android_kernel_oneplus_msm8998/arch/x86
Aaron Lu 3b9d9ec0d8 x86/irq: Do not substract irq_tlb_count from irq_call_count
commit 82ba4faca1bffad429f15c90c980ffd010366c25 upstream.

Since commit:

  52aec3308d ("x86/tlb: replace INVALIDATE_TLB_VECTOR by CALL_FUNCTION_VECTOR")

the TLB remote shootdown is done through call function vector. That
commit didn't take care of irq_tlb_count, which a later commit:

  fd0f586972 ("x86: Distinguish TLB shootdown interrupts from other functions call interrupts")

... tried to fix.

The fix assumes every increase of irq_tlb_count has a corresponding
increase of irq_call_count. So the irq_call_count is always bigger than
irq_tlb_count and we could substract irq_tlb_count from irq_call_count.

Unfortunately this is not true for the smp_call_function_single() case.
The IPI is only sent if the target CPU's call_single_queue is empty when
adding a csd into it in generic_exec_single. That means if two threads
are both adding flush tlb csds to the same CPU's call_single_queue, only
one IPI is sent. In other words, the irq_call_count is incremented by 1
but irq_tlb_count is incremented by 2. Over time, irq_tlb_count will be
bigger than irq_call_count and the substract will produce a very large
irq_call_count value due to overflow.

Considering that:

  1) it's not worth to send more IPIs for the sake of accurate counting of
     irq_call_count in generic_exec_single();

  2) it's not easy to tell if the call function interrupt is for TLB
     shootdown in __smp_call_function_single_interrupt().

Not to exclude TLB shootdown from call function count seems to be the
simplest fix and this patch just does that.

This bug was found by LKP's cyclic performance regression tracking recently
with the vm-scalability test suite. I have bisected to commit:

  3dec0ba0be ("mm/rmap: share the i_mmap_rwsem")

This commit didn't do anything wrong but revealed the irq_call_count
problem. IIUC, the commit makes rwc->remap_one in rmap_walk_file
concurrent with multiple threads.  When remap_one is try_to_unmap_one(),
then multiple threads could queue flush TLB to the same CPU but only
one IPI will be sent.

Since the commit was added in Linux v3.19, the counting problem only
shows up from v3.19 onwards.

Signed-off-by: Aaron Lu <aaron.lu@intel.com>
Cc: Alex Shi <alex.shi@linaro.org>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Huang Ying <ying.huang@intel.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com>
Link: http://lkml.kernel.org/r/20160811074430.GA18163@aaronlu.sh.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-12-25 14:22:09 +01:00
..
boot x86/boot: Add missing declaration of string functions 2017-08-11 09:08:57 -07:00
configs kconfig: tinyconfig: provide whole choice blocks to avoid warnings 2016-09-24 10:07:42 +02:00
crypto crypto: salsa20 - fix blkcipher_walk API usage 2017-12-20 10:04:51 +01:00
entry x86/asm/64: Clear AC on NMI entries 2017-08-24 17:02:36 -07:00
ia32 Merge branch 'x86-headers-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2015-11-03 21:05:40 -08:00
include x86/irq: Do not substract irq_tlb_count from irq_call_count 2017-12-25 14:22:09 +01:00
kernel x86/irq: Do not substract irq_tlb_count from irq_call_count 2017-12-25 14:22:09 +01:00
kvm KVM: nVMX: reset nested_run_pending if the vCPU is going to be reset 2017-12-16 10:33:52 +01:00
lguest x86/paravirt: Prevent rtc_cmos platform device init on PV guests 2015-12-19 21:35:13 +01:00
lib x86/decoder: Add new TEST instruction pattern 2017-11-30 08:37:19 +00:00
math-emu Merge branch 'x86-headers-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2015-11-03 21:05:40 -08:00
mm x86/mm, sched/core: Turn off IRQs in switch_mm() 2017-12-25 14:22:09 +01:00
net ebpf: migrate bpf_prog's flags to bitfield 2015-10-03 05:02:39 -07:00
oprofile x86/oprofile/ppro: Do not use __this_cpu*() in preemptible context 2017-11-15 17:13:14 +01:00
pci x86/PCI: Make broadcom_postcore_init() check acpi_disabled 2017-12-16 10:33:48 +01:00
platform Revert "x86/mm/pat: Ensure cpa->pfn only contains page frame numbers" 2017-12-16 10:33:57 +01:00
power x86/ldt: Make modify_ldt synchronous 2015-07-31 10:23:23 +02:00
purgatory x86/kexec: add -fno-PIE 2016-11-26 09:54:52 +01:00
ras x86/ras/mce_amd_inj: Inject bank 4 errors on the NBC 2015-10-12 16:15:48 +02:00
realmode Merge branch 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2015-02-16 14:58:12 -08:00
tools x86/tools: Fix gcc-7 warning in relocs.c 2017-07-15 11:57:47 +02:00
um x86/entry: Use SYSCALL_DEFINE() macros for sys_modify_ldt() 2017-12-09 18:42:40 +01:00
video
xen xen: adjust early dom0 p2m handling to xen hypervisor behavior 2017-05-20 14:26:58 +02:00
.gitignore x86/build: Add arch/x86/purgatory/ make generated files to gitignore 2014-10-09 09:29:46 +02:00
Kbuild x86/asm/entry, x86/vdso: Move the vDSO code to arch/x86/entry/vdso/ 2015-06-03 18:51:37 +02:00
Kconfig security/keys: add CONFIG_KEYS_COMPAT to Kconfig 2017-11-18 11:11:07 +01:00
Kconfig.cpu x86/Kconfig/cpus: Fix/complete CPU type help texts 2015-10-21 11:12:56 +02:00
Kconfig.debug x86: don't make DEBUG_WX default to 'y' even with DEBUG_RODATA 2015-11-06 09:12:41 -08:00
Makefile Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6 2015-11-04 09:11:12 -08:00
Makefile.um kbuild: use relative path more to include Makefile 2015-04-02 16:42:08 +02:00
Makefile_32.cpu