evie/android_kernel_oneplus_msm8998
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
android_kernel_oneplus_msm8998/arch/arm/include/asm
History
Steven Capper a3a9ea656d ARM: 7858/1: mm: make UACCESS_WITH_MEMCPY huge page aware 
The memory pinning code in uaccess_with_memcpy.c does not check
for HugeTLB or THP pmds, and will enter an infinite loop should
a __copy_to_user or __clear_user occur against a huge page.

This patch adds detection code for huge pages to pin_page_for_write.
As this code can be executed in a fast path it refers to the actual
pmds rather than the vma. If a HugeTLB or THP is found (they have
the same pmd representation on ARM), the page table spinlock is
taken to prevent modification whilst the page is pinned.

On ARM, huge pages are only represented as pmds, thus no huge pud
checks are performed. (For huge puds one would lock the page table
in a similar manner as in the pmd case).

Two helper functions are introduced; pmd_thp_or_huge will check
whether or not a page is huge or transparent huge (which have the
same pmd layout on ARM), and pmd_hugewillfault will detect whether
or not a page fault will occur on write to the page.

Running the following test (with the chunking from read_zero
removed):
 $ dd if=/dev/zero of=/dev/null bs=10M count=1024
Gave:  2.3 GB/s backed by normal pages,
       2.9 GB/s backed by huge pages,
       5.1 GB/s backed by huge pages, with page mask=HPAGE_MASK.

After some discussion, it was decided not to adopt the HPAGE_MASK,
as this would have a significant detrimental effect on the overall
system latency due to page_table_lock being held for too long.
This could be revisited if split huge page locks are adopted.

Signed-off-by: Steve Capper <steve.capper@linaro.org>
Reviewed-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2013-10-29 11:06:15 +00:00
..
hardware ARM: debug: move PL01X debug include into arch/arm/include/debug/ 2013-08-25 17:11:06 +01:00
mach ARM: SoC late changes for v3.12 2013-09-09 16:35:29 -07:00
xen xen/arm and xen/arm64: implement HYPERVISOR_tmem_op 2013-07-04 11:41:12 +00:00
arch_timer.h Merge branch 'timers/clockevents-next' of git://git.linaro.org/people/dlezcano/clockevents into timers/core 2013-08-21 14:59:23 +02:00
asm-offsets.h
assembler.h ARM: barrier: allow options to be passed to memory barrier instructions 2013-08-12 12:25:44 +01:00
atomic.h ARM: 7687/1: atomics: don't use exclusives for atomic64 read/set with LPAE 2013-04-03 17:00:00 +01:00
barrier.h ARM: barrier: allow options to be passed to memory barrier instructions 2013-08-12 12:25:44 +01:00
bitops.h Disintegrate asm/system.h for ARM 2012-03-28 18:30:01 +01:00
bug.h Disintegrate asm/system.h for ARM 2012-03-28 18:30:01 +01:00
bugs.h
cache.h ARM: implement support for read-mostly sections 2010-12-05 08:39:36 +00:00
cacheflush.h Merge branch 'for-rmk/cacheflush-v2' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into devel-stable 2013-08-28 18:38:40 +01:00
cachetype.h ARM: 7062/1: cache: detect PIPT I-cache using CTR 2011-10-17 09:13:41 +01:00
checksum.h
clkdev.h ARM: Consolidate the clkdev header files 2011-07-19 18:09:45 +02:00
cmpxchg.h ARM: 7853/1: cmpxchg: implement cmpxchg64_relaxed 2013-10-29 11:06:09 +00:00
compiler.h Disintegrate asm/system.h for ARM 2012-03-28 18:30:01 +01:00
cp15.h Merge branch 'for-linus' of git://git.linaro.org/people/rmk/linux-arm 2013-07-03 09:46:29 -07:00
cpu.h ARM: kernel: add MIDR to per-CPU information data 2012-11-19 14:51:11 +00:00
cpuidle.h cpuidle: Add common time keeping and irq enabling 2012-03-21 01:59:40 -04:00
cputype.h ARM: 7855/1: Add check for Cortex-A15 errata 798181 ECO 2013-10-29 11:06:13 +00:00
cti.h ARM: coresight: common definition for (OS) Lock Access Register key value 2013-01-10 21:13:05 +00:00
delay.h ARM: 7685/1: delay: use private ticks_per_jiffy field for timer-based delay ops 2013-04-03 16:45:50 +01:00
device.h ARM: dma-mapping: Add macro to_dma_iommu_mapping() 2013-02-25 15:30:41 +01:00
div64.h ARM: 7705/1: use optimized do_div only for EABI 2013-05-15 19:35:53 +01:00
dma-contiguous.h drivers: dma-contiguous: clean source code and prepare for device tree 2013-08-27 09:18:29 +02:00
dma-iommu.h ARM: dma-mapping: Add arm_iommu_detach_device() 2013-02-25 15:30:41 +01:00
dma-mapping.h IOMMU Updates for Linux v3.8 2012-12-20 10:07:25 -08:00
dma.h ARM: disable virt_to_bus/virt_to_bus almost everywhere 2013-02-14 15:04:33 +01:00
domain.h ARM: fix set_domain() macro 2012-07-05 09:50:55 +01:00
ecard.h ARM: io: ecard: move ioaddr() inside __ecard_address 2011-08-17 08:44:16 +01:00
edac.h ARM: 7201/1: add EDAC atomic_scrub function 2011-12-11 08:35:50 +00:00
elf.h Move the EM_ARM and EM_AARCH64 definitions to uapi/linux/elf-em.h 2013-09-02 16:35:50 +01:00
entry-macro-multi.S ARM: gic: consolidate PPI handling 2011-10-23 13:32:29 +01:00
exception.h ARM: 7115/4: move __exception and friends to asm/exception.h 2011-10-17 09:02:44 +01:00
fb.h
fiq.h ARM: 6940/1: fiq: Briefly document driver responsibilities for suspend/resume 2011-05-26 10:31:06 +01:00
firmware.h ARM: Add interface for registering and calling firmware-specific operations 2013-04-09 01:52:06 +09:00
fixmap.h
flat.h ARM: binfmt_flat: unused variable 'persistent' 2012-10-09 20:29:06 +02:00
floppy.h
fncpy.h ARM: 6640/1: Thumb-2: Symbol manipulation macros for function body copying 2011-01-27 11:48:58 +00:00
fpstate.h Fix common misspellings 2011-03-31 11:26:23 -03:00
ftrace.h ARM: 6319/1: ftrace: add Thumb-2 support to dynamic ftrace 2010-09-02 15:28:43 +01:00
futex.h ARM: 7425/1: extable: ensure fixup entries are 4-byte aligned 2012-06-16 16:30:25 +01:00
glue-cache.h Merge branch 'for-next' of git://git.pengutronix.de/git/ukl/linux into devel-stable 2013-05-22 10:52:24 +01:00
glue-df.h ARM: Add base support for ARMv7-M 2013-04-17 21:38:10 +02:00
glue-pf.h ARM: move cache/processor/fault glue to separate include files 2011-02-12 11:52:21 +00:00
glue-proc.h ARM SoC specific changes 2013-07-02 13:43:38 -07:00
glue.h Fix common misspellings 2011-03-31 11:26:23 -03:00
gpio.h ARM: make mach/gpio.h headers optional 2012-09-14 09:21:59 -05:00
hardirq.h ARM: 7536/1: smp: Formalize an IPI for wakeup 2012-09-19 21:52:08 +01:00
highmem.h ARM: 7684/1: errata: Workaround for Cortex-A15 erratum 798181 (TLBI/DSB operations) 2013-04-03 16:45:49 +01:00
hugetlb-3level.h ARM: mm: HugeTLB support for LPAE systems. 2013-06-04 16:52:37 +01:00
hugetlb.h ARM: mm: HugeTLB support for LPAE systems. 2013-06-04 16:52:37 +01:00
hw_breakpoint.h ARM: hw_breakpoint: Check function for OS Save and Restore mechanism 2013-01-10 21:13:06 +00:00
hw_irq.h arm: dove: Use proper irq accessor functions 2011-03-29 14:47:57 +02:00
hwcap.h UAPI: (Scripted) Disintegrate arch/arm/include/asm 2012-10-12 13:05:52 +01:00
hypervisor.h arm: initial Xen support 2012-09-14 13:53:39 +00:00
ide.h
idmap.h ARM: KVM: move to a KVM provided HYP idmap 2013-04-28 22:23:08 -07:00
io.h ARM: 7728/1: mm: Use phys_addr_t properly for ioremap functions 2013-05-23 00:09:44 +01:00
irq.h arm: Move the set_handle_irq and handle_arch_irq declarations to asm/irq.h 2013-03-26 16:11:20 +00:00
irqflags.h ARM: Add base support for ARMv7-M 2013-04-17 21:38:10 +02:00
jump_label.h compiler/gcc4: Add quirk for 'asm goto' miscompilation bug 2013-10-11 07:39:14 +02:00
Kbuild ARM: 7843/1: drop asm/types.h from generic-y 2013-10-03 10:36:38 +01:00
kexec.h [ARM] add machine-specific hook to machine_kexec 2011-03-03 16:26:55 -05:00
kgdb.h kgdb,arm: fix register dump 2010-10-29 13:14:40 -05:00
kmap_types.h arm: remove km_type definitions 2012-07-24 15:27:28 +08:00
kprobes.h Kernel: Audit Support For The ARM Platform 2012-01-17 16:17:01 -05:00
kvm_arm.h ARM: KVM: get rid of S2_PGD_SIZE 2013-06-26 10:50:04 -07:00
kvm_asm.h ARM: KVM: perform save/restore of PAR 2013-06-26 10:50:04 -07:00
kvm_coproc.h KVM: ARM: User space API for getting/setting co-proc registers 2013-01-23 13:29:14 -05:00
kvm_emulate.h ARM: KVM: don't special case PC when doing an MMIO 2013-06-26 10:50:03 -07:00
kvm_host.h arm/kvm: Cleanup KVM_ARM_MAX_VCPUS logic 2013-06-26 10:50:05 -07:00
kvm_mmio.h KVM: ARM: Handle I/O aborts 2013-01-23 13:29:17 -05:00
kvm_mmu.h ARM: KVM: Fix kvm_set_pte assignment 2013-08-30 15:47:37 -07:00
kvm_psci.h KVM: ARM: Power State Coordination Interface implementation 2013-01-23 13:29:18 -05:00
limits.h
linkage.h
mach-types.h
mc146818rtc.h ARM: mc146818rtc: remove unnecessary include of mach/irqs.h 2012-01-25 20:37:45 -06:00
mcpm.h ARM: 7842/1: MCPM: don't explode if invoked without being initialized first 2013-10-03 10:36:37 +01:00
memblock.h ARM: constify machine_desc structure uses 2013-07-26 14:55:59 +01:00
memory.h mm/ARM: fix stale comment about VALID_PAGE() 2013-07-03 16:07:39 -07:00
mmu.h Merge branch 'security-fixes' into fixes 2013-08-01 20:51:13 +01:00
mmu_context.h ARM: 7790/1: Fix deferred mm switch on VIVT processors 2013-07-26 12:02:09 +01:00
module.h ARM: 7829/1: Add ".text.unlikely" and ".text.hot" to arm unwind tables 2013-09-02 13:49:47 +01:00
mpu.h ARM: mpu: protect the vectors page with an MPU region 2013-06-17 15:13:18 +01:00
mtd-xip.h
mutex.h ARM: 7495/1: mutex: use generic atomic_dec-based implementation for ARMv6+ 2012-08-25 09:22:31 +01:00
neon.h ARM: add support for kernel mode NEON 2013-07-08 22:08:51 +01:00
nwflash.h
opcodes-sec.h ARM: opcodes: add opcodes definitions for ARM security extensions 2013-01-10 21:10:20 +00:00
opcodes-virt.h ARM: opcodes: add __ERET/__MSR_ELR_HYP instruction encoding 2012-09-19 08:32:49 +01:00
opcodes.h ARM: opcodes: add missing include of linux/linkage.h 2013-01-10 21:10:20 +00:00
outercache.h ARM: move outer_cache declaration out of ifdef 2013-08-22 20:48:41 -05:00
page-nommu.h
page.h ARM: make vectors page inaccessible from userspace 2013-08-01 14:31:58 +01:00
pci.h PCI: collapse pcibios_resource_to_bus 2012-02-23 20:19:04 -07:00
percpu.h ARM: 7747/1: pcpu: ensure __my_cpu_offset cannot be re-ordered across barrier() 2013-06-05 23:35:56 +01:00
perf_event.h ARM: 7584/1: perf: fix link error when CONFIG_HW_PERF_EVENTS is not selected 2012-11-23 13:01:30 +00:00
pgalloc.h ARM: LPAE: Page table maintenance for the 3-level format 2011-12-08 10:30:39 +00:00
pgtable-2level-hwdef.h ARM: 7077/1: LPAE: Use a mask for physical addresses in page table entries 2011-10-06 15:40:06 +01:00
pgtable-2level-types.h ARM: 7076/1: LPAE: Add (pte|pmd)val_t type definitions as u32 2011-10-06 15:40:05 +01:00
pgtable-2level.h ARM: 7858/1: mm: make UACCESS_WITH_MEMCPY huge page aware 2013-10-29 11:06:15 +00:00
pgtable-3level-hwdef.h Merge branch 'for-rmk/lpae' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into devel-stable 2013-06-18 20:11:32 +01:00
pgtable-3level-types.h ARM: LPAE: Introduce the 3-level page table format definitions 2011-12-08 10:30:39 +00:00
pgtable-3level.h ARM: 7858/1: mm: make UACCESS_WITH_MEMCPY huge page aware 2013-10-29 11:06:15 +00:00
pgtable-hwdef.h ARM: LPAE: Introduce the 3-level page table format definitions 2011-12-08 10:30:39 +00:00
pgtable-nommu.h consolidate io_remap_pfn_range definitions 2013-06-29 12:46:35 +04:00
pgtable.h ARM: 7808/1: KVM: mm: Get rid of L_PTE_USER ref from PAGE_S2_DEVICE 2013-08-13 20:25:06 +01:00
pmu.h ARM: perf: consistently use arm_pmu->name for PMU name 2012-11-09 11:37:26 +00:00
proc-fns.h Merge branch 'for-rmk/lpae' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into devel-stable 2013-06-18 20:11:32 +01:00
processor.h ARM: 7791/1: a.out: remove partial a.out support 2013-07-26 12:02:10 +01:00
procinfo.h
prom.h ARM: constify machine_desc structure uses 2013-07-26 14:55:59 +01:00
psci.h arm: introduce psci_smp_ops 2013-05-21 14:24:11 +00:00
ptrace.h ARM: Add base support for ARMv7-M 2013-04-17 21:38:10 +02:00
scatterlist.h ARM: Allow SoCs to enable scatterlist chaining 2011-06-02 11:16:22 +01:00
sched_clock.h sched_clock: Add temporary asm/sched_clock.h 2013-06-21 10:17:44 -07:00
seccomp.h ARM: SECCOMP support 2010-10-01 22:32:18 -04:00
setup.h UAPI: (Scripted) Disintegrate arch/arm/include/asm 2012-10-12 13:05:52 +01:00
shmparam.h
signal.h arm: switch to generic old sigaction() 2013-02-03 18:15:47 -05:00
smp.h ARM: mpu: add MPU initialisation for secondary cores 2013-06-07 17:02:53 +01:00
smp_plat.h ARM: 7807/1: kexec: validate CPU hotplug support 2013-08-13 20:23:18 +01:00
smp_scu.h ARM: scu: provide inline dummy functions when SCU is not present 2013-07-04 05:10:51 -07:00
smp_twd.h ARM: smp_twd: convert to use CLKSRC_OF init 2013-03-11 08:42:08 -05:00
sparsemem.h
spinlock.h ARM: 7854/1: lockref: add support for lockless lockrefs using cmpxchg64 2013-10-29 11:06:11 +00:00
spinlock_types.h ARM: 7446/1: spinlock: use ticket algorithm for ARMv6+ locking implementation 2012-07-09 17:41:10 +01:00
stackprotector.h
stacktrace.h
string.h
suspend.h ARM: kernel: implement stack pointer save array through MPIDR hashing 2013-06-20 11:24:11 +01:00
swab.h UAPI: (Scripted) Disintegrate arch/arm/include/asm 2012-10-12 13:05:52 +01:00
switch_to.h ARM: spinlock: use inner-shareable dsb variant prior to sev instruction 2013-08-12 12:25:45 +01:00
sync_bitops.h xen/arm: sync_bitops 2012-08-08 16:34:01 +00:00
syscall.h ARM: 7851/1: check for number of arguments in syscall_get/set_arguments() 2013-10-13 18:54:33 +01:00
system.h the only place that needs to include asm/exec.h is linux/binfmts.h 2012-09-20 09:51:13 -04:00
system_info.h ARM: Add base support for ARMv7-M 2013-04-17 21:38:10 +02:00
system_misc.h reboot: arm: change reboot_mode to use enum reboot_mode 2013-07-09 10:33:29 -07:00
tcm.h ARM: 6985/1: export functions to determine the presence of I/DTCM 2011-07-06 20:49:45 +01:00
therm.h
thread_info.h Merge branch 'for-rmk/cacheflush-v2' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into devel-stable 2013-08-28 18:38:40 +01:00
thread_notify.h ARM: 6867/1: Introduce THREAD_NOTIFY_COPY for copy_thread() hooks 2011-04-10 21:13:36 +01:00
timex.h Merge branch 'for-linus' of git://git.linaro.org/people/rmk/linux-arm 2012-10-07 21:20:57 +09:00
tlb.h Fix TLB gather virtual address range invalidation corner cases 2013-08-16 08:52:46 -07:00
tlbflush.h ARM: 7855/1: Add check for Cortex-A15 errata 798181 ECO 2013-10-29 11:06:13 +00:00
tls.h ARM: 7735/2: Preserve the user r/w register TPIDRURW on context switch and fork 2013-06-24 15:21:59 +01:00
topology.h ARM: 7182/1: ARM cpu topology: fix warning 2011-11-30 23:55:21 +00:00
traps.h ARM: earlier initialization of vectors page 2012-01-23 10:24:11 +00:00
types.h ARM: 7822/1: add workaround for ambiguous C99 stdint.h types 2013-08-25 17:09:46 +01:00
uaccess.h ARM: 7836/1: add __get_user_unaligned/__put_user_unaligned 2013-09-18 10:58:23 +01:00
ucontext.h Fix common misspellings 2011-03-31 11:26:23 -03:00
unified.h ARM: make BSYM macro assembly only 2012-01-16 08:56:25 -06:00
unistd.h consolidate cond_syscall and SYSCALL_ALIAS declarations 2013-03-03 22:55:19 -05:00
unwind.h ARM: 7187/1: fix unwinding for XIP kernels 2011-12-06 11:16:13 +00:00
user.h ARM: 6798/1: aout-core: zero thread debug registers in a.out core dump 2011-03-10 15:16:29 +00:00
v7m.h ARM: 7828/1: ARMv7-M: implement restart routine common to all v7-M machines 2013-09-02 13:49:29 +01:00
vfp.h
vfpmacros.h ARM: 7566/1: vfp: fix save and restore when running on pre-VFPv3 and CONFIG_VFPv3 set 2012-10-29 10:04:05 +00:00
vga.h ARM: set vga memory base at run-time 2011-07-12 11:19:29 -05:00
virt.h ARM: 7787/1: virt: ensure visibility of __boot_cpu_mode 2013-07-26 12:01:17 +01:00
word-at-a-time.h ARM: 7450/1: dcache: select DCACHE_WORD_ACCESS for little-endian ARMv6+ CPUs 2012-07-09 17:41:11 +01:00
xor.h ARM: crypto: add NEON accelerated XOR implementation 2013-07-08 22:09:06 +01:00