The existing calculation of vmpressure takes into account only
the ratio of reclaimed to scanned pages, but not the time spent
or the difficulty in reclaiming those pages. For e.g. when there
are quite a number of file pages in the system, an allocation
request can be satisfied by reclaiming the file pages alone. If
such a reclaim is successful, the vmpressure value will remain low
irrespective of the time spent by the reclaim code to free up the
file pages. With a feature like lowmemorykiller, killing a task
can be faster than reclaiming the file pages alone. So if the
vmpressure values reflect the reclaim difficulty level, clients
can make a decision based on that, for e.g. to kill a task early.
This patch monitors the number of pages scanned in the direct
reclaim path and scales the vmpressure level according to that.
Signed-off-by: Vinayak Menon <vinmenon@codeaurora.org>
Change-Id: I6e643d29a9a1aa0814309253a8b690ad86ec0b13
Currently, vmpressure is tied to memcg and its events are
available only to userspace clients. This patch removes
the dependency on CONFIG_MEMCG and adds a mechanism for
in-kernel clients to subscribe for vmpressure events (in
fact raw vmpressure values are delivered instead of vmpressure
levels, to provide clients more flexibility to take actions
on custom pressure levels which are not currently defined
by vmpressure module).
Change-Id: I38010f166546e8d7f12f5f355b5dbfd6ba04d587
Signed-off-by: Vinayak Menon <vinmenon@codeaurora.org>
Add a new config item, CONFIG_FORCE_ALLOC_FROM_DMA_ZONE, which
can be used to optionally force certain allocators to always
return memory from ZONE_DMA.
This option helps ensure that clients who require ZONE_DMA
memory are always using ZONE_DMA memory.
Change-Id: Id2d36214307789f27aa775c2bef2dab5047c4ff0
Signed-off-by: Liam Mark <lmark@codeaurora.org>
Currently we have kmemleak_stack_scan enabled by default.
This can hog the cpu with pre-emption disabled for a long
time starving other tasks.
Make this optional at compile time, since if required
we can always write to sysfs entry and enable this option.
Change-Id: Ie30447861c942337c7ff25ac269b6025a527e8eb
Signed-off-by: Vignesh Radhakrishnan <vigneshr@codeaurora.org>
Signed-off-by: Prasad Sodagudi <psodagud@codeaurora.org>
Currently we have kmemleak_stack_scan enabled by default.
This can hog the cpu with pre-emption disabled for a long
time starving other tasks.
Make this optional at compile time, since if required
we can always write to sysfs entry and enable this option.
Change-Id: Ie30447861c942337c7ff25ac269b6025a527e8eb
Signed-off-by: Vignesh Radhakrishnan <vigneshr@codeaurora.org>
Signed-off-by: Prasad Sodagudi <psodagud@codeaurora.org>
When CONFIG_PAGE_POISONING is enabled, the pages are poisoned
after setting free page in KASan Shadow memory and KASan reports
the read after free warning. The same thing happens in the allocation
path. So change the order of calling KASan_alloc/free API so that
pages poisoning happens when the pages are in alloc status in KASan
shadow memory.
following is the KASan report for reference.
==================================================================
BUG: KASan: use after free in memset+0x24/0x44 at addr ffffffc000000000
Write of size 4096 by task swapper/0
page:ffffffbac5000000 count:0 mapcount:0 mapping: (null) index:0x0
flags: 0x0()
page dumped because: kasan: bad access detected
CPU: 0 PID: 0 Comm: swapper Not tainted 3.18.0-g5a4a5d5-07242-g6938a8b-dirty #1
Hardware name: Qualcomm Technologies, Inc. MSM 8996 v2 + PMI8994 MTP (DT)
Call trace:
[<ffffffc000089ea4>] dump_backtrace+0x0/0x1c4
[<ffffffc00008a078>] show_stack+0x10/0x1c
[<ffffffc0010ecfd8>] dump_stack+0x74/0xc8
[<ffffffc00020faec>] kasan_report_error+0x2b0/0x408
[<ffffffc00020fd20>] kasan_report+0x34/0x40
[<ffffffc00020f138>] __asan_storeN+0x15c/0x168
[<ffffffc00020f374>] memset+0x20/0x44
[<ffffffc0002086e0>] kernel_map_pages+0x238/0x2a8
[<ffffffc0001ba738>] free_pages_prepare+0x21c/0x25c
[<ffffffc0001bc7e4>] __free_pages_ok+0x20/0xf0
[<ffffffc0001bd3bc>] __free_pages+0x34/0x44
[<ffffffc0001bd5d8>] __free_pages_bootmem+0xf4/0x110
[<ffffffc001ca9050>] free_all_bootmem+0x160/0x1f4
[<ffffffc001c97b30>] mem_init+0x70/0x1ec
[<ffffffc001c909f8>] start_kernel+0x2b8/0x4e4
[<ffffffc001c987dc>] kasan_early_init+0x154/0x160
Change-Id: Idbd3dc629be57ed55a383b069a735ae3ee7b9f05
Signed-off-by: Se Wang (Patrick) Oh <sewango@codeaurora.org>
For some use cases, it is not known beforehand, how much
removed (carve-out) region size must be reserved. Hence
the reserved region size might need to be adjusted to
support varying use cases. In such cases maintaining
different device tree configurations to support varying
carve-out region size is difficult.
Introduce an optional device tree property, to
reserved-memory, "no-map-fixup" which works in tandem with
"removed-dma-pool" compatibility that tries to shrink and
adjust the removed area on very first successful allocation.
At end of which it returns the additional (unused) pages
from the region back to the system.
Point to note is this that this adjustment is done on very
first allocation and thereafter the region size is big
enough only to support maximum of first allocation request
size. This fixup is attempted only once upon first
allocation and never after that. Clients can allocate and
free from this region as any other dma region.
As the description suggests this type of region is specific
to certain special needs and is not to be used for common
use cases.
Change-Id: I31f49d6bd957814bc2ef3a94910425b820ccc739
Signed-off-by: Shiraz Hashim <shashim@codeaurora.org>
Using memset generates unaligned access exception for
device type memory on armv8, hence use memset_io for
ioremap region.
Change-Id: I26c82d4bed20f1c163953680aa200c95842d3f21
Signed-off-by: Shiraz Hashim <shashim@codeaurora.org>
removed_region->base should be phy_addr_t type as it
directly holds physical memory address. Fix it.
Change-Id: I80d49d209cf0f319b7a468697387d23e6bcb1b98
Signed-off-by: Shiraz Hashim <shashim@codeaurora.org>
Commit 54329ac (common: dma-mapping: introduce common remapping functions)
factored out common code for remapping arrays of pages. The code before
the refactor relied on setting area->pages with the array of mapped
pages for easy access later. The refactor dropped this, breaking
parts of the ARM DMA API. Fix this by setting the page array in the same
place.
Change-Id: Ie4d085132f350db29eb2aca67156c25b5e842903
Reported-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Large allocations can result in the dma_common_contiguous_remap
call not being able to succeed because it can't find enough
contiguous memory to setup the mapping.
Make dma_common_contiguous_remap more robust by using vmalloc
as a fallback.
Change-Id: I12ca710b4c24f4ef24bc33a0d1d4922196fb7492
Signed-off-by: Liam Mark <lmark@codeaurora.org>
Align size first and then find required number of bits and
order out of it.
Change-Id: I9b12fb45e5c1ff79e24fe7584cd23923b1a88c87
Signed-off-by: Shiraz Hashim <shashim@codeaurora.org>
The current DMA coherent pool assumes that there is a kernel
mapping at all times for hte entire pool. This may not be
what we want for the entire times. Add the dma_removed ops to
support this use case.
Change-Id: Ie4f1e9bdf57b79699fa8fa7e7a6087e6d88ebbfa
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Change the logic which determines the initial readahead window size
such that for small requests (one page) the initial window size
will be x4 the size of the original request, regardless of the
VM_MAX_READAHEAD value. This prevents a rapid ramp-up
that could be caused due to increasing VM_MAX_READAHEAD.
Change-Id: I93d59c515d7e6c6d62348790980ff7bd4f434997
Signed-off-by: Lee Susman <lsusman@codeaurora.org>
Memory watermarks were sometimes preventing CMA allocations
in low memory.
Change-Id: I550ec987cbd6bc6dadd72b4a764df20cd0758479
Signed-off-by: Liam Mark <lmark@codeaurora.org>
CMA allocations rely on being able to migrate pages out
quickly to fulfill the allocations. Most use cases for
movable allocations meet this requirement. File system
allocations may take an unaccpetably long time to
migrate, which creates delays from CMA. Prevent CMA
pages from ending up on the per-cpu lists to avoid
code paths grabbing CMA pages on the fast path. CMA
pages can still be allocated as a fallback under tight
memory pressure.
CRs-Fixed: 452508
Change-Id: I79a28f697275a2a1870caabae53c8ea345b4b47d
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Bring back the is_cma_pageblock definition for determining if a
page is CMA or not.
Change-Id: I39fd546e22e240b752244832c79514f109c8e84b
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Allow the kswapd cpu affinity to be configured.
There can be power benefits on certain targets when limiting kswapd
to run only on certain cores.
CRs-fixed: 752344
Change-Id: I8a83337ff313a7e0324361140398226a09f8be0f
Signed-off-by: Liam Mark <lmark@codeaurora.org>
[imaund@codeaurora.org: Resolved trivial context conflicts.]
Signed-off-by: Ian Maund <imaund@codeaurora.org>
A workaround was added ealier to move a page to active
list if swapping to devices like zram fails. But this
can result in try_to_free_swap being called from
shrink_page_list, without a properly locked page.
Lock the page when we indicate to activate a page
in pageout().
Add a check to ensure that error is on swap, and
clear the error flag before moving the page to
active list.
CRs-fixed: 760049
Change-Id: I77a8bbd6ed13efdec943298fe9448412feeac176
Signed-off-by: Vinayak Menon <vinmenon@codeaurora.org>
split_pmd() calls early_alloc(), which is marked with __init. Mark
split_pmd() similarly. The only current caller of split_pmd() is
remap_pages(), which is already __init, so there was no real danger
here in the first place.
Change-Id: I3bbc4c66f1ced8fe772366b7e5287be5f474f314
Signed-off-by: Matt Wagantall <mattw@codeaurora.org>
Ensure that shrinkers are given the option to completely drop
their caches even when their caches are smaller than the batch size.
This change helps improve memory headroom by ensuring that under
significant memory pressure shrinkers can drop all of their caches.
This change only attempts to more aggressively call the shrinkers
during background memory reclaim, inorder to avoid hurting the
perforamnce of direct memory reclaim.
Change-Id: I8dbc29c054add639e4810e36fd2c8a063e5c52f3
Signed-off-by: Liam Mark <lmark@codeaurora.org>
If the SLUB_DEBUG_PANIC_ON Kconfig option is
selected, also panic for object and slab
errors to allow capturing relevant debug
data.
Change-Id: Idc582ef48d3c0d866fa89cf8660ff0a5402f7e15
Signed-off-by: David Keitel <dkeitel@codeaurora.org>
Add the DEBUG_SLUB_PANIC_ON option to KCONFIG preventing
the existing defconfig option from being overwritten
by make config.
This will induce a panic if slab debug catches corruptions
within the padding of a given object.
The intention here is to induce collection of data
immediately after the corruption is detected with
the goal to catch the possible source of the corruption.
Change-Id: Ide0102d0761022c643a761989360ae5c853870a8
Signed-off-by: David Keitel <dkeitel@codeaurora.org>
[imaund@codeaurora.org: Resolved trivial merge conflicts.]
Signed-off-by: Ian Maund <imaund@codeaurora.org>
[lmark@codeaurora.org: ensure change does not create
arch/arm64/configs/msm8994_defconfig file]
Signed-off-by: Liam Mark <lmark@codeaurora.org>
Allow other functions to dump the list of tasks.
Useful for when debugging memory leaks.
Change-Id: I76c33a118a9765b4c2276e8c76de36399c78dbf6
Signed-off-by: Liam Mark <lmark@codeaurora.org>
KSM is yet another framework which may obfuscate some memory
problems. Use the showmem notifier to show how KSM is being
used to give some insight into potential issues or non-issues.
Change-Id: If82405dc33f212d085e6847f7c511fd4d0a32a10
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Resiliency of slub was added for production systems in an
attempt to restore corruptions and allow production environments
to continue to run.
In debug setups, this may no be desirable. Thus rather than
attempting to restore corrupted bytes in poisoned zones, panic
to attempt to catch more context of what was going on in the
system at the time.
Add the CONFIG_SLUB_DEBUG_PANIC_ON defconfig option to allow
debug builds to turn on this panic option.
Change-Id: I01763e8eea40a4544e9b7e48c4e4d40840b6c82d
Signed-off-by: David Keitel <dkeitel@codeaurora.org>
KSM thread to scan pages is getting schedule on definite timeout.
That wakes up CPU from idle state and hence may affect the power
consumption. Provide an optional support to use deferred timer
which suites low-power use-cases.
To enable deferred timers,
$ echo 1 > /sys/kernel/mm/ksm/deferred_timer
Change-Id: I07fe199f97fe1f72f9a9e1b0b757a3ac533719e8
Signed-off-by: Chintan Pandya <cpandya@codeaurora.org>
When performing memory reclaim support treating anonymous and
file backed pages equally.
Swapping anonymous pages out to memory can be efficient enough
to justify treating anonymous and file backed pages equally.
CRs-Fixed: 648984
Change-Id: I6315b8557020d1e27a34225bb9cefbef1fb43266
Signed-off-by: Liam Mark <lmark@codeaurora.org>
Move pages that fail swapout to the LRU active list to reduce
pressure on swap device when swapping out is already failing.
This helps when using a pseudo swap device such as zram which
starts failing when memory is low.
Change-Id: Ib136cd0a744378aa93d837a24b9143ee818c80b3
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
When using FORCE_PAGES to allocate the kernel memory into pages,
provide an option to mark the the kernel text section as read only.
Since the kernel text pages are always mapped in the kernel, anyone
can write to the page if they have the address.
Enable this option to mark the kernel text pages as read only to
trigger a fault if any code attempts to write to a page that is
part of the kernel text section.
Change-Id: I2a9e105a3340686b4314bb10cc2a6c7bfa19ce8e
Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
free_bootmem_late is currently set up to only be used in init
functions. Some clients need to use this function past initcalls.
The functions themselves have no restrictions on being used later
minus the __init annotations so remove the annotation.
Change-Id: I7c7e15cf2780a8843ebb4610da5b633c9abb0b3d
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
[abhimany@codeaurora.org: resolve minor conflict
and remove __init from nobootmem.c]
Signed-off-by: Abhimanyu Kapur <abhimany@codeaurora.org>
Strict RWX requires poison_init_mem, bring it back
from the dead.
Change-Id: I09b88a12a47c8694e2ba178caad4415981f4f7e3
Signed-off-by: Abhimanyu Kapur <abhimany@codeaurora.org>
With CONFIG_FORCE_PAGES enabled, we break down the section mappings
into 4K page mappings. For 1GB mappings, remapping the pages into 4K
chunks becomes unnecessarily complicated. Skip creating the 1GB mapping
if we know it's going to be separated into 4K mappings.
Change-Id: I991768210ed6e1c1e19faf0d5d851d550e51a8c6
Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
(cherry picked from commit 2528a04cba82ce3d655dabf78bc49c5b64c89647)
Drivers have a tendency to scribble on everything, including free
pages. Make life easier by marking free pages as read only when
on the buddy list and re-marking as read/write when allocating.
Change-Id: I978ed2921394919917307b9c99217fdc22f82c59
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
(cherry picked from commit 752f5aecb0511c4d661dce2538c723675c1e6449)
Add config option to trigger page a fault if any
code attempts to write to a page on the buddy list
Change-Id: Ic5ab791c4117606519c7b9eb4c2876f246d23320
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
For certain debug features the lowmem needs to be
mapped as pages instead of sections. Add config
option to allow remapping of lowmem as 4K pages
Change-Id: I50179311facd91b97ecde720da38ec7e47512e95
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Strongly ordered memory is occasionally needed for some DMA
allocations for specialized use cases. Add the corresponding
DMA attribute.
Change-Id: Idd9e756c242ef57d6fa6700e51cc38d0863b760d
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Although it isn't architecturally required, CMA regions may need
to have attributes changed at runtime. Remap the CMA regions as
pages to allow this to happen.
Change-Id: I7dd7fa150ce69fdf05f8bf6f76a5ae26dd67ff1b
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
[lmark@codeaurora.org: resolve merge conflicts]
Signed-off-by: Liam Mark <lmark@codeaurora.org>
arm64 needs support for both DMA_ATTR_NO_KERNEL_MAPPING and
DMA_ATTR_STRONGLY_ORDERD. Add support for both of them.
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
[imaund@codeaurora.org: Resolved context conflicts]
Signed-off-by: Ian Maund <imaund@codeaurora.org>
After getting an allocation from dma_alloc_coherent, there
may be cases where it is neccessary to remap the handle
into the CPU's address space (e.g. no CPU side mapping was
requested at allocation time but now one is needed). Add
APIs to bring a handle into the CPU address space again.
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
[imaund@codeaurora.org: resolved context conflicts and added support
for remap 'no_warn' argument]
Signed-off-by: Ian Maund <imaund@codeaurora.org>
There are many drivers in the kernel which can hold on
to lots of memory. It can be useful to dump out all those
drivers at key points in the kernel. Introduct a notifier
framework for dumping this information. When the notifiers
are called, drivers can dump out the state of any memory
they may be using.
Change-Id: Ifb2946964bf5d072552dd56d8d6dfdd794af6d84
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Add the ability to individually disable the Instruction and
Data caches on ARM64 CPUs, by means of the SCTLR[I] and
SCTLR[C] bits. This may be useful for performance
profiling, as well as for troubleshooting potential cache
problems.
Change-Id: Ibd7fc796ddf6984c7f05dcd844432164e28bb021
Signed-off-by: Stepan Moskovchenko <stepanm@codeaurora.org>
[abhimany: resolve trivial merge conflicts]
Signed-off-by: Abhimanyu Kapur <abhimany@codeaurora.org>
Add a new function, memblock_overlaps_memory(), to check if a
region overlaps with a memory bank. This will be used by
peripheral loader code to detect when kernel memory would be
overwritten.
Change-Id: I851f8f416a0f36e85c0e19536b5209f7d4bd431c
Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
(cherry picked from commit cc2753448d9f2adf48295f935a7eee36023ba8d3)
Signed-off-by: Josh Cartwright <joshc@codeaurora.org>
This change adds APIs to allocate and instanciate
multi instance gsi function driver using configFS.
Add an entry in kconfig to select diag driver for
configFS. This change fixes all compilation errors
caused by moving gsi driver from android.c to compile
independently.
Change-Id: I0ddb2ca155fd29c6d58fb561cc197f8efee3495a
Signed-off-by: Hemant Kumar <hemantk@codeaurora.org>
rndis APIs are modified to pass rndis params structure
instead of config index to load the rndis device parameters.
Hence update the API calls in gsi driver.
Change-Id: I1492752b315f57866ca39c6fe0021c4adce97c58
Signed-off-by: Hemant Kumar <hemantk@codeaurora.org>
GSI endpoints are now fixed and need to be requested by name.
As the specific GSI endpoints or ops do not change,
it is not required to be updated on every connect/disconnect
of the cable. This will also fix suspend/disconnect race
conditions. Since GSI endpoints are fixed, gadget restart is
no longer required upon composition switch.
CRs-Fixed: 946385
Change-Id: I36de64ea95470cb19eb3f084b518618a2fe9f0df
Signed-off-by: Devdutt Patnaik <dpatnaik@codeaurora.org>
Implement support for allocating an EP based on name.
This allows a function driver to request an EP using an
ep name string. This is useful to request GSI specific
EPs.
CRs-Fixed: 946385
Change-Id: Id38e6e1e3c2d82f440c0507d24f18f808dc3e4dc
Signed-off-by: Devdutt Patnaik <dpatnaik@codeaurora.org>
Register gsi function suspend callback with composite device to
handle super speed function suspend and resume. Also, move
function suspend option masks to composite device layer.
Change-Id: Ie316973d855612ddc5440934344d18b04d49c567
Signed-off-by: Hemant Kumar <hemantk@codeaurora.org>
Adds a generic function driver that will handle all functions
that support h/w acceleration to IPA over GSI.
This snapshot is taken as of msm-3.18 commit 30b0c37abcd9cf0
(Merge "msm: vidc: add support to set low latency property to
venus")
Change-Id: I4f78a4a36727fec24b1c5481acd6d3122eeb9150
Signed-off-by: Hemant Kumar <hemantk@codeaurora.org>
