Currently it is difficult to test the real performance of
allocating from the ion system heap since the heap uses
a page pool. Customers regularly encounter
scenarios where the pool is empty and memory has to
be allocated from the system.
Add support for the ION_FLAG_POOL_FORCE_ALLOC flag so that
the pool can be bypassed in order to measure the performance
of non-pool system heap allocations.
Change-Id: Ia89910abf52282a9696bd5abb9270930078c66a3
Signed-off-by: Liam Mark <lmark@codeaurora.org>
This reverts commit afedf4f2f4c57d2("msm: ion: fix ion cache
maintenance issue") was causing device crash just after
launching camera or adrenotest app. revert the change
till original issue get fixed.
Change-Id: I32e9ff9610de7de8f24fae36ed14ff35a9806f0c
Signed-off-by: Fnu Yagyasen <yagya@codeaurora.org>
ion cache maintenance code while doing cache maintenance
on each sg is only considering page length which may not
always be correct. An sg node can have more than page
length contiguous pages.
Update the code to properly take care of any length sg.
Change-Id: Ied380fa32cbda5384bab8ec2625ceaf1010fd314
Signed-off-by: Shiraz Hashim <shashim@codeaurora.org>
During buffer allocation if the assign call fails, we call
ion_system_secure_heap_free, which in turn does an unassign
operation before freeing the buffer. We don't want to call
unassign if the assign failed.
Fix this by calling the underlying free function directly
during error handling.
Change-Id: I2deec6fd147c4200e16f3779e37330dca1d7d173
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Add the infrastructure to support a secure cma heap
needed for secure usecases. The new heap uses
the existing cma heap infrastructure, along with adding
the assign call to assign the correct VM to the buffers
during allocation and free.
Change-Id: I0c959f46ffa6eb67128b6794e9e60c470a3d292e
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
When freeing a secure buffer, the RWX permissions need to be
passed as a parameter to the hyp_assign call. This will ensure
that when the hypervisor unmaps the buffer, HLOS can use it
for data as well as instructions.
Change-Id: I2f1e67dda8f3ebedee552a2908b7f57c18754d42
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Set the private page bit on each sg of secure buffers to inform
other entities, which import these buffers as an fd, that this
memory is secure.
Change-Id: I767c26fa1feca7505bbb3293cfdb4e09520496a6
Signed-off-by: Shrenuj Bansal <shrenujb@codeaurora.org>
Add new APIs to allow clients to map and unmap dma_buffers created by
ION. The call to the unmap API will not actually do the unmapping from the
IOMMU. The unmapping will occur when the ION/dma_buf buffer is actually
freed. This behavior can be disabled with DMA_ATTR_NO_DELAYED_UNMAP
Change-Id: Ic4dbd3b582eb0388020c650cab5fbb1dad67ae81
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
Presently ion debug interface provides total memory held
in system heap for cached and uncached pool corresponding
to per page order list. Add additional logs to also print
total cumulative bytes held together in these pools so
that it can be accounted easily.
Change-Id: I07dbc45fa99870904cca9b2067b371f4d7247ec5
Signed-off-by: Shiraz Hashim <shashim@codeaurora.org>
The assign call apis have been updated by TZ to
include more usecases. Update the secure_buffer
api files with the same. The system secure heap
needs to be updated to reflect the change in
the api calls.
Change-Id: Idc784ddac222e6ad9f5defafc422e6e3fb88aa0c
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Move all function declarations and types to the header
file to make the macros and definitions available even
when the CONFIG_MSM_SECURE_BUFFER is not enabled. Rename
rid of the msm_ion_secure_table and msm_ion_unsecure_table
to something more generic.
Change-Id: Ia5ca0c52f971a67c7936c64b42cd2522aa1773fa
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
The secure_buffer.c files does not have anything specific
to ion. The functions defined here might be needed by other
drivers as well. If the ion driver is disabled that might
not work. So move the file to a new location.
Change-Id: I3498de791c01019b7933feb65d196152887fcf24
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
ion client can get created and destroyed dynamically.
Client data struct may not be in proper state if one core is reading
client information and other core is destroying the same client.
There is no synchronization between ion_debug_client_show() and
ion_client_destroy(), that leaves client data structures in
uncertain state and which may lead to system crash.
So add a check in ion_debug_client_show(), to verify whether the client
is part of ion device rb tree or not before retrieving client data.
CRs-Fixed: 743714
Change-Id: Ic4c2f9dff905d9bb8fbf2e56b7c6396dc01bdf8f
Signed-off-by: Prasad Sodagudi <psodagud@codeaurora.org>
We're currently using %lu and %ld to print some variables of type
dma_addr_t, which results in the following warning when dma_addr_t is
64-bits wide:
drivers/staging/android/ion/ion_chunk_heap.c: In function 'ion_chunk_heap_create':
drivers/staging/android/ion/ion_chunk_heap.c:176:2: warning: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'dma_addr_t' [-Wformat=]
pr_info("%s: base %lu size %zu align %ld\n", __func__, chunk_heap->base,
^
drivers/staging/android/ion/ion_chunk_heap.c:176:2: warning: format '%ld' expects argument of type 'long int', but argument 5 has type 'dma_addr_t' [-Wformat=]
Fix this by using %pad as instructed in printk-formats.txt.
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
ION handle buffer size should be of type 'size_t' and not
'unsigned long'. Update the API signature of ion_handle_get_size
which reads and returns the size of the ION buffer.
Change-Id: I43040816724318f5f388f0b542be963d497a754b
Signed-off-by: Banajit Goswami <bgoswami@codeaurora.org>
show_ion_usage protects against modification with a down_read.
Close it by protecting with an up_read not an up_write.
Change-Id: I2acd29188cde13cc11b4f6c8d6e35082ba37c08a
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
The buffers passed to the secure world in they hypervisor assign
call need to be flushed, so that they can be accessed by the
secure world. Flush the caches before making the hypervisor call.
CRs-Fixed: 798525
Change-Id: I95862f09daa3b0edc5e32863c6defc657f98b534
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
DMA mappers often (always?) modify the scatterlist they are
given (e.g. dma_address and dma_length) when their dma_map_ops.map_sg
operation is called. Currently Ion always returns the same sg_table
when it's dma_buf_ops.map_dma_buf operation is called. The result is
that when Ion is used as a DMA buf exporter to multiple clients (a
common scenario during zero-copy buffer-sharing) those clients will be
stomping all over each other since they're both trying to put their
mappings into the same sg_table.
Fix this by always duplicating the buffer's sg_table in ion_map_dma_buf
and returning the duplicate. Each buffer will still always have a
single "master" sg_table that should never be exported to the DMA buf
framework.
Test case (pseudo-code):
handle = ion_alloc();
dbuf = ion_share_dma_buf(handle);
attach = dma_buf_attach(dbuf);
table1 = dma_buf_map_attachment(attach);
table2 = dma_buf_map_attachment(attach);
dma_map_sg(table1);
table2 = dma_buf_map_attachment(attach);
dma_map_sg(table2);
/* table1 is now hosed */
check_mappings(table1); /* KABOOM */
Change-Id: Ifefa06ecd464c00819b4ae7ac32ad37f1491373b
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
The dmap_cnt field in struct ion_buffer is not used. Remove it.
Change-Id: I8a5d388a0e7df8612e424ca15cee27498c018508
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
The dma_sync_sg_for_* traverses the scatterlist and performs cache
maintainance using the dma_address of each link. If the scatterlist
is already mapped into an IOMMU, the dma_address is mapped into the
IOMMU, and not the cpu virtual address, which is incorrect for
proper cache maintainance. Instead traverse the scatterlist and
call the cache maintainance on each individual page.
CRs-Fixed: 768053
Change-Id: Ia989fb56236d82184a7f581b8ab4f17c0b93a010
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Ion's deferred freeing feature has been shown to greatly improve user
experience for some use cases. Enable the feature in the system secure
heap by adding the appropriate heap flag.
Change-Id: I9be7f2b3365b2d1a086acc3d182d46bfee2e6cfe
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Currently we initialize the heap free_lock and free list size in
ion_heap_init_deferred_free, which is only called when the
ION_HEAP_FLAG_DEFER_FREE heap flag is given. However, the lock and size
are used in the shrinker path as well as the deferred free path, and we
can register a shrinker *without* enabling deferred freeing. So, if a
heap provides a shrinker but *doesn't* set the DEFER_FREE flag we will
use these parameters uninitialized (resulting in a spinlock bug and
broken shrinker accounting).
Fix these problems by initializing the free list parameters directly in
ion_device_add_heap, which is always called no matter which heap
features are being used.
CRs-Fixed: 772761
Change-Id: Ib6466f940675040812b754690fd132ad95ec91d5
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Set the private page flag on secure chunks to give other entities
a clue that the memory is special.
Change-Id: Ic0dedbad4b41a874a3f80dec75d0721d30307094
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
The cma secure heap allocates contiguous memory. This causes
the memory to be fragmented if several allocation and free
requests are made with buffers of various sizes. Introduce
the new ion flag(ION_FLAG_ALLOW_NON_CONTIG), which can be
passed to indicate a scatter gather list of smaller non
contiguous buffers can be allocated instead of a single
giant chunk.
Change-Id: I28b4ec8ee479e9cd565e1792e732137f0327bca3
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
[imaund@codeaurora.org: resolved context conflicts]
Signed-off-by: Ian Maund <imaund@codeaurora.org>
TZ expects the source and destination vm parameters as u32
integers. Define the parameters as u32 instead of u64.
Change-Id: I819427a1ebbb5823cf68acdbb5ea0c451ed313ea
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Add infrastructure to support a new system secure
heap, needed for secure use cases. The new heap
essentially allocates from the system heap, and
assigns the memory to the correct VM before
returning back to the client.
Change-Id: I2cd900e4399eecc3a9510139c4ebea0b713c39fc
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
We currently pass a NULL device when doing cache maintenance in
msm_ion_heap_buffer_zero. The dma ops that are used when a NULL device
is given rely on s->dma_address being the physical address of the
underlying memory. However, msm_ion_heap_buffer_zero can be called on
buffers that have previously been mapped by a different DMA mapper (like
the IOMMU DMA mapper) which might have set dma_address to something
other than the physical address of the underlying memory. This results
in us trying to do cache maintenance on some stuff that we shouldn't be.
Fix this by putting the physical address for the underlying memory back
into dma_address just before doing cache maintenance.
Change-Id: Ic5df328f5aeac09f7c9280ced887d2ba6098eb88
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
For certain secure use cases, the memory allocated using
ion needs to be re-assigned from one VM's MMU to another
VM's MMU. Call into the hypervisor to make this assingment.
Change-Id: I172f054e1b4385b8fcdaf7fdc2869034dbbd2db3
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
This reverts commit 49f5e2517691c768a884e7d1fb93306919081773.
This change results in some buffers not being zeroed in use
cases which did not show up in testing.
Change-Id: I659e7324bd21344444a0940b3403060254e8fba8
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Ian Maund <imaund@codeaurora.org>
Conflicts:
drivers/staging/android/ion/ion_page_pool.c
To support new usecases, HLOS will permit of sharing
of the CMA pool between the ion heaps. Going forward
we do not need to check if the heaps overlap.
Change-Id: I1dee72147fad994a1c88ea9dfcf7402ba98f3609
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Bugs in Ion clients are semi-common cause of out of memory
errors. Register with the show_mem notification framework
to let Ion dump out data for debugging. To avoid having to
take an excessive amount of locks, keep a running total of
allocated and potentially orphaned memory.
Change-Id: I52c31459d5bf975c67adb5f3fe17033c8e01d91b
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
All clients have been converted over to the new API
for securing. Instead of silently returning success when
secure fails, error out on a secure failure to allow for proper
error handling.
Change-Id: I57a5c1aef999cfc69a2a9cc1b3e28b2fa74fe3a6
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
If unsecure fails, this means the CPU cannot touch the
physical memory. This means if the memory is freed back to
the buddy allocator all accesses will trigger faults. If
unsecure fails, the only option is to leak the memory and
not free it.
Change-Id: I6a98761912598a9f8cdcc3914eca8e5e41147e55
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
The secure APIs are no longer called by external clients
and only called on alloc/free. The reference counting is now
useless so drop it.
Change-Id: Ide0fa6bb3f4183bbbe78d82b98166dc5942c6d3e
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
The usage hint is no longer used and is always 0. Remove
this as an argument to the lock/unlock APIs.
Change-Id: I7bafbe7823f26ba426a887e5c21f2db0852d3294
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
These APIs are deprecated as securing and unsecuring for secure
buffers is done on allocation and free respectively by the secure
heap layer. Clients don't have to call these explicitly.
Change-Id: If88cd1c47cba346446ebdcad494850b54ba954ab
Signed-off-by: Shalaj Jain <shalajj@codeaurora.org>
[mitchelh: dropped all client changes (since they're not in yet)]
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
At one time, the zeroing algorithm used a mixture of
cached and uncached mappings which necessitated an extra
cache operation afterwards. The zeroing algorithm has been
simplified so the cache operation is no longer required if
the mapping used for zeroing matches the cache status of
the buffer (cached or not). Remove the cache operation and
use appropriate page protections instead.
Change-Id: I7d3596ed503811818782e77a0f3f7f63b1456d70
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
The scm library has added support for a new secure world
interface that is more aligned to the ARMv8 SMC calling
convention. Use the new API while maintaining backward
compatibility for the mem-protect call.
Change-Id: I3a4432f2eab4c52d0fb2e8aa5d5e9dd650fd05ef
Signed-off-by: Vikram Mulukutla <markivx@codeaurora.org>
The shrinker interface has now changed with separate count and scan
functions. Update this for the secure CMA shrinker.
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Many of the DT options for reserving memory are no longer used
at all. Simply the reading of bases/sizes from the DT and convert
to the new format.
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Users of dma on arm may call dma_*_writecombine and dma_*_nonconsistent.
Add these functions for arm64 as well.
Change-Id: I213f8314022969bcefbfcd8581725762c385fb18
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
This is a snapshot of the ION support as of msm-3.10 commit
acdce027751d5a7488b283f0ce3111f873a5816d (Merge "defconfig: arm64: Enable
ONESHOT_SYNC for msm8994")
In addition, comment out the shrinker code and skip-zeroing bits as they
aren't yet in the tree.
Change-Id: Id9e1e7fa4c35ce5a9f9348837f05f002258865cf
Signed-off-by: Kumar Gala <galak@codeaurora.org>
[mitchelh: dropped MSM changes to ion_chunk_heap, dropped MSM changes to
ion_carveout_heap]
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
The DMA framework currently zeros all buffers because it (righfully so)
assumes that drivers will soon need to pass the memory to a device.
Some devices/use case may not require zeroed memory and there can
be an increase in performance if we skip the zeroing. Add a DMA_ATTR
to allow skipping of DMA zeroing.
Change-Id: Id9ccab355554b3163d8e7eae1caa82460e171e34
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
[mitchelh: dropped changes to arm32]
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
is_vmalloc_addr currently assumes that all vmalloc addresses
exist between VMALLOC_START and VMALLOC_END. This may not be
the case when interleaving vmalloc and lowmem. Update the
is_vmalloc_addr to properly check for this.
Correspondingly we need to ensure that VMALLOC_TOTAL accounts
for all the vmalloc regions when CONFIG_ENABLE_VMALLOC_SAVING
is enabled.
Change-Id: I5def3d6ae1a4de59ea36f095b8c73649a37b1f36
Signed-off-by: Susheel Khiani <skhiani@codeaurora.org>
Currently on 32 bit systems, virtual space above
PAGE_OFFSET is reserved for direct mapped lowmem
and part of virtual address space is reserved for
vmalloc. We want to optimize such as to have as
much direct mapped memory as possible since there is
penalty for mapping/unmapping highmem. Now, we may
have an image that is expected to have a lifetime of
the entire system and is reserved in physical region
that would be part of direct mapped lowmem. The
physical memory which is thus reserved is never used
by Linux. This means that even though the system is
not actually accessing the virtual memory
corresponding to the reserved physical memory, we
are still losing that portion of direct mapped lowmem
space.
So by allowing lowmem to be non contiguous we can
give this unused virtual address space of reserved
region back for use in vmalloc.
Change-Id: I980b3dfafac71884dcdcb8cd2e4a6363cde5746a
Signed-off-by: Susheel Khiani <skhiani@codeaurora.org>
During early init, all dma areas are remapped to PAGE_SIZE
granularity. Since full pmd regions are cleared to be
remapped into PAGE_SIZE, ensure that address range is pmd
size aligned while not crossing memory boundaries.
This would ensure that even if address region is not pmd
aligned, its mapping would not be cleared but factored in to
PAGE_SIZE regions.
Change-Id: Iad4ad7fd6169cdc693d532821aba453465addb7c
Signed-off-by: Shiraz Hashim <shashim@codeaurora.org>
