Buffers shared with hypervisor might undergo some cache maintenance on
the hypervisor side. Currently, we're passing a buffer that might not
be cache-aligned (source_vm_list), so when the hypervisor does some
cache maintenance on it they might be corrupting data. Fix this by
ensuring that all buffers being shared with the hypervisor are
cache-aligned by kmalloc'ing and memcpy'ing the source_vm_list before
sending it across.
Change-Id: I661f8ca852ade159e3432904086b060be8bfb806
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
It can be useful to convert secure VMID values into strings. Provide a
helper function to do so.
Change-Id: If907a0bac92c5d164154c0e5dfe67933115163c8
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
The hyp_assign_phys() api can be called by different
usecases where it is not guaranteed that the source vm is
always VMID_HLOS.
Pass the responsibility of setting the source_vm to
caller of the function.
Change-Id: I3851a6681f49d4bb6fa5b7a889a16a158497e9e6
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
kmap_flush_unused does not flush kmap_atomic mappings which
are handled separately. Architectures may have use cases to require
these to be flushed. Add an option to let architectures define
kmap_atomic flushing to get rid of extra mappings.
Change-Id: I5a338ed9e215f0f0ad3ab58a3066d2f4c8ce3ba7
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
When calling hyp_assign to free a secure buffer we need to pass
in RWX permissions so that when the Hypervisor unmaps the buffer
and maps it back to HLOS, the HLOS is able to use the same
memory for data and instructions.
Change-Id: I71e06a81df85891fecb11c5f197acd54979d2887
Signed-off-by: Shrenuj Bansal <shrenujb@codeaurora.org>
Buffers must be unmapped first and then freed. Without
this sequence, specifically for secure buffers, TZ throws
error on free if it finds persisting mapping. Fix this.
Change-Id: Iba09cb998b4f87da3e9eb1c2bc186453be02a278
Signed-off-by: Shiraz Hashim <shashim@codeaurora.org>
ion cache maintenance code while doing cache maintenance
on each sg is only considering page length which may not
always be correct. An sg node can have more than page
length contiguous pages.
Update the code to properly take care of any length sg.
Change-Id: Ifdf8a8fa26abfa5bd47185a1039880306246d587
Signed-off-by: Shiraz Hashim <shashim@codeaurora.org>
Let ION compile when CONFIG_CMA is not defined.
Change-Id: I6f3cf19a94e3f966c0973c9f8d765ecb752b7a48
Signed-off-by: Vinayak Menon <vinmenon@codeaurora.org>
If the hypervisor fails to assign the correct vmids to the
memory allocated, we should free the allocated memory. However
the free path depends on certain parameters of the ion_buffer
structure which have not yet been populated.
Update the function definition to accept the scatter gather list
instead of the ion buffer, since we are zero'ing the list
and not the buffer.
Change-Id: I68bef3af6c1b7e8092fbc0e878d3c825580f438e
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Currently it is difficult to test the real performance of
allocating from the ion system heap since the heap uses
a page pool. Customers regularly encounter
scenarios where the pool is empty and memory has to
be allocated from the system.
Add support for the ION_FLAG_POOL_FORCE_ALLOC flag so that
the pool can be bypassed in order to measure the performance
of non-pool system heap allocations.
Change-Id: Ia89910abf52282a9696bd5abb9270930078c66a3
Signed-off-by: Liam Mark <lmark@codeaurora.org>
This reverts commit afedf4f2f4c57d2("msm: ion: fix ion cache
maintenance issue") was causing device crash just after
launching camera or adrenotest app. revert the change
till original issue get fixed.
Change-Id: I32e9ff9610de7de8f24fae36ed14ff35a9806f0c
Signed-off-by: Fnu Yagyasen <yagya@codeaurora.org>
ion cache maintenance code while doing cache maintenance
on each sg is only considering page length which may not
always be correct. An sg node can have more than page
length contiguous pages.
Update the code to properly take care of any length sg.
Change-Id: Ied380fa32cbda5384bab8ec2625ceaf1010fd314
Signed-off-by: Shiraz Hashim <shashim@codeaurora.org>
During buffer allocation if the assign call fails, we call
ion_system_secure_heap_free, which in turn does an unassign
operation before freeing the buffer. We don't want to call
unassign if the assign failed.
Fix this by calling the underlying free function directly
during error handling.
Change-Id: I2deec6fd147c4200e16f3779e37330dca1d7d173
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Add the infrastructure to support a secure cma heap
needed for secure usecases. The new heap uses
the existing cma heap infrastructure, along with adding
the assign call to assign the correct VM to the buffers
during allocation and free.
Change-Id: I0c959f46ffa6eb67128b6794e9e60c470a3d292e
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
When freeing a secure buffer, the RWX permissions need to be
passed as a parameter to the hyp_assign call. This will ensure
that when the hypervisor unmaps the buffer, HLOS can use it
for data as well as instructions.
Change-Id: I2f1e67dda8f3ebedee552a2908b7f57c18754d42
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Set the private page bit on each sg of secure buffers to inform
other entities, which import these buffers as an fd, that this
memory is secure.
Change-Id: I767c26fa1feca7505bbb3293cfdb4e09520496a6
Signed-off-by: Shrenuj Bansal <shrenujb@codeaurora.org>
Add new APIs to allow clients to map and unmap dma_buffers created by
ION. The call to the unmap API will not actually do the unmapping from the
IOMMU. The unmapping will occur when the ION/dma_buf buffer is actually
freed. This behavior can be disabled with DMA_ATTR_NO_DELAYED_UNMAP
Change-Id: Ic4dbd3b582eb0388020c650cab5fbb1dad67ae81
Signed-off-by: Olav Haugan <ohaugan@codeaurora.org>
Presently ion debug interface provides total memory held
in system heap for cached and uncached pool corresponding
to per page order list. Add additional logs to also print
total cumulative bytes held together in these pools so
that it can be accounted easily.
Change-Id: I07dbc45fa99870904cca9b2067b371f4d7247ec5
Signed-off-by: Shiraz Hashim <shashim@codeaurora.org>
The assign call apis have been updated by TZ to
include more usecases. Update the secure_buffer
api files with the same. The system secure heap
needs to be updated to reflect the change in
the api calls.
Change-Id: Idc784ddac222e6ad9f5defafc422e6e3fb88aa0c
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Move all function declarations and types to the header
file to make the macros and definitions available even
when the CONFIG_MSM_SECURE_BUFFER is not enabled. Rename
rid of the msm_ion_secure_table and msm_ion_unsecure_table
to something more generic.
Change-Id: Ia5ca0c52f971a67c7936c64b42cd2522aa1773fa
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
The secure_buffer.c files does not have anything specific
to ion. The functions defined here might be needed by other
drivers as well. If the ion driver is disabled that might
not work. So move the file to a new location.
Change-Id: I3498de791c01019b7933feb65d196152887fcf24
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
ion client can get created and destroyed dynamically.
Client data struct may not be in proper state if one core is reading
client information and other core is destroying the same client.
There is no synchronization between ion_debug_client_show() and
ion_client_destroy(), that leaves client data structures in
uncertain state and which may lead to system crash.
So add a check in ion_debug_client_show(), to verify whether the client
is part of ion device rb tree or not before retrieving client data.
CRs-Fixed: 743714
Change-Id: Ic4c2f9dff905d9bb8fbf2e56b7c6396dc01bdf8f
Signed-off-by: Prasad Sodagudi <psodagud@codeaurora.org>
We're currently using %lu and %ld to print some variables of type
dma_addr_t, which results in the following warning when dma_addr_t is
64-bits wide:
drivers/staging/android/ion/ion_chunk_heap.c: In function 'ion_chunk_heap_create':
drivers/staging/android/ion/ion_chunk_heap.c:176:2: warning: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'dma_addr_t' [-Wformat=]
pr_info("%s: base %lu size %zu align %ld\n", __func__, chunk_heap->base,
^
drivers/staging/android/ion/ion_chunk_heap.c:176:2: warning: format '%ld' expects argument of type 'long int', but argument 5 has type 'dma_addr_t' [-Wformat=]
Fix this by using %pad as instructed in printk-formats.txt.
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
ION handle buffer size should be of type 'size_t' and not
'unsigned long'. Update the API signature of ion_handle_get_size
which reads and returns the size of the ION buffer.
Change-Id: I43040816724318f5f388f0b542be963d497a754b
Signed-off-by: Banajit Goswami <bgoswami@codeaurora.org>
show_ion_usage protects against modification with a down_read.
Close it by protecting with an up_read not an up_write.
Change-Id: I2acd29188cde13cc11b4f6c8d6e35082ba37c08a
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
The buffers passed to the secure world in they hypervisor assign
call need to be flushed, so that they can be accessed by the
secure world. Flush the caches before making the hypervisor call.
CRs-Fixed: 798525
Change-Id: I95862f09daa3b0edc5e32863c6defc657f98b534
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
DMA mappers often (always?) modify the scatterlist they are
given (e.g. dma_address and dma_length) when their dma_map_ops.map_sg
operation is called. Currently Ion always returns the same sg_table
when it's dma_buf_ops.map_dma_buf operation is called. The result is
that when Ion is used as a DMA buf exporter to multiple clients (a
common scenario during zero-copy buffer-sharing) those clients will be
stomping all over each other since they're both trying to put their
mappings into the same sg_table.
Fix this by always duplicating the buffer's sg_table in ion_map_dma_buf
and returning the duplicate. Each buffer will still always have a
single "master" sg_table that should never be exported to the DMA buf
framework.
Test case (pseudo-code):
handle = ion_alloc();
dbuf = ion_share_dma_buf(handle);
attach = dma_buf_attach(dbuf);
table1 = dma_buf_map_attachment(attach);
table2 = dma_buf_map_attachment(attach);
dma_map_sg(table1);
table2 = dma_buf_map_attachment(attach);
dma_map_sg(table2);
/* table1 is now hosed */
check_mappings(table1); /* KABOOM */
Change-Id: Ifefa06ecd464c00819b4ae7ac32ad37f1491373b
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
The dmap_cnt field in struct ion_buffer is not used. Remove it.
Change-Id: I8a5d388a0e7df8612e424ca15cee27498c018508
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
The dma_sync_sg_for_* traverses the scatterlist and performs cache
maintainance using the dma_address of each link. If the scatterlist
is already mapped into an IOMMU, the dma_address is mapped into the
IOMMU, and not the cpu virtual address, which is incorrect for
proper cache maintainance. Instead traverse the scatterlist and
call the cache maintainance on each individual page.
CRs-Fixed: 768053
Change-Id: Ia989fb56236d82184a7f581b8ab4f17c0b93a010
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Ion's deferred freeing feature has been shown to greatly improve user
experience for some use cases. Enable the feature in the system secure
heap by adding the appropriate heap flag.
Change-Id: I9be7f2b3365b2d1a086acc3d182d46bfee2e6cfe
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Currently we initialize the heap free_lock and free list size in
ion_heap_init_deferred_free, which is only called when the
ION_HEAP_FLAG_DEFER_FREE heap flag is given. However, the lock and size
are used in the shrinker path as well as the deferred free path, and we
can register a shrinker *without* enabling deferred freeing. So, if a
heap provides a shrinker but *doesn't* set the DEFER_FREE flag we will
use these parameters uninitialized (resulting in a spinlock bug and
broken shrinker accounting).
Fix these problems by initializing the free list parameters directly in
ion_device_add_heap, which is always called no matter which heap
features are being used.
CRs-Fixed: 772761
Change-Id: Ib6466f940675040812b754690fd132ad95ec91d5
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Set the private page flag on secure chunks to give other entities
a clue that the memory is special.
Change-Id: Ic0dedbad4b41a874a3f80dec75d0721d30307094
Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org>
The cma secure heap allocates contiguous memory. This causes
the memory to be fragmented if several allocation and free
requests are made with buffers of various sizes. Introduce
the new ion flag(ION_FLAG_ALLOW_NON_CONTIG), which can be
passed to indicate a scatter gather list of smaller non
contiguous buffers can be allocated instead of a single
giant chunk.
Change-Id: I28b4ec8ee479e9cd565e1792e732137f0327bca3
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
[imaund@codeaurora.org: resolved context conflicts]
Signed-off-by: Ian Maund <imaund@codeaurora.org>
TZ expects the source and destination vm parameters as u32
integers. Define the parameters as u32 instead of u64.
Change-Id: I819427a1ebbb5823cf68acdbb5ea0c451ed313ea
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Add infrastructure to support a new system secure
heap, needed for secure use cases. The new heap
essentially allocates from the system heap, and
assigns the memory to the correct VM before
returning back to the client.
Change-Id: I2cd900e4399eecc3a9510139c4ebea0b713c39fc
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
We currently pass a NULL device when doing cache maintenance in
msm_ion_heap_buffer_zero. The dma ops that are used when a NULL device
is given rely on s->dma_address being the physical address of the
underlying memory. However, msm_ion_heap_buffer_zero can be called on
buffers that have previously been mapped by a different DMA mapper (like
the IOMMU DMA mapper) which might have set dma_address to something
other than the physical address of the underlying memory. This results
in us trying to do cache maintenance on some stuff that we shouldn't be.
Fix this by putting the physical address for the underlying memory back
into dma_address just before doing cache maintenance.
Change-Id: Ic5df328f5aeac09f7c9280ced887d2ba6098eb88
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
For certain secure use cases, the memory allocated using
ion needs to be re-assigned from one VM's MMU to another
VM's MMU. Call into the hypervisor to make this assingment.
Change-Id: I172f054e1b4385b8fcdaf7fdc2869034dbbd2db3
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
This reverts commit 49f5e2517691c768a884e7d1fb93306919081773.
This change results in some buffers not being zeroed in use
cases which did not show up in testing.
Change-Id: I659e7324bd21344444a0940b3403060254e8fba8
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Ian Maund <imaund@codeaurora.org>
Conflicts:
drivers/staging/android/ion/ion_page_pool.c
To support new usecases, HLOS will permit of sharing
of the CMA pool between the ion heaps. Going forward
we do not need to check if the heaps overlap.
Change-Id: I1dee72147fad994a1c88ea9dfcf7402ba98f3609
Signed-off-by: Neeti Desai <neetid@codeaurora.org>
Bugs in Ion clients are semi-common cause of out of memory
errors. Register with the show_mem notification framework
to let Ion dump out data for debugging. To avoid having to
take an excessive amount of locks, keep a running total of
allocated and potentially orphaned memory.
Change-Id: I52c31459d5bf975c67adb5f3fe17033c8e01d91b
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
All clients have been converted over to the new API
for securing. Instead of silently returning success when
secure fails, error out on a secure failure to allow for proper
error handling.
Change-Id: I57a5c1aef999cfc69a2a9cc1b3e28b2fa74fe3a6
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
If unsecure fails, this means the CPU cannot touch the
physical memory. This means if the memory is freed back to
the buddy allocator all accesses will trigger faults. If
unsecure fails, the only option is to leak the memory and
not free it.
Change-Id: I6a98761912598a9f8cdcc3914eca8e5e41147e55
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
The secure APIs are no longer called by external clients
and only called on alloc/free. The reference counting is now
useless so drop it.
Change-Id: Ide0fa6bb3f4183bbbe78d82b98166dc5942c6d3e
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
The usage hint is no longer used and is always 0. Remove
this as an argument to the lock/unlock APIs.
Change-Id: I7bafbe7823f26ba426a887e5c21f2db0852d3294
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
These APIs are deprecated as securing and unsecuring for secure
buffers is done on allocation and free respectively by the secure
heap layer. Clients don't have to call these explicitly.
Change-Id: If88cd1c47cba346446ebdcad494850b54ba954ab
Signed-off-by: Shalaj Jain <shalajj@codeaurora.org>
[mitchelh: dropped all client changes (since they're not in yet)]
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
At one time, the zeroing algorithm used a mixture of
cached and uncached mappings which necessitated an extra
cache operation afterwards. The zeroing algorithm has been
simplified so the cache operation is no longer required if
the mapping used for zeroing matches the cache status of
the buffer (cached or not). Remove the cache operation and
use appropriate page protections instead.
Change-Id: I7d3596ed503811818782e77a0f3f7f63b1456d70
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
The scm library has added support for a new secure world
interface that is more aligned to the ARMv8 SMC calling
convention. Use the new API while maintaining backward
compatibility for the mem-protect call.
Change-Id: I3a4432f2eab4c52d0fb2e8aa5d5e9dd650fd05ef
Signed-off-by: Vikram Mulukutla <markivx@codeaurora.org>
