nr_pending can go negative if we attempt to decrement it without matching
increment call earlier. If nr_pending goes negative, LLD's runtime suspend
might race with the ongoing request. This change allows decrementing
nr_pending only if it is non-zero.
Change-Id: I5f1e93ab0e0f950307e2e3c4f95c7cb01e83ffdd
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Adding a new element "tsk_dirty" to struct page increases the size
of mem_map/vmemmap, restrict this to a debug only functionality to
save few MB of memory.
Considering a system with 1G of RAM, there will be nearly 262144
pages and thus that many number of page structures in mem_map/vmemmap.
With pointer size of 8 bytes on a 64 bit system, adding this
pointer to "struct page" means an increase of "2MB" for mem_map.
CRs-Fixed: 738692
Change-Id: Idf3217dcbe17cf1ab4d462d2aa8d39da1ffd8b13
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
[venkatg@codeaurora.org: Fixed trivial merge conflict]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
Background writes happen in the context of a background thread.
It is very useful to identify the actual task that generated the
request instead of background task that submited the request.
Hence keep track of the task when a page gets dirtied and dump
this task info while tracing. Not all the pages in the bio are
dirtied by the same task but most likely it will be, since the
sectors accessed on the device must be adjacent.
Change-Id: I6afba85a2063dd3350a0141ba87cf8440ce9f777
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
[venkatg@codeaurora.org: Fixed trivial merge conflicts]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
spin_lock_irq() / spin_unlock_irq() is used so interrupts are
enabled after unlocking the spinlock. However, it is not guaranteed
they were enabled before.
This change uses the proper irqsave / irqrestore variants instead.
Without it, a spinlock recursion on the scsi request completion path
is possible if completion interrupt occurs when used for UFS testing.
Change-Id: I25a9bf6faaa2bbfedc807111fbcb32276cccea2f
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
Expose "sector_range", which will indicate to the low-level driver
unit-tests the size (in sectors, starting from "start_sector") of the
address space in which they can perform I/O operations. This user-defined
variable can be used to change the address space size from the default
512MiB.
Change-Id: I515a6849eb39b78e653f4018993a2c8e64e2a77f
Signed-off-by: Lee Susman <lsusman@codeaurora.org>
Unit tests submit large requests of 512KB made of 128 bios.
Current allocation was done via kmalloc which may not be able
to allocate such a large buffer which is also physically contiguous.
Using kmalloc to allocate each bio separately is also problematic as
it might not be page aligned. Some bio may end up using more than a
single memory segment which will fail the mapping of the bios to
the request which supports up to 128 physical segments.
To avoid such failure, allocate a separate page for each bio
(bio size is single page size).
Change-Id: Id0da394d458942e093d924bc7aa23aa3231cdca7
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
[venkatg@codeaurora.org: Drop changes to mmc_block_test.c]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
Current test-iosched design enables running only a single test
for a single block device.
This change modifies the test-iosched framework to allow running
several tests on several block devices.
Change-Id: I051d842733873488b64e89053d9c4e30e1249870
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
[merez@codeaurora.org: fix conflicts due to removal of BKOPs UT]
Signed-off-by: Maya Erez <merez@codeaurora.org>
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
[venkatg@codeaurora.org: Drop changes to ufs_test.c and
mmc_block_test.c]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
blk_run_queue() takes the queue spinlock and disabled irqs.
Consider the following callstack:
blk_run_queue
->__blk_run_queue
-> scsi_request_fn
-> blk_peek_request
-> __elv_next_request
-> elevator_dispatch_fn
-> test_dispatch_requests
-> test_dispatch_from
test_dispatch_from() will release the test-iosched spinlock
using spin_unlock_irq which will enable interrupts, however,
caller is assuming interrupts are disabled.
An interrupt can occur now and scsi soft-irq may be scheduled
with the following call stack:
scsi_softirq_done
-> scsi_finish_command
-> scsi_device_unbusy
scsi_device_unbusy() tries to lock the queue spinlock which was
previously locked when blk_run_queue was called, resulting in a
spinlock recursion.
Change test_dispatch_from() to use the spinlock irq save/restore variants
to prevent enabling the irq in case they were previously disabled.
Change-Id: Icaea4f9ba54771edb0302c6005047fcc5478ce8d
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
When running a test, a timer was set to detect test timeout
and to unblock the wait_event() function which is waiting for the
test to finish. This is redundant as wait_event timeout variant
gives the same functionality without the overhead of managing a
timer for this purpose and improve code readability.
Change-Id: Icbd3cb0f3fcb5854673f4506b102b0c80e97d6bb
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
The UFS tests are used for testing the functionality and performance
of the UFS driver. Some of the tests call compare_buffer_to_pattern
for data integrity checking. This function should be exposed in order
to allow compilation of ufs_test as a module.
Change-Id: I2279b0ae9dbdf4ecad073fab2b15116be2ea1713
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
Signed-off-by: Maya Erez <merez@codeaurora.org>
The test will verify correctness of sequential data pattern
written to the device while new data (with same pattern) is
written simultaneously.
First this test will run a long sequential write scenario.
This first stage will write the pattern that will be read
later. Second, sequential read requests will read and
compare the same data. The second stage reads, will issue in
Parallel to write requests with the same LBA and size.
NOTE: The test requires a long timeout.
The purpose of this test is to mix read and write requests on the same
LBA while checking for the read data correctness.
Change-Id: I6a437ce689b66233af3055d07a7f62f1e7b40765
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
[venkatg@codeaurora.org: Changes to ufs_test.c are already
present as part of earlier commit, hence drop them here]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
Introduce a new callback 'check_test_completion_fn' to test-iosched
framework. This callback is necessary to determine if a test has
completed or not in situation where the request queue is empty, but the
test was not completed.
Change-Id: I60bd8cccffacab11a5a7cba78caccf53fea3e1d8
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
[venkatg@codeaurora.org: Changes to ufs_test.c are already
present as part of earlier commit, hence drop them here]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
This test adds the ability to test the UFS task management feature
in the driver. It loads the queue with requests in order to allow
the task management to operate in full capacity.
Modify test-iosched infrastructure to support the new tests:
- expose check_test_completion()
Note: we submit 16-bio requests since the current HW is very slow
and we don't want to exceed the timeout duration.
Change-Id: I8ee752cba3c6838d8edc05747fa0288c4b347ef6
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Signed-off-by: Lee Susman <lsusman@codeaurora.org>
[merez@codeaurora.org: fix trivial conflicts in ufs_test.c]
Signed-off-by: Maya Erez <merez@codeaurora.org>
[venkatg@codeaurora.org: Changes to ufs_test.c are already
present as part of earlier commit, hence drop them here]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
Add a define for the test bio size (which is the size of a page),
this is used for allocating the right sized buffer for the bio during
test request creation.
Change-Id: I9505c85c4352009bdee442172eb8ae8f4254cfb0
Signed-off-by: Lee Susman <lsusman@codeaurora.org>
Update logging with:
- prefix with module name
- add '\n' in the end
- test_pr_* removed
Change-Id: I465c9809def9d294dcbb3f7cf7f474c189f5fdbf
Signed-off-by: Konstantin Dorfman <kdorfman@codeaurora.org>
[merez@codeaurora.org: fix conflicts due to removal of bkops tests]
Signed-off-by: Maya Erez <merez@codeaurora.org>
[venkatg@codeaurora.org: Drop changes to mmc_block_test.c]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
The flag REQ_IO_STAT is enabled by default this assumes statistics are
initialized and might cause NULL references in the kernel. To avoid it
this flag is cleared in the request and stats are not updated.
Change-Id: I6a1890dde51dfa8ffdd376b13f4466c9db0ae05b
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Change time measurements in long_sequential_test from jiffies to ktime,
and make the relevant change in test-iosched infrastructure.
In long_sequential_test we measure throughput, and the jiffies resolution
is not sensitive enough for this calculation.
Change-Id: If7c9a03c687f61996609c014e056bcd7132b9012
Signed-off-by: Lee Susman <lsusman@codeaurora.org>
[venkatg@codeaurora.org: Drop changes to mmc_block_test.c]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
The __blk_run_queue function is called from several contexts. The fix is
replacing it with blk_run_queue function, this function is guarded with
a lock, thus making it thread safe and prevents the crashing.
Change-Id: I3e12fa9c8b9e161375fffa3570abfa46b223a60b
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
Change the test design so that requests are dynamically created
and freed. This enables running tests with more than 128 requests,
therefore more than 50MiB can be written/read and makes it possible
to measure driver write/read throughput more accurately.
Change-Id: I56c9d6c1afba5c91a0621a16d97feafd4689521d
Signed-off-by: Lee Susman <lsusman@codeaurora.org>
[merez@codeaurora.org: fix conflicts due to BKOPS tests removal]
Signed-off-by: Maya Erez <merez@codeaurora.org>
[venkatg@codeaurora.org: Drop changes to mmc_block_test.c]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
Some block devices requires the rq_disk field to be assigned.
This patch exposes a new API to the block device test utility for
getting the rq_disk assigned, in the created request.
Change-Id: I61dc4dad50eb7600728156a6cd08bb1ee134df0d
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
The new request notification test checks the following scenario:
A new request arrives after a NULL request was sent to the mmc_queue,
which is waiting for completion of a former request.
Change-Id: I05db0959ded400e292eb5e84e1ecfc579b78ee62
Signed-off-by: Konstantin Dorfman <kdorfman@codeaurora.org>
Signed-off-by: Lee Susman <lsusman@codeaurora.org>
[merez@codeaurora.org: fixed conflicts due to removal of BKOPS tests]
Signed-off-by: Maya Erez <merez@codeaurora.org>
[venkatg@codeaurora.org: Drop changes to mmc_block_test.c]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
Add functionality to test-iosched so that it could simulate the
ROW scheduler behaviour. The main additions are:
- 3 distinct requests queue with counters
- support for urgent request pending
- reinsert request implementation (callback + dispatch behavior)
Change-Id: I83b5d9e3d2b8cd9a2353afa6a3e6a4cbc83b0cd4
Signed-off-by: Konstantin Dorfman <kdorfman@codeaurora.org>
Signed-off-by: Lee Susman <lsusman@codeaurora.org>
[merez@codeaurora.org: fixed conflicts due to bkops tests removal]
Signed-off-by: Maya Erez <merez@codeaurora.org>
[venkatg@codeaurora.org: Dropping elevator is_urgent_fn and
reinsert_req_fn ops fn as they are not present in 3.18 kernel]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
Enable the compilation of eMMC4.5 unit-tests, required by APT team.
This will allow the APT team to test the storage activity on released
builds.
The storage tests are disabled in normal operation and in order to
activate them a test I/O scheduler should be chosen and the test should
be triggered via debugfs. Therefore they have no effect on normal eMMC
driver operation.
Change-Id: I179c567f67cc8fab9ed1edab8246483de18bc76a
Signed-off-by: Maya Erez <merez@codeaurora.org>
[venkatg@codeaurora.org: Fixed trivial merge conflict]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
Change the printout format to be more readable.
Specifically, add quotes around the test case name
strings.
Change-Id: I51b0c1b94389e4b51af84c5e993207b18efc2226
Signed-off-by: Lee Susman <lsusman@codeaurora.org>
[merez@codeaurora.org: fix conflicts as BKOPS tests were removed]
Signed-off-by: Maya Erez <merez@codeaurora.org>
[venkatg@codeaurora.org: Drop changes to mmc_block_test.c]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
Long sequential read test measures read throughput
at the driver level by reading large requests sequentially.
Change-Id: I3b6d685930e1d0faceabbc7d20489111734cc9d4
Signed-off-by: Lee Susman <lsusman@codeaurora.org>
[merez@codeaurora.org: Fix conflicts as BKOPS tests were removed]
Signed-off-by: Maya Erez <merez@codeaurora.org>
[venkatg@codeaurora.org: Drop changes to mmc_block_test.c]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
In order to be sure that the packing statistics collected after the test
reflect *only* requests issued by the test (and not real request from
FS) - sleep before each test in order to give an already dispatched
requests time to complete.
Change-Id: If2f40efad1d79084a8ea85afe93cce58e49ff698
CRs-Fixed: 453712
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
- Fix test-iosched crash when running multiple tests
- Free the BIOs memory when a request is not completed
Change-Id: I1baa916c04ae73c809dee7e67ec63f4546dc71aa
Signed-off-by: Maya Erez <merez@codeaurora.org>
The test scheduler allows testing a block device by dispatching
specific requests according to the test case and declare PASS/FAIL
according to the requests completion error code
Change-Id: Ief91f9fed6e3c3c75627d27264d5252ea14f10ad
Signed-off-by: Maya Erez <merez@codeaurora.org>
A barrier request is used to control ordering of write requests without
clearing the device's cache. LLD support for barrier is optional. If LLD
doesn't support barrier, flush will be issued instead to insure logical
correctness.
To maintain this fallback flush s/w path and flags are appended.
This patch implements the necessary requests marking in order to support
the barrier feature in the block layer.
This patch implements two major changes required for the barrier support.
(1) A new flush execution-policy is added to support "ordered" requests
and a fallback , in case barrier is not supported by LLD.
(2) If there is a flush pending in the flush-queue, the received barrier
is ignored, in order not to miss a demand for an actual flush.
Change-Id: I6072d759e5c3bd983105852d81732e949da3d448
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
A barrier request is a type of flush request used to control ordering of
write requests without clearing the device's cache. LLD support for
barrier is optional. To maintain backward compatibility, barrier request
has to maintain flush s/w path and flags.
This patch introduces those flags to define interface between the block
layer and the LLD.
Change-Id: Iefa8e9a5c1b5e8256eaeb0322c435becd4669de9
Signed-off-by: Dolev Raviv <draviv@codeaurora.org>
[imaund@codeaurora.org: Resolved context conflicts]
Signed-off-by: Ian Maund <imaund@codeaurora.org>
Add new flag "simple_scaling" to on demand governor so that
the clocks can be scaled up only when the load is more than
up threshold and can be scaled down only when the load is less
than down differential data as provided within
struct devfreq_simple_ondemand_data.
Change-Id: Ibc6ab6297c1b64b6e6eaaa76d735d0b9ae0f6477
Signed-off-by: Sahitya Tummala <stummala@codeaurora.org>
[venkatg@codeaurora.org: dereference stat variables]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
This patch adds a new flag to be used in cmd_flags field of struct request
for marking request as urgent.
Urgent request is the one that should be given priority currently handled
(regular) request by the device driver. The decision of a request urgency
is taken by the scheduler.
Change-Id: Ic20470987ef23410f1d0324f96f00578f7df8717
Signed-off-by: Tatyana Brokhman <tlinder@codeaurora.org>
[subhashj@codeaurora.org: resolved trivial merge conflicts]
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
[venkatg@codeaurora.org: resolved trivial merge conflict]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
CONFIG_PM_RUNTIME has been eliminated as of kernel 4.4 in favor of
just using CONFIG_PM, hence remove its use.
Change-Id: I096cde564c7bf2c7a041f0948617a7d74df71dd9
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
FILLER relock on error recovery may create an alignment that is not
compatible with host controller. When this issue happens, host controller
no longer detects symbols until LINE-RESET is done by host controller as
a recovery sequence. This change disables FILLER symbol relock.
Change-Id: Id7147bfb0de6be45de4936fe3429a9ad76a3868b
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Hibernate entry failures have been observed with some UFS devices.
The UFS PHY Rx termination is not disabled within the specified
RX_Min_STALL_NoConfig_Time_Capability of 15 SI. The UFS device enables
mid-termination after RX_Min_STALL_NoConfig_Time_Capability SI. The
combination of Rx terminated and device Tx mid-termination collapses
the line state to near the differential mid-point. The signal detect
may report the line state incorrectly as DIF-P. The incorrect DIF-P
moves the Rx FSM into the HS-Burst state rather than the intended
Hibern8 state. Fix this issue by setting the UFS PHY's
RX_MIN_STALL_NOCONFIG_TIME_CAPABILITY to 40.
Change-Id: I475e03686831e8131fd7ec1c30e6e8f53a6e188c
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
This change updates the initialization sequence of the QRBTC-V2
phy, for msmcobalt RUMI platform.
Change-Id: I369608b41e0b150334e9a7e9ad99b7cde8341dae
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
Some callbacks may not be supported which is a valid case, and should not
be considered as error.
In such cases, and in order to reduce unnecessary logs, we better
remove this error messages.
Change-Id: I75b9e02a2189a330e0ca46387394be4a8c60ef5b
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
This change updates the offsets and calibration values of the
ufs-qmp-v3 phy.
Change-Id: I443a857b7b6620a65acd14e3a805eea9c00fd61c
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
This reverts commit 6ed4bd3af2c153ae3246d89c2d62cfe7919d0320
("phy: qcom-ufs: add ufs phy type selection")
Since a single boot.img should support all phy-qcom-ufs configurations,
all phy-qcom-ufs files should be compiled all together.
Change-Id: I81b3891fb44d927046f39bbe72b5ed24dd9d99d2
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
[venkatg@codeaurora.org: resolved trivial merge conflicts]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
This change adds the UFS PHY initialization table for UFS controller
version 2.2.0.
Change-Id: Ia77dec85433041e2fab3c89df10cc4727e4a47e9
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Add support for QRBTC V2 UFS PHY that is used in msmcobalt rumi platform.
Change-Id: I21ad3f0db23ea16d05ba40593cc7650e1a443702
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
Add support for new QCOM UFS PHY that is used in
future platforms.
Change-Id: I53f162738668ae9f24f5edb9c42a17f947e68b40
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
[venkatg@codeaurora.org: resolved trivial merge conflict]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
This patch removes the hard-coded voltage number of vdda-phy regulator
and instead, gets it by using the appropriate API.
Change-Id: Iab753c7e94424dd3ccce95296258bba264979cf2
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
Since in future UFS Phy's the tx_iface_clk and rx_iface_clk
are no longer exist, we should not fail when their initialization
fail, but rather just report with debug message.
Change-Id: I87e4deca632d4d4ee9e35a03f2fc9c22987255b3
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
Hardware programming guide has updated the PHY power up sequence,
this change adds the relevant changes.
Change-Id: I320e38f501cdafc053d47bf2b21ba7f69b1b12a7
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Currently both 14nm phy and 20nm phy are compiled and built into
the kernel, while only one type is actually present and used.
Add a choice selection to the kernel configuration to specify
which phy type shall be used.
Change-Id: I5608d32fb4815db15e91a970e53099762eabbddd
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
[venkatg@codeaurora.org: resolved trivial merge conflicts]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
In order to allow UFS PHY and UFS driver to be built as
kernel modules, the driver must export its functions so
they can be used by external code.
Change-Id: Ic617daf1038b5727dcd4b24a4c12a1f1bb8d730e
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
[venkatg@codeaurora.org: keep upstream version of EXPORT_SYMBOL_GPL]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
