Current code does not actually handle ICE errors as the
relevant quirk is not set. Also, ICE errors are checked
only if some host controller error occurred.
Removed the quirk and fixed the check for ICE errors.
Also, removed redundant API crypto_engine_get_err()
and crypto_engine_reset_err().
Change-Id: Ic7f4e9a2cd3771f1f52dff97b2be90d12e32d2e5
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
[venkatg@codeaurora.org: dropped msm/ice.c changes]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
Add support for compat ioctl so that 32-bit application can
access the UFS ioctl calls on 64-bit kernel.
Change-Id: Id4793bea84405adc7d692d6cf6f2265d1e491c28
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
"bBootLunEn" attribute value decides the active Boot LUN but there could
be a case where userspace may want to change it during image update.
This change allows writing to this parameter.
Change-Id: I6e8d630606d8caf6197ef662003ec1ee646afe81
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
from include/linux/kobject.h:21:0,
from include/linux/device.h:17,
from include/linux/devfreq.h:16,
from drivers/scsi/ufs/ufshcd.c:42:
drivers/scsi/ufs/ufshcd.c:
In function 'ufshcd_init_clk_gating':
drivers/scsi/ufs/ufshcd.c:1548:19:
error: 'clk_gating' undeclared (first use in this function)
sysfs_attr_init(&clk_gating->enable_attr.attr);
Change-Id: Ie61be8bc4472cf8960dc0afc3e0683c761ea177e
Signed-off-by: Patrick Daly <pdaly@codeaurora.org>
In the old code, there is no way to "disable" the already enabled
error scenarios since the operator in use is a bit-wise OR operator.
This change fixes this issue. Now, the new value overwrites the previous
value so it's possible to disable all error scenarios by:
echo 0 > /d/ufshcd0/err_inj_scenario
Change-Id: I21eadb3345f6587edf5232ea3c03702bfdb930ad
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
The condition in which error message is printed out was incorrect and
resulted error message only if retries exhausted.
But retries happens only if DME command is a peer command, and thus
DME commands which are not peer commands and fail are not printed out.
This change fixes this issue.
Change-Id: I43367befd810da0ebe34efd3a93751b582a19565
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
There is an inconsistency between the actual power mode of the
UFS controller (and UFS device), to the power mode informed to upper
layers in case an error is injected during ERR_INJECT_PWR_CHANGE
scenario. Meaning, power change error is reported although the power
mode has actually changed successfully in the controller and in the
device. This change fixes this inconsistency.
Also, this change fixes a bug where a pointer to u8 is sent as
a pointer to int, which results undesired behavior.
Also, this change removes unused error injection scenarios:
- ERR_INJECT_HIBERN8_ENTER
- ERR_INJECT_HIBERN8_EXIT
Change-Id: I14116656862e3db8f5d6a93b5c798a1470a6b716
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
The removed condition is already checked inside the routine
ufshcd_change_power_mode().
Also, a minor typo is fixed.
Change-Id: Ib238ff9c4bdcd380406505d5319bd2d6c0ed6063
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
In case UFS driver is probed before the phy driver does, the
UFS driver should return a PROBE_DEFER code.
Change-Id: I94ab353fed35889b92fb2f0baf5ee029894b999b
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
This change fixes the signature of the routine
ufshcd_vops_clk_scale_notify() and the callback clk_scale_notify
to receive enum ufs_notify_change_status instead of bool as their 3rd
parameter.
Change-Id: I8463f4f5f0a77bcbc3b0bcdfd05d1a02e334dc8e
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
This change fixes a bug where we check the select_minor value
instead of checking the select_major value.
Change-Id: I852944215a0be484449b578be09b46095fb01931
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
err_inj_scenario debugfs entry used to be a u32 entity.
This change modifies it to be a file entity, so reading it
(cat /d/.../err_inj_scenario) helps the user get useful information
about the enabled/disabled error scenarios.
Change-Id: Ia805e6f5357009752ddcace3ae4ed87061e48759
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
We shouldn't be accessing scsi command's "request" structure after calling
scsi_done callback but it seems pm qos unvoting code path is accessing
this request structure even after caling the scsi_done callback. Fix this
by releasing the pm qos vote before calling scsi_done.
Change-Id: I5c8dd6ba7f0b619397924d73c1fd6c36fe86ebc1
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
This change removes error scenario use-cases that have not been
implemented yet, and one error scenario (ERR_INJECT_LINK_STARTUP)
that anyway is not recoverable.
Change-Id: I78948f64ee3868333ec27e17695980b642248dcf
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
In order to simplify working with error injection feature,
and since the capability of controlling error codes per error scenario
has not been useful in debugging phase of the driver, it's decided
to remove this feature.
Now, the error injection framework randomly choose an error code from
a list of error codes available per error scenario.
Change-Id: Iee34b2aa99321f5f51190f26da364c9c129a3f77
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
Query commands have 100ms timeout and it may timeout if they are
issued in parallel to ongoing read/write SCSI commands, this change
adds the retry (max: 10) in case command timeouts.
Change-Id: If60e2dd85614c8e383018b1286a625a5e53f872f
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
UFS core driver may power collapse the UFS host controller during clock
gating hence we might loose some of the ICE register configuration.
In order to restore ICE register configuration, we should call ICE
driver's resume callback after ungating UFS clocks.
Change-Id: I722a8037d81222c2cd493ebbe608a106f47cedfd
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Currently we have the same gating timeout for both high load and
low load condition but having the different timeouts would help
to save power in low load condition and increase performance
in high load condition. This change also adds support to tune
these timeouts via sysfs.
Change-Id: I0ac79042d0fd0cd6a6a917ebe3b52db2a1abd0b9
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
This change enables the aggressive clock gating along with hibern8 at
every 10ms. As hibern8 enter was already @10ms, we are now merging it
with clock gating which will also be @10ms after this change. This change
is needed to save power for real life usecases.
Change-Id: I297d0f7d1c379f0b402298dc5dddd92ac33f6f25
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Currently we are suspending clock scaling during clock gating which doesn't
allow us to have clock gating timeout lower than clock scaling polling
window. If clock gating timeout is smaller than the clock scaling polling
window then we will mostly suspend the clock scaling before clock scaling
polling window expires and we might get stuck in same state (scaled down
or scaled up) for quite a long time. And for this reason, we have clock
gating timeout (50ms) greater than clock scaling polling window (40ms).
We would like to have aggressive clock gating timeout even lower than the
clock scaling polling window hence this change is decoupling the clock
scaling suspend/resume from clock gate/ungate. We will not suspend the
clock scaling as part of clock gating instead clock scaling context will
schedule scaling suspend work if there are no more pending transfer
requests.
Change-Id: I7c1e7580c63bbbfbad551d17921774afa3d95ca6
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Currently we call the scsi_block_requests()/scsi_unblock_requests()
whenever we want to block/unblock scsi requests but as there is no
reference counting, nesting of these calls could leave us in undesired
state sometime. Consider following call flow sequence:
1. func1() calls scsi_block_requests() but calls func2() before
calling scsi_unblock_requests()
2. func2() calls scsi_block_requests()
3. func2() calls scsi_unblock_requests()
4. func1() calls scsi_unblock_requests()
As there is no reference counting, we will have scsi requests unblocked
after #3 instead of it to be unblocked only after #4. Though we may not
have failures seen with this, we might run into some failures in future.
Better solution would be to fix this by adding reference counting.
Change-Id: I1eaa39c0716cf41120269cf389ebb322e3006da1
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Enabling & disabling host PHY ref_clk generally may have high latencies
hence don't disable them during aggresive clock gating otherwise it could
affect the UFS performance badly.
Change-Id: I1741c75a6295c7c0b0d77ff578ab977213dcc484
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
It is quite possible that we might need multiple clocks to be enabled when
UFS transfers are active and we generally turn these clocks off during
runtime & system suspend. In addition, we also vote these clocks off if
aggressive clock gating feature is enabled. Idle timeout for aggressive
clock gating feature is generally ~50 milliseconds. But turning these
clocks on/off could have huge latencies hence we might only want to turn
off few essential (and low latency) clocks during aggressive clock gating.
This change adds support to let the vendor specific setup_clocks callback
know whether it is called from aggressive clock gating context or normal
clock gating context. Having this context information should help vendor
specific setup_clocks callback to selectively disable clocks.
Change-Id: I5e1523a57bc45a91faef463baac1cea2a2c8d2d6
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
[venkatg@codeaurora.org: resolved trivial merge conflicts]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
Voting/devoting device ref_clk source could be time consuming hence add
support to allow ufs variant to decide if it wants to control ref_clk
during aggressive clock gating or not. This clock should still be gated
off during runtime/system suspend.
Change-Id: Ibec8cc8b3689ff4e2418039e74dbf311e589157e
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Currently we power collapse UFS controller core only during runtime suspend
but we could also power collapse it during clock gating as power collapse
latencies are only few microseconds. This change adds this aggressive power
collapse to save power.
Change-Id: Id550bcbcd044670cec08a6abb7225cb9386993b5
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
[venkatg@codeaurora.org: resolved trivial merge conflicts]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
Some platforms don't support restoring the secure configuration as
secure configuration can never be lost hence scm_restore_sec_cfg() is
expected to be failed for such platforms. This change changes the
failure message level from error to debug.
Change-Id: Ie4b96971d5c4c394576a78444fe5013749375c28
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
If we had called the pm_qos_add_request() with PM QoS request type set to
PM_QOS_REQ_AFFINE_IRQ then freeing up the irq makes the free_irq() to
print out warning with call stack. We don't really need to free up irq
during suspend, disabling it during suspend and reenabling it during
resume should be good enough and that's how it was done on 3.10 kernel
as well.
Change-Id: I58379a05e1f9d5f18b46394d64bdf6f106afa963
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
UFS device ref_clk frequency is set during the one time provisioning,
UFS driver just queries the reference clock frequency from device and
prints it out for informational purpose. Although this seems redundant
as incorrect reference clock frequency would anyway be detected during
bootup phase itself. This patch removes this redundant query and in turn
removes one message from going to serial console.
Change-Id: Id6a4d6d84b05f4b07b5299fd90e78e793e9689dc
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Add debug-fs entry to show current state of UFS PM QoS.
Change-Id: Ic1ae582b5a657f332be8a6f8d3bbcc0434bdf3d3
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
Add sys-fs entries to allow user space control PM QoS
latency parameters and enable/disable PM QoS voting.
The entries are added under:
/sys/bus/platform/devices/624000.ufshc/624000.ufshc:ufs_variant/
pm_qos_enable:
write 0 to disable PM QoS, 1 to enable.
Example: "echo 1 > pm_qos_enable"
pm_qos_latency_us:
write the desired value for each cpu group, separated by a comma.
Example: "echo 10,20 > pm_qos_latency_us"
Change-Id: I9797a1e62c4867ab831b4f18cbb1e0ca9834247b
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
Signed-off-by: Krishna Konda <kkonda@codeaurora.org>
[venkatg@codeaurora.org: resolved trivial merge conflicts]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
Current UFS PM QoS design and implementation do not seem to give
the desired increase in performance. This change revisits the
PM QoS implementation trying to improve performance by making
the following changes:
* de-couple voting from clock scaling decision so voting
occurs from the first request and unvoting on the completion
of the last request regardless to clock scaling state.
Otherwise, suspending the PM QoS voting during the time it takes
to decide on clock up-scaling, seems to degrade random access
performance.
* vote on a per-cluster basis by inspecting the request object's
cpu field. This follows the soft-irq allocation scheme in the
block layer, so the cpu to which the block layer schedules the
soft-irq will not be put into deep LPM.
We should note that PM QoS voting using cpu mask for specific
cpus is a feature of the qcom specific PM QoS implementation.
Change-Id: I427d202aeb45cd284a3bb128e26e519212614801
Signed-off-by: Gilad Broner <gbroner@codeaurora.org>
Signed-off-by: Krishna Konda <kkonda@codeaurora.org>
[venkatg@codeaurora.org: resolved merge conflicts]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
The new UFS Controller (version 3.0.0) has a few changes compared to
previous versions, such as a new vendor specific register, and
different debug register offsets.
Change-Id: Idf455d2dcbdcdadcb1e809a8349fc9adbe9733dd
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
Some platforms do not enable the BUS_SCALING feature.
For such cases, we should wrap the routines that use this feature
with a configuration flag.
Change-Id: Iced5b6492a470eb5700b50a50de8260b6bb59753
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
The UFS HCI v2.1 includes a few additional registers. This change
updates the HCI register, the UFS version register content and
the Interrupt Status register.
Change-Id: Icfeef0e6aaed064b456fac1cd5531fff1328ae37
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
When we are resuming the UFS device rails in HPM mode, we are first
powering on the VCC rail while VCCQ and VCCQ2 rails still being in LPM
mode. Some UFS devices may take VCC on event as hint that host wants
UFS device to be resumed and may start drawing more power from the
VCCQ/VCCQ2 rails (while they are still in LPM mode) causing voltage drop
on these rails. This change fixes this issue by bringing VCCQ & VCCQ2 rails
out of LPM before powering on VCC rail.
Change-Id: I950591d2a571257e08d7697e2b33dd5968f7333f
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
This patch splits the vops by type and creates wrapper
functions for all vops.
Used in order clarify the runtime code and to allow
further types of vops.
This is the first step towards rearchitecting the
variant implementation of the UFS driver - per requirement
from the community. As this will become a full-blown
rearchitecture - incrementing the driver version.
Change-Id: Ifd0a5412a608590bc6c761caf0b3a87aed37f70b
Signed-off-by: Dov Levenglick <dovl@codeaurora.org>
[venkatg@codeaurora.org: resolved merge conflicts,
change ufshcd_pltfrm_init to use ufs_hba_variant
instead of ufs_hba_variant_ops]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
Adds setter/getter functions to set/get a pointer
to the variant control structure in/from the generic
hba control structure.
Change-Id: I4c7b3967366f394472490410e3880fdc6e3608b4
Signed-off-by: Dov Levenglick <dovl@codeaurora.org>
[venkatg@codeaurora.org: use ufshcd_set_variant instead of
ufshcd_bind_variant]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
When compiling with gcc5, it throws below errors:
drivers/scsi/ufs/ufs-qcom.c: In function 'ufs_qcom_hce_enable_notify':
drivers/scsi/ufs/ufs-qcom.c:352:10: warning: switch condition has boolean
value [-Wswitch-bool] error, forbidden warning: ufs-qcom.c:352
This is because we are using the boolean value in switch condition and it
seems to be forbidden by compiler (by default). Fix this by using the
proper enum type instead of boolean.
Change-Id: Iee6950e5986dc51afbf9440cd4773b4d0f02fcdc
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
[venkatg@codeaurora.org: resolved trivial merge conflict]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
We are currently running UFS link in HS-G3 FAST mode during high load
condition for best possible performance and in HS-G2 FAST mode during
low load condition to save power. As we are anyway scaling down from
HS-G3 to HS-G2, we can also change the mode from FAST to FASTAUTO.
So we looked at the performance numbers with HS-G2 FASTAUTO mode and
they are good enough for most of the low bandwidth usecases. But Samsung
UFS memory devices are exception which has really low sequential read
throughput in FAST AUTO mode hence we will only be enabling FAST AUTO mode
for other UFS device vendors.
Change-Id: Ia79da3e82fb87c0e396534979398617388f40af9
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
DME commands such as Hibern8 enter/exit and gear switch generate 2
completion interrupts, one for confirmation that command is received
by local UniPro and 2nd one is the final confirmation after communication
with remote UniPro. Currently both of these completions are registered
as interrupt events which is not quite necessary and instead we can
just wait for the interrupt of 2nd completion, this should reduce
the number of interrupts and could reduce the unnecessary CPU wakeups
to handle extra interrupts.
Change-Id: I5365b8b5cae952fd4bd7bb93cca0cfe76ea8b619
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
UniPro v1.6 specication requires both host and device TX LCC Enable to be
in same state but current driver still attempts to disable just one
side of LCC. With this change, if Host supports UniPro v1.6 (and all
commercial UFS devices anyway supports the UniPro v1.6), we would keep
both host and device TX LCC disabled after the link startup. Keeping LCC
disabled helps avoid multiple compatibility issues between host and device.
Change-Id: I1697089255b63f70a465c8b42a899ee6ab0acd88
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
[venkatg@codeaurora.org: resolved trivial merge conflicts]
Signed-off-by: Venkat Gopalakrishnan <venkatg@codeaurora.org>
DME attributes read/write commands need GenSelectorIndex argument and
according to UFS host controller specification, it should specify the
targeted M-PHY data lane.
This is the valid range of GenSelectorIndex for M-PHY attributes:
0 to (2 * PA_MaxDataLanes - 1)
Example (Note: PA_MaxDataLanes is UniPro protocol constant set to 4):
-----------------------------------------
MPHY Access target | GenSelectorIndex
-----------------------------------------
TX Lane0 | 0
TX Lane1 | 1
RX Lane0 | 4
RX Lane1 | 5
-----------------------------------------
This change makes sure that correct GenSelectorIndex is passed for M-PHY
attributes while it reading via debugfs.
Change-Id: If09e4271958e4bb18d315446cef253caec75584a
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
Quite a few UFS functions (especially those which are invoked in debugfs
context) are calling the ufshcd_hold_all() first before runtime resuming
the UFS host controller. As runtime suspend might have already power
collapsed UFS core controller, attempt to enable clocks (as done by
ufshcd_hold_all()) in power collapsed state will fail. This change fixes
this issue by calling pm_runtime_get_sync() before ufshcd_hold_all().
Change-Id: Iff1d54d815efafbc17d56562de2e9065770c2612
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
During power mode change, PACP_PWR_Req frame sends
PAPowerModeUserData parameters (and they are considered valid by device if
Flags[4] - UserDataValid bit is set in the same frame).
Currently we don't set these PAPowerModeUserData parameters and hardware
always sets UserDataValid bit which would clear all the DL layer timeout
values of the peer device after the power mode change.
This change sets the PAPowerModeUserData[0..2] to UniPro specification
recommended default values, in addition we are also setting the relevant
DME_LOCAL_* timer attributes as required by UFS HCI specification.
Change-Id: I5ff98c03144e6973848d2fa1bf2aafd7637025fb
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
UFS clock scaling may do couple of things such as scaling UFS controller
clock frequencies and UFS interface gear scaling. To ensure that these
operations are completed successfully, driver need to make sure that there
are no regular transfer commands allowed until clock scaling is completed.
Currently we are only blocking the scsi command requests during gear
scaling but not during clock frequency scaling (and related operations).
So first fix is to block scsi requests for entire clock scaling operation,
we are doing this by invoking scsi_block_requests() and then waiting for
all pending requests (doorbells) to be completed. But this approach can not
always ensure that no new request will be issued while clock scaling is in
progress. Think of a scenario where one request just got queued via
->queuecommand() (but still haven't set the doorbell) when no doorbells
were pending and assume that clock scaling function has also started
executing in parallel. In this case clock scaling function blocks the new
scsi requests from being issued and wait for all the pending doorbells
to be cleared. As there were no doorbells pending when it checks for it,
it decides to go ahead with clock scaling but now the command with was just
issued via ->queuecommand() sets the doorbell and gets issued by
controller.
This change fixes such race condition by using the read-write semaphore
approach which basically blocks all the requests until clock scaling work
finishes but would allow multiple requests to be queued in parallel if
clock scaling work isn't running.
Change-Id: I99beed27bbc5b768fb3cee8b84cb5392619cace7
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
It seems arguments passed to memset function are swapped, this change
fixes it.
Change-Id: I1f69d528da9030a2bc061d3c9504058d8b71068b
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
We are only enabling the lane-0 symbol clock in 1-lane configuration
but it seems during disabling we are trying to disable lane-1 symbol
clocks as well even if they were never enable in 1-lane configuration,
this change fixes this minor issue.
Change-Id: I619a7c2e83a96caa338a56d53e61bba3d982ea13
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
This patch adds a new debugfs capability to collect query statistics.
It counts how many times each IDN (index) was sent to device for each
query opcode.
A usage example:
1. to clean current query statistics:
echo 1 > /d/ufshcd0/stats/query_stats
2. to dump the current query statistics:
cat /d/ufshcd0/stats/query_stats
This capability is always on, and can not be turned off.
Change-Id: I46ec405aae480c0dc161dca015b407bde6335cf7
Signed-off-by: Yaniv Gardi <ygardi@codeaurora.org>
