Newer versions of bimc-bwmon counters have the capability to fake
higher byte count than what's actually transferred between a bus
master and DDR if the bus master is being throttled by QoS hardware
logic. Add support to set the throttle adjust field that comes with
this newer version of bimc-bwmon.
Change-Id: I33376c825fb11ab2e378f828b1d2ae46dd582836
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
[junjiew@codeaurora.org: dropped changes in dtsi.]
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
When doing over-voting due to an UP_WAKE event, the governor
pre-initialized the max_mbps value for the next decision window. This was
done to prevent a dropping the vote below the original measurement in case
a DOWN_WAKE event comes before the decision window is complete. This has
the undesirable side-effect of keeping the votes high for two decision
windows following an UP_WAKE event.
Also, DOWN_WAKE event is currently disabled for any decision windows that
didn't do over-voting because doing so had significant performance impacts.
However, with historic max tracking, pattern detection and hysteresis, and
mbps zones awareness that were added later on, the above concerns aren't
really valid anymore. So, simply delete the pre-initialization of the
max_mbps so that one decision window isn't unnecessarily voting higher than
necessary. As part of the change, also fix down_cnt logic to allow it to
work for all decision windows if enabled.
Change-Id: I600e3783da617ed4efef5b05156016d88c301cea
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Currently the lower limit for setting up_scale is 100. Change it
to 0 so that over-voting for spikes in measured bandwidth can be
disabled by setting up_scale to 0.
Change-Id: I1e5e3d51c625163b38074fb1eda26763455ea6c5
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Using debug logs for the short samples would result in excessive logging
and also have an impact on CPU power and performance. The tracepoint that's
already there is a much better alternative. So, delete the debug logs.
Change-Id: Idd1d634f510381d14637f3b84588a118572f3f16
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
The IRQ registration on the resume path was not updated properly when the
high sampling rate algorithm was implemented. Update it so that the IRQ
registration is done correctly.
Change-Id: I17a016dd9c0b50c7b415beda21368cb4586d203c
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Replace measured mrps and frequency vote debug prints with trace
events.
Change-Id: I78370b068e3819a57635cbabaf5cdd053ebabce4
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Change the sequence of registering and freeing the interrupt
handler in suspend/resume. Freeirq needs a guarantee that the
IRQ can't come anymore before we call it. So, we disable the IRQ
before calling freeirq.And register the handler before enabling
the irq to avoid the interrupt getting unhandled.
Change-Id: I3945202d049e16f64a16e456f914f7602b763c89
Signed-off-by: Hanumath Prasad <hpprasad@codeaurora.org>
Due to the way the short samples are triggered, when the traffic is very
low, we could have a decision window where no sample is taken. Fix this by
forcing a sample at the end of every decision window as long as that is the
first sample for that decision window or the sample window won't be smaller
than the sample_ms tunable.
Change-Id: Ia33b59fbff9ff4058c06e24208d1e037d6e9eda9
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
With the rewrite of the algorithm, some tunables are not used. Remove those
tunables. Other tunables don't make a lot of sense to be turned on by
default. Change their default state to be disabled.
Change-Id: I228d275a21765986a7117b335a669deebf590f29
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
When the decision window is terminated early by an IRQ and the frequency
voted by the governor is the max_freq allowed for that device, the AB vote
can be lower than the measured value. This really shouldn't affect power or
performance since the device frequency is already maxed out, but fix it
anyway so that the logs don't look suspicious.
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Change-Id: I1788cbb1af026f1c54f9acc4806cb29cd6069e04
M4M counters are only 28-bit instead of 32-bit. Fix limit calculation
to use the right max value.
Change-Id: I91078842b72da80f6b6755bf8d808ff4b4142f10
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Enable cycle counter along with other counters when m4m_hwmon is
started.
Change-Id: Idd86b99c21e21f64bb91db9b1e64597fbfb2306a
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Some targets have a single irq line which is shared among all
the cci hwmon counters. Enhance the driver to support shared interrupt
handling.
Change-Id: I5fdaecfaa14fa47e8f393fe51c538e5000e6ad5b
Signed-off-by: Arun KS <arunks@codeaurora.org>
Signed-off-by: Hanumath Prasad <hpprasad@codeaurora.org>
For certain implementation, device clock needs to be prepared before
rate voting taking effect. Add support for preparing device clock
during initialization.
Change-Id: Ib22e83952187118342ff2546d4c79d3970a288f9
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
The existing bw_hwmon governor samples the bandwidth every polling_interval
milliseconds and makes decisions. Polling interval of 50ms or even 10ms
gives a very low resolution picture of the DDR/bus traffic. Due to the
lower resolution picture, the existing governor algorithm has to be biased
aggressively towards performance to avoid any performance degradation
compared to using a static mapping between bus master (CPU, GPU, etc)
frequency to DDR/bus BW votes.
While the existing governor uses IRQ to get early notification of traffic
increase, even a 4x early notification for a 50ms polling interval still
takes 12.5ms. This kind of reaction time is still too slow for some bus
masters like CPU.
To take care of these limitations, rewrite the governor algorithm to take
multiple short samples of BW within a decision window (polling interval)
and use that higher resolution picture to make much better and faster
decisions. Doing so allows the governor to have the following features:
- Very low reaction time
- Over voting to stay ahead of increasing traffic
- Historic peak tracking to limit over voting
- Being power aware when doing over voting
- Pattern detection and intelligent hysteresis
- Detection low traffic modes and being less aggressive about BW votes
Change-Id: I69886b7fbeea0b64d10b5a1fb23fcb5f3918f0ce
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Accidentally used a , instead of a ; to end statements. This change fixes
that. Should not have any functional impact.
Change-Id: I36ea6c49a8a8973f1856cac6709f01432e6d60c7
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
A device register is read and printed using pr_debug. Register reads can be
slow. So, refactor the code such that the register read is also compiled
out if the pr_debug is compiled out.
Change-Id: I4de5ab1f2133e19d6ce0b3cb4547b815526c7abd
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
The counter needs to be cleared and acknowledged before clearing the IRQ
bits. Otherwise, the HW could set the IRQ bit again if the old counter
value was higher than the threshold. So, add a memory barrier after
clearing the counter.
Change-Id: I35f9f7905b05b8a185eb94d04d9c0a8ccfc2db51
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Add trace points to log the measurements and the final decision made by the
bw_hwmon governor.
Change-Id: I834bec340f81f02fcccbd6d7568a301dc74dded5
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
The bw_hwmon_meas trace event is to log measurement details and the
bw_hwmon_update trace event is to log the final decision.
Change-Id: I839ace50b1f1686227bcbf7d38a75f89092d26b1
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
[junjiew@codeaurora.org: resolved trivial conflicts]
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Some cache_hwmon devices can have interrupts firing at any time. The
interrupt handler would stop devfreq monitor, update its vote and
restart the monitor again. This introduces a race if
devfreq_supend/resume() or devfreq_interval_update() is called at
the same time. Since devfreq_monitor_start() re-initializes the work,
it could cause corruption while the work is being used elsewhere.
Protect governor monitor start/stops with a new lock.
Change-Id: I143aaaea86494b4c617df46e2c521a19b43861d5
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
On some systems, scm calls for cci registers are not supported.
Use normal writel/readl in those cases. Add device tree flag to
distinguish them.
Change-Id: Icfb609d43f888856786c1881b2ee34ffd501e37a
Signed-off-by: Arun KS <arunks@codeaurora.org>
CCI400 on MSM has additional PMU counters that can be used to monitor
cache requests. MSM CCI hardware monitor device configures these
registers to monitor cache and inform governor. It can also set an
IRQ when count exceeds a programmable limit.
Change-Id: I1d80f57749b91c3972e60e54c75226c4d49d2ec6
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
[junjiew@codeaurora.org: Dropped change in arch/arm64/Kconfig.
Configuration should be selected directly in defconfig.]
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Interrupt storm happens when bwmon is enabled for GPU. This is mainly
due to constant low traffic observed with GPU while doing memory
read/write. So as the data rates read from counters are low and so the
threshold set for triggering the interrupt also set as low, which in
turn causes huge number of interrupts. Avoid this by setting a minimum
floor for the irq threshold.
Change-Id: I190fad5108bc24afcb67bec5809485380ee3662e
Signed-off-by: Hanumath Prasad <hpprasad@codeaurora.org>
Send all the frequency performance levels in KHz units to TZ driver and
convert all the bandwidth recommendations to Bytes/s from Kbytes/s to
accomodate change in the syscall interface calls to TZ SPDM driver.
Change-Id: I209ea0583fdd43f78f51793d7818ea5afd5959c7
Signed-off-by: Dilip Kota <dkota@codeaurora.org>
Signed-off-by: Girish Mahadevan <girishm@codeaurora.org>
The TZ SPDM driver mandates the number of arguments be constant for a
given command id. This patch allows the spdm driver to communicate with
the TZ driver correctly.
Change-Id: Id3cfd2490039c4abde6d5381859d8facf545e85e
Signed-off-by: Dan Sneddon <dsneddon@codeaurora.org>
Signed-off-by: Girish Mahadevan <girishm@codeaurora.org>
A devfreq governor store in parallel with a cpu freq update can cause
deadlock as shown below.
Assume current devfreq governor is cpufreq, and user tries to change
to some other governor.
Write to sysfs store_governor | cpufreq driver updating cpu freq
------------------------------- | -----------------------------------
echo bw_hwmon > governor |
| takes rcu_read_lock and calls all
| cpufreq transition callbacks for
| PRECHANGE or POSTCHANGE
|
GOV_STOP on governor_cpufreq. |
unregister_cpufreq() accquires |
state_lock mutex. |
| try to accquire same state_lock in
| cpufreq_trans_notifier(). Blocked.
unregister from cpufreq |
transition notifier and wait for|
all rcu_readers to finish. |
Deadlock
A similar deadlock can happen with governor change and policy notifier
callbacks.
The state_lock currently protects multiple unrelated critical
sections: registering/unregistering of cpufreq notifiers, read/writing
the device list, and tracking the cpu states and updating device
frequencies. There is no need for register/unregister of the cpufreq
notifiers to be mutually excluded against the other critical sections
using the same lock.
Split state_lock into two locks to protect the register/unregister of
cpufreq notifiers from the rest of the critical sections.
Change-Id: Id06d326748a5cb0c84c4787da5d0910f44eb5c3c
Signed-off-by: Pan Fang <fangpan@codeaurora.org>
Signed-off-by: Arun KS <arunks@codeaurora.org>
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Suggested-by: Saravana Kannan <skannan@codeaurora.org>
Use free_irq to free the interrupt handler for a shared interrupt.
Enable_irqs are not refcounted, whereas disable_irqs are. Depth variable
in irq_desc is actually disable-depth, for nested irq_disable() calls.
It can have value from 0 to N. 0 is when interrupt is enabled and N shows
the irq_disable depth.
Lets say, if disable_irq is called 4 times, driver need to call
enable_irq 4 times to actually enable the interrupt back. But if
enable_irq is called 4 times, only one disable_irq needed to actually
disable the interrupt.
Use request/free_irq instead of disable/enable_irq.
Change-Id: Ie7fe866b403da9bf363f741b1693361b8e2f6a3d
Signed-off-by: Arun KS <arunks@codeaurora.org>
Add ability to read filter properties from debugfs and add ability to
enable and disable spdm processing.
Change-Id: I1b94c5c44180df2a39097657ecc6606562736d1a
Signed-off-by: Dan Sneddon <dsneddon@codeaurora.org>
Signed-off-by: Alok Chauhan <alokc@codeaurora.org>
The files in the debugfs directory for spdm were
set incorrectly using hex instead of octal numbers.
This patch correctly uses octal codes and removes the
execute bit.
CRs-Fixed: 799304
Change-Id: I3bab83073d1ee1f426c402a05556c9ae97ca089d
Signed-off-by: Dan Sneddon <dsneddon@codeaurora.org>
Currently SPDM driver uses hvc calls to support
the dcvs algorithm logic. On some targets this
dcvs algorithm support is present in TZ and which
is accessed via separate calls. Add SCM call to
support TZ based dcvs algorithm.
Change-Id: I197f0f13b4107047151e10e19e4849008607f3e8
Signed-off-by: Dan Sneddon <dsneddon@codeaurora.org>
Signed-off-by: Alok Chauhan <alokc@codeaurora.org>
Correct some of memory free usages check to avoid corruption
later point of time.
Change-Id: Iaeff7bf413157ff324d1678f630de54c22d792a0
Signed-off-by: Alok Chauhan <alokc@codeaurora.org>
Use IPC logging to log calls made to hypervisor for the SPDM driver.
The continuous logging is useful when debugging stability issues and also
to profile code execution.
Change-Id: Ib32fcc998319f4477dc8c1df383ed89b4a9e9214
Signed-off-by: Girish Mahadevan <girishm@codeaurora.org>
put_online_cpus() is not called if an error occurs during cpufreq
notification registration. Fix the error path by calling it properly.
Change-Id: Ia2e6b2debb2db4b39f8fcfcd1ee873538b44d405
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Print out all three levels of requests for debugging.
Change-Id: I1e0d12c46386c1aed6b0bfe878449d070fd1adcc
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Using an array to report monitor stats instead of hard coded variable
names would allow for cleaner implementations of some cache hwmon
device drivers.
Change-Id: I787bdc12f10a0c8ff3c4195ce229a2987acdfce7
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
The cache monitoring devices might have more than one IRQ to handle
or might have notifications from other drivers instead of using actual
IRQs. So, refactor the governor to move the IRQ handling to the cache
monitoring device specific drivers and just provide an API that can be
used to request a re-evaluation.
The device specific driver can call this API to request an immediate
re-evaluation whenever the cache request has exceeded the previously
set limit instead of waiting for the periodic update.
Change-Id: Ib2e9f53f95749d659f440739a1b074b5a0d94fd8
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
The refactor allows the governor to support multiple devfreq devices.
This is done by having different HW monitor instances register their
capability to monitor different devfreq devices and then picking the
right HW monitor based on which devfreq device is using this governor.
Change-Id: I72c0542ce97f3965e422df521e0ce86cad218d93
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
The ARM PMU supports monitoring bus access from each CPU. It also has the
ability to raise an IRQ when the counters overflow. This allows for it to
be used with the bw_hwmon governor to scale the CPU BW requests by
monitoring on the actual bus access traffic.
Change-Id: I0594a6acb846acdc11a18744033636951f22e387
Signed-off-by: Jacob Stevens <jstevens@codeaurora.org>
It is not possible to ensure the synchronization of REMOVE_POLICY
notifications with CPU hotplug lock; {get,put}_online_cpus ensures
that hotplug cannot happen, but it is still possible to receive
REMOVE_POLICY notifications asynchronously while checking for online
CPUs within a {get,put}_online_cpus critical section.
Account for this by detecting that we haven't yet setup a local state
when the REMOVE_POLICY notification comes in.
Change-Id: I3cb750f3984ebe078154734444660675e8d8b5bc
Signed-off-by: Vikram Mulukutla <markivx@codeaurora.org>
When "simple_scaling" flag is enabled for on demand governor then clocks
should be scaled up when the load is more than up threshold and should
be scaled down when load is less than the up threshold minus down
differential threshold. But currently governor is only scaling down
when load is less than the down differential threshold which is definitely
not intentional. This change fixes the above bug.
Change-Id: If2a234155c12989dc0df397cd84eef4a759ecdfc
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
[junjiew@codeaurora.org: resolved trivial conflicts]
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
If the spdm governor fails to connect to the hypervisor correctly
the device it is supposed to be governing will be freed but still
stored in the local list. This patch removes the device from the
list to prevent accessing an invalid pointer if the hypervisor returns
an error.
Change-Id: I536c198b5a25adbd3044ffd37d9951c197b1dfd9
Signed-off-by: Dan Sneddon <dsneddon@codeaurora.org>
The enable and disable calls in the spdm governor are missing
the hypervisor call. This patch adds the hvc call.
Change-Id: Ic8f4feeb9bc0b7066b6620553725aa636c017c03
Signed-off-by: Dan Sneddon <dsneddon@codeaurora.org>
Add debugfs support for the devfreq spdm driver.
The parameters used for determing the SPDM port
threshold value can be updated via debugfs and sent
to the hypervisor to support fine tuning SPDM
performance.
Change-Id: I6f85deacd7d463d90f512f5de18b7e2140c9f492
Signed-off-by: Dan Sneddon <dsneddon@codeaurora.org>
[junjiew@codeaurora.org: resolved trivial conflicts]
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Some generic devfreq governors might not provide AB values since that's a
devbw device specific attribute. In such cases, we might want to make an
average bandwidth (AB) vote that's a percentage of the IB vote to make
sure device BW requirement are not grossly misrepresented. This patch adds
support for that.
Change-Id: I76fbb8d688742058980f0d7568f2e7140023917e
Signed-off-by: Taniya Das <tdas@codeaurora.org>
The devfreq_spdm driver implements support for bandwidth voting
based on input from the SPDM device on MSM SoC's. The SPDM
governor registers for the SPDM interrupt and then calls
the hypervisor to determine the correct bandwidth to vote for.
The devfreq framework poll timer is used to perdiocially
ask the hypervisor for the new bandwidth to request from
the MSM bus scaling code.
Change-Id: I851457e40d49b5929f01c510249d3e6bb4ff2f1d
Signed-off-by: Dan Sneddon <dsneddon@codeaurora.org>
[junjiew@codeaurora.org: resolved trivial conflicts]
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Some devfreq devices might need their frequency to be set only for a short
time after the CPU frequency changes.
For such devices, add a "timeout" tunable that determines for how many
milliseconds the governor sets the device frequency based on the CPU
frequency. After "timeout" milliseconds from the CPU frequency change, the
governor will set the device frequency back to its minimum value.
Change-Id: I17fc972c0b03fab781864ce735013710c2df4647
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
