The existing bw_hwmon governor samples the bandwidth every polling_interval
milliseconds and makes decisions. Polling interval of 50ms or even 10ms
gives a very low resolution picture of the DDR/bus traffic. Due to the
lower resolution picture, the existing governor algorithm has to be biased
aggressively towards performance to avoid any performance degradation
compared to using a static mapping between bus master (CPU, GPU, etc)
frequency to DDR/bus BW votes.
While the existing governor uses IRQ to get early notification of traffic
increase, even a 4x early notification for a 50ms polling interval still
takes 12.5ms. This kind of reaction time is still too slow for some bus
masters like CPU.
To take care of these limitations, rewrite the governor algorithm to take
multiple short samples of BW within a decision window (polling interval)
and use that higher resolution picture to make much better and faster
decisions. Doing so allows the governor to have the following features:
- Very low reaction time
- Over voting to stay ahead of increasing traffic
- Historic peak tracking to limit over voting
- Being power aware when doing over voting
- Pattern detection and intelligent hysteresis
- Detection low traffic modes and being less aggressive about BW votes
Change-Id: I69886b7fbeea0b64d10b5a1fb23fcb5f3918f0ce
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Accidentally used a , instead of a ; to end statements. This change fixes
that. Should not have any functional impact.
Change-Id: I36ea6c49a8a8973f1856cac6709f01432e6d60c7
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
A device register is read and printed using pr_debug. Register reads can be
slow. So, refactor the code such that the register read is also compiled
out if the pr_debug is compiled out.
Change-Id: I4de5ab1f2133e19d6ce0b3cb4547b815526c7abd
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
The counter needs to be cleared and acknowledged before clearing the IRQ
bits. Otherwise, the HW could set the IRQ bit again if the old counter
value was higher than the threshold. So, add a memory barrier after
clearing the counter.
Change-Id: I35f9f7905b05b8a185eb94d04d9c0a8ccfc2db51
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Add trace points to log the measurements and the final decision made by the
bw_hwmon governor.
Change-Id: I834bec340f81f02fcccbd6d7568a301dc74dded5
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Some cache_hwmon devices can have interrupts firing at any time. The
interrupt handler would stop devfreq monitor, update its vote and
restart the monitor again. This introduces a race if
devfreq_supend/resume() or devfreq_interval_update() is called at
the same time. Since devfreq_monitor_start() re-initializes the work,
it could cause corruption while the work is being used elsewhere.
Protect governor monitor start/stops with a new lock.
Change-Id: I143aaaea86494b4c617df46e2c521a19b43861d5
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
On some systems, scm calls for cci registers are not supported.
Use normal writel/readl in those cases. Add device tree flag to
distinguish them.
Change-Id: Icfb609d43f888856786c1881b2ee34ffd501e37a
Signed-off-by: Arun KS <arunks@codeaurora.org>
CCI400 on MSM has additional PMU counters that can be used to monitor
cache requests. MSM CCI hardware monitor device configures these
registers to monitor cache and inform governor. It can also set an
IRQ when count exceeds a programmable limit.
Change-Id: I1d80f57749b91c3972e60e54c75226c4d49d2ec6
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
[junjiew@codeaurora.org: Dropped change in arch/arm64/Kconfig.
Configuration should be selected directly in defconfig.]
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Interrupt storm happens when bwmon is enabled for GPU. This is mainly
due to constant low traffic observed with GPU while doing memory
read/write. So as the data rates read from counters are low and so the
threshold set for triggering the interrupt also set as low, which in
turn causes huge number of interrupts. Avoid this by setting a minimum
floor for the irq threshold.
Change-Id: I190fad5108bc24afcb67bec5809485380ee3662e
Signed-off-by: Hanumath Prasad <hpprasad@codeaurora.org>
Send all the frequency performance levels in KHz units to TZ driver and
convert all the bandwidth recommendations to Bytes/s from Kbytes/s to
accomodate change in the syscall interface calls to TZ SPDM driver.
Change-Id: I209ea0583fdd43f78f51793d7818ea5afd5959c7
Signed-off-by: Dilip Kota <dkota@codeaurora.org>
Signed-off-by: Girish Mahadevan <girishm@codeaurora.org>
The TZ SPDM driver mandates the number of arguments be constant for a
given command id. This patch allows the spdm driver to communicate with
the TZ driver correctly.
Change-Id: Id3cfd2490039c4abde6d5381859d8facf545e85e
Signed-off-by: Dan Sneddon <dsneddon@codeaurora.org>
Signed-off-by: Girish Mahadevan <girishm@codeaurora.org>
A devfreq governor store in parallel with a cpu freq update can cause
deadlock as shown below.
Assume current devfreq governor is cpufreq, and user tries to change
to some other governor.
Write to sysfs store_governor | cpufreq driver updating cpu freq
------------------------------- | -----------------------------------
echo bw_hwmon > governor |
| takes rcu_read_lock and calls all
| cpufreq transition callbacks for
| PRECHANGE or POSTCHANGE
|
GOV_STOP on governor_cpufreq. |
unregister_cpufreq() accquires |
state_lock mutex. |
| try to accquire same state_lock in
| cpufreq_trans_notifier(). Blocked.
unregister from cpufreq |
transition notifier and wait for|
all rcu_readers to finish. |
Deadlock
A similar deadlock can happen with governor change and policy notifier
callbacks.
The state_lock currently protects multiple unrelated critical
sections: registering/unregistering of cpufreq notifiers, read/writing
the device list, and tracking the cpu states and updating device
frequencies. There is no need for register/unregister of the cpufreq
notifiers to be mutually excluded against the other critical sections
using the same lock.
Split state_lock into two locks to protect the register/unregister of
cpufreq notifiers from the rest of the critical sections.
Change-Id: Id06d326748a5cb0c84c4787da5d0910f44eb5c3c
Signed-off-by: Pan Fang <fangpan@codeaurora.org>
Signed-off-by: Arun KS <arunks@codeaurora.org>
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Suggested-by: Saravana Kannan <skannan@codeaurora.org>
Use free_irq to free the interrupt handler for a shared interrupt.
Enable_irqs are not refcounted, whereas disable_irqs are. Depth variable
in irq_desc is actually disable-depth, for nested irq_disable() calls.
It can have value from 0 to N. 0 is when interrupt is enabled and N shows
the irq_disable depth.
Lets say, if disable_irq is called 4 times, driver need to call
enable_irq 4 times to actually enable the interrupt back. But if
enable_irq is called 4 times, only one disable_irq needed to actually
disable the interrupt.
Use request/free_irq instead of disable/enable_irq.
Change-Id: Ie7fe866b403da9bf363f741b1693361b8e2f6a3d
Signed-off-by: Arun KS <arunks@codeaurora.org>
Add ability to read filter properties from debugfs and add ability to
enable and disable spdm processing.
Change-Id: I1b94c5c44180df2a39097657ecc6606562736d1a
Signed-off-by: Dan Sneddon <dsneddon@codeaurora.org>
Signed-off-by: Alok Chauhan <alokc@codeaurora.org>
The files in the debugfs directory for spdm were
set incorrectly using hex instead of octal numbers.
This patch correctly uses octal codes and removes the
execute bit.
CRs-Fixed: 799304
Change-Id: I3bab83073d1ee1f426c402a05556c9ae97ca089d
Signed-off-by: Dan Sneddon <dsneddon@codeaurora.org>
Currently SPDM driver uses hvc calls to support
the dcvs algorithm logic. On some targets this
dcvs algorithm support is present in TZ and which
is accessed via separate calls. Add SCM call to
support TZ based dcvs algorithm.
Change-Id: I197f0f13b4107047151e10e19e4849008607f3e8
Signed-off-by: Dan Sneddon <dsneddon@codeaurora.org>
Signed-off-by: Alok Chauhan <alokc@codeaurora.org>
Correct some of memory free usages check to avoid corruption
later point of time.
Change-Id: Iaeff7bf413157ff324d1678f630de54c22d792a0
Signed-off-by: Alok Chauhan <alokc@codeaurora.org>
Use IPC logging to log calls made to hypervisor for the SPDM driver.
The continuous logging is useful when debugging stability issues and also
to profile code execution.
Change-Id: Ib32fcc998319f4477dc8c1df383ed89b4a9e9214
Signed-off-by: Girish Mahadevan <girishm@codeaurora.org>
put_online_cpus() is not called if an error occurs during cpufreq
notification registration. Fix the error path by calling it properly.
Change-Id: Ia2e6b2debb2db4b39f8fcfcd1ee873538b44d405
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Print out all three levels of requests for debugging.
Change-Id: I1e0d12c46386c1aed6b0bfe878449d070fd1adcc
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Using an array to report monitor stats instead of hard coded variable
names would allow for cleaner implementations of some cache hwmon
device drivers.
Change-Id: I787bdc12f10a0c8ff3c4195ce229a2987acdfce7
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
The cache monitoring devices might have more than one IRQ to handle
or might have notifications from other drivers instead of using actual
IRQs. So, refactor the governor to move the IRQ handling to the cache
monitoring device specific drivers and just provide an API that can be
used to request a re-evaluation.
The device specific driver can call this API to request an immediate
re-evaluation whenever the cache request has exceeded the previously
set limit instead of waiting for the periodic update.
Change-Id: Ib2e9f53f95749d659f440739a1b074b5a0d94fd8
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
The refactor allows the governor to support multiple devfreq devices.
This is done by having different HW monitor instances register their
capability to monitor different devfreq devices and then picking the
right HW monitor based on which devfreq device is using this governor.
Change-Id: I72c0542ce97f3965e422df521e0ce86cad218d93
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
The ARM PMU supports monitoring bus access from each CPU. It also has the
ability to raise an IRQ when the counters overflow. This allows for it to
be used with the bw_hwmon governor to scale the CPU BW requests by
monitoring on the actual bus access traffic.
Change-Id: I0594a6acb846acdc11a18744033636951f22e387
Signed-off-by: Jacob Stevens <jstevens@codeaurora.org>
It is not possible to ensure the synchronization of REMOVE_POLICY
notifications with CPU hotplug lock; {get,put}_online_cpus ensures
that hotplug cannot happen, but it is still possible to receive
REMOVE_POLICY notifications asynchronously while checking for online
CPUs within a {get,put}_online_cpus critical section.
Account for this by detecting that we haven't yet setup a local state
when the REMOVE_POLICY notification comes in.
Change-Id: I3cb750f3984ebe078154734444660675e8d8b5bc
Signed-off-by: Vikram Mulukutla <markivx@codeaurora.org>
When "simple_scaling" flag is enabled for on demand governor then clocks
should be scaled up when the load is more than up threshold and should
be scaled down when load is less than the up threshold minus down
differential threshold. But currently governor is only scaling down
when load is less than the down differential threshold which is definitely
not intentional. This change fixes the above bug.
Change-Id: If2a234155c12989dc0df397cd84eef4a759ecdfc
Signed-off-by: Subhash Jadavani <subhashj@codeaurora.org>
[junjiew@codeaurora.org: resolved trivial conflicts]
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
If the spdm governor fails to connect to the hypervisor correctly
the device it is supposed to be governing will be freed but still
stored in the local list. This patch removes the device from the
list to prevent accessing an invalid pointer if the hypervisor returns
an error.
Change-Id: I536c198b5a25adbd3044ffd37d9951c197b1dfd9
Signed-off-by: Dan Sneddon <dsneddon@codeaurora.org>
The enable and disable calls in the spdm governor are missing
the hypervisor call. This patch adds the hvc call.
Change-Id: Ic8f4feeb9bc0b7066b6620553725aa636c017c03
Signed-off-by: Dan Sneddon <dsneddon@codeaurora.org>
Add debugfs support for the devfreq spdm driver.
The parameters used for determing the SPDM port
threshold value can be updated via debugfs and sent
to the hypervisor to support fine tuning SPDM
performance.
Change-Id: I6f85deacd7d463d90f512f5de18b7e2140c9f492
Signed-off-by: Dan Sneddon <dsneddon@codeaurora.org>
[junjiew@codeaurora.org: resolved trivial conflicts]
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Some generic devfreq governors might not provide AB values since that's a
devbw device specific attribute. In such cases, we might want to make an
average bandwidth (AB) vote that's a percentage of the IB vote to make
sure device BW requirement are not grossly misrepresented. This patch adds
support for that.
Change-Id: I76fbb8d688742058980f0d7568f2e7140023917e
Signed-off-by: Taniya Das <tdas@codeaurora.org>
The devfreq_spdm driver implements support for bandwidth voting
based on input from the SPDM device on MSM SoC's. The SPDM
governor registers for the SPDM interrupt and then calls
the hypervisor to determine the correct bandwidth to vote for.
The devfreq framework poll timer is used to perdiocially
ask the hypervisor for the new bandwidth to request from
the MSM bus scaling code.
Change-Id: I851457e40d49b5929f01c510249d3e6bb4ff2f1d
Signed-off-by: Dan Sneddon <dsneddon@codeaurora.org>
[junjiew@codeaurora.org: resolved trivial conflicts]
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Some devfreq devices might need their frequency to be set only for a short
time after the CPU frequency changes.
For such devices, add a "timeout" tunable that determines for how many
milliseconds the governor sets the device frequency based on the CPU
frequency. After "timeout" milliseconds from the CPU frequency change, the
governor will set the device frequency back to its minimum value.
Change-Id: I17fc972c0b03fab781864ce735013710c2df4647
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
The version 2 of the BIMC BWMON HW doesn't reset the counter to 0 when it
hits the threshold. It also has support for an overflow status register.
Change-Id: I9f18d2153a2e5e762ec9950f26e0e7601468a80a
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Absence of traffic is guaranteed when the device sitting behind a devbw
device is suspended. In such cases, it is a waste of power to make non-zero
bandwidth votes or to scale the devbw device. So, provide APIs to
suspend/resume the devbw device as needed.
Change-Id: Id58072aec7a9710eb917f248d9b9bd08d3a1ec6a
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
- Add more debug logs
- Change the format out the count logs to use hex instead of decimal to be
consistent with the rest of the logs
- Fix the type of the count variable from signed to unsigned to do the
above
Change-Id: I02a2968a3f10ce20ca00618e7aeeac9b9cd52bd3
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
The clearing of the BIMC BWMON IRQ needs clearing bits in two separate
registers. One is a global register and the other is a port specific
register.
The bit in the port specific register needs to be cleared first before
clearing the bit in the global register. Otherwise, the bit in the global
register gets set again before the port specific bit is cleared. Since
these register are in different address regions, we also need memory
barriers around writes to the global register.
Also, clear the counter value before clearing the interrupt status just to
be safe.
Change-Id: Iee8d2caf9bf7d639c65ed19c979036bd5e203bfd
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
The HW workaround margin being too high reduces the effectiveness of the
interrupt. Try using a margin only when the measured bandwidth is too small
and risks the counter wrapping around multiple times before it's read.
Change-Id: Ic1e88ad360b2348dfb9ad314c42c1b0218010c1d
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Some devfreq devices using this governor might need suspend/resume support.
When suspended, those devices won't need any bandwidth votes and there is
no point in monitoring their bandwidth either.
Therefore, upon suspend, vote for zero bandwidth and stop the HW monitor.
Upon resume, vote for the previous bandwidth and start the HW monitor.
Change-Id: I318449995d714959f0ebfe91961bc23fa8edbd04
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
This is a snapshot of the MSM BIMC bwmon driver as of msm-3.10
commit:
acdce027751d5a7488b283f0ce3111f873a5816d (Merge "defconfig: arm64:
Enable ONESHOT_SYNC for msm8994")
Signed-off-by: Kumar Gala <galak@codeaurora.org>
This is a snapshot of the Bandwidth driver as of msm-3.10 commit:
acdce027751d5a7488b283f0ce3111f873a5816d (Merge "defconfig: arm64:
Enable ONESHOT_SYNC for msm8994")
Signed-off-by: Kumar Gala <galak@codeaurora.org>
[junjiew@codeaurora.org: resolved conflicts]
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Change-Id: I30d48abdfe19a421b4d05003c56c47423c6d0456
This is a snapshot of the Generic bandwidth hw monitor driver as of
msm-3.10 commit:
acdce027751d5a7488b283f0ce3111f873a5816d (Merge "defconfig: arm64:
Enable ONESHOT_SYNC for msm8994")
Signed-off-by: Kumar Gala <galak@codeaurora.org>
Commit a0dd7b7 excludes cpufreq header from pm_opp.h. Since the
cpufreq governor uses the cpufreq APIs, include cpufreq header
directly for this governor.
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
This is a snapshot of the Generic cpufreq governor driver as of msm-3.10
commit:
acdce027751d5a7488b283f0ce3111f873a5816d (Merge "defconfig: arm64:
Enable ONESHOT_SYNC for msm8994")
Signed-off-by: Kumar Gala <galak@codeaurora.org>
This is a snapshot of the Krait L2 cache HW monitor driver as of msm-3.10
commit:
acdce027751d5a7488b283f0ce3111f873a5816d (Merge "defconfig: arm64:
Enable ONESHOT_SYNC for msm8994")
Signed-off-by: Kumar Gala <galak@codeaurora.org>
This is a snapshot of the HW monitor governor driver as of msm-3.10
commit:
acdce027751d5a7488b283f0ce3111f873a5816d (Merge "defconfig: arm64:
Enable ONESHOT_SYNC for msm8994")
Signed-off-by: Kumar Gala <galak@codeaurora.org>
This is a snapshot of the simple devfreq device driver as of msm-3.10
commit:
acdce027751d5a7488b283f0ce3111f873a5816d (Merge "defconfig: arm64:
Enable ONESHOT_SYNC for msm8994")
Signed-off-by: Kumar Gala <galak@codeaurora.org>
[junjiew@codeaurora.org: resolved conflicts]
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Change-Id: I37f1781d9192dd0ad2797ea52f9bd3a5ea5847b0
When unregistering devfreq device (devfreq_remove_device()),
there is an additional call to put_device,
after device_unregister().This causes data aborts in case
of access to a kobj in put_device(), that was already freed
by preceding device_unregister()
CRs-Fixed: 841819
Change-Id: I98bd9e4cc9ecfbc48a0bfe72fc47e362a6697741
Signed-off-by: Hanumant Singh <hanumant@codeaurora.org>
