ARCH_MSM has been replaced by ARCH_QCOM. Fix Kconfig for various
devfreq governors and devices.
Change-Id: Ifa85494785cea7eb03be6c02d5664ee6bb2110c6
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
The return value of kstrtouint is erroneously checked while setting
the tunables for mem_latency governor due to which the tunables
cannot be changed from their default values.
This change rectifies that behavior.
Change-Id: Ief7dda4638ede2c97b26229f1188a1559b238920
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Currently stop_hwmon only releases the perf events for each CPU
in the memlat device group. This leaves out the perf_event_attrs
that are not needed anymore leading to memory leaks.
This patch frees perf_event_attrs right after the perf events are
created because they are copied into the perf_event structure
when it is created for a particular event and hence do not need
to exist beyond that point.
Change-Id: If0871739ef477dfbcbbfc6ab2a631ce25deaf71c
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
If any of the CPUs monitored by a memlat device is offline while
trying to register for perf events, the registration fails
returning an error code to the governor's start_monitor function
which then fails the setting of governor and leaves the system in
a bad state.
To avoid this scenario, this patch allows for initialization errors
while registering for perf events and remembers the failing offline
CPUs. Perf event registration is tried for the failing CPUs when they
come back online. Once all the CPUs have registered for perf events
hotplug notifiers callbacks are unregistered.
Change-Id: I36c33d9dc627f60ab9d7db41913661927938e850
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
clk_set_rate is failing for some of the frequencies due to higher
precision introduced in clock driver. Hence call the clk_round_rate
again in dev_target to make sure we send the correct frequency to
clk_set_rate.
Change-Id: I3e40690bf794d99debbcecdcaf05aaac3363caa8
Signed-off-by: Hanumath Prasad <hpprasad@codeaurora.org>
The of_node_put(of_child) call was put in to balance the of_node_get()
that's done inside for_each_child_of_node(). But that's not the right thing
to do because for_each_child_of_node() already takes care of doing
of_node_put(). So, delete the unnecessary of_node_put(of_child) call. We
still need the of_node_put(of_par) though.
Change-Id: Iff7744839cfcf01d5a472449f74859e9030b7e16
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Currently arm-memlat-mon driver uses cpu_coregroup_mask to get
the sibling CPUs for the first CPU specified during the probe.
However if any of the CPUs in a cluster isn't hotplugged in
even once before the probe runs then the call to cpu_coregroup_mask
might not give the updated list of all the sibling CPUs for the
specified CPU.
With this change the initialization of the cpumask for the memlat
device is obtained from the list of CPU phandles specified in the
dtsi file.
Change-Id: Ide97d60d9eecbbe1d33deda72a13951059822896
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
[junjiew@codeaurora.org: dropped changes in dtsi files.]
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Use performance counters to detect the memory latency sensitivity
of CPU workloads and vote for higher DDR frequency if required.
Change-Id: Ie77a3523bc5713fc0315bd0abc3913f485a96e0e
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Suggested-by: Saravana Kannan <skannan@codeaurora.org>
[junjiew@codeaurora.org: dropped changes in arch/arm64/Kconfig]
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Newer versions of bimc-bwmon counters have the capability to fake
higher byte count than what's actually transferred between a bus
master and DDR if the bus master is being throttled by QoS hardware
logic. Add support to set the throttle adjust field that comes with
this newer version of bimc-bwmon.
Change-Id: I33376c825fb11ab2e378f828b1d2ae46dd582836
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
[junjiew@codeaurora.org: dropped changes in dtsi.]
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
When doing over-voting due to an UP_WAKE event, the governor
pre-initialized the max_mbps value for the next decision window. This was
done to prevent a dropping the vote below the original measurement in case
a DOWN_WAKE event comes before the decision window is complete. This has
the undesirable side-effect of keeping the votes high for two decision
windows following an UP_WAKE event.
Also, DOWN_WAKE event is currently disabled for any decision windows that
didn't do over-voting because doing so had significant performance impacts.
However, with historic max tracking, pattern detection and hysteresis, and
mbps zones awareness that were added later on, the above concerns aren't
really valid anymore. So, simply delete the pre-initialization of the
max_mbps so that one decision window isn't unnecessarily voting higher than
necessary. As part of the change, also fix down_cnt logic to allow it to
work for all decision windows if enabled.
Change-Id: I600e3783da617ed4efef5b05156016d88c301cea
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Currently the lower limit for setting up_scale is 100. Change it
to 0 so that over-voting for spikes in measured bandwidth can be
disabled by setting up_scale to 0.
Change-Id: I1e5e3d51c625163b38074fb1eda26763455ea6c5
Signed-off-by: Rohit Gupta <rohgup@codeaurora.org>
Using debug logs for the short samples would result in excessive logging
and also have an impact on CPU power and performance. The tracepoint that's
already there is a much better alternative. So, delete the debug logs.
Change-Id: Idd1d634f510381d14637f3b84588a118572f3f16
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
The IRQ registration on the resume path was not updated properly when the
high sampling rate algorithm was implemented. Update it so that the IRQ
registration is done correctly.
Change-Id: I17a016dd9c0b50c7b415beda21368cb4586d203c
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Replace measured mrps and frequency vote debug prints with trace
events.
Change-Id: I78370b068e3819a57635cbabaf5cdd053ebabce4
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Change the sequence of registering and freeing the interrupt
handler in suspend/resume. Freeirq needs a guarantee that the
IRQ can't come anymore before we call it. So, we disable the IRQ
before calling freeirq.And register the handler before enabling
the irq to avoid the interrupt getting unhandled.
Change-Id: I3945202d049e16f64a16e456f914f7602b763c89
Signed-off-by: Hanumath Prasad <hpprasad@codeaurora.org>
Due to the way the short samples are triggered, when the traffic is very
low, we could have a decision window where no sample is taken. Fix this by
forcing a sample at the end of every decision window as long as that is the
first sample for that decision window or the sample window won't be smaller
than the sample_ms tunable.
Change-Id: Ia33b59fbff9ff4058c06e24208d1e037d6e9eda9
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
With the rewrite of the algorithm, some tunables are not used. Remove those
tunables. Other tunables don't make a lot of sense to be turned on by
default. Change their default state to be disabled.
Change-Id: I228d275a21765986a7117b335a669deebf590f29
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
When the decision window is terminated early by an IRQ and the frequency
voted by the governor is the max_freq allowed for that device, the AB vote
can be lower than the measured value. This really shouldn't affect power or
performance since the device frequency is already maxed out, but fix it
anyway so that the logs don't look suspicious.
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Change-Id: I1788cbb1af026f1c54f9acc4806cb29cd6069e04
M4M counters are only 28-bit instead of 32-bit. Fix limit calculation
to use the right max value.
Change-Id: I91078842b72da80f6b6755bf8d808ff4b4142f10
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Enable cycle counter along with other counters when m4m_hwmon is
started.
Change-Id: Idd86b99c21e21f64bb91db9b1e64597fbfb2306a
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Some targets have a single irq line which is shared among all
the cci hwmon counters. Enhance the driver to support shared interrupt
handling.
Change-Id: I5fdaecfaa14fa47e8f393fe51c538e5000e6ad5b
Signed-off-by: Arun KS <arunks@codeaurora.org>
Signed-off-by: Hanumath Prasad <hpprasad@codeaurora.org>
For certain implementation, device clock needs to be prepared before
rate voting taking effect. Add support for preparing device clock
during initialization.
Change-Id: Ib22e83952187118342ff2546d4c79d3970a288f9
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
The existing bw_hwmon governor samples the bandwidth every polling_interval
milliseconds and makes decisions. Polling interval of 50ms or even 10ms
gives a very low resolution picture of the DDR/bus traffic. Due to the
lower resolution picture, the existing governor algorithm has to be biased
aggressively towards performance to avoid any performance degradation
compared to using a static mapping between bus master (CPU, GPU, etc)
frequency to DDR/bus BW votes.
While the existing governor uses IRQ to get early notification of traffic
increase, even a 4x early notification for a 50ms polling interval still
takes 12.5ms. This kind of reaction time is still too slow for some bus
masters like CPU.
To take care of these limitations, rewrite the governor algorithm to take
multiple short samples of BW within a decision window (polling interval)
and use that higher resolution picture to make much better and faster
decisions. Doing so allows the governor to have the following features:
- Very low reaction time
- Over voting to stay ahead of increasing traffic
- Historic peak tracking to limit over voting
- Being power aware when doing over voting
- Pattern detection and intelligent hysteresis
- Detection low traffic modes and being less aggressive about BW votes
Change-Id: I69886b7fbeea0b64d10b5a1fb23fcb5f3918f0ce
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Accidentally used a , instead of a ; to end statements. This change fixes
that. Should not have any functional impact.
Change-Id: I36ea6c49a8a8973f1856cac6709f01432e6d60c7
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
A device register is read and printed using pr_debug. Register reads can be
slow. So, refactor the code such that the register read is also compiled
out if the pr_debug is compiled out.
Change-Id: I4de5ab1f2133e19d6ce0b3cb4547b815526c7abd
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
The counter needs to be cleared and acknowledged before clearing the IRQ
bits. Otherwise, the HW could set the IRQ bit again if the old counter
value was higher than the threshold. So, add a memory barrier after
clearing the counter.
Change-Id: I35f9f7905b05b8a185eb94d04d9c0a8ccfc2db51
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
Add trace points to log the measurements and the final decision made by the
bw_hwmon governor.
Change-Id: I834bec340f81f02fcccbd6d7568a301dc74dded5
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
The bw_hwmon_meas trace event is to log measurement details and the
bw_hwmon_update trace event is to log the final decision.
Change-Id: I839ace50b1f1686227bcbf7d38a75f89092d26b1
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>
[junjiew@codeaurora.org: resolved trivial conflicts]
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Some cache_hwmon devices can have interrupts firing at any time. The
interrupt handler would stop devfreq monitor, update its vote and
restart the monitor again. This introduces a race if
devfreq_supend/resume() or devfreq_interval_update() is called at
the same time. Since devfreq_monitor_start() re-initializes the work,
it could cause corruption while the work is being used elsewhere.
Protect governor monitor start/stops with a new lock.
Change-Id: I143aaaea86494b4c617df46e2c521a19b43861d5
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
On some systems, scm calls for cci registers are not supported.
Use normal writel/readl in those cases. Add device tree flag to
distinguish them.
Change-Id: Icfb609d43f888856786c1881b2ee34ffd501e37a
Signed-off-by: Arun KS <arunks@codeaurora.org>
CCI400 on MSM has additional PMU counters that can be used to monitor
cache requests. MSM CCI hardware monitor device configures these
registers to monitor cache and inform governor. It can also set an
IRQ when count exceeds a programmable limit.
Change-Id: I1d80f57749b91c3972e60e54c75226c4d49d2ec6
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
[junjiew@codeaurora.org: Dropped change in arch/arm64/Kconfig.
Configuration should be selected directly in defconfig.]
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Interrupt storm happens when bwmon is enabled for GPU. This is mainly
due to constant low traffic observed with GPU while doing memory
read/write. So as the data rates read from counters are low and so the
threshold set for triggering the interrupt also set as low, which in
turn causes huge number of interrupts. Avoid this by setting a minimum
floor for the irq threshold.
Change-Id: I190fad5108bc24afcb67bec5809485380ee3662e
Signed-off-by: Hanumath Prasad <hpprasad@codeaurora.org>
Send all the frequency performance levels in KHz units to TZ driver and
convert all the bandwidth recommendations to Bytes/s from Kbytes/s to
accomodate change in the syscall interface calls to TZ SPDM driver.
Change-Id: I209ea0583fdd43f78f51793d7818ea5afd5959c7
Signed-off-by: Dilip Kota <dkota@codeaurora.org>
Signed-off-by: Girish Mahadevan <girishm@codeaurora.org>
The TZ SPDM driver mandates the number of arguments be constant for a
given command id. This patch allows the spdm driver to communicate with
the TZ driver correctly.
Change-Id: Id3cfd2490039c4abde6d5381859d8facf545e85e
Signed-off-by: Dan Sneddon <dsneddon@codeaurora.org>
Signed-off-by: Girish Mahadevan <girishm@codeaurora.org>
A devfreq governor store in parallel with a cpu freq update can cause
deadlock as shown below.
Assume current devfreq governor is cpufreq, and user tries to change
to some other governor.
Write to sysfs store_governor | cpufreq driver updating cpu freq
------------------------------- | -----------------------------------
echo bw_hwmon > governor |
| takes rcu_read_lock and calls all
| cpufreq transition callbacks for
| PRECHANGE or POSTCHANGE
|
GOV_STOP on governor_cpufreq. |
unregister_cpufreq() accquires |
state_lock mutex. |
| try to accquire same state_lock in
| cpufreq_trans_notifier(). Blocked.
unregister from cpufreq |
transition notifier and wait for|
all rcu_readers to finish. |
Deadlock
A similar deadlock can happen with governor change and policy notifier
callbacks.
The state_lock currently protects multiple unrelated critical
sections: registering/unregistering of cpufreq notifiers, read/writing
the device list, and tracking the cpu states and updating device
frequencies. There is no need for register/unregister of the cpufreq
notifiers to be mutually excluded against the other critical sections
using the same lock.
Split state_lock into two locks to protect the register/unregister of
cpufreq notifiers from the rest of the critical sections.
Change-Id: Id06d326748a5cb0c84c4787da5d0910f44eb5c3c
Signed-off-by: Pan Fang <fangpan@codeaurora.org>
Signed-off-by: Arun KS <arunks@codeaurora.org>
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Suggested-by: Saravana Kannan <skannan@codeaurora.org>
Use free_irq to free the interrupt handler for a shared interrupt.
Enable_irqs are not refcounted, whereas disable_irqs are. Depth variable
in irq_desc is actually disable-depth, for nested irq_disable() calls.
It can have value from 0 to N. 0 is when interrupt is enabled and N shows
the irq_disable depth.
Lets say, if disable_irq is called 4 times, driver need to call
enable_irq 4 times to actually enable the interrupt back. But if
enable_irq is called 4 times, only one disable_irq needed to actually
disable the interrupt.
Use request/free_irq instead of disable/enable_irq.
Change-Id: Ie7fe866b403da9bf363f741b1693361b8e2f6a3d
Signed-off-by: Arun KS <arunks@codeaurora.org>
Add ability to read filter properties from debugfs and add ability to
enable and disable spdm processing.
Change-Id: I1b94c5c44180df2a39097657ecc6606562736d1a
Signed-off-by: Dan Sneddon <dsneddon@codeaurora.org>
Signed-off-by: Alok Chauhan <alokc@codeaurora.org>
The files in the debugfs directory for spdm were
set incorrectly using hex instead of octal numbers.
This patch correctly uses octal codes and removes the
execute bit.
CRs-Fixed: 799304
Change-Id: I3bab83073d1ee1f426c402a05556c9ae97ca089d
Signed-off-by: Dan Sneddon <dsneddon@codeaurora.org>
Currently SPDM driver uses hvc calls to support
the dcvs algorithm logic. On some targets this
dcvs algorithm support is present in TZ and which
is accessed via separate calls. Add SCM call to
support TZ based dcvs algorithm.
Change-Id: I197f0f13b4107047151e10e19e4849008607f3e8
Signed-off-by: Dan Sneddon <dsneddon@codeaurora.org>
Signed-off-by: Alok Chauhan <alokc@codeaurora.org>
Correct some of memory free usages check to avoid corruption
later point of time.
Change-Id: Iaeff7bf413157ff324d1678f630de54c22d792a0
Signed-off-by: Alok Chauhan <alokc@codeaurora.org>
Use IPC logging to log calls made to hypervisor for the SPDM driver.
The continuous logging is useful when debugging stability issues and also
to profile code execution.
Change-Id: Ib32fcc998319f4477dc8c1df383ed89b4a9e9214
Signed-off-by: Girish Mahadevan <girishm@codeaurora.org>
put_online_cpus() is not called if an error occurs during cpufreq
notification registration. Fix the error path by calling it properly.
Change-Id: Ia2e6b2debb2db4b39f8fcfcd1ee873538b44d405
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Print out all three levels of requests for debugging.
Change-Id: I1e0d12c46386c1aed6b0bfe878449d070fd1adcc
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
Using an array to report monitor stats instead of hard coded variable
names would allow for cleaner implementations of some cache hwmon
device drivers.
Change-Id: I787bdc12f10a0c8ff3c4195ce229a2987acdfce7
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
The cache monitoring devices might have more than one IRQ to handle
or might have notifications from other drivers instead of using actual
IRQs. So, refactor the governor to move the IRQ handling to the cache
monitoring device specific drivers and just provide an API that can be
used to request a re-evaluation.
The device specific driver can call this API to request an immediate
re-evaluation whenever the cache request has exceeded the previously
set limit instead of waiting for the periodic update.
Change-Id: Ib2e9f53f95749d659f440739a1b074b5a0d94fd8
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
The refactor allows the governor to support multiple devfreq devices.
This is done by having different HW monitor instances register their
capability to monitor different devfreq devices and then picking the
right HW monitor based on which devfreq device is using this governor.
Change-Id: I72c0542ce97f3965e422df521e0ce86cad218d93
Signed-off-by: Junjie Wu <junjiew@codeaurora.org>
The ARM PMU supports monitoring bus access from each CPU. It also has the
ability to raise an IRQ when the counters overflow. This allows for it to
be used with the bw_hwmon governor to scale the CPU BW requests by
monitoring on the actual bus access traffic.
Change-Id: I0594a6acb846acdc11a18744033636951f22e387
Signed-off-by: Jacob Stevens <jstevens@codeaurora.org>
