[ Upstream commit 0f22e40053bd5378ad1e3250e65c574fd61c0cd6 ]
Make sure the rx_allocator worker is canceled before running the
rx_init routine. rx_init frees and re-allocates all rxb's pages. The
rx_allocator worker also allocates pages for the used rxb's. Running
rx_init and rx_allocator simultaniously causes a kernel panic. Fix
that by canceling the work in rx_init.
Signed-off-by: Shaul Triebitz <shaul.triebitz@intel.com>
Signed-off-by: Luca Coelho <luciano.coelho@intel.com>
Signed-off-by: Sasha Levin <alexander.levin@microsoft.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
The mac80211 patch to pass the NAPI struct only changed iwlwifi to
store the NAPI struct, but we can do better: pass it directly from
the lower transport layer to the opmode during RX, and then on to
mac80211 from there.
When we add multiple RX queues, we can then pass the appropriate
NAPI struct properly.
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
As a preperation for multiple RX queues change the RBD
allocation model.
The new model includes a background allocator. The allocator is
called by the interrupt handler when there are two released
buffers by the queue, and the allocator starts allocating eight
pages per request.
When the queue has released 8 pages it tries claiming the
request. If the pages are not ready - it keeps claiming.
This new model should make sure that RBDs are always available
across the multiple queues.
The RBDs are transferred between the allocator and the queue.
The queue moves the free RBDs upon freeing them to the allocator.
The allocator moves them back to the queue's possession when the
request is claimed.
The allocator has an initial pool to make sure there are always RBDs
available for the request completion.
Release of the buffers at exit is done per pools - the allocator
frees its own initial pool and the queue frees its own pool.
Existing code refactor -
-Queue's initial pool is the size of the queue only as the allocation
of the new buffers no longer uses this pool.
-Removal of replenish background work, and replenish calls in the
interrupt handler and restock().
-The replenish() and the rxq used_list are used only during
initialization.
-Moved page allocation to a new function for code reuse.
New code -
Allocator code - new structure and functions.
Interrupt handler uses the allocator functions for replenishing buffers.
Reuse of the restock() method.
Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Since the CSR_DRAM_INIT_TBL_WRITE_POINTER bit wasn't set
on ict reset, in some flows (like disable ict followed by
immediate reset ict) the driver and hardware went out
of sync (the driver cleared the ict_index, while the hw
kept it intact).
Fix it by setting the flag when resetting ict.
Signed-off-by: Eliad Peller <eliad@wizery.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
When the firmware crashes, we can't expect the Tx queues to
progress. Cancel their timer.
Reviewed-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
With the previous patch series, no opmode continues using the
command or handler_status (i.e. the return value from the RX)
so it can be removed now.
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
This reverts commit 5f17570354.
This patch introduced a high latency in buffer allocation
under extreme load. This latency caused a firmwre crash.
The same scenario works fine with this patch reverted.
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
This allows to ensure that we don't have races between them.
A user reported that stop_device was called twice upon
rfkill interrupt after suspend. When the interrupts are
enabled, and right after when we directly check the rfkill
state.
Reviewed-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
As a preperation for multiple RX queues change the RBD
allocation model.
The new model includes a background allocator. The allocator is
called by the interrupt handler when there are two released
buffers by the queue, and the allocator starts allocating eight
pages per request.
When the queue has released 8 pages it tries claiming the
request. If the pages are not ready - it keeps claiming.
This new model should make sure that RBDs are always available
across the multiple queues.
The RBDs are transferred between the allocator and the queue.
The queue moves the free RBDs upon freeing them to the allocator.
The allocator moves them back to the queue's possession when the
request is claimed.
The allocator has an initial pool to make sure there are always RBDs
available for the request completion.
Release of the buffers at exit is done per pools - the allocator
frees its own initial pool and the queue frees its own pool.
Existing code refactor -
-Queue's initial pool is the size of the queue only as the allocation
of the new buffers no longer uses this pool.
-Removal of replenish background work, and replenish calls in the
interrupt handler and restock().
-The replenish() and the rxq used_list are used only during
initialization.
-Moved page allocation to a new function for code reuse.
New code -
Allocator code - new structure and functions.
Interrupt handler uses the allocator functions for replenishing buffers.
Reuse of the restock() method.
Signed-off-by: Sara Sharon <sara.sharon@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Allow a cleaner way to access those hw-dependent registers,
instead of using the product family type etc.
Signed-off-by: Avri Altman <avri.altman@intel.com>
Reviewed-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
For each RX packet until this patch there only was a debug
print of the HCMD and the offset. This adds also the
sequence number of the packet for easier matching between
what was sent, what came back / was received, and what
got stuck somewhere and was never responded by the FW.
Signed-off-by: Liad Kaufman <liad.kaufman@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
When freeing the structures used for command data, clear their
memory as they may have contained key material at some point.
Also clear the duplicated buffer when freeing it to be safe;
currently key material is never put there but that may change.
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Our legal structure changed at some point (see wikipedia), but
we forgot to immediately switch over to the new copyright
notice.
For files that we have modified in the time since the change,
add the proper copyright notice now.
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
This variable always tracks a constant value (256) so there's
no need to have it. Removing it simplifies code generation,
reducing the .text size (by about 240 bytes on x86-64.)
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Besides the different allocation flags, they are really
the same. Pass the gfp_t flags as a parameter, and unify
them.
Reviewed-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Similar to the recent RX queue patch, this changes the need_update
handling for the TX queues to be clearer and only done when needed.
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
When updating the write pointer, the TX queue should be locked
to get consistent state, fix that in the interrupt handler.
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
When shadow registers are enabled, then need_update never needs
to be set, so move the need_update handling into the function
that really needs to do it (iwl_pcie_rxq_inc_wr_ptr) and also
separate the check when it woke up. While at it, convert it to
bool.
This also clarifies the locking and means the irq_lock needs to
no longer be held for any such updates.
The irq_lock also doesn't have to be held for restocking since
everything else locks the RX queue properly, so remove that and
finally disentangle the two locks entirely so there aren't any
dependencies between the two left.
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Use the new NAPI infrastructure added to mac80211 to get
GRO. We don't really implement NAPI since we don't have
a real poll function and we never schedule a NAPI poll.
Instead of this, we collect all the packets we got from a
single interrupt and then call napi_gro_flush().
This allows us to benefit from GRO. In half duplex medium
like WiFi, its main advantage is that it reduces the number
of TCP Acks, hence improving the TCP Rx performance.
Since we call the Rx path with a spinlock held, remove
the might_sleep mention from the op_mode's API.
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Reviewed-by: Ido Yariv <ido@wizery.com>
[Squash different patches and rewrite the commit message]
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
When indicating RF-kill toggle to the higher layer, that
may in turn call back to the transport (for MVM at least)
to turn off the device quickly. Instead of that, allow it
to return whether or not the device should be turned off,
this gets rid of the call indirection and will help make
the API more consistent when we go back to non-threaded
interrupts again for PCIe.
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
The various code blocks in iwl_pcie_[rt]xq_inc_wr_ptr
finally do the same things, so just merge them
all and make the functions cleaner.
Signed-off-by: Eliad Peller <eliadx.peller@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
In iwl_pcie_int_cause_non_ict, trans_pcie is used for lockdep
purposes only. Since this might not be enabled, trans_pcie
finds itself without user leading to a complaint from gcc.
Avoid using trans_pcie by inlining IWL_TRANS_GET_PCIE_TRANS.
Reviewed-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Add an inline helper function for getting an RX packet's
length or payload length and use it throughout the code
(most of which I did using an spatch.)
While at it, adjust some code, and remove a bogus comment
from the dvm calibration code.
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Reviewed-by: Eran Harary <eran.harary@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
This lock was never acquired in the primary interrupt
handler, but since it was acquired along with irq_lock
which had to disable interrupts, rxq->lock had to disable
interrupts too.
Now that trans_pcie->irq_lock isn't acquired in the primary
interrupt handler, rxq->lock can let interrupt enabled.
Reviewed-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Since we don't take this lock in the primary interrupt
handler, there is no pointin disabling the interrupt
in the critical section protected by trans_pcie->irq_lock.
Reviewed-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Handling interrupt with no cause and printing logs doesn't
need to be ICT / non-ICT specific move this to the common
code.
Reviewed-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
This was useful when the handling was not in the same
context as the interrupt cause retrieval: we could have
several hard interrupts until the handler gets called.
Since we retrieve the interrupt cause in the handler itself,
there is no need to OR the interrupt causes.
Reviewed-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
These functions are meant to return an interrupt cause and
not an irqreturn_t.
We still return IRQ_HANDLED if we had an error and IRQ_NONE
if our device hasn't fired any interrupt.
Reviewed-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Instead of having:
iwl_pcie_irq_handler
iwl_pcie_isr_ict
iwl_pcie_isr_non_ict
we now have:
iwl_pcie_irq_handler:
if (use_ict))
iwl_pcie_int_cause_ict;
else
iwl_pcie_int_cause_non_ict;
This is much clearer.
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
We now disable the interrupts in the hardware from the
upper half and all the rest (including reading the interrupt
cause) is done in the handler.
Reviewed-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
The purpose of this is to be able to call these functions
from the interrupt handler and not from the primary
interrupt handler.
Reviewed-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Separate the code that simply disables interrupt in the
hardware and the code that checks what interrupt fired.
This will be useful to move the second part in the threaded
handler which will be done in a future patch.
Reviewed-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Track the interrupt mask in software, making it exactly
what is configured in the interrupt mask register in the
hardware.
This allows not to access the register from the interrupt
handler. This was the case for ICT interrupt already, but
not for non-ICT interrupt.
Reviewed-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Since iwl_trans_pcie_alloc_ict is called in the PCIe
allocation code, we always set CSR_INT_BIT_RX_PERIODIC.
Move that bit to the default list of interrupts we enable
and simplify the code.
Also use dma_zalloc_ and avoid to memset the memory
afterwards.
trans_pcie->ict_index is 0 since trans_pcie has just been
kzalloced, remove the redundant assignment.
Reviewed-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
In case a sync command timeouts or Tx is stuck while a FW error
interrupt arrives, we might call iwl_op_mode_nic_error twice before
a restart has been initiated. This will cause a reprobe. Unify calls
to this function at the transport level and only call it on the first
FW error in a given by checking the transport FW error flag.
While at it, remove the privately defined iwl_nic_error from PCIE code
and use the common callback instead.
Signed-off-by: Arik Nemtsov <arik@wizery.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
The same bits are employed in all transport layers. Put the status
field in the common transport layer. This allows us to employ them
in common transport code.
Signed-off-by: Arik Nemtsov <arik@wizery.com>
Reviewed-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
inta is checked to be zero in a IRQ_NONE branch so afterwards it
cannot be zero as it is never modified.
Signed-off-by: Michal Nazarewicz <mina86@mina86.com>
[reword the patch title and fix comment]
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
If we call ieee80211_hw_restart, it means that the
firmware is in bad condition and will be reset soon.
Since the firmware will be reset, there is no good
reason to keep sending host commands.
Signed-off-by: Alexander Bondar <alexander.bondar@intel.com>
Reviewed-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
We changed the timeout for the interrupt coealescing for
calibration, but that wasn't effective since we changed
that value back before loading the firmware. Since
calibrations are notification from firmware and not Rx
packets, this doesn't change anyway - the firmware will
fire an interrupt straight away regardless of the interrupt
coalescing value.
Also, a HW issue has been discovered in 7000 devices series.
The work around is to disable the new interrupt coalescing
timeout feature - do this by setting bit 31 in
CSR_INT_COALESCING.
This has been fixed in 7265 which means that we can't rely
on the device family and must have a hint in the iwl_cfg
structure.
Cc: stable@vger.kernel.org [3.10+]
Fixes: 99cd471423 ("iwlwifi: add 7000 series device configuration")
Reviewed-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Instead of having the same code sequentially, fall-through.
Signed-off-by: Ido Yariv <ido@wizery.com>
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Simplify iwl_rxq_space to improve readability and reduce
the ambiguity spares to a single element.
Signed-off-by: Ido Yariv <ido@wizery.com>
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
This reverts commit a53ee0a308.
This fix causes a worse HW Error when entering RF-Kill.
Signed-off-by: Guy Cohen <guy.cohen@intel.com>
Signed-off-by: Dor Shaish <dor.shaish@intel.com>
Reviewed-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
If we forget to do so, we can't send HCMD to firmware while
the NIC is in RFKILL state.
Cc: stable@vger.kernel.org
Signed-off-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
This means it can be shared for different transport
layers in the future.
Signed-off-by: Inbal Hacohen <Inbal.Hacohen@intel.com>
Reviewed-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
A few places use just 'q', use 'rxq' there like all
other places.
Reviewed-by: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
