Commit graph

83723 commits

Author SHA1 Message Date
Srinivas Dasari
350a145574 cfg80211: indicate support for external authentication
Define macro to indicate backport support for
external authentication where authentication can be
offloaded to userspace in specific cases such as SAE.

Change-Id: Ib253b303e82f583f61bc13d14c8d491d5ea2af15
CRs-Fixed: 2468738
Signed-off-by: Srinivas Dasari <dasaris@codeaurora.org>
Signed-off-by: Jiachao Wu <jiacwu@codeaurora.org>
Signed-off-by: Min Liu <minliu@codeaurora.org>
Signed-off-by: stonez <stonez@codeaurora.org>
2019-08-30 09:21:10 +02:00
Pavankumar Kondeti
97fe3984e9 sched/walt: Fix the memory leak of idle task load pointers
The memory for task load pointers are allocated twice for each
idle thread except for the boot CPU. This happens during boot
from idle_threads_init()->idle_init() in the following 2 paths.

1. idle_init()->fork_idle()->copy_process()->
		sched_fork()->init_new_task_load()

2. idle_init()->fork_idle()-> init_idle()->init_new_task_load()

The memory allocation for all tasks happens through the 1st path,
so use the same for idle tasks and kill the 2nd path. Since
the idle thread of boot CPU does not go through fork_idle(),
allocate the memory for it separately.

Change-Id: I4696a414ffe07d4114b56d326463026019e278f1
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
[schikk@codeaurora.org: resolved merge conflicts]
Signed-off-by: Swetha Chikkaboraiah <schikk@codeaurora.org>
2019-08-30 09:21:10 +02:00
Srinivas Dasari
3edd7e38cf cfg80211/nl80211: Optional authentication offload to userspace
This interface allows the host driver to offload the authentication to
user space. This is exclusively defined for host drivers that do not
define separate commands for authentication and association, but rely on
userspace SME (e.g., in wpa_supplicant for the ~WPA_DRIVER_FLAGS_SME
case) for the authentication to happen. This can be used to implement
SAE without full implementation in the kernel/firmware while still being
able to use NL80211_CMD_CONNECT with driver-based BSS selection.

Host driver sends NL80211_CMD_EXTERNAL_AUTH event to start/abort
authentication to the port on which connect is triggered and status
of authentication is further indicated by user space to host
driver through the same command response interface.

User space entities advertise this capability through the
NL80211_ATTR_EXTERNAL_AUTH_SUPP flag in the NL80211_CMD_CONNECT request.
Host drivers shall look at this capability to offload the authentication.

Signed-off-by: Srinivas Dasari <dasaris@qti.qualcomm.com>
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
[add socket connection ownership check]
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Git-commit: 40cbfa90218bc570a7959b436b9d48a18c361041
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
CRs-Fixed: 2468738
Change-Id: Id925dd82d9a9c719b32aac2de75b6ad001f1a958
[dasaris@codeaurora.org: merging with msm-specific changes]
Signed-off-by: Srinivas Dasari <dasaris@codeaurora.org>
Signed-off-by: Jiachao Wu <jiacwu@codeaurora.org>
Signed-off-by: Min Liu <minliu@codeaurora.org>
Signed-off-by: stonez <stonez@codeaurora.org>
2019-08-30 09:21:10 +02:00
Srinivas Dasari
8cb175043f cfg80211: Updated nl80211_commands to be in sync with upstream
Update nl80211_commands to be in sync with upstream.
This is needed to add new commands.

Change-Id: Ib6b71e3f66560b035377c7bc0c115490b04f5c4f
CRs-Fixed: 2468738
Signed-off-by: Srinivas Dasari <dasaris@codeaurora.org>
Signed-off-by: stonez <stonez@codeaurora.org>
2019-08-30 09:21:10 +02:00
codeworkx
f6334e63fa Merge tag 'LA.UM.7.4.r1-05400-8x98.0' into lineage-16.0
"LA.UM.7.4.r1-05400-8x98.0"

Change-Id: Iaa6db184c519b1a6f8de9b989ba402f156bec25c
2019-08-10 14:13:49 +02:00
Andrew Zaborowski
dea74cf7d5 cfg80211: NL80211_ATTR_SOCKET_OWNER support for CMD_CONNECT
Disconnect or deauthenticate when the owning socket is closed if this
flag is supplied to CMD_CONNECT or CMD_ASSOCIATE.  This may be used
to ensure userspace daemon doesn't leave an unmanaged connection behind.

In some situations it would be possible to account for that, to some
degree, in the deamon restart code or in the up/down scripts without
the use of this attribute.  But there will be systems where the daemon
can go away for varying periods without a warning due to local resource
management.

Signed-off-by: Andrew Zaborowski <andrew.zaborowski@intel.com>
Signed-off-by: Johannes Berg <johannes.berg@intel.com>

Git-commit: 36a554cec119bbd20c4ec0cb96bd4712d124bfea
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/jberg/mac80211-next.git
Change-Id: Ic09ee323fc6215059d5c2572ba3e77c56addad32
CRs-Fixed: 2468738
Signed-off-by: Srinivas Dasari <dasaris@codeaurora.org>
Signed-off-by: Jiachao Wu <jiacwu@codeaurora.org>
Signed-off-by: Min Liu <minliu@codeaurora.org>
Signed-off-by: stonez <stonez@codeaurora.org>
2019-06-26 12:40:38 +08:00
codeworkx
15f81a19b5 Merge tag 'LA.UM.7.4.r1-05300-8x98.0' into lineage-16.0
"LA.UM.7.4.r1-05300-8x98.0"

Change-Id: I8e27939efccb7d0bda0ac7c4e32afdf3c6d62507
2019-06-16 08:15:21 +02:00
kunleiz
15c87d11b5 lsm: check payload size validity before using it as array index
Payload size validity is not checked before using it in array index.
Check payload size to avoid out-of-boundary memory.

Change-Id: Ic0b06bb331fc1753ff7543bb218ab12d6a4a3ca8
Signed-off-by: kunleiz <kunleiz@codeaurora.org>
2019-04-29 02:14:22 -07:00
idkwhoiam322
3d23626712 Revert "msm: camera: isp: Handle frame drop due to delay "
Breaks GCAM This reverts commit 5bf83ce8c3.

Change-Id: I632497550243e9fc4b69ba4a0199d8a3e939fd36
2019-04-19 09:17:53 +02:00
Raghavendra Rao Ananta
ea2ee83c92 perf: Cancel the mux hrtimer during CPU hotplug to avoid migration
The current design of hrtimers migrates the pinned timers to a
different CPU upon its hotplug. However, perf-core needs to
maintain the mux-hrtimers on a per CPU basis. That is, each
hrtimer carries the context for that particular CPU and would
lose this context if it gets migrated to a different CPU. As a
result, cancel the hrtimer for the CPU that's about to go down
and restart it (if required) when the perf-events are being created.

Change-Id: I7a1d0456208855e3a99a7d49e59c6dae811d146e
Signed-off-by: Raghavendra Rao Ananta <rananta@codeaurora.org>
[mojha@codeaurora.org: Resolved merge conflict and added missing
 `cpuctx` variable to avoid build failure]
Signed-off-by: Mukesh Ojha <mojha@codeaurora.org>
2019-04-01 21:07:20 -07:00
Sandeep Singh
ab8f63420c icnss: Add Api to Block/Unblock modem shutdown
Add API to Block/Unblock modem graceful shutdown.

Change-Id: I69b061fc7d25762b2c36d9590802addfc170f91f
Signed-off-by: Sandeep Singh <sandsing@codeaurora.org>
2019-03-27 18:10:38 +05:30
codeworkx
6861312cbe Merge tag 'LA.UM.7.4.r1-04700-8x98.0' into auto
"LA.UM.7.4.r1-04700-8x98.0"

Change-Id: I777ab5e199fb463581fbfaf4750c8358d6f9e56f
2019-03-10 16:59:28 +01:00
Linus Torvalds
13fbdf9d9a mm: get rid of vmacache_flush_all() entirely
commit 7a9cdebdcc17e426fb5287e4a82db1dfe86339b2 upstream.

Jann Horn points out that the vmacache_flush_all() function is not only
potentially expensive, it's buggy too.  It also happens to be entirely
unnecessary, because the sequence number overflow case can be avoided by
simply making the sequence number be 64-bit.  That doesn't even grow the
data structures in question, because the other adjacent fields are
already 64-bit.

So simplify the whole thing by just making the sequence number overflow
case go away entirely, which gets rid of all the complications and makes
the code faster too.  Win-win.

[ Oleg Nesterov points out that the VMACACHE_FULL_FLUSHES statistics
  also just goes away entirely with this ]

Change-Id: Ib485c9f33638e844f2378a1ec376e64200d30fd2
Reported-by: Jann Horn <jannh@google.com>
Suggested-by: Will Deacon <will.deacon@arm.com>
Acked-by: Davidlohr Bueso <dave@stgolabs.net>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: stable@kernel.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Git-commit: 84580567f1f856d2c7a610273315852e345bc3ac
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Srinivasarao P <spathi@codeaurora.org>
2019-02-08 01:55:58 -08:00
Wei Wang
4d467694ff ANDROID: mm: add config for default readahead size
Change the VM_MAX_READAHEAD value from the default 128KB to a
configurable value. This will allow the readahead window to grow to a
maximum size bigger than 128KB, which greatly benefits to sequential
read throughput and thus boot performance.

Bug: 62413151
Test: boot walleye 100ms faster
Change-Id: Iad448cf1198056de46654dcb409466802b3b908d
Signed-off-by: Wei Wang <wvw@google.com>
2019-01-05 10:00:14 +01:00
Johannes Berg
da24b5fa50 ipv4: add option to drop gratuitous ARP packets
In certain 802.11 wireless deployments, there will be ARP proxies
that use knowledge of the network to correctly answer requests.
To prevent gratuitous ARP frames on the shared medium from being
a problem, on such deployments wireless needs to drop them.

Enable this by providing an option called "drop_gratuitous_arp".

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
(cherry picked from commit 4078228159c9f54cca7347a8bdace29f2abdef65)

Change-Id: I8772dbd7471085878f8b4161eb2a056d79b8b232
2018-12-30 12:35:55 +01:00
Pranav Vashi
6fe8b6e686 cpufreq: deoneplusify and remove PM QOS changes
Change-Id: I5a4fc49a9238d996cfe372a82e8988df3cfe0e30
2018-12-30 10:12:25 +01:00
Johannes Berg
ef6786d280 ipv6: add option to drop unsolicited neighbor advertisements
In certain 802.11 wireless deployments, there will be NA proxies
that use knowledge of the network to correctly answer requests.
To prevent unsolicitd advertisements on the shared medium from
being a problem, on such deployments wireless needs to drop them.

Enable this by providing an option called "drop_unsolicited_na".

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
(cherry picked from commit aec215e7aa380fe5f85eb6948766b58bf78cb6c3)

Change-Id: Iad429a767a786087b0985632be44932b2e3fd1a8
2018-12-30 10:11:48 +01:00
Johannes Berg
b4a8e0d981 ipv6: add option to drop unicast encapsulated in L2 multicast
In order to solve a problem with 802.11, the so-called hole-196 attack,
add an option (sysctl) called "drop_unicast_in_l2_multicast" which, if
enabled, causes the stack to drop IPv6 unicast packets encapsulated in
link-layer multi- or broadcast frames. Such frames can (as an attack)
be created by any member of the same wireless network and transmitted
as valid encrypted frames since the symmetric key for broadcast frames
is shared between all stations.

Reviewed-by: Julian Anastasov <ja@ssi.bg>
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
(cherry picked from commit dede82143bf1bbf92ea73a519bb0298b19c56cb9)

Change-Id: I76c8f84b53e95c40ad3c2b5adac0ec4964cc920c
2018-12-30 10:11:31 +01:00
Johannes Berg
dc5a07ff18 ipv4: add option to drop unicast encapsulated in L2 multicast
In order to solve a problem with 802.11, the so-called hole-196 attack,
add an option (sysctl) called "drop_unicast_in_l2_multicast" which, if
enabled, causes the stack to drop IPv4 unicast packets encapsulated in
link-layer multi- or broadcast frames. Such frames can (as an attack)
be created by any member of the same wireless network and transmitted
as valid encrypted frames since the symmetric key for broadcast frames
is shared between all stations.

Additionally, enabling this option provides compliance with a SHOULD
clause of RFC 1122.

Change-Id: I8de9fa5bdbea0556802f2ee553d0e73c1349213e
Reviewed-by: Julian Anastasov <ja@ssi.bg>
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2018-12-30 10:11:23 +01:00
codeworkx
3ea60bc578 synaptics S3320: bring inline with lineage-15.1
Change-Id: I36d8cbac90add8cccef212eccae7dede9c9b806a
2018-12-26 08:21:03 +01:00
Srinivas Girigowda
42535d6d10 cfg80211: Support backport of removing ieee80211
Bug: 62058353
Change-Id: Id8725947048bb4ba461dcb77b7b9023991a304be
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
2018-12-26 08:21:02 +01:00
Johannes Berg
37cfc3815d BACKPORT: cfg80211: remove enum ieee80211_band
This enum is already perfectly aliased to enum nl80211_band, and
the only reason for it is that we get IEEE80211_NUM_BANDS out of
it. There's no really good reason to not declare the number of
bands in nl80211 though, so do that and remove the cfg80211 one.

Change-Id: Ifc56e6297146c9095432b757fabd0c463d7cc583
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Bug: 62057517
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
2018-12-26 08:21:02 +01:00
Tom Marshall
e76227950e kernel: Only expose su when daemon is running
It has been claimed that the PG implementation of 'su' has security
vulnerabilities even when disabled.  Unfortunately, the people that
find these vulnerabilities often like to keep them private so they
can profit from exploits while leaving users exposed to malicious
hackers.

In order to reduce the attack surface for vulnerabilites, it is
therefore necessary to make 'su' completely inaccessible when it
is not in use (except by the root and system users).

Change-Id: I79716c72f74d0b7af34ec3a8054896c6559a181d
2018-12-26 08:21:02 +01:00
Scott Mertz
9bf720ca64 pwm: qpnp-pwm: add api for synchronous enable of pwms
Change-Id: Ibe4d8f8f8c9dbb89434cd633a049bd9e216af6f4
2018-12-26 08:21:01 +01:00
liochen
8148b9d900 Synchronize codes for OnePlus5 & 5T OxygenOS 9.0.0
kernel device tree source code for OnePlus 5 & 5T P device

Change-Id: I84f40e66833ea1ce30eb1d9a710d6e1529e9e637
2018-12-26 11:02:39 +08:00
Meera Gande
5bf83ce8c3 msm: camera: isp: Handle frame drop due to delay
In few scenarios, the request frame may get
delayed and current and request frame id may
become same. To handle such scenarios, made
changes to inform user to delay a frame and
process the request.

Change-Id: I31fa04c386922c48a043c511a163c76316e21987
Signed-off-by: Meera Gande <mgande@codeaurora.org>
2018-12-13 06:11:21 -08:00
Hemant Kumar
23db5b808c usb: gadget: Fix double free of device descriptor pointers
Upon driver unbind usb_free_all_descriptors() function frees all
speed descriptor pointers without setting them to NULL. In case
gadget speed changes (i.e from super speed plus to super speed)
after driver unbind only upto super speed descriptor pointers get
populated. Super speed plus desc still holds the stale (already
freed) pointer. As a result next composition switch results into
double free of super speed plus descriptor. Fix this issue by
setting all descriptor pointers to NULL after freeing them in
usb_free_all_descriptors(). Also clean up gsi_unbind() which is
setting up descriptor pointers to NULL already.

Change-Id: I4f28294c165bb3b5dc9feb4f22d819f527ad4d50
Signed-off-by: Hemant Kumar <hemantk@codeaurora.org>
Signed-off-by: Sriharsha Allenki <sallenki@codeaurora.org>
2018-12-04 21:38:53 -08:00
Kaushal Kumar
95645a63ac soc: qcom: scm: Add a noretry variant for scm_call2
scm_call2 can block scm calls up to 2s due to its
retry mechanism whenever the secure firmware is
busy waiting for certain processing by the client
who in turn is waiting upon its scm call to either
complete or return with failure.

Upon early return, client can process the pending
requests to free up secure firmware and unblock
processing of all pending scm calls. Add a noretry
variant for scm_call2 which can be used by clients
who do not intend to wait for 2s for return status.

Change-Id: I1f0849464a64c32a4de4510fa5787b0ab328725c
Signed-off-by: Kaushal Kumar <kaushalk@codeaurora.org>
2018-11-28 22:39:41 -08:00
Anurag Chouhan
33220157ed wcnss: Fix buffer overflow in wcnss_prealloc_get
There is potential integer truncation in the wcnss_prealloc_get api.
size_t is 8 byte on x64 platform and "unsigned int" is 4 byte.
To avoid this integer truncation, pass size as size_t instead
of unsigned int.

CRs-Fixed: 2269610
Change-Id: I14b274dd7cad98b55fdce1aaa27783272231afde
Signed-off-by: Anurag Chouhan <achouhan@codeaurora.org>
2018-10-06 00:22:15 -07:00
Anurag Chouhan
a3c288d3e4 wcnss: Fix buffer overflow in wcnss_prealloc_get
There is potential integer truncation in the wcnss_prealloc_get api.
size_t is 8 byte on x64 platform and "unsigned int" is 4 byte.
To avoid this integer truncation, pass size as size_t instead
of unsigned int.

CRs-Fixed: 2269610
Change-Id: I14b274dd7cad98b55fdce1aaa27783272231afde
Signed-off-by: Anurag Chouhan <achouhan@codeaurora.org>
2018-10-03 01:55:28 -07:00
Manoj Prabhu B
70c285e0b1 diag: Update the logging codes of events, logs and msgs
New msg SSIDs, log and event mask codes for different subsystems
are added as a new requirement for enable logging.

Change-Id: I31e784307e6de388e1de0806baacf00116360c30
Signed-off-by: Manoj Prabhu B <bmanoj@codeaurora.org>
2018-09-19 14:18:32 +05:30
Linux Build Service Account
d90306cf96 Merge "soc: qcom: socinfo: Add support for SDM455 soc-id" 2018-09-14 10:02:34 -07:00
Yue Ma
ee4241b768 cnss2: Add API to check if WLAN PCIe device is down
API provision for WLAN host driver to check if WLAN PCIe device
is down.

Change-Id: I91efcd781af67c72b787c89e6b619c4cc49da34b
Signed-off-by: Yue Ma <yuem@codeaurora.org>
2018-09-13 11:04:22 -07:00
Linux Build Service Account
ecb710876b Merge "mdio: Add support for mdio drivers." 2018-09-11 02:04:20 -07:00
Andrew Lunn
b40bf2be3c mdio: Add support for mdio drivers.
Not all devices on an MDIO bus are PHYs. Meaning not all MDIO drivers
are PHY drivers. Add support for generic MDIO drivers.

Change-Id: I65c7c8a497bbac9ef67b3d21c869818a09378e3c
Signed-off-by: Andrew Lunn <andrew@lunn.ch>
Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Git-Commit: a9049e0c513c4521dbfaa302af8ed08b3366b41f
Git-repo: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Signed-off-by: Anthony Mah <amah@codeaurora.org>
2018-09-10 11:42:06 -07:00
Teng Fei Fan
8cfbfe24f1 soc: qcom: socinfo: Add support for SDM455 soc-id
Add socinfo support for SDM455 Soc and update the
bindings fot the same.

Change-Id: I9b30795e202d84ae06020983b2d656772fb4f313
Signed-off-by: Teng Fei Fan <tengfei@codeaurora.org>
2018-09-10 10:36:23 +08:00
Linux Build Service Account
d6d8c10244 Merge "USB: pd: Notify selfpowered state based on type C current" 2018-09-08 12:33:56 -07:00
Linux Build Service Account
c44da670c2 Merge "Merge android-4.4.153 (5e24b4e) into msm-4.4" 2018-09-06 14:14:24 -07:00
Vijayavardhan Vennapusa
6494e9ee09 USB: pd: Notify selfpowered state based on type C current
It is required to notify device as selfpowered and bmaxpower as zero
even for non PD capable devices based on Type C current. Hence notify
as selfpowerer in bMattributes of configuation descriptor in case of
medium or high Type C current.

Change-Id: Ie552560d93a8195f4c69fdaf6086ef3a52b31d39
Signed-off-by: Vijayavardhan Vennapusa <vvreddy@codeaurora.org>
2018-09-05 14:46:32 +05:30
Linux Build Service Account
7649a71688 Merge "drm: Pass CRTC ID in userspace vblank events" 2018-09-01 16:12:40 -07:00
Linux Build Service Account
4036bbe404 Merge "Merge android-4.4.150 (5541782) into msm-4.4" 2018-08-31 12:34:13 -07:00
Anurag Chouhan
e7ad20af9a icnss: Add a flag to indicare FW rejuvenate
Add a flag to maintain fw rejuvenate state,
set if fw rejuvenate happens and reset at fw ready.
export an API to the wlan host driver to distinguish the
case of ssr or pdr with the FW rejuventae.

Change-Id: I7a01cc4996f68f78aa13eacf36648331a701882a
Signed-off-by: Anurag Chouhan <achouhan@codeaurora.org>
2018-08-30 02:21:41 -07:00
Srinivasarao P
b87d31674a Merge android-4.4.153 (5e24b4e) into msm-4.4
* refs/heads/tmp-5e24b4e
  Linux 4.4.153
  ovl: warn instead of error if d_type is not supported
  ovl: Do d_type check only if work dir creation was successful
  ovl: Ensure upper filesystem supports d_type
  x86/mm: Fix use-after-free of ldt_struct
  x86/mm/pat: Fix L1TF stable backport for CPA
  ANDROID: x86_64_cuttlefish_defconfig: Enable lz4 compression for zram
  UPSTREAM: drivers/block/zram/zram_drv.c: fix bug storing backing_dev
  BACKPORT: zram: introduce zram memory tracking
  BACKPORT: zram: record accessed second
  BACKPORT: zram: mark incompressible page as ZRAM_HUGE
  UPSTREAM: zram: correct flag name of ZRAM_ACCESS
  UPSTREAM: zram: Delete gendisk before cleaning up the request queue
  UPSTREAM: drivers/block/zram/zram_drv.c: make zram_page_end_io() static
  BACKPORT: zram: set BDI_CAP_STABLE_WRITES once
  UPSTREAM: zram: fix null dereference of handle
  UPSTREAM: zram: add config and doc file for writeback feature
  BACKPORT: zram: read page from backing device
  BACKPORT: zram: write incompressible pages to backing device
  BACKPORT: zram: identify asynchronous IO's return value
  BACKPORT: zram: add free space management in backing device
  UPSTREAM: zram: add interface to specif backing device
  UPSTREAM: zram: rename zram_decompress_page to __zram_bvec_read
  UPSTREAM: zram: inline zram_compress
  UPSTREAM: zram: clean up duplicated codes in __zram_bvec_write
  Linux 4.4.152
  reiserfs: fix broken xattr handling (heap corruption, bad retval)
  i2c: imx: Fix race condition in dma read
  PCI: pciehp: Fix use-after-free on unplug
  PCI: Skip MPS logic for Virtual Functions (VFs)
  PCI: hotplug: Don't leak pci_slot on registration failure
  parisc: Remove unnecessary barriers from spinlock.h
  bridge: Propagate vlan add failure to user
  packet: refine ring v3 block size test to hold one frame
  netfilter: conntrack: dccp: treat SYNC/SYNCACK as invalid if no prior state
  xfrm_user: prevent leaking 2 bytes of kernel memory
  parisc: Remove ordered stores from syscall.S
  ext4: fix spectre gadget in ext4_mb_regular_allocator()
  KVM: irqfd: fix race between EPOLLHUP and irq_bypass_register_consumer
  staging: android: ion: check for kref overflow
  tcp: identify cryptic messages as TCP seq # bugs
  net: qca_spi: Fix log level if probe fails
  net: qca_spi: Make sure the QCA7000 reset is triggered
  net: qca_spi: Avoid packet drop during initial sync
  net: usb: rtl8150: demote allmulti message to dev_dbg()
  net/ethernet/freescale/fman: fix cross-build error
  drm/nouveau/gem: off by one bugs in nouveau_gem_pushbuf_reloc_apply()
  tcp: remove DELAYED ACK events in DCTCP
  qlogic: check kstrtoul() for errors
  packet: reset network header if packet shorter than ll reserved space
  ixgbe: Be more careful when modifying MAC filters
  ARM: dts: am3517.dtsi: Disable reference to OMAP3 OTG controller
  ARM: 8780/1: ftrace: Only set kernel memory back to read-only after boot
  perf llvm-utils: Remove bashism from kernel include fetch script
  bnxt_en: Fix for system hang if request_irq fails
  drm/armada: fix colorkey mode property
  ieee802154: fakelb: switch from BUG_ON() to WARN_ON() on problem
  ieee802154: at86rf230: use __func__ macro for debug messages
  ieee802154: at86rf230: switch from BUG_ON() to WARN_ON() on problem
  ARM: pxa: irq: fix handling of ICMR registers in suspend/resume
  netfilter: x_tables: set module owner for icmp(6) matches
  smsc75xx: Add workaround for gigabit link up hardware errata.
  kasan: fix shadow_size calculation error in kasan_module_alloc
  tracing: Use __printf markup to silence compiler
  ARM: imx_v4_v5_defconfig: Select ULPI support
  ARM: imx_v6_v7_defconfig: Select ULPI support
  HID: wacom: Correct touch maximum XY of 2nd-gen Intuos
  m68k: fix "bad page state" oops on ColdFire boot
  bnx2x: Fix receiving tx-timeout in error or recovery state.
  drm/exynos: decon5433: Fix WINCONx reset value
  drm/exynos: decon5433: Fix per-plane global alpha for XRGB modes
  drm/exynos: gsc: Fix support for NV16/61, YUV420/YVU420 and YUV422 modes
  md/raid10: fix that replacement cannot complete recovery after reassemble
  dmaengine: k3dma: Off by one in k3_of_dma_simple_xlate()
  ARM: dts: da850: Fix interrups property for gpio
  selftests/x86/sigreturn/64: Fix spurious failures on AMD CPUs
  perf report powerpc: Fix crash if callchain is empty
  perf test session topology: Fix test on s390
  usb: xhci: increase CRS timeout value
  ARM: dts: am437x: make edt-ft5x06 a wakeup source
  brcmfmac: stop watchdog before detach and free everything
  cxgb4: when disabling dcb set txq dcb priority to 0
  Smack: Mark inode instant in smack_task_to_inode
  ipv6: mcast: fix unsolicited report interval after receiving querys
  locking/lockdep: Do not record IRQ state within lockdep code
  net: davinci_emac: match the mdio device against its compatible if possible
  ARC: Enable machine_desc->init_per_cpu for !CONFIG_SMP
  net: propagate dev_get_valid_name return code
  net: hamradio: use eth_broadcast_addr
  enic: initialize enic->rfs_h.lock in enic_probe
  qed: Add sanity check for SIMD fastpath handler.
  arm64: make secondary_start_kernel() notrace
  scsi: xen-scsifront: add error handling for xenbus_printf
  usb: gadget: dwc2: fix memory leak in gadget_init()
  usb: gadget: composite: fix delayed_status race condition when set_interface
  usb: dwc2: fix isoc split in transfer with no data
  ARM: dts: Cygnus: Fix I2C controller interrupt type
  selftests: sync: add config fragment for testing sync framework
  selftests: zram: return Kselftest Skip code for skipped tests
  selftests: user: return Kselftest Skip code for skipped tests
  selftests: static_keys: return Kselftest Skip code for skipped tests
  selftests: pstore: return Kselftest Skip code for skipped tests
  netfilter: ipv6: nf_defrag: reduce struct net memory waste
  ARC: Explicitly add -mmedium-calls to CFLAGS
  ANDROID: x86_64_cuttlefish_defconfig: Enable zram and zstd
  BACKPORT: crypto: zstd - Add zstd support
  UPSTREAM: zram: add zstd to the supported algorithms list
  UPSTREAM: lib: Add zstd modules
  UPSTREAM: lib: Add xxhash module
  UPSTREAM: zram: rework copy of compressor name in comp_algorithm_store()
  UPSTREAM: zram: constify attribute_group structures.
  UPSTREAM: zram: count same page write as page_stored
  UPSTREAM: zram: reduce load operation in page_same_filled
  UPSTREAM: zram: use zram_free_page instead of open-coded
  UPSTREAM: zram: introduce zram data accessor
  UPSTREAM: zram: remove zram_meta structure
  UPSTREAM: zram: use zram_slot_lock instead of raw bit_spin_lock op
  BACKPORT: zram: partial IO refactoring
  BACKPORT: zram: handle multiple pages attached bio's bvec
  UPSTREAM: zram: fix operator precedence to get offset
  BACKPORT: zram: extend zero pages to same element pages
  BACKPORT: zram: remove waitqueue for IO done
  UPSTREAM: zram: remove obsolete sysfs attrs
  UPSTREAM: zram: support BDI_CAP_STABLE_WRITES
  UPSTREAM: zram: revalidate disk under init_lock
  BACKPORT: mm: support anonymous stable page
  UPSTREAM: zram: use __GFP_MOVABLE for memory allocation
  UPSTREAM: zram: drop gfp_t from zcomp_strm_alloc()
  UPSTREAM: zram: add more compression algorithms
  UPSTREAM: zram: delete custom lzo/lz4
  UPSTREAM: zram: cosmetic: cleanup documentation
  UPSTREAM: zram: use crypto api to check alg availability
  BACKPORT: zram: switch to crypto compress API
  UPSTREAM: zram: rename zstrm find-release functions
  UPSTREAM: zram: introduce per-device debug_stat sysfs node
  UPSTREAM: zram: remove max_comp_streams internals
  UPSTREAM: zram: user per-cpu compression streams
  BACKPORT: zsmalloc: require GFP in zs_malloc()
  UPSTREAM: zram/zcomp: do not zero out zcomp private pages
  UPSTREAM: zram: pass gfp from zcomp frontend to backend
  UPSTREAM: socket: close race condition between sock_close() and sockfs_setattr()
  ANDROID: Refresh x86_64_cuttlefish_defconfig
  Linux 4.4.151
  isdn: Disable IIOCDBGVAR
  Bluetooth: avoid killing an already killed socket
  x86/mm: Simplify p[g4um]d_page() macros
  serial: 8250_dw: always set baud rate in dw8250_set_termios
  ACPI / PM: save NVS memory for ASUS 1025C laptop
  ACPI: save NVS memory for Lenovo G50-45
  USB: option: add support for DW5821e
  USB: serial: sierra: fix potential deadlock at close
  ALSA: vxpocket: Fix invalid endian conversions
  ALSA: memalloc: Don't exceed over the requested size
  ALSA: hda: Correct Asrock B85M-ITX power_save blacklist entry
  ALSA: cs5535audio: Fix invalid endian conversion
  ALSA: virmidi: Fix too long output trigger loop
  ALSA: vx222: Fix invalid endian conversions
  ALSA: hda - Turn CX8200 into D3 as well upon reboot
  ALSA: hda - Sleep for 10ms after entering D3 on Conexant codecs
  net_sched: fix NULL pointer dereference when delete tcindex filter
  vsock: split dwork to avoid reinitializations
  net_sched: Fix missing res info when create new tc_index filter
  llc: use refcount_inc_not_zero() for llc_sap_find()
  l2tp: use sk_dst_check() to avoid race on sk->sk_dst_cache
  dccp: fix undefined behavior with 'cwnd' shift in ccid2_cwnd_restart()

Conflicts:
	drivers/block/zram/zram_drv.c
	drivers/staging/android/ion/ion.c
	include/linux/swap.h
	mm/zsmalloc.c

Change-Id: I1c437ac5133503a939d06d51ec778b65371df6d1
Signed-off-by: Srinivasarao P <spathi@codeaurora.org>
2018-08-28 17:28:39 +05:30
Srinivasarao P
40eed1f500 Merge android-4.4.150 (5541782) into msm-4.4
* refs/heads/tmp-5541782
  Linux 4.4.150
  x86/speculation/l1tf: Exempt zeroed PTEs from inversion
  Linux 4.4.149
  x86/mm: Add TLB purge to free pmd/pte page interfaces
  ioremap: Update pgtable free interfaces with addr
  Bluetooth: hidp: buffer overflow in hidp_process_report
  ASoC: Intel: cht_bsw_max98090_ti: Fix jack initialization
  crypto: ablkcipher - fix crash flushing dcache in error path
  crypto: blkcipher - fix crash flushing dcache in error path
  crypto: vmac - separate tfm and request context
  crypto: vmac - require a block cipher with 128-bit block size
  kbuild: verify that $DEPMOD is installed
  i2c: ismt: fix wrong device address when unmap the data buffer
  kasan: don't emit builtin calls when sanitization is off
  tcp: Fix missing range_truesize enlargement in the backport
  x86/mm: Disable ioremap free page handling on x86-PAE

Conflicts:
	Makefile

Change-Id: I9cbfedbeb3bdb1df021d4f192a2a7392010cd627
Signed-off-by: Srinivasarao P <spathi@codeaurora.org>
2018-08-28 16:40:54 +05:30
Greg Kroah-Hartman
e15716b49f This is the 4.4.152 stable release
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEZH8oZUiU471FcZm+ONu9yGCSaT4FAlt/64YACgkQONu9yGCS
 aT4/pBAAynguZbVbn8QtYi37Kam0R4ZvXe7rKN8H1A7rwo9l9NJDaC4z2M0Iutfj
 1CfIEOeaf7WtxL25xgvDHQEOfB3/DH0xHbP/DtwqzpT0PmOUqMPaboGqOqXb//1J
 WetcluEOQfoYu1DWofPP1YkAR3vU4Bp40ucAGIN2wE4bvMTR2EMbV8Y5QYgIk6mN
 6n+Smg2Xpkq6paOhIrEt3C1P3lXlpY5Hxd54TGTRQ5c+vccXNldczIcP2Z0wue9/
 LRI8veyY1q/IDhVR8wCrlNb3df6kUQ0xixfTNnTkUJjLs3j+NAsaJiO8/nrdCUhJ
 xQORM7gQIMlccSNanKH0MHoCxhT3iMb8S6Hixvai5O+5XjP03TA7aAZ9Cyp7UqHg
 JY5SPbh7YOmvRXbx7/NAgyLYwRcJRt2PamNRApLQKFbot4bSvNJquhrAib5t6kCF
 HfbXjr9N969gLR4WmGkyOi0IHt8kaVwQitfBLZdj2QdlvyYWXmj0MuJ/I4BuZqtj
 0MyzS/v8cxkN/NWO1p1cB7pRzFtaXtHtC6rxzYXKCUycnHW9cJDf5PBgCfDMqyTY
 SdyuCeMrUo4mNEDItrKF8nbswew1T4UsayvJ6UgKHKr3QaH3Xp1mzeyt1GU38tn1
 ogKm9cVbOuAhnic67ikISFsj8oNptrq0w+Zqe3AKGO8B7CwXwis=
 =Q/T6
 -----END PGP SIGNATURE-----

Merge 4.4.152 into android-4.4

Changes in 4.4.152
	ARC: Explicitly add -mmedium-calls to CFLAGS
	netfilter: ipv6: nf_defrag: reduce struct net memory waste
	selftests: pstore: return Kselftest Skip code for skipped tests
	selftests: static_keys: return Kselftest Skip code for skipped tests
	selftests: user: return Kselftest Skip code for skipped tests
	selftests: zram: return Kselftest Skip code for skipped tests
	selftests: sync: add config fragment for testing sync framework
	ARM: dts: Cygnus: Fix I2C controller interrupt type
	usb: dwc2: fix isoc split in transfer with no data
	usb: gadget: composite: fix delayed_status race condition when set_interface
	usb: gadget: dwc2: fix memory leak in gadget_init()
	scsi: xen-scsifront: add error handling for xenbus_printf
	arm64: make secondary_start_kernel() notrace
	qed: Add sanity check for SIMD fastpath handler.
	enic: initialize enic->rfs_h.lock in enic_probe
	net: hamradio: use eth_broadcast_addr
	net: propagate dev_get_valid_name return code
	ARC: Enable machine_desc->init_per_cpu for !CONFIG_SMP
	net: davinci_emac: match the mdio device against its compatible if possible
	locking/lockdep: Do not record IRQ state within lockdep code
	ipv6: mcast: fix unsolicited report interval after receiving querys
	Smack: Mark inode instant in smack_task_to_inode
	cxgb4: when disabling dcb set txq dcb priority to 0
	brcmfmac: stop watchdog before detach and free everything
	ARM: dts: am437x: make edt-ft5x06 a wakeup source
	usb: xhci: increase CRS timeout value
	perf test session topology: Fix test on s390
	perf report powerpc: Fix crash if callchain is empty
	selftests/x86/sigreturn/64: Fix spurious failures on AMD CPUs
	ARM: dts: da850: Fix interrups property for gpio
	dmaengine: k3dma: Off by one in k3_of_dma_simple_xlate()
	md/raid10: fix that replacement cannot complete recovery after reassemble
	drm/exynos: gsc: Fix support for NV16/61, YUV420/YVU420 and YUV422 modes
	drm/exynos: decon5433: Fix per-plane global alpha for XRGB modes
	drm/exynos: decon5433: Fix WINCONx reset value
	bnx2x: Fix receiving tx-timeout in error or recovery state.
	m68k: fix "bad page state" oops on ColdFire boot
	HID: wacom: Correct touch maximum XY of 2nd-gen Intuos
	ARM: imx_v6_v7_defconfig: Select ULPI support
	ARM: imx_v4_v5_defconfig: Select ULPI support
	tracing: Use __printf markup to silence compiler
	kasan: fix shadow_size calculation error in kasan_module_alloc
	smsc75xx: Add workaround for gigabit link up hardware errata.
	netfilter: x_tables: set module owner for icmp(6) matches
	ARM: pxa: irq: fix handling of ICMR registers in suspend/resume
	ieee802154: at86rf230: switch from BUG_ON() to WARN_ON() on problem
	ieee802154: at86rf230: use __func__ macro for debug messages
	ieee802154: fakelb: switch from BUG_ON() to WARN_ON() on problem
	drm/armada: fix colorkey mode property
	bnxt_en: Fix for system hang if request_irq fails
	perf llvm-utils: Remove bashism from kernel include fetch script
	ARM: 8780/1: ftrace: Only set kernel memory back to read-only after boot
	ARM: dts: am3517.dtsi: Disable reference to OMAP3 OTG controller
	ixgbe: Be more careful when modifying MAC filters
	packet: reset network header if packet shorter than ll reserved space
	qlogic: check kstrtoul() for errors
	tcp: remove DELAYED ACK events in DCTCP
	drm/nouveau/gem: off by one bugs in nouveau_gem_pushbuf_reloc_apply()
	net/ethernet/freescale/fman: fix cross-build error
	net: usb: rtl8150: demote allmulti message to dev_dbg()
	net: qca_spi: Avoid packet drop during initial sync
	net: qca_spi: Make sure the QCA7000 reset is triggered
	net: qca_spi: Fix log level if probe fails
	tcp: identify cryptic messages as TCP seq # bugs
	staging: android: ion: check for kref overflow
	KVM: irqfd: fix race between EPOLLHUP and irq_bypass_register_consumer
	ext4: fix spectre gadget in ext4_mb_regular_allocator()
	parisc: Remove ordered stores from syscall.S
	xfrm_user: prevent leaking 2 bytes of kernel memory
	netfilter: conntrack: dccp: treat SYNC/SYNCACK as invalid if no prior state
	packet: refine ring v3 block size test to hold one frame
	bridge: Propagate vlan add failure to user
	parisc: Remove unnecessary barriers from spinlock.h
	PCI: hotplug: Don't leak pci_slot on registration failure
	PCI: Skip MPS logic for Virtual Functions (VFs)
	PCI: pciehp: Fix use-after-free on unplug
	i2c: imx: Fix race condition in dma read
	reiserfs: fix broken xattr handling (heap corruption, bad retval)
	Linux 4.4.152

Change-Id: I1058813031709d20abd0bc45e9ac5fc68ab3a1d7
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
2018-08-24 13:37:12 +02:00
Randy Dunlap
0821ddad49 net/ethernet/freescale/fman: fix cross-build error
[ Upstream commit c133459765fae249ba482f62e12f987aec4376f0 ]

  CC [M]  drivers/net/ethernet/freescale/fman/fman.o
In file included from ../drivers/net/ethernet/freescale/fman/fman.c:35:
../include/linux/fsl/guts.h: In function 'guts_set_dmacr':
../include/linux/fsl/guts.h:165:2: error: implicit declaration of function 'clrsetbits_be32' [-Werror=implicit-function-declaration]
  clrsetbits_be32(&guts->dmacr, 3 << shift, device << shift);
  ^~~~~~~~~~~~~~~

Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: Madalin Bucur <madalin.bucur@nxp.com>
Cc: netdev@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Sasha Levin <alexander.levin@microsoft.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2018-08-24 13:27:00 +02:00
Yuchung Cheng
43707aa8c5 tcp: remove DELAYED ACK events in DCTCP
[ Upstream commit a69258f7aa2623e0930212f09c586fd06674ad79 ]

After fixing the way DCTCP tracking delayed ACKs, the delayed-ACK
related callbacks are no longer needed

Signed-off-by: Yuchung Cheng <ycheng@google.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Acked-by: Neal Cardwell <ncardwell@google.com>
Acked-by: Lawrence Brakmo <brakmo@fb.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Sasha Levin <alexander.levin@microsoft.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2018-08-24 13:26:59 +02:00
Eric Dumazet
8747d9e7d4 netfilter: ipv6: nf_defrag: reduce struct net memory waste
[ Upstream commit 9ce7bc036ae4cfe3393232c86e9e1fea2153c237 ]

It is a waste of memory to use a full "struct netns_sysctl_ipv6"
while only one pointer is really used, considering netns_sysctl_ipv6
keeps growing.

Also, since "struct netns_frags" has cache line alignment,
it is better to move the frags_hdr pointer outside, otherwise
we spend a full cache line for this pointer.

This saves 192 bytes of memory per netns.

Fixes: c038a767cd ("ipv6: add a new namespace for nf_conntrack_reasm")
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: Sasha Levin <alexander.levin@microsoft.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2018-08-24 13:26:53 +02:00
Nick Terrell
460c0d10de UPSTREAM: lib: Add zstd modules
Add zstd compression and decompression kernel modules.
zstd offers a wide varity of compression speed and quality trade-offs.
It can compress at speeds approaching lz4, and quality approaching lzma.
zstd decompressions at speeds more than twice as fast as zlib, and
decompression speed remains roughly the same across all compression levels.

The code was ported from the upstream zstd source repository. The
`linux/zstd.h` header was modified to match linux kernel style.
The cross-platform and allocation code was stripped out. Instead zstd
requires the caller to pass a preallocated workspace. The source files
were clang-formatted [1] to match the Linux Kernel style as much as
possible. Otherwise, the code was unmodified. We would like to avoid
as much further manual modification to the source code as possible, so it
will be easier to keep the kernel zstd up to date.

I benchmarked zstd compression as a special character device. I ran zstd
and zlib compression at several levels, as well as performing no
compression, which measure the time spent copying the data to kernel space.
Data is passed to the compresser 4096 B at a time. The benchmark file is
located in the upstream zstd source repository under
`contrib/linux-kernel/zstd_compress_test.c` [2].

I ran the benchmarks on a Ubuntu 14.04 VM with 2 cores and 4 GiB of RAM.
The VM is running on a MacBook Pro with a 3.1 GHz Intel Core i7 processor,
16 GB of RAM, and a SSD. I benchmarked using `silesia.tar` [3], which is
211,988,480 B large. Run the following commands for the benchmark:

    sudo modprobe zstd_compress_test
    sudo mknod zstd_compress_test c 245 0
    sudo cp silesia.tar zstd_compress_test

The time is reported by the time of the userland `cp`.
The MB/s is computed with

    1,536,217,008 B / time(buffer size, hash)

which includes the time to copy from userland.
The Adjusted MB/s is computed with

    1,536,217,088 B / (time(buffer size, hash) - time(buffer size, none)).

The memory reported is the amount of memory the compressor requests.

| Method   | Size (B) | Time (s) | Ratio | MB/s    | Adj MB/s | Mem (MB) |
|----------|----------|----------|-------|---------|----------|----------|
| none     | 11988480 |    0.100 |     1 | 2119.88 |        - |        - |
| zstd -1  | 73645762 |    1.044 | 2.878 |  203.05 |   224.56 |     1.23 |
| zstd -3  | 66988878 |    1.761 | 3.165 |  120.38 |   127.63 |     2.47 |
| zstd -5  | 65001259 |    2.563 | 3.261 |   82.71 |    86.07 |     2.86 |
| zstd -10 | 60165346 |   13.242 | 3.523 |   16.01 |    16.13 |    13.22 |
| zstd -15 | 58009756 |   47.601 | 3.654 |    4.45 |     4.46 |    21.61 |
| zstd -19 | 54014593 |  102.835 | 3.925 |    2.06 |     2.06 |    60.15 |
| zlib -1  | 77260026 |    2.895 | 2.744 |   73.23 |    75.85 |     0.27 |
| zlib -3  | 72972206 |    4.116 | 2.905 |   51.50 |    52.79 |     0.27 |
| zlib -6  | 68190360 |    9.633 | 3.109 |   22.01 |    22.24 |     0.27 |
| zlib -9  | 67613382 |   22.554 | 3.135 |    9.40 |     9.44 |     0.27 |

I benchmarked zstd decompression using the same method on the same machine.
The benchmark file is located in the upstream zstd repo under
`contrib/linux-kernel/zstd_decompress_test.c` [4]. The memory reported is
the amount of memory required to decompress data compressed with the given
compression level. If you know the maximum size of your input, you can
reduce the memory usage of decompression irrespective of the compression
level.

| Method   | Time (s) | MB/s    | Adjusted MB/s | Memory (MB) |
|----------|----------|---------|---------------|-------------|
| none     |    0.025 | 8479.54 |             - |           - |
| zstd -1  |    0.358 |  592.15 |        636.60 |        0.84 |
| zstd -3  |    0.396 |  535.32 |        571.40 |        1.46 |
| zstd -5  |    0.396 |  535.32 |        571.40 |        1.46 |
| zstd -10 |    0.374 |  566.81 |        607.42 |        2.51 |
| zstd -15 |    0.379 |  559.34 |        598.84 |        4.61 |
| zstd -19 |    0.412 |  514.54 |        547.77 |        8.80 |
| zlib -1  |    0.940 |  225.52 |        231.68 |        0.04 |
| zlib -3  |    0.883 |  240.08 |        247.07 |        0.04 |
| zlib -6  |    0.844 |  251.17 |        258.84 |        0.04 |
| zlib -9  |    0.837 |  253.27 |        287.64 |        0.04 |

Tested in userland using the test-suite in the zstd repo under
`contrib/linux-kernel/test/UserlandTest.cpp` [5] by mocking the kernel
functions. Fuzz tested using libfuzzer [6] with the fuzz harnesses under
`contrib/linux-kernel/test/{RoundTripCrash.c,DecompressCrash.c}` [7] [8]
with ASAN, UBSAN, and MSAN. Additionaly, it was tested while testing the
BtrFS and SquashFS patches coming next.

[1] https://clang.llvm.org/docs/ClangFormat.html
[2] https://github.com/facebook/zstd/blob/dev/contrib/linux-kernel/zstd_compress_test.c
[3] http://sun.aei.polsl.pl/~sdeor/index.php?page=silesia
[4] https://github.com/facebook/zstd/blob/dev/contrib/linux-kernel/zstd_decompress_test.c
[5] https://github.com/facebook/zstd/blob/dev/contrib/linux-kernel/test/UserlandTest.cpp
[6] http://llvm.org/docs/LibFuzzer.html
[7] https://github.com/facebook/zstd/blob/dev/contrib/linux-kernel/test/RoundTripCrash.c
[8] https://github.com/facebook/zstd/blob/dev/contrib/linux-kernel/test/DecompressCrash.c

zstd source repository: https://github.com/facebook/zstd

Signed-off-by: Nick Terrell <terrelln@fb.com>
Signed-off-by: Chris Mason <clm@fb.com>

(cherry picked from commit 73f3d1b48f5069d46ba48aa28c2898dc93185560)
Signed-off-by: Peter Kalauskas <peskal@google.com>
Bug: 112488418
Change-Id: I47b9d43a8065b2b5a1362f8458065f0811cf70b9
2018-08-23 12:00:19 -07:00
Nick Terrell
490b2f1f67 UPSTREAM: lib: Add xxhash module
Adds xxhash kernel module with xxh32 and xxh64 hashes. xxhash is an
extremely fast non-cryptographic hash algorithm for checksumming.
The zstd compression and decompression modules added in the next patch
require xxhash. I extracted it out from zstd since it is useful on its
own. I copied the code from the upstream XXHash source repository and
translated it into kernel style. I ran benchmarks and tests in the kernel
and tests in userland.

I benchmarked xxhash as a special character device. I ran in four modes,
no-op, xxh32, xxh64, and crc32. The no-op mode simply copies the data to
kernel space and ignores it. The xxh32, xxh64, and crc32 modes compute
hashes on the copied data. I also ran it with four different buffer sizes.
The benchmark file is located in the upstream zstd source repository under
`contrib/linux-kernel/xxhash_test.c` [1].

I ran the benchmarks on a Ubuntu 14.04 VM with 2 cores and 4 GiB of RAM.
The VM is running on a MacBook Pro with a 3.1 GHz Intel Core i7 processor,
16 GB of RAM, and a SSD. I benchmarked using the file `filesystem.squashfs`
from `ubuntu-16.10-desktop-amd64.iso`, which is 1,536,217,088 B large.
Run the following commands for the benchmark:

    modprobe xxhash_test
    mknod xxhash_test c 245 0
    time cp filesystem.squashfs xxhash_test

The time is reported by the time of the userland `cp`.
The GB/s is computed with

    1,536,217,008 B / time(buffer size, hash)

which includes the time to copy from userland.
The Normalized GB/s is computed with

    1,536,217,088 B / (time(buffer size, hash) - time(buffer size, none)).

| Buffer Size (B) | Hash  | Time (s) | GB/s | Adjusted GB/s |
|-----------------|-------|----------|------|---------------|
|            1024 | none  |    0.408 | 3.77 |             - |
|            1024 | xxh32 |    0.649 | 2.37 |          6.37 |
|            1024 | xxh64 |    0.542 | 2.83 |         11.46 |
|            1024 | crc32 |    1.290 | 1.19 |          1.74 |
|            4096 | none  |    0.380 | 4.04 |             - |
|            4096 | xxh32 |    0.645 | 2.38 |          5.79 |
|            4096 | xxh64 |    0.500 | 3.07 |         12.80 |
|            4096 | crc32 |    1.168 | 1.32 |          1.95 |
|            8192 | none  |    0.351 | 4.38 |             - |
|            8192 | xxh32 |    0.614 | 2.50 |          5.84 |
|            8192 | xxh64 |    0.464 | 3.31 |         13.60 |
|            8192 | crc32 |    1.163 | 1.32 |          1.89 |
|           16384 | none  |    0.346 | 4.43 |             - |
|           16384 | xxh32 |    0.590 | 2.60 |          6.30 |
|           16384 | xxh64 |    0.466 | 3.30 |         12.80 |
|           16384 | crc32 |    1.183 | 1.30 |          1.84 |

Tested in userland using the test-suite in the zstd repo under
`contrib/linux-kernel/test/XXHashUserlandTest.cpp` [2] by mocking the
kernel functions. A line in each branch of every function in `xxhash.c`
was commented out to ensure that the test-suite fails. Additionally
tested while testing zstd and with SMHasher [3].

[1] https://phabricator.intern.facebook.com/P57526246
[2] https://github.com/facebook/zstd/blob/dev/contrib/linux-kernel/test/XXHashUserlandTest.cpp
[3] https://github.com/aappleby/smhasher

zstd source repository: https://github.com/facebook/zstd
XXHash source repository: https://github.com/cyan4973/xxhash

Signed-off-by: Nick Terrell <terrelln@fb.com>
Signed-off-by: Chris Mason <clm@fb.com>

(cherry picked from commit 5d2405227a9eaea48e8cc95756a06d407b11f141)
Signed-off-by: Peter Kalauskas <peskal@google.com>
Bug: 112488418
Change-Id: I4b63e96457f17cf455591e8f35058dacd7aa9004
2018-08-23 12:00:19 -07:00