evie/android_kernel_oneplus_msm8998
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
android_kernel_oneplus_msm8998/lib
History
Nick Terrell 490b2f1f67 UPSTREAM: lib: Add xxhash module 
Adds xxhash kernel module with xxh32 and xxh64 hashes. xxhash is an
extremely fast non-cryptographic hash algorithm for checksumming.
The zstd compression and decompression modules added in the next patch
require xxhash. I extracted it out from zstd since it is useful on its
own. I copied the code from the upstream XXHash source repository and
translated it into kernel style. I ran benchmarks and tests in the kernel
and tests in userland.

I benchmarked xxhash as a special character device. I ran in four modes,
no-op, xxh32, xxh64, and crc32. The no-op mode simply copies the data to
kernel space and ignores it. The xxh32, xxh64, and crc32 modes compute
hashes on the copied data. I also ran it with four different buffer sizes.
The benchmark file is located in the upstream zstd source repository under
`contrib/linux-kernel/xxhash_test.c` [1].

I ran the benchmarks on a Ubuntu 14.04 VM with 2 cores and 4 GiB of RAM.
The VM is running on a MacBook Pro with a 3.1 GHz Intel Core i7 processor,
16 GB of RAM, and a SSD. I benchmarked using the file `filesystem.squashfs`
from `ubuntu-16.10-desktop-amd64.iso`, which is 1,536,217,088 B large.
Run the following commands for the benchmark:

    modprobe xxhash_test
    mknod xxhash_test c 245 0
    time cp filesystem.squashfs xxhash_test

The time is reported by the time of the userland `cp`.
The GB/s is computed with

    1,536,217,008 B / time(buffer size, hash)

which includes the time to copy from userland.
The Normalized GB/s is computed with

    1,536,217,088 B / (time(buffer size, hash) - time(buffer size, none)).

| Buffer Size (B) | Hash  | Time (s) | GB/s | Adjusted GB/s |
|-----------------|-------|----------|------|---------------|
|            1024 | none  |    0.408 | 3.77 |             - |
|            1024 | xxh32 |    0.649 | 2.37 |          6.37 |
|            1024 | xxh64 |    0.542 | 2.83 |         11.46 |
|            1024 | crc32 |    1.290 | 1.19 |          1.74 |
|            4096 | none  |    0.380 | 4.04 |             - |
|            4096 | xxh32 |    0.645 | 2.38 |          5.79 |
|            4096 | xxh64 |    0.500 | 3.07 |         12.80 |
|            4096 | crc32 |    1.168 | 1.32 |          1.95 |
|            8192 | none  |    0.351 | 4.38 |             - |
|            8192 | xxh32 |    0.614 | 2.50 |          5.84 |
|            8192 | xxh64 |    0.464 | 3.31 |         13.60 |
|            8192 | crc32 |    1.163 | 1.32 |          1.89 |
|           16384 | none  |    0.346 | 4.43 |             - |
|           16384 | xxh32 |    0.590 | 2.60 |          6.30 |
|           16384 | xxh64 |    0.466 | 3.30 |         12.80 |
|           16384 | crc32 |    1.183 | 1.30 |          1.84 |

Tested in userland using the test-suite in the zstd repo under
`contrib/linux-kernel/test/XXHashUserlandTest.cpp` [2] by mocking the
kernel functions. A line in each branch of every function in `xxhash.c`
was commented out to ensure that the test-suite fails. Additionally
tested while testing zstd and with SMHasher [3].

[1] https://phabricator.intern.facebook.com/P57526246
[2] https://github.com/facebook/zstd/blob/dev/contrib/linux-kernel/test/XXHashUserlandTest.cpp
[3] https://github.com/aappleby/smhasher

zstd source repository: https://github.com/facebook/zstd
XXHash source repository: https://github.com/cyan4973/xxhash

Signed-off-by: Nick Terrell <terrelln@fb.com>
Signed-off-by: Chris Mason <clm@fb.com>

(cherry picked from commit 5d2405227a9eaea48e8cc95756a06d407b11f141)
Signed-off-by: Peter Kalauskas <peskal@google.com>
Bug: 112488418
Change-Id: I4b63e96457f17cf455591e8f35058dacd7aa9004
2018-08-23 12:00:19 -07:00
..
842 crypto: 842 - Add CRC and validation support 2015-10-14 22:23:17 +08:00
fonts fonts: Add 6x10 font 2014-10-09 11:35:48 +03:00
lz4 lib: lz4: fixed zram with lz4 on big endian machines 2016-05-04 14:48:41 -07:00
lzo lzo: check for length overrun in variable length encoding. 2014-09-28 11:08:01 +02:00
mpi lib/mpi: Fix umul_ppmm() for MIPS64r6 2018-03-03 10:19:42 +01:00
raid6 md/raid6: delta syndrome for ARM NEON 2015-08-31 19:29:05 +02:00
reed_solomon
xz lib/xz: enable all filters by default in Kconfig 2014-06-04 16:54:18 -07:00
zlib_deflate zlib_deflate/deftree: remove bi_reverse() 2015-09-10 13:29:01 -07:00
zlib_inflate zlib: clean up some dead code 2014-08-06 18:01:24 -07:00
.gitignore
argv_split.c argv_split(): teach it to handle mutable strings 2013-04-29 18:28:19 -07:00
asn1_decoder.c ASN.1: check for error from ASN1_OP_END__ACT actions 2017-12-16 10:33:48 +01:00
assoc_array.c assoc_array: Fix a buggy node-splitting case 2017-11-02 09:40:49 +01:00
atomic64.c atomic: Provide atomic_{or,xor,and} 2015-07-27 14:06:24 +02:00
atomic64_test.c x86/cpufeature: Carve out X86_FEATURE_* 2018-07-17 11:31:44 +02:00
audit.c syscalls: implement execveat() system call 2014-12-13 12:42:51 -08:00
bcd.c
bch.c
bitmap.c lib/bitmap.c: bitmap_parselist can accept string with whitespaces on head or tail 2015-09-10 13:29:01 -07:00
bitrev.c ARM: 8187/1: add CONFIG_HAVE_ARCH_BITREVERSE to support rbit instruction 2014-12-22 16:43:06 +00:00
bsearch.c
btree.c treewide: Remove old email address 2015-11-23 09:44:58 +01:00
bug.c module: Sanitize RCU usage and locking 2015-05-28 11:31:52 +09:30
build_OID_registry X.509: do not emit any informational output 2013-06-19 17:54:06 +02:00
bust_spinlocks.c printk: Provide a wake_up_klogd() off-case 2013-03-22 16:41:20 -07:00
check_signature.c
checksum.c lib/checksum.c: fix build for generic csum_tcpudp_nofold 2015-01-29 11:57:38 -08:00
clz_ctz.c lib/clz_ctz.c: add prototype declarations in lib/clz_ctz.c 2014-04-03 16:21:12 -07:00
clz_tab.c
cmdline.c lib/cmdline.c: fix get_options() overflow while parsing ranges 2017-06-29 12:48:51 +02:00
compat_audit.c audit: Add generic compat syscall support 2014-03-20 10:11:35 -04:00
cordic.c
cpu-notifier-error-inject.c
cpu_rmap.c sched/topology: Rename topology_thread_cpumask() to topology_sibling_cpumask() 2015-05-27 15:22:15 +02:00
cpumask.c revert "cpumask: don't perform while loop in cpumask_next_and()" 2015-06-18 17:00:23 -10:00
crc-ccitt.c
crc-itu-t.c lib: crc-itu-t.[ch] fix 0x0x prefix in integer constants 2015-05-26 15:26:43 +02:00
crc-t10dif.c lib: introduce crc_t10dif_update() 2015-05-30 22:42:24 -07:00
crc7.c lib/crc7: Shift crc7() output left 1 bit 2014-05-16 14:26:52 -04:00
crc8.c
crc16.c
crc32.c lib: crc32: Add some additional __pure annotations 2014-06-25 16:04:00 -07:00
crc32defs.h
ctype.c
debug_info.c kbuild: include core debug info when DEBUG_INFO_REDUCED 2015-06-11 15:08:32 +02:00
debug_locks.c mutex: Add support for wound/wait style locks 2013-06-26 12:10:56 +02:00
debugobjects.c lib/debugobjects.c: convert printk(KERN_DEBUG to pr_debug 2014-06-04 16:53:53 -07:00
dec_and_lock.c
decompress.c lib/decompress: set the compressor name to NULL on error 2015-07-17 16:39:54 -07:00
decompress_bunzip2.c lib/decompressors: use real out buf size for gunzip with kernel 2015-09-10 13:29:01 -07:00
decompress_inflate.c lib/decompressors: use real out buf size for gunzip with kernel 2015-09-10 13:29:01 -07:00
decompress_unlz4.c lib/decompressors: use real out buf size for gunzip with kernel 2015-09-10 13:29:01 -07:00
decompress_unlzma.c lib/decompress_unlzma: Do a NULL check for pointer 2015-09-10 13:29:01 -07:00
decompress_unlzo.c lib/decompressors: use real out buf size for gunzip with kernel 2015-09-10 13:29:01 -07:00
decompress_unxz.c lib/decompressors: use real out buf size for gunzip with kernel 2015-09-10 13:29:01 -07:00
devres.c devres: fix a for loop bounds check 2015-10-05 04:49:54 +01:00
digsig.c lib/digsig: fix dereference of NULL user_key_payload 2017-10-27 10:23:17 +02:00
div64.c remove abs64() 2015-11-09 15:11:24 -08:00
dma-debug.c dma-debug: avoid spinlock recursion when disabling dma-debug 2016-06-07 18:14:37 -07:00
dump_stack.c dump_stack: avoid potential deadlocks 2016-02-25 12:01:23 -08:00
dynamic_debug.c dynamic-debug-howto: fix optional/omitted ending line number to be LARGE instead of 0 2017-12-16 10:33:54 +01:00
dynamic_queue_limits.c lib/dynamic_queue_limits.c: simplify includes 2015-02-12 18:54:15 -08:00
earlycpio.c earlycpio.c: Fix the confusing comment of find_cpio_data(). 2013-08-14 23:24:01 +02:00
extable.c UPSTREAM: extable: add support for relative extables to search and sort routines 2016-09-22 13:38:22 -07:00
fault-inject.c fault-inject: fix inverted interval/probability values in printk 2015-10-23 17:55:10 +09:00
fdt.c
fdt_empty_tree.c lib: add fdt_empty_tree.c 2014-04-30 19:49:37 +01:00
fdt_ro.c
fdt_rw.c
fdt_strerror.c
fdt_sw.c
fdt_wip.c
find_bit.c lib: rename lib/find_next_bit.c to lib/find_bit.c 2015-04-17 09:03:54 -04:00
flex_array.c reciprocal_divide: update/correction of the algorithm 2014-01-21 23:17:20 -08:00
flex_proportions.c proportions: add @gfp to init functions 2014-09-08 09:51:30 +09:00
gcd.c
gen_crc32table.c lib: crc32: constify crc32 lookup table 2015-02-13 21:21:35 -08:00
genalloc.c lib/genalloc.c: make the avail variable an atomic_long_t 2017-12-16 10:33:54 +01:00
glob.c lib/glob.c: add CONFIG_GLOB_SELFTEST 2014-08-06 18:01:25 -07:00
halfmd4.c lib/halfmd4.c: use rol32 inline function in the ROUND macro 2015-11-06 17:50:42 -08:00
hexdump.c lib/hexdump.c: truncate output in case of overflow 2015-11-06 17:50:42 -08:00
hweight.c x86/hweight: Get rid of the special calling convention 2018-04-24 09:32:04 +02:00
idr.c mm, page_alloc: distinguish between being unable to sleep, unwilling to sleep and avoiding waking kswapd 2015-11-06 17:50:42 -08:00
inflate.c
int_sqrt.c lib/int_sqrt.c: optimize square root algorithm 2013-04-29 18:28:19 -07:00
interval_tree.c lib/interval_tree.c: simplify includes 2015-02-12 18:54:15 -08:00
interval_tree_test.c lib: Export interval_tree 2014-05-05 09:09:14 +02:00
iomap.c Kconfig: rename HAS_IOPORT to HAS_IOPORT_MAP 2014-04-07 16:36:11 -07:00
iomap_copy.c
iommu-common.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/sparc 2015-11-05 16:34:48 -08:00
iommu-helper.c
ioremap.c ioremap: Update pgtable free interfaces with addr 2018-08-17 20:56:45 +02:00
iov_iter.c fix iov_iter_fault_in_readable() 2016-09-24 10:07:43 +02:00
irq_regs.c
is_single_threaded.c lib/is_single_threaded.c: change current_is_single_threaded() to use for_each_thread() 2015-11-06 17:50:42 -08:00
jedec_ddr_data.c
kasprintf.c lib/kasprintf.c: introduce kvasprintf_const 2015-11-06 17:50:42 -08:00
Kconfig UPSTREAM: lib: Add xxhash module 2018-08-23 12:00:19 -07:00
Kconfig.debug This is the 4.4.118 stable release 2018-02-26 09:24:57 +01:00
Kconfig.kasan BACKPORT: mm, kasan: switch SLUB to stackdepot, enable memory quarantine for SLUB 2017-12-14 08:20:37 -08:00
Kconfig.kgdb kdb: Allow access to sensitive commands to be restricted by default 2014-11-11 09:31:52 -06:00
Kconfig.kmemcheck
kfifo.c kfifo: use BUG_ON 2014-08-08 15:57:25 -07:00
klist.c klist: fix starting point removed bug in klist iterators 2016-02-25 12:01:16 -08:00
kobject.c kobject: don't use WARN for registration failures 2018-05-02 07:53:42 -07:00
kobject_uevent.c lib/kobject_uevent.c: remove redundant include 2015-02-12 18:54:15 -08:00
kstrtox.c lib: add "on"/"off" support to kstrtobool 2016-10-28 03:01:31 -04:00
kstrtox.h
lcm.c block: fix blk_stack_limits() regression due to lcm() change 2015-03-31 09:45:50 -06:00
libcrc32c.c crypto: Work around deallocated stack frame reference gcc bug on sparc. 2017-10-12 09:35:25 -07:00
list_debug.c
list_sort.c lib/list_sort: use late_initcall to hook in self tests 2015-06-16 14:12:35 -04:00
llist.c lib/llist.c: fix data race in llist_del_first 2015-11-06 17:50:42 -08:00
locking-selftest-hardirq.h
locking-selftest-mutex.h
locking-selftest-rlock-hardirq.h
locking-selftest-rlock-softirq.h
locking-selftest-rlock.h
locking-selftest-rsem.h
locking-selftest-softirq.h
locking-selftest-spin-hardirq.h
locking-selftest-spin-softirq.h
locking-selftest-spin.h
locking-selftest-wlock-hardirq.h
locking-selftest-wlock-softirq.h
locking-selftest-wlock.h
locking-selftest-wsem.h
locking-selftest.c locking/lockdep: Revert qrwlock recusive stuff 2014-10-03 06:09:30 +02:00
lockref.c locking/lockref: Remove homebrew cmpxchg64_relaxed() macro definition 2015-08-12 11:59:04 +02:00
lru_cache.c lru_cache: remove use of seq_printf return value 2015-04-15 16:35:25 -07:00
Makefile UPSTREAM: lib: Add xxhash module 2018-08-23 12:00:19 -07:00
md5.c lib/md5.c: simplify include 2015-02-12 18:54:15 -08:00
memory-notifier-error-inject.c
memweight.c
net_utils.c mac_pton: Use bool not int return 2014-06-25 17:45:43 -07:00
nlattr.c netlink: pad nla_memcpy dest buffer with zeroes 2015-03-31 14:07:24 -04:00
nmi_backtrace.c ARM: 8439/1: Fix backtrace generation when IPI is masked 2015-10-03 16:40:51 +01:00
notifier-error-inject.c mode_t, whack-a-mole at 11... 2013-04-09 14:13:05 -04:00
notifier-error-inject.h
of-reconfig-notifier-error-inject.c powerpc+of: Rename and fix OF reconfig notifier error inject module 2012-12-14 10:32:52 +11:00
oid_registry.c 509: fix printing uninitialized stack memory when OID is empty 2018-02-25 11:03:40 +01:00
once.c once: make helper generic for calling functions once 2015-10-08 05:26:36 -07:00
parser.c lib/parser.c: put EXPORT_SYMBOLs in the conventional place 2014-01-23 16:36:55 -08:00
pci_iomap.c libnvdimm for 4.3: 2015-09-08 14:35:59 -07:00
percpu-refcount.c percpu_ref: make INIT_ATOMIC and switch_to_atomic() sticky 2014-09-24 13:31:50 -04:00
percpu_counter.c percpu_counter: batch size aware __percpu_counter_compare() 2015-05-29 07:39:34 +10:00
percpu_ida.c mm, page_alloc: rename __GFP_WAIT to __GFP_RECLAIM 2015-11-06 17:50:42 -08:00
percpu_test.c percpu: add test module for various percpu operations 2013-11-13 12:09:11 +09:00
plist.c lib/plist.c: remove redundant include 2015-02-12 18:54:16 -08:00
pm-notifier-error-inject.c
proportions.c treewide: Remove old email address 2015-11-23 09:44:58 +01:00
radix-tree.c radix-tree: fix race in gang lookup 2016-02-25 12:01:23 -08:00
random32.c random32: add prandom_init_once helper for own rngs 2015-10-08 05:26:38 -07:00
ratelimit.c
rational.c
rbtree.c rbtree: Make lockless searches non-fatal 2015-05-28 11:32:04 +09:30
rbtree_test.c rbtree/test: test rbtree_postorder_for_each_entry_safe() 2014-01-23 16:37:03 -08:00
reciprocal_div.c reciprocal_divide: update/correction of the algorithm 2014-01-21 23:17:20 -08:00
rhashtable.c lib/rhashtable: consider param->min_size when setting initial table size 2018-07-25 10:18:16 +02:00
scatterlist.c scatterlist: allow limited chaining without ARCH_HAS_SG_CHAIN 2015-08-17 08:12:51 -06:00
seq_buf.c seq_buf: Fix seq_buf_bprintf() truncation 2015-03-04 23:40:19 -05:00
sg_split.c lib: scatterlist: add sg splitting function 2015-08-24 14:28:01 -06:00
sha1.c lib: EXPORT_SYMBOL sha_init 2015-03-23 22:12:08 -04:00
show_mem.c lib/show_mem.c: correct reserved memory calculation 2015-09-08 15:35:28 -07:00
smp_processor_id.c percpu: add preemption checks to __this_cpu ops 2014-04-07 16:36:14 -07:00
sort.c lib/sort: Add 64 bit swap function 2015-06-25 17:00:40 -07:00
stackdepot.c UPSTREAM: lib/stackdepot: export save/fetch stack for drivers 2017-12-14 08:22:26 -08:00
stmp_device.c lib/stmp_device.c: replace module.h include 2015-02-12 18:54:16 -08:00
string.c lib: move strtobool() to kstrtobool() 2016-10-28 03:01:30 -04:00
string_helpers.c string_helpers: fix precision loss for some inputs 2016-02-25 12:01:21 -08:00
strncpy_from_user.c BACKPORT: lib: harden strncpy_from_user 2016-10-18 11:08:46 -07:00
strnlen_user.c UPSTREAM: unsafe_[get|put]_user: change interface to use a error target label 2016-09-06 15:52:54 +00:00
swiotlb.c swiotlb: ensure that page-sized mappings are page-aligned 2017-07-05 14:37:20 +02:00
syscall.c lib/syscall.c: unexport task_current_syscall() 2014-04-03 16:21:06 -07:00
test-hexdump.c hexdump: Make test data really const 2015-06-25 17:00:40 -07:00
test-kstrtox.c kstrto*: accept "-0" for signed conversion 2015-09-10 13:29:01 -07:00
test-string_helpers.c lib/test-string_helpers.c: fix and improve string_get_size() tests 2016-05-11 11:21:26 +02:00
test_bpf.c bpf: fix selftests/bpf test_kmod.sh failure when CONFIG_BPF_JIT_ALWAYS_ON=y 2018-05-30 07:48:56 +02:00
test_firmware.c test: firmware_class: report errors properly on failure 2017-11-15 17:13:13 +01:00
test_kasan.c BACKPORT: kasan: report only the first error by default 2017-12-14 08:25:14 -08:00
test_module.c test: add minimal module for verification testing 2014-01-23 16:36:57 -08:00
test_printf.c test_printf: test printf family at runtime 2015-11-06 17:50:42 -08:00
test_rhashtable.c rhashtable-test: extend to test concurrency 2015-08-17 14:33:47 -07:00
test_static_key_base.c locking/static_keys: Provide a selftest 2015-08-03 11:51:12 +02:00
test_static_keys.c locking/static_keys: Make verify_keys() static 2015-08-05 09:53:40 +02:00
test_user_copy.c usercopy: Adjust tests to deal with SMAP/PAN 2017-06-14 13:16:27 +02:00
textsearch.c lib/textsearch.c: remove textsearch_put reference from comments 2014-10-14 02:18:14 +02:00
timerqueue.c timerqueue: Let timerqueue_add/del return information 2015-04-22 17:06:49 +02:00
ts_bm.c
ts_fsm.c
ts_kmp.c
ucs2_string.c lib/ucs2_string: Correct ucs2 -> utf8 conversion 2016-03-03 15:07:09 -08:00
usercopy.c Kconfig: consolidate CONFIG_DEBUG_STRICT_USER_COPY_CHECKS 2013-04-30 17:04:09 -07:00
uuid.c uuid: use prandom_bytes() 2013-04-29 18:28:42 -07:00
vsprintf.c lib/vsprintf: Remove atomic-unsafe support for %pCr 2018-07-03 11:21:27 +02:00
xxhash.c UPSTREAM: lib: Add xxhash module 2018-08-23 12:00:19 -07:00