Merge android-4.4.163 (0ca3fca) into msm-4.4
* refs/heads/tmp-0ca3fca
Linux 4.4.163
x86/time: Correct the attribute on jiffies' definition
l2tp: hold tunnel socket when handling control frames in l2tp_ip and l2tp_ip6
cpuidle: Do not access cpuidle_devices when !CONFIG_CPU_IDLE
x86/percpu: Fix this_cpu_read()
sched/fair: Fix throttle_list starvation with low CFS quota
Input: elan_i2c - add ACPI ID for Lenovo IdeaPad 330-15IGM
USB: fix the usbfs flag sanitization for control transfers
usb: gadget: storage: Fix Spectre v1 vulnerability
cdc-acm: correct counting of UART states in serial state notification
IB/ucm: Fix Spectre v1 vulnerability
RDMA/ucma: Fix Spectre v1 vulnerability
ptp: fix Spectre v1 vulnerability
cachefiles: fix the race between cachefiles_bury_object() and rmdir(2)
ahci: don't ignore result code of ahci_reset_controller()
crypto: shash - Fix a sleep-in-atomic bug in shash_setkey_unaligned
mremap: properly flush TLB before releasing the page
rtnetlink: Disallow FDB configuration for non-Ethernet device
vhost: Fix Spectre V1 vulnerability
net: drop skb on failure in ip_check_defrag()
sctp: fix race on sctp_id2asoc
r8169: fix NAPI handling under high load
net: stmmac: Fix stmmac_mdio_reset() when building stmmac as modules
net: socket: fix a missing-check bug
net: sched: gred: pass the right attribute to gred_change_table_def()
net/ipv6: Fix index counter for unicast addresses in in6_dump_addrs
ipv6/ndisc: Preserve IPv6 control buffer if protocol error handlers are called
ipv6: mcast: fix a use-after-free in inet6_mc_check
net: bridge: remove ipv6 zero address check in mcast queries
bridge: do not add port to router list when receives query with source 0.0.0.0
perf tools: Disable parallelism for 'make clean'
mtd: spi-nor: Add support for is25wp series chips
fs/fat/fatent.c: add cond_resched() to fat_count_free_clusters()
ARM: dts: imx53-qsb: disable 1.2GHz OPP
MIPS: DEC: Fix an int-handler.S CPU_DADDI_WORKAROUNDS regression
igb: Remove superfluous reset to PHY and page 0 selection
MIPS: microMIPS: Fix decoding of swsp16 instruction
scsi: aacraid: Fix typo in blink status
bonding: avoid defaulting hard_header_len to ETH_HLEN on slave removal
PM / devfreq: tegra: fix error return code in tegra_devfreq_probe()
ASoC: spear: fix error return code in spdif_in_probe()
spi: xlp: fix error return code in xlp_spi_probe()
spi/bcm63xx: fix error return code in bcm63xx_spi_probe()
MIPS: Handle non word sized instructions when examining frame
spi/bcm63xx-hspi: fix error return code in bcm63xx_hsspi_probe()
usb: dwc3: omap: fix error return code in dwc3_omap_probe()
usb: ehci-omap: fix error return code in ehci_hcd_omap_probe()
usb: imx21-hcd: fix error return code in imx21_probe()
gpio: msic: fix error return code in platform_msic_gpio_probe()
sparc64: Fix exception handling in UltraSPARC-III memcpy.
gpu: host1x: fix error return code in host1x_probe()
sparc64 mm: Fix more TSB sizing issues
video: fbdev: pxa3xx_gcu: fix error return code in pxa3xx_gcu_probe()
tty: serial: sprd: fix error return code in sprd_probe()
l2tp: hold socket before dropping lock in l2tp_ip{, 6}_recv()
brcmfmac: Fix glom_skb leak in brcmf_sdiod_recv_chain
gro: Allow tunnel stacking in the case of FOU/GUE
vti6: flush x-netns xfrm cache when vti interface is removed
ALSA: timer: Fix zero-division by continue of uninitialized instance
ixgbe: Correct X550EM_x revision check
ixgbe: fix RSS limit for X550
net/mlx5e: Correctly handle RSS indirection table when changing number of channels
net/mlx5e: Fix LRO modify
ixgbevf: Fix handling of NAPI budget when multiple queues are enabled per vector
fuse: Dont call set_page_dirty_lock() for ITER_BVEC pages for async_dio
drm/nouveau/fbcon: fix oops without fbdev emulation
bpf: generally move prog destruction to RCU deferral
usb-storage: fix bogus hardware error messages for ATA pass-thru devices
sch_red: update backlog as well
sparc/pci: Refactor dev_archdata initialization into pci_init_dev_archdata
scsi: Add STARGET_CREATED_REMOVE state to scsi_target_state
xfrm: Clear sk_dst_cache when applying per-socket policy.
arm64: Fix potential race with hardware DBM in ptep_set_access_flags()
CIFS: handle guest access errors to Windows shares
ASoC: wm8940: Enable cache usage to fix crashes on resume
ASoC: ak4613: Enable cache usage to fix crashes on resume
MIPS: Fix FCSR Cause bit handling for correct SIGFPE issue
usbvision: revert commit 588afcc1
perf/core: Don't leak event in the syscall error path
aacraid: Start adapter after updating number of MSIX vectors
x86/PCI: Mark Broadwell-EP Home Agent 1 as having non-compliant BARs
tpm: fix: return rc when devm_add_action() fails
thermal: allow u8500-thermal driver to be a module
thermal: allow spear-thermal driver to be a module
btrfs: don't create or leak aliased root while cleaning up orphans
sched/cgroup: Fix cgroup entity load tracking tear-down
um: Avoid longjmp/setjmp symbol clashes with libpthread.a
ipv6: orphan skbs in reassembly unit
net/mlx4_en: Resolve dividing by zero in 32-bit system
af_iucv: Move sockaddr length checks to before accessing sa_family in bind and connect handlers
radix-tree: fix radix_tree_iter_retry() for tagged iterators.
x86/mm/pat: Prevent hang during boot when mapping pages
ARM: dts: apq8064: add ahci ports-implemented mask
tracing: Skip more functions when doing stack tracing of events
ser_gigaset: use container_of() instead of detour
net: drop write-only stack variable
ipv6: suppress sparse warnings in IP6_ECN_set_ce()
KEYS: put keyring if install_session_keyring_to_cred() fails
net: cxgb3_main: fix a missing-check bug
perf/ring_buffer: Prevent concurent ring buffer access
smsc95xx: Check for Wake-on-LAN modes
smsc75xx: Check for Wake-on-LAN modes
r8152: Check for supported Wake-on-LAN Modes
sr9800: Check for supported Wake-on-LAN modes
lan78xx: Check for supported Wake-on-LAN modes
ax88179_178a: Check for supported Wake-on-LAN modes
asix: Check for supported Wake-on-LAN modes
pxa168fb: prepare the clock
Bluetooth: SMP: fix crash in unpairing
mac80211_hwsim: do not omit multicast announce of first added radio
xfrm: validate template mode
ARM: 8799/1: mm: fix pci_ioremap_io() offset check
cfg80211: reg: Init wiphy_idx in regulatory_hint_core()
mac80211: Always report TX status
xfrm6: call kfree_skb when skb is toobig
xfrm: Validate address prefix lengths in the xfrm selector.
BACKPORT: xfrm: Allow Output Mark to be Updated Using UPDSA
ANDROID: sdcardfs: Add option to drop unused dentries
f2fs: guarantee journalled quota data by checkpoint
f2fs: cleanup dirty pages if recover failed
f2fs: fix data corruption issue with hardware encryption
f2fs: fix to recover inode->i_flags of inode block during POR
f2fs: spread f2fs_set_inode_flags()
f2fs: fix to spread clear_cold_data()
Revert "f2fs: fix to clear PG_checked flag in set_page_dirty()"
f2fs: account read IOs and use IO counts for is_idle
f2fs: fix to account IO correctly for cgroup writeback
f2fs: fix to account IO correctly
f2fs: remove request_list check in is_idle()
f2fs: allow to mount, if quota is failed
f2fs: update REQ_TIME in f2fs_cross_rename()
f2fs: do not update REQ_TIME in case of error conditions
f2fs: remove unneeded disable_nat_bits()
f2fs: remove unused sbi->trigger_ssr_threshold
f2fs: shrink sbi->sb_lock coverage in set_file_temperature()
f2fs: fix to recover cold bit of inode block during POR
f2fs: submit cached bio to avoid endless PageWriteback
f2fs: checkpoint disabling
f2fs: clear PageError on the read path
f2fs: allow out-place-update for direct IO in LFS mode
f2fs: refactor ->page_mkwrite() flow
Revert: "f2fs: check last page index in cached bio to decide submission"
f2fs: support superblock checksum
f2fs: add to account skip count of background GC
f2fs: add to account meta IO
f2fs: keep lazytime on remount
f2fs: fix missing up_read
f2fs: return correct errno in f2fs_gc
f2fs: avoid f2fs_bug_on if f2fs_get_meta_page_nofail got EIO
f2fs: mark inode dirty explicitly in recover_inode()
f2fs: fix to recover inode's crtime during POR
f2fs: fix to recover inode's i_gc_failures during POR
f2fs: fix to recover inode's i_flags during POR
f2fs: fix to recover inode's project id during POR
f2fs: update i_size after DIO completion
f2fs: report ENOENT correctly in f2fs_rename
f2fs: fix remount problem of option io_bits
f2fs: fix to recover inode's uid/gid during POR
f2fs: avoid infinite loop in f2fs_alloc_nid
f2fs: add new idle interval timing for discard and gc paths
f2fs: split IO error injection according to RW
f2fs: add SPDX license identifiers
f2fs: surround fault_injection related option parsing using CONFIG_F2FS_FAULT_INJECTION
f2fs: avoid sleeping under spin_lock
f2fs: plug readahead IO in readdir()
f2fs: fix to do sanity check with current segment number
f2fs: fix memory leak of percpu counter in fill_super()
f2fs: fix memory leak of write_io in fill_super()
f2fs: cache NULL when both default_acl and acl are NULL
f2fs: fix to flush all dirty inodes recovered in readonly fs
f2fs: report error if quota off error during umount
f2fs: submit bio after shutdown
f2fs: avoid wrong decrypted data from disk
Revert "f2fs: use printk_ratelimited for f2fs_msg"
f2fs: fix unnecessary periodic wakeup of discard thread when dev is busy
f2fs: fix to avoid NULL pointer dereference on se->discard_map
f2fs: add additional sanity check in f2fs_acl_from_disk()
Revert "BACKPORT, FROMLIST: fscrypt: add Speck128/256 support"
Build fix for 076c36fce1.
Revert "BACKPORT, FROMGIT: crypto: speck - add support for the Speck block cipher"
Revert "FROMGIT: crypto: speck - export common helpers"
Revert "BACKPORT, FROMGIT: crypto: arm/speck - add NEON-accelerated implementation of Speck-XTS"
Revert "BACKPORT, FROMGIT: crypto: speck - add test vectors for Speck128-XTS"
Revert "BACKPORT, FROMGIT: crypto: speck - add test vectors for Speck64-XTS"
Revert "BACKPORT, FROMLIST: crypto: arm64/speck - add NEON-accelerated implementation of Speck-XTS"
Revert "fscrypt: add Speck128/256 support"
UPSTREAM: loop: Add LOOP_SET_BLOCK_SIZE in compat ioctl
BACKPORT: block/loop: set hw_sectors
UPSTREAM: loop: add ioctl for changing logical block size
Conflicts:
fs/ext4/crypto.c
fs/ext4/ext4.h
Change-Id: I8cb2f70b27906879f8e8fdd90e67f438e39701b8
Signed-off-by: Srinivasarao P <spathi@codeaurora.org>
This commit is contained in:
Srinivasarao P
commit
b28608bd14
190 changed files with 2165 additions and 4842 deletions
|
|
@ -121,7 +121,22 @@ What: /sys/fs/f2fs/<disk>/idle_interval
|
|||
Date: January 2016
|
||||
Contact: "Jaegeuk Kim" <jaegeuk@kernel.org>
|
||||
Description:
|
||||
Controls the idle timing.
|
||||
Controls the idle timing for all paths other than
|
||||
discard and gc path.
|
||||
|
||||
What: /sys/fs/f2fs/<disk>/discard_idle_interval
|
||||
Date: September 2018
|
||||
Contact: "Chao Yu" <yuchao0@huawei.com>
|
||||
Contact: "Sahitya Tummala" <stummala@codeaurora.org>
|
||||
Description:
|
||||
Controls the idle timing for discard path.
|
||||
|
||||
What: /sys/fs/f2fs/<disk>/gc_idle_interval
|
||||
Date: September 2018
|
||||
Contact: "Chao Yu" <yuchao0@huawei.com>
|
||||
Contact: "Sahitya Tummala" <stummala@codeaurora.org>
|
||||
Description:
|
||||
Controls the idle timing for gc path.
|
||||
|
||||
What: /sys/fs/f2fs/<disk>/iostat_enable
|
||||
Date: August 2017
|
||||
|
|
|
|||
|
|
@ -172,9 +172,10 @@ fault_type=%d Support configuring fault injection type, should be
|
|||
FAULT_DIR_DEPTH 0x000000100
|
||||
FAULT_EVICT_INODE 0x000000200
|
||||
FAULT_TRUNCATE 0x000000400
|
||||
FAULT_IO 0x000000800
|
||||
FAULT_READ_IO 0x000000800
|
||||
FAULT_CHECKPOINT 0x000001000
|
||||
FAULT_DISCARD 0x000002000
|
||||
FAULT_WRITE_IO 0x000004000
|
||||
mode=%s Control block allocation mode which supports "adaptive"
|
||||
and "lfs". In "lfs" mode, there should be no random
|
||||
writes towards main area.
|
||||
|
|
@ -211,6 +212,11 @@ fsync_mode=%s Control the policy of fsync. Currently supports "posix",
|
|||
non-atomic files likewise "nobarrier" mount option.
|
||||
test_dummy_encryption Enable dummy encryption, which provides a fake fscrypt
|
||||
context. The fake fscrypt context is used by xfstests.
|
||||
checkpoint=%s Set to "disable" to turn off checkpointing. Set to "enable"
|
||||
to reenable checkpointing. Is enabled by default. While
|
||||
disabled, any unmounting or unexpected shutdowns will cause
|
||||
the filesystem contents to appear as they did when the
|
||||
filesystem was mounted with that option.
|
||||
|
||||
================================================================================
|
||||
DEBUGFS ENTRIES
|
||||
|
|
|
|||
|
|
@ -1,626 +0,0 @@
|
|||
=====================================
|
||||
Filesystem-level encryption (fscrypt)
|
||||
=====================================
|
||||
|
||||
Introduction
|
||||
============
|
||||
|
||||
fscrypt is a library which filesystems can hook into to support
|
||||
transparent encryption of files and directories.
|
||||
|
||||
Note: "fscrypt" in this document refers to the kernel-level portion,
|
||||
implemented in ``fs/crypto/``, as opposed to the userspace tool
|
||||
`fscrypt <https://github.com/google/fscrypt>`_. This document only
|
||||
covers the kernel-level portion. For command-line examples of how to
|
||||
use encryption, see the documentation for the userspace tool `fscrypt
|
||||
<https://github.com/google/fscrypt>`_. Also, it is recommended to use
|
||||
the fscrypt userspace tool, or other existing userspace tools such as
|
||||
`fscryptctl <https://github.com/google/fscryptctl>`_ or `Android's key
|
||||
management system
|
||||
<https://source.android.com/security/encryption/file-based>`_, over
|
||||
using the kernel's API directly. Using existing tools reduces the
|
||||
chance of introducing your own security bugs. (Nevertheless, for
|
||||
completeness this documentation covers the kernel's API anyway.)
|
||||
|
||||
Unlike dm-crypt, fscrypt operates at the filesystem level rather than
|
||||
at the block device level. This allows it to encrypt different files
|
||||
with different keys and to have unencrypted files on the same
|
||||
filesystem. This is useful for multi-user systems where each user's
|
||||
data-at-rest needs to be cryptographically isolated from the others.
|
||||
However, except for filenames, fscrypt does not encrypt filesystem
|
||||
metadata.
|
||||
|
||||
Unlike eCryptfs, which is a stacked filesystem, fscrypt is integrated
|
||||
directly into supported filesystems --- currently ext4, F2FS, and
|
||||
UBIFS. This allows encrypted files to be read and written without
|
||||
caching both the decrypted and encrypted pages in the pagecache,
|
||||
thereby nearly halving the memory used and bringing it in line with
|
||||
unencrypted files. Similarly, half as many dentries and inodes are
|
||||
needed. eCryptfs also limits encrypted filenames to 143 bytes,
|
||||
causing application compatibility issues; fscrypt allows the full 255
|
||||
bytes (NAME_MAX). Finally, unlike eCryptfs, the fscrypt API can be
|
||||
used by unprivileged users, with no need to mount anything.
|
||||
|
||||
fscrypt does not support encrypting files in-place. Instead, it
|
||||
supports marking an empty directory as encrypted. Then, after
|
||||
userspace provides the key, all regular files, directories, and
|
||||
symbolic links created in that directory tree are transparently
|
||||
encrypted.
|
||||
|
||||
Threat model
|
||||
============
|
||||
|
||||
Offline attacks
|
||||
---------------
|
||||
|
||||
Provided that userspace chooses a strong encryption key, fscrypt
|
||||
protects the confidentiality of file contents and filenames in the
|
||||
event of a single point-in-time permanent offline compromise of the
|
||||
block device content. fscrypt does not protect the confidentiality of
|
||||
non-filename metadata, e.g. file sizes, file permissions, file
|
||||
timestamps, and extended attributes. Also, the existence and location
|
||||
of holes (unallocated blocks which logically contain all zeroes) in
|
||||
files is not protected.
|
||||
|
||||
fscrypt is not guaranteed to protect confidentiality or authenticity
|
||||
if an attacker is able to manipulate the filesystem offline prior to
|
||||
an authorized user later accessing the filesystem.
|
||||
|
||||
Online attacks
|
||||
--------------
|
||||
|
||||
fscrypt (and storage encryption in general) can only provide limited
|
||||
protection, if any at all, against online attacks. In detail:
|
||||
|
||||
fscrypt is only resistant to side-channel attacks, such as timing or
|
||||
electromagnetic attacks, to the extent that the underlying Linux
|
||||
Cryptographic API algorithms are. If a vulnerable algorithm is used,
|
||||
such as a table-based implementation of AES, it may be possible for an
|
||||
attacker to mount a side channel attack against the online system.
|
||||
Side channel attacks may also be mounted against applications
|
||||
consuming decrypted data.
|
||||
|
||||
After an encryption key has been provided, fscrypt is not designed to
|
||||
hide the plaintext file contents or filenames from other users on the
|
||||
same system, regardless of the visibility of the keyring key.
|
||||
Instead, existing access control mechanisms such as file mode bits,
|
||||
POSIX ACLs, LSMs, or mount namespaces should be used for this purpose.
|
||||
Also note that as long as the encryption keys are *anywhere* in
|
||||
memory, an online attacker can necessarily compromise them by mounting
|
||||
a physical attack or by exploiting any kernel security vulnerability
|
||||
which provides an arbitrary memory read primitive.
|
||||
|
||||
While it is ostensibly possible to "evict" keys from the system,
|
||||
recently accessed encrypted files will remain accessible at least
|
||||
until the filesystem is unmounted or the VFS caches are dropped, e.g.
|
||||
using ``echo 2 > /proc/sys/vm/drop_caches``. Even after that, if the
|
||||
RAM is compromised before being powered off, it will likely still be
|
||||
possible to recover portions of the plaintext file contents, if not
|
||||
some of the encryption keys as well. (Since Linux v4.12, all
|
||||
in-kernel keys related to fscrypt are sanitized before being freed.
|
||||
However, userspace would need to do its part as well.)
|
||||
|
||||
Currently, fscrypt does not prevent a user from maliciously providing
|
||||
an incorrect key for another user's existing encrypted files. A
|
||||
protection against this is planned.
|
||||
|
||||
Key hierarchy
|
||||
=============
|
||||
|
||||
Master Keys
|
||||
-----------
|
||||
|
||||
Each encrypted directory tree is protected by a *master key*. Master
|
||||
keys can be up to 64 bytes long, and must be at least as long as the
|
||||
greater of the key length needed by the contents and filenames
|
||||
encryption modes being used. For example, if AES-256-XTS is used for
|
||||
contents encryption, the master key must be 64 bytes (512 bits). Note
|
||||
that the XTS mode is defined to require a key twice as long as that
|
||||
required by the underlying block cipher.
|
||||
|
||||
To "unlock" an encrypted directory tree, userspace must provide the
|
||||
appropriate master key. There can be any number of master keys, each
|
||||
of which protects any number of directory trees on any number of
|
||||
filesystems.
|
||||
|
||||
Userspace should generate master keys either using a cryptographically
|
||||
secure random number generator, or by using a KDF (Key Derivation
|
||||
Function). Note that whenever a KDF is used to "stretch" a
|
||||
lower-entropy secret such as a passphrase, it is critical that a KDF
|
||||
designed for this purpose be used, such as scrypt, PBKDF2, or Argon2.
|
||||
|
||||
Per-file keys
|
||||
-------------
|
||||
|
||||
Master keys are not used to encrypt file contents or names directly.
|
||||
Instead, a unique key is derived for each encrypted file, including
|
||||
each regular file, directory, and symbolic link. This has several
|
||||
advantages:
|
||||
|
||||
- In cryptosystems, the same key material should never be used for
|
||||
different purposes. Using the master key as both an XTS key for
|
||||
contents encryption and as a CTS-CBC key for filenames encryption
|
||||
would violate this rule.
|
||||
- Per-file keys simplify the choice of IVs (Initialization Vectors)
|
||||
for contents encryption. Without per-file keys, to ensure IV
|
||||
uniqueness both the inode and logical block number would need to be
|
||||
encoded in the IVs. This would make it impossible to renumber
|
||||
inodes, which e.g. ``resize2fs`` can do when resizing an ext4
|
||||
filesystem. With per-file keys, it is sufficient to encode just the
|
||||
logical block number in the IVs.
|
||||
- Per-file keys strengthen the encryption of filenames, where IVs are
|
||||
reused out of necessity. With a unique key per directory, IV reuse
|
||||
is limited to within a single directory.
|
||||
- Per-file keys allow individual files to be securely erased simply by
|
||||
securely erasing their keys. (Not yet implemented.)
|
||||
|
||||
A KDF (Key Derivation Function) is used to derive per-file keys from
|
||||
the master key. This is done instead of wrapping a randomly-generated
|
||||
key for each file because it reduces the size of the encryption xattr,
|
||||
which for some filesystems makes the xattr more likely to fit in-line
|
||||
in the filesystem's inode table. With a KDF, only a 16-byte nonce is
|
||||
required --- long enough to make key reuse extremely unlikely. A
|
||||
wrapped key, on the other hand, would need to be up to 64 bytes ---
|
||||
the length of an AES-256-XTS key. Furthermore, currently there is no
|
||||
requirement to support unlocking a file with multiple alternative
|
||||
master keys or to support rotating master keys. Instead, the master
|
||||
keys may be wrapped in userspace, e.g. as done by the `fscrypt
|
||||
<https://github.com/google/fscrypt>`_ tool.
|
||||
|
||||
The current KDF encrypts the master key using the 16-byte nonce as an
|
||||
AES-128-ECB key. The output is used as the derived key. If the
|
||||
output is longer than needed, then it is truncated to the needed
|
||||
length. Truncation is the norm for directories and symlinks, since
|
||||
those use the CTS-CBC encryption mode which requires a key half as
|
||||
long as that required by the XTS encryption mode.
|
||||
|
||||
Note: this KDF meets the primary security requirement, which is to
|
||||
produce unique derived keys that preserve the entropy of the master
|
||||
key, assuming that the master key is already a good pseudorandom key.
|
||||
However, it is nonstandard and has some problems such as being
|
||||
reversible, so it is generally considered to be a mistake! It may be
|
||||
replaced with HKDF or another more standard KDF in the future.
|
||||
|
||||
Encryption modes and usage
|
||||
==========================
|
||||
|
||||
fscrypt allows one encryption mode to be specified for file contents
|
||||
and one encryption mode to be specified for filenames. Different
|
||||
directory trees are permitted to use different encryption modes.
|
||||
Currently, the following pairs of encryption modes are supported:
|
||||
|
||||
- AES-256-XTS for contents and AES-256-CTS-CBC for filenames
|
||||
- AES-128-CBC for contents and AES-128-CTS-CBC for filenames
|
||||
- Speck128/256-XTS for contents and Speck128/256-CTS-CBC for filenames
|
||||
|
||||
It is strongly recommended to use AES-256-XTS for contents encryption.
|
||||
AES-128-CBC was added only for low-powered embedded devices with
|
||||
crypto accelerators such as CAAM or CESA that do not support XTS.
|
||||
|
||||
Similarly, Speck128/256 support was only added for older or low-end
|
||||
CPUs which cannot do AES fast enough -- especially ARM CPUs which have
|
||||
NEON instructions but not the Cryptography Extensions -- and for which
|
||||
it would not otherwise be feasible to use encryption at all. It is
|
||||
not recommended to use Speck on CPUs that have AES instructions.
|
||||
Speck support is only available if it has been enabled in the crypto
|
||||
API via CONFIG_CRYPTO_SPECK. Also, on ARM platforms, to get
|
||||
acceptable performance CONFIG_CRYPTO_SPECK_NEON must be enabled.
|
||||
|
||||
New encryption modes can be added relatively easily, without changes
|
||||
to individual filesystems. However, authenticated encryption (AE)
|
||||
modes are not currently supported because of the difficulty of dealing
|
||||
with ciphertext expansion.
|
||||
|
||||
For file contents, each filesystem block is encrypted independently.
|
||||
Currently, only the case where the filesystem block size is equal to
|
||||
the system's page size (usually 4096 bytes) is supported. With the
|
||||
XTS mode of operation (recommended), the logical block number within
|
||||
the file is used as the IV. With the CBC mode of operation (not
|
||||
recommended), ESSIV is used; specifically, the IV for CBC is the
|
||||
logical block number encrypted with AES-256, where the AES-256 key is
|
||||
the SHA-256 hash of the inode's data encryption key.
|
||||
|
||||
For filenames, the full filename is encrypted at once. Because of the
|
||||
requirements to retain support for efficient directory lookups and
|
||||
filenames of up to 255 bytes, a constant initialization vector (IV) is
|
||||
used. However, each encrypted directory uses a unique key, which
|
||||
limits IV reuse to within a single directory. Note that IV reuse in
|
||||
the context of CTS-CBC encryption means that when the original
|
||||
filenames share a common prefix at least as long as the cipher block
|
||||
size (16 bytes for AES), the corresponding encrypted filenames will
|
||||
also share a common prefix. This is undesirable; it may be fixed in
|
||||
the future by switching to an encryption mode that is a strong
|
||||
pseudorandom permutation on arbitrary-length messages, e.g. the HEH
|
||||
(Hash-Encrypt-Hash) mode.
|
||||
|
||||
Since filenames are encrypted with the CTS-CBC mode of operation, the
|
||||
plaintext and ciphertext filenames need not be multiples of the AES
|
||||
block size, i.e. 16 bytes. However, the minimum size that can be
|
||||
encrypted is 16 bytes, so shorter filenames are NUL-padded to 16 bytes
|
||||
before being encrypted. In addition, to reduce leakage of filename
|
||||
lengths via their ciphertexts, all filenames are NUL-padded to the
|
||||
next 4, 8, 16, or 32-byte boundary (configurable). 32 is recommended
|
||||
since this provides the best confidentiality, at the cost of making
|
||||
directory entries consume slightly more space. Note that since NUL
|
||||
(``\0``) is not otherwise a valid character in filenames, the padding
|
||||
will never produce duplicate plaintexts.
|
||||
|
||||
Symbolic link targets are considered a type of filename and are
|
||||
encrypted in the same way as filenames in directory entries. Each
|
||||
symlink also uses a unique key; hence, the hardcoded IV is not a
|
||||
problem for symlinks.
|
||||
|
||||
User API
|
||||
========
|
||||
|
||||
Setting an encryption policy
|
||||
----------------------------
|
||||
|
||||
The FS_IOC_SET_ENCRYPTION_POLICY ioctl sets an encryption policy on an
|
||||
empty directory or verifies that a directory or regular file already
|
||||
has the specified encryption policy. It takes in a pointer to a
|
||||
:c:type:`struct fscrypt_policy`, defined as follows::
|
||||
|
||||
#define FS_KEY_DESCRIPTOR_SIZE 8
|
||||
|
||||
struct fscrypt_policy {
|
||||
__u8 version;
|
||||
__u8 contents_encryption_mode;
|
||||
__u8 filenames_encryption_mode;
|
||||
__u8 flags;
|
||||
__u8 master_key_descriptor[FS_KEY_DESCRIPTOR_SIZE];
|
||||
};
|
||||
|
||||
This structure must be initialized as follows:
|
||||
|
||||
- ``version`` must be 0.
|
||||
|
||||
- ``contents_encryption_mode`` and ``filenames_encryption_mode`` must
|
||||
be set to constants from ``<linux/fs.h>`` which identify the
|
||||
encryption modes to use. If unsure, use
|
||||
FS_ENCRYPTION_MODE_AES_256_XTS (1) for ``contents_encryption_mode``
|
||||
and FS_ENCRYPTION_MODE_AES_256_CTS (4) for
|
||||
``filenames_encryption_mode``.
|
||||
|
||||
- ``flags`` must be set to a value from ``<linux/fs.h>`` which
|
||||
identifies the amount of NUL-padding to use when encrypting
|
||||
filenames. If unsure, use FS_POLICY_FLAGS_PAD_32 (0x3).
|
||||
|
||||
- ``master_key_descriptor`` specifies how to find the master key in
|
||||
the keyring; see `Adding keys`_. It is up to userspace to choose a
|
||||
unique ``master_key_descriptor`` for each master key. The e4crypt
|
||||
and fscrypt tools use the first 8 bytes of
|
||||
``SHA-512(SHA-512(master_key))``, but this particular scheme is not
|
||||
required. Also, the master key need not be in the keyring yet when
|
||||
FS_IOC_SET_ENCRYPTION_POLICY is executed. However, it must be added
|
||||
before any files can be created in the encrypted directory.
|
||||
|
||||
If the file is not yet encrypted, then FS_IOC_SET_ENCRYPTION_POLICY
|
||||
verifies that the file is an empty directory. If so, the specified
|
||||
encryption policy is assigned to the directory, turning it into an
|
||||
encrypted directory. After that, and after providing the
|
||||
corresponding master key as described in `Adding keys`_, all regular
|
||||
files, directories (recursively), and symlinks created in the
|
||||
directory will be encrypted, inheriting the same encryption policy.
|
||||
The filenames in the directory's entries will be encrypted as well.
|
||||
|
||||
Alternatively, if the file is already encrypted, then
|
||||
FS_IOC_SET_ENCRYPTION_POLICY validates that the specified encryption
|
||||
policy exactly matches the actual one. If they match, then the ioctl
|
||||
returns 0. Otherwise, it fails with EEXIST. This works on both
|
||||
regular files and directories, including nonempty directories.
|
||||
|
||||
Note that the ext4 filesystem does not allow the root directory to be
|
||||
encrypted, even if it is empty. Users who want to encrypt an entire
|
||||
filesystem with one key should consider using dm-crypt instead.
|
||||
|
||||
FS_IOC_SET_ENCRYPTION_POLICY can fail with the following errors:
|
||||
|
||||
- ``EACCES``: the file is not owned by the process's uid, nor does the
|
||||
process have the CAP_FOWNER capability in a namespace with the file
|
||||
owner's uid mapped
|
||||
- ``EEXIST``: the file is already encrypted with an encryption policy
|
||||
different from the one specified
|
||||
- ``EINVAL``: an invalid encryption policy was specified (invalid
|
||||
version, mode(s), or flags)
|
||||
- ``ENOTDIR``: the file is unencrypted and is a regular file, not a
|
||||
directory
|
||||
- ``ENOTEMPTY``: the file is unencrypted and is a nonempty directory
|
||||
- ``ENOTTY``: this type of filesystem does not implement encryption
|
||||
- ``EOPNOTSUPP``: the kernel was not configured with encryption
|
||||
support for this filesystem, or the filesystem superblock has not
|
||||
had encryption enabled on it. (For example, to use encryption on an
|
||||
ext4 filesystem, CONFIG_EXT4_ENCRYPTION must be enabled in the
|
||||
kernel config, and the superblock must have had the "encrypt"
|
||||
feature flag enabled using ``tune2fs -O encrypt`` or ``mkfs.ext4 -O
|
||||
encrypt``.)
|
||||
- ``EPERM``: this directory may not be encrypted, e.g. because it is
|
||||
the root directory of an ext4 filesystem
|
||||
- ``EROFS``: the filesystem is readonly
|
||||
|
||||
Getting an encryption policy
|
||||
----------------------------
|
||||
|
||||
The FS_IOC_GET_ENCRYPTION_POLICY ioctl retrieves the :c:type:`struct
|
||||
fscrypt_policy`, if any, for a directory or regular file. See above
|
||||
for the struct definition. No additional permissions are required
|
||||
beyond the ability to open the file.
|
||||
|
||||
FS_IOC_GET_ENCRYPTION_POLICY can fail with the following errors:
|
||||
|
||||
- ``EINVAL``: the file is encrypted, but it uses an unrecognized
|
||||
encryption context format
|
||||
- ``ENODATA``: the file is not encrypted
|
||||
- ``ENOTTY``: this type of filesystem does not implement encryption
|
||||
- ``EOPNOTSUPP``: the kernel was not configured with encryption
|
||||
support for this filesystem
|
||||
|
||||
Note: if you only need to know whether a file is encrypted or not, on
|
||||
most filesystems it is also possible to use the FS_IOC_GETFLAGS ioctl
|
||||
and check for FS_ENCRYPT_FL, or to use the statx() system call and
|
||||
check for STATX_ATTR_ENCRYPTED in stx_attributes.
|
||||
|
||||
Getting the per-filesystem salt
|
||||
-------------------------------
|
||||
|
||||
Some filesystems, such as ext4 and F2FS, also support the deprecated
|
||||
ioctl FS_IOC_GET_ENCRYPTION_PWSALT. This ioctl retrieves a randomly
|
||||
generated 16-byte value stored in the filesystem superblock. This
|
||||
value is intended to used as a salt when deriving an encryption key
|
||||
from a passphrase or other low-entropy user credential.
|
||||
|
||||
FS_IOC_GET_ENCRYPTION_PWSALT is deprecated. Instead, prefer to
|
||||
generate and manage any needed salt(s) in userspace.
|
||||
|
||||
Adding keys
|
||||
-----------
|
||||
|
||||
To provide a master key, userspace must add it to an appropriate
|
||||
keyring using the add_key() system call (see:
|
||||
``Documentation/security/keys/core.rst``). The key type must be
|
||||
"logon"; keys of this type are kept in kernel memory and cannot be
|
||||
read back by userspace. The key description must be "fscrypt:"
|
||||
followed by the 16-character lower case hex representation of the
|
||||
``master_key_descriptor`` that was set in the encryption policy. The
|
||||
key payload must conform to the following structure::
|
||||
|
||||
#define FS_MAX_KEY_SIZE 64
|
||||
|
||||
struct fscrypt_key {
|
||||
u32 mode;
|
||||
u8 raw[FS_MAX_KEY_SIZE];
|
||||
u32 size;
|
||||
};
|
||||
|
||||
``mode`` is ignored; just set it to 0. The actual key is provided in
|
||||
``raw`` with ``size`` indicating its size in bytes. That is, the
|
||||
bytes ``raw[0..size-1]`` (inclusive) are the actual key.
|
||||
|
||||
The key description prefix "fscrypt:" may alternatively be replaced
|
||||
with a filesystem-specific prefix such as "ext4:". However, the
|
||||
filesystem-specific prefixes are deprecated and should not be used in
|
||||
new programs.
|
||||
|
||||
There are several different types of keyrings in which encryption keys
|
||||
may be placed, such as a session keyring, a user session keyring, or a
|
||||
user keyring. Each key must be placed in a keyring that is "attached"
|
||||
to all processes that might need to access files encrypted with it, in
|
||||
the sense that request_key() will find the key. Generally, if only
|
||||
processes belonging to a specific user need to access a given
|
||||
encrypted directory and no session keyring has been installed, then
|
||||
that directory's key should be placed in that user's user session
|
||||
keyring or user keyring. Otherwise, a session keyring should be
|
||||
installed if needed, and the key should be linked into that session
|
||||
keyring, or in a keyring linked into that session keyring.
|
||||
|
||||
Note: introducing the complex visibility semantics of keyrings here
|
||||
was arguably a mistake --- especially given that by design, after any
|
||||
process successfully opens an encrypted file (thereby setting up the
|
||||
per-file key), possessing the keyring key is not actually required for
|
||||
any process to read/write the file until its in-memory inode is
|
||||
evicted. In the future there probably should be a way to provide keys
|
||||
directly to the filesystem instead, which would make the intended
|
||||
semantics clearer.
|
||||
|
||||
Access semantics
|
||||
================
|
||||
|
||||
With the key
|
||||
------------
|
||||
|
||||
With the encryption key, encrypted regular files, directories, and
|
||||
symlinks behave very similarly to their unencrypted counterparts ---
|
||||
after all, the encryption is intended to be transparent. However,
|
||||
astute users may notice some differences in behavior:
|
||||
|
||||
- Unencrypted files, or files encrypted with a different encryption
|
||||
policy (i.e. different key, modes, or flags), cannot be renamed or
|
||||
linked into an encrypted directory; see `Encryption policy
|
||||
enforcement`_. Attempts to do so will fail with EPERM. However,
|
||||
encrypted files can be renamed within an encrypted directory, or
|
||||
into an unencrypted directory.
|
||||
|
||||
- Direct I/O is not supported on encrypted files. Attempts to use
|
||||
direct I/O on such files will fall back to buffered I/O.
|
||||
|
||||
- The fallocate operations FALLOC_FL_COLLAPSE_RANGE,
|
||||
FALLOC_FL_INSERT_RANGE, and FALLOC_FL_ZERO_RANGE are not supported
|
||||
on encrypted files and will fail with EOPNOTSUPP.
|
||||
|
||||
- Online defragmentation of encrypted files is not supported. The
|
||||
EXT4_IOC_MOVE_EXT and F2FS_IOC_MOVE_RANGE ioctls will fail with
|
||||
EOPNOTSUPP.
|
||||
|
||||
- The ext4 filesystem does not support data journaling with encrypted
|
||||
regular files. It will fall back to ordered data mode instead.
|
||||
|
||||
- DAX (Direct Access) is not supported on encrypted files.
|
||||
|
||||
- The st_size of an encrypted symlink will not necessarily give the
|
||||
length of the symlink target as required by POSIX. It will actually
|
||||
give the length of the ciphertext, which will be slightly longer
|
||||
than the plaintext due to NUL-padding and an extra 2-byte overhead.
|
||||
|
||||
- The maximum length of an encrypted symlink is 2 bytes shorter than
|
||||
the maximum length of an unencrypted symlink. For example, on an
|
||||
EXT4 filesystem with a 4K block size, unencrypted symlinks can be up
|
||||
to 4095 bytes long, while encrypted symlinks can only be up to 4093
|
||||
bytes long (both lengths excluding the terminating null).
|
||||
|
||||
Note that mmap *is* supported. This is possible because the pagecache
|
||||
for an encrypted file contains the plaintext, not the ciphertext.
|
||||
|
||||
Without the key
|
||||
---------------
|
||||
|
||||
Some filesystem operations may be performed on encrypted regular
|
||||
files, directories, and symlinks even before their encryption key has
|
||||
been provided:
|
||||
|
||||
- File metadata may be read, e.g. using stat().
|
||||
|
||||
- Directories may be listed, in which case the filenames will be
|
||||
listed in an encoded form derived from their ciphertext. The
|
||||
current encoding algorithm is described in `Filename hashing and
|
||||
encoding`_. The algorithm is subject to change, but it is
|
||||
guaranteed that the presented filenames will be no longer than
|
||||
NAME_MAX bytes, will not contain the ``/`` or ``\0`` characters, and
|
||||
will uniquely identify directory entries.
|
||||
|
||||
The ``.`` and ``..`` directory entries are special. They are always
|
||||
present and are not encrypted or encoded.
|
||||
|
||||
- Files may be deleted. That is, nondirectory files may be deleted
|
||||
with unlink() as usual, and empty directories may be deleted with
|
||||
rmdir() as usual. Therefore, ``rm`` and ``rm -r`` will work as
|
||||
expected.
|
||||
|
||||
- Symlink targets may be read and followed, but they will be presented
|
||||
in encrypted form, similar to filenames in directories. Hence, they
|
||||
are unlikely to point to anywhere useful.
|
||||
|
||||
Without the key, regular files cannot be opened or truncated.
|
||||
Attempts to do so will fail with ENOKEY. This implies that any
|
||||
regular file operations that require a file descriptor, such as
|
||||
read(), write(), mmap(), fallocate(), and ioctl(), are also forbidden.
|
||||
|
||||
Also without the key, files of any type (including directories) cannot
|
||||
be created or linked into an encrypted directory, nor can a name in an
|
||||
encrypted directory be the source or target of a rename, nor can an
|
||||
O_TMPFILE temporary file be created in an encrypted directory. All
|
||||
such operations will fail with ENOKEY.
|
||||
|
||||
It is not currently possible to backup and restore encrypted files
|
||||
without the encryption key. This would require special APIs which
|
||||
have not yet been implemented.
|
||||
|
||||
Encryption policy enforcement
|
||||
=============================
|
||||
|
||||
After an encryption policy has been set on a directory, all regular
|
||||
files, directories, and symbolic links created in that directory
|
||||
(recursively) will inherit that encryption policy. Special files ---
|
||||
that is, named pipes, device nodes, and UNIX domain sockets --- will
|
||||
not be encrypted.
|
||||
|
||||
Except for those special files, it is forbidden to have unencrypted
|
||||
files, or files encrypted with a different encryption policy, in an
|
||||
encrypted directory tree. Attempts to link or rename such a file into
|
||||
an encrypted directory will fail with EPERM. This is also enforced
|
||||
during ->lookup() to provide limited protection against offline
|
||||
attacks that try to disable or downgrade encryption in known locations
|
||||
where applications may later write sensitive data. It is recommended
|
||||
that systems implementing a form of "verified boot" take advantage of
|
||||
this by validating all top-level encryption policies prior to access.
|
||||
|
||||
Implementation details
|
||||
======================
|
||||
|
||||
Encryption context
|
||||
------------------
|
||||
|
||||
An encryption policy is represented on-disk by a :c:type:`struct
|
||||
fscrypt_context`. It is up to individual filesystems to decide where
|
||||
to store it, but normally it would be stored in a hidden extended
|
||||
attribute. It should *not* be exposed by the xattr-related system
|
||||
calls such as getxattr() and setxattr() because of the special
|
||||
semantics of the encryption xattr. (In particular, there would be
|
||||
much confusion if an encryption policy were to be added to or removed
|
||||
from anything other than an empty directory.) The struct is defined
|
||||
as follows::
|
||||
|
||||
#define FS_KEY_DESCRIPTOR_SIZE 8
|
||||
#define FS_KEY_DERIVATION_NONCE_SIZE 16
|
||||
|
||||
struct fscrypt_context {
|
||||
u8 format;
|
||||
u8 contents_encryption_mode;
|
||||
u8 filenames_encryption_mode;
|
||||
u8 flags;
|
||||
u8 master_key_descriptor[FS_KEY_DESCRIPTOR_SIZE];
|
||||
u8 nonce[FS_KEY_DERIVATION_NONCE_SIZE];
|
||||
};
|
||||
|
||||
Note that :c:type:`struct fscrypt_context` contains the same
|
||||
information as :c:type:`struct fscrypt_policy` (see `Setting an
|
||||
encryption policy`_), except that :c:type:`struct fscrypt_context`
|
||||
also contains a nonce. The nonce is randomly generated by the kernel
|
||||
and is used to derive the inode's encryption key as described in
|
||||
`Per-file keys`_.
|
||||
|
||||
Data path changes
|
||||
-----------------
|
||||
|
||||
For the read path (->readpage()) of regular files, filesystems can
|
||||
read the ciphertext into the page cache and decrypt it in-place. The
|
||||
page lock must be held until decryption has finished, to prevent the
|
||||
page from becoming visible to userspace prematurely.
|
||||
|
||||
For the write path (->writepage()) of regular files, filesystems
|
||||
cannot encrypt data in-place in the page cache, since the cached
|
||||
plaintext must be preserved. Instead, filesystems must encrypt into a
|
||||
temporary buffer or "bounce page", then write out the temporary
|
||||
buffer. Some filesystems, such as UBIFS, already use temporary
|
||||
buffers regardless of encryption. Other filesystems, such as ext4 and
|
||||
F2FS, have to allocate bounce pages specially for encryption.
|
||||
|
||||
Filename hashing and encoding
|
||||
-----------------------------
|
||||
|
||||
Modern filesystems accelerate directory lookups by using indexed
|
||||
directories. An indexed directory is organized as a tree keyed by
|
||||
filename hashes. When a ->lookup() is requested, the filesystem
|
||||
normally hashes the filename being looked up so that it can quickly
|
||||
find the corresponding directory entry, if any.
|
||||
|
||||
With encryption, lookups must be supported and efficient both with and
|
||||
without the encryption key. Clearly, it would not work to hash the
|
||||
plaintext filenames, since the plaintext filenames are unavailable
|
||||
without the key. (Hashing the plaintext filenames would also make it
|
||||
impossible for the filesystem's fsck tool to optimize encrypted
|
||||
directories.) Instead, filesystems hash the ciphertext filenames,
|
||||
i.e. the bytes actually stored on-disk in the directory entries. When
|
||||
asked to do a ->lookup() with the key, the filesystem just encrypts
|
||||
the user-supplied name to get the ciphertext.
|
||||
|
||||
Lookups without the key are more complicated. The raw ciphertext may
|
||||
contain the ``\0`` and ``/`` characters, which are illegal in
|
||||
filenames. Therefore, readdir() must base64-encode the ciphertext for
|
||||
presentation. For most filenames, this works fine; on ->lookup(), the
|
||||
filesystem just base64-decodes the user-supplied name to get back to
|
||||
the raw ciphertext.
|
||||
|
||||
However, for very long filenames, base64 encoding would cause the
|
||||
filename length to exceed NAME_MAX. To prevent this, readdir()
|
||||
actually presents long filenames in an abbreviated form which encodes
|
||||
a strong "hash" of the ciphertext filename, along with the optional
|
||||
filesystem-specific hash(es) needed for directory lookups. This
|
||||
allows the filesystem to still, with a high degree of confidence, map
|
||||
the filename given in ->lookup() back to a particular directory entry
|
||||
that was previously listed by readdir(). See :c:type:`struct
|
||||
fscrypt_digested_name` in the source for more details.
|
||||
|
||||
Note that the precise way that filenames are presented to userspace
|
||||
without the key is subject to change in the future. It is only meant
|
||||
as a way to temporarily present valid filenames so that commands like
|
||||
``rm -r`` work as expected on encrypted directories.
|
||||
2
Makefile
2
Makefile
|
|
@ -1,6 +1,6 @@
|
|||
VERSION = 4
|
||||
PATCHLEVEL = 4
|
||||
SUBLEVEL = 162
|
||||
SUBLEVEL = 163
|
||||
EXTRAVERSION =
|
||||
NAME = Blurry Fish Butt
|
||||
|
||||
|
|
|
|||
|
|
@ -130,6 +130,17 @@
|
|||
};
|
||||
};
|
||||
|
||||
&cpu0 {
|
||||
/* CPU rated to 1GHz, not 1.2GHz as per the default settings */
|
||||
operating-points = <
|
||||
/* kHz uV */
|
||||
166666 850000
|
||||
400000 900000
|
||||
800000 1050000
|
||||
1000000 1200000
|
||||
>;
|
||||
};
|
||||
|
||||
&esdhc1 {
|
||||
pinctrl-names = "default";
|
||||
pinctrl-0 = <&pinctrl_esdhc1>;
|
||||
|
|
|
|||
|
|
@ -577,7 +577,7 @@
|
|||
};
|
||||
|
||||
sata0: sata@29000000 {
|
||||
compatible = "generic-ahci";
|
||||
compatible = "qcom,apq8064-ahci", "generic-ahci";
|
||||
status = "disabled";
|
||||
reg = <0x29000000 0x180>;
|
||||
interrupts = <GIC_SPI 209 IRQ_TYPE_NONE>;
|
||||
|
|
@ -599,6 +599,7 @@
|
|||
|
||||
phys = <&sata_phy0>;
|
||||
phy-names = "sata-phy";
|
||||
ports-implemented = <0x1>;
|
||||
};
|
||||
|
||||
/* Temporary fixed regulator */
|
||||
|
|
|
|||
|
|
@ -120,11 +120,4 @@ config CRYPTO_GHASH_ARM_CE
|
|||
that uses the 64x64 to 128 bit polynomial multiplication (vmull.p64)
|
||||
that is part of the ARMv8 Crypto Extensions
|
||||
|
||||
config CRYPTO_SPECK_NEON
|
||||
tristate "NEON accelerated Speck cipher algorithms"
|
||||
depends on KERNEL_MODE_NEON
|
||||
select CRYPTO_BLKCIPHER
|
||||
select CRYPTO_GF128MUL
|
||||
select CRYPTO_SPECK
|
||||
|
||||
endif
|
||||
|
|
|
|||
|
|
@ -8,7 +8,6 @@ obj-$(CONFIG_CRYPTO_SHA1_ARM) += sha1-arm.o
|
|||
obj-$(CONFIG_CRYPTO_SHA1_ARM_NEON) += sha1-arm-neon.o
|
||||
obj-$(CONFIG_CRYPTO_SHA256_ARM) += sha256-arm.o
|
||||
obj-$(CONFIG_CRYPTO_SHA512_ARM) += sha512-arm.o
|
||||
obj-$(CONFIG_CRYPTO_SPECK_NEON) += speck-neon.o
|
||||
|
||||
ce-obj-$(CONFIG_CRYPTO_AES_ARM_CE) += aes-arm-ce.o
|
||||
ce-obj-$(CONFIG_CRYPTO_SHA1_ARM_CE) += sha1-arm-ce.o
|
||||
|
|
@ -37,7 +36,6 @@ sha1-arm-ce-y := sha1-ce-core.o sha1-ce-glue.o
|
|||
sha2-arm-ce-y := sha2-ce-core.o sha2-ce-glue.o
|
||||
aes-arm-ce-y := aes-ce-core.o aes-ce-glue.o
|
||||
ghash-arm-ce-y := ghash-ce-core.o ghash-ce-glue.o
|
||||
speck-neon-y := speck-neon-core.o speck-neon-glue.o
|
||||
|
||||
quiet_cmd_perl = PERL $@
|
||||
cmd_perl = $(PERL) $(<) > $(@)
|
||||
|
|
|
|||
|
|
@ -1,432 +0,0 @@
|
|||
// SPDX-License-Identifier: GPL-2.0
|
||||
/*
|
||||
* NEON-accelerated implementation of Speck128-XTS and Speck64-XTS
|
||||
*
|
||||
* Copyright (c) 2018 Google, Inc
|
||||
*
|
||||
* Author: Eric Biggers <ebiggers@google.com>
|
||||
*/
|
||||
|
||||
#include <linux/linkage.h>
|
||||
|
||||
.text
|
||||
.fpu neon
|
||||
|
||||
// arguments
|
||||
ROUND_KEYS .req r0 // const {u64,u32} *round_keys
|
||||
NROUNDS .req r1 // int nrounds
|
||||
DST .req r2 // void *dst
|
||||
SRC .req r3 // const void *src
|
||||
NBYTES .req r4 // unsigned int nbytes
|
||||
TWEAK .req r5 // void *tweak
|
||||
|
||||
// registers which hold the data being encrypted/decrypted
|
||||
X0 .req q0
|
||||
X0_L .req d0
|
||||
X0_H .req d1
|
||||
Y0 .req q1
|
||||
Y0_H .req d3
|
||||
X1 .req q2
|
||||
X1_L .req d4
|
||||
X1_H .req d5
|
||||
Y1 .req q3
|
||||
Y1_H .req d7
|
||||
X2 .req q4
|
||||
X2_L .req d8
|
||||
X2_H .req d9
|
||||
Y2 .req q5
|
||||
Y2_H .req d11
|
||||
X3 .req q6
|
||||
X3_L .req d12
|
||||
X3_H .req d13
|
||||
Y3 .req q7
|
||||
Y3_H .req d15
|
||||
|
||||
// the round key, duplicated in all lanes
|
||||
ROUND_KEY .req q8
|
||||
ROUND_KEY_L .req d16
|
||||
ROUND_KEY_H .req d17
|
||||
|
||||
// index vector for vtbl-based 8-bit rotates
|
||||
ROTATE_TABLE .req d18
|
||||
|
||||
// multiplication table for updating XTS tweaks
|
||||
GF128MUL_TABLE .req d19
|
||||
GF64MUL_TABLE .req d19
|
||||
|
||||
// current XTS tweak value(s)
|
||||
TWEAKV .req q10
|
||||
TWEAKV_L .req d20
|
||||
TWEAKV_H .req d21
|
||||
|
||||
TMP0 .req q12
|
||||
TMP0_L .req d24
|
||||
TMP0_H .req d25
|
||||
TMP1 .req q13
|
||||
TMP2 .req q14
|
||||
TMP3 .req q15
|
||||
|
||||
.align 4
|
||||
.Lror64_8_table:
|
||||
.byte 1, 2, 3, 4, 5, 6, 7, 0
|
||||
.Lror32_8_table:
|
||||
.byte 1, 2, 3, 0, 5, 6, 7, 4
|
||||
.Lrol64_8_table:
|
||||
.byte 7, 0, 1, 2, 3, 4, 5, 6
|
||||
.Lrol32_8_table:
|
||||
.byte 3, 0, 1, 2, 7, 4, 5, 6
|
||||
.Lgf128mul_table:
|
||||
.byte 0, 0x87
|
||||
.fill 14
|
||||
.Lgf64mul_table:
|
||||
.byte 0, 0x1b, (0x1b << 1), (0x1b << 1) ^ 0x1b
|
||||
.fill 12
|
||||
|
||||
/*
|
||||
* _speck_round_128bytes() - Speck encryption round on 128 bytes at a time
|
||||
*
|
||||
* Do one Speck encryption round on the 128 bytes (8 blocks for Speck128, 16 for
|
||||
* Speck64) stored in X0-X3 and Y0-Y3, using the round key stored in all lanes
|
||||
* of ROUND_KEY. 'n' is the lane size: 64 for Speck128, or 32 for Speck64.
|
||||
*
|
||||
* The 8-bit rotates are implemented using vtbl instead of vshr + vsli because
|
||||
* the vtbl approach is faster on some processors and the same speed on others.
|
||||
*/
|
||||
.macro _speck_round_128bytes n
|
||||
|
||||
// x = ror(x, 8)
|
||||
vtbl.8 X0_L, {X0_L}, ROTATE_TABLE
|
||||
vtbl.8 X0_H, {X0_H}, ROTATE_TABLE
|
||||
vtbl.8 X1_L, {X1_L}, ROTATE_TABLE
|
||||
vtbl.8 X1_H, {X1_H}, ROTATE_TABLE
|
||||
vtbl.8 X2_L, {X2_L}, ROTATE_TABLE
|
||||
vtbl.8 X2_H, {X2_H}, ROTATE_TABLE
|
||||
vtbl.8 X3_L, {X3_L}, ROTATE_TABLE
|
||||
vtbl.8 X3_H, {X3_H}, ROTATE_TABLE
|
||||
|
||||
// x += y
|
||||
vadd.u\n X0, Y0
|
||||
vadd.u\n X1, Y1
|
||||
vadd.u\n X2, Y2
|
||||
vadd.u\n X3, Y3
|
||||
|
||||
// x ^= k
|
||||
veor X0, ROUND_KEY
|
||||
veor X1, ROUND_KEY
|
||||
veor X2, ROUND_KEY
|
||||
veor X3, ROUND_KEY
|
||||
|
||||
// y = rol(y, 3)
|
||||
vshl.u\n TMP0, Y0, #3
|
||||
vshl.u\n TMP1, Y1, #3
|
||||
vshl.u\n TMP2, Y2, #3
|
||||
vshl.u\n TMP3, Y3, #3
|
||||
vsri.u\n TMP0, Y0, #(\n - 3)
|
||||
vsri.u\n TMP1, Y1, #(\n - 3)
|
||||
vsri.u\n TMP2, Y2, #(\n - 3)
|
||||
vsri.u\n TMP3, Y3, #(\n - 3)
|
||||
|
||||
// y ^= x
|
||||
veor Y0, TMP0, X0
|
||||
veor Y1, TMP1, X1
|
||||
veor Y2, TMP2, X2
|
||||
veor Y3, TMP3, X3
|
||||
.endm
|
||||
|
||||
/*
|
||||
* _speck_unround_128bytes() - Speck decryption round on 128 bytes at a time
|
||||
*
|
||||
* This is the inverse of _speck_round_128bytes().
|
||||
*/
|
||||
.macro _speck_unround_128bytes n
|
||||
|
||||
// y ^= x
|
||||
veor TMP0, Y0, X0
|
||||
veor TMP1, Y1, X1
|
||||
veor TMP2, Y2, X2
|
||||
veor TMP3, Y3, X3
|
||||
|
||||
// y = ror(y, 3)
|
||||
vshr.u\n Y0, TMP0, #3
|
||||
vshr.u\n Y1, TMP1, #3
|
||||
vshr.u\n Y2, TMP2, #3
|
||||
vshr.u\n Y3, TMP3, #3
|
||||
vsli.u\n Y0, TMP0, #(\n - 3)
|
||||
vsli.u\n Y1, TMP1, #(\n - 3)
|
||||
vsli.u\n Y2, TMP2, #(\n - 3)
|
||||
vsli.u\n Y3, TMP3, #(\n - 3)
|
||||
|
||||
// x ^= k
|
||||
veor X0, ROUND_KEY
|
||||
veor X1, ROUND_KEY
|
||||
veor X2, ROUND_KEY
|
||||
veor X3, ROUND_KEY
|
||||
|
||||
// x -= y
|
||||
vsub.u\n X0, Y0
|
||||
vsub.u\n X1, Y1
|
||||
vsub.u\n X2, Y2
|
||||
vsub.u\n X3, Y3
|
||||
|
||||
// x = rol(x, 8);
|
||||
vtbl.8 X0_L, {X0_L}, ROTATE_TABLE
|
||||
vtbl.8 X0_H, {X0_H}, ROTATE_TABLE
|
||||
vtbl.8 X1_L, {X1_L}, ROTATE_TABLE
|
||||
vtbl.8 X1_H, {X1_H}, ROTATE_TABLE
|
||||
vtbl.8 X2_L, {X2_L}, ROTATE_TABLE
|
||||
vtbl.8 X2_H, {X2_H}, ROTATE_TABLE
|
||||
vtbl.8 X3_L, {X3_L}, ROTATE_TABLE
|
||||
vtbl.8 X3_H, {X3_H}, ROTATE_TABLE
|
||||
.endm
|
||||
|
||||
.macro _xts128_precrypt_one dst_reg, tweak_buf, tmp
|
||||
|
||||
// Load the next source block
|
||||
vld1.8 {\dst_reg}, [SRC]!
|
||||
|
||||
// Save the current tweak in the tweak buffer
|
||||
vst1.8 {TWEAKV}, [\tweak_buf:128]!
|
||||
|
||||
// XOR the next source block with the current tweak
|
||||
veor \dst_reg, TWEAKV
|
||||
|
||||
/*
|
||||
* Calculate the next tweak by multiplying the current one by x,
|
||||
* modulo p(x) = x^128 + x^7 + x^2 + x + 1.
|
||||
*/
|
||||
vshr.u64 \tmp, TWEAKV, #63
|
||||
vshl.u64 TWEAKV, #1
|
||||
veor TWEAKV_H, \tmp\()_L
|
||||
vtbl.8 \tmp\()_H, {GF128MUL_TABLE}, \tmp\()_H
|
||||
veor TWEAKV_L, \tmp\()_H
|
||||
.endm
|
||||
|
||||
.macro _xts64_precrypt_two dst_reg, tweak_buf, tmp
|
||||
|
||||
// Load the next two source blocks
|
||||
vld1.8 {\dst_reg}, [SRC]!
|
||||
|
||||
// Save the current two tweaks in the tweak buffer
|
||||
vst1.8 {TWEAKV}, [\tweak_buf:128]!
|
||||
|
||||
// XOR the next two source blocks with the current two tweaks
|
||||
veor \dst_reg, TWEAKV
|
||||
|
||||
/*
|
||||
* Calculate the next two tweaks by multiplying the current ones by x^2,
|
||||
* modulo p(x) = x^64 + x^4 + x^3 + x + 1.
|
||||
*/
|
||||
vshr.u64 \tmp, TWEAKV, #62
|
||||
vshl.u64 TWEAKV, #2
|
||||
vtbl.8 \tmp\()_L, {GF64MUL_TABLE}, \tmp\()_L
|
||||
vtbl.8 \tmp\()_H, {GF64MUL_TABLE}, \tmp\()_H
|
||||
veor TWEAKV, \tmp
|
||||
.endm
|
||||
|
||||
/*
|
||||
* _speck_xts_crypt() - Speck-XTS encryption/decryption
|
||||
*
|
||||
* Encrypt or decrypt NBYTES bytes of data from the SRC buffer to the DST buffer
|
||||
* using Speck-XTS, specifically the variant with a block size of '2n' and round
|
||||
* count given by NROUNDS. The expanded round keys are given in ROUND_KEYS, and
|
||||
* the current XTS tweak value is given in TWEAK. It's assumed that NBYTES is a
|
||||
* nonzero multiple of 128.
|
||||
*/
|
||||
.macro _speck_xts_crypt n, decrypting
|
||||
push {r4-r7}
|
||||
mov r7, sp
|
||||
|
||||
/*
|
||||
* The first four parameters were passed in registers r0-r3. Load the
|
||||
* additional parameters, which were passed on the stack.
|
||||
*/
|
||||
ldr NBYTES, [sp, #16]
|
||||
ldr TWEAK, [sp, #20]
|
||||
|
||||
/*
|
||||
* If decrypting, modify the ROUND_KEYS parameter to point to the last
|
||||
* round key rather than the first, since for decryption the round keys
|
||||
* are used in reverse order.
|
||||
*/
|
||||
.if \decrypting
|
||||
.if \n == 64
|
||||
add ROUND_KEYS, ROUND_KEYS, NROUNDS, lsl #3
|
||||
sub ROUND_KEYS, #8
|
||||
.else
|
||||
add ROUND_KEYS, ROUND_KEYS, NROUNDS, lsl #2
|
||||
sub ROUND_KEYS, #4
|
||||
.endif
|
||||
.endif
|
||||
|
||||
// Load the index vector for vtbl-based 8-bit rotates
|
||||
.if \decrypting
|
||||
ldr r12, =.Lrol\n\()_8_table
|
||||
.else
|
||||
ldr r12, =.Lror\n\()_8_table
|
||||
.endif
|
||||
vld1.8 {ROTATE_TABLE}, [r12:64]
|
||||
|
||||
// One-time XTS preparation
|
||||
|
||||
/*
|
||||
* Allocate stack space to store 128 bytes worth of tweaks. For
|
||||
* performance, this space is aligned to a 16-byte boundary so that we
|
||||
* can use the load/store instructions that declare 16-byte alignment.
|
||||
*/
|
||||
sub sp, #128
|
||||
bic sp, #0xf
|
||||
|
||||
.if \n == 64
|
||||
// Load first tweak
|
||||
vld1.8 {TWEAKV}, [TWEAK]
|
||||
|
||||
// Load GF(2^128) multiplication table
|
||||
ldr r12, =.Lgf128mul_table
|
||||
vld1.8 {GF128MUL_TABLE}, [r12:64]
|
||||
.else
|
||||
// Load first tweak
|
||||
vld1.8 {TWEAKV_L}, [TWEAK]
|
||||
|
||||
// Load GF(2^64) multiplication table
|
||||
ldr r12, =.Lgf64mul_table
|
||||
vld1.8 {GF64MUL_TABLE}, [r12:64]
|
||||
|
||||
// Calculate second tweak, packing it together with the first
|
||||
vshr.u64 TMP0_L, TWEAKV_L, #63
|
||||
vtbl.u8 TMP0_L, {GF64MUL_TABLE}, TMP0_L
|
||||
vshl.u64 TWEAKV_H, TWEAKV_L, #1
|
||||
veor TWEAKV_H, TMP0_L
|
||||
.endif
|
||||
|
||||
.Lnext_128bytes_\@:
|
||||
|
||||
/*
|
||||
* Load the source blocks into {X,Y}[0-3], XOR them with their XTS tweak
|
||||
* values, and save the tweaks on the stack for later. Then
|
||||
* de-interleave the 'x' and 'y' elements of each block, i.e. make it so
|
||||
* that the X[0-3] registers contain only the second halves of blocks,
|
||||
* and the Y[0-3] registers contain only the first halves of blocks.
|
||||
* (Speck uses the order (y, x) rather than the more intuitive (x, y).)
|
||||
*/
|
||||
mov r12, sp
|
||||
.if \n == 64
|
||||
_xts128_precrypt_one X0, r12, TMP0
|
||||
_xts128_precrypt_one Y0, r12, TMP0
|
||||
_xts128_precrypt_one X1, r12, TMP0
|
||||
_xts128_precrypt_one Y1, r12, TMP0
|
||||
_xts128_precrypt_one X2, r12, TMP0
|
||||
_xts128_precrypt_one Y2, r12, TMP0
|
||||
_xts128_precrypt_one X3, r12, TMP0
|
||||
_xts128_precrypt_one Y3, r12, TMP0
|
||||
vswp X0_L, Y0_H
|
||||
vswp X1_L, Y1_H
|
||||
vswp X2_L, Y2_H
|
||||
vswp X3_L, Y3_H
|
||||
.else
|
||||
_xts64_precrypt_two X0, r12, TMP0
|
||||
_xts64_precrypt_two Y0, r12, TMP0
|
||||
_xts64_precrypt_two X1, r12, TMP0
|
||||
_xts64_precrypt_two Y1, r12, TMP0
|
||||
_xts64_precrypt_two X2, r12, TMP0
|
||||
_xts64_precrypt_two Y2, r12, TMP0
|
||||
_xts64_precrypt_two X3, r12, TMP0
|
||||
_xts64_precrypt_two Y3, r12, TMP0
|
||||
vuzp.32 Y0, X0
|
||||
vuzp.32 Y1, X1
|
||||
vuzp.32 Y2, X2
|
||||
vuzp.32 Y3, X3
|
||||
.endif
|
||||
|
||||
// Do the cipher rounds
|
||||
|
||||
mov r12, ROUND_KEYS
|
||||
mov r6, NROUNDS
|
||||
|
||||
.Lnext_round_\@:
|
||||
.if \decrypting
|
||||
.if \n == 64
|
||||
vld1.64 ROUND_KEY_L, [r12]
|
||||
sub r12, #8
|
||||
vmov ROUND_KEY_H, ROUND_KEY_L
|
||||
.else
|
||||
vld1.32 {ROUND_KEY_L[],ROUND_KEY_H[]}, [r12]
|
||||
sub r12, #4
|
||||
.endif
|
||||
_speck_unround_128bytes \n
|
||||
.else
|
||||
.if \n == 64
|
||||
vld1.64 ROUND_KEY_L, [r12]!
|
||||
vmov ROUND_KEY_H, ROUND_KEY_L
|
||||
.else
|
||||
vld1.32 {ROUND_KEY_L[],ROUND_KEY_H[]}, [r12]!
|
||||
.endif
|
||||
_speck_round_128bytes \n
|
||||
.endif
|
||||
subs r6, r6, #1
|
||||
bne .Lnext_round_\@
|
||||
|
||||
// Re-interleave the 'x' and 'y' elements of each block
|
||||
.if \n == 64
|
||||
vswp X0_L, Y0_H
|
||||
vswp X1_L, Y1_H
|
||||
vswp X2_L, Y2_H
|
||||
vswp X3_L, Y3_H
|
||||
.else
|
||||
vzip.32 Y0, X0
|
||||
vzip.32 Y1, X1
|
||||
vzip.32 Y2, X2
|
||||
vzip.32 Y3, X3
|
||||
.endif
|
||||
|
||||
// XOR the encrypted/decrypted blocks with the tweaks we saved earlier
|
||||
mov r12, sp
|
||||
vld1.8 {TMP0, TMP1}, [r12:128]!
|
||||
vld1.8 {TMP2, TMP3}, [r12:128]!
|
||||
veor X0, TMP0
|
||||
veor Y0, TMP1
|
||||
veor X1, TMP2
|
||||
veor Y1, TMP3
|
||||
vld1.8 {TMP0, TMP1}, [r12:128]!
|
||||
vld1.8 {TMP2, TMP3}, [r12:128]!
|
||||
veor X2, TMP0
|
||||
veor Y2, TMP1
|
||||
veor X3, TMP2
|
||||
veor Y3, TMP3
|
||||
|
||||
// Store the ciphertext in the destination buffer
|
||||
vst1.8 {X0, Y0}, [DST]!
|
||||
vst1.8 {X1, Y1}, [DST]!
|
||||
vst1.8 {X2, Y2}, [DST]!
|
||||
vst1.8 {X3, Y3}, [DST]!
|
||||
|
||||
// Continue if there are more 128-byte chunks remaining, else return
|
||||
subs NBYTES, #128
|
||||
bne .Lnext_128bytes_\@
|
||||
|
||||
// Store the next tweak
|
||||
.if \n == 64
|
||||
vst1.8 {TWEAKV}, [TWEAK]
|
||||
.else
|
||||
vst1.8 {TWEAKV_L}, [TWEAK]
|
||||
.endif
|
||||
|
||||
mov sp, r7
|
||||
pop {r4-r7}
|
||||
bx lr
|
||||
.endm
|
||||
|
||||
ENTRY(speck128_xts_encrypt_neon)
|
||||
_speck_xts_crypt n=64, decrypting=0
|
||||
ENDPROC(speck128_xts_encrypt_neon)
|
||||
|
||||
ENTRY(speck128_xts_decrypt_neon)
|
||||
_speck_xts_crypt n=64, decrypting=1
|
||||
ENDPROC(speck128_xts_decrypt_neon)
|
||||
|
||||
ENTRY(speck64_xts_encrypt_neon)
|
||||
_speck_xts_crypt n=32, decrypting=0
|
||||
ENDPROC(speck64_xts_encrypt_neon)
|
||||
|
||||
ENTRY(speck64_xts_decrypt_neon)
|
||||
_speck_xts_crypt n=32, decrypting=1
|
||||
ENDPROC(speck64_xts_decrypt_neon)
|
||||
|
|
@ -1,314 +0,0 @@
|
|||
// SPDX-License-Identifier: GPL-2.0
|
||||
/*
|
||||
* NEON-accelerated implementation of Speck128-XTS and Speck64-XTS
|
||||
*
|
||||
* Copyright (c) 2018 Google, Inc
|
||||
*
|
||||
* Note: the NIST recommendation for XTS only specifies a 128-bit block size,
|
||||
* but a 64-bit version (needed for Speck64) is fairly straightforward; the math
|
||||
* is just done in GF(2^64) instead of GF(2^128), with the reducing polynomial
|
||||
* x^64 + x^4 + x^3 + x + 1 from the original XEX paper (Rogaway, 2004:
|
||||
* "Efficient Instantiations of Tweakable Blockciphers and Refinements to Modes
|
||||
* OCB and PMAC"), represented as 0x1B.
|
||||
*/
|
||||
|
||||
#include <asm/hwcap.h>
|
||||
#include <asm/neon.h>
|
||||
#include <asm/simd.h>
|
||||
#include <crypto/algapi.h>
|
||||
#include <crypto/gf128mul.h>
|
||||
#include <crypto/speck.h>
|
||||
#include <crypto/xts.h>
|
||||
#include <linux/kernel.h>
|
||||
#include <linux/module.h>
|
||||
|
||||
/* The assembly functions only handle multiples of 128 bytes */
|
||||
#define SPECK_NEON_CHUNK_SIZE 128
|
||||
|
||||
/* Speck128 */
|
||||
|
||||
struct speck128_xts_tfm_ctx {
|
||||
struct speck128_tfm_ctx main_key;
|
||||
struct speck128_tfm_ctx tweak_key;
|
||||
};
|
||||
|
||||
asmlinkage void speck128_xts_encrypt_neon(const u64 *round_keys, int nrounds,
|
||||
void *dst, const void *src,
|
||||
unsigned int nbytes, void *tweak);
|
||||
|
||||
asmlinkage void speck128_xts_decrypt_neon(const u64 *round_keys, int nrounds,
|
||||
void *dst, const void *src,
|
||||
unsigned int nbytes, void *tweak);
|
||||
|
||||
typedef void (*speck128_crypt_one_t)(const struct speck128_tfm_ctx *,
|
||||
u8 *, const u8 *);
|
||||
typedef void (*speck128_xts_crypt_many_t)(const u64 *, int, void *,
|
||||
const void *, unsigned int, void *);
|
||||
|
||||
static __always_inline int
|
||||
__speck128_xts_crypt(struct blkcipher_desc *desc, struct scatterlist *dst,
|
||||
struct scatterlist *src, unsigned int nbytes,
|
||||
speck128_crypt_one_t crypt_one,
|
||||
speck128_xts_crypt_many_t crypt_many)
|
||||
{
|
||||
struct crypto_blkcipher *tfm = desc->tfm;
|
||||
const struct speck128_xts_tfm_ctx *ctx = crypto_blkcipher_ctx(tfm);
|
||||
struct blkcipher_walk walk;
|
||||
le128 tweak;
|
||||
int err;
|
||||
|
||||
blkcipher_walk_init(&walk, dst, src, nbytes);
|
||||
err = blkcipher_walk_virt_block(desc, &walk, SPECK_NEON_CHUNK_SIZE);
|
||||
|
||||
crypto_speck128_encrypt(&ctx->tweak_key, (u8 *)&tweak, walk.iv);
|
||||
|
||||
while (walk.nbytes > 0) {
|
||||
unsigned int nbytes = walk.nbytes;
|
||||
u8 *dst = walk.dst.virt.addr;
|
||||
const u8 *src = walk.src.virt.addr;
|
||||
|
||||
if (nbytes >= SPECK_NEON_CHUNK_SIZE && may_use_simd()) {
|
||||
unsigned int count;
|
||||
|
||||
count = round_down(nbytes, SPECK_NEON_CHUNK_SIZE);
|
||||
kernel_neon_begin();
|
||||
(*crypt_many)(ctx->main_key.round_keys,
|
||||
ctx->main_key.nrounds,
|
||||
dst, src, count, &tweak);
|
||||
kernel_neon_end();
|
||||
dst += count;
|
||||
src += count;
|
||||
nbytes -= count;
|
||||
}
|
||||
|
||||
/* Handle any remainder with generic code */
|
||||
while (nbytes >= sizeof(tweak)) {
|
||||
le128_xor((le128 *)dst, (const le128 *)src, &tweak);
|
||||
(*crypt_one)(&ctx->main_key, dst, dst);
|
||||
le128_xor((le128 *)dst, (const le128 *)dst, &tweak);
|
||||
gf128mul_x_ble((be128 *)&tweak, (const be128 *)&tweak);
|
||||
|
||||
dst += sizeof(tweak);
|
||||
src += sizeof(tweak);
|
||||
nbytes -= sizeof(tweak);
|
||||
}
|
||||
err = blkcipher_walk_done(desc, &walk, nbytes);
|
||||
}
|
||||
|
||||
return err;
|
||||
}
|
||||
|
||||
static int speck128_xts_encrypt(struct blkcipher_desc *desc,
|
||||
struct scatterlist *dst,
|
||||
struct scatterlist *src,
|
||||
unsigned int nbytes)
|
||||
{
|
||||
return __speck128_xts_crypt(desc, dst, src, nbytes,
|
||||
crypto_speck128_encrypt,
|
||||
speck128_xts_encrypt_neon);
|
||||
}
|
||||
|
||||
static int speck128_xts_decrypt(struct blkcipher_desc *desc,
|
||||
struct scatterlist *dst,
|
||||
struct scatterlist *src,
|
||||
unsigned int nbytes)
|
||||
{
|
||||
return __speck128_xts_crypt(desc, dst, src, nbytes,
|
||||
crypto_speck128_decrypt,
|
||||
speck128_xts_decrypt_neon);
|
||||
}
|
||||
|
||||
static int speck128_xts_setkey(struct crypto_tfm *tfm, const u8 *key,
|
||||
unsigned int keylen)
|
||||
{
|
||||
struct speck128_xts_tfm_ctx *ctx = crypto_tfm_ctx(tfm);
|
||||
int err;
|
||||
|
||||
if (keylen % 2)
|
||||
return -EINVAL;
|
||||
|
||||
keylen /= 2;
|
||||
|
||||
err = crypto_speck128_setkey(&ctx->main_key, key, keylen);
|
||||
if (err)
|
||||
return err;
|
||||
|
||||
return crypto_speck128_setkey(&ctx->tweak_key, key + keylen, keylen);
|
||||
}
|
||||
|
||||
/* Speck64 */
|
||||
|
||||
struct speck64_xts_tfm_ctx {
|
||||
struct speck64_tfm_ctx main_key;
|
||||
struct speck64_tfm_ctx tweak_key;
|
||||
};
|
||||
|
||||
asmlinkage void speck64_xts_encrypt_neon(const u32 *round_keys, int nrounds,
|
||||
void *dst, const void *src,
|
||||
unsigned int nbytes, void *tweak);
|
||||
|
||||
asmlinkage void speck64_xts_decrypt_neon(const u32 *round_keys, int nrounds,
|
||||
void *dst, const void *src,
|
||||
unsigned int nbytes, void *tweak);
|
||||
|
||||
typedef void (*speck64_crypt_one_t)(const struct speck64_tfm_ctx *,
|
||||
u8 *, const u8 *);
|
||||
typedef void (*speck64_xts_crypt_many_t)(const u32 *, int, void *,
|
||||
const void *, unsigned int, void *);
|
||||
|
||||
static __always_inline int
|
||||
__speck64_xts_crypt(struct blkcipher_desc *desc, struct scatterlist *dst,
|
||||
struct scatterlist *src, unsigned int nbytes,
|
||||
speck64_crypt_one_t crypt_one,
|
||||
speck64_xts_crypt_many_t crypt_many)
|
||||
{
|
||||
struct crypto_blkcipher *tfm = desc->tfm;
|
||||
const struct speck64_xts_tfm_ctx *ctx = crypto_blkcipher_ctx(tfm);
|
||||
struct blkcipher_walk walk;
|
||||
__le64 tweak;
|
||||
int err;
|
||||
|
||||
blkcipher_walk_init(&walk, dst, src, nbytes);
|
||||
err = blkcipher_walk_virt_block(desc, &walk, SPECK_NEON_CHUNK_SIZE);
|
||||
|
||||
crypto_speck64_encrypt(&ctx->tweak_key, (u8 *)&tweak, walk.iv);
|
||||
|
||||
while (walk.nbytes > 0) {
|
||||
unsigned int nbytes = walk.nbytes;
|
||||
u8 *dst = walk.dst.virt.addr;
|
||||
const u8 *src = walk.src.virt.addr;
|
||||
|
||||
if (nbytes >= SPECK_NEON_CHUNK_SIZE && may_use_simd()) {
|
||||
unsigned int count;
|
||||
|
||||
count = round_down(nbytes, SPECK_NEON_CHUNK_SIZE);
|
||||
kernel_neon_begin();
|
||||
(*crypt_many)(ctx->main_key.round_keys,
|
||||
ctx->main_key.nrounds,
|
||||
dst, src, count, &tweak);
|
||||
kernel_neon_end();
|
||||
dst += count;
|
||||
src += count;
|
||||
nbytes -= count;
|
||||
}
|
||||
|
||||
/* Handle any remainder with generic code */
|
||||
while (nbytes >= sizeof(tweak)) {
|
||||
*(__le64 *)dst = *(__le64 *)src ^ tweak;
|
||||
(*crypt_one)(&ctx->main_key, dst, dst);
|
||||
*(__le64 *)dst ^= tweak;
|
||||
tweak = cpu_to_le64((le64_to_cpu(tweak) << 1) ^
|
||||
((tweak & cpu_to_le64(1ULL << 63)) ?
|
||||
0x1B : 0));
|
||||
dst += sizeof(tweak);
|
||||
src += sizeof(tweak);
|
||||
nbytes -= sizeof(tweak);
|
||||
}
|
||||
err = blkcipher_walk_done(desc, &walk, nbytes);
|
||||
}
|
||||
|
||||
return err;
|
||||
}
|
||||
|
||||
static int speck64_xts_encrypt(struct blkcipher_desc *desc,
|
||||
struct scatterlist *dst, struct scatterlist *src,
|
||||
unsigned int nbytes)
|
||||
{
|
||||
return __speck64_xts_crypt(desc, dst, src, nbytes,
|
||||
crypto_speck64_encrypt,
|
||||
speck64_xts_encrypt_neon);
|
||||
}
|
||||
|
||||
static int speck64_xts_decrypt(struct blkcipher_desc *desc,
|
||||
struct scatterlist *dst, struct scatterlist *src,
|
||||
unsigned int nbytes)
|
||||
{
|
||||
return __speck64_xts_crypt(desc, dst, src, nbytes,
|
||||
crypto_speck64_decrypt,
|
||||
speck64_xts_decrypt_neon);
|
||||
}
|
||||
|
||||
static int speck64_xts_setkey(struct crypto_tfm *tfm, const u8 *key,
|
||||
unsigned int keylen)
|
||||
{
|
||||
struct speck64_xts_tfm_ctx *ctx = crypto_tfm_ctx(tfm);
|
||||
int err;
|
||||
|
||||
if (keylen % 2)
|
||||
return -EINVAL;
|
||||
|
||||
keylen /= 2;
|
||||
|
||||
err = crypto_speck64_setkey(&ctx->main_key, key, keylen);
|
||||
if (err)
|
||||
return err;
|
||||
|
||||
return crypto_speck64_setkey(&ctx->tweak_key, key + keylen, keylen);
|
||||
}
|
||||
|
||||
static struct crypto_alg speck_algs[] = {
|
||||
{
|
||||
.cra_name = "xts(speck128)",
|
||||
.cra_driver_name = "xts-speck128-neon",
|
||||
.cra_priority = 300,
|
||||
.cra_flags = CRYPTO_ALG_TYPE_BLKCIPHER,
|
||||
.cra_blocksize = SPECK128_BLOCK_SIZE,
|
||||
.cra_type = &crypto_blkcipher_type,
|
||||
.cra_ctxsize = sizeof(struct speck128_xts_tfm_ctx),
|
||||
.cra_alignmask = 7,
|
||||
.cra_module = THIS_MODULE,
|
||||
.cra_u = {
|
||||
.blkcipher = {
|
||||
.min_keysize = 2 * SPECK128_128_KEY_SIZE,
|
||||
.max_keysize = 2 * SPECK128_256_KEY_SIZE,
|
||||
.ivsize = SPECK128_BLOCK_SIZE,
|
||||
.setkey = speck128_xts_setkey,
|
||||
.encrypt = speck128_xts_encrypt,
|
||||
.decrypt = speck128_xts_decrypt,
|
||||
}
|
||||
}
|
||||
}, {
|
||||
.cra_name = "xts(speck64)",
|
||||
.cra_driver_name = "xts-speck64-neon",
|
||||
.cra_priority = 300,
|
||||
.cra_flags = CRYPTO_ALG_TYPE_BLKCIPHER,
|
||||
.cra_blocksize = SPECK64_BLOCK_SIZE,
|
||||
.cra_type = &crypto_blkcipher_type,
|
||||
.cra_ctxsize = sizeof(struct speck64_xts_tfm_ctx),
|
||||
.cra_alignmask = 7,
|
||||
.cra_module = THIS_MODULE,
|
||||
.cra_u = {
|
||||
.blkcipher = {
|
||||
.min_keysize = 2 * SPECK64_96_KEY_SIZE,
|
||||
.max_keysize = 2 * SPECK64_128_KEY_SIZE,
|
||||
.ivsize = SPECK64_BLOCK_SIZE,
|
||||
.setkey = speck64_xts_setkey,
|
||||
.encrypt = speck64_xts_encrypt,
|
||||
.decrypt = speck64_xts_decrypt,
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
static int __init speck_neon_module_init(void)
|
||||
{
|
||||
if (!(elf_hwcap & HWCAP_NEON))
|
||||
return -ENODEV;
|
||||
return crypto_register_algs(speck_algs, ARRAY_SIZE(speck_algs));
|
||||
}
|
||||
|
||||
static void __exit speck_neon_module_exit(void)
|
||||
{
|
||||
crypto_unregister_algs(speck_algs, ARRAY_SIZE(speck_algs));
|
||||
}
|
||||
|
||||
module_init(speck_neon_module_init);
|
||||
module_exit(speck_neon_module_exit);
|
||||
|
||||
MODULE_DESCRIPTION("Speck block cipher (NEON-accelerated)");
|
||||
MODULE_LICENSE("GPL");
|
||||
MODULE_AUTHOR("Eric Biggers <ebiggers@google.com>");
|
||||
MODULE_ALIAS_CRYPTO("xts(speck128)");
|
||||
MODULE_ALIAS_CRYPTO("xts-speck128-neon");
|
||||
MODULE_ALIAS_CRYPTO("xts(speck64)");
|
||||
MODULE_ALIAS_CRYPTO("xts-speck64-neon");
|
||||
|
|
@ -461,7 +461,7 @@ void pci_ioremap_set_mem_type(int mem_type)
|
|||
|
||||
int pci_ioremap_io(unsigned int offset, phys_addr_t phys_addr)
|
||||
{
|
||||
BUG_ON(offset + SZ_64K > IO_SPACE_LIMIT);
|
||||
BUG_ON(offset + SZ_64K - 1 > IO_SPACE_LIMIT);
|
||||
|
||||
return ioremap_page_range(PCI_IO_VIRT_BASE + offset,
|
||||
PCI_IO_VIRT_BASE + offset + SZ_64K,
|
||||
|
|
|
|||
|
|
@ -58,11 +58,4 @@ config CRYPTO_CRC32_ARM64
|
|||
tristate "CRC32 and CRC32C using optional ARMv8 instructions"
|
||||
depends on ARM64
|
||||
select CRYPTO_HASH
|
||||
|
||||
config CRYPTO_SPECK_NEON
|
||||
tristate "NEON accelerated Speck cipher algorithms"
|
||||
depends on KERNEL_MODE_NEON
|
||||
select CRYPTO_BLKCIPHER
|
||||
select CRYPTO_GF128MUL
|
||||
select CRYPTO_SPECK
|
||||
endif
|
||||
|
|
|
|||
|
|
@ -32,9 +32,6 @@ aes-ce-blk-y := aes-glue-ce.o aes-ce.o
|
|||
obj-$(CONFIG_CRYPTO_AES_ARM64_NEON_BLK) += aes-neon-blk.o
|
||||
aes-neon-blk-y := aes-glue-neon.o aes-neon.o
|
||||
|
||||
obj-$(CONFIG_CRYPTO_SPECK_NEON) += speck-neon.o
|
||||
speck-neon-y := speck-neon-core.o speck-neon-glue.o
|
||||
|
||||
AFLAGS_aes-ce.o := -DINTERLEAVE=4
|
||||
AFLAGS_aes-neon.o := -DINTERLEAVE=4
|
||||
|
||||
|
|
|
|||
|
|
@ -1,352 +0,0 @@
|
|||
// SPDX-License-Identifier: GPL-2.0
|
||||
/*
|
||||
* ARM64 NEON-accelerated implementation of Speck128-XTS and Speck64-XTS
|
||||
*
|
||||
* Copyright (c) 2018 Google, Inc
|
||||
*
|
||||
* Author: Eric Biggers <ebiggers@google.com>
|
||||
*/
|
||||
|
||||
#include <linux/linkage.h>
|
||||
|
||||
.text
|
||||
|
||||
// arguments
|
||||
ROUND_KEYS .req x0 // const {u64,u32} *round_keys
|
||||
NROUNDS .req w1 // int nrounds
|
||||
NROUNDS_X .req x1
|
||||
DST .req x2 // void *dst
|
||||
SRC .req x3 // const void *src
|
||||
NBYTES .req w4 // unsigned int nbytes
|
||||
TWEAK .req x5 // void *tweak
|
||||
|
||||
// registers which hold the data being encrypted/decrypted
|
||||
// (underscores avoid a naming collision with ARM64 registers x0-x3)
|
||||
X_0 .req v0
|
||||
Y_0 .req v1
|
||||
X_1 .req v2
|
||||
Y_1 .req v3
|
||||
X_2 .req v4
|
||||
Y_2 .req v5
|
||||
X_3 .req v6
|
||||
Y_3 .req v7
|
||||
|
||||
// the round key, duplicated in all lanes
|
||||
ROUND_KEY .req v8
|
||||
|
||||
// index vector for tbl-based 8-bit rotates
|
||||
ROTATE_TABLE .req v9
|
||||
ROTATE_TABLE_Q .req q9
|
||||
|
||||
// temporary registers
|
||||
TMP0 .req v10
|
||||
TMP1 .req v11
|
||||
TMP2 .req v12
|
||||
TMP3 .req v13
|
||||
|
||||
// multiplication table for updating XTS tweaks
|
||||
GFMUL_TABLE .req v14
|
||||
GFMUL_TABLE_Q .req q14
|
||||
|
||||
// next XTS tweak value(s)
|
||||
TWEAKV_NEXT .req v15
|
||||
|
||||
// XTS tweaks for the blocks currently being encrypted/decrypted
|
||||
TWEAKV0 .req v16
|
||||
TWEAKV1 .req v17
|
||||
TWEAKV2 .req v18
|
||||
TWEAKV3 .req v19
|
||||
TWEAKV4 .req v20
|
||||
TWEAKV5 .req v21
|
||||
TWEAKV6 .req v22
|
||||
TWEAKV7 .req v23
|
||||
|
||||
.align 4
|
||||
.Lror64_8_table:
|
||||
.octa 0x080f0e0d0c0b0a090007060504030201
|
||||
.Lror32_8_table:
|
||||
.octa 0x0c0f0e0d080b0a090407060500030201
|
||||
.Lrol64_8_table:
|
||||
.octa 0x0e0d0c0b0a09080f0605040302010007
|
||||
.Lrol32_8_table:
|
||||
.octa 0x0e0d0c0f0a09080b0605040702010003
|
||||
.Lgf128mul_table:
|
||||
.octa 0x00000000000000870000000000000001
|
||||
.Lgf64mul_table:
|
||||
.octa 0x0000000000000000000000002d361b00
|
||||
|
||||
/*
|
||||
* _speck_round_128bytes() - Speck encryption round on 128 bytes at a time
|
||||
*
|
||||
* Do one Speck encryption round on the 128 bytes (8 blocks for Speck128, 16 for
|
||||
* Speck64) stored in X0-X3 and Y0-Y3, using the round key stored in all lanes
|
||||
* of ROUND_KEY. 'n' is the lane size: 64 for Speck128, or 32 for Speck64.
|
||||
* 'lanes' is the lane specifier: "2d" for Speck128 or "4s" for Speck64.
|
||||
*/
|
||||
.macro _speck_round_128bytes n, lanes
|
||||
|
||||
// x = ror(x, 8)
|
||||
tbl X_0.16b, {X_0.16b}, ROTATE_TABLE.16b
|
||||
tbl X_1.16b, {X_1.16b}, ROTATE_TABLE.16b
|
||||
tbl X_2.16b, {X_2.16b}, ROTATE_TABLE.16b
|
||||
tbl X_3.16b, {X_3.16b}, ROTATE_TABLE.16b
|
||||
|
||||
// x += y
|
||||
add X_0.\lanes, X_0.\lanes, Y_0.\lanes
|
||||
add X_1.\lanes, X_1.\lanes, Y_1.\lanes
|
||||
add X_2.\lanes, X_2.\lanes, Y_2.\lanes
|
||||
add X_3.\lanes, X_3.\lanes, Y_3.\lanes
|
||||
|
||||
// x ^= k
|
||||
eor X_0.16b, X_0.16b, ROUND_KEY.16b
|
||||
eor X_1.16b, X_1.16b, ROUND_KEY.16b
|
||||
eor X_2.16b, X_2.16b, ROUND_KEY.16b
|
||||
eor X_3.16b, X_3.16b, ROUND_KEY.16b
|
||||
|
||||
// y = rol(y, 3)
|
||||
shl TMP0.\lanes, Y_0.\lanes, #3
|
||||
shl TMP1.\lanes, Y_1.\lanes, #3
|
||||
shl TMP2.\lanes, Y_2.\lanes, #3
|
||||
shl TMP3.\lanes, Y_3.\lanes, #3
|
||||
sri TMP0.\lanes, Y_0.\lanes, #(\n - 3)
|
||||
sri TMP1.\lanes, Y_1.\lanes, #(\n - 3)
|
||||
sri TMP2.\lanes, Y_2.\lanes, #(\n - 3)
|
||||
sri TMP3.\lanes, Y_3.\lanes, #(\n - 3)
|
||||
|
||||
// y ^= x
|
||||
eor Y_0.16b, TMP0.16b, X_0.16b
|
||||
eor Y_1.16b, TMP1.16b, X_1.16b
|
||||
eor Y_2.16b, TMP2.16b, X_2.16b
|
||||
eor Y_3.16b, TMP3.16b, X_3.16b
|
||||
.endm
|
||||
|
||||
/*
|
||||
* _speck_unround_128bytes() - Speck decryption round on 128 bytes at a time
|
||||
*
|
||||
* This is the inverse of _speck_round_128bytes().
|
||||
*/
|
||||
.macro _speck_unround_128bytes n, lanes
|
||||
|
||||
// y ^= x
|
||||
eor TMP0.16b, Y_0.16b, X_0.16b
|
||||
eor TMP1.16b, Y_1.16b, X_1.16b
|
||||
eor TMP2.16b, Y_2.16b, X_2.16b
|
||||
eor TMP3.16b, Y_3.16b, X_3.16b
|
||||
|
||||
// y = ror(y, 3)
|
||||
ushr Y_0.\lanes, TMP0.\lanes, #3
|
||||
ushr Y_1.\lanes, TMP1.\lanes, #3
|
||||
ushr Y_2.\lanes, TMP2.\lanes, #3
|
||||
ushr Y_3.\lanes, TMP3.\lanes, #3
|
||||
sli Y_0.\lanes, TMP0.\lanes, #(\n - 3)
|
||||
sli Y_1.\lanes, TMP1.\lanes, #(\n - 3)
|
||||
sli Y_2.\lanes, TMP2.\lanes, #(\n - 3)
|
||||
sli Y_3.\lanes, TMP3.\lanes, #(\n - 3)
|
||||
|
||||
// x ^= k
|
||||
eor X_0.16b, X_0.16b, ROUND_KEY.16b
|
||||
eor X_1.16b, X_1.16b, ROUND_KEY.16b
|
||||
eor X_2.16b, X_2.16b, ROUND_KEY.16b
|
||||
eor X_3.16b, X_3.16b, ROUND_KEY.16b
|
||||
|
||||
// x -= y
|
||||
sub X_0.\lanes, X_0.\lanes, Y_0.\lanes
|
||||
sub X_1.\lanes, X_1.\lanes, Y_1.\lanes
|
||||
sub X_2.\lanes, X_2.\lanes, Y_2.\lanes
|
||||
sub X_3.\lanes, X_3.\lanes, Y_3.\lanes
|
||||
|
||||
// x = rol(x, 8)
|
||||
tbl X_0.16b, {X_0.16b}, ROTATE_TABLE.16b
|
||||
tbl X_1.16b, {X_1.16b}, ROTATE_TABLE.16b
|
||||
tbl X_2.16b, {X_2.16b}, ROTATE_TABLE.16b
|
||||
tbl X_3.16b, {X_3.16b}, ROTATE_TABLE.16b
|
||||
.endm
|
||||
|
||||
.macro _next_xts_tweak next, cur, tmp, n
|
||||
.if \n == 64
|
||||
/*
|
||||
* Calculate the next tweak by multiplying the current one by x,
|
||||
* modulo p(x) = x^128 + x^7 + x^2 + x + 1.
|
||||
*/
|
||||
sshr \tmp\().2d, \cur\().2d, #63
|
||||
and \tmp\().16b, \tmp\().16b, GFMUL_TABLE.16b
|
||||
shl \next\().2d, \cur\().2d, #1
|
||||
ext \tmp\().16b, \tmp\().16b, \tmp\().16b, #8
|
||||
eor \next\().16b, \next\().16b, \tmp\().16b
|
||||
.else
|
||||
/*
|
||||
* Calculate the next two tweaks by multiplying the current ones by x^2,
|
||||
* modulo p(x) = x^64 + x^4 + x^3 + x + 1.
|
||||
*/
|
||||
ushr \tmp\().2d, \cur\().2d, #62
|
||||
shl \next\().2d, \cur\().2d, #2
|
||||
tbl \tmp\().16b, {GFMUL_TABLE.16b}, \tmp\().16b
|
||||
eor \next\().16b, \next\().16b, \tmp\().16b
|
||||
.endif
|
||||
.endm
|
||||
|
||||
/*
|
||||
* _speck_xts_crypt() - Speck-XTS encryption/decryption
|
||||
*
|
||||
* Encrypt or decrypt NBYTES bytes of data from the SRC buffer to the DST buffer
|
||||
* using Speck-XTS, specifically the variant with a block size of '2n' and round
|
||||
* count given by NROUNDS. The expanded round keys are given in ROUND_KEYS, and
|
||||
* the current XTS tweak value is given in TWEAK. It's assumed that NBYTES is a
|
||||
* nonzero multiple of 128.
|
||||
*/
|
||||
.macro _speck_xts_crypt n, lanes, decrypting
|
||||
|
||||
/*
|
||||
* If decrypting, modify the ROUND_KEYS parameter to point to the last
|
||||
* round key rather than the first, since for decryption the round keys
|
||||
* are used in reverse order.
|
||||
*/
|
||||
.if \decrypting
|
||||
mov NROUNDS, NROUNDS /* zero the high 32 bits */
|
||||
.if \n == 64
|
||||
add ROUND_KEYS, ROUND_KEYS, NROUNDS_X, lsl #3
|
||||
sub ROUND_KEYS, ROUND_KEYS, #8
|
||||
.else
|
||||
add ROUND_KEYS, ROUND_KEYS, NROUNDS_X, lsl #2
|
||||
sub ROUND_KEYS, ROUND_KEYS, #4
|
||||
.endif
|
||||
.endif
|
||||
|
||||
// Load the index vector for tbl-based 8-bit rotates
|
||||
.if \decrypting
|
||||
ldr ROTATE_TABLE_Q, .Lrol\n\()_8_table
|
||||
.else
|
||||
ldr ROTATE_TABLE_Q, .Lror\n\()_8_table
|
||||
.endif
|
||||
|
||||
// One-time XTS preparation
|
||||
.if \n == 64
|
||||
// Load first tweak
|
||||
ld1 {TWEAKV0.16b}, [TWEAK]
|
||||
|
||||
// Load GF(2^128) multiplication table
|
||||
ldr GFMUL_TABLE_Q, .Lgf128mul_table
|
||||
.else
|
||||
// Load first tweak
|
||||
ld1 {TWEAKV0.8b}, [TWEAK]
|
||||
|
||||
// Load GF(2^64) multiplication table
|
||||
ldr GFMUL_TABLE_Q, .Lgf64mul_table
|
||||
|
||||
// Calculate second tweak, packing it together with the first
|
||||
ushr TMP0.2d, TWEAKV0.2d, #63
|
||||
shl TMP1.2d, TWEAKV0.2d, #1
|
||||
tbl TMP0.8b, {GFMUL_TABLE.16b}, TMP0.8b
|
||||
eor TMP0.8b, TMP0.8b, TMP1.8b
|
||||
mov TWEAKV0.d[1], TMP0.d[0]
|
||||
.endif
|
||||
|
||||
.Lnext_128bytes_\@:
|
||||
|
||||
// Calculate XTS tweaks for next 128 bytes
|
||||
_next_xts_tweak TWEAKV1, TWEAKV0, TMP0, \n
|
||||
_next_xts_tweak TWEAKV2, TWEAKV1, TMP0, \n
|
||||
_next_xts_tweak TWEAKV3, TWEAKV2, TMP0, \n
|
||||
_next_xts_tweak TWEAKV4, TWEAKV3, TMP0, \n
|
||||
_next_xts_tweak TWEAKV5, TWEAKV4, TMP0, \n
|
||||
_next_xts_tweak TWEAKV6, TWEAKV5, TMP0, \n
|
||||
_next_xts_tweak TWEAKV7, TWEAKV6, TMP0, \n
|
||||
_next_xts_tweak TWEAKV_NEXT, TWEAKV7, TMP0, \n
|
||||
|
||||
// Load the next source blocks into {X,Y}[0-3]
|
||||
ld1 {X_0.16b-Y_1.16b}, [SRC], #64
|
||||
ld1 {X_2.16b-Y_3.16b}, [SRC], #64
|
||||
|
||||
// XOR the source blocks with their XTS tweaks
|
||||
eor TMP0.16b, X_0.16b, TWEAKV0.16b
|
||||
eor Y_0.16b, Y_0.16b, TWEAKV1.16b
|
||||
eor TMP1.16b, X_1.16b, TWEAKV2.16b
|
||||
eor Y_1.16b, Y_1.16b, TWEAKV3.16b
|
||||
eor TMP2.16b, X_2.16b, TWEAKV4.16b
|
||||
eor Y_2.16b, Y_2.16b, TWEAKV5.16b
|
||||
eor TMP3.16b, X_3.16b, TWEAKV6.16b
|
||||
eor Y_3.16b, Y_3.16b, TWEAKV7.16b
|
||||
|
||||
/*
|
||||
* De-interleave the 'x' and 'y' elements of each block, i.e. make it so
|
||||
* that the X[0-3] registers contain only the second halves of blocks,
|
||||
* and the Y[0-3] registers contain only the first halves of blocks.
|
||||
* (Speck uses the order (y, x) rather than the more intuitive (x, y).)
|
||||
*/
|
||||
uzp2 X_0.\lanes, TMP0.\lanes, Y_0.\lanes
|
||||
uzp1 Y_0.\lanes, TMP0.\lanes, Y_0.\lanes
|
||||
uzp2 X_1.\lanes, TMP1.\lanes, Y_1.\lanes
|
||||
uzp1 Y_1.\lanes, TMP1.\lanes, Y_1.\lanes
|
||||
uzp2 X_2.\lanes, TMP2.\lanes, Y_2.\lanes
|
||||
uzp1 Y_2.\lanes, TMP2.\lanes, Y_2.\lanes
|
||||
uzp2 X_3.\lanes, TMP3.\lanes, Y_3.\lanes
|
||||
uzp1 Y_3.\lanes, TMP3.\lanes, Y_3.\lanes
|
||||
|
||||
// Do the cipher rounds
|
||||
mov x6, ROUND_KEYS
|
||||
mov w7, NROUNDS
|
||||
.Lnext_round_\@:
|
||||
.if \decrypting
|
||||
ld1r {ROUND_KEY.\lanes}, [x6]
|
||||
sub x6, x6, #( \n / 8 )
|
||||
_speck_unround_128bytes \n, \lanes
|
||||
.else
|
||||
ld1r {ROUND_KEY.\lanes}, [x6], #( \n / 8 )
|
||||
_speck_round_128bytes \n, \lanes
|
||||
.endif
|
||||
subs w7, w7, #1
|
||||
bne .Lnext_round_\@
|
||||
|
||||
// Re-interleave the 'x' and 'y' elements of each block
|
||||
zip1 TMP0.\lanes, Y_0.\lanes, X_0.\lanes
|
||||
zip2 Y_0.\lanes, Y_0.\lanes, X_0.\lanes
|
||||
zip1 TMP1.\lanes, Y_1.\lanes, X_1.\lanes
|
||||
zip2 Y_1.\lanes, Y_1.\lanes, X_1.\lanes
|
||||
zip1 TMP2.\lanes, Y_2.\lanes, X_2.\lanes
|
||||
zip2 Y_2.\lanes, Y_2.\lanes, X_2.\lanes
|
||||
zip1 TMP3.\lanes, Y_3.\lanes, X_3.\lanes
|
||||
zip2 Y_3.\lanes, Y_3.\lanes, X_3.\lanes
|
||||
|
||||
// XOR the encrypted/decrypted blocks with the tweaks calculated earlier
|
||||
eor X_0.16b, TMP0.16b, TWEAKV0.16b
|
||||
eor Y_0.16b, Y_0.16b, TWEAKV1.16b
|
||||
eor X_1.16b, TMP1.16b, TWEAKV2.16b
|
||||
eor Y_1.16b, Y_1.16b, TWEAKV3.16b
|
||||
eor X_2.16b, TMP2.16b, TWEAKV4.16b
|
||||
eor Y_2.16b, Y_2.16b, TWEAKV5.16b
|
||||
eor X_3.16b, TMP3.16b, TWEAKV6.16b
|
||||
eor Y_3.16b, Y_3.16b, TWEAKV7.16b
|
||||
mov TWEAKV0.16b, TWEAKV_NEXT.16b
|
||||
|
||||
// Store the ciphertext in the destination buffer
|
||||
st1 {X_0.16b-Y_1.16b}, [DST], #64
|
||||
st1 {X_2.16b-Y_3.16b}, [DST], #64
|
||||
|
||||
// Continue if there are more 128-byte chunks remaining
|
||||
subs NBYTES, NBYTES, #128
|
||||
bne .Lnext_128bytes_\@
|
||||
|
||||
// Store the next tweak and return
|
||||
.if \n == 64
|
||||
st1 {TWEAKV_NEXT.16b}, [TWEAK]
|
||||
.else
|
||||
st1 {TWEAKV_NEXT.8b}, [TWEAK]
|
||||
.endif
|
||||
ret
|
||||
.endm
|
||||
|
||||
ENTRY(speck128_xts_encrypt_neon)
|
||||
_speck_xts_crypt n=64, lanes=2d, decrypting=0
|
||||
ENDPROC(speck128_xts_encrypt_neon)
|
||||
|
||||
ENTRY(speck128_xts_decrypt_neon)
|
||||
_speck_xts_crypt n=64, lanes=2d, decrypting=1
|
||||
ENDPROC(speck128_xts_decrypt_neon)
|
||||
|
||||
ENTRY(speck64_xts_encrypt_neon)
|
||||
_speck_xts_crypt n=32, lanes=4s, decrypting=0
|
||||
ENDPROC(speck64_xts_encrypt_neon)
|
||||
|
||||
ENTRY(speck64_xts_decrypt_neon)
|
||||
_speck_xts_crypt n=32, lanes=4s, decrypting=1
|
||||
ENDPROC(speck64_xts_decrypt_neon)
|
||||
|
|
@ -1,308 +0,0 @@
|
|||
// SPDX-License-Identifier: GPL-2.0
|
||||
/*
|
||||
* NEON-accelerated implementation of Speck128-XTS and Speck64-XTS
|
||||
* (64-bit version; based on the 32-bit version)
|
||||
*
|
||||
* Copyright (c) 2018 Google, Inc
|
||||
*/
|
||||
|
||||
#include <asm/hwcap.h>
|
||||
#include <asm/neon.h>
|
||||
#include <asm/simd.h>
|
||||
#include <crypto/algapi.h>
|
||||
#include <crypto/gf128mul.h>
|
||||
#include <crypto/speck.h>
|
||||
#include <crypto/xts.h>
|
||||
#include <linux/kernel.h>
|
||||
#include <linux/module.h>
|
||||
|
||||
/* The assembly functions only handle multiples of 128 bytes */
|
||||
#define SPECK_NEON_CHUNK_SIZE 128
|
||||
|
||||
/* Speck128 */
|
||||
|
||||
struct speck128_xts_tfm_ctx {
|
||||
struct speck128_tfm_ctx main_key;
|
||||
struct speck128_tfm_ctx tweak_key;
|
||||
};
|
||||
|
||||
asmlinkage void speck128_xts_encrypt_neon(const u64 *round_keys, int nrounds,
|
||||
void *dst, const void *src,
|
||||
unsigned int nbytes, void *tweak);
|
||||
|
||||
asmlinkage void speck128_xts_decrypt_neon(const u64 *round_keys, int nrounds,
|
||||
void *dst, const void *src,
|
||||
unsigned int nbytes, void *tweak);
|
||||
|
||||
typedef void (*speck128_crypt_one_t)(const struct speck128_tfm_ctx *,
|
||||
u8 *, const u8 *);
|
||||
typedef void (*speck128_xts_crypt_many_t)(const u64 *, int, void *,
|
||||
const void *, unsigned int, void *);
|
||||
|
||||
static __always_inline int
|
||||
__speck128_xts_crypt(struct blkcipher_desc *desc, struct scatterlist *dst,
|
||||
struct scatterlist *src, unsigned int nbytes,
|
||||
speck128_crypt_one_t crypt_one,
|
||||
speck128_xts_crypt_many_t crypt_many)
|
||||
{
|
||||
struct crypto_blkcipher *tfm = desc->tfm;
|
||||
const struct speck128_xts_tfm_ctx *ctx = crypto_blkcipher_ctx(tfm);
|
||||
struct blkcipher_walk walk;
|
||||
le128 tweak;
|
||||
int err;
|
||||
|
||||
blkcipher_walk_init(&walk, dst, src, nbytes);
|
||||
err = blkcipher_walk_virt_block(desc, &walk, SPECK_NEON_CHUNK_SIZE);
|
||||
|
||||
crypto_speck128_encrypt(&ctx->tweak_key, (u8 *)&tweak, walk.iv);
|
||||
|
||||
while (walk.nbytes > 0) {
|
||||
unsigned int nbytes = walk.nbytes;
|
||||
u8 *dst = walk.dst.virt.addr;
|
||||
const u8 *src = walk.src.virt.addr;
|
||||
|
||||
if (nbytes >= SPECK_NEON_CHUNK_SIZE && may_use_simd()) {
|
||||
unsigned int count;
|
||||
|
||||
count = round_down(nbytes, SPECK_NEON_CHUNK_SIZE);
|
||||
kernel_neon_begin();
|
||||
(*crypt_many)(ctx->main_key.round_keys,
|
||||
ctx->main_key.nrounds,
|
||||
dst, src, count, &tweak);
|
||||
kernel_neon_end();
|
||||
dst += count;
|
||||
src += count;
|
||||
nbytes -= count;
|
||||
}
|
||||
|
||||
/* Handle any remainder with generic code */
|
||||
while (nbytes >= sizeof(tweak)) {
|
||||
le128_xor((le128 *)dst, (const le128 *)src, &tweak);
|
||||
(*crypt_one)(&ctx->main_key, dst, dst);
|
||||
le128_xor((le128 *)dst, (const le128 *)dst, &tweak);
|
||||
gf128mul_x_ble((be128 *)&tweak, (const be128 *)&tweak);
|
||||
|
||||
dst += sizeof(tweak);
|
||||
src += sizeof(tweak);
|
||||
nbytes -= sizeof(tweak);
|
||||
}
|
||||
err = blkcipher_walk_done(desc, &walk, nbytes);
|
||||
}
|
||||
|
||||
return err;
|
||||
}
|
||||
|
||||
static int speck128_xts_encrypt(struct blkcipher_desc *desc,
|
||||
struct scatterlist *dst,
|
||||
struct scatterlist *src,
|
||||
unsigned int nbytes)
|
||||
{
|
||||
return __speck128_xts_crypt(desc, dst, src, nbytes,
|
||||
crypto_speck128_encrypt,
|
||||
speck128_xts_encrypt_neon);
|
||||
}
|
||||
|
||||
static int speck128_xts_decrypt(struct blkcipher_desc *desc,
|
||||
struct scatterlist *dst,
|
||||
struct scatterlist *src,
|
||||
unsigned int nbytes)
|
||||
{
|
||||
return __speck128_xts_crypt(desc, dst, src, nbytes,
|
||||
crypto_speck128_decrypt,
|
||||
speck128_xts_decrypt_neon);
|
||||
}
|
||||
|
||||
static int speck128_xts_setkey(struct crypto_tfm *tfm, const u8 *key,
|
||||
unsigned int keylen)
|
||||
{
|
||||
struct speck128_xts_tfm_ctx *ctx = crypto_tfm_ctx(tfm);
|
||||
int err;
|
||||
|
||||
if (keylen % 2)
|
||||
return -EINVAL;
|
||||
|
||||
keylen /= 2;
|
||||
|
||||
err = crypto_speck128_setkey(&ctx->main_key, key, keylen);
|
||||
if (err)
|
||||
return err;
|
||||
|
||||
return crypto_speck128_setkey(&ctx->tweak_key, key + keylen, keylen);
|
||||
}
|
||||
|
||||
/* Speck64 */
|
||||
|
||||
struct speck64_xts_tfm_ctx {
|
||||
struct speck64_tfm_ctx main_key;
|
||||
struct speck64_tfm_ctx tweak_key;
|
||||
};
|
||||
|
||||
asmlinkage void speck64_xts_encrypt_neon(const u32 *round_keys, int nrounds,
|
||||
void *dst, const void *src,
|
||||
unsigned int nbytes, void *tweak);
|
||||
|
||||
asmlinkage void speck64_xts_decrypt_neon(const u32 *round_keys, int nrounds,
|
||||
void *dst, const void *src,
|
||||
unsigned int nbytes, void *tweak);
|
||||
|
||||
typedef void (*speck64_crypt_one_t)(const struct speck64_tfm_ctx *,
|
||||
u8 *, const u8 *);
|
||||
typedef void (*speck64_xts_crypt_many_t)(const u32 *, int, void *,
|
||||
const void *, unsigned int, void *);
|
||||
|
||||
static __always_inline int
|
||||
__speck64_xts_crypt(struct blkcipher_desc *desc, struct scatterlist *dst,
|
||||
struct scatterlist *src, unsigned int nbytes,
|
||||
speck64_crypt_one_t crypt_one,
|
||||
speck64_xts_crypt_many_t crypt_many)
|
||||
{
|
||||
struct crypto_blkcipher *tfm = desc->tfm;
|
||||
const struct speck64_xts_tfm_ctx *ctx = crypto_blkcipher_ctx(tfm);
|
||||
struct blkcipher_walk walk;
|
||||
__le64 tweak;
|
||||
int err;
|
||||
|
||||
blkcipher_walk_init(&walk, dst, src, nbytes);
|
||||
err = blkcipher_walk_virt_block(desc, &walk, SPECK_NEON_CHUNK_SIZE);
|
||||
|
||||
crypto_speck64_encrypt(&ctx->tweak_key, (u8 *)&tweak, walk.iv);
|
||||
|
||||
while (walk.nbytes > 0) {
|
||||
unsigned int nbytes = walk.nbytes;
|
||||
u8 *dst = walk.dst.virt.addr;
|
||||
const u8 *src = walk.src.virt.addr;
|
||||
|
||||
if (nbytes >= SPECK_NEON_CHUNK_SIZE && may_use_simd()) {
|
||||
unsigned int count;
|
||||
|
||||
count = round_down(nbytes, SPECK_NEON_CHUNK_SIZE);
|
||||
kernel_neon_begin();
|
||||
(*crypt_many)(ctx->main_key.round_keys,
|
||||
ctx->main_key.nrounds,
|
||||
dst, src, count, &tweak);
|
||||
kernel_neon_end();
|
||||
dst += count;
|
||||
src += count;
|
||||
nbytes -= count;
|
||||
}
|
||||
|
||||
/* Handle any remainder with generic code */
|
||||
while (nbytes >= sizeof(tweak)) {
|
||||
*(__le64 *)dst = *(__le64 *)src ^ tweak;
|
||||
(*crypt_one)(&ctx->main_key, dst, dst);
|
||||
*(__le64 *)dst ^= tweak;
|
||||
tweak = cpu_to_le64((le64_to_cpu(tweak) << 1) ^
|
||||
((tweak & cpu_to_le64(1ULL << 63)) ?
|
||||
0x1B : 0));
|
||||
dst += sizeof(tweak);
|
||||
src += sizeof(tweak);
|
||||
nbytes -= sizeof(tweak);
|
||||
}
|
||||
err = blkcipher_walk_done(desc, &walk, nbytes);
|
||||
}
|
||||
|
||||
return err;
|
||||
}
|
||||
|
||||
static int speck64_xts_encrypt(struct blkcipher_desc *desc,
|
||||
struct scatterlist *dst, struct scatterlist *src,
|
||||
unsigned int nbytes)
|
||||
{
|
||||
return __speck64_xts_crypt(desc, dst, src, nbytes,
|
||||
crypto_speck64_encrypt,
|
||||
speck64_xts_encrypt_neon);
|
||||
}
|
||||
|
||||
static int speck64_xts_decrypt(struct blkcipher_desc *desc,
|
||||
struct scatterlist *dst, struct scatterlist *src,
|
||||
unsigned int nbytes)
|
||||
{
|
||||
return __speck64_xts_crypt(desc, dst, src, nbytes,
|
||||
crypto_speck64_decrypt,
|
||||
speck64_xts_decrypt_neon);
|
||||
}
|
||||
|
||||
static int speck64_xts_setkey(struct crypto_tfm *tfm, const u8 *key,
|
||||
unsigned int keylen)
|
||||
{
|
||||
struct speck64_xts_tfm_ctx *ctx = crypto_tfm_ctx(tfm);
|
||||
int err;
|
||||
|
||||
if (keylen % 2)
|
||||
return -EINVAL;
|
||||
|
||||
keylen /= 2;
|
||||
|
||||
err = crypto_speck64_setkey(&ctx->main_key, key, keylen);
|
||||
if (err)
|
||||
return err;
|
||||
|
||||
return crypto_speck64_setkey(&ctx->tweak_key, key + keylen, keylen);
|
||||
}
|
||||
|
||||
static struct crypto_alg speck_algs[] = {
|
||||
{
|
||||
.cra_name = "xts(speck128)",
|
||||
.cra_driver_name = "xts-speck128-neon",
|
||||
.cra_priority = 300,
|
||||
.cra_flags = CRYPTO_ALG_TYPE_BLKCIPHER,
|
||||
.cra_blocksize = SPECK128_BLOCK_SIZE,
|
||||
.cra_type = &crypto_blkcipher_type,
|
||||
.cra_ctxsize = sizeof(struct speck128_xts_tfm_ctx),
|
||||
.cra_alignmask = 7,
|
||||
.cra_module = THIS_MODULE,
|
||||
.cra_u = {
|
||||
.blkcipher = {
|
||||
.min_keysize = 2 * SPECK128_128_KEY_SIZE,
|
||||
.max_keysize = 2 * SPECK128_256_KEY_SIZE,
|
||||
.ivsize = SPECK128_BLOCK_SIZE,
|
||||
.setkey = speck128_xts_setkey,
|
||||
.encrypt = speck128_xts_encrypt,
|
||||
.decrypt = speck128_xts_decrypt,
|
||||
}
|
||||
}
|
||||
}, {
|
||||
.cra_name = "xts(speck64)",
|
||||
.cra_driver_name = "xts-speck64-neon",
|
||||
.cra_priority = 300,
|
||||
.cra_flags = CRYPTO_ALG_TYPE_BLKCIPHER,
|
||||
.cra_blocksize = SPECK64_BLOCK_SIZE,
|
||||
.cra_type = &crypto_blkcipher_type,
|
||||
.cra_ctxsize = sizeof(struct speck64_xts_tfm_ctx),
|
||||
.cra_alignmask = 7,
|
||||
.cra_module = THIS_MODULE,
|
||||
.cra_u = {
|
||||
.blkcipher = {
|
||||
.min_keysize = 2 * SPECK64_96_KEY_SIZE,
|
||||
.max_keysize = 2 * SPECK64_128_KEY_SIZE,
|
||||
.ivsize = SPECK64_BLOCK_SIZE,
|
||||
.setkey = speck64_xts_setkey,
|
||||
.encrypt = speck64_xts_encrypt,
|
||||
.decrypt = speck64_xts_decrypt,
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
static int __init speck_neon_module_init(void)
|
||||
{
|
||||
if (!(elf_hwcap & HWCAP_ASIMD))
|
||||
return -ENODEV;
|
||||
return crypto_register_algs(speck_algs, ARRAY_SIZE(speck_algs));
|
||||
}
|
||||
|
||||
static void __exit speck_neon_module_exit(void)
|
||||
{
|
||||
crypto_unregister_algs(speck_algs, ARRAY_SIZE(speck_algs));
|
||||
}
|
||||
|
||||
module_init(speck_neon_module_init);
|
||||
module_exit(speck_neon_module_exit);
|
||||
|
||||
MODULE_DESCRIPTION("Speck block cipher (NEON-accelerated)");
|
||||
MODULE_LICENSE("GPL");
|
||||
MODULE_AUTHOR("Eric Biggers <ebiggers@google.com>");
|
||||
MODULE_ALIAS_CRYPTO("xts(speck128)");
|
||||
MODULE_ALIAS_CRYPTO("xts-speck128-neon");
|
||||
MODULE_ALIAS_CRYPTO("xts(speck64)");
|
||||
MODULE_ALIAS_CRYPTO("xts-speck64-neon");
|
||||
|
|
@ -133,26 +133,27 @@ int ptep_set_access_flags(struct vm_area_struct *vma,
|
|||
/* only preserve the access flags and write permission */
|
||||
pte_val(entry) &= PTE_AF | PTE_WRITE | PTE_DIRTY;
|
||||
|
||||
/*
|
||||
* PTE_RDONLY is cleared by default in the asm below, so set it in
|
||||
* back if necessary (read-only or clean PTE).
|
||||
*/
|
||||
/* set PTE_RDONLY if actual read-only or clean PTE */
|
||||
if (!pte_write(entry) || !pte_sw_dirty(entry))
|
||||
pte_val(entry) |= PTE_RDONLY;
|
||||
|
||||
/*
|
||||
* Setting the flags must be done atomically to avoid racing with the
|
||||
* hardware update of the access/dirty state.
|
||||
* hardware update of the access/dirty state. The PTE_RDONLY bit must
|
||||
* be set to the most permissive (lowest value) of *ptep and entry
|
||||
* (calculated as: a & b == ~(~a | ~b)).
|
||||
*/
|
||||
pte_val(entry) ^= PTE_RDONLY;
|
||||
asm volatile("// ptep_set_access_flags\n"
|
||||
" prfm pstl1strm, %2\n"
|
||||
"1: ldxr %0, %2\n"
|
||||
" and %0, %0, %3 // clear PTE_RDONLY\n"
|
||||
" eor %0, %0, %3 // negate PTE_RDONLY in *ptep\n"
|
||||
" orr %0, %0, %4 // set flags\n"
|
||||
" eor %0, %0, %3 // negate final PTE_RDONLY\n"
|
||||
" stxr %w1, %0, %2\n"
|
||||
" cbnz %w1, 1b\n"
|
||||
: "=&r" (old_pteval), "=&r" (tmp), "+Q" (pte_val(*ptep))
|
||||
: "L" (~PTE_RDONLY), "r" (pte_val(entry)));
|
||||
: "L" (PTE_RDONLY), "r" (pte_val(entry)));
|
||||
|
||||
flush_tlb_fix_spurious_fault(vma, address);
|
||||
return 1;
|
||||
|
|
|
|||
|
|
@ -147,23 +147,12 @@
|
|||
* Find irq with highest priority
|
||||
*/
|
||||
# open coded PTR_LA t1, cpu_mask_nr_tbl
|
||||
#if (_MIPS_SZPTR == 32)
|
||||
#if defined(CONFIG_32BIT) || defined(KBUILD_64BIT_SYM32)
|
||||
# open coded la t1, cpu_mask_nr_tbl
|
||||
lui t1, %hi(cpu_mask_nr_tbl)
|
||||
addiu t1, %lo(cpu_mask_nr_tbl)
|
||||
|
||||
#endif
|
||||
#if (_MIPS_SZPTR == 64)
|
||||
# open coded dla t1, cpu_mask_nr_tbl
|
||||
.set push
|
||||
.set noat
|
||||
lui t1, %highest(cpu_mask_nr_tbl)
|
||||
lui AT, %hi(cpu_mask_nr_tbl)
|
||||
daddiu t1, t1, %higher(cpu_mask_nr_tbl)
|
||||
daddiu AT, AT, %lo(cpu_mask_nr_tbl)
|
||||
dsll t1, 32
|
||||
daddu t1, t1, AT
|
||||
.set pop
|
||||
#else
|
||||
#error GCC `-msym32' option required for 64-bit DECstation builds
|
||||
#endif
|
||||
1: lw t2,(t1)
|
||||
nop
|
||||
|
|
@ -214,23 +203,12 @@
|
|||
* Find irq with highest priority
|
||||
*/
|
||||
# open coded PTR_LA t1,asic_mask_nr_tbl
|
||||
#if (_MIPS_SZPTR == 32)
|
||||
#if defined(CONFIG_32BIT) || defined(KBUILD_64BIT_SYM32)
|
||||
# open coded la t1, asic_mask_nr_tbl
|
||||
lui t1, %hi(asic_mask_nr_tbl)
|
||||
addiu t1, %lo(asic_mask_nr_tbl)
|
||||
|
||||
#endif
|
||||
#if (_MIPS_SZPTR == 64)
|
||||
# open coded dla t1, asic_mask_nr_tbl
|
||||
.set push
|
||||
.set noat
|
||||
lui t1, %highest(asic_mask_nr_tbl)
|
||||
lui AT, %hi(asic_mask_nr_tbl)
|
||||
daddiu t1, t1, %higher(asic_mask_nr_tbl)
|
||||
daddiu AT, AT, %lo(asic_mask_nr_tbl)
|
||||
dsll t1, 32
|
||||
daddu t1, t1, AT
|
||||
.set pop
|
||||
#else
|
||||
#error GCC `-msym32' option required for 64-bit DECstation builds
|
||||
#endif
|
||||
2: lw t2,(t1)
|
||||
nop
|
||||
|
|
|
|||
|
|
@ -864,7 +864,7 @@ struct mm16_r3_format { /* Load from global pointer format */
|
|||
struct mm16_r5_format { /* Load/store from stack pointer format */
|
||||
__BITFIELD_FIELD(unsigned int opcode : 6,
|
||||
__BITFIELD_FIELD(unsigned int rt : 5,
|
||||
__BITFIELD_FIELD(signed int simmediate : 5,
|
||||
__BITFIELD_FIELD(unsigned int imm : 5,
|
||||
__BITFIELD_FIELD(unsigned int : 16, /* Ignored */
|
||||
;))))
|
||||
};
|
||||
|
|
|
|||
|
|
@ -211,7 +211,7 @@ static inline int is_ra_save_ins(union mips_instruction *ip, int *poff)
|
|||
if (ip->mm16_r5_format.rt != 31)
|
||||
return 0;
|
||||
|
||||
*poff = ip->mm16_r5_format.simmediate;
|
||||
*poff = ip->mm16_r5_format.imm;
|
||||
*poff = (*poff << 2) / sizeof(ulong);
|
||||
return 1;
|
||||
|
||||
|
|
@ -345,6 +345,7 @@ static int get_frame_info(struct mips_frame_info *info)
|
|||
bool is_mmips = IS_ENABLED(CONFIG_CPU_MICROMIPS);
|
||||
union mips_instruction insn, *ip, *ip_end;
|
||||
const unsigned int max_insns = 128;
|
||||
unsigned int last_insn_size = 0;
|
||||
unsigned int i;
|
||||
|
||||
info->pc_offset = -1;
|
||||
|
|
@ -356,15 +357,19 @@ static int get_frame_info(struct mips_frame_info *info)
|
|||
|
||||
ip_end = (void *)ip + info->func_size;
|
||||
|
||||
for (i = 0; i < max_insns && ip < ip_end; i++, ip++) {
|
||||
for (i = 0; i < max_insns && ip < ip_end; i++) {
|
||||
ip = (void *)ip + last_insn_size;
|
||||
if (is_mmips && mm_insn_16bit(ip->halfword[0])) {
|
||||
insn.halfword[0] = 0;
|
||||
insn.halfword[1] = ip->halfword[0];
|
||||
last_insn_size = 2;
|
||||
} else if (is_mmips) {
|
||||
insn.halfword[0] = ip->halfword[1];
|
||||
insn.halfword[1] = ip->halfword[0];
|
||||
last_insn_size = 4;
|
||||
} else {
|
||||
insn.word = ip->word;
|
||||
last_insn_size = 4;
|
||||
}
|
||||
|
||||
if (is_jump_ins(&insn))
|
||||
|
|
@ -386,8 +391,6 @@ static int get_frame_info(struct mips_frame_info *info)
|
|||
tmp = (ip->halfword[0] >> 1);
|
||||
info->frame_size = -(signed short)(tmp & 0xf);
|
||||
}
|
||||
ip = (void *) &ip->halfword[1];
|
||||
ip--;
|
||||
} else
|
||||
#endif
|
||||
info->frame_size = - ip->i_format.simmediate;
|
||||
|
|
|
|||
|
|
@ -25,6 +25,7 @@
|
|||
#define HPAGE_MASK (~(HPAGE_SIZE - 1UL))
|
||||
#define HUGETLB_PAGE_ORDER (HPAGE_SHIFT - PAGE_SHIFT)
|
||||
#define HAVE_ARCH_HUGETLB_UNMAPPED_AREA
|
||||
#define REAL_HPAGE_PER_HPAGE (_AC(1,UL) << (HPAGE_SHIFT - REAL_HPAGE_SHIFT))
|
||||
#endif
|
||||
|
||||
#ifndef __ASSEMBLY__
|
||||
|
|
|
|||
|
|
@ -245,6 +245,18 @@ static void pci_parse_of_addrs(struct platform_device *op,
|
|||
}
|
||||
}
|
||||
|
||||
static void pci_init_dev_archdata(struct dev_archdata *sd, void *iommu,
|
||||
void *stc, void *host_controller,
|
||||
struct platform_device *op,
|
||||
int numa_node)
|
||||
{
|
||||
sd->iommu = iommu;
|
||||
sd->stc = stc;
|
||||
sd->host_controller = host_controller;
|
||||
sd->op = op;
|
||||
sd->numa_node = numa_node;
|
||||
}
|
||||
|
||||
static struct pci_dev *of_create_pci_dev(struct pci_pbm_info *pbm,
|
||||
struct device_node *node,
|
||||
struct pci_bus *bus, int devfn)
|
||||
|
|
@ -259,13 +271,10 @@ static struct pci_dev *of_create_pci_dev(struct pci_pbm_info *pbm,
|
|||
if (!dev)
|
||||
return NULL;
|
||||
|
||||
op = of_find_device_by_node(node);
|
||||
sd = &dev->dev.archdata;
|
||||
sd->iommu = pbm->iommu;
|
||||
sd->stc = &pbm->stc;
|
||||
sd->host_controller = pbm;
|
||||
sd->op = op = of_find_device_by_node(node);
|
||||
sd->numa_node = pbm->numa_node;
|
||||
|
||||
pci_init_dev_archdata(sd, pbm->iommu, &pbm->stc, pbm, op,
|
||||
pbm->numa_node);
|
||||
sd = &op->dev.archdata;
|
||||
sd->iommu = pbm->iommu;
|
||||
sd->stc = &pbm->stc;
|
||||
|
|
@ -1003,9 +1012,13 @@ int pcibios_add_device(struct pci_dev *dev)
|
|||
* Copy dev_archdata from PF to VF
|
||||
*/
|
||||
if (dev->is_virtfn) {
|
||||
struct dev_archdata *psd;
|
||||
|
||||
pdev = dev->physfn;
|
||||
memcpy(&dev->dev.archdata, &pdev->dev.archdata,
|
||||
sizeof(struct dev_archdata));
|
||||
psd = &pdev->dev.archdata;
|
||||
pci_init_dev_archdata(&dev->dev.archdata, psd->iommu,
|
||||
psd->stc, psd->host_controller, NULL,
|
||||
psd->numa_node);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -145,13 +145,13 @@ ENDPROC(U3_retl_o2_plus_GS_plus_0x08)
|
|||
ENTRY(U3_retl_o2_and_7_plus_GS)
|
||||
and %o2, 7, %o2
|
||||
retl
|
||||
add %o2, GLOBAL_SPARE, %o2
|
||||
add %o2, GLOBAL_SPARE, %o0
|
||||
ENDPROC(U3_retl_o2_and_7_plus_GS)
|
||||
ENTRY(U3_retl_o2_and_7_plus_GS_plus_8)
|
||||
add GLOBAL_SPARE, 8, GLOBAL_SPARE
|
||||
and %o2, 7, %o2
|
||||
retl
|
||||
add %o2, GLOBAL_SPARE, %o2
|
||||
add %o2, GLOBAL_SPARE, %o0
|
||||
ENDPROC(U3_retl_o2_and_7_plus_GS_plus_8)
|
||||
#endif
|
||||
|
||||
|
|
|
|||
|
|
@ -487,6 +487,7 @@ good_area:
|
|||
tsb_grow(mm, MM_TSB_BASE, mm_rss);
|
||||
#if defined(CONFIG_HUGETLB_PAGE) || defined(CONFIG_TRANSPARENT_HUGEPAGE)
|
||||
mm_rss = mm->context.hugetlb_pte_count + mm->context.thp_pte_count;
|
||||
mm_rss *= REAL_HPAGE_PER_HPAGE;
|
||||
if (unlikely(mm_rss >
|
||||
mm->context.tsb_block[MM_TSB_HUGE].tsb_rss_limit)) {
|
||||
if (mm->context.tsb_block[MM_TSB_HUGE].tsb)
|
||||
|
|
|
|||
|
|
@ -174,10 +174,25 @@ void set_pmd_at(struct mm_struct *mm, unsigned long addr,
|
|||
return;
|
||||
|
||||
if ((pmd_val(pmd) ^ pmd_val(orig)) & _PAGE_PMD_HUGE) {
|
||||
if (pmd_val(pmd) & _PAGE_PMD_HUGE)
|
||||
/*
|
||||
* Note that this routine only sets pmds for THP pages.
|
||||
* Hugetlb pages are handled elsewhere. We need to check
|
||||
* for huge zero page. Huge zero pages are like hugetlb
|
||||
* pages in that there is no RSS, but there is the need
|
||||
* for TSB entries. So, huge zero page counts go into
|
||||
* hugetlb_pte_count.
|
||||
*/
|
||||
if (pmd_val(pmd) & _PAGE_PMD_HUGE) {
|
||||
if (is_huge_zero_page(pmd_page(pmd)))
|
||||
mm->context.hugetlb_pte_count++;
|
||||
else
|
||||
mm->context.thp_pte_count++;
|
||||
} else {
|
||||
if (is_huge_zero_page(pmd_page(orig)))
|
||||
mm->context.hugetlb_pte_count--;
|
||||
else
|
||||
mm->context.thp_pte_count--;
|
||||
}
|
||||
|
||||
/* Do not try to allocate the TSB hash table if we
|
||||
* don't have one already. We have various locks held
|
||||
|
|
@ -204,6 +219,9 @@ void set_pmd_at(struct mm_struct *mm, unsigned long addr,
|
|||
}
|
||||
}
|
||||
|
||||
/*
|
||||
* This routine is only called when splitting a THP
|
||||
*/
|
||||
void pmdp_invalidate(struct vm_area_struct *vma, unsigned long address,
|
||||
pmd_t *pmdp)
|
||||
{
|
||||
|
|
@ -213,6 +231,15 @@ void pmdp_invalidate(struct vm_area_struct *vma, unsigned long address,
|
|||
|
||||
set_pmd_at(vma->vm_mm, address, pmdp, entry);
|
||||
flush_tlb_range(vma, address, address + HPAGE_PMD_SIZE);
|
||||
|
||||
/*
|
||||
* set_pmd_at() will not be called in a way to decrement
|
||||
* thp_pte_count when splitting a THP, so do it now.
|
||||
* Sanity check pmd before doing the actual decrement.
|
||||
*/
|
||||
if ((pmd_val(entry) & _PAGE_PMD_HUGE) &&
|
||||
!is_huge_zero_page(pmd_page(entry)))
|
||||
(vma->vm_mm)->context.thp_pte_count--;
|
||||
}
|
||||
|
||||
void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,
|
||||
|
|
|
|||
|
|
@ -489,8 +489,10 @@ retry_tsb_alloc:
|
|||
|
||||
int init_new_context(struct task_struct *tsk, struct mm_struct *mm)
|
||||
{
|
||||
unsigned long mm_rss = get_mm_rss(mm);
|
||||
#if defined(CONFIG_HUGETLB_PAGE) || defined(CONFIG_TRANSPARENT_HUGEPAGE)
|
||||
unsigned long total_huge_pte_count;
|
||||
unsigned long saved_hugetlb_pte_count;
|
||||
unsigned long saved_thp_pte_count;
|
||||
#endif
|
||||
unsigned int i;
|
||||
|
||||
|
|
@ -503,10 +505,12 @@ int init_new_context(struct task_struct *tsk, struct mm_struct *mm)
|
|||
* will re-increment the counters as the parent PTEs are
|
||||
* copied into the child address space.
|
||||
*/
|
||||
total_huge_pte_count = mm->context.hugetlb_pte_count +
|
||||
mm->context.thp_pte_count;
|
||||
saved_hugetlb_pte_count = mm->context.hugetlb_pte_count;
|
||||
saved_thp_pte_count = mm->context.thp_pte_count;
|
||||
mm->context.hugetlb_pte_count = 0;
|
||||
mm->context.thp_pte_count = 0;
|
||||
|
||||
mm_rss -= saved_thp_pte_count * (HPAGE_SIZE / PAGE_SIZE);
|
||||
#endif
|
||||
|
||||
/* copy_mm() copies over the parent's mm_struct before calling
|
||||
|
|
@ -519,11 +523,13 @@ int init_new_context(struct task_struct *tsk, struct mm_struct *mm)
|
|||
/* If this is fork, inherit the parent's TSB size. We would
|
||||
* grow it to that size on the first page fault anyways.
|
||||
*/
|
||||
tsb_grow(mm, MM_TSB_BASE, get_mm_rss(mm));
|
||||
tsb_grow(mm, MM_TSB_BASE, mm_rss);
|
||||
|
||||
#if defined(CONFIG_HUGETLB_PAGE) || defined(CONFIG_TRANSPARENT_HUGEPAGE)
|
||||
if (unlikely(total_huge_pte_count))
|
||||
tsb_grow(mm, MM_TSB_HUGE, total_huge_pte_count);
|
||||
if (unlikely(saved_hugetlb_pte_count + saved_thp_pte_count))
|
||||
tsb_grow(mm, MM_TSB_HUGE,
|
||||
(saved_hugetlb_pte_count + saved_thp_pte_count) *
|
||||
REAL_HPAGE_PER_HPAGE);
|
||||
#endif
|
||||
|
||||
if (unlikely(!mm->context.tsb_block[MM_TSB_BASE].tsb))
|
||||
|
|
|
|||
|
|
@ -59,10 +59,14 @@ KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/um
|
|||
# Same things for in6addr_loopback and mktime - found in libc. For these two we
|
||||
# only get link-time error, luckily.
|
||||
#
|
||||
# -Dlongjmp=kernel_longjmp prevents anything from referencing the libpthread.a
|
||||
# embedded copy of longjmp, same thing for setjmp.
|
||||
#
|
||||
# These apply to USER_CFLAGS to.
|
||||
|
||||
KBUILD_CFLAGS += $(CFLAGS) $(CFLAGS-y) -D__arch_um__ \
|
||||
$(ARCH_INCLUDE) $(MODE_INCLUDE) -Dvmap=kernel_vmap \
|
||||
-Dlongjmp=kernel_longjmp -Dsetjmp=kernel_setjmp \
|
||||
-Din6addr_loopback=kernel_in6addr_loopback \
|
||||
-Din6addr_any=kernel_in6addr_any -Dstrrchr=kernel_strrchr
|
||||
|
||||
|
|
|
|||
|
|
@ -585,6 +585,8 @@ int start_idle_thread(void *stack, jmp_buf *switch_buf)
|
|||
fatal_sigsegv();
|
||||
}
|
||||
longjmp(*switch_buf, 1);
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
void initial_thread_cb_skas(void (*proc)(void *), void *arg)
|
||||
|
|
|
|||
|
|
@ -184,22 +184,22 @@ do { \
|
|||
typeof(var) pfo_ret__; \
|
||||
switch (sizeof(var)) { \
|
||||
case 1: \
|
||||
asm(op "b "__percpu_arg(1)",%0" \
|
||||
asm volatile(op "b "__percpu_arg(1)",%0"\
|
||||
: "=q" (pfo_ret__) \
|
||||
: "m" (var)); \
|
||||
break; \
|
||||
case 2: \
|
||||
asm(op "w "__percpu_arg(1)",%0" \
|
||||
asm volatile(op "w "__percpu_arg(1)",%0"\
|
||||
: "=r" (pfo_ret__) \
|
||||
: "m" (var)); \
|
||||
break; \
|
||||
case 4: \
|
||||
asm(op "l "__percpu_arg(1)",%0" \
|
||||
asm volatile(op "l "__percpu_arg(1)",%0"\
|
||||
: "=r" (pfo_ret__) \
|
||||
: "m" (var)); \
|
||||
break; \
|
||||
case 8: \
|
||||
asm(op "q "__percpu_arg(1)",%0" \
|
||||
asm volatile(op "q "__percpu_arg(1)",%0"\
|
||||
: "=r" (pfo_ret__) \
|
||||
: "m" (var)); \
|
||||
break; \
|
||||
|
|
|
|||
|
|
@ -23,7 +23,7 @@
|
|||
#include <asm/time.h>
|
||||
|
||||
#ifdef CONFIG_X86_64
|
||||
__visible volatile unsigned long jiffies __cacheline_aligned = INITIAL_JIFFIES;
|
||||
__visible volatile unsigned long jiffies __cacheline_aligned_in_smp = INITIAL_JIFFIES;
|
||||
#endif
|
||||
|
||||
unsigned long profile_pc(struct pt_regs *regs)
|
||||
|
|
|
|||
|
|
@ -955,11 +955,11 @@ static void populate_pte(struct cpa_data *cpa,
|
|||
}
|
||||
}
|
||||
|
||||
static int populate_pmd(struct cpa_data *cpa,
|
||||
static long populate_pmd(struct cpa_data *cpa,
|
||||
unsigned long start, unsigned long end,
|
||||
unsigned num_pages, pud_t *pud, pgprot_t pgprot)
|
||||
{
|
||||
unsigned int cur_pages = 0;
|
||||
long cur_pages = 0;
|
||||
pmd_t *pmd;
|
||||
pgprot_t pmd_pgprot;
|
||||
|
||||
|
|
@ -1029,12 +1029,12 @@ static int populate_pmd(struct cpa_data *cpa,
|
|||
return num_pages;
|
||||
}
|
||||
|
||||
static int populate_pud(struct cpa_data *cpa, unsigned long start, pgd_t *pgd,
|
||||
static long populate_pud(struct cpa_data *cpa, unsigned long start, pgd_t *pgd,
|
||||
pgprot_t pgprot)
|
||||
{
|
||||
pud_t *pud;
|
||||
unsigned long end;
|
||||
int cur_pages = 0;
|
||||
long cur_pages = 0;
|
||||
pgprot_t pud_pgprot;
|
||||
|
||||
end = start + (cpa->numpages << PAGE_SHIFT);
|
||||
|
|
@ -1090,7 +1090,7 @@ static int populate_pud(struct cpa_data *cpa, unsigned long start, pgd_t *pgd,
|
|||
|
||||
/* Map trailing leftover */
|
||||
if (start < end) {
|
||||
int tmp;
|
||||
long tmp;
|
||||
|
||||
pud = pud_offset(pgd, start);
|
||||
if (pud_none(*pud))
|
||||
|
|
@ -1116,7 +1116,7 @@ static int populate_pgd(struct cpa_data *cpa, unsigned long addr)
|
|||
pgprot_t pgprot = __pgprot(_KERNPG_TABLE);
|
||||
pud_t *pud = NULL; /* shut up gcc */
|
||||
pgd_t *pgd_entry;
|
||||
int ret;
|
||||
long ret;
|
||||
|
||||
pgd_entry = cpa->pgd + pgd_index(addr);
|
||||
|
||||
|
|
@ -1351,7 +1351,8 @@ static int cpa_process_alias(struct cpa_data *cpa)
|
|||
|
||||
static int __change_page_attr_set_clr(struct cpa_data *cpa, int checkalias)
|
||||
{
|
||||
int ret, numpages = cpa->numpages;
|
||||
unsigned long numpages = cpa->numpages;
|
||||
int ret;
|
||||
|
||||
while (numpages) {
|
||||
/*
|
||||
|
|
|
|||
|
|
@ -541,9 +541,16 @@ static void twinhead_reserve_killing_zone(struct pci_dev *dev)
|
|||
}
|
||||
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x27B9, twinhead_reserve_killing_zone);
|
||||
|
||||
/*
|
||||
* Broadwell EP Home Agent BARs erroneously return non-zero values when read.
|
||||
*
|
||||
* See http://www.intel.com/content/www/us/en/processors/xeon/xeon-e5-v4-spec-update.html
|
||||
* entry BDF2.
|
||||
*/
|
||||
static void pci_bdwep_bar(struct pci_dev *dev)
|
||||
{
|
||||
dev->non_compliant_bars = 1;
|
||||
}
|
||||
DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6f60, pci_bdwep_bar);
|
||||
DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fa0, pci_bdwep_bar);
|
||||
DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fc0, pci_bdwep_bar);
|
||||
|
|
|
|||
|
|
@ -16,9 +16,9 @@
|
|||
|
||||
.text
|
||||
.align 4
|
||||
.globl setjmp
|
||||
.type setjmp, @function
|
||||
setjmp:
|
||||
.globl kernel_setjmp
|
||||
.type kernel_setjmp, @function
|
||||
kernel_setjmp:
|
||||
#ifdef _REGPARM
|
||||
movl %eax,%edx
|
||||
#else
|
||||
|
|
@ -35,13 +35,13 @@ setjmp:
|
|||
movl %ecx,20(%edx) # Return address
|
||||
ret
|
||||
|
||||
.size setjmp,.-setjmp
|
||||
.size kernel_setjmp,.-kernel_setjmp
|
||||
|
||||
.text
|
||||
.align 4
|
||||
.globl longjmp
|
||||
.type longjmp, @function
|
||||
longjmp:
|
||||
.globl kernel_longjmp
|
||||
.type kernel_longjmp, @function
|
||||
kernel_longjmp:
|
||||
#ifdef _REGPARM
|
||||
xchgl %eax,%edx
|
||||
#else
|
||||
|
|
@ -55,4 +55,4 @@ longjmp:
|
|||
movl 16(%edx),%edi
|
||||
jmp *20(%edx)
|
||||
|
||||
.size longjmp,.-longjmp
|
||||
.size kernel_longjmp,.-kernel_longjmp
|
||||
|
|
|
|||
|
|
@ -18,9 +18,9 @@
|
|||
|
||||
.text
|
||||
.align 4
|
||||
.globl setjmp
|
||||
.type setjmp, @function
|
||||
setjmp:
|
||||
.globl kernel_setjmp
|
||||
.type kernel_setjmp, @function
|
||||
kernel_setjmp:
|
||||
pop %rsi # Return address, and adjust the stack
|
||||
xorl %eax,%eax # Return value
|
||||
movq %rbx,(%rdi)
|
||||
|
|
@ -34,13 +34,13 @@ setjmp:
|
|||
movq %rsi,56(%rdi) # Return address
|
||||
ret
|
||||
|
||||
.size setjmp,.-setjmp
|
||||
.size kernel_setjmp,.-kernel_setjmp
|
||||
|
||||
.text
|
||||
.align 4
|
||||
.globl longjmp
|
||||
.type longjmp, @function
|
||||
longjmp:
|
||||
.globl kernel_longjmp
|
||||
.type kernel_longjmp, @function
|
||||
kernel_longjmp:
|
||||
movl %esi,%eax # Return value (int)
|
||||
movq (%rdi),%rbx
|
||||
movq 8(%rdi),%rsp
|
||||
|
|
@ -51,4 +51,4 @@ longjmp:
|
|||
movq 48(%rdi),%r15
|
||||
jmp *56(%rdi)
|
||||
|
||||
.size longjmp,.-longjmp
|
||||
.size kernel_longjmp,.-kernel_longjmp
|
||||
|
|
|
|||
|
|
@ -1390,20 +1390,6 @@ config CRYPTO_SERPENT_AVX2_X86_64
|
|||
See also:
|
||||
<http://www.cl.cam.ac.uk/~rja14/serpent.html>
|
||||
|
||||
config CRYPTO_SPECK
|
||||
tristate "Speck cipher algorithm"
|
||||
select CRYPTO_ALGAPI
|
||||
help
|
||||
Speck is a lightweight block cipher that is tuned for optimal
|
||||
performance in software (rather than hardware).
|
||||
|
||||
Speck may not be as secure as AES, and should only be used on systems
|
||||
where AES is not fast enough.
|
||||
|
||||
See also: <https://eprint.iacr.org/2013/404.pdf>
|
||||
|
||||
If unsure, say N.
|
||||
|
||||
config CRYPTO_TEA
|
||||
tristate "TEA, XTEA and XETA cipher algorithms"
|
||||
select CRYPTO_ALGAPI
|
||||
|
|
|
|||
|
|
@ -98,7 +98,6 @@ obj-$(CONFIG_CRYPTO_TEA) += tea.o
|
|||
obj-$(CONFIG_CRYPTO_KHAZAD) += khazad.o
|
||||
obj-$(CONFIG_CRYPTO_ANUBIS) += anubis.o
|
||||
obj-$(CONFIG_CRYPTO_SEED) += seed.o
|
||||
obj-$(CONFIG_CRYPTO_SPECK) += speck.o
|
||||
obj-$(CONFIG_CRYPTO_SALSA20) += salsa20_generic.o
|
||||
obj-$(CONFIG_CRYPTO_CHACHA20) += chacha20_generic.o
|
||||
obj-$(CONFIG_CRYPTO_POLY1305) += poly1305_generic.o
|
||||
|
|
|
|||
|
|
@ -41,7 +41,7 @@ static int shash_setkey_unaligned(struct crypto_shash *tfm, const u8 *key,
|
|||
int err;
|
||||
|
||||
absize = keylen + (alignmask & ~(crypto_tfm_ctx_alignment() - 1));
|
||||
buffer = kmalloc(absize, GFP_KERNEL);
|
||||
buffer = kmalloc(absize, GFP_ATOMIC);
|
||||
if (!buffer)
|
||||
return -ENOMEM;
|
||||
|
||||
|
|
|
|||
307
crypto/speck.c
307
crypto/speck.c
|
|
@ -1,307 +0,0 @@
|
|||
// SPDX-License-Identifier: GPL-2.0
|
||||
/*
|
||||
* Speck: a lightweight block cipher
|
||||
*
|
||||
* Copyright (c) 2018 Google, Inc
|
||||
*
|
||||
* Speck has 10 variants, including 5 block sizes. For now we only implement
|
||||
* the variants Speck128/128, Speck128/192, Speck128/256, Speck64/96, and
|
||||
* Speck64/128. Speck${B}/${K} denotes the variant with a block size of B bits
|
||||
* and a key size of K bits. The Speck128 variants are believed to be the most
|
||||
* secure variants, and they use the same block size and key sizes as AES. The
|
||||
* Speck64 variants are less secure, but on 32-bit processors are usually
|
||||
* faster. The remaining variants (Speck32, Speck48, and Speck96) are even less
|
||||
* secure and/or not as well suited for implementation on either 32-bit or
|
||||
* 64-bit processors, so are omitted.
|
||||
*
|
||||
* Reference: "The Simon and Speck Families of Lightweight Block Ciphers"
|
||||
* https://eprint.iacr.org/2013/404.pdf
|
||||
*
|
||||
* In a correspondence, the Speck designers have also clarified that the words
|
||||
* should be interpreted in little-endian format, and the words should be
|
||||
* ordered such that the first word of each block is 'y' rather than 'x', and
|
||||
* the first key word (rather than the last) becomes the first round key.
|
||||
*/
|
||||
|
||||
#include <asm/unaligned.h>
|
||||
#include <crypto/speck.h>
|
||||
#include <linux/bitops.h>
|
||||
#include <linux/crypto.h>
|
||||
#include <linux/init.h>
|
||||
#include <linux/module.h>
|
||||
|
||||
/* Speck128 */
|
||||
|
||||
static __always_inline void speck128_round(u64 *x, u64 *y, u64 k)
|
||||
{
|
||||
*x = ror64(*x, 8);
|
||||
*x += *y;
|
||||
*x ^= k;
|
||||
*y = rol64(*y, 3);
|
||||
*y ^= *x;
|
||||
}
|
||||
|
||||
static __always_inline void speck128_unround(u64 *x, u64 *y, u64 k)
|
||||
{
|
||||
*y ^= *x;
|
||||
*y = ror64(*y, 3);
|
||||
*x ^= k;
|
||||
*x -= *y;
|
||||
*x = rol64(*x, 8);
|
||||
}
|
||||
|
||||
void crypto_speck128_encrypt(const struct speck128_tfm_ctx *ctx,
|
||||
u8 *out, const u8 *in)
|
||||
{
|
||||
u64 y = get_unaligned_le64(in);
|
||||
u64 x = get_unaligned_le64(in + 8);
|
||||
int i;
|
||||
|
||||
for (i = 0; i < ctx->nrounds; i++)
|
||||
speck128_round(&x, &y, ctx->round_keys[i]);
|
||||
|
||||
put_unaligned_le64(y, out);
|
||||
put_unaligned_le64(x, out + 8);
|
||||
}
|
||||
EXPORT_SYMBOL_GPL(crypto_speck128_encrypt);
|
||||
|
||||
static void speck128_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in)
|
||||
{
|
||||
crypto_speck128_encrypt(crypto_tfm_ctx(tfm), out, in);
|
||||
}
|
||||
|
||||
void crypto_speck128_decrypt(const struct speck128_tfm_ctx *ctx,
|
||||
u8 *out, const u8 *in)
|
||||
{
|
||||
u64 y = get_unaligned_le64(in);
|
||||
u64 x = get_unaligned_le64(in + 8);
|
||||
int i;
|
||||
|
||||
for (i = ctx->nrounds - 1; i >= 0; i--)
|
||||
speck128_unround(&x, &y, ctx->round_keys[i]);
|
||||
|
||||
put_unaligned_le64(y, out);
|
||||
put_unaligned_le64(x, out + 8);
|
||||
}
|
||||
EXPORT_SYMBOL_GPL(crypto_speck128_decrypt);
|
||||
|
||||
static void speck128_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in)
|
||||
{
|
||||
crypto_speck128_decrypt(crypto_tfm_ctx(tfm), out, in);
|
||||
}
|
||||
|
||||
int crypto_speck128_setkey(struct speck128_tfm_ctx *ctx, const u8 *key,
|
||||
unsigned int keylen)
|
||||
{
|
||||
u64 l[3];
|
||||
u64 k;
|
||||
int i;
|
||||
|
||||
switch (keylen) {
|
||||
case SPECK128_128_KEY_SIZE:
|
||||
k = get_unaligned_le64(key);
|
||||
l[0] = get_unaligned_le64(key + 8);
|
||||
ctx->nrounds = SPECK128_128_NROUNDS;
|
||||
for (i = 0; i < ctx->nrounds; i++) {
|
||||
ctx->round_keys[i] = k;
|
||||
speck128_round(&l[0], &k, i);
|
||||
}
|
||||
break;
|
||||
case SPECK128_192_KEY_SIZE:
|
||||
k = get_unaligned_le64(key);
|
||||
l[0] = get_unaligned_le64(key + 8);
|
||||
l[1] = get_unaligned_le64(key + 16);
|
||||
ctx->nrounds = SPECK128_192_NROUNDS;
|
||||
for (i = 0; i < ctx->nrounds; i++) {
|
||||
ctx->round_keys[i] = k;
|
||||
speck128_round(&l[i % 2], &k, i);
|
||||
}
|
||||
break;
|
||||
case SPECK128_256_KEY_SIZE:
|
||||
k = get_unaligned_le64(key);
|
||||
l[0] = get_unaligned_le64(key + 8);
|
||||
l[1] = get_unaligned_le64(key + 16);
|
||||
l[2] = get_unaligned_le64(key + 24);
|
||||
ctx->nrounds = SPECK128_256_NROUNDS;
|
||||
for (i = 0; i < ctx->nrounds; i++) {
|
||||
ctx->round_keys[i] = k;
|
||||
speck128_round(&l[i % 3], &k, i);
|
||||
}
|
||||
break;
|
||||
default:
|
||||
return -EINVAL;
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
EXPORT_SYMBOL_GPL(crypto_speck128_setkey);
|
||||
|
||||
static int speck128_setkey(struct crypto_tfm *tfm, const u8 *key,
|
||||
unsigned int keylen)
|
||||
{
|
||||
return crypto_speck128_setkey(crypto_tfm_ctx(tfm), key, keylen);
|
||||
}
|
||||
|
||||
/* Speck64 */
|
||||
|
||||
static __always_inline void speck64_round(u32 *x, u32 *y, u32 k)
|
||||
{
|
||||
*x = ror32(*x, 8);
|
||||
*x += *y;
|
||||
*x ^= k;
|
||||
*y = rol32(*y, 3);
|
||||
*y ^= *x;
|
||||
}
|
||||
|
||||
static __always_inline void speck64_unround(u32 *x, u32 *y, u32 k)
|
||||
{
|
||||
*y ^= *x;
|
||||
*y = ror32(*y, 3);
|
||||
*x ^= k;
|
||||
*x -= *y;
|
||||
*x = rol32(*x, 8);
|
||||
}
|
||||
|
||||
void crypto_speck64_encrypt(const struct speck64_tfm_ctx *ctx,
|
||||
u8 *out, const u8 *in)
|
||||
{
|
||||
u32 y = get_unaligned_le32(in);
|
||||
u32 x = get_unaligned_le32(in + 4);
|
||||
int i;
|
||||
|
||||
for (i = 0; i < ctx->nrounds; i++)
|
||||
speck64_round(&x, &y, ctx->round_keys[i]);
|
||||
|
||||
put_unaligned_le32(y, out);
|
||||
put_unaligned_le32(x, out + 4);
|
||||
}
|
||||
EXPORT_SYMBOL_GPL(crypto_speck64_encrypt);
|
||||
|
||||
static void speck64_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in)
|
||||
{
|
||||
crypto_speck64_encrypt(crypto_tfm_ctx(tfm), out, in);
|
||||
}
|
||||
|
||||
void crypto_speck64_decrypt(const struct speck64_tfm_ctx *ctx,
|
||||
u8 *out, const u8 *in)
|
||||
{
|
||||
u32 y = get_unaligned_le32(in);
|
||||
u32 x = get_unaligned_le32(in + 4);
|
||||
int i;
|
||||
|
||||
for (i = ctx->nrounds - 1; i >= 0; i--)
|
||||
speck64_unround(&x, &y, ctx->round_keys[i]);
|
||||
|
||||
put_unaligned_le32(y, out);
|
||||
put_unaligned_le32(x, out + 4);
|
||||
}
|
||||
EXPORT_SYMBOL_GPL(crypto_speck64_decrypt);
|
||||
|
||||
static void speck64_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in)
|
||||
{
|
||||
crypto_speck64_decrypt(crypto_tfm_ctx(tfm), out, in);
|
||||
}
|
||||
|
||||
int crypto_speck64_setkey(struct speck64_tfm_ctx *ctx, const u8 *key,
|
||||
unsigned int keylen)
|
||||
{
|
||||
u32 l[3];
|
||||
u32 k;
|
||||
int i;
|
||||
|
||||
switch (keylen) {
|
||||
case SPECK64_96_KEY_SIZE:
|
||||
k = get_unaligned_le32(key);
|
||||
l[0] = get_unaligned_le32(key + 4);
|
||||
l[1] = get_unaligned_le32(key + 8);
|
||||
ctx->nrounds = SPECK64_96_NROUNDS;
|
||||
for (i = 0; i < ctx->nrounds; i++) {
|
||||
ctx->round_keys[i] = k;
|
||||
speck64_round(&l[i % 2], &k, i);
|
||||
}
|
||||
break;
|
||||
case SPECK64_128_KEY_SIZE:
|
||||
k = get_unaligned_le32(key);
|
||||
l[0] = get_unaligned_le32(key + 4);
|
||||
l[1] = get_unaligned_le32(key + 8);
|
||||
l[2] = get_unaligned_le32(key + 12);
|
||||
ctx->nrounds = SPECK64_128_NROUNDS;
|
||||
for (i = 0; i < ctx->nrounds; i++) {
|
||||
ctx->round_keys[i] = k;
|
||||
speck64_round(&l[i % 3], &k, i);
|
||||
}
|
||||
break;
|
||||
default:
|
||||
return -EINVAL;
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
EXPORT_SYMBOL_GPL(crypto_speck64_setkey);
|
||||
|
||||
static int speck64_setkey(struct crypto_tfm *tfm, const u8 *key,
|
||||
unsigned int keylen)
|
||||
{
|
||||
return crypto_speck64_setkey(crypto_tfm_ctx(tfm), key, keylen);
|
||||
}
|
||||
|
||||
/* Algorithm definitions */
|
||||
|
||||
static struct crypto_alg speck_algs[] = {
|
||||
{
|
||||
.cra_name = "speck128",
|
||||
.cra_driver_name = "speck128-generic",
|
||||
.cra_priority = 100,
|
||||
.cra_flags = CRYPTO_ALG_TYPE_CIPHER,
|
||||
.cra_blocksize = SPECK128_BLOCK_SIZE,
|
||||
.cra_ctxsize = sizeof(struct speck128_tfm_ctx),
|
||||
.cra_module = THIS_MODULE,
|
||||
.cra_u = {
|
||||
.cipher = {
|
||||
.cia_min_keysize = SPECK128_128_KEY_SIZE,
|
||||
.cia_max_keysize = SPECK128_256_KEY_SIZE,
|
||||
.cia_setkey = speck128_setkey,
|
||||
.cia_encrypt = speck128_encrypt,
|
||||
.cia_decrypt = speck128_decrypt
|
||||
}
|
||||
}
|
||||
}, {
|
||||
.cra_name = "speck64",
|
||||
.cra_driver_name = "speck64-generic",
|
||||
.cra_priority = 100,
|
||||
.cra_flags = CRYPTO_ALG_TYPE_CIPHER,
|
||||
.cra_blocksize = SPECK64_BLOCK_SIZE,
|
||||
.cra_ctxsize = sizeof(struct speck64_tfm_ctx),
|
||||
.cra_module = THIS_MODULE,
|
||||
.cra_u = {
|
||||
.cipher = {
|
||||
.cia_min_keysize = SPECK64_96_KEY_SIZE,
|
||||
.cia_max_keysize = SPECK64_128_KEY_SIZE,
|
||||
.cia_setkey = speck64_setkey,
|
||||
.cia_encrypt = speck64_encrypt,
|
||||
.cia_decrypt = speck64_decrypt
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
static int __init speck_module_init(void)
|
||||
{
|
||||
return crypto_register_algs(speck_algs, ARRAY_SIZE(speck_algs));
|
||||
}
|
||||
|
||||
static void __exit speck_module_exit(void)
|
||||
{
|
||||
crypto_unregister_algs(speck_algs, ARRAY_SIZE(speck_algs));
|
||||
}
|
||||
|
||||
module_init(speck_module_init);
|
||||
module_exit(speck_module_exit);
|
||||
|
||||
MODULE_DESCRIPTION("Speck block cipher (generic)");
|
||||
MODULE_LICENSE("GPL");
|
||||
MODULE_AUTHOR("Eric Biggers <ebiggers@google.com>");
|
||||
MODULE_ALIAS_CRYPTO("speck128");
|
||||
MODULE_ALIAS_CRYPTO("speck128-generic");
|
||||
MODULE_ALIAS_CRYPTO("speck64");
|
||||
MODULE_ALIAS_CRYPTO("speck64-generic");
|
||||
|
|
@ -3113,36 +3113,6 @@ static const struct alg_test_desc alg_test_descs[] = {
|
|||
}
|
||||
}
|
||||
}
|
||||
}, {
|
||||
.alg = "ecb(speck128)",
|
||||
.test = alg_test_skcipher,
|
||||
.suite = {
|
||||
.cipher = {
|
||||
.enc = {
|
||||
.vecs = speck128_enc_tv_template,
|
||||
.count = ARRAY_SIZE(speck128_enc_tv_template)
|
||||
},
|
||||
.dec = {
|
||||
.vecs = speck128_dec_tv_template,
|
||||
.count = ARRAY_SIZE(speck128_dec_tv_template)
|
||||
}
|
||||
}
|
||||
}
|
||||
}, {
|
||||
.alg = "ecb(speck64)",
|
||||
.test = alg_test_skcipher,
|
||||
.suite = {
|
||||
.cipher = {
|
||||
.enc = {
|
||||
.vecs = speck64_enc_tv_template,
|
||||
.count = ARRAY_SIZE(speck64_enc_tv_template)
|
||||
},
|
||||
.dec = {
|
||||
.vecs = speck64_dec_tv_template,
|
||||
.count = ARRAY_SIZE(speck64_dec_tv_template)
|
||||
}
|
||||
}
|
||||
}
|
||||
}, {
|
||||
.alg = "ecb(tea)",
|
||||
.test = alg_test_skcipher,
|
||||
|
|
@ -3888,36 +3858,6 @@ static const struct alg_test_desc alg_test_descs[] = {
|
|||
}
|
||||
}
|
||||
}
|
||||
}, {
|
||||
.alg = "xts(speck128)",
|
||||
.test = alg_test_skcipher,
|
||||
.suite = {
|
||||
.cipher = {
|
||||
.enc = {
|
||||
.vecs = speck128_xts_enc_tv_template,
|
||||
.count = ARRAY_SIZE(speck128_xts_enc_tv_template)
|
||||
},
|
||||
.dec = {
|
||||
.vecs = speck128_xts_dec_tv_template,
|
||||
.count = ARRAY_SIZE(speck128_xts_dec_tv_template)
|
||||
}
|
||||
}
|
||||
}
|
||||
}, {
|
||||
.alg = "xts(speck64)",
|
||||
.test = alg_test_skcipher,
|
||||
.suite = {
|
||||
.cipher = {
|
||||
.enc = {
|
||||
.vecs = speck64_xts_enc_tv_template,
|
||||
.count = ARRAY_SIZE(speck64_xts_enc_tv_template)
|
||||
},
|
||||
.dec = {
|
||||
.vecs = speck64_xts_dec_tv_template,
|
||||
.count = ARRAY_SIZE(speck64_xts_dec_tv_template)
|
||||
}
|
||||
}
|
||||
}
|
||||
}, {
|
||||
.alg = "xts(twofish)",
|
||||
.test = alg_test_skcipher,
|
||||
|
|
|
|||
1486
crypto/testmgr.h
1486
crypto/testmgr.h
File diff suppressed because it is too large
Load diff
|
|
@ -619,8 +619,11 @@ static void ahci_pci_save_initial_config(struct pci_dev *pdev,
|
|||
static int ahci_pci_reset_controller(struct ata_host *host)
|
||||
{
|
||||
struct pci_dev *pdev = to_pci_dev(host->dev);
|
||||
int rc;
|
||||
|
||||
ahci_reset_controller(host);
|
||||
rc = ahci_reset_controller(host);
|
||||
if (rc)
|
||||
return rc;
|
||||
|
||||
if (pdev->vendor == PCI_VENDOR_ID_INTEL) {
|
||||
struct ahci_host_priv *hpriv = host->private_data;
|
||||
|
|
|
|||
|
|
@ -1070,6 +1070,7 @@ static int loop_clr_fd(struct loop_device *lo)
|
|||
memset(lo->lo_encrypt_key, 0, LO_KEY_SIZE);
|
||||
memset(lo->lo_crypt_name, 0, LO_NAME_SIZE);
|
||||
memset(lo->lo_file_name, 0, LO_NAME_SIZE);
|
||||
blk_queue_logical_block_size(lo->lo_queue, 512);
|
||||
if (bdev) {
|
||||
bdput(bdev);
|
||||
invalidate_bdev(bdev);
|
||||
|
|
@ -1355,6 +1356,24 @@ static int loop_set_dio(struct loop_device *lo, unsigned long arg)
|
|||
return error;
|
||||
}
|
||||
|
||||
static int loop_set_block_size(struct loop_device *lo, unsigned long arg)
|
||||
{
|
||||
if (lo->lo_state != Lo_bound)
|
||||
return -ENXIO;
|
||||
|
||||
if (arg < 512 || arg > PAGE_SIZE || !is_power_of_2(arg))
|
||||
return -EINVAL;
|
||||
|
||||
blk_mq_freeze_queue(lo->lo_queue);
|
||||
|
||||
blk_queue_logical_block_size(lo->lo_queue, arg);
|
||||
loop_update_dio(lo);
|
||||
|
||||
blk_mq_unfreeze_queue(lo->lo_queue);
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int lo_ioctl(struct block_device *bdev, fmode_t mode,
|
||||
unsigned int cmd, unsigned long arg)
|
||||
{
|
||||
|
|
@ -1403,6 +1422,11 @@ static int lo_ioctl(struct block_device *bdev, fmode_t mode,
|
|||
if ((mode & FMODE_WRITE) || capable(CAP_SYS_ADMIN))
|
||||
err = loop_set_dio(lo, arg);
|
||||
break;
|
||||
case LOOP_SET_BLOCK_SIZE:
|
||||
err = -EPERM;
|
||||
if ((mode & FMODE_WRITE) || capable(CAP_SYS_ADMIN))
|
||||
err = loop_set_block_size(lo, arg);
|
||||
break;
|
||||
default:
|
||||
err = lo->ioctl ? lo->ioctl(lo, cmd, arg) : -EINVAL;
|
||||
}
|
||||
|
|
@ -1557,6 +1581,7 @@ static int lo_compat_ioctl(struct block_device *bdev, fmode_t mode,
|
|||
arg = (unsigned long) compat_ptr(arg);
|
||||
case LOOP_SET_FD:
|
||||
case LOOP_CHANGE_FD:
|
||||
case LOOP_SET_BLOCK_SIZE:
|
||||
err = lo_ioctl(bdev, mode, cmd, arg);
|
||||
break;
|
||||
default:
|
||||
|
|
@ -1789,6 +1814,7 @@ static int loop_add(struct loop_device **l, int i)
|
|||
}
|
||||
lo->lo_queue->queuedata = lo;
|
||||
|
||||
blk_queue_max_hw_sectors(lo->lo_queue, BLK_DEF_MAX_SECTORS);
|
||||
/*
|
||||
* It doesn't make sense to enable merge because the I/O
|
||||
* submitted to backing file is handled page by page.
|
||||
|
|
|
|||
|
|
@ -230,7 +230,11 @@ struct tpm_chip *tpmm_chip_alloc(struct device *dev,
|
|||
chip->cdev.owner = dev->driver->owner;
|
||||
chip->cdev.kobj.parent = &chip->dev.kobj;
|
||||
|
||||
devm_add_action(dev, (void (*)(void *)) put_device, &chip->dev);
|
||||
rc = devm_add_action(dev, (void (*)(void *)) put_device, &chip->dev);
|
||||
if (rc) {
|
||||
put_device(&chip->dev);
|
||||
return ERR_PTR(rc);
|
||||
}
|
||||
|
||||
return chip;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -688,9 +688,9 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
|
|||
}
|
||||
|
||||
irq = platform_get_irq(pdev, 0);
|
||||
if (irq <= 0) {
|
||||
dev_err(&pdev->dev, "Failed to get IRQ\n");
|
||||
return -ENODEV;
|
||||
if (irq < 0) {
|
||||
dev_err(&pdev->dev, "Failed to get IRQ: %d\n", irq);
|
||||
return irq;
|
||||
}
|
||||
|
||||
platform_set_drvdata(pdev, tegra);
|
||||
|
|
|
|||
|
|
@ -266,8 +266,8 @@ static int platform_msic_gpio_probe(struct platform_device *pdev)
|
|||
int i;
|
||||
|
||||
if (irq < 0) {
|
||||
dev_err(dev, "no IRQ line\n");
|
||||
return -EINVAL;
|
||||
dev_err(dev, "no IRQ line: %d\n", irq);
|
||||
return irq;
|
||||
}
|
||||
|
||||
if (!pdata || !pdata->gpio_base) {
|
||||
|
|
|
|||
|
|
@ -235,7 +235,7 @@ void
|
|||
nouveau_fbcon_accel_save_disable(struct drm_device *dev)
|
||||
{
|
||||
struct nouveau_drm *drm = nouveau_drm(dev);
|
||||
if (drm->fbcon) {
|
||||
if (drm->fbcon && drm->fbcon->helper.fbdev) {
|
||||
drm->fbcon->saved_flags = drm->fbcon->helper.fbdev->flags;
|
||||
drm->fbcon->helper.fbdev->flags |= FBINFO_HWACCEL_DISABLED;
|
||||
}
|
||||
|
|
@ -245,7 +245,7 @@ void
|
|||
nouveau_fbcon_accel_restore(struct drm_device *dev)
|
||||
{
|
||||
struct nouveau_drm *drm = nouveau_drm(dev);
|
||||
if (drm->fbcon) {
|
||||
if (drm->fbcon && drm->fbcon->helper.fbdev) {
|
||||
drm->fbcon->helper.fbdev->flags = drm->fbcon->saved_flags;
|
||||
}
|
||||
}
|
||||
|
|
@ -257,6 +257,7 @@ nouveau_fbcon_accel_fini(struct drm_device *dev)
|
|||
struct nouveau_fbdev *fbcon = drm->fbcon;
|
||||
if (fbcon && drm->channel) {
|
||||
console_lock();
|
||||
if (fbcon->helper.fbdev)
|
||||
fbcon->helper.fbdev->flags |= FBINFO_HWACCEL_DISABLED;
|
||||
console_unlock();
|
||||
nouveau_channel_idle(drm->channel);
|
||||
|
|
|
|||
|
|
@ -116,8 +116,8 @@ static int host1x_probe(struct platform_device *pdev)
|
|||
|
||||
syncpt_irq = platform_get_irq(pdev, 0);
|
||||
if (syncpt_irq < 0) {
|
||||
dev_err(&pdev->dev, "failed to get IRQ\n");
|
||||
return -ENXIO;
|
||||
dev_err(&pdev->dev, "failed to get IRQ: %d\n", syncpt_irq);
|
||||
return syncpt_irq;
|
||||
}
|
||||
|
||||
host = devm_kzalloc(&pdev->dev, sizeof(*host), GFP_KERNEL);
|
||||
|
|
|
|||
|
|
@ -46,6 +46,8 @@
|
|||
#include <linux/mutex.h>
|
||||
#include <linux/slab.h>
|
||||
|
||||
#include <linux/nospec.h>
|
||||
|
||||
#include <asm/uaccess.h>
|
||||
|
||||
#include <rdma/ib.h>
|
||||
|
|
@ -1115,6 +1117,7 @@ static ssize_t ib_ucm_write(struct file *filp, const char __user *buf,
|
|||
|
||||
if (hdr.cmd >= ARRAY_SIZE(ucm_cmd_table))
|
||||
return -EINVAL;
|
||||
hdr.cmd = array_index_nospec(hdr.cmd, ARRAY_SIZE(ucm_cmd_table));
|
||||
|
||||
if (hdr.in + sizeof(hdr) > len)
|
||||
return -EINVAL;
|
||||
|
|
|
|||
|
|
@ -44,6 +44,8 @@
|
|||
#include <linux/module.h>
|
||||
#include <linux/nsproxy.h>
|
||||
|
||||
#include <linux/nospec.h>
|
||||
|
||||
#include <rdma/rdma_user_cm.h>
|
||||
#include <rdma/ib_marshall.h>
|
||||
#include <rdma/rdma_cm.h>
|
||||
|
|
@ -1627,6 +1629,7 @@ static ssize_t ucma_write(struct file *filp, const char __user *buf,
|
|||
|
||||
if (hdr.cmd >= ARRAY_SIZE(ucma_cmd_table))
|
||||
return -EINVAL;
|
||||
hdr.cmd = array_index_nospec(hdr.cmd, ARRAY_SIZE(ucma_cmd_table));
|
||||
|
||||
if (hdr.in + sizeof(hdr) > len)
|
||||
return -EINVAL;
|
||||
|
|
|
|||
|
|
@ -1251,6 +1251,7 @@ static const struct acpi_device_id elan_acpi_id[] = {
|
|||
{ "ELAN0611", 0 },
|
||||
{ "ELAN0612", 0 },
|
||||
{ "ELAN0618", 0 },
|
||||
{ "ELAN061C", 0 },
|
||||
{ "ELAN061D", 0 },
|
||||
{ "ELAN0622", 0 },
|
||||
{ "ELAN1000", 0 },
|
||||
|
|
|
|||
|
|
@ -373,13 +373,7 @@ static void gigaset_freecshw(struct cardstate *cs)
|
|||
|
||||
static void gigaset_device_release(struct device *dev)
|
||||
{
|
||||
struct cardstate *cs = dev_get_drvdata(dev);
|
||||
|
||||
if (!cs)
|
||||
return;
|
||||
dev_set_drvdata(dev, NULL);
|
||||
kfree(cs->hw.ser);
|
||||
cs->hw.ser = NULL;
|
||||
kfree(container_of(dev, struct ser_cardstate, dev.dev));
|
||||
}
|
||||
|
||||
/*
|
||||
|
|
@ -408,7 +402,6 @@ static int gigaset_initcshw(struct cardstate *cs)
|
|||
cs->hw.ser = NULL;
|
||||
return rc;
|
||||
}
|
||||
dev_set_drvdata(&cs->hw.ser->dev.dev, cs);
|
||||
|
||||
tasklet_init(&cs->write_tasklet,
|
||||
gigaset_modem_fill, (unsigned long) cs);
|
||||
|
|
|
|||
|
|
@ -1461,13 +1461,6 @@ static int usbvision_probe(struct usb_interface *intf,
|
|||
printk(KERN_INFO "%s: %s found\n", __func__,
|
||||
usbvision_device_data[model].model_string);
|
||||
|
||||
/*
|
||||
* this is a security check.
|
||||
* an exploit using an incorrect bInterfaceNumber is known
|
||||
*/
|
||||
if (ifnum >= USB_MAXINTERFACES || !dev->actconfig->interface[ifnum])
|
||||
return -ENODEV;
|
||||
|
||||
if (usbvision_device_data[model].interface >= 0)
|
||||
interface = &dev->actconfig->interface[usbvision_device_data[model].interface]->altsetting[0];
|
||||
else if (ifnum < dev->actconfig->desc.bNumInterfaces)
|
||||
|
|
|
|||
|
|
@ -2048,6 +2048,7 @@ int db8500_prcmu_config_hotmon(u8 low, u8 high)
|
|||
|
||||
return 0;
|
||||
}
|
||||
EXPORT_SYMBOL_GPL(db8500_prcmu_config_hotmon);
|
||||
|
||||
static int config_hot_period(u16 val)
|
||||
{
|
||||
|
|
@ -2074,11 +2075,13 @@ int db8500_prcmu_start_temp_sense(u16 cycles32k)
|
|||
|
||||
return config_hot_period(cycles32k);
|
||||
}
|
||||
EXPORT_SYMBOL_GPL(db8500_prcmu_start_temp_sense);
|
||||
|
||||
int db8500_prcmu_stop_temp_sense(void)
|
||||
{
|
||||
return config_hot_period(0xFFFF);
|
||||
}
|
||||
EXPORT_SYMBOL_GPL(db8500_prcmu_stop_temp_sense);
|
||||
|
||||
static int prcmu_a9wdog(u8 cmd, u8 d0, u8 d1, u8 d2, u8 d3)
|
||||
{
|
||||
|
|
|
|||
|
|
@ -708,6 +708,12 @@ static const struct flash_info spi_nor_ids[] = {
|
|||
|
||||
/* ISSI */
|
||||
{ "is25cd512", INFO(0x7f9d20, 0, 32 * 1024, 2, SECT_4K) },
|
||||
{ "is25wp032", INFO(0x9d7016, 0, 64 * 1024, 64,
|
||||
SECT_4K | SPI_NOR_DUAL_READ | SPI_NOR_QUAD_READ) },
|
||||
{ "is25wp064", INFO(0x9d7017, 0, 64 * 1024, 128,
|
||||
SECT_4K | SPI_NOR_DUAL_READ | SPI_NOR_QUAD_READ) },
|
||||
{ "is25wp128", INFO(0x9d7018, 0, 64 * 1024, 256,
|
||||
SECT_4K | SPI_NOR_DUAL_READ | SPI_NOR_QUAD_READ) },
|
||||
|
||||
/* Macronix */
|
||||
{ "mx25l512e", INFO(0xc22010, 0, 64 * 1024, 1, SECT_4K) },
|
||||
|
|
|
|||
|
|
@ -1107,11 +1107,11 @@ static void bond_compute_features(struct bonding *bond)
|
|||
gso_max_size = min(gso_max_size, slave->dev->gso_max_size);
|
||||
gso_max_segs = min(gso_max_segs, slave->dev->gso_max_segs);
|
||||
}
|
||||
bond_dev->hard_header_len = max_hard_header_len;
|
||||
|
||||
done:
|
||||
bond_dev->vlan_features = vlan_features;
|
||||
bond_dev->hw_enc_features = enc_features | NETIF_F_GSO_ENCAP_ALL;
|
||||
bond_dev->hard_header_len = max_hard_header_len;
|
||||
bond_dev->gso_max_segs = gso_max_segs;
|
||||
netif_set_gso_max_size(bond_dev, gso_max_size);
|
||||
|
||||
|
|
|
|||
|
|
@ -2147,6 +2147,8 @@ static int cxgb_extension_ioctl(struct net_device *dev, void __user *useraddr)
|
|||
return -EPERM;
|
||||
if (copy_from_user(&t, useraddr, sizeof(t)))
|
||||
return -EFAULT;
|
||||
if (t.cmd != CHELSIO_SET_QSET_PARAMS)
|
||||
return -EINVAL;
|
||||
if (t.qset_idx >= SGE_QSETS)
|
||||
return -EINVAL;
|
||||
if (!in_range(t.intr_lat, 0, M_NEWTIMER) ||
|
||||
|
|
@ -2246,6 +2248,9 @@ static int cxgb_extension_ioctl(struct net_device *dev, void __user *useraddr)
|
|||
if (copy_from_user(&t, useraddr, sizeof(t)))
|
||||
return -EFAULT;
|
||||
|
||||
if (t.cmd != CHELSIO_GET_QSET_PARAMS)
|
||||
return -EINVAL;
|
||||
|
||||
/* Display qsets for all ports when offload enabled */
|
||||
if (test_bit(OFFLOAD_DEVMAP_BIT, &adapter->open_device_map)) {
|
||||
q1 = 0;
|
||||
|
|
@ -2291,6 +2296,8 @@ static int cxgb_extension_ioctl(struct net_device *dev, void __user *useraddr)
|
|||
return -EBUSY;
|
||||
if (copy_from_user(&edata, useraddr, sizeof(edata)))
|
||||
return -EFAULT;
|
||||
if (edata.cmd != CHELSIO_SET_QSET_NUM)
|
||||
return -EINVAL;
|
||||
if (edata.val < 1 ||
|
||||
(edata.val > 1 && !(adapter->flags & USING_MSIX)))
|
||||
return -EINVAL;
|
||||
|
|
@ -2331,6 +2338,8 @@ static int cxgb_extension_ioctl(struct net_device *dev, void __user *useraddr)
|
|||
return -EPERM;
|
||||
if (copy_from_user(&t, useraddr, sizeof(t)))
|
||||
return -EFAULT;
|
||||
if (t.cmd != CHELSIO_LOAD_FW)
|
||||
return -EINVAL;
|
||||
/* Check t.len sanity ? */
|
||||
fw_data = memdup_user(useraddr + sizeof(t), t.len);
|
||||
if (IS_ERR(fw_data))
|
||||
|
|
@ -2354,6 +2363,8 @@ static int cxgb_extension_ioctl(struct net_device *dev, void __user *useraddr)
|
|||
return -EBUSY;
|
||||
if (copy_from_user(&m, useraddr, sizeof(m)))
|
||||
return -EFAULT;
|
||||
if (m.cmd != CHELSIO_SETMTUTAB)
|
||||
return -EINVAL;
|
||||
if (m.nmtus != NMTUS)
|
||||
return -EINVAL;
|
||||
if (m.mtus[0] < 81) /* accommodate SACK */
|
||||
|
|
@ -2395,6 +2406,8 @@ static int cxgb_extension_ioctl(struct net_device *dev, void __user *useraddr)
|
|||
return -EBUSY;
|
||||
if (copy_from_user(&m, useraddr, sizeof(m)))
|
||||
return -EFAULT;
|
||||
if (m.cmd != CHELSIO_SET_PM)
|
||||
return -EINVAL;
|
||||
if (!is_power_of_2(m.rx_pg_sz) ||
|
||||
!is_power_of_2(m.tx_pg_sz))
|
||||
return -EINVAL; /* not power of 2 */
|
||||
|
|
@ -2428,6 +2441,8 @@ static int cxgb_extension_ioctl(struct net_device *dev, void __user *useraddr)
|
|||
return -EIO; /* need the memory controllers */
|
||||
if (copy_from_user(&t, useraddr, sizeof(t)))
|
||||
return -EFAULT;
|
||||
if (t.cmd != CHELSIO_GET_MEM)
|
||||
return -EINVAL;
|
||||
if ((t.addr & 7) || (t.len & 7))
|
||||
return -EINVAL;
|
||||
if (t.mem_id == MEM_CM)
|
||||
|
|
@ -2480,6 +2495,8 @@ static int cxgb_extension_ioctl(struct net_device *dev, void __user *useraddr)
|
|||
return -EAGAIN;
|
||||
if (copy_from_user(&t, useraddr, sizeof(t)))
|
||||
return -EFAULT;
|
||||
if (t.cmd != CHELSIO_SET_TRACE_FILTER)
|
||||
return -EINVAL;
|
||||
|
||||
tp = (const struct trace_params *)&t.sip;
|
||||
if (t.config_tx)
|
||||
|
|
|
|||
|
|
@ -223,17 +223,6 @@ static s32 igb_init_phy_params_82575(struct e1000_hw *hw)
|
|||
hw->bus.func = (rd32(E1000_STATUS) & E1000_STATUS_FUNC_MASK) >>
|
||||
E1000_STATUS_FUNC_SHIFT;
|
||||
|
||||
/* Make sure the PHY is in a good state. Several people have reported
|
||||
* firmware leaving the PHY's page select register set to something
|
||||
* other than the default of zero, which causes the PHY ID read to
|
||||
* access something other than the intended register.
|
||||
*/
|
||||
ret_val = hw->phy.ops.reset(hw);
|
||||
if (ret_val) {
|
||||
hw_dbg("Error resetting the PHY.\n");
|
||||
goto out;
|
||||
}
|
||||
|
||||
/* Set phy->phy_addr and phy->id. */
|
||||
ret_val = igb_get_phy_id_82575(hw);
|
||||
if (ret_val)
|
||||
|
|
|
|||
|
|
@ -312,7 +312,7 @@ enum ixgbe_ring_f_enum {
|
|||
};
|
||||
|
||||
#define IXGBE_MAX_RSS_INDICES 16
|
||||
#define IXGBE_MAX_RSS_INDICES_X550 64
|
||||
#define IXGBE_MAX_RSS_INDICES_X550 63
|
||||
#define IXGBE_MAX_VMDQ_INDICES 64
|
||||
#define IXGBE_MAX_FDIR_INDICES 63 /* based on q_vector limit */
|
||||
#define IXGBE_MAX_FCOE_INDICES 8
|
||||
|
|
|
|||
|
|
@ -3508,7 +3508,7 @@ struct ixgbe_info {
|
|||
|
||||
#define IXGBE_FUSES0_GROUP(_i) (0x11158 + ((_i) * 4))
|
||||
#define IXGBE_FUSES0_300MHZ BIT(5)
|
||||
#define IXGBE_FUSES0_REV1 BIT(6)
|
||||
#define IXGBE_FUSES0_REV_MASK (3 << 6)
|
||||
|
||||
#define IXGBE_KRM_PORT_CAR_GEN_CTRL(P) ((P) ? 0x8010 : 0x4010)
|
||||
#define IXGBE_KRM_LINK_CTRL_1(P) ((P) ? 0x820C : 0x420C)
|
||||
|
|
|
|||
|
|
@ -1873,10 +1873,6 @@ static s32 ixgbe_enter_lplu_t_x550em(struct ixgbe_hw *hw)
|
|||
u32 save_autoneg;
|
||||
bool link_up;
|
||||
|
||||
/* SW LPLU not required on later HW revisions. */
|
||||
if (IXGBE_FUSES0_REV1 & IXGBE_READ_REG(hw, IXGBE_FUSES0_GROUP(0)))
|
||||
return 0;
|
||||
|
||||
/* If blocked by MNG FW, then don't restart AN */
|
||||
if (ixgbe_check_reset_blocked(hw))
|
||||
return 0;
|
||||
|
|
@ -2030,8 +2026,9 @@ static s32 ixgbe_init_phy_ops_X550em(struct ixgbe_hw *hw)
|
|||
}
|
||||
|
||||
/* setup SW LPLU only for first revision */
|
||||
if (!(IXGBE_FUSES0_REV1 & IXGBE_READ_REG(hw,
|
||||
IXGBE_FUSES0_GROUP(0))))
|
||||
if (hw->mac.type == ixgbe_mac_X550EM_x &&
|
||||
!(IXGBE_READ_REG(hw, IXGBE_FUSES0_GROUP(0)) &
|
||||
IXGBE_FUSES0_REV_MASK))
|
||||
phy->ops.enter_lplu = ixgbe_enter_lplu_t_x550em;
|
||||
|
||||
phy->ops.handle_lasi = ixgbe_handle_lasi_ext_t_x550em;
|
||||
|
|
|
|||
|
|
@ -1014,6 +1014,8 @@ static int ixgbevf_poll(struct napi_struct *napi, int budget)
|
|||
ixgbevf_for_each_ring(ring, q_vector->tx)
|
||||
clean_complete &= ixgbevf_clean_tx_irq(q_vector, ring);
|
||||
|
||||
if (budget <= 0)
|
||||
return budget;
|
||||
#ifdef CONFIG_NET_RX_BUSY_POLL
|
||||
if (!ixgbevf_qv_lock_napi(q_vector))
|
||||
return budget;
|
||||
|
|
|
|||
|
|
@ -251,8 +251,11 @@ static u32 freq_to_shift(u16 freq)
|
|||
{
|
||||
u32 freq_khz = freq * 1000;
|
||||
u64 max_val_cycles = freq_khz * 1000 * MLX4_EN_WRAP_AROUND_SEC;
|
||||
u64 tmp_rounded =
|
||||
roundup_pow_of_two(max_val_cycles) > max_val_cycles ?
|
||||
roundup_pow_of_two(max_val_cycles) - 1 : UINT_MAX;
|
||||
u64 max_val_cycles_rounded = is_power_of_2(max_val_cycles + 1) ?
|
||||
max_val_cycles : roundup_pow_of_two(max_val_cycles) - 1;
|
||||
max_val_cycles : tmp_rounded;
|
||||
/* calculate max possible multiplier in order to fit in 64bit */
|
||||
u64 max_mul = div_u64(0xffffffffffffffffULL, max_val_cycles_rounded);
|
||||
|
||||
|
|
|
|||
|
|
@ -586,6 +586,8 @@ int mlx5e_redirect_rqt(struct mlx5e_priv *priv, enum mlx5e_rqt_ix rqt_ix);
|
|||
|
||||
int mlx5e_open_locked(struct net_device *netdev);
|
||||
int mlx5e_close_locked(struct net_device *netdev);
|
||||
void mlx5e_build_default_indir_rqt(u32 *indirection_rqt, int len,
|
||||
int num_channels);
|
||||
|
||||
static inline void mlx5e_tx_notify_hw(struct mlx5e_sq *sq,
|
||||
struct mlx5e_tx_wqe *wqe, int bf_sz)
|
||||
|
|
|
|||
|
|
@ -385,6 +385,8 @@ static int mlx5e_set_channels(struct net_device *dev,
|
|||
mlx5e_close_locked(dev);
|
||||
|
||||
priv->params.num_channels = count;
|
||||
mlx5e_build_default_indir_rqt(priv->params.indirection_rqt,
|
||||
MLX5E_INDIR_RQT_SIZE, count);
|
||||
|
||||
if (was_opened)
|
||||
err = mlx5e_open_locked(dev);
|
||||
|
|
|
|||
|
|
@ -1186,7 +1186,6 @@ static void mlx5e_fill_indir_rqt_rqns(struct mlx5e_priv *priv, void *rqtc)
|
|||
ix = mlx5e_bits_invert(i, MLX5E_LOG_INDIR_RQT_SIZE);
|
||||
|
||||
ix = priv->params.indirection_rqt[ix];
|
||||
ix = ix % priv->params.num_channels;
|
||||
MLX5_SET(rqtc, rqtc, rq_num[i],
|
||||
test_bit(MLX5E_STATE_OPENED, &priv->state) ?
|
||||
priv->channel[ix]->rq.rqn :
|
||||
|
|
@ -1304,7 +1303,7 @@ static void mlx5e_build_tir_ctx_lro(void *tirc, struct mlx5e_priv *priv)
|
|||
lro_timer_supported_periods[2]));
|
||||
}
|
||||
|
||||
static int mlx5e_modify_tir_lro(struct mlx5e_priv *priv, int tt)
|
||||
static int mlx5e_modify_tirs_lro(struct mlx5e_priv *priv)
|
||||
{
|
||||
struct mlx5_core_dev *mdev = priv->mdev;
|
||||
|
||||
|
|
@ -1312,6 +1311,7 @@ static int mlx5e_modify_tir_lro(struct mlx5e_priv *priv, int tt)
|
|||
void *tirc;
|
||||
int inlen;
|
||||
int err;
|
||||
int tt;
|
||||
|
||||
inlen = MLX5_ST_SZ_BYTES(modify_tir_in);
|
||||
in = mlx5_vzalloc(inlen);
|
||||
|
|
@ -1323,7 +1323,11 @@ static int mlx5e_modify_tir_lro(struct mlx5e_priv *priv, int tt)
|
|||
|
||||
mlx5e_build_tir_ctx_lro(tirc, priv);
|
||||
|
||||
for (tt = 0; tt < MLX5E_NUM_TT; tt++) {
|
||||
err = mlx5_core_modify_tir(mdev, priv->tirn[tt], in, inlen);
|
||||
if (err)
|
||||
break;
|
||||
}
|
||||
|
||||
kvfree(in);
|
||||
|
||||
|
|
@ -1870,8 +1874,10 @@ static int mlx5e_set_features(struct net_device *netdev,
|
|||
mlx5e_close_locked(priv->netdev);
|
||||
|
||||
priv->params.lro_en = !!(features & NETIF_F_LRO);
|
||||
mlx5e_modify_tir_lro(priv, MLX5E_TT_IPV4_TCP);
|
||||
mlx5e_modify_tir_lro(priv, MLX5E_TT_IPV6_TCP);
|
||||
err = mlx5e_modify_tirs_lro(priv);
|
||||
if (err)
|
||||
mlx5_core_warn(priv->mdev, "lro modify failed, %d\n",
|
||||
err);
|
||||
|
||||
if (was_opened)
|
||||
err = mlx5e_open_locked(priv->netdev);
|
||||
|
|
@ -1976,12 +1982,20 @@ u16 mlx5e_get_max_inline_cap(struct mlx5_core_dev *mdev)
|
|||
2 /*sizeof(mlx5e_tx_wqe.inline_hdr_start)*/;
|
||||
}
|
||||
|
||||
void mlx5e_build_default_indir_rqt(u32 *indirection_rqt, int len,
|
||||
int num_channels)
|
||||
{
|
||||
int i;
|
||||
|
||||
for (i = 0; i < len; i++)
|
||||
indirection_rqt[i] = i % num_channels;
|
||||
}
|
||||
|
||||
static void mlx5e_build_netdev_priv(struct mlx5_core_dev *mdev,
|
||||
struct net_device *netdev,
|
||||
int num_channels)
|
||||
{
|
||||
struct mlx5e_priv *priv = netdev_priv(netdev);
|
||||
int i;
|
||||
|
||||
priv->params.log_sq_size =
|
||||
MLX5E_PARAMS_DEFAULT_LOG_SQ_SIZE;
|
||||
|
|
@ -2005,8 +2019,8 @@ static void mlx5e_build_netdev_priv(struct mlx5_core_dev *mdev,
|
|||
netdev_rss_key_fill(priv->params.toeplitz_hash_key,
|
||||
sizeof(priv->params.toeplitz_hash_key));
|
||||
|
||||
for (i = 0; i < MLX5E_INDIR_RQT_SIZE; i++)
|
||||
priv->params.indirection_rqt[i] = i % num_channels;
|
||||
mlx5e_build_default_indir_rqt(priv->params.indirection_rqt,
|
||||
MLX5E_INDIR_RQT_SIZE, num_channels);
|
||||
|
||||
priv->params.lro_wqe_sz =
|
||||
MLX5E_PARAMS_DEFAULT_LRO_WQE_SZ;
|
||||
|
|
|
|||
|
|
@ -7540,16 +7540,14 @@ static int rtl8169_poll(struct napi_struct *napi, int budget)
|
|||
struct rtl8169_private *tp = container_of(napi, struct rtl8169_private, napi);
|
||||
struct net_device *dev = tp->dev;
|
||||
u16 enable_mask = RTL_EVENT_NAPI | tp->event_slow;
|
||||
int work_done= 0;
|
||||
int work_done;
|
||||
u16 status;
|
||||
|
||||
status = rtl_get_events(tp);
|
||||
rtl_ack_events(tp, status & ~tp->event_slow);
|
||||
|
||||
if (status & RTL_EVENT_NAPI_RX)
|
||||
work_done = rtl_rx(dev, tp, (u32) budget);
|
||||
|
||||
if (status & RTL_EVENT_NAPI_TX)
|
||||
rtl_tx(dev, tp);
|
||||
|
||||
if (status & tp->event_slow) {
|
||||
|
|
|
|||
|
|
@ -130,7 +130,7 @@ static int stmmac_mdio_write(struct mii_bus *bus, int phyaddr, int phyreg,
|
|||
*/
|
||||
int stmmac_mdio_reset(struct mii_bus *bus)
|
||||
{
|
||||
#if defined(CONFIG_STMMAC_PLATFORM)
|
||||
#if IS_ENABLED(CONFIG_STMMAC_PLATFORM)
|
||||
struct net_device *ndev = bus->priv;
|
||||
struct stmmac_priv *priv = netdev_priv(ndev);
|
||||
unsigned int mii_address = priv->hw->mii.addr;
|
||||
|
|
|
|||
|
|
@ -449,6 +449,9 @@ int asix_set_wol(struct net_device *net, struct ethtool_wolinfo *wolinfo)
|
|||
struct usbnet *dev = netdev_priv(net);
|
||||
u8 opt = 0;
|
||||
|
||||
if (wolinfo->wolopts & ~(WAKE_PHY | WAKE_MAGIC))
|
||||
return -EINVAL;
|
||||
|
||||
if (wolinfo->wolopts & WAKE_PHY)
|
||||
opt |= AX_MONITOR_LINK;
|
||||
if (wolinfo->wolopts & WAKE_MAGIC)
|
||||
|
|
|
|||
|
|
@ -566,6 +566,9 @@ ax88179_set_wol(struct net_device *net, struct ethtool_wolinfo *wolinfo)
|
|||
struct usbnet *dev = netdev_priv(net);
|
||||
u8 opt = 0;
|
||||
|
||||
if (wolinfo->wolopts & ~(WAKE_PHY | WAKE_MAGIC))
|
||||
return -EINVAL;
|
||||
|
||||
if (wolinfo->wolopts & WAKE_PHY)
|
||||
opt |= AX_MONITOR_MODE_RWLC;
|
||||
if (wolinfo->wolopts & WAKE_MAGIC)
|
||||
|
|
|
|||
|
|
@ -1051,19 +1051,10 @@ static int lan78xx_set_wol(struct net_device *netdev,
|
|||
if (ret < 0)
|
||||
return ret;
|
||||
|
||||
pdata->wol = 0;
|
||||
if (wol->wolopts & WAKE_UCAST)
|
||||
pdata->wol |= WAKE_UCAST;
|
||||
if (wol->wolopts & WAKE_MCAST)
|
||||
pdata->wol |= WAKE_MCAST;
|
||||
if (wol->wolopts & WAKE_BCAST)
|
||||
pdata->wol |= WAKE_BCAST;
|
||||
if (wol->wolopts & WAKE_MAGIC)
|
||||
pdata->wol |= WAKE_MAGIC;
|
||||
if (wol->wolopts & WAKE_PHY)
|
||||
pdata->wol |= WAKE_PHY;
|
||||
if (wol->wolopts & WAKE_ARP)
|
||||
pdata->wol |= WAKE_ARP;
|
||||
if (wol->wolopts & ~WAKE_ALL)
|
||||
return -EINVAL;
|
||||
|
||||
pdata->wol = wol->wolopts;
|
||||
|
||||
device_set_wakeup_enable(&dev->udev->dev, (bool)wol->wolopts);
|
||||
|
||||
|
|
|
|||
|
|
@ -3663,6 +3663,9 @@ static int rtl8152_set_wol(struct net_device *dev, struct ethtool_wolinfo *wol)
|
|||
if (!rtl_can_wakeup(tp))
|
||||
return -EOPNOTSUPP;
|
||||
|
||||
if (wol->wolopts & ~WAKE_ANY)
|
||||
return -EINVAL;
|
||||
|
||||
ret = usb_autopm_get_interface(tp->intf);
|
||||
if (ret < 0)
|
||||
goto out_set_wol;
|
||||
|
|
|
|||
|
|
@ -728,6 +728,9 @@ static int smsc75xx_ethtool_set_wol(struct net_device *net,
|
|||
struct smsc75xx_priv *pdata = (struct smsc75xx_priv *)(dev->data[0]);
|
||||
int ret;
|
||||
|
||||
if (wolinfo->wolopts & ~SUPPORTED_WAKE)
|
||||
return -EINVAL;
|
||||
|
||||
pdata->wolopts = wolinfo->wolopts & SUPPORTED_WAKE;
|
||||
|
||||
ret = device_set_wakeup_enable(&dev->udev->dev, pdata->wolopts);
|
||||
|
|
|
|||
|
|
@ -727,6 +727,9 @@ static int smsc95xx_ethtool_set_wol(struct net_device *net,
|
|||
struct smsc95xx_priv *pdata = (struct smsc95xx_priv *)(dev->data[0]);
|
||||
int ret;
|
||||
|
||||
if (wolinfo->wolopts & ~SUPPORTED_WAKE)
|
||||
return -EINVAL;
|
||||
|
||||
pdata->wolopts = wolinfo->wolopts & SUPPORTED_WAKE;
|
||||
|
||||
ret = device_set_wakeup_enable(&dev->udev->dev, pdata->wolopts);
|
||||
|
|
|
|||
|
|
@ -421,6 +421,9 @@ sr_set_wol(struct net_device *net, struct ethtool_wolinfo *wolinfo)
|
|||
struct usbnet *dev = netdev_priv(net);
|
||||
u8 opt = 0;
|
||||
|
||||
if (wolinfo->wolopts & ~(WAKE_PHY | WAKE_MAGIC))
|
||||
return -EINVAL;
|
||||
|
||||
if (wolinfo->wolopts & WAKE_PHY)
|
||||
opt |= SR_MONITOR_LINK;
|
||||
if (wolinfo->wolopts & WAKE_MAGIC)
|
||||
|
|
|
|||
|
|
@ -705,7 +705,7 @@ done:
|
|||
int brcmf_sdiod_recv_chain(struct brcmf_sdio_dev *sdiodev,
|
||||
struct sk_buff_head *pktq, uint totlen)
|
||||
{
|
||||
struct sk_buff *glom_skb;
|
||||
struct sk_buff *glom_skb = NULL;
|
||||
struct sk_buff *skb;
|
||||
u32 addr = sdiodev->sbwad;
|
||||
int err = 0;
|
||||
|
|
@ -726,10 +726,8 @@ int brcmf_sdiod_recv_chain(struct brcmf_sdio_dev *sdiodev,
|
|||
return -ENOMEM;
|
||||
err = brcmf_sdiod_buffrw(sdiodev, SDIO_FUNC_2, false, addr,
|
||||
glom_skb);
|
||||
if (err) {
|
||||
brcmu_pkt_buf_free_skb(glom_skb);
|
||||
if (err)
|
||||
goto done;
|
||||
}
|
||||
|
||||
skb_queue_walk(pktq, skb) {
|
||||
memcpy(skb->data, glom_skb->data, skb->len);
|
||||
|
|
@ -740,6 +738,7 @@ int brcmf_sdiod_recv_chain(struct brcmf_sdio_dev *sdiodev,
|
|||
pktq);
|
||||
|
||||
done:
|
||||
brcmu_pkt_buf_free_skb(glom_skb);
|
||||
return err;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -2547,7 +2547,6 @@ static int mac80211_hwsim_new_radio(struct genl_info *info,
|
|||
list_add_tail(&data->list, &hwsim_radios);
|
||||
spin_unlock_bh(&hwsim_radio_lock);
|
||||
|
||||
if (idx > 0)
|
||||
hwsim_mcast_new_radio(idx, info, param);
|
||||
|
||||
return idx;
|
||||
|
|
|
|||
|
|
@ -23,6 +23,8 @@
|
|||
#include <linux/sched.h>
|
||||
#include <linux/slab.h>
|
||||
|
||||
#include <linux/nospec.h>
|
||||
|
||||
#include "ptp_private.h"
|
||||
|
||||
static int ptp_disable_pinfunc(struct ptp_clock_info *ops,
|
||||
|
|
@ -224,6 +226,7 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd, unsigned long arg)
|
|||
err = -EINVAL;
|
||||
break;
|
||||
}
|
||||
pin_index = array_index_nospec(pin_index, ops->n_pins);
|
||||
if (mutex_lock_interruptible(&ptp->pincfg_mux))
|
||||
return -ERESTARTSYS;
|
||||
pd = ops->pin_config[pin_index];
|
||||
|
|
@ -242,6 +245,7 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd, unsigned long arg)
|
|||
err = -EINVAL;
|
||||
break;
|
||||
}
|
||||
pin_index = array_index_nospec(pin_index, ops->n_pins);
|
||||
if (mutex_lock_interruptible(&ptp->pincfg_mux))
|
||||
return -ERESTARTSYS;
|
||||
err = ptp_set_pinfunc(ptp, pin_index, pd.func, pd.chan);
|
||||
|
|
|
|||
|
|
@ -1416,8 +1416,8 @@ static int aac_acquire_resources(struct aac_dev *dev)
|
|||
/* After EEH recovery or suspend resume, max_msix count
|
||||
* may change, therfore updating in init as well.
|
||||
*/
|
||||
aac_adapter_start(dev);
|
||||
dev->init->Sa_MSIXVectors = cpu_to_le32(dev->max_msix);
|
||||
aac_adapter_start(dev);
|
||||
}
|
||||
return 0;
|
||||
|
||||
|
|
|
|||
|
|
@ -444,7 +444,7 @@ err_out:
|
|||
return -1;
|
||||
|
||||
err_blink:
|
||||
return (status > 16) & 0xFF;
|
||||
return (status >> 16) & 0xFF;
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -381,11 +381,12 @@ static void scsi_target_reap_ref_release(struct kref *kref)
|
|||
= container_of(kref, struct scsi_target, reap_ref);
|
||||
|
||||
/*
|
||||
* if we get here and the target is still in the CREATED state that
|
||||
* if we get here and the target is still in a CREATED state that
|
||||
* means it was allocated but never made visible (because a scan
|
||||
* turned up no LUNs), so don't call device_del() on it.
|
||||
*/
|
||||
if (starget->state != STARGET_CREATED) {
|
||||
if ((starget->state != STARGET_CREATED) &&
|
||||
(starget->state != STARGET_CREATED_REMOVE)) {
|
||||
transport_remove_device(&starget->dev);
|
||||
device_del(&starget->dev);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1213,10 +1213,14 @@ restart:
|
|||
spin_lock_irqsave(shost->host_lock, flags);
|
||||
list_for_each_entry(starget, &shost->__targets, siblings) {
|
||||
if (starget->state == STARGET_DEL ||
|
||||
starget->state == STARGET_REMOVE)
|
||||
starget->state == STARGET_REMOVE ||
|
||||
starget->state == STARGET_CREATED_REMOVE)
|
||||
continue;
|
||||
if (starget->dev.parent == dev || &starget->dev == dev) {
|
||||
kref_get(&starget->reap_ref);
|
||||
if (starget->state == STARGET_CREATED)
|
||||
starget->state = STARGET_CREATED_REMOVE;
|
||||
else
|
||||
starget->state = STARGET_REMOVE;
|
||||
spin_unlock_irqrestore(shost->host_lock, flags);
|
||||
__scsi_remove_target(starget);
|
||||
|
|
|
|||
|
|
@ -336,8 +336,8 @@ static int bcm63xx_hsspi_probe(struct platform_device *pdev)
|
|||
|
||||
irq = platform_get_irq(pdev, 0);
|
||||
if (irq < 0) {
|
||||
dev_err(dev, "no irq\n");
|
||||
return -ENXIO;
|
||||
dev_err(dev, "no irq: %d\n", irq);
|
||||
return irq;
|
||||
}
|
||||
|
||||
res_mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
|
||||
|
|
|
|||
|
|
@ -496,8 +496,8 @@ static int bcm63xx_spi_probe(struct platform_device *pdev)
|
|||
|
||||
irq = platform_get_irq(pdev, 0);
|
||||
if (irq < 0) {
|
||||
dev_err(dev, "no irq\n");
|
||||
return -ENXIO;
|
||||
dev_err(dev, "no irq: %d\n", irq);
|
||||
return irq;
|
||||
}
|
||||
|
||||
clk = devm_clk_get(dev, "spi");
|
||||
|
|
|
|||
|
|
@ -392,8 +392,8 @@ static int xlp_spi_probe(struct platform_device *pdev)
|
|||
|
||||
irq = platform_get_irq(pdev, 0);
|
||||
if (irq < 0) {
|
||||
dev_err(&pdev->dev, "no IRQ resource found\n");
|
||||
return -EINVAL;
|
||||
dev_err(&pdev->dev, "no IRQ resource found: %d\n", irq);
|
||||
return irq;
|
||||
}
|
||||
err = devm_request_irq(&pdev->dev, irq, xlp_spi_interrupt, 0,
|
||||
pdev->name, xspi);
|
||||
|
|
|
|||
|
|
@ -235,7 +235,7 @@ config IMX_THERMAL
|
|||
passive trip is crossed.
|
||||
|
||||
config SPEAR_THERMAL
|
||||
bool "SPEAr thermal sensor driver"
|
||||
tristate "SPEAr thermal sensor driver"
|
||||
depends on PLAT_SPEAR || COMPILE_TEST
|
||||
depends on OF
|
||||
help
|
||||
|
|
@ -277,8 +277,8 @@ config DOVE_THERMAL
|
|||
framework.
|
||||
|
||||
config DB8500_THERMAL
|
||||
bool "DB8500 thermal management"
|
||||
depends on ARCH_U8500
|
||||
tristate "DB8500 thermal management"
|
||||
depends on MFD_DB8500_PRCMU
|
||||
default y
|
||||
help
|
||||
Adds DB8500 thermal management implementation according to the thermal
|
||||
|
|
|
|||
|
|
@ -729,8 +729,8 @@ static int sprd_probe(struct platform_device *pdev)
|
|||
|
||||
irq = platform_get_irq(pdev, 0);
|
||||
if (irq < 0) {
|
||||
dev_err(&pdev->dev, "not provide irq resource\n");
|
||||
return -ENODEV;
|
||||
dev_err(&pdev->dev, "not provide irq resource: %d\n", irq);
|
||||
return irq;
|
||||
}
|
||||
up->irq = irq;
|
||||
|
||||
|
|
|
|||
|
|
@ -332,17 +332,17 @@ static void acm_ctrl_irq(struct urb *urb)
|
|||
|
||||
if (difference & ACM_CTRL_DSR)
|
||||
acm->iocount.dsr++;
|
||||
if (difference & ACM_CTRL_BRK)
|
||||
acm->iocount.brk++;
|
||||
if (difference & ACM_CTRL_RI)
|
||||
acm->iocount.rng++;
|
||||
if (difference & ACM_CTRL_DCD)
|
||||
acm->iocount.dcd++;
|
||||
if (difference & ACM_CTRL_FRAMING)
|
||||
if (newctrl & ACM_CTRL_BRK)
|
||||
acm->iocount.brk++;
|
||||
if (newctrl & ACM_CTRL_RI)
|
||||
acm->iocount.rng++;
|
||||
if (newctrl & ACM_CTRL_FRAMING)
|
||||
acm->iocount.frame++;
|
||||
if (difference & ACM_CTRL_PARITY)
|
||||
if (newctrl & ACM_CTRL_PARITY)
|
||||
acm->iocount.parity++;
|
||||
if (difference & ACM_CTRL_OVERRUN)
|
||||
if (newctrl & ACM_CTRL_OVERRUN)
|
||||
acm->iocount.overrun++;
|
||||
spin_unlock(&acm->read_lock);
|
||||
|
||||
|
|
|
|||
|
|
@ -1329,8 +1329,6 @@ static int proc_do_submiturb(struct usb_dev_state *ps, struct usbdevfs_urb *uurb
|
|||
u = 0;
|
||||
switch (uurb->type) {
|
||||
case USBDEVFS_URB_TYPE_CONTROL:
|
||||
if (is_in)
|
||||
allow_short = true;
|
||||
if (!usb_endpoint_xfer_control(&ep->desc))
|
||||
return -EINVAL;
|
||||
/* min 8 byte setup packet */
|
||||
|
|
@ -1360,6 +1358,8 @@ static int proc_do_submiturb(struct usb_dev_state *ps, struct usbdevfs_urb *uurb
|
|||
is_in = 0;
|
||||
uurb->endpoint &= ~USB_DIR_IN;
|
||||
}
|
||||
if (is_in)
|
||||
allow_short = true;
|
||||
snoop(&ps->dev->dev, "control urb: bRequestType=%02x "
|
||||
"bRequest=%02x wValue=%04x "
|
||||
"wIndex=%04x wLength=%04x\n",
|
||||
|
|
|
|||
|
|
@ -469,8 +469,8 @@ static int dwc3_omap_probe(struct platform_device *pdev)
|
|||
|
||||
irq = platform_get_irq(pdev, 0);
|
||||
if (irq < 0) {
|
||||
dev_err(dev, "missing IRQ resource\n");
|
||||
return -EINVAL;
|
||||
dev_err(dev, "missing IRQ resource: %d\n", irq);
|
||||
return irq;
|
||||
}
|
||||
|
||||
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
|
||||
|
|
|
|||
|
|
@ -220,6 +220,8 @@
|
|||
#include <linux/usb/gadget.h>
|
||||
#include <linux/usb/composite.h>
|
||||
|
||||
#include <linux/nospec.h>
|
||||
|
||||
#include "configfs.h"
|
||||
|
||||
|
||||
|
|
@ -3295,6 +3297,7 @@ static struct config_group *fsg_lun_make(struct config_group *group,
|
|||
fsg_opts = to_fsg_opts(&group->cg_item);
|
||||
if (num >= FSG_MAX_LUNS)
|
||||
return ERR_PTR(-ERANGE);
|
||||
num = array_index_nospec(num, FSG_MAX_LUNS);
|
||||
|
||||
mutex_lock(&fsg_opts->lock);
|
||||
if (fsg_opts->refcnt || fsg_opts->common->luns[num]) {
|
||||
|
|
|
|||
|
|
@ -130,8 +130,8 @@ static int ehci_hcd_omap_probe(struct platform_device *pdev)
|
|||
|
||||
irq = platform_get_irq(pdev, 0);
|
||||
if (irq < 0) {
|
||||
dev_err(dev, "EHCI irq failed\n");
|
||||
return -ENODEV;
|
||||
dev_err(dev, "EHCI irq failed: %d\n", irq);
|
||||
return irq;
|
||||
}
|
||||
|
||||
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
|
||||
|
|
|
|||
|
|
@ -1849,8 +1849,10 @@ static int imx21_probe(struct platform_device *pdev)
|
|||
if (!res)
|
||||
return -ENODEV;
|
||||
irq = platform_get_irq(pdev, 0);
|
||||
if (irq < 0)
|
||||
return -ENXIO;
|
||||
if (irq < 0) {
|
||||
dev_err(&pdev->dev, "Failed to get IRQ: %d\n", irq);
|
||||
return irq;
|
||||
}
|
||||
|
||||
hcd = usb_create_hcd(&imx21_hc_driver,
|
||||
&pdev->dev, dev_name(&pdev->dev));
|
||||
|
|
|
|||
|
|
@ -808,12 +808,24 @@ Retry_Sense:
|
|||
if (result == USB_STOR_TRANSPORT_GOOD) {
|
||||
srb->result = SAM_STAT_GOOD;
|
||||
srb->sense_buffer[0] = 0x0;
|
||||
}
|
||||
|
||||
/*
|
||||
* ATA-passthru commands use sense data to report
|
||||
* the command completion status, and often devices
|
||||
* return Check Condition status when nothing is
|
||||
* wrong.
|
||||
*/
|
||||
else if (srb->cmnd[0] == ATA_16 ||
|
||||
srb->cmnd[0] == ATA_12) {
|
||||
/* leave the data alone */
|
||||
}
|
||||
|
||||
/* If there was a problem, report an unspecified
|
||||
* hardware error to prevent the higher layers from
|
||||
* entering an infinite retry loop.
|
||||
*/
|
||||
} else {
|
||||
else {
|
||||
srb->result = DID_ERROR << 16;
|
||||
if ((sshdr.response_code & 0x72) == 0x72)
|
||||
srb->sense_buffer[1] = HARDWARE_ERROR;
|
||||
|
|
|
|||
|
|
@ -27,6 +27,7 @@
|
|||
#include <linux/cgroup.h>
|
||||
#include <linux/module.h>
|
||||
#include <linux/sort.h>
|
||||
#include <linux/nospec.h>
|
||||
|
||||
#include "vhost.h"
|
||||
|
||||
|
|
@ -748,6 +749,7 @@ long vhost_vring_ioctl(struct vhost_dev *d, int ioctl, void __user *argp)
|
|||
if (idx >= d->nvqs)
|
||||
return -ENOBUFS;
|
||||
|
||||
idx = array_index_nospec(idx, d->nvqs);
|
||||
vq = d->vqs[idx];
|
||||
|
||||
mutex_lock(&vq->mutex);
|
||||
|
|
|
|||
|
|
@ -712,7 +712,7 @@ static int pxa168fb_probe(struct platform_device *pdev)
|
|||
/*
|
||||
* enable controller clock
|
||||
*/
|
||||
clk_enable(fbi->clk);
|
||||
clk_prepare_enable(fbi->clk);
|
||||
|
||||
pxa168fb_set_par(info);
|
||||
|
||||
|
|
@ -767,7 +767,7 @@ static int pxa168fb_probe(struct platform_device *pdev)
|
|||
failed_free_cmap:
|
||||
fb_dealloc_cmap(&info->cmap);
|
||||
failed_free_clk:
|
||||
clk_disable(fbi->clk);
|
||||
clk_disable_unprepare(fbi->clk);
|
||||
failed_free_fbmem:
|
||||
dma_free_coherent(fbi->dev, info->fix.smem_len,
|
||||
info->screen_base, fbi->fb_start_dma);
|
||||
|
|
@ -807,7 +807,7 @@ static int pxa168fb_remove(struct platform_device *pdev)
|
|||
dma_free_writecombine(fbi->dev, PAGE_ALIGN(info->fix.smem_len),
|
||||
info->screen_base, info->fix.smem_start);
|
||||
|
||||
clk_disable(fbi->clk);
|
||||
clk_disable_unprepare(fbi->clk);
|
||||
|
||||
framebuffer_release(info);
|
||||
|
||||
|
|
|
|||
|
|
@ -626,8 +626,8 @@ static int pxa3xx_gcu_probe(struct platform_device *pdev)
|
|||
/* request the IRQ */
|
||||
irq = platform_get_irq(pdev, 0);
|
||||
if (irq < 0) {
|
||||
dev_err(dev, "no IRQ defined\n");
|
||||
return -ENODEV;
|
||||
dev_err(dev, "no IRQ defined: %d\n", irq);
|
||||
return irq;
|
||||
}
|
||||
|
||||
ret = devm_request_irq(dev, irq, pxa3xx_gcu_handle_irq,
|
||||
|
|
|
|||
|
|
@ -1608,7 +1608,7 @@ fail:
|
|||
return ret;
|
||||
}
|
||||
|
||||
static struct btrfs_root *btrfs_lookup_fs_root(struct btrfs_fs_info *fs_info,
|
||||
struct btrfs_root *btrfs_lookup_fs_root(struct btrfs_fs_info *fs_info,
|
||||
u64 root_id)
|
||||
{
|
||||
struct btrfs_root *root;
|
||||
|
|
|
|||
|
|
@ -68,6 +68,8 @@ struct extent_buffer *btrfs_find_tree_block(struct btrfs_fs_info *fs_info,
|
|||
struct btrfs_root *btrfs_read_fs_root(struct btrfs_root *tree_root,
|
||||
struct btrfs_key *location);
|
||||
int btrfs_init_fs_root(struct btrfs_root *root);
|
||||
struct btrfs_root *btrfs_lookup_fs_root(struct btrfs_fs_info *fs_info,
|
||||
u64 root_id);
|
||||
int btrfs_insert_fs_root(struct btrfs_fs_info *fs_info,
|
||||
struct btrfs_root *root);
|
||||
void btrfs_free_fs_roots(struct btrfs_fs_info *fs_info);
|
||||
|
|
|
|||
|
|
@ -272,6 +272,23 @@ int btrfs_find_orphan_roots(struct btrfs_root *tree_root)
|
|||
root_key.objectid = key.offset;
|
||||
key.offset++;
|
||||
|
||||
/*
|
||||
* The root might have been inserted already, as before we look
|
||||
* for orphan roots, log replay might have happened, which
|
||||
* triggers a transaction commit and qgroup accounting, which
|
||||
* in turn reads and inserts fs roots while doing backref
|
||||
* walking.
|
||||
*/
|
||||
root = btrfs_lookup_fs_root(tree_root->fs_info,
|
||||
root_key.objectid);
|
||||
if (root) {
|
||||
WARN_ON(!test_bit(BTRFS_ROOT_ORPHAN_ITEM_INSERTED,
|
||||
&root->state));
|
||||
if (btrfs_root_refs(&root->root_item) == 0)
|
||||
btrfs_add_dead_root(root);
|
||||
continue;
|
||||
}
|
||||
|
||||
root = btrfs_read_fs_root(tree_root, &root_key);
|
||||
err = PTR_ERR_OR_ZERO(root);
|
||||
if (err && err != -ENOENT) {
|
||||
|
|
@ -310,16 +327,8 @@ int btrfs_find_orphan_roots(struct btrfs_root *tree_root)
|
|||
set_bit(BTRFS_ROOT_ORPHAN_ITEM_INSERTED, &root->state);
|
||||
|
||||
err = btrfs_insert_fs_root(root->fs_info, root);
|
||||
/*
|
||||
* The root might have been inserted already, as before we look
|
||||
* for orphan roots, log replay might have happened, which
|
||||
* triggers a transaction commit and qgroup accounting, which
|
||||
* in turn reads and inserts fs roots while doing backref
|
||||
* walking.
|
||||
*/
|
||||
if (err == -EEXIST)
|
||||
err = 0;
|
||||
if (err) {
|
||||
BUG_ON(err == -EEXIST);
|
||||
btrfs_free_fs_root(root);
|
||||
break;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -317,7 +317,7 @@ try_again:
|
|||
trap = lock_rename(cache->graveyard, dir);
|
||||
|
||||
/* do some checks before getting the grave dentry */
|
||||
if (rep->d_parent != dir) {
|
||||
if (rep->d_parent != dir || IS_DEADDIR(d_inode(rep))) {
|
||||
/* the entry was probably culled when we dropped the parent dir
|
||||
* lock */
|
||||
unlock_rename(cache->graveyard, dir);
|
||||
|
|
|
|||
|
|
@ -3674,6 +3674,9 @@ try_mount_again:
|
|||
if (IS_ERR(tcon)) {
|
||||
rc = PTR_ERR(tcon);
|
||||
tcon = NULL;
|
||||
if (rc == -EACCES)
|
||||
goto mount_fail_check;
|
||||
|
||||
goto remote_path_check;
|
||||
}
|
||||
|
||||
|
|
|
|||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Reference in a new issue