evie/android_kernel_oneplus_msm8998
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge android-4.4.142 (8ec9fd8) into msm-4.4

Browse source 
* refs/heads/tmp-8ec9fd8
  ANDROID: sdcardfs: Check stacked filesystem depth
  Fix backport of "tcp: detect malicious patterns in tcp_collapse_ofo_queue()"
  tcp: detect malicious patterns in tcp_collapse_ofo_queue()
  tcp: avoid collapses in tcp_prune_queue() if possible
  x86_64_cuttlefish_defconfig: Enable android-verity
  x86_64_cuttlefish_defconfig: enable verity cert
  Linux 4.4.142
  perf tools: Move syscall number fallbacks from perf-sys.h to tools/arch/x86/include/asm/
  x86/cpu: Probe CPUID leaf 6 even when cpuid_level == 6
  Kbuild: fix # escaping in .cmd files for future Make
  ANDROID: Fix massive cpufreq_times memory leaks
  ANDROID: Reduce use of #ifdef CONFIG_CPU_FREQ_TIMES
  UPSTREAM: binder: replace "%p" with "%pK"
  UPSTREAM: binder: free memory on error
  UPSTREAM: binder: fix proc->files use-after-free
  UPSTREAM: Revert "FROMLIST: binder: fix proc->files use-after-free"
  UPSTREAM: ANDROID: binder: change down_write to down_read
  UPSTREAM: ANDROID: binder: correct the cmd print for BINDER_WORK_RETURN_ERROR
  UPSTREAM: ANDROID: binder: remove 32-bit binder interface.
  UPSTREAM: ANDROID: binder: re-order some conditions
  UPSTREAM: android: binder: use VM_ALLOC to get vm area
  UPSTREAM: android: binder: Use true and false for boolean values
  UPSTREAM: android: binder: Use octal permissions
  UPSTREAM: android: binder: Prefer __func__ to using hardcoded function name
  UPSTREAM: ANDROID: binder: make binder_alloc_new_buf_locked static and indent its arguments
  UPSTREAM: android: binder: Check for errors in binder_alloc_shrinker_init().
  treewide: Use array_size in f2fs_kvzalloc()
  treewide: Use array_size() in f2fs_kzalloc()
  treewide: Use array_size() in f2fs_kmalloc()
  overflow.h: Add allocation size calculation helpers
  f2fs: fix to clear FI_VOLATILE_FILE correctly
  f2fs: let sync node IO interrupt async one
  f2fs: don't change wbc->sync_mode
  f2fs: fix to update mtime correctly
  fs: f2fs: insert space around that ':' and ', '
  fs: f2fs: add missing blank lines after declarations
  fs: f2fs: changed variable type of offset "unsigned" to "loff_t"
  f2fs: clean up symbol namespace
  f2fs: make set_de_type() static
  f2fs: make __f2fs_write_data_pages() static
  f2fs: fix to avoid accessing cross the boundary
  f2fs: fix to let caller retry allocating block address
  disable loading f2fs module on PAGE_SIZE > 4KB
  f2fs: fix error path of move_data_page
  f2fs: don't drop dentry pages after fs shutdown
  f2fs: fix to avoid race during access gc_thread pointer
  f2fs: clean up with clear_radix_tree_dirty_tag
  f2fs: fix to don't trigger writeback during recovery
  f2fs: clear discard_wake earlier
  f2fs: let discard thread wait a little longer if dev is busy
  f2fs: avoid stucking GC due to atomic write
  f2fs: introduce sbi->gc_mode to determine the policy
  f2fs: keep migration IO order in LFS mode
  f2fs: fix to wait page writeback during revoking atomic write
  f2fs: Fix deadlock in shutdown ioctl
  f2fs: detect synchronous writeback more earlier
  mm: remove nr_pages argument from pagevec_lookup_{,range}_tag()
  ceph: use pagevec_lookup_range_nr_tag()
  mm: add variant of pagevec_lookup_range_tag() taking number of pages
  mm: use pagevec_lookup_range_tag() in write_cache_pages()
  mm: use pagevec_lookup_range_tag() in __filemap_fdatawait_range()
  nilfs2: use pagevec_lookup_range_tag()
  gfs2: use pagevec_lookup_range_tag()
  f2fs: use find_get_pages_tag() for looking up single page
  f2fs: simplify page iteration loops
  f2fs: use pagevec_lookup_range_tag()
  ext4: use pagevec_lookup_range_tag()
  ceph: use pagevec_lookup_range_tag()
  btrfs: use pagevec_lookup_range_tag()
  mm: implement find_get_pages_range_tag()
  f2fs: clean up with is_valid_blkaddr()
  f2fs: fix to initialize min_mtime with ULLONG_MAX
  f2fs: fix to let checkpoint guarantee atomic page persistence
  f2fs: fix to initialize i_current_depth according to inode type
  Revert "f2fs: add ovp valid_blocks check for bg gc victim to fg_gc"
  f2fs: don't drop any page on f2fs_cp_error() case
  f2fs: fix spelling mistake: "extenstion" -> "extension"
  f2fs: enhance sanity_check_raw_super() to avoid potential overflows
  f2fs: treat volatile file's data as hot one
  f2fs: introduce release_discard_addr() for cleanup
  f2fs: fix potential overflow
  f2fs: rename dio_rwsem to i_gc_rwsem
  f2fs: move mnt_want_write_file after range check
  f2fs: fix missing clear FI_NO_PREALLOC in some error case
  f2fs: enforce fsync_mode=strict for renamed directory
  f2fs: sanity check for total valid node blocks
  f2fs: sanity check on sit entry
  f2fs: avoid bug_on on corrupted inode
  f2fs: give message and set need_fsck given broken node id
  f2fs: clean up commit_inmem_pages()
  f2fs: do not check F2FS_INLINE_DOTS in recover
  f2fs: remove duplicated dquot_initialize and fix error handling
  f2fs: stop issue discard if something wrong with f2fs
  f2fs: fix return value in f2fs_ioc_commit_atomic_write
  f2fs: allocate hot_data for atomic write more strictly
  f2fs: check if inmem_pages list is empty correctly
  f2fs: fix race in between GC and atomic open
  f2fs: change le32 to le16 of f2fs_inode->i_extra_size
  f2fs: check cur_valid_map_mir & raw_sit block count when flush sit entries
  f2fs: correct return value of f2fs_trim_fs
  f2fs: fix to show missing bits in FS_IOC_GETFLAGS
  f2fs: remove unneeded F2FS_PROJINHERIT_FL
  f2fs: don't use GFP_ZERO for page caches
  f2fs: issue all big range discards in umount process
  f2fs: remove redundant block plug
  f2fs: remove unmatched zero_user_segment when convert inline dentry
  f2fs: introduce private inode status mapping
  fscrypt: log the crypto algorithm implementations
  crypto: api - Add crypto_type_has_alg helper
  crypto: skcipher - Add low-level skcipher interface
  crypto: skcipher - Add helper to retrieve driver name
  crypto: skcipher - Add default key size helper
  fscrypt: add Speck128/256 support
  fscrypt: only derive the needed portion of the key
  fscrypt: separate key lookup from key derivation
  fscrypt: use a common logging function
  fscrypt: remove internal key size constants
  fscrypt: remove unnecessary check for non-logon key type
  fscrypt: make fscrypt_operations.max_namelen an integer
  fscrypt: drop empty name check from fname_decrypt()
  fscrypt: drop max_namelen check from fname_decrypt()
  fscrypt: don't special-case EOPNOTSUPP from fscrypt_get_encryption_info()
  fscrypt: don't clear flags on crypto transform
  fscrypt: remove stale comment from fscrypt_d_revalidate()
  fscrypt: remove error messages for skcipher_request_alloc() failure
  fscrypt: remove unnecessary NULL check when allocating skcipher
  fscrypt: clean up after fscrypt_prepare_lookup() conversions
  fscrypt: use unbound workqueue for decryption
  f2fs: run fstrim asynchronously if runtime discard is on
  f2fs: turn down IO priority of discard from background
  f2fs: don't split checkpoint in fstrim
  f2fs: issue discard commands proactively in high fs utilization
  f2fs: add fsync_mode=nobarrier for non-atomic files
  f2fs: let fstrim issue discard commands in lower priority
  f2fs: avoid fsync() failure caused by EAGAIN in writepage()
  f2fs: clear PageError on writepage - part 2
  f2fs: check cap_resource only for data blocks
  Revert "f2fs: introduce f2fs_set_page_dirty_nobuffer"
  f2fs: clear PageError on writepage
  f2fs: call unlock_new_inode() before d_instantiate()
  f2fs: refactor read path to allow multiple postprocessing steps
  fscrypt: allow synchronous bio decryption
  f2fs: remain written times to update inode during fsync
  f2fs: make assignment of t->dentry_bitmap more readable
  f2fs: truncate preallocated blocks in error case
  f2fs: fix a wrong condition in f2fs_skip_inode_update
  f2fs: reserve bits for fs-verity
  f2fs: Add a segment type check in inplace write
  f2fs: no need to initialize zero value for GFP_F2FS_ZERO
  f2fs: don't track new nat entry in nat set
  f2fs: clean up with F2FS_BLK_ALIGN
  f2fs: check blkaddr more accuratly before issue a bio
  f2fs: Set GF_NOFS in read_cache_page_gfp while doing f2fs_quota_read
  f2fs: introduce a new mount option test_dummy_encryption
  f2fs: introduce F2FS_FEATURE_LOST_FOUND feature
  f2fs: release locks before return in f2fs_ioc_gc_range()
  f2fs: align memory boundary for bitops
  f2fs: remove unneeded set_cold_node()
  f2fs: add nowait aio support
  f2fs: wrap all options with f2fs_sb_info.mount_opt
  f2fs: Don't overwrite all types of node to keep node chain
  f2fs: introduce mount option for fsync mode
  f2fs: fix to restore old mount option in ->remount_fs
  f2fs: wrap sb_rdonly with f2fs_readonly
  f2fs: avoid selinux denial on CAP_SYS_RESOURCE
  f2fs: support hot file extension
  f2fs: fix to avoid race in between atomic write and background GC
  f2fs: do gc in greedy mode for whole range if gc_urgent mode is set
  f2fs: issue discard aggressively in the gc_urgent mode
  f2fs: set readdir_ra by default
  f2fs: add auto tuning for small devices
  f2fs: add mount option for segment allocation policy
  f2fs: don't stop GC if GC is contended
  f2fs: expose extension_list sysfs entry
  f2fs: fix to set KEEP_SIZE bit in f2fs_zero_range
  f2fs: introduce sb_lock to make encrypt pwsalt update exclusive
  f2fs: remove redundant initialization of pointer 'p'
  f2fs: flush cp pack except cp pack 2 page at first
  f2fs: clean up f2fs_sb_has_xxx functions
  f2fs: remove redundant check of page type when submit bio
  f2fs: fix to handle looped node chain during recovery
  f2fs: handle quota for orphan inodes
  f2fs: support passing down write hints to block layer with F2FS policy
  f2fs: support passing down write hints given by users to block layer
  f2fs: fix to clear CP_TRIMMED_FLAG
  f2fs: support large nat bitmap
  f2fs: fix to check extent cache in f2fs_drop_extent_tree
  f2fs: restrict inline_xattr_size configuration
  f2fs: fix heap mode to reset it back
  f2fs: fix potential corruption in area before F2FS_SUPER_OFFSET
  fscrypt: fix build with pre-4.6 gcc versions
  fscrypt: fix up fscrypt_fname_encrypted_size() for internal use
  fscrypt: define fscrypt_fname_alloc_buffer() to be for presented names
  fscrypt: calculate NUL-padding length in one place only
  fscrypt: move fscrypt_symlink_data to fscrypt_private.h
  fscrypt: remove fscrypt_fname_usr_to_disk()
  f2fs: switch to fscrypt_get_symlink()
  f2fs: switch to fscrypt ->symlink() helper functions
  fscrypt: new helper function - fscrypt_get_symlink()
  fscrypt: new helper functions for ->symlink()
  fscrypt: trim down fscrypt.h includes
  fscrypt: move fscrypt_is_dot_dotdot() to fs/crypto/fname.c
  fscrypt: move fscrypt_valid_enc_modes() to fscrypt_private.h
  fscrypt: move fscrypt_operations declaration to fscrypt_supp.h
  fscrypt: split fscrypt_dummy_context_enabled() into supp/notsupp versions
  fscrypt: move fscrypt_ctx declaration to fscrypt_supp.h
  fscrypt: move fscrypt_info_cachep declaration to fscrypt_private.h
  fscrypt: move fscrypt_control_page() to supp/notsupp headers
  fscrypt: move fscrypt_has_encryption_key() to supp/notsupp headers
  f2fs: don't put dentry page in pagecache into highmem
  f2fs: support inode creation time
  f2fs: rebuild sit page from sit info in mem
  f2fs: stop issuing discard if fs is readonly
  f2fs: clean up duplicated assignment in init_discard_policy
  f2fs: use GFP_F2FS_ZERO for cleanup
  f2fs: allow to recover node blocks given updated checkpoint
  f2fs: recover some i_inline flags
  f2fs: correct removexattr behavior for null valued extended attribute
  f2fs: drop page cache after fs shutdown
  f2fs: stop gc/discard thread after fs shutdown
  f2fs: hanlde error case in f2fs_ioc_shutdown
  f2fs: split need_inplace_update
  f2fs: fix to update last_disk_size correctly
  f2fs: kill F2FS_INLINE_XATTR_ADDRS for cleanup
  f2fs: clean up error path of fill_super
  f2fs: avoid hungtask when GC encrypted block if io_bits is set
  f2fs: allow quota to use reserved blocks
  f2fs: fix to drop all inmem pages correctly
  f2fs: speed up defragment on sparse file
  f2fs: support F2FS_IOC_PRECACHE_EXTENTS
  f2fs: add an ioctl to disable GC for specific file
  f2fs: prevent newly created inode from being dirtied incorrectly
  f2fs: support FIEMAP_FLAG_XATTR
  f2fs: fix to cover f2fs_inline_data_fiemap with inode_lock
  f2fs: check node page again in write end io
  f2fs: fix to caclulate required free section correctly
  f2fs: handle newly created page when revoking inmem pages
  f2fs: add resgid and resuid to reserve root blocks
  f2fs: implement cgroup writeback support
  f2fs: remove unused pend_list_tag
  f2fs: avoid high cpu usage in discard thread
  f2fs: make local functions static
  f2fs: add reserved blocks for root user
  f2fs: check segment type in __f2fs_replace_block
  f2fs: update inode info to inode page for new file
  f2fs: show precise # of blocks that user/root can use
  f2fs: clean up unneeded declaration
  f2fs: continue to do direct IO if we only preallocate partial blocks
  f2fs: enable quota at remount from r to w
  f2fs: skip stop_checkpoint for user data writes
  f2fs: fix missing error number for xattr operation
  f2fs: recover directory operations by fsync
  f2fs: return error during fill_super
  f2fs: fix an error case of missing update inode page
  f2fs: fix potential hangtask in f2fs_trace_pid
  f2fs: no need return value in restore summary process
  f2fs: use unlikely for release case
  f2fs: don't return value in truncate_data_blocks_range
  f2fs: clean up f2fs_map_blocks
  f2fs: clean up hash codes
  f2fs: fix error handling in fill_super
  f2fs: spread f2fs_k{m,z}alloc
  f2fs: inject fault to kvmalloc
  f2fs: inject fault to kzalloc
  f2fs: remove a redundant conditional expression
  f2fs: apply write hints to select the type of segment for direct write
  f2fs: switch to fscrypt_prepare_setattr()
  f2fs: switch to fscrypt_prepare_lookup()
  f2fs: switch to fscrypt_prepare_rename()
  f2fs: switch to fscrypt_prepare_link()
  f2fs: switch to fscrypt_file_open()
  f2fs: remove repeated f2fs_bug_on
  f2fs: remove an excess variable
  f2fs: fix lock dependency in between dio_rwsem & i_mmap_sem
  f2fs: remove unused parameter
  f2fs: still write data if preallocate only partial blocks
  f2fs: introduce sysfs readdir_ra to readahead inode block in readdir
  f2fs: fix concurrent problem for updating free bitmap
  f2fs: remove unneeded memory footprint accounting
  f2fs: no need to read nat block if nat_block_bitmap is set
  f2fs: reserve nid resource for quota sysfile
  fscrypt: resolve some cherry-pick bugs
  fscrypt: move to generic async completion
  crypto: introduce crypto wait for async op
  fscrypt: lock mutex before checking for bounce page pool
  fscrypt: new helper function - fscrypt_prepare_setattr()
  fscrypt: new helper function - fscrypt_prepare_lookup()
  fscrypt: new helper function - fscrypt_prepare_rename()
  fscrypt: new helper function - fscrypt_prepare_link()
  fscrypt: new helper function - fscrypt_file_open()
  fscrypt: new helper function - fscrypt_require_key()
  fscrypt: remove unneeded empty fscrypt_operations structs
  fscrypt: remove ->is_encrypted()
  fscrypt: switch from ->is_encrypted() to IS_ENCRYPTED()
  fs, fscrypt: add an S_ENCRYPTED inode flag
  fscrypt: clean up include file mess
  fscrypt: fix dereference of NULL user_key_payload
  fscrypt: make ->dummy_context() return bool
  f2fs: deny accessing encryption policy if encryption is off
  f2fs: inject fault in inc_valid_node_count
  f2fs: fix to clear FI_NO_PREALLOC
  f2fs: expose quota information in debugfs
  f2fs: separate nat entry mem alloc from nat_tree_lock
  f2fs: validate before set/clear free nat bitmap
  f2fs: avoid opened loop codes in __add_ino_entry
  f2fs: apply write hints to select the type of segments for buffered write
  f2fs: introduce scan_curseg_cache for cleanup
  f2fs: optimize the way of traversing free_nid_bitmap
  f2fs: keep scanning until enough free nids are acquired
  f2fs: trace checkpoint reason in fsync()
  f2fs: keep isize once block is reserved cross EOF
  f2fs: avoid race in between GC and block exchange
  f2fs: save a multiplication for last_nid calculation
  f2fs: fix summary info corruption
  f2fs: remove dead code in update_meta_page
  f2fs: remove unneeded semicolon
  f2fs: don't bother with inode->i_version
  f2fs: check curseg space before foreground GC
  f2fs: use rw_semaphore to protect SIT cache
  f2fs: support quota sys files
  f2fs: add quota_ino feature infra
  f2fs: optimize __update_nat_bits
  f2fs: modify for accurate fggc node io stat
  Revert "f2fs: handle dirty segments inside refresh_sit_entry"
  f2fs: add a function to move nid
  f2fs: export SSR allocation threshold
  f2fs: give correct trimmed blocks in fstrim
  f2fs: support bio allocation error injection
  f2fs: support get_page error injection
  f2fs: add missing sysfs description
  f2fs: support soft block reservation
  f2fs: handle error case when adding xattr entry
  f2fs: support flexible inline xattr size
  f2fs: show current cp state
  f2fs: add missing quota_initialize
  f2fs: show # of dirty segments via sysfs
  f2fs: stop all the operations by cp_error flag
  f2fs: remove several redundant assignments
  f2fs: avoid using timespec
  f2fs: fix to correct no_fggc_candidate
  Revert "f2fs: return wrong error number on f2fs_quota_write"
  f2fs: remove obsolete pointer for truncate_xattr_node
  f2fs: retry ENOMEM for quota_read|write
  f2fs: limit # of inmemory pages
  f2fs: update ctx->pos correctly when hitting hole in directory
  f2fs: relocate readahead codes in readdir()
  f2fs: allow readdir() to be interrupted
  f2fs: trace f2fs_readdir
  f2fs: trace f2fs_lookup
  f2fs: skip searching non-exist range in truncate_hole
  f2fs: expose some sectors to user in inline data or dentry case
  f2fs: avoid stale fi->gdirty_list pointer
  f2fs/crypto: drop crypto key at evict_inode only
  f2fs: fix to avoid race when accessing last_disk_size
  f2fs: Fix bool initialization/comparison
  f2fs: give up CP_TRIMMED_FLAG if it drops discards
  f2fs: trace f2fs_remove_discard
  f2fs: reduce cmd_lock coverage in __issue_discard_cmd
  f2fs: split discard policy
  f2fs: wrap discard policy
  f2fs: support issuing/waiting discard in range
  f2fs: fix to flush multiple device in checkpoint
  f2fs: enhance multiple device flush
  f2fs: fix to show ino management cache size correctly
  f2fs: drop FI_UPDATE_WRITE tag after f2fs_issue_flush
  f2fs: obsolete ALLOC_NID_LIST list
  f2fs: convert inline data for direct I/O & FI_NO_PREALLOC
  f2fs: allow readpages with NULL file pointer
  f2fs: show flush list status in sysfs
  f2fs: introduce read_xattr_block
  f2fs: introduce read_inline_xattr
  Revert "f2fs: reuse nids more aggressively"
  Revert "f2fs: node segment is prior to data segment selected victim"
  f2fs: fix potential panic during fstrim
  f2fs: hurry up to issue discard after io interruption
  f2fs: fix to show correct discard_granularity in sysfs
  f2fs: detect dirty inode in evict_inode
  f2fs: clear radix tree dirty tag of pages whose dirty flag is cleared
  f2fs: speed up gc_urgent mode with SSR
  f2fs: better to wait for fstrim completion
  f2fs: avoid race in between read xattr & write xattr
  f2fs: make get_lock_data_page to handle encrypted inode
  f2fs: use generic terms used for encrypted block management
  f2fs: introduce f2fs_encrypted_file for clean-up
  Revert "f2fs: add a new function get_ssr_cost"
  f2fs: constify super_operations
  f2fs: fix to wake up all sleeping flusher
  f2fs: avoid race in between atomic_read & atomic_inc
  f2fs: remove unneeded parameter of change_curseg
  f2fs: update i_flags correctly
  f2fs: don't check inode's checksum if it was dirtied or writebacked
  f2fs: don't need to update inode checksum for recovery
  f2fs: trigger fdatasync for non-atomic_write file
  f2fs: fix to avoid race in between aio and gc
  f2fs: wake up discard_thread iff there is a candidate
  f2fs: return error when accessing insane flie offset
  f2fs: trigger normal fsync for non-atomic_write file
  f2fs: clear FI_HOT_DATA correctly
  f2fs: fix out-of-order execution in f2fs_issue_flush
  f2fs: issue discard commands if gc_urgent is set
  f2fs: introduce discard_granularity sysfs entry
  f2fs: remove unused function overprovision_sections
  f2fs: check hot_data for roll-forward recovery
  f2fs: add tracepoint for f2fs_gc
  f2fs: retry to revoke atomic commit in -ENOMEM case
  f2fs: let fill_super handle roll-forward errors
  f2fs: merge equivalent flags F2FS_GET_BLOCK_[READ|DIO]
  f2fs: support journalled quota
  f2fs: fix potential overflow when adjusting GC cycle
  f2fs: avoid unneeded sync on quota file
  f2fs: introduce gc_urgent mode for background GC
  f2fs: use IPU for cold files
  f2fs: fix the size value in __check_sit_bitmap
  f2fs: add app/fs io stat
  f2fs: do not change the valid_block value if cur_valid_map was wrongly set or cleared
  f2fs: update cur_valid_map_mir together with cur_valid_map
  f2fs: use printk_ratelimited for f2fs_msg
  f2fs: expose features to sysfs entry
  f2fs: support inode checksum
  f2fs: return wrong error number on f2fs_quota_write
  f2fs: provide f2fs_balance_fs to __write_node_page
  f2fs: introduce f2fs_statfs_project
  f2fs: don't need to wait for node writes for atomic write
  f2fs: avoid naming confusion of sysfs init
  f2fs: support project quota
  f2fs: record quota during dot{,dot} recovery
  f2fs: enhance on-disk inode structure scalability
  f2fs: make max inline size changeable
  f2fs: add ioctl to expose current features
  f2fs: make background threads of f2fs being aware of freezing
  f2fs: don't give partially written atomic data from process crash
  f2fs: give a try to do atomic write in -ENOMEM case
  f2fs: preserve i_mode if __f2fs_set_acl() fails
  f2fs: alloc new nids for xattr block in recovery
  f2fs: spread struct f2fs_dentry_ptr for inline path
  f2fs: remove unused input parameter
  f2fs: avoid cpu lockup
  f2fs: include seq_file.h for sysfs.c
  f2fs: Don't clear SGID when inheriting ACLs
  f2fs: remove extra inode_unlock() in error path
  fscrypt: add support for AES-128-CBC
  fscrypt: inline fscrypt_free_filename()
  f2fs: make more close to v4.13-rc1
  f2fs: support plain user/group quota
  f2fs: avoid deadlock caused by lock order of page and lock_op
  f2fs: use spin_{,un}lock_irq{save,restore}
  f2fs: relax migratepage for atomic written page
  f2fs: don't count inode block in in-memory inode.i_blocks
  Revert "f2fs: fix to clean previous mount option when remount_fs"
  f2fs: do not set LOST_PINO for renamed dir
  f2fs: do not set LOST_PINO for newly created dir
  f2fs: skip ->writepages for {mete,node}_inode during recovery
  f2fs: introduce __check_sit_bitmap
  f2fs: stop gc/discard thread in prior during umount
  f2fs: introduce reserved_blocks in sysfs
  f2fs: avoid redundant f2fs_flush after remount
  f2fs: report # of free inodes more precisely
  f2fs: add ioctl to do gc with target block address
  f2fs: don't need to check encrypted inode for partial truncation
  f2fs: measure inode.i_blocks as generic filesystem
  f2fs: set CP_TRIMMED_FLAG correctly
  f2fs: require key for truncate(2) of encrypted file
  f2fs: move sysfs code from super.c to fs/f2fs/sysfs.c
  f2fs: clean up sysfs codes
  f2fs: fix wrong error number of fill_super
  f2fs: fix to show injection rate in ->show_options
  f2fs: Fix a return value in case of error in 'f2fs_fill_super'
  f2fs: use proper variable name
  f2fs: fix to avoid panic when encountering corrupt node
  f2fs: don't track newly allocated nat entry in list
  f2fs: add f2fs_bug_on in __remove_discard_cmd
  f2fs: introduce __wait_one_discard_bio
  f2fs: dax: fix races between page faults and truncating pages
  f2fs: simplify the way of calulating next nat address
  f2fs: sanity check size of nat and sit cache
  f2fs: fix a panic caused by NULL flush_cmd_control
  f2fs: remove the unnecessary cast for PTR_ERR
  f2fs: remove false-positive bug_on
  f2fs: Do not issue small discards in LFS mode
  f2fs: don't bother checking for encryption key in ->write_iter()
  f2fs: don't bother checking for encryption key in ->mmap()
  f2fs: wait discard IO completion without cmd_lock held
  f2fs: wake up all waiters in f2fs_submit_discard_endio
  f2fs: show more info if fail to issue discard
  f2fs: introduce io_list for serialize data/node IOs
  f2fs: split wio_mutex
  f2fs: combine huge num of discard rb tree consistence checks
  f2fs: fix a bug caused by NULL extent tree
  f2fs: try to freeze in gc and discard threads
  f2fs: add a new function get_ssr_cost
  f2fs: declare load_free_nid_bitmap static
  f2fs: avoid f2fs_lock_op for IPU writes
  f2fs: split bio cache
  f2fs: use fio instead of multiple parameters
  f2fs: remove unnecessary read cases in merged IO flow
  f2fs: use f2fs_submit_page_bio for ra_meta_pages
  f2fs: make sure f2fs_gc returns consistent errno
  f2fs: load inode's flag from disk
  f2fs: sanity check checkpoint segno and blkoff
  f2fs, block_dump: give WRITE direction to submit_bio
  fscrypt: correct collision claim for digested names
  f2fs: switch to using fscrypt_match_name()
  fscrypt: introduce helper function for filename matching
  fscrypt: fix context consistency check when key(s) unavailable
  fscrypt: Move key structure and constants to uapi
  fscrypt: remove fscrypt_symlink_data_len()
  fscrypt: remove unnecessary checks for NULL operations
  fscrypt: eliminate ->prepare_context() operation
  fscrypt: remove broken support for detecting keyring key revocation
  fscrypt: avoid collisions when presenting long encrypted filenames
  f2fs: check entire encrypted bigname when finding a dentry
  f2fs: sync f2fs_lookup() with ext4_lookup()
  f2fs: fix a mount fail for wrong next_scan_nid
  f2fs: relocate inode_{,un}lock in F2FS_IOC_SETFLAGS
  f2fs: show available_nids in f2fs/status
  f2fs: flush dirty nats periodically
  f2fs: introduce CP_TRIMMED_FLAG to avoid unneeded discard
  f2fs: allow cpc->reason to indicate more than one reason
  f2fs: release cp and dnode lock before IPU
  f2fs: shrink size of struct discard_cmd
  f2fs: don't hold cmd_lock during waiting discard command
  f2fs: nullify fio->encrypted_page for each writes
  f2fs: sanity check segment count
  f2fs: introduce valid_ipu_blkaddr to clean up
  f2fs: lookup extent cache first under IPU scenario
  f2fs: reconstruct code to write a data page
  f2fs: introduce __wait_discard_cmd
  f2fs: introduce __issue_discard_cmd
  f2fs: enable small discard by default
  f2fs: delay awaking discard thread
  f2fs: seperate read nat page from nat_tree_lock
  f2fs: fix multiple f2fs_add_link() having same name for inline dentry
  f2fs: skip encrypted inode in ASYNC IPU policy
  f2fs: fix out-of free segments
  f2fs: improve definition of statistic macros
  f2fs: assign allocation hint for warm/cold data
  f2fs: fix _IOW usage
  f2fs: add ioctl to flush data from faster device to cold area
  f2fs: introduce async IPU policy
  f2fs: add undiscard blocks stat
  f2fs: unlock cp_rwsem early for IPU writes
  f2fs: introduce __check_rb_tree_consistence
  f2fs: trace __submit_discard_cmd
  f2fs: in prior to issue big discard
  f2fs: clean up discard_cmd_control structure
  f2fs: use rb-tree to track pending discard commands
  f2fs: avoid dirty node pages in check_only recovery
  f2fs: fix not to set fsync/dentry mark
  f2fs: allocate hot_data for atomic writes
  f2fs: give time to flush dirty pages for checkpoint
  f2fs: fix fs corruption due to zero inode page
  f2fs: shrink blk plug region
  f2fs: extract rb-tree operation infrastructure
  f2fs: avoid frequent checkpoint during f2fs_gc
  f2fs: clean up some macros in terms of GET_SEGNO
  f2fs: clean up get_valid_blocks with consistent parameter
  f2fs: use segment number for get_valid_blocks
  f2fs: guard macro variables with braces
  f2fs: fix comment on f2fs_flush_merged_bios() after 86531d6b
  f2fs: prevent waiter encountering incorrect discard states
  f2fs: introduce f2fs_wait_discard_bios
  f2fs: split discard_cmd_list
  Revert "f2fs: put allocate_segment after refresh_sit_entry"
  f2fs: split make_dentry_ptr() into block and inline versions
  f2fs: submit bio of in-place-update pages
  f2fs: remove the redundant variable definition
  f2fs: avoid IO split due to mixed WB_SYNC_ALL and WB_SYNC_NONE
  f2fs: write small sized IO to hot log
  f2fs: use bitmap in discard_entry
  f2fs: clean up destroy_discard_cmd_control
  f2fs: count discard command entry
  f2fs: show issued flush/discard count
  f2fs: relax node version check for victim data in gc
  f2fs: start SSR much eariler to avoid FG_GC
  f2fs: allocate node and hot data in the beginning of partition
  f2fs: fix wrong max cost initialization
  f2fs: allow write page cache when writting cp
  f2fs: don't reserve additional space in xattr block
  f2fs: clean up xattr operation
  f2fs: don't track volatile file in dirty inode list
  f2fs: show the max number of volatile operations
  f2fs: fix race condition in between free nid allocator/initializer
  f2fs: use set_page_private marcro in f2fs_trace_pid
  f2fs: fix recording invalid last_victim
  f2fs: more reasonable mem_size calculating of ino_entry
  f2fs: calculate the f2fs_stat_info into base_mem
  f2fs: avoid stat_inc_atomic_write for non-atomic file
  f2fs: sanity check of crc_offset from raw checkpoint
  f2fs: cleanup the disk level filename updating
  f2fs: cover update_free_nid_bitmap with nid_list_lock
  f2fs: fix bad prefetchw of NULL page
  f2fs: clear FI_DATA_EXIST flag in truncate_inline_inode
  f2fs: move mnt_want_write_file after arguments checking
  f2fs: check new size by inode_newsize_ok in f2fs_insert_range
  f2fs: avoid copy date to user-space if move file range fail
  f2fs: drop duplicate new_size assign in f2fs_zero_range
  f2fs: adjust the way of calculating nat block
  f2fs: add fault injection on f2fs_truncate
  f2fs: check range before defragment
  f2fs: use parameter max_items instead of PIDVEC_SIZE
  f2fs: add a punch discard command function
  f2fs: allocate a bio for discarding when actually issuing it
  f2fs: skip writeback meta pages if cp_mutex acquire failed
  f2fs: show more precise message on orphan recovery failure
  f2fs: remove dead macro PGOFS_OF_NEXT_DNODE
  f2fs: drop duplicate radix tree lookup of nat_entry_set
  f2fs: make sure trace all f2fs_issue_flush
  f2fs: don't allow volatile writes for non-regular file
  f2fs: don't allow atomic writes for not regular files
  f2fs: fix stale ATOMIC_WRITTEN_PAGE private pointer
  f2fs: build stat_info before orphan inode recovery
  f2fs: fix the fault of calculating blkstart twice
  f2fs: fix the fault of checking F2FS_LINK_MAX for rename inode
  f2fs: don't allow to get pino when filename is encrypted
  f2fs: fix wrong error injection for evict_inode
  f2fs: le32_to_cpu for ckpt->cp_pack_total_block_count
  f2fs: le16_to_cpu for xattr->e_value_size
  f2fs: don't need to invalidate wrong node page
  f2fs: fix an error return value in truncate_partial_data_page
  f2fs: combine nat_bits and free_nid_bitmap cache
  f2fs: skip scanning free nid bitmap of full NAT blocks
  f2fs: use __set{__clear}_bit_le
  f2fs: update_free_nid_bitmap() can be static
  f2fs: __update_nat_bits() can be static
  f2fs: le16_to_cpu for xattr->e_value_size
  f2fs: don't overwrite node block by SSR
  f2fs: don't need to invalidate wrong node page
  f2fs: fix an error return value in truncate_partial_data_page
  fscrypt: catch up to v4.11-rc1
  f2fs: avoid to flush nat journal entries
  f2fs: avoid to issue redundant discard commands
  f2fs: fix a plint compile warning
  f2fs: add f2fs_drop_inode tracepoint
  f2fs: Fix zoned block device support
  f2fs: remove redundant set_page_dirty()
  f2fs: fix to enlarge size of write_io_dummy mempool
  f2fs: fix memory leak of write_io_dummy mempool during umount
  f2fs: fix to update F2FS_{CP_}WB_DATA count correctly
  f2fs: use MAX_FREE_NIDS for the free nids target
  f2fs: introduce free nid bitmap
  f2fs: new helper cur_cp_crc() getting crc in f2fs_checkpoint
  f2fs: update the comment of default nr_pages to skipping
  f2fs: drop the duplicate pval in f2fs_getxattr
  f2fs: Don't update the xattr data that same as the exist
  f2fs: kill __is_extent_same
  f2fs: avoid bggc->fggc when enough free segments are avaliable after cp
  f2fs: select target segment with closer temperature in SSR mode
  f2fs: show simple call stack in fault injection message
  fscrypt: catch fscrypto_get_policy in v4.10-rc6
  f2fs: use __clear_bit_le
  f2fs: no need lock_op in f2fs_write_inline_data
  f2fs: add bitmaps for empty or full NAT blocks
  f2fs: replace rw semaphore extent_tree_lock with mutex lock
  f2fs: avoid m_flags overlay when allocating more data blocks
  f2fs: remove unsafe bitmap checking
  f2fs: init local extent_info to avoid stale stack info in tp
  f2fs: remove unnecessary condition check for write_checkpoint in f2fs_gc
  f2fs: do SSR for node segments more aggresively
  f2fs: check discard alignment only for SEQWRITE zones
  f2fs: wait for discard completion after submission
  f2fs: much larger batched trim_fs job
  f2fs: avoid very large discard command
  f2fs: find data segments across all the types
  f2fs: do SSR in higher priority
  f2fs: do SSR for data when there is enough free space
  f2fs: node segment is prior to data segment selected victim
  f2fs: put allocate_segment after refresh_sit_entry
  f2fs: add ovp valid_blocks check for bg gc victim to fg_gc
  f2fs: do not wait for writeback in write_begin
  f2fs: replace __get_victim by dirty_segments in FG_GC
  f2fs: fix multiple f2fs_add_link() calls having same name
  f2fs: show actual device info in tracepoints
  f2fs: use SSR for warm node as well
  f2fs: enable inline_xattr by default
  f2fs: introduce noinline_xattr mount option
  f2fs: avoid reading NAT page by get_node_info
  f2fs: remove build_free_nids() during checkpoint
  f2fs: change recovery policy of xattr node block
  f2fs: super: constify fscrypt_operations structure
  f2fs: show checkpoint version at mount time
  f2fs: remove preflush for nobarrier case
  f2fs: check last page index in cached bio to decide submission
  f2fs: check io submission more precisely
  f2fs: fix trim_fs assignment
  Revert "f2fs: remove batched discard in f2fs_trim_fs"
  f2fs: fix missing bio_alloc(1)
  f2fs: call internal __write_data_page directly
  f2fs: avoid out-of-order execution of atomic writes
  f2fs: move write_node_page above fsync_node_pages
  f2fs: move flush tracepoint
  f2fs: show # of APPEND and UPDATE inodes
  f2fs: fix 446 coding style warnings in f2fs.h
  f2fs: fix 3 coding style errors in f2fs.h
  f2fs: declare missing static function
  f2fs: show the fault injection mount option
  f2fs: fix null pointer dereference when issuing flush in ->fsync
  f2fs: fix to avoid overflow when left shifting page offset
  f2fs: enhance lookup xattr
  f2fs: fix a dead loop in f2fs_fiemap()
  f2fs: do not preallocate blocks which has wrong buffer
  f2fs: show # of on-going flush and discard bios
  f2fs: add a kernel thread to issue discard commands asynchronously
  f2fs: factor out discard command info into discard_cmd_control
  f2fs: remove batched discard in f2fs_trim_fs
  f2fs: reorganize stat information
  f2fs: clean up flush/discard command namings
  f2fs: check in-memory sit version bitmap
  f2fs: check in-memory nat version bitmap
  f2fs: check in-memory block bitmap
  f2fs: introduce FI_ATOMIC_COMMIT
  f2fs: clean up with list_{first, last}_entry
  f2fs: return fs_trim if there is no candidate
  f2fs: avoid needless checkpoint in f2fs_trim_fs
  f2fs: relax async discard commands more
  f2fs: drop exist_data for inline_data when truncated to 0
  f2fs: don't allow encrypted operations without keys
  f2fs: show the max number of atomic operations
  f2fs: get io size bit from mount option
  f2fs: support IO alignment for DATA and NODE writes
  f2fs: add submit_bio tracepoint
  f2fs: reassign new segment for mode=lfs
  f2fs: fix a missing discard prefree segments
  f2fs: use rb_entry_safe
  f2fs: add a case of no need to read a page in write begin
  f2fs: fix a problem of using memory after free
  f2fs: remove unneeded condition
  f2fs: don't cache nat entry if out of memory
  f2fs: remove unused values in recover_fsync_data
  f2fs: support async discard based on v4.9
  f2fs: resolve op and op_flags confilcts
  f2fs: remove wrong backported codes
  f2fs: fix a missing size change in f2fs_setattr
  fs/super.c: fix race between freeze_super() and thaw_super()
  scripts/tags.sh: catch 4.9-rc6
  f2fs: fix to access nullified flush_cmd_control pointer
  f2fs: free meta pages if sanity check for ckpt is failed
  f2fs: detect wrong layout
  f2fs: call sync_fs when f2fs is idle
  Revert "f2fs: use percpu_counter for # of dirty pages in inode"
  f2fs: return AOP_WRITEPAGE_ACTIVATE for writepage
  f2fs: do not activate auto_recovery for fallocated i_size
  f2fs: fix 32-bit build
  f2fs: set ->owner for debugfs status file's file_operations
  f2fs: fix incorrect free inode count in ->statfs
  f2fs: drop duplicate header timer.h
  f2fs: fix wrong AUTO_RECOVER condition
  f2fs: do not recover i_size if it's valid
  f2fs: fix fdatasync
  f2fs: fix to account total free nid correctly
  f2fs: fix an infinite loop when flush nodes in cp
  f2fs: don't wait writeback for datas during checkpoint
  f2fs: fix wrong written_valid_blocks counting
  f2fs: avoid BG_GC in f2fs_balance_fs
  f2fs: fix redundant block allocation
  f2fs: use err for f2fs_preallocate_blocks
  f2fs: support multiple devices
  f2fs: allow dio read for LFS mode
  f2fs: revert segment allocation for direct IO
  f2fs: return directly if block has been removed from the victim
  Revert "f2fs: do not recover from previous remained wrong dnodes"
  f2fs: remove checkpoint in f2fs_freeze
  f2fs: assign segments correctly for direct_io
  f2fs: fix wrong i_atime recovery
  f2fs: record inode updating status correctly
  f2fs: Trace reset zone events
  f2fs: Reset sequential zones on zoned block devices
  f2fs: Cache zoned block devices zone type
  f2fs: Do not allow adaptive mode for host-managed zoned block devices
  f2fs: Always enable discard for zoned blocks devices
  f2fs: Suppress discard warning message for zoned block devices
  f2fs: Check zoned block feature for host-managed zoned block devices
  f2fs: Use generic zoned block device terminology
  f2fs: Add missing break in switch-case
  f2fs: avoid infinite loop in the EIO case on recover_orphan_inodes
  f2fs: report error of f2fs_fill_dentries
  fs/crypto: catch up 4.9-rc6
  f2fs: hide a maybe-uninitialized warning
  f2fs: remove percpu_count due to performance regression
  f2fs: make clean inodes when flushing inode page
  f2fs: keep dirty inodes selectively for checkpoint
  f2fs: Replace CURRENT_TIME_SEC with current_time() for inode timestamps
  f2fs: use BIO_MAX_PAGES for bio allocation
  f2fs: declare static function for __build_free_nids
  f2fs: call f2fs_balance_fs for setattr
  f2fs: count dirty inodes to flush node pages during checkpoint
  f2fs: avoid casted negative value as shrink count
  f2fs: don't interrupt free nids building during nid allocation
  f2fs: clean up free nid list operations
  f2fs: split free nid list
  f2fs: clear nlink if fail to add_link
  f2fs: fix sparse warnings
  f2fs: fix error handling in fsync_node_pages
  f2fs: fix to update largest extent under lock
  f2fs: be aware of extent beyond EOF in fiemap
  f2fs: don't miss any f2fs_balance_fs cases
  f2fs: add missing f2fs_balance_fs in f2fs_zero_range
  f2fs: give a chance to detach from dirty list
  f2fs: fix to release discard entries during checkpoint
  f2fs: exclude free nids building and allocation
  f2fs: fix to determine start_cp_addr by sbi->cur_cp_pack
  f2fs: fix overflow due to condition check order
  posix_acl: Clear SGID bit when setting file permissions
  f2fs: fix wrong sum_page pointer in f2fs_gc
  f2fs: backport from (4c1fad64 - Merge tag 'for-f2fs-4.9' of git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs)

Change-Id: I6c7208efc63ce7b13f26f0ec1cd3c8aef410eff0
Signed-off-by: Blagovest Kolenichev <bkolenichev@codeaurora.org>
Signed-off-by: Srinivasarao P <spathi@codeaurora.org>
This commit is contained in:
Srinivasarao P 2018-08-02 10:10:30 +05:30
commit c2e09fadec
75 changed files with 3961 additions and 1755 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

626
Documentation/filesystems/fscrypt.rst Normal file
View file

@ -0,0 +1,626 @@
=====================================
Filesystem-level encryption (fscrypt)
=====================================
Introduction
============
fscrypt is a library which filesystems can hook into to support
transparent encryption of files and directories.
Note: "fscrypt" in this document refers to the kernel-level portion,
implemented in ``fs/crypto/``, as opposed to the userspace tool
`fscrypt <https://github.com/google/fscrypt>`_. This document only
covers the kernel-level portion. For command-line examples of how to
use encryption, see the documentation for the userspace tool `fscrypt
<https://github.com/google/fscrypt>`_. Also, it is recommended to use
the fscrypt userspace tool, or other existing userspace tools such as
`fscryptctl <https://github.com/google/fscryptctl>`_ or `Android's key
management system
<https://source.android.com/security/encryption/file-based>`_, over
using the kernel's API directly. Using existing tools reduces the
chance of introducing your own security bugs. (Nevertheless, for
completeness this documentation covers the kernel's API anyway.)
Unlike dm-crypt, fscrypt operates at the filesystem level rather than
at the block device level. This allows it to encrypt different files
with different keys and to have unencrypted files on the same
filesystem. This is useful for multi-user systems where each user's
data-at-rest needs to be cryptographically isolated from the others.
However, except for filenames, fscrypt does not encrypt filesystem
metadata.
Unlike eCryptfs, which is a stacked filesystem, fscrypt is integrated
directly into supported filesystems --- currently ext4, F2FS, and
UBIFS. This allows encrypted files to be read and written without
caching both the decrypted and encrypted pages in the pagecache,
thereby nearly halving the memory used and bringing it in line with
unencrypted files. Similarly, half as many dentries and inodes are
needed. eCryptfs also limits encrypted filenames to 143 bytes,
causing application compatibility issues; fscrypt allows the full 255
bytes (NAME_MAX). Finally, unlike eCryptfs, the fscrypt API can be
used by unprivileged users, with no need to mount anything.
fscrypt does not support encrypting files in-place. Instead, it
supports marking an empty directory as encrypted. Then, after
userspace provides the key, all regular files, directories, and
symbolic links created in that directory tree are transparently
encrypted.
Threat model
============
Offline attacks
---------------
Provided that userspace chooses a strong encryption key, fscrypt
protects the confidentiality of file contents and filenames in the
event of a single point-in-time permanent offline compromise of the
block device content. fscrypt does not protect the confidentiality of
non-filename metadata, e.g. file sizes, file permissions, file
timestamps, and extended attributes. Also, the existence and location
of holes (unallocated blocks which logically contain all zeroes) in
files is not protected.
fscrypt is not guaranteed to protect confidentiality or authenticity
if an attacker is able to manipulate the filesystem offline prior to
an authorized user later accessing the filesystem.
Online attacks
--------------
fscrypt (and storage encryption in general) can only provide limited
protection, if any at all, against online attacks. In detail:
fscrypt is only resistant to side-channel attacks, such as timing or
electromagnetic attacks, to the extent that the underlying Linux
Cryptographic API algorithms are. If a vulnerable algorithm is used,
such as a table-based implementation of AES, it may be possible for an
attacker to mount a side channel attack against the online system.
Side channel attacks may also be mounted against applications
consuming decrypted data.
After an encryption key has been provided, fscrypt is not designed to
hide the plaintext file contents or filenames from other users on the
same system, regardless of the visibility of the keyring key.
Instead, existing access control mechanisms such as file mode bits,
POSIX ACLs, LSMs, or mount namespaces should be used for this purpose.
Also note that as long as the encryption keys are *anywhere* in
memory, an online attacker can necessarily compromise them by mounting
a physical attack or by exploiting any kernel security vulnerability
which provides an arbitrary memory read primitive.
While it is ostensibly possible to "evict" keys from the system,
recently accessed encrypted files will remain accessible at least
until the filesystem is unmounted or the VFS caches are dropped, e.g.
using ``echo 2 > /proc/sys/vm/drop_caches``. Even after that, if the
RAM is compromised before being powered off, it will likely still be
possible to recover portions of the plaintext file contents, if not
some of the encryption keys as well. (Since Linux v4.12, all
in-kernel keys related to fscrypt are sanitized before being freed.
However, userspace would need to do its part as well.)
Currently, fscrypt does not prevent a user from maliciously providing
an incorrect key for another user's existing encrypted files. A
protection against this is planned.
Key hierarchy
=============
Master Keys
-----------
Each encrypted directory tree is protected by a *master key*. Master
keys can be up to 64 bytes long, and must be at least as long as the
greater of the key length needed by the contents and filenames
encryption modes being used. For example, if AES-256-XTS is used for
contents encryption, the master key must be 64 bytes (512 bits). Note
that the XTS mode is defined to require a key twice as long as that
required by the underlying block cipher.
To "unlock" an encrypted directory tree, userspace must provide the
appropriate master key. There can be any number of master keys, each
of which protects any number of directory trees on any number of
filesystems.
Userspace should generate master keys either using a cryptographically
secure random number generator, or by using a KDF (Key Derivation
Function). Note that whenever a KDF is used to "stretch" a
lower-entropy secret such as a passphrase, it is critical that a KDF
designed for this purpose be used, such as scrypt, PBKDF2, or Argon2.
Per-file keys
-------------
Master keys are not used to encrypt file contents or names directly.
Instead, a unique key is derived for each encrypted file, including
each regular file, directory, and symbolic link. This has several
advantages:
- In cryptosystems, the same key material should never be used for
different purposes. Using the master key as both an XTS key for
contents encryption and as a CTS-CBC key for filenames encryption
would violate this rule.
- Per-file keys simplify the choice of IVs (Initialization Vectors)
for contents encryption. Without per-file keys, to ensure IV
uniqueness both the inode and logical block number would need to be
encoded in the IVs. This would make it impossible to renumber
inodes, which e.g. ``resize2fs`` can do when resizing an ext4
filesystem. With per-file keys, it is sufficient to encode just the
logical block number in the IVs.
- Per-file keys strengthen the encryption of filenames, where IVs are
reused out of necessity. With a unique key per directory, IV reuse
is limited to within a single directory.
- Per-file keys allow individual files to be securely erased simply by
securely erasing their keys. (Not yet implemented.)
A KDF (Key Derivation Function) is used to derive per-file keys from
the master key. This is done instead of wrapping a randomly-generated
key for each file because it reduces the size of the encryption xattr,
which for some filesystems makes the xattr more likely to fit in-line
in the filesystem's inode table. With a KDF, only a 16-byte nonce is
required --- long enough to make key reuse extremely unlikely. A
wrapped key, on the other hand, would need to be up to 64 bytes ---
the length of an AES-256-XTS key. Furthermore, currently there is no
requirement to support unlocking a file with multiple alternative
master keys or to support rotating master keys. Instead, the master
keys may be wrapped in userspace, e.g. as done by the `fscrypt
<https://github.com/google/fscrypt>`_ tool.
The current KDF encrypts the master key using the 16-byte nonce as an
AES-128-ECB key. The output is used as the derived key. If the
output is longer than needed, then it is truncated to the needed
length. Truncation is the norm for directories and symlinks, since
those use the CTS-CBC encryption mode which requires a key half as
long as that required by the XTS encryption mode.
Note: this KDF meets the primary security requirement, which is to
produce unique derived keys that preserve the entropy of the master
key, assuming that the master key is already a good pseudorandom key.
However, it is nonstandard and has some problems such as being
reversible, so it is generally considered to be a mistake! It may be
replaced with HKDF or another more standard KDF in the future.
Encryption modes and usage
==========================
fscrypt allows one encryption mode to be specified for file contents
and one encryption mode to be specified for filenames. Different
directory trees are permitted to use different encryption modes.
Currently, the following pairs of encryption modes are supported:
- AES-256-XTS for contents and AES-256-CTS-CBC for filenames
- AES-128-CBC for contents and AES-128-CTS-CBC for filenames
- Speck128/256-XTS for contents and Speck128/256-CTS-CBC for filenames
It is strongly recommended to use AES-256-XTS for contents encryption.
AES-128-CBC was added only for low-powered embedded devices with
crypto accelerators such as CAAM or CESA that do not support XTS.
Similarly, Speck128/256 support was only added for older or low-end
CPUs which cannot do AES fast enough -- especially ARM CPUs which have
NEON instructions but not the Cryptography Extensions -- and for which
it would not otherwise be feasible to use encryption at all. It is
not recommended to use Speck on CPUs that have AES instructions.
Speck support is only available if it has been enabled in the crypto
API via CONFIG_CRYPTO_SPECK. Also, on ARM platforms, to get
acceptable performance CONFIG_CRYPTO_SPECK_NEON must be enabled.
New encryption modes can be added relatively easily, without changes
to individual filesystems. However, authenticated encryption (AE)
modes are not currently supported because of the difficulty of dealing
with ciphertext expansion.
For file contents, each filesystem block is encrypted independently.
Currently, only the case where the filesystem block size is equal to
the system's page size (usually 4096 bytes) is supported. With the
XTS mode of operation (recommended), the logical block number within
the file is used as the IV. With the CBC mode of operation (not
recommended), ESSIV is used; specifically, the IV for CBC is the
logical block number encrypted with AES-256, where the AES-256 key is
the SHA-256 hash of the inode's data encryption key.
For filenames, the full filename is encrypted at once. Because of the
requirements to retain support for efficient directory lookups and
filenames of up to 255 bytes, a constant initialization vector (IV) is
used. However, each encrypted directory uses a unique key, which
limits IV reuse to within a single directory. Note that IV reuse in
the context of CTS-CBC encryption means that when the original
filenames share a common prefix at least as long as the cipher block
size (16 bytes for AES), the corresponding encrypted filenames will
also share a common prefix. This is undesirable; it may be fixed in
the future by switching to an encryption mode that is a strong
pseudorandom permutation on arbitrary-length messages, e.g. the HEH
(Hash-Encrypt-Hash) mode.
Since filenames are encrypted with the CTS-CBC mode of operation, the
plaintext and ciphertext filenames need not be multiples of the AES
block size, i.e. 16 bytes. However, the minimum size that can be
encrypted is 16 bytes, so shorter filenames are NUL-padded to 16 bytes
before being encrypted. In addition, to reduce leakage of filename
lengths via their ciphertexts, all filenames are NUL-padded to the
next 4, 8, 16, or 32-byte boundary (configurable). 32 is recommended
since this provides the best confidentiality, at the cost of making
directory entries consume slightly more space. Note that since NUL
(``\0``) is not otherwise a valid character in filenames, the padding
will never produce duplicate plaintexts.
Symbolic link targets are considered a type of filename and are
encrypted in the same way as filenames in directory entries. Each
symlink also uses a unique key; hence, the hardcoded IV is not a
problem for symlinks.
User API
========
Setting an encryption policy
----------------------------
The FS_IOC_SET_ENCRYPTION_POLICY ioctl sets an encryption policy on an
empty directory or verifies that a directory or regular file already
has the specified encryption policy. It takes in a pointer to a
:c:type:`struct fscrypt_policy`, defined as follows::
#define FS_KEY_DESCRIPTOR_SIZE 8
struct fscrypt_policy {
__u8 version;
__u8 contents_encryption_mode;
__u8 filenames_encryption_mode;
__u8 flags;
__u8 master_key_descriptor[FS_KEY_DESCRIPTOR_SIZE];
};
This structure must be initialized as follows:
- ``version`` must be 0.
- ``contents_encryption_mode`` and ``filenames_encryption_mode`` must
be set to constants from ``<linux/fs.h>`` which identify the
encryption modes to use. If unsure, use
FS_ENCRYPTION_MODE_AES_256_XTS (1) for ``contents_encryption_mode``
and FS_ENCRYPTION_MODE_AES_256_CTS (4) for
``filenames_encryption_mode``.
- ``flags`` must be set to a value from ``<linux/fs.h>`` which
identifies the amount of NUL-padding to use when encrypting
filenames. If unsure, use FS_POLICY_FLAGS_PAD_32 (0x3).
- ``master_key_descriptor`` specifies how to find the master key in
the keyring; see `Adding keys`_. It is up to userspace to choose a
unique ``master_key_descriptor`` for each master key. The e4crypt
and fscrypt tools use the first 8 bytes of
``SHA-512(SHA-512(master_key))``, but this particular scheme is not
required. Also, the master key need not be in the keyring yet when
FS_IOC_SET_ENCRYPTION_POLICY is executed. However, it must be added
before any files can be created in the encrypted directory.
If the file is not yet encrypted, then FS_IOC_SET_ENCRYPTION_POLICY
verifies that the file is an empty directory. If so, the specified
encryption policy is assigned to the directory, turning it into an
encrypted directory. After that, and after providing the
corresponding master key as described in `Adding keys`_, all regular
files, directories (recursively), and symlinks created in the
directory will be encrypted, inheriting the same encryption policy.
The filenames in the directory's entries will be encrypted as well.
Alternatively, if the file is already encrypted, then
FS_IOC_SET_ENCRYPTION_POLICY validates that the specified encryption
policy exactly matches the actual one. If they match, then the ioctl
returns 0. Otherwise, it fails with EEXIST. This works on both
regular files and directories, including nonempty directories.
Note that the ext4 filesystem does not allow the root directory to be
encrypted, even if it is empty. Users who want to encrypt an entire
filesystem with one key should consider using dm-crypt instead.
FS_IOC_SET_ENCRYPTION_POLICY can fail with the following errors:
- ``EACCES``: the file is not owned by the process's uid, nor does the
process have the CAP_FOWNER capability in a namespace with the file
owner's uid mapped
- ``EEXIST``: the file is already encrypted with an encryption policy
different from the one specified
- ``EINVAL``: an invalid encryption policy was specified (invalid
version, mode(s), or flags)
- ``ENOTDIR``: the file is unencrypted and is a regular file, not a
directory
- ``ENOTEMPTY``: the file is unencrypted and is a nonempty directory
- ``ENOTTY``: this type of filesystem does not implement encryption
- ``EOPNOTSUPP``: the kernel was not configured with encryption
support for this filesystem, or the filesystem superblock has not
had encryption enabled on it. (For example, to use encryption on an
ext4 filesystem, CONFIG_EXT4_ENCRYPTION must be enabled in the
kernel config, and the superblock must have had the "encrypt"
feature flag enabled using ``tune2fs -O encrypt`` or ``mkfs.ext4 -O
encrypt``.)
- ``EPERM``: this directory may not be encrypted, e.g. because it is
the root directory of an ext4 filesystem
- ``EROFS``: the filesystem is readonly
Getting an encryption policy
----------------------------
The FS_IOC_GET_ENCRYPTION_POLICY ioctl retrieves the :c:type:`struct
fscrypt_policy`, if any, for a directory or regular file. See above
for the struct definition. No additional permissions are required
beyond the ability to open the file.
FS_IOC_GET_ENCRYPTION_POLICY can fail with the following errors:
- ``EINVAL``: the file is encrypted, but it uses an unrecognized
encryption context format
- ``ENODATA``: the file is not encrypted
- ``ENOTTY``: this type of filesystem does not implement encryption
- ``EOPNOTSUPP``: the kernel was not configured with encryption
support for this filesystem
Note: if you only need to know whether a file is encrypted or not, on
most filesystems it is also possible to use the FS_IOC_GETFLAGS ioctl
and check for FS_ENCRYPT_FL, or to use the statx() system call and
check for STATX_ATTR_ENCRYPTED in stx_attributes.
Getting the per-filesystem salt
-------------------------------
Some filesystems, such as ext4 and F2FS, also support the deprecated
ioctl FS_IOC_GET_ENCRYPTION_PWSALT. This ioctl retrieves a randomly
generated 16-byte value stored in the filesystem superblock. This
value is intended to used as a salt when deriving an encryption key
from a passphrase or other low-entropy user credential.
FS_IOC_GET_ENCRYPTION_PWSALT is deprecated. Instead, prefer to
generate and manage any needed salt(s) in userspace.
Adding keys
-----------
To provide a master key, userspace must add it to an appropriate
keyring using the add_key() system call (see:
``Documentation/security/keys/core.rst``). The key type must be
"logon"; keys of this type are kept in kernel memory and cannot be
read back by userspace. The key description must be "fscrypt:"
followed by the 16-character lower case hex representation of the
``master_key_descriptor`` that was set in the encryption policy. The
key payload must conform to the following structure::
#define FS_MAX_KEY_SIZE 64
struct fscrypt_key {
u32 mode;
u8 raw[FS_MAX_KEY_SIZE];
u32 size;
};
``mode`` is ignored; just set it to 0. The actual key is provided in
``raw`` with ``size`` indicating its size in bytes. That is, the
bytes ``raw[0..size-1]`` (inclusive) are the actual key.
The key description prefix "fscrypt:" may alternatively be replaced
with a filesystem-specific prefix such as "ext4:". However, the
filesystem-specific prefixes are deprecated and should not be used in
new programs.
There are several different types of keyrings in which encryption keys
may be placed, such as a session keyring, a user session keyring, or a
user keyring. Each key must be placed in a keyring that is "attached"
to all processes that might need to access files encrypted with it, in
the sense that request_key() will find the key. Generally, if only
processes belonging to a specific user need to access a given
encrypted directory and no session keyring has been installed, then
that directory's key should be placed in that user's user session
keyring or user keyring. Otherwise, a session keyring should be
installed if needed, and the key should be linked into that session
keyring, or in a keyring linked into that session keyring.
Note: introducing the complex visibility semantics of keyrings here
was arguably a mistake --- especially given that by design, after any
process successfully opens an encrypted file (thereby setting up the
per-file key), possessing the keyring key is not actually required for
any process to read/write the file until its in-memory inode is
evicted. In the future there probably should be a way to provide keys
directly to the filesystem instead, which would make the intended
semantics clearer.
Access semantics
================
With the key
------------
With the encryption key, encrypted regular files, directories, and
symlinks behave very similarly to their unencrypted counterparts ---
after all, the encryption is intended to be transparent. However,
astute users may notice some differences in behavior:
- Unencrypted files, or files encrypted with a different encryption
policy (i.e. different key, modes, or flags), cannot be renamed or
linked into an encrypted directory; see `Encryption policy
enforcement`_. Attempts to do so will fail with EPERM. However,
encrypted files can be renamed within an encrypted directory, or
into an unencrypted directory.
- Direct I/O is not supported on encrypted files. Attempts to use
direct I/O on such files will fall back to buffered I/O.
- The fallocate operations FALLOC_FL_COLLAPSE_RANGE,
FALLOC_FL_INSERT_RANGE, and FALLOC_FL_ZERO_RANGE are not supported
on encrypted files and will fail with EOPNOTSUPP.
- Online defragmentation of encrypted files is not supported. The
EXT4_IOC_MOVE_EXT and F2FS_IOC_MOVE_RANGE ioctls will fail with
EOPNOTSUPP.
- The ext4 filesystem does not support data journaling with encrypted
regular files. It will fall back to ordered data mode instead.
- DAX (Direct Access) is not supported on encrypted files.
- The st_size of an encrypted symlink will not necessarily give the
length of the symlink target as required by POSIX. It will actually
give the length of the ciphertext, which will be slightly longer
than the plaintext due to NUL-padding and an extra 2-byte overhead.
- The maximum length of an encrypted symlink is 2 bytes shorter than
the maximum length of an unencrypted symlink. For example, on an
EXT4 filesystem with a 4K block size, unencrypted symlinks can be up
to 4095 bytes long, while encrypted symlinks can only be up to 4093
bytes long (both lengths excluding the terminating null).
Note that mmap *is* supported. This is possible because the pagecache
for an encrypted file contains the plaintext, not the ciphertext.
Without the key
---------------
Some filesystem operations may be performed on encrypted regular
files, directories, and symlinks even before their encryption key has
been provided:
- File metadata may be read, e.g. using stat().
- Directories may be listed, in which case the filenames will be
listed in an encoded form derived from their ciphertext. The
current encoding algorithm is described in `Filename hashing and
encoding`_. The algorithm is subject to change, but it is
guaranteed that the presented filenames will be no longer than
NAME_MAX bytes, will not contain the ``/`` or ``\0`` characters, and
will uniquely identify directory entries.
The ``.`` and ``..`` directory entries are special. They are always
present and are not encrypted or encoded.
- Files may be deleted. That is, nondirectory files may be deleted
with unlink() as usual, and empty directories may be deleted with
rmdir() as usual. Therefore, ``rm`` and ``rm -r`` will work as
expected.
- Symlink targets may be read and followed, but they will be presented
in encrypted form, similar to filenames in directories. Hence, they
are unlikely to point to anywhere useful.
Without the key, regular files cannot be opened or truncated.
Attempts to do so will fail with ENOKEY. This implies that any
regular file operations that require a file descriptor, such as
read(), write(), mmap(), fallocate(), and ioctl(), are also forbidden.
Also without the key, files of any type (including directories) cannot
be created or linked into an encrypted directory, nor can a name in an
encrypted directory be the source or target of a rename, nor can an
O_TMPFILE temporary file be created in an encrypted directory. All
such operations will fail with ENOKEY.
It is not currently possible to backup and restore encrypted files
without the encryption key. This would require special APIs which
have not yet been implemented.
Encryption policy enforcement
=============================
After an encryption policy has been set on a directory, all regular
files, directories, and symbolic links created in that directory
(recursively) will inherit that encryption policy. Special files ---
that is, named pipes, device nodes, and UNIX domain sockets --- will
not be encrypted.
Except for those special files, it is forbidden to have unencrypted
files, or files encrypted with a different encryption policy, in an
encrypted directory tree. Attempts to link or rename such a file into
an encrypted directory will fail with EPERM. This is also enforced
during ->lookup() to provide limited protection against offline
attacks that try to disable or downgrade encryption in known locations
where applications may later write sensitive data. It is recommended
that systems implementing a form of "verified boot" take advantage of
this by validating all top-level encryption policies prior to access.
Implementation details
======================
Encryption context
------------------
An encryption policy is represented on-disk by a :c:type:`struct
fscrypt_context`. It is up to individual filesystems to decide where
to store it, but normally it would be stored in a hidden extended
attribute. It should *not* be exposed by the xattr-related system
calls such as getxattr() and setxattr() because of the special
semantics of the encryption xattr. (In particular, there would be
much confusion if an encryption policy were to be added to or removed
from anything other than an empty directory.) The struct is defined
as follows::
#define FS_KEY_DESCRIPTOR_SIZE 8
#define FS_KEY_DERIVATION_NONCE_SIZE 16
struct fscrypt_context {
u8 format;
u8 contents_encryption_mode;
u8 filenames_encryption_mode;
u8 flags;
u8 master_key_descriptor[FS_KEY_DESCRIPTOR_SIZE];
u8 nonce[FS_KEY_DERIVATION_NONCE_SIZE];
};
Note that :c:type:`struct fscrypt_context` contains the same
information as :c:type:`struct fscrypt_policy` (see `Setting an
encryption policy`_), except that :c:type:`struct fscrypt_context`
also contains a nonce. The nonce is randomly generated by the kernel
and is used to derive the inode's encryption key as described in
`Per-file keys`_.
Data path changes
-----------------
For the read path (->readpage()) of regular files, filesystems can
read the ciphertext into the page cache and decrypt it in-place. The
page lock must be held until decryption has finished, to prevent the
page from becoming visible to userspace prematurely.
For the write path (->writepage()) of regular files, filesystems
cannot encrypt data in-place in the page cache, since the cached
plaintext must be preserved. Instead, filesystems must encrypt into a
temporary buffer or "bounce page", then write out the temporary
buffer. Some filesystems, such as UBIFS, already use temporary
buffers regardless of encryption. Other filesystems, such as ext4 and
F2FS, have to allocate bounce pages specially for encryption.
Filename hashing and encoding
-----------------------------
Modern filesystems accelerate directory lookups by using indexed
directories. An indexed directory is organized as a tree keyed by
filename hashes. When a ->lookup() is requested, the filesystem
normally hashes the filename being looked up so that it can quickly
find the corresponding directory entry, if any.
With encryption, lookups must be supported and efficient both with and
without the encryption key. Clearly, it would not work to hash the
plaintext filenames, since the plaintext filenames are unavailable
without the key. (Hashing the plaintext filenames would also make it
impossible for the filesystem's fsck tool to optimize encrypted
directories.) Instead, filesystems hash the ciphertext filenames,
i.e. the bytes actually stored on-disk in the directory entries. When
asked to do a ->lookup() with the key, the filesystem just encrypts
the user-supplied name to get the ciphertext.
Lookups without the key are more complicated. The raw ciphertext may
contain the ``\0`` and ``/`` characters, which are illegal in
filenames. Therefore, readdir() must base64-encode the ciphertext for
presentation. For most filenames, this works fine; on ->lookup(), the
filesystem just base64-decodes the user-supplied name to get back to
the raw ciphertext.
However, for very long filenames, base64 encoding would cause the
filename length to exceed NAME_MAX. To prevent this, readdir()
actually presents long filenames in an abbreviated form which encodes
a strong "hash" of the ciphertext filename, along with the optional
filesystem-specific hash(es) needed for directory lookups. This
allows the filesystem to still, with a high degree of confidence, map
the filename given in ->lookup() back to a particular directory entry
that was previously listed by readdir(). See :c:type:`struct
fscrypt_digested_name` in the source for more details.
Note that the precise way that filenames are presented to userspace
without the key is subject to change in the future. It is only meant
as a way to temporarily present valid filenames so that commands like
``rm -r`` work as expected on encrypted directories.

2
Makefile
View file

@ -1,6 +1,6 @@
VERSION = 4
PATCHLEVEL = 4
SUBLEVEL = 141
SUBLEVEL = 142
EXTRAVERSION =
NAME = Blurry Fish Butt

9
arch/x86/configs/x86_64_cuttlefish_defconfig
View file

@ -214,13 +214,17 @@ CONFIG_SCSI_CONSTANTS=y
CONFIG_SCSI_SPI_ATTRS=y
CONFIG_SCSI_VIRTIO=y
CONFIG_MD=y
CONFIG_BLK_DEV_MD=y
CONFIG_MD_LINEAR=y
CONFIG_BLK_DEV_DM=y
CONFIG_DM_CRYPT=y
CONFIG_DM_MIRROR=y
CONFIG_DM_ZERO=y
CONFIG_DM_UEVENT=y
CONFIG_DM_VERITY=y
CONFIG_DM_VERITY_HASH_PREFETCH_MIN_SIZE=1
CONFIG_DM_VERITY_FEC=y
CONFIG_DM_ANDROID_VERITY=y
CONFIG_NETDEVICES=y
CONFIG_NETCONSOLE=y
CONFIG_NETCONSOLE_DYNAMIC=y
@ -447,3 +451,8 @@ CONFIG_SECURITY_SELINUX_CHECKREQPROT_VALUE=1
# CONFIG_CRYPTO_MANAGER_DISABLE_TESTS is not set
CONFIG_CRYPTO_ECHAINIV=y
CONFIG_CRYPTO_SHA512=y
CONFIG_ASYMMETRIC_KEY_TYPE=y
CONFIG_ASYMMETRIC_PUBLIC_KEY_SUBTYPE=y
CONFIG_X509_CERTIFICATE_PARSER=y
CONFIG_SYSTEM_TRUSTED_KEYRING=y
CONFIG_SYSTEM_TRUSTED_KEYS="verity_dev_keys.x509"

7
arch/x86/kernel/cpu/common.c
View file

@ -686,13 +686,14 @@ void get_cpu_cap(struct cpuinfo_x86 *c)
c->x86_capability[CPUID_1_EDX] = edx;
}
/* Thermal and Power Management Leaf: level 0x00000006 (eax) */
if (c->cpuid_level >= 0x00000006)
c->x86_capability[CPUID_6_EAX] = cpuid_eax(0x00000006);
/* Additional Intel-defined flags: level 0x00000007 */
if (c->cpuid_level >= 0x00000007) {
cpuid_count(0x00000007, 0, &eax, &ebx, &ecx, &edx);
c->x86_capability[CPUID_7_0_EBX] = ebx;
c->x86_capability[CPUID_6_EAX] = cpuid_eax(0x00000006);
c->x86_capability[CPUID_7_ECX] = ecx;
}

15
crypto/algapi.c
View file

@ -1001,6 +1001,21 @@ unsigned int crypto_alg_extsize(struct crypto_alg *alg)
}
EXPORT_SYMBOL_GPL(crypto_alg_extsize);
int crypto_type_has_alg(const char *name, const struct crypto_type *frontend,
u32 type, u32 mask)
{
int ret = 0;
struct crypto_alg *alg = crypto_find_alg(name, frontend, type, mask);
if (!IS_ERR(alg)) {
crypto_mod_put(alg);
ret = 1;
}
return ret;
}
EXPORT_SYMBOL_GPL(crypto_type_has_alg);
static int __init crypto_algapi_init(void)
{
crypto_init_proc();

3
crypto/internal.h
View file

@ -104,6 +104,9 @@ int crypto_probing_notify(unsigned long val, void *v);
unsigned int crypto_alg_extsize(struct crypto_alg *alg);
int crypto_type_has_alg(const char *name, const struct crypto_type *frontend,
u32 type, u32 mask);
static inline struct crypto_alg *crypto_alg_get(struct crypto_alg *alg)
{
atomic_inc(&alg->cra_refcnt);

200
crypto/skcipher.c
View file

@ -16,7 +16,11 @@
#include <crypto/internal/skcipher.h>
#include <linux/bug.h>
#include <linux/cryptouser.h>
#include <linux/module.h>
#include <linux/rtnetlink.h>
#include <linux/seq_file.h>
#include <net/netlink.h>
#include "internal.h"
@ -25,10 +29,11 @@ static unsigned int crypto_skcipher_extsize(struct crypto_alg *alg)
if (alg->cra_type == &crypto_blkcipher_type)
return sizeof(struct crypto_blkcipher *);
BUG_ON(alg->cra_type != &crypto_ablkcipher_type &&
alg->cra_type != &crypto_givcipher_type);
if (alg->cra_type == &crypto_ablkcipher_type ||
alg->cra_type == &crypto_givcipher_type)
return sizeof(struct crypto_ablkcipher *);
return crypto_alg_extsize(alg);
}
static int skcipher_setkey_blkcipher(struct crypto_skcipher *tfm,
@ -118,7 +123,7 @@ static int crypto_init_skcipher_ops_blkcipher(struct crypto_tfm *tfm)
skcipher->decrypt = skcipher_decrypt_blkcipher;
skcipher->ivsize = crypto_blkcipher_ivsize(blkcipher);
skcipher->has_setkey = calg->cra_blkcipher.max_keysize;
skcipher->keysize = calg->cra_blkcipher.max_keysize;
return 0;
}
@ -211,31 +216,123 @@ static int crypto_init_skcipher_ops_ablkcipher(struct crypto_tfm *tfm)
skcipher->ivsize = crypto_ablkcipher_ivsize(ablkcipher);
skcipher->reqsize = crypto_ablkcipher_reqsize(ablkcipher) +
sizeof(struct ablkcipher_request);
skcipher->has_setkey = calg->cra_ablkcipher.max_keysize;
skcipher->keysize = calg->cra_ablkcipher.max_keysize;
return 0;
}
static void crypto_skcipher_exit_tfm(struct crypto_tfm *tfm)
{
struct crypto_skcipher *skcipher = __crypto_skcipher_cast(tfm);
struct skcipher_alg *alg = crypto_skcipher_alg(skcipher);
alg->exit(skcipher);
}
static int crypto_skcipher_init_tfm(struct crypto_tfm *tfm)
{
struct crypto_skcipher *skcipher = __crypto_skcipher_cast(tfm);
struct skcipher_alg *alg = crypto_skcipher_alg(skcipher);
if (tfm->__crt_alg->cra_type == &crypto_blkcipher_type)
return crypto_init_skcipher_ops_blkcipher(tfm);
BUG_ON(tfm->__crt_alg->cra_type != &crypto_ablkcipher_type &&
tfm->__crt_alg->cra_type != &crypto_givcipher_type);
if (tfm->__crt_alg->cra_type == &crypto_ablkcipher_type ||
tfm->__crt_alg->cra_type == &crypto_givcipher_type)
return crypto_init_skcipher_ops_ablkcipher(tfm);
skcipher->setkey = alg->setkey;
skcipher->encrypt = alg->encrypt;
skcipher->decrypt = alg->decrypt;
skcipher->ivsize = alg->ivsize;
skcipher->keysize = alg->max_keysize;
if (alg->exit)
skcipher->base.exit = crypto_skcipher_exit_tfm;
if (alg->init)
return alg->init(skcipher);
return 0;
}
static void crypto_skcipher_free_instance(struct crypto_instance *inst)
{
struct skcipher_instance *skcipher =
container_of(inst, struct skcipher_instance, s.base);
skcipher->free(skcipher);
}
static void crypto_skcipher_show(struct seq_file *m, struct crypto_alg *alg)
__attribute__ ((unused));
static void crypto_skcipher_show(struct seq_file *m, struct crypto_alg *alg)
{
struct skcipher_alg *skcipher = container_of(alg, struct skcipher_alg,
base);
seq_printf(m, "type : skcipher\n");
seq_printf(m, "async : %s\n",
alg->cra_flags & CRYPTO_ALG_ASYNC ? "yes" : "no");
seq_printf(m, "blocksize : %u\n", alg->cra_blocksize);
seq_printf(m, "min keysize : %u\n", skcipher->min_keysize);
seq_printf(m, "max keysize : %u\n", skcipher->max_keysize);
seq_printf(m, "ivsize : %u\n", skcipher->ivsize);
seq_printf(m, "chunksize : %u\n", skcipher->chunksize);
}
#ifdef CONFIG_NET
static int crypto_skcipher_report(struct sk_buff *skb, struct crypto_alg *alg)
{
struct crypto_report_blkcipher rblkcipher;
struct skcipher_alg *skcipher = container_of(alg, struct skcipher_alg,
base);
strlcpy(rblkcipher.type, "skcipher", sizeof(rblkcipher.type));
strlcpy(rblkcipher.geniv, "<none>", sizeof(rblkcipher.geniv));
rblkcipher.blocksize = alg->cra_blocksize;
rblkcipher.min_keysize = skcipher->min_keysize;
rblkcipher.max_keysize = skcipher->max_keysize;
rblkcipher.ivsize = skcipher->ivsize;
if (nla_put(skb, CRYPTOCFGA_REPORT_BLKCIPHER,
sizeof(struct crypto_report_blkcipher), &rblkcipher))
goto nla_put_failure;
return 0;
nla_put_failure:
return -EMSGSIZE;
}
#else
static int crypto_skcipher_report(struct sk_buff *skb, struct crypto_alg *alg)
{
return -ENOSYS;
}
#endif
static const struct crypto_type crypto_skcipher_type2 = {
.extsize = crypto_skcipher_extsize,
.init_tfm = crypto_skcipher_init_tfm,
.free = crypto_skcipher_free_instance,
#ifdef CONFIG_PROC_FS
.show = crypto_skcipher_show,
#endif
.report = crypto_skcipher_report,
.maskclear = ~CRYPTO_ALG_TYPE_MASK,
.maskset = CRYPTO_ALG_TYPE_BLKCIPHER_MASK,
.type = CRYPTO_ALG_TYPE_BLKCIPHER,
.type = CRYPTO_ALG_TYPE_SKCIPHER,
.tfmsize = offsetof(struct crypto_skcipher, base),
};
int crypto_grab_skcipher2(struct crypto_skcipher_spawn *spawn,
const char *name, u32 type, u32 mask)
{
spawn->base.frontend = &crypto_skcipher_type2;
return crypto_grab_spawn(&spawn->base, name, type, mask);
}
EXPORT_SYMBOL_GPL(crypto_grab_skcipher2);
struct crypto_skcipher *crypto_alloc_skcipher(const char *alg_name,
u32 type, u32 mask)
{
@ -243,5 +340,90 @@ struct crypto_skcipher *crypto_alloc_skcipher(const char *alg_name,
}
EXPORT_SYMBOL_GPL(crypto_alloc_skcipher);
int crypto_has_skcipher2(const char *alg_name, u32 type, u32 mask)
{
return crypto_type_has_alg(alg_name, &crypto_skcipher_type2,
type, mask);
}
EXPORT_SYMBOL_GPL(crypto_has_skcipher2);
static int skcipher_prepare_alg(struct skcipher_alg *alg)
{
struct crypto_alg *base = &alg->base;
if (alg->ivsize > PAGE_SIZE / 8 || alg->chunksize > PAGE_SIZE / 8)
return -EINVAL;
if (!alg->chunksize)
alg->chunksize = base->cra_blocksize;
base->cra_type = &crypto_skcipher_type2;
base->cra_flags &= ~CRYPTO_ALG_TYPE_MASK;
base->cra_flags |= CRYPTO_ALG_TYPE_SKCIPHER;
return 0;
}
int crypto_register_skcipher(struct skcipher_alg *alg)
{
struct crypto_alg *base = &alg->base;
int err;
err = skcipher_prepare_alg(alg);
if (err)
return err;
return crypto_register_alg(base);
}
EXPORT_SYMBOL_GPL(crypto_register_skcipher);
void crypto_unregister_skcipher(struct skcipher_alg *alg)
{
crypto_unregister_alg(&alg->base);
}
EXPORT_SYMBOL_GPL(crypto_unregister_skcipher);
int crypto_register_skciphers(struct skcipher_alg *algs, int count)
{
int i, ret;
for (i = 0; i < count; i++) {
ret = crypto_register_skcipher(&algs[i]);
if (ret)
goto err;
}
return 0;
err:
for (--i; i >= 0; --i)
crypto_unregister_skcipher(&algs[i]);
return ret;
}
EXPORT_SYMBOL_GPL(crypto_register_skciphers);
void crypto_unregister_skciphers(struct skcipher_alg *algs, int count)
{
int i;
for (i = count - 1; i >= 0; --i)
crypto_unregister_skcipher(&algs[i]);
}
EXPORT_SYMBOL_GPL(crypto_unregister_skciphers);
int skcipher_register_instance(struct crypto_template *tmpl,
struct skcipher_instance *inst)
{
int err;
err = skcipher_prepare_alg(&inst->alg);
if (err)
return err;
return crypto_register_instance(tmpl, skcipher_crypto_instance(inst));
}
EXPORT_SYMBOL_GPL(skcipher_register_instance);
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("Symmetric key cipher type");

15
drivers/android/Kconfig
View file

@ -9,7 +9,7 @@ if ANDROID
config ANDROID_BINDER_IPC
bool "Android Binder IPC Driver"
depends on MMU
depends on MMU && !M68K
default n
---help---
Binder is used in Android for both communication between processes,
@ -31,19 +31,6 @@ config ANDROID_BINDER_DEVICES
created. Each binder device has its own context manager, and is
therefore logically separated from the other devices.
config ANDROID_BINDER_IPC_32BIT
bool
depends on !64BIT && ANDROID_BINDER_IPC
default y
---help---
The Binder API has been changed to support both 32 and 64bit
applications in a mixed environment.
Enable this to support an old 32-bit Android user-space (v4.4 and
earlier).
Note that enabling this will break newer Android user-space.
config ANDROID_BINDER_IPC_SELFTEST
bool "Android Binder IPC Driver Selftest"
depends on ANDROID_BINDER_IPC

138
drivers/android/binder.c
View file

@ -71,10 +71,6 @@
#include <linux/security.h>
#include <linux/spinlock.h>
#ifdef CONFIG_ANDROID_BINDER_IPC_32BIT
#define BINDER_IPC_32BIT 1
#endif
#include <uapi/linux/android/binder.h>
#include "binder_alloc.h"
#include "binder_trace.h"
@ -143,7 +139,7 @@ enum {
};
static uint32_t binder_debug_mask = BINDER_DEBUG_USER_ERROR |
BINDER_DEBUG_FAILED_TRANSACTION | BINDER_DEBUG_DEAD_TRANSACTION;
module_param_named(debug_mask, binder_debug_mask, uint, S_IWUSR | S_IRUGO);
module_param_named(debug_mask, binder_debug_mask, uint, 0644);
static char *binder_devices_param = CONFIG_ANDROID_BINDER_DEVICES;
module_param_named(devices, binder_devices_param, charp, S_IRUGO);
@ -162,7 +158,7 @@ static int binder_set_stop_on_user_error(const char *val,
return ret;
}
module_param_call(stop_on_user_error, binder_set_stop_on_user_error,
param_get_int, &binder_stop_on_user_error, S_IWUSR | S_IRUGO);
param_get_int, &binder_stop_on_user_error, 0644);
#define binder_debug(mask, x...) \
do { \
@ -251,7 +247,7 @@ static struct binder_transaction_log_entry *binder_transaction_log_add(
unsigned int cur = atomic_inc_return(&log->cur);
if (cur >= ARRAY_SIZE(log->entry))
log->full = 1;
log->full = true;
e = &log->entry[cur % ARRAY_SIZE(log->entry)];
WRITE_ONCE(e->debug_id_done, 0);
/*
@ -466,8 +462,9 @@ struct binder_ref {
};
enum binder_deferred_state {
BINDER_DEFERRED_FLUSH = 0x01,
BINDER_DEFERRED_RELEASE = 0x02,
BINDER_DEFERRED_PUT_FILES = 0x01,
BINDER_DEFERRED_FLUSH = 0x02,
BINDER_DEFERRED_RELEASE = 0x04,
};
/**
@ -504,6 +501,9 @@ struct binder_priority {
* (invariant after initialized)
* @tsk task_struct for group_leader of process
* (invariant after initialized)
* @files files_struct for process
* (protected by @files_lock)
* @files_lock mutex to protect @files
* @deferred_work_node: element for binder_deferred_list
* (protected by binder_deferred_lock)
* @deferred_work: bitmap of deferred work to perform
@ -548,6 +548,8 @@ struct binder_proc {
struct list_head waiting_threads;
int pid;
struct task_struct *tsk;
struct files_struct *files;
struct mutex files_lock;
struct hlist_node deferred_work_node;
int deferred_work;
bool is_dead;
@ -942,33 +944,27 @@ static void binder_free_thread(struct binder_thread *thread);
static void binder_free_proc(struct binder_proc *proc);
static void binder_inc_node_tmpref_ilocked(struct binder_node *node);
struct files_struct *binder_get_files_struct(struct binder_proc *proc)
{
return get_files_struct(proc->tsk);
}
static int task_get_unused_fd_flags(struct binder_proc *proc, int flags)
{
struct files_struct *files;
unsigned long rlim_cur;
unsigned long irqs;
int ret;
files = binder_get_files_struct(proc);
if (files == NULL)
return -ESRCH;
mutex_lock(&proc->files_lock);
if (proc->files == NULL) {
ret = -ESRCH;
goto err;
}
if (!lock_task_sighand(proc->tsk, &irqs)) {
ret = -EMFILE;
goto err;
}
rlim_cur = task_rlimit(proc->tsk, RLIMIT_NOFILE);
unlock_task_sighand(proc->tsk, &irqs);
ret = __alloc_fd(files, 0, rlim_cur, flags);
ret = __alloc_fd(proc->files, 0, rlim_cur, flags);
err:
put_files_struct(files);
mutex_unlock(&proc->files_lock);
return ret;
}
@ -978,12 +974,10 @@ err:
static void task_fd_install(
struct binder_proc *proc, unsigned int fd, struct file *file)
{
struct files_struct *files = binder_get_files_struct(proc);
if (files) {
__fd_install(files, fd, file);
put_files_struct(files);
}
mutex_lock(&proc->files_lock);
if (proc->files)
__fd_install(proc->files, fd, file);
mutex_unlock(&proc->files_lock);
}
/*
@ -991,21 +985,22 @@ static void task_fd_install(
*/
static long task_close_fd(struct binder_proc *proc, unsigned int fd)
{
struct files_struct *files = binder_get_files_struct(proc);
int retval;
if (files == NULL)
return -ESRCH;
retval = __close_fd(files, fd);
mutex_lock(&proc->files_lock);
if (proc->files == NULL) {
retval = -ESRCH;
goto err;
}
retval = __close_fd(proc->files, fd);
/* can't restart close syscall because file table entry was cleared */
if (unlikely(retval == -ERESTARTSYS ||
retval == -ERESTARTNOINTR ||
retval == -ERESTARTNOHAND ||
retval == -ERESTART_RESTARTBLOCK))
retval = -EINTR;
put_files_struct(files);
err:
mutex_unlock(&proc->files_lock);
return retval;
}
@ -2215,8 +2210,8 @@ static size_t binder_validate_object(struct binder_buffer *buffer, u64 offset)
struct binder_object_header *hdr;
size_t object_size = 0;
if (offset > buffer->data_size - sizeof(*hdr) ||
buffer->data_size < sizeof(*hdr) ||
if (buffer->data_size < sizeof(*hdr) ||
offset > buffer->data_size - sizeof(*hdr) ||
!IS_ALIGNED(offset, sizeof(u32)))
return 0;
@ -2356,7 +2351,7 @@ static void binder_transaction_buffer_release(struct binder_proc *proc,
int debug_id = buffer->debug_id;
binder_debug(BINDER_DEBUG_TRANSACTION,
"%d buffer release %d, size %zd-%zd, failed at %p\n",
"%d buffer release %d, size %zd-%zd, failed at %pK\n",
proc->pid, buffer->debug_id,
buffer->data_size, buffer->offsets_size, failed_at);
@ -2805,7 +2800,7 @@ static bool binder_proc_transaction(struct binder_transaction *t,
if (node->has_async_transaction) {
pending_async = true;
} else {
node->has_async_transaction = 1;
node->has_async_transaction = true;
}
}
@ -3670,7 +3665,7 @@ static int binder_thread_write(struct binder_proc *proc,
w = binder_dequeue_work_head_ilocked(
&buf_node->async_todo);
if (!w) {
buf_node->has_async_transaction = 0;
buf_node->has_async_transaction = false;
} else {
binder_enqueue_work_ilocked(
w, &proc->todo);
@ -3892,7 +3887,7 @@ static int binder_thread_write(struct binder_proc *proc,
}
}
binder_debug(BINDER_DEBUG_DEAD_BINDER,
"%d:%d BC_DEAD_BINDER_DONE %016llx found %p\n",
"%d:%d BC_DEAD_BINDER_DONE %016llx found %pK\n",
proc->pid, thread->pid, (u64)cookie,
death);
if (death == NULL) {
@ -4098,6 +4093,7 @@ retry:
binder_inner_proc_unlock(proc);
if (put_user(e->cmd, (uint32_t __user *)ptr))
return -EFAULT;
cmd = e->cmd;
e->cmd = BR_OK;
ptr += sizeof(uint32_t);
@ -4866,6 +4862,7 @@ static void binder_vma_close(struct vm_area_struct *vma)
(vma->vm_end - vma->vm_start) / SZ_1K, vma->vm_flags,
(unsigned long)pgprot_val(vma->vm_page_prot));
binder_alloc_vma_close(&proc->alloc);
binder_defer_work(proc, BINDER_DEFERRED_PUT_FILES);
}
static int binder_vm_fault(struct vm_area_struct *vma, struct vm_fault *vmf)
@ -4902,16 +4899,22 @@ static int binder_mmap(struct file *filp, struct vm_area_struct *vma)
failure_string = "bad vm_flags";
goto err_bad_arg;
}
vma->vm_flags = (vma->vm_flags | VM_DONTCOPY) & ~VM_MAYWRITE;
vma->vm_flags |= VM_DONTCOPY | VM_MIXEDMAP;
vma->vm_flags &= ~VM_MAYWRITE;
vma->vm_ops = &binder_vm_ops;
vma->vm_private_data = proc;
ret = binder_alloc_mmap_handler(&proc->alloc, vma);
if (ret)
return ret;
mutex_lock(&proc->files_lock);
proc->files = get_files_struct(current);
mutex_unlock(&proc->files_lock);
return 0;
err_bad_arg:
pr_err("binder_mmap: %d %lx-%lx %s failed %d\n",
pr_err("%s: %d %lx-%lx %s failed %d\n", __func__,
proc->pid, vma->vm_start, vma->vm_end, failure_string, ret);
return ret;
}
@ -4921,7 +4924,7 @@ static int binder_open(struct inode *nodp, struct file *filp)
struct binder_proc *proc;
struct binder_device *binder_dev;
binder_debug(BINDER_DEBUG_OPEN_CLOSE, "binder_open: %d:%d\n",
binder_debug(BINDER_DEBUG_OPEN_CLOSE, "%s: %d:%d\n", __func__,
current->group_leader->pid, current->pid);
proc = kzalloc(sizeof(*proc), GFP_KERNEL);
@ -4931,6 +4934,7 @@ static int binder_open(struct inode *nodp, struct file *filp)
spin_lock_init(&proc->outer_lock);
get_task_struct(current->group_leader);
proc->tsk = current->group_leader;
mutex_init(&proc->files_lock);
INIT_LIST_HEAD(&proc->todo);
if (binder_supported_policy(current->policy)) {
proc->default_priority.sched_policy = current->policy;
@ -4966,7 +4970,7 @@ static int binder_open(struct inode *nodp, struct file *filp)
* anyway print all contexts that a given PID has, so this
* is not a problem.
*/
proc->debugfs_entry = debugfs_create_file(strbuf, S_IRUGO,
proc->debugfs_entry = debugfs_create_file(strbuf, 0444,
binder_debugfs_dir_entry_proc,
(void *)(unsigned long)proc->pid,
&binder_proc_fops);
@ -5087,6 +5091,8 @@ static void binder_deferred_release(struct binder_proc *proc)
struct rb_node *n;
int threads, nodes, incoming_refs, outgoing_refs, active_transactions;
BUG_ON(proc->files);
mutex_lock(&binder_procs_lock);
hlist_del(&proc->proc_node);
mutex_unlock(&binder_procs_lock);
@ -5168,6 +5174,8 @@ static void binder_deferred_release(struct binder_proc *proc)
static void binder_deferred_func(struct work_struct *work)
{
struct binder_proc *proc;
struct files_struct *files;
int defer;
do {
@ -5184,11 +5192,23 @@ static void binder_deferred_func(struct work_struct *work)
}
mutex_unlock(&binder_deferred_lock);
files = NULL;
if (defer & BINDER_DEFERRED_PUT_FILES) {
mutex_lock(&proc->files_lock);
files = proc->files;
if (files)
proc->files = NULL;
mutex_unlock(&proc->files_lock);
}
if (defer & BINDER_DEFERRED_FLUSH)
binder_deferred_flush(proc);
if (defer & BINDER_DEFERRED_RELEASE)
binder_deferred_release(proc); /* frees proc */
if (files)
put_files_struct(files);
} while (proc);
}
static DECLARE_WORK(binder_deferred_work, binder_deferred_func);
@ -5217,7 +5237,7 @@ static void print_binder_transaction_ilocked(struct seq_file *m,
spin_lock(&t->lock);
to_proc = t->to_proc;
seq_printf(m,
"%s %d: %p from %d:%d to %d:%d code %x flags %x pri %d:%d r%d",
"%s %d: %pK from %d:%d to %d:%d code %x flags %x pri %d:%d r%d",
prefix, t->debug_id, t,
t->from ? t->from->proc->pid : 0,
t->from ? t->from->pid : 0,
@ -5242,7 +5262,7 @@ static void print_binder_transaction_ilocked(struct seq_file *m,
}
if (buffer->target_node)
seq_printf(m, " node %d", buffer->target_node->debug_id);
seq_printf(m, " size %zd:%zd data %p\n",
seq_printf(m, " size %zd:%zd data %pK\n",
buffer->data_size, buffer->offsets_size,
buffer->data);
}
@ -5777,11 +5797,13 @@ static int __init init_binder_device(const char *name)
static int __init binder_init(void)
{
int ret;
char *device_name, *device_names;
char *device_name, *device_names, *device_tmp;
struct binder_device *device;
struct hlist_node *tmp;
binder_alloc_shrinker_init();
ret = binder_alloc_shrinker_init();
if (ret)
return ret;
atomic_set(&binder_transaction_log.cur, ~0U);
atomic_set(&binder_transaction_log_failed.cur, ~0U);
@ -5796,27 +5818,27 @@ static int __init binder_init(void)
if (binder_debugfs_dir_entry_root) {
debugfs_create_file("state",
S_IRUGO,
0444,
binder_debugfs_dir_entry_root,
NULL,
&binder_state_fops);
debugfs_create_file("stats",
S_IRUGO,
0444,
binder_debugfs_dir_entry_root,
NULL,
&binder_stats_fops);
debugfs_create_file("transactions",
S_IRUGO,
0444,
binder_debugfs_dir_entry_root,
NULL,
&binder_transactions_fops);
debugfs_create_file("transaction_log",
S_IRUGO,
0444,
binder_debugfs_dir_entry_root,
&binder_transaction_log,
&binder_transaction_log_fops);
debugfs_create_file("failed_transaction_log",
S_IRUGO,
0444,
binder_debugfs_dir_entry_root,
&binder_transaction_log_failed,
&binder_transaction_log_fops);
@ -5833,7 +5855,8 @@ static int __init binder_init(void)
}
strcpy(device_names, binder_devices_param);
while ((device_name = strsep(&device_names, ","))) {
device_tmp = device_names;
while ((device_name = strsep(&device_tmp, ","))) {
ret = init_binder_device(device_name);
if (ret)
goto err_init_binder_device_failed;
@ -5847,6 +5870,9 @@ err_init_binder_device_failed:
hlist_del(&device->hlist);
kfree(device);
}
kfree(device_names);
err_alloc_device_names_failed:
debugfs_remove_recursive(binder_debugfs_dir_entry_root);

23
drivers/android/binder_alloc.c
View file

@ -220,7 +220,7 @@ static int binder_update_page_range(struct binder_alloc *alloc, int allocate,
mm = alloc->vma_vm_mm;
if (mm) {
down_write(&mm->mmap_sem);
down_read(&mm->mmap_sem);
vma = alloc->vma;
}
@ -289,7 +289,7 @@ static int binder_update_page_range(struct binder_alloc *alloc, int allocate,
/* vm_insert_page does not seem to increment the refcount */
}
if (mm) {
up_write(&mm->mmap_sem);
up_read(&mm->mmap_sem);
mmput(mm);
}
return 0;
@ -322,13 +322,14 @@ err_page_ptr_cleared:
}
err_no_vma:
if (mm) {
up_write(&mm->mmap_sem);
up_read(&mm->mmap_sem);
mmput(mm);
}
return vma ? -ENOMEM : -ESRCH;
}
struct binder_buffer *binder_alloc_new_buf_locked(struct binder_alloc *alloc,
static struct binder_buffer *binder_alloc_new_buf_locked(
struct binder_alloc *alloc,
size_t data_size,
size_t offsets_size,
size_t extra_buffers_size,
@ -670,7 +671,7 @@ int binder_alloc_mmap_handler(struct binder_alloc *alloc,
goto err_already_mapped;
}
area = get_vm_area(vma->vm_end - vma->vm_start, VM_IOREMAP);
area = get_vm_area(vma->vm_end - vma->vm_start, VM_ALLOC);
if (area == NULL) {
ret = -ENOMEM;
failure_string = "get_vm_area";
@ -1010,8 +1011,14 @@ void binder_alloc_init(struct binder_alloc *alloc)
INIT_LIST_HEAD(&alloc->buffers);
}
void binder_alloc_shrinker_init(void)
int binder_alloc_shrinker_init(void)
{
list_lru_init(&binder_alloc_lru);
register_shrinker(&binder_shrinker);
int ret = list_lru_init(&binder_alloc_lru);
if (ret == 0) {
ret = register_shrinker(&binder_shrinker);
if (ret)
list_lru_destroy(&binder_alloc_lru);
}
return ret;
}

2
drivers/android/binder_alloc.h
View file

@ -130,7 +130,7 @@ extern struct binder_buffer *binder_alloc_new_buf(struct binder_alloc *alloc,
size_t extra_buffers_size,
int is_async);
extern void binder_alloc_init(struct binder_alloc *alloc);
void binder_alloc_shrinker_init(void);
extern int binder_alloc_shrinker_init(void);
extern void binder_alloc_vma_close(struct binder_alloc *alloc);
extern struct binder_buffer *
binder_alloc_prepare_to_free(struct binder_alloc *alloc,

9
drivers/cpufreq/cpufreq_times.c
View file

@ -234,16 +234,19 @@ static int uid_time_in_state_seq_show(struct seq_file *m, void *v)
void cpufreq_task_times_init(struct task_struct *p)
{
void *temp;
unsigned long flags;
unsigned int max_state;
spin_lock_irqsave(&task_time_in_state_lock, flags);
p->time_in_state = NULL;
spin_unlock_irqrestore(&task_time_in_state_lock, flags);
p->max_state = 0;
}
max_state = READ_ONCE(next_offset);
void cpufreq_task_times_alloc(struct task_struct *p)
{
void *temp;
unsigned long flags;
unsigned int max_state = READ_ONCE(next_offset);
/* We use one array to avoid multiple allocs per task */
temp = kcalloc(max_state, sizeof(p->time_in_state[0]), GFP_ATOMIC);

10
drivers/md/dm-table.c
View file

@ -508,14 +508,14 @@ static int adjoin(struct dm_table *table, struct dm_target *ti)
* On the other hand, dm-switch needs to process bulk data using messages and
* excessive use of GFP_NOIO could cause trouble.
*/
static char **realloc_argv(unsigned *array_size, char **old_argv)
static char **realloc_argv(unsigned *size, char **old_argv)
{
char **argv;
unsigned new_size;
gfp_t gfp;
if (*array_size) {
new_size = *array_size * 2;
if (*size) {
new_size = *size * 2;
gfp = GFP_KERNEL;
} else {
new_size = 8;
@ -523,8 +523,8 @@ static char **realloc_argv(unsigned *array_size, char **old_argv)
}
argv = kmalloc(new_size * sizeof(*argv), gfp);
if (argv) {
memcpy(argv, old_argv, *array_size * sizeof(*argv));
*array_size = new_size;
memcpy(argv, old_argv, *size * sizeof(*argv));
*size = new_size;
}
kfree(old_argv);

19
fs/btrfs/extent_io.c
View file

@ -3932,8 +3932,8 @@ retry:
if (wbc->sync_mode == WB_SYNC_ALL)
tag_pages_for_writeback(mapping, index, end);
while (!done && !nr_to_write_done && (index <= end) &&
(nr_pages = pagevec_lookup_tag(&pvec, mapping, &index, tag,
min(end - index, (pgoff_t)PAGEVEC_SIZE-1) + 1))) {
(nr_pages = pagevec_lookup_range_tag(&pvec, mapping, &index, end,
tag))) {
unsigned i;
scanned = 1;
@ -3943,11 +3943,6 @@ retry:
if (!PagePrivate(page))
continue;
if (!wbc->range_cyclic && page->index > end) {
done = 1;
break;
}
spin_lock(&mapping->private_lock);
if (!PagePrivate(page)) {
spin_unlock(&mapping->private_lock);
@ -4076,8 +4071,8 @@ retry:
if (wbc->sync_mode == WB_SYNC_ALL)
tag_pages_for_writeback(mapping, index, end);
while (!done && !nr_to_write_done && (index <= end) &&
(nr_pages = pagevec_lookup_tag(&pvec, mapping, &index, tag,
min(end - index, (pgoff_t)PAGEVEC_SIZE-1) + 1))) {
(nr_pages = pagevec_lookup_range_tag(&pvec, mapping,
&index, end, tag))) {
unsigned i;
scanned = 1;
@ -4101,12 +4096,6 @@ retry:
continue;
}
if (!wbc->range_cyclic && page->index > end) {
done = 1;
unlock_page(page);
continue;
}
if (wbc->sync_mode != WB_SYNC_NONE) {
if (PageWriteback(page))
flush_fn(data);

14
fs/ceph/addr.c
View file

@ -786,8 +786,7 @@ retry:
struct page **pages = NULL;
mempool_t *pool = NULL; /* Becomes non-null if mempool used */
struct page *page;
int want;
u64 offset, len;
u64 offset = 0, len = 0;
long writeback_stat;
next = 0;
@ -796,14 +795,9 @@ retry:
get_more_pages:
first = -1;
want = min(end - index,
min((pgoff_t)PAGEVEC_SIZE,
max_pages - (pgoff_t)locked_pages) - 1)
+ 1;
pvec_pages = pagevec_lookup_tag(&pvec, mapping, &index,
PAGECACHE_TAG_DIRTY,
want);
dout("pagevec_lookup_tag got %d\n", pvec_pages);
pvec_pages = pagevec_lookup_range_tag(&pvec, mapping, &index,
end, PAGECACHE_TAG_DIRTY);
dout("pagevec_lookup_range_tag got %d\n", pvec_pages);
if (!pvec_pages && !locked_pages)
break;
for (i = 0; i < pvec_pages && locked_pages < max_pages; i++) {

47
fs/crypto/crypto.c
View file

@ -162,12 +162,8 @@ int fscrypt_do_page_crypto(const struct inode *inode, fscrypt_direction_t rw,
}
req = skcipher_request_alloc(tfm, gfp_flags);
if (!req) {
printk_ratelimited(KERN_ERR
"%s: crypto_request_alloc() failed\n",
__func__);
if (!req)
return -ENOMEM;
}
skcipher_request_set_callback(
req, CRYPTO_TFM_REQ_MAY_BACKLOG | CRYPTO_TFM_REQ_MAY_SLEEP,
@ -184,9 +180,10 @@ int fscrypt_do_page_crypto(const struct inode *inode, fscrypt_direction_t rw,
res = crypto_wait_req(crypto_skcipher_encrypt(req), &wait);
skcipher_request_free(req);
if (res) {
printk_ratelimited(KERN_ERR
"%s: crypto_skcipher_encrypt() returned %d\n",
__func__, res);
fscrypt_err(inode->i_sb,
"%scryption failed for inode %lu, block %llu: %d",
(rw == FS_DECRYPT ? "de" : "en"),
inode->i_ino, lblk_num, res);
return res;
}
return 0;
@ -332,7 +329,6 @@ static int fscrypt_d_revalidate(struct dentry *dentry, unsigned int flags)
return 0;
}
/* this should eventually be an flag in d_flags */
spin_lock(&dentry->d_lock);
cached_with_key = dentry->d_flags & DCACHE_ENCRYPTED_WITH_KEY;
spin_unlock(&dentry->d_lock);
@ -359,7 +355,6 @@ static int fscrypt_d_revalidate(struct dentry *dentry, unsigned int flags)
const struct dentry_operations fscrypt_d_ops = {
.d_revalidate = fscrypt_d_revalidate,
};
EXPORT_SYMBOL(fscrypt_d_ops);
void fscrypt_restore_control_page(struct page *page)
{
@ -428,13 +423,43 @@ fail:
return res;
}
void fscrypt_msg(struct super_block *sb, const char *level,
const char *fmt, ...)
{
static DEFINE_RATELIMIT_STATE(rs, DEFAULT_RATELIMIT_INTERVAL,
DEFAULT_RATELIMIT_BURST);
struct va_format vaf;
va_list args;
if (!__ratelimit(&rs))
return;
va_start(args, fmt);
vaf.fmt = fmt;
vaf.va = &args;
if (sb)
printk("%sfscrypt (%s): %pV\n", level, sb->s_id, &vaf);
else
printk("%sfscrypt: %pV\n", level, &vaf);
va_end(args);
}
/**
* fscrypt_init() - Set up for fs encryption.
*/
static int __init fscrypt_init(void)
{
/*
* Use an unbound workqueue to allow bios to be decrypted in parallel
* even when they happen to complete on the same CPU. This sacrifices
* locality, but it's worthwhile since decryption is CPU-intensive.
*
* Also use a high-priority workqueue to prioritize decryption work,
* which blocks reads from completing, over regular application tasks.
*/
fscrypt_read_workqueue = alloc_workqueue("fscrypt_read_queue",
WQ_HIGHPRI, 0);
WQ_UNBOUND | WQ_HIGHPRI,
num_online_cpus());
if (!fscrypt_read_workqueue)
goto fail;

32
fs/crypto/fname.c
View file

@ -58,11 +58,8 @@ int fname_encrypt(struct inode *inode, const struct qstr *iname,
/* Set up the encryption request */
req = skcipher_request_alloc(tfm, GFP_NOFS);
if (!req) {
printk_ratelimited(KERN_ERR
"%s: skcipher_request_alloc() failed\n", __func__);
if (!req)
return -ENOMEM;
}
skcipher_request_set_callback(req,
CRYPTO_TFM_REQ_MAY_BACKLOG | CRYPTO_TFM_REQ_MAY_SLEEP,
crypto_req_done, &wait);
@ -73,8 +70,9 @@ int fname_encrypt(struct inode *inode, const struct qstr *iname,
res = crypto_wait_req(crypto_skcipher_encrypt(req), &wait);
skcipher_request_free(req);
if (res < 0) {
printk_ratelimited(KERN_ERR
"%s: Error (error code %d)\n", __func__, res);
fscrypt_err(inode->i_sb,
"Filename encryption failed for inode %lu: %d",
inode->i_ino, res);
return res;
}
@ -95,23 +93,14 @@ static int fname_decrypt(struct inode *inode,
struct skcipher_request *req = NULL;
DECLARE_CRYPTO_WAIT(wait);
struct scatterlist src_sg, dst_sg;
struct fscrypt_info *ci = inode->i_crypt_info;
struct crypto_skcipher *tfm = ci->ci_ctfm;
struct crypto_skcipher *tfm = inode->i_crypt_info->ci_ctfm;
int res = 0;
char iv[FS_CRYPTO_BLOCK_SIZE];
unsigned lim;
lim = inode->i_sb->s_cop->max_namelen(inode);
if (iname->len <= 0 || iname->len > lim)
return -EIO;
/* Allocate request */
req = skcipher_request_alloc(tfm, GFP_NOFS);
if (!req) {
printk_ratelimited(KERN_ERR
"%s: crypto_request_alloc() failed\n", __func__);
if (!req)
return -ENOMEM;
}
skcipher_request_set_callback(req,
CRYPTO_TFM_REQ_MAY_BACKLOG | CRYPTO_TFM_REQ_MAY_SLEEP,
crypto_req_done, &wait);
@ -126,8 +115,9 @@ static int fname_decrypt(struct inode *inode,
res = crypto_wait_req(crypto_skcipher_decrypt(req), &wait);
skcipher_request_free(req);
if (res < 0) {
printk_ratelimited(KERN_ERR
"%s: Error (error code %d)\n", __func__, res);
fscrypt_err(inode->i_sb,
"Filename decryption failed for inode %lu: %d",
inode->i_ino, res);
return res;
}
@ -340,12 +330,12 @@ int fscrypt_setup_filename(struct inode *dir, const struct qstr *iname,
return 0;
}
ret = fscrypt_get_encryption_info(dir);
if (ret && ret != -EOPNOTSUPP)
if (ret)
return ret;
if (dir->i_crypt_info) {
if (!fscrypt_fname_encrypted_size(dir, iname->len,
dir->i_sb->s_cop->max_namelen(dir),
dir->i_sb->s_cop->max_namelen,
&fname->crypto_buf.len))
return -ENAMETOOLONG;
fname->crypto_buf.name = kmalloc(fname->crypto_buf.len,

17
fs/crypto/fscrypt_private.h
View file

@ -17,14 +17,6 @@
/* Encryption parameters */
#define FS_IV_SIZE 16
#define FS_AES_128_ECB_KEY_SIZE 16
#define FS_AES_128_CBC_KEY_SIZE 16
#define FS_AES_128_CTS_KEY_SIZE 16
#define FS_AES_256_GCM_KEY_SIZE 32
#define FS_AES_256_CBC_KEY_SIZE 32
#define FS_AES_256_CTS_KEY_SIZE 32
#define FS_AES_256_XTS_KEY_SIZE 64
#define FS_KEY_DERIVATION_NONCE_SIZE 16
/**
@ -119,6 +111,15 @@ extern int fscrypt_do_page_crypto(const struct inode *inode,
gfp_t gfp_flags);
extern struct page *fscrypt_alloc_bounce_page(struct fscrypt_ctx *ctx,
gfp_t gfp_flags);
extern const struct dentry_operations fscrypt_d_ops;
extern void __printf(3, 4) __cold
fscrypt_msg(struct super_block *sb, const char *level, const char *fmt, ...);
#define fscrypt_warn(sb, fmt, ...) \
fscrypt_msg(sb, KERN_WARNING, fmt, ##__VA_ARGS__)
#define fscrypt_err(sb, fmt, ...) \
fscrypt_msg(sb, KERN_ERR, fmt, ##__VA_ARGS__)
/* fname.c */
extern int fname_encrypt(struct inode *inode, const struct qstr *iname,

3
fs/crypto/hooks.c
View file

@ -39,7 +39,8 @@ int fscrypt_file_open(struct inode *inode, struct file *filp)
dir = dget_parent(file_dentry(filp));
if (IS_ENCRYPTED(d_inode(dir)) &&
!fscrypt_has_permitted_context(d_inode(dir), inode)) {
pr_warn_ratelimited("fscrypt: inconsistent encryption contexts: %lu/%lu",
fscrypt_warn(inode->i_sb,
"inconsistent encryption contexts: %lu/%lu",
d_inode(dir)->i_ino, inode->i_ino);
err = -EPERM;
}

286
fs/crypto/keyinfo.c
View file

@ -18,17 +18,16 @@
static struct crypto_shash *essiv_hash_tfm;
/**
* derive_key_aes() - Derive a key using AES-128-ECB
* @deriving_key: Encryption key used for derivation.
* @source_key: Source key to which to apply derivation.
* @derived_raw_key: Derived raw key.
/*
* Key derivation function. This generates the derived key by encrypting the
* master key with AES-128-ECB using the inode's nonce as the AES key.
*
* Return: Zero on success; non-zero otherwise.
* The master key must be at least as long as the derived key. If the master
* key is longer, then only the first 'derived_keysize' bytes are used.
*/
static int derive_key_aes(u8 deriving_key[FS_AES_128_ECB_KEY_SIZE],
const struct fscrypt_key *source_key,
u8 derived_raw_key[FS_MAX_KEY_SIZE])
static int derive_key_aes(const u8 *master_key,
const struct fscrypt_context *ctx,
u8 *derived_key, unsigned int derived_keysize)
{
int res = 0;
struct skcipher_request *req = NULL;
@ -50,14 +49,13 @@ static int derive_key_aes(u8 deriving_key[FS_AES_128_ECB_KEY_SIZE],
skcipher_request_set_callback(req,
CRYPTO_TFM_REQ_MAY_BACKLOG | CRYPTO_TFM_REQ_MAY_SLEEP,
crypto_req_done, &wait);
res = crypto_skcipher_setkey(tfm, deriving_key,
FS_AES_128_ECB_KEY_SIZE);
res = crypto_skcipher_setkey(tfm, ctx->nonce, sizeof(ctx->nonce));
if (res < 0)
goto out;
sg_init_one(&src_sg, source_key->raw, source_key->size);
sg_init_one(&dst_sg, derived_raw_key, source_key->size);
skcipher_request_set_crypt(req, &src_sg, &dst_sg, source_key->size,
sg_init_one(&src_sg, master_key, derived_keysize);
sg_init_one(&dst_sg, derived_key, derived_keysize);
skcipher_request_set_crypt(req, &src_sg, &dst_sg, derived_keysize,
NULL);
res = crypto_wait_req(crypto_skcipher_encrypt(req), &wait);
out:
@ -66,103 +64,147 @@ out:
return res;
}
static int validate_user_key(struct fscrypt_info *crypt_info,
struct fscrypt_context *ctx, u8 *raw_key,
const char *prefix, int min_keysize)
/*
* Search the current task's subscribed keyrings for a "logon" key with
* description prefix:descriptor, and if found acquire a read lock on it and
* return a pointer to its validated payload in *payload_ret.
*/
static struct key *
find_and_lock_process_key(const char *prefix,
const u8 descriptor[FS_KEY_DESCRIPTOR_SIZE],
unsigned int min_keysize,
const struct fscrypt_key **payload_ret)
{
char *description;
struct key *keyring_key;
struct fscrypt_key *master_key;
struct key *key;
const struct user_key_payload *ukp;
int res;
const struct fscrypt_key *payload;
description = kasprintf(GFP_NOFS, "%s%*phN", prefix,
FS_KEY_DESCRIPTOR_SIZE,
ctx->master_key_descriptor);
FS_KEY_DESCRIPTOR_SIZE, descriptor);
if (!description)
return -ENOMEM;
return ERR_PTR(-ENOMEM);
keyring_key = request_key(&key_type_logon, description, NULL);
key = request_key(&key_type_logon, description, NULL);
kfree(description);
if (IS_ERR(keyring_key))
return PTR_ERR(keyring_key);
down_read(&keyring_key->sem);
if (IS_ERR(key))
return key;
if (keyring_key->type != &key_type_logon) {
printk_once(KERN_WARNING
"%s: key type must be logon\n", __func__);
res = -ENOKEY;
goto out;
}
ukp = user_key_payload(keyring_key);
if (!ukp) {
/* key was revoked before we acquired its semaphore */
res = -EKEYREVOKED;
goto out;
}
if (ukp->datalen != sizeof(struct fscrypt_key)) {
res = -EINVAL;
goto out;
}
master_key = (struct fscrypt_key *)ukp->data;
BUILD_BUG_ON(FS_AES_128_ECB_KEY_SIZE != FS_KEY_DERIVATION_NONCE_SIZE);
down_read(&key->sem);
ukp = user_key_payload(key);
if (master_key->size < min_keysize || master_key->size > FS_MAX_KEY_SIZE
|| master_key->size % AES_BLOCK_SIZE != 0) {
printk_once(KERN_WARNING
"%s: key size incorrect: %d\n",
__func__, master_key->size);
res = -ENOKEY;
goto out;
}
res = derive_key_aes(ctx->nonce, master_key, raw_key);
out:
up_read(&keyring_key->sem);
key_put(keyring_key);
return res;
if (!ukp) /* was the key revoked before we acquired its semaphore? */
goto invalid;
payload = (const struct fscrypt_key *)ukp->data;
if (ukp->datalen != sizeof(struct fscrypt_key) ||
payload->size < 1 || payload->size > FS_MAX_KEY_SIZE) {
fscrypt_warn(NULL,
"key with description '%s' has invalid payload",
key->description);
goto invalid;
}
static const struct {
if (payload->size < min_keysize) {
fscrypt_warn(NULL,
"key with description '%s' is too short (got %u bytes, need %u+ bytes)",
key->description, payload->size, min_keysize);
goto invalid;
}
*payload_ret = payload;
return key;
invalid:
up_read(&key->sem);
key_put(key);
return ERR_PTR(-ENOKEY);
}
/* Find the master key, then derive the inode's actual encryption key */
static int find_and_derive_key(const struct inode *inode,
const struct fscrypt_context *ctx,
u8 *derived_key, unsigned int derived_keysize)
{
struct key *key;
const struct fscrypt_key *payload;
int err;
key = find_and_lock_process_key(FS_KEY_DESC_PREFIX,
ctx->master_key_descriptor,
derived_keysize, &payload);
if (key == ERR_PTR(-ENOKEY) && inode->i_sb->s_cop->key_prefix) {
key = find_and_lock_process_key(inode->i_sb->s_cop->key_prefix,
ctx->master_key_descriptor,
derived_keysize, &payload);
}
if (IS_ERR(key))
return PTR_ERR(key);
err = derive_key_aes(payload->raw, ctx, derived_key, derived_keysize);
up_read(&key->sem);
key_put(key);
return err;
}
static struct fscrypt_mode {
const char *friendly_name;
const char *cipher_str;
int keysize;
bool logged_impl_name;
} available_modes[] = {
[FS_ENCRYPTION_MODE_AES_256_XTS] = { "xts(aes)",
FS_AES_256_XTS_KEY_SIZE },
[FS_ENCRYPTION_MODE_AES_256_CTS] = { "cts(cbc(aes))",
FS_AES_256_CTS_KEY_SIZE },
[FS_ENCRYPTION_MODE_AES_128_CBC] = { "cbc(aes)",
FS_AES_128_CBC_KEY_SIZE },
[FS_ENCRYPTION_MODE_AES_128_CTS] = { "cts(cbc(aes))",
FS_AES_128_CTS_KEY_SIZE },
[FS_ENCRYPTION_MODE_SPECK128_256_XTS] = { "xts(speck128)", 64 },
[FS_ENCRYPTION_MODE_SPECK128_256_CTS] = { "cts(cbc(speck128))", 32 },
[FS_ENCRYPTION_MODE_AES_256_XTS] = {
.friendly_name = "AES-256-XTS",
.cipher_str = "xts(aes)",
.keysize = 64,
},
[FS_ENCRYPTION_MODE_AES_256_CTS] = {
.friendly_name = "AES-256-CTS-CBC",
.cipher_str = "cts(cbc(aes))",
.keysize = 32,
},
[FS_ENCRYPTION_MODE_AES_128_CBC] = {
.friendly_name = "AES-128-CBC",
.cipher_str = "cbc(aes)",
.keysize = 16,
},
[FS_ENCRYPTION_MODE_AES_128_CTS] = {
.friendly_name = "AES-128-CTS-CBC",
.cipher_str = "cts(cbc(aes))",
.keysize = 16,
},
[FS_ENCRYPTION_MODE_SPECK128_256_XTS] = {
.friendly_name = "Speck128/256-XTS",
.cipher_str = "xts(speck128)",
.keysize = 64,
},
[FS_ENCRYPTION_MODE_SPECK128_256_CTS] = {
.friendly_name = "Speck128/256-CTS-CBC",
.cipher_str = "cts(cbc(speck128))",
.keysize = 32,
},
};
static int determine_cipher_type(struct fscrypt_info *ci, struct inode *inode,
const char **cipher_str_ret, int *keysize_ret)
static struct fscrypt_mode *
select_encryption_mode(const struct fscrypt_info *ci, const struct inode *inode)
{
u32 mode;
if (!fscrypt_valid_enc_modes(ci->ci_data_mode, ci->ci_filename_mode)) {
pr_warn_ratelimited("fscrypt: inode %lu uses unsupported encryption modes (contents mode %d, filenames mode %d)\n",
inode->i_ino,
ci->ci_data_mode, ci->ci_filename_mode);
return -EINVAL;
fscrypt_warn(inode->i_sb,
"inode %lu uses unsupported encryption modes (contents mode %d, filenames mode %d)",
inode->i_ino, ci->ci_data_mode,
ci->ci_filename_mode);
return ERR_PTR(-EINVAL);
}
if (S_ISREG(inode->i_mode)) {
mode = ci->ci_data_mode;
} else if (S_ISDIR(inode->i_mode) || S_ISLNK(inode->i_mode)) {
mode = ci->ci_filename_mode;
} else {
if (S_ISREG(inode->i_mode))
return &available_modes[ci->ci_data_mode];
if (S_ISDIR(inode->i_mode) || S_ISLNK(inode->i_mode))
return &available_modes[ci->ci_filename_mode];
WARN_ONCE(1, "fscrypt: filesystem tried to load encryption info for inode %lu, which is not encryptable (file type %d)\n",
inode->i_ino, (inode->i_mode & S_IFMT));
return -EINVAL;
}
*cipher_str_ret = available_modes[mode].cipher_str;
*keysize_ret = available_modes[mode].keysize;
return 0;
return ERR_PTR(-EINVAL);
}
static void put_crypt_info(struct fscrypt_info *ci)
@ -185,7 +227,8 @@ static int derive_essiv_salt(const u8 *key, int keysize, u8 *salt)
tfm = crypto_alloc_shash("sha256", 0, 0);
if (IS_ERR(tfm)) {
pr_warn_ratelimited("fscrypt: error allocating SHA-256 transform: %ld\n",
fscrypt_warn(NULL,
"error allocating SHA-256 transform: %ld",
PTR_ERR(tfm));
return PTR_ERR(tfm);
}
@ -246,8 +289,7 @@ int fscrypt_get_encryption_info(struct inode *inode)
struct fscrypt_info *crypt_info;
struct fscrypt_context ctx;
struct crypto_skcipher *ctfm;
const char *cipher_str;
int keysize;
struct fscrypt_mode *mode;
u8 *raw_key = NULL;
int res;
@ -291,57 +333,59 @@ int fscrypt_get_encryption_info(struct inode *inode)
memcpy(crypt_info->ci_master_key, ctx.master_key_descriptor,
sizeof(crypt_info->ci_master_key));
res = determine_cipher_type(crypt_info, inode, &cipher_str, &keysize);
if (res)
mode = select_encryption_mode(crypt_info, inode);
if (IS_ERR(mode)) {
res = PTR_ERR(mode);
goto out;
}
/*
* This cannot be a stack buffer because it is passed to the scatterlist
* crypto API as part of key derivation.
*/
res = -ENOMEM;
raw_key = kmalloc(FS_MAX_KEY_SIZE, GFP_NOFS);
raw_key = kmalloc(mode->keysize, GFP_NOFS);
if (!raw_key)
goto out;
res = validate_user_key(crypt_info, &ctx, raw_key, FS_KEY_DESC_PREFIX,
keysize);
if (res && inode->i_sb->s_cop->key_prefix) {
int res2 = validate_user_key(crypt_info, &ctx, raw_key,
inode->i_sb->s_cop->key_prefix,
keysize);
if (res2) {
if (res2 == -ENOKEY)
res = -ENOKEY;
res = find_and_derive_key(inode, &ctx, raw_key, mode->keysize);
if (res)
goto out;
ctfm = crypto_alloc_skcipher(mode->cipher_str, 0, 0);
if (IS_ERR(ctfm)) {
res = PTR_ERR(ctfm);
fscrypt_warn(inode->i_sb,
"error allocating '%s' transform for inode %lu: %d",
mode->cipher_str, inode->i_ino, res);
goto out;
}
} else if (res) {
goto out;
}
ctfm = crypto_alloc_skcipher(cipher_str, 0, 0);
if (!ctfm || IS_ERR(ctfm)) {
res = ctfm ? PTR_ERR(ctfm) : -ENOMEM;
pr_debug("%s: error %d (inode %lu) allocating crypto tfm\n",
__func__, res, inode->i_ino);
goto out;
if (unlikely(!mode->logged_impl_name)) {
/*
* fscrypt performance can vary greatly depending on which
* crypto algorithm implementation is used. Help people debug
* performance problems by logging the ->cra_driver_name the
* first time a mode is used. Note that multiple threads can
* race here, but it doesn't really matter.
*/
mode->logged_impl_name = true;
pr_info("fscrypt: %s using implementation \"%s\"\n",
mode->friendly_name,
crypto_skcipher_alg(ctfm)->base.cra_driver_name);
}
crypt_info->ci_ctfm = ctfm;
crypto_skcipher_clear_flags(ctfm, ~0);
crypto_skcipher_set_flags(ctfm, CRYPTO_TFM_REQ_WEAK_KEY);
/*
* if the provided key is longer than keysize, we use the first
* keysize bytes of the derived key only
*/
res = crypto_skcipher_setkey(ctfm, raw_key, keysize);
res = crypto_skcipher_setkey(ctfm, raw_key, mode->keysize);
if (res)
goto out;
if (S_ISREG(inode->i_mode) &&
crypt_info->ci_data_mode == FS_ENCRYPTION_MODE_AES_128_CBC) {
res = init_essiv_generator(crypt_info, raw_key, keysize);
res = init_essiv_generator(crypt_info, raw_key, mode->keysize);
if (res) {
pr_debug("%s: error %d (inode %lu) allocating essiv tfm\n",
__func__, res, inode->i_ino);
fscrypt_warn(inode->i_sb,
"error initializing ESSIV generator for inode %lu: %d",
inode->i_ino, res);
goto out;
}
}

14
fs/ext4/inode.c
View file

@ -2430,24 +2430,14 @@ static int mpage_prepare_extent_to_map(struct mpage_da_data *mpd)
mpd->map.m_len = 0;
mpd->next_page = index;
while (index <= end) {
nr_pages = pagevec_lookup_tag(&pvec, mapping, &index, tag,
min(end - index, (pgoff_t)PAGEVEC_SIZE-1) + 1);
nr_pages = pagevec_lookup_range_tag(&pvec, mapping, &index, end,
tag);
if (nr_pages == 0)
goto out;
for (i = 0; i < nr_pages; i++) {
struct page *page = pvec.pages[i];
/*
* At this point, the page may be truncated or
* invalidated (changing page->mapping to NULL), or
* even swizzled back from swapper_space to tmpfs file
* mapping. However, page->index will not change
* because we have a reference on the page.
*/
if (page->index > end)
goto out;
/*
* Accumulated enough dirty pages? This doesn't apply
* to WB_SYNC_ALL mode. For integrity sync we have to

163
fs/f2fs/checkpoint.c
View file

@ -24,7 +24,7 @@
#include <trace/events/f2fs.h>
static struct kmem_cache *ino_entry_slab;
struct kmem_cache *inode_entry_slab;
struct kmem_cache *f2fs_inode_entry_slab;
void f2fs_stop_checkpoint(struct f2fs_sb_info *sbi, bool end_io)
{
@ -36,7 +36,7 @@ void f2fs_stop_checkpoint(struct f2fs_sb_info *sbi, bool end_io)
/*
* We guarantee no failure on the returned page.
*/
struct page *grab_meta_page(struct f2fs_sb_info *sbi, pgoff_t index)
struct page *f2fs_grab_meta_page(struct f2fs_sb_info *sbi, pgoff_t index)
{
struct address_space *mapping = META_MAPPING(sbi);
struct page *page = NULL;
@ -100,24 +100,27 @@ repeat:
* readonly and make sure do not write checkpoint with non-uptodate
* meta page.
*/
if (unlikely(!PageUptodate(page)))
if (unlikely(!PageUptodate(page))) {
memset(page_address(page), 0, PAGE_SIZE);
f2fs_stop_checkpoint(sbi, false);
}
out:
return page;
}
struct page *get_meta_page(struct f2fs_sb_info *sbi, pgoff_t index)
struct page *f2fs_get_meta_page(struct f2fs_sb_info *sbi, pgoff_t index)
{
return __get_meta_page(sbi, index, true);
}
/* for POR only */
struct page *get_tmp_page(struct f2fs_sb_info *sbi, pgoff_t index)
struct page *f2fs_get_tmp_page(struct f2fs_sb_info *sbi, pgoff_t index)
{
return __get_meta_page(sbi, index, false);
}
bool is_valid_blkaddr(struct f2fs_sb_info *sbi, block_t blkaddr, int type)
bool f2fs_is_valid_meta_blkaddr(struct f2fs_sb_info *sbi,
block_t blkaddr, int type)
{
switch (type) {
case META_NAT:
@ -151,7 +154,7 @@ bool is_valid_blkaddr(struct f2fs_sb_info *sbi, block_t blkaddr, int type)
/*
* Readahead CP/NAT/SIT/SSA pages
*/
int ra_meta_pages(struct f2fs_sb_info *sbi, block_t start, int nrpages,
int f2fs_ra_meta_pages(struct f2fs_sb_info *sbi, block_t start, int nrpages,
int type, bool sync)
{
struct page *page;
@ -174,7 +177,7 @@ int ra_meta_pages(struct f2fs_sb_info *sbi, block_t start, int nrpages,
blk_start_plug(&plug);
for (; nrpages-- > 0; blkno++) {
if (!is_valid_blkaddr(sbi, blkno, type))
if (!f2fs_is_valid_meta_blkaddr(sbi, blkno, type))
goto out;
switch (type) {
@ -218,7 +221,7 @@ out:
return blkno - start;
}
void ra_meta_pages_cond(struct f2fs_sb_info *sbi, pgoff_t index)
void f2fs_ra_meta_pages_cond(struct f2fs_sb_info *sbi, pgoff_t index)
{
struct page *page;
bool readahead = false;
@ -229,7 +232,7 @@ void ra_meta_pages_cond(struct f2fs_sb_info *sbi, pgoff_t index)
f2fs_put_page(page, 0);
if (readahead)
ra_meta_pages(sbi, index, BIO_MAX_PAGES, META_POR, true);
f2fs_ra_meta_pages(sbi, index, BIO_MAX_PAGES, META_POR, true);
}
static int __f2fs_write_meta_page(struct page *page,
@ -250,7 +253,7 @@ static int __f2fs_write_meta_page(struct page *page,
if (wbc->for_reclaim && page->index < GET_SUM_BLOCK(sbi, 0))
goto redirty_out;
write_meta_page(sbi, page, io_type);
f2fs_do_write_meta_page(sbi, page, io_type);
dec_page_count(sbi, F2FS_DIRTY_META);
if (wbc->for_reclaim)
@ -295,7 +298,7 @@ static int f2fs_write_meta_pages(struct address_space *mapping,
trace_f2fs_writepages(mapping->host, wbc, META);
diff = nr_pages_to_write(sbi, META, wbc);
written = sync_meta_pages(sbi, META, wbc->nr_to_write, FS_META_IO);
written = f2fs_sync_meta_pages(sbi, META, wbc->nr_to_write, FS_META_IO);
mutex_unlock(&sbi->cp_mutex);
wbc->nr_to_write = max((long)0, wbc->nr_to_write - written - diff);
return 0;
@ -306,13 +309,14 @@ skip_write:
return 0;
}
long sync_meta_pages(struct f2fs_sb_info *sbi, enum page_type type,
long f2fs_sync_meta_pages(struct f2fs_sb_info *sbi, enum page_type type,
long nr_to_write, enum iostat_type io_type)
{
struct address_space *mapping = META_MAPPING(sbi);
pgoff_t index = 0, end = ULONG_MAX, prev = ULONG_MAX;
pgoff_t index = 0, prev = ULONG_MAX;
struct pagevec pvec;
long nwritten = 0;
int nr_pages;
struct writeback_control wbc = {
.for_reclaim = 0,
};
@ -322,13 +326,9 @@ long sync_meta_pages(struct f2fs_sb_info *sbi, enum page_type type,
blk_start_plug(&plug);
while (index <= end) {
int i, nr_pages;
nr_pages = pagevec_lookup_tag(&pvec, mapping, &index,
PAGECACHE_TAG_DIRTY,
min(end - index, (pgoff_t)PAGEVEC_SIZE-1) + 1);
if (unlikely(nr_pages == 0))
break;
while ((nr_pages = pagevec_lookup_tag(&pvec, mapping, &index,
PAGECACHE_TAG_DIRTY))) {
int i;
for (i = 0; i < nr_pages; i++) {
struct page *page = pvec.pages[i];
@ -459,20 +459,20 @@ static void __remove_ino_entry(struct f2fs_sb_info *sbi, nid_t ino, int type)
spin_unlock(&im->ino_lock);
}
void add_ino_entry(struct f2fs_sb_info *sbi, nid_t ino, int type)
void f2fs_add_ino_entry(struct f2fs_sb_info *sbi, nid_t ino, int type)
{
/* add new dirty ino entry into list */
__add_ino_entry(sbi, ino, 0, type);
}
void remove_ino_entry(struct f2fs_sb_info *sbi, nid_t ino, int type)
void f2fs_remove_ino_entry(struct f2fs_sb_info *sbi, nid_t ino, int type)
{
/* remove dirty ino entry from list */
__remove_ino_entry(sbi, ino, type);
}
/* mode should be APPEND_INO or UPDATE_INO */
bool exist_written_data(struct f2fs_sb_info *sbi, nid_t ino, int mode)
bool f2fs_exist_written_data(struct f2fs_sb_info *sbi, nid_t ino, int mode)
{
struct inode_management *im = &sbi->im[mode];
struct ino_entry *e;
@ -483,7 +483,7 @@ bool exist_written_data(struct f2fs_sb_info *sbi, nid_t ino, int mode)
return e ? true : false;
}
void release_ino_entry(struct f2fs_sb_info *sbi, bool all)
void f2fs_release_ino_entry(struct f2fs_sb_info *sbi, bool all)
{
struct ino_entry *e, *tmp;
int i;
@ -502,13 +502,13 @@ void release_ino_entry(struct f2fs_sb_info *sbi, bool all)
}
}
void set_dirty_device(struct f2fs_sb_info *sbi, nid_t ino,
void f2fs_set_dirty_device(struct f2fs_sb_info *sbi, nid_t ino,
unsigned int devidx, int type)
{
__add_ino_entry(sbi, ino, devidx, type);
}
bool is_dirty_device(struct f2fs_sb_info *sbi, nid_t ino,
bool f2fs_is_dirty_device(struct f2fs_sb_info *sbi, nid_t ino,
unsigned int devidx, int type)
{
struct inode_management *im = &sbi->im[type];
@ -523,7 +523,7 @@ bool is_dirty_device(struct f2fs_sb_info *sbi, nid_t ino,
return is_dirty;
}
int acquire_orphan_inode(struct f2fs_sb_info *sbi)
int f2fs_acquire_orphan_inode(struct f2fs_sb_info *sbi)
{
struct inode_management *im = &sbi->im[ORPHAN_INO];
int err = 0;
@ -546,7 +546,7 @@ int acquire_orphan_inode(struct f2fs_sb_info *sbi)
return err;
}
void release_orphan_inode(struct f2fs_sb_info *sbi)
void f2fs_release_orphan_inode(struct f2fs_sb_info *sbi)
{
struct inode_management *im = &sbi->im[ORPHAN_INO];
@ -556,14 +556,14 @@ void release_orphan_inode(struct f2fs_sb_info *sbi)
spin_unlock(&im->ino_lock);
}
void add_orphan_inode(struct inode *inode)
void f2fs_add_orphan_inode(struct inode *inode)
{
/* add new orphan ino entry into list */
__add_ino_entry(F2FS_I_SB(inode), inode->i_ino, 0, ORPHAN_INO);
update_inode_page(inode);
f2fs_update_inode_page(inode);
}
void remove_orphan_inode(struct f2fs_sb_info *sbi, nid_t ino)
void f2fs_remove_orphan_inode(struct f2fs_sb_info *sbi, nid_t ino)
{
/* remove orphan entry from orphan list */
__remove_ino_entry(sbi, ino, ORPHAN_INO);
@ -573,7 +573,7 @@ static int recover_orphan_inode(struct f2fs_sb_info *sbi, nid_t ino)
{
struct inode *inode;
struct node_info ni;
int err = acquire_orphan_inode(sbi);
int err = f2fs_acquire_orphan_inode(sbi);
if (err)
goto err_out;
@ -591,16 +591,17 @@ static int recover_orphan_inode(struct f2fs_sb_info *sbi, nid_t ino)
}
err = dquot_initialize(inode);
if (err)
if (err) {
iput(inode);
goto err_out;
}
dquot_initialize(inode);
clear_nlink(inode);
/* truncate all the data during iput */
iput(inode);
get_node_info(sbi, ino, &ni);
f2fs_get_node_info(sbi, ino, &ni);
/* ENOMEM was fully retried in f2fs_evict_inode. */
if (ni.blk_addr != NULL_ADDR) {
@ -618,7 +619,7 @@ err_out:
return err;
}
int recover_orphan_inodes(struct f2fs_sb_info *sbi)
int f2fs_recover_orphan_inodes(struct f2fs_sb_info *sbi)
{
block_t start_blk, orphan_blocks, i, j;
unsigned int s_flags = sbi->sb->s_flags;
@ -646,10 +647,10 @@ int recover_orphan_inodes(struct f2fs_sb_info *sbi)
start_blk = __start_cp_addr(sbi) + 1 + __cp_payload(sbi);
orphan_blocks = __start_sum_addr(sbi) - 1 - __cp_payload(sbi);
ra_meta_pages(sbi, start_blk, orphan_blocks, META_CP, true);
f2fs_ra_meta_pages(sbi, start_blk, orphan_blocks, META_CP, true);
for (i = 0; i < orphan_blocks; i++) {
struct page *page = get_meta_page(sbi, start_blk + i);
struct page *page = f2fs_get_meta_page(sbi, start_blk + i);
struct f2fs_orphan_block *orphan_blk;
orphan_blk = (struct f2fs_orphan_block *)page_address(page);
@ -699,7 +700,7 @@ static void write_orphan_inodes(struct f2fs_sb_info *sbi, block_t start_blk)
/* loop for each orphan inode entry and write them in Jornal block */
list_for_each_entry(orphan, head, list) {
if (!page) {
page = grab_meta_page(sbi, start_blk++);
page = f2fs_grab_meta_page(sbi, start_blk++);
orphan_blk =
(struct f2fs_orphan_block *)page_address(page);
memset(orphan_blk, 0, sizeof(*orphan_blk));
@ -741,7 +742,7 @@ static int get_checkpoint_version(struct f2fs_sb_info *sbi, block_t cp_addr,
size_t crc_offset = 0;
__u32 crc = 0;
*cp_page = get_meta_page(sbi, cp_addr);
*cp_page = f2fs_get_meta_page(sbi, cp_addr);
*cp_block = (struct f2fs_checkpoint *)page_address(*cp_page);
crc_offset = le32_to_cpu((*cp_block)->checksum_offset);
@ -794,7 +795,7 @@ invalid_cp1:
return NULL;
}
int get_valid_checkpoint(struct f2fs_sb_info *sbi)
int f2fs_get_valid_checkpoint(struct f2fs_sb_info *sbi)
{
struct f2fs_checkpoint *cp_block;
struct f2fs_super_block *fsb = sbi->raw_super;
@ -806,7 +807,8 @@ int get_valid_checkpoint(struct f2fs_sb_info *sbi)
block_t cp_blk_no;
int i;
sbi->ckpt = f2fs_kzalloc(sbi, cp_blks * blk_size, GFP_KERNEL);
sbi->ckpt = f2fs_kzalloc(sbi, array_size(blk_size, cp_blks),
GFP_KERNEL);
if (!sbi->ckpt)
return -ENOMEM;
/*
@ -838,7 +840,7 @@ int get_valid_checkpoint(struct f2fs_sb_info *sbi)
memcpy(sbi->ckpt, cp_block, blk_size);
/* Sanity checking of checkpoint */
if (sanity_check_ckpt(sbi))
if (f2fs_sanity_check_ckpt(sbi))
goto free_fail_no_cp;
if (cur_page == cp1)
@ -857,7 +859,7 @@ int get_valid_checkpoint(struct f2fs_sb_info *sbi)
void *sit_bitmap_ptr;
unsigned char *ckpt = (unsigned char *)sbi->ckpt;
cur_page = get_meta_page(sbi, cp_blk_no + i);
cur_page = f2fs_get_meta_page(sbi, cp_blk_no + i);
sit_bitmap_ptr = page_address(cur_page);
memcpy(ckpt + i * blk_size, sit_bitmap_ptr, blk_size);
f2fs_put_page(cur_page, 1);
@ -902,7 +904,7 @@ static void __remove_dirty_inode(struct inode *inode, enum inode_type type)
stat_dec_dirty_inode(F2FS_I_SB(inode), type);
}
void update_dirty_page(struct inode *inode, struct page *page)
void f2fs_update_dirty_page(struct inode *inode, struct page *page)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
enum inode_type type = S_ISDIR(inode->i_mode) ? DIR_INODE : FILE_INODE;
@ -921,7 +923,7 @@ void update_dirty_page(struct inode *inode, struct page *page)
f2fs_trace_pid(page);
}
void remove_dirty_inode(struct inode *inode)
void f2fs_remove_dirty_inode(struct inode *inode)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
enum inode_type type = S_ISDIR(inode->i_mode) ? DIR_INODE : FILE_INODE;
@ -938,7 +940,7 @@ void remove_dirty_inode(struct inode *inode)
spin_unlock(&sbi->inode_lock[type]);
}
int sync_dirty_inodes(struct f2fs_sb_info *sbi, enum inode_type type)
int f2fs_sync_dirty_inodes(struct f2fs_sb_info *sbi, enum inode_type type)
{
struct list_head *head;
struct inode *inode;
@ -1021,7 +1023,7 @@ int f2fs_sync_inode_meta(struct f2fs_sb_info *sbi)
/* it's on eviction */
if (is_inode_flag_set(inode, FI_DIRTY_INODE))
update_inode_page(inode);
f2fs_update_inode_page(inode);
iput(inode);
}
}
@ -1061,7 +1063,7 @@ retry_flush_dents:
/* write all the dirty dentry pages */
if (get_pages(sbi, F2FS_DIRTY_DENTS)) {
f2fs_unlock_all(sbi);
err = sync_dirty_inodes(sbi, DIR_INODE);
err = f2fs_sync_dirty_inodes(sbi, DIR_INODE);
if (err)
goto out;
cond_resched();
@ -1089,7 +1091,9 @@ retry_flush_nodes:
if (get_pages(sbi, F2FS_DIRTY_NODES)) {
up_write(&sbi->node_write);
err = sync_node_pages(sbi, &wbc, false, FS_CP_NODE_IO);
atomic_inc(&sbi->wb_sync_req[NODE]);
err = f2fs_sync_node_pages(sbi, &wbc, false, FS_CP_NODE_IO);
atomic_dec(&sbi->wb_sync_req[NODE]);
if (err) {
up_write(&sbi->node_change);
f2fs_unlock_all(sbi);
@ -1183,10 +1187,10 @@ static void commit_checkpoint(struct f2fs_sb_info *sbi,
/*
* pagevec_lookup_tag and lock_page again will take
* some extra time. Therefore, update_meta_pages and
* sync_meta_pages are combined in this function.
* some extra time. Therefore, f2fs_update_meta_pages and
* f2fs_sync_meta_pages are combined in this function.
*/
struct page *page = grab_meta_page(sbi, blk_addr);
struct page *page = f2fs_grab_meta_page(sbi, blk_addr);
int err;
memcpy(page_address(page), src, PAGE_SIZE);
@ -1224,7 +1228,7 @@ static int do_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc)
/* Flush all the NAT/SIT pages */
while (get_pages(sbi, F2FS_DIRTY_META)) {
sync_meta_pages(sbi, META, LONG_MAX, FS_CP_META_IO);
f2fs_sync_meta_pages(sbi, META, LONG_MAX, FS_CP_META_IO);
if (unlikely(f2fs_cp_error(sbi)))
return -EIO;
}
@ -1233,7 +1237,7 @@ static int do_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc)
* modify checkpoint
* version number is already updated
*/
ckpt->elapsed_time = cpu_to_le64(get_mtime(sbi));
ckpt->elapsed_time = cpu_to_le64(get_mtime(sbi, true));
ckpt->free_segment_count = cpu_to_le32(free_segments(sbi));
for (i = 0; i < NR_CURSEG_NODE_TYPE; i++) {
ckpt->cur_node_segno[i] =
@ -1253,7 +1257,7 @@ static int do_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc)
}
/* 2 cp + n data seg summary + orphan inode blocks */
data_sum_blocks = npages_for_summary_flush(sbi, false);
data_sum_blocks = f2fs_npages_for_summary_flush(sbi, false);
spin_lock_irqsave(&sbi->cp_lock, flags);
if (data_sum_blocks < NR_CURSEG_DATA_TYPE)
__set_ckpt_flags(ckpt, CP_COMPACT_SUM_FLAG);
@ -1298,22 +1302,23 @@ static int do_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc)
blk = start_blk + sbi->blocks_per_seg - nm_i->nat_bits_blocks;
for (i = 0; i < nm_i->nat_bits_blocks; i++)
update_meta_page(sbi, nm_i->nat_bits +
f2fs_update_meta_page(sbi, nm_i->nat_bits +
(i << F2FS_BLKSIZE_BITS), blk + i);
/* Flush all the NAT BITS pages */
while (get_pages(sbi, F2FS_DIRTY_META)) {
sync_meta_pages(sbi, META, LONG_MAX, FS_CP_META_IO);
f2fs_sync_meta_pages(sbi, META, LONG_MAX,
FS_CP_META_IO);
if (unlikely(f2fs_cp_error(sbi)))
return -EIO;
}
}
/* write out checkpoint buffer at block 0 */
update_meta_page(sbi, ckpt, start_blk++);
f2fs_update_meta_page(sbi, ckpt, start_blk++);
for (i = 1; i < 1 + cp_payload_blks; i++)
update_meta_page(sbi, (char *)ckpt + i * F2FS_BLKSIZE,
f2fs_update_meta_page(sbi, (char *)ckpt + i * F2FS_BLKSIZE,
start_blk++);
if (orphan_num) {
@ -1321,7 +1326,7 @@ static int do_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc)
start_blk += orphan_blocks;
}
write_data_summaries(sbi, start_blk);
f2fs_write_data_summaries(sbi, start_blk);
start_blk += data_sum_blocks;
/* Record write statistics in the hot node summary */
@ -1332,7 +1337,7 @@ static int do_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc)
seg_i->journal->info.kbytes_written = cpu_to_le64(kbytes_written);
if (__remain_node_summaries(cpc->reason)) {
write_node_summaries(sbi, start_blk);
f2fs_write_node_summaries(sbi, start_blk);
start_blk += NR_CURSEG_NODE_TYPE;
}
@ -1341,7 +1346,7 @@ static int do_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc)
percpu_counter_set(&sbi->alloc_valid_block_count, 0);
/* Here, we have one bio having CP pack except cp pack 2 page */
sync_meta_pages(sbi, META, LONG_MAX, FS_CP_META_IO);
f2fs_sync_meta_pages(sbi, META, LONG_MAX, FS_CP_META_IO);
/* wait for previous submitted meta pages writeback */
wait_on_all_pages_writeback(sbi);
@ -1358,7 +1363,7 @@ static int do_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc)
commit_checkpoint(sbi, ckpt, start_blk);
wait_on_all_pages_writeback(sbi);
release_ino_entry(sbi, false);
f2fs_release_ino_entry(sbi, false);
if (unlikely(f2fs_cp_error(sbi)))
return -EIO;
@ -1383,7 +1388,7 @@ static int do_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc)
/*
* We guarantee that this checkpoint procedure will not fail.
*/
int write_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc)
int f2fs_write_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc)
{
struct f2fs_checkpoint *ckpt = F2FS_CKPT(sbi);
unsigned long long ckpt_ver;
@ -1416,7 +1421,7 @@ int write_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc)
/* this is the case of multiple fstrims without any changes */
if (cpc->reason & CP_DISCARD) {
if (!exist_trim_candidates(sbi, cpc)) {
if (!f2fs_exist_trim_candidates(sbi, cpc)) {
unblock_operations(sbi);
goto out;
}
@ -1424,8 +1429,8 @@ int write_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc)
if (NM_I(sbi)->dirty_nat_cnt == 0 &&
SIT_I(sbi)->dirty_sentries == 0 &&
prefree_segments(sbi) == 0) {
flush_sit_entries(sbi, cpc);
clear_prefree_segments(sbi, cpc);
f2fs_flush_sit_entries(sbi, cpc);
f2fs_clear_prefree_segments(sbi, cpc);
unblock_operations(sbi);
goto out;
}
@ -1440,15 +1445,15 @@ int write_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc)
ckpt->checkpoint_ver = cpu_to_le64(++ckpt_ver);
/* write cached NAT/SIT entries to NAT/SIT area */
flush_nat_entries(sbi, cpc);
flush_sit_entries(sbi, cpc);
f2fs_flush_nat_entries(sbi, cpc);
f2fs_flush_sit_entries(sbi, cpc);
/* unlock all the fs_lock[] in do_checkpoint() */
err = do_checkpoint(sbi, cpc);
if (err)
release_discard_addrs(sbi);
f2fs_release_discard_addrs(sbi);
else
clear_prefree_segments(sbi, cpc);
f2fs_clear_prefree_segments(sbi, cpc);
unblock_operations(sbi);
stat_inc_cp_count(sbi->stat_info);
@ -1465,7 +1470,7 @@ out:
return err;
}
void init_ino_entry_info(struct f2fs_sb_info *sbi)
void f2fs_init_ino_entry_info(struct f2fs_sb_info *sbi)
{
int i;
@ -1483,23 +1488,23 @@ void init_ino_entry_info(struct f2fs_sb_info *sbi)
F2FS_ORPHANS_PER_BLOCK;
}
int __init create_checkpoint_caches(void)
int __init f2fs_create_checkpoint_caches(void)
{
ino_entry_slab = f2fs_kmem_cache_create("f2fs_ino_entry",
sizeof(struct ino_entry));
if (!ino_entry_slab)
return -ENOMEM;
inode_entry_slab = f2fs_kmem_cache_create("f2fs_inode_entry",
f2fs_inode_entry_slab = f2fs_kmem_cache_create("f2fs_inode_entry",
sizeof(struct inode_entry));
if (!inode_entry_slab) {
if (!f2fs_inode_entry_slab) {
kmem_cache_destroy(ino_entry_slab);
return -ENOMEM;
}
return 0;
}
void destroy_checkpoint_caches(void)
void f2fs_destroy_checkpoint_caches(void)
{
kmem_cache_destroy(ino_entry_slab);
kmem_cache_destroy(inode_entry_slab);
kmem_cache_destroy(f2fs_inode_entry_slab);
}

191
fs/f2fs/data.c
View file

@ -48,6 +48,8 @@ static bool __is_cp_guaranteed(struct page *page)
if (inode->i_ino == F2FS_META_INO(sbi) ||
inode->i_ino == F2FS_NODE_INO(sbi) ||
S_ISDIR(inode->i_mode) ||
(S_ISREG(inode->i_mode) &&
is_inode_flag_set(inode, FI_ATOMIC_FILE)) ||
is_cold_data(page))
return true;
return false;
@ -244,7 +246,7 @@ static struct bio *__bio_alloc(struct f2fs_sb_info *sbi, block_t blk_addr,
} else {
bio->bi_end_io = f2fs_write_end_io;
bio->bi_private = sbi;
bio->bi_write_hint = io_type_to_rw_hint(sbi, type, temp);
bio->bi_write_hint = f2fs_io_type_to_rw_hint(sbi, type, temp);
}
if (wbc)
wbc_init_bio(wbc, bio);
@ -459,13 +461,12 @@ int f2fs_submit_page_bio(struct f2fs_io_info *fio)
return 0;
}
int f2fs_submit_page_write(struct f2fs_io_info *fio)
void f2fs_submit_page_write(struct f2fs_io_info *fio)
{
struct f2fs_sb_info *sbi = fio->sbi;
enum page_type btype = PAGE_TYPE_OF_BIO(fio->type);
struct f2fs_bio_info *io = sbi->write_io[btype] + fio->temp;
struct page *bio_page;
int err = 0;
f2fs_bug_on(sbi, is_read_io(fio->op));
@ -475,7 +476,7 @@ next:
spin_lock(&io->io_lock);
if (list_empty(&io->io_list)) {
spin_unlock(&io->io_lock);
goto out_fail;
goto out;
}
fio = list_first_entry(&io->io_list,
struct f2fs_io_info, list);
@ -483,7 +484,7 @@ next:
spin_unlock(&io->io_lock);
}
if (fio->old_blkaddr != NEW_ADDR)
if (is_valid_blkaddr(fio->old_blkaddr))
verify_block_addr(fio, fio->old_blkaddr);
verify_block_addr(fio, fio->new_blkaddr);
@ -502,9 +503,9 @@ alloc_new:
if (io->bio == NULL) {
if ((fio->type == DATA || fio->type == NODE) &&
fio->new_blkaddr & F2FS_IO_SIZE_MASK(sbi)) {
err = -EAGAIN;
dec_page_count(sbi, WB_DATA_TYPE(bio_page));
goto out_fail;
fio->retry = true;
goto skip;
}
io->bio = __bio_alloc(sbi, fio->new_blkaddr, fio->io_wbc,
BIO_MAX_PAGES, false,
@ -524,12 +525,11 @@ alloc_new:
f2fs_trace_ios(fio, 0);
trace_f2fs_submit_page_write(fio->page, fio);
skip:
if (fio->in_list)
goto next;
out_fail:
out:
up_write(&io->io_rwsem);
return err;
}
static struct bio *f2fs_grab_read_bio(struct inode *inode, block_t blkaddr,
@ -603,7 +603,7 @@ static void __set_data_blkaddr(struct dnode_of_data *dn)
* ->node_page
* update block addresses in the node page
*/
void set_data_blkaddr(struct dnode_of_data *dn)
void f2fs_set_data_blkaddr(struct dnode_of_data *dn)
{
f2fs_wait_on_page_writeback(dn->node_page, NODE, true);
__set_data_blkaddr(dn);
@ -614,12 +614,12 @@ void set_data_blkaddr(struct dnode_of_data *dn)
void f2fs_update_data_blkaddr(struct dnode_of_data *dn, block_t blkaddr)
{
dn->data_blkaddr = blkaddr;
set_data_blkaddr(dn);
f2fs_set_data_blkaddr(dn);
f2fs_update_extent_cache(dn);
}
/* dn->ofs_in_node will be returned with up-to-date last block pointer */
int reserve_new_blocks(struct dnode_of_data *dn, blkcnt_t count)
int f2fs_reserve_new_blocks(struct dnode_of_data *dn, blkcnt_t count)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(dn->inode);
int err;
@ -653,12 +653,12 @@ int reserve_new_blocks(struct dnode_of_data *dn, blkcnt_t count)
}
/* Should keep dn->ofs_in_node unchanged */
int reserve_new_block(struct dnode_of_data *dn)
int f2fs_reserve_new_block(struct dnode_of_data *dn)
{
unsigned int ofs_in_node = dn->ofs_in_node;
int ret;
ret = reserve_new_blocks(dn, 1);
ret = f2fs_reserve_new_blocks(dn, 1);
dn->ofs_in_node = ofs_in_node;
return ret;
}
@ -668,12 +668,12 @@ int f2fs_reserve_block(struct dnode_of_data *dn, pgoff_t index)
bool need_put = dn->inode_page ? false : true;
int err;
err = get_dnode_of_data(dn, index, ALLOC_NODE);
err = f2fs_get_dnode_of_data(dn, index, ALLOC_NODE);
if (err)
return err;
if (dn->data_blkaddr == NULL_ADDR)
err = reserve_new_block(dn);
err = f2fs_reserve_new_block(dn);
if (err || need_put)
f2fs_put_dnode(dn);
return err;
@ -692,7 +692,7 @@ int f2fs_get_block(struct dnode_of_data *dn, pgoff_t index)
return f2fs_reserve_block(dn, index);
}
struct page *get_read_data_page(struct inode *inode, pgoff_t index,
struct page *f2fs_get_read_data_page(struct inode *inode, pgoff_t index,
int op_flags, bool for_write)
{
struct address_space *mapping = inode->i_mapping;
@ -711,7 +711,7 @@ struct page *get_read_data_page(struct inode *inode, pgoff_t index,
}
set_new_dnode(&dn, inode, NULL, NULL, 0);
err = get_dnode_of_data(&dn, index, LOOKUP_NODE);
err = f2fs_get_dnode_of_data(&dn, index, LOOKUP_NODE);
if (err)
goto put_err;
f2fs_put_dnode(&dn);
@ -730,7 +730,8 @@ got_it:
* A new dentry page is allocated but not able to be written, since its
* new inode page couldn't be allocated due to -ENOSPC.
* In such the case, its blkaddr can be remained as NEW_ADDR.
* see, f2fs_add_link -> get_new_data_page -> init_inode_metadata.
* see, f2fs_add_link -> f2fs_get_new_data_page ->
* f2fs_init_inode_metadata.
*/
if (dn.data_blkaddr == NEW_ADDR) {
zero_user_segment(page, 0, PAGE_SIZE);
@ -750,7 +751,7 @@ put_err:
return ERR_PTR(err);
}
struct page *find_data_page(struct inode *inode, pgoff_t index)
struct page *f2fs_find_data_page(struct inode *inode, pgoff_t index)
{
struct address_space *mapping = inode->i_mapping;
struct page *page;
@ -760,7 +761,7 @@ struct page *find_data_page(struct inode *inode, pgoff_t index)
return page;
f2fs_put_page(page, 0);
page = get_read_data_page(inode, index, REQ_SYNC, false);
page = f2fs_get_read_data_page(inode, index, REQ_SYNC, false);
if (IS_ERR(page))
return page;
@ -780,13 +781,13 @@ struct page *find_data_page(struct inode *inode, pgoff_t index)
* Because, the callers, functions in dir.c and GC, should be able to know
* whether this page exists or not.
*/
struct page *get_lock_data_page(struct inode *inode, pgoff_t index,
struct page *f2fs_get_lock_data_page(struct inode *inode, pgoff_t index,
bool for_write)
{
struct address_space *mapping = inode->i_mapping;
struct page *page;
repeat:
page = get_read_data_page(inode, index, REQ_SYNC, for_write);
page = f2fs_get_read_data_page(inode, index, REQ_SYNC, for_write);
if (IS_ERR(page))
return page;
@ -812,7 +813,7 @@ repeat:
* Note that, ipage is set only by make_empty_dir, and if any error occur,
* ipage should be released by this function.
*/
struct page *get_new_data_page(struct inode *inode,
struct page *f2fs_get_new_data_page(struct inode *inode,
struct page *ipage, pgoff_t index, bool new_i_size)
{
struct address_space *mapping = inode->i_mapping;
@ -851,7 +852,7 @@ struct page *get_new_data_page(struct inode *inode,
/* if ipage exists, blkaddr should be NEW_ADDR */
f2fs_bug_on(F2FS_I_SB(inode), ipage);
page = get_lock_data_page(inode, index, true);
page = f2fs_get_lock_data_page(inode, index, true);
if (IS_ERR(page))
return page;
}
@ -883,15 +884,15 @@ static int __allocate_data_block(struct dnode_of_data *dn, int seg_type)
return err;
alloc:
get_node_info(sbi, dn->nid, &ni);
f2fs_get_node_info(sbi, dn->nid, &ni);
set_summary(&sum, dn->nid, dn->ofs_in_node, ni.version);
allocate_data_block(sbi, NULL, dn->data_blkaddr, &dn->data_blkaddr,
f2fs_allocate_data_block(sbi, NULL, dn->data_blkaddr, &dn->data_blkaddr,
&sum, seg_type, NULL, false);
set_data_blkaddr(dn);
f2fs_set_data_blkaddr(dn);
/* update i_size */
fofs = start_bidx_of_node(ofs_of_node(dn->node_page), dn->inode) +
fofs = f2fs_start_bidx_of_node(ofs_of_node(dn->node_page), dn->inode) +
dn->ofs_in_node;
if (i_size_read(dn->inode) < ((loff_t)(fofs + 1) << PAGE_SHIFT))
f2fs_i_size_write(dn->inode,
@ -929,7 +930,7 @@ int f2fs_preallocate_blocks(struct kiocb *iocb, struct iov_iter *from)
map.m_seg_type = NO_CHECK_TYPE;
if (direct_io) {
map.m_seg_type = rw_hint_to_seg_type(iocb->ki_hint);
map.m_seg_type = f2fs_rw_hint_to_seg_type(iocb->ki_hint);
flag = f2fs_force_buffered_io(inode, WRITE) ?
F2FS_GET_BLOCK_PRE_AIO :
F2FS_GET_BLOCK_PRE_DIO;
@ -1019,7 +1020,7 @@ next_dnode:
/* When reading holes, we need its node page */
set_new_dnode(&dn, inode, NULL, NULL, 0);
err = get_dnode_of_data(&dn, pgofs, mode);
err = f2fs_get_dnode_of_data(&dn, pgofs, mode);
if (err) {
if (flag == F2FS_GET_BLOCK_BMAP)
map->m_pblk = 0;
@ -1027,10 +1028,10 @@ next_dnode:
err = 0;
if (map->m_next_pgofs)
*map->m_next_pgofs =
get_next_page_offset(&dn, pgofs);
f2fs_get_next_page_offset(&dn, pgofs);
if (map->m_next_extent)
*map->m_next_extent =
get_next_page_offset(&dn, pgofs);
f2fs_get_next_page_offset(&dn, pgofs);
}
goto unlock_out;
}
@ -1043,7 +1044,7 @@ next_dnode:
next_block:
blkaddr = datablock_addr(dn.inode, dn.node_page, dn.ofs_in_node);
if (blkaddr == NEW_ADDR || blkaddr == NULL_ADDR) {
if (!is_valid_blkaddr(blkaddr)) {
if (create) {
if (unlikely(f2fs_cp_error(sbi))) {
err = -EIO;
@ -1116,7 +1117,7 @@ skip:
(pgofs == end || dn.ofs_in_node == end_offset)) {
dn.ofs_in_node = ofs_in_node;
err = reserve_new_blocks(&dn, prealloc);
err = f2fs_reserve_new_blocks(&dn, prealloc);
if (err)
goto sync_out;
@ -1235,7 +1236,7 @@ static int get_data_block_dio(struct inode *inode, sector_t iblock,
{
return __get_data_block(inode, iblock, bh_result, create,
F2FS_GET_BLOCK_DEFAULT, NULL,
rw_hint_to_seg_type(
f2fs_rw_hint_to_seg_type(
inode->i_write_hint));
}
@ -1280,7 +1281,7 @@ static int f2fs_xattr_fiemap(struct inode *inode,
if (!page)
return -ENOMEM;
get_node_info(sbi, inode->i_ino, &ni);
f2fs_get_node_info(sbi, inode->i_ino, &ni);
phys = (__u64)blk_to_logical(inode, ni.blk_addr);
offset = offsetof(struct f2fs_inode, i_addr) +
@ -1307,7 +1308,7 @@ static int f2fs_xattr_fiemap(struct inode *inode,
if (!page)
return -ENOMEM;
get_node_info(sbi, xnid, &ni);
f2fs_get_node_info(sbi, xnid, &ni);
phys = (__u64)blk_to_logical(inode, ni.blk_addr);
len = inode->i_sb->s_blocksize;
@ -1610,12 +1611,12 @@ static inline bool check_inplace_update_policy(struct inode *inode,
if (policy & (0x1 << F2FS_IPU_FORCE))
return true;
if (policy & (0x1 << F2FS_IPU_SSR) && need_SSR(sbi))
if (policy & (0x1 << F2FS_IPU_SSR) && f2fs_need_SSR(sbi))
return true;
if (policy & (0x1 << F2FS_IPU_UTIL) &&
utilization(sbi) > SM_I(sbi)->min_ipu_util)
return true;
if (policy & (0x1 << F2FS_IPU_SSR_UTIL) && need_SSR(sbi) &&
if (policy & (0x1 << F2FS_IPU_SSR_UTIL) && f2fs_need_SSR(sbi) &&
utilization(sbi) > SM_I(sbi)->min_ipu_util)
return true;
@ -1636,7 +1637,7 @@ static inline bool check_inplace_update_policy(struct inode *inode,
return false;
}
bool should_update_inplace(struct inode *inode, struct f2fs_io_info *fio)
bool f2fs_should_update_inplace(struct inode *inode, struct f2fs_io_info *fio)
{
if (f2fs_is_pinned_file(inode))
return true;
@ -1648,7 +1649,7 @@ bool should_update_inplace(struct inode *inode, struct f2fs_io_info *fio)
return check_inplace_update_policy(inode, fio);
}
bool should_update_outplace(struct inode *inode, struct f2fs_io_info *fio)
bool f2fs_should_update_outplace(struct inode *inode, struct f2fs_io_info *fio)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
@ -1671,22 +1672,13 @@ static inline bool need_inplace_update(struct f2fs_io_info *fio)
{
struct inode *inode = fio->page->mapping->host;
if (should_update_outplace(inode, fio))
if (f2fs_should_update_outplace(inode, fio))
return false;
return should_update_inplace(inode, fio);
return f2fs_should_update_inplace(inode, fio);
}
static inline bool valid_ipu_blkaddr(struct f2fs_io_info *fio)
{
if (fio->old_blkaddr == NEW_ADDR)
return false;
if (fio->old_blkaddr == NULL_ADDR)
return false;
return true;
}
int do_write_data_page(struct f2fs_io_info *fio)
int f2fs_do_write_data_page(struct f2fs_io_info *fio)
{
struct page *page = fio->page;
struct inode *inode = page->mapping->host;
@ -1700,7 +1692,7 @@ int do_write_data_page(struct f2fs_io_info *fio)
f2fs_lookup_extent_cache(inode, page->index, &ei)) {
fio->old_blkaddr = ei.blk + page->index - ei.fofs;
if (valid_ipu_blkaddr(fio)) {
if (is_valid_blkaddr(fio->old_blkaddr)) {
ipu_force = true;
fio->need_lock = LOCK_DONE;
goto got_it;
@ -1711,7 +1703,7 @@ int do_write_data_page(struct f2fs_io_info *fio)
if (fio->need_lock == LOCK_REQ && !f2fs_trylock_op(fio->sbi))
return -EAGAIN;
err = get_dnode_of_data(&dn, page->index, LOOKUP_NODE);
err = f2fs_get_dnode_of_data(&dn, page->index, LOOKUP_NODE);
if (err)
goto out;
@ -1727,7 +1719,8 @@ got_it:
* If current allocation needs SSR,
* it had better in-place writes for updated data.
*/
if (ipu_force || (valid_ipu_blkaddr(fio) && need_inplace_update(fio))) {
if (ipu_force || (is_valid_blkaddr(fio->old_blkaddr) &&
need_inplace_update(fio))) {
err = encrypt_one_page(fio);
if (err)
goto out_writepage;
@ -1737,7 +1730,7 @@ got_it:
f2fs_put_dnode(&dn);
if (fio->need_lock == LOCK_REQ)
f2fs_unlock_op(fio->sbi);
err = rewrite_data_page(fio);
err = f2fs_inplace_write_data(fio);
trace_f2fs_do_write_data_page(fio->page, IPU);
set_inode_flag(inode, FI_UPDATE_WRITE);
return err;
@ -1759,7 +1752,7 @@ got_it:
ClearPageError(page);
/* LFS mode write path */
write_data_page(&dn, fio);
f2fs_outplace_write_data(&dn, fio);
trace_f2fs_do_write_data_page(page, OPU);
set_inode_flag(inode, FI_APPEND_WRITE);
if (page->index == 0)
@ -1805,6 +1798,12 @@ static int __write_data_page(struct page *page, bool *submitted,
/* we should bypass data pages to proceed the kworkder jobs */
if (unlikely(f2fs_cp_error(sbi))) {
mapping_set_error(page->mapping, -EIO);
/*
* don't drop any dirty dentry pages for keeping lastest
* directory structure.
*/
if (S_ISDIR(inode->i_mode))
goto redirty_out;
goto out;
}
@ -1829,13 +1828,13 @@ write:
/* we should not write 0'th page having journal header */
if (f2fs_is_volatile_file(inode) && (!page->index ||
(!wbc->for_reclaim &&
available_free_memory(sbi, BASE_CHECK))))
f2fs_available_free_memory(sbi, BASE_CHECK))))
goto redirty_out;
/* Dentry blocks are controlled by checkpoint */
if (S_ISDIR(inode->i_mode)) {
fio.need_lock = LOCK_DONE;
err = do_write_data_page(&fio);
err = f2fs_do_write_data_page(&fio);
goto done;
}
@ -1854,10 +1853,10 @@ write:
}
if (err == -EAGAIN) {
err = do_write_data_page(&fio);
err = f2fs_do_write_data_page(&fio);
if (err == -EAGAIN) {
fio.need_lock = LOCK_REQ;
err = do_write_data_page(&fio);
err = f2fs_do_write_data_page(&fio);
}
}
@ -1882,7 +1881,7 @@ out:
if (wbc->for_reclaim) {
f2fs_submit_merged_write_cond(sbi, inode, 0, page->index, DATA);
clear_inode_flag(inode, FI_HOT_DATA);
remove_dirty_inode(inode);
f2fs_remove_dirty_inode(inode);
submitted = NULL;
}
@ -1932,6 +1931,7 @@ static int f2fs_write_cache_pages(struct address_space *mapping,
int ret = 0;
int done = 0;
struct pagevec pvec;
struct f2fs_sb_info *sbi = F2FS_M_SB(mapping);
int nr_pages;
pgoff_t uninitialized_var(writeback_index);
pgoff_t index;
@ -1976,8 +1976,8 @@ retry:
while (!done && (index <= end)) {
int i;
nr_pages = pagevec_lookup_tag(&pvec, mapping, &index, tag,
min(end - index, (pgoff_t)PAGEVEC_SIZE - 1) + 1);
nr_pages = pagevec_lookup_range_tag(&pvec, mapping, &index, end,
tag);
if (nr_pages == 0)
break;
@ -1985,7 +1985,9 @@ retry:
struct page *page = pvec.pages[i];
bool submitted = false;
if (page->index > end) {
/* give a priority to WB_SYNC threads */
if (atomic_read(&sbi->wb_sync_req[DATA]) &&
wbc->sync_mode == WB_SYNC_NONE) {
done = 1;
break;
}
@ -2044,9 +2046,7 @@ continue_unlock:
last_idx = page->index;
}
/* give a priority to WB_SYNC threads */
if ((atomic_read(&F2FS_M_SB(mapping)->wb_sync_req) ||
--wbc->nr_to_write <= 0) &&
if (--wbc->nr_to_write <= 0 &&
wbc->sync_mode == WB_SYNC_NONE) {
done = 1;
break;
@ -2072,7 +2072,7 @@ continue_unlock:
return ret;
}
int __f2fs_write_data_pages(struct address_space *mapping,
static int __f2fs_write_data_pages(struct address_space *mapping,
struct writeback_control *wbc,
enum iostat_type io_type)
{
@ -2095,7 +2095,7 @@ int __f2fs_write_data_pages(struct address_space *mapping,
if (S_ISDIR(inode->i_mode) && wbc->sync_mode == WB_SYNC_NONE &&
get_dirty_pages(inode) < nr_pages_to_skip(sbi, DATA) &&
available_free_memory(sbi, DIRTY_DENTS))
f2fs_available_free_memory(sbi, DIRTY_DENTS))
goto skip_write;
/* skip writing during file defragment */
@ -2106,8 +2106,8 @@ int __f2fs_write_data_pages(struct address_space *mapping,
/* to avoid spliting IOs due to mixed WB_SYNC_ALL and WB_SYNC_NONE */
if (wbc->sync_mode == WB_SYNC_ALL)
atomic_inc(&sbi->wb_sync_req);
else if (atomic_read(&sbi->wb_sync_req))
atomic_inc(&sbi->wb_sync_req[DATA]);
else if (atomic_read(&sbi->wb_sync_req[DATA]))
goto skip_write;
blk_start_plug(&plug);
@ -2115,13 +2115,13 @@ int __f2fs_write_data_pages(struct address_space *mapping,
blk_finish_plug(&plug);
if (wbc->sync_mode == WB_SYNC_ALL)
atomic_dec(&sbi->wb_sync_req);
atomic_dec(&sbi->wb_sync_req[DATA]);
/*
* if some pages were truncated, we cannot guarantee its mapping->host
* to detect pending bios.
*/
remove_dirty_inode(inode);
f2fs_remove_dirty_inode(inode);
return ret;
skip_write:
@ -2148,7 +2148,7 @@ static void f2fs_write_failed(struct address_space *mapping, loff_t to)
if (to > i_size) {
down_write(&F2FS_I(inode)->i_mmap_sem);
truncate_pagecache(inode, i_size);
truncate_blocks(inode, i_size, true);
f2fs_truncate_blocks(inode, i_size, true);
up_write(&F2FS_I(inode)->i_mmap_sem);
}
}
@ -2180,7 +2180,7 @@ static int prepare_write_begin(struct f2fs_sb_info *sbi,
}
restart:
/* check inline_data */
ipage = get_node_page(sbi, inode->i_ino);
ipage = f2fs_get_node_page(sbi, inode->i_ino);
if (IS_ERR(ipage)) {
err = PTR_ERR(ipage);
goto unlock_out;
@ -2190,7 +2190,7 @@ restart:
if (f2fs_has_inline_data(inode)) {
if (pos + len <= MAX_INLINE_DATA(inode)) {
read_inline_data(page, ipage);
f2fs_do_read_inline_data(page, ipage);
set_inode_flag(inode, FI_DATA_EXIST);
if (inode->i_nlink)
set_inline_node(ipage);
@ -2208,7 +2208,7 @@ restart:
dn.data_blkaddr = ei.blk + index - ei.fofs;
} else {
/* hole case */
err = get_dnode_of_data(&dn, index, LOOKUP_NODE);
err = f2fs_get_dnode_of_data(&dn, index, LOOKUP_NODE);
if (err || dn.data_blkaddr == NULL_ADDR) {
f2fs_put_dnode(&dn);
__do_map_lock(sbi, F2FS_GET_BLOCK_PRE_AIO,
@ -2255,7 +2255,7 @@ static int f2fs_write_begin(struct file *file, struct address_space *mapping,
trace_f2fs_write_begin(inode, pos, len, flags);
if (f2fs_is_atomic_file(inode) &&
!available_free_memory(sbi, INMEM_PAGES)) {
!f2fs_available_free_memory(sbi, INMEM_PAGES)) {
err = -ENOMEM;
drop_atomic = true;
goto fail;
@ -2339,7 +2339,7 @@ fail:
f2fs_put_page(page, 1);
f2fs_write_failed(mapping, pos + len);
if (drop_atomic)
drop_inmem_pages_all(sbi);
f2fs_drop_inmem_pages_all(sbi, false);
return err;
}
@ -2437,17 +2437,17 @@ static ssize_t f2fs_direct_IO(struct kiocb *iocb, struct iov_iter *iter,
if (rw == WRITE && whint_mode == WHINT_MODE_OFF)
iocb->ki_hint = WRITE_LIFE_NOT_SET;
if (!down_read_trylock(&F2FS_I(inode)->dio_rwsem[rw])) {
if (!down_read_trylock(&F2FS_I(inode)->i_gc_rwsem[rw])) {
if (iocb->ki_flags & IOCB_NOWAIT) {
iocb->ki_hint = hint;
err = -EAGAIN;
goto out;
}
down_read(&F2FS_I(inode)->dio_rwsem[rw]);
down_read(&F2FS_I(inode)->i_gc_rwsem[rw]);
}
err = blockdev_direct_IO(iocb, inode, iter, offset, get_data_block_dio);
up_read(&F2FS_I(inode)->dio_rwsem[rw]);
up_read(&F2FS_I(inode)->i_gc_rwsem[rw]);
if (rw == WRITE) {
if (whint_mode == WHINT_MODE_OFF)
@ -2490,13 +2490,13 @@ void f2fs_invalidate_page(struct page *page, unsigned int offset,
dec_page_count(sbi, F2FS_DIRTY_NODES);
} else {
inode_dec_dirty_pages(inode);
remove_dirty_inode(inode);
f2fs_remove_dirty_inode(inode);
}
}
/* This is atomic written page, keep Private */
if (IS_ATOMIC_WRITTEN_PAGE(page))
return drop_inmem_page(inode, page);
return f2fs_drop_inmem_page(inode, page);
set_page_private(page, 0);
ClearPagePrivate(page);
@ -2529,7 +2529,7 @@ static int f2fs_set_data_page_dirty(struct page *page)
if (f2fs_is_atomic_file(inode) && !f2fs_is_commit_atomic_write(inode)) {
if (!IS_ATOMIC_WRITTEN_PAGE(page)) {
register_inmem_page(inode, page);
f2fs_register_inmem_page(inode, page);
return 1;
}
/*
@ -2541,7 +2541,7 @@ static int f2fs_set_data_page_dirty(struct page *page)
if (!PageDirty(page)) {
__set_page_dirty_nobuffers(page);
update_dirty_page(inode, page);
f2fs_update_dirty_page(inode, page);
return 1;
}
return 0;
@ -2634,6 +2634,17 @@ const struct address_space_operations f2fs_dblock_aops = {
#endif
};
void f2fs_clear_radix_tree_dirty_tag(struct page *page)
{
struct address_space *mapping = page_mapping(page);
unsigned long flags;
spin_lock_irqsave(&mapping->tree_lock, flags);
radix_tree_tag_clear(&mapping->page_tree, page_index(page),
PAGECACHE_TAG_DIRTY);
spin_unlock_irqrestore(&mapping->tree_lock, flags);
}
int __init f2fs_init_post_read_processing(void)
{
bio_post_read_ctx_cache = KMEM_CACHE(bio_post_read_ctx, 0);

6
fs/f2fs/debug.c
View file

@ -104,6 +104,8 @@ static void update_general_status(struct f2fs_sb_info *sbi)
si->avail_nids = NM_I(sbi)->available_nids;
si->alloc_nids = NM_I(sbi)->nid_cnt[PREALLOC_NID];
si->bg_gc = sbi->bg_gc;
si->skipped_atomic_files[BG_GC] = sbi->skipped_atomic_files[BG_GC];
si->skipped_atomic_files[FG_GC] = sbi->skipped_atomic_files[FG_GC];
si->util_free = (int)(free_user_blocks(sbi) >> sbi->log_blocks_per_seg)
* 100 / (int)(sbi->user_block_count >> sbi->log_blocks_per_seg)
/ 2;
@ -342,6 +344,10 @@ static int stat_show(struct seq_file *s, void *v)
si->bg_data_blks);
seq_printf(s, " - node blocks : %d (%d)\n", si->node_blks,
si->bg_node_blks);
seq_printf(s, "Skipped : atomic write %llu (%llu)\n",
si->skipped_atomic_files[BG_GC] +
si->skipped_atomic_files[FG_GC],
si->skipped_atomic_files[BG_GC]);
seq_puts(s, "\nExtent Cache:\n");
seq_printf(s, " - Hit Count: L1-1:%llu L1-2:%llu L2:%llu\n",
si->hit_largest, si->hit_cached,

76
fs/f2fs/dir.c
View file

@ -60,12 +60,12 @@ static unsigned char f2fs_type_by_mode[S_IFMT >> S_SHIFT] = {
[S_IFLNK >> S_SHIFT] = F2FS_FT_SYMLINK,
};
void set_de_type(struct f2fs_dir_entry *de, umode_t mode)
static void set_de_type(struct f2fs_dir_entry *de, umode_t mode)
{
de->file_type = f2fs_type_by_mode[(mode & S_IFMT) >> S_SHIFT];
}
unsigned char get_de_type(struct f2fs_dir_entry *de)
unsigned char f2fs_get_de_type(struct f2fs_dir_entry *de)
{
if (de->file_type < F2FS_FT_MAX)
return f2fs_filetype_table[de->file_type];
@ -97,14 +97,14 @@ static struct f2fs_dir_entry *find_in_block(struct page *dentry_page,
dentry_blk = (struct f2fs_dentry_block *)page_address(dentry_page);
make_dentry_ptr_block(NULL, &d, dentry_blk);
de = find_target_dentry(fname, namehash, max_slots, &d);
de = f2fs_find_target_dentry(fname, namehash, max_slots, &d);
if (de)
*res_page = dentry_page;
return de;
}
struct f2fs_dir_entry *find_target_dentry(struct fscrypt_name *fname,
struct f2fs_dir_entry *f2fs_find_target_dentry(struct fscrypt_name *fname,
f2fs_hash_t namehash, int *max_slots,
struct f2fs_dentry_ptr *d)
{
@ -171,7 +171,7 @@ static struct f2fs_dir_entry *find_in_level(struct inode *dir,
for (; bidx < end_block; bidx++) {
/* no need to allocate new dentry pages to all the indices */
dentry_page = find_data_page(dir, bidx);
dentry_page = f2fs_find_data_page(dir, bidx);
if (IS_ERR(dentry_page)) {
if (PTR_ERR(dentry_page) == -ENOENT) {
room = true;
@ -210,7 +210,7 @@ struct f2fs_dir_entry *__f2fs_find_entry(struct inode *dir,
if (f2fs_has_inline_dentry(dir)) {
*res_page = NULL;
de = find_in_inline_dir(dir, fname, res_page);
de = f2fs_find_in_inline_dir(dir, fname, res_page);
goto out;
}
@ -319,7 +319,7 @@ static void init_dent_inode(const struct qstr *name, struct page *ipage)
set_page_dirty(ipage);
}
void do_make_empty_dir(struct inode *inode, struct inode *parent,
void f2fs_do_make_empty_dir(struct inode *inode, struct inode *parent,
struct f2fs_dentry_ptr *d)
{
struct qstr dot = QSTR_INIT(".", 1);
@ -340,23 +340,23 @@ static int make_empty_dir(struct inode *inode,
struct f2fs_dentry_ptr d;
if (f2fs_has_inline_dentry(inode))
return make_empty_inline_dir(inode, parent, page);
return f2fs_make_empty_inline_dir(inode, parent, page);
dentry_page = get_new_data_page(inode, page, 0, true);
dentry_page = f2fs_get_new_data_page(inode, page, 0, true);
if (IS_ERR(dentry_page))
return PTR_ERR(dentry_page);
dentry_blk = page_address(dentry_page);
make_dentry_ptr_block(NULL, &d, dentry_blk);
do_make_empty_dir(inode, parent, &d);
f2fs_do_make_empty_dir(inode, parent, &d);
set_page_dirty(dentry_page);
f2fs_put_page(dentry_page, 1);
return 0;
}
struct page *init_inode_metadata(struct inode *inode, struct inode *dir,
struct page *f2fs_init_inode_metadata(struct inode *inode, struct inode *dir,
const struct qstr *new_name, const struct qstr *orig_name,
struct page *dpage)
{
@ -365,7 +365,7 @@ struct page *init_inode_metadata(struct inode *inode, struct inode *dir,
int err;
if (is_inode_flag_set(inode, FI_NEW_INODE)) {
page = new_inode_page(inode);
page = f2fs_new_inode_page(inode);
if (IS_ERR(page))
return page;
@ -395,7 +395,7 @@ struct page *init_inode_metadata(struct inode *inode, struct inode *dir,
goto put_error;
}
} else {
page = get_node_page(F2FS_I_SB(dir), inode->i_ino);
page = f2fs_get_node_page(F2FS_I_SB(dir), inode->i_ino);
if (IS_ERR(page))
return page;
}
@ -418,19 +418,19 @@ struct page *init_inode_metadata(struct inode *inode, struct inode *dir,
* we should remove this inode from orphan list.
*/
if (inode->i_nlink == 0)
remove_orphan_inode(F2FS_I_SB(dir), inode->i_ino);
f2fs_remove_orphan_inode(F2FS_I_SB(dir), inode->i_ino);
f2fs_i_links_write(inode, true);
}
return page;
put_error:
clear_nlink(inode);
update_inode(inode, page);
f2fs_update_inode(inode, page);
f2fs_put_page(page, 1);
return ERR_PTR(err);
}
void update_parent_metadata(struct inode *dir, struct inode *inode,
void f2fs_update_parent_metadata(struct inode *dir, struct inode *inode,
unsigned int current_depth)
{
if (inode && is_inode_flag_set(inode, FI_NEW_INODE)) {
@ -448,7 +448,7 @@ void update_parent_metadata(struct inode *dir, struct inode *inode,
clear_inode_flag(inode, FI_INC_LINK);
}
int room_for_filename(const void *bitmap, int slots, int max_slots)
int f2fs_room_for_filename(const void *bitmap, int slots, int max_slots)
{
int bit_start = 0;
int zero_start, zero_end;
@ -537,12 +537,12 @@ start:
(le32_to_cpu(dentry_hash) % nbucket));
for (block = bidx; block <= (bidx + nblock - 1); block++) {
dentry_page = get_new_data_page(dir, NULL, block, true);
dentry_page = f2fs_get_new_data_page(dir, NULL, block, true);
if (IS_ERR(dentry_page))
return PTR_ERR(dentry_page);
dentry_blk = page_address(dentry_page);
bit_pos = room_for_filename(&dentry_blk->dentry_bitmap,
bit_pos = f2fs_room_for_filename(&dentry_blk->dentry_bitmap,
slots, NR_DENTRY_IN_BLOCK);
if (bit_pos < NR_DENTRY_IN_BLOCK)
goto add_dentry;
@ -558,7 +558,7 @@ add_dentry:
if (inode) {
down_write(&F2FS_I(inode)->i_sem);
page = init_inode_metadata(inode, dir, new_name,
page = f2fs_init_inode_metadata(inode, dir, new_name,
orig_name, NULL);
if (IS_ERR(page)) {
err = PTR_ERR(page);
@ -576,7 +576,7 @@ add_dentry:
f2fs_put_page(page, 1);
}
update_parent_metadata(dir, inode, current_depth);
f2fs_update_parent_metadata(dir, inode, current_depth);
fail:
if (inode)
up_write(&F2FS_I(inode)->i_sem);
@ -586,7 +586,7 @@ fail:
return err;
}
int __f2fs_do_add_link(struct inode *dir, struct fscrypt_name *fname,
int f2fs_add_dentry(struct inode *dir, struct fscrypt_name *fname,
struct inode *inode, nid_t ino, umode_t mode)
{
struct qstr new_name;
@ -610,7 +610,7 @@ int __f2fs_do_add_link(struct inode *dir, struct fscrypt_name *fname,
* Caller should grab and release a rwsem by calling f2fs_lock_op() and
* f2fs_unlock_op().
*/
int __f2fs_add_link(struct inode *dir, const struct qstr *name,
int f2fs_do_add_link(struct inode *dir, const struct qstr *name,
struct inode *inode, nid_t ino, umode_t mode)
{
struct fscrypt_name fname;
@ -639,7 +639,7 @@ int __f2fs_add_link(struct inode *dir, const struct qstr *name,
} else if (IS_ERR(page)) {
err = PTR_ERR(page);
} else {
err = __f2fs_do_add_link(dir, &fname, inode, ino, mode);
err = f2fs_add_dentry(dir, &fname, inode, ino, mode);
}
fscrypt_free_filename(&fname);
return err;
@ -651,7 +651,7 @@ int f2fs_do_tmpfile(struct inode *inode, struct inode *dir)
int err = 0;
down_write(&F2FS_I(inode)->i_sem);
page = init_inode_metadata(inode, dir, NULL, NULL, NULL);
page = f2fs_init_inode_metadata(inode, dir, NULL, NULL, NULL);
if (IS_ERR(page)) {
err = PTR_ERR(page);
goto fail;
@ -683,9 +683,9 @@ void f2fs_drop_nlink(struct inode *dir, struct inode *inode)
up_write(&F2FS_I(inode)->i_sem);
if (inode->i_nlink == 0)
add_orphan_inode(inode);
f2fs_add_orphan_inode(inode);
else
release_orphan_inode(sbi);
f2fs_release_orphan_inode(sbi);
}
/*
@ -698,14 +698,12 @@ void f2fs_delete_entry(struct f2fs_dir_entry *dentry, struct page *page,
struct f2fs_dentry_block *dentry_blk;
unsigned int bit_pos;
int slots = GET_DENTRY_SLOTS(le16_to_cpu(dentry->name_len));
struct address_space *mapping = page_mapping(page);
unsigned long flags;
int i;
f2fs_update_time(F2FS_I_SB(dir), REQ_TIME);
if (F2FS_OPTION(F2FS_I_SB(dir)).fsync_mode == FSYNC_MODE_STRICT)
add_ino_entry(F2FS_I_SB(dir), dir->i_ino, TRANS_DIR_INO);
f2fs_add_ino_entry(F2FS_I_SB(dir), dir->i_ino, TRANS_DIR_INO);
if (f2fs_has_inline_dentry(dir))
return f2fs_delete_inline_entry(dentry, page, dir, inode);
@ -731,17 +729,13 @@ void f2fs_delete_entry(struct f2fs_dir_entry *dentry, struct page *page,
f2fs_drop_nlink(dir, inode);
if (bit_pos == NR_DENTRY_IN_BLOCK &&
!truncate_hole(dir, page->index, page->index + 1)) {
spin_lock_irqsave(&mapping->tree_lock, flags);
radix_tree_tag_clear(&mapping->page_tree, page_index(page),
PAGECACHE_TAG_DIRTY);
spin_unlock_irqrestore(&mapping->tree_lock, flags);
!f2fs_truncate_hole(dir, page->index, page->index + 1)) {
f2fs_clear_radix_tree_dirty_tag(page);
clear_page_dirty_for_io(page);
ClearPagePrivate(page);
ClearPageUptodate(page);
inode_dec_dirty_pages(dir);
remove_dirty_inode(dir);
f2fs_remove_dirty_inode(dir);
}
f2fs_put_page(page, 1);
}
@ -758,7 +752,7 @@ bool f2fs_empty_dir(struct inode *dir)
return f2fs_empty_inline_dir(dir);
for (bidx = 0; bidx < nblock; bidx++) {
dentry_page = get_lock_data_page(dir, bidx, false);
dentry_page = f2fs_get_lock_data_page(dir, bidx, false);
if (IS_ERR(dentry_page)) {
if (PTR_ERR(dentry_page) == -ENOENT)
continue;
@ -806,7 +800,7 @@ int f2fs_fill_dentries(struct dir_context *ctx, struct f2fs_dentry_ptr *d,
continue;
}
d_type = get_de_type(de);
d_type = f2fs_get_de_type(de);
de_name.name = d->filename[bit_pos];
de_name.len = le16_to_cpu(de->name_len);
@ -830,7 +824,7 @@ int f2fs_fill_dentries(struct dir_context *ctx, struct f2fs_dentry_ptr *d,
return 1;
if (sbi->readdir_ra == 1)
ra_node_page(sbi, le32_to_cpu(de->ino));
f2fs_ra_node_page(sbi, le32_to_cpu(de->ino));
bit_pos += GET_DENTRY_SLOTS(le16_to_cpu(de->name_len));
ctx->pos = start_pos + bit_pos;
@ -880,7 +874,7 @@ static int f2fs_readdir(struct file *file, struct dir_context *ctx)
page_cache_sync_readahead(inode->i_mapping, ra, file, n,
min(npages - n, (pgoff_t)MAX_DIR_RA_PAGES));
dentry_page = get_lock_data_page(inode, n, false);
dentry_page = f2fs_get_lock_data_page(inode, n, false);
if (IS_ERR(dentry_page)) {
err = PTR_ERR(dentry_page);
if (err == -ENOENT) {

22
fs/f2fs/extent_cache.c
View file

@ -49,7 +49,7 @@ static struct rb_entry *__lookup_rb_tree_slow(struct rb_root *root,
return NULL;
}
struct rb_entry *__lookup_rb_tree(struct rb_root *root,
struct rb_entry *f2fs_lookup_rb_tree(struct rb_root *root,
struct rb_entry *cached_re, unsigned int ofs)
{
struct rb_entry *re;
@ -61,7 +61,7 @@ struct rb_entry *__lookup_rb_tree(struct rb_root *root,
return re;
}
struct rb_node **__lookup_rb_tree_for_insert(struct f2fs_sb_info *sbi,
struct rb_node **f2fs_lookup_rb_tree_for_insert(struct f2fs_sb_info *sbi,
struct rb_root *root, struct rb_node **parent,
unsigned int ofs)
{
@ -92,7 +92,7 @@ struct rb_node **__lookup_rb_tree_for_insert(struct f2fs_sb_info *sbi,
* in order to simpfy the insertion after.
* tree must stay unchanged between lookup and insertion.
*/
struct rb_entry *__lookup_rb_tree_ret(struct rb_root *root,
struct rb_entry *f2fs_lookup_rb_tree_ret(struct rb_root *root,
struct rb_entry *cached_re,
unsigned int ofs,
struct rb_entry **prev_entry,
@ -159,7 +159,7 @@ lookup_neighbors:
return re;
}
bool __check_rb_tree_consistence(struct f2fs_sb_info *sbi,
bool f2fs_check_rb_tree_consistence(struct f2fs_sb_info *sbi,
struct rb_root *root)
{
#ifdef CONFIG_F2FS_CHECK_FS
@ -390,7 +390,7 @@ static bool f2fs_lookup_extent_tree(struct inode *inode, pgoff_t pgofs,
goto out;
}
en = (struct extent_node *)__lookup_rb_tree(&et->root,
en = (struct extent_node *)f2fs_lookup_rb_tree(&et->root,
(struct rb_entry *)et->cached_en, pgofs);
if (!en)
goto out;
@ -470,7 +470,7 @@ static struct extent_node *__insert_extent_tree(struct inode *inode,
goto do_insert;
}
p = __lookup_rb_tree_for_insert(sbi, &et->root, &parent, ei->fofs);
p = f2fs_lookup_rb_tree_for_insert(sbi, &et->root, &parent, ei->fofs);
do_insert:
en = __attach_extent_node(sbi, et, ei, parent, p);
if (!en)
@ -520,7 +520,7 @@ static void f2fs_update_extent_tree_range(struct inode *inode,
__drop_largest_extent(inode, fofs, len);
/* 1. lookup first extent node in range [fofs, fofs + len - 1] */
en = (struct extent_node *)__lookup_rb_tree_ret(&et->root,
en = (struct extent_node *)f2fs_lookup_rb_tree_ret(&et->root,
(struct rb_entry *)et->cached_en, fofs,
(struct rb_entry **)&prev_en,
(struct rb_entry **)&next_en,
@ -773,7 +773,7 @@ void f2fs_update_extent_cache(struct dnode_of_data *dn)
else
blkaddr = dn->data_blkaddr;
fofs = start_bidx_of_node(ofs_of_node(dn->node_page), dn->inode) +
fofs = f2fs_start_bidx_of_node(ofs_of_node(dn->node_page), dn->inode) +
dn->ofs_in_node;
f2fs_update_extent_tree_range(dn->inode, fofs, blkaddr, 1);
}
@ -788,7 +788,7 @@ void f2fs_update_extent_cache_range(struct dnode_of_data *dn,
f2fs_update_extent_tree_range(dn->inode, fofs, blkaddr, len);
}
void init_extent_cache_info(struct f2fs_sb_info *sbi)
void f2fs_init_extent_cache_info(struct f2fs_sb_info *sbi)
{
INIT_RADIX_TREE(&sbi->extent_tree_root, GFP_NOIO);
mutex_init(&sbi->extent_tree_lock);
@ -800,7 +800,7 @@ void init_extent_cache_info(struct f2fs_sb_info *sbi)
atomic_set(&sbi->total_ext_node, 0);
}
int __init create_extent_cache(void)
int __init f2fs_create_extent_cache(void)
{
extent_tree_slab = f2fs_kmem_cache_create("f2fs_extent_tree",
sizeof(struct extent_tree));
@ -815,7 +815,7 @@ int __init create_extent_cache(void)
return 0;
}
void destroy_extent_cache(void)
void f2fs_destroy_extent_cache(void)
{
kmem_cache_destroy(extent_node_slab);
kmem_cache_destroy(extent_tree_slab);

425
fs/f2fs/f2fs.h
View file

@ -26,6 +26,7 @@
#include <linux/quotaops.h>
#include <crypto/hash.h>
#include <linux/writeback.h>
#include <linux/overflow.h>
#define __FS_HAS_ENCRYPTION IS_ENABLED(CONFIG_F2FS_FS_ENCRYPTION)
#include <linux/fscrypt.h>
@ -242,6 +243,7 @@ enum {
#define DEF_MAX_DISCARD_REQUEST 8 /* issue 8 discards per round */
#define DEF_MAX_DISCARD_LEN 512 /* Max. 2MB per discard */
#define DEF_MIN_DISCARD_ISSUE_TIME 50 /* 50 ms, if exists */
#define DEF_MID_DISCARD_ISSUE_TIME 500 /* 500 ms, if device busy */
#define DEF_MAX_DISCARD_ISSUE_TIME 60000 /* 60 s, if no candidates */
#define DEF_DISCARD_URGENT_UTIL 80 /* do more discard over 80% */
#define DEF_CP_INTERVAL 60 /* 60 secs */
@ -344,6 +346,7 @@ enum {
struct discard_policy {
int type; /* type of discard */
unsigned int min_interval; /* used for candidates exist */
unsigned int mid_interval; /* used for device busy */
unsigned int max_interval; /* used for candidates not exist */
unsigned int max_requests; /* # of discards issued per round */
unsigned int io_aware_gran; /* minimum granularity discard not be aware of I/O */
@ -676,15 +679,20 @@ enum {
#define DEF_DIR_LEVEL 0
enum {
GC_FAILURE_PIN,
GC_FAILURE_ATOMIC,
MAX_GC_FAILURE
};
struct f2fs_inode_info {
struct inode vfs_inode; /* serve a vfs inode */
unsigned long i_flags; /* keep an inode flags for ioctl */
unsigned char i_advise; /* use to give file attribute hints */
unsigned char i_dir_level; /* use for dentry level for large dir */
union {
unsigned int i_current_depth; /* only for directory depth */
unsigned short i_gc_failures; /* only for regular file */
};
/* for gc failure statistic */
unsigned int i_gc_failures[MAX_GC_FAILURE];
unsigned int i_pino; /* parent inode number */
umode_t i_acl_mode; /* keep file acl mode temporarily */
@ -712,7 +720,9 @@ struct f2fs_inode_info {
struct task_struct *inmem_task; /* store inmemory task */
struct mutex inmem_lock; /* lock for inmemory pages */
struct extent_tree *extent_tree; /* cached extent_tree entry */
struct rw_semaphore dio_rwsem[2];/* avoid racing between dio and gc */
/* avoid racing between foreground op and gc */
struct rw_semaphore i_gc_rwsem[2];
struct rw_semaphore i_mmap_sem;
struct rw_semaphore i_xattr_sem; /* avoid racing between reading and changing EAs */
@ -1062,6 +1072,7 @@ struct f2fs_io_info {
int need_lock; /* indicate we need to lock cp_rwsem */
bool in_list; /* indicate fio is in io_list */
bool is_meta; /* indicate borrow meta inode mapping or not */
bool retry; /* need to reallocate block address */
enum iostat_type io_type; /* io type */
struct writeback_control *io_wbc; /* writeback control */
};
@ -1123,6 +1134,13 @@ enum {
MAX_TIME,
};
enum {
GC_NORMAL,
GC_IDLE_CB,
GC_IDLE_GREEDY,
GC_URGENT,
};
enum {
WHINT_MODE_OFF, /* not pass down write hints */
WHINT_MODE_USER, /* try to pass down hints given by users */
@ -1171,6 +1189,8 @@ struct f2fs_sb_info {
struct f2fs_bio_info *write_io[NR_PAGE_TYPE]; /* for write bios */
struct mutex wio_mutex[NR_PAGE_TYPE - 1][NR_TEMP_TYPE];
/* bio ordering for NODE/DATA */
/* keep migration IO order for LFS mode */
struct rw_semaphore io_order_lock;
mempool_t *write_io_dummy; /* Dummy pages */
/* for checkpoint */
@ -1241,7 +1261,7 @@ struct f2fs_sb_info {
struct percpu_counter alloc_valid_block_count;
/* writeback control */
atomic_t wb_sync_req; /* count # of WB_SYNC threads */
atomic_t wb_sync_req[META]; /* count # of WB_SYNC threads */
/* valid inode count */
struct percpu_counter total_valid_inode_count;
@ -1252,9 +1272,9 @@ struct f2fs_sb_info {
struct mutex gc_mutex; /* mutex for GC */
struct f2fs_gc_kthread *gc_thread; /* GC thread */
unsigned int cur_victim_sec; /* current victim section num */
/* threshold for converting bg victims for fg */
u64 fggc_threshold;
unsigned int gc_mode; /* current GC state */
/* for skip statistic */
unsigned long long skipped_atomic_files[2]; /* FG_GC and BG_GC */
/* threshold for gc trials on pinned files */
u64 gc_pin_file_threshold;
@ -1644,18 +1664,6 @@ static inline bool __exist_node_summaries(struct f2fs_sb_info *sbi)
is_set_ckpt_flags(sbi, CP_FASTBOOT_FLAG));
}
/*
* Check whether the given nid is within node id range.
*/
static inline int check_nid_range(struct f2fs_sb_info *sbi, nid_t nid)
{
if (unlikely(nid < F2FS_ROOT_INO(sbi)))
return -EINVAL;
if (unlikely(nid >= NM_I(sbi)->max_nid))
return -EINVAL;
return 0;
}
/*
* Check whether the inode has blocks or not
*/
@ -2214,9 +2222,60 @@ static inline void f2fs_change_bit(unsigned int nr, char *addr)
*addr ^= mask;
}
#define F2FS_REG_FLMASK (~(FS_DIRSYNC_FL | FS_TOPDIR_FL))
#define F2FS_OTHER_FLMASK (FS_NODUMP_FL | FS_NOATIME_FL)
#define F2FS_FL_INHERITED (FS_PROJINHERIT_FL)
/*
* Inode flags
*/
#define F2FS_SECRM_FL 0x00000001 /* Secure deletion */
#define F2FS_UNRM_FL 0x00000002 /* Undelete */
#define F2FS_COMPR_FL 0x00000004 /* Compress file */
#define F2FS_SYNC_FL 0x00000008 /* Synchronous updates */
#define F2FS_IMMUTABLE_FL 0x00000010 /* Immutable file */
#define F2FS_APPEND_FL 0x00000020 /* writes to file may only append */
#define F2FS_NODUMP_FL 0x00000040 /* do not dump file */
#define F2FS_NOATIME_FL 0x00000080 /* do not update atime */
/* Reserved for compression usage... */
#define F2FS_DIRTY_FL 0x00000100
#define F2FS_COMPRBLK_FL 0x00000200 /* One or more compressed clusters */
#define F2FS_NOCOMPR_FL 0x00000400 /* Don't compress */
#define F2FS_ENCRYPT_FL 0x00000800 /* encrypted file */
/* End compression flags --- maybe not all used */
#define F2FS_INDEX_FL 0x00001000 /* hash-indexed directory */
#define F2FS_IMAGIC_FL 0x00002000 /* AFS directory */
#define F2FS_JOURNAL_DATA_FL 0x00004000 /* file data should be journaled */
#define F2FS_NOTAIL_FL 0x00008000 /* file tail should not be merged */
#define F2FS_DIRSYNC_FL 0x00010000 /* dirsync behaviour (directories only) */
#define F2FS_TOPDIR_FL 0x00020000 /* Top of directory hierarchies*/
#define F2FS_HUGE_FILE_FL 0x00040000 /* Set to each huge file */
#define F2FS_EXTENTS_FL 0x00080000 /* Inode uses extents */
#define F2FS_EA_INODE_FL 0x00200000 /* Inode used for large EA */
#define F2FS_EOFBLOCKS_FL 0x00400000 /* Blocks allocated beyond EOF */
#define F2FS_INLINE_DATA_FL 0x10000000 /* Inode has inline data. */
#define F2FS_PROJINHERIT_FL 0x20000000 /* Create with parents projid */
#define F2FS_RESERVED_FL 0x80000000 /* reserved for ext4 lib */
#define F2FS_FL_USER_VISIBLE 0x304BDFFF /* User visible flags */
#define F2FS_FL_USER_MODIFIABLE 0x204BC0FF /* User modifiable flags */
/* Flags we can manipulate with through F2FS_IOC_FSSETXATTR */
#define F2FS_FL_XFLAG_VISIBLE (F2FS_SYNC_FL | \
F2FS_IMMUTABLE_FL | \
F2FS_APPEND_FL | \
F2FS_NODUMP_FL | \
F2FS_NOATIME_FL | \
F2FS_PROJINHERIT_FL)
/* Flags that should be inherited by new inodes from their parent. */
#define F2FS_FL_INHERITED (F2FS_SECRM_FL | F2FS_UNRM_FL | F2FS_COMPR_FL |\
F2FS_SYNC_FL | F2FS_NODUMP_FL | F2FS_NOATIME_FL |\
F2FS_NOCOMPR_FL | F2FS_JOURNAL_DATA_FL |\
F2FS_NOTAIL_FL | F2FS_DIRSYNC_FL |\
F2FS_PROJINHERIT_FL)
/* Flags that are appropriate for regular files (all but dir-specific ones). */
#define F2FS_REG_FLMASK (~(F2FS_DIRSYNC_FL | F2FS_TOPDIR_FL))
/* Flags that are appropriate for non-directories/regular files. */
#define F2FS_OTHER_FLMASK (F2FS_NODUMP_FL | F2FS_NOATIME_FL)
static inline __u32 f2fs_mask_flags(umode_t mode, __u32 flags)
{
@ -2259,6 +2318,7 @@ enum {
FI_EXTRA_ATTR, /* indicate file has extra attribute */
FI_PROJ_INHERIT, /* indicate file inherits projectid */
FI_PIN_FILE, /* indicate file should not be gced */
FI_ATOMIC_REVOKE_REQUEST, /* request to drop atomic data */
};
static inline void __mark_inode_dirty_flag(struct inode *inode,
@ -2357,7 +2417,7 @@ static inline void f2fs_i_depth_write(struct inode *inode, unsigned int depth)
static inline void f2fs_i_gc_failures_write(struct inode *inode,
unsigned int count)
{
F2FS_I(inode)->i_gc_failures = count;
F2FS_I(inode)->i_gc_failures[GC_FAILURE_PIN] = count;
f2fs_mark_inode_dirty_sync(inode, true);
}
@ -2646,7 +2706,7 @@ static inline int get_inline_xattr_addrs(struct inode *inode)
return F2FS_I(inode)->i_inline_xattr_size;
}
#define get_inode_mode(i) \
#define f2fs_get_inode_mode(i) \
((is_inode_flag_set(i, FI_ACL_MODE)) ? \
(F2FS_I(i)->i_acl_mode) : ((i)->i_mode))
@ -2685,18 +2745,25 @@ static inline void f2fs_update_iostat(struct f2fs_sb_info *sbi,
spin_unlock(&sbi->iostat_lock);
}
static inline bool is_valid_blkaddr(block_t blkaddr)
{
if (blkaddr == NEW_ADDR || blkaddr == NULL_ADDR)
return false;
return true;
}
/*
* file.c
*/
int f2fs_sync_file(struct file *file, loff_t start, loff_t end, int datasync);
void truncate_data_blocks(struct dnode_of_data *dn);
int truncate_blocks(struct inode *inode, u64 from, bool lock);
void f2fs_truncate_data_blocks(struct dnode_of_data *dn);
int f2fs_truncate_blocks(struct inode *inode, u64 from, bool lock);
int f2fs_truncate(struct inode *inode);
int f2fs_getattr(struct vfsmount *mnt, struct dentry *dentry,
struct kstat *stat);
int f2fs_setattr(struct dentry *dentry, struct iattr *attr);
int truncate_hole(struct inode *inode, pgoff_t pg_start, pgoff_t pg_end);
void truncate_data_blocks_range(struct dnode_of_data *dn, int count);
int f2fs_truncate_hole(struct inode *inode, pgoff_t pg_start, pgoff_t pg_end);
void f2fs_truncate_data_blocks_range(struct dnode_of_data *dn, int count);
int f2fs_precache_extents(struct inode *inode);
long f2fs_ioctl(struct file *filp, unsigned int cmd, unsigned long arg);
long f2fs_compat_ioctl(struct file *file, unsigned int cmd, unsigned long arg);
@ -2710,38 +2777,37 @@ bool f2fs_inode_chksum_verify(struct f2fs_sb_info *sbi, struct page *page);
void f2fs_inode_chksum_set(struct f2fs_sb_info *sbi, struct page *page);
struct inode *f2fs_iget(struct super_block *sb, unsigned long ino);
struct inode *f2fs_iget_retry(struct super_block *sb, unsigned long ino);
int try_to_free_nats(struct f2fs_sb_info *sbi, int nr_shrink);
void update_inode(struct inode *inode, struct page *node_page);
void update_inode_page(struct inode *inode);
int f2fs_try_to_free_nats(struct f2fs_sb_info *sbi, int nr_shrink);
void f2fs_update_inode(struct inode *inode, struct page *node_page);
void f2fs_update_inode_page(struct inode *inode);
int f2fs_write_inode(struct inode *inode, struct writeback_control *wbc);
void f2fs_evict_inode(struct inode *inode);
void handle_failed_inode(struct inode *inode);
void f2fs_handle_failed_inode(struct inode *inode);
/*
* namei.c
*/
int update_extension_list(struct f2fs_sb_info *sbi, const char *name,
int f2fs_update_extension_list(struct f2fs_sb_info *sbi, const char *name,
bool hot, bool set);
struct dentry *f2fs_get_parent(struct dentry *child);
/*
* dir.c
*/
void set_de_type(struct f2fs_dir_entry *de, umode_t mode);
unsigned char get_de_type(struct f2fs_dir_entry *de);
struct f2fs_dir_entry *find_target_dentry(struct fscrypt_name *fname,
unsigned char f2fs_get_de_type(struct f2fs_dir_entry *de);
struct f2fs_dir_entry *f2fs_find_target_dentry(struct fscrypt_name *fname,
f2fs_hash_t namehash, int *max_slots,
struct f2fs_dentry_ptr *d);
int f2fs_fill_dentries(struct dir_context *ctx, struct f2fs_dentry_ptr *d,
unsigned int start_pos, struct fscrypt_str *fstr);
void do_make_empty_dir(struct inode *inode, struct inode *parent,
void f2fs_do_make_empty_dir(struct inode *inode, struct inode *parent,
struct f2fs_dentry_ptr *d);
struct page *init_inode_metadata(struct inode *inode, struct inode *dir,
struct page *f2fs_init_inode_metadata(struct inode *inode, struct inode *dir,
const struct qstr *new_name,
const struct qstr *orig_name, struct page *dpage);
void update_parent_metadata(struct inode *dir, struct inode *inode,
void f2fs_update_parent_metadata(struct inode *dir, struct inode *inode,
unsigned int current_depth);
int room_for_filename(const void *bitmap, int slots, int max_slots);
int f2fs_room_for_filename(const void *bitmap, int slots, int max_slots);
void f2fs_drop_nlink(struct inode *dir, struct inode *inode);
struct f2fs_dir_entry *__f2fs_find_entry(struct inode *dir,
struct fscrypt_name *fname, struct page **res_page);
@ -2758,9 +2824,9 @@ void f2fs_update_dentry(nid_t ino, umode_t mode, struct f2fs_dentry_ptr *d,
int f2fs_add_regular_entry(struct inode *dir, const struct qstr *new_name,
const struct qstr *orig_name,
struct inode *inode, nid_t ino, umode_t mode);
int __f2fs_do_add_link(struct inode *dir, struct fscrypt_name *fname,
int f2fs_add_dentry(struct inode *dir, struct fscrypt_name *fname,
struct inode *inode, nid_t ino, umode_t mode);
int __f2fs_add_link(struct inode *dir, const struct qstr *name,
int f2fs_do_add_link(struct inode *dir, const struct qstr *name,
struct inode *inode, nid_t ino, umode_t mode);
void f2fs_delete_entry(struct f2fs_dir_entry *dentry, struct page *page,
struct inode *dir, struct inode *inode);
@ -2769,7 +2835,7 @@ bool f2fs_empty_dir(struct inode *dir);
static inline int f2fs_add_link(struct dentry *dentry, struct inode *inode)
{
return __f2fs_add_link(d_inode(dentry->d_parent), &dentry->d_name,
return f2fs_do_add_link(d_inode(dentry->d_parent), &dentry->d_name,
inode, inode->i_ino, inode->i_mode);
}
@ -2784,7 +2850,7 @@ int f2fs_commit_super(struct f2fs_sb_info *sbi, bool recover);
int f2fs_sync_fs(struct super_block *sb, int sync);
extern __printf(3, 4)
void f2fs_msg(struct super_block *sb, const char *level, const char *fmt, ...);
int sanity_check_ckpt(struct f2fs_sb_info *sbi);
int f2fs_sanity_check_ckpt(struct f2fs_sb_info *sbi);
/*
* hash.c
@ -2798,138 +2864,146 @@ f2fs_hash_t f2fs_dentry_hash(const struct qstr *name_info,
struct dnode_of_data;
struct node_info;
bool available_free_memory(struct f2fs_sb_info *sbi, int type);
int need_dentry_mark(struct f2fs_sb_info *sbi, nid_t nid);
bool is_checkpointed_node(struct f2fs_sb_info *sbi, nid_t nid);
bool need_inode_block_update(struct f2fs_sb_info *sbi, nid_t ino);
void get_node_info(struct f2fs_sb_info *sbi, nid_t nid, struct node_info *ni);
pgoff_t get_next_page_offset(struct dnode_of_data *dn, pgoff_t pgofs);
int get_dnode_of_data(struct dnode_of_data *dn, pgoff_t index, int mode);
int truncate_inode_blocks(struct inode *inode, pgoff_t from);
int truncate_xattr_node(struct inode *inode);
int wait_on_node_pages_writeback(struct f2fs_sb_info *sbi, nid_t ino);
int remove_inode_page(struct inode *inode);
struct page *new_inode_page(struct inode *inode);
struct page *new_node_page(struct dnode_of_data *dn, unsigned int ofs);
void ra_node_page(struct f2fs_sb_info *sbi, nid_t nid);
struct page *get_node_page(struct f2fs_sb_info *sbi, pgoff_t nid);
struct page *get_node_page_ra(struct page *parent, int start);
void move_node_page(struct page *node_page, int gc_type);
int fsync_node_pages(struct f2fs_sb_info *sbi, struct inode *inode,
int f2fs_check_nid_range(struct f2fs_sb_info *sbi, nid_t nid);
bool f2fs_available_free_memory(struct f2fs_sb_info *sbi, int type);
int f2fs_need_dentry_mark(struct f2fs_sb_info *sbi, nid_t nid);
bool f2fs_is_checkpointed_node(struct f2fs_sb_info *sbi, nid_t nid);
bool f2fs_need_inode_block_update(struct f2fs_sb_info *sbi, nid_t ino);
void f2fs_get_node_info(struct f2fs_sb_info *sbi, nid_t nid,
struct node_info *ni);
pgoff_t f2fs_get_next_page_offset(struct dnode_of_data *dn, pgoff_t pgofs);
int f2fs_get_dnode_of_data(struct dnode_of_data *dn, pgoff_t index, int mode);
int f2fs_truncate_inode_blocks(struct inode *inode, pgoff_t from);
int f2fs_truncate_xattr_node(struct inode *inode);
int f2fs_wait_on_node_pages_writeback(struct f2fs_sb_info *sbi, nid_t ino);
int f2fs_remove_inode_page(struct inode *inode);
struct page *f2fs_new_inode_page(struct inode *inode);
struct page *f2fs_new_node_page(struct dnode_of_data *dn, unsigned int ofs);
void f2fs_ra_node_page(struct f2fs_sb_info *sbi, nid_t nid);
struct page *f2fs_get_node_page(struct f2fs_sb_info *sbi, pgoff_t nid);
struct page *f2fs_get_node_page_ra(struct page *parent, int start);
void f2fs_move_node_page(struct page *node_page, int gc_type);
int f2fs_fsync_node_pages(struct f2fs_sb_info *sbi, struct inode *inode,
struct writeback_control *wbc, bool atomic);
int sync_node_pages(struct f2fs_sb_info *sbi, struct writeback_control *wbc,
int f2fs_sync_node_pages(struct f2fs_sb_info *sbi,
struct writeback_control *wbc,
bool do_balance, enum iostat_type io_type);
void build_free_nids(struct f2fs_sb_info *sbi, bool sync, bool mount);
bool alloc_nid(struct f2fs_sb_info *sbi, nid_t *nid);
void alloc_nid_done(struct f2fs_sb_info *sbi, nid_t nid);
void alloc_nid_failed(struct f2fs_sb_info *sbi, nid_t nid);
int try_to_free_nids(struct f2fs_sb_info *sbi, int nr_shrink);
void recover_inline_xattr(struct inode *inode, struct page *page);
int recover_xattr_data(struct inode *inode, struct page *page);
int recover_inode_page(struct f2fs_sb_info *sbi, struct page *page);
void restore_node_summary(struct f2fs_sb_info *sbi,
void f2fs_build_free_nids(struct f2fs_sb_info *sbi, bool sync, bool mount);
bool f2fs_alloc_nid(struct f2fs_sb_info *sbi, nid_t *nid);
void f2fs_alloc_nid_done(struct f2fs_sb_info *sbi, nid_t nid);
void f2fs_alloc_nid_failed(struct f2fs_sb_info *sbi, nid_t nid);
int f2fs_try_to_free_nids(struct f2fs_sb_info *sbi, int nr_shrink);
void f2fs_recover_inline_xattr(struct inode *inode, struct page *page);
int f2fs_recover_xattr_data(struct inode *inode, struct page *page);
int f2fs_recover_inode_page(struct f2fs_sb_info *sbi, struct page *page);
void f2fs_restore_node_summary(struct f2fs_sb_info *sbi,
unsigned int segno, struct f2fs_summary_block *sum);
void flush_nat_entries(struct f2fs_sb_info *sbi, struct cp_control *cpc);
int build_node_manager(struct f2fs_sb_info *sbi);
void destroy_node_manager(struct f2fs_sb_info *sbi);
int __init create_node_manager_caches(void);
void destroy_node_manager_caches(void);
void f2fs_flush_nat_entries(struct f2fs_sb_info *sbi, struct cp_control *cpc);
int f2fs_build_node_manager(struct f2fs_sb_info *sbi);
void f2fs_destroy_node_manager(struct f2fs_sb_info *sbi);
int __init f2fs_create_node_manager_caches(void);
void f2fs_destroy_node_manager_caches(void);
/*
* segment.c
*/
bool need_SSR(struct f2fs_sb_info *sbi);
void register_inmem_page(struct inode *inode, struct page *page);
void drop_inmem_pages_all(struct f2fs_sb_info *sbi);
void drop_inmem_pages(struct inode *inode);
void drop_inmem_page(struct inode *inode, struct page *page);
int commit_inmem_pages(struct inode *inode);
bool f2fs_need_SSR(struct f2fs_sb_info *sbi);
void f2fs_register_inmem_page(struct inode *inode, struct page *page);
void f2fs_drop_inmem_pages_all(struct f2fs_sb_info *sbi, bool gc_failure);
void f2fs_drop_inmem_pages(struct inode *inode);
void f2fs_drop_inmem_page(struct inode *inode, struct page *page);
int f2fs_commit_inmem_pages(struct inode *inode);
void f2fs_balance_fs(struct f2fs_sb_info *sbi, bool need);
void f2fs_balance_fs_bg(struct f2fs_sb_info *sbi);
int f2fs_issue_flush(struct f2fs_sb_info *sbi, nid_t ino);
int create_flush_cmd_control(struct f2fs_sb_info *sbi);
int f2fs_create_flush_cmd_control(struct f2fs_sb_info *sbi);
int f2fs_flush_device_cache(struct f2fs_sb_info *sbi);
void destroy_flush_cmd_control(struct f2fs_sb_info *sbi, bool free);
void invalidate_blocks(struct f2fs_sb_info *sbi, block_t addr);
bool is_checkpointed_data(struct f2fs_sb_info *sbi, block_t blkaddr);
void drop_discard_cmd(struct f2fs_sb_info *sbi);
void stop_discard_thread(struct f2fs_sb_info *sbi);
void f2fs_destroy_flush_cmd_control(struct f2fs_sb_info *sbi, bool free);
void f2fs_invalidate_blocks(struct f2fs_sb_info *sbi, block_t addr);
bool f2fs_is_checkpointed_data(struct f2fs_sb_info *sbi, block_t blkaddr);
void f2fs_drop_discard_cmd(struct f2fs_sb_info *sbi);
void f2fs_stop_discard_thread(struct f2fs_sb_info *sbi);
bool f2fs_wait_discard_bios(struct f2fs_sb_info *sbi);
void clear_prefree_segments(struct f2fs_sb_info *sbi, struct cp_control *cpc);
void release_discard_addrs(struct f2fs_sb_info *sbi);
int npages_for_summary_flush(struct f2fs_sb_info *sbi, bool for_ra);
void allocate_new_segments(struct f2fs_sb_info *sbi);
void f2fs_clear_prefree_segments(struct f2fs_sb_info *sbi,
struct cp_control *cpc);
void f2fs_release_discard_addrs(struct f2fs_sb_info *sbi);
int f2fs_npages_for_summary_flush(struct f2fs_sb_info *sbi, bool for_ra);
void f2fs_allocate_new_segments(struct f2fs_sb_info *sbi);
int f2fs_trim_fs(struct f2fs_sb_info *sbi, struct fstrim_range *range);
bool exist_trim_candidates(struct f2fs_sb_info *sbi, struct cp_control *cpc);
struct page *get_sum_page(struct f2fs_sb_info *sbi, unsigned int segno);
void update_meta_page(struct f2fs_sb_info *sbi, void *src, block_t blk_addr);
void write_meta_page(struct f2fs_sb_info *sbi, struct page *page,
bool f2fs_exist_trim_candidates(struct f2fs_sb_info *sbi,
struct cp_control *cpc);
struct page *f2fs_get_sum_page(struct f2fs_sb_info *sbi, unsigned int segno);
void f2fs_update_meta_page(struct f2fs_sb_info *sbi, void *src,
block_t blk_addr);
void f2fs_do_write_meta_page(struct f2fs_sb_info *sbi, struct page *page,
enum iostat_type io_type);
void write_node_page(unsigned int nid, struct f2fs_io_info *fio);
void write_data_page(struct dnode_of_data *dn, struct f2fs_io_info *fio);
int rewrite_data_page(struct f2fs_io_info *fio);
void __f2fs_replace_block(struct f2fs_sb_info *sbi, struct f2fs_summary *sum,
void f2fs_do_write_node_page(unsigned int nid, struct f2fs_io_info *fio);
void f2fs_outplace_write_data(struct dnode_of_data *dn,
struct f2fs_io_info *fio);
int f2fs_inplace_write_data(struct f2fs_io_info *fio);
void f2fs_do_replace_block(struct f2fs_sb_info *sbi, struct f2fs_summary *sum,
block_t old_blkaddr, block_t new_blkaddr,
bool recover_curseg, bool recover_newaddr);
void f2fs_replace_block(struct f2fs_sb_info *sbi, struct dnode_of_data *dn,
block_t old_addr, block_t new_addr,
unsigned char version, bool recover_curseg,
bool recover_newaddr);
void allocate_data_block(struct f2fs_sb_info *sbi, struct page *page,
void f2fs_allocate_data_block(struct f2fs_sb_info *sbi, struct page *page,
block_t old_blkaddr, block_t *new_blkaddr,
struct f2fs_summary *sum, int type,
struct f2fs_io_info *fio, bool add_list);
void f2fs_wait_on_page_writeback(struct page *page,
enum page_type type, bool ordered);
void f2fs_wait_on_block_writeback(struct f2fs_sb_info *sbi, block_t blkaddr);
void write_data_summaries(struct f2fs_sb_info *sbi, block_t start_blk);
void write_node_summaries(struct f2fs_sb_info *sbi, block_t start_blk);
int lookup_journal_in_cursum(struct f2fs_journal *journal, int type,
void f2fs_write_data_summaries(struct f2fs_sb_info *sbi, block_t start_blk);
void f2fs_write_node_summaries(struct f2fs_sb_info *sbi, block_t start_blk);
int f2fs_lookup_journal_in_cursum(struct f2fs_journal *journal, int type,
unsigned int val, int alloc);
void flush_sit_entries(struct f2fs_sb_info *sbi, struct cp_control *cpc);
int build_segment_manager(struct f2fs_sb_info *sbi);
void destroy_segment_manager(struct f2fs_sb_info *sbi);
int __init create_segment_manager_caches(void);
void destroy_segment_manager_caches(void);
int rw_hint_to_seg_type(enum rw_hint hint);
enum rw_hint io_type_to_rw_hint(struct f2fs_sb_info *sbi, enum page_type type,
enum temp_type temp);
void f2fs_flush_sit_entries(struct f2fs_sb_info *sbi, struct cp_control *cpc);
int f2fs_build_segment_manager(struct f2fs_sb_info *sbi);
void f2fs_destroy_segment_manager(struct f2fs_sb_info *sbi);
int __init f2fs_create_segment_manager_caches(void);
void f2fs_destroy_segment_manager_caches(void);
int f2fs_rw_hint_to_seg_type(enum rw_hint hint);
enum rw_hint f2fs_io_type_to_rw_hint(struct f2fs_sb_info *sbi,
enum page_type type, enum temp_type temp);
/*
* checkpoint.c
*/
void f2fs_stop_checkpoint(struct f2fs_sb_info *sbi, bool end_io);
struct page *grab_meta_page(struct f2fs_sb_info *sbi, pgoff_t index);
struct page *get_meta_page(struct f2fs_sb_info *sbi, pgoff_t index);
struct page *get_tmp_page(struct f2fs_sb_info *sbi, pgoff_t index);
bool is_valid_blkaddr(struct f2fs_sb_info *sbi, block_t blkaddr, int type);
int ra_meta_pages(struct f2fs_sb_info *sbi, block_t start, int nrpages,
struct page *f2fs_grab_meta_page(struct f2fs_sb_info *sbi, pgoff_t index);
struct page *f2fs_get_meta_page(struct f2fs_sb_info *sbi, pgoff_t index);
struct page *f2fs_get_tmp_page(struct f2fs_sb_info *sbi, pgoff_t index);
bool f2fs_is_valid_meta_blkaddr(struct f2fs_sb_info *sbi,
block_t blkaddr, int type);
int f2fs_ra_meta_pages(struct f2fs_sb_info *sbi, block_t start, int nrpages,
int type, bool sync);
void ra_meta_pages_cond(struct f2fs_sb_info *sbi, pgoff_t index);
long sync_meta_pages(struct f2fs_sb_info *sbi, enum page_type type,
void f2fs_ra_meta_pages_cond(struct f2fs_sb_info *sbi, pgoff_t index);
long f2fs_sync_meta_pages(struct f2fs_sb_info *sbi, enum page_type type,
long nr_to_write, enum iostat_type io_type);
void add_ino_entry(struct f2fs_sb_info *sbi, nid_t ino, int type);
void remove_ino_entry(struct f2fs_sb_info *sbi, nid_t ino, int type);
void release_ino_entry(struct f2fs_sb_info *sbi, bool all);
bool exist_written_data(struct f2fs_sb_info *sbi, nid_t ino, int mode);
void set_dirty_device(struct f2fs_sb_info *sbi, nid_t ino,
void f2fs_add_ino_entry(struct f2fs_sb_info *sbi, nid_t ino, int type);
void f2fs_remove_ino_entry(struct f2fs_sb_info *sbi, nid_t ino, int type);
void f2fs_release_ino_entry(struct f2fs_sb_info *sbi, bool all);
bool f2fs_exist_written_data(struct f2fs_sb_info *sbi, nid_t ino, int mode);
void f2fs_set_dirty_device(struct f2fs_sb_info *sbi, nid_t ino,
unsigned int devidx, int type);
bool is_dirty_device(struct f2fs_sb_info *sbi, nid_t ino,
bool f2fs_is_dirty_device(struct f2fs_sb_info *sbi, nid_t ino,
unsigned int devidx, int type);
int f2fs_sync_inode_meta(struct f2fs_sb_info *sbi);
int acquire_orphan_inode(struct f2fs_sb_info *sbi);
void release_orphan_inode(struct f2fs_sb_info *sbi);
void add_orphan_inode(struct inode *inode);
void remove_orphan_inode(struct f2fs_sb_info *sbi, nid_t ino);
int recover_orphan_inodes(struct f2fs_sb_info *sbi);
int get_valid_checkpoint(struct f2fs_sb_info *sbi);
void update_dirty_page(struct inode *inode, struct page *page);
void remove_dirty_inode(struct inode *inode);
int sync_dirty_inodes(struct f2fs_sb_info *sbi, enum inode_type type);
int write_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc);
void init_ino_entry_info(struct f2fs_sb_info *sbi);
int __init create_checkpoint_caches(void);
void destroy_checkpoint_caches(void);
int f2fs_acquire_orphan_inode(struct f2fs_sb_info *sbi);
void f2fs_release_orphan_inode(struct f2fs_sb_info *sbi);
void f2fs_add_orphan_inode(struct inode *inode);
void f2fs_remove_orphan_inode(struct f2fs_sb_info *sbi, nid_t ino);
int f2fs_recover_orphan_inodes(struct f2fs_sb_info *sbi);
int f2fs_get_valid_checkpoint(struct f2fs_sb_info *sbi);
void f2fs_update_dirty_page(struct inode *inode, struct page *page);
void f2fs_remove_dirty_inode(struct inode *inode);
int f2fs_sync_dirty_inodes(struct f2fs_sb_info *sbi, enum inode_type type);
int f2fs_write_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc);
void f2fs_init_ino_entry_info(struct f2fs_sb_info *sbi);
int __init f2fs_create_checkpoint_caches(void);
void f2fs_destroy_checkpoint_caches(void);
/*
* data.c
@ -2942,34 +3016,31 @@ void f2fs_submit_merged_write_cond(struct f2fs_sb_info *sbi,
enum page_type type);
void f2fs_flush_merged_writes(struct f2fs_sb_info *sbi);
int f2fs_submit_page_bio(struct f2fs_io_info *fio);
int f2fs_submit_page_write(struct f2fs_io_info *fio);
void f2fs_submit_page_write(struct f2fs_io_info *fio);
struct block_device *f2fs_target_device(struct f2fs_sb_info *sbi,
block_t blk_addr, struct bio *bio);
int f2fs_target_device_index(struct f2fs_sb_info *sbi, block_t blkaddr);
void set_data_blkaddr(struct dnode_of_data *dn);
void f2fs_set_data_blkaddr(struct dnode_of_data *dn);
void f2fs_update_data_blkaddr(struct dnode_of_data *dn, block_t blkaddr);
int reserve_new_blocks(struct dnode_of_data *dn, blkcnt_t count);
int reserve_new_block(struct dnode_of_data *dn);
int f2fs_reserve_new_blocks(struct dnode_of_data *dn, blkcnt_t count);
int f2fs_reserve_new_block(struct dnode_of_data *dn);
int f2fs_get_block(struct dnode_of_data *dn, pgoff_t index);
int f2fs_preallocate_blocks(struct kiocb *iocb, struct iov_iter *from);
int f2fs_reserve_block(struct dnode_of_data *dn, pgoff_t index);
struct page *get_read_data_page(struct inode *inode, pgoff_t index,
struct page *f2fs_get_read_data_page(struct inode *inode, pgoff_t index,
int op_flags, bool for_write);
struct page *find_data_page(struct inode *inode, pgoff_t index);
struct page *get_lock_data_page(struct inode *inode, pgoff_t index,
struct page *f2fs_find_data_page(struct inode *inode, pgoff_t index);
struct page *f2fs_get_lock_data_page(struct inode *inode, pgoff_t index,
bool for_write);
struct page *get_new_data_page(struct inode *inode,
struct page *f2fs_get_new_data_page(struct inode *inode,
struct page *ipage, pgoff_t index, bool new_i_size);
int do_write_data_page(struct f2fs_io_info *fio);
int f2fs_do_write_data_page(struct f2fs_io_info *fio);
int f2fs_map_blocks(struct inode *inode, struct f2fs_map_blocks *map,
int create, int flag);
int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
u64 start, u64 len);
bool should_update_inplace(struct inode *inode, struct f2fs_io_info *fio);
bool should_update_outplace(struct inode *inode, struct f2fs_io_info *fio);
int __f2fs_write_data_pages(struct address_space *mapping,
struct writeback_control *wbc,
enum iostat_type io_type);
bool f2fs_should_update_inplace(struct inode *inode, struct f2fs_io_info *fio);
bool f2fs_should_update_outplace(struct inode *inode, struct f2fs_io_info *fio);
void f2fs_invalidate_page(struct page *page, unsigned int offset,
unsigned int length);
int f2fs_release_page(struct page *page, gfp_t wait);
@ -2978,22 +3049,23 @@ int f2fs_migrate_page(struct address_space *mapping, struct page *newpage,
struct page *page, enum migrate_mode mode);
#endif
bool f2fs_overwrite_io(struct inode *inode, loff_t pos, size_t len);
void f2fs_clear_radix_tree_dirty_tag(struct page *page);
/*
* gc.c
*/
int start_gc_thread(struct f2fs_sb_info *sbi);
void stop_gc_thread(struct f2fs_sb_info *sbi);
block_t start_bidx_of_node(unsigned int node_ofs, struct inode *inode);
int f2fs_start_gc_thread(struct f2fs_sb_info *sbi);
void f2fs_stop_gc_thread(struct f2fs_sb_info *sbi);
block_t f2fs_start_bidx_of_node(unsigned int node_ofs, struct inode *inode);
int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, bool background,
unsigned int segno);
void build_gc_manager(struct f2fs_sb_info *sbi);
void f2fs_build_gc_manager(struct f2fs_sb_info *sbi);
/*
* recovery.c
*/
int recover_fsync_data(struct f2fs_sb_info *sbi, bool check_only);
bool space_for_roll_forward(struct f2fs_sb_info *sbi);
int f2fs_recover_fsync_data(struct f2fs_sb_info *sbi, bool check_only);
bool f2fs_space_for_roll_forward(struct f2fs_sb_info *sbi);
/*
* debug.c
@ -3031,6 +3103,7 @@ struct f2fs_stat_info {
int bg_node_segs, bg_data_segs;
int tot_blks, data_blks, node_blks;
int bg_data_blks, bg_node_blks;
unsigned long long skipped_atomic_files[2];
int curseg[NR_CURSEG_TYPE];
int cursec[NR_CURSEG_TYPE];
int curzone[NR_CURSEG_TYPE];
@ -3197,29 +3270,31 @@ extern const struct inode_operations f2fs_dir_inode_operations;
extern const struct inode_operations f2fs_symlink_inode_operations;
extern const struct inode_operations f2fs_encrypted_symlink_inode_operations;
extern const struct inode_operations f2fs_special_inode_operations;
extern struct kmem_cache *inode_entry_slab;
extern struct kmem_cache *f2fs_inode_entry_slab;
/*
* inline.c
*/
bool f2fs_may_inline_data(struct inode *inode);
bool f2fs_may_inline_dentry(struct inode *inode);
void read_inline_data(struct page *page, struct page *ipage);
void truncate_inline_inode(struct inode *inode, struct page *ipage, u64 from);
void f2fs_do_read_inline_data(struct page *page, struct page *ipage);
void f2fs_truncate_inline_inode(struct inode *inode,
struct page *ipage, u64 from);
int f2fs_read_inline_data(struct inode *inode, struct page *page);
int f2fs_convert_inline_page(struct dnode_of_data *dn, struct page *page);
int f2fs_convert_inline_inode(struct inode *inode);
int f2fs_write_inline_data(struct inode *inode, struct page *page);
bool recover_inline_data(struct inode *inode, struct page *npage);
struct f2fs_dir_entry *find_in_inline_dir(struct inode *dir,
bool f2fs_recover_inline_data(struct inode *inode, struct page *npage);
struct f2fs_dir_entry *f2fs_find_in_inline_dir(struct inode *dir,
struct fscrypt_name *fname, struct page **res_page);
int make_empty_inline_dir(struct inode *inode, struct inode *parent,
int f2fs_make_empty_inline_dir(struct inode *inode, struct inode *parent,
struct page *ipage);
int f2fs_add_inline_entry(struct inode *dir, const struct qstr *new_name,
const struct qstr *orig_name,
struct inode *inode, nid_t ino, umode_t mode);
void f2fs_delete_inline_entry(struct f2fs_dir_entry *dentry, struct page *page,
struct inode *dir, struct inode *inode);
void f2fs_delete_inline_entry(struct f2fs_dir_entry *dentry,
struct page *page, struct inode *dir,
struct inode *inode);
bool f2fs_empty_inline_dir(struct inode *dir);
int f2fs_read_inline_dir(struct file *file, struct dir_context *ctx,
struct fscrypt_str *fstr);
@ -3240,17 +3315,17 @@ void f2fs_leave_shrinker(struct f2fs_sb_info *sbi);
/*
* extent_cache.c
*/
struct rb_entry *__lookup_rb_tree(struct rb_root *root,
struct rb_entry *f2fs_lookup_rb_tree(struct rb_root *root,
struct rb_entry *cached_re, unsigned int ofs);
struct rb_node **__lookup_rb_tree_for_insert(struct f2fs_sb_info *sbi,
struct rb_node **f2fs_lookup_rb_tree_for_insert(struct f2fs_sb_info *sbi,
struct rb_root *root, struct rb_node **parent,
unsigned int ofs);
struct rb_entry *__lookup_rb_tree_ret(struct rb_root *root,
struct rb_entry *f2fs_lookup_rb_tree_ret(struct rb_root *root,
struct rb_entry *cached_re, unsigned int ofs,
struct rb_entry **prev_entry, struct rb_entry **next_entry,
struct rb_node ***insert_p, struct rb_node **insert_parent,
bool force);
bool __check_rb_tree_consistence(struct f2fs_sb_info *sbi,
bool f2fs_check_rb_tree_consistence(struct f2fs_sb_info *sbi,
struct rb_root *root);
unsigned int f2fs_shrink_extent_tree(struct f2fs_sb_info *sbi, int nr_shrink);
bool f2fs_init_extent_tree(struct inode *inode, struct f2fs_extent *i_ext);
@ -3262,9 +3337,9 @@ bool f2fs_lookup_extent_cache(struct inode *inode, pgoff_t pgofs,
void f2fs_update_extent_cache(struct dnode_of_data *dn);
void f2fs_update_extent_cache_range(struct dnode_of_data *dn,
pgoff_t fofs, block_t blkaddr, unsigned int len);
void init_extent_cache_info(struct f2fs_sb_info *sbi);
int __init create_extent_cache(void);
void destroy_extent_cache(void);
void f2fs_init_extent_cache_info(struct f2fs_sb_info *sbi);
int __init f2fs_create_extent_cache(void);
void f2fs_destroy_extent_cache(void);
/*
* sysfs.c

283
fs/f2fs/file.c
View file

@ -98,7 +98,8 @@ static int f2fs_vm_page_mkwrite(struct vm_area_struct *vma,
/* page is wholly or partially inside EOF */
if (((loff_t)(page->index + 1) << PAGE_SHIFT) >
i_size_read(inode)) {
unsigned offset;
loff_t offset;
offset = i_size_read(inode) & ~PAGE_MASK;
zero_user_segment(page, offset, PAGE_SIZE);
}
@ -160,17 +161,18 @@ static inline enum cp_reason_type need_do_checkpoint(struct inode *inode)
cp_reason = CP_SB_NEED_CP;
else if (file_wrong_pino(inode))
cp_reason = CP_WRONG_PINO;
else if (!space_for_roll_forward(sbi))
else if (!f2fs_space_for_roll_forward(sbi))
cp_reason = CP_NO_SPC_ROLL;
else if (!is_checkpointed_node(sbi, F2FS_I(inode)->i_pino))
else if (!f2fs_is_checkpointed_node(sbi, F2FS_I(inode)->i_pino))
cp_reason = CP_NODE_NEED_CP;
else if (test_opt(sbi, FASTBOOT))
cp_reason = CP_FASTBOOT_MODE;
else if (F2FS_OPTION(sbi).active_logs == 2)
cp_reason = CP_SPEC_LOG_NUM;
else if (F2FS_OPTION(sbi).fsync_mode == FSYNC_MODE_STRICT &&
need_dentry_mark(sbi, inode->i_ino) &&
exist_written_data(sbi, F2FS_I(inode)->i_pino, TRANS_DIR_INO))
f2fs_need_dentry_mark(sbi, inode->i_ino) &&
f2fs_exist_written_data(sbi, F2FS_I(inode)->i_pino,
TRANS_DIR_INO))
cp_reason = CP_RECOVER_DIR;
return cp_reason;
@ -181,7 +183,7 @@ static bool need_inode_page_update(struct f2fs_sb_info *sbi, nid_t ino)
struct page *i = find_get_page(NODE_MAPPING(sbi), ino);
bool ret = false;
/* But we need to avoid that there are some inode updates */
if ((i && PageDirty(i)) || need_inode_block_update(sbi, ino))
if ((i && PageDirty(i)) || f2fs_need_inode_block_update(sbi, ino))
ret = true;
f2fs_put_page(i, 0);
return ret;
@ -241,14 +243,14 @@ static int f2fs_do_sync_file(struct file *file, loff_t start, loff_t end,
* if there is no written data, don't waste time to write recovery info.
*/
if (!is_inode_flag_set(inode, FI_APPEND_WRITE) &&
!exist_written_data(sbi, ino, APPEND_INO)) {
!f2fs_exist_written_data(sbi, ino, APPEND_INO)) {
/* it may call write_inode just prior to fsync */
if (need_inode_page_update(sbi, ino))
goto go_write;
if (is_inode_flag_set(inode, FI_UPDATE_WRITE) ||
exist_written_data(sbi, ino, UPDATE_INO))
f2fs_exist_written_data(sbi, ino, UPDATE_INO))
goto flush_out;
goto out;
}
@ -275,7 +277,9 @@ go_write:
goto out;
}
sync_nodes:
ret = fsync_node_pages(sbi, inode, &wbc, atomic);
atomic_inc(&sbi->wb_sync_req[NODE]);
ret = f2fs_fsync_node_pages(sbi, inode, &wbc, atomic);
atomic_dec(&sbi->wb_sync_req[NODE]);
if (ret)
goto out;
@ -285,7 +289,7 @@ sync_nodes:
goto out;
}
if (need_inode_block_update(sbi, ino)) {
if (f2fs_need_inode_block_update(sbi, ino)) {
f2fs_mark_inode_dirty_sync(inode, true);
f2fs_write_inode(inode, NULL);
goto sync_nodes;
@ -300,21 +304,21 @@ sync_nodes:
* given fsync mark.
*/
if (!atomic) {
ret = wait_on_node_pages_writeback(sbi, ino);
ret = f2fs_wait_on_node_pages_writeback(sbi, ino);
if (ret)
goto out;
}
/* once recovery info is written, don't need to tack this */
remove_ino_entry(sbi, ino, APPEND_INO);
f2fs_remove_ino_entry(sbi, ino, APPEND_INO);
clear_inode_flag(inode, FI_APPEND_WRITE);
flush_out:
if (!atomic && F2FS_OPTION(sbi).fsync_mode != FSYNC_MODE_NOBARRIER)
ret = f2fs_issue_flush(sbi, inode->i_ino);
if (!ret) {
remove_ino_entry(sbi, ino, UPDATE_INO);
f2fs_remove_ino_entry(sbi, ino, UPDATE_INO);
clear_inode_flag(inode, FI_UPDATE_WRITE);
remove_ino_entry(sbi, ino, FLUSH_INO);
f2fs_remove_ino_entry(sbi, ino, FLUSH_INO);
}
f2fs_update_time(sbi, REQ_TIME);
out:
@ -333,18 +337,19 @@ int f2fs_sync_file(struct file *file, loff_t start, loff_t end, int datasync)
static pgoff_t __get_first_dirty_index(struct address_space *mapping,
pgoff_t pgofs, int whence)
{
struct pagevec pvec;
struct page *page;
int nr_pages;
if (whence != SEEK_DATA)
return 0;
/* find first dirty page index */
pagevec_init(&pvec, 0);
nr_pages = pagevec_lookup_tag(&pvec, mapping, &pgofs,
PAGECACHE_TAG_DIRTY, 1);
pgofs = nr_pages ? pvec.pages[0]->index : ULONG_MAX;
pagevec_release(&pvec);
nr_pages = find_get_pages_tag(mapping, &pgofs, PAGECACHE_TAG_DIRTY,
1, &page);
if (!nr_pages)
return ULONG_MAX;
pgofs = page->index;
put_page(page);
return pgofs;
}
@ -354,7 +359,7 @@ static bool __found_offset(block_t blkaddr, pgoff_t dirty, pgoff_t pgofs,
switch (whence) {
case SEEK_DATA:
if ((blkaddr == NEW_ADDR && dirty == pgofs) ||
(blkaddr != NEW_ADDR && blkaddr != NULL_ADDR))
is_valid_blkaddr(blkaddr))
return true;
break;
case SEEK_HOLE:
@ -394,13 +399,13 @@ static loff_t f2fs_seek_block(struct file *file, loff_t offset, int whence)
for (; data_ofs < isize; data_ofs = (loff_t)pgofs << PAGE_SHIFT) {
set_new_dnode(&dn, inode, NULL, NULL, 0);
err = get_dnode_of_data(&dn, pgofs, LOOKUP_NODE);
err = f2fs_get_dnode_of_data(&dn, pgofs, LOOKUP_NODE);
if (err && err != -ENOENT) {
goto fail;
} else if (err == -ENOENT) {
/* direct node does not exists */
if (whence == SEEK_DATA) {
pgofs = get_next_page_offset(&dn, pgofs);
pgofs = f2fs_get_next_page_offset(&dn, pgofs);
continue;
} else {
goto found;
@ -414,6 +419,7 @@ static loff_t f2fs_seek_block(struct file *file, loff_t offset, int whence)
dn.ofs_in_node++, pgofs++,
data_ofs = (loff_t)pgofs << PAGE_SHIFT) {
block_t blkaddr;
blkaddr = datablock_addr(dn.inode,
dn.node_page, dn.ofs_in_node);
@ -488,7 +494,7 @@ static int f2fs_file_open(struct inode *inode, struct file *filp)
return dquot_file_open(inode, filp);
}
void truncate_data_blocks_range(struct dnode_of_data *dn, int count)
void f2fs_truncate_data_blocks_range(struct dnode_of_data *dn, int count)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(dn->inode);
struct f2fs_node *raw_node;
@ -504,12 +510,13 @@ void truncate_data_blocks_range(struct dnode_of_data *dn, int count)
for (; count > 0; count--, addr++, dn->ofs_in_node++) {
block_t blkaddr = le32_to_cpu(*addr);
if (blkaddr == NULL_ADDR)
continue;
dn->data_blkaddr = NULL_ADDR;
set_data_blkaddr(dn);
invalidate_blocks(sbi, blkaddr);
f2fs_set_data_blkaddr(dn);
f2fs_invalidate_blocks(sbi, blkaddr);
if (dn->ofs_in_node == 0 && IS_INODE(dn->node_page))
clear_inode_flag(dn->inode, FI_FIRST_BLOCK_WRITTEN);
nr_free++;
@ -521,7 +528,7 @@ void truncate_data_blocks_range(struct dnode_of_data *dn, int count)
* once we invalidate valid blkaddr in range [ofs, ofs + count],
* we will invalidate all blkaddr in the whole range.
*/
fofs = start_bidx_of_node(ofs_of_node(dn->node_page),
fofs = f2fs_start_bidx_of_node(ofs_of_node(dn->node_page),
dn->inode) + ofs;
f2fs_update_extent_cache_range(dn, fofs, 0, len);
dec_valid_block_count(sbi, dn->inode, nr_free);
@ -533,15 +540,15 @@ void truncate_data_blocks_range(struct dnode_of_data *dn, int count)
dn->ofs_in_node, nr_free);
}
void truncate_data_blocks(struct dnode_of_data *dn)
void f2fs_truncate_data_blocks(struct dnode_of_data *dn)
{
truncate_data_blocks_range(dn, ADDRS_PER_BLOCK);
f2fs_truncate_data_blocks_range(dn, ADDRS_PER_BLOCK);
}
static int truncate_partial_data_page(struct inode *inode, u64 from,
bool cache_only)
{
unsigned offset = from & (PAGE_SIZE - 1);
loff_t offset = from & (PAGE_SIZE - 1);
pgoff_t index = from >> PAGE_SHIFT;
struct address_space *mapping = inode->i_mapping;
struct page *page;
@ -557,7 +564,7 @@ static int truncate_partial_data_page(struct inode *inode, u64 from,
return 0;
}
page = get_lock_data_page(inode, index, true);
page = f2fs_get_lock_data_page(inode, index, true);
if (IS_ERR(page))
return PTR_ERR(page) == -ENOENT ? 0 : PTR_ERR(page);
truncate_out:
@ -572,7 +579,7 @@ truncate_out:
return 0;
}
int truncate_blocks(struct inode *inode, u64 from, bool lock)
int f2fs_truncate_blocks(struct inode *inode, u64 from, bool lock)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct dnode_of_data dn;
@ -591,21 +598,21 @@ int truncate_blocks(struct inode *inode, u64 from, bool lock)
if (lock)
f2fs_lock_op(sbi);
ipage = get_node_page(sbi, inode->i_ino);
ipage = f2fs_get_node_page(sbi, inode->i_ino);
if (IS_ERR(ipage)) {
err = PTR_ERR(ipage);
goto out;
}
if (f2fs_has_inline_data(inode)) {
truncate_inline_inode(inode, ipage, from);
f2fs_truncate_inline_inode(inode, ipage, from);
f2fs_put_page(ipage, 1);
truncate_page = true;
goto out;
}
set_new_dnode(&dn, inode, ipage, NULL, 0);
err = get_dnode_of_data(&dn, free_from, LOOKUP_NODE_RA);
err = f2fs_get_dnode_of_data(&dn, free_from, LOOKUP_NODE_RA);
if (err) {
if (err == -ENOENT)
goto free_next;
@ -618,13 +625,13 @@ int truncate_blocks(struct inode *inode, u64 from, bool lock)
f2fs_bug_on(sbi, count < 0);
if (dn.ofs_in_node || IS_INODE(dn.node_page)) {
truncate_data_blocks_range(&dn, count);
f2fs_truncate_data_blocks_range(&dn, count);
free_from += count;
}
f2fs_put_dnode(&dn);
free_next:
err = truncate_inode_blocks(inode, free_from);
err = f2fs_truncate_inode_blocks(inode, free_from);
out:
if (lock)
f2fs_unlock_op(sbi);
@ -663,7 +670,7 @@ int f2fs_truncate(struct inode *inode)
return err;
}
err = truncate_blocks(inode, i_size_read(inode), true);
err = f2fs_truncate_blocks(inode, i_size_read(inode), true);
if (err)
return err;
@ -689,16 +696,16 @@ int f2fs_getattr(struct vfsmount *mnt,
stat->btime.tv_nsec = fi->i_crtime.tv_nsec;
}
flags = fi->i_flags & (FS_FL_USER_VISIBLE | FS_PROJINHERIT_FL);
if (flags & FS_APPEND_FL)
flags = fi->i_flags & F2FS_FL_USER_VISIBLE;
if (flags & F2FS_APPEND_FL)
stat->attributes |= STATX_ATTR_APPEND;
if (flags & FS_COMPR_FL)
if (flags & F2FS_COMPR_FL)
stat->attributes |= STATX_ATTR_COMPRESSED;
if (f2fs_encrypted_inode(inode))
stat->attributes |= STATX_ATTR_ENCRYPTED;
if (flags & FS_IMMUTABLE_FL)
if (flags & F2FS_IMMUTABLE_FL)
stat->attributes |= STATX_ATTR_IMMUTABLE;
if (flags & FS_NODUMP_FL)
if (flags & F2FS_NODUMP_FL)
stat->attributes |= STATX_ATTR_NODUMP;
stat->attributes_mask |= (STATX_ATTR_APPEND |
@ -814,7 +821,7 @@ int f2fs_setattr(struct dentry *dentry, struct iattr *attr)
__setattr_copy(inode, attr);
if (attr->ia_valid & ATTR_MODE) {
err = posix_acl_chmod(inode, get_inode_mode(inode));
err = posix_acl_chmod(inode, f2fs_get_inode_mode(inode));
if (err || is_inode_flag_set(inode, FI_ACL_MODE)) {
inode->i_mode = F2FS_I(inode)->i_acl_mode;
clear_inode_flag(inode, FI_ACL_MODE);
@ -856,7 +863,7 @@ static int fill_zero(struct inode *inode, pgoff_t index,
f2fs_balance_fs(sbi, true);
f2fs_lock_op(sbi);
page = get_new_data_page(inode, NULL, index, false);
page = f2fs_get_new_data_page(inode, NULL, index, false);
f2fs_unlock_op(sbi);
if (IS_ERR(page))
@ -869,7 +876,7 @@ static int fill_zero(struct inode *inode, pgoff_t index,
return 0;
}
int truncate_hole(struct inode *inode, pgoff_t pg_start, pgoff_t pg_end)
int f2fs_truncate_hole(struct inode *inode, pgoff_t pg_start, pgoff_t pg_end)
{
int err;
@ -878,10 +885,11 @@ int truncate_hole(struct inode *inode, pgoff_t pg_start, pgoff_t pg_end)
pgoff_t end_offset, count;
set_new_dnode(&dn, inode, NULL, NULL, 0);
err = get_dnode_of_data(&dn, pg_start, LOOKUP_NODE);
err = f2fs_get_dnode_of_data(&dn, pg_start, LOOKUP_NODE);
if (err) {
if (err == -ENOENT) {
pg_start = get_next_page_offset(&dn, pg_start);
pg_start = f2fs_get_next_page_offset(&dn,
pg_start);
continue;
}
return err;
@ -892,7 +900,7 @@ int truncate_hole(struct inode *inode, pgoff_t pg_start, pgoff_t pg_end)
f2fs_bug_on(F2FS_I_SB(inode), count == 0 || count > end_offset);
truncate_data_blocks_range(&dn, count);
f2fs_truncate_data_blocks_range(&dn, count);
f2fs_put_dnode(&dn);
pg_start += count;
@ -948,7 +956,7 @@ static int punch_hole(struct inode *inode, loff_t offset, loff_t len)
blk_end - 1);
f2fs_lock_op(sbi);
ret = truncate_hole(inode, pg_start, pg_end);
ret = f2fs_truncate_hole(inode, pg_start, pg_end);
f2fs_unlock_op(sbi);
up_write(&F2FS_I(inode)->i_mmap_sem);
}
@ -966,7 +974,7 @@ static int __read_out_blkaddrs(struct inode *inode, block_t *blkaddr,
next_dnode:
set_new_dnode(&dn, inode, NULL, NULL, 0);
ret = get_dnode_of_data(&dn, off, LOOKUP_NODE_RA);
ret = f2fs_get_dnode_of_data(&dn, off, LOOKUP_NODE_RA);
if (ret && ret != -ENOENT) {
return ret;
} else if (ret == -ENOENT) {
@ -983,7 +991,7 @@ next_dnode:
for (i = 0; i < done; i++, blkaddr++, do_replace++, dn.ofs_in_node++) {
*blkaddr = datablock_addr(dn.inode,
dn.node_page, dn.ofs_in_node);
if (!is_checkpointed_data(sbi, *blkaddr)) {
if (!f2fs_is_checkpointed_data(sbi, *blkaddr)) {
if (test_opt(sbi, LFS)) {
f2fs_put_dnode(&dn);
@ -1016,10 +1024,10 @@ static int __roll_back_blkaddrs(struct inode *inode, block_t *blkaddr,
continue;
set_new_dnode(&dn, inode, NULL, NULL, 0);
ret = get_dnode_of_data(&dn, off + i, LOOKUP_NODE_RA);
ret = f2fs_get_dnode_of_data(&dn, off + i, LOOKUP_NODE_RA);
if (ret) {
dec_valid_block_count(sbi, inode, 1);
invalidate_blocks(sbi, *blkaddr);
f2fs_invalidate_blocks(sbi, *blkaddr);
} else {
f2fs_update_data_blkaddr(&dn, *blkaddr);
}
@ -1049,18 +1057,18 @@ static int __clone_blkaddrs(struct inode *src_inode, struct inode *dst_inode,
pgoff_t ilen;
set_new_dnode(&dn, dst_inode, NULL, NULL, 0);
ret = get_dnode_of_data(&dn, dst + i, ALLOC_NODE);
ret = f2fs_get_dnode_of_data(&dn, dst + i, ALLOC_NODE);
if (ret)
return ret;
get_node_info(sbi, dn.nid, &ni);
f2fs_get_node_info(sbi, dn.nid, &ni);
ilen = min((pgoff_t)
ADDRS_PER_PAGE(dn.node_page, dst_inode) -
dn.ofs_in_node, len - i);
do {
dn.data_blkaddr = datablock_addr(dn.inode,
dn.node_page, dn.ofs_in_node);
truncate_data_blocks_range(&dn, 1);
f2fs_truncate_data_blocks_range(&dn, 1);
if (do_replace[i]) {
f2fs_i_blocks_write(src_inode,
@ -1083,10 +1091,11 @@ static int __clone_blkaddrs(struct inode *src_inode, struct inode *dst_inode,
} else {
struct page *psrc, *pdst;
psrc = get_lock_data_page(src_inode, src + i, true);
psrc = f2fs_get_lock_data_page(src_inode,
src + i, true);
if (IS_ERR(psrc))
return PTR_ERR(psrc);
pdst = get_new_data_page(dst_inode, NULL, dst + i,
pdst = f2fs_get_new_data_page(dst_inode, NULL, dst + i,
true);
if (IS_ERR(pdst)) {
f2fs_put_page(psrc, 1);
@ -1097,7 +1106,8 @@ static int __clone_blkaddrs(struct inode *src_inode, struct inode *dst_inode,
f2fs_put_page(pdst, 1);
f2fs_put_page(psrc, 1);
ret = truncate_hole(src_inode, src + i, src + i + 1);
ret = f2fs_truncate_hole(src_inode,
src + i, src + i + 1);
if (ret)
return ret;
i++;
@ -1119,12 +1129,14 @@ static int __exchange_data_block(struct inode *src_inode,
olen = min((pgoff_t)4 * ADDRS_PER_BLOCK, len);
src_blkaddr = f2fs_kvzalloc(F2FS_I_SB(src_inode),
sizeof(block_t) * olen, GFP_KERNEL);
array_size(olen, sizeof(block_t)),
GFP_KERNEL);
if (!src_blkaddr)
return -ENOMEM;
do_replace = f2fs_kvzalloc(F2FS_I_SB(src_inode),
sizeof(int) * olen, GFP_KERNEL);
array_size(olen, sizeof(int)),
GFP_KERNEL);
if (!do_replace) {
kvfree(src_blkaddr);
return -ENOMEM;
@ -1150,7 +1162,7 @@ static int __exchange_data_block(struct inode *src_inode,
return 0;
roll_back:
__roll_back_blkaddrs(src_inode, src_blkaddr, do_replace, src, len);
__roll_back_blkaddrs(src_inode, src_blkaddr, do_replace, src, olen);
kvfree(src_blkaddr);
kvfree(do_replace);
return ret;
@ -1193,7 +1205,7 @@ static int f2fs_collapse_range(struct inode *inode, loff_t offset, loff_t len)
pg_end = (offset + len) >> PAGE_SHIFT;
/* avoid gc operation during block exchange */
down_write(&F2FS_I(inode)->dio_rwsem[WRITE]);
down_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
down_write(&F2FS_I(inode)->i_mmap_sem);
/* write out all dirty pages from offset */
@ -1214,12 +1226,12 @@ static int f2fs_collapse_range(struct inode *inode, loff_t offset, loff_t len)
new_size = i_size_read(inode) - len;
truncate_pagecache(inode, new_size);
ret = truncate_blocks(inode, new_size, true);
ret = f2fs_truncate_blocks(inode, new_size, true);
if (!ret)
f2fs_i_size_write(inode, new_size);
out_unlock:
up_write(&F2FS_I(inode)->i_mmap_sem);
up_write(&F2FS_I(inode)->dio_rwsem[WRITE]);
up_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
return ret;
}
@ -1239,7 +1251,7 @@ static int f2fs_do_zero_range(struct dnode_of_data *dn, pgoff_t start,
}
dn->ofs_in_node = ofs_in_node;
ret = reserve_new_blocks(dn, count);
ret = f2fs_reserve_new_blocks(dn, count);
if (ret)
return ret;
@ -1248,7 +1260,7 @@ static int f2fs_do_zero_range(struct dnode_of_data *dn, pgoff_t start,
dn->data_blkaddr = datablock_addr(dn->inode,
dn->node_page, dn->ofs_in_node);
/*
* reserve_new_blocks will not guarantee entire block
* f2fs_reserve_new_blocks will not guarantee entire block
* allocation.
*/
if (dn->data_blkaddr == NULL_ADDR) {
@ -1256,9 +1268,9 @@ static int f2fs_do_zero_range(struct dnode_of_data *dn, pgoff_t start,
break;
}
if (dn->data_blkaddr != NEW_ADDR) {
invalidate_blocks(sbi, dn->data_blkaddr);
f2fs_invalidate_blocks(sbi, dn->data_blkaddr);
dn->data_blkaddr = NEW_ADDR;
set_data_blkaddr(dn);
f2fs_set_data_blkaddr(dn);
}
}
@ -1324,7 +1336,7 @@ static int f2fs_zero_range(struct inode *inode, loff_t offset, loff_t len,
f2fs_lock_op(sbi);
set_new_dnode(&dn, inode, NULL, NULL, 0);
ret = get_dnode_of_data(&dn, index, ALLOC_NODE);
ret = f2fs_get_dnode_of_data(&dn, index, ALLOC_NODE);
if (ret) {
f2fs_unlock_op(sbi);
goto out;
@ -1395,10 +1407,10 @@ static int f2fs_insert_range(struct inode *inode, loff_t offset, loff_t len)
f2fs_balance_fs(sbi, true);
/* avoid gc operation during block exchange */
down_write(&F2FS_I(inode)->dio_rwsem[WRITE]);
down_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
down_write(&F2FS_I(inode)->i_mmap_sem);
ret = truncate_blocks(inode, i_size_read(inode), true);
ret = f2fs_truncate_blocks(inode, i_size_read(inode), true);
if (ret)
goto out;
@ -1436,7 +1448,7 @@ static int f2fs_insert_range(struct inode *inode, loff_t offset, loff_t len)
f2fs_i_size_write(inode, new_size);
out:
up_write(&F2FS_I(inode)->i_mmap_sem);
up_write(&F2FS_I(inode)->dio_rwsem[WRITE]);
up_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
return ret;
}
@ -1559,13 +1571,13 @@ static int f2fs_release_file(struct inode *inode, struct file *filp)
/* some remained atomic pages should discarded */
if (f2fs_is_atomic_file(inode))
drop_inmem_pages(inode);
f2fs_drop_inmem_pages(inode);
if (f2fs_is_volatile_file(inode)) {
clear_inode_flag(inode, FI_VOLATILE_FILE);
stat_dec_volatile_write(inode);
set_inode_flag(inode, FI_DROP_CACHE);
filemap_fdatawrite(inode->i_mapping);
clear_inode_flag(inode, FI_DROP_CACHE);
clear_inode_flag(inode, FI_VOLATILE_FILE);
stat_dec_volatile_write(inode);
}
return 0;
}
@ -1582,7 +1594,7 @@ static int f2fs_file_flush(struct file *file, fl_owner_t id)
*/
if (f2fs_is_atomic_file(inode) &&
F2FS_I(inode)->inmem_task == current)
drop_inmem_pages(inode);
f2fs_drop_inmem_pages(inode);
return 0;
}
@ -1590,7 +1602,15 @@ static int f2fs_ioc_getflags(struct file *filp, unsigned long arg)
{
struct inode *inode = file_inode(filp);
struct f2fs_inode_info *fi = F2FS_I(inode);
unsigned int flags = fi->i_flags & FS_FL_USER_VISIBLE;
unsigned int flags = fi->i_flags;
if (file_is_encrypt(inode))
flags |= F2FS_ENCRYPT_FL;
if (f2fs_has_inline_data(inode) || f2fs_has_inline_dentry(inode))
flags |= F2FS_INLINE_DATA_FL;
flags &= F2FS_FL_USER_VISIBLE;
return put_user(flags, (int __user *)arg);
}
@ -1624,15 +1644,15 @@ static int f2fs_ioc_setflags(struct file *filp, unsigned long arg)
oldflags = fi->i_flags;
if ((flags ^ oldflags) & (FS_APPEND_FL | FS_IMMUTABLE_FL)) {
if ((flags ^ oldflags) & (F2FS_APPEND_FL | F2FS_IMMUTABLE_FL)) {
if (!capable(CAP_LINUX_IMMUTABLE)) {
ret = -EPERM;
goto unlock_out;
}
}
flags = flags & FS_FL_USER_MODIFIABLE;
flags |= oldflags & ~FS_FL_USER_MODIFIABLE;
flags = flags & (F2FS_FL_USER_MODIFIABLE);
flags |= oldflags & ~(F2FS_FL_USER_MODIFIABLE);
fi->i_flags = flags;
inode->i_ctime = current_time(inode);
@ -1668,6 +1688,8 @@ static int f2fs_ioc_start_atomic_write(struct file *filp)
inode_lock(inode);
down_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
if (f2fs_is_atomic_file(inode))
goto out;
@ -1675,28 +1697,25 @@ static int f2fs_ioc_start_atomic_write(struct file *filp)
if (ret)
goto out;
set_inode_flag(inode, FI_ATOMIC_FILE);
set_inode_flag(inode, FI_HOT_DATA);
f2fs_update_time(F2FS_I_SB(inode), REQ_TIME);
if (!get_dirty_pages(inode))
goto inc_stat;
goto skip_flush;
f2fs_msg(F2FS_I_SB(inode)->sb, KERN_WARNING,
"Unexpected flush for atomic writes: ino=%lu, npages=%u",
inode->i_ino, get_dirty_pages(inode));
ret = filemap_write_and_wait_range(inode->i_mapping, 0, LLONG_MAX);
if (ret) {
clear_inode_flag(inode, FI_ATOMIC_FILE);
clear_inode_flag(inode, FI_HOT_DATA);
if (ret)
goto out;
}
skip_flush:
set_inode_flag(inode, FI_ATOMIC_FILE);
clear_inode_flag(inode, FI_ATOMIC_REVOKE_REQUEST);
f2fs_update_time(F2FS_I_SB(inode), REQ_TIME);
inc_stat:
F2FS_I(inode)->inmem_task = current;
stat_inc_atomic_write(inode);
stat_update_max_atomic_write(inode);
out:
up_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
inode_unlock(inode);
mnt_drop_write_file(filp);
return ret;
@ -1716,27 +1735,33 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp)
inode_lock(inode);
down_write(&F2FS_I(inode)->dio_rwsem[WRITE]);
down_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
if (f2fs_is_volatile_file(inode))
if (f2fs_is_volatile_file(inode)) {
ret = -EINVAL;
goto err_out;
}
if (f2fs_is_atomic_file(inode)) {
ret = commit_inmem_pages(inode);
ret = f2fs_commit_inmem_pages(inode);
if (ret)
goto err_out;
ret = f2fs_do_sync_file(filp, 0, LLONG_MAX, 0, true);
if (!ret) {
clear_inode_flag(inode, FI_ATOMIC_FILE);
clear_inode_flag(inode, FI_HOT_DATA);
F2FS_I(inode)->i_gc_failures[GC_FAILURE_ATOMIC] = 0;
stat_dec_atomic_write(inode);
}
} else {
ret = f2fs_do_sync_file(filp, 0, LLONG_MAX, 1, false);
}
err_out:
up_write(&F2FS_I(inode)->dio_rwsem[WRITE]);
if (is_inode_flag_set(inode, FI_ATOMIC_REVOKE_REQUEST)) {
clear_inode_flag(inode, FI_ATOMIC_REVOKE_REQUEST);
ret = -EINVAL;
}
up_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
inode_unlock(inode);
mnt_drop_write_file(filp);
return ret;
@ -1821,7 +1846,7 @@ static int f2fs_ioc_abort_volatile_write(struct file *filp)
inode_lock(inode);
if (f2fs_is_atomic_file(inode))
drop_inmem_pages(inode);
f2fs_drop_inmem_pages(inode);
if (f2fs_is_volatile_file(inode)) {
clear_inode_flag(inode, FI_VOLATILE_FILE);
stat_dec_volatile_write(inode);
@ -1849,9 +1874,11 @@ static int f2fs_ioc_shutdown(struct file *filp, unsigned long arg)
if (get_user(in, (__u32 __user *)arg))
return -EFAULT;
if (in != F2FS_GOING_DOWN_FULLSYNC) {
ret = mnt_want_write_file(filp);
if (ret)
return ret;
}
switch (in) {
case F2FS_GOING_DOWN_FULLSYNC:
@ -1876,7 +1903,7 @@ static int f2fs_ioc_shutdown(struct file *filp, unsigned long arg)
f2fs_stop_checkpoint(sbi, false);
break;
case F2FS_GOING_DOWN_METAFLUSH:
sync_meta_pages(sbi, META, LONG_MAX, FS_META_IO);
f2fs_sync_meta_pages(sbi, META, LONG_MAX, FS_META_IO);
f2fs_stop_checkpoint(sbi, false);
break;
default:
@ -1884,14 +1911,15 @@ static int f2fs_ioc_shutdown(struct file *filp, unsigned long arg)
goto out;
}
stop_gc_thread(sbi);
stop_discard_thread(sbi);
f2fs_stop_gc_thread(sbi);
f2fs_stop_discard_thread(sbi);
drop_discard_cmd(sbi);
f2fs_drop_discard_cmd(sbi);
clear_opt(sbi, DISCARD);
f2fs_update_time(sbi, REQ_TIME);
out:
if (in != F2FS_GOING_DOWN_FULLSYNC)
mnt_drop_write_file(filp);
return ret;
}
@ -2051,15 +2079,15 @@ static int f2fs_ioc_gc_range(struct file *filp, unsigned long arg)
if (f2fs_readonly(sbi->sb))
return -EROFS;
end = range.start + range.len;
if (range.start < MAIN_BLKADDR(sbi) || end >= MAX_BLKADDR(sbi)) {
return -EINVAL;
}
ret = mnt_want_write_file(filp);
if (ret)
return ret;
end = range.start + range.len;
if (range.start < MAIN_BLKADDR(sbi) || end >= MAX_BLKADDR(sbi)) {
ret = -EINVAL;
goto out;
}
do_more:
if (!range.sync) {
if (!mutex_trylock(&sbi->gc_mutex)) {
@ -2079,7 +2107,7 @@ out:
return ret;
}
static int f2fs_ioc_write_checkpoint(struct file *filp, unsigned long arg)
static int f2fs_ioc_f2fs_write_checkpoint(struct file *filp, unsigned long arg)
{
struct inode *inode = file_inode(filp);
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
@ -2117,7 +2145,7 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
int err;
/* if in-place-update policy is enabled, don't waste time here */
if (should_update_inplace(inode, NULL))
if (f2fs_should_update_inplace(inode, NULL))
return -EINVAL;
pg_start = range->start >> PAGE_SHIFT;
@ -2212,7 +2240,7 @@ do_map:
while (idx < map.m_lblk + map.m_len && cnt < blk_per_seg) {
struct page *page;
page = get_lock_data_page(inode, idx, true);
page = f2fs_get_lock_data_page(inode, idx, true);
if (IS_ERR(page)) {
err = PTR_ERR(page);
goto clear_out;
@ -2323,12 +2351,12 @@ static int f2fs_move_file_range(struct file *file_in, loff_t pos_in,
}
inode_lock(src);
down_write(&F2FS_I(src)->dio_rwsem[WRITE]);
down_write(&F2FS_I(src)->i_gc_rwsem[WRITE]);
if (src != dst) {
ret = -EBUSY;
if (!inode_trylock(dst))
goto out;
if (!down_write_trylock(&F2FS_I(dst)->dio_rwsem[WRITE])) {
if (!down_write_trylock(&F2FS_I(dst)->i_gc_rwsem[WRITE])) {
inode_unlock(dst);
goto out;
}
@ -2390,11 +2418,11 @@ static int f2fs_move_file_range(struct file *file_in, loff_t pos_in,
f2fs_unlock_op(sbi);
out_unlock:
if (src != dst) {
up_write(&F2FS_I(dst)->dio_rwsem[WRITE]);
up_write(&F2FS_I(dst)->i_gc_rwsem[WRITE]);
inode_unlock(dst);
}
out:
up_write(&F2FS_I(src)->dio_rwsem[WRITE]);
up_write(&F2FS_I(src)->i_gc_rwsem[WRITE]);
inode_unlock(src);
return ret;
}
@ -2521,12 +2549,14 @@ int f2fs_pin_file_control(struct inode *inode, bool inc)
/* Use i_gc_failures for normal file as a risk signal. */
if (inc)
f2fs_i_gc_failures_write(inode, fi->i_gc_failures + 1);
f2fs_i_gc_failures_write(inode,
fi->i_gc_failures[GC_FAILURE_PIN] + 1);
if (fi->i_gc_failures > sbi->gc_pin_file_threshold) {
if (fi->i_gc_failures[GC_FAILURE_PIN] > sbi->gc_pin_file_threshold) {
f2fs_msg(sbi->sb, KERN_WARNING,
"%s: Enable GC = ino %lx after %x GC trials\n",
__func__, inode->i_ino, fi->i_gc_failures);
__func__, inode->i_ino,
fi->i_gc_failures[GC_FAILURE_PIN]);
clear_inode_flag(inode, FI_PIN_FILE);
return -EAGAIN;
}
@ -2557,14 +2587,14 @@ static int f2fs_ioc_set_pin_file(struct file *filp, unsigned long arg)
inode_lock(inode);
if (should_update_outplace(inode, NULL)) {
if (f2fs_should_update_outplace(inode, NULL)) {
ret = -EINVAL;
goto out;
}
if (!pin) {
clear_inode_flag(inode, FI_PIN_FILE);
F2FS_I(inode)->i_gc_failures = 1;
F2FS_I(inode)->i_gc_failures[GC_FAILURE_PIN] = 1;
goto done;
}
@ -2577,7 +2607,7 @@ static int f2fs_ioc_set_pin_file(struct file *filp, unsigned long arg)
goto out;
set_inode_flag(inode, FI_PIN_FILE);
ret = F2FS_I(inode)->i_gc_failures;
ret = F2FS_I(inode)->i_gc_failures[GC_FAILURE_PIN];
done:
f2fs_update_time(F2FS_I_SB(inode), REQ_TIME);
out:
@ -2592,7 +2622,7 @@ static int f2fs_ioc_get_pin_file(struct file *filp, unsigned long arg)
__u32 pin = 0;
if (is_inode_flag_set(inode, FI_PIN_FILE))
pin = F2FS_I(inode)->i_gc_failures;
pin = F2FS_I(inode)->i_gc_failures[GC_FAILURE_PIN];
return put_user(pin, (u32 __user *)arg);
}
@ -2616,9 +2646,9 @@ int f2fs_precache_extents(struct inode *inode)
while (map.m_lblk < end) {
map.m_len = end - map.m_lblk;
down_write(&fi->dio_rwsem[WRITE]);
down_write(&fi->i_gc_rwsem[WRITE]);
err = f2fs_map_blocks(inode, &map, 0, F2FS_GET_BLOCK_PRECACHE);
up_write(&fi->dio_rwsem[WRITE]);
up_write(&fi->i_gc_rwsem[WRITE]);
if (err)
return err;
@ -2670,7 +2700,7 @@ long f2fs_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
case F2FS_IOC_GARBAGE_COLLECT_RANGE:
return f2fs_ioc_gc_range(filp, arg);
case F2FS_IOC_WRITE_CHECKPOINT:
return f2fs_ioc_write_checkpoint(filp, arg);
return f2fs_ioc_f2fs_write_checkpoint(filp, arg);
case F2FS_IOC_DEFRAGMENT:
return f2fs_ioc_defragment(filp, arg);
case F2FS_IOC_MOVE_RANGE:
@ -2694,7 +2724,6 @@ static ssize_t f2fs_file_write_iter(struct kiocb *iocb, struct iov_iter *from)
{
struct file *file = iocb->ki_filp;
struct inode *inode = file_inode(file);
struct blk_plug plug;
ssize_t ret;
if (unlikely(f2fs_cp_error(F2FS_I_SB(inode))))
@ -2724,6 +2753,8 @@ static ssize_t f2fs_file_write_iter(struct kiocb *iocb, struct iov_iter *from)
iov_iter_count(from)) ||
f2fs_has_inline_data(inode) ||
f2fs_force_buffered_io(inode, WRITE)) {
clear_inode_flag(inode,
FI_NO_PREALLOC);
inode_unlock(inode);
return -EAGAIN;
}
@ -2739,9 +2770,7 @@ static ssize_t f2fs_file_write_iter(struct kiocb *iocb, struct iov_iter *from)
return err;
}
}
blk_start_plug(&plug);
ret = __generic_file_write_iter(iocb, from);
blk_finish_plug(&plug);
clear_inode_flag(inode, FI_NO_PREALLOC);
/* if we couldn't write data, we should deallocate blocks. */

168
fs/f2fs/gc.c
View file

@ -76,7 +76,7 @@ static int gc_thread_func(void *data)
* invalidated soon after by user update or deletion.
* So, I'd like to wait some time to collect dirty segments.
*/
if (gc_th->gc_urgent) {
if (sbi->gc_mode == GC_URGENT) {
wait_ms = gc_th->urgent_sleep_time;
mutex_lock(&sbi->gc_mutex);
goto do_gc;
@ -114,7 +114,7 @@ next:
return 0;
}
int start_gc_thread(struct f2fs_sb_info *sbi)
int f2fs_start_gc_thread(struct f2fs_sb_info *sbi)
{
struct f2fs_gc_kthread *gc_th;
dev_t dev = sbi->sb->s_bdev->bd_dev;
@ -131,8 +131,6 @@ int start_gc_thread(struct f2fs_sb_info *sbi)
gc_th->max_sleep_time = DEF_GC_THREAD_MAX_SLEEP_TIME;
gc_th->no_gc_sleep_time = DEF_GC_THREAD_NOGC_SLEEP_TIME;
gc_th->gc_idle = 0;
gc_th->gc_urgent = 0;
gc_th->gc_wake= 0;
sbi->gc_thread = gc_th;
@ -148,7 +146,7 @@ out:
return err;
}
void stop_gc_thread(struct f2fs_sb_info *sbi)
void f2fs_stop_gc_thread(struct f2fs_sb_info *sbi)
{
struct f2fs_gc_kthread *gc_th = sbi->gc_thread;
if (!gc_th)
@ -158,21 +156,19 @@ void stop_gc_thread(struct f2fs_sb_info *sbi)
sbi->gc_thread = NULL;
}
static int select_gc_type(struct f2fs_gc_kthread *gc_th, int gc_type)
static int select_gc_type(struct f2fs_sb_info *sbi, int gc_type)
{
int gc_mode = (gc_type == BG_GC) ? GC_CB : GC_GREEDY;
if (!gc_th)
return gc_mode;
if (gc_th->gc_idle) {
if (gc_th->gc_idle == 1)
switch (sbi->gc_mode) {
case GC_IDLE_CB:
gc_mode = GC_CB;
else if (gc_th->gc_idle == 2)
break;
case GC_IDLE_GREEDY:
case GC_URGENT:
gc_mode = GC_GREEDY;
break;
}
if (gc_th->gc_urgent)
gc_mode = GC_GREEDY;
return gc_mode;
}
@ -187,7 +183,7 @@ static void select_policy(struct f2fs_sb_info *sbi, int gc_type,
p->max_search = dirty_i->nr_dirty[type];
p->ofs_unit = 1;
} else {
p->gc_mode = select_gc_type(sbi->gc_thread, gc_type);
p->gc_mode = select_gc_type(sbi, gc_type);
p->dirty_segmap = dirty_i->dirty_segmap[DIRTY];
p->max_search = dirty_i->nr_dirty[DIRTY];
p->ofs_unit = sbi->segs_per_sec;
@ -195,7 +191,7 @@ static void select_policy(struct f2fs_sb_info *sbi, int gc_type,
/* we need to check every dirty segments in the FG_GC case */
if (gc_type != FG_GC &&
(sbi->gc_thread && !sbi->gc_thread->gc_urgent) &&
(sbi->gc_mode != GC_URGENT) &&
p->max_search > sbi->max_victim_search)
p->max_search = sbi->max_victim_search;
@ -234,10 +230,6 @@ static unsigned int check_bg_victims(struct f2fs_sb_info *sbi)
for_each_set_bit(secno, dirty_i->victim_secmap, MAIN_SECS(sbi)) {
if (sec_usage_check(sbi, secno))
continue;
if (no_fggc_candidate(sbi, secno))
continue;
clear_bit(secno, dirty_i->victim_secmap);
return GET_SEG_FROM_SEC(sbi, secno);
}
@ -377,9 +369,6 @@ static int get_victim_by_default(struct f2fs_sb_info *sbi,
goto next;
if (gc_type == BG_GC && test_bit(secno, dirty_i->victim_secmap))
goto next;
if (gc_type == FG_GC && p.alloc_mode == LFS &&
no_fggc_candidate(sbi, secno))
goto next;
cost = get_gc_cost(sbi, segno, &p);
@ -440,7 +429,7 @@ static void add_gc_inode(struct gc_inode_list *gc_list, struct inode *inode)
iput(inode);
return;
}
new_ie = f2fs_kmem_cache_alloc(inode_entry_slab, GFP_NOFS);
new_ie = f2fs_kmem_cache_alloc(f2fs_inode_entry_slab, GFP_NOFS);
new_ie->inode = inode;
f2fs_radix_tree_insert(&gc_list->iroot, inode->i_ino, new_ie);
@ -454,7 +443,7 @@ static void put_gc_inode(struct gc_inode_list *gc_list)
radix_tree_delete(&gc_list->iroot, ie->inode->i_ino);
iput(ie->inode);
list_del(&ie->list);
kmem_cache_free(inode_entry_slab, ie);
kmem_cache_free(f2fs_inode_entry_slab, ie);
}
}
@ -484,12 +473,16 @@ static void gc_node_segment(struct f2fs_sb_info *sbi,
block_t start_addr;
int off;
int phase = 0;
bool fggc = (gc_type == FG_GC);
start_addr = START_BLOCK(sbi, segno);
next_step:
entry = sum;
if (fggc && phase == 2)
atomic_inc(&sbi->wb_sync_req[NODE]);
for (off = 0; off < sbi->blocks_per_seg; off++, entry++) {
nid_t nid = le32_to_cpu(entry->nid);
struct page *node_page;
@ -503,39 +496,42 @@ next_step:
continue;
if (phase == 0) {
ra_meta_pages(sbi, NAT_BLOCK_OFFSET(nid), 1,
f2fs_ra_meta_pages(sbi, NAT_BLOCK_OFFSET(nid), 1,
META_NAT, true);
continue;
}
if (phase == 1) {
ra_node_page(sbi, nid);
f2fs_ra_node_page(sbi, nid);
continue;
}
/* phase == 2 */
node_page = get_node_page(sbi, nid);
node_page = f2fs_get_node_page(sbi, nid);
if (IS_ERR(node_page))
continue;
/* block may become invalid during get_node_page */
/* block may become invalid during f2fs_get_node_page */
if (check_valid_map(sbi, segno, off) == 0) {
f2fs_put_page(node_page, 1);
continue;
}
get_node_info(sbi, nid, &ni);
f2fs_get_node_info(sbi, nid, &ni);
if (ni.blk_addr != start_addr + off) {
f2fs_put_page(node_page, 1);
continue;
}
move_node_page(node_page, gc_type);
f2fs_move_node_page(node_page, gc_type);
stat_inc_node_blk_count(sbi, 1, gc_type);
}
if (++phase < 3)
goto next_step;
if (fggc)
atomic_dec(&sbi->wb_sync_req[NODE]);
}
/*
@ -545,7 +541,7 @@ next_step:
* as indirect or double indirect node blocks, are given, it must be a caller's
* bug.
*/
block_t start_bidx_of_node(unsigned int node_ofs, struct inode *inode)
block_t f2fs_start_bidx_of_node(unsigned int node_ofs, struct inode *inode)
{
unsigned int indirect_blks = 2 * NIDS_PER_BLOCK + 4;
unsigned int bidx;
@ -576,11 +572,11 @@ static bool is_alive(struct f2fs_sb_info *sbi, struct f2fs_summary *sum,
nid = le32_to_cpu(sum->nid);
ofs_in_node = le16_to_cpu(sum->ofs_in_node);
node_page = get_node_page(sbi, nid);
node_page = f2fs_get_node_page(sbi, nid);
if (IS_ERR(node_page))
return false;
get_node_info(sbi, nid, dni);
f2fs_get_node_info(sbi, nid, dni);
if (sum->version != dni->version) {
f2fs_msg(sbi->sb, KERN_WARNING,
@ -603,7 +599,7 @@ static bool is_alive(struct f2fs_sb_info *sbi, struct f2fs_summary *sum,
* This can be used to move blocks, aka LBAs, directly on disk.
*/
static void move_data_block(struct inode *inode, block_t bidx,
unsigned int segno, int off)
int gc_type, unsigned int segno, int off)
{
struct f2fs_io_info fio = {
.sbi = F2FS_I_SB(inode),
@ -614,6 +610,7 @@ static void move_data_block(struct inode *inode, block_t bidx,
.op_flags = REQ_SYNC,
.encrypted_page = NULL,
.in_list = false,
.retry = false,
};
struct dnode_of_data dn;
struct f2fs_summary sum;
@ -621,6 +618,7 @@ static void move_data_block(struct inode *inode, block_t bidx,
struct page *page;
block_t newaddr;
int err;
bool lfs_mode = test_opt(fio.sbi, LFS);
/* do not read out */
page = f2fs_grab_cache_page(inode->i_mapping, bidx, false);
@ -630,8 +628,11 @@ static void move_data_block(struct inode *inode, block_t bidx,
if (!check_valid_map(F2FS_I_SB(inode), segno, off))
goto out;
if (f2fs_is_atomic_file(inode))
if (f2fs_is_atomic_file(inode)) {
F2FS_I(inode)->i_gc_failures[GC_FAILURE_ATOMIC]++;
F2FS_I_SB(inode)->skipped_atomic_files[gc_type]++;
goto out;
}
if (f2fs_is_pinned_file(inode)) {
f2fs_pin_file_control(inode, true);
@ -639,7 +640,7 @@ static void move_data_block(struct inode *inode, block_t bidx,
}
set_new_dnode(&dn, inode, NULL, NULL, 0);
err = get_dnode_of_data(&dn, bidx, LOOKUP_NODE);
err = f2fs_get_dnode_of_data(&dn, bidx, LOOKUP_NODE);
if (err)
goto out;
@ -654,14 +655,17 @@ static void move_data_block(struct inode *inode, block_t bidx,
*/
f2fs_wait_on_page_writeback(page, DATA, true);
get_node_info(fio.sbi, dn.nid, &ni);
f2fs_get_node_info(fio.sbi, dn.nid, &ni);
set_summary(&sum, dn.nid, dn.ofs_in_node, ni.version);
/* read page */
fio.page = page;
fio.new_blkaddr = fio.old_blkaddr = dn.data_blkaddr;
allocate_data_block(fio.sbi, NULL, fio.old_blkaddr, &newaddr,
if (lfs_mode)
down_write(&fio.sbi->io_order_lock);
f2fs_allocate_data_block(fio.sbi, NULL, fio.old_blkaddr, &newaddr,
&sum, CURSEG_COLD_DATA, NULL, false);
fio.encrypted_page = f2fs_pagecache_get_page(META_MAPPING(fio.sbi),
@ -701,8 +705,8 @@ static void move_data_block(struct inode *inode, block_t bidx,
fio.op = REQ_OP_WRITE;
fio.op_flags = REQ_SYNC | REQ_NOIDLE;
fio.new_blkaddr = newaddr;
err = f2fs_submit_page_write(&fio);
if (err) {
f2fs_submit_page_write(&fio);
if (fio.retry) {
if (PageWriteback(fio.encrypted_page))
end_page_writeback(fio.encrypted_page);
goto put_page_out;
@ -717,8 +721,10 @@ static void move_data_block(struct inode *inode, block_t bidx,
put_page_out:
f2fs_put_page(fio.encrypted_page, 1);
recover_block:
if (lfs_mode)
up_write(&fio.sbi->io_order_lock);
if (err)
__f2fs_replace_block(fio.sbi, &sum, newaddr, fio.old_blkaddr,
f2fs_do_replace_block(fio.sbi, &sum, newaddr, fio.old_blkaddr,
true, true);
put_out:
f2fs_put_dnode(&dn);
@ -731,15 +737,18 @@ static void move_data_page(struct inode *inode, block_t bidx, int gc_type,
{
struct page *page;
page = get_lock_data_page(inode, bidx, true);
page = f2fs_get_lock_data_page(inode, bidx, true);
if (IS_ERR(page))
return;
if (!check_valid_map(F2FS_I_SB(inode), segno, off))
goto out;
if (f2fs_is_atomic_file(inode))
if (f2fs_is_atomic_file(inode)) {
F2FS_I(inode)->i_gc_failures[GC_FAILURE_ATOMIC]++;
F2FS_I_SB(inode)->skipped_atomic_files[gc_type]++;
goto out;
}
if (f2fs_is_pinned_file(inode)) {
if (gc_type == FG_GC)
f2fs_pin_file_control(inode, true);
@ -773,16 +782,21 @@ retry:
f2fs_wait_on_page_writeback(page, DATA, true);
if (clear_page_dirty_for_io(page)) {
inode_dec_dirty_pages(inode);
remove_dirty_inode(inode);
f2fs_remove_dirty_inode(inode);
}
set_cold_data(page);
err = do_write_data_page(&fio);
if (err == -ENOMEM && is_dirty) {
err = f2fs_do_write_data_page(&fio);
if (err) {
clear_cold_data(page);
if (err == -ENOMEM) {
congestion_wait(BLK_RW_ASYNC, HZ/50);
goto retry;
}
if (is_dirty)
set_page_dirty(page);
}
}
out:
f2fs_put_page(page, 1);
@ -825,13 +839,13 @@ next_step:
continue;
if (phase == 0) {
ra_meta_pages(sbi, NAT_BLOCK_OFFSET(nid), 1,
f2fs_ra_meta_pages(sbi, NAT_BLOCK_OFFSET(nid), 1,
META_NAT, true);
continue;
}
if (phase == 1) {
ra_node_page(sbi, nid);
f2fs_ra_node_page(sbi, nid);
continue;
}
@ -840,7 +854,7 @@ next_step:
continue;
if (phase == 2) {
ra_node_page(sbi, dni.ino);
f2fs_ra_node_page(sbi, dni.ino);
continue;
}
@ -858,16 +872,16 @@ next_step:
}
if (!down_write_trylock(
&F2FS_I(inode)->dio_rwsem[WRITE])) {
&F2FS_I(inode)->i_gc_rwsem[WRITE])) {
iput(inode);
continue;
}
start_bidx = start_bidx_of_node(nofs, inode);
data_page = get_read_data_page(inode,
start_bidx = f2fs_start_bidx_of_node(nofs, inode);
data_page = f2fs_get_read_data_page(inode,
start_bidx + ofs_in_node, REQ_RAHEAD,
true);
up_write(&F2FS_I(inode)->dio_rwsem[WRITE]);
up_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
if (IS_ERR(data_page)) {
iput(inode);
continue;
@ -885,11 +899,11 @@ next_step:
bool locked = false;
if (S_ISREG(inode->i_mode)) {
if (!down_write_trylock(&fi->dio_rwsem[READ]))
if (!down_write_trylock(&fi->i_gc_rwsem[READ]))
continue;
if (!down_write_trylock(
&fi->dio_rwsem[WRITE])) {
up_write(&fi->dio_rwsem[READ]);
&fi->i_gc_rwsem[WRITE])) {
up_write(&fi->i_gc_rwsem[READ]);
continue;
}
locked = true;
@ -898,17 +912,18 @@ next_step:
inode_dio_wait(inode);
}
start_bidx = start_bidx_of_node(nofs, inode)
start_bidx = f2fs_start_bidx_of_node(nofs, inode)
+ ofs_in_node;
if (f2fs_post_read_required(inode))
move_data_block(inode, start_bidx, segno, off);
move_data_block(inode, start_bidx, gc_type,
segno, off);
else
move_data_page(inode, start_bidx, gc_type,
segno, off);
if (locked) {
up_write(&fi->dio_rwsem[WRITE]);
up_write(&fi->dio_rwsem[READ]);
up_write(&fi->i_gc_rwsem[WRITE]);
up_write(&fi->i_gc_rwsem[READ]);
}
stat_inc_data_blk_count(sbi, 1, gc_type);
@ -947,12 +962,12 @@ static int do_garbage_collect(struct f2fs_sb_info *sbi,
/* readahead multi ssa blocks those have contiguous address */
if (sbi->segs_per_sec > 1)
ra_meta_pages(sbi, GET_SUM_BLOCK(sbi, segno),
f2fs_ra_meta_pages(sbi, GET_SUM_BLOCK(sbi, segno),
sbi->segs_per_sec, META_SSA, true);
/* reference all summary page */
while (segno < end_segno) {
sum_page = get_sum_page(sbi, segno++);
sum_page = f2fs_get_sum_page(sbi, segno++);
unlock_page(sum_page);
}
@ -1018,6 +1033,8 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync,
.ilist = LIST_HEAD_INIT(gc_list.ilist),
.iroot = RADIX_TREE_INIT(GFP_NOFS),
};
unsigned long long last_skipped = sbi->skipped_atomic_files[FG_GC];
unsigned int skipped_round = 0, round = 0;
trace_f2fs_gc_begin(sbi->sb, sync, background,
get_pages(sbi, F2FS_DIRTY_NODES),
@ -1046,7 +1063,7 @@ gc_more:
* secure free segments which doesn't need fggc any more.
*/
if (prefree_segments(sbi)) {
ret = write_checkpoint(sbi, &cpc);
ret = f2fs_write_checkpoint(sbi, &cpc);
if (ret)
goto stop;
}
@ -1069,17 +1086,27 @@ gc_more:
sec_freed++;
total_freed += seg_freed;
if (gc_type == FG_GC) {
if (sbi->skipped_atomic_files[FG_GC] > last_skipped)
skipped_round++;
last_skipped = sbi->skipped_atomic_files[FG_GC];
round++;
}
if (gc_type == FG_GC)
sbi->cur_victim_sec = NULL_SEGNO;
if (!sync) {
if (has_not_enough_free_secs(sbi, sec_freed, 0)) {
if (skipped_round > MAX_SKIP_ATOMIC_COUNT &&
skipped_round * 2 >= round)
f2fs_drop_inmem_pages_all(sbi, true);
segno = NULL_SEGNO;
goto gc_more;
}
if (gc_type == FG_GC)
ret = write_checkpoint(sbi, &cpc);
ret = f2fs_write_checkpoint(sbi, &cpc);
}
stop:
SIT_I(sbi)->last_victim[ALLOC_NEXT] = 0;
@ -1103,19 +1130,10 @@ stop:
return ret;
}
void build_gc_manager(struct f2fs_sb_info *sbi)
void f2fs_build_gc_manager(struct f2fs_sb_info *sbi)
{
u64 main_count, resv_count, ovp_count;
DIRTY_I(sbi)->v_ops = &default_v_ops;
/* threshold of # of valid blocks in a section for victims of FG_GC */
main_count = SM_I(sbi)->main_segments << sbi->log_blocks_per_seg;
resv_count = SM_I(sbi)->reserved_segments << sbi->log_blocks_per_seg;
ovp_count = SM_I(sbi)->ovp_segments << sbi->log_blocks_per_seg;
sbi->fggc_threshold = div64_u64((main_count - ovp_count) *
BLKS_PER_SEC(sbi), (main_count - resv_count));
sbi->gc_pin_file_threshold = DEF_GC_FAILED_PINNED_FILES;
/* give warm/cold data area from slower device */

2
fs/f2fs/gc.h
View file

@ -36,8 +36,6 @@ struct f2fs_gc_kthread {
unsigned int no_gc_sleep_time;
/* for changing gc mode */
unsigned int gc_idle;
unsigned int gc_urgent;
unsigned int gc_wake;
};

75
fs/f2fs/inline.c
View file

@ -43,7 +43,7 @@ bool f2fs_may_inline_dentry(struct inode *inode)
return true;
}
void read_inline_data(struct page *page, struct page *ipage)
void f2fs_do_read_inline_data(struct page *page, struct page *ipage)
{
struct inode *inode = page->mapping->host;
void *src_addr, *dst_addr;
@ -65,7 +65,8 @@ void read_inline_data(struct page *page, struct page *ipage)
SetPageUptodate(page);
}
void truncate_inline_inode(struct inode *inode, struct page *ipage, u64 from)
void f2fs_truncate_inline_inode(struct inode *inode,
struct page *ipage, u64 from)
{
void *addr;
@ -97,7 +98,7 @@ int f2fs_read_inline_data(struct inode *inode, struct page *page)
path, current->comm);
}
ipage = get_node_page(F2FS_I_SB(inode), inode->i_ino);
ipage = f2fs_get_node_page(F2FS_I_SB(inode), inode->i_ino);
if (IS_ERR(ipage)) {
trace_android_fs_dataread_end(inode, page_offset(page),
PAGE_SIZE);
@ -115,7 +116,7 @@ int f2fs_read_inline_data(struct inode *inode, struct page *page)
if (page->index)
zero_user_segment(page, 0, PAGE_SIZE);
else
read_inline_data(page, ipage);
f2fs_do_read_inline_data(page, ipage);
if (!PageUptodate(page))
SetPageUptodate(page);
@ -149,7 +150,7 @@ int f2fs_convert_inline_page(struct dnode_of_data *dn, struct page *page)
f2fs_bug_on(F2FS_P_SB(page), PageWriteback(page));
read_inline_data(page, dn->inode_page);
f2fs_do_read_inline_data(page, dn->inode_page);
set_page_dirty(page);
/* clear dirty state */
@ -160,18 +161,18 @@ int f2fs_convert_inline_page(struct dnode_of_data *dn, struct page *page)
ClearPageError(page);
fio.old_blkaddr = dn->data_blkaddr;
set_inode_flag(dn->inode, FI_HOT_DATA);
write_data_page(dn, &fio);
f2fs_outplace_write_data(dn, &fio);
f2fs_wait_on_page_writeback(page, DATA, true);
if (dirty) {
inode_dec_dirty_pages(dn->inode);
remove_dirty_inode(dn->inode);
f2fs_remove_dirty_inode(dn->inode);
}
/* this converted inline_data should be recovered. */
set_inode_flag(dn->inode, FI_APPEND_WRITE);
/* clear inline data and flag after data writeback */
truncate_inline_inode(dn->inode, dn->inode_page, 0);
f2fs_truncate_inline_inode(dn->inode, dn->inode_page, 0);
clear_inline_node(dn->inode_page);
clear_out:
stat_dec_inline_inode(dn->inode);
@ -196,7 +197,7 @@ int f2fs_convert_inline_inode(struct inode *inode)
f2fs_lock_op(sbi);
ipage = get_node_page(sbi, inode->i_ino);
ipage = f2fs_get_node_page(sbi, inode->i_ino);
if (IS_ERR(ipage)) {
err = PTR_ERR(ipage);
goto out;
@ -222,12 +223,10 @@ int f2fs_write_inline_data(struct inode *inode, struct page *page)
{
void *src_addr, *dst_addr;
struct dnode_of_data dn;
struct address_space *mapping = page_mapping(page);
unsigned long flags;
int err;
set_new_dnode(&dn, inode, NULL, NULL, 0);
err = get_dnode_of_data(&dn, 0, LOOKUP_NODE);
err = f2fs_get_dnode_of_data(&dn, 0, LOOKUP_NODE);
if (err)
return err;
@ -245,10 +244,7 @@ int f2fs_write_inline_data(struct inode *inode, struct page *page)
kunmap_atomic(src_addr);
set_page_dirty(dn.inode_page);
spin_lock_irqsave(&mapping->tree_lock, flags);
radix_tree_tag_clear(&mapping->page_tree, page_index(page),
PAGECACHE_TAG_DIRTY);
spin_unlock_irqrestore(&mapping->tree_lock, flags);
f2fs_clear_radix_tree_dirty_tag(page);
set_inode_flag(inode, FI_APPEND_WRITE);
set_inode_flag(inode, FI_DATA_EXIST);
@ -258,7 +254,7 @@ int f2fs_write_inline_data(struct inode *inode, struct page *page)
return 0;
}
bool recover_inline_data(struct inode *inode, struct page *npage)
bool f2fs_recover_inline_data(struct inode *inode, struct page *npage)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct f2fs_inode *ri = NULL;
@ -279,7 +275,7 @@ bool recover_inline_data(struct inode *inode, struct page *npage)
if (f2fs_has_inline_data(inode) &&
ri && (ri->i_inline & F2FS_INLINE_DATA)) {
process_inline:
ipage = get_node_page(sbi, inode->i_ino);
ipage = f2fs_get_node_page(sbi, inode->i_ino);
f2fs_bug_on(sbi, IS_ERR(ipage));
f2fs_wait_on_page_writeback(ipage, NODE, true);
@ -297,20 +293,20 @@ process_inline:
}
if (f2fs_has_inline_data(inode)) {
ipage = get_node_page(sbi, inode->i_ino);
ipage = f2fs_get_node_page(sbi, inode->i_ino);
f2fs_bug_on(sbi, IS_ERR(ipage));
truncate_inline_inode(inode, ipage, 0);
f2fs_truncate_inline_inode(inode, ipage, 0);
clear_inode_flag(inode, FI_INLINE_DATA);
f2fs_put_page(ipage, 1);
} else if (ri && (ri->i_inline & F2FS_INLINE_DATA)) {
if (truncate_blocks(inode, 0, false))
if (f2fs_truncate_blocks(inode, 0, false))
return false;
goto process_inline;
}
return false;
}
struct f2fs_dir_entry *find_in_inline_dir(struct inode *dir,
struct f2fs_dir_entry *f2fs_find_in_inline_dir(struct inode *dir,
struct fscrypt_name *fname, struct page **res_page)
{
struct f2fs_sb_info *sbi = F2FS_SB(dir->i_sb);
@ -321,7 +317,7 @@ struct f2fs_dir_entry *find_in_inline_dir(struct inode *dir,
void *inline_dentry;
f2fs_hash_t namehash;
ipage = get_node_page(sbi, dir->i_ino);
ipage = f2fs_get_node_page(sbi, dir->i_ino);
if (IS_ERR(ipage)) {
*res_page = ipage;
return NULL;
@ -332,7 +328,7 @@ struct f2fs_dir_entry *find_in_inline_dir(struct inode *dir,
inline_dentry = inline_data_addr(dir, ipage);
make_dentry_ptr_inline(dir, &d, inline_dentry);
de = find_target_dentry(fname, namehash, NULL, &d);
de = f2fs_find_target_dentry(fname, namehash, NULL, &d);
unlock_page(ipage);
if (de)
*res_page = ipage;
@ -342,7 +338,7 @@ struct f2fs_dir_entry *find_in_inline_dir(struct inode *dir,
return de;
}
int make_empty_inline_dir(struct inode *inode, struct inode *parent,
int f2fs_make_empty_inline_dir(struct inode *inode, struct inode *parent,
struct page *ipage)
{
struct f2fs_dentry_ptr d;
@ -351,7 +347,7 @@ int make_empty_inline_dir(struct inode *inode, struct inode *parent,
inline_dentry = inline_data_addr(inode, ipage);
make_dentry_ptr_inline(inode, &d, inline_dentry);
do_make_empty_dir(inode, parent, &d);
f2fs_do_make_empty_dir(inode, parent, &d);
set_page_dirty(ipage);
@ -386,7 +382,6 @@ static int f2fs_move_inline_dirents(struct inode *dir, struct page *ipage,
goto out;
f2fs_wait_on_page_writeback(page, DATA, true);
zero_user_segment(page, MAX_INLINE_DATA(dir), PAGE_SIZE);
dentry_blk = page_address(page);
@ -410,7 +405,7 @@ static int f2fs_move_inline_dirents(struct inode *dir, struct page *ipage,
set_page_dirty(page);
/* clear inline dir and flag after data writeback */
truncate_inline_inode(dir, ipage, 0);
f2fs_truncate_inline_inode(dir, ipage, 0);
stat_dec_inline_dir(dir);
clear_inode_flag(dir, FI_INLINE_DENTRY);
@ -453,7 +448,7 @@ static int f2fs_add_inline_entries(struct inode *dir, void *inline_dentry)
new_name.len = le16_to_cpu(de->name_len);
ino = le32_to_cpu(de->ino);
fake_mode = get_de_type(de) << S_SHIFT;
fake_mode = f2fs_get_de_type(de) << S_SHIFT;
err = f2fs_add_regular_entry(dir, &new_name, NULL, NULL,
ino, fake_mode);
@ -465,8 +460,8 @@ static int f2fs_add_inline_entries(struct inode *dir, void *inline_dentry)
return 0;
punch_dentry_pages:
truncate_inode_pages(&dir->i_data, 0);
truncate_blocks(dir, 0, false);
remove_dirty_inode(dir);
f2fs_truncate_blocks(dir, 0, false);
f2fs_remove_dirty_inode(dir);
return err;
}
@ -484,7 +479,7 @@ static int f2fs_move_rehashed_dirents(struct inode *dir, struct page *ipage,
}
memcpy(backup_dentry, inline_dentry, MAX_INLINE_DATA(dir));
truncate_inline_inode(dir, ipage, 0);
f2fs_truncate_inline_inode(dir, ipage, 0);
unlock_page(ipage);
@ -533,14 +528,14 @@ int f2fs_add_inline_entry(struct inode *dir, const struct qstr *new_name,
struct page *page = NULL;
int err = 0;
ipage = get_node_page(sbi, dir->i_ino);
ipage = f2fs_get_node_page(sbi, dir->i_ino);
if (IS_ERR(ipage))
return PTR_ERR(ipage);
inline_dentry = inline_data_addr(dir, ipage);
make_dentry_ptr_inline(dir, &d, inline_dentry);
bit_pos = room_for_filename(d.bitmap, slots, d.max);
bit_pos = f2fs_room_for_filename(d.bitmap, slots, d.max);
if (bit_pos >= d.max) {
err = f2fs_convert_inline_dir(dir, ipage, inline_dentry);
if (err)
@ -551,7 +546,7 @@ int f2fs_add_inline_entry(struct inode *dir, const struct qstr *new_name,
if (inode) {
down_write(&F2FS_I(inode)->i_sem);
page = init_inode_metadata(inode, dir, new_name,
page = f2fs_init_inode_metadata(inode, dir, new_name,
orig_name, ipage);
if (IS_ERR(page)) {
err = PTR_ERR(page);
@ -572,7 +567,7 @@ int f2fs_add_inline_entry(struct inode *dir, const struct qstr *new_name,
f2fs_put_page(page, 1);
}
update_parent_metadata(dir, inode, 0);
f2fs_update_parent_metadata(dir, inode, 0);
fail:
if (inode)
up_write(&F2FS_I(inode)->i_sem);
@ -618,7 +613,7 @@ bool f2fs_empty_inline_dir(struct inode *dir)
void *inline_dentry;
struct f2fs_dentry_ptr d;
ipage = get_node_page(sbi, dir->i_ino);
ipage = f2fs_get_node_page(sbi, dir->i_ino);
if (IS_ERR(ipage))
return false;
@ -649,7 +644,7 @@ int f2fs_read_inline_dir(struct file *file, struct dir_context *ctx,
if (ctx->pos == d.max)
return 0;
ipage = get_node_page(F2FS_I_SB(inode), inode->i_ino);
ipage = f2fs_get_node_page(F2FS_I_SB(inode), inode->i_ino);
if (IS_ERR(ipage))
return PTR_ERR(ipage);
@ -675,7 +670,7 @@ int f2fs_inline_data_fiemap(struct inode *inode,
struct page *ipage;
int err = 0;
ipage = get_node_page(F2FS_I_SB(inode), inode->i_ino);
ipage = f2fs_get_node_page(F2FS_I_SB(inode), inode->i_ino);
if (IS_ERR(ipage))
return PTR_ERR(ipage);
@ -691,7 +686,7 @@ int f2fs_inline_data_fiemap(struct inode *inode,
ilen = start + len;
ilen -= start;
get_node_info(F2FS_I_SB(inode), inode->i_ino, &ni);
f2fs_get_node_info(F2FS_I_SB(inode), inode->i_ino, &ni);
byteaddr = (__u64)ni.blk_addr << inode->i_sb->s_blocksize_bits;
byteaddr += (char *)inline_data_addr(inode, ipage) -
(char *)F2FS_INODE(ipage);

113
fs/f2fs/inode.c
View file

@ -36,15 +36,15 @@ void f2fs_set_inode_flags(struct inode *inode)
unsigned int flags = F2FS_I(inode)->i_flags;
unsigned int new_fl = 0;
if (flags & FS_SYNC_FL)
if (flags & F2FS_SYNC_FL)
new_fl |= S_SYNC;
if (flags & FS_APPEND_FL)
if (flags & F2FS_APPEND_FL)
new_fl |= S_APPEND;
if (flags & FS_IMMUTABLE_FL)
if (flags & F2FS_IMMUTABLE_FL)
new_fl |= S_IMMUTABLE;
if (flags & FS_NOATIME_FL)
if (flags & F2FS_NOATIME_FL)
new_fl |= S_NOATIME;
if (flags & FS_DIRSYNC_FL)
if (flags & F2FS_DIRSYNC_FL)
new_fl |= S_DIRSYNC;
if (f2fs_encrypted_inode(inode))
new_fl |= S_ENCRYPTED;
@ -72,7 +72,7 @@ static bool __written_first_block(struct f2fs_inode *ri)
{
block_t addr = le32_to_cpu(ri->i_addr[offset_in_addr(ri)]);
if (addr != NEW_ADDR && addr != NULL_ADDR)
if (is_valid_blkaddr(addr))
return true;
return false;
}
@ -117,7 +117,6 @@ static void __recover_inline_status(struct inode *inode, struct page *ipage)
static bool f2fs_enable_inode_chksum(struct f2fs_sb_info *sbi, struct page *page)
{
struct f2fs_inode *ri = &F2FS_NODE(page)->i;
int extra_isize = le32_to_cpu(ri->i_extra_isize);
if (!f2fs_sb_has_inode_chksum(sbi->sb))
return false;
@ -125,7 +124,8 @@ static bool f2fs_enable_inode_chksum(struct f2fs_sb_info *sbi, struct page *page
if (!RAW_IS_INODE(F2FS_NODE(page)) || !(ri->i_inline & F2FS_EXTRA_ATTR))
return false;
if (!F2FS_FITS_IN_INODE(ri, extra_isize, i_inode_checksum))
if (!F2FS_FITS_IN_INODE(ri, le16_to_cpu(ri->i_extra_isize),
i_inode_checksum))
return false;
return true;
@ -185,6 +185,21 @@ void f2fs_inode_chksum_set(struct f2fs_sb_info *sbi, struct page *page)
ri->i_inode_checksum = cpu_to_le32(f2fs_inode_chksum(sbi, page));
}
static bool sanity_check_inode(struct inode *inode)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
if (f2fs_sb_has_flexible_inline_xattr(sbi->sb)
&& !f2fs_has_extra_attr(inode)) {
set_sbi_flag(sbi, SBI_NEED_FSCK);
f2fs_msg(sbi->sb, KERN_WARNING,
"%s: corrupted inode ino=%lx, run fsck to fix.",
__func__, inode->i_ino);
return false;
}
return true;
}
static int do_read_inode(struct inode *inode)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
@ -194,14 +209,10 @@ static int do_read_inode(struct inode *inode)
projid_t i_projid;
/* Check if ino is within scope */
if (check_nid_range(sbi, inode->i_ino)) {
f2fs_msg(inode->i_sb, KERN_ERR, "bad inode number: %lu",
(unsigned long) inode->i_ino);
WARN_ON(1);
if (f2fs_check_nid_range(sbi, inode->i_ino))
return -EINVAL;
}
node_page = get_node_page(sbi, inode->i_ino);
node_page = f2fs_get_node_page(sbi, inode->i_ino);
if (IS_ERR(node_page))
return PTR_ERR(node_page);
@ -221,8 +232,11 @@ static int do_read_inode(struct inode *inode)
inode->i_ctime.tv_nsec = le32_to_cpu(ri->i_ctime_nsec);
inode->i_mtime.tv_nsec = le32_to_cpu(ri->i_mtime_nsec);
inode->i_generation = le32_to_cpu(ri->i_generation);
if (S_ISDIR(inode->i_mode))
fi->i_current_depth = le32_to_cpu(ri->i_current_depth);
else if (S_ISREG(inode->i_mode))
fi->i_gc_failures[GC_FAILURE_PIN] =
le16_to_cpu(ri->i_gc_failures);
fi->i_xattr_nid = le32_to_cpu(ri->i_xattr_nid);
fi->i_flags = le32_to_cpu(ri->i_flags);
fi->flags = 0;
@ -239,7 +253,6 @@ static int do_read_inode(struct inode *inode)
le16_to_cpu(ri->i_extra_isize) : 0;
if (f2fs_sb_has_flexible_inline_xattr(sbi->sb)) {
f2fs_bug_on(sbi, !f2fs_has_extra_attr(inode));
fi->i_inline_xattr_size = le16_to_cpu(ri->i_inline_xattr_size);
} else if (f2fs_has_inline_xattr(inode) ||
f2fs_has_inline_dentry(inode)) {
@ -265,10 +278,10 @@ static int do_read_inode(struct inode *inode)
if (__written_first_block(ri))
set_inode_flag(inode, FI_FIRST_BLOCK_WRITTEN);
if (!need_inode_block_update(sbi, inode->i_ino))
if (!f2fs_need_inode_block_update(sbi, inode->i_ino))
fi->last_disk_size = inode->i_size;
if (fi->i_flags & FS_PROJINHERIT_FL)
if (fi->i_flags & F2FS_PROJINHERIT_FL)
set_inode_flag(inode, FI_PROJ_INHERIT);
if (f2fs_has_extra_attr(inode) && f2fs_sb_has_project_quota(sbi->sb) &&
@ -317,13 +330,17 @@ struct inode *f2fs_iget(struct super_block *sb, unsigned long ino)
ret = do_read_inode(inode);
if (ret)
goto bad_inode;
if (!sanity_check_inode(inode)) {
ret = -EINVAL;
goto bad_inode;
}
make_now:
if (ino == F2FS_NODE_INO(sbi)) {
inode->i_mapping->a_ops = &f2fs_node_aops;
mapping_set_gfp_mask(inode->i_mapping, GFP_F2FS_ZERO);
mapping_set_gfp_mask(inode->i_mapping, GFP_NOFS);
} else if (ino == F2FS_META_INO(sbi)) {
inode->i_mapping->a_ops = &f2fs_meta_aops;
mapping_set_gfp_mask(inode->i_mapping, GFP_F2FS_ZERO);
mapping_set_gfp_mask(inode->i_mapping, GFP_NOFS);
} else if (S_ISREG(inode->i_mode)) {
inode->i_op = &f2fs_file_inode_operations;
inode->i_fop = &f2fs_file_operations;
@ -373,7 +390,7 @@ retry:
return inode;
}
void update_inode(struct inode *inode, struct page *node_page)
void f2fs_update_inode(struct inode *inode, struct page *node_page)
{
struct f2fs_inode *ri;
struct extent_tree *et = F2FS_I(inode)->extent_tree;
@ -408,7 +425,12 @@ void update_inode(struct inode *inode, struct page *node_page)
ri->i_atime_nsec = cpu_to_le32(inode->i_atime.tv_nsec);
ri->i_ctime_nsec = cpu_to_le32(inode->i_ctime.tv_nsec);
ri->i_mtime_nsec = cpu_to_le32(inode->i_mtime.tv_nsec);
ri->i_current_depth = cpu_to_le32(F2FS_I(inode)->i_current_depth);
if (S_ISDIR(inode->i_mode))
ri->i_current_depth =
cpu_to_le32(F2FS_I(inode)->i_current_depth);
else if (S_ISREG(inode->i_mode))
ri->i_gc_failures =
cpu_to_le16(F2FS_I(inode)->i_gc_failures[GC_FAILURE_PIN]);
ri->i_xattr_nid = cpu_to_le32(F2FS_I(inode)->i_xattr_nid);
ri->i_flags = cpu_to_le32(F2FS_I(inode)->i_flags);
ri->i_pino = cpu_to_le32(F2FS_I(inode)->i_pino);
@ -454,12 +476,12 @@ void update_inode(struct inode *inode, struct page *node_page)
F2FS_I(inode)->i_disk_time[3] = F2FS_I(inode)->i_crtime;
}
void update_inode_page(struct inode *inode)
void f2fs_update_inode_page(struct inode *inode)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct page *node_page;
retry:
node_page = get_node_page(sbi, inode->i_ino);
node_page = f2fs_get_node_page(sbi, inode->i_ino);
if (IS_ERR(node_page)) {
int err = PTR_ERR(node_page);
if (err == -ENOMEM) {
@ -470,7 +492,7 @@ retry:
}
return;
}
update_inode(inode, node_page);
f2fs_update_inode(inode, node_page);
f2fs_put_page(node_page, 1);
}
@ -489,7 +511,7 @@ int f2fs_write_inode(struct inode *inode, struct writeback_control *wbc)
* We need to balance fs here to prevent from producing dirty node pages
* during the urgent cleaning time when runing out of free sections.
*/
update_inode_page(inode);
f2fs_update_inode_page(inode);
if (wbc && wbc->nr_to_write)
f2fs_balance_fs(sbi, true);
return 0;
@ -506,7 +528,7 @@ void f2fs_evict_inode(struct inode *inode)
/* some remained atomic pages should discarded */
if (f2fs_is_atomic_file(inode))
drop_inmem_pages(inode);
f2fs_drop_inmem_pages(inode);
trace_f2fs_evict_inode(inode);
truncate_inode_pages_final(&inode->i_data);
@ -516,7 +538,7 @@ void f2fs_evict_inode(struct inode *inode)
goto out_clear;
f2fs_bug_on(sbi, get_dirty_pages(inode));
remove_dirty_inode(inode);
f2fs_remove_dirty_inode(inode);
f2fs_destroy_extent_tree(inode);
@ -525,9 +547,9 @@ void f2fs_evict_inode(struct inode *inode)
dquot_initialize(inode);
remove_ino_entry(sbi, inode->i_ino, APPEND_INO);
remove_ino_entry(sbi, inode->i_ino, UPDATE_INO);
remove_ino_entry(sbi, inode->i_ino, FLUSH_INO);
f2fs_remove_ino_entry(sbi, inode->i_ino, APPEND_INO);
f2fs_remove_ino_entry(sbi, inode->i_ino, UPDATE_INO);
f2fs_remove_ino_entry(sbi, inode->i_ino, FLUSH_INO);
sb_start_intwrite(inode->i_sb);
set_inode_flag(inode, FI_NO_ALLOC);
@ -544,7 +566,7 @@ retry:
#endif
if (!err) {
f2fs_lock_op(sbi);
err = remove_inode_page(inode);
err = f2fs_remove_inode_page(inode);
f2fs_unlock_op(sbi);
if (err == -ENOENT)
err = 0;
@ -557,7 +579,7 @@ retry:
}
if (err)
update_inode_page(inode);
f2fs_update_inode_page(inode);
dquot_free_inode(inode);
sb_end_intwrite(inode->i_sb);
no_delete:
@ -580,16 +602,19 @@ no_delete:
invalidate_mapping_pages(NODE_MAPPING(sbi), xnid, xnid);
if (inode->i_nlink) {
if (is_inode_flag_set(inode, FI_APPEND_WRITE))
add_ino_entry(sbi, inode->i_ino, APPEND_INO);
f2fs_add_ino_entry(sbi, inode->i_ino, APPEND_INO);
if (is_inode_flag_set(inode, FI_UPDATE_WRITE))
add_ino_entry(sbi, inode->i_ino, UPDATE_INO);
f2fs_add_ino_entry(sbi, inode->i_ino, UPDATE_INO);
}
if (is_inode_flag_set(inode, FI_FREE_NID)) {
alloc_nid_failed(sbi, inode->i_ino);
f2fs_alloc_nid_failed(sbi, inode->i_ino);
clear_inode_flag(inode, FI_FREE_NID);
} else {
f2fs_bug_on(sbi, err &&
!exist_written_data(sbi, inode->i_ino, ORPHAN_INO));
/*
* If xattr nid is corrupted, we can reach out error condition,
* err & !f2fs_exist_written_data(sbi, inode->i_ino, ORPHAN_INO)).
* In that case, f2fs_check_nid_range() is enough to give a clue.
*/
}
out_clear:
fscrypt_put_encryption_info(inode, NULL);
@ -597,7 +622,7 @@ out_clear:
}
/* caller should call f2fs_lock_op() */
void handle_failed_inode(struct inode *inode)
void f2fs_handle_failed_inode(struct inode *inode)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct node_info ni;
@ -612,7 +637,7 @@ void handle_failed_inode(struct inode *inode)
* we must call this to avoid inode being remained as dirty, resulting
* in a panic when flushing dirty inodes in gdirty_list.
*/
update_inode_page(inode);
f2fs_update_inode_page(inode);
f2fs_inode_synced(inode);
/* don't make bad inode, since it becomes a regular file. */
@ -623,18 +648,18 @@ void handle_failed_inode(struct inode *inode)
* so we can prevent losing this orphan when encoutering checkpoint
* and following suddenly power-off.
*/
get_node_info(sbi, inode->i_ino, &ni);
f2fs_get_node_info(sbi, inode->i_ino, &ni);
if (ni.blk_addr != NULL_ADDR) {
int err = acquire_orphan_inode(sbi);
int err = f2fs_acquire_orphan_inode(sbi);
if (err) {
set_sbi_flag(sbi, SBI_NEED_FSCK);
f2fs_msg(sbi->sb, KERN_WARNING,
"Too many orphan inodes, run fsck to fix.");
} else {
add_orphan_inode(inode);
f2fs_add_orphan_inode(inode);
}
alloc_nid_done(sbi, inode->i_ino);
f2fs_alloc_nid_done(sbi, inode->i_ino);
} else {
set_inode_flag(inode, FI_FREE_NID);
}

67
fs/f2fs/namei.c
View file

@ -37,7 +37,7 @@ static struct inode *f2fs_new_inode(struct inode *dir, umode_t mode)
return ERR_PTR(-ENOMEM);
f2fs_lock_op(sbi);
if (!alloc_nid(sbi, &ino)) {
if (!f2fs_alloc_nid(sbi, &ino)) {
f2fs_unlock_op(sbi);
err = -ENOSPC;
goto fail;
@ -54,6 +54,9 @@ static struct inode *f2fs_new_inode(struct inode *dir, umode_t mode)
F2FS_I(inode)->i_crtime = current_time(inode);
inode->i_generation = sbi->s_next_generation++;
if (S_ISDIR(inode->i_mode))
F2FS_I(inode)->i_current_depth = 1;
err = insert_inode_locked(inode);
if (err) {
err = -EINVAL;
@ -61,7 +64,7 @@ static struct inode *f2fs_new_inode(struct inode *dir, umode_t mode)
}
if (f2fs_sb_has_project_quota(sbi->sb) &&
(F2FS_I(dir)->i_flags & FS_PROJINHERIT_FL))
(F2FS_I(dir)->i_flags & F2FS_PROJINHERIT_FL))
F2FS_I(inode)->i_projid = F2FS_I(dir)->i_projid;
else
F2FS_I(inode)->i_projid = make_kprojid(&init_user_ns,
@ -116,9 +119,9 @@ static struct inode *f2fs_new_inode(struct inode *dir, umode_t mode)
f2fs_mask_flags(mode, F2FS_I(dir)->i_flags & F2FS_FL_INHERITED);
if (S_ISDIR(inode->i_mode))
F2FS_I(inode)->i_flags |= FS_INDEX_FL;
F2FS_I(inode)->i_flags |= F2FS_INDEX_FL;
if (F2FS_I(inode)->i_flags & FS_PROJINHERIT_FL)
if (F2FS_I(inode)->i_flags & F2FS_PROJINHERIT_FL)
set_inode_flag(inode, FI_PROJ_INHERIT);
trace_f2fs_new_inode(inode, 0);
@ -193,7 +196,7 @@ static inline void set_file_temperature(struct f2fs_sb_info *sbi, struct inode *
up_read(&sbi->sb_lock);
}
int update_extension_list(struct f2fs_sb_info *sbi, const char *name,
int f2fs_update_extension_list(struct f2fs_sb_info *sbi, const char *name,
bool hot, bool set)
{
__u8 (*extlist)[F2FS_EXTENSION_LEN] = sbi->raw_super->extension_list;
@ -292,7 +295,7 @@ static int f2fs_create(struct inode *dir, struct dentry *dentry, umode_t mode,
goto out;
f2fs_unlock_op(sbi);
alloc_nid_done(sbi, ino);
f2fs_alloc_nid_done(sbi, ino);
d_instantiate(dentry, inode);
unlock_new_inode(inode);
@ -303,7 +306,7 @@ static int f2fs_create(struct inode *dir, struct dentry *dentry, umode_t mode,
f2fs_balance_fs(sbi, true);
return 0;
out:
handle_failed_inode(inode);
f2fs_handle_failed_inode(inode);
return err;
}
@ -398,7 +401,7 @@ static int __recover_dot_dentries(struct inode *dir, nid_t pino)
err = PTR_ERR(page);
goto out;
} else {
err = __f2fs_add_link(dir, &dot, NULL, dir->i_ino, S_IFDIR);
err = f2fs_do_add_link(dir, &dot, NULL, dir->i_ino, S_IFDIR);
if (err)
goto out;
}
@ -409,7 +412,7 @@ static int __recover_dot_dentries(struct inode *dir, nid_t pino)
else if (IS_ERR(page))
err = PTR_ERR(page);
else
err = __f2fs_add_link(dir, &dotdot, NULL, pino, S_IFDIR);
err = f2fs_do_add_link(dir, &dotdot, NULL, pino, S_IFDIR);
out:
if (!err)
clear_inode_flag(dir, FI_INLINE_DOTS);
@ -521,7 +524,7 @@ static int f2fs_unlink(struct inode *dir, struct dentry *dentry)
f2fs_balance_fs(sbi, true);
f2fs_lock_op(sbi);
err = acquire_orphan_inode(sbi);
err = f2fs_acquire_orphan_inode(sbi);
if (err) {
f2fs_unlock_op(sbi);
f2fs_put_page(page, 0);
@ -583,9 +586,9 @@ static int f2fs_symlink(struct inode *dir, struct dentry *dentry,
f2fs_lock_op(sbi);
err = f2fs_add_link(dentry, inode);
if (err)
goto out_handle_failed_inode;
goto out_f2fs_handle_failed_inode;
f2fs_unlock_op(sbi);
alloc_nid_done(sbi, inode->i_ino);
f2fs_alloc_nid_done(sbi, inode->i_ino);
err = fscrypt_encrypt_symlink(inode, symname, len, &disk_link);
if (err)
@ -619,8 +622,8 @@ err_out:
f2fs_balance_fs(sbi, true);
goto out_free_encrypted_link;
out_handle_failed_inode:
handle_failed_inode(inode);
out_f2fs_handle_failed_inode:
f2fs_handle_failed_inode(inode);
out_free_encrypted_link:
if (disk_link.name != (unsigned char *)symname)
kfree(disk_link.name);
@ -656,7 +659,7 @@ static int f2fs_mkdir(struct inode *dir, struct dentry *dentry, umode_t mode)
goto out_fail;
f2fs_unlock_op(sbi);
alloc_nid_done(sbi, inode->i_ino);
f2fs_alloc_nid_done(sbi, inode->i_ino);
d_instantiate(dentry, inode);
unlock_new_inode(inode);
@ -669,7 +672,7 @@ static int f2fs_mkdir(struct inode *dir, struct dentry *dentry, umode_t mode)
out_fail:
clear_inode_flag(inode, FI_INC_LINK);
handle_failed_inode(inode);
f2fs_handle_failed_inode(inode);
return err;
}
@ -708,7 +711,7 @@ static int f2fs_mknod(struct inode *dir, struct dentry *dentry,
goto out;
f2fs_unlock_op(sbi);
alloc_nid_done(sbi, inode->i_ino);
f2fs_alloc_nid_done(sbi, inode->i_ino);
d_instantiate(dentry, inode);
unlock_new_inode(inode);
@ -719,7 +722,7 @@ static int f2fs_mknod(struct inode *dir, struct dentry *dentry,
f2fs_balance_fs(sbi, true);
return 0;
out:
handle_failed_inode(inode);
f2fs_handle_failed_inode(inode);
return err;
}
@ -748,7 +751,7 @@ static int __f2fs_tmpfile(struct inode *dir, struct dentry *dentry,
}
f2fs_lock_op(sbi);
err = acquire_orphan_inode(sbi);
err = f2fs_acquire_orphan_inode(sbi);
if (err)
goto out;
@ -760,8 +763,8 @@ static int __f2fs_tmpfile(struct inode *dir, struct dentry *dentry,
* add this non-linked tmpfile to orphan list, in this way we could
* remove all unused data of tmpfile after abnormal power-off.
*/
add_orphan_inode(inode);
alloc_nid_done(sbi, inode->i_ino);
f2fs_add_orphan_inode(inode);
f2fs_alloc_nid_done(sbi, inode->i_ino);
if (whiteout) {
f2fs_i_links_write(inode, false);
@ -777,9 +780,9 @@ static int __f2fs_tmpfile(struct inode *dir, struct dentry *dentry,
return 0;
release_out:
release_orphan_inode(sbi);
f2fs_release_orphan_inode(sbi);
out:
handle_failed_inode(inode);
f2fs_handle_failed_inode(inode);
return err;
}
@ -886,7 +889,7 @@ static int f2fs_rename(struct inode *old_dir, struct dentry *old_dentry,
f2fs_lock_op(sbi);
err = acquire_orphan_inode(sbi);
err = f2fs_acquire_orphan_inode(sbi);
if (err)
goto put_out_dir;
@ -900,9 +903,9 @@ static int f2fs_rename(struct inode *old_dir, struct dentry *old_dentry,
up_write(&F2FS_I(new_inode)->i_sem);
if (!new_inode->i_nlink)
add_orphan_inode(new_inode);
f2fs_add_orphan_inode(new_inode);
else
release_orphan_inode(sbi);
f2fs_release_orphan_inode(sbi);
} else {
f2fs_balance_fs(sbi, true);
@ -970,8 +973,12 @@ static int f2fs_rename(struct inode *old_dir, struct dentry *old_dentry,
f2fs_put_page(old_dir_page, 0);
f2fs_i_links_write(old_dir, false);
}
if (F2FS_OPTION(sbi).fsync_mode == FSYNC_MODE_STRICT)
add_ino_entry(sbi, new_dir->i_ino, TRANS_DIR_INO);
if (F2FS_OPTION(sbi).fsync_mode == FSYNC_MODE_STRICT) {
f2fs_add_ino_entry(sbi, new_dir->i_ino, TRANS_DIR_INO);
if (S_ISDIR(old_inode->i_mode))
f2fs_add_ino_entry(sbi, old_inode->i_ino,
TRANS_DIR_INO);
}
f2fs_unlock_op(sbi);
@ -1122,8 +1129,8 @@ static int f2fs_cross_rename(struct inode *old_dir, struct dentry *old_dentry,
f2fs_mark_inode_dirty_sync(new_dir, false);
if (F2FS_OPTION(sbi).fsync_mode == FSYNC_MODE_STRICT) {
add_ino_entry(sbi, old_dir->i_ino, TRANS_DIR_INO);
add_ino_entry(sbi, new_dir->i_ino, TRANS_DIR_INO);
f2fs_add_ino_entry(sbi, old_dir->i_ino, TRANS_DIR_INO);
f2fs_add_ino_entry(sbi, new_dir->i_ino, TRANS_DIR_INO);
}
f2fs_unlock_op(sbi);

347
fs/f2fs/node.c
View file

@ -23,13 +23,28 @@
#include "trace.h"
#include <trace/events/f2fs.h>
#define on_build_free_nids(nmi) mutex_is_locked(&(nm_i)->build_lock)
#define on_f2fs_build_free_nids(nmi) mutex_is_locked(&(nm_i)->build_lock)
static struct kmem_cache *nat_entry_slab;
static struct kmem_cache *free_nid_slab;
static struct kmem_cache *nat_entry_set_slab;
bool available_free_memory(struct f2fs_sb_info *sbi, int type)
/*
* Check whether the given nid is within node id range.
*/
int f2fs_check_nid_range(struct f2fs_sb_info *sbi, nid_t nid)
{
if (unlikely(nid < F2FS_ROOT_INO(sbi) || nid >= NM_I(sbi)->max_nid)) {
set_sbi_flag(sbi, SBI_NEED_FSCK);
f2fs_msg(sbi->sb, KERN_WARNING,
"%s: out-of-range nid=%x, run fsck to fix.",
__func__, nid);
return -EINVAL;
}
return 0;
}
bool f2fs_available_free_memory(struct f2fs_sb_info *sbi, int type)
{
struct f2fs_nm_info *nm_i = NM_I(sbi);
struct sysinfo val;
@ -87,18 +102,10 @@ bool available_free_memory(struct f2fs_sb_info *sbi, int type)
static void clear_node_page_dirty(struct page *page)
{
struct address_space *mapping = page->mapping;
unsigned int long flags;
if (PageDirty(page)) {
spin_lock_irqsave(&mapping->tree_lock, flags);
radix_tree_tag_clear(&mapping->page_tree,
page_index(page),
PAGECACHE_TAG_DIRTY);
spin_unlock_irqrestore(&mapping->tree_lock, flags);
f2fs_clear_radix_tree_dirty_tag(page);
clear_page_dirty_for_io(page);
dec_page_count(F2FS_M_SB(mapping), F2FS_DIRTY_NODES);
dec_page_count(F2FS_P_SB(page), F2FS_DIRTY_NODES);
}
ClearPageUptodate(page);
}
@ -106,7 +113,7 @@ static void clear_node_page_dirty(struct page *page)
static struct page *get_current_nat_page(struct f2fs_sb_info *sbi, nid_t nid)
{
pgoff_t index = current_nat_addr(sbi, nid);
return get_meta_page(sbi, index);
return f2fs_get_meta_page(sbi, index);
}
static struct page *get_next_nat_page(struct f2fs_sb_info *sbi, nid_t nid)
@ -123,8 +130,8 @@ static struct page *get_next_nat_page(struct f2fs_sb_info *sbi, nid_t nid)
dst_off = next_nat_addr(sbi, src_off);
/* get current nat block page with lock */
src_page = get_meta_page(sbi, src_off);
dst_page = grab_meta_page(sbi, dst_off);
src_page = f2fs_get_meta_page(sbi, src_off);
dst_page = f2fs_grab_meta_page(sbi, dst_off);
f2fs_bug_on(sbi, PageDirty(src_page));
src_addr = page_address(src_page);
@ -260,7 +267,7 @@ static unsigned int __gang_lookup_nat_set(struct f2fs_nm_info *nm_i,
start, nr);
}
int need_dentry_mark(struct f2fs_sb_info *sbi, nid_t nid)
int f2fs_need_dentry_mark(struct f2fs_sb_info *sbi, nid_t nid)
{
struct f2fs_nm_info *nm_i = NM_I(sbi);
struct nat_entry *e;
@ -277,7 +284,7 @@ int need_dentry_mark(struct f2fs_sb_info *sbi, nid_t nid)
return need;
}
bool is_checkpointed_node(struct f2fs_sb_info *sbi, nid_t nid)
bool f2fs_is_checkpointed_node(struct f2fs_sb_info *sbi, nid_t nid)
{
struct f2fs_nm_info *nm_i = NM_I(sbi);
struct nat_entry *e;
@ -291,7 +298,7 @@ bool is_checkpointed_node(struct f2fs_sb_info *sbi, nid_t nid)
return is_cp;
}
bool need_inode_block_update(struct f2fs_sb_info *sbi, nid_t ino)
bool f2fs_need_inode_block_update(struct f2fs_sb_info *sbi, nid_t ino)
{
struct f2fs_nm_info *nm_i = NM_I(sbi);
struct nat_entry *e;
@ -364,8 +371,7 @@ static void set_node_addr(struct f2fs_sb_info *sbi, struct node_info *ni,
new_blkaddr == NULL_ADDR);
f2fs_bug_on(sbi, nat_get_blkaddr(e) == NEW_ADDR &&
new_blkaddr == NEW_ADDR);
f2fs_bug_on(sbi, nat_get_blkaddr(e) != NEW_ADDR &&
nat_get_blkaddr(e) != NULL_ADDR &&
f2fs_bug_on(sbi, is_valid_blkaddr(nat_get_blkaddr(e)) &&
new_blkaddr == NEW_ADDR);
/* increment version no as node is removed */
@ -376,7 +382,7 @@ static void set_node_addr(struct f2fs_sb_info *sbi, struct node_info *ni,
/* change address */
nat_set_blkaddr(e, new_blkaddr);
if (new_blkaddr == NEW_ADDR || new_blkaddr == NULL_ADDR)
if (!is_valid_blkaddr(new_blkaddr))
set_nat_flag(e, IS_CHECKPOINTED, false);
__set_nat_cache_dirty(nm_i, e);
@ -391,7 +397,7 @@ static void set_node_addr(struct f2fs_sb_info *sbi, struct node_info *ni,
up_write(&nm_i->nat_tree_lock);
}
int try_to_free_nats(struct f2fs_sb_info *sbi, int nr_shrink)
int f2fs_try_to_free_nats(struct f2fs_sb_info *sbi, int nr_shrink)
{
struct f2fs_nm_info *nm_i = NM_I(sbi);
int nr = nr_shrink;
@ -413,7 +419,8 @@ int try_to_free_nats(struct f2fs_sb_info *sbi, int nr_shrink)
/*
* This function always returns success
*/
void get_node_info(struct f2fs_sb_info *sbi, nid_t nid, struct node_info *ni)
void f2fs_get_node_info(struct f2fs_sb_info *sbi, nid_t nid,
struct node_info *ni)
{
struct f2fs_nm_info *nm_i = NM_I(sbi);
struct curseg_info *curseg = CURSEG_I(sbi, CURSEG_HOT_DATA);
@ -443,7 +450,7 @@ void get_node_info(struct f2fs_sb_info *sbi, nid_t nid, struct node_info *ni)
/* Check current segment summary */
down_read(&curseg->journal_rwsem);
i = lookup_journal_in_cursum(journal, NAT_JOURNAL, nid, 0);
i = f2fs_lookup_journal_in_cursum(journal, NAT_JOURNAL, nid, 0);
if (i >= 0) {
ne = nat_in_journal(journal, i);
node_info_from_raw_nat(ni, &ne);
@ -458,7 +465,7 @@ void get_node_info(struct f2fs_sb_info *sbi, nid_t nid, struct node_info *ni)
index = current_nat_addr(sbi, nid);
up_read(&nm_i->nat_tree_lock);
page = get_meta_page(sbi, index);
page = f2fs_get_meta_page(sbi, index);
nat_blk = (struct f2fs_nat_block *)page_address(page);
ne = nat_blk->entries[nid - start_nid];
node_info_from_raw_nat(ni, &ne);
@ -471,7 +478,7 @@ cache:
/*
* readahead MAX_RA_NODE number of node pages.
*/
static void ra_node_pages(struct page *parent, int start, int n)
static void f2fs_ra_node_pages(struct page *parent, int start, int n)
{
struct f2fs_sb_info *sbi = F2FS_P_SB(parent);
struct blk_plug plug;
@ -485,13 +492,13 @@ static void ra_node_pages(struct page *parent, int start, int n)
end = min(end, NIDS_PER_BLOCK);
for (i = start; i < end; i++) {
nid = get_nid(parent, i, false);
ra_node_page(sbi, nid);
f2fs_ra_node_page(sbi, nid);
}
blk_finish_plug(&plug);
}
pgoff_t get_next_page_offset(struct dnode_of_data *dn, pgoff_t pgofs)
pgoff_t f2fs_get_next_page_offset(struct dnode_of_data *dn, pgoff_t pgofs)
{
const long direct_index = ADDRS_PER_INODE(dn->inode);
const long direct_blks = ADDRS_PER_BLOCK;
@ -606,7 +613,7 @@ got:
* f2fs_unlock_op() only if ro is not set RDONLY_NODE.
* In the case of RDONLY_NODE, we don't need to care about mutex.
*/
int get_dnode_of_data(struct dnode_of_data *dn, pgoff_t index, int mode)
int f2fs_get_dnode_of_data(struct dnode_of_data *dn, pgoff_t index, int mode)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(dn->inode);
struct page *npage[4];
@ -625,7 +632,7 @@ int get_dnode_of_data(struct dnode_of_data *dn, pgoff_t index, int mode)
npage[0] = dn->inode_page;
if (!npage[0]) {
npage[0] = get_node_page(sbi, nids[0]);
npage[0] = f2fs_get_node_page(sbi, nids[0]);
if (IS_ERR(npage[0]))
return PTR_ERR(npage[0]);
}
@ -649,24 +656,24 @@ int get_dnode_of_data(struct dnode_of_data *dn, pgoff_t index, int mode)
if (!nids[i] && mode == ALLOC_NODE) {
/* alloc new node */
if (!alloc_nid(sbi, &(nids[i]))) {
if (!f2fs_alloc_nid(sbi, &(nids[i]))) {
err = -ENOSPC;
goto release_pages;
}
dn->nid = nids[i];
npage[i] = new_node_page(dn, noffset[i]);
npage[i] = f2fs_new_node_page(dn, noffset[i]);
if (IS_ERR(npage[i])) {
alloc_nid_failed(sbi, nids[i]);
f2fs_alloc_nid_failed(sbi, nids[i]);
err = PTR_ERR(npage[i]);
goto release_pages;
}
set_nid(parent, offset[i - 1], nids[i], i == 1);
alloc_nid_done(sbi, nids[i]);
f2fs_alloc_nid_done(sbi, nids[i]);
done = true;
} else if (mode == LOOKUP_NODE_RA && i == level && level > 1) {
npage[i] = get_node_page_ra(parent, offset[i - 1]);
npage[i] = f2fs_get_node_page_ra(parent, offset[i - 1]);
if (IS_ERR(npage[i])) {
err = PTR_ERR(npage[i]);
goto release_pages;
@ -681,7 +688,7 @@ int get_dnode_of_data(struct dnode_of_data *dn, pgoff_t index, int mode)
}
if (!done) {
npage[i] = get_node_page(sbi, nids[i]);
npage[i] = f2fs_get_node_page(sbi, nids[i]);
if (IS_ERR(npage[i])) {
err = PTR_ERR(npage[i]);
f2fs_put_page(npage[0], 0);
@ -720,15 +727,15 @@ static void truncate_node(struct dnode_of_data *dn)
struct f2fs_sb_info *sbi = F2FS_I_SB(dn->inode);
struct node_info ni;
get_node_info(sbi, dn->nid, &ni);
f2fs_get_node_info(sbi, dn->nid, &ni);
/* Deallocate node address */
invalidate_blocks(sbi, ni.blk_addr);
f2fs_invalidate_blocks(sbi, ni.blk_addr);
dec_valid_node_count(sbi, dn->inode, dn->nid == dn->inode->i_ino);
set_node_addr(sbi, &ni, NULL_ADDR, false);
if (dn->nid == dn->inode->i_ino) {
remove_orphan_inode(sbi, dn->nid);
f2fs_remove_orphan_inode(sbi, dn->nid);
dec_valid_inode_count(sbi);
f2fs_inode_synced(dn->inode);
}
@ -753,7 +760,7 @@ static int truncate_dnode(struct dnode_of_data *dn)
return 1;
/* get direct node */
page = get_node_page(F2FS_I_SB(dn->inode), dn->nid);
page = f2fs_get_node_page(F2FS_I_SB(dn->inode), dn->nid);
if (IS_ERR(page) && PTR_ERR(page) == -ENOENT)
return 1;
else if (IS_ERR(page))
@ -762,7 +769,7 @@ static int truncate_dnode(struct dnode_of_data *dn)
/* Make dnode_of_data for parameter */
dn->node_page = page;
dn->ofs_in_node = 0;
truncate_data_blocks(dn);
f2fs_truncate_data_blocks(dn);
truncate_node(dn);
return 1;
}
@ -783,13 +790,13 @@ static int truncate_nodes(struct dnode_of_data *dn, unsigned int nofs,
trace_f2fs_truncate_nodes_enter(dn->inode, dn->nid, dn->data_blkaddr);
page = get_node_page(F2FS_I_SB(dn->inode), dn->nid);
page = f2fs_get_node_page(F2FS_I_SB(dn->inode), dn->nid);
if (IS_ERR(page)) {
trace_f2fs_truncate_nodes_exit(dn->inode, PTR_ERR(page));
return PTR_ERR(page);
}
ra_node_pages(page, ofs, NIDS_PER_BLOCK);
f2fs_ra_node_pages(page, ofs, NIDS_PER_BLOCK);
rn = F2FS_NODE(page);
if (depth < 3) {
@ -859,7 +866,7 @@ static int truncate_partial_nodes(struct dnode_of_data *dn,
/* get indirect nodes in the path */
for (i = 0; i < idx + 1; i++) {
/* reference count'll be increased */
pages[i] = get_node_page(F2FS_I_SB(dn->inode), nid[i]);
pages[i] = f2fs_get_node_page(F2FS_I_SB(dn->inode), nid[i]);
if (IS_ERR(pages[i])) {
err = PTR_ERR(pages[i]);
idx = i - 1;
@ -868,7 +875,7 @@ static int truncate_partial_nodes(struct dnode_of_data *dn,
nid[i + 1] = get_nid(pages[i], offset[i + 1], false);
}
ra_node_pages(pages[idx], offset[idx + 1], NIDS_PER_BLOCK);
f2fs_ra_node_pages(pages[idx], offset[idx + 1], NIDS_PER_BLOCK);
/* free direct nodes linked to a partial indirect node */
for (i = offset[idx + 1]; i < NIDS_PER_BLOCK; i++) {
@ -905,7 +912,7 @@ fail:
/*
* All the block addresses of data and nodes should be nullified.
*/
int truncate_inode_blocks(struct inode *inode, pgoff_t from)
int f2fs_truncate_inode_blocks(struct inode *inode, pgoff_t from)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
int err = 0, cont = 1;
@ -921,7 +928,7 @@ int truncate_inode_blocks(struct inode *inode, pgoff_t from)
if (level < 0)
return level;
page = get_node_page(sbi, inode->i_ino);
page = f2fs_get_node_page(sbi, inode->i_ino);
if (IS_ERR(page)) {
trace_f2fs_truncate_inode_blocks_exit(inode, PTR_ERR(page));
return PTR_ERR(page);
@ -1001,7 +1008,7 @@ fail:
}
/* caller must lock inode page */
int truncate_xattr_node(struct inode *inode)
int f2fs_truncate_xattr_node(struct inode *inode)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
nid_t nid = F2FS_I(inode)->i_xattr_nid;
@ -1011,7 +1018,7 @@ int truncate_xattr_node(struct inode *inode)
if (!nid)
return 0;
npage = get_node_page(sbi, nid);
npage = f2fs_get_node_page(sbi, nid);
if (IS_ERR(npage))
return PTR_ERR(npage);
@ -1026,17 +1033,17 @@ int truncate_xattr_node(struct inode *inode)
* Caller should grab and release a rwsem by calling f2fs_lock_op() and
* f2fs_unlock_op().
*/
int remove_inode_page(struct inode *inode)
int f2fs_remove_inode_page(struct inode *inode)
{
struct dnode_of_data dn;
int err;
set_new_dnode(&dn, inode, NULL, NULL, inode->i_ino);
err = get_dnode_of_data(&dn, 0, LOOKUP_NODE);
err = f2fs_get_dnode_of_data(&dn, 0, LOOKUP_NODE);
if (err)
return err;
err = truncate_xattr_node(inode);
err = f2fs_truncate_xattr_node(inode);
if (err) {
f2fs_put_dnode(&dn);
return err;
@ -1045,7 +1052,7 @@ int remove_inode_page(struct inode *inode)
/* remove potential inline_data blocks */
if (S_ISREG(inode->i_mode) || S_ISDIR(inode->i_mode) ||
S_ISLNK(inode->i_mode))
truncate_data_blocks_range(&dn, 1);
f2fs_truncate_data_blocks_range(&dn, 1);
/* 0 is possible, after f2fs_new_inode() has failed */
f2fs_bug_on(F2FS_I_SB(inode),
@ -1056,7 +1063,7 @@ int remove_inode_page(struct inode *inode)
return 0;
}
struct page *new_inode_page(struct inode *inode)
struct page *f2fs_new_inode_page(struct inode *inode)
{
struct dnode_of_data dn;
@ -1064,10 +1071,10 @@ struct page *new_inode_page(struct inode *inode)
set_new_dnode(&dn, inode, NULL, NULL, inode->i_ino);
/* caller should f2fs_put_page(page, 1); */
return new_node_page(&dn, 0);
return f2fs_new_node_page(&dn, 0);
}
struct page *new_node_page(struct dnode_of_data *dn, unsigned int ofs)
struct page *f2fs_new_node_page(struct dnode_of_data *dn, unsigned int ofs)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(dn->inode);
struct node_info new_ni;
@ -1085,7 +1092,7 @@ struct page *new_node_page(struct dnode_of_data *dn, unsigned int ofs)
goto fail;
#ifdef CONFIG_F2FS_CHECK_FS
get_node_info(sbi, dn->nid, &new_ni);
f2fs_get_node_info(sbi, dn->nid, &new_ni);
f2fs_bug_on(sbi, new_ni.blk_addr != NULL_ADDR);
#endif
new_ni.nid = dn->nid;
@ -1137,7 +1144,7 @@ static int read_node_page(struct page *page, int op_flags)
if (PageUptodate(page))
return LOCKED_PAGE;
get_node_info(sbi, page->index, &ni);
f2fs_get_node_info(sbi, page->index, &ni);
if (unlikely(ni.blk_addr == NULL_ADDR)) {
ClearPageUptodate(page);
@ -1151,14 +1158,15 @@ static int read_node_page(struct page *page, int op_flags)
/*
* Readahead a node page
*/
void ra_node_page(struct f2fs_sb_info *sbi, nid_t nid)
void f2fs_ra_node_page(struct f2fs_sb_info *sbi, nid_t nid)
{
struct page *apage;
int err;
if (!nid)
return;
f2fs_bug_on(sbi, check_nid_range(sbi, nid));
if (f2fs_check_nid_range(sbi, nid))
return;
rcu_read_lock();
apage = radix_tree_lookup(&NODE_MAPPING(sbi)->page_tree, nid);
@ -1182,7 +1190,8 @@ static struct page *__get_node_page(struct f2fs_sb_info *sbi, pgoff_t nid,
if (!nid)
return ERR_PTR(-ENOENT);
f2fs_bug_on(sbi, check_nid_range(sbi, nid));
if (f2fs_check_nid_range(sbi, nid))
return ERR_PTR(-EINVAL);
repeat:
page = f2fs_grab_cache_page(NODE_MAPPING(sbi), nid, false);
if (!page)
@ -1198,7 +1207,7 @@ repeat:
}
if (parent)
ra_node_pages(parent, start + 1, MAX_RA_NODE);
f2fs_ra_node_pages(parent, start + 1, MAX_RA_NODE);
lock_page(page);
@ -1232,12 +1241,12 @@ out_err:
return page;
}
struct page *get_node_page(struct f2fs_sb_info *sbi, pgoff_t nid)
struct page *f2fs_get_node_page(struct f2fs_sb_info *sbi, pgoff_t nid)
{
return __get_node_page(sbi, nid, NULL, 0);
}
struct page *get_node_page_ra(struct page *parent, int start)
struct page *f2fs_get_node_page_ra(struct page *parent, int start)
{
struct f2fs_sb_info *sbi = F2FS_P_SB(parent);
nid_t nid = get_nid(parent, start, false);
@ -1272,7 +1281,7 @@ static void flush_inline_data(struct f2fs_sb_info *sbi, nid_t ino)
ret = f2fs_write_inline_data(inode, page);
inode_dec_dirty_pages(inode);
remove_dirty_inode(inode);
f2fs_remove_dirty_inode(inode);
if (ret)
set_page_dirty(page);
page_out:
@ -1283,21 +1292,17 @@ iput_out:
static struct page *last_fsync_dnode(struct f2fs_sb_info *sbi, nid_t ino)
{
pgoff_t index, end;
pgoff_t index;
struct pagevec pvec;
struct page *last_page = NULL;
int nr_pages;
pagevec_init(&pvec, 0);
index = 0;
end = ULONG_MAX;
while (index <= end) {
int i, nr_pages;
nr_pages = pagevec_lookup_tag(&pvec, NODE_MAPPING(sbi), &index,
PAGECACHE_TAG_DIRTY,
min(end - index, (pgoff_t)PAGEVEC_SIZE-1) + 1);
if (nr_pages == 0)
break;
while ((nr_pages = pagevec_lookup_tag(&pvec, NODE_MAPPING(sbi), &index,
PAGECACHE_TAG_DIRTY))) {
int i;
for (i = 0; i < nr_pages; i++) {
struct page *page = pvec.pages[i];
@ -1363,11 +1368,8 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted,
trace_f2fs_writepage(page, NODE);
if (unlikely(f2fs_cp_error(sbi))) {
dec_page_count(sbi, F2FS_DIRTY_NODES);
unlock_page(page);
return 0;
}
if (unlikely(f2fs_cp_error(sbi)))
goto redirty_out;
if (unlikely(is_sbi_flag_set(sbi, SBI_POR_DOING)))
goto redirty_out;
@ -1383,7 +1385,7 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted,
down_read(&sbi->node_write);
}
get_node_info(sbi, nid, &ni);
f2fs_get_node_info(sbi, nid, &ni);
/* This page is already truncated */
if (unlikely(ni.blk_addr == NULL_ADDR)) {
@ -1400,7 +1402,7 @@ static int __write_node_page(struct page *page, bool atomic, bool *submitted,
set_page_writeback(page);
ClearPageError(page);
fio.old_blkaddr = ni.blk_addr;
write_node_page(nid, &fio);
f2fs_do_write_node_page(nid, &fio);
set_node_addr(sbi, &ni, fio.new_blkaddr, is_fsync_dnode(page));
dec_page_count(sbi, F2FS_DIRTY_NODES);
up_read(&sbi->node_write);
@ -1429,7 +1431,7 @@ redirty_out:
return AOP_WRITEPAGE_ACTIVATE;
}
void move_node_page(struct page *node_page, int gc_type)
void f2fs_move_node_page(struct page *node_page, int gc_type)
{
if (gc_type == FG_GC) {
struct writeback_control wbc = {
@ -1466,16 +1468,17 @@ static int f2fs_write_node_page(struct page *page,
return __write_node_page(page, false, NULL, wbc, false, FS_NODE_IO);
}
int fsync_node_pages(struct f2fs_sb_info *sbi, struct inode *inode,
int f2fs_fsync_node_pages(struct f2fs_sb_info *sbi, struct inode *inode,
struct writeback_control *wbc, bool atomic)
{
pgoff_t index, end;
pgoff_t index;
pgoff_t last_idx = ULONG_MAX;
struct pagevec pvec;
int ret = 0;
struct page *last_page = NULL;
bool marked = false;
nid_t ino = inode->i_ino;
int nr_pages;
if (atomic) {
last_page = last_fsync_dnode(sbi, ino);
@ -1485,15 +1488,10 @@ int fsync_node_pages(struct f2fs_sb_info *sbi, struct inode *inode,
retry:
pagevec_init(&pvec, 0);
index = 0;
end = ULONG_MAX;
while (index <= end) {
int i, nr_pages;
nr_pages = pagevec_lookup_tag(&pvec, NODE_MAPPING(sbi), &index,
PAGECACHE_TAG_DIRTY,
min(end - index, (pgoff_t)PAGEVEC_SIZE-1) + 1);
if (nr_pages == 0)
break;
while ((nr_pages = pagevec_lookup_tag(&pvec, NODE_MAPPING(sbi), &index,
PAGECACHE_TAG_DIRTY))) {
int i;
for (i = 0; i < nr_pages; i++) {
struct page *page = pvec.pages[i];
@ -1537,9 +1535,9 @@ continue_unlock:
if (IS_INODE(page)) {
if (is_inode_flag_set(inode,
FI_DIRTY_INODE))
update_inode(inode, page);
f2fs_update_inode(inode, page);
set_dentry_mark(page,
need_dentry_mark(sbi, ino));
f2fs_need_dentry_mark(sbi, ino));
}
/* may be written by other thread */
if (!PageDirty(page))
@ -1589,33 +1587,37 @@ out:
return ret ? -EIO: 0;
}
int sync_node_pages(struct f2fs_sb_info *sbi, struct writeback_control *wbc,
int f2fs_sync_node_pages(struct f2fs_sb_info *sbi,
struct writeback_control *wbc,
bool do_balance, enum iostat_type io_type)
{
pgoff_t index, end;
pgoff_t index;
struct pagevec pvec;
int step = 0;
int nwritten = 0;
int ret = 0;
int nr_pages, done = 0;
pagevec_init(&pvec, 0);
next_step:
index = 0;
end = ULONG_MAX;
while (index <= end) {
int i, nr_pages;
nr_pages = pagevec_lookup_tag(&pvec, NODE_MAPPING(sbi), &index,
PAGECACHE_TAG_DIRTY,
min(end - index, (pgoff_t)PAGEVEC_SIZE-1) + 1);
if (nr_pages == 0)
break;
while (!done && (nr_pages = pagevec_lookup_tag(&pvec,
NODE_MAPPING(sbi), &index, PAGECACHE_TAG_DIRTY))) {
int i;
for (i = 0; i < nr_pages; i++) {
struct page *page = pvec.pages[i];
bool submitted = false;
/* give a priority to WB_SYNC threads */
if (atomic_read(&sbi->wb_sync_req[NODE]) &&
wbc->sync_mode == WB_SYNC_NONE) {
done = 1;
break;
}
/*
* flushing sequence with step:
* 0. indirect nodes
@ -1694,29 +1696,22 @@ continue_unlock:
return ret;
}
int wait_on_node_pages_writeback(struct f2fs_sb_info *sbi, nid_t ino)
int f2fs_wait_on_node_pages_writeback(struct f2fs_sb_info *sbi, nid_t ino)
{
pgoff_t index = 0, end = ULONG_MAX;
pgoff_t index = 0;
struct pagevec pvec;
int ret2 = 0, ret = 0;
int nr_pages;
pagevec_init(&pvec, 0);
while (index <= end) {
int i, nr_pages;
nr_pages = pagevec_lookup_tag(&pvec, NODE_MAPPING(sbi), &index,
PAGECACHE_TAG_WRITEBACK,
min(end - index, (pgoff_t)PAGEVEC_SIZE-1) + 1);
if (nr_pages == 0)
break;
while ((nr_pages = pagevec_lookup_tag(&pvec, NODE_MAPPING(sbi), &index,
PAGECACHE_TAG_WRITEBACK))) {
int i;
for (i = 0; i < nr_pages; i++) {
struct page *page = pvec.pages[i];
/* until radix tree lookup accepts end_index */
if (unlikely(page->index > end))
continue;
if (ino && ino_of_node(page) == ino) {
f2fs_wait_on_page_writeback(page, NODE, true);
if (TestClearPageError(page))
@ -1753,14 +1748,21 @@ static int f2fs_write_node_pages(struct address_space *mapping,
if (get_pages(sbi, F2FS_DIRTY_NODES) < nr_pages_to_skip(sbi, NODE))
goto skip_write;
if (wbc->sync_mode == WB_SYNC_ALL)
atomic_inc(&sbi->wb_sync_req[NODE]);
else if (atomic_read(&sbi->wb_sync_req[NODE]))
goto skip_write;
trace_f2fs_writepages(mapping->host, wbc, NODE);
diff = nr_pages_to_write(sbi, NODE, wbc);
wbc->sync_mode = WB_SYNC_NONE;
blk_start_plug(&plug);
sync_node_pages(sbi, wbc, true, FS_NODE_IO);
f2fs_sync_node_pages(sbi, wbc, true, FS_NODE_IO);
blk_finish_plug(&plug);
wbc->nr_to_write = max((long)0, wbc->nr_to_write - diff);
if (wbc->sync_mode == WB_SYNC_ALL)
atomic_dec(&sbi->wb_sync_req[NODE]);
return 0;
skip_write:
@ -1906,20 +1908,20 @@ static bool add_free_nid(struct f2fs_sb_info *sbi,
* Thread A Thread B
* - f2fs_create
* - f2fs_new_inode
* - alloc_nid
* - f2fs_alloc_nid
* - __insert_nid_to_list(PREALLOC_NID)
* - f2fs_balance_fs_bg
* - build_free_nids
* - __build_free_nids
* - f2fs_build_free_nids
* - __f2fs_build_free_nids
* - scan_nat_page
* - add_free_nid
* - __lookup_nat_cache
* - f2fs_add_link
* - init_inode_metadata
* - new_inode_page
* - new_node_page
* - f2fs_init_inode_metadata
* - f2fs_new_inode_page
* - f2fs_new_node_page
* - set_node_addr
* - alloc_nid_done
* - f2fs_alloc_nid_done
* - __remove_nid_from_list(PREALLOC_NID)
* - __insert_nid_to_list(FREE_NID)
*/
@ -2051,7 +2053,8 @@ out:
up_read(&nm_i->nat_tree_lock);
}
static void __build_free_nids(struct f2fs_sb_info *sbi, bool sync, bool mount)
static void __f2fs_build_free_nids(struct f2fs_sb_info *sbi,
bool sync, bool mount)
{
struct f2fs_nm_info *nm_i = NM_I(sbi);
int i = 0;
@ -2064,7 +2067,7 @@ static void __build_free_nids(struct f2fs_sb_info *sbi, bool sync, bool mount)
if (nm_i->nid_cnt[FREE_NID] >= NAT_ENTRY_PER_BLOCK)
return;
if (!sync && !available_free_memory(sbi, FREE_NIDS))
if (!sync && !f2fs_available_free_memory(sbi, FREE_NIDS))
return;
if (!mount) {
@ -2076,7 +2079,7 @@ static void __build_free_nids(struct f2fs_sb_info *sbi, bool sync, bool mount)
}
/* readahead nat pages to be scanned */
ra_meta_pages(sbi, NAT_BLOCK_OFFSET(nid), FREE_NID_PAGES,
f2fs_ra_meta_pages(sbi, NAT_BLOCK_OFFSET(nid), FREE_NID_PAGES,
META_NAT, true);
down_read(&nm_i->nat_tree_lock);
@ -2106,14 +2109,14 @@ static void __build_free_nids(struct f2fs_sb_info *sbi, bool sync, bool mount)
up_read(&nm_i->nat_tree_lock);
ra_meta_pages(sbi, NAT_BLOCK_OFFSET(nm_i->next_scan_nid),
f2fs_ra_meta_pages(sbi, NAT_BLOCK_OFFSET(nm_i->next_scan_nid),
nm_i->ra_nid_pages, META_NAT, false);
}
void build_free_nids(struct f2fs_sb_info *sbi, bool sync, bool mount)
void f2fs_build_free_nids(struct f2fs_sb_info *sbi, bool sync, bool mount)
{
mutex_lock(&NM_I(sbi)->build_lock);
__build_free_nids(sbi, sync, mount);
__f2fs_build_free_nids(sbi, sync, mount);
mutex_unlock(&NM_I(sbi)->build_lock);
}
@ -2122,7 +2125,7 @@ void build_free_nids(struct f2fs_sb_info *sbi, bool sync, bool mount)
* from second parameter of this function.
* The returned nid could be used ino as well as nid when inode is created.
*/
bool alloc_nid(struct f2fs_sb_info *sbi, nid_t *nid)
bool f2fs_alloc_nid(struct f2fs_sb_info *sbi, nid_t *nid)
{
struct f2fs_nm_info *nm_i = NM_I(sbi);
struct free_nid *i = NULL;
@ -2140,8 +2143,8 @@ retry:
return false;
}
/* We should not use stale free nids created by build_free_nids */
if (nm_i->nid_cnt[FREE_NID] && !on_build_free_nids(nm_i)) {
/* We should not use stale free nids created by f2fs_build_free_nids */
if (nm_i->nid_cnt[FREE_NID] && !on_f2fs_build_free_nids(nm_i)) {
f2fs_bug_on(sbi, list_empty(&nm_i->free_nid_list));
i = list_first_entry(&nm_i->free_nid_list,
struct free_nid, list);
@ -2158,14 +2161,14 @@ retry:
spin_unlock(&nm_i->nid_list_lock);
/* Let's scan nat pages and its caches to get free nids */
build_free_nids(sbi, true, false);
f2fs_build_free_nids(sbi, true, false);
goto retry;
}
/*
* alloc_nid() should be called prior to this function.
* f2fs_alloc_nid() should be called prior to this function.
*/
void alloc_nid_done(struct f2fs_sb_info *sbi, nid_t nid)
void f2fs_alloc_nid_done(struct f2fs_sb_info *sbi, nid_t nid)
{
struct f2fs_nm_info *nm_i = NM_I(sbi);
struct free_nid *i;
@ -2180,9 +2183,9 @@ void alloc_nid_done(struct f2fs_sb_info *sbi, nid_t nid)
}
/*
* alloc_nid() should be called prior to this function.
* f2fs_alloc_nid() should be called prior to this function.
*/
void alloc_nid_failed(struct f2fs_sb_info *sbi, nid_t nid)
void f2fs_alloc_nid_failed(struct f2fs_sb_info *sbi, nid_t nid)
{
struct f2fs_nm_info *nm_i = NM_I(sbi);
struct free_nid *i;
@ -2195,7 +2198,7 @@ void alloc_nid_failed(struct f2fs_sb_info *sbi, nid_t nid)
i = __lookup_free_nid_list(nm_i, nid);
f2fs_bug_on(sbi, !i);
if (!available_free_memory(sbi, FREE_NIDS)) {
if (!f2fs_available_free_memory(sbi, FREE_NIDS)) {
__remove_free_nid(sbi, i, PREALLOC_NID);
need_free = true;
} else {
@ -2212,7 +2215,7 @@ void alloc_nid_failed(struct f2fs_sb_info *sbi, nid_t nid)
kmem_cache_free(free_nid_slab, i);
}
int try_to_free_nids(struct f2fs_sb_info *sbi, int nr_shrink)
int f2fs_try_to_free_nids(struct f2fs_sb_info *sbi, int nr_shrink)
{
struct f2fs_nm_info *nm_i = NM_I(sbi);
struct free_nid *i, *next;
@ -2240,14 +2243,14 @@ int try_to_free_nids(struct f2fs_sb_info *sbi, int nr_shrink)
return nr - nr_shrink;
}
void recover_inline_xattr(struct inode *inode, struct page *page)
void f2fs_recover_inline_xattr(struct inode *inode, struct page *page)
{
void *src_addr, *dst_addr;
size_t inline_size;
struct page *ipage;
struct f2fs_inode *ri;
ipage = get_node_page(F2FS_I_SB(inode), inode->i_ino);
ipage = f2fs_get_node_page(F2FS_I_SB(inode), inode->i_ino);
f2fs_bug_on(F2FS_I_SB(inode), IS_ERR(ipage));
ri = F2FS_INODE(page);
@ -2265,11 +2268,11 @@ void recover_inline_xattr(struct inode *inode, struct page *page)
f2fs_wait_on_page_writeback(ipage, NODE, true);
memcpy(dst_addr, src_addr, inline_size);
update_inode:
update_inode(inode, ipage);
f2fs_update_inode(inode, ipage);
f2fs_put_page(ipage, 1);
}
int recover_xattr_data(struct inode *inode, struct page *page)
int f2fs_recover_xattr_data(struct inode *inode, struct page *page)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
nid_t prev_xnid = F2FS_I(inode)->i_xattr_nid;
@ -2282,25 +2285,25 @@ int recover_xattr_data(struct inode *inode, struct page *page)
goto recover_xnid;
/* 1: invalidate the previous xattr nid */
get_node_info(sbi, prev_xnid, &ni);
invalidate_blocks(sbi, ni.blk_addr);
f2fs_get_node_info(sbi, prev_xnid, &ni);
f2fs_invalidate_blocks(sbi, ni.blk_addr);
dec_valid_node_count(sbi, inode, false);
set_node_addr(sbi, &ni, NULL_ADDR, false);
recover_xnid:
/* 2: update xattr nid in inode */
if (!alloc_nid(sbi, &new_xnid))
if (!f2fs_alloc_nid(sbi, &new_xnid))
return -ENOSPC;
set_new_dnode(&dn, inode, NULL, NULL, new_xnid);
xpage = new_node_page(&dn, XATTR_NODE_OFFSET);
xpage = f2fs_new_node_page(&dn, XATTR_NODE_OFFSET);
if (IS_ERR(xpage)) {
alloc_nid_failed(sbi, new_xnid);
f2fs_alloc_nid_failed(sbi, new_xnid);
return PTR_ERR(xpage);
}
alloc_nid_done(sbi, new_xnid);
update_inode_page(inode);
f2fs_alloc_nid_done(sbi, new_xnid);
f2fs_update_inode_page(inode);
/* 3: update and set xattr node page dirty */
memcpy(F2FS_NODE(xpage), F2FS_NODE(page), VALID_XATTR_BLOCK_SIZE);
@ -2311,14 +2314,14 @@ recover_xnid:
return 0;
}
int recover_inode_page(struct f2fs_sb_info *sbi, struct page *page)
int f2fs_recover_inode_page(struct f2fs_sb_info *sbi, struct page *page)
{
struct f2fs_inode *src, *dst;
nid_t ino = ino_of_node(page);
struct node_info old_ni, new_ni;
struct page *ipage;
get_node_info(sbi, ino, &old_ni);
f2fs_get_node_info(sbi, ino, &old_ni);
if (unlikely(old_ni.blk_addr != NULL_ADDR))
return -EINVAL;
@ -2372,7 +2375,7 @@ retry:
return 0;
}
void restore_node_summary(struct f2fs_sb_info *sbi,
void f2fs_restore_node_summary(struct f2fs_sb_info *sbi,
unsigned int segno, struct f2fs_summary_block *sum)
{
struct f2fs_node *rn;
@ -2389,10 +2392,10 @@ void restore_node_summary(struct f2fs_sb_info *sbi,
nrpages = min(last_offset - i, BIO_MAX_PAGES);
/* readahead node pages */
ra_meta_pages(sbi, addr, nrpages, META_POR, true);
f2fs_ra_meta_pages(sbi, addr, nrpages, META_POR, true);
for (idx = addr; idx < addr + nrpages; idx++) {
struct page *page = get_tmp_page(sbi, idx);
struct page *page = f2fs_get_tmp_page(sbi, idx);
rn = F2FS_NODE(page);
sum_entry->nid = rn->footer.nid;
@ -2534,7 +2537,7 @@ static void __flush_nat_entry_set(struct f2fs_sb_info *sbi,
f2fs_bug_on(sbi, nat_get_blkaddr(ne) == NEW_ADDR);
if (to_journal) {
offset = lookup_journal_in_cursum(journal,
offset = f2fs_lookup_journal_in_cursum(journal,
NAT_JOURNAL, nid, 1);
f2fs_bug_on(sbi, offset < 0);
raw_ne = &nat_in_journal(journal, offset);
@ -2571,7 +2574,7 @@ static void __flush_nat_entry_set(struct f2fs_sb_info *sbi,
/*
* This function is called during the checkpointing process.
*/
void flush_nat_entries(struct f2fs_sb_info *sbi, struct cp_control *cpc)
void f2fs_flush_nat_entries(struct f2fs_sb_info *sbi, struct cp_control *cpc)
{
struct f2fs_nm_info *nm_i = NM_I(sbi);
struct curseg_info *curseg = CURSEG_I(sbi, CURSEG_HOT_DATA);
@ -2634,7 +2637,7 @@ static int __get_nat_bitmaps(struct f2fs_sb_info *sbi)
nat_bits_addr = __start_cp_addr(sbi) + sbi->blocks_per_seg -
nm_i->nat_bits_blocks;
for (i = 0; i < nm_i->nat_bits_blocks; i++) {
struct page *page = get_meta_page(sbi, nat_bits_addr++);
struct page *page = f2fs_get_meta_page(sbi, nat_bits_addr++);
memcpy(nm_i->nat_bits + (i << F2FS_BLKSIZE_BITS),
page_address(page), F2FS_BLKSIZE);
@ -2753,8 +2756,10 @@ static int init_free_nid_cache(struct f2fs_sb_info *sbi)
struct f2fs_nm_info *nm_i = NM_I(sbi);
int i;
nm_i->free_nid_bitmap = f2fs_kzalloc(sbi, nm_i->nat_blocks *
sizeof(unsigned char *), GFP_KERNEL);
nm_i->free_nid_bitmap =
f2fs_kzalloc(sbi, array_size(sizeof(unsigned char *),
nm_i->nat_blocks),
GFP_KERNEL);
if (!nm_i->free_nid_bitmap)
return -ENOMEM;
@ -2770,14 +2775,16 @@ static int init_free_nid_cache(struct f2fs_sb_info *sbi)
if (!nm_i->nat_block_bitmap)
return -ENOMEM;
nm_i->free_nid_count = f2fs_kvzalloc(sbi, nm_i->nat_blocks *
sizeof(unsigned short), GFP_KERNEL);
nm_i->free_nid_count =
f2fs_kvzalloc(sbi, array_size(sizeof(unsigned short),
nm_i->nat_blocks),
GFP_KERNEL);
if (!nm_i->free_nid_count)
return -ENOMEM;
return 0;
}
int build_node_manager(struct f2fs_sb_info *sbi)
int f2fs_build_node_manager(struct f2fs_sb_info *sbi)
{
int err;
@ -2797,11 +2804,11 @@ int build_node_manager(struct f2fs_sb_info *sbi)
/* load free nid status from nat_bits table */
load_free_nid_bitmap(sbi);
build_free_nids(sbi, true, true);
f2fs_build_free_nids(sbi, true, true);
return 0;
}
void destroy_node_manager(struct f2fs_sb_info *sbi)
void f2fs_destroy_node_manager(struct f2fs_sb_info *sbi)
{
struct f2fs_nm_info *nm_i = NM_I(sbi);
struct free_nid *i, *next_i;
@ -2873,7 +2880,7 @@ void destroy_node_manager(struct f2fs_sb_info *sbi)
kfree(nm_i);
}
int __init create_node_manager_caches(void)
int __init f2fs_create_node_manager_caches(void)
{
nat_entry_slab = f2fs_kmem_cache_create("nat_entry",
sizeof(struct nat_entry));
@ -2899,7 +2906,7 @@ fail:
return -ENOMEM;
}
void destroy_node_manager_caches(void)
void f2fs_destroy_node_manager_caches(void)
{
kmem_cache_destroy(nat_entry_set_slab);
kmem_cache_destroy(free_nid_slab);

65
fs/f2fs/recovery.c
View file

@ -47,7 +47,7 @@
static struct kmem_cache *fsync_entry_slab;
bool space_for_roll_forward(struct f2fs_sb_info *sbi)
bool f2fs_space_for_roll_forward(struct f2fs_sb_info *sbi)
{
s64 nalloc = percpu_counter_sum_positive(&sbi->alloc_valid_block_count);
@ -162,7 +162,7 @@ retry:
goto out_put;
}
err = acquire_orphan_inode(F2FS_I_SB(inode));
err = f2fs_acquire_orphan_inode(F2FS_I_SB(inode));
if (err) {
iput(einode);
goto out_put;
@ -173,7 +173,7 @@ retry:
} else if (IS_ERR(page)) {
err = PTR_ERR(page);
} else {
err = __f2fs_do_add_link(dir, &fname, inode,
err = f2fs_add_dentry(dir, &fname, inode,
inode->i_ino, inode->i_mode);
}
if (err == -ENOMEM)
@ -204,8 +204,6 @@ static void recover_inline_flags(struct inode *inode, struct f2fs_inode *ri)
set_inode_flag(inode, FI_DATA_EXIST);
else
clear_inode_flag(inode, FI_DATA_EXIST);
if (!(ri->i_inline & F2FS_INLINE_DOTS))
clear_inode_flag(inode, FI_INLINE_DOTS);
}
static void recover_inode(struct inode *inode, struct page *page)
@ -254,10 +252,10 @@ static int find_fsync_dnodes(struct f2fs_sb_info *sbi, struct list_head *head,
while (1) {
struct fsync_inode_entry *entry;
if (!is_valid_blkaddr(sbi, blkaddr, META_POR))
if (!f2fs_is_valid_meta_blkaddr(sbi, blkaddr, META_POR))
return 0;
page = get_tmp_page(sbi, blkaddr);
page = f2fs_get_tmp_page(sbi, blkaddr);
if (!is_recoverable_dnode(page))
break;
@ -271,7 +269,7 @@ static int find_fsync_dnodes(struct f2fs_sb_info *sbi, struct list_head *head,
if (!check_only &&
IS_INODE(page) && is_dent_dnode(page)) {
err = recover_inode_page(sbi, page);
err = f2fs_recover_inode_page(sbi, page);
if (err)
break;
quota_inode = true;
@ -312,7 +310,7 @@ next:
blkaddr = next_blkaddr_of_node(page);
f2fs_put_page(page, 1);
ra_meta_pages_cond(sbi, blkaddr);
f2fs_ra_meta_pages_cond(sbi, blkaddr);
}
f2fs_put_page(page, 1);
return err;
@ -355,7 +353,7 @@ static int check_index_in_prev_nodes(struct f2fs_sb_info *sbi,
}
}
sum_page = get_sum_page(sbi, segno);
sum_page = f2fs_get_sum_page(sbi, segno);
sum_node = (struct f2fs_summary_block *)page_address(sum_page);
sum = sum_node->entries[blkoff];
f2fs_put_page(sum_page, 1);
@ -375,7 +373,7 @@ got_it:
}
/* Get the node page */
node_page = get_node_page(sbi, nid);
node_page = f2fs_get_node_page(sbi, nid);
if (IS_ERR(node_page))
return PTR_ERR(node_page);
@ -400,7 +398,8 @@ got_it:
inode = dn->inode;
}
bidx = start_bidx_of_node(offset, inode) + le16_to_cpu(sum.ofs_in_node);
bidx = f2fs_start_bidx_of_node(offset, inode) +
le16_to_cpu(sum.ofs_in_node);
/*
* if inode page is locked, unlock temporarily, but its reference
@ -410,11 +409,11 @@ got_it:
unlock_page(dn->inode_page);
set_new_dnode(&tdn, inode, NULL, NULL, 0);
if (get_dnode_of_data(&tdn, bidx, LOOKUP_NODE))
if (f2fs_get_dnode_of_data(&tdn, bidx, LOOKUP_NODE))
goto out;
if (tdn.data_blkaddr == blkaddr)
truncate_data_blocks_range(&tdn, 1);
f2fs_truncate_data_blocks_range(&tdn, 1);
f2fs_put_dnode(&tdn);
out:
@ -427,7 +426,7 @@ out:
truncate_out:
if (datablock_addr(tdn.inode, tdn.node_page,
tdn.ofs_in_node) == blkaddr)
truncate_data_blocks_range(&tdn, 1);
f2fs_truncate_data_blocks_range(&tdn, 1);
if (dn->inode->i_ino == nid && !dn->inode_page_locked)
unlock_page(dn->inode_page);
return 0;
@ -443,25 +442,25 @@ static int do_recover_data(struct f2fs_sb_info *sbi, struct inode *inode,
/* step 1: recover xattr */
if (IS_INODE(page)) {
recover_inline_xattr(inode, page);
f2fs_recover_inline_xattr(inode, page);
} else if (f2fs_has_xattr_block(ofs_of_node(page))) {
err = recover_xattr_data(inode, page);
err = f2fs_recover_xattr_data(inode, page);
if (!err)
recovered++;
goto out;
}
/* step 2: recover inline data */
if (recover_inline_data(inode, page))
if (f2fs_recover_inline_data(inode, page))
goto out;
/* step 3: recover data indices */
start = start_bidx_of_node(ofs_of_node(page), inode);
start = f2fs_start_bidx_of_node(ofs_of_node(page), inode);
end = start + ADDRS_PER_PAGE(page, inode);
set_new_dnode(&dn, inode, NULL, NULL, 0);
retry_dn:
err = get_dnode_of_data(&dn, start, ALLOC_NODE);
err = f2fs_get_dnode_of_data(&dn, start, ALLOC_NODE);
if (err) {
if (err == -ENOMEM) {
congestion_wait(BLK_RW_ASYNC, HZ/50);
@ -472,7 +471,7 @@ retry_dn:
f2fs_wait_on_page_writeback(dn.node_page, NODE, true);
get_node_info(sbi, dn.nid, &ni);
f2fs_get_node_info(sbi, dn.nid, &ni);
f2fs_bug_on(sbi, ni.ino != ino_of_node(page));
f2fs_bug_on(sbi, ofs_of_node(dn.node_page) != ofs_of_node(page));
@ -488,7 +487,7 @@ retry_dn:
/* dest is invalid, just invalidate src block */
if (dest == NULL_ADDR) {
truncate_data_blocks_range(&dn, 1);
f2fs_truncate_data_blocks_range(&dn, 1);
continue;
}
@ -502,19 +501,19 @@ retry_dn:
* and then reserve one new block in dnode page.
*/
if (dest == NEW_ADDR) {
truncate_data_blocks_range(&dn, 1);
reserve_new_block(&dn);
f2fs_truncate_data_blocks_range(&dn, 1);
f2fs_reserve_new_block(&dn);
continue;
}
/* dest is valid block, try to recover from src to dest */
if (is_valid_blkaddr(sbi, dest, META_POR)) {
if (f2fs_is_valid_meta_blkaddr(sbi, dest, META_POR)) {
if (src == NULL_ADDR) {
err = reserve_new_block(&dn);
err = f2fs_reserve_new_block(&dn);
#ifdef CONFIG_F2FS_FAULT_INJECTION
while (err)
err = reserve_new_block(&dn);
err = f2fs_reserve_new_block(&dn);
#endif
/* We should not get -ENOSPC */
f2fs_bug_on(sbi, err);
@ -569,12 +568,12 @@ static int recover_data(struct f2fs_sb_info *sbi, struct list_head *inode_list,
while (1) {
struct fsync_inode_entry *entry;
if (!is_valid_blkaddr(sbi, blkaddr, META_POR))
if (!f2fs_is_valid_meta_blkaddr(sbi, blkaddr, META_POR))
break;
ra_meta_pages_cond(sbi, blkaddr);
f2fs_ra_meta_pages_cond(sbi, blkaddr);
page = get_tmp_page(sbi, blkaddr);
page = f2fs_get_tmp_page(sbi, blkaddr);
if (!is_recoverable_dnode(page)) {
f2fs_put_page(page, 1);
@ -612,11 +611,11 @@ next:
f2fs_put_page(page, 1);
}
if (!err)
allocate_new_segments(sbi);
f2fs_allocate_new_segments(sbi);
return err;
}
int recover_fsync_data(struct f2fs_sb_info *sbi, bool check_only)
int f2fs_recover_fsync_data(struct f2fs_sb_info *sbi, bool check_only)
{
struct list_head inode_list;
struct list_head dir_list;
@ -691,7 +690,7 @@ skip:
struct cp_control cpc = {
.reason = CP_RECOVERY,
};
err = write_checkpoint(sbi, &cpc);
err = f2fs_write_checkpoint(sbi, &cpc);
}
kmem_cache_destroy(fsync_entry_slab);

390
fs/f2fs/segment.c
View file

File diff suppressed because it is too large Load diff

32
fs/f2fs/segment.h
View file

@ -85,7 +85,7 @@
(GET_SEGOFF_FROM_SEG0(sbi, blk_addr) & ((sbi)->blocks_per_seg - 1))
#define GET_SEGNO(sbi, blk_addr) \
((((blk_addr) == NULL_ADDR) || ((blk_addr) == NEW_ADDR)) ? \
((!is_valid_blkaddr(blk_addr)) ? \
NULL_SEGNO : GET_L2R_SEGNO(FREE_I(sbi), \
GET_SEGNO_FROM_SEG0(sbi, blk_addr)))
#define BLKS_PER_SEC(sbi) \
@ -215,6 +215,8 @@ struct segment_allocation {
#define IS_DUMMY_WRITTEN_PAGE(page) \
(page_private(page) == (unsigned long)DUMMY_WRITTEN_PAGE)
#define MAX_SKIP_ATOMIC_COUNT 16
struct inmem_pages {
struct list_head list;
struct page *page;
@ -375,6 +377,7 @@ static inline void seg_info_to_sit_page(struct f2fs_sb_info *sbi,
int i;
raw_sit = (struct f2fs_sit_block *)page_address(page);
memset(raw_sit, 0, PAGE_SIZE);
for (i = 0; i < end - start; i++) {
rs = &raw_sit->entries[i];
se = get_seg_entry(sbi, start + i);
@ -742,11 +745,23 @@ static inline void set_to_next_sit(struct sit_info *sit_i, unsigned int start)
#endif
}
static inline unsigned long long get_mtime(struct f2fs_sb_info *sbi)
static inline unsigned long long get_mtime(struct f2fs_sb_info *sbi,
bool base_time)
{
struct sit_info *sit_i = SIT_I(sbi);
return sit_i->elapsed_time + CURRENT_TIME_SEC.tv_sec -
sit_i->mounted_time;
time64_t diff, now = ktime_get_real_seconds();
if (now >= sit_i->mounted_time)
return sit_i->elapsed_time + now - sit_i->mounted_time;
/* system time is set to the past */
if (!base_time) {
diff = sit_i->mounted_time - now;
if (sit_i->elapsed_time >= diff)
return sit_i->elapsed_time - diff;
return 0;
}
return sit_i->elapsed_time;
}
static inline void set_summary(struct f2fs_summary *sum, nid_t nid,
@ -770,15 +785,6 @@ static inline block_t sum_blk_addr(struct f2fs_sb_info *sbi, int base, int type)
- (base + 1) + type;
}
static inline bool no_fggc_candidate(struct f2fs_sb_info *sbi,
unsigned int secno)
{
if (get_valid_blocks(sbi, GET_SEG_FROM_SEC(sbi, secno), true) >
sbi->fggc_threshold)
return true;
return false;
}
static inline bool sec_usage_check(struct f2fs_sb_info *sbi, unsigned int secno)
{
if (IS_CURSEC(sbi, secno) || (sbi->cur_victim_sec == secno))

4
fs/f2fs/shrinker.c
View file

@ -109,11 +109,11 @@ unsigned long f2fs_shrink_scan(struct shrinker *shrink,
/* shrink clean nat cache entries */
if (freed < nr)
freed += try_to_free_nats(sbi, nr - freed);
freed += f2fs_try_to_free_nats(sbi, nr - freed);
/* shrink free nids cache entries */
if (freed < nr)
freed += try_to_free_nids(sbi, nr - freed);
freed += f2fs_try_to_free_nids(sbi, nr - freed);
spin_lock(&f2fs_list_lock);
p = p->next;

214
fs/f2fs/super.c
View file

@ -830,15 +830,14 @@ static struct inode *f2fs_alloc_inode(struct super_block *sb)
/* Initialize f2fs-specific inode info */
atomic_set(&fi->dirty_pages, 0);
fi->i_current_depth = 1;
init_rwsem(&fi->i_sem);
INIT_LIST_HEAD(&fi->dirty_list);
INIT_LIST_HEAD(&fi->gdirty_list);
INIT_LIST_HEAD(&fi->inmem_ilist);
INIT_LIST_HEAD(&fi->inmem_pages);
mutex_init(&fi->inmem_lock);
init_rwsem(&fi->dio_rwsem[READ]);
init_rwsem(&fi->dio_rwsem[WRITE]);
init_rwsem(&fi->i_gc_rwsem[READ]);
init_rwsem(&fi->i_gc_rwsem[WRITE]);
init_rwsem(&fi->i_mmap_sem);
init_rwsem(&fi->i_xattr_sem);
@ -866,7 +865,7 @@ static int f2fs_drop_inode(struct inode *inode)
/* some remained atomic pages should discarded */
if (f2fs_is_atomic_file(inode))
drop_inmem_pages(inode);
f2fs_drop_inmem_pages(inode);
/* should remain fi->extent_tree for writepage */
f2fs_destroy_extent_node(inode);
@ -1003,7 +1002,7 @@ static void f2fs_put_super(struct super_block *sb)
struct cp_control cpc = {
.reason = CP_UMOUNT,
};
write_checkpoint(sbi, &cpc);
f2fs_write_checkpoint(sbi, &cpc);
}
/* be sure to wait for any on-going discard commands */
@ -1013,17 +1012,17 @@ static void f2fs_put_super(struct super_block *sb)
struct cp_control cpc = {
.reason = CP_UMOUNT | CP_TRIMMED,
};
write_checkpoint(sbi, &cpc);
f2fs_write_checkpoint(sbi, &cpc);
}
/* write_checkpoint can update stat informaion */
/* f2fs_write_checkpoint can update stat informaion */
f2fs_destroy_stats(sbi);
/*
* normally superblock is clean, so we need to release this.
* In addition, EIO will skip do checkpoint, we need this as well.
*/
release_ino_entry(sbi, true);
f2fs_release_ino_entry(sbi, true);
f2fs_leave_shrinker(sbi);
mutex_unlock(&sbi->umount_mutex);
@ -1035,8 +1034,8 @@ static void f2fs_put_super(struct super_block *sb)
iput(sbi->meta_inode);
/* destroy f2fs internal modules */
destroy_node_manager(sbi);
destroy_segment_manager(sbi);
f2fs_destroy_node_manager(sbi);
f2fs_destroy_segment_manager(sbi);
kfree(sbi->ckpt);
@ -1079,7 +1078,7 @@ int f2fs_sync_fs(struct super_block *sb, int sync)
cpc.reason = __get_cp_reason(sbi);
mutex_lock(&sbi->gc_mutex);
err = write_checkpoint(sbi, &cpc);
err = f2fs_write_checkpoint(sbi, &cpc);
mutex_unlock(&sbi->gc_mutex);
}
f2fs_trace_ios(NULL, 1);
@ -1482,11 +1481,11 @@ static int f2fs_remount(struct super_block *sb, int *flags, char *data)
*/
if ((*flags & MS_RDONLY) || !test_opt(sbi, BG_GC)) {
if (sbi->gc_thread) {
stop_gc_thread(sbi);
f2fs_stop_gc_thread(sbi);
need_restart_gc = true;
}
} else if (!sbi->gc_thread) {
err = start_gc_thread(sbi);
err = f2fs_start_gc_thread(sbi);
if (err)
goto restore_opts;
need_stop_gc = true;
@ -1509,9 +1508,9 @@ static int f2fs_remount(struct super_block *sb, int *flags, char *data)
*/
if ((*flags & MS_RDONLY) || !test_opt(sbi, FLUSH_MERGE)) {
clear_opt(sbi, FLUSH_MERGE);
destroy_flush_cmd_control(sbi, false);
f2fs_destroy_flush_cmd_control(sbi, false);
} else {
err = create_flush_cmd_control(sbi);
err = f2fs_create_flush_cmd_control(sbi);
if (err)
goto restore_gc;
}
@ -1529,11 +1528,11 @@ skip:
return 0;
restore_gc:
if (need_restart_gc) {
if (start_gc_thread(sbi))
if (f2fs_start_gc_thread(sbi))
f2fs_msg(sbi->sb, KERN_WARNING,
"background gc thread has stopped");
} else if (need_stop_gc) {
stop_gc_thread(sbi);
f2fs_stop_gc_thread(sbi);
}
restore_opts:
#ifdef CONFIG_QUOTA
@ -1805,7 +1804,7 @@ static int f2fs_quota_on(struct super_block *sb, int type, int format_id,
inode = d_inode(path->dentry);
inode_lock(inode);
F2FS_I(inode)->i_flags |= FS_NOATIME_FL | FS_IMMUTABLE_FL;
F2FS_I(inode)->i_flags |= F2FS_NOATIME_FL | F2FS_IMMUTABLE_FL;
inode_set_flags(inode, S_NOATIME | S_IMMUTABLE,
S_NOATIME | S_IMMUTABLE);
inode_unlock(inode);
@ -1829,7 +1828,7 @@ static int f2fs_quota_off(struct super_block *sb, int type)
goto out_put;
inode_lock(inode);
F2FS_I(inode)->i_flags &= ~(FS_NOATIME_FL | FS_IMMUTABLE_FL);
F2FS_I(inode)->i_flags &= ~(F2FS_NOATIME_FL | F2FS_IMMUTABLE_FL);
inode_set_flags(inode, 0, S_NOATIME | S_IMMUTABLE);
inode_unlock(inode);
f2fs_mark_inode_dirty_sync(inode, false);
@ -1938,19 +1937,13 @@ static bool f2fs_dummy_context(struct inode *inode)
return DUMMY_ENCRYPTION_ENABLED(F2FS_I_SB(inode));
}
static unsigned f2fs_max_namelen(struct inode *inode)
{
return S_ISLNK(inode->i_mode) ?
inode->i_sb->s_blocksize : F2FS_NAME_LEN;
}
static const struct fscrypt_operations f2fs_cryptops = {
.key_prefix = "f2fs:",
.get_context = f2fs_get_context,
.set_context = f2fs_set_context,
.dummy_context = f2fs_dummy_context,
.empty_dir = f2fs_empty_dir,
.max_namelen = f2fs_max_namelen,
.max_namelen = F2FS_NAME_LEN,
};
#endif
@ -1960,7 +1953,7 @@ static struct inode *f2fs_nfs_get_inode(struct super_block *sb,
struct f2fs_sb_info *sbi = F2FS_SB(sb);
struct inode *inode;
if (check_nid_range(sbi, ino))
if (f2fs_check_nid_range(sbi, ino))
return ERR_PTR(-ESTALE);
/*
@ -2143,6 +2136,8 @@ static inline bool sanity_check_area_boundary(struct f2fs_sb_info *sbi,
static int sanity_check_raw_super(struct f2fs_sb_info *sbi,
struct buffer_head *bh)
{
block_t segment_count, segs_per_sec, secs_per_zone;
block_t total_sections, blocks_per_seg;
struct f2fs_super_block *raw_super = (struct f2fs_super_block *)
(bh->b_data + F2FS_SUPER_OFFSET);
struct super_block *sb = sbi->sb;
@ -2199,6 +2194,72 @@ static int sanity_check_raw_super(struct f2fs_sb_info *sbi,
return 1;
}
segment_count = le32_to_cpu(raw_super->segment_count);
segs_per_sec = le32_to_cpu(raw_super->segs_per_sec);
secs_per_zone = le32_to_cpu(raw_super->secs_per_zone);
total_sections = le32_to_cpu(raw_super->section_count);
/* blocks_per_seg should be 512, given the above check */
blocks_per_seg = 1 << le32_to_cpu(raw_super->log_blocks_per_seg);
if (segment_count > F2FS_MAX_SEGMENT ||
segment_count < F2FS_MIN_SEGMENTS) {
f2fs_msg(sb, KERN_INFO,
"Invalid segment count (%u)",
segment_count);
return 1;
}
if (total_sections > segment_count ||
total_sections < F2FS_MIN_SEGMENTS ||
segs_per_sec > segment_count || !segs_per_sec) {
f2fs_msg(sb, KERN_INFO,
"Invalid segment/section count (%u, %u x %u)",
segment_count, total_sections, segs_per_sec);
return 1;
}
if ((segment_count / segs_per_sec) < total_sections) {
f2fs_msg(sb, KERN_INFO,
"Small segment_count (%u < %u * %u)",
segment_count, segs_per_sec, total_sections);
return 1;
}
if (segment_count > (le32_to_cpu(raw_super->block_count) >> 9)) {
f2fs_msg(sb, KERN_INFO,
"Wrong segment_count / block_count (%u > %u)",
segment_count, le32_to_cpu(raw_super->block_count));
return 1;
}
if (secs_per_zone > total_sections) {
f2fs_msg(sb, KERN_INFO,
"Wrong secs_per_zone (%u > %u)",
secs_per_zone, total_sections);
return 1;
}
if (le32_to_cpu(raw_super->extension_count) > F2FS_MAX_EXTENSION ||
raw_super->hot_ext_count > F2FS_MAX_EXTENSION ||
(le32_to_cpu(raw_super->extension_count) +
raw_super->hot_ext_count) > F2FS_MAX_EXTENSION) {
f2fs_msg(sb, KERN_INFO,
"Corrupted extension count (%u + %u > %u)",
le32_to_cpu(raw_super->extension_count),
raw_super->hot_ext_count,
F2FS_MAX_EXTENSION);
return 1;
}
if (le32_to_cpu(raw_super->cp_payload) >
(blocks_per_seg - F2FS_CP_PACKS)) {
f2fs_msg(sb, KERN_INFO,
"Insane cp_payload (%u > %u)",
le32_to_cpu(raw_super->cp_payload),
blocks_per_seg - F2FS_CP_PACKS);
return 1;
}
/* check reserved ino info */
if (le32_to_cpu(raw_super->node_ino) != 1 ||
le32_to_cpu(raw_super->meta_ino) != 2 ||
@ -2211,13 +2272,6 @@ static int sanity_check_raw_super(struct f2fs_sb_info *sbi,
return 1;
}
if (le32_to_cpu(raw_super->segment_count) > F2FS_MAX_SEGMENT) {
f2fs_msg(sb, KERN_INFO,
"Invalid segment count (%u)",
le32_to_cpu(raw_super->segment_count));
return 1;
}
/* check CP/SIT/NAT/SSA/MAIN_AREA area boundary */
if (sanity_check_area_boundary(sbi, bh))
return 1;
@ -2225,7 +2279,7 @@ static int sanity_check_raw_super(struct f2fs_sb_info *sbi,
return 0;
}
int sanity_check_ckpt(struct f2fs_sb_info *sbi)
int f2fs_sanity_check_ckpt(struct f2fs_sb_info *sbi)
{
unsigned int total, fsmeta;
struct f2fs_super_block *raw_super = F2FS_RAW_SUPER(sbi);
@ -2306,13 +2360,15 @@ static void init_sb_info(struct f2fs_sb_info *sbi)
for (i = 0; i < NR_COUNT_TYPE; i++)
atomic_set(&sbi->nr_pages[i], 0);
atomic_set(&sbi->wb_sync_req, 0);
for (i = 0; i < META; i++)
atomic_set(&sbi->wb_sync_req[i], 0);
INIT_LIST_HEAD(&sbi->s_list);
mutex_init(&sbi->umount_mutex);
for (i = 0; i < NR_PAGE_TYPE - 1; i++)
for (j = HOT; j < NR_TEMP_TYPE; j++)
mutex_init(&sbi->wio_mutex[i][j]);
init_rwsem(&sbi->io_order_lock);
spin_lock_init(&sbi->cp_lock);
sbi->dirty_device = 0;
@ -2367,8 +2423,10 @@ static int init_blkz_info(struct f2fs_sb_info *sbi, int devi)
#define F2FS_REPORT_NR_ZONES 4096
zones = f2fs_kzalloc(sbi, sizeof(struct blk_zone) *
F2FS_REPORT_NR_ZONES, GFP_KERNEL);
zones = f2fs_kzalloc(sbi,
array_size(F2FS_REPORT_NR_ZONES,
sizeof(struct blk_zone)),
GFP_KERNEL);
if (!zones)
return -ENOMEM;
@ -2512,8 +2570,10 @@ static int f2fs_scan_devices(struct f2fs_sb_info *sbi)
* Initialize multiple devices information, or single
* zoned block device information.
*/
sbi->devs = f2fs_kzalloc(sbi, sizeof(struct f2fs_dev_info) *
max_devices, GFP_KERNEL);
sbi->devs = f2fs_kzalloc(sbi,
array_size(max_devices,
sizeof(struct f2fs_dev_info)),
GFP_KERNEL);
if (!sbi->devs)
return -ENOMEM;
@ -2735,8 +2795,10 @@ try_onemore:
int n = (i == META) ? 1: NR_TEMP_TYPE;
int j;
sbi->write_io[i] = f2fs_kmalloc(sbi,
n * sizeof(struct f2fs_bio_info),
sbi->write_io[i] =
f2fs_kmalloc(sbi,
array_size(n,
sizeof(struct f2fs_bio_info)),
GFP_KERNEL);
if (!sbi->write_io[i]) {
err = -ENOMEM;
@ -2777,7 +2839,7 @@ try_onemore:
goto free_io_dummy;
}
err = get_valid_checkpoint(sbi);
err = f2fs_get_valid_checkpoint(sbi);
if (err) {
f2fs_msg(sb, KERN_ERR, "Failed to get valid F2FS checkpoint");
goto free_meta_inode;
@ -2807,18 +2869,18 @@ try_onemore:
spin_lock_init(&sbi->inode_lock[i]);
}
init_extent_cache_info(sbi);
f2fs_init_extent_cache_info(sbi);
init_ino_entry_info(sbi);
f2fs_init_ino_entry_info(sbi);
/* setup f2fs internal modules */
err = build_segment_manager(sbi);
err = f2fs_build_segment_manager(sbi);
if (err) {
f2fs_msg(sb, KERN_ERR,
"Failed to initialize F2FS segment manager");
goto free_sm;
}
err = build_node_manager(sbi);
err = f2fs_build_node_manager(sbi);
if (err) {
f2fs_msg(sb, KERN_ERR,
"Failed to initialize F2FS node manager");
@ -2836,7 +2898,7 @@ try_onemore:
sbi->kbytes_written =
le64_to_cpu(seg_i->journal->info.kbytes_written);
build_gc_manager(sbi);
f2fs_build_gc_manager(sbi);
/* get an inode for node space */
sbi->node_inode = f2fs_iget(sb, F2FS_NODE_INO(sbi));
@ -2888,7 +2950,7 @@ try_onemore:
}
#endif
/* if there are nt orphan nodes free them */
err = recover_orphan_inodes(sbi);
err = f2fs_recover_orphan_inodes(sbi);
if (err)
goto free_meta;
@ -2910,7 +2972,7 @@ try_onemore:
if (!retry)
goto skip_recovery;
err = recover_fsync_data(sbi, false);
err = f2fs_recover_fsync_data(sbi, false);
if (err < 0) {
need_fsck = true;
f2fs_msg(sb, KERN_ERR,
@ -2918,7 +2980,7 @@ try_onemore:
goto free_meta;
}
} else {
err = recover_fsync_data(sbi, true);
err = f2fs_recover_fsync_data(sbi, true);
if (!f2fs_readonly(sb) && err > 0) {
err = -EINVAL;
@ -2928,7 +2990,7 @@ try_onemore:
}
}
skip_recovery:
/* recover_fsync_data() cleared this already */
/* f2fs_recover_fsync_data() cleared this already */
clear_sbi_flag(sbi, SBI_POR_DOING);
/*
@ -2937,7 +2999,7 @@ skip_recovery:
*/
if (test_opt(sbi, BG_GC) && !f2fs_readonly(sb)) {
/* After POR, we can run background GC thread.*/
err = start_gc_thread(sbi);
err = f2fs_start_gc_thread(sbi);
if (err)
goto free_meta;
}
@ -2968,10 +3030,10 @@ free_meta:
#endif
f2fs_sync_inode_meta(sbi);
/*
* Some dirty meta pages can be produced by recover_orphan_inodes()
* Some dirty meta pages can be produced by f2fs_recover_orphan_inodes()
* failed by EIO. Then, iput(node_inode) can trigger balance_fs_bg()
* followed by write_checkpoint() through f2fs_write_node_pages(), which
* falls into an infinite loop in sync_meta_pages().
* followed by f2fs_write_checkpoint() through f2fs_write_node_pages(), which
* falls into an infinite loop in f2fs_sync_meta_pages().
*/
truncate_inode_pages_final(META_MAPPING(sbi));
#ifdef CONFIG_QUOTA
@ -2984,13 +3046,13 @@ free_root_inode:
free_stats:
f2fs_destroy_stats(sbi);
free_node_inode:
release_ino_entry(sbi, true);
f2fs_release_ino_entry(sbi, true);
truncate_inode_pages_final(NODE_MAPPING(sbi));
iput(sbi->node_inode);
free_nm:
destroy_node_manager(sbi);
f2fs_destroy_node_manager(sbi);
free_sm:
destroy_segment_manager(sbi);
f2fs_destroy_segment_manager(sbi);
free_devices:
destroy_device_list(sbi);
kfree(sbi->ckpt);
@ -3036,8 +3098,8 @@ static void kill_f2fs_super(struct super_block *sb)
{
if (sb->s_root) {
set_sbi_flag(F2FS_SB(sb), SBI_IS_CLOSE);
stop_gc_thread(F2FS_SB(sb));
stop_discard_thread(F2FS_SB(sb));
f2fs_stop_gc_thread(F2FS_SB(sb));
f2fs_stop_discard_thread(F2FS_SB(sb));
}
kill_block_super(sb);
}
@ -3075,21 +3137,27 @@ static int __init init_f2fs_fs(void)
{
int err;
if (PAGE_SIZE != F2FS_BLKSIZE) {
printk("F2FS not supported on PAGE_SIZE(%lu) != %d\n",
PAGE_SIZE, F2FS_BLKSIZE);
return -EINVAL;
}
f2fs_build_trace_ios();
err = init_inodecache();
if (err)
goto fail;
err = create_node_manager_caches();
err = f2fs_create_node_manager_caches();
if (err)
goto free_inodecache;
err = create_segment_manager_caches();
err = f2fs_create_segment_manager_caches();
if (err)
goto free_node_manager_caches;
err = create_checkpoint_caches();
err = f2fs_create_checkpoint_caches();
if (err)
goto free_segment_manager_caches;
err = create_extent_cache();
err = f2fs_create_extent_cache();
if (err)
goto free_checkpoint_caches;
err = f2fs_init_sysfs();
@ -3118,13 +3186,13 @@ free_shrinker:
free_sysfs:
f2fs_exit_sysfs();
free_extent_cache:
destroy_extent_cache();
f2fs_destroy_extent_cache();
free_checkpoint_caches:
destroy_checkpoint_caches();
f2fs_destroy_checkpoint_caches();
free_segment_manager_caches:
destroy_segment_manager_caches();
f2fs_destroy_segment_manager_caches();
free_node_manager_caches:
destroy_node_manager_caches();
f2fs_destroy_node_manager_caches();
free_inodecache:
destroy_inodecache();
fail:
@ -3138,10 +3206,10 @@ static void __exit exit_f2fs_fs(void)
unregister_filesystem(&f2fs_fs_type);
unregister_shrinker(&f2fs_shrinker_info);
f2fs_exit_sysfs();
destroy_extent_cache();
destroy_checkpoint_caches();
destroy_segment_manager_caches();
destroy_node_manager_caches();
f2fs_destroy_extent_cache();
f2fs_destroy_checkpoint_caches();
f2fs_destroy_segment_manager_caches();
f2fs_destroy_node_manager_caches();
destroy_inodecache();
f2fs_destroy_trace_ios();
}

58
fs/f2fs/sysfs.c
View file

@ -147,13 +147,13 @@ static ssize_t f2fs_sbi_show(struct f2fs_attr *a,
int len = 0, i;
len += snprintf(buf + len, PAGE_SIZE - len,
"cold file extenstion:\n");
"cold file extension:\n");
for (i = 0; i < cold_count; i++)
len += snprintf(buf + len, PAGE_SIZE - len, "%s\n",
extlist[i]);
len += snprintf(buf + len, PAGE_SIZE - len,
"hot file extenstion:\n");
"hot file extension:\n");
for (i = cold_count; i < cold_count + hot_count; i++)
len += snprintf(buf + len, PAGE_SIZE - len, "%s\n",
extlist[i]);
@ -165,7 +165,7 @@ static ssize_t f2fs_sbi_show(struct f2fs_attr *a,
return snprintf(buf, PAGE_SIZE, "%u\n", *ui);
}
static ssize_t f2fs_sbi_store(struct f2fs_attr *a,
static ssize_t __sbi_store(struct f2fs_attr *a,
struct f2fs_sb_info *sbi,
const char *buf, size_t count)
{
@ -201,13 +201,13 @@ static ssize_t f2fs_sbi_store(struct f2fs_attr *a,
down_write(&sbi->sb_lock);
ret = update_extension_list(sbi, name, hot, set);
ret = f2fs_update_extension_list(sbi, name, hot, set);
if (ret)
goto out;
ret = f2fs_commit_super(sbi, false);
if (ret)
update_extension_list(sbi, name, hot, !set);
f2fs_update_extension_list(sbi, name, hot, !set);
out:
up_write(&sbi->sb_lock);
return ret ? ret : count;
@ -248,17 +248,51 @@ out:
if (!strcmp(a->attr.name, "trim_sections"))
return -EINVAL;
if (!strcmp(a->attr.name, "gc_urgent")) {
if (t >= 1) {
sbi->gc_mode = GC_URGENT;
if (sbi->gc_thread) {
wake_up_interruptible_all(
&sbi->gc_thread->gc_wait_queue_head);
wake_up_discard_thread(sbi, true);
}
} else {
sbi->gc_mode = GC_NORMAL;
}
return count;
}
if (!strcmp(a->attr.name, "gc_idle")) {
if (t == GC_IDLE_CB)
sbi->gc_mode = GC_IDLE_CB;
else if (t == GC_IDLE_GREEDY)
sbi->gc_mode = GC_IDLE_GREEDY;
else
sbi->gc_mode = GC_NORMAL;
return count;
}
*ui = t;
if (!strcmp(a->attr.name, "iostat_enable") && *ui == 0)
f2fs_reset_iostat(sbi);
if (!strcmp(a->attr.name, "gc_urgent") && t == 1 && sbi->gc_thread) {
sbi->gc_thread->gc_wake = 1;
wake_up_interruptible_all(&sbi->gc_thread->gc_wait_queue_head);
wake_up_discard_thread(sbi, true);
return count;
}
return count;
static ssize_t f2fs_sbi_store(struct f2fs_attr *a,
struct f2fs_sb_info *sbi,
const char *buf, size_t count)
{
ssize_t ret;
bool gc_entry = (!strcmp(a->attr.name, "gc_urgent") ||
a->struct_type == GC_THREAD);
if (gc_entry)
down_read(&sbi->sb->s_umount);
ret = __sbi_store(a, sbi, buf, count);
if (gc_entry)
up_read(&sbi->sb->s_umount);
return ret;
}
static ssize_t f2fs_attr_show(struct kobject *kobj,
@ -349,8 +383,8 @@ F2FS_RW_ATTR(GC_THREAD, f2fs_gc_kthread, gc_urgent_sleep_time,
F2FS_RW_ATTR(GC_THREAD, f2fs_gc_kthread, gc_min_sleep_time, min_sleep_time);
F2FS_RW_ATTR(GC_THREAD, f2fs_gc_kthread, gc_max_sleep_time, max_sleep_time);
F2FS_RW_ATTR(GC_THREAD, f2fs_gc_kthread, gc_no_gc_sleep_time, no_gc_sleep_time);
F2FS_RW_ATTR(GC_THREAD, f2fs_gc_kthread, gc_idle, gc_idle);
F2FS_RW_ATTR(GC_THREAD, f2fs_gc_kthread, gc_urgent, gc_urgent);
F2FS_RW_ATTR(F2FS_SBI, f2fs_sb_info, gc_idle, gc_mode);
F2FS_RW_ATTR(F2FS_SBI, f2fs_sb_info, gc_urgent, gc_mode);
F2FS_RW_ATTR(SM_INFO, f2fs_sm_info, reclaim_segments, rec_prefree_segments);
F2FS_RW_ATTR(DCC_INFO, discard_cmd_control, max_small_discards, max_discards);
F2FS_RW_ATTR(DCC_INFO, discard_cmd_control, discard_granularity, discard_granularity);

26
fs/f2fs/xattr.c
View file

@ -299,7 +299,7 @@ static int read_inline_xattr(struct inode *inode, struct page *ipage,
if (ipage) {
inline_addr = inline_xattr_addr(inode, ipage);
} else {
page = get_node_page(sbi, inode->i_ino);
page = f2fs_get_node_page(sbi, inode->i_ino);
if (IS_ERR(page))
return PTR_ERR(page);
@ -320,7 +320,7 @@ static int read_xattr_block(struct inode *inode, void *txattr_addr)
void *xattr_addr;
/* The inode already has an extended attribute block. */
xpage = get_node_page(sbi, xnid);
xpage = f2fs_get_node_page(sbi, xnid);
if (IS_ERR(xpage))
return PTR_ERR(xpage);
@ -444,7 +444,7 @@ static inline int write_all_xattrs(struct inode *inode, __u32 hsize,
int err = 0;
if (hsize > inline_size && !F2FS_I(inode)->i_xattr_nid)
if (!alloc_nid(sbi, &new_nid))
if (!f2fs_alloc_nid(sbi, &new_nid))
return -ENOSPC;
/* write to inline xattr */
@ -452,9 +452,9 @@ static inline int write_all_xattrs(struct inode *inode, __u32 hsize,
if (ipage) {
inline_addr = inline_xattr_addr(inode, ipage);
} else {
in_page = get_node_page(sbi, inode->i_ino);
in_page = f2fs_get_node_page(sbi, inode->i_ino);
if (IS_ERR(in_page)) {
alloc_nid_failed(sbi, new_nid);
f2fs_alloc_nid_failed(sbi, new_nid);
return PTR_ERR(in_page);
}
inline_addr = inline_xattr_addr(inode, in_page);
@ -464,8 +464,8 @@ static inline int write_all_xattrs(struct inode *inode, __u32 hsize,
NODE, true);
/* no need to use xattr node block */
if (hsize <= inline_size) {
err = truncate_xattr_node(inode);
alloc_nid_failed(sbi, new_nid);
err = f2fs_truncate_xattr_node(inode);
f2fs_alloc_nid_failed(sbi, new_nid);
if (err) {
f2fs_put_page(in_page, 1);
return err;
@ -478,10 +478,10 @@ static inline int write_all_xattrs(struct inode *inode, __u32 hsize,
/* write to xattr node block */
if (F2FS_I(inode)->i_xattr_nid) {
xpage = get_node_page(sbi, F2FS_I(inode)->i_xattr_nid);
xpage = f2fs_get_node_page(sbi, F2FS_I(inode)->i_xattr_nid);
if (IS_ERR(xpage)) {
err = PTR_ERR(xpage);
alloc_nid_failed(sbi, new_nid);
f2fs_alloc_nid_failed(sbi, new_nid);
goto in_page_out;
}
f2fs_bug_on(sbi, new_nid);
@ -489,13 +489,13 @@ static inline int write_all_xattrs(struct inode *inode, __u32 hsize,
} else {
struct dnode_of_data dn;
set_new_dnode(&dn, inode, NULL, NULL, new_nid);
xpage = new_node_page(&dn, XATTR_NODE_OFFSET);
xpage = f2fs_new_node_page(&dn, XATTR_NODE_OFFSET);
if (IS_ERR(xpage)) {
err = PTR_ERR(xpage);
alloc_nid_failed(sbi, new_nid);
f2fs_alloc_nid_failed(sbi, new_nid);
goto in_page_out;
}
alloc_nid_done(sbi, new_nid);
f2fs_alloc_nid_done(sbi, new_nid);
}
xattr_addr = page_address(xpage);
@ -733,7 +733,7 @@ int f2fs_setxattr(struct inode *inode, int index, const char *name,
if (err)
return err;
/* this case is only from init_inode_metadata */
/* this case is only from f2fs_init_inode_metadata */
if (ipage)
return __f2fs_setxattr(inode, index, name, value,
size, ipage, flags);

20
fs/gfs2/aops.c
View file

@ -249,22 +249,6 @@ static int gfs2_write_jdata_pagevec(struct address_space *mapping,
for(i = 0; i < nr_pages; i++) {
struct page *page = pvec->pages[i];
/*
* At this point, the page may be truncated or
* invalidated (changing page->mapping to NULL), or
* even swizzled back from swapper_space to tmpfs file
* mapping. However, page->index will not change
* because we have a reference on the page.
*/
if (page->index > end) {
/*
* can't be range_cyclic (1st pass) because
* end == -1 in that case.
*/
ret = 1;
break;
}
*done_index = page->index;
lock_page(page);
@ -382,8 +366,8 @@ retry:
tag_pages_for_writeback(mapping, index, end);
done_index = index;
while (!done && (index <= end)) {
nr_pages = pagevec_lookup_tag(&pvec, mapping, &index, tag,
min(end - index, (pgoff_t)PAGEVEC_SIZE-1) + 1);
nr_pages = pagevec_lookup_range_tag(&pvec, mapping, &index, end,
tag);
if (nr_pages == 0)
break;

4
fs/nilfs2/btree.c
View file

@ -2147,8 +2147,8 @@ static void nilfs_btree_lookup_dirty_buffers(struct nilfs_bmap *btree,
pagevec_init(&pvec, 0);
while (pagevec_lookup_tag(&pvec, btcache, &index, PAGECACHE_TAG_DIRTY,
PAGEVEC_SIZE)) {
while (pagevec_lookup_tag(&pvec, btcache, &index,
PAGECACHE_TAG_DIRTY)) {
for (i = 0; i < pagevec_count(&pvec); i++) {
bh = head = page_buffers(pvec.pages[i]);
do {

7
fs/nilfs2/page.c
View file

@ -262,8 +262,7 @@ int nilfs_copy_dirty_pages(struct address_space *dmap,
pagevec_init(&pvec, 0);
repeat:
if (!pagevec_lookup_tag(&pvec, smap, &index, PAGECACHE_TAG_DIRTY,
PAGEVEC_SIZE))
if (!pagevec_lookup_tag(&pvec, smap, &index, PAGECACHE_TAG_DIRTY))
return 0;
for (i = 0; i < pagevec_count(&pvec); i++) {
@ -382,8 +381,8 @@ void nilfs_clear_dirty_pages(struct address_space *mapping, bool silent)
pagevec_init(&pvec, 0);
while (pagevec_lookup_tag(&pvec, mapping, &index, PAGECACHE_TAG_DIRTY,
PAGEVEC_SIZE)) {
while (pagevec_lookup_tag(&pvec, mapping, &index,
PAGECACHE_TAG_DIRTY)) {
for (i = 0; i < pagevec_count(&pvec); i++) {
struct page *page = pvec.pages[i];

12
fs/nilfs2/segment.c
View file

@ -705,18 +705,14 @@ static size_t nilfs_lookup_dirty_data_buffers(struct inode *inode,
pagevec_init(&pvec, 0);
repeat:
if (unlikely(index > last) ||
!pagevec_lookup_tag(&pvec, mapping, &index, PAGECACHE_TAG_DIRTY,
min_t(pgoff_t, last - index,
PAGEVEC_SIZE - 1) + 1))
!pagevec_lookup_range_tag(&pvec, mapping, &index, last,
PAGECACHE_TAG_DIRTY))
return ndirties;
for (i = 0; i < pagevec_count(&pvec); i++) {
struct buffer_head *bh, *head;
struct page *page = pvec.pages[i];
if (unlikely(page->index > last))
break;
lock_page(page);
if (!page_has_buffers(page))
create_empty_buffers(page, i_blocksize(inode), 0);
@ -753,8 +749,8 @@ static void nilfs_lookup_dirty_node_buffers(struct inode *inode,
pagevec_init(&pvec, 0);
while (pagevec_lookup_tag(&pvec, mapping, &index, PAGECACHE_TAG_DIRTY,
PAGEVEC_SIZE)) {
while (pagevec_lookup_tag(&pvec, mapping, &index,
PAGECACHE_TAG_DIRTY)) {
for (i = 0; i < pagevec_count(&pvec); i++) {
bh = head = page_buffers(pvec.pages[i]);
do {

7
fs/sdcardfs/main.c
View file

@ -295,6 +295,13 @@ static int sdcardfs_read_super(struct vfsmount *mnt, struct super_block *sb,
atomic_inc(&lower_sb->s_active);
sdcardfs_set_lower_super(sb, lower_sb);
sb->s_stack_depth = lower_sb->s_stack_depth + 1;
if (sb->s_stack_depth > FILESYSTEM_MAX_STACK_DEPTH) {
pr_err("sdcardfs: maximum fs stacking depth exceeded\n");
err = -EINVAL;
goto out_sput;
}
/* inherit maxbytes from lower file system */
sb->s_maxbytes = lower_sb->s_maxbytes;

87
include/crypto/internal/skcipher.h
View file

@ -19,12 +19,46 @@
struct rtattr;
struct skcipher_instance {
void (*free)(struct skcipher_instance *inst);
union {
struct {
char head[offsetof(struct skcipher_alg, base)];
struct crypto_instance base;
} s;
struct skcipher_alg alg;
};
};
struct crypto_skcipher_spawn {
struct crypto_spawn base;
};
extern const struct crypto_type crypto_givcipher_type;
static inline struct crypto_instance *skcipher_crypto_instance(
struct skcipher_instance *inst)
{
return &inst->s.base;
}
static inline struct skcipher_instance *skcipher_alg_instance(
struct crypto_skcipher *skcipher)
{
return container_of(crypto_skcipher_alg(skcipher),
struct skcipher_instance, alg);
}
static inline void *skcipher_instance_ctx(struct skcipher_instance *inst)
{
return crypto_instance_ctx(skcipher_crypto_instance(inst));
}
static inline void skcipher_request_complete(struct skcipher_request *req, int err)
{
req->base.complete(&req->base, err);
}
static inline void crypto_set_skcipher_spawn(
struct crypto_skcipher_spawn *spawn, struct crypto_instance *inst)
{
@ -33,6 +67,8 @@ static inline void crypto_set_skcipher_spawn(
int crypto_grab_skcipher(struct crypto_skcipher_spawn *spawn, const char *name,
u32 type, u32 mask);
int crypto_grab_skcipher2(struct crypto_skcipher_spawn *spawn,
const char *name, u32 type, u32 mask);
struct crypto_alg *crypto_lookup_skcipher(const char *name, u32 type, u32 mask);
@ -47,6 +83,12 @@ static inline struct crypto_alg *crypto_skcipher_spawn_alg(
return spawn->base.alg;
}
static inline struct skcipher_alg *crypto_spawn_skcipher_alg(
struct crypto_skcipher_spawn *spawn)
{
return container_of(spawn->base.alg, struct skcipher_alg, base);
}
static inline struct crypto_ablkcipher *crypto_spawn_skcipher(
struct crypto_skcipher_spawn *spawn)
{
@ -55,6 +97,25 @@ static inline struct crypto_ablkcipher *crypto_spawn_skcipher(
crypto_skcipher_mask(0)));
}
static inline struct crypto_skcipher *crypto_spawn_skcipher2(
struct crypto_skcipher_spawn *spawn)
{
return crypto_spawn_tfm2(&spawn->base);
}
static inline void crypto_skcipher_set_reqsize(
struct crypto_skcipher *skcipher, unsigned int reqsize)
{
skcipher->reqsize = reqsize;
}
int crypto_register_skcipher(struct skcipher_alg *alg);
void crypto_unregister_skcipher(struct skcipher_alg *alg);
int crypto_register_skciphers(struct skcipher_alg *algs, int count);
void crypto_unregister_skciphers(struct skcipher_alg *algs, int count);
int skcipher_register_instance(struct crypto_template *tmpl,
struct skcipher_instance *inst);
int skcipher_null_givencrypt(struct skcipher_givcrypt_request *req);
int skcipher_null_givdecrypt(struct skcipher_givcrypt_request *req);
const char *crypto_default_geniv(const struct crypto_alg *alg);
@ -122,5 +183,31 @@ static inline u32 skcipher_request_flags(struct skcipher_request *req)
return req->base.flags;
}
static inline unsigned int crypto_skcipher_alg_min_keysize(
struct skcipher_alg *alg)
{
if ((alg->base.cra_flags & CRYPTO_ALG_TYPE_MASK) ==
CRYPTO_ALG_TYPE_BLKCIPHER)
return alg->base.cra_blkcipher.min_keysize;
if (alg->base.cra_ablkcipher.encrypt)
return alg->base.cra_ablkcipher.min_keysize;
return alg->min_keysize;
}
static inline unsigned int crypto_skcipher_alg_max_keysize(
struct skcipher_alg *alg)
{
if ((alg->base.cra_flags & CRYPTO_ALG_TYPE_MASK) ==
CRYPTO_ALG_TYPE_BLKCIPHER)
return alg->base.cra_blkcipher.max_keysize;
if (alg->base.cra_ablkcipher.encrypt)
return alg->base.cra_ablkcipher.max_keysize;
return alg->max_keysize;
}
#endif /* _CRYPTO_INTERNAL_SKCIPHER_H */

147
include/crypto/skcipher.h
View file

@ -60,12 +60,80 @@ struct crypto_skcipher {
unsigned int ivsize;
unsigned int reqsize;
bool has_setkey;
unsigned int keysize;
struct crypto_tfm base;
};
/**
* struct skcipher_alg - symmetric key cipher definition
* @min_keysize: Minimum key size supported by the transformation. This is the
* smallest key length supported by this transformation algorithm.
* This must be set to one of the pre-defined values as this is
* not hardware specific. Possible values for this field can be
* found via git grep "_MIN_KEY_SIZE" include/crypto/
* @max_keysize: Maximum key size supported by the transformation. This is the
* largest key length supported by this transformation algorithm.
* This must be set to one of the pre-defined values as this is
* not hardware specific. Possible values for this field can be
* found via git grep "_MAX_KEY_SIZE" include/crypto/
* @setkey: Set key for the transformation. This function is used to either
* program a supplied key into the hardware or store the key in the
* transformation context for programming it later. Note that this
* function does modify the transformation context. This function can
* be called multiple times during the existence of the transformation
* object, so one must make sure the key is properly reprogrammed into
* the hardware. This function is also responsible for checking the key
* length for validity. In case a software fallback was put in place in
* the @cra_init call, this function might need to use the fallback if
* the algorithm doesn't support all of the key sizes.
* @encrypt: Encrypt a scatterlist of blocks. This function is used to encrypt
* the supplied scatterlist containing the blocks of data. The crypto
* API consumer is responsible for aligning the entries of the
* scatterlist properly and making sure the chunks are correctly
* sized. In case a software fallback was put in place in the
* @cra_init call, this function might need to use the fallback if
* the algorithm doesn't support all of the key sizes. In case the
* key was stored in transformation context, the key might need to be
* re-programmed into the hardware in this function. This function
* shall not modify the transformation context, as this function may
* be called in parallel with the same transformation object.
* @decrypt: Decrypt a single block. This is a reverse counterpart to @encrypt
* and the conditions are exactly the same.
* @init: Initialize the cryptographic transformation object. This function
* is used to initialize the cryptographic transformation object.
* This function is called only once at the instantiation time, right
* after the transformation context was allocated. In case the
* cryptographic hardware has some special requirements which need to
* be handled by software, this function shall check for the precise
* requirement of the transformation and put any software fallbacks
* in place.
* @exit: Deinitialize the cryptographic transformation object. This is a
* counterpart to @init, used to remove various changes set in
* @init.
* @ivsize: IV size applicable for transformation. The consumer must provide an
* IV of exactly that size to perform the encrypt or decrypt operation.
* @chunksize: Equal to the block size except for stream ciphers such as
* CTR where it is set to the underlying block size.
*
* All fields except @ivsize are mandatory and must be filled.
*/
struct skcipher_alg {
int (*setkey)(struct crypto_skcipher *tfm, const u8 *key,
unsigned int keylen);
int (*encrypt)(struct skcipher_request *req);
int (*decrypt)(struct skcipher_request *req);
int (*init)(struct crypto_skcipher *tfm);
void (*exit)(struct crypto_skcipher *tfm);
unsigned int min_keysize;
unsigned int max_keysize;
unsigned int ivsize;
unsigned int chunksize;
struct crypto_alg base;
};
#define SKCIPHER_REQUEST_ON_STACK(name, tfm) \
char __##name##_desc[sizeof(struct skcipher_request) + \
crypto_skcipher_reqsize(tfm)] CRYPTO_MINALIGN_ATTR; \
@ -232,6 +300,43 @@ static inline int crypto_has_skcipher(const char *alg_name, u32 type,
crypto_skcipher_mask(mask));
}
/**
* crypto_has_skcipher2() - Search for the availability of an skcipher.
* @alg_name: is the cra_name / name or cra_driver_name / driver name of the
* skcipher
* @type: specifies the type of the skcipher
* @mask: specifies the mask for the skcipher
*
* Return: true when the skcipher is known to the kernel crypto API; false
* otherwise
*/
int crypto_has_skcipher2(const char *alg_name, u32 type, u32 mask);
static inline const char *crypto_skcipher_driver_name(
struct crypto_skcipher *tfm)
{
return crypto_tfm_alg_name(crypto_skcipher_tfm(tfm));
}
static inline struct skcipher_alg *crypto_skcipher_alg(
struct crypto_skcipher *tfm)
{
return container_of(crypto_skcipher_tfm(tfm)->__crt_alg,
struct skcipher_alg, base);
}
static inline unsigned int crypto_skcipher_alg_ivsize(struct skcipher_alg *alg)
{
if ((alg->base.cra_flags & CRYPTO_ALG_TYPE_MASK) ==
CRYPTO_ALG_TYPE_BLKCIPHER)
return alg->base.cra_blkcipher.ivsize;
if (alg->base.cra_ablkcipher.encrypt)
return alg->base.cra_ablkcipher.ivsize;
return alg->ivsize;
}
/**
* crypto_skcipher_ivsize() - obtain IV size
* @tfm: cipher handle
@ -246,6 +351,36 @@ static inline unsigned int crypto_skcipher_ivsize(struct crypto_skcipher *tfm)
return tfm->ivsize;
}
static inline unsigned int crypto_skcipher_alg_chunksize(
struct skcipher_alg *alg)
{
if ((alg->base.cra_flags & CRYPTO_ALG_TYPE_MASK) ==
CRYPTO_ALG_TYPE_BLKCIPHER)
return alg->base.cra_blocksize;
if (alg->base.cra_ablkcipher.encrypt)
return alg->base.cra_blocksize;
return alg->chunksize;
}
/**
* crypto_skcipher_chunksize() - obtain chunk size
* @tfm: cipher handle
*
* The block size is set to one for ciphers such as CTR. However,
* you still need to provide incremental updates in multiples of
* the underlying block size as the IV does not have sub-block
* granularity. This is known in this API as the chunk size.
*
* Return: chunk size in bytes
*/
static inline unsigned int crypto_skcipher_chunksize(
struct crypto_skcipher *tfm)
{
return crypto_skcipher_alg_chunksize(crypto_skcipher_alg(tfm));
}
/**
* crypto_skcipher_blocksize() - obtain block size of cipher
* @tfm: cipher handle
@ -309,7 +444,13 @@ static inline int crypto_skcipher_setkey(struct crypto_skcipher *tfm,
static inline bool crypto_skcipher_has_setkey(struct crypto_skcipher *tfm)
{
return tfm->has_setkey;
return tfm->keysize;
}
static inline unsigned int crypto_skcipher_default_keysize(
struct crypto_skcipher *tfm)
{
return tfm->keysize;
}
/**

6
include/linux/cpufreq_times.h
View file

@ -22,6 +22,7 @@
#ifdef CONFIG_CPU_FREQ_TIMES
void cpufreq_task_times_init(struct task_struct *p);
void cpufreq_task_times_alloc(struct task_struct *p);
void cpufreq_task_times_exit(struct task_struct *p);
int proc_time_in_state_show(struct seq_file *m, struct pid_namespace *ns,
struct pid *pid, struct task_struct *p);
@ -31,6 +32,11 @@ void cpufreq_times_record_transition(struct cpufreq_freqs *freq);
void cpufreq_task_times_remove_uids(uid_t uid_start, uid_t uid_end);
int single_uid_time_in_state_open(struct inode *inode, struct file *file);
#else
static inline void cpufreq_task_times_init(struct task_struct *p) {}
static inline void cpufreq_task_times_alloc(struct task_struct *p) {}
static inline void cpufreq_task_times_exit(struct task_struct *p) {}
static inline void cpufreq_acct_update_power(struct task_struct *p,
u64 cputime) {}
static inline void cpufreq_times_create_policy(struct cpufreq_policy *policy) {}
static inline void cpufreq_times_record_transition(
struct cpufreq_freqs *freq) {}

1
include/linux/crypto.h
View file

@ -48,6 +48,7 @@
#define CRYPTO_ALG_TYPE_AEAD 0x00000003
#define CRYPTO_ALG_TYPE_BLKCIPHER 0x00000004
#define CRYPTO_ALG_TYPE_ABLKCIPHER 0x00000005
#define CRYPTO_ALG_TYPE_SKCIPHER 0x00000005
#define CRYPTO_ALG_TYPE_GIVCIPHER 0x00000006
#define CRYPTO_ALG_TYPE_DIGEST 0x00000008
#define CRYPTO_ALG_TYPE_HASH 0x00000008

10
include/linux/fscrypt_notsupp.h
View file

@ -67,16 +67,6 @@ static inline void fscrypt_restore_control_page(struct page *page)
return;
}
static inline void fscrypt_set_d_op(struct dentry *dentry)
{
return;
}
static inline void fscrypt_set_encrypted_dentry(struct dentry *dentry)
{
return;
}
/* policy.c */
static inline int fscrypt_ioctl_set_policy(struct file *filp,
const void __user *arg)

16
include/linux/fscrypt_supp.h
View file

@ -28,7 +28,7 @@ struct fscrypt_operations {
int (*set_context)(struct inode *, const void *, size_t, void *);
bool (*dummy_context)(struct inode *);
bool (*empty_dir)(struct inode *);
unsigned (*max_namelen)(struct inode *);
unsigned int max_namelen;
};
struct fscrypt_ctx {
@ -74,20 +74,6 @@ static inline struct page *fscrypt_control_page(struct page *page)
extern void fscrypt_restore_control_page(struct page *);
extern const struct dentry_operations fscrypt_d_ops;
static inline void fscrypt_set_d_op(struct dentry *dentry)
{
d_set_d_op(dentry, &fscrypt_d_ops);
}
static inline void fscrypt_set_encrypted_dentry(struct dentry *dentry)
{
spin_lock(&dentry->d_lock);
dentry->d_flags |= DCACHE_ENCRYPTED_WITH_KEY;
spin_unlock(&dentry->d_lock);
}
/* policy.c */
extern int fscrypt_ioctl_set_policy(struct file *, const void __user *);
extern int fscrypt_ioctl_get_policy(struct file *, void __user *);

411
include/linux/fscrypto.h Normal file
View file

@ -0,0 +1,411 @@
/*
* General per-file encryption definition
*
* Copyright (C) 2015, Google, Inc.
*
* Written by Michael Halcrow, 2015.
* Modified by Jaegeuk Kim, 2015.
*/
#ifndef _LINUX_FSCRYPTO_H
#define _LINUX_FSCRYPTO_H
#include <linux/key.h>
#include <linux/fs.h>
#include <linux/mm.h>
#include <linux/bio.h>
#include <linux/dcache.h>
#include <crypto/skcipher.h>
#include <uapi/linux/fs.h>
#define FS_KEY_DERIVATION_NONCE_SIZE 16
#define FS_ENCRYPTION_CONTEXT_FORMAT_V1 1
#define FS_POLICY_FLAGS_PAD_4 0x00
#define FS_POLICY_FLAGS_PAD_8 0x01
#define FS_POLICY_FLAGS_PAD_16 0x02
#define FS_POLICY_FLAGS_PAD_32 0x03
#define FS_POLICY_FLAGS_PAD_MASK 0x03
#define FS_POLICY_FLAGS_VALID 0x03
/* Encryption algorithms */
#define FS_ENCRYPTION_MODE_INVALID 0
#define FS_ENCRYPTION_MODE_AES_256_XTS 1
#define FS_ENCRYPTION_MODE_AES_256_GCM 2
#define FS_ENCRYPTION_MODE_AES_256_CBC 3
#define FS_ENCRYPTION_MODE_AES_256_CTS 4
/**
* Encryption context for inode
*
* Protector format:
* 1 byte: Protector format (1 = this version)
* 1 byte: File contents encryption mode
* 1 byte: File names encryption mode
* 1 byte: Flags
* 8 bytes: Master Key descriptor
* 16 bytes: Encryption Key derivation nonce
*/
struct fscrypt_context {
u8 format;
u8 contents_encryption_mode;
u8 filenames_encryption_mode;
u8 flags;
u8 master_key_descriptor[FS_KEY_DESCRIPTOR_SIZE];
u8 nonce[FS_KEY_DERIVATION_NONCE_SIZE];
} __packed;
/* Encryption parameters */
#define FS_XTS_TWEAK_SIZE 16
#define FS_AES_128_ECB_KEY_SIZE 16
#define FS_AES_256_GCM_KEY_SIZE 32
#define FS_AES_256_CBC_KEY_SIZE 32
#define FS_AES_256_CTS_KEY_SIZE 32
#define FS_AES_256_XTS_KEY_SIZE 64
#define FS_MAX_KEY_SIZE 64
#define FS_KEY_DESC_PREFIX "fscrypt:"
#define FS_KEY_DESC_PREFIX_SIZE 8
/* This is passed in from userspace into the kernel keyring */
struct fscrypt_key {
u32 mode;
u8 raw[FS_MAX_KEY_SIZE];
u32 size;
} __packed;
struct fscrypt_info {
u8 ci_data_mode;
u8 ci_filename_mode;
u8 ci_flags;
struct crypto_skcipher *ci_ctfm;
struct key *ci_keyring_key;
u8 ci_master_key[FS_KEY_DESCRIPTOR_SIZE];
};
#define FS_CTX_REQUIRES_FREE_ENCRYPT_FL 0x00000001
#define FS_WRITE_PATH_FL 0x00000002
struct fscrypt_ctx {
union {
struct {
struct page *bounce_page; /* Ciphertext page */
struct page *control_page; /* Original page */
} w;
struct {
struct bio *bio;
struct work_struct work;
} r;
struct list_head free_list; /* Free list */
};
u8 flags; /* Flags */
u8 mode; /* Encryption mode for tfm */
};
struct fscrypt_completion_result {
struct completion completion;
int res;
};
#define DECLARE_FS_COMPLETION_RESULT(ecr) \
struct fscrypt_completion_result ecr = { \
COMPLETION_INITIALIZER((ecr).completion), 0 }
#define FS_FNAME_NUM_SCATTER_ENTRIES 4
#define FS_CRYPTO_BLOCK_SIZE 16
#define FS_FNAME_CRYPTO_DIGEST_SIZE 32
/**
* For encrypted symlinks, the ciphertext length is stored at the beginning
* of the string in little-endian format.
*/
struct fscrypt_symlink_data {
__le16 len;
char encrypted_path[1];
} __packed;
/**
* This function is used to calculate the disk space required to
* store a filename of length l in encrypted symlink format.
*/
static inline u32 fscrypt_symlink_data_len(u32 l)
{
if (l < FS_CRYPTO_BLOCK_SIZE)
l = FS_CRYPTO_BLOCK_SIZE;
return (l + sizeof(struct fscrypt_symlink_data) - 1);
}
struct fscrypt_str {
unsigned char *name;
u32 len;
};
struct fscrypt_name {
const struct qstr *usr_fname;
struct fscrypt_str disk_name;
u32 hash;
u32 minor_hash;
struct fscrypt_str crypto_buf;
};
#define FSTR_INIT(n, l) { .name = n, .len = l }
#define FSTR_TO_QSTR(f) QSTR_INIT((f)->name, (f)->len)
#define fname_name(p) ((p)->disk_name.name)
#define fname_len(p) ((p)->disk_name.len)
/*
* crypto opertions for filesystems
*/
struct fscrypt_operations {
int (*get_context)(struct inode *, void *, size_t);
int (*key_prefix)(struct inode *, u8 **);
int (*prepare_context)(struct inode *);
int (*set_context)(struct inode *, const void *, size_t, void *);
int (*dummy_context)(struct inode *);
bool (*is_encrypted)(struct inode *);
bool (*empty_dir)(struct inode *);
unsigned (*max_namelen)(struct inode *);
};
static inline bool fscrypt_dummy_context_enabled(struct inode *inode)
{
if (inode->i_sb->s_cop->dummy_context &&
inode->i_sb->s_cop->dummy_context(inode))
return true;
return false;
}
static inline bool fscrypt_valid_contents_enc_mode(u32 mode)
{
return (mode == FS_ENCRYPTION_MODE_AES_256_XTS);
}
static inline bool fscrypt_valid_filenames_enc_mode(u32 mode)
{
return (mode == FS_ENCRYPTION_MODE_AES_256_CTS);
}
static inline bool fscrypt_is_dot_dotdot(const struct qstr *str)
{
if (str->len == 1 && str->name[0] == '.')
return true;
if (str->len == 2 && str->name[0] == '.' && str->name[1] == '.')
return true;
return false;
}
static inline struct page *fscrypt_control_page(struct page *page)
{
#if IS_ENABLED(CONFIG_FS_ENCRYPTION)
return ((struct fscrypt_ctx *)page_private(page))->w.control_page;
#else
WARN_ON_ONCE(1);
return ERR_PTR(-EINVAL);
#endif
}
static inline int fscrypt_has_encryption_key(struct inode *inode)
{
#if IS_ENABLED(CONFIG_FS_ENCRYPTION)
return (inode->i_crypt_info != NULL);
#else
return 0;
#endif
}
static inline void fscrypt_set_encrypted_dentry(struct dentry *dentry)
{
#if IS_ENABLED(CONFIG_FS_ENCRYPTION)
spin_lock(&dentry->d_lock);
dentry->d_flags |= DCACHE_ENCRYPTED_WITH_KEY;
spin_unlock(&dentry->d_lock);
#endif
}
#if IS_ENABLED(CONFIG_FS_ENCRYPTION)
extern const struct dentry_operations fscrypt_d_ops;
#endif
static inline void fscrypt_set_d_op(struct dentry *dentry)
{
#if IS_ENABLED(CONFIG_FS_ENCRYPTION)
d_set_d_op(dentry, &fscrypt_d_ops);
#endif
}
#if IS_ENABLED(CONFIG_FS_ENCRYPTION)
/* crypto.c */
extern struct kmem_cache *fscrypt_info_cachep;
int fscrypt_initialize(void);
extern struct fscrypt_ctx *fscrypt_get_ctx(struct inode *, gfp_t);
extern void fscrypt_release_ctx(struct fscrypt_ctx *);
extern struct page *fscrypt_encrypt_page(struct inode *, struct page *, gfp_t);
extern int fscrypt_decrypt_page(struct page *);
extern void fscrypt_decrypt_bio_pages(struct fscrypt_ctx *, struct bio *);
extern void fscrypt_pullback_bio_page(struct page **, bool);
extern void fscrypt_restore_control_page(struct page *);
extern int fscrypt_zeroout_range(struct inode *, pgoff_t, sector_t,
unsigned int);
/* policy.c */
extern int fscrypt_ioctl_set_policy(struct file *, const void __user *);
extern int fscrypt_ioctl_get_policy(struct file *, void __user *);
extern int fscrypt_has_permitted_context(struct inode *, struct inode *);
extern int fscrypt_inherit_context(struct inode *, struct inode *,
void *, bool);
/* keyinfo.c */
extern int get_crypt_info(struct inode *);
extern int fscrypt_get_encryption_info(struct inode *);
extern void fscrypt_put_encryption_info(struct inode *, struct fscrypt_info *);
/* fname.c */
extern int fscrypt_setup_filename(struct inode *, const struct qstr *,
int lookup, struct fscrypt_name *);
extern void fscrypt_free_filename(struct fscrypt_name *);
extern u32 fscrypt_fname_encrypted_size(struct inode *, u32);
extern int fscrypt_fname_alloc_buffer(struct inode *, u32,
struct fscrypt_str *);
extern void fscrypt_fname_free_buffer(struct fscrypt_str *);
extern int fscrypt_fname_disk_to_usr(struct inode *, u32, u32,
const struct fscrypt_str *, struct fscrypt_str *);
extern int fscrypt_fname_usr_to_disk(struct inode *, const struct qstr *,
struct fscrypt_str *);
#endif
/* crypto.c */
static inline struct fscrypt_ctx *fscrypt_notsupp_get_ctx(struct inode *i,
gfp_t f)
{
return ERR_PTR(-EOPNOTSUPP);
}
static inline void fscrypt_notsupp_release_ctx(struct fscrypt_ctx *c)
{
return;
}
static inline struct page *fscrypt_notsupp_encrypt_page(struct inode *i,
struct page *p, gfp_t f)
{
return ERR_PTR(-EOPNOTSUPP);
}
static inline int fscrypt_notsupp_decrypt_page(struct page *p)
{
return -EOPNOTSUPP;
}
static inline void fscrypt_notsupp_decrypt_bio_pages(struct fscrypt_ctx *c,
struct bio *b)
{
return;
}
static inline void fscrypt_notsupp_pullback_bio_page(struct page **p, bool b)
{
return;
}
static inline void fscrypt_notsupp_restore_control_page(struct page *p)
{
return;
}
static inline int fscrypt_notsupp_zeroout_range(struct inode *i, pgoff_t p,
sector_t s, unsigned int f)
{
return -EOPNOTSUPP;
}
/* policy.c */
static inline int fscrypt_notsupp_ioctl_set_policy(struct file *f,
const void __user *arg)
{
return -EOPNOTSUPP;
}
static inline int fscrypt_notsupp_ioctl_get_policy(struct file *f,
void __user *arg)
{
return -EOPNOTSUPP;
}
static inline int fscrypt_notsupp_has_permitted_context(struct inode *p,
struct inode *i)
{
return 0;
}
static inline int fscrypt_notsupp_inherit_context(struct inode *p,
struct inode *i, void *v, bool b)
{
return -EOPNOTSUPP;
}
/* keyinfo.c */
static inline int fscrypt_notsupp_get_encryption_info(struct inode *i)
{
return -EOPNOTSUPP;
}
static inline void fscrypt_notsupp_put_encryption_info(struct inode *i,
struct fscrypt_info *f)
{
return;
}
/* fname.c */
static inline int fscrypt_notsupp_setup_filename(struct inode *dir,
const struct qstr *iname,
int lookup, struct fscrypt_name *fname)
{
if (dir->i_sb->s_cop->is_encrypted(dir))
return -EOPNOTSUPP;
memset(fname, 0, sizeof(struct fscrypt_name));
fname->usr_fname = iname;
fname->disk_name.name = (unsigned char *)iname->name;
fname->disk_name.len = iname->len;
return 0;
}
static inline void fscrypt_notsupp_free_filename(struct fscrypt_name *fname)
{
return;
}
static inline u32 fscrypt_notsupp_fname_encrypted_size(struct inode *i, u32 s)
{
/* never happens */
WARN_ON(1);
return 0;
}
static inline int fscrypt_notsupp_fname_alloc_buffer(struct inode *inode,
u32 ilen, struct fscrypt_str *crypto_str)
{
return -EOPNOTSUPP;
}
static inline void fscrypt_notsupp_fname_free_buffer(struct fscrypt_str *c)
{
return;
}
static inline int fscrypt_notsupp_fname_disk_to_usr(struct inode *inode,
u32 hash, u32 minor_hash,
const struct fscrypt_str *iname,
struct fscrypt_str *oname)
{
return -EOPNOTSUPP;
}
static inline int fscrypt_notsupp_fname_usr_to_disk(struct inode *inode,
const struct qstr *iname,
struct fscrypt_str *oname)
{
return -EOPNOTSUPP;
}
#endif /* _LINUX_FSCRYPTO_H */

278
include/linux/overflow.h Normal file
View file

@ -0,0 +1,278 @@
/* SPDX-License-Identifier: GPL-2.0 OR MIT */
#ifndef __LINUX_OVERFLOW_H
#define __LINUX_OVERFLOW_H
#include <linux/compiler.h>
/*
* In the fallback code below, we need to compute the minimum and
* maximum values representable in a given type. These macros may also
* be useful elsewhere, so we provide them outside the
* COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW block.
*
* It would seem more obvious to do something like
*
* #define type_min(T) (T)(is_signed_type(T) ? (T)1 << (8*sizeof(T)-1) : 0)
* #define type_max(T) (T)(is_signed_type(T) ? ((T)1 << (8*sizeof(T)-1)) - 1 : ~(T)0)
*
* Unfortunately, the middle expressions, strictly speaking, have
* undefined behaviour, and at least some versions of gcc warn about
* the type_max expression (but not if -fsanitize=undefined is in
* effect; in that case, the warning is deferred to runtime...).
*
* The slightly excessive casting in type_min is to make sure the
* macros also produce sensible values for the exotic type _Bool. [The
* overflow checkers only almost work for _Bool, but that's
* a-feature-not-a-bug, since people shouldn't be doing arithmetic on
* _Bools. Besides, the gcc builtins don't allow _Bool* as third
* argument.]
*
* Idea stolen from
* https://mail-index.netbsd.org/tech-misc/2007/02/05/0000.html -
* credit to Christian Biere.
*/
#define is_signed_type(type) (((type)(-1)) < (type)1)
#define __type_half_max(type) ((type)1 << (8*sizeof(type) - 1 - is_signed_type(type)))
#define type_max(T) ((T)((__type_half_max(T) - 1) + __type_half_max(T)))
#define type_min(T) ((T)((T)-type_max(T)-(T)1))
#ifdef COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW
/*
* For simplicity and code hygiene, the fallback code below insists on
* a, b and *d having the same type (similar to the min() and max()
* macros), whereas gcc's type-generic overflow checkers accept
* different types. Hence we don't just make check_add_overflow an
* alias for __builtin_add_overflow, but add type checks similar to
* below.
*/
#define check_add_overflow(a, b, d) ({ \
typeof(a) __a = (a); \
typeof(b) __b = (b); \
typeof(d) __d = (d); \
(void) (&__a == &__b); \
(void) (&__a == __d); \
__builtin_add_overflow(__a, __b, __d); \
})
#define check_sub_overflow(a, b, d) ({ \
typeof(a) __a = (a); \
typeof(b) __b = (b); \
typeof(d) __d = (d); \
(void) (&__a == &__b); \
(void) (&__a == __d); \
__builtin_sub_overflow(__a, __b, __d); \
})
#define check_mul_overflow(a, b, d) ({ \
typeof(a) __a = (a); \
typeof(b) __b = (b); \
typeof(d) __d = (d); \
(void) (&__a == &__b); \
(void) (&__a == __d); \
__builtin_mul_overflow(__a, __b, __d); \
})
#else
/* Checking for unsigned overflow is relatively easy without causing UB. */
#define __unsigned_add_overflow(a, b, d) ({ \
typeof(a) __a = (a); \
typeof(b) __b = (b); \
typeof(d) __d = (d); \
(void) (&__a == &__b); \
(void) (&__a == __d); \
*__d = __a + __b; \
*__d < __a; \
})
#define __unsigned_sub_overflow(a, b, d) ({ \
typeof(a) __a = (a); \
typeof(b) __b = (b); \
typeof(d) __d = (d); \
(void) (&__a == &__b); \
(void) (&__a == __d); \
*__d = __a - __b; \
__a < __b; \
})
/*
* If one of a or b is a compile-time constant, this avoids a division.
*/
#define __unsigned_mul_overflow(a, b, d) ({ \
typeof(a) __a = (a); \
typeof(b) __b = (b); \
typeof(d) __d = (d); \
(void) (&__a == &__b); \
(void) (&__a == __d); \
*__d = __a * __b; \
__builtin_constant_p(__b) ? \
__b > 0 && __a > type_max(typeof(__a)) / __b : \
__a > 0 && __b > type_max(typeof(__b)) / __a; \
})
/*
* For signed types, detecting overflow is much harder, especially if
* we want to avoid UB. But the interface of these macros is such that
* we must provide a result in *d, and in fact we must produce the
* result promised by gcc's builtins, which is simply the possibly
* wrapped-around value. Fortunately, we can just formally do the
* operations in the widest relevant unsigned type (u64) and then
* truncate the result - gcc is smart enough to generate the same code
* with and without the (u64) casts.
*/
/*
* Adding two signed integers can overflow only if they have the same
* sign, and overflow has happened iff the result has the opposite
* sign.
*/
#define __signed_add_overflow(a, b, d) ({ \
typeof(a) __a = (a); \
typeof(b) __b = (b); \
typeof(d) __d = (d); \
(void) (&__a == &__b); \
(void) (&__a == __d); \
*__d = (u64)__a + (u64)__b; \
(((~(__a ^ __b)) & (*__d ^ __a)) \
& type_min(typeof(__a))) != 0; \
})
/*
* Subtraction is similar, except that overflow can now happen only
* when the signs are opposite. In this case, overflow has happened if
* the result has the opposite sign of a.
*/
#define __signed_sub_overflow(a, b, d) ({ \
typeof(a) __a = (a); \
typeof(b) __b = (b); \
typeof(d) __d = (d); \
(void) (&__a == &__b); \
(void) (&__a == __d); \
*__d = (u64)__a - (u64)__b; \
((((__a ^ __b)) & (*__d ^ __a)) \
& type_min(typeof(__a))) != 0; \
})
/*
* Signed multiplication is rather hard. gcc always follows C99, so
* division is truncated towards 0. This means that we can write the
* overflow check like this:
*
* (a > 0 && (b > MAX/a || b < MIN/a)) ||
* (a < -1 && (b > MIN/a || b < MAX/a) ||
* (a == -1 && b == MIN)
*
* The redundant casts of -1 are to silence an annoying -Wtype-limits
* (included in -Wextra) warning: When the type is u8 or u16, the
* __b_c_e in check_mul_overflow obviously selects
* __unsigned_mul_overflow, but unfortunately gcc still parses this
* code and warns about the limited range of __b.
*/
#define __signed_mul_overflow(a, b, d) ({ \
typeof(a) __a = (a); \
typeof(b) __b = (b); \
typeof(d) __d = (d); \
typeof(a) __tmax = type_max(typeof(a)); \
typeof(a) __tmin = type_min(typeof(a)); \
(void) (&__a == &__b); \
(void) (&__a == __d); \
*__d = (u64)__a * (u64)__b; \
(__b > 0 && (__a > __tmax/__b || __a < __tmin/__b)) || \
(__b < (typeof(__b))-1 && (__a > __tmin/__b || __a < __tmax/__b)) || \
(__b == (typeof(__b))-1 && __a == __tmin); \
})
#define check_add_overflow(a, b, d) \
__builtin_choose_expr(is_signed_type(typeof(a)), \
__signed_add_overflow(a, b, d), \
__unsigned_add_overflow(a, b, d))
#define check_sub_overflow(a, b, d) \
__builtin_choose_expr(is_signed_type(typeof(a)), \
__signed_sub_overflow(a, b, d), \
__unsigned_sub_overflow(a, b, d))
#define check_mul_overflow(a, b, d) \
__builtin_choose_expr(is_signed_type(typeof(a)), \
__signed_mul_overflow(a, b, d), \
__unsigned_mul_overflow(a, b, d))
#endif /* COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW */
/**
* array_size() - Calculate size of 2-dimensional array.
*
* @a: dimension one
* @b: dimension two
*
* Calculates size of 2-dimensional array: @a * @b.
*
* Returns: number of bytes needed to represent the array or SIZE_MAX on
* overflow.
*/
static inline __must_check size_t array_size(size_t a, size_t b)
{
size_t bytes;
if (check_mul_overflow(a, b, &bytes))
return SIZE_MAX;
return bytes;
}
/**
* array3_size() - Calculate size of 3-dimensional array.
*
* @a: dimension one
* @b: dimension two
* @c: dimension three
*
* Calculates size of 3-dimensional array: @a * @b * @c.
*
* Returns: number of bytes needed to represent the array or SIZE_MAX on
* overflow.
*/
static inline __must_check size_t array3_size(size_t a, size_t b, size_t c)
{
size_t bytes;
if (check_mul_overflow(a, b, &bytes))
return SIZE_MAX;
if (check_mul_overflow(bytes, c, &bytes))
return SIZE_MAX;
return bytes;
}
static inline __must_check size_t __ab_c_size(size_t n, size_t size, size_t c)
{
size_t bytes;
if (check_mul_overflow(n, size, &bytes))
return SIZE_MAX;
if (check_add_overflow(bytes, c, &bytes))
return SIZE_MAX;
return bytes;
}
/**
* struct_size() - Calculate size of structure with trailing array.
* @p: Pointer to the structure.
* @member: Name of the array member.
* @n: Number of elements in the array.
*
* Calculates size of memory needed for structure @p followed by an
* array of @n @member elements.
*
* Return: number of bytes needed or SIZE_MAX on overflow.
*/
#define struct_size(p, member, n) \
__ab_c_size(n, \
sizeof(*(p)->member) + __must_be_array((p)->member),\
sizeof(*(p)))
#endif /* __LINUX_OVERFLOW_H */

12
include/linux/pagemap.h
View file

@ -359,8 +359,16 @@ unsigned find_get_pages(struct address_space *mapping, pgoff_t start,
unsigned int nr_pages, struct page **pages);
unsigned find_get_pages_contig(struct address_space *mapping, pgoff_t start,
unsigned int nr_pages, struct page **pages);
unsigned find_get_pages_tag(struct address_space *mapping, pgoff_t *index,
int tag, unsigned int nr_pages, struct page **pages);
unsigned find_get_pages_range_tag(struct address_space *mapping, pgoff_t *index,
pgoff_t end, int tag, unsigned int nr_pages,
struct page **pages);
static inline unsigned find_get_pages_tag(struct address_space *mapping,
pgoff_t *index, int tag, unsigned int nr_pages,
struct page **pages)
{
return find_get_pages_range_tag(mapping, index, (pgoff_t)-1, tag,
nr_pages, pages);
}
struct page *grab_cache_page_write_begin(struct address_space *mapping,
pgoff_t index, unsigned flags);

14
include/linux/pagevec.h
View file

@ -29,9 +29,17 @@ unsigned pagevec_lookup_entries(struct pagevec *pvec,
void pagevec_remove_exceptionals(struct pagevec *pvec);
unsigned pagevec_lookup(struct pagevec *pvec, struct address_space *mapping,
pgoff_t start, unsigned nr_pages);
unsigned pagevec_lookup_tag(struct pagevec *pvec,
struct address_space *mapping, pgoff_t *index, int tag,
unsigned nr_pages);
unsigned pagevec_lookup_range_tag(struct pagevec *pvec,
struct address_space *mapping, pgoff_t *index, pgoff_t end,
int tag);
unsigned pagevec_lookup_range_nr_tag(struct pagevec *pvec,
struct address_space *mapping, pgoff_t *index, pgoff_t end,
int tag, unsigned max_pages);
static inline unsigned pagevec_lookup_tag(struct pagevec *pvec,
struct address_space *mapping, pgoff_t *index, int tag)
{
return pagevec_lookup_range_tag(pvec, mapping, index, (pgoff_t)-1, tag);
}
static inline void pagevec_init(struct pagevec *pvec, int cold)
{

4
kernel/exit.c
View file

@ -54,7 +54,6 @@
#include <linux/writeback.h>
#include <linux/shm.h>
#include <linux/kcov.h>
#include <linux/cpufreq_times.h>
#include "sched/tune.h"
@ -174,9 +173,6 @@ void release_task(struct task_struct *p)
{
struct task_struct *leader;
int zap_leader;
#ifdef CONFIG_CPU_FREQ_TIMES
cpufreq_task_times_exit(p);
#endif
repeat:
/* don't need to get the RCU readlock here - the process is dead and
* can't be modifying its own credentials. But shut RCU-lockdep up */

6
kernel/fork.c
View file

@ -78,6 +78,7 @@
#include <linux/compiler.h>
#include <linux/sysctl.h>
#include <linux/kcov.h>
#include <linux/cpufreq_times.h>
#include <asm/pgtable.h>
#include <asm/pgalloc.h>
@ -228,6 +229,7 @@ static void account_kernel_stack(unsigned long *stack, int account)
void free_task(struct task_struct *tsk)
{
cpufreq_task_times_exit(tsk);
account_kernel_stack(tsk->stack, -1);
arch_release_thread_stack(tsk->stack);
free_thread_stack(tsk->stack);
@ -1366,6 +1368,8 @@ static struct task_struct *copy_process(unsigned long clone_flags,
if (!p)
goto fork_out;
cpufreq_task_times_init(p);
ftrace_graph_init_task(p);
rt_mutex_init_task(p);
@ -1798,6 +1802,8 @@ long _do_fork(unsigned long clone_flags,
struct completion vfork;
struct pid *pid;
cpufreq_task_times_alloc(p);
trace_sched_process_fork(current, p);
pid = get_task_pid(p, PIDTYPE_PID);

4
kernel/sched/core.c
View file

@ -2366,10 +2366,6 @@ static void __sched_fork(unsigned long clone_flags, struct task_struct *p)
memset(&p->se.statistics, 0, sizeof(p->se.statistics));
#endif
#ifdef CONFIG_CPU_FREQ_TIMES
cpufreq_task_times_init(p);
#endif
RB_CLEAR_NODE(&p->dl.rb_node);
init_dl_task_timer(&p->dl);
__dl_clear_params(p);

5
kernel/sched/cputime.c
View file

@ -162,10 +162,8 @@ void account_user_time(struct task_struct *p, cputime_t cputime,
/* Account for user time used */
acct_account_cputime(p);
#ifdef CONFIG_CPU_FREQ_TIMES
/* Account power usage for user time */
cpufreq_acct_update_power(p, cputime);
#endif
}
/*
@ -216,10 +214,9 @@ void __account_system_time(struct task_struct *p, cputime_t cputime,
/* Account for system time used */
acct_account_cputime(p);
#ifdef CONFIG_CPU_FREQ_TIMES
/* Account power usage for system time */
cpufreq_acct_update_power(p, cputime);
#endif
}
/*

47
mm/filemap.c
View file

@ -396,19 +396,17 @@ static int __filemap_fdatawait_range(struct address_space *mapping,
goto out;
pagevec_init(&pvec, 0);
while ((index <= end) &&
(nr_pages = pagevec_lookup_tag(&pvec, mapping, &index,
PAGECACHE_TAG_WRITEBACK,
min(end - index, (pgoff_t)PAGEVEC_SIZE-1) + 1)) != 0) {
while (index <= end) {
unsigned i;
nr_pages = pagevec_lookup_range_tag(&pvec, mapping, &index,
end, PAGECACHE_TAG_WRITEBACK);
if (!nr_pages)
break;
for (i = 0; i < nr_pages; i++) {
struct page *page = pvec.pages[i];
/* until radix tree lookup accepts end_index */
if (page->index > end)
continue;
wait_on_page_writeback(page);
if (TestClearPageError(page))
ret = -EIO;
@ -1426,9 +1424,10 @@ repeat:
EXPORT_SYMBOL(find_get_pages_contig);
/**
* find_get_pages_tag - find and return pages that match @tag
* find_get_pages_range_tag - find and return pages in given range matching @tag
* @mapping: the address_space to search
* @index: the starting page index
* @end: The final page index (inclusive)
* @tag: the tag index
* @nr_pages: the maximum number of pages
* @pages: where the resulting pages are placed
@ -1436,8 +1435,9 @@ EXPORT_SYMBOL(find_get_pages_contig);
* Like find_get_pages, except we only return pages which are tagged with
* @tag. We update @index to index the next page for the traversal.
*/
unsigned find_get_pages_tag(struct address_space *mapping, pgoff_t *index,
int tag, unsigned int nr_pages, struct page **pages)
unsigned find_get_pages_range_tag(struct address_space *mapping, pgoff_t *index,
pgoff_t end, int tag, unsigned int nr_pages,
struct page **pages)
{
struct radix_tree_iter iter;
void **slot;
@ -1451,6 +1451,9 @@ restart:
radix_tree_for_each_tagged(slot, &mapping->page_tree,
&iter, *index, tag) {
struct page *page;
if (iter.index > end)
break;
repeat:
page = radix_tree_deref_slot(slot);
if (unlikely(!page))
@ -1489,18 +1492,28 @@ repeat:
}
pages[ret] = page;
if (++ret == nr_pages)
break;
if (++ret == nr_pages) {
*index = pages[ret - 1]->index + 1;
goto out;
}
}
/*
* We come here when we got at @end. We take care to not overflow the
* index @index as it confuses some of the callers. This breaks the
* iteration when there is page at index -1 but that is already broken
* anyway.
*/
if (end == (pgoff_t)-1)
*index = (pgoff_t)-1;
else
*index = end + 1;
out:
rcu_read_unlock();
if (ret)
*index = pages[ret - 1]->index + 1;
return ret;
}
EXPORT_SYMBOL(find_get_pages_tag);
EXPORT_SYMBOL(find_get_pages_range_tag);
/*
* CD/DVDs are error prone. When a medium error occurs, the driver may fail

20
mm/page-writeback.c
View file

@ -2204,30 +2204,14 @@ retry:
while (!done && (index <= end)) {
int i;
nr_pages = pagevec_lookup_tag(&pvec, mapping, &index, tag,
min(end - index, (pgoff_t)PAGEVEC_SIZE-1) + 1);
nr_pages = pagevec_lookup_range_tag(&pvec, mapping, &index, end,
tag);
if (nr_pages == 0)
break;
for (i = 0; i < nr_pages; i++) {
struct page *page = pvec.pages[i];
/*
* At this point, the page may be truncated or
* invalidated (changing page->mapping to NULL), or
* even swizzled back from swapper_space to tmpfs file
* mapping. However, page->index will not change
* because we have a reference on the page.
*/
if (page->index > end) {
/*
* can't be range_cyclic (1st pass) because
* end == -1 in that case.
*/
done = 1;
break;
}
done_index = page->index;
lock_page(page);

20
mm/swap.c
View file

@ -1130,15 +1130,25 @@ unsigned pagevec_lookup(struct pagevec *pvec, struct address_space *mapping,
}
EXPORT_SYMBOL(pagevec_lookup);
unsigned pagevec_lookup_tag(struct pagevec *pvec, struct address_space *mapping,
pgoff_t *index, int tag, unsigned nr_pages)
unsigned pagevec_lookup_range_tag(struct pagevec *pvec,
struct address_space *mapping, pgoff_t *index, pgoff_t end,
int tag)
{
pvec->nr = find_get_pages_tag(mapping, index, tag,
nr_pages, pvec->pages);
pvec->nr = find_get_pages_range_tag(mapping, index, end, tag,
PAGEVEC_SIZE, pvec->pages);
return pagevec_count(pvec);
}
EXPORT_SYMBOL(pagevec_lookup_tag);
EXPORT_SYMBOL(pagevec_lookup_range_tag);
unsigned pagevec_lookup_range_nr_tag(struct pagevec *pvec,
struct address_space *mapping, pgoff_t *index, pgoff_t end,
int tag, unsigned max_pages)
{
pvec->nr = find_get_pages_range_tag(mapping, index, end, tag,
min_t(unsigned int, max_pages, PAGEVEC_SIZE), pvec->pages);
return pagevec_count(pvec);
}
EXPORT_SYMBOL(pagevec_lookup_range_nr_tag);
/*
* Perform any setup for the swap system
*/

15
net/ipv4/tcp_input.c
View file

@ -4780,6 +4780,7 @@ restart:
static void tcp_collapse_ofo_queue(struct sock *sk)
{
struct tcp_sock *tp = tcp_sk(sk);
u32 range_truesize, sum_tiny = 0;
struct sk_buff *skb = skb_peek(&tp->out_of_order_queue);
struct sk_buff *head;
u32 start, end;
@ -4789,6 +4790,7 @@ static void tcp_collapse_ofo_queue(struct sock *sk)
start = TCP_SKB_CB(skb)->seq;
end = TCP_SKB_CB(skb)->end_seq;
range_truesize = skb->truesize;
head = skb;
for (;;) {
@ -4803,14 +4805,24 @@ static void tcp_collapse_ofo_queue(struct sock *sk)
if (!skb ||
after(TCP_SKB_CB(skb)->seq, end) ||
before(TCP_SKB_CB(skb)->end_seq, start)) {
/* Do not attempt collapsing tiny skbs */
if (range_truesize != head->truesize ||
end - start >= SKB_WITH_OVERHEAD(SK_MEM_QUANTUM)) {
tcp_collapse(sk, &tp->out_of_order_queue,
head, skb, start, end);
} else {
sum_tiny += range_truesize;
if (sum_tiny > sk->sk_rcvbuf >> 3)
return;
}
head = skb;
if (!skb)
break;
/* Start new segment */
start = TCP_SKB_CB(skb)->seq;
end = TCP_SKB_CB(skb)->end_seq;
range_truesize = skb->truesize;
} else {
if (before(TCP_SKB_CB(skb)->seq, start))
start = TCP_SKB_CB(skb)->seq;
@ -4866,6 +4878,9 @@ static int tcp_prune_queue(struct sock *sk)
else if (tcp_under_memory_pressure(sk))
tp->rcv_ssthresh = min(tp->rcv_ssthresh, 4U * tp->advmss);
if (atomic_read(&sk->sk_rmem_alloc) <= sk->sk_rcvbuf)
return 0;
tcp_collapse_ofo_queue(sk);
if (!skb_queue_empty(&sk->sk_receive_queue))
tcp_collapse(sk, &sk->sk_receive_queue,

5
scripts/Kbuild.include
View file

@ -7,6 +7,7 @@ quote := "
squote := '
empty :=
space := $(empty) $(empty)
pound := \#
###
# Name of target with a '.' as filename prefix. foo/bar.o => foo/.bar.o
@ -250,11 +251,11 @@ endif
# Replace >$< with >$$< to preserve $ when reloading the .cmd file
# (needed for make)
# Replace >#< with >\#< to avoid starting a comment in the .cmd file
# Replace >#< with >$(pound)< to avoid starting a comment in the .cmd file
# (needed for make)
# Replace >'< with >'\''< to be able to enclose the whole string in '...'
# (needed for the shell)
make-cmd = $(call escsq,$(subst \#,\\\#,$(subst $$,$$$$,$(cmd_$(1)))))
make-cmd = $(call escsq,$(subst $(pound),$$(pound),$(subst $$,$$$$,$(cmd_$(1)))))
# Find any prerequisites that is newer than target or that does not exist.
# PHONY targets skipped in both cases.

222
scripts/tags.sh
View file

@ -1,4 +1,4 @@
#!/bin/sh
#!/bin/bash
# Generate tags or cscope files
# Usage tags.sh <mode>
#
@ -135,11 +135,6 @@ all_kconfigs()
find_other_sources 'Kconfig*'
}
all_defconfigs()
{
find_sources $ALLSOURCE_ARCHS "defconfig"
}
docscope()
{
(echo \-k; echo \-q; all_target_sources) > cscope.files
@ -151,8 +146,111 @@ dogtags()
all_target_sources | gtags -i -f -
}
# Basic regular expressions with an optional /kind-spec/ for ctags and
# the following limitations:
# - No regex modifiers
# - Use \{0,1\} instead of \?, because etags expects an unescaped ?
# - \s is not working with etags, use a space or [ \t]
# - \w works, but does not match underscores in etags
# - etags regular expressions have to match at the start of a line;
# a ^[^#] is prepended by setup_regex unless an anchor is already present
regex_asm=(
'/^\(ENTRY\|_GLOBAL\)(\([[:alnum:]_\\]*\)).*/\2/'
)
regex_c=(
'/^SYSCALL_DEFINE[0-9](\([[:alnum:]_]*\).*/sys_\1/'
'/^COMPAT_SYSCALL_DEFINE[0-9](\([[:alnum:]_]*\).*/compat_sys_\1/'
'/^TRACE_EVENT(\([[:alnum:]_]*\).*/trace_\1/'
'/^TRACE_EVENT(\([[:alnum:]_]*\).*/trace_\1_rcuidle/'
'/^DEFINE_EVENT([^,)]*, *\([[:alnum:]_]*\).*/trace_\1/'
'/^DEFINE_EVENT([^,)]*, *\([[:alnum:]_]*\).*/trace_\1_rcuidle/'
'/^DEFINE_INSN_CACHE_OPS(\([[:alnum:]_]*\).*/get_\1_slot/'
'/^DEFINE_INSN_CACHE_OPS(\([[:alnum:]_]*\).*/free_\1_slot/'
'/^PAGEFLAG(\([[:alnum:]_]*\).*/Page\1/'
'/^PAGEFLAG(\([[:alnum:]_]*\).*/SetPage\1/'
'/^PAGEFLAG(\([[:alnum:]_]*\).*/ClearPage\1/'
'/^TESTSETFLAG(\([[:alnum:]_]*\).*/TestSetPage\1/'
'/^TESTPAGEFLAG(\([[:alnum:]_]*\).*/Page\1/'
'/^SETPAGEFLAG(\([[:alnum:]_]*\).*/SetPage\1/'
'/\<__SETPAGEFLAG(\([[:alnum:]_]*\).*/__SetPage\1/'
'/\<TESTCLEARFLAG(\([[:alnum:]_]*\).*/TestClearPage\1/'
'/\<__TESTCLEARFLAG(\([[:alnum:]_]*\).*/TestClearPage\1/'
'/\<CLEARPAGEFLAG(\([[:alnum:]_]*\).*/ClearPage\1/'
'/\<__CLEARPAGEFLAG(\([[:alnum:]_]*\).*/__ClearPage\1/'
'/^__PAGEFLAG(\([[:alnum:]_]*\).*/__SetPage\1/'
'/^__PAGEFLAG(\([[:alnum:]_]*\).*/__ClearPage\1/'
'/^PAGEFLAG_FALSE(\([[:alnum:]_]*\).*/Page\1/'
'/\<TESTSCFLAG(\([[:alnum:]_]*\).*/TestSetPage\1/'
'/\<TESTSCFLAG(\([[:alnum:]_]*\).*/TestClearPage\1/'
'/\<SETPAGEFLAG_NOOP(\([[:alnum:]_]*\).*/SetPage\1/'
'/\<CLEARPAGEFLAG_NOOP(\([[:alnum:]_]*\).*/ClearPage\1/'
'/\<__CLEARPAGEFLAG_NOOP(\([[:alnum:]_]*\).*/__ClearPage\1/'
'/\<TESTCLEARFLAG_FALSE(\([[:alnum:]_]*\).*/TestClearPage\1/'
'/^PAGE_MAPCOUNT_OPS(\([[:alnum:]_]*\).*/Page\1/'
'/^PAGE_MAPCOUNT_OPS(\([[:alnum:]_]*\).*/__SetPage\1/'
'/^PAGE_MAPCOUNT_OPS(\([[:alnum:]_]*\).*/__ClearPage\1/'
'/^TASK_PFA_TEST([^,]*, *\([[:alnum:]_]*\))/task_\1/'
'/^TASK_PFA_SET([^,]*, *\([[:alnum:]_]*\))/task_set_\1/'
'/^TASK_PFA_CLEAR([^,]*, *\([[:alnum:]_]*\))/task_clear_\1/'
'/^DEF_MMIO_\(IN\|OUT\)_[XD](\([[:alnum:]_]*\),[^)]*)/\2/'
'/^DEBUGGER_BOILERPLATE(\([[:alnum:]_]*\))/\1/'
'/^DEF_PCI_AC_\(\|NO\)RET(\([[:alnum:]_]*\).*/\2/'
'/^PCI_OP_READ(\(\w*\).*[1-4])/pci_bus_read_config_\1/'
'/^PCI_OP_WRITE(\(\w*\).*[1-4])/pci_bus_write_config_\1/'
'/\<DEFINE_\(MUTEX\|SEMAPHORE\|SPINLOCK\)(\([[:alnum:]_]*\)/\2/v/'
'/\<DEFINE_\(RAW_SPINLOCK\|RWLOCK\|SEQLOCK\)(\([[:alnum:]_]*\)/\2/v/'
'/\<DECLARE_\(RWSEM\|COMPLETION\)(\([[:alnum:]_]\+\)/\2/v/'
'/\<DECLARE_BITMAP(\([[:alnum:]_]*\)/\1/v/'
'/\(^\|\s\)\(\|L\|H\)LIST_HEAD(\([[:alnum:]_]*\)/\3/v/'
'/\(^\|\s\)RADIX_TREE(\([[:alnum:]_]*\)/\2/v/'
'/\<DEFINE_PER_CPU([^,]*, *\([[:alnum:]_]*\)/\1/v/'
'/\<DEFINE_PER_CPU_SHARED_ALIGNED([^,]*, *\([[:alnum:]_]*\)/\1/v/'
'/\<DECLARE_WAIT_QUEUE_HEAD(\([[:alnum:]_]*\)/\1/v/'
'/\<DECLARE_\(TASKLET\|WORK\|DELAYED_WORK\)(\([[:alnum:]_]*\)/\2/v/'
'/\(^\s\)OFFSET(\([[:alnum:]_]*\)/\2/v/'
'/\(^\s\)DEFINE(\([[:alnum:]_]*\)/\2/v/'
'/\<DEFINE_HASHTABLE(\([[:alnum:]_]*\)/\1/v/'
)
regex_kconfig=(
'/^[[:blank:]]*\(menu\|\)config[[:blank:]]\+\([[:alnum:]_]\+\)/\2/'
'/^[[:blank:]]*\(menu\|\)config[[:blank:]]\+\([[:alnum:]_]\+\)/CONFIG_\2/'
)
setup_regex()
{
local mode=$1 lang tmp=() r
shift
regex=()
for lang; do
case "$lang" in
asm) tmp=("${regex_asm[@]}") ;;
c) tmp=("${regex_c[@]}") ;;
kconfig) tmp=("${regex_kconfig[@]}") ;;
esac
for r in "${tmp[@]}"; do
if test "$mode" = "exuberant"; then
regex[${#regex[@]}]="--regex-$lang=${r}b"
else
# Remove ctags /kind-spec/
case "$r" in
/*/*/?/)
r=${r%?/}
esac
# Prepend ^[^#] unless already anchored
case "$r" in
/^*) ;;
*)
r="/^[^#]*${r#/}"
esac
regex[${#regex[@]}]="--regex=$r"
fi
done
done
}
exuberant()
{
setup_regex exuberant asm c
all_target_sources | xargs $1 -a \
-I __initdata,__exitdata,__initconst, \
-I __initdata_memblock \
@ -166,118 +264,22 @@ exuberant()
-I EXPORT_SYMBOL,EXPORT_SYMBOL_GPL,ACPI_EXPORT_SYMBOL \
-I DEFINE_TRACE,EXPORT_TRACEPOINT_SYMBOL,EXPORT_TRACEPOINT_SYMBOL_GPL \
-I static,const \
--extra=+f --c-kinds=+px \
--regex-asm='/^(ENTRY|_GLOBAL)\(([^)]*)\).*/\2/' \
--regex-c='/^SYSCALL_DEFINE[[:digit:]]?\(([^,)]*).*/sys_\1/' \
--regex-c='/^COMPAT_SYSCALL_DEFINE[[:digit:]]?\(([^,)]*).*/compat_sys_\1/' \
--regex-c++='/^TRACE_EVENT\(([^,)]*).*/trace_\1/' \
--regex-c++='/^TRACE_EVENT\(([^,)]*).*/trace_\1_rcuidle/' \
--regex-c++='/^DEFINE_EVENT\([^,)]*, *([^,)]*).*/trace_\1/' \
--regex-c++='/^DEFINE_EVENT\([^,)]*, *([^,)]*).*/trace_\1_rcuidle/' \
--regex-c++='/PAGEFLAG\(([^,)]*).*/Page\1/' \
--regex-c++='/PAGEFLAG\(([^,)]*).*/SetPage\1/' \
--regex-c++='/PAGEFLAG\(([^,)]*).*/ClearPage\1/' \
--regex-c++='/TESTSETFLAG\(([^,)]*).*/TestSetPage\1/' \
--regex-c++='/TESTPAGEFLAG\(([^,)]*).*/Page\1/' \
--regex-c++='/SETPAGEFLAG\(([^,)]*).*/SetPage\1/' \
--regex-c++='/__SETPAGEFLAG\(([^,)]*).*/__SetPage\1/' \
--regex-c++='/TESTCLEARFLAG\(([^,)]*).*/TestClearPage\1/' \
--regex-c++='/__TESTCLEARFLAG\(([^,)]*).*/TestClearPage\1/' \
--regex-c++='/CLEARPAGEFLAG\(([^,)]*).*/ClearPage\1/' \
--regex-c++='/__CLEARPAGEFLAG\(([^,)]*).*/__ClearPage\1/' \
--regex-c++='/__PAGEFLAG\(([^,)]*).*/__SetPage\1/' \
--regex-c++='/__PAGEFLAG\(([^,)]*).*/__ClearPage\1/' \
--regex-c++='/PAGEFLAG_FALSE\(([^,)]*).*/Page\1/' \
--regex-c++='/TESTSCFLAG\(([^,)]*).*/TestSetPage\1/' \
--regex-c++='/TESTSCFLAG\(([^,)]*).*/TestClearPage\1/' \
--regex-c++='/SETPAGEFLAG_NOOP\(([^,)]*).*/SetPage\1/' \
--regex-c++='/CLEARPAGEFLAG_NOOP\(([^,)]*).*/ClearPage\1/' \
--regex-c++='/__CLEARPAGEFLAG_NOOP\(([^,)]*).*/__ClearPage\1/' \
--regex-c++='/TESTCLEARFLAG_FALSE\(([^,)]*).*/TestClearPage\1/' \
--regex-c++='/__TESTCLEARFLAG_FALSE\(([^,)]*).*/__TestClearPage\1/' \
--regex-c++='/_PE\(([^,)]*).*/PEVENT_ERRNO__\1/' \
--regex-c++='/TASK_PFA_TEST\([^,]*,\s*([^)]*)\)/task_\1/' \
--regex-c++='/TASK_PFA_SET\([^,]*,\s*([^)]*)\)/task_set_\1/' \
--regex-c++='/TASK_PFA_CLEAR\([^,]*,\s*([^)]*)\)/task_clear_\1/'\
--regex-c++='/DEF_MMIO_(IN|OUT)_(X|D)\(([^,]*),\s*[^)]*\)/\3/' \
--regex-c++='/DEBUGGER_BOILERPLATE\(([^,]*)\)/\1/' \
--regex-c='/PCI_OP_READ\((\w*).*[1-4]\)/pci_bus_read_config_\1/' \
--regex-c='/PCI_OP_WRITE\((\w*).*[1-4]\)/pci_bus_write_config_\1/' \
--regex-c='/DEFINE_(MUTEX|SEMAPHORE|SPINLOCK)\((\w*)/\2/v/' \
--regex-c='/DEFINE_(RAW_SPINLOCK|RWLOCK|SEQLOCK)\((\w*)/\2/v/' \
--regex-c='/DECLARE_(RWSEM|COMPLETION)\((\w*)/\2/v/' \
--regex-c='/DECLARE_BITMAP\((\w*)/\1/v/' \
--regex-c='/(^|\s)(|L|H)LIST_HEAD\((\w*)/\3/v/' \
--regex-c='/(^|\s)RADIX_TREE\((\w*)/\2/v/' \
--regex-c='/DEFINE_PER_CPU\(([^,]*,\s*)(\w*).*\)/\2/v/' \
--regex-c='/DEFINE_PER_CPU_SHARED_ALIGNED\(([^,]*,\s*)(\w*).*\)/\2/v/' \
--regex-c='/DECLARE_WAIT_QUEUE_HEAD\((\w*)/\1/v/' \
--regex-c='/DECLARE_(TASKLET|WORK|DELAYED_WORK)\((\w*)/\2/v/' \
--regex-c='/DEFINE_PCI_DEVICE_TABLE\((\w*)/\1/v/' \
--regex-c='/(^\s)OFFSET\((\w*)/\2/v/' \
--regex-c='/(^\s)DEFINE\((\w*)/\2/v/' \
--regex-c='/DEFINE_HASHTABLE\((\w*)/\1/v/'
--extra=+fq --c-kinds=+px --fields=+iaS --langmap=c:+.h \
"${regex[@]}"
setup_regex exuberant kconfig
all_kconfigs | xargs $1 -a \
--langdef=kconfig --language-force=kconfig \
--regex-kconfig='/^[[:blank:]]*(menu|)config[[:blank:]]+([[:alnum:]_]+)/\2/'
--langdef=kconfig --language-force=kconfig "${regex[@]}"
all_kconfigs | xargs $1 -a \
--langdef=kconfig --language-force=kconfig \
--regex-kconfig='/^[[:blank:]]*(menu|)config[[:blank:]]+([[:alnum:]_]+)/CONFIG_\2/'
all_defconfigs | xargs -r $1 -a \
--langdef=dotconfig --language-force=dotconfig \
--regex-dotconfig='/^#?[[:blank:]]*(CONFIG_[[:alnum:]_]+)/\1/'
}
emacs()
{
all_target_sources | xargs $1 -a \
--regex='/^\(ENTRY\|_GLOBAL\)(\([^)]*\)).*/\2/' \
--regex='/^SYSCALL_DEFINE[0-9]?(\([^,)]*\).*/sys_\1/' \
--regex='/^COMPAT_SYSCALL_DEFINE[0-9]?(\([^,)]*\).*/compat_sys_\1/' \
--regex='/^TRACE_EVENT(\([^,)]*\).*/trace_\1/' \
--regex='/^TRACE_EVENT(\([^,)]*\).*/trace_\1_rcuidle/' \
--regex='/^DEFINE_EVENT([^,)]*, *\([^,)]*\).*/trace_\1/' \
--regex='/^DEFINE_EVENT([^,)]*, *\([^,)]*\).*/trace_\1_rcuidle/' \
--regex='/PAGEFLAG(\([^,)]*\).*/Page\1/' \
--regex='/PAGEFLAG(\([^,)]*\).*/SetPage\1/' \
--regex='/PAGEFLAG(\([^,)]*\).*/ClearPage\1/' \
--regex='/TESTSETFLAG(\([^,)]*\).*/TestSetPage\1/' \
--regex='/TESTPAGEFLAG(\([^,)]*\).*/Page\1/' \
--regex='/SETPAGEFLAG(\([^,)]*\).*/SetPage\1/' \
--regex='/__SETPAGEFLAG(\([^,)]*\).*/__SetPage\1/' \
--regex='/TESTCLEARFLAG(\([^,)]*\).*/TestClearPage\1/' \
--regex='/__TESTCLEARFLAG(\([^,)]*\).*/TestClearPage\1/' \
--regex='/CLEARPAGEFLAG(\([^,)]*\).*/ClearPage\1/' \
--regex='/__CLEARPAGEFLAG(\([^,)]*\).*/__ClearPage\1/' \
--regex='/__PAGEFLAG(\([^,)]*\).*/__SetPage\1/' \
--regex='/__PAGEFLAG(\([^,)]*\).*/__ClearPage\1/' \
--regex='/PAGEFLAG_FALSE(\([^,)]*\).*/Page\1/' \
--regex='/TESTSCFLAG(\([^,)]*\).*/TestSetPage\1/' \
--regex='/TESTSCFLAG(\([^,)]*\).*/TestClearPage\1/' \
--regex='/SETPAGEFLAG_NOOP(\([^,)]*\).*/SetPage\1/' \
--regex='/CLEARPAGEFLAG_NOOP(\([^,)]*\).*/ClearPage\1/' \
--regex='/__CLEARPAGEFLAG_NOOP(\([^,)]*\).*/__ClearPage\1/' \
--regex='/TESTCLEARFLAG_FALSE(\([^,)]*\).*/TestClearPage\1/' \
--regex='/__TESTCLEARFLAG_FALSE(\([^,)]*\).*/__TestClearPage\1/' \
--regex='/TASK_PFA_TEST\([^,]*,\s*([^)]*)\)/task_\1/' \
--regex='/TASK_PFA_SET\([^,]*,\s*([^)]*)\)/task_set_\1/' \
--regex='/TASK_PFA_CLEAR\([^,]*,\s*([^)]*)\)/task_clear_\1/' \
--regex='/_PE(\([^,)]*\).*/PEVENT_ERRNO__\1/' \
--regex='/PCI_OP_READ(\([a-z]*[a-z]\).*[1-4])/pci_bus_read_config_\1/' \
--regex='/PCI_OP_WRITE(\([a-z]*[a-z]\).*[1-4])/pci_bus_write_config_\1/'\
--regex='/[^#]*DEFINE_HASHTABLE(\([^,)]*\)/\1/'
setup_regex emacs asm c
all_target_sources | xargs $1 -a "${regex[@]}"
all_kconfigs | xargs $1 -a \
--regex='/^[ \t]*\(\(menu\)*config\)[ \t]+\([a-zA-Z0-9_]+\)/\3/'
all_kconfigs | xargs $1 -a \
--regex='/^[ \t]*\(\(menu\)*config\)[ \t]+\([a-zA-Z0-9_]+\)/CONFIG_\3/'
all_defconfigs | xargs -r $1 -a \
--regex='/^#?[ \t]?\(CONFIG_[a-zA-Z0-9_]+\)/\1/'
setup_regex emacs kconfig
all_kconfigs | xargs $1 -a "${regex[@]}"
}
xtags()

9
tools/arch/x86/include/asm/unistd_32.h Normal file
View file

@ -0,0 +1,9 @@
#ifndef __NR_perf_event_open
# define __NR_perf_event_open 336
#endif
#ifndef __NR_futex
# define __NR_futex 240
#endif
#ifndef __NR_gettid
# define __NR_gettid 224
#endif

9
tools/arch/x86/include/asm/unistd_64.h Normal file
View file

@ -0,0 +1,9 @@
#ifndef __NR_perf_event_open
# define __NR_perf_event_open 298
#endif
#ifndef __NR_futex
# define __NR_futex 202
#endif
#ifndef __NR_gettid
# define __NR_gettid 186
#endif

5
tools/build/Build.include
View file

@ -12,6 +12,7 @@
# Convenient variables
comma := ,
squote := '
pound := \#
###
# Name of target with a '.' as filename prefix. foo/bar.o => foo/.bar.o
@ -43,11 +44,11 @@ echo-cmd = $(if $($(quiet)cmd_$(1)),\
###
# Replace >$< with >$$< to preserve $ when reloading the .cmd file
# (needed for make)
# Replace >#< with >\#< to avoid starting a comment in the .cmd file
# Replace >#< with >$(pound)< to avoid starting a comment in the .cmd file
# (needed for make)
# Replace >'< with >'\''< to be able to enclose the whole string in '...'
# (needed for the shell)
make-cmd = $(call escsq,$(subst \#,\\\#,$(subst $$,$$$$,$(cmd_$(1)))))
make-cmd = $(call escsq,$(subst $(pound),$$(pound),$(subst $$,$$$$,$(cmd_$(1)))))
###
# Find any prerequisites that is newer than target or that does not exist.

1
tools/perf/config/Makefile
View file

@ -200,6 +200,7 @@ CFLAGS += -I$(src-perf)/arch/$(ARCH)/include
CFLAGS += -I$(srctree)/tools/include/
CFLAGS += -I$(srctree)/arch/$(ARCH)/include/uapi
CFLAGS += -I$(srctree)/arch/$(ARCH)/include
CFLAGS += -I$(srctree)/tools/arch/$(ARCH)/include
CFLAGS += -I$(srctree)/include/uapi
CFLAGS += -I$(srctree)/include

18
tools/perf/perf-sys.h
View file

@ -11,29 +11,11 @@
#if defined(__i386__)
#define cpu_relax() asm volatile("rep; nop" ::: "memory");
#define CPUINFO_PROC {"model name"}
#ifndef __NR_perf_event_open
# define __NR_perf_event_open 336
#endif
#ifndef __NR_futex
# define __NR_futex 240
#endif
#ifndef __NR_gettid
# define __NR_gettid 224
#endif
#endif
#if defined(__x86_64__)
#define cpu_relax() asm volatile("rep; nop" ::: "memory");
#define CPUINFO_PROC {"model name"}
#ifndef __NR_perf_event_open
# define __NR_perf_event_open 298
#endif
#ifndef __NR_futex
# define __NR_futex 202
#endif
#ifndef __NR_gettid
# define __NR_gettid 186
#endif
#endif
#ifdef __powerpc__

1
tools/perf/util/include/asm/unistd_32.h
View file

@ -1 +0,0 @@

1
tools/perf/util/include/asm/unistd_64.h
View file

@ -1 +0,0 @@

2
tools/scripts/Makefile.include
View file

@ -92,3 +92,5 @@ ifneq ($(silent),1)
QUIET_INSTALL = @printf ' INSTALL %s\n' $1;
endif
endif
pound := \#

24
verity_dev_keys.x509 Normal file
View file

@ -0,0 +1,24 @@
-----BEGIN CERTIFICATE-----
MIID/TCCAuWgAwIBAgIJAJcPmDkJqolJMA0GCSqGSIb3DQEBBQUAMIGUMQswCQYD
VQQGEwJVUzETMBEGA1UECAwKQ2FsaWZvcm5pYTEWMBQGA1UEBwwNTW91bnRhaW4g
VmlldzEQMA4GA1UECgwHQW5kcm9pZDEQMA4GA1UECwwHQW5kcm9pZDEQMA4GA1UE
AwwHQW5kcm9pZDEiMCAGCSqGSIb3DQEJARYTYW5kcm9pZEBhbmRyb2lkLmNvbTAe
Fw0xNDExMDYxOTA3NDBaFw00MjAzMjQxOTA3NDBaMIGUMQswCQYDVQQGEwJVUzET
MBEGA1UECAwKQ2FsaWZvcm5pYTEWMBQGA1UEBwwNTW91bnRhaW4gVmlldzEQMA4G
A1UECgwHQW5kcm9pZDEQMA4GA1UECwwHQW5kcm9pZDEQMA4GA1UEAwwHQW5kcm9p
ZDEiMCAGCSqGSIb3DQEJARYTYW5kcm9pZEBhbmRyb2lkLmNvbTCCASIwDQYJKoZI
hvcNAQEBBQADggEPADCCAQoCggEBAOjreE0vTVSRenuzO9vnaWfk0eQzYab0gqpi
6xAzi6dmD+ugoEKJmbPiuE5Dwf21isZ9uhUUu0dQM46dK4ocKxMRrcnmGxydFn6o
fs3ODJMXOkv2gKXL/FdbEPdDbxzdu8z3yk+W67udM/fW7WbaQ3DO0knu+izKak/3
T41c5uoXmQ81UNtAzRGzGchNVXMmWuTGOkg6U+0I2Td7K8yvUMWhAWPPpKLtVH9r
AL5TzjYNR92izdKcz3AjRsI3CTjtpiVABGeX0TcjRSuZB7K9EK56HV+OFNS6I1NP
jdD7FIShyGlqqZdUOkAUZYanbpgeT5N7QL6uuqcGpoTOkalu6kkCAwEAAaNQME4w
HQYDVR0OBBYEFH5DM/m7oArf4O3peeKO0ZIEkrQPMB8GA1UdIwQYMBaAFH5DM/m7
oArf4O3peeKO0ZIEkrQPMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADggEB
AHO3NSvDE5jFvMehGGtS8BnFYdFKRIglDMc4niWSzhzOVYRH4WajxdtBWc5fx0ix
NF/+hVKVhP6AIOQa+++sk+HIi7RvioPPbhjcsVlZe7cUEGrLSSveGouQyc+j0+m6
JF84kszIl5GGNMTnx0XRPO+g8t6h5LWfnVydgZfpGRRg+WHewk1U2HlvTjIceb0N
dcoJ8WKJAFWdcuE7VIm4w+vF/DYX/A2Oyzr2+QRhmYSv1cusgAeC1tvH4ap+J1Lg
UnOu5Kh/FqPLLSwNVQp4Bu7b9QFfqK8Moj84bj88NqRGZgDyqzuTrFxn6FW7dmyA
yttuAJAEAymk1mipd9+zp38=
-----END CERTIFICATE-----