Re: [GIT PULL] Btrfs updates for 5.12-rc2
The pull request you sent on Mon, 1 Mar 2021 14:44:16 +0100: > git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-5.12-rc1-tag has been merged into torvalds/linux.git: https://git.kernel.org/torvalds/c/c608aca57dd034d09f307b109b670d1cfb829279 Thank you! -- Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/prtracker.html
[GIT PULL] Btrfs updates for 5.12-rc2
From: David Sterba Hi, first batch of fixes that usually arrive during the merge window code freeze. Regressions and stable material. Please pull, thanks. Regressions: - fix deadlock in log sync in zoned mode - fix bugs in subpage mode still wrongly assuming sectorsize == page size Fixes: - fix missing kunmap of the Q stripe in RAID6 - block group fixes: - fix race between extent freeing/allocation when using bitmaps - avoid double put of block group when emptying cluster - swapfile fixes: - fix swapfile writes vs running scrub - fix swapfile activation vs snapshot creation - fix stale data exposure after cloning a hole with NO_HOLES enabled - remove tree-checker check that does not work in case information from other leaves is necessary The following changes since commit 9d294a685fbcb256ce8c5f7fd88a7596d0f52a8a: btrfs: zoned: enable to mount ZONED incompat flag (2021-02-09 02:52:24 +0100) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-5.12-rc1-tag for you to fetch changes up to 6e37d245994189ba757df7dc2950a44d31421ac6: btrfs: zoned: fix deadlock on log sync (2021-02-22 18:08:48 +0100) Filipe Manana (4): btrfs: avoid checking for RO block group twice during nocow writeback btrfs: fix race between writes to swap files and scrub btrfs: fix race between swap file activation and snapshot creation btrfs: fix stale data exposure after cloning a hole with NO_HOLES enabled Ira Weiny (1): btrfs: fix raid6 qstripe kmap Johannes Thumshirn (1): btrfs: zoned: fix deadlock on log sync Josef Bacik (2): btrfs: tree-checker: do not error out if extent ref hash doesn't match btrfs: avoid double put of block group when emptying cluster Nikolay Borisov (1): btrfs: fix race between extent freeing/allocation when using bitmaps Qu Wenruo (2): btrfs: make btrfs_submit_compressed_read() subpage compatible btrfs: make check_compressed_csum() to be subpage compatible fs/btrfs/block-group.c | 33 +++- fs/btrfs/block-group.h | 9 +++ fs/btrfs/compression.c | 62 +++-- fs/btrfs/ctree.h| 5 fs/btrfs/free-space-cache.c | 14 +- fs/btrfs/inode.c| 44 +++- fs/btrfs/raid56.c | 21 --- fs/btrfs/reflink.c | 18 + fs/btrfs/scrub.c| 9 ++- fs/btrfs/tree-checker.c | 16 +++- fs/btrfs/tree-log.c | 3 --- 11 files changed, 175 insertions(+), 59 deletions(-)
Re: [GIT PULL] Btrfs updates for 5.12
The pull request you sent on Tue, 16 Feb 2021 13:56:46 +0100: > git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-5.12-tag has been merged into torvalds/linux.git: https://git.kernel.org/torvalds/c/6f3952cbe00b74739f540981d1afe84cd4dac879 Thank you! -- Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/prtracker.html
[GIT PULL] Btrfs updates for 5.12
Hi, this update brings updates of space handling, performance improvements or bug fixes. The subpage block size and zoned mode features have reached state where they're usable but with limitations. The branch merges cleanly on top of current master, there are some minor conflicts reported by linux-next in iov_iter or kmap conversion. Please pull, thanks. - Performance or related - do not block on deleted block group mutex in the cleaner, avoids some long stalls - improved flushing: make it work better with ticket space reservations and avoid excessive transaction commits in some scenarios, slightly improves throughput for random write load - preemptive background flushing: separate the logic from ticket reservations, improve the accounting and decisions when to flush in low space conditions - less lock contention related to running delayed refs, let just one thread do the flushing when there are many inside transaction commit - dbench workload improvements: avoid unnecessary work when logging inodes, fewer fallbacks to transaction commit and thus less waiting for it (+7% throughput, -20% latency) - Core - subpage block size - currently read-only support - refactor and generalize code where sectorsize is assumed to be page size, add the subpage handling everywhere - the read-write support is on the way, page sizes are still limited to 4K or 64K - zoned mode, first working version but with limitations - SMR/ZBC/ZNS friendly allocation mode, utilizing the "no fixed location for structures" and chunked allocation - superblock as the only fixed data structure needs special handling, uses 2 consecutive zones as a ring buffer - tree-log support with a dedicated block group to avoid unordered writes - emulated zones on non-zoned devices - not yet working - all non-single block group profiles, requires more zone write pointer synchronization between the multiple block groups - fitrim due to dependency on space cache, can be implemented - Fixes - ref-verify: proper tree owner and node level tracking - fix pinned byte accounting, causing some early ENOSPC now more likely due to other changes in delayed refs - Other - error handling fixes and improvements - more error injection points - more function documentation - more and updated tracepoints - subset of W=1 checked by default - update comments to allow more automatic kdoc parameter checks The following changes since commit e0756cfc7d7cd08c98a53b6009c091a3f6a50be6: Merge tag 'trace-v5.11-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace (2021-02-08 11:32:39 -0800) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-5.12-tag for you to fetch changes up to 9d294a685fbcb256ce8c5f7fd88a7596d0f52a8a: btrfs: zoned: enable to mount ZONED incompat flag (2021-02-09 02:52:24 +0100) Abaci Team (1): btrfs: simplify condition in __btrfs_run_delayed_items Filipe Manana (10): btrfs: send: remove stale code when checking for shared extents btrfs: remove wrong comment for can_nocow_extent() btrfs: remove unnecessary directory inode item update when deleting dir entry btrfs: stop setting nbytes when filling inode item for logging btrfs: avoid logging new ancestor inodes when logging new inode btrfs: skip logging directories already logged when logging all parents btrfs: skip logging inodes already logged when logging new entries btrfs: remove unnecessary check_parent_dirs_for_sync() btrfs: make concurrent fsyncs wait less when waiting for a transaction commit btrfs: fix extent buffer leak on failure to copy root Johannes Thumshirn (7): block: add bio_add_zone_append_page btrfs: release path before calling to btrfs_load_block_group_zone_info btrfs: zoned: do not load fs_info::zoned from incompat flag btrfs: zoned: allow zoned filesystems on non-zoned block devices btrfs: zoned: check if bio spans across an ordered extent btrfs: zoned: cache if block group is on a sequential zone btrfs: save irq flags when looking up an ordered extent Josef Bacik (34): btrfs: fix error handling in commit_fs_roots btrfs: allow error injection for btrfs_search_slot and btrfs_cow_block btrfs: noinline btrfs_should_cancel_balance btrfs: ref-verify: pass down tree block level when building refs btrfs: ref-verify: make sure owner is set for all refs btrfs: keep track of the root owner for relocation reads btrfs: do not cleanup upper nodes in btrfs_backref_cleanup_node btrfs: handle space_info::total_bytes_pinned inside the delayed ref itself btrfs: account
Re: [GIT PULL] Btrfs updates for 5.11
The pull request you sent on Mon, 14 Dec 2020 16:15:53 +0100: > git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-5.11-tag has been merged into torvalds/linux.git: https://git.kernel.org/torvalds/c/f1ee3b8829006b3fda999f00f0059aa327e3f3d0 Thank you! -- Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/prtracker.html
Re: [GIT PULL] Btrfs updates for 5.4
The pull request you sent on Mon, 16 Sep 2019 13:55:05 +0200: > git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-5.4 has been merged into torvalds/linux.git: https://git.kernel.org/torvalds/c/7d14df2d280fb7411eba2eb96682da0683ad97f6 Thank you! -- Deet-doot-dot, I am a bot. https://korg.wiki.kernel.org/userdoc/prtracker
Re: [GIT PULL] Btrfs updates for 5.4
On Mon, Sep 16, 2019 at 01:55:05PM +0200, David Sterba wrote: > > The following changes since commit f74c2bb98776e2de508f4d607cd519873065118e: > > Linux 5.3-rc8 (2019-09-08 13:33:15 -0700) > > are available in the Git repository at: > > git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-5.4 Correction: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-5.4-tag
[GIT PULL] Btrfs updates for 5.4
Hi, this update continues with work on code refactoring, sanity checks and space handling. There are some less user visible changes, nothing that would particularly stand out. Please pull, thanks. User visible changes: - tree checker, more sanity checks of: - ROOT_ITEM (key, size, generation, level, alignment, flags) - EXTENT_ITEM and METADATA_ITEM checks (key, size, offset, alignment, refs) - tree block reference items - EXTENT_DATA_REF (key, hash, offset) - deprecate flag BTRFS_SUBVOL_CREATE_ASYNC for subvolume creation ioctl, scheduled removal in 5.7 - delete stale and unused UAPI definitions BTRFS_DEV_REPLACE_ITEM_STATE_* - improved export of debugging information available via existing sysfs directory structure - try harder to delete relations between qgroups and allow to delete orphan entries - remove unreliable space checks before relocation starts Core: - space handling: - improved ticket reservations and other high level logic in order to remove special cases - factor flushing infrastructure and use it for different contexts, allows to remove some special case handling - reduce metadata reservation when only updating inodes - reduce global block reserve minimum size (affects small filesystems) - improved overcommit logic wrt global block reserve - tests: - fix memory leaks in extent IO tree - catch all TRIM range Fixes: - fix ENOSPC errors, leading to transaction aborts, when cloning extents - several fixes for inode number cache (mount option inode_cache) - fix potential soft lockups during send when traversing large trees - fix unaligned access to space cache pages with SLUB debug on (PowerPC) Other: - refactoring public/private functions, moving to new or more appropriate files - defines converted to enums - error handling improvements - more assertions and comments - old code deletion The following changes since commit f74c2bb98776e2de508f4d607cd519873065118e: Linux 5.3-rc8 (2019-09-08 13:33:15 -0700) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-5.4 for you to fetch changes up to 6af112b11a4bc1b560f60a618ac9c1dcefe9836e: btrfs: Relinquish CPUs in btrfs_compare_trees (2019-09-09 14:59:20 +0200) Anand Jain (7): btrfs: reset device stat using btrfs_dev_stat_set btrfs: opencode reset of all device stats btrfs: replace: BTRFS_DEV_REPLACE_ITEM_STATE_x defines should go btrfs: dev stats item key conversion per cpu type is not needed btrfs: dev stat drop useless goto btrfs: proper error handling when invalid device is found in find_next_devid btrfs: use proper error values on allocation failure in clone_fs_devices Arnd Bergmann (1): btrfs: reduce stack usage for btrfsic_process_written_block Christophe Leroy (1): btrfs: fix allocation of free space cache v1 bitmap pages Dan Carpenter (1): btrfs: fix error pointer check in __btrfs_map_block() David Sterba (33): btrfs: assert extent map tree lock in add_extent_mapping btrfs: assert tree mod log lock in __tree_mod_log_insert btrfs: remove unused btrfs_device::flush_bio_sent btrfs: remove unused key type set/get helpers btrfs: tree-log: convert defines to enums btrfs: async-thread: convert defines to enums btrfs: tree-log: use symbolic name for first replay stage btrfs: sysfs: add debugging exports btrfs: delete debugfs code btrfs: move sysfs declarations out of ctree.h btrfs: factor sysfs code out of link_block_group btrfs: sysfs: unexport btrfs_raid_ktype btrfs: factor out sysfs code for creating space infos btrfs: sysfs: unexport space_info_ktype btrfs: sysfs: replace direct access to feature set names with a helper btrfs: factor out sysfs code for sending device uevent btrfs: factor out sysfs code for deleting block group and space infos btrfs: factor out sysfs code for updating sprout fsid btrfs: cleanup kobject.h includes btrfs: sysfs: move type conversion helpers to sysfs.c btrfs: sysfs: move helper macros to sysfs.c btrfs: define compression levels statically btrfs: compression: replace set_level callbacks by a common helper btrfs: move cond_wake_up functions out of ctree btrfs: move math functions to misc.h btrfs: move private raid56 definitions from ctree.h btrfs: rename and export read_node_slot btrfs: move functions for tree compare to send.c btrfs: move struct io_ctl to free-space-cache.h btrfs: move dev_stats helpers to volumes.c btrfs: define separate btrfs_set/get_XX helpers btrfs: assume valid token for btrfs_set/get_token helpers btrfs: tie extent buffer and it's token together Eric Sandeen (1): btrfs: us
Re: [GIT PULL] Btrfs updates for 5.3
The pull request you sent on Mon, 15 Jul 2019 10:57:01 +0200: > git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-5.3-tag has been merged into torvalds/linux.git: https://git.kernel.org/torvalds/c/a18f8775419d3df282dd83efdb51c5a64d092f31 Thank you! -- Deet-doot-dot, I am a bot. https://korg.wiki.kernel.org/userdoc/prtracker
[GIT PULL] Btrfs updates for 5.3
Hi,
there's majority of cleanups and refactoring, no big new features made
it to the final branch, the rest are fixes.
No merge conflicts. Please pull, thanks.
Hilights:
- chunks that have been trimmed and unchanged since last mount are
tracked and skipped on repeated trims
- use hw assissed crc32c on more arches, speedups if native instructions
or optimized implementation is available
- the RAID56 incompat bit is automatically removed when the last block
group of that type is removed
Fixes:
- fsync fix for reflink on NODATACOW files that could lead to ENOSPC
- fix data loss after inode eviction, renaming it, and fsync it
- fix fsync not persisting dentry deletions due to inode evictions
- update ctime/mtime/iversion after hole punching
- fix compression type validation (reported by KASAN)
- send won't be allowed to start when relocation is in progress, this
can cause spurious errors or produce incorrect send stream
Core:
- new tracepoints for space update
- tree-checker: better check for end of extents for some tree items
- preparatory work for more checksum algorithms
- run delayed iput at unlink time and don't push the work to cleaner
thread where it's not properly throttled
- wrap block mapping to structures and helpers, base for further
refactoring
- split large files, part 1:
- space info handling
- block group reservations
- delayed refs
- delayed allocation
- other cleanups and refactoring
The following changes since commit 6fbc7275c7a9ba97877050335f290341a1fd8dbf:
Linux 5.2-rc7 (2019-06-30 11:25:36 +0800)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-5.3-tag
for you to fetch changes up to e02d48eaaed77f6c36916a7aa65c451e1f9d9aab:
btrfs: fix memory leak of path on error return path (2019-07-05 18:47:57
+0200)
Arnd Bergmann (1):
btrfs: shut up bogus -Wmaybe-uninitialized warning
Colin Ian King (1):
btrfs: fix memory leak of path on error return path
David Sterba (32):
btrfs: detect fast implementation of crc32c on all architectures
btrfs: fiemap: preallocate ulists for btrfs_check_shared
btrfs: fix minimum number of chunk errors for DUP
btrfs: raid56: allow the exact minimum number of devices for balance
convert
btrfs: remove mapping tree structures indirection
btrfs: use raid_attr table in get_profile_num_devs
btrfs: use raid_attr in btrfs_chunk_max_errors
btrfs: use raid_attr to get allowed profiles for balance conversion
btrfs: use raid_attr table to find profiles for integrity lowering
btrfs: use raid_attr table for btrfs_bg_type_to_factor
btrfs: factor out helper for counting data stripes
btrfs: use u8 for raid_array members
btrfs: factor out devs_max setting in __btrfs_alloc_chunk
btrfs: refactor helper for bg flags to name conversion
btrfs: constify map parameter for nr_parity_stripes and nr_data_stripes
btrfs: read number of data stripes from map only once
btrfs: use file:line format for assertion report
btrfs: tests: add locks around add_extent_mapping
btrfs: assert delayed ref lock in btrfs_find_delayed_ref_head
btrfs: switch extent_buffer blocking_writers from atomic to int
btrfs: switch extent_buffer spinning_writers from atomic to int
btrfs: switch extent_buffer write_locks from atomic to int
btrfs: raid56: clear incompat block group flags after removing the last
one
btrfs: add mask for all RAID1 types
btrfs: use mask for RAID56 profiles
btrfs: document BTRFS_MAX_MIRRORS
btrfs: improve messages when updating feature flags
btrfs: use common helpers for extent IO state insertion messages
btrfs: drop default value assignments in enums
btrfs: use raid_attr to adjust minimal stripe size in
btrfs_calc_avail_data_space
btrfs: use raid_attr for minimum stripe count in
btrfs_calc_avail_data_space
btrfs: lift bio_set_dev from bio allocation helpers
Filipe Manana (4):
Btrfs: fix data loss after inode eviction, renaming it, and fsync it
Btrfs: prevent send failures and crashes due to concurrent relocation
Btrfs: fix fsync not persisting dentry deletions due to inode evictions
Btrfs: add missing inode version, ctime and mtime updates when punching
hole
Goldwyn Rodrigues (3):
btrfs: Remove unused variable mode in btrfs_mount
btrfs: Simplify update of space_info in __reserve_metadata_bytes()
btrfs: Evaluate io_tree in find_lock_delalloc_range()
Johannes Thumshirn (13):
btrfs: use btrfs_csum_data() instead of directly calling crc32c
btrfs: resurrect btrfs_crc32c()
btrfs: use btrfs_crc32c{,_final}() in for free space cache
btrfs: don't assume ordered sums to be 4 bytes
btrfs: don't
Re: [GIT PULL] Btrfs updates for 5.1, part 1
The pull request you sent on Mon, 4 Mar 2019 20:20:53 +0100: > git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git > for-5.1-part1-tag has been merged into torvalds/linux.git: https://git.kernel.org/torvalds/c/b1e243957e9b3ba8e820fb8583bdf18e7c737aa2 Thank you! -- Deet-doot-dot, I am a bot. https://korg.wiki.kernel.org/userdoc/prtracker
[GIT PULL] Btrfs updates for 5.1, part 1
Hi, the branch contains usual mix of new features, core changes and fixes; full list below. I'm planning 2nd pull request, with a few more fixes that arrived recently but too close to merge window, will send it next week. Please pull, thanks. New features: - support zstd compression levels - new ioctl to unregister a device from the module (ie. reverse of device scan) - scrub prints a message to log when it's about to start or finish Core changes: - qgroups can now skip part of a tree that does not get updated during relocation, because this does not affect the quota accounting, estimated speedup in run time is about 20% - the compression workspace management had to be enhanced due to zstd requirements - various enospc fixes, when there's high fragmentation the over-reservation can cause ENOSPC that might not happen after a flush, in such cases try to wait if the situation improves Fixes: - various ioctls could overwrite previous return value if copy_to_user fails, fix this so the original error is reported - more reclaim vs GFP_KERNEL fixes - other cleanups and refactoring - fix a (valid) lockdep warning in a test when device replace is destroying worker threads - make qgroup async transaction commit more aggressive, this avoids some 'quota limit reached' errors if there are not enough data to trigger transaction in order to flush - fix deadlock between snapshot deletion and quotas when backref walking is called from context that already holds the same locks - fsync fixes: - fix fsync after succession of renames of different files - fix fsync after succession of renames and unlink/rmdir The following changes since commit 5908e6b738e3357af42c10e1183753c70a0117a9: Linux 5.0-rc8 (2019-02-24 16:46:45 -0800) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-5.1-part1-tag for you to fetch changes up to f65e25e343cfc0e6f4db9a687c4085fad268325d: btrfs: Remove unnecessary casts in btrfs_read_root_item (2019-02-25 14:19:23 +0100) Anand Jain (11): btrfs: merge btrfs_find_device_missing_or_by_path() into parent btrfs: cleanup btrfs_find_device_by_devspec() btrfs: refactor btrfs_find_device() take fs_devices as argument btrfs: refactor btrfs_free_stale_devices() to get return value btrfs: merge btrfs_find_device and find_device btrfs: scrub: print messages when started or finished btrfs: introduce new ioctl to unregister a btrfs device btrfs: fix comment its device list mutex not volume lock btrfs: scrub: fix circular locking dependency warning btrfs: scrub: add scrub_lock lockdep check in scrub_workers_get btrfs: scrub: convert scrub_workers_refcnt to refcount_t Anders Roxell (1): btrfs: let the assertion expression compile in all configs Dan Carpenter (1): btrfs: drop the lock on error in btrfs_dev_replace_cancel Dan Robertson (1): btrfs: init csum_list before possible free David Sterba (9): btrfs: simplify workqueue name when allocating btrfs: split btrfs_set_lock_blocking_rw to read and write helpers btrfs: split btrfs_clear_lock_blocking_rw to read and write helpers btrfs: replace btrfs_set_lock_blocking_rw with appropriate helpers btrfs: open code now trivial btrfs_set_lock_blocking btrfs: simplify waiting loop in btrfs_tree_lock btrfs: merge btrfs_set_lock_blocking_rw with it's caller btrfs: scrub: add assertions for worker pointers btrfs: scrub: remove unused nocow worker pointer Dennis Zhou (12): btrfs: add helpers for compression type and level btrfs: rename workspaces_list to workspace_manager btrfs: manage heuristic workspace as index 0 btrfs: unify compression ops with workspace_manager btrfs: add helper methods for workspace manager init and cleanup btrfs: add compression interface in (get/put)_workspace btrfs: move to function pointers for get/put workspaces btrfs: plumb level through the compression interface btrfs: change set_level() to bound the level passed in btrfs: zstd use the passed through level instead of default btrfs: make zstd memory requirements monotonic btrfs: add zstd compression level support Filipe Manana (14): Btrfs: do not overwrite scrub error with fault error in scrub ioctl Btrfs: do not overwrite error return value in scrub progress ioctl Btrfs: do not overwrite error return value in the get device stats ioctl Btrfs: setup a nofs context for memory allocation at btrfs_create_tree() Btrfs: setup a nofs context for memory allocation at __btrfs_set_acl Btrfs: remove redundant check for swapfiles when reflinking Btrfs: do not overwrite error return value in the device replace ioctl
Re: [GIT PULL] Btrfs updates for 4.21
The pull request you sent on Wed, 19 Dec 2018 16:11:04 +0100: > git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-4.21-tag has been merged into torvalds/linux.git: https://git.kernel.org/torvalds/c/32ee34eddad13cd44ad0cb3e659fe6fd49143b62 Thank you! -- Deet-doot-dot, I am a bot. https://korg.wiki.kernel.org/userdoc/prtracker
[GIT PULL] Btrfs updates for 4.21
Hi, I've planned to merge a few more patches to the 1st pull branch that belong to a core change mentioned below as 'delayed refs reserve'. There are review comments that I think should be addressed, so the patches are postponed. Technically they're fixes and I presume it's ok to take them in the -rc. I didn't want to touch the pull request branch a few days before sending it, though the patches have been in linux-next for some time. The bugs fixed are not critical for testing so this should not block anybody. No merge conflicts, please pull. Thanks. -- New features: * swapfile support - after a long time it's here, with some limitations where COW design does not work well with the swap implementation (nodatacow file, no compression, cannot be snapshotted, not possible on multiple devices, ...), as this is the most restricted but working setup, we'll try to improve that in the future * metadata uuid - an optional incompat feature to assign a new filesystem UUID without overwriting all metadata blocks, stored only in superblock * more balance messages are printed to system log, initial is in the format of the command line that would be used to start it Fixes: * tag pages of a snapshot to better separate pages that are involved in the snapshot (and need to get synced) from newly dirtied pages that could slow down or even livelock the snapshot operation * improved check of filesystem id associated with a device during scan to detect duplicate devices that could be mixed up during mount * fix device replace state transitions, eg. when it ends up interrupted and reboot tries to restart balance too, or when start/cancel ioctls race * fix a crash due to a race when quotas are enabled during snapshot creation * GFP_NOFS/memalloc_nofs_* fixes due to GFP_KERNEL allocations in transaction context * fix fsync of files with multiple hard links in new directories * fix race of send with transaction commits that create snapshots Core changes: * cleanups * further removals of now-dead fsync code * core function for finding free extent has been split and provides a base for further cleanups to make the logic more understandable * removed lot of indirect callbacks for data and metadata inodes * simplified refcounting and locking for cloned extent buffers * removed redundant function arguments * defines converted to enums where appropriate * separate reserve for delayed refs from global reserve, update logic to do less trickery and ad-hoc heuristics, move out some related expensive operations from transaction commit or file truncate * dev-replace switched from custom locking scheme to semaphore * remove first phase of balance that tried to make some space for the relocation by calling shrink and grow, this did not work as expected and only introduced more error states due to potential resize failures, slightly improves the runtime as the chunks on all devices are not needlessly enumerated * clone and deduplication now use generic helper that adds a few more checks that were missing from the original btrfs implementation of the ioctls The following changes since commit 7566ec393f4161572ba6f11ad5171fd5d59b0fbd: Linux 4.20-rc7 (2018-12-16 15:46:55 -0800) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-4.21-tag for you to fetch changes up to 52042d8e82ff50d40e76a275ac0b97aa663328b0: btrfs: Fix typos in comments and strings (2018-12-17 14:51:50 +0100) Anand Jain (14): btrfs: remove redundant replace_state init btrfs: harden agaist duplicate fsid on scanned devices btrfs: mark btrfs_dev_replace_start as static btrfs: dev-replace: go back to suspended state if target device is missing btrfs: dev-replace: go back to suspend state if another EXCL_OP is running btrfs: fix use-after-free due to race between replace start and cancel btrfs: dev-replace: set result code of cancel by status of scrub btrfs: dev-replace: replace's scrub must not be running in suspended state btrfs: dev-replace: add explicit check for replace result "no error" btrfs: silence warning if replace is canceled btrfs: don't report user-requested cancel as an error btrfs: add helper to describe block group flags btrfs: balance: print args during start and resume btrfs: balance: print to system log when balance ends or is paused Andrea Gelmini (1): btrfs: Fix typos in comments and strings David Sterba (18): btrfs: merge btrfs_submit_bio_done to its caller btrfs: replace async_cow::root with fs_info btrfs: remove redundant csum buffer in btrfs_io_bio btrfs: replace btrfs_io_bio::end_io with a simple helper btrfs: switch BTRFS_FS_STATE_* to enums btrfs: switch BTRFS_BLOCK
Re: [GIT PULL] Btrfs updates for 4.20-rc4
The pull request you sent on Wed, 28 Nov 2018 13:28:05 +0100: > git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-4.20-rc4-tag has been merged into torvalds/linux.git: https://git.kernel.org/torvalds/c/121b018f8c74b4e0ba81b4b8ee73a82db3f24b7b Thank you! -- Deet-doot-dot, I am a bot. https://korg.wiki.kernel.org/userdoc/prtracker
[GIT PULL] Btrfs updates for 4.20-rc4
Hi, please pull the following fixes. Some of them are being hit during testing so we'd like to get them merged, otherwise there are usual stability fixes for stable trees. Thanks. The following changes since commit d6fd0ae25c6495674dc5a41a8d16bc8e0073276d: Btrfs: fix missing delayed iputs on unmount (2018-11-07 20:17:45 +0100) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-4.20-rc4-tag for you to fetch changes up to 42a657f57628402c73237547f0134e083e2f6764: btrfs: relocation: set trans to be NULL after ending transaction (2018-11-23 13:47:46 +0100) Filipe Manana (3): Btrfs: fix rare chances for data loss when doing a fast fsync Btrfs: ensure path name is null terminated at btrfs_control_ioctl Btrfs: fix race between enabling quotas and subvolume creation Nikolay Borisov (1): btrfs: Always try all copies when reading extent buffers Pan Bian (1): btrfs: relocation: set trans to be NULL after ending transaction Robbie Ko (1): Btrfs: send, fix infinite loop due to directory rename dependencies fs/btrfs/disk-io.c| 11 +-- fs/btrfs/file.c | 24 fs/btrfs/qgroup.c | 3 ++- fs/btrfs/relocation.c | 1 + fs/btrfs/send.c | 11 --- fs/btrfs/super.c | 1 + 6 files changed, 37 insertions(+), 14 deletions(-)
Re: [GIT PULL] Btrfs updates for 4.20, part 2
On Tue, Oct 30, 2018 at 6:24 AM David Sterba wrote: > > this part contains a few minor updates and fixes that were under testing > or arrived shortly after the merge window freeze, mostly stable material. Pulled, Linus
[GIT PULL] Btrfs updates for 4.20, part 2
Hi, this part contains a few minor updates and fixes that were under testing or arrived shortly after the merge window freeze, mostly stable material. Please pull, thanks. The following changes since commit d9352794dad9f28535439d85a815978878c141ab: btrfs: switch return_bigger to bool in find_ref_head (2018-10-15 17:23:41 +0200) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-4.20-part2-tag for you to fetch changes up to 9084cb6a24bf5838a665af92ded1af8363f9e563: Btrfs: fix use-after-free when dumping free space (2018-10-22 20:31:22 +0200) Filipe Manana (5): Btrfs: fix null pointer dereference on compressed write path error Btrfs: fix assertion on fsync of regular file when using no-holes feature Btrfs: fix deadlock when writing out free space caches Btrfs: fix use-after-free during inode eviction Btrfs: fix use-after-free when dumping free space Josef Bacik (8): MAINTAINERS: update my email address for btrfs btrfs: reset max_extent_size properly btrfs: set max_extent_size properly btrfs: don't use ctl->free_space for max_extent_size btrfs: only free reserved extent if we didn't insert it btrfs: fix insert_reserved error handling btrfs: don't run delayed_iputs in commit btrfs: move the dio_sem higher up the callchain Lu Fengqi (1): btrfs: delayed-ref: extract find_first_ref_head from find_ref_head MAINTAINERS | 2 +- fs/btrfs/ctree.c| 17 +++ fs/btrfs/delayed-ref.c | 50 - fs/btrfs/extent-tree.c | 37 +++-- fs/btrfs/file.c | 12 +++ fs/btrfs/free-space-cache.c | 32 - fs/btrfs/inode.c| 15 -- fs/btrfs/transaction.c | 9 fs/btrfs/tree-log.c | 5 ++--- 9 files changed, 111 insertions(+), 68 deletions(-)
Re: [GIT PULL] Btrfs updates for 4.20, part 1
On Mon, Oct 22, 2018 at 6:22 PM David Sterba wrote: > > this is the first batch with fixes and some nice performance improvements. Pulled (at 30,000 ft, somewhere between Ireland and Iceland), Linus
Re: [GIT PULL] Btrfs updates for 4.20, part 1
On Mon, Oct 22, 2018 at 05:23:36PM -0700, Liu Bo wrote: > > Performance improvements: > > > > * blocking mode of path is gone, means that only the spinning mode is used; > > I'd like to do a few corrections here, the transition from the > spinning mode to blocking mode is removed, we still need blocking mode > of path for sleeping context. Ok, thanks. So for the purpose of merge changelog, the updated text: * transition between blocking and spinning modes of path is gone, which originally resulted to more unnecessary wakeups and updates to the path locks, the effects are measurable and improve latency and scalability
Re: [GIT PULL] Btrfs updates for 4.20, part 1
On Mon, Oct 22, 2018 at 10:24 AM David Sterba wrote: > > Hi, > > this is the first batch with fixes and some nice performance improvements. > > Preliminary results show eg. more files/sec in fsmark, better perf on > multi-threaded workloads (filebench, dbench), fewer context switches and > overall better memory allocation characteristics (multiple benchmarks). > > Apart from general performance, there's an improvement for qgroups + > balance workload that's been troubling our users. > > Note for stable: there are 20+ patches tagged for stable, out of 90. Not > all of them apply cleanly on all stable versions but the conflicts are > mostly due to simple cleanups and resolving should be obvious. The fixes > are otherwise independent. > > No merge conflicts expected. Please pull, thanks. > > > Performance improvements: > > * blocking mode of path is gone, means that only the spinning mode is used; I'd like to do a few corrections here, the transition from the spinning mode to blocking mode is removed, we still need blocking mode of path for sleeping context. thanks, liubo > the blocking resulted in more unnecessary wakeups and updates to the path > locks, the effects are measurable and improve latency and scaleability > > * qgroups: first batch of changes that should speedup balancing with qgroups > on, skip quota accounting on unchanged subtrees, overall gain is about 30+% > in runtime > > * use rb-tree with cached first node for several structures, small improvement > to avoid pointer chasing > > Fixes: > > * trim > * fix: some blockgroups could have been missed if their logical address was > past the total filesystem size (ie. after a lot of balancing) > * better error reporting, after processing blockgroups and whole device > * fix: continue trimming block groups after an error is encountered > * check for trim support of the device earlier and avoid some unnecessary > work > * less interaction with transaction commit that improves latency on slower > storage (eg. image files over NFS) > > * fsync > * fix warning when replaying log after fsync of a O_TMPFILE > * fix wrong dentries after fsync of file that got its parent replaced > > * qgroups: fix rescan that might misc some dirty groups > > * don't clean dirty pages during buffered writes, this could lead to lost > updates in some corner cases > > * some block groups could have been delayed in creation, if the allocation > triggered another one > > * error handling improvements > > Cleanups: > > * removed unused struct members and variables > * function return type cleanups > * delayed refs code refactoring > > * protect against deadlock that could be caused by crafted image that tries to > allocate from a tree that's locked already > > > The following changes since commit 35a7f35ad1b150ddf59a41dcac7b2fa32982be0e: > > Linux 4.19-rc8 (2018-10-15 07:20:24 +0200) > > are available in the Git repository at: > > git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git > for-4.20-part1-tag > > for you to fetch changes up to d9352794dad9f28535439d85a815978878c141ab: > > btrfs: switch return_bigger to bool in find_ref_head (2018-10-15 17:23:41 > +0200) > > > Anand Jain (2): > btrfs: add assertions where number of devices could go below 0 > btrfs: add helper to obtain number of devices with ongoing dev-replace > > Chris Mason (1): > Btrfs: don't clean dirty pages during buffered writes > > Colin Ian King (2): > btrfs: remove unused pointer inode in relink_file_extents > btrfs: remove unused pointer 'tree' in btrfs_submit_compressed_read > > David Sterba (12): > btrfs: tests: add separate stub for find_lock_delalloc_range > btrfs: tests: move testing members of struct btrfs_root to the end > btrfs: tests: group declarations of self-test helpers > btrfs: tests: polish ifdefs around testing helper > btrfs: use common helper instead of open coding a bit test > btrfs: remove btrfs_dev_replace::read_locks > btrfs: open code btrfs_dev_replace_clear_lock_blocking > btrfs: open code btrfs_dev_replace_stats_inc > btrfs: open code btrfs_after_dev_replace_commit > btrfs: dev-replace: avoid useless lock on error handling path > btrfs: dev-replace: move replace members out of fs_info > btrfs: dev-replace: remove pointless assert in write unlock > > Filipe Manana (2): > Btrfs: fix warning when replaying log after fsync of a tmpfile > Btrfs: fix wrong dentries after fsync of file that got its parent > replaced > > Jeff Mahoney (5): > btrfs: fix error handling in free_log_tree > btrfs: fix error handling in btrfs_dev_replace_start > btrfs: iterate all devices during trim, instead of > fs_devices::alloc_list > btrfs: don't attempt to trim devices that don
[GIT PULL] Btrfs updates for 4.20, part 1
Hi, this is the first batch with fixes and some nice performance improvements. Preliminary results show eg. more files/sec in fsmark, better perf on multi-threaded workloads (filebench, dbench), fewer context switches and overall better memory allocation characteristics (multiple benchmarks). Apart from general performance, there's an improvement for qgroups + balance workload that's been troubling our users. Note for stable: there are 20+ patches tagged for stable, out of 90. Not all of them apply cleanly on all stable versions but the conflicts are mostly due to simple cleanups and resolving should be obvious. The fixes are otherwise independent. No merge conflicts expected. Please pull, thanks. Performance improvements: * blocking mode of path is gone, means that only the spinning mode is used; the blocking resulted in more unnecessary wakeups and updates to the path locks, the effects are measurable and improve latency and scaleability * qgroups: first batch of changes that should speedup balancing with qgroups on, skip quota accounting on unchanged subtrees, overall gain is about 30+% in runtime * use rb-tree with cached first node for several structures, small improvement to avoid pointer chasing Fixes: * trim * fix: some blockgroups could have been missed if their logical address was past the total filesystem size (ie. after a lot of balancing) * better error reporting, after processing blockgroups and whole device * fix: continue trimming block groups after an error is encountered * check for trim support of the device earlier and avoid some unnecessary work * less interaction with transaction commit that improves latency on slower storage (eg. image files over NFS) * fsync * fix warning when replaying log after fsync of a O_TMPFILE * fix wrong dentries after fsync of file that got its parent replaced * qgroups: fix rescan that might misc some dirty groups * don't clean dirty pages during buffered writes, this could lead to lost updates in some corner cases * some block groups could have been delayed in creation, if the allocation triggered another one * error handling improvements Cleanups: * removed unused struct members and variables * function return type cleanups * delayed refs code refactoring * protect against deadlock that could be caused by crafted image that tries to allocate from a tree that's locked already The following changes since commit 35a7f35ad1b150ddf59a41dcac7b2fa32982be0e: Linux 4.19-rc8 (2018-10-15 07:20:24 +0200) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-4.20-part1-tag for you to fetch changes up to d9352794dad9f28535439d85a815978878c141ab: btrfs: switch return_bigger to bool in find_ref_head (2018-10-15 17:23:41 +0200) Anand Jain (2): btrfs: add assertions where number of devices could go below 0 btrfs: add helper to obtain number of devices with ongoing dev-replace Chris Mason (1): Btrfs: don't clean dirty pages during buffered writes Colin Ian King (2): btrfs: remove unused pointer inode in relink_file_extents btrfs: remove unused pointer 'tree' in btrfs_submit_compressed_read David Sterba (12): btrfs: tests: add separate stub for find_lock_delalloc_range btrfs: tests: move testing members of struct btrfs_root to the end btrfs: tests: group declarations of self-test helpers btrfs: tests: polish ifdefs around testing helper btrfs: use common helper instead of open coding a bit test btrfs: remove btrfs_dev_replace::read_locks btrfs: open code btrfs_dev_replace_clear_lock_blocking btrfs: open code btrfs_dev_replace_stats_inc btrfs: open code btrfs_after_dev_replace_commit btrfs: dev-replace: avoid useless lock on error handling path btrfs: dev-replace: move replace members out of fs_info btrfs: dev-replace: remove pointless assert in write unlock Filipe Manana (2): Btrfs: fix warning when replaying log after fsync of a tmpfile Btrfs: fix wrong dentries after fsync of file that got its parent replaced Jeff Mahoney (5): btrfs: fix error handling in free_log_tree btrfs: fix error handling in btrfs_dev_replace_start btrfs: iterate all devices during trim, instead of fs_devices::alloc_list btrfs: don't attempt to trim devices that don't support it btrfs: keep trim from interfering with transaction commits Josef Bacik (7): btrfs: wait on caching when putting the bg cache btrfs: release metadata before running delayed refs btrfs: protect space cache inode alloc with GFP_NOFS btrfs: reset max_extent_size on clear in a bitmap btrfs: make sure we create all new block groups btrfs: assert on non-empty delayed iputs btrfs: drop mi
Re: [GIT PULL] Btrfs updates for 4.18
On 06/29/2018 02:26 AM, David Sterba wrote:
On Thu, Jun 28, 2018 at 07:22:59PM +0800, Anand Jain wrote:
The circular locking dependency warning occurs at FSSTRESS_PROG.
And in particular at doproc() in xfstests/ltp/fsstress.c, randomly
at any of the command at
opdesc_tops[] = { ..}
which involves calling mmap file operation and if there is something
to commit.
The commit transaction does need device_list_mutex which is also being
used for the btrfs_open_devices() in the commit 542c5908abfe84f7.
But btrfs_open_devices() is only called at mount, and mmap() can
establish only be established after the mount has completed. With
this give its unclear to me why the circular locking dependency check
is warning about this.
I feel until we have clarity about this and also solve other problem
related to the streamlining of uuid_mutex, I suggest we revert
542c5908abfe84f7. Sorry for the inconvenience.
Ok, the revert is one option. I'm cosidering adding both the locks, like
is in https://patchwork.kernel.org/patch/10478443/ . This would have no
effect, as btrfs_open_devices is called only from mount path and the
list_sort is done only for the first time when there are not other
users of the list that would not also be under the uuid_mutex.
This passed the syzbot and other tests, so this does not break things
and goes towards pushing the device_list_mutex as the real protection
mechanism for the fs_devices members.
Let me know what you think, the revert should be the last option if we
don't have anything better.
With this patch [1] as well I find the circular lock warning[2].
[1]
https://patchwork.kernel.org/patch/10478443/
Test case:
mkfs.btrfs -fq /dev/sdc && mount /dev/sdc /btrfs &&
/xfstests/ltp/fsstress -d /btrfs -w -p 1 -n 2000
However when the device_list_mutex is removed, the warning goes away.
Let me investigate bit more about circular locking dependency.
About using uuid_mutex in btrfs_open_devices().
I am planning to be more conceivable about the using the
bit map for the volume flags and which shall also include the
EXCL OPS in progress flag for the fs_devices. Which means we hold
uuid_mutex and set/reset EXCL OPS flag for the fs_devices. And so
the other fsids like fsid2 can still hold the uuid_mutex while
fsid1 is still mounting/opening (which may sleep).
I hope you would agree to use bit map for volume, we also need this
bit map to manage the volume status. Or if there is a better solution
I am fine. However uuid_mutex isn't as it blocks fsids2 to mount.
Thanks, Anand
[2]
---
kernel:
kernel: ==
kernel: WARNING: possible circular locking dependency detected
kernel: 4.18.0-rc1+ #63 Not tainted
kernel: --
kernel: fsstress/3062 is trying to acquire lock:
kernel: 7d28aeca (&fs_info->reloc_mutex){+.+.}, at:
btrfs_record_root_in_trans+0x43/0x70 [btrfs]
kernel:
but task is already holding lock:
kernel: 2fc78565 (&mm->mmap_sem){}, at:
vm_mmap_pgoff+0x9f/0x110
kernel:
which lock already depends on the new lock.
kernel:
the existing dependency chain (in reverse
order) is:
kernel:
-> #5 (&mm->mmap_sem){}:
kernel:_copy_from_user+0x1e/0x90
kernel:scsi_cmd_ioctl+0x2ba/0x480
kernel:cdrom_ioctl+0x3b/0xb2e
kernel:sr_block_ioctl+0x7e/0xc0
kernel:blkdev_ioctl+0x4ea/0x980
kernel:block_ioctl+0x39/0x40
kernel:do_vfs_ioctl+0xa2/0x6c0
kernel:ksys_ioctl+0x70/0x80
kernel:__x64_sys_ioctl+0x16/0x20
kernel:do_syscall_64+0x4a/0x180
kernel:entry_SYSCALL_64_after_hwframe+0x49/0xbe
kernel:
-> #4 (sr_mutex){+.+.}:
kernel:sr_block_open+0x24/0xd0
kernel:__blkdev_get+0xcb/0x480
kernel:blkdev_get+0x144/0x3a0
kernel:do_dentry_open+0x1b1/0x2d0
kernel:path_openat+0x57b/0xcc0
kernel:do_filp_open+0x9b/0x110
kernel:do_sys_open+0x1bd/0x250
kernel:do_syscall_64+0x4a/0x180
kernel:entry_SYSCALL_64_after_hwframe+0x49/0xbe
kernel:
-> #3 (&bdev->bd_mutex){+.+.}:
kernel:__blkdev_get+0x5d/0x480
kernel:blkdev_get+0x243/0x3a0
kernel:blkdev_get_by_path+0x4a/0x80
kernel:btrfs_get_bdev_and_sb+0x1b/0xa0 [btrfs]
kernel:open_fs_devices+0x85/0x270 [btrfs]
kernel:btrfs_open_devices+0x6b/0x70 [btrfs]
kernel:btrfs_mount_root+0x41a/0x7e0 [btrfs]
kernel:mount_fs+0x30/0x150
kernel:vfs_kern_mount.part.31+0x54/0x140
kernel:btrfs_mount+0x175/0x920 [btrfs]
kernel:mount_fs+0x30/0x150
kernel:vfs_kern_mount.part.31+0x
Re: [GIT PULL] Btrfs updates for 4.18
On Thu, Jun 28, 2018 at 07:22:59PM +0800, Anand Jain wrote:
> The circular locking dependency warning occurs at FSSTRESS_PROG.
> And in particular at doproc() in xfstests/ltp/fsstress.c, randomly
> at any of the command at
> opdesc_tops[] = { ..}
> which involves calling mmap file operation and if there is something
> to commit.
>
> The commit transaction does need device_list_mutex which is also being
> used for the btrfs_open_devices() in the commit 542c5908abfe84f7.
>
> But btrfs_open_devices() is only called at mount, and mmap() can
> establish only be established after the mount has completed. With
> this give its unclear to me why the circular locking dependency check
> is warning about this.
>
> I feel until we have clarity about this and also solve other problem
> related to the streamlining of uuid_mutex, I suggest we revert
> 542c5908abfe84f7. Sorry for the inconvenience.
Ok, the revert is one option. I'm cosidering adding both the locks, like
is in https://patchwork.kernel.org/patch/10478443/ . This would have no
effect, as btrfs_open_devices is called only from mount path and the
list_sort is done only for the first time when there are not other
users of the list that would not also be under the uuid_mutex.
This passed the syzbot and other tests, so this does not break things
and goes towards pushing the device_list_mutex as the real protection
mechanism for the fs_devices members.
Let me know what you think, the revert should be the last option if we
don't have anything better.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates for 4.18
On 06/12/2018 12:16 AM, David Sterba wrote:
On Mon, Jun 11, 2018 at 10:50:54AM +0100, Filipe Manana wrote:
btrfs: replace uuid_mutex by device_list_mutex in
btrfs_open_devices
*
* the mutex can be very coarse and can cover long-running operations
*
* protects: updates to fs_devices counters like missing devices, rw
devices,
* seeding, structure cloning, openning/closing devices at mount/umount
time
generates some confusion since btrfs_open_devices(), after that
commit, no longer takes the uuid_mutex and it
updates some fs_devices counters (opened, open_devices, etc).
As uuid_mutex is a global fs_uuids lock for the per fsid operations
doesn't make any sense.
This problem is reproducible only for-4.18, misc-next if fine.
I am looking deeper.
What about the unprotected updates (increments) to fs_devices->opened
and fs_devices->open_devices?
Other functions are accessing/updating them while holding the uuid mutex.
The goal is to reduce usage of uuid_mutex only to protect search or
update of the fs_uuids list, everything else should be protected by the
device_list_mutex.
The commit 542c5908abfe84f7 (use device_list_mutex in
btrfs_open_devices) implements that but then the access to the ->opened
member is not protected consistently. There are patches that convert the
use to device_list_mutex but haven't been merged due to refinements or
pending review.
At this point I think we should revert the one commit 542c5908abfe84f7
as it introduces the locking problems and revisit the whole fs_devices
locking scheme again in the dex dev cycle. That will be post rc1 as
there might be more to revert.
I tried to narrow this, it appears some of the things that
circular locking dependency check report doesn't make sense.
Here below is what I find.. as of now.
The test case btrfs/004 can be simplified to.. which also
reproduces the problem.
-8<-
$ cat 165
#! /bin/bash
# FS QA Test No. btrfs/165
#
seq=`basename $0`
seqres=$RESULT_DIR/$seq
echo "QA output created by $seq"
here=`pwd`
tmp=/tmp/$$
status=1
noise_pid=0
_cleanup()
{
wait
rm -f $tmp.*
}
trap "_cleanup; exit \$status" 0 1 2 3 15
# get standard environment, filters and checks
. ./common/rc
. ./common/filter
# real QA test starts here
_supported_fs btrfs
_supported_os Linux
_require_scratch
rm -f $seqres.full
run_check _scratch_mkfs_sized $((2000 * 1024 * 1024))
run_check _scratch_mount
run_check $FSSTRESS_PROG -d $SCRATCH_MNT -w -p 1 -n 2000 $FSSTRESS_AVOID
run_check _scratch_unmount
echo "done"
status=0
exit
-8<-
The circular locking dependency warning occurs at FSSTRESS_PROG.
And in particular at doproc() in xfstests/ltp/fsstress.c, randomly
at any of the command at
opdesc_tops[] = { ..}
which involves calling mmap file operation and if there is something
to commit.
The commit transaction does need device_list_mutex which is also being
used for the btrfs_open_devices() in the commit 542c5908abfe84f7.
But btrfs_open_devices() is only called at mount, and mmap() can
establish only be established after the mount has completed. With
this give its unclear to me why the circular locking dependency check
is warning about this.
I feel until we have clarity about this and also solve other problem
related to the streamlining of uuid_mutex, I suggest we revert
542c5908abfe84f7. Sorry for the inconvenience.
Thanks, Anand
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates for 4.18
On Mon, Jun 11, 2018 at 10:50:54AM +0100, Filipe Manana wrote: > >>>btrfs: replace uuid_mutex by device_list_mutex in > >>> btrfs_open_devices > >> * > >> * the mutex can be very coarse and can cover long-running operations > >> * > >> * protects: updates to fs_devices counters like missing devices, rw > >> devices, > >> * seeding, structure cloning, openning/closing devices at mount/umount > >> time > >> > >> generates some confusion since btrfs_open_devices(), after that > >> commit, no longer takes the uuid_mutex and it > >> updates some fs_devices counters (opened, open_devices, etc). > > > > As uuid_mutex is a global fs_uuids lock for the per fsid operations > > doesn't make any sense. > > > > This problem is reproducible only for-4.18, misc-next if fine. > > I am looking deeper. > > What about the unprotected updates (increments) to fs_devices->opened > and fs_devices->open_devices? > Other functions are accessing/updating them while holding the uuid mutex. The goal is to reduce usage of uuid_mutex only to protect search or update of the fs_uuids list, everything else should be protected by the device_list_mutex. The commit 542c5908abfe84f7 (use device_list_mutex in btrfs_open_devices) implements that but then the access to the ->opened member is not protected consistently. There are patches that convert the use to device_list_mutex but haven't been merged due to refinements or pending review. At this point I think we should revert the one commit 542c5908abfe84f7 as it introduces the locking problems and revisit the whole fs_devices locking scheme again in the dex dev cycle. That will be post rc1 as there might be more to revert. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates for 4.18
On Mon, Jun 11, 2018 at 9:14 AM, Anand Jain wrote:
>
>
> On 06/10/2018 12:21 AM, Filipe Manana wrote:
>>
>> On Mon, Jun 4, 2018 at 4:43 PM, David Sterba wrote:
>>>
>>> Hi,
>>>
>>> there are some new features and a usual load of cleanups, more details
>>> below.
>>>
>>> Specifically, there's a set of new non-privileged ioctls to allow
>>> subvolume listing. It works but still needs a security review as it's a
>>> new interface and we might need to do some tweaks to the data
>>> structures. The fixes could be considred regressions but may touch the
>>> interfaces too.
>>>
>>> Currently there are no merge conflicts but linux-next has reported a few
>>> in the past, originating from other *FS trees.
>>>
>>> Please pull, thanks.
>>>
>>> ---
>>>
>>> User visible features:
>>>
>>> - added support for the ioctl FS_IOC_FSGETXATTR, per-inode flags,
>>> successor
>>>of GET/SETFLAGS; now supports only existing flags: append, immutable,
>>>noatime, nodump, sync
>>>
>>> - 3 new unprivileged ioctls to allow users to enumerate subvolumes
>>>
>>> - dedupe syscall implementation does not restrict the range to 16MiB,
>>> though it
>>>still splits the whole range to 16MiB chunks
>>>
>>> - on user demand, rmdir() is able to delete an empty subvolume, export
>>> the
>>>capability in sysfs
>>>
>>> - fix inode number types in tracepoints, other cleanups
>>>
>>> - send: improved speed when dealing with a large removed directory,
>>>measurements show decrease from 2000 minutes to 2 minutes on a
>>> directory with
>>>2 million entries
>>>
>>> - pre-commit check of superblock to detect a mysterious in-memory
>>> corruption
>>>
>>> - log message updates
>>>
>>>
>>> Other changes:
>>>
>>> - orphan inode cleanup improved, does no keep long-standing reservations
>>> that
>>>could lead up to early ENOSPC in some cases
>>>
>>> - slight improvement of handling snapshotted NOCOW files by avoiding some
>>>unnecessary tree searches
>>>
>>> - avoid OOM when dealing with many unmergeable small extents at flush
>>> time
>>>
>>> - speedup conversion of free space tree representations from/to
>>> bitmap/tree
>>>
>>> - code refactoring, deletion, cleanups
>>>- delayed refs
>>>- delayed iput
>>>- redundant argument removals
>>>- memory barrier cleanups
>>>- remove a redundant mutex supposedly excluding several ioctls to run
>>> in
>>> parallel
>>>
>>> - new tracepoints for blockgroup manipulation
>>>
>>> - more sanity checks of compressed headers
>>>
>>>
>>> The following changes since commit
>>> b04e217704b7f879c6b91222b066983a44a7a09f:
>>>
>>>Linux 4.17-rc7 (2018-05-27 13:01:47 -0700)
>>>
>>> are available in the Git repository at:
>>>
>>>git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git
>>> for-4.18-tag
>>>
>>> for you to fetch changes up to 23d0b79dfaed2305b500b0215b0421701ada6b1a:
>>>
>>>btrfs: Add unprivileged version of ino_lookup ioctl (2018-05-31
>>> 11:35:24 +0200)
>>>
>>>
>>> Al Viro (1):
>>>btrfs: take the last remnants of ->d_fsdata use out
>>>
>>> Anand Jain (19):
>>>btrfs: add comment about BTRFS_FS_EXCL_OP
>>>btrfs: rename struct btrfs_fs_devices::list
>>>btrfs: cleanup __btrfs_open_devices() drop head pointer
>>>btrfs: rename __btrfs_close_devices to close_fs_devices
>>>btrfs: rename __btrfs_open_devices to open_fs_devices
>>>btrfs: cleanup find_device() drop list_head pointer
>>>btrfs: cleanup btrfs_rm_device() promote fs_devices pointer
>>>btrfs: move btrfs_raid_type_names values to btrfs_raid_attr table
>>>btrfs: move btrfs_raid_group values to btrfs_raid_attr table
>>>btrfs: move btrfs_raid_mindev_errorvalues to btrfs_raid_attr table
>>>btrfs: reduce uuid_mutex critical section while scanning devices
>>>btrfs: use existing cur_devices, cleanup btrfs_rm_device
>>>btrfs: document uuid_mutex uasge in read_chunk_tree
>>>btrfs: replace uuid_mutex by device_list_mutex in
>>> btrfs_open_devices
>>
>>
>> This change (commit 542c5908abfe84f7b4c1717492ecc92ea0ea328d, "btrfs:
>> replace uuid_mutex by device_list_mutex in btrfs_open_devices"), at
>> the very least
>> introduces a lockdep warning:
>>
>> [ 865.021049] ==
>> [ 865.021950] WARNING: possible circular locking dependency detected
>> [ 865.022828] 4.17.0-rc7-btrfs-next-59+ #1 Not tainted
>> [ 865.023491] --
>> [ 865.024342] fsstress/27897 is trying to acquire lock:
>> [ 865.025070] 99260c12 (&fs_info->reloc_mutex){+.+.}, at:
>> btrfs_record_root_in_trans+0x43/0x62 [btrfs]
>> [ 865.026369]
>> [ 865.026369] but task is already holding lock:
>> [ 865.027206] 8dc17c22 (&mm->mmap_sem){}, at:
>> vm_mmap_pgoff+0x77/0xe8
>> [ 865.0
Re: [GIT PULL] Btrfs updates for 4.18
On 06/10/2018 12:21 AM, Filipe Manana wrote:
On Mon, Jun 4, 2018 at 4:43 PM, David Sterba wrote:
Hi,
there are some new features and a usual load of cleanups, more details below.
Specifically, there's a set of new non-privileged ioctls to allow
subvolume listing. It works but still needs a security review as it's a
new interface and we might need to do some tweaks to the data
structures. The fixes could be considred regressions but may touch the
interfaces too.
Currently there are no merge conflicts but linux-next has reported a few
in the past, originating from other *FS trees.
Please pull, thanks.
---
User visible features:
- added support for the ioctl FS_IOC_FSGETXATTR, per-inode flags, successor
of GET/SETFLAGS; now supports only existing flags: append, immutable,
noatime, nodump, sync
- 3 new unprivileged ioctls to allow users to enumerate subvolumes
- dedupe syscall implementation does not restrict the range to 16MiB, though it
still splits the whole range to 16MiB chunks
- on user demand, rmdir() is able to delete an empty subvolume, export the
capability in sysfs
- fix inode number types in tracepoints, other cleanups
- send: improved speed when dealing with a large removed directory,
measurements show decrease from 2000 minutes to 2 minutes on a directory with
2 million entries
- pre-commit check of superblock to detect a mysterious in-memory corruption
- log message updates
Other changes:
- orphan inode cleanup improved, does no keep long-standing reservations that
could lead up to early ENOSPC in some cases
- slight improvement of handling snapshotted NOCOW files by avoiding some
unnecessary tree searches
- avoid OOM when dealing with many unmergeable small extents at flush time
- speedup conversion of free space tree representations from/to bitmap/tree
- code refactoring, deletion, cleanups
- delayed refs
- delayed iput
- redundant argument removals
- memory barrier cleanups
- remove a redundant mutex supposedly excluding several ioctls to run in
parallel
- new tracepoints for blockgroup manipulation
- more sanity checks of compressed headers
The following changes since commit b04e217704b7f879c6b91222b066983a44a7a09f:
Linux 4.17-rc7 (2018-05-27 13:01:47 -0700)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-4.18-tag
for you to fetch changes up to 23d0b79dfaed2305b500b0215b0421701ada6b1a:
btrfs: Add unprivileged version of ino_lookup ioctl (2018-05-31 11:35:24
+0200)
Al Viro (1):
btrfs: take the last remnants of ->d_fsdata use out
Anand Jain (19):
btrfs: add comment about BTRFS_FS_EXCL_OP
btrfs: rename struct btrfs_fs_devices::list
btrfs: cleanup __btrfs_open_devices() drop head pointer
btrfs: rename __btrfs_close_devices to close_fs_devices
btrfs: rename __btrfs_open_devices to open_fs_devices
btrfs: cleanup find_device() drop list_head pointer
btrfs: cleanup btrfs_rm_device() promote fs_devices pointer
btrfs: move btrfs_raid_type_names values to btrfs_raid_attr table
btrfs: move btrfs_raid_group values to btrfs_raid_attr table
btrfs: move btrfs_raid_mindev_errorvalues to btrfs_raid_attr table
btrfs: reduce uuid_mutex critical section while scanning devices
btrfs: use existing cur_devices, cleanup btrfs_rm_device
btrfs: document uuid_mutex uasge in read_chunk_tree
btrfs: replace uuid_mutex by device_list_mutex in btrfs_open_devices
This change (commit 542c5908abfe84f7b4c1717492ecc92ea0ea328d, "btrfs:
replace uuid_mutex by device_list_mutex in btrfs_open_devices"), at
the very least
introduces a lockdep warning:
[ 865.021049] ==
[ 865.021950] WARNING: possible circular locking dependency detected
[ 865.022828] 4.17.0-rc7-btrfs-next-59+ #1 Not tainted
[ 865.023491] --
[ 865.024342] fsstress/27897 is trying to acquire lock:
[ 865.025070] 99260c12 (&fs_info->reloc_mutex){+.+.}, at:
btrfs_record_root_in_trans+0x43/0x62 [btrfs]
[ 865.026369]
[ 865.026369] but task is already holding lock:
[ 865.027206] 8dc17c22 (&mm->mmap_sem){}, at:
vm_mmap_pgoff+0x77/0xe8
[ 865.028251]
[ 865.028251] which lock already depends on the new lock.
[ 865.028251]
[ 865.029482]
[ 865.029482] the existing dependency chain (in reverse order) is:
[ 865.030523]
[ 865.030523] -> #7 (&mm->mmap_sem){}:
[ 865.031241]_copy_to_user+0x1e/0x63
[ 865.031745]filldir+0x9e/0xef
[ 865.032285]dir_emit_dots+0x3b/0xbd
[ 865.032881]dcache_readdir+0x22/0xbb
[ 865.033502]iterate_dir+0xa3/0x13e
[ 865.034131]__do_sys_getdents+0xa1/0x106
[ 865.034821]do_syscall_6
Re: [GIT PULL] Btrfs updates for 4.18
On Mon, Jun 4, 2018 at 4:43 PM, David Sterba wrote:
> Hi,
>
> there are some new features and a usual load of cleanups, more details below.
>
> Specifically, there's a set of new non-privileged ioctls to allow
> subvolume listing. It works but still needs a security review as it's a
> new interface and we might need to do some tweaks to the data
> structures. The fixes could be considred regressions but may touch the
> interfaces too.
>
> Currently there are no merge conflicts but linux-next has reported a few
> in the past, originating from other *FS trees.
>
> Please pull, thanks.
>
> ---
>
> User visible features:
>
> - added support for the ioctl FS_IOC_FSGETXATTR, per-inode flags, successor
> of GET/SETFLAGS; now supports only existing flags: append, immutable,
> noatime, nodump, sync
>
> - 3 new unprivileged ioctls to allow users to enumerate subvolumes
>
> - dedupe syscall implementation does not restrict the range to 16MiB, though
> it
> still splits the whole range to 16MiB chunks
>
> - on user demand, rmdir() is able to delete an empty subvolume, export the
> capability in sysfs
>
> - fix inode number types in tracepoints, other cleanups
>
> - send: improved speed when dealing with a large removed directory,
> measurements show decrease from 2000 minutes to 2 minutes on a directory
> with
> 2 million entries
>
> - pre-commit check of superblock to detect a mysterious in-memory corruption
>
> - log message updates
>
>
> Other changes:
>
> - orphan inode cleanup improved, does no keep long-standing reservations that
> could lead up to early ENOSPC in some cases
>
> - slight improvement of handling snapshotted NOCOW files by avoiding some
> unnecessary tree searches
>
> - avoid OOM when dealing with many unmergeable small extents at flush time
>
> - speedup conversion of free space tree representations from/to bitmap/tree
>
> - code refactoring, deletion, cleanups
> - delayed refs
> - delayed iput
> - redundant argument removals
> - memory barrier cleanups
> - remove a redundant mutex supposedly excluding several ioctls to run in
> parallel
>
> - new tracepoints for blockgroup manipulation
>
> - more sanity checks of compressed headers
>
>
> The following changes since commit b04e217704b7f879c6b91222b066983a44a7a09f:
>
> Linux 4.17-rc7 (2018-05-27 13:01:47 -0700)
>
> are available in the Git repository at:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-4.18-tag
>
> for you to fetch changes up to 23d0b79dfaed2305b500b0215b0421701ada6b1a:
>
> btrfs: Add unprivileged version of ino_lookup ioctl (2018-05-31 11:35:24
> +0200)
>
>
> Al Viro (1):
> btrfs: take the last remnants of ->d_fsdata use out
>
> Anand Jain (19):
> btrfs: add comment about BTRFS_FS_EXCL_OP
> btrfs: rename struct btrfs_fs_devices::list
> btrfs: cleanup __btrfs_open_devices() drop head pointer
> btrfs: rename __btrfs_close_devices to close_fs_devices
> btrfs: rename __btrfs_open_devices to open_fs_devices
> btrfs: cleanup find_device() drop list_head pointer
> btrfs: cleanup btrfs_rm_device() promote fs_devices pointer
> btrfs: move btrfs_raid_type_names values to btrfs_raid_attr table
> btrfs: move btrfs_raid_group values to btrfs_raid_attr table
> btrfs: move btrfs_raid_mindev_errorvalues to btrfs_raid_attr table
> btrfs: reduce uuid_mutex critical section while scanning devices
> btrfs: use existing cur_devices, cleanup btrfs_rm_device
> btrfs: document uuid_mutex uasge in read_chunk_tree
> btrfs: replace uuid_mutex by device_list_mutex in btrfs_open_devices
This change (commit 542c5908abfe84f7b4c1717492ecc92ea0ea328d, "btrfs:
replace uuid_mutex by device_list_mutex in btrfs_open_devices"), at
the very least
introduces a lockdep warning:
[ 865.021049] ==
[ 865.021950] WARNING: possible circular locking dependency detected
[ 865.022828] 4.17.0-rc7-btrfs-next-59+ #1 Not tainted
[ 865.023491] --
[ 865.024342] fsstress/27897 is trying to acquire lock:
[ 865.025070] 99260c12 (&fs_info->reloc_mutex){+.+.}, at:
btrfs_record_root_in_trans+0x43/0x62 [btrfs]
[ 865.026369]
[ 865.026369] but task is already holding lock:
[ 865.027206] 8dc17c22 (&mm->mmap_sem){}, at:
vm_mmap_pgoff+0x77/0xe8
[ 865.028251]
[ 865.028251] which lock already depends on the new lock.
[ 865.028251]
[ 865.029482]
[ 865.029482] the existing dependency chain (in reverse order) is:
[ 865.030523]
[ 865.030523] -> #7 (&mm->mmap_sem){}:
[ 865.031241]_copy_to_user+0x1e/0x63
[ 865.031745]filldir+0x9e/0xef
[ 865.032285]dir_emit_dots+0x3b/0xbd
[ 865.032881]dcache_readdir+0x22/0xbb
[ 865.033502]iterate_dir+0xa3/0x
[GIT PULL] Btrfs updates for 4.17-rc3
Hi, please pull the following branch with 2 regression fixes and one fix for stable. Thanks. The following changes since commit c0872323746e11fc79344e3738b283a8cda86654: btrfs: print-tree: debugging output enhancement (2018-04-20 19:18:16 +0200) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-4.17-rc3-tag for you to fetch changes up to a6aa10c70bf72fb28504cb5de5deac75da78b0f5: Btrfs: send, fix missing truncate for inode with prealloc extent past eof (2018-05-02 11:55:29 +0200) Filipe Manana (1): Btrfs: send, fix missing truncate for inode with prealloc extent past eof Qu Wenruo (1): btrfs: Fix wrong first_key parameter in replace_path ethanwu (1): btrfs: Take trans lock before access running trans in check_delayed_ref fs/btrfs/extent-tree.c | 7 +++ fs/btrfs/relocation.c | 2 +- fs/btrfs/send.c| 4 3 files changed, 12 insertions(+), 1 deletion(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates for 4.17, part 2
Hi, we have queued a few more fixes (error handling, log replay, softlockup) and the rest is SPDX update that touches almost all files so the diffstat is long. The top patch is a fixup for excessive warning and was not in linux-next but I've tested it locally. Please pull, thanks. The following changes since commit 57599c7e7722daf5f8c2dba4b0e4628f5c500771: btrfs: lift errors from add_extent_changeset to the callers (2018-03-31 02:03:25 +0200) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-4.17-part2-tag for you to fetch changes up to 5d41be6f702f19f72db816c17175caf9dbdcdfa6: btrfs: Only check first key for committed tree blocks (2018-04-13 16:16:15 +0200) David Sterba (3): btrfs: replace GPL boilerplate by SPDX -- headers btrfs: replace GPL boilerplate by SPDX -- sources btrfs: add SPDX header to Kconfig Filipe Manana (1): Btrfs: fix loss of prealloc extents past i_size after fsync log replay Liu Bo (3): Btrfs: fix NULL pointer dereference in log_dir_items Btrfs: bail out on error during replay_dir_deletes Btrfs: clean up resources during umount after trans is aborted Nikolay Borisov (1): btrfs: Fix possible softlock on single core machines Qu Wenruo (1): btrfs: Only check first key for committed tree blocks fs/btrfs/Kconfig | 2 + fs/btrfs/acl.c | 15 +- fs/btrfs/async-thread.c| 15 +- fs/btrfs/async-thread.h| 21 ++-- fs/btrfs/backref.c | 15 +- fs/btrfs/backref.h | 19 ++-- fs/btrfs/btrfs_inode.h | 19 ++-- fs/btrfs/check-integrity.c | 15 +- fs/btrfs/check-integrity.h | 19 ++-- fs/btrfs/compression.c | 15 +- fs/btrfs/compression.h | 19 ++-- fs/btrfs/ctree.c | 15 +- fs/btrfs/ctree.h | 20 ++-- fs/btrfs/dedupe.h | 20 ++-- fs/btrfs/delayed-inode.c | 15 +- fs/btrfs/delayed-inode.h | 19 ++-- fs/btrfs/delayed-ref.c | 15 +- fs/btrfs/delayed-ref.h | 21 ++-- fs/btrfs/dev-replace.c | 16 +- fs/btrfs/dev-replace.h | 20 ++-- fs/btrfs/dir-item.c| 15 +- fs/btrfs/disk-io.c | 26 +- fs/btrfs/disk-io.h | 20 ++-- fs/btrfs/export.c | 1 + fs/btrfs/export.h | 1 + fs/btrfs/extent-tree.c | 17 ++- fs/btrfs/extent_io.c | 1 + fs/btrfs/extent_io.h | 6 ++- fs/btrfs/extent_map.c | 1 + fs/btrfs/extent_map.h | 6 ++- fs/btrfs/file-item.c | 15 +- fs/btrfs/file.c| 15 +- fs/btrfs/free-space-cache.c| 15 +- fs/btrfs/free-space-cache.h| 19 ++-- fs/btrfs/free-space-tree.c | 15 +- fs/btrfs/free-space-tree.h | 19 ++-- fs/btrfs/inode-item.c | 15 +- fs/btrfs/inode-map.c | 15 +- fs/btrfs/inode-map.h | 5 +- fs/btrfs/inode.c | 15 +- fs/btrfs/ioctl.c | 15 +- fs/btrfs/locking.c | 16 +- fs/btrfs/locking.h | 19 ++-- fs/btrfs/lzo.c | 15 +- fs/btrfs/math.h| 20 ++-- fs/btrfs/ordered-data.c| 15 +- fs/btrfs/ordered-data.h| 20 ++-- fs/btrfs/orphan.c | 15 +- fs/btrfs/print-tree.c | 15 +- fs/btrfs/print-tree.h | 21 ++-- fs/btrfs/props.c | 15 +- fs/btrfs/props.h | 19 ++-- fs/btrfs/qgroup.c | 15 +- fs/btrfs/qgroup.h | 22 ++--- fs/btrfs/raid56.c | 16 +- fs/btrfs/raid56.h | 21 ++-- fs/btrfs/rcu-string.h | 20 +++- fs/btrfs/reada.c | 15 +- fs/btrfs/ref-verify.c | 15 +- fs/btrfs/ref-verify.h | 23 +++-- fs/btrfs/relocation.c | 15 +- fs/btrfs/root-tree.c | 15 +- fs/btrfs/scrub.c | 15 +- fs/btrfs/send.c| 15 +- fs/btrfs/send.h| 20 +++- fs/btrfs/struct-funcs.c| 15 +- fs/btrfs/super.c | 15 +- fs
[GIT PULL] Btrfs updates for 4.17
Hi, please pull the following btrfs changes. There are a several user visible changes, the rest is mostly invisible and continues to clean up the whole code base. There are no merge conflicts with current master. Please pull, thanks. User visible changes: - new mount option nossd_spread (pair for ssd_spread) - mount option subvolid will detect junk after the number and fail the mount - add message after cancelled device replace - direct module dependency on libcrc32, removed own crc wrappers - removed user space transaction ioctls - use lighter locking when reading /proc/self/mounts, RCU instead of mutex to avoid unnecessary contention Enhancements: - skip writeback of last page when truncating file to same size - send: do not issue unnecessary truncate operations - mount option token specifiers: use %u for unsigned values, more validation - selftests: more tree block validations qgroups: - preparatory work for splitting reservation types for data and metadata, this should allow for more accurate tracking and fix some issues with underflows or do further enhancements - split metadata reservations for started and joined transaction so they do not get mixed up and are accounted correctly at commit time - with the above, it's possible to revert patch that potentially deadlocks when trying to make more space by explicitly committing when the quota limit is hit - fix root item corruption when multiple same source snapshots are created with quota enabled RAID56: - make sure target is identical to source when raid56 rebuild fails after dev-replace - faster rebuild during scrub, batch by stripes and not block-by-block - make more use of cached data when rebuilding from a missing device Fixes: - null pointer deref when device replace target is missing - fix fsync after hole punching when using no-holes feature - fix lockdep splat when allocating percpu data with wrong GFP flags Cleanups, refactoring, core changes: - drop redunant parameters from various functions - kill and opencode trivial helpers - __cold/__exit function annotations - dead code removal - continued audit and documentation of memory barriers - error handling: handle removal from uuid tree - error handling: remove handling of impossible condtitons - more debugging or error messages - updated tracepoints - 1 VLA use removal (1 still left) The following changes since commit 3eb2ce825ea1ad89d20f7a3b5780df850e4be274: Linux 4.16-rc7 (2018-03-25 12:44:30 -1000) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-4.17-tag for you to fetch changes up to 57599c7e7722daf5f8c2dba4b0e4628f5c500771: btrfs: lift errors from add_extent_changeset to the callers (2018-03-31 02:03:25 +0200) Anand Jain (25): btrfs: open code btrfs_dev_replace_cancel() btrfs: rename __btrfs_dev_replace_cancel() btrfs: btrfs_dev_replace_cancel() can return int btrfs: open code btrfs_init_dev_replace_tgtdev_for_resume() btrfs: extent_buffer_uptodate() make it static and inline btrfs: manage thread_pool mount option as %u btrfs: manage metadata_ratio mount option as %u btrfs: manage check_int_print_mask mount option as %u btrfs: manage commit mount option as %u btrfs: add a comment to mark the deprecated mount option btrfs: fix null pointer deref when target device is missing btrfs: log, when replace, is canceled by the user btrfs: remove unused function btrfs_async_submit_limit() btrfs: cow_file_range() num_bytes and disk_num_bytes are same btrfs: use ASSERT to report logical error in cow_file_range() btrfs: not a disk error if the bio_add_page fails btrfs: keep device list sorted btrfs: insert newly opened device to the end of the list btrfs: verify subvolid mount parameter btrfs: remove assert in btrfs_init_dev_replace_tgtdev() btrfs: unify types for metadata_ratio and data_chunk_allocations btrfs: rename btrfs_close_extra_device to btrfs_free_extra_devids btrfs: add define for oldest generation btrfs: drop num argument from find_live_mirror() btrfs: drop optimal argument from find_live_mirror() Colin Ian King (2): btrfs: remove redundant check on ret and goto Btrfs: extent map selftest: add missing void parameter to btrfs_test_extent_map David Sterba (37): btrfs: add (the only possible) __exit annotation btrfs: add more __cold annotations btrfs: drop underscores from exported xattr functions btrfs: drop extern from function declarations btrfs: adjust return type of btrfs_getxattr btrfs: move btrfs_listxattr prototype to xattr.h btrfs: open code trivial helper
[GIT PULL] Btrfs updates
Hi Linus, Please pull my for-linus branch: git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus Miao Xie has been very busy, fixing races and enospc problems and many other small but important pieces. Alexandre Oliva discovered some problems with how our error handling was interacting with the block layer and for now has disabled our partial handling of sub-page writes. The real sub-page work is in a series of patches from IBM that we still need to integrate and test. The code Alexandre has turned off was really incomplete. Josef has more error handling fixes and an important fix for the new skinny extent format. This also has my fix for the tracepoint crash from late in 3.9. It's the first stage in a larger clean up to get rid of btrfs_bio and make a proper bioset for all the items we need to tack into the bio. For now the bioset only holds our mirror_num and stripe_index, but for the next merge window I'll shuffle more in. Miao Xie (10) commits (+87/-69): Btrfs: don't steal the reserved space from the global reserve if their space type is different (+4/-2) Btrfs: don't abort the current transaction if there is no enough space for inode cache (+2/-1) Btrfs: don't invoke btrfs_invalidate_inodes() in the spin lock context (+6/-0) Btrfs: don't use global block reservation for inode cache truncation (+34/-22) Btrfs: fix unprotected root node of the subvolume's inode rb-tree (+3/-4) Btrfs: remove BUG_ON() in btrfs_read_fs_tree_no_radix() (+0/-1) Btrfs: pause the space balance when remounting to R/O (+1/-0) Btrfs: optimize the error handle of use_block_rsv() (+28/-37) Btrfs: update the global reserve if it is empty (+8/-1) Btrfs: fix accessing a freed tree root (+1/-1) Josef Bacik (4) commits (+35/-32): Btrfs: make sure roots are assigned before freeing their nodes (+21/-18) Btrfs: handle running extent ops with skinny metadata (+12/-10) Btrfs: remove warn on in free space cache writeout (+1/-3) Btrfs: don't null pointer deref on abort (+1/-1) Stefan Behrens (3) commits (+8/-1): Btrfs: explicitly use global_block_rsv for quota_tree (+2/-0) Btrfs: fix possible memory leak in replace_path() (+1/-1) Btrfs: don't allow device replace on RAID5/RAID6 (+5/-0) Liu Bo (2) commits (+8/-4): Btrfs: return errno if possible when we fail to allocate memory (+6/-2) Btrfs: fix off-by-one in fiemap (+2/-2) Gabriel de Perthuis (1) commits (+5/-5): btrfs: don't stop searching after encountering the wrong item Alexandre Oliva (1) commits (+30/-55): btrfs: do away with non-whole_page extent I/O Chris Mason (1) commits (+120/-72): Btrfs: use a btrfs bioset instead of abusing bio internals David Sterba (1) commits (+4/-4): btrfs: annotate quota tree for lockdep Wang Shilong (1) commits (+2/-1): Btrfs: fix possible memory leak in the find_parent_nodes() Andreas Philipp (1) commits (+6/-7): Correct allowed raid levels on balance. Total: (25) commits (+305/-250) fs/btrfs/backref.c | 3 +- fs/btrfs/check-integrity.c | 2 +- fs/btrfs/ctree.c| 4 +- fs/btrfs/ctree.h| 8 +-- fs/btrfs/delayed-ref.h | 1 + fs/btrfs/dev-replace.c | 5 ++ fs/btrfs/disk-io.c | 52 ++--- fs/btrfs/extent-tree.c | 94 -- fs/btrfs/extent_io.c| 138 +++- fs/btrfs/extent_io.h| 2 + fs/btrfs/free-space-cache.c | 43 +++--- fs/btrfs/free-space-cache.h | 2 + fs/btrfs/inode-map.c| 8 ++- fs/btrfs/inode.c| 81 +- fs/btrfs/ioctl.c| 10 ++-- fs/btrfs/raid56.c | 2 +- fs/btrfs/relocation.c | 7 ++- fs/btrfs/scrub.c| 10 ++-- fs/btrfs/super.c| 1 + fs/btrfs/volumes.c | 54 - fs/btrfs/volumes.h | 20 +++ 21 files changed, 301 insertions(+), 246 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi Linus, Please pull my for-linus branch: git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus We've had a busy two weeks of bug fixing. The biggest patches in here are some long standing early-enospc problems (Josef) and a very old race where compression and mmap combine forces to lose writes (me). I'm fairly sure the mmap bug goes all the way back to the introduction of the compression code, which is proof that fsx doesn't trigger every possible mmap corner after all. I'm sure you'll notice one of these is from this morning, it's a small and isolated use-after-free fix in our scrub error reporting. I double checked it here. Josef Bacik (6) commits (+90/-18): Btrfs: hold the ordered operations mutex when waiting on ordered extents (+2/-0) Btrfs: don't drop path when printing out tree errors in scrub (+2/-1) Btrfs: fix space leak when we fail to reserve metadata space (+41/-6) Btrfs: fix space accounting for unlink and rename (+2/-4) Btrfs: limit the global reserve to 512mb (+1/-1) Btrfs: handle a bogus chunk tree nicely (+42/-6) Jan Schmidt (2) commits (+24/-16): Btrfs: fix EIO from btrfs send in is_extent_unchanged for punched holes (+4/-6) Btrfs: fix locking on ROOT_REPLACE operations in tree mod log (+20/-10) Wang Shilong (2) commits (+10/-2): Btrfs: fix double free in the btrfs_qgroup_account_ref() (+1/-2) Btrfs: fix missing qgroup reservation before fallocating (+9/-0) Miao Xie (2) commits (+5/-3): Btrfs: fix wrong return value of btrfs_lookup_csum() (+3/-1) Btrfs: fix wrong reservation of csums (+2/-2) Chris Mason (1) commits (+49/-0): Btrfs: fix race between mmap writes and compression Liu Bo (1) commits (+1/-1): Btrfs: update to use fs_state bit Tsutomu Itoh (1) commits (+9/-3): Btrfs: fix memory leak in btrfs_create_tree() Total: (15) commits fs/btrfs/ctree.c| 30 -- fs/btrfs/disk-io.c | 14 ++--- fs/btrfs/extent-tree.c | 84 ++--- fs/btrfs/extent_io.c| 33 +++ fs/btrfs/extent_io.h| 2 ++ fs/btrfs/file-item.c| 6 ++-- fs/btrfs/file.c | 9 ++ fs/btrfs/inode.c| 22 ++--- fs/btrfs/ordered-data.c | 2 ++ fs/btrfs/qgroup.c | 3 +- fs/btrfs/scrub.c| 3 +- fs/btrfs/send.c | 10 +++--- fs/btrfs/volumes.c | 13 +++- 13 files changed, 188 insertions(+), 43 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi Linus, Please grab my for-linus: git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus These are scattered fixes and one performance improvement. The biggest functional change is in how we throttle metadata changes. The new code bumps our average file creation rate up by ~13% in fs_mark, and lowers CPU usage. Stefan bisected out a regression in our allocation code that made balance loop on extents larger than 256MB. Liu Bo (6) commits (+71/-19): Btrfs: build up error handling for merge_reloc_roots (+35/-12) Btrfs: check for NULL pointer in updating reloc roots (+2/-0) Btrfs: avoid deadlock on transaction waiting list (+7/-0) Btrfs: free all recorded tree blocks on error (+6/-3) Btrfs: do not BUG_ON on aborted situation (+12/-3) Btrfs: do not BUG_ON in prepare_to_reloc (+9/-1) Chris Mason (2) commits (+96/-63): Btrfs: enforce min_bytes parameter during extent allocation (+4/-2) Btrfs: improve the delayed inode throttling (+92/-61) Miao Xie (2) commits (+45/-39): Btrfs: fix unclosed transaction handler when the async transaction commitment fails (+4/-0) Btrfs: fix wrong handle at error path of create_snapshot() when the commit fails (+41/-39) Stefan Behrens (1) commits (+0/-8): Btrfs: allow running defrag in parallel to administrative tasks Ilya Dryomov (1) commits (+5/-0): Btrfs: fix a mismerge in btrfs_balance() Josef Bacik (1) commits (+4/-1): Btrfs: use set_nlink if our i_nlink is 0 Total: (13) commits (+221/-130) fs/btrfs/delayed-inode.c | 151 --- fs/btrfs/delayed-inode.h | 2 + fs/btrfs/disk-io.c | 16 +++-- fs/btrfs/inode.c | 6 +- fs/btrfs/ioctl.c | 18 ++ fs/btrfs/relocation.c| 74 +-- fs/btrfs/transaction.c | 65 fs/btrfs/tree-log.c | 5 +- fs/btrfs/volumes.c | 14 - 9 files changed, 221 insertions(+), 130 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi Linus, If you're doing another RC, please grab these two. Otherwise I'll send them off to -stable. git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus This fixes a long standing problem where the btrfs scan ioctl was racing with mkfs.btrfs and dropping dirty pages created by mkfs. It also fixes a crash during tree log replay with quota enabled. David Sterba (1) commits (+64/-6): btrfs: access superblock via pagecache in scan_one_device Arne Jansen (1) commits (+1/-1): Btrfs: fix crash in log replay with qgroups enabled Total: (2) commits (+65/-7) fs/btrfs/ctree.c | 2 +- fs/btrfs/volumes.c | 70 +- 2 files changed, 65 insertions(+), 7 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
On Wed, Dec 19, 2012 at 07:09:57PM +0100, Roy Sigurd Karlsbakk wrote: > > [ sorry, resend. My lbdb autocompleted with an extra r in kernel.org ] > > > > Hi everyone, > > > > My for-linus branch has a big set of fixes and features: > > Does this include raid-[56]? As Chris's last sentence in that mail says, not in this pull request, but it should be ready for Friday. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- We demand rigidly defined areas of doubt and uncertainty! --- signature.asc Description: Digital signature
Re: [GIT PULL] Btrfs updates
> [ sorry, resend. My lbdb autocompleted with an extra r in kernel.org ] > > Hi everyone, > > My for-linus branch has a big set of fixes and features: Does this include raid-[56]? Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 98013356 r...@karlsbakk.net http://blogg.karlsbakk.net/ GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
[ sorry, resend. My lbdb autocompleted with an extra r in kernel.org ]
Hi everyone,
My for-linus branch has a big set of fixes and features:
git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus
This was against 3.7, and it has two easy conflicts against Linus'
current head. My -next branch has the resolutions, but Linus wills
surely fix these himself.
In terms of line count, most of the code comes from Stefan, who added
the ability to replace a single drive in place. This is different from
how btrfs normally replaces drives, and is much much much faster.
Josef is plowing through our synchronous write performance. This pull
request does not include the DIO_OWN_WAITING patch that was discussed on
the list, but it has a number of other improvements to cut down our
latencies and CPU time during fsync/O_DIRECT writes.
Miao Xie has a big series of fixes and is spreading out ordered
operations over more CPUs. This improves performance and reduces
contention.
I've put in fixes for error handling around hash collisions. These are
going back to individual stable kernels as I test against them.
Otherwise we have a lot of fixes and cleanups, thanks everyone! raid5/6
is being rebased against the device replacement code. I'll have it
posted this Friday along with a nice series of benchmarks.
Stefan Behrens (33) commits (+3446/-831):
Btrfs: change core code of btrfs to support the device replace operations
(+111/-14)
Btrfs: allow repair code to include target disk when searching mirrors
(+154/-5)
Btrfs: changes to live filesystem are also written to replacement disk
(+49/-1)
Btrfs: introduce GET_READ_MIRRORS functionality for btrfs_map_block()
(+11/-7)
Btrfs: remove the block device pointer from the scrub context struct
(+73/-60)
Btrfs: disallow some operations on the device replace target device
(+54/-18)
Btrfs: disallow mutually exclusive admin operations from user mode (+40/-17)
Btrfs: Pass fs_info to btrfs_num_copies() instead of mapping_tree (+15/-15)
Btrfs: fix BUG() in scrub when first superblock reading gives EIO (+11/-0)
Btrfs: pass fs_info to btrfs_map_block() instead of mapping_tree (+32/-37)
Btrfs: Don't trust the superblock label and simply printk("%s") it (+5/-2)
Btrfs: don't allow degraded mount if too many devices are missing (+16/-0)
Btrfs: optionally avoid reads from device replace source drive (+35/-11)
Btrfs: fix race in check-integrity caused by usage of bitfield (+1/-1)
Btrfs: in scrub repair code, optimize the reading of mirrors (+12/-23)
Btrfs: add code to scrub to copy read data to another disk (+851/-73)
Btrfs: in scrub repair code, simplify alloc error handling (+26/-35)
Btrfs: enhance btrfs structures for device replace support (+44/-0)
Btrfs: make the scrub page array dynamically allocated (+121/-74)
Btrfs: increase BTRFS_MAX_MIRRORS by one for dev replace (+1/-1)
Btrfs: handle errors from btrfs_map_bio() everywhere (+65/-33)
Btrfs: avoid risk of a deadlock in btrfs_handle_error (+10/-1)
Btrfs: fix a scrub regression in case of write errors (+2/-1)
Btrfs: add new sources for device replace code (+1069/-1)
Btrfs: move some common code into a subfunction (+50/-43)
Btrfs: introduce a btrfs_dev_replace_item type (+69/-0)
Btrfs: add btrfs_scratch_superblock() function (+19/-0)
Btrfs: cleanup scrub bio and worker wait code (+71/-35)
Btrfs: fix a build warning for an unused label (+0/-1)
Btrfs: add support for device replace ioctls (+52/-3)
Btrfs: rename the scrub context structure (+253/-253)
Btrfs: add two more find_device() methods (+64/-0)
Btrfs: pass fs_info instead of root (+60/-66)
Miao Xie (32) commits (+840/-471):
Btrfs: fix missing reserved space release in error path of delalloc
reservation (+7/-0)
Btrfs: fix off-by-one error of the same page check in btrfs_punch_hole()
(+2/-2)
Btrfs: fix unprotected extent map operation when logging file extents
(+2/-0)
Btrfs: fix unnecessary while loop when search the free space, cache
(+10/-20)
Btrfs: fix off-by-one error of the reserved size of btrfs_allocate() (+2/-2)
Btrfs: fix joining the same transaction handler more than 2 times (+48/-30)
Btrfs: fix missing log when BTRFS_INODE_NEEDS_FULL_SYNC is set (+4/-1)
Btrfs: pass root object into btrfs_ioctl_{start, wait}_sync() (+6/-6)
Btrfs: get write access when setting the default subvolume (+28/-12)
Btrfs: make ordered operations be handled by multi-task (+45/-21)
Btrfs: fix wrong return value of btrfs_wait_for_commit() (+8/-7)
Btrfs: don't start a new transaction when starting sync (+18/-9)
Btrfs: fix missing flush when committing a transaction (+47/-35)
Btrfs: fix wrong return value of btrfs_truncate_page() (+1/-2)
Btrfs: make delalloc inodes be flushed by multi-task (+103/-8)
Btrfs: use existing align macros in btrfs_allocate() (+4/-4)
Btrfs: make o
[GIT PULL] Btrfs updates
Hi everyone,
My for-linus branch has a big set of fixes and features:
git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus
This was against 3.7, and it has two easy conflicts against Linus'
current head. My -next branch has the resolutions, but Linus wills
surely fix these himself.
In terms of line count, most of the code comes from Stefan, who added
the ability to replace a single drive in place. This is different from
how btrfs normally replaces drives, and is much much much faster.
Josef is plowing through our synchronous write performance. This pull
request does not include the DIO_OWN_WAITING patch that was discussed on
the list, but it has a number of other improvements to cut down our
latencies and CPU time during fsync/O_DIRECT writes.
Miao Xie has a big series of fixes and is spreading out ordered
operations over more CPUs. This improves performance and reduces
contention.
I've put in fixes for error handling around hash collisions. These are
going back to individual stable kernels as I test against them.
Otherwise we have a lot of fixes and cleanups, thanks everyone! raid5/6
is being rebased against the device replacement code. I'll have it
posted this Friday along with a nice series of benchmarks.
Stefan Behrens (33) commits (+3446/-831):
Btrfs: change core code of btrfs to support the device replace operations
(+111/-14)
Btrfs: allow repair code to include target disk when searching mirrors
(+154/-5)
Btrfs: changes to live filesystem are also written to replacement disk
(+49/-1)
Btrfs: introduce GET_READ_MIRRORS functionality for btrfs_map_block()
(+11/-7)
Btrfs: remove the block device pointer from the scrub context struct
(+73/-60)
Btrfs: disallow some operations on the device replace target device
(+54/-18)
Btrfs: disallow mutually exclusive admin operations from user mode (+40/-17)
Btrfs: Pass fs_info to btrfs_num_copies() instead of mapping_tree (+15/-15)
Btrfs: fix BUG() in scrub when first superblock reading gives EIO (+11/-0)
Btrfs: pass fs_info to btrfs_map_block() instead of mapping_tree (+32/-37)
Btrfs: Don't trust the superblock label and simply printk("%s") it (+5/-2)
Btrfs: don't allow degraded mount if too many devices are missing (+16/-0)
Btrfs: optionally avoid reads from device replace source drive (+35/-11)
Btrfs: fix race in check-integrity caused by usage of bitfield (+1/-1)
Btrfs: in scrub repair code, optimize the reading of mirrors (+12/-23)
Btrfs: add code to scrub to copy read data to another disk (+851/-73)
Btrfs: in scrub repair code, simplify alloc error handling (+26/-35)
Btrfs: enhance btrfs structures for device replace support (+44/-0)
Btrfs: make the scrub page array dynamically allocated (+121/-74)
Btrfs: increase BTRFS_MAX_MIRRORS by one for dev replace (+1/-1)
Btrfs: handle errors from btrfs_map_bio() everywhere (+65/-33)
Btrfs: avoid risk of a deadlock in btrfs_handle_error (+10/-1)
Btrfs: fix a scrub regression in case of write errors (+2/-1)
Btrfs: add new sources for device replace code (+1069/-1)
Btrfs: move some common code into a subfunction (+50/-43)
Btrfs: introduce a btrfs_dev_replace_item type (+69/-0)
Btrfs: add btrfs_scratch_superblock() function (+19/-0)
Btrfs: cleanup scrub bio and worker wait code (+71/-35)
Btrfs: fix a build warning for an unused label (+0/-1)
Btrfs: add support for device replace ioctls (+52/-3)
Btrfs: rename the scrub context structure (+253/-253)
Btrfs: add two more find_device() methods (+64/-0)
Btrfs: pass fs_info instead of root (+60/-66)
Miao Xie (32) commits (+840/-471):
Btrfs: fix missing reserved space release in error path of delalloc
reservation (+7/-0)
Btrfs: fix off-by-one error of the same page check in btrfs_punch_hole()
(+2/-2)
Btrfs: fix unprotected extent map operation when logging file extents
(+2/-0)
Btrfs: fix unnecessary while loop when search the free space, cache
(+10/-20)
Btrfs: fix off-by-one error of the reserved size of btrfs_allocate() (+2/-2)
Btrfs: fix joining the same transaction handler more than 2 times (+48/-30)
Btrfs: fix missing log when BTRFS_INODE_NEEDS_FULL_SYNC is set (+4/-1)
Btrfs: pass root object into btrfs_ioctl_{start, wait}_sync() (+6/-6)
Btrfs: get write access when setting the default subvolume (+28/-12)
Btrfs: make ordered operations be handled by multi-task (+45/-21)
Btrfs: fix wrong return value of btrfs_wait_for_commit() (+8/-7)
Btrfs: don't start a new transaction when starting sync (+18/-9)
Btrfs: fix missing flush when committing a transaction (+47/-35)
Btrfs: fix wrong return value of btrfs_truncate_page() (+1/-2)
Btrfs: make delalloc inodes be flushed by multi-task (+103/-8)
Btrfs: use existing align macros in btrfs_allocate() (+4/-4)
Btrfs: make ordered extent be flushed by multi-task (+37/-9)
Btrfs: don't auto def
[GIT PULL] Btrfs updates
Hi Linus, I've split out the big send/receive update from my last pull request and now have just the fixes in my for-linus branch: git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus For anyone who wants send/receive updates, they are maintained as well. But it is has enough cleanups (without fixes) that we shouldn't be asking Linus to take it right now. The send/recv branch will wander over to linux-next shortly though. git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git send-recv The largest patches in this pull are Josef's patches to fix DIO locking problems and his patch to fix a crash during balance. They are both well tested. The rest are smaller fixes that we've had queued. The last rc came out while I was hacking new and exciting ways to recover from a misplaced rm -rf on my dev box, so these missed rc3. Josef Bacik (9) commits (+322/-216): Btrfs: don't allocate a seperate csums array for direct reads (+19/-32) Btrfs: do not use missing devices when showing devname (+2/-0) Btrfs: fix enospc problems when deleting a subvol (+1/-1) Btrfs: increase the size of the free space cache (+7/-8) Btrfs: lock extents as we map them in DIO (+127/-129) Btrfs: fix deadlock with freeze and sync V2 (+9/-4) Btrfs: allow delayed refs to be merged (+142/-27) Btrfs: do not strdup non existent strings (+5/-3) Btrfs: barrier before waitqueue_active (+10/-12) Stefan Behrens (5) commits (+16/-77): Btrfs: fix that repair code is spuriously executed for transid failures (+6/-2) Btrfs: revert checksum error statistic which can cause a BUG() (+2/-39) Btrfs: fix a misplaced address operator in a condition (+1/-1) Btrfs: remove superblock writing after fatal error (+5/-33) Btrfs: fix that error value is changed by mistake (+2/-2) Dan Carpenter (4) commits (+16/-8): Btrfs: unlock on error in btrfs_delalloc_reserve_metadata() (+3/-1) Btrfs: fix some error codes in btrfs_qgroup_inherit() (+6/-2) Btrfs: fix some endian bugs handling the root times (+4/-4) Btrfs: checking for NULL instead of IS_ERR (+3/-1) Liu Bo (2) commits (+25/-6): Btrfs: fix ordered extent leak when failing to start a transaction (+5/-2) Btrfs: fix a dio write regression (+20/-4) Arne Jansen (2) commits (+38/-73): Btrfs: fix deadlock in wait_for_more_refs (+21/-73) Btrfs: fix race in run_clustered_refs (+17/-0) Chris Mason (1) commits (+3/-0): Btrfs: don't run __tree_mod_log_free_eb on leaves Fengguang Wu (1) commits (+3/-2): btrfs: fix second lock in btrfs_delete_delayed_items() Miao Xie (1) commits (+1/-0): Btrfs: fix wrong mtime and ctime when creating snapshots Total: (25) commits (+424/-382) fs/btrfs/backref.c | 4 +- fs/btrfs/compression.c | 1 + fs/btrfs/ctree.c | 9 +- fs/btrfs/ctree.h | 3 +- fs/btrfs/delayed-inode.c | 12 +- fs/btrfs/delayed-ref.c | 163 +++- fs/btrfs/delayed-ref.h | 4 + fs/btrfs/disk-io.c | 53 ++-- fs/btrfs/disk-io.h | 2 +- fs/btrfs/extent-tree.c | 123 +- fs/btrfs/extent_io.c | 17 +-- fs/btrfs/file-item.c | 4 +- fs/btrfs/inode.c | 326 --- fs/btrfs/ioctl.c | 2 +- fs/btrfs/locking.c | 2 +- fs/btrfs/qgroup.c| 12 +- fs/btrfs/root-tree.c | 4 +- fs/btrfs/super.c | 15 ++- fs/btrfs/transaction.c | 3 +- fs/btrfs/volumes.c | 33 + fs/btrfs/volumes.h | 2 - 21 files changed, 418 insertions(+), 376 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi Linus I held off on my rc5 pull because I hit an oops during log recovery after a crash. I wanted to make sure it wasn't a regression because we have some logging fixes in here. It turns out that a commit during the merge window just made it much more likely to trigger directory logging instead of full commits, which exposed an old bug. Please grab my for-linus branch: git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus The new backref walking code got some additional fixes. This should be the final set of them. Josef fixed up a corner where our O_DIRECT writes and buffered reads could expose old file contents (not stale, just not the most recent). He and Liu Bo fixed crashes during tree log recover as well. Ilya fixed errors while we resume disk balancing operations on readonly mounts. Jan Schmidt (7) commits (+50/-36): Btrfs: leave critical region in btrfs_find_all_roots as soon as possible (+1/-2) Btrfs: always put insert_ptr modifications into the tree mod log (+7/-7) Btrfs: resolve tree mod log locking issue in btrfs_next_leaf (+12/-0) Btrfs: fix tree mod log for root replacements at leaf level (+15/-13) Btrfs: support root level changes in __resolve_indirect_ref (+8/-4) Btrfs: avoid waiting for delayed refs when we must not (+6/-5) Btrfs: fix tree mod log rewind of ADD operations (+1/-5) Josef Bacik (3) commits (+116/-111): Btrfs: hold a ref on the inode during writepages (+14/-0) Btrfs: fix tree log remove space corner case (+52/-93) Btrfs: fix dio write vs buffered read race (+50/-18) Ilya Dryomov (2) commits (+73/-43): Btrfs: resume balance on rw (re)mounts properly (+47/-18) Btrfs: restore restriper state on all mounts (+26/-25) Stefan Behrens (1) commits (+12/-10): Btrfs: don't count I/O statistic read errors for missing devices Liu Bo (1) commits (+1/-1): Btrfs: fix wrong check during log recovery Alexander Block (1) commits (+1/-1): Btrfs: use _IOR for BTRFS_IOC_SUBVOL_GETFLAGS Chris Mason (1) commits (+6/-0): Btrfs: run delayed directory updates during log replay Total: (16) commits (+259/-202) fs/btrfs/backref.c | 15 +++-- fs/btrfs/ctree.c| 60 ++ fs/btrfs/disk-io.c | 34 +++ fs/btrfs/extent-tree.c | 11 ++-- fs/btrfs/extent_io.c| 14 + fs/btrfs/file.c | 13 fs/btrfs/free-space-cache.c | 145 fs/btrfs/inode.c| 57 +++-- fs/btrfs/ioctl.h| 2 +- fs/btrfs/super.c| 4 ++ fs/btrfs/tree-log.c | 6 ++ fs/btrfs/volumes.c | 95 + fs/btrfs/volumes.h | 3 +- 13 files changed, 258 insertions(+), 201 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi Linus, Please grab my for-linus branch: git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus This is a small pull with btrfs fixes. The biggest of the bunch is another fix for the new backref walking code. We're still hammering out one btrfs dio vs buffered reads problem, but that one will have to wait for the next rc. Josef Bacik (2) commits (+3/-2): Btrfs: delay iput with async extents (+2/-2) Btrfs: add a missing spin_lock (+1/-0) Alexander Block (2) commits (+59/-44): Btrfs: don't assume to be on the correct extent in add_all_parents (+52/-42) Btrfs: introduce btrfs_next_old_item (+7/-2) Total: (4) commits (+62/-46) fs/btrfs/backref.c | 94 +--- fs/btrfs/ctree.h |9 +++-- fs/btrfs/disk-io.c |1 + fs/btrfs/inode.c |4 +-- 4 files changed, 62 insertions(+), 46 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
On Fri, Jun 15, 2012 at 05:57:42PM -0600, Linus Torvalds wrote: > On Fri, Jun 15, 2012 at 11:09 AM, Chris Mason > wrote: > > > > Please pull my for-linus branch: > > > > git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git > > for-linus > > This seems to introduce a new warning: > > In file included from fs/btrfs/ctree.c:22:0: > fs/btrfs/ctree.c: In function ‘btrfs_search_old_slot’: > fs/btrfs/ctree.h:2117:240: warning: ‘old_generation’ may be used > uninitialized in this function [-Wmaybe-uninitialized] > fs/btrfs/ctree.c:1178:6: note: ‘old_generation’ was declared here > > which looks like just gcc being unable to see that it is only used > when set, but it's still annoying. I'd suggest initializing it to 0 > just to shut up the compiler. Ok? > Yes, this does look like noise. It is only used when old_root is set, but it'll be really easy to add bugs there later. I've pushed two commits out, the second one is a printk cast on 32 bit machines. Neither of these are fatal but they are in for-linus: git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus Chris Mason (2) commits (+3/-2): Btrfs: cast devid to unsigned long long for printk %llu (+2/-1) Btrfs: init old_generation in get_old_root (+1/-1) Total: (2) commits (+3/-2) fs/btrfs/ctree.c |2 +- fs/btrfs/ioctl.c |3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
On Fri, Jun 15, 2012 at 11:09 AM, Chris Mason wrote: > > Please pull my for-linus branch: > > git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus This seems to introduce a new warning: In file included from fs/btrfs/ctree.c:22:0: fs/btrfs/ctree.c: In function ‘btrfs_search_old_slot’: fs/btrfs/ctree.h:2117:240: warning: ‘old_generation’ may be used uninitialized in this function [-Wmaybe-uninitialized] fs/btrfs/ctree.c:1178:6: note: ‘old_generation’ was declared here which looks like just gcc being unable to see that it is only used when set, but it's still annoying. I'd suggest initializing it to 0 just to shut up the compiler. Ok? Linus -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi Linus, Please pull my for-linus branch: git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus The dates look like I had to rebase this morning because there was a compiler warning for a printk arg that I had missed earlier. These are all fixes, including one to prevent using stale pointers for device names, and lots of fixes around transaction abort cleanups (Josef, Liu Bo). Jan Schmidt also sent in a number of fixes for the new reference number tracking code. Liu Bo beat me to updating the MAINTAINERS file. Since he thought to also fix the git url, I kept his commit. Josef Bacik (10) commits (+303/-102): Btrfs: pass locked_page into extent_clear_unlock_delalloc if theres an error (+2/-2) Btrfs: unlock everything properly in the error case for nocow (+35/-2) Btrfs: wake up transaction waiters when aborting a transaction (+7/-6) Btrfs: call filemap_fdatawrite twice for compression (+31/-7) Btrfs: fix locking in btrfs_destroy_delayed_refs (+17/-13) Btrfs: keep inode pinned when compressing writes (+6/-2) Btrfs: abort the transaction if the commit fails (+8/-2) Btrfs: use rcu to protect device->name (+162/-64) Btrfs: fix btrfs_destroy_marked_extents (+2/-4) Btrfs: implement ->show_devname (+33/-0) Jan Schmidt (6) commits (+72/-37): Btrfs: remove obsolete btrfs_next_leaf call from __resolve_indirect_ref (+1/-9) Btrfs: remove call to btrfs_header_nritems with no effect (+0/-3) Btrfs: fix return value for __tree_mod_log_oldest_root (+20/-13) Btrfs: use btrfs_read_lock_root_node in get_old_root (+16/-4) Btrfs: fix race in tree mod log addition (+19/-4) Btrfs: add btrfs_next_old_leaf (+16/-4) Liu Bo (5) commits (+29/-5): Btrfs: avoid memory leak of extent state in error handling routine (+2/-0) Btrfs: make sure that we've made everything in pinned tree clean (+11/-0) Btrfs: update MAINTAINERS info for BTRFS FILE SYSTEM (+2/-2) Btrfs: fix missing inherited flag in rename (+6/-3) Btrfs: do not resize a seeding device (+8/-0) Li Zefan (2) commits (+50/-49): Btrfs: fix incompat flags setting (+1/-1) Btrfs: fix defrag regression (+49/-48) Miao Xie (1) commits (+27/-0): Btrfs: destroy the items of the delayed inodes in error handling routine Total: (24) commits (+481/-193) MAINTAINERS|4 +- fs/btrfs/backref.c | 17 ++- fs/btrfs/btrfs_inode.h |1 + fs/btrfs/check-integrity.c | 16 +++--- fs/btrfs/ctree.c | 86 +++- fs/btrfs/ctree.h |2 + fs/btrfs/delayed-inode.c | 18 +++ fs/btrfs/delayed-inode.h |3 ++ fs/btrfs/disk-io.c | 76 +--- fs/btrfs/extent_io.c |7 +-- fs/btrfs/inode.c | 73 +-- fs/btrfs/ioctl.c | 118 +--- fs/btrfs/ordered-data.c| 22 - fs/btrfs/rcu-string.h | 56 + fs/btrfs/scrub.c | 30 ++- fs/btrfs/super.c | 33 + fs/btrfs/transaction.c | 14 +- fs/btrfs/volumes.c | 92 +- fs/btrfs/volumes.h |2 +- 19 files changed, 479 insertions(+), 191 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi everyone, My for-linus branch is updated for 3.5: git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus This includes a fairly large change from Josef around data writeback completion. Before, the writeback wasn't completed until the metadata insertions for the extent were done, and this made for fairly large latency spikes on the last page of each ordered extent. We already had a separate mechanism for tracking pending metadata insertions, so Josef just needed to tweak things a little to end writeback earlier on the page. Overall it makes us much friendly to memory reclaim and lowers latencies quite a lot for synchronous IO. Jan Schmidt has finished some background work required to track btree blocks as they go through changes in ownership. It's the missing piece he needed for both btrfs send/receive and subvolume quotas. Neither of those are ready yet, but the new tracking code is included here. Most of the time, the new code is off. It is only used by scrub and other backref walkers. Stefan Behrens has added io failure tracking. This includes counters for which drives are causing the most trouble so the admin (or an automated tool) can choose to kick them out. We're tracking IO errors, crc errors, and generation checks we do on each metadata block. RAID5/6 did miss the cut this time because I'm having trouble with corruptions. I'll nail it down next week and post as a beta testing before 3.6 Jan Schmidt (21) commits (+1405/-290): Btrfs: bugfix: ignore the wrong key for indirect tree block backrefs (+135/-50) Btrfs: add del_ptr and insert_ptr modifications to the tree mod log (+32/-10) Btrfs: add inodes before dropping the extent lock in find_all_leafs (+43/-6) Btrfs: fix tree mod log rewinded level and rewinding of moved keys (+4/-2) Btrfs: use delayed ref sequence numbers for all fs-tree updates (+13/-23) Btrfs: add missing spin_lock for insertion into tree mod log (+18/-5) Btrfs: don't set for_cow parameter for tree block functions (+20/-20) Btrfs: use the tree modification log for backref resolving (+29/-17) Btrfs: put all block modifications into the tree mod log (+36/-0) Btrfs: tree mod log sanity checks in join_transaction (+18/-0) Btrfs: look into the extent during find_all_leafs (+158/-84) Btrfs: fs_info variable for join_transaction (+19/-18) Btrfs: dummy extent buffers for tree mod log (+76/-7) Btrfs: add tree modification log functions (+412/-1) Btrfs: bugfix in btrfs_find_parent_nodes (+3/-2) Btrfs: move struct seq_list to ctree.h (+7/-5) Btrfs: add tree_mod_dont_log helper (+15/-9) Btrfs: add tree mod log to fs_info (+14/-0) Btrfs: add btrfs_search_old_slot (+317/-4) Btrfs: fix tree mod log del_ptr (+7/-6) Btrfs: ulist realloc bugfix (+29/-21) Josef Bacik (13) commits (+330/-322): Btrfs: convert the inode bit field to use the actual bit operations (+44/-44) Btrfs: check to see if the inode is in the log before fsyncing (+16/-17) Btrfs: merge contigous regions when loading free space cache (+41/-0) Btrfs: fall back to non-inline if we don't have enough space (+4/-1) Btrfs: do not check delalloc when updating disk_i_size (+3/-16) Btrfs: do not do filemap_write_and_wait_range in fsync (+6/-5) Btrfs: finish ordered extents in their own thread (+164/-191) Btrfs: remove useless waiting and extra filemap work (+2/-15) Btrfs: fix how we deal with the orphan block rsv (+24/-22) Btrfs: use i_version instead of our own sequence (+19/-10) Btrfs: fix return code in drop_objectid_items (+2/-0) Btrfs: fix compile warnings in extent_io.c (+1/-1) Btrfs: cache no acl on new inodes (+4/-0) Stefan Behrens (7) commits (+985/-174): Btrfs: read device stats on mount, write modified ones during commit (+232/-0) Btrfs: add device counters for detected IO and checksum errors (+230/-24) Btrfs: fix runtime warning in check-integrity check data mode (+22/-3) Btrfs: change integrity checker to support big blocks (+415/-146) Btrfs: fix false positive in check-integrity on unmount (+1/-1) Btrfs: add ioctl to get and reset the device stats (+77/-0) Btrfs: set ioprio of scrub readahead to idle (+8/-0) Wang Sheng-Hui (5) commits (+7/-8): Btrfs: remove the useless assignment to *entry in function tree_insert of file extent_io.c (+0/-1) Btrfs: fix btrfs_release_extent_buffer_page with the right usage of num_extent_pages (+3/-1) btrfs/ctree.c: remove the unnecessary 'return -1;' at the end of bin_search (+2/-4) Btrfs: cleanup the comment for clear_state_bit in extent_io.c (+1/-1) Btrfs: fix the comment for find_first_extent_bit (+1/-1) Jim Meyering (3) commits (+32/-44): Btrfs: NUL-terminate path buffer in DEV_INFO ioctl result (+4/-2) Btrfs: avoid buffer overrun in mount option handling (+26/-41) Btrfs: avoid buffer overrun in btrfs_printk (+2/-1) Liu Bo (3) commits (+28/-30):
Re: [GIT PULL] Btrfs updates
Hi Chris, Please apply following bug fix patch: http://marc.info/?l=linux-btrfs&m=18438913851&w=2 ([PATCH] btrfs: add missing unlocks to transaction abort paths) Thanks, Tsutomu (2012/04/13 22:38), Chris Mason wrote: > Hi Linus, > > My for-linus branch has btrfs fixes and updates: > > git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus > > The top commit is the only one that isn't strictly a bug fix. It > updates the btrfs metadata checker code to support metadata blocks > larger than the page size. This code isn't compiled in by default, and > it is only used by developers. I put it at the top so you could just > peel it off if you didn't want it. I also put a for-linus-min branch > without this commit: > > git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git > for-linus-min > > The others are pure bug fixes. We have a use-after-free in there, along > with errors when mount -o discard is enabled, and a BUG_ON(we should > compile with UP more often). > > Stefan Behrens (1) commits (+415/-146): > Btrfs: change integrity checker to support big blocks > > Chris Mason (2) commits (+2/-2): > Revert "Btrfs: increase the global block reserve estimates" (+1/-1) > Btrfs: fix uninit variable in repair_eb_io_failure (+1/-1) > > Dave Jones (1) commits (+5/-4): > Btrfs: fix use-after-free in __btrfs_end_transaction > > Liu Bo (1) commits (+18/-2): > Btrfs: fix eof while discarding extents > > Ilya Dryomov (1) commits (+2/-3): > Btrfs: remove lock assert from get_restripe_target() > > Josef Bacik (1) commits (+3/-10): > Btrfs: use commit root when loading free space cache > > Tsutomu Itoh (1) commits (+10/-0): > Btrfs: check return value of bio_alloc() properly > > Total: (8) commits (+455/-167) > > fs/btrfs/check-integrity.c | 561 > --- > fs/btrfs/compression.c |2 + > fs/btrfs/extent-tree.c | 11 +- > fs/btrfs/extent_io.c|6 +- > fs/btrfs/free-space-cache.c |9 +- > fs/btrfs/scrub.c|4 + > fs/btrfs/transaction.c |9 +- > fs/btrfs/volumes.c | 20 +- > 8 files changed, 455 insertions(+), 167 deletions(-) > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi Linus, My for-linus branch has btrfs fixes and updates: git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus The top commit is the only one that isn't strictly a bug fix. It updates the btrfs metadata checker code to support metadata blocks larger than the page size. This code isn't compiled in by default, and it is only used by developers. I put it at the top so you could just peel it off if you didn't want it. I also put a for-linus-min branch without this commit: git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus-min The others are pure bug fixes. We have a use-after-free in there, along with errors when mount -o discard is enabled, and a BUG_ON(we should compile with UP more often). Stefan Behrens (1) commits (+415/-146): Btrfs: change integrity checker to support big blocks Chris Mason (2) commits (+2/-2): Revert "Btrfs: increase the global block reserve estimates" (+1/-1) Btrfs: fix uninit variable in repair_eb_io_failure (+1/-1) Dave Jones (1) commits (+5/-4): Btrfs: fix use-after-free in __btrfs_end_transaction Liu Bo (1) commits (+18/-2): Btrfs: fix eof while discarding extents Ilya Dryomov (1) commits (+2/-3): Btrfs: remove lock assert from get_restripe_target() Josef Bacik (1) commits (+3/-10): Btrfs: use commit root when loading free space cache Tsutomu Itoh (1) commits (+10/-0): Btrfs: check return value of bio_alloc() properly Total: (8) commits (+455/-167) fs/btrfs/check-integrity.c | 561 --- fs/btrfs/compression.c |2 + fs/btrfs/extent-tree.c | 11 +- fs/btrfs/extent_io.c|6 +- fs/btrfs/free-space-cache.c |9 +- fs/btrfs/scrub.c|4 + fs/btrfs/transaction.c |9 +- fs/btrfs/volumes.c | 20 +- 8 files changed, 455 insertions(+), 167 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi Linus, I have two additional and btrfs fixes in my for-linus branch git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus One is a casting error that leads to memory corruption on i386 during scrub, and the other fixes a corner case in the backref walking code (also triggered by scrub). These are reviewed and tested. I was holding off on this while trying to track enospc bug reports, but it isn't yet clear if those are regressions. These are valid fixes either way. Chris Mason (1) commits (+1/-1): Btrfs: fix casting error in scrub reada code Li Zefan (1) commits (+6/-2): btrfs: fix locking issues in find_parent_nodes() Total: (2) commits (+7/-3) fs/btrfs/backref.c |8 ++-- fs/btrfs/reada.c |2 +- 2 files changed, 7 insertions(+), 3 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi Linus, Please pull the for-linus branch of the btrfs repo: git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus This is later than I wanted because I got backed up running through btrfs bugs from the Oracle QA teams. But they are all bug fixes that we've queued and tested since rc1. Nothing in particular stands out, this just reflects bug fixing and QA done in parallel by all the btrfs developers. The most user visible of these is: Btrfs: clear the extent uptodate bits during parent transid failures Because that helps deal with out of date drives (say an iscsi disk that has gone away and come back). The old code wasn't always properly retrying the other mirror for this type of failure. Liu Bo (6) commits (+24/-15): Btrfs: be less strict on finding next node in clear_extent_bit (+1/-2) Btrfs: skip states when they does not contain bits to clear (+10/-5) Btrfs: kick out redundant stuff in convert_extent_bit (+0/-5) Btrfs: increase the global block reserve estimates (+1/-1) Btrfs: fix trim 0 bytes after a device delete (+8/-1) Btrfs: fix a bug on overcommit stuff (+4/-1) Chris Mason (6) commits (+93/-47): Btrfs: clear the extent uptodate bits during parent transid failures (+3/-4) Btrfs: add extra sanity checks on the path names in btrfs_mksubvol (+6/-0) Btrfs: don't reserve data with extents locked in btrfs_fallocate (+10/-15) Btrfs: improve error handling for btrfs_insert_dir_item callers (+26/-7) Btrfs: fix compiler warnings on 32 bit systems (+26/-20) Btrfs: make sure we update latest_bdev (+22/-1) Tsutomu Itoh (3) commits (+15/-8): Btrfs: check return value of lookup_extent_mapping() correctly (+4/-2) Btrfs: fix memory leak in load_free_space_cache() (+1/-0) Btrfs: fix return value check of extent_io_ops (+10/-6) David Sterba (2) commits (+17/-3): btrfs: fix structs where bitfields and spinlock/atomic share 8B word (+3/-3) btrfs: silence warning in raid array setup (+14/-0) Jan Schmidt (1) commits (+2/-0): Btrfs: avoid positive number with ERR_PTR Jeff Mahoney (1) commits (+53/-27): btrfs: delalloc for page dirtied out-of-band in fixup worker Keith Mannthey (1) commits (+6/-0): btrfs: Sector Size check during Mount Arne Jansen (1) commits (+5/-3): btrfs: don't check DUP chunks twice Jeff Liu (1) commits (+2/-2): Btrfs: return the internal error unchanged if btrfs_get_extent_fiemap() call failed for SEEK_DATA/SEEK_HOLE inquiry Miao Xie (1) commits (+29/-24): Btrfs: fix deadlock on page lock when doing auto-defragment Florian Albrechtskirchinger (1) commits (+4/-2): btrfs: honor umask when creating subvol root Total: (24) commits (+250/-131) fs/btrfs/backref.c |2 + fs/btrfs/check-integrity.c |2 +- fs/btrfs/compression.c |2 + fs/btrfs/ctree.h|2 +- fs/btrfs/disk-io.c | 12 + fs/btrfs/extent-tree.c | 51 --- fs/btrfs/extent_io.c| 113 +-- fs/btrfs/extent_io.h|1 + fs/btrfs/extent_map.h |4 +- fs/btrfs/file.c | 29 +--- fs/btrfs/free-space-cache.c |1 + fs/btrfs/inode-map.c|6 ++- fs/btrfs/inode.c| 40 +-- fs/btrfs/ioctl.c| 59 +- fs/btrfs/scrub.c|8 ++- fs/btrfs/transaction.c | 16 --- fs/btrfs/volumes.c | 33 - 17 files changed, 250 insertions(+), 131 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
On Mon, 5 Dec 2011 08:14:13 -0500, Chris Mason wrote: > On Mon, Dec 05, 2011 at 04:10:49PM +0800, Miao Xie wrote: >> Hi, Chris and Oliva >> >> On thu, 1 Dec 2011 10:39:55 -0500, Chris Mason wrote: >>> git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git >>> for-linus >>> >>> Has our current set of fixes. This is fairly small, Alexandre Oliva has >>> been chasing problems in our block allocator and kicked out important >>> fixes. >>> >>> Jan Schmidt fixed a merge error in the raid repair code, we're now >>> properly repairing failed blocks (io errors or crc errors) without >>> having to run a scrub. >>> >>> Alexandre Oliva (5) commits (+8/-8): >>> Btrfs: skip block groups without enough space for a cluster (+1/-1) >> >> This patch introduce a bug that we can not allocate blocks from the cluster >> with enough space and it may make the block allocation fail. >> >> This is because the check that the above patch introduced make the allocator >> skip the cluster allocation, and jump to the uncluster allocation without >> reclaiming all the blocks in the cluster, At this time, if all the free space >> is in the cluster, and no space in the block group, the allocation will fail. >> (we can trigger this bug on SSD.) >> >> Fortunately, the following patch written by Oliva can fix this bug. >> >> [PATCH 08/20] Btrfs: try to allocate from cluster even at LOOP_NO_EMPTY_SIZE > > Thanks, I'll push this 08/20 out as well. I'm sorry for my careless test. I tested it again just now, I found the above patch could not fix the bug completely, we still need [PATCH 16/20] Btrfs: try cluster but don't advance in search list After applying these two patch, all my test can pass. Thanks Miao -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
On Mon, Dec 05, 2011 at 08:14:13AM -0500, Chris Mason wrote: > On Mon, Dec 05, 2011 at 04:10:49PM +0800, Miao Xie wrote: > > [PATCH 08/20] Btrfs: try to allocate from cluster even at LOOP_NO_EMPTY_SIZE > > Thanks, I'll push this 08/20 out as well. Please pick Li Zefan: Btrfs: check if the to-be-added device is writable Liu Bo: Btrfs: drop spin lock when memory alloc fails (collected in branch fixes-20111205 at my repo) I overlooked them and forgot to include in the fixes branch before the last pull request, sorry. david -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
On Mon, Dec 05, 2011 at 04:10:49PM +0800, Miao Xie wrote: > Hi, Chris and Oliva > > On thu, 1 Dec 2011 10:39:55 -0500, Chris Mason wrote: > > git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git > > for-linus > > > > Has our current set of fixes. This is fairly small, Alexandre Oliva has > > been chasing problems in our block allocator and kicked out important > > fixes. > > > > Jan Schmidt fixed a merge error in the raid repair code, we're now > > properly repairing failed blocks (io errors or crc errors) without > > having to run a scrub. > > > > Alexandre Oliva (5) commits (+8/-8): > > Btrfs: skip block groups without enough space for a cluster (+1/-1) > > This patch introduce a bug that we can not allocate blocks from the cluster > with enough space and it may make the block allocation fail. > > This is because the check that the above patch introduced make the allocator > skip the cluster allocation, and jump to the uncluster allocation without > reclaiming all the blocks in the cluster, At this time, if all the free space > is in the cluster, and no space in the block group, the allocation will fail. > (we can trigger this bug on SSD.) > > Fortunately, the following patch written by Oliva can fix this bug. > > [PATCH 08/20] Btrfs: try to allocate from cluster even at LOOP_NO_EMPTY_SIZE Thanks, I'll push this 08/20 out as well. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
Hi, Chris and Oliva On thu, 1 Dec 2011 10:39:55 -0500, Chris Mason wrote: > git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus > > Has our current set of fixes. This is fairly small, Alexandre Oliva has > been chasing problems in our block allocator and kicked out important > fixes. > > Jan Schmidt fixed a merge error in the raid repair code, we're now > properly repairing failed blocks (io errors or crc errors) without > having to run a scrub. > > Alexandre Oliva (5) commits (+8/-8): > Btrfs: skip block groups without enough space for a cluster (+1/-1) This patch introduce a bug that we can not allocate blocks from the cluster with enough space and it may make the block allocation fail. This is because the check that the above patch introduced make the allocator skip the cluster allocation, and jump to the uncluster allocation without reclaiming all the blocks in the cluster, At this time, if all the free space is in the cluster, and no space in the block group, the allocation will fail. (we can trigger this bug on SSD.) Fortunately, the following patch written by Oliva can fix this bug. [PATCH 08/20] Btrfs: try to allocate from cluster even at LOOP_NO_EMPTY_SIZE Thanks Miao > Btrfs: start search for new cluster at the beginning (+2/-4) > Btrfs: reset cluster's max_size when creating bitmap (+1/-0) > Btrfs: skip allocation attempt from empty cluster (+3/-3) > Btrfs: initialize new bitmaps' list (+1/-0) > > Li Zefan (1) commits (+3/-3): > Btrfs: fix oops when calling statfs on readonly device > > Arnd Hannemann (1) commits (+2/-2): > Fix URL of btrfs-progs git repository in docs > > Jan Schmidt (1) commits (+20/-7): > Btrfs: fix meta data raid-repair merge problem > > Dan Carpenter (1) commits (+5/-0): > btrfs scrub: handle -ENOMEM from init_ipath() > > Mike Fleetwood (1) commits (+1/-1): > Btrfs: Don't error on resizing FS to same size > > Miao Xie (1) commits (+22/-5): > Btrfs: fix deadlock on metadata reservation when evicting a inode > > Total: (11) commits (+60/-25) > > Documentation/filesystems/btrfs.txt |4 ++-- > fs/btrfs/ctree.h|3 +++ > fs/btrfs/extent-tree.c | 34 +++--- > fs/btrfs/extent_io.c| 27 --- > fs/btrfs/free-space-cache.c |2 ++ > fs/btrfs/inode.c|2 +- > fs/btrfs/ioctl.c|2 +- > fs/btrfs/scrub.c|5 + > fs/btrfs/super.c|6 +++--- > 9 files changed, 60 insertions(+), 25 deletions(-) > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi everyone, The for-linus branch of the btrfs tree: git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus Has our current set of fixes. This is fairly small, Alexandre Oliva has been chasing problems in our block allocator and kicked out important fixes. Jan Schmidt fixed a merge error in the raid repair code, we're now properly repairing failed blocks (io errors or crc errors) without having to run a scrub. Alexandre Oliva (5) commits (+8/-8): Btrfs: skip block groups without enough space for a cluster (+1/-1) Btrfs: start search for new cluster at the beginning (+2/-4) Btrfs: reset cluster's max_size when creating bitmap (+1/-0) Btrfs: skip allocation attempt from empty cluster (+3/-3) Btrfs: initialize new bitmaps' list (+1/-0) Li Zefan (1) commits (+3/-3): Btrfs: fix oops when calling statfs on readonly device Arnd Hannemann (1) commits (+2/-2): Fix URL of btrfs-progs git repository in docs Jan Schmidt (1) commits (+20/-7): Btrfs: fix meta data raid-repair merge problem Dan Carpenter (1) commits (+5/-0): btrfs scrub: handle -ENOMEM from init_ipath() Mike Fleetwood (1) commits (+1/-1): Btrfs: Don't error on resizing FS to same size Miao Xie (1) commits (+22/-5): Btrfs: fix deadlock on metadata reservation when evicting a inode Total: (11) commits (+60/-25) Documentation/filesystems/btrfs.txt |4 ++-- fs/btrfs/ctree.h|3 +++ fs/btrfs/extent-tree.c | 34 +++--- fs/btrfs/extent_io.c| 27 --- fs/btrfs/free-space-cache.c |2 ++ fs/btrfs/inode.c|2 +- fs/btrfs/ioctl.c|2 +- fs/btrfs/scrub.c|5 + fs/btrfs/super.c|6 +++--- 9 files changed, 60 insertions(+), 25 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
Excerpts from Sage Weil's message of 2011-08-18 17:51:54 -0400: > Hi Chris, Josef, > > Can some form of the clone ioctl transaction start reservation fix go in > soon as well? That hits a BUG_ON every time. Sorry Sage, I thought I had that one. I'll get it in the next pull. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
Hi Chris, Josef, Can some form of the clone ioctl transaction start reservation fix go in soon as well? That hits a BUG_ON every time. Thanks! sage On Thu, 18 Aug 2011, Chris Mason wrote: > Hi everyone, > > The for-linus branch of the btrfs-unstable tree: > > git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git > for-linus > > Has our current pull request for 3.1-rc. The for-linus branch includes > 3.1-rc2 because it two fixes from Dan Carpenter and Jeff Mahoney that > only apply to 3.1. > > The master branch of btrfs-unstable is based on 3.0, and includes > everything except those two changes. > > This is a variety pack of fixes. We do have another fix pending for a > race in our readdir optimizations, but Josef will work that out with Al > Viro and send it in later this week (or early next). > > Chris Mason (1) commits (+17/-0): > Btrfs: force unplugs when switching from high to regular priority bios > > Dan Carpenter (2) commits (+10/-4): > btrfs: unlock on error in btrfs_file_llseek() (+8/-4) > btrfs: memory leak in btrfs_add_inode_defrag() (+2/-0) > > Jeff Mahoney (1) commits (+8/-4): > btrfs: btrfs_permission's RO check shouldn't apply to device nodes > > Josef Bacik (2) commits (+43/-2): > Btrfs: set i_size properly when fallocating (+14/-0) > Btrfs: detect wether a device supports discard (+29/-2) > > Li Zefan (1) commits (+2/-4): > Btrfs: use plain page_address() in header fields setget functions > > Miao Xie (2) commits (+12/-6): > Btrfs: fix uninitialized sync_pending (+1/-1) > Btrfs: fix wrong free space information (+11/-5) > > Sage Weil (1) commits (+4/-0): > Btrfs: truncate pages from clone ioctl target range > > Tsutomu Itoh (1) commits (+16/-10): > Btrfs: forced readonly when btrfs_drop_snapshot() fails > > liubo (3) commits (+72/-14): > Btrfs: check if there is enough space for balancing smarter (+35/-6) > Btrfs: fix a bug of balance on full multi-disk partitions (+13/-4) > Btrfs: fix an oops of log replay (+24/-4) > > Total: (14) commits (+183/-43) > > fs/btrfs/ctree.h| 10 ++--- > fs/btrfs/extent-tree.c | 75 +- > fs/btrfs/file.c | 28 ++-- > fs/btrfs/free-space-cache.c | 16 ++--- > fs/btrfs/inode.c| 12 -- > fs/btrfs/ioctl.c|4 ++ > fs/btrfs/tree-log.c | 28 ++-- > fs/btrfs/volumes.c | 51 +++-- > fs/btrfs/volumes.h |2 + > 9 files changed, 183 insertions(+), 43 deletions(-) > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi everyone, The for-linus branch of the btrfs-unstable tree: git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git for-linus Has our current pull request for 3.1-rc. The for-linus branch includes 3.1-rc2 because it two fixes from Dan Carpenter and Jeff Mahoney that only apply to 3.1. The master branch of btrfs-unstable is based on 3.0, and includes everything except those two changes. This is a variety pack of fixes. We do have another fix pending for a race in our readdir optimizations, but Josef will work that out with Al Viro and send it in later this week (or early next). Chris Mason (1) commits (+17/-0): Btrfs: force unplugs when switching from high to regular priority bios Dan Carpenter (2) commits (+10/-4): btrfs: unlock on error in btrfs_file_llseek() (+8/-4) btrfs: memory leak in btrfs_add_inode_defrag() (+2/-0) Jeff Mahoney (1) commits (+8/-4): btrfs: btrfs_permission's RO check shouldn't apply to device nodes Josef Bacik (2) commits (+43/-2): Btrfs: set i_size properly when fallocating (+14/-0) Btrfs: detect wether a device supports discard (+29/-2) Li Zefan (1) commits (+2/-4): Btrfs: use plain page_address() in header fields setget functions Miao Xie (2) commits (+12/-6): Btrfs: fix uninitialized sync_pending (+1/-1) Btrfs: fix wrong free space information (+11/-5) Sage Weil (1) commits (+4/-0): Btrfs: truncate pages from clone ioctl target range Tsutomu Itoh (1) commits (+16/-10): Btrfs: forced readonly when btrfs_drop_snapshot() fails liubo (3) commits (+72/-14): Btrfs: check if there is enough space for balancing smarter (+35/-6) Btrfs: fix a bug of balance on full multi-disk partitions (+13/-4) Btrfs: fix an oops of log replay (+24/-4) Total: (14) commits (+183/-43) fs/btrfs/ctree.h| 10 ++--- fs/btrfs/extent-tree.c | 75 +- fs/btrfs/file.c | 28 ++-- fs/btrfs/free-space-cache.c | 16 ++--- fs/btrfs/inode.c| 12 -- fs/btrfs/ioctl.c|4 ++ fs/btrfs/tree-log.c | 28 ++-- fs/btrfs/volumes.c | 51 +++-- fs/btrfs/volumes.h |2 + 9 files changed, 183 insertions(+), 43 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi everyone, The for-linus branch of the btrfs-unstable repo is reading for pulling: git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git for-linus This started off as a larger pull, but I had to pull out a number of cleanups from Fujitsu, Novell and a few others (sorry guys) while hunting a crash during stress.sh. It looks like it is unrelated to those commits, but I had to pull out a bunch of them until I can be sure I understand the bug. I'll have another pull request with the fully tested cleanups on Monday. Depending on how Linus does rc1, they might end up as the start of my 3.2 branch. This pull has the commits that I've been able to run through extensive testing. The biggest change here is switching the btrfs tree locks to a reader/writer lock. This has been one of our biggest bottlenecks for some time, and it was consistently at the top of profiles on large machines. The new locks do away with all the adaptive spinning inside of btrfs and rely on the spinning/blocking hints in the code to decide when it must block. The reader/writer locks break the code I had in place to use kmap on metadata buffers, so all of our metadata is now in lowmem. I did test this on a 32 bit VM, but x86-32 users will want to poke gently. I also adapted Tejun's lockdep fixes for the btrfs locks, and so far I haven't seen any lockdep warnings. Josef has a series of enospc fixes and tweaks here as well. His bigger patch to start reworking the enospc reservations seems to be causing the corruptions during stress.sh, so it will wait for 3.2. Josef Bacik (11) commits (+320/-322): Btrfs: use the normal checksumming infrastructure for free space cache (+59/-110) Btrfs: do transaction space reservation before joining the transaction (+17/-42) Btrfs: fix how we merge extent states and deal with cached states (+11/-12) Btrfs: try to only do one btrfs_search_slot in do_setxattr (+42/-33) Btrfs: use find_or_create_page instead of grab_cache_page (+9/-7) Btrfs: serialize flushers in reserve_metadata_bytes (+51/-21) Btrfs: fix deadlock when throttling transactions (+9/-2) Btrfs: fix enospc problems with delalloc (+86/-60) Btrfs: use a worker thread to do caching (+27/-29) Btrfs: don't flush delalloc arbitrarily (+0/-3) Btrfs: tag pages for writeback in sync (+9/-3) Chris Mason (7) commits (+642/-699): Btrfs: make sure reserve_metadata_bytes doesn't leak out strange errors (+6/-1) Btrfs: use the commit_root for reading free_space_inode crcs (+28/-19) Btrfs: reduce extent_state lock contention for metadata (+41/-14) Btrfs: switch the btrfs tree locks to reader/writer (+431/-218) Btrfs: remove lockdep magic from btrfs_next_leaf (+5/-31) Btrfs: stop using highmem for extent_buffers (+52/-378) Btrfs: make a lockdep class for each root (+79/-38) Miao Xie (1) commits (+21/-7): Btrfs: fix BUG_ON() caused by ENOSPC when relocating space Total: (19) fs/btrfs/btrfs_inode.h | 16 ++- fs/btrfs/ctree.c| 457 +++ fs/btrfs/ctree.h| 14 +- fs/btrfs/delayed-inode.c|2 +- fs/btrfs/dir-item.c |9 +- fs/btrfs/disk-io.c | 116 fs/btrfs/disk-io.h | 10 +- fs/btrfs/extent-tree.c | 285 +++- fs/btrfs/extent_io.c| 168 fs/btrfs/extent_io.h| 35 ++-- fs/btrfs/file-item.c| 41 +--- fs/btrfs/file.c | 11 +- fs/btrfs/free-space-cache.c | 173 ++--- fs/btrfs/inode.c| 90 -- fs/btrfs/ioctl.c|8 +- fs/btrfs/locking.c | 280 ++- fs/btrfs/locking.h | 36 - fs/btrfs/relocation.c |3 +- fs/btrfs/struct-funcs.c | 100 +++--- fs/btrfs/transaction.c | 47 +++-- fs/btrfs/tree-log.c |6 +- fs/btrfs/volumes.c |2 +- fs/btrfs/xattr.c| 66 --- 23 files changed, 965 insertions(+), 1010 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates for 3.1
On 07/10/2011 08:20 PM, Mitch Harder wrote: > On Sat, Jul 2, 2011 at 4:25 PM, Josef Bacik wrote: >> On 07/01/2011 04:39 PM, Josef Bacik wrote: >>> Hey Chris, >>> >>> Since I'm going on vacation next week I wanted to get everything ready for >>> you >>> in case you get bored with fsck and want to put together a 3.1 tree :). If >>> you >>> can pull >>> >>> git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-work.git for-chris >>> >>> It is based on your for-linus branch. Here is the shortlog and diffstat >>> >> >> Ugh sorry I had to update the don't panic patch because I'm an idiot. >> Thanks, >> >> Josef >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in >> the body of a message to majord...@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > > Josef: > > I've been testing this series of patches in a 2.6.39.3 kernel merged > with the latest 'for-linus' branch in Chris btrfs-unstable repository. > > I'm getting a kernel BUG that looks like it could be attributed to the > "Btrfs: try to only do one btrfs_search_slot in do_setxattr" patch. > > [ 5959.860027] [ cut here ] > [ 5959.860032] kernel BUG at include/linux/spinlock.h:380! > [ 5959.860035] invalid opcode: [#1] SMP > [ 5959.860038] last sysfs file: /sys/kernel/uevent_seqnum > [ 5959.860041] CPU 1 > [ 5959.860042] Modules linked in: snd_seq_midi nvidia(P) lgdt330x > cx88_dvb cx88_vp3054_i2c videobuf_dvb snd_ens1371 tuner_simple > tuner_types snd_rawmidi tda9887 tda8290 tuner cx8800 cx88_alsa cx8802 > snd_ac97_codec r8169 i2c_i801 cx88xx tveeprom videobuf_dma_sg > btcx_risc videobuf_core ac97_bus > [ 5959.860062] > [ 5959.860065] Pid: 13521, comm: install Tainted: P > 2.6.39.3-git-local-v10+ #1 Gigabyte Technology Co., Ltd. > P35-DS3L/P35-DS3L > [ 5959.860071] RIP: 0010:[] [] > btrfs_assert_tree_locked+0x21/0x30 > [ 5959.860079] RSP: 0018:880005dffaf8 EFLAGS: 00010246 > [ 5959.860082] RAX: 3300 RBX: 88007be51b40 RCX: > 0001 > [ 5959.860084] RDX: RSI: 880005dfe000 RDI: > 880025993900 > [ 5959.860087] RBP: 880005dffaf8 R08: 0001 R09: > 0001 > [ 5959.860089] R10: R11: R12: > 88005a8c2800 > [ 5959.860092] R13: 0001 R14: 000b R15: > 88003e41ebd0 > [ 5959.860095] FS: 7f495b017700() GS:88007fd0() > knlGS: > [ 5959.860098] CS: 0010 DS: ES: CR0: 80050033 > [ 5959.860100] CR2: 7f495afdc000 CR3: 3482b000 CR4: > 06e0 > [ 5959.860103] DR0: DR1: DR2: > > [ 5959.860106] DR3: DR6: 0ff0 DR7: > 0400 > [ 5959.860109] Process install (pid: 13521, threadinfo > 880005dfe000, task 880056aaac40) > [ 5959.860111] Stack: > [ 5959.860113] 880005dffb88 813536f7 88000343f000 > 8138add4 > [ 5959.860117] 880078546700 000b 0072 > 0001 > [ 5959.860122] 880005dffb88 00010001 > 0efb0343f000 > [ 5959.860126] Call Trace: > [ 5959.860132] [] push_leaf_left+0xc7/0x190 > [ 5959.860136] [] ? btrfs_item_size+0xe4/0xf0 > [ 5959.860140] [] btrfs_del_items+0x39d/0x570 > [ 5959.860144] [] btrfs_delete_one_dir_name+0xf9/0x100 > [ 5959.860148] [] do_setxattr+0x191/0x1f0 > [ 5959.860152] [] ? join_transaction.clone.24+0x21/0x220 > [ 5959.860156] [] __btrfs_setxattr+0x93/0xf0 > [ 5959.860159] [] btrfs_set_acl+0x103/0x230 > [ 5959.860163] [] btrfs_acl_chmod+0xea/0xf0 > [ 5959.860167] [] ? __mark_inode_dirty+0x68/0x200 > [ 5959.860171] [] btrfs_setattr+0x9e/0xd0 > [ 5959.860175] [] notify_change+0x189/0x370 > [ 5959.860179] [] sys_fchmodat+0xcd/0x100 > [ 5959.860183] [] ? do_munmap+0x2eb/0x360 > [ 5959.860187] [] sys_chmod+0x18/0x20 > [ 5959.860191] [] system_call_fastpath+0x16/0x1b > [ 5959.860193] Code: 66 2e 0f 1f 84 00 00 00 00 00 55 48 89 e5 0f 1f > 44 00 00 48 8b 47 38 a8 02 75 0e 8b 57 64 89 d0 c1 f8 08 31 d0 84 c0 > 74 02 c9 c3 <0f> 0b 90 90 90 90 90 90 90 90 90 90 90 90 90 55 48 89 e5 > 48 83 > [ 5959.860226] RIP [] btrfs_assert_tree_locked+0x21/0x30 > [ 5959.860230] RSP > [ 5959.860233] ---[ end trace 56253344f3b39a7c ]--- > > I'll try dropping the patch to confirm the source, but it's been an > intermittent bug. I haven't run across a simple way to reliably > reproduce it. Oh I see what's going on, if we're trying to overwrite an existing xattr we will panic because I just take the path and try to delete the sucker, but because the path was originally setup for insertion it unlocked the parent's parent and boom. I will fix this up, thanks for reporting it, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.ker
Re: [GIT PULL] Btrfs updates for 3.1
On Sat, Jul 2, 2011 at 4:25 PM, Josef Bacik wrote: > On 07/01/2011 04:39 PM, Josef Bacik wrote: >> Hey Chris, >> >> Since I'm going on vacation next week I wanted to get everything ready for >> you >> in case you get bored with fsck and want to put together a 3.1 tree :).  If >> you >> can pull >> >> git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-work.git for-chris >> >> It is based on your for-linus branch.  Here is the shortlog and diffstat >> > > Ugh sorry I had to update the don't panic patch because I'm an idiot. > Thanks, > > Josef > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at  http://vger.kernel.org/majordomo-info.html > Josef: I've been testing this series of patches in a 2.6.39.3 kernel merged with the latest 'for-linus' branch in Chris btrfs-unstable repository. I'm getting a kernel BUG that looks like it could be attributed to the "Btrfs: try to only do one btrfs_search_slot in do_setxattr" patch. [ 5959.860027] [ cut here ] [ 5959.860032] kernel BUG at include/linux/spinlock.h:380! [ 5959.860035] invalid opcode: [#1] SMP [ 5959.860038] last sysfs file: /sys/kernel/uevent_seqnum [ 5959.860041] CPU 1 [ 5959.860042] Modules linked in: snd_seq_midi nvidia(P) lgdt330x cx88_dvb cx88_vp3054_i2c videobuf_dvb snd_ens1371 tuner_simple tuner_types snd_rawmidi tda9887 tda8290 tuner cx8800 cx88_alsa cx8802 snd_ac97_codec r8169 i2c_i801 cx88xx tveeprom videobuf_dma_sg btcx_risc videobuf_core ac97_bus [ 5959.860062] [ 5959.860065] Pid: 13521, comm: install Tainted: P 2.6.39.3-git-local-v10+ #1 Gigabyte Technology Co., Ltd. P35-DS3L/P35-DS3L [ 5959.860071] RIP: 0010:[] [] btrfs_assert_tree_locked+0x21/0x30 [ 5959.860079] RSP: 0018:880005dffaf8 EFLAGS: 00010246 [ 5959.860082] RAX: 3300 RBX: 88007be51b40 RCX: 0001 [ 5959.860084] RDX: RSI: 880005dfe000 RDI: 880025993900 [ 5959.860087] RBP: 880005dffaf8 R08: 0001 R09: 0001 [ 5959.860089] R10: R11: R12: 88005a8c2800 [ 5959.860092] R13: 0001 R14: 000b R15: 88003e41ebd0 [ 5959.860095] FS: 7f495b017700() GS:88007fd0() knlGS: [ 5959.860098] CS: 0010 DS: ES: CR0: 80050033 [ 5959.860100] CR2: 7f495afdc000 CR3: 3482b000 CR4: 06e0 [ 5959.860103] DR0: DR1: DR2: [ 5959.860106] DR3: DR6: 0ff0 DR7: 0400 [ 5959.860109] Process install (pid: 13521, threadinfo 880005dfe000, task 880056aaac40) [ 5959.860111] Stack: [ 5959.860113] 880005dffb88 813536f7 88000343f000 8138add4 [ 5959.860117] 880078546700 000b 0072 0001 [ 5959.860122] 880005dffb88 00010001 0efb0343f000 [ 5959.860126] Call Trace: [ 5959.860132] [] push_leaf_left+0xc7/0x190 [ 5959.860136] [] ? btrfs_item_size+0xe4/0xf0 [ 5959.860140] [] btrfs_del_items+0x39d/0x570 [ 5959.860144] [] btrfs_delete_one_dir_name+0xf9/0x100 [ 5959.860148] [] do_setxattr+0x191/0x1f0 [ 5959.860152] [] ? join_transaction.clone.24+0x21/0x220 [ 5959.860156] [] __btrfs_setxattr+0x93/0xf0 [ 5959.860159] [] btrfs_set_acl+0x103/0x230 [ 5959.860163] [] btrfs_acl_chmod+0xea/0xf0 [ 5959.860167] [] ? __mark_inode_dirty+0x68/0x200 [ 5959.860171] [] btrfs_setattr+0x9e/0xd0 [ 5959.860175] [] notify_change+0x189/0x370 [ 5959.860179] [] sys_fchmodat+0xcd/0x100 [ 5959.860183] [] ? do_munmap+0x2eb/0x360 [ 5959.860187] [] sys_chmod+0x18/0x20 [ 5959.860191] [] system_call_fastpath+0x16/0x1b [ 5959.860193] Code: 66 2e 0f 1f 84 00 00 00 00 00 55 48 89 e5 0f 1f 44 00 00 48 8b 47 38 a8 02 75 0e 8b 57 64 89 d0 c1 f8 08 31 d0 84 c0 74 02 c9 c3 <0f> 0b 90 90 90 90 90 90 90 90 90 90 90 90 90 55 48 89 e5 48 83 [ 5959.860226] RIP [] btrfs_assert_tree_locked+0x21/0x30 [ 5959.860230] RSP [ 5959.860233] ---[ end trace 56253344f3b39a7c ]--- I'll try dropping the patch to confirm the source, but it's been an intermittent bug. I haven't run across a simple way to reliably reproduce it. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates for 3.1
On 07/01/2011 04:39 PM, Josef Bacik wrote: > Hey Chris, > > Since I'm going on vacation next week I wanted to get everything ready for you > in case you get bored with fsck and want to put together a 3.1 tree :). If > you > can pull > > git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-work.git for-chris > > It is based on your for-linus branch. Here is the shortlog and diffstat > Ugh sorry I had to update the don't panic patch because I'm an idiot. Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates for 3.1
Hey Chris, Since I'm going on vacation next week I wanted to get everything ready for you in case you get bored with fsck and want to put together a 3.1 tree :). If you can pull git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-work.git for-chris It is based on your for-linus branch. Here is the shortlog and diffstat Josef Bacik (7): Btrfs: try to only do one btrfs_search_slot in do_setxattr Btrfs: do transaction space reservation before joining the transaction Btrfs: serialize flushers in reserve_metadata_bytes Btrfs: use the normal checksumming infrastructure for free space cache Btrfs: fix how we merge extent states and deal with cached states Btrfs: use a worker thread to do caching Btrfs: don't panic if we get an error while balancing fs/btrfs/ctree.h| 10 ++- fs/btrfs/dir-item.c |9 +-- fs/btrfs/disk-io.c |6 ++ fs/btrfs/extent-tree.c | 136 +-- fs/btrfs/extent_io.c| 23 +++--- fs/btrfs/free-space-cache.c | 169 +++ fs/btrfs/transaction.c | 36 +- fs/btrfs/volumes.c |3 +- fs/btrfs/xattr.c| 54 +++--- 9 files changed, 196 insertions(+), 250 deletions(-) Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi everyone, The for-linus branch of the btrfs unstable tree: git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git for-linus Has our current set of fixes. Linus will probably notice the head commit was from this morning. I reordered a refcount bump inside the lock that protects the delayed inode tree and did a commit --amend. This is just for clarity, the delayed_inode can't go away because it is pinned by the inode itself. But, I don't want that code copy & pasted elsewhere to create bugs later on. These fix a regression in the new delayed_inode code where inodes that are dropped from ram need to check for a delayed inode before we load them back in. Josef's commit avoids oopsen if the free extent cache had to be discarded, and my commit makes sure a newly created empty file gets logged properly if someone calls fsync on it. Miao Xie (1) commits (+91/-26): btrfs: fix inconsonant inode information Josef Bacik (1) commits (+6/-3): Btrfs: make sure to update total_bitmaps when freeing cache V3 Ilya Dryomov (1) commits (+2/-2): Btrfs: fix type mismatch in find_free_extent() Chris Mason (1) commits (+1/-0): Btrfs: make sure to record the transid in new inodes Total: (4) commits (+100/-31) fs/btrfs/delayed-inode.c| 104 +-- fs/btrfs/delayed-inode.h|1 + fs/btrfs/extent-tree.c |4 +- fs/btrfs/free-space-cache.c |9 +++- fs/btrfs/inode.c| 13 +- 5 files changed, 100 insertions(+), 31 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi everyone, The for-linus branch of the btrfs unstable repo: git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git for-linus Has our current queue of fixes. One of the fixes removes our unused sysfs code, mostly because gcc likes to complain about it since we removed some dead code in rc1. We do plan on eventually putting more info into the sysfs dir, so the directory itself is still there. Otherwise these are regression fixes. The new delayed metadata insertion code and the new transaction locking code had races with the code to balance space between drives, and races against snapshot creation. There are also two ENOSPC regression fixes included as well. Chris Mason (4) commits (+136/-28): Btrfs: drop the delalloc_bytes check in shrink_delalloc (+0/-4) Btrfs: avoid delayed metadata items during commits (+28/-10) Btrfs: check the return value from set_anon_super (+3/-1) Btrfs: fix relocation races (+105/-13) Josef Bacik (3) commits (+14/-3): Btrfs: set no_trans_join after trying to expand the transaction (+11/-3) Btrfs: protect the pending_snapshots list with trans_lock (+2/-0) Btrfs: fix path leakage on subvol deletion (+1/-0) David Sterba (2) commits (+4/-4): btrfs: fix dereference of ERR_PTR value (+1/-1) btrfs: fix uninitialized return value (+3/-3) Miao Xie (1) commits (+20/-6): btrfs: fix wrong reservation when doing delayed inode operations Maarten Lankhorst (1) commits (+0/-148): btrfs: Remove unused sysfs code Total: (11) commits (+174/-189) fs/btrfs/ctree.h | 15 +- fs/btrfs/delayed-inode.c | 32 +-- fs/btrfs/delayed-inode.h |5 +- fs/btrfs/disk-io.c | 12 ++-- fs/btrfs/extent-tree.c |4 - fs/btrfs/inode.c |1 + fs/btrfs/ioctl.c |2 + fs/btrfs/relocation.c| 30 +++--- fs/btrfs/sysfs.c | 146 -- fs/btrfs/transaction.c | 114 ++- fs/btrfs/tree-log.c |2 +- 11 files changed, 174 insertions(+), 189 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
09:02, Andi Kleen wrote: > Chris Mason writes: > >> Hi everyone, >> >> The for-linus branch of the btrfs unstable tree: >> >> git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git >> for-linus > >> >> Has our current queue of fixes. Josef's is the biggest pile, mostly in >> the allocator. Josef and I both managed to merge his patch to avoid >> mapping the extent buffer if skip_locking was set, git merge is just a >> little too easy sometimes (I double checked the resulting code). > > The new in 3.0 btrfs warnings on every build are still there: > > fs/btrfs/sysfs.c:76: warning: ‘btrfs_root_attrs’ defined but not used > fs/btrfs/sysfs.c:97: warning: ‘btrfs_super_attrs’ defined but not used > fs/btrfs/sysfs.c:153: warning: ‘btrfs_super_release’ defined but not used > fs/btrfs/sysfs.c:160: warning: ‘btrfs_root_release’ defined but not used > > These are not even used inside any ifdef. It's unclear to me: were > these supposed to be used or removed? > > Probably better to remove since they seem to be untested, unless > it was a merge error? > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
Excerpts from Andi Kleen's message of 2011-06-12 21:02:54 -0400: > Chris Mason writes: > > > Hi everyone, > > > > The for-linus branch of the btrfs unstable tree: > > > > git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git > > for-linus > > > > > Has our current queue of fixes. Josef's is the biggest pile, mostly in > > the allocator. Josef and I both managed to merge his patch to avoid > > mapping the extent buffer if skip_locking was set, git merge is just a > > little too easy sometimes (I double checked the resulting code). > > The new in 3.0 btrfs warnings on every build are still there: > > fs/btrfs/sysfs.c:76: warning: ‘btrfs_root_attrs’ defined but not used > fs/btrfs/sysfs.c:97: warning: ‘btrfs_super_attrs’ defined but not used > fs/btrfs/sysfs.c:153: warning: ‘btrfs_super_release’ defined but not used > fs/btrfs/sysfs.c:160: warning: ‘btrfs_root_release’ defined but not used > > These are not even used inside any ifdef. It's unclear to me: were > these supposed to be used or removed? > > Probably better to remove since they seem to be untested, unless > it was a merge error? Right, I've been trying to decide how much of the sysfs interface to rip out. We're not using it the way I had planned to, and Kay's proposed udev changes are better than my original plans for sysfs. One way or another I'll kill these off in the next rc. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
Chris Mason writes: > Hi everyone, > > The for-linus branch of the btrfs unstable tree: > > git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git > for-linus > > Has our current queue of fixes. Josef's is the biggest pile, mostly in > the allocator. Josef and I both managed to merge his patch to avoid > mapping the extent buffer if skip_locking was set, git merge is just a > little too easy sometimes (I double checked the resulting code). The new in 3.0 btrfs warnings on every build are still there: fs/btrfs/sysfs.c:76: warning: ‘btrfs_root_attrs’ defined but not used fs/btrfs/sysfs.c:97: warning: ‘btrfs_super_attrs’ defined but not used fs/btrfs/sysfs.c:153: warning: ‘btrfs_super_release’ defined but not used fs/btrfs/sysfs.c:160: warning: ‘btrfs_root_release’ defined but not used These are not even used inside any ifdef. It's unclear to me: were these supposed to be used or removed? Probably better to remove since they seem to be untested, unless it was a merge error? -Andi -- a...@linux.intel.com -- Speaking for myself only -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
On Sun, Jun 12, 2011 at 4:57 AM, Chris Mason wrote: > > The for-linus branch of the btrfs unstable tree: Chris, this is getting ridiculous. You guys need to start honoring the merge window. None of these big pulls afterwards. If the code wasn't ready, it damn well shouldn't have been pushed to me in the first place. I pulled this round, but next pull request had better make it very clear that it's only minimal *regression* fixes. Nothing more. Linus -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi everyone, The for-linus branch of the btrfs unstable tree: git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git for-linus Has our current queue of fixes. Josef's is the biggest pile, mostly in the allocator. Josef and I both managed to merge his patch to avoid mapping the extent buffer if skip_locking was set, git merge is just a little too easy sometimes (I double checked the resulting code). We've also got some fixes for the new scrub code, and a regression fix for the new use of current->journal_info in nested transactions. Josef Bacik (9) commits (+178/-65): Btrfs: don't commit the transaction if we dont have enough pinned bytes (+7/-0) Btrfs: don't map extent buffer if path->skip_locking is set (+7/-3) Btrfs: cache bitmaps when searching for a cluster (+49/-5) Btrfs: noinline the cluster searching functions (+10/-8) Btrfs: unlock the trans lock properly (+1/-1) Btrfs: fix the allocator loop logic (+25/-23) Btrfs: fix duplicate checking logic (+3/-3) Btrfs: fix bitmap regression (+69/-19) Arne Jansen (3) commits (+39/-32): btrfs: remove unneeded includes from scrub.c (+0/-6) btrfs: scrub: errors in tree enumeration (+34/-23) btrfs: reinitialize scrub workers (+5/-3) Li Zefan (2) commits (+15/-10): Btrfs: use join_transaction in btrfs_evict_inode() (+1/-1) Btrfs: avoid stack bloat in btrfs_ioctl_fs_info() (+14/-9) Chris Mason (1) commits (+5/-4): Btrfs: make sure to recheck for bitmaps in clusters Sage Weil (1) commits (+4/-1): Btrfs: clear current->journal_info on async transaction commit Ilya Dryomov (1) commits (+2/-6): Btrfs - use %pU to print fsid Jan Schmidt (1) commits (+1/-1): Btrfs: fix extent state leak on failed nodatasum reads David Sterba (1) commits (+1/-2): btrfs: fix unlocked access of delalloc_inodes richard kennedy (1) commits (+1/-1): btrfs: remove 64bit alignment padding to allow extent_buffer to fit into one fewer cacheline Total: (20) commits (+246/-122) fs/btrfs/ctree.c| 10 ++- fs/btrfs/disk-io.c |5 +- fs/btrfs/extent-tree.c | 55 +-- fs/btrfs/extent_io.h|2 +- fs/btrfs/free-space-cache.c | 163 ++- fs/btrfs/inode.c|4 +- fs/btrfs/ioctl.c| 23 --- fs/btrfs/scrub.c| 69 ++ fs/btrfs/transaction.c |7 ++- fs/btrfs/volumes.c |8 +-- 10 files changed, 233 insertions(+), 113 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi everyone, The for-linus branch of the btrfs unstable repo: git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git for-linus Has our collection of fixes. It's a little bigger than usual for rc2 because it includes Josef's queue of Btrfs changes. It seemed best to split them so we could concentrate on looking for any issues in the new btrfs rc1 code from Fujitsu. His tree is bug fixes and journal lock reduction. Some people have reported the initial caching of the free inode number map (which happens only once when it is first enabled) is sucking down too much CPU and IO time on their systems. We don't have that one fixed yet, but this pull does clean up a few other problems in the new inode number allocatgor. It also turns it off by default (mount -o inode_cache to enable). I was on the fence for turning this on by default, but we've already kicked out three bugs so it seems best to keep it optional until 3.1. Josef Bacik (15) commits (+478/-386): Btrfs: don't try to allocate from a block group that doesn't have enough space (+8/-0) Btrfs: take away the num_items argument from btrfs_join_transaction (+42/-48) Btrfs: make sure to use the delalloc reserve when filling delalloc (+2/-0) Btrfs: don't save the inode cache if we are deleting this root (+5/-0) Btrfs: don't look at the extent buffer level 3 times in a row (+0/-3) Btrfs: map the node block when looking for readahead targets (+21/-2) Btrfs: set range_start to the right start in count_range_bits (+1/-1) Btrfs: if we've already started a trans handle, use that one (+19/-0) Btrfs: check for duplicate entries in the free space cache (+24/-3) Btrfs: try not to sleep as much when doing slow caching (+11/-8) Btrfs: fix how we do space reservation for truncate (+123/-37) Btrfs: leave spinning on lookup and map the leaf (+12/-0) Btrfs: kill BTRFS_I(inode)->block_group (+13/-110) Btrfs: don't always do readahead (+20/-5) Btrfs: kill trans_mutex (+177/-169) Chris Mason (3) commits (+54/-9): Btrfs: make sure we don't overflow the free space cache crc page (+19/-8) Btrfs: fix uninit variable in the delayed inode code (+1/-0) Btrfs: add mount -o inode_cache (+34/-1) David Sterba (3) commits (+26/-21): btrfs: use btrfs_ino to access inode number (+5/-4) btrfs: fix uninitialized variable warning (+1/-1) btrfs: add helper for fs_info->closing (+20/-16) Arne Jansen (3) commits (+70/-53): btrfs: scrub: don't reuse bios and pages (+65/-49) btrfs: scrub: add explicit plugging (+4/-3) btrfs: false BUG_ON when degraded (+1/-1) liubo (1) commits (+6/-0): Btrfs: don't save the inode cache in non-FS roots Total: (25) commits fs/btrfs/btrfs_inode.h |3 - fs/btrfs/ctree.c| 28 +++- fs/btrfs/ctree.h| 22 +++- fs/btrfs/delayed-inode.c|8 +- fs/btrfs/disk-io.c | 36 +++--- fs/btrfs/extent-tree.c | 103 ++- fs/btrfs/extent_io.c|2 +- fs/btrfs/file.c | 10 +- fs/btrfs/free-space-cache.c | 70 --- fs/btrfs/inode-map.c| 34 +- fs/btrfs/inode.c| 261 +++-- fs/btrfs/ioctl.c| 26 ++--- fs/btrfs/relocation.c | 34 +++-- fs/btrfs/scrub.c| 123 ++ fs/btrfs/super.c|8 +- fs/btrfs/transaction.c | 302 +++ fs/btrfs/transaction.h | 29 +--- fs/btrfs/volumes.c |2 +- fs/btrfs/xattr.c|2 - 19 files changed, 635 insertions(+), 468 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
One question. Will the autodefrag option be snapshot aware? Would enabling this option double the amount of used space if there is a snapshot present? On Fri, May 27, 2011 at 2:55 PM, Chris Mason wrote: > > Hi everyone, > > I always thought that I'd be retired and with my flying car at the > beach by the time 3.0 came out, but I've setup the for-linus branch of > the btrfs-unstable tree for pulling: > > git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git > for-linus > > This pull request is probably the biggest I've sent, but it isn't > a code dump into our shiny new .0 rc. Â The bulk of the changes are > three separate projects that have been going on for 6-8 months: > > Â Â Â Â A new btrfs inode allocation cache (Li Zefan) > Â Â Â Â Delayed metadata insertion into the btree (Miao Xe) > Â Â Â Â Device scrubbing (Arne Jansen) > > On top of that Dave Sterba kicked in a series of code cleanups and Josef > Bacik did some really nice tuning. Â The short log lists a few other > cleanups and fixes too. > > I coded up a mount -o autodefrag that will detect random writes > into existing files and kick off background defragging. Â It is well > suited to bdb or sqlite databases, but not virtualization images or big > databases (yet). Â Once I make sure it doesn't defrag files over and over > again, I'll move this toward the default. > > David Sterba (17) commits (+310/-3253): > Â Â btrfs: rename variables clashing with global function names (+53/-53) > Â Â btrfs: use printk_ratelimited instead of printk_ratelimit (+10/-24) > Â Â btrfs: drop unused parameter from btrfs_release_path (+160/-160) > Â Â btrfs: drop unused parameter from extent_map_tree_init (+5/-7) > Â Â btrfs: drop unused argument from extent_io_tree_init (+10/-12) > Â Â btrfs: remove nested duplicate variable declarations (+0/-4) > Â Â btrfs: drop gfp parameter from alloc_extent_buffer (+7/-9) > Â Â btrfs: drop gfp parameter from find_extent_buffer (+4/-6) > Â Â btrfs: drop gfp parameter from alloc_extent_map (+16/-17) > Â Â btrfs: use unsigned type for single bit bitfield (+4/-4) > Â Â btrfs: remove old unused commented out code (+1/-2071) > Â Â btrfs: Document a mutex lock/unlock sequence (+12/-0) > Â Â btrfs: make functions static when possible (+7/-7) > Â Â btrfs: unify checking of IS_ERR and null (+17/-17) > Â Â btrfs: remove unused function prototypes (+0/-43) > Â Â btrfs: remove all unused functions (+1/-817) > Â Â btrfs: fix dereference before check (+3/-2) > > Li Zefan (8) commits (+1449/-665): > Â Â Btrfs: Make the code for reading/writing free space cache generic > (+204/-154) > Â Â Btrfs: setup free ino caching in a more asynchronous way (+22/-6) > Â Â Btrfs: Support reading/writing on disk free ino cache (+236/-19) > Â Â Btrfs: Remove unused btrfs_block_group_free_space() (+0/-16) > Â Â Btrfs: Make free space cache code generic (+271/-223) > Â Â Btrfs: Cache free inode numbers in memory (+500/-53) > Â Â Btrfs: Always use 64bit inode number (+208/-182) > Â Â Btrfs: Use bitmap_set/clear() (+8/-12) > > Xiao Guangrong (7) commits (+134/-59): > Â Â Btrfs: allocate extent state and check the result properly (+26/-8) > Â Â Btrfs: using rcu lock in the reader side of devices list (+72/-36) > Â Â Btrfs: fix the race between reading and updating devices (+9/-0) > Â Â Btrfs: fix the race between remove dev and alloc chunk (+6/-0) > Â Â Btrfs: fix bh leak on __btrfs_open_devices path (+1/-0) > Â Â Btrfs: fix unsafe usage of merge_state (+14/-8) > Â Â Btrfs: drop unnecessary device lock (+6/-7) > > Arne Jansen (6) commits (+1822/-361): > Â Â btrfs scrub: don't coalesce pages that are logically discontiguous (+2/-1) > Â Â btrfs: move btrfs_cmp_device_free_bytes to super.c (+26/-28) > Â Â btrfs: quasi-round-robin for chunk allocation (+177/-305) > Â Â btrfs: add readonly flag (+16/-12) > Â Â btrfs: heed alloc_start (+1/-4) > Â Â btrfs: scrub (+1600/-11) > > Tsutomu Itoh (5) commits (+43/-36): > Â Â Btrfs: BUG_ON is deleted from the caller of btrfs_truncate_item & > btrfs_extend_item (+2/-17) > Â Â Btrfs: return error code to caller when btrfs_previous_item fails (+3/-2) > Â Â Btrfs: return error code to caller when btrfs_del_item fails (+19/-11) > Â Â Btrfs: return error to caller if read_one_inode() fails (+18/-6) > Â Â Btrfs: check return value of btrfs_inc_extent_ref() (+1/-0) > > Chris Mason (4) commits (+689/-144): > Â Â Btrfs: update the delayed inode code to use the btrfs_ino helper. (+7/-6) > Â Â Btrfs: use the device_list_mutex during write_dev_supers (+2/-2) > Â Â Btrfs: return -ENOMEM in clear_extent_bit (+2/-1) > Â Â Btrfs: add mount -o auto_defrag (+678/-135) > > Sergei Trofimovich (3) commits (+7/-3): > Â Â btrfs: don't spin in shrink_delalloc if there is nothing to free (+4/-0) > Â Â btrfs: fix typo 'testeing' -> 'testing' (+2/-2) > Â Â btrfs: typo: 'btrfS' -> 'btrfs' (+1/-1) > > Jan Schmidt (1) commits (+169/-2): > Â Â btrfs: new ioctls for scrub > > liubo (1) commits (+3/-0): > Â Â Btrfs: do not flush csum items of unchanged file data during treelog > > Miao Xie (1) commi
[GIT PULL] Btrfs updates
Hi everyone, I always thought that I'd be retired and with my flying car at the beach by the time 3.0 came out, but I've setup the for-linus branch of the btrfs-unstable tree for pulling: git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git for-linus This pull request is probably the biggest I've sent, but it isn't a code dump into our shiny new .0 rc. The bulk of the changes are three separate projects that have been going on for 6-8 months: A new btrfs inode allocation cache (Li Zefan) Delayed metadata insertion into the btree (Miao Xe) Device scrubbing (Arne Jansen) On top of that Dave Sterba kicked in a series of code cleanups and Josef Bacik did some really nice tuning. The short log lists a few other cleanups and fixes too. I coded up a mount -o autodefrag that will detect random writes into existing files and kick off background defragging. It is well suited to bdb or sqlite databases, but not virtualization images or big databases (yet). Once I make sure it doesn't defrag files over and over again, I'll move this toward the default. David Sterba (17) commits (+310/-3253): btrfs: rename variables clashing with global function names (+53/-53) btrfs: use printk_ratelimited instead of printk_ratelimit (+10/-24) btrfs: drop unused parameter from btrfs_release_path (+160/-160) btrfs: drop unused parameter from extent_map_tree_init (+5/-7) btrfs: drop unused argument from extent_io_tree_init (+10/-12) btrfs: remove nested duplicate variable declarations (+0/-4) btrfs: drop gfp parameter from alloc_extent_buffer (+7/-9) btrfs: drop gfp parameter from find_extent_buffer (+4/-6) btrfs: drop gfp parameter from alloc_extent_map (+16/-17) btrfs: use unsigned type for single bit bitfield (+4/-4) btrfs: remove old unused commented out code (+1/-2071) btrfs: Document a mutex lock/unlock sequence (+12/-0) btrfs: make functions static when possible (+7/-7) btrfs: unify checking of IS_ERR and null (+17/-17) btrfs: remove unused function prototypes (+0/-43) btrfs: remove all unused functions (+1/-817) btrfs: fix dereference before check (+3/-2) Li Zefan (8) commits (+1449/-665): Btrfs: Make the code for reading/writing free space cache generic (+204/-154) Btrfs: setup free ino caching in a more asynchronous way (+22/-6) Btrfs: Support reading/writing on disk free ino cache (+236/-19) Btrfs: Remove unused btrfs_block_group_free_space() (+0/-16) Btrfs: Make free space cache code generic (+271/-223) Btrfs: Cache free inode numbers in memory (+500/-53) Btrfs: Always use 64bit inode number (+208/-182) Btrfs: Use bitmap_set/clear() (+8/-12) Xiao Guangrong (7) commits (+134/-59): Btrfs: allocate extent state and check the result properly (+26/-8) Btrfs: using rcu lock in the reader side of devices list (+72/-36) Btrfs: fix the race between reading and updating devices (+9/-0) Btrfs: fix the race between remove dev and alloc chunk (+6/-0) Btrfs: fix bh leak on __btrfs_open_devices path (+1/-0) Btrfs: fix unsafe usage of merge_state (+14/-8) Btrfs: drop unnecessary device lock (+6/-7) Arne Jansen (6) commits (+1822/-361): btrfs scrub: don't coalesce pages that are logically discontiguous (+2/-1) btrfs: move btrfs_cmp_device_free_bytes to super.c (+26/-28) btrfs: quasi-round-robin for chunk allocation (+177/-305) btrfs: add readonly flag (+16/-12) btrfs: heed alloc_start (+1/-4) btrfs: scrub (+1600/-11) Tsutomu Itoh (5) commits (+43/-36): Btrfs: BUG_ON is deleted from the caller of btrfs_truncate_item & btrfs_extend_item (+2/-17) Btrfs: return error code to caller when btrfs_previous_item fails (+3/-2) Btrfs: return error code to caller when btrfs_del_item fails (+19/-11) Btrfs: return error to caller if read_one_inode() fails (+18/-6) Btrfs: check return value of btrfs_inc_extent_ref() (+1/-0) Chris Mason (4) commits (+689/-144): Btrfs: update the delayed inode code to use the btrfs_ino helper. (+7/-6) Btrfs: use the device_list_mutex during write_dev_supers (+2/-2) Btrfs: return -ENOMEM in clear_extent_bit (+2/-1) Btrfs: add mount -o auto_defrag (+678/-135) Sergei Trofimovich (3) commits (+7/-3): btrfs: don't spin in shrink_delalloc if there is nothing to free (+4/-0) btrfs: fix typo 'testeing' -> 'testing' (+2/-2) btrfs: typo: 'btrfS' -> 'btrfs' (+1/-1) Jan Schmidt (1) commits (+169/-2): btrfs: new ioctls for scrub liubo (1) commits (+3/-0): Btrfs: do not flush csum items of unchanged file data during treelog Miao Xie (1) commits (+2074/-91): btrfs: implement delayed inode items operation Julia Lawall (1) commits (+4/-1): fs/btrfs: Add missing btrfs_free_path Andi Kleen (1) commits (+0/-4): BTRFS: Remove unused node_lock Ilya Dryomov (1) commits (+80/-207): btrfs scrub: make fixups sync Jamey Sharp (1) commits (+0/-43): btrfs: Delete
Re: [GIT PULL] Btrfs updates
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Any plans to update the wiki ? https://btrfs.wiki.kernel.org/index.php/Changelog :P On 05/15/11 at 10:47am, Chris Mason wrote: > > Hi everyone, > > The master branch of the btrfs unstable tree has a few more fixes: > > git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git master > > These include small fixes in the new per-file flags, an oops in > the btrfs acl code and ENOSPC fixes for mixed block groups (used in very > small filesystems). > > Li Zefan (3) commits (+15/-10): > Btrfs: fix FS_IOC_SETFLAGS ioctl (+2/-0) > Btrfs: fix FS_IOC_GETFLAGS ioctl (+7/-0) > fs: remove FS_COW_FL (+6/-10) > > Daniel J Blueman (1) commits (+3/-2): > Prevent oopsing in posix_acl_valid() > > liubo (1) commits (+26/-11): > Btrfs: fix easily get into ENOSPC in mixed case > > Total: (5) commits (+44/-23) > > fs/btrfs/acl.c |5 +++-- > fs/btrfs/extent-tree.c | 37 ++--- > fs/btrfs/ioctl.c | 24 +++- > include/linux/fs.h |1 - > 4 files changed, 44 insertions(+), 23 deletions(-) > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iQIcBAEBAgAGBQJNz/RGAAoJEFWnpyuyLRXa6M8P/0Ev5byjWO7bRheomupHtUUe Oo4OVsQAN7dvD/GSfRCoCEnNEFAn70wNnARW4bPgWC8jvM5HSPx+fFc4u4URBS6q 3mZUEyB2XMsAKqrDiXuIHgsUFcCZWW9IwhS5q8vMJ6F5ZZR9lIVt1Uh8l9qghdNh 6pdohUCJ6gRQZyaY+P/Cv0eZJ1KdOcnmqL7PU8EMS/0OZWAgE/SMoYBuiX8qftTP jiNdxlIATgVeUA1r7Q7qaBiYQ4x76RuzINe5rtLK04Gg5eYZ1FPyrWZc12/347u3 soOcGKsKKRQ7WvSDnaK2ecz4r5XaeL7s5lnGFcfVlP+71qPpGe6i/JVQKR7+AZpN gBSbXAO/VkkW8mx2BbNNTx93LFkr962XyLzS44mzik4VzPdZ7QM5Bd3q2sWQfRKO ePHoHiH5ZeDsukLZw4cHKVbsHr1ZcaBn1vEqof047Sy79BlEuy0wLr1R7kG/FZMy S+y1gU3gku+3BMb+NuHJwWEFVx1zmAhp/r9fRdyOaHmgxdBsi0bDtCuYPoSxl68O 2m9WlJNxBkuMZ66UAYnhbX97x/WCOvNByLCTzFm25H1ISzHgF2z/pbuK2iz+OGG9 I+vDFX06LJcc/NTOx2dv86Wxaw2MsEOXNoBomIXvn2O+8IeltTLbxd/+8NGavMFc kw0DNwrL8tMx3M7nVfbD =OD9q -END PGP SIGNATURE- -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi everyone, The master branch of the btrfs unstable tree has a few more fixes: git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git master These include small fixes in the new per-file flags, an oops in the btrfs acl code and ENOSPC fixes for mixed block groups (used in very small filesystems). Li Zefan (3) commits (+15/-10): Btrfs: fix FS_IOC_SETFLAGS ioctl (+2/-0) Btrfs: fix FS_IOC_GETFLAGS ioctl (+7/-0) fs: remove FS_COW_FL (+6/-10) Daniel J Blueman (1) commits (+3/-2): Prevent oopsing in posix_acl_valid() liubo (1) commits (+26/-11): Btrfs: fix easily get into ENOSPC in mixed case Total: (5) commits (+44/-23) fs/btrfs/acl.c |5 +++-- fs/btrfs/extent-tree.c | 37 ++--- fs/btrfs/ioctl.c | 24 +++- include/linux/fs.h |1 - 4 files changed, 44 insertions(+), 23 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi everyone, The master branch of the btrfs unstable repo: git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git master Has a few bug fixes. Josef fixed up a crash in the free space cache error handling code, and it should close out the open bugs there. Li Zefan found a slab corruption in btrfs fitrim support. Otherwise, these are smaller fixes for corner cases. This is a separate pull from my plugging fixes because this is against 2.6.38, while the plugging stuff was on top of 2.6.39-rc4. David Sterba (1) commits (+1/-0): btrfs: add missing spin_unlock to a rare exit path Itaru Kitayama (1) commits (+1/-1): btrfs: fix wrong allocating flag when reading page Josef Bacik (2) commits (+8/-5): Btrfs: free bitmaps properly when evicting the cache (+7/-4) Btrfs: put the right bio if we have an error (+1/-1) Li Zefan (1) commits (+1/-1): Btrfs: Free free_space item properly in btrfs_trim_block_group() Tsutomu Itoh (3) commits (+21/-8): Btrfs: fix missing mutex_unlock in btrfs_del_dir_entries_in_log() (+5/-2) Btrfs: cleanup error handling in inode.c (+9/-6) Btrfs: check return value of kmalloc() (+7/-0) Total: (8) commits (+32/-15) fs/btrfs/disk-io.c |1 + fs/btrfs/extent-tree.c |4 fs/btrfs/extent_io.c|2 +- fs/btrfs/free-space-cache.c | 13 - fs/btrfs/inode.c| 20 +--- fs/btrfs/tree-log.c |7 +-- 6 files changed, 32 insertions(+), 15 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi everyone, The master branch of the btrfs unstable repo: git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git master Has a nice pile of btrfs fixes. Josef tackled a variety of DIO problems including latencies and crc errors. He also fixed a crash in the mount -o free_space_cache mode, and dramatically reduced contention on the journal mutex. Less contention means we finish up endio processing faster and we're able to process delalloc much faster as space gets tight. Our ENOSPC fixes have returned. The cluster allocation code was sometimes getting out of hand and leaving us with a filesystem full of metadata chunks. Fujitsu continues a long series of error handling improvements and fixes, and Sergei Trofimovich fixed a memcpy vs memmove bug. I've removed a duplicate commit from the shortlog below. Josef and I both managed to take his iov_base fix. Git did the right thing and the resulting code is correct. Josef Bacik (10) commits (+224/-133): Btrfs: reuse the extent_map we found when calling btrfs_get_extent (+28/-8) Btrfs: do not call btrfs_update_inode in endio if nothing changed (+10/-5) Btrfs: deal with the case that we run out of space in the cache (+69/-74) Btrfs: avoid taking the trans_mutex in btrfs_end_transaction (+19/-24) Btrfs: check for duplicate iov_base's when doing dio reads (+16/-1) Btrfs: avoid taking the chunk_mutex in do_chunk_alloc (+28/-6) Btrfs: map the inode item when doing fill_inode_item (+12/-0) Btrfs: do not use async submit for small DIO io's (+19/-5) Btrfs: don't split dio bios if we don't have to (+13/-7) Btrfs: only retry transaction reservation once (+10/-1) Chris Mason (4) commits (+78/-29): Btrfs end_bio_extent_readpage should look for locked bits (+1/-1) Btrfs: don't force chunk allocation in find_free_extent (+73/-22) Btrfs: make uncache_state unconditional (+3/-5) Btrfs: fix free space cache leak (+1/-1) Li Zefan (2) commits (+22/-39): Btrfs: Check if btrfs_next_leaf() returns error in btrfs_real_readdir() (+10/-18) Btrfs: Check if btrfs_next_leaf() returns error in btrfs_listxattr() (+12/-21) Miao Xie (2) commits (+8/-7): Btrfs: Fix incorrect inode nlink in btrfs_link() (+3/-3) Btrfs: Check validity before setting an acl (+5/-4) Sergei Trofimovich (1) commits (+11/-3): btrfs: properly handle overlapping areas in memmove_extent_buffer Yoshinori Sano (1) commits (+4/-1): Btrfs: fix memory leaks in btrfs_new_inode() Xin Zhong (1) commits (+33/-9): Btrfs: fix subvolume mount by name problem when default mount subvolume is set Daniel J Blueman (1) commits (+1/-1): fix user annotation in ioctl.c Arne Jansen (1) commits (+55/-19): btrfs: using cached extent_state in set/unlock combinations Total: (24) commits (+452/-240) fs/btrfs/acl.c |9 ++- fs/btrfs/ctree.h|9 ++- fs/btrfs/disk-io.c |2 +- fs/btrfs/extent-tree.c | 125 +--- fs/btrfs/extent_io.c| 82 - fs/btrfs/extent_io.h|2 +- fs/btrfs/file.c | 21 +++--- fs/btrfs/free-space-cache.c | 119 +++ fs/btrfs/inode.c| 165 ++- fs/btrfs/ioctl.c|2 +- fs/btrfs/super.c| 42 +--- fs/btrfs/transaction.c | 48 +++-- fs/btrfs/transaction.h |4 +- fs/btrfs/xattr.c| 33 +++-- 14 files changed, 430 insertions(+), 233 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates for 2.6.39
Excerpts from David Sterba's message of 2011-03-28 09:25:28 -0400: > Hi, > > Chris, you did not add my sign-of on the unaligned fix I've posted, nor > the tested-by or reported-by. Looking again into the mail, it seems like > some automated-misprocessing picked the first part of the mail altough > there was properly formated patch after '--' marker (and I've verified > this is correct before sending the mail). > > Can you please fix that? > Patch reference: https://patchwork.kernel.org/patch/645671/ Sorry, looks like I messed that one up. patchwork did bring in the comments in a funny way and I didn't notice before the pull request went out. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates for 2.6.39
David Sterba wrote: > Hi, > > Chris, you did not add my sign-of on the unaligned fix I've posted, nor > the tested-by or reported-by. Looking again into the mail, it seems like > some automated-misprocessing picked the first part of the mail altough > there was properly formated patch after '--' marker (and I've verified > this is correct before sending the mail). > > Can you please fix that? > Patch reference: https://patchwork.kernel.org/patch/645671/ > It can't be fixed as Linus has pulled the tree. Fortunetely it's not a big deal, and you are still recorded as the author of that patch. ;) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates for 2.6.39
On Mon, Mar 28, 2011 at 5:08 AM, Chris Mason wrote: > > Linus, I've pushed out two branches for you, for-linus and > for-linus-unmerged Thanks. I took the unmerged one, because I do like to see what's going on. But I did end up verifying the end result against your merge, and it ended up identical, so we're all good. Linus -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates for 2.6.39
Hi, Chris, you did not add my sign-of on the unaligned fix I've posted, nor the tested-by or reported-by. Looking again into the mail, it seems like some automated-misprocessing picked the first part of the mail altough there was properly formated patch after '--' marker (and I've verified this is correct before sending the mail). Can you please fix that? Patch reference: https://patchwork.kernel.org/patch/645671/ thanks, dave -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates for 2.6.39
Hi everyone, The kernel.org mirror is making me pay for a mistake in my push last night. I'm still waiting for it to update the mirrors with the correct commits, so I've got the proper sha down below. Linus, I've pushed out two branches for you, for-linus and for-linus-unmerged git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git for-linus The correct top commit in for-linus: 08aca535a2150e951c6f1d882bea6ef1a1c31822 git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git for-linus-unmerged The correct top commit in for-linus-unmerged: d9d04879321af570ea7285c6dad92d9c3cd108a1 The reason there are two is because there was a conflict in fs/btrfs/volumes.c. Jens' unplugging code and the new btrfs fitrim support were both changing the same code. The merge isn't too bad, his stuff was just removing the tests for an unplugging function, but I wanted to provide a wrapped up merge commit just in case. So, for-linus-unmerged is against 2.6.38, and for-linus is against your git tree as of Friday. The shortlog/diffstat below don't include the merge commit. This pull has per-file controls in btrfs for the compression and nocow flags. This goes in through the standard chattr interface, and Ted did ack the new flags. We also have support for the FITRIM ioctl, which will let us discard in bulk. It will be very interesting to compare the performance of online discard vs bulk trim, and it is possible to run both at the same time. There are also many error handling fixes, allocator performance fixes, code cleanups, a switch to the new truncate handling code, and a few other small fixes. This might be the first pull in a long time without the words ENOSPC in it, but I'm sure we've fixed that too. Josef Bacik (22) commits (+790/-674): Btrfs: check free space in block group before searching for a cluster (+10/-0) Btrfs: deal with min_bytes appropriately when looking for a cluster (+4/-5) Btrfs: make sure to remove the orphan item from the in-memory list (+6/-0) Btrfs: mark the bio with an error if we have a failure in dio (+8/-0) Btrfs: fix how we deal with the pages array in the write path (+5/-4) Btrfs: add a comment explaining what btrfs_cont_expand does (+6/-0) Btrfs: return error if the range we want to map is bogus (+1/-0) Btrfs: check return value of btrfs_search_slot properly (+2/-0) Btrfs: use mark_inode_dirty when expanding the file (+1/-15) Btrfs: cleanup error handling in the truncate path (+45/-19) Btrfs: cleanup how we setup free space clusters (+182/-185) Btrfs: don't allocate dip->csums when doing writes (+5/-2) Btrfs: don't be as aggressive about using bitmaps (+16/-3) Btrfs: add checks to verify dir items are correct (+50/-0) Btrfs: check items for correctness as we search (+95/-124) Btrfs: change reserved_extents to an atomic_t (+29/-21) Btrfs: only add orphan items when truncating (+18/-27) Btrfs: handle errors in btrfs_orphan_cleanup (+50/-23) Btrfs: use a slab for the free space entries (+29/-16) Btrfs: convert to the new truncate sequence (+40/-47) Btrfs: simplify our write path (+180/-175) Btrfs: fix formatting in file.c (+8/-8) liubo (6) commits (+779/-36): Btrfs: fix memory leak of empty filesystem after balance (+6/-0) Btrfs: fix OOPS of empty filesystem after balance (+30/-0) Btrfs: add initial tracepoint support for btrfs (+727/-11) btrfs: make inode ref log recovery faster (+11/-24) Btrfs: fix return value of setflags ioctl (+3/-1) Btrfs: add datacow flag in inode flag (+2/-0) Li Dongyang (4) commits (+359/-51): Btrfs: make btrfs_map_block() return entire free extent for each device of RAID0/1/10/DUP (+129/-22) Btrfs: adjust btrfs_discard_extent() return errors and trimmed bytes (+31/-19) Btrfs: add btrfs_trim_fs() to handle FITRIM (+190/-1) Btrfs: make update_reserved_bytes() public (+9/-9) Tsutomu Itoh (3) commits (+96/-37): Btrfs: check return value of btrfs_alloc_path() (+27/-14) Btrfs: check return value of read_tree_block() (+15/-0) Btrfs: cleanup some BUG_ON() (+54/-23) Miao Xie (2) commits (+6/-3): btrfs: fix possible deadlock by clearing __GFP_FS flag (+4/-1) btrfs: use GFP_NOFS instead of GFP_KERNEL (+2/-2) Chris Mason (2) commits (+28/-25): Btrfs: use RCU instead of a spinlock to protect the root node (+8/-19) Btrfs: fix __btrfs_map_block on 32 bit machines (+20/-6) Al Viro (2) commits (+28/-11): btrfs: don't mess with i_nlink of unlocked inode in rename() (+25/-11) btrfs: check link counter overflow in link(2) (+3/-0) Yoshinori Sano (1) commits (+12/-0): Btrfs: fix uncheck memory allocations Mark Fasheh (1) commits (+1/-1): btrfs: return EXDEV when linking from different subvolumes Liu Bo (1) commits (+72/-7): Btrfs: Per file/directory controls for COW and compression David Sterba (1) commits (+2/-1): btrfs: properly access unaligned checksum bu
[GIT PULL] Btrfs updates for -rc1
Hey Chris, Please pull from git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-work.git for-chris It is based on v2.6.38. Mostly just cleanups and fixes, the biggest actual change would be moving us to the new truncate sequence. Here is the shortlog and such Josef Bacik (13): Btrfs: fix formatting in file.c Btrfs: simplify our write path Btrfs: fix how we deal with the pages array in the write path Btrfs: change reserved_extents to an atomic_t Btrfs: use a slab for the free space entries Btrfs: convert to the new truncate sequence Btrfs: cleanup error handling in the truncate path Btrfs: handle errors in btrfs_orphan_cleanup Btrfs: make sure to remove the orphan item from the in-memory list Btrfs: only add orphan items when truncating Btrfs: use mark_inode_dirty when expanding the file Btrfs: add a comment explaining what btrfs_cont_expand does Btrfs: return error if the range we want to map is bogus fs/btrfs/btrfs_inode.h |3 +- fs/btrfs/ctree.h|5 +- fs/btrfs/disk-io.c | 15 ++- fs/btrfs/extent-tree.c | 45 +++-- fs/btrfs/extent_io.c|1 + fs/btrfs/file.c | 385 ++- fs/btrfs/free-space-cache.c | 34 ++-- fs/btrfs/inode.c| 241 +++ fs/btrfs/ioctl.c|4 +- fs/btrfs/relocation.c |2 +- 10 files changed, 399 insertions(+), 336 deletions(-) Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
On Lunes, 17 de Enero de 2011 22:13:01 Chris Mason escribió: > Li Zefan also added readonly snapshot support, and I'll have the > corresponding btrfs-progs changes integrated this week. I think you forgot this. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi everyone, The master branch of the btrfs unstable tree has some important btrfs fixes: git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git master I was seeing very rare metadata corruptions during long stress runs, and eventually tracked it down to two different races in the btrfs releasepage code. One was making btrfs think pages were up to date when they really weren't and the other lead to corrupted csum fields in metadata. Chris Mason (2) commits (+50/-6): Btrfs: don't release pages when we can't clear the uptodate bits (+9/-1) Btrfs: fix page->private races (+41/-5) Zheng Yan (1) commits (+1/-0): Btrfs: Fix balance panic Dan Rosenberg (1) commits (+8/-2): btrfs: prevent heap corruption in btrfs_ioctl_space_info() Tsutomu Itoh (1) commits (+7/-3): Btrfs: check return value of alloc_extent_map() Ilya Dryomov (1) commits (+2/-0): Btrfs - Fix memory leak in btrfs_init_new_device() Total: (6) commits (+68/-11) fs/btrfs/disk-io.c |8 ++-- fs/btrfs/extent-tree.c |2 +- fs/btrfs/extent_io.c | 48 fs/btrfs/extent_map.c |4 ++-- fs/btrfs/file.c|1 + fs/btrfs/inode.c |3 +++ fs/btrfs/ioctl.c | 10 -- fs/btrfs/relocation.c |1 + fs/btrfs/volumes.c |2 ++ 9 files changed, 68 insertions(+), 11 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
Hallo, Chris, Du meintest am 07.02.11: [...] > So, here is our collection of bug fixes since rc1. We have an > assortment of ENOSPC tweaks, allocator fixes and better error > handling in general. > Linus please pull the master branch of the btrfs-unstable repo: > git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.gi > t master 32 entries in the 2.6.38-rc4 ChangeLog ... Viele Gruesse! Helmut -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi everyone, This pull was delayed by a week because my test rig kept throwing out corruptions. After a long series of bisects, it looks like the hardware is just silently corrupting blocks and btrfs is catching it early and often. So, here is our collection of bug fixes since rc1. We have an assortment of ENOSPC tweaks, allocator fixes and better error handling in general. Linus please pull the master branch of the btrfs-unstable repo: git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git master Li Zefan (8) commits (+107/-65): btrfs: Fix threshold calculation for block groups smaller than 1GB (+6/-2) btrfs: Check mergeable free space when removing a cluster (+20/-6) btrfs: Update stats when allocating from a cluster (+14/-3) Btrfs: Fix file clone when source offset is not 0 (+4/-1) btrfs: Add a helper try_merge_free_space() (+43/-32) btrfs: Free fully occupied bitmap in cluster (+2/-0) btrfs: Add helper function free_bitmap() (+16/-21) Btrfs: Fix memory leak at umount (+2/-0) Josef Bacik (8) commits (+86/-18): Btrfs: do not release more reserved bytes to the global_block_rsv than we need (+14/-2) Btrfs: make sure search_bitmap finds something in remove_from_bitmap (+1/-0) Btrfs: use the global block reserve if we cannot reserve space (+27/-1) Btrfs: fix check_path_shared so it returns the right value (+3/-2) Btrfs: exclude super blocks when we read in block groups (+18/-8) Btrfs: handle no memory properly in prepare_pages (+6/-2) Btrfs: make shrink_delalloc a little friendlier (+14/-3) Btrfs: do error checking in btrfs_del_csums (+3/-0) Tsutomu Itoh (5) commits (+100/-21): btrfs: check return value of btrfs_start_ioctl_transaction() properly (+1/-1) btrfs: fix return value check of btrfs_start_transaction() (+37/-6) btrfs: fix return value check of btrfs_join_transaction() (+51/-13) btrfs: cleanup error handling in btrfs_unlink_inode() (+1/-1) btrfs: checking NULL or not in some functions (+10/-0) liubo (3) commits (+59/-4): btrfs: fix uncheck memory allocation in btrfs_submit_compressed_read (+25/-4) btrfs: fix several uncheck memory allocations (+33/-0) btrfs: fix missing break in switch phrase (+1/-0) Miao Xie (2) commits (+7/-0): Btrfs: Don't return acl info when mounting with noacl option (+6/-0) Btrfs: Fix memory leak in writepage fixup work (+1/-0) Chris Mason (2) commits (+3/-2): Btrfs: avoid uninit variable warnings in ordered-data.c (+1/-1) Btrfs: catch errors from btrfs_sync_log (+2/-1) Zheng Yan (1) commits (+4/-4): Btrfs: Fix page count calculation Julia Lawall (1) commits (+1/-1): fs/btrfs/inode.c: Add missing IS_ERR test Tero Roponen (1) commits (+3/-2): Btrfs: Free correct pointer after using strsep Ian Kent (1) commits (+2/-0): Btrfs: Fix memory leak on finding existing super Alexey Charkov (1) commits (+1/-1): btrfs: Drop __exit attribute on btrfs_exit_compress Total: (33) commits (+373/-118) fs/btrfs/acl.c |6 ++ fs/btrfs/compression.c | 27 +++- fs/btrfs/disk-io.c |7 ++ fs/btrfs/export.c |2 + fs/btrfs/extent-tree.c | 98 +- fs/btrfs/extent_io.c|6 +- fs/btrfs/file-item.c|5 ++ fs/btrfs/file.c | 20 -- fs/btrfs/free-space-cache.c | 162 ++ fs/btrfs/inode.c| 35 ++--- fs/btrfs/ioctl.c| 19 -- fs/btrfs/ordered-data.c |2 +- fs/btrfs/print-tree.c |1 + fs/btrfs/relocation.c | 29 +++- fs/btrfs/super.c|9 ++- fs/btrfs/transaction.c |5 ++ fs/btrfs/tree-log.c | 35 +- fs/btrfs/volumes.c | 19 +- 18 files changed, 371 insertions(+), 116 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
On Sat, Jan 22, 2011 at 5:41 PM, Clemens Eisserer wrote: >> E.g. suppose you have a LZO compressed file, then a program rewrites some >> data which is in the middle of the file, and suppose the newly written data >> is less compressible. > > Any idea how this is handled? I would be interested in the answer as well. > > Thank you in advance, Clemens The same way any write is handled: the data is written elsewhere. Compression isn't special in this regard. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
> E.g. suppose you have a LZO compressed file, then a program rewrites some > data which is in the middle of the file, and suppose the newly written data > is less compressible. Any idea how this is handled? I would be interested in the answer as well. Thank you in advance, Clemens -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
On 01/18/2011 04:22 PM, C Anthony Risinger wrote: On Tue, Jan 18, 2011 at 4:14 AM, Felix Blanke wrote: i don't know about the readonly snapshots, but the LZO stuff is a mount option; should be in the pull. and for the record, i'm totally stoked to run LZO on all my btrfs machines (esp. the netbooks/SSDs :-) Excuse the newbie question: how does Btrfs deal with the compressed data not being block aligned anymore? E.g. suppose you have a LZO compressed file, then a program rewrites some data which is in the middle of the file, and suppose the newly written data is less compressible. Btrfs is a copy-on-write FS so this simplifies things a bit, nevertheless it has to support the fact that after the rewrite there will in general be 2 or 3 blocks in the middle of the file which are not completely full: probably the one where old data ends and new data starts will be truncated, a few new blocks will be allocated for newly written data (which won't in general end at a block bounday so this is the second block which is not completely full) and then the place where old data restarts will again be not a full block. Is btrfs capable of that? If yes, calculating offsets in files for random reads is going to be quite a bit slower... Also, if yes, this could significantly impact data deduplication, in both positive and negative ways: positive: (with a lot of work) btrfs could detect and merge equal parts at different offsets in different files, even not block aligned. negative: (without a lot of work) if for some reason one of the first blocks in a file gets truncated and all the rest of the file gets hence shifted forward (but this could probably happen only during the first write of that file, maybe if lzo was turned on and then off again), data deduplication won't probably work for that file... ok this might not be big issue. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
Hi Chris, there are a lot of patches related to the btrfs-progs[*]. Do you think that it is reasonable that you find a bit of time to care about that ? Regards G.Baroncelli [*] For example see my email "[REPOST][BTRFS-PROGS] Repost of some patches" dated 2010-12-05. On 01/18/2011 11:14 AM, Felix Blanke wrote: > Hi, > > just slap me if the question is stupid, but when will all those new features > be added > to the btrfs-progs? > > The last commit is at > "git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs-unstable.git" > is from oct. 2010?!? > > > Regards, > Felix > > > On 17. January 2011 - 16:13, Chris Mason wrote: >> Date:Mon, 17 Jan 2011 16:13:01 -0500 >> From: Chris Mason >> To: Linus Torvalds , Linux Btrfs >> , linux-kernel >> Subject: [GIT PULL] Btrfs updates >> >> Hi everyone, >> >> The for-linus branch of the btrfs-unstable git tree: >> >> git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git >> for-linus >> >> Has our collection of features and fixes for 2.6.38-rc1. The git tree >> is actually against 2.6.36 and these commits have been tested against >> .36, .37 and Linus' current git. There's no particular reason for the >> .36 part, other than it's what one of my build boxes was running. >> >> The highlights this time around are LZO compression from Li Zefan and a >> large collection of multi-device fixes from Miao Xie. Liubo kicked in >> the start of our forced-readonly support, which will be the building >> block for ripping out a big collection of BUG_ONs and replacing them >> with kinder gentler error messages. >> >> Li Zefan also added readonly snapshot support, and I'll have the >> corresponding btrfs-progs changes integrated this week. >> >> This pull does have a duplicate for the btrfs ctree.c build fix that >> Linus already has in his tree. Git merge seems to do magic on this one, >> it applies without doing a duplicate #include. >> >> Li Zefan (9) commits (+1354/-542): >> Btrfs: Add BTRFS_IOC_SUBVOL_GETFLAGS/SETFLAGS ioctls (+85/-0) >> btrfs: Allow to specify compress method when defrag (+26/-2) >> btrfs: Allow to add new compression algorithm (+473/-282) >> btrfs: Extract duplicate decompress code (+115/-194) >> Btrfs: Refactor btrfs_ioctl_snap_create() (+40/-44) >> Btrfs: Add readonly snapshots support (+75/-10) >> btrfs: Add lzo compression support (+527/-8) >> btrfs: Fix error handling in zlib (+8/-0) >> btrfs: Fix bugs in zlib workspace (+5/-2) >> >> Miao Xie (6) commits (+695/-186): >> btrfs: make the chunk allocator utilize the devices better (+300/-103) >> btrfs: try to reclaim some space when chunk allocation fails (+7/-2) >> btrfs: fix wrong free space information of btrfs (+286/-7) >> btrfs: restructure find_free_dev_extent() (+91/-68) >> btrfs: fix wrong calculation of stripe size (+8/-2) >> btrfs: fix wrong data space statistics (+3/-4) >> >> Jesper Juhl (2) commits (+5/-1): >> btrfs: Don't pass NULL ptr to func that may deref it. (+2/-0) >> btrfs: Mem leak in btrfs_get_acl() (+3/-1) >> >> Tsutomu Itoh (2) commits (+17/-0): >> btrfs: Fix memory leak in btrfs_read_fs_root_no_radix() (+1/-0) >> btrfs: check NULL or not (+16/-0) >> >> Stefan Schmidt (1) commits (+1/-0): >> fs/btrfs: Fix build of ctree >> >> Ben Hutchings (1) commits (+4/-0): >> btrfs: Require CAP_SYS_ADMIN for filesystem rebalance >> >> Chris Mason (1) commits (+2/-1): >> Btrfs: fix off by one while setting block groups readonly >> >> Dave Young (1) commits (+8/-4): >> btrfs: mount failure return value fix >> >> liubo (1) commits (+523/-2): >> Btrfs: forced readonly mounts on errors >> >> Josef Bacik (1) commits (+0/-5): >> Btrfs: don't warn if we get ENOSPC in btrfs_block_rsv_check >> >> Total: (25) commits (+2609/-741) >> >> fs/btrfs/Kconfig|2 + >> fs/btrfs/Makefile |2 +- >> fs/btrfs/acl.c |4 +- >> fs/btrfs/btrfs_inode.h |2 +- >> fs/btrfs/compression.c | 329 - >> fs/btrfs/compression.h | 72 -- >> fs/btrfs/ctree.c|8 + >> fs/btrfs/ctree.h| 49 - >> fs/btrfs/disk-io.c | 412 ++- >> fs/btrfs/disk-io.h |1 + >> fs/btrfs/extent-tree.c | 90 ++- &g
Re: [GIT PULL] Btrfs updates
On Tue, Jan 18, 2011 at 11:56 AM, Mitch Harder wrote: > On Tue, Jan 18, 2011 at 9:22 AM, C Anthony Risinger wrote: >> On Tue, Jan 18, 2011 at 4:14 AM, Felix Blanke wrote: >>> Hi, >>> >>> just slap me if the question is stupid, but when will all those new >>> features be added >>> to the btrfs-progs? >>> >>> The last commit is at >>> "git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs-unstable.git" >>> is from oct. 2010?!? >> >> i don't know about the readonly snapshots, but the LZO stuff is a >> mount option; should be in the pull. > > There was also a patch submitted by Li Zefan for LZO support in btrfs-progs: > > http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg06931.html > > If I recall correctly, btrfs-progs will throw out an error once you > use LZO compression since btrfs-progs needs to be made aware of the > INCOMPAT flag in the patch submitted above. > > My local btrfs-progs git repository also incorporates patches to > support mixed data+metadata (which also uses an INCOMPAT flag aleady > in the kernel code) and for the filesystem diskinfo command. > > A quick search for "btrfs-progs" in my Mailing List archive turns up a > number of other btrfs-progs patch proposals since October. ah, indeed. should probably include all that before .38 then... and maybe even cut a .20 btrfs-progs "release" so everyone is on the same page :-) there have been several questions regarding that alone. C Anthony -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
On Tue, Jan 18, 2011 at 9:22 AM, C Anthony Risinger wrote: > On Tue, Jan 18, 2011 at 4:14 AM, Felix Blanke wrote: >> Hi, >> >> just slap me if the question is stupid, but when will all those new features >> be added >> to the btrfs-progs? >> >> The last commit is at >> "git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs-unstable.git" >> is from oct. 2010?!? > > i don't know about the readonly snapshots, but the LZO stuff is a > mount option; should be in the pull. > > and for the record, i'm totally stoked to run LZO on all my btrfs > machines (esp. the netbooks/SSDs :-) > > C Anthony There was also a patch submitted by Li Zefan for LZO support in btrfs-progs: http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg06931.html If I recall correctly, btrfs-progs will throw out an error once you use LZO compression since btrfs-progs needs to be made aware of the INCOMPAT flag in the patch submitted above. My local btrfs-progs git repository also incorporates patches to support mixed data+metadata (which also uses an INCOMPAT flag aleady in the kernel code) and for the filesystem diskinfo command. A quick search for "btrfs-progs" in my Mailing List archive turns up a number of other btrfs-progs patch proposals since October. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
On Tue, Jan 18, 2011 at 4:14 AM, Felix Blanke wrote: > Hi, > > just slap me if the question is stupid, but when will all those new features > be added > to the btrfs-progs? > > The last commit is at > "git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs-unstable.git" > is from oct. 2010?!? i don't know about the readonly snapshots, but the LZO stuff is a mount option; should be in the pull. and for the record, i'm totally stoked to run LZO on all my btrfs machines (esp. the netbooks/SSDs :-) C Anthony -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates
Hi, just slap me if the question is stupid, but when will all those new features be added to the btrfs-progs? The last commit is at "git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs-unstable.git" is from oct. 2010?!? Regards, Felix On 17. January 2011 - 16:13, Chris Mason wrote: > Date: Mon, 17 Jan 2011 16:13:01 -0500 > From: Chris Mason > To: Linus Torvalds , Linux Btrfs > , linux-kernel > Subject: [GIT PULL] Btrfs updates > > Hi everyone, > > The for-linus branch of the btrfs-unstable git tree: > > git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git > for-linus > > Has our collection of features and fixes for 2.6.38-rc1. The git tree > is actually against 2.6.36 and these commits have been tested against > .36, .37 and Linus' current git. There's no particular reason for the > .36 part, other than it's what one of my build boxes was running. > > The highlights this time around are LZO compression from Li Zefan and a > large collection of multi-device fixes from Miao Xie. Liubo kicked in > the start of our forced-readonly support, which will be the building > block for ripping out a big collection of BUG_ONs and replacing them > with kinder gentler error messages. > > Li Zefan also added readonly snapshot support, and I'll have the > corresponding btrfs-progs changes integrated this week. > > This pull does have a duplicate for the btrfs ctree.c build fix that > Linus already has in his tree. Git merge seems to do magic on this one, > it applies without doing a duplicate #include. > > Li Zefan (9) commits (+1354/-542): > Btrfs: Add BTRFS_IOC_SUBVOL_GETFLAGS/SETFLAGS ioctls (+85/-0) > btrfs: Allow to specify compress method when defrag (+26/-2) > btrfs: Allow to add new compression algorithm (+473/-282) > btrfs: Extract duplicate decompress code (+115/-194) > Btrfs: Refactor btrfs_ioctl_snap_create() (+40/-44) > Btrfs: Add readonly snapshots support (+75/-10) > btrfs: Add lzo compression support (+527/-8) > btrfs: Fix error handling in zlib (+8/-0) > btrfs: Fix bugs in zlib workspace (+5/-2) > > Miao Xie (6) commits (+695/-186): > btrfs: make the chunk allocator utilize the devices better (+300/-103) > btrfs: try to reclaim some space when chunk allocation fails (+7/-2) > btrfs: fix wrong free space information of btrfs (+286/-7) > btrfs: restructure find_free_dev_extent() (+91/-68) > btrfs: fix wrong calculation of stripe size (+8/-2) > btrfs: fix wrong data space statistics (+3/-4) > > Jesper Juhl (2) commits (+5/-1): > btrfs: Don't pass NULL ptr to func that may deref it. (+2/-0) > btrfs: Mem leak in btrfs_get_acl() (+3/-1) > > Tsutomu Itoh (2) commits (+17/-0): > btrfs: Fix memory leak in btrfs_read_fs_root_no_radix() (+1/-0) > btrfs: check NULL or not (+16/-0) > > Stefan Schmidt (1) commits (+1/-0): > fs/btrfs: Fix build of ctree > > Ben Hutchings (1) commits (+4/-0): > btrfs: Require CAP_SYS_ADMIN for filesystem rebalance > > Chris Mason (1) commits (+2/-1): > Btrfs: fix off by one while setting block groups readonly > > Dave Young (1) commits (+8/-4): > btrfs: mount failure return value fix > > liubo (1) commits (+523/-2): > Btrfs: forced readonly mounts on errors > > Josef Bacik (1) commits (+0/-5): > Btrfs: don't warn if we get ENOSPC in btrfs_block_rsv_check > > Total: (25) commits (+2609/-741) > > fs/btrfs/Kconfig|2 + > fs/btrfs/Makefile |2 +- > fs/btrfs/acl.c |4 +- > fs/btrfs/btrfs_inode.h |2 +- > fs/btrfs/compression.c | 329 - > fs/btrfs/compression.h | 72 -- > fs/btrfs/ctree.c|8 + > fs/btrfs/ctree.h| 49 - > fs/btrfs/disk-io.c | 412 ++- > fs/btrfs/disk-io.h |1 + > fs/btrfs/extent-tree.c | 90 ++- > fs/btrfs/extent_io.c|7 +- > fs/btrfs/extent_io.h| 17 ++- > fs/btrfs/extent_map.c |2 + > fs/btrfs/extent_map.h |3 +- > fs/btrfs/file.c | 13 + > fs/btrfs/inode.c| 90 +--- > fs/btrfs/ioctl.c| 220 + > fs/btrfs/ioctl.h| 12 +- > fs/btrfs/lzo.c | 420 +++ > fs/btrfs/ordered-data.c | 18 ++- > fs/btrfs/ordered-data.h |8 +- > fs/btrfs/super.c| 281 -- > fs/btrfs/transaction.c | 11 + > fs/btrfs/transaction.h |1 + > fs/btrfs/volumes.c | 626 > ++- > fs/btrfs/volumes.h | 27 ++ > fs/btrfs/xattr.
[GIT PULL] Btrfs updates
Hi everyone, The for-linus branch of the btrfs-unstable git tree: git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git for-linus Has our collection of features and fixes for 2.6.38-rc1. The git tree is actually against 2.6.36 and these commits have been tested against .36, .37 and Linus' current git. There's no particular reason for the .36 part, other than it's what one of my build boxes was running. The highlights this time around are LZO compression from Li Zefan and a large collection of multi-device fixes from Miao Xie. Liubo kicked in the start of our forced-readonly support, which will be the building block for ripping out a big collection of BUG_ONs and replacing them with kinder gentler error messages. Li Zefan also added readonly snapshot support, and I'll have the corresponding btrfs-progs changes integrated this week. This pull does have a duplicate for the btrfs ctree.c build fix that Linus already has in his tree. Git merge seems to do magic on this one, it applies without doing a duplicate #include. Li Zefan (9) commits (+1354/-542): Btrfs: Add BTRFS_IOC_SUBVOL_GETFLAGS/SETFLAGS ioctls (+85/-0) btrfs: Allow to specify compress method when defrag (+26/-2) btrfs: Allow to add new compression algorithm (+473/-282) btrfs: Extract duplicate decompress code (+115/-194) Btrfs: Refactor btrfs_ioctl_snap_create() (+40/-44) Btrfs: Add readonly snapshots support (+75/-10) btrfs: Add lzo compression support (+527/-8) btrfs: Fix error handling in zlib (+8/-0) btrfs: Fix bugs in zlib workspace (+5/-2) Miao Xie (6) commits (+695/-186): btrfs: make the chunk allocator utilize the devices better (+300/-103) btrfs: try to reclaim some space when chunk allocation fails (+7/-2) btrfs: fix wrong free space information of btrfs (+286/-7) btrfs: restructure find_free_dev_extent() (+91/-68) btrfs: fix wrong calculation of stripe size (+8/-2) btrfs: fix wrong data space statistics (+3/-4) Jesper Juhl (2) commits (+5/-1): btrfs: Don't pass NULL ptr to func that may deref it. (+2/-0) btrfs: Mem leak in btrfs_get_acl() (+3/-1) Tsutomu Itoh (2) commits (+17/-0): btrfs: Fix memory leak in btrfs_read_fs_root_no_radix() (+1/-0) btrfs: check NULL or not (+16/-0) Stefan Schmidt (1) commits (+1/-0): fs/btrfs: Fix build of ctree Ben Hutchings (1) commits (+4/-0): btrfs: Require CAP_SYS_ADMIN for filesystem rebalance Chris Mason (1) commits (+2/-1): Btrfs: fix off by one while setting block groups readonly Dave Young (1) commits (+8/-4): btrfs: mount failure return value fix liubo (1) commits (+523/-2): Btrfs: forced readonly mounts on errors Josef Bacik (1) commits (+0/-5): Btrfs: don't warn if we get ENOSPC in btrfs_block_rsv_check Total: (25) commits (+2609/-741) fs/btrfs/Kconfig|2 + fs/btrfs/Makefile |2 +- fs/btrfs/acl.c |4 +- fs/btrfs/btrfs_inode.h |2 +- fs/btrfs/compression.c | 329 - fs/btrfs/compression.h | 72 -- fs/btrfs/ctree.c|8 + fs/btrfs/ctree.h| 49 - fs/btrfs/disk-io.c | 412 ++- fs/btrfs/disk-io.h |1 + fs/btrfs/extent-tree.c | 90 ++- fs/btrfs/extent_io.c|7 +- fs/btrfs/extent_io.h| 17 ++- fs/btrfs/extent_map.c |2 + fs/btrfs/extent_map.h |3 +- fs/btrfs/file.c | 13 + fs/btrfs/inode.c| 90 +--- fs/btrfs/ioctl.c| 220 + fs/btrfs/ioctl.h| 12 +- fs/btrfs/lzo.c | 420 +++ fs/btrfs/ordered-data.c | 18 ++- fs/btrfs/ordered-data.h |8 +- fs/btrfs/super.c| 281 -- fs/btrfs/transaction.c | 11 + fs/btrfs/transaction.h |1 + fs/btrfs/volumes.c | 626 ++- fs/btrfs/volumes.h | 27 ++ fs/btrfs/xattr.c| 18 ++ fs/btrfs/zlib.c | 369 ++-- 29 files changed, 2491 insertions(+), 623 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates
Hi everyone, The master branch of the btrfs unstable tree: git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git master Has an assortment of bug fixes. It includes some corruption fixes when mounting the filesystem in degraded mode, and related oopsen as we try to mount damaged filesystems. The async snapshot ioctl added in rc1 is fixed to properly do synchronous snapshots unless the new async ioctl is used. It is also updated to be ready for the readonly snapshot feature we'll add in 2.6.38. Xin Zhong from Intel pitched in a deadlock during copy_from_user and Josef has fixes for the new fast caching code and a few other bugs. Josef Bacik (5) commits (+38/-21): Btrfs: do not BUG if we fail to remove the orphan item for dead snapshots (+7/-3) Btrfs: do not do fast caching if we are allocating blocks for tree_root (+12/-6) Btrfs: fixup return code for btrfs_del_orphan_item (+5/-1) Btrfs: deal with space cache errors better (+13/-9) Btrfs: fix use after free in O_DIRECT (+1/-2) Chris Mason (3) commits (+59/-5): Btrfs: account for missing devices in RAID allocation profiles (+36/-3) Btrfs: prevent RAID level downgrades when space is low (+19/-1) Btrfs: EIO when we fail to read tree roots (+4/-1) Li Zefan (2) commits (+37/-23): Btrfs: Make async snapshot ioctl more generic (+36/-22) Btrfs: Fix a crash when mounting a subvolume (+1/-1) Zheng Yan (1) commits (+1/-1): Btrfs: Fix page leak in compressed writeback path Xin Zhong (1) commits (+60/-32): Btrfs: pwrite blocked when writing from the mmaped buffer of the same page Sage Weil (1) commits (+11/-9): Btrfs: fix sync subvol/snapshot creation Jan Beulich (1) commits (+5/-7): Btrfs: fix compiler warnings Total: (14) commits (+211/-98) fs/btrfs/disk-io.c | 11 +++-- fs/btrfs/extent-tree.c | 75 +++--- fs/btrfs/file.c | 92 --- fs/btrfs/free-space-cache.c | 12 +++-- fs/btrfs/inode.c| 11 ++--- fs/btrfs/ioctl.c| 56 -- fs/btrfs/ioctl.h| 14 -- fs/btrfs/orphan.c |6 ++- fs/btrfs/super.c|2 +- fs/btrfs/volumes.c | 20 +- fs/btrfs/volumes.h |2 + 11 files changed, 207 insertions(+), 94 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[GIT PULL] Btrfs updates for 2.6.37-rc
Hi everyone, The master branch of the btrfs unstable tree: git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git master Has a collection of btrfs bug fixes. The three most important fixes here address crashes in the btrfs O_DIRECT code, add a migrate_page operation to avoid metadata corruption as btree pages go through migration, and fix up our NFS support. Otherwise we have assorted correctness fixes, many of which were kicked out by xfsqa. -chris Josef Bacik (11) commits (+241/-54): Btrfs: make btrfs_add_nondir take parent inode as an argument (+16/-22) Btrfs: fix typo in fallocate to make it honor actual size (+5/-4) Btrfs: hold i_mutex when calling btrfs_log_dentry_safe (+7/-0) Btrfs: setup blank root and fs_info for mount time (+33/-7) Btrfs: make sure new inode size is ok in fallocate (+4/-0) Btrfs: use dget_parent where we can UPDATED (+43/-12) Btrfs: handle the space_cache option properly (+1/-0) Btrfs: update inode ctime when using links (+1/-0) Btrfs: fix more ESTALE problems with NFS (+1/-0) Btrfs: handle NFS lookups properly (+76/-0) Btrfs: fix fiemap (+54/-9) Chris Mason (4) commits (+124/-9): Btrfs: deal with DIO bios that span more than one ordered extent (+89/-4) Btrfs: avoid NULL pointer deref in try_release_extent_buffer (+4/-2) Btrfs: don't use migrate page without CONFIG_MIGRATION (+6/-1) Btrfs: add migrate page for metadata inode (+25/-2) Li Zefan (3) commits (+6/-6): btrfs: Check if dest_offset is block-size aligned before cloning file (+3/-4) btrfs: Show device attr correctly for symlinks (+1/-0) btrfs: Set file size correctly in file clone (+2/-2) Miao Xie (3) commits (+195/-45): btrfs: cleanup duplicate bio allocating functions (+8/-18) btrfs: fix free dip and dip->csums twice (+3/-6) btrfs: fix panic caused by direct IO (+184/-21) Mariusz Kozlowski (1) commits (+3/-3): btrfs: make 1-bit signed fileds unsigned Arne Jansen (1) commits (+1/-1): btrfs: Fix early enospc because 'unused' calculated with wrong sign. Ian Kent (1) commits (+6/-0): Btrfs - fix race between btrfs_get_sb() and umount Total: (24) commits (+576/-118) fs/btrfs/compression.c | 15 +--- fs/btrfs/ctree.h|6 +- fs/btrfs/disk-io.c | 38 +- fs/btrfs/export.c | 76 fs/btrfs/extent-tree.c |2 +- fs/btrfs/extent_io.c| 77 ++--- fs/btrfs/extent_io.h|3 + fs/btrfs/file.c |7 + fs/btrfs/inode.c| 294 ++- fs/btrfs/ioctl.c| 31 -- fs/btrfs/ordered-data.c | 67 +++ fs/btrfs/ordered-data.h |3 + fs/btrfs/super.c| 41 ++- fs/btrfs/transaction.c |5 +- fs/btrfs/tree-log.c | 21 +++- 15 files changed, 572 insertions(+), 114 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] Btrfs updates for 2.6.37
On Sat, Oct 30, 2010 at 6:51 AM, Chris Mason wrote: > > There were some minor conflicts with Linus' current tree, so my branch > is merged with Linus' tree as of this morning. Gaah. Please don't do this. Unless it's a _really_ messy merge, I really do want to do the merge. It's fine to have an alternate pre-merged branch for me to compare against, but please do that separately. So what I did was to just instead merge the state before your merge, and in the process I: (a) noticed that your merge was incorrect (you had left around a unused "error:" label in btrfs_mount()), since I did use your merge as something to compare against (see above). That label had been removed in your branch by commit 0e78340f3c1f, but your merge resurrected it. (b) saw just how horribly nasty your writeback_inodes_sb() end result was, and decided to clean up the estimation of dirty pages in order to not end up with the function call argument from hell. Now, it's obviously totally possible that I screwed things up entirely in the process, but as mentioned elsewhere, I do feel that actually seeing the merge conflicts really does help me get a feel for what I'm merging, and what the points of conflict are. And yes, maybe it's just me showing my insecurities again. I have various mental hangups, and liking to feel like I know roughly what is going on is one of them. Doing the merges and looking at the code that clashes makes me feel like I have some kind of awareness of how things are interacting in the development process. Linus -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
