Re: weird bash autocomplete issue
On Tue, 2008-12-16 at 22:41 +0100, Kay Sievers wrote:
> On Tue, Dec 16, 2008 at 21:46, wrote:
> >> On Tue, Dec 16, 2008 at 20:37, Roland wrote:
> >> > i have come across a weird autocomplete issue i assume it is related to
> >> > btrfs.
> >> >
> >> > let`s have some dirs:
> >> >
> >> > /non-btrfs-mount
> >> > ./linux
> >> > ./testdir
> >> >
> >> > /brtfs-mount
> >> > ./linux
> >> > ./testdir
> >> >
> >> > now, if i do "cd t" in /non-btrfs-mount, "t" autocompletes to
> >> > "testdir"
> >> > same for linux - bash autocompletes as expected.
> >> >
> >> > now, the weird thing is, that on /btrfs-mount this behaves different.
> >> >
> >> > autocompletion for testdir works, but not for linux dir. weird.
> >> >
> >> > can someone reproduce this ?
> >>
> >> Open another shell, find the bash process pid of the first shell with:
> >> ps afx
> >> and do:
> >> strace -p
> >> Go back to the first shell, hit , and the trace should show
> >> what's going on. You see a significant difference there?
> >
> >
> > ok, here we go (i hope i did not cut important parts).
> > i don`t see the real issue, but i did another interesting finding - see
> > below
> >
> >
> > bad (cd l):
> >
> > open(".", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY|O_CLOEXEC) = 3
> > fstat64(3, {st_dev=makedev(0, 19), st_ino=256, st_mode=S_IFDIR|0555,
> > st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=8, st_size=18,
> > st_atime=2008/12/16-21:32:38, st_mtime=2008/12/16-21:32:37,
> > st_ctime=2008/12/16-21:32:37}) = 0
> > getdents64(3, {{d_ino=256, d_off=2, d_type=DT_DIR, d_reclen=24, d_name="."}
> > {d_ino=256, d_off=2, d_type=DT_DIR, d_reclen=24, d_name=".."} {d_ino=257,
> > d_off=3, d_type=DT_DIR, d_reclen=24, d_name="test"} {d_ino=258,
> > d_off=9223372036854775807, d_type=DT_DIR, d_reclen=32, d_name="linux"}},
> > 4096) = 104
> > _llseek(3, 3, [3], SEEK_SET)= 0
> > getdents64(3, {{d_ino=258, d_off=9223372036854775807, d_type=DT_DIR,
> > d_reclen=32, d_name="linux"}}, 4096) = 32
>
> On Tue, Dec 16, 2008 at 22:26, wrote:
> > i assume it has something to do with the large value for d_off of the last
> > dirent ?
>
> Looks like, 9223372036854775807 is just LLONG_MAX.
I can not reproduce that (on openSUSE 11.1). I also don't see
the _llseek() calls.
open(".", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3
fstat(3, {st_dev=makedev(0, 18), ...
getdents64(3, {
{d_ino=260, d_off=2, d_type=DT_DIR, d_reclen=24, d_name="."}
{d_ino=256, d_off=2, d_type=DT_DIR, d_reclen=24, d_name=".."}
{d_ino=261, d_off=3, d_type=DT_REG, d_reclen=24, d_name="a"}
{d_ino=262, d_off=4, d_type=DT_REG, d_reclen=24, d_name="b"}
{d_ino=263, d_off=5, d_type=DT_REG, d_reclen=24, d_name="c"}
{d_ino=264, d_off=6, d_type=DT_DIR, d_reclen=24, d_name="test"}
{d_ino=265, d_off=9223372036854775807, d_type=DT_DIR, d_reclen=32,
d_name="linux"}
}, 4096) = 176
getdents64(3, {}, 4096) = 0
close(3)
This is with today's git kernel and today's standalone btrfs unstable.
You are using the distro kernel and compile the standalone btrfs module?
Kay
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: weird bash autocomplete issue
On Tue, Dec 16, 2008 at 21:46, wrote:
>> On Tue, Dec 16, 2008 at 20:37, Roland wrote:
>> > i have come across a weird autocomplete issue i assume it is related to
>> > btrfs.
>> >
>> > let`s have some dirs:
>> >
>> > /non-btrfs-mount
>> > ./linux
>> > ./testdir
>> >
>> > /brtfs-mount
>> > ./linux
>> > ./testdir
>> >
>> > now, if i do "cd t" in /non-btrfs-mount, "t" autocompletes to
>> > "testdir"
>> > same for linux - bash autocompletes as expected.
>> >
>> > now, the weird thing is, that on /btrfs-mount this behaves different.
>> >
>> > autocompletion for testdir works, but not for linux dir. weird.
>> >
>> > can someone reproduce this ?
>>
>> Open another shell, find the bash process pid of the first shell with:
>> ps afx
>> and do:
>> strace -p
>> Go back to the first shell, hit , and the trace should show
>> what's going on. You see a significant difference there?
>
>
> ok, here we go (i hope i did not cut important parts).
> i don`t see the real issue, but i did another interesting finding - see below
>
>
> bad (cd l):
>
> open(".", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY|O_CLOEXEC) = 3
> fstat64(3, {st_dev=makedev(0, 19), st_ino=256, st_mode=S_IFDIR|0555,
> st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=8, st_size=18,
> st_atime=2008/12/16-21:32:38, st_mtime=2008/12/16-21:32:37,
> st_ctime=2008/12/16-21:32:37}) = 0
> getdents64(3, {{d_ino=256, d_off=2, d_type=DT_DIR, d_reclen=24, d_name="."}
> {d_ino=256, d_off=2, d_type=DT_DIR, d_reclen=24, d_name=".."} {d_ino=257,
> d_off=3, d_type=DT_DIR, d_reclen=24, d_name="test"} {d_ino=258,
> d_off=9223372036854775807, d_type=DT_DIR, d_reclen=32, d_name="linux"}},
> 4096) = 104
> _llseek(3, 3, [3], SEEK_SET)= 0
> getdents64(3, {{d_ino=258, d_off=9223372036854775807, d_type=DT_DIR,
> d_reclen=32, d_name="linux"}}, 4096) = 32
On Tue, Dec 16, 2008 at 22:26, wrote:
> i assume it has something to do with the large value for d_off of the last
> dirent ?
Looks like, 9223372036854775807 is just LLONG_MAX.
Kay
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
kernel BUG: kernel bug caught at 55% mark of portage cache update
A kernel 'BUG' was caught while I had compression enabled and was performing a 'portage' update on my gentoo installation (which had btrfs mounted for portage for some basic perf testing). It has finished the rsync and was at the 55% mark of the portage cache update... if that helps at all. Inlined is the kernel bug output... + some extra debug info that had been spit out beforehand that i thought might be useful... didn't appear to be at that 85% freespace mark... should I attach a btrfs-image dump? /dev/mapper/vg-btrfsTest 1.0G 309M 716M 31% /mnt/btrfsTest device fsid 6c44efb8af8339de-fdcb30fce5264c82 devid 1 transid 22 /dev/ mapper/vg-btrfsTest space info full 36 we were searching for 4096 bytes, num_bytes 4096, loop 2, allowed_alloc 1 allocation failed flags 36, wanted 4096 space_info has 65536 free, is full block group 29360128 has 53673984 bytes, 50790400 used 2818048 pinned 0 reserved 0 blocks of free space at or bigger than bytes is block group 190382080 has 53673984 bytes, 50040832 used 3633152 pinned 0 reserved 0 blocks of free space at or bigger than bytes is block group 244056064 has 53673984 bytes, 50913280 used 2760704 pinned 0 reserved 0 blocks of free space at or bigger than bytes is block group 297730048 has 53673984 bytes, 50917376 used 2756608 pinned 0 reserved 0 blocks of free space at or bigger than bytes is block group 351404032 has 53673984 bytes, 36831232 used 16842752 pinned 0 reserved 0 blocks of free space at or bigger than bytes is block group 405078016 has 53673984 bytes, 19693568 used 33980416 pinned 0 reserved 0 blocks of free space at or bigger than bytes is block group 566099968 has 53673984 bytes, 49348608 used 4325376 pinned 0 reserved 0 blocks of free space at or bigger than bytes is [ cut here ] kernel BUG at /home/harningt/gitrepo/kernel/btrfs-unstable-standalone/ extent-tree.c:3096! invalid opcode: [1] CPU 0 Modules linked in: btrfs crc32c libcrc32c zlib_deflate nvidia(P) fuse snd_pcm_oss snd_mixer_oss snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_atiixp snd_ac97_codec ac97_bus snd_pcm snd_timer snd snd_page_alloc Pid: 4539, comm: emerge Tainted: P 2.6.27-00080-g13d5f30 #1 RIP: 0010:[] [] __btrfs_reserve_extent+0x278/0x2b0 [btrfs] RSP: 0018:880029a25908 EFLAGS: 00010296 RAX: 880048118560 RBX: 880044468400 RCX: RDX: ff01 RSI: RDI: 880048118558 RBP: 880048118548 R08: 8b1a R09: R10: R11: 000a R12: 1000 R13: 880048118558 R14: 880048118548 R15: 1000 FS: 7f671a6896f0() GS:8087fe80() knlGS: f7c7d6c0 CS: 0010 DS: ES: CR0: 80050033 CR2: 01bd1024 CR3: 2b39c000 CR4: 06e0 DR0: DR1: DR2: DR3: DR6: 0ff0 DR7: 0400 Process emerge (pid: 4539, threadinfo 880029a24000, task 880049991740) Stack: 880029a25a08 00040024 1000 880031723800 880029a25a08 0005 8800437a9070 Call Trace: [] ? btrfs_alloc_extent+0x4c/0xc0 [btrfs] [] ? btrfs_alloc_free_block+0x64/0xa0 [btrfs] [] ? __btrfs_cow_block+0x1f6/0x8a0 [btrfs] [] ? btrfs_cow_block+0x134/0x210 [btrfs] [] ? btrfs_search_slot+0x1f4/0x910 [btrfs] [] ? __pagevec_lru_add+0x95/0xa0 [] ? extent_readpages+0x15e/0x180 [btrfs] [] ? __alloc_pages_internal+0xb6/0x470 [] ? btrfs_lookup_inode+0x32/0xb0 [btrfs] [] ? btrfs_update_inode+0x3d/0xc0 [btrfs] [] ? btrfs_dirty_inode+0x43/0x60 [btrfs] [] ? __mark_inode_dirty+0x34/0x190 [] ? touch_atime+0xfe/0x170 [] ? generic_file_aio_read+0x581/0x5e0 [] ? do_sync_read+0xd9/0x120 [] ? autoremove_wake_function+0x0/0x30 [] ? arch_get_unmapped_area_topdown+0x206/0x2b0 [] ? vfs_read+0xc5/0x180 [] ? sys_read+0x53/0x90 [] ? system_call_fastpath+0x16/0x1b Code: ff 31 c0 e8 cb bf a0 df 4c 89 e6 48 89 df e8 40 93 03 00 48 8b 6d 00 48 8b 45 00 4c 39 f5 0f 18 08 75 b3 4c 89 ef e8 a8 0d a2 df <0f> 0b eb fe be 03 0c 00 00 48 c7 c7 30 00 86 a0 e8 63 af a0 df RIP [] __btrfs_reserve_extent+0x278/0x2b0 [btrfs] RSP ---[ end trace cdb8403d4defddf6 ]--- -- Thomas Harning TrustBearer Labs TB OpenID: https://openid.trustbearer.com/harningt -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: weird bash autocomplete issue
> On Tue, Dec 16, 2008 at 20:37, Roland wrote:
> > i have come across a weird autocomplete issue i assume it is related to
> > btrfs.
> >
> > let`s have some dirs:
> >
> > /non-btrfs-mount
> > ./linux
> > ./testdir
> >
> > /brtfs-mount
> > ./linux
> > ./testdir
> >
> > now, if i do "cd t" in /non-btrfs-mount, "t" autocompletes to "testdir"
> > same for linux - bash autocompletes as expected.
> >
> > now, the weird thing is, that on /btrfs-mount this behaves different.
> >
> > autocompletion for testdir works, but not for linux dir. weird.
> >
> > can someone reproduce this ?
>
> Open another shell, find the bash process pid of the first shell with:
> ps afx
> and do:
> strace -p
> Go back to the first shell, hit , and the trace should show
> what's going on. You see a significant difference there?
ok, here we go (i hope i did not cut important parts).
i don`t see the real issue, but i did another interesting finding - see below
bad (cd l):
open(".", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY|O_CLOEXEC) = 3
fstat64(3, {st_dev=makedev(0, 19), st_ino=256, st_mode=S_IFDIR|0555,
st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=8, st_size=18,
st_atime=2008/12/16-21:32:38, st_mtime=2008/12/16-21:32:37,
st_ctime=2008/12/16-21:32:37}) = 0
getdents64(3, {{d_ino=256, d_off=2, d_type=DT_DIR, d_reclen=24, d_name="."}
{d_ino=256, d_off=2, d_type=DT_DIR, d_reclen=24, d_name=".."} {d_ino=257,
d_off=3, d_type=DT_DIR, d_reclen=24, d_name="test"} {d_ino=258,
d_off=9223372036854775807, d_type=DT_DIR, d_reclen=32, d_name="linux"}}, 4096)
= 104
_llseek(3, 3, [3], SEEK_SET)= 0
getdents64(3, {{d_ino=258, d_off=9223372036854775807, d_type=DT_DIR,
d_reclen=32, d_name="linux"}}, 4096) = 32
close(3)= 0
write(2, "\7", 1) = 1
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
good (cd t):
open(".", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY|O_CLOEXEC) = 3
fstat64(3, {st_dev=makedev(0, 19), st_ino=256, st_mode=S_IFDIR|0555,
st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=8, st_size=18,
st_atime=2008/12/16-21:38:13, st_mtime=2008/12/16-21:38:11,
st_ctime=2008/12/16-21:38:11}) = 0
getdents64(3, {{d_ino=256, d_off=2, d_type=DT_DIR, d_reclen=24, d_name="."}
{d_ino=256, d_off=2, d_type=DT_DIR, d_reclen=24, d_name=".."} {d_ino=257,
d_off=3, d_type=DT_DIR, d_reclen=24, d_name="test"} {d_ino=258,
d_off=9223372036854775807, d_type=DT_DIR, d_reclen=32, d_name="linux"}}, 4096)
= 104
_llseek(3, 3, [3], SEEK_SET)= 0
getdents64(3, {{d_ino=258, d_off=9223372036854775807, d_type=DT_DIR,
d_reclen=32, d_name="linux"}}, 4096) = 32
close(3)= 0
stat64("test", {st_dev=makedev(0, 19), st_ino=257, st_mode=S_IFDIR|0755,
st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=0, st_size=0,
st_atime=2008/12/16-21:15:29, st_mtime=2008/12/16-21:15:29,
st_ctime=2008/12/16-21:15:29}) = 0
stat64("test", {st_dev=makedev(0, 19), st_ino=257, st_mode=S_IFDIR|0755,
st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=0, st_size=0,
st_atime=2008/12/16-21:15:29, st_mtime=2008/12/16-21:15:29,
st_ctime=2008/12/16-21:15:29}) = 0
write(2, "est/", 4) = 4
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
and now, after adding a file to that dir with "touch abcd", for my curiousity
this makes "cd l" work again.
open(".", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY|O_CLOEXEC) = 3
fstat64(3, {st_dev=makedev(0, 19), st_ino=256, st_mode=S_IFDIR|0555,
st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=8, st_size=26,
st_atime=2008/12/16-21:34:45, st_mtime=2008/12/16-21:34:44,
st_ctime=2008/12/16-21:34:44}) = 0
getdents64(3, {{d_ino=256, d_off=2, d_type=DT_DIR, d_reclen=24, d_name="."}
{d_ino=256, d_off=2, d_type=DT_DIR, d_reclen=24, d_name=".."} {d_ino=257,
d_off=3, d_type=DT_DIR, d_reclen=24, d_name="test"} {d_ino=258, d_off=17,
d_type=DT_DIR, d_reclen=32, d_name="linux"} {d_ino=272,
d_off=9223372036854775807, d_type=DT_REG, d_reclen=24, d_name="abcd"}}, 4096) =
128
_llseek(3, 17, [17], SEEK_SET) = 0
getdents64(3, {{d_ino=272, d_off=9223372036854775807, d_type=DT_REG,
d_reclen=24, d_name="abcd"}}, 4096) = 24
close(3)= 0
stat64("linux", {st_dev=makedev(0, 19), st_ino=258, st_mode=S_IFDIR|0755,
st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=0, st_size=0,
st_atime=2008/12/16-21:15:33, st_mtime=2008/12/16-21:15:33,
st_ctime=2008/12/16-21:15:33}) = 0
stat64("linux", {st_dev=makedev(0, 19), st_ino=258, st_mode=S_IFDIR|0755,
st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=0, st_size=0,
st_atime=2008/12/16-21:15:33, st_mtime=2008/12/16-21:15:33,
st_ctime=2008/12/16-21:15:33}) = 0
write(2, "inux/", 5)= 5
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
regards
roland
___
Täglich 1.000.000 Euro gewinnen! Jetzt kostenlos WEB.DE Milli
Re: weird bash autocomplete issue
On Tue, Dec 16, 2008 at 20:37, Roland wrote: > i have come across a weird autocomplete issue i assume it is related to > btrfs. > > let`s have some dirs: > > /non-btrfs-mount > ./linux > ./testdir > > /brtfs-mount > ./linux > ./testdir > > now, if i do "cd t" in /non-btrfs-mount, "t" autocompletes to "testdir" > same for linux - bash autocompletes as expected. > > now, the weird thing is, that on /btrfs-mount this behaves different. > > autocompletion for testdir works, but not for linux dir. weird. > > can someone reproduce this ? Open another shell, find the bash process pid of the first shell with: ps afx and do: strace -p Go back to the first shell, hit , and the trace should show what's going on. You see a significant difference there? Kay -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Compressed Filesystem
I agree that adding more options will add more complexity but it seems the same amount of work in kernel space will have to be done regarding lzo compression itself - it`s already there(since july 2007). the in-kernel lzo is equivalent to minilzo. (http://www.oberhumer.com/opensource/lzo/) regards roland - Original Message - From: "Lee Trager" To: "Chris Mason" Cc: "Lee Trager" ; ; Sent: Tuesday, December 16, 2008 5:25 PM Subject: Re: Compressed Filesystem I agree that adding more options will add more complexity but it seems the same amount of work in kernel space will have to be done. If we support mutiple compression schemes somewhere the compression scheme used will have to be stored so we know what to use in the future. If we store it on the super block the user will have to choose when they format at which point they may not see the need to use compression. Or they may choose one compression scheme and later want to change to something else. It doesn't make sence to have to reformat your drive just to change compression scheme. This leaves us with storing what the compression scheme is on each inode. We currently store if compression is used on a per inode basis so storing the type wouldn't be a huge leap. Lee On Tue, Dec 16, 2008 at 10:26:10AM -0500, Chris Mason wrote: On Tue, 2008-12-16 at 10:20 -0500, Lee Trager wrote: > While I agree that the command you send should be possible it wasn't > exactly what I was thinking. Currently I am working on a way for the > user to individually set which files/directories they want compressed > or > not. What I was saying is that assuming you are in a mounted btrfs > directory you could do something like > > chattr -R +c zlib dir1 Compress dir1 and all its contents with zlib > chattr -R +c bzip dir2 Compress dir2 and all its contents with bzip > chattr +c lzo file1 Compress fil1 with lzo > chattr -c file2 Uncompress file2 > chattr +c none dir3 Uncompress dir3 but leave contents as is > > If the user did something like > mount -o compress,cscheme=zlib /dev/xyz /mntpoint > and then > chattr +c /mntpoint/dir > /mntpoint/dir would default to zlib as would anything else written to > the disk. > This is one of those places where more options isn't always better. Every option adds complexity to the filesystem and the testing matrix. I'd much rather have just one compression scheme per FS. If people need a specific compression scheme for a specific file, they can just compress it in userland. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
weird bash autocomplete issue
hi, i have come across a weird autocomplete issue i assume it is related to btrfs. let`s have some dirs: /non-btrfs-mount ./linux ./testdir /brtfs-mount ./linux ./testdir now, if i do "cd t" in /non-btrfs-mount, "t" autocompletes to "testdir" same for linux - bash autocompletes as expected. now, the weird thing is, that on /btrfs-mount this behaves different. autocompletion for testdir works, but not for linux dir. weird. can someone reproduce this ? i`m on opensuse 11.1 with 2.6.27.8-1-default and latest btrfs-unstable from git. regards roland -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Compressed Filesystem
> I'd much rather have just one compression scheme per FS. If people need > a specific compression scheme for a specific file, they can just > compress it in userland. yes, i also think one compression scheme per FS is absolutely sufficient. > -Ursprüngliche Nachricht- > Von: "Chris Mason" > Gesendet: 16.12.08 16:26:28 > An: Lee Trager > CC: linux-btrfs@vger.kernel.org > Betreff: Re: Compressed Filesystem > On Tue, 2008-12-16 at 10:20 -0500, Lee Trager wrote: > > While I agree that the command you send should be possible it wasn't > > exactly what I was thinking. Currently I am working on a way for the > > user to individually set which files/directories they want compressed or > > not. What I was saying is that assuming you are in a mounted btrfs > > directory you could do something like > > > > chattr -R +c zlib dir1 Compress dir1 and all its contents with zlib > > chattr -R +c bzip dir2 Compress dir2 and all its contents with bzip > > chattr +c lzo file1 Compress fil1 with lzo > > chattr -c file2 Uncompress file2 > > chattr +c none dir3 Uncompress dir3 but leave contents as is > > > > If the user did something like > > mount -o compress,cscheme=zlib /dev/xyz /mntpoint > > and then > > chattr +c /mntpoint/dir > > /mntpoint/dir would default to zlib as would anything else written to > > the disk. > > > > This is one of those places where more options isn't always better. > Every option adds complexity to the filesystem and the testing matrix. > > I'd much rather have just one compression scheme per FS. If people need > a specific compression scheme for a specific file, they can just > compress it in userland. > > -chris > > > ___ Sensationsangebot verlängert: WEB.DE FreeDSL - Telefonanschluss + DSL für nur 16,37 Euro/mtl.!* http://dsl.web.de/?ac=OM.AD.AD008K15039B7069a -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Compressed Filesystem
I agree that adding more options will add more complexity but it seems the same amount of work in kernel space will have to be done. If we support mutiple compression schemes somewhere the compression scheme used will have to be stored so we know what to use in the future. If we store it on the super block the user will have to choose when they format at which point they may not see the need to use compression. Or they may choose one compression scheme and later want to change to something else. It doesn't make sence to have to reformat your drive just to change compression scheme. This leaves us with storing what the compression scheme is on each inode. We currently store if compression is used on a per inode basis so storing the type wouldn't be a huge leap. Lee On Tue, Dec 16, 2008 at 10:26:10AM -0500, Chris Mason wrote: > On Tue, 2008-12-16 at 10:20 -0500, Lee Trager wrote: > > While I agree that the command you send should be possible it wasn't > > exactly what I was thinking. Currently I am working on a way for the > > user to individually set which files/directories they want compressed or > > not. What I was saying is that assuming you are in a mounted btrfs > > directory you could do something like > > > > chattr -R +c zlib dir1 Compress dir1 and all its contents with zlib > > chattr -R +c bzip dir2 Compress dir2 and all its contents with bzip > > chattr +c lzo file1 Compress fil1 with lzo > > chattr -c file2 Uncompress file2 > > chattr +c none dir3 Uncompress dir3 but leave contents as is > > > > If the user did something like > > mount -o compress,cscheme=zlib /dev/xyz /mntpoint > > and then > > chattr +c /mntpoint/dir > > /mntpoint/dir would default to zlib as would anything else written to > > the disk. > > > > This is one of those places where more options isn't always better. > Every option adds complexity to the filesystem and the testing matrix. > > I'd much rather have just one compression scheme per FS. If people need > a specific compression scheme for a specific file, they can just > compress it in userland. > > -chris > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
More performance results
Just realized that I never posted the link to the results from a few weeks ago. These are from the November 20 git tree, just before the move to the 2.6.28-rc based kernels. We have been having some issues with the new kernel (unrelated to btrfs) that have kept us from getting newer results. Will post as soon as we have them. http://btrfs.boxacle.net/repository/raid/history/History.html For these results I show a history of the last 3 btrfs runs so we can see how performance is changing with new releases. Also of note, in the analysis/oprofile.breakout.001 directories of any benchmark, you can now find oprofile call graphs (for new runs only). Steve -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Compressed Filesystem
On Tue, 2008-12-16 at 10:20 -0500, Lee Trager wrote: > While I agree that the command you send should be possible it wasn't > exactly what I was thinking. Currently I am working on a way for the > user to individually set which files/directories they want compressed or > not. What I was saying is that assuming you are in a mounted btrfs > directory you could do something like > > chattr -R +c zlib dir1Compress dir1 and all its contents with zlib > chattr -R +c bzip dir2Compress dir2 and all its contents with bzip > chattr +c lzo file1 Compress fil1 with lzo > chattr -c file2 Uncompress file2 > chattr +c none dir3 Uncompress dir3 but leave contents as is > > If the user did something like > mount -o compress,cscheme=zlib /dev/xyz /mntpoint > and then > chattr +c /mntpoint/dir > /mntpoint/dir would default to zlib as would anything else written to > the disk. > This is one of those places where more options isn't always better. Every option adds complexity to the filesystem and the testing matrix. I'd much rather have just one compression scheme per FS. If people need a specific compression scheme for a specific file, they can just compress it in userland. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Compressed Filesystem
While I agree that the command you send should be possible it wasn't exactly what I was thinking. Currently I am working on a way for the user to individually set which files/directories they want compressed or not. What I was saying is that assuming you are in a mounted btrfs directory you could do something like chattr -R +c zlib dir1 Compress dir1 and all its contents with zlib chattr -R +c bzip dir2 Compress dir2 and all its contents with bzip chattr +c lzo file1 Compress fil1 with lzo chattr -c file2 Uncompress file2 chattr +c none dir3 Uncompress dir3 but leave contents as is If the user did something like mount -o compress,cscheme=zlib /dev/xyz /mntpoint and then chattr +c /mntpoint/dir /mntpoint/dir would default to zlib as would anything else written to the disk. Lee On Tue, Dec 16, 2008 at 12:19:13AM +0100, devz...@web.de wrote: > > If multiple compression schemes are implemented how should the user go > > about choosing which one they want? Should it be done at kernel time? Or > > with the userland tools on a per file basis(maybe zlib is the default > > but a user could say I want this directory to be bzip)? > > yes, why not... > > doing that at mounttime like > > mount -o compress,cscheme=myzip /dev/xyz /mntpoint > > would be a good start > > > > > -Urspr?ngliche Nachricht- > > Von: "Lee Trager" > > Gesendet: 16.12.08 00:07:32 > > An: devz...@web.de > > CC: linux-btrfs@vger.kernel.org > > Betreff: Re: Compressed Filesystem > > > > If multiple compression schemes are implemented how should the user go > > about choosing which one they want? Should it be done at kernel time? Or > > with the userland tools on a per file basis(maybe zlib is the default > > but a user could say I want this directory to be bzip)? > > > > On Mon, Dec 15, 2008 at 11:14:01PM +0100, devz...@web.de wrote: > > > fantastic feature! > > > > > > i`m curious: can btrfs support more than one compression scheme at the > > > same time, i.e. is compression "pluggable" ? > > If you look at compression.c, compression.h, and ctree.h you can clearly > > see that support for multiple compression scheme was in mind. Implmented > > a new one shouldn't be to hard but you probably want to make the current > > system a little bit more pluggable and move all the zlib stuff into > > zlib.c. > > > > > > lzo compression coming to my mind, as this is giving real-time compession > > > and may even speed up disk access. > > > > > > compression ratio isn`t too bad, but speed is awesome and doesn`t need as > > > much cpu as gzip. > > > > > In some tests I've run zlib is actually faster then nocompression > > because of the lesser amount of data that has to transfer to and from > > the disk. It would be instresting to see how bzip works with this to. > > > experimental lzo compression in zfs-fuse showed that it could compress > > > tarred kernel-source with 2.99x compressratio (where gzip gave 3.41x), so > > > maybe lzo is a better algorithm for realtime filesystem compression... > > > > > > regards > > > roland > > > > > > > > > > > > From: Chris Mason oracle.com> > > > Subject: Re: Compressed Filesystem > > > Newsgroups: gmane.comp.file-systems.btrfs > > > Date: 2008-10-29 20:08:42 GMT (6 weeks, 5 days, 1 hour and 53 minutes ago) > > > > > > On Wed, 2008-10-29 at 12:14 -0600, Anthony Roberts wrote: > > > > Hi, I have a few questions about this: > > > > > > > > > Compression is optional and off by default (mount -o compress to > > > > > enable > > > > > it). When enabled, every file is compressed. > > > > > > > > Do you know what the CPU load is like with this enabled? > > > > > > Now that I've finally pushed the code out, you can try it ;) One part > > > of the implementation I need to revisit is the place in the code where I > > > do compression means that most of the time the single threaded pdflush > > > is the one compressing. > > > > > > This doesn't spread the load very well across the cpus. It can be > > > fixed, but I wanted to get the code out there. > > > > > > The decompression does spread across cpus, and I've gotten about 800MB/s > > > doing decompress and checksumming on a zero filled compressed file. At > > > the time, the disk was reading 14MB/s. > > > > > > > > > > > Do you know whether data can be compressed at a sufficient rate to still > > > > saturate the disk on recent-ish AMD/Intel CPUs? > > > > > > My recentish intel cpu can compress and checksum at about 120MB/s. > > > > > > > > If no, is the effective pre-compression I/O rate still comparable to the > > > > disk without compression? > > > > > > > > > > It depends on your disks... > > > > > > > I'm pretty sure that won't even matter in many cases (eg you're seeking > > > > too much to care, or you're on a VM with lots of cores but congested > > > > disks, or you're dealing with media files that it doesn't bother > > > > compressing, etc), but I'm curious what sort of overhead this adds. :) > > >
