Re: [GIT PULL] scrub updates for 3.1

2011-07-06 Thread Arne Jansen
On 06.07.2011 03:06, Li Zefan wrote:
 21:00, Arne Jansen worte:
 Hi Chris,

 since rc-6 seems to be the last rc for 3.0 and in case you're already
 preparing your pull request for 3.1, can you please pull the following
 updates for scrub, based on your for-linus tree (2f7e33d432)?

 git://git.kernel.org/pub/scm/linux/kernel/git/arne/btrfs-unstable-arne.git
 for-chris

 It just contains the readahead patch, which gives a significant
 performance improvement for scrub. Currently scrub is the only
 consumer.

 Thanks,
 Arne

 Arne Jansen (7):
   btrfs: add an extra wait mode to read_extent_buffer_pages
   btrfs: add READAHEAD extent buffer flag
   btrfs: state information for readahead
   btrfs: initial readahead code and prototypes
   btrfs: hooks for readahead
   btrfs: test ioctl for readahead
 
 Do we really want this ioctl that is merely for testing some kernel
 APIs in our upstream kernel?

Oh, I completely forgot about that. You're right of course. I pushed
the for-chris branch again, it now looks like this:

Arne Jansen (6):
  btrfs: add an extra wait mode to read_extent_buffer_pages
  btrfs: add READAHEAD extent buffer flag
  btrfs: state information for readahead
  btrfs: initial readahead code and prototypes
  btrfs: hooks for readahead
  btrfs: use readahead API for scrub

 fs/btrfs/Makefile|3 +-
 fs/btrfs/ctree.h |   21 ++
 fs/btrfs/disk-io.c   |   85 +-
 fs/btrfs/disk-io.h   |2 +
 fs/btrfs/extent_io.c |9 +-
 fs/btrfs/extent_io.h |4 +
 fs/btrfs/reada.c |  949
++
 fs/btrfs/scrub.c |  116 +++
 fs/btrfs/volumes.c   |8 +
 fs/btrfs/volumes.h   |8 +
 10 files changed, 1133 insertions(+), 72 deletions(-)
 create mode 100644 fs/btrfs/reada.c

Thanks,
Arne
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Memory leak?

2011-07-06 Thread Stephane Chazelas
2011-07-03 13:38:57 -0600, cwillu:
 On Sun, Jul 3, 2011 at 1:09 PM, Stephane Chazelas
 stephane_chaze...@yahoo.fr wrote:
[...]
  Now, on a few occasions (actually, most of the time), when I
  rsynced the data (about 2.5TB) onto the external drive, the
  system would crash after some time with Out of memory and no
  killable process. Basically, something in kernel was allocating
  the whole memory, then oom mass killed everybody and crash.
[...]
 Look at the output of slabtop (should be installed by default, procfs
 package), before rsync for comparison, and during.

Hi,

so, no crash this time, but at the end of the rsync, there's a
whole chunk of memory that is no longer available to processes
(about 3.5G). As suggested by Carey, I watched /proc/slabinfo
during the rsync process (see below for a report of the most
significant ones over time).

Does that mean that if the system had had less than 3G of RAM, it
would have crashed?

I tried to reclaim the space without success. I had a process
allocate as much memory as it could. Then I unmounted the btrfs
fs that rsync was copying onto (the one on LUKS).
btrfs_inode_cache hardly changed. Then I tried to unmount the
source (the one on 3 hard disks and plenty of subvolumes).
umount hung. The FS disappeared from /proc/mounts. Here is the
backtrace:

[169270.268005] umount  D 880145ebe770 0 24079   1290 0x0004
[169270.268005]  880145ebe770 0086  
8160b020
[169270.268005]  00012840 880123bc7fd8 880123bc7fd8 
00012840
[169270.268005]  880145ebe770 880123bc6010 7fffac84f4a8 
0001
[169270.268005] Call Trace:
[169270.268005]  [81337d50] ? rwsem_down_failed_common+0xda/0x10e
[169270.268005]  [811aca63] ? call_rwsem_down_write_failed+0x13/0x20
[169270.268005]  [813376c7] ? down_write+0x25/0x27
[169270.268005]  [810ff6eb] ? deactivate_super+0x30/0x3d
[169270.268005]  [81114135] ? sys_umount+0x2ea/0x315
[169270.268005]  [8133d412] ? system_call_fastpath+0x16/0x1b

iostat shows nothing being written to the drives.

extent_map delayed_node btrfs_inode_cache btrfs_free_space_cache
(in bytes)
09:40 131560  0  0128
09:50 131560  0   2000128 rsync started at 9:52
10:00   15832608   87963264 146583 325440
10:10 386056   85071168 1444656000 832960
10:20 237600   33988032 15497690001355584
10:30   23300288  145209600  4872740002237056
10:40   24667104  148610016  5064920002304448
10:50   22479776  139655808  5118930002382464
11:004137672104169645920002425344
11:1054985923211776   127420002443648
11:204567904140889695990002452608
11:302276736130982446850002453696
11:402225696 42192019870002455424
11:501971552  90432 3830002466176
12:001761672  74016 3270002469760
12:101939608  85824 4010002473536
12:202136288 121824 5510002479168
12:302367288 135648 6190002486016
12:401380984 181152 9110002485696
12:501053272 20217610270002483712
13:001938200 21974411520002491712
13:102037112 22377612040002494528
13:201775664 24912012440002497216
13:301704560 36604817320002500608
13:401433344 4688642462501824
13:508553248   20505888   673320002503168
14:00   12682208   34351200  1419680002494208
14:10   18488800   50282784  1778030002500544
14:20   19435592   46767744  1635820002505920
14:30   18734936   44863488  1565010002507200
14:40   21865184   46053504  1601850002484928
14:50   24457664   46473120  1624460002499200
15:00   24401344   47700576  1667230002502784
15:10   31390304   63426240  2211790002521472
15:20   34594560   61365600  2142430002524160
15:30   33836704   60934752  2126950002526400
15:40   33358776   60598944  2114550002528320
15:50   34909952   62583840  2184920002526272
16:00   44326656   65875392  2301230002529792
16:10   45840608   66373632  2321140002532736
16:20   47848064   66577536  2320480002535872
16:30   48013152   6160  2406510002536128
16:40   47594184   67766976  2362410002536576
16:50   48144184   67739904  236122542144
17:00   48005848   67639392  2352980002544000
17:10   48253920   67661280  2353760002537216
17:20   48857952   67612032  2349950002536000
17:30   48514752   67611168  2349860002535488
17:40   48436872   67609728  2349240002534528
17:50   48902216   67765248  2356540002542400
18:00   49055160   67763520  236022542912
18:10   48749712   67727520  235742550464
18:20   48631088   67705344  2355570002553280
18:30   49101096   6344  235713000220
18:40   48609264   67782816  2356010002558912
18:50  

Re: [btrfs-transacti] btrfs-endio-wri] - WAS: Re: [btrfs-delalloc-]

2011-07-06 Thread Proskurin Kirill

On 07/01/2011 06:07 PM, Josef Bacik wrote:


I found what my btrfs partition now is really slow. Something wrong
happend. I try on 2.6.32 and 2.6.39 - same result. It is because
partition is almost full?

I run a simple cycle to clean some old files:
for i in `cat /tmp/delit`; do rm -f $i ; done

And it is takes about 5-10 second per file to delete.
I get sysrq+w while rm is work - it is attached.



Just a shot in the dark, but will you give this a shot and let me know
how it goes?  Thanks


Sorry for late answer. Im afraid I can`t test it now - then problem 
occurs I have no time to wait and format partition. But I will save 
this patch and if I run at this problem again I will try it and report 
here.


Thanks for your help anyway.

--
Best regards,
Proskurin Kirill
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


free space defragmentation?

2011-07-06 Thread Andreas Philipp

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
 
Hi,
 
Since the defragmentation tool of btrfs-more or less-just copies the
file to the free space the file gets fragmented again (maybe even worse)
if the free space is fragmented. Thus I wanted to ask whether there is
some tool or idea for a free space defragmentation.
 
Thanks,
Andreas Philipp
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
 
iQIcBAEBAgAGBQJOFMLXAAoJEJIcBJ3+XkgiK8cQAJqYE9VpDak555/5U1YHrrkH
eVJKn/N2nR0qO9pUh/XsMyStpntZ+yv/cVQ/i12mfG5ZZaCS6VBEZzNxooGQP6J2
jLgxYtfKUKaR8AHC0dzOkDN3OE78U1/grZV9SEE1wx05wXGMOipKcuVQZQSMOS1n
jP6FHYRVd22GGbMob9GxxNv+j/2ZSx5ThgYI1ZWbYRwD/XAEWHuMZuWElGTZrQaQ
7TZ4IzlQ4sMpZLbrp5m60EjdmW1G4mwREygSh8jmnZp0ndhWVzfbJorWJYDLLXIP
prlptDH0KJsAf5MBmxbG4fe7nKX+XagvbSaYEKaORS0di61NripELc3QgalPrfdN
r/NjirXuIhnmQMowV288ntsuOe/QHYn7RQbxOgLxxw0ageCZGay/Rwfcq9deI2pe
QoSbC2PBecGhGe7jd0jVpB3TmYil8uWs7VzzdxIhknadtYTotEP4WQYQyK4xQqU1
GDfYXnqLXXhFrJkSZ3EFEeRlfSVdw33VGEEdjnqoRoKkKTvXpwE+JhrdiJvSkOC2
7sIbocvU+KFWLjtXYqDZgOFndCx+6NQIRR/pP9NOYnSk4x8qqthi9h3EqiCq1jRo
JIttYHS3IEXv0P58VmCUwMaaXdNfWEqXr93j12WdvcoNDm/oCwdTB8u9Ckk9rhQQ
yuygcOK6z0XAAh/vlZFm
=jT/r
-END PGP SIGNATURE-

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Announcing btrfs-gui 0.3-rc1

2011-07-06 Thread Hugo Mills
   Finally, I've found some time to finish off the latest slice of
btrfs-gui development. I've put up version 0.3-rc1 on the git repo at

http://git.darksatanic.net/repo/btrfs-gui.git/

and the repository can be browsed through gitweb at [1].

   New features since 0.2:

 * mkfs
 * add/remove devices
 * assorted bug-fixes

   Hugo.

[1] http://git.darksatanic.net/cgi/gitweb.cgi?p=btrfs-gui.git;a=summary

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
  --- You are demons,  and I am in Hell! Well, technically, it's ---  
   London,  but it's an easy mistake to make.   


signature.asc
Description: Digital signature