Re: [zfs-discuss] Zfs send speed. Was: User quota design discussion..

2009-05-27 Thread Jorgen Lundman


I changed to try zfs send on a UFS on zvolume as well:

received 92.9GB stream in 2354 seconds (40.4MB/sec)

Still fast enough to use. I have yet to get around to trying something 
considerably larger in size.


Lund


Jorgen Lundman wrote:



So you recommend I also do speed test on larger volumes? The test data I 
had on the b114 server was only 90GB. Previous tests included 500G ufs 
on zvol etc.  It is just it will take 4 days to send it to the b114 
server to start with ;) (From Sol10 servers).


Lund

Dirk Wriedt wrote:

Jorgen,

what is the size of the sending zfs?

I thought replication speed depends on the size of the sending fs, too 
not only size of the snapshot being sent.


Regards
Dirk


--On Freitag, Mai 22, 2009 19:19:34 +0900 Jorgen Lundman 
lund...@gmo.jp wrote:



Sorry, yes. It is straight;

# time zfs send zpool1/leroy_c...@speedtest | nc 172.20.12.232 3001
real19m48.199s

# /var/tmp/nc -l -p 3001 -vvv | time zfs recv -v zpool1/le...@speedtest
received 82.3GB stream in 1195 seconds (70.5MB/sec)


Sending is osol-b114.
Receiver is Solaris 10 10/08

When we tested Solaris 10 10/08 - Solaris 10 10/08 these were the 
results;


zfs send | nc | zfs recv - 1 MB/s
tar -cvf /zpool/leroy | nc | tar -xvf -  - 2.5 MB/s
ufsdump | nc | ufsrestore- 5.0 MB/s

So, none of those solutions was usable with regular Sol 10. Note most 
our volumes are ufs in

zvol, but even zfs volumes were slow.

Someone else had mentioned the speed was fixed in an earlier release, 
I had not had a chance to
upgrade. But since we wanted to try zfs user-quotas, I finally had 
the chance.


Lund


Brent Jones wrote:
On Thu, May 21, 2009 at 10:17 PM, Jorgen Lundman lund...@gmo.jp 
wrote:

To finally close my quest. I tested zfs send in osol-b114 version:

received 82.3GB stream in 1195 seconds (70.5MB/sec)

Yeeaahh!

That makes it completely usable! Just need to change our support 
contract to

allow us to run b114 and we're set! :)


Thanks,

Lund


Jorgen Lundman wrote:

We finally managed to upgrade the production x4500s to Sol 10 10/08
(unrelated to this) but with the hope that it would also make zfs 
send

usable.

Exactly how does build 105 translate to Solaris 10 10/08?  My 
current
speed test has sent 34Gb in 24 hours, which isn't great. Perhaps 
the next

version of Solaris 10 will have the improvements.

1



Robert Milkowski wrote:

Hello Jorgen,

If you look at the list archives you will see that it made a huge
difference for some people including me. Now I'm easily able to
saturate GbE linke while zfs send|recv'ing.


Since build 105 it should be *MUCH* for faster.



--
Jorgen Lundman   | lund...@lundman.net
Unix Administrator   | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo| +81 (0)90-5578-8500  (cell)
Japan| +81 (0)3 -3375-1767  (home)
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss



Can you give any details about your data set, what you piped zfs
send/receive through (SSH?), hardware/network, etc?
I'm envious of your speeds!




--
Dirk Wriedt, dirk.wri...@sun.com, Sun Microsystems GmbH
Systemingenieur Strategic Accounts
Nagelsweg 55, 20097 Hamburg, Germany
Tel.: +49-40-251523-132 Fax: +49-40-251523-425 Mobile: +49 172 848 4166
Never been afraid of chances I been takin' - Joan Jett

Sitz der Gesellschaft: Sun Microsystems GmbH, Sonnenallee 1, D-85551 
Kirchheim-Heimstetten

Amtsgericht Muenchen: HRB 161028
Geschaeftsfuehrer: Thomas Schroeder, Wolfgang Engels, Wolf Frenkel
Vorsitzender des Aufsichtsrates: Martin Haering






--
Jorgen Lundman   | lund...@lundman.net
Unix Administrator   | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo| +81 (0)90-5578-8500  (cell)
Japan| +81 (0)3 -3375-1767  (home)
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] Zfs send speed. Was: User quota design discussion..

2009-05-27 Thread Dirk Wriedt

Au contraire...

From what I have seen, larger file systems and large numbers of files
seem to slow down zfs send/receive, worsening the problem. So it may be
a good idea to partition your file system, subdividing it into smaller
ones, replicating each one separately. 

Dirk

Am Di, den 26.05.2009 schrieb Jorgen Lundman um 12:46:
 So you recommend I also do speed test on larger volumes? The test data I 
 had on the b114 server was only 90GB. Previous tests included 500G ufs 
 on zvol etc.  It is just it will take 4 days to send it to the b114 
 server to start with ;) (From Sol10 servers).
 
 Lund
 
 Dirk Wriedt wrote:
  Jorgen,
  
  what is the size of the sending zfs?
  
  I thought replication speed depends on the size of the sending fs, too 
  not only size of the snapshot being sent.
  
  Regards
  Dirk
  
  
  --On Freitag, Mai 22, 2009 19:19:34 +0900 Jorgen Lundman 
  lund...@gmo.jp wrote:
  
  Sorry, yes. It is straight;
 
  # time zfs send zpool1/leroy_c...@speedtest | nc 172.20.12.232 3001
  real19m48.199s
 
  # /var/tmp/nc -l -p 3001 -vvv | time zfs recv -v zpool1/le...@speedtest
  received 82.3GB stream in 1195 seconds (70.5MB/sec)
 
 
  Sending is osol-b114.
  Receiver is Solaris 10 10/08
 
  When we tested Solaris 10 10/08 - Solaris 10 10/08 these were the 
  results;
 
  zfs send | nc | zfs recv - 1 MB/s
  tar -cvf /zpool/leroy | nc | tar -xvf -  - 2.5 MB/s
  ufsdump | nc | ufsrestore- 5.0 MB/s
 
  So, none of those solutions was usable with regular Sol 10. Note most 
  our volumes are ufs in
  zvol, but even zfs volumes were slow.
 
  Someone else had mentioned the speed was fixed in an earlier release, 
  I had not had a chance to
  upgrade. But since we wanted to try zfs user-quotas, I finally had the 
  chance.
 
  Lund
 
 
  Brent Jones wrote:
  On Thu, May 21, 2009 at 10:17 PM, Jorgen Lundman lund...@gmo.jp wrote:
  To finally close my quest. I tested zfs send in osol-b114 version:
 
  received 82.3GB stream in 1195 seconds (70.5MB/sec)
 
  Yeeaahh!
 
  That makes it completely usable! Just need to change our support 
  contract to
  allow us to run b114 and we're set! :)
 
 
  Thanks,
 
  Lund
 
 
  Jorgen Lundman wrote:
  We finally managed to upgrade the production x4500s to Sol 10 10/08
  (unrelated to this) but with the hope that it would also make zfs 
  send
  usable.
 
  Exactly how does build 105 translate to Solaris 10 10/08?  My 
  current
  speed test has sent 34Gb in 24 hours, which isn't great. Perhaps 
  the next
  version of Solaris 10 will have the improvements.
  1
 
 
  Robert Milkowski wrote:
  Hello Jorgen,
 
  If you look at the list archives you will see that it made a huge
  difference for some people including me. Now I'm easily able to
  saturate GbE linke while zfs send|recv'ing.
 
  Since build 105 it should be *MUCH* for faster.
 
  -- 
  Jorgen Lundman   | lund...@lundman.net
  Unix Administrator   | +81 (0)3 -5456-2687 ext 1017 (work)
  Shibuya-ku, Tokyo| +81 (0)90-5578-8500  (cell)
  Japan| +81 (0)3 -3375-1767  (home)
  ___
  zfs-discuss mailing list
  zfs-discuss@opensolaris.org
  http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
 
 
  Can you give any details about your data set, what you piped zfs
  send/receive through (SSH?), hardware/network, etc?
  I'm envious of your speeds!
  
  
  
  -- 
  Dirk Wriedt, dirk.wri...@sun.com, Sun Microsystems GmbH
  Systemingenieur Strategic Accounts
  Nagelsweg 55, 20097 Hamburg, Germany
  Tel.: +49-40-251523-132 Fax: +49-40-251523-425 Mobile: +49 172 848 4166
  Never been afraid of chances I been takin' - Joan Jett
  
  Sitz der Gesellschaft: Sun Microsystems GmbH, Sonnenallee 1, D-85551 
  Kirchheim-Heimstetten
  Amtsgericht Muenchen: HRB 161028
  Geschaeftsfuehrer: Thomas Schroeder, Wolfgang Engels, Wolf Frenkel
  Vorsitzender des Aufsichtsrates: Martin Haering
  
  
 
 -- 
 Jorgen Lundman   | lund...@lundman.net
 Unix Administrator   | +81 (0)3 -5456-2687 ext 1017 (work)
 Shibuya-ku, Tokyo| +81 (0)90-5578-8500  (cell)
 Japan| +81 (0)3 -3375-1767  (home)
 ___
 zfs-discuss mailing list
 zfs-discuss@opensolaris.org
 http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
-- 
Sitz der Gesellschaft: Sun Microsystems GmbH, Sonnenallee 1, D-85551
Kirchheim-Heimstetten, Amtsgericht Muenchen: HRB 161028
Geschaeftsfuehrer: Thomas Schröder, Wolfgang Engels, Wolf Frenkel
Vorsitzender des Aufsichtsrates: Martin Haering 

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] Zfs send speed. Was: User quota design discussion..

2009-05-26 Thread Dirk Wriedt

Jorgen,

what is the size of the sending zfs?

I thought replication speed depends on the size of the sending fs, too not only size of the 
snapshot being sent.


Regards
Dirk


--On Freitag, Mai 22, 2009 19:19:34 +0900 Jorgen Lundman lund...@gmo.jp wrote:


Sorry, yes. It is straight;

# time zfs send zpool1/leroy_c...@speedtest | nc 172.20.12.232 3001
real19m48.199s

# /var/tmp/nc -l -p 3001 -vvv | time zfs recv -v zpool1/le...@speedtest
received 82.3GB stream in 1195 seconds (70.5MB/sec)


Sending is osol-b114.
Receiver is Solaris 10 10/08

When we tested Solaris 10 10/08 - Solaris 10 10/08 these were the results;

zfs send | nc | zfs recv - 1 MB/s
tar -cvf /zpool/leroy | nc | tar -xvf -  - 2.5 MB/s
ufsdump | nc | ufsrestore- 5.0 MB/s

So, none of those solutions was usable with regular Sol 10. Note most our 
volumes are ufs in
zvol, but even zfs volumes were slow.

Someone else had mentioned the speed was fixed in an earlier release, I had not 
had a chance to
upgrade. But since we wanted to try zfs user-quotas, I finally had the chance.

Lund


Brent Jones wrote:

On Thu, May 21, 2009 at 10:17 PM, Jorgen Lundman lund...@gmo.jp wrote:

To finally close my quest. I tested zfs send in osol-b114 version:

received 82.3GB stream in 1195 seconds (70.5MB/sec)

Yeeaahh!

That makes it completely usable! Just need to change our support contract to
allow us to run b114 and we're set! :)


Thanks,

Lund


Jorgen Lundman wrote:

We finally managed to upgrade the production x4500s to Sol 10 10/08
(unrelated to this) but with the hope that it would also make zfs send
usable.

Exactly how does build 105 translate to Solaris 10 10/08?  My current
speed test has sent 34Gb in 24 hours, which isn't great. Perhaps the next
version of Solaris 10 will have the improvements.

1



Robert Milkowski wrote:

Hello Jorgen,

If you look at the list archives you will see that it made a huge
difference for some people including me. Now I'm easily able to
saturate GbE linke while zfs send|recv'ing.


Since build 105 it should be *MUCH* for faster.



--
Jorgen Lundman   | lund...@lundman.net
Unix Administrator   | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo| +81 (0)90-5578-8500  (cell)
Japan| +81 (0)3 -3375-1767  (home)
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss



Can you give any details about your data set, what you piped zfs
send/receive through (SSH?), hardware/network, etc?
I'm envious of your speeds!




--
Dirk Wriedt, dirk.wri...@sun.com, Sun Microsystems GmbH
Systemingenieur Strategic Accounts
Nagelsweg 55, 20097 Hamburg, Germany
Tel.: +49-40-251523-132 Fax: +49-40-251523-425 Mobile: +49 172 848 4166
Never been afraid of chances I been takin' - Joan Jett

Sitz der Gesellschaft: Sun Microsystems GmbH, Sonnenallee 1, D-85551 
Kirchheim-Heimstetten
Amtsgericht Muenchen: HRB 161028
Geschaeftsfuehrer: Thomas Schroeder, Wolfgang Engels, Wolf Frenkel
Vorsitzender des Aufsichtsrates: Martin Haering

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] Zfs send speed. Was: User quota design discussion..

2009-05-26 Thread Jorgen Lundman



So you recommend I also do speed test on larger volumes? The test data I 
had on the b114 server was only 90GB. Previous tests included 500G ufs 
on zvol etc.  It is just it will take 4 days to send it to the b114 
server to start with ;) (From Sol10 servers).


Lund

Dirk Wriedt wrote:

Jorgen,

what is the size of the sending zfs?

I thought replication speed depends on the size of the sending fs, too 
not only size of the snapshot being sent.


Regards
Dirk


--On Freitag, Mai 22, 2009 19:19:34 +0900 Jorgen Lundman 
lund...@gmo.jp wrote:



Sorry, yes. It is straight;

# time zfs send zpool1/leroy_c...@speedtest | nc 172.20.12.232 3001
real19m48.199s

# /var/tmp/nc -l -p 3001 -vvv | time zfs recv -v zpool1/le...@speedtest
received 82.3GB stream in 1195 seconds (70.5MB/sec)


Sending is osol-b114.
Receiver is Solaris 10 10/08

When we tested Solaris 10 10/08 - Solaris 10 10/08 these were the 
results;


zfs send | nc | zfs recv - 1 MB/s
tar -cvf /zpool/leroy | nc | tar -xvf -  - 2.5 MB/s
ufsdump | nc | ufsrestore- 5.0 MB/s

So, none of those solutions was usable with regular Sol 10. Note most 
our volumes are ufs in

zvol, but even zfs volumes were slow.

Someone else had mentioned the speed was fixed in an earlier release, 
I had not had a chance to
upgrade. But since we wanted to try zfs user-quotas, I finally had the 
chance.


Lund


Brent Jones wrote:

On Thu, May 21, 2009 at 10:17 PM, Jorgen Lundman lund...@gmo.jp wrote:

To finally close my quest. I tested zfs send in osol-b114 version:

received 82.3GB stream in 1195 seconds (70.5MB/sec)

Yeeaahh!

That makes it completely usable! Just need to change our support 
contract to

allow us to run b114 and we're set! :)


Thanks,

Lund


Jorgen Lundman wrote:

We finally managed to upgrade the production x4500s to Sol 10 10/08
(unrelated to this) but with the hope that it would also make zfs 
send

usable.

Exactly how does build 105 translate to Solaris 10 10/08?  My 
current
speed test has sent 34Gb in 24 hours, which isn't great. Perhaps 
the next

version of Solaris 10 will have the improvements.

1



Robert Milkowski wrote:

Hello Jorgen,

If you look at the list archives you will see that it made a huge
difference for some people including me. Now I'm easily able to
saturate GbE linke while zfs send|recv'ing.


Since build 105 it should be *MUCH* for faster.



--
Jorgen Lundman   | lund...@lundman.net
Unix Administrator   | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo| +81 (0)90-5578-8500  (cell)
Japan| +81 (0)3 -3375-1767  (home)
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss



Can you give any details about your data set, what you piped zfs
send/receive through (SSH?), hardware/network, etc?
I'm envious of your speeds!




--
Dirk Wriedt, dirk.wri...@sun.com, Sun Microsystems GmbH
Systemingenieur Strategic Accounts
Nagelsweg 55, 20097 Hamburg, Germany
Tel.: +49-40-251523-132 Fax: +49-40-251523-425 Mobile: +49 172 848 4166
Never been afraid of chances I been takin' - Joan Jett

Sitz der Gesellschaft: Sun Microsystems GmbH, Sonnenallee 1, D-85551 
Kirchheim-Heimstetten

Amtsgericht Muenchen: HRB 161028
Geschaeftsfuehrer: Thomas Schroeder, Wolfgang Engels, Wolf Frenkel
Vorsitzender des Aufsichtsrates: Martin Haering




--
Jorgen Lundman   | lund...@lundman.net
Unix Administrator   | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo| +81 (0)90-5578-8500  (cell)
Japan| +81 (0)3 -3375-1767  (home)
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] Zfs send speed. Was: User quota design discussion..

2009-05-22 Thread Robert Milkowski



btw: caching data fro zfs send anf zfs recv on another side could make it 
even faster. you could use something like mbuffer with buffers of 1-2GB 
for example.



On Fri, 22 May 2009, Jorgen Lundman wrote:



To finally close my quest. I tested zfs send in osol-b114 version:

received 82.3GB stream in 1195 seconds (70.5MB/sec)

Yeeaahh!

That makes it completely usable! Just need to change our support contract to 
allow us to run b114 and we're set! :)



Thanks,

Lund


Jorgen Lundman wrote:


We finally managed to upgrade the production x4500s to Sol 10 10/08 
(unrelated to this) but with the hope that it would also make zfs send 
usable.


Exactly how does build 105 translate to Solaris 10 10/08?  My current 
speed test has sent 34Gb in 24 hours, which isn't great. Perhaps the next 
version of Solaris 10 will have the improvements.




Robert Milkowski wrote:

Hello Jorgen,

If you look at the list archives you will see that it made a huge
difference for some people including me. Now I'm easily able to
saturate GbE linke while zfs send|recv'ing.


Since build 105 it should be *MUCH* for faster.





--
Jorgen Lundman   | lund...@lundman.net
Unix Administrator   | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo| +81 (0)90-5578-8500  (cell)
Japan| +81 (0)3 -3375-1767  (home)
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] Zfs send speed. Was: User quota design discussion..

2009-05-22 Thread Ian Collins

Brent Jones wrote:

On Thu, May 21, 2009 at 10:17 PM, Jorgen Lundman lund...@gmo.jp wrote:
  

To finally close my quest. I tested zfs send in osol-b114 version:

received 82.3GB stream in 1195 seconds (70.5MB/sec)


Can you give any details about your data set, what you piped zfs
send/receive through (SSH?), hardware/network, etc?
I'm envious of your speeds!

  
I've managed close to that for full sends on Solaris 10 using direct 
socket connections and a few seconds of buffering.


--
Ian.

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] Zfs send speed. Was: User quota design discussion..

2009-05-22 Thread Jorgen Lundman

Sorry, yes. It is straight;

# time zfs send zpool1/leroy_c...@speedtest | nc 172.20.12.232 3001
real19m48.199s

# /var/tmp/nc -l -p 3001 -vvv | time zfs recv -v zpool1/le...@speedtest
received 82.3GB stream in 1195 seconds (70.5MB/sec)


Sending is osol-b114.
Receiver is Solaris 10 10/08

When we tested Solaris 10 10/08 - Solaris 10 10/08 these were the results;

zfs send | nc | zfs recv - 1 MB/s
tar -cvf /zpool/leroy | nc | tar -xvf -  - 2.5 MB/s
ufsdump | nc | ufsrestore- 5.0 MB/s

So, none of those solutions was usable with regular Sol 10. Note most 
our volumes are ufs in zvol, but even zfs volumes were slow.


Someone else had mentioned the speed was fixed in an earlier release, I 
had not had a chance to upgrade. But since we wanted to try zfs 
user-quotas, I finally had the chance.


Lund


Brent Jones wrote:

On Thu, May 21, 2009 at 10:17 PM, Jorgen Lundman lund...@gmo.jp wrote:

To finally close my quest. I tested zfs send in osol-b114 version:

received 82.3GB stream in 1195 seconds (70.5MB/sec)

Yeeaahh!

That makes it completely usable! Just need to change our support contract to
allow us to run b114 and we're set! :)


Thanks,

Lund


Jorgen Lundman wrote:

We finally managed to upgrade the production x4500s to Sol 10 10/08
(unrelated to this) but with the hope that it would also make zfs send
usable.

Exactly how does build 105 translate to Solaris 10 10/08?  My current
speed test has sent 34Gb in 24 hours, which isn't great. Perhaps the next
version of Solaris 10 will have the improvements.

1



Robert Milkowski wrote:

Hello Jorgen,

If you look at the list archives you will see that it made a huge
difference for some people including me. Now I'm easily able to
saturate GbE linke while zfs send|recv'ing.


Since build 105 it should be *MUCH* for faster.



--
Jorgen Lundman   | lund...@lundman.net
Unix Administrator   | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo| +81 (0)90-5578-8500  (cell)
Japan| +81 (0)3 -3375-1767  (home)
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss



Can you give any details about your data set, what you piped zfs
send/receive through (SSH?), hardware/network, etc?
I'm envious of your speeds!



--
Jorgen Lundman   | lund...@lundman.net
Unix Administrator   | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo| +81 (0)90-5578-8500  (cell)
Japan| +81 (0)3 -3375-1767  (home)
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] Zfs send speed. Was: User quota design discussion..

2009-05-22 Thread Eric D. Mudama

On Fri, May 22 at 11:05, Robert Milkowski wrote:



btw: caching data fro zfs send anf zfs recv on another side could make it 
even faster. you could use something like mbuffer with buffers of 1-2GB  
for example.


As another datapoint, the 111a opensolaris preview got me ~29MB/s
through an SSH tunnel with no tuning on a 40GB dataset.

Sender was a Core2Duo E4500 reading from SSDs and receiver was a Xeon
E5520 writing to a few mirrored 7200RPM SATA vdevs in a single pool.
Network was a $35 8-port gigabit netgear switch.

--eric

--
Eric D. Mudama
edmud...@mail.bounceswoosh.org

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] Zfs send speed. Was: User quota design discussion..

2009-05-22 Thread Nicolas Williams
On Fri, May 22, 2009 at 04:40:43PM -0600, Eric D. Mudama wrote:
 As another datapoint, the 111a opensolaris preview got me ~29MB/s
 through an SSH tunnel with no tuning on a 40GB dataset.
 
 Sender was a Core2Duo E4500 reading from SSDs and receiver was a Xeon
 E5520 writing to a few mirrored 7200RPM SATA vdevs in a single pool.
 Network was a $35 8-port gigabit netgear switch.

Unfortunately the SunSSH doesn't know how to grow SSHv2 channel windows
to take full advantage of the TCP BDP, so you could probably have gone
faster.
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] Zfs send speed. Was: User quota design discussion..

2009-05-21 Thread Jorgen Lundman


To finally close my quest. I tested zfs send in osol-b114 version:

received 82.3GB stream in 1195 seconds (70.5MB/sec)

Yeeaahh!

That makes it completely usable! Just need to change our support 
contract to allow us to run b114 and we're set! :)



Thanks,

Lund


Jorgen Lundman wrote:


We finally managed to upgrade the production x4500s to Sol 10 10/08 
(unrelated to this) but with the hope that it would also make zfs send 
usable.


Exactly how does build 105 translate to Solaris 10 10/08?  My current 
speed test has sent 34Gb in 24 hours, which isn't great. Perhaps the 
next version of Solaris 10 will have the improvements.




Robert Milkowski wrote:

Hello Jorgen,

If you look at the list archives you will see that it made a huge
difference for some people including me. Now I'm easily able to
saturate GbE linke while zfs send|recv'ing.


Since build 105 it should be *MUCH* for faster.





--
Jorgen Lundman   | lund...@lundman.net
Unix Administrator   | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo| +81 (0)90-5578-8500  (cell)
Japan| +81 (0)3 -3375-1767  (home)
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] Zfs send speed. Was: User quota design discussion..

2009-04-10 Thread Vladimir Kotal

Jorgen Lundman wrote:


We finally managed to upgrade the production x4500s to Sol 10 10/08 
(unrelated to this) but with the hope that it would also make zfs send 
usable.


Exactly how does build 105 translate to Solaris 10 10/08?  My current 


There is no easy/obvious mapping of Solaris Nevada builds to Solaris 10 
update releases. Solaris Nevada started as a branch of S10 after it was 
released and is the place where new features (RFEs) are developed. For a 
bug fix or RFE to end up in Solaris 10 update release it needs to match 
certain criteria. Basically, only those CRs which are found necessary 
(and this applies to both bugs and features) are backported to S10uX.



v.
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] Zfs send speed. Was: User quota design discussion..

2009-04-09 Thread Jorgen Lundman


We finally managed to upgrade the production x4500s to Sol 10 10/08 
(unrelated to this) but with the hope that it would also make zfs send 
usable.


Exactly how does build 105 translate to Solaris 10 10/08?  My current 
speed test has sent 34Gb in 24 hours, which isn't great. Perhaps the 
next version of Solaris 10 will have the improvements.




Robert Milkowski wrote:

Hello Jorgen,

If you look at the list archives you will see that it made a huge
difference for some people including me. Now I'm easily able to
saturate GbE linke while zfs send|recv'ing.


Since build 105 it should be *MUCH* for faster.



--
Jorgen Lundman   | lund...@lundman.net
Unix Administrator   | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo| +81 (0)90-5578-8500  (cell)
Japan| +81 (0)3 -3375-1767  (home)
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss