Re: [zfs-discuss] [Fwd: Re: Presales support on ZFS]

2008-02-11 Thread Jim Dunham

Enrico,


Is there any forecast to improve the efficiency of the replication
mechanisms of ZFS ? Fishwork - new NAS release 


I would take some time to talk with and understand exactly what the  
customer's expectation are for replication. i would not base my  
decision on the cost of replicating 10 bytes, regardless of how  
inefficient it may be.


These two documents should help:

http://www.sun.com/storagetek/white-papers/data_replication_strategies.pdf
http://www.sun.com/storagetek/white-papers/enterprise_continuity.pdf

Two key metrics of replication are:

Recovery Point Objective (RPO), is the amount of data lost (or less),  
measured as a unit time. Once a day backups yield a 24 hour RPO, once  
an hour snapshots yields ~1 hour RPO, asynchronous replication yields  
zero seconds to a few minutes RPO, and synchronous replication means  
zero seconds RPO.


Recovery Time Objective (RTO), is the amount of time after a failure,  
until normal operations are restored. Tapes backups could be minutes  
to hours, local snapshots could be nearly instantaneous, assuming the  
local site survived the failure. Remote snapshots or replicas could be  
minutes, hours or days, depending on the amount of data to  
resynchronize, impacted by network bandwidth and latency.


Availability Suite has a unique feature in this last area, called on- 
demand pull. Assuming that the primary site's volumes are lost, after  
they have been re-provisioned, a reverse update can be initiated.  
Besides the background resilvering in the reverse direction being  
active, eventually restoring all lost data, on-demand pull performs  
synchronous replication of data blocks on demand, as needed by the  
filesystem, database or application. Although the performance will be  
less then synchronous replication, the RTO is quite low. This type of  
recovery is analogous to loosing one's entire email account, having  
recovery initiated, but also selected email can be open as needed  
before the entire volume is restored, using  on demand requests to  
satisfy data blocks for relevant email requests.


Jim





Considering the solution we are offering to our customer ( 5 remote
sites replicating in one central data-center ) with ZFS ( cheapest
solution )  I should consider
3 times the network load of a solution based on SNDR-AVS and 3 times  
the

storage space too..correct ?

I there any documentation on that ?
Thanks

Richard Elling ha scritto:

Enrico Rampazzo wrote:

Hello
I'm offering a solution based on our disks where replication and
storage management should be made using only ZFS...
The test change few bytes on one file ( 10bytes ) and check how
many bytes the source sends to target.
The customer tried the replication between 2 volume...They  
compared

ZFS replica with true copy replica and they realized the following
considerations:

 1. ZFS uses a block bigger than HDS true copy



ZFS uses dynamic block sizes.  Depending on the configuration and
workload, just a few disk blocks will change, or a bunch of redundant
metadata might change.  In either case, changing the ZFS recordsize
will make little, if any, change.

 2. true copy sends 32Kbytes and ZFS 100K and more changing only  
10

file bytes

Can we configure ZFS to improve replication efficiencies ?



By default, ZFS writes two copies of metadata. I would not recommend
reducing this because it will increase your exposure to faults.   
What may

be happening here is that a 10 byte write may cause a metadata change
resulting in a minimum of three 512 byte physical blocks being
changed. The metadata copies are on spatially diverse, so you may see
these three
blocks starting at non-contiguous boundaries.  If Truecopy sends only
32kByte blocks (speculation), then the remote transfer will be  
96kBytes

for 3 local, physical block writes.

OTOH, ZFS will coalesce writes.  So you may be able to update a
number of files yet still only replicate 96kBytes through Truecopy.
YMMV.

Since the customer is performing replication, I'll assume they are  
very

interested in data protection, so keeping the redundant metadata is a
good idea. The customer should also be aware that replication at the
application level is *always* more efficient than replicating  
somewhere

down the software stack where you lose data context.
-- richard


The solution should consider 5 remote site replicating on one
central data-center. Considering the zfs block overhead the
customer is thinking to buy a solution based on traditional  
storage

arrays like HDS entry level arrays ( our 2530/2540 ). If so ..with
the ZFS the network traffic, storage space become big problems for
the customer infrastructures.

Are there any documentation explaining internal ZFS replication
mechanism to face the customer doubts ? Thanks


Do we need of AVS in our solution to solve the problem ?


Thanks



___
zfs-discuss mailing list
zfs-discuss@opensolaris.org

Re: [zfs-discuss] [Fwd: Re: Presales support on ZFS]

2008-02-11 Thread Enrico Rampazzo
Is there any forecast to improve the efficiency of the replication 
mechanisms of ZFS ? Fishwork - new NAS release 

Considering the solution we are offering to our customer ( 5 remote 
sites replicating in one central data-center ) with ZFS ( cheapest 
solution )  I should consider
3 times the network load of a solution based on SNDR-AVS and 3 times the 
storage space too..correct ?

I there any documentation on that ?
Thanks

Richard Elling ha scritto:
 Enrico Rampazzo wrote:
 Hello
 I'm offering a solution based on our disks where replication and 
 storage management should be made using only ZFS...
 The test change few bytes on one file ( 10bytes ) and check how 
 many bytes the source sends to target.
 The customer tried the replication between 2 volume...They compared 
 ZFS replica with true copy replica and they realized the following 
 considerations:

   1. ZFS uses a block bigger than HDS true copy
   

 ZFS uses dynamic block sizes.  Depending on the configuration and
 workload, just a few disk blocks will change, or a bunch of redundant
 metadata might change.  In either case, changing the ZFS recordsize
 will make little, if any, change.

   2. true copy sends 32Kbytes and ZFS 100K and more changing only 10
  file bytes

 Can we configure ZFS to improve replication efficiencies ?
   

 By default, ZFS writes two copies of metadata. I would not recommend
 reducing this because it will increase your exposure to faults.  What may
 be happening here is that a 10 byte write may cause a metadata change
 resulting in a minimum of three 512 byte physical blocks being 
 changed. The metadata copies are on spatially diverse, so you may see 
 these three
 blocks starting at non-contiguous boundaries.  If Truecopy sends only
 32kByte blocks (speculation), then the remote transfer will be 96kBytes
 for 3 local, physical block writes.

 OTOH, ZFS will coalesce writes.  So you may be able to update a
 number of files yet still only replicate 96kBytes through Truecopy.
 YMMV.

 Since the customer is performing replication, I'll assume they are very
 interested in data protection, so keeping the redundant metadata is a
 good idea. The customer should also be aware that replication at the
 application level is *always* more efficient than replicating somewhere
 down the software stack where you lose data context.
 -- richard

 The solution should consider 5 remote site replicating on one 
 central data-center. Considering the zfs block overhead the 
 customer is thinking to buy a solution based on traditional storage 
 arrays like HDS entry level arrays ( our 2530/2540 ). If so ..with 
 the ZFS the network traffic, storage space become big problems for 
 the customer infrastructures.

 Are there any documentation explaining internal ZFS replication 
 mechanism to face the customer doubts ? Thanks
   
 Do we need of AVS in our solution to solve the problem ?
  
 Thanks
   

 ___
 zfs-discuss mailing list
 zfs-discuss@opensolaris.org
 http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
   

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] [Fwd: Re: Presales support on ZFS]

2008-01-29 Thread Jim Dunham
Enrico,


 Hello
 I'm offering a solution based on our disks where replication and
 storage management should be made using only ZFS...
 The test change few bytes on one file ( 10bytes ) and check how many
 bytes the source sends to target.
 The customer tried the replication between 2 volume...They compared
 ZFS replica with true copy replica and they realized the following
 considerations:

  1. ZFS uses a block bigger than HDS true copy
  2. true copy sends 32Kbytes and ZFS 100K and more changing only 10
 file bytes

 Can we configure ZFS to improve replication efficiencies ?

 The solution should consider 5 remote site replicating on one  
 central
 data-center. Considering the zfs block overhead the customer is
 thinking to buy a solution based on traditional storage arrays like
 HDS entry level arrays ( our 2530/2540 ). If so ..with the ZFS the
 network traffic, storage space become big problems for the customer
 infrastructures.

 Are there any documentation explaining internal ZFS replication
 mechanism to face the customer doubts ? Thanks
 Do we need of AVS in our solution to solve the problem ?

AVS, not unlike HDS, does block-based replication based on actual  
write I/Os to configured devices.  Therefore if the means for ZFS to  
change 10 bytes, results in ZFS writing 100KB or more, AVS will be  
essentially be no different than HDS in this specific area.

Of course this begs to question, is the measure of a 10 byte change to  
a given file, a viable metric for choosing one form of replication  
over another? I think, or would hope not.  What is need is a  
characterization of the application(s) write-rate to one or more ZFS  
filesystems, over the customers requirements for data replication.

A good place to start is:
http://www.sun.com/storagetek/white-papers/data_replication_strategies.pdf
http://www.sun.com/storagetek/white-papers/enterprise_continuity.pdf



 Thanks

 ___
 zfs-discuss mailing list
 zfs-discuss@opensolaris.org
 http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Jim Dunham
Storage Platform Software Group
Sun Microsystems, Inc.
wk: 781.442.4042

http://blogs.sun.com/avs
http://www.opensolaris.org/os/project/avs/
http://www.opensolaris.org/os/project/iscsitgt/
http://www.opensolaris.org/os/community/storage/

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] [Fwd: Re: Presales support on ZFS]

2008-01-29 Thread Richard Elling
Enrico Rampazzo wrote:
 Hello
 I'm offering a solution based on our disks where replication and 
 storage management should be made using only ZFS...
 The test change few bytes on one file ( 10bytes ) and check how many 
 bytes the source sends to target.
 The customer tried the replication between 2 volume...They compared 
 ZFS replica with true copy replica and they realized the following 
 considerations:

   1. ZFS uses a block bigger than HDS true copy
   

ZFS uses dynamic block sizes.  Depending on the configuration and
workload, just a few disk blocks will change, or a bunch of redundant
metadata might change.  In either case, changing the ZFS recordsize
will make little, if any, change.

   2. true copy sends 32Kbytes and ZFS 100K and more changing only 10
  file bytes

 Can we configure ZFS to improve replication efficiencies ?
   

By default, ZFS writes two copies of metadata. I would not recommend
reducing this because it will increase your exposure to faults.  What may
be happening here is that a 10 byte write may cause a metadata change
resulting in a minimum of three 512 byte physical blocks being changed. 
The metadata copies are on spatially diverse, so you may see these three
blocks starting at non-contiguous boundaries.  If Truecopy sends only
32kByte blocks (speculation), then the remote transfer will be 96kBytes
for 3 local, physical block writes.

OTOH, ZFS will coalesce writes.  So you may be able to update a
number of files yet still only replicate 96kBytes through Truecopy.
YMMV.

Since the customer is performing replication, I'll assume they are very
interested in data protection, so keeping the redundant metadata is a
good idea. The customer should also be aware that replication at the
application level is *always* more efficient than replicating somewhere
down the software stack where you lose data context.
 -- richard

 The solution should consider 5 remote site replicating on one central 
 data-center. Considering the zfs block overhead the customer is 
 thinking to buy a solution based on traditional storage arrays like 
 HDS entry level arrays ( our 2530/2540 ). If so ..with the ZFS the 
 network traffic, storage space become big problems for the customer 
 infrastructures.

 Are there any documentation explaining internal ZFS replication 
 mechanism to face the customer doubts ? Thanks
   
 Do we need of AVS in our solution to solve the problem ?
   
 Thanks
   

 ___
 zfs-discuss mailing list
 zfs-discuss@opensolaris.org
 http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
   

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss