date:20060906

Re: [zfs-discuss] Re: ZFS uses 1.1GB more space, reports conflicting information...

2006-09-06 Thread Wee Yeh Tan


On 9/6/06, UNIX admin <[EMAIL PROTECTED]> wrote:

Yes, the man page says that. However, it is possible to mix disks of different 
sizes in a RAIDZ, and this works. Why does it work? Because RAIDZ stripes are 
dynamic in size. From that I infer that disks can be any size because the 
stripes can be any size. Should one or more of the smaller disks become full, 
the stripe information is written across the remaining space on the rest of the 
bigger disks. Or should be anyway. If you think about it, it makes sense.


You are right that it can be done in theory.  The practical difficulty
is to do it efficiently.  My raidz pool tend to have a strong
preference to utilize all the disks so disks with different sizes
might result in smaller column widths later.  I'm not sure if that
spells any performance issues but at least the capacity difference
between the 2 largest disk will not be usable.

Is maximising use of capacity of all available disk in the pipeline
for RAID-Z or is it to be avoided intentionally?


--
Just me,
Wire ...
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

[zfs-discuss] Overview (rollup) of recent activity on zfs-discuss

2006-09-06 Thread Eric Boutilier


For background on what this is, see:

http://www.opensolaris.org/jive/message.jspa?messageID=24416#24416
http://www.opensolaris.org/jive/message.jspa?messageID=25200#25200

=
zfs-discuss 08/16 - 08/31
=

Size of all threads during period:

Thread size Topic
--- -
 22   ZFS + rsync, backup on steroids.
 13   SCSI synchronize cache cmd
 13   Niagara and ZFS compression?
 12   ZFS Boot Disk
 10   zpool status panics server
 10   ZFS & se6920
  9   Oracle on ZFS
  8   zpool core dumped
  8   in-kernel gzip compression
  8   Significant "pauses" during zfs writes
  7   Home Server with ZFS
  7   3510 - some new tests
  6   unaccounted for daily growth in ZFS disk space usage
  6   ZFS write performance problem with compression set to ON
  6   Why is ZFS raidz slower than simple ufs mount?
  6   Tape backup
  6   SPEC SFS97 benchmark of ZFS,UFS,VxFS
  6   Proposal: user-defined properties
  6   Porting ZFS file system to FreeBSD.
  6   Need Help: didn't create the pool as radiz but stripes
  6   Information on ZFS API?
  5   zpool import: snv_33 to S10 6/06
  5   multi-layer ZFS filesystems and exporting: my stupid question for 
the day
  5   commercial backup software and zfs
  5   ZFS compression
  5   ZFS Filesytem Corrpution
  5   Issue with zfs snapshot replication from version2 to version3 
pool.
  5   File level compression
  4   zpool import dropped a core.
  4   zpool import - cannot mount [...] directory is not empty
  4   zpool hangs
  4   destroyed pools signatures
  4   [fbsd] Porting ZFS file system to FreeBSD.
  4   ZFS with expanding LUNs
  4   ZFS web admin - No items found.
  4   ZFS and very large directories
  4   ZFS Load-balancing over vdevs vs. real disks?
  4   Storage Compatibilty list
  4   Find the difference between two snapshots
  4   Encryption on ZFS / Disk Usage
  4   Compression with fixed record sizes
  3   zpool iostat, scrubbing increases used disk space
  3   zpool import/export
  3   system unresponsive after issuing a zpool attach
  3   problem with zfs receive -i
  3   does zfs can be failover between two mechine?
  3   does anybody port the zfs webadmin to webmin?
  3   can't create snapshot
  3   ZFS questions with mirrors
  3   ZFS compression / space efficiency
  3   ZFS Performance compared to UFS & VxFS
  3   Want to try ZFS, but "format" got  message
  3   Newbie questions about drive problems
  3   Can a zfs storage pool be imported readonly?
  2   zfs and vmware
  2   new ZFS links page
  2   neopath vs automount
  2   migrating data across boxes
  2   libzfs question
  2   delete acl not working on zfs.v3?
  2   ZFS 'quot' command
  2   Tunable parameter to zfs memory use
  2   Solaris 6/06 ZFS and OpenSolaris ZFS
  2   Interesting zfs destroy failure
  1   zpool status inconsistent after user error?
  1   zfs list - unexpected error 17 at line 1684
  1   space accounting with RAID-Z
  1   pool ID
  1   fdatasync
  1   ZFS Performance compared to UFS & VxFS - offtopic
  1   ZFS ACL: append_data didn't do what I expected
  1   Sol 10 x86_64 intermittent SATA device locks up server
  1   Question on Zones and memory usage (65120349)
  1   Query on ZFS
  1   Equivalent of command quot
  1   Apple Time Machine


Posting activity by person for period:

# of posts  By
--   --
 35   rmilkowski at task.gda.pl (robert milkowski)
 17   ahrens at eng.sun.com (matthew ahrens)
 15   eric.schrock at sun.com (eric schrock)
 11   roch.bourbonnais at sun.com (roch)
 11   fcusack at fcusack.com (frank cusack)
 10   matthew.ahrens at sun.com (matthew ahrens)
  9   jamesd.wi at gmail.com (james dickens)
  8   george.wilson at sun.com (george wilson)
  7   rasputnik at gmail.com (dick davies)
  6   mgerdts at gmail.com (mike gerdts)
  6   jmlittle at gmail.com (joe little)
  6   anantha.srirama at cdc.hhs.gov (anantha n. srirama)
  5   torrey.mcmahon at sun.com (torrey mcmahon)
  5   richard.elling at sun.com (richard elling - pae)
  5   nicolas.williams at sun.com (nicolas williams)
  5   michael.schuster at sun.com (michael schuster - sun microsystems)

Re: [zfs-discuss] Re: Re: ZFS forces system to paging to the point it is

2006-09-06 Thread Mark Maybee


Robert Milkowski wrote:



On Wed, 6 Sep 2006, Mark Maybee wrote:


Robert Milkowski wrote:


::dnlc!wc



 1048545 3145811 76522461


Well, that explains half your problem... and maybe all of it:




After I reduced vdev prefetch from 64K to 8K for last few hours system 
is working properly without workaround and free memory stays at about 1GB.


Reducing vdev prefetch to 8K alse reduced read thruoutput 10x.

I belive this is somehow related - maybe vdev cache was so aggressive (I 
got 40-100MB/s of reads) and consuming memory so fast that thread which 
is supposed to regain some memory couldn't keep up?


I suppose, although the data volume doesn't seem that high... maybe you
are just operating at the hairy edge here.  Anyway, I have filed a bug
to track this issue:

6467963 do_dnlc_reduce_cache() can be blocked by ZFS_OBJ_HOLD_ENTER()

-Mark
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] Re: Recommendation ZFS on StorEdge 3320

2006-09-06 Thread Torrey McMahon

Roch - PAE wrote:

Thinking some more about this. If your requirements does
mandate some form of mirroring, then it truly seems that ZFS
should take that in charge if only because of the

self-healing characteristics. So I feel the storage array's
job is to export low latency Luns to ZFS.

The hard part is getting a set of simple requirements. As you go into
more complex data center environments you get hit with older Solaris
revs, other OSs, SOX compliance issues, etc. etc. etc. The world where
most of us seem to be playing with ZFS is on the lower end of the
complexity scale. Sure, throw your desktop some fast SATA drives. No
problem. Oh wait, you've got ten Oracle DBs on three E25Ks that need to
be backed up every other blue moon ...

I agree with the general idea that an array, be it one disk or some raid
combination, should simply export low latency LUNs. However, its the
features offered by the array - Like site to site replication - used to
meet more complex requirements that literally slow things down. In many
cases you'll see years old operational procedures causing those low
latency LUNs to slow down even more. Something really hard to get a
customer to undo because a new fangled file system is out. ;)

I'd be happy to live with those simple Luns but I guess some
storage will just refuse to export non-protected luns. Now
we can definitively take advantage of the Array's capability
of exporting highly resilient Luns; RAID-5 seems to fit the
bill rather well here. Even an 9+1 luns will be quite
resilient and have a low block overhead.

I think 99x0 used to do 3+1 only. Now it's 7+1 if I recall. Close enough
I suppose.

So we benefit from the arrays resiliency as well as it's low
latency characteristics. And we mirror data at the ZFS level
which means great performance and great data integrity and

great availability.

Note that ZFS write characteristics (all sequential) means
that we will commonly be filling full stripes on the luns
thus avoiding the partial stripe performance pitfall.

One thing comes to mind in that case. Many arrays do sequential detect
on the blocks that come in to the front end ports.
If things get split up to much or out of order or array characteristic here> then you could induce more latency as the
array does cartwheels trying to figure out whats going on.

48 matches

Mail list logo