Re: [zfs-discuss] stupid ZFS question - floating point operations
Thanks for the clarification. I guess I need to go back and figure out how ZFS crypto keying is performed. I guess most likely the key is generated from some sort of one-way hash from a passphrase? - Garrett -Original Message- From: Darren J Moffat [mailto:darren.mof...@oracle.com] Sent: Thu 12/23/2010 1:32 AM To: Garrett D'Amore Cc: Erik Trimble; Jerry Kemp; zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] stupid ZFS question - floating point operations On 22/12/2010 20:27, Garrett D'Amore wrote: That said, some operations -- and cryptographic ones in particular -- may use floating point registers and operations because for some architectures (sun4u rings a bell) this can make certain expensive Well remembered! There are sun4u optimisations that use the floating point unit but those only apply to the bignum code which in kernel is only used by RSA. operations go faster. I don't think this is the case for secure hash/message digest algorithms, but if you use ZFS encryption as found in Solaris 11 Express you might find that on certain systems these registers are used for performance reasons, either on the bulk crypto or on the keying operations. (More likely the latter, but my memory of these optimizations is still hazy.) RSA isn't used at all by ZFS encryption, everything is AES (including key wrapping) and SHA256. So those optimistations for floating point don't come into play for ZFS encryption. -- Darren J Moffat ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] stupid ZFS question - floating point operations
On 22/12/2010 20:27, Garrett D'Amore wrote: That said, some operations -- and cryptographic ones in particular -- may use floating point registers and operations because for some architectures (sun4u rings a bell) this can make certain expensive Well remembered! There are sun4u optimisations that use the floating point unit but those only apply to the bignum code which in kernel is only used by RSA. operations go faster. I don't think this is the case for secure hash/message digest algorithms, but if you use ZFS encryption as found in Solaris 11 Express you might find that on certain systems these registers are used for performance reasons, either on the bulk crypto or on the keying operations. (More likely the latter, but my memory of these optimizations is still hazy.) RSA isn't used at all by ZFS encryption, everything is AES (including key wrapping) and SHA256. So those optimistations for floating point don't come into play for ZFS encryption. -- Darren J Moffat ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] stupid ZFS question - floating point operations
Darren J Moffat darren.mof...@oracle.com wrote: On 22/12/2010 20:27, Garrett D'Amore wrote: That said, some operations -- and cryptographic ones in particular -- may use floating point registers and operations because for some architectures (sun4u rings a bell) this can make certain expensive Well remembered! There are sun4u optimisations that use the floating point unit but those only apply to the bignum code which in kernel is only used by RSA. It may be a guess caused by the fact that integer division and multiplication is inside the FPU on SPARC processors. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] stupid ZFS question - floating point operations
On Thu, Dec 23, 2010 at 09:32:13AM +, Darren J Moffat wrote: On 22/12/2010 20:27, Garrett D'Amore wrote: That said, some operations -- and cryptographic ones in particular -- may use floating point registers and operations because for some architectures (sun4u rings a bell) this can make certain expensive Well remembered! There are sun4u optimisations that use the floating point unit but those only apply to the bignum code which in kernel is only used by RSA. operations go faster. I don't think this is the case for secure hash/message digest algorithms, but if you use ZFS encryption as found in Solaris 11 Express you might find that on certain systems these registers are used for performance reasons, either on the bulk crypto or on the keying operations. (More likely the latter, but my memory of these optimizations is still hazy.) RSA isn't used at all by ZFS encryption, everything is AES (including key wrapping) and SHA256. So those optimistations for floating point don't come into play for ZFS encryption. Moreover, we have platform-specific crypto optimizations. If there were FPU operations that help speed up symmetric crypto on an M4000 but not on UltraSPARC T2s, then we'd use that on the one but not on the other. Nico -- ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] stupid ZFS question - floating point operations
On 23/12/2010 15:18, Garrett D'Amore wrote: Thanks for the clarification. I guess I need to go back and figure out how ZFS crypto keying is performed. I guess most likely the key is generated from some sort of one-way hash from a passphrase? See http://blogs.sun.com/darren/entry/zfs_encryption_what_is_on where I explain all the type of keys used and how they are generated as well as how passphrases are turned into AES wrapping keys (using PKCS#5 PBE). -- Darren J Moffat ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] stupid ZFS question - floating point operations
On 23/12/2010 17:09, joerg.schill...@fokus.fraunhofer.de wrote: Darren J Moffatdarren.mof...@oracle.com wrote: On 22/12/2010 20:27, Garrett D'Amore wrote: That said, some operations -- and cryptographic ones in particular -- may use floating point registers and operations because for some architectures (sun4u rings a bell) this can make certain expensive Well remembered! There are sun4u optimisations that use the floating point unit but those only apply to the bignum code which in kernel is only used by RSA. It may be a guess caused by the fact that integer division and multiplication is inside the FPU on SPARC processors. Not a guess it is code to do big number integer arithmetic that is optimised for sun4u to explicitly (ab)using the FPU. This isn't guessing it is was a deliberate design choice. Specifically this code here: http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/common/bignum/sun4u/ Note that there are separate kernel and user land variants of that. -- Darren J Moffat ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] stupid ZFS question - floating point operations
On 22 December, 2010 - Jerry Kemp sent me these 1,0K bytes:
I have a coworker, who's primary expertise is in another flavor of Unix.
This coworker lists floating point operations as one of ZFS detriments.
I's not really sure what he means specifically, or where he got this
reference from.
Then maybe ask him first? Guilty until proven innocent isn't the regular
path...
In an effort to refute what I believe is an error or misunderstanding on
his part, I have spent time on Yahoo, Google, the ZFS section of
OpenSolaris.org, etc. I really haven't turned up much of anything that
would prove or disprove his comments. The one thing I haven't done is
to go through the ZFS source code, but its been years since I have done
any serious programming.
If someone from Oracle, or anyone on this mailing list could point me
towards any documentation, or give me a definitive word, I would sure
appreciate it. If there were floating point operations going on within
ZFS, at this point I am uncertain as to what they would be.
TIA for any comments,
Jerry
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
/Tomas
--
Tomas Ögren, st...@acc.umu.se, http://www.acc.umu.se/~stric/
|- Student at Computing Science, University of Umeå
`- Sysadmin at {cs,acc}.umu.se
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] stupid ZFS question - floating point operations
On 12/23/10 08:44 AM, Jerry Kemp wrote: I have a coworker, who's primary expertise is in another flavor of Unix. This coworker lists floating point operations as one of ZFS detriments. I's not really sure what he means specifically, or where he got this reference from. It sounds like your colleague is from another planet where filesystems perform computations! -- Ian. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] stupid ZFS question - floating point operations
On 22/12/10 2:44 PM, Jerry Kemp wrote: I have a coworker, who's primary expertise is in another flavor of Unix. This coworker lists floating point operations as one of ZFS detriments. Perhaps he can point you also to the equally mythical competing filesystem which offers ZFS' advantages. --Toby I's not really sure what he means specifically, or where he got this reference from. ... Jerry ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] stupid ZFS question - floating point operations
If I remember correctly Solaris like most other operating system does not save or restore the floating point registers when context switching from User to Kernel so doing any floating point ops in the kernel would corrupt user floating point state. This means ZFS cannot be doing any floating point ops in the kernel context. Others wiser than I may be able to asert this with more certainty -Angelo On Dec 22, 2010, at 2:44 PM, Jerry Kemp wrote: I have a coworker, who's primary expertise is in another flavor of Unix. This coworker lists floating point operations as one of ZFS detriments. I's not really sure what he means specifically, or where he got this reference from. In an effort to refute what I believe is an error or misunderstanding on his part, I have spent time on Yahoo, Google, the ZFS section of OpenSolaris.org, etc. I really haven't turned up much of anything that would prove or disprove his comments. The one thing I haven't done is to go through the ZFS source code, but its been years since I have done any serious programming. If someone from Oracle, or anyone on this mailing list could point me towards any documentation, or give me a definitive word, I would sure appreciate it. If there were floating point operations going on within ZFS, at this point I am uncertain as to what they would be. TIA for any comments, Jerry ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] stupid ZFS question - floating point operations
On 12/22/2010 11:49 AM, Tomas Ögren wrote: On 22 December, 2010 - Jerry Kemp sent me these 1,0K bytes: I have a coworker, who's primary expertise is in another flavor of Unix. This coworker lists floating point operations as one of ZFS detriments. I's not really sure what he means specifically, or where he got this reference from. Then maybe ask him first? Guilty until proven innocent isn't the regular path... In an effort to refute what I believe is an error or misunderstanding on his part, I have spent time on Yahoo, Google, the ZFS section of OpenSolaris.org, etc. I really haven't turned up much of anything that would prove or disprove his comments. The one thing I haven't done is to go through the ZFS source code, but its been years since I have done any serious programming. If someone from Oracle, or anyone on this mailing list could point me towards any documentation, or give me a definitive word, I would sure appreciate it. If there were floating point operations going on within ZFS, at this point I am uncertain as to what they would be. TIA for any comments, Jerry ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss /Tomas So far as my understanding of the codebase goes (and, while I've read a significant portion, I'm not really an expert here): Assuming he means that ZFS has a weakness of heavy floating-point calculation requirements (i.e using ZFS requires heavy FP usage), that's wrong. Like all normal filesystems, the ordinary operations are all integer, load, and store. The ordinary work of caching, block allocation, and fetching/writing is of course all integer-based. I can't imagine someone writing a filesystem which does such operations using floating point. A quick grep through the main ZFS sources doesn't find anything of type double or float. I think he might be confused with what is happening on Checksums (which is still all Integer, but looks/sounds expensive). Yes, ZFS is considerably *more* compute intensive than other filesystems. However, it's all Integer, and one of the base assumptions of ZFS is that modern systems have lots of excess CPU cycles around, so stealing 5% for use with ZFS won't impact performance much, and the added features of ZFS more than make up for any CPU cycles lost. -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA Timezone: US/Pacific (GMT-0800) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] stupid ZFS question - floating point operations
There are no floating points operations in zfs, however even if there would that wouldn't be a bad thing, as modern CPU are float monsters indeed its likely some things would be faster if converted to use the float ALU (note however those operations would have to account for the different properties of the ALUs). The most complex ALU parts of zfs are the Galois field math in the RAID codes and the checksum operations. All of which are integer ops (tho it might be interesting to see if the Galois field could be accelerated using the float ALU). Deano -Original Message- From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] On Behalf Of Jerry Kemp Sent: 22 December 2010 19:44 To: zfs-discuss@opensolaris.org Subject: [zfs-discuss] stupid ZFS question - floating point operations I have a coworker, who's primary expertise is in another flavor of Unix. This coworker lists floating point operations as one of ZFS detriments. I's not really sure what he means specifically, or where he got this reference from. In an effort to refute what I believe is an error or misunderstanding on his part, I have spent time on Yahoo, Google, the ZFS section of OpenSolaris.org, etc. I really haven't turned up much of anything that would prove or disprove his comments. The one thing I haven't done is to go through the ZFS source code, but its been years since I have done any serious programming. If someone from Oracle, or anyone on this mailing list could point me towards any documentation, or give me a definitive word, I would sure appreciate it. If there were floating point operations going on within ZFS, at this point I am uncertain as to what they would be. TIA for any comments, Jerry ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] stupid ZFS question - floating point operations
Generally, ZFS does not use floating point. And further, use of floating point in the kernel is exceptionally rare. The kernel does not save floating point context automatically, which means that code that uses floating point needs to take special care to make sure any context from userland is saved and restored before it can use the registers itself. This rather onerous burden tends to exclude easy consumption of floating point. That said, some operations -- and cryptographic ones in particular -- may use floating point registers and operations because for some architectures (sun4u rings a bell) this can make certain expensive operations go faster. I don't think this is the case for secure hash/message digest algorithms, but if you use ZFS encryption as found in Solaris 11 Express you might find that on certain systems these registers are used for performance reasons, either on the bulk crypto or on the keying operations. (More likely the latter, but my memory of these optimizations is still hazy.) Note that *if* this is done, it is only done where such an operation is a performance win, and not because any of the math is inherently floating point. So in this case, I would say that this optimization would be an advantage, rather than a disadvantage. Oh, and this usage only applies to Solaris, and is optional. I doubt FreeBSD has these particular enhancements -- indeed, IIRC, these optimizations are specific to certain classes of SPARC cpus and probably are not performed at all for x86 cpus. - Garrett -Original Message- From: zfs-discuss-boun...@opensolaris.org on behalf of Erik Trimble Sent: Wed 12/22/2010 12:08 PM To: Jerry Kemp; zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] stupid ZFS question - floating point operations On 12/22/2010 11:49 AM, Tomas Ögren wrote: On 22 December, 2010 - Jerry Kemp sent me these 1,0K bytes: I have a coworker, who's primary expertise is in another flavor of Unix. This coworker lists floating point operations as one of ZFS detriments. I's not really sure what he means specifically, or where he got this reference from. Then maybe ask him first? Guilty until proven innocent isn't the regular path... In an effort to refute what I believe is an error or misunderstanding on his part, I have spent time on Yahoo, Google, the ZFS section of OpenSolaris.org, etc. I really haven't turned up much of anything that would prove or disprove his comments. The one thing I haven't done is to go through the ZFS source code, but its been years since I have done any serious programming. If someone from Oracle, or anyone on this mailing list could point me towards any documentation, or give me a definitive word, I would sure appreciate it. If there were floating point operations going on within ZFS, at this point I am uncertain as to what they would be. TIA for any comments, Jerry ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss /Tomas So far as my understanding of the codebase goes (and, while I've read a significant portion, I'm not really an expert here): Assuming he means that ZFS has a weakness of heavy floating-point calculation requirements (i.e using ZFS requires heavy FP usage), that's wrong. Like all normal filesystems, the ordinary operations are all integer, load, and store. The ordinary work of caching, block allocation, and fetching/writing is of course all integer-based. I can't imagine someone writing a filesystem which does such operations using floating point. A quick grep through the main ZFS sources doesn't find anything of type double or float. I think he might be confused with what is happening on Checksums (which is still all Integer, but looks/sounds expensive). Yes, ZFS is considerably *more* compute intensive than other filesystems. However, it's all Integer, and one of the base assumptions of ZFS is that modern systems have lots of excess CPU cycles around, so stealing 5% for use with ZFS won't impact performance much, and the added features of ZFS more than make up for any CPU cycles lost. -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA Timezone: US/Pacific (GMT-0800) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] stupid ZFS question - floating point operations
On 2010-Dec-23 04:48:19 +0800, Deano de...@rattie.demon.co.uk wrote: modern CPU are float monsters indeed its likely some things would be faster if converted to use the float ALU _Some_ modern CPUs are good at FP, a lot aren't. The SPARC T-1 was particularly poor as it only had a single FPU. Likewise, performance in the x86 world is highly variable, depending on the vendor and core you pick. AFAIK, iA64 and PPC are consistently good - but neither are commonly found in conjunction with ZFS. You may also need to allow for software assist: Very few CPUs implement all of the IEEE FP standard in hardware and most (including SPARC) require software to implement parts of the standard. If your algorithm happens to make significant use of things other than normalised numbers and zero, your performance may be severely affected by the resultant traps and software assistance. Any use of floating point within the kernel also means changes to when FPU context is saved - and, unless this can be implemented lazily, it will adversely impact the cost of all context switches and potentially system calls. -- Peter Jeremy pgphVXYz2zc3s.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] stupid ZFS question - floating point operations
-Original Message- From: Peter Jeremy [mailto:peter.jer...@alcatel-lucent.com] Sent: 22 December 2010 21:17 To: Deano Cc: zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] stupid ZFS question - floating point operations On 2010-Dec-23 04:48:19 +0800, Deano de...@rattie.demon.co.uk wrote: modern CPU are float monsters indeed its likely some things would be faster if converted to use the float ALU Peter wrote _Some_ modern CPUs are good at FP, a lot aren't. The SPARC T-1 was particularly poor as it only had a single FPU. Likewise, performance in the x86 world is highly variable, depending on the vendor and core you pick. AFAIK, iA64 and PPC are consistently good - but neither are commonly found in conjunction with ZFS. You may also need to allow for software assist: Very few CPUs implement all of the IEEE FP standard in hardware and most (including SPARC) require software to implement parts of the standard. If your algorithm happens to make significant use of things other than normalised numbers and zero, your performance may be severely affected by the resultant traps and software assistance. I can't speak for old architecture like SPARC but all modern ALU designs support most of the subset of useful IEEE in hardware and at high speed. In particular x86 has extremely good float ALU performance, compared to some architectures it is relatively low but certainly not something to avoid. A CPU design that isn't at *least* 5 GFLOPS per core is archaic. This is only accelerating due to the consumer market that many CPUs end up in. From graphics to video decoding to audio synthesis, floating point math dominants. Not to say they aren't able to perform very many integer ALU ops as well, just that the old mantra that FPU is to be avoided hasn't been true for years. Any use of floating point within the kernel also means changes to when FPU context is saved - and, unless this can be implemented lazily, it will adversely impact the cost of all context switches and potentially system calls. Of course the cost of the extra register movement involved in context switches is a concern, but this cost can be evaluated against the gains. I like to see someone actually profile the costs in SunOs as many kernel architectures I know accept FPU (and other specialist registers) restoration when needed as a worthwhile cost. Bye, Deano ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
