Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 That line of argument could be used to justify putting anything and everything in core. I think that our extensible architecture is an important feature and one we should not hesitate to use to the fullest. I agree, but part of the problem here is that pgcrypto is extraordinary overkill for people who just want a better hash function than md5. Our extensible architecture is a feature, but our contrib/packaging/gborg/pgfoundry situation is a mess. It's only the efforts of the distro package maintainers that's kept things from being even worse. Here's what it boils down to for me: 1) Postgres has the md5() function, which is not ever getting removed. 2) Since it exists, people are using it. 3) Not having a builtin sha1() means we are less compatible with other databases. Fair? Perhaps not. But requiring an installation of pgcrypto, or plperl, is another hurdle to be cleared by people porting and using applications with Postgres as a backend. 4) We're also encouraging the use of md5() by making it the only option. Yes, we can talk about why people *shouldn't* use it for this purpose or that, but they will. 5) It seems unwise to go through the trouble of just adding sha1(), when we could easily add some better hashes, which has the nice side effect of making us stand out more and push the envelope, rather than play follow the leader, as was mentioned at PGCon East. - -- Greg Sabino Mullane [EMAIL PROTECTED] PGP Key: 0x14964AC8 200804031020 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iEYEAREDAAYFAkf06AIACgkQvJuQZxSWSshJGACcDlE/sUBTJNx36zMW7C9G2FqE n0QAoLOj50gGura/g2JCk+3sFxR0cLb1 =K8sl -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Greg Sabino Mullane wrote: 4) We're also encouraging the use of md5() by making it the only option. Yes, we can talk about why people *shouldn't* use it for this purpose or that, but they will. There is always the Java route - internal classes have package-scope constructors to specifically prevent them from being accidentally used (and relied on). I prefer the let them use it, but warn them not to have expectations route, which is what PostgreSQL is doing today. The above is not a legitimate reason to provide additional functions in the core. 5) It seems unwise to go through the trouble of just adding sha1(), when we could easily add some better hashes, which has the nice side effect of making us stand out more and push the envelope, rather than play follow the leader, as was mentioned at PGCon East This presumes that better hashes truly exist. It is basic math to show that all hashes will include collisions. Ignoring the possibility that one hash has theoretical better distribution for real documents, the real benefit of SHA-1 over MD5, is that it has more bits. The ultimate solution here, is to store the original using the full copy hash technique, with 0 chance of collision. This extreme defeats the purpose of a hash to start with. Why does PostgreSQL need something better than md5 as part of core? Bragging rights? Cheers, mark -- Mark Mielke [EMAIL PROTECTED] -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Mark Mielke wrote: This presumes that better hashes truly exist. It is basic math to show that all hashes will include collisions. Ignoring the possibility that one hash has theoretical better distribution for real documents, the real benefit of SHA-1 over MD5, is that it has more bits. The ultimate solution here, is to store the original using the full copy hash technique, with 0 chance of collision. This extreme defeats the purpose of a hash to start with. Why does PostgreSQL need something better than md5 as part of core? Bragging rights? Having more than one hash algorithm significantly decreases the risk of (common) collisions. As a non-developer (who does track most messages on the list anyways), I surely find the SHA* functions will add significantly value and they should be easy to install (well-defined functions) with no maintainance afterwards. Hashes are an absolute minimum for keeping passwords stored somehat safely in a database. More two or even three different hashes with different collion-points will strongly increase the security. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Svenne Krap wrote: Mark Mielke wrote: This presumes that better hashes truly exist. It is basic math to show that all hashes will include collisions. Ignoring the possibility that one hash has theoretical better distribution for real documents, the real benefit of SHA-1 over MD5, is that it has more bits. The ultimate solution here, is to store the original using the full copy hash technique, with 0 chance of collision. This extreme defeats the purpose of a hash to start with. Why does PostgreSQL need something better than md5 as part of core? Bragging rights? Having more than one hash algorithm significantly decreases the risk of (common) collisions. No it doesn't. More bits reduces risk of collisions. Additional algorithms just muddy the waters. As a non-developer (who does track most messages on the list anyways), I surely find the SHA* functions will add significantly value and they should be easy to install (well-defined functions) with no maintainance afterwards. Hashes are an absolute minimum for keeping passwords stored somehat safely in a database. It has yet to be proven that MD5 is insufficient for this purpose. Significant value being what? More two or even three different hashes with different collion-points will strongly increase the security. No it doesn't unless you are thinking about a security through obscurity argument. Cheers, mark -- Mark Mielke [EMAIL PROTECTED] -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
On Thu, Apr 03, 2008 at 06:14:17PM +0200, Svenne Krap wrote: Hashes are an absolute minimum for keeping passwords stored somehat safely in a database. More two or even three different hashes with different collion-points will strongly increase the security. Not only that, but they also increase the complexity of the system. Increases in complexity tend to mean decreases in reliability and, by implication, security. As an example, someone may do some fancy cryptanalysis and discover that having lots of hashes will actually make it easier. As another point, most passwords have significantly less state than a 128bit hash allowing attacks like rainbow tables become viable. Sam -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Mark Mielke wrote: More two or even three different hashes with different collion-points will strongly increase the security. No it doesn't unless you are thinking about a security through obscurity argument. It is really the same argument on all your questions If I have a simple table now ID serial Username varchar Password varchar I currently save only md5(id || username || 'password')* into password, if I had access to sha1 (for example) i would add another password column so, having for example ID serial Username varchar Password_md5 varchar Password_sha1 varchar No matter how you see it, I get more bits of hash to check against. I would drop md5 totally and use sha1 and ripemd-160 if possible.. but currently i use only md5 as it is the only available one.. Loading pgcrypto is overkill for something as simple as hash-functions. Svenne * I prepend the id and the username to guard users with weak passwords against known hashvalues (rainbow tables) should the box ever get comprised ... if you are in doubt about the value of this, try google for 40e94aa51dc5c0ccc5aad4e6aefdde2a and guess the secret password... -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Svenne Krap wrote: If I have a simple table now ID serial Username varchar Password varchar I currently save only md5(id || username || 'password')* into password, if I had access to sha1 (for example) i would add another password column so, having for example ID serial Username varchar Password_md5 varchar Password_sha1 varchar No matter how you see it, I get more bits of hash to check against. Really? Why stop at two, then? How many hash functions is enough? cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Mark Mielke wrote: Svenne Krap wrote: Mark Mielke wrote: Svenne Krap wrote: More two or even three different hashes with different collion-points will strongly increase the security. No it doesn't unless you are thinking about a security through obscurity argument Your logic is invalid - the best quality would be to not use a hash at all, and store in plain text, or ROT-13. Then you will have no collisions. If you truly believe more bits are better, don't use a hash to start with. Ooops, went offlist by a wrong click. Putting it back onliste I am aware that plain text (or any 1:1 mapping) has no chance of collision, but on the other hand if the box is compromised it gives an easy target for stealing passwords (and a lot of users use the same passwords a lot of places). I believe that hashing through one hash function is an acceptable compromise between collisions (i.e. people get in with the wrong password) and password safety (evil hacker cannot read passwords) given you deploy anti rainbow table meassures. I would still prefer two hash functions as they do add a better safeguard towards collisions (the gentoo distribtion actually hashes the files by three different algorithms SHA1, SHA256 and RMD160) - i would be inclined to use three hashes too, if they were instantly available. Svenne -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
On Thu, Apr 03, 2008 at 07:07:56PM +0200, Svenne Krap wrote: I currently save only md5(id || username || 'password')* into password, if I had access to sha1 (for example) i would add another password column so, having for example ID serial Username varchar Password_md5 varchar Password_sha1 varchar No matter how you see it, I get more bits of hash to check against. Are you a cryptanalyst and are you sure that this doesn't actually make things worse? I'm sure it gives you a warm fuzzy feeling that it's *got* to be better, but unless someone has done some hard maths I'm not sure how you can be so sure. Why not just use SHA-512, you get many more quality bits that way. I would drop md5 totally and use sha1 and ripemd-160 if possible.. but currently i use only md5 as it is the only available one.. Loading pgcrypto is overkill for something as simple as hash-functions. Sounds like a good reason for moving the current md5 function out into pgcrypto as well! :) * I prepend the id and the username to guard users with weak passwords against known hashvalues (rainbow tables) should the box ever get comprised ... I take it your threat model doesn't include the attacker logging incoming queries to look for the clear-text password. Sam -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Svenne Krap wrote: I would still prefer two hash functions as they do add a better safeguard towards collisions (the gentoo distribtion actually hashes the files by three different algorithms SHA1, SHA256 and RMD160) - i would be inclined to use three hashes too, if they were instantly available. Technically MD5 (128 bits) + SHA1 (160 bits) gives better strength than MD5 on its own, or SHA1 on its own, in that finding one collision is likely to be insufficient to break in, however, I doubt you could mathematically prove that it would perform equal to a 128 + 160 = 288 bit equivalent strength hash function. At the bare minimum, I refer you to the fact that each component on its own represents a self-contained hash of the entire document, and a small change in a real-life document will presumedly affect both values (this is how hash functions are designed), therefore, for real-life documents, there are bit patterns that will never coincide, therefore, not all 288 bits are being used. For real-life documents, it is highly likely that the MD5 and the SHA1 will always be a ~1:1 mapping (how many collision have truly been found in practice?), and the effective bit strength of the total for real documents approaches the highest of the two - which is 160 bits. I suggest that MD5 + SHA1 is not 128 bits better than SHA1 on its own. In any case, this is all irrelevant, because md5 passwords are still very useful, and the argument that more = better is a never ending infinite resource trap. More is not better. Better is better. If you can prove md5 is insufficient for PostgreSQL passwords, the correct decision would be to switch to something better, and deprecate md5 from the core. Cheers, mark -- Mark Mielke [EMAIL PROTECTED] -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Mark Mielke wrote: In any case, this is all irrelevant, because md5 passwords are still very useful, and the argument that more = better is a never ending infinite resource trap. More is not better. Better is better. If you can prove md5 is insufficient for PostgreSQL passwords, the correct decision would be to switch to something better, and deprecate md5 from the core. Agreed. One must also remember that if you use two hashes, if *either* one of them is broken in the future so that you can reconstruct the password from the hash, you're screwed. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Heikki Linnakangas wrote: Mark Mielke wrote: One must also remember that if you use two hashes, if *either* one of them is broken in the future so that you can reconstruct the password from the hash, you're screwed. That is quite a good argument actually :) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Sam Mason wrote: Are you a cryptanalyst and are you sure that this doesn't actually make things worse? I'm sure it gives you a warm fuzzy feeling that it's *got* to be better, but unless someone has done some hard maths I'm not sure how you can be so sure. No sadly I am no cryptoanalyst. Why not just use SHA-512, you get many more quality bits that way. I would, if it was available in core. I would drop md5 totally and use sha1 and ripemd-160 if possible.. but currently i use only md5 as it is the only available one.. Loading pgcrypto is overkill for something as simple as hash-functions. Sounds like a good reason for moving the current md5 function out into pgcrypto as well! :) I am not sure how I am to understand that comment. But again I am just a user... * I prepend the id and the username to guard users with weak passwords against known hashvalues (rainbow tables) should the box ever get comprised ... I take it your threat model doesn't include the attacker logging incoming queries to look for the clear-text password. No it doesn't, I am mostly concerned with the grab and run scenario. I am still convinced having more (and better) hash-functions in core is a gain for some users. And it is fairly un-intrusive as the hash functions are well-defined and never going to change (new ones can be added and old ones deleted, but SHA256 for example will never change). I think I will drop the issue as I cannot present formal proof of my case, sorry to have wasted your time. Svenne
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Sam Mason wrote: On Thu, Apr 03, 2008 at 07:07:56PM +0200, Svenne Krap wrote: ID serial Username varchar Password_md5 varchar Password_sha1 varchar ... Why not just use SHA-512, you get many more quality bits that way. Or if he just wanted to use builtin tools and reduce accidental collisions almost exactly the same much as he propopses, he could use password_md5_with_salt_xxx varchar password_md5_with_salt_yyy varchar but I also can't see the point. Won't he have more collisions from cosmic rays turning the results of his comparisons from false to true anyway? I would drop md5 totally and use sha1 and ripemd-160 if possible.. but currently i use only md5 as it is the only available one.. Loading pgcrypto is overkill for something as simple as hash-functions. Sounds like a good reason for moving the current md5 function out into pgcrypto as well! :) I'd rephrase it as saying a good reason for making it less intimidating to install modules. +1 to all the approaches that make this less scary. For Perl Digest-SHA1's in CPAN http://search.cpan.org/dist/Digest-SHA1/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
On Fri, Apr 04, 2008 at 12:06:03AM +0200, Svenne Krap wrote: Sam Mason wrote: Are you a cryptanalyst and are you sure that this doesn't actually make things worse? I'm sure it gives you a warm fuzzy feeling that it's *got* to be better, but unless someone has done some hard maths I'm not sure how you can be so sure. No sadly I am no cryptoanalyst. Neither am I. I'm not sure if my comment came across right though.. Why not just use SHA-512, you get many more quality bits that way. I would, if it was available in core. I'm still not sure why being in core is so special. I would drop md5 totally and use sha1 and ripemd-160 if possible.. but currently i use only md5 as it is the only available one.. Loading pgcrypto is overkill for something as simple as hash-functions. Sounds like a good reason for moving the current md5 function out into pgcrypto as well! :) I am not sure how I am to understand that comment. But again I am just a user... Within (most? larger anyway) software projects there's a big tension between adding features that are directly useful to users, versus providing tools that can be used to do more powerful things. Modularity and abstractions are about the only tools we have of taming complexity, without them codebases become impossibly convoluted and difficult to understand or modify. The module system in PG is part of this, it allows various parts of the code to be separated off (reasonably cleanly) allowing the rest of PG to be modified without too much concern of unrelated breakage. Lets see how much flack I get for that! :) I take it your threat model doesn't include the attacker logging incoming queries to look for the clear-text password. No it doesn't, I am mostly concerned with the grab and run scenario. OK, this sort of thing is very application specific. I have a feeling that my comments before came across a bit harshly, sorry if they did. I am still convinced having more (and better) hash-functions in core is a gain for some users. Yes, having more functionality in the code is always better for users! But the down side is that the rest of the software changes more slowly. I.e. a lot of the correctness/features/performance improvements that have been happening recently probably wouldn't have happened if all the code that's in the various modules was in core. And it is fairly un-intrusive as the hash functions are well-defined and never going to change (new ones can be added and old ones deleted, but SHA256 for example will never change). That's the sort of engineering problem I'm very bad at dealing with, I'm more interested in the computer science parts. I think I will drop the issue as I cannot present formal proof of my case, sorry to have wasted your time. Sorry, that wasn't the intention of my mail---I just enjoy the formal bits. I think they can be pretty beautiful, but most of the time they're irrelevant. Sam -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
On Thu, Apr 03, 2008 at 04:42:47PM -0700, Ron Mayer wrote: Sam Mason wrote: On Thu, Apr 03, 2008 at 07:07:56PM +0200, Svenne Krap wrote: ID serial Username varchar Password_md5 varchar Password_sha1 varchar ... Why not just use SHA-512, you get many more quality bits that way. Or if he just wanted to use builtin tools and reduce accidental collisions almost exactly the same much as he propopses, he could use password_md5_with_salt_xxx varchar password_md5_with_salt_yyy varchar but I also can't see the point. Won't he have more collisions from cosmic rays turning the results of his comparisons from false to true anyway? There's a difference between random bit flips, which in large systems happen surprisingly regularly, and a determined attacker. You'd be able to get somewhere against the former by duplicating everything, and you'd be able to get somewhere against the latter by using stronger hashes. Or have I missed the point entirely. Sounds like a good reason for moving the current md5 function out into pgcrypto as well! :) I'd rephrase it as saying a good reason for making it less intimidating to install modules. +1 to all the approaches that make this less scary. That's a much nicer way of saying it! I think that from an ISPs perspective (I can't remember the real use case that was given recently) the main problem with modules is that you have to build trust in each one individually. I.e. Doesn't the code running in modules run as the postgres user? If so, what would stop a malicious module from doing anything that the postgres user can? I'd probably phrase this as saying the modules are within PG's trust boundary, rather than outside. It's therefore the responsibility of the person installing the module to verify (I think this will generally be blind human trust) that the code isn't malicious. I'd guess that's a reason why so few modules are installed, it basically opens your code up to another source, with whom the installer has no prior trust relationship. There are operating systems that allow authority to be attenuated arbitrarily finely (i.e. not the very coarse user/role based ACL systems we have at the moment) which would allow mutually suspicious code to function. I.e. PG would be able to assume that the module was malicious and that the module wanted to crash PG, the module would assume the reverse, and the ISP wouldn't have to worry at all. It'll be a few years before we get there though. Sam -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 There isn't enough agreement to move some things from pgcrypto to the core so this thread is being removed from the patch queue. I don't agree that we should just close discussion. Nobody seems happy with the status quo, which is that we provide md5 but not sha1, and are thus encouraging people to use md5 everywhere. At the very least, I think we need to add sha1. Adding sha* would be better, and adding other hashes would be better still (and make PG a better product, in my opinion: having things builtin vs. contrib is a huge distinction). I'd also like to emphasize that this is not a pgcrypto issue: while it provides the same functionality that this proposal does, so does creating a Pl/Perl function, which is the route I usually go, as it is much easier and portable. So I see this as adding missing features to core. We will obviously never replace pgcrypto entirely, due to the silly state of encryption legislation. - -- Greg Sabino Mullane [EMAIL PROTECTED] End Point Corporation PGP Key: 0x14964AC8 200804020906 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iEYEAREDAAYFAkfzhMQACgkQvJuQZxSWSsiTCwCgvauRh/hqOK0BEMEPoKXhzCf5 +GkAn0ma+cZy3fas1Y/uQ0Zf8WAqfLo8 =tTlK -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Greg Sabino Mullane [EMAIL PROTECTED] writes: I don't agree that we should just close discussion. Nobody seems happy with the status quo, which is that we provide md5 but not sha1, There may be a few people who are unhappy, but the above claim seems vastly overblown. md5 is sufficient for the purpose it is intended for in core postgres (namely, obscuring the true source text of passwords), and if you have needs much beyond that you'll soon be installing pgcrypto anyway. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
On Wed, Apr 02, 2008 at 01:07:01PM -, Greg Sabino Mullane wrote: There isn't enough agreement to move some things from pgcrypto to the core so this thread is being removed from the patch queue. I don't agree that we should just close discussion. Nobody seems happy with the status quo, which is that we provide md5 but not sha1, and are thus encouraging people to use md5 everywhere. At the very least, I think we need to add sha1. Adding sha* would be better, and adding other hashes would be better still (and make PG a better product, in my opinion: having things builtin vs. contrib is a huge distinction). I'd also like to emphasize that this is not a pgcrypto issue: while it provides the same functionality that this proposal does, so does creating a Pl/Perl function, which is the route I usually go, as it is much easier and portable. So I see this as adding missing features to core. We will obviously never replace pgcrypto entirely, due to the silly state of encryption legislation. Just exactly which encryption legislation are we talking about here? I know there was some fuss about this issue back in the early 1990s, but that was many, many law changes and court cases ago, world-wide. It's far from clear to me that there's any reason other than inertia not to roll the crypto stuff into the core functionality and have done. Cheers, David. -- David Fetter [EMAIL PROTECTED] http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: [EMAIL PROTECTED] Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
David Fetter [EMAIL PROTECTED] writes: Just exactly which encryption legislation are we talking about here? I know there was some fuss about this issue back in the early 1990s, but that was many, many law changes and court cases ago, world-wide. It's far from clear to me that there's any reason other than inertia not to roll the crypto stuff into the core functionality and have done. This seems a very USA-centric view of the problem. It's true that the US export regulations no longer pose much of an issue for us (but who's to say they might not become tighter again in future?); the problem is there are lots of places where the laws are still strict. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
On Wed, Apr 02, 2008 at 12:27:15PM -0400, Tom Lane wrote: David Fetter [EMAIL PROTECTED] writes: Just exactly which encryption legislation are we talking about here? I know there was some fuss about this issue back in the early 1990s, but that was many, many law changes and court cases ago, world-wide. It's far from clear to me that there's any reason other than inertia not to roll the crypto stuff into the core functionality and have done. This seems a very USA-centric view of the problem. It's true that the US export regulations no longer pose much of an issue for us (but who's to say they might not become tighter again in future?); the problem is there are lots of places where the laws are still strict. Which places, and what laws? Re: compulsively attempting to comply with every law everywhere, let's recall that a certain Search! engine company managed to get itself in a morally pretty reprehensible spot by following the laws of an oppressive regime. We needn't do the same. Cheers, David. -- David Fetter [EMAIL PROTECTED] http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: [EMAIL PROTECTED] Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Hi Tom, md5 is not being recommended anywhere because it contains hash collision. Therefore either it should be replaced with SHA1 or any other good hash algorithm or taken out of core completely. md5 in core is worthless now.I am not using it in my application. I am using SHA1 in client/web tier for password hashing. Would replacing md5 with SHA1 in core involve much work? Sanjay Sharma To: [EMAIL PROTECTED] CC: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3 Date: Wed, 2 Apr 2008 11:38:31 -0400 From: [EMAIL PROTECTED] Greg Sabino Mullane [EMAIL PROTECTED] writes: I don't agree that we should just close discussion. Nobody seems happy with the status quo, which is that we provide md5 but not sha1, There may be a few people who are unhappy, but the above claim seems vastly overblown. md5 is sufficient for the purpose it is intended for in core postgres (namely, obscuring the true source text of passwords), and if you have needs much beyond that you'll soon be installing pgcrypto anyway. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers _ Tried the new MSN Messenger? It’s cool! Download now. http://messenger.msn.com/Download/Default.aspx?mkt=en-in
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
David Fetter wrote: On Wed, Apr 02, 2008 at 12:27:15PM -0400, Tom Lane wrote: David Fetter [EMAIL PROTECTED] writes: Just exactly which encryption legislation are we talking about here? I know there was some fuss about this issue back in the early 1990s, but that was many, many law changes and court cases ago, world-wide. It's far from clear to me that there's any reason other than inertia not to roll the crypto stuff into the core functionality and have done. This seems a very USA-centric view of the problem. It's true that the US export regulations no longer pose much of an issue for us (but who's to say they might not become tighter again in future?); the problem is there are lots of places where the laws are still strict. Which places, and what laws? http://rechten.uvt.nl/koops/cryptolaw/cls-sum.htm (Info only - I have not spent time considering the issue at hand thus, counter to net tradition, offer no opinion.) Cheers, Steve -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
David Fetter [EMAIL PROTECTED] writes: Re: compulsively attempting to comply with every law everywhere, let's recall that a certain Search! engine company managed to get itself in a morally pretty reprehensible spot by following the laws of an oppressive regime. We needn't do the same. We aren't actually doing any such thing. We are just making sure that our distribution is split up in such a way that if someone *else* needs to comply with such a law, it's not impractical to do so. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
On Wed, Apr 02, 2008 at 12:49:38PM -0400, Tom Lane wrote: David Fetter [EMAIL PROTECTED] writes: Re: compulsively attempting to comply with every law everywhere, let's recall that a certain Search! engine company managed to get itself in a morally pretty reprehensible spot by following the laws of an oppressive regime. We needn't do the same. We aren't actually doing any such thing. We are just making sure that our distribution is split up in such a way that if someone *else* needs to comply with such a law, it's not impractical to do so. That cat is already out of the bag with every Linux distribution. There's just no point in our going three sides around the square in order to have someone, somewhere claim that our tarball isn't violating their law. Cheers, David. -- David Fetter [EMAIL PROTECTED] http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: [EMAIL PROTECTED] Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
David Fetter [EMAIL PROTECTED] writes: That cat is already out of the bag with every Linux distribution. There's just no point in our going three sides around the square in order to have someone, somewhere claim that our tarball isn't violating their law. [shrug...] It's not exactly terribly complicated to keep this stuff in a contrib module. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
sanjay sharma [EMAIL PROTECTED] writes: md5 is not being recommended anywhere because it contains hash collision. For the purposes we are using it for, that's just about 100% irrelevant. Would replacing md5 with SHA1 in core involve much work? Yes, it would be a tremendous problem, because the use of md5 is part of our password protocol. We'd have to change client-side code in sync with the servers to do that. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Am Mittwoch, 2. April 2008 schrieb David Fetter: That cat is already out of the bag with every Linux distribution. There are other operating systems besides Linux. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
sanjay sharma wrote: Hi Tom, md5 is not being recommended anywhere because it contains hash collision. Therefore either it should be replaced with SHA1 or any other good hash algorithm or taken out of core completely. md5 in core is worthless now.I am not using it in my application. I am using SHA1 in client/web tier for password hashing. Would replacing md5 with SHA1 in core involve much work? sanjay - please do not top-answer, especially when others have put their answers below. MD5 is not broken for purposes that would require a pre-image attack, AIUI. That means there is a whole series of uses for which it is still quite OK, including password hashing. That said, there might well be a reason for including a collision-resistant hash function in core without including the whole of pg_crypto. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
On Wed, Apr 02, 2008 at 07:16:53PM +0200, Peter Eisentraut wrote: Am Mittwoch, 2. April 2008 schrieb David Fetter: That cat is already out of the bag with every Linux distribution. There are other operating systems besides Linux. And the cat's out of the bag there, too. Have you tried to get a system that doesn't include strong crypto running any time recently? Cheers, David. -- David Fetter [EMAIL PROTECTED] http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: [EMAIL PROTECTED] Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Am Mittwoch, 2. April 2008 schrieb sanjay sharma: md5 is not being recommended anywhere because it contains hash collision. Therefore either it should be replaced with SHA1 or any other good hash algorithm or taken out of core completely. md5 in core is worthless now.I am not using it in my application. I am using SHA1 in client/web tier for password hashing. Would replacing md5 with SHA1 in core involve much work? The vulnerabilities that exist for MD5 and SHA1 involve finding two random input strings that create the same hash values. This is possible for MD5 *and* SHA1 now, so asking for SHA1 to replace MD5 is completely pointless. What is not possible with either MD5 or SHA1 is finding an input string that creates the same hash value as a given input string (except by googling, but that affects all algorithms). So using MD5 for encrypting passwords or digesting known data values or tarballs can be considered secure at the moment. If you are dealing with certificate infrastructures, where the hash collision vulnerability described above might be relevant, you are certainly going to use some library such as openssl, and those have already moved away from using MD5 and SHA1 anyway. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
David Fetter wrote: And the cat's out of the bag there, too. Have you tried to get a system that doesn't include strong crypto running any time recently? Well yes, I have recently been involved in a large deployment of such systems. The details are a bit too involved for this thread, but the notion expressed elsewhere in this thread to make it easy to separate or rip out the different parts are exactly what facilitated that project. Now I am not saying that we must keep pgcrypto away from the core code forever for the benefit of these specific and well-funded external requirements. But these requirements do exist, and I haven't seen a good reason _for_ moving pgcrypto in this thread. Note that the original post actually wanted an sha1 function for MySQL compatibility. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Tom Lane wrote: Greg Sabino Mullane [EMAIL PROTECTED] writes: I don't agree that we should just close discussion. Nobody seems happy with the status quo, which is that we provide md5 but not sha1, There may be a few people who are unhappy, but the above claim seems vastly overblown. md5 is sufficient for the purpose it is intended for in core postgres (namely, obscuring the true source text of passwords), and if you have needs much beyond that you'll soon be installing pgcrypto anyway. I think that claim is completely incorrect. A lot of people use the md5() function in PostgreSQL today to hash the passwords for the users of whatever webbapp they are running. It only uses one account to connect to PostgreSQL and handles the rest of the auth elsewhere in the app. These users would like to have sha1 (and/or other securer hashes). And they would like it in -core, because their hosting company don't install the contrib modules. //Magnus -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Was that really the conclusion? My memory of this thread showed that most people who actually deal with hashes and cryptography *wanted* a SHA based hash in core (because our users ask for it!) and the only disagreement was in *what* should be included. //Magnus Bruce Momjian wrote: There isn't enough agreement to move some things from pgcrypto to the core so this thread is being removed from the patch queue. --- Bruce Momjian wrote: I am not thrilled about moving _some_ of pgcrypto into the backend --- pgcrypto right now seems well designed and if we pull part of it out it seems it will be less clear than what we have now. Perhaps we just need to document that md5() isn't for general use and some function in pgcrypto should be used instead? --- Marko Kreen wrote: On 1/21/08, Tom Lane [EMAIL PROTECTED] wrote: MD5 is broken in the sense that you can create two or more meaningful documents with the same hash. Note that this isn't actually very interesting for the purpose for which the md5() function was put into core: namely, hashing passwords before they are stored in pg_authid. Note: this was bad idea. The function that should have been added to core would be pg_password_hash(username, password). Adding md5() lessens incentive to install pgcrypto or push/accept digest() into core and gives impression there will be sha1(), etc in the future. Now users who want to store passwords in database (the most popular usage) will probably go with md5() without bothering with pgcrypto. They probably see Postgres itself uses MD5 too, without realizing their situation is totally different from pg_authid one. It's like we have solution that is ACID-compliant 99% of the time in core, so why bother with 100% one. -- marko ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Magnus Hagander wrote: I think that claim is completely incorrect. A lot of people use the md5() function in PostgreSQL today to hash the passwords for the users of whatever webbapp they are running. It only uses one account to connect to PostgreSQL and handles the rest of the auth elsewhere in the app. These users would like to have sha1 (and/or other securer hashes). And they would like it in -core, because their hosting company don't install the contrib modules. Hi Magnus: I don't think this is a compelling argument, and I mostly agree with Tom. PHP, Perl and Java are just three languages at the tip of my tongue that have built in support for MD5 and SHA1, and in all cases I can think of in a few seconds (I might be missing something?), it's far more desirable to do the MD5 / SHA1 in the language. If the document being encoded is large, doing it in the client is more efficient from a network transport perspective, as well as allowing ensuring that performance cost is on the web side, not the database side. If the text to be encoded requires security, then transmitting the password in clear text to the server only to be MD5 / SHA1 summed is not a great solution, as it involves transmission of the password. In both cases, I would do it client side, inside the web app. So, I believe your argument that web apps need it is faulty. I think a legitimate use would involve around using such a function in pl/pgsql. I can't think of a case where I've ever needed to do that. Cheers, mark -- Mark Mielke [EMAIL PROTECTED] -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Magnus Hagander [EMAIL PROTECTED] writes: ... These users would like to have sha1 (and/or other securer hashes). And they would like it in -core, because their hosting company don't install the contrib modules. That line of argument could be used to justify putting anything and everything in core. I think that our extensible architecture is an important feature and one we should not hesitate to use to the fullest. There is clearly work that could/should be done to improve it, such as having better system support for packages or modules or whatever you want to call them; and maybe we also need some marketing-type work to encourage people such as hosting companies to allow access to non-core packages. But allowing arguments such as the above to drive our design is not the way to go. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Tom Lane wrote: There is clearly work that could/should be done to improve it, such as having better system support for packages or modules or whatever you want to call them; and maybe we also need some marketing-type work to encourage people such as hosting companies to allow access to non-core packages. Funny you should mention that. I was just thinking the other day that I needed to re-raise the question of getting rid of contrib, which has long been a horrible misnomer, and calling its contents The PostgreSQL Standard Modules. Maybe we could rename the directory modules. IIRC that was the consensus name last time this topic was discussed. We've already vastly improved their usefulness by adding documentation. Let's take the next jump and start advertising them as exemplars of our extensibility. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
On Wed, Apr 02, 2008 at 05:09:14PM -0400, Andrew Dunstan wrote: Standard Modules. Maybe we could rename the directory modules. IIRC This seems like an easy and practical answer. A -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Magnus Hagander wrote: Was that really the conclusion? My memory of this thread showed that most people who actually deal with hashes and cryptography *wanted* a SHA based hash in core (because our users ask for it!) and the only disagreement was in *what* should be included. Sorry if it looked like I was jumping to conclusions on this. I was reporting the summary of the comments I had from the patch queue. Based on the subsequent discussion, it seems it was accurate none-the-less. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
There isn't enough agreement to move some things from pgcrypto to the core so this thread is being removed from the patch queue. --- Bruce Momjian wrote: I am not thrilled about moving _some_ of pgcrypto into the backend --- pgcrypto right now seems well designed and if we pull part of it out it seems it will be less clear than what we have now. Perhaps we just need to document that md5() isn't for general use and some function in pgcrypto should be used instead? --- Marko Kreen wrote: On 1/21/08, Tom Lane [EMAIL PROTECTED] wrote: MD5 is broken in the sense that you can create two or more meaningful documents with the same hash. Note that this isn't actually very interesting for the purpose for which the md5() function was put into core: namely, hashing passwords before they are stored in pg_authid. Note: this was bad idea. The function that should have been added to core would be pg_password_hash(username, password). Adding md5() lessens incentive to install pgcrypto or push/accept digest() into core and gives impression there will be sha1(), etc in the future. Now users who want to store passwords in database (the most popular usage) will probably go with md5() without bothering with pgcrypto. They probably see Postgres itself uses MD5 too, without realizing their situation is totally different from pg_authid one. It's like we have solution that is ACID-compliant 99% of the time in core, so why bother with 100% one. -- marko ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
On 1/28/08, Greg Sabino Mullane [EMAIL PROTECTED] wrote: I am not thrilled about moving _some_ of pgcrypto into the backend --- pgcrypto right now seems well designed and if we pull part of it out it seems it will be less clear than what we have now. Perhaps we just need to document that md5() isn't for general use and some function in pgcrypto should be used instead? I don't think docs will help much. The md5() has broken the policy of all crypto in external module so I'm now thinking we should just bite the bullet and add digest() to core. Exctracting the part from pgcrypto is no-brainer, the hashes have very thin wrapper around them, only thing common with rest of pgcrypto is error handling, which hashes use the least. Only thing that needs minor thought is that it would be nice to access hashes from external module - that means exporting the find_digest() function to it. Which should be no problem. I think looking at this as putting some of pg_crypto into core is looking at this the wrong way. We are never going to put the whole thing into core given the current state of cryptography laws, as obviously the current status of giving users md5() and nothing else is not ideal. What we're looking for is a middle ground. It seems to me we've narrowed it down to two questions: 1) Does sha1(), or other hashing algorithms risk running afoul of cryptography regulations? I'm 100% sure that sha1() itself is not a problem (it's even a PHP builtin, and good luck finding a box these days wihout that monstrosity installed). I highly doubt any of the rest (SHA*, HMAC, etc.) are a problem either: we're doing a one-way hash, not encrypting data. But common sense and cryptography have seldom been seen together since the start of the cold war, so I'll hold my final judgement. 2) Which ones do we include? Putting sha1() seems a no-brainer, but as Joe points out, why not add all the rest in at the same time? Considering we have DES and MD5 already, then how can SHA2 be a problem when SHA1 isnt? -- marko ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
* Bruce Momjian: I am not thrilled about moving _some_ of pgcrypto into the backend --- pgcrypto right now seems well designed and if we pull part of it out it seems it will be less clear than what we have now. Perhaps we just need to document that md5() isn't for general use and some function in pgcrypto should be used instead? Yes, that would probably help those folks doing checklist-based security audits. -- Florian Weimer[EMAIL PROTECTED] BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
I am not thrilled about moving _some_ of pgcrypto into the backend --- pgcrypto right now seems well designed and if we pull part of it out it seems it will be less clear than what we have now. Perhaps we just need to document that md5() isn't for general use and some function in pgcrypto should be used instead? --- Marko Kreen wrote: On 1/21/08, Tom Lane [EMAIL PROTECTED] wrote: MD5 is broken in the sense that you can create two or more meaningful documents with the same hash. Note that this isn't actually very interesting for the purpose for which the md5() function was put into core: namely, hashing passwords before they are stored in pg_authid. Note: this was bad idea. The function that should have been added to core would be pg_password_hash(username, password). Adding md5() lessens incentive to install pgcrypto or push/accept digest() into core and gives impression there will be sha1(), etc in the future. Now users who want to store passwords in database (the most popular usage) will probably go with md5() without bothering with pgcrypto. They probably see Postgres itself uses MD5 too, without realizing their situation is totally different from pg_authid one. It's like we have solution that is ACID-compliant 99% of the time in core, so why bother with 100% one. -- marko ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 I am not thrilled about moving _some_ of pgcrypto into the backend --- pgcrypto right now seems well designed and if we pull part of it out it seems it will be less clear than what we have now. Perhaps we just need to document that md5() isn't for general use and some function in pgcrypto should be used instead? I think looking at this as putting some of pg_crypto into core is looking at this the wrong way. We are never going to put the whole thing into core given the current state of cryptography laws, as obviously the current status of giving users md5() and nothing else is not ideal. What we're looking for is a middle ground. It seems to me we've narrowed it down to two questions: 1) Does sha1(), or other hashing algorithms risk running afoul of cryptography regulations? I'm 100% sure that sha1() itself is not a problem (it's even a PHP builtin, and good luck finding a box these days wihout that monstrosity installed). I highly doubt any of the rest (SHA*, HMAC, etc.) are a problem either: we're doing a one-way hash, not encrypting data. But common sense and cryptography have seldom been seen together since the start of the cold war, so I'll hold my final judgement. 2) Which ones do we include? Putting sha1() seems a no-brainer, but as Joe points out, why not add all the rest in at the same time? - -- Greg Sabino Mullane [EMAIL PROTECTED] PGP Key: 0x14964AC8 200801281506 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iD8DBQFHnjeJvJuQZxSWSsgRA2fWAKCljvbj5BVaFQ5mEDvckNGhVz6rDgCg0DRc zaIu/rT1vdDrL61JTsXdIZ8= =7DKm -END PGP SIGNATURE- ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
On 1/20/08, Greg Sabino Mullane [EMAIL PROTECTED] wrote: In summary: what would objections be to my writing a sha1() patch? Well. If you do start adding hashes to core then _please_ pick a path that allows having all the standard hashes in advance. That means both md5 and sha-1, sha2 (4 hashes) and there is also sha-3 in the horizon. Basically there seems to be 2 variants: 1) Continue the md5() style: md5(), sha1(), sha224(), sha256(), sha384(), sha512(), plus another 4 for SHA-3. 2) Move hashing functions from pgcrypto to core. That means digest() and I would suggest hmac() and crypt() too. I'm also starting to think it may be worth having hexdigest(). I prefer 2). There is some common infrastructure in pgcrypto, the hash specific parts can be either split out or rewritten from scratch, hashes need very small amount of code. I agree that having all of pgcrypto in core is bit overkill, so please don't think of it as all-or-nothing affair. -- marko ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Florian Weimer [EMAIL PROTECTED] writes: * David Fetter: Is there any country with laws so benighted that they restrict secure hashing algorithms? Right now, there's a contest between SHA1 and MD5 as to which one gets broken first, and SHA1 appears to be in the lead. SHAn for n1 could preempt the awfulness of losing this race. MD5 is broken in the sense that you can create two or more meaningful documents with the same hash. Note that this isn't actually very interesting for the purpose for which the md5() function was put into core: namely, hashing passwords before they are stored in pg_authid. pg_authid is already secured against viewing by non-superusers, and a superuser can crack into your database account anyway, nyet? So the only reason we bother with hashing here is to keep a superuser from finding out your cleartext password, which might possibly let him crack into non-database services that you foolishly used the same password for. Therefore, it doesn't really matter if he can find another password with the same hash --- that's not guarding against anything interesting. Of course, if you want to store other sorts of protected stuff in the database, you might not want md5, but at that point you're a candidate to use contrib/pgcrypto. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
* Tom Lane: MD5 is broken in the sense that you can create two or more meaningful documents with the same hash. Note that this isn't actually very interesting for the purpose for which the md5() function was put into core: namely, hashing passwords before they are stored in pg_authid. No doubt about that. But there are checklists out there, and if you use MD5 at some point, you need to go to some lengths to explain that it's okay. That's why I can understand the desire to have sha1 easily available (even though SHA-1 isn't much better, really, and the difference doesn't actually matter for many application). It's a bit like justifying that you don't need a virus scanner on your non-Windows server or database server. 8-P BTW, I'd like to see MD5/SHA-1 for BYTEA, not just TEXT, and with a BYTEA return value. Does pgcrypto provide that? -- Florian Weimer[EMAIL PROTECTED] BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
On 1/21/08, Florian Weimer [EMAIL PROTECTED] wrote: BTW, I'd like to see MD5/SHA-1 for BYTEA, not just TEXT, and with a BYTEA return value. Does pgcrypto provide that? Yes. -- marko ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
On 1/21/08, Tom Lane [EMAIL PROTECTED] wrote: MD5 is broken in the sense that you can create two or more meaningful documents with the same hash. Note that this isn't actually very interesting for the purpose for which the md5() function was put into core: namely, hashing passwords before they are stored in pg_authid. Note: this was bad idea. The function that should have been added to core would be pg_password_hash(username, password). Adding md5() lessens incentive to install pgcrypto or push/accept digest() into core and gives impression there will be sha1(), etc in the future. Now users who want to store passwords in database (the most popular usage) will probably go with md5() without bothering with pgcrypto. They probably see Postgres itself uses MD5 too, without realizing their situation is totally different from pg_authid one. It's like we have solution that is ACID-compliant 99% of the time in core, so why bother with 100% one. -- marko ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 In MySQL I have a function SHA1, which is critical for storing and authenticating passwords. It sure would be nice to have this in core. Yeah, there's pgcrypto, but it's a bit overkill for people who simply want to do a SHA1, especially when they see we already have a md5(). I also realize that SHA1 is not a great solution these days either, but I'd at least like to see a discussion on moving Postgres to somewhere between only has md5() and all pg_crypto functions inside core, even if it only means a handful of SHA functions. Moving this over to -hackers. In summary: what would objections be to my writing a sha1() patch? - -- Greg Sabino Mullane [EMAIL PROTECTED] PGP Key: 0x14964AC8 200801201218 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iD8DBQFHk4NIvJuQZxSWSsgRA83ZAJ0SIk36sYvLF30q7hog2sBaQU1+LACeOv15 WkDwrzgzHyrwmNFP85plbBA= =6P4y -END PGP SIGNATURE- ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Greg Sabino Mullane wrote: I also realize that SHA1 is not a great solution these days either, but I'd at least like to see a discussion on moving Postgres to somewhere between only has md5() and all pg_crypto functions inside core, even if it only means a handful of SHA functions. Moving this over to -hackers. In summary: what would objections be to my writing a sha1() patch? Isn't sha1 considered broken for some uses anyway? Perhaps if you're going to do that it would make sense to move the whole pgcrypto/sha2.c stuff to core, I think. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
On Sun, Jan 20, 2008 at 05:24:11PM -, Greg Sabino Mullane wrote: It sure would be nice to have this in core. Yeah, there's pgcrypto, but it's a bit overkill for people who simply want to do a SHA1, especially when they see we already have a md5(). md5() was added with the following commit message: Attached are two small patches to expose md5 as a user function -- including documentation and regression test mods. It seemed small and unobtrusive enough to not require a specific proposal on the hackers list -- but if not, let me know and I'll make a pitch. Otherwise, if there are no objections please apply. http://developer.postgresql.org/cvsweb.cgi/pgsql/src/backend/utils/adt/varlena.c#rev1.94 In summary: what would objections be to my writing a sha1() patch? There wasn't any discussion about it last time. It does seem a bit wierd to support one but not the other. It's also interesting to note that the implementation in the backed is commented with: I do not expect this file to be used for general purpose MD5'ing of large amounts of data, only for generating hashed passwords from limited input. Have a nice day, -- Martijn van Oosterhout [EMAIL PROTECTED] http://svana.org/kleptog/ Those who make peaceful revolution impossible will make violent revolution inevitable. -- John F Kennedy signature.asc Description: Digital signature
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Alvaro Herrera wrote: Greg Sabino Mullane wrote: I also realize that SHA1 is not a great solution these days either, but I'd at least like to see a discussion on moving Postgres to somewhere between only has md5() and all pg_crypto functions inside core, even if it only means a handful of SHA functions. Moving this over to -hackers. In summary: what would objections be to my writing a sha1() patch? Isn't sha1 considered broken for some uses anyway? Perhaps if you're going to do that it would make sense to move the whole pgcrypto/sha2.c stuff to core, I think. IIRC not anymore than md5, which we already do... That said, it would make sense to include sha1() for compatibility reasons and a stronger sha for people that need something better. //Magnus ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Martijn van Oosterhout wrote: In summary: what would objections be to my writing a sha1() patch? There wasn't any discussion about it last time. It does seem a bit wierd to support one but not the other. It's also interesting to note that the implementation in the backed is commented with: I proposed md5 without sha1 because we already had md5 code in the backend, and we did not have sha1 (and still don't). At the time I was afraid that if I proposed sha1 as well it would become a debate and we would have ended up with neither. Personally I'm in favor of having sha1 and one or more of the newer replacements in the backend. I'd also like to see HMAC built in. But I think we need to be careful about running afoul of various export regulations. Keeping the crypto stuff separate allows distributions to leave the crypto out if they need to. Perhaps cryptographic hashes/HMAC are not an issue though. Anyone know? Joe ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
Greg Sabino Mullane [EMAIL PROTECTED] writes: In summary: what would objections be to my writing a sha1() patch? Mainly that no one else is dissatisfied with the current split between core and pgcrypto. The only reason md5() is in core is to support encryption of passwords in pg_shadow. There are good reasons not to have any more crypto capability in core than we absolutely have to; mainly to do with benighted laws in some countries. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
On Sun, Jan 20, 2008 at 01:42:21PM -0500, Tom Lane wrote: Greg Sabino Mullane [EMAIL PROTECTED] writes: In summary: what would objections be to my writing a sha1() patch? Mainly that no one else is dissatisfied with the current split between core and pgcrypto. The only reason md5() is in core is to support encryption of passwords in pg_shadow. There are good reasons not to have any more crypto capability in core than we absolutely have to; mainly to do with benighted laws in some countries. Is there any country with laws so benighted that they restrict secure hashing algorithms? Right now, there's a contest between SHA1 and MD5 as to which one gets broken first, and SHA1 appears to be in the lead. SHAn for n1 could preempt the awfulness of losing this race. Cheers, David. -- David Fetter [EMAIL PROTECTED] http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: [EMAIL PROTECTED] Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] [GENERAL] SHA1 on postgres 8.3
* David Fetter: Is there any country with laws so benighted that they restrict secure hashing algorithms? Right now, there's a contest between SHA1 and MD5 as to which one gets broken first, and SHA1 appears to be in the lead. SHAn for n1 could preempt the awfulness of losing this race. MD5 is broken in the sense that you can create two or more meaningful documents with the same hash. This is not currently possible for SHA-1 (at least no one has publicly demonstrated this capability). SHA-256 etc. are sufficiently similar to MD5 and SHA-1, so it's not clear if they add significant additional security. (Sorry if this is what you've said.) -- Florian Weimer[EMAIL PROTECTED] BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99 ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
