[HACKERS] Selectivity estimation for equality and range queries
I have been observing a case where the row count estimation for LIKE 'foo' is (much) higher than for LIKE 'foo%', the rest of the query being the same. This is a special case of the estimation for equality being higher than for a range query that includes the value used in the equality. I haven't been able to get a copy of the data from the client yet, but considering the nature of the data and the description of the selectivity estimation algorithms (http://www.postgresql.org/docs/8.3/static/row-estimation-examples.html), this behavior appears to be mathematically plausible. I have been wondering whether in general the eqsel should try to compare its result with the estimation of (x = 'foo' AND x = 'foo') and use that as a ceiling or something. Has anyone else observed something similar? -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Spoofing as the postmaster
On Thu, 27 Dec 2007, Stephen Frost wrote: Debian also has SELinux, if one wishes to configure it. I suspect other Debian-derived distributions also have it as a result. It can certainly be a pain to configure but it's far from impossible That's a good summary. As of Debian Etch (April of this year) the base distribution now include enough SELinux compatible userland packages for the fundamental utilities (ssh, svsvinit, pam, cron, some others) that you don't have to run around hacking a set of patches anymore just to get the base system working. There is also a Hardened Gentoo with SELinux. The most notable distribution where SELinux support is seriously dead is SuSE. RHEL/Fedora are the only distributions where SELinux is taken seriously enough that most packages/daemons are patched and have policies setup in a useful state out of the box. But with some work you can customize a reasonable setup on some other distributions. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Spoofing as the postmaster
The problem with forcing authentication is that an auth-unaware client connecting to a legitimate postmaster would have its connections refused. That same client would have its connections accepted by an impostor postmaster. Thus, there is no way to stop impostor postmasters from carrying out these attacks on auth-unaware clients. The proper solution, as I see it, would be to have an authentication system in the postmaster that was not enforced. If the client requests authentication, postmaster will provide it, if not, then postmaster will connect normally without it. This would not result in *any* change in the default behavior of postmaster, and as far as users who don't want to use it are concerned, they don't even need to bother to turn it off (assuming that having it turned on does not consume extra resources and I don't think having an unused authentication mechanism sitting in the postmaster connection establishment routine would). This does not appear to result in greater security, however it does. It allows DBAs who suspect that they are likely going to be the target of these attacks to deploy authentication procedures in their client packages. This could be a modification to their applications, or whatever steps are necessary to mandate authenticated connections within their organization. There is no point forcing some auth mechanism within postmaster, as attackers would simply catch users using software that did not require the server to auth before sending passwords. For this reason it is not postmaster's responsibility to check that unknown clients do not connect to impostors, it is postmaster's responsibility however to authenticate itself, if the client asks for it. So the onus (rightfully in my opinion) falls upon network administrators / DBAs to ensure that all of their users are using auth-enabled client packages which will not allow connections to be established with a postmaster until authentication has passed, and disallow the use of other client software to connect to postmaster. In my view, this puts the security responsibility where it rightfully belongs *and* maintains a non-breaking of client packages in the wild. Making a server or anything that *requires* auth and disallows non-authed clients is pointless, as there is nothing stopping attackers from setting up an auth-disabled impostor and waiting for someone to just connect using psql or some other vanilla connection method. The onus really ought to be with the administrators who give their users the software they use to connect to ensure that the software they use adheres to the relevant security policy, in the same way that its their responsibility to ensure that the client software does not contain keyloggers and other such trashware. In the web world, it is the client's responsibility to ensure that they check the SSL cert and don't do their banking at www.bankofamerica.hax0r.ru and there is nothing that the real banking site can do to stop them using their malware infested PC to connect to the phishing site. They can only provide a site that provides authentication. This is analogous to postmaster: It is only the responsibility of postmaster to provide the option of authentication, it is the client's responsibility to know if they should use it, and if so, to ensure they do so properly. Regards, - MrNaz.com ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Spoofing as the postmaster
On Sat, Dec 29, 2007 at 02:09:23AM +1100, Naz Gassiep wrote: In the web world, it is the client's responsibility to ensure that they check the SSL cert and don't do their banking at www.bankofamerica.hax0r.ru and there is nothing that the real banking site can do to stop them using their malware infested PC to connect to the phishing site. The above security model is exactly how we got into the mess we're in: relying entirely on the good sense of a wide community of users is how compromises happen. Strong authentication authenticates both ways. For instance, the web world you describe is not the only one. Banks who take security seriously have multiple levels of authentication, have trained their users how to do this, and regularly provide scan tools to clients in an attempt (IMO possibly doomed) to reduce the chances of input-device sniffing. A ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Spoofing as the postmaster
On 12/28/07, Andrew Sullivan [EMAIL PROTECTED] wrote: On Sat, Dec 29, 2007 at 02:09:23AM +1100, Naz Gassiep wrote: In the web world, it is the client's responsibility to ensure that they check the SSL cert and don't do their banking at www.bankofamerica.hax0r.ru and there is nothing that the real banking site can do to stop them using their malware infested PC to connect to the phishing site. The above security model is exactly how we got into the mess we're in: relying entirely on the good sense of a wide community of users is how compromises happen. Strong authentication authenticates both ways. For instance, the web world you describe is not the only one. Banks who take security seriously have multiple levels of authentication, have trained their users how to do this, and regularly provide scan tools to clients in an attempt (IMO possibly doomed) to reduce the chances of input-device sniffing. I don't follow. What are banks doing on the web now to force clients to authenticate them, and how is it any different from the model of training users to check the SSL certificate? There's a fundamental problem that you can't make someone else do authentication if they don't want to, and that's exactly the situation clients are in. I don't see how this can possibly be fixed anywhere other than the client. ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Spoofing as the postmaster
Trevor Talbot [EMAIL PROTECTED] writes: There's a fundamental problem that you can't make someone else do authentication if they don't want to, and that's exactly the situation clients are in. I don't see how this can possibly be fixed anywhere other than the client. The point of requiring authentication from the server side is that it will get people to configure their client code properly. Then if a MITM attack is subsequently attempted, the client code will detect it. It's true that this doesn't offer much defense in the case where a new user is getting set up and a MITM attack is already active. But a user who blindly trusts a server that he's never connected to before is open to all sorts of attacks, starting for instance with mistyping the host name. The fact that this approach doesn't (by itself) solve that problem doesn't make it useless. Also, getting people in the habit of setting up for mutual authentication does have value in that scenario too; it makes the new user perhaps a bit more likely to distrust a server that isn't presenting the right certificate. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Selectivity estimation for equality and range queries
Peter Eisentraut [EMAIL PROTECTED] writes: I have been observing a case where the row count estimation for LIKE 'foo' is (much) higher than for LIKE 'foo%', the rest of the query being the same. This is a special case of the estimation for equality being higher than for a range query that includes the value used in the equality. Not really --- LIKE estimation is only weakly related to range estimation. Relevant questions here include exactly which PG version is in use and what's the database encoding/locale. If it's not C locale, the fixes I made in selfuncs.c during November might be relevant. regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Spoofing as the postmaster
On 12/28/07, Tom Lane [EMAIL PROTECTED] wrote: Trevor Talbot [EMAIL PROTECTED] writes: There's a fundamental problem that you can't make someone else do authentication if they don't want to, and that's exactly the situation clients are in. I don't see how this can possibly be fixed anywhere other than the client. The point of requiring authentication from the server side is that it will get people to configure their client code properly. Then if a MITM attack is subsequently attempted, the client code will detect it. But this is essentially just an education/training issue; the security model itself is unchanged. Bank web sites are only going to accept clients via SSL, but if a client does not try to authenticate the site, whether it connects via SSL or not is rather irrelevant. I have no problem with the idea of encouraging clients to authenticate the server, but this configuration doesn't help with defaults. It's just available as a tool for site administrators to use. Also, getting people in the habit of setting up for mutual authentication does have value in that scenario too; it makes the new user perhaps a bit more likely to distrust a server that isn't presenting the right certificate. I see Naz's argument as addressing this goal. The problem with forcing authentication is that it's an all-or-nothing proposition: either the server and all the clients do it, or none of them do. That's fine when you control all the pieces and are willing to put in the work to configure them all, but not effective for encouraging default behavior. Instead, give the server credentials by default, but let clients choose whether to request them. That makes deployment easier in that all you have to do is configure clients as needed to get authentication of the server. Easier deployment means it's more likely to be used. IOW, put up both http and https out of the box. You might even want to have newer clients default to caching credentials on the first connect. That still doesn't change the security model, but should be more effective at getting clients to do something useful by default. ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Spoofing as the postmaster
On Fri, Dec 28, 2007 at 07:48:22AM -0800, Trevor Talbot wrote: I don't follow. What are banks doing on the web now to force clients to authenticate them, and how is it any different from the model of training users to check the SSL certificate? Some banks (mostly Swiss and German, from what I've seen) are requiring two-token authentication, and that second token is really the way that the client authenticates the server: when you install your banking application, you're really installing the keys you need to authenticate the server and for the server to authenticate you. There's a fundamental problem that you can't make someone else do authentication if they don't want to, and that's exactly the situation clients are in. Right, but you can train users to expect authentication of the server. One way to do that is to require them to use an intrusive enough system that they end up learning what to look for in a phish attack. That said, I tend to agree with you: if we had dnssec everywhere today, it's totally unclear to me what client applications would do in the event they got a bogus resolution. A ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Spoofing as the postmaster
Andrew Sullivan wrote: On Fri, Dec 28, 2007 at 07:48:22AM -0800, Trevor Talbot wrote: I don't follow. What are banks doing on the web now to force clients to authenticate them, and how is it any different from the model of training users to check the SSL certificate? Some banks (mostly Swiss and German, from what I've seen) are requiring two-token authentication, and that second token is really the way that the client authenticates the server: when you install your banking application, you're really installing the keys you need to authenticate the server and for the server to authenticate you. I have done this for my own application before. Although the client and server use standard TLS 1.0 to speak to each other with a required authentication of RSA 1024-bit and a required encryption of AES 128-bit, it still requires that passwords sent from the client to the server are RSA encrypted using the server public certificate, making it impossible for anybody except for the legitimate server to see the password. One benefit of this is that the password itself can be '\0'd out as soon as we have RSA encrypted it, and things like a core dump of the client have a lower chance of including the password in plain text. In my case, the reason I did it is because I was trying to navigate around the US export control regulations that prevent greater than 1024 bit assymetric or 128 bit symmetric from leaving the US. I was able to use the standard Java SSL and crypto libraries to achieve greater than 128 bit symmetric encryption by combining the two. Now, my implementation isn't perfect with regard to Andrew's comments, as I encrypt using the server's public certificate after authenticating it. Technically, however, I could actually have two server certificates - one to use for authentication, and one to use for encryption. I believe this is becoming common in some circles, and you will find that gpg uses DSA keys for authentication, and signs the RSA keys used for encryption with the DSA key. The DSA key can be more bits, or have a longer life time. At what point does prudence become paranoia? I don't know. In my case, I felt 128-bit encryption was insufficient for protecting the passwords in my application. 256-bit encryption would have been sufficient, but that cannot yet be safely exported from the US to the countries I required. Cheers, mark -- Mark Mielke [EMAIL PROTECTED]
Re: [HACKERS] Spoofing as the postmaster
Andrew Sullivan wrote: On Fri, Dec 28, 2007 at 07:48:22AM -0800, Trevor Talbot wrote: I don't follow. What are banks doing on the web now to force clients to authenticate them, and how is it any different from the model of training users to check the SSL certificate? Some banks (mostly Swiss and German, from what I've seen) are requiring two-token authentication, and that second token is really the way that the client authenticates the server: when you install your banking application, you're really installing the keys you need to authenticate the server and for the server to authenticate you. Most actually secure banks would be using standalone tokens, and not something that runs on your local machine and can easily be compromised. There needs to be air between the token and the computer. The exact difference in security is always debatable, but air gap tokens is what's been used for most banks here for many years - in many cases since they first started doing internet banking 10+ years ago. But. That's for authenticating the *client*. Authenticating the server in the end requires you to trust the security of the client machine, and requiring special applications for that just makes it worse :-( And in the end, the only thing they really do is implement the browser the way it should've been implemented in the first place. The bottom line is still that the security against that has to happen on the client side. We could make it so that we *require* the root certificate to be present on the client and make the check, and simply refuse to connect without it. But my guess is that it'll just increase the bar for SSL adoption at all, whilst most people will find some insecure way to get the root key over there anyway. Unless we want to start shipping our own batch of trusted roots, and only support paid-for certificates or something... There's a fundamental problem that you can't make someone else do authentication if they don't want to, and that's exactly the situation clients are in. Right, but you can train users to expect authentication of the server. One way to do that is to require them to use an intrusive enough system that they end up learning what to look for in a phish attack. That said, I tend to agree with you: if we had dnssec everywhere today, it's totally unclear to me what client applications would do in the event they got a bogus resolution. Well, we all know how well the big warning boxes in the web browsers work... You can't really trust the user to make such a decision, in the end. You can get to a point, but not all the way by far. But what you can do is that as an administrator, you can require these checks. If you only allow connections from machines that are trusted, and you make sure those are configured to require verification of the server cert, then you're safe. //Magnus ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Spoofing as the postmaster
Mark Mielke wrote: Andrew Sullivan wrote: On Fri, Dec 28, 2007 at 07:48:22AM -0800, Trevor Talbot wrote: I don't follow. What are banks doing on the web now to force clients to authenticate them, and how is it any different from the model of training users to check the SSL certificate? Some banks (mostly Swiss and German, from what I've seen) are requiring two-token authentication, and that second token is really the way that the client authenticates the server: when you install your banking application, you're really installing the keys you need to authenticate the server and for the server to authenticate you. I have done this for my own application before. Although the client and server use standard TLS 1.0 to speak to each other with a required authentication of RSA 1024-bit and a required encryption of AES 128-bit, it still requires that passwords sent from the client to the server are RSA encrypted using the server public certificate, making it impossible for anybody except for the legitimate server to see the password. One benefit of this is that the password itself can be '\0'd out as soon as we have RSA encrypted it, and things like a core dump of the client have a lower chance of including the password in plain text. Why are you even using a password in this case, and not just key-based auth? Wouldn't that be even easier and more secure? At what point does prudence become paranoia? I don't know. In my case, I felt 128-bit encryption was insufficient for protecting the passwords in my application. 256-bit encryption would have been sufficient, but that cannot yet be safely exported from the US to the countries I required. How do you protect the certificate store on the client? Or the binary that ends up prompting for the password on the client? //Magnus ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Spoofing as the postmaster
Magnus Hagander wrote: Mark Mielke wrote: I have done this for my own application before. Although the client and server use standard TLS 1.0 to speak to each other with a required authentication of RSA 1024-bit and a required encryption of AES 128-bit, it still requires that passwords sent from the client to the server are RSA encrypted using the server public certificate, making it impossible for anybody except for the legitimate server to see the password. One benefit of this is that the password itself can be '\0'd out as soon as we have RSA encrypted it, and things like a core dump of the client have a lower chance of including the password in plain text. Why are you even using a password in this case, and not just key-based auth? Wouldn't that be even easier and more secure? Users of this product don't have keys - they have passwords. The username/password is for per-user authentication. The username defines the access level. Many users will use the same client. The client does have its own private RSA key and public certificate, however, this grants entry to the system. Password login is still required by the users of the client. At what point does prudence become paranoia? I don't know. In my case, I felt 128-bit encryption was insufficient for protecting the passwords in my application. 256-bit encryption would have been sufficient, but that cannot yet be safely exported from the US to the countries I required. How do you protect the certificate store on the client? Or the binary that ends up prompting for the password on the client The certificate on the client grants access to the system. It does not grant access to the resources on the system. Two-level authentication with mandatory server authentication. You see similar things in physical security instances. A security badge lets you in the door - but you still need to login to the computer once you get in. As for protecting the binary that prompts for a password on the client - I didn't bother with this, although Java does allow for signed jar files that would allow the user to be assured that the client is legitimate. There are always loops though, just because the client is legitimate doesn't mean that the keyboard is, and so on. You end up putting in enough effort to mitigate the risk. The risk always exists, but through clever, cryptographic, or obfuscatory measures, the risk can be greatly reduced. Cheers, mark -- Mark Mielke [EMAIL PROTECTED]
Re: [HACKERS] minimal update
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Tom Lane wrote: Well, you could write the trigger in C and it'd work for any table. I think it could be as simple as a memcmp of the tuples' data areas, since we now require padding bytes to be 0 ... Something like this fragment? newtuple = trigdata-tg_newtuple; oldtuple = trigdata-tg_trigtuple; rettuple = newtuple; if (newtuple-t_len == oldtuple-t_len newtuple-t_data-t_hoff == oldtuple-t_data-t_hoff memcmp(GETSTRUCT(newtuple),GETSTRUCT(oldtuple), newtuple-t_len - newtuple-t_data-t_hoff) == 0) rettuple = NULL; return PointerGetDatum(rettuple); Close, but I think you also need to take care to compare natts and the null bitmaps (if any). Might be worth comparing OIDs too, though AFAIR there is no mechanism for substituting a different OID during UPDATE. Probably the easiest coding is to memcmp all the way from offsetof(t_bits) to t_len, after comparing natts and the HASNULL and HASOID flags. How does this look? if (newtuple-t_len == oldtuple-t_len newtuple-t_data-t_hoff == oldtuple-t_data-t_hoff HeapTupleHeaderGetNatts(newtuple) == HeapTupleHeaderGetNatts(oldtuple) (newtuple-t_data-t_infomask (HEAP_HASOID|HEAP_HASNULL)) == (oldtuple-t_data-t_infomask (HEAP_HASOID|HEAP_HASNULL)) memcmp(newtuple-t_data + offsetof(HeapTupleHeaderData, t_bits), oldtuple-t_data + offsetof(HeapTupleHeaderData, t_bits) newtuple-t_len - offsetof(HeapTupleHeaderData, t_bits)) == 0) rettuple = NULL; return PointerGetDatum(rettuple); cheers andrew Also, when did we first require padding bytes to be 0? The 8.3 varvarlena patch is what requires it, but in practice heap_formtuple has always started with a palloc0, so I think it would work a long ways back. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] minimal update
Andrew Dunstan [EMAIL PROTECTED] writes: How does this look? if (newtuple-t_len == oldtuple-t_len newtuple-t_data-t_hoff == oldtuple-t_data-t_hoff HeapTupleHeaderGetNatts(newtuple) == HeapTupleHeaderGetNatts(oldtuple) (newtuple-t_data-t_infomask (HEAP_HASOID|HEAP_HASNULL)) == (oldtuple-t_data-t_infomask (HEAP_HASOID|HEAP_HASNULL)) memcmp(newtuple-t_data + offsetof(HeapTupleHeaderData, t_bits), oldtuple-t_data + offsetof(HeapTupleHeaderData, t_bits) newtuple-t_len - offsetof(HeapTupleHeaderData, t_bits)) == 0) rettuple = NULL; Looks sane. It might be even saner if you compare all of the non-visibility-related infomask bits, viz (newtuple-t_data-t_infomask ~HEAP_XACT_MASK) == (oldtuple-t_data-t_infomask ~HEAP_XACT_MASK) rather than just HASOID and HASNULL. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Archiver behavior at shutdown
On Thu, 2007-12-27 at 18:54 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: On Thu, 2007-12-27 at 17:29 -0500, Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: then a subsequent postmaster start could initiate a second archiver process which would cause issues with whatever the first archiver is doing. That's a problem that the archiver itself should fix (perhaps it needs its own lockfile). http://archives.postgresql.org/pgsql-hackers/2006-05/msg00920.php I thought that sounded familiar ;-). As you say, I'm beginning to know where the bodies are buried... What was the outcome of that discussion? No patch for this ever got applied AFAICS. The patch as posted had a few issues, per the thread, and I don't see a followup version. (The alleged replacement patch did something else entirely.) We applied a one line change in preference to the lockfile approach for 8.2, requested by you, agreed to by me and applied by Bruce. This would be the behaviour I would have, if I had a blank canvas: - keep archiver alive at shutdown, rather than bouncing it - send SIGUSR2 to do finish-up and close, just like bgwriter - put a lockfile in for the archiver that prevents a new archiver from starting, but everything else comes up OK. In postmaster if PgArchPID == 0 then we check for archiver.pid, if present, read it and send a SIGUSR2 to it. If rc = ESRCH then process no present, so start up new archiver - lets keep archiving, if there is work to do, right up until the last possible moment, even if the postmaster has gone - ensure people understand that an archive_command call can be interrupted and may need to handle the consequences if the command is not atomic With those changes the use cases would look like this... System Shutdown System shuts down, postmaster shuts down, archiver works furiously until the end trying to archive things away. Archiver gets caught half way through copy, so crashes, leaving archiver.pid. Subsequent startup sees archiver.pid, postmaster reads file to get pid, then sends signal to archiver to see if it is still alive, it isn't so remove archiver.pid and allow next archiver to start. First call to archive_command handles partially copied file in archive. Server Crash Something takes down server, archiver stays up trying to archive things away. Crash recovery kicks in and finishes very quickly, new archiver tries to start up but cannot because first archiver is still working. At the end of its cycle, first archiver goes away and allows new archiver to start and continue operating. Server Restart Server shuts down, but there is work to do so first archiver stays around to finish it. Newly started server tries to start archiver but cannot because of pid file. Reads pid file, sends signal. Archiver is already shutting down, so continues its cycle and then quites. New archiver starts up under new postmaster. ...but that's too much change for me to personally stomach at this stage of 8.3. My main issue is that I don't have the time to be able to do a retest of start/stop/restart/crash behaviour and catching all the side cases is fairly hard, and yet also critical at this stage of play. For me, the behaviour is close enough now, with the main issue being the additional wait at the end of pgarch_MainLoop(). It's been there since 8.2, so a simple fix there would be non-invasive and backpatchable also. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Archiver behavior at shutdown
On Sat, 29 Dec 2007, Simon Riggs wrote: System Shutdown System shuts down, postmaster shuts down, archiver works furiously until the end trying to archive things away. Archiver gets caught half way through copy, so crashes, leaving archiver.pid. Subsequent startup sees archiver.pid, postmaster reads file to get pid, then sends signal to archiver to see if it is still alive, it isn't so remove archiver.pid and allow next archiver to start. Isn't it possible some other process may have started with that pid if the database server was down for long enough? In that case sending a signal presuming it's the archive process that used to have that pid might be bad form. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] minimal update
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: How does this look? if (newtuple-t_len == oldtuple-t_len newtuple-t_data-t_hoff == oldtuple-t_data-t_hoff HeapTupleHeaderGetNatts(newtuple) == HeapTupleHeaderGetNatts(oldtuple) (newtuple-t_data-t_infomask (HEAP_HASOID|HEAP_HASNULL)) == (oldtuple-t_data-t_infomask (HEAP_HASOID|HEAP_HASNULL)) memcmp(newtuple-t_data + offsetof(HeapTupleHeaderData, t_bits), oldtuple-t_data + offsetof(HeapTupleHeaderData, t_bits) newtuple-t_len - offsetof(HeapTupleHeaderData, t_bits)) == 0) rettuple = NULL; Looks sane. It might be even saner if you compare all of the non-visibility-related infomask bits, viz (newtuple-t_data-t_infomask ~HEAP_XACT_MASK) == (oldtuple-t_data-t_infomask ~HEAP_XACT_MASK) rather than just HASOID and HASNULL. Sadly, the memcmp is failing on my test (update foo set bar = bar) on 8.2. Looks like I'm in for weekend with my fave debugger :-( cheers andrew ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Spoofing as the postmaster
Magnus Hagander wrote: We could make it so that we *require* the root certificate to be present on the client and make the check, and simply refuse to connect without it. But my guess is that it'll just increase the bar for SSL adoption at all, whilst most people will find some insecure way to get the root key over there anyway. Unless we want to start shipping our own batch of trusted roots, and only support paid-for certificates or something... Agreed. Requiring client root certificate checking is heavy-handed. At most we could emit a server log message when a client has no certificate. Of course I am not sure anyone knows how to get that information from SSL. We could do it in the clients we ship but a malicious client will just remove the check. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Spoofing as the postmaster
Bruce Momjian [EMAIL PROTECTED] writes: Agreed. Requiring client root certificate checking is heavy-handed. There seems to be some confusion here. I didn't think anyone was proposing that we force every installation to require client root certificate checking. What was under discussion (I thought) was providing the ability for a DBA to *choose* to require it. Of course I am not sure anyone knows how to get that information from SSL. Yeah, if OpenSSL doesn't support testing for this then the discussion is moot... regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Spoofing as the postmaster
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Agreed. Requiring client root certificate checking is heavy-handed. There seems to be some confusion here. I didn't think anyone was proposing that we force every installation to require client root certificate checking. What was under discussion (I thought) was providing the ability for a DBA to *choose* to require it. Oh, yea, that would be OK. I am a little worried that the extra configuration required to turn this on/off might be added complexity for little gain. It might be simpler to allow the administrator to control whether non-checking clients are logged, rather than refusing the connection. I think this makes it clearer the root client check is to make sure all your clients are doing it right, rather than an actual security enhancement (if that makes sense). Of course I am not sure anyone knows how to get that information from SSL. Yeah, if OpenSSL doesn't support testing for this then the discussion is moot... Yea. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Spoofing as the postmaster
Tomasz Ostrowski wrote: On Sun, 23 Dec 2007, Tom Lane wrote: ISTM we have these action items: 1. Improve the code so that SSL authentication can be used across a Unix-socket connection (we can disable encryption though). I've just realised that there's a problem with SSL with disabled encryption on a unix socket / localhost connections for cpu-saving. Any local user using this attack would be able to eavesdrop everything comming through a socket. If an attacker just acts as a tunnel, highjacking a unix-socket and talking to a server using any other interface (or the other way around), then he would not be able to modify information flow, but he would be able to read and save everything going to and from a server. It is again not obvious as normally local connections are not susceptible to eavesdropping. And could go unnoticed for a long time as everything would just work normally. So I think no cpu-saving by turning off encryption should be done. And this would all not help for a denial-of-service attack. Good point. I have added the last two sentences to the documentation paragraph to highlight this issue: productnameOpenSSL/productname supports a wide range of ciphers and authentication algorithms, of varying strength. While a list of ciphers can be specified in the productnameOpenSSL/productname configuration file, you can specify ciphers specifically for use by the database server by modifying xref linkend=guc-ssl-ciphers in filenamepostgresql.conf/. It is possible to have authentication without the overhead of encryption by using literalNULL-SHA/ or literalNULL-MD5/ ciphers. However, a man-in-the-middle could read and pass communications between client and server. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Spoofing as the postmaster
Bruce Momjian wrote: Good point. I have added the last two sentences to the documentation paragraph to highlight this issue: productnameOpenSSL/productname supports a wide range of ciphers and authentication algorithms, of varying strength. While a list of ciphers can be specified in the productnameOpenSSL/productname configuration file, you can specify ciphers specifically for use by the database server by modifying xref linkend=guc-ssl-ciphers in filenamepostgresql.conf/. It is possible to have authentication without the overhead of encryption by using literalNULL-SHA/ or literalNULL-MD5/ ciphers. However, a man-in-the-middle could read and pass communications between client and server. A fact that the above misses, is that symmetric key encryption is actually quite cheap. It is asymmetric key encryption that is expensive. If you look up information on SSL accelerators, you will find claims that the initial SSL authentication negotiation is 1000X as expensive as the actual data encryption for a running session, and that SSL web services are usually limited by their ability to negotiate NEW sessions. In other words, as well intentioned and accurate as the claim you make above, it may be irrelevant in many real world scenarios. If you are going to go through all the expensive processing of having authentication enabled, you may as well have encryption enabled too. Cheers, mark -- Mark Mielke [EMAIL PROTECTED] ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Spoofing as the postmaster
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Agreed. Requiring client root certificate checking is heavy-handed. There seems to be some confusion here. I didn't think anyone was proposing that we force every installation to require client root certificate checking. What was under discussion (I thought) was providing the ability for a DBA to *choose* to require it. Of course I am not sure anyone knows how to get that information from SSL. Yeah, if OpenSSL doesn't support testing for this then the discussion is moot.. I believe SSL is only capable of letting you know whether authentication for each end point was 1) not requested, 2) optional requested, or 3) required. Note that even if the authentication is required, there is no way to know how authentication was performed. For example, did it check the signature chain, requiring it to map to a public root certificate lists used by most web browsers? If so, did it check the contents of the certificate, or is only checking that it exists? Did it check a local key store that has a copy of the public key certificate? Or did it just log the certificate subject? OpenSSH, for instance, presents the user with the finger print of the certificate and asks you: $ ssh 192.168.0.1 The authenticity of host '192.168.0.1 (192.168.0.1)' can't be established. RSA key fingerprint is 3e:a7:0f:04:60:7e:8e:64:52:bf:81:92:a9:05:c7:36. Are you sure you want to continue connecting (yes/no)? While this certainly gives you the opportunity to challenge it, I don't know of any person who actually checks this finger print. Luckily, it stores it to ~/.ssh/known_hosts, and so the real issue is if it suddenly changes, you get a warning. Still, I've seen the warning before, and realized that oh yes, that machine was upgraded, so it probably has a new public key. I have never personally checked the finger print against a known source. Authentication is only as strong as the person or process confirming it. In the case of trying to force a client to authenticate the server, this requires the client to know who the server is. As most clients will not know who the server is, I see clients implementing an OpenSSH-style authentication model (shown above), or providing their own no-op authentication routine to OpenSSL. I don't think it is worth it, and I don't think it would work. Cheers, mark -- Mark Mielke [EMAIL PROTECTED]
Re: [HACKERS] Spoofing as the postmaster
Mark Mielke wrote: Bruce Momjian wrote: Good point. I have added the last two sentences to the documentation paragraph to highlight this issue: productnameOpenSSL/productname supports a wide range of ciphers and authentication algorithms, of varying strength. While a list of ciphers can be specified in the productnameOpenSSL/productname configuration file, you can specify ciphers specifically for use by the database server by modifying xref linkend=guc-ssl-ciphers in filenamepostgresql.conf/. It is possible to have authentication without the overhead of encryption by using literalNULL-SHA/ or literalNULL-MD5/ ciphers. However, a man-in-the-middle could read and pass communications between client and server. A fact that the above misses, is that symmetric key encryption is actually quite cheap. It is asymmetric key encryption that is expensive. If you look up information on SSL accelerators, you will find claims that the initial SSL authentication negotiation is 1000X as expensive as the actual data encryption for a running session, and that SSL web services are usually limited by their ability to negotiate NEW sessions. In other words, as well intentioned and accurate as the claim you make above, it may be irrelevant in many real world scenarios. If you are going to go through all the expensive processing of having authentication enabled, you may as well have encryption enabled too. OK, updated paragraph: It is possible to have authentication without encryption overhead by using literalNULL-SHA/ or literalNULL-MD5/ ciphers. However, a man-in-the-middle could read and pass communications between client and server. Also, encryption overhead is minimal compared to the overhead of authentication. For these reasons NULL ciphers are not recommended. -- Bruce Momjian [EMAIL PROTECTED]http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Spoofing as the postmaster
Bruce Momjian wrote: OK, updated paragraph: It is possible to have authentication without encryption overhead by using literalNULL-SHA/ or literalNULL-MD5/ ciphers. However, a man-in-the-middle could read and pass communications between client and server. Also, encryption overhead is minimal compared to the overhead of authentication. For these reasons NULL ciphers are not recommended. Looks good! Cheers, mark -- Mark Mielke [EMAIL PROTECTED] ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] minimal update
Andrew Dunstan wrote: Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: How does this look? if (newtuple-t_len == oldtuple-t_len newtuple-t_data-t_hoff == oldtuple-t_data-t_hoff HeapTupleHeaderGetNatts(newtuple) == HeapTupleHeaderGetNatts(oldtuple) (newtuple-t_data-t_infomask (HEAP_HASOID|HEAP_HASNULL)) == (oldtuple-t_data-t_infomask (HEAP_HASOID|HEAP_HASNULL)) memcmp(newtuple-t_data + offsetof(HeapTupleHeaderData, t_bits), oldtuple-t_data + offsetof(HeapTupleHeaderData, t_bits) newtuple-t_len - offsetof(HeapTupleHeaderData, t_bits)) == 0) rettuple = NULL; Looks sane. It might be even saner if you compare all of the non-visibility-related infomask bits, viz (newtuple-t_data-t_infomask ~HEAP_XACT_MASK) == (oldtuple-t_data-t_infomask ~HEAP_XACT_MASK) rather than just HASOID and HASNULL. Sadly, the memcmp is failing on my test (update foo set bar = bar) on 8.2. Looks like I'm in for weekend with my fave debugger :-( Turns out we needed those pointers used in the arguments to memcmp cast to char * so the pointer arithmetic would work right. I'll be suggesting we add a utility function like this for 8.4. cheers andrew ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
