Re: [HACKERS] libpq compression

2012-08-30 Thread Bruce Momjian
On Sun, Jun 17, 2012 at 11:45:54PM +0800, Magnus Hagander wrote: > On Sun, Jun 17, 2012 at 11:42 PM, Tom Lane wrote: > > Magnus Hagander writes: > >> Is there a reason why we don't have a parameter on the client > >> mirroring ssl_ciphers? > > > > Dunno, do we need one?  I am not sure what the ci

Re: [HACKERS] libpq compression

2012-07-13 Thread Magnus Hagander
On Mon, Jun 25, 2012 at 2:26 PM, Magnus Hagander wrote: > On Mon, Jun 25, 2012 at 4:04 AM, Robert Haas wrote: >> On Fri, Jun 22, 2012 at 12:38 PM, Euler Taveira wrote: >>> On 20-06-2012 17:40, Marko Kreen wrote: On Wed, Jun 20, 2012 at 10:05 PM, Florian Pflug wrote: > I'm starting to t

Re: [HACKERS] libpq compression

2012-06-26 Thread Tom Lane
Robert Haas writes: > On Mon, Jun 25, 2012 at 4:25 PM, k...@rice.edu wrote: >> Here is the benchmark list from the Google lz4 page: >> >> NameRatio C.speed D.speed >> LZ4 (r59) 2.084 330 915 >> LZO 2.05 1x_1 2.038 311 480 >> QuickLZ 1.5 -1 2.233 257 27

Re: [HACKERS] libpq compression

2012-06-26 Thread Euler Taveira
On 26-06-2012 12:23, Robert Haas wrote: > At the risk of making everyone laugh at me, has anyone tested pglz? I > observe that if the compression ration and performance are good, we > might consider using it for this purpose, too, which would avoid > having to add dependencies. Conversely, if the

Re: [HACKERS] libpq compression

2012-06-26 Thread Robert Haas
On Mon, Jun 25, 2012 at 4:25 PM, k...@rice.edu wrote: > On Mon, Jun 25, 2012 at 09:45:26PM +0200, Florian Pflug wrote: >> On Jun25, 2012, at 21:21 , Dimitri Fontaine wrote: >> > Magnus Hagander writes: >> >> Or that it takes less code/generates cleaner code... >> > >> > So we're talking about som

Re: [HACKERS] libpq compression

2012-06-25 Thread Merlin Moncure
On Mon, Jun 25, 2012 at 3:38 PM, Euler Taveira wrote: > On 25-06-2012 16:45, Florian Pflug wrote: >> Agreed. If we extend the protocol to support compression, and not rely >> on SSL, then let's pick one of these LZ77-style compressors, and let's >> integrate it into our tree. >> > If we have an op

Re: [HACKERS] libpq compression

2012-06-25 Thread Euler Taveira
On 25-06-2012 14:30, Greg Jaskiewicz wrote: > Wasn't this more of an issue in de-coupling compression from encryption ? > Let me give a try to take some conclusion. If we decide for an independent compression code instead of an SSL-based one, that is a possibility to be tested: SSL + SSL-based com

Re: [HACKERS] libpq compression

2012-06-25 Thread Euler Taveira
On 25-06-2012 16:45, Florian Pflug wrote: > Agreed. If we extend the protocol to support compression, and not rely > on SSL, then let's pick one of these LZ77-style compressors, and let's > integrate it into our tree. > If we have an option to have it out of our tree, good; if not, let's integrate

Re: [HACKERS] libpq compression

2012-06-25 Thread k...@rice.edu
On Mon, Jun 25, 2012 at 09:45:26PM +0200, Florian Pflug wrote: > On Jun25, 2012, at 21:21 , Dimitri Fontaine wrote: > > Magnus Hagander writes: > >> Or that it takes less code/generates cleaner code... > > > > So we're talking about some LZO things such as snappy from google, and > > that would b

Re: [HACKERS] libpq compression

2012-06-25 Thread Phil Sorber
On Mon, Jun 25, 2012 at 3:45 PM, Florian Pflug wrote: > On Jun25, 2012, at 21:21 , Dimitri Fontaine wrote: >> Magnus Hagander writes: >>> Or that it takes less code/generates cleaner code... >> >> So we're talking about some LZO things such as snappy from google, and >> that would be another run

Re: [HACKERS] libpq compression

2012-06-25 Thread Florian Pflug
On Jun25, 2012, at 21:21 , Dimitri Fontaine wrote: > Magnus Hagander writes: >> Or that it takes less code/generates cleaner code... > > So we're talking about some LZO things such as snappy from google, and > that would be another run time dependency IIUC. > > I think it's time to talk about fa

Re: [HACKERS] libpq compression

2012-06-25 Thread Dimitri Fontaine
Magnus Hagander writes: > Or that it takes less code/generates cleaner code... So we're talking about some LZO things such as snappy from google, and that would be another run time dependency IIUC. I think it's time to talk about fastlz: http://fastlz.org/ http://code.google.com/p/fastlz/so

Re: [HACKERS] libpq compression

2012-06-25 Thread Greg Jaskiewicz
Wasn't this more of an issue in de-coupling compression from encryption ? On 25 Jun 2012, at 16:36, Euler Taveira wrote: > On 24-06-2012 23:04, Robert Haas wrote: >> So I think we really >> need someone to try this both ways and compare. Right now it seems >> like we're mostly speculating on h

Re: [HACKERS] libpq compression

2012-06-25 Thread Euler Taveira
On 24-06-2012 23:04, Robert Haas wrote: > So I think we really > need someone to try this both ways and compare. Right now it seems > like we're mostly speculating on how well either approach would work, > which is not as good as having some experimental results. > Not a problem. That's what I'm

Re: [HACKERS] libpq compression

2012-06-25 Thread k...@rice.edu
On Mon, Jun 25, 2012 at 03:12:46PM +0200, Florian Pflug wrote: > On Jun25, 2012, at 04:04 , Robert Haas wrote: > > If, for > > example, someone can demonstrate that an awesomebsdlz compresses 10x > > as fast as OpenSSL... that'd be pretty compelling. > > That, actually, is demonstrably the case f

Re: [HACKERS] libpq compression

2012-06-25 Thread Florian Pflug
On Jun25, 2012, at 04:04 , Robert Haas wrote: > If, for > example, someone can demonstrate that an awesomebsdlz compresses 10x > as fast as OpenSSL... that'd be pretty compelling. That, actually, is demonstrably the case for at least Google's snappy. (and LZO, but that's not an option since its l

Re: [HACKERS] libpq compression

2012-06-25 Thread Magnus Hagander
On Mon, Jun 25, 2012 at 4:04 AM, Robert Haas wrote: > On Fri, Jun 22, 2012 at 12:38 PM, Euler Taveira wrote: >> On 20-06-2012 17:40, Marko Kreen wrote: >>> On Wed, Jun 20, 2012 at 10:05 PM, Florian Pflug wrote: I'm starting to think that relying on SSL/TLS for compression of unencrypte

Re: [HACKERS] libpq compression

2012-06-24 Thread Robert Haas
On Fri, Jun 22, 2012 at 12:38 PM, Euler Taveira wrote: > On 20-06-2012 17:40, Marko Kreen wrote: >> On Wed, Jun 20, 2012 at 10:05 PM, Florian Pflug wrote: >>> I'm starting to think that relying on SSL/TLS for compression of >>> unencrypted connections might not be such a good idea after all. We'd

Re: [HACKERS] libpq compression

2012-06-22 Thread Euler Taveira
On 20-06-2012 17:40, Marko Kreen wrote: > On Wed, Jun 20, 2012 at 10:05 PM, Florian Pflug wrote: >> I'm starting to think that relying on SSL/TLS for compression of >> unencrypted connections might not be such a good idea after all. We'd >> be using the protocol in a way it quite clearly never was

Re: [HACKERS] libpq compression

2012-06-20 Thread Florian Pflug
On Jun20, 2012, at 22:40 , Marko Kreen wrote: > On Wed, Jun 20, 2012 at 10:05 PM, Florian Pflug wrote: >> I'm starting to think that relying on SSL/TLS for compression of >> unencrypted connections might not be such a good idea after all. We'd >> be using the protocol in a way it quite clearly nev

Re: [HACKERS] libpq compression

2012-06-20 Thread Marko Kreen
On Wed, Jun 20, 2012 at 10:05 PM, Florian Pflug wrote: > I'm starting to think that relying on SSL/TLS for compression of > unencrypted connections might not be such a good idea after all. We'd > be using the protocol in a way it quite clearly never was intended to > be used... Maybe, but what is

Re: [HACKERS] libpq compression

2012-06-20 Thread Florian Pflug
On Jun20, 2012, at 18:42 , Magnus Hagander wrote: > That is a very good point. Before we design *another* feature that > relies on it, we should verify if the syntax is compatible in the > other libraries that would be interesting (gnutls and NSS primarily), > and if it's not that at least the *fun

Re: [HACKERS] libpq compression

2012-06-20 Thread Magnus Hagander
On Wed, Jun 20, 2012 at 12:35 PM, Florian Pflug wrote: > On Jun19, 2012, at 17:36 , Robert Haas wrote: >> On Mon, Jun 18, 2012 at 1:42 PM, Martijn van Oosterhout >> wrote: >>> On Sun, Jun 17, 2012 at 12:29:53PM -0400, Tom Lane wrote: The fly in the ointment with any of these ideas is that th

Re: [HACKERS] libpq compression

2012-06-20 Thread Alvaro Herrera
Excerpts from Tom Lane's message of mié jun 20 11:49:51 -0400 2012: > > Alvaro Herrera writes: > > I looked at the code (apps/ciphers.c) and it looks pretty easy to obtain > > the list of ciphers starting from the stringified configuration > > parameter and iterate on them. > > Do you mean that

Re: [HACKERS] libpq compression

2012-06-20 Thread Tom Lane
Alvaro Herrera writes: > I looked at the code (apps/ciphers.c) and it looks pretty easy to obtain > the list of ciphers starting from the stringified configuration > parameter and iterate on them. Do you mean that it will produce an expansion of the set of ciphers meeting criteria like "!aNULL"?

Re: [HACKERS] libpq compression

2012-06-20 Thread Florian Pflug
On Jun20, 2012, at 17:34 , Tom Lane wrote: > Florian Pflug writes: >> I wonder though if shouldn't restrict the allowed ciphers list to being >> a simple list of supported ciphers. If our goal is to support multiple >> SSL libraries transparently then surely having openssl-specific syntax >> in th

Re: [HACKERS] libpq compression

2012-06-20 Thread Tom Lane
Florian Pflug writes: > I wonder though if shouldn't restrict the allowed ciphers list to being > a simple list of supported ciphers. If our goal is to support multiple > SSL libraries transparently then surely having openssl-specific syntax > in the config file isn't exactly great anyway... No,

Re: [HACKERS] libpq compression

2012-06-20 Thread Alvaro Herrera
Excerpts from Florian Pflug's message of mié jun 20 06:35:29 -0400 2012: > On Jun19, 2012, at 17:36 , Robert Haas wrote: > > On Mon, Jun 18, 2012 at 1:42 PM, Martijn van Oosterhout > > wrote: > >> On Sun, Jun 17, 2012 at 12:29:53PM -0400, Tom Lane wrote: > >>> The fly in the ointment with any of

Re: [HACKERS] libpq compression

2012-06-20 Thread Florian Pflug
On Jun19, 2012, at 17:36 , Robert Haas wrote: > On Mon, Jun 18, 2012 at 1:42 PM, Martijn van Oosterhout > wrote: >> On Sun, Jun 17, 2012 at 12:29:53PM -0400, Tom Lane wrote: >>> The fly in the ointment with any of these ideas is that the "configure >>> list" is not a list of exact cipher names, as

Re: [HACKERS] libpq compression

2012-06-19 Thread Robert Haas
On Mon, Jun 18, 2012 at 1:42 PM, Martijn van Oosterhout wrote: > On Sun, Jun 17, 2012 at 12:29:53PM -0400, Tom Lane wrote: >> The fly in the ointment with any of these ideas is that the "configure >> list" is not a list of exact cipher names, as per Magnus' comment that >> the current default incl

Re: [HACKERS] libpq compression

2012-06-18 Thread Martijn van Oosterhout
On Sun, Jun 17, 2012 at 12:29:53PM -0400, Tom Lane wrote: > The fly in the ointment with any of these ideas is that the "configure > list" is not a list of exact cipher names, as per Magnus' comment that > the current default includes tests like "!aNULL". I am not sure that > we know how to evalua

Re: [HACKERS] libpq compression

2012-06-17 Thread Euler Taveira
On 17-06-2012 12:45, Magnus Hagander wrote: > On Sun, Jun 17, 2012 at 11:42 PM, Tom Lane wrote: >> Magnus Hagander writes: >>> Is there a reason why we don't have a parameter on the client >>> mirroring ssl_ciphers? >> >> Dunno, do we need one? I am not sure what the cipher negotiation process >

Re: [HACKERS] libpq compression

2012-06-17 Thread Euler Taveira
On 17-06-2012 12:45, Magnus Hagander wrote: > Uh. We have the ! notation in our default *now*. What openssl also > supports is the text "DEFAULT", which is currently the equivalent of > "ALL!aNULL!eNULL". The question, which is valid of course, should be > if "DEFAULT" works with all openssl versio

Re: [HACKERS] libpq compression

2012-06-17 Thread Euler Taveira
On 17-06-2012 12:42, Tom Lane wrote: > If our default isn't the same as the underlying default, I have to > question why not. But are you sure this "!" notation will work with > all openssl versions? > What is all for you? It seems we don't claim support for an specific version or later in docs o

Re: [HACKERS] libpq compression

2012-06-17 Thread Tom Lane
Florian Pflug writes: > Would we still tell openssl to only negotiate ciphers in the configured > list of available ciphers + NULL? If not, what happens if a connection > happens to use a cipher that is actually stronger than any cipher on > the "list of acceptable ciphers" list? The DBA wouldn't

Re: [HACKERS] libpq compression

2012-06-17 Thread Florian Pflug
On Jun16, 2012, at 17:15 , Tom Lane wrote: > Magnus Hagander writes: >> On Sat, Jun 16, 2012 at 12:55 PM, Tom Lane wrote: >>> It's not obvious to me that we actually *need* anything except the >>> ability to recognize that a null-encrypted SSL connection probably >>> shouldn't be treated as match

Re: [HACKERS] libpq compression

2012-06-17 Thread Dave Page
On Sun, Jun 17, 2012 at 4:45 PM, Magnus Hagander wrote: > On Sun, Jun 17, 2012 at 11:42 PM, Tom Lane wrote: >> Magnus Hagander writes: >>> Is there a reason why we don't have a parameter on the client >>> mirroring ssl_ciphers? >> >> Dunno, do we need one?  I am not sure what the cipher negotiat

Re: [HACKERS] libpq compression

2012-06-17 Thread Magnus Hagander
On Sun, Jun 17, 2012 at 11:42 PM, Tom Lane wrote: > Magnus Hagander writes: >> Is there a reason why we don't have a parameter on the client >> mirroring ssl_ciphers? > > Dunno, do we need one?  I am not sure what the cipher negotiation process > looks like or which side has the freedom to choose

Re: [HACKERS] libpq compression

2012-06-17 Thread Tom Lane
Magnus Hagander writes: > Is there a reason why we don't have a parameter on the client > mirroring ssl_ciphers? Dunno, do we need one? I am not sure what the cipher negotiation process looks like or which side has the freedom to choose. > That, or just have DEFAULT as being the default (which

Re: [HACKERS] libpq compression

2012-06-17 Thread Magnus Hagander
On Sat, Jun 16, 2012 at 11:15 PM, Tom Lane wrote: > Magnus Hagander writes: >> On Sat, Jun 16, 2012 at 12:55 PM, Tom Lane wrote: >>> It's not obvious to me that we actually *need* anything except the >>> ability to recognize that a null-encrypted SSL connection probably >>> shouldn't be treated

Re: [HACKERS] libpq compression

2012-06-16 Thread k...@rice.edu
On Sat, Jun 16, 2012 at 11:15:30AM -0400, Tom Lane wrote: > Magnus Hagander writes: > > On Sat, Jun 16, 2012 at 12:55 PM, Tom Lane wrote: > >> It's not obvious to me that we actually *need* anything except the > >> ability to recognize that a null-encrypted SSL connection probably > >> shouldn't

Re: [HACKERS] libpq compression

2012-06-16 Thread Tom Lane
Magnus Hagander writes: > On Sat, Jun 16, 2012 at 12:55 PM, Tom Lane wrote: >> It's not obvious to me that we actually *need* anything except the >> ability to recognize that a null-encrypted SSL connection probably >> shouldn't be treated as matching a hostssl line; which is not something >> tha

Re: [HACKERS] libpq compression

2012-06-15 Thread Magnus Hagander
On Sat, Jun 16, 2012 at 12:40 PM, Tom Lane wrote: > Marko Kreen writes: >> On Sat, Jun 16, 2012 at 6:39 AM, Magnus Hagander wrote: >>> Fair enough if we decide that - but we should make that decision >>> knowing that we're leaving the JDBC and .Net people in a bad position >>> where they are not

Re: [HACKERS] libpq compression

2012-06-15 Thread Magnus Hagander
On Sat, Jun 16, 2012 at 12:55 PM, Tom Lane wrote: > Magnus Hagander writes: >> Yes, but there's also a lot of such awkward logic we need to add if we >> *do* go with the SSL library doing the compression: > >> For example, we can no longer trust the SSL library to always do >> encryption, since w

Re: [HACKERS] libpq compression

2012-06-15 Thread Tom Lane
Magnus Hagander writes: > Yes, but there's also a lot of such awkward logic we need to add if we > *do* go with the SSL library doing the compression: > For example, we can no longer trust the SSL library to always do > encryption, since we specifically want to support null encryption. True, but

Re: [HACKERS] libpq compression

2012-06-15 Thread Tom Lane
Marko Kreen writes: > On Sat, Jun 16, 2012 at 6:39 AM, Magnus Hagander wrote: >> Fair enough if we decide that - but we should make that decision >> knowing that we're leaving the JDBC and .Net people in a bad position >> where they are not likely to be able to implement his. >> >> The JDBC peop

Re: [HACKERS] libpq compression

2012-06-15 Thread Marko Kreen
On Sat, Jun 16, 2012 at 6:39 AM, Magnus Hagander wrote: > On Sat, Jun 16, 2012 at 4:04 AM, Euler Taveira wrote: >> On 15-06-2012 11:39, Magnus Hagander wrote: >>> As long as a free implementation exists, it can be ported to >>> Java/.Net. Sure, it takes more work, but it *can be done*. >>> >> Goo

Re: [HACKERS] libpq compression

2012-06-15 Thread Euler Taveira
On 16-06-2012 00:43, Magnus Hagander wrote: > For example, we can no longer trust the SSL library to always do > encryption, since we specifically want to support null encryption. > Meaning we need to teach pg_hba to treat a connection with null > encryption as hostnossl, even if it's an openssl-ba

Re: [HACKERS] libpq compression

2012-06-15 Thread Magnus Hagander
On Sat, Jun 16, 2012 at 6:37 AM, Tom Lane wrote: > I wrote: >> Euler Taveira writes: >>> I see the point in not adding another dependencies or reinventing the wheel >>> but I see more drawbacks than benefits in adopting a SSL-based compression. > >> In the end, judging this tradeoff is a matter o

Re: [HACKERS] libpq compression

2012-06-15 Thread Magnus Hagander
On Sat, Jun 16, 2012 at 4:04 AM, Euler Taveira wrote: > On 15-06-2012 11:39, Magnus Hagander wrote: >> As long as a free implementation exists, it can be ported to >> Java/.Net. Sure, it takes more work, but it *can be done*. >> > Good point. IMHO, if there isn't a solution that cover all PostgreS

Re: [HACKERS] libpq compression

2012-06-15 Thread Magnus Hagander
On Sat, Jun 16, 2012 at 12:03 AM, Heikki Linnakangas wrote: > On 15.06.2012 18:28, Magnus Hagander wrote: >> >> On Fri, Jun 15, 2012 at 11:24 PM, Heikki Linnakangas >>  wrote: >>> >>> On 15.06.2012 17:58, Magnus Hagander wrote: On Fri, Jun 15, 2012 at 10:56 PM, Heikki Linnakangas >

Re: [HACKERS] libpq compression

2012-06-15 Thread Bruce Momjian
On Fri, Jun 15, 2012 at 12:48:24PM +0200, Florian Pflug wrote: > > Yeah, but that alone is IMO a rather big blocker for claiming that > > this is the only way to do it :( And I think the fact that that > > wikipedia page doesn't list any other ones, is a sign that there might > > not be a lot of ot

Re: [HACKERS] libpq compression

2012-06-15 Thread Tom Lane
I wrote: > Euler Taveira writes: >> I see the point in not adding another dependencies or reinventing the wheel >> but I see more drawbacks than benefits in adopting a SSL-based compression. > In the end, judging this tradeoff is a matter of opinion, but I come to > the opposite conclusion. BTW,

Re: [HACKERS] libpq compression

2012-06-15 Thread Tom Lane
Euler Taveira writes: > I see the point in not adding another dependencies or reinventing the wheel > but I see more drawbacks than benefits in adopting a SSL-based compression. In the end, judging this tradeoff is a matter of opinion, but I come to the opposite conclusion. Transport-level compr

Re: [HACKERS] libpq compression

2012-06-15 Thread Euler Taveira
On 15-06-2012 11:10, k...@rice.edu wrote: > I agree and think that the SSL-based compression is an excellent default > compression scheme. The plugable compression approach allows for the > choice of the most appropriate compression implementation based on the > application needs. It really address

Re: [HACKERS] libpq compression

2012-06-15 Thread Euler Taveira
On 15-06-2012 11:39, Magnus Hagander wrote: > As long as a free implementation exists, it can be ported to > Java/.Net. Sure, it takes more work, but it *can be done*. > Good point. IMHO, if there isn't a solution that cover all PostgreSQL (it seems it is not), we should pick the most appropriate

Re: [HACKERS] libpq compression

2012-06-15 Thread Heikki Linnakangas
On 15.06.2012 18:28, Magnus Hagander wrote: On Fri, Jun 15, 2012 at 11:24 PM, Heikki Linnakangas wrote: On 15.06.2012 17:58, Magnus Hagander wrote: On Fri, Jun 15, 2012 at 10:56 PM, Heikki Linnakangas wrote: You could write a dummy SSL implementation that only does compression, not enc

Re: [HACKERS] libpq compression

2012-06-15 Thread Ryan Kelly
On Fri, Jun 15, 2012 at 11:28:48PM +0800, Magnus Hagander wrote: > On Fri, Jun 15, 2012 at 11:24 PM, Heikki Linnakangas > wrote: > > On 15.06.2012 17:58, Magnus Hagander wrote: > >> > >> On Fri, Jun 15, 2012 at 10:56 PM, Heikki Linnakangas > >>  wrote: > >>> > >>> On 15.06.2012 17:39, Magnus Haga

Re: [HACKERS] libpq compression

2012-06-15 Thread Tom Lane
Magnus Hagander writes: > On Fri, Jun 15, 2012 at 11:24 PM, Heikki Linnakangas > wrote: >> Umm, then you use a real SSL libray, not the dummy one? > But (in this scenario, and so far nobody has proven it to be wrong) > there exists no real SSL library that does support compression. I do not thi

Re: [HACKERS] libpq compression

2012-06-15 Thread Magnus Hagander
On Fri, Jun 15, 2012 at 11:24 PM, Heikki Linnakangas wrote: > On 15.06.2012 17:58, Magnus Hagander wrote: >> >> On Fri, Jun 15, 2012 at 10:56 PM, Heikki Linnakangas >>  wrote: >>> >>> On 15.06.2012 17:39, Magnus Hagander wrote: On Fri, Jun 15, 2012 at 6:48 PM, Florian Pflug    wrot

Re: [HACKERS] libpq compression

2012-06-15 Thread Heikki Linnakangas
On 15.06.2012 17:58, Magnus Hagander wrote: On Fri, Jun 15, 2012 at 10:56 PM, Heikki Linnakangas wrote: On 15.06.2012 17:39, Magnus Hagander wrote: On Fri, Jun 15, 2012 at 6:48 PM, Florian Pflugwrote: The way I see it, if we use SSL-based compression then non-libpq clients there's at

Re: [HACKERS] libpq compression

2012-06-15 Thread Magnus Hagander
On Fri, Jun 15, 2012 at 10:56 PM, Heikki Linnakangas wrote: > On 15.06.2012 17:39, Magnus Hagander wrote: >> >> On Fri, Jun 15, 2012 at 6:48 PM, Florian Pflug  wrote: >>> >>> The way I see it, if we use SSL-based compression then non-libpq clients >>> >>> there's at least a chance of those clients

Re: [HACKERS] libpq compression

2012-06-15 Thread Heikki Linnakangas
On 15.06.2012 17:39, Magnus Hagander wrote: On Fri, Jun 15, 2012 at 6:48 PM, Florian Pflug wrote: The way I see it, if we use SSL-based compression then non-libpq clients there's at least a chance of those clients being able to use it easily (if their SSL implementation supports it). If we go w

Re: [HACKERS] libpq compression

2012-06-15 Thread Magnus Hagander
On Fri, Jun 15, 2012 at 6:48 PM, Florian Pflug wrote: > On Jun15, 2012, at 12:09 , Magnus Hagander wrote: >> On Fri, Jun 15, 2012 at 5:52 PM, Florian Pflug wrote: >>> On Jun15, 2012, at 07:50 , Magnus Hagander wrote: Second, we also have things like the JDBC driver and the .Net driver t

Re: [HACKERS] libpq compression

2012-06-15 Thread k...@rice.edu
On Fri, Jun 15, 2012 at 07:18:34AM -0500, Merlin Moncure wrote: > On Fri, Jun 15, 2012 at 5:48 AM, Florian Pflug wrote: > > On Jun15, 2012, at 12:09 , Magnus Hagander wrote: > >> On Fri, Jun 15, 2012 at 5:52 PM, Florian Pflug wrote: > >>> On Jun15, 2012, at 07:50 , Magnus Hagander wrote: > S

Re: [HACKERS] libpq compression

2012-06-15 Thread Merlin Moncure
On Fri, Jun 15, 2012 at 5:48 AM, Florian Pflug wrote: > On Jun15, 2012, at 12:09 , Magnus Hagander wrote: >> On Fri, Jun 15, 2012 at 5:52 PM, Florian Pflug wrote: >>> On Jun15, 2012, at 07:50 , Magnus Hagander wrote: Second, we also have things like the JDBC driver and the .Net driver t

Re: [HACKERS] libpq compression

2012-06-15 Thread Florian Pflug
On Jun15, 2012, at 12:09 , Magnus Hagander wrote: > On Fri, Jun 15, 2012 at 5:52 PM, Florian Pflug wrote: >> On Jun15, 2012, at 07:50 , Magnus Hagander wrote: >>> Second, we also have things like the JDBC driver and the .Net driver >>> that don't use libpq. the JDBC driver uses the native java ssl

Re: [HACKERS] libpq compression

2012-06-15 Thread Magnus Hagander
On Fri, Jun 15, 2012 at 5:52 PM, Florian Pflug wrote: > On Jun15, 2012, at 07:50 , Magnus Hagander wrote: So I've got very little patience with the idea of "let's put in some hooks and then great things will happen".  It would be far better all around if we supported exactly one, we

Re: [HACKERS] libpq compression

2012-06-15 Thread Florian Pflug
On Jun15, 2012, at 07:50 , Magnus Hagander wrote: >>> So I've got very little patience with the idea of "let's put in some >>> hooks and then great things will happen". It would be far better all >>> around if we supported exactly one, well-chosen, method. But really >>> I still don't see a reaso

Re: [HACKERS] libpq compression

2012-06-14 Thread Magnus Hagander
On Fri, Jun 15, 2012 at 10:19 AM, Bruce Momjian wrote: > On Thu, Jun 14, 2012 at 02:43:04PM -0400, Tom Lane wrote: >> Euler Taveira writes: >> > On 14-06-2012 02:19, Tom Lane wrote: >> >> ...  I especially think that importing bzip2 >> >> is a pretty bad idea --- it's not only a new dependency, b

Re: [HACKERS] libpq compression

2012-06-14 Thread Bruce Momjian
On Thu, Jun 14, 2012 at 02:43:04PM -0400, Tom Lane wrote: > Euler Taveira writes: > > On 14-06-2012 02:19, Tom Lane wrote: > >> ... I especially think that importing bzip2 > >> is a pretty bad idea --- it's not only a new dependency, but bzip2's > >> compression versus speed tradeoff is entirely

Re: [HACKERS] libpq compression

2012-06-14 Thread k...@rice.edu
On Thu, Jun 14, 2012 at 02:38:02PM -0500, Merlin Moncure wrote: > On Thu, Jun 14, 2012 at 1:43 PM, Tom Lane wrote: > > So I've got very little patience with the idea of "let's put in some > > hooks and then great things will happen".  It would be far better all > > around if we supported exactly o

Re: [HACKERS] libpq compression

2012-06-14 Thread Merlin Moncure
On Thu, Jun 14, 2012 at 1:43 PM, Tom Lane wrote: > So I've got very little patience with the idea of "let's put in some > hooks and then great things will happen".  It would be far better all > around if we supported exactly one, well-chosen, method.  But really > I still don't see a reason not to

Re: [HACKERS] libpq compression

2012-06-14 Thread Tom Lane
Euler Taveira writes: > On 14-06-2012 02:19, Tom Lane wrote: >> ... I especially think that importing bzip2 >> is a pretty bad idea --- it's not only a new dependency, but bzip2's >> compression versus speed tradeoff is entirely inappropriate for this >> use-case. > I see, the idea is that bzip2

Re: [HACKERS] libpq compression

2012-06-14 Thread Merlin Moncure
On Thu, Jun 14, 2012 at 9:57 AM, Phil Sorber wrote: > It doesn't sound like there is a lot of support for this idea, but I > think it would be nice to get something like lz4 > (http://code.google.com/p/lz4/) or snappy > (http://code.google.com/p/snappy/) support. Both are BSD-ish licensed. > It co

Re: [HACKERS] libpq compression

2012-06-14 Thread Phil Sorber
On Thu, Jun 14, 2012 at 10:14 AM, Florian Pflug wrote: > On Jun14, 2012, at 15:28 , Euler Taveira wrote: >> On 14-06-2012 02:19, Tom Lane wrote: >>> I still think that pushing this off to openssl (not an ssh tunnel, but >>> the underlying transport library) would be an adequate solution. >>> If yo

Re: [HACKERS] libpq compression

2012-06-14 Thread Florian Pflug
On Jun14, 2012, at 15:28 , Euler Taveira wrote: > On 14-06-2012 02:19, Tom Lane wrote: >> I still think that pushing this off to openssl (not an ssh tunnel, but >> the underlying transport library) would be an adequate solution. >> If you are shoving data over a connection that is long enough to ne

Re: [HACKERS] libpq compression

2012-06-14 Thread Euler Taveira
On 14-06-2012 02:19, Tom Lane wrote: > I still think that pushing this off to openssl (not an ssh tunnel, but > the underlying transport library) would be an adequate solution. > If you are shoving data over a connection that is long enough to need > compression, the odds that every bit of it is tr

Re: [HACKERS] libpq compression

2012-06-14 Thread Albe Laurenz
Euler Taveira wrote: > There was already some discussion about compressing libpq data [1][2][3]. > Recently, I faced a scenario that would become less problematic if we have had > compression support. The scenario is frequent data load (aka COPY) over > slow/unstable links. It should be executed in

Re: [HACKERS] libpq compression

2012-06-13 Thread Tom Lane
Euler Taveira writes: > There was already some discussion about compressing libpq data [1][2][3]. > Recently, I faced a scenario that would become less problematic if we have had > compression support. The scenario is frequent data load (aka COPY) over > slow/unstable links. It should be executed