Re: [HACKERS] Wire protocol compression

2016-04-22 Thread Craig Ringer 
On 21 April 2016 at 22:21, Andreas Karlsson  wrote:


> Wouldn't such a solution be just as vulnerable to CRIME as TLS is? I
> thought the reason for removing compression from TLS is to discourage
> people from writing applications which are vulnerable to compression based
> attacks by not proving an easy for people to just compress everything.
> 
>

Probably... but you could then use compression without encryption without
hacks like forcing a noop cipher. If you're running traffic over a VPN WAN
link or something that could be a pretty sensible option. OTOH, such a VPN
may have its own compression, rendering compression by Pg unnecessary. The
downside is that the VPN overheads can be considerable as can the general
performance impact.

Personally I admit I just don't care that much about the CRIME attack for
most of the deployments I'm interested in. It requires the attacker be able
to make alterations on the server.

"The attacker then observes the change in size of the compressed request
payload, which contains both the secret cookie that is sent by the browser
only to the target site, and variable content created by the attacker, as
the variable content is altered."

https://en.wikipedia.org/wiki/CRIME

I'm not especially concerned that authorized user 'unpriv-1' can hijack
user 'super' 's connections if unpriv-1 can somehow modify tables super is
reading *and* snoop super's traffic, doing it all on a tight schedule.
We've probably got bigger security issues than that.

Our auth salting helps a lot anyway, and while there are situations where a
privileged user could be fetching some slow-changing short and
security-critical content repeatedly from a table as part of a join the
unprivileged user can modify data in, I'm again just not that concerned.

-- 
 Craig Ringer   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Wire protocol compression

2016-04-22 Thread Craig Ringer 
On 21 April 2016 at 22:07, Tom Lane  wrote:


>
> Yeah.  I think this should definitely be in the list of things we want
> to add when we do the fabled 4.0 protocol revision (and, indeed, it's
> been in the above-cited list for awhile).  Whether we've yet gotten to
> the point of having critical mass for a revision ... meh, I doubt it.
>
>
Well, the critical mass is determined as much as anything by who's willing
to put the large amount of work into implementing the protocol rev, testing
clients, verifying performance, etc etc etc. So far I don't think anyone's
leaping to volunteer.

We could use various hacks to enable wire compression without a protocol
rev, like sending a gzip header before the startup packet so a server that
doesn't understand it just sees a nonsensical startup packet and borks. But
they all either boil down to uglier versions of genuine protocol bump or
are potentially subject to the issues you raised earlier with
GUC-controlled protocol changes.


In retrospect it's a huge pity there was no set of client-library-only
key/value fields defined in the startup packet in addition to GUCs that
libpq lets clients set via PGOPTIONS etc. That would've let us extend the
protocol much more easily by sending capabilities like "I understand
protocol compression".

Personally I think we should do that anyway - allow a startup-only GUC like
proto_compression=gzip .

To help protect against abuse via

  PGOPTIONS="-c proto_compression=gzip" ./exploitable-old-libpq-based-binary

we could do something like sending an ErrorResponse with a newly defined
SQLSTATUS like 08P02 Client must support gzip compression . A client that
supports gzip may ignore this error and proceed normally. A client that
doesn't will not recognise the SQLSTATUS and since it's in a fatal category
08 (or rather, not the two non-fatal categories 00, 01 and 02) it'll die
and won't see any upsetting new protocol messages. Clients that don't
support it will even get a nice error message:

psql: FATAL:  unrecognized configuration parameter "proto_compression"

It's really just a hack around bumping the protocol to add capability
negotiation though, and it'd scale very poorly if it was then used for more
things. Lots of reconnections needed because we can't do a two-way
negotiation.

I think retrofitting protocol compression might be compelling enough to
justify that. But it's not really significantly better than just sending a
v4 protoversion field and reconnecting if the server gets upset with that.


-- 
 Craig Ringer   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Wire protocol compression

2016-04-21 Thread Andreas Karlsson 

On 04/21/2016 03:04 PM, Aleksander Alekseev wrote:

I guess since the usual answer for compression was "use what SSL
provides you for free", it's rather unlikely that someone bothered to
make a proxy just for that purpose, and really, a proxy is just
another moving part in your setup: not everyone will be thrilled to
add that.


It just doesn't sound like a feature that should be implemented
separately for every single application that uses TCP. Granted TCP proxy
is not the most convenient way to solve a task. Maybe it could be
implemented in OpenVPN or on Linux TCP/IP stack level.


Wouldn't such a solution be just as vulnerable to CRIME as TLS is? I 
thought the reason for removing compression from TLS is to discourage 
people from writing applications which are vulnerable to compression 
based attacks by not proving an easy for people to just compress everything.


Andreas


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Wire protocol compression

2016-04-21 Thread Tom Lane 
Craig Ringer  writes:
> On 21 April 2016 at 14:19, Shay Rojansky  wrote:
>> There are potentially huge bandwidth savings which could benefit both WAN
>> and non-WAN scenarios, and decoupling this problem from TLS would make it
>> both accessible to everyone (assuming PostgreSQL clients follow). It would
>> be a protocol change though.

> The problem there is that suddenly everyone wants to get their desired
> protocol features in, since we're changing things anyway, and "enabling
> protocol compression" becomes ... rather more.
> https://wiki.postgresql.org/wiki/Todo#Wire_Protocol_Changes

Yeah.  I think this should definitely be in the list of things we want
to add when we do the fabled 4.0 protocol revision (and, indeed, it's
been in the above-cited list for awhile).  Whether we've yet gotten to
the point of having critical mass for a revision ... meh, I doubt it.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Wire protocol compression

2016-04-21 Thread Craig Ringer 
On 21 April 2016 at 14:19, Shay Rojansky  wrote:


> Does it make sense to you guys to discuss compression outside of TLS?
>

Yes.


> There are potentially huge bandwidth savings which could benefit both WAN
> and non-WAN scenarios, and decoupling this problem from TLS would make it
> both accessible to everyone (assuming PostgreSQL clients follow). It would
> be a protocol change though.
>

It would, but not necessarily a drastic one.

The only reason it has to be any drama at all is that we can't securely use
some GUC like allow_compression=on set in the startup message. As Tom has
pointed out in the past, users may be able to set these "through" a  client
library that is not expecting compressed responses, causing the client
library to fail in a possibly-exploitable manner. I'm not nearly as
concerned about that, but I don't deny that it's theoretically possible.

So we'd have to do a protocol version bump from the 3.0 protocol, allowing
clients to send 3.1 (or 4.0 or whatever) protocol requests. Newer clients
connecting to older servers would get rejected and have to reconnect with
3.0 protocol, or be configured with protocol=3.0 in their setup (JDBC URI,
libpq connstring, etc). Much like we already do for SSL when the server
doesn't support it.

The problem there is that suddenly everyone wants to get their desired
protocol features in, since we're changing things anyway, and "enabling
protocol compression" becomes ... rather more.

https://wiki.postgresql.org/wiki/Todo#Wire_Protocol_Changes

It'd be pretty stupid not to add some form of capabilities negotiation, at
least, so we can avoid having the same drama next time. In the process we'd
probably want to get proper STARTTLS-like support in place to avoid the
disconnect/reconnect dance when connecting to a host that doesn't support
SSL.

So this isn't quite as simple as just adding compression if both client and
server support it. But I do think it's very worthwhile, and that we've been
relying on a pretty sad cop-out for some time by telling people to use SSL
protocol layer compression.

-- 
 Craig Ringer   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] Wire protocol compression

2016-04-21 Thread Shulgin, Oleksandr 
On Thu, Apr 21, 2016 at 3:17 PM, Aleksander Alekseev <
a.aleks...@postgrespro.ru> wrote:

> > > or on Linux TCP/IP stack level.
> > >
> >
> > Yes, but if you want to have both compression and encryption it is
> > crucial to apply compression *before* encryption and I don't see how
> > this can happen with this approach.
>
> If I'm not mistaken IPSec gives you both compression and encryption.
> Just as an example.
>

True, but that setup (as well as any other hypothetical TCP/IP stack level
solution) would require network tweaking on both server and the clients.
With protocol-level compression you can avoid that altogether and
encryption is already solved with use of TLS.

--
Alex

Re: [HACKERS] Wire protocol compression

2016-04-21 Thread Aleksander Alekseev 
> > or on Linux TCP/IP stack level.
> >
> 
> Yes, but if you want to have both compression and encryption it is
> crucial to apply compression *before* encryption and I don't see how
> this can happen with this approach.

If I'm not mistaken IPSec gives you both compression and encryption.
Just as an example. 

-- 
Best regards,
Aleksander Alekseev
http://eax.me/


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Wire protocol compression

2016-04-21 Thread Shulgin, Oleksandr 
On Thu, Apr 21, 2016 at 3:04 PM, Aleksander Alekseev <
a.aleks...@postgrespro.ru> wrote:

> > I guess since the usual answer for compression was "use what SSL
> > provides you for free", it's rather unlikely that someone bothered to
> > make a proxy just for that purpose, and really, a proxy is just
> > another moving part in your setup: not everyone will be thrilled to
> > add that.
>
> It just doesn't sound like a feature that should be implemented
> separately for every single application that uses TCP. Granted TCP proxy
> is not the most convenient way to solve a task. Maybe it could be
> implemented in OpenVPN


Which is another moving part with its own setup and maintenance overhead.


> or on Linux TCP/IP stack level.
>

Yes, but if you want to have both compression and encryption it is crucial
to apply compression *before* encryption and I don't see how this can
happen with this approach.

--
Alex

Re: [HACKERS] Wire protocol compression

2016-04-21 Thread Aleksander Alekseev 
> I guess since the usual answer for compression was "use what SSL
> provides you for free", it's rather unlikely that someone bothered to
> make a proxy just for that purpose, and really, a proxy is just
> another moving part in your setup: not everyone will be thrilled to
> add that.

It just doesn't sound like a feature that should be implemented
separately for every single application that uses TCP. Granted TCP proxy
is not the most convenient way to solve a task. Maybe it could be
implemented in OpenVPN or on Linux TCP/IP stack level.

-- 
Best regards,
Aleksander Alekseev
http://eax.me/


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Wire protocol compression

2016-04-21 Thread Shulgin, Oleksandr 
On Thu, Apr 21, 2016 at 11:07 AM, Aleksander Alekseev <
a.aleks...@postgrespro.ru> wrote:

> > Does it make sense to you guys to discuss compression outside of TLS?
> > There are potentially huge bandwidth savings which could benefit both
> > WAN and non-WAN scenarios, and decoupling this problem from TLS would
> > make it both accessible to everyone (assuming PostgreSQL clients
> > follow). It would be a protocol change though.
>
> I personally don't think it's something that should be implemented in
> PostgreSQL core. As a third-party TCP-proxy (on both client and server
> sides) with gzip/lz4 support perhaps. I'll be not surprised if it turns
> out that such projects already exist.
>

Hm, did you see this recent discussion on -hackers:
http://www.postgresql.org/message-id/flat/CAMkU=1zt9cjaQjVYAmywcP9iyxMJxFBUaVeB1eiaqBP=gej...@mail.gmail.com#CAMkU=1zt9cjaQjVYAmywcP9iyxMJxFBUaVeB1eiaqBP=gej...@mail.gmail.com
?

I guess since the usual answer for compression was "use what SSL provides
you for free", it's rather unlikely that someone bothered to make a proxy
just for that purpose, and really, a proxy is just another moving part in
your setup: not everyone will be thrilled to add that.

--
Alex

Re: [HACKERS] Wire protocol compression

2016-04-21 Thread Aleksander Alekseev 
> Does it make sense to you guys to discuss compression outside of TLS?
> There are potentially huge bandwidth savings which could benefit both
> WAN and non-WAN scenarios, and decoupling this problem from TLS would
> make it both accessible to everyone (assuming PostgreSQL clients
> follow). It would be a protocol change though.

I personally don't think it's something that should be implemented in
PostgreSQL core. As a third-party TCP-proxy (on both client and server
sides) with gzip/lz4 support perhaps. I'll be not surprised if it turns
out that such projects already exist.

-- 
Best regards,
Aleksander Alekseev
http://eax.me/


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Wire protocol compression

2016-04-21 Thread Shay Rojansky 
I know this has been discussed before (
http://postgresql.nabble.com/Compression-on-SSL-links-td2261205.html,
http://www.postgresql.org/message-id/BANLkTi=Ba1ZCmBuwwn7M1wvPFioT=6n...@mail.gmail.com),
but it seems to make sense to revisit this in 2016.

Since CRIME in 2012, AFAIK compression with encryption is considered
insecure, and the feature is removed entirely in the TLS 1.3 draft. In
addition (and because of that), many (most?) client TLS implementations
don't support compression (Java, .NET), meaning that a considerable number
of PostgreSQL users don't have access to compression.

Does it make sense to you guys to discuss compression outside of TLS? There
are potentially huge bandwidth savings which could benefit both WAN and
non-WAN scenarios, and decoupling this problem from TLS would make it both
accessible to everyone (assuming PostgreSQL clients follow). It would be a
protocol change though.

Shay