Re: [HACKERS] PostgreSQL Auditing

[Joshua D. Drake]

On 02/02/2016 07:26 PM, Robert Haas wrote:

On Tue, Feb 2, 2016 at 8:25 PM, Curtis Ruck
 wrote:

Additionally Robert, given your professional status, you are by no means an
unbiased contributor in this discussion.  Your stance on this matter shows
that you don't necessarily want the open source solution to succeed in the
commercial/compliance required space.  Instead of arguing blankly against
inclusion can you at least provide actionable based feedback that if met
would allow patches of this magnitude in?


I am *the* first person to criticise EDB. There isn't a person in EDB 
that would find that surprising.


That said, Robert, Dave, and all the other contributors that work for 
EDB are stand up people. They deserve our respect. Period.


I appreciate the thought process here but if you want any kind of 
legitimacy in this community I suggest you do your homework.


JD

--
Command Prompt, Inc.  http://the.postgres.company/
+1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PostgreSQL Auditing

[Robert Haas]

On Tue, Feb 2, 2016 at 8:25 PM, Curtis Ruck
 wrote:
> Additionally Robert, given your professional status, you are by no means an
> unbiased contributor in this discussion.  Your stance on this matter shows
> that you don't necessarily want the open source solution to succeed in the
> commercial/compliance required space.  Instead of arguing blankly against
> inclusion can you at least provide actionable based feedback that if met
> would allow patches of this magnitude in?

I feel the need to respond to this.  I don't appreciate being called a
liar.  I do not block patches because having them included in
PostgreSQL would be to the detriment of my employer.  Ever.  That
would be dishonest and a betrayal of the community's trust.  Full
stop.  I have a track record of committing multiple large patches that
were developed by people working for EnterpriseDB's competitors (like
logical decoding) or that competed with proprietary features
EnterpriseDB already had (like foreign tables).  I spend countless
hours working on countless patches written by people who do not work
for, and have no commercial relationship with, my employer: and who
are sometimes working for a competitor.  I have worked hard to ensure
that EnterpriseDB makes major contributions to the PostgreSQL
community, such as parallel query.

Even if all of the above were not true, EnterpriseDB does not
currently have a feature that competes with pgaudit, and has no
current plans to try to develop one.  EDBAS does have an auditing
feature, but that feature is radically different from what pgaudit
does; arguing that I am trying to block pgaudit from going into core
because that feature exists is like arguing that I don't want
PostgreSQL to get a new frying pan because EnterpriseDB has a toy
boat.  Furthermore, to the extent that EnterpriseDB does have an
interest in having a feature like pgaudit, it would be to my advantage
for that feature to go *into* core.  After all, everything in
PostgreSQL is in EDBAS, but things on PGXN generally aren't.  In
short, your accusations are both  false and illogical.

I'm going to go ahead and make a suggestion: instead of showing up on
this mailing list and accusing the people who spend their time and
energy here trying to make PostgreSQL a better of being pigheaded
liars, I think that you should try to really understand how this
community works, how it makes decisions, what it does well, and what
it does poorly.  Then, I think you should argue for your positions in
a respectful way, carefully avoiding accusations of bad faith even
(and perhaps especially) in cases where you believe it may be present.
You will find that almost everyone here behaves in that way, and that
is what enables us to get along as well as we do and create a great
piece of software together.  Every single person who has responded to
your emails - and there have been a bunch - has done so with courtesy
and integrity, and yet you seem convinced (without a shred of
evidence, at least that you've presented) that anyone who doesn't
think pgaudit should go into core is either an idiot or part of some
sort of cabal.  Yet, if that were really true, there would be little
point in arguing, because the cabalists won't listen to you anyway,
and the idiots will make stupid decisions no matter what.  Perhaps you
should try starting from a different premise.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PostgreSQL Auditing

[David G. Johnston]

​So, Noah's excellent response has been ignored (from what my threaded
Gmail view tells me) at this point...​

On Tue, Feb 2, 2016 at 6:25 PM, Curtis Ruck <
curtis.ruck+pgsql.hack...@gmail.com> wrote:

> Robert,
>
> This isn't wrong.  I don't see anyone else trying to submit anything in
> reference to auditing.  Just because there have been multiple
> implementations in the past, based on commit histories, there have only
> been a few that tried getting into core or contrib (that i can find in
> mailing list history).
>

​And so if pgaudit is such an improvement then by this logic a poor
implementation would have been committed in the past just because someone
wrote something and proposed it for commit.​  Noah summarized his quite
weighty point-of-view on the outcome of the patch that has been put forth.
One dynamic of which is whether it would be easier - and overall more
productive - to make installing auditing as an extension easier compared to
getting it into core.


> I think if there was a valid commit path laid out for getting auditing
> into core, or into contrib at least, most users would probably find that
> sufficient.  But it seems that every time someone tries to incorporate
> auditing, there are countless and varied reasons to deny its inclusion.
>

​You pre-suppose that people are willing and able to spend their time
trying to predict the future when in reality they would rather be convinced
that what has been put in front of them is worth committing.  For what I
presume are myriad of both technical and political reasons the people who
have the final say have not been so convinced.


> It just seems after reading the mailing list history, that there is a lack
> of interest by people with commit authority, even though there is a decent
> interest in it from the community, and honestly, no one really likes
> auditing, but its one of those damned if you do (in performance) and damned
> if you don't (in legal) things.
>

​So either through persuasion or compensation you need to increase
interest...​



> Additionally Robert, given your professional status, you are by no means
> an unbiased contributor in this discussion.  Your stance on this matter
> shows that you don't necessarily want the open source solution to succeed
> in the commercial/compliance required space.  Instead of arguing blankly
> against inclusion can you at least provide actionable based feedback that
> if met would allow patches of this magnitude in?
>

​Even if that were to be true that would not prevent others involved from
moving things forward - an objection like "I don't want this in -core
because it will eat into my employer's profits" won't persuade many
people.  Nit-picking patches might get a bit further but if there is
sufficient buy-in from the community as a whole it won't last long.​  So
maybe Robert doesn't actively help as much as he could but a lack of
helping is not the same as hindering.


> I'm personally fine with fiscally rewarding organizations that assist my
> customer in succeeding, but its hard to convince my customer to fund open
> source, even though they wouldn't be able to do 75% of what they do without
> it.  Based on past experience this is the same most open source
> organizations face, especially when they don't have the marketing engine
> that the large commercial players have.
>

​Which is why the model seems to be for the service provides who leverage
PostgreSQL to charge their customers enough so that part of the profit can
flow back into the community that they have based their livelihood upon.
Otherwise, maybe features such as this should end up in commercial
offerings and those customers who won't charitably contribute will instead
have an easier time seeing that at least they are spending less for
PostgreSQL licensing than they would for similar features in competitors
offerings.


​In my estimation this ​whole thread started poorly and thus likely will
not garner much if any forward progress on the issue.  Noah's response was
spot-on despite that fact and a proper response to it would likely move
things forward in a more positive manner.  A separate thread could be
started to discuss what can be done to improve the scenario where excellent
patches and/or extensions are available but are not in core.  These two
dynamics are separate and trying to hit both in one thread is just going to
dilute things so that likely neither topic gets progressed.

David J.

Re: [HACKERS] PostgreSQL Auditing

[Jim Nasby]

On 2/2/16 7:25 PM, Curtis Ruck wrote:

I'm opening to testing and evaluating to see if it meets our compliance
requirements, but I am no where close to being a C developer, or having
C developers that could actually provide a meaningful review.  One issue
along this thread already pops up, concerning the client_min_messages,
and how other patches in the pipeline for 9.6 would be required to
enable the auditing to meet compliance requirements.


There's other ways you can help besides reviewing. Providing real-world 
use cases helps. Even better is maintaining things on the wiki that 
assist with moving things forward (use cases, discussion/decision 
highlights, really anything that helps move the discussion).



It just seems after reading the mailing list history, that there is a
lack of interest by people with commit authority, even though there is a
decent interest in it from the community, and honestly, no one really
likes auditing, but its one of those damned if you do (in performance)
and damned if you don't (in legal) things.


Yeah, no one that's volunteering time (as opposed to being paid to work 
on PG) is going to pick up something as unsexy and painful to deal with 
as auditing.



Additionally Robert, given your professional status, you are by no means
an unbiased contributor in this discussion.  Your stance on this matter
shows that you don't necessarily want the open source solution to
succeed in the commercial/compliance required space.  Instead of arguing


I'm sorry, but that's just ridiculous, and I completely agree with 
Robert's initial sentiment: there needs to be a damn good reason for the 
community to pick one specific implementation of something when there 
are competing solutions.



blankly against inclusion can you at least provide actionable based
feedback that if met would allow patches of this magnitude in?


It works just like any other patch does: the community has to come to a 
*consensus* that not only is the feature desired and well designed, but 
that the implementation is high quality. I haven't followed the auditing 
discussions closely, but it seems that there are still questions around 
how the feature should work.



I'm personally fine with fiscally rewarding organizations that assist my
customer in succeeding, but its hard to convince my customer to fund
open source, even though they wouldn't be able to do 75% of what they do
without it.  Based on past experience this is the same most open source
organizations face, especially when they don't have the marketing engine
that the large commercial players have.


I really don't understand that, given what most of the alternative 
solutions cost. If they balk at putting money towards developing 
Postgres they really need to get a quote for running the same amount of 
MSSQL (let alone Oracle, which is even more expensive).


I do think the community could do a better job of at least encouraging 
companies to fund development. Unfortunately there's always going to be 
some amount of friction here though, because of the question of how to 
allocate funds to the different companies that are involved. Another 
problem is no commercial company can actually guarantee anything will 
make it into community Postgres, and it's very difficult to even 
estimate the amount of effort (read as: what to charge) for getting a 
feature committed.


Commercial development is certainly possible though. 2nd Quadrant was 
able to raise a good amount of money to fund the development of hot 
standby. IIRC that was before sites like kickstarter existed too, so it 
would probably be even easier to do today.

--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PostgreSQL Auditing

[Curtis Ruck]

Robert,

This isn't wrong.  I don't see anyone else trying to submit anything in
reference to auditing.  Just because there have been multiple
implementations in the past, based on commit histories, there have only
been a few that tried getting into core or contrib (that i can find in
mailing list history).  Currently, in 9.6, there is one version that is
trying to make it in.  If Crunchy, or 2nd Quadrant, tried to get their
versions incorporated we would have a disagreement in implementation, but
either they are lying in wait, or they passively concur, by not actively
disagreeing.

I think if there was a valid commit path laid out for getting auditing into
core, or into contrib at least, most users would probably find that
sufficient.  But it seems that every time someone tries to incorporate
auditing, there are countless and varied reasons to deny its inclusion.

David Steele's version of auditing builds on most of the prior pgaudit code
and incorporates a large amount of the feedback from 9.5's attempt.

I'm opening to testing and evaluating to see if it meets our compliance
requirements, but I am no where close to being a C developer, or having C
developers that could actually provide a meaningful review.  One issue
along this thread already pops up, concerning the client_min_messages, and
how other patches in the pipeline for 9.6 would be required to enable the
auditing to meet compliance requirements.

It just seems after reading the mailing list history, that there is a lack
of interest by people with commit authority, even though there is a decent
interest in it from the community, and honestly, no one really likes
auditing, but its one of those damned if you do (in performance) and damned
if you don't (in legal) things.

Additionally Robert, given your professional status, you are by no means an
unbiased contributor in this discussion.  Your stance on this matter shows
that you don't necessarily want the open source solution to succeed in the
commercial/compliance required space.  Instead of arguing blankly against
inclusion can you at least provide actionable based feedback that if met
would allow patches of this magnitude in?

I'm personally fine with fiscally rewarding organizations that assist my
customer in succeeding, but its hard to convince my customer to fund open
source, even though they wouldn't be able to do 75% of what they do without
it.  Based on past experience this is the same most open source
organizations face, especially when they don't have the marketing engine
that the large commercial players have.

Curtis

…

On Tue, Feb 2, 2016 at 5:23 PM Robert Haas  wrote:

> On Tue, Feb 2, 2016 at 5:16 PM, David Steele  wrote:
> > This sort of confusion is one of the main reasons I pursued inclusion in
> > core.
>
> But that's exactly wrong.  When there is not agreement on one code
> base over another, that's the time it is most important not to pick
> one of them arbitrarily privilege it over the others.  The *only* time
> it's appropriate to move something that could just as well as an
> extension into core is when (1) we think it's highly likely that users
> will want that particular thing rather than some other thing and (2)
> we think it's worth the burden of maintaining that code forever.
>
> --
> Robert Haas
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>

Re: [HACKERS] PostgreSQL Auditing

[curtis . ruck]

Its not available in rpm or premade binary forms in its current instantiation, 
not a big deal for me, but it raises the barrier to entry.

If it was packaged into an RPM, what would be the process to get it added to 
postgresql's yum repositories?

February 2 2016 10:24 AM, "Joshua D. Drake"  wrote:
> On 02/02/2016 02:47 AM, José Luis Tallón wrote:
> 
>> On 02/02/2016 02:05 AM, Curtis Ruck wrote:
>>> [snip]
>>> 
>>> P.S., do you know what sucks, having a highly performant PostGIS
>>> database that works great, and being told to move to Oracle or SQL
>>> Server (because they have auditing). Even though they charge extra
>>> for Geospatial support (seriously?) or when they don't even have
>>> geospatial support (10 years ago). My customer would prefer to
>>> re-engineer software designed around PostgreSQL and pay the overpriced
>>> licenses, than not have auditing. I agree that their cost analysis is
>>> probably way off, even 10 years later, my only solution would be to
>>> move to Oracle, SQL Server, a NoSQL solution, or pay EnterpriseDB for
>>> their 2 year old version that doesn't have all the cool/modern jsonb
>>> support.
> 
> PostgreSQL has auditing. It is available now, just not in core. Postgis isn't 
> available in core
> either and it seems to do just fine.
> 
> JD
> 
> -- Command Prompt, Inc. http://the.postgres.company
> +1-503-667-4564
> PostgreSQL Centered full stack support, consulting and development.
> Everyone appreciates your honesty, until you are honest with them.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PostgreSQL Auditing

[Robert Haas]

On Tue, Feb 2, 2016 at 5:16 PM, David Steele  wrote:
> This sort of confusion is one of the main reasons I pursued inclusion in
> core.

But that's exactly wrong.  When there is not agreement on one code
base over another, that's the time it is most important not to pick
one of them arbitrarily privilege it over the others.  The *only* time
it's appropriate to move something that could just as well as an
extension into core is when (1) we think it's highly likely that users
will want that particular thing rather than some other thing and (2)
we think it's worth the burden of maintaining that code forever.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PostgreSQL Auditing

[David Steele]

On 2/2/16 4:50 PM, Michael Banck wrote:
> 
> We are looking into packaging pgaudit for Debian.
> 
> However, then another question comes up: Should the 2nd Quadrant or the
> Crunchy Data codebase be added to the distribution? Who gets to decide?

For my 2 cents I think that the version I submitted recently is better
for PostgreSQL >= 9.5:

https://github.com/pgaudit/pgaudit

For this version I started with 2ndQuadrant's excellent work then spent
months refactoring, testing and adding additional configuration options.
 In addition a number of reviewers found issues which were fixed and in
general it went through a trial by fire.  This version includes more
comprehensive documentation and extensive regression tests.

However, the original 2ndQuadrant version will happily work with
PostgreSQL 9.3 or 9.4:

https://github.com/2ndQuadrant/pgaudit

> Alternatively, both could be added, but that will likely confuse users.

This sort of confusion is one of the main reasons I pursued inclusion in
core.

-- 
-David
da...@pgmasters.net



signature.asc
Description: OpenPGP digital signature

Re: [HACKERS] PostgreSQL Auditing

[Michael Banck]

On Tue, Feb 02, 2016 at 08:28:54AM -0800, Joshua D. Drake wrote:
> On 02/02/2016 07:31 AM, curtis.r...@gmail.com wrote:
> >Its not available in rpm or premade binary forms in its current
> >instantiation, not a big deal for me, but it raises the barrier to
> >entry.
> >
> >If it was packaged into an RPM, what would be the process to get it
> >added to postgresql's yum repositories?
> 
> Assuming it is not placed into core, we would need to work with the
> deb and rpm projects. Which I am sure we could (and would) do.

We are looking into packaging pgaudit for Debian.

However, then another question comes up: Should the 2nd Quadrant or the
Crunchy Data codebase be added to the distribution? Who gets to decide?

Alternatively, both could be added, but that will likely confuse users.


Michael

-- 
Michael Banck
Projektleiter / Berater
Tel.: +49 (2161) 4643-171
Fax:  +49 (2161) 4643-100
Email: michael.ba...@credativ.de

credativ GmbH, HRB Mönchengladbach 12080
USt-ID-Nummer: DE204566209
Hohenzollernstr. 133, 41061 Mönchengladbach
Geschäftsführung: Dr. Michael Meskes, Jörg Folz, Sascha Heuer


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PostgreSQL Auditing

[Jim Nasby]

On 2/2/16 5:00 AM, Simon Riggs wrote:

Since you've written the email here, I'd ask that you join our community
and use your knowledge and passion to make things happen.


+1. Kudos for speaking up in the first place, but it's clear that right 
now the biggest thing holding Postgres back is lack of reviewers, 
followed by lack of developers. If your company put even 10% of what it 
would pay for Oracle or MSSQL licensing back into the community (either 
via direct employee involvement or by funding development) then auditing 
could have moved a lot faster than it did.


It would also help if the community better publicized ways that 
companies could give back.

--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PostgreSQL Auditing

[Jim Nasby]

On 2/2/16 10:34 AM, Joshua D. Drake wrote:

Auditing is a pretty security/enterprisey-related thing that could do
with the "officially considered to of the PostgreSQL project standard
and ready for production" rubber-stamp that tends to go along with most
end-user/admin-oriented stuff shipped in the tarball.


Which is exactly why I think .Org needs an official "Extensions" project
which would completely eliminate these arguments. A project team
explicitly for vetting extensions.


Yeah, it's disappointing that PGXN doesn't seem to have really taken 
off. I'm sure a big part of that is the need for even SQL extensions to 
have server access, but I suspect part of it is because it's a separate 
project.

--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PostgreSQL Auditing

[Robert Haas]

On Tue, Feb 2, 2016 at 11:13 AM, Michael Banck
 wrote:
> On Tue, Feb 02, 2016 at 07:24:23AM -0800, Joshua D. Drake wrote:
>> PostgreSQL has auditing. It is available now, just not in core. Postgis
>> isn't available in core either and it seems to do just fine.
>
> I don't really buy that argument. For one, PostGIS has a pretty narrow
> functional use-case (spatial), while auditing is a horizontal use-case
> that could be required for any kind of database usage.

If you're saying you think the user base for pgaudit will be larger
than the user base for PostGIS, color me doubtful.

> Now, whether or not the currently submitted approach actually meets the
> above rubber-stamp requirements is a different story, but at least I
> think it would be quite useful to ship auditing capabilites in the
> distribution.

That is a sensible position.  :-)

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PostgreSQL Auditing

[Joshua D. Drake]

On 02/02/2016 08:13 AM, Michael Banck wrote:

On Tue, Feb 02, 2016 at 07:24:23AM -0800, Joshua D. Drake wrote:

PostgreSQL has auditing. It is available now, just not in core. Postgis
isn't available in core either and it seems to do just fine.


I don't really buy that argument. For one, PostGIS has a pretty narrow
functional use-case (spatial), while auditing is a horizontal use-case
that could be required for any kind of database usage.


The argument was made specifically for the user because they were using 
PostGIS.




Second, PostGIS had 10+ (?) years to build a reputation so that people
say "if I have to choose between PostGIS and buying Oracle Spatial, of
course I choose PostGIS", the pgaudit extension does not have that.


True enough but so what? At some point, someone has to use it. Just 
because it doesn't have 10 years of experience doesn't mean we should 
shove it into core. Those that need it, will use it. My customers (for 
example) use what I tell them to use.



Auditing is a pretty security/enterprisey-related thing that could do
with the "officially considered to of the PostgreSQL project standard
and ready for production" rubber-stamp that tends to go along with most
end-user/admin-oriented stuff shipped in the tarball.


Which is exactly why I think .Org needs an official "Extensions" project 
which would completely eliminate these arguments. A project team 
explicitly for vetting extensions.




I am aware that
2nd Quadrant, Crunchy Data and EnterpriseDB (different codebase via
PPAS) all support their auditing extensions commercially, so that there
is certainly some form of credibility, but still.


Meh, commercial solutions aren't a consideration here. This is 
PostgreSQL not EDB or Crunchy.


Sincerely,

JD

--
Command Prompt, Inc.  http://the.postgres.company/
+1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PostgreSQL Auditing

[Joshua D. Drake]

On 02/02/2016 07:31 AM, curtis.r...@gmail.com wrote:

Its not available in rpm or premade binary forms in its current instantiation, 
not a big deal for me, but it raises the barrier to entry.

If it was packaged into an RPM, what would be the process to get it added to 
postgresql's yum repositories?



Assuming it is not placed into core, we would need to work with the deb 
and rpm projects. Which I am sure we could (and would) do.


Sincerely,
JD


--
Command Prompt, Inc.  http://the.postgres.company/
+1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PostgreSQL Auditing

[Michael Banck]

On Tue, Feb 02, 2016 at 07:24:23AM -0800, Joshua D. Drake wrote:
> PostgreSQL has auditing. It is available now, just not in core. Postgis
> isn't available in core either and it seems to do just fine.

I don't really buy that argument. For one, PostGIS has a pretty narrow
functional use-case (spatial), while auditing is a horizontal use-case
that could be required for any kind of database usage.

Second, PostGIS had 10+ (?) years to build a reputation so that people
say "if I have to choose between PostGIS and buying Oracle Spatial, of
course I choose PostGIS", the pgaudit extension does not have that.

Auditing is a pretty security/enterprisey-related thing that could do
with the "officially considered to of the PostgreSQL project standard
and ready for production" rubber-stamp that tends to go along with most
end-user/admin-oriented stuff shipped in the tarball.  I am aware that
2nd Quadrant, Crunchy Data and EnterpriseDB (different codebase via
PPAS) all support their auditing extensions commercially, so that there
is certainly some form of credibility, but still.

Now, whether or not the currently submitted approach actually meets the
above rubber-stamp requirements is a different story, but at least I
think it would be quite useful to ship auditing capabilites in the
distribution.


Michael

-- 
Michael Banck
Projektleiter / Berater
Tel.: +49 (2161) 4643-171
Fax:  +49 (2161) 4643-100
Email: michael.ba...@credativ.de

credativ GmbH, HRB Mönchengladbach 12080
USt-ID-Nummer: DE204566209
Hohenzollernstr. 133, 41061 Mönchengladbach
Geschäftsführung: Dr. Michael Meskes, Jörg Folz, Sascha Heuer


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PostgreSQL Auditing

[Joshua D. Drake]

On 02/02/2016 02:47 AM, José Luis Tallón wrote:

On 02/02/2016 02:05 AM, Curtis Ruck wrote:

[snip]

P.S., do you know what sucks, having a highly performant PostGIS
database that works great, and being told to move to Oracle or SQL
Server (because they have auditing).  Even though they charge extra
for Geospatial support (seriously?) or when they don't even have
geospatial support (10 years ago).  My customer would prefer to
re-engineer software designed around PostgreSQL and pay the overpriced
licenses, than not have auditing.  I agree that their cost analysis is
probably way off, even 10 years later, my only solution would be to
move to Oracle, SQL Server, a NoSQL solution, or pay EnterpriseDB for
their 2 year old version that doesn't have all the cool/modern jsonb
support.


PostgreSQL has auditing. It is available now, just not in core. Postgis 
isn't available in core either and it seems to do just fine.


JD

--
Command Prompt, Inc.  http://the.postgres.company/
+1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PostgreSQL Auditing

[Simon Riggs]

On 2 February 2016 at 02:05, Curtis Ruck <
curtis.ruck+pgsql.hack...@gmail.com> wrote:


> Just because auditing isn't sexy sharding, parallel partitioning, creative
> indexing (BRIN), or hundreds of thousands of transactions a second, doesn't
> make it any less of a requirement to countless organizations that would
> like to use postgresql, but find the audit requirement a must have.
>
> So, in summary, what would it take to get the core PostgreSQL team to
> actually let auditing patches into the next version?
>

I appreciate your frustration, though I'd say you're making a few
conceptual leaps in what you've said. I can help with a few answers.

For example, 2ndQuadrant developed the original pgAudit extension and
currently provide commercial support for users. So whether this gets
included into core PostgreSQL or not, is not the gating factor on whether
commercial support is available for open source software.

Security is an important thing round here, which also means that we follow
a default-deny approach to new features. So it can take some time to
include new features in core. The process is the same whether its sexy or
not. I agree it can be frustrating at times though overall we maintain a
high throughput of new features into PostgreSQL.

The original version of PgAudit sat in the queue unreviewed for about 7
months, which was a huge factor in it not being accepted into 9.5. We are
very short of reviewers and detailed reviews are accepted from any source.
So yourself or a colleague could make a difference here and I encourage
people with specialist knowledge and passion to take part.

P.S., do you know what sucks, having a highly performant PostGIS database
> that works great, and being told to move to Oracle or SQL Server (because
> they have auditing).  Even though they charge extra for Geospatial support
> (seriously?) or when they don't even have geospatial support (10 years
> ago).  My customer would prefer to re-engineer software designed around
> PostgreSQL and pay the overpriced licenses, than not have auditing.  I
> agree that their cost analysis is probably way off, even 10 years later, my
> only solution would be to move to Oracle, SQL Server, a NoSQL solution, or
> pay EnterpriseDB for their 2 year old version that doesn't have all the
> cool/modern jsonb support.
>

I agree it sucks when other people make money and you don't. That limits
funds available to allocate people on tasks, even when we see them as
important. But there are many companies who would be willing to implement
solutions or extend open source code for you, allowing that problem to be
solved. We don't usually discuss that option here, since this is an
engineering list.

Since you've written the email here, I'd ask that you join our community
and use your knowledge and passion to make things happen.

-- 
Simon Riggshttp://www.2ndQuadrant.com/

PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] PostgreSQL Auditing

[José Luis Tallón]

On 02/02/2016 02:05 AM, Curtis Ruck wrote:

[snip]

P.S., do you know what sucks, having a highly performant PostGIS 
database that works great, and being told to move to Oracle or SQL 
Server (because they have auditing).  Even though they charge extra 
for Geospatial support (seriously?) or when they don't even have 
geospatial support (10 years ago).  My customer would prefer to 
re-engineer software designed around PostgreSQL and pay the overpriced 
licenses, than not have auditing.  I agree that their cost analysis is 
probably way off, even 10 years later, my only solution would be to 
move to Oracle, SQL Server, a NoSQL solution, or pay EnterpriseDB for 
their 2 year old version that doesn't have all the cool/modern jsonb 
support.


Huh?  PPAS 9.5.0.5 is already out there since at least last week; Before 
that PPAS 9.4.5.y or so was there ...

(Not affiliated with EDB, but precision is important)

I agree that auditing is a big selling point and frequently used... But 
it's got to be done "the Postgres way", and that takes time (and usually 
provides superior results).



Just my .02€


/ J.L.




--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PostgreSQL Auditing

[Dave Page]

On Tue, Feb 2, 2016 at 1:05 AM, Curtis Ruck
 wrote:
> or pay
> EnterpriseDB for their 2 year old version that doesn't have all the
> cool/modern jsonb support.

Just for the record, anyone who pays for our "2 year old version that
doesn't have all the cool/modern jsonb support" is also entitled to
use either our 9.4 or 9.5 versions which do have JSONB support. Our
new versions are typically released within a couple of months of
PostgreSQL, and in the case of 9.5, the gap was more like 3 weeks.

-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PostgreSQL Auditing

[Noah Misch]

On Tue, Feb 02, 2016 at 01:05:46AM +, Curtis Ruck wrote:
> So Auditing, it seems that some people want auditing (myself, David Steele,
> 2nd quadrant, and probably others).  I personally love postgresql, but
> until it can meet my annoying compliance requirements, I can't leverage it
> fully as my organization spends more time on meeting compliance, than
> actually doing development and engineering.

> So, in summary, what would it take to get the core PostgreSQL team to
> actually let auditing patches into the next version?

[PostgreSQL has a "core team" (pgsql-core@) that rarely rules on specific
features.  I'm interpreting "core PostgreSQL team" more broadly as "people who
determine whether to accept patches".]

Based on the last discussion, I expect it would take resolving these concerns:

1. Does PostgreSQL gain enough by having the extension in core instead of
   leaving the same extension in Github? (Tom)

2. Will the feature set stand the test of time as the one core auditing
   feature set, pleasing to most of the PostgreSQL users who seek auditing?
   It's fine if the initial patch has a subset of the eventual feature set,
   but it's bad if we later wish to undo UI decisions. (Tom, Heikki, Robert)

   Your own input would be especially helpful on this point.  Would you
   describe the compliance requirements you have faced lately?  References to
   external standards are fine, and lists of private requirements are also
   fine.  If you can annotate a requirements list with the pgaudit feature
   used to meet each requirement, that would be even more helpful.

3. Does the implementation do what it claims to do, correctly?  Any defects
   are likely to be security vulnerabilities, which amplifies the cost of
   fixing them after release. (Noah, Robert)

Points (1) and (2) relax considerably for narrow core changes to support an
external auditing system.  For example, David's errhidefromclient() proposal
faces a simpler journey, because it raises far fewer design questions.

> P.S., do you know what sucks, having a highly performant PostGIS database
> that works great, and being told to move to Oracle or SQL Server (because
> they have auditing).  Even though they charge extra for Geospatial support

In the same sense that PostgreSQL has geospatial support, PostgreSQL 9.3 and
later do have auditing.  Get it from https://github.com/2ndQuadrant/pgaudit,
just like you get PostGIS separately.  I realize some sites forbid extensions
like pgaudit and PostGIS, but this particular case you describe is fortunate
not to have that problem.

Thanks,
nm


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers