Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-08 Thread Noah Misch
On Sat, Aug 08, 2015 at 02:30:47AM +0200, Andres Freund wrote: On 2015-08-07 20:16:20 -0400, Noah Misch wrote: I agree that lock.h offers little to frontend code. Headers that the lockdefs.h patch made usable in the frontend, particularly genam.h and hash.h, are no better. It's not

[HACKERS] Assert in pg_stat_statements

2015-08-08 Thread Satoshi Nagayasu
Hi, I just found that pg_stat_statements causes assert when queryId is set by other module, which is loaded prior to pg_stat_statements in the shared_preload_libraries parameter. Theoretically, queryId in the shared memory could be set by other module by design. So, IMHO, pg_stat_statements

[HACKERS] statement_timeout affects query results fetching?

2015-08-08 Thread Shay Rojansky
Hi everyone, I'm seeing some strange behavior and wanted to confirm it. When executing a query that selects a long result set, if the code processing the results takes its time (i.e.g more than statement_timeout), a timeout occurs. My expectation was that statement_timeout only affects query

[HACKERS] [sqlsmith] ERROR: failed to build any %d-way joins

2015-08-08 Thread Andreas Seltenreich
Hi, there's a 1/1e6 chance that a sqlsmith query on the regression db of master (c124cef) fails with ERROR: failed to build any {4..8}-way joins They all appear to work fine on REL9_5_STABLE. sample query: select 1 from information_schema.collations as rel_60113935 inner join

Re: [HACKERS] [sqlsmith] ERROR: failed to build any %d-way joins

2015-08-08 Thread Andreas Seltenreich
I wrote: They all appear to work fine on REL9_5_STABLE. oops, that branch was slightly stale. Updating it with the latest planner changes (8dccf03..0853388), the queries fail there as well. regards, Andreas -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

[HACKERS] pg_filedump patch for 9.5

2015-08-08 Thread Satoshi Nagayasu
Hi, I have created a patch for pg_filedump to work with 9.5. Here is a list of changes. * Fix to rename CRC32 macros to work with 9.5. * Fix to add missing DBState: DB_SHUTDOWNED_IN_RECOVERY. * Fix to add missing page flags for Btree and GIN. * Update copyright date. Please take a look. Any

Re: [HACKERS] 9.5 release notes

2015-08-08 Thread Bruce Momjian
On Thu, Aug 6, 2015 at 10:35:50PM -0400, Bruce Momjian wrote: On Thu, Aug 6, 2015 at 06:42:38PM -0700, Peter Geoghegan wrote: On Thu, Aug 6, 2015 at 6:08 PM, Bruce Momjian br...@momjian.us wrote: I though tabout this, and it is really an issue for FDW authors, not for end users, so I

Re: [HACKERS] 9.5 release notes

2015-08-08 Thread David Rowley
On 7 August 2015 at 14:24, Bruce Momjian br...@momjian.us wrote: On Tue, Jun 30, 2015 at 09:00:44PM +0200, Andres Freund wrote: * 2014-12-08 [519b075] Simon ..: Use GetSystemTimeAsFileTime directly in win32 2014-12-08 [8001fe6] Simon ..: Windows: use GetSystemTimePreciseAsFileTime if ..

Re: [HACKERS] WIP: SCRAM authentication

2015-08-08 Thread Robert Haas
On Fri, Aug 7, 2015 at 6:54 PM, Michael Paquier michael.paqu...@gmail.com wrote: This filtering machinery definitely looks like a GUC to me, something like password_forbidden_encryption that PASSWORD VERIFIERS looks at and discards the methods listed in there. This definitely needs to be

Re: [HACKERS] tap tests remove working directories

2015-08-08 Thread Robert Haas
On Fri, Aug 7, 2015 at 7:17 PM, Andrew Dunstan and...@dunslane.net wrote: That certainly isn't what happens, and given the way this is done in TestLib.pm, using the CLEANUP parameter of File::Temp's tempdir() function, it's not clear how we could do that easily. shot-in-the-dark Set cleanup

Re: [HACKERS] statement_timeout affects query results fetching?

2015-08-08 Thread Robert Haas
On Sat, Aug 8, 2015 at 5:31 AM, Shay Rojansky r...@roji.org wrote: Hi everyone, I'm seeing some strange behavior and wanted to confirm it. When executing a query that selects a long result set, if the code processing the results takes its time (i.e.g more than statement_timeout), a timeout

Re: [HACKERS] [DOCS] max_worker_processes on the standby

2015-08-08 Thread Robert Haas
On Tue, Aug 4, 2015 at 6:13 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I think it's totally reasonable for the standby to follow the master's behavior rather than the config file. That should be documented, but otherwise, no problem. If it were technologically possible for the

Re: [HACKERS] Patch for ginCombineData

2015-08-08 Thread Robert Haas
On Wed, Aug 5, 2015 at 6:17 AM, Robert Abraham robert.abraha...@googlemail.com wrote: we are using gin indexes on big tables. these tables happen to have several billion rows. the index creation fails in ginCombineData in src/backend/access/ginbulk.c because repalloc is limited to 1 gb. this

Re: [HACKERS] Add column-name hint to log messages generated by inserts when varchars don't fit

2015-08-08 Thread Robert Haas
On Wed, Aug 5, 2015 at 6:39 AM, Stepan Rutz stepan.r...@gmx.de wrote: on our production servers I have quite some errors due to excessively long varchar-values which application-code tries to insert into tables and which don't fit. The errors look like ERROR: value too long for type

Re: [HACKERS] Dependency between bgw_notify_pid and bgw_flags

2015-08-08 Thread Robert Haas
On Wed, Aug 5, 2015 at 3:33 AM, Ashutosh Bapat ashutosh.ba...@enterprisedb.com wrote: This idea looks good. Thanks. It needs testing though to see if it really works as intended. Can you look into that? Looking at larger picture, we should also enable this feature to be used by auxilliary

Re: [HACKERS] [sqlsmith] subplan variable reference / unassigned NestLoopParams

2015-08-08 Thread Andreas Seltenreich
Tom Lane writes: Andreas Seltenreich seltenre...@gmx.de writes: Tom Lane writes: Well, I certainly think all of these represent bugs: 3 | ERROR: plan should not reference subplan's variable 2 | ERROR: failed to assign all NestLoopParams to plan nodes These appear to be related. The

Re: [HACKERS] WIP: SCRAM authentication

2015-08-08 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 08/08/2015 06:27 AM, Robert Haas wrote: terminology. I think we should store (1) your password, either encrypted or unencrypted; and (2) the method used to encrypt it. And that's it. A petty complaint, but it has always bothered me that we

Re: [HACKERS] WIP: SCRAM authentication

2015-08-08 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Fri, Aug 7, 2015 at 6:54 PM, Michael Paquier michael.paqu...@gmail.com wrote: This filtering machinery definitely looks like a GUC to me, something like password_forbidden_encryption that PASSWORD VERIFIERS looks at and discards the methods

Re: [HACKERS] statement_timeout affects query results fetching?

2015-08-08 Thread Tom Lane
Shay Rojansky r...@roji.org writes: Hi everyone, I'm seeing some strange behavior and wanted to confirm it. When executing a query that selects a long result set, if the code processing the results takes its time (i.e.g more than statement_timeout), a timeout occurs. My expectation was that

Re: [HACKERS] statement_timeout affects query results fetching?

2015-08-08 Thread Shay Rojansky
Thanks for your responses. I'm not using cursors or anything fancy. The expected behavior (as far as I can tell) for a .NET database driver is to read one row at a time from the database and make it available. There's even a standard API option for fetching data on a column-by-column basis: this

Re: [HACKERS] statement_timeout affects query results fetching?

2015-08-08 Thread Shay Rojansky
I'd also recommend adding a sentence about this aspect of statement_timeout in the docs to prevent confusion... On Sat, Aug 8, 2015 at 5:30 PM, Shay Rojansky r...@roji.org wrote: Thanks for your responses. I'm not using cursors or anything fancy. The expected behavior (as far as I can tell)

Re: [HACKERS] tap tests remove working directories

2015-08-08 Thread Andrew Dunstan
On 08/08/2015 09:31 AM, Robert Haas wrote: On Fri, Aug 7, 2015 at 7:17 PM, Andrew Dunstan and...@dunslane.net wrote: That certainly isn't what happens, and given the way this is done in TestLib.pm, using the CLEANUP parameter of File::Temp's tempdir() function, it's not clear how we could do

[HACKERS] Test code is worth the space

2015-08-08 Thread Noah Misch
We've too often criticized patches for carrying many lines/bytes of test case additions. Let's continue to demand debuggable, secure tests that fail only when something is wrong, but let's stop pushing for test minimalism. Such objections may improve the individual patch, but that doesn't make

Re: [HACKERS] WIP: SCRAM authentication

2015-08-08 Thread Heikki Linnakangas
On 08/08/2015 04:27 PM, Robert Haas wrote: I don't see that there's any good reason to allow the same password to be stored in the catalog encrypted more than one way, Sure there is. If you want to be able to authenticate using different mechanism, you need the same password encrypted in

Re: [HACKERS] checkpointer continuous flushing

2015-08-08 Thread Heikki Linnakangas
On 07/26/2015 06:01 PM, Fabien COELHO wrote: Attached is very minor v5 update which does a rebase completes the cleanup of doing a full sort instead of a chuncked sort. Some thoughts on this: * I think we should drop the flush part of this for now. It's not as clearly beneficial as the

Re: [HACKERS] 9.5 release notes

2015-08-08 Thread Bruce Momjian
On Fri, Aug 7, 2015 at 09:02:09PM +0200, Andres Freund wrote: With regard to the point about the number of buffer mappings: This has forced peoples sites down. People have found this out themselves and patched postgres. That's an entirely different league. I was not aware of the magnitude of

Re: [HACKERS] WIP: SCRAM authentication

2015-08-08 Thread Greg Stark
On Sat, Aug 8, 2015 at 6:23 PM, Heikki Linnakangas hlinn...@iki.fi wrote: Like Joe and Stephen, I actually find it highly confusing that we call the MD5 hash an encrypted password. The term password verifier is fairly common in the specifications of authentication mechanisms. I think we should

Re: [HACKERS] 9.5 release notes

2015-08-08 Thread Bruce Momjian
On Fri, Aug 7, 2015 at 09:02:09PM +0200, Andres Freund wrote: On 2015-08-07 14:43:11 -0400, Bruce Momjian wrote: Well, we could just throw a Postgres 9.5 is faster release note item in there and call it a day. ;-) Based on my experience one of the prime reason people move to a new

Re: [HACKERS] Test code is worth the space

2015-08-08 Thread Peter Geoghegan
On Sat, Aug 8, 2015 at 9:47 AM, Noah Misch n...@leadboat.com wrote: We've too often criticized patches for carrying many lines/bytes of test case additions. Let's continue to demand debuggable, secure tests that fail only when something is wrong, but let's stop pushing for test minimalism.

Re: [HACKERS] Test code is worth the space

2015-08-08 Thread Josh Berkus
On 08/08/2015 12:24 PM, Peter Geoghegan wrote: I think that there needs to be a way of running an extended set of regression tests. I could definitely respect the desire for minimalism when it comes to adding tests to the regression tests proper if there was an extended set of tests that could

Re: [HACKERS] Test code is worth the space

2015-08-08 Thread Andrew Dunstan
On 08/08/2015 05:09 PM, Josh Berkus wrote: On 08/08/2015 12:24 PM, Peter Geoghegan wrote: I think that there needs to be a way of running an extended set of regression tests. I could definitely respect the desire for minimalism when it comes to adding tests to the regression tests proper if

Re: [HACKERS] 9.5 release notes

2015-08-08 Thread Bruce Momjian
On Sat, Aug 8, 2015 at 02:49:21PM -0400, Bruce Momjian wrote: What I _am_ saying is that you should use the same criteria I am using, and just disagree on the place for the line, rather than use a different criteria, which will lead to perpetual complaints. We can change the

Re: [HACKERS] Test code is worth the space

2015-08-08 Thread Greg Stark
On Sat, Aug 8, 2015 at 8:24 PM, Peter Geoghegan p...@heroku.com wrote: I think that there needs to be a way of running an extended set of regression tests. I could definitely respect the desire for minimalism The larger expense in having extensive test suites is the cost to maintain them. With

Re: [HACKERS] WIP: SCRAM authentication

2015-08-08 Thread Josh Berkus
On 08/08/2015 10:23 AM, Heikki Linnakangas wrote: On 08/08/2015 04:27 PM, Robert Haas wrote: I don't see that there's any good reason to allow the same password to be stored in the catalog encrypted more than one way, Sure there is. If you want to be able to authenticate using different

Re: [HACKERS] WIP: SCRAM authentication

2015-08-08 Thread Michael Paquier
On Sun, Aug 9, 2015 at 6:51 AM, Josh Berkus j...@agliodbs.com wrote: Obviously the backwards-compatibility issues are pretty major ... it'll be years before all drivers support SCRAM ... but we also want to provide a path forwards for secure installations in which no md5 hashes are stored.

Re: [HACKERS] WIP: SCRAM authentication

2015-08-08 Thread Sehrope Sarkuni
It'd be nice if the new auth mechanism supports multiple passwords in the same format as well (not just one per format). That way you could have two different passwords for a user that are active at the same time. This would simplify rolling database credentials as it wouldn't have to be done all

Re: [HACKERS] WIP: Make timestamptz_out less slow.

2015-08-08 Thread David Rowley
On 29 July 2015 at 03:25, Andres Freund and...@anarazel.de wrote: On 2015-07-29 03:10:41 +1200, David Rowley wrote: timestamp_out() = 2015-07-29 02:24:33.34 in 3.506000 timestamp_out_old() = 2015-07-29 02:24:33.034 in 64.518000 timestamp_out_af() = 2015-07-29 02:24:33.034 in 2.981000

Re: [HACKERS] 9.5 release notes

2015-08-08 Thread Bruce Momjian
On Sun, Aug 9, 2015 at 01:24:33AM +1200, David Rowley wrote: On 7 August 2015 at 14:24, Bruce Momjian br...@momjian.us wrote: On Tue, Jun 30, 2015 at 09:00:44PM +0200, Andres Freund wrote: * 2014-12-08 [519b075] Simon ..: Use GetSystemTimeAsFileTime directly in win32    

[HACKERS] Broken src/test/regress/mb/* tests

2015-08-08 Thread Tatsuo Ishii
Commit: http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=9043ef390f4f0b4586cfe59cbd22314b9c3e2957 broke src/test/regress/mb/* tests because the message: WARNING: hash indexes are not WAL-logged and their use is discouraged does not emit any more in the test. I have committed and

Re: [HACKERS] WIP: SCRAM authentication

2015-08-08 Thread Josh Berkus
On 08/08/2015 03:21 PM, Michael Paquier wrote: On Sun, Aug 9, 2015 at 6:51 AM, Josh Berkus j...@agliodbs.com wrote: 1. we drop the parameter password_encryption 2. we add the parameter password_storage, which takes a list: - plain : plain text - md5 : current md5 hashes - scram :

Re: [HACKERS] Assert in pg_stat_statements

2015-08-08 Thread Satoshi Nagayasu
On 2015/08/08 22:32, Robert Haas wrote: On Sat, Aug 8, 2015 at 5:00 AM, Satoshi Nagayasu sn...@uptime.jp wrote: I just found that pg_stat_statements causes assert when queryId is set by other module, which is loaded prior to pg_stat_statements in the shared_preload_libraries parameter.