Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-09-28 Thread Kouhei Kaigai
> -Original Message- > From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Etsuro Fujita > Sent: Tuesday, September 29, 2015 12:15 PM > To: Kaigai Kouhei(海外 浩平); Robert Haas > Cc: PostgreSQL-development; 花田茂 > Subject: Re: [HACKERS] Foreign

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-28 Thread Tom Lane
Stephen Frost writes: > * Jim Nasby (jim.na...@bluetreble.com) wrote: >> 2 years ago is when they first released the enterprise edition, >> which according to [1] had "The most important new feature is that >> you can now add members to groups of projects." > It needed a lot

Re: [HACKERS] Comment update to pathnode.c

2015-09-28 Thread Etsuro Fujita
On 2015/09/12 4:26, Robert Haas wrote: On Fri, Sep 11, 2015 at 6:22 AM, Etsuro Fujita wrote: The comments for create_foreignscan_path says as follows, but that it's now possible that the function is called by GetForeignJoinPaths, which was added in 9.5. 1450 /*

Re: [HACKERS] Doubt in pgbench TPS number

2015-09-28 Thread Fabien COELHO
(Note: I've not read the patch, so this is not an opinion about its correctness.) As Fabien anayzed the problem was that pgbench simply uses wrong variable: nthreads (number of threads, specified by "-j" option) vs. nclients (number of clients, specified by "-c" option). @@ -2616,7 +2616,7

Re: [HACKERS] unclear about row-level security USING vs. CHECK

2015-09-28 Thread Charles Clavadetscher
Good morning > -Original Message- > From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Stephen Frost > Sent: Montag, 28. September 2015 21:16 > To: Peter Eisentraut ; Robert Haas > Cc: pgsql-hackers

Re: [HACKERS] Parallel Seq Scan

2015-09-28 Thread Amit Kapila
On Sun, Sep 27, 2015 at 1:39 AM, Robert Haas wrote: > > I intend to commit this patch (but not the crappy test code, of > course) pretty soon, and then I'm going to start working on the > portion of the patch that actually adds the Funnel node, which I think > you are

[HACKERS] ToDo: possibility to set sqlcode, detail, hint in elog like functions in plpython and plperl

2015-09-28 Thread Pavel Stehule
Hi now, isn't possible to raise rich exception from plpython or plperl. Should be fixed. Regards Pavel

Re: [HACKERS] row_security GUC, BYPASSRLS

2015-09-28 Thread Stephen Frost
* Noah Misch (n...@leadboat.com) wrote: > On Thu, Sep 10, 2015 at 03:23:13PM -0400, Stephen Frost wrote: > > First off, thanks again for your review and comments on RLS. Hopefully > > this addresses the last remaining open item from that review. > > Item (3a), too, remains open. Provided I

Re: [HACKERS] 9.5: Can't connect with PGSSLMODE=require on Windows

2015-09-28 Thread Asif Naeem
I have spent sometime to investigate the issue, it is reproduciable. In case of Windows, when pqsecure_raw_read() function error code WSAEWOULDBLOCK (EWOULDBLOCK) when no data queued to be read from the non blocking socket there is a need to log retry flag. Related error code can be retrieved via

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-28 Thread Shulgin, Oleksandr
On Mon, Sep 28, 2015 at 7:09 PM, Pavel Stehule wrote: > > 2015-09-28 12:37 GMT+02:00 Shulgin, Oleksandr < > oleksandr.shul...@zalando.de>: > >> >>> I didn't propose too different solution. There is only one difference - >>> sharing DSM for smaller data. It is similar to

Re: [HACKERS] optimizing vacuum truncation scans

2015-09-28 Thread Robert Haas
On Mon, Sep 28, 2015 at 2:05 AM, Haribabu Kommi wrote: > I got the following way to get the whether data file is present in the > DISK or SSD. > > 1. Get the device file system that table data file is mapped using the > following or similar. > > df -P "filename" | awk

Re: [HACKERS] track_commit_timestamp and COMMIT PREPARED

2015-09-28 Thread Petr Jelinek
On 2015-09-28 18:59, Robert Haas wrote: The patch looks good to me except the following minor points. * or not. Normal path through RecordTransactionCommit() will be related * to a transaction commit XLog record, and so should pass "false" here. The above source comment of

Re: [HACKERS] 9.3.9 and pg_multixact corruption

2015-09-28 Thread Josh Berkus
On 09/28/2015 08:10 AM, Robert Haas wrote: > -1 on that idea. I really don't think that we should categorically > decide we don't support higher optimization levels. If the compiler > has a bug, then the compiler manufacturer should fix it, and it's not > our fault. If the compiler doesn't have

Re: [HACKERS] unclear about row-level security USING vs. CHECK

2015-09-28 Thread Stephen Frost
* Peter Eisentraut (pete...@gmx.net) wrote: > I see. But it is a bit odd to hide this very fundamental behavior > somewhere in a paragraph that starts out with something about roles. I'm happy to change that. You're right, it should be a paragraph by itself. > There is also a mistake, I

Re: [HACKERS] unclear about row-level security USING vs. CHECK

2015-09-28 Thread Robert Haas
On Mon, Sep 28, 2015 at 3:15 PM, Stephen Frost wrote: > I listed out the various alternatives but didn't end up getting any > responses to it. I'm still of the opinion that the documentation is the > main thing which needs improving here, but we can also change CREATE >

Re: [HACKERS] Rename withCheckOptions to insertedCheckClauses

2015-09-28 Thread Stephen Frost
* Dean Rasheed (dean.a.rash...@gmail.com) wrote: > None of this renaming seems like an improvement to me. I have to admit that I'm not entirely sure it's improving things either. Certainly, we shouldn't be dumping out the withCheckOptions node and perhaps we should move its place in Query and

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-28 Thread Pavel Stehule
2015-09-28 12:37 GMT+02:00 Shulgin, Oleksandr : > On Mon, Sep 28, 2015 at 12:05 PM, Pavel Stehule > wrote: > >> >> 2015-09-28 12:01 GMT+02:00 Shulgin, Oleksandr < >> oleksandr.shul...@zalando.de>: >> >>> On Sun, Sep 27, 2015 at 8:05 AM,

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-28 Thread Pavel Stehule
2015-09-28 19:13 GMT+02:00 Shulgin, Oleksandr : > On Mon, Sep 28, 2015 at 7:09 PM, Pavel Stehule > wrote: > >> >> 2015-09-28 12:37 GMT+02:00 Shulgin, Oleksandr < >> oleksandr.shul...@zalando.de>: >> >>> I didn't propose too different

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-28 Thread Andres Freund
On 2015-09-28 09:41:18 -0700, David Fetter wrote: > Since you're convinced that this is an unqualified win, please put > together a project plan for switching from our current system to > Github. Err, no. That's a waste of all our time. It has been stated pretty clearly in this thread by a

Re: [HACKERS] COPY planning

2015-09-28 Thread Tom Lane
Alvaro Herrera writes: > Yup. However I notice that there are a few other callers of > expression_planner() that do not involve the optimizer for anything > else. Maybe it makes sense to have a separate header file that's just > #include "nodes/primnodes.h" > extern

Re: [HACKERS] track_commit_timestamp and COMMIT PREPARED

2015-09-28 Thread Robert Haas
On Mon, Sep 28, 2015 at 2:07 PM, Petr Jelinek wrote: > Sorry, missed your reply. To be clear, that was actually Fujii Masao's reply, not mine. I hope he can have a look at this version. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL

Re: [HACKERS] 9.3.9 and pg_multixact corruption

2015-09-28 Thread Robert Haas
On Mon, Sep 28, 2015 at 2:34 PM, Josh Berkus wrote: > On 09/28/2015 08:10 AM, Robert Haas wrote: >> -1 on that idea. I really don't think that we should categorically >> decide we don't support higher optimization levels. If the compiler >> has a bug, then the compiler

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-28 Thread David Fetter
On Mon, Sep 28, 2015 at 07:14:40PM +0300, YUriy Zhuravlev wrote: > On Monday 28 September 2015 08:23:46 David Fetter wrote: > > They may well be, but until we decide it's worth the switching > > costs to move to a totally different way of doing things, that > > system will stay in place. > Until

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-28 Thread Joshua D. Drake
On 09/28/2015 05:50 AM, YUriy Zhuravlev wrote: On Thursday 24 September 2015 12:10:07 Ryan Pedela wrote: Kam Lasater wrote: I'd suggest: Github Issues, Pivotal Tracker or Redmine (probably in that order). There are tens to hundreds of other great ones out there, I'm sure one of them would also

Re: [HACKERS] COPY planning

2015-09-28 Thread Alvaro Herrera
Stephen Frost wrote: > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > Alvaro Herrera writes: > > > I noticed that COPY calls planner() (this was introduced in 85188ab88). > > > I think it should be calling pg_plan_query() instead. > > > > +1 --- AFAICS, this is the *only*

Re: [HACKERS] track_commit_timestamp and COMMIT PREPARED

2015-09-28 Thread Robert Haas
On Fri, Sep 18, 2015 at 12:53 AM, Fujii Masao wrote: > On Sat, Sep 5, 2015 at 7:48 PM, Petr Jelinek wrote: >> On 2015-09-02 16:14, Fujii Masao wrote: >>> >>> On Wed, Aug 5, 2015 at 2:16 AM, Robert Haas wrote: On Mon,

Re: [HACKERS] RLS open items are vague and unactionable

2015-09-28 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Tue, Sep 15, 2015 at 10:22 AM, Stephen Frost wrote: > > Unless there are other concerns or issues raised, I'll push this later > > today. > > So does this mean that the first RLS open item is addressed? If so, > can it be

Re: [HACKERS] 9.5: Can't connect with PGSSLMODE=require on Windows

2015-09-28 Thread Alvaro Herrera
Asif Naeem wrote: > I have spent sometime to investigate the issue, it is reproduciable. In > case of Windows, when pqsecure_raw_read() function error code > WSAEWOULDBLOCK (EWOULDBLOCK) when no data queued to be read from the non > blocking socket there is a need to log retry flag. Related error

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-09-28 Thread Robert Haas
On Mon, Sep 28, 2015 at 3:34 AM, Kouhei Kaigai wrote: > The attached patch allows FDW driver to handle EPQ recheck by its own > preferable way, even if it is alternative local join or ExecQual to > the expression being pushed down. Thanks. I was all set to commit this, or

Re: [HACKERS] 9.5: Can't connect with PGSSLMODE=require on Windows

2015-09-28 Thread Tom Lane
Asif Naeem writes: > I have spent sometime to investigate the issue, it is reproduciable. In > case of Windows, when pqsecure_raw_read() function error code > WSAEWOULDBLOCK (EWOULDBLOCK) when no data queued to be read from the non > blocking socket there is a need to log

Re: [HACKERS] 9.5: Can't connect with PGSSLMODE=require on Windows

2015-09-28 Thread Tom Lane
I wrote: > ... and the reassignment to errno at the > bottom of my_sock_read needs to be SOCK_ERRNO_SET(); and why doesn't > my_sock_write have a reassignment at all? Comparison to the backend versions of these routines, which have been through quite a few releases, suggests that the reassignment

Re: [HACKERS] 9.5: Can't connect with PGSSLMODE=require on Windows

2015-09-28 Thread Andres Freund
On 2015-09-28 17:28:48 -0400, Tom Lane wrote: > > What I do find curious is that afaics before 680513ab79 the code also > > looked at errno, not SOCK_ERRNO. And apparently things worked back then? > > No; AFAICS, before that commit, libpq did not use a custom BIO at all. > That commit cloned the

Re: [HACKERS] 9.5: Can't connect with PGSSLMODE=require on Windows

2015-09-28 Thread Tom Lane
Andres Freund writes: > We now probably could remove > * XXX OpenSSL 1.0.1e considers many more errcodes than just EINTR as reasons > * to retry; do we need to adopt their logic for that? > since we now actually check for more tahn just EINTR. Well, that comment is cloned

Re: [HACKERS] row_security GUC, BYPASSRLS

2015-09-28 Thread Stephen Frost
* Noah Misch (n...@leadboat.com) wrote: > On Tue, Sep 22, 2015 at 10:38:53AM -0400, Stephen Frost wrote: > > This would lead to trivial to cause (and likely hard to diagnose) > > referential integrity data corruption issues. I find that a very hard > > pill to swallow in favor of a theoretical

Re: [HACKERS] Rework the way multixact truncations work

2015-09-28 Thread Jim Nasby
On 9/27/15 2:25 PM, Andres Freund wrote: On 2015-09-27 14:21:08 -0500, Jim Nasby wrote: IMHO doing just a log of something this serious; it should at least be a WARNING. In postgres LOG, somewhat confusingly, is more severe than WARNING. Ahh, right. Which in this case stinks, because

[HACKERS] Has anyone run Cachegrind against the code?

2015-09-28 Thread Jim Nasby
Has anyone ever run cachegrind [1] against Postgres? I see little about it on the mailing list so I'm guessing no... [1] http://valgrind.org/docs/manual/cg-manual.html -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in

Re: [HACKERS] 9.5: Can't connect with PGSSLMODE=require on Windows

2015-09-28 Thread Tom Lane
Andres Freund writes: > On 2015-09-28 16:57:24 -0400, Tom Lane wrote: >> Great detective work! But if that's broken, then surely the identical >> code in my_sock_write is as well; and the reassignment to errno at the >> bottom of my_sock_read needs to be SOCK_ERRNO_SET(); and

Re: [HACKERS] Rename withCheckOptions to insertedCheckClauses

2015-09-28 Thread Tom Lane
Stephen Frost writes: > * Dean Rasheed (dean.a.rash...@gmail.com) wrote: >> Also, these were added in 9.4, so introducing this many differences >> between 9.4 and 9.5+ will make back-patching harder. > That's certainly true, but we don't want current or future hackers to be >

Re: [HACKERS] 9.5: Can't connect with PGSSLMODE=require on Windows

2015-09-28 Thread Andres Freund
On 2015-09-28 16:57:24 -0400, Tom Lane wrote: > Asif Naeem writes: > > I have spent sometime to investigate the issue, it is reproduciable. In > > case of Windows, when pqsecure_raw_read() function error code > > WSAEWOULDBLOCK (EWOULDBLOCK) when no data queued to be read

Re: [HACKERS] PGXS "check" target forcing an install ?

2015-09-28 Thread Robert Haas
On Thu, Sep 24, 2015 at 4:21 PM, Noah Misch wrote: > On Fri, Jun 26, 2015 at 09:09:15AM -0400, Robert Haas wrote: >> On Tue, Jun 23, 2015 at 1:31 AM, Michael Paquier >> wrote: >> >> I tracked the dangerous -rf to come from Makefile.global and it's

Re: [HACKERS] RLS open items are vague and unactionable

2015-09-28 Thread Robert Haas
On Tue, Sep 15, 2015 at 10:22 AM, Stephen Frost wrote: > Dean, > > * Dean Rasheed (dean.a.rash...@gmail.com) wrote: >> A minor point -- this comment isn't quite right: > > Fixed. > >> because the policies that are fetched there are only used for >> add_security_quals(), not

Re: [HACKERS] BRIN Scan: Optimize memory allocation in function 'bringetbitmap'

2015-09-28 Thread Simon Riggs
On 27 September 2015 at 02:15, Jinyu Zhang wrote: > > BRIN Scan: Optimize memory allocation in function 'bringetbitmap'. > We can allocate memory for some pointer before do long loop instead of > allocating > memory in long loop. > > Before optimizing code (warm run) >

Re: [HACKERS] Tab completion for ALTER COLUMN SET STATISTICS

2015-09-28 Thread Robert Haas
On Sat, Sep 26, 2015 at 7:24 AM, Michael Paquier wrote: > On Sat, Sep 26, 2015 at 7:18 AM, Jeff Janes wrote: >> If I have "alter table foo alter COLUMN bar SET STATISTICS" in the line >> buffer, >> it tab completes to add " TO", which is not

Re: [HACKERS] unclear about row-level security USING vs. CHECK

2015-09-28 Thread Robert Haas
On Sat, Sep 26, 2015 at 9:46 PM, Peter Eisentraut wrote: > On 9/23/15 3:41 PM, Stephen Frost wrote: > I see. But it is a bit odd to hide this very fundamental behavior > somewhere in a paragraph that starts out with something about roles. > > There is also a mistake, I believe:

Re: [HACKERS] Doubt in pgbench TPS number

2015-09-28 Thread Tom Lane
Fabien COELHO writes: >> Yeah, that's definitely a bug but I'm afraid the fix will change the >> TPS number and may break the backward compatibility. Since we have >> lived with bug for years, I hesitate to back port to older stable >> branches... > My 2¥: I do not think of

Re: [HACKERS] BRIN indexes for MAX, MIN, ORDER BY?

2015-09-28 Thread Marti Raudsepp
Hi Gavin Note that Alexander Korotkov already started work in 2013 on a somewhat similar feature called partial sort: http://www.postgresql.org/message-id/CAPpHfdscOX5an71nHd8WSUH6GNOCf=V7wgDaTXdDd9=gon-...@mail.gmail.com In particular, see the 2nd patch for KNN sort -- it uses known bounding

Re: [HACKERS] pgbench stats per script & other stuff

2015-09-28 Thread Robert Haas
On Sat, Sep 26, 2015 at 3:27 AM, Fabien COELHO wrote: > Here is a v10, which is a rebase because of the "--progress-timestamp" > option addition. I do not see it attached. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent

Re: [HACKERS] Manual bitswizzling -> LOCKBIT_ON

2015-09-28 Thread Robert Haas
On Thu, Sep 24, 2015 at 10:24 PM, Thomas Munro wrote: > While studying lmgr code, I noticed that there are a couple of places > that use 1 << x to convert a LOCKMODE to a LOCKMASK instead of the > macro that is used elsewhere. Should that be changed for

Re: [HACKERS] CustomScan support on readfuncs.c

2015-09-28 Thread Robert Haas
On Thu, Sep 24, 2015 at 9:19 PM, Kouhei Kaigai wrote: > Then, let's look back a bit. Next issue is how to reproduce > the "methods" pointer from the text representation. > I try to lookup the methods table using a pair of library > and symbol name; probably, it is a

Re: [HACKERS] GIN vacuum bug

2015-09-28 Thread Robert Haas
On Thu, Sep 24, 2015 at 11:47 AM, Teodor Sigaev wrote: > Hi! > > After reading thread > http://www.postgresql.org/message-id/flat/CAMkU=1xalflhuuohfp5v33rzedlvb5aknnujceum9knbkrb...@mail.gmail.com#CAMkU=1xalflhuuohfp5v33rzedlvb5aknnujceum9knbkrb...@mail.gmail.com > > I agree,

Re: [HACKERS] Doubt in pgbench TPS number

2015-09-28 Thread Fabien COELHO
Hello Tom, FWIW, I vote with Tatsuo-san. Such a change will break comparability of results I would not classify a performance measure as a "result compatibility" issue. What matters is the *correction* of the results. When a bug is fixed anywhere in pg it may change performance

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-09-28 Thread Robert Haas
On Thu, Sep 24, 2015 at 8:37 AM, Masahiko Sawada wrote: > When we run "VACUUM;", the all tables of current database will be vacuumed. > So pg_stat_vacuum_progress should have these oid in order to show > which table is vacuumed now. Hmm, I would tend to instead show the

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-28 Thread David Fetter
On Mon, Sep 28, 2015 at 03:50:29PM +0300, YUriy Zhuravlev wrote: > On Thursday 24 September 2015 12:10:07 Ryan Pedela wrote: > > Kam Lasater wrote: > > > I'd suggest: Github Issues, Pivotal Tracker or Redmine (probably > > > in that order). There are tens to hundreds of other great ones > > > out

Re: [HACKERS] BRIN indexes for MAX, MIN, ORDER BY?

2015-09-28 Thread Heikki Linnakangas
On 09/28/2015 05:28 PM, Marti Raudsepp wrote: Note that Alexander Korotkov already started work in 2013 on a somewhat similar feature called partial sort: http://www.postgresql.org/message-id/CAPpHfdscOX5an71nHd8WSUH6GNOCf=V7wgDaTXdDd9=gon-...@mail.gmail.com In particular, see the 2nd patch for

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-28 Thread Shulgin, Oleksandr
On Fri, Sep 25, 2015 at 7:13 PM, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > > Some implementation details: > > For every backend that might be running (MaxBackends) we reserve a > dsm_handle slot in the addins shared memory. When the new option is turned > on, the ExecutorRun

[HACKERS] Re: BRIN Scan: Optimize memory allocation in function 'bringetbitmap'

2015-09-28 Thread zhangjinyu
Sorry, I forgot attaching patch. But I have send path in another thread. Please see this thread "Patch: Optimize memory allocation in function 'bringetbitmap' ". -- View this message in context:

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-28 Thread YUriy Zhuravlev
On Monday 28 September 2015 08:23:46 David Fetter wrote: > They may well be, but until we decide it's worth the switching costs > to move to a totally different way of doing things, that system will > stay in place. Until we decide we're wasting time >Neither magic wands nor a green field project

Re: [HACKERS] pageinspect patch, for showing tuple data

2015-09-28 Thread Nikolay Shaplov
В письме от 26 сентября 2015 20:57:25 Вы написали: > >> Thanks! I just had a short look at it: > >> - I am not convinced that it is worth declaring 3 versions of > >> tuple_data_split. > > > > How which of them should we leave? > > The one with the most arguments. Now perhaps we could have as

Re: [HACKERS] pgbench stats per script & other stuff

2015-09-28 Thread Fabien COELHO
Here is a v10, which is a rebase because of the "--progress-timestamp" option addition. I do not see it attached. Indeed. Here it is. -- Fabien.diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml index 0ac40f1..e3a90e5 100644 --- a/doc/src/sgml/ref/pgbench.sgml +++

[HACKERS] [PATCH] Skip ALTER x SET SCHEMA if the schema didn't change

2015-09-28 Thread Marti Raudsepp
Hi list The attached patch changes the behavior of multiple ALTER x SET SCHEMA commands, to skip, rather than fail, when the old and new schema is the same. The advantage is that it's now easier to write DDL scripts that are indempotent. This already matches the behavior of ALTER EXTENSION SET

Re: [HACKERS] Patch: Optimize memory allocation in function 'bringetbitmap'

2015-09-28 Thread zhangjinyu
Yes, I forgot disable-c-assert last test. The following show the test result when disable-c-assert. I think it's still worthwhile. *After optimize code (warm run)* postgres=# select count(*) from lineitem where l_orderkey=1; count --- 6 (1 row) Time: 327.143 ms *Before optimizing

Re: [HACKERS] 9.3.9 and pg_multixact corruption

2015-09-28 Thread Robert Haas
On Fri, Sep 25, 2015 at 3:41 AM, Andreas Seltenreich wrote: > I think the intention was to make configure complain if there's a -O > 2 > in CFLAGS. -1 on that idea. I really don't think that we should categorically decide we don't support higher optimization

Re: [HACKERS] Partial Aggregation / GROUP BY before JOIN

2015-09-28 Thread Amit Langote
On 2015/09/28 13:31, David Rowley wrote: > I've been spending time working on allowing the planner to perform > aggregation before the final join relation is created. > ... > > The patch is however so far capable of giving us extremely nice performance > improvements for some (likely artificial)

Re: [HACKERS] Doubt in pgbench TPS number

2015-09-28 Thread Fabien COELHO
Hello Tatsuo-san, I think that the degree of parallelism to consider is nclients, not nthreads: while connection time is accumulated in conn_time, other clients are possibly doing their transactions, in parallel, I'm not sure about this. I think pgbench will not start transactions until all

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-09-28 Thread Kouhei Kaigai
> -Original Message- > From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Robert Haas > Sent: Saturday, September 12, 2015 1:39 AM > To: Etsuro Fujita > Cc: Kaigai Kouhei(海外 浩平); PostgreSQL-development; 花田茂 > Subject: Re: [HACKERS] Foreign

Re: [HACKERS] Doubt in pgbench TPS number

2015-09-28 Thread Tatsuo Ishii
> I have tested your patch. It seems the tolerance is much better than > before with your patch. [snip] > I'm going to commit your patch if there's no objection. I think we should commit this to master and 9.5 stable branch only because the fix significantly changes the benchmark result of

Re: [HACKERS] Doubt in pgbench TPS number

2015-09-28 Thread Tatsuo Ishii
>> I'm not sure about this. I think pgbench will not start transactions >> until all clients establish connections to PostgreSQL. > > I think that is true if "!is_connect", all client connections are > performed at the beginning of threadRun, but under -C each client has > its own

Re: [HACKERS] Partial Aggregation / GROUP BY before JOIN

2015-09-28 Thread David Rowley
On 28 September 2015 at 20:36, Amit Langote wrote: > > Did you perhaps attach a version of the patch you didn't intend to? > Oops. It seems so. Please find the correct version attached. Thanks for checking and letting me know. -- David Rowley

Re: [HACKERS] optimizing vacuum truncation scans

2015-09-28 Thread Haribabu Kommi
On Tue, Aug 4, 2015 at 2:18 AM, Jeff Janes wrote: > On Mon, Jul 27, 2015 at 1:40 PM, Simon Riggs wrote: >> >> On 22 July 2015 at 17:11, Jeff Janes wrote: >>> >>> On Wed, Jul 22, 2015 at 6:59 AM, Robert Haas

Re: [HACKERS] Doubt in pgbench TPS number

2015-09-28 Thread Tatsuo Ishii
> I think that the degree of parallelism to consider is nclients, not > nthreads: while connection time is accumulated in conn_time, other > clients are possibly doing their transactions, in parallel, I'm not sure about this. I think pgbench will not start transactions until all clients

Re: [HACKERS] Partial Aggregation / GROUP BY before JOIN

2015-09-28 Thread David Rowley
On 28 September 2015 at 23:17, Amit Langote wrote: > On 2015/09/28 17:04, David Rowley wrote: > > On 28 September 2015 at 20:36, Amit Langote < > langote_amit...@lab.ntt.co.jp> > > wrote: > > > >> > >> Did you perhaps attach a version of the patch you didn't intend

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-28 Thread Shulgin, Oleksandr
On Mon, Sep 28, 2015 at 12:05 PM, Pavel Stehule wrote: > > 2015-09-28 12:01 GMT+02:00 Shulgin, Oleksandr < > oleksandr.shul...@zalando.de>: > >> On Sun, Sep 27, 2015 at 8:05 AM, Pavel Stehule >> wrote: >> >> the preparing of content before

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-28 Thread Shulgin, Oleksandr
On Sun, Sep 27, 2015 at 8:05 AM, Pavel Stehule wrote: the preparing of content before execution is interesting idea, that can be > used more. The almost queries and plans are not too big, so when the size > of content is not too big - less than 1MB, then can be used one

Re: [HACKERS] Doubt in pgbench TPS number

2015-09-28 Thread Fabien COELHO
#5 0x00402b2b in doConnect () at pgbench.c:650 #6 0x00404591 in doCustom (thread=0x25c2f40, st=0x25c2770, conn_time=0x25c2f90, logfile=0x0, agg=0x7fffba224260) at pgbench.c:1353 #7 0x0040a1d5 in threadRun (arg=0x25c2f40) at pgbench.c:3581 #8 0x00409ab4 in

Re: [HACKERS] Doubt in pgbench TPS number

2015-09-28 Thread Tatsuo Ishii
>> Yeah, that's definitely a bug but I'm afraid the fix will change the >> TPS number and may break the backward compatibility. Since we have >> lived with bug for years, I hesitate to back port to older stable >> branches... > > My 2¥: I do not think of a good argument to keep wrong tps numbers

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-28 Thread Pavel Stehule
2015-09-28 12:01 GMT+02:00 Shulgin, Oleksandr : > On Sun, Sep 27, 2015 at 8:05 AM, Pavel Stehule > wrote: > > the preparing of content before execution is interesting idea, that can be >> used more. The almost queries and plans are not too

Re: [HACKERS] Partial Aggregation / GROUP BY before JOIN

2015-09-28 Thread Amit Langote
On 2015/09/28 17:04, David Rowley wrote: > On 28 September 2015 at 20:36, Amit Langote > wrote: > >> >> Did you perhaps attach a version of the patch you didn't intend to? >> > > Oops. It seems so. > > Please find the correct version attached. Thanks, this one

Re: [HACKERS] 9.5: Can't connect with PGSSLMODE=require on Windows

2015-09-28 Thread Tom Lane
Thom Brown writes: > With 9.5 alpha 2 on Windows 8 (64-bit), trying to require SSL results > in a blocking error: I've pushed a patch for this; can you verify it on Windows? regards, tom lane -- Sent via pgsql-hackers mailing list

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-28 Thread Jim Nasby
On 9/18/15 5:05 AM, Shulgin, Oleksandr wrote: It should not be true - the data sender create DSM and fills it. Then set caller slot and send signal to caller. Caller can free DSM any time, because data sender send newer touch it. But the requester can timeout on waiting for reply

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-28 Thread Tom Lane
Jim Nasby writes: > On 9/28/15 11:43 AM, Andres Freund wrote: >> It has been stated pretty clearly in this thread by a number of senior >> community people that we're not going to use a closed source system. > GitLab OTOH is released under a MIT license, so it is an

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-28 Thread Ryan Pedela
On Mon, Sep 28, 2015 at 4:09 PM, Jim Nasby wrote: > On 9/28/15 11:43 AM, Andres Freund wrote: > >> On 2015-09-28 09:41:18 -0700, David Fetter wrote: >> >>> Since you're convinced that this is an unqualified win, please put >>> together a project plan for switching from

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-28 Thread Joshua D. Drake
On 09/28/2015 03:40 PM, Alvaro Herrera wrote: Tom Lane wrote: Now, running gitlab on community-owned hardware would potentially be an option, if we find gitlab attractive from a functionality standpoint. The question I'd have about that is whether it has a real development community, or is

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-28 Thread Tom Lane
Josh Berkus writes: > The infra team seems to be good with debbugs, and several committers > seem to like it, why not go with it? It certainly seems like debbugs is the proposal to beat at this point. In the end though, what matters is somebody doing the dogwork to make it

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-28 Thread Jim Nasby
On 9/28/15 5:34 PM, Tom Lane wrote: Now, running gitlab on community-owned hardware would potentially be an option, if we find gitlab attractive from a functionality standpoint. The question I'd have about that is whether it has a real development community, or is open-source in name only. If

Re: [HACKERS] LISTEN denial of service with aborted transaction

2015-09-28 Thread Tom Lane
Matt Newell writes: > 1. When a connection issues it's first LISTEN command, in > Exec_ListenPreCommit > QUEUE_BACKEND_POS(MyBackendId) = QUEUE_TAIL; > this causes the connection to iterate through every notify queued in the > slru, > even though at that point I

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-28 Thread Jim Nasby
On 9/28/15 6:06 PM, Tom Lane wrote: Josh Berkus writes: The infra team seems to be good with debbugs, and several committers seem to like it, why not go with it? It certainly seems like debbugs is the proposal to beat at this point. In the end though, what matters is

[HACKERS] LISTEN denial of service with aborted transaction

2015-09-28 Thread Matt Newell
This morning with our production database I began receiving reports of the database being "down". I checked the log and was surprised to see extremely long durations for a LISTEN that happens after each connection is made by our database library. This coincided with many(approx 600) new

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-28 Thread Gavin Flower
On 29/09/15 11:54, Joshua D. Drake wrote: On 09/28/2015 03:40 PM, Alvaro Herrera wrote: Tom Lane wrote: Now, running gitlab on community-owned hardware would potentially be an option, if we find gitlab attractive from a functionality standpoint. The question I'd have about that is whether it

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-28 Thread Jim Nasby
On 9/28/15 11:43 AM, Andres Freund wrote: On 2015-09-28 09:41:18 -0700, David Fetter wrote: Since you're convinced that this is an unqualified win, please put together a project plan for switching from our current system to Github. Err, no. That's a waste of all our time. It has been stated

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-28 Thread Alvaro Herrera
Tom Lane wrote: > Now, running gitlab on community-owned hardware would potentially be an > option, if we find gitlab attractive from a functionality standpoint. > The question I'd have about that is whether it has a real development > community, or is open-source in name only. If github did go

Re: [HACKERS] track_commit_timestamp and COMMIT PREPARED

2015-09-28 Thread Alvaro Herrera
Fujii Masao wrote: > +if (replorigin_sesssion_origin == InvalidRepOriginId || > > This is not the problem of the patch, but I started wondering what > "sesssion" in the variable name "replorigin_sesssion_origin" means. > Is it just a typo and it should be "session"? Or it's the abbreviation

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-28 Thread Josh Berkus
On 09/28/2015 03:40 PM, Alvaro Herrera wrote: > Tom Lane wrote: > >> Now, running gitlab on community-owned hardware would potentially be an >> option, if we find gitlab attractive from a functionality standpoint. >> The question I'd have about that is whether it has a real development >>

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-28 Thread Joshua D. Drake
On 09/28/2015 04:08 PM, Gavin Flower wrote: JD Linux kernel project uses bugzilla (https://bugzilla.kernel.org) and so does LibreOffice (https://bugs.documentfoundation.org) I think they are both fairly big projects in for the long haul. I am not anti-bugzilla, just not all that familiar

Re: [HACKERS] Rework the way multixact truncations work

2015-09-28 Thread Robert Haas
On Mon, Sep 28, 2015 at 5:47 PM, Jim Nasby wrote: > There was discussion about making this a PANIC instead of a LOG, which I > think is a good idea... but then there'd need to be some way to not PANIC if > you were doing an upgrade. I think you're worrying about a

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-28 Thread Stephen Frost
* Ryan Pedela (rped...@datalanche.com) wrote: > I haven't used Gitlab extensively, but it has a feature set similar to > Github and then some [1]. The OSS project does seem active [2], but it is > still relatively new. I've set it up and used it for a relatively small environment and was *not*

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-28 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Josh Berkus writes: > > The infra team seems to be good with debbugs, and several committers > > seem to like it, why not go with it? > > It certainly seems like debbugs is the proposal to beat at this point. Agreed. > In the end

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-28 Thread Stephen Frost
JD, * Joshua D. Drake (j...@commandprompt.com) wrote: > On 09/28/2015 03:40 PM, Alvaro Herrera wrote: > >We already made a similar choice some years ago when we started > >depending on the then-recently open sourced SourceForge code for > >pgFoundry. That didn't turn out all that well in the

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-09-28 Thread Kouhei Kaigai
> -Original Message- > From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Robert Haas > Sent: Tuesday, September 29, 2015 5:46 AM > To: Kaigai Kouhei(海外 浩平) > Cc: Etsuro Fujita; PostgreSQL-development; 花田茂 > Subject: Re: [HACKERS] Foreign

Re: [HACKERS] Partial Aggregation / GROUP BY before JOIN

2015-09-28 Thread Amit Langote
On 2015/09/28 20:58, David Rowley wrote: > On 28 September 2015 at 23:17, Amit Langote > wrote: >> Moreover, would partial aggregation work below Append? >> > > Do you mean for cases like: > > create table a as select x.x a from generate_series(1,100) x(x); >

Re: [HACKERS] CustomScan support on readfuncs.c

2015-09-28 Thread Kouhei Kaigai
> -Original Message- > From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Robert Haas > Sent: Tuesday, September 29, 2015 12:08 AM > To: Kaigai Kouhei(海外 浩平) > Cc: Amit Kapila; Andres Freund; pgsql-hackers > Subject: Re: [HACKERS] CustomScan

  1   2   >