Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2015-07-23 Thread Laurent Laborde
Le 23 juil. 2015 19:27, "Alvaro Herrera" a écrit : > > Laurent Laborde wrote: > > > Friendly greetings ! > > > > What's the status of parallel clusterdb please ? > > I'm having fun (and troubles) applying the vacuumdb patch to clusterdb. > > > > This thread also talk about unifying code for parall

Re: [HACKERS] Improving replay of XLOG_BTREE_VACUUM records

2015-07-23 Thread Heikki Linnakangas
On 05/02/2015 02:10 AM, Jim Nasby wrote: This looks like a good way to address this until the more significant work can be done. I'm not a fan of "RBM_ZERO_NO_BM_VALID"; how about RBM_ZERO_BM_INVALID? or BM_NOT_VALID? Or maybe I'm just trying to impose too much English on the code; I see the log

[HACKERS] pg_dump -Fd and compression level

2015-07-23 Thread Marc Mamin
Hello, After our last upgrade, we've noticed a 10-20% size increase of our dump size. This comes from our backup scripts were pg_dump was called without setting -Z So it seems, that this fix did modify the default compression to use: http://michael.otacoo.com/postgresql-2/pg_dump-directory-format

Re: [HACKERS] WAL logging problem in 9.4.3?

2015-07-23 Thread Heikki Linnakangas
On 07/23/2015 09:38 PM, Robert Haas wrote: On Wed, Jul 22, 2015 at 12:21 PM, Heikki Linnakangas wrote: This is more invasive than I'd like to backpatch, but I think it's the simplest approach that works, and doesn't disable any of the important optimizations we have. Hmm, isn't HeapNeedsWAL(

Re: [HACKERS] Asynchronous execution on FDW

2015-07-23 Thread Kyotaro HORIGUCHI
Hello, At Thu, 23 Jul 2015 09:38:39 +, Kouhei Kaigai wrote in <9a28c8860f777e439aa12e8aea7694f80111b...@bpxm15gp.gisp.nec.co.jp> > I expected workloads like single shot scan on a partitioned large > fact table on DWH system. Yep, if workload is expected to rescan > so frequently, its expecte

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-07-23 Thread Fabien COELHO
Hello Kyotaro-san, If you feel that this feature only deserve a lexer solution, then the patch should be "returned with feedback". It's unfortunate to abandon this idea so I tried this and made it run with psql's parser. I think it works as expected. Wow, you are much more courageous than

Re: [HACKERS] Autonomous Transaction is back

2015-07-23 Thread Ashutosh Bapat
On Thu, Jul 23, 2015 at 11:01 AM, Rajeev rastogi wrote: > After few failed attempt to propose Autonomous transaction earlier. I > along with Simon Riggs would like to propose again but completely different > in approach. > > > > We also had discussion about this feature in last PGCon2015 Unconfe

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-07-23 Thread Amit Kapila
On Fri, Jul 24, 2015 at 12:31 AM, Ildus Kurbangaliev < i.kurbangal...@postgrespro.ru> wrote: > Hello. > I’ve changed the previous patch. `group` field in LWLock is removed, so > the size of LWLock will not increase. > Instead of the `group` field I've created new tranches for LWLocks from > MainLW

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-07-23 Thread Kyotaro HORIGUCHI
Hi, > If you feel that this feature only deserve a lexer solution, then the > patch should be "returned with feedback". It's unfortunate to abandon this idea so I tried this and made it run with psql's parser. I think it works as expected. The attached files are as follwoing. - 0001-Prepare-fo

Re: [HACKERS] BRIN index and aborted transaction

2015-07-23 Thread Alvaro Herrera
Tatsuo Ishii wrote: > > Hm, well, I am not sure that we want to pay the overhead of > > re-summarization every time we prune a single tuple from a block range. > > That's going to make vacuum much slower, I assume (without measuring); > > many page ranges are going to be re-summarized without this

Re: [HACKERS] fdw_scan_tlist for foreign table scans breaks EPQ testing, doesn't it?

2015-07-23 Thread Kouhei Kaigai
> On Wed, Jul 22, 2015 at 8:24 PM, Kouhei Kaigai wrote: > > Indeed, this commit allows ForeignScan to have fdw_scan_tlist, even if > > scanrelid > 0, however, I'm uncertain about its reason/intention. > > Does it a preparation for the upcoming target-list-pushdown?? > > I guess Tom would have to

Re: [HACKERS] Proposing COPY .. WITH PERMISSIVE

2015-07-23 Thread dinesh kumar
On Thu, Jul 23, 2015 at 9:21 AM, Robert Haas wrote: > On Thu, Jul 23, 2015 at 12:19 PM, dinesh kumar > wrote: > > Sorry for my unclear description about the proposal. > > > > "WITH PERMISSIVE" is equal to our existing behavior. That is, chmod=644 > on > > the created files. > > > > If User don'

Re: [HACKERS] TABLESAMPLE patch is really in pretty sad shape

2015-07-23 Thread Petr Jelinek
On 2015-07-24 01:26, Petr Jelinek wrote: On 2015-07-24 00:39, Tom Lane wrote: I wrote: OK, so "InitSampleScan" for a function called at ExecInitSampleScan time (which we might as well make optional), and then we'll use BeginSampleScan for the function that gets the parameters. The restart/ReSc

Re: [HACKERS] TABLESAMPLE patch is really in pretty sad shape

2015-07-23 Thread Petr Jelinek
On 2015-07-24 00:39, Tom Lane wrote: I wrote: OK, so "InitSampleScan" for a function called at ExecInitSampleScan time (which we might as well make optional), and then we'll use BeginSampleScan for the function that gets the parameters. The restart/ReScan function goes away since BeginSampleSca

Re: [HACKERS] Planner debug views

2015-07-23 Thread Tatsuo Ishii
> Here is a proposal introducing some debugging infrastructure into the > core. The basic idea is to allow us to query the planner search space. > To so do, we can dump related information to csv files and use foreign > table to query them. So here I propose two foreign tables: Sounds like a great

Re: [HACKERS] BRIN index and aborted transaction

2015-07-23 Thread Tatsuo Ishii
> Hm, well, I am not sure that we want to pay the overhead of > re-summarization every time we prune a single tuple from a block range. > That's going to make vacuum much slower, I assume (without measuring); > many page ranges are going to be re-summarized without this actually > changing the rang

Re: [HACKERS] Doubt about AccessExclusiveLock in ALTER TABLE .. SET ( .. );

2015-07-23 Thread Fabrízio de Royes Mello
On Thu, Jul 2, 2015 at 2:03 PM, Simon Riggs wrote: > > The following review has been posted through the commitfest application: > make installcheck-world: not tested > Implements feature: not tested > Spec compliant: not tested > Documentation:not tested > > Looks func

Re: [HACKERS] All-zero page in GIN index causes assertion failure

2015-07-23 Thread Alvaro Herrera
Heikki Linnakangas wrote: > BRIN update is not quite right, however. brin_getinsertbuffer() can > initialize a page, but the caller might bail out without using the page and > WAL-logging the change. If that happens, the next update that uses the same > page will WAL-log the change but it will not

Re: [HACKERS] "A huge debt of gratitude" - Michael Stonebraker

2015-07-23 Thread Oleg Bartunov
Nice to hear you again Jolly ! On Wed, Jul 22, 2015 at 1:42 AM, Jolly Chen wrote: > Hey everyone, > > You have probably heard that Mike Stonebraker recently won the Turing > award. A recording of his award lecture is available at: > https://www.youtube.com/watch?v=BbGeKi6T6QI > > It is an enter

Re: [HACKERS] Speakers Wanted for pgDay Cuba

2015-07-23 Thread Oleg Bartunov
I'm probably going, but my company supports this trip. On Thu, Jul 23, 2015 at 10:35 PM, Josh Berkus wrote: > Contributors: > > We are looking for one or more PostgreSQL contributors to travel to > Havana, Cuba, to present at the second pgDay Cuba. In order to > encourage this young community,

Re: [HACKERS] raw output from copy

2015-07-23 Thread Dickson S. Guedes
2015-07-07 3:32 GMT-03:00 Pavel Stehule : > > Hi > > previous patch was broken, and buggy > > Here is new version with fixed upload and more tests > > The interesting is so I should not to modify interface or client - so it > should to work with any current driver with protocol support >= 3. Hi P

Re: [HACKERS] Autonomous Transaction is back

2015-07-23 Thread Josh Berkus
On 07/23/2015 12:40 PM, Robert Haas wrote: > That's a problem, but I think it is rather unfair to say that it has > anything to do with autonomous transactions. "Run a procedure without > needing to hold a snapshot" is a completely separate feature request > from "allow autonomous transactions", a

Re: [HACKERS] Restore-reliability mode

2015-07-23 Thread Alvaro Herrera
Noah Misch wrote: > - Add buildfarm members. This entails reporting any bugs that prevent an > initial passing run. Once you have a passing run, schedule regular runs. > Examples of useful additions: > - "./configure ac_cv_func_getopt_long=no, ac_cv_func_snprintf=no ..." to > enable al

Re: [HACKERS] Restore-reliability mode

2015-07-23 Thread Alvaro Herrera
Peter Geoghegan wrote: > On Sat, Jun 6, 2015 at 12:58 PM, Noah Misch wrote: > > - Call VALGRIND_MAKE_MEM_NOACCESS() on a shared buffer when its local pin > > count falls to zero. Under CLOBBER_FREED_MEMORY, wipe a shared buffer > > when its global pin count falls to zero. > > Did a pat

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-23 Thread Robert Haas
On Wed, Jul 22, 2015 at 11:28 AM, Jim Nasby wrote: > If we want to expose that level of detail, I think either JSON or arrays > would make more sense, so we're not stuck with a limited amount of info. > Perhaps DDL would be OK with the numbers you suggested, but > https://www.pgcon.org/2013/schedu

Re: [HACKERS] What is HeapScanDescData.rs_initblock good for?

2015-07-23 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Tom Lane wrote: > >> I'm inclined to let it call heap_setscanlimits only if not allow_sync. > > > It is possible for a partial range scan to join an existing herd of > > scans that happens to be processing that part of the table, in which > > case this

Re: [HACKERS] Autonomous Transaction is back

2015-07-23 Thread Robert Haas
On Thu, Jul 23, 2015 at 2:49 PM, Josh Berkus wrote: > On 07/23/2015 11:39 AM, Robert Haas wrote: >> On Thu, Jul 23, 2015 at 2:33 PM, Josh Berkus wrote: Requesting for everyone's opinion regarding this based on which we can proceed to enhance/tune/re-write our design. >>> >>> So, one of

[HACKERS] Speakers Wanted for pgDay Cuba

2015-07-23 Thread Josh Berkus
Contributors: We are looking for one or more PostgreSQL contributors to travel to Havana, Cuba, to present at the second pgDay Cuba. In order to encourage this young community, we will be funding[1] up to $3000 in total travel funds for one or two speakers. Information on the event is here: http

Re: [HACKERS] Planner debug views

2015-07-23 Thread Qingqing Zhou
On Tue, Jul 21, 2015 at 5:15 PM, Qingqing Zhou wrote: > Here is a proposal introducing some debugging infrastructure into the > core. The basic idea is to allow us to query the planner search space. > To so do, we can dump related information to csv files and use foreign > table to query them. FW

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-07-23 Thread Ildus Kurbangaliev
Hello. I’ve changed the previous patch. `group` field in LWLock is removed, so the size of LWLock will not increase. Instead of the `group` field I've created new tranches for LWLocks from MainLWLocksArray. This allowed to remove a loop from the previous version of the patch. Now the names for L

Re: [HACKERS] BRIN index and aborted transaction

2015-07-23 Thread Josh Berkus
On 07/23/2015 11:18 AM, Robert Haas wrote: > Cool. I'm not sure exactly what the right solution is either, but it > seems like the current situation could very well lead to degrading > index performance over time, with no way to put that right except to > rebuild the index completely. So it seems

Re: [HACKERS] Autonomous Transaction is back

2015-07-23 Thread Josh Berkus
On 07/23/2015 11:39 AM, Robert Haas wrote: > On Thu, Jul 23, 2015 at 2:33 PM, Josh Berkus wrote: >>> Requesting for everyone's opinion regarding this based on which we can >>> proceed to enhance/tune/re-write our design. >> >> So, one of the things which came up during the discussion was advancing

Re: [HACKERS] Eliminating CREATE INDEX comparator TID tie-breaker overhead

2015-07-23 Thread Peter Geoghegan
On Thu, Jul 23, 2015 at 11:28 AM, Robert Haas wrote: > If other people feel strongly about this issue, then they can weigh in > and we'll see where we end up. If they don't, then there's no > consensus to proceed with this, and we shouldn't *have* to spend a lot > of time on it. If no one weighs

Re: [HACKERS] Autonomous Transaction is back

2015-07-23 Thread Robert Haas
On Thu, Jul 23, 2015 at 2:33 PM, Josh Berkus wrote: >> Requesting for everyone's opinion regarding this based on which we can >> proceed to enhance/tune/re-write our design. > > So, one of the things which came up during the discussion was advancing > XMIN, which is not important to the audit logg

Re: [HACKERS] WAL logging problem in 9.4.3?

2015-07-23 Thread Robert Haas
On Wed, Jul 22, 2015 at 12:21 PM, Heikki Linnakangas wrote: > > This is more invasive than I'd like to backpatch, but I think it's the > simplest approach that works, and doesn't disable any of the important > optimizations we have. Hmm, isn't HeapNeedsWAL() a lot more costly than RelationNeedsWA

Re: [HACKERS] Autonomous Transaction is back

2015-07-23 Thread Josh Berkus
> Requesting for everyone's opinion regarding this based on which we can > proceed to enhance/tune/re-write our design. So, one of the things which came up during the discussion was advancing XMIN, which is not important to the audit logging use case, but is very important for the batch job us

Re: [HACKERS] Eliminating CREATE INDEX comparator TID tie-breaker overhead

2015-07-23 Thread Robert Haas
On Thu, Jul 23, 2015 at 2:19 PM, Peter Geoghegan wrote: > Our priorities are different, but mine are changing; I simply don't > want to spend a lot of time arguing with you about things like this. Good. If other people feel strongly about this issue, then they can weigh in and we'll see where we

Re: [HACKERS] Eliminating CREATE INDEX comparator TID tie-breaker overhead

2015-07-23 Thread Peter Geoghegan
On Thu, Jul 23, 2015 at 8:19 AM, Robert Haas wrote: > My priorities are different from yours. Your conclusion is basically > that it's OK to burden everyone who comes along and does future > development that may use the sorting code differently from the way > it's used now with dealing with this i

Re: [HACKERS] BRIN index and aborted transaction

2015-07-23 Thread Robert Haas
On Wed, Jul 22, 2015 at 3:20 PM, Alvaro Herrera wrote: > Hm, well, I am not sure that we want to pay the overhead of > re-summarization every time we prune a single tuple from a block range. > That's going to make vacuum much slower, I assume (without measuring); > many page ranges are going to be

Re: [HACKERS] A little RLS oversight?

2015-07-23 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Jul 22, 2015 at 5:17 PM, Dean Rasheed > wrote: > > There's another issue here though -- just adding filters to the > > pg_stats view won't prevent a determined user from seeing the contents > > of the underlying table. For that, the view needs to have the > > security

Re: [HACKERS] What is HeapScanDescData.rs_initblock good for?

2015-07-23 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> I'm inclined to let it call heap_setscanlimits only if not allow_sync. > It is possible for a partial range scan to join an existing herd of > scans that happens to be processing that part of the table, in which > case this wouldn't be sufficient. Howe

Re: [HACKERS] Retrieve the snapshot's LSN

2015-07-23 Thread Florent Guiliani
On Fri, Jul 17, 2015 at 6:53 PM, Robert Haas wrote: > On Fri, Jul 17, 2015 at 8:31 AM, Florent Guiliani wrote: >> A pg_export_snapshot_for_slot(...) would work very well. >> >> Let me explain the use case. You have many downstream systems that are >> replicated with logical decoding. Using a dedi

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2015-07-23 Thread Alvaro Herrera
Laurent Laborde wrote: > Friendly greetings ! > > What's the status of parallel clusterdb please ? > I'm having fun (and troubles) applying the vacuumdb patch to clusterdb. > > This thread also talk about unifying code for parallelizing clusterdb and > reindex. > Was anything done about it ? Bec

Re: [HACKERS] What is HeapScanDescData.rs_initblock good for?

2015-07-23 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Tom Lane wrote: > >> The BRIN patch added a HeapScanDescData field rs_initblock, but so far as > >> I can see it's utterly without use, and it's quite confusing (people might > >> mistake it for rs_startblock, for example). Any objection to taking it >

[HACKERS] Revoke [admin option for] role

2015-07-23 Thread Egor Rogov
Hi, I found an inconsistency between documentation and real behavior of REVOKE [ADMIN OPTION FOR] ROLE. As per documentation (http://www.postgresql.org/docs/current/static/sql-revoke.html): -- If GRANT OPTION FOR is specified, only the grant option for the privilege is revoked, not the priv

Re: [HACKERS] BRIN index and aborted transaction

2015-07-23 Thread Alvaro Herrera
Robert Haas wrote: > Maybe I'm confused here, but it seems like the only time > re-summarization can be needed is when tuples are pruned. The mere > act of deleting a tuple, even if the delete goes on to commit, doesn't > create a scenario where re-summarization can work out to a win, > because t

Re: [HACKERS] pgbench stats per script & other stuff

2015-07-23 Thread Alvaro Herrera
Fabien COELHO wrote: > > >>[...] and that a subsequent -w modifies the meaning of the > >>script-specifiying argument already read. That strikes me as a very > >>unintuitive interface. > > > >Ok, I understand this "afterward modification" objection. > > > >What if the -w would be required *before*

Re: [BUGS] [HACKERS] object_classes array is broken, again

2015-07-23 Thread Jaimin Pan
Hi all, How about this patch. I believe it will never missing someone and be detected while compiling. 2015-07-21 19:38 GMT+08:00 Alvaro Herrera : > Tom Lane wrote: > > I wrote: > > > +1 to this patch, in fact I think we could remove MAX_OCLASS altogether > > > which would be very nice for switc

Re: [HACKERS] Volatility of pg_xact_commit_timestamp() and pg_last_committed_xact()

2015-07-23 Thread Alvaro Herrera
Robert Haas wrote: > On Thu, Jul 16, 2015 at 9:49 AM, Fujii Masao wrote: > > Volatilities of pg_xact_commit_timestamp() and pg_last_committed_xact() > > are now STABLE. But ISTM that those functions can return different results > > even within a single statement. So we should change the volatiliti

Re: [HACKERS] All-zero page in GIN index causes assertion failure

2015-07-23 Thread Jeff Janes
On Mon, Jul 20, 2015 at 1:23 PM, Heikki Linnakangas wrote: > > I came up with the attached, for GIN and SP-GiST. Instead of WAL-logging > the page initialization in SP-GiST vacuum, I changed it so that it simply > leaves the page as all-zeroes, and adds it to the FSM. The code that grabs > a targ

Re: [HACKERS] security labels on databases are bad for dump & restore

2015-07-23 Thread Robert Haas
On Wed, Jul 22, 2015 at 3:42 PM, Adam Brightwell wrote: >> I don't think there's any line near pg_dumpall. That tool seems to >> have grown out of desperation without much actual design. I think it >> makes more sense to plan around that's the best pg_dump behavior for the >> various use cases.

Re: [HACKERS] pgbench stats per script & other stuff

2015-07-23 Thread Robert Haas
On Thu, Jul 23, 2015 at 12:15 PM, Fabien COELHO wrote: >> Yes, I think that's a good idea. I don't know whether : is the right >> separator; I kind of line @. But that's bikeshedding. > > Possible ASCII contenders should avoid shell and filename interaction, which > exclude * ? ! & / < > [ ] . -

Re: [HACKERS] Proposing COPY .. WITH PERMISSIVE

2015-07-23 Thread Robert Haas
On Thu, Jul 23, 2015 at 12:19 PM, dinesh kumar wrote: > Sorry for my unclear description about the proposal. > > "WITH PERMISSIVE" is equal to our existing behavior. That is, chmod=644 on > the created files. > > If User don't specify "PERMISSIVE" as an option, then the chmod=600 on > created fil

Re: [HACKERS] Proposing COPY .. WITH PERMISSIVE

2015-07-23 Thread dinesh kumar
Hi Robert/Andres, On Thu, Jul 23, 2015 at 8:34 AM, Robert Haas wrote: > On Thu, Jul 23, 2015 at 11:32 AM, Andres Freund > wrote: > > On 2015-07-23 11:26:27 -0400, Robert Haas wrote: > >> On Wed, Jul 22, 2015 at 11:29 PM, dinesh kumar > wrote: > >> > Would like to propose PERMISSIVE mode for th

Re: [HACKERS] pgbench stats per script & other stuff

2015-07-23 Thread Fabien COELHO
Yes, I think that's a good idea. I don't know whether : is the right separator; I kind of line @. But that's bikeshedding. Possible ASCII contenders should avoid shell and filename interaction, which exclude * ? ! & / < > [ ] . - $ and so on: those that seem to remain are @ , = : % # +. I

Re: [HACKERS] security labels on databases are bad for dump & restore

2015-07-23 Thread Adam Brightwell
> While I'd favor optional --no-create if we were designing fresh, it's not > worth breaking user scripts by changing that now. Agreed. So, --create would not be enabled by default. >> How would this handle related global objects? It seems like this part >> could get a little tricky. > > Like ro

Re: [HACKERS] proposal: multiple psql option -c

2015-07-23 Thread Robert Haas
On Fri, Jul 17, 2015 at 12:36 AM, Pavel Stehule wrote: >> Or just properly understand the ; ? >> >> -c "select * from foo; update bar set baz = 'bing'; vacuum bar;" > > there is a risk of compatibility issues - all statements runs under one > transaction implicitly So what? -- Robert Haas Enter

Re: [HACKERS] Proposing COPY .. WITH PERMISSIVE

2015-07-23 Thread Robert Haas
On Thu, Jul 23, 2015 at 11:32 AM, Andres Freund wrote: > On 2015-07-23 11:26:27 -0400, Robert Haas wrote: >> On Wed, Jul 22, 2015 at 11:29 PM, dinesh kumar >> wrote: >> > Would like to propose PERMISSIVE mode for the COPY created out files. >> > I mean, at this moment, if do "COPY" as postgres i

Re: [HACKERS] Autonomous Transaction is back

2015-07-23 Thread Robert Haas
On Thu, Jul 23, 2015 at 1:31 AM, Rajeev rastogi wrote: > 2.It should be allowed to deadlock with master transaction. We > need to work-out a solution to avoid deadlock. This sentence seems to contradict itself. I thought the consensus was that the transaction should NOT conflict with

Re: [HACKERS] Proposing COPY .. WITH PERMISSIVE

2015-07-23 Thread Andres Freund
On 2015-07-23 11:26:27 -0400, Robert Haas wrote: > On Wed, Jul 22, 2015 at 11:29 PM, dinesh kumar > wrote: > > Would like to propose PERMISSIVE mode for the COPY created out files. > > I mean, at this moment, if do "COPY" as postgres instance owner, i can able > > to read the file as non instance

Re: [HACKERS] Proposing COPY .. WITH PERMISSIVE

2015-07-23 Thread Robert Haas
On Wed, Jul 22, 2015 at 11:29 PM, dinesh kumar wrote: > Would like to propose PERMISSIVE mode for the COPY created out files. > I mean, at this moment, if do "COPY" as postgres instance owner, i can able > to read the file as non instance user as well, and would like to restrict > this to > instan

Re: [HACKERS] pgbench stats per script & other stuff

2015-07-23 Thread Robert Haas
On Wed, Jul 22, 2015 at 1:54 PM, Josh Berkus wrote: > If so, I would vote for: > > -f script1.bench:3 -f script2.bench:1 > > over: > > -f script1.bench -w 3 -f script2.bench -w 1 > > Making command-line options order-dependant breaks a lot of system call > libraries in various languages, as well a

Re: [HACKERS] Eliminating CREATE INDEX comparator TID tie-breaker overhead

2015-07-23 Thread Robert Haas
On Wed, Jul 22, 2015 at 3:17 PM, Peter Geoghegan wrote: >>> I have a hard time imagining anything (beyond synchronous scans) >>> breaking my assumption that index tuplesorts receive tuples in heap >>> physical order. If anything was to break that in the future (e.g. >>> parallelizing the heap scan

Re: [HACKERS] optimizing vacuum truncation scans

2015-07-23 Thread Andres Freund
On 2015-07-23 11:06:25 -0400, Robert Haas wrote: > I don't know what to do about the fact that it loses to the VM-bit > based approach, but I think it's a bad bet that we will never care > about setting visibility map bits anyplace other than VACUUM. +1 A background process that tries to hint all

Re: [HACKERS] optimizing vacuum truncation scans

2015-07-23 Thread Robert Haas
On Wed, Jul 22, 2015 at 12:11 PM, Jeff Janes wrote: >> Urgh. So if we do this, that forever precludes having HOT pruning set >> the all-visible bit. > > I wouldn't say forever, as it would be easy to revert the change if > something more important came along that conflicted with it. Well, the pe

Re: [HACKERS] [PATCH] postgres_fdw extension support

2015-07-23 Thread Andres Freund
On 2015-07-23 07:48:49 -0700, Paul Ramsey wrote: > fdw=# ALTER SERVER foreign_server OPTIONS ( extensions 'postgis' ); > ALTER SERVER > fdw=# ALTER SERVER foreign_server OPTIONS ( extensions 'postgis,seg' ); > ERROR: option "extensions" provided more than once > > Once set, an option seems to be

Re: [HACKERS] fdw_scan_tlist for foreign table scans breaks EPQ testing, doesn't it?

2015-07-23 Thread Robert Haas
On Wed, Jul 22, 2015 at 8:24 PM, Kouhei Kaigai wrote: > Indeed, this commit allows ForeignScan to have fdw_scan_tlist, even if > scanrelid > 0, however, I'm uncertain about its reason/intention. > Does it a preparation for the upcoming target-list-pushdown?? I guess Tom would have to comment on w

Re: [HACKERS] [PATCH] postgres_fdw extension support

2015-07-23 Thread Paul Ramsey
On Thu, Jul 23, 2015 at 7:48 AM, Paul Ramsey wrote: > In testing it I came across an unrelated issue which could make it > hard for users to manage the options on their wrappers/servers > > fdw=# ALTER SERVER foreign_server OPTIONS ( extensions 'postgis' ); > ALTER SERVER > fdw=# ALTER SERVER for

Re: [HACKERS] [PATCH] postgres_fdw extension support

2015-07-23 Thread Paul Ramsey
On Wed, Jul 22, 2015 at 12:19 PM, Paul Ramsey wrote: > > I’ll have a look at doing invalidation for the case of changes to the FDW > wrappers and servers. Here's an updated patch that clears the cache on changes to foreign wrappers and servers. In testing it I came across an unrelated issue which

Re: [HACKERS] ALTER TABLE .. ADD PRIMARY KEY .. USING INDEX has dump-restore hazard

2015-07-23 Thread Michael Paquier
On Thu, Jul 23, 2015 at 9:06 AM, Michael Paquier wrote: > Yeah, I think we should be able to define a collation in this case. > For example it is as well possible to pass a WITH clause with storage > parameters, though we do not document it in > table_constraint_using_index > (http://www.postgresq

Re: [HACKERS] Parallel Seq Scan

2015-07-23 Thread Kouhei Kaigai
Hi Amit, The latest v16 patch cannot be applied to the latest master as is. 434873806a9b1c0edd53c2a9df7c93a8ba021147 changed various lines in heapam.c, so it probably conflicts with this. [kaigai@magro sepgsql]$ cat ~/patch/parallel_seqscan_v16.patch | patch -p1 patching file src/backend/access/c

Re: [HACKERS] "A huge debt of gratitude" - Michael Stonebraker

2015-07-23 Thread Gurjeet Singh
On Jul 22, 2015 12:07 PM, "Jolly Chen" wrote: > > Hey everyone, > > You have probably heard that Mike Stonebraker recently won the Turing award. A recording of his award lecture is available at: > https://www.youtube.com/watch?v=BbGeKi6T6QI > > It is an entertaining talk overall. If you fast forw

[HACKERS] Several memory leaks for pg_rewind caused by missing PQclear calls

2015-07-23 Thread Michael Paquier
Hi all, After a run of valgrind on pg_rewind, I found a couple of code paths missing some PQclear calls after running a query. Attached is a patch to fix all those leaks. Regards, -- Michael diff --git a/src/bin/pg_rewind/libpq_fetch.c b/src/bin/pg_rewind/libpq_fetch.c index 1979fbc..c670fe2 1006

Re: [HACKERS] TABLESAMPLE patch is really in pretty sad shape

2015-07-23 Thread Tom Lane
Petr Jelinek writes: > On 2015-07-23 02:01, Tom Lane wrote: >> This needs to work more like LIMIT, which doesn't try to compute the >> limit parameters until the first fetch. So what we need is an Init >> function that does very darn little indeed (maybe we don't even need >> it at all), and then

Re: [HACKERS] extend pgbench expressions with functions

2015-07-23 Thread Fabien COELHO
Hello Heikki, This patch extends pgbench expression with functions. Currently only one "abs" function is added. The point is rather to bootstrap the infrastructure for other functions (such as hash, random variants...) to be added later. I think it would actually be good to add at least some

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2015-07-23 Thread Laurent Laborde
On Fri, Jan 2, 2015 at 3:18 PM, Amit Kapila wrote: > > > Okay, I have marked this patch as "Ready For Committer" > > Notes for Committer - > There is one behavioural difference in the handling of --analyze-in-stages > switch, when individual tables (by using -t option) are analyzed by > using thi

[HACKERS] REVOKE [ADMIN OPTION FOR] ROLE

2015-07-23 Thread Egor Rogov
Hi, I found an inconsistency between documentation and real behavior of REVOKE [ADMIN OPTION FOR] ROLE. As per documentation (http://www.postgresql.org/docs/9.4/static/sql-revoke.html): -- If GRANT OPTION FOR is specified, only the grant option for the privilege is revoked, not the privileg

Re: [HACKERS] Parallel Seq Scan

2015-07-23 Thread Amit Kapila
On Wed, Jul 22, 2015 at 9:14 PM, Robert Haas wrote: > > One thing I noticed that is a bit dismaying is that we don't get a lot > of benefit from having more workers. Look at the 0.1 data. At 2 > workers, if we scaled perfectly, we would be 3x faster (since the > master can do work too), but we a

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-23 Thread Thakur, Sameer
Hello, >logged > 25 times Sorry, it is much lower at 7 times. Does not change overall point though regards Sameer Thakur | Senior Software Specialist | NTTDATA Global Delivery Services Private Ltd | w. +91.20.6641.7146 | VoIP: 8834.8146 | m. +91 989.016.6656 | sameer.tha...@nttdata.com | Follow u

Re: [HACKERS] TABLESAMPLE patch is really in pretty sad shape

2015-07-23 Thread Petr Jelinek
On 2015-07-23 02:01, Tom Lane wrote: This needs to work more like LIMIT, which doesn't try to compute the limit parameters until the first fetch. So what we need is an Init function that does very darn little indeed (maybe we don't even need it at all), and then a ParamInspect function that is c

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-23 Thread Thakur, Sameer
Hello, >Yes. Any percent completion calculation will have to account for the case of >needing multiple passes through all the indexes. >Each dead tuple requires 6 bytes (IIRC) of maintenance work mem. So if you're >deleting 5M rows with m_w_m=1MB you should be getting many passes through the >i

Re: [HACKERS] Asynchronous execution on FDW

2015-07-23 Thread Kouhei Kaigai
> > If we have ParallelAppend node that kicks a background worker process for > > each underlying child node in parallel, does ForeignScan need to do > > something > > special? > > Although I don't see the point of the background worker in your > story but at least for ParalleMergeAppend, it woul

Re: [HACKERS] compress method for spgist - 2

2015-07-23 Thread Teodor Sigaev
Poorly, by hanging boxes that straddled dividing lines off the parent node in a big linear list. The hope would be that the case was Ok, I see, but that's not really what I was wondering. My question is this: SP-GiST partitions the space into non-overlapping sections. How can you store polygons -

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-07-23 Thread Ildus Kurbangaliev
On 07/23/2015 05:57 AM, Kyotaro HORIGUCHI wrote: At Wed, 22 Jul 2015 17:50:35 +0300, Ildus Kurbangaliev wrote in<55afadbb.9090...@postgrespro.ru> >On 07/22/2015 09:10 AM, Kyotaro HORIGUCHI wrote: > >Hello, > > > >At Tue, 21 Jul 2015 14:28:25 +0300, Ildus Kurbangaliev > > wrote in > ><55ae2cd

Re: [HACKERS] WAL logging problem in 9.4.3?

2015-07-23 Thread Simon Riggs
On 22 July 2015 at 17:21, Heikki Linnakangas wrote: > > When a WAL-skipping COPY begins, we add an entry for that relation in a > "pending-fsyncs" hash table. Whenever we perform any action on a heap that > would normally be WAL-logged, we check if the relation is in the hash > table, and skip WA

Re: [HACKERS] extend pgbench expressions with functions

2015-07-23 Thread Heikki Linnakangas
On 03/06/2015 11:41 AM, Fabien COELHO wrote: This patch extends pgbench expression with functions. Currently only one "abs" function is added. The point is rather to bootstrap the infrastructure for other functions (such as hash, random variants...) to be added later. I think it would actually

Re: [HACKERS] Idea: closing the loop for "pg_ctl reload"

2015-07-23 Thread Heikki Linnakangas
On 07/04/2015 04:24 AM, Jan de Visser wrote: On July 3, 2015 06:21:09 PM Tom Lane wrote: Jan de Visser writes: Attached a new patch, rebased against the current head. Errors in pg_hba.conf and pg_ident.conf are now also noticed. I checked the documentation for pg_ctl reload, and the only plac

Re: [HACKERS] compress method for spgist - 2

2015-07-23 Thread Heikki Linnakangas
On 03/04/2015 06:58 PM, Paul Ramsey wrote: On Wed, Feb 25, 2015 at 6:13 AM, Heikki Linnakangas wrote: In the original post on this, you mentioned that the PostGIS guys planned to use this to store polygons, as bounding boxes (http://www.postgresql.org/message-id/5447b3ff.2080...@sigaev.ru). Any