Re: General purpose hashing func in pgbench

Hello Ildar, Following up the recent discussion on zipfian distribution I was trying to reproduce some YCSB-like workloads. As this paper [1] describes, YCSB uses zipfian distribution to generate keys in order simulate intensive load on small number of records as it happens in real world applic

Re: User defined data types in Logical Replication

On Wed, Dec 20, 2017 at 2:28 PM, Huong Dangminh wrote: > Hi Sawada-san, > >> >> eventually we'll need to map types to local oid (and possibly more) >> >> where the local info is cached so that we can interpret binary >> >> representation of replicated data (which we'll add at some point >> >> sinc

Re: Shouldn't execParallel.c null-terminate query_string in the parallel DSM?

On Wed, Dec 20, 2017 at 7:58 AM, Thomas Munro wrote: > Hi hackers, > > I just saw some trailing garbage in my log file emanating from a > parallel worker when my query happened to be a BUFFERALIGNed length > (in this case 64 characters). Did commit 4c728f382970 forget to make > sure the null ter

Re: TRAP: FailedAssertion("!(TransactionIdPrecedesOrEquals

On Wed, Dec 20, 2017 at 3:33 PM, Erik Rijkers wrote: > (logical replication does the initial syncing of the instances one by one > (sequentially) so it isn't as busy as expected; it just takes a long time) This is quite different then. I thought that you meant physical replication with a set of c

Re: TRAP: FailedAssertion("!(TransactionIdPrecedesOrEquals

On 2017-12-20 06:27, Michael Paquier wrote: On Wed, Dec 20, 2017 at 7:46 AM, Erik Rijkers wrote: TRAP: FailedAssertion("!(TransactionIdPrecedesOrEquals(safeXid, snap->xmin))", File: "snapbuild.c", Line: 580) Sorry, that was probably too terse, I should explain that a little. After initing

domain cast in parameterized vs. non-parameterized query

I've recently come across a query that produces different plans depending on whether it's parameterized or not. The parameterized query takes ~50ms while the non-parameterized query takes ~4s. The issue seems to be that the query contains a STABLE function (uid_langvar) whose parameter is a domain

Re: [HACKERS] static assertions in C++

Peter Eisentraut writes: > On 12/11/17 17:12, Tom Lane wrote: >> Hmm, well, surely there's more than one way to do that; the sizeof >> is just a convenient way to wrap it in C. Wouldn't a typedef serve >> just as well? > Here is another attempt, which has the desired effect with the handful > of

RE: User defined data types in Logical Replication

Hi Sawada-san, > >> eventually we'll need to map types to local oid (and possibly more) > >> where the local info is cached so that we can interpret binary > >> representation of replicated data (which we'll add at some point > >> since it's big performance boost). > > Sounds good. > > >> > >> S

Re: TRAP: FailedAssertion("!(TransactionIdPrecedesOrEquals

On Wed, Dec 20, 2017 at 7:46 AM, Erik Rijkers wrote: > Sorry, that was probably too terse, I should explain that a little. > > After initing 50 instances, I set up and run a pgbench session in the master > session; the pgbench lines are: > > init: pgbench --port=6515 --quiet --initialize --scale

Re: Add hint about replication slots when nearing wraparound

On Wed, Dec 20, 2017 at 12:46 AM, Robert Haas wrote: > On Tue, Dec 19, 2017 at 5:27 AM, Feike Steenbergen > wrote: >> While doing some wraparound debugging, I saw the hint regarding upcoming >> wraparound did not include the problem of having a stale replication >> slot (which I'm actually using

Re: Basebackups reported as idle

On Tue, Dec 19, 2017 at 11:50 PM, David Steele wrote: > On 12/19/17 4:56 AM, Magnus Hagander wrote: >> AFAICT, base backups running on the replication protocol are always >> reported as "idle" in pg_stat_activity. This seems to have been an >> oversight in the "include walsender backends in pg_sta

Re: User defined data types in Logical Replication

On Mon, Dec 18, 2017 at 11:25 PM, Huong Dangminh wrote: >> eventually we'll need to map types to >> local oid (and possibly more) where the local info is cached so that we >> can interpret binary representation of replicated data (which we'll add >> at some point since it's big performance boost).

Re: [HACKERS] parallel.c oblivion of worker-startup failures

On Tue, Dec 19, 2017 at 9:01 PM, Robert Haas wrote: > On Tue, Dec 19, 2017 at 5:01 AM, Amit Kapila wrote: >> I think it would have been much easier to fix this problem if we would >> have some way to differentiate whether the worker has stopped >> gracefully or not. Do you think it makes sense t

Re: explain analyze output with parallel workers - question about meaning of information for explain.depesz.com

On Tue, Dec 19, 2017 at 11:37 PM, Robert Haas wrote: > On Wed, Dec 13, 2017 at 9:18 PM, Amit Kapila wrote: >> Thanks. I think now we can proceed with >> fix_accum_instr_parallel_workers_v8.patch posted above which will fix >> the original issue and the problem we have found in sort and hash >> n

Re: [HACKERS] logical decoding of two-phase transactions

>> Note that this patch does not contain the HeapTupleSatisfiesVacuum >> changes. I believe we need changes to HeapTupleSatisfiesVacuum given >> than logical decoding changes the assumption that catalog tuples >> belonging to a transaction which never committed can be reclaimed >> immediately. With

Re: Bitmap table scan cost per page formula

On Tue, Dec 19, 2017 at 07:55:32PM +, Haisheng Yuan wrote: > Hi hackers, > > This is Haisheng Yuan from Greenplum Database. > > We had some query in production showing that planner favors seqscan over > bitmapscan, and the execution of seqscan is 5x slower than using > bitmapscan, but the cos

Re: WIP: BRIN multi-range indexes

> On Dec 19, 2017, at 5:16 PM, Tomas Vondra > wrote: > > > > On 12/19/2017 08:38 PM, Mark Dilger wrote: >> >>> On Nov 18, 2017, at 12:45 PM, Tomas Vondra >>> wrote: >>> >>> Hi, >>> >>> Apparently there was some minor breakage due to duplicate OIDs, so here >>> is the patch series updated

Re: Estimate maintenance_work_mem for CREATE INDEX

On Tue, Dec 19, 2017 at 11:14 PM, Greg Stark wrote: > You're not the first to propose something like that. I think an > EXPLAIN ALTER TABLE would also be very handy -- it's currently > impossible to tell without carefully reading the source code whether a > given DDL change will require a full tab

Shouldn't execParallel.c null-terminate query_string in the parallel DSM?

Hi hackers, I just saw some trailing garbage in my log file emanating from a parallel worker when my query happened to be a BUFFERALIGNed length (in this case 64 characters). Did commit 4c728f382970 forget to make sure the null terminator is copied? See attached proposed fix. -- Thomas Munro h

Re: Boolean partitions syntax

Hi Mark, On 2017/12/20 6:46, Mark Dilger wrote: >> On Dec 12, 2017, at 10:32 PM, Amit Langote >> wrote: >> Added to CF: https://commitfest.postgresql.org/16/1410/ > > This compiles and passes the regression tests for me. Thanks for the review. > I extended your test a bit to check whether par

Re: WIP: BRIN multi-range indexes

> On Dec 19, 2017, at 5:16 PM, Tomas Vondra > wrote: > > > > On 12/19/2017 08:38 PM, Mark Dilger wrote: >> >>> On Nov 18, 2017, at 12:45 PM, Tomas Vondra >>> wrote: >>> >>> Hi, >>> >>> Apparently there was some minor breakage due to duplicate OIDs, so here >>> is the patch series updated

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

> On Dec 19, 2017, at 4:31 PM, Tomas Vondra > wrote: > > Hi, > > On 12/19/2017 08:17 PM, Mark Dilger wrote: >> >> I tested your latest patches on my mac os x laptop and got one test >> failure due to the results of 'explain' coming up differently. For the >> record, >> I followed these step

Re: WIP: BRIN multi-range indexes

On 12/19/2017 08:38 PM, Mark Dilger wrote: > >> On Nov 18, 2017, at 12:45 PM, Tomas Vondra >> wrote: >> >> Hi, >> >> Apparently there was some minor breakage due to duplicate OIDs, so here >> is the patch series updated to current master. >> >> regards >> >> -- >> Tomas Vondra

Re: [HACKERS] replace GrantObjectType with ObjectType

On Wed, Dec 20, 2017 at 5:43 AM, Alvaro Herrera wrote: > Peter Eisentraut wrote: >> On 12/15/17 17:34, Michael Paquier wrote: >> > On Sat, Dec 16, 2017 at 2:39 AM, Peter Eisentraut >> > wrote: > >> > That's the whole point of not having "default" in the switches, no? >> > Any object additions wou

Re: Statically linking ICU with Postgres

I was able to do it. I had to build ICU statically first with --enable-static --enable-shared=no options. During configuring postgres I had to use the right flags for ICU: /configure --prefix=/leoaaryan/postgres-10 ... --with-icu ICU_CFLAGS="-I/leoaaryan/postgres-10/include -DU_STATIC_IMPLEMENTATI

Re: access/parallel.h lacks PGDLLIMPORT

On 19 December 2017 at 23:24, Robert Haas wrote: > On Tue, Dec 19, 2017 at 3:36 AM, Amit Kapila > wrote: > > I also think it is good to allow ParallelWorkerNumber to be used in > > extensions. Attached is the patch for same. I think for other two we > > should wait till there is really a good

Re: Using ProcSignal to get memory context stats from a running backend

On 20 December 2017 at 02:35, Andres Freund wrote: > > > > Yeah. But please don't mess with MemoryContextStats per se --- > > I dunno about you guys but "call MemoryContextStats(TopMemoryContext)" > > is kinda wired into my gdb reflexes. I think what'd make sense > > is a new function "MemoryC

Re: [JDBC] [HACKERS] Channel binding support for SCRAM-SHA-256

On Wed, Dec 20, 2017 at 1:19 AM, Peter Eisentraut wrote: > I have committed 0001 and 0002 (renaming to scram_channel_binding). Thanks! > The 0003 patch looks mostly fine as well. The only concern I have is > that the way it is set up now, we make the server compute the channel > binding data fo

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

Hi, On 12/19/2017 08:17 PM, Mark Dilger wrote: > > I tested your latest patches on my mac os x laptop and got one test > failure due to the results of 'explain' coming up differently. For the > record, > I followed these steps: > > cd postgresql/ > git pull > # this got my directory up to 8526

Re: TRAP: FailedAssertion("!(TransactionIdPrecedesOrEquals

On 2017-12-19 23:35, Erik Rijkers wrote: I saw this just now: TRAP: FailedAssertion("!(TransactionIdPrecedesOrEquals(safeXid, snap->xmin))", File: "snapbuild.c", Line: 580) while running 50 cascading instances on a single machine. Sorry, that was probably too terse, I should explain that a li

Re: Tracking of page changes for backup purposes. PTRACK [POC]

Hi, a couple of months ago there was proposal / patch with the similar goals, from Andrey Borodin. See these two threads [1] https://www.postgresql.org/message-id/flat/843D96CC-7C55-4296-ADE0-622A7ACD4978%40yesql.se#843d96cc-7c55-4296-ade0-622a7acd4...@yesql.se [2] https://www.postgresql.org/mes

TRAP: FailedAssertion("!(TransactionIdPrecedesOrEquals

I saw this just now: TRAP: FailedAssertion("!(TransactionIdPrecedesOrEquals(safeXid, snap->xmin))", File: "snapbuild.c", Line: 580) while running 50 cascading instances on a single machine. select version(): PostgreSQL 11devel_HEAD_20171219_2158_7d3583ad9ae5 on x86_64-pc-linux-gnu, compiled

Re: Estimate maintenance_work_mem for CREATE INDEX

On Tue, Dec 19, 2017 at 3:15 PM Greg Stark wrote: > On 19 December 2017 at 10:00, Oleksandr Shulgin > wrote: > > > If there would be an option in the database itself to provide those > > estimation, we wouldn't even need to figure out estimation queries. > > "EXPLAIN CREATE INDEX" anyone? > > Yo

Re: File name as application name in regression tests and elsewhere

On 12/18/2017 03:00 PM, Peter Eisentraut wrote: > On 12/18/17 06:59, Andrew Dunstan wrote: >> I was doing some work over the weekend and it occurred to me that it >> would be quite nice to have the input file name from regression tests >> set as the application name, and then use a log_line_prefi

Re: Boolean partitions syntax

> On Dec 12, 2017, at 10:32 PM, Amit Langote > wrote: > > On 2017/12/12 15:39, Amit Langote wrote: >> On 2017/12/12 15:35, Amit Langote wrote: >>> Works for me, updated patch attached. >> >> Oops, attached the old one with the last email. >> >> Updated one really attached this time. > > Adde

Re: [HACKERS] postgres_fdw: Add support for INSERT OVERRIDING clause

On 11/29/17 19:59, Michael Paquier wrote: > On Wed, Nov 29, 2017 at 1:53 PM, Michael Paquier > wrote: >> On Wed, Nov 29, 2017 at 8:12 AM, Tom Lane wrote: >>> IIRC, this issue was debated at great length back when we first put >>> in foreign tables, because early drafts of postgres_fdw did what yo

Re: [HACKERS] replace GrantObjectType with ObjectType

Peter Eisentraut wrote: > On 12/15/17 17:34, Michael Paquier wrote: > > On Sat, Dec 16, 2017 at 2:39 AM, Peter Eisentraut > > wrote: > > That's the whole point of not having "default" in the switches, no? > > Any object additions would be caught at compilation time. > > I think the purpose of Ev

Re: genomic locus

Uncertain type is a great idea. I needed type like this to store information about premiere date or people birth date in movie database. Finally I made two columns: and . Precocious could me 'year', 'month' and so on and if precocious was 'year' the date should always had to be like '-01-01'.

Re: explain analyze output with parallel workers - question about meaning of information for explain.depesz.com

On Tue, Dec 19, 2017 at 1:29 PM, Robert Haas wrote: > Well, not really, because the context surrounding the lines you've > added seems to refer to SQL that I can't find in join.sql or anywhere > else in the tree. So my suspicion is that this patch is based on your > parallel hash patch set rather

Re: genomic locus

On Fri, Dec 15, 2017 at 10:49 PM, Gene Selkov wrote: > Greetings everyone, Привет ! > > I need a data type to represent genomic positions, which will consist of a > string and a pair of integers with interval logic and access methods. Sort > of like my seg type, but more straightforward. Why no

Bitmap table scan cost per page formula

Hi hackers, This is Haisheng Yuan from Greenplum Database. We had some query in production showing that planner favors seqscan over bitmapscan, and the execution of seqscan is 5x slower than using bitmapscan, but the cost of bitmapscan is 2x the cost of seqscan. The statistics were updated and qu

Re: [HACKERS] replace GrantObjectType with ObjectType

On 12/15/17 17:34, Michael Paquier wrote: > On Sat, Dec 16, 2017 at 2:39 AM, Peter Eisentraut > wrote: >> On 12/13/17 02:35, Michael Paquier wrote: >>> Patch 0001 is simply removing EventTriggerSupportsGrantObjectType(), >>> but shouldn't we keep it and return an error for objects that have no >>>

Re: [HACKERS] replace GrantObjectType with ObjectType

On 12/18/17 02:38, Rushabh Lathia wrote: > Only motivation is, earlier approach looks more cleaner. Also patch is > getting bigger - so if we continue with old approach it will make review > easy. Just in case switch/case approach is a go to, then it can be > done as part of separate clean up patch

Re: vacuum vs heap_update_tuple() and multixactids

On Tue, Dec 19, 2017 at 1:31 PM, Andres Freund wrote: > Could I perhaps convince somebody to add that as a feature to > isolationtester? I'm willing to work on a bugfix for the bug itself, but > I've already spent tremendous amounts of time, energy and pain on > multixact bugs, and I'm at the mome

Re: [HACKERS] replace GrantObjectType with ObjectType

On 12/15/17 14:10, Robert Haas wrote: >> There is an argument for having a big array versus the switch/case >> approach. But most existing code around object addresses uses the >> switch/case approach, so it's better to align it that way, I think. >> It's weird to have to maintain two different st

Re: [HACKERS] logical decoding of two-phase transactions

On 12/19/17 03:37, Nikhil Sontakke wrote: > Note that this patch does not contain the HeapTupleSatisfiesVacuum > changes. I believe we need changes to HeapTupleSatisfiesVacuum given > than logical decoding changes the assumption that catalog tuples > belonging to a transaction which never committed

Re: WIP: BRIN multi-range indexes

> On Nov 18, 2017, at 12:45 PM, Tomas Vondra > wrote: > > Hi, > > Apparently there was some minor breakage due to duplicate OIDs, so here > is the patch series updated to current master. > > regards > > -- > Tomas Vondra http://www.2ndQuadrant.com > PostgreSQL Development,

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

> On Nov 27, 2017, at 8:47 AM, Tomas Vondra > wrote: > > Hi, > > Attached is an updated version of the patch series, fixing the issues > reported by Mark Dilger: > > 1) Fix fabs() issue in histogram.c. > > 2) Do not rely on extra_data being StdAnalyzeData, and instead lookup > the LT operato

Re: WIP: a way forward on bootstrap data

On Thu, Dec 14, 2017 at 05:59:12PM +0700, John Naylor wrote: > On 12/13/17, Peter Eisentraut wrote: > > On 12/13/17 04:06, John Naylor wrote: > >> There doesn't seem to be any interest in bootstrap data at the moment, > >> but rather than give up just yet, I've added a couple features to make > >>

Re: vacuum vs heap_update_tuple() and multixactids

On 2017-12-19 15:35:12 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > > I think the bugfix is going to have to essentially be something similar > > to FreezeMultiXactId(). I.e. when reusing an old tuple's xmax for a new > > tuple version, we need to prune dead multixact members. I think w

improve type conversion of SPI_processed in Python

Here is a patch to improves how PL/Python deals with very large values of SPI_processed. The previous code converts anything that does not fit into a C long into a Python float. But Python long has unlimited precision, so we should be using that instead. And in Python 3, int and long as the same

Re: Using ProcSignal to get memory context stats from a running backend

Andres Freund writes: > On 2017-12-19 13:17:52 -0500, Tom Lane wrote: >> Yeah. But please don't mess with MemoryContextStats per se --- >> I dunno about you guys but "call MemoryContextStats(TopMemoryContext)" >> is kinda wired into my gdb reflexes. I think what'd make sense >> is a new function

Re: Using ProcSignal to get memory context stats from a running backend

On 2017-12-19 13:17:52 -0500, Tom Lane wrote: > Robert Haas writes: > > On Tue, Dec 19, 2017 at 8:44 AM, Craig Ringer wrote: > >> I didn't want to mess with the MemoryContextMethods and expose a > >> printf-wrapper style typedef in memnodes.h, so I went with a hook global. > > > That looks prett

Re: vacuum vs heap_update_tuple() and multixactids

Andres Freund wrote: > I think the bugfix is going to have to essentially be something similar > to FreezeMultiXactId(). I.e. when reusing an old tuple's xmax for a new > tuple version, we need to prune dead multixact members. I think we can > do so unconditionally and rely on multixact id caching

vacuum vs heap_update_tuple() and multixactids

Hi, In [1] I'd discovered a only mildly related bug while reading code to make sure my fix [2] et al was correct. Quoting a couple messages by myself: > Staring at the vacuumlazy hunk I think I might have found a related bug: > heap_update_tuple() just copies the old xmax to the new tuple's xmax

Re: explain analyze output with parallel workers - question about meaning of information for explain.depesz.com

On Tue, Dec 5, 2017 at 4:23 PM, Thomas Munro wrote: > The hash version of this code is now committed as 5bcf389e. Here is a > patch for discussion that adds some extra tests to join.sql to > exercise rescans of a hash join under a Gather node. It fails on > head, because it loses track of the in

Re: Using ProcSignal to get memory context stats from a running backend

Robert Haas writes: > On Tue, Dec 19, 2017 at 8:44 AM, Craig Ringer wrote: >> I didn't want to mess with the MemoryContextMethods and expose a >> printf-wrapper style typedef in memnodes.h, so I went with a hook global. > That looks pretty grotty to me. I think if you want to elog/ereport > thi

Re: Protect syscache from bloating with negative cache entries

Robert Haas writes: > On Tue, Dec 19, 2017 at 3:31 AM, Kyotaro HORIGUCHI > wrote: >> I see three kinds of syscache entries. >> >> A. An entry for an actually existing object. >> B. An entry for an object which once existed but no longer. >> C. An entry for a just non-existent objects. > I'm not

Re: explain analyze output with parallel workers - question about meaning of information for explain.depesz.com

On Wed, Dec 13, 2017 at 9:18 PM, Amit Kapila wrote: > Thanks. I think now we can proceed with > fix_accum_instr_parallel_workers_v8.patch posted above which will fix > the original issue and the problem we have found in sort and hash > nodes. Committed and back-patched to v10. While I was doing

Letting plpgsql in on the fun with the new expression eval stuff

I'm looking at ways to get plpgsql expression evaluation to go faster, and one thing I'm noticing is the rather large overhead of going through ExecEvalParamExtern and plpgsql_param_fetch to get to the useful work (exec_eval_datum). We've ameliorated that for DTYPE_VAR variables by keeping a pre-s

update portal-related memory context names and API

ISTM that some of the portal-related memory context naming is a bit antiquated and at odds with current terminology. In this patch, I propose to rename PortalMemory to TopPortalContext and rename Portal->heap to Portal->portalContext, and then clean up some surrounding APIs. -- Peter Eisentraut

Re: Using ProcSignal to get memory context stats from a running backend

On Tue, Dec 19, 2017 at 8:44 AM, Craig Ringer wrote: > I didn't want to mess with the MemoryContextMethods and expose a > printf-wrapper style typedef in memnodes.h, so I went with a hook global. That looks pretty grotty to me. I think if you want to elog/ereport this, you need to pass another a

Re: [JDBC] [HACKERS] Channel binding support for SCRAM-SHA-256

On 11/30/17 21:11, Michael Paquier wrote: > OK, here is a reworked version with the following changes: > - renamed saslchannelbinding to scramchannelbinding, with a default > set to tls-unique. > - An empty value of scramchannelbinding allows client to not use > channel binding, or in short use use

Re: Protect syscache from bloating with negative cache entries

On Tue, Dec 19, 2017 at 3:31 AM, Kyotaro HORIGUCHI wrote: > I see three kinds of syscache entries. > > A. An entry for an actually existing object. > B. An entry for an object which once existed but no longer. > C. An entry for a just non-existent objects. I'm not convinced that it's useful to di

Re: Add hint about replication slots when nearing wraparound

On Tue, Dec 19, 2017 at 5:27 AM, Feike Steenbergen wrote: > While doing some wraparound debugging, I saw the hint regarding upcoming > wraparound did not include the problem of having a stale replication > slot (which I'm actually using to force wraparound issues). > > I remember a few discussions

Re: Top-N sorts verses parallelism

On Tue, Dec 19, 2017 at 6:54 AM, Thomas Munro wrote: > On Mon, Dec 18, 2017 at 9:29 AM, Robert Haas wrote: >> I went through the callers to create_sort_path and the only one that >> looks like it can pass a limit is the one you and Jeff already >> identified. So I think the question is just whet

Re: [HACKERS] parallel.c oblivion of worker-startup failures

On Tue, Dec 19, 2017 at 5:01 AM, Amit Kapila wrote: > I think it would have been much easier to fix this problem if we would > have some way to differentiate whether the worker has stopped > gracefully or not. Do you think it makes sense to introduce such a > state in the background worker machin

Re: access/parallel.h lacks PGDLLIMPORT

On Tue, Dec 19, 2017 at 3:36 AM, Amit Kapila wrote: > I also think it is good to allow ParallelWorkerNumber to be used in > extensions. Attached is the patch for same. I think for other two we > should wait till there is really a good use case for them. I think waiting for a "really good" use c

Re: MemoryContextCreate change in PG 11 how should contexts be created

On 12/19/17 10:11 AM, Paul Ramsey wrote: > On Tue, Dec 19, 2017 at 7:00 AM, Tom Lane wrote: >> Paul Ramsey writes: > > If I'm reading right, using MemoryContextRegisterResetCallback on a > AllocSetContext created under our PortalContext should do the trick, > with less direct mucking about into

Re: MemoryContextCreate change in PG 11 how should contexts be created

On Tue, Dec 19, 2017 at 7:00 AM, Tom Lane wrote: > Paul Ramsey writes: >> Our use of MemoryContextCreate is solely in order to get use >> MemoryContextDelete as a callback so that, at the end of a statement, >> we can clean up externally allocated memory that we're holding in a >> cache. If we ha

Re: MemoryContextCreate change in PG 11 how should contexts be created

On Tue, Dec 19, 2017 at 6:54 AM, Alvaro Herrera wrote: > Paul Ramsey wrote: > >> Our use of MemoryContextCreate is solely in order to get use >> MemoryContextDelete as a callback so that, at the end of a statement, >> we can clean up externally allocated memory that we're holding in a >> cache. >

Re: MemoryContextCreate change in PG 11 how should contexts be created

Paul Ramsey writes: > Our use of MemoryContextCreate is solely in order to get use > MemoryContextDelete as a callback so that, at the end of a statement, > we can clean up externally allocated memory that we're holding in a > cache. If we had some other callback to use for "the statement is > com

Re: MemoryContextCreate change in PG 11 how should contexts be created

Paul Ramsey wrote: > Our use of MemoryContextCreate is solely in order to get use > MemoryContextDelete as a callback so that, at the end of a statement, > we can clean up externally allocated memory that we're holding in a > cache. You should not use MemoryContextCreate at all -- it's somewhat o

Re: Basebackups reported as idle

Hi Magnus, On 12/19/17 4:56 AM, Magnus Hagander wrote: > AFAICT, base backups running on the replication protocol are always > reported as "idle" in pg_stat_activity. This seems to have been an > oversight in the "include walsender backends in pg_stat_activity" in 10, > which does include it for w

Re: MemoryContextCreate change in PG 11 how should contexts be created

On Tue, Dec 19, 2017 at 12:24 AM, Regina Obe wrote: > On December 13th this change to context creation was committed, which broke > PostGIS trunk compile against PostgreSQL 11 head. > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=9fa6f00b1308d > d10da4eca2f31ccbfc7b35bb461 > > Tic

Re: WIP Patch: Pgbench Serialization and deadlock errors

On 19-12-2017 17:11, Fabien COELHO wrote: Consider adding the patch to the next commitfest? Hi! Yes, here it is: https://commitfest.postgresql.org/16/1420/ I think you may have to ask the cf app admin to remove the duplicate. https://commitfest.postgresql.org/16/1419/ Thanks, I'm trying to

Re: Estimate maintenance_work_mem for CREATE INDEX

On 19 December 2017 at 10:00, Oleksandr Shulgin wrote: > If there would be an option in the database itself to provide those > estimation, we wouldn't even need to figure out estimation queries. > "EXPLAIN CREATE INDEX" anyone? You're not the first to propose something like that. I think an EXPL

Re: WIP Patch: Pgbench Serialization and deadlock errors

Consider adding the patch to the next commitfest? Hi! Yes, here it is: https://commitfest.postgresql.org/16/1420/ I think you may have to ask the cf app admin to remove the duplicate. https://commitfest.postgresql.org/16/1419/ -- Fabien.

Re: Notes about Pl/PgSQL assignment performance

Andrey Zhidenkov wrote: > I've digged into the source code a little bit and found that chain: > > PLPGSQL_STMT_ASSIGN -> exec_stmt_assign() -> exec_assign_expr() > -> exec_eval_expr() -> exec_run_select() > -> SPI_execute_plan_with_paramlist() -> _SPI_execute_plan() which finnaly > calls PushActiv

Re: WIP Patch: Pgbench Serialization and deadlock errors

On 19-12-2017 16:52, Fabien COELHO wrote: Hello Marina, This is the fourth version of the patch for pgbench. Consider adding the patch to the next commitfest? Hi! Yes, here it is: https://commitfest.postgresql.org/16/1420/ -- Marina Polyakova Postgres Professional: http://www.postgrespro.c

Re: Notes about Pl/PgSQL assignment performance

On 20 December 2017 at 02:48, Andrey Zhidenkov wrote: > PLPGSQL_STMT_ASSIGN -> exec_stmt_assign() -> exec_assign_expr() -> > exec_eval_expr() -> exec_run_select() -> SPI_execute_plan_with_paramlist() > -> _SPI_execute_plan() which finnaly calls PushActiveSnapshot() and > PopActiveSnapshot() wich j

Re: Using ProcSignal to get memory context stats from a running backend

Hi 2017-12-19 14:44 GMT+01:00 Craig Ringer : > On 18 December 2017 at 10:05, Robert Haas wrote: > >> On Thu, Dec 14, 2017 at 9:34 PM, Craig Ringer >> wrote: >> > On 15 December 2017 at 09:24, Greg Stark wrote: >> >> Another simpler option would be to open up a new file in the log >> >> directo

Re: WIP Patch: Pgbench Serialization and deadlock errors

Hello Marina, This is the fourth version of the patch for pgbench. Consider adding the patch to the next commitfest? -- Fabien.

Re: Notes about Pl/PgSQL assignment performance

I've digged into the source code a little bit and found that chain: PLPGSQL_STMT_ASSIGN -> exec_stmt_assign() -> exec_assign_expr() -> exec_eval_expr() -> exec_run_select() -> SPI_execute_plan_with_paramlist() -> _SPI_execute_plan() which finnaly calls PushActiveSnapshot() and PopActiveSnapshot()

Re: Using ProcSignal to get memory context stats from a running backend

On 18 December 2017 at 10:05, Robert Haas wrote: > On Thu, Dec 14, 2017 at 9:34 PM, Craig Ringer > wrote: > > On 15 December 2017 at 09:24, Greg Stark wrote: > >> Another simpler option would be to open up a new file in the log > >> directory > > > > ... if we have one. > > > > We might be logg

Re: [HACKERS] path toward faster partition pruning

On 19 December 2017 at 17:36, David Rowley wrote: > I'm sorry to say this is another micro review per code I'm stumbling > over when looking at the run-time partition pruning stuff. Again, another micro review. I apologise for the slow trickle of review. Again, these are just things I'm noticing

Re: Package version in PG_VERSION and version()

Re: Robert Haas 2017-12-17 > Unfortunately, actually modifying the main version number breaks large > numbers of tools and drivers that think they know what a PostgreSQL > version number looks like, as many people who work for my employer can > testify to from personal experience with a piece of

Re: Notes about Pl/PgSQL assignment performance

Andrey Zhidenkov wrote: > When I run this test in 2 threads I expect that running time will be the > same, because PostgreSQL will fork process for the second connection and > this process will be served by a separate CPU core because I have more than > 2 cores. > Yes, IMMUTABLE flag helps, but I t

Re: Tracking of page changes for backup purposes. PTRACK [POC]

Hello Robert, > I think this doesn't really show much because it's apparently limited > by the speed of fsync() on your filesystem. You might try running the > test with synchronous_commit=off. You are right, synchronous_commit=off revealed a noticeable performance degradation. Also I realized t

Re: [HACKERS] Add support for tuple routing to foreign partitions

(2017/12/18 23:25), Alvaro Herrera wrote: InitResultRelInfo becomes unintelligible after this patch -- it was straightforward but adding partition_root makes things shaky. Please add a proper comment indicating what each argument is. I was thiking that the comment I added to the definition of

Re: Top-N sorts verses parallelism

On Tue, Dec 19, 2017 at 5:24 PM, Thomas Munro wrote: > On Mon, Dec 18, 2017 at 9:29 AM, Robert Haas wrote: >> I went through the callers to create_sort_path and the only one that >> looks like it can pass a limit is the one you and Jeff already >> identified. So I think the question is just whet

Re: Top-N sorts verses parallelism

On Mon, Dec 18, 2017 at 9:29 AM, Robert Haas wrote: > I went through the callers to create_sort_path and the only one that > looks like it can pass a limit is the one you and Jeff already > identified. So I think the question is just whether > create_gather_merge_path needs a similar fix. I migh

Re: Notes about Pl/PgSQL assignment performance

2017-12-19 12:46 GMT+01:00 Pavel Stehule : > > > 2017-12-19 12:40 GMT+01:00 Hannu Krosing : > >> On 19.12.2017 11:36, Pavel Stehule wrote: >> >> Hi >> >> 2017-12-19 12:28 GMT+01:00 Андрей Жиденков : >> >>> Few day ago a faced a problem: Pl/PgSQL procedure works slower when >>> running in parallel

Re: Notes about Pl/PgSQL assignment performance

2017-12-19 12:45 GMT+01:00 Andrey Zhidenkov : > When I run this test in 2 threads I expect that running time will be the > same, because PostgreSQL will fork process for the second connection and > this process will be served by a separate CPU core because I have more than > 2 cores. > Yes, IMMUTA

Re: Notes about Pl/PgSQL assignment performance

2017-12-19 12:40 GMT+01:00 Hannu Krosing : > On 19.12.2017 11:36, Pavel Stehule wrote: > > Hi > > 2017-12-19 12:28 GMT+01:00 Андрей Жиденков : > >> Few day ago a faced a problem: Pl/PgSQL procedure works slower when >> running in parallel threads. I found the correlation between number of >> assig

Re: Notes about Pl/PgSQL assignment performance

When I run this test in 2 threads I expect that running time will be the same, because PostgreSQL will fork process for the second connection and this process will be served by a separate CPU core because I have more than 2 cores. Yes, IMMUTABLE flag helps, but I think It's just because Postgres ac

Re: Notes about Pl/PgSQL assignment performance

On 19.12.2017 11:36, Pavel Stehule wrote: > Hi > > 2017-12-19 12:28 GMT+01:00 Андрей Жиденков >: > > Few day ago a faced a problem: Pl/PgSQL procedure works slower > when running in parallel threads. I found the correlation between > number of assignments in

Re: Notes about Pl/PgSQL assignment performance

Hi 2017-12-19 12:28 GMT+01:00 Андрей Жиденков : > Few day ago a faced a problem: Pl/PgSQL procedure works slower when > running in parallel threads. I found the correlation between number of > assignments in procedure code and performance. I decided to write the > simple benchmark procedures and

Notes about Pl/PgSQL assignment performance

Few day ago a faced a problem: Pl/PgSQL procedure works slower when running in parallel threads. I found the correlation between number of assignments in procedure code and performance. I decided to write the simple benchmark procedures and perform some test on PostgreSQL 9.6.5 database installed o

Re: New gist vacuum.

Hi hackers! Here is the patch that deletes pages during GiST VACUUM. > 12 нояб. 2017 г., в 23:20, Andrey Borodin написал(а): > > If author and community do not object, I want to continue work on > Konstantin's patch. ==Purpose== Long story short, some time ago Konstantin Kuznetsov hacked out

  1   2   >