Re: [HACKERS] NULL passed as an argument to memcmp() in parse_func.c

It appears that, according to the standard, passing NULL to memcmp is undefined behavior, even if the count is 0. See http://stackoverflow.com/questions/16362925/can-i-pass-a-null-pointer-to-memcmp for C99 and C++ standard references. I didn't see a good reference for C89 but I find it almost impos

Re: [HACKERS] PGXS "check" target forcing an install ?

On Tue, Jun 23, 2015 at 12:11 AM, Sandro Santilli wrote: > I've noted that upgrading from PostgreSQL 9.3 to 9.5 I'm suddenly > unable to specify a "check" rule in the Makefile that includes the > PGXS one. The error is: > > $ make check > rm -rf ''/tmp_install > make -C '/home/postgresql-9.5/li

Re: [HACKERS] 9.5 make world failing due to sgml tools missing

libxml2 has been a required documentation build tool since PostgreSQL 9.0. The only thing that's new is that xmllint is in a different subpackage on some systems. So just install that and you're all set for the foreseeable future. Well, something is different in 9.5. On this same system (Lin

Re: [HACKERS] A couple of newlines missing in pg_rewind log entries

On Tue, Jun 23, 2015 at 1:39 PM, Michael Paquier wrote: > Hi all, > > Some grepping is showing up that a couple of newlines are missing in > pg_rewind, leading to unreadable log entries: > libpq_fetch.c:pg_log(PG_DEBUG, "getting file chunks"); > logging.c:pg_log(PG_PROGRESS, "%*s/%s kB (%d

Re: [HACKERS] 9.5 make world failing due to sgml tools missing

On Sun, Jun 21, 2015 at 10:56 AM, Peter Eisentraut wrote: > On 6/18/15 8:54 AM, Tom Lane wrote: > > Sure; the point is that libxml2 has suddenly been reclassified as a > > documentation build tool, which is at least a surprising categorization. > > libxml2 has been a req

Re: [HACKERS] upper planner path-ification

On 23 June 2015 at 13:55, Kouhei Kaigai wrote: > Once we support to add aggregation path during path consideration, > we need to pay attention morphing of the final target-list according > to the intermediate path combination, tentatively chosen. > For example, if partial-aggregation makes sense

Re: [HACKERS] checkpointer continuous flushing

Hello Amit, medium2: scale=300 shared_buffers=5GB checkpoint_timeout=30min max_wal_size=4GB warmup=1200 time=7500 flsh | full speed tps | percent of late tx, 4 clients /srt | 1 client | 4 clients | 100 | 200 | 400 | N/N | 173 +- 289* | 198 +- 531* | 27.61

Re: [HACKERS] pg_rewind failure by file deletion in source server

On Fri, Jun 19, 2015 at 9:22 PM, Robert Haas wrote: > On Fri, Jun 19, 2015 at 8:18 AM, Robert Haas wrote: >> On Fri, Jun 19, 2015 at 12:14 AM, Michael Paquier >> wrote: Listing the directories with pg_ls_dir() has the same problem. >>> >>> (After some discussion on IM with Heikki on this on

[HACKERS] A couple of newlines missing in pg_rewind log entries

Hi all, Some grepping is showing up that a couple of newlines are missing in pg_rewind, leading to unreadable log entries: libpq_fetch.c:pg_log(PG_DEBUG, "getting file chunks"); logging.c:pg_log(PG_PROGRESS, "%*s/%s kB (%d%%) copied", filemap.c:pg_fatal("could not stat file \"%

Re: [HACKERS] checkpointer continuous flushing

On Mon, Jun 22, 2015 at 1:41 PM, Fabien COELHO wrote: > > > > >> It'd be interesting to see numbers for tiny, without the overly small >> checkpoint timeout value. 30s is below the OS's writeback time. > > > Here are some tests with longer timeout: > > tiny2: scale=10 shared_buffers=1GB checkpoin

Re: [HACKERS] pretty bad n_distinct estimate, causing HashAgg OOM on TPC-H

Hi, On 06/22/2015 07:47 AM, Jeff Janes wrote: On Sat, Jun 20, 2015 at 8:28 AM, Tomas Vondra mailto:tomas.von...@2ndquadrant.com>> wrote: Hi Tomas, I've lobotomized the sampling a bit to really produce a random set of blocks first, and that produces way better estimates: stati

Re: [HACKERS] upper planner path-ification

> -Original Message- > From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Robert Haas > Sent: Thursday, May 14, 2015 10:39 AM > To: pgsql-hackers@postgresql.org; Tom Lane > Subject: [HACKERS] upper planner path-ification > > Hi, > > I've be

Re: [HACKERS] less log level for success dynamic background workers for 9.5

Michael Paquier writes: > On Tue, Jun 23, 2015 at 10:07 AM, Robert Haas wrote: >> On Mon, Jun 22, 2015 at 8:19 PM, Jim Nasby wrote: >>> Anything ever happen with this? I agree that LOG is to high for reporting >>> most (if not all) of these things. >> I think we should consider having a flag for

Re: [HACKERS] Further issues with jsonb semantics, documentation

On Wed, Jun 10, 2015 at 11:48 AM, Andrew Dunstan wrote: > Please submit a patch to adjust the treatment of negative integers in the > old functions to be consistent with their treatment in the new functions. > i.e. in the range [-n,-1] they should refer to the corresponding element > counting from

Re: [HACKERS] less log level for success dynamic background workers for 9.5

On Tue, Jun 23, 2015 at 10:07 AM, Robert Haas wrote: > On Mon, Jun 22, 2015 at 8:19 PM, Jim Nasby wrote: >> Anything ever happen with this? I agree that LOG is to high for reporting >> most (if not all) of these things. > > I think we should consider having a flag for this behavior rather than > ch

Re: [HACKERS] less log level for success dynamic background workers for 9.5

On Mon, Jun 22, 2015 at 8:19 PM, Jim Nasby wrote: > On 6/14/15 12:25 AM, Pavel Stehule wrote: >> I am working on scheduler extension for 9.5. It use bgworkers >> intensively for any task. This is reason, why I need to decrease a log >> level - and I am thinking so parallel computing needs it due h

Re: [HACKERS] pg_stat_*_columns?

On Sun, Jun 21, 2015 at 12:52 PM, Andres Freund wrote: > On 2015-06-21 12:40:50 -0400, Tom Lane wrote: >> Andres Freund writes: >> > We could also just mmap() the stats file into memory in various >> > processes. With a bit care it should be quite possible to only mmap a >> > subsets of the file

Re: [HACKERS] pg_stat_*_columns?

On Sun, Jun 21, 2015 at 11:43 AM, Magnus Hagander wrote: > On Sat, Jun 20, 2015 at 11:55 PM, Robert Haas wrote: >> On Sat, Jun 20, 2015 at 7:01 PM, Tom Lane wrote: >> >> But if the structure >> >> got too big to map (on a 32-bit system), then you'd be sort of hosed, >> >> because there's no way

Re: [HACKERS] Hash index creation warning

On Tue, Jun 23, 2015 at 9:06 AM, Jim Nasby wrote: > On 6/12/15 5:00 PM, Thom Brown wrote: >> >> On 18 October 2014 at 15:36, Bruce Momjian wrote: >>> >>> On Fri, Oct 17, 2014 at 02:36:55PM -0400, Bruce Momjian wrote: On Fri, Oct 17, 2014 at 12:56:52PM -0400, Tom Lane wrote: > >

Re: [HACKERS] less log level for success dynamic background workers for 9.5

On 6/14/15 12:25 AM, Pavel Stehule wrote: Hi I am working on scheduler extension for 9.5. It use bgworkers intensively for any task. This is reason, why I need to decrease a log level - and I am thinking so parallel computing needs it due high number of created and finished workers. It should b

Re: [HACKERS] Hash index creation warning

On 6/12/15 5:00 PM, Thom Brown wrote: On 18 October 2014 at 15:36, Bruce Momjian wrote: On Fri, Oct 17, 2014 at 02:36:55PM -0400, Bruce Momjian wrote: On Fri, Oct 17, 2014 at 12:56:52PM -0400, Tom Lane wrote: David G Johnston writes: The question is whether we explain the implications of no

Re: [HACKERS] proposal: row_to_array function

On 6/22/15 2:46 AM, Pavel Stehule wrote: FOREACH key, val IN RECORD myrow LOOP IF pg_typeof(val) IN ('int4', 'double precision', 'numeric') THEN val := val + 1; -- these variables can be mutable -- or maybe in futore myrow[key] := val + 1; END IF; END LOOP; What is important

Re: [HACKERS] [Proposal] Progress bar for pg_dump/pg_restore

On 6/21/15 9:45 PM, Craig Ringer wrote: And, I also understood your concern about "CREATE INDEX", but we have no way to get progress information of "CREATE INDEX". >At present, I think it may be good to refer to the same time as estimated time to execute "COPY TO". You could probably get a han

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

On 6/22/15 12:37 PM, Robert Haas wrote: It's not my goal here to create some kind of a performance counter system, even though that would be valuable and could possibly be based on the same infrastructure, but rather just to create a very simple system that lets people know, without any developer

Re: [HACKERS] NULL passed as an argument to memcmp() in parse_func.c

Robert Haas writes: > On Mon, Jun 22, 2015 at 2:55 PM, Tom Lane wrote: >> If I recall that code correctly, the assumption was that if the third >> argument is zero then memcmp() must not fetch any bytes (not should not, >> but MUST not) and therefore it doesn't matter if we pass a NULL. Are >> y

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

Merlin Moncure writes: > On Mon, Jun 22, 2015 at 12:37 PM, Robert Haas wrote: >> ... The basic idea is that pg_stat_activity.waiting would be >> replaced by a new column pg_stat_activity.wait_event, which would >> display the reason why that backend is waiting. > Instead of changing the column,

Re: [HACKERS] 9.5 release notes

On 11 June 2015 09:45, Bruce Momjian Wrote: > > I have committed the first draft of the 9.5 release notes. You can > view the output here: > > http://momjian.us/pgsql_docs/release-9-5.html > > and it will eventually appear here: > > http://www.postgresql.org/docs/devel/static/rele

Re: [HACKERS] NULL passed as an argument to memcmp() in parse_func.c

On Mon, Jun 22, 2015 at 2:55 PM, Tom Lane wrote: > Piotr Stefaniak writes: >> There are two places in parse_func.c where memcmp() conditionally gets a >> NULL as its first argument, which invokes undefined behavior. I guess >> gcc -O2 will make some assumptions based on memcpy's __nonnull attribu

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

On Mon, Jun 22, 2015 at 4:40 PM, Merlin Moncure wrote: > Instead of changing the column, can't we add a new one? Adjusting > columns in PSA requires the innumerable queries written against it to > be adjusted along with all the wiki instructions to dev ops for > emergency stuck query detection et

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

On Mon, Jun 22, 2015 at 12:37 PM, Robert Haas wrote: > When a PostgreSQL system wedges, or when it becomes dreadfully slow > for some reason, I often find myself relying on tools like strace, > gdb, or perf to figure out what is happening. This doesn't tend to > instill customers with confidence;

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

On Mon, Jun 22, 2015 at 4:09 PM, Robert Haas wrote: > On Mon, Jun 22, 2015 at 1:59 PM, David G. Johnston > wrote: > > In addition to the codes themselves I think it would aid less-experienced > > operators if we would provide a meta-data categorization of the codes. > > Something like, I/O Sub-S

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

On Mon, Jun 22, 2015 at 1:59 PM, David G. Johnston wrote: > In addition to the codes themselves I think it would aid less-experienced > operators if we would provide a meta-data categorization of the codes. > Something like, I/O Sub-System; Storage Maintenance; Concurrency, etc.. > > There could b

Re: [HACKERS] Time to get rid of PQnoPasswordSupplied?

On 6/22/15 9:00 AM, Tom Lane wrote: Jim Nasby writes: On 6/19/15 10:35 AM, Tom Lane wrote: On the other hand, you could argue that improving the string is going to break clients that do the right thing (even if klugily) in order to help clients that are doing the wrong thing (ie, failing witho

Re: [HACKERS] NULL passed as an argument to memcmp() in parse_func.c

Piotr Stefaniak writes: > On 06/22/2015 08:55 PM, Tom Lane wrote: >> If I recall that code correctly, the assumption was that if the third >> argument is zero then memcmp() must not fetch any bytes (not should not, >> but MUST not) and therefore it doesn't matter if we pass a NULL. > It's not abo

Re: [HACKERS] NULL passed as an argument to memcmp() in parse_func.c

Piotr Stefaniak writes: > There are two places in parse_func.c where memcmp() conditionally gets a > NULL as its first argument, which invokes undefined behavior. I guess > gcc -O2 will make some assumptions based on memcpy's __nonnull attribute. If I recall that code correctly, the assumption

Re: [HACKERS] Further issues with jsonb semantics, documentation

On 6/5/15 3:51 PM, Alvaro Herrera wrote: Jim Nasby wrote: On 6/5/15 2:08 PM, Petr Jelinek wrote: That's a good point, and it won't get any better if/when we add the json point support in 9.6 since the syntax would be something like select jsonb '{"a":"1", "b":"2", "c": {"a": "2"}}' - '/c/a'; an

Re: [HACKERS] [PATCH] pg_upgrade fails when postgres/template1 isn't in default tablespace

On Fri, Jun 19, 2015 at 12:10 PM, Marti Raudsepp wrote: > One of my databases failed to upgrade successfully and produced this error > in the copying phase: > > error while copying relation "pg_catalog.pg_largeobject" > ("/srv/ssd/PG_9.3_201306121/1/12023" to "/PG_9.4_201409291/1/12130"): No > suc

Re: [HACKERS] pg_receivexlog --create-slot-if-not-exists

On Fri, Jun 19, 2015 at 12:00 PM, Cédric Villemain wrote: > +1 for it in 9.5 If we can do it soon, sure. But not in September. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] Tab completion for TABLESAMPLE

On Fri, Jun 19, 2015 at 4:01 PM, Brendan Jurd wrote: > On Fri, 19 Jun 2015 at 21:05 Petr Jelinek wrote: >> >> On 2015-06-19 09:08, Brendan Jurd wrote: >> > I >> > think it would be convenient and user-friendly to complete the opening >> > bracket -- it would make it perfectly clear that an argume

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

On Mon, Jun 22, 2015 at 1:37 PM, Robert Haas wrote: > and doesn't require a developer to > interpret the results, ​[...]​ > We could > also invent codes for things like "I'm doing a pg_usleep because I've > exceeded max_spins_per_delay" and "I'm waiting for a cleanup lock on a > buffer" and m

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

On 06/22/2015 10:37 AM, Robert Haas wrote: I'm less sure about this next part, but I think we might also want to report ourselves as waiting when we are doing an OS read or an OS write, because it's pretty common for people to think that a PostgreSQL bug is to blame when in fact it's the opera

[HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

When a PostgreSQL system wedges, or when it becomes dreadfully slow for some reason, I often find myself relying on tools like strace, gdb, or perf to figure out what is happening. This doesn't tend to instill customers with confidence; they would like (quite understandably) a process that doesn't

Re: [HACKERS] Auto-vacuum is not running in 9.1.12

Hi Tom/Alvaro, Kindly let us know if the correction provided in previous mail is fine or not! Current code any way handle scenario-1 whereas it is still vulnerable to scenario-2. >From previous mail: *Scenario-1:* current_time (2015) -> changed_to_past (1995) -> stays-here-for-half-day -> correct

Re: [HACKERS] pretty bad n_distinct estimate, causing HashAgg OOM on TPC-H

On 06/22/2015 07:21 AM, Jeff Janes wrote: On Sat, Jun 20, 2015 at 9:55 AM, Tomas Vondra mailto:tomas.von...@2ndquadrant.com>> wrote: Hi, On 06/20/2015 03:01 AM, Jeff Janes wrote: I don't think we need to really assume the density to be constant, ma

[HACKERS] PGXS "check" target forcing an install ?

I've noted that upgrading from PostgreSQL 9.3 to 9.5 I'm suddenly unable to specify a "check" rule in the Makefile that includes the PGXS one. The error is: $ make check rm -rf ''/tmp_install make -C '/home/postgresql-9.5/lib/pgxs/src/makefiles/../..' DESTDIR=''/tmp_install install make[1]: E

Re: [HACKERS] Time to get rid of PQnoPasswordSupplied?

Jim Nasby writes: > On 6/19/15 10:35 AM, Tom Lane wrote: >> On the other hand, you could argue that improving the string is going >> to break clients that do the right thing (even if klugily) in order >> to help clients that are doing the wrong thing (ie, failing without >> offering the opportunit

Re: [HACKERS] Extension support for postgres_fdw

Jim Nasby writes: > On 6/20/15 12:19 PM, Tom Lane wrote: >> Note that no matter what the details are, something like this is putting >> the onus on the DBA to mark as transmittable only functions that actually >> are safe to transmit, ie they exist*and have identical semantics* on the >> remote.

Re: [HACKERS] user space function "is_power_user"

2015-06-22 11:20 GMT+02:00 Heikki Linnakangas : > On 06/22/2015 09:51 AM, Pavel Stehule wrote: > >> Hi >> >> I often need a function for identification if current user is database >> owner or is superuser. >> >> It can be pretty simply implemented in C level. >> >> Do you think it should be availa

Re: [HACKERS] user space function "is_power_user"

On 06/22/2015 09:51 AM, Pavel Stehule wrote: Hi I often need a function for identification if current user is database owner or is superuser. It can be pretty simply implemented in C level. Do you think it should be available in core? current_setting('is_superuser'); - Heikki -- Sent via

Re: [HACKERS] checkpointer continuous flushing

It'd be interesting to see numbers for tiny, without the overly small checkpoint timeout value. 30s is below the OS's writeback time. Here are some tests with longer timeout: tiny2: scale=10 shared_buffers=1GB checkpoint_timeout=5min max_wal_size=1GB warmup=600 time=4000 flsh |

Re: [HACKERS] proposal: row_to_array function

Hi 2015-06-22 5:18 GMT+02:00 Craig Ringer : > On 2 April 2015 at 01:59, Merlin Moncure wrote: > > On Sun, Mar 29, 2015 at 1:27 PM, Tom Lane wrote: > >> Pavel Stehule writes: > >>> here is rebased patch. > >>> It contains both patches - row_to_array function and foreach array > support. > >> >

Re: [HACKERS] checkpointer continuous flushing

Hello Jim, The small problem I see is that for a very large setting there could be several seconds or even minutes of sorting, which may or may not be desirable, so having some control on that seems a good idea. ISTM a more elegant way to handle that would be to start off with a very small n