Re: [HACKERS] pg_rewind failure by file deletion in source server

On 06/23/2015 05:03 PM, Fujii Masao wrote: On Tue, Jun 23, 2015 at 9:19 PM, Heikki Linnakangas wrote: On 06/23/2015 07:51 AM, Michael Paquier wrote: So... Attached are a set of patches dedicated at fixing this issue: Thanks for working on this! - 0001, add if_not_exists to

Re: [HACKERS] A couple of newlines missing in pg_rewind log entries

On 06/23/2015 07:39 AM, Michael Paquier wrote: Hi all, Some grepping is showing up that a couple of newlines are missing in pg_rewind, leading to unreadable log entries: libpq_fetch.c:pg_log(PG_DEBUG, "getting file chunks"); Fixed. logging.c:pg_log(PG_PROGRESS, "%*s/%s kB (%d%%) copi

Re: [HACKERS] pg_rewind failure by file deletion in source server

On 06/23/2015 07:51 AM, Michael Paquier wrote: So... Attached are a set of patches dedicated at fixing this issue: Thanks for working on this! - 0001, add if_not_exists to pg_tablespace_location, returning NULL if path does not exist - 0002, same with pg_stat_file, returning NULL if file does

Re: [HACKERS] user space function "is_power_user"

On 06/22/2015 09:51 AM, Pavel Stehule wrote: Hi I often need a function for identification if current user is database owner or is superuser. It can be pretty simply implemented in C level. Do you think it should be available in core? current_setting('is_superuser'); - Heikki -- Sent via

Re: [HACKERS] pg_stat_*_columns?

On 06/20/2015 11:32 AM, Tom Lane wrote: Magnus Hagander writes: On Sat, Jun 20, 2015 at 10:55 AM, Tom Lane wrote: I dunno that tweaking the format would accomplish much. Where I'd love to get to is to not have to write the data to disk at all (except at shutdown). But that seems to require

Re: [HACKERS] Extension support for postgres_fdw

On 06/20/2015 10:20 AM, Paul Ramsey wrote: I would like to enhance the postgres_fdw to allow more complete support for user-defined types. Right now, postgres_fdw already does a good job of passing user-defined type data back and forth, which is pretty nice. However, it will not pass functions

Re: [HACKERS] pg_rewind failure by file deletion in source server

On 06/16/2015 02:04 AM, Michael Paquier wrote: In order to reduce the risk of failure to a minimum and to preserve the performance of the tool when using --source-server, I think that we should add some check using pg_stat_file to see if a file is still present or not, and if it is missing we can

Re: [HACKERS] Memory leak with XLogFileCopy since de768844 (WAL file with .partial)

On 06/08/2015 09:04 PM, Fujii Masao wrote: On Mon, Jun 8, 2015 at 11:52 AM, Michael Paquier wrote: On Fri, Jun 5, 2015 at 10:45 PM, Fujii Masao wrote: Why don't we call InstallXLogFileSegment() at the end of XLogFileCopy()? If we do that, the risk of memory leak you're worried will disappear a

Re: [HACKERS] gcc -ansi versus SSE4.2 detection

On 06/05/2015 10:07 PM, Tom Lane wrote: It looks like the actual reason that we aren't using the runtime-check CRC implementation is that we can't find a way to do "cpuid" on this old version of OS X. Not sure if it's worth the time to look for one; modern versions of OS X do have __get_cpuid().

Re: [HACKERS] gcc -ansi versus SSE4.2 detection

On 06/05/2015 09:27 PM, Tom Lane wrote: [ this is a bit roundabout, bear with me ] I noticed that, contrary to project policy, a //-style comment snuck into pg_regress.c a month or two back. I had had the idea that buildfarm member pademelon would complain about such comments, given its stone-a

Re: [CORE] [HACKERS] postpone next week's release

On 06/04/2015 12:17 PM, Andres Freund wrote: On 2015-06-04 11:51:44 +0300, Heikki Linnakangas wrote: So, I'm all for refactoring and adding abstractions where it makes sense, but it's not going to solve design problems. I personally don't really see the multixact changes being

Re: [CORE] [HACKERS] postpone next week's release

On 05/30/2015 11:47 PM, Andres Freund wrote: I don't think it's primarily a problem of lack of review; although that is a large problem. I think the biggest systematic problem is that the compound complexity of postgres has increased dramatically over the years. Features have added complexity l

Re: [HACKERS] [COMMITTERS] pgsql: Add pg_audit, an auditing extension

On 05/28/2015 11:14 AM, Stephen Frost wrote: * Heikki Linnakangas (hlinn...@iki.fi) wrote: 1. it's not flexible enough. How do you specify that all READs on super_secret table must be logged, but on less_secret table, it's enough to log only WRITEs? This is actually what that pg_

Re: [HACKERS] [COMMITTERS] pgsql: Add pg_audit, an auditing extension

On 05/28/2015 06:04 AM, Joshua D. Drake wrote: On 05/27/2015 07:02 PM, Stephen Frost wrote: regardless of if they are included in the main git repo or not. Being in the repo represents the support of the overall community and PGDG, which is, understandably in my view, highly valuable to the org

Re: [HACKERS] Buggy logic in nodeIndexscan.c queue handling

On 05/25/2015 06:35 PM, Tom Lane wrote: When deciding whether to pop entries from the queue (line 191), the comparison is done against scandesc->xs_orderbyvals, which as the comment states are the last values physically returned by the index. I think this is wrong, or at least pretty inefficient

[HACKERS] Re: [COMMITTERS] pgsql: At promotion, archive last segment from old timeline with .parti

On 05/22/2015 12:35 PM, Fujii Masao wrote: Doesn't this change break the case where we want to PITR to the recovery target location in the last partial WAL file with the old timeline? In this case, that partial WAL file needs to be read and replayed. But since the suffix of its filename is .parti

Re: [HACKERS] Add a hint when postgresql.conf hot_standby = 'off' and recovery.conf standby = 'on'

On 05/22/2015 06:53 AM, Matthew Kelly wrote: Why anybody in practice would want hot_standby off while in standby mode eludes me, but these are our default values (recovery.conf was generated by pg_basebackup -R). That's how you set up a cold standby system. It seems worth adding a hint and/or

Re: [HACKERS] GiST KNN Crasher

On 05/21/2015 11:28 PM, Tom Lane wrote: Paul Ramsey writes: I'm implementing the recheck functionality for PostGIS so we can support it when 9.5 comes out, and came across this fun little crasher. Hmm, works in 9.3 and 9.4, so it's been broken recently. It was clearly broken by knn-with-rec

Re: [HACKERS] pg_basebackup and replication slots

On 05/21/2015 03:42 PM, Peter Eisentraut wrote: I wonder why pg_basebackup doesn't have any support for replication slots. When relying on replication slots to hang on to WAL data, there is a gap between when pg_basebackup finishes and streaming replication is started where WAL data could be thr

Re: [HACKERS] Re: [COMMITTERS] pgsql: Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.

On 05/21/2015 05:08 AM, Peter Geoghegan wrote: On Wed, May 20, 2015 at 6:22 PM, Tom Lane wrote: I am not really sure that it was a good idea to invent this command tag. In fact, I'm pretty sure it was a *bad* idea --- what will happen if we ever create a statement actually named UPSERT? Why

[HACKERS] Archiving last incomplete segment as .partial issues

rt the XLogFileCopy() changes, although reverting might make backporting future patches slightly easier. - Heikki From a6dab4c8db9f077ecd4b784f390ac161961c90c6 Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Thu, 21 May 2015 15:28:22 +0300 Subject: [PATCH 1/1] At promotion, don't le

Re: [HACKERS] Typo patch

On 05/20/2015 07:29 PM, CharSyam wrote: Hi, I changed typos error. and attached patch for this. Thanks you. I only changed comments only Thanks, committed. Except for this one: --- src/backend/utils/sort/logtape.c +++ src/backend/utils/sort/logtape.c @@ -926,7 +926,7 @@ LogicalTapeBackspace(

Re: [HACKERS] PostgreSQL 8.3 index page count clarification

On 05/20/2015 04:14 PM, Srinivas Karthik V wrote: Hi, For the user created indexes in PostgreSQL 8.3.6, I would like to know in which table (eg: pg_tablename) the index-tuple-count and index-page-count meta-data statistics are stored. pg_class.reltuples and pg_class.relpages. (I'm sure

Re: [HACKERS] Typo in tablecmds.c

On 05/20/2015 12:40 PM, Etsuro Fujita wrote: The attached patch fixes a typo in a comment in tablecmds.c. Fixed, along with dozens more similar typos I found with some grepping. - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscripti

Re: [HACKERS] small typo

On 05/20/2015 06:55 AM, Euler Taveira wrote: Attached is a small typo. Fixed, thanks. - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Wrong Assert in PageIndexMultiDelete?

On 05/19/2015 07:15 PM, Tom Lane wrote: Anastasia Lubennikova writes: I am trying to create new index access method. And I found strange Assert in PageIndexMultiDelete function. Assert

Re: [HACKERS] KNN-GiST with recheck

On 05/16/2015 12:42 AM, Jim Nasby wrote: On 5/14/15 6:30 PM, Heikki Linnakangas wrote: On 05/15/2015 02:28 AM, Heikki Linnakangas wrote: I think this is now ready for committing, but I'm pretty tired now so I'll read through this one more time in the morning, so that I won't w

Re: [HACKERS] 9.5 open items

On 05/18/2015 06:35 AM, Josh Berkus wrote: On 05/15/2015 05:58 AM, Bruce Momjian wrote: I have processed all the open email items I can through mid-March, though I do have two pg_upgrade fixes pending application today. I will continue processing doc fixes and major bug fixes for 9.5, but every

[HACKERS] Re: [COMMITTERS] pgsql: Allow GiST distance function to return merely a lower-bound.

On 05/15/2015 03:17 PM, Heikki Linnakangas wrote: On 05/15/2015 03:05 PM, Fujii Masao wrote: Seems this patch causes the regression test of pg_trgm fail. The regression diff that I got is: *** /home/postgres/pgsql/head/contrib/pg_trgm/expected/pg_trgm.out 2013-07-23 16:46:22.212488785 +0900

Re: [HACKERS] KNN-GiST with recheck

On 05/15/2015 11:31 AM, Alexander Korotkov wrote: On Fri, May 15, 2015 at 2:30 AM, Heikki Linnakangas wrote: On 05/15/2015 02:28 AM, Heikki Linnakangas wrote: I think this is now ready for committing, but I'm pretty tired now so I'll read through this one more time in the morning,

Re: [HACKERS] KNN-GiST with recheck

On 05/15/2015 02:28 AM, Heikki Linnakangas wrote: I think this is now ready for committing, but I'm pretty tired now so I'll read through this one more time in the morning, so that I won't wake up to a red buildfarm. Forgot to attach the latest patch, here you go.

Re: [HACKERS] KNN-GiST with recheck

On 05/14/2015 01:43 PM, Alexander Korotkov wrote: On Wed, May 13, 2015 at 10:17 PM, Alexander Korotkov wrote: One quick comment: It would be good to avoid the extra comparisons of the distances, when the index doesn't return any lossy items. As the patch stands, it adds one extra copyDistanc

Re: [HACKERS] KNN-GiST with recheck

On 04/17/2015 12:05 PM, Alexander Korotkov wrote: On Wed, Feb 25, 2015 at 12:15 PM, Alexander Korotkov wrote: Hi! On Tue, Feb 24, 2015 at 5:39 PM, Tomas Vondra < tomas.von...@2ndquadrant.com> wrote: On 17.2.2015 14:21, Alexander Korotkov wrote: On Sun, Feb 15, 2015 at 2:08 PM, Alexander Ko

Re: [HACKERS] Streaming replication and WAL archive interactions

On 05/13/2015 04:29 PM, Robert Haas wrote: On Wed, May 13, 2015 at 8:53 AM, Heikki Linnakangas wrote: Our manual says that archive_command should refuse to overwrite an existing file. But to work-around the double-archival problem, where the same file is archived twice, it would be even better

Re: [HACKERS] Streaming replication and WAL archive interactions

On 05/13/2015 03:36 PM, Robert Haas wrote: On Mon, May 11, 2015 at 12:00 PM, Heikki Linnakangas wrote: And here is a new version of the patch. I kept the approach of using pgstat, but it now only polls pgstat every 10 seconds, and doesn't block to wait for updated stats. It's not

Re: [HACKERS] Sequence Access Method WIP

On 05/13/2015 02:12 PM, Simon Riggs wrote: This has had around 2 years of thought at this point. I don't agree it needs more thought. Noted. There is one clear use case for this and it is of benefit to many distributed architectures. Right. What's your point? I don't see what calamity wil

Re: [HACKERS] Sequence Access Method WIP

On 05/13/2015 07:10 AM, Alvaro Herrera wrote: Heikki, do you have time to go through this at this point? I'm afraid I won't :-(. I did intend to, but looking at the calendar, I won't have the time to review this thoroughly enough to commit. Sorry. I haven't looked at the CREATE/DROP ACCESS M

Re: [HACKERS] Default Roles

On 05/13/2015 06:07 AM, Stephen Frost wrote: This does change the XLOG functions to require pg_monitor, as discussed on the other thread where it was pointed out by Heikki that the XLOG location information could be used to extract sensitive information based on what happens during compression.

Re: [HACKERS] Let PostgreSQL's On Schedule checkpoint write buffer smooth spread cycle by tuning IsCheckpointOnSchedule?

(please keep the mailing list CC'd, and please don't top-post) On 05/13/2015 05:00 AM, digoal zhou wrote: I test it, but use exponent not very perfect in any environment. why cann't use time only? As you mentioned yourself earlier, if you only use time but you reach checkpoint_segments before

Re: [HACKERS] BRIN range operator class

On 05/12/2015 10:49 PM, Alvaro Herrera wrote: If in the future, for instance, we come up with a way to store the ipv4 plus ipv6 info, we will want to change the page format. If we add a page version to the metapage, we can detect the change at pg_upgrade time and force a reindex of the index.

Re: [HACKERS] Let PostgreSQL's On Schedule checkpoint write buffer smooth spread cycle by tuning IsCheckpointOnSchedule?

On 05/12/2015 03:27 AM, digoal zhou wrote: PostgreSQL (<=9.4) trend to smooth buffer write smooth in a checkpoint_completion_target (checkpoint_timeout or checkpoint_segments), but when we use synchronous_commit=off, there is a little problem for the checkpoint_segments target, because xlog w

Re: [HACKERS] Multixid hindsight design

On 05/12/2015 01:51 AM, Tom Lane wrote: Heikki Linnakangas writes: So the lesson here is that having a permanent pg_multixact is not nice, and we should get rid of it. Here's how to do that: That would be cool, but ... Looking at the tuple header, the CID and CTID fields are only n

Re: [HACKERS] Multi-xacts and our process problem

On 05/12/2015 12:00 AM, Bruce Momjian wrote: Multi-xacts were made durable in Postgres 9.3 (released 2013-09-09) to allow primary-key-column-only locks. 1.7 years later, we are still dealing with bugs related to this feature. Obviously, something is wrong. There were many 9.3 minor releases co

[HACKERS] Multixid hindsight design

I'd like to discuss how we should've implemented the infamous 9.3 multixid/row-locking stuff, and perhaps still should in 9.6. Hindsight is always 20/20 - I'll readily admit that I didn't understand the problems until well after the release - so this isn't meant to bash what's been done. Rather

Re: [HACKERS] Streaming replication and WAL archive interactions

On 05/08/2015 04:21 PM, Heikki Linnakangas wrote: On 04/22/2015 10:07 AM, Michael Paquier wrote: On Wed, Apr 22, 2015 at 3:38 PM, Heikki Linnakangas wrote: I feel that the best approach is to archive the last, partial segment, but with the .partial suffix. I don't see any plausible real

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE/IGNORE 4.0

On 05/08/2015 08:25 PM, Tom Lane wrote: Andres Freund writes: On 2015-05-08 12:32:10 -0400, Tom Lane wrote: Looks like there's a CLOBBER_CACHE_ALWAYS issue ... http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=jaguarundi&dt=2015-05-08%2011%3A52%3A00 Currently index inferrence ignores i

Re: [HACKERS] Streaming replication and WAL archive interactions

On 04/22/2015 10:07 AM, Michael Paquier wrote: On Wed, Apr 22, 2015 at 3:38 PM, Heikki Linnakangas wrote: I feel that the best approach is to archive the last, partial segment, but with the .partial suffix. I don't see any plausible real-wold setup where the current behavior would be bett

Re: [HACKERS] Broken --dry-run mode in pg_rewind

On 05/08/2015 03:39 PM, Michael Paquier wrote: On Fri, May 8, 2015 at 9:34 PM, Heikki Linnakangas wrote: On 05/08/2015 03:25 PM, Vladimir Borodin wrote: Seems, that pg_rewind does not account --dry-run option properly. A simple fix for that is attached. No, the --dry-run takes effect later

Re: [HACKERS] Broken --dry-run mode in pg_rewind

On 05/08/2015 03:25 PM, Vladimir Borodin wrote: Hi all. Seems, that pg_rewind does not account --dry-run option properly. A simple fix for that is attached. No, the --dry-run takes effect later. It performs all the actions it normally would, including reading files from the source, except for

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

On 05/07/2015 12:01 AM, Andres Freund wrote: 6. The tablename and EXCLUDED? Possibility with the ability to specify an AS for INSERT INTO foo AS whatever? If we don't allow "AS whatever", and you create a table called "excluded", you're stuck with the ambiguity in the DO UPDATE statement

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

On 05/07/2015 12:01 AM, Andres Freund wrote: How about 6. The tablename and EXCLUDED? Possibility with the ability to specify an AS for INSERT INTO foo AS whatever? From an implementation pov that'd be simple ;) I did this, because as you say it's simple to implement, and it resolves the

Re: [HACKERS] Disabling trust/ident authentication configure option

On 05/07/2015 01:32 AM, Jim Nasby wrote: On 5/6/15 12:56 PM, Peter Eisentraut wrote: I think this is a sufficiently general requirement to warrant including an option to disable this, as most hardening guides I have seen for PostgreSQL unconditionally require to disable trust authentication and

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

On 05/07/2015 12:01 AM, Andres Freund wrote: On 2015-05-06 23:48:18 +0300, Heikki Linnakangas wrote: I'll see about fixing that. It's not just a matter of creating another alias for the same rel, I'm afraid: "foo.t" is supposed to refer to the tuple that we attempte

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

On 05/07/2015 12:18 AM, Andres Freund wrote: On 2015-05-07 00:10:22 +0300, Heikki Linnakangas wrote: Right, that's the idea. Indexes are just an implementation detail - I think that's a distinction just about no user out there cares about. Unfortunately you can't cr

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

On 05/06/2015 11:05 PM, Peter Geoghegan wrote: On Wed, May 6, 2015 at 7:53 AM, Andres Freund wrote: In this variant, you explicitly specify the constraint by name. I do think it's a bit sad to not be able to specify unique indexes that aren't constraints. So I'd like to have a corresponding O

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

Andres pointed out on IM that the TARGET alias is a bit crummy. In particular, adding an ON CONFLICT DO UPDATE can make a RETURNING clause invalid, because we change the alias of the target rel: create table foo (id int4 primary key, t text); This works: postgres=# insert into foo (id, t) val

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE/IGNORE 4.0

On 05/06/2015 10:47 PM, Peter Geoghegan wrote: On Wed, May 6, 2015 at 8:20 AM, Andres Freund wrote: On 2015-05-05 15:00:56 -0700, Peter Geoghegan wrote: Locking the row is not "nothing", though. If you want to lock the row, use an UPSERT with a tautologically false WHERE clause (like "WHERE fa

Re: [HACKERS] Where are the detoast function called in select * from table_name case?

On 05/06/2015 04:00 PM, Rui Hai Jiang wrote: I've been reading the PostgreSQL code for weeks to figure out how TOAST works. I couldn't find where are the TOAST function called to detoast a tuple comes from a select query, for example, select * from table_name. Does anyone know this? Can you give

[HACKERS] INSERT ... ON CONFLICT error messages

I've read through all the error messages in the patch. Some comments: postgres=# create table foo (id int4); CREATE TABLE postgres=# create unique index foo_y on foo (id) where id > 0; CREATE INDEX postgres=# insert into foo values (-1) on conflict (id) where id > 0 do nothing; ERROR: in

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

On 05/05/2015 12:16 AM, Peter Geoghegan wrote: On Sun, Apr 26, 2015 at 2:22 AM, Heikki Linnakangas wrote: The ability to specify a constraint by name hasn't been implemented, but that would read quite naturally as: INSERT INTO mytable ON CONFLICT ON CONSTRAINT my_constraint UPDATE ...

Re: [HACKERS] BUG in XLogRecordAssemble

On 05/04/2015 07:04 PM, Zhang Zq wrote: hi, I found the code in 'backend/access/transam/xloginsert.c' as that: XLogRecordAssemble: if (prev_regbuf && RelFileNodeEquals(regbuf->rnode, prev_regbuf->rnode)) { samerel = true; bkpb.fork_flags |= BKPBLOCK_SAME_R

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE/IGNORE 4.0

On 04/30/2015 11:09 PM, Peter Geoghegan wrote: I've been unable to reproduce the unprincipled deadlock using the same test case as before. However, the exclusion constraint code now livelocks. Here is example output from a stress-testing session: ... [Fri May 1 04:45:35 2015] normal exit at 14

Re: [HACKERS] pg_rewind test race condition..?

On 04/29/2015 06:03 AM, Stephen Frost wrote: * Heikki Linnakangas (hlinn...@iki.fi) wrote: --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -7173,7 +7173,10 @@ StartupXLOG(void) * than is appropriate now that we're not in standby mode an

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE/IGNORE 4.0

On 04/27/2015 11:02 PM, Peter Geoghegan wrote: On Mon, Apr 27, 2015 at 8:31 PM, Heikki Linnakangas wrote: I thought we had an ironclad scheme to prevent deadlocks like this, so I'd like to understand why that happens. Okay. I think I know how it happens (I was always skeptical of the

Re: [HACKERS] PATCH: adaptive ndistinct estimator v4

On 04/30/2015 01:57 PM, Robert Haas wrote: 2. There should be a compatibility GUC to restore the old behavior. The new behavior should be the default, because if we're not confident that the new behavior will be better for most people, we have no business installing it in the first place (plus f

Re: [HACKERS] pg_rewind test race condition..?

On 04/28/2015 11:02 AM, Stephen Frost wrote: Heikki, Not sure if anyone else is seeing this, but I'm getting regression test failures when running the pg_rewind tests pretty consistently with 'make check'. Specifically with "basic remote", I'm getting: source and target cluster are on

Re: [HACKERS] INSERT ... ON CONFLICT UPDATE/IGNORE 4.0

On 04/27/2015 07:02 PM, Peter Geoghegan wrote: So, this can still happen, but is now happening less often than before, I believe. On a 16 core server, with continual 128 client jjanes_upsert exclusion constraint only runs, with fsync=off, I started at this time: 2015-04-27 21:22:28 UTC [ 0 ]: LO

Re: [HACKERS] INSERT ... ON CONFLICT syntax issues

On 04/25/2015 12:01 PM, Andres Freund wrote: INSERT ... ON CONFLICT (cola, colb [WHERE predicate_for_partial]) UPDATE|IGNORE My problem with the WHERE being inside the parens in the above is that it's a) different from CREATE INDEX b) unclear whether the WHERE belongs to colb or the whole index

Re: [HACKERS] Moving ExecInsertIndexTuples and friends to new file

On 04/24/2015 09:36 AM, Heikki Linnakangas wrote: On 04/24/2015 06:30 AM, Stephen Frost wrote: * Peter Geoghegan (p...@heroku.com) wrote: On Thu, Apr 23, 2015 at 12:05 PM, Heikki Linnakangas wrote: While looking at Peter's INSERT ... ON CONFLICT patch, I started to feel

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

On 04/24/2015 02:52 AM, Peter Geoghegan wrote: I found a bug that seems to be down to commit e3144183562d08e347f832f0b29daefe8bac617b on Github: """ commit e3144183562d08e347f832f0b29daefe8bac617b Author: Heikki Linnakangas Date: Thu Apr 23 18:38:11 2015 +0300 M

Re: [HACKERS] Moving ExecInsertIndexTuples and friends to new file

On 04/24/2015 06:30 AM, Stephen Frost wrote: * Peter Geoghegan (p...@heroku.com) wrote: On Thu, Apr 23, 2015 at 12:05 PM, Heikki Linnakangas wrote: While looking at Peter's INSERT ... ON CONFLICT patch, I started to feel that ExecInsertIndexTuples() and friends would deserve a file of

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

On 04/23/2015 10:53 PM, Andres Freund wrote: On 2015-04-23 12:45:59 -0700, Peter Geoghegan wrote: On Thu, Apr 23, 2015 at 12:55 AM, Andres Freund wrote: I think you misread my statement: I'm saying we don't need the new argument anymore, even if we still do the super-deletion in heap_delete().

[HACKERS] Moving ExecInsertIndexTuples and friends to new file

While looking at Peter's INSERT ... ON CONFLICT patch, I started to feel that ExecInsertIndexTuples() and friends would deserve a file of their own, and not be buried in the middle of execUtils.c. I propose that we split execUtils.c into two, moving ExecOpenIndices(), ExecCloseIndices() ExecIns

Re: [HACKERS] adding more information about process(es) cpu and memory usage

On 04/23/2015 08:00 PM, Radovan Jablonovsky wrote: During current encounters with amazon web services - RDS, the DBA does not have access to OS/linux shell of underlying instance. That render some postgresql monitoring technique of process CPU and memory usage, not useful. Even if the AWS provide

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

On 04/20/2015 07:37 AM, Peter Geoghegan wrote: if (wco->commandType == CMD_INSERT) command = "INSERT-applicable "; else if (wco->commandType == CMD_UPDATE) command = "UPDATE-applicable

Re: [HACKERS] Freeze avoidance of very large table.

On 04/23/2015 06:38 PM, Bruce Momjian wrote: On Thu, Apr 23, 2015 at 10:42:59AM +0300, Heikki Linnakangas wrote: On 04/22/2015 09:24 PM, Robert Haas wrote: I would feel safer if we added a completely new "epoch" counter to the page header, instead of reusing LSNs. But as we all know

Re: [HACKERS] Freeze avoidance of very large table.

On 04/23/2015 06:39 PM, Petr Jelinek wrote: On 23/04/15 17:24, Heikki Linnakangas wrote: On 04/23/2015 05:52 PM, Jim Nasby wrote: I've often wondered if there was some way we could consolidate XMIN/XMAX from multiple tuples at the page level; that could be a big win for OLAP environments

Re: [HACKERS] Freeze avoidance of very large table.

On 04/23/2015 05:52 PM, Jim Nasby wrote: On 4/23/15 2:42 AM, Heikki Linnakangas wrote: On 04/22/2015 09:24 PM, Robert Haas wrote: Yeah. We have a serious need to reduce the size of our on-disk format. On a TPC-C-like workload Jan Wieck recently tested, our data set was 34% larger than

Re: [HACKERS] Streaming replication and WAL archive interactions

On 04/22/2015 11:58 PM, Robert Haas wrote: On Wed, Apr 22, 2015 at 3:34 PM, Heikki Linnakangas wrote: On 04/22/2015 10:21 PM, Robert Haas wrote: On Wed, Apr 22, 2015 at 3:01 PM, Heikki Linnakangas wrote: For example, imagine that perform point-in-time recovery to WAL position 0/1237E568, on

Re: [HACKERS] Freeze avoidance of very large table.

On 04/22/2015 09:24 PM, Robert Haas wrote: I would feel safer if we added a completely new "epoch" counter to the page >header, instead of reusing LSNs. But as we all know, changing the page >format is a problem for in-place upgrade, and takes some space too. Yeah. We have a serious need to red

Re: [HACKERS] Streaming replication and WAL archive interactions

On 04/22/2015 10:21 PM, Robert Haas wrote: On Wed, Apr 22, 2015 at 3:01 PM, Heikki Linnakangas wrote: For example, imagine that perform point-in-time recovery to WAL position 0/1237E568, on timeline 1. That falls within segment 00010012. Then we end recovery, and switch to

Re: [HACKERS] Streaming replication and WAL archive interactions

On 04/22/2015 09:30 PM, Robert Haas wrote: On Wed, Apr 22, 2015 at 2:17 AM, Heikki Linnakangas wrote: Note that it's a bit complicated to set up that scenario today. Archiving is never enabled in recovery mode, so you'll need to use a custom cron job or something to maintain the arch

Re: [HACKERS] Freeze avoidance of very large table.

On 04/22/2015 05:33 PM, Robert Haas wrote: On Tue, Apr 21, 2015 at 7:24 PM, Jim Nasby wrote: On 4/21/15 3:21 PM, Robert Haas wrote: I'm not saying those ideas don't have problems, because they do. But I think they are worth further exploring. The main reason I gave up on that is because Heik

Re: [HACKERS] Improve sleep processing of pg_rewind TAP tests

On 04/20/2015 05:21 AM, Michael Paquier wrote: I have just made a run of the TAP tests of pg_rewind on my raspberry PI 1 (hamster), where the tests are very slow, and I noticed that it takes up to 10s to get confirmation from standby that it has caught up with the changes from master, and 5s to g

Re: [HACKERS] Improve sleep processing of pg_rewind TAP tests

On 04/16/2015 06:51 AM, Alvaro Herrera wrote: Seems reasonable, but why are you sleeping 1s if pg_ctl -w is in use? I thought the -w would wait until promotion has taken effect, so there's no need to sleep additional time. -w is not supported with pg_ctl promote. Only start, stop and restart.

Re: [HACKERS] Streaming replication and WAL archive interactions

On 04/22/2015 03:30 AM, Michael Paquier wrote: This is going to change a behavior that people are used to for a couple of releases. I would not mind having this patch do "archive_mode = on during recovery" => archive only segments generated by this node + the last partial segment on the old timel

Re: [HACKERS] Streaming replication and WAL archive interactions

On 04/22/2015 12:42 AM, Robert Haas wrote: On Tue, Apr 21, 2015 at 6:55 AM, Heikki Linnakangas wrote: On 04/21/2015 12:04 PM, Michael Paquier wrote: On Tue, Apr 21, 2015 at 4:38 PM, Heikki Linnakangas wrote: Note that even though we don't archive the partial last segment on the pre

Re: [HACKERS] installcheck missing in src/bin/pg_rewind/Makefile

On 04/21/2015 07:13 AM, Michael Paquier wrote: Hi all, As mentioned in $subject, the TAP tests of pg_rewind are currently not run by buildfarm machines as the buildfarm client uses installcheck to run the tests in src/bin. A patch is attached to correct the problem. Thanks, applied. (I left o

Re: [HACKERS] Streaming replication and WAL archive interactions

On 04/21/2015 12:04 PM, Michael Paquier wrote: On Tue, Apr 21, 2015 at 4:38 PM, Heikki Linnakangas wrote: Note that even though we don't archive the partial last segment on the previous timeline, the same WAL is copied to the first segment on the new timeline. So the WAL isn't lost

Re: [HACKERS] Streaming replication and WAL archive interactions

On 04/21/2015 09:53 AM, Michael Paquier wrote: On Thu, Apr 16, 2015 at 8:57 PM, Heikki Linnakangas wrote: Oh, hang on, that's not necessarily true. On promotion, the standby archives the last, partial WAL segment from the old timeline. That's just wrong (http://www.postgresql.org/

Re: [HACKERS] Sequence Access Method WIP

On 03/15/2015 09:07 PM, Petr Jelinek wrote: Slightly updated version of the patch. Mainly rebased against current master (there were several conflicts) and fixed some typos, no real functional change. I also attached initial version of the API sgml doc. Thanks! With the patch, pg_class.relam

Re: [HACKERS] Replication identifiers, take 4

On 04/17/2015 11:54 AM, Andres Freund wrote: I've attached a rebased patch, that adds decision about origin logging to the relevant XLogInsert() callsites for "external" 2 byte identifiers and removes the pad-reusing version in the interest of moving forward. Putting aside the 2 vs. 4 byte iden

Re: [HACKERS] Replication identifiers, take 4

On 04/17/2015 11:45 PM, Petr Jelinek wrote: >The argument to move to 4 bytes is a poor one. If it was reasonable in >terms of code or cosmetic value then all values used in the backend >would be 4 bytes. We wouldn't have any 2 byte values anywhere. But we >don't do that. > >The change does nothin

Re: [HACKERS] Replication identifiers, take 4

On 04/17/2015 11:36 PM, Simon Riggs wrote: On 17 April 2015 at 19:18, Heikki Linnakangas wrote: To be honest, I'm not entirely sure what we're arguing over. When arguing over something you consider small, it is customary to allow the author precedence. We can't do things our

Re: [HACKERS] alternative compression algorithms?

On 04/19/2015 11:51 PM, Tomas Vondra wrote: Hi there, in the past we've repeatedly discussed the option of using a different compression algorithm (e.g. lz4), but every time the discussion died off because of fear of possible patent issues [1] [2] and many other threads. Have we decided it's not

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

On 04/17/2015 09:02 PM, Peter Geoghegan wrote: I'm slightly surprised that Heikki now wants to use an infomask2 bit (if that is what you mean), No, I don't. because he wanted to preserve those by doing the MagicOffsetNumber thing. Yes, that's the way to go. Glad we cleared that up :-). -

Re: [HACKERS] Replication identifiers, take 4

On 04/17/2015 08:36 PM, Simon Riggs wrote: On 17 April 2015 at 18:12, Heikki Linnakangas wrote: On 04/17/2015 12:04 PM, Simon Riggs wrote: On 17 April 2015 at 09:54, Andres Freund wrote: Hrmpf. Says the person that used a lot of padding, without much discussion, for the WAL level

Re: [HACKERS] Replication identifiers, take 4

On 04/17/2015 12:04 PM, Simon Riggs wrote: On 17 April 2015 at 09:54, Andres Freund wrote: Hrmpf. Says the person that used a lot of padding, without much discussion, for the WAL level infrastructure making pg_rewind more maintainable. Sounds bad. What padding are we talking about? In the

Re: [HACKERS] Manipulating complex types as non-contiguous structures in-memory

On 04/17/2015 03:58 PM, Tom Lane wrote: Heikki Linnakangas writes: On 03/28/2015 11:24 PM, Tom Lane wrote: + * Macros for iterating through elements of a flat or expanded array. How about a struct instead? struct ArrayIter { Datum datumptr; bool isnullptr; char

Re: [HACKERS] Manipulating complex types as non-contiguous structures in-memory

On 03/28/2015 11:24 PM, Tom Lane wrote: + /* + * Macros for iterating through elements of a flat or expanded array. + * Use "ARRAY_ITER ARRAY_ITER_VARS(name);" to declare the local variables + * needed for an iterator (more than one set can be used in the same function, + * if they have diff

<    2   3   4   5   6   7   8   9   10   11   >