Re: [HACKERS] libpq compression

Euler Taveira writes: > There was already some discussion about compressing libpq data [1][2][3]. > Recently, I faced a scenario that would become less problematic if we have had > compression support. The scenario is frequent data load (aka COPY) over > slow/unstable links. It should be executed

Re: [HACKERS] Ability to listen on two unix sockets

Robert Haas writes: > Maybe: > listen_addresses = { host | host:port | * | *:port } [, ...] > unix_socket_directory = { directory | directory:port ] [,...] > ...except that colon is a valid character in a directory name. Not > sure what to do about that. Do we need to do anything? There are n

[HACKERS] libpq compression

Hi, There was already some discussion about compressing libpq data [1][2][3]. Recently, I faced a scenario that would become less problematic if we have had compression support. The scenario is frequent data load (aka COPY) over slow/unstable links. It should be executed in a few hundreds of Postg

Re: [HACKERS] Ability to listen on two unix sockets

On Thu, Jun 14, 2012 at 12:18 AM, Tom Lane wrote: > Peter Eisentraut writes: >> On mån, 2012-06-11 at 18:07 -0400, Tom Lane wrote: >>> Peter Eisentraut writes: So you do need to create M*N sockets. I don't really see a problem with that. > >>> I do: first, it's a lotta sockets, and sec

Re: [HACKERS] Ability to listen on two unix sockets

Peter Eisentraut writes: > On mån, 2012-06-11 at 18:07 -0400, Tom Lane wrote: >> Peter Eisentraut writes: >>> So you do need to create M*N sockets. >>> I don't really see a problem with that. >> I do: first, it's a lotta sockets, and second, it's not real hard to >> foresee cases where somebody

Re: [HACKERS] Re: [COMMITTERS] pgsql: Send new protocol keepalive messages to standby servers.

Dimitri Fontaine writes: > Tom Lane writes: >> I thought I already pointed that out, but: we have *extensions*. What >> we don't have is a convenient method of dealing with functions that need >> to be migrated across extensions, or from an extension to core, between >> one major release and the

[HACKERS] Re: [COMMITTERS] pgsql: Send new protocol keepalive messages to standby servers.

On Mon, Jun 4, 2012 at 8:49 PM, Christopher Browne wrote: >> What if the two servers are in different time zones? > > NTP shouldn't have any problem; it uses UTC underneath.  As does > PostgreSQL, underneath. As an aside, this is not strictly speaking true. NTP doesn't "use UTC" -- afaik it doesn

Re: [HACKERS] Minimising windows installer password confusion

On 06/13/2012 06:32 PM, Florian Pflug wrote: Some further googling indicates that, yes, the service account passwords are stored in the registry, but are only accessible to the LocalSystem account [2]. Querying them from the postgres installer thus isn't really an option. But what you could do, I

Re: [HACKERS] Minimising windows installer password confusion

On 06/13/2012 05:18 PM, Dave Page wrote: On Wed, Jun 13, 2012 at 3:07 AM, Craig Ringer Why "using the windows control panel" ? Because when I wrote the email I was looking for a simple solution that wouldn't require writing code that has potential to fail depending on how the users environme

Re: [HACKERS] Ability to listen on two unix sockets

On Thu, Jun 14, 2012 at 01:31:31AM +0300, Peter Eisentraut wrote: > On ons, 2012-06-13 at 15:25 +0200, Honza Horak wrote: > > It seems unix_socket_directory could be turned into list and probably > > renamed to unix_socket_directories, since it would be confusing if a > > list value is in singula

Re: [HACKERS] Minimising windows installer password confusion

On 06/13/2012 05:14 PM, Dave Page wrote: On Wed, Jun 13, 2012 at 2:18 AM, Craig Ringer wrote: On 06/12/2012 08:08 PM, Dave Page wrote: Some background: By default the installer will use 'postgres' for both the service (OS) account, and the database superuser account. It will use the same passw

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

Robert Haas writes: >>> ! DETAIL: Character with value 0x0a must be escaped. >>> ! CONTEXT: JSON data, line 1: "abc >>> ! ... >>> >>> This seems an odd way to present this, especially if the goal is to >>> NOT include the character needing escaping in the log unescaped, which >>> I thought was t

Re: [HACKERS] Minimising windows installer password confusion

On 06/13/2012 05:10 PM, Dave Page wrote: The idea of storing the password in clear text in the registry gives me nervous twitches. Me too. It's horrible, and I really dislike the idea. I can't imagine that Microsoft don't have a better solution to this. I talked to some Microsoft people at a

[HACKERS] SP-GiST for ranges based on 2d-mapping and quad-tree

Hackers, attached patch implements quad-tree on ranges. Some performance results in comparison with current GiST indexing. Index creation is slightly slower. Probably, it need some investigation. Search queries on SP-GiST use much more pages. However this comparison can be not really correct, beca

Re: [HACKERS] [COMMITTERS] pgsql: Add ERROR msg for GLOBAL/LOCAL TEMP is not yet implemented

Noah Misch writes: > On Wed, Jun 13, 2012 at 05:50:36PM -0400, Tom Lane wrote: > The SQL standard also distinguishes between global and local temporary > tables, where a local temporary table is only visible within a specific SQL > module, though its definition is still shared across session

Re: [HACKERS] Ability to listen on two unix sockets

On ons, 2012-06-13 at 15:25 +0200, Honza Horak wrote: > It seems unix_socket_directory could be turned into list and probably > renamed to unix_socket_directories, since it would be confusing if a > list value is in singular. Well, it would also be annoying to rename the parameter name for a mar

Re: [HACKERS] Ability to listen on two unix sockets

On tis, 2012-06-12 at 14:47 +0200, Honza Horak wrote: > This could be true in case all listening ports are equal, which I > guess isn't a good idea, because IIUIC the port number as a part of > the socket name is used for distinguish sockets of various postmasters > in the same directory. In that s

Re: [HACKERS] Ability to listen on two unix sockets

On mån, 2012-06-11 at 18:45 -0500, Michael Nolan wrote: > What about entries in pg_hba.conf? Will they need to be able to specify > both the directory and the port number? I think the philosophy behind pg_hba.conf is that you distinguish client *systems*. So one client might be Kerberos-capable,

[HACKERS] Re: [COMMITTERS] pgsql: Add ERROR msg for GLOBAL/LOCAL TEMP is not yet implemented

On Wed, Jun 13, 2012 at 05:50:36PM -0400, Tom Lane wrote: > Applied with some further wordsmithing on docs and comments. We can > still tweak this if anyone objects, of course, but I thought it'd > probably save work to get it in before the branch. Thanks. The SQL standard also distinguishes b

Re: [HACKERS] Ability to listen on two unix sockets

On mån, 2012-06-11 at 18:07 -0400, Tom Lane wrote: > Peter Eisentraut writes: > > On sön, 2012-06-10 at 17:24 -0400, Robert Haas wrote: > >>> Why would that matter? If you configure M ports and N Unix socket > >>> locations, you get M*N actual sockets created. > > >> ...I *seriously* doubt that

Re: [HACKERS] Ability to listen on two unix sockets

On ons, 2012-06-13 at 15:14 +0200, Honza Horak wrote: > Thus, I'd like to ask if anybody is aware of any issue connected with > simply patching pg_config_manual.h, same as Debian does it already? > For example, is there any piece of software, that simply rely on /tmp > location of the socket and

Re: [HACKERS] [PATCH] Support for foreign keys with arrays

On Wed, Jun 13, 2012 at 10:12:18PM +0200, Gabriele Bartolini wrote: > Our goal is to work on this patch from the next commit fest. > > What we are about to do for this commit fest is to split the previous > patch and send a small one just for the array_remove() and > array_replace() functions.

Re: [HACKERS] [COMMITTERS] pgsql: Add ERROR msg for GLOBAL/LOCAL TEMP is not yet implemented

Noah Misch writes: > On Wed, Jun 13, 2012 at 02:56:58PM -0400, Tom Lane wrote: >> Any thoughts on the wording of the WARNING message? > My patch used "GLOBAL is deprecated in temporary table creation", which still > seems fine to me. Here's an update based on this discussion. Applied with some

Re: [HACKERS] Is cachedFetchXidStatus provably valid?

On Wed, Jun 13, 2012 at 3:55 PM, Tom Lane wrote: > Merlin Moncure writes: >> It's probably an academic concern, but what happens if a backend saves >> off cachedFetchXidStatus and then sleeps for a very long time.  During >> that time an xid wraparound happens and the backend wakes up and >> happ

Re: [HACKERS] temporal support patch

On Wed, Jun 13, 2012 at 4:10 PM, Miroslav Šimulčík wrote: > I have working patch for postgresql version 9.0.4, but it needs refactoring > before i can submit it, because some parts don't > meet formatting requirements yet. And yes, changes are large, so it will be > better to discuss design first

Re: [HACKERS] temporal support patch

> > > * I'd very much like to see you make use of Range Types from 9.2; in > particular, TSTZRANGE would be much better than holding two timestamps. > If a standard requires you to display two timestamps in certain > situations, perhaps you could use ranges internally and display the > boundaries a

Re: [HACKERS] Is cachedFetchXidStatus provably valid?

Merlin Moncure writes: > It's probably an academic concern, but what happens if a backend saves > off cachedFetchXidStatus and then sleeps for a very long time. During > that time an xid wraparound happens and the backend wakes up and > happens to read another unhinted tuple with the same xid and

Re: [HACKERS] temporal support patch

2012/5/30 Jim Nasby > On 5/18/12 2:06 AM, Miroslav Šimulčík wrote: > >> - no data redundancy - in my extension current versions of entries are >> stored only once in original table (in table_log - entries are inserted to >> both original and log table) >> > > That's not necessarily a benefit... i

Re: [HACKERS] [PATCH] Support for foreign keys with arrays

Hi Noah, Il 10/06/12 22:53, Noah Misch ha scritto: This has bitrotted; please refresh. Also, please evaluate Peter's feedback: http://archives.postgresql.org/message-id/1333693277.32606.9.ca...@vanquo.pezone.net Our goal is to work on this patch from the next commit fest. What we are about t

Re: [HACKERS] temporal support patch

There would be no problem to make my solution compatible with SQL 2011, but the standard is not freely available. Can anybody provide me with this standard? 2012/5/20 Pavel Stehule > Hello > > 2012/5/18 Miroslav Šimulčík : > > Hello. > > > > SQL 2011 standard wasn't available in time I started t

[HACKERS] Re: [COMMITTERS] pgsql: Add ERROR msg for GLOBAL/LOCAL TEMP is not yet implemented

On Wed, Jun 13, 2012 at 02:56:58PM -0400, Tom Lane wrote: > Noah Misch writes: > > Given that, how about warning on GLOBAL only but having the documentation > > equally discourage use of both? > > Yeah, that's about what I was thinking, too. > > Any thoughts on the wording of the WARNING message

Re: [HACKERS] Re: [COMMITTERS] pgsql: Send new protocol keepalive messages to standby servers.

Tom Lane writes: > I thought I already pointed that out, but: we have *extensions*. What > we don't have is a convenient method of dealing with functions that need > to be migrated across extensions, or from an extension to core, between > one major release and the next. It would clearly be nice

Re: [HACKERS] Backup docs

Magnus Hagander writes: > I would like to see that page changed to list pg_basebackup as the > "default" way of doing base backups, and then list the "manual way" as > an option if you need more flexibility. +1 > I'd also like to add "pg_basebackup -x" under standalone hot backups, > again as th

Re: [HACKERS] [PATCH 14/16] Add module to apply changes from an apply-cache using low-level functions

On Wednesday, June 13, 2012 08:50:42 PM Christopher Browne wrote: > On Wed, Jun 13, 2012 at 7:28 AM, Andres Freund wrote: > > From: Andres Freund > > > > We decided to use low level functions to do the apply instead of > > producing sql statements containing the data (or using prepared > > stat

Re: [HACKERS] [COMMITTERS] pgsql: Add ERROR msg for GLOBAL/LOCAL TEMP is not yet implemented

Noah Misch writes: > Given that, how about warning on GLOBAL only but having the documentation > equally discourage use of both? Yeah, that's about what I was thinking, too. Any thoughts on the wording of the WARNING message? regards, tom lane -- Sent via pgsql-hackers

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

Robert Haas writes: > On Wed, Jun 13, 2012 at 1:04 PM, Tom Lane wrote: >> Robert Haas writes: >>> ! DETAIL: Character with value 0x0a must be escaped. >>> ! CONTEXT: JSON data, line 1: "abc >>> ! ... >>> >>> This seems an odd way to present this, especially if the goal is to >>> NOT include t

Re: [HACKERS] [PATCH 14/16] Add module to apply changes from an apply-cache using low-level functions

On Wed, Jun 13, 2012 at 7:28 AM, Andres Freund wrote: > From: Andres Freund > > We decided to use low level functions to do the apply instead of producing sql > statements containing the data (or using prepared statements) because both, > the > text conversion and the full executor overhead aren

[HACKERS] Re: [COMMITTERS] pgsql: Add ERROR msg for GLOBAL/LOCAL TEMP is not yet implemented

On Mon, Jun 11, 2012 at 09:18:39PM -0400, Robert Haas wrote: > I guess the > remaining question is whether to do it only for LOCAL TEMP tables or > also for GLOBAL TEMP ones. A survey of what other products do might > be of some value. Thanks for investigating. > Sybase ASE, which I include only

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

On Wed, Jun 13, 2012 at 1:04 PM, Tom Lane wrote: > Robert Haas writes: >> I like some of these changes - in particular, the use of errcontext(), >> but some of them still seem off. > >> ! DETAIL:  Token "'" is invalid. >> ! CONTEXT:  JSON data, line 1: '... > >> This doesn't make sense to me. > >

Re: [HACKERS] [RFC][PATCH] BDR Prototype startup instructions (not prematurely sent this time)

Hi, The patch as of yet doesn't contain how you actually can use the prototype... Obviously at this point its not very convenient: I have two config files: Node 1: port = 5501 wal_level = logical max_wal_senders = 10 wal_keep_segments = 200 multimaster_conninfo = 'port=5502 host=/tmp' multimaste

Re: [HACKERS] [RFC][PATCH] Logical Replication/BDR prototype and architecture

Hi, The patch as of yet doesn't contain how you actually can use the prototype... Obviously at this point its not very convenient: I have two config files: Node 1: port = 5501 wal_level = logical max_wal_senders = 10 wal_keep_segments = 200 multimaster_conninfo = 'port=5502 host=/tmp' multimaste

Re: [HACKERS] uncataloged tables are a vestigial husk

Robert Haas writes: > On Wed, Jun 13, 2012 at 1:13 PM, Tom Lane wrote: >>> The attached patch cleans it up by removing RELKIND_UNCATALOGED and >>> teaching RelationBuildLocalRelation() to set the relkind itself. >> I think there are probably some places to fix in the docs too. > catalogs.sgml d

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

The previous mail contained a patch with a mismerge caused by reording commits. Corrected version attached. Thanks to Steve Singer for noticing this quickly. Andres -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services From

Re: [HACKERS] [RFC][PATCH] Logical Replication/BDR prototype and architecture

On Wednesday, June 13, 2012 07:11:40 PM Steve Singer wrote: > On 12-06-13 07:27 AM, Andres Freund wrote: > > Its also available in the 'cabal-rebasing' branch on > > git.postgresql.org/users/andresfreund/postgres.git . That branch will > > modify history though. > > That branch has a merge error i

[HACKERS] Is cachedFetchXidStatus provably valid?

It's probably an academic concern, but what happens if a backend saves off cachedFetchXidStatus and then sleeps for a very long time. During that time an xid wraparound happens and the backend wakes up and happens to read another unhinted tuple with the same xid and a different commit status. Thi

Re: [HACKERS] uncataloged tables are a vestigial husk

On Wed, Jun 13, 2012 at 1:13 PM, Tom Lane wrote: >> The attached patch cleans it up by removing RELKIND_UNCATALOGED and >> teaching RelationBuildLocalRelation() to set the relkind itself. > > I think there are probably some places to fix in the docs too. catalogs.sgml doesn't include it in the li

Re: [HACKERS] [RFC][PATCH] Logical Replication/BDR prototype and architecture

On Wednesday, June 13, 2012 07:11:40 PM Steve Singer wrote: > On 12-06-13 07:27 AM, Andres Freund wrote: > > Its also available in the 'cabal-rebasing' branch on > > git.postgresql.org/users/andresfreund/postgres.git . That branch will > > modify history though. > > That branch has a merge error i

Re: [HACKERS] uncataloged tables are a vestigial husk

Robert Haas writes: > While working on some code today, I noticed that RELKIND_UNCATALOGED > appears to serve no useful purpose. In the few places where we check > for it at all, we treat it in exactly the same way as > RELKIND_RELATION. It seems that it's only purpose is to serve as a > placeho

Re: [HACKERS] [RFC][PATCH] Logical Replication/BDR prototype and architecture

On 12-06-13 07:27 AM, Andres Freund wrote: Its also available in the 'cabal-rebasing' branch on git.postgresql.org/users/andresfreund/postgres.git . That branch will modify history though. That branch has a merge error in f685a11ce43b9694cbe61ffa42e396c9fbc32b05 gcc -O2 -Wall -Wmissing-proto

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

Robert Haas writes: > I like some of these changes - in particular, the use of errcontext(), > but some of them still seem off. > ! DETAIL: Token "'" is invalid. > ! CONTEXT: JSON data, line 1: '... > This doesn't make sense to me. Well, the input is two single quotes and the complaint is tha

Re: [HACKERS] initdb and fsync

e much larger. > There are no updates to the initdb man page included in your patch, > which would be a suitable place to discuss this briefly. Thank you, I added that. Regards, Jeff Davis initdb-fsync-20120613.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-h

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

On Wed, Jun 13, 2012 at 12:27 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Jun 13, 2012 at 11:55 AM, Tom Lane wrote: >>> In any case, the proposed scheme for providing context requires that >>> you know where the error is before you can identify the context.  I >>> considered schemes tha

[HACKERS] uncataloged tables are a vestigial husk

While working on some code today, I noticed that RELKIND_UNCATALOGED appears to serve no useful purpose. In the few places where we check for it at all, we treat it in exactly the same way as RELKIND_RELATION. It seems that it's only purpose is to serve as a placeholder inside each newly-created

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

Robert Haas writes: > On Wed, Jun 13, 2012 at 11:55 AM, Tom Lane wrote: >> In any case, the proposed scheme for providing context requires that >> you know where the error is before you can identify the context. I >> considered schemes that would keep track of the last N characters or >> line br

Re: [HACKERS] Re: [GENERAL] pg_upgrade from 9.0.7 to 9.1.3: duplicate key pg_authid_oid_index

On Mon, Jun 04, 2012 at 10:16:45AM -0400, Bruce Momjian wrote: > > I think the checks that are actually needed here are (1) bootstrap > > superusers are named the same, and (2) there are no roles other than the > > bootstrap superuser in the new cluster. > > You are right that it is more complex t

Re: [HACKERS] [RFC][PATCH] Logical Replication/BDR prototype and architecture

On Wednesday, June 13, 2012 05:53:31 PM Robert Haas wrote: > On Wed, Jun 13, 2012 at 7:27 AM, Andres Freund wrote: > > Unless somebody objects I will add most of the individual marked as RFC > > to the current commitfest. I hope that with comments stemming from that > > round we can get several o

Re: [HACKERS] [PATCH 16/16] current version of the design document

Hi, On Wednesday, June 13, 2012 05:39:36 PM Merlin Moncure wrote: > On Wed, Jun 13, 2012 at 9:40 AM, Andres Freund wrote: > >> Let's take the case where I have N small-ish schema identical database > >> shards that I want to aggregate into a single warehouse -- something > >> that HS/SR currentl

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

On Wed, Jun 13, 2012 at 11:55 AM, Tom Lane wrote: > Andres Freund writes: >> On Wednesday, June 13, 2012 05:18:22 PM Robert Haas wrote: >>> According to my testing, the main cost of an exception block catching >>> a division-by-zero error is that of generating the error message, >>> primarily spr

Re: [HACKERS] 9.3devel branch

On Wed, Jun 13, 2012 at 11:57 AM, Tom Lane wrote: > Robert Haas writes: >> Seeing as we're about to start a CommitFest, I think it's about time >> to create the 9.3devel branch. > >> I'm happy to do it, but I believe it's usually Tom's gig. > > There are a couple open issues (like reversion of th

Re: [HACKERS] [PATCH 07/16] Log enough data into the wal to reconstruct logical changes from it if wal_level=logical

On Wednesday, June 13, 2012 05:27:06 PM Kevin Grittner wrote: > Andres Freund wrote: > > This adds a new wal_level value 'logical' > > > > Missing cases: > > - heap_multi_insert > > - primary key changes for updates > > - no primary key > > - LOG_NEWPAGE > > First, Wow! Thanks ;) I hope you will

Re: [HACKERS] 9.3devel branch

Robert Haas writes: > Seeing as we're about to start a CommitFest, I think it's about time > to create the 9.3devel branch. > I'm happy to do it, but I believe it's usually Tom's gig. There are a couple open issues (like reversion of that checkpoint-skipping patch) that probably ought to get dea

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

Andres Freund writes: > On Wednesday, June 13, 2012 05:18:22 PM Robert Haas wrote: >> According to my testing, the main cost of an exception block catching >> a division-by-zero error is that of generating the error message, >> primarily sprintf()-type stuff. The cost of scanning a multi-megabyte

Re: [HACKERS] [RFC][PATCH] Logical Replication/BDR prototype and architecture

On Wed, Jun 13, 2012 at 7:27 AM, Andres Freund wrote: > Unless somebody objects I will add most of the individual marked as RFC to the > current commitfest. I hope that with comments stemming from that round we can > get several of the patches into the first or second commitfest. As soon as the >

Re: [HACKERS] [PATCH 16/16] current version of the design document

On Wed, Jun 13, 2012 at 9:40 AM, Andres Freund wrote: > Hi Merlin, > > On Wednesday, June 13, 2012 04:21:12 PM Merlin Moncure wrote: >> On Wed, Jun 13, 2012 at 6:28 AM, Andres Freund > wrote: >> > +synchronized catalog at the decoding site. That adds some complexity to >> > use +cases like replic

Re: [HACKERS] [PATCH 07/16] Log enough data into the wal to reconstruct logical changes from it if wal_level=logical

Andres Freund wrote: > This adds a new wal_level value 'logical' > > Missing cases: > - heap_multi_insert > - primary key changes for updates > - no primary key > - LOG_NEWPAGE First, Wow! I look forward to the point where we can replace our trigger-based replication with this! Your "missi

[HACKERS] 9.3devel branch

Seeing as we're about to start a CommitFest, I think it's about time to create the 9.3devel branch. I'm happy to do it, but I believe it's usually Tom's gig. Tom? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

On Wednesday, June 13, 2012 05:18:22 PM Robert Haas wrote: > On Wed, Jun 13, 2012 at 11:06 AM, Andres Freund wrote: > > On Wednesday, June 13, 2012 05:03:38 PM Robert Haas wrote: > >> On Wed, Jun 13, 2012 at 10:35 AM, Tom Lane wrote: > >> > Robert Haas writes: > >> >> On Tue, Jun 12, 2012 at 9:

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

On Wed, Jun 13, 2012 at 11:06 AM, Andres Freund wrote: > On Wednesday, June 13, 2012 05:03:38 PM Robert Haas wrote: >> On Wed, Jun 13, 2012 at 10:35 AM, Tom Lane wrote: >> > Robert Haas writes: >> >> On Tue, Jun 12, 2012 at 9:52 PM, Tom Lane wrote: >> >>> The code for this is as attached.  Note

Re: [HACKERS] Ability to listen on two unix sockets

On Jun13, 2012, at 15:14 , Honza Horak wrote: > Since systemd with it's PrivateTmp feature is going to be used in more and > more distros, there will probably be a bigger need to solve in-accessible > default unix socket directory /tmp in the future. > > Thus, I'd like to ask if anybody is aware

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

On Wednesday, June 13, 2012 05:03:38 PM Robert Haas wrote: > On Wed, Jun 13, 2012 at 10:35 AM, Tom Lane wrote: > > Robert Haas writes: > >> On Tue, Jun 12, 2012 at 9:52 PM, Tom Lane wrote: > >>> The code for this is as attached. Note that I'd rip out the > >>> normal-path tracking of line bound

Re: [HACKERS] hint bit i/o reduction

On Wed, Jun 13, 2012 at 9:02 AM, Amit Kapila wrote: >> Yes, but only in the unhinted case -- in oltp workloads tuples get >> hinted fairly quickly so I doubt this would be a huge impact.  Hinted >> scans will work exactly as they do now.  In the unhinted case for OLTP >> a  few tests are added but

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

On Wed, Jun 13, 2012 at 10:35 AM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Jun 12, 2012 at 9:52 PM, Tom Lane wrote: >>> The code for this is as attached.  Note that I'd rip out the normal-path >>> tracking of line boundaries; it seems better to have a second scan of >>> the data in the e

Re: [HACKERS] [PATCH 16/16] current version of the design document

Hi Merlin, On Wednesday, June 13, 2012 04:21:12 PM Merlin Moncure wrote: > On Wed, Jun 13, 2012 at 6:28 AM, Andres Freund wrote: > > +synchronized catalog at the decoding site. That adds some complexity to > > use +cases like replicating into a different database or cross-version > > +replicatio

Re: [HACKERS] [COMMITTERS] pgsql: Mark JSON error detail messages for translation.

Robert Haas writes: > On Tue, Jun 12, 2012 at 9:52 PM, Tom Lane wrote: >> The code for this is as attached.  Note that I'd rip out the normal-path >> tracking of line boundaries; it seems better to have a second scan of >> the data in the error case and save the cycles in non-error cases. > Real

Re: [HACKERS] [PATCH 16/16] current version of the design document

On Wed, Jun 13, 2012 at 6:28 AM, Andres Freund wrote: > +synchronized catalog at the decoding site. That adds some complexity to use > +cases like replicating into a different database or cross-version > +replication. For those it is relatively straight-forward to develop a proxy > pg > +instance

Re: [HACKERS] WIP patch for Todo Item : Provide fallback_application_name in contrib/pgbench, oid2name, and dblink

I have created the patch by including fallback_application_name for dblink as well. In this I have used the name of fallback_application_name as dblink. Please let me know your suggestions regarding the same. -Original Message- From: Robert Haas [mailto:robertmh...@gmail.com] Sent: Wedne

Re: [HACKERS] hint bit i/o reduction

> Yes, but only in the unhinted case -- in oltp workloads tuples get > hinted fairly quickly so I doubt this would be a huge impact. Hinted > scans will work exactly as they do now. In the unhinted case for OLTP > a few tests are added but that's noise compared to the other stuff > going on. I

Re: [HACKERS] Ability to listen on two unix sockets

On 06/10/2012 03:41 PM, Robert Haas wrote: On Sun, Jun 10, 2012 at 8:36 AM, Tom Lane wrote: Peter Eisentraut writes: On lör, 2012-06-09 at 18:26 -0400, Tom Lane wrote: That's not actually quite the same thing as what I suggest above. Currently, unix_socket_directory *overrides* the compiled-

Re: [HACKERS] hint bit i/o reduction

On Wed, Jun 13, 2012 at 3:57 AM, Amit Kapila wrote: >> 1) Keep track # times the last transaction id was repeatedly seen in >> tqual.c (resetting as soon as a new xid was touched.  We can do this >> just for xmin, or separately for both xmin and xmax. > > Will this be done when we see a new xid du

Re: [HACKERS] Ability to listen on two unix sockets

On 06/10/2012 12:37 AM, Peter Eisentraut wrote: On sön, 2012-06-10 at 00:25 +0200, Andres Freund wrote: We already have the ability to configure the Unix socket directory in postgresql.conf. All you need to do is turn that into a list. That doesn't help libpq using clients. There is no mand

[HACKERS] [PATCH 05/16] Preliminary: Introduce Bgworker process

From: Simon Riggs Early prototype that allows for just 1 bgworker which calls a function called do_applyprocess(). Expect major changes in this, but not in ways that would effect the apply process. --- src/backend/postmaster/Makefile |4 +- src/backend/postmaster/bgworker.c

[HACKERS] [PATCH 13/16] Introduction of pair of logical walreceiver/sender

From: Andres Freund A logical WALReceiver is started directly by Postmaster when we enter PM_RUN state and the new parameter multimaster_conninfo is set. For now only one of those is started, but the code doesn't rely on that. In future multiple ones should be allowed. To transfer that data a ne

[HACKERS] [PATCH 14/16] Add module to apply changes from an apply-cache using low-level functions

From: Andres Freund We decided to use low level functions to do the apply instead of producing sql statements containing the data (or using prepared statements) because both, the text conversion and the full executor overhead aren't introduce a significant overhead which is unneccesary if youre u

[HACKERS] [PATCH 15/16] Activate/Implement the "apply process" which applies received updates from another node

From: Andres Freund One apply process currently can only apply changes from one database in another cluster (with a specific node_id). Currently synchronous_commit=off is statically set in the apply process because after a crash we can safely recover all changes which we didn't apply so there is

[HACKERS] [PATCH 16/16] current version of the design document

From: Andres Freund --- src/backend/replication/logical/DESIGN | 209 1 file changed, 209 insertions(+) create mode 100644 src/backend/replication/logical/DESIGN diff --git a/src/backend/replication/logical/DESIGN b/src/backend/replication/logical/DESIGN new

[HACKERS] [PATCH 06/16] Add support for a generic wal reading facility dubbed XLogReader

From: Andres Freund Features: - streaming reading/writing - filtering - reassembly of records Reusing the ReadRecord infrastructure in situations where the code that wants to do so is not tightly integrated into xlog.c is rather hard and would require changes to rather integral parts of the reco

[HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

From: Andres Freund One solution to avoid loops when doing wal based logical replication in topologies which are more complex than one unidirectional transport is introducing the concept of a 'origin_id' into the wal stream. Luckily there is some padding in the XLogRecord struct that allows us to

[HACKERS] [PATCH 11/16] Add infrastructure for manipulating multiple streams of wal on a segment handling level

From: Andres Freund For that add a 'node_id' parameter to most commands dealing with wal segments. A node_id thats 'InvalidMultimasterNodeId' references local wal, every other node_id referes to wal in a new pg_lcr directory. Using duplicated code would reduce the impact of that change but the l

[HACKERS] [PATCH 09/16] Decode wal (with wal_level=logical) into changes in an ApplyCache instance

From: Andres Freund This requires an up2date catalog and can thus only be run on a replica. Missing: - HEAP_NEWPAGE support - HEAP2_MULTI_INSERT support - DDL integration. *No* ddl, including TRUNCATE is possible atm --- src/backend/replication/logical/Makefile |2 +- src/backend/replicatio

[HACKERS] [PATCH 12/16] Add state to keep track of logical replication

From: Andres Freund In order to have restartable replication with minimal additional writes its very useful to know up to which point we have replayed/received changes from a foreign node. One representation of that is the lsn of changes at the originating cluster. We need to keep track of the

[HACKERS] [PATCH 07/16] Log enough data into the wal to reconstruct logical changes from it if wal_level=logical

From: Andres Freund This adds a new wal_level value 'logical' Missing cases: - heap_multi_insert - primary key changes for updates - no primary key - LOG_NEWPAGE --- src/backend/access/heap/heapam.c| 135 --- src/backend/access/transam/xlog.c |1 +

[HACKERS] [PATCH 01/16] Overhaul walsender wakeup handling

From: Andres Freund The previous coding could miss xlog writeouts at several places. E.g. when wal was written out by the background writer or even after a commit if synchronous_commit=off. This could lead to delays in sending data to the standby of up to 7 seconds. To fix this move the responsi

[HACKERS] [PATCH 08/16] Introduce the ApplyCache module which can reassemble transactions from a stream of interspersed changes

From: Andres Freund The individual changes need to be identified by an xid. The xid can be a subtransaction or a toplevel one, at commit those can be reintegrated by doing a k-way mergesort between the individual transaction. Callbacks for apply_begin, apply_change and apply_commit are provided

[HACKERS] [PATCH 02/16] Add zeroRecPtr as a shortcut for initializing a local variable to {0, 0}

From: Andres Freund This is locally defined in lots of places and would get introduced frequently in the next commits. It is expected that this can be defined in a header-only manner as soon as the XLogInsert scalability groundwork from Heikki gets in. --- src/backend/access/transam/xlogutils.c

[HACKERS] [PATCH 04/16] Add embedded list interface (header only)

From: Andres Freund Adds a single and a double linked list which can easily embedded into other datastructures and can be used without any additional allocations. Problematic: It requires USE_INLINE to be used. It could be remade to fallback to to externally defined functions if that is not avai

[HACKERS] [PATCH 03/16] Add a new syscache to fetch a pg_class entry via its relfilenode

From: Andres Freund This patch is problematic because formally indexes used by syscaches needs to be unique, this one is not though because of 0/InvalidOids entries for nailed/shared catalog entries. Those values aren't allowed to be queried though. It might be nicer to add infrastructure to do

[HACKERS] [RFC][PATCH] Logical Replication/BDR prototype and architecture

Hi everyone, This mail contains the highlevel design description of how our prototype of in-core logical replication works. The individual patches will be posted as replies to this email. I obviously welcome all sorts of productive comments to both the individual patches and the architecture. Unl

Re: [HACKERS] initdb and fsync

On tis, 2012-06-12 at 21:09 -0700, Jeff Davis wrote: > On Sun, 2012-03-25 at 19:59 -0700, Jeff Davis wrote: > > On Sat, 2012-03-17 at 17:48 +0100, Cédric Villemain wrote: > > > I agree with Andres. > > > > > > > > > I believe we should use sync_file_range (_before?) with linux. > > > > > > And w

Re: [HACKERS] Minimising windows installer password confusion

On Jun13, 2012, at 11:10 , Dave Page wrote: > On Wed, Jun 13, 2012 at 2:12 AM, Craig Ringer > wrote: >> >> Users don't remember passwords, though. It's one of those constants, and is >> why practically every web site etc out there offers password recovery. >> >> The installer IMO needs to store

Re: [HACKERS] Minimising windows installer password confusion

On Wed, Jun 13, 2012 at 3:07 AM, Craig Ringer wrote: > On 06/13/2012 01:19 AM, Sachin Srivastava wrote: >> >> >> On Tue, Jun 12, 2012 at 7:43 PM, Dave Page > > wrote: >> >>    On Tue, Jun 12, 2012 at 2:57 PM, Robert Haas >    > wrote: > > >>

  1   2   >