[HACKERS] [PATCH 0/4] COPY to a UDF: COPY ... TO FUNCTION ...

2009-11-23 Thread Daniel Farina
is Truviso, Inc in 2009. Daniel Farina (4): Add COPY ... TO FUNCTION ... support Add tests for COPY ... TO FUNCTION ... Add dblink functions for use with COPY ... TO FUNCTION ... Add tests to dblink covering use of COPY TO FUNCTION contrib/dblink/dblink.c | 190

[HACKERS] [PATCH 3/4] Add dblink functions for use with COPY ... TO FUNCTION ...

2009-11-23 Thread Daniel Farina
connection is still in the COPY state, one -- can even copy some more data in multiple steps... COPY bar_2 TO FUNCTION dblink_copy_write; SELECT dblink_copy_end(); finally: SELECT dblink_copy_reset('myconn'); Signed-off-by: Daniel Farina dfar...@truviso.com

[HACKERS] [PATCH 2/4] Add tests for COPY ... TO FUNCTION ...

2009-11-23 Thread Daniel Farina
Signed-off-by: Daniel Farina dfar...@truviso.com --- src/test/regress/input/copy.source | 38 +++ src/test/regress/output/copy.source | 69 +++ src/test/regress/regress.c | 56 3 files changed, 163

[HACKERS] [PATCH 1/4] Add COPY ... TO FUNCTION ... support

2009-11-23 Thread Daniel Farina
and then reusing it, as opposed to simply using one of the convenience functions in the fmgr. - Add and expose the makeNameListFromRangeVar procedure to src/catalog/namespace.c, the inverse of makeRangeVarFromNameList. Signed-off-by: Daniel Farina dfar...@truviso.com --- src/backend/catalog

[HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-11-23 Thread Daniel Farina
Signed-off-by: Daniel Farina dfar...@truviso.com --- contrib/dblink/expected/dblink.out | 272 contrib/dblink/sql/dblink.sql | 112 +++ 2 files changed, 384 insertions(+), 0 deletions(-) diff --git a/contrib/dblink/expected/dblink.out b

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-11-23 Thread Daniel Farina
On Mon, Nov 23, 2009 at 2:16 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Nov 23, 2009 at 4:34 PM, Daniel Farina dfar...@truviso.com wrote: Signed-off-by: Daniel Farina dfar...@truviso.com Thanks for the patch.  You may want to take a look at this: http://wiki.postgresql.org/wiki

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-11-23 Thread Daniel Farina
On Mon, Nov 23, 2009 at 3:46 PM, Andrew Dunstan and...@dunslane.net wrote: I recently found myself trying to push data through dblink() and ended up writing code to make a call to the target to call a function which called back to the source to select the data and insert it. The speedup was

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-11-23 Thread Daniel Farina
On Mon, Nov 23, 2009 at 4:03 PM, Daniel Farina dfar...@truviso.com wrote: Yes.  Take a look at the tests introduced to core PostgeSQL (see patch 2), where instead of returning a text[] I return just a single text of the verbatim output of the copy.  You could imagine making that an SRF instead

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-11-23 Thread Daniel Farina
On Mon, Nov 23, 2009 at 4:20 PM, Tom Lane t...@sss.pgh.pa.us wrote: pgsql-hackers had some preliminary discussions a couple months back on refactoring COPY to allow things like this --- see the thread starting here: http://archives.postgresql.org/pgsql-hackers/2009-09/msg00486.php While I

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-11-24 Thread Daniel Farina
On Tue, Nov 24, 2009 at 12:29 AM, Hannu Krosing ha...@krosing.net wrote: COPY stdin TO udf(); If stdin becomes (is?) a legitimate source of records, then this patch will Just Work. The patch is already quite useful in the COPY (SELECT ...) TO FUNCTION ... scenario. COPY udf() FROM stdin;

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-11-24 Thread Daniel Farina
On Tue, Nov 24, 2009 at 12:38 AM, Hannu Krosing ha...@2ndquadrant.com wrote: COPY func(rowtype) FROM stdin; I didn't consider rowtype...I did consider a type list, such as: COPY func(typea, typeb, typec) FROM ... Which would then operate just like a table, but be useless for the data-cleaning

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-11-24 Thread Daniel Farina
On Tue, Nov 24, 2009 at 2:10 AM, Pavel Stehule pavel.steh...@gmail.com wrote: Hello I thing, so this patch is maybe good idea. I am missing better function specification. Specification by name isn't enough - we can have a overloaded functions. This syntax doesn't allow to use explicit cast -

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-11-24 Thread Daniel Farina
On Tue, Nov 24, 2009 at 2:50 AM, Hannu Krosing ha...@2ndquadrant.com wrote: Can't you use existing aggregate function design ? CREATE AGGREGATE name ( input_data_type [ , ... ] ) (    SFUNC = sfunc,    STYPE = state_data_type    [ , FINALFUNC = ffunc ]    [ , INITCOND = initial_condition ]

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-11-24 Thread Daniel Farina
On Tue, Nov 24, 2009 at 2:51 AM, Daniel Farina drfar...@gmail.com wrote: On Tue, Nov 24, 2009 at 2:50 AM, Hannu Krosing ha...@2ndquadrant.com wrote: Can't you use existing aggregate function design ? CREATE AGGREGATE name ( input_data_type [ , ... ] ) (    SFUNC = sfunc,    STYPE

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-11-24 Thread Daniel Farina
On Tue, Nov 24, 2009 at 3:25 AM, Hannu Krosing ha...@2ndquadrant.com wrote: On Tue, 2009-11-24 at 02:56 -0800, Daniel Farina wrote: On Tue, Nov 24, 2009 at 2:51 AM, Daniel Farina drfar...@gmail.com wrote: On Tue, Nov 24, 2009 at 2:50 AM, Hannu Krosing ha...@2ndquadrant.com wrote: Can't

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-11-24 Thread Daniel Farina
On Tue, Nov 24, 2009 at 4:37 AM, Pavel Stehule pavel.steh...@gmail.com wrote: 2009/11/24 Daniel Farina drfar...@gmail.com: On Tue, Nov 24, 2009 at 2:10 AM, Pavel Stehule pavel.steh...@gmail.com wrote: Hello I thing, so this patch is maybe good idea. I am missing better function

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-11-24 Thread Daniel Farina
On Tue, Nov 24, 2009 at 8:45 PM, Pavel Stehule pavel.steh...@gmail.com wrote: It depends on design. I don't thing so internal is necessary. It is just wrong design. Depends on how lean you want to be when doing large COPY...right now the cost is restricted to having to call a function pointer

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-11-24 Thread Daniel Farina
On Tue, Nov 24, 2009 at 9:13 PM, Jeff Davis pg...@j-davis.com wrote: I still don't see any reason to force it to be record by record though. If the point is to push data from a table into a remote table, why should the copied data be translated out of binary format into a record, and then

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-11-24 Thread Daniel Farina
On Tue, Nov 24, 2009 at 9:35 PM, Pavel Stehule pavel.steh...@gmail.com wrote: 2009/11/25 Daniel Farina drfar...@gmail.com: On Tue, Nov 24, 2009 at 8:45 PM, Pavel Stehule pavel.steh...@gmail.com wrote: It depends on design. I don't thing so internal is necessary. It is just wrong design

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-11-24 Thread Daniel Farina
On Tue, Nov 24, 2009 at 10:23 PM, Jeff Davis pg...@j-davis.com wrote: On Wed, 2009-11-25 at 06:35 +0100, Pavel Stehule wrote: I believe so using an internal minimalize necessary changes in COPY implementation. Using a funcapi needs more work inside COPY -  you have to take some functionality

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-11-25 Thread Daniel Farina
On Tue, Nov 24, 2009 at 10:39 PM, Pavel Stehule pavel.steh...@gmail.com wrote: yes - it is two features - and should be solved independently There are some common problems though. I was thinking about this with some mind towards my existing mental model of thinking of specifying some parameters

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-11-25 Thread Daniel Farina
On Wed, Nov 25, 2009 at 9:35 PM, Andrew Dunstan and...@dunslane.net wrote: On Wed, November 25, 2009 3:56 pm, Jeff Davis wrote: I worry that we're getting further away from the original problem. Let's allow functions to get the bytes of data from a COPY, like the original proposal. I am not

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-11-26 Thread Daniel Farina
On Thu, Nov 26, 2009 at 6:13 PM, David Fetter da...@fetter.org wrote: It'd be nice to make this available to the next revision of DBI-Link and it'll be pretty handy for our SQL/MED whenever that happens. Okay, so this thread sort of wandered into how we could refactor other elements of COPY.

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-11-29 Thread Daniel Farina
On Sun, Nov 29, 2009 at 6:35 PM, Jeff Davis pg...@j-davis.com wrote: What if the network buffer is flushed in the middle of a line? Is that possible, or is there a guard against that somewhere? What do you mean? They both catenate onto one stream of bytes, it shouldn't matter where the flush

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-12-05 Thread Daniel Farina
On Mon, Nov 30, 2009 at 12:14 PM, Greg Smith g...@2ndquadrant.com wrote: Jeff Davis wrote: COPY target FROM FUNCTION foo() WITH RECORDS; In what format would the records be? As a not-quite aside, I think I have a better syntax suggestion. I have recently been digging around in the grammar

Re: [HACKERS] A sniffer for the buffer

2009-12-07 Thread Daniel Farina
On Sun, Dec 6, 2009 at 9:04 AM, Greg Smith g...@2ndquadrant.com wrote: And that won't work at all.  Buffer is a structure, not an integer.  You need to wait until it's been locked, then save the same data as on the read side (relation and block number) from inside the structure.  You probably

Re: [HACKERS] questions about concurrency control in Postgresql

2009-12-07 Thread Daniel Farina
2009/12/7 黄晓骋 huangxcl...@gmail.com: Hello, I think in Postgresql, concurrency control acts like this: tuple's infomask shows if it is being updated. If it is being updated now, the latter transaction should reread the tuple and get the newer tuple. During the progress of getting the newer

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-12-29 Thread Daniel Farina
On Tue, Dec 29, 2009 at 5:57 PM, Robert Haas robertmh...@gmail.com wrote: I am very fuzzy on where we stand with this patch.  There's a link to the initial version on the commitfest application, but there has been so much discussion since then that I think there are probably some revisions to

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-12-29 Thread Daniel Farina
On Tue, Dec 29, 2009 at 6:48 PM, Robert Haas robertmh...@gmail.com wrote: I think there's clear support for a version of COPY that returns rows like a SELECT statement, particularly for use with CTEs.  There seems to be support both for a mode that returns text[] or something like it and also

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-12-29 Thread Daniel Farina
On Tue, Dec 29, 2009 at 8:11 PM, Robert Haas robertmh...@gmail.com wrote: It might be possible to do this without introducing a notion of an explicit object, but there are a couple of problems with that.  First, it requires the user to specify a lot of details on every COPY invocation, which

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-12-29 Thread Daniel Farina
On Tue, Dec 29, 2009 at 9:11 PM, Robert Haas robertmh...@gmail.com wrote: I think we should try to put an interface layer in place in the first iteration.  I don't really see this as having much value without that.  If we implement this strictly as a series of COPY options, we're going to be

Re: [HACKERS] [PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION

2009-12-29 Thread Daniel Farina
On Tue, Dec 29, 2009 at 9:46 PM, Robert Haas robertmh...@gmail.com wrote: Sure.  If you think you can make it work using bytea, that seems clearly better than using an internal type, all things being equal. I think both are perfectly viable and can be supported simultaneously, actually...I

[HACKERS] Windowing Qual Pushdown

2010-03-20 Thread Daniel Farina
In the function subquery_is_pushdown_safe, there is an immediate false returned if the subquery has a windowing function. While that seems true in general, are there cases where we can push down a qual if it is on the partitioning key? Or do NULLs or some other detail get in the way? fdr --

Re: [HACKERS] Windowing Qual Pushdown

2010-03-23 Thread Daniel Farina
On Tue, Mar 23, 2010 at 12:19 AM, Hitoshi Harada umi.tan...@gmail.com wrote: If you implement that optimization, we need have kind of implicit, homologous qual information. Sure, it's possible. I'm not sure precisely what you mean here. Do you predict the mechanism will be complicated? It's

Re: [HACKERS] Windowing Qual Pushdown

2010-03-23 Thread Daniel Farina
On Tue, Mar 23, 2010 at 12:40 AM, Hitoshi Harada umi.tan...@gmail.com wrote: I believe the changes will probably not be 2-3 lines (ie. a member added to Query structure, etc) if I try it. But the optimizer part is too complicated to me so that I am not sure, either. My idea above is that the

Re: [HACKERS] Windowing Qual Pushdown

2010-03-23 Thread Daniel Farina
On Tue, Mar 23, 2010 at 8:23 AM, Tom Lane t...@sss.pgh.pa.us wrote: The real question is what benefit you expect to get.  If the filter condition can't be pushed below the window functions (which AFAICS Even on the partition key? Right now if you define a view with a windowing + PARTITION BY

Re: [HACKERS] Performance Enhancement/Fix for Array Utility Functions

2010-06-01 Thread Daniel Farina
On Wed, Mar 31, 2010 at 9:47 AM, Mike Lewis mikelikes...@gmail.com wrote: Thanks. Added it. https://commitfest.postgresql.org/action/patch_view?id=292 I have reviewed this patch; this is my review: Regression tests pass with assertions enabled. Performance gains reported by author confirmed.

Re: [HACKERS] exporting raw parser

2010-06-07 Thread Daniel Farina
On Wed, May 26, 2010 at 6:02 PM, Tatsuo Ishii is...@postgresql.org wrote: I'm thinking about exporting the raw parser and related modules as a C library. Though this will not be an immediate benefit of PostgreSQL itself, it will be a huge benefit for any PostgreSQL applications/middle ware

Re: [HACKERS] Error with GIT Repository

2010-06-30 Thread Daniel Farina
On Fri, Jun 11, 2010 at 10:19 AM, Magnus Hagander mag...@hagander.net wrote: Especially if someone has a clue how to do it. The last time I fixed it by runnin repack, but that didn't work this time. I have no clue why it's asking for a file that doesn't exist. Does the repo run  

Re: [HACKERS] documentation for committing with git

2010-07-28 Thread Daniel Farina
On Wed, Jul 21, 2010 at 9:22 AM, Robert Haas robertmh...@gmail.com wrote: On the other hand, if you have technical corrections, or if you have suggestions on how to do the same things better (rather than suggestions on what to do differently), that would be greatly appreciated. Somewhere in

Re: [HACKERS] documentation for committing with git

2010-08-04 Thread Daniel Farina
On Wed, Aug 4, 2010 at 6:29 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: All those issues can be avoided if you only run git gc when all the working directories are in a clean state, with no staged but uncommitted changes or other funny things. I can live with that gun tied

Re: [HACKERS] Extensions and 9.2

2011-12-23 Thread Daniel Farina
On Wed, Dec 21, 2011 at 5:46 AM, Robert Haas robertmh...@gmail.com wrote: Assuming the command in question can be stuffed inside a function, the most you're gaining is a little notational convenience I can answer that one (why a full-blown mechanism for a notational convenience). It has

Re: [HACKERS] backup_label during crash recovery: do we know how to solve it?

2011-12-29 Thread Daniel Farina
On Sat, Dec 3, 2011 at 8:04 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: At the moment, if the situation is ambiguous, the system assumes that you're restoring from a backup. What your suggestion amounts to is to reverse tht assumption, and assume instead that you're doing

Re: [HACKERS] backup_label during crash recovery: do we know how to solve it?

2012-01-01 Thread Daniel Farina
On Sun, Jan 1, 2012 at 5:18 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: That's awfully complicated. If we're going to require co-operation from the backup/archiving software, we might as well just change the procedure so that backup_label is not stored in the data

Re: [HACKERS] backup_label during crash recovery: do we know how to solve it?

2012-01-01 Thread Daniel Farina
On Sun, Jan 1, 2012 at 6:13 AM, Magnus Hagander mag...@hagander.net wrote: It also doesn't affect backups taken through pg_basebackup - but I guess you have good reasons for not being able to use that? Parallel archiving/de-archiving and segmentation of the backup into pieces and rate limiting

[HACKERS] pg_internal.init and an index file have the same inode

2012-01-03 Thread Daniel Farina
I'm not sure if this is an XFS problem, or Postgres. There's enough suspicious evidence that it's too hard to say. Today, I get an interesting issue raised whereby a reasonably simple query fails on a system that does take successful pg_dumps regularly. To make a short story shorter, I end up

[HACKERS] Review of: pg_stat_statements with query tree normalization

2012-01-15 Thread Daniel Farina
I've *finally* gotten around to reviewing this patch. My first step was to de-bitrot it very slightly. More on that in a moment. After that, I tried using it. Installation worked nicely -- I did CREATE EXTENSION and then tried reading from pg_stat_statements. I was then given an error message

[HACKERS] Should we add crc32 in libpgport?

2012-01-16 Thread Daniel Farina
I have been working with xlogdump and noticed that unfortunately it cannot be installed without access to a postgres build directory, which makes the exported functionality in src/include/utils/pg_crc.h useless unless one has access to pg_crc.o -- which would only happen if a build directory is

Re: [HACKERS] Review of: pg_stat_statements with query tree normalization

2012-01-16 Thread Daniel Farina
On Mon, Jan 16, 2012 at 3:53 PM, Tom Lane t...@sss.pgh.pa.us wrote: Well, short of seeing an acceptable patch for the larger thing, I don't want to accept a patch to add that field to Const, because I think it's a kluge.  I'm still feeling that there must be a better way ... Hm. Maybe it is

Re: [HACKERS] Should we add crc32 in libpgport?

2012-01-16 Thread Daniel Farina
On Mon, Jan 16, 2012 at 4:56 PM, Daniel Farina dan...@heroku.com wrote: I have been working with xlogdump and noticed that unfortunately it cannot be installed without access to a postgres build directory, which makes the exported functionality in src/include/utils/pg_crc.h useless unless one

Re: [HACKERS] Should we add crc32 in libpgport?

2012-01-16 Thread Daniel Farina
On Mon, Jan 16, 2012 at 6:18 PM, Robert Haas robertmh...@gmail.com wrote: I think you make a compelling case. That's enough for me to just do it. Expect a patch soon. -- fdr -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Inline Extension

2012-01-20 Thread Daniel Farina
On Thu, Jan 19, 2012 at 8:15 AM, Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Frankly I don't see the point of this. If the extension is an independent piece of (SQL) code, developed separately from an application, with its own lifecycle, a

Re: [HACKERS] Inline Extension

2012-01-20 Thread Daniel Farina
On Fri, Jan 20, 2012 at 2:48 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Even if you give the version number in the CREATE EXTENSION command, it's by convention that people actually maintain a sane versioning policy. If people don't take version management seriously, you

Re: [HACKERS] Inline Extension

2012-01-22 Thread Daniel Farina
On Fri, Jan 20, 2012 at 3:33 PM, Daniel Farina dan...@heroku.com wrote: On Fri, Jan 20, 2012 at 2:48 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Even if you give the version number in the CREATE EXTENSION command, it's by convention that people actually maintain a sane

Re: [HACKERS] Inline Extension

2012-01-23 Thread Daniel Farina
On Sun, Jan 22, 2012 at 8:42 PM, Robert Haas robertmh...@gmail.com wrote: On Sun, Jan 22, 2012 at 3:20 PM, Daniel Farina dan...@heroku.com wrote: A few anecdotes does not constitute evidence, but it does look like some people pay attention to any additional versioning foothold they can get

Re: [HACKERS] Inline Extension

2012-01-23 Thread Daniel Farina
On Mon, Jan 23, 2012 at 8:17 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: Ok, but then, what about .so files?  Wouldn't it make sense to be able to ship also the executable modules needed, and if not, why not? Now you can dump/restore any

Re: [HACKERS] Hot standby fails if any backend crashes

2012-02-03 Thread Daniel Farina
On Thu, Feb 2, 2012 at 8:48 PM, Tom Lane t...@sss.pgh.pa.us wrote: It's a bit disturbing that nobody has reported this from the field yet. Seems to imply that hot standby isn't being used much. I have seen this, but didn't get to dig in, as I thought it could be a problem from other things done

Re: [HACKERS] Should we add crc32 in libpgport?

2012-02-03 Thread Daniel Farina
On Tue, Jan 31, 2012 at 3:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: Daniel Farina dan...@heroku.com writes: On Tue, Jan 17, 2012 at 2:14 AM, Daniel Farina dan...@heroku.com wrote: See the attached patch.  It has a detailed cover letter/comment at the top of the file. I have amended

Re: [HACKERS] Should we add crc32 in libpgport?

2012-02-22 Thread Daniel Farina
On Thu, Feb 16, 2012 at 6:09 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Feb 3, 2012 at 7:33 PM, Daniel Farina dan...@heroku.com wrote: Ah, yes, I think my optimizations were off when building, or something.  I didn't get such verbosity at first, and then I remember doing something

[HACKERS] pg_stat_statements normalization: re-review

2012-02-23 Thread Daniel Farina
Hello again, I'm reviewing the revised version of pg_stat_statements again. In particular, this new version has done away with the mechanical but invasive change to the lexing that preserved *both* the position and length of constant values. (See

Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-23 Thread Daniel Farina
On Tue, Feb 21, 2012 at 1:34 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Sandro Santilli s...@keybit.net writes: Please see the inline extension thread where answers to your problem have been discussed. I'm pretty sure Sandro is hacking PostGIS, so inline extensions are of no help here.

[HACKERS] ISO8601 nitpicking

2012-02-23 Thread Daniel Farina
As it turns out, evidence would suggests that the ISO output in Postgres isn't, unless there's an ISO standard for date and time that is referring to other than 8601. It does not permit use of a space between the date and the time, as seen in: SELECT now(); now

Re: [HACKERS] pg_stat_statements normalization: re-review

2012-02-24 Thread Daniel Farina
]: https://github.com/fdr/postgres/tree/wrench-pgss From c3ad77375062cfbeee7d4ce7e0fe274a5db76453 Mon Sep 17 00:00:00 2001 From: Daniel Farina dan...@heroku.com Date: Fri, 24 Feb 2012 01:31:54 -0800 Subject: [PATCH] Introduce NodeKey as a service to extensions in the backend Signed-off-by: Daniel

Re: [HACKERS] pg_stat_statements normalization: re-review

2012-02-24 Thread Daniel Farina
On Fri, Feb 24, 2012 at 3:14 AM, Daniel Farina dan...@heroku.com wrote: At ExecutorFinish (already hookable) all NodeKeys remembered by an extension should be invalidated, as that memory is free and ready to be used again. I think this statement is false; I thought it was true because

Re: [HACKERS] ISO8601 nitpicking

2012-02-24 Thread Daniel Farina
On Fri, Feb 24, 2012 at 4:45 AM, Peter Eisentraut pete...@gmx.net wrote: On tor, 2012-02-23 at 23:41 -0800, Daniel Farina wrote: As it turns out, evidence would suggests that the ISO output in Postgres isn't, unless there's an ISO standard for date and time that is referring to other than 8601

Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-24 Thread Daniel Farina
On Fri, Feb 24, 2012 at 10:21 AM, Peter Eisentraut pete...@gmx.net wrote: On fre, 2012-02-24 at 17:26 +0100, Sandro Santilli wrote: We don't initdb with PostGIS regression testing framework but I've considered doing it for this specific case and it stroke me that even then we couldn't control

Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-24 Thread Daniel Farina
On Fri, Feb 24, 2012 at 6:31 PM, Andrew Dunstan and...@dunslane.net wrote: Really? Here's what I just got on a severely under-resourced SL6 VM: 1.5s doesn't seem terribly slow. You are right. Come to think of it, I do seem to recall that initdb got some speed improvements; these were in 8.3

Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-28 Thread Daniel Farina
On Sun, Feb 26, 2012 at 7:36 AM, Peter Eisentraut pete...@gmx.net wrote: On lör, 2012-02-25 at 14:21 +0100, Christoph Berg wrote: Well, I'm trying to invoke the extension's make check target at extension build time. I do have a temporary installation I own somehwere in my $HOME, but that is

Re: [HACKERS] Re: pg_stat_statements normalisation without invasive changes to the parser (was: Next steps on pg_stat_statements normalisation)

2012-02-29 Thread Daniel Farina
On Mon, Feb 27, 2012 at 4:26 AM, Peter Geoghegan pe...@2ndquadrant.com wrote: This does not appear to have any user-visible effect on caret position for all variations in coercion syntax, while giving me everything that I need. I had assumed that we were relying on things being this way, but

Re: [HACKERS] Re: pg_stat_statements normalisation without invasive changes to the parser (was: Next steps on pg_stat_statements normalisation)

2012-03-01 Thread Daniel Farina
On Thu, Mar 1, 2012 at 1:27 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: My expectation is that this feature will make life a lot easier for a lot of Postgres users. Yes. It's hard to overstate the apparent utility of this feature in the general category of visibility and profiling. --

Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-03-02 Thread Daniel Farina
On Fri, Mar 2, 2012 at 10:37 AM, Peter Eisentraut pete...@gmx.net wrote: On tis, 2012-02-28 at 11:00 -0800, Daniel Farina wrote: I'd really like to support libraries (C or otherwise) of multiple versions at the same time, when the underlying library permits. What's preventing you from doing

Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-03-02 Thread Daniel Farina
On Fri, Mar 2, 2012 at 1:50 PM, Robert Haas robertmh...@gmail.com wrote: But is it unsurmountable? -- dlsym returns a function pointer, and one would build up the operator table for the version of the extension at hand, so one might have ltree version 1.01 and ltree version 2.3 fields in the

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-05 Thread Daniel Farina
On Mon, Mar 5, 2012 at 12:17 PM, Pavel Stehule pavel.steh...@gmail.com wrote: Hello 2012/3/5 Alvaro Herrera alvhe...@commandprompt.com: Excerpts from Artur Litwinowicz's message of lun mar 05 16:18:56 -0300 2012: Dear Developers,    I am looking for elegant and effective way for running

Re: [HACKERS] elegant and effective way for running jobs inside a database

2012-03-06 Thread Daniel Farina
On Tue, Mar 6, 2012 at 3:31 PM, Andrew Dunstan and...@dunslane.net wrote: We don't slavishly need to reproduce every piece of cron. In any case, on my Linux machine at least, batch is part of the at package, not the cron package. If you want anything at all done, then I'd suggest starting with

Re: [HACKERS] pg_stat_statements and planning time

2012-03-07 Thread Daniel Farina
On Wed, Mar 7, 2012 at 8:07 AM, Tom Lane t...@sss.pgh.pa.us wrote: Fujii Masao masao.fu...@gmail.com writes: In the patch, I didn't change the column name total_time meaning the time spent in the executor because of the backward compatibility. But once new column plan_time is added, total_time

Re: [HACKERS] query planner does not canonicalize infix operators

2012-03-12 Thread Daniel Farina
On Mon, Mar 12, 2012 at 7:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: Will Leinweber w...@heroku.com writes: I created an index on an hstore function, fetchval(hstore, text), however when I use the - infix operator which resolves to the very same function, this index is not used. It should be

Re: [HACKERS] query planner does not canonicalize infix operators

2012-03-12 Thread Daniel Farina
On Mon, Mar 12, 2012 at 12:22 PM, Tom Lane t...@sss.pgh.pa.us wrote: Daniel Farina dan...@heroku.com writes: On Mon, Mar 12, 2012 at 7:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: Will Leinweber w...@heroku.com writes: I created an index on an hstore function, fetchval(hstore, text), however

[HACKERS] pg_upgrade and statistics

2012-03-12 Thread Daniel Farina
As noted by the manual, pg_statistic is ported in any way when performing pg_upgrade. I have been investigating what it would take to (even via just a connected SQL superuser client running UPDATE or INSERT against pg_statistic) get at least some baseline statistics into the database as quickly

Re: [HACKERS] pg_upgrade and statistics

2012-03-13 Thread Daniel Farina
On Mon, Mar 12, 2012 at 8:10 PM, Bruce Momjian br...@momjian.us wrote: To answer your specific question, I think clearing the last analyzed fields should cause autovacuum to run on analyze those tables.  What I don't know is whether not clearing the last vacuum datetime will cause the table

Re: [HACKERS] pg_upgrade and statistics

2012-03-13 Thread Daniel Farina
On Mon, Mar 12, 2012 at 9:12 PM, Tom Lane t...@sss.pgh.pa.us wrote: Bruce Momjian br...@momjian.us writes: Copying the statistics from the old server is on the pg_upgrade TODO list.  I have avoided it because it will add an additional requirement that will make pg_upgrade more fragile in case

Re: [HACKERS] pg_upgrade and statistics

2012-03-13 Thread Daniel Farina
On Tue, Mar 13, 2012 at 3:30 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Mar 13, 2012 at 5:42 PM, Bruce Momjian br...@momjian.us wrote: What is the target=10 duration?  I think 10 is as low as we can acceptably recommend.  Should we recommend they run vacuumdb twice, once with

Re: [HACKERS] Chronic performance issue with Replication Failover and FSM.

2012-03-13 Thread Daniel Farina
On Tue, Mar 13, 2012 at 4:53 PM, Josh Berkus j...@agliodbs.com wrote: All, I've discovered a built-in performance issue with replication failover at one site, which I couldn't find searching the archives.  I don't really see what we can do to fix it, so I'm posting it here in case others

Re: [HACKERS] Chronic performance issue with Replication Failover and FSM.

2012-03-13 Thread Daniel Farina
On Tue, Mar 13, 2012 at 7:05 PM, Fujii Masao masao.fu...@gmail.com wrote: If it's really a high-UPDATE workload, wouldn't autovacuum start soon? Also, while vacuum cleanup records are applied, could not the standby also update its free space map, without having to send the actual FSM updates? I

[HACKERS] Faster compression, again

2012-03-14 Thread Daniel Farina
For 9.3 at a minimum. The topic of LZO became mired in doubts about: * Potential Patents * The author's intention for the implementation to be GPL Since then, Google released Snappy, also an LZ77-class implementation, and it has been ported to C (recently, and with some quirks, like no LICENSE

Re: [HACKERS] Faster compression, again

2012-03-14 Thread Daniel Farina
On Wed, Mar 14, 2012 at 2:03 PM, Merlin Moncure mmonc...@gmail.com wrote: er, typo: I meant to say: *non-gpl* lz based...  :-). Given that, few I would say have had the traction that LZO and Snappy have had, even though in many respects they are interchangeable in the general trade-off spectrum.

Re: [HACKERS] Faster compression, again

2012-03-14 Thread Daniel Farina
On Wed, Mar 14, 2012 at 2:58 PM, Tom Lane t...@sss.pgh.pa.us wrote: Daniel Farina dan...@heroku.com writes: Given that, few I would say have had the traction that LZO and Snappy have had, even though in many respects they are interchangeable in the general trade-off spectrum. The question

[HACKERS] Another review of URI for libpq, v7 submission

2012-03-15 Thread Daniel Farina
I reviewed this and so far have not found any serious problems, although as is par for the course it contains some of the fiddly bits involved in any string manipulations in C. I made a few edits -- none strictly necessary for correctness -- that the original author is free audit and/or

Re: [HACKERS] Another review of URI for libpq, v7 submission

2012-03-15 Thread Daniel Farina
On Thu, Mar 15, 2012 at 7:36 AM, Alex Shulgin a...@commandprompt.com wrote: I wonder if there's any evidence as to that mangling the email addresses helps to reduce spam at all?  I mean replacing (at) back to @ and (dot) to . is piece of cake for a spam crawler. I suspect we're long past the

Re: [HACKERS] Keystone auth in PostgreSQL

2012-03-15 Thread Daniel Farina
On Thu, Mar 15, 2012 at 1:14 PM, Bruce Momjian br...@momjian.us wrote: On Wed, Mar 14, 2012 at 11:38:19AM +0530, Vivek Singh Raghuwanshi wrote: Hi All, Can i use keystone auth with PostgreSQL, it is very helpful when i am using OpenStack as a cloud service and implement DBaaS. I don't think

Re: [HACKERS] Faster compression, again

2012-03-15 Thread Daniel Farina
On Wed, Mar 14, 2012 at 11:06 AM, Daniel Farina dan...@heroku.com wrote: ...and it has been ported to C (recently, and with some quirks, like no LICENSE file...yet, although it is linked from the original Snappy project). I poked the author about the license and he fixed it in a jiffy. Now

Re: [HACKERS] Faster compression, again

2012-03-15 Thread Daniel Farina
On Thu, Mar 15, 2012 at 3:14 PM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, Mar 14, 2012 at 6:06 PM, Daniel Farina dan...@heroku.com wrote: If we're curious how it affects replication traffic, I could probably gather statistics on LZO-compressed WAL traffic, of which we have a pretty

[HACKERS] pg_terminate_backend for same-role

2012-03-15 Thread Daniel Farina
Parallel to pg_cancel_backend, it'd be nice to allow the user to just outright kill a backend that they own (politely, with a SIGTERM), aborting any transactions in progress, including the idle transaction, and closing the socket. I imagine the problem is a race condition whereby a pid might be

Re: [HACKERS] Keystone auth in PostgreSQL

2012-03-15 Thread Daniel Farina
On Thu, Mar 15, 2012 at 6:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: Our standard answer when someone asks for $random-auth-method is to suggest that they find a PAM module for it and use PAM.  I wouldn't want to claim that PAM is a particularly great interface for this sort of thing, but it's

Re: [HACKERS] pg_terminate_backend for same-role

2012-03-15 Thread Daniel Farina
On Thu, Mar 15, 2012 at 9:39 PM, Fujii Masao masao.fu...@gmail.com wrote: Shall we just do everything using the MyCancelKey (which I think could just be called SessionKey, SessionSecret, or even just Session) as to ensure we have no case of mistaken identity? Or does that end up being

Re: [HACKERS] pg_terminate_backend for same-role

2012-03-15 Thread Daniel Farina
On Thu, Mar 15, 2012 at 10:33 PM, Tom Lane t...@sss.pgh.pa.us wrote: Daniel Farina dan...@heroku.com writes: The way MyCancelKey is checked now is backwards, in my mind.  It seems like it would be better checked by the receiving PID (one can use a check/recheck also, if so inclined

Re: [HACKERS] pg_terminate_backend for same-role

2012-03-16 Thread Daniel Farina
On Thu, Mar 15, 2012 at 10:45 PM, Tom Lane t...@sss.pgh.pa.us wrote: Daniel Farina dan...@heroku.com writes: On Thu, Mar 15, 2012 at 10:33 PM, Tom Lane t...@sss.pgh.pa.us wrote: But actually I don't see what you hope to gain from such a change, even if it can be made to work.  Anyone who can

Re: [HACKERS] pg_terminate_backend for same-role

2012-03-16 Thread Daniel Farina
On Thu, Mar 15, 2012 at 11:01 PM, Daniel Farina dan...@heroku.com wrote: Okay, well, I believe there is a race in handling common administrative signals that *might* possibly matter.  In the past, pg_cancel_backend was superuser only, which is a lot like saying only available to people who can

Re: [HACKERS] pg_terminate_backend for same-role

2012-03-16 Thread Daniel Farina
On Fri, Mar 16, 2012 at 3:42 PM, Noah Misch n...@leadboat.com wrote: On Thu, Mar 15, 2012 at 04:14:03PM -0700, Daniel Farina wrote: Parallel to pg_cancel_backend, it'd be nice to allow the user to just outright kill a backend that they own (politely, with a SIGTERM), aborting any transactions

Re: [HACKERS] pg_terminate_backend for same-role

2012-03-16 Thread Daniel Farina
On Fri, Mar 16, 2012 at 4:42 PM, Daniel Farina dan...@heroku.com wrote: Hmm. Well, here's a patch that implements exactly that, I think, That version had some screws loose due to some editor snafus. Hopefully all better. -- fdr Implement-race-free-sql-originated-backend-cancellation-v3

[HACKERS] Cross-backend signals and administration (Was: Re: pg_terminate_backend for same-role)

2012-03-17 Thread Daniel Farina
This thread evolved out of an attempt to implement pg_terminate_backend for non-superusers. I thought -- probably erroneously -- that the major objection to that was the known possibility of a PID-cycling race condition, whereby a signal could be misdirected, in the case of terminate_backend,

  1   2   3   4   5   >