Re: [HACKERS] COPY as a set returning function

2017-01-31 Thread Corey Huinker
> > > Here is a 4: Refactoring BeginCopyFrom so as instead of a Relation are > used a TupleDesc, a default expression list, and a relation name. You > could as well make NextCopyFrom() smarter so as it does nothing if no > expression contexts are given by the caller, which is the case of your >

Re: [HACKERS] COPY as a set returning function

2017-01-30 Thread Michael Paquier
On Tue, Jan 31, 2017 at 3:05 AM, Corey Huinker wrote: > On Fri, Jan 27, 2017 at 9:42 PM, Peter Eisentraut > wrote: >> >> On 1/25/17 8:51 PM, Corey Huinker wrote: >> > # select * from copy_srf('echo "x\ty"',true) as t(x text, y text); >>

Re: [HACKERS] COPY as a set returning function

2017-01-30 Thread Corey Huinker
On Fri, Jan 27, 2017 at 9:42 PM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 1/25/17 8:51 PM, Corey Huinker wrote: > > # select * from copy_srf('echo "x\ty"',true) as t(x text, y text); > > I find these parameters weird. Just looking at this, one has no idea > what the

Re: [HACKERS] COPY as a set returning function

2017-01-27 Thread Peter Eisentraut
On 1/27/17 8:07 PM, David Fetter wrote: > There are still neither regression tests nor SGML documentation. > > Are we at a point where we should add these things? One could argue about the documentation at this point, since the function is somewhat self-documenting for the time being. But

Re: [HACKERS] COPY as a set returning function

2017-01-27 Thread Peter Eisentraut
On 1/25/17 8:51 PM, Corey Huinker wrote: > # select * from copy_srf('echo "x\ty"',true) as t(x text, y text); I find these parameters weird. Just looking at this, one has no idea what the "true" means. Why not have a "filename" and a "program" parameter and make them mutually exclusive? --

Re: [HACKERS] COPY as a set returning function

2017-01-27 Thread David Fetter
On Wed, Jan 25, 2017 at 08:51:38PM -0500, Corey Huinker wrote: > I've put in some more work on this patch, mostly just taking Alvaro's > suggestions, which resulted in big code savings. The patch applies atop master. The test (ls) which previously crashed the backend now doesn't, and works in a

Re: [HACKERS] COPY as a set returning function

2017-01-25 Thread Corey Huinker
> > >> I don't understand why do we have all these checks. Can't we just pass >> the values to COPY and let it apply the checks? That way, when COPY is >> updated to support multibyte escape chars (for example) we don't need to >> touch this code. Together with removing the unneeded braces that

Re: [HACKERS] COPY as a set returning function

2017-01-25 Thread Corey Huinker
> > > + /* param 7: escape text default null, -- defaults to whatever > quote is */ > > + if (PG_ARGISNULL(7)) > > + { > > + copy_state.escape = copy_state.quote; > > + } > > + else > > + { > > + if (copy_state.csv_mode) > > + { > > +

Re: [HACKERS] COPY as a set returning function

2017-01-25 Thread Corey Huinker
On Wed, Jan 25, 2017 at 1:10 PM, David Fetter wrote: > On Wed, Jan 25, 2017 at 12:23:23PM -0500, Corey Huinker wrote: > > > > Feel free to mark it returned as feedback. The concept didn't > > generate as much enthusiasm as I had hoped, so I think the right > > thing to do now

Re: [HACKERS] COPY as a set returning function

2017-01-25 Thread David Fetter
On Wed, Jan 25, 2017 at 12:23:23PM -0500, Corey Huinker wrote: > > Feel free to mark it returned as feedback. The concept didn't > generate as much enthusiasm as I had hoped, so I think the right > thing to do now is scale it back to a patch that makes > CopyFromRawFields() externally visible so

Re: [HACKERS] COPY as a set returning function

2017-01-25 Thread Corey Huinker
On Wed, Jan 25, 2017 at 11:57 AM, Alvaro Herrera wrote: > David Fetter wrote: > > > @@ -562,7 +563,6 @@ CopyGetData(CopyState cstate, void *databuf, int > minread, int maxread) > >errmsg("could not read > from COPY file:

Re: [HACKERS] COPY as a set returning function

2017-01-25 Thread Alvaro Herrera
David Fetter wrote: > @@ -562,7 +563,6 @@ CopyGetData(CopyState cstate, void *databuf, int minread, > int maxread) >errmsg("could not read from > COPY file: %m"))); > break; > case COPY_OLD_FE: > - >

Re: [HACKERS] COPY as a set returning function

2017-01-25 Thread David Fetter
On Wed, Jan 25, 2017 at 06:16:16AM -0800, David Fetter wrote: > On Mon, Oct 31, 2016 at 04:45:40PM -0400, Corey Huinker wrote: > > Attached is a patch that implements copy_srf(). > > > > The function signature reflects cstate more than it reflects the COPY > > options (filename+is_program instead

Re: [HACKERS] COPY as a set returning function

2017-01-25 Thread David Fetter
On Mon, Oct 31, 2016 at 04:45:40PM -0400, Corey Huinker wrote: > Attached is a patch that implements copy_srf(). > > The function signature reflects cstate more than it reflects the COPY > options (filename+is_program instead of FILENAME or PROGRAM, etc) The patch as it stands needs a rebase.

Re: [HACKERS] COPY as a set returning function

2017-01-25 Thread David Fetter
On Wed, Jan 25, 2017 at 02:37:57PM +0900, Michael Paquier wrote: > On Mon, Dec 5, 2016 at 2:10 PM, Haribabu Kommi > wrote: > > On Tue, Nov 1, 2016 at 7:45 AM, Corey Huinker > > wrote: > >> > >> Attached is a patch that implements copy_srf(). >

Re: [HACKERS] COPY as a set returning function

2017-01-24 Thread Michael Paquier
On Mon, Dec 5, 2016 at 2:10 PM, Haribabu Kommi wrote: > On Tue, Nov 1, 2016 at 7:45 AM, Corey Huinker > wrote: >> >> Attached is a patch that implements copy_srf(). > > Moved to next CF with "needs review" status. This patch is still waiting

Re: [HACKERS] COPY as a set returning function

2016-12-04 Thread Haribabu Kommi
On Tue, Nov 1, 2016 at 7:45 AM, Corey Huinker wrote: > Attached is a patch that implements copy_srf(). > Moved to next CF with "needs review" status. Regards, Hari Babu Fujitsu Australia

Re: [HACKERS] COPY as a set returning function

2016-10-31 Thread Corey Huinker
Attached is a patch that implements copy_srf(). The function signature reflects cstate more than it reflects the COPY options (filename+is_program instead of FILENAME or PROGRAM, etc) CREATE OR REPLACE FUNCTION copy_srf( filenametext DEFAULT null, is_program boolean DEFAULT false,

Re: [HACKERS] COPY as a set returning function

2016-10-17 Thread Merlin Moncure
On Fri, Sep 30, 2016 at 9:56 PM, Tom Lane wrote: > Craig Ringer writes: >> On 1 Oct. 2016 05:20, "Tom Lane" wrote: >>> I think the last of those suggestions has come up before. It has the >>> large advantage that you don't

Re: [HACKERS] COPY as a set returning function

2016-10-17 Thread Corey Huinker
On Sun, Oct 16, 2016 at 9:01 AM, Craig Ringer wrote: > On 15 Oct. 2016 04:56, "Corey Huinker" wrote: > > > I would like to make COPY itself a SRF. That's a bit beyond my > capabilities, so if that is the route we want to go, I will need

Re: [HACKERS] COPY as a set returning function

2016-10-16 Thread Craig Ringer
On 15 Oct. 2016 04:56, "Corey Huinker" wrote: > I would like to make COPY itself a SRF. That's a bit beyond my capabilities, so if that is the route we want to go, I will need help. > > The syntax would probably look like this (new bits in bold): > >> WITH my_copy AS (

Re: [HACKERS] COPY as a set returning function

2016-10-14 Thread Corey Huinker
> > > That sounds fantastic. It'd help this copy variant retain festure parity > > with normal copy. And it'd bring us closer to being able to FETCH in non > > queries. > > On second thought, though, this couldn't exactly duplicate the existing > COPY syntax, because COPY relies heavily on the

Re: [HACKERS] COPY as a set returning function

2016-09-30 Thread Tom Lane
Craig Ringer writes: > On 1 Oct. 2016 05:20, "Tom Lane" wrote: >> I think the last of those suggestions has come up before. It has the >> large advantage that you don't have to remember a different syntax for >> copy-as-a-function. > That

Re: [HACKERS] COPY as a set returning function

2016-09-30 Thread Craig Ringer
On 1 Oct. 2016 05:20, "Tom Lane" wrote: > > Corey Huinker writes: > > Attached is a _very_ rough patch implementing a proof-of-concept function > > copy_srf(); > > ... > > As for that future direction, we could either have: > > - a robust function

Re: [HACKERS] COPY as a set returning function

2016-09-30 Thread Tom Lane
Corey Huinker writes: > Attached is a _very_ rough patch implementing a proof-of-concept function > copy_srf(); > ... > As for that future direction, we could either have: > - a robust function named something like copy_srf(), with parameters for > all of the relevant