Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-04-10 Thread Alvaro Herrera
Tom Lane wrote: > So I looked into this, and found that persuading psql to let backslash > commands cross line boundaries is a much bigger deal than just fixing the > lexer. The problem is that MainLoop would need to grow an understanding > of having received only a partial backslash command and

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-03-22 Thread Kyotaro HORIGUCHI
First, thank you all involved, and thank you for polishing this and committing, Tom. At Mon, 21 Mar 2016 17:15:18 -0400, Tom Lane wrote in <1596.1458594...@sss.pgh.pa.us> > So I looked into this, and found that persuading psql to let backslash > commands cross line

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-03-21 Thread Tom Lane
So I looked into this, and found that persuading psql to let backslash commands cross line boundaries is a much bigger deal than just fixing the lexer. The problem is that MainLoop would need to grow an understanding of having received only a partial backslash command and needing to go back to

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-03-21 Thread Tom Lane
Robert Haas writes: > Mmph. I just don't see any benefit in being able to start a command > in the middle of a line. I agree that there's no functional benefit; it's a matter of consistency. In particular, psql has always allowed you to write multiple SQL commands per

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-03-21 Thread Robert Haas
On Mon, Mar 21, 2016 at 3:42 PM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Mar 21, 2016 at 3:01 PM, Tom Lane wrote: >>> Um, why exactly? That psql behavior is of really ancient standing, and >>> we have not had complaints

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-03-21 Thread Tom Lane
Robert Haas writes: > On Mon, Mar 21, 2016 at 3:01 PM, Tom Lane wrote: >> Um, why exactly? That psql behavior is of really ancient standing, and >> we have not had complaints about it. > I think that's mostly because the psql metacommands are

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-03-21 Thread Robert Haas
On Mon, Mar 21, 2016 at 3:01 PM, Tom Lane wrote: >> Wait, was it really? I'd been thinking it was mostly to continue >> queries, not metacommands, but maybe I missed the boat. > > Nah, you're right, it was about continuing queries. Still, we've had > complaints about the

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-03-21 Thread Tom Lane
Robert Haas writes: > On Sun, Mar 20, 2016 at 1:07 PM, Tom Lane wrote: >> This solves the problem of allowing SQL commands in scripts to span >> lines, ... > Excellent. >> but it doesn't do anything about backslash commands, which was >> the original

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-03-20 Thread Robert Haas
On Sun, Mar 20, 2016 at 1:07 PM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Mar 18, 2016 at 10:02 AM, Tom Lane wrote: >>> This is mostly a flex/bison hack, isn't it? If you like I'll take it. > >> I would be delighted if you

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-03-20 Thread David G. Johnston
On Sunday, March 20, 2016, Tom Lane wrote: > > * Allow backslash commands to span lines, probably by adopting the > rule that backslash immediately followed by newline is to be ignored > within a backslash command. This would not be compatible with psql, > though, at least

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-03-20 Thread Tom Lane
Robert Haas writes: > On Fri, Mar 18, 2016 at 10:02 AM, Tom Lane wrote: >> This is mostly a flex/bison hack, isn't it? If you like I'll take it. > I would be delighted if you would. I've committed changes equivalent to Horiguchi-san's 0001 and 0002

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-03-19 Thread Robert Haas
On Thu, Mar 17, 2016 at 4:12 AM, Kyotaro HORIGUCHI wrote: > Thank you for the comment, but could you please tell me what kind > of criteria should I take to split this patch? The discussion > about splitting criteria is in the following reply (in the > sentence

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-03-19 Thread Robert Haas
On Thu, Feb 18, 2016 at 6:54 AM, Kyotaro HORIGUCHI < horiguchi.kyot...@lab.ntt.co.jp> wrote: > It is the SQL part of old psqlscan.l but the difference between > them is a bit bothersome to see. I attached the diff between them > as "psqlscanbody.l.diff" for convenience. > This is a huge diff,

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-03-19 Thread Tom Lane
Robert Haas writes: > Well, I'm trying to find a piece of this patch that is small enough > that I can understand it and in good enough shape that I can commit it > independently, but I am having some difficulty with that. I keep > hoping some other committer is going to

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-03-19 Thread Kyotaro HORIGUCHI
Thank you for the comment, but could you please tell me what kind of criteria should I take to split this patch? The discussion about splitting criteria is in the following reply (in the sentence begins with "By the way"). At Wed, 16 Mar 2016 11:57:25 -0400, Robert Haas

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-03-19 Thread Robert Haas
On Fri, Mar 18, 2016 at 10:02 AM, Tom Lane wrote: > Robert Haas writes: >> Well, I'm trying to find a piece of this patch that is small enough >> that I can understand it and in good enough shape that I can commit it >> independently, but I am having

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-03-15 Thread Fabien COELHO
I intend to have a look at it, I had a look at a previous instance, but I'm ok if someone wants to proceed. There's not exactly a long line of reviewers at the moment so if you could do a followup review that would be great. Ok. It is in the queue, not for right know, though. -- Fabien.

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-03-14 Thread David Steele
Hi Fabien, On 3/14/16 3:27 PM, Fabien COELHO wrote: >> Any takers to review this updated patch? > > I intend to have a look at it, I had a look at a previous instance, but > I'm ok if someone wants to proceed. There's not exactly a long line of reviewers at the moment so if you could do a

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-03-14 Thread Fabien COELHO
Hello David, Any takers to review this updated patch? I intend to have a look at it, I had a look at a previous instance, but I'm ok if someone wants to proceed. -- Fabien. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-03-14 Thread David Steele
On 2/18/16 6:54 AM, Kyotaro HORIGUCHI wrote: First, I rebased the previous patch set and merged three of them. Now they are of three patches. 1. Making SQL parser part of psqlscan independent from psql. Moved psql's baskslsh command stuff out of original psqlscan.l and some psql

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-02-16 Thread Robert Haas
On Mon, Feb 15, 2016 at 1:04 AM, Kyotaro HORIGUCHI wrote: > I'm sorry, but I didn't understood the 'submission notes' exactly > means. Is it precise descriptions in source comments? or commit > message of git-commit? Write a detailed email explaining each change

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-02-14 Thread Kyotaro HORIGUCHI
Hello, thank you for reviewing this. > On Thu, Jan 7, 2016 at 3:36 AM, Kyotaro HORIGUCHI > wrote: > > - 0001-Prepare-for-sharing-psqlscan-with-pgbench.patch > > > > This diff looks a bit large but most of them is cut'n-paste > > work and the substantial

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-02-02 Thread Kyotaro HORIGUCHI
Hello, thank you, Febien, Micael. # Though I have made almost no activity in the last month... At Tue, 26 Jan 2016 13:53:33 +0100 (CET), Fabien COELHO wrote in > > Hello Kyotaro-san, > > > Thank you very much Michael but the CF

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-02-02 Thread Robert Haas
On Thu, Jan 7, 2016 at 3:36 AM, Kyotaro HORIGUCHI wrote: > - 0001-Prepare-for-sharing-psqlscan-with-pgbench.patch > > This diff looks a bit large but most of them is cut'n-paste > work and the substantial change is rather small. > > This refactors psqlscan.l

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-01-26 Thread Michael Paquier
On Tue, Jan 26, 2016 at 6:51 PM, Kyotaro HORIGUCHI wrote: > Mmm. I believed that this is on CF app.. > > At Tue, 19 Jan 2016 15:41:54 +0900, Michael Paquier > wrote in >

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-01-26 Thread Kyotaro HORIGUCHI
Mmm. I believed that this is on CF app.. At Tue, 19 Jan 2016 15:41:54 +0900, Michael Paquier wrote in

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-01-26 Thread Fabien COELHO
Hello Kyotaro-san, Thank you very much Michael but the CF app doesn't allow me to regsiter new one. Filling the Description field with "pgbench - allow backslash-continuations in custom scripts" and chose a topic then "Find thread" shows nothing. Filling the search text field on the "Attach

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2016-01-18 Thread Michael Paquier
On Thu, Jan 7, 2016 at 5:36 PM, Kyotaro HORIGUCHI wrote: > Finally, PsqlScanState has four callback funcions and all pgbench > needs to do to use it is setting NULL to all of them and link the > object file in psql directory. No link switch/ifdef are necessary.

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-12-24 Thread Kyotaro HORIGUCHI
Hello, At Thu, 24 Dec 2015 07:40:19 +0100 (CET), Fabien COELHO wrote in > > Hello Michaël, > > >> If I read you correctly, I should cut it out into a new file and > >> include it. Is it correct? > > > > Not really, I meant to see

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-12-23 Thread Michael Paquier
On Tue, Dec 22, 2015 at 5:34 PM, Kyotaro HORIGUCHI wrote: > I wrote: >> Wouldn't it be better with something say in src/common >> which is frontend-only? We could start with a set of routines allowing >> commands to be parsed. That gives us more room for future

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-12-23 Thread Fabien COELHO
Hello Michaël, If I read you correctly, I should cut it out into a new file and include it. Is it correct? Not really, I meant to see if it would be possible to include this set of routines directly in libpqcommon (as part of OBJS_FRONTEND). This way any client applications could easily

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-12-22 Thread Kyotaro HORIGUCHI
I'm terribly sorry to have overlooked Febien's comment. I'll rework on this considering Febien's previous comment and Michael's this comment in the next CF. At Tue, 22 Dec 2015 16:17:07 +0900, Michael Paquier wrote in

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-12-21 Thread Michael Paquier
On Wed, Oct 14, 2015 at 5:49 PM, Fabien COELHO wrote: > > Hello, > > Here is a review, sorry for the delay... > >> This is done as the additional fourth patch, not merged into >> previous ones, to show what's changed in the manner of command >> storing. >> [...] >>> >>> -

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-10-14 Thread Fabien COELHO
Hello, Here is a review, sorry for the delay... This is done as the additional fourth patch, not merged into previous ones, to show what's changed in the manner of command storing. [...] - SQL multi-statement. SELECT 1; SELECT 2; I think this is really "SELECT 1\; SELECT 2;" I join a

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-09-03 Thread Kyotaro HORIGUCHI
Hello, Thank you for registering this to CF-Sep. I missed the regtration window.. It ended earlier than usual.. Most troubles have gone and I'll be back next week. > The work to be left is eliminating double-format of Command > struct. This is done as the additional fourth patch, not merged

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-08-18 Thread Kyotaro HORIGUCHI
Hi, all. I don't think we actually want backslash-continuations. The feature we want is allow SQL statements span multiple lines, and using the psql lexer solves that. We don't need the backslash-continuations when we have that. Sure. The feature *I* initially wanted was to have

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-08-05 Thread Heikki Linnakangas
On 07/24/2015 11:36 AM, Kyotaro HORIGUCHI wrote: At Fri, 24 Jul 2015 07:39:16 +0200 (CEST), Fabien COELHO coe...@cri.ensmp.fr wrote in alpine.DEB.2.10.1507240731050.12839@sto - backslash commands is handled as the same as before: multiline is not allowed. Hmm... that is really the feature

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-08-05 Thread Fabien COELHO
Hello Heikki, I don't think we actually want backslash-continuations. The feature we want is allow SQL statements span multiple lines, and using the psql lexer solves that. We don't need the backslash-continuations when we have that. Sure. The feature *I* initially wanted was to have

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-07-27 Thread Fabien COELHO
Attatched is the revised version of this patch. The first patch is not changed from before. The second is fixed a kind of bug. Ths third is the new one to allow backslash continuation for backslash commands. Ah, thanks:-) Would you consider adding the patch to the next commitfest? I may

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-07-26 Thread Kyotaro HORIGUCHI
Hello, Attatched is the revised version of this patch. The first patch is not changed from before. The second is fixed a kind of bug. Ths third is the new one to allow backslash continuation for backslash commands. Ah, thanks:-) Would you consider adding the patch to the

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-07-25 Thread Fabien COELHO
oops, sorry, stalled post because of wrong from, posting again... Attatched is the revised version of this patch. The first patch is not changed from before. The second is fixed a kind of bug. Ths third is the new one to allow backslash continuation for backslash commands. Ah, thanks:-)

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-07-24 Thread Kyotaro HORIGUCHI
Hi, all. Attatched is the revised version of this patch. The first patch is not changed from before. The second is fixed a kind of bug. Ths third is the new one to allow backslash continuation for backslash commands. hoge.sql is the test custom script. == At Fri, 24 Jul 2015 07:39:16

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-07-23 Thread Fabien COELHO
Hello Kyotaro-san, If you feel that this feature only deserve a lexer solution, then the patch should be returned with feedback. It's unfortunate to abandon this idea so I tried this and made it run with psql's parser. I think it works as expected. Wow, you are much more courageous than I

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-07-23 Thread Kyotaro HORIGUCHI
Hi, If you feel that this feature only deserve a lexer solution, then the patch should be returned with feedback. It's unfortunate to abandon this idea so I tried this and made it run with psql's parser. I think it works as expected. The attached files are as follwoing. -

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-07-17 Thread Fabien COELHO
Hello Tom, (although actually, why wouldn't we want to just implement variable substitution exactly like it is in psql? Pgbench variable substitution is performed when the script is run, not while the file is being processed for being split, which is when a lexer would be used. The

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-07-03 Thread Tom Lane
Fabien COELHO coe...@cri.ensmp.fr writes: I'm pretty clearly in favor of doing correct lexing. I think we should generalize that and make it reusable. psql has it's own hacked up version already, there seems little point in having variedly good copies around. I must admit that I do not know

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-07-03 Thread Tom Lane
I wrote: As it stands, psqlscan.l has some external dependencies on the rest of psql, but we could perhaps refactor some of those away, and provide dummy implementations to satisfy others (eg pgbench could provide a dummy GetVariable() that just always returns NULL). So I'm imagining

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-07-03 Thread Fabien COELHO
(although actually, why wouldn't we want to just implement variable substitution exactly like it is in psql? Pgbench variable substitution is performed when the script is run, not while the file is being processed for being split, which is when a lexer would be used. The situation is not

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-07-03 Thread Andres Freund
On 2015-07-03 13:50:02 +0300, Heikki Linnakangas wrote: As Tom pointed out, you need the full lexer to do this correctly. You can argue that something that handles the most common cases is enough, but realistically, by the time you've handled all the common cases correctly, you've just

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-07-03 Thread Heikki Linnakangas
On 06/21/2015 11:12 AM, Fabien COELHO wrote: Hello Josh, Add backslash continuations to pgbench custom scripts. [...] IMHO this approach is the best compromise. I don't personally agree. I believe that it it worth breaking backwards compatibility to support line breaks in pgbench

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-07-03 Thread Fabien COELHO
Hello Heikki, I'm not clear on why we'd need a full SQL lexer. Attached is a without lexer version which does ;-terminated SQL commands and \-continuated meta commands (may be useful for \shell and long \set expressions). As Tom pointed out, you need the full lexer to do this correctly.

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-07-03 Thread Fabien COELHO
I'm pretty clearly in favor of doing correct lexing. I think we should generalize that and make it reusable. psql has it's own hacked up version already, there seems little point in having variedly good copies around. I must admit that I do not know how to share lexer rules but have

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-06-25 Thread Tatsuo Ishii
Look, how many people in the world develop their own pgbench scripts? And how many of those aren't on this list right now, reading this thread? I expect I could count them on my fingers and toes. I'm not against you break the backward compatibility of pgbench custom scripts. However I just

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-06-25 Thread Jeff Janes
On Wed, Jun 24, 2015 at 1:22 PM, Josh Berkus j...@agliodbs.com wrote: On 06/21/2015 01:37 PM, Fabien COELHO wrote: The worst case with pgbench is that we break two people's test scripts, they read the release notes, and fix them. Sure, I agree that breaking pgbench custom scripts

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-06-25 Thread Tatsuo Ishii
I'm not against you break the backward compatibility of pgbench custom scripts. However I just want to let you know that PostgreSQL Enterprise Consortium has been published couple of scripts along with reports on evaluating PostgreSQL, which have been downloaded considerable numbers.

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-06-25 Thread Michael Paquier
On Thu, Jun 25, 2015 at 10:51 PM, Tatsuo Ishii is...@postgresql.org wrote: Look, how many people in the world develop their own pgbench scripts? And how many of those aren't on this list right now, reading this thread? I expect I could count them on my fingers and toes. I'm not against you

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-06-25 Thread Michael Paquier
On Fri, Jun 26, 2015 at 9:01 AM, Tatsuo Ishii is...@postgresql.org wrote: I'm not against you break the backward compatibility of pgbench custom scripts. However I just want to let you know that PostgreSQL Enterprise Consortium has been published couple of scripts along with reports on

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-06-24 Thread Josh Berkus
On 06/21/2015 01:37 PM, Fabien COELHO wrote: The worst case with pgbench is that we break two people's test scripts, they read the release notes, and fix them. Sure, I agree that breaking pgbench custom scripts compatibility is no big deal, and having pgbench consistent with psql is

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-06-21 Thread Josh Berkus
Fabien, Without a lexer it is possible to fool pgbench with somehow contrived examples, say with: SELECT 'hello; world'; The ; within the string will be considered as end-of-line. Also, comments intermixed with sql on the same line would generate errors. SELECT 1 -- one

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-06-21 Thread Fabien COELHO
The worst case with pgbench is that we break two people's test scripts, they read the release notes, and fix them. Sure, I agree that breaking pgbench custom scripts compatibility is no big deal, and having pgbench consistent with psql is useful. -- Fabien. -- Sent via pgsql-hackers

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-06-21 Thread Fabien COELHO
Hello Josh, Add backslash continuations to pgbench custom scripts. [...] IMHO this approach is the best compromise. I don't personally agree. I believe that it it worth breaking backwards compatibility to support line breaks in pgbench statements, and that if we're not going to do that,

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-06-20 Thread Fabien COELHO
I tend to agree on that bottom line; having this be inconsistent with psql does not seem like a win. I'm not clear on why we'd need a full SQL lexer. So you don't get fooled by semicolons embedded in string literals or comments. I take it we ignore those now? I mean, personally, it

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-06-19 Thread Josh Berkus
On 06/19/2015 02:51 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: On 05/14/2015 12:10 PM, Fabien COELHO wrote: Add backslash continuations to pgbench custom scripts. I don't personally agree. I believe that it it worth breaking backwards compatibility to support line breaks in

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-06-19 Thread Josh Berkus
On 05/14/2015 12:10 PM, Fabien COELHO wrote: Add backslash continuations to pgbench custom scripts. The benefit of this approach is that it is upward compatible, and it is also pretty simple to implement. The downside is that backslash continuation is not the best syntax ever invented, but

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-06-19 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: On 05/14/2015 12:10 PM, Fabien COELHO wrote: Add backslash continuations to pgbench custom scripts. I don't personally agree. I believe that it it worth breaking backwards compatibility to support line breaks in pgbench statements, and that if we're not