Re: [HACKERS] pgbench - allow to store select results into variables

Hello Pavel, Here is a v13, which is just a rebase after the documentation xml-ization. Here is a v14, after yet another rebase, and some comments added to answer your new comments. I am looking to this patch. Not sure if "cset" is best name - maybe "eset" .. like embeded set? I used

Re: [HACKERS] pgbench - allow to store select results into variables

Hi 2017-10-20 18:37 GMT+02:00 Fabien COELHO : > > Here is a v12. >> > > Here is a v13, which is just a rebase after the documentation xml-ization. > I am looking to this patch. Not sure if "cset" is best name - maybe "eset" .. like embeded set? The code of

Re: [HACKERS] pgbench - allow to store select results into variables

Here is a v12. Here is a v13, which is just a rebase after the documentation xml-ization. -- Fabien.diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml index e509e6c..44e8896 100644 --- a/doc/src/sgml/ref/pgbench.sgml +++ b/doc/src/sgml/ref/pgbench.sgml @@ -818,6

Re: [HACKERS] pgbench - allow to store select results into variables

Here is a v12. There is no changes in the code or documentation, only TAP tests are added. -- Fabien.diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml index f5db8d1..9ad82d4 100644 --- a/doc/src/sgml/ref/pgbench.sgml +++ b/doc/src/sgml/ref/pgbench.sgml @@ -818,6

Re: [HACKERS] pgbench - allow to store select results into variables

Looks good to me. I have marked the patch status as "Ready for committer". LGTM too. Pushed with a minor adjustment to make parseVariable() have exactly the same test as valid_variable_name(). \set ありがとうございました 1 \set 谢谢 2 \set dankeschön 3 :-) -- Fabien. -- Sent via pgsql-hackers

Re: [HACKERS] pgbench - allow to store select results into variables

Tatsuo Ishii writes: >> Here is a v3 with a more precise comment. > Looks good to me. I have marked the patch status as "Ready for > committer". LGTM too. Pushed with a minor adjustment to make parseVariable() have exactly the same test as valid_variable_name().

Re: [HACKERS] pgbench - allow to store select results into variables

Hi 2017-08-13 20:33 GMT+02:00 Fabien COELHO : > > Here is a v11. > > It is basically a simple rebase after Tom committed the "pgbench -M order" > patch. It interfered because the compound command management also needs > to delay part of the SQL command initialization. Some

Re: [HACKERS] pgbench - allow to store select results into variables

Here is a v11. It is basically a simple rebase after Tom committed the "pgbench -M order" patch. It interfered because the compound command management also needs to delay part of the SQL command initialization. Some patch are luckier than others:-) Here is a v10: - does not talk about

Re: [HACKERS] pgbench - allow to store select results into variables

Here is a v3 with a more precise comment. Looks good to me. I have marked the patch status as "Ready for committer". Ok. Thanks. When/if committed, it might trigger a few rebase of other pending patches. I'll see about that then. -- Fabien. -- Sent via pgsql-hackers mailing list

Re: [HACKERS] pgbench - allow to store select results into variables

> Hello, > >> I think you'd better to change the following comments because there's >> no psqlscan.l or psqlscanslash.l in pgbench source tree. >> >> + * underscore. Keep this in sync with the definition of >> variable_char in >> + * psqlscan.l and psqlscanslash.l. > > Here is a v3 with a more

Re: [HACKERS] pgbench - allow to store select results into variables

Hello, I think you'd better to change the following comments because there's no psqlscan.l or psqlscanslash.l in pgbench source tree. + * underscore. Keep this in sync with the definition of variable_char in + * psqlscan.l and psqlscanslash.l. Here is a v3 with a more precise comment. --

Re: [HACKERS] pgbench - allow to store select results into variables

Fabien, > Hello Tatsuo-san, > >> Thank you for the patch. I tested a little bit and found that it does >> not allow value replacement against non ascii variables in given SQL >> statements . Is it intentional? > > No, this is a bug. > >> If not, I think you need to fix parseVariable() as well.

Re: [HACKERS] pgbench - allow to store select results into variables

It seems the new feature \gset doesn't work with tables having none ascii column names: Indeed. The same error is triggered with the \set syntax, which does not involve any query execution. I have added a sentence mentionning the restriction when variables are first discussed in the

Re: [HACKERS] pgbench - allow to store select results into variables

Hello Tatsuo-san, Thank you for the patch. I tested a little bit and found that it does not allow value replacement against non ascii variables in given SQL statements . Is it intentional? No, this is a bug. If not, I think you need to fix parseVariable() as well. Indeed. Here is v2. --

Re: [HACKERS] pgbench - allow to store select results into variables

Fabien, >> As the variable infrastructures are pretty different between psql & >> pgbench (typed vs untyped values, sorted array vs linked list data >> structure, no hook vs 2 hooks, name spaces vs no such thing...), I >> have chosen the simplest option of just copying the name checking >>

Re: [HACKERS] pgbench - allow to store select results into variables

As the variable infrastructures are pretty different between psql & pgbench (typed vs untyped values, sorted array vs linked list data structure, no hook vs 2 hooks, name spaces vs no such thing...), I have chosen the simplest option of just copying the name checking function and extending

Re: [HACKERS] pgbench - allow to store select results into variables

As the variable infrastructures are pretty different between psql & pgbench (typed vs untyped values, sorted array vs linked list data structure, no hook vs 2 hooks, name spaces vs no such thing...), I have chosen the simplest option of just copying the name checking function and extending

Re: [HACKERS] pgbench - allow to store select results into variables

Hello Tatsuo-san, If I understand correctly, the patch is moved because of the unrelated issue that variables cannot be utf8 in pgbench, and it is a condition to consider this patch that existing pgbench variables (set with \set) can be utf8? I'm not sure if it is "unrelated" because the new

Re: [HACKERS] pgbench - allow to store select results into variables

If I understand correctly, the patch is moved because of the unrelated issue that variables cannot be utf8 in pgbench, and it is a condition to consider this patch that existing pgbench variables (set with \set) can be utf8? I'm not sure if it is "unrelated" because the new feature relies on

Re: [HACKERS] pgbench - allow to store select results into variables

> If I understand correctly, the patch is moved because of the unrelated > issue that variables cannot be utf8 in pgbench, and it is a condition > to consider this patch that existing pgbench variables (set with \set) > can be utf8? I'm not sure if it is "unrelated" because the new feature relies

Re: [HACKERS] pgbench - allow to store select results into variables

Hello Tatsuo, Ok, I will move the patch to the next cf. Done. If I understand correctly, the patch is moved because of the unrelated issue that variables cannot be utf8 in pgbench, and it is a condition to consider this patch that existing pgbench variables (set with \set) can be utf8?

Re: [HACKERS] pgbench - allow to store select results into variables

>> Having said all that, I think we're at the point in the commitfest >> where if there's any design question at all about a patch, it should >> get booted to the next cycle. We are going to have more than enough >> to do to stabilize what's already committed, we don't need to be >> adding more

Re: [HACKERS] pgbench - allow to store select results into variables

> Well, personally, as an all-ASCII guy I'm not too fussed about that, > but I can see that other people might be annoyed. > > The main problem in dealing with it seems to be whether you're willing > to support pgbench running in non-backend-safe encodings (eg SJIS). > If we rejected that case

Re: [HACKERS] pgbench - allow to store select results into variables

On 2017-04-05 20:24:19 -0400, Tom Lane wrote: > Having said all that, I think we're at the point in the commitfest > where if there's any design question at all about a patch, it should > get booted to the next cycle. We are going to have more than enough > to do to stabilize what's already

Re: [HACKERS] pgbench - allow to store select results into variables

Tatsuo Ishii writes: > I still wonder whether I should commit this or not because this patch > does not allow none ascii column names. Well, personally, as an all-ASCII guy I'm not too fussed about that, but I can see that other people might be annoyed. The main problem in

Re: [HACKERS] pgbench - allow to store select results into variables

Tom and others, I still wonder whether I should commit this or not because this patch does not allow none ascii column names. We know pgbench variable name has been restricted since the functionality was born. When users explicitly define a pgbench variable using \set, it is not a too strong

Re: [HACKERS] pgbench - allow to store select results into variables

Hello Tatsuo-san, It seems the new feature \gset doesn't work with tables having none ascii column names: Indeed. The same error is triggered with the \set syntax, which does not involve any query execution. I have added a sentence mentionning the restriction when variables are first

Re: [HACKERS] pgbench - allow to store select results into variables

>> Please find attached a v8 which hopefully fixes these two issues. > Looks good to me, marking as ready for committer. I have looked into this a little bit. It seems the new feature \gset doesn't work with tables having none ascii column names: $ src/bin/pgbench/pgbench -t 1 -f /tmp/f test

Re: [HACKERS] pgbench - allow to store select results into variables

On Fri, Mar 24, 2017 at 8:59 PM, Fabien COELHO wrote: > > Hello Rafia, > >> if (my_command->argc > 2) >> + syntax_error(source, lineno, my_command->line, my_command->argv[0], >> + "at most on argument expected", NULL, -1); >> >> I suppose you mean 'one' argument here. > > >

Re: [HACKERS] pgbench - allow to store select results into variables

Hello Rafia, if (my_command->argc > 2) + syntax_error(source, lineno, my_command->line, my_command->argv[0], + "at most on argument expected", NULL, -1); I suppose you mean 'one' argument here. Indeed. Apart from that indentation is not correct as per pgindent, please check. I guess

Re: [HACKERS] pgbench - allow to store select results into variables

On Thu, Mar 16, 2017 at 12:45 AM, Fabien COELHO wrote: > > Hello Rafia, > >> I was reviewing v7 of this patch, to start with I found following white >> space errors when applying with git apply, >> /home/edb/Desktop/patches/others/pgbench-into-7.patch:66: trailing >>

Re: [HACKERS] pgbench - allow to store select results into variables

Hello Rafia, I was reviewing v7 of this patch, to start with I found following white space errors when applying with git apply, /home/edb/Desktop/patches/others/pgbench-into-7.patch:66: trailing whitespace. Yep. I do not know why "git apply" sometimes complains. All is fine for me both

Re: [HACKERS] pgbench - allow to store select results into variables

On Tue, Jan 31, 2017 at 11:54 AM, Fabien COELHO wrote: > > Bonjour Michaël, > > Attached are the patch, a test script for the feature, and various test >>> scripts to trigger error cases. >>> >> >> I have moved this patch to next CF >> > > Ok. > > as the last status is a new

Re: [HACKERS] pgbench - allow to store select results into variables

Bonjour Michaël, Attached are the patch, a test script for the feature, and various test scripts to trigger error cases. I have moved this patch to next CF Ok. as the last status is a new patch set with no further reviews. Indeed. I did not check if the comments have been applied

Re: [HACKERS] pgbench - allow to store select results into variables

On Sat, Jan 7, 2017 at 6:25 PM, Fabien COELHO wrote: > I think that I have done what you required. > > I have documented the fact that now the feature does not work if compound > commands contain empty queries, which is a very minor drawback for a pgbench > script anyway. > >

Re: [HACKERS] pgbench - allow to store select results into variables

Please pardon the redondance: this is a slightly edited repost from another thread where motivation for this patch was discussed, so that it appear in the relevant thread. Tom> [...] there was immediately objection as to whether his idea of TPC-B Tom> compliance was actually right.

Re: [HACKERS] pgbench - allow to store select results into variables

Hello Tom, Please look at changing \into to be a SQL-command-ending backslash command as we previously discussed. Done. There are two variants: \gset & \gcset for compound (end SQL query, set variables, but do not end command, so that several settings are allowed in a compound command, a

Re: [HACKERS] pgbench - allow to store select results into variables

Hello Tom, Please look at changing \into to be a SQL-command-ending backslash command as we previously discussed. Hmmm. I do want storing results & compound command ending to be orthogonal. In order to keep this feature, I think that I can move the "into/ginto/gset/..." at the end of the

Re: [HACKERS] pgbench - allow to store select results into variables

Fabien COELHO writes: > Indeed. Here is the rebased version, which still get through my various > tests. I looked through this again, and I still think that the syntactic design of the new command is seriously misguided, leading to an ugly and unmaintainable implementation

Re: [HACKERS] pgbench - allow to store select results into variables

On Mon, Oct 3, 2016 at 12:43 AM, Michael Paquier wrote: > On Tue, Sep 27, 2016 at 10:41 AM, Amit Langote > wrote: > > On 2016/09/26 20:27, Fabien COELHO wrote: > >> > >> Hello Amit, > >> > I am marking the pgbench-into-5.patch [1]

Re: [HACKERS] pgbench - allow to store select results into variables

On Tue, Sep 27, 2016 at 10:41 AM, Amit Langote wrote: > On 2016/09/26 20:27, Fabien COELHO wrote: >> >> Hello Amit, >> I am marking the pgbench-into-5.patch [1] as "Ready for Committer" as I have no further comments at the moment. >>> >>> Wait... Heikki's

Re: [HACKERS] pgbench - allow to store select results into variables

On 2016/09/26 20:27, Fabien COELHO wrote: > > Hello Amit, > >>> I am marking the pgbench-into-5.patch [1] as "Ready for Committer" as I >>> have no further comments at the moment. >> >> Wait... Heikki's latest commit now requires this patch to be rebased. > > Indeed. Here is the rebased

Re: [HACKERS] pgbench - allow to store select results into variables

Hello Amit, I am marking the pgbench-into-5.patch [1] as "Ready for Committer" as I have no further comments at the moment. Wait... Heikki's latest commit now requires this patch to be rebased. Indeed. Here is the rebased version, which still get through my various tests. -- Fabien.diff

Re: [HACKERS] pgbench - allow to store select results into variables

On 2016/09/26 16:12, Amit Langote wrote: > I am marking the pgbench-into-5.patch [1] as "Ready for Committer" as I > have no further comments at the moment. Wait... Heikki's latest commit now requires this patch to be rebased. commit 12788ae49e1933f463bc59a6efe46c4a01701b76 Author: Heikki

Re: [HACKERS] pgbench - allow to store select results into variables

Hi Fabien, I am marking the pgbench-into-5.patch [1] as "Ready for Committer" as I have no further comments at the moment. Thanks, Amit [1] https://www.postgresql.org/message-id/alpine.DEB.2.20.1609130730380.10870%40lancre -- Sent via pgsql-hackers mailing list

Re: [HACKERS] pgbench - allow to store select results into variables

Hello Amit, [...] Then you would get the -r cut at the end of the compound command. Thus the current version gives full control of what will appear in the summary. If I change "\into xxx\n" to mean "also cut here", then there is less control on when the cut occurs when into is used. So it

Re: [HACKERS] pgbench - allow to store select results into variables

Hi Fabien, On 2016/09/13 17:41, Fabien COELHO wrote: > > Hello Amit, > >> [...] >> There still seems to be a change in behavior of the -r option due to the >> patch. Consider the following example: >> >> select a from a where a = 1 \; >> select a+1 from a where a = 1; >> ... >> - statement

Re: [HACKERS] pgbench - allow to store select results into variables

Hello Amit, [...] There still seems to be a change in behavior of the -r option due to the patch. Consider the following example: select a from a where a = 1 \; select a+1 from a where a = 1; ... - statement latencies in milliseconds: 2.889 select a from a where a = 1 ; vs select

Re: [HACKERS] pgbench - allow to store select results into variables

Hi Fabien, On 2016/09/07 23:01, Fabien COELHO wrote: >> Custom script looks like: >> >> \; >> select a \into a >>from tab where a = 1; >> \set i debug(:a) >> >> I get the following error: >> >> undefined variable "a" >> client 0 aborted in state 1; execution of meta-command failed > > Good

Re: [HACKERS] pgbench - allow to store select results into variables

Hello Amit, Custom script looks like: \; select a \into a from tab where a = 1; \set i debug(:a) I get the following error: undefined variable "a" client 0 aborted in state 1; execution of meta-command failed Good catch! Indeed, there is a problem with empty commands which are simply

Re: [HACKERS] pgbench - allow to store select results into variables

Hi Fabien, On 2016/09/03 2:47, Fabien COELHO wrote: >> This patch needs to be rebased because of commit 64710452 (on 2016-08-19). > > Here it is! Thanks for sending the updated patch. Here are some (mostly cosmetic) comments. Before the comments, let me confirm whether the following result is

Re: [HACKERS] pgbench - allow to store select results into variables

Hello Amit, This patch needs to be rebased because of commit 64710452 (on 2016-08-19). Here it is! -- Fabien.diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml index 285608d..a879e59 100644 --- a/doc/src/sgml/ref/pgbench.sgml +++ b/doc/src/sgml/ref/pgbench.sgml @@

Re: [HACKERS] pgbench - allow to store select results into variables

Hi Fabien, On 2016/07/16 1:33, Fabien COELHO wrote: > Here is a v2 with more or less this approach, although \into does not end > the query, but applies to the current or last sql command. A query is > still terminated with a ";". This patch needs to be rebased because of commit 64710452 (on

Re: [HACKERS] pgbench - allow to store select results into variables

Hello again, I'd be okay with SELECT 1, 2 \into one two SELECT 3 \into three Here is a v2 with more or less this approach, although \into does not end the query, but applies to the current or last sql command. A query is still terminated with a ";". Now it handles things like : --

Re: [HACKERS] pgbench - allow to store select results into variables

Hello Tom, Yeah, that's seriously nasty action-at-a-distance in my view. I'd be okay with SELECT 1, 2 \into one two SELECT 3 \into three After giving it some thoughts, it could work on compound commands if \into does not close the current sql command. Something like: SELECT 1, 2 ;

Re: [HACKERS] pgbench - allow to store select results into variables

Hello Tom, SELECT 1, 2 \; SELECT 3; \into one two three Yeah, that's seriously nasty action-at-a-distance in my view. I'd be okay with SELECT 1, 2 \into one two SELECT 3 \into three ISTM that is not the same, because then you would have two queries (over the network) instead of one, so

Re: [HACKERS] pgbench - allow to store select results into variables

On Wed, Jul 13, 2016 at 4:02 PM, Tom Lane wrote: > Robert Haas writes: > > On Sat, Jul 9, 2016 at 7:52 AM, Fabien COELHO > wrote: > >> If someone thinks that "gset" is a good idea for pgbench, which I > don't, it > >> could be

Re: [HACKERS] pgbench - allow to store select results into variables

Hello Tom, Sending a batch of requests is a feature of libpq which is accessible through pgbench by using "\;", although the fact is not documented. It makes sense for a client to send independent queries together so as to reduce latency. You're putting an awful lot of weight on an

Re: [HACKERS] pgbench - allow to store select results into variables

Fabien COELHO writes: > Sending a batch of requests is a feature of libpq which is accessible > through pgbench by using "\;", although the fact is not documented. It > makes sense for a client to send independent queries together so as to > reduce latency. You're putting

Re: [HACKERS] pgbench - allow to store select results into variables

Hello Robert, I agree: I like \into. Great! But: SELECT 1, 2 \; SELECT 3; \into one two three I think that's pretty weird. I agree that it is weird, but I do not think that it is bad. Sending a batch of requests is a feature of libpq which is accessible through pgbench by using

Re: [HACKERS] pgbench - allow to store select results into variables

Robert Haas writes: > On Wed, Jul 13, 2016 at 3:02 PM, Tom Lane wrote: >> I note also that we were talking a couple months ago >> about trying to align psql and pgbench backslash commands more closely. >> This would not be a good step in that direction.

Re: [HACKERS] pgbench - allow to store select results into variables

On Wed, Jul 13, 2016 at 3:02 PM, Tom Lane wrote: > Robert Haas writes: >> On Sat, Jul 9, 2016 at 7:52 AM, Fabien COELHO wrote: >>> If someone thinks that "gset" is a good idea for pgbench, which I don't, it >>> could be

Re: [HACKERS] pgbench - allow to store select results into variables

Robert Haas writes: > On Sat, Jul 9, 2016 at 7:52 AM, Fabien COELHO wrote: >> If someone thinks that "gset" is a good idea for pgbench, which I don't, it >> could be implemented. I think that an "into" feature, like PL/pgSQL & ECPG, >> makes more sense

Re: [HACKERS] pgbench - allow to store select results into variables

On Sat, Jul 9, 2016 at 7:52 AM, Fabien COELHO wrote: > If someone thinks that "gset" is a good idea for pgbench, which I don't, it > could be implemented. I think that an "into" feature, like PL/pgSQL & ECPG, > makes more sense for scripting. I agree: I like \into. But: >

Re: [HACKERS] pgbench - allow to store select results into variables

Also a more subjective argument: I do not like the gset automagic naming feature. I got more inspired by PL/pgSQL and ECPG which both have an "into" syntax with explicit variable names that let nothing to guessing. I like things to be simple and explicit, hence the proposed into. the gset was

Re: [HACKERS] pgbench - allow to store select results into variables

2016-07-09 11:19 GMT+02:00 Fabien COELHO : > > Hello Pavel, > > Why you are introducing \into and not \gset like psql does? >> > > Good question. > > The \into syntax I implemented is more generic, you can send a bunch of > queries together and extract the results, which

Re: [HACKERS] pgbench - allow to store select results into variables

Hello Pavel, Why you are introducing \into and not \gset like psql does? Good question. The \into syntax I implemented is more generic, you can send a bunch of queries together and extract the results, which makes sense from a client perspective where reducing latency is important:

Re: [HACKERS] pgbench - allow to store select results into variables

Hi 2016-07-09 10:20 GMT+02:00 Fabien COELHO : > > Hello devs, > > I mentionned my intention to add some features to pgbench back in March: > > https://www.postgresql.org/message-id/alpine.DEB.2.10.1603301618570.5677@sto > > The attached patch adds an \into meta command to