Re: [HACKERS] PATCH: Batch/pipelining support for libpq

Thanks a lot for the merge. I did some tests and the master branch runs up to 15% faster than the last patch I tried (v22). Amazing! Cheers, Matthieu Garrigues On Tue, Mar 16, 2021 at 9:00 PM Andres Freund wrote: > > Hi, > > On 2021-03-05 21:35:59 -0300, Alvaro Herrera wrote: > > I'll take the

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

Hi, On 2021-03-05 21:35:59 -0300, Alvaro Herrera wrote: > I'll take the weekend to think about the issue with conn->last_query and > conn->queryclass that I mentioned yesterday; other than that detail my > feeling is that this is committable, so I'll be looking at getting this > pushed early next

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

On 2021-Mar-15, Justin Pryzby wrote: > Are you going to update the assertion ? > > +#if 0 > >

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

Are you going to update the assertion ? +#if 0 Assert((meta ==

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

Here's what seems a final version of the patch. I renamed one more function: PQsendPipeline is now PQpipelineSync. I also reworded the docs in a couple of places, added a few tests to the pgbench patch, and made it work. Note the pgbench results in pipeline mode: ./pgbench -r -Mextended -n -f

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

On 2021-Mar-11, Tom Lane wrote: > I think the changes in pqParseInput3() are broken. You should have > kept the else-structure as-is and inserted the check for "not really > idle" inside the else-clause that reports an error. As it stands, > after successfully processing an

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

Alvaro Herrera writes: > [ v35-libpq-pipeline.patch ] I think the changes in pqParseInput3() are broken. You should have kept the else-structure as-is and inserted the check for "not really idle" inside the else-clause that reports an error. As it stands, after successfully processing an

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

On 2021-Mar-05, Alvaro Herrera wrote: > I'll take the weekend to think about the issue with conn->last_query and > conn->queryclass that I mentioned yesterday; other than that detail my > feeling is that this is committable, so I'll be looking at getting this > pushed early next weeks, barring

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

v33 was indeed marked a pass by cfbot. However, it only *builds* the test program, it does not *run* it. I guess we'll have to wait for the buildfarm to tell us more. In the meantime, I implemented PQsendQuery() as callable in pipeline mode; it does that by using the extended-query protocol

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

Hi, + \gset and \aset cannot be used + pipeline mode, since query results are not immediately 'used pipeline mode' -> 'used in pipeline mode' --- /dev/null +++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c @@ -0,0 +1,1144 @@ +/* + *

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

On 2021-Mar-04, Alvaro Herrera wrote: > I don't know where do __WSAFDIsSet and __imp_select come from or what to > do about them. Let's see if adding pgport and pgcommon fixes things. Indeed all those other problems were fixed and these remain. New failure is:

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

On 2021-Mar-04, Alvaro Herrera wrote: > v31. Got this: libpq_pipeline.obj : error LNK2019: unresolved external symbol __WSAFDIsSet referenced in function test_pipelined_insert [C:\projects\postgresql\libpq_pipeline.vcxproj] 5019libpq_pipeline.obj : error LNK2019: unresolved external symbol

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

On 2021-Mar-04, Alvaro Herrera wrote: > I think the problem is that the project is called pipeline and not > test_libpq, so there's no match in the name. I'm going to rename the > whole thing to src/test/modules/libpq_pipeline/ and see if the msvc > tooling likes that better. v31. -- Álvaro

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

On 2021-Mar-04, Alvaro Herrera wrote: > v30 contains changes to hopefully make it build on MSVC. Hm, that didn't work -- appveyor still says: Project "C:\projects\postgresql\pgsql.sln" (1) is building "C:\projects\postgresql\pipeline.vcxproj" (75) on node 1 (default targets). PrepareForBuild:

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

v30 contains changes to hopefully make it build on MSVC. -- Álvaro Herrera Valdivia, Chile diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml index 0553279314..c16befa314 100644 --- a/doc/src/sgml/libpq.sgml +++ b/doc/src/sgml/libpq.sgml @@ -3173,6 +3173,33 @@ ExecStatusType

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

> I think it's just because you forgot the patch. > https://www.postgresql.org/message-id/20210304142627.GA5978%40alvherre.pgsql -- Álvaro Herrera39°49'30"S 73°17'W diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml index 0553279314..c16befa314 100644 ---

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

On Thu, Mar 04, 2021 at 12:01:37PM -0300, Alvaro Herrera wrote: > Apparently, the archives system or the commitfest system is not picking > up new messages to the thread, so the CF app is trying to apply a > very old patch version. I'm not sure what's up with that. Thomas, any > clues on where

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

On Thu, Mar 4, 2021 at 8:01 PM Alvaro Herrera wrote: > Apparently, the archives system or the commitfest system is not picking > up new messages to the thread, so the CF app is trying to apply a > very old patch version. I'm not sure what's up with that. Thomas, any > clues on where to look? >

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

Apparently, the archives system or the commitfest system is not picking up new messages to the thread, so the CF app is trying to apply a very old patch version. I'm not sure what's up with that. Thomas, any clues on where to look? -- Álvaro Herrera Valdivia, Chile "Oh, great altar of

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

On 2021-Mar-04, Ibrar Ahmed wrote: > The build is failing for this patch, can you please take a look at this? > > https://cirrus-ci.com/task/4568547922804736 > https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.129221 > > > I am marking the patch "Waiting on Author" I don't

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

On Wed, Jul 15, 2020 at 12:18 AM Andres Freund wrote: > Hi, > > On 2020-07-10 19:01:49 -0400, Alvaro Herrera wrote: > > Totally unasked for, here's a rebase of this patch series. I didn't do > > anything other than rebasing to current master, solving a couple of very > > trivial conflicts,

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

Hi, On 2020-07-10 19:01:49 -0400, Alvaro Herrera wrote: > Totally unasked for, here's a rebase of this patch series. I didn't do > anything other than rebasing to current master, solving a couple of very > trivial conflicts, fixing some whitespace complaints by git apply, and > running tests to

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

On 2019-Sep-09, Amit Kapila wrote: > Thanks for picking up this. However, I noticed that previously > Horiguchi-San has given some comments on this patch [1] which doesn't > seem to be addressed or at least not all of them are addressed. It is > possible that you would have already addressed

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

On Fri, Aug 30, 2019 at 7:06 PM Nikhil Sontakke wrote: > > Hi, > > > > This patch has been around for some time now, the last version fails to > > > apply cleanly and in-depth reviews have happened. I am moving that to > > > the next CF, waiting on its author. > > > > Unfortunately, nothing was

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

Hi, > > This patch has been around for some time now, the last version fails to > > apply cleanly and in-depth reviews have happened. I am moving that to > > the next CF, waiting on its author. > > Unfortunately, nothing was changed since then, so there is already some amount > of unaddressed

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

> On Mon, Oct 1, 2018 at 8:34 AM Michael Paquier wrote: > > On Fri, Mar 23, 2018 at 02:18:09PM +0100, Daniel Verite wrote: > > There's a batch mode for pgbench in a patch posted in [1], > > with \beginbatch and \endbatch commands, but nothing > > for psql AFAICS. > > psql is more complicated

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

On Fri, Mar 23, 2018 at 02:18:09PM +0100, Daniel Verite wrote: > There's a batch mode for pgbench in a patch posted in [1], > with \beginbatch and \endbatch commands, but nothing > for psql AFAICS. > psql is more complicated because currently it uses a > blocking PQexec() call at its core. Craig

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

Kyotaro HORIGUCHI wrote: > A disucssion on psql batch mode was held in another branch of > this thread. How do we treat that? There's a batch mode for pgbench in a patch posted in [1], with \beginbatch and \endbatch commands, but nothing for psql AFAICS. psql is more complicated because

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

On Fri, Jan 5, 2018 at 4:55 PM, Vaishnavi Prabakaran < vaishnaviprabaka...@gmail.com> wrote: > > > On Tue, Nov 28, 2017 at 12:57 PM, Michael Paquier < > michael.paqu...@gmail.com> wrote: > >> On Thu, Oct 5, 2017 at 9:58 AM, Vaishnavi Prabakaran >> wrote: >> >

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

On Tue, Nov 28, 2017 at 12:57 PM, Michael Paquier wrote: > On Thu, Oct 5, 2017 at 9:58 AM, Vaishnavi Prabakaran > wrote: > > Thanks for the suggestion and, OK I will create a new patch in upcoming > > commitfest with attached patch