Re: TAP test module - PostgresClient
At Sat, 3 Mar 2018 09:46:11 -0500, Peter Eisentraut wrote in <7f1e5f2f-4902-2c29-de82-381de8cc6...@2ndquadrant.com> > On 3/1/18 23:39, Michael Paquier wrote: > > On Thu, Mar 01, 2018 at 02:27:13AM -0800, Andres Freund wrote: > >> If I understand correctly there's been no progress on this since, and > >> there'd definitely need to be major work to get something we can agree > >> upon. Doesn't seem v11 material. I think we should mark this as returned > >> with feedback. Arguments against? > > > > Agreed with your position. The TAP tests rely on IPC::Run as a pillar > > of its infrastructure. I think that if we need a base API to do such > > capabilities we ought to prioritize what we can do with it first instead > > of trying to reinvent the wheel as this patch proposes in such a > > complicated way. > > I haven't seen any explanation for a problem this is solving. The > original submission contained a sample test case, by I don't see why > that couldn't be done with the existing infrastructure. > > Patch closed for now. Agreed. This is not a v11 matter. Thanks. regards, -- Kyotaro Horiguchi NTT Open Source Software Center
Re: TAP test module - PostgresClient
On 3/1/18 23:39, Michael Paquier wrote: > On Thu, Mar 01, 2018 at 02:27:13AM -0800, Andres Freund wrote: >> If I understand correctly there's been no progress on this since, and >> there'd definitely need to be major work to get something we can agree >> upon. Doesn't seem v11 material. I think we should mark this as returned >> with feedback. Arguments against? > > Agreed with your position. The TAP tests rely on IPC::Run as a pillar > of its infrastructure. I think that if we need a base API to do such > capabilities we ought to prioritize what we can do with it first instead > of trying to reinvent the wheel as this patch proposes in such a > complicated way. I haven't seen any explanation for a problem this is solving. The original submission contained a sample test case, by I don't see why that couldn't be done with the existing infrastructure. Patch closed for now. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: TAP test module - PostgresClient
On Thu, Mar 01, 2018 at 02:27:13AM -0800, Andres Freund wrote: > If I understand correctly there's been no progress on this since, and > there'd definitely need to be major work to get something we can agree > upon. Doesn't seem v11 material. I think we should mark this as returned > with feedback. Arguments against? Agreed with your position. The TAP tests rely on IPC::Run as a pillar of its infrastructure. I think that if we need a base API to do such capabilities we ought to prioritize what we can do with it first instead of trying to reinvent the wheel as this patch proposes in such a complicated way. -- Michael signature.asc Description: PGP signature
Re: TAP test module - PostgresClient
Hi, On 2018-01-11 16:54:10 +0900, Kyotaro HORIGUCHI wrote: > Thank you for the discussion. If I understand correctly there's been no progress on this since, and there'd definitely need to be major work to get something we can agree upon. Doesn't seem v11 material. I think we should mark this as returned with feedback. Arguments against? Greetings, Andres Freund
Re: TAP test module - PostgresClient
Thank you for the discussion. # I didn't noticed that the license has been changed. At Sat, 30 Dec 2017 14:35:27 -0500, Andrew Dunstan wrote in <4ab7546e-dd48-c985-2b26-e98d58920...@2ndquadrant.com> > > > On 12/30/2017 10:45 AM, Tom Lane wrote: > > Andrew Dunstan writes: > >> As for out-dating, if we used DBD::PgPP we'd not be not in great danger > >> there - it doesn't appear to have changed for many years - latest > >> version is dated 2010. If we were to use it we'd have a dependency on > >> DBI, but that in itself doesn't seem a great burden. > > [ blowing the dust off my old red fedora... ] Actually, there's a > > different problem with this proposal: you can bet that DBD::Pg has got a > > build dependency on Postgres. If Postgres starts to depend on DBD::Pg > > then we've created circular-dependency hell for packagers. > > The Pure Perl driver has no such dependency, since it doesn't require > libpq. But ... > > > I much prefer the other line of thought about doing whatever we need > > to do to make psql workable for the desired type of tests. > > ... agreed ... The module intends to perform multiple operations interactively on a session, or a transaction while performing test. We must keep the session by something persistent to do that. The PostgresClient is that for TAP tests. If we want to let psql have such feature, it would be something like "psql server" or "reconnectable session" of frontend protocol. Both seem too much or leading to something dangerous. > > Or just > > write a bespoke testing tool. > > > > > > ... that's pretty much where we came in. Agreed. And we can add anything PostgreSQL or test specific features to this. regards, -- Kyotaro Horiguchi NTT Open Source Software Center
Re: TAP test module - PostgresClient
On 12/30/2017 10:45 AM, Tom Lane wrote: > Andrew Dunstan writes: >> As for out-dating, if we used DBD::PgPP we'd not be not in great danger >> there - it doesn't appear to have changed for many years - latest >> version is dated 2010. If we were to use it we'd have a dependency on >> DBI, but that in itself doesn't seem a great burden. > [ blowing the dust off my old red fedora... ] Actually, there's a > different problem with this proposal: you can bet that DBD::Pg has got a > build dependency on Postgres. If Postgres starts to depend on DBD::Pg > then we've created circular-dependency hell for packagers. The Pure Perl driver has no such dependency, since it doesn't require libpq. But ... > I much prefer the other line of thought about doing whatever we need > to do to make psql workable for the desired type of tests. ... agreed ... > Or just > write a bespoke testing tool. > > ... that's pretty much where we came in. cheers andrew -- Andrew Dunstanhttps://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: TAP test module - PostgresClient
Andrew Dunstan writes: > As for out-dating, if we used DBD::PgPP we'd not be not in great danger > there - it doesn't appear to have changed for many years - latest > version is dated 2010. If we were to use it we'd have a dependency on > DBI, but that in itself doesn't seem a great burden. [ blowing the dust off my old red fedora... ] Actually, there's a different problem with this proposal: you can bet that DBD::Pg has got a build dependency on Postgres. If Postgres starts to depend on DBD::Pg then we've created circular-dependency hell for packagers. We could only make that work if we carefully kept the DBD::Pg requirement *out* of "make check" and anything else that a packager might care to run during package sanity checks. I suppose maybe we could live with a restriction like that, if we treat this like the SSL tests as something that doesn't get run except by special manual invocation --- but that'd reduce its utility greatly don't you think? And I fear there would be quite a risk of somebody breaking the restriction because they weren't thinking about it. I note that there are no buildfarm members running any distro packaging script, so we wouldn't find out about unintended-dependency bugs until packagers were trying to build a release. I much prefer the other line of thought about doing whatever we need to do to make psql workable for the desired type of tests. Or just write a bespoke testing tool. regards, tom lane
Re: TAP test module - PostgresClient
On 12/29/2017 08:12 AM, Tels wrote: > On Thu, December 28, 2017 10:14 pm, Tom Lane wrote: >> Craig Ringer writes: >>> Another option might be to teach the TAP infrastructure and the >>> buildfarm >>> client how to fetch cpanminus and build DBD::Pg against our build-tree, >>> so >>> we could use Perl DBI. >> As a buildfarm owner, I'd take a *really* dim view of the buildfarm >> trying to auto-fetch code off the internet. As a developer, the >> idea that "make check-world" would try to do that is even scarier. >> Buildfarm owners are likely to have taken at least some steps to >> sandbox their builds, but developers not so much. >> >> I do not think we should even think about going there. > Well, while I couldn't agree more on the "running code from the internet > is dangerous" thing, there are a few points for it, tho: > > * if you use Perl modules on your system, you are likely doing already, > anyway, as the Perl modules come, you guessed it, from the internet :) > Just because a random $PackageMaintainer signed it does mean it is really > safe. > > * And a lot of Perl modules are not in say, Debian repositories, so you > need to use CPAN (or re-invent everything). Unfortunately, the trend for > other languages seems to go into the same direction, with Ruby gems, the > python package manager, and almost everybody else re-inventing their own > packaging system, often poorly. So you might already have fallen in the > trap of "use random code from the internet". (Of course, that is not > really an argument for doing it, too...) > > * the other option seems to be "re-invent the wheel, again, locally", > which isn't always the best, either. > > I do agree tho that having "setup" or "make check" auto-fetching stuff > from the internet is not a good idea, however. Mostly because it becomes > suddenly much harder to run in closed networks w/o access and such > side-loading installations can bypass your systems packaging system, which > doesn't sound good, either. > > OTOH, it is possible to setup local repositories, or maybe even > pre-bundling modules into some sort of "approved TAP bundle" hosted on an > official server. > > The last resort would be to pre-bundle the wanted modules, but this has > the risk of outdating them fast. Plus, pre-bundled modules are not more > security vetted than the ones from the internet, so you might as well use > the CPAN version directly. > > The best course seems to me to have dependencies on the OS packackes for > the Perl modules you want to use. Not sure, however, if the build farm > client has "proper" Debian etc. packages and if it is even possible to add > these dependencies in this way. > The buildfarm client isn't even packaged as a CPAN module let alone as a bunch of OS-level packages (and if I supported Debian packaging I'd need to support every other packaging system on the planet too, including Windows). It's always seemed to me unnecessary to use something beyond a simple tarball for something that has a target of less than 100 tolerably savvy users and which requires no actual build. In any case, I agree with Craig that we'd be much better off spending time working out why we can't get IPC::Run to do everything we want on Windows. As for out-dating, if we used DBD::PgPP we'd not be not in great danger there - it doesn't appear to have changed for many years - latest version is dated 2010. If we were to use it we'd have a dependency on DBI, but that in itself doesn't seem a great burden. cheers andrew -- Andrew Dunstanhttps://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: TAP test module - PostgresClient
On Thu, December 28, 2017 10:14 pm, Tom Lane wrote: > Craig Ringer writes: >> Another option might be to teach the TAP infrastructure and the >> buildfarm >> client how to fetch cpanminus and build DBD::Pg against our build-tree, >> so >> we could use Perl DBI. > > As a buildfarm owner, I'd take a *really* dim view of the buildfarm > trying to auto-fetch code off the internet. As a developer, the > idea that "make check-world" would try to do that is even scarier. > Buildfarm owners are likely to have taken at least some steps to > sandbox their builds, but developers not so much. > > I do not think we should even think about going there. Well, while I couldn't agree more on the "running code from the internet is dangerous" thing, there are a few points for it, tho: * if you use Perl modules on your system, you are likely doing already, anyway, as the Perl modules come, you guessed it, from the internet :) Just because a random $PackageMaintainer signed it does mean it is really safe. * And a lot of Perl modules are not in say, Debian repositories, so you need to use CPAN (or re-invent everything). Unfortunately, the trend for other languages seems to go into the same direction, with Ruby gems, the python package manager, and almost everybody else re-inventing their own packaging system, often poorly. So you might already have fallen in the trap of "use random code from the internet". (Of course, that is not really an argument for doing it, too...) * the other option seems to be "re-invent the wheel, again, locally", which isn't always the best, either. I do agree tho that having "setup" or "make check" auto-fetching stuff from the internet is not a good idea, however. Mostly because it becomes suddenly much harder to run in closed networks w/o access and such side-loading installations can bypass your systems packaging system, which doesn't sound good, either. OTOH, it is possible to setup local repositories, or maybe even pre-bundling modules into some sort of "approved TAP bundle" hosted on an official server. The last resort would be to pre-bundle the wanted modules, but this has the risk of outdating them fast. Plus, pre-bundled modules are not more security vetted than the ones from the internet, so you might as well use the CPAN version directly. The best course seems to me to have dependencies on the OS packackes for the Perl modules you want to use. Not sure, however, if the build farm client has "proper" Debian etc. packages and if it is even possible to add these dependencies in this way. Best wishes, Tels
Re: TAP test module - PostgresClient
Craig Ringer writes: > Another option might be to teach the TAP infrastructure and the buildfarm > client how to fetch cpanminus and build DBD::Pg against our build-tree, so > we could use Perl DBI. As a buildfarm owner, I'd take a *really* dim view of the buildfarm trying to auto-fetch code off the internet. As a developer, the idea that "make check-world" would try to do that is even scarier. Buildfarm owners are likely to have taken at least some steps to sandbox their builds, but developers not so much. I do not think we should even think about going there. Your thoughts about making psql have some more automation-friendly output formats seem worth pursuing though. regards, tom lane
Re: TAP test module - PostgresClient
On 28 December 2017 at 16:31, Kyotaro HORIGUCHI < horiguchi.kyot...@lab.ntt.co.jp> wrote: > Hello. > > It would be useful if we have interactive sessions for TAP > tests. My first attempt was apparently unstable one that was > using psql. > > https://www.postgresql.org/message-id/20170720.152533. > 252230418.horiguchi.kyot...@lab.ntt.co.jp > > Finally the test for the patch in the thread made not need such a > machinery but it is still potentially useful. > > I found that there's already a implementation of Perl client but > I saw it was GPL. If you mean http://search.cpan.org/~arc/DBD-PgPP-0.08/lib/DBD/PgPP.pm#COPYRIGHT_AND_LICENCE then it says "The GNU General Public License as published by the Free Software Foundation; either version 2, or (at your option) any later version, or The "Artistic License" which comes with Perl." IOW, it's under the Perl license. These are the exact same terms as used by Perl its self: https://dev.perl.org/licenses/ so I don't think there's any need for hesitation in using it. But... I wonder if time is better spent figuring out what's wrong with IPC::Run on Win32 so we can use psql properly. That way we don't start writing a new test-only client, and we expand our psql coverage too. One thing that'd help a lot, IMO, would be teaching psql how to offer a better batch mode interface, where you can clearly determine: - end of resultset - start and end of error msg - end of query (psql has reported any errors and results and is ready for a new query) I'm sure this would offer benefits well beyond testing, too, given the number of pretty hairy scripts I've seen with psql and 'expect' etc. Another option might be to teach the TAP infrastructure and the buildfarm client how to fetch cpanminus and build DBD::Pg against our build-tree, so we could use Perl DBI. I know prior discussions of relying on DBI and DBD::Pg have stalled on arguments that we shouldn't require buildfarm admins and especially random people running the test suite to install and maintain them. But if we teach the test suite how to build and run them scoped to the current Pg build using PERL5LIB, users wouldn't *have* to. Internet access could be a sticking point for some, but the same would be true of using something like PgPP unless we bundled it in the Pg tree. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
