Re: New 'pg' consolidated metacommand patch

[Robert Haas]

On Wed, May 27, 2020 at 4:00 PM Peter Eisentraut
 wrote:
> First, consider that git has over 170 subcommands.  PostgreSQL currently
> has 36, and we're probably not going to add dozens more any time soon.
> So the issue is not of the same scope.  It also seems to me that the way
> git is organized this becomes a self-perpetuating system:  They are
> adding subcommands all the time without much care where you might in
> other situations think harder about combining them and keeping the
> surface area smaller.  For example, we wouldn't really need separate
> commands clusterdb, reindexdb, vacuumdb if we had better support in psql
> for "run this command in each database [in parallel]".

I see your point here, but I think it's clear that some people are
already concerned about the proliferation of binaries on namespace
grounds. For example, it was proposed that pg_verifybackup should be
part of pg_basebackup, and then later and by someone else that it
should be part of pg_checksums. It's quite different from either of
those things and I'm pretty confident it shouldn't be merged with
either one, but there was certainly pressure in that direction. So
apparently for some people 36 is already too many. It's not clear to
me why it should be a problem to have a lot of commands, as long as
they all start with "pg_", but if it is then we should do something
about it rather than waiting until we have more of them.

> git (and svn etc. before it) also has a much more consistent operating
> model that is sensible to reflect in the command structure.  They all
> more or less operate on a git repository, in apparently 170 different
> ways.  The 36 PostgreSQL commands don't all work in the same way.  Now
> if someone were to propose a way to combine server tools, perhaps like
> pginstancetool {init|controldata|resetwal|checksum}, and perhaps also in
> a way that actually saves code duplication and inconsistency, that would
> be something to consider.  Or maybe a client-side tool that does
> pgclienttool {create user|drop user|create database|...} -- but that
> pretty much already exists by the name of psql.  But just renaming
> everything that's shipped with PostgreSQL to one common bucket without
> regard to how it actually works and what role it plays would be
> unnecessarily confusing.

This doesn't strike me as a very practical proposal because
"pginstancetool checksums" is really stinking long compared to
"pg_checksums", where as "pg checksums" is no different, or one
keystroke better if you assume that the underscore requires pressing
shift.

> Also consider some practical concerns with the command structure you
> describe: Tab completion of commands wouldn't work anymore, unless you
> supply custom tab completion setups.  The direct association between a
> command and its man page would be broken.  Shell scripting becomes more
> challenging:  Instead of writing common things like "if which
> pg_waldump; then" you'd need some custom code, to be determined.  These
> are all solvable, but just a sum of slight annoyances, for no real benefit.

There are some potential benefits, I think, such as:

1. It seems to be the emerging standard for command line interfaces.
There's not only the 'git' example but also things like 'aws', which
is perhaps more similar to the case proposed here in that there are a
bunch of subcommands that do quite different sorts of things. I think
a lot of developers are now quite familiar with the idea of a main
command with a bunch of subcommands, and they expect to be able to
type 'git help' or 'aws help' or 'pg help' to get a list of commands,
and then 'pg help createdb' for help with that. If you don't know what
pg commands exist today, how do you discover them? You're right that
not everyone is going this way but it seems to be pretty common
(kubectl, yum, brew, npm, heroku, ...).

2. It lowers the barrier to adding more commands. For example, as
Chris Browne says, we could have a 'pg completion' command to emit
completion information for various shells. True, that would be more
necessary with this proposal. But I bet there's stuff that could be
done even today -- I think most modern shells have pretty powerful
completion facilities. Someone could propose it, but what are the
chances that a patch adding a pg_completion binary would be accepted?
I feel like the argument that this is too narrow to justify the
namespace pollution is almost inevitable.

3. It might help us achieve some better consistency between commands.
Right now we have a lot of warts in the way things are named, like
pgbench vs. pg_ctl vs. createdb, and also pg_basebackup vs.
pg_test_fsync (why not pg_base_backup or pg_testfsync?). Standardizing
on something like this would probably help us be more consistent
there. Over time, we might be able to also clean other things up.
Maybe we could get all of our client-side utilities to share a common
config file, and have a 'pg config' utility to configure it. Maybe we
could have 

Re: New 'pg' consolidated metacommand patch

[Peter Eisentraut]

On 2020-05-27 23:42, Christopher Browne wrote:
d) systemd is a Controversial System; the folk that seem particularly 
irate about it seem to be "Old Bearded Sysadmins" that hate the idea of 
redoing their understandings of how Unix systems initialize. Personally, 
my feelings are ambivalent; I'm using it where I find some use, and have 
not been displeased with my results.  And since modern systems now have 
USB and network devices added and dropped on a whim, there's a critical 
need for something newer with more dynamic responses than old SysV 
Init.  But I certainly "get" that some aren't so happy with it, and I'm 
not thrilled at the ongoing scope creep that never seems to end.


It is worth noting that systemd did not go for a one-binary-for-all 
approach.  It has different binaries for different parts of the 
functionality.  systemctl for controlling services, journalctl for 
controlling the journal, etc.  Just as a data point to show that there 
is no single "new" way to do things.


--
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: New 'pg' consolidated metacommand patch

[Mark Dilger]

> On May 27, 2020, at 2:42 PM, Christopher Browne  wrote:
> 
> There is merit to having a new, harmonious set of "pg commands."  But it's 
> eminently easy to get into trouble (and get people mad) by changing things 
> that have been working fine for many years.  Half the battle (against the 
> "getting people mad" part) is making sure that it's clear that people were 
> listened to.  Listening to the community is one of the important things to do 
> :-).

I totally agree.

There are options for keeping the existing tools and not modifying them.  If 
the "pg" command (or "pgsql" command, if we use that naming) knows, for 
example, how to execute pg_ctl, that's no harm to people who prefer to just run 
pg_ctl directly.  It only becomes a problem when this patch, or one like it, 
decides that "pg_ctl" needs to work differently, have a different set of 
command line options, etc.  The only thing I changed about pg_ctl and friends 
in the v1 patch is that they moved from BINDIR to LIBEXECDIR, and internally 
they were updated to be able to still work despite the move.  That change was 
partly designed to spark conversation.  If people prefer they get moved back 
into BINDIR, fine by me.  If people instead prefer that the patch include links 
between the old BINDIR location and the new LIBEXECDIR location, that's also 
fine by me.  The "pg" command doesn't really care either.  I'm intentionally 
not calling the shots here.  I'm asking the community members, many of whom 
expressed an interest in something along the lines of this patch.  I'm happy to 
do the grunt work of the patch to meet the community needs.

Dave Page expressed an interest upthread in standardizing the interfaces of the 
various commands.  He didn't say this, but I assume he is thinking about things 
like -d meaning --debug in initdb but meaning --dbname=CONNSTR in 
pg_basebackup.  We could break backwards compatibility by changing one or both 
of those commands to interpret those options in some new standardized way.  Or, 
we could preserve backwards compatibililty by having "pg" take --dbname and 
--debug options and pass them to the subcommand according to the grandfathered 
rules of the subcommand.  I tend towards preserving compability, but maybe 
somebody on this list wants to argue for the other side?  For new commands 
introduced after this patch gets committed (assuming it does), options could be 
passed from "pg" through to the subcommand unmolested.  That supports Robert's 
idea that people could install new subcommands from contrib modules without the 
"pg" command needing to know anything about them.  This, too, is still open to 
conversation and debate.

I'd like to hear from more community members on this.  I'm listening.

—
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: New 'pg' consolidated metacommand patch

[Christopher Browne]

On Wed, 27 May 2020 at 16:49, Isaac Morland  wrote:

> On Wed, 27 May 2020 at 16:00, Peter Eisentraut <
> peter.eisentr...@2ndquadrant.com> wrote:
>
>
>> Also consider some practical concerns with the command structure you
>> describe: Tab completion of commands wouldn't work anymore, unless you
>> supply custom tab completion setups.  The direct association between a
>> command and its man page would be broken.  Shell scripting becomes more
>> challenging:  Instead of writing common things like "if which
>> pg_waldump; then" you'd need some custom code, to be determined.  These
>> are all solvable, but just a sum of slight annoyances, for no real
>> benefit.
>>
>
> I don’t think the man page concern is justified. We could have a “help”
> subcommand, just like git; “git help add” is (to a casual observer;
> probably not precisely) the same as “man git-add”.
>

There's some very small gulf in between the concerns...

- On the one hand, git (and systems with similar "keyword" subsystems) have
arrived at reasonable solutions to cope with various of the systematic
issues, so that these shouldn't be considered to be gigantic insurmountable
barriers.

Indeed, some of these tools present systematic solutions to additional
matters.  I was very pleased when I found that some of the Kubernetes tools
I was working with included subcommands to configure my shell to know how
to do command completion.  Seems like a fine thing to me to have that
become systematically *easier*, and I think that would be a good new
subcommand...  "pg completion bash" and "pg completion zsh" would be mighty
fine things.

- On the other hand, mapping old commands that are names of programs onto
"pg subcommands" is some additional effort, and we haven't yet started
bikeshedding on the favoured names :-)

I have lately noticed some interesting looking apps wandering about that
briefly attracted my attention, but, which, due to being painfully
different from the existing commands and tools that I have already learned,
and have "muscle memory" for, am loath to leave.   I'll throw out 4
examples, 3 of them personal:
a) nnn is a terminal-based file manager.  It has some nifty features
surrounding the concept that you can set up custom file filters to look for
sorts of files that you find interesting, and then offers customizable UI
for running favorite actions against them.  I'm 25 years into using Emacs
Dired mode; as neat as nnn seems, it's not enough of an improvement to be
worth the pain in the neck of relearning stuff.
b) 3mux is a redo of tmux (which was a redo of GNU Screen), and has key
mappings that make it way easier for a new user to learn.  I'm 20-ish years
into Screen/Tmux; I wasn't looking for it to be easier to learn, because I
did that quite a while ago.
c) Kakoune is a vi-like editor which rotates from vi's "verb/object"
approach to commands to a "object/verb" approach, for apparent more
efficiency.  I think I already mentioned that my "muscle memory" is biased
by Emacs features...  I'm not adding a "rotated-vi-like" thing into my mix
:-(
d) systemd is a Controversial System; the folk that seem particularly irate
about it seem to be "Old Bearded Sysadmins" that hate the idea of redoing
their understandings of how Unix systems initialize.  Personally, my
feelings are ambivalent; I'm using it where I find some use, and have not
been displeased with my results.  And since modern systems now have USB and
network devices added and dropped on a whim, there's a critical need for
something newer with more dynamic responses than old SysV Init.  But I
certainly "get" that some aren't so happy with it, and I'm not thrilled at
the ongoing scope creep that never seems to end.

There is merit to having a new, harmonious set of "pg commands."  But it's
eminently easy to get into trouble (and get people mad) by changing things
that have been working fine for many years.  Half the battle (against the
"getting people mad" part) is making sure that it's clear that people were
listened to.  Listening to the community is one of the important things to
do :-).
-- 
When confronted by a difficult problem, solve it by reducing it to the
question, "How would the Lone Ranger handle this?"

Re: New 'pg' consolidated metacommand patch

[Isaac Morland]

On Wed, 27 May 2020 at 16:00, Peter Eisentraut <
peter.eisentr...@2ndquadrant.com> wrote:


> Also consider some practical concerns with the command structure you
> describe: Tab completion of commands wouldn't work anymore, unless you
> supply custom tab completion setups.  The direct association between a
> command and its man page would be broken.  Shell scripting becomes more
> challenging:  Instead of writing common things like "if which
> pg_waldump; then" you'd need some custom code, to be determined.  These
> are all solvable, but just a sum of slight annoyances, for no real benefit.
>

I don’t think the man page concern is justified. We could have a “help”
subcommand, just like git; “git help add” is (to a casual observer;
probably not precisely) the same as “man git-add”.

Re: New 'pg' consolidated metacommand patch

[Peter Eisentraut]

On 2020-05-27 01:19, Mark Dilger wrote:

Attached is a patch for a `pg' command that consolidates various PostgreSQL 
functionality into a single command, along the lines of how `git' commands are 
run from a single 'git' executable.  In other words,

   `pg upgrade`   # instead of `pg_upgrade`
   `pg resetwal`   # instead of `pg_resetwal`

This has been discussed before on the -hackers list, but I don't recall seeing 
a patch.  I'm submitting this patch mostly as a way of moving the conversation 
along, fully expecting the community to want some (or all) of what I wrote to 
be changed.

I'd also appreciate +1 and -1 votes on the overall idea, in case this entire 
feature, regardless of implementation, is simply something the community does 
not want.


I'm not excited about this.

First, consider that git has over 170 subcommands.  PostgreSQL currently 
has 36, and we're probably not going to add dozens more any time soon. 
So the issue is not of the same scope.  It also seems to me that the way 
git is organized this becomes a self-perpetuating system:  They are 
adding subcommands all the time without much care where you might in 
other situations think harder about combining them and keeping the 
surface area smaller.  For example, we wouldn't really need separate 
commands clusterdb, reindexdb, vacuumdb if we had better support in psql 
for "run this command in each database [in parallel]".


git (and svn etc. before it) also has a much more consistent operating 
model that is sensible to reflect in the command structure.  They all 
more or less operate on a git repository, in apparently 170 different 
ways.  The 36 PostgreSQL commands don't all work in the same way.  Now 
if someone were to propose a way to combine server tools, perhaps like 
pginstancetool {init|controldata|resetwal|checksum}, and perhaps also in 
a way that actually saves code duplication and inconsistency, that would 
be something to consider.  Or maybe a client-side tool that does 
pgclienttool {create user|drop user|create database|...} -- but that 
pretty much already exists by the name of psql.  But just renaming 
everything that's shipped with PostgreSQL to one common bucket without 
regard to how it actually works and what role it plays would be 
unnecessarily confusing.


Also consider some practical concerns with the command structure you 
describe: Tab completion of commands wouldn't work anymore, unless you 
supply custom tab completion setups.  The direct association between a 
command and its man page would be broken.  Shell scripting becomes more 
challenging:  Instead of writing common things like "if which 
pg_waldump; then" you'd need some custom code, to be determined.  These 
are all solvable, but just a sum of slight annoyances, for no real benefit.


--
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: New 'pg' consolidated metacommand patch

[Isaac Morland]

On Wed, 27 May 2020 at 12:35, Robert Haas  wrote:

> Ugh, yeah, please don't do that. Renaming them just to make it "look more
> modern" helps nobody, really. Especially if the suggestion is people should
> be using the shared-launcher binary anyway.
>
> The way things like 'git' work is that 'git thunk' just looks in a
> designated directory for an executable called git-thunk, and invokes
> it if it's found. If you want to invent your own git subcommand, you
> can. I guess 'git help' wouldn't know to list it, but you can still
> get the metacommand to execute it. That only works if you use a
> standard naming, though. If the meta-executable has to hard-code the
> names of all the individual executables that it calls, then you can't
> really make that work.
>

You could make the legacy names symlinks to the new systematic names.

Re: New 'pg' consolidated metacommand patch

[Robert Haas]

On Wed, May 27, 2020 at 4:51 AM Magnus Hagander  wrote:
>> For that reason, I did not change the names of the executables, merely their 
>> location.  During conversations with Robert off-list, we discussed renaming 
>> the executables to things like 'pg-ctl' (hyphen rather than underscore), 
>> mostly because that's the more modern way of doing it and follows what 'git' 
>> does.  To avoid breaking scripts that execute these commands by the old 
>> name, this patch doesn't go that far.  It also leaves the usage() functions 
>> alone such that when they report their own progname in the usage text, they 
>> do so under the old name.  This would need to change at some point, but I'm 
>> unclear on whether that would be for v14 or if it would be delayed.
>
> Ugh, yeah, please don't do that. Renaming them just to make it "look more 
> modern" helps nobody, really. Especially if the suggestion is people should 
> be using the shared-launcher binary anyway.

The way things like 'git' work is that 'git thunk' just looks in a
designated directory for an executable called git-thunk, and invokes
it if it's found. If you want to invent your own git subcommand, you
can. I guess 'git help' wouldn't know to list it, but you can still
get the metacommand to execute it. That only works if you use a
standard naming, though. If the meta-executable has to hard-code the
names of all the individual executables that it calls, then you can't
really make that work.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: New 'pg' consolidated metacommand patch

[Robert Haas]

On Wed, May 27, 2020 at 4:51 AM Magnus Hagander  wrote:
> As mentioned at least once before, the "pg" name is already taken in posix. 
> Granted it has been removed now, but it was removed from posix in 2018, which 
> I think is nowhere near soon enough to "steal. See for example 
> https://en.wikipedia.org/wiki/Pg_(Unix)

The previous discussion of this general topic starts at
http://postgr.es/m/ca+tgmozqmdy7nlrq96nlm-wrnmnpy90qdmvz6ltjo941gwg...@mail.gmail.com
and the discussion of this particular issue starts at
https://www.postgresql.org/message-id/15135.1586703479%40sss.pgh.pa.us

I think I agree with what Andres said on that thread: rather than
waiting a long time to see what happens, we should grab the name
before somebody else does. As also discussed on that thread, perhaps
we should have the official name of the binary be 'pgsql' with 'pg' as
a symlink that some packagers might choose to omit.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: New 'pg' consolidated metacommand patch

[Christoph Moench-Tegeder]

## Magnus Hagander (mag...@hagander.net):

> Ugh, yeah, please don't do that. Renaming them just to make it "look more
> modern" helps nobody, really. Especially if the suggestion is people should
> be using the shared-launcher binary anyway.

Quick, let's invent a fancy name like "microcommand" for doing this
like we're used to; then we tell people that's the new "modern" (anybody
cares to write a Medium article for that? (why Medium? it's neither
Rare nor Well Done)). What might make sense for (some) version control
systems and is tempting in languages which have forgotten howto shared
library might not be the best architecture for everything. What has
become of the collection of small dedicated tools?

Regards,
Christoph

-- 
Spare Space

Re: New 'pg' consolidated metacommand patch

[Dave Page]

On Wed, May 27, 2020 at 3:00 PM Mark Dilger 
wrote:

>
>
> > On May 26, 2020, at 4:59 PM, David G. Johnston <
> david.g.johns...@gmail.com> wrote:
> >
> > On Tue, May 26, 2020 at 4:19 PM Mark Dilger <
> mark.dil...@enterprisedb.com> wrote:
> > I'd also appreciate +1 and -1 votes on the overall idea, in case this
> entire feature, regardless of implementation, is simply something the
> community does not want.
> >
> > -1, at least as part of core.  My question would be how much of this is
> would be needed if someone were to create an external project that
> installed a "pg" command on top of an existing PostgreSQL installation.  Or
> put differently, how many of the changes to the existing binaries are
> required versus nice-to-have?
>
> If the only goal of something like this were to have a frontend that could
> execute the various postgres binaries, then I'd say no changes to those
> binaries would be needed, and the frontend would not be worth very much.
> The value in having the frontend is that it makes it less difficult to
> introduce new commands to the postgres suite of commands, as you don't need
> to worry about whether another executable by the same name might happen to
> already exist somewhere.  Even introducing a command named "pg" has already
> gotten such a response on this thread.  By having the commands installed in
> postgres's libexec rather than bin, you can put whatever commands you want
> in libexec without worrying about conflicts.  That still leaves open the
> question of whether existing commands get moved into libexec, and if so, if
> they keep the same name.  An external project for this would be worthless
> in this regard, as the community wouldn't get any benefit when debating the
> merits of introducing a new command vs. the potential for conflicts.
>

The issue you raise can almost certainly be resolved simply by prefixing
pg- or something similar on all the existing binary names.

I think the beauty of having a single CLI executable is that we can
redesign the user interface to make it nice and consistent for all the
different functions it offers, and to cleanup old cruft such as createuser
vs. createrole and pgbench vs. pg_* and so on.

-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: New 'pg' consolidated metacommand patch

[Mark Dilger]

> On May 27, 2020, at 1:50 AM, Magnus Hagander  wrote:
> 
> On Wed, May 27, 2020 at 1:19 AM Mark Dilger  
> wrote:
> Hackers,
> 
> Attached is a patch for a `pg' command that consolidates various PostgreSQL 
> functionality into a single command, along the lines of how `git' commands 
> are run from a single 'git' executable.  In other words,
> 
>   `pg upgrade`   # instead of `pg_upgrade`
>   `pg resetwal`   # instead of `pg_resetwal`
> 
> This has been discussed before on the -hackers list, but I don't recall 
> seeing a patch.  I'm submitting this patch mostly as a way of moving the 
> conversation along, fully expecting the community to want some (or all) of 
> what I wrote to be changed.
> 
> As mentioned at least once before, the "pg" name is already taken in posix. 
> Granted it has been removed now, but it was removed from posix in 2018, which 
> I think is nowhere near soon enough to "steal. See for example 
> https://en.wikipedia.org/wiki/Pg_(Unix)

Care to recommend a different name?

> All other executables have been moved to LIBEXECDIR where they retain their 
> old names and can still be run directly from the command line.  If we 
> committed this patch for v14, I think it makes sense that packagers could put 
> the LIBEXECDIR in the PATH so that 3rd-party scripts which call pg_ctl, 
> initdb, etc. continue to work. 
> 
> I would definitely not expect a packager to change the PATH, as also 
> mentioned by others. More likely options would be to symlink the binaries 
> into the actual  bindir, or just set both those directories to the same one 
> (in the path) for a number of releases as a transition.

There is nothing in the patch that expects packagers to muck with the PATH.  
The idea, badly phrased, was that by keeping the names of the executables and 
only changing locations, people would have more options for how to deal with 
the change.

> But you should definitely poll the packagers separately to make sure 
> something is done that works well for them -- especially when it comes to 
> integrating with for example the debian/ubuntu wrapper system that already 
> supports multiple parallel installs. And mind that they don't typically 
> follow hackers actively (I think), so it would be worthwhile to bring their 
> attention specifically to the thread. In many ways I'd find them more 
> important to get input from than most "other hackers" :)

Yeah, good advice.  Since I've already floated this on -hackers, I might wait a 
few days for comment, then if it looks encouraging, ask on other lists.

> For that reason, I did not change the names of the executables, merely their 
> location.  During conversations with Robert off-list, we discussed renaming 
> the executables to things like 'pg-ctl' (hyphen rather than underscore), 
> mostly because that's the more modern way of doing it and follows what 'git' 
> does.  To avoid breaking scripts that execute these commands by the old name, 
> this patch doesn't go that far.  It also leaves the usage() functions alone 
> such that when they report their own progname in the usage text, they do so 
> under the old name.  This would need to change at some point, but I'm unclear 
> on whether that would be for v14 or if it would be delayed.
> 
> Ugh, yeah, please don't do that. Renaming them just to make it "look more 
> modern" helps nobody, really. Especially if the suggestion is people should 
> be using the shared-launcher binary anyway. 
> 
> usage() seems more reasonable to change as part of a patch like this.
> 
> 
> The binaries 'createuser' and 'dropuser' might be better named 'createrole' 
> and 'droprole'.  I don't currently have aliases in this patch, but it might 
> make sense to allow 'pg createrole' as a synonym for 'pg createuser' and 'pg 
> droprole' as a synonym for 'pg dropuser'.  I have not pursued that yet, 
> largely because as soon as you go that route, it starts making sense to have 
> things like 'pg drop user', 'pg cluster db' and so forth, with the extra 
> spaces.  How far would people want me to go in this direction?
> 
> I'd say a createrole would make sense, but certainly not a "create role". 
> You'd end up with unlimited number of commands. But in either of them, I'd 
> say keep aliases completely out of it for a first iteration.

Ok.

> Prior to this patch, postgres binaries that need to execute other postgres 
> binaries determine the BINDIR using find_my_exec and trimming off their own 
> executable name.  They can then assume the other binary is in that same 
> directory.  After this patch, binaries need to find the common prefix ROOTDIR 
> = commonprefix(BINDIR,LIBEXECDIR) and then assume the other binary is either 
> in ROOTDIR/binsuffix or ROOTDIR/libexecsuffix.  This may cause problems on 
> windows if BINDIR and LIBEXECDIR are configured on different drives, as there 
> won't be a common prefix of C:\my\pg\bin and D:\my\pg\libexec.  I'm hoping 
> somebody with more Windows savvy expresses an opinion about 

Re: New 'pg' consolidated metacommand patch

[Mark Dilger]

> On May 27, 2020, at 1:13 AM, Dave Page  wrote:
> 
> Hi
> 
> On Wed, May 27, 2020 at 12:19 AM Mark Dilger  
> wrote:
> 
> I think it makes sense that packagers could put the LIBEXECDIR in the PATH so 
> that 3rd-party scripts which call pg_ctl, initdb, etc. continue to work.  
> 
> Having packages that futz with the PATH is generally a bad idea, especially 
> those that support side-by-side installations of different versions. None of 
> ours (EDBs) will be doing so.

I probably phrased that badly.  The operative word in that sentence was 
"could".  If we rename the binaries, people can still make links to them from 
the old name, but if we don't rename them, then either links or PATH changes 
*could* be used.  I'm not trying to recommend any particular approach.  
Mentioning "packagers" probably wasn't helpful, as "people" works just as well 
in that sentence.

There is also the option of not moving the binaries at all, and only putting 
new commands into libexec, while grandfathering existing ones in bin.

—
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: New 'pg' consolidated metacommand patch

[Mark Dilger]

> On May 26, 2020, at 4:59 PM, David G. Johnston  
> wrote:
> 
> On Tue, May 26, 2020 at 4:19 PM Mark Dilger  
> wrote:
> I'd also appreciate +1 and -1 votes on the overall idea, in case this entire 
> feature, regardless of implementation, is simply something the community does 
> not want.
> 
> -1, at least as part of core.  My question would be how much of this is would 
> be needed if someone were to create an external project that installed a "pg" 
> command on top of an existing PostgreSQL installation.  Or put differently, 
> how many of the changes to the existing binaries are required versus 
> nice-to-have?

If the only goal of something like this were to have a frontend that could 
execute the various postgres binaries, then I'd say no changes to those 
binaries would be needed, and the frontend would not be worth very much.  The 
value in having the frontend is that it makes it less difficult to introduce 
new commands to the postgres suite of commands, as you don't need to worry 
about whether another executable by the same name might happen to already exist 
somewhere.  Even introducing a command named "pg" has already gotten such a 
response on this thread.  By having the commands installed in postgres's 
libexec rather than bin, you can put whatever commands you want in libexec 
without worrying about conflicts.  That still leaves open the question of 
whether existing commands get moved into libexec, and if so, if they keep the 
same name.  An external project for this would be worthless in this regard, as 
the community wouldn't get any benefit when debating the merits of introducing 
a new command vs. the potential for conflicts.

—
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: New 'pg' consolidated metacommand patch

[Magnus Hagander]

On Wed, May 27, 2020 at 1:19 AM Mark Dilger 
wrote:

> Hackers,
>
> Attached is a patch for a `pg' command that consolidates various
> PostgreSQL functionality into a single command, along the lines of how
> `git' commands are run from a single 'git' executable.  In other words,
>
>   `pg upgrade`   # instead of `pg_upgrade`
>   `pg resetwal`   # instead of `pg_resetwal`
>
> This has been discussed before on the -hackers list, but I don't recall
> seeing a patch.  I'm submitting this patch mostly as a way of moving the
> conversation along, fully expecting the community to want some (or all) of
> what I wrote to be changed.
>

As mentioned at least once before, the "pg" name is already taken in posix.
Granted it has been removed now, but it was removed from posix in 2018,
which I think is nowhere near soon enough to "steal. See for example
https://en.wikipedia.org/wiki/Pg_(Unix)



All other executables have been moved to LIBEXECDIR where they retain their
> old names and can still be run directly from the command line.  If we
> committed this patch for v14, I think it makes sense that packagers could
> put the LIBEXECDIR in the PATH so that 3rd-party scripts which call pg_ctl,
> initdb, etc. continue to work.


I would definitely not expect a packager to change the PATH, as also
mentioned by others. More likely options would be to symlink the binaries
into the actual  bindir, or just set both those directories to the same one
(in the path) for a number of releases as a transition.

But you should definitely poll the packagers separately to make sure
something is done that works well for them -- especially when it comes to
integrating with for example the debian/ubuntu wrapper system that already
supports multiple parallel installs. And mind that they don't typically
follow hackers actively (I think), so it would be worthwhile to bring their
attention specifically to the thread. In many ways I'd find them more
important to get input from than most "other hackers" :)



> For that reason, I did not change the names of the executables, merely
> their location.  During conversations with Robert off-list, we discussed
> renaming the executables to things like 'pg-ctl' (hyphen rather than
> underscore), mostly because that's the more modern way of doing it and
> follows what 'git' does.  To avoid breaking scripts that execute these
> commands by the old name, this patch doesn't go that far.  It also leaves
> the usage() functions alone such that when they report their own progname
> in the usage text, they do so under the old name.  This would need to
> change at some point, but I'm unclear on whether that would be for v14 or
> if it would be delayed.
>

Ugh, yeah, please don't do that. Renaming them just to make it "look more
modern" helps nobody, really. Especially if the suggestion is people should
be using the shared-launcher binary anyway.

usage() seems more reasonable to change as part of a patch like this.


The binaries 'createuser' and 'dropuser' might be better named 'createrole'
> and 'droprole'.  I don't currently have aliases in this patch, but it might
> make sense to allow 'pg createrole' as a synonym for 'pg createuser' and
> 'pg droprole' as a synonym for 'pg dropuser'.  I have not pursued that yet,
> largely because as soon as you go that route, it starts making sense to
> have things like 'pg drop user', 'pg cluster db' and so forth, with the
> extra spaces.  How far would people want me to go in this direction?
>

I'd say a createrole would make sense, but certainly not a "create role".
You'd end up with unlimited number of commands. But in either of them, I'd
say keep aliases completely out of it for a first iteration.


Prior to this patch, postgres binaries that need to execute other postgres
> binaries determine the BINDIR using find_my_exec and trimming off their own
> executable name.  They can then assume the other binary is in that same
> directory.  After this patch, binaries need to find the common prefix
> ROOTDIR = commonprefix(BINDIR,LIBEXECDIR) and then assume the other binary
> is either in ROOTDIR/binsuffix or ROOTDIR/libexecsuffix.  This may cause
> problems on windows if BINDIR and LIBEXECDIR are configured on different
> drives, as there won't be a common prefix of C:\my\pg\bin and
> D:\my\pg\libexec.  I'm hoping somebody with more Windows savvy expresses an
> opinion about how to handle this.
>

Maybe the "pg" binary could just pass down it's own location as a parameter
to the binary it calls, thereby making sure that binary has direct access
to both?


-- 
 Magnus Hagander
 Me: https://www.hagander.net/ 
 Work: https://www.redpill-linpro.com/ 

Re: New 'pg' consolidated metacommand patch

[Dave Page]

Hi

On Wed, May 27, 2020 at 12:19 AM Mark Dilger 
wrote:

>
> I think it makes sense that packagers could put the LIBEXECDIR in the PATH
> so that 3rd-party scripts which call pg_ctl, initdb, etc. continue to
> work.


Having packages that futz with the PATH is generally a bad idea, especially
those that support side-by-side installations of different versions. None
of ours (EDBs) will be doing so.

-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: New 'pg' consolidated metacommand patch

[David G. Johnston]

On Tue, May 26, 2020 at 4:19 PM Mark Dilger 
wrote:

> I'd also appreciate +1 and -1 votes on the overall idea, in case this
> entire feature, regardless of implementation, is simply something the
> community does not want.
>

-1, at least as part of core.  My question would be how much of this is
would be needed if someone were to create an external project that
installed a "pg" command on top of an existing PostgreSQL installation.  Or
put differently, how many of the changes to the existing binaries are
required versus nice-to-have?

David J.