Re: Postgres Windows build system doesn't work with python installed in Program Files
On Sat, Jul 04, 2020 at 09:17:52PM +0200, Juan José Santamaría Flecha wrote: > Going through the open items in the commitfest, I see that this patch has > not been pushed. It still applies and solves the warning so, I am marking > it as RFC. Thanks, applied. I was actually waiting to see if you had more comments. > Adding a help option is a new feature, that can have its own patch without > delaying this one any further. Yep. And I am not sure if that's worth worrying either. -- Michael signature.asc Description: PGP signature
Re: Postgres Windows build system doesn't work with python installed in Program Files
On Tue, Jun 2, 2020 at 2:06 AM David Zhang wrote: > > On 2020-05-06 10:45 p.m., Michael Paquier wrote: > > On Wed, May 06, 2020 at 12:17:03AM +0200, Juan José Santamaría Flecha > wrote: > >> Please forgive me if I am being too nitpicky, but I find the comments a > >> little too verbose, a usage format might be more visual and easier to > >> explain: > >> > >> Usage: build [[CONFIGURATION] COMPONENT] > >> > >> The options are case-insensitive. > >> CONFIGURATION sets the configuration to build, "debug" or "release" (by > >> default). > >> COMPONENT defines a component to build. An empty option means all > >> components. > > Your comment makes sense to me. What about the attached then? On top > > of documenting the script usage in the code, let's trigger it if it > > gets called with more than 3 arguments. What do you think? > > > > FWIW, I forgot to mention that I don't think those warnings are worth > > a backpatch. No objections with improving things on HEAD of course. > > It would be a bonus if the build.pl can support the "help" in Windows' > way. > Going through the open items in the commitfest, I see that this patch has not been pushed. It still applies and solves the warning so, I am marking it as RFC. Adding a help option is a new feature, that can have its own patch without delaying this one any further. Regards, Juan José Santamaría Flecha
Re: Postgres Windows build system doesn't work with python installed in Program Files
Hi Michael, I performed a quick test for the path "msvc-build-init-v2.patch" using below cases: 1. perl build.pl 2. perl build.pl debug psql 3. perl build.pl RELEASE psql 4. perl build.pl deBUG psql 5. perl build.pl psql The above cases (case-insensitive) are all working great without any warning for latest master branch. When build with more than 3 parameters, yes, I got below expected message as well. c:\Users\david\Downloads\postgres\src\tools\msvc>perl build.pl DEbug psql pg_baseback Usage: build.pl [ [ ] ] Options are case-insensitive. configuration: Release | Debug. This sets the configuration to build. Default is Release. component: name of component to build. An empty value means to build all components. However, if I ask for help in a typical Windows' way, i.e, "perl build.pl -help" or "perl build.pl /?", I got some messages like below, c:\Users\david\Downloads\postgres\src\tools\msvc>perl build.pl -help Detected hardware platform: Win32 Files src/bin/pgbench/exprscan.l Files src/bin/pgbench/exprparse.y Files src/bin/psql/psqlscanslash.l Files contrib/cube/cubescan.l Files contrib/cube/cubeparse.y Files contrib/seg/segscan.l Files contrib/seg/segparse.y Generating configuration headers... Microsoft (R) Build Engine version 16.5.1+4616136f8 for .NET Framework Copyright (C) Microsoft Corporation. All rights reserved. MSBUILD : error MSB1001: Unknown switch. Switch: -help.vcxproj For switch syntax, type "MSBuild -help" c:\Users\david\Downloads\postgres\src\tools\msvc>perl build.pl /? Detected hardware platform: Win32 Files src/bin/pgbench/exprscan.l Files src/bin/pgbench/exprparse.y Files src/bin/psql/psqlscanslash.l Files contrib/cube/cubescan.l Files contrib/cube/cubeparse.y Files contrib/seg/segscan.l Files contrib/seg/segparse.y Generating configuration headers... Microsoft (R) Build Engine version 16.5.1+4616136f8 for .NET Framework Copyright (C) Microsoft Corporation. All rights reserved. MSBUILD : error MSB1001: Unknown switch. Switch: /?.vcxproj For switch syntax, type "MSBuild -help" It would be a bonus if the build.pl can support the "help" in Windows' way. Thanks, David On 2020-05-06 10:45 p.m., Michael Paquier wrote: On Wed, May 06, 2020 at 12:17:03AM +0200, Juan José Santamaría Flecha wrote: Please forgive me if I am being too nitpicky, but I find the comments a little too verbose, a usage format might be more visual and easier to explain: Usage: build [[CONFIGURATION] COMPONENT] The options are case-insensitive. CONFIGURATION sets the configuration to build, "debug" or "release" (by default). COMPONENT defines a component to build. An empty option means all components. Your comment makes sense to me. What about the attached then? On top of documenting the script usage in the code, let's trigger it if it gets called with more than 3 arguments. What do you think? FWIW, I forgot to mention that I don't think those warnings are worth a backpatch. No objections with improving things on HEAD of course. -- Michael -- David Software Engineer Highgo Software Inc. (Canada) www.highgo.ca
Re: Postgres Windows build system doesn't work with python installed in Program Files
Em qui., 7 de mai. de 2020 às 02:04, Victor Wagner
escreveu:
> В Thu, 7 May 2020 09:14:33 +0900
> Michael Paquier пишет:
>
> > On Wed, May 06, 2020 at 03:58:15PM -0300, Ranier Vilela wrote:
> > > Hacking pgbison.pl, to print PATH, shows that the path inside
> > > pgbison.pl, returned to being the original, without the addition of
> > > c:\perl\bin;c:\bin. my $out = $ENV{PATH};
> > > print "Path after system call=$out\n";
> > > Path after system
> > > call=...C:\Users\ranier\AppData\Local\Microsoft\WindowsApps;;
> > > The final part lacks: c:\perl\bin;c:\bin
> > >
> > > Now I need to find out why the path is being reset, within the perl
> > > scripts.
> >
> > FWIW, we have a buildfarm animal called drongo that runs with VS 2019,
> > that uses Python, and that is now happy. One of my own machines uses
> > VS 2019 as well and I have yet to see what you are describing here.
> > Perhaps that's related to a difference in the version of perl you are
> > using and the version of that any others?
>
>
> I doubt so. I have different machines with perl from 5.22 to 5.30 but
> none of tham exibits such weird behavoir.
>
The perl is the same,when it was working ok.
>
> Perhaps problem is that Ranier calls vcvars64.bat from the menu, and
> then calls msbuild such way that is becames unrelated process.
>
It also worked previously, using this same process, link menu and manual
path configuration.
What has changed:
1 In the environment, the python installation, which added entries to the
path.
2. Perl scripts: Use perl's $/ more idiomatically
commit beb2516e961490723fb1a2f193406afb3d71ea9c
3. Msbuild and others, have been updated.They are not the same ones that
were working before.
>
> Obvoisly buildfarm animal doesn't use menu and then starts build
> process from same CMD.EXE process, that it called vcvarsall.but into.
>
> It is same in all OSes - Windows, *nix and even MS-DOS - there is no
> way to change environment of parent process. You can change environment
> of current process (and if this process is command interpreter you can
> do so by sourcing script into it. In windows this misleadingly called
> 'CALL', but it executes commands from command file in the current
> shell, not in subshell) you can pass enivronment to the child
> processes. But you can never affect environment of the parent or
> sibling process.
>
Maybe that's what is happening, calling system, perl or msbuild, would be
creating a new environment, transferring the path that is configured in
Windows, and not the path that is in the environment that was manually
configured.
>
> The only exception is - if you know that some process would at startup
> read environment vars from some file or registry, you can modify this
> source in unrelated process.
>
> So, if you want perl in path of msbuild, started from Visual Studio,
> you should either first set path in CMD.EXE, then type command to start
> Studio from this very command window, or set path via control panel
> dialog (which modified registry). Later is what I usially do on machines
> wher I compile postgres.
>
buidfarm aninal, uses a more secure and reliable process, the path is
already configured and does not change.
Perhaps this is the way for me and for others.
It would then remain to document, to warn that to work correctly, the path
must be configured before entering the compilation environment.
regards,
Ranier Vilela
Re: Postgres Windows build system doesn't work with python installed in Program Files
On Wed, May 06, 2020 at 12:17:03AM +0200, Juan José Santamaría Flecha wrote:
> Please forgive me if I am being too nitpicky, but I find the comments a
> little too verbose, a usage format might be more visual and easier to
> explain:
>
> Usage: build [[CONFIGURATION] COMPONENT]
>
> The options are case-insensitive.
> CONFIGURATION sets the configuration to build, "debug" or "release" (by
> default).
> COMPONENT defines a component to build. An empty option means all
> components.
Your comment makes sense to me. What about the attached then? On top
of documenting the script usage in the code, let's trigger it if it
gets called with more than 3 arguments. What do you think?
FWIW, I forgot to mention that I don't think those warnings are worth
a backpatch. No objections with improving things on HEAD of course.
--
Michael
diff --git a/src/tools/msvc/build.pl b/src/tools/msvc/build.pl
index de50554e7e..b66f2c3b12 100644
--- a/src/tools/msvc/build.pl
+++ b/src/tools/msvc/build.pl
@@ -1,7 +1,9 @@
# -*-perl-*- hey - emacs - this is a perl file
-
+#
+# Script that provides 'make' functionality for msvc builds.
+#
# src/tools/msvc/build.pl
-
+#
use strict;
use warnings;
@@ -12,10 +14,22 @@ use Cwd;
use Mkvcbuild;
+sub usage
+{
+ die("Usage: build.pl [ [ ] ]\n"
+ . "Options are case-insensitive.\n"
+ . " configuration: Release | Debug. This sets the configuration\n"
+ . "to build. Default is Release.\n"
+ . " component: name of component to build. An empty value means\n"
+ . "to build all components.\n");
+}
+
chdir('../../..') if (-d '../msvc' && -d '../../../src');
die 'Must run from root or msvc directory'
unless (-d 'src/tools/msvc' && -d 'src');
+usage() unless scalar(@ARGV) <= 2;
+
# buildenv.pl is for specifying the build environment settings
# it should contain lines like:
# $ENV{PATH} = "c:/path/to/bison/bin;$ENV{PATH}";
@@ -37,17 +51,20 @@ do "./src/tools/msvc/config.pl" if (-f "src/tools/msvc/config.pl");
my $vcver = Mkvcbuild::mkvcbuild($config);
# check what sort of build we are doing
-
my $bconf = $ENV{CONFIG} || "Release";
my $msbflags = $ENV{MSBFLAGS} || "";
my $buildwhat = $ARGV[1] || "";
-if (uc($ARGV[0]) eq 'DEBUG')
+
+if (defined($ARGV[0]))
{
- $bconf = "Debug";
-}
-elsif (uc($ARGV[0]) ne "RELEASE")
-{
- $buildwhat = $ARGV[0] || "";
+ if (uc($ARGV[0]) eq 'DEBUG')
+ {
+ $bconf = "Debug";
+ }
+ elsif (uc($ARGV[0]) ne "RELEASE")
+ {
+ $buildwhat = $ARGV[0] || "";
+ }
}
# ... and do it
signature.asc
Description: PGP signature
Re: Postgres Windows build system doesn't work with python installed in Program Files
В Wed, 6 May 2020 21:23:57 -0300 Ranier Vilela пишет: > > The perl is: > Win32 strawberry-perl 5.30.1.1 > This perl would have problems when compiling PL/Perl (see my letter about week ago), but it have no problems running various build scripts for Postgres. I'm using it with MSVisualStudio 2019 and only unexpected thing I've encountered is that it comes with its own patch.exe, which doesn't like unix-style end-of-lines in patches (but OK with them in patched files). > regards, > Ranier VIlela --
Re: Postgres Windows build system doesn't work with python installed in Program Files
В Thu, 7 May 2020 09:14:33 +0900
Michael Paquier пишет:
> On Wed, May 06, 2020 at 03:58:15PM -0300, Ranier Vilela wrote:
> > Hacking pgbison.pl, to print PATH, shows that the path inside
> > pgbison.pl, returned to being the original, without the addition of
> > c:\perl\bin;c:\bin. my $out = $ENV{PATH};
> > print "Path after system call=$out\n";
> > Path after system
> > call=...C:\Users\ranier\AppData\Local\Microsoft\WindowsApps;;
> > The final part lacks: c:\perl\bin;c:\bin
> >
> > Now I need to find out why the path is being reset, within the perl
> > scripts.
>
> FWIW, we have a buildfarm animal called drongo that runs with VS 2019,
> that uses Python, and that is now happy. One of my own machines uses
> VS 2019 as well and I have yet to see what you are describing here.
> Perhaps that's related to a difference in the version of perl you are
> using and the version of that any others?
I doubt so. I have different machines with perl from 5.22 to 5.30 but
none of tham exibits such weird behavoir.
Perhaps problem is that Ranier calls vcvars64.bat from the menu, and
then calls msbuild such way that is becames unrelated process.
Obvoisly buildfarm animal doesn't use menu and then starts build
process from same CMD.EXE process, that it called vcvarsall.but into.
It is same in all OSes - Windows, *nix and even MS-DOS - there is no
way to change environment of parent process. You can change environment
of current process (and if this process is command interpreter you can
do so by sourcing script into it. In windows this misleadingly called
'CALL', but it executes commands from command file in the current
shell, not in subshell) you can pass enivronment to the child
processes. But you can never affect environment of the parent or
sibling process.
The only exception is - if you know that some process would at startup
read environment vars from some file or registry, you can modify this
source in unrelated process.
So, if you want perl in path of msbuild, started from Visual Studio,
you should either first set path in CMD.EXE, then type command to start
Studio from this very command window, or set path via control panel
dialog (which modified registry). Later is what I usially do on machines
wher I compile postgres.
--
> --
> Michael
Re: Postgres Windows build system doesn't work with python installed in Program Files
Em qua., 6 de mai. de 2020 às 21:14, Michael Paquier
escreveu:
> On Wed, May 06, 2020 at 03:58:15PM -0300, Ranier Vilela wrote:
> > Hacking pgbison.pl, to print PATH, shows that the path inside pgbison.pl
> ,
> > returned to being the original, without the addition of
> c:\perl\bin;c:\bin.
> > my $out = $ENV{PATH};
> > print "Path after system call=$out\n";
> > Path after system
> > call=...C:\Users\ranier\AppData\Local\Microsoft\WindowsApps;;
> > The final part lacks: c:\perl\bin;c:\bin
> >
> > Now I need to find out why the path is being reset, within the perl
> scripts.
>
> FWIW, we have a buildfarm animal called drongo that runs with VS 2019,
> that uses Python, and that is now happy. One of my own machines uses
> VS 2019 as well and I have yet to see what you are describing here.
> Perhaps that's related to a difference in the version of perl you are
> using and the version of that any others?
>
I really don't know what to say, I know very little about perl.
The perl is:
Win32 strawberry-perl 5.30.1.1
regards,
Ranier VIlela
Re: Postgres Windows build system doesn't work with python installed in Program Files
Em qua., 6 de mai. de 2020 às 21:08, Michael Paquier escreveu: > On Wed, May 06, 2020 at 02:11:34PM -0400, Andrew Dunstan wrote: > > We assume perl, flex and bison are in the PATH. That doesn't seem > > unreasonable, it's worked well for quite a long time. > > I recall that it is an assumption we rely on since MSVC scripts are > around, and that's rather easy to configure, so it seems to me that > changing things now would just introduce annoying changes for anybody > (developers, maintainers) using this stuff. > Ah yes, better to leave it as is. No problem for me, I already got around the difficulty. regards, Ranier Vilela
Re: Postgres Windows build system doesn't work with python installed in Program Files
On Wed, May 06, 2020 at 03:58:15PM -0300, Ranier Vilela wrote:
> Hacking pgbison.pl, to print PATH, shows that the path inside pgbison.pl,
> returned to being the original, without the addition of c:\perl\bin;c:\bin.
> my $out = $ENV{PATH};
> print "Path after system call=$out\n";
> Path after system
> call=...C:\Users\ranier\AppData\Local\Microsoft\WindowsApps;;
> The final part lacks: c:\perl\bin;c:\bin
>
> Now I need to find out why the path is being reset, within the perl scripts.
FWIW, we have a buildfarm animal called drongo that runs with VS 2019,
that uses Python, and that is now happy. One of my own machines uses
VS 2019 as well and I have yet to see what you are describing here.
Perhaps that's related to a difference in the version of perl you are
using and the version of that any others?
--
Michael
signature.asc
Description: PGP signature
Re: Postgres Windows build system doesn't work with python installed in Program Files
On Wed, May 06, 2020 at 02:11:34PM -0400, Andrew Dunstan wrote: > We assume perl, flex and bison are in the PATH. That doesn't seem > unreasonable, it's worked well for quite a long time. I recall that it is an assumption we rely on since MSVC scripts are around, and that's rather easy to configure, so it seems to me that changing things now would just introduce annoying changes for anybody (developers, maintainers) using this stuff. -- Michael signature.asc Description: PGP signature
Re: Postgres Windows build system doesn't work with python installed in Program Files
Em qua., 6 de mai. de 2020 às 15:19, Ranier Vilela escreveu: > Em qua., 6 de mai. de 2020 às 14:14, Victor Wagner > escreveu: > >> В Wed, 6 May 2020 10:21:41 -0300 >> Ranier Vilela пишет: >> >> > Em qua., 6 de mai. de 2020 às 09:53, Michael Paquier >> > escreveu: >> > >> > > On Tue, May 05, 2020 at 10:16:23AM +0300, Victor Wagner wrote: >> > > > I agree, it is better. >> > > >> > > Thanks, applied and back-patched down to 9.5. Now for the second >> > > problem of this thread.. >> > > >> > Sorry, no clue yet. >> > I hacked the perl sources, to hardcoded perl, bison and flex with >> > path.It works like this. >> >> Perl has "magic" variable $^X which expands to full path to perl >> executable, I wonder why build.pl doesn't use it to invoke secondary >> perl scripts. >> > I still don't think it's necessary, it was working well. > My main suspicions are: > 1. Path with spaces; > 2. Incompatibility with < symbol, some suggest use > >
Re: Postgres Windows build system doesn't work with python installed in Program Files
Em qua., 6 de mai. de 2020 às 14:14, Victor Wagner escreveu: > В Wed, 6 May 2020 10:21:41 -0300 > Ranier Vilela пишет: > > > Em qua., 6 de mai. de 2020 às 09:53, Michael Paquier > > escreveu: > > > > > On Tue, May 05, 2020 at 10:16:23AM +0300, Victor Wagner wrote: > > > > I agree, it is better. > > > > > > Thanks, applied and back-patched down to 9.5. Now for the second > > > problem of this thread.. > > > > > Sorry, no clue yet. > > I hacked the perl sources, to hardcoded perl, bison and flex with > > path.It works like this. > > Perl has "magic" variable $^X which expands to full path to perl > executable, I wonder why build.pl doesn't use it to invoke secondary > perl scripts. > I still don't think it's necessary, it was working well. My main suspicions are: 1. Path with spaces; 2. Incompatibility with < symbol, some suggest use
Re: Postgres Windows build system doesn't work with python installed in Program Files
On 5/6/20 1:14 PM, Victor Wagner wrote: > В Wed, 6 May 2020 10:21:41 -0300 > Ranier Vilela пишет: > >> Em qua., 6 de mai. de 2020 às 09:53, Michael Paquier >> escreveu: >> >>> On Tue, May 05, 2020 at 10:16:23AM +0300, Victor Wagner wrote: I agree, it is better. >>> Thanks, applied and back-patched down to 9.5. Now for the second >>> problem of this thread.. >>> >> Sorry, no clue yet. >> I hacked the perl sources, to hardcoded perl, bison and flex with >> path.It works like this. > Perl has "magic" variable $^X which expands to full path to perl > executable, I wonder why build.pl doesn't use it to invoke secondary > perl scripts. > We assume perl, flex and bison are in the PATH. That doesn't seem unreasonable, it's worked well for quite a long time. cheers andrew -- Andrew Dunstanhttps://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: Postgres Windows build system doesn't work with python installed in Program Files
В Wed, 6 May 2020 10:21:41 -0300 Ranier Vilela пишет: > Em qua., 6 de mai. de 2020 às 09:53, Michael Paquier > escreveu: > > > On Tue, May 05, 2020 at 10:16:23AM +0300, Victor Wagner wrote: > > > I agree, it is better. > > > > Thanks, applied and back-patched down to 9.5. Now for the second > > problem of this thread.. > > > Sorry, no clue yet. > I hacked the perl sources, to hardcoded perl, bison and flex with > path.It works like this. Perl has "magic" variable $^X which expands to full path to perl executable, I wonder why build.pl doesn't use it to invoke secondary perl scripts. --
Re: Postgres Windows build system doesn't work with python installed in Program Files
Em qua., 6 de mai. de 2020 às 10:25, Ranier Vilela
escreveu:
> Em qua., 6 de mai. de 2020 às 10:21, Ranier Vilela
> escreveu:
>
>> Em qua., 6 de mai. de 2020 às 09:53, Michael Paquier
>> escreveu:
>>
>>> On Tue, May 05, 2020 at 10:16:23AM +0300, Victor Wagner wrote:
>>> > I agree, it is better.
>>>
>>> Thanks, applied and back-patched down to 9.5. Now for the second
>>> problem of this thread..
>>>
>> Sorry, no clue yet.
>> I hacked the perl sources, to hardcoded perl, bison and flex with path.It
>> works like this.
>> For some reason, which I haven't yet discovered, msbuild is ignoring the
>> path, where perl and bison and flex are.
>> Although it is being set, within the 64-bit compilation environment of
>> msvc 2019.
>> I'm still investigating.
>>
> In fact perl, it is found, otherwise, neither build.pl would be working.
> But within the perl environment, when the system call is made, in this
> case, neither perl, bison, nor flex is found.
>
I'm using it like this, for now.
File pgbison.pl:
system("c:\\bin\\bison $headerflag $input -o $output");
File pgflex.pl:
system("c:\\bin\\flex $flexflags -o$output $input");
system("c:\\perl\\bin\\perl src\\tools\\fix-old-flex-code.pl
$output");
File Solution.pm:
system(
system('perl generate-lwlocknames.pl lwlocknames.txt');
system(
system(
system(
system(
system(
system(
system("perl create_help.pl ../../../doc/src/sgml/ref
sql_help");
system(
system(
system(
system(
system(
system('perl parse.pl < ../../../backend/parser/gram.y >
preproc.y');
system(
C:\dll\postgres\src\tools\msvc>\bin\grep bison *pm
File MSBuildProject.pm:
Running
bison on $grammarFile
c:\\perl\\bin\\perl
"src\\tools\\msvc\\pgbison.pl" "$grammarFile"
Running
bison on $grammarFile
c:\\perl\\bin\\perl
"src\\tools\\msvc\\pgbison.pl" "$grammarFile"
C:\dll\postgres\src\tools\msvc>\bin\grep flex *pm
File MSBuildProject.pm:
Running
flex on $grammarFile
c:\\perl\\bin\\perl
"src\\tools\\msvc\\pgflex.pl" "$grammarFile"
Running
flex on $grammarFile
c:\\perl\\bin\\perl
"src\\tools\\msvc\\pgflex.pl" "$grammarFile"
regards,
Ranier Vilela
Re: Postgres Windows build system doesn't work with python installed in Program Files
Em qua., 6 de mai. de 2020 às 10:21, Ranier Vilela escreveu: > Em qua., 6 de mai. de 2020 às 09:53, Michael Paquier > escreveu: > >> On Tue, May 05, 2020 at 10:16:23AM +0300, Victor Wagner wrote: >> > I agree, it is better. >> >> Thanks, applied and back-patched down to 9.5. Now for the second >> problem of this thread.. >> > Sorry, no clue yet. > I hacked the perl sources, to hardcoded perl, bison and flex with path.It > works like this. > For some reason, which I haven't yet discovered, msbuild is ignoring the > path, where perl and bison and flex are. > Although it is being set, within the 64-bit compilation environment of > msvc 2019. > I'm still investigating. > In fact perl, it is found, otherwise, neither build.pl would be working. But within the perl environment, when the system call is made, in this case, neither perl, bison, nor flex is found. regards, Ranier Vilela
Re: Postgres Windows build system doesn't work with python installed in Program Files
Em qua., 6 de mai. de 2020 às 09:53, Michael Paquier escreveu: > On Tue, May 05, 2020 at 10:16:23AM +0300, Victor Wagner wrote: > > I agree, it is better. > > Thanks, applied and back-patched down to 9.5. Now for the second > problem of this thread.. > Sorry, no clue yet. I hacked the perl sources, to hardcoded perl, bison and flex with path.It works like this. For some reason, which I haven't yet discovered, msbuild is ignoring the path, where perl and bison and flex are. Although it is being set, within the 64-bit compilation environment of msvc 2019. I'm still investigating. regards, Ranier Vilela
Re: Postgres Windows build system doesn't work with python installed in Program Files
On Tue, May 05, 2020 at 10:16:23AM +0300, Victor Wagner wrote: > I agree, it is better. Thanks, applied and back-patched down to 9.5. Now for the second problem of this thread.. -- Michael signature.asc Description: PGP signature
Re: Postgres Windows build system doesn't work with python installed in Program Files
On Tue, May 5, 2020 at 4:06 PM Michael Paquier wrote: > On Mon, May 04, 2020 at 03:42:20PM +0200, Juan José Santamaría Flecha > wrote: > > +1, seems like the way to go to me. > > Attached is a patch for that and I have gone with a simple solution, > with some bonus comments about the way things happen. > This solves the issue. Please forgive me if I am being too nitpicky, but I find the comments a little too verbose, a usage format might be more visual and easier to explain: Usage: build [[CONFIGURATION] COMPONENT] The options are case-insensitive. CONFIGURATION sets the configuration to build, "debug" or "release" (by default). COMPONENT defines a component to build. An empty option means all components. Regards, Juan José Santamaría Flecha
Re: Postgres Windows build system doesn't work with python installed in Program Files
On Mon, May 04, 2020 at 03:42:20PM +0200, Juan José Santamaría Flecha wrote:
> +1, seems like the way to go to me.
Attached is a patch for that and I have gone with a simple solution,
with some bonus comments about the way things happen. Here are the
patterns I tested for build.pl and the commands it generates, making
sure that we have the same commands with HEAD and the patch:
1) perl build.pl
msbuild pgsql.sln /verbosity:normal /p:Configuration=Release
2) perl build.pl debug
msbuild pgsql.sln /verbosity:normal /p:Configuration=Debug
3) perl build.pl release
msbuild pgsql.sln /verbosity:normal /p:Configuration=Release
4) perl build.pl foo
msbuild foo.vcxproj /verbosity:normal /p:Configuration=Release
5) perl build.pl debug foo
msbuild foo.vcxproj /verbosity:normal /p:Configuration=Debug
6) perl build.pl release foo
msbuild foo.vcxproj /verbosity:normal /p:Configuration=Release
The two warnings show up only in the first case, of course.
--
Michael
diff --git a/src/tools/msvc/build.pl b/src/tools/msvc/build.pl
index de50554e7e..782c9f4f26 100644
--- a/src/tools/msvc/build.pl
+++ b/src/tools/msvc/build.pl
@@ -1,7 +1,9 @@
# -*-perl-*- hey - emacs - this is a perl file
-
+#
+# Script that provides 'make' functionality for msvc builds.
+#
# src/tools/msvc/build.pl
-
+#
use strict;
use warnings;
@@ -36,18 +38,36 @@ do "./src/tools/msvc/config.pl" if (-f "src/tools/msvc/config.pl");
my $vcver = Mkvcbuild::mkvcbuild($config);
-# check what sort of build we are doing
+# Check what sort of build we are doing. Several patterns are
+# supported here depending on how many arguments this script is called
+# with:
+# - No arguments provided imply that all components should be built
+# with a release configuration.
+# - A first optional, case-insensitive, argument can be provided:
+# -- "Release" to trigger a build for all components with a release
+#configuration.
+# -- "Debug" to trigger a build for all components with a debug
+#build.
+# -- Specifying any other value will create a build with a release
+#configuration for the component specified as argument.
+# - A second optional argument can be provided to define a component
+# to build. This can be used with the first argument to build one
+# component at a time with a debug or release configuration.
my $bconf = $ENV{CONFIG} || "Release";
my $msbflags = $ENV{MSBFLAGS} || "";
my $buildwhat = $ARGV[1] || "";
-if (uc($ARGV[0]) eq 'DEBUG')
+
+if (defined($ARGV[0]))
{
- $bconf = "Debug";
-}
-elsif (uc($ARGV[0]) ne "RELEASE")
-{
- $buildwhat = $ARGV[0] || "";
+ if (uc($ARGV[0]) eq 'DEBUG')
+ {
+ $bconf = "Debug";
+ }
+ elsif (uc($ARGV[0]) ne "RELEASE")
+ {
+ $buildwhat = $ARGV[0] || "";
+ }
}
# ... and do it
signature.asc
Description: PGP signature
Re: Postgres Windows build system doesn't work with python installed in Program Files
В Tue, 5 May 2020 15:45:48 +0900
Michael Paquier пишет:
> On Fri, May 01, 2020 at 12:48:17PM +0300, Victor Wagner wrote:
> > Maybe. But probably original author of this code was afraid of using
> > too long chain of ->{} in the string substitution.
> >
> > So, I left this style n place.
> >
> > Nonetheless, using qq wouldn't save us from doubling backslashes.
>
> Looking at this part in more details, I find the attached much more
> readable. I have been able to test it on my own Windows environment
I agree, it is better.
> and the problem gets fixed (I have reproduced the original problem as
> well).
--
Re: Postgres Windows build system doesn't work with python installed in Program Files
On Fri, May 01, 2020 at 12:48:17PM +0300, Victor Wagner wrote:
> Maybe. But probably original author of this code was afraid of using
> too long chain of ->{} in the string substitution.
>
> So, I left this style n place.
>
> Nonetheless, using qq wouldn't save us from doubling backslashes.
Looking at this part in more details, I find the attached much more
readable. I have been able to test it on my own Windows environment
and the problem gets fixed (I have reproduced the original problem as
well).
--
Michael
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 72a21dbd41..6daa18f70e 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -498,7 +498,7 @@ sub mkvcbuild
my $pythonprog = "import sys;print(sys.prefix);"
. "print(str(sys.version_info[0])+str(sys.version_info[1]))";
my $prefixcmd =
- $solution->{options}->{python} . "\\python -c \"$pythonprog\"";
+ qq("$solution->{options}->{python}\\python" -c "$pythonprog");
my $pyout = `$prefixcmd`;
die "Could not query for python version!\n" if $?;
my ($pyprefix, $pyver) = split(/\r?\n/, $pyout);
signature.asc
Description: PGP signature
Re: Postgres Windows build system doesn't work with python installed in Program Files
On Mon, May 4, 2020 at 2:18 PM Michael Paquier wrote: > On Mon, May 04, 2020 at 09:45:54AM +0200, Juan José Santamaría Flecha > wrote: > > I think these are two different issues, python PATH and build.pl > warnings. > > For the later, you can check woodloose logs and see the warning after > > commit 8f00d84afc. > > Oh, indeed. I somewhat managed to miss these in the logs of the > buildfarm. What if we refactored the code of build.pl so as we'd > check first if $ARGV[0] is defined or not? If not defined, then we > need to have a release-quality build for all the components. How does > that sound? Something not documented is that using "release" as first > argument enforces also a release-quality build for all the components, > so we had better not break that part. > +1, seems like the way to go to me. Regards, Juan José Santamaría Flecha
Re: Postgres Windows build system doesn't work with python installed in Program Files
On Mon, May 04, 2020 at 09:45:54AM +0200, Juan José Santamaría Flecha wrote: > I think these are two different issues, python PATH and build.pl warnings. > For the later, you can check woodloose logs and see the warning after > commit 8f00d84afc. Oh, indeed. I somewhat managed to miss these in the logs of the buildfarm. What if we refactored the code of build.pl so as we'd check first if $ARGV[0] is defined or not? If not defined, then we need to have a release-quality build for all the components. How does that sound? Something not documented is that using "release" as first argument enforces also a release-quality build for all the components, so we had better not break that part. -- Michael signature.asc Description: PGP signature
Re: Postgres Windows build system doesn't work with python installed in Program Files
Em seg., 4 de mai. de 2020 às 02:58, Michael Paquier escreveu: > On Sun, May 03, 2020 at 04:23:24PM -0300, Ranier Vilela wrote: > > I don't know if it applies to the same case, but from the moment I > > installed python on the development machine, the Postgres build stopped > > working correctly. > > Although perl, flex and bison are available in the path, the build does > not > > generate files that depend on flex and bison. > > Are you following the instructions of the documentation? Here is a > link to them: > https://www.postgresql.org/docs/devel/install-windows-full.html My environment was ok and worked 100%, compiling with msvc 2019 (64 bits). > > > My guess is that you would be just missing a PATH configuration or > such because python enforced a new setting? > Perl and flex and bison, are in the path, no doubt. > > > Warning from build.pl > > Use of uninitialized value $ARGV[0] in uc at build.pl line 44. > > Use of uninitialized value $ARGV[0] in uc at build.pl line 48. > > Hmm. We have buildfarm machines using the MSVC scripts and Python, > see for example woodloose. And note that @ARGV would be normally > defined, so your warning looks fishy to me. > I'll redo from the beginning. 1. Make empty directory postgres 2. Clone postgres 3. Call msvc 2019 (64 bits) env batch 4. Setup path to perl, bison and flex set path=%path%;c:\perl;\bin;c:\bin 5. C:\dll>perl -V Summary of my perl5 (revision 5 version 30 subversion 1) configuration: Platform: osname=MSWin32 osvers=10.0.18363.476 archname=MSWin32-x64-multi-thread uname='Win32 strawberry-perl 5.30.1.1 #1 Fri Nov 22 02:24:29 2019 x64' 6. C:\dll>bison -V bison (GNU Bison) 2.7 Written by Robert Corbett and Richard Stallman. 7. C:\dll>flex -V flex 2.6.4 8. cd\dll\postgres\src\tools\msvc 9. build results: ... PrepareForBuild: Creating directory ".\Release\postgres\". Creating directory ".\Release\postgres\postgres.tlog\". InitializeBuildStatus: Creating ".\Release\postgres\postgres.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified. CustomBuild: Running bison on src/backend/bootstrap/bootparse.y 'perl' nao é reconhecido como um comando interno ou externo, um programa operável ou um arquivo em lotes. Running flex on src/backend/bootstrap/bootscanner.l 'perl' nao é reconhecido como um comando interno ou externo, um programa operável ou um arquivo em lotes. Running bison on src/backend/parser/gram.y 'perl' nao é reconhecido como um comando interno ou externo, um programa operável ou um arquivo em lotes. Running flex on src/backend/parser/scan.l 'perl' nao é reconhecido como um comando interno ou externo, um programa operável ou um arquivo em lotes. Running bison on src/backend/replication/repl_gram.y 'perl' nao é reconhecido como um comando interno ou externo, um programa operável ou um arquivo em lotes. Running flex on src/backend/replication/repl_scanner.l Running bison on src/backend/replication/syncrep_gram.y 'perl' nao é reconhecido como um comando interno ou externo, um programa operável ou um arquivo em lotes. Running flex on src/backend/replication/syncrep_scanner.l 'perl' nao é reconhecido como um comando interno ou externo, um programa operável ou um arquivo em lotes. Running bison on src/backend/utils/adt/jsonpath_gram.y Running flex on src/backend/utils/adt/jsonpath_scan.l 'perl' nao é reconhecido como um comando interno ou externo, um programa operável ou um arquivo em lotes. 'perl' nao é reconhecido como um comando interno ou externo, um programa operável ou um arquivo em lotes. 'perl' nao é reconhecido como um comando interno ou externo, um programa operável ou um arquivo em lotes. Running flex on src/backend/utils/misc/guc-file.l 'perl' nao é reconhecido como um comando interno ou externo, um programa operável ou um arquivo em lotes. C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(231,5): error MSB6006: "cmd.exe" exited with code 9009. [C:\dll\postgres\postgres.vcxproj] Done Building Project "C:\dll\postgres\postgres.vcxproj" (default targets) -- FAILED. Done Building Project "C:\dll\postgres\pgsql.sln" (default targets) -- FAILED. ... Build FAILED. "C:\dll\postgres\pgsql.sln" (default target) (1) -> "C:\dll\postgres\cyrillic_and_mic.vcxproj" (default target) (2) -> "C:\dll\postgres\postgres.vcxproj" (default target) (3) -> (CustomBuild target) -> C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(231,5): error MSB6006: "cmd.exe" exited with code 900 9. [C:\dll\postgres\postgres.vcxproj] "C:\dll\postgres\pgsql.sln" (default target) (1) -> "C:\dll\postgres\initdb.vcxproj" (default target) (32) -> "C:\dll\postgres\libpgfeutils.vcxproj" (default target) (33) -> C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(231,5): error
Re: Postgres Windows build system doesn't work with python installed in Program Files
On Mon, May 4, 2020 at 7:58 AM Michael Paquier wrote: > > > Warning from build.pl > > Use of uninitialized value $ARGV[0] in uc at build.pl line 44. > > Use of uninitialized value $ARGV[0] in uc at build.pl line 48. > > Hmm. We have buildfarm machines using the MSVC scripts and Python, > see for example woodloose. And note that @ARGV would be normally > defined, so your warning looks fishy to me. I think these are two different issues, python PATH and build.pl warnings. For the later, you can check woodloose logs and see the warning after commit 8f00d84afc. Regards, Juan José Santamaría Flecha > >
Re: Postgres Windows build system doesn't work with python installed in Program Files
On Sun, May 03, 2020 at 04:23:24PM -0300, Ranier Vilela wrote: > I don't know if it applies to the same case, but from the moment I > installed python on the development machine, the Postgres build stopped > working correctly. > Although perl, flex and bison are available in the path, the build does not > generate files that depend on flex and bison. Are you following the instructions of the documentation? Here is a link to them: https://www.postgresql.org/docs/devel/install-windows-full.html My guess is that you would be just missing a PATH configuration or such because python enforced a new setting? > Warning from build.pl > Use of uninitialized value $ARGV[0] in uc at build.pl line 44. > Use of uninitialized value $ARGV[0] in uc at build.pl line 48. Hmm. We have buildfarm machines using the MSVC scripts and Python, see for example woodloose. And note that @ARGV would be normally defined, so your warning looks fishy to me. -- Michael signature.asc Description: PGP signature
Re: Postgres Windows build system doesn't work with python installed in Program Files
Em qui., 30 de abr. de 2020 às 09:06, Victor Wagner
escreveu:
>
> Collegues,
>
> Accidently I've come over minor bug in the Mkvcbuild.pm.
> It happens, that it doesn't tolerate spaces in the $config->{python}
> path, because it want to call python in order to find out version,
> prefix and so on, and doesn't properly quote command.
>
> Fix is very simple, see attach.
>
> Patch is made against REL_12_STABLE, but probably applicable to other
> versions as well.
>
I don't know if it applies to the same case, but from the moment I
installed python on the development machine, the Postgres build stopped
working correctly.
Although perl, flex and bison are available in the path, the build does not
generate files that depend on flex and bison.
Running bison on src/backend/bootstrap/bootparse.y
'perl' nao é reconhecido como um comando interno
ou externo, um programa operável ou um arquivo em lotes.
Running flex on src/backend/bootstrap/bootscanner.l
'perl' nao é reconhecido como um comando interno
ou externo, um programa operável ou um arquivo em lotes.
Running bison on src/backend/parser/gram.y
'perl' nao é reconhecido como um comando interno
ou externo, um programa operável ou um arquivo em lotes.
etc
Warning from build.pl
Use of uninitialized value $ARGV[0] in uc at build.pl line 44.
Use of uninitialized value $ARGV[0] in uc at build.pl line 48.
regards,
Ranier Vilela
Re: Postgres Windows build system doesn't work with python installed in Program Files
В Fri, 1 May 2020 17:52:15 +0900
Michael Paquier пишет:
> On Thu, Apr 30, 2020 at 03:06:08PM +0300, Victor Wagner wrote:
> > Fix is very simple, see attach.
> >
> > Patch is made against REL_12_STABLE, but probably applicable to
> > other versions as well.
>
> Indeed, thanks.
>
> > my $pythonprog = "import sys;print(sys.prefix);"
> > .
> > "print(str(sys.version_info[0])+str(sys.version_info[1]))"; my
> > $prefixcmd =
> > - $solution->{options}->{python} . "\\python -c
> > \"$pythonprog\"";
> > + '"' . $solution->{options}->{python} . "\\python\"
> > -c \"$pythonprog\""; my $pyout = `$prefixcmd`;
> > die "Could not query for python version!\n" if $?;
> > my ($pyprefix, $pyver) = split(/\r?\n/, $pyout);
>
> This reminds me of ad7595b. Wouldn't it be better to use qq() here?
Maybe. But probably original author of this code was afraid of using
too long chain of ->{} in the string substitution.
So, I left this style n place.
Nonetheless, using qq wouldn't save us from doubling backslashes.
--
Re: Postgres Windows build system doesn't work with python installed in Program Files
On Thu, Apr 30, 2020 at 03:06:08PM +0300, Victor Wagner wrote:
> Fix is very simple, see attach.
>
> Patch is made against REL_12_STABLE, but probably applicable to other
> versions as well.
Indeed, thanks.
> my $pythonprog = "import sys;print(sys.prefix);"
> . "print(str(sys.version_info[0])+str(sys.version_info[1]))";
> my $prefixcmd =
> - $solution->{options}->{python} . "\\python -c
> \"$pythonprog\"";
> + '"' . $solution->{options}->{python} . "\\python\" -c
> \"$pythonprog\"";
> my $pyout = `$prefixcmd`;
> die "Could not query for python version!\n" if $?;
> my ($pyprefix, $pyver) = split(/\r?\n/, $pyout);
This reminds me of ad7595b. Wouldn't it be better to use qq() here?
--
Michael
signature.asc
Description: PGP signature
Postgres Windows build system doesn't work with python installed in Program Files
Collegues,
Accidently I've come over minor bug in the Mkvcbuild.pm.
It happens, that it doesn't tolerate spaces in the $config->{python}
path, because it want to call python in order to find out version,
prefix and so on, and doesn't properly quote command.
Fix is very simple, see attach.
Patch is made against REL_12_STABLE, but probably applicable to other
versions as well.
--
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 99f39caa522..6c95b7068d8 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -497,7 +497,7 @@ sub mkvcbuild
my $pythonprog = "import sys;print(sys.prefix);"
. "print(str(sys.version_info[0])+str(sys.version_info[1]))";
my $prefixcmd =
- $solution->{options}->{python} . "\\python -c \"$pythonprog\"";
+ '"' . $solution->{options}->{python} . "\\python\" -c \"$pythonprog\"";
my $pyout = `$prefixcmd`;
die "Could not query for python version!\n" if $?;
my ($pyprefix, $pyver) = split(/\r?\n/, $pyout);
