Ray Donnelly writes:
> On Sun, Dec 20, 2015 at 9:15 PM, <[email protected]> wrote:
> >
> > Hello,
> >
> > We're using msys2 (from 2015.05.12) as part of a front-end for a
> > build system.
> >
> > With the original msys, we experienced crashes in the automatic
> > conversion of the command line from Windows to POSIX-style
> > arguments. Using MSYS2_ARG_CONV_EXCL="*" with msys2 to disable this
> > automatic conversion has allowed us to make steady progress for
> > seven months, but recently the command tail passed to the build
> > system has changed, and we're again seeing this crash issue.
> >
>
> Do you mean that you are seeing arguments being converted from POSIX
> form to Windows form? I implemented the ="*" thing and never used it,
> so it's quite likely that some changes outside it have caused it to no
> longer work, however MSYS2_ARG_CONV_EXCL itself is used extensively.
> I'm keen for us not to crash here so if you have a simple testcase I'd
> appreciate it. To be honest, if the ground has shifted around
> arg_heuristic_with_exclusions so that things that once held true no
> long do, it should be easy to track down and fix.
I do not have a simple test case that I can release; we have not
pinned down exactly what the crash is, but have some high level
commands that do reproduce faithfully. Narrowing it down further
hits a meta stable state -- further reductions work sometimes.
We know exactly which command is failing, but have not been able to
reproduce the crash by _only_ executing the command that ultimately
crashes. This might not be corruption in Bash, but may be the version
of Gnu Make? We are still investigating.
In short, we have a tool that generates a sophisticated build process
that uses Gnu Make, after the gnerated Makefile performs a
significant amount of bookkeeping work, it attempts to start a
subordinate build process. The subordinate process is written in
Python, and the command line that's executed from within Make takes
the form:
program --option-0 value --option-1 value -- command-tail
The command tail is many thousands of bytes long, contains things
that look just like environment variable settings:
V0="value" V1="value" V2="value value"
Some of the values are pathnames with drive letters, but no '\', only
'/'.
>
<snip>
>
> There's a CYGWIN env var that we renamed to MSYS and it has an
> option (wincmdln) for whether to fill in the Windows process
> command lines or not, and by default this is set to false, so I'm
> not sure if you are talking about that here or not. i.e. by
> default, Cygwin and MSYS2 processes don't show any command lines in
> the Windows Task Manager.
> https://cygwin.com/cygwin-ug-net/using-cygwinenv.html. I guess you
> probably don't mean the Windows process' view of the command line
> though.
No, not talking about anything with Windows tools. Here's an
example, for clarity; the Makefile executes:
mkdir --parents <path>
When run from the command line, that works reliably.
When run from the infrastructure described above, using the
2015.05.12 version, it works reliably.
When run from the infrastructure described above, with the msys64
that I just installed, I see:
mkdir: missing operand
This is reproduced on the command line with invocations like:
mkdir --parents
or
mkdir
I conclude that 'something' is not executing mkdir properly. The
Makefile is using the bash shell to execute the commands, so the
natural assumption is that something in bash (or futher down in the
posix/windows layer) is having issues. This behavior is with and
without my changes.
> >
> > I'd like to get this resolved, but being new to looking at this
> > code, I have a few issues. I'm hoping you folks can point me to
> > documentation, or better workflows:
> >
> > o How do I efficiently build?
>
> Get a 20+ core workstation, with as much memory as possible and run
> everything from an ImDisk Virtual Disk.
Speed's not the issue; the issue is efficiency in my work.... you
address some a bit below.
>
> Ok, other ideas .. for many packages you can get away with using the
> fact that package () calls make install, and "make install" depends on
> "make all" and "pacman -R" (repackage) avoids re-extracting,
> rebuilding etc etc
>
> .. but, unfortunately for msys2-runtime, this isn't possible (it's
> probably not impossible either, it's just not setup that way for some
> reason).
Drat.
> It seems at the bottom of the msys2-runtime/PKGBUILD I added some
> instructions for myself for just this kind of situation, for i686,
> modify for x86_64 and fix the paths. With no msys-2.0.dll-linked-to
> programs or shells running:
Below, what is your E: drive?
> open cmd.exe
> set "PATH=C:\\msys32\\usr\\bin;%PATH%"
> E:
> pushd
> E:\m2\repo-MSYS2\msys2-runtime\src\build-i686-pc-msys\i686-pc-msys\winsup\cygwin
> C:/msys32/usr/bin/bash -c "LANG=C && make"
> copy /y new-msys-2.0.dll C:\msys32\usr\bin\msys-2.0.dll
> popd
> C:
> C:/msys32/usr/bin/strace ls / > C:/strace.txt 2>&1
Is the strace for getting new debug messages?
Is there any way to get the debug messages without strace?
> >
> > Right now, I'm just modifying msys2-runtime. I can modify the
> > source code, and run 'makepkg -sLf', but this fetches the current
> > sources from git, overwriting my changes.
> >
> > For now, I put my changes back while all the ./configure work is
> > being done, but this is really cumbersome and error prone. (It's
> > worked so far, but I don't trust it.)
> >
> > What's the best way to build without having my changes overwritten
> > with each build?
> >
>
> To make sure you aren't losing your changes when you're hacking on it
> and building a package the normal way, since this package is based on
> a git repository it's quite easy to get into the flow of using git to
> generate patches and adding them to the PKGBUILD and iterating on
> that. Here's how I do it:
> .. assuming you're running an msys2 shell, in the
> MSYS2-packages/msys2-runtime folder:
> pushd src/msys2-runtime
> nano winsup/cygwin/path.cc
> <make some changes and exit>
> # If this is a brand new patch then:
> git commit -a -m "path.cc: Attempted fix for segfault"
> git format-patch -6 # <5+1> # where 5 is the amount of patches there
> were before this new one
> <now add the new patch to the PKGBUILD in two places, source array
> and in the git -am part of perpare() >
> # Else if this is modification to a patch you're iterating on then:
> git add path.cc
> git commit --amend
> # EndIf
> git format-patch -6
> mv *.patch ../..
> popd
> updpkgsums
> git add *.patch PKGBUILD
This is the kind of information that I was hoping would appear.
Thanks. (Saw your other message, too).
>
> > o What is the preferred mechanism for contributing patches?
> >
>
> Fork https://github.com/Alexpux/MSYS2-packages or
> https://github.com/Alexpux/MINGW-packages and make pull requests.
Ok. Th
>
> > o Where is the source to 'mkdir'?
> >
>
<snip>
> It's going to be a little bit fiddly to be honest. You're going to
> have to clone msys2-runtime (ok no problem) and also:
> https://github.com/Alexpux/Cygwin.git (msys2-master branch)
> then run "git log ." in the msys2-runtime folder and redirect the
> output to a file. This folder has changed about 20 times since then.
> At each of those date points, you need
> to figure out where the HEAD was of msys2-master of Cygwin.git with a
> bit of git-fu (it may not have changed very often), then you need to
> do a bisect by hand probably:
That's what I feared.
> Anyway, I hope this has been useful to some extent.
Yes, very much so. It gives me a lot of information to make faster
progress.
thutt
--
------------------------------------------------------------------------------
_______________________________________________
Msys2-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/msys2-users
