RE: Command line processing in dcrt0.cc does not match Microsoft parsing rules
On 9/5/19 9:26 PM, Eric Blake wrote: > Rather, go is not passing the command line to CreateProcess in the way > that is unambiguously parseable in the manner expected by > CommandLineToArgvW. The specific example I gave is unambiguous and is parsed correctly by CommandLineToArgvW, so if the goal is for Cygwin to effectively simulate this function, I can confirm that it is missing this case. It's reasonable that Go's algorithm should be changed to have a better chance of working with Windows programs that manually implement command line parsing and may not match expectations for all cases. I'll follow up with them and for the time being, work around the issue with my own implementation as I've since figured out how to do that. FWIW, here's the most definitive reference I've found for how Windows binaries compiled with the Microsoft C/C++ compilers do command line parsing, in case there is any desire to address this issue at some point: http://daviddeley.com/autohotkey/parameters/parameters.htm#WINCRULES Thanks for entertaining my persistence on this topic! Stephen -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] TEST: Cygwin 3.1.0-0.3
On Thu, Sep 5, 2019 at 8:23 PM Ken Brown wrote: > 3.1.0-0.3 had some bugs that were fixed in today's 3.1.0-0.4 release. Please > report back after trying it. I'm glad to hear this, because I upgraded to 3.1.0-0.4 at work today and could not duplicate the problem. I just upgraded at home and was not easily able to demonstrate the problem there either. I'll keep watching for any strangeness. -- Jim Reisert AD1C, , http://www.ad1c.us -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] TEST: Cygwin 3.1.0-0.3
On 9/5/2019 2:11 PM, Jim Reisert AD1C wrote: > Since I started using this version, I've been having some strange > hangs running g++ (called from make). The compilation just hangs > (probably at the link stage). The .o file is generated but the .exe > is not. If I remove the .o file and try again, sometimes it works. > > I had been running the August 09 version of Cygwin at work and have > never seen the problem there. > > Does this ring any bells for anyone? I just updated work to the 5 > September test version, I'll see if the problem shows up. 3.1.0-0.3 had some bugs that were fixed in today's 3.1.0-0.4 release. Please report back after trying it. Ken -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: Command line processing in dcrt0.cc does not match Microsoft parsing rules
On 9/5/19 6:45 PM, Stephen Provine via cygwin wrote: > > To prove it is not going through cmd.exe, I debugged the Go program > to the point that it calls the Win32 CreateProcess function, and the > first two arguments are: > > lpApplicationName: "C:\\cygwin64\\bin\\bash.exe" > lpCommandLine: "C:\\cygwin64\\bin\\bash.exe test.sh foo bar\\\"baz bat" And according to https://blogs.msdn.microsoft.com/twistylittlepassagesallalike/2011/04/23/everyone-quotes-command-line-arguments-the-wrong-way/ that is NOT the correct command line to be handing to CreateProcess, at least not if you want things preserved. If I read that page correctly, the unambiguously correct command line should be: "C:\\cygwin64\\bin\\bash.exe test.sh foo \"bar\\\"baz\" bat" > > So unless I'm missing something, bash.exe is not interpreting the command line > following the rules pointed to by the documentation for CommandLineToArgvW. Rather, go is not passing the command line to CreateProcess in the way that is unambiguously parseable in the manner expected by CommandLineToArgvW. And because Go is relying on a corner case of ambiguous parsing instead of well-balanced quoting, it's no surprise if cygwin doesn't parse that corner case in the manner expected. A patch to teach cygwin to parse the corner case identically would be welcome, but fixing recipient processes does not scale as well as fixing the culprit source process. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org signature.asc Description: OpenPGP digital signature
Re: RE: Command line processing in dcrt0.cc does not match Microsoft parsing rules
On Thu, 5 Sep 2019 23:45:44, "Stephen Provine via cygwin" wrote:
package main
import (
"log"
"os"
"os/exec"
)
func main() {
cmd :=3D exec.Command("C:\\cygwin64\\bin\\bash.exe", "test.sh", "foo",
"ba=
r\"baz", "bat")
cmd.Stdout =3D os.Stdout
cmd.Stderr =3D os.Stderr
if err :=3D cmd.Run(); err !=3D nil {
log.Fatal(err)
}
}
Why are you doing this? I hate to be that guy, but examples are important.
Arguably the most important lesson I have learned with computer programming is:
use the right tool for the job.
So when I need to do something, I start with a shell script. Then once a shell
script doesnt cut it anymore, I move to AWK, then Python, the Go. Substitute
your language of choice.
What I dont do is call a shell script from Go or anything else. I might call
"git.exe" or "ffmpeg.exe", but even then you could argue against it as those
binaries have libraries too.
I agree that Cygwin should be parsing to and from cmd.exe correctly. But unless
you have a valid use case, its kind of like "Cygwin theory". I have found that
historically those type issues are less likely to be resolved in timely manner,
if at all.
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
RE: Command line processing in dcrt0.cc does not match Microsoft parsing rules
On 9/5/19 5:46 PM, Eric Blake wrote:
> If you start a cygwin process from Windows, then cygwin1.dll is given
> only a single string, which it must parse into argv according to windows
> conventions (if it does not produce the same argv[] as a windows process
> using CommandLineToArgvW, then that's a bug in cygwin1.dll). But on top
> of that, if you are using cmd.exe to generate your command line, then
> you must use proper escaping, otherwise, cmd.exe can produce a command
> line that has unexpected quoting in the string handed to
> CommandLineToArgvW, and the Windows parsing when there are unbalanced
> quotes can be screwy
Great explanation, it's very helpful.
I've been using cmd.exe to generate the command line for my tests, but the
original problem was when my compiled Go binary directly executes another
Windows process using the Win32 APIs like CreateProcess directly. Here's a
simple Go program that reproduces the issue:
package main
import (
"log"
"os"
"os/exec"
)
func main() {
cmd := exec.Command("C:\\cygwin64\\bin\\bash.exe", "test.sh", "foo",
"bar\"baz", "bat")
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
if err := cmd.Run(); err != nil {
log.Fatal(err)
}
}
The output of this process is:
foo
bar\baz bat
To prove it is not going through cmd.exe, I debugged the Go program
to the point that it calls the Win32 CreateProcess function, and the
first two arguments are:
lpApplicationName: "C:\\cygwin64\\bin\\bash.exe"
lpCommandLine: "C:\\cygwin64\\bin\\bash.exe test.sh foo bar\\\"baz bat"
So unless I'm missing something, bash.exe is not interpreting the command line
following the rules pointed to by the documentation for CommandLineToArgvW.
Thanks,
Stephen
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: Command line processing in dcrt0.cc does not match Microsoft parsing rules
On 9/5/19 5:01 PM, Stephen Provine via cygwin wrote: > On 9/5/19 2:05 PM, Eric Blake wrote: >> On 9/5/19 1:31 PM, Stephen Provine via cygwin wrote: >>> Not expected. > >> Why not? That obeyed cmd's odd rules: The moment you have a " in the >> command line, that argument continues until end of line or the next " >> (regardless of how many \ precede the "). > > Now I'm really confused. Brian seemed to indicate that the POSIX rules were > followed, but you're indicating that the Windows command line parsing rules > are followed. So I assume the reality is that it is actually some mix of the > two. > Is the effective parsing logic implemented by Cygwin documented anywhere? If you start a Cygwin process from another cygwin process, then only POSIX rules are in effect. The bash shell parses its command line according to POSIX rules, creates an argv[] to pass to exec(), then cygwin1.dll manages to get that argv[], unscathed, to the new child process (bypassing Window's mechanisms), which uses the argv[] as-is. If you start a Windows process from a cygwin process, then cygwin1.dll must quote the arguments into a single concatenated string that will be reparsed in the manner that the Windows runtime expects, because the Windows process only gets a single string, not an argv[]. But cygwin should be providing the correct escaping so that windows then parses it back into the same intended argv[] (if not, that's a bug in cygwin1.dll). If you start a cygwin process from Windows, then cygwin1.dll is given only a single string, which it must parse into argv according to windows conventions (if it does not produce the same argv[] as a windows process using CommandLineToArgvW, then that's a bug in cygwin1.dll). But on top of that, if you are using cmd.exe to generate your command line, then you must use proper escaping, otherwise, cmd.exe can produce a command line that has unexpected quoting in the string handed to CommandLineToArgvW, and the Windows parsing when there are unbalanced quotes can be screwy (if it encounters a " inside an argument that was not quoted with ", then that groups remaining text into the same argument until a balanced " or end of string is encountered). So it is not always obvious at first glance if what you type in cmd.exe provides the argv[] that you intended, because of the two layers of interpretation (one from cmd to Windows, and one from Windows convention into argv[]). -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org signature.asc Description: OpenPGP digital signature
RE: Command line processing in dcrt0.cc does not match Microsoft parsing rules
On 9/5/19 2:05 PM, Eric Blake wrote: > On 9/5/19 1:31 PM, Stephen Provine via cygwin wrote: > > Not expected. > Why not? That obeyed cmd's odd rules: The moment you have a " in the > command line, that argument continues until end of line or the next " > (regardless of how many \ precede the "). Now I'm really confused. Brian seemed to indicate that the POSIX rules were followed, but you're indicating that the Windows command line parsing rules are followed. So I assume the reality is that it is actually some mix of the two. Is the effective parsing logic implemented by Cygwin documented anywhere? Thanks, Stephen -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
[ANNOUNCEMENT] wxWidgets3.0 3.0.4-1
The following packages have been uploaded to the Cygwin distribution: * libwx_baseu3.0_0-3.0.4-1 * libwx_baseu3.0-devel-3.0.4-1 * libwx_gtk2u3.0_0-3.0.4-1 * libwx_gtk2u3.0-devel-3.0.4-1 * libwx_gtk3u3.0_0-3.0.4-1 * libwx_gtk3u3.0-devel-3.0.4-1 * wxWidgets3.0-doc-3.0.4-1 wxWidgets is a set of libraries that allows C++ applications to compile and run on several different types of computer, with minimal source code changes. There is one library per supported GUI. As well as providing a common API for GUI functionality, it provides functionality for accessing some commonly-used operating system facilities, from copying and deleting files to socket and thread support. This is an update to the latest upstream release. -- Yaakov -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: Command line processing in dcrt0.cc does not match Microsoft parsing rules
On 9/5/19 1:31 PM, Stephen Provine via cygwin wrote: > My mistake - I'm very aware of the quoting rules, yet in my test script for > this > scenario I forgot to quote the arguments. However, if POSIX rules are being > implemented, there is still something I didn't expect. Here's my bash script: > > #!/bin/bash > echo "$1" > echo "$2" > echo "$3" > > And I invoke it like this from a Windows command prompt: > > C:\> bash -x script.sh foo bar\"baz bat > + echo foo > foo > + echo 'bar\baz bat' > bar\baz bat > + echo '' > > Not expected. Why not? That obeyed cmd's odd rules: The moment you have a " in the command line, that argument continues until end of line or the next " (regardless of how many \ precede the "). https://blogs.msdn.microsoft.com/twistylittlepassagesallalike/2011/04/23/everyone-quotes-command-line-arguments-the-wrong-way/ Perhaps you meant to try: c:\> bash -x script.sh foo ^"bar\^"baz^" bat > Called from within Cygwin, the behavior is correct: > > $ bash -x script.sh foo bar\"baz bat > + echo foo > foo > + echo 'bar"baz' > bar"baz > + echo bat > bat Moral of the story: POSIX rules are saner than cmd rules. > > Can you explain this difference? The reason I ask is that if this worked, > the way Go constructs the command line string would be just fine. If Go is not constructing the command line string in a manner that matches that blog post, the bug would be in Go. Presumably, Cygwin is correctly quoting things any time it calls into a non-Cygwin process (but if not, give us a test case for us to patch cygwin, or even better submit the patch). -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org signature.asc Description: OpenPGP digital signature
RE: Command line processing in dcrt0.cc does not match Microsoft parsing rules
On 2019-09-04 23:29, Brian Inglis wrote: > As standard on Unix systems, just add another level of quoting for each level > of > interpretation, as bash will process that command line, then bash will process > the script command line. My mistake - I'm very aware of the quoting rules, yet in my test script for this scenario I forgot to quote the arguments. However, if POSIX rules are being implemented, there is still something I didn't expect. Here's my bash script: #!/bin/bash echo "$1" echo "$2" echo "$3" And I invoke it like this from a Windows command prompt: C:\> bash -x script.sh foo bar\"baz bat + echo foo foo + echo 'bar\baz bat' bar\baz bat + echo '' Not expected. Called from within Cygwin, the behavior is correct: $ bash -x script.sh foo bar\"baz bat + echo foo foo + echo 'bar"baz' bar"baz + echo bat bat Can you explain this difference? The reason I ask is that if this worked, the way Go constructs the command line string would be just fine. Thanks, Stephen -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] TEST: Cygwin 3.1.0-0.3
Since I started using this version, I've been having some strange hangs running g++ (called from make). The compilation just hangs (probably at the link stage). The .o file is generated but the .exe is not. If I remove the .o file and try again, sometimes it works. I had been running the August 09 version of Cygwin at work and have never seen the problem there. Does this ring any bells for anyone? I just updated work to the 5 September test version, I'll see if the problem shows up. -- Jim Reisert AD1C, , http://www.ad1c.us -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
[ANNOUNCEMENT] mesa 19.1.6-1
The following packages have been uploaded to the Cygwin distribution: * dri-drivers-19.1.6-1 * libglapi0-19.1.6-1 * libGL1-19.1.6-1 * libGL-devel-19.1.6-1 * libOSMesa8-19.1.6-1 * libOSMesa-devel-19.1.6-1 * libEGL1-19.1.6-1 * libEGL-devel-19.1.6-1 * libGLESv2_2-19.1.6-1 * libGLESv2-devel-19.1.6-1 * khrplatform-devel-19.1.6-1 * windowsdriproto-19.1.6-1 Mesa is an open-source implementation of the OpenGL, OpenGL ES, and EGL specifications for rendering interactive 3D graphics. Complete documentation on OpenGL usage and configuration can be found here: http://x.cygwin.com/docs/ug/using-glx.html This is an update to the latest upstream release, with a few new modifications: * set process-default DPI awareness with WGL (thanks Jon Turney); * fix devel packaging for inclusion of KHR headers in GL. -- Yaakov -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
[ANNOUNCEMENT] cygwin 3.1.0-0.4 (TEST)
The following packages have been uploaded to the Cygwin distribution as test releases: * cygwin-3.1.0-0.4 * cygwin-devel-3.1.0-0.4 * cygwin-doc-3.1.0-0.4 This release comes with a couple of new features and quite a few bug fixes. The most interesting changes: - A revamp of the old FIFO code. It should now be possible to open FIFOs multiple times for writing, something the old code failed on. Courtesy Ken Brown. - Support the new pseudo console in PTY. Pseudo console is a new feature in Windows 10 1809, which provides console APIs on virtual terminal. With this patch, native console applications can work in Cygwin PTYs. Courtesy Takashi Yano. Please test. === What's new: --- - Add 24 bit color support using xterm compatibility mode in Windows 10 1703 or later. Add fake 24 bit color support for legacy console, which uses the nearest color from 16 system colors. - Support pseudo console in PTY. Pseudo console is a new feature in Windows 10 1809, which provides console APIs on virtual terminal. With this patch, native console applications can work in PTYs such as mintty, ssh, gnu screen or tmux. - New APIs: sched_getaffinity, sched_setaffinity, pthread_getaffinity_np, pthread_setaffinity_np, plus CPU_SET macros. - New APIs: dbm_clearerr, dbm_close, dbm_delete, dbm_dirfno, dbm_error, dbm_fetch, dbm_firstkey, dbm_nextkey, dbm_open, dbm_store. What changed: - - FIFOs can now be opened multiple times for writing. Addresses: https://cygwin.com/ml/cygwin/2015-03/msg00047.html https://cygwin.com/ml/cygwin/2015-12/msg00311.html - If a SA_SIGINFO signal handler changes the ucontext_t pointed to by the third parameter, follow it after returning from the handler. - Eliminate a header file name collision with on case insensitive filesystems by reverting back to . Bug Fixes - - Fix select() on console in canonical mode. Return after one line is completed, instead of when only one key is typed. - Make console I/O functions thread-safe. - Define missing MSG_EOR. It's unsupported by the underlying Winsock layer so using it in send(2), sendto(2), or sendmsg(2) will return -1 with errno set to EOPNOTSUPP and recvmsg(2) will never return it. - Fix a timerfd deadlock. Addresses: https://cygwin.com/ml/cygwin/2019-06/msg00096.html - Fix sigpending() incorrectly returning signals for unrelated threads. Addresses: https://cygwin.com/ml/cygwin/2019-07/msg00051.html - Fix a hang when opening a FIFO with O_PATH. Addresses: https://cygwin.com/ml/cygwin-developers/2019-06/msg1.html - Don't append ".lnk" when renaming a socket file. Addresses: https://cygwin.com/ml/cygwin/2019-07/msg00139.html - Make tcsetpgrp() return -1 if its argument is negative. Addresses: https://cygwin.com/ml/cygwin/2019-07/msg00166.html - Avoid mistakenly moving a process under debugger control into the process group of the debugger. Addresses a problem visible in GDB 8.1.1, related to https://cygwin.com/ml/cygwin/2019-07/msg00166.html - Return ENOEXEC from execve for arbitrary files only if the files are executable. Addresses: https://cygwin.com/ml/cygwin/2019-08/msg00054.html - Fix off-by-one in environment evaluation leading to an abort. Addresses: https://cygwin.com/ml/cygwin-patches/2019-q3/msg00069.html - Make output of /proc/[PID]/stat consistent with getpriority(). Addresses: https://cygwin.com/ml/cygwin/2019-08/msg00082.html - 64 bit only: Avoid collisions between memory maps created with shmat and Windows datastructures during fork. Addresses: https://cygwin.com/ml/cygwin/2019-08/msg00107.html === Have fun, Ken Brown, on behalf of Corinna -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: createevent access denied
Thanks for the tip - I already had logger installed, but without syslog-ng, it doesn't seem to create windows events. I also discovered that I had to run passwd -R in order to get createevent to work from a cygwin mintty session. On Thu, Sep 5, 2019 at 12:12 AM Brian Inglis < brian.ing...@systematicsw.ab.ca> wrote: > On 2019-09-04 19:29, Cary Lewis wrote: > > I am trying to write to the windows events sub system using createevent, > > but I am getting access denied. > > > > Is there a cygwin utility that provides a way to create a windows event? > > Cygwin package util-linux contains logger which should create Windows > events > with Unix fields. > > Unless you have the Cygwin syslog or syslog-ng daemon installed and > running as a > service, in which case it will write to /dev/log socket, which will be > logged to > /var/log/messages, or whatever file or processing is configured. > > Search the mail archive for previous topics including syslog in the > subject. > > -- > Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada > > This email may be disturbing to some readers as it contains > too much technical detail. Reader discretion is advised. > > -- > Problem reports: http://cygwin.com/problems.html > FAQ: http://cygwin.com/faq/ > Documentation: http://cygwin.com/docs.html > Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple > > -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Issues with multiple displays and openGL libraries after latest updates to X, GL, and Mesa
Hello, After installing the latest updates to cygwin, including the new X server, openGL libraries, and mesa, I'm having issues with a configure script that was previously working. It no longer finds the openGL libraries. Additionally, the xwin server refuses to start if more than one display is present, for reasons I don't understand. If it's of any use, the package I'm trying to build is wxWidgets 3.0.4, in the hope of eventually getting it and wxPython 4 into the cygwin package repositories. This happens on both the 32-bit and 64-bit versions of Cygwin. Does anyone have any idea why this could be happening? Hamish
Re: netinet/* in Cygwin
W dniu 2019-09-05 09:14, Csaba Raduly napisał(a): Hi Lukasz, Please don't top-post on this list. On Thu, Sep 5, 2019 at 8:49 AM Lukasz Swierczewski wrote: Thanks! It looks like it works ... You can tell where I can find: #include #include Have you tried a Linux system ? Csaba I use Windows (Cygwin on Windows 7). Code work well on Linux, but I have a problem ( Windows: and ) Luk -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: netinet/* in Cygwin
Hi Lukasz, Please don't top-post on this list. On Thu, Sep 5, 2019 at 8:49 AM Lukasz Swierczewski wrote: > > Thanks! > > It looks like it works ... > > You can tell where I can find: > > #include > #include Have you tried a Linux system ? Csaba -- You can get very substantial performance improvements by not doing the right thing. - Scott Meyers, An Effective C++11/14 Sampler So if you're looking for a completely portable, 100% standards-conformant way to get the wrong information: this is what you want. - Scott Meyers (C++TDaWYK) -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
