Re: silent option not silencing cURL error

[Brian Inglis]

On 2016-11-30 20:00, Steven Penny wrote:
> $ curl --version
> curl 7.51.0
> Piping cURL to head gives me an error, as expected:
> $ curl https://github.com/github | head
> [...]
> (23) Failed writing body
> However adding the silent option, is not silencing it:
> $ curl --silent https://github.com/github | head
> [...]
> (23) Failed writing body
> I can quiet the error with:
> curl https://github.com/github 2>/dev/null | head
> but silent should be taking care of this. Note the Windows version
> here works as expected:
> http://bintray.com/vszakats/generic/curl
> cross post:
> http://stackoverflow.com/q/40901579/silent-option-not-silencing-curl-error

This is a common problem: not handling a closed pipe silently.
Testing curl on WSL/UfW and Debian does not give this error.
This may be a regression in the newer Cygwin curl version 7.51 vs older 
UfW 7.35 and Debian 7.38, or it could be a design decision.
See if has been reported upstream at curl.haxx.se, and report it if not.
Using | sed '10q' also gives the error.
You can work around it using | awk 'NR<=10', | awk -vn=10 'NR<=n', or 
| awk 'NR<=n' n=10 to get a similar result.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

silent option not silencing cURL error

[Steven Penny]

$ curl --version
curl 7.51.0

Piping cURL to head gives me an error, as expected:

$ curl https://github.com/github | head
[...]
(23) Failed writing body

However adding the silent option, is not silencing it:

$ curl --silent https://github.com/github | head
[...]
(23) Failed writing body

I can quiet the error with:

curl https://github.com/github 2>/dev/null | head

but silent should be taking care of this. Note the Windows version here works as
expected:

http://bintray.com/vszakats/generic/curl

cross post:

http://stackoverflow.com/q/40901579/silent-option-not-silencing-curl-error

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: pdfseparate does nothing for me?

[Yaakov Selkowitz]

On 2016-11-14 13:52, Ian Lambert wrote:

I occasionally use pdfseparate and pdfjoin or pdfunite to manipulate pdf files.
Now (not sure since when), pdfseparate produces no output, whether in an xterm 
or mintty.
I'd appreciate knowing if anyone else has this problem, or if you see anything 
I should fix.


You have a mix of old and new packages on your system.  Run setup again 
and allow it to update everything.


--
Yaakov

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin TCP slow

[Lee]

I don't know if this qualifies as a simple test case, but

if you don't already have wireshark, get it from
https://www.wireshark.org/download.html
get iperf-2.0.9.tar.gz from https://sourceforge.net/projects/iperf2/files/
change the setsockopt calls on lines 125 & 132 of src/tcp_window_size.c to
  rc = 0;

./configure  --prefix=/usr/local
make && make install

start wireshark & add a column for bytes in flight:
edit / preferences
appearance / columns
click on the "+" button to get a new row
double click on the "New Column", type BIF
double click the empty bit in the Fields column
type tcp.an  which pulls up a pick list; click on tcp.analysis.bytes_in_flight
click on OK
find a public iperf server -- I got lucky & found one ~65ms away, so
thruput is going to be constrained by the 130ms round trip time.

start the wireshark capture
iperf  -c 

when it's done stop the capture & click on the BIF column header

What I get is a max bytes in flight of 66560

recompile iperf using the windows cross-compiler
make clean
make distclean
./configure  --prefix=/usr/local --build=i686-pc-cygwin --host=i686-w64-mingw32
make && make install

start capturing
iperf  -c 

when it's done stop the capture & sort on the BIF column
What I get is a max bytes in flight of 196608

So, for me, it's about a 3X difference between the native & cygwin app.


If the problem really is in net.cc as the OP said, have a look at
https://cygwin.com/git/gitweb.cgi?p=newlib-cygwin.git;a=blob;f=winsup/cygwin/net.cc;h=e4805d3e11c3cea09b1cdfa27170dfe626265125;hb=HEAD

starting at line 587
/* Raise default buffer sizes (instead of WinSock default 8K).

I think starting with Windows Vista the default is tcp autotuning.  So
unless there's some other reason for setting the send/receive buffer
it seems that cygwin apps would be better off going with the defaults.

Lee

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin TCP slow

[Brian Inglis]

On 2016-11-30 07:49, Lee wrote:
>> If you also provided a Cygwin port of iperf3
> Any hints on how to do that?

If you are running under Cygwin, first install cygport (and let setup 
install all its dependencies), and any iperf3 dependencies, then read 
the cygport html doc, and create a minimal package .cygport file (or 
download iperf source package and modify its .cygport for iperf3).
It defaults to build with autotools, so not much more needs to be 
done if that is used.
Otherwise you may have to inherit one or more other cygclasses, 
or modify default high level shell functions to override the 
behaviour, obeying the rules about which directory variables to use 
and in which to perform work.

If you only have Linux, you will have to check the setup.ini entry 
for cygport and install it and the appropriate (cross) tools, 
packages, and libraries, and the same for iperf3, for the host/target 
platform.

Check the Source*Ware* cygwin-apps and Source*Forge* 
cygwin-ports-general mailing lists archives, also available on 
mail-archive.com and other ml mirrors, for more info on cygports and 
cross-compiling.

> Trying to build using the i686-w64-mingw32-gcc cross-compiler dies 
> because nanosleep is missing. How to figure out what needs to be 
> installed to get that function?
> $ ./configure --prefix=/usr/local  --build=i686-pc-cygwin
> --host=i686-w64-mingw32
>   <.. snip ..>
> checking for ANSI C header files... (cached) yes
> checking for library containing floor... none required
> checking for library containing nanosleep... no
> nanosleep() required for timing operations.

For nanosleep, there appear to be two different interfaces in time.h, 
depending on whether _POSIX_TIMERS or _POSIX_CLOCK_SELECTION are 
defined: check the .h for which interface is used and set the 
appropriate definition to a POSIX release date e.g. 200809L.
The headers, newlib, and gcc support a variety of standards and 
flavours of platform extensions, not all of which are compatible.

> Trying to build using the cygwin gcc compiler does because of
> "conflicting types for 'iprintf'"
> 
> aside: Why would one redefine a standard routine instead of making
> their own routine that calls the standard routine??  The conflicting
> bit is in iperf_api.h:
> /* Custom printf routine. */
> int iprintf(struct iperf_test *test, const char *format, ...)
> __attribute__ ((format(printf,2,3)));
> 
> In any case, how does one reconcile
>   int iprintf( ...
> with
>   int _EXFUN(iprintf, ( ...

Original iperf predates POSIX additions, and they did not heed the C 
and POSIX standards notes to avoid defining functions containing 
certain substrings.
One strategic #define iprintf iperf_printf in the right app header, 
included after all standard headers, before other app headers, may 
fix things up, or you may have to dink around with #undefs as well to 
avoid problems.

If your source defines no specific feature test macros, and you 
compile with no standard language version, or a gnu version, _GNU_SOURCE 
and many POSIX and glibc-compatible interfaces to newlib will be defined.

The same header may be included in the builds of the Windows cygwin 
dll, the newlib library for Cygwin and many embedded platforms, tools 
for many host platforms, and apps for many target platforms, so there 
are macros which provide the appropriate decorations depending on 
platform.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Fwd: Re: DMARC - gmail.com or is it yahoo.com

[Christopher Faylor]

On Wed, Nov 30, 2016 at 10:55:49AM -0500, Frank Ch. Eigler wrote:
>cygsimple wrote:
>
>> See the description below of a problem that is occurring because of
>> DMARC rules in place by yahoo.com.  Is there anything you can do for this?
>
>cgf, any chance of updating our copy of ezmlm-toaster etc.?
>Newer-than-2014 versions of ezmlm-idx seem to have some DMARC
>capabilities.

fche, I read overseers.  No need to clutter my personal inbox with this.

I started looking at this when it came in.  No ETA yet.

cgf

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Fwd: Re: DMARC - gmail.com or is it yahoo.com

[Frank Ch. Eigler]

Hi -

cygsimple wrote:

> See the description below of a problem that is occurring because of
> DMARC rules in place by yahoo.com.  Is there anything you can do for this?

cgf, any chance of updating our copy of ezmlm-toaster etc.?
Newer-than-2014 versions of ezmlm-idx seem to have some DMARC
capabilities.


- FChE

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Fwd: Re: DMARC - gmail.com or is it yahoo.com

[cyg Simple]

Overseers,

See the description below of a problem that is occurring because of
DMARC rules in place by yahoo.com.  Is there anything you can do for this?

Regards,
cygSimple


 Forwarded Message 
Subject: Re: DMARC - gmail.com or is it yahoo.com
Date: Tue, 29 Nov 2016 11:59:53 -0500
From: Erik Soderquist 
To: cygwin@cygwin.com

On Tue, Nov 29, 2016 at 11:19 AM, David Macek wrote:
> What happens is that ezmlm (the mailing list software) broadcasts this
> message (after adding a bunch of links at the bottom) to everyone
> subscribed, but with the sender unchanged. Yahoo has got a DMARC policy
> set up that is supposed to ensure that only Yahoo servers can send
> messages marked as "from ...@yahoo.com". As you can see, Google is
> enforcing the policy and correctly rejecting the message as fake (it
> really wasn't sent by Yahoo, but by Sourceware). This is only going to
> happen more and more often as email providers up the battle against
> spam.
>
> The best solution is to change the way ezmlm re-sends the messages, I
> bet there's a big discussion about it somewhere on the Internet.

Probably hundreds if not thousands of such discussions!

Who is in charge of the list server configuration?  Sourceware itself, or
someone from Cygwin?


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: chmod failing on user's permissions

[Corinna Vinschen]

On Nov 30 09:23, Ben Altman wrote:
> On 11/28/2016 7:00 PM, L. A. Walsh wrote:
> > Ben Altman wrote: When I get a directory listing, it shows for each
> > > file "Unknown+User Unknown+Group" while on the desktop the same files
> > > show my user name that I logged in with and "Domain Users" as the
> > > group.
> > ---
> > Is your laptop a member of the domain?  How do each of them
> > resolve User ID's?
> > 
> > 
> > > It also shows as unknown if I create a file on the laptop.
> > 
> > It doesn't sound like your laptop is looking up the ID's from the same
> > location as your desktop, and perhaps more likely,
> > it doesn't sound like your laptop is looking up ID's by checking
> > with the domain.
> > 
> Thanks for you response. I tracked down the issue to there not being any
> /etc/group file. I did a "mkgroup > /etc/group" and the issue went away. I
> am not sure why the file wasn't there.

Because you don't need it in usual circumstances.  You onle need
these files for offline work, if you're disconnected from your AD.

See https://cygwin.com/cygwin-ug-net/ntsec.html


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


signature.asc
Description: PGP signature

Re: Cygwin TCP slow

[Lee]

> If you also provided a Cygwin port of iperf3

Any hints on how to do that?

Trying to build using the i686-w64-mingw32-gcc cross-compiler dies
because nanosleep is missing.  How to figure out what needs to be
installed to get that function?

$ ./configure --prefix=/usr/local  --build=i686-pc-cygwin
--host=i686-w64-mingw32
  <.. snip ..>
checking for ANSI C header files... (cached) yes
checking for library containing floor... none required
checking for library containing nanosleep... no
nanosleep() required for timing operations.

/source/iperf-3.1.4
$



Trying to build using the cygwin gcc compiler does because of
"conflicting types for 'iprintf'"

aside: Why would one redefine a standard routine instead of making
their own routine that calls the standard routine??  The conflicting
bit is in iperf_api.h:
/* Custom printf routine. */
int iprintf(struct iperf_test *test, const char *format, ...)
__attribute__ ((format(printf,2,3)));

In any case, how does one reconcile
  int iprintf( ...
with
  int   _EXFUN(iprintf, ( ...


$ ./configure --prefix=/usr/local
$ make
Making all in src
make[1]: Entering directory '/source/iperf-3.1.4/src'
make  all-am
make[2]: Entering directory '/source/iperf-3.1.4/src'
/bin/sh ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I.
   -g -O2 -Wall -MT cjson.lo -MD -MP -MF .deps/cjson.Tpo -c -o
cjson.lo cjson.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -g -O2 -Wall -MT cjson.lo
-MD -MP -MF .deps/cjson.Tpo -c cjson.c  -DDLL_EXPORT -DPIC -o
.libs/cjson.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -g -O2 -Wall -MT cjson.lo
-MD -MP -MF .deps/cjson.Tpo -c cjson.c -o cjson.o >/dev/null 2>&1
mv -f .deps/cjson.Tpo .deps/cjson.Plo
/bin/sh ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I.
   -g -O2 -Wall -MT iperf_api.lo -MD -MP -MF .deps/iperf_api.Tpo -c -o
iperf_api.lo iperf_api.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -g -O2 -Wall -MT
iperf_api.lo -MD -MP -MF .deps/iperf_api.Tpo -c iperf_api.c
-DDLL_EXPORT -DPIC -o .libs/iperf_api.o
In file included from iperf_api.c:66:0:
iperf_api.h:262:5: error: conflicting types for 'iprintf'
 int iprintf(struct iperf_test *test, const char *format, ...)
__attribute__ ((format(printf,2,3)));
 ^
In file included from /usr/include/stdio.h:29:0,
 from iperf_api.c:32:
/usr/include/stdio.h:298:5: note: previous declaration of 'iprintf' was here
 int _EXFUN(iprintf, (const char *, ...)
 ^
iperf_api.c:3113:1: error: conflicting types for 'iprintf'
 iprintf(struct iperf_test *test, const char* format, ...)
 ^
In file included from /usr/include/stdio.h:29:0,
 from iperf_api.c:32:
/usr/include/stdio.h:298:5: note: previous declaration of 'iprintf' was here
 int _EXFUN(iprintf, (const char *, ...)
 ^
make[2]: *** [Makefile:856: iperf_api.lo] Error 1
make[2]: Leaving directory '/source/iperf-3.1.4/src'
make[1]: *** [Makefile:632: all] Error 2
make[1]: Leaving directory '/source/iperf-3.1.4/src'
make: *** [Makefile:381: all-recursive] Error 1

/source/iperf-3.1.4
$

TIA,
Lee

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: chmod failing on user's permissions

[Ben Altman]

On 11/28/2016 7:00 PM, L. A. Walsh wrote:

Ben Altman wrote: When I get a directory listing, it shows for each

file "Unknown+User Unknown+Group" while on the desktop the same files
show my user name that I logged in with and "Domain Users" as the
group.

---
Is your laptop a member of the domain?  How do each of them
resolve User ID's?



It also shows as unknown if I create a file on the laptop.


It doesn't sound like your laptop is looking up the ID's from the same 
location as your desktop, and perhaps more likely,

it doesn't sound like your laptop is looking up ID's by checking
with the domain.

Thanks for you response. I tracked down the issue to there not being any 
/etc/group file. I did a "mkgroup > /etc/group" and the issue went away. 
I am not sure why the file wasn't there.


Ben

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

remote GTK apps nor workin correctly in Cygwin/X

[David Cunningham]

I'm not sure if this is a cygwin problem, a gtk problem or something 
else (please be gentle).


I have 3 remote machines, one running servers, one running Ubuntu 16.04 
and the other 2 Debian stretch.


The Ubuntu machine has version 3.18.9 of libgtk-3 installed. The Debian 
machines version 3.22.4.


I run remote applications on these machines using x11-forwarding with 
ssh -Y (or putty).


I have no problem running applications on the Ubuntu machine from either 
the Debian machines or windows logged in from a cygwin xterm (or putty 
session).


I have no problem running applications on the Debian machines from 
either the Ubuntu machine or the other Debian machine.


Remote applications on the Debian machines from a Cygwin xterm (or 
putty) don't work correctly. An application (gnome-terminal and nemiver 
being the two mostly use) will open correctly and keyboard is (mostly) 
OK but mouse input doesn't work.


For example clicking a menu heading will place a line under the heading 
in the GUI but the menu doesn't open. Note that I get the same failure 
when using the keyboard shortcut keys (e.g ALT-F <> File). Sometimes a 
part of the menu (maybe one or 2 lines) will appear in the top left of 
the Desktop and can be scrolled through and selected using the keyboard. 
The window controls (minimise, maximise, exit) work correctly.


I get (lots of ) GTK error messages in terminal like this as I try to 
use the GUI:



(nemiver:6694): Gtk-WARNING **: Allocating size to gtkmm__GtkWindow 
0xe42260 without calling gtk_widget_get_preferred_width/height(). How 
does the code know the size to allocate?


(nemiver:6694): Gtk-CRITICAL **: 
gtk_widget_get_preferred_height_for_width: assertion 'width >= 0' failed


*** BUG ***
In pixman_region32_init_rect: Invalid rectangle passed
Set a breakpoint on '_pixman_log_error' to debug

(nemiver:6694): Gtk-WARNING **: Negative content width -7 (allocation 1, 
extents 4x4) while allocating gadget (node arrow, owner GtkMenu)



(nemiver:6694): Gtk-WARNING **: Negative content width -11 (allocation 
1, extents 6x6) while allocating gadget (node menuitem, owner 
GtkImageMenuItem)



I'v come across this bug filed against gtk in Debian which seems 
somewhat similar:


.

I'm unclear then whether this is a Debian/gtk issue or a Cywin/X issue 
(given that Ubuntu->Debian is OK).


Can anyone advise wher I go from here?

--
Thanks,
David


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Retrieving per-process environment block?

[Corinna Vinschen]

On Nov 30 12:35, Erik Bray wrote:
> On Wed, Nov 30, 2016 at 11:43 AM, Corinna Vinschen
>  wrote:
> > So how do we contact the other process to ask for information?
> >
> > We have a mechanism inside Cygwin to request info from another process.
> > It's part of the signal handling and consists basically of two
> > functions.  The applicant calls _pinfo::commune_request(), this will
> > send a request into the signal pipe of the target process, this in turn
> > will call commune_process(), a callback function, within the target
> > process.
> >
> > Have a look into an example:
> >
> > Start in fhandler_process.cc, function format_process_cmdline()
> > which implements /proc/$PID/cmdline.
> >
> > It fetches the _pinfo pointer of the target process and calls
> > _pinfo::cmdline.
> >
> > _pinfo::cmdline (in pinfo.cc) checks if the target process is a
> > native Windows process a Cygwin process, or if its itself.
> >
> > In the native Windows case, it opens a handle to the target process,
> > fetches its RTL_USER_PROCESS_PARAMETERS block (function
> > open_commune_proc_parms).  and fetches the info from the target process
> > by directly reading from its memory.  If it's itself it just constructs
> > the info as desired and returns.  Boring.
> >
> > In the Cygwin case, it calls _pinfo::commune_request (PICOM_CMDLINE).
> > PICOM_CMDLINE is defined in pinfo.h, together with other values for the
> > commune requests.  It send the request over the signal pipe to the
> > target process.
> >
> > The target process receives the request and calls commune_process().  It
> > checks what info is requested, prepares the info and send it back over
> > over the signal pipe.
> >
> > The (waiting) _pinfo::commune_request fetches the info from the pipe and
> > returns to _pinfo::cmdline, which in turn returns to 
> > format_process_cmdline().
> >
> > So, ultimately, just copy the functionality of format_process_cmdline,
> > _pinfo::cmdline, as well as the handling of PICOM_CMDLINE in
> > _pinfo::commune_request and commune_process and you're done.
> 
> Hi Corinna,
> 
> Thank you for the detailed guidance on that--that all makes sense.  I
> feel silly for missing that--I was actually poking around to see if
> there was already something like that in the source but missed it.
> For some reason I thought the cmdline was just stored directly in the
> pinfo object itself.

There's a catch.  The shared mem regions maintained by Cygwin are
supposed to fit in one allocation page, that is, 64K, at a reproducible
slot in the processes VM (think fork).  Variable size information,
especially information which can have an almost arbitrary size, is quite
tricky to be held in shared mem.

> But I knew, considering how signals are handled, that there had to be
> some communication channel between processes; I just didn't know how
> to use it for general purposes.  I just played around with it a bit
> and I get the gist of how __SIGCOMMUNE works, etc.  Very clever.

Curtesy cgf.  The commune mechanism obviously depends on the reliability
of the signal handling.  It also has a problem in that, since it's a part
of signal handling itself, it can't be interrupted by signals.  In earlier
stages it was pretty bad, but at one point we just changed the pipe calls
to timeout after 1 second each, so either it works or it times out after
a max of 2 secs and the information is just not available on the client
side.  There's worse than that.

> I can see what to do now--thanks.

Cool.


Thanks,
Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


signature.asc
Description: PGP signature

Re: Retrieving per-process environment block?

[Erik Bray]

On Wed, Nov 30, 2016 at 11:43 AM, Corinna Vinschen
 wrote:
> Hi Erik,
>
> On Nov 29 14:26, Erik Bray wrote:
>> On Thu, Nov 17, 2016 at 3:00 PM, Corinna Vinschen
>>  wrote:
>> > On Nov 17 14:30, Erik Bray wrote:
>> >> Hi all,
>> >>
>> >> For a quick bit of background, I'm working on porting the highly
>> >> useful psutil [1] Python library to Cygwin.  This has proved an
>> >> interesting exercise, as much of the functionality of psutil works on
>> >> Cygwin through existing POSIX interfaces, and a handful of
>> >> Linux-specific interfaces as well.  But there are some bits that
>> >> simply don't map at all.
>> >>
>> >> The one I'm struggling with right now is retrieving Cygwin environment
>> >> variables for a process (under inspection--i.e. not listing a
>> >> process's environment from within that process which is obviously
>> >> trivial).
>> >>
>> >> I've looked at every route I could conceive of but as far as I can
>> >> tell this is currently impossible.  That's fine for now--I simply
>> >> disable that functionality in psutil.  But it is unfortunate, though,
>> >> since the information is there.
>> >>
>> >> There are a couple avenues I could see to this.  The most "obvious"
>> >> (to me) being to implement /proc//environ.
>> >>
>> >> I would be willing to provide a patch for this if it would be
>> >> accepted.  Is there some particular non-obvious hurdle to this that it
>> >> hasn't been implemented?  Obviously there are security
>> >> implications--the /proc//environ should only be readable to the
>> >> process's owner, but that is already within Cygwin's capabilities, and
>> >> works for other /proc files.
>> >
>> > Patch welcome.  Implementing this should be fairly straightforward.
>> > The only hurdle is winsup/CONTRIBUTORS ;)
>>
>> Thanks--I went to go work on this finally but it turns out not to be
>> straightforward after all, as the process's environment is not shared
>> in any way between processes.
>>
>> I could do this, if each process kept a copy of its environment block
>> in shared memory, which would in turn have to be updated every time
>> the process's environment is updated.  But I don't know what the
>> impact of that would be performance-wise.
>
> You're right, it's not *that* straightforward.  I got carried away by
> the idea but didn't think this through when replying to you.
>
> So, how to implement this?
>
> We have two types of information in /proc/$PID, one is information
> readily available without having to contact the target process, the
> other information is local to the target process and we have to get the
> information with the target processes consent.  Argc, argv pointers
> are in the second group.
>
> So how do we contact the other process to ask for information?
>
> We have a mechanism inside Cygwin to request info from another process.
> It's part of the signal handling and consists basically of two
> functions.  The applicant calls _pinfo::commune_request(), this will
> send a request into the signal pipe of the target process, this in turn
> will call commune_process(), a callback function, within the target
> process.
>
> Have a look into an example:
>
> Start in fhandler_process.cc, function format_process_cmdline()
> which implements /proc/$PID/cmdline.
>
> It fetches the _pinfo pointer of the target process and calls
> _pinfo::cmdline.
>
> _pinfo::cmdline (in pinfo.cc) checks if the target process is a
> native Windows process a Cygwin process, or if its itself.
>
> In the native Windows case, it opens a handle to the target process,
> fetches its RTL_USER_PROCESS_PARAMETERS block (function
> open_commune_proc_parms).  and fetches the info from the target process
> by directly reading from its memory.  If it's itself it just constructs
> the info as desired and returns.  Boring.
>
> In the Cygwin case, it calls _pinfo::commune_request (PICOM_CMDLINE).
> PICOM_CMDLINE is defined in pinfo.h, together with other values for the
> commune requests.  It send the request over the signal pipe to the
> target process.
>
> The target process receives the request and calls commune_process().  It
> checks what info is requested, prepares the info and send it back over
> over the signal pipe.
>
> The (waiting) _pinfo::commune_request fetches the info from the pipe and
> returns to _pinfo::cmdline, which in turn returns to format_process_cmdline().
>
> So, ultimately, just copy the functionality of format_process_cmdline,
> _pinfo::cmdline, as well as the handling of PICOM_CMDLINE in
> _pinfo::commune_request and commune_process and you're done.

Hi Corinna,

Thank you for the detailed guidance on that--that all makes sense.  I
feel silly for missing that--I was actually poking around to see if
there was already something like that in the source but missed it.
For some reason I thought the cmdline was just stored directly in the
pinfo object itself.

But I knew, considering how signals are handled, that there had to 

Re: DMARC - gmail.com or is it yahoo.com

[Corinna Vinschen]

On Nov 29 13:25, Stephen John Smoogen wrote:
> On 29 November 2016 at 11:59, Erik Soderquist  
> wrote:
> > On Tue, Nov 29, 2016 at 11:19 AM, David Macek wrote:
> >> What happens is that ezmlm (the mailing list software) broadcasts this
> >> message (after adding a bunch of links at the bottom) to everyone
> >> subscribed, but with the sender unchanged. Yahoo has got a DMARC policy
> >> set up that is supposed to ensure that only Yahoo servers can send
> >> messages marked as "from ...@yahoo.com". As you can see, Google is
> >> enforcing the policy and correctly rejecting the message as fake (it
> >> really wasn't sent by Yahoo, but by Sourceware). This is only going to
> >> happen more and more often as email providers up the battle against
> >> spam.
> >>
> >> The best solution is to change the way ezmlm re-sends the messages, I
> >> bet there's a big discussion about it somewhere on the Internet.
> >
> > Probably hundreds if not thousands of such discussions!
> >
> > Who is in charge of the list server configuration?  Sourceware itself, or
> > someone from Cygwin?
> 
> Sourceware runs the lists

Indeed.  The problem is, I have no idea what this is about.  If you have
problems in terms of the sourceware mailer stuff, please feel free to
politely explain what the problem is and ask if that can be fixed on
either the overseers AT sourceware DOT org mailing list, or the Freenode
IRC channel #overseers.


Thanks,
Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


signature.asc
Description: PGP signature

Re: Retrieving per-process environment block?

[Corinna Vinschen]

On Nov 30 11:48, Corinna Vinschen wrote:
> On Nov 30 10:15, Peter Rosin wrote:
> > On 2016-11-30 04:00, Eliot Moss wrote:
> > > On 11/29/2016 7:28 PM, Herbert Stocker wrote:
> > > 
> > >> Or we simply copy the variables unconverted to the Windows environment
> > >> block, as the Windows env block is not used for anything else - up to
> > >> now it is not updated at all.
> > > 
> > > Maybe we could stash them inside one large variable?  Would that help
> > > at all?  (It does demand a quoting convention, but since this is all
> > > internal stuff, that may not be too hard.)
> > 
> > I believe this is a dead end. The Windows environment block has an
> > unacceptable size limit which is circumvented by the internal Cygwin
> > handling. Since this is about exposing said Cygwin environment, it
> > would simply not work for big environments.
> 
> ACK

Uhm, sorry, not ACK, this is nonsense.  The Windows env is not limited
to 32K characters, only the CreateProcess call is, unless you use the
option CREATE_UNICODE_ENVIRONMENT.  In that case, you can pass bigger
envs.  Cygwin uses it, too, when calling native Windows processes.


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


signature.asc
Description: PGP signature

Re: Retrieving per-process environment block?

[Corinna Vinschen]

On Nov 30 10:15, Peter Rosin wrote:
> On 2016-11-30 04:00, Eliot Moss wrote:
> > On 11/29/2016 7:28 PM, Herbert Stocker wrote:
> > 
> >> Or we simply copy the variables unconverted to the Windows environment
> >> block, as the Windows env block is not used for anything else - up to
> >> now it is not updated at all.
> > 
> > Maybe we could stash them inside one large variable?  Would that help
> > at all?  (It does demand a quoting convention, but since this is all
> > internal stuff, that may not be too hard.)
> 
> I believe this is a dead end. The Windows environment block has an
> unacceptable size limit which is circumvented by the internal Cygwin
> handling. Since this is about exposing said Cygwin environment, it
> would simply not work for big environments.

ACK


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


signature.asc
Description: PGP signature

Re: Retrieving per-process environment block?

[Corinna Vinschen]

On Nov 29 19:27, Andrey Repin wrote:
> Greetings, Erik Bray!
> 
> > Thanks for the reply.  The issue here isn't getting the Windows
> > process environment--that can be done, albeit trickily [1].  While
> > it's true there are security implications, in this case that is
> > handled at the OpenProcess call required to obtain a handle to the
> > process.  It's true you can't just read the environment from a process
> > you don't have permission to.
> 
> > The issue here though is reading the Cygwin process's environment
> > which is separate from its Windows environment, and the above
> > technique is inapplicable.
> 
> Update Windows environment block when Cygwin environment is updated.

Nope.


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


signature.asc
Description: PGP signature

Re: Retrieving per-process environment block?

[Corinna Vinschen]

On Nov 29 09:28, Eliot Moss wrote:
> On 11/29/2016 8:26 AM, Erik Bray wrote:
> > I could do this, if each process kept a copy of its environment block
> > in shared memory, which would in turn have to be updated every time
> > the process's environment is updated.  But I don't know what the
> > impact of that would be performance-wise.
> 
> Sorry, no advice getting around this, but a thought about why it may have
> been hard -- it can be a security hole.

Using the mechanism outlined in my mail from 2 minutes ago, there's
no security problem.  Opening the signal pipe is only allowed to
same user processes or administrative processes.  Other will get a
/proc/$PID/environ content of ""


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


signature.asc
Description: PGP signature

Re: Retrieving per-process environment block?

[Corinna Vinschen]

On Nov 29 09:11, cyg Simple wrote:
> On 11/29/2016 8:26 AM, Erik Bray wrote:
> > I could do this, if each process kept a copy of its environment block
> > in shared memory, which would in turn have to be updated every time
> > the process's environment is updated.  But I don't know what the
> > impact of that would be performance-wise.
> > 
> > Any advice?
> > 
> 
> Sounds like a job for a thread that wakes every X time units to check
> the contents of the environment.  Or is there a notification API that
> could be used to wake the thread?  I know there is a disk change
> notification API for this; maybe one for environment changes as well.

Cygwin env == POSIX env != Windows env.  Not only the content, but
also the internal memory handling is different.


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


signature.asc
Description: PGP signature

Re: Retrieving per-process environment block?

[Corinna Vinschen]

Hi Erik,

On Nov 29 14:26, Erik Bray wrote:
> On Thu, Nov 17, 2016 at 3:00 PM, Corinna Vinschen
>  wrote:
> > On Nov 17 14:30, Erik Bray wrote:
> >> Hi all,
> >>
> >> For a quick bit of background, I'm working on porting the highly
> >> useful psutil [1] Python library to Cygwin.  This has proved an
> >> interesting exercise, as much of the functionality of psutil works on
> >> Cygwin through existing POSIX interfaces, and a handful of
> >> Linux-specific interfaces as well.  But there are some bits that
> >> simply don't map at all.
> >>
> >> The one I'm struggling with right now is retrieving Cygwin environment
> >> variables for a process (under inspection--i.e. not listing a
> >> process's environment from within that process which is obviously
> >> trivial).
> >>
> >> I've looked at every route I could conceive of but as far as I can
> >> tell this is currently impossible.  That's fine for now--I simply
> >> disable that functionality in psutil.  But it is unfortunate, though,
> >> since the information is there.
> >>
> >> There are a couple avenues I could see to this.  The most "obvious"
> >> (to me) being to implement /proc//environ.
> >>
> >> I would be willing to provide a patch for this if it would be
> >> accepted.  Is there some particular non-obvious hurdle to this that it
> >> hasn't been implemented?  Obviously there are security
> >> implications--the /proc//environ should only be readable to the
> >> process's owner, but that is already within Cygwin's capabilities, and
> >> works for other /proc files.
> >
> > Patch welcome.  Implementing this should be fairly straightforward.
> > The only hurdle is winsup/CONTRIBUTORS ;)
> 
> Thanks--I went to go work on this finally but it turns out not to be
> straightforward after all, as the process's environment is not shared
> in any way between processes.
> 
> I could do this, if each process kept a copy of its environment block
> in shared memory, which would in turn have to be updated every time
> the process's environment is updated.  But I don't know what the
> impact of that would be performance-wise.

You're right, it's not *that* straightforward.  I got carried away by
the idea but didn't think this through when replying to you.

So, how to implement this?

We have two types of information in /proc/$PID, one is information
readily available without having to contact the target process, the
other information is local to the target process and we have to get the
information with the target processes consent.  Argc, argv pointers
are in the second group.

So how do we contact the other process to ask for information?

We have a mechanism inside Cygwin to request info from another process.
It's part of the signal handling and consists basically of two
functions.  The applicant calls _pinfo::commune_request(), this will
send a request into the signal pipe of the target process, this in turn
will call commune_process(), a callback function, within the target
process.

Have a look into an example:

Start in fhandler_process.cc, function format_process_cmdline()
which implements /proc/$PID/cmdline.

It fetches the _pinfo pointer of the target process and calls
_pinfo::cmdline.

_pinfo::cmdline (in pinfo.cc) checks if the target process is a
native Windows process a Cygwin process, or if its itself.

In the native Windows case, it opens a handle to the target process,
fetches its RTL_USER_PROCESS_PARAMETERS block (function
open_commune_proc_parms).  and fetches the info from the target process
by directly reading from its memory.  If it's itself it just constructs
the info as desired and returns.  Boring.

In the Cygwin case, it calls _pinfo::commune_request (PICOM_CMDLINE).
PICOM_CMDLINE is defined in pinfo.h, together with other values for the
commune requests.  It send the request over the signal pipe to the
target process.

The target process receives the request and calls commune_process().  It
checks what info is requested, prepares the info and send it back over
over the signal pipe.

The (waiting) _pinfo::commune_request fetches the info from the pipe and
returns to _pinfo::cmdline, which in turn returns to format_process_cmdline().

So, ultimately, just copy the functionality of format_process_cmdline,
_pinfo::cmdline, as well as the handling of PICOM_CMDLINE in
_pinfo::commune_request and commune_process and you're done.


Does that help?


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


signature.asc
Description: PGP signature

Re: Retrieving per-process environment block?

[Erik Bray]

On Wed, Nov 30, 2016 at 10:15 AM, Peter Rosin  wrote:
> On 2016-11-30 04:00, Eliot Moss wrote:
>> On 11/29/2016 7:28 PM, Herbert Stocker wrote:
>>
>>> Or we simply copy the variables unconverted to the Windows environment
>>> block, as the Windows env block is not used for anything else - up to
>>> now it is not updated at all.
>>
>> Maybe we could stash them inside one large variable?  Would that help
>> at all?  (It does demand a quoting convention, but since this is all
>> internal stuff, that may not be too hard.)
>
> I believe this is a dead end. The Windows environment block has an
> unacceptable size limit which is circumvented by the internal Cygwin
> handling. Since this is about exposing said Cygwin environment, it
> would simply not work for big environments.

I agree--although I can't off the top of my head recall where I've
read this, I believe there's a deliberate effort in Cygwin to keep the
Windows environment to the bare minimum for Cygwin processes.  Among
other things I imagine it keeps the number of conversions necessary to
a minimum.

And in any case, trying to cache the entire Cygwin environment back
into the Windows environment would be even more overhead than what I'm
proposing which is to store it in shared memory.  My goal is not to
make the Cygwin environment of a process readable from outside
Cygwin--only make it so it can be read from other Cygwin processes by
way of /proc (given the necessary permissions).

Perhaps I'll go ahead and cook up a patch and see what impact it has.
With luck it won't be much--I just don't know how much overhead there
is in open_shared.

Thanks,
Erik

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Retrieving per-process environment block?

[Peter Rosin]

On 2016-11-30 04:00, Eliot Moss wrote:
> On 11/29/2016 7:28 PM, Herbert Stocker wrote:
> 
>> Or we simply copy the variables unconverted to the Windows environment
>> block, as the Windows env block is not used for anything else - up to
>> now it is not updated at all.
> 
> Maybe we could stash them inside one large variable?  Would that help
> at all?  (It does demand a quoting convention, but since this is all
> internal stuff, that may not be too hard.)

I believe this is a dead end. The Windows environment block has an
unacceptable size limit which is circumvented by the internal Cygwin
handling. Since this is about exposing said Cygwin environment, it
would simply not work for big environments.

Cheers,
Peter

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple