Re: Clang is using the wrong memory model
Thanks a lot for your help in clarifying this.
When I complained here about the wasteful 64-bit addresses you said that
it was an LLVM issue. When I complained to LLVM they said it was a
Cygwin issue, and that you were using the wrong memory model.
All this confusion is due to a terrible lack of documentation of everything.
I had to do a lot of reverse engineering to figure out what is
happening. What I have found out so far is listed below. Much of this is
undocumented. Obviously, I would like to know if any or this is wrong or
if specific documentation is available other than the SysV ABI and
Windows ABI:
* Cygwin is using its own loader which is different from the Windows
loader.
* The Cygwin loader emulates the behavior of Linux shared objects. This
includes the ability to directly access a variable inside a DLL
* Access to a variable in a different DLL requires a 64-bit address.
This is obtained by using the medium memory model with a gcc or Clang
compiler.
* The small memory model works differently on different targets. A
-mcmodel=small with a Linux target puts everything below 2GB addresses.
32-bit absolute addresses are allowed. -mcmodel=small with a Windows or
Mac target allows addresses above 2GB, but limits the distance between
code and data in the same executable to 2GB. 32-bit absolute addresses
are not allowed. 32-bit relative addresses are used instead.
* The memory models work differently in gcc an Clang. Gcc with a medium
or large memory model is using 64-bit address tables to access a
variable in a different C/CPP file. Clang with a medium or large memory
model is using 64-bit addresses not only for external variables, but
also for local static data. This includes floating point constants,
string constants, array initializers, jump tables, global variables, and
more.
* Cygwin uses a medium memory model by default. The medium memory model
is necessary only for a program that makes direct access to a variable
in a different DLL. The medium memory model is wasteful, and more so
with Clang than with gcc.
Now I am speculating what we can do to avoid the wasteful 64-bit
address-load instructions to improve the performance of Cygwin programs.
We can improve performance by using the small memory model when
possible. The medium memory model is needed only for programs that link
to a variable in a different DLL. The DLL that contains the link target
does not need the medium memory model.
Direct access to a variable in a different DLL is considered bad
programming practice by modern standards. This should occur only in old
Linux code.
A link to a variable in a different DLL may be replaced by function
calls (this is done with errno). In some cases, static linking can be an
efficient alternative.
It would be helpful if the Cygwin loader could print the name of the
offending variable when relocation fails with the small memory model.
This could help programmers remove any obstacles to using the more
efficient small memory model.
Agner
On 17/08/2019 10.16, Corinna Vinschen wrote:
Oe Aug 17 07:31, Agner Fog wrote:
So errno was a bad example but you can try accessing e.g. __ctype_ptr__,
__progname, optarg, h_errno, or use FE_DFL_ENV from another DLL, just
for kicks.
__ctype_ptr__ is a function
h_errno works like errno with an imported function
FE_DFL_ENV is a macro
__progname and optarg are local variables to each exe or dll
That would contradict what, e.g., __progname is for. Here's a test:
$ cat > dll.c <
extern char *__progname;
void
printprog ()
{
printf ("progname: %s\n", __progname);
}
EOF
$ cat > main.c <
gcc is using the small memory model by default in Cygwin64, and it works.
No, it's not, see above.
clang is using the small memory by default when cross-compiling for a Cygwin64
target from Linux, and it works.
...in *your* example code.
Corinna
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: Win7 system update hosed something in Cygwin
On 2019/08/16 18:44, David Karr wrote: > > > I would most like to understand what changed to make this suddenly > start happening. You really need to read the cygwin section on nt-security. in the user manual @ https://www.cygwin.com/cygwin-ug-net/ntsec.html. If you are into things appearing a certain way -- I think you will really find this invaluable. It also gives background and why things changed. As for how or why a windows update changed a behavior in cygwin -- MS doesn't released documentation in that detail. I came up with my mappings on my linux/samba box because I have a pretty simple setup. But I wanted the common well-known ID's in there so they would also make some sense: >From the very generic sids: Null Authority:x:10010:S-1-0,builtin: Nobody:x:10100:S-1-0-0,builtin: World Authority:x:10101:S-1-1,builtin: Everyone:x:11100:S-1-1-0,builtin: Local Authority:x:10102:S-1-2,builtin: ... to my domain id's: Domain Administrator:x::S-1-5-21-3-7-3-500,domain: Domain Admins:x::S-1-5-21-3-7-3-512,domain: Domain Users:x::S-1-5-21-3-7-3-513,domain: ... to local roles: Administrators:x::S-1-5-32-544,builtin: Users:x::S-1-5-32-545,builtin: Backup Operators:x::S-1-5-32-551,builtin: ... and authentication + trust labels: NTLM Authentication:x::S-1-5-64-10,builtin: Schannel Authentication:x::S-1-5-64-10,builtin: NT Service:x::S-1-5-80,builtin Untrusted Mandatory Level:x::S-1-16-0,builtin: Low Mandatory Level:x:11604096:S-1-16-4096,builtin: High Mandatory Level:x:11612288:S-1-16-12288,builtin: System Mandatory Level:x:11616384:S-1-16-16384,builtin: So in cygwin, when I display my 'id' output, I see the various groups and labels on my userid: uid=5013(Bliss\law) gid=201(Bliss\lawgroup) groups=201(Bliss\lawgroup), 1015(lawgroup), 1018(Netmon Users), 1017(pulse-access), 1016(pulse-rt), 1023(WinRMRemoteWMIUsers__), 544(Administrators), 555(Remote Desktop Users), 559(Performance Log Users), 545(Users), 11504(Interactive), 11201(Console Login), 11511(Authenticated Users), 4095(CurrentSession), 66048(LOCAL), 260(Bliss\Media), 512(Bliss\Domain Admins), 513(Bliss\Domain Users), 1053(Bliss\Trusted Local Net Users), 1156410(NTLM Athentication), 11612288(High Mandatory Level) If you look at your security blob using something like 'processhacker' (goog), you can see all those labels on your login. Setup right, you also see them in your cygwin shell, which I thought was cool (being a computer scientist). Sorry if this was too much, I can't say my setup is necessarily recommended, BUT, I wanted my Win+linux machines to be joined as 1 machine (all my docs & content is on the linux box, with progs on my Win Desktop box) joined by a 10gbit ethernet cable: > I've been running this Cygwin installation for a long time on this > laptop without any problem like this. Yeah, you get used to a well tuned setup (though mine does often have problems because of the weird things I've tried). -linda -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: More on Win7 update messing up HOME
On Sat, Aug 17, 2019 at 12:12 AM Achim Gratz wrote: > David Karr writes: > > I noticed that when I do an "ls -lt" in my homedir, instead of the owner > > and group of files being my simple uid, it's something like > > "+User(1944941)" and "2993517" (group and user, respectively, I > > believe). > > There's your problem. For whatever reason, the DC did not resolve your > user ID when it was asked by Cygwin and so all the other information > associated with your account is likewise unavailable. Are you starting > any Cygwin services at boot? If so, make them depend on tcp_ip so they > don't start before networking is available and/or set them to delayed > start. If that doesn't fix it, check the user they are running under, > both cygserver and sshd should run under LocalSystem for an up-to-date > Cygwin. > About a month ago I worked on getting crontab working. I hadn't done it before, and I found there were lots of details that were hard to get right. I'd have to review it again to remember how that is set up. In the older setup, without the physical passwd or group files, what would happen if I rebooted at a point when the laptop wouldn't have been able to reach the domain server? > > Regards, > Achim. > -- > +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ > > Wavetables for the Terratec KOMPLEXER: > http://Synth.Stromeko.net/Downloads.html#KomplexerWaves > > -- > Problem reports: http://cygwin.com/problems.html > FAQ: http://cygwin.com/faq/ > Documentation: http://cygwin.com/docs.html > Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple > > -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: Perl Unicode-Normalize
On 8/17/2019 12:59 PM, Achim Gratz wrote: > Ken Brown writes: >> The upcoming release of Biber requires Unicode-Normalize-1.26, but we >> currently >> have 1.25. I know this module is now bundled with perl. Would it be >> difficult >> for you to update it? > > I'd have to start provifding it as an extra package again of course, but > otherwise it should be OK. How imminent is this update? Not sure. I just received a note this morning from the author saying that biber-2.13 will be released "soon", and asking me to build the Cygwin binaries for the upstream biber release (which provides packed perl archives for various platforms). If you can get to it within the next few days, that would be great. If not, I can just install it from CPAN temporarily for the purpose of building the packed archive. Ken
Re: Perl Unicode-Normalize
Ken Brown writes: > The upcoming release of Biber requires Unicode-Normalize-1.26, but we > currently > have 1.25. I know this module is now bundled with perl. Would it be > difficult > for you to update it? I'd have to start provifding it as an extra package again of course, but otherwise it should be OK. How imminent is this update? Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Wavetables for the Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] TEST: Cygwin 3.1.0-0.1
Hi Corinna, On Fri, 16 Aug 2019 16:48:11 +0200 Corinna Vinschen wrote: > I now had an idea, but I'm not entirely sure if it's the right thing to > do. Can you please test this? It consists of two patches, one with the > revamped signalfd handling, and one with the revert of the signalfd > patch I applied a couple of days ago. > > Quick description: I dropped signalfd_select_wait entirely. Instead, > wait_sig sets or resets a manual event object to indicate if there are > signals pending in the queue, even after trying to handle them the > normal way. That usually means they are blocked. > > select() uses the event to wake up from WFMO, if at least one signalfd > is present in the read descriptor set. The rest is done via the peek > and verify functions in select, which basically just check if this > signalfd is waiting for one of the pending signals. > > The reversion of my patch from a couple days ago is not required as > such, but after thinking about this a while I'm convinced that this was > just me not getting the full picture. Also, reverting this patch would > revert to seeing a SEGV in your testcase and thus a bug in the new code, > too. > > I attached both patches. It would be pretty nice if you could test them > and point out any problems you get with this new code. > > Please note that you should ideally perform a full rebuild due to the > slight change in TLS layout. I confirmed that my STC and script command works as expected with these patches. Thank you for greate work! -- Takashi Yano -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Perl Unicode-Normalize
Achim, The upcoming release of Biber requires Unicode-Normalize-1.26, but we currently have 1.25. I know this module is now bundled with perl. Would it be difficult for you to update it? Thanks. Ken
[ANNOUNCEMENT] Updated: postgresql-11.5-1
Version 11.5-1 of packages libecpg-compat3 libecpg-devel libecpg6 libpgtypes3 libpq-devel libpq5 postgresql postgresql-client postgresql-contrib postgresql-devel postgresql-doc postgresql-plperl postgresql-plpython are available in the Cygwin distribution: CHANGES This is the latest upstream version, upstream info on https://www.postgresql.org/about/news/1960 Migration to Version 11.x http://www.postgresql.org/docs/11/static/release-11.html ADVISE for major version UPGRADE http://www.postgresql.org/support/versioning/ Major releases usually change the internal format of system tables and data files. These changes are often complex, so we do not maintain backward compatibility of all stored data. A dump/reload of the database or use of the pg_upgrade module is required for major upgrades. DESCRIPTION PostgreSQL is a powerful, open source object-relational database system. It has a proven architecture that has earned it a strong reputation for reliability, data integrity, and correctness. It is fully ACID compliant, has full support for foreign keys, joins, views, triggers, and stored procedures (in multiple languages). It includes most SQL:2008 data types HOMEPAGE http://www.postgresql.org Marco Atzeri If you have questions or comments, please send them to the cygwin mailing list at: cygwin (at) cygwin (dot) com . -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
[ANNOUNCEMENT] Updated: openblas-0.3.7-1
New releases 0.3.7-1 of openblas (source) libopenblas (dinamic library) are available in the Cygwin distribution : CHANGES Last upstream release bugfix and performance improvment https://github.com/xianyi/OpenBLAS/releases OpenBLAS will now provide enough buffer space for at least 50 threads by default. DESCRIPTION OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version and further extended. OpenBLAS is licensed under the 3-clause BSD license. HOMEPAGE http://www.openblas.net/ CYGWIN NOTES 1) As BLAS it is around 5-10 times faster than Netlib reference, included in the liblapack0 package. 2) No devel package is provided as liblapack-devel already provide the needed headers and import. Openblas is fully compatible with Netlib BLAS. 3) libopenblas consist of a single file /usr/bin/cygblas-0.dll that will precede in PATH the liblapack0 /usr/lib/lapack/cygblas-0.dll and used instead. Removing libopenblas will restore the usage of Netlib BLAS 4) CPU architecture covered up to now 64 bit ATOM BARCELONA BOBCAT BULLDOZER CORE2 DUNNINGTON EXCAVATOR HASWELL NANO NEHALEM OPTERON OPTERON_SSE3 PENRYN PILEDRIVER PRESCOTT SANDYBRIDGE STEAMROLLER ZEN 32 bit ATHLON ATOM BANIAS BARCELONA BOBCAT COPPERMINE CORE2 DUNNINGTON KATMAI NANO NEHALEM NORTHWOOD OPTERON OPTERON_SSE3 PENRYN PRESCOTT Regards Marco Atzeri If you have questions or comments, please send them to the cygwin mailing list at: cygwin (at) cygwin (dot) com . -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: Clang is using the wrong memory model
Oe Aug 17 07:31, Agner Fog wrote:
> > So errno was a bad example but you can try accessing e.g. __ctype_ptr__,
> > __progname, optarg, h_errno, or use FE_DFL_ENV from another DLL, just
> > for kicks.
> __ctype_ptr__ is a function
>
> h_errno works like errno with an imported function
>
> FE_DFL_ENV is a macro
>
> __progname and optarg are local variables to each exe or dll
That would contradict what, e.g., __progname is for. Here's a test:
$ cat > dll.c <
extern char *__progname;
void
printprog ()
{
printf ("progname: %s\n", __progname);
}
EOF
$ cat > main.c < gcc is using the small memory model by default in Cygwin64, and it works.
No, it's not, see above.
> clang is using the small memory by default when cross-compiling for a
> Cygwin64 target from Linux, and it works.
...in *your* example code.
Corinna
--
Corinna Vinschen
Cygwin Maintainer
signature.asc
Description: PGP signature
Re: More on Win7 update messing up HOME
David Karr writes: > I noticed that when I do an "ls -lt" in my homedir, instead of the owner > and group of files being my simple uid, it's something like > "+User(1944941)" and "2993517" (group and user, respectively, I > believe). There's your problem. For whatever reason, the DC did not resolve your user ID when it was asked by Cygwin and so all the other information associated with your account is likewise unavailable. Are you starting any Cygwin services at boot? If so, make them depend on tcp_ip so they don't start before networking is available and/or set them to delayed start. If that doesn't fix it, check the user they are running under, both cygserver and sshd should run under LocalSystem for an up-to-date Cygwin. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Wavetables for the Terratec KOMPLEXER: http://Synth.Stromeko.net/Downloads.html#KomplexerWaves -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
