Re: Error accessing mapped drive >2TB?

[Corinna Vinschen]

On Nov  3 15:18, Warren Young wrote:
> On Oct 26, 2015, at 5:15 AM, Corinna Vinschen wrote:
> > 
> > On Oct 23 16:15, Warren Young wrote:
> >> On Oct 23, 2015, at 8:30 AM, Jeffrey Altman wrote:
> >>> While Apple's design choices do not fit with the expectations of Cygwin
> >>> they are not necessarily wrong.
> >> 
> >> So, should I send Apple this code, or not?
> >> 
> >>  http://pastebin.com/uZdDZPgi
> > 
> > Wouldn't hurt I guess.  They are free to ignore us, of course :}
> 
> I finally got around to writing this up.  It’s radar ID 23381990 at
> https://bugreport.apple.com/
> 
> You might want to read it over.  I’m not certain I’m describing the
> problem clearly or completely.

"An Apple ID is required to use this tool."


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


pgpK0FGDBwgCJ.pgp
Description: PGP signature

Re: Error accessing mapped drive >2TB?

[Warren Young]

On Oct 26, 2015, at 5:15 AM, Corinna Vinschen wrote:
> 
> On Oct 23 16:15, Warren Young wrote:
>> On Oct 23, 2015, at 8:30 AM, Jeffrey Altman wrote:
>>> While Apple's design choices do not fit with the expectations of Cygwin
>>> they are not necessarily wrong.
>> 
>> So, should I send Apple this code, or not?
>> 
>>  http://pastebin.com/uZdDZPgi
> 
> Wouldn't hurt I guess.  They are free to ignore us, of course :}

I finally got around to writing this up.  It’s radar ID 23381990 at 
https://bugreport.apple.com/

You might want to read it over.  I’m not certain I’m describing the problem 
clearly or completely.
--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Error accessing mapped drive >2TB?

[Corinna Vinschen]

Hi Jeffrey,

On Oct 23 10:30, Jeffrey Altman wrote:
> Lets discuss what is going on in this particular case.   Apple
> constructs their file namespace as the machine name representing a
> virtual directory containing mount points to partitions and end user
> folders.
> [...]
> Apple's SMB file namespace (as is also true for the /afs file namespace)
> permits the crossing of mount points without the use of a DFS referral
> to a share representing the root directory of the target.  As a result
> the Apple SMB server does expose the existence of the mount point via
> the use of the RP file attribute.
> [...]
> ]getfileattrsex.exe  r:\
> Attribute: 0x410
> Directory
> Reparse Point
> Size:  0x0:0
> 
> Note that the size of the reparse data is zero.  There is no reparse
> data to read.  This is a UNIX mount point not an NTFS junction.
> 
> ]getfileinfobyhandle.exe r:\
> Attribute: 0x410
> Directory
> Reparse Point
> ReparseTag: 0xa003 Microsoft Surrogate
> Size:  0x0:0
> EOF:   0x0:0
> Links: 1
> Delete?no
> Directory? yes
> 
> Apple should have registered with Microsoft their own reparse point tag.
>  Instead they broke the rules and used Microsoft's
> IO_REPARSE_TAG_MOUNT_POINT which can lead to confusion except for the
> fact that the SMB server is clearly indicating that there is no reparse
> data to read by setting the size to 0.  The error status
> STATUS_NOT_A_REPARSE_POINT (0xC275L) is what is generated when there
> is no reparse data to read.

Are you saying that the file size is supposed to be non-0 if reparse
data is present?  The problem is, I can't reproduce this.  When calling
NtQueryInformationFile(FileNetworkOpenInformation) on the reparse point,
the size returned in the EndOfFile member is 0, even if it's a standard
directory junction created by mklink.  There's no visible difference
between real and faked mount point in the results of this call.  I don't
see how Cygwin could avoid checking the reparse data altogether in this
scenario :(

> Remember that there is no requirement that SMB servers expose reparse
> points at all nor is there any requirement that they expose volume
> information.
> 
> While Apple's design choices do not fit with the expectations of Cygwin
> they are not necessarily wrong.

Well, I'm not sure.  Either they don't expose the reparse point, or they
should expose them in a way which matches the expectations.  Cygwin is
only evaluating the default Microsoft mount points and symlinks anyway,
and when using those tags, they should be used as the original provider
does, me thinks.

> I hope this e-mail is helpful.

It is, thank you!


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


pgp1P9p08nm1B.pgp
Description: PGP signature

Re: Error accessing mapped drive >2TB?

[Corinna Vinschen]

On Oct 23 16:15, Warren Young wrote:
> On Oct 23, 2015, at 8:30 AM, Jeffrey Altman  
> wrote:
> > While Apple's design choices do not fit with the expectations of Cygwin
> > they are not necessarily wrong.
> 
> So, should I send Apple this code, or not?
> 
>   http://pastebin.com/uZdDZPgi

Wouldn't hurt I guess.  They are free to ignore us, of course :}


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


pgpyxWzKziGEc.pgp
Description: PGP signature

Re: Error accessing mapped drive >2TB?

[Corinna Vinschen]

On Oct 22 12:34, Warren Young wrote:
> On Oct 22, 2015, at 2:34 AM, Corinna Vinschen wrote:
> > 
> > On Oct 21 11:26, Warren Young wrote:
> >> On Oct 21, 2015, at 10:22 AM, Corinna Vinschen wrote:
> >>> 
> >>> On Oct 21 09:52, Warren Young wrote:
>  
>  I mean, I know how to snag a stream of SMB packets with Wireshark, but
>  I don’t know what I’d be looking for in the dump.
> >>> 
> >>> Me neither, the Samba guys might be able to help there, perhaps.
> >> 
> >> Apple hasn’t shipped Samba as part of OS X since 10.6, quite a few
> >> years ago now.  In 10.7, they switched to an internally-developed SMB
> >> server.
> > 
> > Yes, but the SMB guys can recite the wire format of SMB asleep, probably.
> 
> Why would the Samba project’s wizards be motivated to help debug a
> closed-source piece of software they didn’t write?

No idea, it was just a suggestion.

> If anything, I’d expect them to give Apple the finger for booting
> Samba out of OS X in the first place.
> 
> > There's also
> > https://msdn.microsoft.com/en-us/library/windows/desktop/aa365233%28v=vs.85%29.aspx
> 
> Now we’re talking about *my* motivation, which has only been to test
> the original report and the fix.

Well, it was *you* asking "How could we prove that the problem is the
Apple SMB server?"  I was just trying to help.  If that's not desired,
I don't have to.

> >>> HANDLE handle = CreateFile ("P:\\", ...);
> >> 
> >> I guess I’m not seeing what values to pass to CreateFile()
> > 
> > Opening a directory requires to use the FILE_FLAG_BACKUP_SEMANTICS
> > flag.
> 
> Yes, silly me for not guessing that Windows requires that I tell it I
> am about to do a backup before I attempt to open a directory for
> reading.  What was so wrong about the design of opendir() that MS had
> to reinvent it this way?

Think DOS/early Windows.  CreateFile was not meant to open directories
to perform a directory search and you couldn't do that on Windows 9x
anyway.  On NT the functions to do that were hidden in the NT API
originally.  Directory searches were meant to be performed using
FindFirstFile, etc., just as on 9x.

> I also had to fix an error in your original code: GetFileAttributes()
> takes a path string, not a HANDLE.

Oh, sorry, that should have been GetFileInformationByHandle.  It does
not help to call GetFileAttributes with a path since that obviously
won't use the previously opened handle.

> > Oh and, you have to use the FILE_FLAG_OPEN_REPARSE_POINT flag, of
> > course.
> 
> Adding that doesn’t affect the error I get from the program, so I left
> it out in the Pastebin version.

Without this flag the testcase won't do what Cygwin does.


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


pgpVqxRLXzxRm.pgp
Description: PGP signature

Re: Error accessing mapped drive >2TB?

[Jeffrey Altman]

In this thread there appears to be a small amount of misunderstanding of
what a reparse point is and how it should be used.

A reparse point can be thought of as a special form of extended
attribute on a file system object (directory or file) that can stored a
single {tag-value, opaque-data} pair.  In general, applications are not
expected to understand the meaning of the opaque-data nor should they
need to know what all of the registered tag-values are.

The reparse data is meant for the file system to interpret.  Some of the
opaque data structures necessary to interpret tag-values registered for
NTFS are included in the Windows DDK ntifs.h header.   For example, the
structures for the IO_REPARSE_TAG_SYMLINK (0xA00CL) tag and
IO_REPARSE_TAG_MOUNT_POINT (0xA003L) tag are included in the struct
_REPARSE_DATA_BUFFER.  But there are nearly a dozen other Microsoft
registered tag values and dozens on non-Microsoft registered tag values
for which the data structures are not published.

The high bits of the tag-value convey information.  There is a Microsoft
bit (0x8000L) and a Surrogate bit (0x2000).  If the Surrogate
bit is set in the tag value then the directory entry is a placeholder
for another object.  Examples, a junction has the surrogate bit set to
indicate that the target of the junction is the root directory of
another file system.   A symlink has the surrogate bit set to indicate
that the target is a different file or directory that might or might not
be in a different file system.

In this thread a statement was made that the Explorer Shell does not pay
attention to reparse points.  Actually, the Explorer Shell maintains a
cache of all objects and in particular where there reparse points are.
Each surrogate reparse point is a potential bridge to a new file system.
 That means that volume information queries must be evaluated on the
target side of the reparse point.   There was a long standing bug in
this caching that has only been fixed in Windows 10.

The CreateFile() API when applied to a reparse point will by default
allocate a handle to the reparse point target.  In order to obtain a
handle to the object on which the reparse point attribute is set the
FILE_FLAG_OPEN_REPARSE_POINT flag must be set.

The GetFileAttributes() and GetFileAttributesEx() API calls set the
FILE_FLAG_OPEN_REPARSE_POINT flag which is why GetFileAttributesEx()
returns the size of the reparse point data and not the size of the
target object.  This is a point of confusion especially when working
with NTFS Symlinks because .NET and Java are unaware of Symlinks and use
the wrong size when parse the target data stream.

Lets discuss what is going on in this particular case.   Apple
constructs their file namespace as the machine name representing a
virtual directory containing mount points to partitions and end user
folders.

Apple's SMB security requires that an authenticated connection be
established before viewing the available shares.   I have created a
drive letter mapping of R: to one of the partitions.

For example:

]net view \\172.16.16.xx
Shared resources at \\172.16.16.xx


Share nameType  Used as  Comment

---
BOOTCAMP  Disk
jaltman   Disk
Jeffrey Altman's Public FolderDisk
Macintosh HD  Disk  R:
The command completed successfully.

Apple's SMB file namespace (as is also true for the /afs file namespace)
permits the crossing of mount points without the use of a DFS referral
to a share representing the root directory of the target.  As a result
the Apple SMB server does expose the existence of the mount point via
the use of the RP file attribute.

]getfileattrs.exe r:\
Attribute: 0x410
Directory
Reparse Point

]getfileattrsex.exe  r:\
Attribute: 0x410
Directory
Reparse Point
Size:  0x0:0

Note that the size of the reparse data is zero.  There is no reparse
data to read.  This is a UNIX mount point not an NTFS junction.

]getfileinfobyhandle.exe r:\
Attribute: 0x410
Directory
Reparse Point
ReparseTag: 0xa003 Microsoft Surrogate
Size:  0x0:0
EOF:   0x0:0
Links: 1
Delete?no
Directory? yes

Apple should have registered with Microsoft their own reparse point tag.
 Instead they broke the rules and used Microsoft's
IO_REPARSE_TAG_MOUNT_POINT which can lead to confusion except for the
fact that the SMB server is clearly indicating that there is no reparse
data to read by setting the size to 0.  The error status
STATUS_NOT_A_REPARSE_POINT (0xC275L) is what is generated when there
is no reparse data to read.

]getvolumeinfobyhandle.exe r:\
File System: NTFS
Volume Name: Macintosh HD
Serial: 0
Max Component Length: 255
System Flags: 0x40086
Case Preservation
Named Streams
Supports Reparse Points
Unicode on Disk

Apple also broke the rules 

Re: Error accessing mapped drive >2TB?

[Andrey Repin]

Greetings, Warren Young!

>> Apple should have registered with Microsoft their own reparse point tag.
>> Instead they broke the rules and used Microsoft's
>> IO_REPARSE_TAG_MOUNT_POINT

> If Apple uses their own tags, wouldn’t that cause the Windows SMB client to
> be unable to understand Unix mount points, when if it comes across them?

My understanding is that the data stored in reparse point is not intended for
end user (in this case, client system's Explorer) consumption, but solely
exists for the benefits of the host file system management driver.

> I don’t see that the Apple SMB server really needs to report Unix mount
> points at the root of a share, but they could also appear in the middle of a
> share, at which point I assume there are important implications to SMB,
> equivalent to the inode uniqueness problem on Unix.

> Therefore, I can see that Apple’s SMB server needs a way to tell the client
> that it is crossing a filesystem boundary.
> The question is, is the way Apple chose  a sensible one?

The very presence of the reparse point attribute is all that client system
needs to know. The data stored in it is (supposedly) of little utility outside
the host filesystem.


-- 
With best regards,
Andrey Repin
Saturday, October 24, 2015 03:32:00

Sorry for my terrible english...

Re: Error accessing mapped drive >2TB?

[Warren Young]

On Oct 23, 2015, at 4:04 PM, Warren Young wrote:
> 
> I’ve made the suggested changes to the program, here:
> 
>  http://pastebin.com/uZdDZPgi

By the way, if you look at scream_and_die() and wonder why I’ve badly 
overcomplicated it, it’s because a previous version presented a printf-like 
interface to its callers.  In stages, the callers stopped using it that way, 
and the function itself evolved to where it couldn’t do printf-like things 
anyway.

This simpler replacement suffices now:

void scream_and_die(const char* complaint)
{
LPTSTR syserr = 0;
FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM |
FORMAT_MESSAGE_ALLOCATE_BUFFER, 0, GetLastError(),
MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT),
(LPTSTR), 0, 0);
fprintf(stderr, "%s: %s (0x%x)\n", complaint, syserr, GetLastError());
exit(1);
}


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Error accessing mapped drive >2TB?

[Warren Young]

On Oct 23, 2015, at 3:20 AM, Corinna Vinschen wrote:
> 
> Well, it was *you* asking "How could we prove that the problem is the
> Apple SMB server?"  I was just trying to help.  If that's not desired,
> I don't have to.

I shouldn’t have suggested attacking the problem at the SMB protocol layer to 
begin with.  That requires the sort of expertise that the Samba and Apple 
developers have, and I doubt I can get access to either.  Since I’m not going 
to go and acquire such expertise myself just to answer the question, it’s a 
hopeless line of inquiry.

Instead, it would be better if we can refine the Windows API C code we’ve been 
playing with to show the problem.  Then I can attach it to an Apple bug report. 
I’m sure they like STCs, too. :)

I’ve made the suggested changes to the program, here:

  http://pastebin.com/uZdDZPgi

Is that enough to take to Apple?  I mean, does this let them wiggle out and 
say, “You’re doing it wrong”?

> HANDLE handle = CreateFile ("P:\\", ...);
 
 I guess I’m not seeing what values to pass to CreateFile()
>>> 
>>> Opening a directory requires to use the FILE_FLAG_BACKUP_SEMANTICS
>>> flag.
>> 
>> Yes, silly me for not guessing that Windows requires that I tell it I
>> am about to do a backup before I attempt to open a directory for
>> reading.  What was so wrong about the design of opendir() that MS had
>> to reinvent it this way?
> 
> Think DOS/early Windows.  CreateFile was not meant to open directories
> to perform a directory search

I was reacting more to the revelation that Windows has workarounds for the 
known-problematic file access semantics, and that they’re specifically labeled 
as “for backup programs” when in reality they’re just ad hoc post facto fixes 
to a poorly thought out initial design.

They’ve reinvented Unix, poorly.
--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Error accessing mapped drive >2TB?

[Warren Young]

On Oct 23, 2015, at 8:30 AM, Jeffrey Altman  
wrote:
> 
> In this thread there appears to be a small amount of misunderstanding of
> what a reparse point is and how it should be used.

Thank you for clearing all of this up.  It was a fascinating read.

> the Apple SMB server does expose the existence of the mount point via
> the use of the RP file attribute.

Which is legal so far as it goes, right?

> Note that the size of the reparse data is zero.  There is no reparse
> data to read.  This is a UNIX mount point not an NTFS junction.

So is that wrong, or is it a valid way of shoehorning Unix filesystem behavior 
(mount points and such) into the SMB framework?

> Apple should have registered with Microsoft their own reparse point tag.
> Instead they broke the rules and used Microsoft's
> IO_REPARSE_TAG_MOUNT_POINT

If Apple uses their own tags, wouldn’t that cause the Windows SMB client to be 
unable to understand Unix mount points, when if it comes across them?

I don’t see that the Apple SMB server really needs to report Unix mount points 
at the root of a share, but they could also appear in the middle of a share, at 
which point I assume there are important implications to SMB, equivalent to the 
inode uniqueness problem on Unix.

Therefore, I can see that Apple’s SMB server needs a way to tell the client 
that it is crossing a filesystem boundary.  The question is, is the way Apple 
chose  a sensible one?

> applications cannot rely on the serial numbers to distinguish
> between devices.  Instead, the applications must do as the Explorer
> Shell does and track the locations of the RP attributes in paths as they
> are encountered.

Isn’t the Explorer behavior more robust, anyway?  Are device serial numbers 
GUID-like, so that there is no need for central coordination to avoid 
collisions?  If not, I don’t see that a robust application should rely on them, 
anyway.

I’m not including things like the udev rules on Linux, where you can use a 
serial number to work out where to automount a removable volume regardless of 
which bus it appears on.  In that case, you’re dealing with a small number of 
serial numbers, so the chances of collision are small.

I don’t think Explorer has the luxury of making such assumptions because it has 
to work in all possible combinations of hardware and software, by its nature.  
It is not possible to fix collisions by configuration, as with udev.

> While Apple's design choices do not fit with the expectations of Cygwin
> they are not necessarily wrong.

So, should I send Apple this code, or not?

  http://pastebin.com/uZdDZPgi
--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Error accessing mapped drive >2TB?

[Corinna Vinschen]

On Oct 21 11:26, Warren Young wrote:
> On Oct 21, 2015, at 10:22 AM, Corinna Vinschen wrote:
> > 
> > On Oct 21 09:52, Warren Young wrote:
> >> 
> >> I mean, I know how to snag a stream of SMB packets with Wireshark, but
> >> I don’t know what I’d be looking for in the dump.
> > 
> > Me neither, the Samba guys might be able to help there, perhaps.
> 
> Apple hasn’t shipped Samba as part of OS X since 10.6, quite a few
> years ago now.  In 10.7, they switched to an internally-developed SMB
> server.

Yes, but the SMB guys can recite the wire format of SMB asleep, probably.
There's also
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365233%28v=vs.85%29.aspx

> [...]
> >  HANDLE handle = CreateFile ("P:\\", ...);
> 
> I guess I’m not seeing what values to pass to CreateFile() because I
> get an error with the values I’m trying here.  I’ve put my fleshed-out
> test program here:
> 
>   http://pastebin.com/BfN2fNBQ
> 
> Its complaint is:
> 
>   Bad handle: The filename, directory name, or volume label syntax is
>   incorrect.  (0x7b)
> 
> I double-checked, and P: is still mapped.

Opening a directory requires to use the FILE_FLAG_BACKUP_SEMANTICS flag.
See the Remarks section of
https://msdn.microsoft.com/en-us/library/windows/desktop/aa363858%28v=vs.85%29.aspx

Oh and, you have to use the FILE_FLAG_OPEN_REPARSE_POINT flag, of
course.

This might be the difference to Explorer.  If the server accidentally
returns the FILE_ATTRIBUTE_REPARSE_POINT flag only if the dir has been
opened with FILE_FLAG_OPEN_REPARSE_POINT, Explorer would never see
this.  In contrast to Cygwin it's not interested in the fact whether
the dir is a reparse point or not.


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


pgpI_Wz36gFMY.pgp
Description: PGP signature

Re: Error accessing mapped drive >2TB?

[Warren Young]

On Oct 22, 2015, at 2:34 AM, Corinna Vinschen wrote:
> 
> On Oct 21 11:26, Warren Young wrote:
>> On Oct 21, 2015, at 10:22 AM, Corinna Vinschen wrote:
>>> 
>>> On Oct 21 09:52, Warren Young wrote:
 
 I mean, I know how to snag a stream of SMB packets with Wireshark, but
 I don’t know what I’d be looking for in the dump.
>>> 
>>> Me neither, the Samba guys might be able to help there, perhaps.
>> 
>> Apple hasn’t shipped Samba as part of OS X since 10.6, quite a few
>> years ago now.  In 10.7, they switched to an internally-developed SMB
>> server.
> 
> Yes, but the SMB guys can recite the wire format of SMB asleep, probably.

Why would the Samba project’s wizards be motivated to help debug a 
closed-source piece of software they didn’t write?

If anything, I’d expect them to give Apple the finger for booting Samba out of 
OS X in the first place.

> There's also
> https://msdn.microsoft.com/en-us/library/windows/desktop/aa365233%28v=vs.85%29.aspx

Now we’re talking about *my* motivation, which has only been to test the 
original report and the fix.  I never had this problem — I don’t export whole 
drives as a matter of principle — and the symptom is now fixed, leaving me 
without a motivation to learn the SMB protocol.

The only people left with both means and motivation would be at Apple, and then 
only if they had important software that broke because of their implementation 
choices.  You just fixed the only other known piece of software that breaks.

(Please don’t back out the fix just to put pressure on Apple! :) )

>> [...]
>>> HANDLE handle = CreateFile ("P:\\", ...);
>> 
>> I guess I’m not seeing what values to pass to CreateFile()
> 
> Opening a directory requires to use the FILE_FLAG_BACKUP_SEMANTICS flag.

Yes, silly me for not guessing that Windows requires that I tell it I am about 
to do a backup before I attempt to open a directory for reading.  What was so 
wrong about the design of opendir() that MS had to reinvent it this way?

I also had to fix an error in your original code: GetFileAttributes() takes a 
path string, not a HANDLE.

After doing both of these things, I now get a new complaint:

Failed to get reparse point: The file or directory is not a reparse point.
 (0x1126)

I’m reporting this in case it affects the way you want your patch for this 
problem to react.

Here’s the new program: http://pastebin.com/kUwPrk8v

> Oh and, you have to use the FILE_FLAG_OPEN_REPARSE_POINT flag, of
> course.

Adding that doesn’t affect the error I get from the program, so I left it out 
in the Pastebin version.
--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Error accessing mapped drive >2TB?

[Corinna Vinschen]

On Oct 21 16:03, Andrey Repin wrote:
> Greetings, Corinna Vinschen!

Your mailer is STILL breaking the history, btw.  Could you reconfigure
so the "blah wrote:" header is preserved for each mail in the thread you
still quote?  Thanks.

> > On Oct 21 15:33, Andrey Repin wrote:
> 
> >> Windows does not allow for reparse points to networked locations.
> >> Symlinks all right, directory junctions no.
> 
> > Fine, but then the question is, why is the FILE_ATTRIBUTE_REPARSE_POINT
> > flag set in the first place?
> 
> May be the reparse point was manually tampered with.

How so?  You can't set or clear the FILE_ATTRIBUTE_REPARSE_POINT flag
on an existing file.  The flag seems to be set by the server providing
the drive for some reason.


Corinna


-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


pgpNJEExj7NUI.pgp
Description: PGP signature

Re: Error accessing mapped drive >2TB?

[Corinna Vinschen]

On Oct 21 09:52, Warren Young wrote:
> On Oct 21, 2015, at 4:03 AM, Corinna Vinschen wrote:
> > 
> > The FILE_ATTRIBUTE_REPARSE_POINT flag is set but when calling a function
> > to fetch the reparse data it's no reparse point anymore?  How dumb is
> > that?
> > 
> > I can't reproduce this bug with my Samba drives, but it's not actually a
> > bug in Cygwin from my perspective.
> > 
> > Yeah, that observation doesn't really help, so I applied a potential
> > workaround to Cygwin.
> 
> How could we prove that the problem is the Apple SMB server?
> 
> I mean, I know how to snag a stream of SMB packets with Wireshark, but
> I don’t know what I’d be looking for in the dump.

Me neither, the Samba guys might be able to help there, perhaps.
Or, does wireshark know how SMB packages look like OTW?

Alternatively, a native Windows test application like this should
show the same symptom:

  HANDLE handle = CreateFile ("P:\\", ...);
  DWORD attributes = GetFileAttributes (handle);
  if (attributes != (DWORD) -1
  && (attributes & FILE_ATTRIBUTE_REPARSE_POINT))
{
  char buf[32000];
  DWORD len;

  if (!DeviceIoControl(handle, FSCTL_GET_REPARSE_POINT, NULL, 0,
   buf, 32000, , NULL))
printf ("error %u\n", GetLastError ());
}

If so, the error code would be 4390.

> Keep in mind that Windows Explorer seems to cope with this situation,
> so Explorer may be doing something like in your patch.

Yeah, maybe.  Or it doesn't even test for a reparse point in this
scenario.

> > If you're set up to build your own Cygwin, please give it a try.
> 
> Yes, that fixes the symptom here.
> 
> Thanks for chasing this down.

Thanks for testing!


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


pgp_7kvVX9CdG.pgp
Description: PGP signature

Re: Error accessing mapped drive >2TB?

[Warren Young]

On Oct 21, 2015, at 4:03 AM, Corinna Vinschen wrote:
> 
> The FILE_ATTRIBUTE_REPARSE_POINT flag is set but when calling a function
> to fetch the reparse data it's no reparse point anymore?  How dumb is
> that?
> 
> I can't reproduce this bug with my Samba drives, but it's not actually a
> bug in Cygwin from my perspective.
> 
> Yeah, that observation doesn't really help, so I applied a potential
> workaround to Cygwin.

How could we prove that the problem is the Apple SMB server?

I mean, I know how to snag a stream of SMB packets with Wireshark, but I don’t 
know what I’d be looking for in the dump.

Keep in mind that Windows Explorer seems to cope with this situation, so 
Explorer may be doing something like in your patch.

> If you're set up to build your own Cygwin, please give it a try.

Yes, that fixes the symptom here.

Thanks for chasing this down.
--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Error accessing mapped drive >2TB?

[Warren Young]

On Oct 21, 2015, at 10:22 AM, Corinna Vinschen wrote:
> 
> On Oct 21 09:52, Warren Young wrote:
>> 
>> I mean, I know how to snag a stream of SMB packets with Wireshark, but
>> I don’t know what I’d be looking for in the dump.
> 
> Me neither, the Samba guys might be able to help there, perhaps.

Apple hasn’t shipped Samba as part of OS X since 10.6, quite a few years ago 
now.  In 10.7, they switched to an internally-developed SMB server.

> Or, does wireshark know how SMB packages look like OTW?

The build of Wireshark on the machine I’m using right now has about 1,700 
protocol dissectors, which covers pretty much every protocol you’ve ever heard 
of, and hundreds you haven’t.

The trick is, dissecting the packets is only useful if the protocol is human 
readable (SMB isn’t) or you know the protocol (I don’t) or you’re lucky and 
happen to see something you can make sense of.  I was hoping not to have to 
rely on blind luck.

>  HANDLE handle = CreateFile ("P:\\", ...);

I guess I’m not seeing what values to pass to CreateFile() because I get an 
error with the values I’m trying here.  I’ve put my fleshed-out test program 
here:

  http://pastebin.com/BfN2fNBQ

Its complaint is:

  Bad handle: The filename, directory name, or volume label syntax is incorrect.
 (0x7b)

I double-checked, and P: is still mapped.

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Error accessing mapped drive >2TB?

[Andrey Repin]

Greetings, Corinna Vinschen!

> On Oct 21 15:33, Andrey Repin wrote:

>> Windows does not allow for reparse points to networked locations.
>> Symlinks all right, directory junctions no.

> Fine, but then the question is, why is the FILE_ATTRIBUTE_REPARSE_POINT
> flag set in the first place?

May be the reparse point was manually tampered with.

I just ran strace on two ls calls.
On symlink to a network share

 1962   29951 [main] ls 28776 lstat64: entering
   18   29969 [main] ls 28776 normalize_posix_path: src /c/arc
   15   29984 [main] ls 28776 normalize_posix_path: /c/arc = 
normalize_posix_path (/c/arc)
   16   3 [main] ls 28776 mount_info::conv_to_win32_path: 
conv_to_win32_path (/c/arc)
   14   30014 [main] ls 28776 mount_info::cygdrive_win32_path: src '/c/arc', 
dst 'C:\arc'
   13   30027 [main] ls 28776 set_flags: flags: binary (0x2)
   13   30040 [main] ls 28776 mount_info::conv_to_win32_path: src_path /c/arc, 
dst C:\arc, flags 0x6022, rc 0
   38   30078 [main] ls 28776 symlink_info::check: 0x0 = NtCreateFile 
(\??\C:\arc)
   38   30116 [main] ls 28776 symlink_info::check: 28 = symlink.check(C:\arc, 
0x22B600) (0x4406023)
   15   30131 [main] ls 28776 path_conv::check: this->path(C:\arc), has_acls(1)
   17   30148 [main] ls 28776 build_fh_pc: fh 0x180329B00, dev 00C3
   15   30163 [main] ls 28776 stat_worker: (\??\C:\arc, 0x60008A2F0, 
0x180329B00), file_attributes 1024
   19   30182 [main] ls 28776 fhandler_base::fstat_helper: 0 = fstat 
(\??\C:\arc, 0x60008A2F0) st_size=28, st_mode=0120777, 
st_ino=443041613342573138st_atim=54BC1392.15238F60 st_ctim=54BC1392.15238F60 
st_mtim=54BC1392.15238F60 st_birthtim=54BC1392.15238F60
   17   30199 [main] ls 28776 stat_worker: 0 = (\??\C:\arc,0x60008A2F0)
   37   30236 [main] ls 28776 normalize_posix_path: src /c/arc
   13   30249 [main] ls 28776 normalize_posix_path: /c/arc = 
normalize_posix_path (/c/arc)
   13   30262 [main] ls 28776 mount_info::conv_to_win32_path: 
conv_to_win32_path (/c/arc)
   14   30276 [main] ls 28776 mount_info::cygdrive_win32_path: src '/c/arc', 
dst 'C:\arc'
   13   30289 [main] ls 28776 set_flags: flags: binary (0x2)
   13   30302 [main] ls 28776 mount_info::conv_to_win32_path: src_path /c/arc, 
dst C:\arc, flags 0x6022, rc 0
   32   30334 [main] ls 28776 symlink_info::check: 0x0 = NtCreateFile 
(\??\C:\arc)
   37   30371 [main] ls 28776 symlink_info::check: 28 = symlink.check(C:\arc, 
0x22B570) (0x4006023)
   15   30386 [main] ls 28776 path_conv::check: 
this->path(//DAEMON1.DARKDRAGON.LAN/arc), has_acls(1)

On directory junction to another local drive:

 1825   26534 [main] ls 26712 lstat64: entering
   19   26553 [main] ls 26712 normalize_posix_path: src /c/Users
   14   26567 [main] ls 26712 normalize_posix_path: /c/Users = 
normalize_posix_path (/c/Users)
   14   26581 [main] ls 26712 mount_info::conv_to_win32_path: 
conv_to_win32_path (/c/Users)
   13   26594 [main] ls 26712 mount_info::cygdrive_win32_path: src '/c/Users', 
dst 'C:\Users'
   12   26606 [main] ls 26712 set_flags: flags: binary (0x2)
   12   26618 [main] ls 26712 mount_info::conv_to_win32_path: src_path 
/c/Users, dst C:\Users, flags 0x6022, rc 0
   45   26663 [main] ls 26712 symlink_info::check: 0x0 = NtCreateFile 
(\??\C:\Users)
  108   26771 [main] ls 26712 symlink_info::check: not a symlink
   20   26791 [main] ls 26712 symlink_info::check: 0 = symlink.check(C:\Users, 
0x22B600) (0x6022)
   14   26805 [main] ls 26712 path_conv::check: this->path(C:\Users), 
has_acls(1)
   16   26821 [main] ls 26712 build_fh_pc: fh 0x180329B00, dev 00C3
   14   26835 [main] ls 26712 stat_worker: (\??\C:\Users, 0x60008A2F0, 
0x180329B00), file_attributes 1072
   13   26848 [main] ls 26712 fhandler_base::open: (\??\C:\Users, 0x11)
   54   26902 [main] ls 26712 fhandler_base::set_flags: flags 0x11, 
supplied_bin 0x1
   15   26917 [main] ls 26712 fhandler_base::set_flags: O_TEXT/O_BINARY set in 
flags 0x1
   12   26929 [main] ls 26712 fhandler_base::set_flags: filemode set to binary
   12   26941 [main] ls 26712 fhandler_base::open: 0x0 = NtCreateFile (0x29C, 
0x8010, \??\C:\Users, io, NULL, 0x0, 0x7, 0x1, 0x4020, NULL, 0)
   13   26954 [main] ls 26712 fhandler_base::open: 1 = 
fhandler_base::open(\??\C:\Users, 0x11)
   15   26969 [main] ls 26712 fhandler_base::open_fs: 1 = 
fhandler_disk_file::open(\??\C:\Users, 0x1)
   17   26986 [main] ls 26712 fhandler_base::fstat_helper: 0 = fstat 
(\??\C:\Users, 0x60008A2F0) st_size=0, st_mode=040755, 
st_ino=562949953508329st_atim=540F4952.1CDEF9D8 st_ctim=5405F6F7.8AFF78C 
st_mtim=5405F6F7.8AFF78C st_birthtim=53DFAC4D.59682F0
   14   27000 [main] ls 26712 fhandler_base::close: closing '/c/Users' handle 
0x29C
   20   27020 [main] ls 26712 stat_worker: 0 = (\??\C:\Users,0x60008A2F0)
   17   27037 [main] ls 26712 normalize_posix_path: src /c/Users
   12   27049 [main] ls 26712 normalize_posix_path: /c/Users = 
normalize_posix_path (/c/Users)
   12   27061 [main] ls 26712 

Re: Error accessing mapped drive >2TB?

[Corinna Vinschen]

On Sep 14 14:34, Warren Young wrote:
> On Sep 12, 2015, at 11:14 AM, Nem W Schlecht  wrote:
> > 
> > The only thing I can think of is that the 2nd drive is >2TB.
> 
> The same thing happens here with a 3.1 TB Fusion drive and a 500 GB external 
> drive, so no, I don’t think the volume size is the real issue.
> 
> I assume you are seeing the same thing that I am, that Explorer shows the 
> root contents of both drives just fine?
> 
> When I strace’d an ls of one of these root shares here, I got 764 lines 
> of…emissions, of which this section seems the most interesting to me:
> 
> 1051  263166 [main] ls 4720 stat64: entering
>   625  263791 [main] ls 4720 normalize_posix_path: src /p
>   609  264400 [main] ls 4720 normalize_posix_path: /p = normalize_posix_path 
> (/p)
>   630  265030 [main] ls 4720 mount_info::conv_to_win32_path: 
> conv_to_win32_path (/p)
>   653  265683 [main] ls 4720 mount_info::cygdrive_win32_path: src '/p', dst 
> 'P:\'
>   668  266351 [main] ls 4720 set_flags: flags: binary (0x2)
>   674  267025 [main] ls 4720 mount_info::conv_to_win32_path: src_path /p, dst 
> P:\, flags 0x4022, rc 0
>  1168  268193 [main] ls 4720 symlink_info::check: 0x0 = NtCreateFile (\??\P:\)
> 10641  278834 [main] ls 4720 symlink_info::check_reparse_point: 
> NtFsControlFile(FSCTL_GET_REPARSE_POINT) failed, 0xC275
>   839  279673 [main] ls 4720 symlink_info::check: not a symlink
>  1049  280722 [main] ls 4720 symlink_info::check: 0 = symlink.check(P:\, 
> 0x24B620) (0x4022)
>   655  281377 [main] ls 4720 path_conv::check: this->path(P:\), has_acls(0)
>   640  282017 [main] ls 4720 stat_worker: got 5 error from path_conv
>   757  282774 [main] ls 4720 __set_errno: int stat_worker(path_conv&, 
> stat*):1933 setting errno 5
>   714  283488 [main] ls 4720 stat_worker: -1 = (\??\P:\,0x600042080)
> 
> Why errno 5 from path_conv?

Probably because of the above

  symlink_info::check_reparse_point: NtFsControlFile(FSCTL_GET_REPARSE_POINT)
  failed, 0xC275

This is in fact a weird error code in this scenario.

Consider that symlink_info::check_reparse_point is *only* called, if the
file to check (here "P:\") has the FILE_ATTRIBUTE_REPARSE_POINT flag
set.  So from the Windows perspective it is certainly a reparse point.

Cygwin checks the flag to allow evaluating of reparse points as symlinks.
If the flag is set, it calls symlink_info::check_reparse_point which in
turn calls

  status = NtFsControlFile (..., FSCTL_GET_REPARSE_POINT, ...);

to fetch the target information the reparse point points to, in POSIX
terms the symlink target.  But *this* call returns a status of 0xC275,
which means STATUS_NOT_A_REPARSE_POINT.  And since it's totally unexpected
that NtFsControlFile fails on a reparse point, the code sets errno to EIO.

Hang on.

The FILE_ATTRIBUTE_REPARSE_POINT flag is set but when calling a function
to fetch the reparse data it's no reparse point anymore?  How dumb is
that?

I can't reproduce this bug with my Samba drives, but it's not actually a
bug in Cygwin from my perspective.

Yeah, that observation doesn't really help, so I applied a potential
workaround to Cygwin.  If you're set up to build your own Cygwin, please
give it a try.  Otherwise I'll upload a new snapshot in the next couple of
days.


Thanks,
Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


pgpChY3LL4nd1.pgp
Description: PGP signature

Re: Error accessing mapped drive >2TB?

[Andrey Repin]

Greetings, Corinna Vinschen!

>> > The only thing I can think of is that the 2nd drive is >2TB.
>> 
>> The same thing happens here with a 3.1 TB Fusion drive and a 500 GB external 
>> drive, so no, I don’t think the volume size is the real issue.
>> 
>> I assume you are seeing the same thing that I am, that Explorer shows the 
>> root contents of both drives just fine?
>> 
>> When I strace’d an ls of one of these root shares here, I got 764 lines 
>> of…emissions, of which this section seems the most interesting to me:
>> 
>> 1051  263166 [main] ls 4720 stat64: entering
>>   625  263791 [main] ls 4720 normalize_posix_path: src /p
>>   609  264400 [main] ls 4720 normalize_posix_path: /p = normalize_posix_path 
>> (/p)
>>   630  265030 [main] ls 4720 mount_info::conv_to_win32_path: 
>> conv_to_win32_path (/p)
>>   653  265683 [main] ls 4720 mount_info::cygdrive_win32_path: src '/p', dst 
>> 'P:\'
>>   668  266351 [main] ls 4720 set_flags: flags: binary (0x2)
>>   674  267025 [main] ls 4720 mount_info::conv_to_win32_path: src_path /p, 
>> dst P:\, flags 0x4022, rc 0
>>  1168  268193 [main] ls 4720 symlink_info::check: 0x0 = NtCreateFile 
>> (\??\P:\)
>> 10641  278834 [main] ls 4720 symlink_info::check_reparse_point: 
>> NtFsControlFile(FSCTL_GET_REPARSE_POINT) failed, 0xC275
>>   839  279673 [main] ls 4720 symlink_info::check: not a symlink
>>  1049  280722 [main] ls 4720 symlink_info::check: 0 = symlink.check(P:\, 
>> 0x24B620) (0x4022)
>>   655  281377 [main] ls 4720 path_conv::check: this->path(P:\), has_acls(0)
>>   640  282017 [main] ls 4720 stat_worker: got 5 error from path_conv
>>   757  282774 [main] ls 4720 __set_errno: int stat_worker(path_conv&, 
>> stat*):1933 setting errno 5
>>   714  283488 [main] ls 4720 stat_worker: -1 = (\??\P:\,0x600042080)
>> 
>> Why errno 5 from path_conv?

> Probably because of the above

>   symlink_info::check_reparse_point: NtFsControlFile(FSCTL_GET_REPARSE_POINT)
>   failed, 0xC275

> This is in fact a weird error code in this scenario.

> Consider that symlink_info::check_reparse_point is *only* called, if the
> file to check (here "P:\") has the FILE_ATTRIBUTE_REPARSE_POINT flag
> set.  So from the Windows perspective it is certainly a reparse point.

> Cygwin checks the flag to allow evaluating of reparse points as symlinks.
> If the flag is set, it calls symlink_info::check_reparse_point which in
> turn calls

>   status = NtFsControlFile (..., FSCTL_GET_REPARSE_POINT, ...);

> to fetch the target information the reparse point points to, in POSIX
> terms the symlink target.  But *this* call returns a status of 0xC275,
> which means STATUS_NOT_A_REPARSE_POINT.  And since it's totally unexpected
> that NtFsControlFile fails on a reparse point, the code sets errno to EIO.

> Hang on.

> The FILE_ATTRIBUTE_REPARSE_POINT flag is set but when calling a function
> to fetch the reparse data it's no reparse point anymore?  How dumb is
> that?

That's… interesting.
Windows does not allow for reparse points to networked locations.
Symlinks all right, directory junctions no.

> I can't reproduce this bug with my Samba drives, but it's not actually a
> bug in Cygwin from my perspective.

> Yeah, that observation doesn't really help, so I applied a potential
> workaround to Cygwin.  If you're set up to build your own Cygwin, please
> give it a try.  Otherwise I'll upload a new snapshot in the next couple of
> days.


-- 
With best regards,
Andrey Repin
Wednesday, October 21, 2015 15:32:13

Sorry for my terrible english...

Re: Error accessing mapped drive >2TB?

[Corinna Vinschen]

On Oct 21 15:33, Andrey Repin wrote:
> Greetings, Corinna Vinschen!

Your mailer is breaking the history, btw.  Could you reconfigure so the
"blah wrote:" header is preserved for each mail in the thread you still
quote?  Thanks.

> > Probably because of the above
> 
> >   symlink_info::check_reparse_point: 
> > NtFsControlFile(FSCTL_GET_REPARSE_POINT)
> >   failed, 0xC275
> 
> > This is in fact a weird error code in this scenario.
> 
> > Consider that symlink_info::check_reparse_point is *only* called, if the
> > file to check (here "P:\") has the FILE_ATTRIBUTE_REPARSE_POINT flag
> > set.  So from the Windows perspective it is certainly a reparse point.
> 
> > Cygwin checks the flag to allow evaluating of reparse points as symlinks.
> > If the flag is set, it calls symlink_info::check_reparse_point which in
> > turn calls
> 
> >   status = NtFsControlFile (..., FSCTL_GET_REPARSE_POINT, ...);
> 
> > to fetch the target information the reparse point points to, in POSIX
> > terms the symlink target.  But *this* call returns a status of 0xC275,
> > which means STATUS_NOT_A_REPARSE_POINT.  And since it's totally unexpected
> > that NtFsControlFile fails on a reparse point, the code sets errno to EIO.
> 
> > Hang on.
> 
> > The FILE_ATTRIBUTE_REPARSE_POINT flag is set but when calling a function
> > to fetch the reparse data it's no reparse point anymore?  How dumb is
> > that?
> 
> That's… interesting.
> Windows does not allow for reparse points to networked locations.
> Symlinks all right, directory junctions no.

Fine, but then the question is, why is the FILE_ATTRIBUTE_REPARSE_POINT
flag set in the first place?


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


pgpgXlvB7H26r.pgp
Description: PGP signature

Re: Error accessing mapped drive >2TB?

[Andrey Repin]

Greetings, Brian Inglis!

>> > Windows Explorer drive mappings are not visible from cmd or mintty/bash
>> > windows, but "net use x: \\live.sysinternals.com\tools" mappings are 
>> > visible
>> > from separate cmd and mintty/bash windows as drive X: or /cygdrive/x, but
>> > x:\ is not accessible from Windows Explorer. 
>> 
>> Are you sure you're looking from the same user session?

> Same user session, different (elevated) cmd and mintty/bash console sessions. 

Elevated process uses different user session.
Network mapped drive letters are session-bound.

>> If you start mintty/cmd elevated, you won't see drives mapped in Explorer
>> (non-elevated).

> Kind of figures on Windows - I would expect elevated sessions to be able to
> see mappings on non-elevated sessions. 

That's not how it works. And is a part of reason why using mapped drives is
discouraged. For 15 years now, at least.

>> For me, everything works as expected. Drives mapped in batch file on system
>> start visible in Explorer without an issue.

> Found that when I started using W7 console sessions, net utilities, both MS
> and Cygwin nslookup, ping, tracert, etc. could not see the network unless
> they were run in elevated sessions, so set up all console sessions to start
> elevated. Was that something that happened on early releases and was fixed
> later, or some incorrect assumption or setting on my part years ago? 

The latter.
If your logon script attempts to mount (net use) drive letters, but Explorer
don't see them, double check that it did indeed mounted them. I.e. insert
"pause" before the end of the script and check for any errors after its
execution.
Another possible solution (Cygwin-specific) is to use fstab bind mount for
network paths.
Third one, and that I've gone with myself is to use Vista+ NTFS functionality
to symlink network paths to local FS. Just remember to use FQDN host names or
IPs when doing so, else you could intermittently lose connectivity when using
VPN connections to other networks with different domain search suffixes.


-- 
With best regards,
Andrey Repin
Tuesday, September 22, 2015 11:20:20

Sorry for my terrible english...


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Error accessing mapped drive >2TB?

[Andrey Repin]

Greetings, Brian Inglis!

> $ ls $(cygpath '\\live.sysinternals.com\tools\') # works - list contents
> About_This_Site.txt  ctrl2cap.amd.sys*  Eula.txt
> ...
> $ ls `cygpath '\\live.sysinternals.com\tools\'`  # fails - should be same as
> above 2
> ls: cannot access /cygdrive/c/live.sysinternals.com/tools/: No such file or
> directory

Not necessarily. This is why using the backticks is discouraged.
They have weird escaping rules and generally non-obvious.
Stick to $( ... ).

> Windows Explorer drive mappings are not visible from cmd or mintty/bash
> windows, but "net use x: \\live.sysinternals.com\tools" mappings are visible
> from separate cmd and mintty/bash windows as drive X: or /cygdrive/x, but
> x:\ is not accessible from Windows Explorer. 

Are you sure you're looking from the same user session?
If you start mintty/cmd elevated, you won't see drives mapped in Explorer
(non-elevated).

For me, everything works as expected. Drives mapped in batch file on system
start visible in Explorer without an issue.


-- 
With best regards,
Andrey Repin
Monday, September 21, 2015 08:53:08

Sorry for my terrible english...


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Error accessing mapped drive >2TB?

[Brian Inglis]

Andrey Repin  yandex.ru> writes:

> 
> Greetings, Brian Inglis!
> 
> > $ ls $(cygpath '\\live.sysinternals.com\tools\') # works - list contents
> > About_This_Site.txt  ctrl2cap.amd.sys*  Eula.txt
> > ...
> > $ ls `cygpath '\\live.sysinternals.com\tools\'`  # fails - should be same as
> > above 2
> > ls: cannot access /cygdrive/c/live.sysinternals.com/tools/: No such file or
> > directory
> 
> Not necessarily. This is why using the backticks is discouraged.
> They have weird escaping rules and generally non-obvious.
> Stick to $( ... ).

Never noticed any issues previous to this but generally don't use
backslashes except as escape character, and MS utilities which require
legacy path separators (even under DOS, many non-MS utilities would accept
normal path separators, as the underlying APIs didn't care, and neither do
most of the Windows APIs). 

> > Windows Explorer drive mappings are not visible from cmd or mintty/bash
> > windows, but "net use x: \\live.sysinternals.com\tools" mappings are visible
> > from separate cmd and mintty/bash windows as drive X: or /cygdrive/x, but
> > x:\ is not accessible from Windows Explorer. 
> 
> Are you sure you're looking from the same user session?

Same user session, different (elevated) cmd and mintty/bash console sessions. 

> If you start mintty/cmd elevated, you won't see drives mapped in Explorer
> (non-elevated).

Kind of figures on Windows - I would expect elevated sessions to be able to
see mappings on non-elevated sessions. 

> For me, everything works as expected. Drives mapped in batch file on system
> start visible in Explorer without an issue.

Found that when I started using W7 console sessions, net utilities, both MS
and Cygwin nslookup, ping, tracert, etc. could not see the network unless
they were run in elevated sessions, so set up all console sessions to start
elevated. Was that something that happened on early releases and was fixed
later, or some incorrect assumption or setting on my part years ago? 



--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Error accessing mapped drive >2TB?

[Brian Inglis]

Warren Young  etr-usa.com> writes:
> On Sep 16, 2015, at 7:39 AM, Nem W Schlecht  gmail.com> wrote:
> > I would think if it was an issue with Mac's SMB implementation, then
> > *Windows* would also have some sort of issues with it.  But it shows
> > up fine on Windows, I would assume it should show fine in Cygwin as
> > well.
> More than that: Cygwin doesn’t contain an SMB client.  (I mean, not at the
cygwin1.dll level.)  It is just
> using what Windows gives it.
> That’s why I’m grasping at straws like the path error in the strace
output, since the only thing that
> makes sense to me is a problem in the way Cygwin is interpreting what it
gets from Windows, rather than the
> SMB protocol peers doing the wrong thing.
> I suppose it could be that Windows Explorer and cygwin1.dll are making
different NT kernel syscalls, and
> that explains the problem.  Lacking a “real” strace on Windows, I’m not
sure how to test that.  Maybe
> Process Monitor could do this?
>   https://technet.microsoft.com/en-us/sysinternals/bb896645

I remembered using the live site for SysInternals, giving the interesting
result: 

$ cygstart '\\live.sysinternals.com\tools\' # works - opens site in
Windows Explorer
$ cygpath '\\live.sysinternals.com\tools\'
//live.sysinternals.com/tools/
$ ls //live.sysinternals.com/tools/  # works - list contents
About_This_Site.txt  ctrl2cap.amd.sys*  Eula.txt
...
$ ls $(cygpath '\\live.sysinternals.com\tools\') # works - list contents
About_This_Site.txt  ctrl2cap.amd.sys*  Eula.txt
...
$ ls `cygpath '\\live.sysinternals.com\tools\'`  # fails - should be same as
above 2
ls: cannot access /cygdrive/c/live.sysinternals.com/tools/: No such file or
directory

Windows Explorer drive mappings are not visible from cmd or mintty/bash
windows, but "net use x: \\live.sysinternals.com\tools" mappings are visible
from separate cmd and mintty/bash windows as drive X: or /cygdrive/x, but
x:\ is not accessible from Windows Explorer. 
It does not seem to matter whether the drive mapping is made persistent from
either the console or Windows Explorer - it is only visible from whichever
type of window interface was used to make the mapping. 
I have not tested making the mapping persistent, logging out and back in, to
see if it makes any difference to visibility: it shouldn't, but with
Windows...who knows? 

Testing was on a standalone non-domain W7 desktop with:
Microsoft Windows [Version 6.1.7601]
CYGWIN_NT-6.1 ... 2.2.1(0.289/5/3) 2015-08-20 11:42 x86_64 Cygwin

Re: Error accessing mapped drive >2TB?

[Nem W Schlecht]

On Tue, Sep 15, 2015 at 6:21 PM, Warren Young  wrote:
> On Sep 15, 2015, at 9:14 AM, Nem W Schlecht  wrote:
>>
>> I'm going to be spinning up an Ubuntu box in the next few days that
>> will have a 5TB drive in it as well, so I'll be able to test to see if
>> the source OS (and/or version of SMB server) makes a difference.
>
> Good.
>
> Please test by exporting the filesystem root, since exporting a folder makes 
> the problem go away here.  Maybe Cygwin’s UNC path parsing simply can’t cope 
> with a bare slash where it expected at least one folder name?

The filesystem I'm having troubles with on the mac side is
/Volumes/project and being mapped on Windows from \\hostname\project
(or accessed via UNC in Cygwin as //hostname/project) so I'm already
thinking its not an issue with the path.

I would think if it was an issue with Mac's SMB implementation, then
*Windows* would also have some sort of issues with it.  But it shows
up fine on Windows, I would assume it should show fine in Cygwin as
well.

-- 
Nem W Schlecht

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Error accessing mapped drive >2TB?

[Warren Young]

On Sep 16, 2015, at 7:39 AM, Nem W Schlecht  wrote:
> 
> I would think if it was an issue with Mac's SMB implementation, then
> *Windows* would also have some sort of issues with it.  But it shows
> up fine on Windows, I would assume it should show fine in Cygwin as
> well.

More than that: Cygwin doesn’t contain an SMB client.  (I mean, not at the 
cygwin1.dll level.)  It is just using what Windows gives it.

That’s why I’m grasping at straws like the path error in the strace output, 
since the only thing that makes sense to me is a problem in the way Cygwin is 
interpreting what it gets from Windows, rather than the SMB protocol peers 
doing the wrong thing.

I suppose it could be that Windows Explorer and cygwin1.dll are making 
different NT kernel syscalls, and that explains the problem.  Lacking a “real” 
strace on Windows, I’m not sure how to test that.  Maybe Process Monitor could 
do this?

  https://technet.microsoft.com/en-us/sysinternals/bb896645

There is also this, one of the Google results for “Windows strace”:

  http://www.drmemory.org/strace_for_windows.html
--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Error accessing mapped drive >2TB?

[Andrey Repin]

Greetings, Warren Young!

>> I'm going to be spinning up an Ubuntu box in the next few days that
>> will have a 5TB drive in it as well, so I'll be able to test to see if
>> the source OS (and/or version of SMB server) makes a difference.

> Good.

> Please test by exporting the filesystem root, since exporting a folder
> makes the problem go away here.  Maybe Cygwin’s UNC path parsing simply
> can’t cope with a bare slash where it expected at least one folder name?
Even if you export whole root, it will have a "folder name" :)


-- 
With best regards,
Andrey Repin
Wednesday, September 16, 2015 03:24:48

Sorry for my terrible english...

Re: Error accessing mapped drive >2TB?

[Warren Young]

On Sep 14, 2015, at 9:46 PM, Andrey Repin  wrote:
> 
>>> The only thing I can think of is that the 2nd drive is >2TB.
> 
>> The same thing happens here with a 3.1 TB Fusion drive and a 500 GB
>> external drive…

[snip]

>> Why errno 5 from path_conv?
> 
> That's an interesting point... can you walk me through your setup

Pretty simple: I shared the entire drive from my OS X box to the Cygwin VM 
hosted on the same machine, and got this error with commands like “ls /p” when 
the drive is mapped as P:.

If I map my OS X Desktop folder to Q: on the Windows side, the problem doesn’t 
occur.

There are no ACLs on the root of either Mac OS X drive (ls -led /) so if it’s 
an OS X-side permission problem, it’s happening at a different layer than the 
filesystem.  Perhaps one of the MAC layers?  (Gatekeeper, etc.)

> Or is this Mac-specific?

Possibly.  OS X switched from Samba to an Apple-specific smbd in 10.7:

  
https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man8/smbd.8.html

I’ll repeat that Windows Explorer seems to have no problem showing the contents 
of the root of the P: share, though.
--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Error accessing mapped drive >2TB?

[Warren Young]

On Sep 15, 2015, at 9:14 AM, Nem W Schlecht  wrote:
> 
> I'm going to be spinning up an Ubuntu box in the next few days that
> will have a 5TB drive in it as well, so I'll be able to test to see if
> the source OS (and/or version of SMB server) makes a difference.

Good.

Please test by exporting the filesystem root, since exporting a folder makes 
the problem go away here.  Maybe Cygwin’s UNC path parsing simply can’t cope 
with a bare slash where it expected at least one folder name?
--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Error accessing mapped drive >2TB?

[Nem W Schlecht]

I'm going to be spinning up an Ubuntu box in the next few days that
will have a 5TB drive in it as well, so I'll be able to test to see if
the source OS (and/or version of SMB server) makes a difference.

I did an strace on '/bin/ls /cygdrive/t' on my host and saw almost the
exact same thing Warren did (slight re-ordering of events).  I tried
creating a new mount mount with 'noacl,notexec' as options since it
looked like maybe the has_acls() call was throwing the error in the
strace output, but my attempt did not work.  I'm going to play with
that some more this afternoon, though.

On Mon, Sep 14, 2015 at 10:46 PM, Andrey Repin  wrote:
> Greetings, Warren Young!
>
>>> The only thing I can think of is that the 2nd drive is >2TB.
>
>> The same thing happens here with a 3.1 TB Fusion drive and a 500 GB
>> external drive, so no, I don’t think the volume size is the real issue.
>
>> I assume you are seeing the same thing that I am, that Explorer shows the
>> root contents of both drives just fine?
>
>> When I strace’d an ls of one of these root shares here, I got 764 lines
>> of…emissions, of which this section seems the most interesting to me:
>
>> 1051  263166 [main] ls 4720 stat64: entering
>>   625  263791 [main] ls 4720 normalize_posix_path: src /p
>>   609  264400 [main] ls 4720 normalize_posix_path: /p = normalize_posix_path 
>> (/p)
>>   630  265030 [main] ls 4720 mount_info::conv_to_win32_path: 
>> conv_to_win32_path (/p)
>>   653  265683 [main] ls 4720 mount_info::cygdrive_win32_path: src '/p', dst 
>> 'P:\'
>>   668  266351 [main] ls 4720 set_flags: flags: binary (0x2)
>>   674  267025 [main] ls 4720 mount_info::conv_to_win32_path: src_path /p, 
>> dst P:\, flags 0x4022, rc 0
>>  1168  268193 [main] ls 4720 symlink_info::check: 0x0 = NtCreateFile 
>> (\??\P:\)
>> 10641  278834 [main] ls 4720 symlink_info::check_reparse_point:
>> NtFsControlFile(FSCTL_GET_REPARSE_POINT) failed, 0xC275
>>   839  279673 [main] ls 4720 symlink_info::check: not a symlink
>>  1049  280722 [main] ls 4720 symlink_info::check: 0 = symlink.check(P:\, 
>> 0x24B620) (0x4022)
>>   655  281377 [main] ls 4720 path_conv::check: this->path(P:\), has_acls(0)
>>   640  282017 [main] ls 4720 stat_worker: got 5 error from path_conv
>>   757  282774 [main] ls 4720 __set_errno: int stat_worker(path_conv&, 
>> stat*):1933 setting errno 5
>>   714  283488 [main] ls 4720 stat_worker: -1 = (\??\P:\,0x600042080)
>
>> Why errno 5 from path_conv?
>
>> Yes, I am one of those evil people who edit cygdrive out of /etc/fstab so I 
>> can use /x, where x=drive letter. :)
>
> That's an interesting point... can you walk me through your setup, so I could
> try to recreate the issue?
> Or is this Mac-specific?
>
>
> --
> With best regards,
> Andrey Repin
> Tuesday, September 15, 2015 06:45:22
>
> Sorry for my terrible english...



-- 
Nem W Schlecht   n...@emptec.com
Empyreal Technologieshttp://www.emptec.com/
 "Perl did the magic.  I just waved the wand."

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Error accessing mapped drive >2TB?

[Warren Young]

On Sep 12, 2015, at 11:14 AM, Nem W Schlecht  wrote:
> 
> The only thing I can think of is that the 2nd drive is >2TB.

The same thing happens here with a 3.1 TB Fusion drive and a 500 GB external 
drive, so no, I don’t think the volume size is the real issue.

I assume you are seeing the same thing that I am, that Explorer shows the root 
contents of both drives just fine?

When I strace’d an ls of one of these root shares here, I got 764 lines 
of…emissions, of which this section seems the most interesting to me:

1051  263166 [main] ls 4720 stat64: entering
  625  263791 [main] ls 4720 normalize_posix_path: src /p
  609  264400 [main] ls 4720 normalize_posix_path: /p = normalize_posix_path 
(/p)
  630  265030 [main] ls 4720 mount_info::conv_to_win32_path: conv_to_win32_path 
(/p)
  653  265683 [main] ls 4720 mount_info::cygdrive_win32_path: src '/p', dst 
'P:\'
  668  266351 [main] ls 4720 set_flags: flags: binary (0x2)
  674  267025 [main] ls 4720 mount_info::conv_to_win32_path: src_path /p, dst 
P:\, flags 0x4022, rc 0
 1168  268193 [main] ls 4720 symlink_info::check: 0x0 = NtCreateFile (\??\P:\)
10641  278834 [main] ls 4720 symlink_info::check_reparse_point: 
NtFsControlFile(FSCTL_GET_REPARSE_POINT) failed, 0xC275
  839  279673 [main] ls 4720 symlink_info::check: not a symlink
 1049  280722 [main] ls 4720 symlink_info::check: 0 = symlink.check(P:\, 
0x24B620) (0x4022)
  655  281377 [main] ls 4720 path_conv::check: this->path(P:\), has_acls(0)
  640  282017 [main] ls 4720 stat_worker: got 5 error from path_conv
  757  282774 [main] ls 4720 __set_errno: int stat_worker(path_conv&, 
stat*):1933 setting errno 5
  714  283488 [main] ls 4720 stat_worker: -1 = (\??\P:\,0x600042080)

Why errno 5 from path_conv?

Yes, I am one of those evil people who edit cygdrive out of /etc/fstab so I can 
use /x, where x=drive letter. :)
--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Error accessing mapped drive >2TB?

[Andrey Repin]

Greetings, Warren Young!

>> The only thing I can think of is that the 2nd drive is >2TB.

> The same thing happens here with a 3.1 TB Fusion drive and a 500 GB
> external drive, so no, I don’t think the volume size is the real issue.

> I assume you are seeing the same thing that I am, that Explorer shows the
> root contents of both drives just fine?

> When I strace’d an ls of one of these root shares here, I got 764 lines
> of…emissions, of which this section seems the most interesting to me:

> 1051  263166 [main] ls 4720 stat64: entering
>   625  263791 [main] ls 4720 normalize_posix_path: src /p
>   609  264400 [main] ls 4720 normalize_posix_path: /p = normalize_posix_path 
> (/p)
>   630  265030 [main] ls 4720 mount_info::conv_to_win32_path: 
> conv_to_win32_path (/p)
>   653  265683 [main] ls 4720 mount_info::cygdrive_win32_path: src '/p', dst 
> 'P:\'
>   668  266351 [main] ls 4720 set_flags: flags: binary (0x2)
>   674  267025 [main] ls 4720 mount_info::conv_to_win32_path: src_path /p, dst 
> P:\, flags 0x4022, rc 0
>  1168  268193 [main] ls 4720 symlink_info::check: 0x0 = NtCreateFile (\??\P:\)
> 10641  278834 [main] ls 4720 symlink_info::check_reparse_point:
> NtFsControlFile(FSCTL_GET_REPARSE_POINT) failed, 0xC275
>   839  279673 [main] ls 4720 symlink_info::check: not a symlink
>  1049  280722 [main] ls 4720 symlink_info::check: 0 = symlink.check(P:\, 
> 0x24B620) (0x4022)
>   655  281377 [main] ls 4720 path_conv::check: this->path(P:\), has_acls(0)
>   640  282017 [main] ls 4720 stat_worker: got 5 error from path_conv
>   757  282774 [main] ls 4720 __set_errno: int stat_worker(path_conv&, 
> stat*):1933 setting errno 5
>   714  283488 [main] ls 4720 stat_worker: -1 = (\??\P:\,0x600042080)

> Why errno 5 from path_conv?

> Yes, I am one of those evil people who edit cygdrive out of /etc/fstab so I 
> can use /x, where x=drive letter. :)

That's an interesting point... can you walk me through your setup, so I could
try to recreate the issue?
Or is this Mac-specific?


-- 
With best regards,
Andrey Repin
Tuesday, September 15, 2015 06:45:22

Sorry for my terrible english...

Error accessing mapped drive >2TB?

[Nem W Schlecht]

I have 2 filesystem shares from a Mac that I'm mapping on Windows 7 as
my 'S' and 'T' drives.  On the Mac, both directories are owned by the
same user/group and shared the same.  One drive is 80GB (S) and the
other drive is 3TB (T).  I can access /cydrive/s just fine.  But
/cygdrive/t shows up with "?" for user and group and gives me
"Input/output error" when I try to cd or ls it.

When I try via a UNC path, I get the same thing - 80GB works, 3TB does not.

The only thing I can think of is that the 2nd drive is >2TB.  Cygwin
can access a 5TB local drive just fine, but won't access this 3TB
remote drive.  I can access the 3TB drive just fine from Windows,
though.  Is there a limit of some sort of limit when mounting a drive?
 Any settings I can tweak?

-- 
Nem W Schlecht

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple