Re: [Tutor] Case Insensitive Globing
On 20/05/2019 09:49, Alan Gauld via Tutor wrote: > On 19/05/2019 19:19, Alan Gauld via Tutor wrote: > ... >> So I always end up with two copies - the original file and the >> edited version. > I forgot I had moved all my photos onto my NAS box > and then mounted that in my pictures library under > Windows 10. On closer study it was slightly more complex. In fact, the errant editor is actually a Linux utility program that I only rarely use but which uses the same NAS folder as is used by the Windows box(obviously, that's why it's on the NAS!) Now, because I do 90% of my photo editing on Windows (Affinity Photo really is superb!), I only noticed the duplicate file names there and so assumed it was a Windows thing. So,to summarize: I copy the files from the camera to the (Linux based) NAS. I edit the file on my Linux PC (which saves with the .jpg extension) Then I open the folder for serious editing on Windows and see two filenames differing only in the case of the extension. No NTFS involved. Just memory loss from a befuddled senior... :-( -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Case Insensitive Globing
On 19/05/2019 19:19, Alan Gauld via Tutor wrote: > Hmm, odd. My NTFS filesystems on Windows all appear to be case > sensitive. For example I have a photo editor that saves its files > with a jpg extension but the files from my camera all end in JPG. > So I always end up with two copies - the original file and the > edited version. Amazing how you forget the simple things over time. I forgot I had moved all my photos onto my NAS box and then mounted that in my pictures library under Windows 10. Because you access them via the Library feature I'd completely forgotten they were on the NAS. So the filesystem is really ext3 and I'm assuming that's why the double names exist... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Case Insensitive Globing
On 5/19/19, Alan Gauld via Tutor wrote: > > Hmm, odd. My NTFS filesystems on Windows all appear to be case > sensitive. For example I have a photo editor that saves its files > with a jpg extension but the files from my camera all end in JPG. > So I always end up with two copies - the original file and the > edited version. > > I'm not aware of having done anything to cause that. > More investigation required I think... Maybe you have Explorer configured to hide file extensions, and you have one file named "filename.JPG" and another named "filename.jpg.JPG". On a related note, the new support for case-sensitive directories in Windows 10 can lead to an issue when running commands. Shells use the PATHEXT environment variable to supply a list of default extensions when searching PATH for a command. These are usually upper case, so if we run "script" and it searches for "script.PY", it won't find "script.py" in a case-sensitive directory. The same applies to Python's shutil.which(). Regarding this topic, glob.py and fnmatch.py are unreliable in Windows for case-sensitive NTFS directories. Sometimes they rely on regular expressions and os.path.normcase (for ntpath.normcase, this replaces slash with backslash and converts to lowercase), and sometimes they rely on stat-based functions such as os.path.lexists, which will be case sensitive. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Case Insensitive Globing
On 19/05/2019 01:37, Steven D'Aprano wrote: > That's not quite right -- case sensitivity of the OS isn't important, > case sensitivity of the *file system* is. And the standard file system > on Mac OS, HFS+, defaults to case-preserving but case-insensitive. > > (There is an option to turn case-sensitivity off, but hardly anyone uses > it because too many applications break.) Oops, my mistake. Almost all my MacOS knowledge is based on my ancient iBook. Having had a look, it appears it's formatted in UFS rather than HFS+ so I assume I must have done something to force that when I first installed the system back in 2001... It certainly appears to be case sensitive but life is too short for me to do extensive testing right now! > Fun fact: even NTFS supports a case-sensitive mode! But again, hardly > anyone uses it. Hmm, odd. My NTFS filesystems on Windows all appear to be case sensitive. For example I have a photo editor that saves its files with a jpg extension but the files from my camera all end in JPG. So I always end up with two copies - the original file and the edited version. I'm not aware of having done anything to cause that. More investigation required I think... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Case Insensitive Globing
On 5/18/19, Steven D'Aprano wrote: > > That means that, like Windows file systems FAT and NTFS, file names are > case-insensitive: files "Foo", "foo" and "FOO" are all considered the > same. But unlike Windows, the file system preserves the case of the file > as you created it, so if you created it as "foO" that's how it will be > recorded on the disk rather than normalizied to "foo". NTFS and FAT32 are case preserving. > Fun fact: even NTFS supports a case-sensitive mode! But again, hardly > anyone uses it. There's nothing inherent in the design of NTFS that prevents case-sensitive names in a directory. It's strictly a function of the OS and filesystem driver. On non-Windows platforms, an NTFS driver can create names that differ only in case. An example is the Linux ntfs-3g driver, which allows this unless the "windows_names" option is set. In a Windows system, the NT object namespace is case sensitive, and overriding a create or open to be case insensitive requires the flag OBJ_CASE_INSENSITIVE. The Windows API uses this flag by default in the file and registry APIs. However, beginning with Windows XP, the kernel takes it a step further. Regardless of the user-mode subsytem, it forces this flag for object types that are defined as case insensitive, such as Object Manager "Directory" and "SymbolicLink" objects (used in the root object namespace, akin to the root filesystem in Unix), Configuration Manager "Key" objects (registry), and I/O manager "Device" and "File" objects. Prior to Windows XP, it was possible to force a CreateFile or FindFirstFileEx call to be case sensitive, respectively via the flags FILE_FLAG_POSIX_SEMANTICS and FIND_FIRST_EX_CASE_SENSITIVE. Internally, this is implemented by *omitting* the OBJ_CASE_INSENSITIVE flag that the Windows API usually includes for device and file operations. This didn't change in XP, but, as mentioned above, it's impotent now since the kernel itself adds the flag for case-insensitive object types. The only way to change this is to restart the system after zeroing the "obcaseinsensitive" value in the "Session Manager\kernel" registry key. That's not tenable in practice, so we have to just accept that device and file names are case insensitive. That said, in Windows 10, the Windows Subsystem for Linux requires a way to change the filesystem default to case sensitive for individual directories. If a directory is case sensitive, then create, open, and control operations in or on it always ignore OBJ_CASE_INSENSITIVE. This directory attribute can be queried and set with a new file information class in the NT API named "FileCaseSensitiveInformation", or on the command line via `fsutil file queryCaseSensitiveInfo ` and `fsutil file setCaseSensitiveInfo [enable|disable]`. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Case Insensitive Globing
On Sun, May 19, 2019 at 10:37:56AM +1000, Steven D'Aprano wrote:
> That's not quite right -- case sensitivity of the OS isn't important,
> case sensitivity of the *file system* is. And the standard file system
> on Mac OS, HFS+, defaults to case-preserving but case-insensitive.
>
> (There is an option to turn case-sensitivity off, but hardly anyone uses
> it because too many applications break.)
Oops, I meant to say there is an option to turn case-sensitivity ON, but
hardly anyone uses it.
By the way, Linux users can experiment with case-insensitive file
systems provided they have privileges to mount file systems. Here's an
example. At the shell:
# allocate 100K for a (tiny) file system
dd if=/dev/zero of=fat.fs bs=1024 count=100
# format as FAT-12 with an optional label
/sbin/mkfs.vfat -n "FAT image" fat.fs
# mount it and create some files
sudo mount -o loop fat.fs /mnt
sudo touch /mnt/{lower.txt,UPPER.TXT,mIxEd.TxT}
Now you have a file system and some empty files to experiment with.
--
Steven
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Case Insensitive Globing
On Sat, May 18, 2019 at 11:52:29AM +0100, Alan Gauld via Tutor wrote: > On 18/05/2019 03:14, Richard Damon wrote: > > > The same directory, running the same program under Mac OS X, which also > > is a case insensitive file system, > > That is your mistake. Darwin, the core of the MacOS X system > is a version of BSD Unix and like all Unix OS is very much > case sensitive. That's not quite right -- case sensitivity of the OS isn't important, case sensitivity of the *file system* is. And the standard file system on Mac OS, HFS+, defaults to case-preserving but case-insensitive. (There is an option to turn case-sensitivity off, but hardly anyone uses it because too many applications break.) https://stackoverflow.com/questions/4706215/mac-os-x-how-to-determine-if-filesystem-is-case-sensitive https://apple.stackexchange.com/questions/71357/how-to-check-if-my-hd-is-case-sensitive-or-not That means that, like Windows file systems FAT and NTFS, file names are case-insensitive: files "Foo", "foo" and "FOO" are all considered the same. But unlike Windows, the file system preserves the case of the file as you created it, so if you created it as "foO" that's how it will be recorded on the disk rather than normalizied to "foo". Fun fact: even NTFS supports a case-sensitive mode! But again, hardly anyone uses it. Likewise, if you are using Samba on Unix/Linux, you can have case-insensitive file operations on an underlying case-sensitive file system. > Some of the GUI tools in MacOS X may work as if they were case > insensitive (presumably for backwards compatibility to MacOS 9) > but the underlying OS is case sensitive and all the Terminal > type tools and commands, including Python, follow suit. I don't think that is correct: https://apple.stackexchange.com/questions/22297/is-bash-in-osx-case-insensitive -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Case Insensitive Globing
On 5/17/19 8:14 PM, Richard Damon wrote: > I am working on a program to process some files created by an old > windows program that created it files with varying case with a python > program. > > Using glob.glob on Windows seems to ignore the case, and find all the > matching files. > > The same directory, running the same program under Mac OS X, which also > is a case insensitive file system, is only files that match the case of > the glob, and is missing many of the files that were found under windows. > > Is there an easy was to make glob match files as a case insensitive manner? > > Or a simple way to do this with something else. > > I am trying to do something like: > > for file in glob.glob(pattern): processfile(file) > here's a little snippet I've used in the past - uses fnmatch to help translate the pattern into a regular expression: import os, re, fnmatch def findfiles(pattern, path='.'): rule = re.compile(fnmatch.translate(pattern), re.IGNORECASE) return [name for name in os.listdir(path) if rule.match(name)] ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Case Insensitive Globing
On 18/05/2019 03:14, Richard Damon wrote:
> The same directory, running the same program under Mac OS X, which also
> is a case insensitive file system,
That is your mistake. Darwin, the core of the MacOS X system
is a version of BSD Unix and like all Unix OS is very much
case sensitive.
Some of the GUI tools in MacOS X may work as if they were case
insensitive (presumably for backwards compatibility to MacOS 9)
but the underlying OS is case sensitive and all the Terminal
type tools and commands, including Python, follow suit.
> Is there an easy was to make glob match files as a case insensitive manner?
Depends on what you mean by easy :-)
You could include both upper and lower case letters in the search:
glob.glob("[aA][mM][iI][xX][eE][dD][nN][aA][mM][eE]")
And you could write a helper function to generate the search strings.
But personally I'd probably just use listdir and compare with
my search string expressed as a regex.
for fname in os.listdir('.'):
if re.match("aregex", fname)
...
Another option might be to use fnmatch against the uppercase
version of the name
for fname in os.listdir('.'):
if fnmatch(pattern, fname.upper())
...
HTH
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
