Unable to open files including Korean names
I've updated to cygwin dll 1.5.10 a few days ago.
(I'm using Cygwin with Windows XP SP1, Korean version.)
And today I found a problem with the accessing files.
I was trying to use rsync to sync some of my files to another place,
but rsync was telling me those files had vanished, like:
file has vanished: /home/netj/image/SPARCS/2004//DSC00137.JPG
I thought it was a permission problem at the first time,
but I couldn't even chmod or open the file from cygwin environment. :(
I did some test and found out that ``every file whose path includes
Korean characters weren't openable.'' Still, I could move arround those
paths from my shell, and get the file/directory listings with ls
normally. I could still create directories with Korean names.
$ cd ~/tmp
$ date ascii
$ cat ascii
Sun Jun 13 03:23:04 2004
$ date
-bash: : No such file or directory
$ mkdir ascii
$ date ascii/date
$ cat ascii/date
Sun Jun 13 03:24:07 2004
$ mkdir .d
$ date .d/date
-bash: .d/date: No such file or directory
I tried playing with codepage:* in CYGWIN, and LANG variables, but
nothing helped.
As I remember, there wasn't any problem like this before my update few
days ago. Not sure, but since I remember the last time I had an update
was near Apr 20, I should have been using cygwin dll 1.5.9 then.
Attached my result of cygcheck -s -v -r.
--
| Jaeho Shin [EMAIL PROTECTED] | http://netj.org/
System Programmers' Association for Researching Computer Systems
Division of Computer Science, Department of EECS, KAIST
Cygwin Configuration Diagnostics
Current System Time: Sun Jun 13 03:17:53 2004
Windows XP Professional Ver 5.1 Build 2600 Service Pack 1
Path: C:\cygwin\usr\local\bin
C:\cygwin\bin
C:\cygwin\bin
C:\cygwin\usr\X11R6\bin
C:\cygwin\bin
C:\cygwin\usr\X11R6\bin
C:\cygwin\opt\gnome2\bin
c:\TeX\texmf\miktex\bin\
c:\Program Files\j2sdk1.4.2\bin
c:\Program Files\gs\gs8.00\bin
c:\Program Files\emacs\bin
c:\Program Files\Vim\vim62
c:\WINDOWS\system32
c:\WINDOWS
c:\WINDOWS\System32\Wbem
c:\Program Files\Common Files\Adaptec Shared\System
D:\netj\bin
C:\cygwin\bin
C:\cygwin\usr\X11R6\bin
C:\cygwin\opt\gnome2\bin
c:\TeX\texmf\miktex\bin\
c:\Program Files\j2sdk1.4.2\bin
c:\Program Files\gs\gs8.00\bin
c:\Program Files\emacs\bin
c:\Program Files\Vim\vim62
c:\WINDOWS\system32
c:\WINDOWS
c:\WINDOWS\System32\Wbem
c:\Program Files\Common Files\Adaptec Shared\System
Output from C:\cygwin\bin\id.exe (nontsec)
UID: 1003(netj) GID: 513()
513()
Output from C:\cygwin\bin\id.exe (ntsec)
UID: 1003(netj) GID: 513()
513()544(Administrators)
545(Users)
SysDir: C:\WINDOWS\System32
WinDir: C:\WINDOWS
CYGWIN = `ntsec binmode'
HOME = `D:\netj'
MAKE_MODE = `unix'
PWD = `/home/netj'
USER = `netj'
ALLUSERSPROFILE = `D:\Documents and Settings\All Users'
APPDATA = `D:\netj\.AppData'
CLIENTNAME = `Console'
COMMONPROGRAMFILES = `C:\Program Files\Common Files'
COMPUTERNAME = `SAB'
COMSPEC = `C:\WINDOWS\system32\cmd.exe'
CVS_RSH = `/bin/ssh'
DISPLAY = `localhost:0.0'
EDITOR = `vim'
HOMEDRIVE = `D:'
HOMEPATH = `\netj'
HOSTNAME = `sab'
INCLUDE = `C:\Program Files\Microsoft Visual Studio .NET\FrameworkSDK\include\'
INFOPATH =
`/usr/local/info:/usr/info:/usr/share/info:/usr/autotool/devel/info:/usr/autotool/stable/info:'
LANG = `ko_KR.eucKR'
LESSBINFMT = `*n%c'
LIB = `C:\Program Files\Microsoft Visual Studio .NET\FrameworkSDK\Lib\'
LOGNAME = `netj'
LOGONSERVER = `\\SAB'
MANPATH = `/usr/man:/usr/share/man:/usr/X11R6/man:/usr/ssl/man'
NUMBER_OF_PROCESSORS = `1'
OLDPWD = `/home/netj'
OS = `Windows_NT'
PAGER = `less'
PATHEXT = `.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH'
PKG_CONFIG_PATH = `/usr/X11R6/lib/pkgconfig'
PROCESSOR_ARCHITECTURE = `x86'
PROCESSOR_IDENTIFIER = `x86 Family 6 Model 9 Stepping 5, GenuineIntel'
PROCESSOR_LEVEL = `6'
PROCESSOR_REVISION = `0905'
PROGRAMFILES = `C:\Program Files'
PS1 = `[0;37m([0;31mnetj[0;[EMAIL PROTECTED];32mtty2[0;[EMAIL
PROTECTED];34msab[0;37m)-(\D{[0;36m%Y-%m-%d [0;35m%a}
[0m\t[0;37m)---([0;33m~[0;37m)[0m\r\n(\j:\#:$?)\$
'
RSYNC_RSH = `ssh'
SESSIONNAME = `Console'
SHLVL = `2'
SSH_AGENT_PID = `1980'
SSH_AUTH_SOCK = `/tmp/ssh-oZrfIw1908/agent.1908'
SYSTEMDRIVE = `C:'
SYSTEMROOT = `C:\WINDOWS'
TEMP = `D:\tmp'
TERM = `xterm'
TERMCAP = `xterm-r6|xterm|xterm X11R6
version:am:km:mi:ms:xn:co#80:it#8:li#24:AL=\E[%dL:DC=\E[%dP:DL=\E[%dM:DO=\E[%dB:LE=\E[%dD:RI=\E[%dC:UP=\E[%dA:ae=^O:al=\E[L:as=^N:bl=^G:cd=\E[J:ce=\E[K:cl=\E[H\E[2J:cm=\E[%i%d;%dH:cr=^M:cs=\E[%i%d;%dr:ct=\E[3g:dc=\E[P:dl=\E[M:do=^J:ei=\E[4l:ho=\E[H:im=\E[4h:is=\E7\E[r\E[m\E[?7h\E[?1;3;4;6l\E[4l\E8\E:k1=\EOP:k2=\EOQ:k3=\EOR:k4=\EOS:k5=\E[15~:k6=\E[17~:k7=\E[18~:k8=\E[19~:k9=\E[20~:kD=\E[3~:kI=\E[2~:kN=\E[6~:kP=\E[5~:kd=\EOB:ke=\E[?1l\E:kh=\E[1~:kl=\EOD:kr=\EOC:ks=\E[?1h\E=:ku
Re: Unable to open files including Korean names
On Sun, Jun 13, 2004 at 03:30:00AM +0900, Jaeho Shin ([EMAIL PROTECTED]) wrote: As I remember, there wasn't any problem like this before my update few days ago. Not sure, but since I remember the last time I had an update was near Apr 20, I should have been using cygwin dll 1.5.9 then. Yes. I just downgraded cygwin dll (and only cygwin dll) to 1.5.9-1, and now the problem is gone. Looks like 1.5.10 does have some problem handling non-ascii paths. I've checked the changes in 1.5.10 from the announce (at http://cygwin.com/ml/cygwin-announce/2004-05/msg00010.html) and found an entry related to this problem: - Simplify and speed up path handling. (Pierre Humblet) Perhaps it was TOO simplified? :-( I'll try examining the codes as soon as I get some free time. Or, maybe Pierre can fix the problem? -- | Jaeho Shin [EMAIL PROTECTED] | http://netj.org/ System Programmers' Association for Researching Computer Systems Division of Computer Science, Department of EECS, KAIST pgpGdekb62hlv.pgp Description: PGP signature
Re: Unable to open files including Korean names
On Sun, Jun 13, 2004 at 12:15:41PM -0400, Gregg C Levine ([EMAIL PROTECTED]) wrote: As for your problem Jaeho Shin, I think the switching of binaries for the Cygwin DLLs should fix things. At least temporarily. Yeah, I already switched to 1.5.9-1. Everything's fine here. Trying SP2 might help, but Gary's story about fork() made me afraid of trying it, and besides I don't have much time to play with it. However, thanks for the advice. :-) -- | Jaeho Shin [EMAIL PROTECTED] | http://netj.org/ System Programmers' Association for Researching Computer Systems Division of Computer Science, Department of EECS, KAIST pgpQ65OghhUOS.pgp Description: PGP signature
Re: Unable to open files including Korean names
On Mon, 2004-06-14 16:19:17 -0400, Pierre A. Humblet wrote: Pierre A. Humblet wrote: I am surprised that ls -l works, as it also calls NtCreateFile. Could you strace it too? Actually the best would be to have a Korean directory with one file in it and to strace -o strace.txt ls -l dir dir/file Here you go. (gzip'ed it due to big size again) I ran strace -o strace-ls-10.txt ls -l /testfile. It's quite a troublesome job to shut all those process running from 1.5.9 and then upgrade to 1.5.10 for strace'ing, and again back to 1.5.9 launching everything back. However, I doesn't matter if I can help solving this problem. :-) -- | Jaeho Shin [EMAIL PROTECTED] | http://netj.org/ System Programmers' Association for Researching Computer Systems Division of Computer Science, Department of EECS, KAIST strace-ls-10.txt.gz Description: Binary data pgpdNNnJ7pgtu.pgp Description: PGP signature
Re: Unable to open files including Korean names
On Mon, 2004-06-14 14:02:48 +0200, Corinna Vinschen wrote: On Jun 14 20:12, Jaeho Shin wrote: On Sun, Jun 13, 2004 at 02:55:23PM -0400, Pierre A. Humblet ([EMAIL PROTECTED]) wrote: Please don't quote raw email addresses in a reply! http://cygwin.com/acronyms/index.html#PCYMTNQREAIYR Oh, sorry about that. Fixed it now. Your original mail stated I did some test and found out that ``every file whose path includes Korean characters weren't openable.'' Still, I could move arround those paths from my shell, and get the file/directory listings with ls normally. I could still create directories with Korean names. That would indicate that it's not the path handling code, but the use of NtCreateFile in 1.5.10. Can you ls -l those paths? Sorry about my complaints against you. I misunderstood ``path handling'' as everything done when openning files with paths in Cygwin. Yes, I can ls -l them, so the problem is really in the file openning part, not the path handling. :) I'm under the impression, the important difference is that Cygwin does the MultiByte to WideChar conversion instead of Windows itself. Could you please add codepage:oem to your CYGWIN environment variable and try again? Does that change the behaviour? Nothing changed. I ran strace -o strace-ls-10-codepage_oem.txt ls -l /testfile, and strace -o strace-touch-10-codepage_oem.txt touch file. Both are attached and gzip'ed due to size limit. -- | Jaeho Shin [EMAIL PROTECTED] | http://netj.org/ System Programmers' Association for Researching Computer Systems Division of Computer Science, Department of EECS, KAIST strace-ls-10-codepage_oem.txt.gz Description: Binary data strace-touch-10-codepage_oem.txt.gz Description: Binary data pgpEYeIPDOw7F.pgp Description: PGP signature
Re: Unable to open files including Korean names
On Tue, 2004-06-15 09:14:22 -0400, Pierre A. Humblet wrote:
Thanks. Nothing conclusive.
Could you compile and run the following one line program?
#include windows.h
#include stdio.h
main()
{
printf(AreFileApisANSI %d\n, AreFileApisANSI());
}
Compile it with
gcc -mno-cygwin try_ansi.c
With the -mno-cygwin, the value of CYGWIN=codepage:oem
shouldn't matter. When compiled without that switch
codepage:oem or codepage:ansi should matter.
Running on 1.5.9 is OK.
Here's the result:
$ gcc -mno-cygwin try_ansi.c
$ ./a.exe
AreFileApisANSI 1
$
Also, the Korean directory name has numerical value
~ od -x xx.txt
000 d1c7 dbb1
Do you know what encoding that is? Is it Unicode or UTF8?
If it is UTF8, do you know what the Unicode values should be?
Well, that's in EUC-KR and CP949. CP949 has some more characters
defined in the empty areas of EUC-KR. The directory name I used,
``'', which is pronounced ``hangeul'' and means Korean (written
language) in Korean, is consisted of two characters:
U+D55C: Hangul syllable Hieuh A Nieun,
U+AE00: Hangul syllable Kiyeok Eu Rieul.
(Perhaps, you may be able to find it from Windows charmap)
Neither character is in CP949's extension, so they have identical values
in both EUC-KR and CP949 encoding.
Yes, you gave me the identical numerical value I use.
Running, `echo -n | od -x -` tells me:
000 d1c7 dbb1
Now, `echo -n | iconv -f euc-kr -t utf-8 | od -x -` tells me:
000 95ed ea9c 80b8
Yes, it's in EUC-KR (or CP949 equivalently in this case). I don't use
unicode environment yet. Actually, I don't know how to change encoding
from Windows. Korean version of Windows just uses CP949 as default.
Looks like od's output is in little-endian. This identifies them as
U+D55C and U+AE00, `echo -n | iconv -f euc-kr -t ucs-2 | od -x -`:
000 5cd5 00ae
Thanks for your help
My pleasure. :)
BTW, is there any reason you not sending your msgs to cygwin ML?
If not, I'll just keep Cc'ing to it.
--
| Jaeho Shin [EMAIL PROTECTED] | http://netj.org/
System Programmers' Association for Researching Computer Systems
Division of Computer Science, Department of EECS, KAIST
pgpvJCbfw6IZ9.pgp
Description: PGP signature
Re: Unable to open files including Korean names
On Wed, 2004-06-23 21:48:38 -0400, Pierre A. Humblet wrote: Hello, the problem you discovered should now be fixed in the snapshots, e.g. http://cygwin.com/snapshots/cygwin1-20040623.dll.bz2 Could you try it and post the result to the list? It may be necessry to set the codepage in CYGWIN. Yes. Every Korean path works fine now with the snapshot DLL. (I haven't set any codepage in CYGWIN, as before.) Thanks for your help. No problem. Thanks for the fix. -- | Jaeho Shin [EMAIL PROTECTED] | http://netj.org/ System Programmers' Association for Researching Computer Systems Division of Computer Science, Department of EECS, KAIST pgpkCpp6Wp6of.pgp Description: PGP signature
Re: setting windows environment variables
On Tue, 2004-07-20 15:18:34 -0500, Jason Joines wrote: I have a Cygwin BASH script which takes user input, uses ldapsearch, etc., to set a bunch of environment variables. I can then use these from bash with no problem. I would also like to make these environment variables available to the windows command interpreter. Is there anyway to do this? Try setx: http://www.microsoft.com/windows2000/techinfo/reskit/tools/existing/setx-o.asp You can change Windows environment variables from the command line, so that you can call setx and change them from bash scripts. It works excellent for me to share a single ssh-agent Windows-wide. :) You can use it only on NT/2000/XP, perhaps. I'm not sure what's available on Windows 9X/ME. Also, is there a BASH equivalent of the windows cmd start command that lets you start many processes without having to wait on each to finish? ``cygstart'' should be what you are looking for. -- | Jaeho Shin [EMAIL PROTECTED] | http://netj.org/ Research On Program Analysis System, Seoul National University pgpusjagi6Bsg.pgp Description: PGP signature
mutt not working with libiconv 1.8-3
I'm using mutt 1.4-1 and recently I've upgraded libiconv and stuffs to
1.8-3. After then, I couldn't start composing any mail in mutt. :(
mutt was spitting out some error messages like,
assertion errno == EISL || (BUGGY_ICONV ...) || errno == ENOENT
failure ../lib/sendlib.c:743
I forgot to capture the exact error msg, but it's easy to reproduce:
just press 'm' inside mutt.
Downgrading libiconv{,2} and libcharset to 1.8-2, everything became OK
as usual.
--
netj | Jaeho Shin | ; http://netj.org/
System Programmers' Association for Researching Computer Systems
Division of Computer Science, Department of EECS, KAIST
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/
Unicode in filenames support?
I'm having problem with accessing files that have Unicode in their
filenames.
1. I use Windows XP Korean version (so the codepage must be 949?).
2. I use iTunes to listen to my music.
3. Files in iTunes Library have filenames in the following format:
{Artist}/{Album}/{Track#} {Title}.mp3
where names inside braces are values from its ID3-tag.
4. Some of my mp3s have Japanese or Latin characters,
e.g. (Latin small letter e with acute).
In ID3-tags, those characters seem to be in UCS-2 encoding or so,
but not in CP949 or EUC-KR.
5. I want to rsync those files to my other Linux machine.
6. But rsync complains some files (whose name contains such
special/Unicode characters perhaps?) have vanished! :'(
With Windows Explorer, I can copy them to a Samba share (with utf-8
encoding) without any problem. However, from the Cygwin environment, it
seems that there is no way I can access those files.
I tried the mount -o managed option which escapes capitals and other
non-ascii characters in filenames. It wasn't a solution for me since
iTunes (not Cygwin) mainly manages the files.
Since I really want to use rsync, I hope Cygwin to be able to access
Unicode filenames. It would be great if I could mount a filesystem with
a charset or encoding specified. Is there any nice way already I can
solve this problem?
--
| Jaeho Shin [EMAIL PROTECTED] | http://netj.org/
Programming Research Laboratory, Seoul National University
signature.asc
Description: OpenPGP digital signature
