Re: Unable to open files including Korean names
On Wed, 2004-06-23 21:48:38 -0400, Pierre A. Humblet wrote: Hello, the problem you discovered should now be fixed in the snapshots, e.g. http://cygwin.com/snapshots/cygwin1-20040623.dll.bz2 Could you try it and post the result to the list? It may be necessry to set the codepage in CYGWIN. Yes. Every Korean path works fine now with the snapshot DLL. (I haven't set any codepage in CYGWIN, as before.) Thanks for your help. No problem. Thanks for the fix. -- | Jaeho Shin [EMAIL PROTECTED] | http://netj.org/ System Programmers' Association for Researching Computer Systems Division of Computer Science, Department of EECS, KAIST pgpkCpp6Wp6of.pgp Description: PGP signature
Re: Unable to open files including Korean names
On Mon, 2004-06-14 16:19:17 -0400, Pierre A. Humblet wrote: Pierre A. Humblet wrote: I am surprised that ls -l works, as it also calls NtCreateFile. Could you strace it too? Actually the best would be to have a Korean directory with one file in it and to strace -o strace.txt ls -l dir dir/file Here you go. (gzip'ed it due to big size again) I ran strace -o strace-ls-10.txt ls -l /testfile. It's quite a troublesome job to shut all those process running from 1.5.9 and then upgrade to 1.5.10 for strace'ing, and again back to 1.5.9 launching everything back. However, I doesn't matter if I can help solving this problem. :-) -- | Jaeho Shin [EMAIL PROTECTED] | http://netj.org/ System Programmers' Association for Researching Computer Systems Division of Computer Science, Department of EECS, KAIST strace-ls-10.txt.gz Description: Binary data pgpdNNnJ7pgtu.pgp Description: PGP signature
Re: Unable to open files including Korean names
On Mon, 2004-06-14 14:02:48 +0200, Corinna Vinschen wrote: On Jun 14 20:12, Jaeho Shin wrote: On Sun, Jun 13, 2004 at 02:55:23PM -0400, Pierre A. Humblet ([EMAIL PROTECTED]) wrote: Please don't quote raw email addresses in a reply! http://cygwin.com/acronyms/index.html#PCYMTNQREAIYR Oh, sorry about that. Fixed it now. Your original mail stated I did some test and found out that ``every file whose path includes Korean characters weren't openable.'' Still, I could move arround those paths from my shell, and get the file/directory listings with ls normally. I could still create directories with Korean names. That would indicate that it's not the path handling code, but the use of NtCreateFile in 1.5.10. Can you ls -l those paths? Sorry about my complaints against you. I misunderstood ``path handling'' as everything done when openning files with paths in Cygwin. Yes, I can ls -l them, so the problem is really in the file openning part, not the path handling. :) I'm under the impression, the important difference is that Cygwin does the MultiByte to WideChar conversion instead of Windows itself. Could you please add codepage:oem to your CYGWIN environment variable and try again? Does that change the behaviour? Nothing changed. I ran strace -o strace-ls-10-codepage_oem.txt ls -l /testfile, and strace -o strace-touch-10-codepage_oem.txt touch file. Both are attached and gzip'ed due to size limit. -- | Jaeho Shin [EMAIL PROTECTED] | http://netj.org/ System Programmers' Association for Researching Computer Systems Division of Computer Science, Department of EECS, KAIST strace-ls-10-codepage_oem.txt.gz Description: Binary data strace-touch-10-codepage_oem.txt.gz Description: Binary data pgpEYeIPDOw7F.pgp Description: PGP signature
Re: Unable to open files including Korean names
On Tue, 2004-06-15 09:14:22 -0400, Pierre A. Humblet wrote:
Thanks. Nothing conclusive.
Could you compile and run the following one line program?
#include windows.h
#include stdio.h
main()
{
printf(AreFileApisANSI %d\n, AreFileApisANSI());
}
Compile it with
gcc -mno-cygwin try_ansi.c
With the -mno-cygwin, the value of CYGWIN=codepage:oem
shouldn't matter. When compiled without that switch
codepage:oem or codepage:ansi should matter.
Running on 1.5.9 is OK.
Here's the result:
$ gcc -mno-cygwin try_ansi.c
$ ./a.exe
AreFileApisANSI 1
$
Also, the Korean directory name has numerical value
~ od -x xx.txt
000 d1c7 dbb1
Do you know what encoding that is? Is it Unicode or UTF8?
If it is UTF8, do you know what the Unicode values should be?
Well, that's in EUC-KR and CP949. CP949 has some more characters
defined in the empty areas of EUC-KR. The directory name I used,
``'', which is pronounced ``hangeul'' and means Korean (written
language) in Korean, is consisted of two characters:
U+D55C: Hangul syllable Hieuh A Nieun,
U+AE00: Hangul syllable Kiyeok Eu Rieul.
(Perhaps, you may be able to find it from Windows charmap)
Neither character is in CP949's extension, so they have identical values
in both EUC-KR and CP949 encoding.
Yes, you gave me the identical numerical value I use.
Running, `echo -n | od -x -` tells me:
000 d1c7 dbb1
Now, `echo -n | iconv -f euc-kr -t utf-8 | od -x -` tells me:
000 95ed ea9c 80b8
Yes, it's in EUC-KR (or CP949 equivalently in this case). I don't use
unicode environment yet. Actually, I don't know how to change encoding
from Windows. Korean version of Windows just uses CP949 as default.
Looks like od's output is in little-endian. This identifies them as
U+D55C and U+AE00, `echo -n | iconv -f euc-kr -t ucs-2 | od -x -`:
000 5cd5 00ae
Thanks for your help
My pleasure. :)
BTW, is there any reason you not sending your msgs to cygwin ML?
If not, I'll just keep Cc'ing to it.
--
| Jaeho Shin [EMAIL PROTECTED] | http://netj.org/
System Programmers' Association for Researching Computer Systems
Division of Computer Science, Department of EECS, KAIST
pgpvJCbfw6IZ9.pgp
Description: PGP signature
OT: Beeing 'endian' (RE: Unable to open files including Korean names)
From: Jaeho Shin
Sent: Tuesday, June 15, 2004 4:17 PM
Looks like od's output is in little-endian. This identifies them as
U+D55C and U+AE00, `echo -n XX | iconv -f euc-kr -t ucs-2 | od -x -`:
000 5cd5 00ae
A better way to explore things is by bypassing the endianness with
$ ... | od -t x1z
As soon as there is more than 8 bits bunched together,
you'll find the end(ian) grin?.
Another way might be something like this:
$ type -a od
od is a function
od ()
{
odargs=$@;
if [ -z $odargs ] || [ -f $odargs ]; then
odargs=-A x -w24 -t x1z $odargs;
fi;
command od $odargs;
unset odargs
}
od is /usr/bin/od
od is /bin/od
( -w24 above = $(echo $COLUMNS) = 112
/Hannu E K Nevalainen, B.Sc. EE - 59+16.37'N, 17+12.60'E --76--
--
inline short isLittleEndian(void) {
const short testvalue=1;
char *p=(char *) testvalue;
return (short)(*p!=0);
}
--
--END OF MESSAGE-- File: ATT00010.dat
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/
Re: Unable to open files including Korean names
On Sun, Jun 13, 2004 at 12:15:41PM -0400, Gregg C Levine ([EMAIL PROTECTED]) wrote: As for your problem Jaeho Shin, I think the switching of binaries for the Cygwin DLLs should fix things. At least temporarily. Yeah, I already switched to 1.5.9-1. Everything's fine here. Trying SP2 might help, but Gary's story about fork() made me afraid of trying it, and besides I don't have much time to play with it. However, thanks for the advice. :-) -- | Jaeho Shin [EMAIL PROTECTED] | http://netj.org/ System Programmers' Association for Researching Computer Systems Division of Computer Science, Department of EECS, KAIST pgpQ65OghhUOS.pgp Description: PGP signature
Re: Unable to open files including Korean names
On Jun 14 20:12, Jaeho Shin wrote: On Sun, Jun 13, 2004 at 02:55:23PM -0400, Pierre A. Humblet ([EMAIL PROTECTED]) wrote: Please don't quote raw email addresses in a reply! http://cygwin.com/acronyms/index.html#PCYMTNQREAIYR Your original mail stated I did some test and found out that ``every file whose path includes Korean characters weren't openable.'' Still, I could move arround those paths from my shell, and get the file/directory listings with ls normally. I could still create directories with Korean names. That would indicate that it's not the path handling code, but the use of NtCreateFile in 1.5.10. Can you ls -l those paths? Sorry about my complaints against you. I misunderstood ``path handling'' as everything done when openning files with paths in Cygwin. Yes, I can ls -l them, so the problem is really in the file openning part, not the path handling. :) I'm under the impression, the important difference is that Cygwin does the MultiByte to WideChar conversion instead of Windows itself. Could you please add codepage:oem to your CYGWIN environment variable and try again? Does that change the behaviour? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Co-Project Leader mailto:[EMAIL PROTECTED] Red Hat, Inc. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Unable to open files including Korean names
Can you run some very simple program demonstrating the problem under
strace, something like
strace -o strace-10.txt touch some short Korean path
both under 1.5.10 and 1.5.9?
Sure. strace-{10,9}.txt are attached, and strace-env.txt, too.
It's the output of ``env'' where I ran strace.
1.5.10 fails after calling NtCreateFile, as expected.
I agree with Corinna's explanation.
374 38792 [main] touch 4888 seterrno_from_win_error:
/netrel/src/cygwin-1.5.10-3/winsup/cygwin/fhandler.cc:651 windows error 123
53 38910 [main] touch 4888 geterrno_from_win_error: windows error 123 == errno 2
208 39118 [main] touch 4888 fhandler_base::open: 0 = fhandler_base::open
(D:\netj\tmp\ÇѱÛ.txt, 0x10C201)
The Windows file name (displayed in single byte chars) D:\netj\tmp\ÇѱÛ.txt
is the same is 1.5.9 and 1.5.10
Windows error 123 is:
The filename, directory name, or volume label syntax is incorrect.
I am surprised that ls -l works, as it also calls NtCreateFile.
Could you strace it too?
Pierre
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/
Re: Unable to open files including Korean names
On Sun, Jun 13, 2004 at 03:30:00AM +0900, Jaeho Shin ([EMAIL PROTECTED]) wrote: As I remember, there wasn't any problem like this before my update few days ago. Not sure, but since I remember the last time I had an update was near Apr 20, I should have been using cygwin dll 1.5.9 then. Yes. I just downgraded cygwin dll (and only cygwin dll) to 1.5.9-1, and now the problem is gone. Looks like 1.5.10 does have some problem handling non-ascii paths. I've checked the changes in 1.5.10 from the announce (at http://cygwin.com/ml/cygwin-announce/2004-05/msg00010.html) and found an entry related to this problem: - Simplify and speed up path handling. (Pierre Humblet) Perhaps it was TOO simplified? :-( I'll try examining the codes as soon as I get some free time. Or, maybe Pierre can fix the problem? -- | Jaeho Shin [EMAIL PROTECTED] | http://netj.org/ System Programmers' Association for Researching Computer Systems Division of Computer Science, Department of EECS, KAIST pgpGdekb62hlv.pgp Description: PGP signature
RE: Unable to open files including Korean names
Hello from Gregg C Levine Actually Gary, its called SP2 RC1, for XP. Its going through that phase, with SP1 RC2 due out towards the end of the month, and the actual SP2 going to the RTM phase so that its in time to be released by the 21 July, date. (If you can believe that.) Also, can you tell me, off list, where you got your copy? I was at an OEM meeting for that guy's operating system, yesterday. The meeting was held here in NYC. As for your problem Jaeho Shin, I think the switching of binaries for the Cygwin DLLs should fix things. At least temporarily. --- Gregg C Levine [EMAIL PROTECTED] The Force will be with you...Always. Obi-Wan Kenobi Use the Force, Luke. Obi-Wan Kenobi -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of GARY VANSICKLE Sent: Saturday, June 12, 2004 11:13 PM To: [EMAIL PROTECTED] Subject: RE: Unable to open files including Korean names I did some test and found out that ``every file whose path includes Korean characters weren't openable.'' Still, I could move arround those [snip] I just installed the XP SP2 preview or whatever they call it. One of the things in the very long list of things it claims to fix is some problem with... Japanese I think?... filenames. Perhaps this is in some way related to that... As I remember, there wasn't any problem like this before my update few days ago. [snip] ...or possibly not. -- Gary R. Van Sickle -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Unable to open files including Korean names
At 12:02 AM 6/14/2004 +0900, Jaeho Shin wrote: On Sun, Jun 13, 2004 at 03:30:00AM +0900, Jaeho Shin ([EMAIL PROTECTED]) wrote: As I remember, there wasn't any problem like this before my update few days ago. Not sure, but since I remember the last time I had an update was near Apr 20, I should have been using cygwin dll 1.5.9 then. Yes. I just downgraded cygwin dll (and only cygwin dll) to 1.5.9-1, and now the problem is gone. Looks like 1.5.10 does have some problem handling non-ascii paths. I've checked the changes in 1.5.10 from the announce (at http://cygwin.com/ml/cygwin-announce/2004-05/msg00010.html) and found an entry related to this problem: - Simplify and speed up path handling. (Pierre Humblet) Perhaps it was TOO simplified? :-( I'll try examining the codes as soon as I get some free time. Or, maybe Pierre can fix the problem? Do you see anything resembling what's described in http://cygwin.com/ml/cygwin/2004-05/msg00659.html What character representation are you using? Is a character represented by one or by two (or more) bytes? Your original mail stated I did some test and found out that ``every file whose path includes Korean characters weren't openable.'' Still, I could move arround those paths from my shell, and get the file/directory listings with ls normally. I could still create directories with Korean names. That would indicate that it's not the path handling code, but the use of NtCreateFile in 1.5.10. Can you ls -l those paths? Can you run some very simple program demonstrating the problem under strace, something like strace -o strace-10.txt touch some short Korean path both under 1.5.10 and 1.5.9? Pierre -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
[SEMI-OT]: XP SP2 (was: RE: Unable to open files including Korean names)
Hello from Gregg C Levine Actually Gary, its called SP2 RC1, for XP. Its going through that phase, with SP1 RC2 due out towards the end of the month, and the actual SP2 going to the RTM phase so that its in time to be released by the 21 July, date. (If you can believe that.) I of course cannot. ;-) Also, can you tell me, off list, where you got your copy? I was at an OEM meeting for that guy's operating system, yesterday. The meeting was held here in NYC. Well, I'll reply to both in case anybody else is interested. It's available for download right off of msdn.microsoft.com, 200MB plus tax. I've only been using it for a few days now, but everything Cygwin that I use (gcc et al, perl, rxvt, etc) seems to still work fine. There's all kinds of massive security-related changes that I'm guessing could raise hell with a few programs though. Helpful hint though to anybody brave/foolish enough to install it: don't bother reporting any Cygwin problems to Cygwin until SP2 is actually released; doing so will serve no purpose. Probably can't hurt to send bug reports to MS though. One change that they're making could be a gigantic problem for Cygwin: Finally, after all these years, Windows will be keeping the hot side hot and the cool side cool, or in non-McDLT-speak, tagging program and data memory areas and preventing code execution in data areas. This is a good (fantabulous?) if belated feature, but if I understand fork() correctly (which I don't), doesn't fork() allocate memory, copy some data and code over, and jump to the code? From what I read I think this would no longer work. -- Gary R. Van Sickle -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: Unable to open files including Korean names
I did some test and found out that ``every file whose path includes Korean characters weren't openable.'' Still, I could move arround those [snip] I just installed the XP SP2 preview or whatever they call it. One of the things in the very long list of things it claims to fix is some problem with... Japanese I think?... filenames. Perhaps this is in some way related to that... As I remember, there wasn't any problem like this before my update few days ago. [snip] ...or possibly not. -- Gary R. Van Sickle -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
