Re: [Pharo-dev] File path encoding

2018-04-16 Thread Sven Van Caekenberghe 
Actually, (from another thread) I think that it is related to how environment 
variables are accessed.

Consider:

$ FOO=benoît ./pharo Pharo.image eval 'OSEnvironment current associations' 
{'TERM_PROGRAM'->'Apple_Terminal'. 'TERM'->'xterm-256color'. 
'SHELL'->'/bin/bash'. 
'TMPDIR'->'/var/folders/sy/sndrtj9j1tq06j0lfnshmrl8gn/T/'. 
'FOO'->'benoît'. 
'Apple_PubSub_Socket_Render'->'/private/tmp/com.apple.launchd.uWk7pivcLT/Render'.
 'TERM_PROGRAM_VERSION'->'404'. 
'TERM_SESSION_ID'->'845BECCD-0AB0-4686-B7F9-3A0FF84BDCB7'. 'USER'->'sven'. 
'SSH_AUTH_SOCK'->'/private/tmp/com.apple.launchd.y5oCwdUyaG/Listeners'. 
'PATH'->'/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin:/opt/X11/bin'.
 'PWD'->'/tmp/benoît'. 'XPC_FLAGS'->'0x0'. 'XPC_SERVICE_NAME'->'0'. 
'HOME'->'/Users/sven'. 'SHLVL'->'2'. 'LOGNAME'->'sven'. 'LC_CTYPE'->'UTF-8'. 
'DISPLAY'->'/private/tmp/com.apple.launchd.lsgASYFiWW/org.macosforge.xquartz:0'.
 'SECURITYSESSIONID'->'186a9'. 'OLDPWD'->'/tmp/benoît'. 
'_'->'/tmp/benoît/pharo-vm/Pharo.app/Contents/MacOS/Pharo'. 
'__CF_USER_TEXT_ENCODING'->'0x1F5:0x0:0x0'}

As you can see, several values are wrongly decoded - and this is on macOS as 
well.

> On 16 Apr 2018, at 15:10, Sven Van Caekenberghe  wrote:
> 
> 
> 
>> On 16 Apr 2018, at 14:51, Alistair Grant  wrote:
>> 
>> Hi Sven,
>> 
>> On 16 April 2018 at 13:57, Sven Van Caekenberghe  wrote:
>>> 
>>> Technically, at the File (FilePlugin) level we seem to be able to handle 
>>> paths with strange characters, even on Windows, but somehow, somewhere a 
>>> bogus call is made during image startup.
>> 
>> I'm not going to have time to look at this in the next few days, sorry.
>> 
>> But I have a vague memory that some of the other primitives outside
>> FilePlugin don't add the "\\?\" prefix, so they possibly also don't do
>> wide character conversion.
> 
> Then I hope some other Windows user/developer can have a look. 
> Hell, I believe we got a few French people working on Pharo, n'est pas ?
> 
>> Cheers,
>> Alistair

Re: [Pharo-dev] File path encoding

2018-04-16 Thread Ben Coman 
>> On 16 Apr 2018, at 13:05, Ben Coman  wrote:
>>
>> On 16 April 2018 at 18:52, Ben Coman  wrote:
>>> On 16 April 2018 at 17:51, Sven Van Caekenberghe  wrote:
 Hi,

 Some people raise issues with file path encoding with non-ASCII
characters.

 The following works fine for me (macOS, Pharo 7, using new File class):

 "works"

 (FileLocator temp / 'foo' , 'txt') writeStreamDo: [ :out | out << 'Les
élèves Français' ; crlf ].

 (FileLocator temp / 'foo' , 'txt') readStreamDo: [ :in | in upToEnd ].

 "works"

 (FileLocator temp / 'benoît' , 'txt') writeStreamDo: [ :out | out <<
'Les élèves Français' ; crlf ].

 (FileLocator temp / 'benoît' , 'txt') readStreamDo: [ :in | in upToEnd
].

 "works"

 (FileLocator temp / 'benoît') ensureCreateDirectory.

 (FileLocator temp / 'benoît' / 'benoît' , 'txt') writeStreamDo: [ :out
| out << 'Les élèves Français' ; crlf ].

 (FileLocator temp / 'benoît' / 'benoît' , 'txt') readStreamDo: [ :in |
in upToEnd ].


 Can Linux/Windows users confirm that it works on their platform ?
>>>
>>> Thanks for attacking this.  All three work on Windows 10 English locale.
>>> I'd hope it wasn't different with French locale, but wait for a report
on that.
>>>
>>> cheers -ben
>>
>> I unzipped a fresh download of
http://files.pharo.org/platform/Pharo6.1-win.zip
>> * into C:\Temp\Pharo6.1
>> and immediately copied that directory
>> * into C:\Temp\Pharo6.1.benoît
>>
>> Double-clicking on Pharo.exe
>> * in the former, opens Pharo an expected.
>> * in the latter, does nothing.

On 16 April 2018 at 19:57, Sven Van Caekenberghe  wrote:
> Ben,
>
> On my machine, macOS 10.13.4,
>
> $ pwd
> /tmp/benoît
>
> $ curl get.pharo.org/70+vm | bash
>   % Total% Received % Xferd  Average Speed   TimeTime Time
 Current
>  Dload  Upload   Total   SpentLeft
 Speed
> 100  3036  100  30360 0  38694  0 --:--:-- --:--:-- --:--:--
38923
> Downloading the latest 70 Image:
> http://files.pharo.org/get-files/70/pharo.zip
> Pharo.image
> Downloading the latest pharoVM:
> http://files.pharo.org/get-files/70/pharo-mac-stable.zip
> pharo-vm/Pharo.app/Contents/MacOS/Pharo
> Creating starter scripts pharo and pharo-ui
>
> $ ./pharo Pharo.image printVersion
> [version] 'Pharo7.0'
'Pharo-7.0+alpha.build.769.sha.244850d3ffc1b49ab716c8093f4bcd9af3917be6 (32
Bit)'
>
> $ ./pharo Pharo.image eval 'FileLocator image resolve'
> File @ /private/tmp/benoît/Pharo.image
>
> And the UI version works too.
>
> So we're back at 'it is a Windows problem' (but maybe it is only visible
on Windows).
>
> Can you see what is going on ?
>
> Technically, at the File (FilePlugin) level we seem to be able to handle
paths with strange characters, even on Windows, but somehow, somewhere a
bogus call is made during image startup.

I tried logging the startup like this...

WorkingSession >> runStartup: isImageStarting
'C:\Temp\StartupLog.txt' asFileReference writeStreamDo: [ :log |
log nextPutAll: log className; flush.
self
runList: manager startupList
do: [ :each |
log
nextPutAll: each printString;
nextPutAll: String crlf;
flush.
each startup: isImageStarting ].
self executeDeferredStartupActions: isImageStarting
]

which worked in the former and with the latter(benoît) nothing.
But poking around I bumped into an interesting symptom.

The following does nothing
C:\Temp\Pharo6.1.benoît> pharo

but the following successfully opens the Image...
C:\Temp\Pharo6.1.benoît> pharo Pharo6.1.image

Not sure where to go from there.  Maybe needs a Windows Debug-VM,
but I'm out of time for now.

cheers -ben

Re: [Pharo-dev] File path encoding

2018-04-16 Thread Sven Van Caekenberghe 


> On 16 Apr 2018, at 14:51, Alistair Grant  wrote:
> 
> Hi Sven,
> 
> On 16 April 2018 at 13:57, Sven Van Caekenberghe  wrote:
>> 
>> Technically, at the File (FilePlugin) level we seem to be able to handle 
>> paths with strange characters, even on Windows, but somehow, somewhere a 
>> bogus call is made during image startup.
> 
> I'm not going to have time to look at this in the next few days, sorry.
> 
> But I have a vague memory that some of the other primitives outside
> FilePlugin don't add the "\\?\" prefix, so they possibly also don't do
> wide character conversion.

Then I hope some other Windows user/developer can have a look. 
Hell, I believe we got a few French people working on Pharo, n'est pas ?

> Cheers,
> Alistair
>

Re: [Pharo-dev] File path encoding

2018-04-16 Thread Alistair Grant 
Hi Sven,

On 16 April 2018 at 13:57, Sven Van Caekenberghe  wrote:
>
> Technically, at the File (FilePlugin) level we seem to be able to handle 
> paths with strange characters, even on Windows, but somehow, somewhere a 
> bogus call is made during image startup.

I'm not going to have time to look at this in the next few days, sorry.

But I have a vague memory that some of the other primitives outside
FilePlugin don't add the "\\?\" prefix, so they possibly also don't do
wide character conversion.

Cheers,
Alistair

Re: [Pharo-dev] File path encoding

2018-04-16 Thread Sven Van Caekenberghe 
Ben,

On my machine, macOS 10.13.4,

$ pwd
/tmp/benoît

$ curl get.pharo.org/70+vm | bash
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
100  3036  100  30360 0  38694  0 --:--:-- --:--:-- --:--:-- 38923
Downloading the latest 70 Image:
http://files.pharo.org/get-files/70/pharo.zip
Pharo.image
Downloading the latest pharoVM:
http://files.pharo.org/get-files/70/pharo-mac-stable.zip
pharo-vm/Pharo.app/Contents/MacOS/Pharo
Creating starter scripts pharo and pharo-ui

$ ./pharo Pharo.image printVersion
[version] 'Pharo7.0' 
'Pharo-7.0+alpha.build.769.sha.244850d3ffc1b49ab716c8093f4bcd9af3917be6 (32 
Bit)'

$ ./pharo Pharo.image eval 'FileLocator image resolve'
File @ /private/tmp/benoît/Pharo.image

And the UI version works too.

So we're back at 'it is a Windows problem' (but maybe it is only visible on 
Windows).

Can you see what is going on ?

Technically, at the File (FilePlugin) level we seem to be able to handle paths 
with strange characters, even on Windows, but somehow, somewhere a bogus call 
is made during image startup.

> On 16 Apr 2018, at 13:05, Ben Coman  wrote:
> 
> On 16 April 2018 at 18:52, Ben Coman  wrote:
>> On 16 April 2018 at 17:51, Sven Van Caekenberghe  wrote:
>>> Hi,
>>> 
>>> Some people raise issues with file path encoding with non-ASCII characters.
>>> 
>>> The following works fine for me (macOS, Pharo 7, using new File class):
>>> 
>>> "works"
>>> 
>>> (FileLocator temp / 'foo' , 'txt') writeStreamDo: [ :out | out << 'Les 
>>> élèves Français' ; crlf ].
>>> 
>>> (FileLocator temp / 'foo' , 'txt') readStreamDo: [ :in | in upToEnd ].
>>> 
>>> "works"
>>> 
>>> (FileLocator temp / 'benoît' , 'txt') writeStreamDo: [ :out | out << 'Les 
>>> élèves Français' ; crlf ].
>>> 
>>> (FileLocator temp / 'benoît' , 'txt') readStreamDo: [ :in | in upToEnd ].
>>> 
>>> "works"
>>> 
>>> (FileLocator temp / 'benoît') ensureCreateDirectory.
>>> 
>>> (FileLocator temp / 'benoît' / 'benoît' , 'txt') writeStreamDo: [ :out | 
>>> out << 'Les élèves Français' ; crlf ].
>>> 
>>> (FileLocator temp / 'benoît' / 'benoît' , 'txt') readStreamDo: [ :in | in 
>>> upToEnd ].
>>> 
>>> 
>>> Can Linux/Windows users confirm that it works on their platform ?
>> 
>> Thanks for attacking this.  All three work on Windows 10 English locale.
>> I'd hope it wasn't different with French locale, but wait for a report on 
>> that.
>> 
>> cheers -ben
> 
> I unzipped a fresh download of 
> http://files.pharo.org/platform/Pharo6.1-win.zip
> * into C:\Temp\Pharo6.1
> and immediately copied that directory
> * into C:\Temp\Pharo6.1.benoît
> 
> Double-clicking on Pharo.exe
> * in the former, opens Pharo an expected.
> * in the latter, does nothing.
> 
> cheers -ben
>

Re: [Pharo-dev] File path encoding

2018-04-16 Thread Ben Coman 
On 16 April 2018 at 18:52, Ben Coman  wrote:
> On 16 April 2018 at 17:51, Sven Van Caekenberghe  wrote:
>> Hi,
>>
>> Some people raise issues with file path encoding with non-ASCII characters.
>>
>> The following works fine for me (macOS, Pharo 7, using new File class):
>>
>> "works"
>>
>> (FileLocator temp / 'foo' , 'txt') writeStreamDo: [ :out | out << 'Les 
>> élèves Français' ; crlf ].
>>
>> (FileLocator temp / 'foo' , 'txt') readStreamDo: [ :in | in upToEnd ].
>>
>> "works"
>>
>> (FileLocator temp / 'benoît' , 'txt') writeStreamDo: [ :out | out << 'Les 
>> élèves Français' ; crlf ].
>>
>> (FileLocator temp / 'benoît' , 'txt') readStreamDo: [ :in | in upToEnd ].
>>
>> "works"
>>
>> (FileLocator temp / 'benoît') ensureCreateDirectory.
>>
>> (FileLocator temp / 'benoît' / 'benoît' , 'txt') writeStreamDo: [ :out | out 
>> << 'Les élèves Français' ; crlf ].
>>
>> (FileLocator temp / 'benoît' / 'benoît' , 'txt') readStreamDo: [ :in | in 
>> upToEnd ].
>>
>>
>> Can Linux/Windows users confirm that it works on their platform ?
>
> Thanks for attacking this.  All three work on Windows 10 English locale.
> I'd hope it wasn't different with French locale, but wait for a report on 
> that.
>
> cheers -ben

I unzipped a fresh download of http://files.pharo.org/platform/Pharo6.1-win.zip
* into C:\Temp\Pharo6.1
and immediately copied that directory
* into C:\Temp\Pharo6.1.benoît

Double-clicking on Pharo.exe
* in the former, opens Pharo an expected.
* in the latter, does nothing.

cheers -ben

Re: [Pharo-dev] File path encoding

2018-04-16 Thread Ben Coman 
On 16 April 2018 at 17:51, Sven Van Caekenberghe  wrote:
> Hi,
>
> Some people raise issues with file path encoding with non-ASCII characters.
>
> The following works fine for me (macOS, Pharo 7, using new File class):
>
> "works"
>
> (FileLocator temp / 'foo' , 'txt') writeStreamDo: [ :out | out << 'Les élèves 
> Français' ; crlf ].
>
> (FileLocator temp / 'foo' , 'txt') readStreamDo: [ :in | in upToEnd ].
>
> "works"
>
> (FileLocator temp / 'benoît' , 'txt') writeStreamDo: [ :out | out << 'Les 
> élèves Français' ; crlf ].
>
> (FileLocator temp / 'benoît' , 'txt') readStreamDo: [ :in | in upToEnd ].
>
> "works"
>
> (FileLocator temp / 'benoît') ensureCreateDirectory.
>
> (FileLocator temp / 'benoît' / 'benoît' , 'txt') writeStreamDo: [ :out | out 
> << 'Les élèves Français' ; crlf ].
>
> (FileLocator temp / 'benoît' / 'benoît' , 'txt') readStreamDo: [ :in | in 
> upToEnd ].
>
>
> Can Linux/Windows users confirm that it works on their platform ?

Thanks for attacking this.  All three work on Windows 10 English locale.
I'd hope it wasn't different with French locale, but wait for a report on that.

cheers -ben