Re: Unicode is not "everywhere"...

This post regards the future of "the detailed files". I'm resurrecting this thread to reply to a couple of very important statements after helping to create a fairly fast, cross-platform tentative workaround (still in testing, and I only mention this since the code was already posted) for

Re: Unicode is not "everywhere"...

On 2019-08-27 10:55, Dar Scott Consulting via use-livecode wrote: The added parameter approach looks very similar to an enhancement suggestion that has been around for a while. I'd mention the bug number, but I and bugs are not getting along at the moment. Indeed - it is very unlikely a new

Re: Unicode is not "everywhere"...

The added parameter approach looks very similar to an enhancement suggestion that has been around for a while. I'd mention the bug number, but I and bugs are not getting along at the moment. Dar > On Aug 27, 2019, at 5:54 AM, Mark Waddingham via use-livecode > wrote: > > On 2019-08-22

Re: Unicode is not "everywhere"...

Mark, 1) Fine with shell remaining as is. Thanks to Dar and others, there are ways to execute a windows or OSX shell command that returns Unicode and get Unicode back. The Dictionary entries just need to be documented with  a caution or warning about how to handle Unicode with Shell on each

Re: Unicode is not "everywhere"...

On 2019-08-22 21:07, Paul Dupuis via use-livecode wrote: I reported what I thought was 3 bugs in 1 report in https://quality.livecode.com/show_bug.cgi?id=22213. I have edited that report to focus on a single bug - that the detailed files (and probably the detailed folders) is broken for Unicode

Re: Unicode is not "everywhere"...

On 2019-08-22 20:53, Paul Dupuis via use-livecode wrote: I just want it consistent and documented and able to return more than just ASCII data Currently, OSX shell returns UTF8 which may mean that it is returning binary as it is returning 8-bit bytes where Unicode text has been encoded as UTF8

Re: Unicode is not "everywhere"...

You know, I don't know. I have spent so much time on these bugs that my brain is burned out. Someone like Mark Waddingham can determine the architecturally correct return values and then make sure the Dictionary reflects the right way to use them. At this point, I have work-arounds to shell

Re: Unicode is not "everywhere"...

If shell returns text, does that mean shellCommand has to go away? > On Aug 23, 2019, at 9:05 AM, Paul Dupuis via use-livecode > wrote: > > On 8/23/2019 10:30 AM, Dar Scott Consulting via use-livecode wrote: >> SHELL() >> >> Related: https://quality.livecode.com/show_bug.cgi?id=22334 >>

Re: Unicode is not "everywhere"...

On 8/23/2019 10:30 AM, Dar Scott Consulting via use-livecode wrote: SHELL() Related: https://quality.livecode.com/show_bug.cgi?id=22334 I was on the fence concerning whether shell() returns a text string or binary data. I have fallen

Re: Unicode is not "everywhere"...

Hi Dar, Yes, this is a viable work-around for the Windows shell bug. function newShell pCommand   local tResult   set the hideConsoleWindows to true   set the shellCommand to "cmd.exe /u" --/U for UTF16 output   put shell(pCommand) into tResult   if the result is empty then -- no errors     put

Re: Unicode is not "everywhere"...

SHELL() Related: https://quality.livecode.com/show_bug.cgi?id=22334 I was on the fence concerning whether shell() returns a text string or binary data. I have fallen to one side: BINARY. Here is a summary of my position. Tomatoes are

Re: Unicode is not "everywhere"...

Paul, does this solve the problem with shell() on Windows? > On Aug 22, 2019, at 10:27 PM, dsc--- via use-livecode > wrote: > > I tried it. > > on mouseUp > set the shellCommand to "cmd.exe /u" > put textDecode( shell("dir"), "UTF-16" ) > end mouseUp > > That does seem to double up the

Re: Unicode is not "everywhere"...

I tried it. on mouseUp set the shellCommand to "cmd.exe /u" put textDecode( shell("dir"), "UTF-16" ) end mouseUp That does seem to double up the newlines. It looks like it works otherwise. Dar > On Aug 22, 2019, at 12:15 PM, dsc--- via use-livecode > wrote: > > The shellCommand

Re: Unicode is not "everywhere"...

> On 23 Aug 2019, at 11:07 am, Paul Dupuis via use-livecode > wrote: > > I reported what I thought was 3 bugs in 1 report in > https://quality.livecode.com/show_bug.cgi?id=22213 > . I have edited that > report to focus on a single bug -

Re: Unicode is not "everywhere"...

All encoding is binary, but not all binary is valid Unicode encoding. > On Aug 22, 2019, at 6:53 PM, Paul Dupuis via use-livecode > wrote: > > Currently, OSX shell returns UTF8 which may mean that it is returning binary > as it is returning 8-bit bytes where Unicode text has been encoded as

Re: Unicode is not "everywhere"...

But that isn't the issue. Well, not what I thought the issue was. If shell() only returns binary, as suggested by the mention of textDecode(), then binary data must always have an interpretation as a string to keep from breaking legacy code. Currently, that is interpreted as an 8-bit character

Re: Unicode is not "everywhere"...

On 8/22/2019 8:46 PM, Monte Goulding via use-livecode wrote: Both of these are anomalies we could only resolve with new syntax I think… at lease the urlEncode one is. I’m not sure if the expectation of shell is it returns text or binary data… The workaround there would be to open process for

Re: Unicode is not "everywhere"...

I just want it consistent and documented and able to return more than just ASCII data Currently, OSX shell returns UTF8 which may mean that it is returning binary as it is returning 8-bit bytes where Unicode text has been encoded as UTF8 Windows returns CP1252 text, not binary and any

Re: Unicode is not "everywhere"...

Both of these are anomalies we could only resolve with new syntax I think… at lease the urlEncode one is. I’m not sure if the expectation of shell is it returns text or binary data… The workaround there would be to open process for UTF8 text read instead of using shell... not sure if UTF8 would

Re: Unicode is not "everywhere"...

You can't have it both ways. Either shell() only characters of the broadest character set, Unicode, or it returns binary. If it returns characters, then it will not work for binary responses; that will have to be documented as excluded. If Unicode is returned then textDecode is not needed and

Re: Unicode is not "everywhere"...

On 8/22/2019 4:08 PM, Richard Gaskin via use-livecode wrote: Dar Scott wrote: > If the result of shell() is run through textDecode, should then > shell() return binary data? This puts it outside the scope of > "everywhere". I like the idea, as long as simple ASCII characters > work

Re: Unicode is not "everywhere"...

Dar Scott wrote: > If the result of shell() is run through textDecode, should then > shell() return binary data? This puts it outside the scope of > "everywhere". I like the idea, as long as simple ASCII characters > work transparently. Isn't that the goal of UTF-8, which is why so many

Re: Unicode is not "everywhere"...

If the result of shell() is run through textDecode, should then shell() return binary data? This puts it outside the scope of "everywhere". I like the idea, as long as simple ASCII characters work transparently. Currently, "is strictly" says it is a string, not a binary string. As far as the

Re: Unicode is not "everywhere"...

Dar Scott wrote: > Concerning 22335... > > 1. I wonder if anyone uses shell() to return binary values. In the modern Unicode world, what is "text" and what is "binary"? I believe MacOS' HFS+ and Win's NFS use UTF-16 natively, Linux EXT4 uses UTF-8. It might seem simple to have a habit of

Re: Unicode is not "everywhere"...

The shellCommand property for Windows is "cmd.exe", not "command.com " as described in the dictionary. I wonder how setting that to "cmd.exe /u" would affect this behavior. > On Aug 22, 2019, at 10:19 AM, Dar Scott Consulting via use-livecode > wrote: > > Concerning

Re: Unicode is not "everywhere"...

Concerning 22335... 1. I wonder if anyone uses shell() to return binary values. 2. I wonder about whether using the shellCommand property would help. > On Aug 22, 2019, at 9:37 AM, Paul Dupuis via use-livecode > wrote: > > I have filed two bug reports that are in LC905rc1 and go back to 7.0

Unicode is not "everywhere"...

I have filed two bug reports that are in LC905rc1 and go back to 7.0 where LC functions that should deal with Unicode properly do not. These are: https://quality.livecode.com/show_bug.cgi?id=22213 -- The "detailed files" function fails for any files with Unicode in the name, returning the