Re: Extracting Unicode text from Oracle over ODBC

2022-03-01 Thread Ben Rubinstein via use-livecode
rally fine, except that where the text contains Unicode characters beyond the Windows Latin1/CP512 character set, I'm not getting them. I'm not getting random garbage; I'm getting a sensible transliteration, e.g. "n" for "ň" (n-caron). So this isn't simply getting the b

Re: Extracting Unicode text from Oracle over ODBC

2022-02-28 Thread J. Landman Gay via use-livecode
_19_5".) Generally fine, except that where the text contains Unicode characters beyond the Windows Latin1/CP512 character set, I'm not getting them. I'm not getting random garbage; I'm getting a sensible transliteration, e.g. "n" for "ň" (n-caron). So this isn't simply getting

Extracting Unicode text from Oracle over ODBC

2022-02-28 Thread Ben Rubinstein via use-livecode
I'm trying to extract text from an Oracle database, via ODBC, on Windows. (Using the 64 bit "Oracle in instantclient_19_5".) Generally fine, except that where the text contains Unicode characters beyond the Windows Latin1/CP512 character set, I'm not getting them. I'm not gett

Re: Converting from unicode to ASCII

2020-09-26 Thread J. Landman Gay via use-livecode
On 9/24/20 12:09 PM, J. Landman Gay via use-livecode wrote: My original goal was to get the canonical version directly from LC somehow. Neville Smythe contacted me privately with this brilliant solution, posted here with his consent: function stripAccents pInput local tDecomposed local

Re: Converting from unicode to ASCII

2020-09-24 Thread J. Landman Gay via use-livecode
That's what I was hoping for when I started this thread, and it was suggested (without the ID) a while back, but then I'd need another lookup table. Probably one for each language. My original goal was to get the canonical version directly from LC somehow. -- Jacqueline Landman Gay |

Re: Converting from unicode to ASCII

2020-09-24 Thread Alex Tweedly via use-livecode
You could even decide that, rather than strip out non-ascii characters, you would convert (reduce?) each one to a canonical equivalent (where there is one), and hence instead of l’Académie française---> lAcadmiefranaise_1234.livecode it would become l’Académie française--->

Re: Converting from unicode to ASCII

2020-09-24 Thread Dave Cragg via use-livecode
That's what I was thinking. So the filename for " l’Académie française" might becomes something like lAcadmiefranaise_1234.livecode. Kind of readable, but guaranteed unique. (And also allows identifying the database record from the filename if that is needed.) (Apologies if this appears

Re: Converting from unicode to ASCII

2020-09-24 Thread J. Landman Gay via use-livecode
I'm pretty sure each record has an ID. This would be for ensuring unique file names, right? -- Jacqueline Landman Gay | jac...@hyperactivesw.com HyperActive Software | http://www.hyperactivesw.com On September 24, 2020 2:00:50 AM Dave Cragg via use-livecode wrote: Jacqueline, You said

Re: Converting from unicode to ASCII

2020-09-24 Thread J. Landman Gay via use-livecode
It's all automated already except for the uploading. The file organization on AWS is complex and the stacks don't all go in the same place. -- Jacqueline Landman Gay | jac...@hyperactivesw.com HyperActive Software | http://www.hyperactivesw.com On September 23, 2020 4:53:36 PM Richard Gaskin

Re: Converting from unicode to ASCII

2020-09-24 Thread Dave Cragg via use-livecode
Jacqueline, You said earlier you don't have a field in the database for the file name. But does the database table have a unique numerical ID field for each record? If so, could you strip out the non-ASCII characters and then append the numerical ID to the file name? > On 23 Sep 2020, at

Re: Converting from unicode to ASCII

2020-09-23 Thread Mark Wieder via use-livecode
On 9/23/20 4:50 PM, J. Landman Gay via use-livecode wrote: Heh. Now you understand why I didn't want another lookup table. :) OTOH, one of the cardinal rules of data design is *not* to use real data as an index into data. YMMV. -- Mark Wieder ahsoftw...@gmail.com

Re: Converting from unicode to ASCII

2020-09-23 Thread J. Landman Gay via use-livecode
Heh. Now you understand why I didn't want another lookup table. :) -- Jacqueline Landman Gay | jac...@hyperactivesw.com HyperActive Software | http://www.hyperactivesw.com On September 23, 2020 5:27:06 PM Mark Wieder via use-livecode wrote: On 9/22/20 11:10 PM, J. Landman Gay via

Re: Converting from unicode to ASCII

2020-09-23 Thread Mark Wieder via use-livecode
On 9/22/20 11:10 PM, J. Landman Gay via use-livecode wrote: There's more to it than that; the server runs a cron job hourly that indexes all its files and creates AWS secure URLs for each. The app downloads that lookup file on demand. When the user selects a name from a list, the selection is

Re: Converting from unicode to ASCII

2020-09-23 Thread Bob Sneidar via use-livecode
FYI I have a rudimentary document storage system developed where I can “check out” a document from my app so that no one else can check it out, which downloads the file from it’s repository into a temp folder. The user can then edit or work with the file, then check it in. It RETAINS the

Re: Converting from unicode to ASCII

2020-09-23 Thread Richard Gaskin via use-livecode
For an ongoing need like that on a substantial project, I'd automate it: She works on her master copy, then presses a button. Done. The button saves the stack, copies it to the munged name, and uploads it for her, even verifying the integrity of the upload afterward (machines don't mind the

Re: Converting from unicode to ASCII

2020-09-23 Thread Lagi Pittas via use-livecode
unicode file names. Since most names will only have a maximum of 3 or 4 diacritics or non ascii characters within them (big assumption) - removing the non ascii but tagging their # on the end with a positional value gives you the readable filename and the uniqueness in 1 hit - or am i missing

Re: Converting from unicode to ASCII

2020-09-23 Thread J. Landman Gay via use-livecode
On 9/23/20 1:26 PM, Richard Gaskin via use-livecode wrote: My only suggestion was to change how the existing munger works to satisfy the two problem areas identified: that names not be too long, and that any munger not remove so many characters as to make the file name non-unique or empty.

Re: Converting from unicode to ASCII

2020-09-23 Thread Bob Sneidar via use-livecode
Yes I understand that I was thinking of using the method for something I need. Bob S On Sep 23, 2020, at 11:47 AM, Richard Gaskin via use-livecode mailto:use-livecode@lists.runrev.com>> wrote: No lookup table is needed at all if the relationship between the original string and the resulting

Re: Converting from unicode to ASCII

2020-09-23 Thread J. Landman Gay via use-livecode
On 9/23/20 1:47 PM, Richard Gaskin via use-livecode wrote: But so far I haven't read anything requiring this to work in both directions.  Did I miss something?  Does she also rely on an unmunger function? No, you're correct, I only need the conversion to go one-way. The cron job creates

Re: Converting from unicode to ASCII

2020-09-23 Thread Richard Gaskin via use-livecode
No lookup table is needed at all if the relationship between the original string and the resulting munged file name never needs to also work the other direction. If bidirectional derivation is needed, given the limitations imposed by AWS' naming limitations I would see no way to avoid

Re: Converting from unicode to ASCII

2020-09-23 Thread Bob Sneidar via use-livecode
Understood, but if it were reversible, it would eliminate the necessity of a lookup table as an intermediary. Bob S > On Sep 23, 2020, at 11:26 AM, Richard Gaskin via use-livecode > wrote: > > If I understand her problem correctly, file identification need only be in > one direction. > >

Re: Converting from unicode to ASCII

2020-09-23 Thread Richard Gaskin via use-livecode
If I understand her problem correctly, file identification need only be in one direction. As far as I can tell from the description, everything that needs to determine which file to access does so by using a string from which the hashed file name can be derived. That she already has a

Re: Converting from unicode to ASCII

2020-09-23 Thread Bob Sneidar via use-livecode
Will binaryEncode get you back to the filename? Bob S On Sep 23, 2020, at 8:03 AM, Richard Gaskin via use-livecode mailto:use-livecode@lists.runrev.com>> wrote: J. Landman Gay write: > I'm looking for a way to create non-unicode file names > based on the string that comes out of

Re: Converting from unicode to ASCII

2020-09-23 Thread Bob Sneidar via use-livecode
to:use-livecode@lists.runrev.com>> wrote: J. Landman Gay write: > I'm looking for a way to create non-unicode file names > based on the string that comes out of the database. Ah, public clouds... Amazon's S3 docs say just encoding in UTF-8 should suffice, but then they also list a

Re: Converting from unicode to ASCII

2020-09-23 Thread Richard Gaskin via use-livecode
J. Landman Gay write: > I'm looking for a way to create non-unicode file names > based on the string that comes out of the database. Ah, public clouds... Amazon's S3 docs say just encoding in UTF-8 should suffice, but then they also list a lot of characters they consider "special&q

Re: Converting from unicode to ASCII

2020-09-23 Thread Bob Sneidar via use-livecode
You could extract the filename part of the path returned by tempfile() and use that anywhere. That would require something to track the visible name linked to the stored filename tho’. Bob S On Sep 22, 2020, at 11:10 PM, J. Landman Gay via use-livecode mailto:use-livecode@lists.runrev.com>>

Re: Converting from unicode to ASCII

2020-09-23 Thread Paul Dupuis via use-livecode
displayed. We are now translating some stacks to other languages which require unicode text. AWS recommends not using accented characters in file names, so we need to translate the clicktext into plain ASCII. Is there a good way to do that without using a lookup table or metadata? The file nam

Re: Converting from unicode to ASCII

2020-09-23 Thread Lagi Pittas via use-livecode
in the string where that character was. so instead of #&123 you would put #&5123 which means the character was in the fifth position. This means that (assuming not chinese) the temporary file name will give a hint as to the original name and you could still recreate the original unicode

Re: Converting from unicode to ASCII

2020-09-23 Thread scott--- via use-livecode
now translating some stacks to other languages which require unicode > text. AWS recommends not using accented characters in file names, so we need > to translate the clicktext into plain ASCII. > > Is there a good way to do that without using a lookup table or metadata? The >

Re: Converting from unicode to ASCII

2020-09-23 Thread Richmond Mathewson via use-livecode
"This communication may be unlawfully collected and stored by the Agents of a large number of governments in secret. The parties to this email do not consent to the retrieving or storing of this communication and any related metadata, as well as printing, copying, re-transmitting,disseminating, or

Re: Converting from unicode to ASCII

2020-09-23 Thread Richmond Mathewson via use-livecode
Personally I think deleting everything that is not inwith the ASCII range is potentially a bit dangerous [suppose ALL the letters in the title are not inwith the ASCII range], so I would favour using some sort of lookup table/substitution list. Certainly letters such as accented 'e' can just be

Re: Converting from unicode to ASCII

2020-09-23 Thread J. Landman Gay via use-livecode
-unicode file names based on the string that comes out of the database. There's more to it than that; the server runs a cron job hourly that indexes all its files and creates AWS secure URLs for each. The app downloads that lookup file on demand. When the user selects a name from a list

Re: Converting from unicode to ASCII

2020-09-22 Thread Mark Wieder via use-livecode
On 9/22/20 7:58 PM, J. Landman Gay via use-livecode wrote: Is this just a temporary filename (not long-term storage)? No, the stacks are uploaded to AWS and remain there, retrieved from the server on request. There are currently hundreds of them with more added frequently. That's why I'm

Re: Converting from unicode to ASCII

2020-09-22 Thread J. Landman Gay via use-livecode
Combining responses: "NormalizeText" always returns unicode for all four of its variations, so no go. And as Paul pointed out, if the language is Chinese, deleting all non-ascii characters would leave nothing. On the other hand, we are only converting to Roman languages rig

Re: Converting from unicode to ASCII

2020-09-22 Thread John Balgenorth via use-livecode
You could easily convert it to HEX but that would make the file name exactly twice as long. JB > On Sep 22, 2020, at 4:43 PM, Bob Sneidar via use-livecode > wrote: > > There’s a tempname() function??? Ohhh fun!! > > Bob S > > > On Sep 22, 2020, at 4:22 PM, Mark Wieder via use-livecode >

Re: Converting from unicode to ASCII

2020-09-22 Thread Bob Sneidar via use-livecode
There’s a tempname() function??? Ohhh fun!! Bob S On Sep 22, 2020, at 4:22 PM, Mark Wieder via use-livecode mailto:use-livecode@lists.runrev.com>> wrote: Can you use tempname() to create and retrieve the stack? ___ use-livecode mailing list

RE: Converting from unicode to ASCII

2020-09-22 Thread Ralph DiMola via use-livecode
...@evergreeninfo.net -Original Message- From: use-livecode [mailto:use-livecode-boun...@lists.runrev.com] On Behalf Of J. Landman Gay via use-livecode Sent: Tuesday, September 22, 2020 6:49 PM To: LiveCode Mailing List Cc: J. Landman Gay Subject: Converting from unicode to ASCII I have a stack

Re: Converting from unicode to ASCII

2020-09-22 Thread Mark Wieder via use-livecode
displayed. We are now translating some stacks to other languages which require unicode text. AWS recommends not using accented characters in file names, so we need to translate the clicktext into plain ASCII. Is there a good way to do that without using a lookup table or metadata? The file nam

Re: Converting from unicode to ASCII

2020-09-22 Thread Paul Dupuis via use-livecode
On 9/22/2020 6:58 PM, Devin Asay via use-livecode wrote: But it that doesn’t help, and if nobody ever sees the filenames, why not just loop through the string and delete anything that’s not in ASCII range? Well, if the name is in Chinese,  you would delete the entire name.

Re: Converting from unicode to ASCII

2020-09-22 Thread Devin Asay via use-livecode
AWS server and > displayed. > > We are now translating some stacks to other languages which require unicode > text. AWS recommends not using accented characters in file names, so we need > to translate the clicktext into plain ASCII. > > Is there a good way to do that without us

Converting from unicode to ASCII

2020-09-22 Thread J. Landman Gay via use-livecode
ages which require unicode text. AWS recommends not using accented characters in file names, so we need to translate the clicktext into plain ASCII. Is there a good way to do that without using a lookup table or metadata? The file names doesn't need to make sense syntactically. For example,

Re: text manipulation with Chinese characters (unicode): guide?

2020-09-09 Thread Peter Bogdanoff via use-livecode
in Chinese are somewhat of a black hole. Peter Bogdanoff > On Sep 9, 2020, at 5:37 AM, Paul Dupuis via use-livecode > wrote: > > As mentions, Unicode in a field just works (there is a known issue with Thai > word breaks and another with printing Hindi) > > If you want to get

Re: text manipulation with Chinese characters (unicode): guide?

2020-09-09 Thread Curry Kenworthy via use-livecode
Livecode 中文支持非常好。 您可以复制和粘贴。 อย่างไรก็ตามการตัดข้อความของ Livecode สำหรับภาษาไทยนั้นไม่ดีนัก มันทำให้ตัวละครแตก :) Best wishes, Curry Kenworthy Custom Software Development "Better Methods, Better Results" LiveCode Training and Consulting http://livecodeconsulting.com/

Re: text manipulation with Chinese characters (unicode): guide?

2020-09-09 Thread Paul Dupuis via use-livecode
As mentions, Unicode in a field just works (there is a known issue with Thai word breaks and another with printing Hindi) If you want to get Chinese (traditional or simplified) into a field, you can: 1)  get the text from a Utf8 ot utf16 text file and textDecode it i.e put url ("

Re: text manipulation with Chinese characters (unicode): guide?

2020-09-09 Thread Richmond via use-livecode
use-livecode wrote: Hi! I would like to be able to copy and paste into fields and scripts Chinese characters, which are of course unicode and then manipulate the resulting strings. Is this possible in livecode? How? Are there any good Recent guides to working with unicode in livecode

text manipulation with Chinese characters (unicode): guide?

2020-09-08 Thread Eric A. Engle via use-livecode
Hi! I would like to be able to copy and paste into fields and scripts Chinese characters, which are of course unicode and then manipulate the resulting strings. Is this possible in livecode? How? Are there any good Recent guides to working with unicode in livecode

Re: Unicode and languages

2020-06-07 Thread David V Glasgow via use-livecode
, Paul Dupuis via use-livecode wrote: >> In all the added stuff the LC7 and higher Unicode engine includes, is there >> any way to determine the LANGUAGE of a range of text? >> >> USE-CASE >> >> We have a tool that helps researchers transcribe text from di

Re: Unicode and languages

2020-06-06 Thread Jim Lambert via use-livecode
Not LC native but take a look at Google Translate API

Re: Unicode and languages

2020-06-06 Thread Alex Tweedly via use-livecode
mismatch.  - and if not, proceed to do the spellcheck. Adjust the numbers until it gives protection without too many false positives. Alex. On 05/06/2020 18:15, Paul Dupuis via use-livecode wrote: In all the added stuff the LC7 and higher Unicode engine includes, is there any way to determine

Re: Unicode and languages

2020-06-05 Thread Richmond via use-livecode
I doubt that. But if you can determine the Unicode range that is being used you can at least know which writing system is being used. You could then trap for individual glyphs (such as 'џ', which is only used in Macedonian) to narrow things down a spot. On 5.06.20 20:15, Paul Dupuis via use

Re: Unicode and languages

2020-06-05 Thread Paul Dupuis via use-livecode
On 6/5/2020 1:46 PM, Mark Waddingham via use-livecode wrote: On 2020-06-05 18:15, Paul Dupuis via use-livecode wrote: I don't even know if language exists in the IBM Unicode engine as some exportable property a future version of LiveCode could expose. Any clever ideas or thoughts

Re: Unicode and languages

2020-06-05 Thread Mark Waddingham via use-livecode
On 2020-06-05 18:15, Paul Dupuis via use-livecode wrote: I don't even know if language exists in the IBM Unicode engine as some exportable property a future version of LiveCode could expose. Any clever ideas or thoughts on this problem are welcome. Unicode doesn't deal in languages

Unicode and languages

2020-06-05 Thread Paul Dupuis via use-livecode
In all the added stuff the LC7 and higher Unicode engine includes, is there any way to determine the LANGUAGE of a range of text? USE-CASE We have a tool that helps researchers transcribe text from digital media. It is used internationally. We have added spell checking using lclSpell form

Re: Damn You Unicode!

2020-04-14 Thread doc hawk via use-livecode
I’m thinking that possibly there is a violated assumption as to what the other end is using within that, yes. Or that the other end makes an assumption, and puts the “wrong” type back in. Which kind of SQL server is that? Microsofts? I can probably rig access to my Postgres server if you

Re: Damn You Unicode!

2020-04-14 Thread Mark Wieder via use-livecode
On 4/14/20 2:34 PM, doc hawk via use-livecode wrote: Mark mentioned Try a picnic basket instead Given that someone actually implemented RFC 1149, why not? Ha! I totally missed the fact that someone (other than Terry Pratchett) had actually done a real-world implementation. "And, as

Re: Damn You Unicode!

2020-04-14 Thread doc hawk via use-livecode
Mark mumbled > and in doing so you could unUnicode it. OK I made up that word, but still… Verbing nouns again, the man is . . . ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage

Re: Damn You Unicode!

2020-04-14 Thread Mark Wieder via use-livecode
On 4/14/20 12:56 PM, Bob Sneidar via use-livecode wrote: I have some code in a field that I put into a variable then I “do” the variable. Works fine on a Mac. On Windows however, the command throws an error. I build the code in the field by getting values from different objects on the card.

Re: Damn You Unicode!

2020-04-14 Thread doc hawk via use-livecode
Mark mentioned > > Try a picnic basket instead Given that someone actually implemented RFC 1149, why not? ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription

Re: Damn You Unicode!

2020-04-14 Thread Mark Wieder via use-livecode
On 4/14/20 1:09 PM, doc hawk via use-livecode wrote: Shooting from the hip, but could it be differences in cr, lf, and the like getting processed differently? That is, being intrereted by your yogi as a break in one implementation and not another? (Which, of course, would be a boo-boo.)

Re: Damn You Unicode!

2020-04-14 Thread doc hawk via use-livecode
Shooting from the hip, but could it be differences in cr, lf, and the like getting processed differently? That is, being intrereted by your yogi as a break in one implementation and not another? (Which, of course, would be a boo-boo.) pawk, who couldn’t manage to get “Cindy” or “ranger smith”

Re: Damn You Unicode!

2020-04-14 Thread Bob Sneidar via use-livecode
gt; The problem then is that text is unicode, so when I attempt to do the code, > it fails, and the error indicates that the last character of the code is the > first character of the dgProp [“columns”]. > > Example: > > sqlquery_set qObject, “select clause”, “E > > That is

Damn You Unicode!

2020-04-14 Thread Bob Sneidar via use-livecode
”] of a data grid. The problem then is that text is unicode, so when I attempt to do the code, it fails, and the error indicates that the last character of the code is the first character of the dgProp [“columns”]. Example: sqlquery_set qObject, “select clause”, “E That is as much as sqlYoga

Re: Unicode mysteries

2020-03-27 Thread Neville Smythe via use-livecode
I have filed a bug report bug_22561 <https://quality.livecode.com/show_bug.cgi?id=22651> re the misreporting of codepoints for unicode characters. I am pretty convinced the treatment of the Rainbow flag emoji as three separate characters should be treated as incorrect behaviour and the

Re: Unicode mysteries

2020-03-26 Thread Neville Smythe via use-livecode
eally* need the actual font the system uses to display unsupported codepoints. I was thinking of using it as a lazy way to find out which single codepoints are supported rather than having to parse the cmap tables in the font file. As a way of learning about unicode I was trying to writing an LC

Re: Unicode mysteries

2020-03-26 Thread Bob Sneidar via use-livecode
I see the flag here. Bob S On Mar 25, 2020, at 11:53 PM, Neville Smythe via use-livecode mailto:use-livecode@lists.runrev.com>> wrote: Taking a look at the Emoji ‘flag of Scotland’ character gbsct which won’t display here but exists in the Apple Color Emoji font and in corresponding fonts

Re: Unicode mysteries

2020-03-26 Thread Mark Waddingham via use-livecode
On 2020-03-26 06:53, Neville Smythe via use-livecode wrote: Which should correspond to codepoints 1F3F4 E0067 E0062 E0073 E0063 E0074 E007F And indeed if I manually build a UTF-16 string with these code points it does display as the flag of Scotland. So the lesson is that the reported

Unicode mysteries

2020-03-26 Thread Neville Smythe via use-livecode
I am trying to understand the mysteries of unicode encodings; the following may (or may not) be useful (or confusing) to others. The docs say the full chunk expression for a unicode character is byte i of codeunit j of codepoint k of character c of str (with the warning

Re: Unicode is not "everywhere"...

2019-10-27 Thread Curry Kenworthy via use-livecode
h the same methods or for the same purpose. Sometimes the existing format is VERY efficient for certain tasks. Back to Unicode: > For the same reason that urlEncode and urlDecode cannot be > changed, the detailed files/folders cannot be changed. Nope. Unicode? Absolutely! This is an intentional

Another LC905rc1 function NOT Unicode aware

2019-09-24 Thread Paul Dupuis via use-livecode
And 'open printing to PDF' is yet another part of LC905rc1 still not Unicode aware. https://quality.livecode.com/show_bug.cgi?id=22385 Filename/paths and PDF document property options can not be Unicode. ___ use-livecode mailing list use-livecode

RE: Merge and unicode

2019-09-11 Thread Ralph DiMola via use-livecode
-livecode-boun...@lists.runrev.com] On Behalf Of J. Landman Gay via use-livecode Sent: Wednesday, September 11, 2019 4:54 PM To: How to use LiveCode Cc: J. Landman Gay Subject: Re: Merge and unicode On 9/11/19 12:28 AM, Richard Gaskin via use-livecode wrote: > I'll wager using a styledText ar

Re: Merge and unicode

2019-09-11 Thread J. Landman Gay via use-livecode
On 9/11/19 12:28 AM, Richard Gaskin via use-livecode wrote: I'll wager using a styledText array for this will be fun to write and will perform very well. I took a look. You'd win that wager. I didn't test performance (haven't written the handler yet) but getting the styledText of an existing

Re: Merge and unicode

2019-09-10 Thread Richard Gaskin via use-livecode
I'll wager using a styledText array for this will be fun to write and will perform very well.Richard GaskinFourth World Systems ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your

Re: Merge and unicode

2019-09-10 Thread J. Landman Gay via use-livecode
I wondered about using htmltext in the merge too, but before I started using merge I was setting the properties one by one in a handler. Here's part of my original handler, where pResults is a list of lines that match search criteria: repeat for each line l in pResults put item 1 of l &

Re: Merge and unicode

2019-09-10 Thread Dar Scott Consulting via use-livecode
Because htmlText does not set the metadata to interesting characters, you can consider this workaround: Change the quotes in tCONCEPT to and . And, thus, in tMETADATA. > On Sep 10, 2019, at 12:47 PM, J. Landman Gay via use-livecode > wrote: > > :) > 1. Jacque is very confused too, but is

Re: Merge and unicode

2019-09-10 Thread Dar Scott Consulting via use-livecode
I looked at this some more on OS X. I'm not seeing a problem with merge. And I'm not seeing a problem with metadata per se, I don't think. But I am seeing a problem with setting metadata with htmlText. > On Sep 10, 2019, at 1:32 PM, J. Landman Gay via use-livecode > wrote: > > On 9/10/19

Re: Merge and unicode

2019-09-10 Thread J. Landman Gay via use-livecode
On 9/10/19 1:47 PM, J. Landman Gay via use-livecode wrote: :) 1. Jacque is very confused too, but is afraid of big sticks. 2. Encoding should be identical throughout. I'm working with a large text block, pulling out sections to create a list. All data is retrieved from othe same variable,

Re: Merge and unicode

2019-09-10 Thread J. Landman Gay via use-livecode
:) 1. Jacque is very confused too, but is afraid of big sticks. 2. Encoding should be identical throughout. I'm working with a large text block, pulling out sections to create a list. All data is retrieved from othe same variable, which is UTF16 native LC text. 3. The metadata is only set at

Re: Merge and unicode

2019-09-10 Thread J. Landman Gay via use-livecode
eld 1 after field 1 end mouseup On Sep 9, 2019, at 8:32 PM, J. Landman Gay via use-livecode wrote: Well, I've made some changes to the code since I started urlEncoding the text before merging so I'll check that again. Paul is right that unicode in htmltext needs to be in hex, but the numbers I'm

Re: Merge and unicode

2019-09-10 Thread dsc--- via use-livecode
Jacque, these are my latest thoughts as far as possible problems. 1. Dar is very confused and off in the wrong direction. Use big stick. 2. Binary data is in an 8-bit char set encoding causing problems with UTF-8 decode. Check encoding. 3. Field, line and character metadata are interfering.

Re: Merge and unicode

2019-09-10 Thread Dar Scott Consulting via use-livecode
Trusting... Also, interpreting Latin-1 as UTF-8 can generate some weird characters and lots of ?-diamond symbols. > On Sep 10, 2019, at 8:36 AM, Bob Sneidar via use-livecode > wrote: > > Trust me it's better than a feral gander persuit. > > Bob S > > >> On Sep 9, 2019, at 17:23 , Dar

Re: Merge and unicode

2019-09-10 Thread Bob Sneidar via use-livecode
Trust me it's better than a feral gander persuit. Bob S > On Sep 9, 2019, at 17:23 , Dar Scott Consulting via use-livecode > wrote: > > Sorry, if I am off on a bunny trail... > > Dar ___ use-livecode mailing list use-livecode@lists.runrev.com

Re: Merge and unicode

2019-09-09 Thread dsc--- via use-livecode
etadata of char 1 of field 1 to "é" >> >> put the metadata of char 1 of field 1 after field 1 >> >> end mouseup >> >> >>> On Sep 9, 2019, at 8:32 PM, J. Landman Gay via use-livecode >>> wrote: >>> >>> Well,

Re: Merge and unicode

2019-09-09 Thread dsc--- via use-livecode
p 9, 2019, at 8:32 PM, J. Landman Gay via use-livecode >> wrote: >> >> Well, I've made some changes to the code since I started urlEncoding the >> text before merging so I'll check that again. Paul is right that unicode in >> htmltext needs to be in hex, but the numbers I'm

Re: Merge and unicode

2019-09-09 Thread Dar Scott Consulting via use-livecode
gt; Well, I've made some changes to the code since I started urlEncoding the text > before merging so I'll check that again. Paul is right that unicode in > htmltext needs to be in hex, but the numbers I'm getting back are very high > (8,000+) and render in the field as strange pictogra

Re: Merge and unicode

2019-09-09 Thread J. Landman Gay via use-livecode
Well, I've made some changes to the code since I started urlEncoding the text before merging so I'll check that again. Paul is right that unicode in htmltext needs to be in hex, but the numbers I'm getting back are very high (8,000+) and render in the field as strange pictographs. Elsewhere

Re: Merge and unicode

2019-09-09 Thread Dar Scott Consulting via use-livecode
t. And then try merge. (Which still might have >> problems, I suppose.) >>> On Sep 9, 2019, at 12:08 PM, J. Landman Gay via use-livecode >>> wrote: >>> >>> It seems that the merge command doesn't respect unicode. Does anyone have a >>> workarou

Re: Merge and unicode

2019-09-09 Thread Paul Dupuis via use-livecode
Doesn't any Unicode in htmlText of a field need to be in HTML form (i.e. #;? I thought htmlText turns any non ASCII into either hex encoded html or, where html entity names exists, uses html entity names. On 9/9/2019 6:35 PM, J. Landman Gay via use-livecode wrote: On 9/9/19 2:39 PM, Paul

Re: Merge and unicode

2019-09-09 Thread J. Landman Gay via use-livecode
mean "encoded to UTF-16"? In that case you should decode that to convert it to internal text. And then try merge. (Which still might have problems, I suppose.) On Sep 9, 2019, at 12:08 PM, J. Landman Gay via use-livecode wrote: It seems that the merge command doesn't respect unicode. D

Re: Merge and unicode

2019-09-09 Thread J. Landman Gay via use-livecode
On 9/9/19 2:39 PM, Paul Dupuis via use-livecode wrote: On 9/9/2019 2:13 PM, J. Landman Gay via use-livecode wrote: On 9/9/19 1:08 PM, J. Landman Gay via use-livecode wrote: It seems that the merge command doesn't respect unicode. Does anyone have a workaround? The text I'm inserting is already

Re: Merge and unicode

2019-09-09 Thread Dar Scott Consulting via use-livecode
e: > > It seems that the merge command doesn't respect unicode. Does anyone have a > workaround? The text I'm inserting is already decoded to UTF16. > > -- > Jacqueline Landman Gay | jac...@hyperactivesw.com > HyperActive Software

Re: Merge and unicode

2019-09-09 Thread Paul Dupuis via use-livecode
On 9/9/2019 2:13 PM, J. Landman Gay via use-livecode wrote: On 9/9/19 1:08 PM, J. Landman Gay via use-livecode wrote: It seems that the merge command doesn't respect unicode. Does anyone have a workaround? The text I'm inserting is already decoded to UTF16. I misspoke, sorry. It's

Re: Merge and unicode

2019-09-09 Thread J. Landman Gay via use-livecode
On 9/9/19 1:08 PM, J. Landman Gay via use-livecode wrote: It seems that the merge command doesn't respect unicode. Does anyone have a workaround? The text I'm inserting is already decoded to UTF16. I misspoke, sorry. It's the metadata that doesn't respect unicode. -- Jacqueline Landman Gay

Merge and unicode

2019-09-09 Thread J. Landman Gay via use-livecode
It seems that the merge command doesn't respect unicode. Does anyone have a workaround? The text I'm inserting is already decoded to UTF16. -- Jacqueline Landman Gay | jac...@hyperactivesw.com HyperActive Software | http://www.hyperactivesw.com

Re: Unicode is not "everywhere"...

2019-08-27 Thread Mark Waddingham via use-livecode
On 2019-08-27 10:55, Dar Scott Consulting via use-livecode wrote: The added parameter approach looks very similar to an enhancement suggestion that has been around for a while. I'd mention the bug number, but I and bugs are not getting along at the moment. Indeed - it is very unlikely a new

Re: Unicode is not "everywhere"...

2019-08-27 Thread Dar Scott Consulting via use-livecode
-08-22 20:53, Paul Dupuis via use-livecode wrote: >> I just want it consistent and documented and able to return more than >> just ASCII data >> Currently, OSX shell returns UTF8 which may mean that it is returning >> binary as it is returning 8-bit bytes where Unic

Re: Unicode is not "everywhere"...

2019-08-27 Thread Paul Dupuis via use-livecode
Mark, 1) Fine with shell remaining as is. Thanks to Dar and others, there are ways to execute a windows or OSX shell command that returns Unicode and get Unicode back. The Dictionary entries just need to be documented with  a caution or warning about how to handle Unicode with Shell on each

Re: Unicode is not "everywhere"...

2019-08-27 Thread Mark Waddingham via use-livecode
On 2019-08-22 21:07, Paul Dupuis via use-livecode wrote: I reported what I thought was 3 bugs in 1 report in https://quality.livecode.com/show_bug.cgi?id=22213. I have edited that report to focus on a single bug - that the detailed files (and probably the detailed folders) is broken for Unicode

Re: Unicode is not "everywhere"...

2019-08-27 Thread Mark Waddingham via use-livecode
On 2019-08-22 20:53, Paul Dupuis via use-livecode wrote: I just want it consistent and documented and able to return more than just ASCII data Currently, OSX shell returns UTF8 which may mean that it is returning binary as it is returning 8-bit bytes where Unicode text has been encoded as UTF8

Re: Unicode is not "everywhere"...

2019-08-23 Thread Paul Dupuis via use-livecode
an UTF-16. (1) this matches shell() on OSX which returns results from the shell command UTF-8 encoded, or would you suggest changing OSX?; (2) UTF8 encoding means that people expecting ASCII can just work with the returned results as they always have. People expecting Unicode, will need to p

Re: Unicode is not "everywhere"...

2019-08-23 Thread Dar Scott Consulting via use-livecode
should ensure that for each OS. > > Well this is why I advocate that the encoding should be UTF-8 rather than > UTF-16. (1) this matches shell() on OSX which returns results from the shell > command UTF-8 encoded, or would you suggest changing OSX?; (2) UTF8 encoding > means that

  1   2   3   4   5   6   7   8   9   >