Re: Format text data records for processing in Livecode

[Peter Bogdanoff via use-livecode]

I must say that once I started using arrays for data storage and access, I 
never looked back. Compact, fast, malleable.

Many years ago I thought I would take a continuing education C programming 
class at a university. Knowing only HyperTalk, I muddled along until 
mid-semester when the curriculum now turned to arrays. I withdrew from the 
class; I wasn’t ready.

Now, with more water under the bridge, I embrace the array.

Peter Bogdanoff
ArtsInteractive

> On Mar 1, 2019, at 2:44 AM, Kaveh Bazargan via use-livecode 
>  wrote:
> 
> On Fri, 1 Mar 2019 at 03:44, Richard Gaskin via use-livecode <
> use-livecode@lists.runrev.com> wrote:
> 
>> Kaveh Bazargan wrote:
>> 
>> 
>> If it's working and you're happy keep doing it.
>> 
>> At some point you may find arrays very powerful. This circumstance seems
>> a natural fit for them, as they keep data separated -- even binary data
>> -- while remaining blindingly fast to get at the piece you want whenever
>> you want it.
>> 
>> 
> You are right Richard. It is a testament to LiveCode that I can do
> sophisticated stuff (by my standards) and not use arrays, but you have now
> rightly pushed me to get into arrays, especially the speed gain that is
> important for me. Thanks for all the good advice you give on this list. :-)
> 
> 
>> --
>>  Richard Gaskin
>>  Fourth World Systems
>>  Software Design and Development for the Desktop, Mobile, and the Web
>>  
>>  ambassa...@fourthworld.comhttp://www.FourthWorld.com
>> 
>> ___
>> use-livecode mailing list
>> use-livecode@lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your
>> subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>> 
> 
> 
> -- 
> Kaveh Bazargan PhD
> Director
> River Valley Technologies  • Twitter
>  • LinkedIn
> 
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription 
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Format text data records for processing in Livecode

[Kaveh Bazargan via use-livecode]

On Fri, 1 Mar 2019 at 03:44, Richard Gaskin via use-livecode <
use-livecode@lists.runrev.com> wrote:

> Kaveh Bazargan wrote:
>
>
> If it's working and you're happy keep doing it.
>
> At some point you may find arrays very powerful. This circumstance seems
> a natural fit for them, as they keep data separated -- even binary data
> -- while remaining blindingly fast to get at the piece you want whenever
> you want it.
>
>
You are right Richard. It is a testament to LiveCode that I can do
sophisticated stuff (by my standards) and not use arrays, but you have now
rightly pushed me to get into arrays, especially the speed gain that is
important for me. Thanks for all the good advice you give on this list. :-)


> --
>   Richard Gaskin
>   Fourth World Systems
>   Software Design and Development for the Desktop, Mobile, and the Web
>   
>   ambassa...@fourthworld.comhttp://www.FourthWorld.com
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>


-- 
Kaveh Bazargan PhD
Director
River Valley Technologies  • Twitter
 • LinkedIn

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Format text data records for processing in Livecode

[Richard Gaskin via use-livecode]

Kaveh Bazargan wrote:

> On Tue, 26 Feb 2019 at 23:12, Richard Gaskin wrote:
>
>> Kaveh Bazargan wrote:
>>
>>  > My use case might be unusual in that I have several itemdelimiters
>>  > in the same text. I need to record what each one is, but they must
>>  > not be the same. I might be going off topic here, but I thought it
>>  > is a simple way of generating a text that is guaranteed to be
>>  > unique.
>>
>> The UUID function is ideal for getting a unique string.
>>
>> But maybe simpler, have you considered using an array?
>>
> Hi Richard
>
> So far I have used arrays very rarely. Not sure why.
>
> I did not know about UUID.
>
> One advantage of using the milliseconds is that it gives me an
> automated timeline and speed of events, just by comparing the
> itemdelimiter. But my use case is probably very specific.

If it's working and you're happy keep doing it.

At some point you may find arrays very powerful. This circumstance seems 
a natural fit for them, as they keep data separated -- even binary data 
-- while remaining blindingly fast to get at the piece you want whenever 
you want it.


--
 Richard Gaskin
 Fourth World Systems
 Software Design and Development for the Desktop, Mobile, and the Web
 
 ambassa...@fourthworld.comhttp://www.FourthWorld.com

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Format text data records for processing in Livecode

[Kaveh Bazargan via use-livecode]

On Tue, 26 Feb 2019 at 23:12, Richard Gaskin via use-livecode <
use-livecode@lists.runrev.com> wrote:

> Kaveh Bazargan wrote:
>
>  > My use case might be unusual in that I have several itemdelimiters in
>  > the same text. I need to record what each one is, but they must not be
>  > the same. I might be going off topic here, but I thought it is a
>  > simple way of generating a text that is guaranteed to be unique.
>
> The UUID function is ideal for getting a unique string.
>
> But maybe simpler, have you considered using an array?
>
>
Hi Richard

So far I have used arrays very rarely. Not sure why.

I did not know about UUID.

One advantage of using the milliseconds is that it gives me an automated
timeline and speed of events, just by comparing the itemdelimiter. But my
use case is probably very specific. But I feel it is a good example of the
flexibility of LiveCode. :-)


> --
>   Richard Gaskin
>   Fourth World Systems
>   Software Design and Development for the Desktop, Mobile, and the Web
>   
>   ambassa...@fourthworld.comhttp://www.FourthWorld.com
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>


-- 
Kaveh Bazargan
Director
River Valley Technologies  • Twitter
 • LinkedIn

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Format text data records for processing in Livecode

[Richard Gaskin via use-livecode]

Kaveh Bazargan wrote:

> My use case might be unusual in that I have several itemdelimiters in
> the same text. I need to record what each one is, but they must not be
> the same. I might be going off topic here, but I thought it is a
> simple way of generating a text that is guaranteed to be unique.

The UUID function is ideal for getting a unique string.

But maybe simpler, have you considered using an array?

--
 Richard Gaskin
 Fourth World Systems
 Software Design and Development for the Desktop, Mobile, and the Web
 
 ambassa...@fourthworld.comhttp://www.FourthWorld.com

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Format text data records for processing in Livecode

[Simon Knight via use-livecode]

Some interesting ideas although I’m not sure why a changing separator is 
useful.  I generally attempt to stick with TSV but it won’t work in this case 
unless I parse the data to remove the tabs from the text, however I may well 
try this.  At the moment I’m trying to use a simple tagging system.  The 
AppleScript that creates the data looks like this:

-- build the data in a useful form to be read by Livecode
set tNewRec to "" & return
set tNewRec to tNewRec & "" & "Sent : " & tDateSent & 
"" & return
set tNewRec to tNewRec & "" & "From : " & tSender & 
"" & return
set tNewRec to tNewRec & "" & "To : " & tAddresseeList & 
"" & return
set tNewRec to tNewRec & "" & "Subject : " & tSubject & 
"" & return
set tNewRec to tNewRec & "" & "Content : " & tContent & 
"" & return
set tNewRec to tNewRec & "" & return

The final version will probably not add the closing tags as they seem 
unnecessary.  I have two routines that extract the fields and put them into an 
array, these make use of itemdelimiter.  It all works but the leading REC and 
FLD tags have to be ignored otherwise blank records and fields are processed.  
I think that replacing tabs and line returns in the source data and then saving 
as TSV will be simpler.

best wishes

Simon





___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Format text data records for processing in Livecode

[kee nethery via use-livecode]

What about accidentally choosing a codepoint that is already in the text?

I tend to use tab and return for spreadsheet data.

For text that needs to include tabs and returns, I use the ASCII codepoints 
designed for delimiting data; Unit Separator ACSII37, Record Separator ASCII36, 
Group Separator ASCII35, File Separator ASCII34. And before I use them I always 
make sure the text does not contain any of these characters. If it does (for 
example contains a Unit Separator) I tend to see if the text contains 
“<>” and if not I’ll replace all Unit Separators in the 
text with “<>”. Where (the seconds) is the actual value 
for the seconds. Then I’ll use the ASCII codepoints that were allocated for 
delimiters. And I’ll add my replacement separators at the front of the data 
stream knowing that I can see what that entire string should get replaced with.

My two cents.

Kee Nethery



> On Feb 26, 2019, at 2:10 PM, Kaveh Bazargan via use-livecode 
>  wrote:
> 
> My use case might be unusual in that I have several itemdelimiters in the
> same text. I need to record what each one is, but they must not be the
> same. I might be going off topic here, but I thought it is a simple way of
> generating a text that is guaranteed to be unique.
> 
> On Tue, 26 Feb 2019 at 21:51, dunbarxx via use-livecode <
> use-livecode@lists.runrev.com> wrote:
> 
>> I am missing the point of using an ephemeral and unknown string as an
>> itemDelimiter. The whole point of such a gadget is that it can be used at
>> another time to undo, or redo, whatever data parsing was done earlier.
>> 
>> Is it that the delimiter is stored somewhere, and just for security it is
>> generated in an unpredictable way? But that it still can be recalled as
>> needed? But then, as described, that several unique delimiters in quick
>> succession may be created and used, are these stored as well for future
>> use?
>> 
>> In other words, if a delimiter is created somehow, and data is parsed into
>> items defined by that delimiter, is there any purpose in NOT saving it
>> somewhere?
>> 
>> Craig Newman
>> 
>> 
>> 
>> --
>> Sent from:
>> http://runtime-revolution.278305.n4.nabble.com/Revolution-User-f278306.html
>> 
>> ___
>> use-livecode mailing list
>> use-livecode@lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your
>> subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>> 
> 
> 
> -- 
> Kaveh Bazargan
> Director
> River Valley Technologies  • Twitter
>  • LinkedIn
> 
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription 
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Format text data records for processing in Livecode

[Kaveh Bazargan via use-livecode]

My use case might be unusual in that I have several itemdelimiters in the
same text. I need to record what each one is, but they must not be the
same. I might be going off topic here, but I thought it is a simple way of
generating a text that is guaranteed to be unique.

On Tue, 26 Feb 2019 at 21:51, dunbarxx via use-livecode <
use-livecode@lists.runrev.com> wrote:

> I am missing the point of using an ephemeral and unknown string as an
> itemDelimiter. The whole point of such a gadget is that it can be used at
> another time to undo, or redo, whatever data parsing was done earlier.
>
> Is it that the delimiter is stored somewhere, and just for security it is
> generated in an unpredictable way? But that it still can be recalled as
> needed? But then, as described, that several unique delimiters in quick
> succession may be created and used, are these stored as well for future
> use?
>
> In other words, if a delimiter is created somehow, and data is parsed into
> items defined by that delimiter, is there any purpose in NOT saving it
> somewhere?
>
> Craig Newman
>
>
>
> --
> Sent from:
> http://runtime-revolution.278305.n4.nabble.com/Revolution-User-f278306.html
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>


-- 
Kaveh Bazargan
Director
River Valley Technologies  • Twitter
 • LinkedIn

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Format text data records for processing in Livecode

[dunbarxx via use-livecode]

I am missing the point of using an ephemeral and unknown string as an
itemDelimiter. The whole point of such a gadget is that it can be used at
another time to undo, or redo, whatever data parsing was done earlier.

Is it that the delimiter is stored somewhere, and just for security it is
generated in an unpredictable way? But that it still can be recalled as
needed? But then, as described, that several unique delimiters in quick
succession may be created and used, are these stored as well for future use?

In other words, if a delimiter is created somehow, and data is parsed into
items defined by that delimiter, is there any purpose in NOT saving it
somewhere?

Craig Newman



--
Sent from: 
http://runtime-revolution.278305.n4.nabble.com/Revolution-User-f278306.html

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Format text data records for processing in Livecode

[Kaveh Bazargan via use-livecode]

I have an handler that needs the itemdelimiter to be unique every time I
call it, and that can be several times in quick succession. I use:

wait 1 millisecond
set the itemdelimiter to the milliseconds

The first line ensures that two successive loops do not have the same
itemdelimiter, and it is almost guaranteed to be unique every time.

On Tue, 26 Feb 2019 at 17:41, Ralph DiMola via use-livecode <
use-livecode@lists.runrev.com> wrote:

> Option 2: The item delimiter can be multiple characters so you can set it
> to
> something that would never likely be in the text.
> Example: Set the itemdel to ""
>
> Ralph DiMola
> IT Director
> Evergreen Information Services
> rdim...@evergreeninfo.net
>
> -Original Message-
> From: use-livecode [mailto:use-livecode-boun...@lists.runrev.com] On
> Behalf
> Of dunbarxx via use-livecode
> Sent: Tuesday, February 26, 2019 10:58 AM
> To: use-revolut...@lists.runrev.com
> Cc: dunbarxx
> Subject: Re: Format text data records for processing in Livecode
>
> Hi.
>
> Since you may have any number and kind of characters in any of the fields
> in
> each record, I would append a character that is unlikely ever to appear in
> any context. I like numToChar(202), but others like numToChar(3) or
> numtoChar(8).
>
> Then you can set the itemDelimiter to that char, and be sure that you are
> parsing the totality of each field within any record.
>
> Craig Newman
>
>
>
> --
> Sent from:
> http://runtime-revolution.278305.n4.nabble.com/Revolution-User-f278306.html
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>


-- 
Kaveh Bazargan
Director
River Valley Technologies <http://rivervalleytechnologies.com/> • Twitter
<https://twitter.com/kaveh1000> • LinkedIn
<https://www.linkedin.com/in/bazargankaveh/>
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

RE: Format text data records for processing in Livecode

[Ralph DiMola via use-livecode]

Option 2: The item delimiter can be multiple characters so you can set it to
something that would never likely be in the text.
Example: Set the itemdel to ""

Ralph DiMola
IT Director
Evergreen Information Services
rdim...@evergreeninfo.net

-Original Message-
From: use-livecode [mailto:use-livecode-boun...@lists.runrev.com] On Behalf
Of dunbarxx via use-livecode
Sent: Tuesday, February 26, 2019 10:58 AM
To: use-revolut...@lists.runrev.com
Cc: dunbarxx
Subject: Re: Format text data records for processing in Livecode

Hi.

Since you may have any number and kind of characters in any of the fields in
each record, I would append a character that is unlikely ever to appear in
any context. I like numToChar(202), but others like numToChar(3) or
numtoChar(8).

Then you can set the itemDelimiter to that char, and be sure that you are
parsing the totality of each field within any record.

Craig Newman



--
Sent from:
http://runtime-revolution.278305.n4.nabble.com/Revolution-User-f278306.html

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Format text data records for processing in Livecode

[dunbarxx via use-livecode]

Hi.

Since you may have any number and kind of characters in any of the fields in
each record, I would append a character that is unlikely ever to appear in
any context. I like numToChar(202), but others like numToChar(3) or
numtoChar(8).

Then you can set the itemDelimiter to that char, and be sure that you are
parsing the totality of each field within any record.

Craig Newman



--
Sent from: 
http://runtime-revolution.278305.n4.nabble.com/Revolution-User-f278306.html

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Format text data records for processing in Livecode

[Simon Knight via use-livecode]

Hi,

I have written an Applescript that reads the headers and contents of selected 
emails in Apple Mail.  I am trying to decide to simplest method of importing 
this data into Livecode to allow it to be sorted and formatted etc.  Each email 
is a record with a number of fields the largest field is the contents of the 
email and this may contain quotes, tabs and other characters.  I aim to do the 
minimum in Applecript so I don’t wish to process the data but I can use any 
output format that I devise.

A record contains the following fields : DateSent; From; To; Subject and Content


Any ideas on a simple methodology of how to format the data so that it is 
simple to process using Livecode’s chunk expressions?

best wishes

Simon

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode