Re: Making "read from file" less blocky.
The trick is to create an independent process. A standalone that listens for messages. Pass it some parameters and go on about your business. Multithreaded computing on demand. AppleScript or open process doesn’t matter. Sent from my iPhone > On Aug 3, 2019, at 19:09, Dar Scott Consulting via use-livecode > wrote: > > Alas, only read from socket allows a message to be sent upon completion. The > step siblings read from file, read from process and read from driver do not. > > Here are a few things you might do: > > 1. Try making the file loading very fast and don't do it in the background. > Change the cursor if need be. Try using URL with file: or binfile:, maybe > that is fast. > > 2. Use a "send loop" to read in portions and update a progress bar. You can > get help here on how to do that. > > 3. Process the file lazily and bring in parts as needed. > > 4. Figure out how to make the file read through networking. Somehow. Maybe. > > >> On Aug 3, 2019, at 6:56 PM, Tom Glod via use-livecode >> wrote: >> >> Hey folks, >> >> I'm having trouble finding a combination of settings that allows my file >> loading to seem to happen in the background. >> >> repeat while read_result is not "eof" >>read from file ThisFile for (1024 * 1000) bytes >>put the result into read_result >>put it after IntoThisVariable >>add length(it) to amount_read >>TSTProgress amount_read,ExpectedSize,"%","Loading File ..." >> (Progress Indicator Handler) >>wait 10 milliseconds with messages >> end repeat >> >> no matter what I try, its still sluggish, and it seems like messages are >> still accumilating instead of being processed by the engine. >> >> Am I missing something? Normally waiting with messages sufficiently frees >> the engine to allow the UI to remain responsive. >> >> Thanks, >> >> Tom >> ___ >> use-livecode mailing list >> use-livecode@lists.runrev.com >> Please visit this url to subscribe, unsubscribe and manage your subscription >> preferences: >> http://lists.runrev.com/mailman/listinfo/use-livecode > > > ___ > use-livecode mailing list > use-livecode@lists.runrev.com > Please visit this url to subscribe, unsubscribe and manage your subscription > preferences: > http://lists.runrev.com/mailman/listinfo/use-livecode ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Making "read from file" less blocky.
OK. Skip "cat" for now. Everything else should work on Windows.
To load a file in one swoop...
put URL("binfile:" & ThisFile) into IntoThisVariable
And if you saved the file with compress, then decompress it.
To load portions of a file just-in-time...
Use "at" and "for" in "read from"
That wont help if you immediately want to find the average or something. But it
might help if you want to read a header and first record, then move to another
record as needed.
I am awful at explaining things so twist my arm until I do it right.
> On Aug 4, 2019, at 5:28 PM, Tom Glod via use-livecode
> wrote:
>
> heheh its always a fun exercise.
>
> Dar, these are linux based solutions right? using windows here at the
> moment, so I can't test, but when I test and optimize my application for
> linux i will try these.
>
> On Sun, Aug 4, 2019 at 4:43 PM dsc--- via use-livecode <
> use-livecode@lists.runrev.com> wrote:
>
>> Even more "really out there" of the "really out there".
>>
>> Don't read in the file.
>>
>> Access portions JIT, that is, lazily.
>>
>> Create a function that pulls in segments of the file. Kinda like this:
>>
>>function segmentOfFile pStartIndex, pEndIndex, pThisFile
>>
>> Or this:
>>
>>function segmentOfCurrentFile pStartIndex, pEndIndex
>>
>> Dar
>>
>>> On Aug 4, 2019, at 1:22 PM, dsc--- via use-livecode <
>> use-livecode@lists.runrev.com> wrote:
>>>
>>> More "really out there".
>>>
>>> I like the idea of trying to speed up an upfront foreground load.
>> Something simple like this:
>>>
>>> put blah-plah into IntoThisVariable.
>>>
>>> where blah-blah is nana-nana or decompress( nana-nana )
>>> where nana-nana is one of these:
>>> URL ("binfile:" & ThisFile)
>>> shell("cat " & ThisFile)
>>>
>>> Function decompress() makes two RAM hits and requires control over the
>> loaded files.
>>>
>>>
On Aug 4, 2019, at 12:47 PM, Dar Scott Consulting via use-livecode <
>> use-livecode@lists.runrev.com> wrote:
I love "really out there". I wanna play.
At the start of any solution, try this. It might speed up any method
>> but would take some time at the start.
get shell( "cat " & ThisFile & " > dev/null" )
I think that is likely to pre-load the system file buffers for you.
If one is feeling adventurous, one can try open process (cat) to avoid
>> the wait; it will probably move through the file faster than the script and
>> sectors will already be loaded when you ask for them.
Dar
> On Aug 4, 2019, at 7:59 AM, Alex Tweedly via use-livecode <
>> use-livecode@lists.runrev.com> wrote:
>
> OK, here's a "really out there" suggestion
>
> 1. Run a local web server to serve files (locally only).
>
> Can be done various ways, including (easily) via LC and the httpd
>> library,
>
> (build that server as a standalone and have it running - started
>> from your app if need be...)
>
> 2. in your stack, just do
>
> load url ("http://localhost:8080/myfilename;) with message
>> "mycallback"
>
> and handle the file once it has been read in the "mycallback" handler
>
> -- Alex.
>
>
> On 04/08/2019 01:56, Tom Glod via use-livecode wrote:
>> Hey folks,
>>
>> I'm having trouble finding a combination of settings that allows my
>> file
>> loading to seem to happen in the background.
>>
>> repeat while read_result is not "eof"
>> read from file ThisFile for (1024 * 1000) bytes
>> put the result into read_result
>> put it after IntoThisVariable
>> add length(it) to amount_read
>> TSTProgress amount_read,ExpectedSize,"%","Loading File ..."
>> (Progress Indicator Handler)
>> wait 10 milliseconds with messages
>>end repeat
>>
>> no matter what I try, its still sluggish, and it seems like messages
>> are
>> still accumilating instead of being processed by the engine.
>>
>> Am I missing something? Normally waiting with messages sufficiently
>> frees
>> the engine to allow the UI to remain responsive.
>>
>> Thanks,
>>
>> Tom
>> ___
>> use-livecode mailing list
>> use-livecode@lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your
>> subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
>> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe,
Re: Making "read from file" less blocky.
On 04/08/2019 19:07, Tom Glod via use-livecode wrote: Hmmm interesting. I was sending binary variables to it, and the headers came through ok, but the binary data didn't when it was over a certain size. What sort of data sizes have you been been sending to your httpd standalone? Currently I am reading multi-Mbytes *from* the httpd server, and it all seems OK, though I haven't paid that much attention to how large the transfers were. Currently I am only sending small datasets *to* the server, haven't tied anything large. And, in fact, I never will !! What I am writing must run in the community version, and must be completely non-blocking; therefore, *everything* is done with "load URL ...", i.e. the data is being sent as a (rather long) parameter to the HTTP/GET. (because without tsNet there is no non-blocking POST). So in the future when I need to send larger datasets, I will need to stitch the data back together in my server code, and use many "load url ..." calls to do the transfer. Alex. ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Making "read from file" less blocky.
heheh its always a fun exercise.
Dar, these are linux based solutions right? using windows here at the
moment, so I can't test, but when I test and optimize my application for
linux i will try these.
On Sun, Aug 4, 2019 at 4:43 PM dsc--- via use-livecode <
use-livecode@lists.runrev.com> wrote:
> Even more "really out there" of the "really out there".
>
> Don't read in the file.
>
> Access portions JIT, that is, lazily.
>
> Create a function that pulls in segments of the file. Kinda like this:
>
> function segmentOfFile pStartIndex, pEndIndex, pThisFile
>
> Or this:
>
> function segmentOfCurrentFile pStartIndex, pEndIndex
>
> Dar
>
> > On Aug 4, 2019, at 1:22 PM, dsc--- via use-livecode <
> use-livecode@lists.runrev.com> wrote:
> >
> > More "really out there".
> >
> > I like the idea of trying to speed up an upfront foreground load.
> Something simple like this:
> >
> > put blah-plah into IntoThisVariable.
> >
> > where blah-blah is nana-nana or decompress( nana-nana )
> > where nana-nana is one of these:
> > URL ("binfile:" & ThisFile)
> > shell("cat " & ThisFile)
> >
> > Function decompress() makes two RAM hits and requires control over the
> loaded files.
> >
> >
> >> On Aug 4, 2019, at 12:47 PM, Dar Scott Consulting via use-livecode <
> use-livecode@lists.runrev.com> wrote:
> >>
> >> I love "really out there". I wanna play.
> >>
> >> At the start of any solution, try this. It might speed up any method
> but would take some time at the start.
> >>
> >> get shell( "cat " & ThisFile & " > dev/null" )
> >>
> >> I think that is likely to pre-load the system file buffers for you.
> >>
> >> If one is feeling adventurous, one can try open process (cat) to avoid
> the wait; it will probably move through the file faster than the script and
> sectors will already be loaded when you ask for them.
> >>
> >> Dar
> >>
> >>> On Aug 4, 2019, at 7:59 AM, Alex Tweedly via use-livecode <
> use-livecode@lists.runrev.com> wrote:
> >>>
> >>> OK, here's a "really out there" suggestion
> >>>
> >>> 1. Run a local web server to serve files (locally only).
> >>>
> >>>Can be done various ways, including (easily) via LC and the httpd
> library,
> >>>
> >>> (build that server as a standalone and have it running - started
> from your app if need be...)
> >>>
> >>> 2. in your stack, just do
> >>>
> >>> load url ("http://localhost:8080/myfilename;) with message
> "mycallback"
> >>>
> >>> and handle the file once it has been read in the "mycallback" handler
> >>>
> >>> -- Alex.
> >>>
> >>>
> >>> On 04/08/2019 01:56, Tom Glod via use-livecode wrote:
> Hey folks,
>
> I'm having trouble finding a combination of settings that allows my
> file
> loading to seem to happen in the background.
>
> repeat while read_result is not "eof"
> read from file ThisFile for (1024 * 1000) bytes
> put the result into read_result
> put it after IntoThisVariable
> add length(it) to amount_read
> TSTProgress amount_read,ExpectedSize,"%","Loading File ..."
> (Progress Indicator Handler)
> wait 10 milliseconds with messages
> end repeat
>
> no matter what I try, its still sluggish, and it seems like messages
> are
> still accumilating instead of being processed by the engine.
>
> Am I missing something? Normally waiting with messages sufficiently
> frees
> the engine to allow the UI to remain responsive.
>
> Thanks,
>
> Tom
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
> >>>
> >>> ___
> >>> use-livecode mailing list
> >>> use-livecode@lists.runrev.com
> >>> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> >>> http://lists.runrev.com/mailman/listinfo/use-livecode
> >>>
> >>
> >>
> >> ___
> >> use-livecode mailing list
> >> use-livecode@lists.runrev.com
> >> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> >> http://lists.runrev.com/mailman/listinfo/use-livecode
> >>
> >
> >
> > ___
> > use-livecode mailing list
> > use-livecode@lists.runrev.com
> > Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> > http://lists.runrev.com/mailman/listinfo/use-livecode
>
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
Re: Making "read from file" less blocky.
Even more "really out there" of the "really out there".
Don't read in the file.
Access portions JIT, that is, lazily.
Create a function that pulls in segments of the file. Kinda like this:
function segmentOfFile pStartIndex, pEndIndex, pThisFile
Or this:
function segmentOfCurrentFile pStartIndex, pEndIndex
Dar
> On Aug 4, 2019, at 1:22 PM, dsc--- via use-livecode
> wrote:
>
> More "really out there".
>
> I like the idea of trying to speed up an upfront foreground load. Something
> simple like this:
>
> put blah-plah into IntoThisVariable.
>
> where blah-blah is nana-nana or decompress( nana-nana )
> where nana-nana is one of these:
> URL ("binfile:" & ThisFile)
> shell("cat " & ThisFile)
>
> Function decompress() makes two RAM hits and requires control over the loaded
> files.
>
>
>> On Aug 4, 2019, at 12:47 PM, Dar Scott Consulting via use-livecode
>> wrote:
>>
>> I love "really out there". I wanna play.
>>
>> At the start of any solution, try this. It might speed up any method but
>> would take some time at the start.
>>
>> get shell( "cat " & ThisFile & " > dev/null" )
>>
>> I think that is likely to pre-load the system file buffers for you.
>>
>> If one is feeling adventurous, one can try open process (cat) to avoid the
>> wait; it will probably move through the file faster than the script and
>> sectors will already be loaded when you ask for them.
>>
>> Dar
>>
>>> On Aug 4, 2019, at 7:59 AM, Alex Tweedly via use-livecode
>>> wrote:
>>>
>>> OK, here's a "really out there" suggestion
>>>
>>> 1. Run a local web server to serve files (locally only).
>>>
>>>Can be done various ways, including (easily) via LC and the httpd
>>> library,
>>>
>>> (build that server as a standalone and have it running - started from
>>> your app if need be...)
>>>
>>> 2. in your stack, just do
>>>
>>> load url ("http://localhost:8080/myfilename;) with message "mycallback"
>>>
>>> and handle the file once it has been read in the "mycallback" handler
>>>
>>> -- Alex.
>>>
>>>
>>> On 04/08/2019 01:56, Tom Glod via use-livecode wrote:
Hey folks,
I'm having trouble finding a combination of settings that allows my file
loading to seem to happen in the background.
repeat while read_result is not "eof"
read from file ThisFile for (1024 * 1000) bytes
put the result into read_result
put it after IntoThisVariable
add length(it) to amount_read
TSTProgress amount_read,ExpectedSize,"%","Loading File ..."
(Progress Indicator Handler)
wait 10 milliseconds with messages
end repeat
no matter what I try, its still sluggish, and it seems like messages are
still accumilating instead of being processed by the engine.
Am I missing something? Normally waiting with messages sufficiently frees
the engine to allow the UI to remain responsive.
Thanks,
Tom
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your
subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
>>>
>>> ___
>>> use-livecode mailing list
>>> use-livecode@lists.runrev.com
>>> Please visit this url to subscribe, unsubscribe and manage your
>>> subscription preferences:
>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>>
>>
>>
>> ___
>> use-livecode mailing list
>> use-livecode@lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your subscription
>> preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>
>
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Making "read from file" less blocky.
More "really out there".
I like the idea of trying to speed up an upfront foreground load. Something
simple like this:
put blah-plah into IntoThisVariable.
where blah-blah is nana-nana or decompress( nana-nana )
where nana-nana is one of these:
URL ("binfile:" & ThisFile)
shell("cat " & ThisFile)
Function decompress() makes two RAM hits and requires control over the loaded
files.
> On Aug 4, 2019, at 12:47 PM, Dar Scott Consulting via use-livecode
> wrote:
>
> I love "really out there". I wanna play.
>
> At the start of any solution, try this. It might speed up any method but
> would take some time at the start.
>
> get shell( "cat " & ThisFile & " > dev/null" )
>
> I think that is likely to pre-load the system file buffers for you.
>
> If one is feeling adventurous, one can try open process (cat) to avoid the
> wait; it will probably move through the file faster than the script and
> sectors will already be loaded when you ask for them.
>
> Dar
>
>> On Aug 4, 2019, at 7:59 AM, Alex Tweedly via use-livecode
>> wrote:
>>
>> OK, here's a "really out there" suggestion
>>
>> 1. Run a local web server to serve files (locally only).
>>
>> Can be done various ways, including (easily) via LC and the httpd
>> library,
>>
>>(build that server as a standalone and have it running - started from
>> your app if need be...)
>>
>> 2. in your stack, just do
>>
>> load url ("http://localhost:8080/myfilename;) with message "mycallback"
>>
>> and handle the file once it has been read in the "mycallback" handler
>>
>> -- Alex.
>>
>>
>> On 04/08/2019 01:56, Tom Glod via use-livecode wrote:
>>> Hey folks,
>>>
>>> I'm having trouble finding a combination of settings that allows my file
>>> loading to seem to happen in the background.
>>>
>>> repeat while read_result is not "eof"
>>> read from file ThisFile for (1024 * 1000) bytes
>>> put the result into read_result
>>> put it after IntoThisVariable
>>> add length(it) to amount_read
>>> TSTProgress amount_read,ExpectedSize,"%","Loading File ..."
>>> (Progress Indicator Handler)
>>> wait 10 milliseconds with messages
>>> end repeat
>>>
>>> no matter what I try, its still sluggish, and it seems like messages are
>>> still accumilating instead of being processed by the engine.
>>>
>>> Am I missing something? Normally waiting with messages sufficiently frees
>>> the engine to allow the UI to remain responsive.
>>>
>>> Thanks,
>>>
>>> Tom
>>> ___
>>> use-livecode mailing list
>>> use-livecode@lists.runrev.com
>>> Please visit this url to subscribe, unsubscribe and manage your
>>> subscription preferences:
>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>
>> ___
>> use-livecode mailing list
>> use-livecode@lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your subscription
>> preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>
>
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Making "read from file" less blocky.
I love "really out there". I wanna play.
At the start of any solution, try this. It might speed up any method but would
take some time at the start.
get shell( "cat " & ThisFile & " > dev/null" )
I think that is likely to pre-load the system file buffers for you.
If one is feeling adventurous, one can try open process (cat) to avoid the
wait; it will probably move through the file faster than the script and sectors
will already be loaded when you ask for them.
Dar
> On Aug 4, 2019, at 7:59 AM, Alex Tweedly via use-livecode
> wrote:
>
> OK, here's a "really out there" suggestion
>
> 1. Run a local web server to serve files (locally only).
>
> Can be done various ways, including (easily) via LC and the httpd
> library,
>
> (build that server as a standalone and have it running - started from
> your app if need be...)
>
> 2. in your stack, just do
>
> load url ("http://localhost:8080/myfilename;) with message "mycallback"
>
> and handle the file once it has been read in the "mycallback" handler
>
> -- Alex.
>
>
> On 04/08/2019 01:56, Tom Glod via use-livecode wrote:
>> Hey folks,
>>
>> I'm having trouble finding a combination of settings that allows my file
>> loading to seem to happen in the background.
>>
>> repeat while read_result is not "eof"
>> read from file ThisFile for (1024 * 1000) bytes
>> put the result into read_result
>> put it after IntoThisVariable
>> add length(it) to amount_read
>> TSTProgress amount_read,ExpectedSize,"%","Loading File ..."
>> (Progress Indicator Handler)
>> wait 10 milliseconds with messages
>> end repeat
>>
>> no matter what I try, its still sluggish, and it seems like messages are
>> still accumilating instead of being processed by the engine.
>>
>> Am I missing something? Normally waiting with messages sufficiently frees
>> the engine to allow the UI to remain responsive.
>>
>> Thanks,
>>
>> Tom
>> ___
>> use-livecode mailing list
>> use-livecode@lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your subscription
>> preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Making "read from file" less blocky.
Hmmm interesting.
I was sending binary variables to it, and the headers came through ok, but
the binary data didn't when it was over a certain size.
What sort of data sizes have you been been sending to your httpd standalone?
My tests, were 18 months ago at least, but I am hoping it was just me doing
something wrong. I did see a "//todo" in the library code under a section
that looked like it dealt with multiple chunks.
On Sun, Aug 4, 2019 at 1:21 PM Alex Tweedly via use-livecode <
use-livecode@lists.runrev.com> wrote:
> Oh, that's a bit worrying I'm just starting a project that will use
> httpd, and it might in the medium term need to receive large data sets.
>
> But for now, similar to this case of yours for "serving" files, it only
> needs to send large data sets, and I have tested that pretty thoroughly
> already so I'm comfortable it'll do that just fine.
>
> Alex.
>
> On 04/08/2019 18:12, Tom Glod via use-livecode wrote:
> > Hi Alex, yes that would definitely be a great option for a high
> performance
> > solution that would work well in the background.
> >
> > I did tests on such a solution a while back (for a similar task), but
> found
> > that the httpd library was not able to receive large pieces of binary
> > data. It worked beautiful with small chunks.
> >
> > So I don't know if it was me, if it was a missing feature in the library,
> > but I will be investigating it again soon enough.
> >
> > Thanks for the reminder
> >
> > On Sun, Aug 4, 2019 at 9:59 AM Alex Tweedly via use-livecode <
> > use-livecode@lists.runrev.com> wrote:
> >
> >> OK, here's a "really out there" suggestion
> >>
> >> 1. Run a local web server to serve files (locally only).
> >>
> >>Can be done various ways, including (easily) via LC and the httpd
> >> library,
> >>
> >> (build that server as a standalone and have it running -
> >> started from your app if need be...)
> >>
> >> 2. in your stack, just do
> >>
> >> load url ("http://localhost:8080/myfilename;) with message
> "mycallback"
> >>
> >> and handle the file once it has been read in the "mycallback" handler
> >>
> >> -- Alex.
> >>
> >>
> >> On 04/08/2019 01:56, Tom Glod via use-livecode wrote:
> >>> Hey folks,
> >>>
> >>> I'm having trouble finding a combination of settings that allows my
> file
> >>> loading to seem to happen in the background.
> >>>
> >>> repeat while read_result is not "eof"
> >>>read from file ThisFile for (1024 * 1000) bytes
> >>>put the result into read_result
> >>>put it after IntoThisVariable
> >>>add length(it) to amount_read
> >>>TSTProgress amount_read,ExpectedSize,"%","Loading File ..."
> >>> (Progress Indicator Handler)
> >>>wait 10 milliseconds with messages
> >>> end repeat
> >>>
> >>> no matter what I try, its still sluggish, and it seems like messages
> are
> >>> still accumilating instead of being processed by the engine.
> >>>
> >>> Am I missing something? Normally waiting with messages sufficiently
> >> frees
> >>> the engine to allow the UI to remain responsive.
> >>>
> >>> Thanks,
> >>>
> >>> Tom
> >>> ___
> >>> use-livecode mailing list
> >>> use-livecode@lists.runrev.com
> >>> Please visit this url to subscribe, unsubscribe and manage your
> >> subscription preferences:
> >>> http://lists.runrev.com/mailman/listinfo/use-livecode
> >> ___
> >> use-livecode mailing list
> >> use-livecode@lists.runrev.com
> >> Please visit this url to subscribe, unsubscribe and manage your
> >> subscription preferences:
> >> http://lists.runrev.com/mailman/listinfo/use-livecode
> > ___
> > use-livecode mailing list
> > use-livecode@lists.runrev.com
> > Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> > http://lists.runrev.com/mailman/listinfo/use-livecode
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Making "read from file" less blocky.
Oh, that's a bit worrying I'm just starting a project that will use
httpd, and it might in the medium term need to receive large data sets.
But for now, similar to this case of yours for "serving" files, it only
needs to send large data sets, and I have tested that pretty thoroughly
already so I'm comfortable it'll do that just fine.
Alex.
On 04/08/2019 18:12, Tom Glod via use-livecode wrote:
Hi Alex, yes that would definitely be a great option for a high performance
solution that would work well in the background.
I did tests on such a solution a while back (for a similar task), but found
that the httpd library was not able to receive large pieces of binary
data. It worked beautiful with small chunks.
So I don't know if it was me, if it was a missing feature in the library,
but I will be investigating it again soon enough.
Thanks for the reminder
On Sun, Aug 4, 2019 at 9:59 AM Alex Tweedly via use-livecode <
use-livecode@lists.runrev.com> wrote:
OK, here's a "really out there" suggestion
1. Run a local web server to serve files (locally only).
Can be done various ways, including (easily) via LC and the httpd
library,
(build that server as a standalone and have it running -
started from your app if need be...)
2. in your stack, just do
load url ("http://localhost:8080/myfilename;) with message "mycallback"
and handle the file once it has been read in the "mycallback" handler
-- Alex.
On 04/08/2019 01:56, Tom Glod via use-livecode wrote:
Hey folks,
I'm having trouble finding a combination of settings that allows my file
loading to seem to happen in the background.
repeat while read_result is not "eof"
read from file ThisFile for (1024 * 1000) bytes
put the result into read_result
put it after IntoThisVariable
add length(it) to amount_read
TSTProgress amount_read,ExpectedSize,"%","Loading File ..."
(Progress Indicator Handler)
wait 10 milliseconds with messages
end repeat
no matter what I try, its still sluggish, and it seems like messages are
still accumilating instead of being processed by the engine.
Am I missing something? Normally waiting with messages sufficiently
frees
the engine to allow the UI to remain responsive.
Thanks,
Tom
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your
subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your
subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Making "read from file" less blocky.
Hi Alex, yes that would definitely be a great option for a high performance
solution that would work well in the background.
I did tests on such a solution a while back (for a similar task), but found
that the httpd library was not able to receive large pieces of binary
data. It worked beautiful with small chunks.
So I don't know if it was me, if it was a missing feature in the library,
but I will be investigating it again soon enough.
Thanks for the reminder
On Sun, Aug 4, 2019 at 9:59 AM Alex Tweedly via use-livecode <
use-livecode@lists.runrev.com> wrote:
> OK, here's a "really out there" suggestion
>
> 1. Run a local web server to serve files (locally only).
>
> Can be done various ways, including (easily) via LC and the httpd
> library,
>
> (build that server as a standalone and have it running -
> started from your app if need be...)
>
> 2. in your stack, just do
>
>load url ("http://localhost:8080/myfilename;) with message "mycallback"
>
> and handle the file once it has been read in the "mycallback" handler
>
> -- Alex.
>
>
> On 04/08/2019 01:56, Tom Glod via use-livecode wrote:
> > Hey folks,
> >
> > I'm having trouble finding a combination of settings that allows my file
> > loading to seem to happen in the background.
> >
> >repeat while read_result is not "eof"
> > read from file ThisFile for (1024 * 1000) bytes
> > put the result into read_result
> > put it after IntoThisVariable
> > add length(it) to amount_read
> > TSTProgress amount_read,ExpectedSize,"%","Loading File ..."
> > (Progress Indicator Handler)
> > wait 10 milliseconds with messages
> >end repeat
> >
> > no matter what I try, its still sluggish, and it seems like messages are
> > still accumilating instead of being processed by the engine.
> >
> > Am I missing something? Normally waiting with messages sufficiently
> frees
> > the engine to allow the UI to remain responsive.
> >
> > Thanks,
> >
> > Tom
> > ___
> > use-livecode mailing list
> > use-livecode@lists.runrev.com
> > Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> > http://lists.runrev.com/mailman/listinfo/use-livecode
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Making "read from file" less blocky.
OK, here's a "really out there" suggestion
1. Run a local web server to serve files (locally only).
Can be done various ways, including (easily) via LC and the httpd
library,
(build that server as a standalone and have it running -
started from your app if need be...)
2. in your stack, just do
load url ("http://localhost:8080/myfilename;) with message "mycallback"
and handle the file once it has been read in the "mycallback" handler
-- Alex.
On 04/08/2019 01:56, Tom Glod via use-livecode wrote:
Hey folks,
I'm having trouble finding a combination of settings that allows my file
loading to seem to happen in the background.
repeat while read_result is not "eof"
read from file ThisFile for (1024 * 1000) bytes
put the result into read_result
put it after IntoThisVariable
add length(it) to amount_read
TSTProgress amount_read,ExpectedSize,"%","Loading File ..."
(Progress Indicator Handler)
wait 10 milliseconds with messages
end repeat
no matter what I try, its still sluggish, and it seems like messages are
still accumilating instead of being processed by the engine.
Am I missing something? Normally waiting with messages sufficiently frees
the engine to allow the UI to remain responsive.
Thanks,
Tom
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Making "read from file" less blocky.
Memory returns slowly… I take back what I said about eof maybe working for serial ports. I’m pretty sure it doesn’t. When I did this before I was taking data from 2 incoming asynchronous serial ports, one then the other then repeat. They only had tiny buffers, so I had to read small pieces of data from each with minimum delays between. When nobody was talking I handled the accumulated data. It it is! .Jerry > On Aug 3, 2019, at 8:02 PM, Tom Glod via use-livecode > wrote: > > Jerry, that simple tweak worked to improve the performance of the UI a > great deal.. its a slower method of loading...but not too slow > to use as background loading. ...Its actually better than what I > expected. Awesome. > I always love to use the forever loop. its just so fun to write. lol > I gotta get some error handling in there but thats another story. > Consider this one SOLVED. Thanks again. > > > > On Sat, Aug 3, 2019 at 10:47 PM Tom Glod wrote: > >> Dar, Yes...I understand.it is a blocking operation no matter what I >> can just introduce breaks in the blockiness. So I'm not expecting a socket >> like experience for sure. >> >> Those are very good suggestions to try thank you. >> >> Jerry, I will give that a try, it looks promising.even a little >> improvement would be better and sufficient for this need, and I can use it >> sparingly. Thanks alot! Onward indeed. >> >> On Sat, Aug 3, 2019 at 10:36 PM Jerry Jensen via use-livecode < >> use-livecode@lists.runrev.com> wrote: >> >>> Hi Tom, >>> >>> I’ve done this in the past watching it instead of the result. I seem to >>> remember eof was not useful. Maybe for a serial port it would be? >>> >>> As in: >>> >>> repeat forever >>> read from file ThisFile for (1024 * 1000) bytes >>> if it is empty then exit repeat >>> — do your stuff >>> wait 0 with messages >>> end repeat >>> >>> Onward, >>> .Jerry non-sphere (tetrahedron?) >>> >>> It was a long time ago in, I think, LC 5, so YMMV. >>> >>> On Aug 3, 2019, at 5:56 PM, Tom Glod via use-livecode < >>> use-livecode@lists.runrev.com> wrote: Hey folks, I'm having trouble finding a combination of settings that allows my file loading to seem to happen in the background. repeat while read_result is not "eof" read from file ThisFile for (1024 * 1000) bytes put the result into read_result put it after IntoThisVariable add length(it) to amount_read TSTProgress amount_read,ExpectedSize,"%","Loading File ..." (Progress Indicator Handler) wait 10 milliseconds with messages end repeat no matter what I try, its still sluggish, and it seems like messages are still accumilating instead of being processed by the engine. Am I missing something? Normally waiting with messages sufficiently >>> frees the engine to allow the UI to remain responsive. Thanks, Tom ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your >>> subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode >>> >>> >>> ___ >>> use-livecode mailing list >>> use-livecode@lists.runrev.com >>> Please visit this url to subscribe, unsubscribe and manage your >>> subscription preferences: >>> http://lists.runrev.com/mailman/listinfo/use-livecode >> >> > ___ > use-livecode mailing list > use-livecode@lists.runrev.com > Please visit this url to subscribe, unsubscribe and manage your subscription > preferences: > http://lists.runrev.com/mailman/listinfo/use-livecode ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Making "read from file" less blocky.
Jerry, that simple tweak worked to improve the performance of the UI a great deal.. its a slower method of loading...but not too slow to use as background loading. ...Its actually better than what I expected. Awesome. I always love to use the forever loop. its just so fun to write. lol I gotta get some error handling in there but thats another story. Consider this one SOLVED. Thanks again. On Sat, Aug 3, 2019 at 10:47 PM Tom Glod wrote: > Dar, Yes...I understand.it is a blocking operation no matter what I > can just introduce breaks in the blockiness. So I'm not expecting a socket > like experience for sure. > > Those are very good suggestions to try thank you. > > Jerry, I will give that a try, it looks promising.even a little > improvement would be better and sufficient for this need, and I can use it > sparingly. Thanks alot! Onward indeed. > > On Sat, Aug 3, 2019 at 10:36 PM Jerry Jensen via use-livecode < > use-livecode@lists.runrev.com> wrote: > >> Hi Tom, >> >> I’ve done this in the past watching it instead of the result. I seem to >> remember eof was not useful. Maybe for a serial port it would be? >> >> As in: >> >> repeat forever >> read from file ThisFile for (1024 * 1000) bytes >> if it is empty then exit repeat >> — do your stuff >> wait 0 with messages >> end repeat >> >> Onward, >> .Jerry non-sphere (tetrahedron?) >> >> It was a long time ago in, I think, LC 5, so YMMV. >> >> >> > On Aug 3, 2019, at 5:56 PM, Tom Glod via use-livecode < >> use-livecode@lists.runrev.com> wrote: >> > >> > Hey folks, >> > >> > I'm having trouble finding a combination of settings that allows my file >> > loading to seem to happen in the background. >> > >> > repeat while read_result is not "eof" >> > read from file ThisFile for (1024 * 1000) bytes >> > put the result into read_result >> > put it after IntoThisVariable >> > add length(it) to amount_read >> > TSTProgress amount_read,ExpectedSize,"%","Loading File ..." >> > (Progress Indicator Handler) >> > wait 10 milliseconds with messages >> > end repeat >> > >> > no matter what I try, its still sluggish, and it seems like messages are >> > still accumilating instead of being processed by the engine. >> > >> > Am I missing something? Normally waiting with messages sufficiently >> frees >> > the engine to allow the UI to remain responsive. >> > >> > Thanks, >> > >> > Tom >> > ___ >> > use-livecode mailing list >> > use-livecode@lists.runrev.com >> > Please visit this url to subscribe, unsubscribe and manage your >> subscription preferences: >> > http://lists.runrev.com/mailman/listinfo/use-livecode >> > >> >> >> ___ >> use-livecode mailing list >> use-livecode@lists.runrev.com >> Please visit this url to subscribe, unsubscribe and manage your >> subscription preferences: >> http://lists.runrev.com/mailman/listinfo/use-livecode > > ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Making "read from file" less blocky.
Dar, Yes...I understand.it is a blocking operation no matter what I can just introduce breaks in the blockiness. So I'm not expecting a socket like experience for sure. Those are very good suggestions to try thank you. Jerry, I will give that a try, it looks promising.even a little improvement would be better and sufficient for this need, and I can use it sparingly. Thanks alot! Onward indeed. On Sat, Aug 3, 2019 at 10:36 PM Jerry Jensen via use-livecode < use-livecode@lists.runrev.com> wrote: > Hi Tom, > > I’ve done this in the past watching it instead of the result. I seem to > remember eof was not useful. Maybe for a serial port it would be? > > As in: > > repeat forever > read from file ThisFile for (1024 * 1000) bytes > if it is empty then exit repeat > — do your stuff > wait 0 with messages > end repeat > > Onward, > .Jerry non-sphere (tetrahedron?) > > It was a long time ago in, I think, LC 5, so YMMV. > > > > On Aug 3, 2019, at 5:56 PM, Tom Glod via use-livecode < > use-livecode@lists.runrev.com> wrote: > > > > Hey folks, > > > > I'm having trouble finding a combination of settings that allows my file > > loading to seem to happen in the background. > > > > repeat while read_result is not "eof" > > read from file ThisFile for (1024 * 1000) bytes > > put the result into read_result > > put it after IntoThisVariable > > add length(it) to amount_read > > TSTProgress amount_read,ExpectedSize,"%","Loading File ..." > > (Progress Indicator Handler) > > wait 10 milliseconds with messages > > end repeat > > > > no matter what I try, its still sluggish, and it seems like messages are > > still accumilating instead of being processed by the engine. > > > > Am I missing something? Normally waiting with messages sufficiently > frees > > the engine to allow the UI to remain responsive. > > > > Thanks, > > > > Tom > > ___ > > use-livecode mailing list > > use-livecode@lists.runrev.com > > Please visit this url to subscribe, unsubscribe and manage your > subscription preferences: > > http://lists.runrev.com/mailman/listinfo/use-livecode > > > > > ___ > use-livecode mailing list > use-livecode@lists.runrev.com > Please visit this url to subscribe, unsubscribe and manage your > subscription preferences: > http://lists.runrev.com/mailman/listinfo/use-livecode ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Making "read from file" less blocky.
Hi Tom, I’ve done this in the past watching it instead of the result. I seem to remember eof was not useful. Maybe for a serial port it would be? As in: repeat forever read from file ThisFile for (1024 * 1000) bytes if it is empty then exit repeat — do your stuff wait 0 with messages end repeat Onward, .Jerry non-sphere (tetrahedron?) It was a long time ago in, I think, LC 5, so YMMV. > On Aug 3, 2019, at 5:56 PM, Tom Glod via use-livecode > wrote: > > Hey folks, > > I'm having trouble finding a combination of settings that allows my file > loading to seem to happen in the background. > > repeat while read_result is not "eof" > read from file ThisFile for (1024 * 1000) bytes > put the result into read_result > put it after IntoThisVariable > add length(it) to amount_read > TSTProgress amount_read,ExpectedSize,"%","Loading File ..." > (Progress Indicator Handler) > wait 10 milliseconds with messages > end repeat > > no matter what I try, its still sluggish, and it seems like messages are > still accumilating instead of being processed by the engine. > > Am I missing something? Normally waiting with messages sufficiently frees > the engine to allow the UI to remain responsive. > > Thanks, > > Tom > ___ > use-livecode mailing list > use-livecode@lists.runrev.com > Please visit this url to subscribe, unsubscribe and manage your subscription > preferences: > http://lists.runrev.com/mailman/listinfo/use-livecode > ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Making "read from file" less blocky.
Alas, only read from socket allows a message to be sent upon completion. The step siblings read from file, read from process and read from driver do not. Here are a few things you might do: 1. Try making the file loading very fast and don't do it in the background. Change the cursor if need be. Try using URL with file: or binfile:, maybe that is fast. 2. Use a "send loop" to read in portions and update a progress bar. You can get help here on how to do that. 3. Process the file lazily and bring in parts as needed. 4. Figure out how to make the file read through networking. Somehow. Maybe. > On Aug 3, 2019, at 6:56 PM, Tom Glod via use-livecode > wrote: > > Hey folks, > > I'm having trouble finding a combination of settings that allows my file > loading to seem to happen in the background. > > repeat while read_result is not "eof" > read from file ThisFile for (1024 * 1000) bytes > put the result into read_result > put it after IntoThisVariable > add length(it) to amount_read > TSTProgress amount_read,ExpectedSize,"%","Loading File ..." > (Progress Indicator Handler) > wait 10 milliseconds with messages > end repeat > > no matter what I try, its still sluggish, and it seems like messages are > still accumilating instead of being processed by the engine. > > Am I missing something? Normally waiting with messages sufficiently frees > the engine to allow the UI to remain responsive. > > Thanks, > > Tom > ___ > use-livecode mailing list > use-livecode@lists.runrev.com > Please visit this url to subscribe, unsubscribe and manage your subscription > preferences: > http://lists.runrev.com/mailman/listinfo/use-livecode ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
Making "read from file" less blocky.
Hey folks, I'm having trouble finding a combination of settings that allows my file loading to seem to happen in the background. repeat while read_result is not "eof" read from file ThisFile for (1024 * 1000) bytes put the result into read_result put it after IntoThisVariable add length(it) to amount_read TSTProgress amount_read,ExpectedSize,"%","Loading File ..." (Progress Indicator Handler) wait 10 milliseconds with messages end repeat no matter what I try, its still sluggish, and it seems like messages are still accumilating instead of being processed by the engine. Am I missing something? Normally waiting with messages sufficiently frees the engine to allow the UI to remain responsive. Thanks, Tom ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
