Re: Go to card has become slow

[Neville Smythe via use-livecode]

Mark Waddingham wrote
> Each paragraph has at least one style run - even if no styles are 
> applied so this is a general thing.
> 
> Most of the text in your generated test (data) stack was unstyled anyway 
> (as it uses 'the text of' to replicate the data) so minimizing those API 
> calls should affect all stacks with a reasonable amount of field data in 
> them.

Ah, my bad. I have updated the SlowSaveWithStyle test stack.

So table fields and datagrids  will indeed be affected. Excellent!

Neville

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Trevor DeVore via use-livecode]

On Mon, Apr 13, 2020 at 7:48 AM Mark Waddingham via use-livecode <
use-livecode@lists.runrev.com> wrote:

> On 2020-04-13 07:06, Trevor DeVore via use-livecode wrote:
> > Will this caching of the DPI setting affect accessing the styledtext
> > property in general on Windows or will it only affect calls to it
> > during
> > saves?
>
> I did a quick test:
>
>- 'the styledText of' is unaffected
>
>- 'the effective styledText if' is about 25% faster
>
> This was using the sample text on card 1 of Neville's test data stack -
> each fetched 1000 times.
>

Thanks for testing. I use `the effective styledText` in a few instances
where it can be called often so it will be nice to get a speed bump on
Windows.

-- 
Trevor DeVore
ScreenSteps
www.screensteps.com
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Mark Waddingham via use-livecode]

On 2020-04-13 07:06, Trevor DeVore via use-livecode wrote:

Will this caching of the DPI setting affect accessing the styledtext
property in general on Windows or will it only affect calls to it 
during

saves?


I did a quick test:

  - 'the styledText of' is unaffected

  - 'the effective styledText if' is about 25% faster

This was using the sample text on card 1 of Neville's test data stack - 
each fetched 1000 times.


Warmest Regards,

Mark.

--
Mark Waddingham ~ m...@livecode.com ~ http://www.livecode.com/
LiveCode: Everyone can create apps

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Trevor DeVore via use-livecode]

On Mon, Apr 13, 2020 at 12:58 AM Mark Waddingham via use-livecode <
use-livecode@lists.runrev.com> wrote:

> On 2020-04-13 02:23, Neville Smythe via use-livecode wrote:
> > But no, it shaved only 1 second off a 7 second save for styled text
> > (elderly MacPro running Windows 10 VM) So your engine modifications
> > must be having a much wider effect, not just on reducing the 3 API
> > calls for styled as opposed to plain text ??
>
> Each paragraph has at least one style run - even if no styles are
> applied so this is a general thing.
>
> Most of the text in your generated test (data) stack was unstyled anyway
> (as it uses 'the text of' to replicate the data) so minimizing those API
> calls should affect all stacks with a reasonable amount of field data in
> them.


Hi Mark,

Will this caching of the DPI setting affect accessing the styledtext
property in general on Windows or will it only affect calls to it during
saves?

-- 
Trevor DeVore
ScreenSteps

>
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Mark Waddingham via use-livecode]

On 2020-04-13 02:23, Neville Smythe via use-livecode wrote:

But no, it shaved only 1 second off a 7 second save for styled text
(elderly MacPro running Windows 10 VM) So your engine modifications
must be having a much wider effect, not just on reducing the 3 API
calls for styled as opposed to plain text ??


Each paragraph has at least one style run - even if no styles are 
applied so this is a general thing.


Most of the text in your generated test (data) stack was unstyled anyway 
(as it uses 'the text of' to replicate the data) so minimizing those API 
calls should affect all stacks with a reasonable amount of field data in 
them.


Warmest Regards,

Mark.

--
Mark Waddingham ~ m...@livecode.com ~ http://www.livecode.com/
LiveCode: Everyone can create apps

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Neville Smythe via use-livecode]

Mark Waddingham wrote
> There appear to be two reasons for the difference:
> 
>   1) When saving each style run (block) in a field the engine is making 
> three Win32 API calls.
> 
>   2) The saving method uses (and always has used) direct Win32 system 
> calls to manipulate files without any buffering (on other platforms we 
> wrap the standard C library stdio FILE which has its own buffer).


Wonderful Mark!

I hadn’t really flagged the fact that I was using styled test in my test set. 
Given your comment that seemed to point to styled text being the bottleneck I 
tweaked my test stack to give the option of plain text. It is available at

https://www.dropbox.com/sh/kgc7u7xpqu63hef/AAAdblOyIcx8SIuurB_QfHyHa?dl=0 


But no, it shaved only 1 second off a 7 second save for styled text (elderly 
MacPro running Windows 10 VM) So your engine modifications must be having a 
much wider effect, not just on reducing the 3 API calls for styled as opposed 
to plain text ??

I hope so because the actual problem in the production stack was large numbers 
of table fields, not styled text. I only used ordinary text fields in the 
sample stack I submitted to QC in order to simplify and isolate the problem. 
Will we get a speed-up for table fields (and DataGrids) when it rolls out?

Neville
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Bob Sneidar via use-livecode]

Mark, you are a freaking computing god! (small G). 

Bob S


> On Apr 11, 2020, at 8:41 AM, Mark Waddingham via use-livecode 
>  wrote:
> 
> On 2020-04-11 13:54, Neville Smythe via use-livecode wrote:
>> Sorry Brian, I uploaded from the wrong folder. Here is the update test
>> set. But the only difference is that you don’t need to hold down the
>> shift key to see the binary save timing.
>> https://www.dropbox.com/sh/cb2r9jbohxqv6bp/AAAQ1weLLlzrKYQ21yn1apf9a?dl=0
>> 
>> Colin, the test set here shows the timing of saving of stacks; the
>> times for Windows 10 are a large multiple of Mac/Unix and we
>> discussing possible reasons before updating the bug report  bug 21305
>>   for Quality
>> Control
> 
> So I also had a look at this today - using Visual Studio's builtin CPU 
> profiling diagnostic when running the engine in release mode.
> 
> There appear to be two reasons for the difference:
> 
>  1) When saving each style run (block) in a field the engine is making three 
> Win32 API calls.
> 
>  2) The saving method uses (and always has used) direct Win32 system calls to 
> manipulate files without any buffering (on other platforms we wrap the 
> standard C library stdio FILE which has its own buffer).
> 
> I tested your very neat little stack with 1323 cards (about 36Mb of data).
> 
> On my machine with the engine as it is it takes:
> 
>  - Windows 10 (VM) [ Community 'develop' ] - 10s
> 
>  - macOS Catalina (Native) [ Business 9.5.0 ] - 0.45s
> 
> After doing a hacky patch to the sites using the Win32 API calls (i.e. 
> caching the results and only calling them once) I got:
> 
>  - Windows 10 (VM) [ Community 'develop' ] - 1.8s
> 
> After adding a very simple write buffer to the Win32 implementation of the 
> system file stream class we have. I got:
> 
>  - Windows 10 (VM) [ Community 'develop' ] - 0.4s
> 
> Anyway, the patches I've made are quite rough at the moment, but I'll 
> endeavour to submit them as PRs in the next few days :)
> 
> Warmest Regards,
> 
> Mark.
> 
> -- 
> Mark Waddingham ~ m...@livecode.com ~ http://www.livecode.com/
> LiveCode: Everyone can create apps

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Bob Sneidar via use-livecode]

Well that would explain why my Datagrid heavy application is a pig saving on 
Windows! 

Bob S


> On Apr 10, 2020, at 8:46 AM, Brian Milby via use-livecode 
>  wrote:
> 
> I'm going to be that guy who replies to himself...
> 
> I just imported a 500kb image to the card, removed the text from the field,
> created 300 cards.
> 0.756 seconds normal (171MB).  1.124 from a variable. (image instead of
> text)
> 8.311 seconds normal (8.2MB).  0.047 from a variable.  (original text)
> 
> I'll need to clean this up with a better image (something I can share) and
> try on my Mac for comparison when I get off today, but this points squarely
> at the serialization of the field object being the difference.
> 
> Thanks,
> Brian


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[scott--- via use-livecode]

Awesomeness! I really appreciate all the posting on this. It was informative to 
observe the process even if I didn’t participate.

--
Scott Morrow

Elementary Software
(Now with 20% less chalk dust!)
web   https://elementarysoftware.com/
email sc...@elementarysoftware.com
booth1-800-615-0867
--

> On Apr 11, 2020, at 8:41 AM, Mark Waddingham via use-livecode 
>  wrote:
> 
> On 2020-04-11 13:54, Neville Smythe via use-livecode wrote:
>> Sorry Brian, I uploaded from the wrong folder. Here is the update test
>> set. But the only difference is that you don’t need to hold down the
>> shift key to see the binary save timing.
>> https://www.dropbox.com/sh/cb2r9jbohxqv6bp/AAAQ1weLLlzrKYQ21yn1apf9a?dl=0
>> 
>> Colin, the test set here shows the timing of saving of stacks; the
>> times for Windows 10 are a large multiple of Mac/Unix and we
>> discussing possible reasons before updating the bug report  bug 21305
>>   for Quality
>> Control
> 
> So I also had a look at this today - using Visual Studio's builtin CPU 
> profiling diagnostic when running the engine in release mode.
> 
> There appear to be two reasons for the difference:
> 
>  1) When saving each style run (block) in a field the engine is making three 
> Win32 API calls.
> 
>  2) The saving method uses (and always has used) direct Win32 system calls to 
> manipulate files without any buffering (on other platforms we wrap the 
> standard C library stdio FILE which has its own buffer).
> 
> I tested your very neat little stack with 1323 cards (about 36Mb of data).
> 
> On my machine with the engine as it is it takes:
> 
>  - Windows 10 (VM) [ Community 'develop' ] - 10s
> 
>  - macOS Catalina (Native) [ Business 9.5.0 ] - 0.45s
> 
> After doing a hacky patch to the sites using the Win32 API calls (i.e. 
> caching the results and only calling them once) I got:
> 
>  - Windows 10 (VM) [ Community 'develop' ] - 1.8s
> 
> After adding a very simple write buffer to the Win32 implementation of the 
> system file stream class we have. I got:
> 
>  - Windows 10 (VM) [ Community 'develop' ] - 0.4s
> 
> Anyway, the patches I've made are quite rough at the moment, but I'll 
> endeavour to submit them as PRs in the next few days :)
> 
> Warmest Regards,
> 
> Mark.
> 
> -- 
> Mark Waddingham ~ m...@livecode.com ~ http://www.livecode.com/
> LiveCode: Everyone can create apps

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Mark Waddingham via use-livecode]

On 2020-04-11 16:41, Mark Waddingham via use-livecode wrote:

Anyway, the patches I've made are quite rough at the moment, but I'll
endeavour to submit them as PRs in the next few days :)


I've pushed a WIP PR with the changes I made to get the results 
mentioned in my last email - it is here for anyone who is interested in 
such things:




The thing which makes the biggest difference is definitely the caching 
of the system metrics used (the first commit in that patch) - with that 
removed and just the write buffer you get what you'd expect - it takes 
9s (rather than 10s).


I'm always highly dubious about rushing anything to do with I/O of any 
sort - so the write buffer might not appear in a build for a while.


The caching of system metrics though should be absolutely fine once I've 
hooked it up to the system message which gets sent when they change - so 
that will hopefully appear much sooner.


Warmest Regards,

Mark.

--
Mark Waddingham ~ m...@livecode.com ~ http://www.livecode.com/
LiveCode: Everyone can create apps

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Mark Wieder via use-livecode]

On 4/11/20 8:41 AM, Mark Waddingham via use-livecode wrote:

After adding a very simple write buffer to the Win32 implementation of 
the system file stream class we have. I got:


   - Windows 10 (VM) [ Community 'develop' ] - 0.4s

Anyway, the patches I've made are quite rough at the moment, but I'll 
endeavour to submit them as PRs in the next few days :)


!

--
 Mark Wieder
 ahsoftw...@gmail.com

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Mark Waddingham via use-livecode]

On 2020-04-11 13:54, Neville Smythe via use-livecode wrote:

Sorry Brian, I uploaded from the wrong folder. Here is the update test
set. But the only difference is that you don’t need to hold down the
shift key to see the binary save timing.

https://www.dropbox.com/sh/cb2r9jbohxqv6bp/AAAQ1weLLlzrKYQ21yn1apf9a?dl=0


Colin, the test set here shows the timing of saving of stacks; the
times for Windows 10 are a large multiple of Mac/Unix and we
discussing possible reasons before updating the bug report  bug 21305
  for Quality
Control


So I also had a look at this today - using Visual Studio's builtin CPU 
profiling diagnostic when running the engine in release mode.


There appear to be two reasons for the difference:

  1) When saving each style run (block) in a field the engine is making 
three Win32 API calls.


  2) The saving method uses (and always has used) direct Win32 system 
calls to manipulate files without any buffering (on other platforms we 
wrap the standard C library stdio FILE which has its own buffer).


I tested your very neat little stack with 1323 cards (about 36Mb of 
data).


On my machine with the engine as it is it takes:

  - Windows 10 (VM) [ Community 'develop' ] - 10s

  - macOS Catalina (Native) [ Business 9.5.0 ] - 0.45s

After doing a hacky patch to the sites using the Win32 API calls (i.e. 
caching the results and only calling them once) I got:


  - Windows 10 (VM) [ Community 'develop' ] - 1.8s

After adding a very simple write buffer to the Win32 implementation of 
the system file stream class we have. I got:


  - Windows 10 (VM) [ Community 'develop' ] - 0.4s

Anyway, the patches I've made are quite rough at the moment, but I'll 
endeavour to submit them as PRs in the next few days :)


Warmest Regards,

Mark.

--
Mark Waddingham ~ m...@livecode.com ~ http://www.livecode.com/
LiveCode: Everyone can create apps

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Brian Milby via use-livecode]

I did 3 additional tests this morning and believe that the issue lies
somewhere in the de/encoding of text or the way that text is streamed to
the file.  My additional tests were using a non-shared text field (due to
the way the text of a shared field is stored in memory, was wondering if
that was a contributor - does not seem to be that significant), putting the
text into a custom property of the card, and setting the script of the card
to the text.  In each of these new tests, the time to save was very close.
The non-BG one was .55 seconds faster but that could just be an artifact of
not doing enough tests on each to get a solid average.

This does make looking into the source code a little easier though since we
can rule out something specific to the field object and can focus on what
is common between the different ways text is streamed to the file.

My tests were on a Win10 laptop (i5-6300 @ 2.4GHZ; 8GB RAM)

On Sat, Apr 11, 2020 at 8:55 AM Neville Smythe via use-livecode <
use-livecode@lists.runrev.com> wrote:

> Sorry Brian, I uploaded from the wrong folder. Here is the update test
> set. But the only difference is that you don’t need to hold down the shift
> key to see the binary save timing.
>
> https://www.dropbox.com/sh/cb2r9jbohxqv6bp/AAAQ1weLLlzrKYQ21yn1apf9a?dl=0
>  >
>
> Colin, the test set here shows the timing of saving of stacks; the times
> for Windows 10 are a large multiple of Mac/Unix and we discussing possible
> reasons before updating the bug report  bug 21305 <
> https://quality.livecode.com/show_bug.cgi?id=21305>  for Quality Control
>
> Neville
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Neville Smythe via use-livecode]

Sorry Brian, I uploaded from the wrong folder. Here is the update test set. But 
the only difference is that you don’t need to hold down the shift key to see 
the binary save timing.

https://www.dropbox.com/sh/cb2r9jbohxqv6bp/AAAQ1weLLlzrKYQ21yn1apf9a?dl=0 


Colin, the test set here shows the timing of saving of stacks; the times for 
Windows 10 are a large multiple of Mac/Unix and we discussing possible reasons 
before updating the bug report  bug 21305 
  for Quality Control

Neville
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Brian Milby via use-livecode]

I just ran another set of tests with a much smaller image.

Text: 5.85s / 0.032s / 8.183708MB
Image: 0.211s / 0.034s / 9.29539MB

One other thing to try tomorrow is to use an individual text field on each
card instead of a shared group.  I tried to download the update, but still
got the original version.  I could put mine up on GitHub with permission.

On Fri, Apr 10, 2020 at 10:54 PM Neville Smythe via use-livecode <
use-livecode@lists.runrev.com> wrote:

> > On 11 Apr 2020, at 2:00 am, Brian Milby wrote
> >
> > I just imported a 500kb image to the card, removed the text from the
> field,
> > created 300 cards.
> > 0.756 seconds normal (171MB).  1.124 from a variable. (image instead of
> > text)
> > 8.311 seconds normal (8.2MB).  0.047 from a variable.  (original text)
> >
> > I'll need to clean this up with a better image (something I can share)
> and
> > try on my Mac for comparison when I get off today, but this points
> squarely
> > at the serialization of the field object being the difference.
>
> I am intrigued by the fact that save stack (for a stack with on-disk file
> size 171MB) took *less* time than saving its binary data representation.
> Doesn’t that mean that the images were already on disk, and the in-memory
> stack was very much smaller than 171 MB, containing only image references.
> In which case I’m not sure we are comparing like with like, as compared
> with the text version (unless "save stack as newStackFile" might be give
> more comparable info?).
>
> My hunch is that Richard’s point about the Windows write-to-file overhead
> is still the key, not the serialisation. One would to need to compare the
> number of writes used by the engine for the two stack versions. That I do
> not know how to do , but I think the incomparable Brian could do it!
>
> Neville
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Colin Holgate via use-livecode]

I have an interest in this topic, and even some ideas, but I’ve been swamped 
with other demands and not able to read every post. Is there a short way to say 
where you are at with the discussion, or a test script that will show a problem?
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Neville Smythe via use-livecode]

> On 11 Apr 2020, at 2:00 am, Brian Milby wrote
> 
> I just imported a 500kb image to the card, removed the text from the field,
> created 300 cards.
> 0.756 seconds normal (171MB).  1.124 from a variable. (image instead of
> text)
> 8.311 seconds normal (8.2MB).  0.047 from a variable.  (original text)
> 
> I'll need to clean this up with a better image (something I can share) and
> try on my Mac for comparison when I get off today, but this points squarely
> at the serialization of the field object being the difference.

I am intrigued by the fact that save stack (for a stack with on-disk file size 
171MB) took *less* time than saving its binary data representation. Doesn’t 
that mean that the images were already on disk, and the in-memory stack was 
very much smaller than 171 MB, containing only image references. In which case 
I’m not sure we are comparing like with like, as compared with the text version 
(unless "save stack as newStackFile" might be give more comparable info?).

My hunch is that Richard’s point about the Windows write-to-file overhead is 
still the key, not the serialisation. One would to need to compare the number 
of writes used by the engine for the two stack versions. That I do not know how 
to do , but I think the incomparable Brian could do it!

Neville
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Richard Gaskin via use-livecode]

Brian Milby wrote:

> I'm going to be that guy who replies to himself...
>
> I just imported a 500kb image to the card, removed the text from the
> field, created 300 cards. 0.756 seconds normal (171MB).  1.124 from a
> variable. (image instead of text) 8.311 seconds normal (8.2MB).  0.047
> from a variable.  (original text)
>
> I'll need to clean this up with a better image (something I can share)
> and try on my Mac for comparison when I get off today, but this points
> squarely at the serialization of the field object being the
> difference.

Thanks for exploring this, Brian.  It's mystifying how such generalized 
code could have such a pronounced impact on only one platform, but with 
your assistance I'm more confident we'll find it.


--
 Richard Gaskin
 Fourth World Systems
 Software Design and Development for the Desktop, Mobile, and the Web
 
 ambassa...@fourthworld.comhttp://www.FourthWorld.com


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Brian Milby via use-livecode]

I'm going to be that guy who replies to himself...

I just imported a 500kb image to the card, removed the text from the field,
created 300 cards.
0.756 seconds normal (171MB).  1.124 from a variable. (image instead of
text)
8.311 seconds normal (8.2MB).  0.047 from a variable.  (original text)

I'll need to clean this up with a better image (something I can share) and
try on my Mac for comparison when I get off today, but this points squarely
at the serialization of the field object being the difference.

Thanks,
Brian

On Fri, Apr 10, 2020 at 9:48 AM Brian Milby  wrote:

> That's a good question.  It has been over a year since I looked at that
> code, but I can't think of anything in the serialization piece that would
> be an issue.  That code is all common (writing to a file buffer).  I did
> not look at the actual platform file code though - could be something
> there.  In this example, there is a good bit of text that is being
> serialized.  It may be a good test to generate a file the same size but
> with binary data (images) to see if it has anything to do with text
> serialization.
>
> On Fri, Apr 10, 2020 at 12:00 AM Richard Gaskin via use-livecode <
> use-livecode@lists.runrev.com> wrote:
>
>> Earlier I wrote:
>>  > Bob Sneidar wrote:
>>  >
>>  >  >> On Apr 8, 2020, at 11:16 PM, Richard Gaskin wrote:
>>  >  >>
>>  >  >> Better still, see the last comment on that page. :)
>>  >  >>
>>  >  >> It kinda fits Brian's observation about how the stack file is
>>  >  >> written in a series of small writes, one per object.
>>  >  >
>>  >  > Yes, but I have saved stacks with both Defender and Kaspersky
>>  >  > disabled. In our case I do not think that is the entire issue.
>>  >
>>  > Probably not exclusively. But to assess the contributory effect, what
>>  > are your measurements using Neville's test with and without those AV
>>  > packages active?
>>
>> It may not be worth the trouble.
>>
>> When I run Neville's stack saving test here with Windows Defender's RTP
>> on the 8MB stack save takes a little over 8 seconds.
>>
>> And when I turn RTP off it takes a bit over 7 seconds.
>>
>> So while RTP is a clear conributor, it doesn't account for most of the
>> speed loss on stack saves.
>>
>>
>> @Brian Milby: When you were looking up the stack serialization in the
>> code, did you see anything which might explain why this slowdown is only
>> evident on Windows but not Mac or Linux?
>>
>> --
>>   Richard Gaskin
>>   Fourth World Systems
>>   Software Design and Development for the Desktop, Mobile, and the Web
>>   
>>   ambassa...@fourthworld.comhttp://www.FourthWorld.com
>>
>>
>> ___
>> use-livecode mailing list
>> use-livecode@lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your
>> subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>
>
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Brian Milby via use-livecode]

That's a good question.  It has been over a year since I looked at that
code, but I can't think of anything in the serialization piece that would
be an issue.  That code is all common (writing to a file buffer).  I did
not look at the actual platform file code though - could be something
there.  In this example, there is a good bit of text that is being
serialized.  It may be a good test to generate a file the same size but
with binary data (images) to see if it has anything to do with text
serialization.

On Fri, Apr 10, 2020 at 12:00 AM Richard Gaskin via use-livecode <
use-livecode@lists.runrev.com> wrote:

> Earlier I wrote:
>  > Bob Sneidar wrote:
>  >
>  >  >> On Apr 8, 2020, at 11:16 PM, Richard Gaskin wrote:
>  >  >>
>  >  >> Better still, see the last comment on that page. :)
>  >  >>
>  >  >> It kinda fits Brian's observation about how the stack file is
>  >  >> written in a series of small writes, one per object.
>  >  >
>  >  > Yes, but I have saved stacks with both Defender and Kaspersky
>  >  > disabled. In our case I do not think that is the entire issue.
>  >
>  > Probably not exclusively. But to assess the contributory effect, what
>  > are your measurements using Neville's test with and without those AV
>  > packages active?
>
> It may not be worth the trouble.
>
> When I run Neville's stack saving test here with Windows Defender's RTP
> on the 8MB stack save takes a little over 8 seconds.
>
> And when I turn RTP off it takes a bit over 7 seconds.
>
> So while RTP is a clear conributor, it doesn't account for most of the
> speed loss on stack saves.
>
>
> @Brian Milby: When you were looking up the stack serialization in the
> code, did you see anything which might explain why this slowdown is only
> evident on Windows but not Mac or Linux?
>
> --
>   Richard Gaskin
>   Fourth World Systems
>   Software Design and Development for the Desktop, Mobile, and the Web
>   
>   ambassa...@fourthworld.comhttp://www.FourthWorld.com
>
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Richard Gaskin via use-livecode]

Earlier I wrote:
> Bob Sneidar wrote:
>
>  >> On Apr 8, 2020, at 11:16 PM, Richard Gaskin wrote:
>  >>
>  >> Better still, see the last comment on that page. :)
>  >>
>  >> It kinda fits Brian's observation about how the stack file is
>  >> written in a series of small writes, one per object.
>  >
>  > Yes, but I have saved stacks with both Defender and Kaspersky
>  > disabled. In our case I do not think that is the entire issue.
>
> Probably not exclusively. But to assess the contributory effect, what
> are your measurements using Neville's test with and without those AV
> packages active?

It may not be worth the trouble.

When I run Neville's stack saving test here with Windows Defender's RTP 
on the 8MB stack save takes a little over 8 seconds.


And when I turn RTP off it takes a bit over 7 seconds.

So while RTP is a clear conributor, it doesn't account for most of the 
speed loss on stack saves.



@Brian Milby: When you were looking up the stack serialization in the 
code, did you see anything which might explain why this slowdown is only 
evident on Windows but not Mac or Linux?


--
 Richard Gaskin
 Fourth World Systems
 Software Design and Development for the Desktop, Mobile, and the Web
 
 ambassa...@fourthworld.comhttp://www.FourthWorld.com


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Richard Gaskin via use-livecode]

Bob Sneidar wrote:

>> On Apr 8, 2020, at 11:16 PM, Richard Gaskin wrote:
>>
>> Better still, see the last comment on that page. :)
>>
>> It kinda fits Brian's observation about how the stack file is
>> written in a series of small writes, one per object.
>
> Yes, but I have saved stacks with both Defender and Kaspersky
> disabled. In our case I do not think that is the entire issue.

Probably not exclusively. But to assess the contributory effect, what 
are your measurements using Neville's test with and without those AV 
packages active?


--
 Richard Gaskin
 Fourth World Systems
 Software Design and Development for the Desktop, Mobile, and the Web
 
 ambassa...@fourthworld.comhttp://www.FourthWorld.com


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Bob Sneidar via use-livecode]

Are we saying we can use binFile to save stacks as opposed to the normal way? 
With no side effects??

Bob S


> On Apr 8, 2020, at 7:04 AM, Bob Sneidar  wrote:
> 
>>   put url ("binfile:"&fName) into tData
>>   put the long seconds into t
>>   put tData into url ("binfile:"& fName)
>>   put the long seconds - t


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Bob Sneidar via use-livecode]

Yes, but I have saved stacks with both Defender and Kaspersky disabled. In our 
case I do not think that is the entire issue. 

Bob S


> On Apr 8, 2020, at 11:16 PM, Richard Gaskin via use-livecode 
>  wrote:
> 
> Better still, see the last comment on that page. :)
> 
> It kinda fits Brian's observation about how the stack file is written in a 
> series of small writes, one per object.
> 
> -- 
> Richard Gaskin


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

RE: Go to card has become slow

[Ralph DiMola via use-livecode]

Why would saving be so much faster on a Mac than Windows when writing to the 
same a SMB served by a Win 10 PC? I looks like the Windows problem is at higher 
abstraction layer than SMB.

Ralph DiMola
IT Director
Evergreen Information Services
rdim...@evergreeninfo.net

-Original Message-
From: use-livecode [mailto:use-livecode-boun...@lists.runrev.com] On Behalf Of 
Neville Smythe via use-livecode
Sent: Thursday, April 09, 2020 1:08 AM
To: use-livecode@lists.runrev.com
Cc: Neville Smythe
Subject: Re: Go to card has become slow

I have updated the SlowSave test stack with Richard's enhancement to also show 
the time to save the binary data (no need to use the shift button)

https://www.dropbox.com/sh/cb2r9jbohxqv6bp/AAAQ1weLLlzrKYQ21yn1apf9a?dl=0

Pleased to see everyone is seeing the problem at last, it doesn’t seem to be 
related to the particular Windows 10 installation. But 38 seconds for 8 MB!! 

Neville
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Richard Gaskin via use-livecode]

The speed difference with stack files seems to be that there's high 
overhead for each write on Windows 10, and stack files are written on a 
series of small writes.


Given this, which would make more sense?:

a) Serialize objects to one buffer in memory
   and flush to disk in one write
   (changes current behavior, but I can't think
of why anyone would mind)

b) Extend the "save file" command with a flag
   to tell the engine to flush in one write
   (keeps current default behavior, allowing
   the single-flush method as an option)

--
 Richard Gaskin
 Fourth World Systems
 Software Design and Development for the Desktop, Mobile, and the Web
 
 ambassa...@fourthworld.comhttp://www.FourthWorld.com

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Richard Gaskin via use-livecode]

Neville Smythe wrote:

> I have updated the SlowSave test stack with Richard's enhancement to
> also show the time to save the binary data (no need to use the shift
> button)
>
> https://www.dropbox.com/sh/cb2r9jbohxqv6bp/AAAQ1weLLlzrKYQ21yn1apf9a?dl=0
>
> Pleased to see everyone is seeing the problem at last, it doesn’t seem
> to be related to the particular Windows 10 installation. But 38
> seconds for 8 MB!!

Yes, the difference is quite pronounced.

Thank you for updating the stack.

Brian Milby's comment was most illuminating:

   Internally, each object is serialized and written to the
   file buffer in turn.  So the question then becomes how
   much of the difference is due to writing the file in
   pieces compared to the process of serializing the data.


I just did a search at DuckDuckGo for "difference in write speed windows 
vs mac programming":


https://stackoverflow.com/questions/48356177/c-program-runs-much-slower-on-windows-vs-linux-when-doing-file-operations


Among the first results was this discussion from stack overflow, where 
the problem is described as:


  When you execute a build operation with ChatScript, it scans
  all the script files that comprise your chat-bot. In my case,
  that's hundreds of files. This process takes nearly 30 times
  longer on Windows 8.1 than it does on Ubuntu 16.04. Therefore
  I do use Linux for much of my work, but there is a part of
  my work that I have to do on Windows because of certain
  associated tools, so I would like to modify the code base so
  that Windows ChatScript compiles are as fast as on Linux.

  Can anyone think of a reason the code would run so much slower
  on Windows vs. Linux? Are there some C++ file operation codes
  (read/write/etc.) that are known to be much slower on Windows
  compared to Linux due to variances in the C++ run-time libraries
  running on each platform?

Sounds kinda like us, no?

But it gets better.

The first reply suggests a few common things to speed up disk I/O, like 
defragging and double-checking compiler options, but then he kinda gives 
up and says, "But doing all those things for me has never got the 
Windows compile to be quicker than on linux using equivalent disk 
hardware, not once."


Better still, see the last comment on that page. :)

It kinda fits Brian's observation about how the stack file is written in 
a series of small writes, one per object.


--
 Richard Gaskin
 Fourth World Systems
 Software Design and Development for the Desktop, Mobile, and the Web
 
 ambassa...@fourthworld.comhttp://www.FourthWorld.com



___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Neville Smythe via use-livecode]

I have updated the SlowSave test stack with Richard's enhancement to also show 
the time to save the binary data (no need to use the shift button)

https://www.dropbox.com/sh/cb2r9jbohxqv6bp/AAAQ1weLLlzrKYQ21yn1apf9a?dl=0

Pleased to see everyone is seeing the problem at last, it doesn’t seem to be 
related to the particular Windows 10 installation. But 38 seconds for 8 MB!! 

Neville
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Neville Smythe via use-livecode]

Richard:

OK, so my impression, that LC saving a (binary) file on Windows 10 is not the 
problem, is correct, as you deduce it lies in the serialisation of the stack.

Now why would that be platform, indeed OS, dependent? Could the LC 
cache/virtual memory settings be different for Windows 10 from those for MacOS 
or Linux? But if that were the case you would expect to see a slow down in 
other parts of LC, such as perhaps script editing in the IDE, and that doesn’t 
happen, does it (!!)

Neville
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Brian Milby via use-livecode]

On my Win10 Dell (i5):

6.154 to 6.806 without shift
0.036 to 0.15 with shift

This is measuring a difference in doing a bulk 8MB write with a streamed
write of the stack to the file piece by piece.  Internally, each object is
serialized and written to the file buffer in turn.  So the question then
becomes how much of the difference is due to writing the file in pieces
compared to the process of serializing the data.

Brian
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Niggemann, Bernd via use-livecode]

Saving the 300 cards 8.x MB stack on a 2017 MacBook Pro SSD:

0.127276 seconds

Same stack saving as binfile:

0.013656 seconds

Kind regards
Bernd

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Richard Gaskin via use-livecode]

Ralph DiMola wrote:

> 0.033 shift key down
> 28-30 seconds shift key up.

It's almost like the stack file takes longer, if only the difference 
were more clearly evident. :)


I'll include your results with the test data I'll add to the report. 
Thanks for running that.


--
 Richard Gaskin
 Fourth World Systems
 Software Design and Development for the Desktop, Mobile, and the Web
 
 ambassa...@fourthworld.comhttp://www.FourthWorld.com


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

RE: Go to card has become slow

[Ralph DiMola via use-livecode]

0.033 shift key down
28-30 seconds shift key up.

Ralph DiMola
IT Director
Evergreen Information Services
rdim...@evergreeninfo.net


-Original Message-
From: use-livecode [mailto:use-livecode-boun...@lists.runrev.com] On Behalf
Of Richard Gaskin via use-livecode
Sent: Wednesday, April 08, 2020 1:46 PM
To: use-livecode@lists.runrev.com
Cc: Richard Gaskin
Subject: Re: Go to card has become slow

Ralph DiMola wrote:

> LC 9.5.1
> Win 10 Hyper-V VM with 8 processors assigned Stack and data on smb 
> served by VM host.
> Host ==> i7 6700 3.40Ghz 16 Gig
> 
> saving data took  38.46 seconds
> 8.183708 MB
> 
> 
> 
> LC 9.5.1
> MacBook Pro Early 2011 OSX 10.13.1 16 Gig Stack and data on smb served 
> by the aforementioned Win 10 VM host.
> 
> saving data took  0.400926 seconds
> 8.183708 MB

Thanks for testing that, Ralph.

If you use this script, what do you get with a second run with the Shift key
down to measure only a straight write, separate from the "save stack"
routine?:


on mouseUp
put word 2 of the long name of stack "data" into fName
replace quote with "" in fName
if the shiftKey is "up" then
  put the long seconds into t0
  save stack "data"
  put the long seconds - t0 into t1
  put "saving data took " && t1 && "seconds" into fld "timeInfo"
  put 0.01*(the length of URL ("file:" & fName)) && "MB" into fld
"lengthInfo"
else
  put url ("binfile:"&fName) into tData
  put the long seconds into t
  put tData into url ("binfile:"& fName)
  put the long seconds - t
end if
end mouseUp



--
  Richard Gaskin
  Fourth World Systems
  Software Design and Development for the Desktop, Mobile, and the Web
  
  ambassa...@fourthworld.comhttp://www.FourthWorld.com

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Richard Gaskin via use-livecode]

Ralph DiMola wrote:


LC 9.5.1
Win 10 Hyper-V VM with 8 processors assigned
Stack and data on smb served by VM host.
Host ==> i7 6700 3.40Ghz 16 Gig

saving data took  38.46 seconds
8.183708 MB



LC 9.5.1
MacBook Pro Early 2011 OSX 10.13.1 16 Gig
Stack and data on smb served by the aforementioned Win 10 VM host.

saving data took  0.400926 seconds
8.183708 MB


Thanks for testing that, Ralph.

If you use this script, what do you get with a second run with the Shift 
key down to measure only a straight write, separate from the "save 
stack" routine?:



on mouseUp
   put word 2 of the long name of stack "data" into fName
   replace quote with "" in fName
   if the shiftKey is "up" then
 put the long seconds into t0
 save stack "data"
 put the long seconds - t0 into t1
 put "saving data took " && t1 && "seconds" into fld "timeInfo"
 put 0.01*(the length of URL ("file:" & fName)) && "MB" into fld
"lengthInfo"
   else
 put url ("binfile:"&fName) into tData
 put the long seconds into t
 put tData into url ("binfile:"& fName)
 put the long seconds - t
   end if
end mouseUp



--
 Richard Gaskin
 Fourth World Systems
 Software Design and Development for the Desktop, Mobile, and the Web
 
 ambassa...@fourthworld.comhttp://www.FourthWorld.com

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

RE: Go to card has become slow

[Ralph DiMola via use-livecode]

LC 9.5.1
Win 10 Hyper-V VM with 8 processors assigned
Stack and data on smb served by VM host.
Host ==> i7 6700 3.40Ghz 16 Gig

saving data took  38.46 seconds
8.183708 MB



LC 9.5.1
MacBook Pro Early 2011 OSX 10.13.1 16 Gig
Stack and data on smb served by the aforementioned Win 10 VM host.

saving data took  0.400926 seconds
8.183708 MB


Ralph DiMola
IT Director
Evergreen Information Services
rdim...@evergreeninfo.net

-Original Message-
From: use-livecode [mailto:use-livecode-boun...@lists.runrev.com] On Behalf Of 
Neville Smythe via use-livecode
Sent: Tuesday, April 07, 2020 11:17 PM
To: use-livecode@lists.runrev.com
Cc: Neville Smythe
Subject: Re: Go to card has become slow

Richard

Here is a link to the test stack for testing the speed of save stack

https://www.dropbox.com/sh/cb2r9jbohxqv6bp/AAAQ1weLLlzrKYQ21yn1apf9a?dl=0

I don’t know why the test stacks I supplied disappeared from the QC bug report.

You need the SlowSave.livecode and the data.livecode stacks. They are both 
small, but SlowSave creates a large stack with 300 cards and the lorem ipsum 
text data to build an 8 MB stack, and displays the timing for saving. On my 
MacBook Air it takes 0.13 seconds to save; on Windows 10 installations I have 
seen times from 3 seconds to 10 seconds.

In this test, the stack has just 300 cards, 1 field on each, and lots of ascii 
text (so there should be no problems converting to utf-8 before saving in the 
unlikely event that is happening, and in any case would happen on all 
platforms). I haven’t tested stacks with large numbers of controls or large 
images rather than large text data.

Other apps on Windows 10 certainly don’t display this behaviour, which  
indicates it is a LC problem not a Windows problem. Antivirus software and 
Defender were off during the test. I was under the impression that saving files 
from LC, rather than stacks, was not slower than normal, but Bob’s recently 
experience with saving data to a database may contradict that.

Some people have had trouble verifying my observations (other LC Forum users 
have confirmed). QC originally could not confirm the bug until they tried it 
“on an older PC”. However one of my users recently updated to a *very* fast HP 
box; it cut the save time down significantly but was still at least 20 times 
slower than on a typical Mac, ie almost usable. So maybe it has something to do 
with the Windows installation, though the ones I used were stock standard. I 
can’t see how fonts could be involved, for example. 

Neville
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Bob Sneidar via use-livecode]

Interesting results Richard. I hope the mothership is monitoring this thread. 
I’ll give the test a go later today. I have to go out (believe it or not) to 
some customers this morning and early afternoon. 

Bob S


> On Apr 7, 2020, at 10:56 PM, Richard Gaskin via use-livecode 
>  wrote:
> 
> Great test set, Neville. Thanks for posting that link.
> 
> The hardware on my Linux and Win boxes is so different I don't have a strong 
> opinion there. But I did modify your main test script to see what I might 
> learn from isolating the file I/O from the stack serialization:
> 
> on mouseUp
>  put word 2 of the long name of stack "data" into fName
>  replace quote with "" in fName
>  if the shiftKey is "up" then
>put the long seconds into t0
>save stack "data"
>put the long seconds - t0 into t1
>put "saving data took " && t1 && "seconds" into fld "timeInfo"
>put 0.01*(the length of URL ("file:" & fName)) && "MB" into fld 
> "lengthInfo"
>  else
>put url ("binfile:"&fName) into tData
>put the long seconds into t
>put tData into url ("binfile:"& fName)
>put the long seconds - t
>  end if
> end mouseUp
> 
> So I ran your tool, making a stack with 300 cards, which produces a file a 
> bit over 8MB.  These are my times:
> 
> Ubuntu 18.04 w/SSD
> Stack: 0.097182
> File:  0.008475
> 
> Windows 10 w/HDD
> Stack: 8.183708
> File:  0.014
> 
> Even with the differences in the hardware, it looks like the bottleneck is in 
> the stack serialization, and the file I/O seems good.
> 
> Anyone here in a position to run Neville's test with this modification on 
> similarly-configured systems but different OSes?
> 
> -- 
> Richard Gaskin
> Fourth World Systems
> 
> 
> Neville Smythe wrote:
> 
>> Richard
>> Here is a link to the test stack for testing the speed of save stack
>> https://www.dropbox.com/sh/cb2r9jbohxqv6bp/AAAQ1weLLlzrKYQ21yn1apf9a?dl=0
>> I don’t know why the test stacks I supplied disappeared from the QC bug 
>> report.
>> You need the SlowSave.livecode and the data.livecode stacks. They are both 
>> small, but SlowSave creates a large stack with 300 cards and the lorem ipsum 
>> text data to build an 8 MB stack, and displays the timing for saving. On my 
>> MacBook Air it takes 0.13 seconds to save; on Windows 10 installations I 
>> have seen times from 3 seconds to 10 seconds.
>> In this test, the stack has just 300 cards, 1 field on each, and lots of 
>> ascii text (so there should be no problems converting to utf-8 before saving 
>> in the unlikely event that is happening, and in any case would happen on all 
>> platforms). I haven’t tested stacks with large numbers of controls or large 
>> images rather than large text data.
>> Other apps on Windows 10 certainly don’t display this behaviour, which  
>> indicates it is a LC problem not a Windows problem. Antivirus software and 
>> Defender were off during the test. I was under the impression that saving 
>> files from LC, rather than stacks, was not slower than normal, but Bob’s 
>> recently experience with saving data to a database may contradict that.
>> Some people have had trouble verifying my observations (other LC Forum users 
>> have confirmed). QC originally could not confirm the bug until they tried it 
>> “on an older PC”. However one of my users recently updated to a *very* fast 
>> HP box; it cut the save time down significantly but was still at least 20 
>> times slower than on a typical Mac, ie almost usable. So maybe it has 
>> something to do with the Windows installation, though the ones I used were 
>> stock standard. I can’t see how fonts could be involved, for example. Neville
> 
> 
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription 
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Richard Gaskin via use-livecode]

Great test set, Neville. Thanks for posting that link.

The hardware on my Linux and Win boxes is so different I don't have a 
strong opinion there. But I did modify your main test script to see what 
I might learn from isolating the file I/O from the stack serialization:


on mouseUp
  put word 2 of the long name of stack "data" into fName
  replace quote with "" in fName
  if the shiftKey is "up" then
put the long seconds into t0
save stack "data"
put the long seconds - t0 into t1
put "saving data took " && t1 && "seconds" into fld "timeInfo"
put 0.01*(the length of URL ("file:" & fName)) && "MB" into fld 
"lengthInfo"

  else
put url ("binfile:"&fName) into tData
put the long seconds into t
put tData into url ("binfile:"& fName)
put the long seconds - t
  end if
end mouseUp

So I ran your tool, making a stack with 300 cards, which produces a file 
a bit over 8MB.  These are my times:


Ubuntu 18.04 w/SSD
Stack: 0.097182
File:  0.008475

Windows 10 w/HDD
Stack: 8.183708
File:  0.014

Even with the differences in the hardware, it looks like the bottleneck 
is in the stack serialization, and the file I/O seems good.


Anyone here in a position to run Neville's test with this modification 
on similarly-configured systems but different OSes?


--
 Richard Gaskin
 Fourth World Systems


Neville Smythe wrote:


Richard

Here is a link to the test stack for testing the speed of save stack

https://www.dropbox.com/sh/cb2r9jbohxqv6bp/AAAQ1weLLlzrKYQ21yn1apf9a?dl=0

I don’t know why the test stacks I supplied disappeared from the QC bug report.

You need the SlowSave.livecode and the data.livecode stacks. They are both 
small, but SlowSave creates a large stack with 300 cards and the lorem ipsum 
text data to build an 8 MB stack, and displays the timing for saving. On my 
MacBook Air it takes 0.13 seconds to save; on Windows 10 installations I have 
seen times from 3 seconds to 10 seconds.

In this test, the stack has just 300 cards, 1 field on each, and lots of ascii 
text (so there should be no problems converting to utf-8 before saving in the 
unlikely event that is happening, and in any case would happen on all 
platforms). I haven’t tested stacks with large numbers of controls or large 
images rather than large text data.

Other apps on Windows 10 certainly don’t display this behaviour, which  
indicates it is a LC problem not a Windows problem. Antivirus software and 
Defender were off during the test. I was under the impression that saving files 
from LC, rather than stacks, was not slower than normal, but Bob’s recently 
experience with saving data to a database may contradict that.

Some people have had trouble verifying my observations (other LC Forum users have confirmed). QC originally could not confirm the bug until they tried it “on an older PC”. However one of my users recently updated to a *very* fast HP box; it cut the save time down significantly but was still at least 20 times slower than on a typical Mac, ie almost usable. So maybe it has something to do with the Windows installation, though the ones I used were stock standard. I can’t see how fonts could be involved, for example. 


Neville



___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Neville Smythe via use-livecode]

Richard

Here is a link to the test stack for testing the speed of save stack

https://www.dropbox.com/sh/cb2r9jbohxqv6bp/AAAQ1weLLlzrKYQ21yn1apf9a?dl=0

I don’t know why the test stacks I supplied disappeared from the QC bug report.

You need the SlowSave.livecode and the data.livecode stacks. They are both 
small, but SlowSave creates a large stack with 300 cards and the lorem ipsum 
text data to build an 8 MB stack, and displays the timing for saving. On my 
MacBook Air it takes 0.13 seconds to save; on Windows 10 installations I have 
seen times from 3 seconds to 10 seconds.

In this test, the stack has just 300 cards, 1 field on each, and lots of ascii 
text (so there should be no problems converting to utf-8 before saving in the 
unlikely event that is happening, and in any case would happen on all 
platforms). I haven’t tested stacks with large numbers of controls or large 
images rather than large text data.

Other apps on Windows 10 certainly don’t display this behaviour, which  
indicates it is a LC problem not a Windows problem. Antivirus software and 
Defender were off during the test. I was under the impression that saving files 
from LC, rather than stacks, was not slower than normal, but Bob’s recently 
experience with saving data to a database may contradict that.

Some people have had trouble verifying my observations (other LC Forum users 
have confirmed). QC originally could not confirm the bug until they tried it 
“on an older PC”. However one of my users recently updated to a *very* fast HP 
box; it cut the save time down significantly but was still at least 20 times 
slower than on a typical Mac, ie almost usable. So maybe it has something to do 
with the Windows installation, though the ones I used were stock standard. I 
can’t see how fonts could be involved, for example. 

Neville
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Bob Sneidar via use-livecode]

Richard, I can verify that my mainStack is pretty big in terms of objects and 
code. When I create a standalone for Windows however, it breaks out all the 
substacks into individual stacks, and the saving just the mainStack in a 
Windows standalone still takes longer than saving the entire Mainstack with 
Substacks on the Mac. 

As a countermeasure, I suppose I can work on getting all the scripts I can 
moved into script only behaviors. I still make extensive use of Datagrids 
however which have a lot of objects. Most of my substacks have multiple grids 
and the main stack has 7. 

I will also say that I have seen a considerable speed difference between Mac 
and Windows when expanding a fairly large zip file contains lots of individual 
files and folders (copier firmware to be exact), so I do not thing the issue is 
with the LC engine, or at least not entirely. It’s so prevalent that I don’t 
even use windows to expand these files anymore. 

Bob S


> > Yes we see this. But as you know in 2018 I reported this slow saving
> > of stacks under Windows10, using LC timing of just the save handler
> > rather than the visual perception of the screen update, and it has
> > been confirmed as a bug.  bug 21305  > /show_bug.cgi?id=21305> The time to save is extreme for large stacks.
> > It took some time for QualityControl to confirm, because apparently
> > it doesn’t seem happen on every Windows 10 installation - just every
> > one  that I or my users have tried. It has not been fixed since 2018.
> 
> Thank you for the link to that report, Nevile.  While I have been a 
> subscriber to it I also follow many reports, so your link is very helpful.
> 
> I was eager to run the test stack there, but discovered there isn't one.  I 
> don't mind creating one, but it would be helpful to know what's required.
> 
> Have you seen any difference between a stack with little content but a large 
> number of objects vs one with a small number of objects but a lot of text or 
> images?
> 
> Once I can reproduce the test, I'll modify it to isolate the time devoted to 
> stack files separately from other file I/O for equivalently-sized files.
> 
> -- 
> Richard Gaskin
> Fourth World Systems
> Software Design and Development for the Desktop, Mobile, and the Web
> 
> ambassa...@fourthworld.comhttp://www.FourthWorld.com`
> 
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription 
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Richard Gaskin via use-livecode]

Neville Smythe wrote:
> Richard wrote
>> Bob Sneidar wrote:
>>
>>> In review, I tested saving stacks on a standalone Windows
>>> Workstation, a VMWARE VM on a very robust server host, a
>>> Parallels VM on a workstation and my Mac. As I am saving
>>> the stack, I am watching the folder the stack is in. I see
>>> the tilde version pop up and go away. On Mac it?s almost
>>> instantaneous. On Windows it can be 3 to 4 seconds.
>>
>> My messages don't seem to be getting through, because each time this
>> observation method of measuring write throughput comes up I post the
>> same reply, yet it keeps coming up.  Please confirm if you can see
>> this:
>
> Yes we see this. But as you know in 2018 I reported this slow saving
> of stacks under Windows10, using LC timing of just the save handler
> rather than the visual perception of the screen update, and it has
> been confirmed as a bug.  bug 21305  /show_bug.cgi?id=21305> The time to save is extreme for large stacks.
> It took some time for QualityControl to confirm, because apparently
> it doesn’t seem happen on every Windows 10 installation - just every
> one  that I or my users have tried. It has not been fixed since 2018.

Thank you for the link to that report, Nevile.  While I have been a 
subscriber to it I also follow many reports, so your link is very helpful.


I was eager to run the test stack there, but discovered there isn't one. 
 I don't mind creating one, but it would be helpful to know what's 
required.


Have you seen any difference between a stack with little content but a 
large number of objects vs one with a small number of objects but a lot 
of text or images?


Once I can reproduce the test, I'll modify it to isolate the time 
devoted to stack files separately from other file I/O for 
equivalently-sized files.


--
 Richard Gaskin
 Fourth World Systems
 Software Design and Development for the Desktop, Mobile, and the Web
 
 ambassa...@fourthworld.comhttp://www.FourthWorld.com`

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Neville Smythe via use-livecode]

Richard wrote 

> Bob Sneidar wrote:
> 
>> In review, I tested saving stacks on a standalone Windows Workstation,
>> a VMWARE VM on a very robust server host, a Parallels VM on a
>> workstation and my Mac. As I am saving the stack, I am watching the
>> folder the stack is in. I see the tilde version pop up and go away. On
>> Mac it?s almost instantaneous. On Windows it can be 3 to 4 seconds.
> 
> My messages don't seem to be getting through, because each time this 
> observation method of measuring write throughput comes up I post the 
> same reply, yet it keeps coming up.  Please confirm if you can see this:

Yes we see this. But as you know in 2018 I reported this slow saving of stacks 
under Windows10, using LC timing of just the save handler rather than the 
visual perception of the screen update, and it has been confirmed as a bug.  
bug 21305  The time to save 
is extreme for large stacks. It took some time for QualityControl to confirm, 
because apparently  it doesn’t seem happen on every Windows 10 installation - 
just every one  that I or my users have tried. It has not been fixed since 
2018. 

Regards

Neville
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Richard Gaskin via use-livecode]

Bob Sneidar wrote:

> On Apr 6, 2020, at 9:28 AM, Richard Gaskin wrote:
>
>> Your application writing data to disk is a very different thing from
>> Windows Desktop Explorer automatically refreshing a directory view.
>
> Control doesn’t return to the application until the tilde version goes
> away.

True, but LC is done with that tilde file and deletes it LONG before the 
separate process of the Windows File Explorer polls and redraws the 
graphical illustration of the deletion that has already happened.



> On Apr 6, 2020, at 9:28 AM, Richard Gaskin wrote:
>
>> on mouseUp
>>  put the long seconds into t
>>  write "somedata" to url ("file:whatever")
>>  if the result is not empty then
>> answer "Couldn't write file (" &sysError()&")"
>>exit to top
>>  end if
>>  put the long seconds - t
>> end mouseUp
>
> You are testing low level file operations. I am testing the saving of
> a Livecode stack. I don’t think this test applies.

Comforting to know that there is no perceived difference in generic file 
writes between Mac and Win, and that we can focus exclusively on the 
small handful of additional steps LC takes when saving a stack file.


This is my understanding of the steps LC takes to save a stack file 
(hopefully Mark Waddingham will chime in if any of this is incorrect or 
incomplete), where "*" denotes steps unique to stack files not taken 
with other files:


* 1. Serialize stack data
* 2. If file exists, rename it with tilde
  3. Open the file
  4. Write the data to disk
  5. Close the file
* 6. Delete the tilde'd file


This test will measure that:

on mouseUp
  put the long seconds into t
  save this stack
  put the long seconds - t
end mouseUp


If the stack contains little data, as with other benchmarking you may 
need to put the task in a loop in between the timing statements to have 
a duration long enough to be meaningfully measurable, e.g.:


on mouseUp
  put the long seconds into t
  repeat 100
 save this stack
  end repeat
  put the long seconds - t
end mouseUp

Run that on similar Mac and Win iron and let us know how it goes.

--
 Richard Gaskin
 Fourth World Systems
 Software Design and Development for the Desktop, Mobile, and the Web
 
 ambassa...@fourthworld.comhttp://www.FourthWorld.com


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Bob Sneidar via use-livecode]

You are testing low level file operations. I am testing the saving of a 
Livecode stack. I don’t think this test applies.

Bob S


On Apr 6, 2020, at 9:28 AM, Richard Gaskin via use-livecode 
mailto:use-livecode@lists.runrev.com>> wrote:

on mouseUp
 put the long seconds into t
 write "somedata" to url ("file:whatever")
 if the result is not empty then
answer "Couldn't write file (" &sysError()&")"
exit to top
 end if
 put the long seconds - t
end mouseUp

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Bob Sneidar via use-livecode]

Control doesn’t return to the application until the tilde version goes away.

On Apr 6, 2020, at 9:28 AM, Richard Gaskin via use-livecode 
mailto:use-livecode@lists.runrev.com>> wrote:

Your application writing data to disk is a very different thing from Windows 
Desktop Explorer automatically refreshing a directory view.

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Richard Gaskin via use-livecode]

Bob Sneidar wrote:

> In review, I tested saving stacks on a standalone Windows Workstation,
> a VMWARE VM on a very robust server host, a Parallels VM on a
> workstation and my Mac. As I am saving the stack, I am watching the
> folder the stack is in. I see the tilde version pop up and go away. On
> Mac it’s almost instantaneous. On Windows it can be 3 to 4 seconds.

My messages don't seem to be getting through, because each time this 
observation method of measuring write throughput comes up I post the 
same reply, yet it keeps coming up.  Please confirm if you can see this:



Your application writing data to disk is a very different thing from 
Windows Desktop Explorer automatically refreshing a directory view.


Any GUI file manager on any OS will use some form of timer/polling for 
the refresh, and the refresh rate for macOS' Finder is much shorter than 
the one in Windows Desktop Explorer.


Observing those GUI file managers only tells us the refresh rate of that 
GUI, not the write speed for the app saving a file.


To measure write speed from LC it would be better to measure within LC 
itself, e.g.:


on mouseUp
  put the long seconds into t
  write "somedata" to url ("file:whatever")
  if the result is not empty then
 answer "Couldn't write file (" &sysError()&")"
 exit to top
  end if
  put the long seconds - t
end mouseUp


When run on similarly-configured iron, is there a significant difference 
between macOS and Windows?


When I isolate write speeds in this manner, I don't see a difference 
larger than could be explained by differences in hardware.


But given the frequent notes about perceived differences, it would be 
great if we could pin down the source of those disparities with some 
sort of test isolating the cause.


--
 Richard Gaskin
 Fourth World Systems
 Software Design and Development for the Desktop, Mobile, and the Web
 
 ambassa...@fourthworld.comhttp://www.FourthWorld.com



___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[doc hawk via use-livecode]

On Apr 5, 2020, at 11:18 AM, Bob Sneidar via use-livecode 
 wrote:
> 
> I don’t think LC accepts multiple statement transactions at all, does it?

I’d be dead int the water if it didn’t. ;)

I’m flat out *dependent* on these—opening a debtor in my software requires 
several hundred entries to be pulled, and at event few milliseconds of latency, 
this would be brutal.  I had an old version running with livecode’s mySQL 
version several years ago, and even after coming back for it to finish loading, 
the overhead from latenciy for repeated transactions made it simply unusable ( 
I think this was before I was stashing in a memory SQLite.  [I started with a 
file-bases SQLITE and an array of internal variables, and with a couple of 
intermediate steps, ended up with :memory: SQLite, and asynchronously  tapping 
Postgres every 30 seconds or so with updates.
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Bob Sneidar via use-livecode]

Hi Richard.

In review, I tested saving stacks on a standalone Windows Workstation, a VMWARE 
VM on a very robust server host, a Parallels VM on a workstation and my Mac. As 
I am saving the stack, I am watching the folder the stack is in. I see the 
tilde version pop up and go away. On Mac it’s almost instantaneous. On Windows 
it can be 3 to 4 seconds.

I do the same thing for SQL queries, querying the exact same data on each 
device I tested on. There are seven individual queries performed for each 
“transaction” in the Main Form.

On my Mac, it takes about 1 to 2 seconds. On any of the Windows systems above, 
it takes 5 to 6 seconds, and one of the systems is the ACTUAL mySQL SERVER 
itself.

I am using sqlYoga to make the queries, but I’ve compared the time it takes to 
query using sqlYoga to mySQL Workbench, and the lag is very minimal (once I 
commented out the statements in libsqlYoga that saved the parent stack every 
time a query was run!)

I disabled Kaspersky during these tests to eliminate that possibility, and 
Defender is disabled domain wide by Group Policy.

My conclusion remains the same. Networking and File Operations on Windows is 
generally slower than on a Mac, or let’s call it *nix.

I can throw together a test app for you if you care to check my numbers.

Bob S


On Apr 5, 2020, at 10:30 AM, Richard Gaskin via use-livecode 
mailto:use-livecode@lists.runrev.com>> wrote:

I've been trying to sort through several reports about write speeds on Windows, 
but most examples I've seen conflate several different tasks. For example, the 
latest I came across had a mix of JSON translation and array manipulation in 
addition to writes, so diligent analysis would require pulling that example 
stack apart into three different tests to pinpoint the actual bottleneck.

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Bob Sneidar via use-livecode]

I don’t think LC accepts multiple statement transactions at all, does it?

Bob S


On Apr 5, 2020, at 11:00 AM, doc hawk via use-livecode 
mailto:use-livecode@lists.runrev.com>> wrote:

If server performance is at all an issue, SQLite may be the wrong choice.

It has a kind of, very reluctant, simultaneous access that the developers warn 
you not to use.

They suggest PostgreSQL for heavy loads.

I use SQLite within the application (the in-memory access is blindingly fast), 
and an external Postgres server (where I basically stash the statements to 
create the sqlite records, and update).

I tried mysql first, but LiveCode  doesn’t support multiple statement 
transactions on it (or at least didn’t at the time), resulting in hundreds of 
transactions where I could have used one

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[doc hawk via use-livecode]

If server performance is at all an issue, SQLite may be the wrong choice.

It has a kind of, very reluctant, simultaneous access that the developers warn 
you not to use.

They suggest PostgreSQL for heavy loads.

I use SQLite within the application (the in-memory access is blindingly fast), 
and an external Postgres server (where I basically stash the statements to 
create the sqlite records, and update).

I tried mysql first, but LiveCode  doesn’t support multiple statement 
transactions on it (or at least didn’t at the time), resulting in hundreds of 
transactions where I could have used one 
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Richard Gaskin via use-livecode]

Bob Sneidar wrote:

> Thanks Terry I did the same thing. The focus is on server performance,
> not client. What I gleaned is that for an individual transaction, the
> difference between a transactional select statement and non is
> indistinguishable to the end user.

The rubric I was given back in the day is that transactions benefit 
writes but not reads. Thanks for confirming.



> Also, filtering out any save operations (including those occurring in
> plugins) increased performance on Windows by an astounding amount.
> While queries still take 3 to 4 times as long on Windows as they do on
> Mac, the lag has become acceptable and comparable to other apps doing
> similar operations.

I've been trying to sort through several reports about write speeds on 
Windows, but most examples I've seen conflate several different tasks. 
For example, the latest I came across had a mix of JSON translation and 
array manipulation in addition to writes, so diligent analysis would 
require pulling that example stack apart into three different tests to 
pinpoint the actual bottleneck.


It may be that there's something about the APIs LC uses on Win which are 
less than optimal, or perhaps something inherent in NTFS.


But I so often see speedy writes on Win in LC that I haven't yet pinned 
down a recipe that exposes a write issue specific to LC on Windows.


Does anyone have a clue to the question that comes up in various forms 
here lately: what factors could make writes from LC slower on Windows 
than on Mac or Linux?


--
 Richard Gaskin
 Fourth World Systems
 Software Design and Development for the Desktop, Mobile, and the Web
 
 ambassa...@fourthworld.comhttp://www.FourthWorld.com


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Bob Sneidar via use-livecode]

Thanks Terry I did the same thing. The focus is on server performance, not 
client. What I gleaned is that for an individual transaction, the difference 
between a transactional select statement and non is indistinguishable to the 
end user. 

Also, filtering out any save operations (including those occurring in plugins) 
increased performance on Windows by an astounding amount. While queries still 
take 3 to 4 times as long on Windows as they do on Mac, the lag has become 
acceptable and comparable to other apps doing similar operations. 

I have always wondered why Windows apps that access databases seem to take so 
long to start up. I think I know now. 

Bob S


> On Apr 5, 2020, at 12:07 AM, Terence Heaford via use-livecode 
>  wrote:
> 
> There is an interesting debate here as to whether it is necessary or not to 
> wrap select statements in transactions for optimisation:
> 
> https://stackoverflow.com/questions/7349189/optimizing-select-with-transaction-under-sqlite-3
>  
> 
> 
> I think there is probably widespread debate about this subject.
> 
> I don’t know the answer?
> 
> Best regards
> 
> Terry
> 
> 
> 
>> On 3 Apr 2020, at 17:00, Bob Sneidar via use-livecode 
>>  wrote:
>> 
>> Mark, you may have answered why it is my Forms Generator app is running so 
>> slow on Windows. I don’t use transactions and I make multiple queries for 
>> most transactions.
>> 
>> Bob S
>> 
> 
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription 
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Terence Heaford via use-livecode]

There is an interesting debate here as to whether it is necessary or not to 
wrap select statements in transactions for optimisation:

https://stackoverflow.com/questions/7349189/optimizing-select-with-transaction-under-sqlite-3
 


I think there is probably widespread debate about this subject.

I don’t know the answer?

Best regards

Terry



> On 3 Apr 2020, at 17:00, Bob Sneidar via use-livecode 
>  wrote:
> 
> Mark, you may have answered why it is my Forms Generator app is running so 
> slow on Windows. I don’t use transactions and I make multiple queries for 
> most transactions.
> 
> Bob S
> 

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Bob Sneidar via use-livecode]

Mark, you may have answered why it is my Forms Generator app is running so slow 
on Windows. I don’t use transactions and I make multiple queries for most 
transactions.

Bob S


On Apr 3, 2020, at 1:42 AM, Mark Waddingham via use-livecode 
mailto:use-livecode@lists.runrev.com>> wrote:

On 2020-04-03 09:25, Terence Heaford via use-livecode wrote:
Fortunately I have Time Machine running on my mac.
I have gone back to a previous copy from the 01 April 2020 and this
project is running at full speed.
I have made no modifications to the underlying functionality of this
project other than some modifications to SQLite
scripts to improve performance.
I will make those adjustments again and report on the outcome.
I do not believe it’s anything I have done.

I'd humbly  suggest that modifications to SQLite scripts constitute 
modifications ;)

Have you modified the SQLite database in any way?

Like most db's SQLite is heavily dependent on indices to prevent operations 
being slow - if some have been removed which existing queries relied upon, or 
one hasn't been added to support a new/modified query then that could cause a 
heavy performance loss.

Similarly, SQLite is transactional, so if you do lots of mutation queries 
outside of one each individual query will take a lot longer (as each will be 
under its own transaction requiring a OS disk flush/sync after each) - if 
you've tweaked how you are doing such queries and not taken into account 
transactions that could also cause a heavy performance loss.

Warmest Regards,

Mark.

--
Mark Waddingham ~ m...@livecode.com ~ 
http://www.livecode.com/
LiveCode: Everyone can create apps

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Bob Sneidar via use-livecode]

What may help in the future to eliminate possible script issues, is to use a 
text editor that has the capability of comparing two scripts and showing you 
the difference. Most of the code editors have addams that will do this.

Bob S


On Apr 3, 2020, at 1:25 AM, Terence Heaford via use-livecode 
mailto:use-livecode@lists.runrev.com>> wrote:

Fortunately I have Time Machine running on my mac.

I have gone back to a previous copy from the 01 April 2020 and this project is 
running at full speed.

I have made no modifications to the underlying functionality of this project 
other than some modifications to SQLite
scripts to improve performance.

I will make those adjustments again and report on the outcome.

I do not believe it’s anything I have done.

I believe (no proof) that something within LC has corrupted the project.

I have looked at copying the old project into a new project but that within LC 
seems to be a nightmare scenario.

If that was easy and it worked it would be a future solution or if there was a 
livecode maintenance type project that could do this sort of thing that would 
be good.

All the best for now.

Will report back

Thanks

Terry

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

RE: Go to card has become slow

[Ralph DiMola via use-livecode]

On Windows I have seen a slow down on some but not all cards when the SE is 
open. Close the SE and everything speeds back up. 

Ralph DiMola
IT Director
Evergreen Information Services
rdim...@evergreeninfo.net


-Original Message-
From: use-livecode [mailto:use-livecode-boun...@lists.runrev.com] On Behalf Of 
Terence Heaford via use-livecode
Sent: Friday, April 03, 2020 8:55 AM
To: How to use LiveCode
Cc: Terence Heaford
Subject: Re: Go to card has become slow

I have now copied the modified scripts from the project version that was slow 
and reapplied them to the previous version which was running fast.

Applying the modified scripts has not resulted in any slowdown of the version 
they have been applied to.

So, I cannot understand why the project slowed down in the first place but…

all is now well until it happens again.


Thanks

Terry

> On 3 Apr 2020, at 09:42, Mark Waddingham via use-livecode 
>  wrote:
> 
> On 2020-04-03 09:25, Terence Heaford via use-livecode wrote:
>> Fortunately I have Time Machine running on my mac.
>> I have gone back to a previous copy from the 01 April 2020 and this 
>> project is running at full speed.
>> I have made no modifications to the underlying functionality of this 
>> project other than some modifications to SQLite scripts to improve 
>> performance.
>> I will make those adjustments again and report on the outcome.
>> I do not believe it’s anything I have done.
> 
> I'd humbly  suggest that modifications to SQLite scripts constitute 
> modifications ;)
> 
> Have you modified the SQLite database in any way?
> 
> Like most db's SQLite is heavily dependent on indices to prevent operations 
> being slow - if some have been removed which existing queries relied upon, or 
> one hasn't been added to support a new/modified query then that could cause a 
> heavy performance loss.
> 
> Similarly, SQLite is transactional, so if you do lots of mutation queries 
> outside of one each individual query will take a lot longer (as each will be 
> under its own transaction requiring a OS disk flush/sync after each) - if 
> you've tweaked how you are doing such queries and not taken into account 
> transactions that could also cause a heavy performance loss.
> 
> Warmest Regards,
> 
> Mark.
> 
> --
> Mark Waddingham ~ m...@livecode.com ~ http://www.livecode.com/
> LiveCode: Everyone can create apps
> 
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription 
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Terence Heaford via use-livecode]

I have now copied the modified scripts from the project version that was slow 
and reapplied them to the previous version which was running fast.

Applying the modified scripts has not resulted in any slowdown of the version 
they have been applied to.

So, I cannot understand why the project slowed down in the first place but…

all is now well until it happens again.


Thanks

Terry

> On 3 Apr 2020, at 09:42, Mark Waddingham via use-livecode 
>  wrote:
> 
> On 2020-04-03 09:25, Terence Heaford via use-livecode wrote:
>> Fortunately I have Time Machine running on my mac.
>> I have gone back to a previous copy from the 01 April 2020 and this
>> project is running at full speed.
>> I have made no modifications to the underlying functionality of this
>> project other than some modifications to SQLite
>> scripts to improve performance.
>> I will make those adjustments again and report on the outcome.
>> I do not believe it’s anything I have done.
> 
> I'd humbly  suggest that modifications to SQLite scripts constitute 
> modifications ;)
> 
> Have you modified the SQLite database in any way?
> 
> Like most db's SQLite is heavily dependent on indices to prevent operations 
> being slow - if some have been removed which existing queries relied upon, or 
> one hasn't been added to support a new/modified query then that could cause a 
> heavy performance loss.
> 
> Similarly, SQLite is transactional, so if you do lots of mutation queries 
> outside of one each individual query will take a lot longer (as each will be 
> under its own transaction requiring a OS disk flush/sync after each) - if 
> you've tweaked how you are doing such queries and not taken into account 
> transactions that could also cause a heavy performance loss.
> 
> Warmest Regards,
> 
> Mark.
> 
> -- 
> Mark Waddingham ~ m...@livecode.com ~ http://www.livecode.com/
> LiveCode: Everyone can create apps
> 
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription 
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Mark Waddingham via use-livecode]

On 2020-04-03 09:25, Terence Heaford via use-livecode wrote:

Fortunately I have Time Machine running on my mac.

I have gone back to a previous copy from the 01 April 2020 and this
project is running at full speed.

I have made no modifications to the underlying functionality of this
project other than some modifications to SQLite
scripts to improve performance.

I will make those adjustments again and report on the outcome.

I do not believe it’s anything I have done.


I'd humbly  suggest that modifications to SQLite scripts constitute 
modifications ;)


Have you modified the SQLite database in any way?

Like most db's SQLite is heavily dependent on indices to prevent 
operations being slow - if some have been removed which existing queries 
relied upon, or one hasn't been added to support a new/modified query 
then that could cause a heavy performance loss.


Similarly, SQLite is transactional, so if you do lots of mutation 
queries outside of one each individual query will take a lot longer (as 
each will be under its own transaction requiring a OS disk flush/sync 
after each) - if you've tweaked how you are doing such queries and not 
taken into account transactions that could also cause a heavy 
performance loss.


Warmest Regards,

Mark.

--
Mark Waddingham ~ m...@livecode.com ~ http://www.livecode.com/
LiveCode: Everyone can create apps

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Terence Heaford via use-livecode]

Fortunately I have Time Machine running on my mac.

I have gone back to a previous copy from the 01 April 2020 and this project is 
running at full speed.

I have made no modifications to the underlying functionality of this project 
other than some modifications to SQLite
scripts to improve performance.

I will make those adjustments again and report on the outcome.

I do not believe it’s anything I have done.

I believe (no proof) that something within LC has corrupted the project.

I have looked at copying the old project into a new project but that within LC 
seems to be a nightmare scenario.

If that was easy and it worked it would be a future solution or if there was a 
livecode maintenance type project that could do this sort of thing that would 
be good.

All the best for now.

Will report back

Thanks

Terry


> On 2 Apr 2020, at 22:45, Devin Asay via use-livecode 
>  wrote:
> 
> I have seen delays when going to a card containing a large image with Resize 
> Quality set to “Best”. It was more of an issue on a mobile device, but it’s 
> worth looking into.
> 
> Devin
> 
>> On Apr 2, 2020, at 1:27 PM, Terence Heaford via use-livecode 
>>  wrote:
>> 
>> I have a project that opens at a particular card.
>> 
>> When I go to another card (any card) it’s very snappy.
>> 
>> When I return to the original card (from any card) I have noticed a second 
>> or two delay before it happens.
>> 
>> There would appear to be nothing in any script that would account for this.
>> 
>> Can anyone give some insight as to why this has happened?
>> 
>> I reinstalled LC to no avail.
>> 
>> Community 9.5.1
>> 
>> MacOS 10.15.4
>> 
>> I read something about font changes on the forum but that seemed to relate 
>> to iOS not MacOS.
>> 
>> I’m at a loss
>> 
>> 
>> Thanks
>> 
>> Terry
>> 
>> 
>> 
>> 
>> 
>> ___
>> use-livecode mailing list
>> use-livecode@lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your subscription 
>> preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
> 
> Devin Asay
> Director
> Office of Digital Humanities
> Brigham Young University
> 
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription 
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Devin Asay via use-livecode]

I have seen delays when going to a card containing a large image with Resize 
Quality set to “Best”. It was more of an issue on a mobile device, but it’s 
worth looking into.

Devin

> On Apr 2, 2020, at 1:27 PM, Terence Heaford via use-livecode 
>  wrote:
> 
> I have a project that opens at a particular card.
> 
> When I go to another card (any card) it’s very snappy.
> 
> When I return to the original card (from any card) I have noticed a second or 
> two delay before it happens.
> 
> There would appear to be nothing in any script that would account for this.
> 
> Can anyone give some insight as to why this has happened?
> 
> I reinstalled LC to no avail.
> 
> Community 9.5.1
> 
> MacOS 10.15.4
> 
> I read something about font changes on the forum but that seemed to relate to 
> iOS not MacOS.
> 
> I’m at a loss
> 
> 
> Thanks
> 
> Terry
> 
> 
> 
> 
> 
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription 
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode

Devin Asay
Director
Office of Digital Humanities
Brigham Young University

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Peter Bogdanoff via use-livecode]

Also, any audio/video files that load?


> On Apr 2, 2020, at 1:46 PM, Pi Digital via use-livecode 
>  wrote:
> 
> Hi Terry
> 
> Are there any big images on the first card? Any layer effects or blend modes? 
> Anything graphic based that could be attributed to the slow down. 
> 
> Sean Cole
> Pi
> 
>> On 2 Apr 2020, at 20:27, Terence Heaford via use-livecode 
>>  wrote:
>> 
>> I have a project that opens at a particular card.
>> 
>> When I go to another card (any card) it’s very snappy.
>> 
>> When I return to the original card (from any card) I have noticed a second 
>> or two delay before it happens.
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription 
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Go to card has become slow

[Pi Digital via use-livecode]

Hi Terry

Are there any big images on the first card? Any layer effects or blend modes? 
Anything graphic based that could be attributed to the slow down. 

Sean Cole
Pi

> On 2 Apr 2020, at 20:27, Terence Heaford via use-livecode 
>  wrote:
> 
> I have a project that opens at a particular card.
> 
> When I go to another card (any card) it’s very snappy.
> 
> When I return to the original card (from any card) I have noticed a second or 
> two delay before it happens.
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Go to card has become slow

[Terence Heaford via use-livecode]

I have a project that opens at a particular card.

When I go to another card (any card) it’s very snappy.

When I return to the original card (from any card) I have noticed a second or 
two delay before it happens.

There would appear to be nothing in any script that would account for this.

Can anyone give some insight as to why this has happened?

I reinstalled LC to no avail.

Community 9.5.1

MacOS 10.15.4

I read something about font changes on the forum but that seemed to relate to 
iOS not MacOS.

I’m at a loss


Thanks

Terry





___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode