Re: [tw5] Re: File-based TW on node.js crashes when the number of tiddlers is more than ~2000

2019-06-03 Thread PMario 
Hi, 

With your attachment, I can reproduce the problem. ... I'm pretty sure it's 
because there is no text field. 

So if you could create your tiddlers with at least 3 line breaks it may 
work. 

-m

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/d0496360-55bc-40a5-90bd-7d83b90917de%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [tw5] Re: File-based TW on node.js crashes when the number of tiddlers is more than ~2000

2019-06-03 Thread 'Mark S.' via TiddlyWiki 
It's always useful to see things exactly the way the user sees them.

I got the exact same error you did, when loading it as

tiddlywiki tw-benchmark --listen port=8090

The error occurs after 1862 "loads".  However, when I check the internal 
contents, it appears that all tiddler titles have actually loaded (I can 
check these numbers because the advanced search works even when the server 
has crashed).

But I was able to load the wiki via Bob just fine. So ... maybe Bob doesn't 
invoke all the "loads"? Or uses a different loading mechanism?

I'm using Windows 7, so my system is very different from yours, but we're 
getting the same error.

So anyone testing should use your data and the standard mechanism for using 
it on node.js.

Good luck

On Monday, June 3, 2019 at 11:09:55 AM UTC-7, solex wrote:
>
> Hi Mark,
>
> Sure, attached. 
> But like I said, it really is just a fresh wiki with a bunch of generated 
> simple tiddlers
>
>
>
> On Mon, Jun 3, 2019 at 6:46 PM 'Mark S.' via TiddlyWiki <
> tiddl...@googlegroups.com > wrote:
>
>> I'm wondering about the data. Could you zip up your tiddlers and post it?
>>
>> About the email. Google bounces mail from certain domains (or at least 
>> that seems to be the problem).  
>>
>> -- Mark
>>
>> On Monday, June 3, 2019 at 2:03:49 AM UTC-7, solex wrote:
>>>
>>> Hi all,
>>>
>>> I am rather new to TiddlyWiki, and I've been doing some experiments 
>>> recently with the automatic importing of data to a file-based wiki.
>>> But I found that when serving a wiki with a few thousand tiddlers, the 
>>> node server crashes with the following error, while in the middle of 
>>> dispatching "load" tasks:
>>>
>>> RangeError: Maximum call stack size exceeded
 at Console.(anonymous function) 
 (internal/console/constructor.js:243:9)
 at Console.log (internal/console/constructor.js:283:24)
 at Logger.log ($:/core/modules/utils/logger.js:32:25)
 at Syncer.dispatchTask ($:/core/modules/syncer.js:515:15)
 at Syncer.processTaskQueue ($:/core/modules/syncer.js:437:9)
 at $:/core/modules/syncer.js:445:27
 at $:/core/modules/syncer.js:525:4
 at FileSystemAdaptor.loadTiddler 
 ($:/plugins/tiddlywiki/filesystem/filesystemadaptor.js:193:2)
 at Syncer.dispatchTask ($:/core/modules/syncer.js:516:20)
 at Syncer.processTaskQueue ($:/core/modules/syncer.js:437:9)
>>>
>>>
>>> My system:
>>> tiddlywiki: 5.1.19
>>> node: 11.9.0
>>> MacOs: 10.13.6
>>>
>>> I experimented with the total number, and found that the issue occurs 
>>> somewhere between 1500 and 2000 tiddlers. It looks like the content doesn't 
>>> matter: the problem occurs even with the simplest possible tiddlers (title 
>>> only).
>>>
>>> I also disabled the TW's logger just to rule out a problem in the logger 
>>> itself, and the problem persists.
>>>
>>> To reproduce the error:
>>>
>>>1. Create a fresh "server edition" wiki
>>>2. Populate with 2000 tiddlers (I used a basic script for that)
>>>3. Start a server with a "listen" command
>>>4. Open the TW's page in the browser
>>>
>>>
>>> 2000 tiddlers doesn't seem like a very large number for an in-browser 
>>> wiki, nor it should be a problem for a server per se. I believe the 
>>> bottleneck is somewhere in the synchronization process, but so far I wasn't 
>>> able to pinpoint it exactly.
>>>
>>> Has anyone experienced the same problem and is there is anything that 
>>> can be done to improve the sync performance?
>>>
>>> (Also, please let me know if this sort of question should be posted to 
>>> TiddlyWikiDev)
>>>
>>> Thanks!
>>>
>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "TiddlyWiki" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to tiddl...@googlegroups.com .
>> To post to this group, send email to tiddl...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/tiddlywiki.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/tiddlywiki/c320a3b8-3d90-4bfd-8135-9d7518f0111b%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/0c7700b8-ddfe-49b9-8e12-9a266aa8679b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[tw5] Re: File-based TW on node.js crashes when the number of tiddlers is more than ~2000

2019-06-03 Thread 'Mark S.' via TiddlyWiki 
I'm wondering about the data. Could you zip up your tiddlers and post it?

About the email. Google bounces mail from certain domains (or at least that 
seems to be the problem).  

-- Mark

On Monday, June 3, 2019 at 2:03:49 AM UTC-7, solex wrote:
>
> Hi all,
>
> I am rather new to TiddlyWiki, and I've been doing some experiments 
> recently with the automatic importing of data to a file-based wiki.
> But I found that when serving a wiki with a few thousand tiddlers, the 
> node server crashes with the following error, while in the middle of 
> dispatching "load" tasks:
>
> RangeError: Maximum call stack size exceeded
>> at Console.(anonymous function) 
>> (internal/console/constructor.js:243:9)
>> at Console.log (internal/console/constructor.js:283:24)
>> at Logger.log ($:/core/modules/utils/logger.js:32:25)
>> at Syncer.dispatchTask ($:/core/modules/syncer.js:515:15)
>> at Syncer.processTaskQueue ($:/core/modules/syncer.js:437:9)
>> at $:/core/modules/syncer.js:445:27
>> at $:/core/modules/syncer.js:525:4
>> at FileSystemAdaptor.loadTiddler 
>> ($:/plugins/tiddlywiki/filesystem/filesystemadaptor.js:193:2)
>> at Syncer.dispatchTask ($:/core/modules/syncer.js:516:20)
>> at Syncer.processTaskQueue ($:/core/modules/syncer.js:437:9)
>
>
> My system:
> tiddlywiki: 5.1.19
> node: 11.9.0
> MacOs: 10.13.6
>
> I experimented with the total number, and found that the issue occurs 
> somewhere between 1500 and 2000 tiddlers. It looks like the content doesn't 
> matter: the problem occurs even with the simplest possible tiddlers (title 
> only).
>
> I also disabled the TW's logger just to rule out a problem in the logger 
> itself, and the problem persists.
>
> To reproduce the error:
>
>1. Create a fresh "server edition" wiki
>2. Populate with 2000 tiddlers (I used a basic script for that)
>3. Start a server with a "listen" command
>4. Open the TW's page in the browser
>
>
> 2000 tiddlers doesn't seem like a very large number for an in-browser 
> wiki, nor it should be a problem for a server per se. I believe the 
> bottleneck is somewhere in the synchronization process, but so far I wasn't 
> able to pinpoint it exactly.
>
> Has anyone experienced the same problem and is there is anything that can 
> be done to improve the sync performance?
>
> (Also, please let me know if this sort of question should be posted to 
> TiddlyWikiDev)
>
> Thanks!
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/c320a3b8-3d90-4bfd-8135-9d7518f0111b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[tw5] Re: File-based TW on node.js crashes when the number of tiddlers is more than ~2000

2019-06-03 Thread solex 
It looks like my original post doesn't appear in the email exchange (while 
visible in Google groups UI).
That probably explains some some questions and suggestions (which I already 
addressed in the first post)

On Monday, June 3, 2019 at 12:03:49 PM UTC+3, solex wrote:
>
> Hi all,
>
> I am rather new to TiddlyWiki, and I've been doing some experiments 
> recently with the automatic importing of data to a file-based wiki.
> But I found that when serving a wiki with a few thousand tiddlers, the 
> node server crashes with the following error, while in the middle of 
> dispatching "load" tasks:
>
> RangeError: Maximum call stack size exceeded
>> at Console.(anonymous function) 
>> (internal/console/constructor.js:243:9)
>> at Console.log (internal/console/constructor.js:283:24)
>> at Logger.log ($:/core/modules/utils/logger.js:32:25)
>> at Syncer.dispatchTask ($:/core/modules/syncer.js:515:15)
>> at Syncer.processTaskQueue ($:/core/modules/syncer.js:437:9)
>> at $:/core/modules/syncer.js:445:27
>> at $:/core/modules/syncer.js:525:4
>> at FileSystemAdaptor.loadTiddler 
>> ($:/plugins/tiddlywiki/filesystem/filesystemadaptor.js:193:2)
>> at Syncer.dispatchTask ($:/core/modules/syncer.js:516:20)
>> at Syncer.processTaskQueue ($:/core/modules/syncer.js:437:9)
>
>
> My system:
> tiddlywiki: 5.1.19
> node: 11.9.0
> MacOs: 10.13.6
>
> I experimented with the total number, and found that the issue occurs 
> somewhere between 1500 and 2000 tiddlers. It looks like the content doesn't 
> matter: the problem occurs even with the simplest possible tiddlers (title 
> only).
>
> I also disabled the TW's logger just to rule out a problem in the logger 
> itself, and the problem persists.
>
> To reproduce the error:
>
>1. Create a fresh "server edition" wiki
>2. Populate with 2000 tiddlers (I used a basic script for that)
>3. Start a server with a "listen" command
>4. Open the TW's page in the browser
>
>
> 2000 tiddlers doesn't seem like a very large number for an in-browser 
> wiki, nor it should be a problem for a server per se. I believe the 
> bottleneck is somewhere in the synchronization process, but so far I wasn't 
> able to pinpoint it exactly.
>
> Has anyone experienced the same problem and is there is anything that can 
> be done to improve the sync performance?
>
> (Also, please let me know if this sort of question should be posted to 
> TiddlyWikiDev)
>
> Thanks!
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/a790148a-f303-42ba-a48f-dc76f1954952%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [tw5] Re: File-based TW on node.js crashes when the number of tiddlers is more than ~2000

2019-06-03 Thread solex 
Hi BJ,

No, turning off the logs was the first thing I tried. The error was the 
same (just without the logger-related traceback)


On Monday, June 3, 2019 at 6:09:12 PM UTC+3, BJ wrote:
>
> maybe there is a problem with the logging...
>
> you can turn off the logs from the syncer in startup.js
>
> $tw.syncer = new $tw.Syncer({wiki: $tw.wiki, syncadaptor: $tw.syncadaptor, 
> logging : false});
>
>
> On Monday, June 3, 2019 at 3:13:04 PM UTC+2, solex wrote:
>>
>> Hi Jeremy,
>>
>> It didn't occur to me to tweak the Node's memory usage, because I didn't 
>> think it was the cause of the problem, and I still doubt that processing 
>> 2000 one-line tiddlers consumes all of the default 1GB memory - something 
>> else must be the problem. But I'll try to experiment with the parameters to 
>> see if that at least increases the number.
>> I did try to close most of the other apps/processes on my machine just in 
>> case, to no avail.
>>
>> My test wiki doesn't use any recursive templates - or anything besides 
>> the built-in functionality - because I created a fresh server wiki just to 
>> investigate the problem. All I am doing is generating 2000 simple tiddlers 
>> with an external script (I described a sequence of steps in my original 
>> post).
>>
>> Yes, the same error occurs when I am trying to build a static version 
>> with:
>> tiddlywiki tw-benchmark --build index
>>
>> -
>> Thanks,
>> solex
>>
>>
>> On Mon, Jun 3, 2019 at 3:39 PM Jeremy Ruston  wrote:
>>
>>> Hi Solex
>>>
>>> I run many wikis with over 2,000 tiddlers, so I think there maybe 
>>> something else going on here.
>>>
>>> Firstly, Node.js is by default conservative about memory usage, using 
>>> only a maximum of 512MB of 32-bit systems and 1GB on 64-bit systems. You 
>>> can manually specify the total amount of memory in MB to be used with the 
>>> "max-old-space-size" flag. For example:
>>>
>>> node --max-old-space-size=16384  >> arguments>
>>>
>>> Secondly, the easiest way to cause out-of-memory errors is to try to 
>>> render a template that includes recursive elements. Are you using a custom 
>>> raw markup tiddlers? Do you get the same error if you try to build the wiki 
>>> as a single file? 
>>>
>>> Best wishes
>>>
>>> Jeremy
>>>
>>>
>>>
>>> On Monday, June 3, 2019 at 1:04:51 PM UTC+1, solex wrote:

 Hi mario,

 My tiddlers only have title as an attribute, and the corresponding file 
 name.
 e.g.

 tiddlers/test-tiddler-1.tid

 title: test-tiddler-1

 Initially they had other fields as well (including content-type), but 
 I reduced them to a minimum while investigating, and the particular set of 
 fields didn't seem to make any difference. 
 Also, when the total number is lower (e.g. <1500), the wiki is 
 functioning as expected, so I'm sure the problem is not in the tiddler 
 generation.
 The browser doesn't matter too (as expected).

 Apparently the only difference is environment (Node version + OS), so 
 perhaps I will try later on a different machine.

 Thanks,
 solex



 On Monday, June 3, 2019 at 1:03:41 PM UTC+3, PMario wrote:
>
> Hi solex, 
>
> I did create 10'000 tiddlers. They all look similar to this one and 
> are about 800 bytes in size. 
>
> created: 20190603094535523
> modified: 20190603094535523
> tags: t-0
> title: a-0-0
> type: text/vnd.tiddlywiki
>
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 
>
>
> There is no problem for me. 
>  - I use node v10.16.0 LTS. 
>  - FF latest 
>  - TW 5.1.19
>  - win 10
>
> The tw server side needs about 20 seconds to start. 
> It transferes about 12MByte to the browser, which needs about 4 
> seconds. 
>
> So how do you create your files?
>
> -mario
>
>
> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "TiddlyWiki" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to tiddl...@googlegroups.com.
>>> To post to this group, send email to tiddl...@googlegroup

Re: [tw5] Re: File-based TW on node.js crashes when the number of tiddlers is more than ~2000

2019-06-03 Thread BJ 
maybe there is a problem with the logging...

you can turn off the logs from the syncer in startup.js

$tw.syncer = new $tw.Syncer({wiki: $tw.wiki, syncadaptor: $tw.syncadaptor, 
logging : false});


On Monday, June 3, 2019 at 3:13:04 PM UTC+2, solex wrote:
>
> Hi Jeremy,
>
> It didn't occur to me to tweak the Node's memory usage, because I didn't 
> think it was the cause of the problem, and I still doubt that processing 
> 2000 one-line tiddlers consumes all of the default 1GB memory - something 
> else must be the problem. But I'll try to experiment with the parameters to 
> see if that at least increases the number.
> I did try to close most of the other apps/processes on my machine just in 
> case, to no avail.
>
> My test wiki doesn't use any recursive templates - or anything besides the 
> built-in functionality - because I created a fresh server wiki just to 
> investigate the problem. All I am doing is generating 2000 simple tiddlers 
> with an external script (I described a sequence of steps in my original 
> post).
>
> Yes, the same error occurs when I am trying to build a static version with:
> tiddlywiki tw-benchmark --build index
>
> -
> Thanks,
> solex
>
>
> On Mon, Jun 3, 2019 at 3:39 PM Jeremy Ruston  > wrote:
>
>> Hi Solex
>>
>> I run many wikis with over 2,000 tiddlers, so I think there maybe 
>> something else going on here.
>>
>> Firstly, Node.js is by default conservative about memory usage, using 
>> only a maximum of 512MB of 32-bit systems and 1GB on 64-bit systems. You 
>> can manually specify the total amount of memory in MB to be used with the 
>> "max-old-space-size" flag. For example:
>>
>> node --max-old-space-size=16384  > arguments>
>>
>> Secondly, the easiest way to cause out-of-memory errors is to try to 
>> render a template that includes recursive elements. Are you using a custom 
>> raw markup tiddlers? Do you get the same error if you try to build the wiki 
>> as a single file? 
>>
>> Best wishes
>>
>> Jeremy
>>
>>
>>
>> On Monday, June 3, 2019 at 1:04:51 PM UTC+1, solex wrote:
>>>
>>> Hi mario,
>>>
>>> My tiddlers only have title as an attribute, and the corresponding file 
>>> name.
>>> e.g.
>>>
>>> tiddlers/test-tiddler-1.tid
>>>
>>> title: test-tiddler-1
>>>
>>> Initially they had other fields as well (including content-type), but I 
>>> reduced them to a minimum while investigating, and the particular set of 
>>> fields didn't seem to make any difference. 
>>> Also, when the total number is lower (e.g. <1500), the wiki is 
>>> functioning as expected, so I'm sure the problem is not in the tiddler 
>>> generation.
>>> The browser doesn't matter too (as expected).
>>>
>>> Apparently the only difference is environment (Node version + OS), so 
>>> perhaps I will try later on a different machine.
>>>
>>> Thanks,
>>> solex
>>>
>>>
>>>
>>> On Monday, June 3, 2019 at 1:03:41 PM UTC+3, PMario wrote:

 Hi solex, 

 I did create 10'000 tiddlers. They all look similar to this one and are 
 about 800 bytes in size. 

 created: 20190603094535523
 modified: 20190603094535523
 tags: t-0
 title: a-0-0
 type: text/vnd.tiddlywiki

 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
 123456789 123456789 123456789 


 There is no problem for me. 
  - I use node v10.16.0 LTS. 
  - FF latest 
  - TW 5.1.19
  - win 10

 The tw server side needs about 20 seconds to start. 
 It transferes about 12MByte to the browser, which needs about 4 
 seconds. 

 So how do you create your files?

 -mario


 -- 
>> You received this message because you are subscribed to the Google Groups 
>> "TiddlyWiki" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to tiddl...@googlegroups.com .
>> To post to this group, send email to tiddl...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/tiddlywiki.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/tiddlywiki/cdbfa07c-80bd-4ed9-b378-242ab9180f02%40googlegroups.com
>>  
>>

Re: [tw5] Re: File-based TW on node.js crashes when the number of tiddlers is more than ~2000

2019-06-03 Thread Oleksiy Solyanyk 
Hi Jeremy,

It didn't occur to me to tweak the Node's memory usage, because I didn't
think it was the cause of the problem, and I still doubt that processing
2000 one-line tiddlers consumes all of the default 1GB memory - something
else must be the problem. But I'll try to experiment with the parameters to
see if that at least increases the number.
I did try to close most of the other apps/processes on my machine just in
case, to no avail.

My test wiki doesn't use any recursive templates - or anything besides the
built-in functionality - because I created a fresh server wiki just to
investigate the problem. All I am doing is generating 2000 simple tiddlers
with an external script (I described a sequence of steps in my original
post).

Yes, the same error occurs when I am trying to build a static version with:
tiddlywiki tw-benchmark --build index

-
Thanks,
solex


On Mon, Jun 3, 2019 at 3:39 PM Jeremy Ruston 
wrote:

> Hi Solex
>
> I run many wikis with over 2,000 tiddlers, so I think there maybe
> something else going on here.
>
> Firstly, Node.js is by default conservative about memory usage, using only
> a maximum of 512MB of 32-bit systems and 1GB on 64-bit systems. You can
> manually specify the total amount of memory in MB to be used with the
> "max-old-space-size" flag. For example:
>
> node --max-old-space-size=16384   arguments>
>
> Secondly, the easiest way to cause out-of-memory errors is to try to
> render a template that includes recursive elements. Are you using a custom
> raw markup tiddlers? Do you get the same error if you try to build the wiki
> as a single file?
>
> Best wishes
>
> Jeremy
>
>
>
> On Monday, June 3, 2019 at 1:04:51 PM UTC+1, solex wrote:
>>
>> Hi mario,
>>
>> My tiddlers only have title as an attribute, and the corresponding file
>> name.
>> e.g.
>>
>> tiddlers/test-tiddler-1.tid
>>
>> title: test-tiddler-1
>>
>> Initially they had other fields as well (including content-type), but I
>> reduced them to a minimum while investigating, and the particular set of
>> fields didn't seem to make any difference.
>> Also, when the total number is lower (e.g. <1500), the wiki is
>> functioning as expected, so I'm sure the problem is not in the tiddler
>> generation.
>> The browser doesn't matter too (as expected).
>>
>> Apparently the only difference is environment (Node version + OS), so
>> perhaps I will try later on a different machine.
>>
>> Thanks,
>> solex
>>
>>
>>
>> On Monday, June 3, 2019 at 1:03:41 PM UTC+3, PMario wrote:
>>>
>>> Hi solex,
>>>
>>> I did create 10'000 tiddlers. They all look similar to this one and are
>>> about 800 bytes in size.
>>>
>>> created: 20190603094535523
>>> modified: 20190603094535523
>>> tags: t-0
>>> title: a-0-0
>>> type: text/vnd.tiddlywiki
>>>
>>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789
>>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789
>>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789
>>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789
>>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789
>>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789
>>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789
>>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789
>>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789
>>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789
>>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789
>>> 123456789 123456789 123456789
>>>
>>>
>>> There is no problem for me.
>>>  - I use node v10.16.0 LTS.
>>>  - FF latest
>>>  - TW 5.1.19
>>>  - win 10
>>>
>>> The tw server side needs about 20 seconds to start.
>>> It transferes about 12MByte to the browser, which needs about 4 seconds.
>>>
>>> So how do you create your files?
>>>
>>> -mario
>>>
>>>
>>> --
> You received this message because you are subscribed to the Google Groups
> "TiddlyWiki" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to tiddlywiki+unsubscr...@googlegroups.com.
> To post to this group, send email to tiddlywiki@googlegroups.com.
> Visit this group at https://groups.google.com/group/tiddlywiki.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/tiddlywiki/cdbfa07c-80bd-4ed9-b378-242ab9180f02%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddl

[tw5] Re: File-based TW on node.js crashes when the number of tiddlers is more than ~2000

2019-06-03 Thread Jeremy Ruston 
Hi Solex

I run many wikis with over 2,000 tiddlers, so I think there maybe something 
else going on here.

Firstly, Node.js is by default conservative about memory usage, using only 
a maximum of 512MB of 32-bit systems and 1GB on 64-bit systems. You can 
manually specify the total amount of memory in MB to be used with the 
"max-old-space-size" flag. For example:

node --max-old-space-size=16384  

Secondly, the easiest way to cause out-of-memory errors is to try to render 
a template that includes recursive elements. Are you using a custom raw 
markup tiddlers? Do you get the same error if you try to build the wiki as 
a single file? 

Best wishes

Jeremy



On Monday, June 3, 2019 at 1:04:51 PM UTC+1, solex wrote:
>
> Hi mario,
>
> My tiddlers only have title as an attribute, and the corresponding file 
> name.
> e.g.
>
> tiddlers/test-tiddler-1.tid
>
> title: test-tiddler-1
>
> Initially they had other fields as well (including content-type), but I 
> reduced them to a minimum while investigating, and the particular set of 
> fields didn't seem to make any difference. 
> Also, when the total number is lower (e.g. <1500), the wiki is functioning 
> as expected, so I'm sure the problem is not in the tiddler generation.
> The browser doesn't matter too (as expected).
>
> Apparently the only difference is environment (Node version + OS), so 
> perhaps I will try later on a different machine.
>
> Thanks,
> solex
>
>
>
> On Monday, June 3, 2019 at 1:03:41 PM UTC+3, PMario wrote:
>>
>> Hi solex, 
>>
>> I did create 10'000 tiddlers. They all look similar to this one and are 
>> about 800 bytes in size. 
>>
>> created: 20190603094535523
>> modified: 20190603094535523
>> tags: t-0
>> title: a-0-0
>> type: text/vnd.tiddlywiki
>>
>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
>> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
>> 123456789 123456789 123456789 
>>
>>
>> There is no problem for me. 
>>  - I use node v10.16.0 LTS. 
>>  - FF latest 
>>  - TW 5.1.19
>>  - win 10
>>
>> The tw server side needs about 20 seconds to start. 
>> It transferes about 12MByte to the browser, which needs about 4 seconds. 
>>
>> So how do you create your files?
>>
>> -mario
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/cdbfa07c-80bd-4ed9-b378-242ab9180f02%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[tw5] Re: File-based TW on node.js crashes when the number of tiddlers is more than ~2000

2019-06-03 Thread solex 
Hi mario,

My tiddlers only have title as an attribute, and the corresponding file 
name.
e.g.

tiddlers/test-tiddler-1.tid

title: test-tiddler-1

Initially they had other fields as well (including content-type), but I 
reduced them to a minimum while investigating, and the particular set of 
fields didn't seem to make any difference. 
Also, when the total number is lower (e.g. <1500), the wiki is functioning 
as expected, so I'm sure the problem is not in the tiddler generation.
The browser doesn't matter too (as expected).

Apparently the only difference is environment (Node version + OS), so 
perhaps I will try later on a different machine.

Thanks,
solex



On Monday, June 3, 2019 at 1:03:41 PM UTC+3, PMario wrote:
>
> Hi solex, 
>
> I did create 10'000 tiddlers. They all look similar to this one and are 
> about 800 bytes in size. 
>
> created: 20190603094535523
> modified: 20190603094535523
> tags: t-0
> title: a-0-0
> type: text/vnd.tiddlywiki
>
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 123456789 123456789 123456789 123456789 
> 123456789 123456789 123456789 
>
>
> There is no problem for me. 
>  - I use node v10.16.0 LTS. 
>  - FF latest 
>  - TW 5.1.19
>  - win 10
>
> The tw server side needs about 20 seconds to start. 
> It transferes about 12MByte to the browser, which needs about 4 seconds. 
>
> So how do you create your files?
>
> -mario
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/eba7c040-6e3d-49e0-bc75-17e0cdcf6c74%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[tw5] Re: File-based TW on node.js crashes when the number of tiddlers is more than ~2000

2019-06-03 Thread PMario 
Hi solex, 

I did create 10'000 tiddlers. They all look similar to this one and are 
about 800 bytes in size. 

created: 20190603094535523
modified: 20190603094535523
tags: t-0
title: a-0-0
type: text/vnd.tiddlywiki

123456789 123456789 123456789 123456789 123456789 123456789 123456789 
123456789 123456789 123456789 123456789 123456789 123456789 123456789 
123456789 123456789 123456789 123456789 123456789 123456789 123456789 
123456789 123456789 123456789 123456789 123456789 123456789 123456789 
123456789 123456789 123456789 123456789 123456789 123456789 123456789 
123456789 123456789 123456789 123456789 123456789 123456789 123456789 
123456789 123456789 123456789 123456789 123456789 123456789 123456789 
123456789 123456789 123456789 123456789 123456789 123456789 123456789 
123456789 123456789 123456789 123456789 123456789 123456789 123456789 
123456789 123456789 123456789 123456789 123456789 123456789 123456789 
123456789 123456789 123456789 123456789 123456789 123456789 123456789 
123456789 123456789 123456789 


There is no problem for me. 
 - I use node v10.16.0 LTS. 
 - FF latest 
 - TW 5.1.19
 - win 10

The tw server side needs about 20 seconds to start. 
It transferes about 12MByte to the browser, which needs about 4 seconds. 

So how do you create your files?

-mario


-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/7484d536-1423-425c-bd67-2e8eb5b05674%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.