I am trying to parse a large text file using TW in the browser. I already
know how to process text files using .Net, I enjpy learing what
programmatic capabilities are available in every environment I pick up.
The text file is simple enough to find records using splitregexp[/n] .
However, I know this particular file has a unique key for each record. I am
able to convert the file using .Net into a json file describing a
DataTiddler that I can import. This way I can reference each record by its
primary key. If there was a way to auto-number the list entries and persist
them to a DataTiddler, this would be close enough for me to selectively
process the records further without having to re-split the original list
over and over.
I saw several requests for adding a variable to the ListWidget, calling it
an iterator (Tobias) or addposition (Evans) on GitHub issue 3384. It was
closed as duplicate of issue 1523. Both issues mention concerns over how
the ListWidget creating a variable would actually cause re-rendering.
Instead of modifying the ListWidget directly, I was proposing an
intermediate step of preprocessing a list into a DataTiddler with
auto-numbered indexes. Then you can iterate over the DataTiddler indexes
to get the list position and use getindex to return the record.
When trying to use the iterator feature in a loop to call a set-field
action on a DataTiddler for each record, the button code generated can end
up being huge. I would just like to internally side-step the generating of
so many individual actions in the button. It would be like a 'copy tiiddler
text to DataTiddler' action with a list process in the miiddle and
auto-number on the insertions.
Just saying that out loud sounds like I'm probablly barking up the wrong
tree, because TW lists are not designed as an imperitive language.
DataTiddlers as an internal database table seem to work well as long as the
tables are generated outside of TW.
On Wednesday, December 16, 2020 at 3:07:06 PM UTC-8 TW Tones wrote:
> FYI:
>
> We can get an "index" number for items in a list, and we could use this in
> a button to set an index field in each tiddler or a data tiddler for later
> reference using the allbefore and count + 1
> <$list filter="[tag[TableOfContents]]">
> {{{ [tag[TableOfContents]allbefore<currentTiddler>count[]add[1]] }}}
> <$link/><br>
> </$list><br>
>
> However this will need the whole list to be processed for each item in the
> list. and could take a lot of time.
>
> - With a little trickery you may be able to make the numbering
> "re-start able", ie get the last issued and add more from there.
>
> Tones
> On Thursday, 17 December 2020 at 09:49:47 UTC+11 TW Tones wrote:
>
>> Is your primary purpose to assign a number to a set of tiddlers and
>> retain them indefinitely?
>> Is processing the large number of items only the first time, or on import
>> because we could assign "serial numbers" as created or as used?
>>
>> I suppose I need to understand why you need this, or to what purpose to
>> present one of a number of different methods.
>>
>> - The created date is almost, if not actually unique, and there is a
>> way to make them unique in retrospect if necessary.
>> - It is possible to store tiddler titles in a list that is itself in
>> the order required, without introducing index's
>> - List item can be generated automatically
>> - It is common in tiddlywiki to access tiddlers in a list by more
>> than one sort method, in different orders the serial number is less
>> important.
>> - I have built serial number assignment for tiddler if that would
>> satisfy your needs.
>>
>> Tones
>>
>> On Thursday, 17 December 2020 at 04:48:44 UTC+11 [email protected]
>> wrote:
>>
>>> Would it be desirable to add a new button action to persist a list
>>> filter directly into a DataTiddler, with the row number as the index?
>>>
>>> I can construct a DataTiddler like this manually using:
>>>
>>> * A count of entries from a filter
>>> * A range up to the count, subtracting one from each to make it base zero
>>> * Re-run the first filter, butfirst<current range value>, first entry
>>> * Set-field action to save the filter result to a DataTiddler, using the
>>> current range value as the index
>>>
>>> If the range filter exceed 10,000 records, this process cannot run. Even
>>> when I break the process out into smaller chunks, the throughput gets very
>>> slow after 20,000 records.
>>>
>>> Does anyone else think this could be useful when trying to pre-process a
>>> large amount of data?
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/tiddlywiki/440c444d-a5f3-4371-9912-e8ad9d6f3000n%40googlegroups.com.
