[twdev] Re: Making a CSV deserializer module... Error reports & wizard steps in the importer?

[Tobias Beer]

Hi Evan,

I like the general capability to import from csv genericaly.

Here's how I would think the process makes most sense:

   1. you drag a csv into the wiki
   2. the import then creates a tiddler from the overall csv whereas
  1. *text*: corresponds to the csv body
  2. *title*: corresponds to the csv title
  3. *csv-preset*: the parsing preset chosen to handle the csv (below)
  3. only *after* the importer created the tiddler from the csv you 
   then use a wizard allows to turn csv rows into tiddlers, and for that you 
   would have...
   4. the ability to define a number of csv parser definitions, each with...
  1. a csv to tiddler fields mapping
  2. tiddler field defaults, when source field was blank or as field 
  additions (e.g. "always add tag XYZ")
  3. csv delimiter
  4. csv quote character
  5. whether to overwrite existing tiddlers (or what else to do, e.g. 
  only update imported fields)
   5. *only after* you set up your csv parser rule right, you hit some 
   "create tiddlers" button that eventually creates those tiddlers
   6. before that, however, you should have a "*preview*" / preview button 
   (perhaps for the first row) which more or less does the same thing, only 
   just it puts the created tiddler(s) undder $:/temp for (p)review
   7. imported tiddlers should have accompanying metadata-fields regarding 
   the csv import (those meta fields could exist under some system tiddler 
   namespace) such as:
   1. *csv-imported*: the datestamp when *the individual tiddler* imported 
  / updated
  2. *csv-source*: the name of the csv source file
  3. *csv-preset*: the preset with which the tiddler was parsed / 
  imported
   8. a way to delete all created tiddlers (or metadata tiddlers)
  1. based on the above "csv-source" field
  2. so, there could be lists under the parser rule (grouped by *both* 
  import date and file name) showing imported tiddlers each, and a button 
to 
  batch delete them (or just delete related metadata concerning the csv 
  import, which also removes the tiddlers from the related list)
   
best --tb

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/b5dd25c8-c626-448b-afbe-e00bce8ca764%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [twdev] Making a CSV deserializer module... Error reports & wizard steps in the importer?

[Evan Balster]

Hey, Jeremy —

OK, good to know this stuff is on your radar.  For the time being, my 
plugin is functional (just less convenient than it could be).

My inclination would be to stand by for any future developments in 
TiddlyWiki's import workflow, and jump in and upgrade the plugin if and 
when those should occur.  I'm happy to participate in any conversations 
about the design of this system.

On Wednesday, 20 December 2017 16:59:57 UTC-6, Jeremy Ruston wrote:
>
> Hi Evan
>
> I'm working on this CSV "unpacker" plugin:  
> http://evanbalster.com/tiddlywiki/csv_unpack.html
>
>
> Good stuff.
>
> The introduction of deserializer modules, which I only discovered half-way 
> through the process, made things a lot easier.  But I'm a bit troubled at 
> the lack of a good mechanism for reporting errors in the import, or 
> allowing the user to customize the re-formatting of the data.
>
>
> The deserializer modules were originally primarily used in the boot 
> process where there is no possibility of showing a UI. (Similarly, they are 
> constrained to being synchronous because that's convenient for the boot 
> process).
>
> For the things I've built, I've used a few different workarounds:
>
> * For the xlsx plugin, there is a UI in the control panel, and the 
> deserialiser then refers to the tiddlers managed by that UI. That means 
> that the UI is separate to the import process
> * For the latest release of the text-slicer plugin, the functionality is 
> implemented as a toolbar button (and widget message under the covers), and 
> a modal dialogue is presented before the operation is performed
>
> None of them are particularly satisfactory. I think we need to:
>
> * extend the spec for deserializers to allow them to be asynchronous and 
> to present a UI
> * enhance the import mechanism to present a UI for selecting/overriding 
> the deserializer, and for interacting with the chosen deserializer
>
> There are some other enhancements that are overdue for the import 
> mechanism: diffs, and better handling of overwriting.
>
> There are a few scenarios I'm thinking about:
>
>- The CSV can't be parsed due to some error.
>   - The wiki opens a tiddler describing the error, in addition to the 
>   $:/Import tiddler.
>   - OR:  The wiki displays error information in the $:/import tiddler.
>   
>   - The CSV is parsed successfully and the user needs to map CSV 
>columns to field names in the wiki, previewing their effects.
>   - A configuration tiddler appears alongside $:/Import and allows 
>   the user to modify the imported data before finalizing the import.
>   - OR:  A "wizard" appears in the $:/Import tiddler and the user 
>   configures the import before proceeding to preview the tiddlers.
>
> Indeed, that's a good summary of what is needed.
>
> So, my questions:
>
>- Is it bad practice to create and/or display tiddlers from a 
>deserializer?
>
> Deserializers shouldn't have any side effects besides converting their 
> block of text to an array of tiddlers.
>
>
>- Is it possible to incorporate this sort of functionality (wizard 
>steps and error reporting) into the import mechanism in current or future 
>versions of TiddlyWiki?
>
> It would need some reengineering to make it possible, covering both the 
> deserializers and the wider import mechanism. Detailed discussion about the 
> code might be better at GitHub; if you're interested in exploring it I'd 
> recommend digging into the existing implementation first.
>
> Currently my plugin requires all setup to be performed in advance, using 
> the plugin's settings.  I notice that one of the built-in deserializers 
> displays import errors by creating an imported tiddler to act as an error 
> message; I might adopt this "hack" in lieu of something better.
>
>
> Both those techniques are useful.
>
> Best wishes
>
> Jeremy.
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "TiddlyWikiDev" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to tiddlywikide...@googlegroups.com .
> To post to this group, send email to tiddly...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/tiddlywikidev.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/tiddlywikidev/e1dd0adf-d80c-450e-91bb-3b6c8d20fdbe%40googlegroups.com
>  
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at 

Re: [twdev] Making a CSV deserializer module... Error reports & wizard steps in the importer?

[Jeremy Ruston]

Hi Evan

> I'm working on this CSV "unpacker" plugin:  
> http://evanbalster.com/tiddlywiki/csv_unpack.html

Good stuff.

> The introduction of deserializer modules, which I only discovered half-way 
> through the process, made things a lot easier.  But I'm a bit troubled at the 
> lack of a good mechanism for reporting errors in the import, or allowing the 
> user to customize the re-formatting of the data.

The deserializer modules were originally primarily used in the boot process 
where there is no possibility of showing a UI. (Similarly, they are constrained 
to being synchronous because that's convenient for the boot process).

For the things I've built, I've used a few different workarounds:

* For the xlsx plugin, there is a UI in the control panel, and the deserialiser 
then refers to the tiddlers managed by that UI. That means that the UI is 
separate to the import process
* For the latest release of the text-slicer plugin, the functionality is 
implemented as a toolbar button (and widget message under the covers), and a 
modal dialogue is presented before the operation is performed

None of them are particularly satisfactory. I think we need to:

* extend the spec for deserializers to allow them to be asynchronous and to 
present a UI
* enhance the import mechanism to present a UI for selecting/overriding the 
deserializer, and for interacting with the chosen deserializer

There are some other enhancements that are overdue for the import mechanism: 
diffs, and better handling of overwriting.

> There are a few scenarios I'm thinking about:
> The CSV can't be parsed due to some error.
> The wiki opens a tiddler describing the error, in addition to the $:/Import 
> tiddler.
> OR:  The wiki displays error information in the $:/import tiddler.
> 
> The CSV is parsed successfully and the user needs to map CSV columns to field 
> names in the wiki, previewing their effects.
> A configuration tiddler appears alongside $:/Import and allows the user to 
> modify the imported data before finalizing the import.
> OR:  A "wizard" appears in the $:/Import tiddler and the user configures the 
> import before proceeding to preview the tiddlers.
Indeed, that's a good summary of what is needed.

> So, my questions:
> Is it bad practice to create and/or display tiddlers from a deserializer?
Deserializers shouldn't have any side effects besides converting their block of 
text to an array of tiddlers.
> Is it possible to incorporate this sort of functionality (wizard steps and 
> error reporting) into the import mechanism in current or future versions of 
> TiddlyWiki?
It would need some reengineering to make it possible, covering both the 
deserializers and the wider import mechanism. Detailed discussion about the 
code might be better at GitHub; if you're interested in exploring it I'd 
recommend digging into the existing implementation first.

> Currently my plugin requires all setup to be performed in advance, using the 
> plugin's settings.  I notice that one of the built-in deserializers displays 
> import errors by creating an imported tiddler to act as an error message; I 
> might adopt this "hack" in lieu of something better.

Both those techniques are useful.

Best wishes

Jeremy.

> -- 
> You received this message because you are subscribed to the Google Groups 
> "TiddlyWikiDev" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to tiddlywikidev+unsubscr...@googlegroups.com.
> To post to this group, send email to tiddlywikidev@googlegroups.com.
> Visit this group at https://groups.google.com/group/tiddlywikidev.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/tiddlywikidev/e1dd0adf-d80c-450e-91bb-3b6c8d20fdbe%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/E5C7973E-9900-4BA1-9B79-2FE502EFA29E%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

[twdev] Re: Making a CSV deserializer module... Error reports & wizard steps in the importer?

[Joshua Fontany]

I just started looking at importing CSVs as well, to use with my 
RenderTable plugin. I will check out your code. Following with interest. :)

Best,
Joshua Fontany

On Wednesday, December 20, 2017 at 11:16:45 AM UTC-8, Evan Balster wrote:
>
> Hey, all —
>
> I'm working on this CSV "unpacker" plugin:  
> http://evanbalster.com/tiddlywiki/csv_unpack.html
>
> The introduction of deserializer modules, which I only discovered half-way 
> through the process, made things a lot easier.  But I'm a bit troubled at 
> the lack of a good mechanism for reporting errors in the import, or 
> allowing the user to customize the re-formatting of the data.
>
> There are a few scenarios I'm thinking about:
>
>- The CSV can't be parsed due to some error.
>   - The wiki opens a tiddler describing the error, in addition to the 
>   $:/Import tiddler.
>   - OR:  The wiki displays error information in the $:/import tiddler.
>   
>   - The CSV is parsed successfully and the user needs to map CSV 
>columns to field names in the wiki, previewing their effects.
>   - A configuration tiddler appears alongside $:/Import and allows 
>   the user to modify the imported data before finalizing the import.
>   - OR:  A "wizard" appears in the $:/Import tiddler and the user 
>   configures the import before proceeding to preview the tiddlers.
>
> So, my questions:
>
>- Is it bad practice to create and/or display tiddlers from a 
>deserializer?
>- Is it possible to incorporate this sort of functionality (wizard 
>steps and error reporting) into the import mechanism in current or future 
>versions of TiddlyWiki?
>
>
> Currently my plugin requires all setup to be performed in advance, using 
> the plugin's settings.  I notice that one of the built-in deserializers 
> displays import errors by creating an imported tiddler to act as an error 
> message; I might adopt this "hack" in lieu of something better.
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/bc98871d-f53a-43ce-b54e-a6d46473007a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[twdev] Making a CSV deserializer module... Error reports & wizard steps in the importer?

[Evan Balster]

Hey, all —

I'm working on this CSV "unpacker" plugin:  
http://evanbalster.com/tiddlywiki/csv_unpack.html

The introduction of deserializer modules, which I only discovered half-way 
through the process, made things a lot easier.  But I'm a bit troubled at 
the lack of a good mechanism for reporting errors in the import, or 
allowing the user to customize the re-formatting of the data.

There are a few scenarios I'm thinking about:

   - The CSV can't be parsed due to some error.
  - The wiki opens a tiddler describing the error, in addition to the 
  $:/Import tiddler.
  - OR:  The wiki displays error information in the $:/import tiddler.
  
  - The CSV is parsed successfully and the user needs to map CSV 
   columns to field names in the wiki, previewing their effects.
  - A configuration tiddler appears alongside $:/Import and allows the 
  user to modify the imported data before finalizing the import.
  - OR:  A "wizard" appears in the $:/Import tiddler and the user 
  configures the import before proceeding to preview the tiddlers.
   
So, my questions:

   - Is it bad practice to create and/or display tiddlers from a 
   deserializer?
   - Is it possible to incorporate this sort of functionality (wizard steps 
   and error reporting) into the import mechanism in current or future 
   versions of TiddlyWiki?
   

Currently my plugin requires all setup to be performed in advance, using 
the plugin's settings.  I notice that one of the built-in deserializers 
displays import errors by creating an imported tiddler to act as an error 
message; I might adopt this "hack" in lieu of something better.

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/e1dd0adf-d80c-450e-91bb-3b6c8d20fdbe%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [twdev] Proposing more sane fault tolerance for "unclosed" tags & markup

[Evan Balster]

Hey, Tobias —

I fear we will just delay the parsing error ...to the next closing tag 
> which now operates as a new opening tag.
>
> So, with...
>
> ''foo //bar'' baz// mumble and then some //more italics// but is it really?
>
> ...what happens if we forcefully close the italics at the second '', we 
> are then facing an opening italics marker that will screw up the subsequent 
> output right after *baz*.
> How do you suggest we ensure that *more italics* is indeed displayed in 
> *italics*, in the end?
>

So yes, this is one problem with the scheme — in trying to prevent one 
formatting "leak" we create another.  However, based on my inspection 
TiddlyWiki already parses every formatting run separately, which implies 
there might be a more graceful solution.  If I understand correctly, 
TiddlyWiki will identify the first two pairs of bold-quotes and the first 
two pairs of italic-slashes prior to generating a parse tree.  The trouble 
is in how it resolves them.  I propose that parse resolution could either 
accommodate the overlap or could close the italic section at the end of the 
bold section (leaving the text intact).  Either behavior would require more 
intricate logic in that resolution.

Jeremy, correct me if my appraisal of the parsing mechanism is incorrect 
here.  This is based on a shallow inspection of parse rules.


So, here's the thing: Rather than try and be smart or "graceful" about it,
> we should *stop all parsing completely* and throw an error,
> that the wikitext contains invalid formatting at the very first spot it 
> occurs.


To be a little blunt, this is contrary to web programming conventions.  
When a browser encounters something like this, it muddles through.  
Highlighting any syntax errors rather than attempting to emulate the 
"intended" behavior could be a productive compromise.  Halting the parser 
would be useful as an option, especially when making widget structures.


An alternative that I have seen in other markup implementations is to treat 
> line break as closing all open inline formatting elements. That would also 
> have significant impact on backwards compatibility, but might be easier to 
> implement.


I'm not keen to break backwards compatibility.  As a compromise, paragraphs 
(which are containers) could close formatting.


It occurs to me that the most productive parse rule to experiment with, at 
a baseline, would be HTML tags and widgets.


On Wednesday, 20 December 2017 11:21:23 UTC-6, @TiddlyTweeter wrote:
>
> Ciao Evan, Jeremy & Tobias
>
> Jeremy Ruston wrote:
>>
>> An alternative that I have seen in other markup implementations is to 
>> treat line break as closing all open inline formatting elements. 
>>
>
> Right. That behaviour is what I have see too. I suspect that's because the 
> simpler versions parse only a line at a time and just substitute HTML for 
> inline markup shorthand using regex that matches pairs sequentially. 
>
> That would also have significant impact on backwards compatibility, but 
>> might be easier to implement. 
>>
>
> Right. How much and often it would break back compatibility I'm unsure of.
>
> IMO all the lightweight markups got a bit out of control (see: CommonMark 
> & Babelmark 2 [A Markdown comparator]). They are easy. But parsers tended 
> to make it too easy to get away with fouling the footpath with malformed 
> markup by, typically, second guessing closing tags when only opening tags 
> had been created.
>
> I think Tobias' point is worth attention ...
>
>> Rather than try and be smart or "graceful" about it,
>> we should *stop all parsing completely* and throw an error ...
>
>
> Why? Because if you try to automate on all anomalies you end up with an 
> over complex smorgasbord to maintain. 
>
> I think one issue behind the problem is NOT just about poor final render 
> in edge cases. Its also about tolerance (or not) of user behaviours on 
> their original marking-up that code ends up over-supporting.
>
> Best wishes
> Josiah
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/9253b7fd-c888-4be4-9a8c-ee5a73d49cbd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [twdev] Proposing more sane fault tolerance for "unclosed" tags & markup

[@TiddlyTweeter]

Ciao Evan, Jeremy & Tobias

Jeremy Ruston wrote:
>
> An alternative that I have seen in other markup implementations is to 
> treat line break as closing all open inline formatting elements. 
>

Right. That behaviour is what I have see too. I suspect that's because the 
simpler versions parse only a line at a time and just substitute HTML for 
inline markup shorthand using regex that matches pairs sequentially. 

That would also have significant impact on backwards compatibility, but 
> might be easier to implement. 
>

Right. How much and often it would break back compatibility I'm unsure of.

IMO all the lightweight markups got a bit out of control (see: CommonMark & 
Babelmark 2 [A Markdown comparator]). They are easy. But parsers tended to 
make it too easy to get away with fouling the footpath with malformed 
markup by, typically, second guessing closing tags when only opening tags 
had been created.

I think Tobias' point is worth attention ...

> Rather than try and be smart or "graceful" about it,
> we should *stop all parsing completely* and throw an error ...


Why? Because if you try to automate on all anomalies you end up with an 
over complex smorgasbord to maintain. 

I think the one issue behind the problem is NOT just about poor final 
render in edge cases. Its also about tolerance (or not) of user behaviours 
on their original marking-up that code ends up over-supporting.

Best wishes
Josiah

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/80ec16df-fff9-4fbb-ac95-901661feefc8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[twdev] Re: Proposing more sane fault tolerance for "unclosed" tags & markup

[Tobias Beer]

Hi Evan,
 

> Aside from being spooky-unintuitive, the issues here can get quite severe 
> when editing complex widget code or upper sections of a long article, with 
> deletrious effects on browser performance.  I've seen browser lockups when 
> editing widget trees with fragmentary HTML.  For these reasons, I'm most 
> concerned with use-cases involving widgets nested in HTML structures like 
> tables (which unravel spectacularly while editing).
>

Put simply: do not use preview for editing such complex tiddlers.

I would like to propose an alternative fault-tolerance behavior for 
> unclosed tags and markup, which may require some changes to parsing logic.  
> Specifically, unclosed tags should be "contained" to their parent in the 
> parse tree, and closed when the parent is closed.
>

You would have to store the tag-nesting somewhere alongside the closed 
state.

Also, you would need to be aware that this only applies to //enclosing// 
parser rules, that have a beginning AND a closing pattern.

Now, if we're closing a parent, while the child is not closed, we 
forcefully also close the child.

I fear we will just delay the parsing error ...to the next closing tag 
which now operates as a new opening tag.

So, with...

''foo //bar'' baz// mumble and then some //more italics//

...what happens if we forcefully close the italics at the second '', we are 
then facing an opening italics marker that will screw up the subsequent 
output.
How do you suggest we ensure that *more italics* is indeed displayed in 
*italics*, in the end?

So, here's the thing: Rather than try and be smart or "graceful" about it,
we should *stop all parsing completely* and throw an error,
that the wikitext contains invalid formatting at the very first spot it 
occurs.

best -tb

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/3631cc7d-3da0-400b-9586-83c71a8592c2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[twdev] Re: Multiuser TW implementation idea

[Tobias Beer]

At second thought,

In my last examples, the server state tiddlers accompanying a tiddler 
should be relative to the user, so:

title: $:/state/server/FOO/JonDoe/AnyTiddler

instead of just

title: $:/state/server/FOO/AnyTiddler

best -tb

On Monday, 18 December 2017 14:32:12 UTC+1, Tobias Beer wrote:
>
> Hi David,
>
> The tiddler should not define / store server details such as who can 
> access it or who last did and when.
>
> As for me, TiddlyWeb had/has it all neatly figured out in terms of 
> separation of concerns, mostly:
>
> https://tank.peermore.com/tanks/tiddlyweb/resource
>
> So, I think, for TW5, there should not just be a tiddlywiki.info
> but also recipes that define what's served to whom based on profiles.
>
> However, on TiddlySpace as well, tiddlers were overloaded
> with server-meta data in the form of fields prefixed "server."
>
> To me, none of that stuff belongs into a tiddler.
> It is the server that has to know and be aware of how to handle a tiddler.
> In TW5 there now are system tiddlers that should encode accompanying 
> server metadata.
>
> Say you have a server (connection) called *FOO*.
>
> Then, for *AnyTiddler* loaded via / from *FOO*, there should be something 
> like:
>
> title: $:/state/server/FOO/AnyTiddler
> server-foo-bar: dfgw4324535z
> server-foo-baz: dfhjlkertlerku
> server-foo-fingerprint: iwherkfh34pq5up3qweurqpwiue
>
> ...so, a system tiddler that holds the metadata for that server connection,
> used by the syncer module to track state.
> And when the syncer module talks to the server,
> it will always wrap the actual tiddler with the state metadata.
>
> However, those details should *never* be stored with the actual tiddler.
> So, no *editableBy* fields at *AnyTiddler*. That should only go into the 
> state tiddler for the connection.
> Also consider that you may want to handle bags, rather than individual 
> tiddlers.
>
> Say you have two server connections one to *FOO* and one to *MUMBLE*.
> (Yes, not just one server and two clients but one client and two servers, 
> both handling the same tiddlers,
> e.g. you're distriuting authored contents, like plugins or other defaults.)
>
> So, you would have another config for the same tiddler:
>
> title: $:/state/server/MUMBLE/AnyTiddler
> server-mumble-stk: eqeoriu3pq
> server-mumble-xyz: dfhjlkertlerku
> server-mumble-fingerprint: lkdfjkjqwöerluqwoeriufoi802
>
> Now, those fields above, they mean something to each server
> and when there is a change written to or read from the server,
> the client needs some server specific syncer module that knows how to 
> compare
> what's coming with what's already there and handle conflicts adequately.
>
> Different server implementations might implement their own syncer modules
> as well as fields representing the server state of a tiddler.
>
> However, if you drag *AnyTiddler* to some other wiki,
> you will never drag along (any of) its server config, ever.
>
> So, cut short, the overall architecture that I would think needs 
> specifying is:
>
>- a server state namespace, so it's generally agreed upon where to 
>find connection details
>
>$:/state/server/SERVERXYZ/...
>
>- *server connection tiddler states*
>as exemplified above, under that namespace
>
>- *server connection tiddler state fields*
>which might be entirely different across servers, but some might 
>actually be the same
>
>- *a server mechanism to handle auth*
>
>that's probably the first and most important piece to the puzzle that 
>needs solving
>
>=> and it should probably never be in the form of tiddlers (at the 
>server)
>
> best -tb
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/121724a0-1c07-4bcc-9ee0-a8afa2922bfa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [twdev] Proposing more sane fault tolerance for "unclosed" tags & markup

[Jeremy Ruston]

Hi Evan

I agree that this is a troublesome problem, and I’d also be interested in 
exploring fixes.

I think that the approach you are suggesting is for the parser to check for the 
closure of parent components within a particular component; in your example, 
the second pair of double single quotes would be recognised as closing the 
first pair, despite the prevailing italic component. We could do that with a 
bit of refactoring but as you note there is a risk that we’d break the existing 
behaviour of nested components.

An alternative that I have seen in other markup implementations is to treat 
line break as closing all open inline formatting elements. That would also have 
significant impact on backwards compatibility, but might be easier to implement.

Best wishes

Jeremy



> On 19 Dec 2017, at 22:05, Evan Balster  wrote:
> 
> Hello, all —
> 
> I've been malcontent for some time with the way TiddlyWiki handles 
> "un-closed" tags and markup.  To give a simple example:
> 
> plain ''bold //bold-italic'' italic// plain
> 
> While the above is clearly wrong, TiddlyWiki's resolution is 
> 
> plain bold bold-italic italic 
> plain
> 
> which renders as
> 
> plain bold bold-italic italic plain
> 
> ...So that this localized markup error ends up affecting the entire remaining 
> content of the tiddler.  This is a "quirk" we're all familiar with, and the 
> example given here is innocuous, but this issue also affects unclosed HTML 
> tags and widgets.
> 
> Here's a slightly more onerous example:
> 
> <<<
> Evan Balster says ''Fault tolerance is awesome!
> <<< 
>  
> He later died of exhaustion
> 
> This results in 
> 
> Evan Balster says Fault tolerance is awesome! <<< He later died of exhaustion
> 
> ...which clearly wasn't our intention, and also implies the rest of our 
> tiddler will be a mess.  Compare your browser's behavior 
>  with analogs to both of these examples.
> 
> Aside from being spooky-unintuitive, the issues here can get quite severe 
> when editing complex widget code or upper sections of a long article, with 
> deletrious effects on browser performance.  I've seen browser lockups when 
> editing widget trees with fragmentary HTML.  For these reasons, I'm most 
> concerned with use-cases involving widgets nested in HTML structures like 
> tables (which unravel spectacularly while editing).
> 
> 
> I would like to propose an alternative fault-tolerance behavior for unclosed 
> tags and markup, which may require some changes to parsing logic.  
> Specifically, unclosed tags should be "contained" to their parent in the 
> parse tree, and closed when the parent is closed.
> 
> Under this approach:
> The first example will render as one of these depending on implementation:
> "plain bold bold-italic italic plain"  (italic markers are parsed separately)
> "plain bold bold-italic italic plain"  (parse tree resolution constrains the 
> italic section to the bold section, its parent)
> "plain bold bold-italic italic plain"  (parse tree resolution resolves the 
> intersection)
> The second example will terminate the bold section at the end of the block 
> quote.
> An unclosed <$set> or <$list> widget in a  tag should be closed prior to 
>  (rather than destructuring the entire table).
> Obviously this is a high-impact change.  It could be it's too extreme to do 
> it for non-tag elements, but I'd like to see what the architecture nerds 
> think.
> 
> 
> Here's a bit of code for a "sandbox" to tinker with this stuff firsthand:
> 
> <$edit-text tiddler="ParseTest/Data"/>
> 
> <<<
> {{ParseTest/Data}}
> <<<
> 
> HTML:
> 
> <$wikify name=html output=html text={{ParseTest/Data}}>
> <$text text=<>/>
> 
> 
> Parse tree:
> 
> <$wikify name=parsetree output=parsetree text={{ParseTest/Data}}>
> <$text text=<>/>
> 
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "TiddlyWikiDev" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to tiddlywikidev+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to tiddlywikidev@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/tiddlywikidev 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/tiddlywikidev/5c724111-0eda-402e-87bf-621e144932cb%40googlegroups.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to