[tw] Re: pandoc and markdown to TW format

[TonyM]

Mark,

I would think changing // to __ as an example is trivial. As long as one 
can reliable search and replace one markup for another in a set of search 
and replace steps you could build a macro for this purpose in NotePadd++ 
and run it against material constructed in TiddlyWiki mark(down). Even 
leading characters can be replaced using find "\n!!!" replace with "\n###", 
of course one should replace longer strings before shorter ones like find 
"\n!" replace with "\n#" after the previous example. Oh and line Trailing 
"]]\n".

Unfortunately you need to step through every conversion step before you 
will know if it is easy or not.

Regards
Tony

On Tuesday, November 21, 2017 at 1:39:12 AM UTC+11, Mark S. wrote:
>
> Looking at pandoc, it looks like one might have it do markdown to plain 
> text, and then create filters in between for each of the markdown types to 
> wrap the plain text the way TW text wants.
>
> But how much markdown to have to convert?
>
> -- Mark
>
> On Friday, November 17, 2017 at 9:16:46 AM UTC-8, Shay Shaked wrote:
>>
>> So I installed the markdown plugin, works ok, but the problem is that it 
>> doesn't play nice inside TW. some markings (like strikethrough) and my own 
>> Macros, highlight to text, journal entries and the like do not play nice in 
>> TW. So, I was looking at pandoc to help me convert markdown to WikiText, 
>> which exists, but then TW is not exactly WikiText
>>
>> Did anyone build a TW extension to pandoc perhaps? Can this be done? Or, 
>> is there a plugin to TW that understands WikiText? 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/7c8279ac-3c37-4212-87d4-3a76d088aeca%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [tw] Re: pandoc and markdown to TW format

[Shay Shaked]

This is all new stuff to me. I'm reading up on parsers, so this is beyond
me at this point.

It's simple enough to go over the md file for now and put in the formatting
myself. Since I know I will be doing this, I don't bother too much with
formatting; i create a list at the button of the entry with notes for
format, usually notes to what I want to link or what picture to add, and
that's it. I also use the search function in the browser to highlight
underscores when adding an article, so I known to use // for example.

On Mon, Nov 20, 2017, 18:58 TonyM  wrote:

> Josiah,
>
> The following is possible more for my own understanding, but may help
> describe the situation.
>
> When it comes to IMPLICIT rather than explicit there are hidden characters
> in any text be it new lines, a pair of new lines and more. It seems to me
> that along with some smart CSS determining if you can easily process an
> alternate markup can only be determined in its details, ie stepping through
> each of the detailed markup features and finding a way to display according
> the required standard. The idea would be to always retain the text in its
> custom markup and only display it according to standards just in time. This
> allows it to be imported/exported/viewed and edited in its native form.
> Since the act of displaying it, is one of the standards, HTML the fact is
> we can copy or save it in this display format as well for transfer to other
> documents or environments.
>
> The other approach would be to use tiddlywiki markup and macros as needed
> to display (not in finished presentation format) but in fountain markup
> which can be copied and pasted into a converter or tool for the final
> display.
>
> I suppose it all depends on which part(s) of the workflow you want
> tiddlywiki to be responsible.
>
> Regards
> Tony
>
>
> On Tuesday, November 21, 2017 at 12:01:03 AM UTC+11, @TiddlyTweeter wrote:
>>
>> Ciao TonyM
>>
>> TonyM wrote:
>>>
>>> Is it a third approach, or one of your two to use tiddler type to keep
>>> and edit in any desired markup standard and only render it as needed to
>>> html as currently happens with wikitext?
>>>
>>
>> For some types of specialised Markup, closely linked to application
>> types, it makes most sense to create a new Content Type and go much of the
>> way to creating final HTML for the rendered version.
>>
>> FOUNTAIN markup is an example that is widely used in writing screenplays. *It
>> makes much sense to author using Fountain when writing screenplays*
>> inTW. But its markup conflicts with TW standard markup.
>> https://fountain.io/syntax
>>
>> There are TWO ways forward (which you can mix) ... you convert for TW
>> compliant markup where you can (in other words change F. markup to
>> compliant TW markup in a pre-parser before the TW parser runs--after that
>> you get what you'd expect) AND/OR directly convert F. Markup to HTML.
>>
>> Fountain markup is interesting because its largely IMPLICIT (how you
>> space paragraphs & how you start them) whilst most normal wiki markup is
>> initiated by explicit characters (*/-/! etc). F. is great for screenplay
>> authors because *you rarely ever have to enter any explicit markup* if
>> you have your line layout right.
>>
>> But to use the F. approach in TW you need more than straight conversion
>> of markup systems because the "implicit" markup of F. TW has no equivalents
>> for. Its still pretty easy once you can access a pre-parser just using
>> regular expressions to do it though.
>>
>> All of this works under point (1) of my last post.
>>
>> Point (2) about CONVERSION is more a one-way-ticket. Its for cases where
>> you don't need to retain original markup, just permanently convert for
>> native TW normal.
>>
>> Best wishes
>> Josiah
>>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "TiddlyWiki" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/tiddlywiki/QcBqV2Cvxrw/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> tiddlywiki+unsubscr...@googlegroups.com.
> To post to this group, send email to tiddlywiki@googlegroups.com.
> Visit this group at https://groups.google.com/group/tiddlywiki.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/tiddlywiki/649b92f5-274a-431d-91ee-526295276c1c%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion 

[tw] Re: pandoc and markdown to TW format

[TonyM]

Josiah,

The following is possible more for my own understanding, but may help 
describe the situation.

When it comes to IMPLICIT rather than explicit there are hidden characters 
in any text be it new lines, a pair of new lines and more. It seems to me 
that along with some smart CSS determining if you can easily process an 
alternate markup can only be determined in its details, ie stepping through 
each of the detailed markup features and finding a way to display according 
the required standard. The idea would be to always retain the text in its 
custom markup and only display it according to standards just in time. This 
allows it to be imported/exported/viewed and edited in its native form. 
Since the act of displaying it, is one of the standards, HTML the fact is 
we can copy or save it in this display format as well for transfer to other 
documents or environments.

The other approach would be to use tiddlywiki markup and macros as needed 
to display (not in finished presentation format) but in fountain markup 
which can be copied and pasted into a converter or tool for the final 
display.

I suppose it all depends on which part(s) of the workflow you want 
tiddlywiki to be responsible.

Regards
Tony

On Tuesday, November 21, 2017 at 12:01:03 AM UTC+11, @TiddlyTweeter wrote:
>
> Ciao TonyM
>
> TonyM wrote:
>>
>> Is it a third approach, or one of your two to use tiddler type to keep 
>> and edit in any desired markup standard and only render it as needed to 
>> html as currently happens with wikitext?
>>
>
> For some types of specialised Markup, closely linked to application types, 
> it makes most sense to create a new Content Type and go much of the way to 
> creating final HTML for the rendered version.
>
> FOUNTAIN markup is an example that is widely used in writing screenplays. *It 
> makes much sense to author using Fountain when writing screenplays* inTW. 
> But its markup conflicts with TW standard markup. 
> https://fountain.io/syntax
>
> There are TWO ways forward (which you can mix) ... you convert for TW 
> compliant markup where you can (in other words change F. markup to 
> compliant TW markup in a pre-parser before the TW parser runs--after that 
> you get what you'd expect) AND/OR directly convert F. Markup to HTML. 
>
> Fountain markup is interesting because its largely IMPLICIT (how you space 
> paragraphs & how you start them) whilst most normal wiki markup is 
> initiated by explicit characters (*/-/! etc). F. is great for screenplay 
> authors because *you rarely ever have to enter any explicit markup* if 
> you have your line layout right.
>
> But to use the F. approach in TW you need more than straight conversion of 
> markup systems because the "implicit" markup of F. TW has no equivalents 
> for. Its still pretty easy once you can access a pre-parser just using 
> regular expressions to do it though.
>
> All of this works under point (1) of my last post.
>
> Point (2) about CONVERSION is more a one-way-ticket. Its for cases where 
> you don't need to retain original markup, just permanently convert for 
> native TW normal.
>
> Best wishes
> Josiah
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/649b92f5-274a-431d-91ee-526295276c1c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[tw] Re: pandoc and markdown to TW format

['Mark S.' via TiddlyWiki]

Looking at pandoc, it looks like one might have it do markdown to plain 
text, and then create filters in between for each of the markdown types to 
wrap the plain text the way TW text wants.

But how much markdown to have to convert?

-- Mark

On Friday, November 17, 2017 at 9:16:46 AM UTC-8, Shay Shaked wrote:
>
> So I installed the markdown plugin, works ok, but the problem is that it 
> doesn't play nice inside TW. some markings (like strikethrough) and my own 
> Macros, highlight to text, journal entries and the like do not play nice in 
> TW. So, I was looking at pandoc to help me convert markdown to WikiText, 
> which exists, but then TW is not exactly WikiText
>
> Did anyone build a TW extension to pandoc perhaps? Can this be done? Or, 
> is there a plugin to TW that understands WikiText? 
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/27413422-cd8a-48bf-9232-e8cc89dee7c0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[tw] Re: pandoc and markdown to TW format

[@TiddlyTweeter]

Ciao TonyM

TonyM wrote:
>
> Is it a third approach, or one of your two to use tiddler type to keep and 
> edit in any desired markup standard and only render it as needed to html as 
> currently happens with wikitext?
>

For some types of specialised Markup, closely linked to application types, 
it makes most sense to create a new Content Type and go much of the way to 
creating final HTML for the rendered version.

FOUNTAIN markup is an example that is widely used in writing screenplays. *It 
makes much sense to author using Fountain when writing screenplays* inTW. 
But its markup conflicts with TW standard markup. https://fountain.io/syntax

There are TWO ways forward (which you can mix) ... you convert for TW 
compliant markup where you can (in other words change F. markup to 
compliant TW markup in a pre-parser before the TW parser runs--after that 
you get what you'd expect) AND/OR directly convert F. Markup to HTML. 

Fountain markup is interesting because its largely IMPLICIT (how you space 
paragraphs & how you start them) whilst most normal wiki markup is 
initiated by explicit characters (*/-/! etc). F. is great for screenplay 
authors because *you rarely ever have to enter any explicit markup* if you 
have your line layout right.

But to use the F. approach in TW you need more than straight conversion of 
markup systems because the "implicit" markup of F. TW has no equivalents 
for. Its still pretty easy once you can access a pre-parser just using 
regular expressions to do it though.

All of this works under point (1) of my last post.

Point (2) about CONVERSION is more a one-way-ticket. Its for cases where 
you don't need to retain original markup, just permanently convert for 
native TW normal.

Best wishes
Josiah

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/06396888-4480-40f9-9970-6586bdb244cc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[tw] Re: pandoc and markdown to TW format

[TonyM]

Josiah,

Is it a third approach, or one of your two to use tiddler type to keep and edit 
in any desired markup standard and only render it as needed to html as 
currently happens with wikitext?

regards tony

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/b881ce12-bf1d-40aa-a833-27709f698db5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[tw] Re: pandoc and markdown to TW format

[@TiddlyTweeter]

Ciao Shay & PMario

I'm finding this thread both really interesting AND confusing. We need be 
clear how TW parsing works.

Let's take the case that you want a non-TiddlyWiki markup system converting 
to Tiddly-Wiki markup. The issue could be thought about at least two ways 
...

1 - you PRESERVE the original markup of your source and dynamically convert 
it to TW markup. For that the BJ plugins are relevant. In the current TW 
architecture it is a reasonable route. Both Flexity (mentioned before, that 
uses regular expressions) and the Flexible Types plugin 
that requires you to be 
able to write new JavaScript components would work. The point is both take 
hold of the parsing system and allows you to pre-run your own markup 
conversions.

2 - you CONVERT (permanently) the original markup to TW markup. Either via 
an import mechanism
or via (1) saving to a new Tiddler a rendered version of the final code.

A lot of the work in all such is done via regular expressions. For simple 
substitution, like "^ *-" (meaning "-" is the first non-space character in 
the line) for "*" are quite easy. The in-built parsers in TW have more 
sophistication allowing for more complex situations--but I think most 
simple markup can be "translated" dynamically easily to its TW equivalent 
and let TW then do the HTML render in the normal way. Or you could directly 
convert to HTML (more complex).

Best wishes
Josiah

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/29dde8bc-49b3-434b-a448-4815be54dca7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[tw] Re: pandoc and markdown to TW format

[PMario]

On Sunday, November 19, 2017 at 1:30:19 AM UTC+1, Shay Shaked wrote:
>
> I _know_ it can't be that simple, but don't know why.
>

You are absolute right. ... The basic parts are probably simple eg: 

static lists in markdown

 - element 1
 - element 2

static lists in tiddlywiki

* element 1
* element 2

That is simple, but boring, cumbersome and errorprone. 
Whenever a new element should be added, you have to search all the 
different occurrencies and edit them.
We use dynamic lists, built with tags, fields,  like so:

<> ... TW wikitext macros

{{{"[tag[myList]]"}}} ... which is a shorthand for: <$list 
filter="[tag[myList]]"/>

<$list filter="[tag[myList]]" template="myCustomListTemplate"/>

and a lot more: https://tiddlywiki.com/#ListWidget

That's just one reason why it's not simple.  

While pandoc is an exceptional piece of work, it only translates simple 
text a to simple text b.  

With TW we use transclusions a lot. see: 
https://tiddlywiki.com/#Transclusion
And that's the second reason why it isn't simple. 

and so on, and so on, ... 

have fun!
mario

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/f9ccc570-fb69-4417-8752-cbb82e631d31%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[tw] Re: pandoc and markdown to TW format

[Shay Shaked]

Here's my naaive, newbie theory:

I use Typora (Markdown) and TiddlyWiki (TW text). Typora had panodoc built into 
it, so it can convert to HTML, Wiki text, and bunch of other things. Pandoc has 
"extensions" (for lack of a better term) that "translate" Markdown to Wikitext. 

Ok, so, why not find the code that translate md to Wikitext, and change it so 
instead of translating # to , translate it to  for example. Continue 
with the rest. 

I _know_ it can't be that simple, but don't know why. Also, TW understands 
HTML, so that's a work around, but not exactly what I want. 

Can you, wise people, explain it to me in terms that I can grasp maybe? Thanks 
:-)

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/aa11c9df-77c5-47e2-a1fd-ee58cdd2cce7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[tw] Re: pandoc and markdown to TW format

[@TiddlyTweeter]

Ciao Shay

IF you understand how to write complex regular expressions I highly 
recommend BJ's FLEXITY PLUGIN  that 
will allow you to create **your own parser**.

In a previous post I adumbrated that TiddlyWiki has the potential to be a 
Universal Markup Engine--at least for the simpler markups and downs. I 
think there is a lot of mileage in that idea for inter-conversion of the 
simpler markup systems. 

The older systems of markup are looking a bit tired. There is no real 
reason that they should not be inter-changeable easily. Its more an inertia 
to vision forward than any technical obstacle.

Pandoc is something else. Its scope is huge and its DEDICATED to the one 
purpose of complex conversion. Unless you need complex XML etc I think its 
overkill.

Best wishes
Josiah

On Friday, 17 November 2017 18:16:46 UTC+1, Shay Shaked wrote:
>
> So I installed the markdown plugin, works ok, but the problem is that it 
> doesn't play nice inside TW. some markings (like strikethrough) and my own 
> Macros, highlight to text, journal entries and the like do not play nice in 
> TW. So, I was looking at pandoc to help me convert markdown to WikiText, 
> which exists, but then TW is not exactly WikiText
>
> Did anyone build a TW extension to pandoc perhaps? Can this be done? Or, 
> is there a plugin to TW that understands WikiText? 
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/00e48b85-01f8-4d20-b24c-34c8ad965fef%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[tw] Re: pandoc and markdown to TW format

[TonyM]

Shay,

There are so many formats there, I imagine it could be easy to add 
TiddlyWiki markup to the set.

Of more interest to me would be being able to import/export tiddlers in 
anyone of these nominated markups.

However I often find value customising the output differently to the source 
text anyway.

Regards
Tony

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/355e3bb7-3023-46d7-a6e9-2e9ecd055514%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[tw] Re: pandoc and markdown to TW format

[PMario]

On Friday, November 17, 2017 at 6:16:46 PM UTC+1, Shay Shaked wrote:
>
> ... but then TW is not exactly WikiText
>

That's right. TiddlyWiki has its own syntax. The TiddlyWiki syntax, which 
has much more possibilities than most other wiki system. 

TiddlyWiki wikitext exists since 2004. That's much longer then many other 
wiki systems. ... But as so often, it's not the the most powerfull 
wikitext, "that dominates the world". It's the one, that is "just good 
enough" to be usefull. ... 

Did anyone build a TW extension to pandoc perhaps? 
>

I don't know one. 
 

> Can this be done?
>

Sure, If someone, who knows how to program in Haskell, does it.  

-mario

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/af62580f-e1eb-4b25-9279-4039fb05a82f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.