Re: [fossil-users] Support for commonmark markdown in fossil

[Jan Nijtmans]

2017-03-14 16:27 GMT+01:00 Mark Janssen:
> Or in patch form:
...
http://fossil-scm.org/index.html/info/dd41f85acf57290a

Thanks!
 Jan Nijtmans
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Support for commonmark markdown in fossil

[Mark Janssen]

Or in patch form:

@@ -460,15 +460,16 @@

   while( i=size ) return 0;
-if( data[i]==c ) return i;

 /* not counting escaped chars */
 if( i && data[i-1]=='\\' ){
   i++;
   continue;
 }
+
+if( data[i]==c ) return i;

 /* skipping a code span */
 if( data[i]=='`' ){
   size_t span_nb = 0, bt;

On Tue, Mar 14, 2017 at 4:23 PM, Natacha Porté  wrote:
> Hello,
>
> on Tuesday 14 March 2017 at 15:44, Mark Janssen wrote:
>> I did notice a (IMO) bug during the conversion:
>>
>> $ cat markdown-test2.md
>> _test\_embedded_
>> *test\*embedded*
>> $ fossil test-markdown-render markdown-test2.md
>> 
>>
>> testembedded_
>> testembedded*
>>
>> 
>>
>> The escaped delimiters (\_ and \*) terminate the  but they shouldn't.
>
> I completely agree that this is a bug, and I'm very ashamed by it.
>
> In case somebody with more commit fluency than me wants to commit the
> fix, it's in markdown.c, almost at the beginning of the function
> `find_emph_char`, the line `if( data[i]==c ) return i;` should be moved
> blow the block `if( i && data[i-1]=='\\' ){`.
>
> As of this writing (head of trunk is 6f52316955), that's line 464 to
> move after the closing brace of line 470.
>
> If nobody beats me to it, and if I still have some commit rights, and if
> I figure whether it should go directly to trunk or into a to-be-reviewed
> branch (I'm very confident in the fix, but I might not be trusted enough
> to muddle directly with trunk yet), I will commit the fix myself.
>
>
> Thanks a lot for the report and sorry for the inconvenience,
> Natacha
> ___
> fossil-users mailing list
> fossil-users@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Support for commonmark markdown in fossil

[Natacha Porté]

Hello,

on Tuesday 14 March 2017 at 15:44, Mark Janssen wrote:
> I did notice a (IMO) bug during the conversion:
> 
> $ cat markdown-test2.md
> _test\_embedded_
> *test\*embedded*
> $ fossil test-markdown-render markdown-test2.md
> 
> 
> testembedded_
> testembedded*
> 
> 
> 
> The escaped delimiters (\_ and \*) terminate the  but they shouldn't.

I completely agree that this is a bug, and I'm very ashamed by it.

In case somebody with more commit fluency than me wants to commit the
fix, it's in markdown.c, almost at the beginning of the function
`find_emph_char`, the line `if( data[i]==c ) return i;` should be moved
blow the block `if( i && data[i-1]=='\\' ){`.

As of this writing (head of trunk is 6f52316955), that's line 464 to
move after the closing brace of line 470.

If nobody beats me to it, and if I still have some commit rights, and if
I figure whether it should go directly to trunk or into a to-be-reviewed
branch (I'm very confident in the fix, but I might not be trusted enough
to muddle directly with trunk yet), I will commit the fix myself.


Thanks a lot for the report and sorry for the inconvenience,
Natacha
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Support for commonmark markdown in fossil

[Mark Janssen]

On Sat, Mar 11, 2017 at 6:18 PM, Natacha Porté  wrote:
>
>
> I should really clarify that it was about *fenced* code blocks, since
> traditional markdown code blocks are correctly supported (unless there
> is a serious bug in there).
>
> I do understand the use of code blocks, and use them myself frequently.
> But only the indented version, because the fenced one feels to me like
> trading a lot of ease of reading for a little bit of ease of writing (or
> rather cut-and-pasting, I don't think it changes much for real writing).
> And that's without taking into account that on top of that most of my
> documents (even in markdown) are read much more often than written.
>

I must admit that after converting all the documents, I don't really
see the lack of fenced code blocks as an issue and I agree that the
readability is better when the blocks are indented (and yes I do use a
proper editor :))

I did notice a (IMO) bug during the conversion:

$ cat markdown-test2.md
_test\_embedded_
*test\*embedded*
$ fossil test-markdown-render markdown-test2.md


testembedded_
testembedded*



The escaped delimiters (\_ and \*) terminate the  but they shouldn't.

Regards,
Mark
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Support for commonmark markdown in fossil

[Warren Young]

On Mar 13, 2017, at 12:57 PM, Martin S. Weber  wrote:
> 
> On 03/13/17 19:00, Warren Young wrote:
>> On Mar 13, 2017, at 10:32 AM, Natacha Porté  wrote:
>>> Or do you expect it to count parentheses and prevent you from ever
>>> linking to a URL with unbalanced parentheses?
>> 
>> Yes.  Other Markdown processors do that.
> 
> It doesn't necessarily have to count parens, in the specific case mentioned 
> by Warren opting for the longest instead of the shortest syntactically 
> correct match would do as well. In other words, use
> regexen naively, duh :)

Regex matching of URLs is difficult:

   https://mathiasbynens.be/demo/url-regex

It doesn’t seem that src/markdown.c uses regexes.  At least, I don’t see any 
“regex” or “regcomp” text in that file.

A simpler method that may also work is:

1. Count parens and don’t close the URL until the paren count goes to 0.  (Open 
paren = +1, closed = -1.)

2. …unless you hit a whitespace, EOL, or other thing after a closed paren that 
clearly means the URL is finished.

That would let you have unbalanced parens, too.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Support for commonmark markdown in fossil

[David Vines]

On 13/03/2017 15:50, Warren Young wrote:


A related wish that comes up here now and then is some kind of pretty-printer 
support, so that common programming languages are colored nicely.  Google’s 
code-prettify JS library would work for this:

https://github.com/google/code-prettify



Admittedly it only does syntax colouring (and not formatting), but I found the instructions at the 
link below of how to add syntax colouring to a repository pretty straight forward to follow:


https://www.fossil-scm.org/xfer/wiki?name=Cookbook#source-hilight

Dave
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Support for commonmark markdown in fossil

[Martin S. Weber]

On 03/13/17 19:00, Warren Young wrote:

On Mar 13, 2017, at 10:32 AM, Natacha Porté  wrote:

Or do you expect it to count parentheses and prevent you from ever
linking to a URL with unbalanced parentheses?


Yes.  Other Markdown processors do that.


It doesn't necessarily have to count parens, in the specific case 
mentioned by Warren opting for the longest instead of the shortest 
syntactically correct match would do as well. In other words, use

regexen naively, duh :)
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Support for commonmark markdown in fossil

[Warren Young]

On Mar 13, 2017, at 10:32 AM, Natacha Porté  wrote:
> 
> on Monday 13 March 2017 at 09:50, Warren Young wrote:
>> I have found bugs in Fossil’s Markdown implementation.  For example, it 
>> doesn’t deal properly with hyperlinks to Wikipedia documents that end in a 
>> parenthesis, as when the Wikipedia topic needs disambiguation:
>> 
>>   I like [Fossil](https://en.wikipedia.org/wiki/Fossil_(software))!
> 
> ...
> Or do you expect it to count parentheses and prevent you from ever
> linking to a URL with unbalanced parentheses?

Yes.  Other Markdown processors do that.  Stack Exchange and GitHub, for two.

https://gist.github.com/wyoung/8cb1b16f128c7e6050fd71a3040488ee
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Support for commonmark markdown in fossil

[Natacha Porté]

Hello,

on Monday 13 March 2017 at 09:50, Warren Young wrote:
> I have found bugs in Fossil’s Markdown implementation.  For example, it 
> doesn’t deal properly with hyperlinks to Wikipedia documents that end in a 
> parenthesis, as when the Wikipedia topic needs disambiguation:
> 
>I like [Fossil](https://en.wikipedia.org/wiki/Fossil_(software))!

I'm not sure I really understand the problem here. The links ends at the
first unescaped closing parenthesis, so I would expect anyone wanting it
to be part of the URL to escape it, just like any literal star or
underscore or bracket or backtick in the text.

Or do you expect it to count parentheses and prevent you from ever
linking to a URL with unbalanced parentheses?


Hoping this helps,
Natacha
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Support for commonmark markdown in fossil

[Warren Young]

On Mar 11, 2017, at 7:07 AM, Mark Janssen  wrote:
> 
> the fossil markdown support is fairly limited (for example there are no code 
> blocks)

We must have different definitions of “code block.”

This is a Fossil wiki page with a very large code block down at the end:

https://tangentsoft.com/pidp8i/wiki?name=PEP001.PA

And it’s an interesting programming story besides, IMHO. :)

That code block — the amber-on-black box — is simply the boxed text you see 
indented four spaces, the same as specified by CommonMark.

Fossil also allows  and  in Markdown, if you want more control over 
what you get in the HTML output.

Fossil *doesn’t* support this “fenced code block” feature I see in CommonMark.  
Is that what you wanted?  I assume that has value for some indenting or quoting 
reason, but I have yet to find the older indented code block style unsuitable.

A related wish that comes up here now and then is some kind of pretty-printer 
support, so that common programming languages are colored nicely.  Google’s 
code-prettify JS library would work for this:

https://github.com/google/code-prettify

I have found bugs in Fossil’s Markdown implementation.  For example, it doesn’t 
deal properly with hyperlinks to Wikipedia documents that end in a parenthesis, 
as when the Wikipedia topic needs disambiguation:

   I like [Fossil](https://en.wikipedia.org/wiki/Fossil_(software))!

The URL is missing the trailing closing paren and an extra closing paren shows 
up in the HTML rendered output.

It would also be nice to have Markdown support in tickets.  I write Markdown 
text faster than I write Wiki formatted text.

> I have made a version of fossil which supports the commonmark markdown format

If it’s simply “stronger” than the current Markdown implementation on the 
common subset of the two Markdown flavors, I’m in favor of the change.  I don’t 
mind going back and tweaking my Markdown text to format correctly under the new 
system, especially if the differences amount to removing workarounds for 
Fossil’s current Markdown rendering engine.

> - (Maybe) add a repo wide markdown flavour setting

That sounds necessary, at least through the transition period, for much the 
same reason we how have “fossil set hash-policy”.  

If I update my local Fossil with your version and say “fossil ui” in a repo 
made by someone who doesn’t use a Fossil binary with your Markdown engine, the 
Markdown might not render properly if my Fossil doesn’t also know how to use 
the “legacy” rendering engine.

> would there be any interest in adding commonmark support?  

For now, I’m torn until you give me better examples than “no code block 
support.” :)
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Support for commonmark markdown in fossil

[Natacha Porté]

Hello,

on Saturday 11 March 2017 at 16:23, Mark Janssen wrote:
> I wasn't aware that the author of the current implementation was a fossil
> user as well :)

Being the author really helps integrating the code into fossil paradigm,
deal with licencing stuff, and adding new features efficiently. It might
even take less time for me to write support for fenced code blocks than
both e-mails on this mailing list :-)


> On Sat, 11 Mar 2017, 16:22 Natacha Porté,  wrote:
> > I never understood the appeal for code blocks, but if it's only that
> > it's very easy to add to the existing implementation.
> >
> 
> Code blocks are very convenient for displaying TIP examples or other pieces
> of example code.

I should really clarify that it was about *fenced* code blocks, since
traditional markdown code blocks are correctly supported (unless there
is a serious bug in there).

I do understand the use of code blocks, and use them myself frequently.
But only the indented version, because the fenced one feels to me like
trading a lot of ease of reading for a little bit of ease of writing (or
rather cut-and-pasting, I don't think it changes much for real writing).
And that's without taking into account that on top of that most of my
documents (even in markdown) are read much more often than written.

Of course, I'm bias by my using only decently-featured text editor. If I
had to insert manually a tabulation at the beginning of every line, I
would consider changing the editor before condering chaning the
language.


Natacha
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Support for commonmark markdown in fossil

[Mark Janssen]

I wasn't aware that the author of the current implementation was a fossil
user as well :)

On Sat, 11 Mar 2017, 16:22 Natacha Porté,  wrote:

>
> I never understood the appeal for code blocks, but if it's only that
> it's very easy to add to the existing implementation.
>

Code blocks are very convenient for displaying TIP examples or other pieces
of example code.

>
> However I would be very interested to know what other features it lacks,
> or how a single feature missing makes it "fairly limited".
>

Limited was maybe a bit harsh, personally I am just missing code blocks.
The rest was based on the score on the commonmark test suite.


> Don't get me wrong, I fully recognize the value of CommonMark, just not
> in features but rather in disambiguation and standardization.
>
>
> 
>
> And a fair warning, if anyone really wants standard-compliant
> CommonMark, and not just a few extra features, don't bother with the
> existing implementation, it's architectually at odds with the choices
> made in CommonMark.
>

Fair warning. I did consider adapting the current implementation, but
re-using the Commonmark reference implementation was less work :)

>
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Support for commonmark markdown in fossil

[Natacha Porté]

Hello,

on Saturday 11 March 2017 at 15:07, Mark Janssen wrote:
> Recently I have been looking to use fossil as a backend for managing the
> Tcl tip collection.
> An obvious format for the new tip format would be markdown, but currently
> the fossil markdown support is fairly limited (for example there are no
> code blocks)

I never understood the appeal for code blocks, but if it's only that
it's very easy to add to the existing implementation.

However I would be very interested to know what other features it lacks,
or how a single feature missing makes it "fairly limited".

Don't get me wrong, I fully recognize the value of CommonMark, just not
in features but rather in disambiguation and standardization.

Regarding the compatibility, as long as everybody kept their markdown
valid and non-contrieved, it should be completely transparent. People
who have stuff *like [that* ][link] might see quite some change there.

And a fair warning, if anyone really wants standard-compliant
CommonMark, and not just a few extra features, don't bother with the
existing implementation, it's architectually at odds with the choices
made in CommonMark. Better scratch it and write/import something
entirely different.


Natacha
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Support for commonmark markdown in fossil

[Scott Robison]

On Sat, Mar 11, 2017 at 8:08 AM, Mark Janssen  wrote:

> Good questions. Currently it replaces the existing markdown parser which
> can break existing files. This is why I suggested the repo wide setting.
>
Sorry, I missed that part.

> There are other possible solutions (switch on extension being one as well)
> but having different markdown flavours in one repo just feels wrong to me.
>
I'm torn on the two. I think I'd prefer the repo wide setting myself, or
even just replacing wholesale, but I can appreciate why people who've
already starting using the existing markdown might not like that. Switching
on an extension feels ugly to me, but it would allow people to gradually
change their repo over to using commonmark over time rather than having to
do it all at once.

In any case, I'm all for it.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Support for commonmark markdown in fossil

[Mark Janssen]

Good questions. Currently it replaces the existing markdown parser which
can break existing files. This is why I suggested the repo wide setting.
There are other possible solutions (switch on extension being one as well)
but having different markdown flavours in one repo just feels wrong to me.

On Sat, 11 Mar 2017, 15:38 Scott Robison,  wrote:

> On Sat, Mar 11, 2017 at 7:07 AM, Mark Janssen 
> wrote:
>
> My question to you all is, would there be any interest in adding
> commonmark support?
>
>
> I like the idea of a more fully featured markdown implementation. Would
> this replace the existing markdown support or be in addition to it? If
> replace, would it "break" existing repos that are using markdown? If people
> have .md files with the existing markdown support, might this need a
> different extension?
>
> --
> Scott Robison
>
> ___
> fossil-users mailing list
> fossil-users@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Support for commonmark markdown in fossil

[Scott Robison]

On Sat, Mar 11, 2017 at 7:07 AM, Mark Janssen  wrote:
>
> My question to you all is, would there be any interest in adding
> commonmark support?
>

I like the idea of a more fully featured markdown implementation. Would
this replace the existing markdown support or be in addition to it? If
replace, would it "break" existing repos that are using markdown? If people
have .md files with the existing markdown support, might this need a
different extension?

-- 
Scott Robison
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users