Re: [Wikitech-l] Editing JSON in MediaWiki

2017-04-11 Thread Daniel Kinzler 
Am 11.04.2017 um 02:35 schrieb Denny Vrandečić:
> Unfortunately, I am unsure whether I can restrict the usage this tightly,
> and whether I'd rather let the users shoot themselves in the foot or
> restrict them from doing so... age old question!

That indeed is one of the ultimate questions, from UI design to law making!

-- 
Daniel Kinzler
Senior Software Developer

Wikimedia Deutschland
Gesellschaft zur Förderung Freien Wissens e.V.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Editing JSON in MediaWiki

2017-04-11 Thread C. Scott Ananian 
...and inspired, I just submitted
https://wikimania2017.wikimedia.org/wiki/Submissions/Writing_Visual_Editor_and_Parsoid_extensions
  --scott

On Mon, Apr 10, 2017 at 3:15 PM, C. Scott Ananian 
wrote:

> More related functionality: Parsoid supports JSON as a contenttype, and
> emits a special type of table (the same as the HTML table generated by the
> HTML handler for JSON in mediawiki).  You can edit this in VE, although w/o
> any special support, and Parsoid will serialize it back to JSON.
>
> This could be turned into a very pleasant type-aware editor for JSON in VE
> pretty easily.
>  --scott
>
> On Sun, Apr 9, 2017 at 2:23 AM, Denny Vrandečić 
> wrote:
>
>> Here's my requirement:
>> - a wiki page is one JSON document
>> - when editing, the user edits the JSON directly
>> - when viewing, I have a viewer that turns the JSON into wikitext, and
>> that
>> wikitext gets rendered as wikitext and turned into HTML by MediaWiki
>>
>> I have several options, including:
>>
>> 1) hook for a tag like , and write an extension that parses the
>> content between the tags and turns it into wikitext (not ideal, as I don't
>> use any of the existing support for JSON stuff, and also I could have
>> several such tags per page, which does not fit with my requirements)
>>
>> 2) I found the JsonConfig extension by yurik. This allows me to do almost
>> all of the things above - but it returns HTML directly, not wikitext. It
>> doesn't seem trivial to be able to return wikitext instead of HTML, but
>> hopefully I am wrong? Also, this ties in nicely with the Code Editor.
>>
>> 3) there is actually a JsonContentHandler in core. But looking through it
>> it seems that this suffers from the same limitations - I can return HTML,
>> but not wikitext.
>>
>> 3 seems to have the advantage to be more actively worked on that 2 (which
>> is not based on 3, probably because it is older than 3). So future goodies
>> like a Json Schema validator will probably go to 3, but not to 2, so I
>> should probably go to 3.
>>
>> Writing this down, one solution could be to create the wikitext, and then
>> call the wikitext parser manually and have it create HTML?
>>
>> I have already developed the extension in 1, and then fully rewritten it
>> in
>> 2.  Before I go and rewrite it again in 3, I wanted to ask whether I am
>> doing it right, or if should do it completely differently, and also if
>> there are examples of stuff developed in 3, i.e. of extensions or features
>> using the JsonContent class.
>>
>> Example:
>> I have a JSON document
>> { "username": "Denny" }
>> which gets turned into wikitext
>> ''Hello, [[User:Denny|Denny]]!''
>> which then gets turned into the right HTML and displayed to the user, e.g.
>> Hello, Denny!
>>
>> Cheers,
>> Denny
>> ___
>> Wikitech-l mailing list
>> Wikitech-l@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
>
>
>
> --
> (http://cscott.net)
>



-- 
(http://cscott.net)
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Editing JSON in MediaWiki

2017-04-10 Thread Denny Vrandečić 
Good points. All too familiar from trying to parse and understand the ways
the templating system has been used :)

Unfortunately, I am unsure whether I can restrict the usage this tightly,
and whether I'd rather let the users shoot themselves in the foot or
restrict them from doing so... age old question!

On Mon, Apr 10, 2017 at 2:42 PM Bartosz Dziewoński 
wrote:

> On 2017-04-10 06:17, Denny Vrandečić wrote:
> > On Sat, Apr 8, 2017 at 11:30 PM James Hare  wrote:
> >
> >> Why, exactly, do you want a wikitext intermediary between your JSON and
> >> your HTML? The value of wikitext is that it’s a syntax that is easier to
> >> edit than HTML. But if it’s not the native format of your data, nor can
> >> browsers render it directly, what’s the point of having it?
> >>
> >
> > Ah, good question indeed. The reason is that users would be actually
> > putting fragments of wikitext into the JSON structure, and then the JSON
> > structure gets assembled into wikitext. Not only would I prefer to have
> the
> > users work with fragments of wikitext than fragments of HTML, but some
> > things are almost impossible with HTML - e.g. making internal links red
> or
> > blue depending on the existence of the article, etc.
>
> It would be more reliable to parse each fragment of wikitext separately,
> and then build the HTML out of them. If you try to make a big chunk of
> wikitext with user-supplied fragments, you'll soon run into two problems:
>
> * Users will accidentally put '' or something into one of the
> fields and completely mess up the rendering.
> * Users will intentionally put '[[Foo|' in one field and ']]' into
> another and then be mad at you when you change the structure in a minor
> way and their link no longer works.
>
> --
> Bartosz Dziewoński
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Editing JSON in MediaWiki

2017-04-10 Thread Bartosz Dziewoński 

On 2017-04-10 06:17, Denny Vrandečić wrote:

On Sat, Apr 8, 2017 at 11:30 PM James Hare  wrote:


Why, exactly, do you want a wikitext intermediary between your JSON and
your HTML? The value of wikitext is that it’s a syntax that is easier to
edit than HTML. But if it’s not the native format of your data, nor can
browsers render it directly, what’s the point of having it?



Ah, good question indeed. The reason is that users would be actually
putting fragments of wikitext into the JSON structure, and then the JSON
structure gets assembled into wikitext. Not only would I prefer to have the
users work with fragments of wikitext than fragments of HTML, but some
things are almost impossible with HTML - e.g. making internal links red or
blue depending on the existence of the article, etc.


It would be more reliable to parse each fragment of wikitext separately, 
and then build the HTML out of them. If you try to make a big chunk of 
wikitext with user-supplied fragments, you'll soon run into two problems:


* Users will accidentally put '' or something into one of the 
fields and completely mess up the rendering.
* Users will intentionally put '[[Foo|' in one field and ']]' into 
another and then be mad at you when you change the structure in a minor 
way and their link no longer works.


--
Bartosz Dziewoński

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Editing JSON in MediaWiki

2017-04-10 Thread Denny Vrandečić 
Uh, that sounds nifty. Thanks for that!

On Mon, Apr 10, 2017 at 12:15 PM C. Scott Ananian 
wrote:

> More related functionality: Parsoid supports JSON as a contenttype, and
> emits a special type of table (the same as the HTML table generated by the
> HTML handler for JSON in mediawiki).  You can edit this in VE, although w/o
> any special support, and Parsoid will serialize it back to JSON.
>
> This could be turned into a very pleasant type-aware editor for JSON in VE
> pretty easily.
>  --scott
>
> On Sun, Apr 9, 2017 at 2:23 AM, Denny Vrandečić 
> wrote:
>
> > Here's my requirement:
> > - a wiki page is one JSON document
> > - when editing, the user edits the JSON directly
> > - when viewing, I have a viewer that turns the JSON into wikitext, and
> that
> > wikitext gets rendered as wikitext and turned into HTML by MediaWiki
> >
> > I have several options, including:
> >
> > 1) hook for a tag like , and write an extension that parses the
> > content between the tags and turns it into wikitext (not ideal, as I
> don't
> > use any of the existing support for JSON stuff, and also I could have
> > several such tags per page, which does not fit with my requirements)
> >
> > 2) I found the JsonConfig extension by yurik. This allows me to do almost
> > all of the things above - but it returns HTML directly, not wikitext. It
> > doesn't seem trivial to be able to return wikitext instead of HTML, but
> > hopefully I am wrong? Also, this ties in nicely with the Code Editor.
> >
> > 3) there is actually a JsonContentHandler in core. But looking through it
> > it seems that this suffers from the same limitations - I can return HTML,
> > but not wikitext.
> >
> > 3 seems to have the advantage to be more actively worked on that 2 (which
> > is not based on 3, probably because it is older than 3). So future
> goodies
> > like a Json Schema validator will probably go to 3, but not to 2, so I
> > should probably go to 3.
> >
> > Writing this down, one solution could be to create the wikitext, and then
> > call the wikitext parser manually and have it create HTML?
> >
> > I have already developed the extension in 1, and then fully rewritten it
> in
> > 2.  Before I go and rewrite it again in 3, I wanted to ask whether I am
> > doing it right, or if should do it completely differently, and also if
> > there are examples of stuff developed in 3, i.e. of extensions or
> features
> > using the JsonContent class.
> >
> > Example:
> > I have a JSON document
> > { "username": "Denny" }
> > which gets turned into wikitext
> > ''Hello, [[User:Denny|Denny]]!''
> > which then gets turned into the right HTML and displayed to the user,
> e.g.
> > Hello, Denny!
> >
> > Cheers,
> > Denny
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
>
>
>
> --
> (http://cscott.net)
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Editing JSON in MediaWiki

2017-04-10 Thread Denny Vrandečić 
Ah, just an internal data structure. In the end, the UI will be form-based
anyway. But in these forms, the user will be able to enter some wikitext
fragments. Yucky, I know.

I prefer JSON over XML only because it is the Zeitgeist, and I expect the
tool support for JSON to grow whereas I don't see similar activity around
XML in the MediaWiki development community.

Or, put differently, the same reason Wikibase is using JSON.

On Mon, Apr 10, 2017 at 11:06 AM Daniel Kinzler 
wrote:

> Am 10.04.2017 um 06:17 schrieb Denny Vrandečić:
> > Ah, good question indeed. The reason is that users would be actually
> > putting fragments of wikitext into the JSON structure, and then the JSON
> > structure gets assembled into wikitext. Not only would I prefer to have
> the
> > users work with fragments of wikitext than fragments of HTML, but some
> > things are almost impossible with HTML - e.g. making internal links red
> or
> > blue depending on the existence of the article, etc.
>
> Why JSON? JSON is an absolute pain to edit by hand. HTML/XML, as annoying
> as it
> is, is still much better for that than JSON is. And Wikitext is designed
> to mix
> well with HTML.
>
> --
> Daniel Kinzler
> Principal Platform Engineer
>
> Wikimedia Deutschland
> Gesellschaft zur Förderung Freien Wissens e.V.
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Editing JSON in MediaWiki

2017-04-10 Thread C. Scott Ananian 
More related functionality: Parsoid supports JSON as a contenttype, and
emits a special type of table (the same as the HTML table generated by the
HTML handler for JSON in mediawiki).  You can edit this in VE, although w/o
any special support, and Parsoid will serialize it back to JSON.

This could be turned into a very pleasant type-aware editor for JSON in VE
pretty easily.
 --scott

On Sun, Apr 9, 2017 at 2:23 AM, Denny Vrandečić  wrote:

> Here's my requirement:
> - a wiki page is one JSON document
> - when editing, the user edits the JSON directly
> - when viewing, I have a viewer that turns the JSON into wikitext, and that
> wikitext gets rendered as wikitext and turned into HTML by MediaWiki
>
> I have several options, including:
>
> 1) hook for a tag like , and write an extension that parses the
> content between the tags and turns it into wikitext (not ideal, as I don't
> use any of the existing support for JSON stuff, and also I could have
> several such tags per page, which does not fit with my requirements)
>
> 2) I found the JsonConfig extension by yurik. This allows me to do almost
> all of the things above - but it returns HTML directly, not wikitext. It
> doesn't seem trivial to be able to return wikitext instead of HTML, but
> hopefully I am wrong? Also, this ties in nicely with the Code Editor.
>
> 3) there is actually a JsonContentHandler in core. But looking through it
> it seems that this suffers from the same limitations - I can return HTML,
> but not wikitext.
>
> 3 seems to have the advantage to be more actively worked on that 2 (which
> is not based on 3, probably because it is older than 3). So future goodies
> like a Json Schema validator will probably go to 3, but not to 2, so I
> should probably go to 3.
>
> Writing this down, one solution could be to create the wikitext, and then
> call the wikitext parser manually and have it create HTML?
>
> I have already developed the extension in 1, and then fully rewritten it in
> 2.  Before I go and rewrite it again in 3, I wanted to ask whether I am
> doing it right, or if should do it completely differently, and also if
> there are examples of stuff developed in 3, i.e. of extensions or features
> using the JsonContent class.
>
> Example:
> I have a JSON document
> { "username": "Denny" }
> which gets turned into wikitext
> ''Hello, [[User:Denny|Denny]]!''
> which then gets turned into the right HTML and displayed to the user, e.g.
> Hello, Denny!
>
> Cheers,
> Denny
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l




-- 
(http://cscott.net)
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Editing JSON in MediaWiki

2017-04-10 Thread Daniel Kinzler 
Am 10.04.2017 um 06:17 schrieb Denny Vrandečić:
> Ah, good question indeed. The reason is that users would be actually
> putting fragments of wikitext into the JSON structure, and then the JSON
> structure gets assembled into wikitext. Not only would I prefer to have the
> users work with fragments of wikitext than fragments of HTML, but some
> things are almost impossible with HTML - e.g. making internal links red or
> blue depending on the existence of the article, etc.

Why JSON? JSON is an absolute pain to edit by hand. HTML/XML, as annoying as it
is, is still much better for that than JSON is. And Wikitext is designed to mix
well with HTML.

-- 
Daniel Kinzler
Principal Platform Engineer

Wikimedia Deutschland
Gesellschaft zur Förderung Freien Wissens e.V.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Editing JSON in MediaWiki

2017-04-10 Thread Denny Vrandečić 
Thanks Brad, that's perfect! I'll proceed as suggested.

Thanks list, that was really helpful and saved me plenty of trial and
errors!

On Mon, Apr 10, 2017 at 7:07 AM Brad Jorsch (Anomie) 
wrote:

> On Sun, Apr 9, 2017 at 11:38 AM, Daniel Kinzler <
> daniel.kinz...@wikimedia.de
> > wrote:
>
> > Generating wikitext from some other thing is what Scribunto does.
>
>
> Not really. What Scribunto does is let you run a program to generate
> wikitext.
>
> If you wanted to write code that took some JSON and turned it into wikitext
> without going through all the trouble of writing an extension and getting
> it deployed, you might write that code in Lua as a Scribunto module.
>
>
> On Mon, Apr 10, 2017 at 12:17 AM, Denny Vrandečić 
> wrote:
>
> > On Sat, Apr 8, 2017 at 11:30 PM James Hare  wrote:
> >
> > > Why, exactly, do you want a wikitext intermediary between your JSON and
> > > your HTML? The value of wikitext is that it’s a syntax that is easier
> to
> > > edit than HTML. But if it’s not the native format of your data, nor can
> > > browsers render it directly, what’s the point of having it?
> >
> > Ah, good question indeed. The reason is that users would be actually
> > putting fragments of wikitext into the JSON structure, and then the JSON
> > structure gets assembled into wikitext. Not only would I prefer to have
> the
> > users work with fragments of wikitext than fragments of HTML, but some
> > things are almost impossible with HTML - e.g. making internal links red
> or
> > blue depending on the existence of the article, etc.
> >
>
> What you probably want to do then is to extend JsonContent and
> JsonContentHandler. In the fillParserOutput() method, you'd convert the
> JSON to wikitext and then pass that wikitext to the Parser; for the latter
> step you could look at how WikitextContent does it.
>
> You might also look at implementing Content::getWikitextForTransclusion()
> to let people transclude the resulting wikitext.
>
>
> --
> Brad Jorsch (Anomie)
> Senior Software Engineer
> Wikimedia Foundation
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Editing JSON in MediaWiki

2017-04-10 Thread Brad Jorsch (Anomie) 
On Sun, Apr 9, 2017 at 11:38 AM, Daniel Kinzler  wrote:

> Generating wikitext from some other thing is what Scribunto does.


Not really. What Scribunto does is let you run a program to generate
wikitext.

If you wanted to write code that took some JSON and turned it into wikitext
without going through all the trouble of writing an extension and getting
it deployed, you might write that code in Lua as a Scribunto module.


On Mon, Apr 10, 2017 at 12:17 AM, Denny Vrandečić 
wrote:

> On Sat, Apr 8, 2017 at 11:30 PM James Hare  wrote:
>
> > Why, exactly, do you want a wikitext intermediary between your JSON and
> > your HTML? The value of wikitext is that it’s a syntax that is easier to
> > edit than HTML. But if it’s not the native format of your data, nor can
> > browsers render it directly, what’s the point of having it?
>
> Ah, good question indeed. The reason is that users would be actually
> putting fragments of wikitext into the JSON structure, and then the JSON
> structure gets assembled into wikitext. Not only would I prefer to have the
> users work with fragments of wikitext than fragments of HTML, but some
> things are almost impossible with HTML - e.g. making internal links red or
> blue depending on the existence of the article, etc.
>

What you probably want to do then is to extend JsonContent and
JsonContentHandler. In the fillParserOutput() method, you'd convert the
JSON to wikitext and then pass that wikitext to the Parser; for the latter
step you could look at how WikitextContent does it.

You might also look at implementing Content::getWikitextForTransclusion()
to let people transclude the resulting wikitext.


-- 
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Editing JSON in MediaWiki

2017-04-09 Thread Denny Vrandečić 
Thanks Tpt, that sounds relevant! I will take a look into the code. More
code examples are really useful to figure this all out :)

On Sun, Apr 9, 2017 at 9:53 AM Thomas PT  wrote:

> An other maybe relevant example : ProofreadPage has a content model for
> proofreading pages with multiple areas of Wikitext and use some for
> transclusion and others for rendering:
>
> *
> https://github.com/wikimedia/mediawiki-extensions-ProofreadPage/blob/master/includes/page/PageContent.php
> *
> https://github.com/wikimedia/mediawiki-extensions-ProofreadPage/blob/master/includes/page/PageContentHandler.php
>
> Thomas
>
> > Le 9 avr. 2017 à 18:44, Isarra Yos  a écrit :
> >
> > Sounds like what they want is what collaborationlistcontent/handler
> does, specifically, at least for now.
> >
> > On 09/04/17 06:30, James Hare wrote:
> >> Why, exactly, do you want a wikitext intermediary between your JSON and
> >> your HTML? The value of wikitext is that it’s a syntax that is easier to
> >> edit than HTML. But if it’s not the native format of your data, nor can
> >> browsers render it directly, what’s the point of having it?
> >>
> >> The CollaborationKit extension [0] has two content models,
> >> CollaborationHubContent and CollaborationListContent, and they have two
> >> different strategies for parsing. CollaborationHubContent takes
> validated
> >> JSON and puts out raw HTML; this is the most straightforward to work
> with.
> >> CollaborationListContent instead puts out wikitext that is fed into the
> >> parser, since it is expected that lists can be transcluded onto other
> >> pages, and this means using wikitext (as transcluded HTML is not an
> >> option). However, this creates a lot of limitations, including parser
> >> restrictions that make sense in the context of arbitrary wikitext
> parsing
> >> but not when the markup is provided directly by an extension. The long
> term
> >> plan is for CollaborationListContent to put out HTML, since it’s more
> >> straightforward than using a wikitext intermediary that the user does
> not
> >> see anyway.
> >>
> >> [0] https://www.mediawiki.org/wiki/Extension:CollaborationKit
> >>
> >> On April 8, 2017 at 11:23:38 PM, Denny Vrandečić (vrande...@gmail.com)
> >> wrote:
> >>
> >> Here's my requirement:
> >> - a wiki page is one JSON document
> >> - when editing, the user edits the JSON directly
> >> - when viewing, I have a viewer that turns the JSON into wikitext, and
> that
> >> wikitext gets rendered as wikitext and turned into HTML by MediaWiki
> >>
> >> I have several options, including:
> >>
> >> 1) hook for a tag like , and write an extension that parses the
> >> content between the tags and turns it into wikitext (not ideal, as I
> don't
> >> use any of the existing support for JSON stuff, and also I could have
> >> several such tags per page, which does not fit with my requirements)
> >>
> >> 2) I found the JsonConfig extension by yurik. This allows me to do
> almost
> >> all of the things above - but it returns HTML directly, not wikitext. It
> >> doesn't seem trivial to be able to return wikitext instead of HTML, but
> >> hopefully I am wrong? Also, this ties in nicely with the Code Editor.
> >>
> >> 3) there is actually a JsonContentHandler in core. But looking through
> it
> >> it seems that this suffers from the same limitations - I can return
> HTML,
> >> but not wikitext.
> >>
> >> 3 seems to have the advantage to be more actively worked on that 2
> (which
> >> is not based on 3, probably because it is older than 3). So future
> goodies
> >> like a Json Schema validator will probably go to 3, but not to 2, so I
> >> should probably go to 3.
> >>
> >> Writing this down, one solution could be to create the wikitext, and
> then
> >> call the wikitext parser manually and have it create HTML?
> >>
> >> I have already developed the extension in 1, and then fully rewritten
> it in
> >> 2. Before I go and rewrite it again in 3, I wanted to ask whether I am
> >> doing it right, or if should do it completely differently, and also if
> >> there are examples of stuff developed in 3, i.e. of extensions or
> features
> >> using the JsonContent class.
> >>
> >> Example:
> >> I have a JSON document
> >> { "username": "Denny" }
> >> which gets turned into wikitext
> >> ''Hello, [[User:Denny|Denny]]!''
> >> which then gets turned into the right HTML and displayed to the user,
> e.g.
> >> Hello, Denny!
> >>
> >> Cheers,
> >> Denny
> >> ___
> >> Wikitech-l mailing list
> >> Wikitech-l@lists.wikimedia.org
> >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >> ___
> >> Wikitech-l mailing list
> >> Wikitech-l@lists.wikimedia.org
> >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >
> >
> >
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> >

Re: [Wikitech-l] Editing JSON in MediaWiki

2017-04-09 Thread Denny Vrandečić 
On Sun, Apr 9, 2017 at 8:38 AM Daniel Kinzler 
wrote:

> Am 09.04.2017 um 08:23 schrieb Denny Vrandečić:
> > Here's my requirement:
> > - a wiki page is one JSON document
> > - when editing, the user edits the JSON directly
> > - when viewing, I have a viewer that turns the JSON into wikitext, and
> that
> > wikitext gets rendered as wikitext and turned into HTML by MediaWiki
>
> Quick thoughts off the top of my head:
>
> Generating wikitext from some other thing is what Scribunto does. Instead
> of
> using the Lua handler, you would make a handler for Json, with whetever
> rules
> you like for generating wikitext.
>

Thanks, I will take a look into how Scribunto does that.

>
> I have ne4ver looked at customizing Scribunto, but this seems to right
> place to
> plug this in.
>
> The alternative is to try to have some kind of "cascading" ContentHandler,
> that
> generates a WikiTextContent object first, and then turns that into HTML.
>
>
Not sure what you mean here - just create wikitext and then run it through
the parser?


>
> --
> Daniel Kinzler
> Senior Software Developer
>
> Wikimedia Deutschland
> Gesellschaft zur Förderung Freien Wissens e.V.
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Editing JSON in MediaWiki

2017-04-09 Thread Denny Vrandečić 
On Sun, Apr 9, 2017 at 4:11 AM Gergo Tisza  wrote:

> You probably want to subclass JsonContentHandler and add wikitext transform
>

What does it mean to add wikitext transform?


> and whatever else you need. For schemas, have a look
> at JsonSchemaContentHandler in EventLogging.
>

Thanks, that looks very helpful! I am surprised this is not part of Core,
or a stand-alone component, but it is in EventLogging. Thanks!




> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Editing JSON in MediaWiki

2017-04-09 Thread Denny Vrandečić 
On Sat, Apr 8, 2017 at 11:30 PM James Hare  wrote:

> Why, exactly, do you want a wikitext intermediary between your JSON and
> your HTML? The value of wikitext is that it’s a syntax that is easier to
> edit than HTML. But if it’s not the native format of your data, nor can
> browsers render it directly, what’s the point of having it?
>

Ah, good question indeed. The reason is that users would be actually
putting fragments of wikitext into the JSON structure, and then the JSON
structure gets assembled into wikitext. Not only would I prefer to have the
users work with fragments of wikitext than fragments of HTML, but some
things are almost impossible with HTML - e.g. making internal links red or
blue depending on the existence of the article, etc.


>
> The CollaborationKit extension [0] has two content models,
> CollaborationHubContent and CollaborationListContent, and they have two
> different strategies for parsing. CollaborationHubContent takes validated
> JSON and puts out raw HTML; this is the most straightforward to work with.
> CollaborationListContent instead puts out wikitext that is fed into the
> parser, since it is expected that lists can be transcluded onto other
> pages, and this means using wikitext (as transcluded HTML is not an
> option). However, this creates a lot of limitations, including parser
> restrictions that make sense in the context of arbitrary wikitext parsing
> but not when the markup is provided directly by an extension. The long term
> plan is for CollaborationListContent to put out HTML, since it’s more
> straightforward than using a wikitext intermediary that the user does not
> see anyway.
>

Thanks, that is super helpful to know! And thanks for the rationale in
particular.


>
> [0] https://www.mediawiki.org/wiki/Extension:CollaborationKit
>
> On April 8, 2017 at 11:23:38 PM, Denny Vrandečić (vrande...@gmail.com)
> wrote:
>
> Here's my requirement:
> - a wiki page is one JSON document
> - when editing, the user edits the JSON directly
> - when viewing, I have a viewer that turns the JSON into wikitext, and that
> wikitext gets rendered as wikitext and turned into HTML by MediaWiki
>
> I have several options, including:
>
> 1) hook for a tag like , and write an extension that parses the
> content between the tags and turns it into wikitext (not ideal, as I don't
> use any of the existing support for JSON stuff, and also I could have
> several such tags per page, which does not fit with my requirements)
>
> 2) I found the JsonConfig extension by yurik. This allows me to do almost
> all of the things above - but it returns HTML directly, not wikitext. It
> doesn't seem trivial to be able to return wikitext instead of HTML, but
> hopefully I am wrong? Also, this ties in nicely with the Code Editor.
>
> 3) there is actually a JsonContentHandler in core. But looking through it
> it seems that this suffers from the same limitations - I can return HTML,
> but not wikitext.
>
> 3 seems to have the advantage to be more actively worked on that 2 (which
> is not based on 3, probably because it is older than 3). So future goodies
> like a Json Schema validator will probably go to 3, but not to 2, so I
> should probably go to 3.
>
> Writing this down, one solution could be to create the wikitext, and then
> call the wikitext parser manually and have it create HTML?
>
> I have already developed the extension in 1, and then fully rewritten it in
> 2. Before I go and rewrite it again in 3, I wanted to ask whether I am
> doing it right, or if should do it completely differently, and also if
> there are examples of stuff developed in 3, i.e. of extensions or features
> using the JsonContent class.
>
> Example:
> I have a JSON document
> { "username": "Denny" }
> which gets turned into wikitext
> ''Hello, [[User:Denny|Denny]]!''
> which then gets turned into the right HTML and displayed to the user, e.g.
> Hello, Denny!
>
> Cheers,
> Denny
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Editing JSON in MediaWiki

2017-04-09 Thread Thomas PT 
An other maybe relevant example : ProofreadPage has a content model for 
proofreading pages with multiple areas of Wikitext and use some for 
transclusion and others for rendering:

* 
https://github.com/wikimedia/mediawiki-extensions-ProofreadPage/blob/master/includes/page/PageContent.php
* 
https://github.com/wikimedia/mediawiki-extensions-ProofreadPage/blob/master/includes/page/PageContentHandler.php

Thomas

> Le 9 avr. 2017 à 18:44, Isarra Yos  a écrit :
> 
> Sounds like what they want is what collaborationlistcontent/handler does, 
> specifically, at least for now.
> 
> On 09/04/17 06:30, James Hare wrote:
>> Why, exactly, do you want a wikitext intermediary between your JSON and
>> your HTML? The value of wikitext is that it’s a syntax that is easier to
>> edit than HTML. But if it’s not the native format of your data, nor can
>> browsers render it directly, what’s the point of having it?
>> 
>> The CollaborationKit extension [0] has two content models,
>> CollaborationHubContent and CollaborationListContent, and they have two
>> different strategies for parsing. CollaborationHubContent takes validated
>> JSON and puts out raw HTML; this is the most straightforward to work with.
>> CollaborationListContent instead puts out wikitext that is fed into the
>> parser, since it is expected that lists can be transcluded onto other
>> pages, and this means using wikitext (as transcluded HTML is not an
>> option). However, this creates a lot of limitations, including parser
>> restrictions that make sense in the context of arbitrary wikitext parsing
>> but not when the markup is provided directly by an extension. The long term
>> plan is for CollaborationListContent to put out HTML, since it’s more
>> straightforward than using a wikitext intermediary that the user does not
>> see anyway.
>> 
>> [0] https://www.mediawiki.org/wiki/Extension:CollaborationKit
>> 
>> On April 8, 2017 at 11:23:38 PM, Denny Vrandečić (vrande...@gmail.com)
>> wrote:
>> 
>> Here's my requirement:
>> - a wiki page is one JSON document
>> - when editing, the user edits the JSON directly
>> - when viewing, I have a viewer that turns the JSON into wikitext, and that
>> wikitext gets rendered as wikitext and turned into HTML by MediaWiki
>> 
>> I have several options, including:
>> 
>> 1) hook for a tag like , and write an extension that parses the
>> content between the tags and turns it into wikitext (not ideal, as I don't
>> use any of the existing support for JSON stuff, and also I could have
>> several such tags per page, which does not fit with my requirements)
>> 
>> 2) I found the JsonConfig extension by yurik. This allows me to do almost
>> all of the things above - but it returns HTML directly, not wikitext. It
>> doesn't seem trivial to be able to return wikitext instead of HTML, but
>> hopefully I am wrong? Also, this ties in nicely with the Code Editor.
>> 
>> 3) there is actually a JsonContentHandler in core. But looking through it
>> it seems that this suffers from the same limitations - I can return HTML,
>> but not wikitext.
>> 
>> 3 seems to have the advantage to be more actively worked on that 2 (which
>> is not based on 3, probably because it is older than 3). So future goodies
>> like a Json Schema validator will probably go to 3, but not to 2, so I
>> should probably go to 3.
>> 
>> Writing this down, one solution could be to create the wikitext, and then
>> call the wikitext parser manually and have it create HTML?
>> 
>> I have already developed the extension in 1, and then fully rewritten it in
>> 2. Before I go and rewrite it again in 3, I wanted to ask whether I am
>> doing it right, or if should do it completely differently, and also if
>> there are examples of stuff developed in 3, i.e. of extensions or features
>> using the JsonContent class.
>> 
>> Example:
>> I have a JSON document
>> { "username": "Denny" }
>> which gets turned into wikitext
>> ''Hello, [[User:Denny|Denny]]!''
>> which then gets turned into the right HTML and displayed to the user, e.g.
>> Hello, Denny!
>> 
>> Cheers,
>> Denny
>> ___
>> Wikitech-l mailing list
>> Wikitech-l@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>> ___
>> Wikitech-l mailing list
>> Wikitech-l@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> 
> 
> 
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l



signature.asc
Description: Message signed with OpenPGP
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Editing JSON in MediaWiki

2017-04-09 Thread Isarra Yos 
Sounds like what they want is what collaborationlistcontent/handler 
does, specifically, at least for now.


On 09/04/17 06:30, James Hare wrote:

Why, exactly, do you want a wikitext intermediary between your JSON and
your HTML? The value of wikitext is that it’s a syntax that is easier to
edit than HTML. But if it’s not the native format of your data, nor can
browsers render it directly, what’s the point of having it?

The CollaborationKit extension [0] has two content models,
CollaborationHubContent and CollaborationListContent, and they have two
different strategies for parsing. CollaborationHubContent takes validated
JSON and puts out raw HTML; this is the most straightforward to work with.
CollaborationListContent instead puts out wikitext that is fed into the
parser, since it is expected that lists can be transcluded onto other
pages, and this means using wikitext (as transcluded HTML is not an
option). However, this creates a lot of limitations, including parser
restrictions that make sense in the context of arbitrary wikitext parsing
but not when the markup is provided directly by an extension. The long term
plan is for CollaborationListContent to put out HTML, since it’s more
straightforward than using a wikitext intermediary that the user does not
see anyway.

[0] https://www.mediawiki.org/wiki/Extension:CollaborationKit

On April 8, 2017 at 11:23:38 PM, Denny Vrandečić (vrande...@gmail.com)
wrote:

Here's my requirement:
- a wiki page is one JSON document
- when editing, the user edits the JSON directly
- when viewing, I have a viewer that turns the JSON into wikitext, and that
wikitext gets rendered as wikitext and turned into HTML by MediaWiki

I have several options, including:

1) hook for a tag like , and write an extension that parses the
content between the tags and turns it into wikitext (not ideal, as I don't
use any of the existing support for JSON stuff, and also I could have
several such tags per page, which does not fit with my requirements)

2) I found the JsonConfig extension by yurik. This allows me to do almost
all of the things above - but it returns HTML directly, not wikitext. It
doesn't seem trivial to be able to return wikitext instead of HTML, but
hopefully I am wrong? Also, this ties in nicely with the Code Editor.

3) there is actually a JsonContentHandler in core. But looking through it
it seems that this suffers from the same limitations - I can return HTML,
but not wikitext.

3 seems to have the advantage to be more actively worked on that 2 (which
is not based on 3, probably because it is older than 3). So future goodies
like a Json Schema validator will probably go to 3, but not to 2, so I
should probably go to 3.

Writing this down, one solution could be to create the wikitext, and then
call the wikitext parser manually and have it create HTML?

I have already developed the extension in 1, and then fully rewritten it in
2. Before I go and rewrite it again in 3, I wanted to ask whether I am
doing it right, or if should do it completely differently, and also if
there are examples of stuff developed in 3, i.e. of extensions or features
using the JsonContent class.

Example:
I have a JSON document
{ "username": "Denny" }
which gets turned into wikitext
''Hello, [[User:Denny|Denny]]!''
which then gets turned into the right HTML and displayed to the user, e.g.
Hello, Denny!

Cheers,
Denny
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l




___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Editing JSON in MediaWiki

2017-04-09 Thread zppix e 
That seems like a performance hog in the making in my opinion to not mention, 
mostly unneeded as far as I can tell.

Sent from my iPhone

> On Apr 9, 2017, at 6:11 AM, Gergo Tisza  wrote:
> 
> You probably want to subclass JsonContentHandler and add wikitext transform
> and whatever else you need. For schemas, have a look
> at JsonSchemaContentHandler in EventLogging.
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Editing JSON in MediaWiki

2017-04-09 Thread Daniel Kinzler 
Am 09.04.2017 um 08:23 schrieb Denny Vrandečić:
> Here's my requirement:
> - a wiki page is one JSON document
> - when editing, the user edits the JSON directly
> - when viewing, I have a viewer that turns the JSON into wikitext, and that
> wikitext gets rendered as wikitext and turned into HTML by MediaWiki

Quick thoughts off the top of my head:

Generating wikitext from some other thing is what Scribunto does. Instead of
using the Lua handler, you would make a handler for Json, with whetever rules
you like for generating wikitext.

I have ne4ver looked at customizing Scribunto, but this seems to right place to
plug this in.

The alternative is to try to have some kind of "cascading" ContentHandler, that
generates a WikiTextContent object first, and then turns that into HTML.


-- 
Daniel Kinzler
Senior Software Developer

Wikimedia Deutschland
Gesellschaft zur Förderung Freien Wissens e.V.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Editing JSON in MediaWiki

2017-04-09 Thread Gergo Tisza 
You probably want to subclass JsonContentHandler and add wikitext transform
and whatever else you need. For schemas, have a look
at JsonSchemaContentHandler in EventLogging.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Editing JSON in MediaWiki

2017-04-09 Thread James Hare 
Why, exactly, do you want a wikitext intermediary between your JSON and
your HTML? The value of wikitext is that it’s a syntax that is easier to
edit than HTML. But if it’s not the native format of your data, nor can
browsers render it directly, what’s the point of having it?

The CollaborationKit extension [0] has two content models,
CollaborationHubContent and CollaborationListContent, and they have two
different strategies for parsing. CollaborationHubContent takes validated
JSON and puts out raw HTML; this is the most straightforward to work with.
CollaborationListContent instead puts out wikitext that is fed into the
parser, since it is expected that lists can be transcluded onto other
pages, and this means using wikitext (as transcluded HTML is not an
option). However, this creates a lot of limitations, including parser
restrictions that make sense in the context of arbitrary wikitext parsing
but not when the markup is provided directly by an extension. The long term
plan is for CollaborationListContent to put out HTML, since it’s more
straightforward than using a wikitext intermediary that the user does not
see anyway.

[0] https://www.mediawiki.org/wiki/Extension:CollaborationKit

On April 8, 2017 at 11:23:38 PM, Denny Vrandečić (vrande...@gmail.com)
wrote:

Here's my requirement:
- a wiki page is one JSON document
- when editing, the user edits the JSON directly
- when viewing, I have a viewer that turns the JSON into wikitext, and that
wikitext gets rendered as wikitext and turned into HTML by MediaWiki

I have several options, including:

1) hook for a tag like , and write an extension that parses the
content between the tags and turns it into wikitext (not ideal, as I don't
use any of the existing support for JSON stuff, and also I could have
several such tags per page, which does not fit with my requirements)

2) I found the JsonConfig extension by yurik. This allows me to do almost
all of the things above - but it returns HTML directly, not wikitext. It
doesn't seem trivial to be able to return wikitext instead of HTML, but
hopefully I am wrong? Also, this ties in nicely with the Code Editor.

3) there is actually a JsonContentHandler in core. But looking through it
it seems that this suffers from the same limitations - I can return HTML,
but not wikitext.

3 seems to have the advantage to be more actively worked on that 2 (which
is not based on 3, probably because it is older than 3). So future goodies
like a Json Schema validator will probably go to 3, but not to 2, so I
should probably go to 3.

Writing this down, one solution could be to create the wikitext, and then
call the wikitext parser manually and have it create HTML?

I have already developed the extension in 1, and then fully rewritten it in
2. Before I go and rewrite it again in 3, I wanted to ask whether I am
doing it right, or if should do it completely differently, and also if
there are examples of stuff developed in 3, i.e. of extensions or features
using the JsonContent class.

Example:
I have a JSON document
{ "username": "Denny" }
which gets turned into wikitext
''Hello, [[User:Denny|Denny]]!''
which then gets turned into the right HTML and displayed to the user, e.g.
Hello, Denny!

Cheers,
Denny
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Editing JSON in MediaWiki

2017-04-09 Thread Denny Vrandečić 
Here's my requirement:
- a wiki page is one JSON document
- when editing, the user edits the JSON directly
- when viewing, I have a viewer that turns the JSON into wikitext, and that
wikitext gets rendered as wikitext and turned into HTML by MediaWiki

I have several options, including:

1) hook for a tag like , and write an extension that parses the
content between the tags and turns it into wikitext (not ideal, as I don't
use any of the existing support for JSON stuff, and also I could have
several such tags per page, which does not fit with my requirements)

2) I found the JsonConfig extension by yurik. This allows me to do almost
all of the things above - but it returns HTML directly, not wikitext. It
doesn't seem trivial to be able to return wikitext instead of HTML, but
hopefully I am wrong? Also, this ties in nicely with the Code Editor.

3) there is actually a JsonContentHandler in core. But looking through it
it seems that this suffers from the same limitations - I can return HTML,
but not wikitext.

3 seems to have the advantage to be more actively worked on that 2 (which
is not based on 3, probably because it is older than 3). So future goodies
like a Json Schema validator will probably go to 3, but not to 2, so I
should probably go to 3.

Writing this down, one solution could be to create the wikitext, and then
call the wikitext parser manually and have it create HTML?

I have already developed the extension in 1, and then fully rewritten it in
2.  Before I go and rewrite it again in 3, I wanted to ask whether I am
doing it right, or if should do it completely differently, and also if
there are examples of stuff developed in 3, i.e. of extensions or features
using the JsonContent class.

Example:
I have a JSON document
{ "username": "Denny" }
which gets turned into wikitext
''Hello, [[User:Denny|Denny]]!''
which then gets turned into the right HTML and displayed to the user, e.g.
Hello, Denny!

Cheers,
Denny
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l