from:"C. Scott Ananian $Code Review$"

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Sync parserTests with core.

2017-01-25 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/334202 )

Change subject: Sync parserTests with core.
..

Sync parserTests with core.

New failing parser tests, added to blacklist:
* T156308: We don't properly strip whitespace around interlanguage links.
* T156309: We don't support {{PAGELANGUAGE}}.  (We actually need to have
  the page language available for LanguageConverter.)
* T43716: LanguageConverter syntax in attributes.

There were also two changes to the html2html stored in the blacklist,
caused by a change in the localized name of the image thumbnail attribute.
This is a side-effect of Id36550e864acf0df472b6a6a12f750eb28cbdd1d.

Change-Id: I09b794af5a8eb83b99f90f128f1c67efabec8576

blaclist extra

Change-Id: Ib95c4dfb27c2680873f11be823297d975b48cfea
---
M tests/parserTests-blacklist.js
M tests/parserTests.txt
M tools/fetch-parserTests.txt.js
3 files changed, 74 insertions(+), 7 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/02/334202/1

diff --git a/tests/parserTests-blacklist.js b/tests/parserTests-blacklist.js
index 29310ae..b9eb71f 100644
--- a/tests/parserTests-blacklist.js
+++ b/tests/parserTests-blacklist.js
@@ -58,6 +58,7 @@
 add("wt2html", "Self-link to numeric title", "0");
 add("wt2html", " inside a link", "Main Page the main page [it's not very good]");
 add("wt2html", "Interlanguage link with spacing", "Blah blah blah\nhttp://zh.wikipedia.org/wiki/%20%20%20%20Chinese%20%20%20%20%20\; 
data-parsoid='{\"stx\":\"simple\",\"a\":{\"href\":\"http://zh.wikipedia.org/wiki/%20%20%20%20Chinese%20%20%20%20%20\"},\"sa\":{\"href\":\;
   zh  :Chinese \"},\"dsr\":[15,43,null,null]}'/>");
+add("wt2html", "Escaping of interlanguage links (T129218, T156308)", "Blah blah blah\nhttp://es.wikipedia.org/wiki/Spanish\; title=\"es:Spanish\" 
data-parsoid='{\"stx\":\"simple\",\"a\":{\"href\":\"http://es.wikipedia.org/wiki/Spanish\"},\"sa\":{\"href\":\":es:Spanish\"},\"isIW\":true,\"dsr\":[15,30,2,2]}'>es:Spanish\n : zh : Chinese ");
 add("wt2html", "Space and question mark encoding in interlanguage links 
(T95473)", "Blah blah blah\nhttp://es.wikipedia.org/wiki/Foo%20bar?\; 
data-parsoid='{\"stx\":\"simple\",\"a\":{\"href\":\"http://es.wikipedia.org/wiki/Foo%20bar?\"},\"sa\":{\"href\":\"es:Foo
 bar?\"},\"dsr\":[15,30,null,null]}'/>");
 add("wt2html", "2. Lists with start-of-line-transparent tokens before bullets: 
Template close", "foo
 bar\n*baz");
 add("wt2html", "Multiple list tags generated by templates", "a\nb\nc");
@@ -93,6 +94,8 @@
 add("wt2html", "Magic Word: {{REVISIONID}}", "Warning:
 Page/template fetching disabled, and no cache for 
Template:REVISIONID");
 add("wt2html", "Magic Word: {{SCRIPTPATH}}", "/");
 add("wt2html", "Magic Word: {{STYLEPATH}}", "Warning:
 Page/template fetching disabled, and no cache for 
Template:STYLEPATH");
+add("wt2html", "Magic Word: {{PAGELANGUAGE}}", "Warning:
 Page/template fetching disabled, and no cache for 
Modèle:PAGELANGUAGE");
+add("wt2html", "Magic Word: {{PAGELANGUAGE}} on a page with no explicitly set 
language", "Warning:
 Page/template fetching disabled, and no cache for 
Template:PAGELANGUAGE");
 add("wt2html", "Case-sensitive magic words, when cased differently, should 
just be template transclusions", "Warning:
 Page/template fetching disabled, and no cache for 
Template:CurrentMonth\nWarning:
 Page/template fetching disabled, and no cache for 
Template:Currentday\nWarning:
 Page/template fetching disabled, and no cache for 
Template:CURreNTweEK\nWarning:
 Page/template fetching disabled, and no cache for 
Template:CurrentHour");
 add("wt2html", "Parsoid: Template-generated DISPLAYTITLE", "");
 add("wt2html", "Namespace 1 {{ns:1}}", "1");
@@ -214,6 +217,7 @@
 add("wt2html", "Unidirectional converter rule entries with an empty \"from\" 
string should be ignored (T53551)", "-{H|=>zh-cn:foo;}-foobar");
 add("wt2html", "Empty converter rule entries shouldn't be inserted into the 
conversion table (T53551)", "-{H|}-foobar");
 add("wt2html", "Nested using of manual convert syntax", "Nested: -{zh-hans:Hi 
-{zh-cn:China;zh-sg:Singapore;}-;zh-hant:Hello -{zh-tw:Taiwan;zh-hk:H-{ong}- 
K-{}-ong;}-;}-!");
+add("wt2html", "HTML markups with conversion syntax in attribs, nested in 
other conversion blocks", "-{zh;zh-hans;zh-hant|A}-");
 add("wt2html", "Proper conversion of text in external links", "http://www.google.com\; 
data-parsoid='{\"stx\":\"url\",\"dsr\":[0,21,0,0]}'>http://www.google.com\ngopher://www.google.com\; 
data-parsoid='{\"stx\":\"url\",\"dsr\":[22,45,0,0]}'>gopher://www.google.com\nhttp://www.google.com\; 
data-parsoid='{\"targetOff\":69,\"contentOffsets\":[69,90],\"dsr\":[46,91,23,1]}'>http://www.google.com\ngopher://www.google.com\; 
data-parsoid='{\"targetOff\":117,\"contentOffsets\":[117,140],\"dsr\":[92,141,25,1]}'>gopher://www.google.com\nhttps://www.google.com\;

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Sync up with Parsoid parserTests.

2017-01-25 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/334161 )

Change subject: Sync up with Parsoid parserTests.
..

Sync up with Parsoid parserTests.

This now aligns with Parsoid commit 643d5392bcf4dfebf906102627c51e8a608125bf

Change-Id: I4d7dc7378ca7cfdb3919f33959f58eb5c4d88ca8
---
M tests/parser/parserTests.txt
1 file changed, 176 insertions(+), 164 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/61/334161/1

diff --git a/tests/parser/parserTests.txt b/tests/parser/parserTests.txt
index ed09203..983c172 100644
--- a/tests/parser/parserTests.txt
+++ b/tests/parser/parserTests.txt
@@ -933,6 +933,7 @@
 ###
 ### multiple quote sequences in a line
 ###
+
 !! test
 Italics and bold: multiple quote sequences: (2,4,2)
 !! options
@@ -942,8 +943,7 @@
 !! html/*
 foo'bar
 
-!!end
-
+!! end
 
 # same html as previous, but wikitext adjusted to match parsoid html2wt
 !! test
@@ -955,7 +955,6 @@
 
 !! end
 
-
 !! test
 Italics and bold: multiple quote sequences: (2,4,3)
 !! options
@@ -965,8 +964,7 @@
 !! html/*
 foo'bar
 
-!!end
-
+!! end
 
 # same html as previous, but wikitext adjusted to match parsoid html2wt
 !! test
@@ -978,7 +976,6 @@
 
 !! end
 
-
 !! test
 Italics and bold: multiple quote sequences: (2,4,4)
 !! options
@@ -988,8 +985,7 @@
 !! html/*
 foo'bar'
 
-!!end
-
+!! end
 
 # same html as previous, but wikitext adjusted to match parsoid html2wt
 !! test
@@ -1000,7 +996,6 @@
 foo'bar'
 
 !! end
-
 
 # The PHP parser strips the empty tags out for giggles; parsoid doesn't.
 !! test
@@ -1015,23 +1010,20 @@
 !! html/parsoid
 foo'bar
 
-!!end
+!! end
 
 # same html as previous, but wikitext adjusted to match parsoid html2wt
 !! test
 Italics and bold: multiple quote sequences: (3,4,2+2) w/ nowiki
-!! options
-parsoid
 !! wikitext
-'''foobar
+'''foobar
 !! html/php
 foo'bar
 
 !! html/parsoid
-foo'bar
+foo'bar
 
 !! end
-
 
 # The PHP parser strips the empty tags out for giggles; parsoid doesn't.
 !! test
@@ -1046,18 +1038,18 @@
 !! html/parsoid
 foo'bar
 
-!!end
+!! end
 
 # same html as previous, but wikitext adjusted to match parsoid html2wt
 !! test
 Italics and bold: multiple quote sequences: (3,4,3+3) w/ nowiki
 !! wikitext
-'''foobar''
+'''foobar''
 !! html/php
 foo'bar
 
 !! html/parsoid
-foo'bar
+foo'bar
 
 !! end
 
@@ -1135,7 +1127,7 @@
 The Main Page's talk 
page.
 
 !! html/parsoid
-The Main 
Page's talk page.
+The Main 
Page's talk page.
 !! end
 
 !! test
@@ -2861,9 +2853,9 @@
 !! html/php+tidy
 http://foo.com/a%7Cb;>http://foo.com/a%7Cb
 !! html/parsoid
-http://foo.com/a|b" about="#mwt1"
+http://foo.com/a%7Cb; about="#mwt1"
 typeof="mw:Transclusion"
-data-mw='{"parts":[{"template":{"target":{"wt":"echo","href":"./Template:Echo"},"params":{"1":{"wt":"http://foo.com/a#124;b"}},"i":0}}]}'>http://foo.com/a|b
+data-mw='{"parts":[{"template":{"target":{"wt":"echo","href":"./Template:Echo"},"params":{"1":{"wt":"http://foo.com/a#124;b"}},"i":0}}]}'>http://foo.com/a%7Cb
 !! end
 
 !! test
@@ -4799,6 +4791,17 @@
 !! end
 
 !! test
+External links: with entity
+!! wikitext
+[http://www.librarieswithoutborders.org Libraries without borders]
+!! html/php
+http://+www.librarieswithoutborders.org;>Libraries without borders
+
+!! html/parsoid
+http://+www.librarieswithoutborders.org; 
data-parsoid='{"a":{"href":"http://+www.librarieswithoutborders.org"},"sa":{"href":"http://#x20;www.librarieswithoutborders.org"}}'>Libraries
 without borders
+!! end
+
+!! test
 External links: Lone protocols are never linked (T105697)
 !! wikitext
 http://
@@ -5460,7 +5463,7 @@
 !! html/parsoid
 http://example.com/index.php?foozoid%5B%5D=bar;>http://example.com/index.php?foozoid%5B%5D=bar
 
-http://example.com/index.php?foozoid[]=bar;>http://example.com/index.php?foozoid[]=bar
+http://example.com/index.php?foozoid%5B%5D=bar; 
data-parsoid='{"stx":"url","a":{"href":"http://example.com/index.php?foozoid%5B%5D=bar"},"sa":{"href":"http://example.com/index.php?foozoid#x5B;#x5D;=bar"}}'>http://example.com/index.php?foozoid%5B%5D=bar
 !! end
 
 !! test
@@ -6407,7 +6410,7 @@
 !! html/parsoid
 
 
-" 
onmouseover="alert(document.cookie)">test
+[ftp://%7Cx; 
data-parsoid='{"stx":"url","a":{"href":"ftp://%7Cx"},"sa":{"href":"ftp://|x"}}'>ftp://%7Cx]" 
onmouseover="alert(document.cookie)">test
 !! end
 
 !! test
@@ -7485,7 +7488,7 @@
 |The|Main|Page|
 
 !! html/parsoid
-|The|Main|Page|
+|The|Main|Page|
 !! end
 
 !! test
@@ -7842,6 +7845,17 @@
 !! end
 
 !! test
+Link containing double quotes and spaces
+!! wikitext
+[[Cool "Gator"]]
+!! html/php
+Cool 
"Gator"
+
+!! html/parsoid
+Cool 
"Gator"
+!! end
+
+!! test
 Link containing double-single-quotes '' in text (bug 4598 sanity check)
 !! wikitext
 Some [[Link|pretty ''italics'' and stuff]]!
@@ -7849,7 +7863,7 @@
 Some pretty italics and 
stuff!
 
 !! html/parsoid
-Some pretty italics 
and stuff!
+Some pretty

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Update html/php clauses for subpage parserTests.

2017-01-24 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/333996 )

Change subject: Update html/php clauses for subpage parserTests.
..

Update html/php clauses for subpage parserTests.

The NS_USER namespace needs to have subpages enabled before these tests
work on the PHP parser.

Change-Id: I8e5e3bbd0dea6fc12f3b9ff9feeb58812fc51af1
---
M tests/parser/ParserTestRunner.php
M tests/parser/parserTests.txt
2 files changed, 21 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/96/333996/1

diff --git a/tests/parser/ParserTestRunner.php 
b/tests/parser/ParserTestRunner.php
index 281e1df..298feae 100644
--- a/tests/parser/ParserTestRunner.php
+++ b/tests/parser/ParserTestRunner.php
@@ -974,7 +974,10 @@
'wgEnableUploads' => self::getOptionValue( 
'wgEnableUploads', $opts, true ),
'wgLanguageCode' => $langCode,
'wgRawHtml' => self::getOptionValue( 'wgRawHtml', 
$opts, false ),
-   'wgNamespacesWithSubpages' => [ 0 => isset( 
$opts['subpage'] ) ],
+   'wgNamespacesWithSubpages' => [
+   0 => isset( $opts['subpage'] ),
+   2 => isset( $opts['subpage'] ),
+],
'wgMaxTocLevel' => $maxtoclevel,
'wgAllowExternalImages' => self::getOptionValue( 
'wgAllowExternalImages', $opts, true ),
'wgThumbLimits' => [ self::getOptionValue( 'thumbsize', 
$opts, 180 ) ],
diff --git a/tests/parser/parserTests.txt b/tests/parser/parserTests.txt
index 3853a12..42a1d1c 100644
--- a/tests/parser/parserTests.txt
+++ b/tests/parser/parserTests.txt
@@ -8134,19 +8134,32 @@
 Handle title parsing for subpages
 !! options
 title=[[/123123]]
+subpage
 !! wikitext
 123
+!! html/php
+123
+
 !! html/parsoid
 123
 !! end
 
-## FIXME: Add a working php section here
+!! article
+User:Test/123
+!! text
+test 123
+!! endarticle
+
 !! test
 Link to a subpage from a namespace other than main
 !! options
-title=[[User:test]]
+title=[[User:Test]]
+subpage
 !! wikitext
 [[/123]]
+!! html/php
+/123
+
 !! html/parsoid
 /123
 !! end
@@ -8168,7 +8181,8 @@
 !! test
 Purely hash wikilink
 !! options
-title=[[User:test/123]]
+title=[[User:Test/123]]
+subpage
 !! wikitext
 [[#a|b]]
 !! html/php

-- 
To view, visit https://gerrit.wikimedia.org/r/333996
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8e5e3bbd0dea6fc12f3b9ff9feeb58812fc51af1
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Protect language converter markup in the preprocessor (take 2).

2017-01-24 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/333997 )

Change subject: Protect language converter markup in the preprocessor (take 2).
..

Protect language converter markup in the preprocessor (take 2).

This revises 28774022769d2273be16c6c6e1cca710a1fd97ef, which was
reverted in master due to unexpected issues with `-{{...}} ` markup
on translatewiki and enwiki.  Test cases are added to ensure that this
is parsed as a template, not as language converter markup.

https://www.mediawiki.org/wiki/Preprocessor_ABNF is the canonical
documentation for the preprocessor; this will be updated after this
patch is merged.  The basic principles described in that page are
maintained in this patch:

* Rightmost opening structure has precedence: `-{{` is parsed as a
dash followed by template opening.

* `{{{` has precedence over `{{` and `-{`: `-` is parsed as
`-{` `{{{` since we first grab the rightmost `{{{`.

A bunch of test cases were added to verify the "ideal precedence"
order described on that wiki page.

Bug: T153761
Change-Id: I2f0c186c75e392c95e1a3d89266cae2586349150
---
M includes/parser/Preprocessor.php
M includes/parser/Preprocessor_DOM.php
M includes/parser/Preprocessor_Hash.php
M tests/parser/parserTests.txt
4 files changed, 245 insertions(+), 35 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/97/333997/1

diff --git a/includes/parser/Preprocessor.php b/includes/parser/Preprocessor.php
index 426b550..cb8e3a7 100644
--- a/includes/parser/Preprocessor.php
+++ b/includes/parser/Preprocessor.php
@@ -51,9 +51,9 @@
],
'-{' => [
'end' => '}-',
-   'names' => [ 1 => null ],
-   'min' => 1,
-   'max' => 1,
+   'names' => [ 2 => null ],
+   'min' => 2,
+   'max' => 2,
],
];
 
diff --git a/includes/parser/Preprocessor_DOM.php 
b/includes/parser/Preprocessor_DOM.php
index 661318b..3cdd38c 100644
--- a/includes/parser/Preprocessor_DOM.php
+++ b/includes/parser/Preprocessor_DOM.php
@@ -223,8 +223,7 @@
 
$searchBase = "[{<\n"; # }
if ( !$wgDisableLangConversion ) {
-   // FIXME: disabled due to T153761
-   // $searchBase .= '-';
+   $searchBase .= '-';
}
 
// For fast reverse searches
@@ -277,6 +276,13 @@
$search = $searchBase;
if ( $stack->top === false ) {
$currentClosing = '';
+   } else if (
+   $stack->top->close === '}-' &&
+   $stack->top->count > 2
+   ) {
+   # adjust closing for -{{{...{{
+   $currentClosing = '}';
+   $search .= $currentClosing;
} else {
$currentClosing = $stack->top->close;
$search .= $currentClosing;
@@ -333,11 +339,15 @@
} elseif ( isset( 
$this->rules[$curChar] ) ) {
$found = 'open';
$rule = $this->rules[$curChar];
-   } elseif ( $curChar == '-' ) {
-   $found = 'dash';
} else {
-   # Some versions of PHP have a 
strcspn which stops on null characters
-   # Ignore and continue
+   # Some versions of PHP have a 
strcspn which stops on
+   # null characters; ignore these 
and continue.
+   # We also may get '-' and '}' 
characters here which
+   # don't match -{ or 
$currentClosing.  Add these to
+   # output and continue.
+   if ( $curChar == '-' || 
$curChar == '}' ) {
+   $accum .= $curChar;
+   }
++$i;
continue;
}
@@ -615,7 +625,10 @@
} elseif ( $found == 'open' ) {
# count opening brace

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Sync parserTests with core.

2017-01-25 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/334171 )

Change subject: Sync parserTests with core.
..

Sync parserTests with core.

Change-Id: I09b794af5a8eb83b99f90f128f1c67efabec8576
---
M tests/parserTests.txt
M tools/fetch-parserTests.txt.js
2 files changed, 61 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/71/334171/1

diff --git a/tests/parserTests.txt b/tests/parserTests.txt
index 52209d2..cc202ac 100644
--- a/tests/parserTests.txt
+++ b/tests/parserTests.txt
@@ -8600,6 +8600,22 @@
 http://es.wikipedia.org/wiki/Foo_bar; />
 !! end
 
+!! test
+Escaping of interlanguage links
+!! wikitext
+Blah blah blah
+[[:es:Spanish]]
+[[ : zh : Chinese ]]
+!! html/php
+Blah blah blah
+http://es.wikipedia.org/wiki/Spanish; class="extiw" 
title="es:Spanish">es:Spanish
+http://zh.wikipedia.org/wiki/Chinese; class="extiw" 
title="zh:Chinese"> zh : Chinese 
+
+!! html/parsoid
+http://es.wikipedia.org/wiki/Spanish; 
title="es:Spanish">es:Spanish
+http://zh.wikipedia.org/wiki/Chinese; 
title="zh:Chinese"> zh : Chinese 
+!! end
+
 ## parsoid html2wt will normalize the space to _
 !! test
 Space and question mark encoding in interlanguage links (T95473)
@@ -10296,6 +10312,26 @@
 {{SITENAME}}
 !! html
 MediaWiki
+
+!! end
+
+!! test
+Magic Word: {{PAGELANGUAGE}}
+!! options
+language=fr
+!! wikitext
+{{PAGELANGUAGE}}
+!! html
+fr
+
+!! end
+
+!! test
+Magic Word: {{PAGELANGUAGE}} on a page with no explicitly set language
+!! wikitext
+{{PAGELANGUAGE}}
+!! html
+en
 
 !! end
 
@@ -16057,7 +16093,7 @@
 !! wikitext
 Foo
 !! html/php
-Foo
+Foo
 
 !! html/parsoid
 Foo
@@ -20678,6 +20714,28 @@
 # [http://example.com http://example.com] will not RT back to that
 # form anymore.
 !! test
+HTML markups with conversion syntax in attribs, nested in other conversion 
blocks
+!! options
+language=zh variant=zh-cn
+!! wikitext
+-{zh;zh-hans;zh-hant|A}-
+!! html
+A
+
+!! end
+
+!! test
+HTML markups with conversion syntax in attribs, nested in other conversion 
blocks (not working yet)
+!! options
+language=zh variant=zh-cn disabled
+!! wikitext
+-{A}-
+!! html
+A
+
+!! end
+
+!! test
 Proper conversion of text in external links
 !! options
 language=sr variant=sr-ec
diff --git a/tools/fetch-parserTests.txt.js b/tools/fetch-parserTests.txt.js
index f801139..90f971d 100755
--- a/tools/fetch-parserTests.txt.js
+++ b/tools/fetch-parserTests.txt.js
@@ -12,9 +12,9 @@
 // and update these hashes automatically.
 //
 // You can use 'sha1sum -b tests/parser/parserTests.txt' to compute this value:
-var expectedSHA1 = "d358d2a1401d9d9ba06099d1f8862282acf00b87";
+var expectedSHA1 = "e356e32b279671d5c662aa4409d64425c14adb6f";
 // git log --pretty=oneline -1 tests/parser/parserTests.txt
-var latestCommit = "6b97c822726c98f9b3cbc821e631dc18e52489d2";
+var latestCommit = "4a07505402a56a32be23a570f4bd1fe33dd542a9";
 
 var fs = require('fs');
 var path = require('path');

-- 
To view, visit https://gerrit.wikimedia.org/r/334171
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I09b794af5a8eb83b99f90f128f1c67efabec8576
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Update baseconfig; add frwiki.

2017-01-25 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/334170 )

Change subject: Update baseconfig; add frwiki.
..

Update baseconfig; add frwiki.

Add a README which explains how to regenerate the baseconfig JSON files,
and regenerate them from the current wiki configurations.

Add frwiki, since this is needed in an upcoming parserTests merge.

Change-Id: Id36550e864acf0df472b6a6a12f750eb28cbdd1d
---
A lib/config/baseconfig/README
M lib/config/baseconfig/be-taraskwiki.json
M lib/config/baseconfig/cawiki.json
M lib/config/baseconfig/cswiki.json
M lib/config/baseconfig/dewiki.json
M lib/config/baseconfig/enwiki.json
M lib/config/baseconfig/eswiki.json
M lib/config/baseconfig/fawiki.json
M lib/config/baseconfig/fiwiki.json
A lib/config/baseconfig/frwiki.json
M lib/config/baseconfig/iswiki.json
M lib/config/baseconfig/kaawiki.json
M lib/config/baseconfig/lnwiki.json
M lib/config/baseconfig/nlwiki.json
M lib/config/baseconfig/srwiki.json
M lib/config/baseconfig/trwiki.json
M lib/config/baseconfig/zhwiki.json
17 files changed, 100,541 insertions(+), 7,813 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/70/334170/1


-- 
To view, visit https://gerrit.wikimedia.org/r/334170
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Id36550e864acf0df472b6a6a12f750eb28cbdd1d
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Sync up with Parsoid parserTests.

2017-01-25 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/334166 )

Change subject: Sync up with Parsoid parserTests.
..

Sync up with Parsoid parserTests.

This now aligns with Parsoid commit 643d5392bcf4dfebf906102627c51e8a608125bf

Change-Id: I8de82d19bcb6ba6a15bbc0289dba7db625fa5c2f
---
M tests/parser/parserTests.txt
1 file changed, 176 insertions(+), 164 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/66/334166/1

diff --git a/tests/parser/parserTests.txt b/tests/parser/parserTests.txt
index aa79208..cc202ac 100644
--- a/tests/parser/parserTests.txt
+++ b/tests/parser/parserTests.txt
@@ -933,6 +933,7 @@
 ###
 ### multiple quote sequences in a line
 ###
+
 !! test
 Italics and bold: multiple quote sequences: (2,4,2)
 !! options
@@ -942,8 +943,7 @@
 !! html/*
 foo'bar
 
-!!end
-
+!! end
 
 # same html as previous, but wikitext adjusted to match parsoid html2wt
 !! test
@@ -955,7 +955,6 @@
 
 !! end
 
-
 !! test
 Italics and bold: multiple quote sequences: (2,4,3)
 !! options
@@ -965,8 +964,7 @@
 !! html/*
 foo'bar
 
-!!end
-
+!! end
 
 # same html as previous, but wikitext adjusted to match parsoid html2wt
 !! test
@@ -978,7 +976,6 @@
 
 !! end
 
-
 !! test
 Italics and bold: multiple quote sequences: (2,4,4)
 !! options
@@ -988,8 +985,7 @@
 !! html/*
 foo'bar'
 
-!!end
-
+!! end
 
 # same html as previous, but wikitext adjusted to match parsoid html2wt
 !! test
@@ -1000,7 +996,6 @@
 foo'bar'
 
 !! end
-
 
 # The PHP parser strips the empty tags out for giggles; parsoid doesn't.
 !! test
@@ -1015,23 +1010,20 @@
 !! html/parsoid
 foo'bar
 
-!!end
+!! end
 
 # same html as previous, but wikitext adjusted to match parsoid html2wt
 !! test
 Italics and bold: multiple quote sequences: (3,4,2+2) w/ nowiki
-!! options
-parsoid
 !! wikitext
-'''foobar
+'''foobar
 !! html/php
 foo'bar
 
 !! html/parsoid
-foo'bar
+foo'bar
 
 !! end
-
 
 # The PHP parser strips the empty tags out for giggles; parsoid doesn't.
 !! test
@@ -1046,18 +1038,18 @@
 !! html/parsoid
 foo'bar
 
-!!end
+!! end
 
 # same html as previous, but wikitext adjusted to match parsoid html2wt
 !! test
 Italics and bold: multiple quote sequences: (3,4,3+3) w/ nowiki
 !! wikitext
-'''foobar''
+'''foobar''
 !! html/php
 foo'bar
 
 !! html/parsoid
-foo'bar
+foo'bar
 
 !! end
 
@@ -1135,7 +1127,7 @@
 The Main Page's talk 
page.
 
 !! html/parsoid
-The Main 
Page's talk page.
+The Main 
Page's talk page.
 !! end
 
 !! test
@@ -2861,9 +2853,9 @@
 !! html/php+tidy
 http://foo.com/a%7Cb;>http://foo.com/a%7Cb
 !! html/parsoid
-http://foo.com/a|b" about="#mwt1"
+http://foo.com/a%7Cb; about="#mwt1"
 typeof="mw:Transclusion"
-data-mw='{"parts":[{"template":{"target":{"wt":"echo","href":"./Template:Echo"},"params":{"1":{"wt":"http://foo.com/a#124;b"}},"i":0}}]}'>http://foo.com/a|b
+data-mw='{"parts":[{"template":{"target":{"wt":"echo","href":"./Template:Echo"},"params":{"1":{"wt":"http://foo.com/a#124;b"}},"i":0}}]}'>http://foo.com/a%7Cb
 !! end
 
 !! test
@@ -4799,6 +4791,17 @@
 !! end
 
 !! test
+External links: with entity
+!! wikitext
+[http://www.librarieswithoutborders.org Libraries without borders]
+!! html/php
+http://+www.librarieswithoutborders.org;>Libraries without borders
+
+!! html/parsoid
+http://+www.librarieswithoutborders.org; 
data-parsoid='{"a":{"href":"http://+www.librarieswithoutborders.org"},"sa":{"href":"http://#x20;www.librarieswithoutborders.org"}}'>Libraries
 without borders
+!! end
+
+!! test
 External links: Lone protocols are never linked (T105697)
 !! wikitext
 http://
@@ -5460,7 +5463,7 @@
 !! html/parsoid
 http://example.com/index.php?foozoid%5B%5D=bar;>http://example.com/index.php?foozoid%5B%5D=bar
 
-http://example.com/index.php?foozoid[]=bar;>http://example.com/index.php?foozoid[]=bar
+http://example.com/index.php?foozoid%5B%5D=bar; 
data-parsoid='{"stx":"url","a":{"href":"http://example.com/index.php?foozoid%5B%5D=bar"},"sa":{"href":"http://example.com/index.php?foozoid#x5B;#x5D;=bar"}}'>http://example.com/index.php?foozoid%5B%5D=bar
 !! end
 
 !! test
@@ -6407,7 +6410,7 @@
 !! html/parsoid
 
 
-" 
onmouseover="alert(document.cookie)">test
+[ftp://%7Cx; 
data-parsoid='{"stx":"url","a":{"href":"ftp://%7Cx"},"sa":{"href":"ftp://|x"}}'>ftp://%7Cx]" 
onmouseover="alert(document.cookie)">test
 !! end
 
 !! test
@@ -7485,7 +7488,7 @@
 |The|Main|Page|
 
 !! html/parsoid
-|The|Main|Page|
+|The|Main|Page|
 !! end
 
 !! test
@@ -7842,6 +7845,17 @@
 !! end
 
 !! test
+Link containing double quotes and spaces
+!! wikitext
+[[Cool "Gator"]]
+!! html/php
+Cool 
"Gator"
+
+!! html/parsoid
+Cool 
"Gator"
+!! end
+
+!! test
 Link containing double-single-quotes '' in text (bug 4598 sanity check)
 !! wikitext
 Some [[Link|pretty ''italics'' and stuff]]!
@@ -7849,7 +7863,7 @@
 Some pretty italics and 
stuff!
 
 !! html/parsoid
-Some pretty italics 
and stuff!
+Some pretty

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: WIP: Implement {{PAGELANGUAGE}}

2017-01-25 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/334195 )

Change subject: WIP: Implement {{PAGELANGUAGE}}
..

WIP: Implement {{PAGELANGUAGE}}

Infrastructure needed for LanguageConverter support, since conversion
is enabled based on whether the page language has variants.

FIXME: not sure that the pagelanguage returned by the API is actually
correct; see https://phabricator.wikimedia.org/T59603#2970423

Bug: T59603
Change-Id: I26d7972a54a214a17bb92d735aa41f1b030a38d6
---
M lib/mw/ApiRequest.js
M lib/wt2html/tt/ParserFunctions.js
2 files changed, 10 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/95/334195/1

diff --git a/lib/mw/ApiRequest.js b/lib/mw/ApiRequest.js
index 2e4b440..7261a55 100644
--- a/lib/mw/ApiRequest.js
+++ b/lib/mw/ApiRequest.js
@@ -585,6 +585,8 @@
metadata.ns = page.ns;
metadata.latest = page.lastrevid;
metadata.revision = page.revisions[0];
+   metadata.pagelanguage = page.pagelanguage;
+   metadata.pagelanguagedir = page.pagelanguagedir;
 
if (metadata.revision.texthidden || 
!metadata.revision.hasOwnProperty("*")) {
error = new DoesNotExistError("Source is hidden 
for " + self.title);
diff --git a/lib/wt2html/tt/ParserFunctions.js 
b/lib/wt2html/tt/ParserFunctions.js
index 38342d9..3981d8b 100644
--- a/lib/wt2html/tt/ParserFunctions.js
+++ b/lib/wt2html/tt/ParserFunctions.js
@@ -746,6 +746,10 @@
var target = args[0].k;
cb({ tokens: [ target || this.env.page.name || '' ] });
 };
+ParserFunctions.prototype.pf_pagelanguage = function(token, frame, cb, args) {
+   // The language (code) of the current page.
+   cb({ tokens: [ this.env.page.language || 'en' ] });
+};
 // This should be doable with the information in the envirionment
 // (this.env) already.
 ParserFunctions.prototype.pf_fullurl = function(token, frame, cb, args) {
@@ -810,8 +814,10 @@
var target = args[0].k;
cb({ tokens: [target] });
 };
-ParserFunctions.prototype.pf_contentlang = function(token, frame, cb, args) {
-   cb({ tokens: ['en'] });
+ParserFunctions.prototype.pf_contentlang =
+ParserFunctions.prototype.pf_contentlanguage = function(token, frame, cb, 
args) {
+   // The wiki's default interface language ($wgLanguageCode)
+   cb({ tokens: [ this.env.conf.wiki.lang || 'en' ] });
 };
 ParserFunctions.prototype.pf_numberoffiles = function(token, frame, cb, args) {
cb({ tokens: ['2'] });

-- 
To view, visit https://gerrit.wikimedia.org/r/334195
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I26d7972a54a214a17bb92d735aa41f1b030a38d6
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Update baseconfig; add frwiki.

2017-01-25 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/334170 )

Change subject: Update baseconfig; add frwiki.
..


Update baseconfig; add frwiki.

Add a README which explains how to regenerate the baseconfig JSON files
using tools/sync-baseconfig.js.

Regenerate all baseconfigs *except for enwiki* from the current wiki
configurations.  Add frwiki, since this is needed in an upcoming
parserTests merge.

The enwiki configuration appears to have changed in a way which causes
numerous parserTests failures (T156295, T156296).  So we haven't updated
it here; we'll tackle those issues in a follow-up patch.

Change-Id: Id36550e864acf0df472b6a6a12f750eb28cbdd1d
---
A lib/config/baseconfig/README
M lib/config/baseconfig/be-taraskwiki.json
M lib/config/baseconfig/cawiki.json
M lib/config/baseconfig/cswiki.json
M lib/config/baseconfig/dewiki.json
M lib/config/baseconfig/eswiki.json
M lib/config/baseconfig/fawiki.json
M lib/config/baseconfig/fiwiki.json
A lib/config/baseconfig/frwiki.json
M lib/config/baseconfig/iswiki.json
M lib/config/baseconfig/kaawiki.json
M lib/config/baseconfig/lnwiki.json
M lib/config/baseconfig/nlwiki.json
M lib/config/baseconfig/srwiki.json
M lib/config/baseconfig/trwiki.json
M lib/config/baseconfig/zhwiki.json
A tools/sync-baseconfig.js
17 files changed, 94,635 insertions(+), 7,384 deletions(-)




-- 
To view, visit https://gerrit.wikimedia.org/r/334170
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Id36550e864acf0df472b6a6a12f750eb28cbdd1d
Gerrit-PatchSet: 4
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 
Gerrit-Reviewer: Arlolra 
Gerrit-Reviewer: C. Scott Ananian 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...TemplateData[master]: WIP: Further "custom format" spec tweaks.

2017-01-17 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/332569 )

Change subject: WIP: Further "custom format" spec tweaks.
..

WIP: Further "custom format" spec tweaks.

Change how newlines are handled in the EndFormat production.

Change-Id: Iae5181e65c7afe25810406a39f387f0c1f3cb1b3
---
M Specification.md
1 file changed, 2 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/TemplateData 
refs/changes/69/332569/1

diff --git a/Specification.md b/Specification.md
index d0133b9..32b43b1 100644
--- a/Specification.md
+++ b/Specification.md
@@ -296,9 +296,9 @@
 
 Begin with `StartFormat`, and replace the `Hole` with the name of the template 
to create the "output string".  If `StartFormat` begins with a newline and 
template is already at the start of a line (the character preceding this 
template invocation is a newline or the template is at the start of the 
output), delete the initial newline from the output string.
 
-For each parameter, append the `ParameterFormat` to the output string after 
replacing the first `Hole` with the name of the parameter and the second `Hole` 
with the value of the parameter.
+For each parameter, if `ParameterFormat` begins with a newline and the last 
character in the output string is a newline, then delete the last character in 
the output string.  Then append `ParameterFormat` to the output string after 
replacing the first `Hole` with the name of the parameter and the second `Hole` 
with the value of the parameter.
 
-Finally, append the `EndFormat` to the output string.
+Finally, if `EndFormat` begins with a newline and the last character in the 
output string is a newline, then delete the last character in the output 
string.  Append the `EndFormat` to the output string.
 
 Some example format strings:
 

-- 
To view, visit https://gerrit.wikimedia.org/r/332569
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Iae5181e65c7afe25810406a39f387f0c1f3cb1b3
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/TemplateData
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Add an option to force strong normalization of parser tests.

2017-02-28 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/340329 )

Change subject: Add an option to force strong normalization of parser tests.
..

Add an option to force strong normalization of parser tests.

This allows us to remove html2wt and html2html entries from the blacklist,
even when the test doesn't merit adding specific html/parsoid clauses.

Change-Id: I03a8dc7ffcc180a05128c4c63807cee281abb1e9
---
M bin/parserTests.js
M tests/parserTests-blacklist.js
M tests/parserTests.txt
3 files changed, 50 insertions(+), 39 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/29/340329/1

diff --git a/bin/parserTests.js b/bin/parserTests.js
index 985cd6e..9fb79db 100755
--- a/bin/parserTests.js
+++ b/bin/parserTests.js
@@ -718,7 +718,9 @@
var endsAtWikitext = mode === 'wt2wt' || mode === 'selser' || mode === 
'html2wt';
var endsAtHtml = mode === 'wt2html' || mode === 'html2html';
 
-   var parsoidOnly = ('html/parsoid' in item) || (item.options.parsoid !== 
undefined);
+   var parsoidOnly =
+   ('html/parsoid' in item) ||
+   (item.options.parsoid !== undefined && 
!item.options.parsoid.normalizePhp);
 
// Source preparation
if (startsAtHtml) {
@@ -892,7 +894,8 @@
 ParserTests.prototype.checkHTML = function(item, out, options, mode) {
var normalizedOut, normalizedExpected;
var parsoidOnly =
-   ('html/parsoid' in item) || (item.options.parsoid !== 
undefined);
+   ('html/parsoid' in item) ||
+   (item.options.parsoid !== undefined && 
!item.options.parsoid.normalizePhp);
 
normalizedOut = DU.normalizeOut(out, parsoidOnly);
out = DU.toXML(out, { innerXML: true });
diff --git a/tests/parserTests-blacklist.js b/tests/parserTests-blacklist.js
index dc5b9cb..b85a9f1 100644
--- a/tests/parserTests-blacklist.js
+++ b/tests/parserTests-blacklist.js
@@ -787,43 +787,6 @@
 add("html2wt", "Magic Word: {{CURRENTYEAR}}", "1970\n");
 add("html2wt", "Magic Word: {{CURRENTTIMESTAMP}}", "19700101000203\n");
 add("html2wt", "Magic Words LOCAL (UTC)", "* 01\n* 1\n* January\n* January\n* 
Jan\n* 1\n* 01\n* Thursday\n* 1970\n* 00:02\n* 00\n* 1\n* 4\n* 
19700101000203\n");
-add("html2wt", "Magic Word: {{FULLPAGENAME}}", "User:Ævar Arnfjörð 
Bjarmason\n");
-add("html2wt", "Magic Word: {{FULLPAGENAMEE}}", 
"User:%C3%86var_Arnfj%C3%B6r%C3%B0_Bjarmason\n");
-add("html2wt", "Magic Word: {{TALKSPACE}}", "User talk\n");
-add("html2wt", "Magic Word: {{TALKSPACE}}, same namespace", "User talk\n");
-add("html2wt", "Magic Word: {{TALKSPACE}}, main namespace", "Talk\n");
-add("html2wt", "Magic Word: {{TALKSPACEE}}", "User_talk\n");
-add("html2wt", "Magic Word: {{SUBJECTSPACE}}", "User\n");
-add("html2wt", "Magic Word: {{SUBJECTSPACE}}, same namespace", "User\n");
-add("html2wt", "Magic Word: {{SUBJECTSPACE}}, main namespace", "");
-add("html2wt", "Magic Word: {{SUBJECTSPACEE}}", "User\n");
-add("html2wt", "Magic Word: {{NAMESPACE}}", "User\n");
-add("html2wt", "Magic Word: {{NAMESPACEE}}", "User\n");
-add("html2wt", "Magic Word: {{NAMESPACENUMBER}}", "2\n");
-add("html2wt", "Magic Word: {{SUBPAGENAME}}", "sub ö\n");
-add("html2wt", "Magic Word: {{SUBPAGENAMEE}}", "sub_%C3%B6\n");
-add("html2wt", "Magic Word: {{ROOTPAGENAME}}", "Ævar Arnfjörð Bjarmason\n");
-add("html2wt", "Magic Word: {{ROOTPAGENAMEE}}", 
"%C3%86var_Arnfj%C3%B6r%C3%B0_Bjarmason\n");
-add("html2wt", "Magic Word: {{BASEPAGENAME}}", "Ævar Arnfjörð Bjarmason\n");
-add("html2wt", "Magic Word: {{BASEPAGENAMEE}}", 
"%C3%86var_Arnfj%C3%B6r%C3%B0_Bjarmason\n");
-add("html2wt", "Magic Word: {{TALKPAGENAME}}", "User talk:Ævar Arnfjörð 
Bjarmason\n");
-add("html2wt", "Magic Word: {{TALKPAGENAMEE}}", 
"User_talk:%C3%86var_Arnfj%C3%B6r%C3%B0_Bjarmason\n");
-add("html2wt", "Magic Word: {{SUBJECTPAGENAME}}", "User:Ævar Arnfjörð 
Bjarmason\n");
-add("html2wt", "Magic Word: {{SUBJECTPAGENAMEE}}", 
"User:%C3%86var_Arnfj%C3%B6r%C3%B0_Bjarmason\n");
-add("html2wt", "Magic Word: {{NUMBEROFFILES}}", "7\n");
-add("html2wt", "Magic Word: {{PAGENAME}}", "Ævar Arnfjörð Bjarmason\n");
-add("html2wt", "Magic Word: {{PAGENAME}} with metacharacters", 
"'''foo & bar = baz'''\n");
-add("html2wt", "Magic Word: {{PAGENAME}} with metacharacters (T28781)", 
"*RFC 1234 
http://example.com/\n");
-add("html2wt", "Magic Word: {{PAGENAMEE}}", 
"%C3%86var_Arnfj%C3%B6r%C3%B0_Bjarmason\n");
-add("html2wt", "Magic Word: {{PAGENAMEE}} with metacharacters (T28781)", 
"*RFC_1234_http://example.com/\n;);
-add("html2wt", "Magic Word: {{REVISIONID}}", "1337\n");
-add("html2wt", "Magic Word: {{SCRIPTPATH}}", "");
-add("html2wt", "Magic Word: {{STYLEPATH}}", "/skins\n");
-add("html2wt", "Magic Word: {{SERVER}}", "http://example.org\n;);
-add("html2wt", "Magic Word: {{SERVERNAME}}", "example.org\n");
-add("html2wt", "Magic Word: {{SITENAME}}",

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: T153341: Export list of languages which will trigger languag...

2017-02-28 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/340373 )

Change subject: T153341: Export list of languages which will trigger language 
conversion.
..

T153341: Export list of languages which will trigger language conversion.

LanguageConverter is enabled on a per-page basis, based on the page
language and whether it implements `$lang->getConverter()`.  Export
this in siteinfo so that Parsoid knows whether it should parse
language converter markup on a given page (based on the page language).

Change-Id: I010aa3812051725166ab79ea5ee3eaf56615fe94
---
M includes/api/ApiQuerySiteinfo.php
M includes/api/i18n/en.json
M includes/api/i18n/qqq.json
3 files changed, 43 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/73/340373/1

diff --git a/includes/api/ApiQuerySiteinfo.php 
b/includes/api/ApiQuerySiteinfo.php
index a8c037e..0af303e 100644
--- a/includes/api/ApiQuerySiteinfo.php
+++ b/includes/api/ApiQuerySiteinfo.php
@@ -88,6 +88,9 @@
case 'languages':
$fit = $this->appendLanguages( $p );
break;
+   case 'variants':
+   $fit = $this->appendVariants( $p );
+   break;
case 'skins':
$fit = $this->appendSkins( $p );
break;
@@ -713,6 +716,43 @@
return $this->getResult()->addValue( 'query', $property, $data 
);
}
 
+   // Export information about which page languages will trigger
+   // language conversion. (T153341)
+   public function appendVariants( $property ) {
+
+   global $wgDisableLangConversion;
+   $langNames = LanguageConverter::$languagesWithVariants;
+   sort( $langNames );
+
+   $data = [];
+   foreach ( $langNames as $langCode ) {
+   $lang = Language::factory( $langCode );
+   if (
+   $wgDisableLangConversion ||
+   $lang->getConverter() instanceof FakeConverter
+   ) {
+   // Only languages which are not instances of 
FakeConverter
+   // implement language conversion.
+   continue;
+   }
+   $data[$langCode] = [];
+   ApiResult::setIndexedTagName( $data[$langCode], 
'variant' );
+
+   $variants = $lang->getVariants();
+   sort( $variants );
+   foreach ( $variants as $v ) {
+   if ( $v === $langCode ) {
+   continue;
+   }
+   $data[$langCode][] = $v;
+   }
+   }
+   ApiResult::setIndexedTagName( $data, 'lang' );
+   ApiResult::setArrayType( $data, 'kvp', 'code' );
+
+   return $this->getResult()->addValue( 'query', $property, $data 
);
+   }
+
public function appendSkins( $property ) {
$data = [];
$allowed = Skin::getAllowedSkins();
@@ -851,6 +891,7 @@
'rightsinfo',
'restrictions',
'languages',
+   'variants',
'skins',
'extensiontags',
'functionhooks',
diff --git a/includes/api/i18n/en.json b/includes/api/i18n/en.json
index f9de725..fc2dedc 100644
--- a/includes/api/i18n/en.json
+++ b/includes/api/i18n/en.json
@@ -1139,6 +1139,7 @@
"apihelp-query+siteinfo-paramvalue-prop-rightsinfo": "Returns wiki 
rights (license) information if available.",
"apihelp-query+siteinfo-paramvalue-prop-restrictions": "Returns 
information on available restriction (protection) types.",
"apihelp-query+siteinfo-paramvalue-prop-languages": "Returns a list of 
languages MediaWiki supports (optionally localised by using 
$1inlanguagecode).",
+   "apihelp-query+siteinfo-paramvalue-prop-variants": "Returns a list of 
languages for which [[mw:LanguageConverter|LanguageConverter]] is enabled, and 
the variants supported for each.",
"apihelp-query+siteinfo-paramvalue-prop-skins": "Returns a list of all 
enabled skins (optionally localised by using $1inlanguagecode, 
otherwise in the content language).",
"apihelp-query+siteinfo-paramvalue-prop-extensiontags": "Returns a list 
of parser extension

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Clean up parserTests output.

2017-03-01 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/340664 )

Change subject: Clean up parserTests output.
..

Clean up parserTests output.

The whitelist is deprecated.  Warn when it's used, not when it's missing.

Print out the name of the test file in the results summary, and colorize it
so we know which test files resulted in errors.

Only emit the node version and initialization message once at startup.

Change-Id: I443f80102007cc5b542f48fdc091bc41a8e35f88
---
M bin/parserTests.js
M tests/parserTests.utils.js
2 files changed, 6 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/64/340664/1

diff --git a/bin/parserTests.js b/bin/parserTests.js
index 5d7030c..681f2e1 100755
--- a/bin/parserTests.js
+++ b/bin/parserTests.js
@@ -51,8 +51,8 @@
this.whiteListPath = path.resolve(parseFilePath.dir, whiteListName);
try {
this.testWhiteList = require(this.whiteListPath).testWhiteList;
+   console.warn('Using whitelist from ' + this.whiteListPath);
} catch (e) {
-   console.warn('No whitelist found at ' + this.whiteListPath);
this.testWhiteList = {};
}
 
@@ -1492,6 +1492,7 @@
var ptests = new ParserTests(testFilePath, options.modes);
return ptests.main(options, mockURL)
.then(function(s) {
+   options.reportStart = function() { }; // quiet startup 
message
return status + s;  // Add up the status codes
});
}, 0);
diff --git a/tests/parserTests.utils.js b/tests/parserTests.utils.js
index 04834b0..f98379c 100644
--- a/tests/parserTests.utils.js
+++ b/tests/parserTests.utils.js
@@ -50,9 +50,12 @@
 var reportSummary = function(modesRan, stats, file, loggedErrorCount, 
testFilter) {
var curStr, mode, thisMode;
var failTotalTests = stats.failedTests;
+   var happiness = (
+   stats.passedTestsUnexpected === 0 && 
stats.failedTestsUnexpected === 0
+   );
 

console.log("==");
-   console.log("SUMMARY: ");
+   console.log("SUMMARY:", happiness ? file.green : file.red);
if (console.time && console.timeEnd) {
console.timeEnd('Execution time');
}

-- 
To view, visit https://gerrit.wikimedia.org/r/340664
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I443f80102007cc5b542f48fdc091bc41a8e35f88
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Fix bin/parserTests.js --help

2017-03-01 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/340663 )

Change subject: Fix bin/parserTests.js --help
..

Fix bin/parserTests.js --help

Change-Id: Ib3a31953c2795ca1f759e52bca623ee7322bd790
---
M bin/parserTests.js
1 file changed, 7 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/63/340663/1

diff --git a/bin/parserTests.js b/bin/parserTests.js
index 9fb79db..5d7030c 100755
--- a/bin/parserTests.js
+++ b/bin/parserTests.js
@@ -1470,9 +1470,13 @@
 };
 
 // Start the mock api server and kick off parser tests
-serviceWrapper.runServices({ skipParsoid: true })
-.then(function(ret) {
+Promise.resolve(null).then(function() {
var options = PTUtils.prepareOptions();
+   return serviceWrapper.runServices({ skipParsoid: true })
+   .then(function(ret) {
+   return [ options, ret.mockURL ];
+   });
+}).spread(function(options, mockURL) {
var testFilePaths;
if (options._[0]) {
testFilePaths = [path.resolve(process.cwd(), options._[0])];
@@ -1486,7 +1490,7 @@
}
return Promise.reduce(testFilePaths, function(status, testFilePath) {
var ptests = new ParserTests(testFilePath, options.modes);
-   return ptests.main(options, ret.mockURL)
+   return ptests.main(options, mockURL)
.then(function(s) {
return status + s;  // Add up the status codes
});

-- 
To view, visit https://gerrit.wikimedia.org/r/340663
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib3a31953c2795ca1f759e52bca623ee7322bd790
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Collect and print summary information at end of parser tests.

2017-03-01 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/340682 )

Change subject: Collect and print summary information at end of parser tests.
..

Collect and print summary information at end of parser tests.

Return statistics from each parser tests file, so we can sum and
report a summary at the end.  The final line printed will make it
unambiguous whether unexpected results were found.

Change-Id: I95382f54df2ecff9041af84d85af87fea3c6ac25
---
M bin/parserTests.js
M tests/parserTests.utils.js
2 files changed, 70 insertions(+), 29 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/82/340682/1

diff --git a/bin/parserTests.js b/bin/parserTests.js
index a471b7f..462fbce 100755
--- a/bin/parserTests.js
+++ b/bin/parserTests.js
@@ -1267,11 +1267,20 @@
var failures = options.reportSummary(options.modes, this.stats, 
this.testFileName, this.loggedErrorCount, this.testFilter);
 
// we're done!
+   // exit status 1 == uncaught exception
+   var exitCode = failures ? 2 : 0;
if (Util.booleanOption(options['exit-zero'])) {
-   failures = false;
+   exitCode = 0;
}
 
-   return failures ? 2 : 0;  // exit status 1 == uncaught exception
+   return {
+   exitCode: exitCode,
+   stats: Object.assign({
+   failures: failures,
+   loggedErrorCount: this.loggedErrorCount,
+   }, this.stats),
+   file: this.testFileName,
+   };
}
 };
 
@@ -1381,8 +1390,7 @@
// Skip test whose title does not match --filter
// or which is disabled or php-only
this.comments = [];
-   setImmediate(nextCallback);
-   return;
+   return setImmediate(nextCallback);
}
// Add comments to following test.
item.comments = item.comments || this.comments;
@@ -1405,8 +1413,7 @@
});
}
if (!targetModes.length) {
-   setImmediate(nextCallback);
-   return;
+   return setImmediate(nextCallback);
}
// Honor language option in parserTests.txt
var prefix = item.options.language || 'enwiki';
@@ -1492,15 +1499,31 @@
return path.join(testDir, f);
});
}
-   return Promise.reduce(testFilePaths, function(status, testFilePath) {
+   var stats = {
+   passedTests: 0,
+   passedTestsWhitelisted: 0,
+   passedTestsUnexpected: 0,
+   failedTests: 0,
+   failedTestsUnexpected: 0,
+   loggedErrorCount: 0,
+   failures: 0,
+   };
+   var exitCode = 0;
+   return Promise.reduce(testFilePaths, function(stats, testFilePath) {
var ptests = new ParserTests(testFilePath, options.modes);
return ptests.main(options, mockURL)
-   .then(function(s) {
-   return status + s;  // Add up the status codes
-   });
-   }, 0);
+   .then(function(result) {
+   Object.keys(stats).forEach(function(k) {
+   stats[k] += result.stats[k]; // Sum all stats
+   });
+   exitCode = exitCode || result.exitCode;
+   }).return(stats);
+   }, stats).then(function(stats) {
+   options.reportSummary([], stats, null, stats.loggedErrorCount, 
null);
+   return exitCode;
+   });
 })
-.then(function(status) {
-   process.exit(status);
+.then(function(exitCode) {
+   process.exit(exitCode);
 })
 .done();
diff --git a/tests/parserTests.utils.js b/tests/parserTests.utils.js
index a9f7bbf..88e42b2 100644
--- a/tests/parserTests.utils.js
+++ b/tests/parserTests.utils.js
@@ -53,10 +53,12 @@
var happiness = (
stats.passedTestsUnexpected === 0 && 
stats.failedTestsUnexpected === 0
);
+   var filename = (file === null) ? "ALL TESTS" : file;
 
+   if (file === null) { console.log(); }

console.log("==");
-   console.log("SUMMARY:", happiness ? file.green : file.red);
-   if (console.time && console.timeEnd) {
+   console.log("SUMMARY:", happiness ? filename.green : filename.red);
+   if (console.time && console.timeEnd && file !== null) {
console.timeEnd('Execution time');
}
 
@@ -81,17 +83,14 @@
curStr += colorizeCount(stats.failedTestsUnexpected, 'red') + ' 
unexpected)';
console.log(curStr);
 
-

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: T159174: Strip U+0000 in wikitext

2017-02-27 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/340225 )

Change subject: T159174: Strip U+ in wikitext
..

T159174: Strip U+ in wikitext

U+ is not allowed in HTML5, there's no reason to allow it in wikitext.

It simplifies our code if we can just strip them at the start.  Strip in
PST as well so they don't sneak into our database either.

Tweaked the EXT_LINK URLs to account for the fact that invalid characters
get transformed into U+FFFD when using Preprocessor_DOM.  See
https://github.com/wikimedia/mediawiki/commit/73649741ed1e2f557aec22a485598b199fdd2d09
for context on that change.

Change-Id: I3f67e92b61aacc87a40c3662085c84d1dac08bfb
---
M includes/parser/Parser.php
M languages/LanguageConverter.php
M tests/parser/extraParserTests.txt
3 files changed, 12 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/25/340225/1

diff --git a/includes/parser/Parser.php b/includes/parser/Parser.php
index 86aa06a..5f64c92 100644
--- a/includes/parser/Parser.php
+++ b/includes/parser/Parser.php
@@ -89,13 +89,15 @@
# Everything except bracket, space, or control characters
# \p{Zs} is unicode 'separator, space' category. It covers the space 
0x20
# as well as U+3000 is IDEOGRAPHIC SPACE for T21052
-   const EXT_LINK_URL_CLASS = '[^][<>"\\x00-\\x20\\x7F\p{Zs}]';
+   # \x{FFFD} is the Unicode replacement character, which Preprocessor_DOM
+   # uses to replace invalid HTML characters.
+   const EXT_LINK_URL_CLASS = '[^][<>"\\x00-\\x20\\x7F\p{Zs}\x{FFFD}]';
# Simplified expression to match an IPv4 or IPv6 address, or
# at least one character of a host name (embeds EXT_LINK_URL_CLASS)
-   const EXT_LINK_ADDR = 
'(?:[0-9.]+|\\[(?i:[0-9a-f:.]+)\\]|[^][<>"\\x00-\\x20\\x7F\p{Zs}])';
+   const EXT_LINK_ADDR = 
'(?:[0-9.]+|\\[(?i:[0-9a-f:.]+)\\]|[^][<>"\\x00-\\x20\\x7F\p{Zs}\x{FFFD}])';
# RegExp to make image URLs (embeds IPv6 part of EXT_LINK_ADDR)
// @codingStandardsIgnoreStart Generic.Files.LineLength
-   const EXT_IMAGE_REGEX = 
'/^(http:\/\/|https:\/\/)((?:\\[(?i:[0-9a-f:.]+)\\])?[^][<>"\\x00-\\x20\\x7F\p{Zs}]+)
+   const EXT_IMAGE_REGEX = 
'/^(http:\/\/|https:\/\/)((?:\\[(?i:[0-9a-f:.]+)\\])?[^][<>"\\x00-\\x20\\x7F\p{Zs}\x{FFFD}]+)

\\/([A-Za-z0-9_.,~%\\-+&;#*?!=()@\\x80-\\xFF]+)\\.((?i)gif|png|jpg|jpeg)$/Sxu';
// @codingStandardsIgnoreEnd
 
@@ -264,7 +266,7 @@
$this->mUrlProtocols = wfUrlProtocols();
$this->mExtLinkBracketedRegex = '/\[(((?i)' . 
$this->mUrlProtocols . ')' .
self::EXT_LINK_ADDR .
-   self::EXT_LINK_URL_CLASS . 
'*)\p{Zs}*([^\]\\x00-\\x08\\x0a-\\x1F]*?)\]/Su';
+   self::EXT_LINK_URL_CLASS . 
'*)\p{Zs}*([^\]\\x00-\\x08\\x0a-\\x1F\\x{FFFD}]*?)\]/Su';
if ( isset( $conf['preprocessorClass'] ) ) {
$this->mPreprocessorClass = $conf['preprocessorClass'];
} elseif ( defined( 'HPHP_VERSION' ) ) {
@@ -417,6 +419,8 @@
$text = strtr( $text, "\x7f", "?" );
$magicScopeVariable = $this->lock();
}
+   // Strip U+ NULL (T159174)
+   $text = str_replace( "\000", '', $text );
 
$this->startParse( $title, $options, self::OT_HTML, $clearState 
);
 
@@ -4462,6 +4466,9 @@
$this->startParse( $title, $options, self::OT_WIKI, $clearState 
);
$this->setUser( $user );
 
+   // Strip U+ NULL (T159174)
+   $text = str_replace( "\000", '', $text );
+
// We still normalize line endings for backwards-compatibility
// with other code that just calls PST, but this should already
// be handled in TextContent subclasses
diff --git a/languages/LanguageConverter.php b/languages/LanguageConverter.php
index 06fec44..8607d59 100644
--- a/languages/LanguageConverter.php
+++ b/languages/LanguageConverter.php
@@ -380,6 +380,7 @@
$literalBlob = '';
 
// Guard against delimiter nulls in the input
+   // (should never happen: see T159174)
$text = str_replace( "\000", '', $text );
 
$markupMatches = null;
diff --git a/tests/parser/extraParserTests.txt 
b/tests/parser/extraParserTests.txt
index a48087e..8d042d7 100644
--- a/tests/parser/extraParserTests.txt
+++ b/tests/parser/extraParserTests.txt
Binary files differ

-- 
To view, visit https://gerrit.wikimedia.org/r/340225
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I3f67e92b61aacc87a40c3662085c84d1dac08bfb
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian

[MediaWiki-commits] [Gerrit] mediawiki...ocg-collection[master]: Smash the global promise

2016-09-02 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has submitted this change and it was merged.

Change subject: Smash the global promise
..


Smash the global promise

 * Also, allow me to override what node is named.

Change-Id: If8904b24530f2bf066cc84f9caa1577dd60f5a25
---
M Makefile
M unify-package-json.js
2 files changed, 4 insertions(+), 2 deletions(-)

Approvals:
  C. Scott Ananian: Verified; Looks good to me, approved



diff --git a/Makefile b/Makefile
index 5532455..b591b28 100644
--- a/Makefile
+++ b/Makefile
@@ -1,10 +1,12 @@
+NODE?=/usr/bin/env nodejs
+
 all: debug
 
 core:
export LINK=g++
rm -f ./package.json
npm install prfun semver
-   ./unify-package-json.js
+   $(NODE) ./unify-package-json.js
npm install
npm update
npm dedupe
diff --git a/unify-package-json.js b/unify-package-json.js
index 74254f2..9dbfaf0 100755
--- a/unify-package-json.js
+++ b/unify-package-json.js
@@ -18,7 +18,7 @@
util = require( 'util' );
 
 try {
-   require('prfun');
+   require('prfun/smash');
 } catch ( err ) {
console.err( "The 'prfun' library could not be loaded. Please `npm 
install prfun`")
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/308118
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: If8904b24530f2bf066cc84f9caa1577dd60f5a25
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ocg-collection
Gerrit-Branch: master
Gerrit-Owner: Arlolra 
Gerrit-Reviewer: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Allow specifying a specific revision for roundtrip-test.js.

2016-09-02 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/308293

Change subject: Allow specifying a specific revision for roundtrip-test.js.
..

Allow specifying a specific revision for roundtrip-test.js.

Change-Id: I25768ce0eca778e7c2756da5440bdae824851571
---
M bin/roundtrip-test.js
1 file changed, 12 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/93/308293/1

diff --git a/bin/roundtrip-test.js b/bin/roundtrip-test.js
index 4dbb068..57c0eb2 100755
--- a/bin/roundtrip-test.js
+++ b/bin/roundtrip-test.js
@@ -598,13 +598,18 @@
uri: uri + domain + '/v3/',
title: encodeURIComponent(title),
contentVersion: options.contentVersion || defaultContentVersion,
+   oldid: options.oldid,
};
 
var data = {};
return Promise[err ? 'reject' : 'resolve'](err).then(function() {
+   var uri = parsoidOptions.uri + 'page/wikitext/' + 
parsoidOptions.title;
+   if (parsoidOptions.oldid) {
+   uri += '/' + parsoidOptions.oldid;
+   }
return Util.retryingHTTPRequest(10, {
method: 'GET',
-   uri: parsoidOptions.uri + 'page/wikitext/' + 
parsoidOptions.title,
+   uri: uri,
});
}).spread(function(res, body) {
profile.start = Date.now();
@@ -706,6 +711,12 @@
boolean: false,
default: '',  // Add a default when `prefix` is removed.
},
+   oldid: {
+   description: 'Optional oldid of the given page. If not 
given,'+
+   ' will use the latest revision.',
+   boolean: false,
+   default: null,
+   },
parsoidURL: {
description: 'The URL for the Parsoid API',
boolean: false,

-- 
To view, visit https://gerrit.wikimedia.org/r/308293
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I25768ce0eca778e7c2756da5440bdae824851571
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...Collection[master]: Pass request source and user information to backend.

2016-10-04 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/314178

Change subject: Pass request source and user information to backend.
..

Pass request source and user information to backend.

This allows the backend to do rate-limiting, blacklisting, etc, based
on the user agent, wiki username, ip address, or article/collection title.

Bug: T97030
Bug: T147211
Change-Id: I5faae8e6caa77287ac4cfd6b249fd591837e6bd2
---
M Collection.body.php
M RenderingAPI.php
2 files changed, 56 insertions(+), 12 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/Collection 
refs/changes/78/314178/1

diff --git a/Collection.body.php b/Collection.body.php
index c41b0f2..d5fd051 100644
--- a/Collection.body.php
+++ b/Collection.body.php
@@ -498,6 +498,7 @@
$collection = CollectionSession::getCollection();
$collection['title'] = $title;
$collection['subtitle'] = $subtitle;
+   unset( $collection['source'] );
CollectionSession::setCollection( $collection );
}
 
@@ -510,6 +511,7 @@
$collection['settings'] = array();
}
$collection['settings'] = $settings + $collection['settings'];
+   unset( $collection['source'] );
CollectionSession::setCollection( $collection );
}
 
@@ -542,6 +544,7 @@
}
usort( $articles, array( __CLASS__, 'title_cmp' ) );
$collection['items'] = array_merge( $new_items, $articles );
+   unset( $collection['source'] );
CollectionSession::setCollection( $collection );
}
 
@@ -557,6 +560,7 @@
'type' => 'chapter',
'title' => $name,
) );
+   unset( $collection['source'] );
CollectionSession::setCollection( $collection );
}
 
@@ -573,6 +577,7 @@
return;
}
$collection['items'][$index]['title'] = $name;
+   unset( $collection['source'] );
CollectionSession::setCollection( $collection );
}
 
@@ -633,6 +638,7 @@
);
 
$collection['items'][] = $item;
+   unset( $collection['source'] );
CollectionSession::setCollection( $collection );
return true;
}
@@ -662,6 +668,7 @@
if ( $index != - 1 ) {
array_splice( $collection['items'], $index, 1 );
}
+   unset( $collection['source'] );
CollectionSession::setCollection( $collection );
return true;
}
@@ -738,6 +745,7 @@
}
$collection = CollectionSession::getCollection();
array_splice( $collection['items'], $index, 1 );
+   unset( $collection['source'] );
CollectionSession::setCollection( $collection );
return true;
}
@@ -755,6 +763,7 @@
$saved = $collection['items'][$index + $delta];
$collection['items'][$index + $delta] = 
$collection['items'][$index];
$collection['items'][$index] = $saved;
+   unset( $collection['source'] );
CollectionSession::setCollection( $collection );
return true;
}
@@ -773,6 +782,7 @@
$new_items[$new_index] = isset( $old_items[$old_index] 
) ? $old_items[$old_index] : null;
}
$collection['items'] = $new_items;
+   unset( $collection['source'] );
CollectionSession::setCollection( $collection );
}
 
@@ -905,6 +915,8 @@
}
}
$collection['items'] = $items;
+   // Mark the source of this multi-page collection.
+   $collection['source'] = $title->getCanonicalURL();
return $collection;
}
 
@@ -919,6 +931,7 @@
return false;
}
$collection = CollectionSession::getCollection();
+   $collection['source'] = $title->getCanonicalURL();
$articleText = "{{" . $this->msg( 'coll-savedbook_template' 
)->inContentLanguage()->text();
if ( !empty( $collection['settings'] ) ) {
$articleText .= "\n";
@@ -979,6 +992,8 @@
);
$api = new ApiMain( $req, true );
$api->execute();
+   // Store new 'source' for this collection.
+   CollectionSession::setCollection( $collection );
return true;
}
 
@@ -997,7 +1012,8 @@
}
 
$api = CollectionRenderingAPI::instance( $writer );
-   $response =

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Parser tests: don't use user's tidy configuration for initia...

2016-09-21 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/312048

Change subject: Parser tests: don't use user's tidy configuration for initial 
setup.
..

Parser tests: don't use user's tidy configuration for initial setup.

We were previously using the user's inherited tidy configuration when
parsing the comments associated with image and article "uploads" prior
to running parser tests.  This causes inconsistent results when
running with different tidy configurations (or none at all) and
confuses a developer who ought to be using the `--use-tidy-config` flag.

Change-Id: I89db16f715d1280be1d3b3c94373d21adea31ad9
---
M tests/parser/ParserTestRunner.php
1 file changed, 15 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/48/312048/1

diff --git a/tests/parser/ParserTestRunner.php 
b/tests/parser/ParserTestRunner.php
index 4ef778d..2d148d1 100644
--- a/tests/parser/ParserTestRunner.php
+++ b/tests/parser/ParserTestRunner.php
@@ -1189,6 +1189,13 @@
// Create a user
$user = User::createNew( 'WikiSysop' );
 
+   // For consistency when running tests with different tidy
+   // configurations, explicitly disable tidy for uploads:
+   // recordUpload2 will eventually call $wikiPage->doEditContent 
in
+   // filerepo/file/LocalFile.php, which parses the comment text
+   // applying the current tidy configuration.
+   MWTidy::setInstance( false );
+
// Register the uploads in the database
 
$image = wfLocalFile( Title::makeTitle( NS_FILE, 'Foobar.jpg' ) 
);
@@ -1313,6 +1320,8 @@
'sha1' => Wikimedia\base_convert( '', 16, 36, 31 ),
'fileExists' => true
], $this->db->timestamp( '20010115123600' ), $user );
+
+   MWTidy::destroySingleton();
 
return $this->createTeardownObject( $teardown, $nextTeardown );
}
@@ -1517,7 +1526,13 @@
throw new MWException( "duplicate article '$name' at 
$file:$line\n" );
}
 
+   // For consistency when running tests with different tidy
+   // configurations, explicitly disable tidy here; otherwise
+   // the inherited local tidy configuration will be used when
+   // parsing the comment text for this revision.
+   MWTidy::setInstance( false );
$status = $page->doEditContent( ContentHandler::makeContent( 
$text, $title ), '', EDIT_NEW );
+   MWTidy::destroySingleton();
if ( !$status->isOK() ) {
throw new MWException( $status->getWikiText( false, 
false, 'en' ) );
}

-- 
To view, visit https://gerrit.wikimedia.org/r/312048
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I89db16f715d1280be1d3b3c94373d21adea31ad9
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Other language converter bugs (test case tweaks)

2016-09-21 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/312066

Change subject: Other language converter bugs (test case tweaks)
..

Other language converter bugs (test case tweaks)

Change-Id: I018ac758cd88f3b6139b5c83e35781b6bfaf2807
---
M tests/parser/parserTests.txt
1 file changed, 3 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/66/312066/1

diff --git a/tests/parser/parserTests.txt b/tests/parser/parserTests.txt
index fcad339..053608b 100644
--- a/tests/parser/parserTests.txt
+++ b/tests/parser/parserTests.txt
@@ -20262,19 +20262,18 @@
 
 !! end
 
-# FIXME: This test is currently broken in the PHP parser (bug 52661)
 !! test
 Don't break image parsing if language converter markup is in the caption.
 !! options
 language=sr
 !! wikitext
 [[File:Foobar.jpg|-{R|caption}-]]
+!! html/php
 !! html/parsoid
 http://example.com/images/3/3a/Foobar.jpg; width="1941" 
height="220" />
 
 !! end
 
-# FIXME: This test is currently broken in the PHP parser (bug 52661)
 !! test
 Don't break list handling if language converter markup is in the item.
 !! options
@@ -20283,13 +20282,12 @@
 ;-{zh-cn:AAA;zh-tw:BBB}-
 !! html/php
 在手动语言转换规则中检测到错误
-
+xxx
 !! html/parsoid
 AAA
 
 !! end
 
-# FIXME: This test is currently broken in the PHP parser (bug 52661)
 !! test
 Don't break table handling if language converter markup is in the cell.
 !! options
@@ -20300,6 +20298,7 @@
 | -{R|B}-
 |}
 !! html/php
+
 
 
 

-- 
To view, visit https://gerrit.wikimedia.org/r/312066
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I018ac758cd88f3b6139b5c83e35781b6bfaf2807
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: WIP: protect language converter markup in the preprocessor.

2016-09-20 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/311849

Change subject: WIP: protect language converter markup in the preprocessor.
..

WIP: protect language converter markup in the preprocessor.

TODO: disable preprocessor protection if $wgDisableLangConversion ?

Bug: T54661
Bug: T54190
Bug: T54192
Change-Id: I709d007c70a3fd19264790055042c615999b2f67
---
M includes/parser/Preprocessor.php
M includes/parser/Preprocessor_DOM.php
M includes/parser/Preprocessor_Hash.php
3 files changed, 57 insertions(+), 9 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/49/311849/1

diff --git a/includes/parser/Preprocessor.php b/includes/parser/Preprocessor.php
index cc98abd..cb8e3a7 100644
--- a/includes/parser/Preprocessor.php
+++ b/includes/parser/Preprocessor.php
@@ -48,7 +48,13 @@
'names' => [ 2 => null ],
'min' => 2,
'max' => 2,
-   ]
+   ],
+   '-{' => [
+   'end' => '}-',
+   'names' => [ 2 => null ],
+   'min' => 2,
+   'max' => 2,
+   ],
];
 
/**
diff --git a/includes/parser/Preprocessor_DOM.php 
b/includes/parser/Preprocessor_DOM.php
index 5da7cd7..ea67b19 100644
--- a/includes/parser/Preprocessor_DOM.php
+++ b/includes/parser/Preprocessor_DOM.php
@@ -219,7 +219,7 @@
 
$stack = new PPDStack;
 
-   $searchBase = "[{<\n"; # }
+   $searchBase = "[{<\n-"; # }
// For fast reverse searches
$revText = strrev( $text );
$lengthText = strlen( $text );
@@ -298,7 +298,10 @@
break;
}
} else {
-   $curChar = $text[$i];
+   $curChar = $curTwoChar = $text[$i];
+   if ( ( $i + 1 ) < $lengthText ) {
+   $curTwoChar .= $text[$i + 1];
+   }
if ( $curChar == '|' ) {
$found = 'pipe';
} elseif ( $curChar == '=' ) {
@@ -311,11 +314,20 @@
} else {
$found = 'line-start';
}
+   } elseif ( $curTwoChar == 
$currentClosing ) {
+   $found = 'close';
+   $curChar = $curTwoChar;
} elseif ( $curChar == $currentClosing 
) {
$found = 'close';
+   } elseif ( isset( 
$this->rules[$curTwoChar] ) ) {
+   $curChar = $curTwoChar;
+   $found = 'open';
+   $rule = $this->rules[$curChar];
} elseif ( isset( 
$this->rules[$curChar] ) ) {
$found = 'open';
$rule = $this->rules[$curChar];
+   } else if ( $curChar == '-' ) {
+   $found = 'dash';
} else {
# Some versions of PHP have a 
strcspn which stops on null characters
# Ignore and continue
@@ -595,7 +607,10 @@
// input pointer.
} elseif ( $found == 'open' ) {
# count opening brace characters
-   $count = strspn( $text, $curChar, $i );
+   $count = strlen( $curChar );
+   if ( $count === 1 ) {
+   $count = strspn( $text, $curChar, $i );
+   }
 
# we need to add to stack only if opening brace 
count is enough for one of the rules
if ( $count >= $rule['min'] ) {
@@ -620,7 +635,10 @@
$piece = $stack->top;
# lets check if there are enough characters for 
closing brace
$maxCount = $piece->count;
-   $count = strspn( $text, $curChar, $i, $maxCount 
);
+

[MediaWiki-commits] [Gerrit] mediawiki...OfflineContentGenerator[master]: Allow administrators to blacklist renders of certain metabooks.

2016-10-03 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/313876

Change subject: Allow administrators to blacklist renders of certain metabooks.
..

Allow administrators to blacklist renders of certain metabooks.

This is a ham-handed but hopefully effective way to deal with denial of
service/abuse.

Bug: T147211
Change-Id: I4c1c3e155e399dc80d501aa9d8ee286f23a92db4
---
M lib/threads/frontend.js
1 file changed, 17 insertions(+), 8 deletions(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/Collection/OfflineContentGenerator
 refs/changes/76/313876/1

diff --git a/lib/threads/frontend.js b/lib/threads/frontend.js
index c67ba94..617d37f 100644
--- a/lib/threads/frontend.js
+++ b/lib/threads/frontend.js
@@ -569,6 +569,11 @@
 * @returns {bool|Promise}
 */
var canIRender = Promise.method(function() {
+   if (config.frontend.blacklist instanceof RegExp && metabookObj) 
{
+   if 
(config.frontend.blacklist.exec(JSON.stringify(metabookObj))) {
+   return [false, 'Administrative blacklist.'];
+   }
+   }
if (config.redis.max_job_queue_length) {
return redisClient
.llen(config.redis.job_queue_name)
@@ -579,10 +584,13 @@
job: { id: collectionId },
});
statsd.gauge('job_queue_length', len);
-   return len <= 
config.redis.max_job_queue_length;
+   return [
+   len <= 
config.redis.max_job_queue_length,
+   'Job queue is full'
+   ];
});
} else {
-   return true;
+   return [true, 'Success.'];
}
});
 
@@ -625,16 +633,17 @@
return determineCachedStatus().then(function(cacheStatus) {
isCached = cacheStatus;
if (!isCached) {
-   return canIRender().then(function(canRender)  {
-   if (canRender) {
-   return addRenderJob();
-   } else {
-   console.warn('Refusing 
new job because job queue is full', {
+   return canIRender().spread(function(canRender, 
whyNot)  {
+   whyNot = whyNot || "Unknown reason.";
+   if (canRender) {
+   return addRenderJob();
+   } else {
+   console.warn('Refusing new job 
because: '+whyNot, {
channel: 
'frontend',
request: { id: 
requestId },
job: { id: 
collectionId },
});
-   throw new 
FrontendError(503, 'Job queue is full.');
+   throw new 
FrontendError(503, whyNot);
}
});
} else {

-- 
To view, visit https://gerrit.wikimedia.org/r/313876
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I4c1c3e155e399dc80d501aa9d8ee286f23a92db4
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/Collection/OfflineContentGenerator
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...Collection[master]: Remove unused 'OfflineContentGenerator' submodule.

2016-10-03 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/313882

Change subject: Remove unused 'OfflineContentGenerator' submodule.
..

Remove unused 'OfflineContentGenerator' submodule.

Change-Id: Id272952e42fddb22911e687548ea63f4ae1410e1
---
M .gitmodules
D OfflineContentGenerator
2 files changed, 0 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/Collection 
refs/changes/82/313882/1

diff --git a/.gitmodules b/.gitmodules
index a3f4a54..e69de29 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -1,3 +0,0 @@
-[submodule "OfflineContentGenerator"]
-   path = OfflineContentGenerator
-   url = 
https://gerrit.wikimedia.org/r/mediawiki/extensions/Collection/OfflineContentGenerator
diff --git a/OfflineContentGenerator b/OfflineContentGenerator
deleted file mode 16
index a7a79cc..000
--- a/OfflineContentGenerator
+++ /dev/null
@@ -1 +0,0 @@
-Subproject commit a7a79ccc3f7cf774cf42f91991381b2799df0ff6

-- 
To view, visit https://gerrit.wikimedia.org/r/313882
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Id272952e42fddb22911e687548ea63f4ae1410e1
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/Collection
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ocg-collection[master]: Synchronize package.json unification code with wmf-deploy br...

2016-10-03 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has submitted this change and it was merged.

Change subject: Synchronize package.json unification code with wmf-deploy 
branch.
..


Synchronize package.json unification code with wmf-deploy branch.

This is a cherry-pick of 4533acd571ec9a5114a1a53233552c16a298d4fe.

Change-Id: I7baf0f84ae8b8b0627d68107033a860623684858
---
M Makefile
M unify-package-json.js
2 files changed, 29 insertions(+), 8 deletions(-)

Approvals:
  C. Scott Ananian: Verified; Looks good to me, approved



diff --git a/Makefile b/Makefile
index b591b28..3ff2eab 100644
--- a/Makefile
+++ b/Makefile
@@ -5,7 +5,7 @@
 core:
export LINK=g++
rm -f ./package.json
-   npm install prfun semver
+   npm install core-js@0.9.x prfun@2.1.x semver@4.1.x
$(NODE) ./unify-package-json.js
npm install
npm update
diff --git a/unify-package-json.js b/unify-package-json.js
index 9dbfaf0..1b99a77 100755
--- a/unify-package-json.js
+++ b/unify-package-json.js
@@ -11,17 +11,26 @@
  * library (since we want to use fancy non-standard Promise features).
  */
 
+var haveAllPackages = true;
+['core-js','prfun','semver'].forEach(function(pkg) {
+   try {
+   require(pkg);
+   } catch (err) {
+   console.err( "The '"+pkg+"' library could not be loaded.  
Please `npm install "+pkg+"`" );
+   haveAllPackages = false;
+   }
+});
+if (!haveAllPackages) {
+   process.exit(1);
+}
+
+require('core-js'); // for Map/Set
 var child_process = require( 'child_process' ),
semver = require( 'semver' ),
fs = require( 'fs' ),
path = require( 'path' ),
-   util = require( 'util' );
-
-try {
-   require('prfun/smash');
-} catch ( err ) {
-   console.err( "The 'prfun' library could not be loaded. Please `npm 
install prfun`")
-}
+   util = require( 'util' ),
+   Promise = require( 'prfun' );
 
 function findPackageJson() {
var readdir = Promise.promisify(fs.readdir, false, fs);
@@ -94,6 +103,18 @@
optdeps = iterate( 'optionalDependencies', packageObjs );
console.info( 'Unifying unified development dependencies' );
glodevdeps = iterate( 'devDependencies', packageObjs );
+   // Hack around the 'mw-ocg-bundler' dev dependency; don't make it global
+   delete glodevdeps['mw-ocg-bundler'];
+   // Remove optional and dev dependencies if they are already in 
dependencies
+   // (In particular we need to remove the `request` package from
+   // devDependencies so that `npm prune --production` doesn't delete it!)
+   [ optdeps, glodevdeps ].forEach( function( d ) {
+   Object.keys(d).forEach( function( pkg ) {
+   if ( d[pkg] === glodeps[pkg] ) {
+   delete d[pkg];
+   }
+   } );
+   } );
 
return {
'dependencies': glodeps,

-- 
To view, visit https://gerrit.wikimedia.org/r/313912
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I7baf0f84ae8b8b0627d68107033a860623684858
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ocg-collection
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 
Gerrit-Reviewer: Arlolra 
Gerrit-Reviewer: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ocg-collection[master]: Updating to latest masters

2016-10-03 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has submitted this change and it was merged.

Change subject: Updating to latest masters
..


Updating to latest masters

Deployed new service (d0f8e126f5d1ad53358c7c7380ca02f41c33ead6):
* T147211: Allow administrators to blacklist renders of certain metabooks.

Deployed new latexer (45c30840d22a7590cee904aff5f2e272c75d3bdb):
* T144120: Add `\FloatBarrier`s after articles.

Change-Id: I3bd1339214d5d752d934ed80a92d0647bd1f2efa
---
M mw-ocg-latexer
M mw-ocg-service
2 files changed, 2 insertions(+), 2 deletions(-)

Approvals:
  C. Scott Ananian: Verified; Looks good to me, approved



diff --git a/mw-ocg-latexer b/mw-ocg-latexer
index 544a26a..45c3084 16
--- a/mw-ocg-latexer
+++ b/mw-ocg-latexer
@@ -1 +1 @@
-Subproject commit 544a26ae19eb0e0c0ae9bea98e24630b8ca7982b
+Subproject commit 45c30840d22a7590cee904aff5f2e272c75d3bdb
diff --git a/mw-ocg-service b/mw-ocg-service
index 51d2eb6..d0f8e12 16
--- a/mw-ocg-service
+++ b/mw-ocg-service
@@ -1 +1 @@
-Subproject commit 51d2eb6d47dfc5225d8d5bc45444e81cc233b1c7
+Subproject commit d0f8e126f5d1ad53358c7c7380ca02f41c33ead6

-- 
To view, visit https://gerrit.wikimedia.org/r/313900
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I3bd1339214d5d752d934ed80a92d0647bd1f2efa
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ocg-collection
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 
Gerrit-Reviewer: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ocg-collection[wmf-deploy]: Add en.wiktionary.org to blacklist temporarily to prevent DoS.

2016-10-03 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has submitted this change and it was merged.

Change subject: Add en.wiktionary.org to blacklist temporarily to prevent DoS.
..


Add en.wiktionary.org to blacklist temporarily to prevent DoS.

Bug: T147211
Change-Id: I9e14710601bb8a3311e10971a8975fb641ee1a1b
---
M LocalSettings.js
1 file changed, 3 insertions(+), 0 deletions(-)

Approvals:
  C. Scott Ananian: Verified; Looks good to me, approved



diff --git a/LocalSettings.js b/LocalSettings.js
index 86e998d..60afb3e 100644
--- a/LocalSettings.js
+++ b/LocalSettings.js
@@ -10,6 +10,9 @@
config.backend.writers.rdf2latex.bin = 
"/srv/deployment/ocg/ocg/mw-ocg-latexer/bin/mw-ocg-latexer";
config.backend.writers.rdf2text.bin = 
"/srv/deployment/ocg/ocg/mw-ocg-texter/bin/mw-ocg-texter";
 
+   // DoS prevention: stop spider of wiktionary (2016-10-03, T147211)
+   config.frontend.blacklist = /"baseurl":"[^"]*en\.wiktionary\.org/;
+
// DoS prevention: limit bundle sizes to 512MB

config.backend.bundler.additionalArgs.push('--image-size-limit=5');

config.backend.bundler.additionalArgs.push('--bundle-size-limit=536870912');

-- 
To view, visit https://gerrit.wikimedia.org/r/313915
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I9e14710601bb8a3311e10971a8975fb641ee1a1b
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ocg-collection
Gerrit-Branch: wmf-deploy
Gerrit-Owner: C. Scott Ananian 
Gerrit-Reviewer: Arlolra 
Gerrit-Reviewer: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ocg-collection[master]: Updating to latest masters

2016-10-03 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/313900

Change subject: Updating to latest masters
..

Updating to latest masters

Deployed new service (d0f8e126f5d1ad53358c7c7380ca02f41c33ead6):
* T147211: Allow administrators to blacklist renders of certain metabooks.

Deployed new latexer (45c30840d22a7590cee904aff5f2e272c75d3bdb):
* T144120: Add `\FloatBarrier`s after articles.

Change-Id: I3bd1339214d5d752d934ed80a92d0647bd1f2efa
---
M mw-ocg-latexer
M mw-ocg-service
2 files changed, 2 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ocg-collection 
refs/changes/00/313900/1

diff --git a/mw-ocg-latexer b/mw-ocg-latexer
index 544a26a..45c3084 16
--- a/mw-ocg-latexer
+++ b/mw-ocg-latexer
@@ -1 +1 @@
-Subproject commit 544a26ae19eb0e0c0ae9bea98e24630b8ca7982b
+Subproject commit 45c30840d22a7590cee904aff5f2e272c75d3bdb
diff --git a/mw-ocg-service b/mw-ocg-service
index 51d2eb6..d0f8e12 16
--- a/mw-ocg-service
+++ b/mw-ocg-service
@@ -1 +1 @@
-Subproject commit 51d2eb6d47dfc5225d8d5bc45444e81cc233b1c7
+Subproject commit d0f8e126f5d1ad53358c7c7380ca02f41c33ead6

-- 
To view, visit https://gerrit.wikimedia.org/r/313900
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I3bd1339214d5d752d934ed80a92d0647bd1f2efa
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ocg-collection
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ocg-collection[master]: Synchronize package.json unification code with wmf-deploy br...

2016-10-03 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/313912

Change subject: Synchronize package.json unification code with wmf-deploy 
branch.
..

Synchronize package.json unification code with wmf-deploy branch.

This is a cherry-pick of 4533acd571ec9a5114a1a53233552c16a298d4fe.

Change-Id: I7baf0f84ae8b8b0627d68107033a860623684858
---
M Makefile
M unify-package-json.js
2 files changed, 29 insertions(+), 8 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ocg-collection 
refs/changes/12/313912/1

diff --git a/Makefile b/Makefile
index b591b28..3ff2eab 100644
--- a/Makefile
+++ b/Makefile
@@ -5,7 +5,7 @@
 core:
export LINK=g++
rm -f ./package.json
-   npm install prfun semver
+   npm install core-js@0.9.x prfun@2.1.x semver@4.1.x
$(NODE) ./unify-package-json.js
npm install
npm update
diff --git a/unify-package-json.js b/unify-package-json.js
index 9dbfaf0..1b99a77 100755
--- a/unify-package-json.js
+++ b/unify-package-json.js
@@ -11,17 +11,26 @@
  * library (since we want to use fancy non-standard Promise features).
  */
 
+var haveAllPackages = true;
+['core-js','prfun','semver'].forEach(function(pkg) {
+   try {
+   require(pkg);
+   } catch (err) {
+   console.err( "The '"+pkg+"' library could not be loaded.  
Please `npm install "+pkg+"`" );
+   haveAllPackages = false;
+   }
+});
+if (!haveAllPackages) {
+   process.exit(1);
+}
+
+require('core-js'); // for Map/Set
 var child_process = require( 'child_process' ),
semver = require( 'semver' ),
fs = require( 'fs' ),
path = require( 'path' ),
-   util = require( 'util' );
-
-try {
-   require('prfun/smash');
-} catch ( err ) {
-   console.err( "The 'prfun' library could not be loaded. Please `npm 
install prfun`")
-}
+   util = require( 'util' ),
+   Promise = require( 'prfun' );
 
 function findPackageJson() {
var readdir = Promise.promisify(fs.readdir, false, fs);
@@ -94,6 +103,18 @@
optdeps = iterate( 'optionalDependencies', packageObjs );
console.info( 'Unifying unified development dependencies' );
glodevdeps = iterate( 'devDependencies', packageObjs );
+   // Hack around the 'mw-ocg-bundler' dev dependency; don't make it global
+   delete glodevdeps['mw-ocg-bundler'];
+   // Remove optional and dev dependencies if they are already in 
dependencies
+   // (In particular we need to remove the `request` package from
+   // devDependencies so that `npm prune --production` doesn't delete it!)
+   [ optdeps, glodevdeps ].forEach( function( d ) {
+   Object.keys(d).forEach( function( pkg ) {
+   if ( d[pkg] === glodeps[pkg] ) {
+   delete d[pkg];
+   }
+   } );
+   } );
 
return {
'dependencies': glodeps,

-- 
To view, visit https://gerrit.wikimedia.org/r/313912
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I7baf0f84ae8b8b0627d68107033a860623684858
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ocg-collection
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ocg-collection[wmf-deploy]: Merge branch 'master' into wmf-deploy

2016-10-03 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/313913

Change subject: Merge branch 'master' into wmf-deploy
..

Merge branch 'master' into wmf-deploy

Change-Id: Ibc86a4ca0bb8e01cdaaf39ce9078214c7de99c85
---
M Makefile
1 file changed, 1 insertion(+), 9 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ocg-collection 
refs/changes/13/313913/1

diff --git a/Makefile b/Makefile
index 113bd4d..a56d6ee 100644
--- a/Makefile
+++ b/Makefile
@@ -1,20 +1,12 @@
-<<< HEAD   (e3e0fd Merge branch 'master' into wmf-deploy)
-all: production
-===
 NODE?=/usr/bin/env nodejs
 
-all: debug
->>> BRANCH (02e565 Synchronize package.json unification code with 
wmf-deploy br)
+all: production
 
 core:
export LINK=g++
rm -f ./package.json
npm install core-js@0.9.x prfun@2.1.x semver@4.1.x
-<<< HEAD   (e3e0fd Merge branch 'master' into wmf-deploy)
-   ./unify-package-json.js
-===
$(NODE) ./unify-package-json.js
->>> BRANCH (02e565 Synchronize package.json unification code with 
wmf-deploy br)
npm install
npm update
npm dedupe

-- 
To view, visit https://gerrit.wikimedia.org/r/313913
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibc86a4ca0bb8e01cdaaf39ce9078214c7de99c85
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ocg-collection
Gerrit-Branch: wmf-deploy
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ocg-collection[wmf-deploy]: Merge branch 'master' into wmf-deploy

2016-10-03 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has submitted this change and it was merged.

Change subject: Merge branch 'master' into wmf-deploy
..


Merge branch 'master' into wmf-deploy

Change-Id: Ibc86a4ca0bb8e01cdaaf39ce9078214c7de99c85
---
M Makefile
1 file changed, 1 insertion(+), 9 deletions(-)

Approvals:
  C. Scott Ananian: Verified; Looks good to me, approved



diff --git a/Makefile b/Makefile
index 113bd4d..a56d6ee 100644
--- a/Makefile
+++ b/Makefile
@@ -1,20 +1,12 @@
-<<< HEAD   (e3e0fd Merge branch 'master' into wmf-deploy)
-all: production
-===
 NODE?=/usr/bin/env nodejs
 
-all: debug
->>> BRANCH (02e565 Synchronize package.json unification code with 
wmf-deploy br)
+all: production
 
 core:
export LINK=g++
rm -f ./package.json
npm install core-js@0.9.x prfun@2.1.x semver@4.1.x
-<<< HEAD   (e3e0fd Merge branch 'master' into wmf-deploy)
-   ./unify-package-json.js
-===
$(NODE) ./unify-package-json.js
->>> BRANCH (02e565 Synchronize package.json unification code with 
wmf-deploy br)
npm install
npm update
npm dedupe

-- 
To view, visit https://gerrit.wikimedia.org/r/313913
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ibc86a4ca0bb8e01cdaaf39ce9078214c7de99c85
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ocg-collection
Gerrit-Branch: wmf-deploy
Gerrit-Owner: C. Scott Ananian 
Gerrit-Reviewer: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ocg-collection[wmf-deploy]: Add en.wiktionary.org to blacklist temporarily to prevent DoS.

2016-10-03 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/313915

Change subject: Add en.wiktionary.org to blacklist temporarily to prevent DoS.
..

Add en.wiktionary.org to blacklist temporarily to prevent DoS.

Bug: T147211
Change-Id: I9e14710601bb8a3311e10971a8975fb641ee1a1b
---
M LocalSettings.js
1 file changed, 3 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ocg-collection 
refs/changes/15/313915/1

diff --git a/LocalSettings.js b/LocalSettings.js
index 86e998d..60afb3e 100644
--- a/LocalSettings.js
+++ b/LocalSettings.js
@@ -10,6 +10,9 @@
config.backend.writers.rdf2latex.bin = 
"/srv/deployment/ocg/ocg/mw-ocg-latexer/bin/mw-ocg-latexer";
config.backend.writers.rdf2text.bin = 
"/srv/deployment/ocg/ocg/mw-ocg-texter/bin/mw-ocg-texter";
 
+   // DoS prevention: stop spider of wiktionary (2016-10-03, T147211)
+   config.frontend.blacklist = /"baseurl":"[^"]*en\.wiktionary\.org/;
+
// DoS prevention: limit bundle sizes to 512MB

config.backend.bundler.additionalArgs.push('--image-size-limit=5');

config.backend.bundler.additionalArgs.push('--bundle-size-limit=536870912');

-- 
To view, visit https://gerrit.wikimedia.org/r/313915
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I9e14710601bb8a3311e10971a8975fb641ee1a1b
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ocg-collection
Gerrit-Branch: wmf-deploy
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: POC: Translate extension experiment using Annotation service.

2016-11-11 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/321005

Change subject: POC: Translate extension experiment using Annotation service.
..

POC: Translate extension experiment using Annotation service.

Change-Id: I86f1fbe5bceef93b2654ded636719550daf72010
---
M lib/api/ParsoidService.js
M lib/api/apiUtils.js
M lib/api/routes.js
A lib/translate/HRequest.js
A lib/translate/Translator.js
5 files changed, 203 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/05/321005/1

diff --git a/lib/api/ParsoidService.js b/lib/api/ParsoidService.js
index 7aa1bb5..7d6d765 100644
--- a/lib/api/ParsoidService.js
+++ b/lib/api/ParsoidService.js
@@ -183,6 +183,7 @@
// v3 API routes
app.get('/:domain/v3/page/:format/:title/:revision?', v3, u, p, a, 
routes.v3Get);
app.post('/:domain/v3/transform/:from/to/:format/:title?/:revision?', 
v3, u, p, a, routes.v3Post);
+   app.get('/:domain/v3/translate/:lang/:format/:title/:revision?', v3, u, 
p, a, routes.v3Translate);
 
var server;
return new Promise(function(resolve, reject) {
diff --git a/lib/api/apiUtils.js b/lib/api/apiUtils.js
index e3d77e9..ad3f947 100644
--- a/lib/api/apiUtils.js
+++ b/lib/api/apiUtils.js
@@ -402,7 +402,7 @@
  * @param {Request} req
  * @param {Response} res
  */
-apiUtils.redirectToOldid = function(req, res) {
+apiUtils.redirectToOldid = function(req, res, isTranslate) {
var opts = res.locals.opts;
var env = res.locals.env;
var metrics = env.conf.parsoid.metrics;
@@ -418,7 +418,12 @@
format,
encodeURIComponent(target),
revid,
-   ].join('/');
+   ];
+   // HACK
+   if (isTranslate) {
+   path.splice(3, 1, 'translate', opts.lang);
+   }
+   path = path.join('/');
if (Object.keys(req.query).length > 0) {
path += '?' + qs.stringify(req.query);
}
diff --git a/lib/api/routes.js b/lib/api/routes.js
index 9a56651..c1e2ff3 100644
--- a/lib/api/routes.js
+++ b/lib/api/routes.js
@@ -14,6 +14,7 @@
 var Promise = require('../utils/promise.js');
 var LogData = require('../logger/LogData.js').LogData;
 var ApiRequest = require('../mw/ApiRequest.js');
+var Translator = require('../translate/Translator.js');
 
 var TemplateRequest = ApiRequest.TemplateRequest;
 
@@ -744,6 +745,47 @@
}
return p.catch(apiUtils.timeoutResp.bind(null, env));
};
+   // GET requests
+   routes.v3Translate = function(req, res) {
+   var opts = res.locals.opts;
+   var env = res.locals.env;
+   var lang = opts.lang = req.params.lang;
+   var oldid = res.locals.oldid;
+   var target = env.normalizeAndResolvePageTitle();
+   // XXX explicit oldid is still going through this path!
+   if (!oldid) {
+   return TemplateRequest.setPageSrcInfo(env, 
target).then(function(){
+   return apiUtils.redirectToOldid(req, res, 
'translate');
+   });
+   }
+   return Promise.resolve().then(function() {
+   if (req.headers.cookie) {
+   // Don't cache requests with a session.
+   apiUtils.setHeader(res, env, 'Cache-Control', 
'private,no-cache,s-maxage=0');
+   }
+   return TemplateRequest.setPageSrcInfo(env, target, 
oldid);
+   }).then(function() {
+   apiUtils.setHeader(res, env, 'content-revision-id', 
oldid);
+   return env.getContentHandler().toHTML(env);
+   }).then(function(doc) {
+   return new Translator(env, lang).translate(doc);
+   }).then(function(doc) {
+   var html, pb;
+   if (env.pageBundle) {
+   var out = 
DU.extractDpAndSerialize(res.locals.bodyOnly ? doc.body : doc, {
+   innerXML: res.locals.bodyOnly,
+   });
+   html = out.str;
+   pb = out.pb;
+   } else {
+   html = DU.toXML(res.locals.bodyOnly ? doc.body 
: doc, {
+   innerXML: res.locals.bodyOnly,
+   });
+   }
+   apiUtils.wt2htmlRes(env, res, html, pb);
+   }).catch(apiUtils.timeoutResp.bind(null, env));
+   };
+
 
// POST requests
routes.v3Post = function(req, res) {
diff --git a/lib/translate/HRequest.js b/lib/translate/HRequest.js
new file mode 100644
index 000..eec57d2
--- /dev/null

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: fixup! WIP: parse language converter markup.

2016-10-16 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/316235

Change subject: fixup! WIP: parse language converter markup.
..

fixup! WIP: parse language converter markup.

Change-Id: I0344d3b14c827defe0c14bbe91598a8cfadedee9
---
M lib/html2wt/LanguageVariantHandler.js
M lib/html2wt/escapeWikitext.js
M lib/wt2html/pegTokenizer.pegjs.txt
3 files changed, 6 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/35/316235/1

diff --git a/lib/html2wt/LanguageVariantHandler.js 
b/lib/html2wt/LanguageVariantHandler.js
index 2f3d3f4..cedbb3d 100644
--- a/lib/html2wt/LanguageVariantHandler.js
+++ b/lib/html2wt/LanguageVariantHandler.js
@@ -69,7 +69,7 @@
};
// Helper function: combine the three parts of the -{ }- string
var combine = function(flagStr, bodyStr, useTrailingSemi) {
-   if (flagStr) { flagStr = flagStr + '|'; }
+   if (flagStr || /|/.test(bodyStr)) { flagStr = flagStr + '|'; }
if (useTrailingSemi) { bodyStr = bodyStr + ';'; }
return flagStr + bodyStr;
};
diff --git a/lib/html2wt/escapeWikitext.js b/lib/html2wt/escapeWikitext.js
index 56cd406..0b2980a 100644
--- a/lib/html2wt/escapeWikitext.js
+++ b/lib/html2wt/escapeWikitext.js
@@ -653,6 +653,7 @@
var hasMagicWord = /(^|\W)(RFC|ISBN|PMID)\s/.test(text);
var hasAutolink = state.env.conf.wiki.findValidProtocol(text);
var fullCheckNeeded = !state.inLink && (hasMagicWord || hasAutolink);
+   var hasLanguageConverter = false;
var hasQuoteChar = false;
var indentPreUnsafe = false;
var hasNonQuoteEscapableChars = false;
@@ -662,6 +663,8 @@
hasQuoteChar = /'/.test(text);
indentPreUnsafe = (!indentPreSafeMode && (/\n 
+[^\r\n]*?[^\s]+/).test(text) || sol && (/^ +[^\r\n]*?[^\s]+/).test(text));
hasNonQuoteEscapableChars = 
/[<>\[\]\-\+\|!=#\*:;~{}]|__[^_]*__/.test(text);
+   hasLanguageConverter = /-{|}-/.test(text);
+   if (hasLanguageConverter) { fullCheckNeeded = true; }
}
 
// Quick check for the common case (useful to kill a majority of 
requests)
diff --git a/lib/wt2html/pegTokenizer.pegjs.txt 
b/lib/wt2html/pegTokenizer.pegjs.txt
index f311e85..67286ae 100644
--- a/lib/wt2html/pegTokenizer.pegjs.txt
+++ b/lib/wt2html/pegTokenizer.pegjs.txt
@@ -852,11 +852,12 @@
 rest:(space_or_newline* ";" space_or_newline* ff:lang_variant_flag
   { return ff; })*
 { return [ f ].concat(rest); }
+  / "" { return []; }
 
 lang_variant_flag
   = f:[-+A-Z]   { return { flag: f }; }
   / v:lang_variant_name { return { variant: v }; }
-  / b:(!space_or_newline [^|;])+ { return { bogus: b.join('') }; /* bad flag 
*/}
+  / b:(!space_or_newline [^{}|;])+ { return { bogus: b.join('') }; /* bad flag 
*/}
 
 lang_variant_name // language variant name, like zh, zh-cn, etc.
   = h:[a-z] t:[-a-z]+ { return h + t.join(''); }

-- 
To view, visit https://gerrit.wikimedia.org/r/316235
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0344d3b14c827defe0c14bbe91598a8cfadedee9
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: WIP: allstar wikitext serialization.

2016-10-16 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/316237

Change subject: WIP: allstar wikitext serialization.
..

WIP: allstar wikitext serialization.

Change-Id: Ibbf98aa4420650707813ec67addcac1bf827a7b5
---
A bin/allstar.js
M package.json
2 files changed, 488 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/37/316237/1

diff --git a/bin/allstar.js b/bin/allstar.js
new file mode 100755
index 000..1329bbc
--- /dev/null
+++ b/bin/allstar.js
@@ -0,0 +1,487 @@
+#!/usr/bin/env node
+/**
+ * Allstar serialization.
+ * Read from STDIN, write to STDOUT.
+ */
+'use strict';
+require('../core-upgrade.js');
+
+var ParserEnv = 
require('../lib/config/MWParserEnvironment.js').MWParserEnvironment;
+var ParsoidConfig = require('../lib/config/ParsoidConfig.js').ParsoidConfig;
+var Parse = require('./parse.js');
+var TemplateRequest = require('../lib/mw/ApiRequest.js').TemplateRequest;
+var Util = require('../lib/utils/Util.js').Util;
+var DU = require('../lib/utils/DOMUtils.js').DOMUtils;
+var DOMTraverser = require('../lib/utils/DOMTraverser.js').DOMTraverser;
+var Promise = require('../lib/utils/promise.js');
+var fs = require('fs');
+var path = require('path');
+var yargs = require('yargs');
+var yaml = require('js-yaml');
+
+var standardOpts = Util.addStandardOptions({
+   'config': {
+   description: "Path to a config.yaml file.  Use --config w/ no 
argument to default to the server's config.yaml",
+   'default': false,
+   },
+   'prefix': {
+   description: 'Which wiki prefix to use; e.g. "enwiki" for 
English wikipedia, "eswiki" for Spanish, "mediawikiwiki" for mediawiki.org',
+   'boolean': false,
+   'default': null,
+   },
+   'domain': {
+   description: 'Which wiki to use; e.g. "en.wikipedia.org" for 
English wikipedia, "es.wikipedia.org" for Spanish, "mediawiki.org" for 
mediawiki.org',
+   'boolean': false,
+   'default': null,
+   },
+   'page': {
+   description: 'The page name, returned for {{PAGENAME}}. If no 
input is given (ie. empty/stdin closed), it downloads and parses the page. This 
should be the actual title of the article (that is, not including any 
URL-encoding that might be necessary in wikitext).',
+   'boolean': false,
+   'default': ParserEnv.prototype.defaultPageName,
+   },
+   'inputfile': {
+   description: 'File containing input as an alternative to stdin',
+   'boolean': false,
+   'default': false,
+   },
+   'contentVersion': {
+   description: 'The acceptable content version.',
+   'boolean': false,
+   'default': ParserEnv.prototype.contentVersion,
+   },
+   'loadWMF': {
+   description: 'Use WMF mediawiki API config',
+   'boolean': true,
+   'default': true,
+   },
+   'offline': {
+   description: 'Shortcut to turn off various network fetches 
during parse.',
+   'boolean': true,
+   'default': false,
+   },
+});
+
+var serialize = function(env, doc) {
+   var result = [];
+   var dt = new DOMTraverser(env);
+   var atSOL = function() {
+   if (result.length === 0) { return true; }
+   var i = result.length - 1;
+   while (i>0 && result[i].length === 0) { i--; }
+   return /\n$/.test(result[i]);
+   };
+   var atListSOL = function() {
+   if (result.length === 0) { return true; }
+   var i = result.length - 1;
+   while (i>0 && /^[*#:;]*$/.test(result[i])) { i--; }
+   if (i==0) { return /(^|\n)[*#:;]*$/.test(result[i]); }
+   return /\n[*#:;]*$/.test(result[i]);
+   };
+   var forceSOL = function() {
+   if (!atSOL()) { result.push('\n'); }
+   };
+   var typeofContains = function(node, type) {
+   if (node.nodeType !== 1) { return false; }
+   var ty = (node.getAttribute('typeof') || '').split(/\s+/g);
+   if (typeof type === 'string') {
+   type = new RegExp('^' + Util.escapeRegExp(type) + '$');
+   }
+   for (var i=0; i

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Typo fix in comment.

2016-10-16 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/316238

Change subject: Typo fix in comment.
..

Typo fix in comment.

Change-Id: Ib1d93af2f7318ec887530c63e25d4a00ab118a68
---
M lib/config/MWParserEnvironment.js
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/38/316238/1

diff --git a/lib/config/MWParserEnvironment.js 
b/lib/config/MWParserEnvironment.js
index e7de92d..828b628 100644
--- a/lib/config/MWParserEnvironment.js
+++ b/lib/config/MWParserEnvironment.js
@@ -484,7 +484,7 @@
  * shouldn't be treated like links at all.
  *
  * This function handles strings that are page-fragments or subpage references
- * and resolves hose w.r.t the current page name so that title-handling code 
elsewhere
+ * and resolves those w.r.t the current page name so that title-handling code 
elsewhere
  * only deal with non-relative title strings.
  */
 MWParserEnvironment.prototype.resolveTitle = function(urlDecodedStr, 
resolveOnly) {

-- 
To view, visit https://gerrit.wikimedia.org/r/316238
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib1d93af2f7318ec887530c63e25d4a00ab118a68
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: WIP: fix more language converter bugs.

2016-10-16 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/316236

Change subject: WIP: fix more language converter bugs.
..

WIP: fix more language converter bugs.

Change-Id: Id6688d72175470148fb3295771f10cf2e802caa5
---
M lib/config/WikiConfig.js
M lib/html2wt/LanguageVariantHandler.js
M lib/mw/ApiRequest.js
M lib/wt2html/tt/LanguageVariantHandler.js
M tests/parserTests.txt
5 files changed, 150 insertions(+), 51 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/36/316236/1

diff --git a/lib/config/WikiConfig.js b/lib/config/WikiConfig.js
index 37a4f2a..30cac7e 100644
--- a/lib/config/WikiConfig.js
+++ b/lib/config/WikiConfig.js
@@ -514,6 +514,17 @@
// Function hooks on this wiki, indexed by their normalized form
this.functionHooks = new Set(resultConf.functionhooks || []);
 
+   // Somewhat annoyingly, LanguageConversion is turned on by default
+   // for all wikis (ie, $wgDisableLangConversion = false, as reported
+   // by siteinfo general.langconversion), but the
+   // -{ }- syntax is only parsed when the currently selected language
+   // has variants.  For the most part, the "language" is
+   // siteinfo general.lang and the variants are in siteinfo as lists in
+   // general.fallback and general.variants. *However* various mechanisms
+   // (like the Translate extension) can change the default language
+   // for a given page, and then we need to use other mechanisms to see
+   // if variants are active.
+
// Match a wikitext line containing just whitespace, comments, and
// sol transparent links and behavior switches.
// Redirects should not contain any preceding non-whitespace chars.
diff --git a/lib/html2wt/LanguageVariantHandler.js 
b/lib/html2wt/LanguageVariantHandler.js
index cedbb3d..1f9cc0a 100644
--- a/lib/html2wt/LanguageVariantHandler.js
+++ b/lib/html2wt/LanguageVariantHandler.js
@@ -61,15 +61,19 @@
};
 
// Helper function: protect characters not allowed in -{ }- strings.
+   // XXX escape := in left side of unidir rules, and in language names (!)
var protect = function(t) {
-   // XXX escape - characters that would form -{ }- markup.
-   // XXX escape confusing | : ; characters?
-   // XXX ensure that this wikitext won't break the -{}- context
-   return t;
+   return true ? t : t.replace(/[\{\}|:]/g, function(c) {
+   // Escape {} characters that would form -{ }- markup
+   // Escape | that would form a lang_variant_flag
+   // Escape : that would form a lang_variant_option
+   return '

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Balancer: remove unnecessary extra argument

2016-10-12 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/315609

Change subject: Balancer: remove unnecessary extra argument
..

Balancer: remove unnecessary extra argument

The full HTML5 spec clones element attributes when they are added to
the ActiveFormattingElements list, so that when an element on that
list is later cloned and reinserted the attributes are the *original*
attributes, not reflecting any changes which embedded JavaScript
in an inline

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Protect -{...}- variant constructs in definition lists.

2016-12-13 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/327112 )

Change subject: Protect -{...}- variant constructs in definition lists.
..

Protect -{...}- variant constructs in definition lists.

Given the wikitext:

;-{zh-cn:AAA;zh-tw:BBB}-

Prevent `doBlockLevels` from trying to split the definition list at the
embedded colon and using `AAA;zh-tw:BBB}-` as the `` portion.

Bug: T153135
Change-Id: I3a4d02f1fbd0d0fe8278d6b7c66005f0dd3dd36b
---
M includes/parser/BlockLevelPass.php
M tests/parser/parserTests.txt
2 files changed, 40 insertions(+), 31 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/12/327112/1

diff --git a/includes/parser/BlockLevelPass.php 
b/includes/parser/BlockLevelPass.php
index cbacd34..1bb3c49 100644
--- a/includes/parser/BlockLevelPass.php
+++ b/includes/parser/BlockLevelPass.php
@@ -38,6 +38,7 @@
const COLON_STATE_COMMENT = 5;
const COLON_STATE_COMMENTDASH = 6;
const COLON_STATE_COMMENTDASHDASH = 7;
+   const COLON_STATE_LC = 8;
 
/**
 * Make lists from lines starting with ':', '*', '#', etc.
@@ -389,15 +390,14 @@
 * @return string The position of the ':', or false if none found
 */
private function findColonNoLinks( $str, &$before, &$after ) {
-   $colonPos = strpos( $str, ':' );
-   if ( $colonPos === false ) {
+   if ( !preg_match( '/:|<|-\{/', $str, $m, PREG_OFFSET_CAPTURE ) 
) {
# Nothing to find!
return false;
}
 
-   $ltPos = strpos( $str, '<' );
-   if ( $ltPos === false || $ltPos > $colonPos ) {
+   if ( $m[0][0] === ':' ) {
# Easy; no tag nesting to worry about
+   $colonPos = $m[0][1];
$before = substr( $str, 0, $colonPos );
$after = substr( $str, $colonPos + 1 );
return $colonPos;
@@ -405,9 +405,10 @@
 
# Ugly state machine to walk through avoiding tags.
$state = self::COLON_STATE_TEXT;
-   $level = 0;
+   $ltLevel = 0;
+   $lcLevel = 0;
$len = strlen( $str );
-   for ( $i = 0; $i < $len; $i++ ) {
+   for ( $i = $m[0][1]; $i < $len; $i++ ) {
$c = $str[$i];
 
switch ( $state ) {
@@ -418,7 +419,7 @@
$state = self::COLON_STATE_TAGSTART;
break;
case ":":
-   if ( $level === 0 ) {
+   if ( $ltLevel === 0 ) {
# We found it!
$before = substr( $str, 0, $i );
$after = substr( $str, $i + 1 );
@@ -428,35 +429,44 @@
break;
default:
# Skip ahead looking for something 
interesting
-   $colonPos = strpos( $str, ':', $i );
-   if ( $colonPos === false ) {
+   if ( !preg_match( '/:|<|-\{/', $str, 
$m, PREG_OFFSET_CAPTURE, $i ) ) {
# Nothing else interesting
return false;
}
-   $ltPos = strpos( $str, '<', $i );
-   if ( $level === 0 ) {
-   if ( $ltPos === false || 
$colonPos < $ltPos ) {
-   # We found it!
-   $before = substr( $str, 
0, $colonPos );
-   $after = substr( $str, 
$colonPos + 1 );
-   return $i;
-   }
+   if ( $m[0][0] === '-{' ) {
+   $state = self::COLON_STATE_LC;
+   $lcLevel++;
+   $i = $m[0][1] + 1;
+   } else {
+   # Skip ahead to next 
interesting character.
+   $i = $m[0][1] - 1;
}
-   if ( $ltPos === false ) {
-

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Don't parse language converter markup as a cell parameter in...

2016-12-13 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/327127 )

Change subject: Don't parse language converter markup as a cell parameter in 
tables.
..

Don't parse language converter markup as a cell parameter in tables.

Bug: T153140
Change-Id: I799363727162a0f337652b26bb69fe35c61a8553
---
M includes/parser/Parser.php
M tests/parser/parserTests.txt
2 files changed, 4 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/27/327127/1

diff --git a/includes/parser/Parser.php b/includes/parser/Parser.php
index 435c8fc..13b3744 100644
--- a/includes/parser/Parser.php
+++ b/includes/parser/Parser.php
@@ -1183,7 +1183,8 @@
 
# Bug 553: Note that a '|' inside an 
invalid link should not
# be mistaken as delimiting cell 
parameters
-   if ( strpos( $cell_data[0], '[[' ) !== 
false ) {
+   # Bug T153140: Neither should language 
converter markup.
+   if ( preg_match( '/\[\[|-\{/', 
$cell_data[0] ) === 1 ) {
$cell = 
"{$previous}<{$last_tag}>{$cell}";
} elseif ( count( $cell_data ) == 1 ) {
$cell = 
"{$previous}<{$last_tag}>{$cell_data[0]}";
diff --git a/tests/parser/parserTests.txt b/tests/parser/parserTests.txt
index 2a41822..ec50d86 100644
--- a/tests/parser/parserTests.txt
+++ b/tests/parser/parserTests.txt
@@ -20373,9 +20373,8 @@
 
 !! end
 
-# FIXME: This test is currently broken in the PHP parser (bug 52661)
 !! test
-Don't break table handling if language converter markup is in the cell.
+T153140: Don't break table handling if language converter markup is in the 
cell.
 !! options
 language=sr variant=sr-ec
 !! wikitext
@@ -20387,7 +20386,7 @@
 
 
 
-Б}-
+ B
 
 
 !! html/parsoid

-- 
To view, visit https://gerrit.wikimedia.org/r/327127
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I799363727162a0f337652b26bb69fe35c61a8553
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Protect -{...}- variant constructs in images.

2016-12-13 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/327042 )

Change subject: Protect -{...}- variant constructs in images.
..

Protect -{...}- variant constructs in images.

Bug: T146305
Change-Id: I1327685e9e8c07ef476dceaa6f6dae4ba40989ef
---
M includes/parser/Parser.php
M tests/parser/parserTests.txt
2 files changed, 12 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/42/327042/1

diff --git a/includes/parser/Parser.php b/includes/parser/Parser.php
index 13dae8f..20f079a 100644
--- a/includes/parser/Parser.php
+++ b/includes/parser/Parser.php
@@ -5151,7 +5151,13 @@
#  * bottom
#  * text-bottom
 
-   $parts = StringUtils::explode( "|", $options );
+   $placeholder = Parser::MARKER_PREFIX . wfRandomString( 16 );
+   $separator = '|';
+   $options = str_replace( $placeholder, '', $options );
+   $replacer = new DoubleReplacer( $separator, $placeholder );
+   // Protect LanguageConverter markup
+   $cleaned = StringUtils::delimiterReplaceCallback( '-{', '}-', 
$replacer->cb(), $options );
+   $parts = StringUtils::explode( $separator, $cleaned );
 
# Give extensions a chance to select the file revision for us
$options = [];
@@ -5176,7 +5182,8 @@
'horizAlign' => [], 'vertAlign' => [] ];
$seenformat = false;
foreach ( $parts as $part ) {
-   $part = trim( $part );
+   // Put replaced separators back in.
+   $part = trim( str_replace( $placeholder, $separator, 
$part ) );
list( $magicName, $value ) = 
$mwArray->matchVariableStartToEnd( $part );
$validated = false;
if ( isset( $paramMap[$magicName] ) ) {
diff --git a/tests/parser/parserTests.txt b/tests/parser/parserTests.txt
index c7d84d2..9fd268d 100644
--- a/tests/parser/parserTests.txt
+++ b/tests/parser/parserTests.txt
@@ -20305,13 +20305,15 @@
 
 !! end
 
-# FIXME: This test is currently broken in the PHP parser (bug 52661)
 !! test
 Don't break image parsing if language converter markup is in the caption.
 !! options
 language=sr
 !! wikitext
 [[File:Foobar.jpg|-{R|caption}-]]
+!! html/php
+http://example.com/images/3/3a/Foobar.jpg; width="1941" height="220" />
+
 !! html/parsoid
 http://example.com/images/3/3a/Foobar.jpg; width="1941" 
height="220" />
 

-- 
To view, visit https://gerrit.wikimedia.org/r/327042
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I1327685e9e8c07ef476dceaa6f6dae4ba40989ef
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Sync parserTests with core.

2016-12-14 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/327286 )

Change subject: Sync parserTests with core.
..

Sync parserTests with core.

Change-Id: I4bc976d3a861052b201ad985aac0a647999d2cc4
---
M tests/parserTests-blacklist.js
M tests/parserTests.txt
M tools/fetch-parserTests.txt.js
3 files changed, 57 insertions(+), 17 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/86/327286/1

diff --git a/tests/parserTests-blacklist.js b/tests/parserTests-blacklist.js
index 4571014..6b4288b 100644
--- a/tests/parserTests-blacklist.js
+++ b/tests/parserTests-blacklist.js
@@ -645,7 +645,6 @@
 add("html2wt", "Templates: Indent-Pre: 1f: Wrapping should be based on 
expanded content", " a\n\n\n\n a\n\n\n\n b\n\na\n\n b\n\na\n\n b\n");
 add("html2wt", "Templates: Strip leading and trailing whitespace from 
named-param values", "a\n\nb\n\nc\n\n* d\n");
 add("html2wt", "Templates: Don't strip whitespace from positional-param 
values", "a \n\nb \n\n c \n\n\n\n d\n\n\n\n e\n\n\n\n* f\n\n\n g\n");
-add("html2wt", "Templates: Handle empty comment-and-ws-only lines correctly", 
"foo\nbar\n");
 add("html2wt", "Templates: Handle comments in the target", "{{echo |foo}}\n\n{{echo |foo}}\n\n{{echo|foo}}\n\n{{echo|foo}}\n\n{{echo|foo}}");
 add("html2wt", "Templates: Handle comments in parameter names (bug 67657)", 
"{{echo|1 =foo}}\n\n{{echo| 
1=foo}}\n\n{{echo|1=foo}}\n\n{{echo|1=foo}}");
 add("html2wt", "1a. Indent-Pre and Comments", " a\n\nc\n");
diff --git a/tests/parserTests.txt b/tests/parserTests.txt
index 47401bc..07d50a8 100644
--- a/tests/parserTests.txt
+++ b/tests/parserTests.txt
@@ -1,5 +1,5 @@
 # MediaWiki Parser test cases
-# Some taken from http://meta.wikimedia.org/wiki/Parser_testing
+# Some taken from https://meta.wikimedia.org/wiki/Parser_testing
 # All (C) their respective authors and released under the GPL
 #
 # The syntax should be fairly self-explanatory.
@@ -539,7 +539,7 @@
 !! end
 
 # Note that the html+tidy output removes the spaces after the ,
-# which is a bug (http://sourceforge.net/p/tidy/bugs/945/, etc).
+# which is a bug (https://sourceforge.net/p/tidy/bugs/945/, etc).
 # This is an issue for all tests with lists.  We intentionally do
 # *not* add html+tidy clauses for these, as we don't want to
 # document/test the broken behavior.  (Parsoid matches the non-tidy
@@ -1245,7 +1245,7 @@
 !! end
 
 # test cases taken from
-# http://www.w3.org/TR/html5/text-level-semantics.html#the-ruby-element
+# https://www.w3.org/TR/html5/text-level-semantics.html#the-ruby-element
 !! test
 Ruby markup (W3C-style)
 !! wikitext
@@ -1308,7 +1308,7 @@
 
 !! end
 
-# There is a tidy bug here: http://sourceforge.net/p/tidy/bugs/946/
+# There is a tidy bug here: https://sourceforge.net/p/tidy/bugs/946/
 # If the non-word-character tag made it through the sanitizer, tidy
 # would munge it up.
 !! test
@@ -1433,6 +1433,15 @@
 
 :and a colon
 
+!! end
+
+!! test
+Don't parse  (T149622)
+!! wikitext
+
+!! html/php
+span class="error"
+
 !! end
 
 !! test
@@ -2718,10 +2727,12 @@
 
  
 bar}}
-!! html
+!! html/php
 foo
 bar
 
+!! html/parsoid
+foo 
  bar
 !! end
 
 !! test
@@ -2741,7 +2752,14 @@
 {{echo|foo}}
 
 {{echo|foo}}
-!!html/parsoid
+!! html/php
+foo
+foo
+foo
+foo
+foo
+
+!! html/parsoid
 foo
 
 foo
@@ -2767,7 +2785,13 @@
 {{echo|1=foo}}
 
 {{echo|1=foo}}
-!!html/parsoid
+!! html/php
+foo
+foo
+foo
+foo
+
+!! html/parsoid
 foo
 
 foo
@@ -2781,11 +2805,11 @@
 Templates: Other wikitext in parameter names (bug 67657)
 !! wikitext
 {{echo|''1''=foo}}
-!!html/parsoid
-{{{1}}}
-!!html/php
+!! html/php
 {{{1}}}
 
+!! html/parsoid
+{{{1}}}
 !!end
 
 #
@@ -3860,7 +3884,7 @@
 ## All Parsoid only definition list tests have this difference.
 ##
 ## See also: https://phabricator.wikimedia.org/T8569
-## and 
http://lists.wikimedia.org/pipermail/wikitext-l/2011-November/000483.html
+## and 
https://lists.wikimedia.org/pipermail/wikitext-l/2011-November/000483.html
 
 !! test
 Table / list interaction: indented table with lists in table contents
@@ -5214,7 +5238,7 @@
 http://www.example.com/?title=AT%26T;>http://www.example.com/?title=AT%26T
 !! end
 
-# According to 
http://www.w3.org/TR/2011/WD-html5-20110525/Overview.html#parsing-urls a plain
+# According to 
https://www.w3.org/TR/2011/WD-html5-20110525/Overview.html#parsing-urls a plain
 # % is actually legal in HTML5. Any change in output would need testing though.
 !! test
 Bug 4781, 5267: %25 in URL
@@ -5801,7 +5825,7 @@
 
 # This should not produce  as 
 # is the bare minimum required by the spec, see:
-# 
http://www.w3.org/TR/xhtml-modularization/dtd_module_defs.html#a_module_Basic_Tables
+# 
https://www.w3.org/TR/xhtml-modularization/dtd_module_defs.html#a_module_Basic_Tables
 # Parsoid team replies: empty table tags are legal in HTML5
 !! test
 A table with

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Add html/parsoid version of test added in T149622.

2016-12-14 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/327287 )

Change subject: Add html/parsoid version of test added in T149622.
..

Add html/parsoid version of test added in T149622.

Change-Id: Ia2d4c95284a244ab0c0608d6ed61be445917c0ff
---
M tests/parserTests.txt
1 file changed, 2 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/87/327287/1

diff --git a/tests/parserTests.txt b/tests/parserTests.txt
index 07d50a8..0131d00 100644
--- a/tests/parserTests.txt
+++ b/tests/parserTests.txt
@@ -1442,6 +1442,8 @@
 !! html/php
 span class="error"
 
+!! html/parsoid
+span class="error">
 !! end
 
 !! test

-- 
To view, visit https://gerrit.wikimedia.org/r/327287
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia2d4c95284a244ab0c0608d6ed61be445917c0ff
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Sync up with Parsoid parserTests.

2016-12-14 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/327278 )

Change subject: Sync up with Parsoid parserTests.
..

Sync up with Parsoid parserTests.

This now aligns with Parsoid commit 73798df0632e10313b82987d0b99e93c73407ca7

Change-Id: Ia0e511311eb05276617cc7bdff72b07347591ca3
---
M tests/parser/parserTests.txt
1 file changed, 426 insertions(+), 129 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/78/327278/1

diff --git a/tests/parser/parserTests.txt b/tests/parser/parserTests.txt
index 5b17eac..07d50a8 100644
--- a/tests/parser/parserTests.txt
+++ b/tests/parser/parserTests.txt
@@ -271,6 +271,12 @@
 
 !!endarticle
 
+!! article
+Template:CircularRef
+!! text
+{{CircularRef}}
+!! endarticle
+
 ###
 ### Basic tests
 ###
@@ -280,6 +286,15 @@
 !! html
 !! end
 
+!! test
+CircularRef
+!! wikitext
+{{CircularRef}}
+
+!! html/parsoid
+[1]
+↑  Error: Expansion loop detected at 
Template:CircularRef
+!! end
 
 !! test
 Simple paragraph
@@ -2727,6 +2742,10 @@
 
 |foo}}
 
+{{echo
+
+  |foo}}
+
 {{echo
 |foo}}
 
@@ -2738,9 +2757,12 @@
 foo
 foo
 foo
+foo
 
 !! html/parsoid
 foo
+
+foo
 
 foo
 
@@ -3686,7 +3708,6 @@
 !! end
 
 !! test
-
 Definition list with empty definition and following paragraph
 !! wikitext
 ; term:
@@ -7596,8 +7617,6 @@
 
 !! end
 
-# Note that parsoid does not munge anchor text; all non-space
-# characters are valid in HTML5 ids.
 !! test
 Anchor containing a #. (bug 63430)
 !! wikitext
@@ -7606,7 +7625,7 @@
 Main 
Page#And#Link
 
 !! html/parsoid
-Main 
Page#And#Link
+Main Page#And#Link
 !! end
 
 !! test
@@ -7721,18 +7740,16 @@
 
 !!end
 
-# note that parsoid does not munge anchor text; all non-space
-# characters are valid in HTML5 anchors.
+## Example for such a section: == < ==
 !! test
 Link containing "#<" and "#>" % as a hex sequences- these are valid section 
anchors
-Example for such a section: == < ==
 !! wikitext
 [[%23%3c]][[%23%3e]]
 !! html/php
 ##
 
 !! html/parsoid
-##>
+##>
 !! end
 
 !! test
@@ -8252,7 +8269,7 @@
 Foo#bar
 
 !! html/parsoid
-Foo#bar>
+Foo#bar>
 !! end
 
 ###
@@ -8308,6 +8325,23 @@
 !! html
 http://www.usemod.com/cgi-bin/mb.pl?SoftSecurity#foo; 
class="extiw" title="meatball:SoftSecurity">MeatBall:SoftSecurity#foo
 
+!! end
+
+!! test
+Link scenarios with escaped fragments
+!! wikitext
+[[#Is this great?]]
+[[Foo#Is this great?]]
+[[meatball:Foo#Is this great?]]
+!! html/php
+#Is this great?
+Foo#Is this great?
+http://www.usemod.com/cgi-bin/mb.pl?Foo#Is_this_great.3F; 
class="extiw" title="meatball:Foo">meatball:Foo#Is this great?
+
+!! html/parsoid
+#Is this great?
+Foo#Is this great?
+http://www.usemod.com/cgi-bin/mb.pl?Foo#Is_this_great.3F; 
title="meatball:Foo" 
data-parsoid='{"stx":"simple","a":{"href":"http://www.usemod.com/cgi-bin/mb.pl?Foo#Is_this_great.3F"},"sa":{"href":"meatball:Foo#Is
 this great?"},"isIW":true}'>meatball:Foo#Is this great?
 !! end
 
 # Ideally the wikipedia: prefix here should be proto-relative too
@@ -10374,7 +10408,6 @@
 
 !! end
 
-
 !! test
 Urlencode
 !! wikitext
@@ -10382,7 +10415,7 @@
 {{urlencode:hi world?!|WIKI}}
 {{urlencode:hi world?!|PATH}}
 {{urlencode:hi world?!|QUERY}}
-!! html
+!! html/php
 hi+world%3F%21
 hi_world%3F!
 hi%20world%3F%21
@@ -15929,7 +15962,7 @@
 HTML rocks
 
 !! html/parsoid
-HTML rocks
+HTML rocks
 !! end
 
 !! test
@@ -17107,9 +17140,11 @@
 Sanitizer: Escaping of spaces, multibyte characters, colons & other stuff in 
id=""
 !! wikitext
 byte[[#æ: v|backlink]]
-!! html
+!! html/php
 bytebacklink
 
+!! html/parsoid
+bytebacklink
 !! end
 
 # In HTML5, the restrictions are that id must contain at least one character,
@@ -18858,8 +18893,14 @@
 !! end
 ### Note: Above tests excludes the "{{NUMBEROFADMINS}}" magic word because it 
generates a MySQL error when included.
 
+## Parsoid thinks the "centre" here is a property, not a caption.
 !! test
 Gallery
+!! options
+parsoid={
+  "modes": ["wt2html"],
+  "nativeGallery": true
+}
 !! wikitext
 
 image1.png |
@@ -18871,7 +18912,7 @@
 [[x|xx
 * image6
 
-!! html
+!! html/php
 

Image1.png
@@ -18909,19 +18950,33 @@

 
 
+!! html/parsoid
+
+image1.png 
+image2.gif
+image3
+image4
+ image5.svg http:/;>http:/
+* image6
+
 !! end
 
 !! test
-Gallery (with options)
+Gallery (with options, html)
+!! options
+parsoid={
+  "modes": ["wt2html", "html2html"],
+  "nativeGallery": true
+}
 !! wikitext
-
+
 File:Nonexistent.jpg|caption
 File:Nonexistent.jpg
 image:foobar.jpg|some '''caption''' [[Main Page]]
 image:foobar.jpg
 image:foobar.jpg|Blabla|alt=This is a foo-bar.|blabla.
 
-!! html
+!! html/php
 
Foo Main Page

@@ -18957,17 +19012,92 @@

 
 
+!! html/parsoid
+
+Foo Main Page
+File:Nonexistent.jpgcaption
+File:Nonexistent.jpg
+some 
caption Main 
Page
+
+blabla.
+
+!! end
+
+!! test
+Gallery (with

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: DON'T MERGE: update parsertests with LC fixes in core.

2016-12-14 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/327382 )

Change subject: DON'T MERGE: update parsertests with LC fixes in core.
..

DON'T MERGE: update parsertests with LC fixes in core.

This does a preemptive sync of Parsoid parser tests with core, assuming
that the following core patches are merged:

https://gerrit.wikimedia.org/r/311849 - template arguments
https://gerrit.wikimedia.org/r/327042 - images
https://gerrit.wikimedia.org/r/76341 - gallery
https://gerrit.wikimedia.org/r/327112 - definition lists
https://gerrit.wikimedia.org/r/327127 - tables

Once they are actually merged into core, then we'll do a normal Parsoid
parsertests sync to accomplish the same thing as this patch.

Change-Id: Ide584deea52f7720d71da1ce6b0216d4245b0c8e
---
M tests/parserTests-blacklist.js
M tests/parserTests.txt
2 files changed, 190 insertions(+), 26 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/82/327382/1

diff --git a/tests/parserTests-blacklist.js b/tests/parserTests-blacklist.js
index 6b4288b..58aca78 100644
--- a/tests/parserTests-blacklist.js
+++ b/tests/parserTests-blacklist.js
@@ -226,9 +226,10 @@
 add("wt2html", "Do not convert roman numbers to language variants", "Fridrih IV je car.");
 add("wt2html", "Don't convert raw rule \"-{R|=}-\" to \"=>\"", "-{R|=>}-");
 add("wt2html", "Don't break link parsing if language converter markup is in 
the caption.", "-{R|main page}-");
-add("wt2html", "Don't break image parsing if language converter markup is in 
the caption.", "");
-add("wt2html", "Don't break list handling if language converter markup is in 
the item.", "-{zh-cnAAA;zh-tw:BBB}-");
-add("wt2html", "Don't break table handling if language converter markup is in 
the cell.", "\n\nB}-\n");
+add("wt2html", "T146305: Don't break image parsing if language converter 
markup is in the caption.", "");
+add("wt2html", "Don't break gallery if language converter markup is inside.", 
"\nbat}-\nThis
 is a test template\n");
+add("wt2html", "T153135: Don't break list handling if language converter 
markup is in the item.", "-{zh-cnAAA;zh-tw:BBB}-");
+add("wt2html", "T153140: Don't break table handling if language converter 
markup is in the cell.", "\n\nB}-\n");
 add("wt2html", "Bug 529: Uncovered bullet in parser function result", " 
Foo  bar");
 add("wt2html", "Bug 5678: Double-parsed template invocation", "{{paramtest 
 param = hello }}");
 add("wt2html", "Case insensitivity of parser functions for non-ASCII 
characters (bug 8143)", "Warning:
 Page/template fetching disabled, and no cache for 
PRVNÍVELKÉ:ěščř\nWarning:
 Page/template fetching disabled, and no cache for 
Prvnívelké:ěščř\nWarning:
 Page/template fetching disabled, and no cache for PRVNÍMALÉ:ěščř\nWarning:
 Page/template fetching disabled, and no cache for Prvnímalé:ěščř\nWarning:
 Page/template fetching disabled, and no cache for MALÁ:ěščř\nWarning:
 Page/template fetching disabled, and no cache for Malá:ěščř\nWarning:
 Page/template fetching disabled, and no cache for VELKÁ:ěščř\nWarning:
 Page/template fetching disabled, and no cache for Velká:ěščř");
@@ -258,6 +259,11 @@
 add("wt2html", "Check noCommafy in formatNum", "123456.78\n123456.78");
 add("wt2html", "Wrong option for formatNum (bug 56199)", "1,234.56\n1,234.56\n1234.56");
 add("wt2html", "Strip marker in grammar", "Parser
 function implementation for pf_grammar missing in Parsoid.");
+add("wt2html", "Gallery override link with WikiLink (bug 34852)", "\ncaption\n");
+add("wt2html", "Gallery override link with absolute external link (bug 
34852)", "\nhttp://www.example.org\;>caption\n");
+add("wt2html", "Gallery override link with absolute external link with 
LanguageConverter", "\nhttp://www.example.org\;>caption\n");
+add("wt2html", "Gallery override link with malicious javascript (bug 34852)", 
"\ncaption\n");
+add("wt2html", "Gallery with invalid title as link (bug 43964)", "\nlink=\n");
 add("wt2html", "Language parser function", "ar");
 add("wt2html", "Special parser function", "Parser
 function implementation for pf_#special missing in Parsoid.\nParser
 function implementation for pf_#special missing in Parsoid.\nParser
 function implementation for pf_#special missing in Parsoid.");
 add("wt2html", "1. SOL-sensitive wikitext tokens as template-args", "*a\n#a\n:a");
@@ -345,7 +351,7 @@
 add("wt2wt", "Image with page parameter", "[[File:LoremIpsum.djvu]]\n");
 add("wt2wt", "Don't fall for the self-closing div", "hello world");
 add("wt2wt", "Parsing of overlapping (improperly nested) inline html tags", 
"x\n");
-add("wt2wt", "Don't break table handling if language converter markup is in 
the cell.", "{|\n|-\n| -{R |B}-\n|}");
+add("wt2wt", "T153140: Don't break table handling if language converter markup 
is in the cell.", "{|\n|-\n| -{R |B}-\n|}");
 add("wt2wt", "Bug 529: Uncovered bullet leaving empty list, normally removed

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Preserve null bytes in LanguageConverter conversions.

2016-12-16 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/327779 )

Change subject: Preserve null bytes in LanguageConverter conversions.
..

Preserve null bytes in LanguageConverter conversions.

Use a replacement string to allow null bytes to be preserved.
This is a follow-up to I9b099273203482ffb570a5654d8ba50c833e526d,
and fixes a bug discovered while working on
I7fa2d85d6364958c5138366e8b4504a2697a8731.

Change-Id: I02d33ada52e3fb9ee037bc03b1a83e170bd330dd
---
M languages/LanguageConverter.php
M tests/parser/extraParserTests.txt
2 files changed, 7 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/79/327779/1

diff --git a/languages/LanguageConverter.php b/languages/LanguageConverter.php
index 06fec44..f0621aa 100644
--- a/languages/LanguageConverter.php
+++ b/languages/LanguageConverter.php
@@ -360,8 +360,12 @@
   1. HTML markups (anything between < and >)
   2. HTML entities
   3. placeholders created by the parser
+  4. the replacement string we use ourselves for null bytes
*/
-   $marker = '|' . Parser::MARKER_PREFIX . '[\-a-zA-Z0-9]+';
+   $nullReplacement =
+   Parser::MARKER_PREFIX . '-lc-null-' . 
Parser::MARKER_SUFFIX;
+   $marker = '|' . Parser::MARKER_PREFIX . '[\-a-zA-Z0-9]+'
+   . '(' . Parser::MARKER_SUFFIX . ')?';
 
// this one is needed when the text is inside an HTML markup
$htmlfix = '|<[^>]+$|^[^<>]*>';
@@ -380,7 +384,7 @@
$literalBlob = '';
 
// Guard against delimiter nulls in the input
-   $text = str_replace( "\000", '', $text );
+   $text = str_replace( "\000", $nullReplacement, $text );
 
$markupMatches = null;
$elementMatches = null;
@@ -441,6 +445,7 @@
$translatedIter->next();
$literalIter->next();
}
+   $output = str_replace( $nullReplacement, "\000", $output );
 
return $output;
}
diff --git a/tests/parser/extraParserTests.txt 
b/tests/parser/extraParserTests.txt
index a48087e..553d0e8 100644
--- a/tests/parser/extraParserTests.txt
+++ b/tests/parser/extraParserTests.txt
Binary files differ

-- 
To view, visit https://gerrit.wikimedia.org/r/327779
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I02d33ada52e3fb9ee037bc03b1a83e170bd330dd
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Allow HTML tags in LanguageConverter output.

2016-12-16 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/327778 )

Change subject: Allow HTML tags in LanguageConverter output.
..

Allow HTML tags in LanguageConverter output.

A "remove HTML tags to avoid disrupting the layout" block is removed
(previously added in f16d1e4ed70cd5a8fa6ae6ca8bb71bfe62f4f47e).

This is a follow-up to I9b099273203482ffb570a5654d8ba50c833e526d.

Bug: T54192
Change-Id: I565fac58b3b0da7bfaedf64f5001c364f52e2244
---
M languages/LanguageConverter.php
M tests/parser/parserTests.txt
2 files changed, 13 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/78/327778/1

diff --git a/languages/LanguageConverter.php b/languages/LanguageConverter.php
index 5a9f652..06fec44 100644
--- a/languages/LanguageConverter.php
+++ b/languages/LanguageConverter.php
@@ -415,8 +415,6 @@
$attr = 
$this->recursiveConvertTopLevel( $attr, $toVariant );
}
 
-   // Remove HTML tags to avoid disrupting 
the layout
-   $attr = preg_replace( '/<[^>]+>/', '', 
$attr );
if ( $attr !== $attrs[$attrName] ) {
$attrs[$attrName] = $attr;
$changed = true;
diff --git a/tests/parser/parserTests.txt b/tests/parser/parserTests.txt
index 18e7e96..43a44bf 100644
--- a/tests/parser/parserTests.txt
+++ b/tests/parser/parserTests.txt
@@ -14144,6 +14144,19 @@
 !! end
 
 !! test
+Escape HTML special chars in image alt text with LanguageConverter
+!! options
+language=zh
+!! wikitext
+[[File:Foobar.jpg|& < > "]]
+!! html/php
+http://example.com/images/3/3a/Foobar.jpg; width="1941" height="220" />
+
+!! html/parsoid
+
+!! end
+
+!! test
 Entities in file name and attributes
 !! wikitext
 [[File:7%25 solution.gif|manualthumb=7%25 solution.gif|link=7%25 
solution|[[7%25 solution

-- 
To view, visit https://gerrit.wikimedia.org/r/327778
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I565fac58b3b0da7bfaedf64f5001c364f52e2244
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: API improvements needed for ProofreadPage extension.

2017-01-11 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/331817 )

Change subject: API improvements needed for ProofreadPage extension.
..

API improvements needed for ProofreadPage extension.

ProofreadPage wants to fetch its content w/ a specific non-wikitext
contentformat.

Change-Id: I3e039e36fd8202b5a068a7dcb2303e99bada8ad6
---
M lib/config/extapi.js
M lib/mw/ApiRequest.js
2 files changed, 11 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/17/331817/1

diff --git a/lib/config/extapi.js b/lib/config/extapi.js
index 193e30e..5b01687 100644
--- a/lib/config/extapi.js
+++ b/lib/config/extapi.js
@@ -28,6 +28,7 @@
// and explicitly mark the exported functions as 
"stable", ie
// we need to bump Parsoid's major version if the 
exported
// functions are changed.
+   TemplateRequest: 
require('../mw/ApiRequest.js').TemplateRequest,
Util: require('../utils/Util.js').Util,
DOMUtils: require('../utils/DOMUtils.js').DOMUtils,
JSUtils: require('../utils/jsutils.js').JSUtils,
diff --git a/lib/mw/ApiRequest.js b/lib/mw/ApiRequest.js
index 7421c17..dd2ef3f 100644
--- a/lib/mw/ApiRequest.js
+++ b/lib/mw/ApiRequest.js
@@ -488,12 +488,13 @@
  * @param {string} title The template (or really, page) we should fetch from 
the wiki
  * @param {string} oldid The revision ID you want to get, defaults to "latest 
revision"
  */
-function TemplateRequest(env, title, oldid) {
+function TemplateRequest(env, title, oldid, opts) {
ApiRequest.call(this, env, title);
// IMPORTANT: Set queueKey to the 'title'
// since TemplateHandler uses it for recording listeners
this.queueKey = title;
this.reqType = "Template Fetch";
+   opts = opts || {}; // optional extra arguments
 
var apiargs = {
format: 'json',
@@ -502,6 +503,12 @@
rawcontinue: 1,
rvprop: PARSOID_RVPROP,
};
+
+   // Allow request to give a specific contentformat
+   // (otherwise wikitext is the default)
+   if (opts.contentformat) {
+   apiargs.rvcontentformat = opts.contentformat;
+   }
 
if (oldid) {
this.oldid = oldid;
@@ -613,8 +620,8 @@
 TemplateRequest.promise = promiseFor(TemplateRequest);
 
 // Function which returns a promise to set page src info.
-TemplateRequest.setPageSrcInfo = function(env, target, oldid) {
-   return TemplateRequest.promise(env, target, oldid).then(function(src) {
+TemplateRequest.setPageSrcInfo = function(env, target, oldid, opts) {
+   return TemplateRequest.promise(env, target, oldid, 
opts).then(function(src) {
env.setPageSrcInfo(src);
});
 };

-- 
To view, visit https://gerrit.wikimedia.org/r/331817
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I3e039e36fd8202b5a068a7dcb2303e99bada8ad6
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Revert "Protect language converter markup in the preprocessor."

2017-01-03 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/330315 )

Change subject: Revert "Protect language converter markup in the preprocessor."
..

Revert "Protect language converter markup in the preprocessor."

This effectively reverts commit 28774022769d2273be16c6c6e1cca710a1fd97ef in
order to unblock the deploy train.  The underlying behavior might not be
incorrect, but it was unexpected.

Bug: T153761
Change-Id: Ifc9c7cf3482dd5d222ff4da24a6d4cc401e9d965
---
M includes/parser/Preprocessor_DOM.php
M includes/parser/Preprocessor_Hash.php
M tests/parser/parserTests.txt
3 files changed, 6 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/15/330315/1

diff --git a/includes/parser/Preprocessor_DOM.php 
b/includes/parser/Preprocessor_DOM.php
index 950d66d..25929ec 100644
--- a/includes/parser/Preprocessor_DOM.php
+++ b/includes/parser/Preprocessor_DOM.php
@@ -223,7 +223,8 @@
 
$searchBase = "[{<\n"; # }
if ( !$wgDisableLangConversion ) {
-   $searchBase .= '-';
+   // FIXME: disabled due to T153761
+   //$searchBase .= '-';
}
 
// For fast reverse searches
diff --git a/includes/parser/Preprocessor_Hash.php 
b/includes/parser/Preprocessor_Hash.php
index 1317e60..ee11ecb 100644
--- a/includes/parser/Preprocessor_Hash.php
+++ b/includes/parser/Preprocessor_Hash.php
@@ -155,7 +155,8 @@
 
$searchBase = "[{<\n";
if ( !$wgDisableLangConversion ) {
-   $searchBase .= '-';
+   // FIXME: disabled due to T153761
+   //$searchBase .= '-';
}
 
// For fast reverse searches
diff --git a/tests/parser/parserTests.txt b/tests/parser/parserTests.txt
index b6f2830..f6b6bce 100644
--- a/tests/parser/parserTests.txt
+++ b/tests/parser/parserTests.txt
@@ -20633,10 +20633,12 @@
 
 !! end
 
+# FIXME: This test is currently broken in the PHP parser T153761
 !! test
 T146304: Don't break template parsing if language converter markup is in the 
parameter.
 !! options
 language=sr variant=sr-ec
+disabled
 !! wikitext
 {{echo|-{R|foo}-}}
 !! html/php

-- 
To view, visit https://gerrit.wikimedia.org/r/330315
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ifc9c7cf3482dd5d222ff4da24a6d4cc401e9d965
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Don't bail on single-line definition list due to excess clos...

2016-12-22 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/328697 )

Change subject: Don't bail on single-line definition list due to excess close 
tags.
..

Don't bail on single-line definition list due to excess close tags.

When parsing a single line definition list, we track nested tags so that:

; foo:bar: baz

breaks before `baz`, not between `foo` and `bar`.  But we currently bail
out of this algorithm entirely if we see a mismatched close tag.  We should
just ignore the unmatched tag, like Parsoid does.

Change-Id: I6306dcad6347abeb6ab001d35562f1ab9f374bd1
---
M includes/parser/BlockLevelPass.php
M tests/parser/parserTests.txt
2 files changed, 30 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/97/328697/1

diff --git a/includes/parser/BlockLevelPass.php 
b/includes/parser/BlockLevelPass.php
index e16cfd4..2ef599a 100644
--- a/includes/parser/BlockLevelPass.php
+++ b/includes/parser/BlockLevelPass.php
@@ -496,10 +496,12 @@
case self::COLON_STATE_CLOSETAG:
# In a 
if ( $c === ">" ) {
-   $ltLevel--;
-   if ( $ltLevel < 0 ) {
+   if ( $ltLevel > 0 ) {
+   $ltLevel--;
+   } else {
+   # ignore the excess close tag, 
but keep looking for
+   # colons. (This matches Parsoid 
behavior.)
wfDebug( __METHOD__ . ": 
Invalid input; too many close tags\n" );
-   return false;
}
$state = self::COLON_STATE_TEXT;
}
diff --git a/tests/parser/parserTests.txt b/tests/parser/parserTests.txt
index b0e3d81..f38643c 100644
--- a/tests/parser/parserTests.txt
+++ b/tests/parser/parserTests.txt
@@ -3686,6 +3686,31 @@
 !! end
 
 !! test
+Definition lists: ignore colons inside tags
+!! wikitext
+;one two : tag fun::: def
+!! html
+one two: tag fun::
+ def
+
+!! end
+
+!! test
+Definition lists: excess closed tags
+!! wikitext
+;onetwo : bad tag fun
+!! html/php
+one/btwo
+ bad tag fun
+
+!! html/parsoid
+
+onetwo
+bad tag fun
+
+!! end
+
+!! test
 Bug 11748: Literal closing tags
 !! wikitext
 

-- 
To view, visit https://gerrit.wikimedia.org/r/328697
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I6306dcad6347abeb6ab001d35562f1ab9f374bd1
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: DON'T PUSH: remove a bunch of tests which ought to pass fr...

2016-12-22 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/328716 )

Change subject: DON'T PUSH: remove a bunch of tests which *ought* to pass from 
blacklist.
..

DON'T PUSH: remove a bunch of tests which *ought* to pass from blacklist.

Remove all the language-converter related tests from blacklist, so I
can see more clearly what is still failing.

Change-Id: Ibc16988fb1e3230362bbdd36809716df08d91919
---
M tests/parserTests-blacklist.js
1 file changed, 0 insertions(+), 120 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/16/328716/1

diff --git a/tests/parserTests-blacklist.js b/tests/parserTests-blacklist.js
index bb03503..e20068c 100644
--- a/tests/parserTests-blacklist.js
+++ b/tests/parserTests-blacklist.js
@@ -163,7 +163,6 @@
 add("wt2html", "CSS line continuation 2", "");
 add("wt2html", "Sanitizer: Closing of closed but not open table tags", "Table 
not started");
 add("wt2html", "Sanitizer: Validating that  and  work, but only 
for Microdata", "\n\tmeta itemprop=\"hello\" 
content=\"world\">\n\tmeta http-equiv=\"refresh\" 
content=\"5\">\n\tmeta itemprop=\"hello\" http-equiv=\"refresh\" 
content=\"5\">\n\tlink itemprop=\"hello\" href=\"{{SERVER}}\">\n\tlink 
rel=\"stylesheet\" href=\"{{SERVER}}\">\n\tlink rel=\"stylesheet\" 
itemprop=\"hello\" href=\"{{SERVER}}\">\n");
-add("wt2html", "Language converter: output gets cut off unexpectedly (bug 
5757)", "this bit is safe: }-\n\nbut if we add a conversion instance: 
-{zh-cn:xxx;zh-tw:yyy}-\n\nthen we 
get cut off here: }-\n\nall 
additional text is vanished");
 add("wt2html", "Self closed html pairs (bug 5487)", "Centered
 text\nIn
 div text");
 add("wt2html", "Fuzz testing: Parser14", " onmouseover= \nhttp://__TOC__\; 
data-parsoid='{\"stx\":\"url\",\"dsr\":[19,33,0,0]}'>http://__TOC__");
 add("wt2html", "Fuzz testing: Parser24", "{{{|\n\nMOVE
 YOUR MOUSE CURSOR OVER THIS TEXT\n\n\n\n");
@@ -192,45 +191,6 @@
 add("wt2html", "anchorencode trims spaces", "__pretty__please__");
 add("wt2html", "anchorencode deals with links", "world 
hi");
 add("wt2html", "anchorencode encodes like the TOC generator: (bug 18431)", 
" _ 
+:.3A%3A]]
 \n_ 
+:.3A%3A]]\n");
-add("wt2html", "Self-link in language variants", "Both Dunav
 and Дунав
 are names for this river.");
-add("wt2html", "Link to another existing title shouldn't be parsed as 
self-link even if it's a variant of this title", "Дуна
 is not a self-link while Duna
 and Dуна
 are still self-links.");
-add("wt2html", "Link to a section of a variant of this title shouldn't be 
parsed as self-link", "Dуна
 is a self-link while Dunа#Foo
 and Dуна#Foo
 are not self-links.");
-add("wt2html", "Link to pages in language variants", "Main Page can be written as Маин Паге");
-add("wt2html", "Multiple links to pages in language variants", "Main Page can be written as Маин Паге same as Маин Паге.");
-add("wt2html", "Simple template in language variants", "Warning:
 Page/template fetching disabled, and no cache for Шаблон:Тест");
-add("wt2html", "Template with explicit namespace in language variants", "Warning:
 Page/template fetching disabled, and no cache for Шаблон:Тест");
-add("wt2html", "Basic test for template parameter in language variants", "Warning:
 Page/template fetching disabled, and no cache for 
Шаблон:Парамтест");
-add("wt2html", "Stripping -{}- tags (language variants)", "Latin proverb: -{Ne nuntium necare}-");
-add("wt2html", "Prevent conversion with -{}- tags (language variants)", "Latinski: -{Ne nuntium necare}-");
-add("wt2html", "Prevent conversion of text with -{}- tags (language 
variants)", "Latinski: -{Ne nuntium 
necare}-");
-add("wt2html", "Prevent conversion of links with -{}- tags (language 
variants)", "-{Main Page}-");
-add("wt2html", "-{}- tags within headlines (within html for parserConvert())", 
" -{Naslov}- 
");
-add("wt2html", "Explicit definition of language variant alternatives", "-{zh:China;zh-tw:Taiwan}-, not China");
-add("wt2html", "Conversion around HTML tags", "-{H|span=>sr-ec:script;title=>sr-ec:src;}-\nski");
-add("wt2html", "Explicit session-wise language variant mapping (A flag and - 
flag)", "Taiwan is not China.\nBut 
-{A|zh:China;zh-tw:Taiwan}- is China,\n(This-{-|zh:China;zh-tw:Taiwan}- should 
be stripped!)\nand -{China}- is China.");
-add("wt2html", "Explicit session-wise language variant mapping (H flag for 
hide)", "(This-{H|zh:China;zh-tw:Taiwan}- should be 
stripped!)\nTaiwan is China.");
-add("wt2html", "Adding explicit conversion rule for title (T flag)", "Should be 
stripped-{T|zh:China;zh-tw:Taiwan}-!");
-add("wt2html", "Testing that changing the language variant here in the tests 
actually works", "Should be 
stripped-{T|zh:China;zh-tw:Taiwan}-!");
-add("wt2html", "Recursive conversion of alt and title attrs shouldn't clear 
converter state", "-{H|zh-cn:Exclamation;zh-tw:exclamation;}-\nShould
 be

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: WIP add new languageconverter testcases

2017-04-12 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/347979 )

Change subject: WIP add new languageconverter testcases
..

WIP add new languageconverter testcases

Change-Id: If75a9c9d1ad63c598686838125484dd2f5f6c5a4
---
M tests/parser/parserTests.txt
1 file changed, 59 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/79/347979/1

diff --git a/tests/parser/parserTests.txt b/tests/parser/parserTests.txt
index 0e2e5a6..32a29aa 100644
--- a/tests/parser/parserTests.txt
+++ b/tests/parser/parserTests.txt
@@ -11744,8 +11744,11 @@
 language=zh
 !! wikitext
 {{-{R|raw}}}-
+
+[[-{R|raw]]}-
 !! html
 {{raw}}
+[[raw]]
 
 !! end
 
@@ -11881,7 +11884,63 @@
 
 !! end
 
+!! test
+Preprocessor precedence 12: broken language converter closed by brace.
+!! wikitext
+This form breaks the template, which is unfortunate:
+* {{echo|foo-{bar}bat}}
 
+But if the broken language converter markup is inside an extension
+tag, nothing bad happens:
+* foo-{bar}bat
+* {{echo|foo-{bar}bat}}
+* foo-{bar}bat
+* {{echo|foo-{bar}bat}}
+* foo-{bar}bat
+* {{echo|foo-{bar}bat}}
+
+
+{{cite web|title=foo-{bar} bat|date=}}
+Foo -{bar} bat
+x
+
+
+[http://example.com Example in -{foo} link]
+
+[http://example.com Example in -{foo} link]
+
+This can be confusing, since by default...
+
+* -{foo} ...should be ignored inside extension markup
+* [http://foo/bar-{bat} Bat] ...should be ignored inside URL string
+*  ...should be ok
+*  ...extra dashes
+* {{echo|foobat}} ...should be ok
+
+This only occurs inside template context:
+* {{Foo -{R|raw}yep }-}}
+* [[Foo -{R|raw}nopge }-]]
+* -{R|raw}nope }-
+
+This behavior is for backwards-compatibility with existing text
+(mostly chemical formulas); other brace types don't do this.  Compare:
+* {{echo|foo[[bar}bat}}
+* {{echo|foo[[bar]bat}}
+* {{echo|foo{{{bar}bat}}
+
+Headings have a similar behavior, though.
+===foo[[bar===
+
+[http://example.com-{foo sasd]
+
+{{echo|foo}}
+
+{{echo|[http://example.com-{foo sasd]}}
+
+[http://example.com-{foo sasd] {{echo|foo}}
+
+!! html
+!! end
 
 ###
 ### Token Stream Patcher tests

-- 
To view, visit https://gerrit.wikimedia.org/r/347979
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If75a9c9d1ad63c598686838125484dd2f5f6c5a4
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Match [[:mw:Preprocessor_ABNF]] behavior for "broken link".

2017-04-19 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/348992 )

Change subject: Match [[:mw:Preprocessor_ABNF]] behavior for "broken link".
..

Match [[:mw:Preprocessor_ABNF]] behavior for "broken link".

Match the behavior of the PHP preprocessor for links missing their
close brackets.

Change-Id: Ied94e2224d5d72d6b36906f4bb5fd8b55e645210
---
M lib/wt2html/pegTokenizer.pegjs
1 file changed, 20 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/92/348992/1

diff --git a/lib/wt2html/pegTokenizer.pegjs b/lib/wt2html/pegTokenizer.pegjs
index cd25d08..4abf3ac 100644
--- a/lib/wt2html/pegTokenizer.pegjs
+++ b/lib/wt2html/pegTokenizer.pegjs
@@ -954,18 +954,34 @@
 target:wikilink_preprocessor_text?
 tpos:("" { return endOffset(); })
 lcontent:wikilink_content?
-"]]"
+closed:"]]"?
   {
   if (lcontent === null) {
   lcontent = { content: [] };
   }
 
-  if (target === null || lcontent.pipetrick) {
-return [text()];
+  var textTokens = [];
+  if (target === null || lcontent.pipetrick || !closed) {
+// !closed => "broken-link" rule; see [[:mw:Preprocessor_ABNF]]
+textTokens.push("[[");
+if (target) {
+  textTokens.push(target);
+}
+if (lcontent.pipetrick) {
+  textTokens.push("|");
+}
+lcontent.content.forEach(function(a) {
+  // a is a mw:maybeContent attribute
+  textTokens.push("|");
+  textTokens.push(a.v);
+});
+if (closed) {
+  textTokens.push("]]");
+}
+return textTokens;
   }
 
   var obj = new SelfclosingTagTk('wikilink');
-  var textTokens = [];
   var hrefKV = new KV('href', target);
   hrefKV.vsrc = input.substring(startOffset() + 2, tpos);
   // XXX: Point to object with path, revision and input information

-- 
To view, visit https://gerrit.wikimedia.org/r/348992
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ied94e2224d5d72d6b36906f4bb5fd8b55e645210
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Fetch LanguageConverter status as part of wiki configuration.

2017-04-19 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/348998 )

Change subject: Fetch LanguageConverter status as part of wiki configuration.
..

Fetch LanguageConverter status as part of wiki configuration.

Bug: T153341
Change-Id: I7440a51727d24260689e0dc1c4972decea3a7a67
---
M lib/config/MWParserEnvironment.js
M lib/config/WikiConfig.js
A lib/config/variants.json
M lib/mw/ApiRequest.js
4 files changed, 42 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/98/348998/1

diff --git a/lib/config/MWParserEnvironment.js 
b/lib/config/MWParserEnvironment.js
index a8d15af..9b6286e 100644
--- a/lib/config/MWParserEnvironment.js
+++ b/lib/config/MWParserEnvironment.js
@@ -796,6 +796,18 @@
return this.conf.wiki.extContentModel.get(contentmodel);
 };
 
+/**
+ * @method
+ *
+ * Determine if LanguageConverter markup should be parsed on this page,
+ * based on the wiki configuration and the current page language.
+ *
+ * @return {Boolean}
+ */
+MWParserEnvironment.prototype.langConverterEnabled = function() {
+   var lang = this.page.pagelanguage || this.conf.wiki.lang || 'en';
+   return this.conf.wiki.langConverterEnabled.has(lang);
+};
 
 if (typeof module === "object") {
module.exports.MWParserEnvironment = MWParserEnvironment;
diff --git a/lib/config/WikiConfig.js b/lib/config/WikiConfig.js
index 9fe20ea..f216798 100644
--- a/lib/config/WikiConfig.js
+++ b/lib/config/WikiConfig.js
@@ -557,6 +557,34 @@
// Function hooks on this wiki, indexed by their normalized form
this.functionHooks = new Set(resultConf.functionhooks || []);
 
+   // Somewhat annoyingly, although LanguageConversion is turned on by
+   // default for all WMF wikis (ie, $wgDisableLangConversion = false, as
+   // reported by `general.langconversion` in siteinfo), but the
+   // -{ }- syntax is only parsed when the current *page language*
+   // has variants.  We can't use the "UI language" (in siteinfo
+   // `general.lang`) and "UI variants" (in `general.fallback` and
+   // `general.variants`), because the *page language* could be quite
+   // different.  Use the mechanism introduced in T153341 instead.
+   var languagevariants = resultConf.languagevariants;
+   if (languagevariants === undefined) {
+   // Hard-coded list of variants and fallbacks, for mediawiki
+   // releases before 1.29 without T153341
+   languagevariants = require('./variants.json');
+   }
+   this.variants = new Map();
+   this.langConverterEnabled = new Set();
+   Object.keys(languagevariants).forEach(function(code) {
+   if (general.langconversion !== undefined) {
+   this.langConverterEnabled.add(code);
+   }
+   Object.keys(languagevariants[code]).forEach(function(v) {
+   this.variants.set(v, {
+   base: code,
+   fallbacks: languagevariants[code][v].fallbacks,
+   });
+   }.bind(this));
+   }.bind(this));
+
// Match a wikitext line containing just whitespace, comments, and
// sol transparent links and behavior switches.
// Redirects should not contain any preceding non-whitespace chars.
diff --git a/lib/config/variants.json b/lib/config/variants.json
new file mode 100644
index 000..7ef417a
--- /dev/null
+++ b/lib/config/variants.json
@@ -0,0 +1 @@

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Fixes for preprocessor precedence.

2017-04-19 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/348997 )

Change subject: Fixes for preprocessor precedence.
..

Fixes for preprocessor precedence.

The rightmost opening set of braces/brackets take precedence: see
[[:mw:Preprocessor_ABNF]] for details.  So:
  [[Foo|bar}}]]
and
  [[Foo}}|bar]]
are both valid, even if there's an template opening somewhere to the left.
Note however that the latter example will not become a link in the output
because }} is not a valid title character -- it is unlinked at a later stage.

Change-Id: I4d7b1ff638aa6bec8becbb8ecfd649d7a59b1eb2
---
M lib/wt2html/pegTokenizer.pegjs
M tests/parserTests-blacklist.js
M tests/parserTests.txt
3 files changed, 11 insertions(+), 23 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/97/348997/1

diff --git a/lib/wt2html/pegTokenizer.pegjs b/lib/wt2html/pegTokenizer.pegjs
index 4abf3ac..61689ec 100644
--- a/lib/wt2html/pegTokenizer.pegjs
+++ b/lib/wt2html/pegTokenizer.pegjs
@@ -1002,6 +1002,7 @@
/ (!inline_breaks
   r:( inline_element / '[' text_char+ ']' / . ) { return r; }
  )
+   / '}' // "rightmost opening" rule
 )+ {
   return tu.flattenStringlist(c);
   }
@@ -1902,6 +1903,7 @@
 // XXX gwicke: any more chars we need to allow here?
 / !inline_breaks wr:( directive / $( !"]]" ( text_char / [!<\-] ) ) )
 { return wr; }
+/ $"}" // if [[ was rightmost it takes precedence over {{
 )+ {
   return tu.flattenStringlist(r);
   }
diff --git a/tests/parserTests-blacklist.js b/tests/parserTests-blacklist.js
index f895151..5441057 100644
--- a/tests/parserTests-blacklist.js
+++ b/tests/parserTests-blacklist.js
@@ -107,7 +107,6 @@
 add("wt2html", "Template with invalid target containing wikilink", "{{Main Page}}");
 add("wt2html", "Template with complex arguments", "link This is a test template with parameter (test)");
 add("wt2html", "Template infinite loop", "");
-add("wt2html", "Template with targets containing wikilinks", "{{foo}}\n\n{{foo}}\n\n{{[[foo]]}}");
 add("wt2html", "int keyword", "Parser function 
implementation for pf_int missing in Parsoid.");
 add("wt2html", "int keyword - non-existing message", "Parser
 function implementation for pf_int missing in Parsoid.");
 add("wt2html", "Self-closed noinclude, includeonly, onlyinclude tags", "\n\n");
@@ -297,7 +296,6 @@
 add("wt2wt", "Template with argument in separate line", "{{templateasargtest  
|\n templ = simple }}");
 add("wt2wt", "T2553: link with two variables in a piped link", 
"{|\n|[[{{{1}}}|{{{2}}}]]\n|}");
 add("wt2wt", "Abort table cell attribute parsing on wikilink", "{|\n| testing 
[[one|two]] | three || four\n| testing one two | three || 
four\n| testing=\"[[one|two]]\" | three || four\n|}");
-add("wt2wt", "Template with targets containing wikilinks", 
"{{[[foo]]}}\n\n{{[[{{echo|foo}}]]}}\n\n{{[[foo]]}}\n");
 add("wt2wt", "2. includeonly in html attr value", "v1v2\">bar\nv1v2\">bar\n");
 add("wt2wt", "1. Table tag in SOL posn. should get reparsed correctly with 
valid TSR", "{{echo|}}\n{| width=\"100%\"\n|foo\n|}");
 add("wt2wt", "Templates: HTML Tag: 2. Generation of HTML attr. value", "foo");
@@ -425,7 +423,6 @@
 add("html2html", "Template parameter as link source", "link\n");
 add("html2html", "Template as link source", "Main Page\n\nMain Page\n\nMain PagePage\n");
 add("html2html", "Template infinite loop", "Template loop detected: 
Template:Loop1\n");
-add("html2html", "Template with targets containing wikilinks", "{{foo}}\n\n{{foo}}\n\n{{[[foo}}]]\n");
 add("html2html", "Self-closed noinclude, includeonly, onlyinclude tags", "\n");
 add("html2html", "T8563: Section extraction for section shown by 
", "==Section 
T-2==");
 add("html2html", "T8563: Section extraction for section suppressed by 
", "==Section 
1==");
@@ -824,7 +821,6 @@
 add("html2wt", "T2529: Template with table, not included at beginning of 
line", "foo \n\n{|\n| 1 \n| 2\n|-\n| 3 \n| 4\n|}\n");
 add("html2wt", "T2523: Template shouldn't eat newline (or add an extra one 
before table)", "foo\n\n{|\n| 1 \n| 2\n|-\n| 3 \n| 4\n|}\n");
 add("html2wt", "T2041: Template parameters shown as broken links", 
"{{{parameter}}}\n");
-add("html2wt", "Template with targets containing wikilinks", 
"{{[[wiki/Foo|foo]]}}\n\n{{[[wiki/Foo|foo]]}}\n\n{{[[foo}}]]\n");
 add("html2wt", "int keyword", "You have lots of money (not!).\n");
 add("html2wt", "int keyword - non-existing message", "⧼var⧽\n");
 add("html2wt", " and  being included", "Foobar\n");
@@ -1573,22 +1569,6 @@
 add("selser", "Abort table cell attribute parsing on wikilink [[3,1]]", "{|\n| 
testing [[one|two]] | three || four\n| testing one two | three || four\n| 
testing=\"[[one|two]]\" | three || four\n|}");
 add("selser", "Abort table cell attribute parsing on wikilink [[0,2]]", 
"{|\n| testing [[one|two]] | three || four\n| testing

[MediaWiki-commits] [Gerrit] mediawiki...Wikibase[master]: Don't trust LanguageConverter::$languagesWithVariants

2017-04-14 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/348229 )

Change subject: Don't trust LanguageConverter::$languagesWithVariants
..

Don't trust LanguageConverter::$languagesWithVariants

Use $language->hasVariants() as the authoritative source, since variants
could have been disabled with $wgDisabledVariants.

Change-Id: Ibc7197b7cb8f35c389ccd260f2fcd61a2692a101
---
M lib/includes/LanguageFallbackChainFactory.php
1 file changed, 6 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/Wikibase 
refs/changes/29/348229/1

diff --git a/lib/includes/LanguageFallbackChainFactory.php 
b/lib/includes/LanguageFallbackChainFactory.php
index 7d12a73..4f99cfd 100644
--- a/lib/includes/LanguageFallbackChainFactory.php
+++ b/lib/includes/LanguageFallbackChainFactory.php
@@ -130,7 +130,12 @@
if ( is_string( $language ) ) {
$language = Language::factory( 
$language );
}
-   $parentLanguage = 
$language->getParentLanguage();
+   // $languagesWithVariants is just an 
approximation; variants
+   // can be disabled via $wgDisabledVariants, 
etc.  The
+   // $language->hasVariants() API is 
authoritative.
+   if ( $language->hasVariants() ) {
+   $parentLanguage = 
$language->getParentLanguage();
+   }
}
 
if ( $parentLanguage ) {

-- 
To view, visit https://gerrit.wikimedia.org/r/348229
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibc7197b7cb8f35c389ccd260f2fcd61a2692a101
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/Wikibase
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Allow ] in template parameters.

2017-04-21 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/349604 )

Change subject: Allow ] in template parameters.
..

Allow ] in template parameters.

Change-Id: I2c3c7b588724b50eb01ff8583e43e13a0def15ec
---
M lib/wt2html/pegTokenizer.pegjs
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/04/349604/1

diff --git a/lib/wt2html/pegTokenizer.pegjs b/lib/wt2html/pegTokenizer.pegjs
index d031bfc..f89b2c5 100644
--- a/lib/wt2html/pegTokenizer.pegjs
+++ b/lib/wt2html/pegTokenizer.pegjs
@@ -911,7 +911,7 @@
 stops.push('tableCellArg', false);
 return stops.inc('template');
 }
-il:(nested_block / newlineToken)+ {
+il:(nested_block / newlineToken / $"]" )+ {
 stops.pop('table');
 stops.pop('extlink');
 stops.pop('templateArg');

-- 
To view, visit https://gerrit.wikimedia.org/r/349604
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I2c3c7b588724b50eb01ff8583e43e13a0def15ec
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Improve config.example.yaml documentation.

2017-04-14 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/348237 )

Change subject: Improve config.example.yaml documentation.
..

Improve config.example.yaml documentation.

Change-Id: I619e2f06ff5ef7a0b02f4395ed919aed9971c435
---
M config.example.yaml
1 file changed, 3 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/37/348237/1

diff --git a/config.example.yaml b/config.example.yaml
index fff190d..e74103a 100644
--- a/config.example.yaml
+++ b/config.example.yaml
@@ -37,8 +37,9 @@
   uri: 'http://localhost/w/api.php'
   # The "domain" is used for communication with Visual Editor
   # and RESTBase.  It defaults to the hostname portion of
-  # the `uri` property below, but you can manually set it
-  # to an arbitrary string.
+  # the `uri` property above, but you can manually set it
+  # to an arbitrary string. It must match the "domain" set
+  # in $wgVirtualRestConfig.
   domain: 'localhost'  # optional
   # To specify a proxy (or proxy headers) specific to this prefix
   # (which overrides defaultAPIProxyURI). Alternatively, set `proxy`

-- 
To view, visit https://gerrit.wikimedia.org/r/348237
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I619e2f06ff5ef7a0b02f4395ed919aed9971c435
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Improve documentation for sync-parserTests a bit.

2017-04-14 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/348235 )

Change subject: Improve documentation for sync-parserTests a bit.
..

Improve documentation for sync-parserTests a bit.

Change-Id: If85b92ce4db9edba9e01fd8dc182bb8dc4e9efd5
---
M tools/sync-parserTests.js
1 file changed, 5 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/35/348235/1

diff --git a/tools/sync-parserTests.js b/tools/sync-parserTests.js
index d58fef5..f5ac020 100755
--- a/tools/sync-parserTests.js
+++ b/tools/sync-parserTests.js
@@ -8,9 +8,11 @@
 
Basic use:
  $PARSOID is the path to a checked out git copy of Parsoid
- $REPO is the path to a checked out git copy of the repo containing the 
parserTest file
- $BRANCH is a branch name for the patch to $REPO (ie, 'pt-sync')
- $TARGET is the key to parserTests.json we're sync'ing
+ $REPO is the path to a checked out git copy of the repo containing
+   the parserTest file. (Check the `repo` key in tests/parserTests.json)
+ $BRANCH is a branch name for the patch to $REPO (ie, 'ptsync-')
+ $TARGET identifies which set of parserTests we're synchronizing.
+   (This should be one of the top-level keys in tests/parserTests.json)
 
$ cd $PARSOID
$ tools/sync-parserTests.js $REPO $BRANCH $TARGET

-- 
To view, visit https://gerrit.wikimedia.org/r/348235
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If85b92ce4db9edba9e01fd8dc182bb8dc4e9efd5
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Bump src/ to 08114f35 for deploy

2017-07-31 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/368844 )

Change subject: Bump src/ to 08114f35 for deploy
..

Bump src/ to 08114f35 for deploy

Change-Id: I6a0b32ef60ac6e899575b5b9a6560c4352460a3a
---
M src
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid/deploy 
refs/changes/44/368844/1

diff --git a/src b/src
index a89a9cc..08114f3 16
--- a/src
+++ b/src
@@ -1 +1 @@
-Subproject commit a89a9cc4a0ab06106dab87c74b15d0f54b3e2a79
+Subproject commit 08114f35d3040f05d47552eb188d3d03cbacc135

-- 
To view, visit https://gerrit.wikimedia.org/r/368844
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I6a0b32ef60ac6e899575b5b9a6560c4352460a3a
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid/deploy
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Fix link prefix/suffixes around Category and Language links.

2017-08-13 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/371735 )

Change subject: Fix link prefix/suffixes around Category and Language links.
..

Fix link prefix/suffixes around Category and Language links.

Bug: T2087
Bug: T10897
Bug: T87753
Change-Id: I943cd9bec0855d9a326b0b50739d686a29995370
---
M includes/parser/Parser.php
M tests/parser/parserTests.txt
2 files changed, 43 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/35/371735/1

diff --git a/includes/parser/Parser.php b/includes/parser/Parser.php
index 3261002..2173ddc 100644
--- a/includes/parser/Parser.php
+++ b/includes/parser/Parser.php
@@ -2323,8 +2323,11 @@

$this->mOutput->addLanguageLink( $nt->getFullText() );
}
 
+   /**
+* Strip the whitespace interwiki links 
produce, see T10897
+*/
$s = rtrim( $s . $prefix );
-   $s .= trim( $trail, "\n" ) == '' ? '' : 
$prefix . $trail;
+   $s .= rtrim( $trail, "\n" );
continue;
}
 
@@ -2349,7 +2352,11 @@
continue;
}
} elseif ( $ns == NS_CATEGORY ) {
-   $s = rtrim( $s . "\n" ); # T2087
+   /**
+* Strip the whitespace Category links 
produce, see T2087
+*/
+   $s = rtrim( $s . $prefix ); # T2087, 
T87753
+   $s .= rtrim( $trail, "\n" );
 
if ( $wasblank ) {
$sortkey = 
$this->getDefaultSort();
@@ -2361,10 +2368,6 @@
$sortkey = 
$this->getConverterLanguage()->convertCategoryKey( $sortkey );
$this->mOutput->addCategory( 
$nt->getDBkey(), $sortkey );
 
-   /**
-* Strip the whitespace Category links 
produce, see T2087
-*/
-   $s .= trim( $prefix . $trail, "\n" ) == 
'' ? '' : $prefix . $trail;
 
continue;
}
diff --git a/tests/parser/parserTests.txt b/tests/parser/parserTests.txt
index ab79b59..2fa3cb0 100644
--- a/tests/parser/parserTests.txt
+++ b/tests/parser/parserTests.txt
@@ -15808,6 +15808,26 @@
 !! end
 
 !! test
+9. Categories and newlines: should behave properly with linkprefix (T87753)
+!! options
+language=ar
+!! wikitext
+foo bar
+foo bar
+[[تصنيف:Foo]]
+[[تصنيف:Bar]]
+!! html/php
+foo bar
+foo bar
+
+!! html/parsoid
+foo bar
+foo bar
+
+
+!! end
+
+!! test
 Category links with multiple namespaces
 !! wikitext
 [[Category:Project:Foo]]
@@ -15855,6 +15875,20 @@
 !! end
 
 !! test
+Link prefix/suffixes aren't applied to language links
+!! options
+parsoid=wt2html
+language=is
+!! wikitext
+x[[es:Foo]]y
+!! html/php
+xy
+
+!! html/parsoid
+xhttp://es.wikipedia.org/wiki/Foo; 
data-parsoid=""/>y
+!! end
+
+!! test
 Parsoid: Serialize link to file page with colon escape
 !! options
 parsoid

-- 
To view, visit https://gerrit.wikimedia.org/r/371735
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I943cd9bec0855d9a326b0b50739d686a29995370
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] purtle[master]: Add JSON-LD support.

2017-08-11 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/371446 )

Change subject: Add JSON-LD support.
..

Add JSON-LD support.

Change-Id: I77d6a771abfef5b00b1b151c48aa950be30896cf
---
M README.md
M composer.json
A src/JsonLdRdfWriter.php
M src/RdfWriterFactory.php
A tests/data/EricMiller.json
A tests/data/LabeledBlankNode.json
A tests/data/NumberedBlankNode.json
A tests/data/Numbers.json
A tests/data/Predicates.json
A tests/data/Resources.json
A tests/data/TextWithSpecialChars.json
A tests/data/Texts.json
A tests/data/Triples.json
A tests/data/Values.json
A tests/phpunit/JsonLdRdfWriterTest.php
M tests/phpunit/NTriplesRdfWriterTest.php
M tests/phpunit/RdfWriterFactoryTest.php
M tests/phpunit/RdfWriterTestBase.php
M tests/phpunit/TurtleRdfWriterTest.php
M tests/phpunit/XmlRdfWriterTest.php
20 files changed, 583 insertions(+), 15 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/purtle refs/changes/46/371446/1

diff --git a/README.md b/README.md
index 95321b5..1d4e39c 100644
--- a/README.md
+++ b/README.md
@@ -1,14 +1,15 @@
 # Purtle
 
 **Purtle** is a fast, lightweight RDF generator. It provides a "fluent" 
interface for
-generating RDF output in Turtle, XML/RDF or N-Triples. The fluent interface 
allows the
+generating RDF output in Turtle, JSON-LD, XML/RDF or N-Triples. The fluent 
interface allows the
 resulting PHP code to be structured just like Turtle notation for RDF, hence 
the name: "Purtle"
 is a contraction of "PHP Turtle".
 
-The three concrete classes implementing the common `RdfWriter` interface are:
+The four concrete classes implementing the common `RdfWriter` interface are:
 * `TurtleRdfWriter` outputs Turtle
 * `XmlRdfWriter` outputs XML/RDF
 * `NTriplesRdfWriter` outputs N-Triples
+* `JsonLdRdfWriter` outputs JSON-LD
 
 The PHP code would look something like this:
 
diff --git a/composer.json b/composer.json
index 1243c49..1c8f823 100644
--- a/composer.json
+++ b/composer.json
@@ -5,7 +5,8 @@
"keywords": [
"RDF",
"Serializer",
-   "Turtle"
+   "Turtle",
+   "JSON-LD"
],
"homepage": "https://mediawiki.org/wiki/Purtle;,
"license": "GPL-2.0+",
@@ -18,6 +19,9 @@
},
{
"name": "Thiemo Mättig"
+   },
+   {
+   "name": "C. Scott Ananian"
}
],
"support": {
diff --git a/src/JsonLdRdfWriter.php b/src/JsonLdRdfWriter.php
new file mode 100644
index 000..184b826
--- /dev/null
+++ b/src/JsonLdRdfWriter.php
@@ -0,0 +1,262 @@
+transitionTable[self::STATE_START][self::STATE_DOCUMENT] 
= function () {
+   $this->beginJson();
+   };
+   
$this->transitionTable[self::STATE_DOCUMENT][self::STATE_FINISH] = function () {
+   $this->finishJson();
+   };
+   
$this->transitionTable[self::STATE_OBJECT][self::STATE_PREDICATE] = function () 
{
+   $this->finishPredicate();
+   };
+   $this->transitionTable[self::STATE_OBJECT][self::STATE_SUBJECT] 
= function () {
+   $this->finishPredicate();
+   $this->finishSubject();
+   };
+   
$this->transitionTable[self::STATE_OBJECT][self::STATE_DOCUMENT] = function () {
+   $this->finishPredicate();
+   $this->finishSubject();
+   $this->finishDocument();
+   };
+   }
+
+   private function isTopLevel() {
+   return $this->role === self::DOCUMENT_ROLE;
+   }
+
+   private function contextify( $base, $local = null ) {
+   $this->expandShorthand( $base, $local );
+
+   if ( $base === 'rdf' && $local === 'type' ) {
+   return '@type';
+   } elseif ( $local === null ) {
+   return $base;
+   } else {
+   if ( $base !== '_' && $this->isPrefix( $base ) ) {
+   $prefixes = $this->getPrefixes();
+   if ( $base === '' ) {
+   // Empty prefix not supported; use full 
IRI
+   return $prefixes[ $base ] . $local;
+   }
+   $this->context[ $base ] = $prefixes[ $base ];
+   }
+   return $base . ':' . $local;
+   }
+   }
+
+   /**
+* Write document header
+*/
+   private function beginJson() {
+   if ( $this->isTopLevel() ) {
+   $this->write( "{\n" );
+   $this->write( function () {
+   // If this buffer is drained

[MediaWiki-commits] [Gerrit] purtle[master]: Minor bug fixes in test suite.

2017-08-11 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/371445 )

Change subject: Minor bug fixes in test suite.
..

Minor bug fixes in test suite.

Change-Id: I05e4b25c775357414c519374574486a9977d59da
---
M tests/data/TextWithSpecialChars.rdf
M tests/phpunit/RdfWriterTestBase.php
2 files changed, 5 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/purtle refs/changes/45/371445/1

diff --git a/tests/data/TextWithSpecialChars.rdf 
b/tests/data/TextWithSpecialChars.rdf
index 7952698..b8d7364 100644
--- a/tests/data/TextWithSpecialChars.rdf
+++ b/tests/data/TextWithSpecialChars.rdf
@@ -10,3 +10,4 @@
http://www.example.org/terms/Bear;>
Bear says: Превед!

+
diff --git a/tests/phpunit/RdfWriterTestBase.php 
b/tests/phpunit/RdfWriterTestBase.php
index 7f7c527..39aaa61 100644
--- a/tests/phpunit/RdfWriterTestBase.php
+++ b/tests/phpunit/RdfWriterTestBase.php
@@ -243,7 +243,7 @@
}
 
public function testEricMiller() {
-   // exampel taken from http://www.w3.org/2007/02/turtle/primer/
+   // example taken from http://www.w3.org/2007/02/turtle/primer/
 
$writer = $this->newWriter();
 
@@ -262,7 +262,7 @@
}
 
public function testLabeledBlankNode() {
-   // exampel taken from http://www.w3.org/2007/02/turtle/primer/
+   // example taken from http://www.w3.org/2007/02/turtle/primer/
 
$writer = $this->newWriter();
 
@@ -284,7 +284,7 @@
}
 
public function testNumberedBlankNodes() {
-   // exampel taken from http://www.w3.org/2007/02/turtle/primer/
+   // example taken from http://www.w3.org/2007/02/turtle/primer/
 
$writer = $this->newWriter();
 
@@ -319,6 +319,7 @@
->text( "Cow says:\n\r 'Moo! \\Moo!'" );
$writer->about( 'exterms', 'Bear' )->say( 'exterms', 'says' )
->text( 'Bear says: Превед!' );
+   $writer->finish();
 
$rdf = $writer->drain();
$this->assertOutputLines( 'TextWithSpecialChars', $rdf );

-- 
To view, visit https://gerrit.wikimedia.org/r/371445
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I05e4b25c775357414c519374574486a9977d59da
Gerrit-PatchSet: 1
Gerrit-Project: purtle
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Update documentation for SelectiveSerializer.

2017-08-12 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/371604 )

Change subject: Update documentation for SelectiveSerializer.
..

Update documentation for SelectiveSerializer.

Change-Id: I557218fd91012ce710c20d4127bca8bb16c8a48c
---
M lib/html2wt/SelectiveSerializer.js
1 file changed, 4 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/04/371604/1

diff --git a/lib/html2wt/SelectiveSerializer.js 
b/lib/html2wt/SelectiveSerializer.js
index b876263..973268a 100644
--- a/lib/html2wt/SelectiveSerializer.js
+++ b/lib/html2wt/SelectiveSerializer.js
@@ -1,8 +1,8 @@
 /*
- * This is a Serializer class that will run through a DOM looking for special
- * change markers, usually supplied by an HTML5 WYSIWYG editor (like the
- * VisualEditor for MediaWiki), and determining what needs to be
- * serialized and what can simply be copied over.
+ * This is a Serializer class that will compare two versions of a DOM
+ * and re-use the original wikitext for unmodified regions of the DOM.
+ * Originally this relied on special change markers inserted by the
+ * editor, but we now generate these ourselves using DOMDiff.
  */
 
 'use strict';

-- 
To view, visit https://gerrit.wikimedia.org/r/371604
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I557218fd91012ce710c20d4127bca8bb16c8a48c
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] VisualEditor/VisualEditor[master]: Fix infinite loop in ve.BranchNode#getNodeFromOffset

2017-07-06 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/363743 )

Change subject: Fix infinite loop in ve.BranchNode#getNodeFromOffset
..

Fix infinite loop in ve.BranchNode#getNodeFromOffset

This is a minor clean up of the refactoring done in
1cbcc2f0bf66fd824c255a7e6ca1e87fdacd46e9.

If the document is empty (consisting only of ve.ce.InternalListNode)
then it is possible for `currentNode.children.length > 0` and
`currentNode.children[ 0 ] instanceof ve.ce.InternalListNode`.
This sets up an infinite loop, unless `offset === nodeOffset`.

This was being triggered when an empty document consisting only of
`, , , ` was
p-unwrapped; for example, prior to paste in
ve.ui.MWWikitextStringTransferHandler.js or prior to update of a
`-{}-` construct by ve.ui.MWLanguageVariantInspector.

Change-Id: I933bf28637f3b3c235aa4b9f8d96d1d09a7b3085
---
M src/ve.BranchNode.js
M src/ve.Document.js
M tests/ce/ve.ce.Document.test.js
3 files changed, 37 insertions(+), 19 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/VisualEditor/VisualEditor 
refs/changes/43/363743/1

diff --git a/src/ve.BranchNode.js b/src/ve.BranchNode.js
index c2404db..477bb4b 100644
--- a/src/ve.BranchNode.js
+++ b/src/ve.BranchNode.js
@@ -155,14 +155,14 @@
while ( currentNode.children.length ) {
for ( i = 0, length = currentNode.children.length; i < length; 
i++ ) {
childNode = currentNode.children[ i ];
-   if ( childNode instanceof ve.ce.InternalListNode ) {
-   break;
-   }
if ( offset === nodeOffset ) {
// The requested offset is right before 
childNode, so it's not
// inside any of currentNode's children, but is 
inside currentNode
return currentNode;
}
+   if ( childNode instanceof ve.ce.InternalListNode ) {
+   break SIBLINGS;
+   }
nodeLength = childNode.getOuterLength();
if ( offset >= nodeOffset && offset < nodeOffset + 
nodeLength ) {
if ( !shallow && childNode.hasChildren() && 
childNode.getChildren().length ) {
diff --git a/src/ve.Document.js b/src/ve.Document.js
index 58d7218..5e909c4 100644
--- a/src/ve.Document.js
+++ b/src/ve.Document.js
@@ -39,7 +39,7 @@
 };
 
 /**
- * Get a node a an offset.
+ * Get a node at an offset.
  *
  * @method
  * @param {number} offset Offset to get node at
diff --git a/tests/ce/ve.ce.Document.test.js b/tests/ce/ve.ce.Document.test.js
index 4984c98..c1eebc8 100644
--- a/tests/ce/ve.ce.Document.test.js
+++ b/tests/ce/ve.ce.Document.test.js
@@ -35,7 +35,7 @@
 // TODO: getDirectionFromRange
 
 QUnit.test( 'getNodeAndOffset', function ( assert ) {
-   var tests, i, iLen, test, parts, view, data, ceDoc, rootNode, 
offsetCount, offset, j, jLen, node;
+   var tests, i, iLen, test, parts, view, data, dmDoc, ceDoc, rootNode, 
offsetCount, offset, j, jLen, node, ex;
 
// Each test below has the following:
// html: an input document
@@ -63,6 +63,14 @@
html: 'x',
data: [ '', '', 'x', '', 
'' ],
positions: "||<#text>|x||"
+   },
+   {
+   title: 'Empty document',
+   html: '',
+   unwrap: [ { type: 'paragraph' } ],
+   data: [],
+   positions: "",
+   dies: [ 1 ]
},
{
title: 'Slugless emptied paragraph',
@@ -121,7 +129,20 @@
test = tests[ i ];
parts = test.positions.split( /[|]/ );
view = ve.test.utils.createSurfaceViewFromHtml( test.html );
-   data = view.getModel().getDocument().data.data
+   dmDoc = view.getModel().getDocument();
+   if ( test.unwrap ) {
+   new ve.dm.Surface( dmDoc ).change(
+   ve.dm.TransactionBuilder.static.newFromWrap(
+   dmDoc,
+   new ve.Range( 0, 
dmDoc.data.countNonInternalElements() ),
+   [],
+   [],
+   test.unwrap,
+   []
+   )
+   );
+   }
+   data = dmDoc.data.data
.slice( 0, -2 )
.map( showModelItem );
ceDoc = view.documentView;
@@ -147,19 +168,6 @@
}
 
for ( offset = 0; offset

[MediaWiki-commits] [Gerrit] VisualEditor/VisualEditor[master]: Doc comment improvement to ve.ui.LanguageInputWidget.

2017-07-25 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/367683 )

Change subject: Doc comment improvement to ve.ui.LanguageInputWidget.
..

Doc comment improvement to ve.ui.LanguageInputWidget.

Change-Id: Id02984fec35b44e1d8188b97b063bbad67196036
---
M src/ui/widgets/ve.ui.LanguageInputWidget.js
1 file changed, 5 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/VisualEditor/VisualEditor 
refs/changes/83/367683/1

diff --git a/src/ui/widgets/ve.ui.LanguageInputWidget.js 
b/src/ui/widgets/ve.ui.LanguageInputWidget.js
index 8710502..393d1c6 100644
--- a/src/ui/widgets/ve.ui.LanguageInputWidget.js
+++ b/src/ui/widgets/ve.ui.LanguageInputWidget.js
@@ -12,7 +12,11 @@
  *
  * @constructor
  * @param {Object} [config] Configuration options
- * @cfg {string} [dirInput="auto"] Set to "none" to hide the directionality 
input; set to "no-auto" to select between LTR and RTL; set to "auto" to add 
"auto" to directionality options.
+ * @cfg {string} [dirInput='auto'] How to display the directionality input. 
Options are:
+ *  - none: Directionality input is hidden.
+ *  - no-auto: Directionality input is visible and options are LTR or RTL.
+ *  - auto: Directionality input is visible and options include "auto" in
+ *addition to LTR and RTL.
  * @cfg {boolean} [hideCodeInput] Prevent user from entering a language code 
as free text
  * @cfg {ve.ui.WindowManager} [dialogManager] Window manager to launch the 
language search dialog in
  * @cfg {string[]} [availableLanguages] Available language codes to show in 
search dialog

-- 
To view, visit https://gerrit.wikimedia.org/r/367683
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Id02984fec35b44e1d8188b97b063bbad67196036
Gerrit-PatchSet: 1
Gerrit-Project: VisualEditor/VisualEditor
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...VisualEditor[master]: WIP: Dialog for editing LanguageConverter markup

2017-06-28 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/361921 )

Change subject: WIP: Dialog for editing LanguageConverter markup
..

WIP: Dialog for editing LanguageConverter markup

Bug: T49411
Change-Id: I5ce29e4bf47abf509afde0a57f64b5d1189f5185
---
M extension.json
M modules/ve-mw/ui/contextitems/ve.ui.MWLanguageVariantNodeContextItem.js
A modules/ve-mw/ui/dialogs/ve.ui.MWLanguageVariantDialog.js
3 files changed, 137 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/VisualEditor 
refs/changes/21/361921/1

diff --git a/extension.json b/extension.json
index d5ce90e..358da8d 100644
--- a/extension.json
+++ b/extension.json
@@ -1927,7 +1927,8 @@
"scripts": [

"modules/ve-mw/dm/nodes/ve.dm.MWLanguageVariantNode.js",

"modules/ve-mw/ce/nodes/ve.ce.MWLanguageVariantNode.js",
-   
"modules/ve-mw/ui/contextitems/ve.ui.MWLanguageVariantNodeContextItem.js"
+   
"modules/ve-mw/ui/contextitems/ve.ui.MWLanguageVariantNodeContextItem.js",
+   
"modules/ve-mw/ui/dialogs/ve.ui.MWLanguageVariantDialog.js"
],
"styles": [

"modules/ve-mw/ui/styles/contextitems/ve.ui.MWLanguageVariantNodeContextItem.css",
diff --git 
a/modules/ve-mw/ui/contextitems/ve.ui.MWLanguageVariantNodeContextItem.js 
b/modules/ve-mw/ui/contextitems/ve.ui.MWLanguageVariantNodeContextItem.js
index 08ec05a..e03313b 100644
--- a/modules/ve-mw/ui/contextitems/ve.ui.MWLanguageVariantNodeContextItem.js
+++ b/modules/ve-mw/ui/contextitems/ve.ui.MWLanguageVariantNodeContextItem.js
@@ -41,7 +41,7 @@
ve.dm.MWLanguageVariantHiddenNode
 ];
 
-ve.ui.MWLanguageVariantNodeContextItem.static.editable = false;
+ve.ui.MWLanguageVariantNodeContextItem.static.commandName = 
'mwLanguageVariant';
 
 /* Methods */
 
@@ -177,3 +177,10 @@
 /* Registration */
 
 ve.ui.contextItemFactory.register( ve.ui.MWLanguageVariantNodeContextItem );
+
+ve.ui.commandRegistry.register(
+   new ve.ui.Command(
+   'mwLanguageVariant', 'window', 'open',
+   { args: [ 'mwLanguageVariant' ], supportedSelections: [ 
'linear' ] }
+   )
+);
diff --git a/modules/ve-mw/ui/dialogs/ve.ui.MWLanguageVariantDialog.js 
b/modules/ve-mw/ui/dialogs/ve.ui.MWLanguageVariantDialog.js
new file mode 100644
index 000..ceea277
--- /dev/null
+++ b/modules/ve-mw/ui/dialogs/ve.ui.MWLanguageVariantDialog.js
@@ -0,0 +1,127 @@
+/*!
+ * VisualEditor user interface MWLanguageVariantDialog class.
+ *
+ * @copyright 2017 VisualEditor Team and others; see AUTHORS.txt
+ * @license The MIT License (MIT); see LICENSE.txt
+ */
+
+/**
+ * Dialog for editing MediaWiki LanguageConverter markup for language
+ * variants.
+ *
+ * @class
+ * @extends ve.ui.NodeDialog
+ *
+ * @constructor
+ * @param {Object} [config] Configuration options
+ */
+ve.ui.MWLanguageVariantDialog = function VeUiMWLanguageVariantDialog() {
+   // Parent constructor
+   ve.ui.MWLanguageVariantDialog.super.apply( this, arguments );
+
+   this.$element.addClass( 've-ui-mwLanguageVariantDialog' );
+};
+
+/* Inheritance */
+
+OO.inheritClass( ve.ui.MWLanguageVariantDialog, ve.ui.NodeDialog );
+
+/* Static properties */
+
+ve.ui.MWLanguageVariantDialog.static.name = 'mwLanguageVariant';
+
+ve.ui.MWLanguageVariantDialog.static.size = 'large';
+
+ve.ui.MWLanguageVariantDialog.static.title = OO.ui.deferMsg( 
'visualeditor-mwlanguagevariantdialog-title' );
+
+ve.ui.MWLanguageVariantDialog.static.modelClasses = [
+   ve.dm.MWLanguageVariantBlockNode,
+   ve.dm.MWLanguageVariantInlineNode,
+   ve.dm.MWLanguageVariantHiddenNode
+];
+
+/* Methods */
+
+/**
+ * @inheritdoc
+ */
+ve.ui.MWLanguageVariantDialog.prototype.initialize = function () {
+   // Parent method
+   ve.ui.MWLanguageVariantDialog.super.prototype.initialize.call( this );
+
+   /*
+   // Properties
+   this.panel = new OO.ui.PanelLayout( {
+   padded: true
+   } );
+   // Note that this overrides this.input from ve.ui.MWExtensionWindow
+   this.input = new ve.ui.MWPreTextInputWidget( {
+   multiline: true,
+   // This number doesn't really matter, it just needs to be large.
+   // The real height is enforced by #getBodyHeight and max-height 
in CSS.
+   rows: 100,
+   classes: [ 've-ui-mwExtensionWindow-input' ]
+   } );
+   this.input.connect( this, { resize: 'updateSize' } );
+
+   // Initialization
+   this.panel.$element.append( this.input.$element );
+   this.$body.append( this.panel.$element );
+   */
+   this.$element.addClass( 've-ui-mwLanguageVariantDialog' );
+};
+
+/**
+ * @inheritdoc
+ */
+/*

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: More carefully preserve whitespace for LC round-tripping.

2017-06-29 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/362333 )

Change subject: More carefully preserve whitespace for LC round-tripping.
..

More carefully preserve whitespace for LC round-tripping.

Change-Id: I3b8ac67e40eaaf59299764c754dc1dd096d40e3a
---
M lib/html2wt/LanguageVariantHandler.js
M lib/wt2html/pegTokenizer.pegjs
M lib/wt2html/tt/LanguageVariantHandler.js
M tests/parserTests.txt
4 files changed, 193 insertions(+), 76 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/33/362333/1

diff --git a/lib/html2wt/LanguageVariantHandler.js 
b/lib/html2wt/LanguageVariantHandler.js
index 5f78972..74a25dd 100644
--- a/lib/html2wt/LanguageVariantHandler.js
+++ b/lib/html2wt/LanguageVariantHandler.js
@@ -6,12 +6,30 @@
 var Util = require('../utils/Util.js').Util;
 var LanguageVariantText = require('./ConstrainedText.js').LanguageVariantText;
 
+var expandSpArray = function(a) {
+   var result = [];
+   if (Array.isArray(a)) {
+   a.forEach(function(el) {
+   if (typeof (el) === 'number') {
+   for (var i = 0; i < el; i++) {
+   result.push('');
+   }
+   } else {
+   result.push(el);
+   }
+   });
+   }
+   return result;
+};
+
 // should be called with this == instance of WikitextSerializer
 var LanguageVariantHandler = function(node) {
var state = this.state;
var dataMWV = DU.getJSONAttribute(node, 'data-mw-variant', {});
var dp = DU.getDataParsoid(node);
-   var trailingSemi = dp.ts;
+   var flSp = expandSpArray(dp.flSp);
+   var textSp = expandSpArray(dp.tSp);
+   var trailingSemi = false;
var textP;
var flags;
var originalFlags = (dp.fl || []).reduce(function(m, k, idx) {
@@ -85,20 +103,34 @@
// Helper function: combine the three parts of the -{ }- string
var combine = function(flagStr, bodyStr, useTrailingSemi) {
if (flagStr || /\|/.test(bodyStr)) { flagStr += '|'; }
-   if (useTrailingSemi) { bodyStr += ';'; }
+   if (useTrailingSemi !== false) { bodyStr += ';' + 
useTrailingSemi; }
return flagStr + bodyStr;
};
 
// Canonicalize combinations of flags.
-   var sortedFlags = function() {
-   return Array.from(flags).filter(function(f) {
+   var sortedFlags = function(flags, noFilter, protectFunc) {
+   var s = Array.from(flags).filter(function(f) {
// Filter out internal-use-only flags
+   if (noFilter) { return true; }
return !/^[$]/.test(f);
}).sort(function(a, b) {
var ai = originalFlags.has(a) ? originalFlags.get(a) : 
-1;
var bi = originalFlags.has(b) ? originalFlags.get(b) : 
-1;
return ai - bi;
+   }).map(function(f) {
+   // Reinsert the original whitespace around the flag (if 
any)
+   var i = originalFlags.get(f);
+   var p = protectFunc ? protectFunc(f) : f;
+   if (i !== undefined && (2 * i + 1) < flSp.length) {
+   return flSp[2 * i] + p + flSp[2 * i + 1];
+   }
+   return p;
}).join(';');
+   if (2 * originalFlags.size + 1 === flSp.length) {
+   if (flSp.length > 1 || s.length) { s += ';'; }
+   s += flSp[2 * originalFlags.size];
+   }
+   return s;
};
 
if (dataMWV.filter && dataMWV.filter.l) {
@@ -107,7 +139,7 @@
resultP = textP.then(function(text) {
console.assert(flags.size === 0);
return combine(
-   dataMWV.filter.l.map(protectLang).join(';'),
+   sortedFlags(dataMWV.filter.l, true, 
protectLang),
text,
false /* no trailing semi */);
});
@@ -118,38 +150,51 @@
if (!/[:;|]/.test(text)) {
maybeDeleteFlag('R');
}
-   return combine(sortedFlags(), text, false);
+   return combine(sortedFlags(flags), text, false);
});
} else if (Array.isArray(dataMWV.bidir)) {
// Bidirectional rules
+   if (textSp.length % 3 === 1) {
+   trailingSemi = textSp[textSp.length - 1];
+   }
var b = (dataMWV.bidir[0] &&

[MediaWiki-commits] [Gerrit] mediawiki...VisualEditor[master]: WIP: use Dialog for LanguageConverter instead of Inspector

2017-06-30 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/362464 )

Change subject: WIP: use Dialog for LanguageConverter instead of Inspector
..

WIP: use Dialog for LanguageConverter instead of Inspector

Change-Id: Ida377d54c75dfe406a2a1d04465becc1fd94a65b
---
M extension.json
A modules/ve-mw/ui/dialogs/ve.ui.MWLanguageVariantDialog.js
2 files changed, 238 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/VisualEditor 
refs/changes/64/362464/1

diff --git a/extension.json b/extension.json
index d5ce90e..358da8d 100644
--- a/extension.json
+++ b/extension.json
@@ -1927,7 +1927,8 @@
"scripts": [

"modules/ve-mw/dm/nodes/ve.dm.MWLanguageVariantNode.js",

"modules/ve-mw/ce/nodes/ve.ce.MWLanguageVariantNode.js",
-   
"modules/ve-mw/ui/contextitems/ve.ui.MWLanguageVariantNodeContextItem.js"
+   
"modules/ve-mw/ui/contextitems/ve.ui.MWLanguageVariantNodeContextItem.js",
+   
"modules/ve-mw/ui/dialogs/ve.ui.MWLanguageVariantDialog.js"
],
"styles": [

"modules/ve-mw/ui/styles/contextitems/ve.ui.MWLanguageVariantNodeContextItem.css",
diff --git a/modules/ve-mw/ui/dialogs/ve.ui.MWLanguageVariantDialog.js 
b/modules/ve-mw/ui/dialogs/ve.ui.MWLanguageVariantDialog.js
new file mode 100644
index 000..f099c8e
--- /dev/null
+++ b/modules/ve-mw/ui/dialogs/ve.ui.MWLanguageVariantDialog.js
@@ -0,0 +1,236 @@
+/*!
+ * VisualEditor user interface MWLanguageVariantDialog class.
+ *
+ * @copyright 2017 VisualEditor Team and others; see AUTHORS.txt
+ * @license The MIT License (MIT); see LICENSE.txt
+ */
+
+/**
+ * Dialog for editing MediaWiki LanguageConverter markup for language
+ * variants.
+ *
+ * @class
+ * @extends ve.ui.NodeDialog
+ *
+ * @constructor
+ * @param {Object} [config] Configuration options
+ */
+ve.ui.MWLanguageVariantDialog = function VeUiMWLanguageVariantDialog() {
+   // Parent constructor
+   ve.ui.MWLanguageVariantDialog.super.apply( this, arguments );
+
+   this.$element.addClass( 've-ui-mwLanguageVariantDialog' );
+};
+
+/* Inheritance */
+
+OO.inheritClass( ve.ui.MWLanguageVariantDialog, ve.ui.NodeDialog );
+
+/* Static properties */
+
+ve.ui.MWLanguageVariantDialog.static.name = 'mwLanguageVariant-disabled';
+
+ve.ui.MWLanguageVariantDialog.static.size = 'large';
+
+ve.ui.MWLanguageVariantDialog.static.title = OO.ui.deferMsg( 
'visualeditor-mwlanguagevariantdialog-title' );
+
+ve.ui.MWLanguageVariantDialog.static.modelClasses = [
+   ve.dm.MWLanguageVariantBlockNode,
+   ve.dm.MWLanguageVariantInlineNode,
+   ve.dm.MWLanguageVariantHiddenNode
+];
+
+/* Methods */
+
+/**
+ * @inheritdoc
+ */
+ve.ui.MWLanguageVariantDialog.prototype.initialize = function () {
+   // Parent method
+   ve.ui.MWLanguageVariantDialog.super.prototype.initialize.call( this );
+
+   // Properties
+   this.panel = new OO.ui.PanelLayout( {
+   padded: true
+   } );
+   // Note that this overrides this.input from ve.ui.MWExtensionWindow
+   this.textTarget = ve.init.target.createTargetWidget( {
+   tools: ve.init.target.constructor.static.toolbarGroups,
+   includeCommands: this.constructor.static.includeCommands,
+   excludeCommands: this.constructor.static.excludeCommands,
+   importRules: this.constructor.static.getImportRules(),
+   inDialog: this.constructor.static.name,
+   placeholder: 'XXX Add some text here XXX'
+   } );
+   this.textTarget.connect( this, { resize: 'updateSize' } );
+
+   // Initialization
+   this.panel.$element.append( this.textTarget.$element );
+   this.$body.append( this.panel.$element );
+
+   this.$element.addClass( 've-ui-mwLanguageVariantDialog' );
+};
+
+ve.ui.MWLanguageVariantDialog.static.includeCommands = null;
+ve.ui.MWLanguageVariantDialog.static.excludeCommands = [
+   // No formatting
+   'paragraph',
+   'heading1',
+   'heading2',
+   'heading3',
+   'heading4',
+   'heading5',
+   'heading6',
+   'preformatted',
+   'blockquote',
+   // TODO: Decide if tables tools should be allowed
+   'tableCellHeader',
+   'tableCellData',
+   // No structure
+   'bullet',
+   'bulletWrapOnce',
+   'number',
+   'numberWrapOnce',
+   'indent',
+   'outdent'
+];
+
+ve.ui.MWLanguageVariantDialog.static.getImportRules = function () {
+   return ve.extendObject(
+   ve.copy( ve.init.target.constructor.static.importRules ),
+   {
+   }
+   );
+};
+
+/**
+ * @inheritdoc
+ */
+ve.ui.MWLanguageVariantDialog.prototype.getReadyProcess

[MediaWiki-commits] [Gerrit] VisualEditor/VisualEditor[master]: LanguageInputWidget: optionally suppress directionality input

2017-07-05 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/363490 )

Change subject: LanguageInputWidget: optionally suppress directionality input
..

LanguageInputWidget: optionally suppress directionality input

This allows use for language selection where directionality isn't
relevant, for example in LanguageConverter markup.

Change-Id: If53084fb1c5f9b71435fd6edd1aa9a547689dd60
---
M src/ui/widgets/ve.ui.LanguageInputWidget.js
1 file changed, 5 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/VisualEditor/VisualEditor 
refs/changes/90/363490/1

diff --git a/src/ui/widgets/ve.ui.LanguageInputWidget.js 
b/src/ui/widgets/ve.ui.LanguageInputWidget.js
index 8b1725b..e17cba3 100644
--- a/src/ui/widgets/ve.ui.LanguageInputWidget.js
+++ b/src/ui/widgets/ve.ui.LanguageInputWidget.js
@@ -16,6 +16,7 @@
  * @cfg {boolean} [hideCodeInput] Prevent user from entering a language code 
as free text
  * @cfg {ve.ui.WindowManager} [dialogManager] Window manager to launch the 
language search dialog in
  * @cfg {string[]} [availableLanguages] Available language codes to show in 
search dialog
+ * @cfg {boolean} [hideDir] Don't display directionality widget
  */
 ve.ui.LanguageInputWidget = function VeUiLanguageInputWidget( config ) {
var dirItems;
@@ -98,10 +99,10 @@
 
this.$element
.addClass( 've-ui-languageInputWidget' )
-   .append(
-   this.languageLayout.$element,
-   this.directionField.$element
-   );
+   .append( this.languageLayout.$element );
+   if ( !config.hideDir ) {
+   this.$element.append( this.directionField.$element );
+   }
 };
 
 /* Inheritance */

-- 
To view, visit https://gerrit.wikimedia.org/r/363490
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If53084fb1c5f9b71435fd6edd1aa9a547689dd60
Gerrit-PatchSet: 1
Gerrit-Project: VisualEditor/VisualEditor
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: tests: Use TestingAccessWrapper to reload LanguageConverter ...

2017-06-27 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/361784 )

Change subject: tests: Use TestingAccessWrapper to reload LanguageConverter 
tables
..

tests: Use TestingAccessWrapper to reload LanguageConverter tables

Make the LanguageConverter::reloadTables method actually private,
and use the TestingAccessWrapper to call it when running parser tests.

Follow-up to I65736520cd04bfe8949b29ade07338a6e1b88a4d.

Change-Id: I43b81b8fef6441ad50b858ff7757732ecb5eef91
---
M languages/FakeConverter.php
M languages/LanguageConverter.php
M tests/parser/ParserTestRunner.php
3 files changed, 14 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/84/361784/1

diff --git a/languages/FakeConverter.php b/languages/FakeConverter.php
index 0bf88c6..22377c2 100644
--- a/languages/FakeConverter.php
+++ b/languages/FakeConverter.php
@@ -126,6 +126,11 @@
public function updateConversionTable( Title $title ) {
}
 
-   public function reloadTables() {
+   /**
+* Used by test suites which need to reset the converter state.
+*
+* @private
+*/
+   private function reloadTables() {
}
 }
diff --git a/languages/LanguageConverter.php b/languages/LanguageConverter.php
index ef88c12..2137786 100644
--- a/languages/LanguageConverter.php
+++ b/languages/LanguageConverter.php
@@ -891,11 +891,11 @@
/**
 * Reload the conversion tables.
 *
-* Public only for use by test suites which need to reset the converter
-* state.
+* Also used by test suites which need to reset the converter state.
 *
+* @private
 */
-   function reloadTables() {
+   private function reloadTables() {
if ( $this->mTables ) {
unset( $this->mTables );
}
diff --git a/tests/parser/ParserTestRunner.php 
b/tests/parser/ParserTestRunner.php
index 26c5586..5aba371 100644
--- a/tests/parser/ParserTestRunner.php
+++ b/tests/parser/ParserTestRunner.php
@@ -28,6 +28,7 @@
 use Wikimedia\Rdbms\IDatabase;
 use MediaWiki\MediaWikiServices;
 use Wikimedia\ScopedCallback;
+use Wikimedia\TestingAccessWrapper;
 
 /**
  * @ingroup Testing
@@ -1046,7 +1047,10 @@
$context->setLanguage( $lang );
$teardown[] = function () use ( $context ) {
// Clear language conversion tables
-   $context->getLanguage()->getConverter()->reloadTables();
+   $wrapper = TestingAccessWrapper::newFromObject(
+   $context->getLanguage()->getConverter()
+   );
+   $wrapper->reloadTables();
// Reset context to the restored globals
$context->setUser( $GLOBALS['wgUser'] );
$context->setLanguage( $GLOBALS['wgContLang'] );

-- 
To view, visit https://gerrit.wikimedia.org/r/361784
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I43b81b8fef6441ad50b858ff7757732ecb5eef91
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Fix typo in comment.

2017-06-27 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/361785 )

Change subject: Fix typo in comment.
..

Fix typo in comment.

Change-Id: Ia656c55ca22fa30b5a911b8cb3a6218c916d3316
---
M languages/ConverterRule.php
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/85/361785/1

diff --git a/languages/ConverterRule.php b/languages/ConverterRule.php
index 0d0d90d..e51a8ed 100644
--- a/languages/ConverterRule.php
+++ b/languages/ConverterRule.php
@@ -230,7 +230,7 @@
if ( $disp === false && array_key_exists( $variant, 
$unidtable ) ) {
$disp = array_values( $unidtable[$variant] )[0];
}
-   // or display frist text under disable manual convert
+   // or display first text under disable manual convert
if ( $disp === false && 
$this->mConverter->mManualLevel[$variant] == 'disable' ) {
if ( count( $bidtable ) > 0 ) {
$disp = array_values( $bidtable )[0];

-- 
To view, visit https://gerrit.wikimedia.org/r/361785
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia656c55ca22fa30b5a911b8cb3a6218c916d3316
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: bin/parserTests.js: Return non-zero exitcode when results ar...

2017-04-24 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/350023 )

Change subject: bin/parserTests.js: Return non-zero exitcode when results are 
unexpected.
..

bin/parserTests.js: Return non-zero exitcode when results are unexpected.

This is a regression introduced by 2a1e26536b0610a630a34a41d8c64e2d1513942f.

Change-Id: I108aef54662b7f6a5bba34a00c32f2892e5f6b00
---
M bin/parserTests.js
1 file changed, 4 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/23/350023/1

diff --git a/bin/parserTests.js b/bin/parserTests.js
index 611597c..34b1f09 100755
--- a/bin/parserTests.js
+++ b/bin/parserTests.js
@@ -1516,9 +1516,12 @@
Object.keys(stats).forEach(function(k) {
stats[k] += result.stats[k]; // Sum all stats
});
-   return [ runner, exitCode || result.exitCode ];
+   return exitCode || result.exitCode;
});
}, 0)
+   .then(function(exitCode) {
+   return [ runner, exitCode ];
+   })
.tap(function() {
options.reportSummary([], stats, null, stats.loggedErrorCount, 
null);
});

-- 
To view, visit https://gerrit.wikimedia.org/r/350023
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I108aef54662b7f6a5bba34a00c32f2892e5f6b00
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Fix self-closed HTML tag test.

2017-04-28 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/350901 )

Change subject: Fix self-closed HTML tag test.
..

Fix self-closed HTML tag test.

The "non-tidy" version of this behavior was deprecated in
Ie1cf3aa40d5483bf395ece539f0240b694ff04ab.

Bug: T7487
Bug: T134423
Change-Id: Ie29034988ad804ce7ab05506b56baf71fc907d72
---
M tests/parserTests-blacklist.js
M tests/parserTests.txt
2 files changed, 4 insertions(+), 7 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/01/350901/1

diff --git a/tests/parserTests-blacklist.js b/tests/parserTests-blacklist.js
index 883dcab..857cb49 100644
--- a/tests/parserTests-blacklist.js
+++ b/tests/parserTests-blacklist.js
@@ -158,7 +158,6 @@
 add("wt2html", "Sanitizer: Closing of closed but not open table tags", "Table 
not started");
 add("wt2html", "Sanitizer: Validating that  and  work, but only 
for Microdata", "\n\tmeta itemprop=\"hello\" 
content=\"world\">\n\tmeta http-equiv=\"refresh\" 
content=\"5\">\n\tmeta itemprop=\"hello\" http-equiv=\"refresh\" 
content=\"5\">\n\tlink itemprop=\"hello\" href=\"{{SERVER}}\">\n\tlink 
rel=\"stylesheet\" href=\"{{SERVER}}\">\n\tlink rel=\"stylesheet\" 
itemprop=\"hello\" href=\"{{SERVER}}\">\n");
 add("wt2html", "Language converter: output gets cut off unexpectedly (T7757)", 
"this bit is safe: }-\n\nbut if we add a conversion instance: 
-{zh-cn:xxx;zh-tw:yyy}-\n\nthen we 
get cut off here: }-\n\nall 
additional text is vanished");
-add("wt2html", "Self closed html pairs (T7487)", "Centered
 text\nIn
 div text");
 add("wt2html", "Fuzz testing: Parser14", " onmouseover= \nhttp://__TOC__\; 
data-parsoid='{\"stx\":\"url\",\"dsr\":[19,33,0,0]}'>http://__TOC__");
 add("wt2html", "Inline HTML vs wiki block nesting", "Bold
 paragraph\n\nNew wiki paragraph");
 add("wt2html", "Special page transclusion", "Parser
 function implementation for pf_special missing in Parsoid.");
@@ -979,7 +978,7 @@
 add("html2wt", "Sanitizer: Closing of open but not closed tags", 
"foo\n");
 add("html2wt", "Sanitizer: Validating that  and  work, but only 
for Microdata", "\n\t\n\t\n\t\n\n\t[http://example.org]\n\thttp://example.org\;>\n\t[http://example.org]\n\n");
 add("html2wt", "Language converter: output gets cut off unexpectedly (T7757)", 
"this bit is safe: }-\n\nbut if we add a conversion instance: xxx\n\nthen we 
get cut off here: }-\n\nall additional text is vanished\n");
-add("html2wt", "Self closed html pairs (T7487)", "Centered text\nIn div text\n");
+add("html2wt", "Self closed html pairs (T7487)", "Centered text\nIn div 
text");
 add("html2wt", "Punctuation: nbsp before exclamation", "C'est grave !\n");
 add("html2wt", "HTML nested bullet list, open tags (T7497)", "* One\n* 
Two:\n** Sub-one\n** Sub-two\n");
 add("html2wt", "HTML nested ordered list, closed tags (T7497)", "# One\n# 
Two:\n## Sub-one\n## Sub-two\n");
diff --git a/tests/parserTests.txt b/tests/parserTests.txt
index e595841..13fce4e 100644
--- a/tests/parserTests.txt
+++ b/tests/parserTests.txt
@@ -17498,14 +17498,12 @@
 
 !! test
 Self closed html pairs (T7487)
-!! options
 !! wikitext
 Centered text
 In div text
-!! html
-font id="bug" /Centered text
-font id="bug2" /In div text
-
+!! html+tidy
+Centered text
+In div text
 !! end
 
 #

-- 
To view, visit https://gerrit.wikimedia.org/r/350901
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie29034988ad804ce7ab05506b56baf71fc907d72
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Preprocessor precedence: rightmost-opening/"broken-link"/"br...

2017-04-28 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/350867 )

Change subject: Preprocessor precedence: 
rightmost-opening/"broken-link"/"broken-template"
..

Preprocessor precedence: rightmost-opening/"broken-link"/"broken-template"

Match the behavior of the PHP preprocessor for links or templates missing
their close brackets or misnested, by using a shared stack for brackets in
inlineBreaks.  We only stop for the close-symbol of the rightmost opened
construct; see [[:mw:Preprocessor_ABNF]] for details.  So:
  [[Foo|bar}}]]
and
  [[Foo}}|bar]]
are both valid, even if there's an template opening somewhere to the left.
Note however that the latter example will not become a link in the output
because }} is not a valid title character -- it is unlinked at a later
stage.  Similarly:
  [[Foo|{{echo|]]}}
is a broken link, because the square brackets are not inline breaks inside
the template.

Change-Id: I8a2dd4471383b389dc17fefb7917305aefcb3141
---
M lib/wt2html/pegTokenizer.pegjs
M lib/wt2html/tokenizer.utils.js
M tests/parserTests-blacklist.js
M tests/parserTests.txt
4 files changed, 149 insertions(+), 46 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/67/350867/1

diff --git a/lib/wt2html/pegTokenizer.pegjs b/lib/wt2html/pegTokenizer.pegjs
index 473f70b..3ebf28d 100644
--- a/lib/wt2html/pegTokenizer.pegjs
+++ b/lib/wt2html/pegTokenizer.pegjs
@@ -780,33 +780,51 @@
  * 5: {·} → {{·{{{·}}}·}}
  * 6: {{·}} → {{{·{{{·}}}·}}}
  * 7: {{{·}}} → {·{{{·{{{·}}}·}}}·}
+ * This is only if close has > 3 braces; otherwise we just match open
+ * and close as we find them.
  */
 tplarg_or_template
-= & {
+  = &'{{' &{
   // Refuse to recurse beyond 40 levels. Default in the PHP parser
   // is $wgMaxTemplateDepth = 40; This is to prevent crashing from
   // buggy wikitext with lots of unclosed template calls, as in
   // eswiki/Usuario:C%C3%A1rdenas/PRUEBAS?oldid=651094
   if (stops.onCount('templatedepth') === undefined ||
   stops.onCount('templatedepth') < 40) {
-return stops.inc('templatedepth');
+return true;
   } else {
 return false;
   }
-}
-r:( &('{{' &('{{{'+ !'{') tplarg) a:template { return a; }
+} t:tplarg_or_template_guarded { return t; }
+
+tplarg_or_template_guarded
+  = &{ return stops.inc('templatedepth'); }
+r:( &('{{' &('{{{'+ !'{') tplarg) a:(template/broken_template) { return a; 
}
   / a:$('{' &('{{{'+ !'{'))? b:tplarg { return [a].concat(b); }
   / a:$('{' &('{{' !'{'))? b:template { return [a].concat(b); }
 ) {
   stops.dec('templatedepth');
   return r;
 }
+// deliberately fail to decrement templatedepth for a broken template
+/ a:broken_template { return a; }
 / & { return stops.dec('templatedepth'); }
 
 tplarg_or_template_or_bust "tplarg_or_template_or_bust"
 = r:(tplarg_or_template / .)+ { return tu.flattenIfArray(r); }
 
 template
+  = &{ return stops.push("preproc", /*{{*/"}}"); }
+t:template_preproc
+{ stops.pop("preproc"); return t; }
+  / &{ return stops.pop("preproc"); }
+
+broken_template
+  = &{ return stops.push("preproc", "broken"); }
+// for broken-template,  deliberately fail to pop the preproc stops stack
+t:"{{" { return t; }
+
+template_preproc
   = "{{" nl_comment_space*
 target:template_param_value
 params:(nl_comment_space* "|"
@@ -829,6 +847,12 @@
 } / $('{{' space_or_newline+ '}}')
 
 tplarg
+  = &{ return stops.push("preproc", /*{{{*/"}}}"); }
+t:tplarg_preproc
+{ stops.pop("preproc"); return t; }
+  / &{ return stops.pop("preproc"); }
+
+tplarg_preproc
   = "{{{"
 p:("" { return endOffset(); })
 target:template_param_value?
@@ -945,8 +969,18 @@
 }
 }
 
-// TODO: handle link prefixes as in al[[Razi]]
 wikilink
+  = &{ return stops.push("preproc", "]]"); }
+w:wikilink_preproc
+{ stops.pop("preproc"); return w; }
+  // `broken-link` (see [[:mw:Preprocessor_ABNF]]), but careful because the
+  // second bracket could start an extlink.  Deliberately leave entry
+  // on preproc stack since we haven't seen a double-close bracket.
+  / &"[[" &{ return stops.replace("preproc", "broken"); }
+a:"[" b:(extlink / "[") { return [a,b]; }
+  / &{ return stops.pop("preproc"); }
+
+wikilink_preproc
   = "[["
 ! url
 //target:link_target
@@ -960,12 +994,25 @@
   lcontent = { content: [] };
   }
 
+  var textTokens = [];
   if (target === null || lcontent.pipetrick) {
-return [text()];
+textTokens.push("[[");
+if (target) {
+  textTokens.push(target);
+}
+if (lcontent.pipetrick) {
+  textTokens.push("|");
+}
+lcontent.content.forEach(function(a) {
+  // a is a mw:maybeContent attribute
+  textTokens.push("|");
+

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Rename bidir/unidir => twoway/oneway in LC markup.

2017-08-09 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/370833 )

Change subject: Rename bidir/unidir => twoway/oneway in LC markup.
..

Rename bidir/unidir => twoway/oneway in LC markup.

Depends-On: I7efb35245d48125b167dc0f0ef8f12aa0fff94e5
Change-Id: I44c3a63670e831cd3e69e29d0cc10bba55758082
---
M lib/html2wt/LanguageVariantHandler.js
M lib/wt2html/pegTokenizer.pegjs
M lib/wt2html/tt/LanguageVariantHandler.js
M tests/parserTests.txt
4 files changed, 93 insertions(+), 92 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/33/370833/1

diff --git a/lib/html2wt/LanguageVariantHandler.js 
b/lib/html2wt/LanguageVariantHandler.js
index 8644926..c9b652b 100644
--- a/lib/html2wt/LanguageVariantHandler.js
+++ b/lib/html2wt/LanguageVariantHandler.js
@@ -38,14 +38,14 @@
}, new Map());
var resultP = Promise.resolve('$E|'); // "error" flag
 
-   // Migration: `twoway` => `bidir` ; `oneway` => `unidir`
-   if (dataMWV.twoway) {
-   dataMWV.bidir = dataMWV.twoway;
-   delete dataMWV.twoway;
+   // Backwards-compatibility: `bidir` => `twoway` ; `unidir` => `oneway`
+   if (dataMWV.bidir) {
+   dataMWV.twoway = dataMWV.bidir;
+   delete dataMWV.bidir;
}
-   if (dataMWV.oneway) {
-   dataMWV.unidir = dataMWV.oneway;
-   delete dataMWV.oneway;
+   if (dataMWV.unidir) {
+   dataMWV.oneway = dataMWV.undir;
+   delete dataMWV.unidir;
}
 
flags = Object.keys(dataMWV).reduce(function(f, k) {
@@ -162,14 +162,14 @@
}
return combine(sortedFlags(flags), text, false);
});
-   } else if (Array.isArray(dataMWV.bidir)) {
-   // Bidirectional rules
+   } else if (Array.isArray(dataMWV.twoway)) {
+   // Two-way rules (most common)
if (textSp.length % 3 === 1) {
trailingSemi = textSp[textSp.length - 1];
}
-   var b = (dataMWV.bidir[0] && dataMWV.bidir[0].l === '*') ?
-   dataMWV.bidir.slice(0, 1) :
-   dataMWV.bidir;
+   var b = (dataMWV.twoway[0] && dataMWV.twoway[0].l === '*') ?
+   dataMWV.twoway.slice(0, 1) :
+   dataMWV.twoway;
textP = Promise.all(b.map(function(rule, idx) {
return ser(rule.t, { protect: /;|\}-/ 
}).then(function(text) {
if (rule.l === '*') {
@@ -187,11 +187,12 @@
maybeDeleteFlag('$S');
return combine(sortedFlags(flags), text, trailingSemi);
});
-   } else if (Array.isArray(dataMWV.unidir)) {
+   } else if (Array.isArray(dataMWV.oneway)) {
+   // One-way rules (uncommon)
if (textSp.length % 4 === 1) {
trailingSemi = textSp[textSp.length - 1];
}
-   textP = Promise.all(dataMWV.unidir.map(function(rule, idx) {
+   textP = Promise.all(dataMWV.oneway.map(function(rule, idx) {
return Promise.all([
ser(rule.f, { protect: /:|;|=>|\}-/ }),
ser(rule.t, { protect: /;|\}-/ })
diff --git a/lib/wt2html/pegTokenizer.pegjs b/lib/wt2html/pegTokenizer.pegjs
index 476598b..8dad166 100644
--- a/lib/wt2html/pegTokenizer.pegjs
+++ b/lib/wt2html/pegTokenizer.pegjs
@@ -1185,7 +1185,7 @@
 lvtext:(lang_variant_nowiki / lang_variant_text_no_semi)
 {
   return {
-bidir: true,
+twoway: true,
 lang: lang,
 text: lvtext,
 sp: [sp1.join(''), sp2.join(''), sp3.join('')]
@@ -1200,7 +1200,7 @@
 to:(lang_variant_nowiki / lang_variant_text_no_semi)
 {
   return {
-unidir: true,
+oneway: true,
 from: from,
 lang: lang,
 to: to,
diff --git a/lib/wt2html/tt/LanguageVariantHandler.js 
b/lib/wt2html/tt/LanguageVariantHandler.js
index 1d5c040..cb9c7fe 100644
--- a/lib/wt2html/tt/LanguageVariantHandler.js
+++ b/lib/wt2html/tt/LanguageVariantHandler.js
@@ -103,10 +103,10 @@
// convert all variant texts to DOM
var isBlock = false;
return Promise.map(dataAttribs.texts, function(t) {
-   if (t.bidir) {
+   if (t.twoway) {
return convertOne(t.text).then(function(text) {
isBlock = isBlock || text.isBlock;
-   return { lang: t.lang, text: text.xmlstr, 
bidir: true, sp: t.sp };
+   return { lang: t.lang, text: text.xmlstr, 
twoway: true, sp: t.sp };
});
} else if (t.lang) {

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Support gradual migration to oneway/twoway in LC markup.

2017-08-09 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/370812 )

Change subject: Support gradual migration to oneway/twoway in LC markup.
..

Support gradual migration to oneway/twoway in LC markup.

Change-Id: Ib59dff22e64f235e30778a5a5b3e525e4fc7fdd3
---
M lib/html2wt/LanguageVariantHandler.js
1 file changed, 10 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/12/370812/1

diff --git a/lib/html2wt/LanguageVariantHandler.js 
b/lib/html2wt/LanguageVariantHandler.js
index 74a25dd..8644926 100644
--- a/lib/html2wt/LanguageVariantHandler.js
+++ b/lib/html2wt/LanguageVariantHandler.js
@@ -38,6 +38,16 @@
}, new Map());
var resultP = Promise.resolve('$E|'); // "error" flag
 
+   // Migration: `twoway` => `bidir` ; `oneway` => `unidir`
+   if (dataMWV.twoway) {
+   dataMWV.bidir = dataMWV.twoway;
+   delete dataMWV.twoway;
+   }
+   if (dataMWV.oneway) {
+   dataMWV.unidir = dataMWV.oneway;
+   delete dataMWV.oneway;
+   }
+
flags = Object.keys(dataMWV).reduce(function(f, k) {
if (Consts.LCNameMap.has(k)) {
f.add(Consts.LCNameMap.get(k));

-- 
To view, visit https://gerrit.wikimedia.org/r/370812
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib59dff22e64f235e30778a5a5b3e525e4fc7fdd3
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...VisualEditor[master]: Rename LanguageConverter markup fields

2017-08-09 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/370816 )

Change subject: Rename LanguageConverter markup fields
..

Rename LanguageConverter markup fields

Avoid confusion by using `twoway` instead of `bidir`, which could be
confused with the Unicode bidirectional layout algorithm.

Change-Id: I7efb35245d48125b167dc0f0ef8f12aa0fff94e5
---
M extension.json
M modules/ve-mw/ce/nodes/ve.ce.MWLanguageVariantNode.js
M modules/ve-mw/dm/nodes/ve.dm.MWLanguageVariantNode.js
M modules/ve-mw/i18n/en.json
M modules/ve-mw/i18n/qqq.json
5 files changed, 48 insertions(+), 23 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/VisualEditor 
refs/changes/16/370816/1

diff --git a/extension.json b/extension.json
index cb88de1..b81f5b6 100644
--- a/extension.json
+++ b/extension.json
@@ -1936,11 +1936,11 @@
],
"messages": [
"visualeditor-changedesc-mwlanguagevariant",
-   "visualeditor-mwlanguagevariant-bidir",
+   "visualeditor-mwlanguagevariant-twoway",
"visualeditor-mwlanguagevariant-disabled",
"visualeditor-mwlanguagevariant-filter",
"visualeditor-mwlanguagevariant-name",
-   "visualeditor-mwlanguagevariant-unidir",
+   "visualeditor-mwlanguagevariant-oneway",
"visualeditor-mwlanguagevariant-unknown"
],
"targets": [
diff --git a/modules/ve-mw/ce/nodes/ve.ce.MWLanguageVariantNode.js 
b/modules/ve-mw/ce/nodes/ve.ce.MWLanguageVariantNode.js
index 2aea13e..1298647 100644
--- a/modules/ve-mw/ce/nodes/ve.ce.MWLanguageVariantNode.js
+++ b/modules/ve-mw/ce/nodes/ve.ce.MWLanguageVariantNode.js
@@ -63,12 +63,12 @@
languageCodes = [ variantInfo.name.t ];
} else if ( variantInfo.filter ) {
languageCodes = variantInfo.filter.l;
-   } else if ( variantInfo.bidir ) {
-   languageCodes = variantInfo.bidir.map( function ( item ) {
+   } else if ( variantInfo.twoway ) {
+   languageCodes = variantInfo.twoway.map( function ( item ) {
return item.l;
} );
-   } else if ( variantInfo.unidir ) {
-   languageCodes = variantInfo.unidir.map( function ( item ) {
+   } else if ( variantInfo.oneway ) {
+   languageCodes = variantInfo.oneway.map( function ( item ) {
return item.l;
} );
}
diff --git a/modules/ve-mw/dm/nodes/ve.dm.MWLanguageVariantNode.js 
b/modules/ve-mw/dm/nodes/ve.dm.MWLanguageVariantNode.js
index fe7c9a7..c39a0c9 100644
--- a/modules/ve-mw/dm/nodes/ve.dm.MWLanguageVariantNode.js
+++ b/modules/ve-mw/dm/nodes/ve.dm.MWLanguageVariantNode.js
@@ -73,6 +73,27 @@
 ve.dm.MWLanguageVariantNode.static.hiddenType = 'mwLanguageVariantHidden';
 
 /**
+ * Migrate field names from old Parsoid spec to new field names.
+ * This method will go away after the next Parsoid flag day.
+ * @static
+ * @method
+ * @private
+ */
+ve.dm.MWLanguageVariantNode.static.migrateFieldNames = function ( dataMwv ) {
+   // Field name migration: `bidir`=>`twoway`; `unidir`=>`oneway`
+   // This will go away eventually.
+   if ( dataMwv.bidir ) {
+   dataMwv.twoway = dataMwv.bidir;
+   delete dataMwv.bidir;
+   }
+   if ( dataMwv.unidir ) {
+   dataMwv.oneway = dataMwv.unidir;
+   delete dataMwv.unidir;
+   }
+   return dataMwv;
+};
+
+/**
  * @inheritdoc
  */
 ve.dm.MWLanguageVariantNode.static.toDataElement = function ( domElements, 
converter ) {
@@ -81,6 +102,8 @@
firstElement = domElements[ 0 ],
dataMwvJSON = firstElement.getAttribute( 'data-mw-variant' ),
dataMwv = dataMwvJSON ? JSON.parse( dataMwvJSON ) : {};
+
+   this.migrateFieldNames( dataMwv );
 
dataElement = {
attributes: {
@@ -112,7 +135,9 @@
// Preserve exact equality of this attribute for selser.
if ( dataElement.attributes.originalVariantInfo ) {
if ( OO.compare(
-   JSON.parse( dataElement.attributes.originalVariantInfo 
),
+   this.migrateFieldNames(
+   JSON.parse( 
dataElement.attributes.originalVariantInfo )
+   ),
variantInfo
) ) {
dataMwvJSON = 
dataElement.attributes.originalVariantInfo;
@@ -206,8 +231,8 @@
return variantInfo.filter.t;
} else if ( variantInfo.describe || ( opts && opts.describeAll ) ) {
$holder = $( '' );
-

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Refactor TemplateHandler to improve commonality of native an...

2017-05-03 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/351669 )

Change subject: Refactor TemplateHandler to improve commonality of native and 
PHP expansion.
..

Refactor TemplateHandler to improve commonality of native and PHP expansion.

Change-Id: I5e2a52ef1248f06e0b69172c84cfa71c6ecb5a75
---
M lib/wt2html/tt/TemplateHandler.js
1 file changed, 41 insertions(+), 42 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/69/351669/1

diff --git a/lib/wt2html/tt/TemplateHandler.js 
b/lib/wt2html/tt/TemplateHandler.js
index 858f553..333de90 100644
--- a/lib/wt2html/tt/TemplateHandler.js
+++ b/lib/wt2html/tt/TemplateHandler.js
@@ -125,6 +125,30 @@
 
var accumReceiveToksFromSibling;
var accumReceiveToksFromChild;
+   var templateName;
+
+   if (this.options.wrapTemplates) {
+   templateName = tgt.target;
+   // We still need to check for loops and depth limit violations
+   // because of the higher precedence of extension tags, which can
+   // result in nested templates even while using the php 
preprocessor
+   // for expansion.
+   var checkRes = tgt && this.checkRes(templateName);
+   if (Array.isArray(checkRes)) {
+   cb({ tokens: checkRes });
+   return;
+   }
+   // Check if we have an expansion for this template in the cache
+   // already
+   var cachedTransclusion = env.transclusionCache[text];
+   if (cachedTransclusion) {
+   // cache hit: reuse the expansion DOM
+   var opts = { setDSR: true, isForeignContent: true };
+   toks = DU.encapsulateExpansionHTML(env, token, 
cachedTransclusion, opts);
+   cb({ tokens: toks });
+   return;
+   }
+   }
 
if (env.conf.parsoid.usePHPPreProcessor) {
if (this.options.wrapTemplates) {
@@ -138,48 +162,25 @@
// It's sufficient to pass `[]` in place of attribs 
since they
// won't be used.  In `usePHPPreProcessor`, there is no 
parameter
// substitution coming from the frame.
-
-   var templateName = tgt.target;
var attribs = [];
 
-   // We still need to check for loops and depth limit 
violations
-   // because of the higher precedence of extension tags, 
which can
-   // result in nested templates even while using the php 
preprocessor
-   // for expansion.
-   var checkRes = this.checkRes(templateName);
-   if (Array.isArray(checkRes)) {
-   cb({ tokens: checkRes });
-   return;
-   }
+   // Use a TokenAccumulator to divide the template 
processing
+   // in two parts: The child part will take care of the 
main
+   // template element (including parameters) and the 
sibling
+   // will process the returned template expansion
+   state.accum = new TokenAccumulator(this.manager, cb);
+   accumReceiveToksFromSibling = 
state.accum.receiveToksFromSibling.bind(state.accum);
+   accumReceiveToksFromChild = 
state.accum.receiveToksFromChild.bind(state.accum);
+   var srcHandler = state.srcCB.bind(
+   this, state, frame,
+   accumReceiveToksFromSibling,
+   { name: templateName, attribs: attribs, 
cacheKey: text });
 
-   // Check if we have an expansion for this template in 
the cache
-   // already
-   var cachedTransclusion = env.transclusionCache[text];
-   if (cachedTransclusion) {
-   // cache hit: reuse the expansion DOM
-   var opts = { setDSR: true, isForeignContent: 
true };
-   toks = DU.encapsulateExpansionHTML(env, token, 
cachedTransclusion, opts);
-   cb({ tokens: toks });
-   } else {
-   // Use a TokenAccumulator to divide the 
template processing
-   // in two parts: The child part will take care 
of the main
-   // template element (including parameters) and 
the sibling
-   // will process the returned template expansion
-   state.accum = new

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Don't run additional test files if --exit-unexpected and a t...

2017-05-02 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/351344 )

Change subject: Don't run additional test files if --exit-unexpected and a test 
fails.
..

Don't run additional test files if --exit-unexpected and a test fails.

Change-Id: I08404991740d22e4c239fc39cfbe58eef94230a0
---
M bin/parserTests.js
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/44/351344/1

diff --git a/bin/parserTests.js b/bin/parserTests.js
index ad1370a..5493cf2 100755
--- a/bin/parserTests.js
+++ b/bin/parserTests.js
@@ -1510,6 +1510,7 @@
failures: 0,
};
return Promise.reduce(testFilePaths, function(exitCode, testFilePath) {
+   if (exitCode !== 0 && options['exit-unexpected']) { return 
exitCode; }
var ptests = new ParserTests(testFilePath, options.modes);
return ptests.main(options, mockURL)
.then(function(result) {

-- 
To view, visit https://gerrit.wikimedia.org/r/351344
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I08404991740d22e4c239fc39cfbe58eef94230a0
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Switch from data-mw to data-mw-variant.

2017-05-02 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/351345 )

Change subject: Switch from data-mw to data-mw-variant.
..

Switch from data-mw to data-mw-variant.

Change-Id: Idaa06024685e7fce11afe2461b2538fde890d6c9
---
M lib/html2wt/LanguageVariantHandler.js
M lib/wt2html/tt/LanguageVariantHandler.js
M tests/parserTests.txt
3 files changed, 87 insertions(+), 86 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/45/351345/1

diff --git a/lib/html2wt/LanguageVariantHandler.js 
b/lib/html2wt/LanguageVariantHandler.js
index 1f9cc0a..826a84a 100644
--- a/lib/html2wt/LanguageVariantHandler.js
+++ b/lib/html2wt/LanguageVariantHandler.js
@@ -16,7 +16,7 @@
 // should be called with this == instance of WikitextSerializer
 var LanguageVariantHandler = function(node) {
var state = this.state;
-   var dataMW = DU.getDataMw(node);
+   var dataMWV = DU.getJSONAttribute(node, 'data-mw-variant', {});
var dp = DU.getDataParsoid(node);
var trailingSemi = dp.ts;
var textP;
@@ -27,7 +27,7 @@
}, new Map());
var resultP = Promise.resolve('E|'); // "error" flag
 
-   flags = Object.keys(dataMW).reduce(function(f, k) {
+   flags = Object.keys(dataMWV).reduce(function(f, k) {
if (LC_NAME_MAP.has(k)) {
f.add(LC_NAME_MAP.get(k));
}
@@ -38,7 +38,7 @@
};
 
// Tweak flag set to account for implicitly-enabled flags.
-   if (!flags.has('S') && !flags.has('T') && dataMW.filter === undefined) {
+   if (!flags.has('S') && !flags.has('T') && dataMWV.filter === undefined) 
{
flags.add('H');
}
if (flags.size === 1 && flags.has('S')) {
@@ -87,25 +87,25 @@
}).join(';');
};
 
-   if (Array.isArray(dataMW.filter)) {
+   if (Array.isArray(dataMWV.filter)) {
// "Restrict possible variants to a limited set"
-   textP = ser(dataMW.text);
+   textP = ser(dataMWV.text);
resultP = textP.then(function(text) {
console.assert(flags.size === 0);
-   return combine(dataMW.filter.join(';'), protect(text), 
false);
+   return combine(dataMWV.filter.join(';'), protect(text), 
false);
});
-   } else if (dataMW.disabled) {
+   } else if (dataMWV.disabled) {
// "Raw" / protect contents from language converter
-   textP = ser(dataMW.text);
+   textP = ser(dataMWV.text);
resultP = textP.then(function(text) {
if (!/[:;|]/.test(text)) {
maybeDeleteFlag('R');
}
return combine(sortedFlags(), protect(text), false);
});
-   } else if (Array.isArray(dataMW.bidir)) {
+   } else if (Array.isArray(dataMWV.bidir)) {
// Bidirectional rules
-   textP = Promise.all(dataMW.bidir.map(function(rule) {
+   textP = Promise.all(dataMWV.bidir.map(function(rule) {
return ser(rule.t).then(function(text) {
return rule.l + ':' + protect(text);
});
@@ -115,8 +115,8 @@
maybeDeleteFlag('S');
return combine(sortedFlags(), text, trailingSemi);
});
-   } else if (dataMW.unidir) {
-   textP = Promise.all(dataMW.unidir.map(function(rule) {
+   } else if (dataMWV.unidir) {
+   textP = Promise.all(dataMWV.unidir.map(function(rule) {
return Promise.all([ser(rule.f), 
ser(rule.t)]).spread(function(from, to) {
return protect(from) + '=>' + rule.l + ':' + 
protect(to);
});
diff --git a/lib/wt2html/tt/LanguageVariantHandler.js 
b/lib/wt2html/tt/LanguageVariantHandler.js
index ca1b4df..f5fe5d6 100644
--- a/lib/wt2html/tt/LanguageVariantHandler.js
+++ b/lib/wt2html/tt/LanguageVariantHandler.js
@@ -143,37 +143,36 @@
}
});
 
-   var dataMW;
+   var dataMWV;
if (flags.length==0 && dataAttribs.variants.length > 0) {
// "Restrict possible variants to a limited set"
-   // XXX can't return directly.
-   dataMW = { filter: dataAttribs.variants, text: 
texts[0].text };
+   dataMWV = { filter: dataAttribs.variants, text: 
texts[0].text };
} else {
-   dataMW = flags.reduce(function(dmw, f) {
+   dataMWV = flags.reduce(function(dmwv, f) {
if (LC_FLAG_MAP.has(f)) {

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Fix 'ISBN with space-delimited number' test case.

2017-05-03 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/351726 )

Change subject: Fix 'ISBN with space-delimited number' test case.
..

Fix 'ISBN with space-delimited number' test case.

Change-Id: I26571b500dad45825145c692586e71fa2e9bd45d
---
M tests/parserTests-blacklist.js
M tests/parserTests.txt
2 files changed, 3 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/26/351726/1

diff --git a/tests/parserTests-blacklist.js b/tests/parserTests-blacklist.js
index e0b2fe8..efda162 100644
--- a/tests/parserTests-blacklist.js
+++ b/tests/parserTests-blacklist.js
@@ -507,7 +507,6 @@
 add("html2html", "Parents of subpages, two levels up, with lots of extra 
trailing slashes.", "[/index.php?title=Subpage_test/L1action=editredlink=1
 Subpage test/L1]\n");
 add("html2html", "RAW magic word", "[/index.php?title=Template:QUERTYaction=editredlink=1
 Template:QUERTY]\n");
 add("html2html", "Out-of-order TOC heading levels", "\n Contents \n\n\n 1 2\n\n 1.1 6\n 1.2 3\n 2 1\n\n 2.1 5\n 2.2 2\n\n\n\n 2 \n\n 6 \n\n 3 \n\n 1 \n\n 5 \n\n 2 \n");
-add("html2html", "ISBN with space-delimited number", "ISBN
 92 9017 032 8\n");
 add("html2html", "ISBN length", "ISBN 
123456789\n\nISBN
 1234567890\n\nISBN 
12345678901\n");
 add("html2html", "ISBN with trailing year (T9110)", "ISBN
 1-234-56789-0 - 2006\n\nISBN
 1 234 56789 0 - 2006\n");
 add("html2html", "Self-link in language variants", "Both Dunav and Дунав are names for this 
river.\n");
@@ -1025,7 +1024,6 @@
 add("html2wt", "Template caching", "This is a test template\nThis is a test 
template\n");
 add("html2wt", "Inclusion of !userCanEdit() content", "== header ==\n");
 add("html2wt", "Out-of-order TOC heading levels", "\n== Contents 
==\n\n\n* [[#2|1 2]]\n\n** [[#6|1.1 
6]]\n** [[#3|1.2 3]]\n* [[#1|2 1]]\n\n** 
[[#5|2.1 5]]\n** [[#2_2|2.2 
2]]\n\n\n\n== 2 ==\n\n== 6 
==\n\n=== 3 ===\n\n= 1 =\n\n= 5 =\n\n== 2 ==\n");
-add("html2wt", "ISBN with space-delimited number", 
"[[wiki/Special:BookSources/9290170328|ISBN 92 9017 032 8]]\n");
 add("html2wt", "ISBN length", "ISBN 
123456789\n\n[[wiki/Special:BookSources/1234567890|ISBN 1234567890]]\n\nISBN 
12345678901\n");
 add("html2wt", "ISBN with trailing year (T9110)", 
"[[wiki/Special:BookSources/1234567890|ISBN 1-234-56789-0]] - 
2006\n\n[[wiki/Special:BookSources/1234567890|ISBN 1 234 56789 0]] - 2006\n");
 add("html2wt", "anchorencode", "foo_bar.C2.A9.23.25n\n");
diff --git a/tests/parserTests.txt b/tests/parserTests.txt
index 13fce4e..33016cb 100644
--- a/tests/parserTests.txt
+++ b/tests/parserTests.txt
@@ -20356,9 +20356,11 @@
 ISBN with space-delimited number
 !! wikitext
 ISBN 92 9017 032 8
-!! html
+!! html/php
 ISBN 92 9017 032 8
 
+!! html/parsoid
+ISBN 92 9017 032 8
 !! end
 
 

-- 
To view, visit https://gerrit.wikimedia.org/r/351726
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I26571b500dad45825145c692586e71fa2e9bd45d
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Fix path to parserTests.php.

2017-05-16 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/354039 )

Change subject: Fix path to parserTests.php.
..

Fix path to parserTests.php.

Change-Id: Ic29143de111d6abfdbd90281a2554e01e54bd314
---
M tools/sync-parserTests.js
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/39/354039/1

diff --git a/tools/sync-parserTests.js b/tools/sync-parserTests.js
index d0ea830..737186c 100755
--- a/tools/sync-parserTests.js
+++ b/tools/sync-parserTests.js
@@ -21,7 +21,7 @@
$ cd $REPO
$ git rebase master
  ... resolve conflicts, sigh ...
-   $ php tests/parserTests.php
+   $ php tests/parser/parserTests.php
  ... fix any failures by marking tests parsoid-only, etc ...
$ git review
 

-- 
To view, visit https://gerrit.wikimedia.org/r/354039
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic29143de111d6abfdbd90281a2554e01e54bd314
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...VisualEditor[master]: Node inspector for LanguageConverter markup

2017-06-12 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/358396 )

Change subject: Node inspector for LanguageConverter markup
..

Node inspector for LanguageConverter markup

Add an inspector giving more information on language converter markup.
A follow-up patch will add the edit dialog.

Bug: T49411
Change-Id: I3c91db84793a9a965dcb760253b24edee5d7bd23
---
M extension.json
M modules/ve-mw/i18n/en.json
M modules/ve-mw/i18n/qqq.json
A modules/ve-mw/ui/contextitems/ve.ui.MWLanguageVariantNodeContextItem.js
A modules/ve-mw/ui/dialogs/ve.ui.MWLanguageVariantDialog.js
A 
modules/ve-mw/ui/styles/contextitems/ve.ui.MWLanguageVariantNodeContextItem.css
6 files changed, 364 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/VisualEditor 
refs/changes/96/358396/1

diff --git a/extension.json b/extension.json
index 307cc61..ad3e883 100644
--- a/extension.json
+++ b/extension.json
@@ -1896,9 +1896,11 @@
"ext.visualEditor.mwlanguage": {
"scripts": [

"modules/ve-mw/dm/nodes/ve.dm.MWLanguageVariantNode.js",
-   
"modules/ve-mw/ce/nodes/ve.ce.MWLanguageVariantNode.js"
+   
"modules/ve-mw/ce/nodes/ve.ce.MWLanguageVariantNode.js",
+   
"modules/ve-mw/ui/contextitems/ve.ui.MWLanguageVariantNodeContextItem.js"
],
"styles": [
+   
"modules/ve-mw/ui/styles/contextitems/ve.ui.MWLanguageVariantNodeContextItem.css",

"modules/ve-mw/ce/styles/nodes/ve.ce.MWLanguageVariantNode.css"
],
"dependencies": [
@@ -1910,7 +1912,17 @@
"visualeditor-mwlanguagevariant-disabled",
"visualeditor-mwlanguagevariant-filter",
"visualeditor-mwlanguagevariant-bidir",
-   "visualeditor-mwlanguagevariant-unidir"
+   "visualeditor-mwlanguagevariant-unidir",
+   
"visualeditor-mwlanguagevariantnodeinspector-rule-name-label",
+   
"visualeditor-mwlanguagevariantnodeinspector-rule-code-label",
+   
"visualeditor-mwlanguagevariantnodeinspector-rule-text-label",
+   
"visualeditor-mwlanguagevariantnodeinspector-rule-text-from-label",
+   
"visualeditor-mwlanguagevariantnodeinspector-rule-text-to-label",
+   
"visualeditor-mwlanguagevariantnodeinspector-title-unknown",
+   
"visualeditor-mwlanguagevariantnodeinspector-title-disabled",
+   
"visualeditor-mwlanguagevariantnodeinspector-title-filter",
+   
"visualeditor-mwlanguagevariantnodeinspector-title-bidir",
+   
"visualeditor-mwlanguagevariantnodeinspector-title-unidir"
],
"targets": [
"desktop",
diff --git a/modules/ve-mw/i18n/en.json b/modules/ve-mw/i18n/en.json
index 8fdfceb..7ba6f53 100644
--- a/modules/ve-mw/i18n/en.json
+++ b/modules/ve-mw/i18n/en.json
@@ -324,6 +324,16 @@
"visualeditor-mwlanguagevariant-filter": "Variants filtered to $1",
"visualeditor-mwlanguagevariant-bidir": "Bidirectional conversion: $1",
"visualeditor-mwlanguagevariant-unidir": "Unidirectional conversion: 
$1",
+   "visualeditor-mwlanguagevariantnodeinspector-rule-name-label": 
"Language",
+   "visualeditor-mwlanguagevariantnodeinspector-rule-code-label": 
"Language code",
+   "visualeditor-mwlanguagevariantnodeinspector-rule-text-label": "Text",
+   "visualeditor-mwlanguagevariantnodeinspector-rule-text-from-label": 
"Source",
+   "visualeditor-mwlanguagevariantnodeinspector-rule-text-to-label": 
"Destination",
+   "visualeditor-mwlanguagevariantnodeinspector-title-unknown": "Language 
variant",
+   "visualeditor-mwlanguagevariantnodeinspector-title-disabled": "Variant 
conversion disabled",
+   "visualeditor-mwlanguagevariantnodeinspector-title-filter": "Variant 
filter",
+   "visualeditor-mwlanguagevariantnodeinspector-title-bidir": 
"Bidirectional conversion rule",
+   "visualeditor-mwlanguagevariantnodeinspector-title-unidir": 
"Unidirectional conversion rule",
"visualeditor-mwpredialog-title": "Preformatted plain text",
"visualeditor-mwpredialog-convert": "Allow text styling",
"visualeditor-mwsignature-tool": "Your signature",
diff --git a/modules/ve-mw/i18n/qqq.json b/modules/ve-mw/i18n/qqq.json
index 1e27c0d..c486f6a 100644
--- a/modules/ve-mw/i18n/qqq.json
+++

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: tests: Reset LanguageConverter conversion tables between tes...

2017-06-26 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/361492 )

Change subject: tests: Reset LanguageConverter conversion tables between test 
cases
..

tests: Reset LanguageConverter conversion tables between test cases

Conversion rules defined in a previous test case were leaking into
subsequent test cases.  Existing tests had worked around this by defining
non-overlapping rules, but it's better to just fix the problem at the
source.

Change-Id: I65736520cd04bfe8949b29ade07338a6e1b88a4d
---
M languages/FakeConverter.php
M languages/LanguageConverter.php
M tests/parser/ParserTestRunner.php
3 files changed, 8 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/92/361492/1

diff --git a/languages/FakeConverter.php b/languages/FakeConverter.php
index 0cddc99..0bf88c6 100644
--- a/languages/FakeConverter.php
+++ b/languages/FakeConverter.php
@@ -125,4 +125,7 @@
 
public function updateConversionTable( Title $title ) {
}
+
+   public function reloadTables() {
+   }
 }
diff --git a/languages/LanguageConverter.php b/languages/LanguageConverter.php
index 19d644c..ef88c12 100644
--- a/languages/LanguageConverter.php
+++ b/languages/LanguageConverter.php
@@ -891,7 +891,9 @@
/**
 * Reload the conversion tables.
 *
-* @private
+* Public only for use by test suites which need to reset the converter
+* state.
+*
 */
function reloadTables() {
if ( $this->mTables ) {
diff --git a/tests/parser/ParserTestRunner.php 
b/tests/parser/ParserTestRunner.php
index f44b0d5..26c5586 100644
--- a/tests/parser/ParserTestRunner.php
+++ b/tests/parser/ParserTestRunner.php
@@ -1045,6 +1045,8 @@
$context->setUser( $user );
$context->setLanguage( $lang );
$teardown[] = function () use ( $context ) {
+   // Clear language conversion tables
+   $context->getLanguage()->getConverter()->reloadTables();
// Reset context to the restored globals
$context->setUser( $GLOBALS['wgUser'] );
$context->setLanguage( $GLOBALS['wgContLang'] );

-- 
To view, visit https://gerrit.wikimedia.org/r/361492
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I65736520cd04bfe8949b29ade07338a6e1b88a4d
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Expand variant list in LC DOM markup.

2017-06-27 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/361692 )

Change subject: Expand variant list in LC DOM markup.
..

Expand variant list in LC DOM markup.

Change-Id: Ia1b649082a9f1224635980ec768357296d0221f8
---
M lib/config/WikiConfig.js
M lib/wt2html/tt/LanguageVariantHandler.js
2 files changed, 15 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/92/361692/1

diff --git a/lib/config/WikiConfig.js b/lib/config/WikiConfig.js
index 6eefc4b..4e590fe 100644
--- a/lib/config/WikiConfig.js
+++ b/lib/config/WikiConfig.js
@@ -632,17 +632,21 @@
// `general.variants`), because the *page language* could be quite
// different.  Use the mechanism introduced in T153341 instead.
this.variants = new Map();
+   this.variantsForBase = new Map();
this.langConverterEnabled = new Set();
Object.keys(languagevariants).forEach(function(code) {
+   var variantList = [];
if (general.langconversion !== undefined) {
this.langConverterEnabled.add(code);
}
Object.keys(languagevariants[code]).forEach(function(v) {
+   variantList.push(v);
this.variants.set(v, {
base: code,
fallbacks: languagevariants[code][v].fallbacks,
});
}.bind(this));
+   this.variantsForBase.set(code, variantList);
}.bind(this));
 
// Match a wikitext line containing just whitespace, comments, and
diff --git a/lib/wt2html/tt/LanguageVariantHandler.js 
b/lib/wt2html/tt/LanguageVariantHandler.js
index b73f6a9..9c885d6 100644
--- a/lib/wt2html/tt/LanguageVariantHandler.js
+++ b/lib/wt2html/tt/LanguageVariantHandler.js
@@ -144,6 +144,17 @@
if (texts.length === 1 && !texts[0].lang  && 
!dataMWV.name) {
if (dataMWV.add || dataMWV.remove) {
var variants = [ '*' ];
+   // Expanding the list of variants 
bloats the DOM.
+   // But if we wanted to be explicit:
+   if (false) {
+   var env = manager.env;
+   var lang =
+   env.page.pagelanguage ||
+   env.conf.wiki.lang ||
+   'en';
+   variants =
+   
manager.env.conf.wiki.variantsForBase.get(lang);
+   }
bidir = variants.map(function(code) {
return { l: code, t: 
texts[0].text };
});

-- 
To view, visit https://gerrit.wikimedia.org/r/361692
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia1b649082a9f1224635980ec768357296d0221f8
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Allow \r\n in wikilink titles.

2017-05-18 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/354235 )

Change subject: Allow \r\n in wikilink titles.
..

Allow \r\n in wikilink titles.

This fixes an RT testing regression on
[[it:Teenage Mutant Ninja Turtles - Tartarughe Ninja]]
caused by this fragment of (bogus) wikitext:

  |autori = [[  Ciro Nieli
  Joshua Sternin
  J. R. Ventimilia]]

In general our practice now is *not* to try to validate the title inside
the tokenizer, but instead to accept the same set of characters here that
the PHP preprocessor does and wait until later title validation to
unlink bogus titles.

Change-Id: I0ed15d6e897d2734ad8436c73de2fdcc4d0f563a
---
M lib/wt2html/pegTokenizer.pegjs
M tests/parserTests.txt
2 files changed, 16 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/35/354235/1

diff --git a/lib/wt2html/pegTokenizer.pegjs b/lib/wt2html/pegTokenizer.pegjs
index 2664c17..72b2d42 100644
--- a/lib/wt2html/pegTokenizer.pegjs
+++ b/lib/wt2html/pegTokenizer.pegjs
@@ -1950,7 +1950,7 @@
 wikilink_preprocessor_text
   = r:( t:$[^<[{\n\r\t|!\]}{ &\-]+
 // XXX gwicke: any more chars we need to allow here?
-/ !inline_breaks wr:( directive / $( !"]]" ( text_char / [!<\-\}\]] ) 
) )
+/ !inline_breaks wr:( directive / $( !"]]" ( text_char / 
[!<\-\}\]\n\r] ) ) )
 { return wr; }
 )+ {
   return tu.flattenStringlist(r);
diff --git a/tests/parserTests.txt b/tests/parserTests.txt
index 2d107e7..0d93294 100644
--- a/tests/parserTests.txt
+++ b/tests/parserTests.txt
@@ -5332,6 +5332,21 @@
 !! end
 
 !! test
+Wikilinks with embedded newlines are not broken
+!! wikitext
+{{echo|[[ Foo
+B
+C]]}}
+!! html/php
+[[ Foo
+B
+C]]
+
+!! html/parsoid
+[[ Foo B C]]
+!! end
+
+!! test
 Broken templates
 !! options
 parsoid=wt2html

-- 
To view, visit https://gerrit.wikimedia.org/r/354235
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0ed15d6e897d2734ad8436c73de2fdcc4d0f563a
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...Collection[master]: Use the recommended `@param ` ordering in d...

2017-05-23 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/355248 )

Change subject: Use the recommended `@param  ` ordering in 
documentation
..

Use the recommended `@param  ` ordering in documentation

Change-Id: I4f3601f34851ff43525b8ac3bbdf9a4691e2d3ef
---
M Collection.body.php
M Collection.hooks.php
M Collection.php
M Collection.session.php
M Collection.suggest.php
M RenderingAPI.php
6 files changed, 62 insertions(+), 62 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/Collection 
refs/changes/48/355248/1

diff --git a/Collection.body.php b/Collection.body.php
index ca3f6a3..54deebc 100644
--- a/Collection.body.php
+++ b/Collection.body.php
@@ -24,7 +24,7 @@
public $tempfile;
 
/**
-* @param $PODPartners bool|array
+* @param bool|array $PODPartners
 */
public function __construct( $PODPartners = false ) {
parent::__construct( "Book" );
@@ -48,7 +48,7 @@
}
 
/**
-* @param $par null|string
+* @param null|string $par
 */
public function execute( $par ) {
global $wgCollectionMaxArticles;
@@ -491,8 +491,8 @@
}
 
/**
-* @param $title string
-* @param $subtitle string
+* @param string $title
+* @param string $subtitle
 */
public static function setTitles( $title, $subtitle ) {
$collection = CollectionSession::getCollection();
@@ -516,8 +516,8 @@
}
 
/**
-* @param $a array
-* @param $b array
+* @param array $a
+* @param array $b
 * @return int
 */
public static function title_cmp( $a, $b ) {
@@ -549,7 +549,7 @@
}
 
/**
-* @param $name string
+* @param string $name
 */
public static function addChapter( $name ) {
$collection = CollectionSession::getCollection();
@@ -565,8 +565,8 @@
}
 
/**
-* @param $index int
-* @param $name string
+* @param int $index
+* @param string $name
 */
public static function renameChapter( $index, $name ) {
if ( !is_int( $index ) ) {
@@ -582,8 +582,8 @@
}
 
/**
-* @param $namespace int
-* @param $name string
+* @param int $namespace
+* @param string $name
 * @param int $oldid
 * @return bool
 */
@@ -596,8 +596,8 @@
}
 
/**
-* @param $title Title
-* @param $oldid int
+* @param Title $title
+* @param int $oldid
 * @return bool
 */
public static function addArticle( $title, $oldid = 0 ) {
@@ -644,9 +644,9 @@
}
 
/**
-* @param $namespace string
-* @param $name string
-* @param $oldid int
+* @param string $namespace
+* @param string $name
+* @param int $oldid
 * @return bool
 */
public static function removeArticleFromName( $namespace, $name, $oldid 
= 0 ) {
@@ -655,8 +655,8 @@
}
 
/**
-* @param $title Title
-* @param $oldid int
+* @param Title $title
+* @param int $oldid
 * @return bool
 */
public static function removeArticle( $title, $oldid = 0 ) {
@@ -674,7 +674,7 @@
}
 
/**
-* @param $name string
+* @param string $name
 * @return bool
 */
public static function addCategoryFromName( $name ) {
@@ -683,7 +683,7 @@
}
 
/**
-* @param $title Title
+* @param Title $title
 * @return bool
 */
public static function addCategory( $title ) {
@@ -733,7 +733,7 @@
}
 
/**
-* @param $index int
+* @param int $index
 * @return bool
 */
public static function removeItem( $index ) {
@@ -769,7 +769,7 @@
}
 
/**
-* @param $items array
+* @param array $items
 */
public static function setSorting( $items ) {
if ( !CollectionSession::hasSession() ) {
@@ -877,8 +877,8 @@
}
 
/**
-* @param $title Title
-* @param $append bool
+* @param Title $title
+* @param bool $append
 * @return array|bool
 */
public function loadCollection( $title, $append = false ) {
@@ -922,8 +922,8 @@
}
 
/**
-* @param $title Title
-* @param $forceOverwrite bool
+* @param Title $title
+* @param bool $forceOverwrite
 * @return bool
 */
public function saveCollection( $title, $forceOverwrite = false ) {
@@ -1003,10 +1003,10 @@
/**
 * Take an array of arrays, each containing information about one item 
to be

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Sync up with Parsoid parserTests.txt

2017-05-23 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/355253 )

Change subject: Sync up with Parsoid parserTests.txt
..

Sync up with Parsoid parserTests.txt

This now aligns with Parsoid commit ebac189004d6edc4801719a5802766113bc84beb

Change-Id: I09bd3a72fd6210abc200bead2a16ad4106bcc6f9
---
M tests/parser/parserTests.txt
1 file changed, 20 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/53/355253/1

diff --git a/tests/parser/parserTests.txt b/tests/parser/parserTests.txt
index 54a8c61..69fee30 100644
--- a/tests/parser/parserTests.txt
+++ b/tests/parser/parserTests.txt
@@ -2311,7 +2311,7 @@
 Foo bar
 
 !! html/parsoid
-Foo →bar
+Foo →bar
 !! end
 
 ## Don't expect this to rt, Parsoid drops the unmatched closing pre tags that
@@ -5329,6 +5329,21 @@
 [http://example.com;>http://example.com x
 [[Foo
 {{echo|[[Foo}}
+!! end
+
+!! test
+Wikilinks with embedded newlines are not broken
+!! wikitext
+{{echo|[[ Foo
+B
+C]]}}
+!! html/php
+[[ Foo
+B
+C]]
+
+!! html/parsoid
+[[ Foo B C]]
 !! end
 
 !! test
@@ -11218,7 +11233,7 @@
 |}
 !! html/parsoid
 
- Test One two "three" four
+ Test One two "three" four
 
 !! end
 
@@ -19529,9 +19544,8 @@
 | is not a magic word here but | is still a magic word here
 
 !! html/parsoid
-|
 is a magic word there and |
 is still a magic word here
-| is not a magic word here but |
 is still a magic word here
-
+|
 is a magic word there and |
 is still a magic word here
+| is not a magic word here but |
 is still a magic word here
 !! end
 
 !! test
@@ -25802,7 +25816,7 @@
 }}
 
 !! html/parsoid
-
+
 
 
 !! end

-- 
To view, visit https://gerrit.wikimedia.org/r/355253
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I09bd3a72fd6210abc200bead2a16ad4106bcc6f9
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Sync parserTests with core

2017-05-23 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/355265 )

Change subject: Sync parserTests with core
..

Sync parserTests with core

Lots of new tests added to the blacklist from core commit
186a182a150f20475f0887cbc261d9be01dbfd98, which will be fixed
by the LanguageConverter support in Parsoid patch
https://gerrit.wikimedia.org/r/140235

One other change was made, to a Santizier test, in core commit
33e4ac5b220b0edbef6774b3d961a3313fbf76e9.  Parsoid never had the
bug which is being fixed there ( confused with ).

Change-Id: I5c61781149bb8e598ccd7b6664f63cc42a346a8c
---
M tests/parserTests-blacklist.js
M tests/parserTests.json
M tests/parserTests.txt
3 files changed, 447 insertions(+), 7 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/65/355265/1

diff --git a/tests/parserTests-blacklist.js b/tests/parserTests-blacklist.js
index e595334..6887613 100644
--- a/tests/parserTests-blacklist.js
+++ b/tests/parserTests-blacklist.js
@@ -115,6 +115,14 @@
 add("wt2html", "T8563: Section extraction for section suppressed by 
", "\nSection 1");
 add("wt2html", "2. includeonly in html attr value", "bar\nbar");
 add("wt2html", "4. includeonly in table attributes", "\n\n\n");
+add("wt2html", "Preprocessor precedence 3: language converter is rightmost 
opening", "hi\n\n{{-{R|echo|hi}}}-\n\n[[-{R|raw]]}-");
+add("wt2html", "Preprocessor precedence 7: broken template is rightmost 
opening", "[[Foo|{{echo|Bar]]\n\nBar]]-x-y-z\n\nCareful: linktrails can move the end of 
the wikilink:\ny']]all");
+add("wt2html", "Preprocessor precedence 10: groups of braces with leading 
dash", "1
 -{R|raw}-\n2 - 
Bar-\n3 
-Three-\n4 -{raw2}-\n5 - Bar-\n6 -Three-\n7 
-{raw2}-");
+add("wt2html", "Preprocessor precedence 11: found during visual diff testing", 
"--x\n\n--x\n\n-x");
+add("wt2html", "Preprocessor precedence 12: broken language converter closed 
by brace.", "This form breaks the 
template, which is unfortunate:\n 
foo-{bar}bat\n\nBut if the broken language converter 
markup is inside an extension\ntag, nothing bad happens:\n foo-{bar}bat\n foo-{bar}bat\n foo-{bar}bat\n foo-{bar}bat\n\n\n\n");
+add("wt2html", "Preprocessor precedence, 13: broken language converter in 
external link", " http://example.com/-{foo\; 
data-parsoid='{\"targetOff\":28,\"contentOffsets\":[28,42],\"dsr\":[2,43,26,1]}'>Example
 in URL\n http://example.com\; 
data-parsoid='{\"targetOff\":66,\"contentOffsets\":[66,96],\"dsr\":[46,97,20,1]}'>Example
 in -{link} description\n http://example.com/-{foo\; about=\"#mwt1\" 
typeof=\"mw:Transclusion\" 
data-parsoid='{\"targetOff\":133,\"contentOffsets\":[133,157],\"dsr\":[100,160,null,null],\"pi\":[[{\"k\":\"1\"}]]}'
 
data-mw='{\"parts\":[{\"template\":{\"target\":{\"wt\":\"echo\",\"href\":\"./Template:Echo\"},\"params\":{\"1\":{\"wt\":\"[http://example.com/-{foo
 Breaks template, however]\"}},\"i\":0}}]}'>Breaks template, 
however");
+add("wt2html", "Preprocessor precedence, 14: broken language converter in 
comment", "  ...should be ok\n  ...extra 
dashes\n foobat ...should be 
ok");
+add("wt2html", "Preprocessor precedence, 15: broken brace markup in headings", 
"
 \n1 foo[bar 1\n1\n2 foo[[bar 2\n2\n3 foo{bar 3\n3\n4 foo{{bar 4\n4\n5 foo{{{bar 5\n5\n6 foo-{bar 6\n6");
 add("wt2html", "Templates: 2. Inside a block tag", "Foo\nFoo");
 add("wt2html", "Templates: HTML Tag: 2. Generation of HTML attr. value", "foo");
 add("wt2html", "Templates: HTML Tag: 3. Generation of HTML attr key and 
value", "foo");
@@ -202,6 +210,8 @@
 add("wt2html", "Partly disable title conversion if variant == main language 
code", "-{T|zh-cn:CN;zh-tw:TW}-");
 add("wt2html", "Partly disable title conversion if variant == main language 
code, more", "-{T|TW}-");
 add("wt2html", "Raw output of variant escape tags (R flag)", "Raw: -{R|zh:China;zh-tw:Taiwan}-");
+add("wt2html", "Nested markup inside raw output of variant escape tags (R 
flag)", "Nested raw: -{R|nested 
-{zh:China;zh-tw:Taiwan}- nested}-");
+add("wt2html", "Templates inside raw output of variant escape tags (R flag)", 
"Nested raw: -{R|nested hi
 templates}-");
 add("wt2html", "Strings evaluating false shouldn't be ignored by Language 
converter (T51072)", "-{zh-cn:0;zh-sg:1;zh-tw:2;zh-hk:3}-");
 add("wt2html", "Conversion rules from [numeric-only string] to [something 
else] (T48634)", "-{H|0=>zh-cn:B}--{H|0=>zh-cn:C;0=>zh-cn:D}--{H|0=>zh-hans:A}-012345-{A|zh-tw:0;zh-cn:E;}-012345");
 add("wt2html", "Bidirectional converter rule entries with an empty value 
should be ignored (T53551)", "-{H|zh-cn:foo;zh-tw:;}-foobar");
@@ -288,6 +298,11 @@
 add("wt2wt", "T2553: link with two variables in a piped link", 
"{|\n|[[{{{1}}}|{{{2}}}]]\n|}");
 add("wt2wt", "Abort table cell attribute parsing on wikilink", "{|\n| testing 
[[one|two]] | three || four\n| testing one two | three || 
four\n| testing=\"[[one|two]]\" | three ||

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Sync up with Parsoid parserTests.txt

2017-05-16 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/354040 )

Change subject: Sync up with Parsoid parserTests.txt
..

Sync up with Parsoid parserTests.txt

This now aligns with Parsoid commit d163deefa3aaeb0926af09a91075d6a611b36363

Change-Id: I9e543f0803247ffc264e634bf66b6bd0e143f187
---
M tests/parser/parserTests.txt
1 file changed, 266 insertions(+), 48 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/40/354040/1

diff --git a/tests/parser/parserTests.txt b/tests/parser/parserTests.txt
index 6477356..2d107e7 100644
--- a/tests/parser/parserTests.txt
+++ b/tests/parser/parserTests.txt
@@ -2737,6 +2737,39 @@
 !! end
 
 !! test
+Templates: Don't recognize targets split by newlines
+!! options
+parsoid=wt2html
+!! wikitext
+{{ech
+o|foo}}
+!! html/php
+{{ech
+o|foo}}
+
+!! html/parsoid
+{{ech
+o|foo}}
+!! end
+
+!! test
+Templates: Recognize targets when newlines and comments don't split the target
+!! options
+parsoid=wt2html
+!! wikitext
+{{  
+  echo  
+  
+
+ |foo}}
+!! html/php
+foo
+
+!! html/parsoid
+foo
+!! end
+
+!! test
 Templates: Handle empty comment-and-ws-only lines correctly
 !! wikitext
 {{echo|foo
@@ -5270,6 +5303,49 @@
 !! html
 [http:/example.com title]
 
+!! end
+
+# wt2html only because Parsoid would want to add s coming from html
+!! test
+Broken wikilinks (but not external links) prevent templates from closing
+!! options
+parsoid=wt2html
+!! wikitext
+[http://example.com x
+
+{{echo|[http://example.com x}}
+
+[[Foo
+
+{{echo|[[Foo}}
+!! html/php
+[http://example.com;>http://example.com x
+[http://example.com;>http://example.com x
+[[Foo
+{{echo|[[Foo}}
+
+!! html/parsoid
+[http://example.com;>http://example.com x
+[http://example.com;>http://example.com x
+[[Foo
+{{echo|[[Foo}}
+!! end
+
+!! test
+Broken templates
+!! options
+parsoid=wt2html
+!! wikitext
+{{echo|[[Foo|}}]]
+
+[[Foo|{{echo|]]}}
+!! html/php
+{{echo|}}
+[[Foo|]]
+
+!! html/parsoid
+{{echo|}}
+[[Foo|]]
 !! end
 
 !! test
@@ -7848,7 +7924,7 @@
 !! wikitext
 [[File:Cool "Gator".png]]
 !! html/parsoid
-
+
 !! end
 
 !! test
@@ -7896,7 +7972,7 @@
 Pentecoste
 
 !! html/parsoid
-
+
 ''Pentecoste''
 Pentecoste
 Pentecoste
@@ -7916,10 +7992,10 @@
 abc
 
 !! html/parsoid
-
-
-
-
+
+
+
+
 !! end
 
 !! test
@@ -10435,15 +10511,26 @@
 !! wikitext
 {{DISPLAYTITLE:''{{PAGENAME}}''}}
 !! html/parsoid
-
+
 !! end
 
+# NOTE: mw:ExpandedAttrs is not the best typeof here. mw:Transclusion is 
better.
+# But, this is a limitation of our representation and is documented in
+# TemplateHandler.js in processSpecialMagicWord
 !! test
 Parsoid: Template-generated DISPLAYTITLE
 !! wikitext
 echo|DISPLAYTITLE}}:Foo}}
+!! options
+showtitle
+!! config
+wgAllowDisplayTitle=true
+wgRestrictDisplayTitle=false
+!! html/php
+Foo
+
 !! html/parsoid
-
+
 !! end
 
 !! test
@@ -11007,6 +11094,39 @@
 !! end
 
 !! test
+Templates with templated name
+!! wikitext
+echo|echo}}|foo}}
+echo|inner list}} }}
+!! html
+foo
+
+ item 1
+
+!! html/parsoid
+foo
+ item 1
+!! end
+
+# Parsoid markup is deliberate "broken". This is an edge case.
+# See long comment in TemplateHandler.js:convertAttribsToString.
+!! test
+Templates with invalid templated targets
+!! wikitext
+{{echo
+{{echo|foo}}
+}}
+!! html/php
+{{echo
+foo
+}}
+
+!! html/parsoid
+{{echo
+foo }}
+!! end
+
+!! test
 Template with thumb image (with link in description)
 !! wikitext
 {{paramtest|param=[[Image:noimage.png|thumb|[[no link|link]] [[no 
link|caption}}
@@ -11021,7 +11141,7 @@
 
 
 !! html/parsoid
-This is a test template with 
parameter link caption
+This is a test template with 
parameter link caption
 !! end
 
 !! article
@@ -11253,17 +11373,23 @@
 
 !! test
 Template with targets containing wikilinks
+!! options
+parsoid=wt2html
 !! wikitext
 {{[[foo]]}}
 
 {{[[{{echo|foo}}]]}}
 
 echo|[[foo}}]]}}
-!! html
+!! html/php
 {{foo}}
 {{foo}}
 {{[[foo}}]]
 
+!! html/parsoid
+{{foo}}
+{{foo}}
+{{[[foo}}]]
 !! end
 
 !! article
@@ -13113,6 +13239,17 @@
 !! end
 
 !! test
+Parsoid: backwards pipe trick
+!! wikitext
+[[|'''bar''']]
+!! html/php
+[[|bar]]
+
+!! html/parsoid
+[[|bar]]
+!! end
+
+!! test
 pre-save transform: trim trailing empty lines
 !! options
 pst
@@ -14318,7 +14455,7 @@
 7% solution
 
 !! html/parsoid
-
+
 !! end
 
 !! test
@@ -14533,7 +14670,7 @@
 File:Foobaz.jpg
 
 !! html/parsoid
-
+
 !! end
 
 # Parsoid-specific testing for images
@@ -14866,6 +15003,7 @@
 [[.]]
 [[..]]
 [[foo././bar]]
+[[foo[http://example.com]xyz]]
 
 [[{{echo|./../foo}}|bar]]
 [[{{echo|foo/.}}|bar]]
@@ -14889,6 +15027,7 @@
 [[.]]
 [[..]]
 [[foo././bar]]
+[[foohttp://example.com;>[1]xyz]]
 [[./../foo|bar]]
 [[foo/.|bar]]
 [[foo/..|bar]]
@@ -14911,7 +15050,8 @@
 [[foo[bar]]
 [[.]]
 [[..]]
-[[foo././bar]]
+[[foo././bar]]
+[[foohttp://example.com;>xyz]]
 
 [[./../foo|bar]]
 [[foo/.|bar]]
@@ -15377,12 +15517,15 @@
 
 !! end
 
+# NOTE:

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Sync parserTests with core

2017-05-16 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/354044 )

Change subject: Sync parserTests with core
..

Sync parserTests with core

Change-Id: Ie87ffed05cf6e4c7b86ebd65d3d90ae859986cd5
---
M tests/parserTests.json
M tests/parserTests.txt
2 files changed, 3 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/44/354044/1

diff --git a/tests/parserTests.json b/tests/parserTests.json
index ebb2a0d..3e11b75 100644
--- a/tests/parserTests.json
+++ b/tests/parserTests.json
@@ -2,8 +2,8 @@
"parserTests.txt": {
"repo": "/wikimedia/mediawiki/",
"path": "tests/parser/parserTests.txt",
-   "expectedSHA1": "b95c6136eac1a685ee499aaf59cd2f6204585804",
-   "latestCommit": "ed1afdee3578bf757dc315a7cfd60525dab9dbd4"
+   "expectedSHA1": "ce0665cff572231b0a7d5cae5ef280a4322a6377",
+   "latestCommit": "f9de807e28c5194ef4e2ddbcf31375041816fab6"
},
"citeParserTests.txt": {
"repo": "/wikimedia/mediawiki-extensions-Cite/",
diff --git a/tests/parserTests.txt b/tests/parserTests.txt
index 37ae227..2d107e7 100644
--- a/tests/parserTests.txt
+++ b/tests/parserTests.txt
@@ -32,6 +32,7 @@
 # local format section links in edit comment text as local links
 # notoc disable table of contents
 # thumbsize=NNN set the default thumb size to NNNpx for this test
+# wrap  include the normal wrapper  
(since 1.30)
 #
 # You can also set the following parser properties via test options:
 #  wgEnableUploads, wgAllowExternalImages, wgMaxTocLevel,

-- 
To view, visit https://gerrit.wikimedia.org/r/354044
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie87ffed05cf6e4c7b86ebd65d3d90ae859986cd5
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...VisualEditor[master]: WIP: Use annotations for inline LanguageConverter markup.

2017-06-02 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/356865 )

Change subject: WIP: Use annotations for inline LanguageConverter markup.
..

WIP: Use annotations for inline LanguageConverter markup.

Change-Id: I9cae1c6c0d6efda5b0ba3c6f454e25834ae6eacc
---
M extension.json
A modules/ve-mw/dm/annotations/ve.dm.MWLanguageConverterRawAnnotation.js
2 files changed, 74 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/VisualEditor 
refs/changes/65/356865/1

diff --git a/extension.json b/extension.json
index 294c8a0..6238300 100644
--- a/extension.json
+++ b/extension.json
@@ -1895,6 +1895,7 @@
},
"ext.visualEditor.mwlanguage": {
"scripts": [
+   
"modules/ve-mw/dm/annotations/ve.dm.MWLanguageConverterRawAnnotation.js",

"modules/ve-mw/dm/nodes/ve.dm.MWLanguageVariantNode.js",

"modules/ve-mw/ce/nodes/ve.ce.MWLanguageVariantNode.js"
],
diff --git 
a/modules/ve-mw/dm/annotations/ve.dm.MWLanguageConverterRawAnnotation.js 
b/modules/ve-mw/dm/annotations/ve.dm.MWLanguageConverterRawAnnotation.js
new file mode 100644
index 000..27925f5
--- /dev/null
+++ b/modules/ve-mw/dm/annotations/ve.dm.MWLanguageConverterRawAnnotation.js
@@ -0,0 +1,73 @@
+/*!
+ * VisualEditor DataModel LanguageConverterRawAnnotation class.
+ *
+ * @copyright 2011-2017 VisualEditor Team and others; see AUTHORS.txt
+ * @license The MIT License (MIT); see LICENSE.txt
+ */
+
+/**
+ * DataModel MediaWiki nowiki annotation
+ *
+ * Represents `` tags (in HTML as ``) and 
unwraps them when they change
+ * so as to retrigger Parsoid's escaping mechanism.
+ *
+ * @class
+ * @extends ve.dm.Annotation
+ * @constructor
+ * @param {Object} element [description]
+ */
+ve.dm.LanguageConverterRawAnnotation = function 
VeDmLanguageConverterRawAnnotation() {
+   // Parent constructor
+   ve.dm.LanguageConverterRawAnnotation.super.apply( this, arguments );
+};
+
+/* Inheritance */
+
+OO.inheritClass( ve.dm.LanguageConverterRawAnnotation, ve.dm.Annotation );
+
+/* Static Properties */
+
+ve.dm.LanguageConverterRawAnnotation.static.name = 'mwLanguageConverterRaw';
+
+ve.dm.LanguageConverterRawAnnotation.static.matchRdfaTypes = [ 
'mw:LanguageVariant' ];
+
+ve.dm.LanguageConverterRawAnnotation.static.matchFunction = function( node ) {
+   var mwvDataJSON = node.getAttribute( 'data-mw-variant' ),
+   mwvData = mwvDataJSON ? JSON.parse( mwvDataJSON ) : {};
+   return mwvData.disabled && false; // XXX
+};
+
+ve.dm.LanguageConverterRawAnnotation.static.toDataElement = function ( 
domElements, converter ) {
+   var node = domElements[ 0 ],
+   mwvDataJSON = node.getAttribute( 'data-mw-variant' ),
+   mwvData = mwvDataJSON ? JSON.parse( mwvDataJSON ) : {},
+   text = mwvData.text || '',
+   tree = $( "" ).append( $.parseHTML(text) )[ 0 ],
+   context = converter.contextStack[ converter.contextStack.length 
- 1 ],
+   childAnnotations = context.annotations.clone(),
+   annotation = converter.annotationFactory.createFromElement(
+   { type: this.name }, converter.store
+   );
+   childAnnotations.push( annotation );
+
+   // XXX needs a stub node here as 1st element in order to reload the
+   // modelClass from in Converter.getDataFromDomSubtree.
+   return converter.getDataFromDomClean( tree, undefined, childAnnotations 
);
+};
+
+ve.dm.LanguageConverterRawAnnotation.static.toDomElements = function ( 
dataElement, doc, converter, childDomElements ) {
+   // XXX choose 'span' or 'div'
+   var domElement = document.createElement( 'span' );
+
+   domElement.setAttribute( 'typeof', 'mw:LanguageVariant' );
+   domElement.setAttribute( 'data-mw-variant', JSON.stringify({
+   disabled: true,
+   show: true,
+   text: "foo" // XXX
+   }));
+   return [ domElement ];
+};
+
+/* Registration */
+
+ve.dm.modelRegistry.register( ve.dm.LanguageConverterRawAnnotation );

-- 
To view, visit https://gerrit.wikimedia.org/r/356865
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I9cae1c6c0d6efda5b0ba3c6f454e25834ae6eacc
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/VisualEditor
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...VisualEditor[master]: WIP: Specialized inspector for LanguageConverter markup

2017-06-01 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/356739 )

Change subject: WIP: Specialized inspector for LanguageConverter markup
..

WIP: Specialized inspector for LanguageConverter markup

Implement a special node type, context item, and inspector for
LanguageConverter markup.

Depends on I4fcdebc2290ec35ba188f4c2e69d578791fbcd67 in Parsoid to
generate the appropriate markup, but is safe to merge independently.

Bug: T49411
Change-Id: Ie11e9301d2513bfe4a36036481cee9a047f46d37
---
M extension.json
A modules/ve-mw/ce/nodes/ve.ce.MWLanguageVariantNode.js
A modules/ve-mw/dm/annotations/ve.dm.MWLanguageConverterRawAnnotation.js
A modules/ve-mw/dm/nodes/ve.dm.MWLanguageVariantNode.js
4 files changed, 461 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/VisualEditor 
refs/changes/39/356739/1

diff --git a/extension.json b/extension.json
index adc3be9..d6dc51e 100644
--- a/extension.json
+++ b/extension.json
@@ -13,6 +13,7 @@
"Rob Moen",
"Timo Tijhof",
"Trevor Parscal",
+   "C. Scott Ananian",
"..."
],
"url": "https://www.mediawiki.org/wiki/Extension:VisualEditor;,
@@ -1377,6 +1378,7 @@
"ext.visualEditor.mwalienextension",
"ext.visualEditor.mwdiffing",
"ext.visualEditor.language",
+   "ext.visualEditor.mwlanguage",
"ext.visualEditor.icons"
],
"targets": [
@@ -1891,6 +1893,20 @@
"mobile"
]
},
+   "ext.visualEditor.mwlanguage": {
+   "scripts": [
+   
"modules/ve-mw/dm/annotations/ve.dm.MWLanguageConverterRawAnnotation.js",
+   
"modules/ve-mw/dm/nodes/ve.dm.MWLanguageVariantNode.js",
+   
"modules/ve-mw/ce/nodes/ve.ce.MWLanguageVariantNode.js"
+   ],
+   "dependencies": [
+   "ext.visualEditor.core"
+   ],
+   "targets": [
+   "desktop",
+   "mobile"
+   ]
+   },
"ext.visualEditor.mwalienextension": {
"scripts": [

"modules/ve-mw/dm/nodes/ve.dm.MWAlienExtensionNode.js",
diff --git a/modules/ve-mw/ce/nodes/ve.ce.MWLanguageVariantNode.js 
b/modules/ve-mw/ce/nodes/ve.ce.MWLanguageVariantNode.js
new file mode 100644
index 000..134b3bf
--- /dev/null
+++ b/modules/ve-mw/ce/nodes/ve.ce.MWLanguageVariantNode.js
@@ -0,0 +1,140 @@
+/*!
+ * VisualEditor ContentEditable MWLanguageVariantNode class.
+ *
+ * @copyright 2011-2017 VisualEditor Team and others; see AUTHORS.txt
+ * @license The MIT License (MIT); see LICENSE.txt
+ */
+
+/**
+ * ContentEditable MediaWiki language variant node, used for
+ * LanguageConverter markup.
+ *
+ * @class
+ * @abstract
+ * @extends ve.ce.FocusableNode
+ * @constructor
+ * @param {ve.dm.MWLanguageVariantNode} model Model to observe
+ * @param {Object} [config] Configuration options
+ */
+ve.ce.MWLanguageVariantNode = function VeCeMWLanguageVariantNode( $focusable, 
config ) {
+   // Parent constructor
+   ve.ce.MWLanguageVariantNode.super.call( this, $focusable, config );
+
+   // DOM changes
+   this.$element.addClass( 've-ce-mwLanguageVariantNode' );
+
+   // Events
+   this.model.connect( this, { update: 'onUpdate' } );
+
+   // Initialization
+   this.onUpdate();
+};
+
+/* Inheritance */
+
+OO.inheritClass( ve.ce.MWLanguageVariantNode, ve.ce.FocusableNode );
+
+/* Static Properties */
+
+/* Static Methods */
+
+/**
+ * @inheritdoc
+ */
+ve.ce.MWLanguageVariantNode.static.getDescription = function ( model ) {
+   // XXX FIXME
+   return model.getAttribute( 'content' );
+};
+
+/* Methods */
+
+/**
+ * @inheritdoc ve.ce.GeneratedContentNode
+ */
+ve.ce.MWLanguageVariantNode.prototype.generateContents = function ( config ) {
+   var deferred = $.Deferred();
+   deferred.resolve( $( '' ).get() );
+   return deferred.promise();
+};
+
+/**
+ * Handle model update events.
+ *
+ * @method
+ */
+ve.ce.MWLanguageVariantNode.prototype.onUpdate = function () {
+// XXX FIX ME
+/*
+   this.$link
+   .attr( 'href', this.model.getHref() )
+   .attr( 'rel', this.model.getRel() )
+   .text( this.model.getAttribute( 'content' ) );
+*/
+};
+
+/**
+ * ContentEditable MediaWiki language variant block node.
+ *
+ * @class
+ * @extends ve.ce.BranchNode
+ * @mixins ve.ce.MWLanguageVariantNode
+ *
+ * @constructor
+

[MediaWiki-commits] [Gerrit] VisualEditor/VisualEditor[master]: Don't let jQuery 3 remove top-level text nodes from generate...

2017-06-07 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/357656 )

Change subject: Don't let jQuery 3 remove top-level text nodes from generated 
content
..

Don't let jQuery 3 remove top-level text nodes from generated content

Bug: T167335
Change-Id: Id3f663ea6b0b9d18959216abaeab775306be5c6c
---
M src/ce/nodes/ve.ce.GeneratedContentNode.js
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/VisualEditor/VisualEditor 
refs/changes/56/357656/1

diff --git a/src/ce/nodes/ve.ce.GeneratedContentNode.js 
b/src/ce/nodes/ve.ce.GeneratedContentNode.js
index afbec9d..09af0b2 100644
--- a/src/ce/nodes/ve.ce.GeneratedContentNode.js
+++ b/src/ce/nodes/ve.ce.GeneratedContentNode.js
@@ -108,7 +108,7 @@
// Filter out link and style tags for bug 50043
// Previously filtered out meta tags, but restore these as they
// can be made visible.
-   $rendering = $rendering.not( 'link, style' );
+   $rendering.filter( 'link, style' ).remove();
// Also remove link and style tags nested inside other tags
$rendering.find( 'link, style' ).remove();
 

-- 
To view, visit https://gerrit.wikimedia.org/r/357656
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Id3f663ea6b0b9d18959216abaeab775306be5c6c
Gerrit-PatchSet: 1
Gerrit-Project: VisualEditor/VisualEditor
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Update Parsoid output for TOC-related parserTests.

2017-05-05 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/352173 )

Change subject: Update Parsoid output for TOC-related parserTests.
..

Update Parsoid output for TOC-related parserTests.

Change-Id: I3662c7c6e3b09f8b1f769044273ce6c0a65d83ae
---
M tests/parserTests-blacklist.js
M tests/parserTests.txt
2 files changed, 51 insertions(+), 38 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/73/352173/1

diff --git a/tests/parserTests-blacklist.js b/tests/parserTests-blacklist.js
index 565485a..b9e5405 100644
--- a/tests/parserTests-blacklist.js
+++ b/tests/parserTests-blacklist.js
@@ -236,11 +236,6 @@
 add("wt2html", "T33490 Turkish: ucfirst 'ix'", "Ix");
 add("wt2html", "T33490 Turkish: ucfırst (with a dotless i)", "{{ucfırst:blah}}");
 add("wt2html", "T33490 ucfırst (with a dotless i) with English language", "{{ucfırst:blah}}");
-add("wt2html", "T35845: Headings become cursive in TOC when they contain an 
image", "\n Image 
 ");
-add("wt2html", "T35845 (2): Headings become bold in TOC when they contain a 
blockquote", "\n Quote 
");
-add("wt2html", "Multiple tags in TOC", "\n Foo Bar \n\n Foo Bar 
");
-add("wt2html", "span tags with directionality in TOC", "\n C++ \n\n זבנג! 
\n\n The attributes on these 
span tags must be deleted from the TOC \n\n All 
attributes on these span tags must be deleted from the TOC \n\n Attributes 
after dir on these span tags must be deleted from the TOC ");
-add("wt2html", "Empty  tag in TOC, removed by Sanitizer (T92892)", "\n x ");
 add("wt2html", "T34057: Title needed when expanding  nodes.", "Parser
 function implementation for pf_int missing in Parsoid.");
 add("wt2html", "Check noCommafy in formatNum", "123456.78\n123456.78");
 add("wt2html", "Wrong option for formatNum (T58199)", "1,234.56\n1,234.56\n1234.56");
@@ -524,16 +519,6 @@
 add("html2html", "T33098 Template which includes system messages which 
includes the template", "Template 
loop detected: Template:Identical\nTemplate loop detected: 
Template:Identical\n");
 add("html2html", "T33490 Turkish: ucfırst (with a dotless i)", "[/index.php?title=%C5%9Eablon:Ucf%C4%B1rst:blahaction=editredlink=1
 Şablon:Ucfırst:blah]\n");
 add("html2html", "T33490 ucfırst (with a dotless i) with English language", 
"[/index.php?title=Template:Ucf%C4%B1rst:blahaction=editredlink=1
 Template:Ucfırst:blah]\n");
-add("html2html", "T28375: TOC with italics", "\n Contents 
\n\n\n 1 Lost 
episodes\n\n\n\n Lost episodes \n");
-add("html2html", "T28375: TOC with bold", "\n Contents 
\n\n\n 1 should be bold then normal 
text\n\n\n\n 
should be bold then normal text 
\n");
-add("html2html", "T35845: Headings become cursive in TOC when they contain an 
image", "\n Contents 
\n\n\n 1 Image\n\n\n\n Image http://example.com/images/3/3a/Foobar.jpg\; alt=\"Foobar.jpg\" 
rel=\"mw:externalImage\" data-parsoid='{\"dsr\":[185,226,null,null]}'/> 
\n");
-add("html2html", "T35845 (2): Headings become bold in TOC when they contain a 
blockquote", "\n\n\n Contents 
\n\n\n 1 Quote\n\n\n\n\n\nQuote\n\n[[/index.php?title=Main_Pageaction=editsection=1
 edit]]\n");
-add("html2html", "Unclosed tags in TOC", "\n Contents 
\n\n\n 1 Proof: 2  
3\n\n\n\n Proof: 2  3 \nHanc marginis exiguitas 
non caperet.\nQED\n");
-add("html2html", "Multiple tags in TOC", "\n\n\n Contents 
\n\n\n 1 Foo Bar\n 2 Foo 
Bar\n\n\n\n Foo Bar \n\n Foo \n\nBar\n\n[[/index.php?title=Parser_testaction=editsection=2
 edit]]\n");
-add("html2html", "span tags with directionality in TOC", "\n Contents \n\n\n 1 C++\n 2 זבנג!\n 3 The attributes on these 
span tags must be deleted from the TOC\n 4 All attributes on these 
span tags must be deleted from the TOC\n 5 Attributes after dir 
on these span tags must be deleted from the 
TOC\n\n\n\n C++[[/index.php?title=Parser_testaction=editsection=1
 edit]] 
\n\n זבנג![[/index.php?title=Parser_testaction=editsection=2
 edit]] 
\n\n The attributes on 
these span tags must be deleted from the TOC[[/index.php?title=Parser_testaction=editsection=3
 edit]] 
\n\n All 
attributes on these span tags must be deleted from the TOC[[/index.php?title=Parser_testaction=editsection=4
 edit]] 
\n\n Attributes after dir 
on these span tags must be deleted from the TOC[[/index.php?title=Parser_testaction=editsection=5
 edit]] 
\n");
-add("html2html", "T74884: bdi element in ToC", "\n Contents 
\n\n\n 1 test\n\n\n\n test \n");
-add("html2html", "T35715: s/strike element in ToC", "\n Contents \n\n\n 1 test test test\n\n\n\n test test test \n");
-add("html2html", "Empty  tag in TOC, removed by Sanitizer (T92892)", 
"\n\n\n Contents 
\n\n\n 1 x\n\n\n\n x \n");
 add("html2html", "nowiki inside link inside heading (T20295)", " xyz
 \n");
 add("html2html", "new support for bdi element (T33817)", "ולדימיר לנין (ברוסית: Владимир Ленин, 24 
באפריל 1870–22 בינואר 1924) הוא מנהיג פוליטי קומוניסטי רוסי.\n");
 add("html2html", "Empty LI

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: WIP: Protect broken wikilinks from being parsed as wikilinks...

2017-05-05 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/352179 )

Change subject: WIP: Protect broken wikilinks from being parsed as wikilinks 
later.
..

WIP: Protect broken wikilinks from being parsed as wikilinks later.

Wikitext such as `[[Foo{{echo]]}}` get parsed as broken links by the
preprocessor, but then end up being parsed as valid wikilinks later.
This exposes the intermediate representation of the preprocessor
and should be avoided.

Change-Id: I0dd4512dc9014fc8e329e682be20184867cafe80
---
M includes/parser/Preprocessor_DOM.php
M includes/parser/Preprocessor_Hash.php
2 files changed, 6 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/79/352179/1

diff --git a/includes/parser/Preprocessor_DOM.php 
b/includes/parser/Preprocessor_DOM.php
index 7539307..9bd1a16 100644
--- a/includes/parser/Preprocessor_DOM.php
+++ b/includes/parser/Preprocessor_DOM.php
@@ -966,6 +966,9 @@
}
$s .= $part->out;
}
+   // Break after first character so this doesn't get 
misparsed
+   // later.
+   $s = substr( $s, 0, 1 ) . '' . substr( $s, 1 );
}
return $s;
}
diff --git a/includes/parser/Preprocessor_Hash.php 
b/includes/parser/Preprocessor_Hash.php
index 597d1f2..0bfd29e 100644
--- a/includes/parser/Preprocessor_Hash.php
+++ b/includes/parser/Preprocessor_Hash.php
@@ -795,6 +795,9 @@
substr( $this->open, -1 ),
$openingCount - strlen( $s )
);
+   // Break after first character so this doesn't get 
misparsed
+   // later.
+   $s = substr( $s, 0, 1 ) . '' . substr( $s, 1 );
$accum = [ $s ];
$lastIndex = 0;
$first = true;

-- 
To view, visit https://gerrit.wikimedia.org/r/352179
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0dd4512dc9014fc8e329e682be20184867cafe80
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Refactor the pipetrick handler to avoid using `text()`.

2017-05-05 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/352177 )

Change subject: Refactor the pipetrick handler to avoid using `text()`.
..

Refactor the pipetrick handler to avoid using `text()`.

Change-Id: I720cc0d75c483068090773f7d5a8a183ea6148ce
---
M lib/wt2html/pegTokenizer.pegjs
1 file changed, 15 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/77/352177/1

diff --git a/lib/wt2html/pegTokenizer.pegjs b/lib/wt2html/pegTokenizer.pegjs
index 9cdf977..7fd6601 100644
--- a/lib/wt2html/pegTokenizer.pegjs
+++ b/lib/wt2html/pegTokenizer.pegjs
@@ -993,12 +993,25 @@
   lcontent = { content: [] };
   }
 
+  var textTokens = [];
   if (target === null || lcontent.pipetrick) {
-return [text()];
+textTokens.push("[[");
+if (target) {
+  textTokens.push(target);
+}
+if (lcontent.pipetrick) {
+  textTokens.push("|");
+}
+lcontent.content.forEach(function(a) {
+  // a is a mw:maybeContent attribute
+  textTokens.push("|");
+  textTokens.push(a.v);
+});
+textTokens.push("]]");
+return textTokens;
   }
 
   var obj = new SelfclosingTagTk('wikilink');
-  var textTokens = [];
   var hrefKV = new KV('href', target);
   hrefKV.vsrc = input.substring(startOffset() + 2, tpos);
   // XXX: Point to object with path, revision and input information

-- 
To view, visit https://gerrit.wikimedia.org/r/352177
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I720cc0d75c483068090773f7d5a8a183ea6148ce
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Sync up with Parsoid parserTests.txt

2017-05-05 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/352178 )

Change subject: Sync up with Parsoid parserTests.txt
..

Sync up with Parsoid parserTests.txt

This now aligns with Parsoid commit 21c9e1b6642e275eb53be922b6a3baf77d02fe59

Change-Id: I7e890e6cd903503e1f586ced13b7db3907931b38
---
M tests/parser/parserTests.txt
1 file changed, 73 insertions(+), 20 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/78/352178/1

diff --git a/tests/parser/parserTests.txt b/tests/parser/parserTests.txt
index e12c136..b1f0117 100644
--- a/tests/parser/parserTests.txt
+++ b/tests/parser/parserTests.txt
@@ -5271,6 +5271,32 @@
 
 !! end
 
+# wt2html only because Parsoid would want to add s coming from html
+!! test
+Broken wikilinks (but not external links) prevent templates from closing
+!! options
+parsoid=wt2html
+!! wikitext
+[http://example.com x
+
+{{echo|[http://example.com x}}
+
+[[Foo
+
+{{echo|[[Foo}}
+!! html/php
+[http://example.com;>http://example.com x
+[http://example.com;>http://example.com x
+[[Foo
+{{echo|[[Foo}}
+
+!! html/parsoid
+[http://example.com;>http://example.com x
+[http://example.com;>http://example.com x
+[[Foo
+{{echo|[[Foo}}
+!! end
+
 !! test
 T4702: Mismatched ,  and  tags are invalid
 !! wikitext
@@ -7847,7 +7873,7 @@
 !! wikitext
 [[File:Cool "Gator".png]]
 !! html/parsoid
-
+
 !! end
 
 !! test
@@ -7895,7 +7921,7 @@
 Pentecoste
 
 !! html/parsoid
-
+
 ''Pentecoste''
 Pentecoste
 Pentecoste
@@ -7915,10 +7941,10 @@
 abc
 
 !! html/parsoid
-
-
-
-
+
+
+
+
 !! end
 
 !! test
@@ -10434,15 +10460,18 @@
 !! wikitext
 {{DISPLAYTITLE:''{{PAGENAME}}''}}
 !! html/parsoid
-
+
 !! end
 
+# NOTE: mw:ExpandedAttrs is not the best typeof here. mw:Transclusion is 
better.
+# But, this is a limitation of our representation and is documented in
+# TemplateHandler.js in processSpecialMagicWord
 !! test
 Parsoid: Template-generated DISPLAYTITLE
 !! wikitext
 echo|DISPLAYTITLE}}:Foo}}
 !! html/parsoid
-
+
 !! end
 
 !! test
@@ -11006,6 +11035,21 @@
 !! end
 
 !! test
+Templates with templated name
+!! wikitext
+echo|echo}}|foo}}
+echo|inner list}} }}
+!! html
+foo
+
+ item 1
+
+!! html/parsoid
+foo
+ item 1
+!! end
+
+!! test
 Template with thumb image (with link in description)
 !! wikitext
 {{paramtest|param=[[Image:noimage.png|thumb|[[no link|link]] [[no 
link|caption}}
@@ -11020,7 +11064,7 @@
 
 
 !! html/parsoid
-This is a test template with 
parameter link caption
+This is a test template with 
parameter link caption
 !! end
 
 !! article
@@ -11252,17 +11296,23 @@
 
 !! test
 Template with targets containing wikilinks
+!! options
+parsoid=wt2html
 !! wikitext
 {{[[foo]]}}
 
 {{[[{{echo|foo}}]]}}
 
 echo|[[foo}}]]}}
-!! html
+!! html/php
 {{foo}}
 {{foo}}
 {{[[foo}}]]
 
+!! html/parsoid
+{{foo}}
+{{foo}}
+{{[[foo}}]]
 !! end
 
 !! article
@@ -14317,7 +14367,7 @@
 7% solution
 
 !! html/parsoid
-
+
 !! end
 
 !! test
@@ -14532,7 +14582,7 @@
 File:Foobaz.jpg
 
 !! html/parsoid
-
+
 !! end
 
 # Parsoid-specific testing for images
@@ -15376,12 +15426,15 @@
 
 !! end
 
+# NOTE: mw:ExpandedAttrs is not the best typeof here. mw:Transclusion is 
better.
+# But, this is a limitation of our representation and is documented in
+# TemplateHandler.js in processSpecialMagicWord
 !! test
 Parsoid: Defaultsort (template-generated)
 !! wikitext
 echo|DEFAULTSORT}}:Foo}}
 !! html/parsoid
-
+
 !! end
 
 ###
@@ -16418,7 +16471,7 @@
 File:No such.jpg
 
 !! html/parsoid
-
+
 !! end
 
 !! test
@@ -17887,8 +17940,8 @@
 
 !! html/parsoid
 {{{|
-
-MOVE
 YOUR MOUSE CURSOR OVER THIS TEXT
+ >
+MOVE
 YOUR MOUSE CURSOR OVER THIS TEXT
 
 
 
@@ -17965,7 +18018,7 @@
 Error 
creating thumbnail:   
 
 !! html/parsoid
-
+
 !! end
 
 !! test
@@ -22081,7 +22134,7 @@
 !! html/parsoid
 User:+%
 Page+title%
 %+ 
%20 
%+ 
 %+r
-% + 

+% + 

 3E
 3E+
 !! end
 
@@ -22095,8 +22148,8 @@
 Title with  ampersand
 
 !! html/parsoid
-
-
+
+
 !! end
 
 !! test

-- 
To view, visit https://gerrit.wikimedia.org/r/352178
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I7e890e6cd903503e1f586ced13b7db3907931b38
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Use wikilinkHandler to escape wikilink fragments.

2017-05-31 Thread C. Scott Ananian (Code Review)

C. Scott Ananian has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/356479 )

Change subject: Use wikilinkHandler to escape wikilink fragments.
..

Use wikilinkHandler to escape wikilink fragments.

Change-Id: Ic8a5bf50dd9087d7c3ed717016a7df626a9c5ccb
---
M lib/html2wt/LinkHandler.js
1 file changed, 11 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/79/356479/1

diff --git a/lib/html2wt/LinkHandler.js b/lib/html2wt/LinkHandler.js
index b2db533..e5e08bf 100644
--- a/lib/html2wt/LinkHandler.js
+++ b/lib/html2wt/LinkHandler.js
@@ -652,16 +652,22 @@
} else {
// serialize as auto-numbered external link
// [http://example.com]
-   var linktext, Construct;
+   var linktext;
// If it's just anchor text, serialize as an internal 
link.
if (/^#/.test(urlStr)) {
-   Construct = WikiLinkText;
-   linktext = '[[' + urlStr + (contentStr ? '|' + 
contentStr : '') + ']]';
+   // Reserialize content using wikilinkHandler
+   return state.serializeLinkChildrenToString(
+   node,
+   
state.serializer.wteHandlers.wikilinkHandler
+   ).then(function(ncontentStr) {
+   linktext = '[[' + urlStr + (ncontentStr 
? '|' + ncontentStr : '') + ']]';
+   state.emitChunk(new 
WikiLinkText(linktext, node, wiki, linkData.type));
+   return;
+   });
} else {
-   Construct = ExtLinkText;
linktext = '[' + urlStr + (contentStr ? ' ' + 
contentStr : '') + ']';
+   state.emitChunk(new ExtLinkText(linktext, node, 
wiki, linkData.type), node);
}
-   state.emitChunk(new Construct(linktext, node, wiki, 
linkData.type), node);
return;
}
});

-- 
To view, visit https://gerrit.wikimedia.org/r/356479
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic8a5bf50dd9087d7c3ed717016a7df626a9c5ccb
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: C. Scott Ananian 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

1 2 >

1 - 100 of 185 matches

Mail list logo