[MediaWiki-commits] [Gerrit] FormatJson::parse( TRY_FIXING ) - remove trailing commas - change (mediawiki/core)
Yurik has uploaded a new change for review.
https://gerrit.wikimedia.org/r/163344
Change subject: FormatJson::parse( TRY_FIXING ) - remove trailing commas
..
FormatJson::parse( TRY_FIXING ) - remove trailing commas
Removes trailing commas from json text when parsing
Solves very common cases like [1,2,3,]
Resulting status will be set to OK but not Good to warn caller
Change-Id: Ic0eb0a711da3ae578d6bb58d7474279d6845a4a7
---
M includes/json/FormatJson.php
M languages/i18n/en.json
M languages/i18n/qqq.json
M tests/phpunit/includes/json/FormatJsonTest.php
4 files changed, 68 insertions(+), 1 deletion(-)
git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core
refs/changes/44/163344/1
diff --git a/includes/json/FormatJson.php b/includes/json/FormatJson.php
index 5565644..2dbbc30 100644
--- a/includes/json/FormatJson.php
+++ b/includes/json/FormatJson.php
@@ -61,7 +61,14 @@
*
* @since 1.24
*/
- const FORCE_ASSOC = 1;
+ const FORCE_ASSOC = 0x1;
+
+ /**
+* If set, attempts to fix invalid json.
+*
+* @since 1.24
+*/
+ const TRY_FIXING = 0x2;
/**
* Regex that matches whitespace inside empty arrays and objects.
@@ -149,6 +156,28 @@
$result = json_decode( $value, $assoc );
$code = json_last_error();
+ if ( $code === JSON_ERROR_SYNTAX ( $options
self::TRY_FIXING ) !== 0 ) {
+ // The most common error is the trailing comma in a
list or an object.
+ // We cannot simply replace /,\s*[}\]]/ because it
could be inside a string value.
+ // But we could use the fact that JSON does not allow
multi-line string values,
+ // And remove trailing commas if they are et the end of
a line.
+ // JSON only allows 4 control characters: [ \t\r\n].
So we must not use '\s' for matching.
+ // Regex match ,]any non-quote chars\n or ,\n]
with optional spaces/tabs.
+ $count = 0;
+ $value =
+ preg_replace( '/,([
\t]*[}\]][^\r\n]*([\r\n]|$)|[ \t]*[\r\n][ \t\r\n]*[}\]])/', '$1',
+ $value, - 1, $count );
+ if ( $count 0 ) {
+ $result = json_decode( $value, $assoc );
+ if ( JSON_ERROR_NONE === json_last_error() ) {
+ // Report warning
+ $st = Status::newGood( $result );
+ $st-warning( wfMessage(
'json-warn-trailing-comma' )-numParams( $count ) );
+ return $st;
+ }
+ }
+ }
+
switch ( $code ) {
case JSON_ERROR_NONE:
return Status::newGood( $result );
diff --git a/languages/i18n/en.json b/languages/i18n/en.json
index e8a5c3c..7a1d2f5 100644
--- a/languages/i18n/en.json
+++ b/languages/i18n/en.json
@@ -3559,6 +3559,7 @@
mediastatistics-header-text: Textual,
mediastatistics-header-executable: Executables,
mediastatistics-header-archive: Compressed formats,
+ json-warn-trailing-comma: $1 trailing {{PLURAL:$1|comma was|commas
were}} removed from JSON,
json-error-unknown: There was a problem with the JSON. Error: $1,
json-error-depth: The maximum stack depth has been exceeded,
json-error-state-mismatch: Invalid or malformed JSON,
diff --git a/languages/i18n/qqq.json b/languages/i18n/qqq.json
index 2049ae5..96e7fac 100644
--- a/languages/i18n/qqq.json
+++ b/languages/i18n/qqq.json
@@ -3721,6 +3721,7 @@
mediastatistics-header-text: Header on [[Special:MediaStatistics]]
for file types that are in the text category. This includes simple text
formats, including plain text formats, json, csv, and xml. Source code of
compiled programming languages may be included here in the future, but isn't
currently.,
mediastatistics-header-executable: Header on
[[Special:MediaStatistics]] for file types that are in the executable category.
This includes things like source files for interpreted programming language
(Shell scripts, javascript, etc).,
mediastatistics-header-archive: Header on
[[Special:MediaStatistics]] for file types that are in the archive category.
Includes things like tar, zip, gzip etc.,
+ json-warn-trailing-comma: A warning message notifying that JSON text
was automatically corrected by removing erroneous commas.\n\nParameters:\n* $1
- number of commas that were removed,
json-error-unknown: User error message when there’s an unknown
error.\n{{Identical|Unknown error}}. This error is shown if we
[MediaWiki-commits] [Gerrit] FormatJson::parse( TRY_FIXING ) - remove trailing commas - change (mediawiki/core)
jenkins-bot has submitted this change and it was merged.
Change subject: FormatJson::parse( TRY_FIXING ) - remove trailing commas
..
FormatJson::parse( TRY_FIXING ) - remove trailing commas
Removes trailing commas from json text when parsing
Solves very common cases like [1,2,3,]
Resulting status will be set to OK but not Good to warn caller
Change-Id: Ic0eb0a711da3ae578d6bb58d7474279d6845a4a7
---
M includes/json/FormatJson.php
M languages/i18n/en.json
M languages/i18n/qqq.json
M tests/phpunit/includes/json/FormatJsonTest.php
4 files changed, 68 insertions(+), 1 deletion(-)
Approvals:
Parent5446: Looks good to me, approved
jenkins-bot: Verified
diff --git a/includes/json/FormatJson.php b/includes/json/FormatJson.php
index 5565644..2dbbc30 100644
--- a/includes/json/FormatJson.php
+++ b/includes/json/FormatJson.php
@@ -61,7 +61,14 @@
*
* @since 1.24
*/
- const FORCE_ASSOC = 1;
+ const FORCE_ASSOC = 0x1;
+
+ /**
+* If set, attempts to fix invalid json.
+*
+* @since 1.24
+*/
+ const TRY_FIXING = 0x2;
/**
* Regex that matches whitespace inside empty arrays and objects.
@@ -149,6 +156,28 @@
$result = json_decode( $value, $assoc );
$code = json_last_error();
+ if ( $code === JSON_ERROR_SYNTAX ( $options
self::TRY_FIXING ) !== 0 ) {
+ // The most common error is the trailing comma in a
list or an object.
+ // We cannot simply replace /,\s*[}\]]/ because it
could be inside a string value.
+ // But we could use the fact that JSON does not allow
multi-line string values,
+ // And remove trailing commas if they are et the end of
a line.
+ // JSON only allows 4 control characters: [ \t\r\n].
So we must not use '\s' for matching.
+ // Regex match ,]any non-quote chars\n or ,\n]
with optional spaces/tabs.
+ $count = 0;
+ $value =
+ preg_replace( '/,([
\t]*[}\]][^\r\n]*([\r\n]|$)|[ \t]*[\r\n][ \t\r\n]*[}\]])/', '$1',
+ $value, - 1, $count );
+ if ( $count 0 ) {
+ $result = json_decode( $value, $assoc );
+ if ( JSON_ERROR_NONE === json_last_error() ) {
+ // Report warning
+ $st = Status::newGood( $result );
+ $st-warning( wfMessage(
'json-warn-trailing-comma' )-numParams( $count ) );
+ return $st;
+ }
+ }
+ }
+
switch ( $code ) {
case JSON_ERROR_NONE:
return Status::newGood( $result );
diff --git a/languages/i18n/en.json b/languages/i18n/en.json
index e8a5c3c..7a1d2f5 100644
--- a/languages/i18n/en.json
+++ b/languages/i18n/en.json
@@ -3559,6 +3559,7 @@
mediastatistics-header-text: Textual,
mediastatistics-header-executable: Executables,
mediastatistics-header-archive: Compressed formats,
+ json-warn-trailing-comma: $1 trailing {{PLURAL:$1|comma was|commas
were}} removed from JSON,
json-error-unknown: There was a problem with the JSON. Error: $1,
json-error-depth: The maximum stack depth has been exceeded,
json-error-state-mismatch: Invalid or malformed JSON,
diff --git a/languages/i18n/qqq.json b/languages/i18n/qqq.json
index 2049ae5..96e7fac 100644
--- a/languages/i18n/qqq.json
+++ b/languages/i18n/qqq.json
@@ -3721,6 +3721,7 @@
mediastatistics-header-text: Header on [[Special:MediaStatistics]]
for file types that are in the text category. This includes simple text
formats, including plain text formats, json, csv, and xml. Source code of
compiled programming languages may be included here in the future, but isn't
currently.,
mediastatistics-header-executable: Header on
[[Special:MediaStatistics]] for file types that are in the executable category.
This includes things like source files for interpreted programming language
(Shell scripts, javascript, etc).,
mediastatistics-header-archive: Header on
[[Special:MediaStatistics]] for file types that are in the archive category.
Includes things like tar, zip, gzip etc.,
+ json-warn-trailing-comma: A warning message notifying that JSON text
was automatically corrected by removing erroneous commas.\n\nParameters:\n* $1
- number of commas that were removed,
json-error-unknown: User error message when there’s an unknown
error.\n{{Identical|Unknown error}}. This error is shown if we received an
unexpected value from PHP.
[MediaWiki-commits] [Gerrit] FormatJson::parse( TRY_FIXING ) - remove trailing commas - change (mediawiki/core)
Yurik has uploaded a new change for review.
https://gerrit.wikimedia.org/r/163456
Change subject: FormatJson::parse( TRY_FIXING ) - remove trailing commas
..
FormatJson::parse( TRY_FIXING ) - remove trailing commas
Removes trailing commas from json text when parsing
Solves very common cases like [1,2,3,]
Resulting status will be set to OK but not Good to warn caller
Change-Id: Ic0eb0a711da3ae578d6bb58d7474279d6845a4a7
(cherry picked from commit 289d3e4f503fac5326e986f27e0b1d924c7f)
---
M includes/json/FormatJson.php
M languages/i18n/en.json
M languages/i18n/qqq.json
M tests/phpunit/includes/json/FormatJsonTest.php
4 files changed, 68 insertions(+), 1 deletion(-)
git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core
refs/changes/56/163456/1
diff --git a/includes/json/FormatJson.php b/includes/json/FormatJson.php
index 5565644..2dbbc30 100644
--- a/includes/json/FormatJson.php
+++ b/includes/json/FormatJson.php
@@ -61,7 +61,14 @@
*
* @since 1.24
*/
- const FORCE_ASSOC = 1;
+ const FORCE_ASSOC = 0x1;
+
+ /**
+* If set, attempts to fix invalid json.
+*
+* @since 1.24
+*/
+ const TRY_FIXING = 0x2;
/**
* Regex that matches whitespace inside empty arrays and objects.
@@ -149,6 +156,28 @@
$result = json_decode( $value, $assoc );
$code = json_last_error();
+ if ( $code === JSON_ERROR_SYNTAX ( $options
self::TRY_FIXING ) !== 0 ) {
+ // The most common error is the trailing comma in a
list or an object.
+ // We cannot simply replace /,\s*[}\]]/ because it
could be inside a string value.
+ // But we could use the fact that JSON does not allow
multi-line string values,
+ // And remove trailing commas if they are et the end of
a line.
+ // JSON only allows 4 control characters: [ \t\r\n].
So we must not use '\s' for matching.
+ // Regex match ,]any non-quote chars\n or ,\n]
with optional spaces/tabs.
+ $count = 0;
+ $value =
+ preg_replace( '/,([
\t]*[}\]][^\r\n]*([\r\n]|$)|[ \t]*[\r\n][ \t\r\n]*[}\]])/', '$1',
+ $value, - 1, $count );
+ if ( $count 0 ) {
+ $result = json_decode( $value, $assoc );
+ if ( JSON_ERROR_NONE === json_last_error() ) {
+ // Report warning
+ $st = Status::newGood( $result );
+ $st-warning( wfMessage(
'json-warn-trailing-comma' )-numParams( $count ) );
+ return $st;
+ }
+ }
+ }
+
switch ( $code ) {
case JSON_ERROR_NONE:
return Status::newGood( $result );
diff --git a/languages/i18n/en.json b/languages/i18n/en.json
index 01a826c..58236d3 100644
--- a/languages/i18n/en.json
+++ b/languages/i18n/en.json
@@ -3562,6 +3562,7 @@
mediastatistics-header-text: Textual,
mediastatistics-header-executable: Executables,
mediastatistics-header-archive: Compressed formats,
+ json-warn-trailing-comma: $1 trailing {{PLURAL:$1|comma was|commas
were}} removed from JSON,
json-error-unknown: There was a problem with the JSON. Error: $1,
json-error-depth: The maximum stack depth has been exceeded,
json-error-state-mismatch: Invalid or malformed JSON,
diff --git a/languages/i18n/qqq.json b/languages/i18n/qqq.json
index cfde0cd..31504bb 100644
--- a/languages/i18n/qqq.json
+++ b/languages/i18n/qqq.json
@@ -3722,6 +3722,7 @@
mediastatistics-header-text: Header on [[Special:MediaStatistics]]
for file types that are in the text category. This includes simple text
formats, including plain text formats, json, csv, and xml. Source code of
compiled programming languages may be included here in the future, but isn't
currently.,
mediastatistics-header-executable: Header on
[[Special:MediaStatistics]] for file types that are in the executable category.
This includes things like source files for interpreted programming language
(Shell scripts, javascript, etc).,
mediastatistics-header-archive: Header on
[[Special:MediaStatistics]] for file types that are in the archive category.
Includes things like tar, zip, gzip etc.,
+ json-warn-trailing-comma: A warning message notifying that JSON text
was automatically corrected by removing erroneous commas.\n\nParameters:\n* $1
- number of commas that were removed,
json-error-unknown: User error message when there’s an unknown
[MediaWiki-commits] [Gerrit] FormatJson::parse( TRY_FIXING ) - remove trailing commas - change (mediawiki/core)
jenkins-bot has submitted this change and it was merged.
Change subject: FormatJson::parse( TRY_FIXING ) - remove trailing commas
..
FormatJson::parse( TRY_FIXING ) - remove trailing commas
Removes trailing commas from json text when parsing
Solves very common cases like [1,2,3,]
Resulting status will be set to OK but not Good to warn caller
Change-Id: Ic0eb0a711da3ae578d6bb58d7474279d6845a4a7
(cherry picked from commit 289d3e4f503fac5326e986f27e0b1d924c7f)
---
M includes/json/FormatJson.php
M languages/i18n/en.json
M languages/i18n/qqq.json
M tests/phpunit/includes/json/FormatJsonTest.php
4 files changed, 68 insertions(+), 1 deletion(-)
Approvals:
Yurik: Looks good to me, approved
jenkins-bot: Verified
diff --git a/includes/json/FormatJson.php b/includes/json/FormatJson.php
index 5565644..2dbbc30 100644
--- a/includes/json/FormatJson.php
+++ b/includes/json/FormatJson.php
@@ -61,7 +61,14 @@
*
* @since 1.24
*/
- const FORCE_ASSOC = 1;
+ const FORCE_ASSOC = 0x1;
+
+ /**
+* If set, attempts to fix invalid json.
+*
+* @since 1.24
+*/
+ const TRY_FIXING = 0x2;
/**
* Regex that matches whitespace inside empty arrays and objects.
@@ -149,6 +156,28 @@
$result = json_decode( $value, $assoc );
$code = json_last_error();
+ if ( $code === JSON_ERROR_SYNTAX ( $options
self::TRY_FIXING ) !== 0 ) {
+ // The most common error is the trailing comma in a
list or an object.
+ // We cannot simply replace /,\s*[}\]]/ because it
could be inside a string value.
+ // But we could use the fact that JSON does not allow
multi-line string values,
+ // And remove trailing commas if they are et the end of
a line.
+ // JSON only allows 4 control characters: [ \t\r\n].
So we must not use '\s' for matching.
+ // Regex match ,]any non-quote chars\n or ,\n]
with optional spaces/tabs.
+ $count = 0;
+ $value =
+ preg_replace( '/,([
\t]*[}\]][^\r\n]*([\r\n]|$)|[ \t]*[\r\n][ \t\r\n]*[}\]])/', '$1',
+ $value, - 1, $count );
+ if ( $count 0 ) {
+ $result = json_decode( $value, $assoc );
+ if ( JSON_ERROR_NONE === json_last_error() ) {
+ // Report warning
+ $st = Status::newGood( $result );
+ $st-warning( wfMessage(
'json-warn-trailing-comma' )-numParams( $count ) );
+ return $st;
+ }
+ }
+ }
+
switch ( $code ) {
case JSON_ERROR_NONE:
return Status::newGood( $result );
diff --git a/languages/i18n/en.json b/languages/i18n/en.json
index 01a826c..58236d3 100644
--- a/languages/i18n/en.json
+++ b/languages/i18n/en.json
@@ -3562,6 +3562,7 @@
mediastatistics-header-text: Textual,
mediastatistics-header-executable: Executables,
mediastatistics-header-archive: Compressed formats,
+ json-warn-trailing-comma: $1 trailing {{PLURAL:$1|comma was|commas
were}} removed from JSON,
json-error-unknown: There was a problem with the JSON. Error: $1,
json-error-depth: The maximum stack depth has been exceeded,
json-error-state-mismatch: Invalid or malformed JSON,
diff --git a/languages/i18n/qqq.json b/languages/i18n/qqq.json
index cfde0cd..31504bb 100644
--- a/languages/i18n/qqq.json
+++ b/languages/i18n/qqq.json
@@ -3722,6 +3722,7 @@
mediastatistics-header-text: Header on [[Special:MediaStatistics]]
for file types that are in the text category. This includes simple text
formats, including plain text formats, json, csv, and xml. Source code of
compiled programming languages may be included here in the future, but isn't
currently.,
mediastatistics-header-executable: Header on
[[Special:MediaStatistics]] for file types that are in the executable category.
This includes things like source files for interpreted programming language
(Shell scripts, javascript, etc).,
mediastatistics-header-archive: Header on
[[Special:MediaStatistics]] for file types that are in the archive category.
Includes things like tar, zip, gzip etc.,
+ json-warn-trailing-comma: A warning message notifying that JSON text
was automatically corrected by removing erroneous commas.\n\nParameters:\n* $1
- number of commas that were removed,
json-error-unknown: User error message when there’s an unknown
error.\n{{Identical|Unknown error}}. This
