This is an automated email from the ASF dual-hosted git repository.
uros-b pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new c9e28f4b36e9 [SPARK-58745][DOC] Document missing JSON and XML data
source options
c9e28f4b36e9 is described below
commit c9e28f4b36e97906595e61587d1fb0fc40418f02
Author: Uros Bojanic <[email protected]>
AuthorDate: Thu Aug 13 05:58:56 2026 +0000
[SPARK-58745][DOC] Document missing JSON and XML data source options
### What changes were proposed in this pull request?
Adds table rows for options that are registered in the JSON and XML data
sources but were missing from the option tables:
- JSON: `pretty`, `writeNonAsciiCharacterAsCodePoint`
- XML: `indent`, `multiLine`, `prefersDecimal`, `preferDate`
### Why are the changes needed?
Each is a real user-facing option registered via `newOption`, so it passes
option validation and can be set by any user, yet none appeared in the
documented tables. The gap was found by diffing the `newOption` registrations
in `JSONOptions`/`XmlOptions` against the `<code>` entries in each doc.
### Does this PR introduce _any_ user-facing change?
No, other than newly documenting existing options.
### How was this patch tested?
Documentation only. Every default and description is taken from source:
`indent` defaults to `XmlOptions.DEFAULT_INDENT` (four spaces); `multiLine`
defaults to `true` for XML; `prefersDecimal`/`preferDate` affect schema
inference and `preferDate` is disabled when `spark.sql.legacy.timeParserPolicy`
is `LEGACY`; `pretty` and `writeNonAsciiCharacterAsCodePoint` are write-side
and default to `false`.
After this change the JSON table is complete. The XML table still omits
`useLegacyXMLParser`, a legacy parser-implementation switch, which is
intentionally left undocumented.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.8)
Closes #57971 from uros-b/doc-datasource-missing-options.
Authored-by: Uros Bojanic <[email protected]>
Signed-off-by: Uros Bojanic <[email protected]>
(cherry picked from commit a2649b96d3a9a8af667f9b82c46359476a023373)
Signed-off-by: Uros Bojanic <[email protected]>
---
docs/sql-data-sources-json.md | 12 ++++++++++++
docs/sql-data-sources-xml.md | 24 ++++++++++++++++++++++++
2 files changed, 36 insertions(+)
diff --git a/docs/sql-data-sources-json.md b/docs/sql-data-sources-json.md
index a30156b7a708..6ac5d690868c 100644
--- a/docs/sql-data-sources-json.md
+++ b/docs/sql-data-sources-json.md
@@ -249,6 +249,18 @@ Data source options of JSON can be set via:
<td>For reading, allows to forcibly set one of standard basic or extended
encoding for the JSON files. For example UTF-16BE, UTF-32LE. For writing,
Specifies encoding (charset) of saved json files. JSON built-in functions
ignore this option.</td>
<td>read/write</td>
</tr>
+ <tr>
+ <td><code>pretty</code></td>
+ <td><code>false</code></td>
+ <td>If true, writes the generated JSON with the default pretty printer,
indenting nested structures over multiple lines instead of emitting each record
on a single line.</td>
+ <td>write</td>
+ </tr>
+ <tr>
+ <td><code>writeNonAsciiCharacterAsCodePoint</code></td>
+ <td><code>false</code></td>
+ <td>If true, writes non-ASCII characters as \uXXXX escape sequences
instead of emitting them literally.</td>
+ <td>write</td>
+ </tr>
<tr>
<td><code>lineSep</code></td>
<td><code>\r</code>, <code>\r\n</code>, <code>\n</code> (for reading),
<code>\n</code> (for writing)</td>
diff --git a/docs/sql-data-sources-xml.md b/docs/sql-data-sources-xml.md
index e714ded0ee28..8a07a486b70d 100644
--- a/docs/sql-data-sources-xml.md
+++ b/docs/sql-data-sources-xml.md
@@ -248,6 +248,30 @@ Data source options of XML can be set via:
<td>Compression codec to use when saving to file. This can be one of the
known case-insensitive shortened names (none, bzip2, gzip, lz4, snappy and
deflate). XML built-in functions ignore this option.</td>
<td>write</td>
</tr>
+ <tr>
+ <td><code>indent</code></td>
+ <td>four spaces</td>
+ <td>String used to indent each nested level of the generated XML. Setting
it to an empty string disables indentation, writing each row on a new line.</td>
+ <td>write</td>
+ </tr>
+ <tr>
+ <td><code>multiLine</code></td>
+ <td><code>true</code></td>
+ <td>Whether to parse one record, which may span multiple lines, per
file.</td>
+ <td>read</td>
+ </tr>
+ <tr>
+ <td><code>prefersDecimal</code></td>
+ <td><code>false</code></td>
+ <td>During schema inference, infers floating-point values as
<code>DecimalType</code> rather than <code>DoubleType</code> when they fit.</td>
+ <td>read</td>
+ </tr>
+ <tr>
+ <td><code>preferDate</code></td>
+ <td><code>true</code></td>
+ <td>During schema inference, tries to infer string columns that contain
dates as <code>DateType</code>. Disabled when
<code>spark.sql.legacy.timeParserPolicy</code> is set to
<code>LEGACY</code>.</td>
+ <td>read</td>
+ </tr>
<tr>
<td><code>validateName</code></td>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
