Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13727#discussion_r67575684
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ---
@@ -276,7 +267,45 @@ class DataFrameReader private[sql](sparkSession:
SparkSession) extends Logging {
* `spark.sql.columnNameOfCorruptRecord`): allows renaming the new field
having malformed string
* created by `PERMISSIVE` mode. This overrides
`spark.sql.columnNameOfCorruptRecord`.</li>
*
- * @since 1.6.0
+ * @since 1.4.0
+ */
+ def json(path: String): DataFrame = {
+ // This method ensures that calls that explicit need single argument
works, see SPARK-16009
+ json(Seq(path): _*)
+ }
+
+ /**
+ * Loads a JSON file (one object per line) and returns the result as a
[[DataFrame]].
+ *
+ * This function goes through the input once to determine the input
schema. If you know the
+ * schema in advance, use the version that specifies the schema to avoid
the extra scan.
+ *
+ * You can set the following JSON-specific options to deal with
non-standard JSON files:
+ * <li>`primitivesAsString` (default `false`): infers all primitive
values as a string type</li>
+ * <li>`prefersDecimal` (default `false`): infers all floating-point
values as a decimal
+ * type. If the values do not fit in decimal, then it infers them as
doubles.</li>
+ * <li>`allowComments` (default `false`): ignores Java/C++ style comment
in JSON records</li>
+ * <li>`allowUnquotedFieldNames` (default `false`): allows unquoted JSON
field names</li>
+ * <li>`allowSingleQuotes` (default `true`): allows single quotes in
addition to double quotes
+ * </li>
+ * <li>`allowNumericLeadingZeros` (default `false`): allows leading
zeros in numbers
+ * (e.g. 00012)</li>
+ * <li>`allowBackslashEscapingAnyCharacter` (default `false`): allows
accepting quoting of all
+ * character using backslash quoting mechanism</li>
+ * <li>`mode` (default `PERMISSIVE`): allows a mode for dealing with
corrupt records
+ * during parsing.</li>
+ * <ul>
+ * <li>`PERMISSIVE` : sets other fields to `null` when it meets a
corrupted record, and puts the
--- End diff --
this does not indent correctly:
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
