Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/20959#discussion_r183226807
--- Diff: python/pyspark/sql/readwriter.py ---
@@ -882,6 +882,9 @@ def csv(self, path, mode=None, compression=None,
sep=None, quote=None, escape=No
the quote character. If None is
set, the default value is
escape character when escape and
quote characters are
different, ``\0`` otherwise..
+ :param samplingRatio: defines fraction of rows (when ``multiLine``
is ``false``) or fraction
+ of files (when ``multiLine`` is ``true``)
used for schema inferring.
--- End diff --
@MaxGekk, I think in CSV's case we shouldn't do sample based on the files
.. JSON's case is a record per file whereas CSV allows multiple lines in a file
...
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
