Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/20666#discussion_r170499102
--- Diff: python/pyspark/sql/readwriter.py ---
@@ -393,13 +395,16 @@ def csv(self, path, schema=None, sep=None,
encoding=None, quote=None, escape=Non
:param mode: allows a mode for dealing with corrupt records during
parsing. If None is
set, it uses the default value, ``PERMISSIVE``.
- * ``PERMISSIVE`` : sets other fields to ``null`` when it
meets a corrupted \
- record, and puts the malformed string into a field
configured by \
- ``columnNameOfCorruptRecord``. To keep corrupt records,
an user can set \
- a string type field named ``columnNameOfCorruptRecord``
in an \
- user-defined schema. If a schema does not have the
field, it drops corrupt \
- records during parsing. When a length of parsed CSV
tokens is shorter than \
- an expected length of a schema, it sets `null` for extra
fields.
+ * ``PERMISSIVE`` : when it meets a corrupted record, puts
the malformed string \
+ into a field configured by
``columnNameOfCorruptRecord``, and sets other \
+ fields to ``null``. To keep corrupt records, an user can
set a string type \
+ field named ``columnNameOfCorruptRecord`` in an
user-defined schema. If a \
+ schema does not have the field, it drops corrupt records
during parsing. \
+ It supports partial result for the records just with
less or more tokens \
+ than the schema. When it meets a malformed record whose
parsed tokens is \
--- End diff --
Ok.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
