Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/21948#discussion_r207240252
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataWriterFactory.java
---
@@ -50,4 +50,15 @@
* this ID will always be 0.
*/
DataWriter<T> createDataWriter(int partitionId, long taskId, long
epochId);
+
+ /**
+ * When true, Spark will reuse the same data object instance when
sending data to the data writer,
+ * for better performance. Data writers should carefully handle the data
objects if it's reused,
+ * e.g. do not buffer the data objects in a list. By default it returns
false for safety, data
--- End diff --
nit: `By default it returns false, data sources` => `By default the method
returns false. Data sources`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
