Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21145#discussion_r186597255
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/ReadTask.java ---
@@ -22,20 +22,20 @@
import org.apache.spark.annotation.InterfaceStability;
/**
- * A reader factory returned by {@link
DataSourceReader#createDataReaderFactories()} and is
+ * A read task returned by {@link DataSourceReader#createReadTasks()} and
is
--- End diff --
So we want to expose 2 things in the naming:
1. it represents an input RDD partition
2. it creates `DataReader`
I think the first one really needs to be pointed out explicitly, while the
second one is not that confusing to create a `DataReader` from a partition. So
+1 on `InputPartition`.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
