sanha commented on a change in pull request #18: [NEMO-45] Distributed
Nemo-Spark
URL: https://github.com/apache/incubator-nemo/pull/18#discussion_r190483000
##########
File path:
compiler/frontend/spark/src/main/java/edu/snu/nemo/compiler/frontend/spark/source/SparkDatasetBoundedSourceVertex.java
##########
@@ -74,25 +73,30 @@ public SparkBoundedSourceVertex getClone() {
}
/**
- * A Readable for SparkBoundedSourceReadablesWrapper.
+ * A Readable wrapper for Spark Dataset.
*/
- private final class SparkBoundedSourceReadable implements Readable<T> {
+ private final class SparkDatasetBoundedSourceReadable implements Readable<T>
{
private final LinkedHashMap<String, Object[]> commands;
private final Map<String, String> sessionInitialConf;
private final int partitionIndex;
+ private final List<String> locations;
/**
* Constructor.
- * @param commands list of commands needed to build the dataset.
+ *
+ * @param partition the partition to wrap.
+ * @param commands list of commands needed to build the dataset.
* @param sessionInitialConf spark session's initial configuration.
- * @param partitionIndex partition for this readable.
+ * @param partitionIndex partition for this readable.
*/
- private SparkBoundedSourceReadable(final LinkedHashMap<String, Object[]>
commands,
- final Map<String, String>
sessionInitialConf,
- final int partitionIndex) {
+ private SparkDatasetBoundedSourceReadable(final Partition partition,
Review comment:
I created [Nemo-85](https://issues.apache.org/jira/browse/NEMO-85).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
