leesf commented on a change in pull request #1405: [HUDI-344] Add partitioner
param to Exporter
URL: https://github.com/apache/incubator-hudi/pull/1405#discussion_r392595777
##########
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieSnapshotExporter.java
##########
@@ -47,36 +49,47 @@
import org.apache.spark.sql.SparkSession;
import org.apache.spark.sql.execution.datasources.DataSource;
-import scala.Tuple2;
-import scala.collection.JavaConversions;
-
import java.io.IOException;
import java.io.Serializable;
import java.util.ArrayList;
-import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;
+import scala.Tuple2;
+import scala.collection.JavaConversions;
+
/**
* Export the latest records of Hudi dataset to a set of external files (e.g.,
plain parquet files).
*
* @experimental This export is an experimental tool. If you want to export
hudi to hudi, please use HoodieSnapshotCopier.
*/
public class HoodieSnapshotExporter {
+
+ @FunctionalInterface
+ public interface Partitioner {
+
+ DataFrameWriter<Row> partition(Dataset<Row> source);
+
+ }
Review comment:
Also need UT for user-defined Partitioner?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
