vinishjail97 commented on code in PR #801:
URL: https://github.com/apache/incubator-xtable/pull/801#discussion_r3336035332
##########
xtable-core/src/main/java/org/apache/xtable/conversion/ConversionTargetFactory.java:
##########
@@ -62,12 +68,39 @@ public ConversionTarget createForFormat(TargetTable
targetTable, Configuration c
* @return
*/
public ConversionTarget createConversionTargetForName(String
tableFormatName) {
+ return createConversionTargetForName(tableFormatName, new Properties());
+ }
+
+ /**
+ * Create an instance of the ConversionTarget for the given table format
name, using the supplied
+ * target properties to disambiguate when more than one implementation is
registered for a format.
+ *
+ * <p>Delta registers two implementations sharing {@link TableFormat#DELTA}:
the default Delta
+ * Standalone {@code DeltaConversionTarget} and the Delta Kernel {@code
+ * DeltaKernelConversionTarget}. The {@link
DeltaConversionTargetConfig#USE_KERNEL} property
+ * (default {@code false}) selects between them. For all other formats
exactly one implementation
+ * matches, so the flag has no effect.
+ *
+ * @param tableFormatName the target table format name
+ * @param properties target table additional properties used to resolve the
implementation
+ * @return an uninitialized {@link ConversionTarget}
+ */
+ public ConversionTarget createConversionTargetForName(
+ String tableFormatName, Properties properties) {
+ boolean useKernel =
+ TableFormat.DELTA.equalsIgnoreCase(tableFormatName)
+ &&
DeltaConversionTargetConfig.fromProperties(properties).isUseKernel();
ServiceLoader<ConversionTarget> loader =
ServiceLoader.load(ConversionTarget.class);
for (ConversionTarget target : loader) {
- if (target.getTableFormat().equalsIgnoreCase(tableFormatName)) {
+ if (target.getTableFormat().equalsIgnoreCase(tableFormatName)
+ && isKernelTarget(target) == useKernel) {
return target;
}
}
throw new NotSupportedException("Target format is not yet supported: " +
tableFormatName);
}
+
+ private static boolean isKernelTarget(ConversionTarget target) {
Review Comment:
Rename to `isDeltaKernelTarget`?
##########
xtable-core/src/main/java/org/apache/xtable/delta/DeltaStatsExtractor.java:
##########
@@ -18,293 +18,70 @@
package org.apache.xtable.delta;
-import java.io.IOException;
-import java.util.ArrayDeque;
-import java.util.Arrays;
-import java.util.Collections;
-import java.util.HashMap;
-import java.util.HashSet;
import java.util.List;
-import java.util.Map;
-import java.util.Queue;
import java.util.Set;
-import java.util.function.Function;
-import java.util.stream.Collectors;
import lombok.AccessLevel;
-import lombok.AllArgsConstructor;
-import lombok.Builder;
-import lombok.Getter;
import lombok.NoArgsConstructor;
-import lombok.Value;
-import lombok.extern.log4j.Log4j2;
-
-import org.apache.commons.lang3.StringUtils;
import org.apache.spark.sql.delta.actions.AddFile;
-import com.fasterxml.jackson.annotation.JsonAnySetter;
-import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.core.JsonProcessingException;
-import com.fasterxml.jackson.databind.ObjectMapper;
import com.google.common.annotations.VisibleForTesting;
-import org.apache.xtable.collectors.CustomCollectors;
-import org.apache.xtable.model.exception.ParseException;
import org.apache.xtable.model.schema.InternalField;
import org.apache.xtable.model.schema.InternalSchema;
-import org.apache.xtable.model.schema.InternalType;
import org.apache.xtable.model.stat.ColumnStat;
import org.apache.xtable.model.stat.FileStats;
-import org.apache.xtable.model.stat.Range;
/**
- * DeltaStatsExtractor extracts column stats and also responsible for their
serialization leveraging
- * {@link DeltaValueConverter}.
+ * Delta Standalone stats extractor - delegates to {@link DeltaStatsUtils} for
shared logic.
+ *
+ * @deprecated This class is a thin wrapper around DeltaStatsUtils. Consider
using DeltaStatsUtils
+ * directly.
*/
-@Log4j2
+@Deprecated
Review Comment:
Is it still used by the existing DeltaConversionTarget? If yes, why is it
marked deprecated?
##########
xtable-core/src/main/java/org/apache/xtable/conversion/ConversionTargetFactory.java:
##########
@@ -62,12 +68,39 @@ public ConversionTarget createForFormat(TargetTable
targetTable, Configuration c
* @return
*/
public ConversionTarget createConversionTargetForName(String
tableFormatName) {
+ return createConversionTargetForName(tableFormatName, new Properties());
+ }
+
+ /**
+ * Create an instance of the ConversionTarget for the given table format
name, using the supplied
Review Comment:
Reduce verbose comments here and keep it more concise?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]