vinishjail97 commented on code in PR #801:
URL: https://github.com/apache/incubator-xtable/pull/801#discussion_r3336095844
##########
xtable-core/src/main/java/org/apache/xtable/conversion/ConversionTargetFactory.java:
##########
@@ -62,12 +68,39 @@ public ConversionTarget createForFormat(TargetTable
targetTable, Configuration c
* @return
*/
public ConversionTarget createConversionTargetForName(String
tableFormatName) {
+ return createConversionTargetForName(tableFormatName, new Properties());
+ }
+
+ /**
+ * Create an instance of the ConversionTarget for the given table format
name, using the supplied
Review Comment:
Done — condensed the Javadoc to a 4-line summary (kept the `USE_KERNEL`
property and default-`false` note).
##########
xtable-core/src/main/java/org/apache/xtable/conversion/ConversionTargetFactory.java:
##########
@@ -62,12 +68,39 @@ public ConversionTarget createForFormat(TargetTable
targetTable, Configuration c
* @return
*/
public ConversionTarget createConversionTargetForName(String
tableFormatName) {
+ return createConversionTargetForName(tableFormatName, new Properties());
+ }
+
+ /**
+ * Create an instance of the ConversionTarget for the given table format
name, using the supplied
+ * target properties to disambiguate when more than one implementation is
registered for a format.
+ *
+ * <p>Delta registers two implementations sharing {@link TableFormat#DELTA}:
the default Delta
+ * Standalone {@code DeltaConversionTarget} and the Delta Kernel {@code
+ * DeltaKernelConversionTarget}. The {@link
DeltaConversionTargetConfig#USE_KERNEL} property
+ * (default {@code false}) selects between them. For all other formats
exactly one implementation
+ * matches, so the flag has no effect.
+ *
+ * @param tableFormatName the target table format name
+ * @param properties target table additional properties used to resolve the
implementation
+ * @return an uninitialized {@link ConversionTarget}
+ */
+ public ConversionTarget createConversionTargetForName(
+ String tableFormatName, Properties properties) {
+ boolean useKernel =
+ TableFormat.DELTA.equalsIgnoreCase(tableFormatName)
+ &&
DeltaConversionTargetConfig.fromProperties(properties).isUseKernel();
ServiceLoader<ConversionTarget> loader =
ServiceLoader.load(ConversionTarget.class);
for (ConversionTarget target : loader) {
- if (target.getTableFormat().equalsIgnoreCase(tableFormatName)) {
+ if (target.getTableFormat().equalsIgnoreCase(tableFormatName)
+ && isKernelTarget(target) == useKernel) {
return target;
}
}
throw new NotSupportedException("Target format is not yet supported: " +
tableFormatName);
}
+
+ private static boolean isKernelTarget(ConversionTarget target) {
Review Comment:
Done — renamed to `isDeltaKernelTarget`.
##########
xtable-core/src/main/java/org/apache/xtable/delta/DeltaStatsExtractor.java:
##########
@@ -18,293 +18,70 @@
package org.apache.xtable.delta;
-import java.io.IOException;
-import java.util.ArrayDeque;
-import java.util.Arrays;
-import java.util.Collections;
-import java.util.HashMap;
-import java.util.HashSet;
import java.util.List;
-import java.util.Map;
-import java.util.Queue;
import java.util.Set;
-import java.util.function.Function;
-import java.util.stream.Collectors;
import lombok.AccessLevel;
-import lombok.AllArgsConstructor;
-import lombok.Builder;
-import lombok.Getter;
import lombok.NoArgsConstructor;
-import lombok.Value;
-import lombok.extern.log4j.Log4j2;
-
-import org.apache.commons.lang3.StringUtils;
import org.apache.spark.sql.delta.actions.AddFile;
-import com.fasterxml.jackson.annotation.JsonAnySetter;
-import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.core.JsonProcessingException;
-import com.fasterxml.jackson.databind.ObjectMapper;
import com.google.common.annotations.VisibleForTesting;
-import org.apache.xtable.collectors.CustomCollectors;
-import org.apache.xtable.model.exception.ParseException;
import org.apache.xtable.model.schema.InternalField;
import org.apache.xtable.model.schema.InternalSchema;
-import org.apache.xtable.model.schema.InternalType;
import org.apache.xtable.model.stat.ColumnStat;
import org.apache.xtable.model.stat.FileStats;
-import org.apache.xtable.model.stat.Range;
/**
- * DeltaStatsExtractor extracts column stats and also responsible for their
serialization leveraging
- * {@link DeltaValueConverter}.
+ * Delta Standalone stats extractor - delegates to {@link DeltaStatsUtils} for
shared logic.
+ *
+ * @deprecated This class is a thin wrapper around DeltaStatsUtils. Consider
using DeltaStatsUtils
+ * directly.
*/
-@Log4j2
+@Deprecated
Review Comment:
Good catch — it is still the active stats extractor for the Standalone path
(DeltaConversionSource, DeltaActionsConverter, DeltaDataFileUpdatesExtractor,
DeltaDataFileExtractor), so the deprecation was misleading. Removed
`@Deprecated` and the `@deprecated` tag (kept the "delegates to
DeltaStatsUtils" note). Applied the same to `DeltaKernelStatsExtractor`, which
is likewise actively used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]