the-other-tim-brown commented on code in PR #957:
URL: https://github.com/apache/incubator-xtable/pull/957#discussion_r4174759483
##########
xtable-core/src/main/java/org/apache/xtable/delta/DeltaPathUtils.java:
##########
@@ -51,6 +56,35 @@ public static String getFullPathToFile(String tableBasePath,
String dataFilePath
: tableBasePath + Path.SEPARATOR + dataFilePath;
}
+ /**
+ * Returns the path to record in an AddFile or RemoveFile action for a data
file.
+ *
+ * <p>A file under the table base path is recorded relative to it, since
relative paths seem more
+ * commonly supported by query engines. A file outside the base path, for
example one written to a
+ * separate data location of an Iceberg source, is recorded with its
absolute path, since a
+ * relative path cannot point to it.
+ *
+ * @param tableBasePath the table base path
+ * @param dataFilePath the path to the data file, relative or absolute
+ * @return the path to record in the Delta log
+ */
+ public static String getPathForDeltaLog(String tableBasePath, String
dataFilePath) {
+ return isUnderBasePath(tableBasePath, dataFilePath)
+ ? PathUtils.getRelativePath(dataFilePath, tableBasePath)
+ : dataFilePath;
+ }
+
+ private static boolean isUnderBasePath(String tableBasePath, String
dataFilePath) {
+ URI file = new Path(dataFilePath).toUri();
Review Comment:
converting to `Path` objects incurs some validation of the path itself which
can add some unexpected latency at scale. Any chance we can pass in as URIs or
do a quick check for `:` in the path as an optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
