Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/877#discussion_r128502337
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/Metadata.java
---
@@ -1851,9 +1860,81 @@ private static String relativize(String baseDir,
String childPath) {
.relativize(fullPathWithoutSchemeAndAuthority.toUri()));
if (relativeFilePath.isAbsolute()) {
throw new IllegalStateException(String.format("Path %s is not a
subpath of %s.",
- basePathWithoutSchemeAndAuthority.toUri().toString(),
fullPathWithoutSchemeAndAuthority.toUri().toString()));
+ basePathWithoutSchemeAndAuthority.toUri().getPath(),
fullPathWithoutSchemeAndAuthority.toUri().getPath()));
+ }
+ return relativeFilePath.toUri().getPath();
+ }
+ }
+
+ /**
+ * Used to identify metadata version by the deserialization
"metadata_version" first property
+ * from the metadata cache file
+ */
+ public static class MetadataVersion {
+ @JsonProperty("metadata_version")
+ public String textVersion;
+
+ /**
+ * Supported metadata versions.
+ * Note: keep them synchronized with {@link ParquetTableMetadataBase}
versions
+ */
+ enum Versions {
+ v1(Constants.V1),
+ v2(Constants.V2),
+ v3(Constants.V3),
+ v3_1(Constants.V3_1);
+
+ private final String version;
+
+ Versions(String version) {
+ this.version = version;
+ }
+
+ public String getVersion() {
+ return version;
+ }
+
+ public static Versions fromString(String version) {
+ for (Versions v : Versions.values()) {
+ if (v.version.equalsIgnoreCase(version)) {
+ return v;
+ }
+ }
+ return null;
+ }
+
+ public static class Constants {
+ public static final String V1 = "v1";
+ public static final String V2 = "v2";
+ public static final String V3 = "v3";
+ public static final String V3_1 = "v3_1";
+ }
+ }
+
+ /**
+ * @param fs current file system
+ * @param path of metadata cache file
+ * @return true if metadata version is supported, false otherwise
+ * @throws IOException if parquet metadata can't be deserialized from
the json file
+ */
+ public static boolean isVersionSupported(FileSystem fs, Path path)
throws IOException {
--- End diff --
That new deserialization persistence class for reading `metadata` version
is removed.
For now we try to deserialize the `metadata` file and in case of getting
any inheritor of `JsonProcessingException` ( for example `JsonMappingException`
or `JsonParseException`) the `metadata` will be null and will be ignored (with
appropriate logging). To avoid of reading such corrupted or unsupported file
again that status is stored in `metadata context`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
