[ https://issues.apache.org/jira/browse/DRILL-3867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16058075#comment-16058075 ]
ASF GitHub Bot commented on DRILL-3867: --------------------------------------- Github user vdiravka commented on a diff in the pull request: https://github.com/apache/drill/pull/824#discussion_r123339909 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/Metadata.java --- @@ -748,6 +771,22 @@ public ParquetTableMetadataDirs(List<String> directories) { return directories; } + /** If directories list contains relative paths, update it to absolute ones + * @param baseDir base parent directory + */ + @JsonIgnore public void updateRelativePaths(Path baseDir) { + if (!directories.isEmpty()) { + // It is enough to check the first path to decide if updating needed + if (!new Path(directories.get(0)).isAbsolute()) { --- End diff -- It is possible to replace String with Path for directories paths due to implementing custom `JsonSerializer<Path>` and `JsonDeserializer<Path>`. But then it will be necessary to convert every `Path` from lists back into `String`, because a String paths are used in a lot of places: `FileSelection`, `Metadata`, `ParquetGroupScan`, `ReadEntryWithPath`, `FileWork`, `FormatSelection`, `FormatPlugin`, `PartitionLocation`and so on. I am totally agree with replacing `String` with `Path` requirement. But it should be done not only for parquet and in context of separate jira. I am going to create it. > Store relative paths in metadata file > ------------------------------------- > > Key: DRILL-3867 > URL: https://issues.apache.org/jira/browse/DRILL-3867 > Project: Apache Drill > Issue Type: Bug > Components: Metadata > Affects Versions: 1.2.0 > Reporter: Rahul Challapalli > Assignee: Vitalii Diravka > Fix For: Future > > > git.commit.id.abbrev=cf4f745 > git.commit.time=29.09.2015 @ 23\:19\:52 UTC > The below sequence of steps reproduces the issue > 1. Create the cache file > {code} > 0: jdbc:drill:zk=10.10.103.60:5181> refresh table metadata > dfs.`/drill/testdata/metadata_caching/lineitem`; > +-------+-------------------------------------------------------------------------------------+ > | ok | summary > | > +-------+-------------------------------------------------------------------------------------+ > | true | Successfully updated metadata for table > /drill/testdata/metadata_caching/lineitem. | > +-------+-------------------------------------------------------------------------------------+ > 1 row selected (1.558 seconds) > {code} > 2. Move the directory > {code} > hadoop fs -mv /drill/testdata/metadata_caching/lineitem /drill/ > {code} > 3. Now run a query on top of it > {code} > 0: jdbc:drill:zk=10.10.103.60:5181> select * from dfs.`/drill/lineitem` limit > 1; > Error: SYSTEM ERROR: FileNotFoundException: Requested file > maprfs:///drill/testdata/metadata_caching/lineitem/2006/1 does not exist. > [Error Id: b456d912-57a0-4690-a44b-140d4964903e on pssc-66.qa.lab:31010] > (state=,code=0) > {code} > This is obvious given the fact that we are storing absolute file paths in the > cache file -- This message was sent by Atlassian JIRA (v6.4.14#64029)