[
https://issues.apache.org/jira/browse/DRILL-3867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16058539#comment-16058539
]
ASF GitHub Bot commented on DRILL-3867:
---------------------------------------
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/824#discussion_r123398337
--- Diff: exec/java-exec/src/test/java/org/apache/drill/BaseTestQuery.java
---
@@ -593,4 +610,49 @@ private void convert(List<QueryDataBatch> batches)
throws SchemaChangeException
}
}
}
+
+ private static String replaceWorkingPathInString(String orig) {
+ return orig.replaceAll(Pattern.quote("[WORKING_PATH]"),
Matcher.quoteReplacement(TestTools.getWorkingPath()));
+ }
+
+ protected static void copyDirectoryIntoTempSpace(String resourcesDir)
throws IOException {
+ copyDirectoryIntoTempSpace(resourcesDir, null);
+ }
+
+ protected static void copyDirectoryIntoTempSpace(String resourcesDir,
String destinationSubDir) throws IOException {
+ Path destination = destinationSubDir != null ? new
Path(getDfsTestTmpSchemaLocation(), destinationSubDir)
+ : new Path(getDfsTestTmpSchemaLocation());
+ fs.copyFromLocalFile(
+ new Path(replaceWorkingPathInString(resourcesDir)),
+ destination);
+ }
+
+ /**
+ * Metadata cache files include full paths to the files that have been
scanned.
--- End diff --
For older files with the marker, should we just replace the marker to be
relative and take advantage of this improvement? Can that be done without
having to edit the old files?
> Store relative paths in metadata file
> -------------------------------------
>
> Key: DRILL-3867
> URL: https://issues.apache.org/jira/browse/DRILL-3867
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
> Affects Versions: 1.2.0
> Reporter: Rahul Challapalli
> Assignee: Vitalii Diravka
> Fix For: Future
>
>
> git.commit.id.abbrev=cf4f745
> git.commit.time=29.09.2015 @ 23\:19\:52 UTC
> The below sequence of steps reproduces the issue
> 1. Create the cache file
> {code}
> 0: jdbc:drill:zk=10.10.103.60:5181> refresh table metadata
> dfs.`/drill/testdata/metadata_caching/lineitem`;
> +-------+-------------------------------------------------------------------------------------+
> | ok | summary
> |
> +-------+-------------------------------------------------------------------------------------+
> | true | Successfully updated metadata for table
> /drill/testdata/metadata_caching/lineitem. |
> +-------+-------------------------------------------------------------------------------------+
> 1 row selected (1.558 seconds)
> {code}
> 2. Move the directory
> {code}
> hadoop fs -mv /drill/testdata/metadata_caching/lineitem /drill/
> {code}
> 3. Now run a query on top of it
> {code}
> 0: jdbc:drill:zk=10.10.103.60:5181> select * from dfs.`/drill/lineitem` limit
> 1;
> Error: SYSTEM ERROR: FileNotFoundException: Requested file
> maprfs:///drill/testdata/metadata_caching/lineitem/2006/1 does not exist.
> [Error Id: b456d912-57a0-4690-a44b-140d4964903e on pssc-66.qa.lab:31010]
> (state=,code=0)
> {code}
> This is obvious given the fact that we are storing absolute file paths in the
> cache file
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
