[
https://issues.apache.org/jira/browse/PIG-5025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15608615#comment-15608615
]
Adam Szita commented on PIG-5025:
---------------------------------
Actually I think we should go with [^PIG-5025.1.patch].
I checked this test today with Windows and it was actually passing without any
of these touches (so using /tmp). It seems like this test doesn't even put
anything into the given working directory on the local FS and the dir doesn't
even have to exist.
Most tests merely just check if the supplied path matches the one found in the
Load operator after creating the query plan (see checkLoadPath)
Those that do write and read files (see those calling testLoadingMultipleFiles)
only do that in HDFS mode and they clean up after their work too.
So the only problem is the case when it exists and contains a file with : in
its name
> Improve TestLoad.java: use own separated folder under /tmp
> ----------------------------------------------------------
>
> Key: PIG-5025
> URL: https://issues.apache.org/jira/browse/PIG-5025
> Project: Pig
> Issue Type: Improvement
> Reporter: Adam Szita
> Assignee: Adam Szita
> Priority: Minor
> Attachments: PIG-5025.1.patch, PIG-5025.2.patch, PIG-5025.patch
>
>
> Test cases testCommaSeparatedString2 and testGlobChars may fail if for some
> reason files (from any other sources) in /tmp have : (colon) in the
> filenames. This is because HDFS doesn't support colon since it has its own
> URI handling. Exception below.
> I propose we separate the working dir of these tests to use their own folder
> in /tmp.
> Failed to parse: java.net.URISyntaxException: Relative path in absolute URI:
> t:2sTest.txt
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:198)
> at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:317)
> at org.apache.pig.test.TestLoad.checkLoadPath(TestLoad.java:299)
> at
> org.apache.pig.test.TestLoad.testCommaSeparatedString2(TestLoad.java:189)
> Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
> Relative path in absolute URI: t:2sTest.txt
> at org.apache.hadoop.fs.Path.initialize(Path.java:206)
> at org.apache.hadoop.fs.Path.<init>(Path.java:172)
> at org.apache.hadoop.fs.Path.<init>(Path.java:94)
> at org.apache.hadoop.fs.Globber.doGlob(Globber.java:260)
> at org.apache.hadoop.fs.Globber.glob(Globber.java:151)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1637)
> at
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.asCollection(HDataStorage.java:215)
> at
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.asCollection(HDataStorage.java:41)
> at
> org.apache.pig.builtin.JsonMetadata.findMetaFile(JsonMetadata.java:119)
> at org.apache.pig.builtin.JsonMetadata.getSchema(JsonMetadata.java:191)
> at org.apache.pig.builtin.PigStorage.getSchema(PigStorage.java:518)
> at
> org.apache.pig.newplan.logical.relational.LOLoad.getSchemaFromMetaData(LOLoad.java:175)
> at
> org.apache.pig.newplan.logical.relational.LOLoad.<init>(LOLoad.java:89)
> at
> org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:866)
> at
> org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3568)
> at
> org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)
> at
> org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
> at
> org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
> at
> org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188)
> Caused by: java.net.URISyntaxException: Relative path in absolute URI:
> t:2sTest.txt
> at java.net.URI.checkPath(URI.java:1823)
> at java.net.URI.<init>(URI.java:745)
> at org.apache.hadoop.fs.Path.initialize(Path.java:203)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
