[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880211#comment-13880211 ] Shuaishuai Nie commented on HIVE-5446: -- Hi [~xuefuz], I tested it on my local machine and the unit test external_table_with_space_in_location_path.q failed. After I added the missing file, the test pass. I checked HIVE QA test report here: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/. From the TestMinimrCliDriver test result you can see lots of minimr tests are missing if you compare with the list of minimr.query.files in the pom.xml in itests/qtest. If this is the problem, I think we need to open a JIRA for this. Thanks Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Fix For: 0.13.0 Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch, HIVE-5446.3.patch, HIVE-5446.4.patch, HIVE-5446.5.patch, HIVE-5446.7.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13880324#comment-13880324 ] Xuefu Zhang commented on HIVE-5446: --- Thanks [~shuainie] for the update. I have committed the data file and created HIVE-6293 to track the ptest issue. Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Fix For: 0.13.0 Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch, HIVE-5446.3.patch, HIVE-5446.4.patch, HIVE-5446.5.patch, HIVE-5446.7.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879391#comment-13879391 ] Jason Dere commented on HIVE-5446: -- Hey, is the commit for this fix missing file data/files/ext_test_space/folder+with space/data.txt? I don't see it on git. I would have thought that any of the pre commit tests run after this commit would have failed on this test since the required data file is missing. Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Fix For: 0.13.0 Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch, HIVE-5446.3.patch, HIVE-5446.4.patch, HIVE-5446.5.patch, HIVE-5446.7.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879396#comment-13879396 ] Shuaishuai Nie commented on HIVE-5446: -- Hi [~xuefuz], seems like the test file is missing. Can you help commit? Thanks Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Fix For: 0.13.0 Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch, HIVE-5446.3.patch, HIVE-5446.4.patch, HIVE-5446.5.patch, HIVE-5446.7.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879446#comment-13879446 ] Xuefu Zhang commented on HIVE-5446: --- Guys, Please help me understand. If the data file is missing, then I would expect that the tests relying on it would fail. Let's fix this once for all. [~shuainie] Could you please check why the new tests didn't fail? If we can confirm that the data file is only the problem, I can commit it. Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Fix For: 0.13.0 Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch, HIVE-5446.3.patch, HIVE-5446.4.patch, HIVE-5446.5.patch, HIVE-5446.7.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13868307#comment-13868307 ] Xuefu Zhang commented on HIVE-5446: --- The failure above seems transient and unrelated. Patch committed to trunk. Thanks to Shuaishuai for the contribution. Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Fix For: 0.13.0 Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch, HIVE-5446.3.patch, HIVE-5446.4.patch, HIVE-5446.5.patch, HIVE-5446.7.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13868315#comment-13868315 ] Shuaishuai Nie commented on HIVE-5446: -- Thanks [~xuefuz]! Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Fix For: 0.13.0 Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch, HIVE-5446.3.patch, HIVE-5446.4.patch, HIVE-5446.5.patch, HIVE-5446.7.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[
https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13866719#comment-13866719
]
Hive QA commented on HIVE-5446:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12622085/HIVE-5446.5.patch
{color:green}SUCCESS:{color} +1 4904 tests passed
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/839/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/839/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12622085
Hive can CREATE an external table but not SELECT from it when file path have
spaces
---
Key: HIVE-5446
URL: https://issues.apache.org/jira/browse/HIVE-5446
Project: Hive
Issue Type: Bug
Reporter: Shuaishuai Nie
Assignee: Shuaishuai Nie
Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch, HIVE-5446.3.patch,
HIVE-5446.4.patch, HIVE-5446.5.patch
Create external table table1 (age int,
gender string, totBil float,
dirBill float, alkphos int,
sgpt int, sgot int, totProt float,
aLB float, aG float, sel int)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
STORED AS TEXTFILE
LOCATION 'hdfs://namenodehost:9000/hive newtable';
select * from table1;
return nothing even there is file in the target folder
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13867229#comment-13867229 ] Xuefu Zhang commented on HIVE-5446: --- +1 Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch, HIVE-5446.3.patch, HIVE-5446.4.patch, HIVE-5446.5.patch, HIVE-5446.7.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[
https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13867582#comment-13867582
]
Hive QA commented on HIVE-5446:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12622272/HIVE-5446.7.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 4917 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/848/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/848/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12622272
Hive can CREATE an external table but not SELECT from it when file path have
spaces
---
Key: HIVE-5446
URL: https://issues.apache.org/jira/browse/HIVE-5446
Project: Hive
Issue Type: Bug
Reporter: Shuaishuai Nie
Assignee: Shuaishuai Nie
Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch, HIVE-5446.3.patch,
HIVE-5446.4.patch, HIVE-5446.5.patch, HIVE-5446.7.patch
Create external table table1 (age int,
gender string, totBil float,
dirBill float, alkphos int,
sgpt int, sgot int, totProt float,
aLB float, aG float, sel int)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
STORED AS TEXTFILE
LOCATION 'hdfs://namenodehost:9000/hive newtable';
select * from table1;
return nothing even there is file in the target folder
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13866280#comment-13866280 ] Xuefu Zhang commented on HIVE-5446: --- Thanks for the update. Some minor comments on the review boards. Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch, HIVE-5446.3.patch, HIVE-5446.4.patch, HIVE-5446.5.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[
https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13864005#comment-13864005
]
Hive QA commented on HIVE-5446:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12621699/HIVE-5446.3.patch
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/817/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/817/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n '' ]]
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-817/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/Context.java'
++ awk '{print $2}'
++ egrep -v '^X|^Performing status on external'
++ svn status --no-ignore
+ rm -rf target datanucleus.log ant/target shims/target shims/0.20/target
shims/0.20S/target shims/0.23/target shims/aggregator/target
shims/common/target shims/common-secure/target packaging/target
hbase-handler/target testutils/target jdbc/target metastore/target
itests/target itests/hcatalog-unit/target itests/test-serde/target
itests/qtest/target itests/hive-unit/target itests/custom-serde/target
itests/util/target hcatalog/target hcatalog/storage-handlers/hbase/target
hcatalog/server-extensions/target hcatalog/core/target
hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target
hcatalog/hcatalog-pig-adapter/target hwi/target common/target common/src/gen
contrib/target service/target serde/target beeline/target odbc/target
cli/target ql/dependency-reduced-pom.xml ql/target
+ svn update
Aql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRpad.java
Aql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLTrim.java
Aql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRTrim.java
Aql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFTrim.java
Aql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFLpad.java
U
ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizationContext.java
Dql/src/java/org/apache/hadoop/hive/ql/udf/UDFTrim.java
Dql/src/java/org/apache/hadoop/hive/ql/udf/UDFLpad.java
Dql/src/java/org/apache/hadoop/hive/ql/udf/UDFRpad.java
Dql/src/java/org/apache/hadoop/hive/ql/udf/UDFLTrim.java
Dql/src/java/org/apache/hadoop/hive/ql/udf/UDFRTrim.java
Aql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRTrim.java
Aql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLTrim.java
Aql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBasePad.java
Aql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFTrim.java
Aql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseTrim.java
Aql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFLpad.java
Aql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRpad.java
Uql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java
Uql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
Fetching external item into 'hcatalog/src/test/e2e/harness'
Updated external to revision 1556142.
Updated to revision 1556142.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12621699
Hive can CREATE an external table but not SELECT from it when file path have
spaces
---
Key: HIVE-5446
URL: https://issues.apache.org/jira/browse/HIVE-5446
Project: Hive
Issue
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13864605#comment-13864605 ] Thejas M Nair commented on HIVE-5446: - Thanks for catching that [~xuefuz] ! Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch, HIVE-5446.3.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13864990#comment-13864990 ] Shuaishuai Nie commented on HIVE-5446: -- Hi [~xuefuz], [~thejas]. The patch HIVE-5446.4 deal with the scenario when + or exist in the file path. I also modified the unit test to cover both cases. Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch, HIVE-5446.3.patch, HIVE-5446.4.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13864995#comment-13864995 ] Xuefu Zhang commented on HIVE-5446: --- Could you please create a review board entry for your code changes? Thanks. Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch, HIVE-5446.3.patch, HIVE-5446.4.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13865019#comment-13865019 ] Shuaishuai Nie commented on HIVE-5446: -- Create review board for this JIRA here: https://reviews.apache.org/r/16727/ Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch, HIVE-5446.3.patch, HIVE-5446.4.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13865065#comment-13865065 ] Xuefu Zhang commented on HIVE-5446: --- [~shuainie] Change looks good. I have a couple comments on the review board. Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch, HIVE-5446.3.patch, HIVE-5446.4.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13863502#comment-13863502 ] Thejas M Nair commented on HIVE-5446: - +1 . Looks good. There is a typo in the error message Invalid UIR , but I can take care of that while committing. Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13863571#comment-13863571 ] Xuefu Zhang commented on HIVE-5446: --- [~shuainie] If you don't mind, could you quickly test a case where the file name contains '+' sign? I am a little concerned that URI.decode() might screw it up, but I'm not sure. It would be perfect if this also works. Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch, HIVE-5446.3.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13863685#comment-13863685 ] Shuaishuai Nie commented on HIVE-5446: -- Hi [~xuefuz]. I modified my unit test and verified that it will work when the path contains '+' sign with my fix. Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch, HIVE-5446.3.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13863688#comment-13863688 ] Shuaishuai Nie commented on HIVE-5446: -- Sorry, just ignore my former comment. The test environment is messed up. I am validating the scenario. Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch, HIVE-5446.3.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13863757#comment-13863757 ] Shuaishuai Nie commented on HIVE-5446: -- Seems the decode function will change character + to space. Need to find another solution for the problem. [~xuefuz] does the fix you made in HIVE-6048 apply to this scenario? Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch, HIVE-5446.3.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13863850#comment-13863850 ] Xuefu Zhang commented on HIVE-5446: --- The fix for HIVE-6048 was not to use URI.decode(). I think this problem is not about the way that table location is stored, but about the way when table location is used when the table is accessed. Storing the value of uri.toString() as the table metadata is fine. When the value is used, the right way should be to deserialize the value back to a URI instance, make a Path instance out of the URI instance using Path(URI uri), and then use the Path instance to access the file. This is just my speculation as I didn't check the code, but I suppose it should fix the problem (plus other encoding/decoding problems as we saw in other JIRAs). Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch, HIVE-5446.3.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[
https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13861663#comment-13861663
]
Hive QA commented on HIVE-5446:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12621210/HIVE-5446.2.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 4873 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_single_sourced_multi_insert
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_reduce_deduplicate
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/795/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/795/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12621210
Hive can CREATE an external table but not SELECT from it when file path have
spaces
---
Key: HIVE-5446
URL: https://issues.apache.org/jira/browse/HIVE-5446
Project: Hive
Issue Type: Bug
Reporter: Shuaishuai Nie
Assignee: Shuaishuai Nie
Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch
Create external table table1 (age int,
gender string, totBil float,
dirBill float, alkphos int,
sgpt int, sgot int, totProt float,
aLB float, aG float, sel int)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
STORED AS TEXTFILE
LOCATION 'hdfs://namenodehost:9000/hive newtable';
select * from table1;
return nothing even there is file in the target folder
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13861700#comment-13861700 ] Xuefu Zhang commented on HIVE-5446: --- With a series of standardizing path in Hive such as via HIVE6048, HIVE-6121, etc, URI decoding might become unnecessary. At least, it's worth reproducing the problem with the latest trunk and fix it accordingly if the problem remains. Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[ https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13861744#comment-13861744 ] Shuaishuai Nie commented on HIVE-5446: -- Hi [~xuefuz], thanks for the advice. I run the unit test with the latest trunk and the problem still exist, so I think we still need the fix. Also I have validated that the failed tests in HIVE QA is not related to the patch. Hive can CREATE an external table but not SELECT from it when file path have spaces --- Key: HIVE-5446 URL: https://issues.apache.org/jira/browse/HIVE-5446 Project: Hive Issue Type: Bug Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Attachments: HIVE-5446.1.patch, HIVE-5446.2.patch Create external table table1 (age int, gender string, totBil float, dirBill float, alkphos int, sgpt int, sgot int, totProt float, aLB float, aG float, sel int) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION 'hdfs://namenodehost:9000/hive newtable'; select * from table1; return nothing even there is file in the target folder -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HIVE-5446) Hive can CREATE an external table but not SELECT from it when file path have spaces
[
https://issues.apache.org/jira/browse/HIVE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13786865#comment-13786865
]
Hive QA commented on HIVE-5446:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12606828/HIVE-5446.1.patch
{color:green}SUCCESS:{color} +1 4052 tests passed
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1036/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1036/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
Hive can CREATE an external table but not SELECT from it when file path have
spaces
---
Key: HIVE-5446
URL: https://issues.apache.org/jira/browse/HIVE-5446
Project: Hive
Issue Type: Bug
Reporter: Shuaishuai Nie
Assignee: Shuaishuai Nie
Attachments: HIVE-5446.1.patch
Create external table table1 (age int,
gender string, totBil float,
dirBill float, alkphos int,
sgpt int, sgot int, totProt float,
aLB float, aG float, sel int)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
STORED AS TEXTFILE
LOCATION 'hdfs://namenodehost:9000/hive newtable';
select * from table1;
return nothing even there is file in the target folder
--
This message was sent by Atlassian JIRA
(v6.1#6144)
