[jira] [Commented] (HIVE-3451) map-reduce jobs does not work for a partition containing sub-directories

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13548071#comment-13548071
 ] 

Hudson commented on HIVE-3451:
--

Integrated in Hive-trunk-hadoop2 #54 (See 
[https://builds.apache.org/job/Hive-trunk-hadoop2/54/])
HIVE-3451 map-reduce jobs does not work for a partition containing 
sub-directories
(Gang Tim Liu via namit) (Revision 1389704)

 Result = ABORTED
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1389704
Files : 
* /hive/trunk/ql/src/test/queries/clientpositive/recursive_dir.q
* /hive/trunk/ql/src/test/results/clientpositive/recursive_dir.q.out


> map-reduce jobs does not work for a partition containing sub-directories
> 
>
> Key: HIVE-3451
> URL: https://issues.apache.org/jira/browse/HIVE-3451
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Gang Tim Liu
> Fix For: 0.10.0
>
> Attachments: HIVE-3451.patch
>
>
> Consider the following test:
> -- The test verifies that sub-directories are supported for versions of hadoop
> -- where MAPREDUCE-1501 is fixed. So, enable this test only for hadoop 23.
> -- INCLUDE_HADOOP_MAJOR_VERSIONS(0.23)
> CREATE TABLE fact_daily(x int) PARTITIONED BY (ds STRING);
> CREATE TABLE fact_tz(x int) PARTITIONED BY (ds STRING, hr STRING) 
> LOCATION 'pfile:${system:test.tmp.dir}/fact_tz';
> INSERT OVERWRITE TABLE fact_tz PARTITION (ds='1', hr='1')
> SELECT key+11 FROM src WHERE key=484;
> ALTER TABLE fact_daily SET TBLPROPERTIES('EXTERNAL'='TRUE');
> ALTER TABLE fact_daily ADD PARTITION (ds='1')
> LOCATION 'pfile:${system:test.tmp.dir}/fact_tz/ds=1';
> set mapred.input.dir.recursive=true;
> SELECT * FROM fact_daily WHERE ds='1';
> SELECT count(1) FROM fact_daily WHERE ds='1';
> Say, the above file was named: recursive_dir.q
> and we ran the test for hadoop 23:
> by executing:
> ant test -Dhadoop.mr.rev=23 -Dtest.print.classpath=true 
> -Dhadoop.version=2.0.0-alpha -Dhadoop.security.version=2.0.0-alpha 
> -Dtestcase=TestCliDriver -Dqfile=recursive_dir.q
> The select * from the table works fine, but the last command does not work
> since it requires a map-reduce job.
> This will prevent other features which are creating sub-directories to add
> any tests which requires a map-reduce job. The work-around is to issue
> queries which do not require map-reduce jobs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3451) map-reduce jobs does not work for a partition containing sub-directories

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462591#comment-13462591
 ] 

Hudson commented on HIVE-3451:
--

Integrated in Hive-trunk-h0.21 #1695 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1695/])
HIVE-3451 map-reduce jobs does not work for a partition containing 
sub-directories
(Gang Tim Liu via namit) (Revision 1389704)

 Result = ABORTED
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1389704
Files : 
* /hive/trunk/ql/src/test/queries/clientpositive/recursive_dir.q
* /hive/trunk/ql/src/test/results/clientpositive/recursive_dir.q.out


> map-reduce jobs does not work for a partition containing sub-directories
> 
>
> Key: HIVE-3451
> URL: https://issues.apache.org/jira/browse/HIVE-3451
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Gang Tim Liu
> Fix For: 0.10.0
>
> Attachments: HIVE-3451.patch
>
>
> Consider the following test:
> -- The test verifies that sub-directories are supported for versions of hadoop
> -- where MAPREDUCE-1501 is fixed. So, enable this test only for hadoop 23.
> -- INCLUDE_HADOOP_MAJOR_VERSIONS(0.23)
> CREATE TABLE fact_daily(x int) PARTITIONED BY (ds STRING);
> CREATE TABLE fact_tz(x int) PARTITIONED BY (ds STRING, hr STRING) 
> LOCATION 'pfile:${system:test.tmp.dir}/fact_tz';
> INSERT OVERWRITE TABLE fact_tz PARTITION (ds='1', hr='1')
> SELECT key+11 FROM src WHERE key=484;
> ALTER TABLE fact_daily SET TBLPROPERTIES('EXTERNAL'='TRUE');
> ALTER TABLE fact_daily ADD PARTITION (ds='1')
> LOCATION 'pfile:${system:test.tmp.dir}/fact_tz/ds=1';
> set mapred.input.dir.recursive=true;
> SELECT * FROM fact_daily WHERE ds='1';
> SELECT count(1) FROM fact_daily WHERE ds='1';
> Say, the above file was named: recursive_dir.q
> and we ran the test for hadoop 23:
> by executing:
> ant test -Dhadoop.mr.rev=23 -Dtest.print.classpath=true 
> -Dhadoop.version=2.0.0-alpha -Dhadoop.security.version=2.0.0-alpha 
> -Dtestcase=TestCliDriver -Dqfile=recursive_dir.q
> The select * from the table works fine, but the last command does not work
> since it requires a map-reduce job.
> This will prevent other features which are creating sub-directories to add
> any tests which requires a map-reduce job. The work-around is to issue
> queries which do not require map-reduce jobs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3451) map-reduce jobs does not work for a partition containing sub-directories

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462434#comment-13462434
 ] 

Namit Jain commented on HIVE-3451:
--

+1

> map-reduce jobs does not work for a partition containing sub-directories
> 
>
> Key: HIVE-3451
> URL: https://issues.apache.org/jira/browse/HIVE-3451
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Gang Tim Liu
> Fix For: 0.10.0
>
> Attachments: HIVE-3451.patch
>
>
> Consider the following test:
> -- The test verifies that sub-directories are supported for versions of hadoop
> -- where MAPREDUCE-1501 is fixed. So, enable this test only for hadoop 23.
> -- INCLUDE_HADOOP_MAJOR_VERSIONS(0.23)
> CREATE TABLE fact_daily(x int) PARTITIONED BY (ds STRING);
> CREATE TABLE fact_tz(x int) PARTITIONED BY (ds STRING, hr STRING) 
> LOCATION 'pfile:${system:test.tmp.dir}/fact_tz';
> INSERT OVERWRITE TABLE fact_tz PARTITION (ds='1', hr='1')
> SELECT key+11 FROM src WHERE key=484;
> ALTER TABLE fact_daily SET TBLPROPERTIES('EXTERNAL'='TRUE');
> ALTER TABLE fact_daily ADD PARTITION (ds='1')
> LOCATION 'pfile:${system:test.tmp.dir}/fact_tz/ds=1';
> set mapred.input.dir.recursive=true;
> SELECT * FROM fact_daily WHERE ds='1';
> SELECT count(1) FROM fact_daily WHERE ds='1';
> Say, the above file was named: recursive_dir.q
> and we ran the test for hadoop 23:
> by executing:
> ant test -Dhadoop.mr.rev=23 -Dtest.print.classpath=true 
> -Dhadoop.version=2.0.0-alpha -Dhadoop.security.version=2.0.0-alpha 
> -Dtestcase=TestCliDriver -Dqfile=recursive_dir.q
> The select * from the table works fine, but the last command does not work
> since it requires a map-reduce job.
> This will prevent other features which are creating sub-directories to add
> any tests which requires a map-reduce job. The work-around is to issue
> queries which do not require map-reduce jobs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3451) map-reduce jobs does not work for a partition containing sub-directories

[Gang Tim Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462209#comment-13462209
 ] 

Gang Tim Liu commented on HIVE-3451:


ready for review https://reviews.facebook.net/D5637

> map-reduce jobs does not work for a partition containing sub-directories
> 
>
> Key: HIVE-3451
> URL: https://issues.apache.org/jira/browse/HIVE-3451
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Gang Tim Liu
>
> Consider the following test:
> -- The test verifies that sub-directories are supported for versions of hadoop
> -- where MAPREDUCE-1501 is fixed. So, enable this test only for hadoop 23.
> -- INCLUDE_HADOOP_MAJOR_VERSIONS(0.23)
> CREATE TABLE fact_daily(x int) PARTITIONED BY (ds STRING);
> CREATE TABLE fact_tz(x int) PARTITIONED BY (ds STRING, hr STRING) 
> LOCATION 'pfile:${system:test.tmp.dir}/fact_tz';
> INSERT OVERWRITE TABLE fact_tz PARTITION (ds='1', hr='1')
> SELECT key+11 FROM src WHERE key=484;
> ALTER TABLE fact_daily SET TBLPROPERTIES('EXTERNAL'='TRUE');
> ALTER TABLE fact_daily ADD PARTITION (ds='1')
> LOCATION 'pfile:${system:test.tmp.dir}/fact_tz/ds=1';
> set mapred.input.dir.recursive=true;
> SELECT * FROM fact_daily WHERE ds='1';
> SELECT count(1) FROM fact_daily WHERE ds='1';
> Say, the above file was named: recursive_dir.q
> and we ran the test for hadoop 23:
> by executing:
> ant test -Dhadoop.mr.rev=23 -Dtest.print.classpath=true 
> -Dhadoop.version=2.0.0-alpha -Dhadoop.security.version=2.0.0-alpha 
> -Dtestcase=TestCliDriver -Dqfile=recursive_dir.q
> The select * from the table works fine, but the last command does not work
> since it requires a map-reduce job.
> This will prevent other features which are creating sub-directories to add
> any tests which requires a map-reduce job. The work-around is to issue
> queries which do not require map-reduce jobs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira