[jira] [Updated] (HIVE-4240) optimize hive.enforce.bucketing and hive.enforce sorting insert
[ https://issues.apache.org/jira/browse/HIVE-4240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-4240: - Resolution: Fixed Fix Version/s: 0.11.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed. Thanks Gang Tim Liu > optimize hive.enforce.bucketing and hive.enforce sorting insert > --- > > Key: HIVE-4240 > URL: https://issues.apache.org/jira/browse/HIVE-4240 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Namit Jain > Fix For: 0.11.0 > > Attachments: hive.4240.1.patch, hive.4240.2.patch, hive.4240.3.patch, > hive.4240.4.patch, hive.4240.5.patch, hive.4240.5.patch-nohcat > > > Consider the following scenario: > set hive.optimize.bucketmapjoin = true; > set hive.optimize.bucketmapjoin.sortedmerge = true; > set hive.input.format = > org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat; > set hive.enforce.bucketing=true; > set hive.enforce.sorting=true; > set hive.exec.reducers.max = 1; > set hive.merge.mapfiles=false; > set hive.merge.mapredfiles=false; > -- Create two bucketed and sorted tables > CREATE TABLE test_table1 (key INT, value STRING) PARTITIONED BY (ds STRING) > CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; > CREATE TABLE test_table2 (key INT, value STRING) PARTITIONED BY (ds STRING) > CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; > FROM src > INSERT OVERWRITE TABLE test_table1 PARTITION (ds = '1') SELECT *; > -- Insert data into the bucketed table by selecting from another bucketed > table > -- This should be a map-only operation > INSERT OVERWRITE TABLE test_table2 PARTITION (ds = '1') > SELECT a.key, a.value FROM test_table1 a WHERE a.ds = '1'; > We should not need a reducer to perform the above operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4240) optimize hive.enforce.bucketing and hive.enforce sorting insert
[ https://issues.apache.org/jira/browse/HIVE-4240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-4240: - Attachment: hive.4240.5.patch-nohcat > optimize hive.enforce.bucketing and hive.enforce sorting insert > --- > > Key: HIVE-4240 > URL: https://issues.apache.org/jira/browse/HIVE-4240 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Namit Jain > Attachments: hive.4240.1.patch, hive.4240.2.patch, hive.4240.3.patch, > hive.4240.4.patch, hive.4240.5.patch, hive.4240.5.patch-nohcat > > > Consider the following scenario: > set hive.optimize.bucketmapjoin = true; > set hive.optimize.bucketmapjoin.sortedmerge = true; > set hive.input.format = > org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat; > set hive.enforce.bucketing=true; > set hive.enforce.sorting=true; > set hive.exec.reducers.max = 1; > set hive.merge.mapfiles=false; > set hive.merge.mapredfiles=false; > -- Create two bucketed and sorted tables > CREATE TABLE test_table1 (key INT, value STRING) PARTITIONED BY (ds STRING) > CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; > CREATE TABLE test_table2 (key INT, value STRING) PARTITIONED BY (ds STRING) > CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; > FROM src > INSERT OVERWRITE TABLE test_table1 PARTITION (ds = '1') SELECT *; > -- Insert data into the bucketed table by selecting from another bucketed > table > -- This should be a map-only operation > INSERT OVERWRITE TABLE test_table2 PARTITION (ds = '1') > SELECT a.key, a.value FROM test_table1 a WHERE a.ds = '1'; > We should not need a reducer to perform the above operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4240) optimize hive.enforce.bucketing and hive.enforce sorting insert
[ https://issues.apache.org/jira/browse/HIVE-4240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-4240: - Attachment: hive.4240.5.patch > optimize hive.enforce.bucketing and hive.enforce sorting insert > --- > > Key: HIVE-4240 > URL: https://issues.apache.org/jira/browse/HIVE-4240 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Namit Jain > Attachments: hive.4240.1.patch, hive.4240.2.patch, hive.4240.3.patch, > hive.4240.4.patch, hive.4240.5.patch > > > Consider the following scenario: > set hive.optimize.bucketmapjoin = true; > set hive.optimize.bucketmapjoin.sortedmerge = true; > set hive.input.format = > org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat; > set hive.enforce.bucketing=true; > set hive.enforce.sorting=true; > set hive.exec.reducers.max = 1; > set hive.merge.mapfiles=false; > set hive.merge.mapredfiles=false; > -- Create two bucketed and sorted tables > CREATE TABLE test_table1 (key INT, value STRING) PARTITIONED BY (ds STRING) > CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; > CREATE TABLE test_table2 (key INT, value STRING) PARTITIONED BY (ds STRING) > CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; > FROM src > INSERT OVERWRITE TABLE test_table1 PARTITION (ds = '1') SELECT *; > -- Insert data into the bucketed table by selecting from another bucketed > table > -- This should be a map-only operation > INSERT OVERWRITE TABLE test_table2 PARTITION (ds = '1') > SELECT a.key, a.value FROM test_table1 a WHERE a.ds = '1'; > We should not need a reducer to perform the above operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4240) optimize hive.enforce.bucketing and hive.enforce sorting insert
[ https://issues.apache.org/jira/browse/HIVE-4240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-4240: - Attachment: hive.4240.4.patch > optimize hive.enforce.bucketing and hive.enforce sorting insert > --- > > Key: HIVE-4240 > URL: https://issues.apache.org/jira/browse/HIVE-4240 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Namit Jain > Attachments: hive.4240.1.patch, hive.4240.2.patch, hive.4240.3.patch, > hive.4240.4.patch > > > Consider the following scenario: > set hive.optimize.bucketmapjoin = true; > set hive.optimize.bucketmapjoin.sortedmerge = true; > set hive.input.format = > org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat; > set hive.enforce.bucketing=true; > set hive.enforce.sorting=true; > set hive.exec.reducers.max = 1; > set hive.merge.mapfiles=false; > set hive.merge.mapredfiles=false; > -- Create two bucketed and sorted tables > CREATE TABLE test_table1 (key INT, value STRING) PARTITIONED BY (ds STRING) > CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; > CREATE TABLE test_table2 (key INT, value STRING) PARTITIONED BY (ds STRING) > CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; > FROM src > INSERT OVERWRITE TABLE test_table1 PARTITION (ds = '1') SELECT *; > -- Insert data into the bucketed table by selecting from another bucketed > table > -- This should be a map-only operation > INSERT OVERWRITE TABLE test_table2 PARTITION (ds = '1') > SELECT a.key, a.value FROM test_table1 a WHERE a.ds = '1'; > We should not need a reducer to perform the above operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4240) optimize hive.enforce.bucketing and hive.enforce sorting insert
[ https://issues.apache.org/jira/browse/HIVE-4240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-4240: - Status: Patch Available (was: Open) > optimize hive.enforce.bucketing and hive.enforce sorting insert > --- > > Key: HIVE-4240 > URL: https://issues.apache.org/jira/browse/HIVE-4240 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Namit Jain > Attachments: hive.4240.1.patch, hive.4240.2.patch, hive.4240.3.patch > > > Consider the following scenario: > set hive.optimize.bucketmapjoin = true; > set hive.optimize.bucketmapjoin.sortedmerge = true; > set hive.input.format = > org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat; > set hive.enforce.bucketing=true; > set hive.enforce.sorting=true; > set hive.exec.reducers.max = 1; > set hive.merge.mapfiles=false; > set hive.merge.mapredfiles=false; > -- Create two bucketed and sorted tables > CREATE TABLE test_table1 (key INT, value STRING) PARTITIONED BY (ds STRING) > CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; > CREATE TABLE test_table2 (key INT, value STRING) PARTITIONED BY (ds STRING) > CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; > FROM src > INSERT OVERWRITE TABLE test_table1 PARTITION (ds = '1') SELECT *; > -- Insert data into the bucketed table by selecting from another bucketed > table > -- This should be a map-only operation > INSERT OVERWRITE TABLE test_table2 PARTITION (ds = '1') > SELECT a.key, a.value FROM test_table1 a WHERE a.ds = '1'; > We should not need a reducer to perform the above operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4240) optimize hive.enforce.bucketing and hive.enforce sorting insert
[ https://issues.apache.org/jira/browse/HIVE-4240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-4240: - Attachment: hive.4240.3.patch > optimize hive.enforce.bucketing and hive.enforce sorting insert > --- > > Key: HIVE-4240 > URL: https://issues.apache.org/jira/browse/HIVE-4240 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Namit Jain > Attachments: hive.4240.1.patch, hive.4240.2.patch, hive.4240.3.patch > > > Consider the following scenario: > set hive.optimize.bucketmapjoin = true; > set hive.optimize.bucketmapjoin.sortedmerge = true; > set hive.input.format = > org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat; > set hive.enforce.bucketing=true; > set hive.enforce.sorting=true; > set hive.exec.reducers.max = 1; > set hive.merge.mapfiles=false; > set hive.merge.mapredfiles=false; > -- Create two bucketed and sorted tables > CREATE TABLE test_table1 (key INT, value STRING) PARTITIONED BY (ds STRING) > CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; > CREATE TABLE test_table2 (key INT, value STRING) PARTITIONED BY (ds STRING) > CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; > FROM src > INSERT OVERWRITE TABLE test_table1 PARTITION (ds = '1') SELECT *; > -- Insert data into the bucketed table by selecting from another bucketed > table > -- This should be a map-only operation > INSERT OVERWRITE TABLE test_table2 PARTITION (ds = '1') > SELECT a.key, a.value FROM test_table1 a WHERE a.ds = '1'; > We should not need a reducer to perform the above operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4240) optimize hive.enforce.bucketing and hive.enforce sorting insert
[ https://issues.apache.org/jira/browse/HIVE-4240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-4240: - Attachment: hive.4240.2.patch > optimize hive.enforce.bucketing and hive.enforce sorting insert > --- > > Key: HIVE-4240 > URL: https://issues.apache.org/jira/browse/HIVE-4240 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Namit Jain > Attachments: hive.4240.1.patch, hive.4240.2.patch > > > Consider the following scenario: > set hive.optimize.bucketmapjoin = true; > set hive.optimize.bucketmapjoin.sortedmerge = true; > set hive.input.format = > org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat; > set hive.enforce.bucketing=true; > set hive.enforce.sorting=true; > set hive.exec.reducers.max = 1; > set hive.merge.mapfiles=false; > set hive.merge.mapredfiles=false; > -- Create two bucketed and sorted tables > CREATE TABLE test_table1 (key INT, value STRING) PARTITIONED BY (ds STRING) > CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; > CREATE TABLE test_table2 (key INT, value STRING) PARTITIONED BY (ds STRING) > CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; > FROM src > INSERT OVERWRITE TABLE test_table1 PARTITION (ds = '1') SELECT *; > -- Insert data into the bucketed table by selecting from another bucketed > table > -- This should be a map-only operation > INSERT OVERWRITE TABLE test_table2 PARTITION (ds = '1') > SELECT a.key, a.value FROM test_table1 a WHERE a.ds = '1'; > We should not need a reducer to perform the above operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4240) optimize hive.enforce.bucketing and hive.enforce sorting insert
[ https://issues.apache.org/jira/browse/HIVE-4240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-4240: - Attachment: hive.4240.1.patch > optimize hive.enforce.bucketing and hive.enforce sorting insert > --- > > Key: HIVE-4240 > URL: https://issues.apache.org/jira/browse/HIVE-4240 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Namit Jain > Attachments: hive.4240.1.patch > > > Consider the following scenario: > set hive.optimize.bucketmapjoin = true; > set hive.optimize.bucketmapjoin.sortedmerge = true; > set hive.input.format = > org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat; > set hive.enforce.bucketing=true; > set hive.enforce.sorting=true; > set hive.exec.reducers.max = 1; > set hive.merge.mapfiles=false; > set hive.merge.mapredfiles=false; > -- Create two bucketed and sorted tables > CREATE TABLE test_table1 (key INT, value STRING) PARTITIONED BY (ds STRING) > CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; > CREATE TABLE test_table2 (key INT, value STRING) PARTITIONED BY (ds STRING) > CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; > FROM src > INSERT OVERWRITE TABLE test_table1 PARTITION (ds = '1') SELECT *; > -- Insert data into the bucketed table by selecting from another bucketed > table > -- This should be a map-only operation > INSERT OVERWRITE TABLE test_table2 PARTITION (ds = '1') > SELECT a.key, a.value FROM test_table1 a WHERE a.ds = '1'; > We should not need a reducer to perform the above operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4240) optimize hive.enforce.bucketing and hive.enforce sorting insert
[ https://issues.apache.org/jira/browse/HIVE-4240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-4240: - Issue Type: Improvement (was: Bug) > optimize hive.enforce.bucketing and hive.enforce sorting insert > --- > > Key: HIVE-4240 > URL: https://issues.apache.org/jira/browse/HIVE-4240 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Namit Jain >Assignee: Namit Jain > > Consider the following scenario: > set hive.optimize.bucketmapjoin = true; > set hive.optimize.bucketmapjoin.sortedmerge = true; > set hive.input.format = > org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat; > set hive.enforce.bucketing=true; > set hive.enforce.sorting=true; > set hive.exec.reducers.max = 1; > set hive.merge.mapfiles=false; > set hive.merge.mapredfiles=false; > -- Create two bucketed and sorted tables > CREATE TABLE test_table1 (key INT, value STRING) PARTITIONED BY (ds STRING) > CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; > CREATE TABLE test_table2 (key INT, value STRING) PARTITIONED BY (ds STRING) > CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS; > FROM src > INSERT OVERWRITE TABLE test_table1 PARTITION (ds = '1') SELECT *; > -- Insert data into the bucketed table by selecting from another bucketed > table > -- This should be a map-only operation > INSERT OVERWRITE TABLE test_table2 PARTITION (ds = '1') > SELECT a.key, a.value FROM test_table1 a WHERE a.ds = '1'; > We should not need a reducer to perform the above operation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
