[jira] [Updated] (HIVE-12963) LIMIT statement with SORT BY creates additional MR job with hardcoded only one reducer
[ https://issues.apache.org/jira/browse/HIVE-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-12963: -- Labels: TODOC2.1 (was: ) > LIMIT statement with SORT BY creates additional MR job with hardcoded only > one reducer > -- > > Key: HIVE-12963 > URL: https://issues.apache.org/jira/browse/HIVE-12963 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.0.0, 1.2.1, 0.13 >Reporter: Alina Abramova >Assignee: Alina Abramova > Labels: TODOC2.1 > Fix For: 2.1.0 > > Attachments: HIVE-12963.1.patch, HIVE-12963.2.patch, > HIVE-12963.3.patch, HIVE-12963.4.patch, HIVE-12963.6.patch > > > I execute query: > hive> select age from test1 sort by age.age limit 10; > Total jobs = 2 > Launching Job 1 out of 2 > Number of reduce tasks not specified. Estimated from input data size: 1 > Launching Job 2 out of 2 > Number of reduce tasks determined at compile time: 1 > When I have a large number of rows then the last stage of the job takes a > long time. I think we could allow to user choose number of reducers of last > job or refuse extra MR job. > The same behavior I observed with querie: > hive> create table new_test as select age from test1 group by age.age limit > 10; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12963) LIMIT statement with SORT BY creates additional MR job with hardcoded only one reducer
[ https://issues.apache.org/jira/browse/HIVE-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-12963: Resolution: Fixed Fix Version/s: 2.1.0 Status: Resolved (was: Patch Available) Committed to master, sorry for the delay. Test result changes with the setting enabled are in the explain plan only... > LIMIT statement with SORT BY creates additional MR job with hardcoded only > one reducer > -- > > Key: HIVE-12963 > URL: https://issues.apache.org/jira/browse/HIVE-12963 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.0.0, 1.2.1, 0.13 >Reporter: Alina Abramova >Assignee: Alina Abramova > Fix For: 2.1.0 > > Attachments: HIVE-12963.1.patch, HIVE-12963.2.patch, > HIVE-12963.3.patch, HIVE-12963.4.patch, HIVE-12963.6.patch > > > I execute query: > hive> select age from test1 sort by age.age limit 10; > Total jobs = 2 > Launching Job 1 out of 2 > Number of reduce tasks not specified. Estimated from input data size: 1 > Launching Job 2 out of 2 > Number of reduce tasks determined at compile time: 1 > When I have a large number of rows then the last stage of the job takes a > long time. I think we could allow to user choose number of reducers of last > job or refuse extra MR job. > The same behavior I observed with querie: > hive> create table new_test as select age from test1 group by age.age limit > 10; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12963) LIMIT statement with SORT BY creates additional MR job with hardcoded only one reducer
[ https://issues.apache.org/jira/browse/HIVE-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alina Abramova updated HIVE-12963: -- Attachment: HIVE-12963.6.patch > LIMIT statement with SORT BY creates additional MR job with hardcoded only > one reducer > -- > > Key: HIVE-12963 > URL: https://issues.apache.org/jira/browse/HIVE-12963 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.0.0, 1.2.1, 0.13 >Reporter: Alina Abramova >Assignee: Alina Abramova > Attachments: HIVE-12963.1.patch, HIVE-12963.2.patch, > HIVE-12963.3.patch, HIVE-12963.4.patch, HIVE-12963.6.patch > > > I execute query: > hive> select age from test1 sort by age.age limit 10; > Total jobs = 2 > Launching Job 1 out of 2 > Number of reduce tasks not specified. Estimated from input data size: 1 > Launching Job 2 out of 2 > Number of reduce tasks determined at compile time: 1 > When I have a large number of rows then the last stage of the job takes a > long time. I think we could allow to user choose number of reducers of last > job or refuse extra MR job. > The same behavior I observed with querie: > hive> create table new_test as select age from test1 group by age.age limit > 10; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12963) LIMIT statement with SORT BY creates additional MR job with hardcoded only one reducer
[ https://issues.apache.org/jira/browse/HIVE-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alina Abramova updated HIVE-12963: -- Attachment: HIVE-12963.4.patch > LIMIT statement with SORT BY creates additional MR job with hardcoded only > one reducer > -- > > Key: HIVE-12963 > URL: https://issues.apache.org/jira/browse/HIVE-12963 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.0.0, 1.2.1, 0.13 >Reporter: Alina Abramova >Assignee: Alina Abramova > Attachments: HIVE-12963.1.patch, HIVE-12963.2.patch, > HIVE-12963.3.patch, HIVE-12963.4.patch > > > I execute query: > hive> select age from test1 sort by age.age limit 10; > Total jobs = 2 > Launching Job 1 out of 2 > Number of reduce tasks not specified. Estimated from input data size: 1 > Launching Job 2 out of 2 > Number of reduce tasks determined at compile time: 1 > When I have a large number of rows then the last stage of the job takes a > long time. I think we could allow to user choose number of reducers of last > job or refuse extra MR job. > The same behavior I observed with querie: > hive> create table new_test as select age from test1 group by age.age limit > 10; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12963) LIMIT statement with SORT BY creates additional MR job with hardcoded only one reducer
[ https://issues.apache.org/jira/browse/HIVE-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alina Abramova updated HIVE-12963: -- Attachment: HIVE-12963.3.patch > LIMIT statement with SORT BY creates additional MR job with hardcoded only > one reducer > -- > > Key: HIVE-12963 > URL: https://issues.apache.org/jira/browse/HIVE-12963 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.0.0, 1.2.1, 0.13 >Reporter: Alina Abramova >Assignee: Alina Abramova > Attachments: HIVE-12963.1.patch, HIVE-12963.2.patch, > HIVE-12963.3.patch > > > I execute query: > hive> select age from test1 sort by age.age limit 10; > Total jobs = 2 > Launching Job 1 out of 2 > Number of reduce tasks not specified. Estimated from input data size: 1 > Launching Job 2 out of 2 > Number of reduce tasks determined at compile time: 1 > When I have a large number of rows then the last stage of the job takes a > long time. I think we could allow to user choose number of reducers of last > job or refuse extra MR job. > The same behavior I observed with querie: > hive> create table new_test as select age from test1 group by age.age limit > 10; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12963) LIMIT statement with SORT BY creates additional MR job with hardcoded only one reducer
[ https://issues.apache.org/jira/browse/HIVE-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alina Abramova updated HIVE-12963: -- Attachment: (was: HIVE-12963.2.patch) > LIMIT statement with SORT BY creates additional MR job with hardcoded only > one reducer > -- > > Key: HIVE-12963 > URL: https://issues.apache.org/jira/browse/HIVE-12963 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.0.0, 1.2.1, 0.13 >Reporter: Alina Abramova >Assignee: Alina Abramova > Attachments: HIVE-12963.1.patch > > > I execute query: > hive> select age from test1 sort by age.age limit 10; > Total jobs = 2 > Launching Job 1 out of 2 > Number of reduce tasks not specified. Estimated from input data size: 1 > Launching Job 2 out of 2 > Number of reduce tasks determined at compile time: 1 > When I have a large number of rows then the last stage of the job takes a > long time. I think we could allow to user choose number of reducers of last > job or refuse extra MR job. > The same behavior I observed with queries: > hive> create table new_test as select age from test1 group by age.age limit > 10; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12963) LIMIT statement with SORT BY creates additional MR job with hardcoded only one reducer
[ https://issues.apache.org/jira/browse/HIVE-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alina Abramova updated HIVE-12963: -- Attachment: HIVE-12963.2.patch > LIMIT statement with SORT BY creates additional MR job with hardcoded only > one reducer > -- > > Key: HIVE-12963 > URL: https://issues.apache.org/jira/browse/HIVE-12963 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.0.0, 1.2.1, 0.13 >Reporter: Alina Abramova >Assignee: Alina Abramova > Attachments: HIVE-12963.1.patch, HIVE-12963.2.patch > > > I execute query: > hive> select age from test1 sort by age.age limit 10; > Total jobs = 2 > Launching Job 1 out of 2 > Number of reduce tasks not specified. Estimated from input data size: 1 > Launching Job 2 out of 2 > Number of reduce tasks determined at compile time: 1 > When I have a large number of rows then the last stage of the job takes a > long time. I think we could allow to user choose number of reducers of last > job or refuse extra MR job. > The same behavior I observed with queries: > hive> create table new_test as select age from test1 group by age.age limit > 10; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12963) LIMIT statement with SORT BY creates additional MR job with hardcoded only one reducer
[ https://issues.apache.org/jira/browse/HIVE-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alina Abramova updated HIVE-12963: -- Description: I execute query: hive> select age from test1 sort by age.age limit 10; Total jobs = 2 Launching Job 1 out of 2 Number of reduce tasks not specified. Estimated from input data size: 1 Launching Job 2 out of 2 Number of reduce tasks determined at compile time: 1 When I have a large number of rows then the last stage of the job takes a long time. I think we could allow to user choose number of reducers of last job or refuse extra MR job. The same behavior I observed with querie: hive> create table new_test as select age from test1 group by age.age limit 10; was: I execute query: hive> select age from test1 sort by age.age limit 10; Total jobs = 2 Launching Job 1 out of 2 Number of reduce tasks not specified. Estimated from input data size: 1 Launching Job 2 out of 2 Number of reduce tasks determined at compile time: 1 When I have a large number of rows then the last stage of the job takes a long time. I think we could allow to user choose number of reducers of last job or refuse extra MR job. The same behavior I observed with queries: hive> create table new_test as select age from test1 group by age.age limit 10; > LIMIT statement with SORT BY creates additional MR job with hardcoded only > one reducer > -- > > Key: HIVE-12963 > URL: https://issues.apache.org/jira/browse/HIVE-12963 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.0.0, 1.2.1, 0.13 >Reporter: Alina Abramova >Assignee: Alina Abramova > Attachments: HIVE-12963.1.patch, HIVE-12963.2.patch > > > I execute query: > hive> select age from test1 sort by age.age limit 10; > Total jobs = 2 > Launching Job 1 out of 2 > Number of reduce tasks not specified. Estimated from input data size: 1 > Launching Job 2 out of 2 > Number of reduce tasks determined at compile time: 1 > When I have a large number of rows then the last stage of the job takes a > long time. I think we could allow to user choose number of reducers of last > job or refuse extra MR job. > The same behavior I observed with querie: > hive> create table new_test as select age from test1 group by age.age limit > 10; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12963) LIMIT statement with SORT BY creates additional MR job with hardcoded only one reducer
[ https://issues.apache.org/jira/browse/HIVE-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alina Abramova updated HIVE-12963: -- Attachment: (was: HIVE-12244.2.patch) > LIMIT statement with SORT BY creates additional MR job with hardcoded only > one reducer > -- > > Key: HIVE-12963 > URL: https://issues.apache.org/jira/browse/HIVE-12963 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.0.0, 1.2.1, 0.13 >Reporter: Alina Abramova >Assignee: Alina Abramova > Attachments: HIVE-12963.1.patch > > > I execute query: > hive> select age from test1 sort by age.age limit 10; > Total jobs = 2 > Launching Job 1 out of 2 > Number of reduce tasks not specified. Estimated from input data size: 1 > Launching Job 2 out of 2 > Number of reduce tasks determined at compile time: 1 > When I have a large number of rows then the last stage of the job takes a > long time. I think we could allow to user choose number of reducers of last > job or refuse extra MR job. > The same behavior I observed with queries: > hive> create table new_test as select age from test1 group by age.age limit > 10; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12963) LIMIT statement with SORT BY creates additional MR job with hardcoded only one reducer
[ https://issues.apache.org/jira/browse/HIVE-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alina Abramova updated HIVE-12963: -- Attachment: HIVE-12244.2.patch > LIMIT statement with SORT BY creates additional MR job with hardcoded only > one reducer > -- > > Key: HIVE-12963 > URL: https://issues.apache.org/jira/browse/HIVE-12963 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.0.0, 1.2.1, 0.13 >Reporter: Alina Abramova >Assignee: Alina Abramova > Attachments: HIVE-12963.1.patch > > > I execute query: > hive> select age from test1 sort by age.age limit 10; > Total jobs = 2 > Launching Job 1 out of 2 > Number of reduce tasks not specified. Estimated from input data size: 1 > Launching Job 2 out of 2 > Number of reduce tasks determined at compile time: 1 > When I have a large number of rows then the last stage of the job takes a > long time. I think we could allow to user choose number of reducers of last > job or refuse extra MR job. > The same behavior I observed with queries: > hive> create table new_test as select age from test1 group by age.age limit > 10; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12963) LIMIT statement with SORT BY creates additional MR job with hardcoded only one reducer
[ https://issues.apache.org/jira/browse/HIVE-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alina Abramova updated HIVE-12963: -- Attachment: HIVE-12963.2.patch > LIMIT statement with SORT BY creates additional MR job with hardcoded only > one reducer > -- > > Key: HIVE-12963 > URL: https://issues.apache.org/jira/browse/HIVE-12963 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.0.0, 1.2.1, 0.13 >Reporter: Alina Abramova >Assignee: Alina Abramova > Attachments: HIVE-12963.1.patch, HIVE-12963.2.patch > > > I execute query: > hive> select age from test1 sort by age.age limit 10; > Total jobs = 2 > Launching Job 1 out of 2 > Number of reduce tasks not specified. Estimated from input data size: 1 > Launching Job 2 out of 2 > Number of reduce tasks determined at compile time: 1 > When I have a large number of rows then the last stage of the job takes a > long time. I think we could allow to user choose number of reducers of last > job or refuse extra MR job. > The same behavior I observed with queries: > hive> create table new_test as select age from test1 group by age.age limit > 10; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12963) LIMIT statement with SORT BY creates additional MR job with hardcoded only one reducer
[ https://issues.apache.org/jira/browse/HIVE-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alina Abramova updated HIVE-12963: -- Attachment: HIVE-12963.1.patch > LIMIT statement with SORT BY creates additional MR job with hardcoded only > one reducer > -- > > Key: HIVE-12963 > URL: https://issues.apache.org/jira/browse/HIVE-12963 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.0.0, 1.2.1, 0.13 >Reporter: Alina Abramova >Assignee: Alina Abramova > Attachments: HIVE-12963.1.patch > > > I execute query: > hive> select age from test1 sort by age.age limit 10; > Total jobs = 2 > Launching Job 1 out of 2 > Number of reduce tasks not specified. Estimated from input data size: 1 > Launching Job 2 out of 2 > Number of reduce tasks determined at compile time: 1 > When I have a large number of rows then the last stage of the job takes a > long time. I think we could allow to user choose number of reducers of last > job or refuse extra MR job. > The same behavior I observed with queries: > hive> create table new_test as select age from test1 group by age.age limit > 10; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12963) LIMIT statement with SORT BY creates additional MR job with hardcoded only one reducer
[ https://issues.apache.org/jira/browse/HIVE-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alina Abramova updated HIVE-12963: -- Attachment: (was: HIVE-12963.patch.1) > LIMIT statement with SORT BY creates additional MR job with hardcoded only > one reducer > -- > > Key: HIVE-12963 > URL: https://issues.apache.org/jira/browse/HIVE-12963 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.0.0, 1.2.1, 0.13 >Reporter: Alina Abramova >Assignee: Alina Abramova > Attachments: HIVE-12963.1.patch > > > I execute query: > hive> select age from test1 sort by age.age limit 10; > Total jobs = 2 > Launching Job 1 out of 2 > Number of reduce tasks not specified. Estimated from input data size: 1 > Launching Job 2 out of 2 > Number of reduce tasks determined at compile time: 1 > When I have a large number of rows then the last stage of the job takes a > long time. I think we could allow to user choose number of reducers of last > job or refuse extra MR job. > The same behavior I observed with queries: > hive> create table new_test as select age from test1 group by age.age limit > 10; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-12963) LIMIT statement with SORT BY creates additional MR job with hardcoded only one reducer
[ https://issues.apache.org/jira/browse/HIVE-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alina Abramova updated HIVE-12963: -- Attachment: HIVE-12963.patch.1 > LIMIT statement with SORT BY creates additional MR job with hardcoded only > one reducer > -- > > Key: HIVE-12963 > URL: https://issues.apache.org/jira/browse/HIVE-12963 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.0.0, 1.2.1, 0.13 >Reporter: Alina Abramova >Assignee: Alina Abramova > Attachments: HIVE-12963.patch.1 > > > I execute query: > hive> select age from test1 sort by age.age limit 10; > Total jobs = 2 > Launching Job 1 out of 2 > Number of reduce tasks not specified. Estimated from input data size: 1 > Launching Job 2 out of 2 > Number of reduce tasks determined at compile time: 1 > When I have a large number of rows then the last stage of the job takes a > long time. I think we could allow to user choose number of reducers of last > job or refuse extra MR job. > The same behavior I observed with queries: > hive> create table new_test as select age from test1 group by age.age limit > 10; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
