[jira] [Commented] (HIVE-8689) handle overflows in statistics better
[ https://issues.apache.org/jira/browse/HIVE-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195078#comment-14195078 ] Sergey Shelukhin commented on HIVE-8689: Thanks! handle overflows in statistics better - Key: HIVE-8689 URL: https://issues.apache.org/jira/browse/HIVE-8689 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.14.0 Attachments: HIVE-8689.01.patch, HIVE-8689.02.patch, HIVE-8689.patch Improve overflow checks in StatsAnnotation optimizer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8689) handle overflows in statistics better
[
https://issues.apache.org/jira/browse/HIVE-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14192975#comment-14192975
]
Hive QA commented on HIVE-8689:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12678641/HIVE-8689.01.patch
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 6608 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_infer_bucket_sort_reducers_power_two
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority
org.apache.hive.minikdc.TestJdbcWithMiniKdc.testNegativeTokenAuth
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1586/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1586/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1586/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12678641 - PreCommit-HIVE-TRUNK-Build
handle overflows in statistics better
-
Key: HIVE-8689
URL: https://issues.apache.org/jira/browse/HIVE-8689
Project: Hive
Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Fix For: 0.14.0
Attachments: HIVE-8689.01.patch, HIVE-8689.patch
Improve overflow checks in StatsAnnotation optimizer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8689) handle overflows in statistics better
[ https://issues.apache.org/jira/browse/HIVE-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14193394#comment-14193394 ] Prasanth J commented on HIVE-8689: -- [~sershe] HIVE-8671 committed now. Can you rebase this patch now? Also can you fix Mostafa's change to reducer estimation. It will estimate one reducer less than the previous code. For example: if totalInputFileSize is 140 and bytesPerReducer is 100 then current change will just say 1 reducer. We should either have Math.ceil or Math.max(totalInputFileSize, totalInputFileSize + bytesPerReducer - 1)/bytesPerReducer.. handle overflows in statistics better - Key: HIVE-8689 URL: https://issues.apache.org/jira/browse/HIVE-8689 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.14.0 Attachments: HIVE-8689.01.patch, HIVE-8689.patch Improve overflow checks in StatsAnnotation optimizer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8689) handle overflows in statistics better
[ https://issues.apache.org/jira/browse/HIVE-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14193503#comment-14193503 ] Sergey Shelukhin commented on HIVE-8689: [~hagleitn] 14? I will address the latest comment handle overflows in statistics better - Key: HIVE-8689 URL: https://issues.apache.org/jira/browse/HIVE-8689 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.14.0 Attachments: HIVE-8689.01.patch, HIVE-8689.patch Improve overflow checks in StatsAnnotation optimizer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8689) handle overflows in statistics better
[ https://issues.apache.org/jira/browse/HIVE-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14193521#comment-14193521 ] Prasanth J commented on HIVE-8689: -- +1 handle overflows in statistics better - Key: HIVE-8689 URL: https://issues.apache.org/jira/browse/HIVE-8689 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.14.0 Attachments: HIVE-8689.01.patch, HIVE-8689.02.patch, HIVE-8689.patch Improve overflow checks in StatsAnnotation optimizer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8689) handle overflows in statistics better
[ https://issues.apache.org/jira/browse/HIVE-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14193522#comment-14193522 ] Prasanth J commented on HIVE-8689: -- [~sershe] minor nit: Can you remove the getMaxIfOverflow() method? Since we are using safeAdd, safeMultiply methods we don't need that anymore. handle overflows in statistics better - Key: HIVE-8689 URL: https://issues.apache.org/jira/browse/HIVE-8689 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.14.0 Attachments: HIVE-8689.01.patch, HIVE-8689.02.patch, HIVE-8689.patch Improve overflow checks in StatsAnnotation optimizer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8689) handle overflows in statistics better
[ https://issues.apache.org/jira/browse/HIVE-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14193537#comment-14193537 ] Gunther Hagleitner commented on HIVE-8689: -- +1 for .14 handle overflows in statistics better - Key: HIVE-8689 URL: https://issues.apache.org/jira/browse/HIVE-8689 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.14.0 Attachments: HIVE-8689.01.patch, HIVE-8689.02.patch, HIVE-8689.patch Improve overflow checks in StatsAnnotation optimizer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8689) handle overflows in statistics better
[
https://issues.apache.org/jira/browse/HIVE-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14193622#comment-14193622
]
Hive QA commented on HIVE-8689:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12678725/HIVE-8689.02.patch
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 6668 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_covar_pop
org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchEmptyCommit
org.apache.hive.minikdc.TestJdbcWithMiniKdc.testNegativeTokenAuth
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1603/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/1603/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-1603/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12678725 - PreCommit-HIVE-TRUNK-Build
handle overflows in statistics better
-
Key: HIVE-8689
URL: https://issues.apache.org/jira/browse/HIVE-8689
Project: Hive
Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Fix For: 0.14.0
Attachments: HIVE-8689.01.patch, HIVE-8689.02.patch, HIVE-8689.patch
Improve overflow checks in StatsAnnotation optimizer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8689) handle overflows in statistics better
[ https://issues.apache.org/jira/browse/HIVE-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14192776#comment-14192776 ] Sergey Shelukhin commented on HIVE-8689: https://reviews.apache.org/r/27455/ handle overflows in statistics better - Key: HIVE-8689 URL: https://issues.apache.org/jira/browse/HIVE-8689 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.14.0 Attachments: HIVE-8689.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8689) handle overflows in statistics better
[ https://issues.apache.org/jira/browse/HIVE-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14192807#comment-14192807 ] Prasanth J commented on HIVE-8689: -- Added some comments to RB handle overflows in statistics better - Key: HIVE-8689 URL: https://issues.apache.org/jira/browse/HIVE-8689 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.14.0 Attachments: HIVE-8689.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
