[jira] [Commented] (HIVE-15717) Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to exception handling
[
https://issues.apache.org/jira/browse/HIVE-15717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15843744#comment-15843744
]
Hive QA commented on HIVE-15717:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12849651/HIVE-15717.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10985 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=140)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
(batchId=93)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23]
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3227/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3227/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3227/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12849651 - PreCommit-HIVE-Build
> Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to
> exception handling
> ---
>
> Key: HIVE-15717
> URL: https://issues.apache.org/jira/browse/HIVE-15717
> Project: Hive
> Issue Type: Improvement
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-15717.1.patch, HIVE-15717.2.patch, Screen Shot
> 2017-01-24 at 3.11.09 PM.png, Screen Shot 2017-01-24 at 3.15.14 PM.png
>
>
> The exception handling from the 3 methods calls (rowDeleted, rowInserted and
> rowUpdated). The implementation of these methods in
> org.apache.hive.jdbc.HiveBaseResultSet class is just throwing
> SQLException("Method not supported”), i.e. no real implementations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15717) Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to exception handling
[
https://issues.apache.org/jira/browse/HIVE-15717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15842313#comment-15842313
]
Tao Li commented on HIVE-15717:
---
[~thejas] Just submitted another iteration.
> Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to
> exception handling
> ---
>
> Key: HIVE-15717
> URL: https://issues.apache.org/jira/browse/HIVE-15717
> Project: Hive
> Issue Type: Improvement
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-15717.1.patch, HIVE-15717.2.patch, Screen Shot
> 2017-01-24 at 3.11.09 PM.png, Screen Shot 2017-01-24 at 3.15.14 PM.png
>
>
> The exception handling from the 3 methods calls (rowDeleted, rowInserted and
> rowUpdated). The implementation of these methods in
> org.apache.hive.jdbc.HiveBaseResultSet class is just throwing
> SQLException("Method not supported”), i.e. no real implementations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15717) Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to exception handling
[
https://issues.apache.org/jira/browse/HIVE-15717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840057#comment-15840057
]
Thejas M Nair commented on HIVE-15717:
--
[~taoli-hwx] can you also add a simple unit test ?
> Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to
> exception handling
> ---
>
> Key: HIVE-15717
> URL: https://issues.apache.org/jira/browse/HIVE-15717
> Project: Hive
> Issue Type: Improvement
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-15717.1.patch, Screen Shot 2017-01-24 at 3.11.09
> PM.png, Screen Shot 2017-01-24 at 3.15.14 PM.png
>
>
> The exception handling from the 3 methods calls (rowDeleted, rowInserted and
> rowUpdated). The implementation of these methods in
> org.apache.hive.jdbc.HiveBaseResultSet class is just throwing
> SQLException("Method not supported”), i.e. no real implementations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15717) Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to exception handling
[
https://issues.apache.org/jira/browse/HIVE-15717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839561#comment-15839561
]
Hive QA commented on HIVE-15717:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12849468/HIVE-15717.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 11000 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_varchar_simple]
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
(batchId=93)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3193/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3193/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3193/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12849468 - PreCommit-HIVE-Build
> Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to
> exception handling
> ---
>
> Key: HIVE-15717
> URL: https://issues.apache.org/jira/browse/HIVE-15717
> Project: Hive
> Issue Type: Improvement
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-15717.1.patch, Screen Shot 2017-01-24 at 3.11.09
> PM.png, Screen Shot 2017-01-24 at 3.15.14 PM.png
>
>
> The exception handling from the 3 methods calls (rowDeleted, rowInserted and
> rowUpdated). The implementation of these methods in
> org.apache.hive.jdbc.HiveBaseResultSet class is just throwing
> SQLException("Method not supported”), i.e. no real implementations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15717) Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to exception handling
[
https://issues.apache.org/jira/browse/HIVE-15717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839340#comment-15839340
]
Tao Li commented on HIVE-15717:
---
The local test shows 50% improvement in terms of beeline output time. The
latency was reduced from 12 sec to 6 sec with the optimization.
> Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to
> exception handling
> ---
>
> Key: HIVE-15717
> URL: https://issues.apache.org/jira/browse/HIVE-15717
> Project: Hive
> Issue Type: Improvement
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: HIVE-15717.1.patch, Screen Shot 2017-01-24 at 3.11.09
> PM.png, Screen Shot 2017-01-24 at 3.15.14 PM.png
>
>
> The exception handling from the 3 methods calls (rowDeleted, rowInserted and
> rowUpdated). The implementation of these methods in
> org.apache.hive.jdbc.HiveBaseResultSet class is just throwing
> SQLException("Method not supported”), i.e. no real implementations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15717) Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to exception handling
[
https://issues.apache.org/jira/browse/HIVE-15717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15838905#comment-15838905
]
Tao Li commented on HIVE-15717:
---
Cool. I should be able to verify the improvement and submit the patch soon.
> Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to
> exception handling
> ---
>
> Key: HIVE-15717
> URL: https://issues.apache.org/jira/browse/HIVE-15717
> Project: Hive
> Issue Type: Improvement
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: Screen Shot 2017-01-24 at 3.11.09 PM.png, Screen Shot
> 2017-01-24 at 3.15.14 PM.png
>
>
> The exception handling from the 3 methods calls (rowDeleted, rowInserted and
> rowUpdated). The implementation of these methods in
> org.apache.hive.jdbc.HiveBaseResultSet class is just throwing
> SQLException("Method not supported”), i.e. no real implementations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15717) Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to exception handling
[
https://issues.apache.org/jira/browse/HIVE-15717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15838798#comment-15838798
]
Thejas M Nair commented on HIVE-15717:
--
[~taoli-hwx] Returning false looks appropriate in this case.
We don't support delete/insert/update in HiveBaseResultSet.
A possible future optimization for beeline would be to recognize
'SQLFeatureNotSupportedException' exception and not make further calls to it.
Created a related jira around use of 'SQLFeatureNotSupportedException' -
HIVE-15730
> Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to
> exception handling
> ---
>
> Key: HIVE-15717
> URL: https://issues.apache.org/jira/browse/HIVE-15717
> Project: Hive
> Issue Type: Improvement
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: Screen Shot 2017-01-24 at 3.11.09 PM.png, Screen Shot
> 2017-01-24 at 3.15.14 PM.png
>
>
> The exception handling from the 3 methods calls (rowDeleted, rowInserted and
> rowUpdated). The implementation of these methods in
> org.apache.hive.jdbc.HiveBaseResultSet class is just throwing
> SQLException("Method not supported”), i.e. no real implementations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15717) Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to exception handling
[
https://issues.apache.org/jira/browse/HIVE-15717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15838788#comment-15838788
]
Tao Li commented on HIVE-15717:
---
[~thejas] Got it. Thanks for the correction.
> Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to
> exception handling
> ---
>
> Key: HIVE-15717
> URL: https://issues.apache.org/jira/browse/HIVE-15717
> Project: Hive
> Issue Type: Improvement
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: Screen Shot 2017-01-24 at 3.11.09 PM.png, Screen Shot
> 2017-01-24 at 3.15.14 PM.png
>
>
> The exception handling from the 3 methods calls (rowDeleted, rowInserted and
> rowUpdated). The implementation of these methods in
> org.apache.hive.jdbc.HiveBaseResultSet class is just throwing
> SQLException("Method not supported”), i.e. no real implementations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15717) Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to exception handling
[
https://issues.apache.org/jira/browse/HIVE-15717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15838774#comment-15838774
]
Thejas M Nair commented on HIVE-15717:
--
Changed (rowDeleted, rowDeleted and rowUpdated) to (rowDeleted, rowInserted
and rowUpdated)
> Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to
> exception handling
> ---
>
> Key: HIVE-15717
> URL: https://issues.apache.org/jira/browse/HIVE-15717
> Project: Hive
> Issue Type: Improvement
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: Screen Shot 2017-01-24 at 3.11.09 PM.png, Screen Shot
> 2017-01-24 at 3.15.14 PM.png
>
>
> The exception handling from the 3 methods calls (rowDeleted, rowInserted and
> rowUpdated). The implementation of these methods in
> org.apache.hive.jdbc.HiveBaseResultSet class is just throwing
> SQLException("Method not supported”), i.e. no real implementations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15717) Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to exception handling
[
https://issues.apache.org/jira/browse/HIVE-15717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15838763#comment-15838763
]
Tao Li commented on HIVE-15717:
---
[~thejas] Looks like you made some changes to the description, however I don't
see any before/after difference.
> Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to
> exception handling
> ---
>
> Key: HIVE-15717
> URL: https://issues.apache.org/jira/browse/HIVE-15717
> Project: Hive
> Issue Type: Improvement
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: Screen Shot 2017-01-24 at 3.11.09 PM.png, Screen Shot
> 2017-01-24 at 3.15.14 PM.png
>
>
> The exception handling from the 3 methods calls (rowDeleted, rowInserted and
> rowUpdated). The implementation of these methods in
> org.apache.hive.jdbc.HiveBaseResultSet class is just throwing
> SQLException("Method not supported”), i.e. no real implementations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15717) Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to exception handling
[
https://issues.apache.org/jira/browse/HIVE-15717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837351#comment-15837351
]
Tao Li commented on HIVE-15717:
---
We want to keep Beeline working with other dbs such as PostgreSQL, so we don't
want to change the logic from org.apache.hive.beeline.Rows.Row class. One
possible solution is modifying the three methods to just return false in
org.apache.hive.jdbc.HiveBaseResultSet class. With the current logic, the 3
methods called from "org.apache.hive.beeline.Rows.Row" just throw exceptions
without updating the boolean variables, so the proposed change should not
change the logic.
> Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to
> exception handling
> ---
>
> Key: HIVE-15717
> URL: https://issues.apache.org/jira/browse/HIVE-15717
> Project: Hive
> Issue Type: Improvement
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: Screen Shot 2017-01-24 at 3.11.09 PM.png, Screen Shot
> 2017-01-24 at 3.15.14 PM.png
>
>
> The exception handling from the 3 methods calls (rowDeleted, rowDeleted and
> rowUpdated). The implementation of these methods in
> org.apache.hive.jdbc.HiveBaseResultSet class is just throwing
> SQLException("Method not supported”), i.e. no real implementations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15717) Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to exception handling
[
https://issues.apache.org/jira/browse/HIVE-15717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836884#comment-15836884
]
Tao Li commented on HIVE-15717:
---
I have attached the screen shot of the CPU profile. Instantiating the exception
instances are CPU intensive which causes extra latency when beeline is used to
display results (up to tens of seconds when displaying 1 million rows).
According the CPU profile, this exception related cost is roughly half of total
CPU cost from beeline, so reducing this cost should significantly improve the
user experience.
[~thejas] Please let me know your thoughts.
> Class "org.apache.hive.beeline.Rows.Row" constructor is CPU consuming due to
> exception handling
> ---
>
> Key: HIVE-15717
> URL: https://issues.apache.org/jira/browse/HIVE-15717
> Project: Hive
> Issue Type: Improvement
>Reporter: Tao Li
>Assignee: Tao Li
> Attachments: Screen Shot 2017-01-24 at 3.15.14 PM.png
>
>
> The exception handling from the 3 methods calls (rowDeleted, rowDeleted and
> rowUpdated). The implementation of these methods in
> org.apache.hive.jdbc.HiveBaseResultSet class is just throwing
> SQLException("Method not supported”), i.e. no real implementations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
