[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=443714=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-443714 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 10/Jun/20 13:15 Start Date: 10/Jun/20 13:15 Worklog Time Spent: 10m Work Description: belugabehr merged pull request #1029: URL: https://github.com/apache/hive/pull/1029 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 443714) Time Spent: 7h 40m (was: 7.5h) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 7h 40m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441762=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441762 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 05/Jun/20 09:39 Start Date: 05/Jun/20 09:39 Worklog Time Spent: 10m Work Description: dengzhhu653 edited a comment on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-639371721 > @dengzhhu653 : absolutely...we are in the process of both changing the test execution infra - and it actually exposes some new problematic tests...I've restarted the job - see if the next build has a better chance... OK. Thank you for all this work. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441762) Time Spent: 7.5h (was: 7h 20m) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 7.5h > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441757=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441757 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 05/Jun/20 09:38 Start Date: 05/Jun/20 09:38 Worklog Time Spent: 10m Work Description: dengzhhu653 commented on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-639371360 > @dengzhhu653 : absolutely...we are in the process of both changing the test execution infra - and it actually exposes some new problematic tests...I've restarted the job - see if the next build has a better chance... OK. Thank you for all this infra work. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441757) Time Spent: 6h 50m (was: 6h 40m) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 6h 50m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441759=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441759 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 05/Jun/20 09:38 Start Date: 05/Jun/20 09:38 Worklog Time Spent: 10m Work Description: dengzhhu653 removed a comment on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-639371360 > @dengzhhu653 : absolutely...we are in the process of both changing the test execution infra - and it actually exposes some new problematic tests...I've restarted the job - see if the next build has a better chance... OK. Thank you for all this infra work. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441759) Time Spent: 7h 10m (was: 7h) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 7h 10m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441760=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441760 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 05/Jun/20 09:38 Start Date: 05/Jun/20 09:38 Worklog Time Spent: 10m Work Description: dengzhhu653 commented on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-639371721 > @dengzhhu653 : absolutely...we are in the process of both changing the test execution infra - and it actually exposes some new problematic tests...I've restarted the job - see if the next build has a better chance... OK. Thank you for all this infra work. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441760) Time Spent: 7h 20m (was: 7h 10m) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 7h 20m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441758=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441758 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 05/Jun/20 09:38 Start Date: 05/Jun/20 09:38 Worklog Time Spent: 10m Work Description: dengzhhu653 edited a comment on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-639371360 > @dengzhhu653 : absolutely...we are in the process of both changing the test execution infra - and it actually exposes some new problematic tests...I've restarted the job - see if the next build has a better chance... OK. Thank you for all this infra work. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441758) Time Spent: 7h (was: 6h 50m) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 7h > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441745=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441745 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 05/Jun/20 08:48 Start Date: 05/Jun/20 08:48 Worklog Time Spent: 10m Work Description: kgyrtkirk commented on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-639347617 @dengzhhu653 : absolutely...we are in the process of both changing the test execution infra - and it actually exposes some new problematic tests...I've restarted the job - see if the next build has a better chance... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441745) Time Spent: 6h 40m (was: 6.5h) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 6h 40m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441732=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441732 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 05/Jun/20 08:10 Start Date: 05/Jun/20 08:10 Worklog Time Spent: 10m Work Description: dengzhhu653 commented on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-639327974 The failed tests seem to be unrelated to the changes... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441732) Time Spent: 6.5h (was: 6h 20m) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 6.5h > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441318=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441318
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 14:53
Start Date: 04/Jun/20 14:53
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r435321340
##
File path: jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java
##
@@ -198,9 +198,9 @@ private void closeStatementIfNeeded() throws SQLException {
} catch (SQLException e) {
throw e;
} catch (TApplicationException tae) {
- String errorMsg = tae.getMessage();
+ String errorMsg = "Failed to close statement";
if (tae.getType() == TApplicationException.BAD_SEQUENCE_ID) {
-errorMsg = "Mismatch thrift sequence id. A previous call to the Thrift
library failed"
+errorMsg += ". Mismatch thrift sequence id. A previous call to the
Thrift library failed"
Review comment:
Done. Thank you @belugabehr
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441318)
Time Spent: 6h 20m (was: 6h 10m)
> Beeline may throw the misleading exception
> --
>
> Key: HIVE-23526
> URL: https://issues.apache.org/jira/browse/HIVE-23526
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
> Environment: Hive 1.2.2
>Reporter: Zhihua Deng
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch,
> HIVE-23526.patch, outofsequence.log
>
> Time Spent: 6h 20m
> Remaining Estimate: 0h
>
> Sometimes we can see 'out of sequence response' message in beeline, for
> example:
> Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of
> sequence response (state=08S01,code=0)
> java.sql.SQLException: org.apache.thrift.TApplicationException:
> CloseOperation failed: out of sequence response
> at
> org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198)
> at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217)
> at org.apache.hive.beeline.Commands.execute(Commands.java:891)
> at org.apache.hive.beeline.Commands.sql(Commands.java:713)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774)
> at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470)
> and there is no other usage message to figured it out, even with --verbose,
> this makes problem puzzled as beeline does not have concurrency problem on
> underlying thrift transport.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441297=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441297
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 14:32
Start Date: 04/Jun/20 14:32
Worklog Time Spent: 10m
Work Description: belugabehr commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r435305205
##
File path: jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java
##
@@ -198,9 +198,9 @@ private void closeStatementIfNeeded() throws SQLException {
} catch (SQLException e) {
throw e;
} catch (TApplicationException tae) {
- String errorMsg = tae.getMessage();
+ String errorMsg = "Failed to close statement";
if (tae.getType() == TApplicationException.BAD_SEQUENCE_ID) {
-errorMsg = "Mismatch thrift sequence id. A previous call to the Thrift
library failed"
+errorMsg += ". Mismatch thrift sequence id. A previous call to the
Thrift library failed"
Review comment:
```
errorMsg = "Failed to close statement. Mismatch thrift sequence id. A
previous call to the Thrift library failed...";
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441297)
Time Spent: 6h 10m (was: 6h)
> Beeline may throw the misleading exception
> --
>
> Key: HIVE-23526
> URL: https://issues.apache.org/jira/browse/HIVE-23526
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
> Environment: Hive 1.2.2
>Reporter: Zhihua Deng
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch,
> HIVE-23526.patch, outofsequence.log
>
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> Sometimes we can see 'out of sequence response' message in beeline, for
> example:
> Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of
> sequence response (state=08S01,code=0)
> java.sql.SQLException: org.apache.thrift.TApplicationException:
> CloseOperation failed: out of sequence response
> at
> org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198)
> at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217)
> at org.apache.hive.beeline.Commands.execute(Commands.java:891)
> at org.apache.hive.beeline.Commands.sql(Commands.java:713)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774)
> at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470)
> and there is no other usage message to figured it out, even with --verbose,
> this makes problem puzzled as beeline does not have concurrency problem on
> underlying thrift transport.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441296=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441296
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 14:31
Start Date: 04/Jun/20 14:31
Worklog Time Spent: 10m
Work Description: belugabehr commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r435304118
##
File path: jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java
##
@@ -198,9 +198,9 @@ private void closeStatementIfNeeded() throws SQLException {
} catch (SQLException e) {
throw e;
} catch (TApplicationException tae) {
- String errorMsg = tae.getMessage();
+ String errorMsg = "Failed to close statement";
if (tae.getType() == TApplicationException.BAD_SEQUENCE_ID) {
-errorMsg = "Mismatch thrift sequence id. A previous call to the Thrift
library failed"
+errorMsg += ". Mismatch thrift sequence id. A previous call to the
Thrift library failed"
Review comment:
Lol. I'm so sorry to do this to you, I've been in your seat quite a few
times myself, but can you please do one more push? Please do not do the string
concatenation. This could be ripe for errors if the code changes later. Just
use a single static string which includes the words "Failed to close statement."
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441296)
Time Spent: 6h (was: 5h 50m)
> Beeline may throw the misleading exception
> --
>
> Key: HIVE-23526
> URL: https://issues.apache.org/jira/browse/HIVE-23526
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
> Environment: Hive 1.2.2
>Reporter: Zhihua Deng
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch,
> HIVE-23526.patch, outofsequence.log
>
> Time Spent: 6h
> Remaining Estimate: 0h
>
> Sometimes we can see 'out of sequence response' message in beeline, for
> example:
> Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of
> sequence response (state=08S01,code=0)
> java.sql.SQLException: org.apache.thrift.TApplicationException:
> CloseOperation failed: out of sequence response
> at
> org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198)
> at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217)
> at org.apache.hive.beeline.Commands.execute(Commands.java:891)
> at org.apache.hive.beeline.Commands.sql(Commands.java:713)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774)
> at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470)
> and there is no other usage message to figured it out, even with --verbose,
> this makes problem puzzled as beeline does not have concurrency problem on
> underlying thrift transport.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441294=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441294
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 14:29
Start Date: 04/Jun/20 14:29
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r435303133
##
File path: jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java
##
@@ -196,6 +197,14 @@ private void closeStatementIfNeeded() throws SQLException {
}
} catch (SQLException e) {
throw e;
+} catch (TApplicationException tae) {
+ String errorMsg = tae.getMessage();
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441294)
Time Spent: 5h 50m (was: 5h 40m)
> Beeline may throw the misleading exception
> --
>
> Key: HIVE-23526
> URL: https://issues.apache.org/jira/browse/HIVE-23526
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
> Environment: Hive 1.2.2
>Reporter: Zhihua Deng
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch,
> HIVE-23526.patch, outofsequence.log
>
> Time Spent: 5h 50m
> Remaining Estimate: 0h
>
> Sometimes we can see 'out of sequence response' message in beeline, for
> example:
> Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of
> sequence response (state=08S01,code=0)
> java.sql.SQLException: org.apache.thrift.TApplicationException:
> CloseOperation failed: out of sequence response
> at
> org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198)
> at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217)
> at org.apache.hive.beeline.Commands.execute(Commands.java:891)
> at org.apache.hive.beeline.Commands.sql(Commands.java:713)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774)
> at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470)
> and there is no other usage message to figured it out, even with --verbose,
> this makes problem puzzled as beeline does not have concurrency problem on
> underlying thrift transport.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441291=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441291 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 04/Jun/20 14:23 Start Date: 04/Jun/20 14:23 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-638881978 Thanks for your work on this and for bringing it to our attention. I think the changes we ended up with will be helpful for others. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441291) Time Spent: 5h 40m (was: 5.5h) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 5h 40m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441289=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441289 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 04/Jun/20 14:21 Start Date: 04/Jun/20 14:21 Worklog Time Spent: 10m Work Description: dengzhhu653 commented on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-638880740 When I firstly see the exception, it's hard to associate the exception with the memory problem. the case can be fixed by set incremental to true, as trunk does. Agree that thrift should better handle the OOM Exception, as long as the connection is still alive, maybe there is a chance that the problem will see. Thanks again for taking time to look at this, @belugabehr @kgyrtkirk This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441289) Time Spent: 5.5h (was: 5h 20m) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 5.5h > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441283=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441283
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 13:56
Start Date: 04/Jun/20 13:56
Worklog Time Spent: 10m
Work Description: belugabehr commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r435277483
##
File path: jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java
##
@@ -196,6 +197,14 @@ private void closeStatementIfNeeded() throws SQLException {
}
} catch (SQLException e) {
throw e;
+} catch (TApplicationException tae) {
+ String errorMsg = tae.getMessage();
Review comment:
The default message should still be `Failed to close statement`. The
message of the `TApplicationException` will be preserved because the entire
Exception is passed into the `SQLException`.
With that said, you can you please append that to the `BAD_SEQUENCE_ID`
error as well?
> Failed to close statement. Mismatch thrift sequence id. A previous ...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441283)
Time Spent: 5h 20m (was: 5h 10m)
> Beeline may throw the misleading exception
> --
>
> Key: HIVE-23526
> URL: https://issues.apache.org/jira/browse/HIVE-23526
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
> Environment: Hive 1.2.2
>Reporter: Zhihua Deng
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch,
> HIVE-23526.patch, outofsequence.log
>
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> Sometimes we can see 'out of sequence response' message in beeline, for
> example:
> Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of
> sequence response (state=08S01,code=0)
> java.sql.SQLException: org.apache.thrift.TApplicationException:
> CloseOperation failed: out of sequence response
> at
> org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198)
> at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217)
> at org.apache.hive.beeline.Commands.execute(Commands.java:891)
> at org.apache.hive.beeline.Commands.sql(Commands.java:713)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774)
> at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470)
> and there is no other usage message to figured it out, even with --verbose,
> this makes problem puzzled as beeline does not have concurrency problem on
> underlying thrift transport.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441281=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441281
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 13:46
Start Date: 04/Jun/20 13:46
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r435268220
##
File path: jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java
##
@@ -197,6 +198,12 @@ private void closeStatementIfNeeded() throws SQLException {
} catch (SQLException e) {
throw e;
} catch (Exception e) {
+ if (e instanceof TApplicationException && ((TApplicationException)e)
+ .getType() == TApplicationException.BAD_SEQUENCE_ID) {
+String errorMsg = "Mismatch thrift sequence id, which usually is
caused by an OutOfMemoryError " +
+"or a SocketTimeoutException, please check the memory status and
previous calls.";
+throw new SQLException(errorMsg, "08S01", e);
Review comment:
Thanks @belugabehr, updated the error messages.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441281)
Time Spent: 5h 10m (was: 5h)
> Beeline may throw the misleading exception
> --
>
> Key: HIVE-23526
> URL: https://issues.apache.org/jira/browse/HIVE-23526
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
> Environment: Hive 1.2.2
>Reporter: Zhihua Deng
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch,
> HIVE-23526.patch, outofsequence.log
>
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> Sometimes we can see 'out of sequence response' message in beeline, for
> example:
> Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of
> sequence response (state=08S01,code=0)
> java.sql.SQLException: org.apache.thrift.TApplicationException:
> CloseOperation failed: out of sequence response
> at
> org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198)
> at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217)
> at org.apache.hive.beeline.Commands.execute(Commands.java:891)
> at org.apache.hive.beeline.Commands.sql(Commands.java:713)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774)
> at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470)
> and there is no other usage message to figured it out, even with --verbose,
> this makes problem puzzled as beeline does not have concurrency problem on
> underlying thrift transport.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441271=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441271
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 13:18
Start Date: 04/Jun/20 13:18
Worklog Time Spent: 10m
Work Description: belugabehr commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r435245485
##
File path: jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java
##
@@ -197,6 +198,12 @@ private void closeStatementIfNeeded() throws SQLException {
} catch (SQLException e) {
throw e;
} catch (Exception e) {
+ if (e instanceof TApplicationException && ((TApplicationException)e)
+ .getType() == TApplicationException.BAD_SEQUENCE_ID) {
+String errorMsg = "Mismatch thrift sequence id, which usually is
caused by an OutOfMemoryError " +
+"or a SocketTimeoutException, please check the memory status and
previous calls.";
+throw new SQLException(errorMsg, "08S01", e);
Review comment:
Please do not say 'is usually caused by'. We do not yet know what
"usual" is.
Try something like:
`
Mismatch thrift sequence id. A previous call to the Thrift library failed
and now position within the input stream is lost. Please enable verbose error
logging and check the status of previous calls.
`
##
File path: jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java
##
@@ -197,6 +198,12 @@ private void closeStatementIfNeeded() throws SQLException {
} catch (SQLException e) {
throw e;
} catch (Exception e) {
+ if (e instanceof TApplicationException && ((TApplicationException)e)
Review comment:
Please do not use `instanceof` here, just create a new `catch` block for
`TApplicationException` types.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441271)
Time Spent: 5h (was: 4h 50m)
> Beeline may throw the misleading exception
> --
>
> Key: HIVE-23526
> URL: https://issues.apache.org/jira/browse/HIVE-23526
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
> Environment: Hive 1.2.2
>Reporter: Zhihua Deng
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch,
> HIVE-23526.patch, outofsequence.log
>
> Time Spent: 5h
> Remaining Estimate: 0h
>
> Sometimes we can see 'out of sequence response' message in beeline, for
> example:
> Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of
> sequence response (state=08S01,code=0)
> java.sql.SQLException: org.apache.thrift.TApplicationException:
> CloseOperation failed: out of sequence response
> at
> org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198)
> at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217)
> at org.apache.hive.beeline.Commands.execute(Commands.java:891)
> at org.apache.hive.beeline.Commands.sql(Commands.java:713)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774)
> at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470)
> and there is no other usage message to figured it out, even with --verbose,
> this makes problem puzzled as beeline does not have concurrency problem on
> underlying thrift transport.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441270=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441270 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 04/Jun/20 13:15 Start Date: 04/Jun/20 13:15 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-638839709 OK. Sorry. I just noticed that you changed your PR quite a bit. OK. I can be on board with that, but I do have some changes I would like to see. I will put comments in. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441270) Time Spent: 4h 50m (was: 4h 40m) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441269=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441269 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 04/Jun/20 13:13 Start Date: 04/Jun/20 13:13 Worklog Time Spent: 10m Work Description: belugabehr edited a comment on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-638838456 @dengzhhu653 By default (Hive 2.3 and earlier), beeline buffers all of the results before displaying them to the user. With a query like `select * from a limit 50`, if there are that many rows in the table, it will have to buffer 500_000 rows and fit them all into 512MB of memory (plus all the other stuff beeline stores). And let's be honest, no human is going to look through that many rows manually. You're better off disabling buffering, and streaming out the results to a CSV for further processing: Something like: `beeline -u "jdbc:hive2://..." -e "select * from a limit 50" --outputformat=csv2 --incremental=true > results.csv` https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=82903124#HiveServer2Clients-Separated-ValueOutputFormats https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-BeelineCommandOptions This is not a problem with Thrift. Maybe Thrift should better handle the OOM Exception, but that will have to be addressed in that project, not Hive. As long as the OOM Exception is being propagated up, and the Statement is closed (which I believe it is) then Beeline is handling this appropriately. I just visually traced the trunk code and it looks like the OOM is being handled correctly. Details should be printed with verbose logging enabled. I think this method is a bit spaghetti and needs some TLC, but it should be working. I am not in favor of this change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441269) Time Spent: 4h 40m (was: 4.5h) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 4h 40m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441268=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441268 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 04/Jun/20 13:13 Start Date: 04/Jun/20 13:13 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-638838456 @dengzhhu653 By default (Hive 2.3 and earlier), beeline buffers all of the results before displaying them to the user. With a query like `select * from a limit 50`, if there are that many rows in the table, it will have to buffer 500_000 rows and fit them all into 512MB of memory (please all the other stuff beeline stores). And let's be honest, no human is going to look through that many rows manually. You're better off disabling buffering, and streaming out the results to a CSV for further processing: Something like: `beeline -u "jdbc:hive2://..." -e "select * from a limit 50" --outputformat=csv2 --incremental=true > results.csv` https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=82903124#HiveServer2Clients-Separated-ValueOutputFormats https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-BeelineCommandOptions This is not a problem with Thrift. Maybe Thrift should better handle the OOM Exception, but that will have to be addressed in that project, not Hive. As long as the OOM Exception is being propagated up, and the Statement is closed (which I believe it is) then Beeline is handling this appropriately. I just visually traced the trunk code and it looks like the OOM is being handled correctly. Details should be printed with verbose logging enabled. I think this method is a bit spaghetti and needs some TLC, but it should be working. I am not in favor of this change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441268) Time Spent: 4.5h (was: 4h 20m) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 4.5h > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441206=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441206 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 04/Jun/20 09:51 Start Date: 04/Jun/20 09:51 Worklog Time Spent: 10m Work Description: dengzhhu653 edited a comment on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-638744541 > just making a bit more clear what could be the underlying situation makes sense to me; so +1 > > @dengzhhu653 can you reproduce the issue? > from the thrift tickets it is unclear if this was addressed since 0.9.3 or not... Yes, everytime I run a user's sql(select * from a limit 50 with -Xmx512m), the problem always shows, maybe the problem have some relationship with his data and the total memory. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441206) Time Spent: 4h 20m (was: 4h 10m) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 4h 20m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441203=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441203 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 04/Jun/20 09:49 Start Date: 04/Jun/20 09:49 Worklog Time Spent: 10m Work Description: dengzhhu653 commented on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-638744541 > just making a bit more clear what could be the underlying situation makes sense to me; so +1 > > @dengzhhu653 can you reproduce the issue? > from the thrift tickets it is unclear if this was addressed since 0.9.3 or not... Yes, everytime I run a user's sql(select * from a limit 5 with -Xmx512m), the problem always shows, maybe the problem have some relationship with his data and the total memory. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441203) Time Spent: 4h (was: 3h 50m) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 4h > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441205=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441205 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 04/Jun/20 09:49 Start Date: 04/Jun/20 09:49 Worklog Time Spent: 10m Work Description: dengzhhu653 edited a comment on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-638744541 > just making a bit more clear what could be the underlying situation makes sense to me; so +1 > > @dengzhhu653 can you reproduce the issue? > from the thrift tickets it is unclear if this was addressed since 0.9.3 or not... Yes, everytime I run a user's sql(select * from a limit 5 with -Xmx512m), the problem always shows, maybe the problem have some relationship with his data and the total memory. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441205) Time Spent: 4h 10m (was: 4h) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 4h 10m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441198=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441198 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 04/Jun/20 09:36 Start Date: 04/Jun/20 09:36 Worklog Time Spent: 10m Work Description: kgyrtkirk commented on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-638737870 just making a bit more clear what could be the underlying situation makes sense to me; so +1 @dengzhhu653 can you reproduce the issue? from the thrift tickets it is unclear if this was addressed since 0.9.3 or not... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441198) Time Spent: 3h 50m (was: 3h 40m) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 3h 50m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441194=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441194 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 04/Jun/20 09:19 Start Date: 04/Jun/20 09:19 Worklog Time Spent: 10m Work Description: dengzhhu653 commented on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-638728373 > I think the best thing would be to modify this: > > https://github.com/apache/hive/blob/2795d3253f5ff09c23b12e5ff980cb14268d37ab/jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java#L189-L202 > > Thrift does not have sub-classes for Exception, but Exceptions generally carry a type. In this case, the type is `TApplicationException.BAD_SEQUENCE_ID` : > > https://github.com/apache/thrift/blob/af7ecd6a2b15efe5c6b742cf4a9ccb31bcc1f362/lib/java/src/org/apache/thrift/TServiceClient.java#L84-L87 > > I think the best course of action here is to catch the Exception in the `HiveStatement`, check the 'type', and if it is `BAD_SEQUENCE_ID`, wrap the Exception in a `SQLException` and provide a better error message that is more clear. > > That `Commands` method is a bit hairy, but all that needs to happen as a quick fix is to wrap the `Statement#close()` with its own try-catch block. There is no need to close the result set. Closing the Statement should close all associated result sets. > > > When a Statement object is closed, its current ResultSet object, if one exists, is also closed. > > https://docs.oracle.com/javase/7/docs/api/java/sql/Statement.html#close() Updated, wrap the exception with some messages. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441194) Time Spent: 3h 40m (was: 3.5h) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 3h 40m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441073=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441073 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 04/Jun/20 02:27 Start Date: 04/Jun/20 02:27 Worklog Time Spent: 10m Work Description: belugabehr edited a comment on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-638563394 I think the best thing would be to modify this: https://github.com/apache/hive/blob/2795d3253f5ff09c23b12e5ff980cb14268d37ab/jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java#L189-L202 Thrift does not have sub-classes for Exception, but Exceptions generally carry a type. In this case, the type is `TApplicationException.BAD_SEQUENCE_ID` : https://github.com/apache/thrift/blob/af7ecd6a2b15efe5c6b742cf4a9ccb31bcc1f362/lib/java/src/org/apache/thrift/TServiceClient.java#L84-L87 I think the best course of action here is to catch the Exception in the `HiveStatement`, check the 'type', and if it is `BAD_SEQUENCE_ID`, wrap the Exception in a `SQLException` and provide a better error message that is more clear. That `Commands` method is a bit hairy, but all that needs to happen as a quick fix is to wrap the `Statement#close()` with its own try-catch block. There is no need to close the result set. Closing the Statement should close all associated result sets. > When a Statement object is closed, its current ResultSet object, if one exists, is also closed. https://docs.oracle.com/javase/7/docs/api/java/sql/Statement.html#close() This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441073) Time Spent: 3.5h (was: 3h 20m) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 3.5h > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441071=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441071 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 04/Jun/20 02:26 Start Date: 04/Jun/20 02:26 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-638563394 I think the best thing would be to modify this: https://github.com/apache/hive/blob/2795d3253f5ff09c23b12e5ff980cb14268d37ab/jdbc/src/java/org/apache/hive/jdbc/HiveStatement.java#L189-L202 Thrift does not have sub-classes for Exception, but Exceptions generally carry a type. In this case, the type is `TApplicationException.BAD_SEQUENCE_ID` : https://github.com/apache/thrift/blob/af7ecd6a2b15efe5c6b742cf4a9ccb31bcc1f362/lib/java/src/org/apache/thrift/TServiceClient.java#L84-L87 I think the best course of action here is to catch the Exception in the `HiveStatement`, check the 'type', and if it is `BAD_SEQUENCE_ID, wrap the Exception in a `SQLException` and provide a better error message that is more clear. That `Commands` method is a bit hairy, but all that needs to happen as a quick fix is to wrap the `Statement#close()` with its own try-catch block. There is no need to close the result set. Closing the Statement should close all associated result sets. > When a Statement object is closed, its current ResultSet object, if one exists, is also closed. https://docs.oracle.com/javase/7/docs/api/java/sql/Statement.html#close() This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441071) Time Spent: 3h 20m (was: 3h 10m) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Minor > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 3h 20m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441070=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441070
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 04/Jun/20 02:16
Start Date: 04/Jun/20 02:16
Worklog Time Spent: 10m
Work Description: belugabehr commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r434953975
##
File path: beeline/src/java/org/apache/hive/beeline/Commands.java
##
@@ -1063,13 +1057,17 @@ private boolean executeInternal(String sql, boolean
call) {
showRemainingLogsIfAny(stmnt);
}
}
+if (rs != null) {
+ rs.close();
Review comment:
Closing the result set can throw a `SQLException` in which case it will
skip over the `Statement#close()` method.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441070)
Time Spent: 3h 10m (was: 3h)
> Beeline may throw the misleading exception
> --
>
> Key: HIVE-23526
> URL: https://issues.apache.org/jira/browse/HIVE-23526
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
> Environment: Hive 1.2.2
>Reporter: Zhihua Deng
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch,
> HIVE-23526.patch, outofsequence.log
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> Sometimes we can see 'out of sequence response' message in beeline, for
> example:
> Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of
> sequence response (state=08S01,code=0)
> java.sql.SQLException: org.apache.thrift.TApplicationException:
> CloseOperation failed: out of sequence response
> at
> org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198)
> at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217)
> at org.apache.hive.beeline.Commands.execute(Commands.java:891)
> at org.apache.hive.beeline.Commands.sql(Commands.java:713)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774)
> at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470)
> and there is no other usage message to figured it out, even with --verbose,
> this makes problem puzzled as beeline does not have concurrency problem on
> underlying thrift transport.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=441031=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441031 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 03/Jun/20 23:48 Start Date: 03/Jun/20 23:48 Worklog Time Spent: 10m Work Description: dengzhhu653 commented on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-638518722 > As I understand it, the code calling the JDBC Driver captures and handles the OOM error and then closes the JDBC Driver. When it closes, it throws another error " CloseOperation failed: out of sequence response". You would like for the close to be silent here. Is that correct? > > If so, there's not a ton that can be done here (other than perhaps wrap the OOM error in a Thrift Exception). > > In this particular case, Thrift client is reading the size of the string off the wire, then trying to create a buffer large enough to read the string. This is where it fails. When it fails here, it loses its place in the stream. The data is still in the stream, but the size is lost since that was read before the OOM error and there is no way to push the size back into the stream. At this point, Thrift client is hosed and it can't read anything else. > > When the JDBC Driver is closed, it sends a "close" request to HS2 to gracefully close the Hive session on the server. However, it is unable to read the response properly because, again, it lost its place and does not know how to find the next message. I'm not sure what the fix is here, but I don't think this is it. Yes, that's the truth, somehow a socket read timeout can also make the same trouble. The fix handles the root cause before beeline throwing the puzzling exception, make the beeline user have a way to fix the problem quickly. Maybe this is not a bug fix but just an improvement. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 441031) Time Spent: 3h (was: 2h 50m) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Bug > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 3h > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=440819=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-440819 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 03/Jun/20 15:00 Start Date: 03/Jun/20 15:00 Worklog Time Spent: 10m Work Description: belugabehr edited a comment on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-638240582 As I understand it, the code calling the JDBC Driver captures and handles the OOM error and then closes the JDBC Driver. When it closes, it throws another error " CloseOperation failed: out of sequence response". You would like for the close to be silent here. Is that correct? If so, there's not a ton that can be done here (other than perhaps wrap the OOM error in a Thrift Exception). In this particular case, Thrift client is reading the size of the string off the wire, then trying to create a buffer large enough to read the string. This is where it fails. When it fails here, it loses its place in the stream. The data is still in the stream, but the size is lost since that was read before the OOM error and there is no way to push the size back into the stream. At this point, Thrift client is hosed and it can't read anything else. When the JDBC Driver is closed, it sends a "close" request to HS2 to gracefully close the Hive session on the server. However, it is unable to read the response properly because, again, it lost its place and does not know how to find the next message. I'm not sure what the fix is here, but I don't think this is it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 440819) Time Spent: 2h 50m (was: 2h 40m) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Bug > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 2h 50m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=440815=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-440815 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 03/Jun/20 14:58 Start Date: 03/Jun/20 14:58 Worklog Time Spent: 10m Work Description: belugabehr edited a comment on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-638240582 As I understand it, the code calling the JDBC Driver captures and handles the OOM error and then closes the JDBC Driver. When it closes, it throws another error " CloseOperation failed: out of sequence response". You would like for the close to be silent here. Is that correct? If so, there's not a ton that can be done here (other than perhaps wrap the OOM error in a Thrift Exception). In this particular case, Thrift client is reading the size of the string off the wire, then trying to create a buffer large enough to read the string. This is where it fails. When it fails here, it loses its place in the stream. The data is still in the stream, but the size is lost since that was read before the OOM error and there is no way to push the size back into the stream. At this point, Thrift client is hosed and it can't read anything else. When the JDBC Driver is closed, it sends a "close" request to HS2 to gracefully shutdown the connection. However, it is unable to read the response properly because, again, it lost its place and does not know how to find the next message. I'm not sure what the fix is here, but I don't think this is it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 440815) Time Spent: 2h 40m (was: 2.5h) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Bug > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 2h 40m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=440814=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-440814 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 03/Jun/20 14:57 Start Date: 03/Jun/20 14:57 Worklog Time Spent: 10m Work Description: belugabehr edited a comment on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-638240582 As I understand it, the code calling the JDBC Driver captures and handles the OOM error and then closes the JDBC Driver. When it closes, it throws another error " CloseOperation failed: out of sequence response". You would like for the close to be silent here. Is that correct? If so, there's not a ton that can be done here (other than perhaps wrap the OOM error in a Thrift Exception). In this particular case, Thrift client is reading the size of the string off the wire, then trying to create a buffer large enough to read the string. This is where it fails. When it fails here, it loses its place in the stream. The data is still in the stream, but the size is lost since that was read before the OOM error. At this point, Thrift client is hosed and it can't read anything else. When the JDBC Driver is closed, it sends a "close" request to HS2 to gracefully shutdown the connection. However, it is unable to read the response properly because, again, it lost its place and does not know how to find the next message. I'm not sure what the fix is here, but I don't think this is it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 440814) Time Spent: 2.5h (was: 2h 20m) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Bug > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 2.5h > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=440813=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-440813 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 03/Jun/20 14:57 Start Date: 03/Jun/20 14:57 Worklog Time Spent: 10m Work Description: belugabehr edited a comment on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-638240582 As I understand it, the code calling the JDBC Driver captures and handles the OOM error and then closes the JDBC Driver. When it closes, it throws another error " CloseOperation failed: out of sequence response". You would like for the close to be silent here. Is that correct? If so, there's not a ton that can be done here (other than perhaps wrap the OOM error). In this particular case, Thrift client is reading the size of the string off the wire, then trying to create a buffer large enough to read the string. This is where it fails. When it fails here, it loses its place in the stream. The data is still in the stream, but the size is lost since that was read before the OOM error. At this point, Thrift client is hosed and it can't read anything else. When the JDBC Driver is closed, it sends a "close" request to HS2 to gracefully shutdown the connection. However, it is unable to read the response properly because, again, it lost its place and does not know how to find the next message. I'm not sure what the fix is here, but I don't think this is it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 440813) Time Spent: 2h 20m (was: 2h 10m) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Bug > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 2h 20m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=440803=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-440803 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 03/Jun/20 14:37 Start Date: 03/Jun/20 14:37 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-638240582 As I understand it, the code calling the JDBC Driver captures and handles the OOM error and then closes the JDBC Driver. When it closes, it throws another error " CloseOperation failed: out of sequence response". You would like for the close to be silent here. Is that correct? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 440803) Time Spent: 2h 10m (was: 2h) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Bug > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=440519=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-440519
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 02/Jun/20 23:45
Start Date: 02/Jun/20 23:45
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r433726195
##
File path: beeline/src/java/org/apache/hive/beeline/Commands.java
##
@@ -974,86 +974,79 @@ private boolean executeInternal(String sql, boolean call)
{
ClientHook hook = ClientCommandHookFactory.get().getHook(beeLine, sql);
-try {
- Statement stmnt = null;
- boolean hasResults;
- Thread logThread = null;
+Statement stmnt = null;
+boolean hasResults;
+Thread logThread = null;
- try {
-long start = System.currentTimeMillis();
+try {
+ long start = System.currentTimeMillis();
-if (call) {
- stmnt =
beeLine.getDatabaseConnection().getConnection().prepareCall(sql);
- hasResults = ((CallableStatement) stmnt).execute();
+ if (call) {
+stmnt =
beeLine.getDatabaseConnection().getConnection().prepareCall(sql);
+hasResults = ((CallableStatement) stmnt).execute();
+ } else {
+stmnt = beeLine.createStatement();
+// In test mode we want the operation logs regardless of the settings
+if (!beeLine.isTestMode() && beeLine.getOpts().isSilent()) {
+ hasResults = stmnt.execute(sql);
} else {
- stmnt = beeLine.createStatement();
- // In test mode we want the operation logs regardless of the settings
- if (!beeLine.isTestMode() && beeLine.getOpts().isSilent()) {
-hasResults = stmnt.execute(sql);
- } else {
-InPlaceUpdateStream.EventNotifier eventNotifier =
-new InPlaceUpdateStream.EventNotifier();
-logThread = new Thread(createLogRunnable(stmnt, eventNotifier));
-logThread.setDaemon(true);
-logThread.start();
-if (stmnt instanceof HiveStatement) {
- HiveStatement hiveStatement = (HiveStatement) stmnt;
- hiveStatement.setInPlaceUpdateStream(
- new BeelineInPlaceUpdateStream(
- beeLine.getErrorStream(),
- eventNotifier
- ));
-}
-hasResults = stmnt.execute(sql);
-logThread.interrupt();
-logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
+ InPlaceUpdateStream.EventNotifier eventNotifier =
+ new InPlaceUpdateStream.EventNotifier();
+ logThread = new Thread(createLogRunnable(stmnt, eventNotifier));
+ logThread.setDaemon(true);
+ logThread.start();
+ if (stmnt instanceof HiveStatement) {
+HiveStatement hiveStatement = (HiveStatement) stmnt;
+hiveStatement.setInPlaceUpdateStream(
+new BeelineInPlaceUpdateStream(
+beeLine.getErrorStream(),
+eventNotifier
+));
}
+ hasResults = stmnt.execute(sql);
+ logThread.interrupt();
+ logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
}
+ }
-beeLine.showWarnings();
-
-if (hasResults) {
- OutputFile outputFile = beeLine.getRecordOutputFile();
- if (beeLine.isTestMode() && outputFile != null &&
outputFile.isActiveConverter()) {
-outputFile.fetchStarted();
-if (!sql.trim().toLowerCase().startsWith("explain")) {
- outputFile.foundQuery(true);
-} else {
- outputFile.foundQuery(false);
-}
- }
- do {
-ResultSet rs = stmnt.getResultSet();
-try {
- int count = beeLine.print(rs);
- long end = System.currentTimeMillis();
-
- if (showReport()) {
-beeLine.output(beeLine.loc("rows-selected", count) + " " +
beeLine.locElapsedTime(end - start),
-true, beeLine.getErrorStream());
- }
-} finally {
- if (logThread != null) {
-logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
-showRemainingLogsIfAny(stmnt);
-logThread = null;
- }
- rs.close();
Review comment:
I wondering if this maybe is not a thrift problem, reading out of
sequence is just a consequence of the upper request unpredictably interrupts
while the connection is still being alive. Fix lands in the jdbc driver maybe
is not a best approach, as user who uses jdbc driver can handle the root cause
at their logic
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=440099=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-440099
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 02/Jun/20 12:21
Start Date: 02/Jun/20 12:21
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r433726195
##
File path: beeline/src/java/org/apache/hive/beeline/Commands.java
##
@@ -974,86 +974,79 @@ private boolean executeInternal(String sql, boolean call)
{
ClientHook hook = ClientCommandHookFactory.get().getHook(beeLine, sql);
-try {
- Statement stmnt = null;
- boolean hasResults;
- Thread logThread = null;
+Statement stmnt = null;
+boolean hasResults;
+Thread logThread = null;
- try {
-long start = System.currentTimeMillis();
+try {
+ long start = System.currentTimeMillis();
-if (call) {
- stmnt =
beeLine.getDatabaseConnection().getConnection().prepareCall(sql);
- hasResults = ((CallableStatement) stmnt).execute();
+ if (call) {
+stmnt =
beeLine.getDatabaseConnection().getConnection().prepareCall(sql);
+hasResults = ((CallableStatement) stmnt).execute();
+ } else {
+stmnt = beeLine.createStatement();
+// In test mode we want the operation logs regardless of the settings
+if (!beeLine.isTestMode() && beeLine.getOpts().isSilent()) {
+ hasResults = stmnt.execute(sql);
} else {
- stmnt = beeLine.createStatement();
- // In test mode we want the operation logs regardless of the settings
- if (!beeLine.isTestMode() && beeLine.getOpts().isSilent()) {
-hasResults = stmnt.execute(sql);
- } else {
-InPlaceUpdateStream.EventNotifier eventNotifier =
-new InPlaceUpdateStream.EventNotifier();
-logThread = new Thread(createLogRunnable(stmnt, eventNotifier));
-logThread.setDaemon(true);
-logThread.start();
-if (stmnt instanceof HiveStatement) {
- HiveStatement hiveStatement = (HiveStatement) stmnt;
- hiveStatement.setInPlaceUpdateStream(
- new BeelineInPlaceUpdateStream(
- beeLine.getErrorStream(),
- eventNotifier
- ));
-}
-hasResults = stmnt.execute(sql);
-logThread.interrupt();
-logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
+ InPlaceUpdateStream.EventNotifier eventNotifier =
+ new InPlaceUpdateStream.EventNotifier();
+ logThread = new Thread(createLogRunnable(stmnt, eventNotifier));
+ logThread.setDaemon(true);
+ logThread.start();
+ if (stmnt instanceof HiveStatement) {
+HiveStatement hiveStatement = (HiveStatement) stmnt;
+hiveStatement.setInPlaceUpdateStream(
+new BeelineInPlaceUpdateStream(
+beeLine.getErrorStream(),
+eventNotifier
+));
}
+ hasResults = stmnt.execute(sql);
+ logThread.interrupt();
+ logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
}
+ }
-beeLine.showWarnings();
-
-if (hasResults) {
- OutputFile outputFile = beeLine.getRecordOutputFile();
- if (beeLine.isTestMode() && outputFile != null &&
outputFile.isActiveConverter()) {
-outputFile.fetchStarted();
-if (!sql.trim().toLowerCase().startsWith("explain")) {
- outputFile.foundQuery(true);
-} else {
- outputFile.foundQuery(false);
-}
- }
- do {
-ResultSet rs = stmnt.getResultSet();
-try {
- int count = beeLine.print(rs);
- long end = System.currentTimeMillis();
-
- if (showReport()) {
-beeLine.output(beeLine.loc("rows-selected", count) + " " +
beeLine.locElapsedTime(end - start),
-true, beeLine.getErrorStream());
- }
-} finally {
- if (logThread != null) {
-logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
-showRemainingLogsIfAny(stmnt);
-logThread = null;
- }
- rs.close();
Review comment:
I wondering if this maybe is not a thrift problem, reading out of
sequence is just a consequence of the upper request unpredictably interrupts by
some reasons. Fix lands in the jdbc driver maybe is not a best approach, as
user who uses jdbc driver can handle the root cause at their logic as beeline
does.
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=440089=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-440089
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 02/Jun/20 12:00
Start Date: 02/Jun/20 12:00
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r433726195
##
File path: beeline/src/java/org/apache/hive/beeline/Commands.java
##
@@ -974,86 +974,79 @@ private boolean executeInternal(String sql, boolean call)
{
ClientHook hook = ClientCommandHookFactory.get().getHook(beeLine, sql);
-try {
- Statement stmnt = null;
- boolean hasResults;
- Thread logThread = null;
+Statement stmnt = null;
+boolean hasResults;
+Thread logThread = null;
- try {
-long start = System.currentTimeMillis();
+try {
+ long start = System.currentTimeMillis();
-if (call) {
- stmnt =
beeLine.getDatabaseConnection().getConnection().prepareCall(sql);
- hasResults = ((CallableStatement) stmnt).execute();
+ if (call) {
+stmnt =
beeLine.getDatabaseConnection().getConnection().prepareCall(sql);
+hasResults = ((CallableStatement) stmnt).execute();
+ } else {
+stmnt = beeLine.createStatement();
+// In test mode we want the operation logs regardless of the settings
+if (!beeLine.isTestMode() && beeLine.getOpts().isSilent()) {
+ hasResults = stmnt.execute(sql);
} else {
- stmnt = beeLine.createStatement();
- // In test mode we want the operation logs regardless of the settings
- if (!beeLine.isTestMode() && beeLine.getOpts().isSilent()) {
-hasResults = stmnt.execute(sql);
- } else {
-InPlaceUpdateStream.EventNotifier eventNotifier =
-new InPlaceUpdateStream.EventNotifier();
-logThread = new Thread(createLogRunnable(stmnt, eventNotifier));
-logThread.setDaemon(true);
-logThread.start();
-if (stmnt instanceof HiveStatement) {
- HiveStatement hiveStatement = (HiveStatement) stmnt;
- hiveStatement.setInPlaceUpdateStream(
- new BeelineInPlaceUpdateStream(
- beeLine.getErrorStream(),
- eventNotifier
- ));
-}
-hasResults = stmnt.execute(sql);
-logThread.interrupt();
-logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
+ InPlaceUpdateStream.EventNotifier eventNotifier =
+ new InPlaceUpdateStream.EventNotifier();
+ logThread = new Thread(createLogRunnable(stmnt, eventNotifier));
+ logThread.setDaemon(true);
+ logThread.start();
+ if (stmnt instanceof HiveStatement) {
+HiveStatement hiveStatement = (HiveStatement) stmnt;
+hiveStatement.setInPlaceUpdateStream(
+new BeelineInPlaceUpdateStream(
+beeLine.getErrorStream(),
+eventNotifier
+));
}
+ hasResults = stmnt.execute(sql);
+ logThread.interrupt();
+ logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
}
+ }
-beeLine.showWarnings();
-
-if (hasResults) {
- OutputFile outputFile = beeLine.getRecordOutputFile();
- if (beeLine.isTestMode() && outputFile != null &&
outputFile.isActiveConverter()) {
-outputFile.fetchStarted();
-if (!sql.trim().toLowerCase().startsWith("explain")) {
- outputFile.foundQuery(true);
-} else {
- outputFile.foundQuery(false);
-}
- }
- do {
-ResultSet rs = stmnt.getResultSet();
-try {
- int count = beeLine.print(rs);
- long end = System.currentTimeMillis();
-
- if (showReport()) {
-beeLine.output(beeLine.loc("rows-selected", count) + " " +
beeLine.locElapsedTime(end - start),
-true, beeLine.getErrorStream());
- }
-} finally {
- if (logThread != null) {
-logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
-showRemainingLogsIfAny(stmnt);
-logThread = null;
- }
- rs.close();
Review comment:
I wondering if this maybe is not a thrift problem, reading out of
sequence is just a consequence of the upper request unpredictably changes by
some reasons. Fix lands in the jdbc driver maybe is not a best approach, as
user who uses jdbc driver can handle the root cause at their logic as beeline
does.
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=440088=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-440088
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 02/Jun/20 12:00
Start Date: 02/Jun/20 12:00
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r433726195
##
File path: beeline/src/java/org/apache/hive/beeline/Commands.java
##
@@ -974,86 +974,79 @@ private boolean executeInternal(String sql, boolean call)
{
ClientHook hook = ClientCommandHookFactory.get().getHook(beeLine, sql);
-try {
- Statement stmnt = null;
- boolean hasResults;
- Thread logThread = null;
+Statement stmnt = null;
+boolean hasResults;
+Thread logThread = null;
- try {
-long start = System.currentTimeMillis();
+try {
+ long start = System.currentTimeMillis();
-if (call) {
- stmnt =
beeLine.getDatabaseConnection().getConnection().prepareCall(sql);
- hasResults = ((CallableStatement) stmnt).execute();
+ if (call) {
+stmnt =
beeLine.getDatabaseConnection().getConnection().prepareCall(sql);
+hasResults = ((CallableStatement) stmnt).execute();
+ } else {
+stmnt = beeLine.createStatement();
+// In test mode we want the operation logs regardless of the settings
+if (!beeLine.isTestMode() && beeLine.getOpts().isSilent()) {
+ hasResults = stmnt.execute(sql);
} else {
- stmnt = beeLine.createStatement();
- // In test mode we want the operation logs regardless of the settings
- if (!beeLine.isTestMode() && beeLine.getOpts().isSilent()) {
-hasResults = stmnt.execute(sql);
- } else {
-InPlaceUpdateStream.EventNotifier eventNotifier =
-new InPlaceUpdateStream.EventNotifier();
-logThread = new Thread(createLogRunnable(stmnt, eventNotifier));
-logThread.setDaemon(true);
-logThread.start();
-if (stmnt instanceof HiveStatement) {
- HiveStatement hiveStatement = (HiveStatement) stmnt;
- hiveStatement.setInPlaceUpdateStream(
- new BeelineInPlaceUpdateStream(
- beeLine.getErrorStream(),
- eventNotifier
- ));
-}
-hasResults = stmnt.execute(sql);
-logThread.interrupt();
-logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
+ InPlaceUpdateStream.EventNotifier eventNotifier =
+ new InPlaceUpdateStream.EventNotifier();
+ logThread = new Thread(createLogRunnable(stmnt, eventNotifier));
+ logThread.setDaemon(true);
+ logThread.start();
+ if (stmnt instanceof HiveStatement) {
+HiveStatement hiveStatement = (HiveStatement) stmnt;
+hiveStatement.setInPlaceUpdateStream(
+new BeelineInPlaceUpdateStream(
+beeLine.getErrorStream(),
+eventNotifier
+));
}
+ hasResults = stmnt.execute(sql);
+ logThread.interrupt();
+ logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
}
+ }
-beeLine.showWarnings();
-
-if (hasResults) {
- OutputFile outputFile = beeLine.getRecordOutputFile();
- if (beeLine.isTestMode() && outputFile != null &&
outputFile.isActiveConverter()) {
-outputFile.fetchStarted();
-if (!sql.trim().toLowerCase().startsWith("explain")) {
- outputFile.foundQuery(true);
-} else {
- outputFile.foundQuery(false);
-}
- }
- do {
-ResultSet rs = stmnt.getResultSet();
-try {
- int count = beeLine.print(rs);
- long end = System.currentTimeMillis();
-
- if (showReport()) {
-beeLine.output(beeLine.loc("rows-selected", count) + " " +
beeLine.locElapsedTime(end - start),
-true, beeLine.getErrorStream());
- }
-} finally {
- if (logThread != null) {
-logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
-showRemainingLogsIfAny(stmnt);
-logThread = null;
- }
- rs.close();
Review comment:
I wondering if this maybe is not a thrift problem, reading out of
sequence is just a consequence of the upper request unpredictably changed by
some reasons. Fix lands in the jdbc driver maybe is not a best approach, as
user who uses jdbc driver can handle the root cause at their logic as beeline
does.
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=440025=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-440025
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 02/Jun/20 09:25
Start Date: 02/Jun/20 09:25
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r433726195
##
File path: beeline/src/java/org/apache/hive/beeline/Commands.java
##
@@ -974,86 +974,79 @@ private boolean executeInternal(String sql, boolean call)
{
ClientHook hook = ClientCommandHookFactory.get().getHook(beeLine, sql);
-try {
- Statement stmnt = null;
- boolean hasResults;
- Thread logThread = null;
+Statement stmnt = null;
+boolean hasResults;
+Thread logThread = null;
- try {
-long start = System.currentTimeMillis();
+try {
+ long start = System.currentTimeMillis();
-if (call) {
- stmnt =
beeLine.getDatabaseConnection().getConnection().prepareCall(sql);
- hasResults = ((CallableStatement) stmnt).execute();
+ if (call) {
+stmnt =
beeLine.getDatabaseConnection().getConnection().prepareCall(sql);
+hasResults = ((CallableStatement) stmnt).execute();
+ } else {
+stmnt = beeLine.createStatement();
+// In test mode we want the operation logs regardless of the settings
+if (!beeLine.isTestMode() && beeLine.getOpts().isSilent()) {
+ hasResults = stmnt.execute(sql);
} else {
- stmnt = beeLine.createStatement();
- // In test mode we want the operation logs regardless of the settings
- if (!beeLine.isTestMode() && beeLine.getOpts().isSilent()) {
-hasResults = stmnt.execute(sql);
- } else {
-InPlaceUpdateStream.EventNotifier eventNotifier =
-new InPlaceUpdateStream.EventNotifier();
-logThread = new Thread(createLogRunnable(stmnt, eventNotifier));
-logThread.setDaemon(true);
-logThread.start();
-if (stmnt instanceof HiveStatement) {
- HiveStatement hiveStatement = (HiveStatement) stmnt;
- hiveStatement.setInPlaceUpdateStream(
- new BeelineInPlaceUpdateStream(
- beeLine.getErrorStream(),
- eventNotifier
- ));
-}
-hasResults = stmnt.execute(sql);
-logThread.interrupt();
-logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
+ InPlaceUpdateStream.EventNotifier eventNotifier =
+ new InPlaceUpdateStream.EventNotifier();
+ logThread = new Thread(createLogRunnable(stmnt, eventNotifier));
+ logThread.setDaemon(true);
+ logThread.start();
+ if (stmnt instanceof HiveStatement) {
+HiveStatement hiveStatement = (HiveStatement) stmnt;
+hiveStatement.setInPlaceUpdateStream(
+new BeelineInPlaceUpdateStream(
+beeLine.getErrorStream(),
+eventNotifier
+));
}
+ hasResults = stmnt.execute(sql);
+ logThread.interrupt();
+ logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
}
+ }
-beeLine.showWarnings();
-
-if (hasResults) {
- OutputFile outputFile = beeLine.getRecordOutputFile();
- if (beeLine.isTestMode() && outputFile != null &&
outputFile.isActiveConverter()) {
-outputFile.fetchStarted();
-if (!sql.trim().toLowerCase().startsWith("explain")) {
- outputFile.foundQuery(true);
-} else {
- outputFile.foundQuery(false);
-}
- }
- do {
-ResultSet rs = stmnt.getResultSet();
-try {
- int count = beeLine.print(rs);
- long end = System.currentTimeMillis();
-
- if (showReport()) {
-beeLine.output(beeLine.loc("rows-selected", count) + " " +
beeLine.locElapsedTime(end - start),
-true, beeLine.getErrorStream());
- }
-} finally {
- if (logThread != null) {
-logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
-showRemainingLogsIfAny(stmnt);
-logThread = null;
- }
- rs.close();
Review comment:
I wondering if this maybe is not a thrift problem, reading out of
sequence is just a consequence of the upper request suddenly changed by some
reasons. Fix lands in the jdbc driver maybe is not a best approach, as user
who uses jdbc driver can handle the root cause at their logic as beeline does.
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=439994=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439994
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 02/Jun/20 09:00
Start Date: 02/Jun/20 09:00
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r433726195
##
File path: beeline/src/java/org/apache/hive/beeline/Commands.java
##
@@ -974,86 +974,79 @@ private boolean executeInternal(String sql, boolean call)
{
ClientHook hook = ClientCommandHookFactory.get().getHook(beeLine, sql);
-try {
- Statement stmnt = null;
- boolean hasResults;
- Thread logThread = null;
+Statement stmnt = null;
+boolean hasResults;
+Thread logThread = null;
- try {
-long start = System.currentTimeMillis();
+try {
+ long start = System.currentTimeMillis();
-if (call) {
- stmnt =
beeLine.getDatabaseConnection().getConnection().prepareCall(sql);
- hasResults = ((CallableStatement) stmnt).execute();
+ if (call) {
+stmnt =
beeLine.getDatabaseConnection().getConnection().prepareCall(sql);
+hasResults = ((CallableStatement) stmnt).execute();
+ } else {
+stmnt = beeLine.createStatement();
+// In test mode we want the operation logs regardless of the settings
+if (!beeLine.isTestMode() && beeLine.getOpts().isSilent()) {
+ hasResults = stmnt.execute(sql);
} else {
- stmnt = beeLine.createStatement();
- // In test mode we want the operation logs regardless of the settings
- if (!beeLine.isTestMode() && beeLine.getOpts().isSilent()) {
-hasResults = stmnt.execute(sql);
- } else {
-InPlaceUpdateStream.EventNotifier eventNotifier =
-new InPlaceUpdateStream.EventNotifier();
-logThread = new Thread(createLogRunnable(stmnt, eventNotifier));
-logThread.setDaemon(true);
-logThread.start();
-if (stmnt instanceof HiveStatement) {
- HiveStatement hiveStatement = (HiveStatement) stmnt;
- hiveStatement.setInPlaceUpdateStream(
- new BeelineInPlaceUpdateStream(
- beeLine.getErrorStream(),
- eventNotifier
- ));
-}
-hasResults = stmnt.execute(sql);
-logThread.interrupt();
-logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
+ InPlaceUpdateStream.EventNotifier eventNotifier =
+ new InPlaceUpdateStream.EventNotifier();
+ logThread = new Thread(createLogRunnable(stmnt, eventNotifier));
+ logThread.setDaemon(true);
+ logThread.start();
+ if (stmnt instanceof HiveStatement) {
+HiveStatement hiveStatement = (HiveStatement) stmnt;
+hiveStatement.setInPlaceUpdateStream(
+new BeelineInPlaceUpdateStream(
+beeLine.getErrorStream(),
+eventNotifier
+));
}
+ hasResults = stmnt.execute(sql);
+ logThread.interrupt();
+ logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
}
+ }
-beeLine.showWarnings();
-
-if (hasResults) {
- OutputFile outputFile = beeLine.getRecordOutputFile();
- if (beeLine.isTestMode() && outputFile != null &&
outputFile.isActiveConverter()) {
-outputFile.fetchStarted();
-if (!sql.trim().toLowerCase().startsWith("explain")) {
- outputFile.foundQuery(true);
-} else {
- outputFile.foundQuery(false);
-}
- }
- do {
-ResultSet rs = stmnt.getResultSet();
-try {
- int count = beeLine.print(rs);
- long end = System.currentTimeMillis();
-
- if (showReport()) {
-beeLine.output(beeLine.loc("rows-selected", count) + " " +
beeLine.locElapsedTime(end - start),
-true, beeLine.getErrorStream());
- }
-} finally {
- if (logThread != null) {
-logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
-showRemainingLogsIfAny(stmnt);
-logThread = null;
- }
- rs.close();
Review comment:
I wondering this maybe is not a thrift problem, reading out of sequence
is just a result caused by the upper request suddenly changed by some reasons.
User using jdbc driver can handle the root cause at their logic as beeline does.
This is an
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=439982=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439982
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 02/Jun/20 08:27
Start Date: 02/Jun/20 08:27
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r433706839
##
File path: beeline/src/java/org/apache/hive/beeline/Commands.java
##
@@ -974,86 +974,79 @@ private boolean executeInternal(String sql, boolean call)
{
ClientHook hook = ClientCommandHookFactory.get().getHook(beeLine, sql);
-try {
- Statement stmnt = null;
- boolean hasResults;
- Thread logThread = null;
+Statement stmnt = null;
+boolean hasResults;
+Thread logThread = null;
- try {
-long start = System.currentTimeMillis();
+try {
+ long start = System.currentTimeMillis();
-if (call) {
- stmnt =
beeLine.getDatabaseConnection().getConnection().prepareCall(sql);
- hasResults = ((CallableStatement) stmnt).execute();
+ if (call) {
+stmnt =
beeLine.getDatabaseConnection().getConnection().prepareCall(sql);
+hasResults = ((CallableStatement) stmnt).execute();
+ } else {
+stmnt = beeLine.createStatement();
+// In test mode we want the operation logs regardless of the settings
+if (!beeLine.isTestMode() && beeLine.getOpts().isSilent()) {
+ hasResults = stmnt.execute(sql);
} else {
- stmnt = beeLine.createStatement();
- // In test mode we want the operation logs regardless of the settings
- if (!beeLine.isTestMode() && beeLine.getOpts().isSilent()) {
-hasResults = stmnt.execute(sql);
- } else {
-InPlaceUpdateStream.EventNotifier eventNotifier =
-new InPlaceUpdateStream.EventNotifier();
-logThread = new Thread(createLogRunnable(stmnt, eventNotifier));
-logThread.setDaemon(true);
-logThread.start();
-if (stmnt instanceof HiveStatement) {
- HiveStatement hiveStatement = (HiveStatement) stmnt;
- hiveStatement.setInPlaceUpdateStream(
- new BeelineInPlaceUpdateStream(
- beeLine.getErrorStream(),
- eventNotifier
- ));
-}
-hasResults = stmnt.execute(sql);
-logThread.interrupt();
-logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
+ InPlaceUpdateStream.EventNotifier eventNotifier =
+ new InPlaceUpdateStream.EventNotifier();
+ logThread = new Thread(createLogRunnable(stmnt, eventNotifier));
+ logThread.setDaemon(true);
+ logThread.start();
+ if (stmnt instanceof HiveStatement) {
+HiveStatement hiveStatement = (HiveStatement) stmnt;
+hiveStatement.setInPlaceUpdateStream(
+new BeelineInPlaceUpdateStream(
+beeLine.getErrorStream(),
+eventNotifier
+));
}
+ hasResults = stmnt.execute(sql);
+ logThread.interrupt();
+ logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
}
+ }
-beeLine.showWarnings();
-
-if (hasResults) {
- OutputFile outputFile = beeLine.getRecordOutputFile();
- if (beeLine.isTestMode() && outputFile != null &&
outputFile.isActiveConverter()) {
-outputFile.fetchStarted();
-if (!sql.trim().toLowerCase().startsWith("explain")) {
- outputFile.foundQuery(true);
-} else {
- outputFile.foundQuery(false);
-}
- }
- do {
-ResultSet rs = stmnt.getResultSet();
-try {
- int count = beeLine.print(rs);
- long end = System.currentTimeMillis();
-
- if (showReport()) {
-beeLine.output(beeLine.loc("rows-selected", count) + " " +
beeLine.locElapsedTime(end - start),
-true, beeLine.getErrorStream());
- }
-} finally {
- if (logThread != null) {
-logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
-showRemainingLogsIfAny(stmnt);
-logThread = null;
- }
- rs.close();
Review comment:
I've just looked around...it seems like a thrift upgrade(HIVE-21000)
would need an accumulo upgrade(HIVE-0) which would need a guava
upgrade(HIVE-21569) ...
This is an automated message from the Apache Git Service.
To respond to the message, please
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[ https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=439979=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439979 ] ASF GitHub Bot logged work on HIVE-23526: - Author: ASF GitHub Bot Created on: 02/Jun/20 08:26 Start Date: 02/Jun/20 08:26 Worklog Time Spent: 10m Work Description: dengzhhu653 commented on pull request #1029: URL: https://github.com/apache/hive/pull/1029#issuecomment-637378452 > From the description in the jira it seems to me that the thrift should have thrown an exception because it was not able to read properly. > > Do you know if a more recent version of thrift have this fixed or something? Have tested on thrift version 0.9.3, same problem occurs, the version >0.9.3 hasn't tried yet. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 439979) Time Spent: 50m (was: 40m) > Beeline may throw the misleading exception > -- > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Bug > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.3.patch, > HIVE-23526.patch, outofsequence.log > > Time Spent: 50m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage message to figured it out, even with --verbose, > this makes problem puzzled as beeline does not have concurrency problem on > underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=439974=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439974
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 02/Jun/20 08:23
Start Date: 02/Jun/20 08:23
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r433704040
##
File path: beeline/src/java/org/apache/hive/beeline/Commands.java
##
@@ -974,86 +974,79 @@ private boolean executeInternal(String sql, boolean call)
{
ClientHook hook = ClientCommandHookFactory.get().getHook(beeLine, sql);
-try {
- Statement stmnt = null;
- boolean hasResults;
- Thread logThread = null;
+Statement stmnt = null;
+boolean hasResults;
+Thread logThread = null;
- try {
-long start = System.currentTimeMillis();
+try {
+ long start = System.currentTimeMillis();
-if (call) {
- stmnt =
beeLine.getDatabaseConnection().getConnection().prepareCall(sql);
- hasResults = ((CallableStatement) stmnt).execute();
+ if (call) {
+stmnt =
beeLine.getDatabaseConnection().getConnection().prepareCall(sql);
+hasResults = ((CallableStatement) stmnt).execute();
+ } else {
+stmnt = beeLine.createStatement();
+// In test mode we want the operation logs regardless of the settings
+if (!beeLine.isTestMode() && beeLine.getOpts().isSilent()) {
+ hasResults = stmnt.execute(sql);
} else {
- stmnt = beeLine.createStatement();
- // In test mode we want the operation logs regardless of the settings
- if (!beeLine.isTestMode() && beeLine.getOpts().isSilent()) {
-hasResults = stmnt.execute(sql);
- } else {
-InPlaceUpdateStream.EventNotifier eventNotifier =
-new InPlaceUpdateStream.EventNotifier();
-logThread = new Thread(createLogRunnable(stmnt, eventNotifier));
-logThread.setDaemon(true);
-logThread.start();
-if (stmnt instanceof HiveStatement) {
- HiveStatement hiveStatement = (HiveStatement) stmnt;
- hiveStatement.setInPlaceUpdateStream(
- new BeelineInPlaceUpdateStream(
- beeLine.getErrorStream(),
- eventNotifier
- ));
-}
-hasResults = stmnt.execute(sql);
-logThread.interrupt();
-logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
+ InPlaceUpdateStream.EventNotifier eventNotifier =
+ new InPlaceUpdateStream.EventNotifier();
+ logThread = new Thread(createLogRunnable(stmnt, eventNotifier));
+ logThread.setDaemon(true);
+ logThread.start();
+ if (stmnt instanceof HiveStatement) {
+HiveStatement hiveStatement = (HiveStatement) stmnt;
+hiveStatement.setInPlaceUpdateStream(
+new BeelineInPlaceUpdateStream(
+beeLine.getErrorStream(),
+eventNotifier
+));
}
+ hasResults = stmnt.execute(sql);
+ logThread.interrupt();
+ logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
}
+ }
-beeLine.showWarnings();
-
-if (hasResults) {
- OutputFile outputFile = beeLine.getRecordOutputFile();
- if (beeLine.isTestMode() && outputFile != null &&
outputFile.isActiveConverter()) {
-outputFile.fetchStarted();
-if (!sql.trim().toLowerCase().startsWith("explain")) {
- outputFile.foundQuery(true);
-} else {
- outputFile.foundQuery(false);
-}
- }
- do {
-ResultSet rs = stmnt.getResultSet();
-try {
- int count = beeLine.print(rs);
- long end = System.currentTimeMillis();
-
- if (showReport()) {
-beeLine.output(beeLine.loc("rows-selected", count) + " " +
beeLine.locElapsedTime(end - start),
-true, beeLine.getErrorStream());
- }
-} finally {
- if (logThread != null) {
-logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
-showRemainingLogsIfAny(stmnt);
-logThread = null;
- }
- rs.close();
Review comment:
in case it should be fixed in some other way `ResultSet` is
`AutoCloseable` so it is expected to call `close` on it no matter what - I now
think that this fix should either land in the jdbc driver - or should be fixed
in thrift
This is an
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=439971=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439971
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 02/Jun/20 08:20
Start Date: 02/Jun/20 08:20
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r433702322
##
File path: beeline/src/java/org/apache/hive/beeline/Commands.java
##
@@ -974,86 +974,79 @@ private boolean executeInternal(String sql, boolean call)
{
ClientHook hook = ClientCommandHookFactory.get().getHook(beeLine, sql);
-try {
- Statement stmnt = null;
- boolean hasResults;
- Thread logThread = null;
+Statement stmnt = null;
+boolean hasResults;
+Thread logThread = null;
- try {
-long start = System.currentTimeMillis();
+try {
+ long start = System.currentTimeMillis();
-if (call) {
- stmnt =
beeLine.getDatabaseConnection().getConnection().prepareCall(sql);
- hasResults = ((CallableStatement) stmnt).execute();
+ if (call) {
+stmnt =
beeLine.getDatabaseConnection().getConnection().prepareCall(sql);
+hasResults = ((CallableStatement) stmnt).execute();
+ } else {
+stmnt = beeLine.createStatement();
+// In test mode we want the operation logs regardless of the settings
+if (!beeLine.isTestMode() && beeLine.getOpts().isSilent()) {
+ hasResults = stmnt.execute(sql);
} else {
- stmnt = beeLine.createStatement();
- // In test mode we want the operation logs regardless of the settings
- if (!beeLine.isTestMode() && beeLine.getOpts().isSilent()) {
-hasResults = stmnt.execute(sql);
- } else {
-InPlaceUpdateStream.EventNotifier eventNotifier =
-new InPlaceUpdateStream.EventNotifier();
-logThread = new Thread(createLogRunnable(stmnt, eventNotifier));
-logThread.setDaemon(true);
-logThread.start();
-if (stmnt instanceof HiveStatement) {
- HiveStatement hiveStatement = (HiveStatement) stmnt;
- hiveStatement.setInPlaceUpdateStream(
- new BeelineInPlaceUpdateStream(
- beeLine.getErrorStream(),
- eventNotifier
- ));
-}
-hasResults = stmnt.execute(sql);
-logThread.interrupt();
-logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
+ InPlaceUpdateStream.EventNotifier eventNotifier =
+ new InPlaceUpdateStream.EventNotifier();
+ logThread = new Thread(createLogRunnable(stmnt, eventNotifier));
+ logThread.setDaemon(true);
+ logThread.start();
+ if (stmnt instanceof HiveStatement) {
+HiveStatement hiveStatement = (HiveStatement) stmnt;
+hiveStatement.setInPlaceUpdateStream(
+new BeelineInPlaceUpdateStream(
+beeLine.getErrorStream(),
+eventNotifier
+));
}
+ hasResults = stmnt.execute(sql);
+ logThread.interrupt();
+ logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
}
+ }
-beeLine.showWarnings();
-
-if (hasResults) {
- OutputFile outputFile = beeLine.getRecordOutputFile();
- if (beeLine.isTestMode() && outputFile != null &&
outputFile.isActiveConverter()) {
-outputFile.fetchStarted();
-if (!sql.trim().toLowerCase().startsWith("explain")) {
- outputFile.foundQuery(true);
-} else {
- outputFile.foundQuery(false);
-}
- }
- do {
-ResultSet rs = stmnt.getResultSet();
-try {
- int count = beeLine.print(rs);
- long end = System.currentTimeMillis();
-
- if (showReport()) {
-beeLine.output(beeLine.loc("rows-selected", count) + " " +
beeLine.locElapsedTime(end - start),
-true, beeLine.getErrorStream());
- }
-} finally {
- if (logThread != null) {
-logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
-showRemainingLogsIfAny(stmnt);
-logThread = null;
- }
- rs.close();
Review comment:
resultset::close() will close remote operation, this can be also
achieved by stmnt::close() in finally block, wondering if this can be removed
to make codes more elegant.
This is an automated message from the Apache Git Service.
To respond to the
[jira] [Work logged] (HIVE-23526) Beeline may throw the misleading exception
[
https://issues.apache.org/jira/browse/HIVE-23526?focusedWorklogId=439959=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-439959
]
ASF GitHub Bot logged work on HIVE-23526:
-
Author: ASF GitHub Bot
Created on: 02/Jun/20 07:40
Start Date: 02/Jun/20 07:40
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on a change in pull request #1029:
URL: https://github.com/apache/hive/pull/1029#discussion_r433678995
##
File path: beeline/src/java/org/apache/hive/beeline/Commands.java
##
@@ -974,86 +974,79 @@ private boolean executeInternal(String sql, boolean call)
{
ClientHook hook = ClientCommandHookFactory.get().getHook(beeLine, sql);
-try {
- Statement stmnt = null;
- boolean hasResults;
- Thread logThread = null;
+Statement stmnt = null;
+boolean hasResults;
+Thread logThread = null;
- try {
-long start = System.currentTimeMillis();
+try {
+ long start = System.currentTimeMillis();
-if (call) {
- stmnt =
beeLine.getDatabaseConnection().getConnection().prepareCall(sql);
- hasResults = ((CallableStatement) stmnt).execute();
+ if (call) {
+stmnt =
beeLine.getDatabaseConnection().getConnection().prepareCall(sql);
+hasResults = ((CallableStatement) stmnt).execute();
+ } else {
+stmnt = beeLine.createStatement();
+// In test mode we want the operation logs regardless of the settings
+if (!beeLine.isTestMode() && beeLine.getOpts().isSilent()) {
+ hasResults = stmnt.execute(sql);
} else {
- stmnt = beeLine.createStatement();
- // In test mode we want the operation logs regardless of the settings
- if (!beeLine.isTestMode() && beeLine.getOpts().isSilent()) {
-hasResults = stmnt.execute(sql);
- } else {
-InPlaceUpdateStream.EventNotifier eventNotifier =
-new InPlaceUpdateStream.EventNotifier();
-logThread = new Thread(createLogRunnable(stmnt, eventNotifier));
-logThread.setDaemon(true);
-logThread.start();
-if (stmnt instanceof HiveStatement) {
- HiveStatement hiveStatement = (HiveStatement) stmnt;
- hiveStatement.setInPlaceUpdateStream(
- new BeelineInPlaceUpdateStream(
- beeLine.getErrorStream(),
- eventNotifier
- ));
-}
-hasResults = stmnt.execute(sql);
-logThread.interrupt();
-logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
+ InPlaceUpdateStream.EventNotifier eventNotifier =
+ new InPlaceUpdateStream.EventNotifier();
+ logThread = new Thread(createLogRunnable(stmnt, eventNotifier));
+ logThread.setDaemon(true);
+ logThread.start();
+ if (stmnt instanceof HiveStatement) {
+HiveStatement hiveStatement = (HiveStatement) stmnt;
+hiveStatement.setInPlaceUpdateStream(
+new BeelineInPlaceUpdateStream(
+beeLine.getErrorStream(),
+eventNotifier
+));
}
+ hasResults = stmnt.execute(sql);
+ logThread.interrupt();
+ logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
}
+ }
-beeLine.showWarnings();
-
-if (hasResults) {
- OutputFile outputFile = beeLine.getRecordOutputFile();
- if (beeLine.isTestMode() && outputFile != null &&
outputFile.isActiveConverter()) {
-outputFile.fetchStarted();
-if (!sql.trim().toLowerCase().startsWith("explain")) {
- outputFile.foundQuery(true);
-} else {
- outputFile.foundQuery(false);
-}
- }
- do {
-ResultSet rs = stmnt.getResultSet();
-try {
- int count = beeLine.print(rs);
- long end = System.currentTimeMillis();
-
- if (showReport()) {
-beeLine.output(beeLine.loc("rows-selected", count) + " " +
beeLine.locElapsedTime(end - start),
-true, beeLine.getErrorStream());
- }
-} finally {
- if (logThread != null) {
-logThread.join(DEFAULT_QUERY_PROGRESS_THREAD_TIMEOUT);
-showRemainingLogsIfAny(stmnt);
-logThread = null;
- }
- rs.close();
Review comment:
removing this line will mean that the resultset will be never closed -
even when the issue doesn't happen...
isn't there a way to fix this w/o removing this close call?
This is an automated message from the Apache Git Service.
To respond to the
