[jira] [Commented] (HIVE-13960) Session timeout may happen before HIVE_SERVER2_IDLE_SESSION_TIMEOUT for back-to-back synchronous operations.
[ https://issues.apache.org/jira/browse/HIVE-13960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340739#comment-15340739 ] zhihai xu commented on HIVE-13960: -- Yes, renaming pendingCount to activeCalls sounds good to me. Will fix it in a follow up JIRA HIVE-14067. thanks for the review [~thejas]! > Session timeout may happen before HIVE_SERVER2_IDLE_SESSION_TIMEOUT for > back-to-back synchronous operations. > > > Key: HIVE-13960 > URL: https://issues.apache.org/jira/browse/HIVE-13960 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: zhihai xu >Assignee: zhihai xu > Fix For: 2.2.0 > > Attachments: HIVE-13960.000.patch > > > Session timeout may happen before > HIVE_SERVER2_IDLE_SESSION_TIMEOUT(hive.server2.idle.session.timeout) for > back-to-back synchronous operations. > This issue can happen with the following two operations op1 and op2: op2 is a > synchronous long running operation, op2 is running right after op1 is closed. > > 1. closeOperation(op1) is called: > this will set {{lastIdleTime}} with value System.currentTimeMillis() because > {{opHandleSet}} becomes empty after {{closeOperation}} remove op1 from > {{opHandleSet}}. > 2. op2 is running for long time by calling {{executeStatement}} right after > closeOperation(op1) is called. > If op2 is running for more than HIVE_SERVER2_IDLE_SESSION_TIMEOUT, then the > session will timeout even when op2 is still running. > We hit this issue when we use PyHive to execute non-async operation > The following is the exception we see: > {code} > File "/usr/local/lib/python2.7/dist-packages/pyhive/hive.py", line 126, in > close > _check_status(response) > File "/usr/local/lib/python2.7/dist-packages/pyhive/hive.py", line 362, in > _check_status > raise OperationalError(response) > OperationalError: TCloseSessionResp(status=TStatus(errorCode=0, > errorMessage='Session does not exist!', sqlState=None, > infoMessages=['*org.apache.hive.service.cli.HiveSQLException:Session does not > exist!:12:11', > 'org.apache.hive.service.cli.session.SessionManager:closeSession:SessionManager.java:311', > 'org.apache.hive.service.cli.CLIService:closeSession:CLIService.java:221', > 'org.apache.hive.service.cli.thrift.ThriftCLIService:CloseSession:ThriftCLIService.java:471', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1273', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1258', > 'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39', > 'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39', > 'org.apache.hive.service.auth.TSetIpAddressProcessor:process:TSetIpAddressProcessor.java:56', > > 'org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:285', > > 'java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1145', > > 'java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:615', > 'java.lang.Thread:run:Thread.java:745'], statusCode=3)) > TCloseSessionResp(status=TStatus(errorCode=0, errorMessage='Session does not > exist!', sqlState=None, > infoMessages=['*org.apache.hive.service.cli.HiveSQLException:Session does not > exist!:12:11', > 'org.apache.hive.service.cli.session.SessionManager:closeSession:SessionManager.java:311', > 'org.apache.hive.service.cli.CLIService:closeSession:CLIService.java:221', > 'org.apache.hive.service.cli.thrift.ThriftCLIService:CloseSession:ThriftCLIService.java:471', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1273', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1258', > 'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39', > 'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39', > 'org.apache.hive.service.auth.TSetIpAddressProcessor:process:TSetIpAddressProcessor.java:56', > > 'org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:285', > > 'java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1145', > > 'java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:615', > 'java.lang.Thread:run:Thread.java:745'], statusCode=3)) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13960) Session timeout may happen before HIVE_SERVER2_IDLE_SESSION_TIMEOUT for back-to-back synchronous operations.
[ https://issues.apache.org/jira/browse/HIVE-13960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340617#comment-15340617 ] Thejas M Nair commented on HIVE-13960: -- Thanks for the patch [~zxu] and [~jxiang] >From the variable name pendingCount, its hard to understand what it >represents. Should we name it activeCalls instead ? If you agree, the change can be done in a follow up jira. > Session timeout may happen before HIVE_SERVER2_IDLE_SESSION_TIMEOUT for > back-to-back synchronous operations. > > > Key: HIVE-13960 > URL: https://issues.apache.org/jira/browse/HIVE-13960 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: zhihai xu >Assignee: zhihai xu > Fix For: 2.2.0 > > Attachments: HIVE-13960.000.patch > > > Session timeout may happen before > HIVE_SERVER2_IDLE_SESSION_TIMEOUT(hive.server2.idle.session.timeout) for > back-to-back synchronous operations. > This issue can happen with the following two operations op1 and op2: op2 is a > synchronous long running operation, op2 is running right after op1 is closed. > > 1. closeOperation(op1) is called: > this will set {{lastIdleTime}} with value System.currentTimeMillis() because > {{opHandleSet}} becomes empty after {{closeOperation}} remove op1 from > {{opHandleSet}}. > 2. op2 is running for long time by calling {{executeStatement}} right after > closeOperation(op1) is called. > If op2 is running for more than HIVE_SERVER2_IDLE_SESSION_TIMEOUT, then the > session will timeout even when op2 is still running. > We hit this issue when we use PyHive to execute non-async operation > The following is the exception we see: > {code} > File "/usr/local/lib/python2.7/dist-packages/pyhive/hive.py", line 126, in > close > _check_status(response) > File "/usr/local/lib/python2.7/dist-packages/pyhive/hive.py", line 362, in > _check_status > raise OperationalError(response) > OperationalError: TCloseSessionResp(status=TStatus(errorCode=0, > errorMessage='Session does not exist!', sqlState=None, > infoMessages=['*org.apache.hive.service.cli.HiveSQLException:Session does not > exist!:12:11', > 'org.apache.hive.service.cli.session.SessionManager:closeSession:SessionManager.java:311', > 'org.apache.hive.service.cli.CLIService:closeSession:CLIService.java:221', > 'org.apache.hive.service.cli.thrift.ThriftCLIService:CloseSession:ThriftCLIService.java:471', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1273', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1258', > 'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39', > 'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39', > 'org.apache.hive.service.auth.TSetIpAddressProcessor:process:TSetIpAddressProcessor.java:56', > > 'org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:285', > > 'java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1145', > > 'java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:615', > 'java.lang.Thread:run:Thread.java:745'], statusCode=3)) > TCloseSessionResp(status=TStatus(errorCode=0, errorMessage='Session does not > exist!', sqlState=None, > infoMessages=['*org.apache.hive.service.cli.HiveSQLException:Session does not > exist!:12:11', > 'org.apache.hive.service.cli.session.SessionManager:closeSession:SessionManager.java:311', > 'org.apache.hive.service.cli.CLIService:closeSession:CLIService.java:221', > 'org.apache.hive.service.cli.thrift.ThriftCLIService:CloseSession:ThriftCLIService.java:471', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1273', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1258', > 'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39', > 'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39', > 'org.apache.hive.service.auth.TSetIpAddressProcessor:process:TSetIpAddressProcessor.java:56', > > 'org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:285', > > 'java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1145', > > 'java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:615', > 'java.lang.Thread:run:Thread.java:745'], statusCode=3)) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13960) Session timeout may happen before HIVE_SERVER2_IDLE_SESSION_TIMEOUT for back-to-back synchronous operations.
[ https://issues.apache.org/jira/browse/HIVE-13960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325835#comment-15325835 ] Hive QA commented on HIVE-13960: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12808547/HIVE-13960.000.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10223 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_globallimit org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_mult_tables org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_constprog_partitioner org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3 {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/85/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/85/console Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-85/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 7 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12808547 - PreCommit-HIVE-MASTER-Build > Session timeout may happen before HIVE_SERVER2_IDLE_SESSION_TIMEOUT for > back-to-back synchronous operations. > > > Key: HIVE-13960 > URL: https://issues.apache.org/jira/browse/HIVE-13960 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: zhihai xu >Assignee: zhihai xu > Attachments: HIVE-13960.000.patch > > > Session timeout may happen before > HIVE_SERVER2_IDLE_SESSION_TIMEOUT(hive.server2.idle.session.timeout) for > back-to-back synchronous operations. > This issue can happen with the following two operations op1 and op2: op2 is a > synchronous long running operation, op2 is running right after op1 is closed. > > 1. closeOperation(op1) is called: > this will set {{lastIdleTime}} with value System.currentTimeMillis() because > {{opHandleSet}} becomes empty after {{closeOperation}} remove op1 from > {{opHandleSet}}. > 2. op2 is running for long time by calling {{executeStatement}} right after > closeOperation(op1) is called. > If op2 is running for more than HIVE_SERVER2_IDLE_SESSION_TIMEOUT, then the > session will timeout even when op2 is still running. > We hit this issue when we use PyHive to execute non-async operation > The following is the exception we see: > {code} > File "/usr/local/lib/python2.7/dist-packages/pyhive/hive.py", line 126, in > close > _check_status(response) > File "/usr/local/lib/python2.7/dist-packages/pyhive/hive.py", line 362, in > _check_status > raise OperationalError(response) > OperationalError: TCloseSessionResp(status=TStatus(errorCode=0, > errorMessage='Session does not exist!', sqlState=None, > infoMessages=['*org.apache.hive.service.cli.HiveSQLException:Session does not > exist!:12:11', > 'org.apache.hive.service.cli.session.SessionManager:closeSession:SessionManager.java:311', > 'org.apache.hive.service.cli.CLIService:closeSession:CLIService.java:221', > 'org.apache.hive.service.cli.thrift.ThriftCLIService:CloseSession:ThriftCLIService.java:471', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1273', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1258', > 'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39', > 'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39', > 'org.apache.hive.service.auth.TSetIpAddressProcessor:process:TSetIpAddressProcessor.java:56', > > 'org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:285', > > 'java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1145', > > 'java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:615', > 'java.lang.Thread:run:Thread.java:745'], statusCode=3)) > TCloseSessionResp(status=TStatus(errorCode=0, errorMessage='Session does not > exist!', sqlState=None, > infoMessages=['*org.apache.hive.service.cli.HiveSQLException:Session does not > exist!:12:11', > 'org.apache.hive.
[jira] [Commented] (HIVE-13960) Session timeout may happen before HIVE_SERVER2_IDLE_SESSION_TIMEOUT for back-to-back synchronous operations.
[ https://issues.apache.org/jira/browse/HIVE-13960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15319671#comment-15319671 ] Jimmy Xiang commented on HIVE-13960: Cool. +1. > Session timeout may happen before HIVE_SERVER2_IDLE_SESSION_TIMEOUT for > back-to-back synchronous operations. > > > Key: HIVE-13960 > URL: https://issues.apache.org/jira/browse/HIVE-13960 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: zhihai xu >Assignee: zhihai xu > Attachments: HIVE-13960.000.patch > > > Session timeout may happen before > HIVE_SERVER2_IDLE_SESSION_TIMEOUT(hive.server2.idle.session.timeout) for > back-to-back synchronous operations. > This issue can happen with the following two operations op1 and op2: op2 is a > synchronous long running operation, op2 is running right after op1 is closed. > > 1. closeOperation(op1) is called: > this will set {{lastIdleTime}} with value System.currentTimeMillis() because > {{opHandleSet}} becomes empty after {{closeOperation}} remove op1 from > {{opHandleSet}}. > 2. op2 is running for long time by calling {{executeStatement}} right after > closeOperation(op1) is called. > If op2 is running for more than HIVE_SERVER2_IDLE_SESSION_TIMEOUT, then the > session will timeout even when op2 is still running. > We hit this issue when we use PyHive to execute non-async operation > The following is the exception we see: > {code} > File "/usr/local/lib/python2.7/dist-packages/pyhive/hive.py", line 126, in > close > _check_status(response) > File "/usr/local/lib/python2.7/dist-packages/pyhive/hive.py", line 362, in > _check_status > raise OperationalError(response) > OperationalError: TCloseSessionResp(status=TStatus(errorCode=0, > errorMessage='Session does not exist!', sqlState=None, > infoMessages=['*org.apache.hive.service.cli.HiveSQLException:Session does not > exist!:12:11', > 'org.apache.hive.service.cli.session.SessionManager:closeSession:SessionManager.java:311', > 'org.apache.hive.service.cli.CLIService:closeSession:CLIService.java:221', > 'org.apache.hive.service.cli.thrift.ThriftCLIService:CloseSession:ThriftCLIService.java:471', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1273', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1258', > 'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39', > 'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39', > 'org.apache.hive.service.auth.TSetIpAddressProcessor:process:TSetIpAddressProcessor.java:56', > > 'org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:285', > > 'java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1145', > > 'java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:615', > 'java.lang.Thread:run:Thread.java:745'], statusCode=3)) > TCloseSessionResp(status=TStatus(errorCode=0, errorMessage='Session does not > exist!', sqlState=None, > infoMessages=['*org.apache.hive.service.cli.HiveSQLException:Session does not > exist!:12:11', > 'org.apache.hive.service.cli.session.SessionManager:closeSession:SessionManager.java:311', > 'org.apache.hive.service.cli.CLIService:closeSession:CLIService.java:221', > 'org.apache.hive.service.cli.thrift.ThriftCLIService:CloseSession:ThriftCLIService.java:471', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1273', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1258', > 'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39', > 'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39', > 'org.apache.hive.service.auth.TSetIpAddressProcessor:process:TSetIpAddressProcessor.java:56', > > 'org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:285', > > 'java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1145', > > 'java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:615', > 'java.lang.Thread:run:Thread.java:745'], statusCode=3)) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13960) Session timeout may happen before HIVE_SERVER2_IDLE_SESSION_TIMEOUT for back-to-back synchronous operations.
[ https://issues.apache.org/jira/browse/HIVE-13960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15319063#comment-15319063 ] zhihai xu commented on HIVE-13960: -- Thanks for the review [~jxiang]! Checking opHandleSet.isEmpty() is to protect asynchronous operations because an operation is only removed in function closeOperation {{opHandleSet.remove(opHandle);}}, but not in function release. After an asynchronous operation such as {{executeStatementAsync}} is running, the pendingCount is zero and opHandleSet is not empty in function release called from {{executeStatementAsync}}. So these two don't match in this case, if we don't check opHandleSet in release for asynchronous operation, the session may be expired when the asynchronous operation is still running (before closeOperation is called). Checking pendingCount in release is to prevent another race condition: if a long running synchronous operation is running in {{executeStatement}} and another one is trying to call {{getInfo}} to get the session information, then lastIdleTime will be changed to a non-zero value in function release called from {{getInfo}} and the session may be expired when the synchronous operation is still running. > Session timeout may happen before HIVE_SERVER2_IDLE_SESSION_TIMEOUT for > back-to-back synchronous operations. > > > Key: HIVE-13960 > URL: https://issues.apache.org/jira/browse/HIVE-13960 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: zhihai xu >Assignee: zhihai xu > Attachments: HIVE-13960.000.patch > > > Session timeout may happen before > HIVE_SERVER2_IDLE_SESSION_TIMEOUT(hive.server2.idle.session.timeout) for > back-to-back synchronous operations. > This issue can happen with the following two operations op1 and op2: op2 is a > synchronous long running operation, op2 is running right after op1 is closed. > > 1. closeOperation(op1) is called: > this will set {{lastIdleTime}} with value System.currentTimeMillis() because > {{opHandleSet}} becomes empty after {{closeOperation}} remove op1 from > {{opHandleSet}}. > 2. op2 is running for long time by calling {{executeStatement}} right after > closeOperation(op1) is called. > If op2 is running for more than HIVE_SERVER2_IDLE_SESSION_TIMEOUT, then the > session will timeout even when op2 is still running. > We hit this issue when we use PyHive to execute non-async operation > The following is the exception we see: > {code} > File "/usr/local/lib/python2.7/dist-packages/pyhive/hive.py", line 126, in > close > _check_status(response) > File "/usr/local/lib/python2.7/dist-packages/pyhive/hive.py", line 362, in > _check_status > raise OperationalError(response) > OperationalError: TCloseSessionResp(status=TStatus(errorCode=0, > errorMessage='Session does not exist!', sqlState=None, > infoMessages=['*org.apache.hive.service.cli.HiveSQLException:Session does not > exist!:12:11', > 'org.apache.hive.service.cli.session.SessionManager:closeSession:SessionManager.java:311', > 'org.apache.hive.service.cli.CLIService:closeSession:CLIService.java:221', > 'org.apache.hive.service.cli.thrift.ThriftCLIService:CloseSession:ThriftCLIService.java:471', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1273', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1258', > 'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39', > 'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39', > 'org.apache.hive.service.auth.TSetIpAddressProcessor:process:TSetIpAddressProcessor.java:56', > > 'org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:285', > > 'java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1145', > > 'java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:615', > 'java.lang.Thread:run:Thread.java:745'], statusCode=3)) > TCloseSessionResp(status=TStatus(errorCode=0, errorMessage='Session does not > exist!', sqlState=None, > infoMessages=['*org.apache.hive.service.cli.HiveSQLException:Session does not > exist!:12:11', > 'org.apache.hive.service.cli.session.SessionManager:closeSession:SessionManager.java:311', > 'org.apache.hive.service.cli.CLIService:closeSession:CLIService.java:221', > 'org.apache.hive.service.cli.thrift.ThriftCLIService:CloseSession:ThriftCLIService.java:471', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1273', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1258', > 'org.apache.thrift.ProcessFuncti
[jira] [Commented] (HIVE-13960) Session timeout may happen before HIVE_SERVER2_IDLE_SESSION_TIMEOUT for back-to-back synchronous operations.
[ https://issues.apache.org/jira/browse/HIVE-13960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318851#comment-15318851 ] Jimmy Xiang commented on HIVE-13960: Good to set lastIdleTime to 0 at acquire(). Why do we need to check both opHandleSet.isEmpty() and pendingCount = 0? When will these two don't match? > Session timeout may happen before HIVE_SERVER2_IDLE_SESSION_TIMEOUT for > back-to-back synchronous operations. > > > Key: HIVE-13960 > URL: https://issues.apache.org/jira/browse/HIVE-13960 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: zhihai xu >Assignee: zhihai xu > Attachments: HIVE-13960.000.patch > > > Session timeout may happen before > HIVE_SERVER2_IDLE_SESSION_TIMEOUT(hive.server2.idle.session.timeout) for > back-to-back synchronous operations. > This issue can happen with the following two operations op1 and op2: op2 is a > synchronous long running operation, op2 is running right after op1 is closed. > > 1. closeOperation(op1) is called: > this will set {{lastIdleTime}} with value System.currentTimeMillis() because > {{opHandleSet}} becomes empty after {{closeOperation}} remove op1 from > {{opHandleSet}}. > 2. op2 is running for long time by calling {{executeStatement}} right after > closeOperation(op1) is called. > If op2 is running for more than HIVE_SERVER2_IDLE_SESSION_TIMEOUT, then the > session will timeout even when op2 is still running. > We hit this issue when we use PyHive to execute non-async operation > The following is the exception we see: > {code} > File "/usr/local/lib/python2.7/dist-packages/pyhive/hive.py", line 126, in > close > _check_status(response) > File "/usr/local/lib/python2.7/dist-packages/pyhive/hive.py", line 362, in > _check_status > raise OperationalError(response) > OperationalError: TCloseSessionResp(status=TStatus(errorCode=0, > errorMessage='Session does not exist!', sqlState=None, > infoMessages=['*org.apache.hive.service.cli.HiveSQLException:Session does not > exist!:12:11', > 'org.apache.hive.service.cli.session.SessionManager:closeSession:SessionManager.java:311', > 'org.apache.hive.service.cli.CLIService:closeSession:CLIService.java:221', > 'org.apache.hive.service.cli.thrift.ThriftCLIService:CloseSession:ThriftCLIService.java:471', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1273', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1258', > 'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39', > 'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39', > 'org.apache.hive.service.auth.TSetIpAddressProcessor:process:TSetIpAddressProcessor.java:56', > > 'org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:285', > > 'java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1145', > > 'java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:615', > 'java.lang.Thread:run:Thread.java:745'], statusCode=3)) > TCloseSessionResp(status=TStatus(errorCode=0, errorMessage='Session does not > exist!', sqlState=None, > infoMessages=['*org.apache.hive.service.cli.HiveSQLException:Session does not > exist!:12:11', > 'org.apache.hive.service.cli.session.SessionManager:closeSession:SessionManager.java:311', > 'org.apache.hive.service.cli.CLIService:closeSession:CLIService.java:221', > 'org.apache.hive.service.cli.thrift.ThriftCLIService:CloseSession:ThriftCLIService.java:471', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1273', > > 'org.apache.hive.service.cli.thrift.TCLIService$Processor$CloseSession:getResult:TCLIService.java:1258', > 'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39', > 'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39', > 'org.apache.hive.service.auth.TSetIpAddressProcessor:process:TSetIpAddressProcessor.java:56', > > 'org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:285', > > 'java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1145', > > 'java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:615', > 'java.lang.Thread:run:Thread.java:745'], statusCode=3)) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13960) Session timeout may happen before HIVE_SERVER2_IDLE_SESSION_TIMEOUT for back-to-back synchronous operations.
[ https://issues.apache.org/jira/browse/HIVE-13960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317774#comment-15317774 ] zhihai xu commented on HIVE-13960: -- I attached a patch HIVE-13960.000.patch which moved {{lastIdleTime = 0;}} from {{release}} to {{acquire}} and also add a variable {{pendingCount}} to make sure only the last {{release}} can change {{lastIdleTime}}. > Session timeout may happen before HIVE_SERVER2_IDLE_SESSION_TIMEOUT for > back-to-back synchronous operations. > > > Key: HIVE-13960 > URL: https://issues.apache.org/jira/browse/HIVE-13960 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: zhihai xu >Assignee: zhihai xu > Attachments: HIVE-13960.000.patch > > > Session timeout may happen before > HIVE_SERVER2_IDLE_SESSION_TIMEOUT(hive.server2.idle.session.timeout) for > back-to-back synchronous operations. > This issue can happen with the following two operations op1 and op2: op2 is a > synchronous long running operation, op2 is running right after op1 is closed. > > 1. closeOperation(op1) is called: > this will set {{lastIdleTime}} with value System.currentTimeMillis() because > {{opHandleSet}} becomes empty after {{closeOperation}} remove op1 from > {{opHandleSet}}. > 2. op2 is running for long time by calling {{executeStatement}} right after > closeOperation(op1) is called. > If op2 is running for more than HIVE_SERVER2_IDLE_SESSION_TIMEOUT, then the > session will timeout even when op2 is still running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)