[
https://issues.apache.org/jira/browse/LENS-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366949#comment-14366949
]
Jaideep Dhok commented on LENS-413:
-----------------------------------
While restarting sessions, we set the parameters of restored sessions on
underlying CLI service. This is done using an execute statement operation which
is creating the file. We are not closing the execute statement operation. This
might be the root cause.
> Lens unable to restart when lot of open session is persisted
> ------------------------------------------------------------
>
> Key: LENS-413
> URL: https://issues.apache.org/jira/browse/LENS-413
> Project: Apache Lens
> Issue Type: Bug
> Reporter: Arshad Matin
> Assignee: Jaideep Dhok
>
> Scenario :
> Started lens server
> Opened around 100 session
> stopped it
> Restarted it and restart failed
> Logs
> {noformat}
> 12 Mar 2015 09:51:48,356 [main] INFO
> org.apache.lens.server.session.HiveSessionService - Request to Set param
> key:hive.server2.authentication.ldap.security.protocol value:ssl
> 12 Mar 2015 09:51:48,356 [main] INFO
> org.apache.hive.service.cli.operation.Operation - Putting temp output to
> file /tmp/dataqa/aba21db9-2ae9-47dc-aa60-358940a1ff2a87568105576796952.pipeout
> 12 Mar 2015 09:51:48,356 [main] ERROR
> org.apache.hive.service.cli.operation.Operation - Error in creating temp
> output file
> java.io.FileNotFoundException:
> /tmp/dataqa/aba21db9-2ae9-47dc-aa60-358940a1ff2a87568105576796952.pipeout
> (Too many open files)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:212)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:160)
> at
> org.apache.hive.service.cli.operation.HiveCommandOperation.setupSessionIO(HiveCommandOperation.java:73)
> at
> org.apache.hive.service.cli.operation.HiveCommandOperation.<init>(HiveCommandOperation.java:65)
> at
> org.apache.hive.service.cli.operation.ExecuteStatementOperation.newExecuteStatementOperation(ExecuteStatementOperation.java:58)
> at
> org.apache.hive.service.cli.operation.OperationManager.newExecuteStatementOperation(OperationManager.java:76)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:292)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(HiveSessionImpl.java:277)
> at
> org.apache.hive.service.cli.CLIService.executeStatement(CLIService.java:238)
> at
> org.apache.lens.server.session.HiveSessionService.setSessionParameter(HiveSessionService.java:304)
> at
> org.apache.lens.server.session.HiveSessionService.start(HiveSessionService.java:369)
> at
> org.apache.hive.service.CompositeService.start(CompositeService.java:70)
> at org.apache.lens.server.LensServices.start(LensServices.java:235)
> at org.apache.lens.server.LensServer.startServices(LensServer.java:138)
> at org.apache.lens.server.LensServer.<init>(LensServer.java:91)
> at
> org.apache.lens.server.LensServer.createLensServer(LensServer.java:79)
> at org.apache.lens.server.LensServer.main(LensServer.java:190)
> 12 Mar 2015 09:51:48,357 [main] INFO
> org.apache.lens.server.session.HiveSessionService - Set param
> key:hive.server2.authentication.ldap.security.protocol value:ssl
> 12 Mar 2015 09:51:48,357 [main] INFO
> org.apache.lens.server.session.HiveSessionService - Request to Set param
> key:lens.session.loggedin.user value:test
> 12 Mar 2015 09:51:48,357 [main] INFO
> org.apache.hive.service.cli.operation.Operation - Putting temp output to
> file /tmp/dataqa/aba21db9-2ae9-47dc-aa60-358940a1ff2a87568105576796952.pipeout
> 12 Mar 2015 09:51:48,357 [main] ERROR
> org.apache.hive.service.cli.operation.Operation - Error in creating temp
> output file
> java.io.FileNotFoundException:
> /tmp/dataqa/aba21db9-2ae9-47dc-aa60-358940a1ff2a87568105576796952.pipeout
> (Too many open files)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:212)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:160)
> at
> org.apache.hive.service.cli.operation.HiveCommandOperation.setupSessionIO(HiveCommandOperation.java:73)
> at
> org.apache.hive.service.cli.operation.HiveCommandOperation.<init>(HiveCommandOperation.java:65)
> at
> org.apache.hive.service.cli.operation.ExecuteStatementOperation.newExecuteStatementOperation(ExecuteStatementOperation.java:58)
> at
> org.apache.hive.service.cli.operation.OperationManager.newExecuteStatementOperation(OperationManager.java:76)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:292)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(HiveSessionImpl.java:277)
> at
> org.apache.hive.service.cli.CLIService.executeStatement(CLIService.java:238)
> at
> org.apache.lens.server.session.HiveSessionService.setSessionParameter(HiveSessionService.java:304)
> at
> org.apache.lens.server.session.HiveSessionService.start(HiveSessionService.java:369)
> at
> org.apache.hive.service.CompositeService.start(CompositeService.java:70)
> at org.apache.lens.server.LensServices.start(LensServices.java:235)
> at org.apache.lens.server.LensServer.startServices(LensServer.java:138)
> at org.apache.lens.server.LensServer.<init>(LensServer.java:91)
> at
> org.apache.lens.server.LensServer.createLensServer(LensServer.java:79)
> at org.apache.lens.server.LensServer.main(LensServer.java:190)
> 12 Mar 2015 09:51:48,357 [main] INFO
> org.apache.lens.server.session.HiveSessionService - Set param
> key:lens.session.loggedin.user value:test
> 12 Mar 2015 09:51:48,358 [main] INFO
> org.apache.lens.server.session.HiveSessionService - Restored session
> aba21db9-2ae9-47dc-aa60-358940a1ff2a
> 12 Mar 2015 09:51:48,366 [main] ERROR
> org.apache.hive.service.CompositeService - Error starting services
> lens_services
> java.lang.RuntimeException: org.apache.lens.api.LensException: Error
> restoring session <?xml version="1.0" encoding="UTF-8"
> standalone="yes"?><lensSessionHandle><publicId>3f6c815b-d7c5-48d4-9671-c78fe04a05ed</publicId><secretId>c413685e-10f2-43ee-b920-6cc26801680c</secretId></lensSessionHandle>
> at
> org.apache.lens.server.session.HiveSessionService.start(HiveSessionService.java:376)
> at
> org.apache.hive.service.CompositeService.start(CompositeService.java:70)
> at org.apache.lens.server.LensServices.start(LensServices.java:235)
> at org.apache.lens.server.LensServer.startServices(LensServer.java:138)
> at org.apache.lens.server.LensServer.<init>(LensServer.java:91)
> at
> org.apache.lens.server.LensServer.createLensServer(LensServer.java:79)
> at org.apache.lens.server.LensServer.main(LensServer.java:190)
> Caused by: org.apache.lens.api.LensException: Error restoring session <?xml
> version="1.0" encoding="UTF-8"
> standalone="yes"?><lensSessionHandle><publicId>3f6c815b-d7c5-48d4-9671-c78fe04a05ed</publicId><secretId>c413685e-10f2-43ee-b920-6cc26801680c</secretId></lensSessionHandle>
> at
> org.apache.lens.server.LensService.restoreSession(LensService.java:173)
> at
> org.apache.lens.server.session.HiveSessionService.start(HiveSessionService.java:350)
> ... 6 more
> Caused by: org.apache.hive.service.cli.HiveSQLException: Cannot initilize
> session class:org.apache.lens.server.session.LensSessionImpl
> at
> org.apache.hive.service.cli.session.SessionManager.createSession(SessionManager.java:217)
> at
> org.apache.hive.service.cli.session.SessionManager.restoreSession(SessionManager.java:244)
> at
> org.apache.hive.service.cli.CLIService.restoreSession(CLIService.java:190)
> at
> org.apache.lens.server.LensService.restoreSession(LensService.java:167)
> ... 7 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.GeneratedConstructorAccessor8.newInstance(Unknown Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:534)
> at
> org.apache.hive.service.cli.session.SessionManager.createSession(SessionManager.java:207)
> ... 10 more
> Caused by: java.lang.RuntimeException: java.io.IOException: Too many open
> files
> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:333)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.<init>(HiveSessionImpl.java:113)
> at
> org.apache.lens.server.session.LensSessionImpl.<init>(LensSessionImpl.java:161)
> ... 14 more
> Caused by: java.io.IOException: Too many open files
> at java.io.UnixFileSystem.createFileExclusively(Native Method)
> at java.io.File.checkAndCreate(File.java:1833)
> at java.io.File.createTempFile0(File.java:1857)
> at java.io.File.createTempFile(File.java:1934)
> at
> org.apache.hadoop.hive.ql.session.SessionState.createTempFile(SessionState.java:435)
> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:331)
> ... 16 more
> 12 Mar 2015 09:51:48,367 [main] INFO org.apache.hive.service.AbstractService
> - Service:database-resources is stopped.
> 12 Mar 2015 09:51:48,367 [main] INFO org.apache.hive.service.AbstractService
> - Service:session is stopped.
> 12 Mar 2015 09:51:48,368 [main] INFO org.apache.lens.server.EventServiceImpl
> - Removed listener
> org.apache.lens.server.stats.store.log.LogStatisticsStore@2d9d4f83
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
