[
https://issues.apache.org/jira/browse/LIVY-752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17048760#comment-17048760
]
Wing Yew Poon commented on LIVY-752:
The NPE is due to checking if a ConcurrentHashMap (connectionsCount) does not
contains a key (instead of contains it) before getting the value and
incrementing or decrementing the count:
{code:java}
private def incrementConnectionsCount(key: String): Unit = {
if (!connectionsCount.containsKey(key))
connectionsCount.get(key).incrementAndGet
else connectionsCount.put(key, new AtomicLong)
}
{code}
However, correcting the condition is not enough. There is a race in the code –
a thread can get a count, find that it's within a limit, create a new session
and then increment the count, while in the meantime, another thread could have
incremented the count and so the limit is now actually exceeded.
I have fixed this in my PR.
> Livy TS does not accept any connections when limits are set on connections
> --
>
> Key: LIVY-752
> URL: https://issues.apache.org/jira/browse/LIVY-752
> Project: Livy
> Issue Type: Bug
> Components: Thriftserver
>Affects Versions: 0.7.0
>Reporter: Wing Yew Poon
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I set livy.server.thrift.limit.connections.per.user=20 on my Livy Server.
> When I try to connect to it, I get
> {noformat}
> 2020-02-28 17:13:30,443 WARN
> org.apache.livy.thriftserver.cli.ThriftBinaryCLIService: Error opening
> session:
> java.lang.NullPointerException
> at
> org.apache.livy.thriftserver.LivyThriftSessionManager.incrementConnectionsCount(LivyThriftSessionManager.scala:438)
> at
> org.apache.livy.thriftserver.LivyThriftSessionManager.incrementConnections(LivyThriftSessionManager.scala:425)
> at
> org.apache.livy.thriftserver.LivyThriftSessionManager.openSession(LivyThriftSessionManager.scala:222)
> at
> org.apache.livy.thriftserver.LivyCLIService.openSessionWithImpersonation(LivyCLIService.scala:121)
> at
> org.apache.livy.thriftserver.cli.ThriftCLIService.getSessionHandle(ThriftCLIService.scala:324)
> at
> org.apache.livy.thriftserver.cli.ThriftCLIService.OpenSession(ThriftCLIService.scala:203)
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1497)
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1482)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
