I think I might have a theory for the non-heap memory leak I observed. Calcite uses code generation when working with adapters. For each issued query, code will be generated at runtime and get compiled and loaded. However, the JIT compiler data and runtime class loaded info are all stored in JVM's non-heap area which rarely get gc unless they get unloaded explicitly. Therefore, it causes non-heap memory explosion as queries come in over time. I've verified the leak using jstat on both cassandra schema and reflective schema.
Could you please let me know if I am correct and how we can fix the leak? Thanks a lot. Shuyi On Thu, Aug 25, 2016 at 4:02 PM, Julian Hyde <[email protected]> wrote: > Calcite doesn’t use off-heap memory (no native code, no direct > ByteBuffers, no Unsafe). So if there are off-heap leaks, look to Calcite’s > libraries (e.g. the Cassandra driver) or to Calcite adapters’ use of those > libraries. > > Julian > > > > On Aug 25, 2016, at 3:45 PM, Shuyi Chen <[email protected]> wrote: > > > > Hi all, I was using Calcite to query data in Cassandra. I found that > there > > is memory leak in non-heap space when doing calcite queries. Has someone > > spot such issues? Below is an example code snippet I used. Thanks a lot. > > > > Statement statement = someCalciteConnection.createStatement(); > > > > ResultSet resultSet = statement.executeQuery(query); > > final JSONArray jsonArray = JSONUtil.convertResultSetIntoJSON( > resultSet); > > statement.close(); > > > > > > Shuyi > > > > -- > > "So you have to trust that the dots will somehow connect in your future." > > -- "So you have to trust that the dots will somehow connect in your future."
