[
https://issues.apache.org/jira/browse/CALCITE-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobo Liao updated CALCITE-2950:
---------------------------------
Description:
Kylin's JDBC
Driver([https://github.com/apache/kylin/tree/master/jdbc)|https://github.com/apache/kylin/tree/master/jdbc),]
is based on Avatica. While using Kylin JDBC Driver, if repeatedly invoking
{code:java}
//code from Application code.
Driver driver = (Driver)
Class.forName("org.apache.kylin.jdbc.Driver").newInstance();
{code}
, which repeatedly invokes
{code:java}
//code from Kylin JDBC
DriverVersion version = DriverVersion.load(Driver.class,
"org-apache-kylin-jdbc.properties", "Kylin JDBC Driver", "unknown version",
"Kylin", "unknown version");
{code}
Then the native memory leak can be observed, see attached memory profile
diagram(from jeprof).
And the leaking stack trace is below.
at java.util.zip.Inflater.inflate
at java.util.zip.InflaterInputStream.read()
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at java.util.Properties$LineReader.readLine(Properties.java:435)
at java.util.Properties.load0(Properties.java:353)
at java.util.Properties.load(Properties.java:341)
at
[org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.DriverVersion.load(DriverVersion.java:104|http://org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.driverversion.load%28driverversion.java:104/])
at org.apache.kylin.jdbc.Driver.createDriverVersion(Driver.java:88)
at
[org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.UnregisteredDriver|http://org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.unregistereddriver/].<init>(UnregisteredDriver.java:56)
at org.apache.kylin.jdbc.Driver.<init>(Driver.java:70)
By inspecting the Avatica code, the InputStream was not closed after use.
{code:java}
//code from Avatica DriverVersion.load
final InputStream inStream =
driverClass.getClassLoader().getResourceAsStream(resourceName);
{code}
Though it is nonsense to calling DriverVersion.load repeatedly, but the leak
is there.
The code to reproduce the leak is at
[https://github.com/leonliao/calcite-avatica-driver-version-test]. Follow the
README to reproduce the problem.
was:
Kylin's JDBC
Driver([https://github.com/apache/kylin/tree/master/jdbc)|https://github.com/apache/kylin/tree/master/jdbc),]
is based on Avatica. While using Kylin JDBC Driver, if repeatedly invokes
DriverVersion.load, native memory leak can be observed, see attached memory
profile diagram(from jeprof). Though it is nonsense to calling
DriverVersion.load repeatedly, but the leak is there.
package org.apache.calcite.avatica.test;
import org.apache.calcite.avatica.DriverVersion;
import org.apache.calcite.avatica.remote.Driver;
/**
* This class demonstrates the native memory leak caused by DriverVersion.load,
* which has not closed the opened InputStream.
*/
public class DriverVersionTest {
public static void main(String[] args) throws Exception {
while (true) {
DriverVersion version = DriverVersion.load(Driver.class,
"org-apache-kylin-jdbc.properties", "Kylin JDBC Driver", "unknown version",
"Kylin", "unknown version"); System.out.println(version);
Thread.sleep(100l);
}
}
By using jemalloc's jeprof utility, the leak is caused by below stack trace
at java.util.zip.Inflater.inflate()
at java.util.zip.InflaterInputStream.read()
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at java.util.Properties$LineReader.readLine(Properties.java:435)
at java.util.Properties.load0(Properties.java:353)
at java.util.Properties.load(Properties.java:341)
at
[org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.DriverVersion.load(DriverVersion.java:104|http://org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.driverversion.load%28driverversion.java:104/])
at org.apache.kylin.jdbc.Driver.createDriverVersion(Driver.java:88)
at
[org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.UnregisteredDriver|http://org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.unregistereddriver/].<init>(UnregisteredDriver.java:56)
at org.apache.kylin.jdbc.Driver.<init>(Driver.java:70)
By inspecting the code, the InputStream was not closed after use.
{code:java}
final InputStream inStream =
driverClass.getClassLoader().getResourceAsStream(resourceName);
{code}
The code to reproduce the leak is at
[https://github.com/leonliao/calcite-avatica-driver-version-test]. Follow the
README to reproduce the problem.
> Avatica DriverVersion.load leaks InputStream
> --------------------------------------------
>
> Key: CALCITE-2950
> URL: https://issues.apache.org/jira/browse/CALCITE-2950
> Project: Calcite
> Issue Type: Bug
> Components: avatica
> Affects Versions: avatica-1.13.0
> Environment: CentOS release 6.9 (Final)
> Reporter: Xiaobo Liao
> Priority: Minor
> Labels: pull-request-available
> Fix For: next
>
> Attachments: stream_not_closed.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Kylin's JDBC
> Driver([https://github.com/apache/kylin/tree/master/jdbc)|https://github.com/apache/kylin/tree/master/jdbc),]
> is based on Avatica. While using Kylin JDBC Driver, if repeatedly invoking
> {code:java}
> //code from Application code.
> Driver driver = (Driver)
> Class.forName("org.apache.kylin.jdbc.Driver").newInstance();
> {code}
> , which repeatedly invokes
>
> {code:java}
> //code from Kylin JDBC
> DriverVersion version = DriverVersion.load(Driver.class,
> "org-apache-kylin-jdbc.properties", "Kylin JDBC Driver", "unknown version",
> "Kylin", "unknown version");
> {code}
> Then the native memory leak can be observed, see attached memory profile
> diagram(from jeprof).
> And the leaking stack trace is below.
> at java.util.zip.Inflater.inflate
> at java.util.zip.InflaterInputStream.read()
> at java.io.FilterInputStream.read(FilterInputStream.java:133)
> at java.io.FilterInputStream.read(FilterInputStream.java:107)
> at java.util.Properties$LineReader.readLine(Properties.java:435)
> at java.util.Properties.load0(Properties.java:353)
> at java.util.Properties.load(Properties.java:341)
> at
> [org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.DriverVersion.load(DriverVersion.java:104|http://org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.driverversion.load%28driverversion.java:104/])
> at org.apache.kylin.jdbc.Driver.createDriverVersion(Driver.java:88)
> at
> [org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.UnregisteredDriver|http://org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.unregistereddriver/].<init>(UnregisteredDriver.java:56)
> at org.apache.kylin.jdbc.Driver.<init>(Driver.java:70)
> By inspecting the Avatica code, the InputStream was not closed after use.
> {code:java}
> //code from Avatica DriverVersion.load
> final InputStream inStream =
> driverClass.getClassLoader().getResourceAsStream(resourceName);
> {code}
> Though it is nonsense to calling DriverVersion.load repeatedly, but the leak
> is there.
> The code to reproduce the leak is at
> [https://github.com/leonliao/calcite-avatica-driver-version-test]. Follow the
> README to reproduce the problem.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
