[
https://issues.apache.org/jira/browse/DRILL-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351280#comment-15351280
]
Arina Ielchiieva commented on DRILL-4177:
-----------------------------------------
Adding configuration properties ?useCursorFetch=true&defaultFetchSize=10000 [1]
helps to solve OOM
Here is some guidelines:
1. Before setting up storage plugin for MySql user needs to add
mysql-connector-java-XXX-bin.jar to <Drill installation
directory>/jars/3rdparty/. Please mind cursor fetch configuration works from
5.0.0 version [2].
2. MySql version should be > 5.0.2.
3. Add properties to storage plugin configuration:
{
"type": "jdbc",
"driver": "com.mysql.jdbc.Driver",
"url": "jdbc:mysql://host:port?useCursorFetch=true&defaultFetchSize=10000",
"username": "XXX",
"password": "XXX",
"enabled": true
}
Fetch size can be adjusted.
[1]
https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-configuration-properties.html
[2] https://dev.mysql.com/downloads/connector/j/
> select * from table;Node ran out of Heap memory,
> exiting.java.lang.OutOfMemoryError: GC overhead limit exceeded
> ---------------------------------------------------------------------------------------------------------------
>
> Key: DRILL-4177
> URL: https://issues.apache.org/jira/browse/DRILL-4177
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
> Affects Versions: 1.3.0
> Environment: drill1.3 jdk7
> Reporter: david_hudavy
> Labels: patch
> Original Estimate: 12h
> Remaining Estimate: 12h
>
> 0: jdbc:drill:zk=local> select * from table;
> Node ran out of Heap memory, exiting.
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> at com.mysql.jdbc.MysqlIO.nextRowFast(MysqlIO.java:2149)
> at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1956)
> at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:3308)
> at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:463)
> at
> com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:3032)
> at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:2280)
> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2673)
> at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2546)
> at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2504)
> at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1370)
> at
> org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
> at
> org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
> at
> org.apache.drill.exec.store.jdbc.JdbcRecordReader.setup(JdbcRecordReader.java:177)
> at
> org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:101)
> at
> org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:128)
> at
> org.apache.drill.exec.store.jdbc.JdbcBatchCreator.getBatch(JdbcBatchCreator.java:40)
> at
> org.apache.drill.exec.store.jdbc.JdbcBatchCreator.getBatch(JdbcBatchCreator.java:33)
> at
> org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:151)
> at
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:174)
> at
> org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:131)
> at
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:174)
> at
> org.apache.drill.exec.physical.impl.ImplCreator.getRootExec(ImplCreator.java:105)
> at
> org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:79)
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:230)
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
