[
https://issues.apache.org/jira/browse/DERBY-6309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13744918#comment-13744918
]
Knut Anders Hatlen commented on DERBY-6309:
-------------------------------------------
I tried to reproduce the OutOfMemoryError by using a 700 MB input file that
contained the following line repeated over and over:
1,2,"abcdefghijklmnopqrstuvwxyzæøåABCDEFGHIJKLMNOPQRSTUVWXYZÆØÅ"
The data import was performed in ij by executing these statements:
connect 'jdbc:derby:db;create=true';
create table t(x int, y int, z varchar(100));
call syscs_util.syscs_import_data(null, 'T', null, null, 'db.csv', null, null,
null, 1);
Unfortunately, this did not cause an OutOfMemoryError when I ran the import
with 18 MB of Java heap space, so it seems like there's something more to the
problem.
If you could add some details about what the table looks like (columns and
types) and any indexes, triggers or constraints on the table, someone might
have better luck reproducing it.
> Embedded Derby throws java.lang.OutOfMemoryError: Java heap space while
> importing data
> --------------------------------------------------------------------------------------
>
> Key: DERBY-6309
> URL: https://issues.apache.org/jira/browse/DERBY-6309
> Project: Derby
> Issue Type: Bug
> Affects Versions: 10.10.1.1
> Reporter: Manikandan Chellappan
>
> While importing large files (> 500 MB) using SYSCS_UTIL.SYSCS_IMPORT_DATA
> procedure, derby runs out of heap space. The java process which is running
> the derby has 2GB of heap space. The following exception is logged in the
> derby.log.
> ERROR 38000: The exception 'java.lang.OutOfMemoryError: Java heap space' was
> thrown while evaluating an expression.
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at
> org.apache.derby.iapi.error.StandardException.unexpectedUserException(Unknown
> Source)
> at
> org.apache.derby.impl.sql.execute.VTIResultSet.getNextRowCore(Unknown Source)
> at
> org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(Unknown
> Source)
> at
> org.apache.derby.impl.sql.execute.NormalizeResultSet.getNextRowCore(Unknown
> Source)
> at
> org.apache.derby.impl.sql.execute.NoPutResultSetImpl.getNextRowFromRowSource(Unknown
> Source)
> at org.apache.derby.impl.store.access.heap.HeapController.load(Unknown
> Source)
> at org.apache.derby.impl.store.access.heap.Heap.load(Unknown Source)
> at
> org.apache.derby.impl.store.access.RAMTransaction.loadConglomerate(Unknown
> Source)
> at
> org.apache.derby.impl.store.access.RAMTransaction.recreateAndLoadConglomerate(Unknown
> Source)
> at
> org.apache.derby.impl.sql.execute.InsertResultSet.bulkInsertCore(Unknown
> Source)
> at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown
> Source)
> at
> org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
> at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown
> Source)
> at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeLargeUpdate(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown
> Source)
> at org.apache.derby.impl.load.Import.performImport(Unknown Source)
> at org.apache.derby.impl.load.Import.importData(Unknown Source)
> at org.apache.derby.catalog.SystemProcedures.SYSCS_IMPORT_DATA(Unknown
> Source)
> at
> org.apache.derby.exe.ace3c78554x013fxcb2cx4da6x00000f7e08680.g0(Unknown
> Source)
> at sun.reflect.GeneratedMethodAccessor176.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown
> Source)
> at
> org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source)
> at
> org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
> at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown
> Source)
> at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown
> Source)
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown
> Source)
> at
> oracle.retail.stores.iddi.IDDIDerbyWriter.importTable(IDDIDerbyWriter.java:233)
> at
> oracle.retail.stores.domain.iddi.OfflineDBProducer$ImportThread.call(OfflineDBProducer.java:488)
> at
> oracle.retail.stores.domain.iddi.OfflineDBProducer$ImportThread.call(OfflineDBProducer.java:1)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.derby.impl.load.ImportReadData.readNextDelimitedRow(Unknown Source)
> at org.apache.derby.impl.load.ImportReadData.readNextRow(Unknown Source)
> at org.apache.derby.impl.load.ImportAbstract.next(Unknown Source)
> at
> org.apache.derby.impl.sql.execute.VTIResultSet.getNextRowCore(Unknown Source)
> at
> org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(Unknown
> Source)
> at
> org.apache.derby.impl.sql.execute.NormalizeResultSet.getNextRowCore(Unknown
> Source)
> at
> org.apache.derby.impl.sql.execute.NoPutResultSetImpl.getNextRowFromRowSource(Unknown
> Source)
> at org.apache.derby.impl.store.access.heap.HeapController.load(Unknown
> Source)
> at org.apache.derby.impl.store.access.heap.Heap.load(Unknown Source)
> at
> org.apache.derby.impl.store.access.RAMTransaction.loadConglomerate(Unknown
> Source)
> at
> org.apache.derby.impl.store.access.RAMTransaction.recreateAndLoadConglomerate(Unknown
> Source)
> at
> org.apache.derby.impl.sql.execute.InsertResultSet.bulkInsertCore(Unknown
> Source)
> at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown
> Source)
> at
> org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
> at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown
> Source)
> at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeLargeUpdate(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown
> Source)
> at org.apache.derby.impl.load.Import.performImport(Unknown Source)
> at org.apache.derby.impl.load.Import.importData(Unknown Source)
> at org.apache.derby.catalog.SystemProcedures.SYSCS_IMPORT_DATA(Unknown
> Source)
> at
> org.apache.derby.exe.ace3c78554x013fxcb2cx4da6x00000f7e08680.g0(Unknown
> Source)
> at sun.reflect.GeneratedMethodAccessor176.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown
> Source)
> at
> org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source)
> at
> org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
> at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown
> Source)
> at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown
> Source)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
