[
https://issues.apache.org/jira/browse/DERBY-6309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13753370#comment-13753370
]
Manikandan Chellappan commented on DERBY-6309:
----------------------------------------------
I am using embedded derby and using jdbc to run the import procedure. The java
program triggers the import based on cron expression, generally the second or
third import cycle fails.
The tables are much larger than the one used for recreation. The tables don't
have triggers, but have constraints and indexes.
One of the larger tables has following columns:
ID_ITM VARCHAR(14) NOT NULL,
LU_UOM_SLS VARCHAR(20) NOT NULL,
ED_CLR VARCHAR(20),
ED_SZ VARCHAR(10),
LU_STYL VARCHAR(4),
ID_STR_RT VARCHAR(5) DEFAULT '0',
QL_HT_PCKG_CNS DECIMAL(9,2) DEFAULT 0,
ID_SPR VARCHAR(20) DEFAULT '0',
NM_NMB_SRZ_ITM_MDL VARCHAR(40),
QL_UOM_WD_PCKG_CNS DECIMAL(9,2) DEFAULT 0,
TY_WST_BLK_SLS VARCHAR(20),
CY_MDL_SRZ_ITM VARCHAR(4),
QU_CPC_HLD DECIMAL(9,2) DEFAULT 0,
LU_UOM VARCHAR(2),
QW_ITM_PCK DECIMAL(9,2) DEFAULT 0,
TY_UN_DPLY VARCHAR(20),
QU_CB_PCK_ITM DECIMAL(9,2) DEFAULT 0,
QL_PCKG_CNS DECIMAL(9,2) DEFAULT 0,
DE_CLR_MF_SRZ_ITM VARCHAR(250),
TY_ITM_STK VARCHAR(20),
PE_WST_BLK_SLS DECIMAL(5,2) DEFAULT 0,
LU_UOM_PCKG_CNS_DMN VARCHAR(20),
LU_SYS_PRMRY_MS VARCHAR(20),
DE_SZ_MF_SRZ_ITM VARCHAR(250),
QU_UN_PCK_ITM DECIMAL(9,2) DEFAULT 0,
DC_UN_DPLY_ST_UP DATE,
LU_WRTY_MF_SRZ_ITM VARCHAR(20),
QW_WT_PCKG_CNS DECIMAL(9,2) DEFAULT 0,
DC_UN_DPLY_TK_DWN DATE,
DE_FBRC VARCHAR(250),
DP_UN_DPLY VARCHAR(20),
LU_CNT_SLS_WT_UN VARCHAR(20),
LU_WRTY_STR_SRZ VARCHAR(20),
LU_UOM_WT_PCKG_CNS VARCHAR(20),
TY_PKP_CT_STK_ITM VARCHAR(20),
LU_UOM_SZ_PCKG_CNS VARCHAR(20),
DE_SLH VARCHAR(250),
FL_VLD_SRZ_ITM CHAR(1),
TY_SRZ_ITM VARCHAR(40),
ID_SRZ_ITM_LB INTEGER,
CD_SRZ_CPT_TM VARCHAR(20) DEFAULT 'Sale',
FL_SRZ_CRT_EXT CHAR(1) DEFAULT '1',
FA_PRC_UN_STK_ITM DECIMAL(9,2) DEFAULT 0,
FL_DSD_AZN CHAR(1),
DI_PRD_SH_LF DECIMAL(3,0) DEFAULT 0,
DI_LF_SH DECIMAL(3,0) DEFAULT 0,
ID_BRKR INTEGER DEFAULT 0,
DC_AVLB_FR_SLS DATE,
TY_ITM_STPL_PRSH VARCHAR(20),
TY_ENV_STK_ITM VARCHAR(20),
NM_LCN_ASL VARCHAR(250),
TY_SCTY_RQ VARCHAR(20),
NM_LCN_SH VARCHAR(250),
NM_LCN_SID VARCHAR(250),
TY_MTR_HZ_STK_ITM VARCHAR(20),
QU_FCG DECIMAL(9,2) DEFAULT 0,
CP_UN_SL_LS_RCV_BS DECIMAL(7,3) DEFAULT 0,
CP_CST_NT_LS_RCV DECIMAL(7,3) DEFAULT 0,
CP_UN_SL_LND DECIMAL(7,3) DEFAULT 0,
DC_CST_EST_LS_RCV DATE,
FL_SHRK_SH_ITM CHAR(1),
FL_SWL_SH_ITM CHAR(1),
FL_RQ_UN_PRC CHAR(1),
FL_FE_RSTK CHAR(1),
TS_CRT_RCRD TIMESTAMP,
TS_MDF_RCRD TIMESTAMP
> Embedded Derby throws java.lang.OutOfMemoryError: Java heap space while
> importing data
> --------------------------------------------------------------------------------------
>
> Key: DERBY-6309
> URL: https://issues.apache.org/jira/browse/DERBY-6309
> Project: Derby
> Issue Type: Bug
> Affects Versions: 10.10.1.1
> Reporter: Manikandan Chellappan
>
> While importing large files (> 500 MB) using SYSCS_UTIL.SYSCS_IMPORT_DATA
> procedure, derby runs out of heap space. The java process which is running
> the derby has 2GB of heap space. The following exception is logged in the
> derby.log.
> ERROR 38000: The exception 'java.lang.OutOfMemoryError: Java heap space' was
> thrown while evaluating an expression.
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at
> org.apache.derby.iapi.error.StandardException.unexpectedUserException(Unknown
> Source)
> at
> org.apache.derby.impl.sql.execute.VTIResultSet.getNextRowCore(Unknown Source)
> at
> org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(Unknown
> Source)
> at
> org.apache.derby.impl.sql.execute.NormalizeResultSet.getNextRowCore(Unknown
> Source)
> at
> org.apache.derby.impl.sql.execute.NoPutResultSetImpl.getNextRowFromRowSource(Unknown
> Source)
> at org.apache.derby.impl.store.access.heap.HeapController.load(Unknown
> Source)
> at org.apache.derby.impl.store.access.heap.Heap.load(Unknown Source)
> at
> org.apache.derby.impl.store.access.RAMTransaction.loadConglomerate(Unknown
> Source)
> at
> org.apache.derby.impl.store.access.RAMTransaction.recreateAndLoadConglomerate(Unknown
> Source)
> at
> org.apache.derby.impl.sql.execute.InsertResultSet.bulkInsertCore(Unknown
> Source)
> at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown
> Source)
> at
> org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
> at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown
> Source)
> at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeLargeUpdate(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown
> Source)
> at org.apache.derby.impl.load.Import.performImport(Unknown Source)
> at org.apache.derby.impl.load.Import.importData(Unknown Source)
> at org.apache.derby.catalog.SystemProcedures.SYSCS_IMPORT_DATA(Unknown
> Source)
> at
> org.apache.derby.exe.ace3c78554x013fxcb2cx4da6x00000f7e08680.g0(Unknown
> Source)
> at sun.reflect.GeneratedMethodAccessor176.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown
> Source)
> at
> org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source)
> at
> org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
> at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown
> Source)
> at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown
> Source)
> at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown
> Source)
> at
> oracle.retail.stores.iddi.IDDIDerbyWriter.importTable(IDDIDerbyWriter.java:233)
> at
> oracle.retail.stores.domain.iddi.OfflineDBProducer$ImportThread.call(OfflineDBProducer.java:488)
> at
> oracle.retail.stores.domain.iddi.OfflineDBProducer$ImportThread.call(OfflineDBProducer.java:1)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.derby.impl.load.ImportReadData.readNextDelimitedRow(Unknown Source)
> at org.apache.derby.impl.load.ImportReadData.readNextRow(Unknown Source)
> at org.apache.derby.impl.load.ImportAbstract.next(Unknown Source)
> at
> org.apache.derby.impl.sql.execute.VTIResultSet.getNextRowCore(Unknown Source)
> at
> org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(Unknown
> Source)
> at
> org.apache.derby.impl.sql.execute.NormalizeResultSet.getNextRowCore(Unknown
> Source)
> at
> org.apache.derby.impl.sql.execute.NoPutResultSetImpl.getNextRowFromRowSource(Unknown
> Source)
> at org.apache.derby.impl.store.access.heap.HeapController.load(Unknown
> Source)
> at org.apache.derby.impl.store.access.heap.Heap.load(Unknown Source)
> at
> org.apache.derby.impl.store.access.RAMTransaction.loadConglomerate(Unknown
> Source)
> at
> org.apache.derby.impl.store.access.RAMTransaction.recreateAndLoadConglomerate(Unknown
> Source)
> at
> org.apache.derby.impl.sql.execute.InsertResultSet.bulkInsertCore(Unknown
> Source)
> at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown
> Source)
> at
> org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
> at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown
> Source)
> at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeLargeUpdate(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown
> Source)
> at org.apache.derby.impl.load.Import.performImport(Unknown Source)
> at org.apache.derby.impl.load.Import.importData(Unknown Source)
> at org.apache.derby.catalog.SystemProcedures.SYSCS_IMPORT_DATA(Unknown
> Source)
> at
> org.apache.derby.exe.ace3c78554x013fxcb2cx4da6x00000f7e08680.g0(Unknown
> Source)
> at sun.reflect.GeneratedMethodAccessor176.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown
> Source)
> at
> org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source)
> at
> org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
> at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown
> Source)
> at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown
> Source)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
