[jira] Commented: (DERBY-3354) Select from large lob table with embedded gives OutOfMemoryError

2008-04-23 Thread Kathey Marsden (JIRA)

[ 
https://issues.apache.org/jira/browse/DERBY-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12591832#action_12591832
 ] 

Kathey Marsden commented on DERBY-3354:
---

I was thinking about whether to close this issue. Is there still a regression 
test coming?


> Select from large lob table with embedded gives OutOfMemoryError
> 
>
> Key: DERBY-3354
> URL: https://issues.apache.org/jira/browse/DERBY-3354
> Project: Derby
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 10.3.1.4, 10.3.2.1, 10.4.0.0
>Reporter: Kathey Marsden
>Assignee: Anurag Shekhar
> Fix For: 10.4.1.0, 10.5.0.0
>
> Attachments: derby-3354.diff, derby-3354_preview.diff, 
> derby-3354_v1.diff, derby-3354_v2.diff, derby-3354_v3.diff, 
> derby-3354_v4.diff, LocLeak.java
>
>
> Retrieving from a large table with lobs gives an OutOfMemoryException, even 
> if free() is explictly called on the lob.   I believe this is because 
> EmbedConnection.addLobMapping is called for every lob creation but is never 
> cleared until commit or rollback, even if the lob is freed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (DERBY-3354) Select from large lob table with embedded gives OutOfMemoryError

2008-04-04 Thread JIRA

[ 
https://issues.apache.org/jira/browse/DERBY-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585633#action_12585633
 ] 

Øystein Grøvlen commented on DERBY-3354:


It would be good if a test code be made based on the program that reproduced 
this error.

> Select from large lob table with embedded gives OutOfMemoryError
> 
>
> Key: DERBY-3354
> URL: https://issues.apache.org/jira/browse/DERBY-3354
> Project: Derby
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 10.3.1.4, 10.3.2.1, 10.4.0.0
>Reporter: Kathey Marsden
>Assignee: Anurag Shekhar
> Fix For: 10.5.0.0
>
> Attachments: derby-3354.diff, derby-3354_preview.diff, 
> derby-3354_v1.diff, derby-3354_v2.diff, derby-3354_v3.diff, 
> derby-3354_v4.diff, LocLeak.java
>
>
> Retrieving from a large table with lobs gives an OutOfMemoryException, even 
> if free() is explictly called on the lob.   I believe this is because 
> EmbedConnection.addLobMapping is called for every lob creation but is never 
> cleared until commit or rollback, even if the lob is freed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (DERBY-3354) Select from large lob table with embedded gives OutOfMemoryError

2008-04-04 Thread JIRA

[ 
https://issues.apache.org/jira/browse/DERBY-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585531#action_12585531
 ] 

Øystein Grøvlen commented on DERBY-3354:


Is the removeLOBMapping put in the right plcae?
Don't you have to do it regardless of whether it is materialized or not?
Maybe it should have been put in a finally block like for EmbedClob?  (Ref.  
Kathey's comment on my initial patch)

> Select from large lob table with embedded gives OutOfMemoryError
> 
>
> Key: DERBY-3354
> URL: https://issues.apache.org/jira/browse/DERBY-3354
> Project: Derby
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 10.3.1.4, 10.3.2.1, 10.4.0.0
>Reporter: Kathey Marsden
>Assignee: Anurag Shekhar
> Attachments: derby-3354.diff, derby-3354_preview.diff, 
> derby-3354_v1.diff, derby-3354_v2.diff, derby-3354_v3.diff, LocLeak.java
>
>
> Retrieving from a large table with lobs gives an OutOfMemoryException, even 
> if free() is explictly called on the lob.   I believe this is because 
> EmbedConnection.addLobMapping is called for every lob creation but is never 
> cleared until commit or rollback, even if the lob is freed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (DERBY-3354) Select from large lob table with embedded gives OutOfMemoryError

2008-04-04 Thread Anurag Shekhar (JIRA)

[ 
https://issues.apache.org/jira/browse/DERBY-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585483#action_12585483
 ] 

Anurag Shekhar commented on DERBY-3354:
---

Thanks Oystein for pointing out. I missed it, I will upload the patch after 
adding this.

> Select from large lob table with embedded gives OutOfMemoryError
> 
>
> Key: DERBY-3354
> URL: https://issues.apache.org/jira/browse/DERBY-3354
> Project: Derby
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 10.3.1.4, 10.3.2.1, 10.4.0.0
>Reporter: Kathey Marsden
>Assignee: Anurag Shekhar
> Attachments: derby-3354.diff, derby-3354_preview.diff, 
> derby-3354_v1.diff, derby-3354_v2.diff, LocLeak.java
>
>
> Retrieving from a large table with lobs gives an OutOfMemoryException, even 
> if free() is explictly called on the lob.   I believe this is because 
> EmbedConnection.addLobMapping is called for every lob creation but is never 
> cleared until commit or rollback, even if the lob is freed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (DERBY-3354) Select from large lob table with embedded gives OutOfMemoryError

2008-04-04 Thread JIRA

[ 
https://issues.apache.org/jira/browse/DERBY-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585477#action_12585477
 ] 

Øystein Grøvlen commented on DERBY-3354:


Thanks for the patch Anurag. Comments:

 - I guess you need to do removeLOBMapping in EmbedBlob#free too.
 - Any resaon to keep references for already freed objects around until end of 
transaction?

> Select from large lob table with embedded gives OutOfMemoryError
> 
>
> Key: DERBY-3354
> URL: https://issues.apache.org/jira/browse/DERBY-3354
> Project: Derby
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 10.3.1.4, 10.3.2.1, 10.4.0.0
>Reporter: Kathey Marsden
>Assignee: Anurag Shekhar
> Attachments: derby-3354.diff, derby-3354_preview.diff, 
> derby-3354_v1.diff, derby-3354_v2.diff, LocLeak.java
>
>
> Retrieving from a large table with lobs gives an OutOfMemoryException, even 
> if free() is explictly called on the lob.   I believe this is because 
> EmbedConnection.addLobMapping is called for every lob creation but is never 
> cleared until commit or rollback, even if the lob is freed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (DERBY-3354) Select from large lob table with embedded gives OutOfMemoryError

2008-04-04 Thread Anurag Shekhar (JIRA)

[ 
https://issues.apache.org/jira/browse/DERBY-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585440#action_12585440
 ] 

Anurag Shekhar commented on DERBY-3354:
---

Thanks Kristian for looking at the patch. 
I will make the changes you have suggested and upload the patch.
I am not planing for any more version of this patch (except for review changes.)

> Select from large lob table with embedded gives OutOfMemoryError
> 
>
> Key: DERBY-3354
> URL: https://issues.apache.org/jira/browse/DERBY-3354
> Project: Derby
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 10.3.1.4, 10.3.2.1, 10.4.0.0
>Reporter: Kathey Marsden
>Assignee: Anurag Shekhar
> Attachments: derby-3354.diff, derby-3354_preview.diff, 
> derby-3354_v1.diff, LocLeak.java
>
>
> Retrieving from a large table with lobs gives an OutOfMemoryException, even 
> if free() is explictly called on the lob.   I believe this is because 
> EmbedConnection.addLobMapping is called for every lob creation but is never 
> cleared until commit or rollback, even if the lob is freed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (DERBY-3354) Select from large lob table with embedded gives OutOfMemoryError

2008-04-03 Thread Kristian Waagan (JIRA)

[ 
https://issues.apache.org/jira/browse/DERBY-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585299#action_12585299
 ] 

Kristian Waagan commented on DERBY-3354:


I haven't validated the approach, but it seems reasonable. I applied the patch 
and compiled Derby.

Some *very minor* things you can consider changing:
 1) Typos: "refrences" -> "references" (5 occurrences)
 2) Use of a space between the method call / declaration and the following 
parenthesis is not consistent.
 3) Trailing whitespace at diff lines 28, 107 and 148.
 4) You could change the comment for the weak hash map to JavaDoc.

Nice and small change :)
Is this the final patch, or are the more coming up?

> Select from large lob table with embedded gives OutOfMemoryError
> 
>
> Key: DERBY-3354
> URL: https://issues.apache.org/jira/browse/DERBY-3354
> Project: Derby
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 10.3.1.4, 10.3.2.1, 10.4.0.0
>Reporter: Kathey Marsden
>Assignee: Anurag Shekhar
> Attachments: derby-3354.diff, derby-3354_preview.diff, 
> derby-3354_v1.diff, LocLeak.java
>
>
> Retrieving from a large table with lobs gives an OutOfMemoryException, even 
> if free() is explictly called on the lob.   I believe this is because 
> EmbedConnection.addLobMapping is called for every lob creation but is never 
> cleared until commit or rollback, even if the lob is freed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (DERBY-3354) Select from large lob table with embedded gives OutOfMemoryError

2008-04-03 Thread JIRA

[ 
https://issues.apache.org/jira/browse/DERBY-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585049#action_12585049
 ] 

Øystein Grøvlen commented on DERBY-3354:


I do think that it is a good idea to put disk operations in a finalizer, but it 
should be possible to record work that needs to be done (e.g., at transaction 
commit).   I do not think delaying clean-up of files until db reboot is 
acceptable since a server may be running for months without a reboot.



> Select from large lob table with embedded gives OutOfMemoryError
> 
>
> Key: DERBY-3354
> URL: https://issues.apache.org/jira/browse/DERBY-3354
> Project: Derby
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 10.3.1.4, 10.3.2.1, 10.4.0.0
>Reporter: Kathey Marsden
>Assignee: Anurag Shekhar
> Attachments: derby-3354.diff, derby-3354_preview.diff, LocLeak.java
>
>
> Retrieving from a large table with lobs gives an OutOfMemoryException, even 
> if free() is explictly called on the lob.   I believe this is because 
> EmbedConnection.addLobMapping is called for every lob creation but is never 
> cleared until commit or rollback, even if the lob is freed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (DERBY-3354) Select from large lob table with embedded gives OutOfMemoryError

2008-04-03 Thread Anurag Shekhar (JIRA)

[ 
https://issues.apache.org/jira/browse/DERBY-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585015#action_12585015
 ] 

Anurag Shekhar commented on DERBY-3354:
---

Thanks Øystein  for look at the patch.

My question is whether it is OK that free is not called on the LOB objects that 
are garbage-collected this way. Are you certain that all associated resources 
(e.g., temp files) will be released? Maybe some finalizer clean-up will also be 
needed?

If there is no free called (or release on internal clob) the temporary file 
will not be cleaned. But the temporary file 
system is cleaned every time db boots up. I was under impression that disk 
operations are not allowed from 
finalize. Probably I was wrong. I will check and if its not restricted I will 
add finalizer for EmbedClob and 
EmbedBlob.

Other comments/questions:
  * It seems getLocator will create a new locator on every call. I think there 
should be just a single locator per object. Otherwise, I think the clean-up 
will be difficult.

I will be checking it to check if the locator is 0 before calling addMapping. 

  * free: Willl removeLOBMapping handle correctly the cases where a locator has 
not been set?
HashMap ignore remove call if key is not found in the Map So there is no 
problem If we call 
remove with invalid key.

  * typo: Refrence 
I will fix this,

> Select from large lob table with embedded gives OutOfMemoryError
> 
>
> Key: DERBY-3354
> URL: https://issues.apache.org/jira/browse/DERBY-3354
> Project: Derby
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 10.3.1.4, 10.3.2.1, 10.4.0.0
>Reporter: Kathey Marsden
>Assignee: Anurag Shekhar
> Attachments: derby-3354.diff, derby-3354_preview.diff, LocLeak.java
>
>
> Retrieving from a large table with lobs gives an OutOfMemoryException, even 
> if free() is explictly called on the lob.   I believe this is because 
> EmbedConnection.addLobMapping is called for every lob creation but is never 
> cleared until commit or rollback, even if the lob is freed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (DERBY-3354) Select from large lob table with embedded gives OutOfMemoryError

2008-04-03 Thread JIRA

[ 
https://issues.apache.org/jira/browse/DERBY-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585005#action_12585005
 ] 

Øystein Grøvlen commented on DERBY-3354:


If I understand correctly, the idea behind using a WeakHashMap is that if this 
is the only reference, it should not prevent the referred object from being 
garbage-collected.
In other words, as long as a user thread or the network server through 
locators, refers to the object, it will not be garbage-collected, but if such 
references does not exist, the object will be garbage-collected.This is an 
interesting idea.  

My question is whether it is OK that free is not called on the LOB objects that 
are garbage-collected this way.  Are you certain that all associated resources 
(e.g., temp files) will be released?  Maybe some finalizer clean-up  will also 
be needed?

Other comments/questions:
  * It seems getLocator will create a new locator on every call.  I think there 
should be just a single locator per object.  Otherwise, I think the clean-up 
will be difficult.
  * free:  Willl removeLOBMapping handle correctly the cases where a locator 
has not been set?
  * typo: Refrence



> Select from large lob table with embedded gives OutOfMemoryError
> 
>
> Key: DERBY-3354
> URL: https://issues.apache.org/jira/browse/DERBY-3354
> Project: Derby
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 10.3.1.4, 10.3.2.1, 10.4.0.0
>Reporter: Kathey Marsden
>Assignee: Anurag Shekhar
> Attachments: derby-3354.diff, derby-3354_preview.diff, LocLeak.java
>
>
> Retrieving from a large table with lobs gives an OutOfMemoryException, even 
> if free() is explictly called on the lob.   I believe this is because 
> EmbedConnection.addLobMapping is called for every lob creation but is never 
> cleared until commit or rollback, even if the lob is freed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (DERBY-3354) Select from large lob table with embedded gives OutOfMemoryError

2008-04-01 Thread Anurag Shekhar (JIRA)

[ 
https://issues.apache.org/jira/browse/DERBY-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12584108#action_12584108
 ] 

Anurag Shekhar commented on DERBY-3354:
---

LobMapping and locater is used by the stored procedures (on behalf of 
network driver) to get hold of a particular lob object. LobMapping is 
additionally 
used to clear the lob object during commit/rollback and connection close. 

Separating these two functionality in two different collections may solve OOM 
problem.

We can continue to have LobMapping only if the LOB is created through client 
driver (entry will 
be made in LOBMapiing in getLocator)

Create methods will make an entry to a new collection holding week references 
of the LOBs. 

This will ensure that LOB will get garbage collected if they unreferenced in 
Embedded 
mode but in case of client server mode the references will be protected from 
being garbage 
collected unless free is called by the stored procedure. 

> Select from large lob table with embedded gives OutOfMemoryError
> 
>
> Key: DERBY-3354
> URL: https://issues.apache.org/jira/browse/DERBY-3354
> Project: Derby
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 10.3.1.4, 10.3.2.1, 10.4.0.0
>Reporter: Kathey Marsden
>Assignee: Anurag Shekhar
> Attachments: derby-3354.diff, LocLeak.java
>
>
> Retrieving from a large table with lobs gives an OutOfMemoryException, even 
> if free() is explictly called on the lob.   I believe this is because 
> EmbedConnection.addLobMapping is called for every lob creation but is never 
> cleared until commit or rollback, even if the lob is freed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (DERBY-3354) Select from large lob table with embedded gives OutOfMemoryError

2008-02-15 Thread JIRA

[ 
https://issues.apache.org/jira/browse/DERBY-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12569268#action_12569268
 ] 

Øystein Grøvlen commented on DERBY-3354:


Kathey, I agreed that more needs to be done for a final patch.  I just put 
together a patch with the core fix to make it possible to try out the fix.

To your issues:
1) Creating a finally block in EmbedBlob seems like a good idea.  I also agree 
that we can remove similar code from the stored procedures.

2) I will think about solution for jdk15. The problem introduced by 10.3 is 
that the lob mapping keeps Blob/Clob objects alive that earlier would be gc'ed 
when it was no longer referred by the user.  I guess another level of 
indirection would make it possible to use finalizers to clean up, but I am not 
sure that is a good idea.
A work-around is to not create Blob objects in the first place, but use 
ResultSet#getBytes etc instead.  So the problem is limited to the case where a 
Blob object need to be accessible after a call to ResultSet#next, but not until 
commit.

3) Seems like there is come concurrent access to the HashMap.  I would guess 
that this is because the iterator created by rollback, detects that the 
underlying collection has been changed.  Probably through the calls to free in 
the loop using the iterator.  I think this show that something needs to be 
reorganized a bit here.


> Select from large lob table with embedded gives OutOfMemoryError
> 
>
> Key: DERBY-3354
> URL: https://issues.apache.org/jira/browse/DERBY-3354
> Project: Derby
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 10.3.1.4, 10.3.2.1, 10.4.0.0
>Reporter: Kathey Marsden
> Attachments: derby-3354.diff, LocLeak.java
>
>
> Retrieving from a large table with lobs gives an OutOfMemoryException, even 
> if free() is explictly called on the lob.   I believe this is because 
> EmbedConnection.addLobMapping is called for every lob creation but is never 
> cleared until commit or rollback, even if the lob is freed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (DERBY-3354) Select from large lob table with embedded gives OutOfMemoryError

2008-02-12 Thread Kathey Marsden (JIRA)

[ 
https://issues.apache.org/jira/browse/DERBY-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12568226#action_12568226
 ] 

Kathey Marsden commented on DERBY-3354:
---

Also should the removeLobMapping calls be removed from 
LobStoredProcedures.BLOBRELEASELOCATOR and CLOBRELEASELOCATOR now that the 
release happens in free()?



> Select from large lob table with embedded gives OutOfMemoryError
> 
>
> Key: DERBY-3354
> URL: https://issues.apache.org/jira/browse/DERBY-3354
> Project: Derby
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 10.3.1.4, 10.3.2.1, 10.4.0.0
>Reporter: Kathey Marsden
> Attachments: derby-3354.diff, LocLeak.java
>
>
> Retrieving from a large table with lobs gives an OutOfMemoryException, even 
> if free() is explictly called on the lob.   I believe this is because 
> EmbedConnection.addLobMapping is called for every lob creation but is never 
> cleared until commit or rollback, even if the lob is freed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (DERBY-3354) Select from large lob table with embedded gives OutOfMemoryError

2008-02-12 Thread Kathey Marsden (JIRA)

[ 
https://issues.apache.org/jira/browse/DERBY-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12568177#action_12568177
 ] 

Kathey Marsden commented on DERBY-3354:
---

Thanks for the patch.  I tried it out and it and have a few comments.

1) In EmbedBlob should the call to  localConn.removeLOBMapping(locator); be in 
a finally block like it is in EmbedClob?

2) In jdk15 we don't have free()  so would still have a leak. Do you have any 
suggestions for a jdk15 solution?

3) I removed the call to free() from the repro and ran 
java LocLeak and got.
Retrieving row 99000
Exception in thread "main" java.sql.SQLException: Java exception: ': 
java.util.ConcurrentModificationException'.
at 
org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:95)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Util.java:88)
at org.apache.derby.impl.jdbc.Util.javaException(Util.java:245)
at 
org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(TransactionResourceImpl.java:403)
at 
org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(TransactionResourceImpl.java:346)
at 
org.apache.derby.impl.jdbc.EmbedConnection.handleException(EmbedConnection.java:1946)
at 
org.apache.derby.impl.jdbc.EmbedConnection.rollback(EmbedConnection.java:1521)
at LocLeak.main(LocLeak.java:14)
Caused by: java.sql.SQLException: Java exception: ': 
java.util.ConcurrentModificationException'.
at 
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(SQLExceptionFactory.java:45)
at 
org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(SQLExceptionFactory40.java:13
5)
at 
org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:70)
... 7 more
Caused by: java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$ValueIterator.next(HashMap.java:821)
at 
org.apache.derby.impl.jdbc.EmbedConnection.clearLOBMapping(EmbedConnection.java:2737)
at 
org.apache.derby.impl.jdbc.EmbedConnection.rollback(EmbedConnection.java:1519)
... 1 more

> Select from large lob table with embedded gives OutOfMemoryError
> 
>
> Key: DERBY-3354
> URL: https://issues.apache.org/jira/browse/DERBY-3354
> Project: Derby
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 10.3.1.4, 10.3.2.1, 10.4.0.0
>Reporter: Kathey Marsden
> Attachments: derby-3354.diff, LocLeak.java
>
>
> Retrieving from a large table with lobs gives an OutOfMemoryException, even 
> if free() is explictly called on the lob.   I believe this is because 
> EmbedConnection.addLobMapping is called for every lob creation but is never 
> cleared until commit or rollback, even if the lob is freed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (DERBY-3354) Select from large lob table with embedded gives OutOfMemoryError

2008-01-29 Thread Knut Anders Hatlen (JIRA)

[ 
https://issues.apache.org/jira/browse/DERBY-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12563539#action_12563539
 ] 

Knut Anders Hatlen commented on DERBY-3354:
---

> I believe this is because EmbedConnection.addLobMapping is called for every 
> lob creation but is never cleared until commit or rollback, even if the lob 
> is freed.

I believe your analysis is correct. free() should remove the mapping. 
Unfortunately, the lob doesn't know about its locator value, but it seems like 
your about to change that by adding a getLocator() method in DERBY-3243.

On DERBY-3243 I wrote:
> Another thing I came to think about: Would it be better to remove 
> addLOBMapping() from the constructors in EmbedBlob/EmbedClob and instead call 
> it lazily from getLocator()? (...) Then we'd also remove the overhead of 
> maintaining the lob mapping in embedded mode.

I think that change would fix the OOME (on embedded, but we'd probably still 
see it in a client/server environment).

Now, it seems like DERBY-2787 added the call to addLOBMapping() intentionally 
to ensure that free() was called on commit/rollback and temporary files would 
be deleted. Moving addLOBMapping() out of the constructors would probably 
reintroduce the problem with temporary files not being deleted. However, the 
lobs only create temporary files when they are modified and the new size of the 
lob is greater than the buffer in LOBStreamControl, so it shouldn't be 
necessary to have commit/rollback call free() on all lobs to delete the 
temporary files. It would probably be enough if each lob added itself to a list 
in the connection each time a temporary file was created in 
LOBStreamControl.init(). (addLOBMapping() is probably OK, but it feels a bit 
strange to piggyback on a locator mechanism to do this cleanup when we don't 
actually care about the locators.)

> Select from large lob table with embedded gives OutOfMemoryError
> 
>
> Key: DERBY-3354
> URL: https://issues.apache.org/jira/browse/DERBY-3354
> Project: Derby
>  Issue Type: Bug
>  Components: SQL
>Reporter: Kathey Marsden
> Attachments: LocLeak.java
>
>
> Retrieving from a large table with lobs gives an OutOfMemoryException, even 
> if free() is explictly called on the lob.   I believe this is because 
> EmbedConnection.addLobMapping is called for every lob creation but is never 
> cleared until commit or rollback, even if the lob is freed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Commented: (DERBY-3354) Select from large lob table with embedded gives OutOfMemoryError

2008-01-28 Thread Kathey Marsden (JIRA)

[ 
https://issues.apache.org/jira/browse/DERBY-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12563275#action_12563275
 ] 

Kathey Marsden commented on DERBY-3354:
---

This regression was caused by the following checkin:
r546881 | bernt | 2007-06-13 07:06:30 -0700 (Wed, 13 Jun 2007) | 1 line

DERBY-2787 make entry for clob in connection so that temporary file is removed 
when a connectionn is commited/rolledback. Submitted by Anurag Shekhar

> Select from large lob table with embedded gives OutOfMemoryError
> 
>
> Key: DERBY-3354
> URL: https://issues.apache.org/jira/browse/DERBY-3354
> Project: Derby
>  Issue Type: Bug
>  Components: SQL
>Reporter: Kathey Marsden
> Attachments: LocLeak.java
>
>
> Retrieving from a large table with lobs gives an OutOfMemoryException, even 
> if free() is explictly called on the lob.   I believe this is because 
> EmbedConnection.addLobMapping is called for every lob creation but is never 
> cleared until commit or rollback, even if the lob is freed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.