Lance Thomas wrote:
Hello XMLBlaster folks,I'd like to thank you for contribution with XMLBlaster. It is a wonderful product, and it seems to be on its way to becoming an industrial strength messaging server. We have been testing out XMLBlaster 0.84 and we ran into the following difficulties. We were wondering if you had seen these before and if you had any suggestions: 1) Difficulties with database persistence: we are using a Postgres database to act as persistent storage. Every so often (around 1/100 times during our tests), XMLBlaster will throw the following error: java.sql.SQLException: ERROR: Cannot insert a duplicate key into unique
This is fixed.
2) Deadlock: XMLBlaster clients occasionally appear to enter into deadlock, under certain simultaneous usage patterns. Other non-deadlocked clients seem to be able to connect and continue processing. One culprit may be postgres, although I am unsure.
This should be fixed. (Please report if it should show again).
I made some investigations and found out it is related to the used JDBC driver.
3) Difficulties with large messages: some of the messages we send are 2
megabytes in size. If there are about five clients publishing messages of
this size simultaneously, meaning a total of 10 MB in motion, the server may
throw an out of memory exception, even when the JVM is set to use 256 MB of
memory. If there are any property settings or other configuration settings
that would help with this, please let us know. If XMLBlaster is not supposed
to be used with such large messages, or if we must figure in a 26x memory
factor in deployment, please let us know.
Running RAM based only we use exactly the RAM which is the size of the message.
Running persistent showed me:
1. Using the deliverd Postgres JDBC driverin in xmlBlaster/lib:
I had up to x40 memory factor used, this is temporary during JDBC operation
and there after freed.
2. I played with newer Postgres JDBC versions (latest devel, latest stable for different
JDK versions):
They behave all the same, but "only" use half the amount of memory (up to x20)
So i have updated the delivered JDBC driver in xmlBlaster/lib.
You can replace it yourself if you find a better driver.
3. Using Oracle 9i JDBC driver.
Here i found a temporar factor of x5.
Note that again after insert completion the memory consumption is down to the
expected amount.
Note: With the default setting the max . size of the cache is 2 MB, so if you send
messages bigger 1MB (until any value) only one message is kept in cache,
the others are swapped to harddisk. Every topic (key oid) has its own cache.
Those properties configure the settings (xmpBlaster.properties):
persistence/msgUnitStore/maxBytesCache= [defaults to 2 MB]
persistence/msgUnitStore/maxBytes= [The max size on harddisk, defaults to Integer.MAX_VALUE]
4) Large amounts of data in persistent storage: Suppose we publish 500 MB worth of messages into XMLBlaster with persistence flagged. XMLBlaster accepts these messages and writes them to persistent storage with no problem. However, if we stop and restart XMLBlaster, it attempts to reload the messages from persistent storage and eventually runs into memory-related errors (on our 256 MB virtual machine). Again, if there are any property settings that will help with this, please let us know.
I think this is covered by 3) Please do a cvs update build.sh delete build.sh all to get the newest. i hope this helps, Marcel
Again, we thank you and commend you for a fine product, and if you have any
ideas about these issues, please let us know.
Take care,
Lance Thomas
