Werner,
Thanks for you response. I agree re: synchronized collections, making
the methods thread-safe makes sense. We will be more than happy to test
and changes/enhancements you make.
Regards,
James.
Werner Guttmann wrote:
James,
Whilst make the collection synchronized is definitely not an option (as
explained by Steven in the same thread), making the missing methois that access
the collection(s) in a write mode is a goal.
We'd definitely appreciate your help in this context, and the more of a test
environment you have, the better for us and yourself.
Steven, shall we create a new Jira issue asking ourselves to have a look at this (making the remaining write methods thread-safe) ?
Regards
Werner
-----Ursprüngliche Nachricht-----
Von: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Im Auftrag von James
Gesendet: Mittwoch, 21. März 2007 13:50
An: [email protected]
Betreff: Re: Re: AW: AW: [castor-user] [XML] XMLClassDescriptorResolver
XMLFieldDescriptorImpl cache possible memory leak
Hi All,
I've just joined the mailing list because I saw a conversation between
Adam Chesney and the Castor development team in the archives, and
thought I would add my two cents. :)
Below is a link to the last email on this thread:
http://archive.castor.codehaus.org/user/00b501c767ea%24da8a1880%240a84010a
%40zultan
We use Castor in much the same way as Adam: 100's of thousands of Castor
generated beans, and which get loaded at runtime depends very much on
the user/runtime profile of our application. I agree with Adam that
initialising the XMLClassDescriptorResolver at startup is not really a
useful solution - we would have to maintain a very messy (and error
prone) configuration to track individual bean classes in what is a huge
application, and it would also cause unnecessarily high memory usage.
Although I can't be sure, looking at the source code for
XMLClassDescriptorResolverImpl.DescriptorCache in Castor 1.1, I think
changing the non-threadsafe collections to synchronized ones would
probably do the trick. In any case, I'm sure making a thread-safe
XMLClassDescriptorResolver implementation would not be that difficult
and we will be more than happy to contribute codes and run some highly
concurrent tests if the Castor developers agree that this is the best
way forward.
My 2 cents!
Kind regards,
James.
P.S. Below is a short investigation into the possible effects of using
HashMap in a multi-threaded environment in a non-thread-safe way.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
As Adam's colleague has done, I have isolated the HashMap race condition
(in both Java 1.4 and 1.5). The problem happens with concurrent writes
to the map, no reads are necessary to recreate the problem (though they
also have a race condition when there are concurrent writes).
Below is the source for HashMap.put:
<code>
public V put(K key, V value) {
if (key == null)
return putForNullKey(value);
int hash = hash(key.hashCode());
int i = indexFor(hash, table.length);
for (Entry<K,V> e = table[i]; e != null; e = e.next) {
Object k;
if (e.hash == hash && ((k = e.key) == key ||
key.equals(k))) {
V oldValue = e.value;
e.value = value;
e.recordAccess(this);
return oldValue;
}
}
modCount++;
addEntry(hash, key, value, i);
return null;
}
</code>
As you can see, the for loop will only complete when "e != null" - that
is to say, that the last entry in the table (Entry[]) is null. The call
to HashMap.addEntry near the end of the method calls the HashMap.resize
if the size of the array that backs the map needs to be resized. It
seems that a circular reference inside one of the bins _can_ occur and
_might_ cause a race condition. See the code for HashMap.transfer below:
<code>
void transfer(Entry[] newTable) {
Entry[] src = table;
int newCapacity = newTable.length;
for (int j = 0; j < src.length; j++) {
Entry<K,V> e = src[j];
if (e != null) {
src[j] = null;
do {
Entry<K,V> next = e.next;
int i = indexFor(e.hash, newCapacity);
e.next = newTable[i];
newTable[i] = e;
e = next;
} while (e != null);
}
}
}
</code>
I am able to recreate the problem with the code below. If the HashMap is
initialised to "maxSize", I have not seen the race condition occur,
however, if it is initalised to 1 (as in the code below), the race
condition happens relatively frequently.
<code>
/**
* HashMapTester
* Show a race condition HashMap
*/
public class HashMapTester{
/**
* @param args
*/
public static void main(String[] args) {
new HashMapTester().run();
}
public void run() {
int max = 10;
int maxSize = 100000;
List<HashMapWriter> writers = new ArrayList<HashMapWriter>(max);
Map<String,String> m = new HashMap<String,String>(1);
for(int i = 0; i < max; i++) {
HashMapWriter writer = new HashMapWriter(m,maxSize);
writers.add(writer);
writer.start();
System.out.println("Num writers created: " + (i + 1));
}
}
class HashMapWriter extends Thread{
Random r = new Random(System.currentTimeMillis());
Map<String, String> m = null;
int maxSize =-1;
@Override
public void run() {
while(true) {
if(m.size() < maxSize || m.size() == 0) {
String something = String.valueOf(m.size());
m.put(something,something);
if(m.size() % 100 == 0) {
System.out.println("Map size is now: " +
m.size());
}
}else {
String something =
String.valueOf(r.nextInt(m.size()));
m.put(something, something);
try {
Thread.sleep(r.nextInt(10) +1 );
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
public HashMapWriter(Map<String, String> m, int maxSize) {
this.m = m;
this.maxSize = maxSize;
}
}
}
</code>
---------------------------------------------------------------------
To unsubscribe from this list please visit:
http://xircles.codehaus.org/manage_email
---------------------------------------------------------------------
To unsubscribe from this list please visit:
http://xircles.codehaus.org/manage_email
---------------------------------------------------------------------
To unsubscribe from this list please visit:
http://xircles.codehaus.org/manage_email
