It sounds a wierd hack to me.
Why would a delayed update be 10 times faster than a normal update? I
would have expected it to be a little less. It can't be the database
overhead so it must be code in MMbase. My guess is the eventsystem and
cache updates.
The example is not very realistic too. A poll will be updated by
different users, which means different cloud sessions. Every cloud
should "commit" its changes to the system before it finishes the
request. A Transaction object would have worked to speed up the
loop-code too. The setValues and node commit are delayed until the
transaction commit takes place.
And yes, a change on the StorageManager should mean an update on the
database. It is called a 'storage layer' for a reason :)
IOW, it is a quick fix for a more fundamental problem or it can be
achieved by already existing code.
Nico
Michiel Meeuwissen wrote:
I was busy with the 'poll' contribution, and figured, that for real good
performance, the updating in the database could better be 'collected'
during a certain amount of time, before actual persistance occurs.
Turns out that something like that can be quite easily achieved using the
'getprocessor/setprocessor' paradigm. I've checked in some experimental
code in the poll contribution:
http://cvs.mmbase.org/viewcvs/contributions/poll/src/org/mmbase/datatypes/processors/
These are used on the 'total_answers' field of the 'answer' builder:
http://cvs.mmbase.org/viewcvs/contributions/poll/config/builders/poll/answer.xml?view=markup
I also need a change in mmbase core though:
~/mmbase/head/src/org/mmbase/storage/implementation/database$ cvs diff
DatabaseStorageManager.java
Index: DatabaseStorageManager.java
===================================================================
RCS file:
/var/cvs/src/org/mmbase/storage/implementation/database/DatabaseStorageManager.java,v
retrieving revision 1.174
diff -r1.174 DatabaseStorageManager.java
1043c1043,1049
< changeFields.add(field);
---
Object value = node.getValues().get(key);
Object origValue = node.getOldValues().get(key);
if (value == null ? origValue == null :
value.equals(origValue)) {
// no actual change, ignore that.
} else {
changeFields.add(field);
}
In other words, because 'setValue's are still called, but the processor
simply returns the old value in stead, the storage layer must be so smart
to not actually perform an update query if nothing actually changed. I first
thought that the 'changed' flag of node would do here, but it doesn't
because that it set with any 'setValue' also if that sets it to the value
it already had. That last thing could also be changed, but it seems rather
more complicated, while DatabaseStorageManager already had the necessary
loop...
The following jsp:
<c:forEach begin="1" end="1000">
<mm:node number="731">
<mm:setfield name="total_answers">${_node.total_answers +
1}</mm:setfield>
</mm:node>
</c:forEach>
executes about 10 times faster now (using mysql 5), while having no latency
in mmbase bridge (because the 'get' procesosr ensures the correct value is
returned), and latency of only 10 seconds (the 'delay' parameter in the
builder xml), in the database.
I think these 'processors' can serve much more general use, because they
could be used on any field which would possible change very often.
Should I make a hack-proposal to do that change in DatabaseStorageManager
and add those processors to mmbase.jar itself? Perhaps even for 1.8?
Any other remarks? Should I have anticipated that there can also be nodes
of which all fields change very often (which can of course all be done by
one sql update)? Does anybody know if it is essential that an actual update
occurs if you call the 'change' method in DatabaseStorageManager?
Michiel
_______________________________________________
Developers mailing list
[email protected]
http://lists.mmbase.org/mailman/listinfo/developers
