The classfiles are downloadable on
http://www.vpro.nl/MMObjectNode.java and
http://www.vpro.nl/MMObjectBuilder.java
Seems that attachments on my side are discarded
-------------------------------------------------------------
Hi,
This bug is mentioned on butracker
MMBase has a performance-issue regarding MMObjectNode::getRelated() and
MMObjectNode::getRelated(type) where type is a builder.
Imagine a person who has 20.000 relations with userratings and 1 relation with an
image. If I where to ask mmbase to get the person.getRelated(Images), the following
will happen:
1.) the builder of type 'type' (image) will be fetched from mmbase
2.) mmbase gets *all* related nodes (20.001 in this example)
3.) it will check the otype of the node against the otype of the builder from step 1
4.) returns 1 image
There are 3 major problems with this:
- if you use inherited builders this current method will not work. If I ask
getRelated(Mediaparts)
the current implementation will return Mediaparts instead of the correct
Audio/Videoparts
- the nodecache will be filled with useless information (it will now contain the
20.001 nodes
from the getRelated in the previous example; the current nodecache is only 16.000
orso,
so a getRelated(type) on a node with many relations will invalidate the whole
cache and is worthless
at this point
- to determine the otype of the node, the relationnumber is being queried on the
object table field otype,
after that the builder for that otype will do a 'SELECT * from buildername' to
fetch the real values
for the node.. This will result in 40.002 queries on the database for just one (1)
image!
My current proposed sollution for getRelated(type) works as follows:
- query with a multilevel from this_node.parent -> type, this will return virtual
nodes (contain otype and number)
- check wether the node is in the nodecache, if so, return that, otherwise remember
the number
- query the builder in 1 search the uncached nodes, fetch those and fill in the real
values (this will put the
nodes automatically into the cache
- returns the nodes
This solves the inheritance-problem, queries the remaining nodes in 1 extra query and
fill the nodecache.
To get the previsous example, the related image-node, if not already in cache, will be
retreived in 2 queries
instead of 40.002..
The code is backwards-compatible. The only case in which it is not, are inherited
builders, which only Rob is
currently using.
There is one point of discussion, which is the Multilevelcache to speed things up a
bit more.If i have time,
I will try to incorporate that one too.
I have included the code-snip and the proposed sollution.You can switch with a boolean
between the old
and new implementation to be sure that if, for a reason I cannot image, it should not
work, can be activated
or deactivated. Look for the static boolean 'NEW_RELATED_TEST'.
gr,
marcel maatkamp
VPRO Digitaal
---
MMObjectNode
public Vector getRelatedNodes() {
Vector results = new Vector();
String type ="object";
MMObjectBuilder builder = (MMObjectBuilder)parent.mmb.getMMObject(type);
// example: we want a thisnode.relatedNodes(mediaparts) where mediaparts are
of type
// audioparts and videoparts. This method will return the real nodes (thus of
type audio/videoparts)
// when asked to get nodes of type mediaparts.
//
// - get a list of virtual nodes from a multilevel("this.parent.name, type")
ordered on otype
// (this will return virtual audio- and/or videoparts ordered on their
*real* parent)
// - construct a list of nodes for each parentbuilder seperately
// - ask the parentbuilder for each list of virtual nodes to get a list of the
real nodes
if( builder != null ) {
// multilevel from table this.parent.name -> type
Vector tables = new Vector();
tables.addElement(parent.getTableName());
tables.addElement(type);
// return type.number (and otype for sorting)
Vector fields = new Vector();
fields.addElement(type + ".number");
fields.addElement(type + ".otype");
// order list UP
Vector directions = new Vector();
directions.addElement("UP");
// and order on otype
Vector ordered = new Vector();
ordered.addElement(type + ".otype");
// retrieve the related nodes (these are virtual)
Vector v = multirelations.searchMultiLevelVector(
getNumber(),fields,"NO",tables,"",ordered,directions);
results = new Vector(getRealNodes(v, type));
} else {
log.error("This type("+type+") is not a valid buildername!");
}
return results;
}
/**
* Get the related nodes of a certain type. The returned nodes are not the
* nodes directly attached to this node (the relation nodes) but the nodes
* attached to the relation nodes of this node.
*
* @param type the type of objects to be returned
* @return a <code>Vector</code> containing <code>MMObjectNode</code>s
*/
public Vector getRelatedNodes(String type) {
Vector results = new Vector();
MMObjectBuilder builder = (MMObjectBuilder)parent.mmb.getMMObject(type);
// example: we want a thisnode.relatedNodes(mediaparts) where mediaparts are
of type
// audioparts and videoparts. This method will return the real nodes (thus of
type audio/videoparts)
// when asked to get nodes of type mediaparts.
//
// - get a list of virtual nodes from a multilevel("this.parent.name, type")
ordered on otype
// (this will return virtual audio- and/or videoparts ordered on their
*real* parent)
// - construct a list of nodes for each parentbuilder seperately
// - ask the parentbuilder for each list of virtual nodes to get a list of the
real nodes
if( builder != null ) {
// multilevel from table this.parent.name -> type
Vector tables = new Vector();
tables.addElement(parent.getTableName());
tables.addElement(type);
// return type.number (and otype for sorting)
Vector fields = new Vector();
fields.addElement(type + ".number");
fields.addElement(type + ".otype");
// order list UP
Vector directions = new Vector();
directions.addElement("UP");
// and order on otype
Vector ordered = new Vector();
ordered.addElement(type + ".otype");
// retrieve the related nodes (these are virtual)
Vector v = multirelations.searchMultiLevelVector(
getNumber(),fields,"NO",tables,"",ordered,directions);
results = new Vector(getRealNodes(v, type));
} else {
log.error("This type("+type+") is not a valid buildername!");
}
return results;
}
/**
* Loop through the virtuals vector, group all same nodes based on parent and
fetch the real nodes from those parents
*
* @param Vector of virtual nodes (only type.number and type.otype fields are set)
* @param type, needed to retreive the otype, which is set in node as type +
".otype"
* @returns List of real nodes
*
* @see getRelatedNodes(String type)
*/
private List getRealNodes(Vector virtuals, String type) {
List result = new ArrayList();
MMObjectBuilder rparent = null;
MMObjectNode node = null;
MMObjectNode convert = null;
Enumeration e = virtuals.elements();
List list = new ArrayList();
int otype = -1;
int ootype = -1;
// fill the list
while(e.hasMoreElements()) {
node = (MMObjectNode)e.nextElement();
otype = node.getIntValue(type + ".otype");
// same type, construct list further
if(ootype == -1 || otype == ootype) {
// convert type.number and type.otype to number and otype
convert = new MMObjectNode();
// parent needs to be set or else mmbase does nag nag nag on a
setValue()
convert.parent =
parent.mmb.getMMObject(parent.mmb.TypeDef.getValue(otype));
convert.setValue("number", node.getValue(type + ".number"));
convert.setValue("otype", otype);
list.add(convert);
// new strain detected, fetch list, add to result and make new list for
new parent
} else {
result.addAll(getRealNodesFromBuilder(list, ootype));
list = new ArrayList();
list.add(node);
ootype = otype;
}
// first and only list or last list, return real values
if(!e.hasMoreElements())
result.addAll(getRealNodesFromBuilder(list, otype));
}
// check that we didnt loose any nodes
// Java 1.4
// assert(virtuals.size() == result.size());
// Below Java 1.4
if(virtuals.size() != result.size()) {
log.error("We lost a few nodes during conversion from
virtualnodes("+virtuals.size()+") to realnodes("+result.size()+")");
}
return result;
}
private List getRealNodesFromBuilder(List list, int otype) {
List result = new ArrayList();
String name = parent.mmb.TypeDef.getValue(otype);
if(name != null) {
MMObjectBuilder rparent = parent.mmb.getMMObject(name);
if(rparent != null) {
result.addAll(rparent.getNodes(list));
} else {
log.error("This otype("+otype+") does not denote a valid
typedef-name("+name+")!");
}
} else {
log.error("This otype("+otype+") gives no name("+name+") from typedef!");
}
return result;
}
MMObjectBuilder
----------------------
/**
* Convert virtual nodes to real nodes based on their otype
*
* Normally a multirelations-search will return virtual nodes. These nodes
* will only contain values which where specified in the field-vector.
* This method will make real nodes of those virtual nodes.
*
* @param List containing virtual nodes
* @return List containing real nodes
*/
public List getNodes(List virtuals) {
List result = new ArrayList();
MMObjectNode node = null;
Integer number = null;
String numbers = "";
Iterator i = virtuals.iterator();
while(i.hasNext()) {
node = (MMObjectNode)i.next();
number = new Integer(node.getIntValue("number"));
// check if this node is already in cache
if(nodeCache.containsKey(number))
result.add(nodeCache.get(number));
// else seek it with a search on builder in db
else
numbers += "" + number;
if(i.hasNext())
numbers += ",";
}
// now that we have a comma seperated string of numbers, we can
// the search with a where-clause containing this list
if(!numbers.equals("")) {
result = new ArrayList(basicSearch("SELECT * FROM "+getFullTableName()+"
WHERE number in ("+numbers+")"));
} // else everything from cache
// check that we didnt loose any nodes
// Java 1.4
// assert(virtuals.size() == result.size());
// Below Java 1.4
if(virtuals.size() != result.size()) {
log.error("We lost a few nodes during conversion from
virtualsnodes("+virtuals.size()+") to realnodes("+result.size()+")");
}
return result;
}
