Hi Guido,
I think I know what is the matter. It has to do with the commit. But first I will explain the criptic commit message a little more.
I asume your query looks something like the following:
<basicsearch> <select> <allprop/> </select> <scope> <href>/files/repo/staging/documents</href> <depth>Infinity</depth> </scope> <where> <eq> <prop><getcontenttype/></prop> <literal>text/xml</literal> </eq> </where>
First off all the RDBMS expression factory does not yet know what properties are to be selected in case of <allprop> so property retrieval is forwarded to the default PropertyProvider. This means only the selection of what resources are in the resultset is optimized.
If on the other hand you would predefine the properties you want within a <prop> element in the select statement then the expression factory is more likely to be able to optimize the search but only on one condition, and that was what the below commit was about: the properties that you select should somehow participate in the where clause. Only in that case the expression factory will compile them to SQL.
So,
<basicsearch> <select> <prop><getcontenttype/></prop> </select> <scope> <href>/files/repo/staging/documents</href> <depth>Infinity</depth> </scope> <where> <eq> <prop><getcontenttype/></prop> <literal>text/xml</literal> </eq> </where>
Will be optimally fast because the getcontenttype property is retrieved by the SQL select.
But in this query:
<basicsearch> <select> <prop><getcontenttype/><displayname/></prop> </select> <scope> <href>/files/repo/staging/documents</href> <depth>Infinity</depth> </scope> <where> <eq> <prop><getcontenttype/></prop> <literal>text/xml</literal> </eq> </where>
the displayname is not retrieved by the SQL select because it does not participate in the where clause. Before the below commit I *was* compiling it into the SQL select, but the way it was done did not match the semantics of the original query. In effect the semantic of the above query was compiled to this:
<basicsearch> <select> <prop><getcontenttype/><displayname/></prop> </select> <scope> <href>/files/repo/staging/documents</href> <depth>Infinity</depth> </scope> <where> <and> <eq> <prop><getcontenttype/></prop> <literal>text/xml</literal> </eq> <is-defined><displayname/></is-defined> </and> </where>
Notice the is-defined expression in the where clause. Now in order to optimize your query you should try to let each selected property participate in the where clause. For instance by using is-defined clause like above.
But now concerning the NPE. I know what is causing it and why it wasn't there before I will commit a fix later in the day.
-- Unico
Guido Casper wrote:
Unico,
I think this change introduced a NPE on <D:allprop/> requests (see stacktrace below).
I'm not quite sure I understand what you're saying in the commit message.
<quote>
Selected properties that don't participate in the where clause cannot be correctly compiled to SQL without using nested select statements.(AFAIK)
</quote>
It worked fine with CVS from 12/07. Do you have any idea what the cause might be?
Thanks Guido
20 Jul 2004 09:11:10 - org.apache.slide.store.impl.rdbms.AbstractRDBMSStore - INFO - executing:
select u.URI_STRING, o.CLASS_NAME from (((OBJECT o inner join URI u on u.URI_ID = o.URI_ID) inner join VERSION_HISTORY vh on vh.URI_ID = u.URI_ID) inner join PROPERTIES p0 on p0.VERSION_ID = vh.VERSION_ID) where (u.URI_STRING = '/files/repo/staging/documents' OR u.URI_STRING LIKE '/files/repo/staging/documents/%') AND (p0.PROPERTY_NAME = 'getcontenttype' AND p0.PROPERTY_NAMESPACE = 'DAV:' AND p0.PROPERTY_VALUE = 'text/xml
java.lang.NullPointerException
at org.apache.slide.search.basic.ComparableResourceImpl.getAllPropertiesNames(ComparableResourceImpl.java:369)
at org.apache.slide.webdav.util.PropertyRetrieverImpl.getAllPropertiesOfObject(PropertyRetrieverImpl.java:422)
at org.apache.slide.webdav.util.PropertyRetrieverImpl.getPropertiesOfObject(PropertyRetrieverImpl.java:301)
at org.apache.slide.webdav.method.SearchMethod$WebdavResult.init(SearchMethod.java:409)
at org.apache.slide.webdav.method.SearchMethod$WebdavResult.<init>(SearchMethod.java:351)
at org.apache.slide.webdav.method.SearchMethod.executeRequest(SearchMethod.java:224)
at org.apache.slide.webdav.method.AbstractWebdavMethod.run(AbstractWebdavMethod.java:364)
at org.apache.slide.webdav.WebdavServlet.service(WebdavServlet.java:166)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:237)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:157)
at org.apache.slide.webdav.filter.LogFilter.doFilter(LogFilter.java:141)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:186)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:157)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:214)
at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
at org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:198)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:152)
at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:540)
at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)
at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)
at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:793)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:702)
at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:571)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:644)
at java.lang.Thread.run(Thread.java:534)
http-9090-Processor22, 20-Jul-2004 09:11:10, gcasper, SEARCH, 500 "Internal Server Error", 40 ms, /files/repo/staging/documents/
[EMAIL PROTECTED] wrote:
unico 2004/07/14 10:02:33
Modified: src/stores/org/apache/slide/store/impl/rdbms
RDBMSComparableResourcesPool.java
Log:
Selected properties that don't participate in the where clause cannot be correctly compiled to SQL without using nested select statements.(AFAIK)
Since not all databases support this we cannot use it in the default implementation.
Selected properties that do not participate in the where clause are separately retrieved by the provided property provider
instead of from the database directly.
This results in a large performance penalty when doing queries where you select properties that don't also participate in the where clause
because for each resource in the result set those properties are retrieved separately.
To optimize your queries you should, if possible let all properties
you want to select participate in the where clause. For instance by using <d:is-defined/> expression:
<select><d:prop><d:displayname/></d:prop></select>
<from><d:scope>somescope</d:scope></from>
<where><d:is-defined><d:prop><d:displayname/></d:prop></d:is-defined></where>
Revision Changes Path
1.5 +14 -75 jakarta-slide/src/stores/org/apache/slide/store/impl/rdbms/RDBMSComparableResourcesPool.java
Index: RDBMSComparableResourcesPool.java
===================================================================
RCS file: /home/cvs/jakarta-slide/src/stores/org/apache/slide/store/impl/rdbms/RDBMSComparableResourcesPool.java,v
retrieving revision 1.4
retrieving revision 1.5
diff -u -r1.4 -r1.5
--- RDBMSComparableResourcesPool.java 13 Jul 2004 12:02:13 -0000 1.4
+++ RDBMSComparableResourcesPool.java 14 Jul 2004 17:02:33 -0000 1.5
@@ -49,7 +49,6 @@
import org.apache.slide.search.basic.ComparableResourceImpl;
import org.apache.slide.search.basic.ComparableResourcesPool;
import org.apache.slide.search.basic.IBasicQuery;
-import org.apache.slide.store.impl.rdbms.expression.RDBMSExpressionFactory;
import org.apache.slide.store.impl.rdbms.expression.RDBMSQueryContext;
import org.apache.slide.structure.ObjectNode;
import org.apache.slide.util.logger.Logger;
@@ -78,20 +77,19 @@
_token = _query.getSearchToken();
_scope = _query.getScope();
_selectProperties = new HashMap();
- _provider = _query.getPropertyProvider();
+ _provider = new RDBMSPropertyProvider(_query.getPropertyProvider(), _selectProperties);
+
if (_query instanceof SearchQuery) {
- RequestedProperties props = ((SearchQuery) _query).requestedProperties();
+ final RequestedProperties props = ((SearchQuery) _query).requestedProperties();
if (!props.isAllProp()) {
- Iterator iter = props.getRequestedProperties();
+ final Iterator iter = props.getRequestedProperties();
while (iter.hasNext()) {
- RequestedProperty property = (RequestedProperty) iter.next();
- if (RDBMSExpressionFactory.isSQLCompilableProperty(property.getNamespace(), property.getName())) {
+ final RequestedProperty property = (RequestedProperty) iter.next();
+ final String selectKey = property.getNamespace() + property.getName();
+ if (_context.selects().containsKey(selectKey)) {
_selectProperties.put(property, new HashMap());
}
}
- if (_selectProperties.size() > 0) {
- _provider = new RDBMSPropertyProvider(_query.getPropertyProvider(), _selectProperties);
- }
}
}
}
@@ -114,10 +112,8 @@
_pool.add(new ComparableResourceImpl(objects[i], _token, _scope, _provider));
}
} catch (ServiceAccessException e) {
- e.printStackTrace();
throw new BadQueryException(e);
} catch (SlideException e) {
- e.printStackTrace();
throw new BadQueryException(e);
}
}
@@ -161,8 +157,8 @@
ArrayList uris = new ArrayList();
try {
final String sql = compileSQL();
- if (_store.getLogger().isEnabled(Logger.DEBUG)) {
- _store.getLogger().log("executing: " + sql, AbstractRDBMSStore.LOG_CHANNEL, Logger.DEBUG);
+ if (_store.getLogger().isEnabled(Logger.INFO)) {
+ _store.getLogger().log("executing: " + sql, AbstractRDBMSStore.LOG_CHANNEL, Logger.INFO);
}
statement = connection.prepareStatement(sql);
result = statement.executeQuery();
@@ -231,28 +227,19 @@
" where " + compileScope(uri);
final String criteria = compileCriteria();
query = (criteria != null && criteria.length() > 0) ? query + " AND " + criteria : query;
-// query += compileOrderBy();
-// query += compileLimit();
return query;
}
private String compileSelect() {
String select = "u.URI_STRING, o.CLASS_NAME";
final Iterator iter = _selectProperties.keySet().iterator();
- int index = 0;
while (iter.hasNext()) {
final RequestedProperty property = (RequestedProperty) iter.next();
final String selectKey = property.getNamespace() + property.getName();
String propSelect = (String) _context.selects().get(selectKey);
- if (propSelect == null) {
- // TODO: qualify alias
- propSelect = "sp" + index + ".PROPERTY_VALUE AS " + property.getName();
-// propSelect = "sp" + index + ".PROPERTY_VALUE AS " + property.getName() + "_value" +
-// "sp.PROPERTY_TYPE AS " + property.getName() + "_type" +
-// "sp.IS_PROTECTED AS " + property.getName() + "_protected";
- index++;
+ if (propSelect != null) {
+ select += ", " + propSelect;
}
- select += ", " + propSelect;
}
return select;
}
@@ -284,25 +271,6 @@
result += iter.next();
}
}
- Iterator iter = _selectProperties.keySet().iterator();
- int index = 0;
- while (iter.hasNext()) {
- final RequestedProperty property = (RequestedProperty) iter.next();
- final String name = property.getName();
- final String namespace = property.getNamespace();
- final String selectKey = namespace + name;
- if (!_context.selects().containsKey(selectKey)) {
- if (result == null) {
- result = "";
- }
- else {
- result += " AND ";
- }
- result += " (sp" + index + ".PROPERTY_NAME = '" + name + - "' AND sp" + index + ".PROPERTY_NAMESPACE = '" + namespace + "') ";
- index++;
- }
- }
return result;
}
@@ -314,34 +282,9 @@
while (iter.hasNext()) {
joins = "(" + joins + " " + iter.next() + ") ";
}
- Iterator iter2 = _selectProperties.keySet().iterator();
- int index = 0;
- while (iter2.hasNext()) {
- final RequestedProperty property = (RequestedProperty) iter2.next();
- final String selectKey = property.getNamespace() + property.getName();
- if (!_context.selects().containsKey(selectKey)) {
- joins = "(" + joins + " inner join PROPERTIES sp" + index + - " on sp" + index + ".VERSION_ID = vh.VERSION_ID)";
-// joins = "(" + joins + " left join " +
-// "(SELECT VERSION_ID, PROPERTY_VALUE " +
-// "FROM PROPERTIES WHERE VERSION_ID = vh.VERSION_ID " +
-// "AND PROPERTY_NAME = '" + property.getName() + "' " +
-// "AND PROPERTY_NAMESPACE= '" + property.getNamespace() + "') " +
-// "AS sp" + index + " ON sp" + index + ".VERSION_ID = vh.VERSION_ID)";
- index++;
- }
- }
return joins;
}
-// private String compileOrderBy() {
-// return "";
-// }
-//
-// private String compileLimit() {
-// return "";
-// }
- private static class RDBMSPropertyProvider implements PropertyProvider {
private final PropertyProvider _propertyProvider;
@@ -360,22 +303,18 @@
}
public Iterator getSupportedPropertiesNames(String resourceUri) throws SlideException {
- // TODO: implement
return null;
}
public NodeProperty getProperty(String resourceUri, String propertyName, String propertyNamespace) throws SlideException {
Map properties = (Map) _selectProperties.get(new RequestedPropertyImpl(propertyName, propertyNamespace));
if (properties != null) {
-// System.out.println("reading property directly");
return (NodeProperty) properties.get(resourceUri);
}
-// System.out.println("not reading property directly");
return _propertyProvider.getProperty(resourceUri, propertyName, propertyNamespace);
}
public Iterator getSupportedProperties(String resourceUri) throws SlideException {
- // TODO: implement
return null;
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
