Rene,
Your question can easily be answered by going to this link (the specification page for the find_business function): http://uddi.org/pubs/uddi-v3.0.2-20041019.htm#_Toc85908086 Lucene may work well for the "name" search (in fact, I believe we talked about using it) but what about searches for identifiers, categories, tmodels, discoveryURLs, etc. Not to mention, there are findQualifiers that instruct the search to either AND or OR the various identifier or category keys. Categories are even more complicated as they can be grouped and you may have to AND some keys and OR other keys. The bottom line is, we are obviously bound to implement the specification and the specification describes a very complicated search that can't easily be handled by abstracting to a full-text search engine. Even the name search has some complications as there are rules we must follow on how you apply wildcards and you have to account for the language attribute. I investigated Lucene a little bit when I started the inquiry code, and decided against using it for the reasons stated above. But, I never got a full flavor for what it could do. If you know more about its capabilities maybe you could take a look at the specification and make a better assessment as to whether it is a viable option. -Jeff From: Rene Schwietzke [mailto:[email protected]] Sent: Saturday, August 01, 2009 4:32 AM To: [email protected] Subject: Re: jUDDI load testing update: Max search results Hi Jeff, That sounds like a implementation that is hard to manage and highly system dependent. Especially with bigger result sets or very generic queries this will hit a limit very often. How about putting Lucene underneath and running all searches by it? Text searching would become very comfortable and very fast. If I am not mistaken, there is a transparent Lucene integration for Hibernate available. Additionally the system scales with application server cpus and does not depend on the database power that much anymore. Also it becomes maintenance friendly, because database maintenance is always a big cost factor and can only be done by a few people in an organization. Throwing cpus and memory at Tomcat/Jetty is way simpler. Rene Jeff Faath wrote: I'm guessing it's the final query that compiles and sorts the results. Searching works by running a series of queries - one for each kind of criteria (names, categories, identifiers, etc) - that return a list of keys that match the criteria. This list is subsequently passed into each query and appended as an IN list and therefore narrowed down. We had a user post an issue that Oracle only allows 1000 terms in the IN list so these criteria queries chunk the IN list into groups of 1000 (there is a parameter that sets this value). However, after all these queries are run and the results are hopefully narrowed down enough, the resulting key list is used in an IN clause for one final query that grabs all the necessary data and sorts the results. In order to take advantage of SQL sorting, this final query does not "chunk" the IN clause. Otherwise, sorting would have to occur as another step within Java. The maximum results parameter for the search doesn't help because the entire query is still run (but only the maximum results are compiled into the returned list). So, I'm thinking that this final query is the one that is causing the error. It could be that either Hibernate or Derby (or JDBC) has an issue with a query that has an IN clause with 2741 terms (Oracle would've died at 1000). The solution would be to chunk this query like the previous criteria queries and then do the sorting within Java. I'm curious to see the results with MySql. Any thoughts? -----Original Message----- From: Jeremi Thebeau [mailto:[email protected]] Sent: Friday, July 31, 2009 11:00 AM To: [email protected] Subject: jUDDI load testing update: Max search results Hi, I tried to determine how many businesses I could publish to juddi before I get to many results back when I search for all businesses. To do this I ran a test where I start with an empty DB (derby), publish a business with a single service, then run a search for any business (by submitting the wild-card "%" as the name of the business I am looking for). I run this test repeatedly until the first error. Three times now I've gotten to the 2741st iteration before the first error. Here is the return SOAP msg and the stack trace: Soap Message: <soap:Envelope xmlns:soap= <http://schemas.xmlsoap.org/soap/envelope/> "http://schemas.xmlsoap.org/soap/envelope/";> <soap:Header /> <soap:Body> <soap:Fault> <faultcode> soap:Server </faultcode> <faultstring> org.hibernate.exception.GenericJDBCException: could not execute query </faultstring> </soap:Fault> </soap:Body> </soap:Envelope> Message Context Properties: - com.sun.xml.internal.ws.api.message.HeaderList = [] - com.sun.xml.internal.ws.api.server.WSEndpoint = null - com.sun.xml.internal.ws.client.ContentNegotiation = none - com.sun.xml.internal.ws.client.handle = null - com.sun.xml.internal.ws.handler.config = com.sun.xml.internal.ws.client.handlerconfigurat...@183ee9a - com.sun.xml.internal.ws.server.OneWayOperation = null - com.xceptance.xlt.ws.handler.requestData = com.xceptance.xlt.api.engine.requestd...@13ada - javax.xml.ws.handler.message.outbound = false - javax.xml.ws.http.response.code = 200 - javax.xml.ws.http.response.headers = {null=[HTTP/1.1 200 OK], Content-type=[text/xml;charset=UTF-8], Content-length=[264], Server=[Apache-Coyote/1.1], Date=[Fri, 31 Jul 2009 12:22:07 GMT]} - javax.xml.ws.reference.parameters = [] - javax.xml.ws.service.endpoint.address = http://192.168.178.200:8080/juddi/services/inquiry - javax.xml.ws.soap.http.soapaction.uri = null - javax.xml.ws.wsdl.interface = {urn:uddi-org:api_v3_portType}UDDI_Inquiry_PortType - javax.xml.ws.wsdl.port = {urn:uddi-org:api_v3_portType}UDDIInquiryServicePort - javax.xml.ws.wsdl.service = {urn:uddi-org:api_v3_portType}UDDIInquiryService [14:21:50,074] ERROR [main] - Execution step failed (user: 'TFindBusinessByName-0', output: '1249042905521'): javax.xml.ws.soap.SOAPFaultException: org.hibernate.exception.GenericJDBCException: could not execute query at com.sun.xml.internal.ws.fault.SOAP11Fault.getProtocolException(SOAP11Fault.j ava:178) at com.sun.xml.internal.ws.fault.SOAPFaultBuilder.createException(SOAPFaultBuil der.java:111) at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandle r.java:108) at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandle r.java:78) at com.sun.xml.internal.ws.client.sei.SEIStub.invoke(SEIStub.java:107) at $Proxy43.findBusiness(Unknown Source) at org.apache.juddi.xlt.action.SOAP.FindBusinessByName.execute(FindBusinessByNa me.java:91) at com.xceptance.xlt.api.actions.AbstractAction.run(AbstractAction.java:315) at org.apache.juddi.xlt.test.TFindBusinessByName.findBusiness(TFindBusinessByNa me.java:45) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39 ) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl .java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod. java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.j ava:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.ja va:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.jav a:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28 ) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.jav a:73) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.jav a:46) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41) at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28 ) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) at org.junit.runners.ParentRunner.run(ParentRunner.java:220) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestRef erence.java:45) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:3 8) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRu nner.java:460) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRu nner.java:673) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner. java:386) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner .java:196) Is this expected behavior? The fault message doesn't exactly tell me that there are too many search results returned. Is there another possible problem? Possibly worth mentioning is that a maximum of 5 results is specified for the business search. I'm using the snapshot release as is found here: http://people.apache.org/repo/m2-snapshot-repository/org/apache/juddi/juddi- portal-bundle/3.0.0.SNAPSHOT/juddi-portal-bundle-3.0.0.20090723.201427-7.zip installed on the following machine: Processor: Intel(R) Core(TM)2 CPU, 6400 @ 2.13GHz; RAM: 2011 MB; OS: Ubuntu 9.04, linux kernel 2.6.28-13-generic. The three XLT test reports can be view here: http://xlt.xceptance.de/download/results/juddi/20090730-195254/ http://xlt.xceptance.de/download/results/juddi/20090731-112335/ These first two runs are basically the same except the first one was planed to run for 15 hours and the second for only 3 hours. They both ran for 1h 25 min http://xlt.xceptance.de/download/results/juddi/20090731-132952/ This run is different only in that it uses the same AuthToken for all queries. Here is the XLT test suite that was used: http://xlt.xceptance.de/download/results/juddi/testsuites/20090731-juddiTest Suite.zip Next I will switch to a mySQL DB and rerun these test. Jeremi
