Thanks for the update. Let me know if there's anything I can do to help. Karl
On Thu, Aug 4, 2011 at 4:10 AM, Piergiorgio Lucidi <[email protected]> wrote: > I found a solution to create the right configuration for the CMIS Repository > Connector in the integration test code, and now it works ;) > > Chemistry guys are supporting me about an issue that I have found in the > OpenCMIS InMemory Repository that I would like to use in the integration > test for Manifold. > > Now I'm finishing the integration test implementation using the public > Alfresco CMIS server. Then I can start to test this new version of the > InMemory Repository with the bugfix provided by Jens. And I have to follow > some useful suggestions provided by Florian. > > I'll let you know soon about all these tasks. > > Piergiorgio > > ---------- Forwarded message ---------- > From: Jens Hübel <[email protected]> > Date: 2011/8/4 > Subject: RE: CMIS and Lucene > To: [email protected] > > > Hi Piergiorgio, > > you are right. I could reproduce the problem. There is a bug in the query > implementation of the InMemory server if you query for predefined properties > like cmis:name, cmis:createdBy, etc. :-( > > I have fixed this, so hopefully this works for you now if you take the > latest build. I have created CMIS-413 for this ( > https://issues.apache.org/jira/browse/CMIS-414). Please reopen if you still > have issues. > > One more note: Your test code is quite fragile if you use a fixed name for > your test document/folder. This implementation does not allow multiple > objects with the same name in one folder. This means unless you restart the > server your code will work only once. I recommend you using a random UUID as > name or something like that. In the default configuration the InMemory > creates a tree of document and folders by default. So it might not be > necessary to create objects at all (use the Workbench to take a look at > that). If you really need this reproducible behavior there is an option to > run the server using the local binding in the same Java VM. Then you can > restart the server with each test. The InMemory JUnit tests use this > mechanism. I do not recommend this however, because you bypass the whole > protocol layer for AtomPub/SOAP. This leaves many issues undetected a client > may see in a real connection later (and of course you lose the option to > switch to another CMIS server just by changing configuration). > > Jens > > > -----Original Message----- > From: Piergiorgio Lucidi [mailto:[email protected]] > Sent: Mittwoch, 3. August 2011 14:48 > To: [email protected] > Subject: Re: CMIS and Lucene > > Hi Jens, > > here the code of my integration test that is used to create some content in > the InMemory Repository, the OpenCMIS server is running because it is > bootstrapped by Jetty / Maven. The CMIS Repository Connector works fine with > Alfresco 3.4d Community, but I would like to have the OpenCMIS server in the > test suite. > > In the following snippet I started to initialize the test environment with a > new folder with a new content, here I don't have any problem, it works with > the InMemory Repository: > > private Session getCmisClientSession(){ >> // default factory implementation >> SessionFactory factory = SessionFactoryImpl.newInstance(); >> Map<String, String> parameters = new HashMap<String, String>(); >> // user credentials >> parameters.put(SessionParameter.USER, "dummyuser"); >> parameters.put(SessionParameter.PASSWORD, "dummysecret"); >> // connection settings >> parameters.put(SessionParameter.ATOMPUB_URL, CMIS_ENDPOINT_TEST_SERVER >> ); >> parameters.put(SessionParameter.BINDING_TYPE, BindingType.ATOMPUB >> .value()); >> // create session >> return factory.getRepositories(parameters).get(0).createSession(); >> } >> @Before >> public void createTestArea() >> throws Exception >> { >> try >> { >> Session session = getCmisClientSession(); >> //creating a new folder >> Folder root = session.getRootFolder(); >> Map<String, Object> folderProperties = new HashMap<String, >> Object>(); >> folderProperties.put(PropertyIds.OBJECT_TYPE_ID, "cmis:folder"); >> folderProperties.put(PropertyIds.NAME, "testdata"); >> >> Folder newFolder = root.createFolder(folderProperties); >> //create a new content in the folder >> String name = "testdata1.txt"; >> // properties >> // (minimal set: name and object type id) >> Map<String, Object> contentProperties = new HashMap<String, >> Object>(); >> contentProperties.put(PropertyIds.OBJECT_TYPE_ID, "cmis:document"); >> contentProperties.put(PropertyIds.NAME, name); >> >> // content >> byte[] content = "CMIS Testdata One".getBytes(); >> InputStream stream = new ByteArrayInputStream(content); >> ContentStream contentStream = new ContentStreamImpl(name, > newBigInteger(content), >> "text/plain", stream); >> >> // create a major version >> Document newContent1 = newFolder.createDocument(contentProperties, >> contentStream, null); > > > But if I try to search the new content in the InMemory Repository, in the > same way I implemented in the CMIS Repository Connector: > > ItemIterable<QueryResult> results = session.query("SELECT * FROM >> cmis:folder WHERE cmis:name='testdata'", false); >> for (QueryResult result : results) { >> String id = result.getPropertyValueById(PropertyIds.OBJECT_ID); >> } > > > > It returns the following exception: > > org.apache.chemistry.opencmis.commons.exceptions.CmisRuntimeException: null >> at >> > org.apache.chemistry.opencmis.client.bindings.spi.atompub.AbstractAtomPubService.convertStatusCode( >> AbstractAtomPubService.java:450) >> at >> > org.apache.chemistry.opencmis.client.bindings.spi.atompub.AbstractAtomPubService.post( >> AbstractAtomPubService.java:568) >> at >> > org.apache.chemistry.opencmis.client.bindings.spi.atompub.DiscoveryServiceImpl.query( >> DiscoveryServiceImpl.java:141) >> at org.apache.chemistry.opencmis.client.runtime.SessionImpl$3.fetchPage( >> SessionImpl.java:557) >> at >> > org.apache.chemistry.opencmis.client.runtime.util.AbstractIterator.getCurrentPage( >> AbstractIterator.java:132) >> at >> > org.apache.chemistry.opencmis.client.runtime.util.CollectionIterator.hasNext( >> CollectionIterator.java:48) >> at org.apache.manifoldcf.cmis_tests.APISanityTest.createTestArea( >> APISanityTest.java:139) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at sun.reflect.NativeMethodAccessorImpl.invoke( >> NativeMethodAccessorImpl.java:39) >> at sun.reflect.DelegatingMethodAccessorImpl.invoke( >> DelegatingMethodAccessorImpl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall( >> FrameworkMethod.java:44) >> at org.junit.internal.runners.model.ReflectiveCallable.run( >> ReflectiveCallable.java:15) >> at org.junit.runners.model.FrameworkMethod.invokeExplosively( >> FrameworkMethod.java:41) >> at org.junit.internal.runners.statements.RunBefores.evaluate( >> RunBefores.java:27) >> at org.junit.internal.runners.statements.RunAfters.evaluate( >> RunAfters.java:31) >> at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored( >> BlockJUnit4ClassRunner.java:79) >> at org.junit.runners.BlockJUnit4ClassRunner.runChild( >> BlockJUnit4ClassRunner.java:71) >> at org.junit.runners.BlockJUnit4ClassRunner.runChild( >> BlockJUnit4ClassRunner.java:49) >> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) >> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) >> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) >> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) >> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) >> at org.junit.runners.ParentRunner.run(ParentRunner.java:236) >> at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run( >> JUnit4TestReference.java:49) >> at org.eclipse.jdt.internal.junit.runner.TestExecution.run( >> TestExecution.java:38) >> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests( >> RemoteTestRunner.java:467) >> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests( >> RemoteTestRunner.java:683) >> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run( >> RemoteTestRunner.java:390) >> at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main( >> RemoteTestRunner.java:197) > > > Let me know if you have any hints! > > Thank you again for your support. > > Regards, > > Piergiorgio > > 2011/8/3 Jens Hübel <[email protected]> > >> Hi Pergiorio, >> >> do you have a code piece that I can take a look at? What is your client >> API? Do you connect to Chemistry client API > (chemistry-opencmis-client-api)? >> Or perhaps you can grab the query sent from the log file? >> >> Thanks Jens >> >> >> -----Original Message----- >> From: Piergiorgio Lucidi [mailto:[email protected]] >> Sent: Mittwoch, 3. August 2011 13:22 >> To: [email protected] >> Subject: Re: CMIS and Lucene >> >> Hi Jens, >> >> I contributed to the Apache ManifoldCF (ex Lucene Connector Framework) >> project implementing the CMIS Connector using OpenCMIS 0.4.0. >> >> Now I'm working on integration test trying to use the OpenCMIS InMemory >> Repository that is available from Apache Chemistry ;) >> >> I have some problems because now I'm finishing my test implementation but >> it >> seems that I can't execute queries on the InMemory Repository, it returns > a >> null string in the convertStatusCode method. >> >> The CMIS Repository Connector that I implemented for ManifoldCF has a job >> parameter that is the CMIS Query, this parameter is used by agents to >> select >> all the contents that needs to be indexed on the Output Connector (for >> example Apache Solr). >> Without a query feature exposed by the InMemory Repository I can't test > the >> connector in the right way. >> >> I tried to use the latest version of the InMemory Repository >> (0.5.0-SNAPSHOT) but with the same result. >> >> Have you got any ideas to solve this problem? >> Thank you for your support. >> >> Regards, >> Piergiorgio >> >> >> 2011/8/3 Jens Hübel <[email protected]> >> >> > Hi Chemistries, >> > >> > >> > >> > is anyone of us already involved in the Lucene project? >> > >> > >> > >> > Reading this here >> > http://search-lucene.com/m/wI38e1K3BWJ&subj=Re+CMIS+Connector+Tests >> > >> > >> > >> > I think it would make sense to give them an introduction to what we > have. >> > >> > >> > >> > if I understand this correctly one of the options they discuss is to >> create >> > an InMemory mock server. It does not make much sense to duplicate what > we >> > already have (and if we need getContentChanges() for that I am happy to >> > implement this). >> > >> > >> > >> > Jens >> > >> > >> > >> > >> >> >> -- >> Piergiorgio Lucidi >> Web: http://about.me/piergiorgiolucidi >> > > > > -- > Piergiorgio Lucidi > Web: http://about.me/piergiorgiolucidi > > > > -- > Piergiorgio Lucidi > Web: http://about.me/piergiorgiolucidi > > 2011/8/3 Piergiorgio Lucidi <[email protected]> > >> Yes, this is a very quick way to test my configuration code for the >> integration test ;) >> I'm going to fix this part, I saw what I need in the JSON retrieved by a >> GET call against the Manifold API service. >> >> I'll let you know tomorrow an update about this. >> >> Anyway we have a problem with the OpenCMIS InMemory server. It seems that >> there is a problem during the execution of CMIS queries, I notified the >> problem to the Chemistry guys (Jens Hubel and Florian Muller) and they are >> trying to reproduce the issue to solve our problem. >> >> I'm going on developing the code using the public Alfresco CMIS server >> exposed at the following address: >> http://cmis.alfresco.com >> >> Piergiorgio >> >> >> 2011/8/3 Karl Wright <[email protected]> >> >>> Another good way to see exactly what you need to do is to call the API >>> to get configuration information for an existing connection. Then, >>> use the toXML() method to convert to XML, or the toJSON() to get it as >>> JSON. Either way you will see the structure. BTW, ManifoldCF in >>> Action Chapter 3 covers this in great detail as well. >>> >>> Karl >>> >>> On Wed, Aug 3, 2011 at 12:50 PM, Karl Wright <[email protected]> wrote: >>> > The ConfigParams class is, I believe, derived from the Configuration >>> > class. So, you can create a ConfigParams object instead of a >>> > Configuration object if you want to use the API in the manner you >>> > describe. >>> > >>> > The reason your commented-out code doesn't work is because the >>> > setParameter() method isn't doing quite what you are expecting. It's >>> > creating a node named "_PARAMETER_" with a "name" attribute and a >>> > value area, and you are creating nodes named by the parameter name. >>> > >>> > Karl >>> > >>> > On Wed, Aug 3, 2011 at 12:41 PM, Piergiorgio Lucidi >>> > <[email protected]> wrote: >>> >> I'm trying to implement tests but I found a problem to set all the >>> needed >>> >> parameters to the CMIS Repository Connector that needs: an username, a >>> >> password and the endpoint (url). >>> >> >>> >> I need to know how to create the configuration nodes for the connector, >>> in >>> >> the connector code I managed the configuration parameters in this way >>> in the >>> >> processConfigurationPost method: >>> >> >>> >> public String processConfigurationPost(IThreadContext threadContext, >>> >>> IPostParameters variableContext, ConfigParams parameters) >>> >>> throws ManifoldCFException { >>> >>> String username = >>> variableContext.getParameter(CONFIG_PARAM_USERNAME); >>> >>> if (StringUtils.isNotEmpty(username)) >>> >>> parameters.setParameter(CONFIG_PARAM_USERNAME, username); >>> >>> String password = >>> variableContext.getParameter(CONFIG_PARAM_PASSWORD); >>> >>> if (StringUtils.isNotEmpty(password)) >>> >>> parameters.setParameter(CONFIG_PARAM_PASSWORD, password); >>> >>> String endpoint = >>> variableContext.getParameter(CONFIG_PARAM_ENDPOINT); >>> >>> if (StringUtils.isNotEmpty(endpoint) && endpoint.length() > 0) >>> >>> parameters.setParameter(CONFIG_PARAM_ENDPOINT, endpoint); >>> >>> String repositoryId = variableContext >>> >>> .getParameter(CONFIG_PARAM_REPOSITORY_ID); >>> >>> if (StringUtils.isNotEmpty(repositoryId)) >>> >>> parameters.setParameter(CONFIG_PARAM_REPOSITORY_ID, >>> repositoryId); >>> >>> return null; >>> >>> } >>> >> >>> >> >>> >> Now I have to setup the same parameters inside my test class >>> APISanityTest >>> >> that doesn't like the following snippet, it works only if CMIS >>> parameters >>> >> are commented as the following: >>> >> >>> >> @Test >>> >>> public void sanityCheck() >>> >>> throws Exception >>> >>> { >>> >>> try >>> >>> { >>> >>> // Hey, we were able to install the file system connector etc. >>> >>> // Now, create a local test job and run it. >>> >>> IThreadContext tc = ThreadContextFactory.make(); >>> >>> int i; >>> >>> IJobManager jobManager = JobManagerFactory.make(tc); >>> >>> // Create a basic file system connection, and save it. >>> >>> ConfigurationNode connectionObject; >>> >>> ConfigurationNode child; >>> >>> Configuration requestObject; >>> >>> Configuration result; >>> >>> >>> >>> connectionObject = new >>> ConfigurationNode("repositoryconnection"); >>> >>> >>> >>> child = new ConfigurationNode("name"); >>> >>> child.setValue("CMIS Connection"); >>> >>> >>> connectionObject.addChild(connectionObject.getChildCount(),child); >>> >>> >>> >>> child = new ConfigurationNode("class_name"); >>> >>> child.setValue( >>> >>> >>> "org.apache.manifoldcf.crawler.connectors.cmis.CmisRepositoryConnector"); >>> >>> >>> connectionObject.addChild(connectionObject.getChildCount(),child); >>> >>> >>> >>> child = new ConfigurationNode("description"); >>> >>> child.setValue("CMIS Connection"); >>> >>> >>> connectionObject.addChild(connectionObject.getChildCount(),child); >>> >>> child = new ConfigurationNode("max_connections"); >>> >>> child.setValue("10"); >>> >>> >>> connectionObject.addChild(connectionObject.getChildCount(),child); >>> >>> >>> >>> //setting the CMIS specific parameters >>> >>> // child = new ConfigurationNode("username"); >>> >>> // child.setValue(CMIS_USERNAME); >>> >>> // >>> connectionObject.addChild(connectionObject.getChildCount(),child); >>> >>> // >>> >>> // child = new ConfigurationNode("password"); >>> >>> // child.setValue(CMIS_PASSWORD); >>> >>> // >>> connectionObject.addChild(connectionObject.getChildCount(),child); >>> >>> // >>> >>> // child = new ConfigurationNode("endpoint"); >>> >>> // child.setValue(CMIS_ENDPOINT_TEST_SERVER); >>> >>> // >>> connectionObject.addChild(connectionObject.getChildCount(),child); >>> >>> requestObject = new Configuration(); >>> >>> requestObject.addChild(0,connectionObject); >>> >>> >>> >>> result = performAPIPutOperationViaNodes( >>> >>> "repositoryconnections/CMIS%20Connection",201,requestObject); >>> >> >>> >> >>> >> How can I set the username, password and endpoint for the CMIS >>> Repository >>> >> Connector parameters in this test class? >>> >> >>> >> Thank you. >>> >> >>> >> Piergiorgio >>> >> >>> >> >>> >> 2011/8/2 Karl Wright <[email protected]> >>> >> >>> >>> Thanks for the status report. I hope to see your patch soon! >>> >>> >>> >>> Also, FWIW, once the documentation is also done I'd like to consider >>> >>> solidifying the 0.3 release. It's got a lot of good stuff in it and I >>> >>> think as soon as we've finished off the new CMIS connector in all >>> >>> dimensions we should go ahead. Thoughts, anyone? >>> >>> >>> >>> Karl >>> >>> >>> >>> >>> >>> On Tue, Aug 2, 2011 at 5:00 AM, Piergiorgio Lucidi >>> >>> <[email protected]> wrote: >>> >>> > Yesterday I started to work on end-to-end integration test for the >>> CMIS >>> >>> > Connector and now I have a full running OpenCMIS test server >>> integrated >>> >>> with >>> >>> > the ManifoldCF Maven build process. >>> >>> > >>> >>> > Now I have to implement: >>> >>> > - a setup method to create the test documents in the CMIS server >>> >>> > - a null output connector using the ManifoldCF api >>> >>> > - tests using the ManifoldCF api to create a mock configuration >>> against >>> >>> the >>> >>> > test CMIS server >>> >>> > >>> >>> > I'll let you know when it works. >>> >>> > >>> >>> > Regards, >>> >>> > Piergiorgio >>> >>> > >>> >>> > 2011/7/29 Piergiorgio Lucidi <[email protected]> >>> >>> > >>> >>> >> Hi Karl, >>> >>> >> >>> >>> >> thank you for the details and as soon as I finish a first version >>> of >>> >>> >> integration and/or unit test I will create a new ticket in the CMIS >>> >>> >> Component to release the patch. >>> >>> >> >>> >>> >> I hope to release this new patch soon. >>> >>> >> I'll let you know during the next week. >>> >>> >> >>> >>> >> Piergiorgio >>> >>> >> >>> >>> >> >>> >>> >> 2011/7/28 Karl Wright <[email protected]> >>> >>> >> >>> >>> >>> The tests/filesystem/src/test/java area are end-to-end tests >>> >>> >>> principally designed to test the filesystem connector. (They also >>> >>> >>> involve other connectors because, obviously, an end-to-end test is >>> not >>> >>> >>> going going to work without them. This is why they are at the >>> root >>> >>> >>> level.) >>> >>> >>> >>> >>> >>> There are also unit tests you can use as models under >>> >>> >>> connectors/filesystem/connector/src/test/.... These tests use >>> only >>> >>> >>> the filesystem connector, no others. >>> >>> >>> >>> >>> >>> If you would be so kind as to create a ticket to cover your test >>> work, >>> >>> >>> that would be great. >>> >>> >>> >>> >>> >>> Also, another workitem I've been meaning to get to but haven't >>> >>> >>> involves the conversion of a python-based HTML browser and >>> Javascript >>> >>> >>> engine I wrote for MetaCarta into a Java equivalent. This would >>> allow >>> >>> >>> direct testing of the UI components of the framework and >>> connectors. >>> >>> >>> The python sources are in: >>> >>> >>> >>> >>> >>> legacy-tests/VirtualBrowser.py >>> >>> >>> and >>> >>> >>> legacy-tests/Javascript.py >>> >>> >>> >>> >>> >>> You may not have time to get that far for a while, but I thought >>> you'd >>> >>> >>> want to know where my thoughts were headed. ;-) >>> >>> >>> >>> >>> >>> Karl >>> >>> >>> >>> >>> >>> On Thu, Jul 28, 2011 at 2:03 PM, Piergiorgio Lucidi >>> >>> >>> <[email protected]> wrote: >>> >>> >>> > I'm starting to think a way to implement tests for the CMIS >>> >>> Connector, >>> >>> >>> > probably I can use the Apache Chemistry (OpenCMIS) local binding >>> to >>> >>> >>> create a >>> >>> >>> > mock of a CMIS Server (an InMemory repository), but I have to >>> check >>> >>> this >>> >>> >>> > possibility because I'm not sure that all the features are now >>> >>> >>> implemented. >>> >>> >>> > >>> >>> >>> > Anyway I saw some test classes for the filesystem connector at >>> the >>> >>> >>> following >>> >>> >>> > package: >>> >>> >>> > >>> >>> >>> > tests/filesystem/src/test/java >>> >>> >>> > >>> >>> >>> > Could you please confirm that these classes could be taken as an >>> >>> example >>> >>> >>> for >>> >>> >>> > tests? >>> >>> >>> > Or do I need to take a look at other classes? >>> >>> >>> > >>> >>> >>> > Thank you. >>> >>> >>> > Piergiorgio >>> >>> >>> > >>> >>> >>> > -- >>> >>> >>> > Piergiorgio Lucidi >>> >>> >>> > Web: http://about.me/piergiorgiolucidi >>> >>> >>> > >>> >>> >>> >>> >>> >> >>> >>> >> >>> >>> >> >>> >>> >> -- >>> >>> >> Piergiorgio Lucidi >>> >>> >> Web: http://about.me/piergiorgiolucidi >>> >>> >> >>> >>> >> >>> >>> > >>> >>> > >>> >>> > -- >>> >>> > Piergiorgio Lucidi >>> >>> > Web: http://about.me/piergiorgiolucidi >>> >>> > >>> >>> >>> >> >>> >> >>> >> >>> >> -- >>> >> Piergiorgio Lucidi >>> >> Web: http://about.me/piergiorgiolucidi >>> >> >>> > >>> >> >> >> >> -- >> Piergiorgio Lucidi >> Web: http://about.me/piergiorgiolucidi >> >> > > > -- > Piergiorgio Lucidi > Web: http://about.me/piergiorgiolucidi >
