Re: [Neo4j] OSGi and Neo4j - superbundle approach
Mmh, not really an idea so far. The steps you describe look OK. Try to build and install the Neo4j bundle alone: cd neo4j-osgi/bundle mvn clean install Then try out this project: https://github.com/jri/neo4j-osgi-test It provisions a standalone Felix runtime (with the aid of Pax Runner). There I also describe the steps that worked for me (seemingly the same as you tried :-| Cheers, Jörg On Jul 26, 2011, at 23:37, Jean-Pierre Bergamin wrote: Helly everyone I'm currently also stuck with the No index provider 'lucene' found error that you all seem to have been able to work around. I have built and installed the 1.5-SNAPSHOT kernel from the bundle-friendly-classloader branch and installed it locally (mvn install). When I run the neo4j-osgi tests I still get the mentioned lucene error: ... [FelixDispatchQueue] DEBUG org.neo4j.examples.osgi.test-bundle - BundleEvent STOPPED ERROR: Bundle org.neo4j.examples.osgi.test-bundle [3] Error starting file:bundles/org.neo4j.examples.osgi.test-bundle_0.1.0.SNAPSHOT.jar (org.osgi.framework.BundleException: Activator start error in bundle org.neo4j.examples.osgi.test-bundle [3].) java.lang.IllegalArgumentException: No index provider 'lucene' found at org.neo4j.kernel.IndexManagerImpl.getIndexProvider(IndexManagerImpl.java:76) ... Failed tests: neo4jStartupTest(org.neo4j.examples.osgi.OSGiTest) bundleStartupTest(org.neo4j.examples.osgi.OSGiTest) Is there anything else I have to to so that I get the examples up and running? Best regards, James ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] OSGi and Neo4j - superbundle approach
Hi Peter,
SUCCESS!!!
No index provider 'lucene' found in OSGi is solved for the moment!
Its exactly as you suggest:
When using pax exam's native container no error shows up.
When using pax exam's pax runner container the error shows up (which is correct
OSGi behavoir).
When using bundle-friendly-classloader branch of neo 1.5-SNAPSHOT kernel the
error disappears!
My neo4j-osgi-test project runs successful now!
I'm looking forward to have a patch like yours in the next neo4j stable.
Thank you and Toni very much for investigation.
Cheers,
Jörg
On Jul 24, 2011, at 22:08, Peter Neubauer wrote:
Jörg,
there where several issues Toni and me found when looking at your
example. Thanks a lot!
First of all, Pax Exam Player using the built-in native OSGi Runner is
not emulating the real platform classloading correctly. Toni is at
this one, the tests are switched to the PaxRunner instead, running in
its own process.
Secondly, the Neo4j Service.java that is loading the classes and
resources is not providing a classloader, which in an OSGi bundle
defaults to a classloader that does not find the interesting classes
and services. With
private static T IterableT java6Loader( ClassT type )
{
try
{
@SuppressWarnings( unchecked ) IterableT result = (IterableT)
Class.forName( java.util.ServiceLoader )
.getMethod( load, Class.class, ClassLoader.class )
.invoke( null, type, Service.class.getClassLoader() );
return filterExceptions( result );
}
catch ( Exception e )
{
return null;
}
catch ( LinkageError e )
{
return null;
}
}
Everything works, but we need to discuss more what classloader to
chose in what contexts, as this might not be great in all
circumstances.
Anyway, you can build the fixed kernel project from
https://github.com/neo4j/community/tree/bundle-friendly-classloader,
then build the https://github.com/neo4j/neo4j-osgi and you should be
able to start up your test project (see my pull request).
Let me know if that works!
/peter
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] OSGi and Neo4j - superbundle approach
Peter,
no problem, take your time.
I see your super bundle test running OK in pax exam, but with an independant
deployment scenario No index provider 'lucene' found is a persisting error.
Meanwhile I tried the approach you suggested: having the neo-super-bundle and
my activator as separate bundles. No success.
And even with your official super-bundle (neo4j/neo4j-osgi/bundle): same
error.
To demonstrate the problem I refactored the test project to utilize your
super-bundle:
https://github.com/jri/neo4j-osgi-test
To run it:
git clone git://github.com/neo4j/neo4j-osgi.git
cd neo4j-osgi
mvn clean install
cd ..
git clone git://github.com/jri/neo4j-osgi-test.git
cd neo4j-osgi-test
mvn clean install pax:run
The test is very small. Just your super-bundle as dependency:
dependency
groupIdorg.neo4j/groupId
artifactIdneo4j-osgi-bundle/artifactId
version0.1-SNAPSHOT/version
scopeprovided/scope
/dependency
And my activator (in package de.deepamehta.neo4j):
public void start(BundleContext context) {
GraphDatabaseService neo4j = new
EmbeddedGraphDatabase(test-db);
IndexNode index = neo4j.index().forNodes(exact);
}
Along with straight-forward bundling instructions for the activator:
configuration
instructions
Export-Packagede.deepamehta.neo4j/Export-Package
Bundle-Activatorde.deepamehta.neo4j.Activator/Bundle-Activator
/instructions
/configuration
I'm eager for knowledge.
Have a nice travel :)
Cheers,
Jörg
On Jul 23, 2011, at 8:11, Peter Neubauer wrote:
Jörg,
I can see your problem in the project. It seems the bundle is inlining
all the classes, AND adding the dependencies as JARs.
I have no time to investigate further because of travel for the next
day, but it might be that the Activator being in the bundle with the
libraries is messing up the build. Could you try the other approach
like in neo4j-osgi - having a library bundle, and your activator in a
separate one?
Cheers,
/peter neubauer
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] OSGi and Neo4j - superbundle approach
I'm seriously trying the superbundle approach and can't get rid of No index
provider 'lucene' found.
The tests at https://github.com/neo4j/neo4j-osgi run successful but I have
serious problems to transfer them to my application (I'm using Felix 3.2.1 on a
Mac).
Just to check my understanding: the No index provider 'lucene' found problem
is with fine granular bundling, OSGi classloaders, and Java's service loader.
One way to solve it is the superbundle approach: putting the user application's
activator and all required neo and lucene bundles/libs into a single bundle.
Correct?
So I took neo4j-osgi/bundle/pom.xml as an template for my application's pom,
especially its bundle plugin configuration:
Import-Package!javax.transaction.*,!com.sun.*,!sun.misc.*,!org.junit.*,!junit.*,*;/Import-Package
Export-Packageorg.neo4j.*;version=1.4,*/Export-Package
Embed-Dependency*;scope=compile|runtime/Embed-Dependency
Embed-Transitivetrue/Embed-Transitive
and added my application's activator:
Bundle-Activator de.deepamehta.neo4j.Activator/Bundle-Activator
The activator's start() method is fairly simple:
public void start(BundleContext context) {
GraphDatabaseService neo4j = new
EmbeddedGraphDatabase(test-db);
IndexNode index = neo4j.index().forNodes(exact);
}
But it throws: java.lang.IllegalArgumentException: No index provider 'lucene'
found
The bundle list looks fine (#1 is the super bundle):
ID|State |Level|Name
0|Active |0|System Bundle (3.2.1)
1|Resolved |5|Neo4j OSGi Test (0.1.0)
2|Active |1|Apache Felix Gogo Command (0.8.0)
3|Active |1|Apache Felix Gogo Runtime (0.8.0)
4|Active |1|Apache Felix Gogo Shell (0.8.0)
To let you reproduce this I packed a minimal test project here:
https://github.com/jri/neo4j-osgi-test
Run it:
git clone git://github.com/jri/neo4j-osgi-test.git
cd neo4j-osgi-test
mvn clean install pax:run
This uses pax runner to provision the super bundle in a Felix runtime.
Deploying the super bundle manually in Felix makes no difference.
Using the latest Maven bundle plugin (2.3.5 instead of 1.4) makes no difference
(after tweaking the pom a little) and also getting rid of the bundle plugin's
warnings (again by tweaking the pom) makes no difference.
Who can help?
Cheers,
Jörg
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] OSGi and Neo4j - superbundle approach
On Jul 23, 2011, at 1:13, Peter Neubauer wrote: Will try to check it out tomorrow. Now sleep. Thanks for the test! Sure. Just one addition for tomorrow: the only neo-bundles my application need are kernel and lucene-index. So I did add just these 2 to the super bundle's dependencies instead the neo4j meta package. Good night! Jörg ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j 1.4 / Lucene 3.1 / OSGi - No index provider 'lucene' found
Thanks for replies! Yes, the super bundle approach (putting *all* neo modules and *all* their dependencies in one bundle) works in principle. But to be a deployable solution ... 1) it must exist in the Maven repo and be on a par with the Neo4j release 2) A configuration mechanism is required to choose the modules actually needed To abandon the super bundle approach and as long as the original problem (Java Service Loader and OSGi) is not solved: how about bringing back the old-style (serviceloader-less) indexing API as an *alternate* entry point? This would allow thousands of OSGi users to use Neo4j 1.2 and they would love it. Cheers, Jörg On Jul 20, 2011, at 9:20, Peter Neubauer wrote: Could you please try the OSGi neo4j super bundle built in github / neo4j / Neo4j - osgi / bundle? Fine grained bundle deployment interferes with the java service loader which only looks in its own classpath. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j 1.4 / Lucene 3.1 / OSGi - No index provider 'lucene' found
Hi Peter, thanks for reply! Is there any chance to get the old index API back, as an *alternative* to the new one? Within an OSGi environment the old index API worked perfectly in Neo4j 1.0, 1.1, and 1.2. In the latter ones the old index API friendly coexists with the new (serviceloader-based) index API but starting with Neo 1.3 the old index API was dropped. With Neo4j 1.2 I just have the neo-kernel and neo-index bundles deployed in Felix along with the Lucene bundle and it works perfectly. It is perfectly OSGi-like and it perfectly fits all the Maven workflow (no publishing of custom-build Neo bundles to a private Maven repo required). From my point of knowledge, Java's service-loader is just not applicable in the context of OSGi. It seems to be a lack in the OSGi Spec. On the Java-side there is ServiceLoader.load(Class, ClassLoader) but a Bundle neither tells you its ClassLoader nor provides a loadService(Class) method. So, my question is, why not returning to a working solution (the old index API) to support the OSGi users *now* and let it friendly coexist with the new index API? Regarding the super bundle approach: From my point of view this is the wrong way as it renders the OSGi idea pointless and complicates the Maven workflow without reason. Anyway, I will give the super bundle approach a try. Please help me with one question: how is my OSGi application supposed to be bundled? 1) *one* single bundle that contains my application (in particular its Activator) *and* the required Neo and Lucene bundles? or 2) *two* bundles: one with my application and another one with Neo and Lucene? Cheers, Jörg On Jul 20, 2011, at 19:26, Peter Neubauer wrote: Hi Jörg, I understand your view. However, as there is no support for scopes in OSGi, the reasonable approach would be to import ALL dependencies for all Neo4j code. That would require quite substancial numbers of bundles for e.g. the enterprise edition that pollute the Bundle Space for the whole system. Also, it seems every OSGi setup has its own granularity level. E.g. some want to use their own Lucene bundle, some installations prefer the Neo4j ones, some like JTA as its own bundle, etc etc. So for the time being I thought a template default setup would be a good start, as refactoring the Java Service Loader to be fully OSGi aware seems very big to get to a stable state across all frameworks and systems. The template is just one pom.xml at https://github.com/neo4j/neo4j-osgi/blob/master/bundle/pom.xml and the resulting default bundle could easily be published. Do you think fine grained atomic bundles instead of a sueprbundle template that is adaptable would be really generic? Secondly, I think the request for an injection-based initialization method for index providers sounds good. Could you add a lighthouse ticket at http://neo4jdb.lighthouseapp.com/projects/77609-neo4j-community/tickets or add you requests to http://neo4jdb.lighthouseapp.com/projects/77609/tickets/3-osgi-environments-not-starting-lucene ? Cheers, /peter neubauer ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Neo4j 1.4 / Lucene 3.1 / OSGi - No index provider 'lucene' found
Neo4j 1.4 / Lucene 3.1 / OSGi = java.lang.IllegalArgumentException: No index provider 'lucene' found I'm running in Felix. All Dependencies are resolved (so version 0.0.0 should not be the problem here). 1|Active |5|wrap_mvn_org.apache.lucene_lucene-core_3.1.0 (0.0.0) 13|Active |5|Neo4j - Graph Database Kernel (1.4.0) 14|Active |5|Neo4j - Lucene Index (1.4.0) I understood Neo4j expects Lucene 3.1. So I wrapped Lucene via pax runner. The latest servicemix Lucene bundle available is 3.0.3 and brings up the same Exception. With the old Index API in Neo4j 1.1 and 1.2 (IndexService) the problem never occured when running in OSGi. But always with the new service-provider based Index API of Neo4j 1.3 and 1.4 when running in OSGi. Two month ago Marco Gerber started the Thread No index provider 'lucene' found - osgi (this time still for Neo4j 1.3) and found out: On May 24, 2011, at 8:53, Marco Gerber wrote: I solved the problem after having a look on the neo4j source code. The code uses java services to load any extensions on runtime. One of them is defined in META-INF/services/org.neo4j.graphdb.index.IndexProvider which is 'org.neo4j.index.impl.lucene.LuceneIndexProvider' (in my opinion the default index provider). However, the package containing this class isn't getting exported with the bundle. So, the class isn't visibible at runtime and no lucene index provider is getting loaded. His solution (tweaking the MANIFEST of the neo4j-lucene-index jar) is no durable solution and Peter Neubauer's approach (moving the LuceneIndexProvider into the public space, where it is now in Neo4j 1.4) looks like a good one but apparently did not solve the problem. I would really love to use Neo4j 1.4 with Lucene indexing while running in OSGi. Cheers, Jörg ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j 1.1 released -- meta-model component is missing for OSGi
It works now! Great! Neo4j 1.1 + meta-model runs together in OSGi environment. Thank you for quick action! Cheers, Jörg On Aug 4, 2010, at 10:00, Peter Neubauer wrote: Jörg, the neo4j-meta-model is not following the core components release cycle, so it will be released separately when it is tested and stabilized. However, I have bumped the versions of the dependencies to 1.1, which mean you should be fine in OSGi using it. Could you please report back if that works for you? The new snapshot should be deployed in a few minutes ... Cheers, /peter neubauer COO and Sales, Neo Technology ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j 1.1 released -- meta-model component is missing for OSGi
Now, with final 1.1 util component, dependencies resolve properly in an OSGi environment. Still missing is the meta-model component. It is at 0.9-SNAPSHOT and declares a dependency of Neo4j 1.1-SNAPSHOT: Import-Package: org.neo4j.graphdb;version=1.1.0.SNAPSHOT. So, Neo4j 1.1 can't yet be deployed in an OSGi environment if the meta-model component is needed: org.osgi.framework.BundleException: Unresolved constraint in bundle org.neo4j.neo4j-meta-model [22]: Unable to resolve 22.0: missing requirement [22.0] package; ((package=org.neo4j.graphdb)(version=1.1.0.SNAPSHOT)) - [22.0] package; ((package=org.neo4j.graphdb)(version=1.1.0.SNAPSHOT)) In OSGi terms, a 1.1 version is apparently not greater than a 1.1.0.SNAPSHOT version. I would love to upgrade to Neo4j 1.1 while using the meta-model component (in an OSGi environment). A new version of meta-model component with updated dependency would do. Congratulations to Neo4j 1.1! Cheers, Jörg On Aug 3, 2010, at 15:02, Anders Nawroth wrote: Hi! If case anyone has missed the neo4j-utils component version 1.1, it has now been deployed to our Maven2 repo: http://m2.neo4j.org/org/neo4j/neo4j-utils/1.1/ /anders On 07/30/2010 04:35 PM, Peter Neubauer wrote: Gentlemen, Neo4j 1.1 has arrived - http://bit.ly/neo4j11 (changelog at http://dist.neo4j.org/CHANGES.txt)! Change your Maven pom.xml and rejoice :) Happy hacking, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Meta-model component: problem with renaming classes
Great! I've tested the 2 new methods. rename() works as expected. With remove() there is an issue: once a class is removed, getMetaClass(name, false) returns a stale MetaModelClass object if -- and only if -- that class has been renamed before. It is expected to return null. If the class has not been renamed before remove() works OK. Example: // create class namespace.getMetaClass(My Class, true); // rename class namespace.rename(My Class, New Class); // remove renamed class namespace.remove(New Class, true); // ERROR: this call returns a stale MetaModelClass object. It is expected to return null. namespace.getMetaClass(New Class, false); Thank you very much for the quick action :-) Cheers, Jörg On Jul 6, 2010, at 2:43, Niels Hoogeveen wrote: The modifications have been applied to trunk. Niels Date: Mon, 5 Jul 2010 20:27:49 +0200 From: matt...@neotechnology.com To: user@lists.neo4j.org Subject: Re: [Neo4j] Meta-model component: problem with renaming classes 2010/7/5 Niels Hoogeveen pd_aficion...@hotmail.com If Mattias agrees, I suggest adding the methods: MetaModelNamespace#rename(String oldName, String newName) MetaModelNamespace#remove(String name, Boolean forced) If forced is false, throw exception when relations exist to other MetaModelObjects or in case of a MetaModelClass, has associated instances. If forced is true, all relationships are removed together with the MetaModelObject. I am willing to make the changes, but want approval from the Neo4J team before making these adjustments. Niels ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Meta-model component: problem with renaming classes
On Jul 5, 2010, at 8:57, Mattias Persson wrote: Maybe there's some bug in the removeIndex(Node,String) method. Instead try to do a: index.removeIndex(classNode1,MetaModelProperty.KEY_NAME,classNode1.getName()); before you set the new name I've tested this already. Indexing works OK. The problem is in namespace.getMetaClass() What is your use case for renaming a meta model class? Is that really functionality that should be used at runtime? My gut feeling says that this is a develpment time thing and once a class is created with a name it sticks with that name. Or are you doing some kind of migration of your data? Renaming a class is required when building a graphical modelling tool. That is, the user creates classes and properties and he want to rename them. This happens at runtime. I'm currently working on a rewrite of the DeepaMehta open source project. DeepaMehta is a platform for collaboration and knowledge management. In existence since 10 years, 6 years in production. Part of DeepaMehta is a modelling feature. For the DeepaMehta rewrite I switched from MySQL to CouchDB, and now to Neo4j. Neo4j fits the DeepaMehta model very well and Neo4j storage runs smooth since 2 months. Now, while working on the modelling feature I hit the wall at meta-model component. I would love to release DeepaMehta 3 v0.4 (the first version based on Neo4j) soon. Is there a reason to not have a setName() method on MetaModelObject? Cheers Jörg ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Meta-model component: problem with renaming classes
On Jul 5, 2010, at 9:13, Niels Hoogeveen wrote: MetaModelClasses are cached in a map, so while the property may be renamed and reindexed, the map still holds the original name value. After a restart of the application the new value should be applied. To make renaming possible a method needs to be added to MetaModelNamespace. Niels Thank you for the info! Restarting the application is not an option in dynamic environments, where classes come and go (and change their names) at runtime. BTW: deleting a class is also an issue with the meta-model component. Is it possible for you to add a setName() method to MetaModelObject (or elsewhere)? This would allow developers to use the meta-model component in dynamic applications. Cheers Jörg ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Meta-model component: problem with renaming classes
Thank you! Highly appreciating that :-) I'm looking forward ... Cheers Jörg On Jul 5, 2010, at 17:54, Niels Hoogeveen wrote: If Mattias agrees, I suggest adding the methods: MetaModelNamespace#rename(String oldName, String newName) MetaModelNamespace#remove(String name, Boolean forced) If forced is false, throw exception when relations exist to other MetaModelObjects or in case of a MetaModelClass, has associated instances. If forced is true, all relationships are removed together with the MetaModelObject. I am willing to make the changes, but want approval from the Neo4J team before making these adjustments. Niels From: j...@deepamehta.de Date: Mon, 5 Jul 2010 13:54:10 +0200 To: user@lists.neo4j.org Subject: Re: [Neo4j] Meta-model component: problem with renaming classes On Jul 5, 2010, at 9:13, Niels Hoogeveen wrote: MetaModelClasses are cached in a map, so while the property may be renamed and reindexed, the map still holds the original name value. After a restart of the application the new value should be applied. To make renaming possible a method needs to be added to MetaModelNamespace. Niels Thank you for the info! Restarting the application is not an option in dynamic environments, where classes come and go (and change their names) at runtime. BTW: deleting a class is also an issue with the meta-model component. Is it possible for you to add a setName() method to MetaModelObject (or elsewhere)? This would allow developers to use the meta-model component in dynamic applications. Cheers Jörg ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Meta-model component: problem with renaming classes
How can a MetaModelClass be renamed? There seems to be no obvious way. I tried to change the meta_model_name property of the wrapped node and re-index the node. This basically works, but namespace.getMetaClass() still returns a MetaModelClass object when called with the old name. IMO it would be correct if null is returned in this case, like for any unknown class. Otherwise the application could get confused. Example code: // creating a class and getting the wrapped node MetaModelClass class1 = namespace.getMetaClass(My Class, true); Node classNode1 = class1.node(); // change the node's name property and re-index classNode1.setProperty(MetaModelProperty.KEY_NAME, New Class Name); index.removeIndex(classNode1, MetaModelProperty.KEY_NAME); index.index(classNode1, MetaModelProperty.KEY_NAME, New Class Name); // both calls return a class, the second one shouldn't namespace.getMetaClass(New Class Name, false); namespace.getMetaClass(My Class, false); Tested with latest snapshots of neo4j kernel, index, and meta-model components and Lucene 2.9.2 API proposal: Class MetaModelObject should have a setName(String) method for easy and consistent renaming of classes and properties. Furthermore, namespace.getMetaClass() should return null when called with a stale name. BTW: one big up to the great Neo4j team and community!! Cheers, Jörg ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
