Exactly. You would/could create a "chicken and egg" problem if the service dependencies needed to be known prior to a recovery or startup options, and since you can't start the db, you couldn't read the metadata.
The "separate file" approach (or even a "mini db" that used Neo constructs but did not have any other dependencies) would work fine. -------- Original Message -------- Subject: Re: [Neo] Persisting store metadata From: Peter Neubauer <[email protected]> Date: Tue, January 05, 2010 5:48 am To: Neo user discussions <[email protected]> Hi there, my only concern with storing metadata in the db itself is the issue of having no way to discover the version of Neo4j needed to open the db, and the inability to get that metainf out with anything but Neo4j itself. I am thinking of things like maven processing to programmatically add info there etc etc, if that may be of interest later. It depends of course on how big this should be and where in the SW stack the meta info comes in. Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn [1]http://www.linkedin.com/in/neubauer Twitter [2]http://twitter.com/peterneubauer [3]http://www.neo4j.org - Relationships count. [4]http://gremlin.tinkerpop.com - PageRank in 2 lines of code. [5]http://www.linkedprocess.org - Computing at LinkedData scale. On Tue, Jan 5, 2010 at 1:07 PM, Mattias Persson <[email protected]> wrote: > 2010/1/5 Tobias Ivarsson <[email protected]>: >> That's an interesting idea. However something in my gut tells me that it's >> not good/better: >> * It's still open for people to mock about with, however now it's in the >> node space using up a relationship type that is now unusable by the >> application (at least as a relationship from the reference node). >> * This is intended (at least by how I envision it) to be read by the kernel >> to know which components to load in order to have a consistent store, it >> seems to me like having the data in the node space would require too much >> bootstrapping before it can be accessed. Remember that this information is >> meant to be part of the bootstrapping process. > Yep I agree that it'd probably require too much bootstrapping to reach > that information. >> >> It does have the advantage that the consistency of the information would be >> guaranteed. > The upside is, as you say, that you get all these nice things such as > consistency for free... as well as that it automatically tags along in > online backup and all such things, which is quite nice. >> >> /Tobias >> >> On Tue, Jan 5, 2010 at 9:34 AM, Mattias Persson >> <[email protected]>wrote: >> >>> Great initiative. >>> >>> Instead of storing such information in a plain-text file it could be >>> stored in the neo4j store itself (as a sub-reference node or >>> something). >>> >>> I don't know if that'd be good/better though. >>> >>> 2010/1/4 Tobias Ivarsson <[email protected]>: >>> > Hi, >>> > >>> > Neo4j today is made up of a core service that persists nodes, >>> relationships >>> > and properties. In addition to that there are a number of additional >>> > services, where index-util is probably the most used and most important >>> at >>> > the moment. index-util is also a good example of the problem I would like >>> to >>> > discuss in this email. >>> > >>> > With the current architecture there is no way of introspecting which >>> > additional services have been initialized with a particular Neo store, >>> for >>> > example there is no way of telling which (if any) IndexService has been >>> > used. This causes problems with the transaction recovery process, where >>> the >>> > recovery mechanism today has to know about all possible extension >>> services >>> > without having a compile time dependency on them. Needless to say this >>> > causes a mess. The code for this is ugly and borderline-buggy. It also >>> > causes a problem when introspecting a store, or even restarting your >>> > software, since it is up to the programmer to remember to restart all of >>> the >>> > same services as last time. >>> > >>> > I would like to propose an addition to the store, in a separate metadata >>> > file in the store directory, where we store a simple list of all started >>> > additional services. Does anyone have any suggestions to what this should >>> > look like to be reasonable future proof (i.e. be able to handle some >>> service >>> > that is not implemented yet as well as the current index-utils). >>> > >>> > What comes to mind is something similar to the Java ServiceLoader API >>> [1], >>> > but simplified with the fact that we can require all classes referenced >>> in >>> > the file to implement one specific interface, and we know where the file >>> > will be, in effect taking out all the complications involving >>> ClassLoaders. >>> > What I'm worrying about is that storing Java classnames in a file will >>> make >>> > this metadata unusable to any future implementations of the NeoStore >>> format >>> > outside of the Java platform. Any suggestions to how information about >>> > loaded services can be stored is welcome, my idea of a clear text file >>> might >>> > not even be good (since people have a tendency to think they can patch >>> those >>> > manually). >>> > >>> > Since we have a community of smart and entrepreneurial individuals I >>> thought >>> > I'd throw the question out here and see if I got any good responses. >>> > >>> > Cheers, >>> > Tobias >>> > >>> > [1] [6]http://java.sun.com/javase/6/docs/api/java/util/ServiceLoader.html >>> > Simple explanation of how it works: >>> > * You have an interface with fully qualified classname: >>> > com.somedomain.someproduct.ServiceInterface >>> > * In each jar that provides implementations of the ServiceInterface you >>> add >>> > a file called >>> > META-INF/services/com.somedomain.someproduct.ServiceInterface >>> > * In this file you put the fully qualified classnames of each >>> implementing >>> > class separated by newline >>> > * In your code you do: ServiceLoader<ServiceInterface> impls = >>> > ServiceLoader.load(ServiceInterface.class); >>> > -- >>> > Tobias Ivarsson <[email protected]> >>> > Hacker, Neo Technology >>> > [7]www.neotechnology.com >>> > Cellphone: +46 706 534857 >>> > _______________________________________________ >>> > Neo mailing list >>> > [email protected] >>> > [8]https://lists.neo4j.org/mailman/listinfo/user >>> > >>> >>> >>> >>> -- >>> Mattias Persson, [[email protected]] >>> Neo Technology, [9]www.neotechnology.com >>> _______________________________________________ >>> Neo mailing list >>> [email protected] >>> [10]https://lists.neo4j.org/mailman/listinfo/user >>> >> >> >> >> -- >> Tobias Ivarsson <[email protected]> >> Hacker, Neo Technology >> [11]www.neotechnology.com >> Cellphone: +46 706 534857 >> _______________________________________________ >> Neo mailing list >> [email protected] >> [12]https://lists.neo4j.org/mailman/listinfo/user >> > > > > -- > Mattias Persson, [[email protected]] > Neo Technology, [13]www.neotechnology.com > _______________________________________________ > Neo mailing list > [email protected] > [14]https://lists.neo4j.org/mailman/listinfo/user > _______________________________________________ Neo mailing list [email protected] [15]https://lists.neo4j.org/mailman/listinfo/user References 1. http://www.linkedin.com/in/neubauer 2. http://twitter.com/peterneubauer 3. http://www.neo4j.org/ 4. http://gremlin.tinkerpop.com/ 5. http://www.linkedprocess.org/ 6. http://java.sun.com/javase/6/docs/api/java/util/ServiceLoader.html 7. http://www.neotechnology.com/ 8. https://lists.neo4j.org/mailman/listinfo/user 9. http://www.neotechnology.com/ 10. https://lists.neo4j.org/mailman/listinfo/user 11. http://www.neotechnology.com/ 12. https://lists.neo4j.org/mailman/listinfo/user 13. http://www.neotechnology.com/ 14. https://lists.neo4j.org/mailman/listinfo/user 15. https://lists.neo4j.org/mailman/listinfo/user _______________________________________________ Neo mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
