Re: [Neo4j] indexquery 12times faster than cypher -- neostore-upgrade-question
Hi, this is great, I was still holding my breath and at first didnt even
notice, then I got the snapshot and changed classpaths in my project to
use 1.5.m, when I noticed the following error
org.neo4j.graphdb.TransactionFailureException: Could not create data
source [nioneodb], see nested exception for cause of error
org.neo4j.kernel.impl.transaction.TxModule.registerDataSource(TxModule.java:153)
org.neo4j.kernel.GraphDbInstance.start(GraphDbInstance.java:112)
at
org.neo4j.kernel.EmbeddedGraphDbImpl.init(EmbeddedGraphDbImpl.java:190)
at
org.neo4j.kernel.EmbeddedGraphDatabase.init(EmbeddedGraphDatabase.java:80)
...
Caused by:
org.neo4j.kernel.impl.storemigration.UpgradeNotAllowedByConfigurationException:
To enable automatic upgrade, please set allow_store_upgrade in
configuration properties
at
org.neo4j.kernel.impl.storemigration.ConfigMapUpgradeConfiguration.checkConfigurationAllowsAutomaticUpgrade(ConfigMapUpgradeConfiguration.java:41)
at
org.neo4j.kernel.impl.storemigration.StoreUpgrader.attemptUpgrade(StoreUpgrader.java:50)
at
org.neo4j.kernel.impl.nioneo.store.NeoStore.tryToUpgradeStores(NeoStore.java:127)
at
org.neo4j.kernel.impl.nioneo.store.NeoStore.verifyCorrectTypeDescriptorAndVersion(NeoStore.java:98)
at
org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.loadStorage(CommonAbstractStore.java:196)
at
org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.init(CommonAbstractStore.java:104)
at
org.neo4j.kernel.impl.nioneo.store.AbstractStore.init(AbstractStore.java:120)
at org.neo4j.kernel.impl.nioneo.store.NeoStore.init(NeoStore.java:72)
at
org.neo4j.kernel.impl.nioneo.xa.NeoStoreXaDataSource.init(NeoStoreXaDataSource.java:137)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.neo4j.kernel.impl.transaction.XaDataSourceManager.create(XaDataSourceManager.java:75)
at
org.neo4j.kernel.impl.transaction.TxModule.registerDataSource(TxModule.java:147)
... 24 more
which tells me to allow the neo-store to be updated. That spooked me so
much that I didnt yet try it in earnest, focusing instead on rewriting
my app to work without thread-interrupts because that was apparently
messing with lucene, and trying to find the reason why the
worker-threads' transactions always keep deadlocking... as much as I
desired that cypher-feature with the RelationshipIndexLookup, there are
work-arounds, but I'll get around to testing it sometime soon.
Before I do upgrade the store,
39 MB neostore.nodestore.db
412 MB neostore.propertystore.db
4315 MB neostore.propertystore.db.arrays
1857 MB neostore.propertystore.db.strings
1090 MB neostore.relationshipstore.db
plus 1.6GB of indices,
any idea how long such a neostore-upgrade might take from 1.4.1. to that
1.5.snapshot? like seconds, minutes, hours, or worse?
Am 02.10.2011 19:25, schrieb Andres Taylor:
And now it's ready.
If you are using SNAPSHOT, please update your dependencies, and check it
out.
http://docs.neo4j.org/chunked/snapshot/query-start.html#start-relationship-by-index-lookup
Thanks again for providing feedback. It's much more fun coding when I know
someone is using it.
Andrés
On Fri, Sep 30, 2011 at 10:37 AM, Andres Taylor
andres.tay...@neotechnology.com wrote:
Hey!
This is something that has been planned since the very first iteration of
Cypher, but no-one seemed to need it, so I have pushed it in front of me.
Now we seem to have an user that wants it, so I'll add it.
Don't hold your breath though - it'll be ready when it's ready.
Andrés
On Thu, Sep 29, 2011 at 1:10 PM, st.past...@web.de wrote:
hi.
when i use a cypher query to get the queued nodes associated via
queue-relationships to a repository-node which has several million other
relationships then it takes about six minutes to fill that result list:
pre
ListNode list = new ArrayListNode();
StringBuffer s = new StringBuffer();
s.append(START repository = ();
s.append((Long) controller.getRepositoryNode().getId());
s.append() MATCH (repository)-[:);
s.append(Rels.REL_REPOSITORY_QUEUE);
s.append(]-(node) RETURN node ORDER BY node.);
s.append(Keys.KEY_TIMESTAMP);
s.append( SKIP );
s.append(Integer.toString(offset));
s.append( LIMIT );
s.append(Integer.toString(buffersize));
Log.log(s.toString());
Query query = this.parser.parse(s.toString());
ExecutionResult result = this.engine.execute(query);
IteratorNode iterator = result.columnAs(node);
while (iterator.hasNext()) {
list.add(iterator.next());
}
/pre
but when i directly query the relationshipIndex containing all
Re: [Neo4j] indexquery 12times faster than cypher -- neostore-upgrade-question
Hi, it is hard to make predictions on how long it will take to migrate a store to 1.5. What the migration time actually depends on is node and relationship population, number of properties and number of dynamic records in use. This correlates to file size but one cannot determine the time based on just that, since not in use records still take up space but do not contribute to the migration time. Also indices are not touched, so they have nothing to do with upgrade time. Based on our runs, a store of the size you mention should take in the order of 1-2 hours to upgrade, depending of course on OS, filesystem, hardware etc. As far as the safety of your data goes: It is highly recommended to do a backup of your data in a safe location before starting the upgrade process and after shutting down cleanly the neo4j database. However, we have taken all the measures we could to make sure that the process is as safe as possible and we also keep a copy of the existing database files under database_dir/upgrade_backup just to be extra cautious. In the unlikely case anything goes south, you can always fix the problem, restore the old database and try the upgrade again. hope that helps, CG On Wed, Oct 12, 2011 at 9:30 AM, st.pa st...@web.de wrote: Hi, this is great, I was still holding my breath and at first didnt even notice, then I got the snapshot and changed classpaths in my project to use 1.5.m, when I noticed the following error org.neo4j.graphdb.TransactionFailureException: Could not create data source [nioneodb], see nested exception for cause of error org.neo4j.kernel.impl.transaction.TxModule.registerDataSource(TxModule.java:153) org.neo4j.kernel.GraphDbInstance.start(GraphDbInstance.java:112) at org.neo4j.kernel.EmbeddedGraphDbImpl.init(EmbeddedGraphDbImpl.java:190) at org.neo4j.kernel.EmbeddedGraphDatabase.init(EmbeddedGraphDatabase.java:80) ... Caused by: org.neo4j.kernel.impl.storemigration.UpgradeNotAllowedByConfigurationException: To enable automatic upgrade, please set allow_store_upgrade in configuration properties at org.neo4j.kernel.impl.storemigration.ConfigMapUpgradeConfiguration.checkConfigurationAllowsAutomaticUpgrade(ConfigMapUpgradeConfiguration.java:41) at org.neo4j.kernel.impl.storemigration.StoreUpgrader.attemptUpgrade(StoreUpgrader.java:50) at org.neo4j.kernel.impl.nioneo.store.NeoStore.tryToUpgradeStores(NeoStore.java:127) at org.neo4j.kernel.impl.nioneo.store.NeoStore.verifyCorrectTypeDescriptorAndVersion(NeoStore.java:98) at org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.loadStorage(CommonAbstractStore.java:196) at org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.init(CommonAbstractStore.java:104) at org.neo4j.kernel.impl.nioneo.store.AbstractStore.init(AbstractStore.java:120) at org.neo4j.kernel.impl.nioneo.store.NeoStore.init(NeoStore.java:72) at org.neo4j.kernel.impl.nioneo.xa.NeoStoreXaDataSource.init(NeoStoreXaDataSource.java:137) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at org.neo4j.kernel.impl.transaction.XaDataSourceManager.create(XaDataSourceManager.java:75) at org.neo4j.kernel.impl.transaction.TxModule.registerDataSource(TxModule.java:147) ... 24 more which tells me to allow the neo-store to be updated. That spooked me so much that I didnt yet try it in earnest, focusing instead on rewriting my app to work without thread-interrupts because that was apparently messing with lucene, and trying to find the reason why the worker-threads' transactions always keep deadlocking... as much as I desired that cypher-feature with the RelationshipIndexLookup, there are work-arounds, but I'll get around to testing it sometime soon. Before I do upgrade the store, 39 MB neostore.nodestore.db 412 MB neostore.propertystore.db 4315 MB neostore.propertystore.db.arrays 1857 MB neostore.propertystore.db.strings 1090 MB neostore.relationshipstore.db plus 1.6GB of indices, any idea how long such a neostore-upgrade might take from 1.4.1. to that 1.5.snapshot? like seconds, minutes, hours, or worse? Am 02.10.2011 19:25, schrieb Andres Taylor: And now it's ready. If you are using SNAPSHOT, please update your dependencies, and check it out. http://docs.neo4j.org/chunked/snapshot/query-start.html#start-relationship-by-index-lookup Thanks again for providing feedback. It's much more fun coding when I know someone is using it. Andrés On Fri, Sep 30, 2011 at 10:37 AM, Andres Taylor andres.tay...@neotechnology.com wrote: Hey! This is something that has been planned since the very
Re: [Neo4j] indexquery 12times faster than cypher -- neostore-upgrade-question
To reiterate what Chris said about backups. The latest release is a milestone, and while we've taken every opportunity to rigorously test it, you should always have a backup of your data before any upgrade especially so if it's not a stable GA release. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Transparent Distributed Queries?
Mmh, I would actually do it the other way round - have a number of replicated Neo4j instances (possibly via Neo4j HA, see http://docs.neo4j.org/chunked/snapshot/ha-setup-tutorial.html) and then push traversals via GoldenOrb onto these instances and merge the results. WDYT? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, Oct 12, 2011 at 7:40 AM, amitp amit.port...@gmail.com wrote: Hey thanks, I've just read goldenOrb wiki, it's seem related BUT if I understand correctly it's helps just with the distribution. I'm looking for a more complete graph package that can be easily distributed (I don't want to implement my own query engine). So I think I'll need to get into the depths of neo4j and see if I can change the implentation to work on GoldenOrb Nodes... Does this make sense? any points before I get lost in the code? Thanks again, Amit -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Transparent-Distributed-Queries-tp3414168p3414876.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Online backups for single instance using community edition
Hi, I am using the neo4j community edition in a PHP webapp that I am currently developing and found it to be a really cool solution for my project. It is a small project and I am just a single guy developing it so I am nowhere near needing HA or sharding capabilities. Here is the problem I see with neo4j in any form of webapp (SaaS): To be able to create an online backup you'll have to use the enterprise edition (=2000 dollars a month) or you are forced to shutdown the service for a short period of time (most likely multiple times per day) which can be an absolute no-go for a webapp. So my question is: Is there another method of doing an online backup for a single neo4j instance? I thought about traversing the whole graph and creating a second graph filled with the result. I could then shutdown the second instance and just backup the files old school style. This sounds super ugly, but maybe it is a way to think about. I am fine with solutions that would require some additional developing. If the only solution is to use the enterprise edition than I would highly suggest some alternative ways for the next neo4j releases as this would prevent a lot of webdevelopers from using the product. It would be nice to have a way to do online backups for a single instance only so the benefits of the advanced and enterprise editions would still exist. Anyways, thanks for your time and answers Greetings from Austria, Hannes -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Online-backups-for-single-instance-using-community-edition-tp3415246p3415246.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Deprecation Warning with Python Neo4j Bindings
Glad you're enjoying it :) The warning seems to be coming from JPype itself, we'll see if they end up pushing an update for it. If it ends up becoming a problem for supporting newer python versions, we could potentially write a patch for JPype.. /Jake On Wed, Oct 12, 2011 at 10:42 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Neo folks, Just wanted to pass along a deprecation warning I'm getting after installing the latest versions of JPype and Neo4j under Python 2.6. from neo4j import GraphDatabase /Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/jpype/_pykeywords.py:18: DeprecationWarning: the sets module is deprecated import sets Stoked to use this for pushing data into Neo4j. Thanks again! Chris Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Mahout, and Migrating from JPA/Hibernate/MySQL to Neo4j?
Hi all, I've heard a lot of hype about Neo4j, including some presentations at Spring conferences. I've got a few questions I was hoping Neo4j vets might be able to help me with: 1. We're using Mahout as a recommendation system. Has anyone had any success plugging Neo4j into this? 2. How does Neo4j distribute in an elastic cloud environment? Our current solution is EHCache+Terracotta. Our main application is based on JPA, Hibernate, and MySQL. We're expecting high levels of load, and also need to be able to do many operations like the equivalent of SELECT COUNT(id) FROM table WHERE score 12345. I'm fed up of battling with the relational model and trying to map to it, and was wondering if Neo4j would allow me to just get on with POJOs and worry less about databases. Is this a fair usage scenario, or should I be looking to Neo4j for much more specific usages? -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Mahout-and-Migrating-from-JPA-Hibernate-MySQL-to-Neo4j-tp3415889p3415889.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Mahout, and Migrating from JPA/Hibernate/MySQL to Neo4j?
Hi Deejay, 1. We're using Mahout as a recommendation system. Has anyone had any success plugging Neo4j into this? I work with various companies using graph databases to do recommendation. Moreover, a couple of them are also experimenting with Mahout over MySQL and Hadoop. I have never thought to think about backing Mahout by a graph database. That is an interesting idea... Here are three thoughts that are related (though do not directly address your question): 1. With Mahout (as I understand it since a few versions back), only single relational data can be processed. That is, it only supports data of the form: X likes[weight] Y, where weight can be binary or rational. When a domain model is sufficiently complex: people liking things, people knowing each other, people working in the same, similar, etc. places, and products having features, designers, etc. there is more information in the domain that can be capitalized on for recommendation. 2. With pure graph-based recommendation, no recommendation model is generated (intermediate data structure) as recommendations are calculated on the fly over the raw graph using traversal techniques. Traversals can propagate over more complex relations and are not limited to X likes[weight] Y or, better yet, such basic relations can be derived through implicit relations (i.e. paths) [ http://markorodriguez.com/2011/02/08/property-graph-algorithms/ ]. Along this line of thought, the raw graph representation of your domain can be used for more than just recommendation --- e.g. path analysis, global ranking, searching, reasoning, abstraction, etc. [ http://markorodriguez.com/2011/07/14/graphs-brains-and-gremlin/ ] 3. With various forms of graph sampling/weighting, it is possible to put as many clock cycles (thus, compute time) as desired into the determination of a recommendation -- generally, more clock cycles yields greater accuracy. However, with accumulative methods, it is possible to reach an ergodic state [ http://en.wikipedia.org/wiki/Ergodicity ] whereby the contribution of more clock cycles does not yield more information (i.e. does not alter the order of the resultant recommendation ranking). While your question was about overlaying Mahout on top of Neo4j, I argue that by using Neo4j in its native form (through its API and its approach to data analysis), there is much more beyond recommendation that you can exploit from your domain model. To conclude, I recently wrote up a post on graph-based recommendation that may be of interest to you: http://markorodriguez.com/2011/09/22/a-graph-based-movie-recommender-engine/ Good luck with your explorations, Marko. http://markorodriguez.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Node index in Neo4j REST server
Thank you , Jim. Cheers, From: Jim Webber j...@neotechnology.com To: Neo4j user discussions user@lists.neo4j.org Sent: Wednesday, October 12, 2011 1:24 AM Subject: Re: [Neo4j] Node index in Neo4j REST server Hi Andrew, I have a question about indexing in Neo4j REST server. For some reason I may need to create multiple node indexes in REST server. Is there any limit number of indexing I am allowed to create? No, there's no logical limit on the number of indexes you can create. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Jobangebot in Zürich/Wallisellen als Java-Entwickler mit Neo4j-Kenntnissen
Liebe Graphistas Ich möchte bekennende Graph-DB-Enthusiasten auf unsere offene Stelle aufmerksam machen: Wir suchen einen Java-Entwickler, der Teil unseres Scrum-Teams werden will. Unsere BSM-Software basiert auf Spring und ist modular mit OSGI (Spring Dynamic Modules) aufgebaut. Als Applikationsserver kommt Virgo zum Einsatz. Die Webapplikation wird mit Spring MVC und JSP/JSTL und mit Dojo als Javascript-Framework umgesetzt. Für die Kommunikation zwischen den Systemen wird JMS mit Camel verwendet. Als Datenbank und sozusagen als Herzstück setzen wir - wie könnte es auch anders sein - neo4j embedded mit Spring Data Neo4j ein. Für die Speicherung von Messdaten (Mrd. von Werten) wird voraussichtlich MongoDB verwendet. Wer sich in einem agilen Team wohlfühlt und bereit ist, sich in einen leading-edge Technologie-Stack einzuarbeiten soll sich melden. Unter http://www.junisphere.net/docs/Software_Entwicklungsingenieur.pdf sind weitere Informationen zum Jobangebot zu finden. Unsere Büros sind im Glattzentrum in Wallisellen (in der Nähe von Zürich). Wir freuen uns auf Bewerbungen von echten Graphistas! :-) Bewerbungen sind an Lindita Eichmann (leichm...@junisphere.net) zu richten. Viele Grüsse Jean-Pierre Bergamin PS: German is a must. This is why this post is written in German... ;-) -- Junisphere Systems AG Glatt Tower P.O. Box 1572 CH-8301 Glattzentrum Phone +41 43 443 31 80 Direct +41 43 443 31 93 jpberga...@junisphere.net www.junisphere.net ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Jobangebot in Zürich/Wallisellen als Java-Entwickler mit Neo4j-Kenntnissen
Also everyone, Rick Bullottas and Emil Eifrems talk at QCon is now online - much along the lines of getting sensory data into a highly dynamic platform. http://www.infoq.com/presentations/Graph-Database-Power-Web-of-Things Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. 2011/10/12 Jean-Pierre Bergamin jpberga...@gmail.com: Liebe Graphistas Ich möchte bekennende Graph-DB-Enthusiasten auf unsere offene Stelle aufmerksam machen: Wir suchen einen Java-Entwickler, der Teil unseres Scrum-Teams werden will. Unsere BSM-Software basiert auf Spring und ist modular mit OSGI (Spring Dynamic Modules) aufgebaut. Als Applikationsserver kommt Virgo zum Einsatz. Die Webapplikation wird mit Spring MVC und JSP/JSTL und mit Dojo als Javascript-Framework umgesetzt. Für die Kommunikation zwischen den Systemen wird JMS mit Camel verwendet. Als Datenbank und sozusagen als Herzstück setzen wir - wie könnte es auch anders sein - neo4j embedded mit Spring Data Neo4j ein. Für die Speicherung von Messdaten (Mrd. von Werten) wird voraussichtlich MongoDB verwendet. Wer sich in einem agilen Team wohlfühlt und bereit ist, sich in einen leading-edge Technologie-Stack einzuarbeiten soll sich melden. Unter http://www.junisphere.net/docs/Software_Entwicklungsingenieur.pdf sind weitere Informationen zum Jobangebot zu finden. Unsere Büros sind im Glattzentrum in Wallisellen (in der Nähe von Zürich). Wir freuen uns auf Bewerbungen von echten Graphistas! :-) Bewerbungen sind an Lindita Eichmann (leichm...@junisphere.net) zu richten. Viele Grüsse Jean-Pierre Bergamin PS: German is a must. This is why this post is written in German... ;-) -- Junisphere Systems AG Glatt Tower P.O. Box 1572 CH-8301 Glattzentrum Phone +41 43 443 31 80 Direct +41 43 443 31 93 jpberga...@junisphere.net www.junisphere.net ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Importing data from oracle to neo4j
Another approach might be to adapt SymmetricDS -- http://symmetricds.codehaus.org/ -- to feed data from your Oracle (or other JDBC accessible Relational database) into Neo4j - live, as the data changes. I've been wanting a SymmetricDS interface to Neo4j for a while. Let me know if you get one working! -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Peter Neubauer Sent: Tuesday, October 11, 2011 6:47 AM To: Neo4j user discussions Subject: Re: [Neo4j] Importing data from oracle to neo4j Hi there, yes, the utility is converting m:n into relationships, see https://github.com/peterneubauer/sql-import and https://github.com/peterneubauer/sql-import/blob/master/src/test/java/com/neo4j/sqlimport/InsertTest.java fro an eample. Let me know if that helps! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Sat, Oct 8, 2011 at 11:14 PM, Michael Hunger michael.hun...@neotechnology.com wrote: What is your use-case for importing the DWH data? How are you going to model the data of the DWH in neo4j. How does the domain-model look like? BI data is normally denormalized. So for creating a good graph structure it would be sensible to normalize it during the import. 10k rows is not that much. You can import them using the normal neo4j transactional facilities in a few seconds. For building up your graph model you probably want to index your data or create category nodes to access certain parts of your domain model. Peter wrote a tool to import relational data into neo4j but that was normalized data where each table was represented by a certain type of node in the graph and foreign key relationships were converted to graph relationships. I don't know if that also handled m:n connection tables efficiently by converting them to relationships too. HTH Michael Am 08.10.2011 um 23:05 schrieb jiteshks: I am very new to neo4j.So I don't know all of its features.I am reading its documentation to understand how it works. My project's requirement is to import the data from a data warehouse( which is an oracle db) once every month.We are thinking of implementing neo4j in our project which means we will have to read the data from oracle db and put it into neo4j. There will be around 1 lakh rows(10^5) to be fetched from oracle db.What is the fast/efficient way of doing it? Thanks! -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Importing-dat a-from-oracle-to-neo4j-tp3406024p3406024.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Importing data from oracle to neo4j
Very interesting - I like! Would love to play around with it. Thanks for the tip Rick! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, Oct 12, 2011 at 10:37 PM, Rick Otten rot...@manta.com wrote: Another approach might be to adapt SymmetricDS -- http://symmetricds.codehaus.org/ -- to feed data from your Oracle (or other JDBC accessible Relational database) into Neo4j - live, as the data changes. I've been wanting a SymmetricDS interface to Neo4j for a while. Let me know if you get one working! -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Peter Neubauer Sent: Tuesday, October 11, 2011 6:47 AM To: Neo4j user discussions Subject: Re: [Neo4j] Importing data from oracle to neo4j Hi there, yes, the utility is converting m:n into relationships, see https://github.com/peterneubauer/sql-import and https://github.com/peterneubauer/sql-import/blob/master/src/test/java/com/neo4j/sqlimport/InsertTest.java fro an eample. Let me know if that helps! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Sat, Oct 8, 2011 at 11:14 PM, Michael Hunger michael.hun...@neotechnology.com wrote: What is your use-case for importing the DWH data? How are you going to model the data of the DWH in neo4j. How does the domain-model look like? BI data is normally denormalized. So for creating a good graph structure it would be sensible to normalize it during the import. 10k rows is not that much. You can import them using the normal neo4j transactional facilities in a few seconds. For building up your graph model you probably want to index your data or create category nodes to access certain parts of your domain model. Peter wrote a tool to import relational data into neo4j but that was normalized data where each table was represented by a certain type of node in the graph and foreign key relationships were converted to graph relationships. I don't know if that also handled m:n connection tables efficiently by converting them to relationships too. HTH Michael Am 08.10.2011 um 23:05 schrieb jiteshks: I am very new to neo4j.So I don't know all of its features.I am reading its documentation to understand how it works. My project's requirement is to import the data from a data warehouse( which is an oracle db) once every month.We are thinking of implementing neo4j in our project which means we will have to read the data from oracle db and put it into neo4j. There will be around 1 lakh rows(10^5) to be fetched from oracle db.What is the fast/efficient way of doing it? Thanks! -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Importing-dat a-from-oracle-to-neo4j-tp3406024p3406024.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
