[Neo4j] SDN w/ write and readonly Graph Database Service?
I'm using SDN to build a graph db I'm expecting to have quite a heavy volume of write and read activity at peak times. We're not ready to start using HA at this point. I was wondering, if using Neo4J using two instances of the Graph Database Service, one EmbeddedGraphDatabase (writes), and one or more EmbeddedReadOnlyGraphDatabase (reads) would make any difference? Are there any benefits for splitting the db access that way? Or would accessing the db through the writable instance for all operations be pretty much the same? -TPP ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] SDN w/ write and readonly Graph Database Service?
Probably the opposite (if you could even do it). You'd lose the LRU caching across the boundary. Is the data being written the same as the data being read, or is there a natural segmentation? If so you could implement a crude form of sharding/partioning to avoid hot spots (concurrency related) during these periods. As always, SSD + plenty of RAM. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Tero Paananen Sent: Tuesday, November 01, 2011 12:16 PM To: user@lists.neo4j.org Subject: [Neo4j] SDN w/ write and readonly Graph Database Service? I'm using SDN to build a graph db I'm expecting to have quite a heavy volume of write and read activity at peak times. We're not ready to start using HA at this point. I was wondering, if using Neo4J using two instances of the Graph Database Service, one EmbeddedGraphDatabase (writes), and one or more EmbeddedReadOnlyGraphDatabase (reads) would make any difference? Are there any benefits for splitting the db access that way? Or would accessing the db through the writable instance for all operations be pretty much the same? -TPP ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] SDN w/ write and readonly Graph Database Service?
On Tue, Nov 1, 2011 at 12:26 PM, Rick Bullotta rick.bullo...@thingworx.com wrote: Probably the opposite (if you could even do it). You'd lose the LRU caching across the boundary. Is the data being written the same as the data being read, or is there a natural segmentation? If so you could implement a crude form of sharding/partioning to avoid hot spots (concurrency related) during these periods. It's largely going to be the same data. It's definitely the same type of data. Basically there will be discreet sets of same type of data (kinda like sub-graphs, but they can be connected) inserted in a batch-like manner. Those discreet sets of data will then immediately be consumed by processes that use it to calculate additional data and store the results to various data stores, incl. the same Neo4J db. After the initial consumption, the data sets will be consumed on-demand, but I'm not sure about how frequently at this point. It'll likely be something like once or twice a day or even less frequently, but I'll have to see how the usage patterns emerge after the solution goes live to be sure. Thanks for the answer Rick. I'll do some quick-and-dirty testing later this week to guide the decision making on this. I'll see if I can post what I find on this thread afterwards. -TPP ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] SDN w/ write and readonly Graph Database Service?
Cool. If you can control access to the data (e.g. no access until the entire subgraph/dataset has been imported) you can probably avoid a lot of the concurrency issues. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Tero Paananen Sent: Tuesday, November 01, 2011 12:49 PM To: Neo4j user discussions Subject: Re: [Neo4j] SDN w/ write and readonly Graph Database Service? On Tue, Nov 1, 2011 at 12:26 PM, Rick Bullotta rick.bullo...@thingworx.com wrote: Probably the opposite (if you could even do it). You'd lose the LRU caching across the boundary. Is the data being written the same as the data being read, or is there a natural segmentation? If so you could implement a crude form of sharding/partioning to avoid hot spots (concurrency related) during these periods. It's largely going to be the same data. It's definitely the same type of data. Basically there will be discreet sets of same type of data (kinda like sub-graphs, but they can be connected) inserted in a batch-like manner. Those discreet sets of data will then immediately be consumed by processes that use it to calculate additional data and store the results to various data stores, incl. the same Neo4J db. After the initial consumption, the data sets will be consumed on-demand, but I'm not sure about how frequently at this point. It'll likely be something like once or twice a day or even less frequently, but I'll have to see how the usage patterns emerge after the solution goes live to be sure. Thanks for the answer Rick. I'll do some quick-and-dirty testing later this week to guide the decision making on this. I'll see if I can post what I find on this thread afterwards. -TPP ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
