You didn't mention before that you used the "superfast" batch-inserter, I
think that version is still work in progress, not sure if it creates a
normal store.
I used my own batch-inserter github.com/jexp/batch-import
with these batch.properties:
dump_configuration=false
cache_type=none
use_memory_mapped_buffers=true
neostore.propertystore.db.index.keys.mapped_memory=5M
neostore.propertystore.db.index.mapped_memory=5M
neostore.nodestore.db.mapped_memory=200M
neostore.relationshipstore.db.mapped_memory=2G
neostore.propertystore.db.mapped_memory=100M
neostore.relationshipgroupstore.db.mapped_memory=10M
neostore.propertystore.db.strings.mapped_memory=100M
batch_array_separator=,
batch_import.csv.quotes=false
#batch_import.csv.delim=,
#batch_import.node_index.source_id=exact
#batch_import.node_index.topic=fulltext
Importing 111111001 Relationships took 478 seconds
Total import time: 520 seconds
Then running your queries, actually without the second limit:
| [Node[103105]{name:"1963-64 Austrian football
championship"},:My_Proximity[102026221]{proximity:13},Node[2513520]{name:"Cowley
plant"},:My_Proximity[108842982]{proximity:28},Node[5523128]{name:"Kinzirô
Miyake"}]
| 41 |
| [Node[103105]{name:"1963-64 Austrian football
championship"},:My_Proximity[102026221]{proximity:13},Node[2513520]{name:"Cowley
plant"},:My_Proximity[25343932]{proximity:27},Node[9598046]{name:"Suzdal
Urban Settlement"}]
| 40 |
| [Node[103105]{name:"1963-64 Austrian football
championship"},:My_Proximity[102026221]{proximity:13},Node[2513520]{name:"Cowley
plant"},:My_Proximity[108581215]{proximity:13},Node[2627627]{name:"DSFA"}]
| 26 |
| [Node[103105]{name:"1963-64 Austrian football
championship"},:My_Proximity[102026221]{proximity:13},Node[2513520]{name:"Cowley
plant"}]
| 13 |
| [Node[103105]{name:"1963-64 Austrian football championship"}]
| 0 |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
2241 rows
129 ms
Add this to the config of the server:
4G heap
# Default values for the low-level graph engine
neostore.nodestore.db.mapped_memory=250M
neostore.relationshipstore.db.mapped_memory=500M
neostore.propertystore.db.mapped_memory=250M
neostore.propertystore.db.strings.mapped_memory=250M
can you try this?
also add this to your neo4j.properties
neostore.relationshipgroupstore.db.mapped_memory=10M
On Fri, Oct 3, 2014 at 11:43 PM, gg4u <[email protected]> wrote:
> Hi,
>
> here my new answer, I got into this issue:
>
> I have a large weighted graph with only one schema index on nodes (Topic):
> 4M topics and 100M rels.
>
> I wanted to find paths between two given nodes.
>
> I tried out with queries like this one:
> since it is a weighted graph, I compute the weighted path between nodes as
> the sum of its weight (I called weight 'proximity' here).
>
> Problem is, a query of this type, on such a large graph, tooks ages:
>
> Note that using an index, either directly the internal id, give same
> responsive results
> *Is there any way to speed up performance to reasonable production time?*
> (lower than 1s ... it means 3 orders of magnitude ... )
>
> MATCH (n) , (m), p = (n)-[*0..2]-(m)
> where id(n) = 103105 and id(m) = 1386672
> with p, n, m
> return p, reduce(totProximity = 0, n IN relationships(p)| totProximity +
> n.proximity) AS pathProximity order by pathProximity DESC;
>
> *~1M ms !!! *
>
>
> same as
> MATCH (n:Topic) , (m:Topic), p = (n)-[*0..2]-(m)
> where n.name = 'title-1' and id(m) = 'title-2'
> with p, n, m
> return p, reduce(totProximity = 0, n IN relationships(p)| totProximity +
> n.proximity) AS pathProximity order by pathProximity DESC;
>
> *~2M ms !!! *
>
> --
> You received this message because you are subscribed to the Google Groups
> "Neo4j" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
