This bug has been found and fixed, so will be included in the next milestone.
2014-03-17 15:13 GMT+01:00 Mattias Persson <[email protected]>: > Thank you, I was just now able to reproduce this! > > > 2014-03-17 11:28 GMT+01:00 Sotiris Beis <[email protected]>: > > Hi Mattias, >> >> here is some more details: >> >> - The function to create the graph >> public void createGraphForMassiveLoad(String dbPath) { >> System.out.println("Creating Neo4j Graph Database for massive load . . . >> ."); >> Map<String, String> config = new HashMap<String, String>(); >> config.put("cache_type", "none"); >> config.put("use_memory_mapped_buffers", "true"); >> config.put("neostore.nodestore.db.mapped_memory", "200M"); >> config.put("neostore.relationshipstore.db.mapped_memory", "1000M"); >> config.put("neostore.propertystore.db.mapped_memory", "250M"); >> config.put("neostore.propertystore.db.strings.mapped_memory", "250M"); >> inserter = BatchInserters.inserter(dbPath, config); >> indexProvider = new LuceneBatchInserterIndexProvider(inserter); >> nodes = indexProvider.nodeIndex("nodes", MapUtil.stringMap("type", >> "exact")); >> } >> >> - The code to load the data >> public void createGraph(String datasetDir) { >> System.out.println("Loading data in massive mode in Neo4j database . . . >> ."); >> >> inserter.createDeferredSchemaIndex(Neo4jGraphDatabase.NODE_LABEL).on("nodeId").create(); >> try { >> BufferedReader reader = new BufferedReader(new InputStreamReader(new >> FileInputStream(datasetDir))); >> String line; >> int lineCounter = 1; >> // Map<String, Object> properties; >> // IndexHits<Long> cache; >> long srcNode, dstNode; >> while((line = reader.readLine()) != null) { >> if(lineCounter > 4) { >> String[] parts = line.split("\t"); >> srcNode = getOrCreate(parts[0]); >> dstNode = getOrCreate(parts[1]); >> inserter.createRelationship(srcNode, dstNode, >> Neo4jGraphDatabase.RelTypes.SIMILAR, null); >> } >> lineCounter++; >> } >> reader.close(); >> } >> catch (IOException e) { >> e.printStackTrace(); >> } >> nodes.flush(); >> } >> private long getOrCreate(String value) { >> Long id = cache.get(Long.valueOf(value)); >> if(id == null) { >> Map<String, Object> properties = MapUtil.map("nodeId", value); >> id = inserter.createNode(properties, Neo4jGraphDatabase.NODE_LABEL); >> cache.put(Long.valueOf(value), id); >> nodes.add(id, properties); >> } >> return id; >> } >> >> - The dataset I made the tests: >> http://snap.stanford.edu/data/email-Enron.html >> >> Unfortunately the database is not available right now, but it's pretty >> easy to reproduce with the above code. >> >> Thanks, >> Sotiris >> >> >> Τη Κυριακή, 16 Μαρτίου 2014 4:48:16 μ.μ. UTC+2, ο χρήστης Mattias Persson >> έγραψε: >>> >>> Yup, it's probably a regression introduced in the recent store format >>> changes. I'd love to track this and be able to reproduce it. How were the >>> relationships added to the node? Distribution of types/directions and also >>> in which order they were added. Could you provide detailed information >>> about that, or provide the database zipped up to me directly ( >>> [email protected]) ? >>> >>> Thanks in advance >>> >>> >>> 2014-03-13 12:26 GMT+01:00 Sotiris Beis <[email protected]>: >>> >>>> Ok, can you suggest me another temporary solution? >>>> >>>> >>>> On 03/13/2014 01:24 PM, Michael Hunger wrote: >>>> >>>> Thanks for the feedback. Could be related to the changes in the store >>>> format for heavily connected nodes. >>>> >>>> We'll investigate. >>>> >>>> Cheers, >>>> >>>> Michael >>>> >>>> ---- >>>> (michael <http://twitter.com/mesirii>)-[:SUPPORTS]->(*YOU*)-[:USE]->( >>>> Neo4j <http://neo4j.org>) >>>> Learn Online <http://neo4j.org/learn/online_course>, >>>> Offline<http://www.neo4j.org/events> or >>>> Read a Book <http://graphdatabases.com> (in Deutsch<http://bit.ly/das-buch> >>>> ) >>>> We're trading T-shirts for cool Graph Models <http://bit.ly/graphgist> >>>> >>>> >>>> >>>> Am 13.03.2014 um 12:22 schrieb Sotiris Beis <[email protected]>: >>>> >>>> Which version of Neo4j are you using? >>>> >>>> I use neo4j-2.1.0-M01 >>>> >>>> You use a Set which elminates duplicates. You probably have >>>> duplicate neighbourId's that are only 100 distinct ones. >>>> >>>> That was my first thought, but I cheched it. There are no dublicates. >>>> Why do you think the result is different when I use this test line >>>> >>>> System.out.println(IteratorUtil.count(n.getRelationships(Direction. >>>> OUTGOING))); >>>> >>>> before the iteration of the relationships? >>>> >>>> n 03/13/2014 01:00 PM, Michael Hunger wrote: >>>> >>>> Which version of Neo4j are you using? >>>> >>>> You use a Set which elminates duplicates. You probably have >>>> duplicate neighbourId's that are only 100 distinct ones. >>>> >>>> And you close the transaction twice. It is an auto-closable resource >>>> so you can remove your manual tx.close() line. >>>> >>>> Cheers, >>>> >>>> Michael >>>> >>>> ---- >>>> (michael <http://twitter.com/mesirii>)-[:SUPPORTS]->(*YOU*)-[:USE]->( >>>> Neo4j <http://neo4j.org/>) >>>> Learn Online <http://neo4j.org/learn/online_course>, >>>> Offline<http://www.neo4j.org/events> or >>>> Read a Book <http://graphdatabases.com/> (in >>>> Deutsch<http://bit.ly/das-buch> >>>> ) >>>> We're trading T-shirts for cool Graph Models <http://bit.ly/graphgist> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> Am 13.03.2014 um 11:21 schrieb Sotiris Beis <[email protected]>: >>>> >>>> I have the following case study. I want to get the neighbors of the >>>> certain node. My function is this one: >>>> public Set<Integer> getNeighborsIds(int nodeId) { >>>> Set<Integer> neighbours = new HashSet<Integer>(); >>>> try (Transaction tx = neo4jGraph.beginTx()) { >>>> Node n = nodeIndex.get("nodeId", nodeId).getSingle(); >>>> for(Relationship relationship : n.getRelationships(Direction.OUTGOING)) >>>> { >>>> Node neighbour = relationship.getEndNode(); >>>> String neighbourId = (String)neighbour.getProperty("nodeId"); >>>> neighbours.add(Integer.valueOf(neighbourId)); >>>> } >>>> tx.success(); >>>> tx.close(); >>>> } >>>> return neighbours; >>>> } >>>> >>>> I know that this node has 255 neighbors but this function return only >>>> 100 nodes. When I put this line >>>> System.out.println(IteratorUtil.count(n.getRelationships(Direction. >>>> OUTGOING))); >>>> >>>> >>>> 4 and 5 line the function returns 255 neighbors. The most strange is >>>> that the above function prints out the value 100. Is this a bug or can >>>> anyone explain this to me? >>>> >>>> Thanks, >>>> Sotiris >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Neo4j" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "Neo4j" group. >>>> To unsubscribe from this topic, visit https://groups.google.com/d/ >>>> topic/neo4j/stHamJpQSBk/unsubscribe. >>>> To unsubscribe from this group and all its topics, send an email to >>>> [email protected]. >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Neo4j" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "Neo4j" group. >>>> To unsubscribe from this topic, visit https://groups.google.com/d/ >>>> topic/neo4j/stHamJpQSBk/unsubscribe. >>>> To unsubscribe from this group and all its topics, send an email to >>>> [email protected]. >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Neo4j" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >>> >>> -- >>> Mattias Persson, [[email protected]] >>> Hacker, Neo Technology >>> www.neotechnology.com >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Neo4j" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Mattias Persson, [[email protected]] > Hacker, Neo Technology > www.neotechnology.com > -- Mattias Persson, [[email protected]] Hacker, Neo Technology www.neotechnology.com -- You received this message because you are subscribed to the Google Groups "Neo4j" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
