Sure Michael! I'll be available then. Well but there are more things broken. And I still have the 2.1.3 version because I'm not able to download the 2.1.4 via http://www.neo4j.org/download/other_versions I always get the download page but not any download.
So could you please fix the download and then I can try the separeted import. Maybe that's it already! Would be great. Am Sonntag, 21. September 2014 21:22:46 UTC+2 schrieb Michael Hunger: > > Ok, it's the two create statements in a row if you split it up it will not > happen: > > // *!! Eager Loading kills your import, everything is pulled from the > file into memory after the first create!* > > profile > LOAD CSV > WITH HEADERS FROM " > https://gist.githubusercontent.com/jexp/909e92c65e715ea91062/raw/cbe617cdf551ef51111fff349f1b1826a7a52d3d/appln.txt"; > > AS csvLine > WITH csvLine > LIMIT 1 > MATCH (appln:Appln) > WHERE appln.ID = toInt(csvLine.appln_id) > CREATE (title:Title { title:csvLine.appln_title }) > CREATE (appln)-[:HAS_TITLE]->(title); > > > +-------------+------+--------+---------------------+-------------------------------------------------------+ > | Operator | Rows | DbHits | Identifiers | > Other | > > +-------------+------+--------+---------------------+-------------------------------------------------------+ > | EmptyResult | 0 | 0 | | > | > | UpdateGraph | 0 | 0 | title, UNNAMED304 | > CreateNode; CreateRelationship | > *| Eager | 0 | 0 | | > |* > | SchemaIndex | 0 | 1 | appln, appln | > ToIntFunction(Property(csvLine,appln_id)); :Appln(ID) | > | Slice | 1 | 0 | | > { AUTOINT0} | > | LoadCSV | 1 | 0 | csvLine | > | > > +-------------+------+--------+---------------------+-------------------------------------------------------+ > > > *// Do it like this:* > > CREATE INDEX ON :Appln(ID); > CREATE INDEX ON :Title(title); > > profile > LOAD CSV > WITH HEADERS FROM " > https://gist.githubusercontent.com/jexp/909e92c65e715ea91062/raw/cbe617cdf551ef51111fff349f1b1826a7a52d3d/appln.txt"; > > AS csvLine > WITH csvLine > LIMIT 1 > CREATE (title:Title { title:csvLine.appln_title }); > > +-------------+------+--------+-------------+--------------+ > | Operator | Rows | DbHits | Identifiers | Other | > +-------------+------+--------+-------------+--------------+ > | EmptyResult | 0 | 0 | | | > | UpdateGraph | 1 | 5 | title | CreateNode | > | Slice | 1 | 0 | | { AUTOINT0} | > | LoadCSV | 1 | 0 | csvLine | | > +-------------+------+--------+-------------+--------------+ > > profile > LOAD CSV > WITH HEADERS FROM " > https://gist.githubusercontent.com/jexp/909e92c65e715ea91062/raw/cbe617cdf551ef51111fff349f1b1826a7a52d3d/appln.txt"; > > AS csvLine > WITH csvLine > LIMIT 1 > MATCH (appln:Appln {ID: csvLine.appln_id}) > MATCH (title:Title { title:csvLine.appln_title }) > CREATE (appln)-[:HAS_TITLE]->(title); > > > +----------------+------+--------+--------------+----------------------------------------------+ > | Operator | Rows | DbHits | Identifiers | > Other | > > +----------------+------+--------+--------------+----------------------------------------------+ > | EmptyResult | 0 | 0 | | > | > | UpdateGraph | 0 | 0 | UNNAMED285 | > CreateRelationship | > | SchemaIndex(0) | 0 | 0 | title, title | > Property(csvLine,appln_title); :Title(title) | > | SchemaIndex(1) | 0 | 1 | appln, appln | > Property(csvLine,appln_id); :Appln(ID) | > | Slice | 1 | 0 | | > { AUTOINT0} | > | LoadCSV | 1 | 0 | csvLine | > | > > +----------------+------+--------+--------------+----------------------------------------------+ > > > On Sun, Sep 21, 2014 at 9:03 PM, Michael Hunger < > michael...@neotechnology.com <javascript:>> wrote: > >> Ah, cool, then we can do a call this week, perhaps tomorrow or on Thu. >> >> That's really weird. It shouldn't build up memory in 2.1.4 as the team >> changed the eagerness requirement for queries where we determine >> independence. >> >> Can you do a a limit 10 and prefix your query with profile? There should >> be no "Eager"-pipe in between the match and merge. >> >> >> On Sun, Sep 21, 2014 at 8:31 PM, 'Curtis Mosters' via Neo4j < >> ne...@googlegroups.com <javascript:>> wrote: >> >>> Sure y. And testing with LIMIT 1000000 works and needs 1 min or so. >>> >>> So this is just a Heap Stack thingi. >>> >>> I'm from Germany like you ;) >>> >>> Am Sonntag, 21. September 2014 19:25:15 UTC+2 schrieb Michael Hunger: >>>> >>>> In what Timezone are you located? Perhaps someone can help you and have >>>> a look at your setup? via Skype >>>> >>>> But you have an index or constraint on the thing you match? >>>> >>>> CREATE INDEX ON :Appln(ID); >>>> >>>> >>>> >>>> On Sun, Sep 21, 2014 at 4:08 PM, 'Curtis Mosters' via Neo4j < >>>> ne...@googlegroups.com> wrote: >>>> >>>>> Jup, same issue with >>>>> >>>>> *-Xmx512M*So you have any other tips? Observing the IO values, it >>>>> seems to idle =/ >>>>> >>>>> Am Samstag, 20. September 2014 12:58:33 UTC+2 schrieb Curtis Mosters: >>>>> >>>>>> Well I tried it with >>>>>> >>>>>> *-Xms4096m -Xmx15000m* >>>>>> >>>>>> but the import did not finish in now 16 hours. >>>>>> >>>>>> Now I try your >>>>>> >>>>>> *-Xmx512M* >>>>>> >>>>>> Am Freitag, 19. September 2014 16:08:42 UTC+2 schrieb Michael Hunger: >>>>>>> >>>>>>> If you specifiy it I'd keep it to a few MB (e.g. -Xmx512M) as the >>>>>>> shell against a running server is just a terminal. >>>>>>> It's different if when you run the shell against a on-disk directory >>>>>>> then it fires up the database itself. >>>>>>> >>>>>>> Your memory config looks ok otherwise, I'd only swap mapped memory >>>>>>> settings for nodes (to 50MB) and rels (to 500MB) >>>>>>> >>>>>>> I think there was one memory issue in 2.1.3 could you try it with >>>>>>> 2.1.4 ? >>>>>>> >>>>>>> Michael >>>>>>> >>>>>>> >>>>>>> On Fri, Sep 19, 2014 at 4:00 PM, 'Curtis Mosters' via Neo4j < >>>>>>> ne...@googlegroups.com> wrote: >>>>>>> >>>>>>>> Here is the complete file, well I still don't know which settings >>>>>>>> are the best in my case. >>>>>>>> >>>>>>>> Should I delete the Xmx thing? Is it better to not specify it? >>>>>>>> >>>>>>>> Am Donnerstag, 18. September 2014 23:14:09 UTC+2 schrieb Michael >>>>>>>> Hunger: >>>>>>>>> >>>>>>>>> If you connect to the server it uses the servers RAM >>>>>>>>> The shell then needs much less like 512m >>>>>>>>> >>>>>>>>> Can you check the server's graph.db/messages.log for the root >>>>>>>>> cause error? >>>>>>>>> >>>>>>>>> Which version? >>>>>>>>> >>>>>>>>> Sent from mobile device >>>>>>>>> >>>>>>>>> Am 18.09.2014 um 14:28 schrieb "'Curtis Mosters' via Neo4j" < >>>>>>>>> ne...@googlegroups.com>: >>>>>>>>> >>>>>>>>> Well I was thinking that the Xmx is the heap and it should be not >>>>>>>>> above 4 GB, so far I have learned. >>>>>>>>> >>>>>>>>> I currently let the import run with "COMMIT 100" so every 100 >>>>>>>>> lines, and it's still running. >>>>>>>>> >>>>>>>>> But looking on the IO it seems to idle... >>>>>>>>> >>>>>>>>> I don't know, but could you tell me exactly where to set which >>>>>>>>> setting? >>>>>>>>> >>>>>>>>> As I said I have 30 GB RAM. To avoid more complications let's >>>>>>>>> maybe set the Heap 4 GB and the maximum RAM to 20 GB. >>>>>>>>> >>>>>>>>> And again I use the Neo4jShell for the import. It's loading a file >>>>>>>>> containing LOAD CSV. >>>>>>>>> >>>>>>>>> Thanks. >>>>>>>>> >>>>>>>>> Am Donnerstag, 18. September 2014 13:42:34 UTC+2 schrieb Clark >>>>>>>>> Richey: >>>>>>>>>> >>>>>>>>>> According to the screenshot you don't give it 30G of ram. You >>>>>>>>>> only provide 3G. Is that your intent? >>>>>>>>>> >>>>>>>>>> Sent from my iPhone >>>>>>>>>> >>>>>>>>>> On Sep 18, 2014, at 02:51, 'Curtis Mosters' via Neo4j < >>>>>>>>>> ne...@googlegroups.com> wrote: >>>>>>>>>> >>>>>>>>>> Well somehow the behaviour of the Shell is odd. >>>>>>>>>> >>>>>>>>>> I imported 76 Mio lines without any problems. But now I have >>>>>>>>>> started a new import over night and now I watched the process and it >>>>>>>>>> said >>>>>>>>>> Java Heap Space Error. >>>>>>>>>> >>>>>>>>>> But that cannot be. I have set the Neo4jShell.bat like this: >>>>>>>>>> >>>>>>>>>> %JAVACMD% -Xmx3G *-Dstorage.diskCache.bufferSize=20000* % >>>>>>>>>> JAVA_OPTS% %EXTRA_JVM_ARGUMENTS% -classpath %CLASSPATH_PREFIX%;% >>>>>>>>>> CLASSPATH% -Dapp.name="neo4j-shell" -Dapp.repo="%REPO%" -Dbasedir >>>>>>>>>> ="%BASEDIR%" org.neo4j.shell.StartClient %CMD_LINE_ARGS% >>>>>>>>>> >>>>>>>>>> That is a great ratio of heap. But yeah why did I get this error? >>>>>>>>>> I have 30 GB RAM. This happens so random that a productive import is >>>>>>>>>> nearly >>>>>>>>>> impossible. >>>>>>>>>> >>>>>>>>>> So my question is, what have I done wrong? Where do I need to set >>>>>>>>>> the Java Settings correctly? Were my edited settings active? >>>>>>>>>> >>>>>>>>>> Here the first file or let's say script i'm loading in and where >>>>>>>>>> this error happens: >>>>>>>>>> >>>>>>>>>> USING PERIODIC COMMIT 2000 >>>>>>>>>> LOAD CSV WITH HEADERS FROM "file:///C:/data/tls202_part01.txt" >>>>>>>>>> AS csvLine WITH csvLine LIMIT 1000 >>>>>>>>>> MATCH (appln:Appln) >>>>>>>>>> WHERE appln.ID = toInt(csvLine.appln_id) >>>>>>>>>> CREATE (title:Title { title:csvLine.appln_title }) >>>>>>>>>> CREATE (appln)-[:HAS_TITLE]->(title); >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "Neo4j" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to neo4j+un...@googlegroups.com. >>>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "Neo4j" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to neo4j+un...@googlegroups.com. >>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>> >>>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Neo4j" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to neo4j+un...@googlegroups.com. >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>> >>>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Neo4j" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to neo4j+un...@googlegroups.com. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Neo4j" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to neo4j+un...@googlegroups.com <javascript:>. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> > -- You received this message because you are subscribed to the Google Groups "Neo4j" group. To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
