Well on 90 mio line it needed 23 hours. I'm currently running it on the last 82 mio lines. But it already runs 19 hours and is at line 52.000.000 . I will need to improve that code even more I think. But for now it should be fine.
Am Samstag, 6. September 2014 15:17:12 UTC+2 schrieb Curtis Mosters: > > Hi Luca, thanks for your response. > > I was already able to fix it. The problem was that I used a float. I just > disabled it and now it runs pretty well. I was testing with 20 mio. And > these 20 mio were now created in 192 minutes. A great result. > > Will now let it run on all 180 mio lines =) > > Till then a great weekend. > > Am Samstag, 6. September 2014 12:48:58 UTC+2 schrieb Lvc@: >> >> Hi Curtis, >> nothing hardcoded at 16M in OrientDB. Can you post the thread dump of >> OrientDB server/JVM? >> >> Lvc@ >> >> >> >> On 6 September 2014 09:48, 'Curtis Mosters' via OrientDB < >> [email protected]> wrote: >> >>> Well I had my Java process running over night. First of all, that is >>> what I already have. >>> >>> I have basically: >>> >>> - *80 *mio entries (stuff Person have written) and >>> - *50 *mio entries of Persons >>> >>> Now I have a CSV file that is connecting both via ID's. >>> >>> My first idea on the Java implementation was by 200 entries/sec. (noTx) >>> >>> While my latest is ~2000/sec. (Tx) >>> >>> But now I'm looking on the current state of the system. And I still see >>> CPU and RAM changing and process is still running. But when I look onto the >>> IO values. It's just reading. >>> >>> So I was thinking that maybe the lines just contain ID's that are not in >>> the database. Maybe! But I have a syso that shows me every 10,000 lines the >>> current state. And it's not coming up anymore. So this cannot be. >>> >>> Btw I'm at line 16.777.000 right now. And it's somehow frozen I would >>> say. It's working really hardcore but doing nothing =/ >>> >>> Btw2 I: >>> >>> - use Transactions every 100 lines >>> - STORAGE_KEEP_OPEN=true >>> - ENVIRONMENT_CONCURRENT=false >>> - OIntentMassiveInsert=true >>> - setUsingLog=false >>> >>> -- >>> >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "OrientDB" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
