Thanks for hint. I made some fixes for better performance and fixed some
typos.
Here is relations.json:
{
>
> "config" : {
>
> "haltOnError": true,
>
> "log": "info",
>
> "parallel": false
>
> },
>
> "source": { "file": { "path": "Relations.csv", "lock": false} },
>
> "extractor": { "row": {} },
>
> "transformers": [
>
> { "csv": {
>
> "separator": ";",
>
> "columnsOnFirstLine": true,
>
> "columns":
>
> ["SOURCEID","DESTID","LABEL","PROPERTY1","PROPERTY2"]
>
> }
>
> },
>
> {
>
> "command": {
>
> "command": "create edge Relation from (select from Node where
>> ID = ${input.SOURCEID}) to (select from Node where ID = ${input.DESTID})",
>
> "output": "relazione"
>
> }
>
> }
>
> ],
>
> "loader": {
>
> "orientdb": {
>
> "dbURL": "remote:localhost/example",
>
> "dbType": "graph",
>
> "dbAutoCreate": true,
>
> "dbAutoCreateProperties": true,
>
> "wal": false,
>
> "tx": false,
>
> "batchCommit": 100,
>
> "classes": [
>
> {"name": "RelationOut", "extends": "E"}
>
> ]
>
> }
>
> }
>
> }
>
>
Is there any better (quicker) solution? ID in Node is indexed column, but
the maximum speed is only cca. 120 edges/sec. I have graph with 100M edges
so it will take 10 days to import :/
If I try change parallel option to true I get many null pointer exception
errors:
Exception in thread "OrientDB ETL pipeline-0"
>> com.orientechnologies.orient.etl.OETLProcessHaltedExce
>
> ption: com.orientechnologies.orient.core.sql.OCommandSQLParsingException:
>> Error on parsing command a
>
> t position #0: Encountered "" at line 1, column 51.
>
> Was expecting one of:
>
>
>> at
>> com.orientechnologies.orient.etl.OETLPipeline.execute(OETLPipeline.java:148)
>
> at
>> com.orientechnologies.orient.etl.OETLProcessor$2.run(OETLProcessor.java:370)
>
> at java.lang.Thread.run(Thread.java:745)
>
> Caused by:
>> com.orientechnologies.orient.core.sql.OCommandSQLParsingException: Error on
>> parsing comma
>
> nd at position #0: Encountered "" at line 1, column 51.
>
> Was expecting one of:
>
>
>> at
>> com.orientechnologies.orient.core.sql.OCommandExecutorSQLAbstract.throwParsingException(O
>
> CommandExecutorSQLAbstract.java:102)
>
> at
>> com.orientechnologies.orient.core.sql.OCommandExecutorSQLAbstract.preParse(OCommandExecut
>
> orSQLAbstract.java:238)
>
> at
>> com.orientechnologies.orient.graph.sql.OCommandExecutorSQLCreateEdge.parse(OCommandExecut
>
> orSQLCreateEdge.java:72)
>
> at
>> com.orientechnologies.orient.graph.sql.OCommandExecutorSQLCreateEdge.parse(OCommandExecut
>
> orSQLCreateEdge.java:55)
>
> at
>> com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.parse(OCommandExecutorS
>
> QLDelegate.java:76)
>
> at
>> com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.parse(OCommandExecutorS
>
> QLDelegate.java:42)
>
> at
>> com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.command(OA
>
> bstractPaginatedStorage.java:1447)
>
> at
>> com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandReq
>
> uestTextAbstract.java:63)
>
> at
>> com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.comman
>
> d(ONetworkProtocolBinary.java:1314)
>
> at
>> com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.execut
>
> eRequest(ONetworkProtocolBinary.java:396)
>
> at
>> com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstrac
>
> t.execute(OBinaryNetworkProtocolAbstract.java:223)
>
> at
>> com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:77)
>
> Waiting for the pipeline to finish, remaining 421 entries to process
>
>
Anyway, thanks for your help!
Marek
--
---
You received this message because you are subscribed to the Google Groups
"OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
