I tested ETL 2 months ago and there I had the same problem, that the ETL
did not create any edges.
Will test again when 2.0 is officially out.
Am Montag, 22. September 2014 23:48:18 UTC+2 schrieb Olakunle Olaniyi:
>
> Hello ,
> I have 2 json files (person.json and alien.json) and they are as below :
>
> =======person.json================
> {
> "config": {
> "log": "debug"
> },
> "extractor" : {
> "jdbc": { "driver": "org.postgresql.Driver",
> "url": "jdbc:postgresql://localhost/testdb",
> "userName": "postgres",
> "userPassword": "password1",
> "query": "select * from person" }
> },
> "transformers" : [
> { "vertex": { "class": "Person"} }
> ],
> "loader" : {
> "orientdb": {
> "dbURL": "plocal:/home/orientdb/db/databases/db_test",
> dbUser: "admin",
> dbPassword: "admin",
> dbAutoDropIfExists: false,
> dbAutoCreate: true,
> tx: false,
> wal: false,
> batchCommit: 1000,
> dbType: "graph",
> indexes: [{class:"Person", fields:["pid:string"],
> type:"UNIQUE_HASH_INDEX" }]
> }
> }
> }
>
>
>
>
>
> ======alien.json=====================
> {
> "config": {
> "log": "debug"
> },
> "extractor" : {
> "jdbc": { "driver": "org.postgresql.Driver",
> "url": "jdbc:postgresql://localhost/testdb",
> "userName": "postgres",
> "userPassword": "password1",
> "query": "select * from alien" }
> },
> "transformers" : [
> { "vertex": { "class": "Alien"} },
> { "edge": { "class": "hasLink", "direction" : "in",
> "joinFieldName": "aid",
> "lookup":"Person.pid", "unresolvedLinkAction":"CREATE"} }
> ],
> "loader" : {
> "orientdb": {
> "dbURL": "plocal:/home/orientdb/db/databases/db_test",
> dbUser: "admin",
> dbPassword: "admin",
> dbAutoDropIfExists: false,
> dbAutoCreate: true,
> tx: false,
> wal: false,
> batchCommit: 1000,
> dbType: "graph",
> indexes: [{class:"Alien", fields:["aid:string"],
> type:"UNIQUE_HASH_INDEX" }]
> }
> }
> }
>
> On Mon, Sep 22, 2014 at 7:13 PM, Luca Garulli <[email protected]
> <javascript:>> wrote:
>
>> Hi,
>> Seems there is no Loader configured. Could you post also your JSON file?
>>
>> Lvc@
>>
>> On 22 September 2014 18:42, Olakunle Olaniyi <[email protected]
>> <javascript:>> wrote:
>>
>>> Hi Luca,
>>> I'd like to start by saying thanks for the explanation on this so far. I
>>> am running orientdb 2.0-M1 and this is not working for me as well. It shows
>>> the joinvalue alright and they are correct but the edges are not be
>>> created. Have an explanation why this is so? Your response will be much
>>> appreciated.
>>>
>>> Olakay
>>>
>>> P.S : the log could be see below :
>>>
>>>
>>> OrientDB etl v.2.0-M1 (build @BUILD@) www.orientechnologies.com
>>> BEGIN ETL PROCESSOR
>>> [1:vertex] DEBUG Transformer input: {aid:1,name:Qualitrail - 1}
>>> [1:vertex] DEBUG Transformer output: v(Alien)[#12:10]
>>> [1:edge] DEBUG Transformer input: v(Alien)[#12:10]
>>> [1:edge] DEBUG WARNING: index Person.mid not found. Lookups could be
>>> really slow
>>> [1:edge] DEBUG joinValue=1,
>>> lookupResult=Person#11:0{mid:1,name:Qualitrail - 1,in_hasLink:[size=1]} v2
>>> [1:edge] DEBUG Transformer output: v(Alien)[#12:10]
>>> [3:vertex] DEBUG Transformer input: {aid:2,name:Qualitrail - 2}
>>> [3:vertex] DEBUG Transformer output: v(Alien)[#12:11]
>>> [3:edge] DEBUG Transformer input: v(Alien)[#12:11]
>>> [3:edge] DEBUG joinValue=2,
>>> lookupResult=Person#11:1{mid:2,name:Qualitrail - 2} v1
>>> [3:edge] DEBUG Transformer output: v(Alien)[#12:11]
>>> [5:vertex] DEBUG Transformer input: {aid:3,name:Qualitrail - 3}
>>> [5:vertex] DEBUG Transformer output: v(Alien)[#12:12]
>>> [5:edge] DEBUG Transformer input: v(Alien)[#12:12]
>>> [5:edge] DEBUG joinValue=3,
>>> lookupResult=Person#11:2{mid:3,name:Qualitrail - 3} v1
>>> [5:edge] DEBUG Transformer output: v(Alien)[#12:12]
>>> [7:vertex] DEBUG Transformer input: {aid:4,name:Qualitrail - 4}
>>> [7:vertex] DEBUG Transformer output: v(Alien)[#12:13]
>>> [7:edge] DEBUG Transformer input: v(Alien)[#12:13]
>>> [7:edge] DEBUG joinValue=4,
>>> lookupResult=Person#11:3{mid:4,name:Qualitrail - 4} v1
>>> [7:edge] DEBUG Transformer output: v(Alien)[#12:13]
>>> [9:vertex] DEBUG Transformer input: {aid:5,name:Qualitrail - 5}
>>> [9:vertex] DEBUG Transformer output: v(Alien)[#12:14]
>>> [9:edge] DEBUG Transformer input: v(Alien)[#12:14]
>>> [9:edge] DEBUG joinValue=5,
>>> lookupResult=Person#11:4{mid:5,name:Qualitrail - 5} v1
>>> [9:edge] DEBUG Transformer output: v(Alien)[#12:14]
>>> [11:vertex] DEBUG Transformer input: {aid:6,name:Qualitrail - 6}
>>> [11:vertex] DEBUG Transformer output: v(Alien)[#12:15]
>>> [11:edge] DEBUG Transformer input: v(Alien)[#12:15]
>>> [11:edge] DEBUG joinValue=6,
>>> lookupResult=Person#11:5{mid:6,name:Qualitrail - 6} v1
>>> [11:edge] DEBUG Transformer output: v(Alien)[#12:15]
>>> [13:vertex] DEBUG Transformer input: {aid:7,name:Qualitrail - 7}
>>> [13:vertex] DEBUG Transformer output: v(Alien)[#12:16]
>>> [13:edge] DEBUG Transformer input: v(Alien)[#12:16]
>>> [13:edge] DEBUG joinValue=7,
>>> lookupResult=Person#11:6{mid:7,name:Qualitrail - 7} v1
>>> [13:edge] DEBUG Transformer output: v(Alien)[#12:16]
>>> [15:vertex] DEBUG Transformer input: {aid:8,name:Qualitrail - 8}
>>> [15:vertex] DEBUG Transformer output: v(Alien)[#12:17]
>>> [15:edge] DEBUG Transformer input: v(Alien)[#12:17]
>>> [15:edge] DEBUG joinValue=8,
>>> lookupResult=Person#11:7{mid:8,name:Qualitrail - 8} v1
>>> [15:edge] DEBUG Transformer output: v(Alien)[#12:17]
>>> [17:vertex] DEBUG Transformer input: {aid:9,name:Qualitrail - 9}
>>> [17:vertex] DEBUG Transformer output: v(Alien)[#12:18]
>>> [17:edge] DEBUG Transformer input: v(Alien)[#12:18]
>>> [17:edge] DEBUG joinValue=9,
>>> lookupResult=Person#11:8{mid:9,name:Qualitrail - 9} v1
>>> [17:edge] DEBUG Transformer output: v(Alien)[#12:18]
>>> [19:vertex] DEBUG Transformer input: {aid:10,name:Qualitrail - 10}
>>> [19:vertex] DEBUG Transformer output: v(Alien)[#12:19]
>>> [19:edge] DEBUG Transformer input: v(Alien)[#12:19]
>>> [19:edge] DEBUG joinValue=10,
>>> lookupResult=Person#11:9{mid:10,name:Qualitrail - 10} v1
>>> [19:edge] DEBUG Transformer output: v(Alien)[#12:19]
>>> END ETL PROCESSOR
>>>
>>>
>>> On Thursday, July 3, 2014 6:06:45 PM UTC+2, Lvc@ wrote:
>>>>
>>>> Hi guys,
>>>> I'm glad to announce a new project under the Orient Technologies
>>>> umbrella: *OrientDB-ETL*!
>>>>
>>>> This is the URL of WiKi:
>>>> *https://github.com/orientechnologies/orientdb-etl/wiki
>>>> <https://github.com/orientechnologies/orientdb-etl/wiki>*
>>>>
>>>> Thanks to OrientDB-ETL project, moving data from/to OrientDB is super
>>>> fast and require no more code writing: only a JSON file.
>>>>
>>>> So far these are the available components:
>>>>
>>>> - *Extractors*:
>>>> - File
>>>> - *Transformers*:
>>>> - CSV, to convert rows in documents
>>>> - Link, to transform JOIN in links
>>>> - Merge, to merge the parsed document with an existent
>>>> - Vertex, to convert a document in Vertex
>>>> - Edge, to create edges
>>>> - Skip, to skip records based on expressions
>>>> - Code, to execute arbitrary Javascript code
>>>> - *Loaders*:
>>>> - OrientDB, to save into a OrientDB database
>>>>
>>>> OrientDB-ETL module is in beta status, and will be final with OrientDB
>>>> v2.0 (September). To use it against OrientDB 2.0-SNAPSHOT follow the
>>>> "Installation" instructions.
>>>>
>>>> Below an example to import *DBPedia* *csv* files (
>>>> http://wiki.dbpedia.org/DBpediaAsTables):
>>>>
>>>> {
>>>> config: {
>>>> verbose: true,
>>>> fileDirectory: "/temp/databases/dbpedia_csv/",
>>>> fileName: "Person.csv.gz"
>>>> },
>>>> begin: [
>>>> { let: { name: "$filePath", value: "$fileDirectory.append(
>>>> $fileName )"} },
>>>> { let: { name: "$className", value: "$fileName.substring( 0,
>>>> $fileName.indexOf('.') )"} }
>>>> ],
>>>> extractor : {
>>>> line: { path: "$filePath", lock : true }
>>>> },
>>>> transformers : [
>>>> { csv: { separator: ",", nullValue: "NULL", skipFrom: 1, skipTo: 3 }
>>>> },
>>>> { merge: { joinFieldName:"URI", lookup:"V.URI" } },
>>>> { vertex: { class: "$className"} }
>>>> ],
>>>> loader : {
>>>> orientdb: {
>>>> dbURL: "plocal:/temp/databases/dbpedia",
>>>> dbUser: "admin",
>>>> dbPassword: "admin",
>>>> dbAutoCreate: true,
>>>> tx: false,
>>>> batchCommit: 1000,
>>>> dbType: "graph",
>>>> indexes: [{class:"V", fields:["URI:string"], type:"UNIQUE" }]
>>>> }
>>>> }
>>>> }
>>>>
>>>> For more information: https://github.com/orientechnologies/
>>>> orientdb-etl/wiki/Import-from-DBPedia.
>>>>
>>>> Lvc@
>>>>
>>>> --
>>>
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "OrientDB" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected] <javascript:>.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
>>
>> ---
>> You received this message because you are subscribed to a topic in the
>> Google Groups "OrientDB" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/orient-database/Xp38b55AbpU/unsubscribe
>> .
>> To unsubscribe from this group and all its topics, send an email to
>> [email protected] <javascript:>.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> Olakunle
>
--
---
You received this message because you are subscribed to the Google Groups
"OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
