Hello ,
I have 2 json files (person.json and alien.json) and they are as below :
=======person.json================
{
"config": {
"log": "debug"
},
"extractor" : {
"jdbc": { "driver": "org.postgresql.Driver",
"url": "jdbc:postgresql://localhost/testdb",
"userName": "postgres",
"userPassword": "password1",
"query": "select * from person" }
},
"transformers" : [
{ "vertex": { "class": "Person"} }
],
"loader" : {
"orientdb": {
"dbURL": "plocal:/home/orientdb/db/databases/db_test",
dbUser: "admin",
dbPassword: "admin",
dbAutoDropIfExists: false,
dbAutoCreate: true,
tx: false,
wal: false,
batchCommit: 1000,
dbType: "graph",
indexes: [{class:"Person", fields:["pid:string"],
type:"UNIQUE_HASH_INDEX" }]
}
}
}
======alien.json=====================
{
"config": {
"log": "debug"
},
"extractor" : {
"jdbc": { "driver": "org.postgresql.Driver",
"url": "jdbc:postgresql://localhost/testdb",
"userName": "postgres",
"userPassword": "password1",
"query": "select * from alien" }
},
"transformers" : [
{ "vertex": { "class": "Alien"} },
{ "edge": { "class": "hasLink", "direction" : "in",
"joinFieldName": "aid",
"lookup":"Person.pid", "unresolvedLinkAction":"CREATE"} }
],
"loader" : {
"orientdb": {
"dbURL": "plocal:/home/orientdb/db/databases/db_test",
dbUser: "admin",
dbPassword: "admin",
dbAutoDropIfExists: false,
dbAutoCreate: true,
tx: false,
wal: false,
batchCommit: 1000,
dbType: "graph",
indexes: [{class:"Alien", fields:["aid:string"],
type:"UNIQUE_HASH_INDEX" }]
}
}
}
On Mon, Sep 22, 2014 at 7:13 PM, Luca Garulli <[email protected]> wrote:
> Hi,
> Seems there is no Loader configured. Could you post also your JSON file?
>
> Lvc@
>
> On 22 September 2014 18:42, Olakunle Olaniyi <[email protected]> wrote:
>
>> Hi Luca,
>> I'd like to start by saying thanks for the explanation on this so far. I
>> am running orientdb 2.0-M1 and this is not working for me as well. It shows
>> the joinvalue alright and they are correct but the edges are not be
>> created. Have an explanation why this is so? Your response will be much
>> appreciated.
>>
>> Olakay
>>
>> P.S : the log could be see below :
>>
>>
>> OrientDB etl v.2.0-M1 (build @BUILD@) www.orientechnologies.com
>> BEGIN ETL PROCESSOR
>> [1:vertex] DEBUG Transformer input: {aid:1,name:Qualitrail - 1}
>> [1:vertex] DEBUG Transformer output: v(Alien)[#12:10]
>> [1:edge] DEBUG Transformer input: v(Alien)[#12:10]
>> [1:edge] DEBUG WARNING: index Person.mid not found. Lookups could be
>> really slow
>> [1:edge] DEBUG joinValue=1,
>> lookupResult=Person#11:0{mid:1,name:Qualitrail - 1,in_hasLink:[size=1]} v2
>> [1:edge] DEBUG Transformer output: v(Alien)[#12:10]
>> [3:vertex] DEBUG Transformer input: {aid:2,name:Qualitrail - 2}
>> [3:vertex] DEBUG Transformer output: v(Alien)[#12:11]
>> [3:edge] DEBUG Transformer input: v(Alien)[#12:11]
>> [3:edge] DEBUG joinValue=2,
>> lookupResult=Person#11:1{mid:2,name:Qualitrail - 2} v1
>> [3:edge] DEBUG Transformer output: v(Alien)[#12:11]
>> [5:vertex] DEBUG Transformer input: {aid:3,name:Qualitrail - 3}
>> [5:vertex] DEBUG Transformer output: v(Alien)[#12:12]
>> [5:edge] DEBUG Transformer input: v(Alien)[#12:12]
>> [5:edge] DEBUG joinValue=3,
>> lookupResult=Person#11:2{mid:3,name:Qualitrail - 3} v1
>> [5:edge] DEBUG Transformer output: v(Alien)[#12:12]
>> [7:vertex] DEBUG Transformer input: {aid:4,name:Qualitrail - 4}
>> [7:vertex] DEBUG Transformer output: v(Alien)[#12:13]
>> [7:edge] DEBUG Transformer input: v(Alien)[#12:13]
>> [7:edge] DEBUG joinValue=4,
>> lookupResult=Person#11:3{mid:4,name:Qualitrail - 4} v1
>> [7:edge] DEBUG Transformer output: v(Alien)[#12:13]
>> [9:vertex] DEBUG Transformer input: {aid:5,name:Qualitrail - 5}
>> [9:vertex] DEBUG Transformer output: v(Alien)[#12:14]
>> [9:edge] DEBUG Transformer input: v(Alien)[#12:14]
>> [9:edge] DEBUG joinValue=5,
>> lookupResult=Person#11:4{mid:5,name:Qualitrail - 5} v1
>> [9:edge] DEBUG Transformer output: v(Alien)[#12:14]
>> [11:vertex] DEBUG Transformer input: {aid:6,name:Qualitrail - 6}
>> [11:vertex] DEBUG Transformer output: v(Alien)[#12:15]
>> [11:edge] DEBUG Transformer input: v(Alien)[#12:15]
>> [11:edge] DEBUG joinValue=6,
>> lookupResult=Person#11:5{mid:6,name:Qualitrail - 6} v1
>> [11:edge] DEBUG Transformer output: v(Alien)[#12:15]
>> [13:vertex] DEBUG Transformer input: {aid:7,name:Qualitrail - 7}
>> [13:vertex] DEBUG Transformer output: v(Alien)[#12:16]
>> [13:edge] DEBUG Transformer input: v(Alien)[#12:16]
>> [13:edge] DEBUG joinValue=7,
>> lookupResult=Person#11:6{mid:7,name:Qualitrail - 7} v1
>> [13:edge] DEBUG Transformer output: v(Alien)[#12:16]
>> [15:vertex] DEBUG Transformer input: {aid:8,name:Qualitrail - 8}
>> [15:vertex] DEBUG Transformer output: v(Alien)[#12:17]
>> [15:edge] DEBUG Transformer input: v(Alien)[#12:17]
>> [15:edge] DEBUG joinValue=8,
>> lookupResult=Person#11:7{mid:8,name:Qualitrail - 8} v1
>> [15:edge] DEBUG Transformer output: v(Alien)[#12:17]
>> [17:vertex] DEBUG Transformer input: {aid:9,name:Qualitrail - 9}
>> [17:vertex] DEBUG Transformer output: v(Alien)[#12:18]
>> [17:edge] DEBUG Transformer input: v(Alien)[#12:18]
>> [17:edge] DEBUG joinValue=9,
>> lookupResult=Person#11:8{mid:9,name:Qualitrail - 9} v1
>> [17:edge] DEBUG Transformer output: v(Alien)[#12:18]
>> [19:vertex] DEBUG Transformer input: {aid:10,name:Qualitrail - 10}
>> [19:vertex] DEBUG Transformer output: v(Alien)[#12:19]
>> [19:edge] DEBUG Transformer input: v(Alien)[#12:19]
>> [19:edge] DEBUG joinValue=10,
>> lookupResult=Person#11:9{mid:10,name:Qualitrail - 10} v1
>> [19:edge] DEBUG Transformer output: v(Alien)[#12:19]
>> END ETL PROCESSOR
>>
>>
>> On Thursday, July 3, 2014 6:06:45 PM UTC+2, Lvc@ wrote:
>>>
>>> Hi guys,
>>> I'm glad to announce a new project under the Orient Technologies
>>> umbrella: *OrientDB-ETL*!
>>>
>>> This is the URL of WiKi:
>>> *https://github.com/orientechnologies/orientdb-etl/wiki
>>> <https://github.com/orientechnologies/orientdb-etl/wiki>*
>>>
>>> Thanks to OrientDB-ETL project, moving data from/to OrientDB is super
>>> fast and require no more code writing: only a JSON file.
>>>
>>> So far these are the available components:
>>>
>>> - *Extractors*:
>>> - File
>>> - *Transformers*:
>>> - CSV, to convert rows in documents
>>> - Link, to transform JOIN in links
>>> - Merge, to merge the parsed document with an existent
>>> - Vertex, to convert a document in Vertex
>>> - Edge, to create edges
>>> - Skip, to skip records based on expressions
>>> - Code, to execute arbitrary Javascript code
>>> - *Loaders*:
>>> - OrientDB, to save into a OrientDB database
>>>
>>> OrientDB-ETL module is in beta status, and will be final with OrientDB
>>> v2.0 (September). To use it against OrientDB 2.0-SNAPSHOT follow the
>>> "Installation" instructions.
>>>
>>> Below an example to import *DBPedia* *csv* files (
>>> http://wiki.dbpedia.org/DBpediaAsTables):
>>>
>>> {
>>> config: {
>>> verbose: true,
>>> fileDirectory: "/temp/databases/dbpedia_csv/",
>>> fileName: "Person.csv.gz"
>>> },
>>> begin: [
>>> { let: { name: "$filePath", value: "$fileDirectory.append( $fileName
>>> )"} },
>>> { let: { name: "$className", value: "$fileName.substring( 0,
>>> $fileName.indexOf('.') )"} }
>>> ],
>>> extractor : {
>>> line: { path: "$filePath", lock : true }
>>> },
>>> transformers : [
>>> { csv: { separator: ",", nullValue: "NULL", skipFrom: 1, skipTo: 3 }
>>> },
>>> { merge: { joinFieldName:"URI", lookup:"V.URI" } },
>>> { vertex: { class: "$className"} }
>>> ],
>>> loader : {
>>> orientdb: {
>>> dbURL: "plocal:/temp/databases/dbpedia",
>>> dbUser: "admin",
>>> dbPassword: "admin",
>>> dbAutoCreate: true,
>>> tx: false,
>>> batchCommit: 1000,
>>> dbType: "graph",
>>> indexes: [{class:"V", fields:["URI:string"], type:"UNIQUE" }]
>>> }
>>> }
>>> }
>>>
>>> For more information: https://github.com/orientechnologies/
>>> orientdb-etl/wiki/Import-from-DBPedia.
>>>
>>> Lvc@
>>>
>>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "OrientDB" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "OrientDB" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/orient-database/Xp38b55AbpU/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> [email protected].
> For more options, visit https://groups.google.com/d/optout.
>
--
Olakunle
--
---
You received this message because you are subscribed to the Google Groups
"OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
