I tried to use ETL as described here
http://www.orientechnologies.com/docs/last/orientdb-etl.wiki/Import-from-DBMS.html
I used slightly modified approach (I don't know is it a correct one). I
tried to import BookAuthor as vertex first (went fast and with no
problems), than I tried to import Book as a vertex and to create out
relation from Book to BookAuthor. This lasted for ages (I have about 50 000
rows in Books table and 150 000 in BookAuthor)
{
"config": {
"log": "error"
},
"extractor" : {
"jdbc": { "driver": "com.microsoft.sqlserver.jdbc.SQLServerDriver",
"url": "jdbc:sqlserver://localhost:1433;DatabaseName=SomeDB",
"userName": "sa",
"fetchSize": 500,
"userPassword": "sa",
"query": "select * from BookAuthor" }
},
"transformers" : [
{ "vertex": { "class": "BookAuthor"} }
],
"loader" : {
"orientdb": {
"dbURL": "plocal:d:/orientdb/myorient",
"dbAutoCreate": true,
"dbType": "graph",
"batchCommit": 1000
}
}
}
{
"config": {
"log": "error"
},
"extractor" : {
"jdbc": { "driver": "com.microsoft.sqlserver.jdbc.SQLServerDriver",
"url": "jdbc:sqlserver://localhost:1433;DatabaseName=SomeDB",
"userName": "sa",
"fetchSize": 500,
"userPassword": "sa",
"query": "select * from Book" }
},
"transformers" : [
{ "vertex": { "class": "Book"} },
{ "edge": { "class": "Book_BookAuthor", "direction" : "out",
"joinFieldName": "bookId",
"lookup":"BookAuthor.bookId", "unresolvedLinkAction":"CREATE"}
}
],
"loader" : {
"orientdb": {
dbURL": "plocal:d:/orientdb/myorient",
"dbAutoCreate": true,
"dbType": "graph",
"batchCommit": 1000
}
}
}
On Monday, 20 October 2014 20:51:22 UTC+2, Lvc@ wrote:
>
> @Bojan, How did you import edges? Can you share the code? It's hard to
> help without any information.
>
> Lvc@
>
>
> On 20 October 2014 13:31, 'Curtis Mosters' via OrientDB <
> [email protected] <javascript:>> wrote:
>
>> Is you code secret? I still don't understand the real issue you have,
>> sorry.
>>
>> Am Montag, 20. Oktober 2014 15:26:27 UTC+2 schrieb Bojan Vukotić:
>>
>>>
>>> Well, that's exactly what I need (and what I already did). I thought
>>> maybe that ETL has some API that we could use to make this easier (I used
>>> 'pure' OrientDB API to implement this)
>>>
>>>
>>>
>>> On Monday, 20 October 2014 15:15:27 UTC+2, Curtis Mosters wrote:
>>>>
>>>> I think I don't understand your issue:
>>>>
>>>> First of all you import all Books. So you have all the data in there.
>>>> Now setting an index on the Book.ID.
>>>>
>>>> After that the Authors are imported. They are matched with the ID of
>>>> the Book. I think in your case the Author matched with a Book is the same
>>>> like the Book.ID? Or do you have another file containing the relations of
>>>> the ID's?
>>>>
>>>> Can you post here the first 10 lines of each file maybe? That would
>>>> help a low.
>>>>
>>>> Am Montag, 20. Oktober 2014 14:55:10 UTC+2 schrieb Bojan Vukotić:
>>>>>
>>>>> I took example from here http://www.orientechnologies.
>>>>> com/docs/last/orientdb-etl.wiki/Import-from-DBMS.html
>>>>>
>>>>> When can I find ETL/Java example?
>>>>>
>>>>>
>>>>>
>>>>> On Monday, 20 October 2014 14:43:11 UTC+2, Curtis Mosters wrote:
>>>>>>
>>>>>> Well I think it's way better to create a Java example. Then you
>>>>>> understand what is happening in the background. Otherwhise in my tests
>>>>>> the
>>>>>> ETL way had the same speed, but these tests are 3-4 month old. I will
>>>>>> redo
>>>>>> them soon. Did you take the example of ETL from OrientDB? Otherwhise
>>>>>> look
>>>>>> above for some examples. Or even post yours here?
>>>>>>
>>>>>> Am Montag, 20. Oktober 2014 12:36:57 UTC+2 schrieb Bojan Vukotić:
>>>>>>>
>>>>>>>
>>>>>>> I prefer do it with ETL, if it is possible, I would like to avoid
>>>>>>> programming. If not, Java is also a good solution.
>>>>>>>
>>>>>>> So, example how to do it in ETL? And regarding ETL, I was playing
>>>>>>> with it, it imports vertices nicely, but when I want to import edges
>>>>>>> (100
>>>>>>> 000 of them) it is extremely slow :( How to improve this?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Monday, 20 October 2014 12:16:08 UTC+2, Curtis Mosters wrote:
>>>>>>>>
>>>>>>>> Well you have several ways. Do you want to do it with JAVA oder the
>>>>>>>> ETL plugin?
>>>>>>>>
>>>>>>>> In any case I think it should be
>>>>>>>>
>>>>>>>> Vertices: Author, Book
>>>>>>>> Edge: WROTE
>>>>>>>>
>>>>>>>> WDYT?
>>>>>>>>
>>>>>>>> Am Montag, 20. Oktober 2014 10:51:03 UTC+2 schrieb Bojan Vukotić:
>>>>>>>>>
>>>>>>>>> Hi guys!
>>>>>>>>>
>>>>>>>>> The whole discussion here is how to create edges from one table to
>>>>>>>>> another, but what to do if we have more complex cases where we have
>>>>>>>>> connected 2 (or even more) tables? Example, n:n relation: book and
>>>>>>>>> authors,
>>>>>>>>> book can have one or more authors and author can work on one or more
>>>>>>>>> books.
>>>>>>>>>
>>>>>>>>> Tables:
>>>>>>>>>
>>>>>>>>> Book {
>>>>>>>>> book_id
>>>>>>>>> book_name,
>>>>>>>>> ....
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> Author {
>>>>>>>>> author_id,
>>>>>>>>> author_name,
>>>>>>>>> ......
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Author_on_Book {
>>>>>>>>> ab_id,
>>>>>>>>> book_id,
>>>>>>>>> author_id,
>>>>>>>>> description // describes what this author did on this book
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> How to migrate this case? Should "Author_on_Book" be migrated as a
>>>>>>>>> vertex or edge? How to write scripts in this case? (in real life we
>>>>>>>>> could
>>>>>>>>> have even more foreign keys in "Author_on_Book" table )
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "OrientDB" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
---
You received this message because you are subscribed to the Google Groups
"OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
