Thanks Luca, I tried it out. The Person import works. But the more complex
vetrex and edge thing does not.
And with...
*{ "edge": { "class": "Person", " ...I get*
*Exception in thread "main" java.lang.IllegalArgumentException: Type error.
The class Person does not extend class 'E' and therefore cannot be
considered an Edge*So I thought it was just a mistake. I changed it to a
Class E called "hasPost". And now I get:
Exception in thread "main" java.lang.IllegalArgumentException: Property
value ca
n not be null
at com.tinkerpop.blueprints.util.ExceptionFactory.
propertyValueCanNotBeN
ull(ExceptionFactory.java:60)
at com.tinkerpop.blueprints.impls.orient.OrientElement.
validateProperty(
OrientElement.java:423)
at com.tinkerpop.blueprints.impls.orient.OrientElement.setProperty(
Orien
tElement.java:139)
at com.orientechnologies.orient.etl.transformer.OEdgeTransformer.
execute
Transform(OEdgeTransformer.java:102)
at com.orientechnologies.orient.etl.transformer.OAbstractTransformer
.tra
nsform(OAbstractTransformer.java:37)
at com.orientechnologies.orient.etl.OETLPipeline.execute(
OETLPipeline.ja
va:97)
at com.orientechnologies.orient.etl.OETLProcessor.
executeSequentially(OE
TLProcessor.java:459)
at com.orientechnologies.orient.etl.OETLProcessor.execute(
OETLProcessor.
java:253)
at com.orientechnologies.orient.etl.OETLProcessor.main(OETLProcessor
.jav
a:221)
+ extracted 1 records (0 records/sec) - 1 records -> loaded 0 vertices (0
vertic
es/sec) Total time: 1000ms [0 warnings, 0 errors]
+ extracted 1 records (0 records/sec) - 1 records -> loaded 0 vertices (0
vertic
es/sec) Total time: 2s [0 warnings, 0 errors]
Am Montag, 25. August 2014 16:11:32 UTC+2 schrieb Lvc@:
>
> Hi Curtis,
> getting your structure on MySQL:
>
> - Person{id,name}
> - Post{person_id,title}
>
> You could configure 2 pipelines:
>
> 1. one with extraction of "select * from Person"
> 2. one with extraction of "select * from Post"
>
>
> *Example for (1)*
>
> {
> "config": {
> "verbose": true
> },
> "extractor" : {
> "jdbc": { "driver": "com.mysql.jdbc.Driver",
> "url": "jdbc:mysql://localhost/mysql",
> "userName": "root",
> "userPassword": "",
> "query": "select * from Person" }
> },
> "transformers" : [
> { "vertex": { "class": "Person"} }
> ],
> "loader" : {
> "orientdb": {
> "dbURL": "plocal:/temp/databases/orientdb",
> "dbAutoCreate": true
> }
> }
> }
>
>
> *And this for (2):*
> {
> "config": {
> "verbose": true
> },
> "extractor" : {
> "jdbc": { "driver": "com.mysql.jdbc.Driver",
> "url": "jdbc:mysql://localhost/mysql",
> "userName": "root",
> "userPassword": "",
> "query": "select * from Post" }
> },
> "transformers" : [
> { "vertex": { "class": "Post"} },
> * { "edge": { "class": "Person", "direction" : "in", *
>
>
> * "joinFieldName": "person_id",
> "lookup":"Person.id", "unresolvedLinkAction":"CREATE"} }* ],
> "loader" : {
> "orientdb": {
> "dbURL": "plocal:/temp/databases/orientdb",
> "dbAutoCreate": true
> }
> }
> }
>
>
> Lvc@
>
> ᐧ
>
>
> On 25 August 2014 15:23, 'Curtis Mosters' via OrientDB <
> [email protected] <javascript:>> wrote:
>
>> Well there is an idea that might be easy to be implemented.
>>
>> So the idea is to *auto-create* an edge. Let's take an example:
>>
>> MySQL classes are:
>>
>> - Person{id,name}
>> - Post{,person_id,title}
>>
>> Now we also create such classes in OrientDB. Now we tell OrientDB if
>> there is INSERT INTO Person or Post -> create edge on "id" to "person_id".
>> I don't but isn't that pretty simple for a database to do?
>> Well I have to say all of that is possible easily done with Java API but
>> I really want it with ETL because there is no real need for me to use the
>> Java API.
>>
>> Am Montag, 25. August 2014 14:15:00 UTC+2 schrieb Curtis Mosters:
>>
>>> Could you maybe take some time to create such an example?
>>>
>>> The current "edge" is bit hard to understand:
>>>
>>> edge: {
>>> class: "hasPost",
>>> joinFieldName: "person_id",
>>> lookup: "person.id",
>>> unresolvedLinkAction: "CREATE",
>>> if: "person_id is not null"
>>> }
>>>
>>> How can I say here to Link from person on post? I just need a small
>>> example and also the MySQL strcuture if possible. That would make
>>> everything clear. Thanks Luca.
>>>
>>> Am Montag, 25. August 2014 13:23:55 UTC+2 schrieb Lvc@:
>>>>
>>>> Curtis,
>>>> You could create 2 or more etl scripts, some that extract vertices and
>>>> other that extract edges.
>>>>
>>>> Lvc@
>>>>
>>>> ᐧ
>>>>
>>>>
>>>> On 25 August 2014 12:25, 'Curtis Mosters' via OrientDB <
>>>> [email protected]> wrote:
>>>>
>>>>> Hi Luca, well your example is just reading one table from MySQL. One
>>>>> table with creating edges on it's entries. But in the usual cases that's
>>>>> not the way how to import from a MySQL. Or maybe I just didn't understood
>>>>> that example completely. But so far as I have seen it, you used one table
>>>>> from MySQL that contained {id,...,friend_id}. You created profiles on
>>>>> that
>>>>> id and then created edge on that id's. Is that right?
>>>>>
>>>>> But just once again I have those data as usually stored in another
>>>>> table. So that cannot work for me.
>>>>>
>>>>> Am Donnerstag, 21. August 2014 15:56:05 UTC+2 schrieb Lvc@:
>>>>>>
>>>>>> ᐧ
>>>>>> On 21 August 2014 14:45, 'Curtis Mosters' via OrientDB <
>>>>>> [email protected]> wrote:
>>>>>>
>>>>>>> This is a great example Luca, thanks first of all.
>>>>>>>
>>>>>>> But could you additionally add information about the MySQL schema
>>>>>>> and do I understand it correctly that you get the classes Person,Friend
>>>>>>> and
>>>>>>> an edge. But where comes the friend data?
>>>>>>>
>>>>>>
>>>>>>
>>>>>> Friend in my case is an Edge class. (create class Friend extends E).
>>>>>> It's useful to create ad-hoc edge classes instead that the generic "E":
>>>>>>
>>>>>> http://www.orientechnologies.com/docs/1.7.8/orientdb.wiki/Gr
>>>>>> aph-Schema.html
>>>>>>
>>>>>> Lvc@
>>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Am Donnerstag, 21. August 2014 13:25:47 UTC+2 schrieb Lvc@:
>>>>>>>>
>>>>>>>> Hi Curtis,
>>>>>>>> ETL is very powerful and we're improving it everyday with users'
>>>>>>>> feedback. We want to let it to be final for 2.0, so we can bundle with
>>>>>>>> it.
>>>>>>>> By the way Enterprise Edition
>>>>>>>> <http://www.orientechnologies.com/orientdb-enterprise> will have a
>>>>>>>> Web Interface to edit it, and in the future also to debug it.
>>>>>>>>
>>>>>>>> Unfortunately I don't understand in your example table1 & 2 with
>>>>>>>> both name_id field. Look at this example about Friendship. I want to
>>>>>>>> connect a social network like app.
>>>>>>>>
>>>>>>>> This is an example of ETL configuration. My comments begins with //
>>>>>>>> but remove them from real config file:
>>>>>>>>
>>>>>>>> {
>>>>>>>> config: {
>>>>>>>> verbose: true
>>>>>>>> },
>>>>>>>> begin: [
>>>>>>>> ],
>>>>>>>> extractor: {
>>>>>>>> "jdbc": { "driver": "com.mysql.jdbc.Driver",
>>>>>>>> "url": "jdbc:mysql://localhost/mysocialnetwork",
>>>>>>>> "userName": "root",
>>>>>>>> "userPassword": "root",
>>>>>>>> "query": "select * from profile"
>>>>>>>> }
>>>>>>>> },
>>>>>>>> transformers : [
>>>>>>>> {
>>>>>>>> // THIS IS ONLY A LOG, USEFUL TO TRACE WHAT ARRIVES FROM MYSQL
>>>>>>>> log: {
>>>>>>>> prefix: "MySQL -> "
>>>>>>>> }
>>>>>>>> },
>>>>>>>> {
>>>>>>>> merge: {
>>>>>>>> // LOOKUP BY ID: IF ALREADY PRESENT MERGE THE FIELDS BY
>>>>>>>> OVERWRITING THE DIFFERENT ONES
>>>>>>>> joinFieldName: "*id*",
>>>>>>>> lookup: "Profile.*id*"
>>>>>>>> }
>>>>>>>> },
>>>>>>>> {
>>>>>>>> vertex: {
>>>>>>>> // TRANSFORM IT IN A VERTEX
>>>>>>>> class: "Profile"
>>>>>>>> }
>>>>>>>> },
>>>>>>>> {
>>>>>>>> edge: {
>>>>>>>> // CONNECT THE FRIEND IN MYSQL COLUMN 'friend_id'
>>>>>>>> class: "Friend", // Friend is the edges' class
>>>>>>>> joinFieldName: "friend_id", //
>>>>>>>> lookup: "Profile.id",
>>>>>>>> unresolvedLinkAction: "CREATE", // IF DOESN'T EXIST YET,
>>>>>>>> CREATE THE VERTEX EMPTY. WILL BE FURTHER UPDATED
>>>>>>>> if: "friend_id is not null"
>>>>>>>> }
>>>>>>>> },
>>>>>>>> {
>>>>>>>> log: {
>>>>>>>> prefix: "Vertex -> "
>>>>>>>> }
>>>>>>>> }
>>>>>>>> ],
>>>>>>>> loader : {
>>>>>>>> orientdb: {
>>>>>>>> dbURL: "remote:localhost/yourdb",
>>>>>>>> dbUser: "admin",
>>>>>>>> dbPassword: "admin",
>>>>>>>> dbAutoCreate: true,
>>>>>>>> tx: true,
>>>>>>>> batchCommit: 1000,
>>>>>>>> dbType: "graph",
>>>>>>>> classes: [
>>>>>>>> ],
>>>>>>>> indexes: [
>>>>>>>> ]
>>>>>>>> }
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>> Lvc@
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 21 August 2014 13:05, 'Curtis Mosters' via OrientDB <
>>>>>>>> [email protected]> wrote:
>>>>>>>>
>>>>>>>>> Hey so I have now some experiences with ETL. I think this is great
>>>>>>>>> for import. That's why I want to improve the importing process now.
>>>>>>>>>
>>>>>>>>> 1. First of all is it possible to run several imports
>>>>>>>>> parallel(not with several consoles) or if one is done start
>>>>>>>>> another
>>>>>>>>> instantly?
>>>>>>>>> 2. How do I create an edge (https://github.com/orientechn
>>>>>>>>> ologies/orientdb-etl/wiki/Transformer#edge
>>>>>>>>>
>>>>>>>>> <https://github.com/orientechnologies/orientdb-etl/wiki/Transformer#edge>
>>>>>>>>>
>>>>>>>>> is not helpful for me)
>>>>>>>>>
>>>>>>>>> Let's say I have 2 tables in MySQL:
>>>>>>>>>
>>>>>>>>> table1 with {*name_id*,*name*} and table2 with {*name_id*,
>>>>>>>>> *text_id*,*text*}
>>>>>>>>>
>>>>>>>>> Now I want to get them connected (name_id) in OrientDB.
>>>>>>>>>
>>>>>>>>> One way would now be to import both tables as Class. That's easy
>>>>>>>>> and then create edges with the *name_id*. Everything is fine with
>>>>>>>>> that, but it would be much easier to generate edges with ETL if that
>>>>>>>>> is
>>>>>>>>> anyway possible. So I don't know what is more efficient. Someone else
>>>>>>>>> also
>>>>>>>>> said that Sails (https://github.com/vjsrinath/sails-orientdb)
>>>>>>>>> might be a good choice.
>>>>>>>>>
>>>>>>>>> If ETL allows that I could imagine about the following way:
>>>>>>>>>
>>>>>>>>> - read line of table1 -> safe in memory
>>>>>>>>> - look in table2 if name_id exists
>>>>>>>>> - -> if yes, create vertex Tabel1 and Table2
>>>>>>>>> - -> if no, create vertex Tabel1 and Table2 and create an
>>>>>>>>> edge
>>>>>>>>> - and so on
>>>>>>>>>
>>>>>>>>> Of course this is very crazy if you imagine 30 tables or so. Was
>>>>>>>>> just an idea, just to check if that is already possible somehow.
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> ---
>>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>>> Groups "OrientDB" group.
>>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>>> send an email to [email protected].
>>>>>>>>>
>>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>
>>>>>>> ---
>>>>>>> You received this message because you are subscribed to the Google
>>>>>>> Groups "OrientDB" group.
>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>> send an email to [email protected].
>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>
>>>>>>
>>>>>> --
>>>>>
>>>>> ---
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "OrientDB" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected].
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>
>>>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "OrientDB" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
---
You received this message because you are subscribed to the Google Groups
"OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
