Curtis, You could create 2 or more etl scripts, some that extract vertices and other that extract edges.
Lvc@ ᐧ On 25 August 2014 12:25, 'Curtis Mosters' via OrientDB < [email protected]> wrote: > Hi Luca, well your example is just reading one table from MySQL. One table > with creating edges on it's entries. But in the usual cases that's not the > way how to import from a MySQL. Or maybe I just didn't understood that > example completely. But so far as I have seen it, you used one table from > MySQL that contained {id,...,friend_id}. You created profiles on that id > and then created edge on that id's. Is that right? > > But just once again I have those data as usually stored in another table. > So that cannot work for me. > > Am Donnerstag, 21. August 2014 15:56:05 UTC+2 schrieb Lvc@: >> >> ᐧ >> On 21 August 2014 14:45, 'Curtis Mosters' via OrientDB < >> [email protected]> wrote: >> >>> This is a great example Luca, thanks first of all. >>> >>> But could you additionally add information about the MySQL schema and do >>> I understand it correctly that you get the classes Person,Friend and an >>> edge. But where comes the friend data? >>> >> >> >> Friend in my case is an Edge class. (create class Friend extends E). It's >> useful to create ad-hoc edge classes instead that the generic "E": >> >> http://www.orientechnologies.com/docs/1.7.8/orientdb.wiki/ >> Graph-Schema.html >> >> Lvc@ >> >> >> >>> >>> Am Donnerstag, 21. August 2014 13:25:47 UTC+2 schrieb Lvc@: >>>> >>>> Hi Curtis, >>>> ETL is very powerful and we're improving it everyday with users' >>>> feedback. We want to let it to be final for 2.0, so we can bundle with it. >>>> By the way Enterprise Edition >>>> <http://www.orientechnologies.com/orientdb-enterprise> will have a Web >>>> Interface to edit it, and in the future also to debug it. >>>> >>>> Unfortunately I don't understand in your example table1 & 2 with both >>>> name_id field. Look at this example about Friendship. I want to connect a >>>> social network like app. >>>> >>>> This is an example of ETL configuration. My comments begins with // but >>>> remove them from real config file: >>>> >>>> { >>>> config: { >>>> verbose: true >>>> }, >>>> begin: [ >>>> ], >>>> extractor: { >>>> "jdbc": { "driver": "com.mysql.jdbc.Driver", >>>> "url": "jdbc:mysql://localhost/mysocialnetwork", >>>> "userName": "root", >>>> "userPassword": "root", >>>> "query": "select * from profile" >>>> } >>>> }, >>>> transformers : [ >>>> { >>>> // THIS IS ONLY A LOG, USEFUL TO TRACE WHAT ARRIVES FROM MYSQL >>>> log: { >>>> prefix: "MySQL -> " >>>> } >>>> }, >>>> { >>>> merge: { >>>> // LOOKUP BY ID: IF ALREADY PRESENT MERGE THE FIELDS BY >>>> OVERWRITING THE DIFFERENT ONES >>>> joinFieldName: "*id*", >>>> lookup: "Profile.*id*" >>>> } >>>> }, >>>> { >>>> vertex: { >>>> // TRANSFORM IT IN A VERTEX >>>> class: "Profile" >>>> } >>>> }, >>>> { >>>> edge: { >>>> // CONNECT THE FRIEND IN MYSQL COLUMN 'friend_id' >>>> class: "Friend", // Friend is the edges' class >>>> joinFieldName: "friend_id", // >>>> lookup: "Profile.id", >>>> unresolvedLinkAction: "CREATE", // IF DOESN'T EXIST YET, CREATE >>>> THE VERTEX EMPTY. WILL BE FURTHER UPDATED >>>> if: "friend_id is not null" >>>> } >>>> }, >>>> { >>>> log: { >>>> prefix: "Vertex -> " >>>> } >>>> } >>>> ], >>>> loader : { >>>> orientdb: { >>>> dbURL: "remote:localhost/yourdb", >>>> dbUser: "admin", >>>> dbPassword: "admin", >>>> dbAutoCreate: true, >>>> tx: true, >>>> batchCommit: 1000, >>>> dbType: "graph", >>>> classes: [ >>>> ], >>>> indexes: [ >>>> ] >>>> } >>>> } >>>> } >>>> >>>> >>>> Lvc@ >>>> >>>> >>>> >>>> On 21 August 2014 13:05, 'Curtis Mosters' via OrientDB < >>>> [email protected]> wrote: >>>> >>>>> Hey so I have now some experiences with ETL. I think this is great for >>>>> import. That's why I want to improve the importing process now. >>>>> >>>>> 1. First of all is it possible to run several imports parallel(not >>>>> with several consoles) or if one is done start another instantly? >>>>> 2. How do I create an edge (https://github.com/orientechn >>>>> ologies/orientdb-etl/wiki/Transformer#edge >>>>> >>>>> <https://github.com/orientechnologies/orientdb-etl/wiki/Transformer#edge> >>>>> is not helpful for me) >>>>> >>>>> Let's say I have 2 tables in MySQL: >>>>> >>>>> table1 with {*name_id*,*name*} and table2 with {*name_id*,*text_id*, >>>>> *text*} >>>>> >>>>> Now I want to get them connected (name_id) in OrientDB. >>>>> >>>>> One way would now be to import both tables as Class. That's easy and >>>>> then create edges with the *name_id*. Everything is fine with that, >>>>> but it would be much easier to generate edges with ETL if that is anyway >>>>> possible. So I don't know what is more efficient. Someone else also said >>>>> that Sails (https://github.com/vjsrinath/sails-orientdb) might be a >>>>> good choice. >>>>> >>>>> If ETL allows that I could imagine about the following way: >>>>> >>>>> - read line of table1 -> safe in memory >>>>> - look in table2 if name_id exists >>>>> - -> if yes, create vertex Tabel1 and Table2 >>>>> - -> if no, create vertex Tabel1 and Table2 and create an edge >>>>> - and so on >>>>> >>>>> Of course this is very crazy if you imagine 30 tables or so. Was just >>>>> an idea, just to check if that is already possible somehow. >>>>> >>>>> -- >>>>> >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "OrientDB" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "OrientDB" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > > --- > You received this message because you are subscribed to the Google Groups > "OrientDB" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
