I'm still very interested in this ETL Loader. Just checking if it's faster than my current Loading. So if interested @sarav I'm tomorrow online in gitter.im and we could try together? :)
Am Sonntag, 7. September 2014 13:15:14 UTC+2 schrieb Lvc@: > > Hi Sarav, > Please could you post your config .json file here? > > Lvc@ > > > > On 7 September 2014 05:41, Sarav <[email protected] <javascript:>> > wrote: > >> I got the same issue - basically not able to load this CSV file. I do >> have a class Country and I am able to insert into Class thru console >> commands. Thanks. >> >> On Friday, 29 August 2014 03:58:22 UTC+8, Lvc@ wrote: >>> >>> Curtis, >>> You have the skip in your cfg: >>> >>> { "csv": { "separator": ",", "nullValue": "NULL", "skipFrom": 2, >>> "skipTo": 2 } }, >>> >>> So all the records are skipped. We tried with big CSV file and >>> everything worked like a sharm. >>> >>> If you have a CSV source you can't import, just send it to me. >>> >>> Lvc@ >>> >>> ᐧ >>> >>> >>> On 28 August 2014 21:34, 'Curtis Mosters' via OrientDB < >>> [email protected]> wrote: >>> >>>> Well just take the example from above. >>>> >>>> When I get this small example to run, I think a bigger one is easy to >>>> manage. =) >>>> >>>> Am Donnerstag, 28. August 2014 20:23:43 UTC+2 schrieb Lvc@: >>>>> >>>>> Hi, >>>>> Ok, we have the JSON for configuration, now we'd need the CSV to >>>>> reproduce your problem. If it's sensitive, can you send me in private? >>>>> >>>>> Lvc@ >>>>> >>>>> ᐧ >>>>> >>>>> >>>>> On 28 August 2014 20:17, Lvc@ <[email protected]> wrote: >>>>> >>>>>> The CSV extractor couldn't find a csv, so it returned null and the >>>>>> pipeline skipped the execution. >>>>>> >>>>>> Lvc@ >>>>>> >>>>>> >>>>>> On Tuesday, 26 August 2014 17:20:01 UTC+2, Curtis Mosters wrote: >>>>>>> >>>>>>> Well instead of using a JDBC connection to MySQL I instead now want >>>>>>> to use a *CSV *file to load from. >>>>>>> >>>>>>> So I tried out many things and again ETL is having huge problems. >>>>>>> Well it still seems pretty unstable and not well documented. The only >>>>>>> tutorial in the internet is the one about the DBPedia. Well I don't >>>>>>> know >>>>>>> that plattform this is but for sure it's the same way of an usual CSV >>>>>>> like: >>>>>>> >>>>>>> id,name >>>>>>> 1,Name1 >>>>>>> 2,Name2 >>>>>>> and so on >>>>>>> >>>>>>> So I tried it with an own example: >>>>>>> >>>>>>> { >>>>>>> "config": { >>>>>>> "verbose": true, >>>>>>> "fileDirectory": "C:/Users/kwoxer/Desktop/DB - >>>>>>> orientdb/bin/backup/csv-etl/", >>>>>>> "fileName": "Person.csv.gz" >>>>>>> }, >>>>>>> "begin": [ >>>>>>> { "let": { "name": "$filePath", "value": "$fileDirectory.append( >>>>>>> $fileName )"} }, >>>>>>> { "let": { "name": "$className", "value": "$fileName.substring( >>>>>>> 0, $fileName.indexOf(".") )"} } >>>>>>> ], >>>>>>> "source" : { >>>>>>> "file": { "path": "$filePath", "lock" : false } >>>>>>> }, >>>>>>> "extractor" : { >>>>>>> "row": {} >>>>>>> }, >>>>>>> "transformers" : [ >>>>>>> { "csv": { "separator": ",", "nullValue": "NULL", "skipFrom": 2, >>>>>>> "skipTo": 2 } }, >>>>>>> { "vertex": { "class": "$className"} } >>>>>>> ], >>>>>>> "loader" : { >>>>>>> "orientdb": { >>>>>>> "dbURL": "plocal:C:\Users\kwoxer\Desktop\DB - >>>>>>> orientdb\databases\Test", >>>>>>> "dbUser": "root", >>>>>>> "dbPassword": "root", >>>>>>> "dbAutoCreate": true, >>>>>>> "tx": false, >>>>>>> "batchCommit": 1000, >>>>>>> "dbType": "graph", >>>>>>> "indexes": [{"class":"V", "fields":["id:string"], "type": >>>>>>> "UNIQUE" }] >>>>>>> } >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> But when I run this I get: >>>>>>> >>>>>>> C:\Users\kwoxer\Desktop\DB - orientdb\bin>oetl.bat backup\csv-etl\ >>>>>>> person.json >>>>>>> OrientDB etl v.1.7.8 (build @BUILD@) www.orientechnologies.com >>>>>>> BEGIN ETL PROCESSOR >>>>>>> >>>>>>> 2014-08-26 17:08:26:501 WARN Transformer [com.orientechnologies. >>>>>>> orient.etl.trans >>>>>>> former.OCSVTransformer@107598d7] returned null, skip rest of >>>>>>> pipeline execution >>>>>>> [OETLPipeline]END ETL PROCESSOR >>>>>>> + extracted 1 rows (0 rows/sec) - 1 rows -> loaded 0 vertices (0 >>>>>>> vertices/sec) T >>>>>>> otal time: 35ms [0 warnings, 0 errors] >>>>>>> >>>>>>> Some Transformer Warning, well due there is no real example about a >>>>>>> normal CSV I cannot do anything. I also don't understand why the >>>>>>> "skip"'s >>>>>>> are mandatory. Why should I skip line in a CSV? Is this just for >>>>>>> DBPedia >>>>>>> where comments might happen? >>>>>>> >>>>>>> Could some please add more examples. I just want to import a CSV >>>>>>> with ETL nothing else. Thanks. >>>>>>> >>>>>>> BTW: I also tried the unzipped version: >>>>>>> >>>>>>> "fileName": "Person.csv" >>>>>>> >>>>>>> Same result...: >>>>>>> >>>>>>> C:\Users\kwoxer\Desktop\DB - orientdb\bin>oetl.bat backup\csv-etl\ >>>>>>> person.json >>>>>>> OrientDB etl v.1.7.8 (build @BUILD@) www.orientechnologies.com >>>>>>> BEGIN ETL PROCESSOR >>>>>>> >>>>>>> 2014-08-26 17:18:21:189 WARN Transformer [com.orientechnologies. >>>>>>> orient.etl.trans >>>>>>> former.OCSVTransformer@1747c] returned null, skip rest of pipeline >>>>>>> execution [OE >>>>>>> TLPipeline]END ETL PROCESSOR >>>>>>> + extracted 1 rows (0 rows/sec) - 1 rows -> loaded 0 vertices (0 >>>>>>> vertices/sec) T >>>>>>> otal time: 25ms [0 warnings, 0 errors] >>>>>>> >>>>>>> >>>>>>> -- >>>>>> >>>>>> --- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "OrientDB" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>> >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "OrientDB" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "OrientDB" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > > -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
