Knut, Unless it was enhanced during the last 2-3 days, then it is still very slow compared to other data imports (e.g importing 5.6 mill datavalue records take around 10 minutes on my laptop - importing ~29,000 events with 250k event values took about twice that)
regards calle On 12 March 2015 at 04:11, Knut Staring <[email protected]> wrote: > Also I think the performance was enhanced recently. > On 11 Mar 2015 23:09, "Pierre Dane" <[email protected]> wrote: > >> hi Calle >> >> FYI I have some database scripts that I used to import events into our >> program - 300k records in 10 mins or so. if you need to bulk insert events >> let me know and I can share. p >> >> >> >> On 11 Mar 2015, at 23:49, Calle Hedberg <[email protected]> wrote: >> >> Hi >> >> There seems to be at least two bugs related to the import of event-data >> in CSV format introduced in 2.17. >> >> Firstly, a dry run importing around 29,000 events (~250,000 trackervalue >> records) into 2.18 (version as of 09 Mar 2015) did not seem to work - no >> feedback within reasonable time. NOTE that the import with dry run set to >> NO worked fine, but it was quite slow - so there is a possibility the dry >> run actually works BUT since it gives no indication of activity for a LOONG >> time it is perceived as not working. If that is the case, it is important >> to add a counter or similar to the user messages (e.g. "1000 events >> imported" .... "2000 events imported" etc). >> >> Secondly, DHIS2 has a confusing set of standards for uppercase and >> lowercase keywords. When you export some event data to CSV, it will export >> the content of "ProvidedElsewhere" as it is stored: e.g. "FALSE" as an >> uppercase word. If you try to import the same file, it is rejected and the >> tomcat log indicates that only "false" and "true" are accepted (lowercase). >> >> Thirdly, from a more "cosmetic" perspective, I find it annoying that a >> tomcat log entry is produced for EVERY event imported - it would be better >> if only "abnormal" issues are logged otherwise any abnormalities or >> warnings drowns in the "info about every event record" clutter. >> >> Regards >> Calle >> >> ******************************************* >> >> Calle Hedberg >> >> 46D Alma Road, 7700 Rosebank, SOUTH AFRICA >> >> Tel/fax (home): +27-21-685-6472 >> >> Cell: +27-82-853-5352 >> >> Iridium SatPhone: +8816-315-19274 >> >> Email: [email protected] >> >> Skype: calle_hedberg >> >> ******************************************* >> >> _______________________________________________ >> Mailing list: https://launchpad.net/~dhis2-devs >> Post to : [email protected] >> Unsubscribe : https://launchpad.net/~dhis2-devs >> More help : https://help.launchpad.net/ListHelp >> >> >> _______________________________________________ >> Mailing list: https://launchpad.net/~dhis2-devs >> Post to : [email protected] >> Unsubscribe : https://launchpad.net/~dhis2-devs >> More help : https://help.launchpad.net/ListHelp >> >> -- ******************************************* Calle Hedberg 46D Alma Road, 7700 Rosebank, SOUTH AFRICA Tel/fax (home): +27-21-685-6472 Cell: +27-82-853-5352 Iridium SatPhone: +8816-315-19274 Email: [email protected] Skype: calle_hedberg *******************************************
_______________________________________________ Mailing list: https://launchpad.net/~dhis2-devs Post to : [email protected] Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
