Hi there Joel. I'm really sorry that the migration of your data from v3 to v4 is becoming frustrating for you.
Please know that we are trying our best to get as much functionality into the system as possible. However, as I'm sure you can appreciate, there are limits to what we can do and tough decisions have to be made on a regular basis, more often than not concerning the most difficult ones--i.e., which functionality we should include/exclude. We specifically discussed writing a parser to help transfer one's data from v3 to v4 when we were defining the scope and budget for v4, but as often is the case, it becomes more complicated the closer you look at it, especially since we want to make any code as generic and possible. In this case, we made the decision that this is likely something that could/would be contributed by someone in the community. The idea was that someone with experience implementing v3 would contribute a script to the community in the process of upgrading to v4. We assumed, perhaps mistakenly, that most would not need to upgrade immediately. You may be the first to upgrade with complex data! My take on the documentation is that it is something that can always be improved on, regardless. But, once again, we are relying on the community to help us out, because only through real use can we tell what needs more clarification. We are continually engaged in improving it. I will look into seeing if I can find someone whom I could encourage to help you out, but I can't promise. It seems that everyone I know is faced with tight deadlines at the moment, and there's more stress than usual going around. If I can find any help, I'll let you know. Best, and sorry once again that this has become frustrating. Alison On Saturday, May 6, 2017 at 12:27:12 AM UTC-7, Joel Aldor wrote: > > Hi Ryan, > > I was able to download the export files (it's 33MB total). But I don't > think we have the time and resources right now to write a parser to convert > the JSON file to a CSV file for version 4. Isn't this something that you > should be supporting instead because you changed the data model? We don't > even know the exact changes v3 to v4 as far as the data models are, and > you're expecting us to write the parser already? > > We need to have this up and running by the end of this month. What are our > other options at the moment to complete this migration process? > > Regards, > > Joel > > On Saturday, May 6, 2017 at 1:37:26 PM UTC+8, Ryan Anderson wrote: >> >> Hi Joel, >> >> >> Have you looked into the v3 export_resources command? (python manage.py >> packages -o export_resources -d 'path_to_destination') This might fit the >> bill for what you're doing. It should export business data as json (with >> all your groups intact I believe). From there you can write a parser to >> convert that json to a v4 csv file. >> >> Ryan >> >> On May 5, 2017 at 10:11 PM, <Joel Aldor> wrote: >> >> Hi all, >> >> I'm trying to export resources from Arches version 3 into a CSV file that >> I can manipulate for version 4 upload, as recommended by Adam >> <https://groups.google.com/d/msg/archesproject/rE9ipvxHso4/fQuB3tGRBgAJ>. >> I'm looking into this documentation for v3 >> <http://arches-hip.readthedocs.io/en/latest/system-administration/#exporting-data> >> >> which lets me configure resource export mappings via a JSON file. I >> realized, however, that the output of this example is best for resources >> that only have single entries per field. Many of our historic resources >> have multiple data entries for several fields (e.g. resource description, >> classification, components, important dates, measurements, condition >> assessments, and lots more). And we are referring to thousands of historic >> resources that have these info. >> >> I tried using the example resource_export_mappings.json file and the >> exported CSV file only populates the latest entry for a certain field. >> Basically there's only one row per historic resource. We must be able to >> export all multiple entries per data field. How do you configure that JSON >> file to be able to populate all those fields in a CSV file? >> >> Requesting for you to look into this matter as this is a very major >> challenge for us to be able to export all business data from version 3 and >> import them to version 4. We hope to finish this upgrade before the end of >> May because we have an upcoming batch of volunteers to train next month. >> Didn't realize how hard this whole migration process is. >> >> Regards, >> >> Joel >> >> >> -- -- To post, send email to archesproject@googlegroups.com. To unsubscribe, send email to archesproject+unsubscr...@googlegroups.com. For more information, visit https://groups.google.com/d/forum/archesproject?hl=en --- You received this message because you are subscribed to the Google Groups "Arches Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to archesproject+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
