Re: [Archivesspace_Users_Group] batch delete dates saga continues
Hey and thanks for the plug, Patrick! My co-workshop lead, Valerie Addonizio, and I tried to make the resources we put up in that GitHub stand on their own without the in-person part of the workshop. It was really important to us that everyone – not just those able to get to and pay for a 1-day workshop – be able to tinker about. As Patrick has said, tinkering is the BEST way to learn. It’s certainly how I did. If anyone wants assistance in working through these workshop materials, definitely give a shout. Like all of the rest of you, I have a super busy day job, but I’m happy to help as time allows! Regarding the time to do this, everyone’s situation is going to be different, but if your institution supports the idea of professional development time/personal goals, it can be a good idea to talk about this with supervisors and make it an articulated part of your job. Last year my “personal goal” was to increase my proficiency with Rails, so I tried to set aside an hour or so every couple weeks to dedicate myself to that, in the same way that I try to stay on top of the literature in ours and allied fields. Your learning style may vary, but I find it is best to just set aside a little bit of time over a long period, rather than trying to “cram” everything at once. Approach it as you would approach learning a foreign language. You aren’t going to do it in one weekend, one month, or, even, one year. It will grow naturally over time as you expose yourself to it. There are SO MANY wonderful resources out there (most for free), and you’ve got a community of folks who can help connect you to them. I won’t go too much into specifics as my answers will vary depending on what you want to do/what your specific setup looks like, but Code Academy, Lynda.com, the Hitchhiker’s Guide to Python (http://docs.python-guide.org/en/latest/), and some of the “better” coding boot camps (https://www.switchup.org/research/best-coding-bootcamps) offer free online courses (but, be forewarned, if you sign up with one of them they WILL hound you to apply to their programs and actively try and recruit you to their school ;-) ). Good luck, all, Lora -- Lora J. Woodford The Sheridan Libraries Johns Hopkins University 3400 North Charles Street Baltimore, MD 21228 (410) 516-5898 lwoodf...@jhu.edu<mailto:lwoodf...@jhu.edu> From: <archivesspace_users_group-boun...@lyralists.lyrasis.org> on behalf of "Galligan, Patrick" <pgalli...@rockarch.org> Reply-To: Archivesspace Users Group <archivesspace_users_group@lyralists.lyrasis.org> Date: Monday, November 13, 2017 at 9:52 AM To: Archivesspace Users Group <archivesspace_users_group@lyralists.lyrasis.org> Subject: Re: [Archivesspace_Users_Group] batch delete dates saga continues Hey all, I think a great place to get started learning about Python, APIs, and ArchivesSpace would be some of the API tutorials that members of the ArchivesSpace community have taught. Lora Woodford from JHU taught a workshop about 4 months ago and made the materials available on Github: https://github.com/jhu-archives-and-manuscripts/ASpace_API_Workshop. Personally, I’ve found that I learn best by doing. So I learned what I did about the AS JSON responses by downloading Postman (https://www.getpostman.com/) and performing a few API calls against our development database. Also, back in 2014, Maureen Callahan and a few other archivists ran a blog about dealing with legacy data. They have some posts about AS API work. I personally liked this post as a beginner: https://icantiemyownshoes.wordpress.com/2014/08/01/a-very-brief-guide-to-deleting-records-with-the-archivesspace-api-from-a-total-tyro/. This is a great resource for learning python: https://automatetheboringstuff.com/. I’m sure there are a ton of other resources out there, but those are the few off the top of my head. I’m willing to jump in and give some more recommendations as needed. -Patrick Galligan From: archivesspace_users_group-boun...@lyralists.lyrasis.org [mailto:archivesspace_users_group-boun...@lyralists.lyrasis.org] On Behalf Of Margaret Kidd Sent: Friday, November 10, 2017 5:31 PM To: Archivesspace Users Group <archivesspace_users_group@lyralists.lyrasis.org> Subject: Re: [Archivesspace_Users_Group] batch delete dates saga continues I second that last part of Lydia's remarks about where to begin learning more about XML, JSON, python, ruby, etc. and applying it to ArchivesSpace. I want to learn, but it is overwhelming to know where to begin and my time to devote to it is rather limited. Every time I start trying to teach myself I have other work priorities that take up all my time and soon forget whatever progress I have made. Thanks, Margaret Margaret T. Kidd Project Archivist, Special Collections & Archives VCU Libraries | Tompkins-McCaw Library for the Health Sciences 509 N. 12th Street / B
Re: [Archivesspace_Users_Group] batch delete dates saga continues
Along this line I’ve been struggling with finding an authoritative data dictionary for all of ASpace’s model classes and all the possible endpoint parameters. The API documentation provides much of this but not seemingly all, esp. mixins. For example: I would like an application developer to modify the Digital Object CSV importer to add specific Note fields that do not exist in the DO import template or data map at http://archivesspace.org/using-archivesspace/migration-tools-and-data-mapping/. I can’t reasonably expect a developer to “learn by doing”, read code or test documents, or watch YouTube videos -- nor will their managers allow them to do any of that. Having such an authoritative data dictionary would help to build external stakeholder confidence in ASpace as a documented, robust, authoritative data management application. Or perhaps I’m missing some boat entirely? John John P. Rees Archivist and Digital Resources Manager History of Medicine Division National Library of Medicine 301-827-4510 From: Galligan, Patrick [mailto:pgalli...@rockarch.org] Sent: Monday, November 13, 2017 9:14 AM To: Archivesspace Users Group <archivesspace_users_group@lyralists.lyrasis.org> Subject: Re: [Archivesspace_Users_Group] batch delete dates saga continues Hey all, I think a great place to get started learning about Python, APIs, and ArchivesSpace would be some of the API tutorials that members of the ArchivesSpace community have taught. Lora Woodford from JHU taught a workshop about 4 months ago and made the materials available on Github: https://github.com/jhu-archives-and-manuscripts/ASpace_API_Workshop. Personally, I’ve found that I learn best by doing. So I learned what I did about the AS JSON responses by downloading Postman (https://www.getpostman.com/) and performing a few API calls against our development database. Also, back in 2014, Maureen Callahan and a few other archivists ran a blog about dealing with legacy data. They have some posts about AS API work. I personally liked this post as a beginner: https://icantiemyownshoes.wordpress.com/2014/08/01/a-very-brief-guide-to-deleting-records-with-the-archivesspace-api-from-a-total-tyro/. This is a great resource for learning python: https://automatetheboringstuff.com/. I’m sure there are a ton of other resources out there, but those are the few off the top of my head. I’m willing to jump in and give some more recommendations as needed. -Patrick Galligan From: archivesspace_users_group-boun...@lyralists.lyrasis.org<mailto:archivesspace_users_group-boun...@lyralists.lyrasis.org> [mailto:archivesspace_users_group-boun...@lyralists.lyrasis.org] On Behalf Of Margaret Kidd Sent: Friday, November 10, 2017 5:31 PM To: Archivesspace Users Group <archivesspace_users_group@lyralists.lyrasis.org<mailto:archivesspace_users_group@lyralists.lyrasis.org>> Subject: Re: [Archivesspace_Users_Group] batch delete dates saga continues I second that last part of Lydia's remarks about where to begin learning more about XML, JSON, python, ruby, etc. and applying it to ArchivesSpace. I want to learn, but it is overwhelming to know where to begin and my time to devote to it is rather limited. Every time I start trying to teach myself I have other work priorities that take up all my time and soon forget whatever progress I have made. Thanks, Margaret Margaret T. Kidd Project Archivist, Special Collections & Archives VCU Libraries | Tompkins-McCaw Library for the Health Sciences 509 N. 12th Street / Box 980582, Richmond, VA 23298-0582 (804) 828-3152 [em_twitter.png]<https://twitter.com/VCUTMLibrary>[em_fb.png]<https://www.facebook.com/VCUTMLib> [http://identity.vcu.edu/images/photos/vcu_brand_mark_email_sig.jpg]<http://www.vcu.edu/> [http://pages.shanti.virginia.edu/Virginia_Heritage/files/2013/01/vhpban.jpg] <http://vaheritage.org> On Fri, Nov 10, 2017 at 12:02 PM, Tang, Lydia <lta...@lib.msu.edu<mailto:lta...@lib.msu.edu>> wrote: Mark, Thank you for identifying my problem! Just for the sake of technological dummies like me, what should I do? Everything there (besides removing the tags is exactly as it spat out of ArchivesSpace, so I wonder if the export allowed the invalid characters? I also wonder if the importing process could be improved within the ArchivesSpace code to search for “aspace_" and not batch add it as well as recognize (or not export out in the first place) the "Linear Feet" / linear_feet controlled vocabulary? Ideally, after establishing with the database that linear_feet should publish as Linear Feet, I wish it would continue to recognize that rule when new stuff is imported in. I was also meaning poll people on “how do I even get started” with learning more about working the ArchivesSpace guts? I can understand EAD but obviously don’t know the wizardry possible with
Re: [Archivesspace_Users_Group] batch delete dates saga continues
Hey all, I think a great place to get started learning about Python, APIs, and ArchivesSpace would be some of the API tutorials that members of the ArchivesSpace community have taught. Lora Woodford from JHU taught a workshop about 4 months ago and made the materials available on Github: https://github.com/jhu-archives-and-manuscripts/ASpace_API_Workshop. Personally, I’ve found that I learn best by doing. So I learned what I did about the AS JSON responses by downloading Postman (https://www.getpostman.com/) and performing a few API calls against our development database. Also, back in 2014, Maureen Callahan and a few other archivists ran a blog about dealing with legacy data. They have some posts about AS API work. I personally liked this post as a beginner: https://icantiemyownshoes.wordpress.com/2014/08/01/a-very-brief-guide-to-deleting-records-with-the-archivesspace-api-from-a-total-tyro/. This is a great resource for learning python: https://automatetheboringstuff.com/. I’m sure there are a ton of other resources out there, but those are the few off the top of my head. I’m willing to jump in and give some more recommendations as needed. -Patrick Galligan From: archivesspace_users_group-boun...@lyralists.lyrasis.org [mailto:archivesspace_users_group-boun...@lyralists.lyrasis.org] On Behalf Of Margaret Kidd Sent: Friday, November 10, 2017 5:31 PM To: Archivesspace Users Group <archivesspace_users_group@lyralists.lyrasis.org> Subject: Re: [Archivesspace_Users_Group] batch delete dates saga continues I second that last part of Lydia's remarks about where to begin learning more about XML, JSON, python, ruby, etc. and applying it to ArchivesSpace. I want to learn, but it is overwhelming to know where to begin and my time to devote to it is rather limited. Every time I start trying to teach myself I have other work priorities that take up all my time and soon forget whatever progress I have made. Thanks, Margaret Margaret T. Kidd Project Archivist, Special Collections & Archives VCU Libraries | Tompkins-McCaw Library for the Health Sciences 509 N. 12th Street / Box 980582, Richmond, VA 23298-0582 (804) 828-3152 [em_twitter.png]<https://twitter.com/VCUTMLibrary>[em_fb.png]<https://www.facebook.com/VCUTMLib> [http://identity.vcu.edu/images/photos/vcu_brand_mark_email_sig.jpg]<http://www.vcu.edu/> [http://pages.shanti.virginia.edu/Virginia_Heritage/files/2013/01/vhpban.jpg] <http://vaheritage.org> On Fri, Nov 10, 2017 at 12:02 PM, Tang, Lydia <lta...@lib.msu.edu<mailto:lta...@lib.msu.edu>> wrote: Mark, Thank you for identifying my problem! Just for the sake of technological dummies like me, what should I do? Everything there (besides removing the tags is exactly as it spat out of ArchivesSpace, so I wonder if the export allowed the invalid characters? I also wonder if the importing process could be improved within the ArchivesSpace code to search for “aspace_" and not batch add it as well as recognize (or not export out in the first place) the "Linear Feet" / linear_feet controlled vocabulary? Ideally, after establishing with the database that linear_feet should publish as Linear Feet, I wish it would continue to recognize that rule when new stuff is imported in. I was also meaning poll people on “how do I even get started” with learning more about working the ArchivesSpace guts? I can understand EAD but obviously don’t know the wizardry possible with XML, there’s JSON, python, etc, and I would be interested in starting courses with Code Academy to learn, but I don’t even know where to begin. Advice appreciated! Lydia From: <archivesspace_users_group-boun...@lyralists.lyrasis.org<mailto:archivesspace_users_group-boun...@lyralists.lyrasis.org>> on behalf of "Custer, Mark" <mark.cus...@yale.edu<mailto:mark.cus...@yale.edu>> Reply-To: Archivesspace Users Group <archivesspace_users_group@lyralists.lyrasis.org<mailto:archivesspace_users_group@lyralists.lyrasis.org>> Date: Friday, November 10, 2017 at 11:50 AM To: Archivesspace Users Group <archivesspace_users_group@lyralists.lyrasis.org<mailto:archivesspace_users_group@lyralists.lyrasis.org>> Subject: Re: [Archivesspace_Users_Group] batch delete dates saga continues Lydia, In the EAD files that you attached, both have invalid XML characters in them (Unicode: 0x14). Those are easy to remove before re-importing in an XML editor like oXygen, but I'm curious how they got into ASpace in the first place? In any event, it's possible that that's what's blocking your imports this time around, and if that's the issue, if you just fix on those issues, then the ASpace importer won't tell you about the next issue until it runs again. In any event, I'd also suggest making the following changes to your XML file before re-importing, so perh
Re: [Archivesspace_Users_Group] batch delete dates saga continues
I second that last part of Lydia's remarks about where to begin learning more about XML, JSON, python, ruby, etc. and applying it to ArchivesSpace. I want to learn, but it is overwhelming to know where to begin and my time to devote to it is rather limited. Every time I start trying to teach myself I have other work priorities that take up all my time and soon forget whatever progress I have made. Thanks, Margaret -- Margaret T. Kidd Project Archivist, Special Collections & Archives VCU Libraries | Tompkins-McCaw Library for the Health Sciences 509 N. 12th Street / Box 980582, Richmond, VA 23298-0582 (804) 828-3152 [image: em_twitter.png] <https://twitter.com/VCUTMLibrary> [image: em_fb.png] <https://www.facebook.com/VCUTMLib> <http://www.vcu.edu/> <http://vaheritage.org> On Fri, Nov 10, 2017 at 12:02 PM, Tang, Lydia <lta...@lib.msu.edu> wrote: > Mark, > Thank you for identifying my problem! Just for the sake of technological > dummies like me, what should I do? Everything there (besides removing the > tags is exactly as it spat out of ArchivesSpace, so I wonder if > the export allowed the invalid characters? I also wonder if the importing > process could be improved within the ArchivesSpace code to search for > “aspace_" and not batch add it as well as recognize (or not export out in > the first place) the "Linear Feet" / linear_feet controlled vocabulary? > Ideally, after establishing with the database that linear_feet should > publish as Linear Feet, I wish it would continue to recognize that rule > when new stuff is imported in. > I was also meaning poll people on “how do I even get started” with > learning more about working the ArchivesSpace guts? I can understand EAD > but obviously don’t know the wizardry possible with XML, there’s JSON, > python, etc, and I would be interested in starting courses with Code > Academy to learn, but I don’t even know where to begin. Advice appreciated! > Lydia > > From: <archivesspace_users_group-boun...@lyralists.lyrasis.org> on behalf > of "Custer, Mark" <mark.cus...@yale.edu> > Reply-To: Archivesspace Users Group <archivesspace_users_group@ > lyralists.lyrasis.org> > Date: Friday, November 10, 2017 at 11:50 AM > To: Archivesspace Users Group <archivesspace_users_group@ > lyralists.lyrasis.org> > Subject: Re: [Archivesspace_Users_Group] batch delete dates saga continues > > Lydia, > > In the EAD files that you attached, both have invalid XML characters in > them (Unicode: 0x14). Those are easy to remove before re-importing in an > XML editor like oXygen, but I'm curious how they got into ASpace in the > first place? In any event, it's possible that that's what's blocking your > imports this time around, and if that's the issue, if you just fix on those > issues, then the ASpace importer won't tell you about the next issue until > it runs again. > > In any event, I'd also suggest making the following changes to your XML > file before re-importing, so perhaps the snag is a good thing for now : > > · ASpace adds "aspace_" to all of the @id values in the EAD file > upon export. If you don't remove those before reimporting, then on the > next export you'll get "aspace_aspace_". Removing them will invalidate the > EAD file, but ASpace doesn't care whether the file is valid or not upon > re-importing it. > · ASpace expects to have the database values for the controlled > value terms in the exports, not the translation values. So, if you don't > change things like "Linear Feet" to linear_feet and "Mixed Materials" to > mixed_materials, then you'll wind up with new database values in ASpace > after the import. Those of course can be merged after the fact, but that's > another step, and it would be unfortunate to have to do that on every > occasion. > > There's a lot more to say on the practice of re-importing EAD into ASpace, > but I haven't still come up with a great strategy for that, so basically we > try to avoid it > > Mark > > > -Original Message----- > From: archivesspace_users_group-boun...@lyralists.lyrasis.org [mailto: > archivesspace_users_group-boun...@lyralists.lyrasis.org] On Behalf Of > Tang, Lydia > Sent: Friday, 10 November, 2017 10:47 AM > To: Archivesspace Users Group <archivesspace_users_group@ > lyralists.lyrasis.org> > Subject: Re: [Archivesspace_Users_Group] batch delete dates saga continues > > Wizardry! Thank you, Olivia! I would have NEVER figured that out. It > seemed to take out the tags perfectly but I ended up getting > snagged on importing because of, um, a seemingly benign line which was > marked
Re: [Archivesspace_Users_Group] batch delete dates saga continues
Lydia, In the EAD files that you attached, both have invalid XML characters in them (Unicode: 0x14). Those are easy to remove before re-importing in an XML editor like oXygen, but I'm curious how they got into ASpace in the first place? In any event, it's possible that that's what's blocking your imports this time around, and if that's the issue, if you just fix on those issues, then the ASpace importer won't tell you about the next issue until it runs again. In any event, I'd also suggest making the following changes to your XML file before re-importing, so perhaps the snag is a good thing for now : • ASpace adds "aspace_" to all of the @id values in the EAD file upon export. If you don't remove those before reimporting, then on the next export you'll get "aspace_aspace_". Removing them will invalidate the EAD file, but ASpace doesn't care whether the file is valid or not upon re-importing it. • ASpace expects to have the database values for the controlled value terms in the exports, not the translation values. So, if you don't change things like "Linear Feet" to linear_feet and "Mixed Materials" to mixed_materials, then you'll wind up with new database values in ASpace after the import. Those of course can be merged after the fact, but that's another step, and it would be unfortunate to have to do that on every occasion. There's a lot more to say on the practice of re-importing EAD into ASpace, but I haven't still come up with a great strategy for that, so basically we try to avoid it Mark -Original Message- From: archivesspace_users_group-boun...@lyralists.lyrasis.org [mailto:archivesspace_users_group-boun...@lyralists.lyrasis.org] On Behalf Of Tang, Lydia Sent: Friday, 10 November, 2017 10:47 AM To: Archivesspace Users Group <archivesspace_users_group@lyralists.lyrasis.org> Subject: Re: [Archivesspace_Users_Group] batch delete dates saga continues Wizardry! Thank you, Olivia! I would have NEVER figured that out. It seemed to take out the tags perfectly but I ended up getting snagged on importing because of, um, a seemingly benign line which was marked as not well-formed. Oof, so close, but snagged up! Lydia From: <archivesspace_users_group-boun...@lyralists.lyrasis.org> on behalf of Olivia S Solis <livso...@utexas.edu> Reply-To: Archivesspace Users Group <archivesspace_users_group@lyralists.lyrasis.org> Date: Friday, November 10, 2017 at 10:36 AM To: Archivesspace Users Group <archivesspace_users_group@lyralists.lyrasis.org> Subject: Re: [Archivesspace_Users_Group] batch delete dates saga continues <unitda[^>]+>[^<]+<\/unitdate> ___ Archivesspace_Users_Group mailing list Archivesspace_Users_Group@lyralists.lyrasis.org http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
Re: [Archivesspace_Users_Group] batch delete dates saga continues
You're welcome! I love me my regular expressions. Glad it worked out! -Olivia On Fri, Nov 10, 2017 at 9:47 AM, Tang, Lydia <lta...@lib.msu.edu> wrote: > Wizardry! Thank you, Olivia! I would have NEVER figured that out. It > seemed to take out the tags perfectly but I ended up getting > snagged on importing because of, um, a seemingly benign line which was > marked as not well-formed. Oof, so close, but snagged up! > Lydia > > From: <archivesspace_users_group-boun...@lyralists.lyrasis.org> on behalf > of Olivia S Solis <livso...@utexas.edu> > Reply-To: Archivesspace Users Group <archivesspace_users_group@ > lyralists.lyrasis.org> > Date: Friday, November 10, 2017 at 10:36 AM > To: Archivesspace Users Group <archivesspace_users_group@ > lyralists.lyrasis.org> > Subject: Re: [Archivesspace_Users_Group] batch delete dates saga continues > > <unitda[^>]+>[^<]+<\/unitdate> > > ___ > Archivesspace_Users_Group mailing list > Archivesspace_Users_Group@lyralists.lyrasis.org > http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group > > -- Olivia Solis, MSIS Metadata Coordinator Dolph Briscoe Center for American History The University of Texas at Austin 2300 Red River St. Stop D1100 Austin TX, 78712-1426 (512) 232-8013 ___ Archivesspace_Users_Group mailing list Archivesspace_Users_Group@lyralists.lyrasis.org http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
Re: [Archivesspace_Users_Group] batch delete dates saga continues
While I’m at it, I want to draw attention to ideas on how to conceptualize a bulk edit/delete functionality in ArchivesSpace with SIEWG. We would definitely appreciate your feedback and additional ideas! Data entry/edit doc (comments enabled) https://docs.google.com/document/d/1poanmnYaVfU4kUoxyMa6r0GQwA74byvZdduc7smQuYQ/edit?usp=sharing Lydia ___ Archivesspace_Users_Group mailing list Archivesspace_Users_Group@lyralists.lyrasis.org http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
Re: [Archivesspace_Users_Group] batch delete dates saga continues
Hi Lydia, Maybe try:]+>.+<\/unitda[^>]+> I think it might be finding the first unitdate begin tag and identifying all text through the last unitdate end tag. The [^>] indicates any character that is not end end angle bracket. Thanks, Olivia On Fri, Nov 10, 2017 at 9:17 AM, Tang, Lydia wrote: > Hi all, > I just wanted to toss out my latest saga of wrangling the dates field for > this particular collection. Instead of opening and editing each record, I > exported it to EAD and have it in an XML editor. All I would like to do is > batch delete everything like this: .* but so far it > just wants to highlight and delete the whole range (including everything > else) of where these tags exist in the finding aid. I just wanted to toss > it out there, in case any smart people know the magic command. In > searching around, all I get are doing command-line options and it seems > like it should be simpler than this! > > Lydia > > ___ > Archivesspace_Users_Group mailing list > Archivesspace_Users_Group@lyralists.lyrasis.org > http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group > > -- Olivia Solis, MSIS Metadata Coordinator Dolph Briscoe Center for American History The University of Texas at Austin 2300 Red River St. Stop D1100 Austin TX, 78712-1426 (512) 232-8013 ___ Archivesspace_Users_Group mailing list Archivesspace_Users_Group@lyralists.lyrasis.org http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group