[Archivesspace_Users_Group] Yet more tools: Ingest and analysis (of ingest) scripts now on Github

2016-08-19 Thread Mayo, Dave
Hello all, I just wanted to make known that, in addition to the EADChecker and our preprocessor, we have produced a set of ruby scripts for doing bulk ingest that might be of general interest, made them available via Github. https://github.com/harvard-library/aspace-utils There are basically

Re: [Archivesspace_Users_Group] Need help Interpreting an EAD ingest error message

2016-11-23 Thread Mayo, Dave
ate, Dave: Can you attach the EAD file? I wonder if this is being caused by an empty container element in the source file, or something else related to a container attribute? Mark From:archivesspace_users_group-boun...@lyralists.lyrasis.org<mailto:archivesspace_users_group-boun...@l

Re: [Archivesspace_Users_Group] Need help Interpreting an EAD ingest error message

2016-11-22 Thread Mayo, Dave
Hi all, I have a partial answer to the original question (what is this meant to tell me). Short version: It's not meant to tell you anything, because it's an uncaught Ruby error in the ead_converter, so, a bug in ASpace, not an error message produced by ASpace intentionally. Longer version:

Re: [Archivesspace_Users_Group] Python ArchivesSpace scripting library

2017-08-03 Thread Mayo, Dave
One thing I think might be useful, especially in planning stages, is collecting the “prior art” in one place to look at – because there are a lot of scripts people have built around the API, and they probably capture at least a fair bit of “what people want to do most with the API.” So,

Re: [Archivesspace_Users_Group] Python ArchivesSpace scripting library

2017-08-03 Thread Mayo, Dave
Hello, I’m definitely interested in contributing to this, and have some experience with writing client-ish things for ASpace in Ruby that might be relevant. +1 for Github - Dave Mayo Senior Digital Library Software Engineer Harvard University > HUIT > LTS > OSC Dev Team From:

Re: [Archivesspace_Users_Group] RFC: EAD import/export: mapping multiple unitid's and external-id's

2017-04-26 Thread Mayo, Dave
uld it make more sense to concatenate multiple unittitles ? I don’t think I’ve actually run into one in the wild. — Steve M. On Apr 24, 2017, at 3:55 PM, Mayo, Dave <dave_m...@harvard.edu<mailto:dave_m...@harvard.edu>> wrote: +1, and also this is true of unittitles as well; unfortunately,

Re: [Archivesspace_Users_Group] ampersand issue with PDF button in 2.1.2 public interface

2017-09-22 Thread Mayo, Dave
Hi Benn, This is a recurring issue I hit over both Harvard and Smith’s collections – it’s a consequence of ASpace not really having a distinction between mixed content and plaintext content. Unfortunately, there isn’t really a good solution. The best solution as far as I’ve been able to

Re: [Archivesspace_Users_Group] EAD Importer and DAOs

2018-05-25 Thread Mayo, Dave
IIRC, also, a lot of the smaller DB-level defaults (and some of the larger ones) are the result of various defaults or arbitrary guesses - before just saying that it's part of the data model and we should just live with it, one should always positively verify that it's legitimately part of the

Re: [Archivesspace_Users_Group] ArchivesSpace spreadsheet import plugin

2018-01-19 Thread Mayo, Dave
Hi Jessika, The file you’d want to edit is the Gemfile file in the top directory of the plugin – contents look like this: https://github.com/harvard-library/aspace-import-excel/blob/master/Gemfile On line 6, you’d remove the comma and everything after. - Dave From:

Re: [Archivesspace_Users_Group] Questions about MariaDB

2018-02-11 Thread Mayo, Dave
Full disclosure, I work as a developer rather than on the sysadmin/devops side. I've been using MariaDB as a drop in replacement for MySQL server for local development instances for years, and it's been completely fine - I have had zero issues with it, and specifically zero issues resulting

Re: [Archivesspace_Users_Group] Duplicate Commas Following Description

2018-08-10 Thread Mayo, Dave
Hi, That’s definitely possible. There are definitely multiple ways to do that, with different risks and benefits. The ones I can think of offhand are: 1. Do that in the MySQL database, then clear out and rebuild ArchivesSpace’s Solr indexes. This would probably be overall fastest, but would

Re: [Archivesspace_Users_Group] FW: [Archivesspace_member_reps] Migrating Data from Eloquent Systems to ArchivesSpace

2018-03-15 Thread Mayo, Dave
Hi Cheryl, Oof, that’s rough. I have a few questions, if you don’t mind. 1. Can you provide more detail on why EAD export is impossible? I only ask because they claim EAD export on their website pretty prominently, and it would be, I think, much easier than doing a migration from proprietary

Re: [Archivesspace_Users_Group] Errors editing older resource records v.2.3.1

2018-04-18 Thread Mayo, Dave
It looks like xmx gets set to 24g by default if no env var exists when it starts up… From: on behalf of "Holland, Andrew S" Reply-To: Archivesspace Users Group

Re: [Archivesspace_Users_Group] Import Data Errors

2019-01-29 Thread Mayo, Dave
What Kate said, and also: for anything that we figured out during our migration, you can check individual EAD files with somewhat better error reporting by putting them through https://eadchecker.lib.harvard.edu/ -- Dave Mayo (he/him) Senior Digital Library Software Engineer Harvard University

Re: [Archivesspace_Users_Group] Mariadb errors from reports: "portion"

2019-07-29 Thread Mayo, Dave
Weird – I just verified that this happens this on my local instance, but haven’t been able to find the thing causing this issue. Looking in the code, it looks like there might be an in-process or forthcoming syntax for doing date ranges with this as a keyword?

Re: [Archivesspace_Users_Group] importing EAD files

2019-11-22 Thread Mayo, Dave
Also, the EADChecker web service mentioned in the same article is still around – it’s not exhaustive, but it should find _most_ potential errors that will cause ingest to fail, and output them in a slightly easier to manage format than the ASpace job reports.

Re: [Archivesspace_Users_Group] Database query to recreate inventory hierarchy?

2019-09-20 Thread Mayo, Dave
It’s possible, but I think to do it in one query requires recursive common table expressions – what version of MySQL/MariaDB are you on? I wrote a thing that’s almost this except I was getting container info attached to the AOs rather than info from the AOs themselves, I can probably find and

Re: [Archivesspace_Users_Group] Database query to recreate inventory hierarchy?

2019-09-20 Thread Mayo, Dave
Engineer Harvard University > HUIT > LTS From: on behalf of "Mayo, Dave" Reply-To: Archivesspace Users Group Date: Friday, September 20, 2019 at 10:13 AM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] Database query to recreate inventory hierarchy?

Re: [Archivesspace_Users_Group] Database query to recreate inventory hierarchy?

2019-09-20 Thread Mayo, Dave
No joy then, I’m sorry to say. Honestly, since the API is going to be in the mix to use eventually, I’d suggest looking at this endpoint:

Re: [Archivesspace_Users_Group] Oddities when updating Agents via the API

2019-10-13 Thread Mayo, Dave
There are a lot of sub-records in ASpace that are essentially treated as ephemeral but which have full concrete tables backing them; they’re not addressable in the system, their representations in the JSONModel are as primitive values (strings, array of strings, etc) rather than objects, etc.

[Archivesspace_Users_Group] Release 3.0.3 and 3.0.4 of aspace-import-excel plugin

2020-01-31 Thread Mayo, Dave
This message may be of interest to those who use, or are considering, the aspace-import-excel plugin (https://github.com/harvard-library/aspace-import-excel [github.com]), or the Tufts Extended Version plugin (https://github.com/tufts-digital-collections-archives/aspace-import-excel . Harvard

Re: [Archivesspace_Users_Group] question about customizing search features in the ArchivesSpace PUI

2020-01-09 Thread Mayo, Dave
types last, even if their title is a precise match, then boost factors like 50 and 100 are appropriate. For a more subtle bias towards certain record types, without them totally dominating, start low and work upwards (the smallest boost you can apply is 1.1). Andrew. On Wed, 2020-01-08 at 15:39 +

Re: [Archivesspace_Users_Group] question about customizing search features in the ArchivesSpace PUI

2020-01-08 Thread Mayo, Dave
Hi Henry! So, good news! As long as you’re on a reasonably current version of ASpace, you can change order by primary type purely via configuration. The way you’d do it is by setting the bq param in AppConfig[:solr_params] – here’s what we’ve got in production right now (we’re setting q.op

Re: [Archivesspace_Users_Group] Search Across Repositories and Get EAD XML - ArchivesSpace API

2020-03-10 Thread Mayo, Dave
exports, the barcodes to containers were being attached to the "label" attribute for the container tag (label="Mixed Materials [5168852364844]"). Yet, these exports are generating the "physloc" element with the barcode as the value. This works for us, I just thought

Re: [Archivesspace_Users_Group] Search Across Repositories and Get EAD XML - ArchivesSpace API

2020-03-09 Thread Mayo, Dave
Hi Corey, Splitting this into two sections – the first is a description of what’s happening and should immediately fix your current issue, the second is some advice going forward re: using ASnake. You appear to have found a bug in the ASpace API! From what I can tell, all_ids is not actually

Re: [Archivesspace_Users_Group] Search Across Repositories and Get EAD XML - ArchivesSpace API

2020-03-09 Thread Mayo, Dave
Oh, I missed the second part: So, that endpoint returns the XML directly as the body of the response in memory. The JSONDecodeError you’re getting is due to trying to decode this body as JSON, when what you actually have is XML! If you want to save it to the filesystem, you just need to write

Re: [Archivesspace_Users_Group] Error updating resource records through API

2020-04-17 Thread Mayo, Dave
So, ArchivesSpace keeps track of whether it’s safe to modify an object via a field called “lock_version”. Any change to an object will increment this value, and this is probably what’s catching you up – something is altering the value between when you download the resource and when you update

Re: [Archivesspace_Users_Group] EAD Import Error

2020-03-16 Thread Mayo, Dave
So, this isn’t actually an error about a character entity – those character entities are the result of a Ruby error being double-escaped in HTML. The error actually being emitted is: "#:0x2346a459>\nDid you mean? instance_of?>" Where that’s coming from, I’m not sure, unfortunately – is that

Re: [Archivesspace_Users_Group] EAD Import Error

2020-03-16 Thread Mayo, Dave
Error Thanks for responding! Yes that’s the complete error message. From: archivesspace_users_group-boun...@lyralists.lyrasis.org On Behalf Of Mayo, Dave Sent: Monday, March 16, 2020 2:18 PM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] EAD Import Error So, this isn’t ac

Re: [Archivesspace_Users_Group] EAD Import Error

2020-03-16 Thread Mayo, Dave
:36 PM To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] EAD Import Error That solved it thank you so much! From: archivesspace_users_group-boun...@lyralists.lyrasis.org On Behalf Of Mayo, Dave Sent: Monday, March 16, 2020 2:31 PM To: Archivesspace Users Group Subje

Re: [Archivesspace_Users_Group] EAD via API - character problem

2020-03-27 Thread Mayo, Dave
Can you be more precise about what “directly from the archivesspace” means? What page (URL) and control on the page you’re using to generate this? I want to make sure it’s the same code path in both cases. Also, when you say “locally,” do you just mean the script is running locally against

Re: [Archivesspace_Users_Group] Question Regarding the REST API batch_imports Operation

2020-05-07 Thread Mayo, Dave
I did a lot of this back in the day, but it’s been a while – IIRC You post an array of objects matching the JSONModel schema in ArchivesSpace. If you’re referencing existing objects in the system, you use the existing URI in refs, if you’re creating an object, you give it an arbitrary URI

Re: [Archivesspace_Users_Group] Custom reports with string params?

2020-10-02 Thread Mayo, Dave
Hi, We have a plugin we wrote with some custom SQL reports – this one uses a string parameter for an EADID or list of EADIDs - https://github.com/harvard-library/harvard_aspace_reports/blob/master/backend/model/resource_top_containers.rb This is currently installed in production and working;

Re: [Archivesspace_Users_Group] Custom reports with string params?

2020-10-05 Thread Mayo, Dave
__ From: archivesspace_users_group-boun...@lyralists.lyrasis.org on behalf of Mayo, Dave Sent: 02 October 2020 17:07 To: Archivesspace Users Group Subject: Re: [Archivesspace_Users_Group] Custom reports with string params? Hi, We have a plugin we wrote with some custom SQL reports

Re: [Archivesspace_Users_Group] API

2020-08-06 Thread Mayo, Dave
It is not – the API is fundamentally required for the application to function at all, so it’s more or less always going to be running if any component is running. On a separate note, many installations don’t necessarily _expose_ it; the API by default is run so that it’s only accessible to

Re: [Archivesspace_Users_Group] Top container ranges

2020-06-18 Thread Mayo, Dave
The two options I see here are essentially: 1. Change the EAD 2. Change the containers after they’re ingested. Of the two, changing the EAD seems _easier_ to me; if you wouldn’t mind going more into why that’s not a viable solution for you, it might help us provide better advice? Either way,

Re: [Archivesspace_Users_Group] Top container ranges

2020-06-18 Thread Mayo, Dave
en any drawbacks? Anything we should be looking to avoid? Thanks again, Dawne From: Mayo, Dave<mailto:dave_m...@harvard.edu> Sent: Thursday, June 18, 2020 9:04 AM To: Archivesspace Users Group<mailto:archivesspace_users_group@lyralists.lyrasis.org> Subject: Re: [Archivesspace_Users

Re: [Archivesspace_Users_Group] Top container ranges

2020-06-18 Thread Mayo, Dave
this specific issue, but it does catch a bunch of issues, some of which cause corrupted data rather than failure to import. -- Dave Mayo (he/him) Senior Digital Library Software Engineer Harvard University > HUIT > LTS From: on behalf of "Mayo, Dave" Reply-To: Archivesspace Us

Re: [Archivesspace_Users_Group] Separating Solr index to separate server?

2021-02-04 Thread Mayo, Dave
We definitely do, and it's definitely been useful performance-wise. We also are able to use a newer Solr revision this way. Additionally, Solr needs a lot of empty disk to be performant, and it's easier to manage that when it's running on its own server. Get Outlook for

Re: [Archivesspace_Users_Group] PUI indexing issues

2021-03-18 Thread Mayo, Dave
This is a little bit of a shot in the dark, but have you looked at disk space on whatever host Solr is resident on? (the ASpace server if you’re not running an external one)? A thing we’ve hit a couple times is that Solr, at least in some configurations, needs substantial headroom on disk to

Re: [Archivesspace_Users_Group] Missing Japanese charactires in a PUI generated PDF

2022-09-14 Thread Mayo, Dave
Hi! This is something we’ve recently had to deal with – I’m not 100% sure from what you’ve posted that it’s the same issue we had, but there are a few issues with the PUI’s current PDF generation support that make font handling challenging. So, first of all – if you’re setting up a fallback