Re: [Archivesspace_Users_Group] Maximum finding aid size

[Custer, Mark]

Kevin,

In addition to what Nancy and Steve have already mentioned, I'll add a few 
other things to consider about the size of the records:

•   Right now, a note cannot exceed 65k characters in ArchivesSpace.  The 
overall file might not even be that large, but if you have any really, really, 
really long notes, you won't be able to import the file without splitting up 
the note.
•   If you have a single archival object that's linked to, say, 1,000 or 
more digital objects, top containers, subjects, agents, or the like, then 
things might not behave so well in the staff interface, etc.  I've found the 
number of linked records clustered on a single record to be more problematic 
than a deeply hierarchical finding aid.  As long as you don't have many (or 
any) linked to over 500 things, I'd say that's good for ASpace.  We have a few 
above that, but not many.
•   The tree view for finding aids has also recently been optimized to work 
better for really flat listing, as well.  See these good-natured notes for a 
great explanation of the work done behind the scenes to accommodate all sorts 
of different finding aids:  
https://github.com/archivesspace/archivesspace/blob/615fe43e946cbf672e2c35a6c9b79ac6b11a136a/backend/app/model/large_tree.rb#L1-L49

I can also say that we don't have any finding aids as large as 100MB 😊  We have 
a few that run the gamut from 10 - 40 MB, though, and those are working well.

Mark



-Original Message-
From: archivesspace_users_group-boun...@lyralists.lyrasis.org 
[mailto:archivesspace_users_group-boun...@lyralists.lyrasis.org] On Behalf Of 
Kennedy, Nancy
Sent: Friday, 05 October, 2018 4:08 PM
To: Archivesspace Users Group 
Subject: Re: [Archivesspace_Users_Group] Maximum finding aid size

We also have finding aids that large (about 10 or so that are larger than 
10MB).  I'd definitely recommend 2.5 if you will need to work with large 
records.  Prior to updating to 2.5, the load times could mean long wait times 
to view or edit.  In the last few weeks, we've added a record that is now over 
100MB, and takes about 15 minutes to export to EAD.

In the past, in previous systems, we were unable to keep records of this size 
united and had split a few very large records into separate resources due to 
size. It's always been hard to re-unite / present that scenario in discovery 
systems.  So far, archivesspace is doing a better job of scaling to handle our 
size extremes.

Nancy






-Original Message-
From: 
archivesspace_users_group-boun...@lyralists.lyrasis.org
 
mailto:archivesspace_users_group-boun...@lyralists.lyrasis.org>>
 On Behalf Of Majewski, Steven Dennis (sdm7g)
Sent: Friday, October 05, 2018 3:28 PM
To: Archivesspace Users Group 
mailto:archivesspace_users_group@lyralists.lyrasis.org>>
Subject: Re: [Archivesspace_Users_Group] Maximum finding aid size


I’ve recently imported a 25MB EAD file as well as some around 10-15MB.
It took a significant amount of time to ingest — about an hour.
This was on a test server with minimal load other than that job, so you may 
want to schedule your ingest at off hours.
Definitely do it on a test server first, because it’s difficult to back out of 
if you need to fix something and try again.

( This was using the current version (v2.5.0). Some very early versions were 
unacceptably slow. )


I think the primary overhead is in creating objects, more than the EAD parsing, 
so it probably scales by some other order of complexity rather than just text 
size.
i.e. creating a very large bioghist or other note is still only creating a 
single object and is probably a single MySQL operation, while turning a long 
list of agent names into agent records and linking them to the resource is many 
operations.


— Steve Majewski



> On Oct 5, 2018, at 3:00 PM, Kevin W. Schlottmann 
> mailto:kws2...@columbia.edu>> wrote:
>
> Hi all,
>
> Does anyone have a sense of where the size limitations are in
> ArchivesSpace?  We have some very large finding aids (10+ MB EAD) that
> we hope to import, manage, and publish using AS. I would image that
> other institutions have large finding aids, and I'm curious to know
> what issues we might run into.
>
> Best,
>
> Kevin
> --
> Kevin Schlottmann
> Head of Archives Processing
> Rare Book & Manuscript Library
> Butler Library, Room 801
> Columbia University
> 535 W. 114th St., New York, NY  10027
> (212) 854-8483
>
> ___
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group@lyralists.lyrasis.org
> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flyrali
> sts.lyrasis.org%2Fmailman%2Flistinfo%2Farchivesspace_users_grou&da
> ta=02%7C01%7Cmark.custer%40yale.edu%7C4ceb88a237be4d5f3f5e08d62afe373d
> %7Cdd8cbebb21394df8b4114e3e87abeb5c%7C0%7C0%7C636743668695267673&s
> data=mbglfUrCPnQFTF1VkDRbbm1hd7nJn

Re: [Archivesspace_Users_Group] Maximum finding aid size

[Kennedy, Nancy]

We also have finding aids that large (about 10 or so that are larger than 
10MB).  I'd definitely recommend 2.5 if you will need to work with large 
records.  Prior to updating to 2.5, the load times could mean long wait times 
to view or edit.  In the last few weeks, we've added a record that is now over 
100MB, and takes about 15 minutes to export to EAD.

In the past, in previous systems, we were unable to keep records of this size 
united and had split a few very large records into separate resources due to 
size. It's always been hard to re-unite / present that scenario in discovery 
systems.  So far, archivesspace is doing a better job of scaling to handle our 
size extremes.

Nancy






-Original Message-
From: archivesspace_users_group-boun...@lyralists.lyrasis.org 
 On Behalf Of 
Majewski, Steven Dennis (sdm7g)
Sent: Friday, October 05, 2018 3:28 PM
To: Archivesspace Users Group 
Subject: Re: [Archivesspace_Users_Group] Maximum finding aid size


I’ve recently imported a 25MB EAD file as well as some around 10-15MB. 
It took a significant amount of time to ingest — about an hour. 
This was on a test server with minimal load other than that job, so you may 
want to schedule your ingest at off hours. 
Definitely do it on a test server first, because it’s difficult to back out of 
if you need to fix something and try again. 

( This was using the current version (v2.5.0). Some very early versions were 
unacceptably slow. )


I think the primary overhead is in creating objects, more than the EAD parsing, 
so it probably scales by some other order of complexity rather than just text 
size. 
i.e. creating a very large bioghist or other note is still only creating a 
single object and is probably a single MySQL operation, while turning a long 
list of agent names into agent records and linking them to the resource is many 
operations. 


— Steve Majewski



> On Oct 5, 2018, at 3:00 PM, Kevin W. Schlottmann  wrote:
> 
> Hi all,
> 
> Does anyone have a sense of where the size limitations are in 
> ArchivesSpace?  We have some very large finding aids (10+ MB EAD) that 
> we hope to import, manage, and publish using AS. I would image that 
> other institutions have large finding aids, and I'm curious to know 
> what issues we might run into.
> 
> Best,
> 
> Kevin
> --
> Kevin Schlottmann
> Head of Archives Processing
> Rare Book & Manuscript Library
> Butler Library, Room 801
> Columbia University
> 535 W. 114th St., New York, NY  10027
> (212) 854-8483
> 
> ___
> Archivesspace_Users_Group mailing list 
> Archivesspace_Users_Group@lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_grou
> p

___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

Re: [Archivesspace_Users_Group] Maximum finding aid size

[Kevin W. Schlottmann]

Thanks Steve.  That's very helpful; we'll definitely run these in a
test environment first.
On Fri, Oct 5, 2018 at 3:27 PM Majewski, Steven Dennis (sdm7g)
 wrote:
>
>
> I’ve recently imported a 25MB EAD file as well as some around 10-15MB.
> It took a significant amount of time to ingest — about an hour.
> This was on a test server with minimal load other than that job,
> so you may want to schedule your ingest at off hours.
> Definitely do it on a test server first, because it’s difficult to back out of
> if you need to fix something and try again.
>
> ( This was using the current version (v2.5.0). Some very early versions were 
> unacceptably slow. )
>
>
> I think the primary overhead is in creating objects, more than the EAD 
> parsing,
> so it probably scales by some other order of complexity rather than just text 
> size.
> i.e. creating a very large bioghist or other note is still only creating a 
> single object
> and is probably a single MySQL operation, while turning a long list of agent 
> names into
> agent records and linking them to the resource is many operations.
>
>
> — Steve Majewski
>
>
>
> > On Oct 5, 2018, at 3:00 PM, Kevin W. Schlottmann  
> > wrote:
> >
> > Hi all,
> >
> > Does anyone have a sense of where the size limitations are in
> > ArchivesSpace?  We have some very large finding aids (10+ MB EAD) that
> > we hope to import, manage, and publish using AS. I would image that
> > other institutions have large finding aids, and I'm curious to know
> > what issues we might run into.
> >
> > Best,
> >
> > Kevin
> > --
> > Kevin Schlottmann
> > Head of Archives Processing
> > Rare Book & Manuscript Library
> > Butler Library, Room 801
> > Columbia University
> > 535 W. 114th St., New York, NY  10027
> > (212) 854-8483
> >
> > ___
> > Archivesspace_Users_Group mailing list
> > Archivesspace_Users_Group@lyralists.lyrasis.org
> > http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>
> ___
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group@lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group



-- 
Kevin Schlottmann
Head of Archives Processing
Rare Book & Manuscript Library
Butler Library, Room 801
Columbia University
535 W. 114th St., New York, NY  10027
(212) 854-8483

___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

Re: [Archivesspace_Users_Group] Maximum finding aid size

[Majewski, Steven Dennis (sdm7g)]

I’ve recently imported a 25MB EAD file as well as some around 10-15MB. 
It took a significant amount of time to ingest — about an hour. 
This was on a test server with minimal load other than that job, 
so you may want to schedule your ingest at off hours. 
Definitely do it on a test server first, because it’s difficult to back out of
if you need to fix something and try again. 

( This was using the current version (v2.5.0). Some very early versions were 
unacceptably slow. )


I think the primary overhead is in creating objects, more than the EAD parsing, 
so it probably scales by some other order of complexity rather than just text 
size. 
i.e. creating a very large bioghist or other note is still only creating a 
single object
and is probably a single MySQL operation, while turning a long list of agent 
names into 
agent records and linking them to the resource is many operations. 


— Steve Majewski



> On Oct 5, 2018, at 3:00 PM, Kevin W. Schlottmann  wrote:
> 
> Hi all,
> 
> Does anyone have a sense of where the size limitations are in
> ArchivesSpace?  We have some very large finding aids (10+ MB EAD) that
> we hope to import, manage, and publish using AS. I would image that
> other institutions have large finding aids, and I'm curious to know
> what issues we might run into.
> 
> Best,
> 
> Kevin
> -- 
> Kevin Schlottmann
> Head of Archives Processing
> Rare Book & Manuscript Library
> Butler Library, Room 801
> Columbia University
> 535 W. 114th St., New York, NY  10027
> (212) 854-8483
> 
> ___
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group@lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group



smime.p7s
Description: S/MIME cryptographic signature
___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

[Archivesspace_Users_Group] Maximum finding aid size

[Kevin W. Schlottmann]

Hi all,

Does anyone have a sense of where the size limitations are in
ArchivesSpace?  We have some very large finding aids (10+ MB EAD) that
we hope to import, manage, and publish using AS. I would image that
other institutions have large finding aids, and I'm curious to know
what issues we might run into.

Best,

Kevin
-- 
Kevin Schlottmann
Head of Archives Processing
Rare Book & Manuscript Library
Butler Library, Room 801
Columbia University
535 W. 114th St., New York, NY  10027
(212) 854-8483

___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

[Archivesspace_Users_Group] fund codes not appearing after installing payments_module plugin

[Eric Loehr]

Hi all,

I've installed the payments_module plugin on our local test version of
ArchivesSpace; we've created the .tsv file with the fund codes, but they
aren't appearing as options.  I initially installed this in v.2.4.1 and
didn't specify the location for the fund codes file on installation,
thinking that I could do it later - -but then wasn't sure how to do that --
after looking through the conifg files I ended up adding it to /plugins/
payments_module/migrations/*002_load_fund_codes.rb.*

I've since upgraded to 2.5.0, and the correct path appeared on installation
(since it was already in the 002_load_fund_codes.rb file copied over during
the upgrade), but the fund codes still aren't appearing.  Does anyone have
an idea where we might be going wrong?

Thanks --

Eric


-- 
Eric Loehr
Head of Library Technology Services
Smith College Libraries
elo...@smith.edu
(413) 585-2969

*Young Library is now our central campus hub. The latest information is on
the Smith College Libraries web site . *

*Please don't hesitate to be in touch with questions, comments or feedback
-- Ask Us! *
___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

[Archivesspace_Users_Group] FW: ArchivesSpace + LYRASIS = Innovation at Work

[Laurie Arp]

Greetings,

We want invite you to the LYRASIS Member Summit.

Invitation and details are below.

Best wishes,

Laurie Gemmill Arp
Director, Collections Services & Community Supported Software
laurie@lyrasis.org
800.999.8558 x 2908


[https://mlsvc01-prod.s3.amazonaws.com/df76fc97001/37b93280-9a94-4031-8b56-388ae8119514.jpg]





[https://mlsvc01-prod.s3.amazonaws.com/df76fc97001/813de38a-d661-4848-814f-e9c370545e4b.jpg]







Collaborate, Learn and Contribute at the LYRASIS Member Summit

As a member of ArchivesSpace and the LYRASIS community, we would like to extend 
a special invitation to attend the LYRASIS Annual Member Summit, in Nashville, 
TN on October 24-25. This two-day meeting is a chance to meet and work directly 
with fellow leaders, and to directly impact the wider field of Galleries, 
Libraries, Archives and Museums (GLAMs) by participating in our breakout 
sessions, contributing to LYRASIS strategic planning and helping to develop and 
foster new and innovative projects through the LYRASIS Catalyst Fund.

Join us at the LYRASIS Member Summit to:

  *   Learn more about fostering innovation and creating an innovative 
organization;
  *   Directly engage with your fellow leaders on the issues of the day;
  *   Share your experience and expertise with the wider community;
  *   Hear from experts about their successes, challenges and next steps; and
  *   Help set the LYRASIS strategic agenda and turn your ideas into real world 
results!
The Member Summit kicks off with a workshop day, where we will be working 
together to discuss how to create and foster innovative organizations, and we 
will shine a light on our Catalyst Fund projects from last year and this year, 
where LYRASIS invested more than $200,000 directly into member projects.

On our second day of the Summit, we have designed breakout sessions focused on 
the uniting thread in the GLAMs community: collections. From creating standards 
for program sustainability to making sure your collections are accessible and 
manageable. Our breakouts give you a chance to learn, contribute and think 
about how to be even more innovative in your own institution. Click 
here
 to see the full agenda.


[https://mlsvc01-prod.s3.amazonaws.com/df76fc97001/c008a57f-9a01-410c-a256-1c6542959e44.jpg]

We would love to have you join us for the Member Summit, at no cost to you. 
Register here or email 
jennifer.bielew...@lyrasis.org for more 
information or help registering.




Also, if you can't make the Summit, we would love to have your feedback on how 
you are fostering innovation in your own institution. Please consider filling 
out this short 5-minute survey so we can understand more about the hurdles and 
benefits of innovation in the community. Take the short 
survey.











STAY CONNECTED

[Like us on 
Facebook]
  [Follow us on Twitter] 






[http://img.constantcontact.com/letters/images/1101116784221/PM_B2BC_BottomShadow.png]




___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

[Archivesspace_Users_Group] Linked agent relator table broken in MARC export?

[Kevin W. Schlottmann]

Hi all,

We've noticed recently (post-version 2.5 upgrade to our hosted
instance) that our linked agent relators are not exporting into MARC
correctly. Instead of a subfield e "collector" we are getting a
subfield 4 "translation missing:
en.enumerations.linked_agent_archival_record_relators.collector".  The
staff interface displays correctly, and the equivalent EAD looks like
it is working, so I suspect it's a MARC export error. Code below. I
looked for a ticket, but all I found was this completed request:
https://archivesspace.atlassian.net/browse/ANW-726

Christine DiBella's 9/14 email mentioned the inclusion of the relators
for agents as well: "One of the additions was including relators for
agents when there is a value in the relator field."

Before I file a bug ticket, had anyone else experienced and/or
reported this issue yet?

MARC:

  Ryan, Dennis B.
  translation missing:
en.enumerations.linked_agent_archival_record_relators.collector


EAD:

Ryan, Dennis B.


Best,

Kevin


-- 
Kevin Schlottmann
Head of Archives Processing
Rare Book & Manuscript Library
Butler Library, Room 801
Columbia University
535 W. 114th St., New York, NY  10027
(212) 854-8483

___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group