Re: [Archivesspace_Users_Group] proxy rewriting gone awry

[Majewski, Steven Dennis (sdm7g)]

You might also try accessing the public web app directly at port 8081, not 
going thru the proxy. 
You may have to temporarily change your firewall settings, if that port isn’t 
served to the outside. 
If the public webapp works properly on it’s own, the the problem is with 
proxying. 

— Steve. 


> On Jul 23, 2019, at 9:29 AM, Blake Carver  wrote:
> 
> I'd start by looking in the logs (Both Apache and ArchivesSpace), any errors 
> in there?
> From: archivesspace_users_group-boun...@lyralists.lyrasis.org 
>  on behalf of 
> Kimberli Kelmor 
> Sent: Tuesday, July 23, 2019 9:18 AM
> To: archivesspace_users_group@lyralists.lyrasis.org 
> 
> Subject: [Archivesspace_Users_Group] proxy rewriting gone awry
>  
> All of a sudden today the public link to the repositories is broken. The 
> public can list the repositories, but when they click on a link it gets a 404 
> not found.
> 
>  The url for the list of repositories is website.edu/public/repositories 
>  -- this works
>   The url for one of the repositories is website.edu/repositories/2 
>  -- this does not work
> 
> If I manually make it website.edu/public/repositories/2 
>  -- this works for the actual 
> content, but the images are broken.
> 
> I'm new to all this, but is something messing up the proxy settings or is 
> that barking up the wrong tree?
> 
> Everything (config.rb and apache conf.d) is set up per the 'running 
> archivesspace under a prefix' documentation.
> 
> Running version 2.6.0 -- but that has been running for about a week with no 
> problem.
> 
> Any help greatly appreciated!
> Kim
> 
> -- 
> Kimberli M. Kelmor
> Head of Law Library Technology
> Georgetown University Law Library
> kk1...@georgetown.edu 
> 202-662-9158
> ___
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group@lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group



smime.p7s
Description: S/MIME cryptographic signature
___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

Re: [Archivesspace_Users_Group] proxy rewriting gone awry

[Blake Carver]

That doesn't sound good, maybe something changed in your Apache settings.

From: archivesspace_users_group-boun...@lyralists.lyrasis.org 
 on behalf of Kimberli 
Kelmor 
Sent: Tuesday, July 23, 2019 3:26 PM
To: archivesspace_users_group@lyralists.lyrasis.org 

Subject: Re: [Archivesspace_Users_Group] proxy rewriting gone awry

I will try the reindexing tonight.

The apache error log had a ton of proxy errors, all of which were blocking 
localhost or refusing connection. A connection will be refused, a connection 
will fail, 'worker' for local host will be disabled for 60 seconds, and finally 
local host is disabled. Then the cycle repeats. All the localhost calls are 
supposed to be re-written aren't they?

Thanks again,
Kim

I’m not very good at this, there are smarter folks than me out there,
But when I run into a funky problem like that I re-index and it fixes it.

https://archivesspace.github.io/archivesspace/user/re-creating-indexes/

From: archivesspace_users_group-bounces at 
lyralists.lyrasis.org
 http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group>>
 On Behalf Of Kimberli Kelmor
Sent: Tuesday, July 23, 2019 9:19 AM
To: archivesspace_users_group at 
lyralists.lyrasis.org
Subject: [Archivesspace_Users_Group] proxy rewriting gone awry

All of a sudden today the public link to the repositories is broken. The public 
can list the repositories, but when they click on a link it gets a 404 not 
found.

 The url for the list of repositories is 
website.edu/public/repositories
 -- this works
  The url for one of the repositories is 
website.edu/repositories/2
 -- this does not work

If I manually make it 
website.edu/public/repositories/2
 -- this works for the actual content, but the images are broken.

I'm new to all this, but is something messing up the proxy settings or is that 
barking up the wrong tree?

Everything (config.rb and apache conf.d) is set up per the 'running 
archivesspace under a prefix' documentation.

Running version 2.6.0 -- but that has been running for about a week with no 
problem.

Any help greatly appreciated!
Kim

--
Kimberli M. Kelmor
Head of Law Library Technology
Georgetown University Law Library
kk1210 at 
georgetown.edu>
202-662-9158

--
Kimberli M. Kelmor
Head of Law Library Technology
Georgetown University Law Library
kk1...@georgetown.edu
202-662-9158
___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

Re: [Archivesspace_Users_Group] proxy rewriting gone awry

[Kimberli Kelmor]

I will try the reindexing tonight.

The apache error log had a ton of proxy errors, all of which were blocking
localhost or refusing connection. A connection will be refused, a
connection will fail, 'worker' for local host will be disabled for 60
seconds, and finally local host is disabled. Then the cycle repeats. All
the localhost calls are supposed to be re-written aren't they?

Thanks again,
Kim

I’m not very good at this, there are smarter folks than me out there,
But when I run into a funky problem like that I re-index and it fixes it.
https://archivesspace.github.io/archivesspace/user/re-creating-indexes/

From: archivesspace_users_group-bounces at lyralists.lyrasis.org

http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group>>
On Behalf Of Kimberli Kelmor
Sent: Tuesday, July 23, 2019 9:19 AM
To: archivesspace_users_group at lyralists.lyrasis.org

Subject: [Archivesspace_Users_Group] proxy rewriting gone awry

All of a sudden today the public link to the repositories is broken.
The public can list the repositories, but when they click on a link it
gets a 404 not found.

 The url for the list of repositories is
website.edu/public/repositories
-- this works
  The url for one of the repositories is
website.edu/repositories/2
-- this does not work

If I manually make it
website.edu/public/repositories/2
-- this works for the actual content, but the images are broken.

I'm new to all this, but is something messing up the proxy settings or
is that barking up the wrong tree?

Everything (config.rb and apache conf.d) is set up per the 'running
archivesspace under a prefix' documentation.

Running version 2.6.0 -- but that has been running for about a week
with no problem.

Any help greatly appreciated!
Kim

--
Kimberli M. Kelmor
Head of Law Library Technology
Georgetown University Law Librarykk1210 at georgetown.edu
>
202-662-9158

-- 
Kimberli M. Kelmor
Head of Law Library Technology
Georgetown University Law Library
kk1...@georgetown.edu
202-662-9158
___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

[Archivesspace_Users_Group] Export ASpace accession records to MARC?

[Erin Louthen]

Hello ASpace Community,

I would appreciate advice regarding the following: is there a way to export
ASpace accession records into MARC records? I know you can do this with
resource records, but can't locate any documentation about exporting
accession records.

Thank you!

Erin

-- 

Erin Louthen
University Archivist
Archives & Special Collections
University Library
Santa Clara University
500 El Camino Real
Santa Clara, CA 95053-0500
elout...@scu.edu
Tel.:408-554-4117
___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

Re: [Archivesspace_Users_Group] Retrieving tree info via API (what are "waypoints"?)

[Majewski, Steven Dennis (sdm7g)]

AH! Thanks 
I hadn’t needed to drill down to deeper levels on what I was initially using 
this endpoint for, so I didn’t catch onto the use of /tree/node?node_uri= 
Sounds right! 

— Steve. 


> On Jul 23, 2019, at 2:22 PM, Trevor Thornton  wrote:
> 
> First of all, great documentation (in the code, API documentation less so but 
> we're working on that) 
> 
> To close the loop on this thread (for anyone still interested):
> For what I'm doing I need the container info, which is included in the 
> .../tree/... responses. Basically I'm re-creating a version of the AS 
> resource tree to provide a browsable view of a resource hierarchy in another 
> application. So the process will be something like this (for a resource with 
> URI  /repositories/1/resources/123):
> Call  /repositories/1/resources/123/tree/root to get the resource-level data 
> and its children (up to 200)
> If the value for "waypoints" in the response is greater than 1, call 
> /repositories/1/resources/123/tree/waypoints=n for each additional 
> waypoint (n = 1 through # of waypoints - 1) to get the rest of the children
> Then for each child record with other children, I'll provide a link to see 
> the next level, which will call 
> /repositories/1/resources/123/tree/node_uri=[URI OF THE RECORD THAT WAS 
> CLICKED]
> (NOTE: node_uri is a required parameter for this endpoint but that's not 
> mentioned in API the documentation)
> This provides a response similar to the .../tree/root endpoint but with data 
> for the archival object record instead of the resource
> Repeat step 2 if there is more than one waypoint at this level, including the 
> current node URI as parent_id in the GET params
> Repeat steps 3 & 4 until you get to the end
> I *think* this is close to right.
> 
> Thanks again for your help!
> 
> On Tue, Jul 23, 2019 at 12:51 PM Majewski, Steven Dennis (sdm7g) 
> mailto:sd...@virginia.edu>> wrote:
> 
> I believe for the next level of archival_objects, you have to get 
> /repositories/$REPO/archival_objects/$ID/children , but check the API docs.
> 
> 
> Note that there is also a GET 
> /repositories/$REPO/resources/$ID/ordered_records method that gives you the 
> whole hierarchy, but minimal info about each resource:  { ref: 
> display_string:, depth:, level: } 
> 
> I don’t think I knew about that one the first time I was wrestling with this 
> sort of task. 
> If you’re doing backend API and not worried about real time display update, 
> it might make more sense to walk the output ordered_records 
> If you want more complete info on resource children. 
> 
> 
> — Steve. 
> 
> 
>> On Jul 23, 2019, at 12:11 PM, Trevor Thornton > > wrote:
>> 
>> Just found that file in the repo before I saw your message and I think I 
>> understand now - thanks!
>> 
>> So, if you're looking at a node below the root (an ArchivalObject) that has 
>> >200 children, you would hit the ".../tree/waypoint" endpoint however many 
>> times and include "parent_node" in the GET params with the ArchivalObject 
>> URI, right?
>> 
>> On Tue, Jul 23, 2019 at 11:57 AM Majewski, Steven Dennis (sdm7g) 
>> mailto:sd...@virginia.edu>> wrote:
>> 
>>> So the next question is how do you make the subsequent calls to retrieve 
>>> the next 200, etc.?
>> 
>> 
>> 
>> You call  /repositories/$repo/resources/$id/tree/waypoint?offset=$N  23 
>> times. 
>> ( You already got the first batch in .precomputed_waypoints in the call to 
>> /ress/root  ) 
>> 
>> 
>> I found the documentation note in the source I was looking for: 
>> https://github.com/archivesspace/archivesspace/blob/master/backend/app/model/large_tree.rb
>>  
>> 
>> 
>> 
>> # What's the big idea?
>> #
>> # ArchivesSpace has some big trees in it, and sometimes they look a lot like 
>> big
>> # sticks.  Back in the dark ages, we used JSTree for our trees, which in 
>> general
>> # is perfectly cromulent.  We recognized the risk of having some very large
>> # collections, so dutifully configured JSTree to lazily load subtrees as the
>> # user expanded them (avoiding having to load the full tree into memory right
>> # away).
>> #
>> # However, time makes fools of us all.  The JSTree approach works fine if 
>> your
>> # tree is fairly well balanced, but that's not what things look like in the 
>> real
>> # world.  Some trees have a single root node and tens of thousands of records
>> # directly underneath it.  Lazy loading at the subtree level doesn't save you
>> # here: as soon as you expand that (single) node, you're toast.
>> #
>> # This "large tree" business is a way around all of this.  It's effectively a
>> # hybrid of trees and pagination, except we call the pages "waypoints" for
>> # reasons known only to me.  So here's the big idea:
>> #
>> #  * You want to show a tree.  You ask the API to give you the root node.
>> #
>> #  * The root node tells you whether or not it has children, how many 
>> 

Re: [Archivesspace_Users_Group] Retrieving tree info via API (what are "waypoints"?)

[Trevor Thornton]

First of all, great documentation (in the code, API documentation less so
but we're working on that) 

To close the loop on this thread (for anyone still interested):
For what I'm doing I need the container info, which is included in the
.../tree/... responses. Basically I'm re-creating a version of the AS
resource tree to provide a browsable view of a resource hierarchy in
another application. So the process will be something like this (for a
resource with URI  */repositories/1/resources/123):*

   1. Call * /repositories/1/resources/123/tree/root* to get the
   resource-level data and its children (up to 200)
   2. If the value for "waypoints" in the response is greater than 1, call
   */repositories/1/resources/123/tree/waypoints=n* for each
   additional waypoint (n = 1 through # of waypoints - 1) to get the rest of
   the children
   3. Then for each child record with other children, I'll provide a link
   to see the next level, which will call
* /repositories/1/resources/123/tree/node_uri=[URI OF THE RECORD THAT
   WAS CLICKED] **(NOTE: node_uri is a required parameter for this endpoint
   but that's not mentioned in API the documentation)*
   This provides a response similar to the *.../tree/root* endpoint but
   with data for the archival object record instead of the resource
   4. Repeat step 2 if there is more than one waypoint at this level,
   including the current node URI as *parent_id* in the GET params
   5. Repeat steps 3 & 4 until you get to the end

I *think* this is close to right.

Thanks again for your help!

On Tue, Jul 23, 2019 at 12:51 PM Majewski, Steven Dennis (sdm7g) <
sd...@virginia.edu> wrote:

>
> I believe for the next level of archival_objects, you have to get
> /repositories/$REPO/archival_objects/$ID/children , but check the API docs.
>
>
> Note that there is also a GET
> /repositories/$REPO/resources/$ID/ordered_records method that gives you the
> whole hierarchy, but minimal info about each resource:  { ref:
> display_string:, depth:, level: }
>
> I don’t think I knew about that one the first time I was wrestling with
> this sort of task.
> If you’re doing backend API and not worried about real time display
> update, it might make more sense to walk the output ordered_records
> If you want more complete info on resource children.
>
>
> — Steve.
>
>
> On Jul 23, 2019, at 12:11 PM, Trevor Thornton  wrote:
>
> Just found that file in the repo before I saw your message and I think I
> understand now - thanks!
>
> So, if you're looking at a node below the root (an ArchivalObject) that
> has >200 children, you would hit the ".../tree/waypoint" endpoint however
> many times and include "parent_node" in the GET params with the
> ArchivalObject URI, right?
>
> On Tue, Jul 23, 2019 at 11:57 AM Majewski, Steven Dennis (sdm7g) <
> sd...@virginia.edu> wrote:
>
>>
>> So the next question is how do you make the subsequent calls to retrieve
>> the next 200, etc.?
>>
>>
>>
>> You call  /repositories/$repo/resources/$id/tree/waypoint?offset=$N  23
>> times.
>> ( You already got the first batch in .precomputed_waypoints in the call
>> to /ress/root  )
>>
>>
>> I found the documentation note in the source I was looking for:
>>
>> https://github.com/archivesspace/archivesspace/blob/master/backend/app/model/large_tree.rb
>>
>>
>> # What's the big idea?
>> #
>> # ArchivesSpace has some big trees in it, and sometimes they look a lot
>> like big
>> # sticks.  Back in the dark ages, we used JSTree for our trees, which in
>> general
>> # is perfectly cromulent.  We recognized the risk of having some very
>> large
>> # collections, so dutifully configured JSTree to lazily load subtrees as
>> the
>> # user expanded them (avoiding having to load the full tree into memory
>> right
>> # away).
>> #
>> # However, time makes fools of us all.  The JSTree approach works fine if
>> your
>> # tree is fairly well balanced, but that's not what things look like in
>> the real
>> # world.  Some trees have a single root node and tens of thousands of
>> records
>> # directly underneath it.  Lazy loading at the subtree level doesn't save
>> you
>> # here: as soon as you expand that (single) node, you're toast.
>> #
>> # This "large tree" business is a way around all of this.  It's
>> effectively a
>> # hybrid of trees and pagination, except we call the pages "waypoints" for
>> # reasons known only to me.  So here's the big idea:
>> #
>> #  * You want to show a tree.  You ask the API to give you the root node.
>> #
>> #  * The root node tells you whether or not it has children, how many
>> children,
>> #and how many waypoints that works out to.
>> #
>> #  * Each waypoint is a fixed-size page of nodes.  If the waypoint size
>> is set
>> #to 200, a node with 1,000 children would have 5 waypoints underneath
>> it.
>> #
>> #  * So, to display the records underneath the root node, you fetch the
>> root
>> #node, then fetch the first waypoint to get the first N nodes.  If
>> you need
>> #to show more 

Re: [Archivesspace_Users_Group] Retrieving tree info via API (what are "waypoints"?)

[Majewski, Steven Dennis (sdm7g)]

I believe for the next level of archival_objects, you have to get 
/repositories/$REPO/archival_objects/$ID/children , but check the API docs.


Note that there is also a GET /repositories/$REPO/resources/$ID/ordered_records 
method that gives you the whole hierarchy, but minimal info about each 
resource:  { ref: display_string:, depth:, level: } 

I don’t think I knew about that one the first time I was wrestling with this 
sort of task. 
If you’re doing backend API and not worried about real time display update, it 
might make more sense to walk the output ordered_records 
If you want more complete info on resource children. 


— Steve. 


> On Jul 23, 2019, at 12:11 PM, Trevor Thornton  wrote:
> 
> Just found that file in the repo before I saw your message and I think I 
> understand now - thanks!
> 
> So, if you're looking at a node below the root (an ArchivalObject) that has 
> >200 children, you would hit the ".../tree/waypoint" endpoint however many 
> times and include "parent_node" in the GET params with the ArchivalObject 
> URI, right?
> 
> On Tue, Jul 23, 2019 at 11:57 AM Majewski, Steven Dennis (sdm7g) 
> mailto:sd...@virginia.edu>> wrote:
> 
>> So the next question is how do you make the subsequent calls to retrieve the 
>> next 200, etc.?
> 
> 
> 
> You call  /repositories/$repo/resources/$id/tree/waypoint?offset=$N  23 
> times. 
> ( You already got the first batch in .precomputed_waypoints in the call to 
> /ress/root  ) 
> 
> 
> I found the documentation note in the source I was looking for: 
> https://github.com/archivesspace/archivesspace/blob/master/backend/app/model/large_tree.rb
>  
> 
> 
> 
> # What's the big idea?
> #
> # ArchivesSpace has some big trees in it, and sometimes they look a lot like 
> big
> # sticks.  Back in the dark ages, we used JSTree for our trees, which in 
> general
> # is perfectly cromulent.  We recognized the risk of having some very large
> # collections, so dutifully configured JSTree to lazily load subtrees as the
> # user expanded them (avoiding having to load the full tree into memory right
> # away).
> #
> # However, time makes fools of us all.  The JSTree approach works fine if your
> # tree is fairly well balanced, but that's not what things look like in the 
> real
> # world.  Some trees have a single root node and tens of thousands of records
> # directly underneath it.  Lazy loading at the subtree level doesn't save you
> # here: as soon as you expand that (single) node, you're toast.
> #
> # This "large tree" business is a way around all of this.  It's effectively a
> # hybrid of trees and pagination, except we call the pages "waypoints" for
> # reasons known only to me.  So here's the big idea:
> #
> #  * You want to show a tree.  You ask the API to give you the root node.
> #
> #  * The root node tells you whether or not it has children, how many 
> children,
> #and how many waypoints that works out to.
> #
> #  * Each waypoint is a fixed-size page of nodes.  If the waypoint size is set
> #to 200, a node with 1,000 children would have 5 waypoints underneath it.
> #
> #  * So, to display the records underneath the root node, you fetch the root
> #node, then fetch the first waypoint to get the first N nodes.  If you 
> need
> #to show more nodes (i.e. if the user has scrolled down), you fetch the
> #second waypoint, and so on.
> #
> #  * The records underneath the root might have their own children, and 
> they'll
> #have their own waypoints that you can fetch in the same way.  It's nodes,
> #waypoints and turtles the whole way down.
> #
> # All of this interacts with the largetree.js code in the staff and public
> # interfaces.  You open a resource record, and largetree.js fetches the root
> # node and inserts placeholders for each waypoint underneath it.  As the user
> # scrolls towards a placeholder, the code starts building tracks ahead of the
> # train, fetching that waypoint and rendering the records it contains.  When a
> # user expands a node to view its children, that process repeats again (the 
> node
> # is fetched, waypoint placeholders inserted, etc.).
> #
> # The public interface runs the same code as the staff interface, but with a
> # small twist: it fetches its nodes and waypoints from Solr, rather than from
> # the live API.  We hit the API endpoints at indexing time and store them as
> # Solr documents, effectively precomputing all of the bits of data we need 
> when
> # displaying trees.
> 
> 
> 
> 
> 
>> On Jul 23, 2019, at 11:08 AM, Trevor Thornton > > wrote:
>> 
>> Thanks, Steve. That makes sense, and I tested with a resource with >1000 top 
>> level children and I see that only 200 of them are included, which 
>> corresponds to the value for "waypoint_size" in the response:
>> 
>> {  
>>"child_count":4780,
>>"waypoints":24,
>>"waypoint_size":200
>> ...
>> 
>> So 

Re: [Archivesspace_Users_Group] Retrieving tree info via API (what are "waypoints"?)

[Trevor Thornton]

Just found that file in the repo before I saw your message and I think I
understand now - thanks!

So, if you're looking at a node below the root (an ArchivalObject) that has
>200 children, you would hit the ".../tree/waypoint" endpoint however many
times and include "parent_node" in the GET params with the ArchivalObject
URI, right?

On Tue, Jul 23, 2019 at 11:57 AM Majewski, Steven Dennis (sdm7g) <
sd...@virginia.edu> wrote:

>
> So the next question is how do you make the subsequent calls to retrieve
> the next 200, etc.?
>
>
>
> You call  /repositories/$repo/resources/$id/tree/waypoint?offset=$N  23
> times.
> ( You already got the first batch in .precomputed_waypoints in the call to
> /ress/root  )
>
>
> I found the documentation note in the source I was looking for:
>
> https://github.com/archivesspace/archivesspace/blob/master/backend/app/model/large_tree.rb
>
>
> # What's the big idea?
> #
> # ArchivesSpace has some big trees in it, and sometimes they look a lot
> like big
> # sticks.  Back in the dark ages, we used JSTree for our trees, which in
> general
> # is perfectly cromulent.  We recognized the risk of having some very large
> # collections, so dutifully configured JSTree to lazily load subtrees as
> the
> # user expanded them (avoiding having to load the full tree into memory
> right
> # away).
> #
> # However, time makes fools of us all.  The JSTree approach works fine if
> your
> # tree is fairly well balanced, but that's not what things look like in
> the real
> # world.  Some trees have a single root node and tens of thousands of
> records
> # directly underneath it.  Lazy loading at the subtree level doesn't save
> you
> # here: as soon as you expand that (single) node, you're toast.
> #
> # This "large tree" business is a way around all of this.  It's
> effectively a
> # hybrid of trees and pagination, except we call the pages "waypoints" for
> # reasons known only to me.  So here's the big idea:
> #
> #  * You want to show a tree.  You ask the API to give you the root node.
> #
> #  * The root node tells you whether or not it has children, how many
> children,
> #and how many waypoints that works out to.
> #
> #  * Each waypoint is a fixed-size page of nodes.  If the waypoint size is
> set
> #to 200, a node with 1,000 children would have 5 waypoints underneath
> it.
> #
> #  * So, to display the records underneath the root node, you fetch the
> root
> #node, then fetch the first waypoint to get the first N nodes.  If you
> need
> #to show more nodes (i.e. if the user has scrolled down), you fetch the
> #second waypoint, and so on.
> #
> #  * The records underneath the root might have their own children, and
> they'll
> #have their own waypoints that you can fetch in the same way.  It's
> nodes,
> #waypoints and turtles the whole way down.
> #
> # All of this interacts with the largetree.js code in the staff and public
> # interfaces.  You open a resource record, and largetree.js fetches the
> root
> # node and inserts placeholders for each waypoint underneath it.  As the
> user
> # scrolls towards a placeholder, the code starts building tracks ahead of
> the
> # train, fetching that waypoint and rendering the records it contains.
> When a
> # user expands a node to view its children, that process repeats again
> (the node
> # is fetched, waypoint placeholders inserted, etc.).
> #
> # The public interface runs the same code as the staff interface, but with
> a
> # small twist: it fetches its nodes and waypoints from Solr, rather than
> from
> # the live API.  We hit the API endpoints at indexing time and store them
> as
> # Solr documents, effectively precomputing all of the bits of data we need
> when
> # displaying trees.
>
>
>
>
>
> On Jul 23, 2019, at 11:08 AM, Trevor Thornton  wrote:
>
> Thanks, Steve. That makes sense, and I tested with a resource with >1000
> top level children and I see that only 200 of them are included, which
> corresponds to the value for "waypoint_size" in the response:
>
> {
>>"child_count":4780,
>>"waypoints":24,
>>"waypoint_size":200
>> ...
>
>
> So the next question is how do you make the subsequent calls to retrieve
> the next 200, etc.?
>
> On Tue, Jul 23, 2019 at 10:52 AM Majewski, Steven Dennis (sdm7g) <
> sd...@virginia.edu> wrote:
>
>> I believe the rationale of the waypoints was that initially, it was
>> expected that resource children/ archival objects would fall into a more
>> balanced tree structure, but it turned out that there were many flat
>> hierarchies with hundreds of top level children, and getting all of the
>> children at once was not working very efficiently. So with they waypoint
>> calls, you may only be getting some of the children, but the display can
>> start populating the tree display while making additional calls for the
>> rest.
>>
>> I may have some postman examples and internal notes around somewhere:
>> I’ll see what I can dig out.
>>
>> — Steve.
>>
>>
>> On Jul 23, 2019, at 

Re: [Archivesspace_Users_Group] Retrieving tree info via API (what are "waypoints"?)

[Majewski, Steven Dennis (sdm7g)]

> So the next question is how do you make the subsequent calls to retrieve the 
> next 200, etc.?



You call  /repositories/$repo/resources/$id/tree/waypoint?offset=$N  23 times. 
( You already got the first batch in .precomputed_waypoints in the call to 
/ress/root  ) 


I found the documentation note in the source I was looking for: 
https://github.com/archivesspace/archivesspace/blob/master/backend/app/model/large_tree.rb
 



# What's the big idea?
#
# ArchivesSpace has some big trees in it, and sometimes they look a lot like big
# sticks.  Back in the dark ages, we used JSTree for our trees, which in general
# is perfectly cromulent.  We recognized the risk of having some very large
# collections, so dutifully configured JSTree to lazily load subtrees as the
# user expanded them (avoiding having to load the full tree into memory right
# away).
#
# However, time makes fools of us all.  The JSTree approach works fine if your
# tree is fairly well balanced, but that's not what things look like in the real
# world.  Some trees have a single root node and tens of thousands of records
# directly underneath it.  Lazy loading at the subtree level doesn't save you
# here: as soon as you expand that (single) node, you're toast.
#
# This "large tree" business is a way around all of this.  It's effectively a
# hybrid of trees and pagination, except we call the pages "waypoints" for
# reasons known only to me.  So here's the big idea:
#
#  * You want to show a tree.  You ask the API to give you the root node.
#
#  * The root node tells you whether or not it has children, how many children,
#and how many waypoints that works out to.
#
#  * Each waypoint is a fixed-size page of nodes.  If the waypoint size is set
#to 200, a node with 1,000 children would have 5 waypoints underneath it.
#
#  * So, to display the records underneath the root node, you fetch the root
#node, then fetch the first waypoint to get the first N nodes.  If you need
#to show more nodes (i.e. if the user has scrolled down), you fetch the
#second waypoint, and so on.
#
#  * The records underneath the root might have their own children, and they'll
#have their own waypoints that you can fetch in the same way.  It's nodes,
#waypoints and turtles the whole way down.
#
# All of this interacts with the largetree.js code in the staff and public
# interfaces.  You open a resource record, and largetree.js fetches the root
# node and inserts placeholders for each waypoint underneath it.  As the user
# scrolls towards a placeholder, the code starts building tracks ahead of the
# train, fetching that waypoint and rendering the records it contains.  When a
# user expands a node to view its children, that process repeats again (the node
# is fetched, waypoint placeholders inserted, etc.).
#
# The public interface runs the same code as the staff interface, but with a
# small twist: it fetches its nodes and waypoints from Solr, rather than from
# the live API.  We hit the API endpoints at indexing time and store them as
# Solr documents, effectively precomputing all of the bits of data we need when
# displaying trees.





> On Jul 23, 2019, at 11:08 AM, Trevor Thornton  wrote:
> 
> Thanks, Steve. That makes sense, and I tested with a resource with >1000 top 
> level children and I see that only 200 of them are included, which 
> corresponds to the value for "waypoint_size" in the response:
> 
> {  
>"child_count":4780,
>"waypoints":24,
>"waypoint_size":200
> ...
> 
> So the next question is how do you make the subsequent calls to retrieve the 
> next 200, etc.?
> 
> On Tue, Jul 23, 2019 at 10:52 AM Majewski, Steven Dennis (sdm7g) 
> mailto:sd...@virginia.edu>> wrote:
> I believe the rationale of the waypoints was that initially, it was expected 
> that resource children/ archival objects would fall into a more balanced tree 
> structure, but it turned out that there were many flat hierarchies with 
> hundreds of top level children, and getting all of the children at once was 
> not working very efficiently. So with they waypoint calls, you may only be 
> getting some of the children, but the display can start populating the tree 
> display while making additional calls for the rest. 
> 
> I may have some postman examples and internal notes around somewhere: I’ll 
> see what I can dig out. 
> 
> — Steve. 
> 
> 
>> On Jul 23, 2019, at 9:05 AM, Trevor Thornton > > wrote:
>> 
>> Hi everybody-
>> 
>> I'm building a service using these API endpoints (or I think I am):
>> [:GET] /repositories/:repo_id/resources/:id/tree/root 
>> 
>> [:GET] /repositories/:repo_id/resources/:id/tree/node 
>> 

Re: [Archivesspace_Users_Group] Retrieving tree info via API (what are "waypoints"?)

[Trevor Thornton]

Thanks, Steve. That makes sense, and I tested with a resource with >1000
top level children and I see that only 200 of them are included, which
corresponds to the value for "waypoint_size" in the response:

{
>"child_count":4780,
>"waypoints":24,
>"waypoint_size":200
> ...


So the next question is how do you make the subsequent calls to retrieve
the next 200, etc.?

On Tue, Jul 23, 2019 at 10:52 AM Majewski, Steven Dennis (sdm7g) <
sd...@virginia.edu> wrote:

> I believe the rationale of the waypoints was that initially, it was
> expected that resource children/ archival objects would fall into a more
> balanced tree structure, but it turned out that there were many flat
> hierarchies with hundreds of top level children, and getting all of the
> children at once was not working very efficiently. So with they waypoint
> calls, you may only be getting some of the children, but the display can
> start populating the tree display while making additional calls for the
> rest.
>
> I may have some postman examples and internal notes around somewhere: I’ll
> see what I can dig out.
>
> — Steve.
>
>
> On Jul 23, 2019, at 9:05 AM, Trevor Thornton  wrote:
>
> Hi everybody-
>
> I'm building a service using these API endpoints (or I think I am):
> [:GET] /repositories/:repo_id/resources/:id/tree/root
> 
> [:GET] /repositories/:repo_id/resources/:id/tree/node
> 
>
> These incorporate the concept of "waypoints", which I admit that I'm not
> familiar with in this context, and it isn't explained very well in the
> documentation. This is what I have to work with (these are elements
> included in the API response):
>
>- child_count – the number of immediate children
>- waypoints – the number of “waypoints” those children are grouped into
>- waypoint_size – the number of children in each waypoint
>- precomputed_waypoints – a collection of arrays (keyed on child URI)
>in the same format as returned by the ’/waypoint’ endpoint. Since a fetch
>for a given node is almost always followed by a fetch of the first
>waypoint, using the information in this structure can save a backend call.
>
> Can anyone explain what exactly waypoints are and how they are different
> from children? In the examples I've seen, the "precomputed_waypoints"
> element in the response looks like a convoluted way (an array value of the
> lone element in an object, which is itself the value of the lone element in
> another object) to provide the children nodes of the given node (or root).
> What's the difference?
>
> Thanks,
> Trevor
>
> --
> Trevor Thornton
> Applications Developer, Digital Library Initiatives
> North Carolina State University Libraries
> ___
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group@lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>
>
> ___
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group@lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
>


-- 
Trevor Thornton
Applications Developer, Digital Library Initiatives
North Carolina State University Libraries
___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

Re: [Archivesspace_Users_Group] Retrieving tree info via API (what are "waypoints"?)

[Majewski, Steven Dennis (sdm7g)]

I believe the rationale of the waypoints was that initially, it was expected 
that resource children/ archival objects would fall into a more balanced tree 
structure, but it turned out that there were many flat hierarchies with 
hundreds of top level children, and getting all of the children at once was not 
working very efficiently. So with they waypoint calls, you may only be getting 
some of the children, but the display can start populating the tree display 
while making additional calls for the rest. 

I may have some postman examples and internal notes around somewhere: I’ll see 
what I can dig out. 

— Steve. 


> On Jul 23, 2019, at 9:05 AM, Trevor Thornton  wrote:
> 
> Hi everybody-
> 
> I'm building a service using these API endpoints (or I think I am):
> [:GET] /repositories/:repo_id/resources/:id/tree/root 
> 
> [:GET] /repositories/:repo_id/resources/:id/tree/node 
> 
> 
> These incorporate the concept of "waypoints", which I admit that I'm not 
> familiar with in this context, and it isn't explained very well in the 
> documentation. This is what I have to work with (these are elements included 
> in the API response):
> child_count – the number of immediate children
> waypoints – the number of “waypoints” those children are grouped into
> waypoint_size – the number of children in each waypoint
> precomputed_waypoints – a collection of arrays (keyed on child URI) in the 
> same format as returned by the ’/waypoint’ endpoint. Since a fetch for a 
> given node is almost always followed by a fetch of the first waypoint, using 
> the information in this structure can save a backend call.
> Can anyone explain what exactly waypoints are and how they are different from 
> children? In the examples I've seen, the "precomputed_waypoints" element in 
> the response looks like a convoluted way (an array value of the lone element 
> in an object, which is itself the value of the lone element in another 
> object) to provide the children nodes of the given node (or root). What's the 
> difference?
> 
> Thanks,
> Trevor
> 
> -- 
> Trevor Thornton
> Applications Developer, Digital Library Initiatives
> North Carolina State University Libraries
> ___
> Archivesspace_Users_Group mailing list
> Archivesspace_Users_Group@lyralists.lyrasis.org
> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group



smime.p7s
Description: S/MIME cryptographic signature
___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

Re: [Archivesspace_Users_Group] proxy rewriting gone awry

[Swanson, Bob]

BTW, I re-index overnight.
It takes a LONG time.

From: archivesspace_users_group-boun...@lyralists.lyrasis.org 
 On Behalf Of Swanson, 
Bob
Sent: Tuesday, July 23, 2019 9:39 AM
To: Archivesspace Users Group 
Subject: Re: [Archivesspace_Users_Group] proxy rewriting gone awry

I’m not very good at this, there are smarter folks than me out there,
But when I run into a funky problem like that I re-index and it fixes it.

https://archivesspace.github.io/archivesspace/user/re-creating-indexes/

From: 
archivesspace_users_group-boun...@lyralists.lyrasis.org
 
mailto:archivesspace_users_group-boun...@lyralists.lyrasis.org>>
 On Behalf Of Kimberli Kelmor
Sent: Tuesday, July 23, 2019 9:19 AM
To: 
archivesspace_users_group@lyralists.lyrasis.org
Subject: [Archivesspace_Users_Group] proxy rewriting gone awry

All of a sudden today the public link to the repositories is broken. The public 
can list the repositories, but when they click on a link it gets a 404 not 
found.

 The url for the list of repositories is 
website.edu/public/repositories
 -- this works
  The url for one of the repositories is 
website.edu/repositories/2
 -- this does not work

If I manually make it 
website.edu/public/repositories/2
 -- this works for the actual content, but the images are broken.

I'm new to all this, but is something messing up the proxy settings or is that 
barking up the wrong tree?

Everything (config.rb and apache conf.d) is set up per the 'running 
archivesspace under a prefix' documentation.

Running version 2.6.0 -- but that has been running for about a week with no 
problem.

Any help greatly appreciated!
Kim

--
Kimberli M. Kelmor
Head of Law Library Technology
Georgetown University Law Library
kk1...@georgetown.edu
202-662-9158
___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

Re: [Archivesspace_Users_Group] proxy rewriting gone awry

[Swanson, Bob]

I’m not very good at this, there are smarter folks than me out there,
But when I run into a funky problem like that I re-index and it fixes it.

https://archivesspace.github.io/archivesspace/user/re-creating-indexes/

From: archivesspace_users_group-boun...@lyralists.lyrasis.org 
 On Behalf Of Kimberli 
Kelmor
Sent: Tuesday, July 23, 2019 9:19 AM
To: archivesspace_users_group@lyralists.lyrasis.org
Subject: [Archivesspace_Users_Group] proxy rewriting gone awry

All of a sudden today the public link to the repositories is broken. The public 
can list the repositories, but when they click on a link it gets a 404 not 
found.

 The url for the list of repositories is 
website.edu/public/repositories
 -- this works
  The url for one of the repositories is 
website.edu/repositories/2
 -- this does not work

If I manually make it 
website.edu/public/repositories/2
 -- this works for the actual content, but the images are broken.

I'm new to all this, but is something messing up the proxy settings or is that 
barking up the wrong tree?

Everything (config.rb and apache conf.d) is set up per the 'running 
archivesspace under a prefix' documentation.

Running version 2.6.0 -- but that has been running for about a week with no 
problem.

Any help greatly appreciated!
Kim

--
Kimberli M. Kelmor
Head of Law Library Technology
Georgetown University Law Library
kk1...@georgetown.edu
202-662-9158
___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

Re: [Archivesspace_Users_Group] proxy rewriting gone awry

[Blake Carver]

I'd start by looking in the logs (Both Apache and ArchivesSpace), any errors in 
there?

From: archivesspace_users_group-boun...@lyralists.lyrasis.org 
 on behalf of Kimberli 
Kelmor 
Sent: Tuesday, July 23, 2019 9:18 AM
To: archivesspace_users_group@lyralists.lyrasis.org 

Subject: [Archivesspace_Users_Group] proxy rewriting gone awry

All of a sudden today the public link to the repositories is broken. The public 
can list the repositories, but when they click on a link it gets a 404 not 
found.

 The url for the list of repositories is 
website.edu/public/repositories -- this 
works
  The url for one of the repositories is 
website.edu/repositories/2 -- this does not 
work

If I manually make it 
website.edu/public/repositories/2 -- 
this works for the actual content, but the images are broken.

I'm new to all this, but is something messing up the proxy settings or is that 
barking up the wrong tree?

Everything (config.rb and apache conf.d) is set up per the 'running 
archivesspace under a prefix' documentation.

Running version 2.6.0 -- but that has been running for about a week with no 
problem.

Any help greatly appreciated!
Kim

--
Kimberli M. Kelmor
Head of Law Library Technology
Georgetown University Law Library
kk1...@georgetown.edu
202-662-9158
___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

[Archivesspace_Users_Group] proxy rewriting gone awry

[Kimberli Kelmor]

All of a sudden today the public link to the repositories is broken. The
public can list the repositories, but when they click on a link it gets a
404 not found.

 The url for the list of repositories is website.edu/public/repositories --
this works
  The url for one of the repositories is website.edu/repositories/2 -- this
does not work

If I manually make it website.edu/public/repositories/2 -- this works for
the actual content, but the images are broken.

I'm new to all this, but is something messing up the proxy settings or is
that barking up the wrong tree?

Everything (config.rb and apache conf.d) is set up per the 'running
archivesspace under a prefix' documentation.

Running version 2.6.0 -- but that has been running for about a week with no
problem.

Any help greatly appreciated!
Kim

-- 
Kimberli M. Kelmor
Head of Law Library Technology
Georgetown University Law Library
kk1...@georgetown.edu
202-662-9158
___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group

[Archivesspace_Users_Group] Retrieving tree info via API (what are "waypoints"?)

[Trevor Thornton]

Hi everybody-

I'm building a service using these API endpoints (or I think I am):
[:GET] /repositories/:repo_id/resources/:id/tree/root

[:GET] /repositories/:repo_id/resources/:id/tree/node


These incorporate the concept of "waypoints", which I admit that I'm not
familiar with in this context, and it isn't explained very well in the
documentation. This is what I have to work with (these are elements
included in the API response):

   - child_count – the number of immediate children
   - waypoints – the number of “waypoints” those children are grouped into
   - waypoint_size – the number of children in each waypoint
   - precomputed_waypoints – a collection of arrays (keyed on child URI) in
   the same format as returned by the ’/waypoint’ endpoint. Since a fetch for
   a given node is almost always followed by a fetch of the first waypoint,
   using the information in this structure can save a backend call.

Can anyone explain what exactly waypoints are and how they are different
from children? In the examples I've seen, the "precomputed_waypoints"
element in the response looks like a convoluted way (an array value of the
lone element in an object, which is itself the value of the lone element in
another object) to provide the children nodes of the given node (or root).
What's the difference?

Thanks,
Trevor

-- 
Trevor Thornton
Applications Developer, Digital Library Initiatives
North Carolina State University Libraries
___
Archivesspace_Users_Group mailing list
Archivesspace_Users_Group@lyralists.lyrasis.org
http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group