> So the next question is how do you make the subsequent calls to retrieve the > next 200, etc.?
You call /repositories/$repo/resources/$id/tree/waypoint?offset=$N 23 times. ( You already got the first batch in .precomputed_waypoints in the call to /ress/root ) I found the documentation note in the source I was looking for: https://github.com/archivesspace/archivesspace/blob/master/backend/app/model/large_tree.rb <https://github.com/archivesspace/archivesspace/blob/master/backend/app/model/large_tree.rb> # What's the big idea? # # ArchivesSpace has some big trees in it, and sometimes they look a lot like big # sticks. Back in the dark ages, we used JSTree for our trees, which in general # is perfectly cromulent. We recognized the risk of having some very large # collections, so dutifully configured JSTree to lazily load subtrees as the # user expanded them (avoiding having to load the full tree into memory right # away). # # However, time makes fools of us all. The JSTree approach works fine if your # tree is fairly well balanced, but that's not what things look like in the real # world. Some trees have a single root node and tens of thousands of records # directly underneath it. Lazy loading at the subtree level doesn't save you # here: as soon as you expand that (single) node, you're toast. # # This "large tree" business is a way around all of this. It's effectively a # hybrid of trees and pagination, except we call the pages "waypoints" for # reasons known only to me. So here's the big idea: # # * You want to show a tree. You ask the API to give you the root node. # # * The root node tells you whether or not it has children, how many children, # and how many waypoints that works out to. # # * Each waypoint is a fixed-size page of nodes. If the waypoint size is set # to 200, a node with 1,000 children would have 5 waypoints underneath it. # # * So, to display the records underneath the root node, you fetch the root # node, then fetch the first waypoint to get the first N nodes. If you need # to show more nodes (i.e. if the user has scrolled down), you fetch the # second waypoint, and so on. # # * The records underneath the root might have their own children, and they'll # have their own waypoints that you can fetch in the same way. It's nodes, # waypoints and turtles the whole way down. # # All of this interacts with the largetree.js code in the staff and public # interfaces. You open a resource record, and largetree.js fetches the root # node and inserts placeholders for each waypoint underneath it. As the user # scrolls towards a placeholder, the code starts building tracks ahead of the # train, fetching that waypoint and rendering the records it contains. When a # user expands a node to view its children, that process repeats again (the node # is fetched, waypoint placeholders inserted, etc.). # # The public interface runs the same code as the staff interface, but with a # small twist: it fetches its nodes and waypoints from Solr, rather than from # the live API. We hit the API endpoints at indexing time and store them as # Solr documents, effectively precomputing all of the bits of data we need when # displaying trees. > On Jul 23, 2019, at 11:08 AM, Trevor Thornton <[email protected]> wrote: > > Thanks, Steve. That makes sense, and I tested with a resource with >1000 top > level children and I see that only 200 of them are included, which > corresponds to the value for "waypoint_size" in the response: > > { > "child_count":4780, > "waypoints":24, > "waypoint_size":200 > ... > > So the next question is how do you make the subsequent calls to retrieve the > next 200, etc.? > > On Tue, Jul 23, 2019 at 10:52 AM Majewski, Steven Dennis (sdm7g) > <[email protected] <mailto:[email protected]>> wrote: > I believe the rationale of the waypoints was that initially, it was expected > that resource children/ archival objects would fall into a more balanced tree > structure, but it turned out that there were many flat hierarchies with > hundreds of top level children, and getting all of the children at once was > not working very efficiently. So with they waypoint calls, you may only be > getting some of the children, but the display can start populating the tree > display while making additional calls for the rest. > > I may have some postman examples and internal notes around somewhere: I’ll > see what I can dig out. > > — Steve. > > >> On Jul 23, 2019, at 9:05 AM, Trevor Thornton <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hi everybody- >> >> I'm building a service using these API endpoints (or I think I am): >> [:GET] /repositories/:repo_id/resources/:id/tree/root >> <http://archivesspace.github.io/archivesspace/api/#fetch-tree-information-for-the-top-level-resource-record> >> [:GET] /repositories/:repo_id/resources/:id/tree/node >> <http://archivesspace.github.io/archivesspace/api/#fetch-tree-information-for-an-archival-object-record-within-a-tree> >> >> These incorporate the concept of "waypoints", which I admit that I'm not >> familiar with in this context, and it isn't explained very well in the >> documentation. This is what I have to work with (these are elements included >> in the API response): >> child_count – the number of immediate children >> waypoints – the number of “waypoints” those children are grouped into >> waypoint_size – the number of children in each waypoint >> precomputed_waypoints – a collection of arrays (keyed on child URI) in the >> same format as returned by the ’/waypoint’ endpoint. Since a fetch for a >> given node is almost always followed by a fetch of the first waypoint, using >> the information in this structure can save a backend call. >> Can anyone explain what exactly waypoints are and how they are different >> from children? In the examples I've seen, the "precomputed_waypoints" >> element in the response looks like a convoluted way (an array value of the >> lone element in an object, which is itself the value of the lone element in >> another object) to provide the children nodes of the given node (or root). >> What's the difference? >> >> Thanks, >> Trevor >> >> -- >> Trevor Thornton >> Applications Developer, Digital Library Initiatives >> North Carolina State University Libraries >> _______________________________________________ >> Archivesspace_Users_Group mailing list >> [email protected] >> <mailto:[email protected]> >> http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group >> <http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group> > > _______________________________________________ > Archivesspace_Users_Group mailing list > [email protected] > <mailto:[email protected]> > http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group > <http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group> > > > -- > Trevor Thornton > Applications Developer, Digital Library Initiatives > North Carolina State University Libraries > _______________________________________________ > Archivesspace_Users_Group mailing list > [email protected] > http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________ Archivesspace_Users_Group mailing list [email protected] http://lyralists.lyrasis.org/mailman/listinfo/archivesspace_users_group
