Hi Matt, TRA's were put into use almost immediately by at least one organization as soon as Dave and I implemented them, and the CRA and DRA's followed because the same organization wanted to further subdivide their data. They have been around for a while, and I'm currently helping a client move to use them, and past clients have adopted them. Always hard to know exactly how much any feature is used, but I have heard other mentions of folks using them, and not heard a failure story yet, so I think so long as your use case is a good fit for them (non-trivial amounts of data, never re-index the same doc with a different routing date, typically data flows in over time, optionally old data needs periodic removal, etc) then they are good. Of course every particular case is individual and there's always the chance that YOU are the lucky one who discovers something subtle, or find a scale at which things break down, but you aren't the first to use it, that's for sure :). They definitely were meant to make it easier to handle large amounts of temporal data.
Also, it's open source so if something needs tweaking the process for that is open and well defined :) As with everything technical, test often and test well. -Gus On Fri, Aug 20, 2021 at 10:07 AM Matt Kuiper <kuipe...@gmail.com> wrote: > Sending this question out again to learn about how well Time Routed Aliases > have worked out for others. > > Would like to know if a number of others have used this approach > successfully as our team is planning for the use of TRAs in a very large > SolrCloud deployment. > > Thanks, > Matt > > On Fri, Aug 13, 2021, 2:56 PM Matt Kuiper <kuipe...@gmail.com> wrote: > > > Thanks David, this test link is helpful. > > > > @David @Gus - From your viewpoint do you see TRAs as an accepted/proven > > technique within SolrCloud? My small POC works great. Would like to > hear > > if others are using TRA in production deployments successfully at scale. > > > > Thanks, > > Matt > > > > On Wed, Aug 11, 2021 at 8:10 PM David Smiley <dsmi...@apache.org> wrote: > > > >> I hope you have success with TRAs! > >> > >> You can delete some number of collections from the rear of the chain, > but > >> you must first update the TRA to exclude these collections. This is > >> tested: > >> > >> > https://github.com/apache/solr/blob/f6c4f8a755603c3049e48eaf9511041252f2dbad/solr/core/src/test/org/apache/solr/update/processor/TimeRoutedAliasUpdateProcessorTest.java#L184 > >> It'd be nice if it would remove itself from the alias. > >> > >> ~ David Smiley > >> Apache Lucene/Solr Search Developer > >> http://www.linkedin.com/in/davidwsmiley > >> > >> > >> On Tue, Aug 10, 2021 at 9:26 PM Matt Kuiper <kuipe...@gmail.com> wrote: > >> > >> > I found some helpful information while testing TRAs: > >> > > >> > For our use-case I am hesitant to set up an autoDeleteAge (unless it > >> can be > >> > modified - still need to test). So I wondered about a little more > >> manual > >> > delete management approach. > >> > > >> > I confirmed that I cannot simply delete a collection that is > registered > >> as > >> > part of a TRA. The delete collection api call will fail with a > message > >> > that the collection is a part of the alias. > >> > > >> > I did learn that I could use the same create TRA api call I used to > >> create > >> > the TRA, but modify the router.start to date more recent than one or > >> more > >> > of the older collections associated with the TRA. Then when I queried > >> the > >> > TRA, I only received documents from the collections after the new > >> > router.start date. Also, I was now able to successfully delete the > older > >> > collections with a standard collection delete command. > >> > > >> > I think this satisfies my initial use-case requirements to be able to > >> > modify an existing TRA and delete older collections. > >> > > >> > Matt > >> > > >> > On Mon, Aug 9, 2021 at 11:27 AM Matt Kuiper <kuipe...@gmail.com> > wrote: > >> > > >> > > Hi Gus, Jan, > >> > > > >> > > I am considering implementing TRA for a large-scale Solr deployment. > >> > Your > >> > > Q&A is helpful! > >> > > > >> > > I am curious if you have experience/ideas regarding modifying the TR > >> > Alias > >> > > when one desires to manually delete old collections or modify the > >> > > router.autoDeleteAge to shorten or extend the delete age. Here's a > >> few > >> > > specific questions? > >> > > > >> > > 1) Can you manually delete an old collection (via collection api) > and > >> > then > >> > > edit the start date (to a more recent date) of the TRA so that it no > >> > longer > >> > > sees/processes the deleted collection? > >> > > 2) Is the only way to manage the deletion of collections within a > TRA > >> > > using the automatic deletion configuration? The router.autoDeleteAge > >> > > parameter. > >> > > 3) If you can only manage deletes using the router.autoDeleteAge > >> > > parameter, are you able to update this parameter to either: > >> > > > >> > > - Set the delete age earlier so that older collections are > >> triggered > >> > > for automatic deletion sooner? > >> > > - Set the delete age to a larger value to extend the life of a > >> > > collection? Say you originally would like the collections to > stay > >> > around > >> > > for 5 years, but then change your mind to 7 years. > >> > > > >> > > I will likely do some experimentation, but am interested to learn if > >> you > >> > > have covered these use-cases with TRA. > >> > > > >> > > Thanks, > >> > > Matt > >> > > > >> > > > >> > > On Fri, Aug 6, 2021 at 8:08 AM Gus Heck <gus.h...@gmail.com> wrote: > >> > > > >> > >> Hi Jan, > >> > >> > >> > >> The key thing to remember about TRA's (or any Routed Alias) is that > >> it > >> > >> only > >> > >> actively does two things: > >> > >> 1) Routes document updates to the correct collection by inspecting > >> the > >> > >> routed field in the document > >> > >> 2) Detects when a new collection is required and creates it. > >> > >> > >> > >> If you don't send it data *nothing* happens. The collections are > not > >> > >> created until data requires them (with an async create possible > when > >> it > >> > >> sees an update that has a timestamp "near" the next interval, see > >> docs > >> > for > >> > >> router.preemptiveCreateMath ) > >> > >> > >> > >> A) Dave's half of our talk at 2018 activate talks about it: > >> > >> https://youtu.be/RB1-7Y5NQeI?t=839 > >> > >> B) Time Routed Aliases are a means by which to automate creation of > >> > >> collections and route documents to the created collections. Sizing, > >> and > >> > >> performance of the individual collections is not otherwise special, > >> and > >> > >> you > >> > >> can interact with the collections individually after they are > >> created, > >> > >> with > >> > >> the obvious caveats that you probably don't want to be doing things > >> that > >> > >> get them out of sync schema wise unless your client programs know > >> how to > >> > >> handle documents of both types etc. A less obvious consequence of > the > >> > >> routing is that your data must not ever republish the same document > >> > with a > >> > >> different route key (date for TRA), since that can lead to > duplicate > >> > id's > >> > >> across collections. The "normal" use case is event data, things > that > >> > >> happened and are done, and are correctly recorded (or at least > their > >> > time > >> > >> is correctly recorded) the first time > >> > >> C) Configure the higher number of replicas, remove old ones > manually > >> if > >> > >> not > >> > >> needed. At query time it's "just an alias". Managing collections > >> based > >> > on > >> > >> recency could be automated here, before autoscaling was deprecated > I > >> was > >> > >> thinking that adding a couple of hooks into autoscaling such that > it > >> > could > >> > >> react to collection creation by a TRA specifically would get us to > a > >> > place > >> > >> much like Elastic's Hot/Warm architecture. I haven't kept track of > >> > what's > >> > >> being done to replace auto scaling however. I think Atri was > >> interested > >> > in > >> > >> that at one point as well. > >> > >> D) TRA's create collections under the hood with a CREATE command > just > >> > like > >> > >> you would manually (based on the config in the TRA). Anything in > Solr > >> > that > >> > >> would influence that placement should apply. > >> > >> E) See D above, for fill rate, Utilizing new nodes over time should > >> be > >> > as > >> > >> simple as adding new nodes and waiting for new collections to be > >> > created. > >> > >> One could also manually move replicas as with any other collection, > >> > >> (aside: > >> > >> be sure to refer to a current version of MOVEREPLICA docs, prior to > >> > >> something like 8.6 they were incomplete and even wrong in a few > >> places). > >> > >> F) If you are talking about router.autoDeleteAge here, old > collection > >> > >> removal is a regular DELETE (just automatically issued), Not sure > >> what > >> > you > >> > >> mean by rotation interval. > >> > >> G) They are just collections with special names that can be parsed > >> > during > >> > >> update to select a destination for the incoming document. > >> > >> H) They are just collections, and there's nothing to prevent you > from > >> > >> upgrading the schema, and new collections will begin using that, > >> > >> individual > >> > >> collections would need to be reloaded, non-safe schema changes (in > >> the > >> > >> usual sense) require a re-index as usual. In a cloud environment > >> where > >> > you > >> > >> can temporarily add machines or disk this is not so bad aside from > >> the > >> > >> time > >> > >> to re-index of course. If you are on-prem then plan to have a > >> > significant > >> > >> level of spare disk to handle this case without running yourself > into > >> > the > >> > >> danger zone for segment merging. > >> > >> H.2) TRA is just an alias with fancy collection creation (and > >> naming). > >> > >> Once > >> > >> they collections exist, it's just an alias. All the action (at this > >> > point) > >> > >> happens at update. So long as the collection is listed in the TRA > in > >> > >> zookeeper in aliases.json ***in the correct, (chronological, desc) > >> > >> order*** > >> > >> and the naming of the collection can be parsed by the TRA code you > >> > should > >> > >> be fine. Incoming updates iterate down the list of collections > >> during an > >> > >> update, and stop at the first one where the collection name matches > >> the > >> > >> date in the routing field for the document for a normal TRA the > vast > >> > >> majority of updates hit one of the most recent two or three > >> collections. > >> > >> Frequent updates to old data in a TRA with very many time slices > (sub > >> > >> collections) might suffer some since this is a simple linear > >> iteration, > >> > >> optimizing that was deferred until it seemed important to someone's > >> less > >> > >> normal use case :). > >> > >> > >> > >> > >> > >> > >> > >> Otherwise it's just an alias of collections with funky looking > names > >> > >> (unless someone added something when I wasn't looking ;) ). > >> > >> > >> > >> -Gus > >> > >> > >> > >> On Fri, Aug 6, 2021 at 4:13 AM Jan Høydahl <jan....@cominvent.com> > >> > wrote: > >> > >> > >> > >> > Hi, > >> > >> > > >> > >> > I have never used TRA, but a client of mine is considering it. A > >> few > >> > >> > questions. > >> > >> > > >> > >> > A) Do you have links to talks (slides/video) on the feature? Or > >> blog > >> > >> posts > >> > >> > going into more detail than the RefGuide? > >> > >> > B) For ingestion performance, sharding may make sense. But only > for > >> > the > >> > >> > current collection. Have anyone tried merging "static" shards? > >> > >> > C) Is there a trick to have more relicas on recent collections > than > >> > old > >> > >> > ones? > >> > >> > D) Is there a way to manage what nodes that get selected for new > >> > >> > collections, or you need to rely on replica placement policies? > >> > >> > E) How do you guys ensure you get a good fill-rate on the nodes, > >> and > >> > >> what > >> > >> > procedure do you use when adding more nodes in the cluster? > >> > >> > * I.e. do you simply add a few new nodes and let Solr > >> > automatically > >> > >> > place new collections onto those? > >> > >> > F) How many sub-collections/cores do you plan for on a single > node? > >> > >> > * You could try to configure the "rotation interval" such > that > >> a > >> > >> node > >> > >> > gets filled by a single core, but that seems hard to predict > >> > >> > * Having a too rapid "rotation interval" will leave behind > too > >> > many > >> > >> > cores per node, causing inefficiencies? > >> > >> > * Have you found a strategy to balance this? I'd likely try > to > >> > plan > >> > >> > for 10 cores per node, and monitor fill-rate such that I > (manually) > >> > add > >> > >> > more HW once a threshold is reached. > >> > >> > G) Have anyone tried backup of a TRA? Does it even work, or do > you > >> > need > >> > >> to > >> > >> > run the command for each single collection? > >> > >> > H) A typical requirement is to migrate all data from one cluster > >> to a > >> > >> new > >> > >> > cluster on a newer version or with a new schema. Have you tried > >> doing > >> > >> that > >> > >> > with a TRA? > >> > >> > * Would you need to migrate each sub collection at a time? > >> > >> > * Will TRA on the new cluster accept that someone "external" > >> adds > >> > >> > collections, and how it is initialized/bootstrapped to fill the > >> > internal > >> > >> > collection registry? > >> > >> > > >> > >> > That's what I could think of before trying the feature. I'm sure > >> there > >> > >> > would be other questions after some trial and error :) > >> > >> > > >> > >> > Jan > >> > >> > >> > >> > >> > >> > >> > >> -- > >> > >> http://www.needhamsoftware.com (work) > >> > >> http://www.the111shift.com (play) > >> > >> > >> > > > >> > > >> > > > -- http://www.needhamsoftware.com (work) http://www.the111shift.com (play)
