[ https://issues.apache.org/jira/browse/SOLR-11653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Smiley updated SOLR-11653: -------------------------------- Attachment: SOLR-11653.patch Updated patch, significantly moving this issue to being done. I tested it which uncovered various bugs which I fixed. I tweaked some but not all metadata param names to align with SOLR-11722; we can fully align them here or there. Unfortunately there's a bit of redundant stuff in that issue and this one (and internally in this one between the URP & Cmd) that we can resolve in a separate issue to tidy up duplication. Some reflection on changes in this patch: Overseer & OverseerTaskProcessor: added checks for "operation" being null to pass a clear error. Of course this should never happen in bug free code, but my code wasn't bug free at first :-). In OTP I left a nocommit because I think the order of the nearby code should be reordered a little (a very minor refactoring). The patch won't easily show this as there isn't enough contextual lines of code. I think operation should be gotten & verified (not null), then check for the asyncId logic, and then acquire the messageHandler (thus immediately before the MH is used). CollectionsHandler.waitForActiveCollection: I made this method public because it's important to check for this level of collection readiness before we actually use it. I had hoped waitForFinalState would be sufficient but tests proved it wasn't. Having to call this feels bad; I have to wonder if Collection creation should internally wait for being "active"; what's the harm in that? IMO CollectionsHandler.handleResponse is a bad name... it's name suggest it only handles the response when it performs the intended action too! I have TODOs to rename to submitToOverseerRPC since it submits to the Overseer queue and it does so in an RPC way (expects a response; it's not just a message on a queue). Input on the name is welcome. Added TZ timezone handling for working with the interval. What I want to do now is improve testing for race conditions by having the test submit documents in parallel. I may also get to a minor TODO involving the addition of a Lock to reduce wasted concurrent calls by the URP to the Overseer to create a new collection. This is minor because the RoutedAliasCreateCollectionCmd doesn't do anything if ifHeadCollName doesn't jive with the actual head collection name. [~gus_heck] can you please review this now? > create next time collection based on a fixed time gap > ----------------------------------------------------- > > Key: SOLR-11653 > URL: https://issues.apache.org/jira/browse/SOLR-11653 > Project: Solr > Issue Type: Sub-task > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud > Reporter: David Smiley > Assignee: David Smiley > Attachments: SOLR-11653.patch, SOLR-11653.patch > > > For time series collections (as part of a collection Alias with certain > metadata), we want to automatically add new collections. In this issue, this > is about creating the next collection based on a configurable fixed time gap. > And we will also add this collection synchronously once a document flowing > through the URP chain exceeds the gap, as opposed to asynchronously in > advance. There will be some Alias metadata to define in this issue. The > preponderance of the implementation will be in TimePartitionedUpdateProcessor > or perhaps a helper to this URP. > note: other issues will implement pre-emptive creation and capping > collections by size. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org