[
https://issues.apache.org/jira/browse/SOLR-13420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16831648#comment-16831648
]
Gus Heck commented on SOLR-13420:
---------------------------------
{quote}At least right now, the watched collections and collection properties
are not bundled
{quote}
I haven't changed anything with collection watches (CollectionStateWatcher,
etc)? My changes only relate to CollectionProperties and their associated
watches (PropsWatch for zk and CollectionPropsWatch for user notifications).
Where do you see a change to or bundling with collection watches?
{quote}all you need to do is register a watcher on the properties somewhere in
the TRA code
{quote}
If I register the watcher in, or accessible by the static
RoutedAliasUpdateProcessor.wrap() method (where i'm consulting the properties)
how does it ever get unregistered when the collection is deleted or moved? I'll
never be notified that the collection is gone, thus I'll never unregister the
watch. This will create the same sort of memory/watch leak you were worried
about earlier.
{quote}Maybe I’m missing something, but if someone calls
{{registerCollectionPropsWatcher}}, then {{collectionPropsWatches}} will have
“collection1”->properties_of_collection1.
{quote}
Yes
{quote}If you call {{getCollectionProperties}} on “collection1”, you get the
values from cache,
{quote}
Yes
{quote}however, the first time there is a change in the properties, the watch
will be unset
{quote}
Yes, this is normal in Zk, but the code re-registers the watch when it makes
it's call to getData() to read the change indicated by the notification:
{code:java}
... events are received by a method on PropsWatch...
public void process(WatchedEvent event) {
... which calls...
refreshAndWatch(true); // also on PropsWatch
... which calls ...
VersionedCollectionProps vcp = fetchCollectionProperties(coll, this);
// this ==> the instance of PropsWatch
... which calls ...
byte[] data = zkClient.getData(znodePath, watcher, stat, true); //
watcher is the parameter that received "this" above{code}
That pattern was in place before my changes. The watch is always re-applied
immediately every time a event is processed, unless there is +no longer a
cached copy+ . In that case refreshAndWatch() is not called due to the check
shown below. This breaks the event refresh cycle, therefore avoiding the leaks
you were worried about previously:
{code:java}
if (!watchedCollectionProps.containsKey(coll)) {
// No one can be notified of the change, we can ignore it and "unset"
the watch
// by not refreshing.
log.debug("Ignoring property change for collection {}, zk watch will
not be renewed", coll);
return;
}
{code}
{quote}(because watchedCollectionProps doesn’t necessarily include the same
collections as collectionPropsWatches).
{quote}
Yes, but the watchedCollectionProps keyset is always a superset of
collectionPropsWatches key set. Registering a CollectionPropsWatcher creates a
matching PropsWatch and a cached entry, neither of which can expire so long as
a the collectionPropsWatches map contains a CollectionPropsWatcher for that
collection exists.
Also recall that there is a separate node for each collection's properties.
(hence the need for this method:
{code:java}
static String getCollectionPropsPath(final String collection) {
return COLLECTIONS_ZKNODE + '/' + collection + '/' +
COLLECTION_PROPS_ZKNODE;
} {code}
Therefore each and every collection's props will have it's own watch
(represented by a PropsWatch instance on the java side) and it's own cache
entry. What happens to one collection does not effect the caching for the
others.
Also, at no time are the user's watches for collection properties (instances of
CollectionPropsWatcher implementations) un-registered for them, so those simply
exist until they are unregistered by the user (no change there).
{quote}Since the watch is unset, new changes in the collection properties won’t
be updated.
{quote}
As discussed above, the watch is re-applied immediately upon read of the data
so we will be notified of further changes.
> Allow Routed Aliases to use Collection Properties instead of core properties
> ----------------------------------------------------------------------------
>
> Key: SOLR-13420
> URL: https://issues.apache.org/jira/browse/SOLR-13420
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
> Affects Versions: master (9.0)
> Reporter: Gus Heck
> Assignee: Gus Heck
> Priority: Major
> Attachments: SOLR-13420.patch, SOLR-13420.patch, SOLR-13420.patch
>
>
> The current routed alias code is relying on a core property named
> routedAliasName to detect when the Routed Alias wrapper URP should be applied
> to Distributed Update Request Processor.
> {code:java}
> #Written by CorePropertiesLocator
> #Sun Mar 03 06:21:14 UTC 2019
> routedAliasName=testalias21
> numShards=2
> collection.configName=_default
> ... etc...
> {code}
> Core properties are not changeable after the core is created, and they are
> written to the file system for every core. To support a unit test for
> SOLR-13419 I need to create some legacy formatted collection names, and
> arrange them into a TRA, but this is impossible due to the fact that I can't
> change the core property from the test. There's a TODO dating back to the
> original TRA implementation in the routed alias code to switch to collection
> properties instead, so this ticket will address that TOD to support the test
> required for SOLR-13419.
> Back compatibility with legacy core based TRA's and CRA's will of course be
> maintained. I also expect that this will facilitate some more nimble handling
> or routed aliases with future auto-scaling capabilities such as possibly
> detaching and archiving collections to cheaper, slower machines rather than
> deleting them. (presently such a collection would still attempt to use the
> routed alias if it received an update even if it were no longer in the list
> of collections for the alias)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
