[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r498982949
##
File path:
solr/core/src/java/org/apache/solr/cluster/events/impl/CollectionsRepairEventListener.java
##
@@ -0,0 +1,185 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.cluster.events.impl;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import org.apache.solr.client.solrj.SolrClient;
+import org.apache.solr.client.solrj.cloud.SolrCloudManager;
+import org.apache.solr.client.solrj.request.CollectionAdminRequest;
+import org.apache.solr.cloud.api.collections.Assign;
+import org.apache.solr.cluster.events.ClusterEvent;
+import org.apache.solr.cluster.events.ClusterEventListener;
+import org.apache.solr.cluster.events.NodesDownEvent;
+import org.apache.solr.cluster.events.ReplicasDownEvent;
+import org.apache.solr.common.cloud.ClusterState;
+import org.apache.solr.common.cloud.Replica;
+import org.apache.solr.common.cloud.ReplicaPosition;
+import org.apache.solr.core.CoreContainer;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This is an illustration how to re-implement the combination of 8x
+ * NodeLostTrigger and AutoAddReplicasPlanAction to maintain the collection's
replication factor.
+ * NOTE: there's no support for 'waitFor' yet.
+ * NOTE 2: this functionality would be probably more reliable when executed
also as a
+ * periodically scheduled check - both as a reactive (listener) and proactive
(scheduled) measure.
+ */
+public class CollectionsRepairEventListener implements ClusterEventListener {
Review comment:
Is this class registered somewhere to be actually called? If so I missed
that part.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r498957288
##
File path:
solr/core/src/java/org/apache/solr/cluster/events/ClusterEventProducer.java
##
@@ -0,0 +1,100 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.cluster.events;
+
+import org.apache.solr.cloud.ClusterSingleton;
+
+import java.util.Collections;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+
+/**
+ * Component that produces {@link ClusterEvent} instances.
+ */
+public interface ClusterEventProducer extends ClusterSingleton {
+
+ String PLUGIN_NAME = "clusterEventProducer";
+
+ default String getName() {
+return PLUGIN_NAME;
+ }
+
+ /**
+ * Returns a modifiable map of event types and listeners to process events
+ * of a given type.
+ */
+ Map> getEventListeners();
+
+ /**
+ * Register an event listener for processing the specified event types.
+ * @param listener non-null listener. If the same instance of the listener is
+ * already registered it will be ignored.
+ * @param eventTypes non-empty array of event types that this listener
+ * is being registered for. If this is null or empty then
all types will be used.
+ */
+ default void registerListener(ClusterEventListener listener,
ClusterEvent.EventType... eventTypes) throws Exception {
+Objects.requireNonNull(listener);
+if (eventTypes == null || eventTypes.length == 0) {
+ eventTypes = ClusterEvent.EventType.values();
+}
+for (ClusterEvent.EventType type : eventTypes) {
+ Set perType =
getEventListeners().computeIfAbsent(type, t -> ConcurrentHashMap.newKeySet());
+ perType.add(listener);
+}
+ }
+
+ /**
+ * Unregister an event listener.
+ * @param listener non-null listener.
+ */
+ default void unregisterListener(ClusterEventListener listener) {
+Objects.requireNonNull(listener);
+getEventListeners().forEach((type, listeners) -> {
+ listeners.remove(listener);
+});
+ }
+
+ /**
+ * Unregister an event listener for specified event types.
+ * @param listener non-null listener.
+ * @param eventTypes event types from which the listener will be
unregistered. If this
+ * is null or empty then all event types will be used
+ */
+ default void unregisterListener(ClusterEventListener listener,
ClusterEvent.EventType... eventTypes) {
+Objects.requireNonNull(listener);
+if (eventTypes == null || eventTypes.length == 0) {
+ eventTypes = ClusterEvent.EventType.values();
+}
+for (ClusterEvent.EventType type : eventTypes) {
+ getEventListeners()
+ .getOrDefault(type, Collections.emptySet())
+ .remove(listener);
+}
+ }
+
+ /**
+ * Fire an event. This method will call registered listeners that subscribed
to the
+ * type of event being passed.
+ * @param event cluster event
+ */
+ default void fireEvent(ClusterEvent event) {
Review comment:
Firing (publishing) events is definitely needed in solr core where
events are created.
Do we need to couple the publishing of events and managing the consumption
of events as done here, or maybe have a simple publishing interface (with this
single method) and a separate one for implementing the even distribution logic?
This could eventually allow keeping in Solr core the things that really
belong there (creating events related to solr internal state change and
publishing them) while allowing to move the event bus logic outside of solr
core.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r498954021
##
File path: solr/core/src/java/org/apache/solr/cluster/events/ClusterEvent.java
##
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.cluster.events;
+
+import org.apache.solr.common.MapWriter;
+
+import java.io.IOException;
+import java.time.Instant;
+
+/**
+ * Cluster-level event.
+ */
+public interface ClusterEvent extends MapWriter {
Review comment:
The `MapWriter` related machinery (including the `writeMap` method)
doesn't belong in the interfaces IMO. It should be hidden in the implementation
if it's needed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r496005660
##
File path:
solr/core/src/java/org/apache/solr/cluster/events/impl/CollectionsRepairEventListener.java
##
@@ -0,0 +1,186 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.cluster.events.impl;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import org.apache.solr.client.solrj.SolrClient;
+import org.apache.solr.client.solrj.cloud.SolrCloudManager;
+import org.apache.solr.client.solrj.request.CollectionAdminRequest;
+import org.apache.solr.cloud.api.collections.Assign;
+import org.apache.solr.cluster.events.ClusterEvent;
+import org.apache.solr.cluster.events.ClusterEventListener;
+import org.apache.solr.cluster.events.NodesDownEvent;
+import org.apache.solr.cluster.events.ReplicasDownEvent;
+import org.apache.solr.common.cloud.ClusterState;
+import org.apache.solr.common.cloud.Replica;
+import org.apache.solr.common.cloud.ReplicaPosition;
+import org.apache.solr.core.CoreContainer;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This is an illustration how to re-implement the combination of 8x
+ * NodeLostTrigger and AutoAddReplicasPlanAction to maintain the collection's
replication factor.
+ * NOTE: there's no support for 'waitFor' yet.
+ * NOTE 2: this functionality would be probably more reliable when executed
also as a
+ * periodically scheduled check - both as a reactive (listener) and proactive
(scheduled) measure.
+ */
+public class CollectionsRepairEventListener implements ClusterEventListener {
+ private static final Logger log =
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+
+ public static final String PLUGIN_NAME = "collectionsRepairListener";
+ private static final String ASYNC_ID_PREFIX = "_async_" + PLUGIN_NAME;
+ private static final AtomicInteger counter = new AtomicInteger();
+
+ private final SolrClient solrClient;
+ private final SolrCloudManager solrCloudManager;
+
+ private boolean running = false;
Review comment:
Likely needs to be volatile (or get proper synchronization).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r491887454
##
File path:
solr/core/src/java/org/apache/solr/cluster/events/ClusterEventProducer.java
##
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.cluster.events;
+
+import org.apache.solr.cloud.ClusterSingleton;
+
+import java.util.Collections;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+
+/**
+ * Component that produces {@link ClusterEvent} instances.
+ */
+public interface ClusterEventProducer extends ClusterSingleton {
+
+ String PLUGIN_NAME = "clusterEventProducer";
+
+ /**
+ * Returns a modifiable map of event types and listeners to process events
+ * of a given type.
+ */
+ Map> getEventListeners();
Review comment:
These implementations will be internal (the `ClusterEvent` values are a
finite defined set so any new events imply changes to internal implementation),
so we likely can accept to use `EnumMap` in these.
Potential efficiency gain vs. flexibility?
I would have used `EnumMap` but ok to keep `Map`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r491887454
##
File path:
solr/core/src/java/org/apache/solr/cluster/events/ClusterEventProducer.java
##
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.cluster.events;
+
+import org.apache.solr.cloud.ClusterSingleton;
+
+import java.util.Collections;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+
+/**
+ * Component that produces {@link ClusterEvent} instances.
+ */
+public interface ClusterEventProducer extends ClusterSingleton {
+
+ String PLUGIN_NAME = "clusterEventProducer";
+
+ /**
+ * Returns a modifiable map of event types and listeners to process events
+ * of a given type.
+ */
+ Map> getEventListeners();
Review comment:
These implementations will be internal (the `ClusterEvent` values are a
finite defined set so any new events imply changes to internal implementation),
so we likely can accept to use `EnumMap` in these.
Potential efficiency gain vs. flexibility?
I would have used `EnumMap` but ok to keep `Map`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r490294923
##
File path:
solr/core/src/java/org/apache/solr/cluster/events/ClusterPropertiesChangedEvent.java
##
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.cluster.events;
+
+import java.util.Map;
+
+/**
+ * Event generated when {@link
org.apache.solr.common.cloud.ZkStateReader#CLUSTER_PROPS} is modified.
+ */
+public interface ClusterPropertiesChangedEvent extends ClusterEvent {
+
+ @Override
+ default EventType getType() {
+return EventType.CLUSTER_PROPERTIES_CHANGED;
+ }
+
+ Map getOldClusterProperties();
Review comment:
Is it really the role of the event to provide the old vs new properties?
I'd prefer the event to provide the change only, let the subscriber fetch the
new state if it wants to assemble the before/after state (and possibly the
listener is keeping its own copy of what the old state is, and that old state
is not necessarily equal to the old state as known by the change event
implementation).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r490288384
##
File path:
solr/core/src/java/org/apache/solr/cluster/events/ClusterEventProducer.java
##
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.cluster.events;
+
+import org.apache.solr.cloud.ClusterSingleton;
+
+import java.util.Collections;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+
+/**
+ * Component that produces {@link ClusterEvent} instances.
+ */
+public interface ClusterEventProducer extends ClusterSingleton {
+
+ String PLUGIN_NAME = "clusterEventProducer";
+
+ /**
+ * Returns a modifiable map of event types and listeners to process events
+ * of a given type.
+ */
+ Map> getEventListeners();
Review comment:
What about using an `EnumMap` here instead?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r490285588
##
File path:
solr/core/src/java/org/apache/solr/cluster/events/ClusterEventListener.java
##
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.cluster.events;
+
+import java.util.Set;
+
+/**
+ * Components that want to be notified of cluster-wide events should use this.
+ *
+ * XXX should this work only for ClusterSingleton-s? some types of events may
be
+ * XXX difficult (or pointless) to propagate to every node.
+ */
+public interface ClusterEventListener {
+
+ /**
+ * The types of events that this listener can process.
+ */
+ Set getEventTypes();
Review comment:
Similarly, it might be useful to unregister a listener on some event
types but not all...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r490284824
##
File path:
solr/core/src/java/org/apache/solr/cluster/events/ClusterEventListener.java
##
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.cluster.events;
+
+import java.util.Set;
+
+/**
+ * Components that want to be notified of cluster-wide events should use this.
+ *
+ * XXX should this work only for ClusterSingleton-s? some types of events may
be
+ * XXX difficult (or pointless) to propagate to every node.
+ */
+public interface ClusterEventListener {
+
+ /**
+ * The types of events that this listener can process.
+ */
+ Set getEventTypes();
Review comment:
Why have event types listened to as a listener attribute rather than as
a parameter passed upon registration?
Even types part of the listener make it harder to reuse listeners. For
example a listener that logs the event and that can be registered wherever
needed (this model would force subclassing it each time).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r490279251
##
File path:
solr/core/src/java/org/apache/solr/cluster/events/CollectionsAddedEvent.java
##
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.cluster.events;
+
+import java.util.Collection;
+
+/**
+ * Event generated when some collections have been added.
+ */
+public interface CollectionsAddedEvent extends ClusterEvent {
+
+ @Override
+ default EventType getType() {
+return EventType.COLLECTIONS_ADDED;
+ }
+
+ Collection getCollectionNames();
Review comment:
Shall we move this to an iterator model?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r473959273
##
File path:
solr/core/src/java/org/apache/solr/cluster/events/ClusterSingleton.java
##
@@ -0,0 +1,14 @@
+package org.apache.solr.cluster.events;
+
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+
+/**
+ * Intended for {@link org.apache.solr.core.CoreContainer} plugins that should
be
+ * enabled only one instance per cluster.
+ * Implementation detail: currently these plugins are instantiated on the
+ * Overseer leader, and closed when the current node loses its leadership.
+ */
+@Retention(RetentionPolicy.RUNTIME)
+public @interface ClusterSingleton {
Review comment:
Another general (semi lame) comment from me: "good practices" are good
guidelines but we also have to be pragmatic/realistic. To some extent it means
understanding the big picture of how a (new) feature needs to be implemented
and then seeing how that plays with the principles of separation etc we all
want. Sometimes it doesn't play well (due to maturity of the rest of the code
or other reasons), and possibly during design phases we didn't anticipate all
the touchpoints.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r473953526
##
File path: solr/core/src/java/org/apache/solr/cluster/events/ScheduledEvent.java
##
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.cluster.events;
+
+/**
+ *
+ */
+public interface ScheduledEvent extends ClusterEvent {
Review comment:
First step is separating interfaces and implementation. If `solr-core`
is to interact with non core modules, it needs a way to talk to them. After
interfaces comes the question of instantiation or dependency injection into
`solr-core`.
Even if package selection/install is not automated, having the right code
for a clean separation will make evolution much easier.
To some extent, if `solr-core` doesn't need to talk (directly or indirectly)
to another module, then maybe the module doesn't belong at all in Solr?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r473052101
##
File path:
solr/core/src/java/org/apache/solr/cluster/events/ClusterSingleton.java
##
@@ -0,0 +1,14 @@
+package org.apache.solr.cluster.events;
+
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+
+/**
+ * Intended for {@link org.apache.solr.core.CoreContainer} plugins that should
be
+ * enabled only one instance per cluster.
+ * Implementation detail: currently these plugins are instantiated on the
+ * Overseer leader, and closed when the current node loses its leadership.
+ */
+@Retention(RetentionPolicy.RUNTIME)
+public @interface ClusterSingleton {
Review comment:
Ok, I agree about keeping module dependencies clean. Didn't realize
"Solr-core" meant just a module in the project.
We do need some code in `solr-core` though, at least interfaces to be able
to interact with other modules without depending on them (+ dependency
injection or configuration to instantiate actual concrete classes).
I believe that understanding what's really needed happens when the
interfaces and the configuration is wired into Solr in some place or another,
then we should move out of solr-core everything left there without reason
(thinking of [PR 1684](https://github.com/apache/lucene-solr/pull/1684/files)
as I write this comment, likely the `impl `and `plugin `packages there can be
moved out of `solr-core`).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r473052101
##
File path:
solr/core/src/java/org/apache/solr/cluster/events/ClusterSingleton.java
##
@@ -0,0 +1,14 @@
+package org.apache.solr.cluster.events;
+
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+
+/**
+ * Intended for {@link org.apache.solr.core.CoreContainer} plugins that should
be
+ * enabled only one instance per cluster.
+ * Implementation detail: currently these plugins are instantiated on the
+ * Overseer leader, and closed when the current node loses its leadership.
+ */
+@Retention(RetentionPolicy.RUNTIME)
+public @interface ClusterSingleton {
Review comment:
Ok, I agree about keeping module dependencies clean. Didn't realize
"Solr-core" meant just a module in the project.
We do need some code in `solr-core` though, at least interfaces to be able
to interact with other modules without depending on them (+ dependency
injection or configuration to instantiate actual concrete classes).
I believe that understanding what's really needed happens when the
interfaces and the configuration is wired into Solr in some place or another,
then we should move out of solr-core everything left there without reason
(thinking of [PR 1684](https://github.com/apache/lucene-solr/pull/1684/files)
as I write this comment, likely the impl and plugin packages there can be moved
out of `solr-core`).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r473052101
##
File path:
solr/core/src/java/org/apache/solr/cluster/events/ClusterSingleton.java
##
@@ -0,0 +1,14 @@
+package org.apache.solr.cluster.events;
+
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+
+/**
+ * Intended for {@link org.apache.solr.core.CoreContainer} plugins that should
be
+ * enabled only one instance per cluster.
+ * Implementation detail: currently these plugins are instantiated on the
+ * Overseer leader, and closed when the current node loses its leadership.
+ */
+@Retention(RetentionPolicy.RUNTIME)
+public @interface ClusterSingleton {
Review comment:
Ok, I agree about keeping module dependencies clean. Didn't realize
"Solr-core" meant just a module in the project.
We do need some code in `solr-core` though, at least interfaces to be able
to interact with other modules without depending on them (+ dependency
injection or configuration to instantiate actual concrete classes).
I believe that understanding what's really needed happens when the
interfaces and the configuration is wired into Solr in some place or another,
then we should move out of solr-core everything left there without reason
(thinking of [PR 1684](https://github.com/apache/lucene-solr/pull/1684/files)
as I write this comment, likely the impl and plugin packages there can be moved
out of solr-core.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r472946169
##
File path:
solr/core/src/java/org/apache/solr/cluster/events/ClusterSingleton.java
##
@@ -0,0 +1,14 @@
+package org.apache.solr.cluster.events;
+
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+
+/**
+ * Intended for {@link org.apache.solr.core.CoreContainer} plugins that should
be
+ * enabled only one instance per cluster.
+ * Implementation detail: currently these plugins are instantiated on the
+ * Overseer leader, and closed when the current node loses its leadership.
+ */
+@Retention(RetentionPolicy.RUNTIME)
+public @interface ClusterSingleton {
Review comment:
My fear is everything that depends on the internal implementation of
Solr and that's not compiled with Solr from the lucene-solr git repo makes our
life harder (refactor breaks things).
I'd rather have such code be in lucene-solr and not be loaded when not in
use, even though its presence in the running JVM if it's not called is really
not an issue. We have loads of loaded but unused code in a running instance
(think 3rd party libraries).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r472343024
##
File path:
solr/core/src/java/org/apache/solr/cluster/events/ClusterSingleton.java
##
@@ -0,0 +1,14 @@
+package org.apache.solr.cluster.events;
+
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+
+/**
+ * Intended for {@link org.apache.solr.core.CoreContainer} plugins that should
be
+ * enabled only one instance per cluster.
+ * Implementation detail: currently these plugins are instantiated on the
+ * Overseer leader, and closed when the current node loses its leadership.
+ */
+@Retention(RetentionPolicy.RUNTIME)
+public @interface ClusterSingleton {
Review comment:
If you don't provide this level of service to plugins (single instance
running) you force them to somehow do it on their own. How would they even
start to do that? I'm strongly against exposing internal implementation
"details" such as ZK. Do we let plugins open TCP ports? Have their own ZK?
Tomorrow we might decide to run plug-in on other containers/VM's that are not
nodes or replace ZK by a DB. Will plugins have to reimplement another leader
election or similar?
It is much cleaner and simpler to do it once in Solr.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r472346084
##
File path: solr/core/src/java/org/apache/solr/cluster/events/ScheduledEvent.java
##
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.cluster.events;
+
+/**
+ *
+ */
+public interface ScheduledEvent extends ClusterEvent {
Review comment:
Leader rebalancing job, cluster rebalancing, removing replicas from a
node to be removed, unnecessary replica removal, core corruption checking etc
etc
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r472343024
##
File path:
solr/core/src/java/org/apache/solr/cluster/events/ClusterSingleton.java
##
@@ -0,0 +1,14 @@
+package org.apache.solr.cluster.events;
+
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+
+/**
+ * Intended for {@link org.apache.solr.core.CoreContainer} plugins that should
be
+ * enabled only one instance per cluster.
+ * Implementation detail: currently these plugins are instantiated on the
+ * Overseer leader, and closed when the current node loses its leadership.
+ */
+@Retention(RetentionPolicy.RUNTIME)
+public @interface ClusterSingleton {
Review comment:
If you don't provide thus level of service to plugins (single instance
running) you force them to somehow do it on their own. How would they even
start to do that? I'm strongly against exposing internal implementation
"details" such as ZK. Do we let plugins open TCP ports? Have their own ZK?
Tomorrow we might decide to run plug-in on other containers/VM's that are not
nodes. Will plugins have to reimplement another leader election or similar?
It is much cleaner and simpler to do it once in Solr.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r472343024
##
File path:
solr/core/src/java/org/apache/solr/cluster/events/ClusterSingleton.java
##
@@ -0,0 +1,14 @@
+package org.apache.solr.cluster.events;
+
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+
+/**
+ * Intended for {@link org.apache.solr.core.CoreContainer} plugins that should
be
+ * enabled only one instance per cluster.
+ * Implementation detail: currently these plugins are instantiated on the
+ * Overseer leader, and closed when the current node loses its leadership.
+ */
+@Retention(RetentionPolicy.RUNTIME)
+public @interface ClusterSingleton {
Review comment:
If you don't provude thus level of servuce to plugins (single instance
running) you force them to somehow do it on their own. How would they do that?
I'm strongly against exposing internal implementation "details" such as ZK. Do
we let plugins open
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758:
URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r472239057
##
File path: solr/core/src/java/org/apache/solr/cloud/events/ScheduledEvent.java
##
@@ -0,0 +1,25 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.cloud.events;
+
+/**
+ *
+ */
+public interface ScheduledEvent extends ClusterEvent {
+ String getScheduleName();
+ Object getScheduleParam(String key);
Review comment:
I'm all for these events to be just like the other ones. Just define
what a schedule is and strong type whatever can be...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
