dsmiley commented on code in PR #2834:
URL: https://github.com/apache/solr/pull/2834#discussion_r1826292404
##########
solr/solrj-zookeeper/src/java/org/apache/solr/client/solrj/impl/SolrClientNodeStateProvider.java:
##########
@@ -80,23 +79,18 @@ protected void readReplicaDetails() throws IOException {
if (clusterState == null) { // zkStateReader still initializing
return;
}
- Map<String, ClusterState.CollectionRef> all =
- clusterStateProvider.getClusterState().getCollectionStates();
- all.forEach(
- (collName, ref) -> {
- DocCollection coll = ref.get();
- if (coll == null) return;
- coll.forEachReplica(
- (shard, replica) -> {
- Map<String, Map<String, List<Replica>>> nodeData =
- nodeVsCollectionVsShardVsReplicaInfo.computeIfAbsent(
- replica.getNodeName(), k -> new HashMap<>());
- Map<String, List<Replica>> collData =
- nodeData.computeIfAbsent(collName, k -> new HashMap<>());
- List<Replica> replicas = collData.computeIfAbsent(shard, k ->
new ArrayList<>());
- replicas.add((Replica) replica.clone());
- });
- });
+ clusterState.forEachCollection(
Review Comment:
simple; nothing special here.
But an aside; it's disappointing to see that NodeStateProvider and thus
AttributeFetcher.fetchAttributes() (it's sole user) and thus some of the
default placement plugins (who call it very indirectly) are burdened by this
scalability issue.
##########
solr/solrj-zookeeper/src/java/org/apache/solr/common/cloud/ZkStateReader.java:
##########
@@ -1432,8 +1432,7 @@ private DocCollection fetchCollectionState(String coll,
Watcher watcher)
zkClient,
Instant.ofEpochMilli(stat.getCtime()));
- ClusterState.CollectionRef collectionRef =
state.getCollectionStates().get(coll);
- return collectionRef == null ? null : collectionRef.get();
+ return state.getCollectionOrNull(coll);
Review Comment:
LOL
##########
solr/core/src/java/org/apache/solr/handler/designer/SchemaDesignerConfigSetHelper.java:
##########
@@ -168,24 +167,12 @@ Map<String, Object> analyzeField(String configSet, String
fieldName, String fiel
}
List<String> listCollectionsForConfig(String configSet) {
- final List<String> collections = new ArrayList<>();
- Map<String, ClusterState.CollectionRef> states =
- zkStateReader().getClusterState().getCollectionStates();
- for (Map.Entry<String, ClusterState.CollectionRef> e : states.entrySet()) {
- final String coll = e.getKey();
- if (coll.startsWith(DESIGNER_PREFIX)) {
- continue; // ignore temp
- }
-
- try {
- if (configSet.equals(e.getValue().get().getConfigName()) &&
e.getValue().get() != null) {
- collections.add(coll);
- }
- } catch (Exception exc) {
- log.warn("Failed to get config name for {}", coll, exc);
- }
Review Comment:
This is weird; it's impossible. Just in case getNonfigName is null, I did
the configSet check in a null-safe way below by putting it on the left side of
calling equals()
##########
solr/solrj-zookeeper/src/java/org/apache/solr/common/cloud/ZkStateReader.java:
##########
@@ -634,7 +634,7 @@ private void constructState(Set<String> changedCollections)
{
collectionWatches.watchedCollections(),
collectionWatches.activeCollections(),
lazyCollectionStates.keySet(),
- clusterState.getCollectionStates());
+ clusterState.collectionStream().toList());
Review Comment:
nice little JDK 17 convenience
##########
solr/core/src/java/org/apache/solr/handler/designer/SchemaDesignerConfigSetHelper.java:
##########
@@ -168,24 +167,12 @@ Map<String, Object> analyzeField(String configSet, String
fieldName, String fiel
}
List<String> listCollectionsForConfig(String configSet) {
- final List<String> collections = new ArrayList<>();
- Map<String, ClusterState.CollectionRef> states =
- zkStateReader().getClusterState().getCollectionStates();
- for (Map.Entry<String, ClusterState.CollectionRef> e : states.entrySet()) {
- final String coll = e.getKey();
- if (coll.startsWith(DESIGNER_PREFIX)) {
- continue; // ignore temp
- }
Review Comment:
This is bizarre -- why would a collection name start with `._designer_`?
That's a prefix for configsets in connection with this schema designer
functionality, not collection names (AFAICT). I did a find-usages on
DESIGNER_PREFIX to see. CC @thelabdude
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
