Noble Paul created SOLR-12993: --------------------------------- Summary: Split the state.json into 2. a small frequently modified data + a large unmodified data Key: SOLR-12993 URL: https://issues.apache.org/jira/browse/SOLR-12993 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Environment: This a just a proposal particularly useful for large cluster.
Reporter: Noble Paul Every time a small atet change occurs for a collection replica the entire following file needs to be updated + read * n times (where n = no of replicas for this collection ). The proposal is to split the main file into 2. {code:json} {"gettingstarted":{ "pullReplicas":"0", "replicationFactor":"2", "router":{"name":"compositeId"}, "maxShardsPerNode":"-1", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0", "shards":{ "shard1":{ "range":"80000000-ffffffff", "replicas":{ "core_node3":{ "core":"gettingstarted_shard1_replica_n1", "base_url":"http://10.0.0.80:8983/solr", "node_name":"10.0.0.80:8983_solr", "state":"active", "type":"NRT", "force_set_state":"false", "leader":"true"}, "core_node5":{ "core":"gettingstarted_shard1_replica_n2", "base_url":"http://10.0.0.80:7574/solr", "node_name":"10.0.0.80:7574_solr", "type":"NRT", "force_set_state":"false"}}}, "shard2":{ "range":"0-7fffffff", "state":"active", "replicas":{ "core_node7":{ "core":"gettingstarted_shard2_replica_n4", "base_url":"http://10.0.0.80:7574/solr", "node_name":"10.0.0.80:7574_solr", "type":"NRT", "force_set_state":"false"}, "core_node8":{ "core":"gettingstarted_shard2_replica_n6", "base_url":"http://10.0.0.80:8983/solr", "node_name":"10.0.0.80:8983_solr", "type":"NRT", "force_set_state":"false", "leader":"true"}}}}}} {code} another file {{status.json}} which is frequently updated and small. {code:json} { "shard1": { "s": 1, "core_node3": {"s": 1}, "core_node5": {"s": 1} }, "shard2": { "s": 1, "core_node7": {"s": 1}, "core_node8": {"s": 1}} } {code} Here the size of the file is roughly one tenth of the other file. This leads to a dramatic reduction in the amount of data written/read to/from ZK. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org