[
https://issues.apache.org/jira/browse/SOLR-4793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445754#comment-16445754
]
Steve Rowe commented on SOLR-4793:
----------------------------------
bq. I think the long term solution could be to implement something like a
BlobStoreResourceLoader
Agreed - see SOLR-8751 and SOLR-9175.
bq. and a configset (as a whole or in parts) could be loaded from ZK or blob
store.
I'm not sure how useful it would be to store whole configsets in the blob
store. In any case, that won't be the first step here.
bq. Can we keep all but large files in zk. When zkClient is asked to upload a
large file it will upload it to blob instead and create a shadow file with same
name in Zk, but with a body telling it is a blob file with a pointer to blob
store ID. Then when zk resource loader gets a file it will detect such files
and serve them from blob transparently.
Hmm, I don't think we should be *starting* with this kind of magic - I'm much
more comfortable with separate blob store upload (already implemented) and
schema reference steps (SOLR-8751/SOLR-9175)
bq. This probably means that backup/restore feature also needs to backup the
blob store?
Yes, but this is already true right now: {{solrconfig.xml}} can load handler
and component classes from blobs in the blob store.
> Solr Cloud can't upload large config files ( > 1MB) to Zookeeper
> -----------------------------------------------------------------
>
> Key: SOLR-4793
> URL: https://issues.apache.org/jira/browse/SOLR-4793
> Project: Solr
> Issue Type: Improvement
> Reporter: Son Nguyen
> Priority: Major
> Attachments: SOLR-4793.patch
>
>
> Zookeeper set znode size limit to 1MB by default. So we can't start Solr
> Cloud with some large config files, like synonyms.txt.
> Jan Høydahl has a good idea:
> "SolrCloud is designed with an assumption that you should be able to upload
> your whole disk-based conf folder into ZK, and that you should be able to add
> an empty Solr node to a cluster and it would download all config from ZK. So
> immediately a splitting strategy automatically handled by ZkSolresourceLoader
> for large files could be one way forward, i.e. store synonyms.txt as e.g.
> __001_synonyms.txt __002_synonyms.txt...."
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
