[
https://issues.apache.org/jira/browse/SOLR-7256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365136#comment-14365136
]
Hari Sekhon commented on SOLR-7256:
-----------------------------------
RAID is fine if you're doing nothing but a purpose built SolrCloud... but one
of the best use cases right now is SolrCloud co-located with Hadoop where there
is a JBOD of multiple disks that you can't utilize the storage from and manage
well without this feature.
Perhaps a workaround would be to add better tooling for multiple shard replicas
per node, one per disk? However this goes back to the different sizes problem
as shards can end up being not that well balanced.
With regards to locking across disks, the two options are 1) Solr locks a file
(can be any location/disk) and then controls the disk writes across all the
disks, or 2) Solr acquires a lock per dataDir as Hadoop does.
> Multiple data dirs
> ------------------
>
> Key: SOLR-7256
> URL: https://issues.apache.org/jira/browse/SOLR-7256
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 4.10.3
> Environment: HDP 2.2 / HDP Search
> Reporter: Hari Sekhon
>
> Request to support multiple dataDirs as indexing a large collection fills up
> only one of many disks in modern servers (think colocating on Hadoop servers
> with many disks).
> While HDFS is another alternative, it results in poor performance and index
> corruption under high online indexing loads (SOLR-7255).
> While it should be possible to do multiple cores with different dataDirs,
> that could be very difficult to manage and not humanly scale well, so I think
> Solr should support use of multiple dataDirs natively.
> Regards,
> Hari Sekhon
> http://www.linkedin.com/in/harisekhon
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
