[
https://issues.apache.org/jira/browse/SOLR-14251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585134#comment-17585134
]
David Smiley commented on SOLR-14251:
-------------------------------------
FYI the linked SOLR-16353 adds a boolean system property to turn it off.
> Shard Split on HDFS
> --------------------
>
> Key: SOLR-14251
> URL: https://issues.apache.org/jira/browse/SOLR-14251
> Project: Solr
> Issue Type: Bug
> Components: hdfs
> Affects Versions: 8.4
> Reporter: Johannes Brucher
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Shard Split on HDFS Index will evaluate local disc space instead of HDFS space
> When performing a shard split on an index that is stored on HDFS the
> SplitShardCmd however evaluates the free disc space on the local file system
> of the server where Solr is installed.
> SplitShardCmd assumes that its main phase (when the Lucene index is being
> split) always executes on the local file system of the shard leader; and
> indeed the ShardSplitCmd.checkDiskSpace() checks the local file system's free
> disk space - even though the actual data is written to the HDFS Directory so
> it (almost) doesn't affect the local FS (except for core.properties file).
> See also:
> [https://lucene.472066.n3.nabble.com/HDFS-Shard-Split-td4449920.html]
> My setup to reproduce the issue:
> * Solr deployed on Openshift with local disc of about 5GB
> * HDFS configuration based on solrconfig.xml with
> {code:java}
> <directoryFactory name="DirectoryFactory" class="solr.HdfsDirectoryFactory">
> <str name="solr.hdfs.home">hdfs://path/to/index/</str>
> ...
> {code}
> * Split command:
> {code:java}
> .../admin/collections?action=SPLITSHARD&collection=collection1&shard=shard1&async=1234{code}
> * Response:
> {code:java}
> {
> "responseHeader":{"status":0,"QTime":32},
> "Operation splitshard caused
> exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> not enough free disk space to perform index split on node <solr
> instance>:8983_solr, required: 294.64909074269235, available:
> 5.4632568359375",
> "exception":{
> "msg":"not enough free disk space to perform index split on node <solr
> instance>:8983_solr, required: 294.64909074269235, available:
> 5.4632568359375",
> "rspCode":500},
> "status":{"state":"failed","msg":"found [1234] in failed tasks"}
> }
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
