[
https://issues.apache.org/jira/browse/SOLR-14251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Johannes Brucher updated SOLR-14251:
------------------------------------
Description:
Shard Split on HDFS Index will evaluate local disc space instead of HDFS space
When performing a shard split on an index that is stored on HDFS the
SplitShardCmd however evaluates the free disc space on the local file system of
the server where Solr is installed.
SplitShardCmd assumes that its main phase (when the Lucene index is being
split) always executes on the local file system of the shard leader; and indeed
the ShardSplitCmd.checkDiskSpace() checks the local file system's free disk
space - even though the actual data is written to the HDFS Directory so it
(almost) doesn't affect the local FS (except for core.properties file).
See also: [https://lucene.472066.n3.nabble.com/HDFS-Shard-Split-td4449920.html]
My setup to reproduce the issue:
* Solr deployed on Openshift with local disc of about 5GB
* HDFS configuration based on solrconfig.xml with
{code:java}
<directoryFactory name="DirectoryFactory" class="solr.HdfsDirectoryFactory">
<str name="solr.hdfs.home">hdfs://path/to/index/</str>
...
{code}
* Split command:
{code:java}
.../admin/collections?action=SPLITSHARD&collection=collection1&shard=shard1&async=1234{code}
* Response:
{code:java}
{
"responseHeader":{"status":0,"QTime":32},
"Operation splitshard caused
exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
not enough free disk space to perform index split on node <solr
instance>:8983_solr, required: 294.64909074269235, available: 5.4632568359375",
"exception":{
"msg":"not enough free disk space to perform index split on node <solr
instance>:8983_solr, required: 294.64909074269235, available: 5.4632568359375",
"rspCode":500},
"status":{"state":"failed","msg":"found [1234] in failed tasks"}
}
{code}
was:
Shard Split on HDFS Index will evaluate local disc space instead of HDFS space
When performing a shard split on an index that is stored on HDFS the
SplitShardCmd however evaluates the free disc space on the local file system of
the server where Solr is installed.
SplitShardCmd assumes that its main phase (when the Lucene index is being
split) always executes on the local file system of the shard leader; and indeed
the ShardSplitCmd.checkDiskSpace() checks the local file system's free disk
space - even though the actual data is written to the HDFS Directory so it
(almost) doesn't affect the local FS (except for core.properties file).
See also: [https://lucene.472066.n3.nabble.com/HDFS-Shard-Split-td4449920.html]
My setup to reproduce the issue:
* Solr deployed on Openshift with local disc of about 5GB
* HDFS configuration based on solrconfig.xml with
{code:java}
<directoryFactory name="DirectoryFactory" class="solr.HdfsDirectoryFactory">
<str name="solr.hdfs.home">hdfs://path/to/index/str>
...
{code}
* Split command:
{code:java}
.../admin/collections?action=SPLITSHARD&collection=collection1&shard=shard1&async=1234{code}
* Response:
{code:java}
{
"responseHeader":{"status":0,"QTime":32},
"Operation splitshard caused
exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
not enough free disk space to perform index split on node <solr
instance>:8983_solr, required: 294.64909074269235, available: 5.4632568359375",
"exception":{
"msg":"not enough free disk space to perform index split on node <solr
instance>:8983_solr, required: 294.64909074269235, available: 5.4632568359375",
"rspCode":500},
"status":{"state":"failed","msg":"found [1234] in failed tasks"}
}
{code}
> Shard Split on HDFS
> --------------------
>
> Key: SOLR-14251
> URL: https://issues.apache.org/jira/browse/SOLR-14251
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: hdfs
> Affects Versions: 8.4
> Reporter: Johannes Brucher
> Priority: Major
>
> Shard Split on HDFS Index will evaluate local disc space instead of HDFS space
> When performing a shard split on an index that is stored on HDFS the
> SplitShardCmd however evaluates the free disc space on the local file system
> of the server where Solr is installed.
> SplitShardCmd assumes that its main phase (when the Lucene index is being
> split) always executes on the local file system of the shard leader; and
> indeed the ShardSplitCmd.checkDiskSpace() checks the local file system's free
> disk space - even though the actual data is written to the HDFS Directory so
> it (almost) doesn't affect the local FS (except for core.properties file).
> See also:
> [https://lucene.472066.n3.nabble.com/HDFS-Shard-Split-td4449920.html]
> My setup to reproduce the issue:
> * Solr deployed on Openshift with local disc of about 5GB
> * HDFS configuration based on solrconfig.xml with
> {code:java}
> <directoryFactory name="DirectoryFactory" class="solr.HdfsDirectoryFactory">
> <str name="solr.hdfs.home">hdfs://path/to/index/</str>
> ...
> {code}
> * Split command:
> {code:java}
> .../admin/collections?action=SPLITSHARD&collection=collection1&shard=shard1&async=1234{code}
> * Response:
> {code:java}
> {
> "responseHeader":{"status":0,"QTime":32},
> "Operation splitshard caused
> exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> not enough free disk space to perform index split on node <solr
> instance>:8983_solr, required: 294.64909074269235, available:
> 5.4632568359375",
> "exception":{
> "msg":"not enough free disk space to perform index split on node <solr
> instance>:8983_solr, required: 294.64909074269235, available:
> 5.4632568359375",
> "rspCode":500},
> "status":{"state":"failed","msg":"found [1234] in failed tasks"}
> }
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
