dtspence opened a new issue, #5960:
URL: https://github.com/apache/accumulo/issues/5960
**Describe the bug**
Using the `ErasureCodeConfigurer` with a default `table.file.ec=inherit`
causes an error when attempting to compact files greater than the size
threshold. The compaction error states the `table.ec.policy` has not been set.
**Versions (OS, Maven, Java, and others, as appropriate):**
- Accumulo 2.1.4
**To Reproduce**
1. Create a new table `test`.
1. Import r-files into `test` that are greater than 123M.
1. Set property `config -t test -s
table.compaction.configurer=org.apache.accumulo.core.client.admin.compaction.ErasureCodeConfigurer`
1. Set property `config -t test -s
table.compaction.configurer.opts.erasure.code.size.conversion=123M`
1. Run `compact -t test`.
1. Check the Accumulo monitor for compaction errors. The error states `Blank
or empty value set for table.file.ec.policy`.
**Expected behavior**
A table with a `table.file.ec=inherit` will use the HDFS volume
configuration when the `ErasureCodeConfigurer` evaluates an estimated size that
is greater than the `erasure.code.size.conversion` threshold.
**Screenshots**
N/A
**Additional context**
The error occurs due to code in the following method when `byPassEC` is
false (i.e. default value):
```java
public Overrides override(InputParameters params) {
Map<String,String> overs = new
HashMap<>(super.override(params).getOverrides());
if (this.byPassEC) {
// Allow for user initiated compactions to pass an options to bypass
EC.
overs.put(Property.TABLE_ENABLE_ERASURE_CODES.getKey(), "disable");
} else {
long inputsSum =
params.getInputFiles().stream().mapToLong(CompactableFile::getEstimatedSize).sum();
if (inputsSum >= this.ecSize) {
overs.put(Property.TABLE_ENABLE_ERASURE_CODES.getKey(), "enable");
if (ecPolicyName != null) {
overs.put(Property.TABLE_ERASURE_CODE_POLICY.getKey(),
ecPolicyName);
}
} else {
overs.put(Property.TABLE_ENABLE_ERASURE_CODES.getKey(), "disable");
}
}
return new Overrides(overs);
}
```
The `byPassEC` else block sets an override that the `table.file.ec=enable`,
which potentially overrides the `inherit` value. In the `inherit` case, the
table may not have an `table.file.ec.policy` defined.
An error is raised later on when the RFileWriter sees the `table.file.ec` is
enabled but a check fails because the table does not have a
`table.file.ec.policy` defined (i.e. the inherit case).
```java
switch (ecEnable) {
case ENABLE:
String ecPolicyName =
options.getTableConfiguration().get(Property.TABLE_ERASURE_CODE_POLICY);
// The default value of this property is empty string. If empty
string is given to this
// builder it will disable erasure coding. So adding an explicit
check for that.
Preconditions.checkArgument(!ecPolicyName.isBlank(), "Blank or
empty value set for %s",
Property.TABLE_ERASURE_CODE_POLICY.getKey());
builder = builder.ecPolicyName(ecPolicyName);
break;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
