ctubbsii opened a new issue, #4273:
URL: https://github.com/apache/accumulo/issues/4273
Had a discussion with @keith-turner and @ddanielr about some of the
compaction configuration and SPI design, and we came up with some of these
ideas that could be implemented:
One major unknown is what to call it. We went back and forth a bit on
whether it should be `CompactionService*` or `CompactionPlanner*`
1. Remove user-facing SPI for CompactionPlanner and related config
2. Replace with CompactionServiceFactory:
a. `compaction.service.factory` holds the class name (singleton ref in
ServerContext?)
b. `compaction.service.factory.config` holds a single string of all the
factory's config
3. Suggested API (names can change later, this is just an idea)
```java
interface CompactionServiceFactory {
void init(PluginEnvironment env);
CompactionService forName(String service);
}
// configuration for a bounded named group / queue
class GroupConfig {
String name;
int maxQueueSize;
}
interface CompactionService {
Set<GroupConfig> getGroups();
List<Job> plan(tabletPlanningInformation);
}
```
4. `init` can validate the config, or re-validate periodically
(implementation dependent)
Our default configuration for the `compaction.service.factory.config` could
be (not pretty-printed, but can be copied/pasted into the property description
in an HTML-friendly pretty-printed way):
```json
[
{
"meta": {
"maxOpenFilesPerJob": "30",
"groups": [
{
"name": "accumulo_meta_small",
"maxSize": "128M",
"maxJobs": "1000"
},
{
"name": "accumulo_meta_large",
"maxJobs": "1000"
}
]
},
"default": {
"maxOpenFilesPerJob": "30",
"groups": [
{
"name": "user_small",
"maxSize": "128M",
"maxJobs": "1000"
},
{
"name": "user_large",
"maxJobs": "1000"
}
]
}
}
]
```
One of the main benefits of this is having simplified configuration for
user's configuration file. Also, we'll be able to set a default value in the
DefaultConfiguration that actually lives in one place, and is easy for users to
view for reference that represents the actual default behavior and override if
they want to.
Re: #3981, #4034 , #4061
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
