[GitHub] [accumulo-website] markxiaotao opened a new pull request #233: Add new contributor
markxiaotao opened a new pull request #233: URL: https://github.com/apache/accumulo-website/pull/233 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [accumulo] ctubbsii commented on pull request #1652: Remove an workaround profile exists in core/pom.xml file since the related issue had been already fixed.
ctubbsii commented on pull request #1652: URL: https://github.com/apache/accumulo/pull/1652#issuecomment-659596703 Thanks for the PR @markxiaotao ! I merged it, and found and fixed two others of the same type, which would have remained if you hadn't brought it to our attention. Thanks! If you wish to be added as a contributor to https://accumulo.apache.org/people/ , please open a pull request to add yourself at https://github.com/apache/accumulo-website/edit/master/pages/people.md and leave a reference to `apache/accumulo#1652` in your commit log. If you intend to be a regular contributor to Accumulo projects, please consider subscribing to our developer mailing list (https://accumulo.apache.org/contact-us/) and introducing yourself. :smiley_cat: This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [accumulo] ctubbsii merged pull request #1652: Remove an workaround profile exists in core/pom.xml file since the related issue had been already fixed.
ctubbsii merged pull request #1652: URL: https://github.com/apache/accumulo/pull/1652 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Accumulo-1.9 - Build # 448 - Fixed
The Apache Jenkins build system has built Accumulo-1.9 (build #448) Status: Fixed Check console output at https://builds.apache.org/job/Accumulo-1.9/448/ to view the results.
Accumulo-Master - Build # 3125 - Fixed
The Apache Jenkins build system has built Accumulo-Master (build #3125) Status: Fixed Check console output at https://builds.apache.org/job/Accumulo-Master/3125/ to view the results.
[GitHub] [accumulo] milleruntime opened a new pull request #1660: Drop MemoryManager from properties
milleruntime opened a new pull request #1660: URL: https://github.com/apache/accumulo/pull/1660 This is work to handle one of the internal pluggable components in #1659. This removes the property and interface. The interface had internal types, was only used by one class and had a method with no implementation. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [accumulo] milleruntime commented on issue #1617: CompactionStrategy plugin interface not compatible with 2.0.0
milleruntime commented on issue #1617: URL: https://github.com/apache/accumulo/issues/1617#issuecomment-659442061 > We could just mention it in the release notes. Yeah I like that. I marked this ticket as a blocker for 2.1 so we remember to add it to the release notes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [accumulo] milleruntime commented on a change in pull request #1649: Addresses four compaction service follow on issues.
milleruntime commented on a change in pull request #1649:
URL: https://github.com/apache/accumulo/pull/1649#discussion_r455810772
##
File path: core/src/main/java/org/apache/accumulo/core/conf/Property.java
##
@@ -415,6 +415,10 @@
TSERV_COMPACTION_SERVICE_ROOT_PLANNER("tserver.compaction.major.service.root.planner",
DefaultCompactionPlanner.class.getName(), PropertyType.CLASSNAME,
"Compaction planner for root tablet service"),
+
TSERV_COMPACTION_SERVICE_ROOT_THROUGHPUT("tserver.compaction.major.service.root.throughput",
"0B",
+ PropertyType.BYTES,
+ "Maximum number of bytes to read or write per second over all major"
+ + " compactions in this compaction service, or 0B for unlimited."),
Review comment:
I think a property that is easier to spell is more user friendly. The
last part of the property could be "limit" or "rate.limit". As a bad speller,
I know a word like "throughput" would trip me up.
##
File path:
test/src/main/java/org/apache/accumulo/test/CompactionRateLimitingIT.java
##
@@ -37,50 +40,74 @@
public static final long BYTES_TO_WRITE = 10 * 1024 * 1024;
public static final long RATE = 1 * 1024 * 1024;
+ protected Property getThroughputProp() {
+return Property.TSERV_COMPACTION_SERVICE_DEFAULT_THROUGHPUT;
+ }
+
@Override
public void configure(MiniAccumuloConfigImpl cfg, Configuration fsConf) {
-cfg.setProperty(Property.TSERV_MAJC_THROUGHPUT, RATE + "B");
+cfg.setProperty(getThroughputProp(), RATE + "B");
cfg.setProperty(Property.TABLE_MAJC_RATIO, "20");
cfg.setProperty(Property.TABLE_FILE_COMPRESSION_TYPE, "none");
+
+cfg.setProperty("tserver.compaction.major.service.test.throughput", RATE +
"B");
+cfg.setProperty("tserver.compaction.major.service.test.planner",
+DefaultCompactionPlanner.class.getName());
+
cfg.setProperty("tserver.compaction.major.service.test.planner.opts.executors",
+"[{'name':'all','numThreads':2}]".replaceAll("'", "\""));
+
}
@Test
public void majorCompactionsAreRateLimited() throws Exception {
long bytesWritten = 0;
-String tableName = getUniqueNames(1)[0];
-AccumuloClient client =
-getCluster().createAccumuloClient("root", new
PasswordToken(ROOT_PASSWORD));
-client.tableOperations().create(tableName);
-try (BatchWriter bw = client.createBatchWriter(tableName)) {
- Random r = new SecureRandom();
- while (bytesWritten < BYTES_TO_WRITE) {
-byte[] rowKey = new byte[32];
-r.nextBytes(rowKey);
-
-byte[] qual = new byte[32];
-r.nextBytes(qual);
-
-byte[] value = new byte[1024];
-r.nextBytes(value);
-
-Mutation m = new Mutation(rowKey);
-m.put(new byte[0], qual, value);
-bw.addMutation(m);
-
-bytesWritten += rowKey.length + qual.length + value.length;
+String[] tableNames = getUniqueNames(1);
+
+try (AccumuloClient client =
+getCluster().createAccumuloClient("root", new
PasswordToken(ROOT_PASSWORD))) {
+
+ for (int i = 0; i < tableNames.length; i++) {
+String tableName = tableNames[i];
+
+NewTableConfiguration ntc = new NewTableConfiguration();
+if (i == 1) {
+ ntc.setProperties(Map.of("table.compaction.dispatcher.opts.service",
"test"));
+}
+
+client.tableOperations().create(tableName, ntc);
+try (BatchWriter bw = client.createBatchWriter(tableName)) {
+ Random r = new SecureRandom();
+ while (bytesWritten < BYTES_TO_WRITE) {
+byte[] rowKey = new byte[32];
+r.nextBytes(rowKey);
+
+byte[] qual = new byte[32];
+r.nextBytes(qual);
+
+byte[] value = new byte[1024];
+r.nextBytes(value);
+
+Mutation m = new Mutation(rowKey);
+m.put(new byte[0], qual, value);
+bw.addMutation(m);
+
+bytesWritten += rowKey.length + qual.length + value.length;
+ }
+}
+
+client.tableOperations().flush(tableName, null, null, true);
+
+long compactionStart = System.currentTimeMillis();
+client.tableOperations().compact(tableName, null, null, false, true);
+long duration = System.currentTimeMillis() - compactionStart;
+// The rate will be "bursty", try to account for that by taking 80% of
the expected rate
+// (allow
+// for 20% under the maximum expected duration)
Review comment:
Comment got awkward by the formatter.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [accumulo] milleruntime commented on a change in pull request #1649: Addresses four compaction service follow on issues.
milleruntime commented on a change in pull request #1649:
URL: https://github.com/apache/accumulo/pull/1649#discussion_r455788993
##
File path:
shell/src/main/java/org/apache/accumulo/shell/commands/CompactCommand.java
##
@@ -147,30 +170,28 @@ public int execute(final String fullCommand, final
CommandLine cl, final Shell s
compactionConfig.setIterators(new ArrayList<>(iterators));
}
-Map selectorOpts = new HashMap<>();
-Map configurerOpts = new HashMap<>();
-getConfigurableCompactionStrategyOpts(cl, selectorOpts, configurerOpts);
+setupConfigurableCompaction(cl, compactionConfig);
if (cl.hasOption(strategyOpt.getOpt())) {
- if (!selectorOpts.isEmpty() || !configurerOpts.isEmpty())
+ if (cl.hasOption(selectorOpt.getLongOpt()) ||
cl.hasOption(configurerOpt.getLongOpt())) {
throw new IllegalArgumentException(
-"Can not specify compaction strategy with file selection and file
output options.");
-
+"Can not specify a strategy with a selector or configurer");
+ }
configureCompactionStrat(cl);
}
-if (!selectorOpts.isEmpty()) {
- PluginConfig selectorCfg = new PluginConfig(
-
"org.apache.accumulo.tserver.compaction.strategies.ConfigurableCompactionStrategy",
- selectorOpts);
- compactionConfig.setSelector(selectorCfg);
+if (cl.hasOption(selectorOpt.getLongOpt())) {
Review comment:
If the new options can't be set along with a strategy, shouldn't these
two if statements be in an else?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [accumulo] milleruntime commented on pull request #1649: Addresses four compaction service follow on issues.
milleruntime commented on pull request #1649: URL: https://github.com/apache/accumulo/pull/1649#issuecomment-659375737 I would like @EdColeman to take a look at the metrics you added for compaction. I will look over the other commits This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (ACCUMULO-4870) 洪洞县花呗提现
[ https://issues.apache.org/jira/browse/ACCUMULO-4870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs resolved ACCUMULO-4870. - Resolution: Invalid Spam > 洪洞县花呗提现 > --- > > Key: ACCUMULO-4870 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4870 > Project: Accumulo > Issue Type: New Feature > Components: client > Environment: 洪洞县花呗提现【联系/薇信:18502021565】 >Reporter: xiaojian >Priority: Major > > 洪洞县花呗提现【联系/薇信:18502021565】 -- This message was sent by Atlassian Jira (v8.3.4#803005)
