[GitHub] EdColeman commented on issue #553: Correct display of BulkImport duration in monitor
EdColeman commented on issue #553: Correct display of BulkImport duration in monitor URL: https://github.com/apache/accumulo/pull/553#issuecomment-402893907 The code for 2.0 has changed such that I do not see the same issue - but I'll need to dig deeper to find the actual display code - a quick look and it seems that milliseconds are being passed around. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on issue #553: Correct display of BulkImport duration in monitor
ctubbsii commented on issue #553: Correct display of BulkImport duration in monitor URL: https://github.com/apache/accumulo/pull/553#issuecomment-402891217 Do you know if a similar issue affects the 2.0 monitor? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (ACCUMULO-551) Experiment with multi-node batch writer
[ https://issues.apache.org/jira/browse/ACCUMULO-551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ACCUMULO-551: Labels: pull-request-available (was: ) > Experiment with multi-node batch writer > --- > > Key: ACCUMULO-551 > URL: https://issues.apache.org/jira/browse/ACCUMULO-551 > Project: Accumulo > Issue Type: Task >Reporter: Keith Turner >Assignee: Keith Turner >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Accumulo has a batch writer that batches mutations by tablet server for > writes. This works well until there are alot of tablet servers being written > to at which point only a small amount of data is being sent to each tablet > server. Would it be better for the client to batch writes for multiple > tablet servers and send them to one server which writes directly to the > tablet servers? > One possible way to do this is to : > > * batch mutations by rack on the client > * send all of those mutations to one random tablet server on the rack > * have the random tablet server write to the other servers on the rack > This cuts down on the number of direct connections the client has to make. > Could have the following benefits. > * Tablet servers can keep connections open to other tablet servers. > * A write pipeline > Would be interesting to run some test and see how well this works. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Accumulo-Pull-Requests - Build # 1347 - Fixed
The Apache Jenkins build system has built Accumulo-Pull-Requests (build #1347) Status: Fixed Check console output at https://builds.apache.org/job/Accumulo-Pull-Requests/1347/ to view the results.
[GitHub] ctubbsii commented on issue #553: ACCUMULO-551 Correct display of BulkImport duration in monitor
ctubbsii commented on issue #553: ACCUMULO-551 Correct display of BulkImport duration in monitor URL: https://github.com/apache/accumulo/pull/553#issuecomment-402890953 @EdColeman A few helpful things for using GitHub: * There is no need to create an issue *and* a PR if you are creating them at the same time. GitHub PRs are their own issue. Sometimes it's useful to create an issue for future work. * It is not necessary to close a PR and create a new one to change the branch. You can click the "Edit" button and select a different branch to do the PR against from the drop-down. * When you use the form "ACCUMULO-NNN", it automatically links against the JIRA issue with that ID, and that JIRA issue (such as https://issues.apache.org/jira/browse/ACCUMULO-551) may be completely unrelated. GitHub issues are mentioned using the pound syntax: #551 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (ACCUMULO-551) Experiment with multi-node batch writer
[ https://issues.apache.org/jira/browse/ACCUMULO-551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs updated ACCUMULO-551: --- Labels: (was: pull-request-available) > Experiment with multi-node batch writer > --- > > Key: ACCUMULO-551 > URL: https://issues.apache.org/jira/browse/ACCUMULO-551 > Project: Accumulo > Issue Type: Task >Reporter: Keith Turner >Assignee: Keith Turner >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > Accumulo has a batch writer that batches mutations by tablet server for > writes. This works well until there are alot of tablet servers being written > to at which point only a small amount of data is being sent to each tablet > server. Would it be better for the client to batch writes for multiple > tablet servers and send them to one server which writes directly to the > tablet servers? > One possible way to do this is to : > > * batch mutations by rack on the client > * send all of those mutations to one random tablet server on the rack > * have the random tablet server write to the other servers on the rack > This cuts down on the number of direct connections the client has to make. > Could have the following benefits. > * Tablet servers can keep connections open to other tablet servers. > * A write pipeline > Would be interesting to run some test and see how well this works. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] EdColeman opened a new pull request #553: ACCUMULO-551 Correct display of BulkImport duration in monitor
EdColeman opened a new pull request #553: ACCUMULO-551 Correct display of BulkImport duration in monitor URL: https://github.com/apache/accumulo/pull/553 The formatting for monitor display accepts milliseconds. This fix corrects issue where seconds (duration / 1000) were used, causing the duration displayed by the monitor to format and display incorrectly. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] EdColeman closed pull request #552: ACCUMULO-551 Correct display of BulkImport duration in monitor
EdColeman closed pull request #552: ACCUMULO-551 Correct display of BulkImport
duration in monitor
URL: https://github.com/apache/accumulo/pull/552
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BulkImportServlet.java
b/server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BulkImportServlet.java
index 3e41218765..f5c1447ba8 100644
---
a/server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BulkImportServlet.java
+++
b/server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BulkImportServlet.java
@@ -42,7 +42,7 @@ protected String getTitle(HttpServletRequest req) {
}
static private long duration(long start) {
-return (System.currentTimeMillis() - start) / 1000L;
+return System.currentTimeMillis() - start;
}
@Override
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (ACCUMULO-551) Experiment with multi-node batch writer
[ https://issues.apache.org/jira/browse/ACCUMULO-551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ACCUMULO-551: Labels: pull-request-available (was: ) > Experiment with multi-node batch writer > --- > > Key: ACCUMULO-551 > URL: https://issues.apache.org/jira/browse/ACCUMULO-551 > Project: Accumulo > Issue Type: Task >Reporter: Keith Turner >Assignee: Keith Turner >Priority: Major > Labels: pull-request-available > > Accumulo has a batch writer that batches mutations by tablet server for > writes. This works well until there are alot of tablet servers being written > to at which point only a small amount of data is being sent to each tablet > server. Would it be better for the client to batch writes for multiple > tablet servers and send them to one server which writes directly to the > tablet servers? > One possible way to do this is to : > > * batch mutations by rack on the client > * send all of those mutations to one random tablet server on the rack > * have the random tablet server write to the other servers on the rack > This cuts down on the number of direct connections the client has to make. > Could have the following benefits. > * Tablet servers can keep connections open to other tablet servers. > * A write pipeline > Would be interesting to run some test and see how well this works. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] EdColeman opened a new pull request #552: ACCUMULO-551 Correct display of BulkImport duration in monitor
EdColeman opened a new pull request #552: ACCUMULO-551 Correct display of BulkImport duration in monitor URL: https://github.com/apache/accumulo/pull/552 The formatting for monitor display accepts milliseconds. This fix corrects issue where seconds (duration / 1000) were used, causing the duration displayed by the monitor to format and display incorrectly. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] EdColeman opened a new issue #551: Monitor - BulkInput displays incorrectly formatted duration for import age.
EdColeman opened a new issue #551: Monitor - BulkInput displays incorrectly formatted duration for import age. URL: https://github.com/apache/accumulo/issues/551 The duration displayed on the monitor / bulk input displays incorrectly - the values passed to the monitor display function assume the duration is in milliseconds, however, the org.apache.accumulo.monitor.servlets.BulkImportServlet is passing seconds. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit closed issue #550: Update formatter and impsort plugins
asfgit closed issue #550: Update formatter and impsort plugins URL: https://github.com/apache/accumulo/issues/550 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch opened a new pull request #91: Updated 2.0 client documentation
mikewalch opened a new pull request #91: Updated 2.0 client documentation URL: https://github.com/apache/accumulo-website/pull/91 * Due to recent changes in master This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii opened a new issue #550: Update formatter and impsort plugins
ctubbsii opened a new issue #550: Update formatter and impsort plugins URL: https://github.com/apache/accumulo/issues/550 Update formatter and impsort plugins to latest versions. Use newer feature to automatically remove unused and unneeded imports. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
Accumulo-Pull-Requests - Build # 1345 - Failure
The Apache Jenkins build system has built Accumulo-Pull-Requests (build #1345) Status: Failure Check console output at https://builds.apache.org/job/Accumulo-Pull-Requests/1345/ to view the results.
[GitHub] milleruntime commented on issue #499: Implement new Encryption interface WIP
milleruntime commented on issue #499: Implement new Encryption interface WIP URL: https://github.com/apache/accumulo/pull/499#issuecomment-402799965 I am going to rebase since there have been a lot of commits between @PircDef and I. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #549: ACCUMULO-4074 Added support for multiple scan executors
ctubbsii commented on a change in pull request #549: ACCUMULO-4074 Added
support for multiple scan executors
URL: https://github.com/apache/accumulo/pull/549#discussion_r200417343
##
File path: core/src/main/java/org/apache/accumulo/core/conf/Property.java
##
@@ -430,12 +433,33 @@
"When a tablet server's SimpleTimer thread triggers to check idle"
+ " sessions, this configurable option will be used to evaluate
update"
+ " sessions to determine if they can be closed due to inactivity"),
+ @Deprecated
TSERV_READ_AHEAD_MAXCONCURRENT("tserver.readahead.concurrent.max", "16",
PropertyType.COUNT,
- "The maximum number of concurrent read ahead that will execute. This
effectively"
- + " limits the number of long running scans that can run
concurrently per tserver."),
+ "This property is deprecated, use tserver.scan.executors.default.threads
instead."),
Review comment:
Should not remove the old description, because it's still useful to know
what it does, if anything. Also, please `s/deprecated/deprecated since 2.0.0/`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #549: ACCUMULO-4074 Added support for multiple scan executors
ctubbsii commented on a change in pull request #549: ACCUMULO-4074 Added
support for multiple scan executors
URL: https://github.com/apache/accumulo/pull/549#discussion_r200421701
##
File path:
server/tserver/src/main/java/org/apache/accumulo/tserver/session/MultiScanSession.java
##
@@ -53,17 +48,24 @@ public MultiScanSession(TCredentials credentials,
KeyExtent threadPoolExtent,
Map> queries, List ssiList,
Map> ssio, Authorizations authorizations,
SamplerConfiguration samplerConfig, long batchTimeOut, String context) {
-super(credentials);
+super(credentials, new HashSet<>(), ssiList, ssio, authorizations);
this.queries = queries;
-this.ssiList = ssiList;
-this.ssio = ssio;
-this.auths = authorizations;
this.threadPoolExtent = threadPoolExtent;
this.samplerConfig = samplerConfig;
this.batchTimeOut = batchTimeOut;
this.context = context;
}
+ @Override
+ public Type getScanType() {
+return Type.MULTI;
+ }
+
+ @Override
+ public String getTableId() {
+return threadPoolExtent.getTableId().canonicalID();
Review comment:
Should not use String version of tableId unless returning through API back
to user.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #549: ACCUMULO-4074 Added support for multiple scan executors
ctubbsii commented on a change in pull request #549: ACCUMULO-4074 Added support for multiple scan executors URL: https://github.com/apache/accumulo/pull/549#discussion_r200420999 ## File path: server/tserver/src/main/java/org/apache/accumulo/tserver/TabletServer.java ## @@ -587,13 +598,14 @@ public InitialScan startScan(TInfo tinfo, TCredentials credentials, TKeyExtent t if (tablet == null) throw new NotServingTabletException(textent); - Set columnSet = new HashSet<>(); + HashSet columnSet = new HashSet<>(); Review comment: Could be: `columns.stream().map(tcol -> new Column(tcol)).collect(Collectors.toSet());` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #549: ACCUMULO-4074 Added support for multiple scan executors
ctubbsii commented on a change in pull request #549: ACCUMULO-4074 Added
support for multiple scan executors
URL: https://github.com/apache/accumulo/pull/549#discussion_r200418608
##
File path: core/src/main/java/org/apache/accumulo/core/conf/Property.java
##
@@ -430,12 +433,33 @@
"When a tablet server's SimpleTimer thread triggers to check idle"
+ " sessions, this configurable option will be used to evaluate
update"
+ " sessions to determine if they can be closed due to inactivity"),
+ @Deprecated
TSERV_READ_AHEAD_MAXCONCURRENT("tserver.readahead.concurrent.max", "16",
PropertyType.COUNT,
- "The maximum number of concurrent read ahead that will execute. This
effectively"
- + " limits the number of long running scans that can run
concurrently per tserver."),
+ "This property is deprecated, use tserver.scan.executors.default.threads
instead."),
+ @Deprecated
TSERV_METADATA_READ_AHEAD_MAXCONCURRENT("tserver.metadata.readahead.concurrent.max",
"8",
PropertyType.COUNT,
- "The maximum number of concurrent metadata read ahead that will
execute."),
+ "This property is deprecated, use tserver.scan.executors.meta.threads
instead."),
+ TSERV_SCAN_EXECUTORS_PREFIX("tserver.scan.executors.", null,
PropertyType.PREFIX,
Review comment:
I thought there was a problem with having a prefix that also had
well-defined properties "beneath" it. Might only apply to table properties,
though.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #549: ACCUMULO-4074 Added support for multiple scan executors
ctubbsii commented on a change in pull request #549: ACCUMULO-4074 Added
support for multiple scan executors
URL: https://github.com/apache/accumulo/pull/549#discussion_r200417539
##
File path: core/src/main/java/org/apache/accumulo/core/conf/Property.java
##
@@ -430,12 +433,33 @@
"When a tablet server's SimpleTimer thread triggers to check idle"
+ " sessions, this configurable option will be used to evaluate
update"
+ " sessions to determine if they can be closed due to inactivity"),
+ @Deprecated
TSERV_READ_AHEAD_MAXCONCURRENT("tserver.readahead.concurrent.max", "16",
PropertyType.COUNT,
- "The maximum number of concurrent read ahead that will execute. This
effectively"
- + " limits the number of long running scans that can run
concurrently per tserver."),
+ "This property is deprecated, use tserver.scan.executors.default.threads
instead."),
+ @Deprecated
TSERV_METADATA_READ_AHEAD_MAXCONCURRENT("tserver.metadata.readahead.concurrent.max",
"8",
PropertyType.COUNT,
- "The maximum number of concurrent metadata read ahead that will
execute."),
+ "This property is deprecated, use tserver.scan.executors.meta.threads
instead."),
Review comment:
Same comment as above about the other property.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] bbux-atg commented on issue #510: ACCUMULO-4074
bbux-atg commented on issue #510: ACCUMULO-4074 URL: https://github.com/apache/accumulo/pull/510#issuecomment-402787253 closing as the changes are updated in https://github.com/apache/accumulo/pull/549 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] bbux-atg closed pull request #510: ACCUMULO-4074
bbux-atg closed pull request #510: ACCUMULO-4074
URL: https://github.com/apache/accumulo/pull/510
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/core/src/main/java/org/apache/accumulo/core/conf/Property.java
b/core/src/main/java/org/apache/accumulo/core/conf/Property.java
index fc423acca5..42fc87a817 100644
--- a/core/src/main/java/org/apache/accumulo/core/conf/Property.java
+++ b/core/src/main/java/org/apache/accumulo/core/conf/Property.java
@@ -436,6 +436,8 @@
TSERV_READ_AHEAD_MAXCONCURRENT("tserver.readahead.concurrent.max", "16",
PropertyType.COUNT,
"The maximum number of concurrent read ahead that will execute. This
effectively"
+ " limits the number of long running scans that can run
concurrently per tserver."),
+ TSERV_READ_AHEAD_PREFIX("tserver.readahead.concurrent.table.", null,
PropertyType.PREFIX,
+ "Properties in this category allow overriding of table specific read
ahead pools"),
TSERV_METADATA_READ_AHEAD_MAXCONCURRENT("tserver.metadata.readahead.concurrent.max",
"8",
PropertyType.COUNT,
"The maximum number of concurrent metadata read ahead that will
execute."),
@@ -542,6 +544,9 @@
TSERV_SUMMARY_RETRIEVAL_THREADS("tserver.summary.retrieval.threads", "10",
PropertyType.COUNT,
"The number of threads on each tablet server available to retrieve"
+ " summary data, that is not currently in cache, from RFiles."),
+ TSERV_SESSION_COMPARATOR_CLASS("tserver.summary.comparator.class", "",
PropertyType.CLASSNAME,
+ "A customizable Scan session comparator. Note that by default, the value
is empty"
+ + " and thus uses no session comparator"),
// accumulo garbage collector properties
GC_PREFIX("gc.", null, PropertyType.PREFIX,
diff --git
a/core/src/main/java/org/apache/accumulo/core/util/AccumuloUncaughtExceptionHandler.java
b/core/src/main/java/org/apache/accumulo/core/util/AccumuloUncaughtExceptionHandler.java
new file mode 100644
index 00..f688010f16
--- /dev/null
+++
b/core/src/main/java/org/apache/accumulo/core/util/AccumuloUncaughtExceptionHandler.java
@@ -0,0 +1,34 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.core.util;
+
+import java.lang.Thread.UncaughtExceptionHandler;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class AccumuloUncaughtExceptionHandler implements
UncaughtExceptionHandler {
+
+ private static final Logger log =
LoggerFactory.getLogger(AccumuloUncaughtExceptionHandler.class);
+
+ @Override
+ public void uncaughtException(Thread t, Throwable e) {
+
+log.error(String.format("Caught an exception in %s. Shutting down.", t),
e);
+ }
+
+}
diff --git
a/core/src/main/java/org/apache/accumulo/core/util/NamingThreadFactory.java
b/core/src/main/java/org/apache/accumulo/core/util/NamingThreadFactory.java
index d8b307c36c..0b4730cc70 100644
--- a/core/src/main/java/org/apache/accumulo/core/util/NamingThreadFactory.java
+++ b/core/src/main/java/org/apache/accumulo/core/util/NamingThreadFactory.java
@@ -26,6 +26,8 @@
public class NamingThreadFactory implements ThreadFactory {
private static final Logger log =
LoggerFactory.getLogger(NamingThreadFactory.class);
+ private static final AccumuloUncaughtExceptionHandler uncaughtHandler = new
AccumuloUncaughtExceptionHandler();
+
private AtomicInteger threadNum = new AtomicInteger(1);
private String name;
@@ -35,7 +37,10 @@ public NamingThreadFactory(String name) {
@Override
public Thread newThread(Runnable r) {
-return new Daemon(new LoggingRunnable(log, r), name + " " +
threadNum.getAndIncrement());
+Thread thread = new Daemon(new LoggingRunnable(log, r),
+name + " " + threadNum.getAndIncrement());
+thread.setUncaughtExceptionHandler(uncaughtHandler);
+return thread;
}
}
diff --git
a/server/tserver/src/main/java/org/apache/accumulo/tserver/TabletServerResourceManager.java
[GitHub] mikewalch closed pull request #90: Created post about Accumulo Summit
mikewalch closed pull request #90: Created post about Accumulo Summit URL: https://github.com/apache/accumulo-website/pull/90 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/_posts/blog/2018-07-05-accumulo-summit.md b/_posts/blog/2018-07-05-accumulo-summit.md new file mode 100644 index ..22851cca --- /dev/null +++ b/_posts/blog/2018-07-05-accumulo-summit.md @@ -0,0 +1,16 @@ +--- +title: "Accumulo Summit is on October 15th!" +--- + +The [Fifth Annual Accumulo Summit][summit] will be held on October 15, 2018 at the [Sheraton Columbia Town Center Hotel][hotel] in Columbia, MD. + +This day-long event offers a unique opportunity for attendees to get introduced to Apache Accumulo, sharpen their skillsets, and connect with leading Accumulo users and developers. + +Have a great idea you'd like to share? Engineers, architects, and business leaders are encouraged to share their experiences or present a topic that would be of interest to the Accumulo community. [Talks can be submitted ][talk] through August 1st. + +[Early bird registration is now open][register]! Sign up before September 1st to save $50 off the regular admission price. + +[summit]: http://accumulosummit.com/ +[hotel]: http://accumulosummit.com/about/venue/ +[talk]: http://accumulosummit.com/program/submit-talk/ +[register]: http://accumulosummit.com/register/ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on issue #549: ACCUMULO-4074 Added support for multiple scan executors
keith-turner commented on issue #549: ACCUMULO-4074 Added support for multiple scan executors URL: https://github.com/apache/accumulo/pull/549#issuecomment-402755063 All ITs passed for 73f7f8d merged into 13a025d This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch closed pull request #15: Added random pausing to continuous ingest
mikewalch closed pull request #15: Added random pausing to continuous ingest
URL: https://github.com/apache/accumulo-testing/pull/15
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/conf/accumulo-testing.properties.example
b/conf/accumulo-testing.properties.example
index 7468443..4fff104 100644
--- a/conf/accumulo-testing.properties.example
+++ b/conf/accumulo-testing.properties.example
@@ -77,6 +77,16 @@ test.ci.ingest.max.cq=32767
test.ci.ingest.visibilities=
# Checksums will be generated during ingest if set to true
test.ci.ingest.checksum=true
+# Enables periodic pausing of ingest
+test.ci.ingest.pause.enabled=false
+# Minimum wait between ingest pauses (in seconds)
+test.ci.ingest.pause.wait.min=120
+# Maximum wait between ingest pauses (in seconds)
+test.ci.ingest.pause.wait.max=180
+# Minimum pause duration (in seconds)
+test.ci.ingest.pause.duration.min=60
+# Maximum pause duration (in seconds)
+test.ci.ingest.pause.duration.max=120
# Batch walker
#
diff --git a/core/src/main/java/org/apache/accumulo/testing/core/TestProps.java
b/core/src/main/java/org/apache/accumulo/testing/core/TestProps.java
index b0927db..638d2db 100644
--- a/core/src/main/java/org/apache/accumulo/testing/core/TestProps.java
+++ b/core/src/main/java/org/apache/accumulo/testing/core/TestProps.java
@@ -92,6 +92,16 @@
public static final String CI_INGEST_VISIBILITIES = CI_INGEST +
"visibilities";
// Checksums will be generated during ingest if set to true
public static final String CI_INGEST_CHECKSUM = CI_INGEST + "checksum";
+ // Enables periodic pausing of ingest
+ public static final String CI_INGEST_PAUSE_ENABLED = CI_INGEST +
"pause.enabled";
+ // Minimum wait between ingest pauses (in seconds)
+ public static final String CI_INGEST_PAUSE_WAIT_MIN = CI_INGEST +
"pause.wait.min";
+ // Maximum wait between ingest pauses (in seconds)
+ public static final String CI_INGEST_PAUSE_WAIT_MAX = CI_INGEST +
"pause.wait.max";
+ // Minimum pause duration (in seconds)
+ public static final String CI_INGEST_PAUSE_DURATION_MIN = CI_INGEST +
"pause.duration.min";
+ // Maximum pause duration (in seconds)
+ public static final String CI_INGEST_PAUSE_DURATION_MAX = CI_INGEST +
"pause.duration.max";
/** Batch Walker **/
// Sleep time between batch scans (in ms)
diff --git
a/core/src/main/java/org/apache/accumulo/testing/core/continuous/ContinuousIngest.java
b/core/src/main/java/org/apache/accumulo/testing/core/continuous/ContinuousIngest.java
index db281a7..4afd00c 100644
---
a/core/src/main/java/org/apache/accumulo/testing/core/continuous/ContinuousIngest.java
+++
b/core/src/main/java/org/apache/accumulo/testing/core/continuous/ContinuousIngest.java
@@ -24,9 +24,11 @@
import java.util.Properties;
import java.util.Random;
import java.util.UUID;
+import java.util.concurrent.TimeUnit;
import java.util.zip.CRC32;
import java.util.zip.Checksum;
+import com.google.common.base.Preconditions;
import org.apache.accumulo.core.client.BatchWriter;
import org.apache.accumulo.core.client.Connector;
import org.apache.accumulo.core.client.MutationsRejectedException;
@@ -39,17 +41,62 @@
import org.apache.accumulo.core.util.FastFormat;
import org.apache.accumulo.testing.core.TestProps;
import org.apache.hadoop.io.Text;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
public class ContinuousIngest {
+ private static final Logger log =
LoggerFactory.getLogger(ContinuousIngest.class);
+
private static final byte[] EMPTY_BYTES = new byte[0];
private static List visibilities;
+ private static long lastPauseNs;
+ private static long pauseWaitSec;
private static ColumnVisibility getVisibility(Random rand) {
return visibilities.get(rand.nextInt(visibilities.size()));
}
+ private static boolean pauseEnabled(Properties props) {
+String value = props.getProperty(TestProps.CI_INGEST_PAUSE_ENABLED);
+return Boolean.parseBoolean(value);
+ }
+
+ private static int getPauseWaitSec(Properties props, Random rand) {
+int waitMin =
Integer.parseInt(props.getProperty(TestProps.CI_INGEST_PAUSE_WAIT_MIN));
+int waitMax =
Integer.parseInt(props.getProperty(TestProps.CI_INGEST_PAUSE_WAIT_MAX));
+Preconditions.checkState(waitMax >= waitMin && waitMin > 0);
+if (waitMax == waitMin) {
+ return waitMin;
+}
+return (rand.nextInt(waitMax - waitMin) + waitMin);
+ }
+
+ private static int getPauseDurationSec(Properties props, Random rand) {
+int durationMin =
Integer.parseInt(props.getProperty(TestProps.CI_INGEST_PAUSE_DURATION_MIN));
+int durationMax =
Integer.parseInt(props.getProperty(TestProps.CI_INGEST_PAUSE_DURATION_MAX));
+Preconditions.checkState(durationMax >= durationMin &&
[GitHub] mikewalch opened a new pull request #90: Created post about Accumulo Summit
mikewalch opened a new pull request #90: Created post about Accumulo Summit URL: https://github.com/apache/accumulo-website/pull/90 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (ACCUMULO-4592) Add since information to properties
[ https://issues.apache.org/jira/browse/ACCUMULO-4592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16533745#comment-16533745 ] Keith Turner commented on ACCUMULO-4592: Could also possibly add an optional deprecated since version parameter. > Add since information to properties > --- > > Key: ACCUMULO-4592 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4592 > Project: Accumulo > Issue Type: Improvement >Reporter: Keith Turner >Priority: Major > > It would be very useful to document, in the user manual, the version in which > each property was added. Not sure what the best way to do this is. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] keith-turner commented on a change in pull request #546: Fix multiple concurrency bugs in Master.gatherTableInformation()
keith-turner commented on a change in pull request #546: Fix multiple
concurrency bugs in Master.gatherTableInformation()
URL: https://github.com/apache/accumulo/pull/546#discussion_r200373311
##
File path: core/src/main/java/org/apache/accumulo/core/conf/Property.java
##
@@ -340,6 +340,10 @@
MASTER_REPLICATION_COORDINATOR_THREADCHECK("master.replication.coordinator.threadcheck.time",
"5s", PropertyType.TIMEDURATION,
"The time between adjustments of the coordinator thread pool"),
+ /**
+ * @deprecated since 1.9.1
Review comment:
I updated the docs. I looked into the the code that generates prop docs
and it generates documentation for deprecated props. So users looking at the
docs will know which props are deprecated, but they will not know when. If
[ACCUMULO-4592](https://issues.apache.org/jira/browse/ACCUMULO-4592) were done,
could also add a deprecated version.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
