[accumulo] branch main updated: Auto Generate Accumulo Client in JShell (#1995)
This is an automated email from the ASF dual-hosted git repository.
ctubbsii pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/accumulo.git
The following commit(s) were added to refs/heads/main by this push:
new f9bc914 Auto Generate Accumulo Client in JShell (#1995)
f9bc914 is described below
commit f9bc91405ea4fb166d50d28a26d438265af5b554
Author: slackwinner <50567198+slackwin...@users.noreply.github.com>
AuthorDate: Wed Apr 21 17:05:47 2021 -0400
Auto Generate Accumulo Client in JShell (#1995)
Auto-generate an Accumulo client on JShell startup,
if a client properties file is found on the class path.
---
assemble/src/main/scripts/create-jshell.sh | 31 --
1 file changed, 29 insertions(+), 2 deletions(-)
diff --git a/assemble/src/main/scripts/create-jshell.sh
b/assemble/src/main/scripts/create-jshell.sh
index 6d58fee..2cc2702 100755
--- a/assemble/src/main/scripts/create-jshell.sh
+++ b/assemble/src/main/scripts/create-jshell.sh
@@ -36,6 +36,31 @@ function addAccumuloAPI() {
echo
}
+function addClientBuild() {
+cat < "$mainBase/assemble/target/jshell-init.jsh"
}
[accumulo] branch main updated: Pom changes to allow reuseForks to be set to true (#2015)
This is an automated email from the ASF dual-hosted git repository. jmanno pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/accumulo.git The following commit(s) were added to refs/heads/main by this push: new 5c4e196 Pom changes to allow reuseForks to be set to true (#2015) 5c4e196 is described below commit 5c4e1962097584e4b4269def73bce6be54c2493b Author: Jeffrey Manno AuthorDate: Wed Apr 21 13:56:01 2021 -0400 Pom changes to allow reuseForks to be set to true (#2015) * Created second execution to run the few tests that cannot run with reuseForks being true. --- core/pom.xml | 33 + pom.xml| 2 +- server/tserver/pom.xml | 25 + 3 files changed, 59 insertions(+), 1 deletion(-) diff --git a/core/pom.xml b/core/pom.xml index c5e9552..36d9dc6 100644 --- a/core/pom.xml +++ b/core/pom.xml @@ -195,6 +195,39 @@ + + org.apache.maven.plugins + maven-surefire-plugin + + + + **/TableIdTest.java + **/HadoopCredentialProviderTest.java + **/IdleRatioScanPrioritizerTest.java + **/TestCfCqSlice.java + **/TestCfCqSliceSeekingFilter.java + + + + + reuseForks-false-tests + +test + + +false + + + **/TableIdTest.java + **/HadoopCredentialProviderTest.java + **/IdleRatioScanPrioritizerTest.java + **/TestCfCqSlice.java + **/TestCfCqSliceSeekingFilter.java + + + + + diff --git a/pom.xml b/pom.xml index 54dca69..b41ac03 100644 --- a/pom.xml +++ b/pom.xml @@ -149,7 +149,7 @@ false 1C -false +true 3.0.0-M4 diff --git a/server/tserver/pom.xml b/server/tserver/pom.xml index 362149a..69bb4b0 100644 --- a/server/tserver/pom.xml +++ b/server/tserver/pom.xml @@ -151,6 +151,31 @@ + + org.apache.maven.plugins + maven-surefire-plugin + + + + **/AssignmentWatcherTest.java + + + + + reuseForks-false-tests + +test + + +false + + + **/AssignmentWatcherTest.java + + + + +
[accumulo] branch 1451-external-compactions-feature updated: Prefix thrift types with T that have duplicate names
This is an automated email from the ASF dual-hosted git repository.
kturner pushed a commit to branch 1451-external-compactions-feature
in repository https://gitbox.apache.org/repos/asf/accumulo.git
The following commit(s) were added to
refs/heads/1451-external-compactions-feature by this push:
new 42de4ef Prefix thrift types with T that have duplicate names
42de4ef is described below
commit 42de4efab84b4dc0e4eda23c6d348be744941c30
Author: Keith Turner
AuthorDate: Wed Apr 21 14:09:45 2021 -0400
Prefix thrift types with T that have duplicate names
---
.../compaction/thrift/CompactionCoordinator.java | 70 +++---
.../accumulo/core/compaction/thrift/Status.java| 24
...{CompactionState.java => TCompactionState.java} | 6 +-
.../core/tabletserver/thrift/ActiveCompaction.java | 48 +++
.../{CompactionKind.java => TCompactionKind.java} | 6 +-
...ompactionReason.java => TCompactionReason.java} | 6 +-
...{CompactionStats.java => TCompactionStats.java} | 60 +--
.../{CompactionType.java => TCompactionType.java} | 6 +-
.../thrift/TExternalCompactionJob.java | 64 ++--
core/src/main/thrift/compaction-coordinator.thrift | 8 +--
core/src/main/thrift/tabletserver.thrift | 20 +++
.../accumulo/server/compaction/CompactionInfo.java | 16 ++---
.../accumulo/server/compaction/Compactor.java | 6 +-
.../coordinator/CompactionCoordinator.java | 12 ++--
.../accumulo/coordinator/CompactionUpdate.java | 8 +--
.../accumulo/coordinator/RunningCompaction.java| 12 ++--
.../coordinator/CompactionCoordinatorTest.java | 5 +-
.../accumulo/compactor/CompactionEnvironment.java | 10 ++--
.../accumulo/compactor/CompactionJobHolder.java| 8 +--
.../org/apache/accumulo/compactor/Compactor.java | 20 +++
.../apache/accumulo/compactor/CompactorTest.java | 16 ++---
.../tserver/compactions/ExternalCompactionJob.java | 16 ++---
.../accumulo/tserver/tablet/CompactableUtils.java | 10 ++--
.../accumulo/tserver/tablet/MinorCompactor.java| 10 ++--
.../accumulo/test/ExternalDoNothingCompactor.java | 4 +-
25 files changed, 236 insertions(+), 235 deletions(-)
diff --git
a/core/src/main/thrift-gen-java/org/apache/accumulo/core/compaction/thrift/CompactionCoordinator.java
b/core/src/main/thrift-gen-java/org/apache/accumulo/core/compaction/thrift/CompactionCoordinator.java
index 43ed0ee..490f729 100644
---
a/core/src/main/thrift-gen-java/org/apache/accumulo/core/compaction/thrift/CompactionCoordinator.java
+++
b/core/src/main/thrift-gen-java/org/apache/accumulo/core/compaction/thrift/CompactionCoordinator.java
@@ -31,11 +31,11 @@ public class CompactionCoordinator {
public void cancelCompaction(org.apache.accumulo.core.trace.thrift.TInfo
tinfo, org.apache.accumulo.core.securityImpl.thrift.TCredentials credentials,
java.lang.String externalCompactionId) throws UnknownCompactionIdException,
org.apache.thrift.TException;
-public void
compactionCompleted(org.apache.accumulo.core.trace.thrift.TInfo tinfo,
org.apache.accumulo.core.securityImpl.thrift.TCredentials credentials,
java.lang.String externalCompactionId,
org.apache.accumulo.core.dataImpl.thrift.TKeyExtent extent,
org.apache.accumulo.core.tabletserver.thrift.CompactionStats stats) throws
UnknownCompactionIdException, org.apache.thrift.TException;
+public void
compactionCompleted(org.apache.accumulo.core.trace.thrift.TInfo tinfo,
org.apache.accumulo.core.securityImpl.thrift.TCredentials credentials,
java.lang.String externalCompactionId,
org.apache.accumulo.core.dataImpl.thrift.TKeyExtent extent,
org.apache.accumulo.core.tabletserver.thrift.TCompactionStats stats) throws
UnknownCompactionIdException, org.apache.thrift.TException;
public org.apache.accumulo.core.tabletserver.thrift.TExternalCompactionJob
getCompactionJob(org.apache.accumulo.core.trace.thrift.TInfo tinfo,
org.apache.accumulo.core.securityImpl.thrift.TCredentials credentials,
java.lang.String queueName, java.lang.String compactor, java.lang.String
externalCompactionId) throws org.apache.thrift.TException;
-public void
updateCompactionStatus(org.apache.accumulo.core.trace.thrift.TInfo tinfo,
org.apache.accumulo.core.securityImpl.thrift.TCredentials credentials,
java.lang.String externalCompactionId, CompactionState state, java.lang.String
message, long timestamp) throws UnknownCompactionIdException,
org.apache.thrift.TException;
+public void
updateCompactionStatus(org.apache.accumulo.core.trace.thrift.TInfo tinfo,
org.apache.accumulo.core.securityImpl.thrift.TCredentials credentials,
java.lang.String externalCompactionId, TCompactionState state, java.lang.String
message, long timestamp) throws UnknownCompactionIdException,
org.apache.thrift.TException;
public void compactionFailed(org.apache.accumulo.core.trace.thrift.TInfo
tinfo,
[accumulo] branch 1451-external-compactions-feature updated: Added test for deleting a table during a system compaction
This is an automated email from the ASF dual-hosted git repository.
dlmarion pushed a commit to branch 1451-external-compactions-feature
in repository https://gitbox.apache.org/repos/asf/accumulo.git
The following commit(s) were added to
refs/heads/1451-external-compactions-feature by this push:
new 70e106f Added test for deleting a table during a system compaction
70e106f is described below
commit 70e106f794c3fbd338bf7735dc7cab39d926bfc7
Author: Dave Marion
AuthorDate: Wed Apr 21 16:30:30 2021 +
Added test for deleting a table during a system compaction
---
.../org/apache/accumulo/compactor/Compactor.java | 26 ---
.../apache/accumulo/test/ExternalCompactionIT.java | 42 ++-
.../accumulo/test/ExternalDoNothingCompactor.java | 83 ++
3 files changed, 140 insertions(+), 11 deletions(-)
diff --git
a/server/compactor/src/main/java/org/apache/accumulo/compactor/Compactor.java
b/server/compactor/src/main/java/org/apache/accumulo/compactor/Compactor.java
index 10d1cfa..528810f 100644
---
a/server/compactor/src/main/java/org/apache/accumulo/compactor/Compactor.java
+++
b/server/compactor/src/main/java/org/apache/accumulo/compactor/Compactor.java
@@ -102,7 +102,7 @@ import com.beust.jcommander.Parameter;
public class Compactor extends AbstractServer
implements org.apache.accumulo.core.compaction.thrift.Compactor.Iface {
- static class CompactorServerOpts extends ServerOpts {
+ public static class CompactorServerOpts extends ServerOpts {
@Parameter(required = true, names = {"-q", "--queue"}, description =
"compaction queue name")
private String queueName = null;
@@ -113,11 +113,12 @@ public class Compactor extends AbstractServer
private static final Logger LOG = LoggerFactory.getLogger(Compactor.class);
private static final long TIME_BETWEEN_GC_CHECKS = 5000;
- private static final CompactionJobHolder JOB_HOLDER = new
CompactionJobHolder();
private static final long TEN_MEGABYTES = 10485760;
private static final CompactionCoordinator.Client.Factory
COORDINATOR_CLIENT_FACTORY =
new CompactionCoordinator.Client.Factory();
+ protected static final CompactionJobHolder JOB_HOLDER = new
CompactionJobHolder();
+
private final GarbageCollectionLogger gcLogger = new
GarbageCollectionLogger();
private final UUID compactorId = UUID.randomUUID();
private final AccumuloConfiguration aconf;
@@ -134,7 +135,7 @@ public class Compactor extends AbstractServer
// Exposed for tests
protected volatile Boolean shutdown = false;
- Compactor(CompactorServerOpts opts, String[] args) {
+ protected Compactor(CompactorServerOpts opts, String[] args) {
super("compactor", opts, args);
queueName = opts.getQueueName();
aconf = getConfiguration();
@@ -599,8 +600,8 @@ public class Compactor extends AbstractServer
final CountDownLatch stopped = new CountDownLatch(1);
final Thread compactionThread = Threads.createThread(
-"Compaction job for tablet " + job.getExtent().toString(),
this.createCompactionJob(job,
-totalInputEntries, totalInputBytes, started, stopped, err));
+"Compaction job for tablet " + job.getExtent().toString(),
+createCompactionJob(job, totalInputEntries, totalInputBytes,
started, stopped, err));
synchronized (JOB_HOLDER) {
JOB_HOLDER.set(job, compactionThread);
@@ -608,7 +609,8 @@ public class Compactor extends AbstractServer
final String tableId = new String(job.getExtent().getTable(), UTF_8);
final ServerContext ctxRef = getContext();
-String tablePath = getContext().getZooKeeperRoot() + Constants.ZTABLES
+ "/" + tableId;
+final String tablePath =
+getContext().getZooKeeperRoot() + Constants.ZTABLES + "/" +
tableId;
Watcher tableNodeWatcher = new Watcher() {
@Override
public void process(WatchedEvent event) {
@@ -682,12 +684,14 @@ public class Compactor extends AbstractServer
e.getMessage());
}
}
+} else {
+ LOG.warn("Waiting on compaction thread to finish, but no RUNNING
compaction");
}
}
compactionThread.join();
LOG.info("Compaction thread finished.");
- if (compactionThread.isInterrupted()
+ if (compactionThread.isInterrupted() || JOB_HOLDER.isCancelled()
|| ((err.get() != null &&
err.get().getClass().equals(InterruptedException.class {
LOG.warn("Compaction thread was interrupted, sending CANCELLED
state");
try {
@@ -736,8 +740,12 @@ public class Compactor extends AbstractServer
LOG.error("Error cancelling compaction.", e2);
}
} finally {
-
getContext().getZooReaderWriter().getZooKeeper().removeWatches(tablePath,
- tableNodeWatcher, WatcherType.Any, true);
+
[accumulo] branch main updated (8dfe443 -> 70e1be4)
This is an automated email from the ASF dual-hosted git repository. ctubbsii pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/accumulo.git. from 8dfe443 Fix WebViewsIT add 8bb4749 Revert "Revert "Ensure that HostRegexTableLoadBalancer propery changes are picked up"" add fb16d7c re #2006: changed method from public to protected add 631386d re #2006: Removed the configuration Observable mechism * The properties are now reloaded on every balance and getAssignments call. Hence the table configuration observable mechanism is no longer required. new 70e1be4 Merge branch '1.10' The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes:
[accumulo] 01/01: Merge branch '1.10'
This is an automated email from the ASF dual-hosted git repository. ctubbsii pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/accumulo.git commit 70e1be41ae5f73fcb669f822674a1440262fe6a9 Merge: 8dfe443 631386d Author: Christopher Tubbs AuthorDate: Wed Apr 21 13:01:33 2021 -0400 Merge branch '1.10' No-op merge of #2006 into main branch. That issue only applies to 1.10
