[accumulo] branch main updated (f389518b6b -> b7d00d068f)
This is an automated email from the ASF dual-hosted git repository. ctubbsii pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/accumulo.git from f389518b6b Merge branch '2.1' add 2931a9b931 Call setDropBehind() when reading/writing WAL files (#3077) new b7d00d068f Merge branch '2.1' The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../src/main/java/org/apache/accumulo/tserver/log/DfsLogger.java | 7 +++ .../src/main/java/org/apache/accumulo/tserver/log/LogSorter.java | 8 2 files changed, 15 insertions(+)
[accumulo] 01/01: Merge branch '2.1'
This is an automated email from the ASF dual-hosted git repository. ctubbsii pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/accumulo.git commit b7d00d068f2f813936bd03ee69a63b7c6a453950 Merge: f389518b6b 2931a9b931 Author: Christopher Tubbs AuthorDate: Wed Nov 16 00:38:10 2022 -0500 Merge branch '2.1' .../src/main/java/org/apache/accumulo/tserver/log/DfsLogger.java | 7 +++ .../src/main/java/org/apache/accumulo/tserver/log/LogSorter.java | 8 2 files changed, 15 insertions(+)
[accumulo-website] branch asf-site updated (487f2ee7 -> 7d2b63ee)
This is an automated email from the ASF dual-hosted git repository. ctubbsii pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/accumulo-website.git from 487f2ee7 Automatic Site Publish by Buildbot add e438900b Automatic Site Publish by Buildbot add 7d2b63ee Automatic Site Publish by Buildbot No new revisions were added by this update. Summary of changes: output/docs/2.x/administration/upgrading.html | 122 output/docs/2.x/getting-started/quickstart.html | 9 +- output/docs/2.x/troubleshooting/tools.html | 2 +- output/feed.xml | 4 +- output/search_data.json | 4 +- 5 files changed, 117 insertions(+), 24 deletions(-)
[accumulo-website] branch asf-staging updated: Automatic Site Publish by Buildbot
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-staging in repository https://gitbox.apache.org/repos/asf/accumulo-website.git The following commit(s) were added to refs/heads/asf-staging by this push: new 7d2b63ee Automatic Site Publish by Buildbot 7d2b63ee is described below commit 7d2b63ee96b4494e9d8f3b19e8b4b9d0f25d5bc6 Author: buildbot AuthorDate: Wed Nov 16 05:21:45 2022 + Automatic Site Publish by Buildbot --- output/docs/2.x/administration/upgrading.html | 158 output/docs/2.x/getting-started/quickstart.html | 6 +- output/docs/2.x/troubleshooting/tools.html | 2 +- output/feed.xml | 4 +- output/search_data.json | 4 +- 5 files changed, 90 insertions(+), 84 deletions(-) diff --git a/output/docs/2.x/administration/upgrading.html b/output/docs/2.x/administration/upgrading.html index 5b6b96c0..18d4f1f8 100644 --- a/output/docs/2.x/administration/upgrading.html +++ b/output/docs/2.x/administration/upgrading.html @@ -430,103 +430,95 @@ -Upgrading from 1.10.x or 2.0.x to 2.1 +Upgrading from 1.10 or 2.0 to 2.1 -The recommended way to upgrade from a prior 1.10.x or 2.0.x release is to stop Accumulo, upgrade -to 2.1 and then start 2.1. To upgrade from a release prior to 1.10, follow the -below steps to upgrade to at least 1.10 and then perform the upgrade to 2.1. A -direct upgrade from releases prior to 1.10 has not been tested. +Please read these directions in their entirety before beginning. Please contact us with any +questions you have about this process. -Important: before starting any upgrade process you need to make sure there are no outstanding FATE transactions. This -includes transactions that have completed with SUCCESS or FAILED but have not been removed by the automatic clean-up -process. This is required because the internal serialization of FATE transactions is not guaranteed to be compatible -between versions, so ANY FATE transaction that is present will fail the upgrade. Procedures to manage FATE -transactions, including commands to fail and delete transactions, are included in the Accumulo documentation -FATE - Administration +IMPORTANT! Before running any Accumulo 2.1 upgrade utilities or services, you will need to +upgrade to Java 11, Hadoop 3, and at least ZooKeeper 3.5 (at least 3.8 was current at the time of +this writing and is recommended). -Two significant changes from 1.10.x/2.0 to 2.1 that are important to note: +The basic upgrade sequence is: + + + upgrade to at least Accumulo 1.10 first (if necessary) + stop Accumulo 1.10 or 2.0 + prepare your installation of Accumulo 2.1 through whatever means you obtain the binaries and +configure it in your environment + (optional - but recommended) create a ZooKeeper snapshot + (required if not using the provided scripts to start 2.1) run the RenameMasterDirInZK utility + (optional) run the pre-upgrade utility to convert the configuration in ZooKeeper + start Accumulo 2.1 for the first time to complete the upgrade + + +IMPORTANT! before starting any upgrade process you need to make sure there are no outstanding +FATE transactions. This includes transactions that have completed with SUCCESS or FAILED but +have not been removed by the automatic clean-up process. This is required because the internal +serialization of FATE transactions is not guaranteed to be compatible between versions, so ANY +FATE transaction that is present will fail the upgrade. Procedures to manage FATE transactions, +including commands to fail and delete transactions, are included in the FATE Administration +documentation. + +Two significant changes occurred in 2.1 that are particularly important to note for upgrades: properties and services that referenced master are renamed manager and - the property storage in ZooKeeper has changed from a ZooKeeper node per property to a single node for all -properties with a single ZooKeeper property node for each table, namespace. and the system config properties. + the internal property storage format in ZooKeeper has changed - instead of each table, namespace, +and the system configuration using separate ZooKeeper nodes for each of their properties, they +each now use only a single ZooKeeper node for all of their respective properties. -Before upgrading to 2.1, it is suggested that you create a snapshot of the current ZooKeeper contents to be a backup -in case issues occur and you need to rollback. There are no provisions to roll back to a previous Accumulo version -once an upgrade process has been completed other than restoring from a snapshot of ZooKeeper. +Details on renaming the properties and the ZooKeeper property conversion are provided in the +following sections. Additional information on configuring 2.1 is available here. -Details on renaming the
[accumulo-website] branch main updated: More improvements to the upgrade docs
This is an automated email from the ASF dual-hosted git repository.
ctubbsii pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/accumulo-website.git
The following commit(s) were added to refs/heads/main by this push:
new ed255cd8 More improvements to the upgrade docs
ed255cd8 is described below
commit ed255cd864d1cad169ff3ebdfe44234d172d6e7d
Author: Christopher Tubbs
AuthorDate: Wed Nov 16 00:16:14 2022 -0500
More improvements to the upgrade docs
Follow up from #367
* eliminate duplicate information
* remove unnecessary details to favor being concise
* add some helpful details
* fix formatting
* fix broken links
---
_docs-2/administration/upgrading.md | 155 ++
_docs-2/getting-started/quickstart.md | 6 +-
_docs-2/troubleshooting/tools.md | 2 +-
3 files changed, 86 insertions(+), 77 deletions(-)
diff --git a/_docs-2/administration/upgrading.md
b/_docs-2/administration/upgrading.md
index 6614d0ca..6c613c2f 100644
--- a/_docs-2/administration/upgrading.md
+++ b/_docs-2/administration/upgrading.md
@@ -4,102 +4,96 @@ category: administration
order: 7
---
-## Upgrading from 1.10.x or 2.0.x to 2.1
+## Upgrading from 1.10 or 2.0 to 2.1
-The recommended way to upgrade from a prior 1.10.x or 2.0.x release is to stop
Accumulo, upgrade
-to 2.1 and then start 2.1. To upgrade from a release prior to 1.10, follow the
-[below steps](#upgrading-from-18910-to-20) to upgrade to at least 1.10 and
then perform the upgrade to 2.1. A
-direct upgrade from releases prior to 1.10 has not been tested.
+Please read these directions in their entirety before beginning. Please
[contact] us with any
+questions you have about this process.
-Important: before starting any upgrade process you need to make sure there are
no outstanding FATE transactions. This
-includes transactions that have completed with `SUCCESS` or `FAILED` but have
not been removed by the automatic clean-up
-process. This is required because the internal serialization of FATE
transactions is not guaranteed to be compatible
-between versions, so *ANY* FATE transaction that is present will fail the
upgrade. Procedures to manage FATE
-transactions, including commands to fail and delete transactions, are included
in the Accumulo documentation
-[FATE - Administration]({% durl administration/fate#administration %})
+**IMPORTANT!** Before running any Accumulo 2.1 upgrade utilities or services,
you will need to
+upgrade to Java 11, Hadoop 3, and at least ZooKeeper 3.5 (at least 3.8 was
current at the time of
+this writing and is recommended).
-Two significant changes from 1.10.x/2.0 to 2.1 that are important to note:
+The basic upgrade sequence is:
-1. properties and services that referenced `master` are renamed `manager` and
-2. the property storage in ZooKeeper has changed from a ZooKeeper node per
property to a single node for all
-properties with a single ZooKeeper property node for each table, namespace.
and the system config properties.
+- upgrade to at least Accumulo 1.10 first (if necessary)
+- stop Accumulo 1.10 or 2.0
+- prepare your installation of Accumulo 2.1 through whatever means you obtain
the binaries and
+ configure it in your environment
+- (optional - but recommended) create a ZooKeeper snapshot
+- (required if not using the provided scripts to start 2.1) run the
`RenameMasterDirInZK` utility
+- (optional) run the pre-upgrade utility to convert the configuration in
ZooKeeper
+- start Accumulo 2.1 for the first time to complete the upgrade
-Before upgrading to 2.1, it is suggested that you create a snapshot of the
current ZooKeeper contents to be a backup
-in case issues occur and you need to rollback. There are no provisions to
roll back to a previous Accumulo version
-once an upgrade process has been completed other than restoring from a
snapshot of ZooKeeper.
+**IMPORTANT!** before starting any upgrade process you need to make sure there
are no outstanding
+FATE transactions. This includes transactions that have completed with
`SUCCESS` or `FAILED` but
+have not been removed by the automatic clean-up process. This is required
because the internal
+serialization of FATE transactions is not guaranteed to be compatible between
versions, so *ANY*
+FATE transaction that is present will fail the upgrade. Procedures to manage
FATE transactions,
+including commands to fail and delete transactions, are included in the [FATE
Administration
+documentation]({% durl administration/fate#administration %}).
-Details on renaming the properties and the ZooKeeper property conversion
provided in sections that follow.
+Two significant changes occurred in 2.1 that are particularly important to
note for upgrades:
-### Rename master Properties, Config Files, and Script References
+1. properties and services that referenced `master` are renamed `manager` and
+2. the internal property storage format in
[accumulo-website] branch asf-staging updated: Automatic Site Publish by Buildbot
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-staging in repository https://gitbox.apache.org/repos/asf/accumulo-website.git The following commit(s) were added to refs/heads/asf-staging by this push: new e438900b Automatic Site Publish by Buildbot e438900b is described below commit e438900b20f0ad1ff95484ad110146539e0046db Author: buildbot AuthorDate: Wed Nov 16 05:20:28 2022 + Automatic Site Publish by Buildbot --- output/docs/2.x/administration/upgrading.html | 86 - output/docs/2.x/getting-started/quickstart.html | 7 +- output/feed.xml | 4 +- output/search_data.json | 4 +- 4 files changed, 94 insertions(+), 7 deletions(-) diff --git a/output/docs/2.x/administration/upgrading.html b/output/docs/2.x/administration/upgrading.html index e803e7e2..5b6b96c0 100644 --- a/output/docs/2.x/administration/upgrading.html +++ b/output/docs/2.x/administration/upgrading.html @@ -434,9 +434,30 @@ The recommended way to upgrade from a prior 1.10.x or 2.0.x release is to stop Accumulo, upgrade to 2.1 and then start 2.1. To upgrade from a release prior to 1.10, follow the -below steps to upgrade to 2.0 and then perform the upgrade to 2.1. A +below steps to upgrade to at least 1.10 and then perform the upgrade to 2.1. A direct upgrade from releases prior to 1.10 has not been tested. +Important: before starting any upgrade process you need to make sure there are no outstanding FATE transactions. This +includes transactions that have completed with SUCCESS or FAILED but have not been removed by the automatic clean-up +process. This is required because the internal serialization of FATE transactions is not guaranteed to be compatible +between versions, so ANY FATE transaction that is present will fail the upgrade. Procedures to manage FATE +transactions, including commands to fail and delete transactions, are included in the Accumulo documentation +FATE - Administration + +Two significant changes from 1.10.x/2.0 to 2.1 that are important to note: + + + properties and services that referenced master are renamed manager and + the property storage in ZooKeeper has changed from a ZooKeeper node per property to a single node for all +properties with a single ZooKeeper property node for each table, namespace. and the system config properties. + + +Before upgrading to 2.1, it is suggested that you create a snapshot of the current ZooKeeper contents to be a backup +in case issues occur and you need to rollback. There are no provisions to roll back to a previous Accumulo version +once an upgrade process has been completed other than restoring from a snapshot of ZooKeeper. + +Details on renaming the properties and the ZooKeeper property conversion provided in sections that follow. + Rename master Properties, Config Files, and Script References Although not required until at least release 3.0, it is strongly recommended as a part of the @@ -463,7 +484,68 @@ will then need to transfer the contents of the current individual files to this Warning: Upgrading a previously encrypted instance is not supported as the implementation and properties have changed. -Upgrading from 1.8/9 to 2.0 +Upgrading from 1.9/1.10/2.0 to 2.1 + +Note: the process for upgrading to 2.1 assumes that you have already upgraded to 1.10 or 2.0. Upgrading from a +version prior to 1.10 may work, but is unsupported and untested. If you are upgrading from 1.10 make sure +that you have adjusted for the master to manager and other configuration changes as described in the +Upgrading from 1.8/9/10 to 2.0 below steps. + +Additional information on configuring 2.1 is available at (/docs/2.x//in-depth-installation) + +The particular order for your installation will differ depending on how you choose to install Accumulo. +The key part being that the upgrade utilities need to be run with 2.1 code on a stopped 1.10 / 2.0 instance. + +You can elect to stop your current instance, install and configure the 2.1 binaries, optionally run the upgrade +utilities and then start 2.1, or you could install and configure the 2.1 binaries while your instance remains +running and then when ready, stop your instance, optionally run the upgrade utilities and then start 2.1. + +The basic upgrade sequence is to: + + + stop Accumulo (1.10/2.0) + update your environment to point to 2.1 executable path. + (optional - but recommended) create a ZooKeeper snapshot + (optional) run the configuration upgrade utility + start Accumulo 2.1 + + +Create ZooKeeper snapshot (optional - but recommended) + +$ACCUMULO_HOME/bin/accumulo dump-zoo --xml --root /accumulo | tee PATH_TO_SNAPSHOT + + +If you need to restore from the ZooKeeper snapshot see (/docs/2.x/troubleshooting/tools ) + +Upgrade the property storage in ZooKeeper (optional) + +The property conversion can be done
[accumulo-website] branch main updated: update upgrade instructions to include prop conversion (#367)
This is an automated email from the ASF dual-hosted git repository.
ctubbsii pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/accumulo-website.git
The following commit(s) were added to refs/heads/main by this push:
new 65ae3ff3 update upgrade instructions to include prop conversion (#367)
65ae3ff3 is described below
commit 65ae3ff39d6b1b6977f491285aa4f7c84fa17848
Author: EdColeman
AuthorDate: Wed Nov 16 00:20:08 2022 -0500
update upgrade instructions to include prop conversion (#367)
---
_docs-2/administration/upgrading.md | 86 ++-
_docs-2/getting-started/quickstart.md | 7 ++-
2 files changed, 90 insertions(+), 3 deletions(-)
diff --git a/_docs-2/administration/upgrading.md
b/_docs-2/administration/upgrading.md
index d9ee2bf2..6614d0ca 100644
--- a/_docs-2/administration/upgrading.md
+++ b/_docs-2/administration/upgrading.md
@@ -8,9 +8,28 @@ order: 7
The recommended way to upgrade from a prior 1.10.x or 2.0.x release is to stop
Accumulo, upgrade
to 2.1 and then start 2.1. To upgrade from a release prior to 1.10, follow the
-[below steps](#upgrading-from-189-to-20) to upgrade to 2.0 and then perform
the upgrade to 2.1. A
+[below steps](#upgrading-from-18910-to-20) to upgrade to at least 1.10 and
then perform the upgrade to 2.1. A
direct upgrade from releases prior to 1.10 has not been tested.
+Important: before starting any upgrade process you need to make sure there are
no outstanding FATE transactions. This
+includes transactions that have completed with `SUCCESS` or `FAILED` but have
not been removed by the automatic clean-up
+process. This is required because the internal serialization of FATE
transactions is not guaranteed to be compatible
+between versions, so *ANY* FATE transaction that is present will fail the
upgrade. Procedures to manage FATE
+transactions, including commands to fail and delete transactions, are included
in the Accumulo documentation
+[FATE - Administration]({% durl administration/fate#administration %})
+
+Two significant changes from 1.10.x/2.0 to 2.1 that are important to note:
+
+1. properties and services that referenced `master` are renamed `manager` and
+2. the property storage in ZooKeeper has changed from a ZooKeeper node per
property to a single node for all
+properties with a single ZooKeeper property node for each table, namespace.
and the system config properties.
+
+Before upgrading to 2.1, it is suggested that you create a snapshot of the
current ZooKeeper contents to be a backup
+in case issues occur and you need to rollback. There are no provisions to
roll back to a previous Accumulo version
+once an upgrade process has been completed other than restoring from a
snapshot of ZooKeeper.
+
+Details on renaming the properties and the ZooKeeper property conversion
provided in sections that follow.
+
### Rename master Properties, Config Files, and Script References
Although not required until at least release 3.0, it is strongly recommended
as a part of the
@@ -38,7 +57,70 @@ will then need to transfer the contents of the current
individual files to this
**Warning**: Upgrading a previously encrypted instance is not supported as the
implementation
and properties have changed.
-## Upgrading from 1.8/9 to 2.0
+## Upgrading from 1.9/1.10/2.0 to 2.1
+
+Note: the process for upgrading to 2.1 assumes that you have already upgraded
to 1.10 or 2.0. Upgrading from a
+version prior to 1.10 may work, but is unsupported and untested. If you are
upgrading from 1.10 make sure
+that you have adjusted for the `master` to `manager` and other configuration
changes as described in the
+Upgrading from 1.8/9/10 to 2.0 [below steps](#upgrading-from-18910-to-20).
+
+Additional information on configuring 2.1 is available at ({% durl
/in-depth-installation%})
+
+The particular order for your installation will differ depending on how you
choose to install Accumulo.
+The key part being that the upgrade utilities need to be run with 2.1 code on
a stopped 1.10 / 2.0 instance.
+
+You can elect to stop your current instance, install and configure the 2.1
binaries, optionally run the upgrade
+utilities and then start 2.1, or you could install and configure the 2.1
binaries while your instance remains
+running and then when ready, stop your instance, optionally run the upgrade
utilities and then start 2.1.
+
+The basic upgrade sequence is to:
+
+- stop Accumulo (1.10/2.0)
+- update your environment to point to 2.1 executable path.
+- (optional - but recommended) create a ZooKeeper snapshot
+- (optional) run the configuration upgrade utility
+- start Accumulo 2.1
+
+### Create ZooKeeper snapshot (optional - but recommended)
+
+```
+$ACCUMULO_HOME/bin/accumulo dump-zoo --xml --root /accumulo | tee
PATH_TO_SNAPSHOT
+```
+
+If you need to restore from the ZooKeeper snapshot see ({% durl
troubleshooting/tools %})
+
+### Upgrade the property storage in ZooKeeper
[accumulo-proxy] branch main updated: Convert try-catch blocks in tests to JUnit assertThrows (#41)
This is an automated email from the ASF dual-hosted git repository.
domgarguilo pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/accumulo-proxy.git
The following commit(s) were added to refs/heads/main by this push:
new d8ab970 Convert try-catch blocks in tests to JUnit assertThrows (#41)
d8ab970 is described below
commit d8ab970930c77e06444c733af0301baf122f7501
Author: Dom G
AuthorDate: Tue Nov 15 10:45:47 2022 -0500
Convert try-catch blocks in tests to JUnit assertThrows (#41)
---
.../org/apache/accumulo/proxy/ProxyServerTest.java | 18 +-
.../apache/accumulo/proxy/its/SimpleProxyBase.java | 649 -
2 files changed, 241 insertions(+), 426 deletions(-)
diff --git a/src/test/java/org/apache/accumulo/proxy/ProxyServerTest.java
b/src/test/java/org/apache/accumulo/proxy/ProxyServerTest.java
index e09c6dc..8ebf886 100644
--- a/src/test/java/org/apache/accumulo/proxy/ProxyServerTest.java
+++ b/src/test/java/org/apache/accumulo/proxy/ProxyServerTest.java
@@ -17,7 +17,7 @@
package org.apache.accumulo.proxy;
import static java.nio.charset.StandardCharsets.UTF_8;
-import static org.junit.jupiter.api.Assertions.fail;
+import static org.junit.jupiter.api.Assertions.assertThrows;
import java.nio.ByteBuffer;
import java.util.HashMap;
@@ -63,12 +63,8 @@ public class ProxyServerTest {
EasyMock.replay(server, writer, mre);
-try {
- server.updateAndFlush(login, tableName, cells);
- fail("Expected updateAndFlush to throw an exception");
-} catch (org.apache.accumulo.proxy.thrift.MutationsRejectedException e) {
- // pass
-}
+
assertThrows(org.apache.accumulo.proxy.thrift.MutationsRejectedException.class,
+() -> server.updateAndFlush(login, tableName, cells));
EasyMock.verify(server, writer, mre);
}
@@ -102,12 +98,8 @@ public class ProxyServerTest {
EasyMock.replay(server, writer, mre);
-try {
- server.updateAndFlush(login, tableName, cells);
- fail("Expected updateAndFlush to throw an exception");
-} catch (org.apache.accumulo.proxy.thrift.MutationsRejectedException e) {
- // pass
-}
+
assertThrows(org.apache.accumulo.proxy.thrift.MutationsRejectedException.class,
+() -> server.updateAndFlush(login, tableName, cells));
EasyMock.verify(server, writer, mre);
}
diff --git a/src/test/java/org/apache/accumulo/proxy/its/SimpleProxyBase.java
b/src/test/java/org/apache/accumulo/proxy/its/SimpleProxyBase.java
index b7ddea8..85a55ea 100644
--- a/src/test/java/org/apache/accumulo/proxy/its/SimpleProxyBase.java
+++ b/src/test/java/org/apache/accumulo/proxy/its/SimpleProxyBase.java
@@ -29,6 +29,7 @@ import static org.junit.jupiter.api.Assertions.fail;
import java.io.BufferedReader;
import java.io.File;
+import java.io.IOException;
import java.io.InputStreamReader;
import java.net.InetAddress;
import java.nio.ByteBuffer;
@@ -45,6 +46,7 @@ import java.util.Set;
import java.util.TreeMap;
import java.util.UUID;
import java.util.concurrent.TimeUnit;
+import java.util.stream.Stream;
import org.apache.accumulo.cluster.ClusterUser;
import org.apache.accumulo.core.client.Accumulo;
@@ -137,6 +139,7 @@ import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInfo;
import org.junit.jupiter.api.Timeout;
+import org.junit.jupiter.api.function.Executable;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -158,13 +161,13 @@ public abstract class SimpleProxyBase extends
SharedMiniClusterBase {
private static int proxyPort;
private TestProxyClient proxyClient;
- private org.apache.accumulo.proxy.thrift.AccumuloProxy.Client client;
+ private static org.apache.accumulo.proxy.thrift.AccumuloProxy.Client client;
private static Map properties = new HashMap<>();
private static String hostname, proxyPrincipal, proxyPrimary,
clientPrincipal;
private static File proxyKeytab, clientKeytab;
- private ByteBuffer creds = null;
+ private static ByteBuffer creds = null;
// Implementations can set this
static TProtocolFactory factory = null;
@@ -397,7 +400,7 @@ public abstract class SimpleProxyBase extends
SharedMiniClusterBase {
@Test
@Timeout(5)
- public void compactTableLoginFailure() throws Exception {
+ public void compactTableLoginFailure() {
assertThrows(AccumuloSecurityException.class,
() -> client.compactTable(badLogin, tableName, null, null, null, true,
false, null));
}
@@ -424,7 +427,7 @@ public abstract class SimpleProxyBase extends
SharedMiniClusterBase {
@Test
@Timeout(5)
- public void deleteRowsLoginFailure() throws Exception {
+ public void deleteRowsLoginFailure() {
assertThrows(AccumuloSecurityException.class,
() -> client.deleteRows(badLogin, tableName, null, null));
}
@@ -451,7 +454,7 @@ public abstract class SimpleProxyBase extends
SharedMiniClusterBase {
@Test
@Timeout(5)
-
[accumulo] branch 2.1 updated: Call setDropBehind() when reading/writing WAL files (#3077)
This is an automated email from the ASF dual-hosted git repository.
dlmarion pushed a commit to branch 2.1
in repository https://gitbox.apache.org/repos/asf/accumulo.git
The following commit(s) were added to refs/heads/2.1 by this push:
new 2931a9b931 Call setDropBehind() when reading/writing WAL files (#3077)
2931a9b931 is described below
commit 2931a9b93172a1acaad5cace13e77c452f2bfe68
Author: Dave Marion
AuthorDate: Tue Nov 15 07:56:39 2022 -0500
Call setDropBehind() when reading/writing WAL files (#3077)
WAL files are not read after being written unless there is a
failure that causes recovery. We can call setDropBehind on the
DFSInputStream and DFSOutputStream to inform the DataNodes that
they don't have to cache the WAL information after the file has
been read or written.
---
.../src/main/java/org/apache/accumulo/tserver/log/DfsLogger.java | 7 +++
.../src/main/java/org/apache/accumulo/tserver/log/LogSorter.java | 8
2 files changed, 15 insertions(+)
diff --git
a/server/tserver/src/main/java/org/apache/accumulo/tserver/log/DfsLogger.java
b/server/tserver/src/main/java/org/apache/accumulo/tserver/log/DfsLogger.java
index c6febc4eb8..b24387050b 100644
---
a/server/tserver/src/main/java/org/apache/accumulo/tserver/log/DfsLogger.java
+++
b/server/tserver/src/main/java/org/apache/accumulo/tserver/log/DfsLogger.java
@@ -417,6 +417,13 @@ public class DfsLogger implements Comparable {
else
logFile = fs.create(logfilePath, true, 0, replication, blockSize);
+ // Tell the DataNode that the write ahead log does not need to be cached
in the OS page cache
+ try {
+logFile.setDropBehind(Boolean.TRUE);
+ } catch (IOException | UnsupportedOperationException e) {
+log.debug("setDropBehind writes not enabled for wal file: {}",
logFile);
+ }
+
// check again that logfile can be sync'd
if (!fs.canSyncAndFlush(logfilePath)) {
log.warn("sync not supported for log file {}. Data loss may occur.",
logPath);
diff --git
a/server/tserver/src/main/java/org/apache/accumulo/tserver/log/LogSorter.java
b/server/tserver/src/main/java/org/apache/accumulo/tserver/log/LogSorter.java
index bcec30621a..6df7729766 100644
---
a/server/tserver/src/main/java/org/apache/accumulo/tserver/log/LogSorter.java
+++
b/server/tserver/src/main/java/org/apache/accumulo/tserver/log/LogSorter.java
@@ -142,6 +142,14 @@ public class LogSorter {
fs.deleteRecursively(new Path(destPath));
input = fs.open(srcPath);
+
+ // Tell the DataNode that the write ahead log does not need to be cached
in the OS page cache
+ try {
+input.setDropBehind(Boolean.TRUE);
+ } catch (IOException | UnsupportedOperationException e) {
+log.debug("setDropBehind reads not enabled for wal file: {}", input);
+ }
+
try {
decryptingInput = DfsLogger.getDecryptingStream(input, cryptoService);
} catch (LogHeaderIncompleteException e) {
