[accumulo-website] branch asf-site updated (df2e3a1 -> db1693e)
This is an automated email from the ASF dual-hosted git repository. ctubbsii pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/accumulo-website.git. from df2e3a1 Automatic Site Publish by Buildbot add 114687f Automatic Site Publish by Buildbot add db1693e Automatic Site Publish by Buildbot No new revisions were added by this update. Summary of changes: output/docs/2.x/troubleshooting/tools.html | 10 -- output/feed.xml| 4 ++-- output/search_data.json| 2 +- 3 files changed, 11 insertions(+), 5 deletions(-)
[accumulo-website] branch asf-staging updated: Automatic Site Publish by Buildbot
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-staging
in repository https://gitbox.apache.org/repos/asf/accumulo-website.git
The following commit(s) were added to refs/heads/asf-staging by this push:
new db1693e Automatic Site Publish by Buildbot
db1693e is described below
commit db1693e128eaff60d43ba941d0025461e9d4f873
Author: buildbot
AuthorDate: Wed Jul 7 19:01:48 2021 +
Automatic Site Publish by Buildbot
---
output/docs/2.x/troubleshooting/tools.html | 15 ---
output/feed.xml| 4 ++--
output/search_data.json| 2 +-
3 files changed, 11 insertions(+), 10 deletions(-)
diff --git a/output/docs/2.x/troubleshooting/tools.html
b/output/docs/2.x/troubleshooting/tools.html
index cc85cad..d20eb93 100644
--- a/output/docs/2.x/troubleshooting/tools.html
+++ b/output/docs/2.x/troubleshooting/tools.html
@@ -532,13 +532,14 @@ every tablet matches, and the start and stop for the
table is empty:
$ accumulo
org.apache.accumulo.server.util.CheckForMetadataProblems -u root --password
Enter the connection password:
-Checking table: accumulo.root
-All is well for table !0
-No problems found
-Checking table: accumulo.metadata
-All is well for table +rep
-All is well for table 1
-No problems found
+Checking tables whose metadata is found in: accumulo.root (+r)
+...All is well for table accumulo.metadata (!0)
+No problems found in accumulo.root (+r)
+
+Checking tables whose metadata is found in: accumulo.metadata (!0)
+...All is well for table accumulo.replication (+rep)
+...All is well for table trace (1)
+No problems found in accumulo.metadata (!0)
RemoveEntriesForMissingFiles
diff --git a/output/feed.xml b/output/feed.xml
index 160cb28..0632150 100644
--- a/output/feed.xml
+++ b/output/feed.xml
@@ -6,8 +6,8 @@
https://accumulo.apache.org/
https://accumulo.apache.org/feed.xml; rel="self"
type="application/rss+xml"/>
-Tue, 15 Jun 2021 14:48:09 +
-Tue, 15 Jun 2021 14:48:09 +
+Wed, 07 Jul 2021 19:01:41 +
+Wed, 07 Jul 2021 19:01:41 +
Jekyll v4.2.0
diff --git a/output/search_data.json b/output/search_data.json
index d0d6464..0fbfbaa 100644
--- a/output/search_data.json
+++ b/output/search_data.json
@@ -289,7 +289,7 @@
"docs-2-x-troubleshooting-tools": {
"title": "Troubleshooting Tools",
- "content" : "The accumulo command can be used to run various
tools and classes from the command line.RFileInfoThe rfile-info tool will
examine an Accumulo storage file and print out basic metadata.$ accumulo
rfile-info /accumulo/tables/1/default_tablet/A00n.rf2013-07-16 08:17:14,778
[util.NativeCodeLoader] INFO : Loaded the native-hadoop libraryLocality group
: lt;DEFAULTgt;Start block : 0Num
blocks : 1Index level 0 [...]
+ "content" : "The accumulo command can be used to run various
tools and classes from the command line.RFileInfoThe rfile-info tool will
examine an Accumulo storage file and print out basic metadata.$ accumulo
rfile-info /accumulo/tables/1/default_tablet/A00n.rf2013-07-16 08:17:14,778
[util.NativeCodeLoader] INFO : Loaded the native-hadoop libraryLocality group
: lt;DEFAULTgt;Start block : 0Num
blocks : 1Index level 0 [...]
"url": " /docs/2.x/troubleshooting/tools",
"categories": "troubleshooting"
},
[accumulo-website] branch main updated: Update tools.md for new CheckForMetadataProblems output (#287)
This is an automated email from the ASF dual-hosted git repository. ctubbsii pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/accumulo-website.git The following commit(s) were added to refs/heads/main by this push: new 2ca6f78 Update tools.md for new CheckForMetadataProblems output (#287) 2ca6f78 is described below commit 2ca6f78ac38ad34221da3fbf29ee20ccf6cb275d Author: Luke Foster <84727868+foste...@users.noreply.github.com> AuthorDate: Wed Jul 7 15:01:20 2021 -0400 Update tools.md for new CheckForMetadataProblems output (#287) --- _docs-2/troubleshooting/tools.md | 15 --- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/_docs-2/troubleshooting/tools.md b/_docs-2/troubleshooting/tools.md index 7083c21..e069793 100644 --- a/_docs-2/troubleshooting/tools.md +++ b/_docs-2/troubleshooting/tools.md @@ -102,13 +102,14 @@ every tablet matches, and the start and stop for the table is empty: $ accumulo org.apache.accumulo.server.util.CheckForMetadataProblems -u root --password Enter the connection password: -Checking table: accumulo.root -All is well for table !0 -No problems found -Checking table: accumulo.metadata -All is well for table +rep -All is well for table 1 -No problems found +Checking tables whose metadata is found in: accumulo.root (+r) +...All is well for table accumulo.metadata (!0) +No problems found in accumulo.root (+r) + +Checking tables whose metadata is found in: accumulo.metadata (!0) +...All is well for table accumulo.replication (+rep) +...All is well for table trace (1) +No problems found in accumulo.metadata (!0) ## RemoveEntriesForMissingFiles
[accumulo] branch main updated: Refactor common table property validation to reduce duplication (#2186)
This is an automated email from the ASF dual-hosted git repository.
edcoleman pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/accumulo.git
The following commit(s) were added to refs/heads/main by this push:
new df7e49e Refactor common table property validation to reduce
duplication (#2186)
df7e49e is described below
commit df7e49e771683f73d228ca02af0c40f475ac90e0
Author: EdColeman
AuthorDate: Wed Jul 7 14:10:56 2021 -0400
Refactor common table property validation to reduce duplication (#2186)
* Refactor common table property validation to reduce duplication
- Move duplcated code for table property validation to one place
- Minor check-style fixes.
- fix possible nulll pointer warning (FateServiceHandler)
---
.../org/apache/accumulo/core/conf/Property.java| 44 +-
.../accumulo/server/util/NamespacePropUtil.java| 8 +---
.../apache/accumulo/server/util/TablePropUtil.java | 8 +---
.../accumulo/manager/FateServiceHandler.java | 8 ++--
4 files changed, 33 insertions(+), 35 deletions(-)
diff --git a/core/src/main/java/org/apache/accumulo/core/conf/Property.java
b/core/src/main/java/org/apache/accumulo/core/conf/Property.java
index 74f22a5..3093990 100644
--- a/core/src/main/java/org/apache/accumulo/core/conf/Property.java
+++ b/core/src/main/java/org/apache/accumulo/core/conf/Property.java
@@ -1171,15 +1171,15 @@ public enum Property {
"This property is deprecated since 2.0.0, use
tserver.scan.executors.meta.threads instead. "
+ "The maximum number of concurrent metadata read ahead that will
execute.");
- private String key;
- private String defaultValue;
- private String description;
+ private final String key;
+ private final String defaultValue;
+ private final String description;
private boolean annotationsComputed = false;
private boolean isSensitive;
private boolean isDeprecated;
private boolean isExperimental;
private Property replacedBy = null;
- private PropertyType type;
+ private final PropertyType type;
Property(String name, String defaultValue, PropertyType type, String
description) {
this.key = name;
@@ -1352,10 +1352,10 @@ public enum Property {
return false;
}
- private static HashSet validTableProperties = null;
- private static HashSet validProperties = null;
- private static HashSet validPrefixes = null;
- private static HashMap propertiesByKey = null;
+ private static final HashSet validTableProperties = new HashSet<>();
+ private static final HashSet validProperties = new HashSet<>();
+ private static final HashSet validPrefixes = new HashSet<>();
+ private static final HashMap propertiesByKey = new
HashMap<>();
private static boolean isKeyValidlyPrefixed(String key) {
for (String prefix : validPrefixes) {
@@ -1368,6 +1368,22 @@ public enum Property {
}
/**
+ * Checks if the given property and value are valid. A property is valid if
the property key is
+ * valid see {@link #isValidTablePropertyKey} and that the value is a valid
format for the type
+ * see {@link PropertyType#isValidFormat}.
+ *
+ * @param key
+ * property key
+ * @param value
+ * property value
+ * @return true if key is valid (recognized, or has a recognized prefix)
+ */
+ public static boolean isTablePropertyValid(final String key, final String
value) {
+Property p = getPropertyByKey(key);
+return (p == null || p.getType().isValidFormat(value)) &&
isValidTablePropertyKey(key);
+ }
+
+ /**
* Checks if the given property key is valid. A valid property key is either
equal to the key of
* some defined property or has a prefix matching some prefix defined in
this class.
*
@@ -1528,21 +1544,15 @@ public enum Property {
// Precomputing information here avoids :
// * Computing it each time a method is called
// * Using synch to compute the first time a method is called
-propertiesByKey = new HashMap<>();
-validPrefixes = new HashSet<>();
-validProperties = new HashSet<>();
-
for (Property p : Property.values()) {
+ propertiesByKey.put(p.getKey(), p);
if (p.getType().equals(PropertyType.PREFIX)) {
validPrefixes.add(p.getKey());
} else {
validProperties.add(p.getKey());
}
- propertiesByKey.put(p.getKey(), p);
-}
-
-validTableProperties = new HashSet<>();
-for (Property p : Property.values()) {
+ // exclude prefix types (prevents setting a prefix type like
table.custom or
+ // table.constraint, directly, since they aren't valid properties on
their own)
if (!p.getType().equals(PropertyType.PREFIX)
&& p.getKey().startsWith(Property.TABLE_PREFIX.getKey())) {
validTableProperties.add(p.getKey());
diff --git
a/server/base/src/main/java/org/apache/accumulo/server/util/NamespacePropUtil.java
[accumulo] branch main updated (908105b -> 7bbe579)
This is an automated email from the ASF dual-hosted git repository. jmanno pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/accumulo.git. from 908105b Refactor LogSorter to use config (#2191) add 7bbe579 Update CheckForMetadataProblems Utility Output (#2180) No new revisions were added by this update. Summary of changes: .../server/util/CheckForMetadataProblems.java | 48 ++ 1 file changed, 31 insertions(+), 17 deletions(-)
[accumulo] branch main updated: Refactor LogSorter to use config (#2191)
This is an automated email from the ASF dual-hosted git repository.
mmiller pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/accumulo.git
The following commit(s) were added to refs/heads/main by this push:
new 908105b Refactor LogSorter to use config (#2191)
908105b is described below
commit 908105bc8fddd64aa2d5907c22ccf2a232209306
Author: Mike Miller
AuthorDate: Wed Jul 7 07:29:44 2021 -0400
Refactor LogSorter to use config (#2191)
* Refactor LogSorter writeBuffer to allow use of system configuration
when writing out sorted rfiles. This will use configured settings on the
sorted files instead of only the defaults.
---
.../java/org/apache/accumulo/tserver/log/LogSorter.java | 13 ++---
.../apache/accumulo/tserver/log/SortedLogRecoveryTest.java | 6 --
2 files changed, 10 insertions(+), 9 deletions(-)
diff --git
a/server/tserver/src/main/java/org/apache/accumulo/tserver/log/LogSorter.java
b/server/tserver/src/main/java/org/apache/accumulo/tserver/log/LogSorter.java
index 293ff82..bd299af 100644
---
a/server/tserver/src/main/java/org/apache/accumulo/tserver/log/LogSorter.java
+++
b/server/tserver/src/main/java/org/apache/accumulo/tserver/log/LogSorter.java
@@ -32,7 +32,6 @@ import java.util.concurrent.ThreadPoolExecutor;
import org.apache.accumulo.core.Constants;
import org.apache.accumulo.core.conf.AccumuloConfiguration;
-import org.apache.accumulo.core.conf.DefaultConfiguration;
import org.apache.accumulo.core.conf.Property;
import org.apache.accumulo.core.data.Key;
import org.apache.accumulo.core.data.Mutation;
@@ -146,7 +145,7 @@ public class LogSorter {
// Creating a 'finished' marker will cause recovery to proceed
normally and the
// empty file will be correctly ignored downstream.
fs.mkdirs(new Path(destPath));
-writeBuffer(context, destPath, Collections.emptyList(), part++);
+writeBuffer(destPath, Collections.emptyList(), part++);
fs.create(SortedLogState.getFinishedMarkerPath(destPath)).close();
return;
}
@@ -164,10 +163,10 @@ public class LogSorter {
value.readFields(decryptingInput);
buffer.add(new Pair<>(key, value));
}
- writeBuffer(context, destPath, buffer, part++);
+ writeBuffer(destPath, buffer, part++);
buffer.clear();
} catch (EOFException ex) {
- writeBuffer(context, destPath, buffer, part++);
+ writeBuffer(destPath, buffer, part++);
break;
}
}
@@ -215,8 +214,8 @@ public class LogSorter {
}
@VisibleForTesting
- public static void writeBuffer(ServerContext context, String destPath,
- List> buffer, int part) throws IOException
{
+ void writeBuffer(String destPath, List>
buffer, int part)
+ throws IOException {
String filename = String.format("part-r-%05d.rf", part);
Path path = new Path(destPath, filename);
FileSystem fs = context.getVolumeManager().getFileSystemByPath(path);
@@ -238,7 +237,7 @@ public class LogSorter {
try (var writer = FileOperations.getInstance().newWriterBuilder()
.forFile(fullPath.toString(), fs, fs.getConf(),
context.getCryptoService())
-.withTableConfiguration(DefaultConfiguration.getInstance()).build()) {
+.withTableConfiguration(conf).build()) {
writer.startDefaultLocalityGroup();
for (var entry : keyListMap.entrySet()) {
LogFileValue val = new LogFileValue();
diff --git
a/server/tserver/src/test/java/org/apache/accumulo/tserver/log/SortedLogRecoveryTest.java
b/server/tserver/src/test/java/org/apache/accumulo/tserver/log/SortedLogRecoveryTest.java
index 2121725..582b9bc 100644
---
a/server/tserver/src/test/java/org/apache/accumulo/tserver/log/SortedLogRecoveryTest.java
+++
b/server/tserver/src/test/java/org/apache/accumulo/tserver/log/SortedLogRecoveryTest.java
@@ -79,6 +79,7 @@ public class SortedLogRecoveryTest {
static final Text cq = new Text("cq");
static final Value value = new Value("value");
static ServerContext context;
+ static LogSorter logSorter;
@Rule
public TemporaryFolder tempFolder =
@@ -87,6 +88,7 @@ public class SortedLogRecoveryTest {
@Before
public void setup() {
context = EasyMock.createMock(ServerContext.class);
+logSorter = new LogSorter(context, DefaultConfiguration.getInstance());
}
static class KeyValue implements Comparable {
@@ -179,11 +181,11 @@ public class SortedLogRecoveryTest {
for (KeyValue pair : entry.getValue()) {
buffer.add(new Pair<>(pair.key, pair.value));
if (buffer.size() >= bufferSize) {
-LogSorter.writeBuffer(context, destPath, buffer, parts++);
+logSorter.writeBuffer(destPath, buffer, parts++);
buffer.clear();
}
}
-LogSorter.writeBuffer(context, destPath, buffer, parts);
+
