ctubbsii commented on pull request #1803:
URL: https://github.com/apache/accumulo/pull/1803#issuecomment-755139190
Thanks, @tynyttie ; I spent some time looking into this, and realized that
the "Stream is closed" error messages are a bit of a red herring. I've created
a PR to fix those in #1856 .
Although I still haven't figured out why the changes in this PR are causing
RestartIT to hang, I suspect it has something to do with the fact that it's
doing recovery using a RawLocalFileSystem, which might behave differently than
the default implementation of the local FileSystem.
In addition, I found a deficiency in this PR's code. It does not update the
SimpleGarbageCollector code that looks for recovery files to clean up. The
following patch is my attempt to fix that, but because of the hanging issue of
RestartIT, I'm not sure if it will be sufficient.
<details>
<summary>Click to expand/collapse patch for SimpleGarbageCollector</summary>
```patch
diff --git
a/server/base/src/main/java/org/apache/accumulo/server/ServerConstants.java
b/server/base/src/main/java/org/apache/accumulo/server/ServerConstants.java
index ae6c0da8c7..1e357e5af5 100644
---
a/server/base/src/main/java/org/apache/accumulo/server/ServerConstants.java
+++
b/server/base/src/main/java/org/apache/accumulo/server/ServerConstants.java
@@ -166,8 +166,8 @@ public class ServerConstants {
return prefix(getBaseUris(context), TABLE_DIR);
}
- public static Set<String> getRecoveryDirs(ServerContext context) {
- return prefix(getBaseUris(context), RECOVERY_DIR);
+ public static Set<String> getRecoveryDirPatterns(ServerContext context) {
+ return prefix(getBaseUris(context), RECOVERY_DIR + "-*");
}
public static Path getInstanceIdLocation(Volume v) {
diff --git
a/server/gc/src/main/java/org/apache/accumulo/gc/GarbageCollectWriteAheadLogs.java
b/server/gc/src/main/java/org/apache/accumulo/gc/GarbageCollectWriteAheadLogs.java
index 40cd9d180d..da432854b0 100644
---
a/server/gc/src/main/java/org/apache/accumulo/gc/GarbageCollectWriteAheadLogs.java
+++
b/server/gc/src/main/java/org/apache/accumulo/gc/GarbageCollectWriteAheadLogs.java
@@ -393,10 +393,9 @@ public class GarbageCollectWriteAheadLogs {
protected Map<UUID,Path> getSortedWALogs() throws IOException {
Map<UUID,Path> result = new HashMap<>();
- for (String dir : ServerConstants.getRecoveryDirs(context)) {
- Path recoveryDir = new Path(dir);
- if (fs.exists(recoveryDir)) {
- for (FileStatus status : fs.listStatus(recoveryDir)) {
+ for (String dir : ServerConstants.getRecoveryDirPatterns(context)) {
+ for (FileStatus recoveryDir : fs.globStatus(new Path(dir))) {
+ for (FileStatus status : fs.listStatus(recoveryDir.getPath())) {
try {
UUID logId = path2uuid(status.getPath());
result.put(logId, status.getPath());
```
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
