milleruntime edited a comment on issue #2179:
URL: https://github.com/apache/accumulo/issues/2179#issuecomment-867763969
I tested an upgrade in Uno from 2.0.1 to 2.1.0-SNAPSHOT with some data and
sorted WAL files. I was surprised to not see any errors trying to recover the
old map files. I noticed that the GC deleted the files rather quickly after the
Upgrader was finished.
<pre>
11:04:24 {main} ~/workspace/uno$ hdfs dfs -ls -R /accumulo/recovery
drwxr-xr-x - mike supergroup 0 2021-06-24 11:04
/accumulo/recovery/980ce402-07dc-4a09-b6ac-e26664db29a3
drwxr-xr-x - mike supergroup 0 2021-06-24 11:04
/accumulo/recovery/980ce402-07dc-4a09-b6ac-e26664db29a3/part-r-00000
-rw-r--r-- 3 mike supergroup 366452 2021-06-24 11:04
/accumulo/recovery/980ce402-07dc-4a09-b6ac-e26664db29a3/part-r-00000/data
-rw-r--r-- 3 mike supergroup 255 2021-06-24 11:04
/accumulo/recovery/980ce402-07dc-4a09-b6ac-e26664db29a3/part-r-00000/index
drwxr-xr-x - mike supergroup 0 2021-06-24 11:04
/accumulo/recovery/980ce402-07dc-4a09-b6ac-e26664db29a3/part-r-00001
-rw-r--r-- 3 mike supergroup 366592 2021-06-24 11:04
/accumulo/recovery/980ce402-07dc-4a09-b6ac-e26664db29a3/part-r-00001/data
-rw-r--r-- 3 mike supergroup 224 2021-06-24 11:04
/accumulo/recovery/980ce402-07dc-4a09-b6ac-e26664db29a3/part-r-00001/index
drwxr-xr-x - mike supergroup 0 2021-06-24 11:04
/accumulo/recovery/980ce402-07dc-4a09-b6ac-e26664db29a3/part-r-00002
-rw-r--r-- 3 mike supergroup 366727 2021-06-24 11:04
/accumulo/recovery/980ce402-07dc-4a09-b6ac-e26664db29a3/part-r-00002/data
-rw-r--r-- 3 mike supergroup 224 2021-06-24 11:04
/accumulo/recovery/980ce402-07dc-4a09-b6ac-e26664db29a3/part-r-00002/index
11:49:32 {main} ~/workspace/uno/install/logs/accumulo$ grep
980ce402-07dc-4a09-b6ac-e26664db29a3 *
gc_ip-10-113-12-25.log:2021-06-24T11:08:05,957
[gc.GarbageCollectWriteAheadLogs] DEBUG: Removing CLOSED WAL
hdfs://localhost:8020/accumulo/wal/ip-10-113-12-25+9997/980ce402-07dc-4a09-b6ac-e26664db29a3
gc_ip-10-113-12-25.log:2021-06-24T11:08:05,965
[gc.GarbageCollectWriteAheadLogs] DEBUG: Removing recovery log
hdfs://localhost:8020/accumulo/recovery/980ce402-07dc-4a09-b6ac-e26664db29a3
gc_ip-10-113-12-25.log:2021-06-24T11:08:05,979 [log.WalStateManager] DEBUG:
Removing 980ce402-07dc-4a09-b6ac-e26664db29a3
</pre>
This could have been because the sorted files were no longer referenced. I
am going to try the test again but use CI to make sure there wasn't data any
data loss.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
