cshannon opened a new issue, #3946: URL: https://github.com/apache/accumulo/issues/3946
As part of investigating #3938 , I also tested deleting a range to see if it was Idempotent. So far after analyzing it and testing it does appear to be idempotent for both 2.1 and also for no-chop merge version which is good. This makes sense because deletion works different than merge because it doesn't copy files to a different tablet, it just fences files in existing tablets or deletes tablets by deleting all data in the tablets and these operations are fine to repeat as on retry it will just either re-fence with the same files or skip over already deleted data so not a problem. In one failure simulation I added some test code and threw an exception [here](https://github.com/apache/accumulo/blob/71f1b0fbc494342b5b659ed607ca67d5339848d1/server/manager/src/main/java/org/apache/accumulo/manager/TabletGroupWatcher.java#L781), after tablets are deleted but before the fate op updates the prev row column and finishes. I set the test to only throw the exception the first time through by using a flag so that on restart of the fate operation it would start over again and continue successfully to see if things were idempotent. When testing various deletion ranges I ran into the following 2 bugs that can occur when the deletion range is infinite (null end row) and we are deleting everything at the end of the table. The bugs apply in 2.1 as well and is not unique to no chop merge. 1. The first issue is we can get a NPE in [this](https://github.com/apache/accumulo/blob/71f1b0fbc494342b5b659ed607ca67d5339848d1/server/manager/src/main/java/org/apache/accumulo/manager/TabletGroupWatcher.java#L798) section on resume. Than occur in some instances when the deletion range goes to the end of the table and we restart (but tablets still exist). Because this is a retry, the first time through had already scanned and removed everything so the `metadataTime` column never gets read so we get an NPE when calling `metadataTime.getType()`. I thnk we could fix this by just simply looking up the time type if it's null by using something like: ``` Optional.ofNullable(metadataTime).map(MetadataTime::getType) .orElse(client.tableOperations().getTimeType(manager.getContext().getTableName(extent.tableId()))); ``` 2. If the delete range includes everything (all rows deleted) then all tablets get removed. If the failure then occurs after deleting all tablets (as I simulated) but before creating the new default tablet then on restart the fate operation will never resume. This is because there will be no tablets that exist so `TabletGroupWatcher` has nothing to process to resume the fate op and it just gets hung. In this scenario I'm not quite sure the best way to handle it, it's definitely going to be an edge case but seems like we'd need to essentially re-create a new default tablet on recovery so it could resume and finish the fate op so it's not stuck forever with no tablets. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
