keith-turner opened a new issue, #3358:
URL: https://github.com/apache/accumulo/issues/3358
This is follow on work for #3350. In #3350 a new refresh column was added to
indicate a hosted tablet needs to reload its metadata. The bulk import fate
code (and probably compactions in the future) only set the refresh column
(using a conditional mutation) if the tablet has current of future location set.
When a tablet loads it currently reads its metadata and then sets the
location. This order can lead to race conditions where the bulk import code
does not set a refresh column and the tablet does not read the files the bulk
import code wrote. If the order is reversed and the bulk makes updates with
conditional mutations that check the tablet location, then that will avoid race
conditions.
For example consider the following scenarios.
1. Bulk code reads metadata for table T1 and sees it has no location
2. Tablet T1 sets its location in the metadata table
3. Bulk code attempts to add files to tablet T1 with a condition that the
location is absent. This fails because the location is not absent
4. Tablet T1 reads it metadata and gets its set of files
5. The bulk code reread metadata for tablet T1 and see the location L1
6. The bulk code attempts to add files to the tablet T1 and a refresh
column with a condition that the tablet location is L1
7. The bulk sends an async rpc to the tablet asking it to refresh
8. Tablet refreshes its metadata and deletes the refresh column
In the scenario above the tablet was made aware of the new bulk files. If
the order of the tablet settings it location and reading its metadata above
were reversed then the tablet would have been aware of the concurrent bulk
load. Also the bulk load would not have been aware of the concurrent tablet
load.
Reading the metadata after setting the location ensures the following.
* Will see all changes that were made before the location was set
* Will be notified of any changes during and after the location was set
The during part assumes other changes are also using conditional mutations.
When making this changes in the tablet code need to document how important
the order is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
