keith-turner commented on code in PR #3447:
URL: https://github.com/apache/accumulo/pull/3447#discussion_r1218413798
##########
server/base/src/main/java/org/apache/accumulo/server/manager/state/TabletManagementScanner.java:
##########
@@ -46,9 +47,11 @@ public class TabletManagementScanner implements
ClosableIterator<TabletManagemen
private final BatchScanner mdScanner;
private final Iterator<Entry<Key,Value>> iter;
private final AtomicBoolean closed = new AtomicBoolean(false);
+ private final ConcurrentLinkedQueue<TabletManagement>
knownTabletModifications;
- TabletManagementScanner(ClientContext context, Range range, CurrentState
state,
- String tableName) {
+ // This constructor is called from TabletStateStore implementations
+ public TabletManagementScanner(ClientContext context, Range range,
CurrentState state,
+ String tableName, ConcurrentLinkedQueue<TabletManagement>
knownTabletModifications) {
Review Comment:
I was not thinking of this in terms of priority, but that is really
interesting (especially for hosting ondemand). I was thinking of this in terms
of "I know these 3 tablets need attention, lets not scan the entire metadata
table to service 3 tablets, but servicing those 3 tablets is not more important
than servicing any other". Was thinking when we focus on only processing the
tablets that need attention that it will speed up the response time for them in
the case where the metadata table is large. So I was thinking of this a bit
differently, but I think both priority and avoiding unneeded scanning of the
entire metadata table are both good goals.
I think only processing the tablets that need attention w/o scanning the
entire metadata table could benefit split, merge, can compaction, and ondemand
hosting. The benefits for compaction and merge would be w/ future changes to
elasticity. I'll look through the usage of EventCoordinator to see what else
may or may not benefit. I think all operations could benefit from avoiding
scanning the entire metadata table.
I think hosting ondemand tablet, user initiated splits, merges and user
initiated compaction could benefit from prioritizing processing them over
system initiated splits and compactions. Prioritizing processing of those user
initiated operations could make things more responsive to a user.
Could we achieve the following goals in the manager?
* Tablets needing attention on behalf of user initiated operations will
take priority over a metadata scan in TGW if its in progress. This could make
user initiated operations more responsive (and may be we don't have to increase
polling frequency in ITs).
* When the tablets are known by the manager that need attention, we only
process those tablets in TGW and do not initiate a scan of the entire metadata
table.
Something else I am wondering is if achieving these goals is possible if we
do need multiple managers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
