keith-turner opened a new pull request, #3429:
URL: https://github.com/apache/accumulo/pull/3429
This PR contains my experimentation with making scans pre load ondemand
tablets. Below are some log messages from running BulkSplitOptimizationIT
which creates around 57 splits in an on demand table and scans it. The scan is
faster that it was before this change, but still slower than if all the tablets
were loaded.
```
2023-05-25T20:49:31,910 [clientImpl.ClientTabletCacheImpl] DEBUG: Requesting
2 ondemand tablets to be hosted.
2023-05-25T20:49:37,046 [clientImpl.ClientTabletCacheImpl] DEBUG: Requesting
2 ondemand tablets to be hosted.
2023-05-25T20:49:43,249 [logging.InternalLoggerFactory] DEBUG: Using SLF4J
as the default logging framework
2023-05-25T20:49:43,308 [clientImpl.ClientTabletCacheImpl] DEBUG: Requesting
4 ondemand tablets to be hosted.
2023-05-25T20:49:49,793 [clientImpl.ClientTabletCacheImpl] DEBUG: Requesting
8 ondemand tablets to be hosted.
2023-05-25T20:49:55,886 [clientImpl.ClientTabletCacheImpl] DEBUG: Requesting
11 ondemand tablets to be hosted.
2023-05-25T20:50:02,054 [clientImpl.ClientTabletCacheImpl] DEBUG: Requesting
11 ondemand tablets to be hosted.
2023-05-25T20:50:09,089 [clientImpl.ClientTabletCacheImpl] DEBUG: Requesting
11 ondemand tablets to be hosted.
2023-05-25T20:50:13,316 [clientImpl.ClientTabletCacheImpl] DEBUG: Requesting
9 ondemand tablets to be hosted.
100,000 records read | 2,369 records/sec | 7,900,000 bytes read |
187,168 bytes/sec | 42.208 secs
```
These changes are not ready to commit, just posting in case anyone want to
look over them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
