Andrew Kyle Purtell created HBASE-27088:
-------------------------------------------
Summary: IntegrationLoadTestCommonCrawl async load improvements
Key: HBASE-27088
URL: https://issues.apache.org/jira/browse/HBASE-27088
Project: HBase
Issue Type: Improvement
Components: integration tests, test
Reporter: Andrew Kyle Purtell
Assignee: Andrew Kyle Purtell
Fix For: 2.5.0, 3.0.0-alpha-3
ITLCC improvements:
- Use an async client and work stealing executor for parallelism during loads.
- Remove the verification read retries, these are not that effective during
replication lag anyway.
- Increase max task attempts because S3 might throttle.
- Implement a side task that exercises Increments by extracting urls from
content and updating a cf that tracks referrer counts. These are not validated
at this time. It could be possible to log the increments, sum them with a
reducer, and then verify the total, but this is left as a future exercise.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
