abhinav-phi commented on PR #2122: URL: https://github.com/apache/stormcrawler/pull/2122#issuecomment-5583400498
Thanks — pushed 77a75c29 addressing the partitioner bypass: - `URLPartitionerBolt` and the `URLPartitioner` used by the URL buffer now derive host, domain and IP keys from `URLUtil.getCanonicalHost` in every mode. An explicitly supplied `ip` metadata value is used unchanged, as you asked. - The bolt's DNS-resolution cache is keyed on the canonical host too, so aliases share cache entries. - Regression coverage: `URLPartitionerTest` pins all three modes for the percent-escaped, trailing-dot and uppercase spellings plus the provided-IP preservation; `URLPartitionerBoltTest` covers the partitioner-to-fetcher path end to end (the emitted `key` field the fetcher's `FetchItem.create` consumes) and asserts the URL itself is emitted unchanged. Local gates: checkstyle clean on the full reactor, format validation clean, all affected suites pass. The workflow run for 77a75c29 will need a maintainer's workflow approval again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
