rzo1 commented on PR #2122: URL: https://github.com/apache/stormcrawler/pull/2122#issuecomment-5589497127
> Thanks — pushed [77a75c2](https://github.com/apache/stormcrawler/commit/77a75c29cf55527438555f2f70414cd2e9d8037a) addressing the partitioner bypass: > > * `URLPartitionerBolt` and the `URLPartitioner` used by the URL buffer now derive host, domain and IP keys from `URLUtil.getCanonicalHost` in every mode. An explicitly supplied `ip` metadata value is used unchanged, as you asked. > * The bolt's DNS-resolution cache is keyed on the canonical host too, so aliases share cache entries. > * Regression coverage: `URLPartitionerTest` pins all three modes for the percent-escaped, trailing-dot and uppercase spellings plus the provided-IP preservation; `URLPartitionerBoltTest` covers the partitioner-to-fetcher path end to end (the emitted `key` field the fetcher's `FetchItem.create` consumes) and asserts the URL itself is emitted unchanged. > > Local gates: checkstyle clean on the full reactor, format validation clean, all affected suites pass. The workflow run for [77a75c2](https://github.com/apache/stormcrawler/commit/77a75c29cf55527438555f2f70414cd2e9d8037a) will need a maintainer's workflow approval again. Build fails: ``` [ERROR] Failed to execute goal com.cosium.code:git-code-format-maven-plugin:6.0:validate-code-format (validate-code-format) on project stormcrawler-core: /home/runner/work/stormcrawler/stormcrawler/core/src/test/java/org/apache/stormcrawler/util/URLPartitionerTest.java is not correctly formatted ! -> [Help 1] ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
