GGraziadei opened a new issue, #2130: URL: https://github.com/apache/stormcrawler/issues/2130
`MetadataTransfer.filter()` is called for every outlink. For each wildcard key (e.g. `cookie.*`) it calls `Metadata.keySet(prefix)`, which streams over all the metadata keys and collects a new `Set`, then copies each matching key through `getValues()`/`setValues()` with key normalisation on both sides. That is one intermediate set per wildcard per outlink, plus repeated key normalisation, on the hottest path of the parser bolts. Proposal: pre-compile the configured keys into exact keys and wildcard prefixes once, and copy the matching entries in a single pass over the metadata map. Micro-benchmark on a 12-key metadata with 3 wildcards and 2 exact keys: ~780 ns to ~265 ns per outlink. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
