dpol1 commented on code in PR #2171:
URL: https://github.com/apache/stormcrawler/pull/2171#discussion_r4052759323
##########
core/src/main/java/org/apache/stormcrawler/protocol/ProtocolResponse.java:
##########
@@ -78,6 +82,36 @@ public class ProtocolResponse {
*/
public static final String PROTOCOL_MD_PREFIX_PARAM = "protocol.md.prefix";
+ /** Prefix of the metadata keys holding the timings a protocol reports
about a fetch. */
+ public static final String METRICS_PREFIX = "metrics.";
+
+ /** Lowercased: metadata keys are normalised that way, and so are header
names. */
+ private static final Set<String> RESERVED_METADATA_KEYS =
+ Stream.of(
+ REQUEST_HEADERS_KEY,
+ RESPONSE_HEADERS_KEY,
+ RESPONSE_IP_KEY,
+ REQUEST_TIME_KEY,
+ PROTOCOL_VERSIONS_KEY,
+ CIPHER_SUITE_KEY,
+ TRIMMED_RESPONSE_KEY,
+ TRIMMED_RESPONSE_REASON_KEY,
+ REDIRECTED_TO_KEY)
Review Comment:
`playwright.protocol.end` is another key a protocol writes and the js
rendering guards read through `skipIfMetadataPresent`: with an empty
`protocol.md.prefix`, or the key configured with it, a server can set it
through okhttp. worth reserving too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
