dpol1 commented on code in PR #2175: URL: https://github.com/apache/stormcrawler/pull/2175#discussion_r4080082618
########## external/urlfrontier/README.md: ########## @@ -24,6 +24,30 @@ urlfrontier.max.buckets: 10 urlfrontier.max.urls.per.bucket:10 ``` +## Transport security + +The gRPC channels to the frontier are plaintext unless TLS is enabled. Plaintext is kept as the +default so that existing deployments keep working. The channel carries the URLs and their metadata, so enable TLS whenever the Review Comment: can you add what you said above? URLFrontier 2.6 needs a TLS proxy in front of it, until crawler-commons/url-frontier#219 is released ########## external/urlfrontier/README.md: ########## @@ -24,6 +24,30 @@ urlfrontier.max.buckets: 10 urlfrontier.max.urls.per.bucket:10 ``` +## Transport security + +The gRPC channels to the frontier are plaintext unless TLS is enabled. Plaintext is kept as the +default so that existing deployments keep working. The channel carries the URLs and their metadata, so enable TLS whenever the +frontier runs on another host: + +```yaml +urlfrontier.tls.enabled: true +# PEM file with the certificates trusted to sign the server certificate; +# the JVM trust store is used if not set +urlfrontier.tls.trust.cert.collection: /etc/stormcrawler/frontier-ca.pem +# client certificate and PKCS#8 private key for mutual TLS, both or neither +urlfrontier.tls.client.cert.chain: /etc/stormcrawler/crawler.pem +urlfrontier.tls.client.private.key: /etc/stormcrawler/crawler-key.pem +# only needed if the private key is encrypted +urlfrontier.tls.client.private.key.password: changeit +``` + +The server certificate must be valid for the host name in `urlfrontier.address` or +`urlfrontier.host`. Setting only one of `urlfrontier.tls.client.cert.chain` and +`urlfrontier.tls.client.private.key`, or pointing a key at a file which cannot be read, fails +the component at startup. The settings apply to `Spout`, `StatusUpdaterBolt` and Review Comment: a failed handshake doesn't fail the startup: `Spout` and `StatusUpdaterBolt` use `withWaitForReady()` without a deadline and just wait. worth a line here -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
