ChrisSamo632 commented on PR #9606: URL: https://github.com/apache/nifi/pull/9606#issuecomment-2781014105
@exceptionfactory thanks for the thoughtful discussion. I understand the reticence around the change to the Elasticsearch processors and components within nifi of there's uncertainty of the level of benefit. Along with the potential use for per-request user credentials such as [JWTs](https://www.elastic.co/guide/en/elasticsearch/reference/current/jwt-auth-realm.html#hmac-oidc-example-request-headers) (which do indeed come with the potential for security concerns, but as noted similar concerns exist elsewhere with other processors), Elasticsearch has a few general request headers that can be useful for audit and debug logging, along with functional use: * [content type](https://www.elastic.co/guide/en/elasticsearch/reference/current/api-conventions.html#_content_type_requirements) * although this isn't likely to be overly useful for nifi use cases, as the flowfile content type isn't likely to change within the flow between requests, and even if it did, all of the Elasticsearch components currently handle only (nd)json * [request origin tracking](https://www.elastic.co/guide/en/elasticsearch/reference/current/api-conventions.html#x-opaque-id) * could be used for general observability between nifi & elasticsearch * [request tracing](https://www.elastic.co/guide/en/elasticsearch/reference/current/api-conventions.html#traceparent) * could be used for application logging between components and, potentially, request auditing The use case of nifi as an intermediary webserver between a client and elasticsearch where the operations are intended to be authenticate with the end user's identity are probably somewhat niche, but I've run into such environments a few times over the years, hence the proposed enhancement. Probably more useful are the cross-system tracing capabilities for the increasingly relevant and popular observability. Use of these headers could be implemented a different way, but this generic approach would allow for each of these use cases, along with catering for environments that have a proxy in between system components potentially requiring additional request headers (although again, those setups aren't likely to need per-request differences in the headers for the most part). There are likely some Elasticsearch endpoints (or equivalent provider systems that follow the same general endpoints as those provided by Elastic) that would make use of additional request headers, but I don't have any available to show here (I'm aware of some such systems I've used in the past, but not with nifi, and not that are generally available). One thing included in this PR was a refactor of how request parameters are handled between the Elasticsearch components within nifi, making such changes simpler in the future. That said, in itself, it probably doesn't warrant the change if the functional benefits aren't thought sufficient. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
