pan3793 opened a new pull request, #6836:
URL: https://github.com/apache/kyuubi/pull/6836
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines:
https://kyuubi.readthedocs.io/en/master/contributing/code/index.html
2. If the PR is related to an issue in
https://github.com/apache/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title,
e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g.,
'[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug, and what versions
are affected.
-->
I'd like to include `kafka-clients` in the Kyuubi binary distribution
tarball to enable the out-of-box support for sinking Kyuubi events to Kafka.
- Kafka is an important component in modern data platforms, and is a defacto
message queue implementation, especially in the big data domain
- `kafka-clients` is quite a light lib, has no third-party deps except for
`slf4j-api` and a few optional compression libs
- `kafka-clients` uses "none" compression as default, in practice,
"gzip"(delegate to JDK gzip algorithm, no additional libs are required) already
performs well for non-extreme cases
Additionally, LOG4J2 has a built-in `KafkaAppender` that supports sinking
logging to Kafka, which also requires `kafka-clients` in the classpath, I have
some initial ideas to forward both Kyuubi server's and engine bootstrap
processes log to Kafka in a structured format, will send another PR to add docs
to guide users in configuring that.
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some
test cases that check the changes thoroughly including negative and positive
cases if possible.
If it was tested in a way different from regular unit tests, please clarify
how you tested step by step, ideally copy and paste-able, so that other
reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why
it was difficult to add.
-->
Review
### Was this patch authored or co-authored using generative AI tooling?
<!--
If a generative AI tooling has been used in the process of authoring this
patch, please include
phrase 'Generated-by: ' followed by the name of the tool and its version.
If no, write 'No'.
Please refer to the [ASF Generative Tooling
Guidance](https://www.apache.org/legal/generative-tooling.html) for details.
-->
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscr...@kyuubi.apache.org
For additional commands, e-mail: notifications-h...@kyuubi.apache.org
