Andy Muir created KAFKA-8629: -------------------------------- Summary: Kafka Streams Apps to support small native images through GraalVM Key: KAFKA-8629 URL: https://issues.apache.org/jira/browse/KAFKA-8629 Project: Kafka Issue Type: Improvement Components: streams Affects Versions: 2.3.0 Environment: OSX Linux on Docker Reporter: Andy Muir
I'm investigating using [GraalVM|http://example.com/] to help with reducing docker image size and required resources for a simple Kafka Streams microservice. To this end, I'm looking at running a microservice which: 1) consumes from a Kafka topic (XML) 2) Transforms into JSON 3) Produces to a new Kafka topic. The Kafka Streams app running in the JVM works fine. When I attempt to build it to a GraalVM native image (binary executable which does not require the JVM, hence smaller image size and less resources), I encountered a few [incompatibilities|https://github.com/oracle/graal/blob/master/substratevm/LIMITATIONS.md] with the source code in Kafka. I've implemented a workaround for each of these in a fork (link to follow) to help establish if it is feasible. I don't intend (at this stage) for the changes to be applied to the broker - I'm only after Kafka Streams for now. I'm not sure whether it'd be a good idea for the broker itself to run as a native image! There were 2 issues getting the native image with kafka streams: 1) Some Reflection use cases using MethodHandle 2) Anything JMX To work around these issues, I have: 1) Replaced use of MethodHandle with alternatives 2) Commented out the JMX code in a few places While the first may be sustainable, I'd expect that the 2nd option should be put behind a configuration switch to allow the existing code to be used by default and turning off JMX if configured. *I haven't created a PR for now, as I'd like feedback to decide if it is going to be feasible to carry this forwards.* -- This message was sent by Atlassian JIRA (v7.6.3#76005)