(lemme know if this belongs on the users email list, I'm not sure where API questions fall)
Hi, I'm Ben Osheroff, I wrote Maxwell (http://github.com/zendesk/maxwell) and have been prototyping an engine to do arbitrary denormalizations of Maxwell's CDC events based on the kafka-streams library; the elevator pitch is that you can write SQL joins which the engine compiles down to stream-joins and aggregations and such. Maxwell partitions its stream by mysql database name, which means that to do stream-joins I need to implement the same (custom) partitioning algorithm somewhere in my stream processor. I'd prefer not drop down to the lower level `addSink()` library calls if possible, and I can't figure out how to mix and match the lower level alls with the higher level DSL (map/filter/etc). So I guess I have two questions: 1. Is it somehow possible to add a custom `Sink` to an otherwise high level stream topology? There's no obvious way to retrieve the topology names that I can see. 2. If not, I'd like to make a feature request that the various stream building functions (.to, .through) accept an optional StreamPartitioner. 3. Any other ideas about how to pull this off? Thanks! - Ben Osheroff zendesk.com
signature.asc
Description: PGP signature
