Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

Submitted the PR .. https://github.com/apache/kafka/pull/4756 and updated the KIP https://cwiki.apache.org/confluence/display/KAFKA/KIP-270+-+A+Scala+Wrapper+Library+for+Kafka+Streams regards. On Tue, Mar 20, 2018 at 4:06 AM, Ismael Juma wrote: > Matthias, > > You have to

Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

Matthias, You have to choose between the package and class names when it comes to adding a suffix (or prefix). I think the package name is the lesser of two evils, but it would be interesting to know what others think. I agree that we should include the information in the KIP (whatever we

Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

About the package name. Would it be better/cleaner to omit the `scala` sub-package? Or is this required to avoid naming conflicts with the Java classes? If yes, please point it out in the KIP. -Matthias On 3/19/18 11:24 AM, Debasish Ghosh wrote: > Cool .. so the changes that u suggested e.g

Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

Cool .. so the changes that u suggested e.g the WordCount example, package name changes etc. will be in the PR only and not in the KIP document .. Or we keep the KIP document updated as well ? regards. On Mon, 19 Mar 2018 at 11:46 PM, Guozhang Wang wrote: > Hi Debasish, >

Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

Hi Debasish, You can work on the PR in parallel to the KIP discussion so that people can start reviewing it. The only restriction is that the PR cannot be merged until the KIP is accepted. Guozhang On Mon, Mar 19, 2018 at 11:09 AM, Debasish Ghosh < debasish.gh...@lightbend.com> wrote: >

Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

Based on this discussion I have a question .. For the actual implementation we need to have a PR on https://github.com/apache/kafka and the new library will be in the package structure org.apache.kafka.streams.scala. My question is, is this PR part of the KIP ? Or we work on the PR once the KIP

Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

Thanks Guozhang for the comments .. we will start working on them .. regards. On Mon, Mar 19, 2018 at 11:02 PM, Guozhang Wang wrote: > And one more comment about the type safety: > > 7. I'm in favor of the approach of enforcing type safety at compile time, > since with

Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

And one more comment about the type safety: 7. I'm in favor of the approach of enforcing type safety at compile time, since with Scala's strong typing system I think it makes more sense to get rid of config-based serde specifications that can cause runtime error in the Scala wrapper. Guozhang

Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

Hello Debasish, Thanks for the KIP. Here are some comments: 1. About the naming: I'd also vote for option 2 and enforce users to rename when import since this scenario seems rare to me. 2. About the dependency: since this KIP can only be merged as early as 1.2, it means that for users who wants

Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

I agree with Sean on name unification. +1 to option 2. On Mon, Mar 19, 2018 at 7:23 AM, Sean Glover wrote: > Type names: I vote for option 2. The user must explicitly add a dependency > to this library and the wrapper types are in a different package. It seems >

Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

Type names: I vote for option 2. The user must explicitly add a dependency to this library and the wrapper types are in a different package. It seems reasonable to expect them to do an import rename if there's a need to drop down to the Java API. Test Utils: The test utils in

Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

> > > Should this be 1.2 (maybe it's even better to not put any version at > all) Actually wanted to emphasize that the support is from 1.0.0 onwards .. Should we make that explicit ? Like .. kafka-streams-scala only depends on the Scala standard library and Kafka > Streams 1.0.0+. In 1.1

Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

Thanks a lot for the KIP! Two questions: 1) the KIP states: > kafka-streams-scala only depends on the Scala standard library and Kafka > Streams 1.0.0. Should this be 1.2 (maybe it's even better to not put any version at all) 2) In 1.1 release, we add a new module `kafka-streams-test-utils`

Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

Import renames seem to be fine. The class names with trailing 'S' look clean. Cheers On Fri, Mar 16, 2018 at 11:04 AM, Ismael Juma wrote: > If this is rare (as it sounds), relying on import renames seems fine to me. > Let's see what others think. > > Ismael > > On Fri, Mar

Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

If this is rare (as it sounds), relying on import renames seems fine to me. Let's see what others think. Ismael On Fri, Mar 16, 2018 at 10:51 AM, Debasish Ghosh < debasish.gh...@lightbend.com> wrote: > I am not sure if this is practical or not. But theoretically a user may > want to extract the

Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

I am not sure if this is practical or not. But theoretically a user may want to extract the unsafe Java abstraction from the Scala ones and use Java APIs on them .. e.g. val userClicksStream: KStreamS[String, Long] = builder.stream(userClicksTopic) // Scala abstraction val jStream:

Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

What does "mixed mode application" mean? What are the cases where a user would want to use both APIs? I think that would help understand the reasoning. Thanks, Ismael On Fri, Mar 16, 2018 at 8:48 AM, Debasish Ghosh < debasish.gh...@lightbend.com> wrote: > Hi Damian - > > We could. But in case

Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

Thanks for the KIP! It will be nice to have an API for Kafka Streams that is more concise _and_ safer. Like Damian, I would prefer if we didn't have a suffix and relied on package names. However, if there's a good reason why this won't work well, we should explain it in the "Rejected

Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

Hi Damian - We could. But in case the user wants to use both Scala and Java APIs (may be for some mixed mode application), won't that be confusing ? She will have to do something like .. import o.a.k.s.scala.{KStream => KStreamS} to rename Scala imports or the other way round for imported Java

Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

Hi Debasish, Thanks for the KIP - will be a great addition to streams. I've only had a quick scan, but seeing as the Scala classes are going to be in their own package could we drop the S at the end of the class names? Thanks, Damian On Fri, 16 Mar 2018 at 15:25 Debasish Ghosh

[DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

Hi - A new KIP, KIP-270 is up for discussion: https://cwiki.apache.org/confluence/display/KAFKA/KIP-270+-+A+Scala+Wrapper+Library+for+Kafka+Streams The relevant JIRA issue: https://issues.apache.org/jira/browse/KAFKA-6670 The library as proposed in the KIP has been implemented at