[jira] [Assigned] (KAFKA-4919) Document that stores must not be closed when Processors are closed

[ https://issues.apache.org/jira/browse/KAFKA-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-4919: - Assignee: Damian Guy > Document that stores must not be closed when Processors are clo

[jira] [Work started] (KAFKA-4919) Document that stores must not be closed when Processors are closed

[ https://issues.apache.org/jira/browse/KAFKA-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4919 started by Damian Guy. - > Document that stores must not be closed when Processors are clo

[jira] [Commented] (KAFKA-4919) Streams job fails with InvalidStateStoreException: Store is currently closed

[ https://issues.apache.org/jira/browse/KAFKA-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15936328#comment-15936328 ] Damian Guy commented on KAFKA-4919: --- Hi [~elevy] Yes that would be the issue. Because we now re-use

Re: [DISCUSS] KIP-132: Augment KStream.print to allow extra parameters in the printed string

Hi Marc, Thanks for the KIP. It mostly looks good to me. The only thing i'd change is using a null argument to use a default mapping. IMO it would be better if the existing print() method delegates to the new one supplying a KeyValueMapper that does the right thing. Thanks, Damian On Sat, 18

[jira] [Work started] (KAFKA-4594) Annotate integration tests and provide gradle build targets to run subsets of tests

[ https://issues.apache.org/jira/browse/KAFKA-4594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4594 started by Damian Guy. - > Annotate integration tests and provide gradle build targets to run subs

[jira] [Resolved] (KAFKA-4909) run integration tests separate from unit tests

[ https://issues.apache.org/jira/browse/KAFKA-4909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy resolved KAFKA-4909. --- Resolution: Duplicate > run integration tests separate from unit te

[jira] [Commented] (KAFKA-4594) Annotate integration tests and provide gradle build targets to run subsets of tests

[ https://issues.apache.org/jira/browse/KAFKA-4594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927957#comment-15927957 ] Damian Guy commented on KAFKA-4594: --- [~manasvigupta] Sorry, i've already begun work on a duplicate

[jira] [Assigned] (KAFKA-4594) Annotate integration tests and provide gradle build targets to run subsets of tests

[ https://issues.apache.org/jira/browse/KAFKA-4594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-4594: - Assignee: Damian Guy (was: Manasvi Gupta) > Annotate integration tests and provide gradle bu

[jira] [Created] (KAFKA-4909) run integration tests separate from unit tests

Damian Guy created KAFKA-4909: - Summary: run integration tests separate from unit tests Key: KAFKA-4909 URL: https://issues.apache.org/jira/browse/KAFKA-4909 Project: Kafka Issue Type: New

[jira] [Commented] (KAFKA-4643) Improve test coverage of StreamsKafkaClient

[ https://issues.apache.org/jira/browse/KAFKA-4643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15926216#comment-15926216 ] Damian Guy commented on KAFKA-4643: --- [~adyachkov] true. I missed that! Though the unit tests

[jira] [Reopened] (KAFKA-4643) Improve test coverage of StreamsKafkaClient

[ https://issues.apache.org/jira/browse/KAFKA-4643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reopened KAFKA-4643: --- > Improve test coverage of StreamsKafkaCli

[jira] [Commented] (KAFKA-4643) Improve test coverage of StreamsKafkaClient

[ https://issues.apache.org/jira/browse/KAFKA-4643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15925830#comment-15925830 ] Damian Guy commented on KAFKA-4643: --- [~adyachkov] it looks like the method has been removed since

[jira] [Resolved] (KAFKA-4643) Improve test coverage of StreamsKafkaClient

[ https://issues.apache.org/jira/browse/KAFKA-4643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy resolved KAFKA-4643. --- Resolution: Won't Fix Method that wasn't covered has subsequently been removed > Improve t

[jira] [Commented] (KAFKA-4885) processstreamwithcachedstatestore and other streams benchmarks fail occasionally

[ https://issues.apache.org/jira/browse/KAFKA-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15925825#comment-15925825 ] Damian Guy commented on KAFKA-4885: --- [~guozhang] 1. We give users a chance to shutdown the whole

[jira] [Comment Edited] (KAFKA-4609) KTable/KTable join followed by groupBy and aggregate/count can result in incorrect results

[ https://issues.apache.org/jira/browse/KAFKA-4609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15924238#comment-15924238 ] Damian Guy edited comment on KAFKA-4609 at 3/14/17 2:07 PM: [~miguno

[jira] [Commented] (KAFKA-4609) KTable/KTable join followed by groupBy and aggregate/count can result in incorrect results

[ https://issues.apache.org/jira/browse/KAFKA-4609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15924238#comment-15924238 ] Damian Guy commented on KAFKA-4609: --- [~miguno] It is because the caches are flushed independently

[jira] [Updated] (KAFKA-4890) State directory being deleted when another thread holds the lock

[ https://issues.apache.org/jira/browse/KAFKA-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4890: -- Description: Looks like a state directory is being cleaned up when another thread already has the lock

[jira] [Created] (KAFKA-4890) State directory being deleted when another thread holds the lock

Damian Guy created KAFKA-4890: - Summary: State directory being deleted when another thread holds the lock Key: KAFKA-4890 URL: https://issues.apache.org/jira/browse/KAFKA-4890 Project: Kafka

[jira] [Resolved] (KAFKA-4880) Add KTable/GlobalKTable Join to Streams

[ https://issues.apache.org/jira/browse/KAFKA-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy resolved KAFKA-4880. --- Resolution: Duplicate > Add KTable/GlobalKTable Join to Stre

Re: [DISCUSS] KIP-129: Kafka Streams Exactly-Once Semantics

same task are publishing to > > different topics. Since the producer batching happens at the > > topic/partition level, using a producer per task may not impact batching > > much. > > > > 2. When processing.guarantee is set to exactly_once, do we want to > enforce > &

[jira] [Created] (KAFKA-4881) Add internal leave.group.on.close config to consumer

Damian Guy created KAFKA-4881: - Summary: Add internal leave.group.on.close config to consumer Key: KAFKA-4881 URL: https://issues.apache.org/jira/browse/KAFKA-4881 Project: Kafka Issue Type

[jira] [Created] (KAFKA-4880) Add KTable/GlobalKTable Join to Streams

Damian Guy created KAFKA-4880: - Summary: Add KTable/GlobalKTable Join to Streams Key: KAFKA-4880 URL: https://issues.apache.org/jira/browse/KAFKA-4880 Project: Kafka Issue Type: New Feature

[jira] [Resolved] (KAFKA-4876) WindowStore.fetch(key, 0, Long.MaxValue) is very slow

[ https://issues.apache.org/jira/browse/KAFKA-4876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy resolved KAFKA-4876. --- Resolution: Duplicate Fix Version/s: 0.10.2.1 > WindowStore.fetch(key, 0, Long.MaxVa

[jira] [Updated] (KAFKA-4863) Querying window store may return unwanted keys

[ https://issues.apache.org/jira/browse/KAFKA-4863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4863: -- Status: Patch Available (was: In Progress) > Querying window store may return unwanted k

[jira] [Work started] (KAFKA-4863) Querying window store may return unwanted keys

[ https://issues.apache.org/jira/browse/KAFKA-4863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4863 started by Damian Guy. - > Querying window store may return unwanted k

[jira] [Assigned] (KAFKA-4863) Querying window store may return unwanted keys

[ https://issues.apache.org/jira/browse/KAFKA-4863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-4863: - Assignee: Damian Guy (was: Eno Thereska) > Querying window store may return unwanted k

[jira] [Updated] (KAFKA-4657) Improve test coverage of CompositeReadOnlyWindowStore

[ https://issues.apache.org/jira/browse/KAFKA-4657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4657: -- Description: No tests that cover what happens when {{windowStore.fetch(...)}} throw

Re: [DISCUSS] KIP-129: Kafka Streams Exactly-Once Semantics

all at once. > >> - Memory usage. We currently default to 64MB per producer. This can be > >> tuned down, but the fact that we are spreading the batching over more > >> producers will fundamentally mean we need a lot more memory to get > good > >> p

[jira] [Created] (KAFKA-4851) SessionStore.fetch(key) is a performance bottleneck

Damian Guy created KAFKA-4851: - Summary: SessionStore.fetch(key) is a performance bottleneck Key: KAFKA-4851 URL: https://issues.apache.org/jira/browse/KAFKA-4851 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP 130: Expose states of active tasks to KafkaStreams public API

Hi Florian, Thanks for the KIP. It seems there is some overlap here with what we already have in KafkaStreams.allMetadata(). This currently returns a Collection where each StreamsMetadata instance holds the state stores and partition assignment for every instance of the KafkaStreams application.

Re: groupBy without auto-repartition topics for Kafka Streams

If you use stream.groupByKey() then there will be no repartitioning as long as there have been no key changing operations preceding it, i.e, map, selectKey, flatMap, transform. If you use stream.groupBy(...) then we see it as a key changing operation, hence we need to repartition the data. On

Re: [DISCUSS] KIP-129: Kafka Streams Exactly-Once Semantics

Hi Guozhang, Thanks for the KIP! This is an important feature for Kafka Streams and will help to unlock a bunch of use cases. I have some concerns/questions: 1. Producer per task: I'm worried about the overhead this is going to put on both the streams app and the Kafka Brokers. You can

Re: [DISCUSS] 0.10.3.0/0.11.0.0 release planning

+1 On Tue, 28 Feb 2017 at 04:32 Vahid S Hashemian wrote: > +1 on 0.11.0.0. > > Can we also include KIP-54 to the list? > The PR for this KIP is ready for review. > > Thanks. > --Vahid > > > > > > From: Ismael Juma > To: dev@kafka.apache.org >

Re: [VOTE] KIP-123: Allow per stream/table timestamp extractor

Thanks for the KIP Jeyhun! +1 On Tue, 28 Feb 2017 at 08:59 Jeyhun Karimov wrote: > Dear community, > > I'd like to start the vote for KIP-123: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=68714788 > > > Cheers, > Jeyhun > -- > -Cheers > > Jeyhun >

[jira] [Commented] (KAFKA-4791) Kafka Streams - unable to add state stores when using wildcard topics on the source

[ https://issues.apache.org/jira/browse/KAFKA-4791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15881280#comment-15881280 ] Damian Guy commented on KAFKA-4791: --- [~mjsax] this is a bug. Bart is calling {{addStateStore}} which

Re: [DISCUSS] KIP-116 - Add State Store Checkpoint Interval Configuration

ach commit we can try that with this as well, since it won't change any > > public APIs and we can still add this config in the future if we do > observe > > some users reporting it has huge perf impacts. > > > > > > > > Guozhang > > > > On Fri, Feb

Re: [DISCUSS] KIP-120: Cleanup Kafka Streams builder API

> And about printing the topology for debuggability: I agrees this is a > > potential drawback, and I'd suggest maintain some functionality to build > a > > "dry topology" as Mathieu suggested; the difficulty is that, internally > we > > need a different "copy" of the topology for each thread so

[jira] [Assigned] (KAFKA-4494) Significant startup delays in KStreams app

[ https://issues.apache.org/jira/browse/KAFKA-4494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-4494: - Assignee: Damian Guy > Significant startup delays in KStreams

[jira] [Work started] (KAFKA-4494) Significant startup delays in KStreams app

[ https://issues.apache.org/jira/browse/KAFKA-4494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4494 started by Damian Guy. - > Significant startup delays in KStreams

Re: [DISCUSS] KIP-116 - Add State Store Checkpoint Interval Configuration

d can disable the checkpoint file for this > case. And even if there is no switch and we enable EoS all the time, we > can get rid of the checkpoint file overall (making the parameter obsolete). > > IMHO, if the config parameter is not really useful, we should not have it. > > >

Re: [DISCUSS] KIP-116 - Add State Store Checkpoint Interval Configuration

gt; May be for fine tuning the performance. > >>> Say we don't need the checkpointing and would like to gain the lil bit > >> of performance improvement by turning it off. > >>> The trade off is between giving people control knobs vs complicating > the > >

Re: [DISCUSS] KIP-116 - Add State Store Checkpoint Interval Configuration

y we can get rid of this extra config variable and just > use the existing commit interval. > > Less tuning knobs. > > Eno > > > On 10 Feb 2017, at 09:27, Damian Guy <damian@gmail.com> wrote: > > > > Gouzhang, > > > > You've

Re: [DISCUSS] KIP-116 - Add State Store Checkpoint Interval Configuration

the checkpoint file in your mind is different? > > > > > > > > > Guozhang > > > > > > > > > > > > On Thu, Feb 9, 2017 at 11:02 AM, Guozhang Wang <wangg...@gmail.com> > > wrote: > > > > > >>

Re: [VOTE] KIP-118: Drop Support for Java 7 in Kafka 0.11

+1 On Thu, 9 Feb 2017 at 15:41 Bill Bejeck wrote: > +1 > > On Thu, Feb 9, 2017 at 10:31 AM, Ismael Juma wrote: > > > Hi everyone, > > > > Since everyone in the discuss thread was in favour (10 people > responded), I > > would like to initiate the voting

[jira] [Updated] (KAFKA-4484) Set more conservative default values on RocksDB for memory usage

[ https://issues.apache.org/jira/browse/KAFKA-4484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4484: -- Status: Patch Available (was: In Progress) > Set more conservative default values on Rock

[jira] [Created] (KAFKA-4752) Streams Simple Benchmark MB/sec calculation is not correct for Join operations

Damian Guy created KAFKA-4752: - Summary: Streams Simple Benchmark MB/sec calculation is not correct for Join operations Key: KAFKA-4752 URL: https://issues.apache.org/jira/browse/KAFKA-4752 Project

[jira] [Assigned] (KAFKA-4484) Set more conservative default values on RocksDB for memory usage

[ https://issues.apache.org/jira/browse/KAFKA-4484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-4484: - Assignee: Damian Guy > Set more conservative default values on RocksDB for memory us

[jira] [Work started] (KAFKA-4484) Set more conservative default values on RocksDB for memory usage

[ https://issues.apache.org/jira/browse/KAFKA-4484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4484 started by Damian Guy. - > Set more conservative default values on RocksDB for memory us

Re: KIP-121 [VOTE]: Add KStream peek method

+1 On Thu, 9 Feb 2017 at 10:26 Michael Noll wrote: > +1 (non-binding) > > And thanks again for suggesting and driving this, Steven! :-) > > Best, > Michael > > > > > On Thu, Feb 9, 2017 at 6:06 AM, Matthias J. Sax > wrote: > > > +1 > > > > On 2/8/17

Re: [DISCUSS] KIP-116 - Add State Store Checkpoint Interval Configuration

at go into picking this > value? > > Right now, in the KIP and in this discussion, "5 mins" looks like a magic > number to me. :-) > > -Michael > > > > On Thu, Feb 9, 2017 at 11:03 AM, Damian Guy <damian@gmail.com> wrote: > > > I've ran

Re: [DISCUSS] KIP-116 - Add State Store Checkpoint Interval Configuration

/33476/298721.4720994145/30.738439479029754 Streams Performance [records/latency/rec-sec/MB-sec source+store]: 1000/33196/301241.1133871551/30.99771056753826 On Wed, 8 Feb 2017 at 09:02 Damian Guy <damian@gmail.com> wrote: > Matthias, > > Fair point. I'll update it the KIP. >

Re: [DISCUSS] KIP-116 - Add State Store Checkpoint Interval Configuration

ve the > advantage to guard users to accidentally switch it off. > > -Matthias > > > On 2/3/17 2:03 AM, Damian Guy wrote: > > Hi Matthias, > > > > It possibly doesn't make sense to disable it, but then i'm sure someone > > will come up with a reason they don't

Re: KIP-121 [Discuss]: Add KStream peek method

Hi Steven, Thanks for the KIP. I think this is a worthy addition to the API. Thanks, Damian On Tue, 7 Feb 2017 at 09:30 Eno Thereska wrote: > Hi, > > I like the proposal, thank you. I have found it frustrating myself not to > be able to understand simple things, like

Re: [DISCUSS] KIP-120: Cleanup Kafka Streams builder API

Hi Matthias, Thanks for the KIP. Should TopologyBuilder#build() and KStreamBuilder#build accept a StreamsConfig as an argument? Should we add a KStreamBuilder#topologyBuilder() for cases where people want to mix and match DSL & PAPI? Or do you think we already provide enough support for that

Re: [DISCUSS] KIP-116 - Add State Store Checkpoint Interval Configuration

blem, but can be convinced otherwise. > > Thanks > Eno > > > > > On 3 Feb 2017, at 10:03, Damian Guy <damian@gmail.com> wrote: > > > > Hi Matthias, > > > > It possibly doesn't make sense to disable it, but then i'm sure someone > > will

Re: [DISCUSS] KIP-118: Drop Support for Java 7 in Kafka 0.11

Thanks Ismael. Makes sense to me. On Fri, 3 Feb 2017 at 10:39 Ismael Juma wrote: > Hi all, > > I have posted a KIP for dropping support for Java 7 in Kafka 0.11: > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-118%3A+Drop+Support+for+Java+7+in+Kafka+0.11 > > Most

Re: [DISCUSS] KIP-116 - Add State Store Checkpoint Interval Configuration

gt; wrote: > Thanks Damian. > > One more question: "Checkpointing is disabled if the checkpoint interval > is set to a value <=0." > > > Does it make sense to disable check pointing? What's the tradeoff here? > > > -Matthias > > > On 2/2/

[jira] [Resolved] (KAFKA-4182) Move the change logger out of RocksDB stores

[ https://issues.apache.org/jira/browse/KAFKA-4182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy resolved KAFKA-4182. --- Resolution: Fixed Fix Version/s: 0.10.2.0 Fixed by: https://github.com/apache/kafka/pull/2360

[jira] [Updated] (KAFKA-4724) Clean up of state directories can possibly remove stores that are about to be used by another thread

[ https://issues.apache.org/jira/browse/KAFKA-4724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4724: -- Summary: Clean up of state directories can possibly remove stores that are about to be used by another

[jira] [Assigned] (KAFKA-4724) Clean up of state directories has a race condition and can possibly remove stores that are still used

[ https://issues.apache.org/jira/browse/KAFKA-4724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-4724: - Assignee: Damian Guy > Clean up of state directories has a race condition and can possi

[jira] [Updated] (KAFKA-4724) Clean up of state directories has a race condition and can possibly remove stores that are still used

[ https://issues.apache.org/jira/browse/KAFKA-4724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4724: -- Summary: Clean up of state directories has a race condition and can possibly remove stores

[jira] [Created] (KAFKA-4724) Clean up of state directories has a race condition and can possibly remove stores that are sill used

Damian Guy created KAFKA-4724: - Summary: Clean up of state directories has a race condition and can possibly remove stores that are sill used Key: KAFKA-4724 URL: https://issues.apache.org/jira/browse/KAFKA-4724

Re: [DISCUSS] KIP-116 - Add State Store Checkpoint Interval Configuration

uot; > > it might be easier to just use an parameter that is "number-or-commit > intervals". > > > -Matthias > > > On 2/1/17 7:29 AM, Damian Guy wrote: > > Thanks for the comments Eno. > > As for exactly once, i don't believe this matters as we are just >

Re: [DISCUSS] KIP-116 - Add State Store Checkpoint Interval Configuration

a, I > believe we'll have to add some support for rolling back checkpoints, e.g., > when a transaction is aborted. We need to be aware of that and ideally > anticipate a bit those needs in the KIP. > > > > Thanks > > Eno > > > > > >> On 1 Feb 2

[DISCUSS] KIP-116 - Add State Store Checkpoint Interval Configuration

Hi all, I would like to start the discussion on KIP-116: https://cwiki.apache.org/confluence/display/KAFKA/KIP-116+-+Add+State+Store+Checkpoint+Interval+Configuration Thanks, Damian

[jira] (KAFKA-4317) RocksDB checkpoint files lost on kill -9

[ https://issues.apache.org/jira/browse/KAFKA-4317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4317: -- Status: Patch Available (was: Open) > RocksDB checkpoint files lost on kill

Re: Re: [DISCUSS] KIP-114: KTable materialization and improved semantics

gt;> Nevertheless I see your point and I think we could provide a better > API > >>>>> for KTable stores including the discovery of remote shards of the > same > >>>>> KTable. > >>>>> > >>>>> @Michael: Yes, right now we do have

Re: [DISCUSS] KIP-114: KTable materialization and improved semantics

erialization and improved >>>>> semantics >>>>> Date: 24 January 2017 at 19:30:10 GMT >>>>> To: dev@kafka.apache.org >>>>> Reply-To: dev@kafka.apache.org >>>>> >>>>> That not what I meant by "huge impact". >&

[jira] [Assigned] (KAFKA-4317) RocksDB checkpoint files lost on kill -9

[ https://issues.apache.org/jira/browse/KAFKA-4317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-4317: - Assignee: Damian Guy > RocksDB checkpoint files lost on kill

[jira] [Assigned] (KAFKA-4650) Improve test coverage org.apache.kafka.streams.kstream.internals

[ https://issues.apache.org/jira/browse/KAFKA-4650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-4650: - Assignee: Damian Guy > Improve test coverage org.apache.kafka.streams.kstream.intern

Re: Fwd: [DISCUSS] KIP-114: KTable materialization and improved semantics

gt;> To: dev@kafka.apache.org > >>> Reply-To: dev@kafka.apache.org > >>> > >>> That not what I meant by "huge impact". > >>> > >>> I refer to the actions related to materialize a KTable: creating a > >>> RocksDB store and a

[jira] [Reopened] (KAFKA-4222) Transient failure in QueryableStateIntegrationTest.queryOnRebalance

[ https://issues.apache.org/jira/browse/KAFKA-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reopened KAFKA-4222: --- re-opening as there was one more case of this recently. However the build has been cleaned up. Just

[jira] [Updated] (KAFKA-4696) Streams standby task assignment should be state-store aware

[ https://issues.apache.org/jira/browse/KAFKA-4696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4696: -- Description: Task Assignment is currently not aware of which tasks have State Stores. This can result

[jira] [Updated] (KAFKA-4649) Improve test coverage GlobalStateManagerImpl

[ https://issues.apache.org/jira/browse/KAFKA-4649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4649: -- Status: Patch Available (was: Open) > Improve test coverage GlobalStateManagerI

[jira] [Assigned] (KAFKA-4649) Improve test coverage GlobalStateManagerImpl

[ https://issues.apache.org/jira/browse/KAFKA-4649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-4649: - Assignee: Damian Guy > Improve test coverage GlobalStateManagerI

[jira] [Updated] (KAFKA-4648) Improve test coverage StreamTask

[ https://issues.apache.org/jira/browse/KAFKA-4648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4648: -- Status: Patch Available (was: Open) > Improve test coverage StreamT

[jira] [Assigned] (KAFKA-4648) Improve test coverage StreamTask

[ https://issues.apache.org/jira/browse/KAFKA-4648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-4648: - Assignee: Damian Guy > Improve test coverage StreamT

[jira] [Updated] (KAFKA-4647) Improve test coverage of GlobalStreamThread

[ https://issues.apache.org/jira/browse/KAFKA-4647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4647: -- Status: Patch Available (was: Open) > Improve test coverage of GlobalStreamThr

[jira] [Assigned] (KAFKA-4647) Improve test coverage of GlobalStreamThread

[ https://issues.apache.org/jira/browse/KAFKA-4647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-4647: - Assignee: Damian Guy > Improve test coverage of GlobalStreamThr

[jira] [Updated] (KAFKA-4644) Improve test coverage of StreamsPartitionAssignor

[ https://issues.apache.org/jira/browse/KAFKA-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4644: -- Status: Patch Available (was: Open) > Improve test coverage of StreamsPartitionAssig

[jira] [Assigned] (KAFKA-4644) Improve test coverage of StreamsPartitionAssignor

[ https://issues.apache.org/jira/browse/KAFKA-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-4644: - Assignee: Damian Guy > Improve test coverage of StreamsPartitionAssig

[jira] [Updated] (KAFKA-4646) Improve test coverage AbstractProcessorContext

[ https://issues.apache.org/jira/browse/KAFKA-4646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4646: -- Status: Patch Available (was: Reopened) > Improve test coverage AbstractProcessorCont

[jira] [Updated] (KAFKA-4645) Improve test coverage of ProcessorTopology

[ https://issues.apache.org/jira/browse/KAFKA-4645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4645: -- Status: Patch Available (was: Open) > Improve test coverage of ProcessorTopol

[jira] [Assigned] (KAFKA-4646) Improve test coverage AbstractProcessorContext

[ https://issues.apache.org/jira/browse/KAFKA-4646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-4646: - Assignee: Damian Guy > Improve test coverage AbstractProcessorCont

[jira] [Assigned] (KAFKA-4645) Improve test coverage of ProcessorTopology

[ https://issues.apache.org/jira/browse/KAFKA-4645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-4645: - Assignee: Damian Guy > Improve test coverage of ProcessorTopol

[jira] [Updated] (KAFKA-4642) Improve test coverage of ProcessorStateManager

[ https://issues.apache.org/jira/browse/KAFKA-4642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4642: -- Status: Patch Available (was: In Progress) > Improve test coverage of ProcessorStateMana

[jira] [Work started] (KAFKA-4642) Improve test coverage of ProcessorStateManager

[ https://issues.apache.org/jira/browse/KAFKA-4642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4642 started by Damian Guy. - > Improve test coverage of ProcessorStateMana

[jira] [Assigned] (KAFKA-4642) Improve test coverage of ProcessorStateManager

[ https://issues.apache.org/jira/browse/KAFKA-4642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-4642: - Assignee: Damian Guy > Improve test coverage of ProcessorStateMana

[jira] [Created] (KAFKA-4696) Streams standby task assignment should be state-store aware

Damian Guy created KAFKA-4696: - Summary: Streams standby task assignment should be state-store aware Key: KAFKA-4696 URL: https://issues.apache.org/jira/browse/KAFKA-4696 Project: Kafka Issue

Re: [DISCUSS] KIP-114: KTable materialization and improved semantics

verbose > but explicit version. > > > -Matthias > > On 1/23/17 1:39 AM, Damian Guy wrote: > > I'm not a fan of materialize. I think it interrupts the flow, i.e, > > > > table.mapValue(..).materialize().join(..).materialize() > > compared to: > > table.mapV

Re: [DISCUSS] KIP-114: KTable materialization and improved semantics

serde) v.s. table.mapValues(/*no > > serde specified*/).materialize(serde). We need to specify what are the > > handling logic here. > > 3. We can remove "KTable#to" call as well, and enforce users to call " > > KTable.toStream.to" to be more c

[jira] [Updated] (KAFKA-4677) Avoid unnecessary task movement across threads of the same process during rebalance

[ https://issues.apache.org/jira/browse/KAFKA-4677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4677: -- Summary: Avoid unnecessary task movement across threads of the same process during rebalance

[jira] [Created] (KAFKA-4677) Make StreamPartitionAssigner more sticky when assigning to threads

Damian Guy created KAFKA-4677: - Summary: Make StreamPartitionAssigner more sticky when assigning to threads Key: KAFKA-4677 URL: https://issues.apache.org/jira/browse/KAFKA-4677 Project: Kafka

[jira] [Updated] (KAFKA-4662) Improve test coverage TopologyBuilder

[ https://issues.apache.org/jira/browse/KAFKA-4662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4662: -- Description: overloaded {{addSource}} methods with {{AutoOffsetReset}} param not tested. Also some

[jira] [Created] (KAFKA-4662) Improve test coverage TopologyBuilder

Damian Guy created KAFKA-4662: - Summary: Improve test coverage TopologyBuilder Key: KAFKA-4662 URL: https://issues.apache.org/jira/browse/KAFKA-4662 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-4661) Improve test coverage UsePreviousTimeOnInvalidTimestamp

Damian Guy created KAFKA-4661: - Summary: Improve test coverage UsePreviousTimeOnInvalidTimestamp Key: KAFKA-4661 URL: https://issues.apache.org/jira/browse/KAFKA-4661 Project: Kafka Issue Type

[jira] [Created] (KAFKA-4660) Improve test coverage KafkaStreams

Damian Guy created KAFKA-4660: - Summary: Improve test coverage KafkaStreams Key: KAFKA-4660 URL: https://issues.apache.org/jira/browse/KAFKA-4660 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-4659) Improve test coverage of CachingKeyValueStore

Damian Guy created KAFKA-4659: - Summary: Improve test coverage of CachingKeyValueStore Key: KAFKA-4659 URL: https://issues.apache.org/jira/browse/KAFKA-4659 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-4658) Improve test coverage InMemoryKeyValueLoggedStore

Damian Guy created KAFKA-4658: - Summary: Improve test coverage InMemoryKeyValueLoggedStore Key: KAFKA-4658 URL: https://issues.apache.org/jira/browse/KAFKA-4658 Project: Kafka Issue Type: Sub

[jira] [Created] (KAFKA-4657) Improve test coverage of CompositeReadOnlyWindowStore

Damian Guy created KAFKA-4657: - Summary: Improve test coverage of CompositeReadOnlyWindowStore Key: KAFKA-4657 URL: https://issues.apache.org/jira/browse/KAFKA-4657 Project: Kafka Issue Type

<    1   2   3   4   5   6   7   8   >