[jira] [Updated] (KAFKA-4366) KafkaStreams.close() blocks indefinitely

[ https://issues.apache.org/jira/browse/KAFKA-4366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4366: -- Status: Patch Available (was: In Progress) > KafkaStreams.close() blocks indefinit

[jira] [Updated] (KAFKA-4379) Remove caching of dirty and removed keys from StoreChangeLogger

[ https://issues.apache.org/jira/browse/KAFKA-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4379: -- Status: Patch Available (was: In Progress) > Remove caching of dirty and removed keys f

[jira] [Work started] (KAFKA-4379) Remove caching of dirty and removed keys from StoreChangeLogger

[ https://issues.apache.org/jira/browse/KAFKA-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4379 started by Damian Guy. - > Remove caching of dirty and removed keys from StoreChangeLog

[jira] [Created] (KAFKA-4379) Remove caching of dirty and removed keys from StoreChangeLogger

Damian Guy created KAFKA-4379: - Summary: Remove caching of dirty and removed keys from StoreChangeLogger Key: KAFKA-4379 URL: https://issues.apache.org/jira/browse/KAFKA-4379 Project: Kafka

[jira] [Work started] (KAFKA-4366) KafkaStreams.close() blocks indefinitely

[ https://issues.apache.org/jira/browse/KAFKA-4366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4366 started by Damian Guy. - > KafkaStreams.close() blocks indefinit

[jira] [Assigned] (KAFKA-4366) KafkaStreams.close() blocks indefinitely

[ https://issues.apache.org/jira/browse/KAFKA-4366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-4366: - Assignee: Damian Guy (was: Guozhang Wang) > KafkaStreams.close() blocks indefinit

[jira] [Updated] (KAFKA-4311) Multi layer cache eviction causes forwarding to incorrect ProcessorNode

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4311: -- Status: Patch Available (was: Open) > Multi layer cache eviction causes forwarding to incorr

[jira] [Commented] (KAFKA-4311) Multi layer cache eviction causes forwarding to incorrect ProcessorNode

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15621909#comment-15621909 ] Damian Guy commented on KAFKA-4311: --- Thanks for confirming > Multi layer cache eviction cau

[jira] [Commented] (KAFKA-4311) Multi layer cache eviction causes forwarding to incorrect ProcessorNode

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15597385#comment-15597385 ] Damian Guy commented on KAFKA-4311: --- Frank, thanks for testing this for me. I just pushed another minor

[jira] [Updated] (KAFKA-4311) Multi layer cache eviction causes forwarding to incorrect ProcessorNode

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4311: -- Summary: Multi layer cache eviction causes forwarding to incorrect ProcessorNode (was: Multi layer

[jira] [Updated] (KAFKA-4311) Multi layer cache eviction causes forwarding to incorrect Processor Node

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4311: -- Description: The two exceptions below were reported by Frank on the dev mailing list. After

[jira] [Updated] (KAFKA-4311) Multi layer cache eviction causes forwarding to incorrect Processor Node

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4311: -- Description: The two exceptions below were reported by Frank on the dev mailing list. After

[jira] [Updated] (KAFKA-4311) Multi layer cache eviction causes forwarding to incorrect Processor Node

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4311: -- Summary: Multi layer cache eviction causes forwarding to incorrect Processor Node (was: Mutli layer

[jira] [Updated] (KAFKA-4311) Mutli layer cache eviction causes forwarding to incorrect Processor Node

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4311: -- Summary: Mutli layer cache eviction causes forwarding to incorrect Processor Node (was: Exception

[jira] [Commented] (KAFKA-4311) Exception in NamedCache.flush - Key found in dirty key set, but entry is null

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15594852#comment-15594852 ] Damian Guy commented on KAFKA-4311: --- Hi Frank, if you have the time, would you mind trying to run your

[jira] [Commented] (KAFKA-4311) Exception in NamedCache.flush - Key found in dirty key set, but entry is null

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15592060#comment-15592060 ] Damian Guy commented on KAFKA-4311: --- when you are running it are you running a single instance of your

[jira] [Commented] (KAFKA-4311) Exception in NamedCache.flush - Key found in dirty key set, but entry is null

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15591872#comment-15591872 ] Damian Guy commented on KAFKA-4311: --- Ok - i've been to reproduce the issue with the test data. It has

[jira] [Commented] (KAFKA-4311) Exception in NamedCache.flush - Key found in dirty key set, but entry is null

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15591675#comment-15591675 ] Damian Guy commented on KAFKA-4311: --- Unfortunately that isn't working. We probably need the zookeeper

[jira] [Commented] (KAFKA-4311) Exception in NamedCache.flush - Key found in dirty key set, but entry is null

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15591605#comment-15591605 ] Damian Guy commented on KAFKA-4311: --- thanks - i'll take a look > Exception in NamedCache.flush -

[jira] [Commented] (KAFKA-4311) Exception in NamedCache.flush - Key found in dirty key set, but entry is null

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15591478#comment-15591478 ] Damian Guy commented on KAFKA-4311: --- Can you commit it to the github repo? > Except

[jira] [Commented] (KAFKA-4311) Exception in NamedCache.flush - Key found in dirty key set, but entry is null

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15591271#comment-15591271 ] Damian Guy commented on KAFKA-4311: --- [~flyaruu] Thanks for the sample code. I've tried running

[jira] [Commented] (KAFKA-4311) Exception in NamedCache.flush - Key found in dirty key set, but entry is null

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15589089#comment-15589089 ] Damian Guy commented on KAFKA-4311: --- Thanks - i'll take a look at it what your repository and get back

[jira] [Commented] (KAFKA-4311) Exception in NamedCache.flush - Key found in dirty key set, but entry is null

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588096#comment-15588096 ] Damian Guy commented on KAFKA-4311: --- [~flyaruu] We are unable to reproduce or find the problem you

[jira] [Commented] (KAFKA-4311) Exception in NamedCache.flush - Key found in dirty key set, but entry is null

[ https://issues.apache.org/jira/browse/KAFKA-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15585914#comment-15585914 ] Damian Guy commented on KAFKA-4311: --- [~flyaruu] by merge do you mean join? Is it possible for you

[jira] [Created] (KAFKA-4311) Exception in NamedCache.flush - Key found in dirty key set, but entry is null

Damian Guy created KAFKA-4311: - Summary: Exception in NamedCache.flush - Key found in dirty key set, but entry is null Key: KAFKA-4311 URL: https://issues.apache.org/jira/browse/KAFKA-4311 Project

Re: KStreams / add support for sink processor with dynamic topics

Hi Florian, Do you know the set of subtypes in advance? I.e, could you use: KStream[] branches = stream.branch(predicates); to split the stream based on the subtypes? Thanks, Damian On Tue, 18 Oct 2016 at 00:43 Matthias J. Sax wrote: > -BEGIN PGP SIGNED

[jira] [Updated] (KAFKA-4300) NamedCache throws an NPE when evict is called and the cache is empty

[ https://issues.apache.org/jira/browse/KAFKA-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4300: -- Status: Patch Available (was: In Progress) > NamedCache throws an NPE when evict is cal

[jira] [Updated] (KAFKA-4300) NamedCache throws an NPE when evict is called and the cache is empty

[ https://issues.apache.org/jira/browse/KAFKA-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4300: -- Description: evict can be called on a NamedCache even though it is empty. This is due to the shared

Re: [kafka-clients] [VOTE] 0.10.1.0 RC2

Hi Jason, Really sorry, but we are going to need to cut another RC. There was a report on the user list w.r.t the NamedCache (in KafkaStreams) throwing a NullPointerException. I've looked into it and it is definitely a bug that needs to be fixed. jira is

[jira] [Work started] (KAFKA-4300) NamedCache throws an NPE when evict is called and the cache is empty

[ https://issues.apache.org/jira/browse/KAFKA-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4300 started by Damian Guy. - > NamedCache throws an NPE when evict is called and the cache is em

[jira] [Created] (KAFKA-4300) NamedCache throws an NPE when evict is called and the cache is empty

Damian Guy created KAFKA-4300: - Summary: NamedCache throws an NPE when evict is called and the cache is empty Key: KAFKA-4300 URL: https://issues.apache.org/jira/browse/KAFKA-4300 Project: Kafka

[jira] [Updated] (KAFKA-4283) records deleted from CachingKeyValueStore still appear in range and all queries

[ https://issues.apache.org/jira/browse/KAFKA-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4283: -- Status: Patch Available (was: In Progress) > records deleted from CachingKeyValueStore still app

[jira] [Created] (KAFKA-4283) records deleted from CachingKeyValueStore still appear in range and all queries

Damian Guy created KAFKA-4283: - Summary: records deleted from CachingKeyValueStore still appear in range and all queries Key: KAFKA-4283 URL: https://issues.apache.org/jira/browse/KAFKA-4283 Project

[jira] [Work started] (KAFKA-4283) records deleted from CachingKeyValueStore still appear in range and all queries

[ https://issues.apache.org/jira/browse/KAFKA-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4283 started by Damian Guy. - > records deleted from CachingKeyValueStore still appear in range and all > q

[jira] [Commented] (KAFKA-4263) QueryableStateIntegrationTest.concurrentAccess is failing occasionally in jenkins builds

[ https://issues.apache.org/jira/browse/KAFKA-4263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554500#comment-15554500 ] Damian Guy commented on KAFKA-4263: --- I've had this running in a loop and it has run 74 times without

[jira] [Created] (KAFKA-4263) QueryableStateIntegrationTest.concurrentAccess is failing occasionally in jenkins builds

Damian Guy created KAFKA-4263: - Summary: QueryableStateIntegrationTest.concurrentAccess is failing occasionally in jenkins builds Key: KAFKA-4263 URL: https://issues.apache.org/jira/browse/KAFKA-4263

[jira] [Updated] (KAFKA-4253) Fix Kafka Stream thread shutting down process ordering

[ https://issues.apache.org/jira/browse/KAFKA-4253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4253: -- Status: Patch Available (was: In Progress) > Fix Kafka Stream thread shutting down process order

[jira] [Assigned] (KAFKA-4253) Fix Kafka Stream thread shutting down process ordering

[ https://issues.apache.org/jira/browse/KAFKA-4253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-4253: - Assignee: Damian Guy > Fix Kafka Stream thread shutting down process order

[jira] [Work started] (KAFKA-4253) Fix Kafka Stream thread shutting down process ordering

[ https://issues.apache.org/jira/browse/KAFKA-4253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4253 started by Damian Guy. - > Fix Kafka Stream thread shutting down process order

[jira] [Assigned] (KAFKA-4186) Transient failure in KStreamAggregationIntegrationTest.shouldGroupByKey

[ https://issues.apache.org/jira/browse/KAFKA-4186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-4186: - Assignee: Damian Guy > Transient failure in KStreamAggregationIntegrationTest.shouldGroupBy

[jira] [Created] (KAFKA-4223) RocksDBStore should close all open iterators on close

Damian Guy created KAFKA-4223: - Summary: RocksDBStore should close all open iterators on close Key: KAFKA-4223 URL: https://issues.apache.org/jira/browse/KAFKA-4223 Project: Kafka Issue Type

[jira] [Created] (KAFKA-4199) When a window store segment is dropped we should also clear any corresponding cached entries

Damian Guy created KAFKA-4199: - Summary: When a window store segment is dropped we should also clear any corresponding cached entries Key: KAFKA-4199 URL: https://issues.apache.org/jira/browse/KAFKA-4199

[jira] [Created] (KAFKA-4182) Move the change logger our of RocksDB stores

Damian Guy created KAFKA-4182: - Summary: Move the change logger our of RocksDB stores Key: KAFKA-4182 URL: https://issues.apache.org/jira/browse/KAFKA-4182 Project: Kafka Issue Type: Improvement

[jira] [Updated] (KAFKA-4175) Can't have StandbyTasks in KafkaStreams where NUM_STREAM_THREADS_CONFIG > 1

[ https://issues.apache.org/jira/browse/KAFKA-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4175: -- Description: When we have StandbyTasks in a Kafka Streams app and we have > 1 threads per instance

[jira] [Assigned] (KAFKA-4175) Can't have StandbyTasks in KafkaStreams where NUM_STREAM_THREADS_CONFIG > 1

[ https://issues.apache.org/jira/browse/KAFKA-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-4175: - Assignee: Damian Guy (was: Guozhang Wang) > Can't have StandbyTasks in KafkaStreams wh

[jira] [Updated] (KAFKA-4175) Can't have StandbyTasks in KafkaStreams where NUM_STREAM_THREADS_CONFIG > 1

[ https://issues.apache.org/jira/browse/KAFKA-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4175: -- Description: When we have StandbyTasks in a Kafka Streams app and we have > 1 threads per instance

[jira] [Created] (KAFKA-4175) Can't have StandbyTasks in KafkaStreams where NUM_STREAM_THREADS_CONFIG > 1

Damian Guy created KAFKA-4175: - Summary: Can't have StandbyTasks in KafkaStreams where NUM_STREAM_THREADS_CONFIG > 1 Key: KAFKA-4175 URL: https://issues.apache.org/jira/browse/KAFKA-4175 Project: Ka

[jira] [Updated] (KAFKA-4175) Can't have StandbyTasks in KafkaStreams where NUM_STREAM_THREADS_CONFIG > 1

[ https://issues.apache.org/jira/browse/KAFKA-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4175: -- Description: When we have {code}StandbyTasks{code} in a Kafka Streams app and we have > 1 threads

[jira] [Work started] (KAFKA-4163) NPE in StreamsMetadataState during re-balance operations

[ https://issues.apache.org/jira/browse/KAFKA-4163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4163 started by Damian Guy. - > NPE in StreamsMetadataState during re-balance operati

[jira] [Updated] (KAFKA-4163) NPE in StreamsMetadataState during re-balance operations

[ https://issues.apache.org/jira/browse/KAFKA-4163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4163: -- Status: Patch Available (was: In Progress) > NPE in StreamsMetadataState during re-balance operati

[jira] [Created] (KAFKA-4163) NPE in StreamsMetadataState during re-balance operations

Damian Guy created KAFKA-4163: - Summary: NPE in StreamsMetadataState during re-balance operations Key: KAFKA-4163 URL: https://issues.apache.org/jira/browse/KAFKA-4163 Project: Kafka Issue Type

Re: Queryable state client read guarantees

d then have the following sequence: > > 18 -> 2 > 19 -> Not found > 20 -> Not found > 21 -> 2 > > From the client perspective, I think this violates the guarantee of always > reading the latest value. > > Am I making the wrong assumptions or is there som

Re: Queryable state client read guarantees

Hi Mikael, A fix for KAFKA-4123 (the issue you found with receiving null values) has now been committed to trunk. I've tried it with your github repo and it appears to be working. You will have to make a small change to your code as we now throw

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

t; >>> below for a non-cached state, and learn about caching later on, and > > >>> think, caching is a cool feature, I want to use it, you would simply > > >>> want to enable caching (without breaking your code). > > >>> > > >>

[jira] [Updated] (KAFKA-4123) Queryable State returning null for key before all stores in instance have been initialized

[ https://issues.apache.org/jira/browse/KAFKA-4123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4123: -- Summary: Queryable State returning null for key before all stores in instance have been initialized

[jira] [Updated] (KAFKA-4123) Queryable State returning null for key before all stores in instance have been intialized

[ https://issues.apache.org/jira/browse/KAFKA-4123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4123: -- Description: A couple of problems: 1. A RocksDBStore instance is currently marked as open before

[jira] [Updated] (KAFKA-4123) Queryable State returning null for key before all stores in instance have been intialized

[ https://issues.apache.org/jira/browse/KAFKA-4123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4123: -- Summary: Queryable State returning null for key before all stores in instance have been intialized

[jira] [Created] (KAFKA-4123) RocksDBStore can be marked as open before it has been initialized

Damian Guy created KAFKA-4123: - Summary: RocksDBStore can be marked as open before it has been initialized Key: KAFKA-4123 URL: https://issues.apache.org/jira/browse/KAFKA-4123 Project: Kafka

Re: Queryable state client read guarantees

Hi Mikael, > > Failed to read key hello, org.mkhq.kafka.Topology$StoreUnavailable > > Failed to read key hello, org.mkhq.kafka.Topology$KeyNotFound > > hello -> 10 > > The case where you get KeyNotFound looks like a bug to me. This shouldn't happen. I can see why it might happen and we will

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

ever, if a "cached state" would be used, it would not work any more. > > > -Matthias > > On 09/04/2016 05:58 PM, Damian Guy wrote: > > Hi Matthias, > > > > Thanks for bringing the conversation across to the thread. > > > > I think a main limita

Re: [DISCUSS] KIP-63: Unify store and downstream caching in streams

Hi Matthias, Thanks for bringing the conversation across to the thread. I think a main limitation would be, that you cannot mix the 4 patterns > within a single application anymore (iff you use a "caches state"). If > you have processor with a "cached state" this disables direct usage of >

[jira] [Assigned] (KAFKA-3708) Rethink exception handling in KafkaStreams

[ https://issues.apache.org/jira/browse/KAFKA-3708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-3708: - Assignee: Damian Guy > Rethink exception handling in KafkaStre

[jira] [Created] (KAFKA-4112) Remove alpha quality label from Kafka Streams in docs

Damian Guy created KAFKA-4112: - Summary: Remove alpha quality label from Kafka Streams in docs Key: KAFKA-4112 URL: https://issues.apache.org/jira/browse/KAFKA-4112 Project: Kafka Issue Type

[jira] [Updated] (KAFKA-4104) Queryable state metadata is sometimes invalid

[ https://issues.apache.org/jira/browse/KAFKA-4104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4104: -- Status: Patch Available (was: In Progress) > Queryable state metadata is sometimes inva

[jira] [Work started] (KAFKA-4069) Forward records in context of cache flushing/eviction

[ https://issues.apache.org/jira/browse/KAFKA-4069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4069 started by Damian Guy. - > Forward records in context of cache flushing/evict

[jira] [Updated] (KAFKA-3595) Add capability to specify replication compact option for stream store

[ https://issues.apache.org/jira/browse/KAFKA-3595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-3595: -- Status: Patch Available (was: In Progress) > Add capability to specify replication compact opt

[jira] [Work started] (KAFKA-3595) Add capability to specify replication compact option for stream store

[ https://issues.apache.org/jira/browse/KAFKA-3595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3595 started by Damian Guy. - > Add capability to specify replication compact option for stream st

[jira] [Work started] (KAFKA-4104) Queryable state metadata is sometimes invalid

[ https://issues.apache.org/jira/browse/KAFKA-4104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4104 started by Damian Guy. - > Queryable state metadata is sometimes inva

Re: [VOTE] KIP-77: Improve Kafka Streams Join Semantics

+1 On Mon, 29 Aug 2016 at 18:07 Eno Thereska wrote: > +1 (non-binding) > > > On 29 Aug 2016, at 12:22, Bill Bejeck wrote: > > > > +1 > > > > On Mon, Aug 29, 2016 at 5:50 AM, Matthias J. Sax > > wrote: > > > >> I’d like to

Re: [DISCUSS] KIP-76: Enable getting password from executable rather than passing as plaintext in config files

I'm in agreement with Gwen. An API would be a better solution. Running executables from Kafka is dangerous. On Sat, 27 Aug 2016 at 12:02, Ismael Juma wrote: > Hi Matthias, > > Improve Kafka Streams Join Semantics is not mentioned on the KIP page and > that is probably the

Re: [VOTE] KIP-63: Unify store and downstream caching in streams

+1 On Thu, 25 Aug 2016 at 11:57 Eno Thereska wrote: > Hi folks, > > We'd like to start the vote for KIP-63. At this point the Wiki addresses > all previous questions and we believe the PoC is feature-complete. > > Thanks > Eno >

[jira] [Updated] (KAFKA-4015) Change cleanup.policy config to accept a list of valid policies

[ https://issues.apache.org/jira/browse/KAFKA-4015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-4015: -- Summary: Change cleanup.policy config to accept a list of valid policies (was: Add new cleanup.policy

Re: [DISCUSS] KIP-71 Enable log compaction and deletion to co-exist

t; > Jun > > On Fri, Aug 19, 2016 at 10:01 AM, Damian Guy <damian@gmail.com> wrote: > > > Hi Grant, > > > > I apologise - I seemed to have skipped over Joel's email. > > It is not something we considered, but seems valid. > > I'm not sure if w

Re: [DISCUSS] KIP-71 Enable log compaction and deletion to co-exist

tially indicate precedence. > > Anyway, it is just a thought - it may end up being very confusing for > > users. > > > > @Jason - I agree this could be used to handle offset expiration as well. > We > > can discuss that separately though; and if we do that we woul

Re: [VOTE] KIP:71 Enable log compaction and deletion to co-exist

wrote: > > > +1 (binding) > > > > Thanks, > > -Ewen > > > > On Mon, Aug 15, 2016 at 4:26 PM, Jun Rao <j...@confluent.io> wrote: > > > > > Thanks for the proposal. +1 > > > > > > Jun > > > > > > On Mon, Aug 15,

[jira] [Commented] (KAFKA-4045) Investigate feasibility of hooking into RocksDb's cache

[ https://issues.apache.org/jira/browse/KAFKA-4045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15424099#comment-15424099 ] Damian Guy commented on KAFKA-4045: --- https://github.com/facebook/rocksdb/issues/1282 > Investig

Re: [VOTE] KIP:71 Enable log compaction and deletion to co-exist

Oops, it appears I was a bit trigger happy on closing the vote. Sorry, it will remain open for the requisite 72 hours. Sorry again. Damian On Mon, 15 Aug 2016 at 19:56, Damian Guy <damian@gmail.com> wrote: > And that concludes the vote. KIP-71 has been accepted. > Thanks everyone

Re: [VOTE] KIP:71 Enable log compaction and deletion to co-exist

..@juma.me.uk> > wrote: > > > > > > > Thanks for the KIP, +1 (binding) > > > > > > > > Ismael > > > > > > > > On Mon, Aug 15, 2016 at 2:20 PM, Damian Guy <damian@gmail.com> > > > wrote: > > > >

[VOTE] KIP:71 Enable log compaction and deletion to co-exist

I would like to initiate the voting process for KIP-71 ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-71%3A+Enable+log+compaction+and+deletion+to+co-exist ). This change will add a new cleanup.policy, compact_and_delete, that when enabled will run both compaction and deletion. Thanks,

Re: [DISCUSS] KIP-71 Enable log compaction and deletion to co-exist

ld do a seek to the start of the > window (as soon as KIP-33 is available) and consume from there in order to > read the current state? > > Thanks, > Jason > > On Fri, Aug 12, 2016 at 8:48 AM, Damian Guy <damian@gmail.com> wrote: > > > Thanks Jun

Re: [DISCUSS] KIP-71 Enable log compaction and deletion to co-exist

size-based retention is infinite, I think > we can just leave the KIP as it is. > > Thanks, > > Jun > > On Fri, Aug 12, 2016 at 12:10 AM, Damian Guy <damian@gmail.com> wrote: > > > Hi, > > > > The only concrete example i can think of is a case for

Re: [DISCUSS] KIP-71 Enable log compaction and deletion to co-exist

Hi Jason, It is still not clear to me how adding a callback would help. You could remove the periodic task, but you would still need to scan the entire cache to remove the expired offsets. Am i missing something? Thanks, Damian On 11 August 2016 at 10:00, Damian Guy <damian@gmail.

[jira] [Resolved] (KAFKA-3909) Queryable state for Kafka Streams

[ https://issues.apache.org/jira/browse/KAFKA-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy resolved KAFKA-3909. --- Resolution: Fixed > Queryable state for Kafka Stre

[jira] [Assigned] (KAFKA-3452) Support session windows besides time interval windows

[ https://issues.apache.org/jira/browse/KAFKA-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-3452: - Assignee: Damian Guy > Support session windows besides time interval wind

[jira] [Assigned] (KAFKA-3595) Add capability to specify replication compact option for stream store

[ https://issues.apache.org/jira/browse/KAFKA-3595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-3595: - Assignee: Damian Guy > Add capability to specify replication compact option for stream st

Re: [DISCUSS] KIP-71 Enable log compaction and deletion to co-exist

gt; >> Thanks for the proposal. It makes sense to use time-based deletion >> retention and compaction together, as you mentioned in the KStream. >> >> Is there a use case where we want to combine size-based deletion retention >> and compaction together? >> >>

[jira] [Commented] (KAFKA-3777) Extract the existing LRU cache out of RocksDBStore

[ https://issues.apache.org/jira/browse/KAFKA-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417518#comment-15417518 ] Damian Guy commented on KAFKA-3777: --- We need to support concurrent access (Queryable State), so

Re: [DISCUSS] KIP-71 Enable log compaction and deletion to co-exist

we want to as part of this KIP. Thanks, Damian > > > On Mon, Aug 8, 2016 at 12:41 AM, Damian Guy <damian@gmail.com> wrote: > > > Hi, > > > > We have created KIP 71: Enable log compaction and deletion to co-exist` > > > > https://cwiki.apache.org/confl

[jira] [Updated] (KAFKA-3936) Validate user parameters as early as possible

[ https://issues.apache.org/jira/browse/KAFKA-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-3936: -- Status: Patch Available (was: Open) > Validate user parameters as early as possi

[DISCUSS] KIP-71 Enable log compaction and deletion to co-exist

Hi, We have created KIP 71: Enable log compaction and deletion to co-exist` https://cwiki.apache.org/confluence/display/KAFKA/KIP-71%3A+Enable+log+compaction+and+deletion+to+co-exist Please take a look. Feedback is appreciated. Thank you

[jira] [Updated] (KAFKA-3929) Add prefix for underlying clients configs in StreamConfig

[ https://issues.apache.org/jira/browse/KAFKA-3929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-3929: -- Status: Patch Available (was: In Progress) > Add prefix for underlying clients configs in StreamCon

[jira] [Work started] (KAFKA-3929) Add prefix for underlying clients configs in StreamConfig

[ https://issues.apache.org/jira/browse/KAFKA-3929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3929 started by Damian Guy. - > Add prefix for underlying clients configs in StreamCon

[jira] [Assigned] (KAFKA-3929) Add prefix for underlying clients configs in StreamConfig

[ https://issues.apache.org/jira/browse/KAFKA-3929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-3929: - Assignee: Damian Guy (was: Ishita Mandhan) > Add prefix for underlying clients conf

[jira] [Updated] (KAFKA-3740) Enable configuration of RocksDBStores

[ https://issues.apache.org/jira/browse/KAFKA-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy updated KAFKA-3740: -- Summary: Enable configuration of RocksDBStores (was: Add configs for RocksDBStores) > Ena

[jira] [Work started] (KAFKA-3855) Guard race conditions in TopologyBuilder

[ https://issues.apache.org/jira/browse/KAFKA-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-3855 started by Damian Guy. - > Guard race conditions in TopologyBuil

[jira] [Assigned] (KAFKA-3855) Guard race conditions in TopologyBuilder

[ https://issues.apache.org/jira/browse/KAFKA-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-3855: - Assignee: Damian Guy > Guard race conditions in TopologyBuil

[jira] [Assigned] (KAFKA-3812) State store locking is incorrect

[ https://issues.apache.org/jira/browse/KAFKA-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-3812: - Assignee: Damian Guy (was: Eno Thereska) > State store locking is incorr

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

Hi, The vote is now complete and KIP-67 has been accepted and adopted. Thanks everyone for the input etc. Regards, Damian On Sat, 16 Jul 2016 at 06:53 Damian Guy <damian@gmail.com> wrote: > Hi, > Jay's interpretation is correct. > Thanks, > Damian > > > On Fri

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

upport a new store, would they need to > > change code in streams? > > > > On Fri, Jul 15, 2016 at 3:15 PM, Jay Kreps <j...@confluent.io> wrote: > > > > > Cool, I'm +1 after the updates. > > > > > > -Jay > > > > > > On Fri, Jul

Re: [VOTE] KIP-67: Queryable state for Kafka Streams

eparate guidance for Streams users, > from Streams developers. > > Other than that, all LGTM. > > Guozhang > > On Fri, Jul 15, 2016 at 9:57 AM, Damian Guy <damian@gmail.com> wrote: > > > Hi All, > > > > I've updated the KIP with changes as discusse

<    2   3   4   5   6   7   8   >