[
https://issues.apache.org/jira/browse/KAFKA-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312912#comment-15312912
]
Greg Fodor commented on KAFKA-3758:
---
Hey, we're running 16 threads -- for this job we have 25 topics,
[
https://issues.apache.org/jira/browse/KAFKA-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312913#comment-15312913
]
Greg Fodor commented on KAFKA-3758:
---
also, if we did not run at an elevated number of threads, we were
Greg Fodor created KAFKA-3785:
-
Summary: Fetcher spending unnecessary time during metrics recording
Key: KAFKA-3785
URL: https://issues.apache.org/jira/browse/KAFKA-3785
Project: Kafka
Issue
[
https://issues.apache.org/jira/browse/KAFKA-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313434#comment-15313434
]
Greg Fodor commented on KAFKA-3770:
---
Cut a new PR since I screwed up the rebase on the previous one.
[
https://issues.apache.org/jira/browse/KAFKA-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Fodor updated KAFKA-3785:
--
Component/s: consumer
> Fetcher spending unnecessary time during metrics recording
>
[
https://issues.apache.org/jira/browse/KAFKA-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313461#comment-15313461
]
Greg Fodor commented on KAFKA-3785:
---
https://github.com/apache/kafka/pull/1464
> Fetcher spending
[
https://issues.apache.org/jira/browse/KAFKA-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15326778#comment-15326778
]
Greg Fodor commented on KAFKA-3769:
---
Discussion/resolution moved to:
[
https://issues.apache.org/jira/browse/KAFKA-3811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324847#comment-15324847
]
Greg Fodor commented on KAFKA-3811:
---
I've also attached a screenshot of YourKit of the relevant call
[
https://issues.apache.org/jira/browse/KAFKA-3811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Fodor updated KAFKA-3811:
--
Attachment: screenshot-1.png
> Introduce Kafka Streams metrics recording levels
>
[
https://issues.apache.org/jira/browse/KAFKA-3811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Fodor updated KAFKA-3811:
--
Attachment: Muon-Snapshot.zip
> Introduce Kafka Streams metrics recording levels
>
[
https://issues.apache.org/jira/browse/KAFKA-3811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Fodor updated KAFKA-3811:
--
Attachment: Muon-latency.zip
> Introduce Kafka Streams metrics recording levels
>
[
https://issues.apache.org/jira/browse/KAFKA-3811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324838#comment-15324838
]
Greg Fodor commented on KAFKA-3811:
---
Hey [~aartigupta], I ran an attached yourkit profiler to one of our
[
https://issues.apache.org/jira/browse/KAFKA-3811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324869#comment-15324869
]
Greg Fodor commented on KAFKA-3811:
---
Also, I've attached a screenshot + snapshot of a second run where I
[
https://issues.apache.org/jira/browse/KAFKA-3811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Fodor updated KAFKA-3811:
--
Attachment: screenshot-latency.png
> Introduce Kafka Streams metrics recording levels
>
[
https://issues.apache.org/jira/browse/KAFKA-3811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Fodor updated KAFKA-3811:
--
Summary: Introduce Kafka Streams metrics recording levels (was: Introduce
KStream metrics recording
Greg Fodor created KAFKA-3811:
-
Summary: Introduce KStream metrics recording levels
Key: KAFKA-3811
URL: https://issues.apache.org/jira/browse/KAFKA-3811
Project: Kafka
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/KAFKA-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Fodor reopened KAFKA-3758:
---
> KStream job fails to recover after Kafka broker stopped
>
[
https://issues.apache.org/jira/browse/KAFKA-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15305501#comment-15305501
]
Greg Fodor commented on KAFKA-3758:
---
Ah yes this looks like the same issue, thanks!
> KStream job fails
[
https://issues.apache.org/jira/browse/KAFKA-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Fodor resolved KAFKA-3758.
---
Resolution: Duplicate
> KStream job fails to recover after Kafka broker stopped
>
[
https://issues.apache.org/jira/browse/KAFKA-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15305503#comment-15305503
]
Greg Fodor commented on KAFKA-3758:
---
Oh, actually, I'm not so sure. This was not during an unclean
Greg Fodor created KAFKA-3770:
-
Summary: KStream job should be able to specify linger.ms
Key: KAFKA-3770
URL: https://issues.apache.org/jira/browse/KAFKA-3770
Project: Kafka
Issue Type:
[
https://issues.apache.org/jira/browse/KAFKA-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306371#comment-15306371
]
Greg Fodor commented on KAFKA-3770:
---
https://github.com/apache/kafka/pull/1448
> KStream job should be
[
https://issues.apache.org/jira/browse/KAFKA-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306306#comment-15306306
]
Greg Fodor commented on KAFKA-3769:
---
https://github.com/apache/kafka/pull/1447
> KStream job spending
[
https://issues.apache.org/jira/browse/KAFKA-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306318#comment-15306318
]
Greg Fodor commented on KAFKA-3769:
---
Actually, additionally it looks like the code path for fetching
Greg Fodor created KAFKA-3769:
-
Summary: KStream job spending 60% of time writing metrics
Key: KAFKA-3769
URL: https://issues.apache.org/jira/browse/KAFKA-3769
Project: Kafka
Issue Type: Bug
[
https://issues.apache.org/jira/browse/KAFKA-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306381#comment-15306381
]
Greg Fodor commented on KAFKA-3769:
---
Consider the PR a first pass, please advise on how we may want to
[
https://issues.apache.org/jira/browse/KAFKA-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306321#comment-15306321
]
Greg Fodor commented on KAFKA-3769:
---
It seems it might be desirable to have a way to just flip off some
[
https://issues.apache.org/jira/browse/KAFKA-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15308306#comment-15308306
]
Greg Fodor commented on KAFKA-3758:
---
No, the kstream job was running across 2 servers, and the kafka
Greg Fodor created KAFKA-3758:
-
Summary: KStream job fails to recover after Kafka broker stopped
Key: KAFKA-3758
URL: https://issues.apache.org/jira/browse/KAFKA-3758
Project: Kafka
Issue Type:
[
https://issues.apache.org/jira/browse/KAFKA-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Fodor updated KAFKA-3758:
--
Attachment: muon.log.1.gz
> KStream job fails to recover after Kafka broker stopped
>
[
https://issues.apache.org/jira/browse/KAFKA-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15302542#comment-15302542
]
Greg Fodor commented on KAFKA-3758:
---
Also, the log is truncated at the top to the point where we shut
[
https://issues.apache.org/jira/browse/KAFKA-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311179#comment-15311179
]
Greg Fodor commented on KAFKA-3769:
---
Thanks Jay!
Guozhang, what are your thoughts on instead of trying
[
https://issues.apache.org/jira/browse/KAFKA-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198422#comment-15198422
]
Greg Fodor edited comment on KAFKA-3360 at 3/16/16 11:39 PM:
-
Apologies as I
[
https://issues.apache.org/jira/browse/KAFKA-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236331#comment-15236331
]
Greg Fodor commented on KAFKA-3542:
---
I might just not be understanding, but a simple example is if I
[
https://issues.apache.org/jira/browse/KAFKA-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236512#comment-15236512
]
Greg Fodor edited comment on KAFKA-3544 at 4/12/16 3:05 AM:
Not sure of the
[
https://issues.apache.org/jira/browse/KAFKA-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236512#comment-15236512
]
Greg Fodor commented on KAFKA-3544:
---
Not sure of the best way to share the topology. Here's the relevant
[
https://issues.apache.org/jira/browse/KAFKA-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236512#comment-15236512
]
Greg Fodor edited comment on KAFKA-3544 at 4/12/16 3:04 AM:
Not sure of the
[
https://issues.apache.org/jira/browse/KAFKA-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236512#comment-15236512
]
Greg Fodor edited comment on KAFKA-3544 at 4/12/16 3:04 AM:
Not sure of the
[
https://issues.apache.org/jira/browse/KAFKA-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236339#comment-15236339
]
Greg Fodor commented on KAFKA-3543:
---
Ah interesting, my assumption was that the items emitted from the
[
https://issues.apache.org/jira/browse/KAFKA-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236417#comment-15236417
]
Greg Fodor commented on KAFKA-3542:
---
great. feel free to close.
> Add "repartition (+ join)" operations
[
https://issues.apache.org/jira/browse/KAFKA-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236374#comment-15236374
]
Greg Fodor commented on KAFKA-3542:
---
Right, this map approach is what I am doing right now before all of
[
https://issues.apache.org/jira/browse/KAFKA-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236427#comment-15236427
]
Greg Fodor commented on KAFKA-3543:
---
Just something akin to a flatTransform where the output of the
[
https://issues.apache.org/jira/browse/KAFKA-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236343#comment-15236343
]
Greg Fodor commented on KAFKA-3542:
---
Ah, I may understand what you're getting at here -- to do the
[
https://issues.apache.org/jira/browse/KAFKA-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236375#comment-15236375
]
Greg Fodor commented on KAFKA-3543:
---
That makes sense, thanks! It may be useful to provide a helper
[
https://issues.apache.org/jira/browse/KAFKA-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Fodor updated KAFKA-3543:
--
Summary: Allow a variant of transform() which can emit multiple values
(was: Allow a variant of
Greg Fodor created KAFKA-3543:
-
Summary: Allow a variant of transform() which allows emitting
multiple values
Key: KAFKA-3543
URL: https://issues.apache.org/jira/browse/KAFKA-3543
Project: Kafka
[
https://issues.apache.org/jira/browse/KAFKA-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236121#comment-15236121
]
Greg Fodor edited comment on KAFKA-3543 at 4/11/16 10:40 PM:
-
Also note that
Greg Fodor created KAFKA-3544:
-
Summary: Missing topics on startup
Key: KAFKA-3544
URL: https://issues.apache.org/jira/browse/KAFKA-3544
Project: Kafka
Issue Type: Bug
Components:
Greg Fodor created KAFKA-3545:
-
Summary: Generalized Serdes for List/Map
Key: KAFKA-3545
URL: https://issues.apache.org/jira/browse/KAFKA-3545
Project: Kafka
Issue Type: Improvement
Greg Fodor created KAFKA-3542:
-
Summary: Add "repartition (+ join)" operations to streams
Key: KAFKA-3542
URL: https://issues.apache.org/jira/browse/KAFKA-3542
Project: Kafka
Issue Type:
[
https://issues.apache.org/jira/browse/KAFKA-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198422#comment-15198422
]
Greg Fodor commented on KAFKA-3360:
---
Apologies as I did not really know the best place to put this, and
Greg Fodor created KAFKA-3619:
-
Summary: State lock file handle leaks
Key: KAFKA-3619
URL: https://issues.apache.org/jira/browse/KAFKA-3619
Project: Kafka
Issue Type: Bug
Components:
[
https://issues.apache.org/jira/browse/KAFKA-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Fodor updated KAFKA-3619:
--
Description:
The .lock files in the state store directories do not seem to be having their
file
[
https://issues.apache.org/jira/browse/KAFKA-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297477#comment-15297477
]
Greg Fodor commented on KAFKA-3745:
---
Yep, I admit this is definitely not the most common case. But when
[
https://issues.apache.org/jira/browse/KAFKA-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297250#comment-15297250
]
Greg Fodor commented on KAFKA-3745:
---
sure. we are left joining a KTable against a KStream. The entry in
Greg Fodor created KAFKA-3745:
-
Summary: Consider adding join key to ValueJoiner interface
Key: KAFKA-3745
URL: https://issues.apache.org/jira/browse/KAFKA-3745
Project: Kafka
Issue Type:
[
https://issues.apache.org/jira/browse/KAFKA-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297733#comment-15297733
]
Greg Fodor commented on KAFKA-3745:
---
Yes, the join key needs to be added to the final joined record. I
[
https://issues.apache.org/jira/browse/KAFKA-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297745#comment-15297745
]
Greg Fodor commented on KAFKA-3745:
---
Actually this seems like a good enough solution, I forgot to
[
https://issues.apache.org/jira/browse/KAFKA-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15240723#comment-15240723
]
Greg Fodor commented on KAFKA-3544:
---
Ah this makes sense. Fortunately thanks to the other responses
[
https://issues.apache.org/jira/browse/KAFKA-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Fodor resolved KAFKA-3544.
---
Resolution: Not A Problem
> Missing topics on startup
> -
>
>
[
https://issues.apache.org/jira/browse/KAFKA-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416351#comment-15416351
]
Greg Fodor commented on KAFKA-3752:
---
Oh apologies for mis-reading the ticket, but in our case it's a
[
https://issues.apache.org/jira/browse/KAFKA-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415664#comment-15415664
]
Greg Fodor commented on KAFKA-3752:
---
Hey [~guozhang], we're hitting this issue as well during a
[
https://issues.apache.org/jira/browse/KAFKA-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Fodor resolved KAFKA-3770.
---
Resolution: Fixed
Fixed via KAFKA-3786
> KStream job should be able to specify linger.ms
>
Greg Fodor created KAFKA-4043:
-
Summary: User-defined handler for topology restart
Key: KAFKA-4043
URL: https://issues.apache.org/jira/browse/KAFKA-4043
Project: Kafka
Issue Type: Improvement
Greg Fodor created KAFKA-4120:
-
Summary: byte[] keys in RocksDB state stores do not work as
expected
Key: KAFKA-4120
URL: https://issues.apache.org/jira/browse/KAFKA-4120
Project: Kafka
Issue
[
https://issues.apache.org/jira/browse/KAFKA-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464171#comment-15464171
]
Greg Fodor edited comment on KAFKA-4120 at 9/5/16 5:34 PM:
---
We were able to work
[
https://issues.apache.org/jira/browse/KAFKA-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15468374#comment-15468374
]
Greg Fodor commented on KAFKA-3769:
---
It's just the sensor calls inside of Selector, not Kafka Streams
[
https://issues.apache.org/jira/browse/KAFKA-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15463861#comment-15463861
]
Greg Fodor commented on KAFKA-3769:
---
I've done some additional profiling and I have found that this
[
https://issues.apache.org/jira/browse/KAFKA-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464171#comment-15464171
]
Greg Fodor commented on KAFKA-4120:
---
We were able to work around it by just creating a proper Avro class
[
https://issues.apache.org/jira/browse/KAFKA-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Fodor resolved KAFKA-4043.
---
Resolution: Not A Problem
> User-defined handler for topology restart
>
[
https://issues.apache.org/jira/browse/KAFKA-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15472271#comment-15472271
]
Greg Fodor commented on KAFKA-4043:
---
Ah, that should work for us. Thanks!
> User-defined handler for
[
https://issues.apache.org/jira/browse/KAFKA-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433738#comment-15433738
]
Greg Fodor commented on KAFKA-3758:
---
Excited to try this, as our state stores have grown, it's become
[
https://issues.apache.org/jira/browse/KAFKA-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15443912#comment-15443912
]
Greg Fodor commented on KAFKA-4043:
---
I suppose it looks like there's also not a handler for when the
[
https://issues.apache.org/jira/browse/KAFKA-4113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587991#comment-15587991
]
Greg Fodor commented on KAFKA-4113:
---
I guess what I would argue is that KStreamBuilder#table should have
[
https://issues.apache.org/jira/browse/KAFKA-4113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587966#comment-15587966
]
Greg Fodor commented on KAFKA-4113:
---
Having played around with Kafka Streams for a while now, I am still
Greg Fodor created KAFKA-4317:
-
Summary: RocksDB checkpoint files lost on kill -9
Key: KAFKA-4317
URL: https://issues.apache.org/jira/browse/KAFKA-4317
Project: Kafka
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/KAFKA-4113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15593018#comment-15593018
]
Greg Fodor commented on KAFKA-4113:
---
Oh, so it should be doing exactly what makes sense to me -- I am on
[
https://issues.apache.org/jira/browse/KAFKA-4113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15627097#comment-15627097
]
Greg Fodor commented on KAFKA-4113:
---
Hey [~guozhang], I have been able to reproduce a bootstrapping
[
https://issues.apache.org/jira/browse/KAFKA-4281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15560765#comment-15560765
]
Greg Fodor commented on KAFKA-4281:
---
PR: https://github.com/apache/kafka/pull/1998
If this approach
Greg Fodor created KAFKA-4281:
-
Summary: Should be able to forward aggregation values immediately
Key: KAFKA-4281
URL: https://issues.apache.org/jira/browse/KAFKA-4281
Project: Kafka
Issue Type:
[
https://issues.apache.org/jira/browse/KAFKA-3545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15583149#comment-15583149
]
Greg Fodor commented on KAFKA-3545:
---
Yes
> Generalized Serdes for List/Map
>
[
https://issues.apache.org/jira/browse/KAFKA-3545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Fodor resolved KAFKA-3545.
---
Resolution: Fixed
Fix Version/s: 0.10.1.0
> Generalized Serdes for List/Map
>
Greg Fodor created KAFKA-5571:
-
Summary: Possible deadlock during shutdown in setState in kafka
streams 10.2
Key: KAFKA-5571
URL: https://issues.apache.org/jira/browse/KAFKA-5571
Project: Kafka
83 matches
Mail list logo
