Re: Missing kafkaOffset metric in storm-kafka-client trident spout

2018-11-23 Thread Stig Rohde Døssing
Your first link points to storm-kafka, while your second link points to storm-kafka-client. Storm-kafka-client is intended as a replacement for storm-kafka, but the two codebases are unrelated. The metric is missing because no one has implemented metrics for the new Trident spout yet. There's an

Missing kafkaOffset metric in storm-kafka-client trident spout

2018-11-23 Thread Jonathan Munz
Hi, I was looking to upgrade from 1.0.5 to 1.2.2. One thing I noticed is that the 'kafkaOffset' metric is no longer being reported by the Kafka Trident spout. In 1.0.5 it was being registered here:

Re: Fields grouping consistency for separate streams

2018-11-23 Thread Stig Rohde Døssing
See https://github.com/apache/storm/blob/21bb1388414d373572779289edc785c7e5aa52aa/storm-client/src/jvm/org/apache/storm/daemon/GrouperFactory.java#L174 and https://github.com/apache/storm/blob/21bb1388414d373572779289edc785c7e5aa52aa/storm-client/src/jvm/org/apache/storm/utils/TupleUtils.java#L38

Re: how to set timestamp field in trident window?

2018-11-23 Thread Stig Rohde Døssing
Also you should be aware of https://issues.apache.org/jira/browse/STORM-3280 if you intend to use Trident based windowing. Den fre. 23. nov. 2018 kl. 12.10 skrev Stig Rohde Døssing < stigdoess...@gmail.com>: > I don't believe you can. The Trident windowing code seems to use >

Re: how to set timestamp field in trident window?

2018-11-23 Thread Stig Rohde Døssing
I don't believe you can. The Trident windowing code seems to use System.currentTimeMillis without allowing you to set a timestamp extractor. If you want to add this functionality, you should look at

Fields grouping consistency for separate streams

2018-11-23 Thread bogun . dmitriy
Hi. I have a question, regarding fields grouping. In documentation we have: ``` Fields grouping: The stream is partitioned by the fields specified in the grouping. For example, if the stream is grouped by the "user-id" field, tuples with the same "user-id" will always go to the same task, but