[jira] [Commented] (STORM-563) Kafka Spout doesn't pick up from the beginning of the queue unless forceFromStart specified

[ASF GitHub Bot (JIRA)]

    [ 
https://issues.apache.org/jira/browse/STORM-563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389225#comment-14389225
 ] 

ASF GitHub Bot commented on STORM-563:
--------------------------------------

Github user ptgoetz commented on a diff in the pull request:

    https://github.com/apache/storm/pull/493#discussion_r27512587
  
    --- Diff: external/storm-kafka/README.md ---
    @@ -120,6 +120,23 @@ spoutConf.scheme = new SchemeAsMultiScheme(new 
StringScheme());
     OpaqueTridentKafkaSpout spout = new OpaqueTridentKafkaSpout(spoutConf);
     ```
     
    +### How KafkaSpout stores offsets of a kafka topic and recovers incase of 
failures
    +
    +As shown in the above KafkaConfig properties , user can control where in 
the topic they can start reading by setting **KafkaConfig.startOffsetTime.**
    +
    +There are two options **kafka.api.OffsetRequest.EarliestTime()** which 
makes the KafkaSpout to read from the begining of the topic and 
    --- End diff --
    
    I would also document the actual values of `EarliestTime()` (`-2`) and 
`LatestTime()` (`-1`), and that it can also be set to a point in time (a la 
`System.currentTimeMillis()`).
    
    My reasoning behind documenting the values (as opposed to the kafka API 
constants) is that the start offset time is likely to be specified via 
configuration (i.e. outside java code). Either that, or add spout constants 
that would get evaluated to `EarliestTime()`/`LatestTime()` if for some reason 
those values were ever changed in the Kafka API -- that seems like a less 
"leaky" solution.
    



> Kafka Spout doesn't pick up from the beginning of the queue unless 
> forceFromStart specified
> -------------------------------------------------------------------------------------------
>
>                 Key: STORM-563
>                 URL: https://issues.apache.org/jira/browse/STORM-563
>             Project: Apache Storm
>          Issue Type: Bug
>            Reporter: Sriharsha Chintalapani
>            Assignee: Sriharsha Chintalapani
>
> KafkaUtil.getOffset starts from LatestTime unless forceFromStart specified. 
> It should pick this from KafkaConfig.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)