cchighman commented on pull request #28841:
URL: https://github.com/apache/spark/pull/28841#issuecomment-650728661


   @HeartSaVioR 
   With_startingOffsetByTimestamp_, you have the ability to indicate start/end 
offsets per topic such as TopicA or TopicB.  If this concept were applied to a 
file data source with the underlying intent that each file name represented a 
topic, problems begin to emerge.  For example, if there are multiple files, 
they would have different file names,  different file names may imply a new 
topic.
   
   This would mean a naming convention would have to be followed if you were 
reading from a file data source by _path_ since that path could have different 
file names...or topics...and you couldn't consider the whole as one stream.
   
   
   Thoughts?
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to