[GitHub] spark issue #15102: [SPARK-17346][SQL] Add Kafka source for Structured Strea...

2016-09-23 Thread jwbear
Github user jwbear commented on the issue: https://github.com/apache/spark/pull/15102 Which offset does getOffset() return one from the partition or something you created? Because it looked like when you were hashing them, you were returning partition offsets. --- If your project

[GitHub] spark issue #15102: [SPARK-17346][SQL] Add Kafka source for Structured Strea...

2016-09-22 Thread jwbear
Github user jwbear commented on the issue: https://github.com/apache/spark/pull/15102 Just curious looking at this, if you are comparing "sequential" offsets across partitions a rebalance would definitely affect this and, unless something has changed, it probably not a go