This is an automated email from the ASF dual-hosted git repository.
kabhwan pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.
from 1034815 [SPARK-33572][SQL] Datetime building should fail if the year,
month, ..., second combination is invalid
add e5bb293 [SPARK-32032][SS] Avoid infinite wait in driver because of
KafkaConsumer.poll(long) API
No new revisions were added by this update.
Summary of changes:
docs/ss-migration-guide.md | 5 +
docs/structured-streaming-kafka-integration.md | 20 +
.../spark/sql/kafka010/ConsumerStrategy.scala | 65 ++-
.../org/apache/spark/sql/kafka010/KafkaBatch.scala | 2 +-
.../spark/sql/kafka010/KafkaOffsetReader.scala | 601 ++-------------------
...etReader.scala => KafkaOffsetReaderAdmin.scala} | 284 +++++-----
...eader.scala => KafkaOffsetReaderConsumer.scala} | 39 +-
.../apache/spark/sql/kafka010/KafkaRelation.scala | 2 +-
.../spark/sql/kafka010/KafkaSourceProvider.scala | 6 +-
.../spark/sql/kafka010/ConsumerStrategySuite.scala | 147 +++++
.../sql/kafka010/KafkaMicroBatchSourceSuite.scala | 42 +-
.../sql/kafka010/KafkaOffsetReaderSuite.scala | 95 +++-
.../spark/sql/kafka010/KafkaRelationSuite.scala | 47 +-
.../org/apache/spark/sql/internal/SQLConf.scala | 13 +
14 files changed, 542 insertions(+), 826 deletions(-)
copy
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/{KafkaOffsetReader.scala
=> KafkaOffsetReaderAdmin.scala} (73%)
copy
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/{KafkaOffsetReader.scala
=> KafkaOffsetReaderConsumer.scala} (96%)
create mode 100644
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/ConsumerStrategySuite.scala
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
