GitHub user szczeles opened a pull request:
https://github.com/apache/spark/pull/14540
[SPARK-16950] [PySpark] fromOffsets parameter support in
KafkaUtils.createDirectStream for python3
## What changes were proposed in this pull request?
Ability to use KafkaUtils.createDirectStream with starting offsets in
python 3 by using java.lang.Number instead of Long during param mapping in
scala helper. This allows py4j to pass Integer or Long to the map and resolves
ClassCastException problems.
## How was this patch tested?
unit tests
@jerryshao - could you please look at this PR?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/szczeles/spark kafka_pyspark
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14540.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14540
----
commit d2a402761bce28ed9b76dc916fa1c8e57ba5cad0
Author: Mariusz Strzelecki <[email protected]>
Date: 2016-08-08T12:46:20Z
[SPARK-16950] fromOffsets parameter support in
KafkaUtils.createDirectStream for python3
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
