GitHub user boneill42 opened a pull request:
https://github.com/apache/spark/pull/13054
Upgrades protobuf dependency to 2.6.1 for the new version of KCL, andâ¦
## What changes were proposed in this pull request?
When running with Kinesis Consumer Library (KCL), against a stream that
contains aggregated data, the KCL needs access to protobuf to de-aggregate the
records. Without this patch, that results in the following error message:
```
Caused by: java.lang.ClassNotFoundException:
com.google.protobuf.ProtocolStringList
```
This PR upgrades the protobuf dependency within the kinesis-asl-assembly,
and relocates that package (as not to conflict with Spark's use of 2.5.0),
which fixes the above CNFE.
## How was this patch tested?
Used kinesis word count example against a stream containing aggregated data.
See: SPARK-14421
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/boneill42/spark protobuf-relocation-for-kcl
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/13054.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #13054
----
commit 0a532c3e1f4afa496e2181ebdc5568914d463ae5
Author: Brian O'Neill <[email protected]>
Date: 2016-05-11T17:42:02Z
Upgrades protobuf dependency to 2.6.1 for the new version of KCL, and
relocates that dependency as not to conflict with Spark's use of 2.5.0.
See: SPARK-14421
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
