[
https://issues.apache.org/jira/browse/BEAM-7029?focusedWorklogId=229613&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-229613
]
ASF GitHub Bot logged work on BEAM-7029:
----------------------------------------
Author: ASF GitHub Bot
Created on: 18/Apr/19 09:43
Start Date: 18/Apr/19 09:43
Worklog Time Spent: 10m
Work Description: mxm commented on pull request #8322: [BEAM-7029] Add
KafkaIO.Write as an external transform
URL: https://github.com/apache/beam/pull/8322#discussion_r276591492
##########
File path: sdks/java/harness/build.gradle
##########
@@ -55,6 +55,8 @@ dependencies {
compile library.java.joda_time
compile library.java.slf4j_api
compile library.java.vendored_grpc_1_13_1
+ // For executing KafkaIO, e.g. as an external transform
+ compile project(path: ":beam-sdks-java-io-kafka", configuration: "shadow")
Review comment:
Thanks for mentioning this. I was also thinking about an option to stage
jars from Python. I think it would be nicer if the Jars were automatically
provided for the user. The user has to download the KafkaIO jar now and supply
its path?
The reason why we can't add KafkaIO to the SDK Harness is that Kafka depends
on the direct-runner which has a dependency on the SDK Harness. I'm not sure
why the direct-runner has to depend on the SDK harness.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 229613)
Time Spent: 9h 50m (was: 9h 40m)
> Support KafkaIO to be configured externally for use with other SDKs
> -------------------------------------------------------------------
>
> Key: BEAM-7029
> URL: https://issues.apache.org/jira/browse/BEAM-7029
> Project: Beam
> Issue Type: New Feature
> Components: io-java-kafka, runner-flink, sdk-py-core
> Reporter: Maximilian Michels
> Assignee: Maximilian Michels
> Priority: Major
> Time Spent: 9h 50m
> Remaining Estimate: 0h
>
> As of BEAM-6730, we can externally configure existing transforms from SDKs.
> We should add more useful transforms then just {{GenerateSequence}}.
> {{KafkaIO}} is a good candidate.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
