charlespnh commented on issue #35099:
URL: https://github.com/apache/beam/issues/35099#issuecomment-2923134231
FYI the working pipeline:
```
pipeline:
type: chain
transforms:
- type: ReadFromText
name: Read from GCS
config:
path: gs://dataflow-samples/shakespeare/kinglear.txt
- type: MapToFields
name: Build Kafka records
config:
language: python
fields:
value:
callable: |
def func(row):
return row.line.encode('utf-8')
output_type: bytes
- type: WriteToKafka
name: Write to Kafka
config:
format: RAW
topic: my-topic
bootstrap_servers: kafka:9092
```
In the MapToField transform, specifying 2 fields in this example will give
error -
https://github.com/apache/beam/blob/master/sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaWriteSchemaTransformProvider.java#L171-L180
I thought the Kafka write transform was expecting KV records...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
