[
https://issues.apache.org/jira/browse/BEAM-10529?focusedWorklogId=763682&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-763682
]
ASF GitHub Bot logged work on BEAM-10529:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 28/Apr/22 16:18
Start Date: 28/Apr/22 16:18
Worklog Time Spent: 10m
Work Description: chamikaramj commented on code in PR #17319:
URL: https://github.com/apache/beam/pull/17319#discussion_r861084554
##########
sdks/python/apache_beam/io/external/xlang_kafkaio_it_test.py:
##########
@@ -75,8 +75,8 @@ def build_write_pipeline(self, pipeline):
pipeline
| 'Generate' >> beam.Create(range(NUM_RECORDS)) # pylint:
disable=bad-option-value
| 'MakeKV' >> beam.Map(lambda x:
- (b'', str(x).encode())).with_output_types(
- typing.Tuple[bytes, bytes])
+ (None, str(x).encode())).with_output_types(
Review Comment:
Could we also add a version that uses a non-null non-empty key ?
Issue Time Tracking
-------------------
Worklog Id: (was: 763682)
Time Spent: 26h (was: 25h 50m)
> Kafka XLang fails for ?empty? key/values
> ----------------------------------------
>
> Key: BEAM-10529
> URL: https://issues.apache.org/jira/browse/BEAM-10529
> Project: Beam
> Issue Type: Bug
> Components: cross-language, io-java-kafka
> Reporter: Luke Cwik
> Assignee: John Casey
> Priority: P1
> Fix For: 2.38.0
>
> Time Spent: 26h
> Remaining Estimate: 0h
>
> It looks like the Javadoc for ByteArrayDeserializer and StringDeserializer
> can return null[1, 2] and we aren't using
> NullableCoder.of(ByteArrayCoder.of()) in the expansion[3]. Note that KafkaIO
> does this correctly in its regular coder inference logic[4].
> 1:
> [https://kafka.apache.org/21/javadoc/org/apache/kafka/common/serialization/ByteArrayDeserializer.html#deserialize-java.lang.String-byte:A-|https://kafka.apache.org/21/javadoc/org/apache/kafka/common/serialization/ByteArrayDeserializer.html#deserialize-java.lang.String-byte:A-2:]
> [2:|https://kafka.apache.org/21/javadoc/org/apache/kafka/common/serialization/ByteArrayDeserializer.html#deserialize-java.lang.String-byte:A-2:]
>
> [https://kafka.apache.org/21/javadoc/org/apache/kafka/common/serialization/StringDeserializer.html#deserialize-java.lang.String-byte:A-]
> 3:
> [https://github.com/apache/beam/blob/af2d6b0379d64b522ecb769d88e9e7e7b8900208/sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java#L478]
> 4:
> [https://github.com/apache/beam/blob/af2d6b0379d64b522ecb769d88e9e7e7b8900208/sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/LocalDeserializerProvider.java#L85]
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
