[
https://issues.apache.org/jira/browse/BEAM-11657?focusedWorklogId=543629&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-543629
]
ASF GitHub Bot logged work on BEAM-11657:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 28/Jan/21 13:10
Start Date: 28/Jan/21 13:10
Worklog Time Spent: 10m
Work Description: scwhittle commented on pull request #13782:
URL: https://github.com/apache/beam/pull/13782#issuecomment-769042068
PTAL, I updated to better match ProducerSpEL by using static initializer and
static methods instead of a singleton object. I removed the map of
deserializer to if it examines headers by instead just delegating to calling
either the method with our without headers provided based upon the existence of
the method in static initialization.
From my understanding this should work because the SDK is compiled with a
kafka version supporting the header method, and if during execution the method
does not exist it is ok to have the implementation reference it as long as it
is not executed. This matches how ConsumerSpEL.getRecordTimestamp works.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 543629)
Time Spent: 1h 10m (was: 1h)
> Kafka read performance regression due to added header support
> -------------------------------------------------------------
>
> Key: BEAM-11657
> URL: https://issues.apache.org/jira/browse/BEAM-11657
> Project: Beam
> Issue Type: Bug
> Components: io-java-kafka
> Reporter: Sam Whittle
> Assignee: Sam Whittle
> Priority: P2
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Support for headers in KafkaIO reads was recently added:
> https://issues.apache.org/jira/browse/BEAM-10865
> This introduced several reflection calls into the path of advancing
> KafkaUnboundedReader. While separately running benchmarks, I noticed this
> regression.
> Calls currently come from:
> ConsumerSpEL.hasHeaders -> can be cached similar to other booleans
> deserialize key and value methods -> could be avoided in cases where headers
> are not being examined (at a minimum can be avoided for known coders like
> ByteArrayDeserializer)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
