[
https://issues.apache.org/jira/browse/BEAM-11657?focusedWorklogId=542185&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-542185
]
ASF GitHub Bot logged work on BEAM-11657:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 26/Jan/21 13:49
Start Date: 26/Jan/21 13:49
Worklog Time Spent: 10m
Work Description: scwhittle commented on pull request #13782:
URL: https://github.com/apache/beam/pull/13782#issuecomment-767552230
Another idea I had was to change the deserialize method to attempt calling
the kafka deserialize method taking headers and catch an exception if the
method is not found during execution (due to previous kafka version).
This could be cached so that future attempts would avoid the exception and
just call the method without headers. We could remove the cache map of
deserializer -> bool in that case.
Thoughts?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 542185)
Time Spent: 50m (was: 40m)
> Kafka read performance regression due to added header support
> -------------------------------------------------------------
>
> Key: BEAM-11657
> URL: https://issues.apache.org/jira/browse/BEAM-11657
> Project: Beam
> Issue Type: Bug
> Components: io-java-kafka
> Reporter: Sam Whittle
> Assignee: Sam Whittle
> Priority: P2
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Support for headers in KafkaIO reads was recently added:
> https://issues.apache.org/jira/browse/BEAM-10865
> This introduced several reflection calls into the path of advancing
> KafkaUnboundedReader. While separately running benchmarks, I noticed this
> regression.
> Calls currently come from:
> ConsumerSpEL.hasHeaders -> can be cached similar to other booleans
> deserialize key and value methods -> could be avoided in cases where headers
> are not being examined (at a minimum can be avoided for known coders like
> ByteArrayDeserializer)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
