[
https://issues.apache.org/jira/browse/BEAM-10543?focusedWorklogId=466111&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-466111
]
ASF GitHub Bot logged work on BEAM-10543:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 04/Aug/20 08:47
Start Date: 04/Aug/20 08:47
Worklog Time Spent: 10m
Work Description: piotr-szuberski commented on pull request #12325:
URL: https://github.com/apache/beam/pull/12325#issuecomment-668469132
@ihji @chamikaramj ping
> Thanks, looks good to me overall.
>
> Is there any way to share the pipeline codes in `CrossLanguageKafkaIOTest`
and `CrossLanguageKafkaIO`? Could we share the pipeline codes and use
`max_num_records` only in `CrossLanguageKafkaIOTest`? The internal test only
checks `sum_counter` so you could keep any value in the `PCollection` afterward
and assert it.
It required a small modification (triggers don't work with bounded source -
max_num_records changes unbounded to bounded) and sum_counter returns [None],
so I use just ReadFromKafka and skip the rest of the read pipeline.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 466111)
Time Spent: 2h 50m (was: 2h 40m)
> Kafka Cross-language configuration lacks of few parameters
> ----------------------------------------------------------
>
> Key: BEAM-10543
> URL: https://issues.apache.org/jira/browse/BEAM-10543
> Project: Beam
> Issue Type: Improvement
> Components: cross-language, io-py-kafka
> Affects Versions: Not applicable
> Reporter: Piotr Szuberski
> Assignee: Piotr Szuberski
> Priority: P2
> Fix For: Not applicable
>
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> Kafka Cross-language configuration lacks of some attributes like
> maxNumRecords, maxReadTime, startReadTime etc.
> For example maxNumRecords would make it possible to write an integration test
> for read transform.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
