[ https://issues.apache.org/jira/browse/BEAM-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ryan Skraba updated BEAM-1847: ------------------------------ Description: Some Beam IOs that expose the ability to turn an unbounded source into a bounded source. For example, KafkaIO uses the underlying Read.from() API to specify the {{withMaxNumRecords}} and/or {{withMaxReadTime}}. If the former is specified, the latter is silently ignored. I would expect that the first stopping condition to be reached (either max records OR max duration) would stop the source. The underlying implementation {{BoundedReadFromUnboundedSource}} has this logic, but it is not supported -in Read.Unbounded- or the Beam IOs that expose this feature. was: Some Beam IOs that expose the ability to turn an unbounded source into a bounded source. For example, KafkaIO uses the underlying Read.from() API to specify the {{withMaxNumRecords}} and/or {{withMaxReadTime}}. If the former is specified, the latter is silently ignored. I would expect that the first stopping condition to be reached (either max records OR max duration) would stop the source. The underlying implementation {{BoundedReadFromUnboundedSource}} has this logic, but it is not supported in Read.Unbounded or the Beam IOs that expose this feature. > Read.Unbounded can't specify both max records and max duration. > --------------------------------------------------------------- > > Key: BEAM-1847 > URL: https://issues.apache.org/jira/browse/BEAM-1847 > Project: Beam > Issue Type: Bug > Components: beam-model > Reporter: Ryan Skraba > Assignee: Ryan Skraba > > Some Beam IOs that expose the ability to turn an unbounded source into a > bounded source. > For example, KafkaIO uses the underlying Read.from() API to specify the > {{withMaxNumRecords}} and/or {{withMaxReadTime}}. If the former is > specified, the latter is silently ignored. > I would expect that the first stopping condition to be reached (either max > records OR max duration) would stop the source. > The underlying implementation {{BoundedReadFromUnboundedSource}} has this > logic, but it is not supported -in Read.Unbounded- or the Beam IOs that > expose this feature. -- This message was sent by Atlassian JIRA (v6.3.15#6346)