[
https://issues.apache.org/jira/browse/BEAM-6756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16796558#comment-16796558
]
Reuven Lax commented on BEAM-6756:
----------------------------------
this would require adding a new ITERABLE schema type (would not support
operations such as size()).
> Support lazy iterables in schemas
> ---------------------------------
>
> Key: BEAM-6756
> URL: https://issues.apache.org/jira/browse/BEAM-6756
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-java-core
> Reporter: Reuven Lax
> Assignee: Reuven Lax
> Priority: Major
>
> The iterables returned by GroupByKey and CoGroupByKey are lazy; this allows a
> runner to page data into memory if the full iterable is too large. We
> currently don't support this in Schemas, so the Schema Group and CoGroup
> transforms materialize all data into memory. We should add support for this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
