[
https://issues.apache.org/jira/browse/BEAM-6756?focusedWorklogId=345341&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-345341
]
ASF GitHub Bot logged work on BEAM-6756:
----------------------------------------
Author: ASF GitHub Bot
Created on: 18/Nov/19 15:14
Start Date: 18/Nov/19 15:14
Worklog Time Spent: 10m
Work Description: TheNeuralBit commented on pull request #10003:
[BEAM-6756] Create Iterable type for Schema
URL: https://github.com/apache/beam/pull/10003#discussion_r347420434
##########
File path:
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/utils/StaticSchemaInference.java
##########
@@ -139,6 +129,23 @@ public static Schema schemaFromClass(
return FieldType.DATETIME;
} else if (type.isSubtypeOf(TypeDescriptor.of(ByteBuffer.class))) {
return FieldType.BYTES;
+ } else if (type.isSubtypeOf(TypeDescriptor.of(Iterable.class))) {
+ TypeDescriptor<Iterable<?>> iterable = type.getSupertype(Iterable.class);
+ if (iterable.getType() instanceof ParameterizedType) {
+ ParameterizedType ptype = (ParameterizedType) iterable.getType();
+ java.lang.reflect.Type[] params = ptype.getActualTypeArguments();
+ checkArgument(params.length == 1);
+ // TODO: should this be AbstractCollection?
Review comment:
Should this be tracked in a JIRA, or can it be resolved now?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 345341)
Time Spent: 1h 20m (was: 1h 10m)
> Support lazy iterables in schemas
> ---------------------------------
>
> Key: BEAM-6756
> URL: https://issues.apache.org/jira/browse/BEAM-6756
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-java-core
> Reporter: Reuven Lax
> Assignee: Reuven Lax
> Priority: Major
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> The iterables returned by GroupByKey and CoGroupByKey are lazy; this allows a
> runner to page data into memory if the full iterable is too large. We
> currently don't support this in Schemas, so the Schema Group and CoGroup
> transforms materialize all data into memory. We should add support for this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
