[
https://issues.apache.org/jira/browse/BEAM-10097?focusedWorklogId=446076&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-446076
]
ASF GitHub Bot logged work on BEAM-10097:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 15/Jun/20 20:20
Start Date: 15/Jun/20 20:20
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #11821:
URL: https://github.com/apache/beam/pull/11821#discussion_r440422483
##########
File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/View.java
##########
@@ -157,7 +166,10 @@ private View() {}
* PCollectionView} mapping each window to a {@link List} containing all of
the elements in the
* window.
*
- * <p>Unlike with {@link #asIterable}, the resulting list is required to fit
in memory.
+ * <p>This view should only be used if random access and/or size of the
PCollection is required.
+ * {@link #asIterable()} will perform significantly better for sequential
access.
Review comment:
No, most runners will be able to easily support an efficient iterable
representation even if they only support an efficient multimap representation
since the mapping on the runners side is trivial. The converse is not true.
Also, the list side input adds additional overhead in the format which the
iterable does not.
Finally, the list side input is about accessing elements in a random access
fashion and the current format is likely to only assign one element underneath
each key so sequential access will be much better under the iterable case.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 446076)
Time Spent: 6h (was: 5h 50m)
> Migrate PCollection views to use both iterable and multimap
> materializations/access patterns
> --------------------------------------------------------------------------------------------
>
> Key: BEAM-10097
> URL: https://issues.apache.org/jira/browse/BEAM-10097
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-core, sdk-java-harness
> Reporter: Luke Cwik
> Assignee: Luke Cwik
> Priority: P2
> Time Spent: 6h
> Remaining Estimate: 0h
>
> Currently all the PCollection views have a trival mapping from KV<void,
> Iterable<T>> to the view that is being requested (singleton, iterable, list,
> map, multimap.
> We should be using the primitive views (iterable, multimap) directly without
> going through the naive mapping.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
