[
https://issues.apache.org/jira/browse/BEAM-9946?focusedWorklogId=435813&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-435813
]
ASF GitHub Bot logged work on BEAM-9946:
----------------------------------------
Author: ASF GitHub Bot
Created on: 21/May/20 01:42
Start Date: 21/May/20 01:42
Worklog Time Spent: 10m
Work Description: darshanj commented on a change in pull request #11682:
URL: https://github.com/apache/beam/pull/11682#discussion_r428399770
##########
File path:
sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Partition.java
##########
@@ -124,21 +190,26 @@ private Partition(PartitionDoFn<T> partitionDoFn) {
private static class PartitionDoFn<X> extends DoFn<X, Void> {
private final int numPartitions;
- private final PartitionFn<? super X> partitionFn;
private final TupleTagList outputTags;
+ private Contextful<Contextful.Fn<X, Integer>> ctxFn;
+ private Object originalFnForDisplayData;
/**
* Constructs a PartitionDoFn.
*
* @throws IllegalArgumentException if {@code numPartitions <= 0}
*/
- public PartitionDoFn(int numPartitions, PartitionFn<? super X>
partitionFn) {
+ public PartitionDoFn(
Review comment:
Done.Thanks
##########
File path:
sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Partition.java
##########
@@ -124,21 +190,26 @@ private Partition(PartitionDoFn<T> partitionDoFn) {
private static class PartitionDoFn<X> extends DoFn<X, Void> {
private final int numPartitions;
- private final PartitionFn<? super X> partitionFn;
private final TupleTagList outputTags;
+ private Contextful<Contextful.Fn<X, Integer>> ctxFn;
+ private Object originalFnForDisplayData;
Review comment:
Done Thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 435813)
Remaining Estimate: 94h 50m (was: 95h)
Time Spent: 1h 10m (was: 1h)
> Enhance Partition transform to provide partitionfn with SideInputs
> ------------------------------------------------------------------
>
> Key: BEAM-9946
> URL: https://issues.apache.org/jira/browse/BEAM-9946
> Project: Beam
> Issue Type: New Feature
> Components: sdk-java-core
> Reporter: Darshan Jani
> Assignee: Darshan Jani
> Priority: P2
> Original Estimate: 96h
> Time Spent: 1h 10m
> Remaining Estimate: 94h 50m
>
> Currently _Partition_ transform can partition a collection into n collections
> based on only _element_ value in _PartitionFn_ to decide on which partition a
> particular element belongs to.
> {code:java}
> public interface PartitionFn<T> extends Serializable {
> int partitionFor(T elem, int numPartitions);
> }
> public static <T> Partition<T> of(int numPartitions, PartitionFn<? super T>
> partitionFn) {
> return new Partition<>(new PartitionDoFn<T>(numPartitions, partitionFn));
> }
> {code}
> It will be useful to introduce new API with additional _sideInputs_ provided
> to partition function. User will be able to write logic to use both _element_
> value and _sideInputs_ to decide on which partition a particular element
> belongs to.
> Option-1: Proposed new API:
> {code:java}
> public interface PartitionWithSideInputsFn<T> extends Serializable {
> int partitionFor(T elem, int numPartitions, Context c);
> }
> public static <T> Partition<T> of(int numPartitions,
> PartitionWithSideInputsFn<? super T> partitionFn, Requirements requirements) {
> ...
> }
> {code}
> User can use any of the two APIs as per there partitioning function logic.
> Option-2: Redesign old API with Builder Pattern which can provide optionally
> a _Requirements_ with _sideInputs._ Deprecate old API.
> {code:java}
> // using sideviews
> Partition.into(numberOfPartitions).via(
> fn(
> (input,c) -> {
> // use c.sideInput(view)
> // use input
> // return partitionnumber
> },requiresSideInputs(view))
> )
> // without using sideviews
> Partition.into(numberOfPartitions).via(
> fn((input,c) -> {
> // use input
> // return partitionnumber
> })
> )
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
