[
https://issues.apache.org/jira/browse/BEAM-9946?focusedWorklogId=440548&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-440548
]
ASF GitHub Bot logged work on BEAM-9946:
----------------------------------------
Author: ASF GitHub Bot
Created on: 03/Jun/20 01:33
Start Date: 03/Jun/20 01:33
Worklog Time Spent: 10m
Work Description: aaltay merged pull request #11682:
URL: https://github.com/apache/beam/pull/11682
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 440548)
Remaining Estimate: 91h 40m (was: 91h 50m)
Time Spent: 4h 20m (was: 4h 10m)
> Enhance Partition transform to provide partitionfn with SideInputs
> ------------------------------------------------------------------
>
> Key: BEAM-9946
> URL: https://issues.apache.org/jira/browse/BEAM-9946
> Project: Beam
> Issue Type: New Feature
> Components: sdk-java-core
> Reporter: Darshan Jani
> Assignee: Darshan Jani
> Priority: P2
> Original Estimate: 96h
> Time Spent: 4h 20m
> Remaining Estimate: 91h 40m
>
> Currently _Partition_ transform can partition a collection into n collections
> based on only _element_ value in _PartitionFn_ to decide on which partition a
> particular element belongs to.
> {code:java}
> public interface PartitionFn<T> extends Serializable {
> int partitionFor(T elem, int numPartitions);
> }
> public static <T> Partition<T> of(int numPartitions, PartitionFn<? super T>
> partitionFn) {
> return new Partition<>(new PartitionDoFn<T>(numPartitions, partitionFn));
> }
> {code}
> It will be useful to introduce new API with additional _sideInputs_ provided
> to partition function. User will be able to write logic to use both _element_
> value and _sideInputs_ to decide on which partition a particular element
> belongs to.
> Option-1: Proposed new API:
> {code:java}
> public interface PartitionWithSideInputsFn<T> extends Serializable {
> int partitionFor(T elem, int numPartitions, Context c);
> }
> public static <T> Partition<T> of(int numPartitions,
> PartitionWithSideInputsFn<? super T> partitionFn, Requirements requirements) {
> ...
> }
> {code}
> User can use any of the two APIs as per there partitioning function logic.
> Option-2: Redesign old API with Builder Pattern which can provide optionally
> a _Requirements_ with _sideInputs._ Deprecate old API.
> {code:java}
> // using sideviews
> Partition.into(numberOfPartitions).via(
> fn(
> (input,c) -> {
> // use c.sideInput(view)
> // use input
> // return partitionnumber
> },requiresSideInputs(view))
> )
> // without using sideviews
> Partition.into(numberOfPartitions).via(
> fn((input,c) -> {
> // use input
> // return partitionnumber
> })
> )
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
