[
https://issues.apache.org/jira/browse/QUARKS-91?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Will Marshall updated QUARKS-91:
--------------------------------
Description:
Quarks should support the ability to join a stream against another stream or
window. The interface would look something like this:
{code}
<J, U, K> TStream<J> join(Function<T, K> keyer, TWindow<U, K> window,
BiFunction<T, List<U>, J> joiner);
<J, U, K> TStream<J> joinLast(TStream<U> lastStream, BiFunction<T, U, J>
joiner);
<J, U, K> TStream<J> joinLast(Function<T, K> keyer, TStream<U> lastStream,
BiFunction<T, U, J> joiner);
{code}
For a stream and a window, streamA and windowB: *streamA.join(keyer, windowB,
joiner)* would call *joiner* on every tuple on streamA with the contents of the
partition associated with the key *keyer.apply(tuple)*.
For two streams, streamA and streamB: *streamA.joinLast(keyer, streamB,
joiner)* would call the supplied *joiner* function on the most recent value of
streamA and the most recent value of streamB whose key matched
*keyer.apply(tuple)*. This happens whenever a tuple passes through streamA.
For two streams, streamA and streamB: *streamA.joinLast(streamB, joiner)* would
call the supplied *joiner* function on the most recent values of streamA and
streamB whenever a tuple passes through streamA.
*Issues:*
- The definition of Oplet<I, O> requires that all input tuples be of the same
type. If streamA and streamB are of different types, it's not clear how to
create a "join" oplet where one input port has the tuples of streamA and
another has the tuples of streamB.
- The windowing library doesn't currently support accessing the contents of a
partition from a quarks.window.Window reference.
was:
Quarks should support the ability to join a stream against another stream or
window. The interface would look something like this:
{code}
<J, U, K> TStream<J> join(Function<T, K> keyer, TWindow<U, K> window,
BiFunction<T, List<U>, J> joiner);
<J, U, K> TStream<J> joinLast(TStream<U> lastStream, BiFunction<T, U, J>
joiner);
<J, U, K> TStream<J> joinLast(Function<T, K> keyer, TStream<U> lastStream,
BiFunction<T, U, J> joiner);
{code}
For a stream and a window, streamA and windowB: *streamA.join(keyer, windowB,
joiner)* would call *joiner* on every tuple on streamA with the contents of the
partition associated with the key *keyer.apply(tuple)*.
For two streams streamA and streamB: *streamA.joinLast(keyer, streamB, joiner)*
would call the supplied *joiner* function on the most recent value of streamA
and the most recent value of streamB whose key matched *keyer.apply(tuple)*.
This happens whenever a tuple passes through streamA.
For two streams streamA and streamB: *streamA.joinLast(streamB, joiner)* would
call the supplied *joiner* function on the most recent values of streamA and
streamB whenever a tuple passes through streamA.
*Issues:*
- The definition of Oplet<I, O> requires that all input tuples be of the same
type. If streamA and streamB are of different types, it's not clear how to
create a "join" oplet where one input port has the tuples of streamA and
another has the tuples of streamB.
- The windowing library doesn't currently support accessing the contents of a
partition from a quarks.window.Window reference.
> Support joining a stream against another stream/window
> ------------------------------------------------------
>
> Key: QUARKS-91
> URL: https://issues.apache.org/jira/browse/QUARKS-91
> Project: Quarks
> Issue Type: Bug
> Reporter: Will Marshall
> Assignee: Will Marshall
>
> Quarks should support the ability to join a stream against another stream or
> window. The interface would look something like this:
> {code}
> <J, U, K> TStream<J> join(Function<T, K> keyer, TWindow<U, K> window,
> BiFunction<T, List<U>, J> joiner);
>
> <J, U, K> TStream<J> joinLast(TStream<U> lastStream, BiFunction<T, U, J>
> joiner);
> <J, U, K> TStream<J> joinLast(Function<T, K> keyer, TStream<U>
> lastStream, BiFunction<T, U, J> joiner);
> {code}
> For a stream and a window, streamA and windowB: *streamA.join(keyer, windowB,
> joiner)* would call *joiner* on every tuple on streamA with the contents of
> the partition associated with the key *keyer.apply(tuple)*.
> For two streams, streamA and streamB: *streamA.joinLast(keyer, streamB,
> joiner)* would call the supplied *joiner* function on the most recent value
> of streamA and the most recent value of streamB whose key matched
> *keyer.apply(tuple)*. This happens whenever a tuple passes through streamA.
> For two streams, streamA and streamB: *streamA.joinLast(streamB, joiner)*
> would call the supplied *joiner* function on the most recent values of
> streamA and streamB whenever a tuple passes through streamA.
> *Issues:*
> - The definition of Oplet<I, O> requires that all input tuples be of the same
> type. If streamA and streamB are of different types, it's not clear how to
> create a "join" oplet where one input port has the tuples of streamA and
> another has the tuples of streamB.
> - The windowing library doesn't currently support accessing the contents of a
> partition from a quarks.window.Window reference.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
