Will Marshall created QUARKS-91:
-----------------------------------
Summary: Support joining a stream against another stream/window
Key: QUARKS-91
URL: https://issues.apache.org/jira/browse/QUARKS-91
Project: Quarks
Issue Type: Bug
Reporter: Will Marshall
Assignee: Will Marshall
Quarks should support the ability to join a stream against another stream or
window. The interface would look something like this:
{code}
<J, U, K> TStream<J> join(Function<T, K> keyer, TWindow<U, K> window,
BiFunction<T, List<U>, J> joiner);
<J, U, K> TStream<J> joinLast(TStream<U> lastStream, BiFunction<T, U, J>
joiner);
<J, U, K> TStream<J> joinLast(Function<T, K> keyer, TStream<U> lastStream,
BiFunction<T, U, J> joiner);
{code}
For a stream and a window, streamA and windowB: *streamA.join(keyer, windowB,
joiner)* would call *joiner* on every tuple on streamA with the contents of the
partition associated with the key *keyer.apply(tuple)*.
For two streams streamA and streamB: *streamA.joinLast(keyer, streamB, joiner)*
would call the supplied *joiner* function on the most recent value of streamA
and the most recent value of streamB whose key matched *keyer.apply(tuple)*.
This happens whenever a tuple passes through streamA.
For two streams streamA and streamB: *streamA.joinLast(streamB, joiner)* would
call the supplied *joiner* function on the most recent values of streamA and
streamB whenever a tuple passes through streamA.
*Issues:*
- The definition of Oplet<I, O> requires that all input tuples be of the same
type. If streamA and streamB are of different types, it's not clear how to
create a "join" oplet where one input port has the tuples of streamA and
another has the tuples of streamB.
- The windowing library doesn't currently support accessing the contents of a
partition from a quarks.window.Window reference.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
