[
https://issues.apache.org/jira/browse/SAMZA-1743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582738#comment-16582738
]
Peter Georgantas commented on SAMZA-1743:
-----------------------------------------
Hi Prateek,
I'm following up with a somewhat related concern while trying the table join.
My use case doesn't seem to be covered by the existing API, but perhaps there's
an opportunity to consider adding support. Or perhaps you might have an idea
for a workaround. Currently we handle it manually in a map function.
My incoming message has a collection of actors where only the key is known.
{code:java}
{id: 'abc', actors: [{key: 1}, {key: 3}]}
{code}
My Table will hold enrichment data for the actors.
{code:java}
[{key: 1, name: "actor 1"}, {key: 2, name: "actor 2"}, {key: 3, name: "actor
3"}]
{code}
My desired outgoing message is similar to the incoming message, but the actors
are now enriched.
{code:java}
{id: 'abc', actors: [{key: 1, name: "actor 1"}, {key: 2, name: "actor 2"}]}
{code}
So a proposal for my use case would require a table join function like this.
{code:java}
public interface StreamTableMultiJoinFunction<K, M, R, JM> extends
InitableFunction, ClosableFunction {
JM apply(M message, Map<K,R> records);
Collection<K> getMessageKeys(M message);
K getRecordKey(R record);
}{code}
> Allow highlevel operators to have access to programmatically created state.
> ---------------------------------------------------------------------------
>
> Key: SAMZA-1743
> URL: https://issues.apache.org/jira/browse/SAMZA-1743
> Project: Samza
> Issue Type: Improvement
> Affects Versions: 0.14.0
> Reporter: Peter Georgantas
> Assignee: Prateek Maheshwari
> Priority: Minor
>
> The window and join highlevel operators must have state by definition. When
> using these operators, the api will programmatically create the config for
> that state. The api should allow more operators (filter, map, etc) access to
> the same programmatic state definition. Having this option would help close
> the gap between code and properties files.
> Relevant code:
> [StatefulOperatorSpec.java|https://github.com/apache/samza/blob/161d1c47a2c7322a7d3197d571a227cce0f1cbbf/samza-core/src/main/java/org/apache/samza/operators/spec/StatefulOperatorSpec.java#L28]
> [JobNode.java|https://github.com/apache/samza/blob/161d1c47a2c7322a7d3197d571a227cce0f1cbbf/samza-core/src/main/java/org/apache/samza/execution/JobNode.java#L166]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
