[jira] [Created] (SAMZA-1714) Creating shared context factory for shared context in high-level API

[Yi Pan (Data Infrastructure) (JIRA)]

Yi Pan (Data Infrastructure) created SAMZA-1714:
---------------------------------------------------

             Summary: Creating shared context factory for shared context in 
high-level API
                 Key: SAMZA-1714
                 URL: https://issues.apache.org/jira/browse/SAMZA-1714
             Project: Samza
          Issue Type: Improvement
            Reporter: Yi Pan (Data Infrastructure)


Currently, ContextManager only provides a single scope, single object shared 
context in the high-level API:
{code:java}

public interface ContextManager {
   public void init(Config config, TaskContext context);
}{code}

It is not sufficient to address the following requirements:
1) different scope of shared objects (i.e. operator-scope, task-scope, 
container-scope)
2) different shared objects instantiated and shared in the same scope (i.e. 
user program has task-level shared objects for remote-http-client vs in-mem 
cache that needs to managed independently)

An initial idea to implement the SharedContextFactory is the following:
{code}
public interface SharedContextFactory {
    <T extends SharedContext> T createSharedContext(Config config, Scope scope, 
final SystemContext sysContext, String id);
}

public interface SharedContext {
  void init(Config, final SystemContext sysContext);
  void close();
}

public interface SystemContext {
  <T extends SharedContext> T getSharedContext(Scope scope, String id);
}

public interface SamzaContainerContext extends SystemContext {
}

public interface TaskContext extends SystemContext {
}
{code}

Note that this is also needed to serialize operator functions used in 
high-level API that depends on non-serializable third-party libraries (e.g. 
Calcite runtime objects used in Samza SQL).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)