Prateek Maheshwari created SAMZA-1995:
-----------------------------------------
Summary: Delay SystemDescriptor uniqueness check until config
generation
Key: SAMZA-1995
URL: https://issues.apache.org/jira/browse/SAMZA-1995
Project: Samza
Issue Type: Improvement
Reporter: Prateek Maheshwari
Assignee: Prateek Maheshwari
Currently, ApplicationDescriptorImpl does not allow adding multiple system
descriptor _instances_ for the same system name.
It's useful to support the following use case:
User wants to separate their input and output system descriptors. In dev
environment, they want to consume inputs from the "staging" cluster, but want
to output to a dev "local" cluster. In staging environment, they want to
produce and consume to the "staging" cluster.
Currently, if the input and output system descriptors have the same system name
(as is the case for staging environment), application description will fail. We
should relax this check to allow registering multiple system descriptors for
the same system as long as they are "equal".
Equality for two SystemDescriptors implies that they generate the same
configuration, and have the same runtime behavior (e.g., for transformers,
expanders and serdes).
Since a SystemDescriptor is a config builder, it's possible that some
implementations are stateful (e.g. EventHubsSystemDescriptor keeps track of its
input streams). This can cause the equality check to pass or fail depending on
when it's done during application description. To be safe, we should delay the
equality check until we actually generate the configs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
