A module is superset of operator before expansion in logical plan, except
operator
attribute are not applicable to modules. As Tim suggested we will have to
add checks
later to validate these attributes settings, and not allow top level
operator and
module to share same name.
The same question comes when we want to add module to application using
property
file API and JSON file API.
dt.operator.<operatorname>.classname=<fqcn of operator>
One approach we can take is, if class is of type module, then use addModule
api
to add module to the dag, else use addOperator api. The initialization of
object and applying property is generic can be reused by module.
Similarly we have Json API where all operators are specified under
operators array
{
"operators": [
{.. operators .. }
],
"streams": [
{.. streams .. }
]
}
We could add module in the operators array, and based on the object type we
can
consider it as module or operator. The mechanism of initializing object and
applying
properties remains same.
- Tushar.
