1) Yes, the parameters need to be specified in properties.xml file.
2 & 3) You can use and specify different properties for multiple instances
of same operator in your logical DAG. Only the operator names should be
different.
For eg. Assume you declare 2 instances of HdfsInputOperator in prepareDag()
as follows
HdfsnputOperator o1 = dag.addOperator("hdfsInputOperator1",
HdfsInputOperator.class);
HdfsnputOperator o2 = dag.addOperator("hdfsInputOperator2",
HdfsInputOperator.class);
To specify which directory/file each operator should read, you can set
their respective property as follows
<property>
<name>dt.operator.hdfsInputOperator1.prop.fileName</name>
<value>data1</value>
</property>
<property>
<name>dt.operator.hdfsInputOperator2.prop.fileName</name>
<value>data2</value>
</property>
On Mon, Jan 16, 2017 at 5:13 AM, Neeraj Vaidya <[email protected]>
wrote:
> Hi,
> With reference to application configuration packages, can the following be
> clarified :
> 1) Where do parameters related application specific business logic need to
> be placed ? In the properties.xml or a separate appname.properties file in
> the classpath folder of the config package ? I am guessing it is the
> properties file, but correct me if I am wrong.
> 2) If I were to use 2 instances of the same operator (e.g. : 2 Socket
> operators listening for messages on different ports), is it possible to
> specify the port number for each in the properties.xml file ? Again, my
> guess is no, because the <name> tag cannot differentiate between multiple
> instances of the same operator.
> 2.1) As an aside, is it ok to have multiple instances of any type of
> operator in each application or rather create a separate application
> altogether with a different config package ? the complexity in creating
> more than one application is sharing in-memory application data among each
> other.
>
> Regards,
> Neeraj
>
