Hi nirmal, DAS features such as scripts scheduling, purging,etc are used to submit the jobs (only spark queries) to external spark cluster, rather those DAS features jars doesn't need to exists within the external spark cluster instance. For example, if we consider spark script scheduled execution scenario which uses Task OSGI services, and the task triggering will be be occurred wihing DAS node (OSGI env), furthermore when the spark is configured externally the job will be handed over to the external cluster, and then results will be given back to DAS node. Therefore I don't think any of the DAS features jars other than DAL feature jars will be required to be inside the external spark cluster.
Can you please explain more on what is your usecase? And how you have configured the setup with DAS features? Thanks, Sinthuja. On Sunday, June 28, 2015, Nirmal Fernando <[email protected]> wrote: > Hi DAS team, > > It appears that we have to think and implement DAS features so that they > will run even in an non-OSGi environment like an external Spark scenario. > We have some DAS features which are dependent on Task Service etc. and they > are failing when we use the from within a Spark job which runs on an > external Spark cluster. > > How can we solve this? > > -- > > Thanks & regards, > Nirmal > > Associate Technical Lead - Data Technologies Team, WSO2 Inc. > Mobile: +94715779733 > Blog: http://nirmalfdo.blogspot.com/ > > >
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
