Re: [Dev] Developing support for tool-box approach for WSO2 CEP
Hi all, On Thu, Oct 30, 2014 at 5:30 AM, Sinthuja Ragendran sinth...@wso2.com wrote: Hi, On Wed, Oct 29, 2014 at 7:30 PM, Lasantha Fernando lasan...@wso2.com wrote: Hi Tishan, Since we can't catch any exceptions properly when simply dropping the files to the relevant directories, I think going for a admin service call approach might be better. Yeah. AFAIU it is better to consume backend OSGi services directly. But in that approach also we need to revisit current implementation. In the current CEP EventProcessorService implementation BE service will write the file to respective deployment directory, prevent automatic triggering of ExecutionPlanDeployer and then manually trigger the ExecutionPlanDeployer so that exceptions can be caught. I do not know whether this was done in order to achieve some requirement. But IMHO this should not be the case. I think we should use registry as common persistence point for configurations and avoid storing a copy of execution plans added in UI in the deployment directory. Then the EventProcessor BE service will store a given configuration in the registry and then do the runtime deplyment. Also the ExecutionPlanDeployer will call EventprocessorService when it is notified of a new deployment. Then we can use EventProcessorService directly in ToolboxDeployer and deploy artifacts. Then we also need to change BAM services to same approach for a unified scenario. Also, I think initially there were discussions on aligning the toolboxes for BAM/CEP. e.g. You have one toolbox for mediation statistics. When you want batch processing, you drop the toolbox to BAM. When you want real-time analytics, you drop the toolbox to CEP. So we might have to discuss the structure of the toolbox with BAM team as well. Can we create a separate new component for the toolbox? I think none of the existing component categorizations would fit in with the toolbox. WDYT? +1. Creating a new deployment directory such as cep-toolobox and completes different toolbox structure, etc is not a good idea. We need to unify the same toolbox for BAM and CEP scenarios, and based on the availablitiy of the relavant components the artefacts needs to be deployed. The initial implementation of the toolbox was not written in a extensible manner as there was no such requirement before. Hence now it would be better to have discussion on this to agree upon a structure for BAM and CEP, and then start implementing a common toolbox deployer which is extensible for new artefacts for the future add-ons. +1 for unifying BAM and CEP toolbox scenarios. If we think about extensibility, we can develop an AbstractToolboxDeployer which will perform generic actions (unzipping)inside deploy() and pass on to abstract doProcess(). There each implementation can handle their artifact accordingly. Also can have general utility classes which does file to OMElement/JSON conversions/etc since confings are either XML or JSON. Thanks, Tishan Thanks, Sinthuja Thanks, Lasantha On 29 October 2014 14:38, Tishan Dahanayakage tis...@wso2.com wrote: Hi all, I was working on $Subject and following scenario came up. My initial design was to create a new deployment directory, say 'cep-toolbox'. Then write a deployer extending AbstractDeployer. Deployer will get deployed toolbox, unzip it and deploy each artifact to respective deployment directory again. (Ex: deploy Execution Plan artifact to executionplan folder). But the problem with this approach is I can't catch any exception occurred when deploying each artifact. So the other approach is to create a separate bundle depending on each AdminService/BE Service and deploy artifacts using those services. This is more like BAM toolbox approach. Any thoughts on these approaches? Also please suggest a component to add the new bundle if we are going through with separate bundle option. -- Tishan Dahanayakage Software Engineer WSO2, Inc. Mobile:+94 716481328 Disclaimer: This communication may contain privileged or other confidential information and is intended exclusively for the addressee/s. If you are not the intended recipient/s, or believe that you may have received this communication in error, please reply to the sender indicating that fact and delete the copy you received and in addition, you should not print, copy, re-transmit, disseminate, or otherwise use the information contained in this communication. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions. -- *Lasantha Fernando* Software Engineer - Data Technologies Team WSO2 Inc. http://wso2.com email: lasan...@wso2.com mobile: (+94) 71 5247551 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev -- *Sinthuja Rajendran* Senior Software Engineer http://wso2.com/ WSO2,
Re: [Dev] Developing support for tool-box approach for WSO2 CEP
On Thu, Oct 30, 2014 at 2:10 AM, Tishan Dahanayakage tis...@wso2.com wrote: Hi all, On Thu, Oct 30, 2014 at 5:30 AM, Sinthuja Ragendran sinth...@wso2.com wrote: Hi, On Wed, Oct 29, 2014 at 7:30 PM, Lasantha Fernando lasan...@wso2.com wrote: Hi Tishan, Since we can't catch any exceptions properly when simply dropping the files to the relevant directories, I think going for a admin service call approach might be better. Yeah. AFAIU it is better to consume backend OSGi services directly. But in that approach also we need to revisit current implementation. In the current CEP EventProcessorService implementation BE service will write the file to respective deployment directory, prevent automatic triggering of ExecutionPlanDeployer and then manually trigger the ExecutionPlanDeployer so that exceptions can be caught. I do not know whether this was done in order to achieve some requirement. But IMHO this should not be the case. I think we should use registry as common persistence point for configurations and avoid storing a copy of execution plans added in UI in the deployment directory. Then the EventProcessor BE service will store a given configuration in the registry and then do the runtime deplyment. Also the ExecutionPlanDeployer will call EventprocessorService when it is notified of a new deployment. Then we can use EventProcessorService directly in ToolboxDeployer and deploy artifacts. Then we also need to change BAM services to same approach for a unified scenario. We had this model long time back and we moved away from this because there were lots of issues in this. The current approach works where CEP backend services store files directly to the filesystem, they have implemented in a way to deploy the artefacts then there, and throw exceptions if there any, hence we can use the CEPs OSGi services without any issue for this purpose. What toolbox deployer should do is to basically call these services. Also, I think initially there were discussions on aligning the toolboxes for BAM/CEP. e.g. You have one toolbox for mediation statistics. When you want batch processing, you drop the toolbox to BAM. When you want real-time analytics, you drop the toolbox to CEP. So we might have to discuss the structure of the toolbox with BAM team as well. Can we create a separate new component for the toolbox? I think none of the existing component categorizations would fit in with the toolbox. WDYT? +1. Creating a new deployment directory such as cep-toolobox and completes different toolbox structure, etc is not a good idea. We need to unify the same toolbox for BAM and CEP scenarios, and based on the availablitiy of the relavant components the artefacts needs to be deployed. The initial implementation of the toolbox was not written in a extensible manner as there was no such requirement before. Hence now it would be better to have discussion on this to agree upon a structure for BAM and CEP, and then start implementing a common toolbox deployer which is extensible for new artefacts for the future add-ons. +1 for unifying BAM and CEP toolbox scenarios. If we think about extensibility, we can develop an AbstractToolboxDeployer which will perform generic actions (unzipping)inside deploy() and pass on to abstract doProcess(). There each implementation can handle their artifact accordingly. Also can have general utility classes which does file to OMElement/JSON conversions/etc since confings are either XML or JSON. +1 for this approach. Please proceed with this approach. We also need to decide on the new folder structure of the Analytics Toolbox. follow the mail in architecture@ with subject : Folder structure of Analytics Toolbox (BAM CEP) Regards Suho Thanks, Tishan Thanks, Sinthuja Thanks, Lasantha On 29 October 2014 14:38, Tishan Dahanayakage tis...@wso2.com wrote: Hi all, I was working on $Subject and following scenario came up. My initial design was to create a new deployment directory, say 'cep-toolbox'. Then write a deployer extending AbstractDeployer. Deployer will get deployed toolbox, unzip it and deploy each artifact to respective deployment directory again. (Ex: deploy Execution Plan artifact to executionplan folder). But the problem with this approach is I can't catch any exception occurred when deploying each artifact. So the other approach is to create a separate bundle depending on each AdminService/BE Service and deploy artifacts using those services. This is more like BAM toolbox approach. Any thoughts on these approaches? Also please suggest a component to add the new bundle if we are going through with separate bundle option. -- Tishan Dahanayakage Software Engineer WSO2, Inc. Mobile:+94 716481328 Disclaimer: This communication may contain privileged or other confidential information and is intended exclusively for the addressee/s. If you are not the intended recipient/s, or believe
[Dev] Developing support for tool-box approach for WSO2 CEP
Hi all, I was working on $Subject and following scenario came up. My initial design was to create a new deployment directory, say 'cep-toolbox'. Then write a deployer extending AbstractDeployer. Deployer will get deployed toolbox, unzip it and deploy each artifact to respective deployment directory again. (Ex: deploy Execution Plan artifact to executionplan folder). But the problem with this approach is I can't catch any exception occurred when deploying each artifact. So the other approach is to create a separate bundle depending on each AdminService/BE Service and deploy artifacts using those services. This is more like BAM toolbox approach. Any thoughts on these approaches? Also please suggest a component to add the new bundle if we are going through with separate bundle option. -- Tishan Dahanayakage Software Engineer WSO2, Inc. Mobile:+94 716481328 Disclaimer: This communication may contain privileged or other confidential information and is intended exclusively for the addressee/s. If you are not the intended recipient/s, or believe that you may have received this communication in error, please reply to the sender indicating that fact and delete the copy you received and in addition, you should not print, copy, re-transmit, disseminate, or otherwise use the information contained in this communication. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions. ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] Developing support for tool-box approach for WSO2 CEP
Hi Tishan, Since we can't catch any exceptions properly when simply dropping the files to the relevant directories, I think going for a admin service call approach might be better. Also, I think initially there were discussions on aligning the toolboxes for BAM/CEP. e.g. You have one toolbox for mediation statistics. When you want batch processing, you drop the toolbox to BAM. When you want real-time analytics, you drop the toolbox to CEP. So we might have to discuss the structure of the toolbox with BAM team as well. Can we create a separate new component for the toolbox? I think none of the existing component categorizations would fit in with the toolbox. WDYT? Thanks, Lasantha On 29 October 2014 14:38, Tishan Dahanayakage tis...@wso2.com wrote: Hi all, I was working on $Subject and following scenario came up. My initial design was to create a new deployment directory, say 'cep-toolbox'. Then write a deployer extending AbstractDeployer. Deployer will get deployed toolbox, unzip it and deploy each artifact to respective deployment directory again. (Ex: deploy Execution Plan artifact to executionplan folder). But the problem with this approach is I can't catch any exception occurred when deploying each artifact. So the other approach is to create a separate bundle depending on each AdminService/BE Service and deploy artifacts using those services. This is more like BAM toolbox approach. Any thoughts on these approaches? Also please suggest a component to add the new bundle if we are going through with separate bundle option. -- Tishan Dahanayakage Software Engineer WSO2, Inc. Mobile:+94 716481328 Disclaimer: This communication may contain privileged or other confidential information and is intended exclusively for the addressee/s. If you are not the intended recipient/s, or believe that you may have received this communication in error, please reply to the sender indicating that fact and delete the copy you received and in addition, you should not print, copy, re-transmit, disseminate, or otherwise use the information contained in this communication. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions. -- *Lasantha Fernando* Software Engineer - Data Technologies Team WSO2 Inc. http://wso2.com email: lasan...@wso2.com mobile: (+94) 71 5247551 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
Re: [Dev] Developing support for tool-box approach for WSO2 CEP
Hi, On Wed, Oct 29, 2014 at 7:30 PM, Lasantha Fernando lasan...@wso2.com wrote: Hi Tishan, Since we can't catch any exceptions properly when simply dropping the files to the relevant directories, I think going for a admin service call approach might be better. Also, I think initially there were discussions on aligning the toolboxes for BAM/CEP. e.g. You have one toolbox for mediation statistics. When you want batch processing, you drop the toolbox to BAM. When you want real-time analytics, you drop the toolbox to CEP. So we might have to discuss the structure of the toolbox with BAM team as well. Can we create a separate new component for the toolbox? I think none of the existing component categorizations would fit in with the toolbox. WDYT? +1. Creating a new deployment directory such as cep-toolobox and completes different toolbox structure, etc is not a good idea. We need to unify the same toolbox for BAM and CEP scenarios, and based on the availablitiy of the relavant components the artefacts needs to be deployed. The initial implementation of the toolbox was not written in a extensible manner as there was no such requirement before. Hence now it would be better to have discussion on this to agree upon a structure for BAM and CEP, and then start implementing a common toolbox deployer which is extensible for new artefacts for the future add-ons. Thanks, Sinthuja Thanks, Lasantha On 29 October 2014 14:38, Tishan Dahanayakage tis...@wso2.com wrote: Hi all, I was working on $Subject and following scenario came up. My initial design was to create a new deployment directory, say 'cep-toolbox'. Then write a deployer extending AbstractDeployer. Deployer will get deployed toolbox, unzip it and deploy each artifact to respective deployment directory again. (Ex: deploy Execution Plan artifact to executionplan folder). But the problem with this approach is I can't catch any exception occurred when deploying each artifact. So the other approach is to create a separate bundle depending on each AdminService/BE Service and deploy artifacts using those services. This is more like BAM toolbox approach. Any thoughts on these approaches? Also please suggest a component to add the new bundle if we are going through with separate bundle option. -- Tishan Dahanayakage Software Engineer WSO2, Inc. Mobile:+94 716481328 Disclaimer: This communication may contain privileged or other confidential information and is intended exclusively for the addressee/s. If you are not the intended recipient/s, or believe that you may have received this communication in error, please reply to the sender indicating that fact and delete the copy you received and in addition, you should not print, copy, re-transmit, disseminate, or otherwise use the information contained in this communication. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions. -- *Lasantha Fernando* Software Engineer - Data Technologies Team WSO2 Inc. http://wso2.com email: lasan...@wso2.com mobile: (+94) 71 5247551 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev -- *Sinthuja Rajendran* Senior Software Engineer http://wso2.com/ WSO2, Inc.:http://wso2.com Blog: http://sinthu-rajan.blogspot.com/ Mobile: +94774273955 ___ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
