[Architecture] Siddhi - Script on It
Hi All, The implementation, $subject is adding some scripting support for Siddhi. It is as follows, 1. Scripting languages are going to be JavaScript and Scala. 2. New language feature is to be introduced like below, Define script[Language]{//script implementation goes here} 3. Siddhi user can choose the scripting language mentioning it like below in Siddhi, Define script[JavaScript] {//JavaScript implementation goes here} or Define script[Scala]{//Scala implementation goes here} 4. The functions or methods defined are to be available to the user to call it like Siddhi inbuilt function. For example, let's just define and call a Scala-method in Siddhi like below, define script[Scala] { Def *getSymbol*(id: String): String = id } define stream cseEventStream (symbol string, price float, volume string) from cseEventStream select *getSymbol*(symbol), convert(volume,long)/1000 as vol, sum(convert(volume,long)) as sumVolume group by symbol insert into StockQuote; Appreciate your thoughts!!! Thanks, -Ayash -- Ayashkantha Ramasinghe Software Engineer WSO2, Inc. email: ayashkan...@wso2.com sanj...@wso2.com; TP: +94 77 7 487 669 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Generic RDBMS Output adapter support
Hi, Thanks Anjana and Lahiru for the suggestions. Please find below the found observations. Following are the DB actions. - DB creation (IF NOT EXIST) - Table creation (IF NOT EXIST) - Insert data to table - update data to table Since all the DBMS's are using same DDL and DML query formats we can keep the query formats static for all DB types. Also we are only considering above db actions only eg:- CREATE TABLE table_name ( column1 datatype [ NULL | NOT NULL ], column2 datatype [ NULL | NOT NULL ], ... ); UPDATE table SET column1 = expression1, column2 = expression2, ... WHERE conditions; INSERT INTO table (column1, column2, ... ) VALUES (expression1, expression2, ... ), (expression1, expression2, ... ), ...; issue for the above implementation is the query syntax differs(Some) from DB type to another (eg.data types) . So decided to use a XML file which has DB type specific data types. So dynamically we can access the db type and retrieve data types to generate the query in prepared statement. Xml file would be like following. mappings mapping db='ansi' types type fromstring/from toVARCHAR/to /type type fromdouble/from toDOUBLE/to /type /types /mapping mapping db='mysql' types //mysqi specific data types if any /types /mapping mapping db='oracle' types type fromstring/from tovarchar2/to /type /types /mapping mapping db='mssql' types type fromstring/from tovarchar2/to /type /types /mapping mapping db='h2' types //h2 specific data types if any /types /mapping /mappings Like wise above xml can be created. Main idea is to use a separate XML to manipulate different data types. According to above XML first checks for specific data type in specific db mappings. If not found then the value is read from the generic data type mapping as stated as mapping db='ansi'. All the data type mappings are not included in above configuration. Your suggestions feedbacks are highly appreciated. On Sat, Nov 29, 2014 at 11:15 AM, Lahiru Cooray lahi...@wso2.com wrote: Hi, Maintenance wise it's nice to keep queries in separate XML. And I think it's better to keep all the generic queries in a one section of the XML and keep DBMS specific queries in separate sections. So you don't need to rewrite common queries for each type. Also better to consider MS SQL Server as well in testing ( if it's supported DB type) as there are many syntax differences compared to Oracle and mysql On Fri, Nov 28, 2014 at 4:49 PM, Damith Wickramasinghe dami...@wso2.com wrote: Hi, Currently we have the support only for Mysql and it is decided to implement a generic adapter to support any RDBMS database. For now adapter implementation will be focused on supporting Oracle, Mysql and H2. I will update the thread on decided architecture for the said requirement soon. Any feedbacks on the requirement will be greatly appreciated. Regards, Damith. -- Software Engineer WSO2 Inc.; http://wso2.com http://www.google.com/url?q=http%3A%2F%2Fwso2.comsa=Dsntz=1usg=AFQjCNEZvyc0uMD1HhBaEGCBxs6e9fBObg lean.enterprise.middleware mobile: *+94728671315 %2B94728671315* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Lahiru Cooray* Software Engineer WSO2, Inc.;http://wso2.com/ lean.enterprise.middleware Mobile: +94 715 654154 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Software Engineer WSO2 Inc.; http://wso2.com http://www.google.com/url?q=http%3A%2F%2Fwso2.comsa=Dsntz=1usg=AFQjCNEZvyc0uMD1HhBaEGCBxs6e9fBObg lean.enterprise.middleware mobile: *+94728671315 %2B94728671315* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Siddhi - Script on It
+1 looks good Suho On Mon, Dec 1, 2014 at 4:57 PM, Ayash ayashkan...@wso2.com wrote: Hi All, The implementation, $subject is adding some scripting support for Siddhi. It is as follows, 1. Scripting languages are going to be JavaScript and Scala. 2. New language feature is to be introduced like below, Define script[Language]{//script implementation goes here} 3. Siddhi user can choose the scripting language mentioning it like below in Siddhi, Define script[JavaScript] {//JavaScript implementation goes here} or Define script[Scala]{//Scala implementation goes here} 4. The functions or methods defined are to be available to the user to call it like Siddhi inbuilt function. For example, let's just define and call a Scala-method in Siddhi like below, define script[Scala] { Def *getSymbol*(id: String): String = id } define stream cseEventStream (symbol string, price float, volume string) from cseEventStream select *getSymbol*(symbol), convert(volume,long)/1000 as vol, sum(convert(volume,long)) as sumVolume group by symbol insert into StockQuote; Appreciate your thoughts!!! Thanks, -Ayash -- Ayashkantha Ramasinghe Software Engineer WSO2, Inc. email: ayashkan...@wso2.com sanj...@wso2.com; TP: +94 77 7 487 669 -- *S. Suhothayan* Technical Lead Team Lead of WSO2 Complex Event Processor *WSO2 Inc. *http://wso2.com * http://wso2.com/* lean . enterprise . middleware *cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/ http://suhothayan.blogspot.com/twitter: http://twitter.com/suhothayan http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan http://lk.linkedin.com/in/suhothayan* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Generic RDBMS Output adapter support
Hi, please note above varchar length will be given as default. eg:- varchar(250). Regards, damith. On Mon, Dec 1, 2014 at 5:10 PM, Damith Wickramasinghe dami...@wso2.com wrote: Hi, Thanks Anjana and Lahiru for the suggestions. Please find below the found observations. Following are the DB actions. - DB creation (IF NOT EXIST) - Table creation (IF NOT EXIST) - Insert data to table - update data to table Since all the DBMS's are using same DDL and DML query formats we can keep the query formats static for all DB types. Also we are only considering above db actions only eg:- CREATE TABLE table_name ( column1 datatype [ NULL | NOT NULL ], column2 datatype [ NULL | NOT NULL ], ... ); UPDATE table SET column1 = expression1, column2 = expression2, ... WHERE conditions; INSERT INTO table (column1, column2, ... ) VALUES (expression1, expression2, ... ), (expression1, expression2, ... ), ...; issue for the above implementation is the query syntax differs(Some) from DB type to another (eg.data types) . So decided to use a XML file which has DB type specific data types. So dynamically we can access the db type and retrieve data types to generate the query in prepared statement. Xml file would be like following. mappings mapping db='ansi' types type fromstring/from toVARCHAR/to /type type fromdouble/from toDOUBLE/to /type /types /mapping mapping db='mysql' types //mysqi specific data types if any /types /mapping mapping db='oracle' types type fromstring/from tovarchar2/to /type /types /mapping mapping db='mssql' types type fromstring/from tovarchar2/to /type /types /mapping mapping db='h2' types //h2 specific data types if any /types /mapping /mappings Like wise above xml can be created. Main idea is to use a separate XML to manipulate different data types. According to above XML first checks for specific data type in specific db mappings. If not found then the value is read from the generic data type mapping as stated as mapping db='ansi'. All the data type mappings are not included in above configuration. Your suggestions feedbacks are highly appreciated. On Sat, Nov 29, 2014 at 11:15 AM, Lahiru Cooray lahi...@wso2.com wrote: Hi, Maintenance wise it's nice to keep queries in separate XML. And I think it's better to keep all the generic queries in a one section of the XML and keep DBMS specific queries in separate sections. So you don't need to rewrite common queries for each type. Also better to consider MS SQL Server as well in testing ( if it's supported DB type) as there are many syntax differences compared to Oracle and mysql On Fri, Nov 28, 2014 at 4:49 PM, Damith Wickramasinghe dami...@wso2.com wrote: Hi, Currently we have the support only for Mysql and it is decided to implement a generic adapter to support any RDBMS database. For now adapter implementation will be focused on supporting Oracle, Mysql and H2. I will update the thread on decided architecture for the said requirement soon. Any feedbacks on the requirement will be greatly appreciated. Regards, Damith. -- Software Engineer WSO2 Inc.; http://wso2.com http://www.google.com/url?q=http%3A%2F%2Fwso2.comsa=Dsntz=1usg=AFQjCNEZvyc0uMD1HhBaEGCBxs6e9fBObg lean.enterprise.middleware mobile: *+94728671315 %2B94728671315* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Lahiru Cooray* Software Engineer WSO2, Inc.;http://wso2.com/ lean.enterprise.middleware Mobile: +94 715 654154 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Software Engineer WSO2 Inc.; http://wso2.com http://www.google.com/url?q=http%3A%2F%2Fwso2.comsa=Dsntz=1usg=AFQjCNEZvyc0uMD1HhBaEGCBxs6e9fBObg lean.enterprise.middleware mobile: *+94728671315 %2B94728671315* -- Software Engineer WSO2 Inc.; http://wso2.com http://www.google.com/url?q=http%3A%2F%2Fwso2.comsa=Dsntz=1usg=AFQjCNEZvyc0uMD1HhBaEGCBxs6e9fBObg lean.enterprise.middleware mobile: *+94728671315* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Generic RDBMS Output adapter support
Hi Damith, I'm personally biased towards just using the whole queries in the XML, where we may get some subtle changes in queries that may not be possible with your parameterized approach. For example, in MySQL, you can give the storage table engine as a part in the DDL query. Likewise, there maybe special sections that is dependent on the RDBMS. So we may not want to over-engineer this that much, a simple mapping of all the queries to a DBMS is much simpler, and has more functionality. The redundancy of possible duplication of standard queries across DBMS mappings can be ignored, looking at the other potential advantages. Also, these table creation queries, you may want to have a set of queries that will initialize the tables, because you would want required indexes to be created too, and index creation is anyway dependent on the target RDBMS. Cheers, Anjana. On Mon, Dec 1, 2014 at 5:17 PM, Damith Wickramasinghe dami...@wso2.com wrote: Hi, please note above varchar length will be given as default. eg:- varchar(250). Regards, damith. On Mon, Dec 1, 2014 at 5:10 PM, Damith Wickramasinghe dami...@wso2.com wrote: Hi, Thanks Anjana and Lahiru for the suggestions. Please find below the found observations. Following are the DB actions. - DB creation (IF NOT EXIST) - Table creation (IF NOT EXIST) - Insert data to table - update data to table Since all the DBMS's are using same DDL and DML query formats we can keep the query formats static for all DB types. Also we are only considering above db actions only eg:- CREATE TABLE table_name ( column1 datatype [ NULL | NOT NULL ], column2 datatype [ NULL | NOT NULL ], ... ); UPDATE table SET column1 = expression1, column2 = expression2, ... WHERE conditions; INSERT INTO table (column1, column2, ... ) VALUES (expression1, expression2, ... ), (expression1, expression2, ... ), ...; issue for the above implementation is the query syntax differs(Some) from DB type to another (eg.data types) . So decided to use a XML file which has DB type specific data types. So dynamically we can access the db type and retrieve data types to generate the query in prepared statement. Xml file would be like following. mappings mapping db='ansi' types type fromstring/from toVARCHAR/to /type type fromdouble/from toDOUBLE/to /type /types /mapping mapping db='mysql' types //mysqi specific data types if any /types /mapping mapping db='oracle' types type fromstring/from tovarchar2/to /type /types /mapping mapping db='mssql' types type fromstring/from tovarchar2/to /type /types /mapping mapping db='h2' types //h2 specific data types if any /types /mapping /mappings Like wise above xml can be created. Main idea is to use a separate XML to manipulate different data types. According to above XML first checks for specific data type in specific db mappings. If not found then the value is read from the generic data type mapping as stated as mapping db='ansi'. All the data type mappings are not included in above configuration. Your suggestions feedbacks are highly appreciated. On Sat, Nov 29, 2014 at 11:15 AM, Lahiru Cooray lahi...@wso2.com wrote: Hi, Maintenance wise it's nice to keep queries in separate XML. And I think it's better to keep all the generic queries in a one section of the XML and keep DBMS specific queries in separate sections. So you don't need to rewrite common queries for each type. Also better to consider MS SQL Server as well in testing ( if it's supported DB type) as there are many syntax differences compared to Oracle and mysql On Fri, Nov 28, 2014 at 4:49 PM, Damith Wickramasinghe dami...@wso2.com wrote: Hi, Currently we have the support only for Mysql and it is decided to implement a generic adapter to support any RDBMS database. For now adapter implementation will be focused on supporting Oracle, Mysql and H2. I will update the thread on decided architecture for the said requirement soon. Any feedbacks on the requirement will be greatly appreciated. Regards, Damith. -- Software Engineer WSO2 Inc.; http://wso2.com http://www.google.com/url?q=http%3A%2F%2Fwso2.comsa=Dsntz=1usg=AFQjCNEZvyc0uMD1HhBaEGCBxs6e9fBObg lean.enterprise.middleware mobile: *+94728671315 %2B94728671315* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Lahiru Cooray* Software Engineer WSO2, Inc.;http://wso2.com/ lean.enterprise.middleware Mobile: +94 715 654154 ___ Architecture
Re: [Architecture] Generic RDBMS Output adapter support
Hi Damitha, As Anjana suggested, separating only the data types would lead to some confusions. Furthermore might, sometimes lead to not using a specific database system to it's fullest potential. For example, given that there are a huge amount of built-in functions in MSSQL we might be able to use a simpler, efficient query to complete our task in MSSQL that in other database systems (just an example). Also apart from the functions you have mentioned, in the future the system might need some more functions and we have to make sure that extensibility is supported. Therefore, having a separate XML for queries in each database system is more suitable in my opinion too. Thanks, Akalanka On Mon, Dec 1, 2014 at 5:26 PM, Anjana Fernando anj...@wso2.com wrote: Hi Damith, I'm personally biased towards just using the whole queries in the XML, where we may get some subtle changes in queries that may not be possible with your parameterized approach. For example, in MySQL, you can give the storage table engine as a part in the DDL query. Likewise, there maybe special sections that is dependent on the RDBMS. So we may not want to over-engineer this that much, a simple mapping of all the queries to a DBMS is much simpler, and has more functionality. The redundancy of possible duplication of standard queries across DBMS mappings can be ignored, looking at the other potential advantages. Also, these table creation queries, you may want to have a set of queries that will initialize the tables, because you would want required indexes to be created too, and index creation is anyway dependent on the target RDBMS. Cheers, Anjana. On Mon, Dec 1, 2014 at 5:17 PM, Damith Wickramasinghe dami...@wso2.com wrote: Hi, please note above varchar length will be given as default. eg:- varchar(250). Regards, damith. On Mon, Dec 1, 2014 at 5:10 PM, Damith Wickramasinghe dami...@wso2.com wrote: Hi, Thanks Anjana and Lahiru for the suggestions. Please find below the found observations. Following are the DB actions. - DB creation (IF NOT EXIST) - Table creation (IF NOT EXIST) - Insert data to table - update data to table Since all the DBMS's are using same DDL and DML query formats we can keep the query formats static for all DB types. Also we are only considering above db actions only eg:- CREATE TABLE table_name ( column1 datatype [ NULL | NOT NULL ], column2 datatype [ NULL | NOT NULL ], ... ); UPDATE table SET column1 = expression1, column2 = expression2, ... WHERE conditions; INSERT INTO table (column1, column2, ... ) VALUES (expression1, expression2, ... ), (expression1, expression2, ... ), ...; issue for the above implementation is the query syntax differs(Some) from DB type to another (eg.data types) . So decided to use a XML file which has DB type specific data types. So dynamically we can access the db type and retrieve data types to generate the query in prepared statement. Xml file would be like following. mappings mapping db='ansi' types type fromstring/from toVARCHAR/to /type type fromdouble/from toDOUBLE/to /type /types /mapping mapping db='mysql' types //mysqi specific data types if any /types /mapping mapping db='oracle' types type fromstring/from tovarchar2/to /type /types /mapping mapping db='mssql' types type fromstring/from tovarchar2/to /type /types /mapping mapping db='h2' types //h2 specific data types if any /types /mapping /mappings Like wise above xml can be created. Main idea is to use a separate XML to manipulate different data types. According to above XML first checks for specific data type in specific db mappings. If not found then the value is read from the generic data type mapping as stated as mapping db='ansi'. All the data type mappings are not included in above configuration. Your suggestions feedbacks are highly appreciated. On Sat, Nov 29, 2014 at 11:15 AM, Lahiru Cooray lahi...@wso2.com wrote: Hi, Maintenance wise it's nice to keep queries in separate XML. And I think it's better to keep all the generic queries in a one section of the XML and keep DBMS specific queries in separate sections. So you don't need to rewrite common queries for each type. Also better to consider MS SQL Server as well in testing ( if it's supported DB type) as there are many syntax differences compared to Oracle and mysql On Fri, Nov 28, 2014 at 4:49 PM, Damith Wickramasinghe dami...@wso2.com wrote: Hi, Currently we have the support only for Mysql and it is decided to implement a generic adapter to support any RDBMS database. For now adapter implementation will
Re: [Architecture] Generic RDBMS Output adapter support
Hi, Thanks Anjana and Akalanka for the suggestions. @Anjana As per the off-line discussion we had, I will add the necessary for the implementation. Regards, Damith. On Mon, Dec 1, 2014 at 5:49 PM, Akalanka Pagoda Arachchi darsha...@wso2.com wrote: Hi Damitha, As Anjana suggested, separating only the data types would lead to some confusions. Furthermore might, sometimes lead to not using a specific database system to it's fullest potential. For example, given that there are a huge amount of built-in functions in MSSQL we might be able to use a simpler, efficient query to complete our task in MSSQL that in other database systems (just an example). Also apart from the functions you have mentioned, in the future the system might need some more functions and we have to make sure that extensibility is supported. Therefore, having a separate XML for queries in each database system is more suitable in my opinion too. Thanks, Akalanka On Mon, Dec 1, 2014 at 5:26 PM, Anjana Fernando anj...@wso2.com wrote: Hi Damith, I'm personally biased towards just using the whole queries in the XML, where we may get some subtle changes in queries that may not be possible with your parameterized approach. For example, in MySQL, you can give the storage table engine as a part in the DDL query. Likewise, there maybe special sections that is dependent on the RDBMS. So we may not want to over-engineer this that much, a simple mapping of all the queries to a DBMS is much simpler, and has more functionality. The redundancy of possible duplication of standard queries across DBMS mappings can be ignored, looking at the other potential advantages. Also, these table creation queries, you may want to have a set of queries that will initialize the tables, because you would want required indexes to be created too, and index creation is anyway dependent on the target RDBMS. Cheers, Anjana. On Mon, Dec 1, 2014 at 5:17 PM, Damith Wickramasinghe dami...@wso2.com wrote: Hi, please note above varchar length will be given as default. eg:- varchar(250). Regards, damith. On Mon, Dec 1, 2014 at 5:10 PM, Damith Wickramasinghe dami...@wso2.com wrote: Hi, Thanks Anjana and Lahiru for the suggestions. Please find below the found observations. Following are the DB actions. - DB creation (IF NOT EXIST) - Table creation (IF NOT EXIST) - Insert data to table - update data to table Since all the DBMS's are using same DDL and DML query formats we can keep the query formats static for all DB types. Also we are only considering above db actions only eg:- CREATE TABLE table_name ( column1 datatype [ NULL | NOT NULL ], column2 datatype [ NULL | NOT NULL ], ... ); UPDATE table SET column1 = expression1, column2 = expression2, ... WHERE conditions; INSERT INTO table (column1, column2, ... ) VALUES (expression1, expression2, ... ), (expression1, expression2, ... ), ...; issue for the above implementation is the query syntax differs(Some) from DB type to another (eg.data types) . So decided to use a XML file which has DB type specific data types. So dynamically we can access the db type and retrieve data types to generate the query in prepared statement. Xml file would be like following. mappings mapping db='ansi' types type fromstring/from toVARCHAR/to /type type fromdouble/from toDOUBLE/to /type /types /mapping mapping db='mysql' types //mysqi specific data types if any /types /mapping mapping db='oracle' types type fromstring/from tovarchar2/to /type /types /mapping mapping db='mssql' types type fromstring/from tovarchar2/to /type /types /mapping mapping db='h2' types //h2 specific data types if any /types /mapping /mappings Like wise above xml can be created. Main idea is to use a separate XML to manipulate different data types. According to above XML first checks for specific data type in specific db mappings. If not found then the value is read from the generic data type mapping as stated as mapping db='ansi'. All the data type mappings are not included in above configuration. Your suggestions feedbacks are highly appreciated. On Sat, Nov 29, 2014 at 11:15 AM, Lahiru Cooray lahi...@wso2.com wrote: Hi, Maintenance wise it's nice to keep queries in separate XML. And I think it's better to keep all the generic queries in a one section of the XML and keep DBMS specific queries in separate sections. So you don't need to rewrite common queries for each type. Also better to consider MS SQL Server as well in testing ( if it's supported DB type) as there are many syntax differences compared to
[Architecture] Moving deployment section from configuration xml and Facilitate AppFactory users to deploy apps with their custom app types if its not already available
Hi All, We are trying to implement a feature on AF which enables the user to deploy their customized app types, Currently this configuration is available in *appfactory.xml *under *Deployer* tag the content would be as [1], likewise we have for each app types. Hence this wouldn't be editable by the users and may not deploy their own app types. If we move out this [1] from *appfactory.xml* and put this in a configurable file would enable the users to customize their need. When we analyzing the [1] we found out , The below content related to Pass artifact storage configuration and common to all app types. RepositoryProvider Property name=Class org.wso2.carbon.appfactory.s4.integration.GITBlitBasedGITRepositoryProvider /Property Property name=BaseURL https://gitblit.s2.wso2.com:8444//Property Property name=URLPattern{@stage}/as/Property Property name=AdminUserNameadmin/Property Property name=AdminPasswordadmin/Property /RepositoryProvider So we are planning to move this to a common configuration file and all the application types can access that. In [1] below properties are not used. Hence, We shall get rid of that from AF Property name=minInstances1/Property Property name=maxInstances1/Property Property name=shouldActivate/Property Property name=subscribeOnDeploymentfalse /Property And the below properties are used. So, We can keep them in the *apptype.xml* as we already have separate apptype.xml for each app types.. Property name=alias162dev/Property Property name=cartridgeType162dev/Property Property name=deploymentPolicyaf-deployment /Property Property name=autoscalePolicyeconomy/Property Property name=repoURL/Property Property name=dataCartridgeType/Property Property name=dataCartridgeAlias/Property Look forward your views in this. Thanks Regards, S.A.Rajeevan Software Engineer WSO2 Inc E-Mail: rajeev...@wso2.com | Mobile : +94776411636 {\rtf1\ansi\ansicpg1252\cocoartf1343\cocoasubrtf140 {\fonttbl\f0\fnil\fcharset0 Monaco;} {\colortbl;\red255\green255\blue255;\red19\green150\blue163;\red178\green178\blue178;} \margl1440\margr1440\vieww10800\viewh8400\viewkind0 \pard\tx560\tx1120\tx1680\tx2240\tx2800\tx3360\tx3920\tx4480\tx5040\tx5600\tx6160\tx6720\pardirnatural \f0\fs20 \cf2 \cb3 \CocoaLigature0 ApplicationType name=*\ \ ClassNameorg.wso2.carbon.appfactory.jenkins.deploy.JenkinsArtifactDeployer/ClassName\ Endpointhttps://sc.s2.AF_HOST:9463/services//Endpoint\ RepositoryProvider\ Property name=Class\ org.wso2.carbon.appfactory.s4.integration.GITBlitBasedGITRepositoryProvider\ /Property\ Property name=BaseURLhttps://gitblit.s2.wso2.com:8444//Property\ Property name=URLPattern\{@stage\}/as/Property\ Property name=AdminUserNameadmin/Property\ Property name=AdminPasswordadmin/Property\ /RepositoryProvider\ Properties\ Property name=minInstances1/Property\ Property name=maxInstances1/Property\ Property name=shouldActivate/Property\ Property name=aliasasdev/Property\ !--mac182dev is the value --\ Property name=cartridgeTypeasdev/Property\ Property name=deploymentPolicyaf-deployment/Property\ Property name=autoscalePolicyeconomy/Property\ !--mac182dev is the value --\ Property name=repoURL/Property\ Property name=dataCartridgeType/Property\ Property name=dataCartridgeAlias/Property\ /Properties\ }___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Moving deployment section from configuration xml and Facilitate AppFactory users to deploy apps with their custom app types if its not already available
Hi Rajeevan, Please see my comments below. On Mon, Dec 1, 2014 at 10:46 PM, Aiyadurai Rajeevan rajeev...@wso2.com wrote: Hi All, We are trying to implement a feature on AF which enables the user to deploy their customized app types, Currently this configuration is available in *appfactory.xml *under *Deployer* tag the content would be as [1], likewise we have for each app types. Hence this wouldn't be editable by the users and may not deploy their own app types. If we move out this [1] from *appfactory.xml* and put this in a configurable file would enable the users to customize their need. When we analyzing the [1] we found out , The below content related to Pass artifact storage configuration and common to all app types. RepositoryProvider Property name=Class org.wso2.carbon.appfactory.s4.integration.GITBlitBasedGITRepositoryProvider /Property Property name=BaseURL https://gitblit.s2.wso2.com:8444//Property Property name=URLPattern{@stage}/as/Property Property name=AdminUserNameadmin/Property Property name=AdminPasswordadmin/Property /RepositoryProvider What is the other xml file are you talking about? Lets call it foo.xml. The only other place that require these information other than App Factory is Jenkins. So rather than putting these in separate xml file, I will put it in the appfactory-plugin.xml file. This is because Jenkins provide and inherent way to read appfactory-plugin.xml file and it doesn't give an inherent way to read foo.xml file. So at this stage the foo.xml file is only used in App Factory. You may merge it with appfactory.xml. The next question you may ask configuration duplication. I say puppet is the answer. If we have code to read foo.xml we have to put the code in both Jenkins side and AF side. I would rather use Jenkins inherent way of config files as oppose to duplicating code in both places and having a special file. So we are planning to move this to a common configuration file and all the application types can access that. In [1] below properties are not used. Hence, We shall get rid of that from AF Property name=minInstances1/Property Property name=maxInstances1/Property Property name=shouldActivate/Property Property name=subscribeOnDeploymentfalse /Property And the below properties are used. So, We can keep them in the *apptype.xml* as we already have separate apptype.xml for each app types.. Property name=alias162dev/Property Property name=cartridgeType162dev/Property Property name=deploymentPolicyaf-deployment /Property Property name=autoscalePolicyeconomy /Property Property name=repoURL/Property Property name=dataCartridgeType/Property Property name=dataCartridgeAlias/Property What is the propose of having the below in apptype.xml itself? What if we move this to a different runtime.xml file? I am proposing because Runtime is a different concept that is required by App Factory.Let me prove it by example. If we use a different xml file for deployment(runtime), in future we can implement multiple deployment environments very easily. For example consider the below relationship. AppType (1) - has -- (n)Deployment Envs Right now it is 1 to 1 relationship. In future it is 1 to n relationship. So if we keep it in a separate XML the work we have to do is simply externalise the deployment xmls to support the below scenario. For example consider app creation page. There is a Application Type drop down. In future we are going to have Runtimes drop down. So if we have deployment as a separate concept in the architecture it is going to be much better. thanks, dimuthu Look forward your views in this. Thanks Regards, S.A.Rajeevan Software Engineer WSO2 Inc E-Mail: rajeev...@wso2.com | Mobile : +94776411636 -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Cipher Tool to work in non-Carbon environments
Please find the PR https://github.com/wso2/cipher-tool/pull/4 On Mon, Dec 1, 2014 at 3:05 PM, Nirmal Fernando nir...@wso2.com wrote: We gonna introduce following properties file to hold the configuration parameters of the ciphertool. cipher-tool-config.properties Content: # this file contains the optional Ciphertool configuration parameters primary.key.location=security/client-truststore.jks primary.key.type=JKS primary.key.alias=wso2carbon cipher.text.properties.file=conf/cipher-text.properties cipher.tool.properties.file=conf/cipher-tool.properties secret.conf.properties.file=conf/secret-conf.properties On Fri, Nov 28, 2014 at 11:41 AM, Nirmal Fernando nir...@wso2.com wrote: On Fri, Nov 28, 2014 at 11:33 AM, Isuru Perera isu...@wso2.com wrote: Hi, +1 for making CipherTool a standalone tool. Rather than using system properties, let's use a configuration file specific to the tool. Yes, since this was implemented as a patch, we have to go with System properties. I'll introduce a config file, when I send a PR to Kernel. Both encrypt and decrypt logic should be independent from Carbon. It is, as of now. On Fri, Nov 28, 2014 at 9:45 AM, Nirmal Fernando nir...@wso2.com wrote: +1 On Fri, Nov 28, 2014 at 9:37 AM, Afkham Azeez az...@wso2.com wrote: I think properties file is clean and extensible. All the relevant properties can be seen configured in one location. On Fri, Nov 28, 2014 at 9:01 AM, Nirmal Fernando nir...@wso2.com wrote: Hi Azeez, On Fri, Nov 28, 2014 at 8:35 AM, Afkham Azeez az...@wso2.com wrote: Have you already started working on this? Yes, had to provide a support patch. I would prefer a ciphertool.properties or ciphertool.conf file, rather than System properties. Ciphertool has a ciphertool.sh file to invoke it. That's why I thought to use System properties. But of course, it's possible to introduce a properties file. If the file is not present, fallback to carbon.xml. However, to make this truly independent from Carbon, you should not take any dependency on Carbon kernel or components. Ciphertool has no dependency for any Carbon component/ Kernel. It's under dependencies and here's the pom file: https://github.com/wso2/commons/blob/master/ciphertool/pom.xml On Thu, Nov 27, 2014 at 11:13 PM, Nirmal Fernando nir...@wso2.com wrote: Hi All, Currently, our Cipher tool implementation is dependent upon Carbon runtime specific configuration files such as carbon.xml [1]. The idea is to, let you configure and run Cipher tool by simply setting System properties, so that you can utilize the tool to encrypt passwords even without having a Carbon runtime available. Of course, the current behavior will also be preserved, that is; if the System properties are not present, the required values will be read from the Carbon runtime specific configurations. Please share your thoughts. [1] https://github.com/wso2/commons/blob/master/ciphertool/src/main/java/org/wso2/ciphertool/CipherTool.java#L290 -- Thanks regards, Nirmal Senior Software Engineer- Platform Technologies Team, WSO2 Inc. Mobile: +94715779733 Blog: http://nirmalfdo.blogspot.com/ -- *Afkham Azeez* Director of Architecture; WSO2, Inc.; http://wso2.com Member; Apache Software Foundation; http://www.apache.org/ * http://www.apache.org/* *email: **az...@wso2.com* az...@wso2.com * cell: +94 77 3320919 %2B94%2077%203320919blog: * *http://blog.afkham.org* http://blog.afkham.org *twitter: **http://twitter.com/afkham_azeez* http://twitter.com/afkham_azeez *linked-in: **http://lk.linkedin.com/in/afkhamazeez http://lk.linkedin.com/in/afkhamazeez* *Lean . Enterprise . Middleware* -- Thanks regards, Nirmal Senior Software Engineer- Platform Technologies Team, WSO2 Inc. Mobile: +94715779733 Blog: http://nirmalfdo.blogspot.com/ -- *Afkham Azeez* Director of Architecture; WSO2, Inc.; http://wso2.com Member; Apache Software Foundation; http://www.apache.org/ * http://www.apache.org/* *email: **az...@wso2.com* az...@wso2.com * cell: +94 77 3320919 %2B94%2077%203320919blog: * *http://blog.afkham.org* http://blog.afkham.org *twitter: **http://twitter.com/afkham_azeez* http://twitter.com/afkham_azeez *linked-in: **http://lk.linkedin.com/in/afkhamazeez http://lk.linkedin.com/in/afkhamazeez* *Lean . Enterprise . Middleware* -- Thanks regards, Nirmal Senior Software Engineer- Platform Technologies Team, WSO2 Inc. Mobile: +94715779733 Blog: http://nirmalfdo.blogspot.com/ ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Isuru Perera Senior Software Engineer | WSO2, Inc. | http://wso2.com/ Lean . Enterprise . Middleware about.me/chrishantha ___ Architecture mailing list Architecture@wso2.org
Re: [Architecture] Moving deployment section from configuration xml and Facilitate AppFactory users to deploy apps with their custom app types if its not already available
Hi, On a side note, I would call it s/Deployment/Runtime. I think it is a more suitable name. thanks, dimuthu On Tue, Dec 2, 2014 at 8:45 AM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi Rajeevan, Please see my comments below. On Mon, Dec 1, 2014 at 10:46 PM, Aiyadurai Rajeevan rajeev...@wso2.com wrote: Hi All, We are trying to implement a feature on AF which enables the user to deploy their customized app types, Currently this configuration is available in *appfactory.xml *under *Deployer* tag the content would be as [1], likewise we have for each app types. Hence this wouldn't be editable by the users and may not deploy their own app types. If we move out this [1] from *appfactory.xml* and put this in a configurable file would enable the users to customize their need. When we analyzing the [1] we found out , The below content related to Pass artifact storage configuration and common to all app types. RepositoryProvider Property name=Class org.wso2.carbon.appfactory.s4.integration.GITBlitBasedGITRepositoryProvider /Property Property name=BaseURL https://gitblit.s2.wso2.com:8444//Property Property name=URLPattern{@stage}/as /Property Property name=AdminUserNameadmin/Property Property name=AdminPasswordadmin/Property /RepositoryProvider What is the other xml file are you talking about? Lets call it foo.xml. The only other place that require these information other than App Factory is Jenkins. So rather than putting these in separate xml file, I will put it in the appfactory-plugin.xml file. This is because Jenkins provide and inherent way to read appfactory-plugin.xml file and it doesn't give an inherent way to read foo.xml file. So at this stage the foo.xml file is only used in App Factory. You may merge it with appfactory.xml. The next question you may ask configuration duplication. I say puppet is the answer. If we have code to read foo.xml we have to put the code in both Jenkins side and AF side. I would rather use Jenkins inherent way of config files as oppose to duplicating code in both places and having a special file. So we are planning to move this to a common configuration file and all the application types can access that. In [1] below properties are not used. Hence, We shall get rid of that from AF Property name=minInstances1/Property Property name=maxInstances1/Property Property name=shouldActivate/Property Property name=subscribeOnDeploymentfalse /Property And the below properties are used. So, We can keep them in the *apptype.xml* as we already have separate apptype.xml for each app types.. Property name=alias162dev/Property Property name=cartridgeType162dev/Property Property name=deploymentPolicyaf-deployment /Property Property name=autoscalePolicyeconomy /Property Property name=repoURL/Property Property name=dataCartridgeType/Property Property name=dataCartridgeAlias/Property What is the propose of having the below in apptype.xml itself? What if we move this to a different runtime.xml file? I am proposing because Runtime is a different concept that is required by App Factory.Let me prove it by example. If we use a different xml file for deployment(runtime), in future we can implement multiple deployment environments very easily. For example consider the below relationship. AppType (1) - has -- (n)Deployment Envs Right now it is 1 to 1 relationship. In future it is 1 to n relationship. So if we keep it in a separate XML the work we have to do is simply externalise the deployment xmls to support the below scenario. For example consider app creation page. There is a Application Type drop down. In future we are going to have Runtimes drop down. So if we have deployment as a separate concept in the architecture it is going to be much better. thanks, dimuthu Look forward your views in this. Thanks Regards, S.A.Rajeevan Software Engineer WSO2 Inc E-Mail: rajeev...@wso2.com | Mobile : +94776411636 -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Moving deployment section from configuration xml and Facilitate AppFactory users to deploy apps with their custom app types if its not already available
Hi, +1 with the proposals. I couldn't find the appfactory-plugin.xml, though. Did you mean to say the jelly files(conf.jelly and global.jelly) which is inside the AppfactoryPluginManager? Thanks Regards, S.A.Rajeevan Software Engineer WSO2 Inc E-Mail: rajeev...@wso2.com | Mobile : +94776411636 On Tue, Dec 2, 2014 at 9:57 AM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi, On a side note, I would call it s/Deployment/Runtime. I think it is a more suitable name. thanks, dimuthu On Tue, Dec 2, 2014 at 8:45 AM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi Rajeevan, Please see my comments below. On Mon, Dec 1, 2014 at 10:46 PM, Aiyadurai Rajeevan rajeev...@wso2.com wrote: Hi All, We are trying to implement a feature on AF which enables the user to deploy their customized app types, Currently this configuration is available in *appfactory.xml *under *Deployer* tag the content would be as [1], likewise we have for each app types. Hence this wouldn't be editable by the users and may not deploy their own app types. If we move out this [1] from *appfactory.xml* and put this in a configurable file would enable the users to customize their need. When we analyzing the [1] we found out , The below content related to Pass artifact storage configuration and common to all app types. RepositoryProvider Property name=Class org.wso2.carbon.appfactory.s4.integration.GITBlitBasedGITRepositoryProvider /Property Property name=BaseURL https://gitblit.s2.wso2.com:8444//Property Property name=URLPattern{@stage}/as /Property Property name=AdminUserNameadmin/Property Property name=AdminPasswordadmin/Property /RepositoryProvider What is the other xml file are you talking about? Lets call it foo.xml. The only other place that require these information other than App Factory is Jenkins. So rather than putting these in separate xml file, I will put it in the appfactory-plugin.xml file. This is because Jenkins provide and inherent way to read appfactory-plugin.xml file and it doesn't give an inherent way to read foo.xml file. So at this stage the foo.xml file is only used in App Factory. You may merge it with appfactory.xml. The next question you may ask configuration duplication. I say puppet is the answer. If we have code to read foo.xml we have to put the code in both Jenkins side and AF side. I would rather use Jenkins inherent way of config files as oppose to duplicating code in both places and having a special file. So we are planning to move this to a common configuration file and all the application types can access that. In [1] below properties are not used. Hence, We shall get rid of that from AF Property name=minInstances1/Property Property name=maxInstances1/Property Property name=shouldActivate/Property Property name=subscribeOnDeploymentfalse /Property And the below properties are used. So, We can keep them in the *apptype.xml* as we already have separate apptype.xml for each app types.. Property name=alias162dev/Property Property name=cartridgeType162dev/Property Property name=deploymentPolicyaf-deployment /Property Property name=autoscalePolicyeconomy /Property Property name=repoURL/Property Property name=dataCartridgeType/Property Property name=dataCartridgeAlias/Property What is the propose of having the below in apptype.xml itself? What if we move this to a different runtime.xml file? I am proposing because Runtime is a different concept that is required by App Factory.Let me prove it by example. If we use a different xml file for deployment(runtime), in future we can implement multiple deployment environments very easily. For example consider the below relationship. AppType (1) - has -- (n)Deployment Envs Right now it is 1 to 1 relationship. In future it is 1 to n relationship. So if we keep it in a separate XML the work we have to do is simply externalise the deployment xmls to support the below scenario. For example consider app creation page. There is a Application Type drop down. In future we are going to have Runtimes drop down. So if we have deployment as a separate concept in the architecture it is going to be much better. thanks, dimuthu Look forward your views in this. Thanks Regards, S.A.Rajeevan Software Engineer WSO2 Inc E-Mail: rajeev...@wso2.com | Mobile : +94776411636 -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise
Re: [Architecture] Invitation: CDM Hackathon @ Mon Dec 1, 2014 1:30pm - 5:30pm (Sameera Perera)
Appending Day 1 Notes: *Goals* 1. Identify a CDM Core that's device/platform agnostic 2. Properly modularize the product (continuing the discussion in the thread [1]) 3. Evaluate how CDM can able to support Dynamic policies and policy merging which are frequently requested features *Terminology* - Feature (more accurately Device Management Feature): This is an operation that the CDM can instruct the device to perform. A MDM example of this would be to Put ringer in silent mode. On a smart thermostat, this could be Reduce Air Conditioning - Policy: A policy is a collection of features. The CDM Core, maintains the policy in a canonical format. Platform bundles may translate the policy in to a device dependent format. Both terms (feature and policy) are misnomers and may better termed as operations and profiles. However, we decided to stick to the terms as these are standard across the MDM industry. - Platform Bundle: Refer [1] *CDM Core* - Instructs the platform bundle to apply a policy to the device - The core determines which policy needs to be applied to the device through a Trigger. EMM 1.0 used 3 types of triggers: Platform, User and Role. CDM will introduce Location and Time. - When device calls back to the CDM, validates whether the device is violating its policy The Core will contain policy management and policy enforcement components as well as a policy editor. *MDM Module* - A MDM Module will sit on top of the core to enable feature parity with EMM 1.1. It will be written specifically with the knowledge that platform bundles for Android, iOS are available. - The MDM will define a common set of features that are available across all phones. By introducing this layer of abstraction we are able to keep manage mobile devices across mobile platforms as we've done in EMM without complicating the device agnostic capabilities of the core. Similar modules can be built by 3rd parties to manage other device categories such as thermostats, smart TVs etc. from different vendors/platforms. *Other notes* - We have demoted the following concepts to second class citizens and pulled them out of the core 1. OS Platform, version 2. Device Platform, version 3. Roles 1 and 2 only matter to the extend that they help us define a set of available features. Bundles will be responsible for managing this set based on these factors. The core will only be aware of the set. Roles were used in the EMM only to select the policy to apply on a user's device. We have moved this responsibility to the Trigger (or handler) chain. The core will only be aware of the chain, not what logic is applied to generate the policy. *For further discussions:* - More discussion around Trigger concept - Policy merging - Feature permissions [1] Proposed Architecture of CDM - architecture@ On Tue, Dec 2, 2014 at 10:10 AM, Asok Perera as...@wso2.com wrote: Hi All, Please find the meeting minutes for the yesterday meeting. *Attendees* : Sameera Perera, Kasun Dananjaya, Prabath Abeysekera, Asok Perera, Afkham Azeez, Geeth Munasinghe, Inosh Perera, Harshan Liyanage, Niranjan Karunanandham, Sumedha Rubasinghe, Srinath Perera, Manoj Dilshan - Knowledge transfer of the current EMM architecture - Moving all the device specific entries from Core EER to bundles (bundles will have their own EERs) - Checked the possibility of separating Mobile phone devices implementation from IoT as too much generalisation seems to make the architecture too complex (Not finalised yet) - Checked the possibility of moving device specific feature information to bundles, and keeping only the definition of features in core for managing policy information (Not finalised yet) - generalising core EER by removing/moving entries like IMEI, MAC etc - Checked the possibility of renaming some entries(role, policy, feature etc) according to the standard Kindly append if I have missed anything here. Regards *Asok Aravinda Perera* Software Engineer WSO2, Inc.;http://wso2.com/ http://www.google.com/url?q=http%3A%2F%2Fwso2.com%2Fsa=Dsntz=1usg=AFQjCNGJuLRux6KkJwXKVUCYOtEsNCmIAQ lean.enterprise.middleware Mobile: +94722241032 On Mon, Dec 1, 2014 at 12:25 PM, Sameera Perera samee...@wso2.com wrote: more details » https://www.google.com/calendar/event?action=VIEWeid=cjdjYmFnamZjdWY5dTFqYTEzdm5ibDlzaDAgZW5naW5lZXJpbmctZ3JvdXBAd3NvMi5jb20tok=MTcjc2FtZWVyYXBAd3NvMi5jb201OTJhZGJmZWMzM2Q4NmViNDczZWFiYzRmNzYyZTFhOTk4MjFhMWJjctz=Asia/Colombohl=en CDM Hackathon *When* Mon Dec 1, 2014 1:30pm – 5:30pm Colombo *Where* LK 3rd Floor Meeting Room - Kernel (map https://maps.google.lk/maps?q=LK+3rd+Floor+Meeting+Room+-+Kernelhl=en) *Video call* https://plus.google.com/hangouts/_/wso2.com/cdm-hackathon
Re: [Architecture] Moving deployment section from configuration xml and Facilitate AppFactory users to deploy apps with their custom app types if its not already available
Hi, We used to have a *plugin*.xml file. I can't find it anymore. Perhaps it is time to re-introduce it. thanks, dimuthu On Tue, Dec 2, 2014 at 10:31 AM, Aiyadurai Rajeevan rajeev...@wso2.com wrote: Hi, +1 with the proposals. I couldn't find the appfactory-plugin.xml, though. Did you mean to say the jelly files(conf.jelly and global.jelly) which is inside the AppfactoryPluginManager? Thanks Regards, S.A.Rajeevan Software Engineer WSO2 Inc E-Mail: rajeev...@wso2.com | Mobile : +94776411636 On Tue, Dec 2, 2014 at 9:57 AM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi, On a side note, I would call it s/Deployment/Runtime. I think it is a more suitable name. thanks, dimuthu On Tue, Dec 2, 2014 at 8:45 AM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi Rajeevan, Please see my comments below. On Mon, Dec 1, 2014 at 10:46 PM, Aiyadurai Rajeevan rajeev...@wso2.com wrote: Hi All, We are trying to implement a feature on AF which enables the user to deploy their customized app types, Currently this configuration is available in *appfactory.xml *under *Deployer* tag the content would be as [1], likewise we have for each app types. Hence this wouldn't be editable by the users and may not deploy their own app types. If we move out this [1] from *appfactory.xml* and put this in a configurable file would enable the users to customize their need. When we analyzing the [1] we found out , The below content related to Pass artifact storage configuration and common to all app types. RepositoryProvider Property name=Class org.wso2.carbon.appfactory.s4.integration.GITBlitBasedGITRepositoryProvider /Property Property name=BaseURL https://gitblit.s2.wso2.com:8444//Property Property name=URLPattern{@stage}/as /Property Property name=AdminUserNameadmin/Property Property name=AdminPasswordadmin /Property /RepositoryProvider What is the other xml file are you talking about? Lets call it foo.xml. The only other place that require these information other than App Factory is Jenkins. So rather than putting these in separate xml file, I will put it in the appfactory-plugin.xml file. This is because Jenkins provide and inherent way to read appfactory-plugin.xml file and it doesn't give an inherent way to read foo.xml file. So at this stage the foo.xml file is only used in App Factory. You may merge it with appfactory.xml. The next question you may ask configuration duplication. I say puppet is the answer. If we have code to read foo.xml we have to put the code in both Jenkins side and AF side. I would rather use Jenkins inherent way of config files as oppose to duplicating code in both places and having a special file. So we are planning to move this to a common configuration file and all the application types can access that. In [1] below properties are not used. Hence, We shall get rid of that from AF Property name=minInstances1/Property Property name=maxInstances1/Property Property name=shouldActivate/Property Property name=subscribeOnDeploymentfalse /Property And the below properties are used. So, We can keep them in the *apptype.xml* as we already have separate apptype.xml for each app types.. Property name=alias162dev/Property Property name=cartridgeType162dev /Property Property name=deploymentPolicyaf-deployment /Property Property name=autoscalePolicyeconomy /Property Property name=repoURL/Property Property name=dataCartridgeType/Property Property name=dataCartridgeAlias/Property What is the propose of having the below in apptype.xml itself? What if we move this to a different runtime.xml file? I am proposing because Runtime is a different concept that is required by App Factory.Let me prove it by example. If we use a different xml file for deployment(runtime), in future we can implement multiple deployment environments very easily. For example consider the below relationship. AppType (1) - has -- (n)Deployment Envs Right now it is 1 to 1 relationship. In future it is 1 to n relationship. So if we keep it in a separate XML the work we have to do is simply externalise the deployment xmls to support the below scenario. For example consider app creation page. There is a Application Type drop down. In future we are going to have Runtimes drop down. So if we have deployment as a separate concept in the architecture it is going to be much better. thanks, dimuthu Look forward your views in this. Thanks Regards, S.A.Rajeevan Software Engineer WSO2 Inc
Re: [Architecture] Moving deployment section from configuration xml and Facilitate AppFactory users to deploy apps with their custom app types if its not already available
HI Dimuthu Please find my comments inline On Tue, Dec 2, 2014 at 8:45 AM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi Rajeevan, Please see my comments below. On Mon, Dec 1, 2014 at 10:46 PM, Aiyadurai Rajeevan rajeev...@wso2.com wrote: Hi All, We are trying to implement a feature on AF which enables the user to deploy their customized app types, Currently this configuration is available in *appfactory.xml *under *Deployer* tag the content would be as [1], likewise we have for each app types. Hence this wouldn't be editable by the users and may not deploy their own app types. If we move out this [1] from *appfactory.xml* and put this in a configurable file would enable the users to customize their need. When we analyzing the [1] we found out , The below content related to Pass artifact storage configuration and common to all app types. RepositoryProvider Property name=Class org.wso2.carbon.appfactory.s4.integration.GITBlitBasedGITRepositoryProvider /Property Property name=BaseURL https://gitblit.s2.wso2.com:8444//Property Property name=URLPattern{@stage}/as /Property Property name=AdminUserNameadmin/Property Property name=AdminPasswordadmin/Property /RepositoryProvider What is the other xml file are you talking about? Lets call it foo.xml. The only other place that require these information other than App Factory is Jenkins. So rather than putting these in separate xml file, I will put it in the appfactory-plugin.xml file. This is because Jenkins provide and inherent way to read appfactory-plugin.xml file and it doesn't give an inherent way to read foo.xml file. So at this stage the foo.xml file is only used in App Factory. You may merge it with appfactory.xml. The next question you may ask configuration duplication. I say puppet is the answer. If we have code to read foo.xml we have to put the code in both Jenkins side and AF side. I would rather use Jenkins inherent way of config files as oppose to duplicating code in both places and having a special file. I was thinking about moving outside the tag but keep it inside the same appfactory.xml. We can pass these to jenkins through property bag when deployment. So we are planning to move this to a common configuration file and all the application types can access that. In [1] below properties are not used. Hence, We shall get rid of that from AF Property name=minInstances1/Property Property name=maxInstances1/Property Property name=shouldActivate/Property Property name=subscribeOnDeploymentfalse /Property And the below properties are used. So, We can keep them in the *apptype.xml* as we already have separate apptype.xml for each app types.. Property name=alias162dev/Property Property name=cartridgeType162dev/Property Property name=deploymentPolicyaf-deployment /Property Property name=autoscalePolicyeconomy /Property Property name=repoURL/Property Property name=dataCartridgeType/Property Property name=dataCartridgeAlias/Property What is the propose of having the below in apptype.xml itself? What if we move this to a different runtime.xml file? I am proposing because Runtime is a different concept that is required by App Factory.Let me prove it by example. If we use a different xml file for deployment(runtime), in future we can implement multiple deployment environments very easily. For example consider the below relationship. What I thought the cartidgeType only being used. But seems all of them used just for cartridge subscription. So we need a separate place to these indeed. And IMO we should add support to add the cartridge configuration file to the apptype as well. AppType (1) - has -- (n)Deployment Envs Question about this. Is it one to many or many to many. If it is many to many its going to be complex I guess. In that case we cannot deploy the deploy / runtime configs with apptype it self. And there should be a way to map which runtimes are available to which apptype. If it is one to many there will be same deployer mentioned in several app types. Should we focus on that feature here. If it is to implement many to many relationship then we are doing that feature now. Right now it is 1 to 1 relationship. In future it is 1 to n relationship. So if we keep it in a separate XML the work we have to do is simply externalise the deployment xmls to support the below scenario. For example consider app creation page. There is a Application Type drop down. In future we are going to have Runtimes drop down.
Re: [Architecture] How to forcefully remove an entry from app factory registry path cache?
Hi Shazni, I checked the code for removeCache method and found that cacheKey is calculated as below. *String connectionId = (dataBaseConfiguration.getUserName() != null? dataBaseConfiguration.getUserName().split(@)[0]:dataBaseConfiguration.getUserName()) + @ + dataBaseConfiguration.getDbUrl(); cacheKey = RegistryUtils.buildRegistryCacheKey(connectionId, tenantId, path);* But in registry.xml we have below element too, *cacheIdroot@jdbc:mysql://mysql1.appfactory.private.wso2.com:3306/dbGovernanceCloud http://mysql1.appfactory.private.wso2.com:3306/dbGovernanceCloud/cacheId* Shouldn't we read the cacheId element first and calculate as above if cacheId element is not defined? Or did I misunderstand the configuration? thank you. On Wed, Nov 12, 2014 at 2:23 PM, Shazni Nazeer sha...@wso2.com wrote: Hi, Given that we know the registry path of the resource of which cache to be deleted and have an instance of the registry we can manually delete the cache with a method like removeCache in the below file. However, it's not a clean or correct way of manipulating the registry cache. https://github.com/wso2-dev/carbon-governance/blob/master/components/governance/org.wso2.carbon.governance.custom.lifecycles.checklist/src/main/java/org/wso2/carbon/governance/custom/lifecycles/checklist/util/LifecycleBeanPopulator.java Shazni Nazeer Senior Software Engineer Mob : +94 37331 LinkedIn : http://lk.linkedin.com/in/shazninazeer Blog : http://shazninazeer.blogspot.com On Wed, Nov 12, 2014 at 1:48 PM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi Pulasthi, So when we are doing global invalidation, what is the method we are going to use to invalidate the cache within the JVM? :) Or are you going to do it by magic? thanks, dimuthu On Wed, Nov 12, 2014 at 1:36 PM, Pulasthi Supun pulas...@wso2.com wrote: Hi Dimuthu, On Tue, Nov 11, 2014 at 4:17 PM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi, On Tue, Nov 11, 2014 at 3:43 PM, Pulasthi Supun pulas...@wso2.com wrote: Hi Dimuthu, On Tue, Nov 11, 2014 at 2:05 PM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi Pulasthi, Yes. We do not need global invalidation (although it would solve the problem), but the request is to sink AF Registry cache with the DB. We are in the same JVM, and we need a method/way to tell registry remove this particular path from Registry path cache. Such a method would need to be accessed through something like the remote registry right?. The Registry api does not provide such a method to remove entries from the Registry cache. It should not be remote. An OSGi level method would be fine. Is there a way to patch the registry that we use? I talked with Azeez regarding this. He also agrees that providing such a method to manipulate the cache is wrong. We need to think of some other solution for this. I am not sure if the global cache invalidation has completed or can be backported into 4.2.0. @Amal is the work on that complete ? Regards, Pulasthi thanks, dimuthu Regards, Pulasthi thanks, dimuthu On Tue, Nov 11, 2014 at 12:16 PM, Pulasthi Supun pulas...@wso2.com wrote: Hi All, From what i understand the AF and SM are in different domains that is why distributed caching will not be able to handle this scenario right?. global cluster cache invalidation has been done with the use of a pub sub mechanism ( discussed in archi under Global cluster cache invalidation code review Notes ) but this will only be available in the next release AFAIK. Regards, Pulasthi On Tue, Nov 11, 2014 at 10:00 AM, Amalka Subasinghe ama...@wso2.com wrote: Hi, The scenario is, we have mounted SM's registry to the App Factory registry to remove the remote call for read the resources. but still the write calls happens via SM. see the image below. The problem is, once we do a write call to the SM's registry, App Factory registry cache won't get updated, so old information will be shown in App Factory pages. Is there a way to remove an entry from the App Factory registry cache when we do a write call to the SM's registry? then when we do a read call, it will load the cache again and show the updated information. Thanks Amalka ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- -- Pulasthi Supun Software Engineer; WSO2 Inc.; http://wso2.com, Email: pulas...@wso2.com Mobile: +94 (71) 9258281 Blog : http://pulasthisupun.blogspot.com/ Git hub profile: https://github.com/pulasthi -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware -- -- Pulasthi Supun Software Engineer; WSO2 Inc.; http://wso2.com, Email: pulas...@wso2.com Mobile: +94 (71) 9258281 Blog :
Re: [Architecture] How to forcefully remove an entry from app factory registry path cache?
Hi Manjula, As my understanding cacheKey is a RegistryCacheKey object, which is different from the cacheId. So we don't need to read the cacheId. Thanks. Mahendran Pirinthapan Software Engineer | WSO2 Inc. Mobile +94772378732. On Tue, Dec 2, 2014 at 12:01 PM, Manjula Rathnayake manju...@wso2.com wrote: Hi Shazni, I checked the code for removeCache method and found that cacheKey is calculated as below. *String connectionId = (dataBaseConfiguration.getUserName() != null? dataBaseConfiguration.getUserName().split(@)[0]:dataBaseConfiguration.getUserName()) + @ + dataBaseConfiguration.getDbUrl(); cacheKey = RegistryUtils.buildRegistryCacheKey(connectionId, tenantId, path);* But in registry.xml we have below element too, *cacheIdroot@jdbc:mysql://mysql1.appfactory.private.wso2.com:3306/dbGovernanceCloud http://mysql1.appfactory.private.wso2.com:3306/dbGovernanceCloud/cacheId* Shouldn't we read the cacheId element first and calculate as above if cacheId element is not defined? Or did I misunderstand the configuration? thank you. On Wed, Nov 12, 2014 at 2:23 PM, Shazni Nazeer sha...@wso2.com wrote: Hi, Given that we know the registry path of the resource of which cache to be deleted and have an instance of the registry we can manually delete the cache with a method like removeCache in the below file. However, it's not a clean or correct way of manipulating the registry cache. https://github.com/wso2-dev/carbon-governance/blob/master/components/governance/org.wso2.carbon.governance.custom.lifecycles.checklist/src/main/java/org/wso2/carbon/governance/custom/lifecycles/checklist/util/LifecycleBeanPopulator.java Shazni Nazeer Senior Software Engineer Mob : +94 37331 LinkedIn : http://lk.linkedin.com/in/shazninazeer Blog : http://shazninazeer.blogspot.com On Wed, Nov 12, 2014 at 1:48 PM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi Pulasthi, So when we are doing global invalidation, what is the method we are going to use to invalidate the cache within the JVM? :) Or are you going to do it by magic? thanks, dimuthu On Wed, Nov 12, 2014 at 1:36 PM, Pulasthi Supun pulas...@wso2.com wrote: Hi Dimuthu, On Tue, Nov 11, 2014 at 4:17 PM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi, On Tue, Nov 11, 2014 at 3:43 PM, Pulasthi Supun pulas...@wso2.com wrote: Hi Dimuthu, On Tue, Nov 11, 2014 at 2:05 PM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi Pulasthi, Yes. We do not need global invalidation (although it would solve the problem), but the request is to sink AF Registry cache with the DB. We are in the same JVM, and we need a method/way to tell registry remove this particular path from Registry path cache. Such a method would need to be accessed through something like the remote registry right?. The Registry api does not provide such a method to remove entries from the Registry cache. It should not be remote. An OSGi level method would be fine. Is there a way to patch the registry that we use? I talked with Azeez regarding this. He also agrees that providing such a method to manipulate the cache is wrong. We need to think of some other solution for this. I am not sure if the global cache invalidation has completed or can be backported into 4.2.0. @Amal is the work on that complete ? Regards, Pulasthi thanks, dimuthu Regards, Pulasthi thanks, dimuthu On Tue, Nov 11, 2014 at 12:16 PM, Pulasthi Supun pulas...@wso2.com wrote: Hi All, From what i understand the AF and SM are in different domains that is why distributed caching will not be able to handle this scenario right?. global cluster cache invalidation has been done with the use of a pub sub mechanism ( discussed in archi under Global cluster cache invalidation code review Notes ) but this will only be available in the next release AFAIK. Regards, Pulasthi On Tue, Nov 11, 2014 at 10:00 AM, Amalka Subasinghe ama...@wso2.com wrote: Hi, The scenario is, we have mounted SM's registry to the App Factory registry to remove the remote call for read the resources. but still the write calls happens via SM. see the image below. The problem is, once we do a write call to the SM's registry, App Factory registry cache won't get updated, so old information will be shown in App Factory pages. Is there a way to remove an entry from the App Factory registry cache when we do a write call to the SM's registry? then when we do a read call, it will load the cache again and show the updated information. Thanks Amalka ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- -- Pulasthi Supun Software Engineer; WSO2 Inc.; http://wso2.com, Email: pulas...@wso2.com Mobile: +94 (71) 9258281 Blog : http://pulasthisupun.blogspot.com/ Git hub profile: https://github.com/pulasthi -- Dimuthu
Re: [Architecture] Moving deployment section from configuration xml and Facilitate AppFactory users to deploy apps with their custom app types if its not already available
Hi Danushka, Please see my comments below. On Tue, Dec 2, 2014 at 12:01 PM, Danushka Fernando danush...@wso2.com wrote: HI Dimuthu Please find my comments inline On Tue, Dec 2, 2014 at 8:45 AM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi Rajeevan, Please see my comments below. On Mon, Dec 1, 2014 at 10:46 PM, Aiyadurai Rajeevan rajeev...@wso2.com wrote: Hi All, We are trying to implement a feature on AF which enables the user to deploy their customized app types, Currently this configuration is available in *appfactory.xml *under *Deployer* tag the content would be as [1], likewise we have for each app types. Hence this wouldn't be editable by the users and may not deploy their own app types. If we move out this [1] from *appfactory.xml* and put this in a configurable file would enable the users to customize their need. When we analyzing the [1] we found out , The below content related to Pass artifact storage configuration and common to all app types. RepositoryProvider Property name=Class org.wso2.carbon.appfactory.s4.integration.GITBlitBasedGITRepositoryProvider /Property Property name=BaseURL https://gitblit.s2.wso2.com:8444//Property Property name=URLPattern{@stage}/as /Property Property name=AdminUserNameadmin/Property Property name=AdminPasswordadmin/Property /RepositoryProvider What is the other xml file are you talking about? Lets call it foo.xml. The only other place that require these information other than App Factory is Jenkins. So rather than putting these in separate xml file, I will put it in the appfactory-plugin.xml file. This is because Jenkins provide and inherent way to read appfactory-plugin.xml file and it doesn't give an inherent way to read foo.xml file. So at this stage the foo.xml file is only used in App Factory. You may merge it with appfactory.xml. The next question you may ask configuration duplication. I say puppet is the answer. If we have code to read foo.xml we have to put the code in both Jenkins side and AF side. I would rather use Jenkins inherent way of config files as oppose to duplicating code in both places and having a special file. I was thinking about moving outside the tag but keep it inside the same appfactory.xml. We can pass these to jenkins through property bag when deployment. So we are planning to move this to a common configuration file and all the application types can access that. In [1] below properties are not used. Hence, We shall get rid of that from AF Property name=minInstances1/Property Property name=maxInstances1/Property Property name=shouldActivate/Property Property name=subscribeOnDeploymentfalse /Property And the below properties are used. So, We can keep them in the *apptype.xml* as we already have separate apptype.xml for each app types.. Property name=alias162dev/Property Property name=cartridgeType162dev/Property Property name=deploymentPolicyaf-deployment /Property Property name=autoscalePolicyeconomy /Property Property name=repoURL/Property Property name=dataCartridgeType/Property Property name=dataCartridgeAlias/Property What is the propose of having the below in apptype.xml itself? What if we move this to a different runtime.xml file? I am proposing because Runtime is a different concept that is required by App Factory.Let me prove it by example. If we use a different xml file for deployment(runtime), in future we can implement multiple deployment environments very easily. For example consider the below relationship. What I thought the cartidgeType only being used. But seems all of them used just for cartridge subscription. So we need a separate place to these indeed. And IMO we should add support to add the cartridge configuration file to the apptype as well. AppType (1) - has -- (n)Deployment Envs Question about this. Is it one to many or many to many. If it is many to many its going to be complex I guess. In that case we cannot deploy the deploy / runtime configs with apptype it self. And there should be a way to map which runtimes are available to which apptype. If it is one to many there will be same deployer mentioned in several app types. Should we focus on that feature here. If it is to implement many to many relationship then we are doing that feature now. You are correct in saying we need to concentrate on this current feature only. But since we are going to be doing refactoring anyway I propose to separate out Runtime concept gradually. There is no harm
Re: [Architecture] How to forcefully remove an entry from app factory registry path cache?
Hi Manjula, Yes. The cacheId that you specify is not the the '*connectionId*' that we create in the method. When a resource is added to the cache we take the ' *connectionId*' the way it is implemented to create the cache key. Therefore while retrieving the cache we should use the same way. Shazni Nazeer Senior Software Engineer Mob : +94 37331 LinkedIn : http://lk.linkedin.com/in/shazninazeer Blog : http://shazninazeer.blogspot.com On Tue, Dec 2, 2014 at 12:20 PM, Pirinthapan Mahendran pirintha...@wso2.com wrote: Hi Manjula, As my understanding cacheKey is a RegistryCacheKey object, which is different from the cacheId. So we don't need to read the cacheId. Thanks. Mahendran Pirinthapan Software Engineer | WSO2 Inc. Mobile +94772378732. On Tue, Dec 2, 2014 at 12:01 PM, Manjula Rathnayake manju...@wso2.com wrote: Hi Shazni, I checked the code for removeCache method and found that cacheKey is calculated as below. *String connectionId = (dataBaseConfiguration.getUserName() != null? dataBaseConfiguration.getUserName().split(@)[0]:dataBaseConfiguration.getUserName()) + @ + dataBaseConfiguration.getDbUrl(); cacheKey = RegistryUtils.buildRegistryCacheKey(connectionId, tenantId, path);* But in registry.xml we have below element too, *cacheIdroot@jdbc:mysql://mysql1.appfactory.private.wso2.com:3306/dbGovernanceCloud http://mysql1.appfactory.private.wso2.com:3306/dbGovernanceCloud/cacheId* Shouldn't we read the cacheId element first and calculate as above if cacheId element is not defined? Or did I misunderstand the configuration? thank you. On Wed, Nov 12, 2014 at 2:23 PM, Shazni Nazeer sha...@wso2.com wrote: Hi, Given that we know the registry path of the resource of which cache to be deleted and have an instance of the registry we can manually delete the cache with a method like removeCache in the below file. However, it's not a clean or correct way of manipulating the registry cache. https://github.com/wso2-dev/carbon-governance/blob/master/components/governance/org.wso2.carbon.governance.custom.lifecycles.checklist/src/main/java/org/wso2/carbon/governance/custom/lifecycles/checklist/util/LifecycleBeanPopulator.java Shazni Nazeer Senior Software Engineer Mob : +94 37331 LinkedIn : http://lk.linkedin.com/in/shazninazeer Blog : http://shazninazeer.blogspot.com On Wed, Nov 12, 2014 at 1:48 PM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi Pulasthi, So when we are doing global invalidation, what is the method we are going to use to invalidate the cache within the JVM? :) Or are you going to do it by magic? thanks, dimuthu On Wed, Nov 12, 2014 at 1:36 PM, Pulasthi Supun pulas...@wso2.com wrote: Hi Dimuthu, On Tue, Nov 11, 2014 at 4:17 PM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi, On Tue, Nov 11, 2014 at 3:43 PM, Pulasthi Supun pulas...@wso2.com wrote: Hi Dimuthu, On Tue, Nov 11, 2014 at 2:05 PM, Dimuthu Leelarathne dimut...@wso2.com wrote: Hi Pulasthi, Yes. We do not need global invalidation (although it would solve the problem), but the request is to sink AF Registry cache with the DB. We are in the same JVM, and we need a method/way to tell registry remove this particular path from Registry path cache. Such a method would need to be accessed through something like the remote registry right?. The Registry api does not provide such a method to remove entries from the Registry cache. It should not be remote. An OSGi level method would be fine. Is there a way to patch the registry that we use? I talked with Azeez regarding this. He also agrees that providing such a method to manipulate the cache is wrong. We need to think of some other solution for this. I am not sure if the global cache invalidation has completed or can be backported into 4.2.0. @Amal is the work on that complete ? Regards, Pulasthi thanks, dimuthu Regards, Pulasthi thanks, dimuthu On Tue, Nov 11, 2014 at 12:16 PM, Pulasthi Supun pulas...@wso2.com wrote: Hi All, From what i understand the AF and SM are in different domains that is why distributed caching will not be able to handle this scenario right?. global cluster cache invalidation has been done with the use of a pub sub mechanism ( discussed in archi under Global cluster cache invalidation code review Notes ) but this will only be available in the next release AFAIK. Regards, Pulasthi On Tue, Nov 11, 2014 at 10:00 AM, Amalka Subasinghe ama...@wso2.com wrote: Hi, The scenario is, we have mounted SM's registry to the App Factory registry to remove the remote call for read the resources. but still the write calls happens via SM. see the image below. The problem is, once we do a write call to the SM's registry, App Factory registry cache won't get updated, so old information will be shown in App Factory pages. Is there a way to remove an entry from the App Factory registry cache when we do