[Architecture] Siddhi - Script on It

2014-12-01 Thread Ayash
Hi All,

The implementation, $subject is adding some scripting support for Siddhi.
It is as follows,

1. Scripting languages are going to be JavaScript and Scala.
2. New language feature is to be introduced like below,
  Define script[Language]{//script implementation goes here}
3. Siddhi user can choose the scripting language mentioning it like below
in Siddhi,
  Define script[JavaScript] {//JavaScript implementation goes here}
  or
  Define script[Scala]{//Scala implementation goes here}
4. The functions or methods defined are to be available to the user to call
it like Siddhi inbuilt function.

For example, let's just define and call a Scala-method in Siddhi like below,

define script[Scala] {
Def *getSymbol*(id: String): String = id
}
define stream cseEventStream (symbol string, price float, volume string)
from cseEventStream
select *getSymbol*(symbol), convert(volume,long)/1000 as vol,
sum(convert(volume,long)) as sumVolume
group by symbol
insert into StockQuote;

Appreciate your thoughts!!!

Thanks,
-Ayash


-- 
Ayashkantha Ramasinghe
Software Engineer WSO2, Inc.
email: ayashkan...@wso2.com sanj...@wso2.com;
TP: +94 77 7 487 669
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture


Re: [Architecture] Generic RDBMS Output adapter support

2014-12-01 Thread Damith Wickramasinghe
Hi,

Thanks Anjana and Lahiru for the suggestions. Please find below the found
observations.


Following are the DB actions.

- DB creation (IF NOT EXIST)
- Table creation (IF NOT EXIST)
- Insert data to table
- update data to table

Since all the DBMS's are using same DDL and DML query formats we can keep
the query formats static for all DB types. Also we are only considering
above db actions only

eg:-

CREATE TABLE table_name
(
  column1 datatype [ NULL | NOT NULL ],
  column2 datatype [ NULL | NOT NULL ],
  ...
);

UPDATE table
SET column1 = expression1,
column2 = expression2,
...
WHERE conditions;


INSERT INTO table
(column1, column2, ... )
VALUES
(expression1, expression2, ... ),
(expression1, expression2, ... ),
...;


issue for the above implementation is the query syntax differs(Some) from
DB type to another (eg.data types) . So decided to use a XML file which has
DB type specific data types. So dynamically we can access the db type and
retrieve data types to generate the query in prepared statement.

Xml file would be like following.

mappings
mapping db='ansi'
types
  type
   fromstring/from
   toVARCHAR/to
  /type
  type
   fromdouble/from
   toDOUBLE/to
  /type

/types
/mapping
mapping db='mysql'
types
//mysqi specific data types if any

/types
/mapping
mapping db='oracle'
types
type
  fromstring/from
  tovarchar2/to
 /type
/types
/mapping
mapping db='mssql'
types
type
  fromstring/from
  tovarchar2/to
 /type
/types
/mapping
mapping db='h2'
types
//h2 specific data types if any
/types
/mapping
/mappings

Like wise above xml can be created.  Main idea is to use a separate XML to
manipulate different data types.  According to above XML first checks for
specific data type in specific db mappings. If not found then the value is
read from the generic data type mapping as stated as mapping db='ansi'.
All the data type mappings are not included in above configuration.

Your suggestions  feedbacks are highly appreciated.






On Sat, Nov 29, 2014 at 11:15 AM, Lahiru Cooray lahi...@wso2.com wrote:

 Hi,
 Maintenance wise it's nice to keep queries in separate XML.
 And I think it's better to keep all the generic queries in a one section
 of the XML and keep DBMS specific queries in separate sections. So you
 don't need to rewrite common queries for each type. Also better to consider
 MS SQL Server as well in testing  ( if it's supported DB type) as there are
 many syntax differences compared to Oracle and mysql

 On Fri, Nov 28, 2014 at 4:49 PM, Damith Wickramasinghe dami...@wso2.com
 wrote:

 Hi,

 Currently we have the support only for Mysql and it is decided to
 implement a generic adapter to support any RDBMS database. For now adapter
 implementation will be focused on supporting Oracle, Mysql and H2.

 I will update the thread on decided architecture for the said requirement
 soon. Any feedbacks on the requirement will be greatly appreciated.

 Regards,
 Damith.

 --
 Software Engineer
 WSO2 Inc.; http://wso2.com
 http://www.google.com/url?q=http%3A%2F%2Fwso2.comsa=Dsntz=1usg=AFQjCNEZvyc0uMD1HhBaEGCBxs6e9fBObg
 lean.enterprise.middleware

 mobile: *+94728671315 %2B94728671315*


 ___
 Architecture mailing list
 Architecture@wso2.org
 https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




 --
 *Lahiru Cooray*
 Software Engineer
 WSO2, Inc.;http://wso2.com/
 lean.enterprise.middleware

 Mobile: +94 715 654154

 ___
 Architecture mailing list
 Architecture@wso2.org
 https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




-- 
Software Engineer
WSO2 Inc.; http://wso2.com
http://www.google.com/url?q=http%3A%2F%2Fwso2.comsa=Dsntz=1usg=AFQjCNEZvyc0uMD1HhBaEGCBxs6e9fBObg
lean.enterprise.middleware

mobile: *+94728671315 %2B94728671315*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture


Re: [Architecture] Siddhi - Script on It

2014-12-01 Thread Sriskandarajah Suhothayan
+1 looks good

Suho

On Mon, Dec 1, 2014 at 4:57 PM, Ayash ayashkan...@wso2.com wrote:

 Hi All,

 The implementation, $subject is adding some scripting support for Siddhi.
 It is as follows,

 1. Scripting languages are going to be JavaScript and Scala.
 2. New language feature is to be introduced like below,
   Define script[Language]{//script implementation goes here}
 3. Siddhi user can choose the scripting language mentioning it like below
 in Siddhi,
   Define script[JavaScript] {//JavaScript implementation goes here}
   or
   Define script[Scala]{//Scala implementation goes here}
 4. The functions or methods defined are to be available to the user to
 call it like Siddhi inbuilt function.

 For example, let's just define and call a Scala-method in Siddhi like
 below,

 define script[Scala] {
 Def *getSymbol*(id: String): String = id
 }
 define stream cseEventStream (symbol string, price float, volume string)
 from cseEventStream
 select *getSymbol*(symbol), convert(volume,long)/1000 as vol,
 sum(convert(volume,long)) as sumVolume
 group by symbol
 insert into StockQuote;

 Appreciate your thoughts!!!

 Thanks,
 -Ayash


 --
 Ayashkantha Ramasinghe
 Software Engineer WSO2, Inc.
 email: ayashkan...@wso2.com sanj...@wso2.com;
 TP: +94 77 7 487 669




-- 

*S. Suhothayan*
Technical Lead  Team Lead of WSO2 Complex Event Processor
 *WSO2 Inc. *http://wso2.com
* http://wso2.com/*
lean . enterprise . middleware


*cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
http://suhothayan.blogspot.com/twitter: http://twitter.com/suhothayan
http://twitter.com/suhothayan | linked-in:
http://lk.linkedin.com/in/suhothayan http://lk.linkedin.com/in/suhothayan*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture


Re: [Architecture] Generic RDBMS Output adapter support

2014-12-01 Thread Damith Wickramasinghe
Hi,

please note above varchar length will be given as default. eg:-
varchar(250).

Regards,
damith.

On Mon, Dec 1, 2014 at 5:10 PM, Damith Wickramasinghe dami...@wso2.com
wrote:

 Hi,

 Thanks Anjana and Lahiru for the suggestions. Please find below the found
 observations.


 Following are the DB actions.

 - DB creation (IF NOT EXIST)
 - Table creation (IF NOT EXIST)
 - Insert data to table
 - update data to table

 Since all the DBMS's are using same DDL and DML query formats we can keep
 the query formats static for all DB types. Also we are only considering
 above db actions only

 eg:-

 CREATE TABLE table_name
 (
   column1 datatype [ NULL | NOT NULL ],
   column2 datatype [ NULL | NOT NULL ],
   ...
 );

 UPDATE table
 SET column1 = expression1,
 column2 = expression2,
 ...
 WHERE conditions;


 INSERT INTO table
 (column1, column2, ... )
 VALUES
 (expression1, expression2, ... ),
 (expression1, expression2, ... ),
 ...;


 issue for the above implementation is the query syntax differs(Some) from
 DB type to another (eg.data types) . So decided to use a XML file which has
 DB type specific data types. So dynamically we can access the db type and
 retrieve data types to generate the query in prepared statement.

 Xml file would be like following.

 mappings
 mapping db='ansi'
 types
   type
fromstring/from
toVARCHAR/to
   /type
   type
fromdouble/from
toDOUBLE/to
   /type

 /types
 /mapping
 mapping db='mysql'
 types
 //mysqi specific data types if any

 /types
 /mapping
 mapping db='oracle'
 types
 type
   fromstring/from
   tovarchar2/to
  /type
 /types
 /mapping
 mapping db='mssql'
 types
 type
   fromstring/from
   tovarchar2/to
  /type
 /types
 /mapping
 mapping db='h2'
 types
 //h2 specific data types if any
 /types
 /mapping
 /mappings

 Like wise above xml can be created.  Main idea is to use a separate XML to
 manipulate different data types.  According to above XML first checks for
 specific data type in specific db mappings. If not found then the value is
 read from the generic data type mapping as stated as mapping db='ansi'.
 All the data type mappings are not included in above configuration.

 Your suggestions  feedbacks are highly appreciated.






 On Sat, Nov 29, 2014 at 11:15 AM, Lahiru Cooray lahi...@wso2.com wrote:

 Hi,
 Maintenance wise it's nice to keep queries in separate XML.
 And I think it's better to keep all the generic queries in a one section
 of the XML and keep DBMS specific queries in separate sections. So you
 don't need to rewrite common queries for each type. Also better to consider
 MS SQL Server as well in testing  ( if it's supported DB type) as there are
 many syntax differences compared to Oracle and mysql

 On Fri, Nov 28, 2014 at 4:49 PM, Damith Wickramasinghe dami...@wso2.com
 wrote:

 Hi,

 Currently we have the support only for Mysql and it is decided to
 implement a generic adapter to support any RDBMS database. For now adapter
 implementation will be focused on supporting Oracle, Mysql and H2.

 I will update the thread on decided architecture for the said
 requirement soon. Any feedbacks on the requirement will be greatly
 appreciated.

 Regards,
 Damith.

 --
 Software Engineer
 WSO2 Inc.; http://wso2.com
 http://www.google.com/url?q=http%3A%2F%2Fwso2.comsa=Dsntz=1usg=AFQjCNEZvyc0uMD1HhBaEGCBxs6e9fBObg
 lean.enterprise.middleware

 mobile: *+94728671315 %2B94728671315*


 ___
 Architecture mailing list
 Architecture@wso2.org
 https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




 --
 *Lahiru Cooray*
 Software Engineer
 WSO2, Inc.;http://wso2.com/
 lean.enterprise.middleware

 Mobile: +94 715 654154

 ___
 Architecture mailing list
 Architecture@wso2.org
 https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




 --
 Software Engineer
 WSO2 Inc.; http://wso2.com
 http://www.google.com/url?q=http%3A%2F%2Fwso2.comsa=Dsntz=1usg=AFQjCNEZvyc0uMD1HhBaEGCBxs6e9fBObg
 lean.enterprise.middleware

 mobile: *+94728671315 %2B94728671315*




-- 
Software Engineer
WSO2 Inc.; http://wso2.com
http://www.google.com/url?q=http%3A%2F%2Fwso2.comsa=Dsntz=1usg=AFQjCNEZvyc0uMD1HhBaEGCBxs6e9fBObg
lean.enterprise.middleware

mobile: *+94728671315*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture


Re: [Architecture] Generic RDBMS Output adapter support

2014-12-01 Thread Anjana Fernando
Hi Damith,

I'm personally biased towards just using the whole queries in the XML,
where we may get some subtle changes in queries that may not be possible
with your parameterized approach. For example, in MySQL, you can give the
storage table engine as a part in the DDL query. Likewise, there maybe
special sections that is dependent on the RDBMS. So we may not want to
over-engineer this that much, a simple mapping of all the queries to a DBMS
is much simpler, and has more functionality. The redundancy of possible
duplication of standard queries across DBMS mappings can be ignored,
looking at the other potential advantages. Also, these table creation
queries, you may want to have a set of queries that will initialize the
tables, because you would want required indexes to be created too, and
index creation is anyway dependent on the target RDBMS.

Cheers,
Anjana.

On Mon, Dec 1, 2014 at 5:17 PM, Damith Wickramasinghe dami...@wso2.com
wrote:

 Hi,

 please note above varchar length will be given as default. eg:-
 varchar(250).

 Regards,
 damith.

 On Mon, Dec 1, 2014 at 5:10 PM, Damith Wickramasinghe dami...@wso2.com
 wrote:

 Hi,

 Thanks Anjana and Lahiru for the suggestions. Please find below the found
 observations.


 Following are the DB actions.

 - DB creation (IF NOT EXIST)
 - Table creation (IF NOT EXIST)
 - Insert data to table
 - update data to table

 Since all the DBMS's are using same DDL and DML query formats we can keep
 the query formats static for all DB types. Also we are only considering
 above db actions only

 eg:-

 CREATE TABLE table_name
 (
   column1 datatype [ NULL | NOT NULL ],
   column2 datatype [ NULL | NOT NULL ],
   ...
 );

 UPDATE table
 SET column1 = expression1,
 column2 = expression2,
 ...
 WHERE conditions;


 INSERT INTO table
 (column1, column2, ... )
 VALUES
 (expression1, expression2, ... ),
 (expression1, expression2, ... ),
 ...;


 issue for the above implementation is the query syntax differs(Some) from
 DB type to another (eg.data types) . So decided to use a XML file which has
 DB type specific data types. So dynamically we can access the db type and
 retrieve data types to generate the query in prepared statement.

 Xml file would be like following.

 mappings
 mapping db='ansi'
 types
   type
fromstring/from
toVARCHAR/to
   /type
   type
fromdouble/from
toDOUBLE/to
   /type

 /types
 /mapping
 mapping db='mysql'
 types
 //mysqi specific data types if any

 /types
 /mapping
 mapping db='oracle'
 types
 type
   fromstring/from
   tovarchar2/to
  /type
 /types
 /mapping
 mapping db='mssql'
 types
 type
   fromstring/from
   tovarchar2/to
  /type
 /types
 /mapping
 mapping db='h2'
 types
 //h2 specific data types if any
 /types
 /mapping
 /mappings

 Like wise above xml can be created.  Main idea is to use a separate XML
 to manipulate different data types.  According to above XML first checks
 for specific data type in specific db mappings. If not found then the value
 is read from the generic data type mapping as stated as mapping
 db='ansi'.  All the data type mappings are not included in above
 configuration.

 Your suggestions  feedbacks are highly appreciated.






 On Sat, Nov 29, 2014 at 11:15 AM, Lahiru Cooray lahi...@wso2.com wrote:

 Hi,
 Maintenance wise it's nice to keep queries in separate XML.
 And I think it's better to keep all the generic queries in a one section
 of the XML and keep DBMS specific queries in separate sections. So you
 don't need to rewrite common queries for each type. Also better to consider
 MS SQL Server as well in testing  ( if it's supported DB type) as there are
 many syntax differences compared to Oracle and mysql

 On Fri, Nov 28, 2014 at 4:49 PM, Damith Wickramasinghe dami...@wso2.com
  wrote:

 Hi,

 Currently we have the support only for Mysql and it is decided to
 implement a generic adapter to support any RDBMS database. For now adapter
 implementation will be focused on supporting Oracle, Mysql and H2.

 I will update the thread on decided architecture for the said
 requirement soon. Any feedbacks on the requirement will be greatly
 appreciated.

 Regards,
 Damith.

 --
 Software Engineer
 WSO2 Inc.; http://wso2.com
 http://www.google.com/url?q=http%3A%2F%2Fwso2.comsa=Dsntz=1usg=AFQjCNEZvyc0uMD1HhBaEGCBxs6e9fBObg
 lean.enterprise.middleware

 mobile: *+94728671315 %2B94728671315*


 ___
 Architecture mailing list
 Architecture@wso2.org
 https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




 --
 *Lahiru Cooray*
 Software Engineer
 WSO2, Inc.;http://wso2.com/
 lean.enterprise.middleware

 Mobile: +94 715 654154

 ___
 Architecture 

Re: [Architecture] Generic RDBMS Output adapter support

2014-12-01 Thread Akalanka Pagoda Arachchi
Hi Damitha,

As Anjana suggested, separating only the data types would lead to some
confusions. Furthermore  might, sometimes lead to not using a specific
database system to it's fullest potential. For example, given that there
are a huge amount of built-in functions in MSSQL we might be able to use a
simpler, efficient query to complete our task in MSSQL that in other
database systems (just an example). Also apart from the functions you have
mentioned, in the future the system might need some more functions and we
have to make sure that extensibility is supported. Therefore, having a
separate XML for queries in each database system is more suitable in my
opinion too.

Thanks,
Akalanka

On Mon, Dec 1, 2014 at 5:26 PM, Anjana Fernando anj...@wso2.com wrote:

 Hi Damith,

 I'm personally biased towards just using the whole queries in the XML,
 where we may get some subtle changes in queries that may not be possible
 with your parameterized approach. For example, in MySQL, you can give the
 storage table engine as a part in the DDL query. Likewise, there maybe
 special sections that is dependent on the RDBMS. So we may not want to
 over-engineer this that much, a simple mapping of all the queries to a DBMS
 is much simpler, and has more functionality. The redundancy of possible
 duplication of standard queries across DBMS mappings can be ignored,
 looking at the other potential advantages. Also, these table creation
 queries, you may want to have a set of queries that will initialize the
 tables, because you would want required indexes to be created too, and
 index creation is anyway dependent on the target RDBMS.

 Cheers,
 Anjana.

 On Mon, Dec 1, 2014 at 5:17 PM, Damith Wickramasinghe dami...@wso2.com
 wrote:

 Hi,

 please note above varchar length will be given as default. eg:-
 varchar(250).

 Regards,
 damith.

 On Mon, Dec 1, 2014 at 5:10 PM, Damith Wickramasinghe dami...@wso2.com
 wrote:

 Hi,

 Thanks Anjana and Lahiru for the suggestions. Please find below the
 found observations.


 Following are the DB actions.

 - DB creation (IF NOT EXIST)
 - Table creation (IF NOT EXIST)
 - Insert data to table
 - update data to table

 Since all the DBMS's are using same DDL and DML query formats we can
 keep the query formats static for all DB types. Also we are only
 considering above db actions only

 eg:-

 CREATE TABLE table_name
 (
   column1 datatype [ NULL | NOT NULL ],
   column2 datatype [ NULL | NOT NULL ],
   ...
 );

 UPDATE table
 SET column1 = expression1,
 column2 = expression2,
 ...
 WHERE conditions;


 INSERT INTO table
 (column1, column2, ... )
 VALUES
 (expression1, expression2, ... ),
 (expression1, expression2, ... ),
 ...;


 issue for the above implementation is the query syntax differs(Some)
 from DB type to another (eg.data types) . So decided to use a XML file
 which has DB type specific data types. So dynamically we can access the db
 type and retrieve data types to generate the query in prepared statement.

 Xml file would be like following.

 mappings
 mapping db='ansi'
 types
   type
fromstring/from
toVARCHAR/to
   /type
   type
fromdouble/from
toDOUBLE/to
   /type

 /types
 /mapping
 mapping db='mysql'
 types
 //mysqi specific data types if any

 /types
 /mapping
 mapping db='oracle'
 types
 type
   fromstring/from
   tovarchar2/to
  /type
 /types
 /mapping
 mapping db='mssql'
 types
 type
   fromstring/from
   tovarchar2/to
  /type
 /types
 /mapping
 mapping db='h2'
 types
 //h2 specific data types if any
 /types
 /mapping
 /mappings

 Like wise above xml can be created.  Main idea is to use a separate XML
 to manipulate different data types.  According to above XML first checks
 for specific data type in specific db mappings. If not found then the value
 is read from the generic data type mapping as stated as mapping
 db='ansi'.  All the data type mappings are not included in above
 configuration.

 Your suggestions  feedbacks are highly appreciated.






 On Sat, Nov 29, 2014 at 11:15 AM, Lahiru Cooray lahi...@wso2.com
 wrote:

 Hi,
 Maintenance wise it's nice to keep queries in separate XML.
 And I think it's better to keep all the generic queries in a one
 section of the XML and keep DBMS specific queries in separate sections. So
 you don't need to rewrite common queries for each type. Also better to
 consider MS SQL Server as well in testing  ( if it's supported DB type) as
 there are many syntax differences compared to Oracle and mysql

 On Fri, Nov 28, 2014 at 4:49 PM, Damith Wickramasinghe 
 dami...@wso2.com wrote:

 Hi,

 Currently we have the support only for Mysql and it is decided to
 implement a generic adapter to support any RDBMS database. For now adapter
 implementation will 

Re: [Architecture] Generic RDBMS Output adapter support

2014-12-01 Thread Damith Wickramasinghe
Hi,

Thanks Anjana and Akalanka for the suggestions. @Anjana As per the off-line
discussion we had, I will add the necessary for the implementation.

Regards,
Damith.

On Mon, Dec 1, 2014 at 5:49 PM, Akalanka Pagoda Arachchi darsha...@wso2.com
 wrote:

 Hi Damitha,

 As Anjana suggested, separating only the data types would lead to some
 confusions. Furthermore  might, sometimes lead to not using a specific
 database system to it's fullest potential. For example, given that there
 are a huge amount of built-in functions in MSSQL we might be able to use a
 simpler, efficient query to complete our task in MSSQL that in other
 database systems (just an example). Also apart from the functions you have
 mentioned, in the future the system might need some more functions and we
 have to make sure that extensibility is supported. Therefore, having a
 separate XML for queries in each database system is more suitable in my
 opinion too.

 Thanks,
 Akalanka

 On Mon, Dec 1, 2014 at 5:26 PM, Anjana Fernando anj...@wso2.com wrote:

 Hi Damith,

 I'm personally biased towards just using the whole queries in the XML,
 where we may get some subtle changes in queries that may not be possible
 with your parameterized approach. For example, in MySQL, you can give the
 storage table engine as a part in the DDL query. Likewise, there maybe
 special sections that is dependent on the RDBMS. So we may not want to
 over-engineer this that much, a simple mapping of all the queries to a DBMS
 is much simpler, and has more functionality. The redundancy of possible
 duplication of standard queries across DBMS mappings can be ignored,
 looking at the other potential advantages. Also, these table creation
 queries, you may want to have a set of queries that will initialize the
 tables, because you would want required indexes to be created too, and
 index creation is anyway dependent on the target RDBMS.

 Cheers,
 Anjana.

 On Mon, Dec 1, 2014 at 5:17 PM, Damith Wickramasinghe dami...@wso2.com
 wrote:

 Hi,

 please note above varchar length will be given as default. eg:-
 varchar(250).

 Regards,
 damith.

 On Mon, Dec 1, 2014 at 5:10 PM, Damith Wickramasinghe dami...@wso2.com
 wrote:

 Hi,

 Thanks Anjana and Lahiru for the suggestions. Please find below the
 found observations.


 Following are the DB actions.

 - DB creation (IF NOT EXIST)
 - Table creation (IF NOT EXIST)
 - Insert data to table
 - update data to table

 Since all the DBMS's are using same DDL and DML query formats we can
 keep the query formats static for all DB types. Also we are only
 considering above db actions only

 eg:-

 CREATE TABLE table_name
 (
   column1 datatype [ NULL | NOT NULL ],
   column2 datatype [ NULL | NOT NULL ],
   ...
 );

 UPDATE table
 SET column1 = expression1,
 column2 = expression2,
 ...
 WHERE conditions;


 INSERT INTO table
 (column1, column2, ... )
 VALUES
 (expression1, expression2, ... ),
 (expression1, expression2, ... ),
 ...;


 issue for the above implementation is the query syntax differs(Some)
 from DB type to another (eg.data types) . So decided to use a XML file
 which has DB type specific data types. So dynamically we can access the db
 type and retrieve data types to generate the query in prepared statement.

 Xml file would be like following.

 mappings
 mapping db='ansi'
 types
   type
fromstring/from
toVARCHAR/to
   /type
   type
fromdouble/from
toDOUBLE/to
   /type

 /types
 /mapping
 mapping db='mysql'
 types
 //mysqi specific data types if any

 /types
 /mapping
 mapping db='oracle'
 types
 type
   fromstring/from
   tovarchar2/to
  /type
 /types
 /mapping
 mapping db='mssql'
 types
 type
   fromstring/from
   tovarchar2/to
  /type
 /types
 /mapping
 mapping db='h2'
 types
 //h2 specific data types if any
 /types
 /mapping
 /mappings

 Like wise above xml can be created.  Main idea is to use a separate XML
 to manipulate different data types.  According to above XML first checks
 for specific data type in specific db mappings. If not found then the value
 is read from the generic data type mapping as stated as mapping
 db='ansi'.  All the data type mappings are not included in above
 configuration.

 Your suggestions  feedbacks are highly appreciated.






 On Sat, Nov 29, 2014 at 11:15 AM, Lahiru Cooray lahi...@wso2.com
 wrote:

 Hi,
 Maintenance wise it's nice to keep queries in separate XML.
 And I think it's better to keep all the generic queries in a one
 section of the XML and keep DBMS specific queries in separate sections. So
 you don't need to rewrite common queries for each type. Also better to
 consider MS SQL Server as well in testing  ( if it's supported DB type) as
 there are many syntax differences compared to 

[Architecture] Moving deployment section from configuration xml and Facilitate AppFactory users to deploy apps with their custom app types if its not already available

2014-12-01 Thread Aiyadurai Rajeevan
Hi All,

We are trying to implement a feature on AF which enables the user to deploy
their customized app types, Currently this configuration is available
in *appfactory.xml
*under *Deployer* tag the content would be as [1], likewise we have for
each app types. Hence this wouldn't be editable by the users and may not
deploy their own app types. If we move out this [1] from *appfactory.xml*
and put this in a configurable file would enable the users to customize
their need.

When we analyzing the [1] we found out , The below content related to Pass
artifact storage configuration and common to all app types.

RepositoryProvider

   Property name=Class


org.wso2.carbon.appfactory.s4.integration.GITBlitBasedGITRepositoryProvider

/Property

   Property name=BaseURL
https://gitblit.s2.wso2.com:8444//Property

Property name=URLPattern{@stage}/as/Property

   Property name=AdminUserNameadmin/Property

 Property name=AdminPasswordadmin/Property

/RepositoryProvider


So we are planning to move this to a common configuration file and all the
application types can access that.

In [1] below properties are not used. Hence, We shall get rid of that from
AF

Property name=minInstances1/Property

Property name=maxInstances1/Property

Property name=shouldActivate/Property
Property name=subscribeOnDeploymentfalse
/Property

And the below properties are used. So, We can keep them in the *apptype.xml* as
we already have separate apptype.xml for each app types..

Property name=alias162dev/Property

Property name=cartridgeType162dev/Property

   Property name=deploymentPolicyaf-deployment
/Property

Property name=autoscalePolicyeconomy/Property

Property name=repoURL/Property

   Property name=dataCartridgeType/Property

   Property name=dataCartridgeAlias/Property


Look forward your views in this.

Thanks  Regards,
S.A.Rajeevan
Software Engineer WSO2 Inc
E-Mail: rajeev...@wso2.com | Mobile : +94776411636
{\rtf1\ansi\ansicpg1252\cocoartf1343\cocoasubrtf140
{\fonttbl\f0\fnil\fcharset0 Monaco;}
{\colortbl;\red255\green255\blue255;\red19\green150\blue163;\red178\green178\blue178;}
\margl1440\margr1440\vieww10800\viewh8400\viewkind0
\pard\tx560\tx1120\tx1680\tx2240\tx2800\tx3360\tx3920\tx4480\tx5040\tx5600\tx6160\tx6720\pardirnatural

\f0\fs20 \cf2 \cb3 \CocoaLigature0  ApplicationType name=*\
\
ClassNameorg.wso2.carbon.appfactory.jenkins.deploy.JenkinsArtifactDeployer/ClassName\
Endpointhttps://sc.s2.AF_HOST:9463/services//Endpoint\
RepositoryProvider\
Property name=Class\
org.wso2.carbon.appfactory.s4.integration.GITBlitBasedGITRepositoryProvider\
/Property\
Property name=BaseURLhttps://gitblit.s2.wso2.com:8444//Property\
Property name=URLPattern\{@stage\}/as/Property\
Property name=AdminUserNameadmin/Property\
Property name=AdminPasswordadmin/Property\
/RepositoryProvider\
Properties\
Property name=minInstances1/Property\
Property name=maxInstances1/Property\
Property name=shouldActivate/Property\
Property name=aliasasdev/Property\
!--mac182dev is the value --\
Property name=cartridgeTypeasdev/Property\
Property name=deploymentPolicyaf-deployment/Property\
Property name=autoscalePolicyeconomy/Property\
!--mac182dev is the value --\
Property name=repoURL/Property\
Property name=dataCartridgeType/Property\
Property name=dataCartridgeAlias/Property\
/Properties\
}___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture


Re: [Architecture] Moving deployment section from configuration xml and Facilitate AppFactory users to deploy apps with their custom app types if its not already available

2014-12-01 Thread Dimuthu Leelarathne
Hi Rajeevan,

Please see my comments below.

On Mon, Dec 1, 2014 at 10:46 PM, Aiyadurai Rajeevan rajeev...@wso2.com
wrote:

 Hi All,

 We are trying to implement a feature on AF which enables the user to
 deploy their customized app types, Currently this configuration is
 available in *appfactory.xml *under *Deployer* tag the content would be
 as [1], likewise we have for each app types. Hence this wouldn't be
 editable by the users and may not deploy their own app types. If we move
 out this [1] from *appfactory.xml* and put this in a configurable file
 would enable the users to customize their need.

 When we analyzing the [1] we found out , The below content related to Pass
 artifact storage configuration and common to all app types.

 RepositoryProvider

Property name=Class


 org.wso2.carbon.appfactory.s4.integration.GITBlitBasedGITRepositoryProvider

 /Property

Property name=BaseURL
 https://gitblit.s2.wso2.com:8444//Property

 Property name=URLPattern{@stage}/as/Property

Property name=AdminUserNameadmin/Property

  Property name=AdminPasswordadmin/Property

 /RepositoryProvider


What is the other xml file are you talking about? Lets call it foo.xml. The
only other place that require these information other than App Factory is
Jenkins. So rather than putting these in separate xml file, I will put it
in the appfactory-plugin.xml file. This is because Jenkins provide and
inherent way to read appfactory-plugin.xml file and it doesn't give an
inherent way to read foo.xml file.

So at this stage the foo.xml file is only used in App Factory. You may
merge it with appfactory.xml. The next question you may ask configuration
duplication. I say puppet is the answer.

If we have code to read foo.xml we have to put the code in both Jenkins
side and AF side. I would rather use Jenkins inherent way of config files
as oppose to duplicating code in both places and having a special file.


So we are planning to move this to a common configuration file and all the
 application types can access that.

 In [1] below properties are not used. Hence, We shall get rid of that from
 AF

 Property name=minInstances1/Property

 Property name=maxInstances1/Property

 Property name=shouldActivate/Property
 Property name=subscribeOnDeploymentfalse
 /Property

 And the below properties are used. So, We can keep them in the
 *apptype.xml* as we already have separate apptype.xml for each app types..

Property name=alias162dev/Property

 Property name=cartridgeType162dev/Property

Property name=deploymentPolicyaf-deployment
 /Property

 Property name=autoscalePolicyeconomy
 /Property

 Property name=repoURL/Property

Property name=dataCartridgeType/Property

Property name=dataCartridgeAlias/Property





What is the propose of having the below in apptype.xml itself? What if we
move this to a different runtime.xml file? I am proposing because Runtime
is a different concept that is required by App Factory.Let me prove it by
example. If we use a different xml file for deployment(runtime), in future
we can implement multiple deployment environments very easily. For example
consider the below relationship.

AppType (1) - has --  (n)Deployment Envs

Right now it is 1 to 1 relationship. In future it is 1 to n relationship.
So if we keep it in a separate XML the work we have to do is simply
externalise the deployment xmls to support the below scenario.

For example consider app creation page.

There is a Application Type drop down. In future we are going to have
Runtimes drop down. So if we have deployment as a separate concept in the
architecture it is going to be much better.

thanks,
dimuthu



 Look forward your views in this.

 Thanks  Regards,
 S.A.Rajeevan
 Software Engineer WSO2 Inc
 E-Mail: rajeev...@wso2.com | Mobile : +94776411636




-- 
Dimuthu Leelarathne
Architect  Product Lead of App Factory

WSO2, Inc. (http://wso2.com)
email: dimut...@wso2.com
Mobile : 0773661935

Lean . Enterprise . Middleware
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture


Re: [Architecture] Cipher Tool to work in non-Carbon environments

2014-12-01 Thread Nirmal Fernando
Please find the PR https://github.com/wso2/cipher-tool/pull/4

On Mon, Dec 1, 2014 at 3:05 PM, Nirmal Fernando nir...@wso2.com wrote:

 We gonna introduce following properties file to hold the configuration
 parameters of the ciphertool.

 cipher-tool-config.properties

 Content:

 # this file contains the optional Ciphertool configuration parameters


 primary.key.location=security/client-truststore.jks

 primary.key.type=JKS

 primary.key.alias=wso2carbon

 cipher.text.properties.file=conf/cipher-text.properties

 cipher.tool.properties.file=conf/cipher-tool.properties

 secret.conf.properties.file=conf/secret-conf.properties

 On Fri, Nov 28, 2014 at 11:41 AM, Nirmal Fernando nir...@wso2.com wrote:



 On Fri, Nov 28, 2014 at 11:33 AM, Isuru Perera isu...@wso2.com wrote:

 Hi,

 +1 for making CipherTool a standalone tool. Rather than using system
 properties, let's use a configuration file specific to the tool.


 Yes, since this was implemented as a patch, we have to go with System
 properties. I'll introduce a config file, when I send a PR to Kernel.



 Both encrypt and decrypt logic should be independent from Carbon.


 It is, as of now.


 On Fri, Nov 28, 2014 at 9:45 AM, Nirmal Fernando nir...@wso2.com
 wrote:

 +1

 On Fri, Nov 28, 2014 at 9:37 AM, Afkham Azeez az...@wso2.com wrote:

 I think properties file is clean and extensible. All the relevant
 properties can be seen  configured in one location.

 On Fri, Nov 28, 2014 at 9:01 AM, Nirmal Fernando nir...@wso2.com
 wrote:

 Hi Azeez,

 On Fri, Nov 28, 2014 at 8:35 AM, Afkham Azeez az...@wso2.com wrote:

 Have you already started working on this?


 Yes, had to provide a support patch.


 I would prefer a ciphertool.properties or ciphertool.conf file,
 rather than System properties.


 Ciphertool has a ciphertool.sh file to invoke it. That's why I
 thought to use System properties. But of course, it's possible to 
 introduce
 a properties file.


 If the file is not present, fallback to carbon.xml.

 However, to make this truly independent from Carbon, you should not
 take any dependency on Carbon kernel or components.


 Ciphertool has no dependency for any Carbon component/ Kernel. It's
 under dependencies and here's the pom file:
 https://github.com/wso2/commons/blob/master/ciphertool/pom.xml


 On Thu, Nov 27, 2014 at 11:13 PM, Nirmal Fernando nir...@wso2.com
 wrote:

 Hi All,

 Currently, our Cipher tool implementation is dependent upon Carbon
 runtime specific configuration files such as carbon.xml [1].

 The idea is to, let you configure and run Cipher tool by simply
 setting System properties, so that you can utilize the tool to encrypt
 passwords even without having a Carbon runtime available.

 Of course, the current behavior will also be preserved, that is; if
 the System properties are not present, the required values will be read
 from the Carbon runtime specific configurations.

 Please share your thoughts.

 [1]
 https://github.com/wso2/commons/blob/master/ciphertool/src/main/java/org/wso2/ciphertool/CipherTool.java#L290

 --

 Thanks  regards,
 Nirmal

 Senior Software Engineer- Platform Technologies Team, WSO2 Inc.
 Mobile: +94715779733
 Blog: http://nirmalfdo.blogspot.com/





 --
 *Afkham Azeez*
 Director of Architecture; WSO2, Inc.; http://wso2.com
 Member; Apache Software Foundation; http://www.apache.org/
 * http://www.apache.org/*
 *email: **az...@wso2.com* az...@wso2.com
 * cell: +94 77 3320919 %2B94%2077%203320919blog: *
 *http://blog.afkham.org* http://blog.afkham.org
 *twitter: **http://twitter.com/afkham_azeez*
 http://twitter.com/afkham_azeez
 *linked-in: **http://lk.linkedin.com/in/afkhamazeez
 http://lk.linkedin.com/in/afkhamazeez*

 *Lean . Enterprise . Middleware*




 --

 Thanks  regards,
 Nirmal

 Senior Software Engineer- Platform Technologies Team, WSO2 Inc.
 Mobile: +94715779733
 Blog: http://nirmalfdo.blogspot.com/





 --
 *Afkham Azeez*
 Director of Architecture; WSO2, Inc.; http://wso2.com
 Member; Apache Software Foundation; http://www.apache.org/
 * http://www.apache.org/*
 *email: **az...@wso2.com* az...@wso2.com
 * cell: +94 77 3320919 %2B94%2077%203320919blog: *
 *http://blog.afkham.org* http://blog.afkham.org
 *twitter: **http://twitter.com/afkham_azeez*
 http://twitter.com/afkham_azeez
 *linked-in: **http://lk.linkedin.com/in/afkhamazeez
 http://lk.linkedin.com/in/afkhamazeez*

 *Lean . Enterprise . Middleware*




 --

 Thanks  regards,
 Nirmal

 Senior Software Engineer- Platform Technologies Team, WSO2 Inc.
 Mobile: +94715779733
 Blog: http://nirmalfdo.blogspot.com/



 ___
 Architecture mailing list
 Architecture@wso2.org
 https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




 --
 Isuru Perera
 Senior Software Engineer | WSO2, Inc. | http://wso2.com/
 Lean . Enterprise . Middleware

 about.me/chrishantha

 ___
 Architecture mailing list
 Architecture@wso2.org
 

Re: [Architecture] Moving deployment section from configuration xml and Facilitate AppFactory users to deploy apps with their custom app types if its not already available

2014-12-01 Thread Dimuthu Leelarathne
Hi,

On a side note, I would call it s/Deployment/Runtime. I think it is a more
suitable name.

thanks,
dimuthu


On Tue, Dec 2, 2014 at 8:45 AM, Dimuthu Leelarathne dimut...@wso2.com
wrote:

 Hi Rajeevan,

 Please see my comments below.

 On Mon, Dec 1, 2014 at 10:46 PM, Aiyadurai Rajeevan rajeev...@wso2.com
 wrote:

 Hi All,

 We are trying to implement a feature on AF which enables the user to
 deploy their customized app types, Currently this configuration is
 available in *appfactory.xml *under *Deployer* tag the content would
 be as [1], likewise we have for each app types. Hence this wouldn't be
 editable by the users and may not deploy their own app types. If we move
 out this [1] from *appfactory.xml* and put this in a configurable file
 would enable the users to customize their need.

 When we analyzing the [1] we found out , The below content related to
 Pass artifact storage configuration and common to all app types.

 RepositoryProvider

Property name=Class


 org.wso2.carbon.appfactory.s4.integration.GITBlitBasedGITRepositoryProvider

 /Property

Property name=BaseURL
 https://gitblit.s2.wso2.com:8444//Property

 Property name=URLPattern{@stage}/as
 /Property

Property name=AdminUserNameadmin/Property

  Property name=AdminPasswordadmin/Property

 /RepositoryProvider


 What is the other xml file are you talking about? Lets call it foo.xml.
 The only other place that require these information other than App Factory
 is Jenkins. So rather than putting these in separate xml file, I will put
 it in the appfactory-plugin.xml file. This is because Jenkins provide and
 inherent way to read appfactory-plugin.xml file and it doesn't give an
 inherent way to read foo.xml file.

 So at this stage the foo.xml file is only used in App Factory. You may
 merge it with appfactory.xml. The next question you may ask configuration
 duplication. I say puppet is the answer.

 If we have code to read foo.xml we have to put the code in both Jenkins
 side and AF side. I would rather use Jenkins inherent way of config files
 as oppose to duplicating code in both places and having a special file.


 So we are planning to move this to a common configuration file and all the
 application types can access that.

 In [1] below properties are not used. Hence, We shall get rid of that
 from AF

 Property name=minInstances1/Property

 Property name=maxInstances1/Property

 Property name=shouldActivate/Property
 Property name=subscribeOnDeploymentfalse
 /Property

 And the below properties are used. So, We can keep them in the
 *apptype.xml* as we already have separate apptype.xml for each app
 types..

 Property name=alias162dev/Property

 Property name=cartridgeType162dev/Property

Property name=deploymentPolicyaf-deployment
 /Property

 Property name=autoscalePolicyeconomy
 /Property

 Property name=repoURL/Property

Property name=dataCartridgeType/Property

Property name=dataCartridgeAlias/Property





 What is the propose of having the below in apptype.xml itself? What if we
 move this to a different runtime.xml file? I am proposing because Runtime
 is a different concept that is required by App Factory.Let me prove it by
 example. If we use a different xml file for deployment(runtime), in future
 we can implement multiple deployment environments very easily. For example
 consider the below relationship.

 AppType (1) - has --  (n)Deployment Envs

 Right now it is 1 to 1 relationship. In future it is 1 to n relationship.
 So if we keep it in a separate XML the work we have to do is simply
 externalise the deployment xmls to support the below scenario.

 For example consider app creation page.

 There is a Application Type drop down. In future we are going to have
 Runtimes drop down. So if we have deployment as a separate concept in the
 architecture it is going to be much better.

 thanks,
 dimuthu



 Look forward your views in this.

 Thanks  Regards,
 S.A.Rajeevan
 Software Engineer WSO2 Inc
 E-Mail: rajeev...@wso2.com | Mobile : +94776411636




 --
 Dimuthu Leelarathne
 Architect  Product Lead of App Factory

 WSO2, Inc. (http://wso2.com)
 email: dimut...@wso2.com
 Mobile : 0773661935

 Lean . Enterprise . Middleware




-- 
Dimuthu Leelarathne
Architect  Product Lead of App Factory

WSO2, Inc. (http://wso2.com)
email: dimut...@wso2.com
Mobile : 0773661935

Lean . Enterprise . Middleware
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture


Re: [Architecture] Moving deployment section from configuration xml and Facilitate AppFactory users to deploy apps with their custom app types if its not already available

2014-12-01 Thread Aiyadurai Rajeevan
Hi,

+1 with the proposals. I couldn't find the appfactory-plugin.xml, though.
Did you mean to say the jelly files(conf.jelly and global.jelly) which is
inside the AppfactoryPluginManager?

Thanks  Regards,
S.A.Rajeevan
Software Engineer WSO2 Inc
E-Mail: rajeev...@wso2.com | Mobile : +94776411636

On Tue, Dec 2, 2014 at 9:57 AM, Dimuthu Leelarathne dimut...@wso2.com
wrote:

 Hi,

 On a side note, I would call it s/Deployment/Runtime. I think it is a more
 suitable name.

 thanks,
 dimuthu


 On Tue, Dec 2, 2014 at 8:45 AM, Dimuthu Leelarathne dimut...@wso2.com
 wrote:

 Hi Rajeevan,

 Please see my comments below.

 On Mon, Dec 1, 2014 at 10:46 PM, Aiyadurai Rajeevan rajeev...@wso2.com
 wrote:

 Hi All,

 We are trying to implement a feature on AF which enables the user to
 deploy their customized app types, Currently this configuration is
 available in *appfactory.xml *under *Deployer* tag the content would
 be as [1], likewise we have for each app types. Hence this wouldn't be
 editable by the users and may not deploy their own app types. If we move
 out this [1] from *appfactory.xml* and put this in a configurable file
 would enable the users to customize their need.

 When we analyzing the [1] we found out , The below content related to
 Pass artifact storage configuration and common to all app types.

 RepositoryProvider

Property name=Class


 org.wso2.carbon.appfactory.s4.integration.GITBlitBasedGITRepositoryProvider

 /Property

Property name=BaseURL
 https://gitblit.s2.wso2.com:8444//Property

 Property name=URLPattern{@stage}/as
 /Property

Property name=AdminUserNameadmin/Property

  Property name=AdminPasswordadmin/Property

 /RepositoryProvider


 What is the other xml file are you talking about? Lets call it foo.xml.
 The only other place that require these information other than App Factory
 is Jenkins. So rather than putting these in separate xml file, I will put
 it in the appfactory-plugin.xml file. This is because Jenkins provide and
 inherent way to read appfactory-plugin.xml file and it doesn't give an
 inherent way to read foo.xml file.

 So at this stage the foo.xml file is only used in App Factory. You may
 merge it with appfactory.xml. The next question you may ask configuration
 duplication. I say puppet is the answer.

 If we have code to read foo.xml we have to put the code in both Jenkins
 side and AF side. I would rather use Jenkins inherent way of config files
 as oppose to duplicating code in both places and having a special file.


 So we are planning to move this to a common configuration file and all
 the application types can access that.

 In [1] below properties are not used. Hence, We shall get rid of that
 from AF

 Property name=minInstances1/Property

 Property name=maxInstances1/Property

 Property name=shouldActivate/Property
 Property name=subscribeOnDeploymentfalse
 /Property

 And the below properties are used. So, We can keep them in the
 *apptype.xml* as we already have separate apptype.xml for each app
 types..

 Property name=alias162dev/Property

 Property name=cartridgeType162dev/Property

Property name=deploymentPolicyaf-deployment
 /Property

 Property name=autoscalePolicyeconomy
 /Property

 Property name=repoURL/Property

Property name=dataCartridgeType/Property

Property name=dataCartridgeAlias/Property





 What is the propose of having the below in apptype.xml itself? What if we
 move this to a different runtime.xml file? I am proposing because Runtime
 is a different concept that is required by App Factory.Let me prove it by
 example. If we use a different xml file for deployment(runtime), in future
 we can implement multiple deployment environments very easily. For example
 consider the below relationship.

 AppType (1) - has --  (n)Deployment Envs

 Right now it is 1 to 1 relationship. In future it is 1 to n relationship.
 So if we keep it in a separate XML the work we have to do is simply
 externalise the deployment xmls to support the below scenario.

 For example consider app creation page.

 There is a Application Type drop down. In future we are going to have
 Runtimes drop down. So if we have deployment as a separate concept in the
 architecture it is going to be much better.

 thanks,
 dimuthu



 Look forward your views in this.

 Thanks  Regards,
 S.A.Rajeevan
 Software Engineer WSO2 Inc
 E-Mail: rajeev...@wso2.com | Mobile : +94776411636




 --
 Dimuthu Leelarathne
 Architect  Product Lead of App Factory

 WSO2, Inc. (http://wso2.com)
 email: dimut...@wso2.com
 Mobile : 0773661935

 Lean . Enterprise 

Re: [Architecture] Invitation: CDM Hackathon @ Mon Dec 1, 2014 1:30pm - 5:30pm (Sameera Perera)

2014-12-01 Thread Sameera Perera
Appending

Day 1 Notes:

*Goals*

   1. Identify a CDM Core that's device/platform agnostic
   2. Properly modularize the product (continuing the discussion in the
   thread [1])
   3. Evaluate how CDM can able to support Dynamic policies and policy
   merging which are frequently requested features

*Terminology*

   - Feature (more accurately Device Management Feature): This is an
   operation that the CDM can instruct the device to perform. A MDM example of
   this would be to Put ringer in silent mode. On a smart thermostat, this
   could be Reduce Air Conditioning
   - Policy: A policy is a collection of features. The CDM Core, maintains
   the policy in a canonical format. Platform bundles may translate the policy
   in to a device dependent format.

Both terms (feature and policy) are misnomers and may better termed as
operations and profiles. However, we decided to stick to the terms as these
are standard across the MDM industry.


   - Platform Bundle: Refer [1]


*CDM Core*

   - Instructs the platform bundle to apply a policy to the device
   - The core determines which policy needs to be applied to the device
   through a Trigger. EMM 1.0 used 3 types of triggers: Platform, User and
   Role. CDM will introduce Location and Time.
   - When device calls back to the CDM, validates whether the device is
   violating its policy

The Core will contain policy management and policy enforcement components
as well as a policy editor.

*MDM Module*

   - A MDM Module will sit on top of the core to enable feature parity with
   EMM 1.1. It will be written specifically with the knowledge that platform
   bundles for Android, iOS are available.
   - The MDM will define a common set of features that are available across
   all phones.

By introducing this layer of abstraction we are able to keep manage mobile
devices across mobile platforms as we've done in EMM without complicating
the device agnostic capabilities of the core. Similar modules can be built
by 3rd parties to manage other device categories such as thermostats, smart
TVs etc. from different vendors/platforms.

*Other notes*

   - We have demoted the following concepts to second class citizens and
   pulled them out of the core
  1. OS Platform, version
  2. Device Platform, version
  3. Roles

1 and 2 only matter to the extend that they help us define a set of
available features. Bundles will be responsible for managing this set based
on these factors. The core will only be aware of the set.
Roles were used in the EMM only to select the policy to apply on a user's
device. We have moved this responsibility to the Trigger (or handler)
chain. The core will only be aware of the chain, not what logic is applied
to generate the policy.

*For further discussions:*

   - More discussion around Trigger concept
   - Policy merging
   - Feature permissions

[1] Proposed Architecture of CDM - architecture@


On Tue, Dec 2, 2014 at 10:10 AM, Asok Perera as...@wso2.com wrote:

 Hi All,

 Please find the meeting minutes for the yesterday meeting.

 *Attendees* : Sameera Perera, Kasun Dananjaya, Prabath Abeysekera, Asok
 Perera, Afkham Azeez, Geeth Munasinghe, Inosh Perera, Harshan Liyanage, 
 Niranjan
 Karunanandham, Sumedha Rubasinghe, Srinath Perera, Manoj Dilshan


- Knowledge transfer of the current EMM architecture
- Moving all the device specific entries from Core EER to bundles
(bundles will have their own EERs)
- Checked the possibility of separating Mobile phone devices
implementation from IoT as too much generalisation seems to make the
architecture too complex (Not finalised yet)
- Checked the possibility of moving device specific feature
information to bundles, and keeping only the definition of features in core
for managing policy information (Not finalised yet)
- generalising core EER by removing/moving entries like IMEI, MAC etc
- Checked the possibility of renaming some entries(role, policy,
feature etc) according to the standard

 Kindly append if I have missed anything here.

 Regards

 *Asok Aravinda Perera*
 Software Engineer
 WSO2, Inc.;http://wso2.com/
 http://www.google.com/url?q=http%3A%2F%2Fwso2.com%2Fsa=Dsntz=1usg=AFQjCNGJuLRux6KkJwXKVUCYOtEsNCmIAQ
 lean.enterprise.middleware

 Mobile: +94722241032

 On Mon, Dec 1, 2014 at 12:25 PM, Sameera Perera samee...@wso2.com wrote:

 more details »
 https://www.google.com/calendar/event?action=VIEWeid=cjdjYmFnamZjdWY5dTFqYTEzdm5ibDlzaDAgZW5naW5lZXJpbmctZ3JvdXBAd3NvMi5jb20tok=MTcjc2FtZWVyYXBAd3NvMi5jb201OTJhZGJmZWMzM2Q4NmViNDczZWFiYzRmNzYyZTFhOTk4MjFhMWJjctz=Asia/Colombohl=en
 CDM Hackathon
 *When*
 Mon Dec 1, 2014 1:30pm – 5:30pm Colombo
 *Where*
 LK 3rd Floor Meeting Room - Kernel (map
 https://maps.google.lk/maps?q=LK+3rd+Floor+Meeting+Room+-+Kernelhl=en)
 *Video call*
 https://plus.google.com/hangouts/_/wso2.com/cdm-hackathon
 

Re: [Architecture] Moving deployment section from configuration xml and Facilitate AppFactory users to deploy apps with their custom app types if its not already available

2014-12-01 Thread Dimuthu Leelarathne
Hi,

We used to have a *plugin*.xml file. I can't find it anymore. Perhaps it is
time to re-introduce it.

thanks,
dimuthu

On Tue, Dec 2, 2014 at 10:31 AM, Aiyadurai Rajeevan rajeev...@wso2.com
wrote:

 Hi,

 +1 with the proposals. I couldn't find the appfactory-plugin.xml, though.
 Did you mean to say the jelly files(conf.jelly and global.jelly) which is
 inside the AppfactoryPluginManager?

 Thanks  Regards,
 S.A.Rajeevan
 Software Engineer WSO2 Inc
 E-Mail: rajeev...@wso2.com | Mobile : +94776411636

 On Tue, Dec 2, 2014 at 9:57 AM, Dimuthu Leelarathne dimut...@wso2.com
 wrote:

 Hi,

 On a side note, I would call it s/Deployment/Runtime. I think it is a
 more suitable name.

 thanks,
 dimuthu


 On Tue, Dec 2, 2014 at 8:45 AM, Dimuthu Leelarathne dimut...@wso2.com
 wrote:

 Hi Rajeevan,

 Please see my comments below.

 On Mon, Dec 1, 2014 at 10:46 PM, Aiyadurai Rajeevan rajeev...@wso2.com
 wrote:

 Hi All,

 We are trying to implement a feature on AF which enables the user to
 deploy their customized app types, Currently this configuration is
 available in *appfactory.xml *under *Deployer* tag the content would
 be as [1], likewise we have for each app types. Hence this wouldn't be
 editable by the users and may not deploy their own app types. If we move
 out this [1] from *appfactory.xml* and put this in a configurable file
 would enable the users to customize their need.

 When we analyzing the [1] we found out , The below content related to
 Pass artifact storage configuration and common to all app types.

 RepositoryProvider

Property name=Class


 org.wso2.carbon.appfactory.s4.integration.GITBlitBasedGITRepositoryProvider

 /Property

Property name=BaseURL
 https://gitblit.s2.wso2.com:8444//Property

 Property name=URLPattern{@stage}/as
 /Property

Property name=AdminUserNameadmin/Property

  Property name=AdminPasswordadmin
 /Property

 /RepositoryProvider


 What is the other xml file are you talking about? Lets call it foo.xml.
 The only other place that require these information other than App Factory
 is Jenkins. So rather than putting these in separate xml file, I will put
 it in the appfactory-plugin.xml file. This is because Jenkins provide and
 inherent way to read appfactory-plugin.xml file and it doesn't give an
 inherent way to read foo.xml file.

 So at this stage the foo.xml file is only used in App Factory. You may
 merge it with appfactory.xml. The next question you may ask configuration
 duplication. I say puppet is the answer.

 If we have code to read foo.xml we have to put the code in both Jenkins
 side and AF side. I would rather use Jenkins inherent way of config files
 as oppose to duplicating code in both places and having a special file.


 So we are planning to move this to a common configuration file and all
 the application types can access that.

 In [1] below properties are not used. Hence, We shall get rid of that
 from AF

 Property name=minInstances1/Property

 Property name=maxInstances1/Property

 Property name=shouldActivate/Property
 Property name=subscribeOnDeploymentfalse
 /Property

 And the below properties are used. So, We can keep them in the
 *apptype.xml* as we already have separate apptype.xml for each app
 types..

 Property name=alias162dev/Property

 Property name=cartridgeType162dev
 /Property

Property name=deploymentPolicyaf-deployment
 /Property

 Property name=autoscalePolicyeconomy
 /Property

 Property name=repoURL/Property

Property name=dataCartridgeType/Property

Property name=dataCartridgeAlias/Property





 What is the propose of having the below in apptype.xml itself? What if
 we move this to a different runtime.xml file? I am proposing because
 Runtime is a different concept that is required by App Factory.Let me prove
 it by example. If we use a different xml file for deployment(runtime), in
 future we can implement multiple deployment environments very easily. For
 example consider the below relationship.

 AppType (1) - has --  (n)Deployment Envs

 Right now it is 1 to 1 relationship. In future it is 1 to n
 relationship. So if we keep it in a separate XML the work we have to do is
 simply externalise the deployment xmls to support the below scenario.

 For example consider app creation page.

 There is a Application Type drop down. In future we are going to have
 Runtimes drop down. So if we have deployment as a separate concept in the
 architecture it is going to be much better.

 thanks,
 dimuthu



 Look forward your views in this.

 Thanks  Regards,
 S.A.Rajeevan
 Software Engineer WSO2 Inc
 

Re: [Architecture] Moving deployment section from configuration xml and Facilitate AppFactory users to deploy apps with their custom app types if its not already available

2014-12-01 Thread Danushka Fernando
HI Dimuthu
Please find my comments inline


On Tue, Dec 2, 2014 at 8:45 AM, Dimuthu Leelarathne dimut...@wso2.com
wrote:

 Hi Rajeevan,

 Please see my comments below.

 On Mon, Dec 1, 2014 at 10:46 PM, Aiyadurai Rajeevan rajeev...@wso2.com
 wrote:

 Hi All,

 We are trying to implement a feature on AF which enables the user to
 deploy their customized app types, Currently this configuration is
 available in *appfactory.xml *under *Deployer* tag the content would
 be as [1], likewise we have for each app types. Hence this wouldn't be
 editable by the users and may not deploy their own app types. If we move
 out this [1] from *appfactory.xml* and put this in a configurable file
 would enable the users to customize their need.

 When we analyzing the [1] we found out , The below content related to
 Pass artifact storage configuration and common to all app types.

 RepositoryProvider

Property name=Class


 org.wso2.carbon.appfactory.s4.integration.GITBlitBasedGITRepositoryProvider

 /Property

Property name=BaseURL
 https://gitblit.s2.wso2.com:8444//Property

 Property name=URLPattern{@stage}/as
 /Property

Property name=AdminUserNameadmin/Property

  Property name=AdminPasswordadmin/Property

 /RepositoryProvider


 What is the other xml file are you talking about? Lets call it foo.xml.
 The only other place that require these information other than App Factory
 is Jenkins. So rather than putting these in separate xml file, I will put
 it in the appfactory-plugin.xml file. This is because Jenkins provide and
 inherent way to read appfactory-plugin.xml file and it doesn't give an
 inherent way to read foo.xml file.

 So at this stage the foo.xml file is only used in App Factory. You may
 merge it with appfactory.xml. The next question you may ask configuration
 duplication. I say puppet is the answer.

 If we have code to read foo.xml we have to put the code in both Jenkins
 side and AF side. I would rather use Jenkins inherent way of config files
 as oppose to duplicating code in both places and having a special file.


I was thinking about moving outside the tag but keep it inside the same
appfactory.xml. We can pass these to jenkins through property bag when
deployment.



 So we are planning to move this to a common configuration file and all the
 application types can access that.

 In [1] below properties are not used. Hence, We shall get rid of that
 from AF

 Property name=minInstances1/Property

 Property name=maxInstances1/Property

 Property name=shouldActivate/Property
 Property name=subscribeOnDeploymentfalse
 /Property

 And the below properties are used. So, We can keep them in the
 *apptype.xml* as we already have separate apptype.xml for each app
 types..

 Property name=alias162dev/Property

 Property name=cartridgeType162dev/Property

Property name=deploymentPolicyaf-deployment
 /Property

 Property name=autoscalePolicyeconomy
 /Property

 Property name=repoURL/Property

Property name=dataCartridgeType/Property

Property name=dataCartridgeAlias/Property





 What is the propose of having the below in apptype.xml itself? What if we
 move this to a different runtime.xml file? I am proposing because Runtime
 is a different concept that is required by App Factory.Let me prove it by
 example. If we use a different xml file for deployment(runtime), in future
 we can implement multiple deployment environments very easily. For example
 consider the below relationship.


What I thought the cartidgeType only being used. But seems all of them used
just for cartridge subscription. So we need a separate place to these
indeed. And IMO we should add support to add the cartridge configuration
file to the apptype as well.


 AppType (1) - has --  (n)Deployment Envs

 Question about this. Is it one to many or many to many. If it is many to
many its going to be complex I guess. In that case we cannot deploy the
deploy / runtime configs with apptype it self. And there should be a way to
map which runtimes are available to which apptype. If it is one to many
there will be same deployer mentioned in several app types.

Should we focus on that feature here. If it is to implement many to many
relationship then we are doing that feature now.


 Right now it is 1 to 1 relationship. In future it is 1 to n relationship.
 So if we keep it in a separate XML the work we have to do is simply
 externalise the deployment xmls to support the below scenario.

 For example consider app creation page.

 There is a Application Type drop down. In future we are going to have
 Runtimes drop down. 

Re: [Architecture] How to forcefully remove an entry from app factory registry path cache?

2014-12-01 Thread Manjula Rathnayake
Hi Shazni,

I checked the code for removeCache method and found that cacheKey is
calculated as below.

*String connectionId = (dataBaseConfiguration.getUserName() != null?
dataBaseConfiguration.getUserName().split(@)[0]:dataBaseConfiguration.getUserName())
+ @ + dataBaseConfiguration.getDbUrl(); cacheKey =
RegistryUtils.buildRegistryCacheKey(connectionId, tenantId, path);*

But in registry.xml we have below element too,
*cacheIdroot@jdbc:mysql://mysql1.appfactory.private.wso2.com:3306/dbGovernanceCloud
http://mysql1.appfactory.private.wso2.com:3306/dbGovernanceCloud/cacheId*

Shouldn't we read the cacheId element first and calculate as above if
cacheId element is not defined?  Or did I misunderstand the configuration?

thank you.

On Wed, Nov 12, 2014 at 2:23 PM, Shazni Nazeer sha...@wso2.com wrote:

 Hi,

 Given that we know the registry path of the resource of which cache to be
 deleted and have an instance of the registry we can manually delete the
 cache with a method like removeCache in the below file. However, it's not a
 clean or correct way of manipulating the registry cache.


 https://github.com/wso2-dev/carbon-governance/blob/master/components/governance/org.wso2.carbon.governance.custom.lifecycles.checklist/src/main/java/org/wso2/carbon/governance/custom/lifecycles/checklist/util/LifecycleBeanPopulator.java

 Shazni Nazeer

 Senior Software Engineer

 Mob : +94 37331
 LinkedIn : http://lk.linkedin.com/in/shazninazeer
 Blog : http://shazninazeer.blogspot.com

 On Wed, Nov 12, 2014 at 1:48 PM, Dimuthu Leelarathne dimut...@wso2.com
 wrote:

 Hi Pulasthi,

 So when we are doing global invalidation, what is the method we are going
 to use to invalidate the cache within the JVM? :) Or are you going to do it
 by magic?

 thanks,
 dimuthu

 On Wed, Nov 12, 2014 at 1:36 PM, Pulasthi Supun pulas...@wso2.com
 wrote:

 Hi Dimuthu,



 On Tue, Nov 11, 2014 at 4:17 PM, Dimuthu Leelarathne dimut...@wso2.com
 wrote:

 Hi,



 On Tue, Nov 11, 2014 at 3:43 PM, Pulasthi Supun pulas...@wso2.com
 wrote:

 Hi Dimuthu,

 On Tue, Nov 11, 2014 at 2:05 PM, Dimuthu Leelarathne 
 dimut...@wso2.com wrote:

 Hi Pulasthi,

 Yes. We do not need  global invalidation (although it would solve the
 problem), but the request is to sink AF Registry cache with the DB. We 
 are
 in the same JVM, and we need a method/way to tell registry remove this
 particular path from Registry path cache.


 Such a method would need to be accessed through something like the
 remote registry right?. The Registry api does not provide such a method to
 remove entries from the Registry cache.


 It should not be remote. An OSGi level method would be fine. Is there a
 way to patch the registry that we use?


 I talked with Azeez regarding this. He also agrees that providing such a
 method to manipulate the cache is wrong. We need to think of some other
 solution for this. I am not sure if the global cache invalidation has
 completed or can be backported into 4.2.0.
 @Amal is the work on that complete ?

 Regards,
 Pulasthi


 thanks,
 dimuthu



 Regards,
 Pulasthi


 thanks,
 dimuthu


 On Tue, Nov 11, 2014 at 12:16 PM, Pulasthi Supun pulas...@wso2.com
 wrote:

 Hi All,


 From what i understand the AF and SM are in different domains that
 is why distributed caching will not be able to handle this scenario 
 right?.
 global cluster cache invalidation has been done with the use of a pub 
 sub
 mechanism ( discussed in archi under Global cluster cache invalidation
 code review Notes ) but this will only be available in the next release
 AFAIK.

 Regards,
 Pulasthi

 On Tue, Nov 11, 2014 at 10:00 AM, Amalka Subasinghe ama...@wso2.com
  wrote:


 Hi,

 The scenario is, we have mounted SM's registry to the App Factory
 registry to remove the remote call for read the resources. but still 
 the
 write calls happens via SM. see the image below.


 ​

 The problem is, once we do a write call to the SM's registry, App
 Factory registry cache won't get updated, so old information will be 
 shown
 in App Factory pages.
 Is there a way to remove an entry from the App Factory registry
 cache when we do a write call to the SM's registry? then when we do a 
 read
 call, it will load the cache again and show the updated information.

 Thanks
 Amalka


 ___
 Architecture mailing list
 Architecture@wso2.org
 https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




 --
 --
 Pulasthi Supun
 Software Engineer; WSO2 Inc.; http://wso2.com,
 Email: pulas...@wso2.com
 Mobile: +94 (71) 9258281
 Blog : http://pulasthisupun.blogspot.com/
 Git hub profile: https://github.com/pulasthi




 --
 Dimuthu Leelarathne
 Architect  Product Lead of App Factory

 WSO2, Inc. (http://wso2.com)
 email: dimut...@wso2.com
 Mobile : 0773661935

 Lean . Enterprise . Middleware




 --
 --
 Pulasthi Supun
 Software Engineer; WSO2 Inc.; http://wso2.com,
 Email: pulas...@wso2.com
 Mobile: +94 (71) 9258281
 Blog : 

Re: [Architecture] How to forcefully remove an entry from app factory registry path cache?

2014-12-01 Thread Pirinthapan Mahendran
Hi Manjula,

As my understanding cacheKey is a RegistryCacheKey object, which is
different from the cacheId. So we don't need to read the cacheId.

Thanks.



Mahendran Pirinthapan
Software Engineer | WSO2 Inc.
Mobile +94772378732.

On Tue, Dec 2, 2014 at 12:01 PM, Manjula Rathnayake manju...@wso2.com
wrote:

 Hi Shazni,

 I checked the code for removeCache method and found that cacheKey is
 calculated as below.

 *String connectionId = (dataBaseConfiguration.getUserName() != null?
 dataBaseConfiguration.getUserName().split(@)[0]:dataBaseConfiguration.getUserName())
 + @ + dataBaseConfiguration.getDbUrl(); cacheKey =
 RegistryUtils.buildRegistryCacheKey(connectionId, tenantId, path);*

 But in registry.xml we have below element too,
 *cacheIdroot@jdbc:mysql://mysql1.appfactory.private.wso2.com:3306/dbGovernanceCloud
 http://mysql1.appfactory.private.wso2.com:3306/dbGovernanceCloud/cacheId*

 Shouldn't we read the cacheId element first and calculate as above if
 cacheId element is not defined?  Or did I misunderstand the configuration?

 thank you.

 On Wed, Nov 12, 2014 at 2:23 PM, Shazni Nazeer sha...@wso2.com wrote:

 Hi,

 Given that we know the registry path of the resource of which cache to be
 deleted and have an instance of the registry we can manually delete the
 cache with a method like removeCache in the below file. However, it's not a
 clean or correct way of manipulating the registry cache.


 https://github.com/wso2-dev/carbon-governance/blob/master/components/governance/org.wso2.carbon.governance.custom.lifecycles.checklist/src/main/java/org/wso2/carbon/governance/custom/lifecycles/checklist/util/LifecycleBeanPopulator.java

 Shazni Nazeer

 Senior Software Engineer

 Mob : +94 37331
 LinkedIn : http://lk.linkedin.com/in/shazninazeer
 Blog : http://shazninazeer.blogspot.com

 On Wed, Nov 12, 2014 at 1:48 PM, Dimuthu Leelarathne dimut...@wso2.com
 wrote:

 Hi Pulasthi,

 So when we are doing global invalidation, what is the method we are
 going to use to invalidate the cache within the JVM? :) Or are you going to
 do it by magic?

 thanks,
 dimuthu

 On Wed, Nov 12, 2014 at 1:36 PM, Pulasthi Supun pulas...@wso2.com
 wrote:

 Hi Dimuthu,



 On Tue, Nov 11, 2014 at 4:17 PM, Dimuthu Leelarathne dimut...@wso2.com
  wrote:

 Hi,



 On Tue, Nov 11, 2014 at 3:43 PM, Pulasthi Supun pulas...@wso2.com
 wrote:

 Hi Dimuthu,

 On Tue, Nov 11, 2014 at 2:05 PM, Dimuthu Leelarathne 
 dimut...@wso2.com wrote:

 Hi Pulasthi,

 Yes. We do not need  global invalidation (although it would solve
 the problem), but the request is to sink AF Registry cache with the DB. 
 We
 are in the same JVM, and we need a method/way to tell registry remove 
 this
 particular path from Registry path cache.


 Such a method would need to be accessed through something like the
 remote registry right?. The Registry api does not provide such a method 
 to
 remove entries from the Registry cache.


 It should not be remote. An OSGi level method would be fine. Is there
 a way to patch the registry that we use?


 I talked with Azeez regarding this. He also agrees that providing such
 a method to manipulate the cache is wrong. We need to think of some other
 solution for this. I am not sure if the global cache invalidation has
 completed or can be backported into 4.2.0.
 @Amal is the work on that complete ?

 Regards,
 Pulasthi


 thanks,
 dimuthu



 Regards,
 Pulasthi


 thanks,
 dimuthu


 On Tue, Nov 11, 2014 at 12:16 PM, Pulasthi Supun pulas...@wso2.com
 wrote:

 Hi All,


 From what i understand the AF and SM are in different domains that
 is why distributed caching will not be able to handle this scenario 
 right?.
 global cluster cache invalidation has been done with the use of a pub 
 sub
 mechanism ( discussed in archi under Global cluster cache invalidation
 code review Notes ) but this will only be available in the next 
 release
 AFAIK.

 Regards,
 Pulasthi

 On Tue, Nov 11, 2014 at 10:00 AM, Amalka Subasinghe 
 ama...@wso2.com wrote:


 Hi,

 The scenario is, we have mounted SM's registry to the App Factory
 registry to remove the remote call for read the resources. but still 
 the
 write calls happens via SM. see the image below.


 ​

 The problem is, once we do a write call to the SM's registry, App
 Factory registry cache won't get updated, so old information will be 
 shown
 in App Factory pages.
 Is there a way to remove an entry from the App Factory registry
 cache when we do a write call to the SM's registry? then when we do a 
 read
 call, it will load the cache again and show the updated information.

 Thanks
 Amalka


 ___
 Architecture mailing list
 Architecture@wso2.org
 https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




 --
 --
 Pulasthi Supun
 Software Engineer; WSO2 Inc.; http://wso2.com,
 Email: pulas...@wso2.com
 Mobile: +94 (71) 9258281
 Blog : http://pulasthisupun.blogspot.com/
 Git hub profile: https://github.com/pulasthi




 --
 Dimuthu 

Re: [Architecture] Moving deployment section from configuration xml and Facilitate AppFactory users to deploy apps with their custom app types if its not already available

2014-12-01 Thread Dimuthu Leelarathne
Hi Danushka,

Please see my comments below.

On Tue, Dec 2, 2014 at 12:01 PM, Danushka Fernando danush...@wso2.com
wrote:

 HI Dimuthu
 Please find my comments inline


 On Tue, Dec 2, 2014 at 8:45 AM, Dimuthu Leelarathne dimut...@wso2.com
 wrote:

 Hi Rajeevan,

 Please see my comments below.

 On Mon, Dec 1, 2014 at 10:46 PM, Aiyadurai Rajeevan rajeev...@wso2.com
 wrote:

 Hi All,

 We are trying to implement a feature on AF which enables the user to
 deploy their customized app types, Currently this configuration is
 available in *appfactory.xml *under *Deployer* tag the content would
 be as [1], likewise we have for each app types. Hence this wouldn't be
 editable by the users and may not deploy their own app types. If we move
 out this [1] from *appfactory.xml* and put this in a configurable file
 would enable the users to customize their need.

 When we analyzing the [1] we found out , The below content related to
 Pass artifact storage configuration and common to all app types.

 RepositoryProvider

Property name=Class


 org.wso2.carbon.appfactory.s4.integration.GITBlitBasedGITRepositoryProvider

 /Property

Property name=BaseURL
 https://gitblit.s2.wso2.com:8444//Property

 Property name=URLPattern{@stage}/as
 /Property

Property name=AdminUserNameadmin/Property

  Property name=AdminPasswordadmin/Property

 /RepositoryProvider


 What is the other xml file are you talking about? Lets call it foo.xml.
 The only other place that require these information other than App Factory
 is Jenkins. So rather than putting these in separate xml file, I will put
 it in the appfactory-plugin.xml file. This is because Jenkins provide and
 inherent way to read appfactory-plugin.xml file and it doesn't give an
 inherent way to read foo.xml file.

 So at this stage the foo.xml file is only used in App Factory. You may
 merge it with appfactory.xml. The next question you may ask configuration
 duplication. I say puppet is the answer.

 If we have code to read foo.xml we have to put the code in both Jenkins
 side and AF side. I would rather use Jenkins inherent way of config files
 as oppose to duplicating code in both places and having a special file.


 I was thinking about moving outside the tag but keep it inside the same
 appfactory.xml. We can pass these to jenkins through property bag when
 deployment.



 So we are planning to move this to a common configuration file and all
 the application types can access that.

 In [1] below properties are not used. Hence, We shall get rid of that
 from AF

 Property name=minInstances1/Property

 Property name=maxInstances1/Property

 Property name=shouldActivate/Property
 Property name=subscribeOnDeploymentfalse
 /Property

 And the below properties are used. So, We can keep them in the
 *apptype.xml* as we already have separate apptype.xml for each app
 types..

 Property name=alias162dev/Property

 Property name=cartridgeType162dev/Property

Property name=deploymentPolicyaf-deployment
 /Property

 Property name=autoscalePolicyeconomy
 /Property

 Property name=repoURL/Property

Property name=dataCartridgeType/Property

Property name=dataCartridgeAlias/Property





 What is the propose of having the below in apptype.xml itself? What if we
 move this to a different runtime.xml file? I am proposing because Runtime
 is a different concept that is required by App Factory.Let me prove it by
 example. If we use a different xml file for deployment(runtime), in future
 we can implement multiple deployment environments very easily. For example
 consider the below relationship.


 What I thought the cartidgeType only being used. But seems all of them
 used just for cartridge subscription. So we need a separate place to these
 indeed. And IMO we should add support to add the cartridge configuration
 file to the apptype as well.


 AppType (1) - has --  (n)Deployment Envs

 Question about this. Is it one to many or many to many. If it is many to
 many its going to be complex I guess. In that case we cannot deploy the
 deploy / runtime configs with apptype it self. And there should be a way to
 map which runtimes are available to which apptype. If it is one to many
 there will be same deployer mentioned in several app types.

 Should we focus on that feature here. If it is to implement many to many
 relationship then we are doing that feature now.


You are correct in saying we need to concentrate on this current feature
only. But since we are going to be doing refactoring anyway I propose to
separate out Runtime concept gradually. There is no harm 

Re: [Architecture] How to forcefully remove an entry from app factory registry path cache?

2014-12-01 Thread Shazni Nazeer
Hi Manjula,

Yes. The cacheId that you specify is not the the '*connectionId*' that we
create in the method. When a resource is added to the cache we take the '
*connectionId*' the way it is implemented to create the cache key.
Therefore while retrieving the cache we should use the same way.

Shazni Nazeer

Senior Software Engineer

Mob : +94 37331
LinkedIn : http://lk.linkedin.com/in/shazninazeer
Blog : http://shazninazeer.blogspot.com

On Tue, Dec 2, 2014 at 12:20 PM, Pirinthapan Mahendran pirintha...@wso2.com
 wrote:

 Hi Manjula,

 As my understanding cacheKey is a RegistryCacheKey object, which is
 different from the cacheId. So we don't need to read the cacheId.

 Thanks.



 Mahendran Pirinthapan
 Software Engineer | WSO2 Inc.
 Mobile +94772378732.

 On Tue, Dec 2, 2014 at 12:01 PM, Manjula Rathnayake manju...@wso2.com
 wrote:

 Hi Shazni,

 I checked the code for removeCache method and found that cacheKey is
 calculated as below.

 *String connectionId = (dataBaseConfiguration.getUserName() != null?
 dataBaseConfiguration.getUserName().split(@)[0]:dataBaseConfiguration.getUserName())
 + @ + dataBaseConfiguration.getDbUrl(); cacheKey =
 RegistryUtils.buildRegistryCacheKey(connectionId, tenantId, path);*

 But in registry.xml we have below element too,
 *cacheIdroot@jdbc:mysql://mysql1.appfactory.private.wso2.com:3306/dbGovernanceCloud
 http://mysql1.appfactory.private.wso2.com:3306/dbGovernanceCloud/cacheId*

 Shouldn't we read the cacheId element first and calculate as above if
 cacheId element is not defined?  Or did I misunderstand the configuration?

 thank you.

 On Wed, Nov 12, 2014 at 2:23 PM, Shazni Nazeer sha...@wso2.com wrote:

 Hi,

 Given that we know the registry path of the resource of which cache to
 be deleted and have an instance of the registry we can manually delete the
 cache with a method like removeCache in the below file. However, it's not a
 clean or correct way of manipulating the registry cache.


 https://github.com/wso2-dev/carbon-governance/blob/master/components/governance/org.wso2.carbon.governance.custom.lifecycles.checklist/src/main/java/org/wso2/carbon/governance/custom/lifecycles/checklist/util/LifecycleBeanPopulator.java

 Shazni Nazeer

 Senior Software Engineer

 Mob : +94 37331
 LinkedIn : http://lk.linkedin.com/in/shazninazeer
 Blog : http://shazninazeer.blogspot.com

 On Wed, Nov 12, 2014 at 1:48 PM, Dimuthu Leelarathne dimut...@wso2.com
 wrote:

 Hi Pulasthi,

 So when we are doing global invalidation, what is the method we are
 going to use to invalidate the cache within the JVM? :) Or are you going to
 do it by magic?

 thanks,
 dimuthu

 On Wed, Nov 12, 2014 at 1:36 PM, Pulasthi Supun pulas...@wso2.com
 wrote:

 Hi Dimuthu,



 On Tue, Nov 11, 2014 at 4:17 PM, Dimuthu Leelarathne 
 dimut...@wso2.com wrote:

 Hi,



 On Tue, Nov 11, 2014 at 3:43 PM, Pulasthi Supun pulas...@wso2.com
 wrote:

 Hi Dimuthu,

 On Tue, Nov 11, 2014 at 2:05 PM, Dimuthu Leelarathne 
 dimut...@wso2.com wrote:

 Hi Pulasthi,

 Yes. We do not need  global invalidation (although it would solve
 the problem), but the request is to sink AF Registry cache with the 
 DB. We
 are in the same JVM, and we need a method/way to tell registry remove 
 this
 particular path from Registry path cache.


 Such a method would need to be accessed through something like the
 remote registry right?. The Registry api does not provide such a method 
 to
 remove entries from the Registry cache.


 It should not be remote. An OSGi level method would be fine. Is there
 a way to patch the registry that we use?


 I talked with Azeez regarding this. He also agrees that providing such
 a method to manipulate the cache is wrong. We need to think of some other
 solution for this. I am not sure if the global cache invalidation has
 completed or can be backported into 4.2.0.
 @Amal is the work on that complete ?

 Regards,
 Pulasthi


 thanks,
 dimuthu



 Regards,
 Pulasthi


 thanks,
 dimuthu


 On Tue, Nov 11, 2014 at 12:16 PM, Pulasthi Supun pulas...@wso2.com
  wrote:

 Hi All,


 From what i understand the AF and SM are in different domains that
 is why distributed caching will not be able to handle this scenario 
 right?.
 global cluster cache invalidation has been done with the use of a pub 
 sub
 mechanism ( discussed in archi under Global cluster cache 
 invalidation
 code review Notes ) but this will only be available in the next 
 release
 AFAIK.

 Regards,
 Pulasthi

 On Tue, Nov 11, 2014 at 10:00 AM, Amalka Subasinghe 
 ama...@wso2.com wrote:


 Hi,

 The scenario is, we have mounted SM's registry to the App Factory
 registry to remove the remote call for read the resources. but still 
 the
 write calls happens via SM. see the image below.


 ​

 The problem is, once we do a write call to the SM's registry, App
 Factory registry cache won't get updated, so old information will be 
 shown
 in App Factory pages.
 Is there a way to remove an entry from the App Factory registry
 cache when we do