Re: [Architecture] [WSO2 Docker Images] Creating Base Docker Images for WSO2 Products

2016-03-23 Thread Imesh Gunaratne
On Thu, Mar 24, 2016 at 11:05 AM, Chamila De Alwis wrote: > >> The "base" images can be started and run in the standalone mode, although > not production readily. Usually base images are used to collect common > configurations, not runnable products. However it's naming issue, so I > guess we can

Re: [Architecture] [IoTS] Is it the right thing to package "analytics" scripts with a plugin implementation?

2016-03-23 Thread Prabath Abeysekera
Hi Suho, On Thu, Mar 24, 2016 at 12:21 AM, Sriskandarajah Suhothayan wrote: > We already have Dev Studio support for "analytics" artifacts. > > In this case it depends, If the plugin developer believes that he his > sensor need some specialised analytics and charts then we should have the > capa

Re: [Architecture] [Dev] [REST APIs][Analytics] GET request with a payload

2016-03-23 Thread Ayoma Wijethunga
Agreed. Then again, if user needs to include more columns than he needs to exclude we can introduce something like below and use the most suitable depending on the length. service?*include*=column1,column2*- only include columns specified by user* service?*exclude*=column1,column2 *- include

Re: [Architecture] [IoTS] Is it the right thing to package "analytics" scripts with a plugin implementation?

2016-03-23 Thread Prabath Abeysekera
Hi Ruwan, Analytics is indeed a primary requirement in any IoT "eco-system". There's no doubt whatsoever about it. In fact, analytics is what inspires folks to achieve operational efficiency and all that stuff end-to-end with proper use of monitoring against organization-specific KPIs, as you also

Re: [Architecture] Session Affinity Alternatives

2016-03-23 Thread Srinath Perera
Hi Frank, Sorry, maybe we have not thought this through. When I said "state transfer" I mean "full session state in the message (REST-style)" Q1: What can we do at the platform level to support "state transfer"? Q1. Case 1: for stateful Apps and Services written by end users - end user can choo

Re: [Architecture] [Dev] [REST APIs][Analytics] GET request with a payload

2016-03-23 Thread Udara Liyanage
Hi, IMO by using POST to do a GET we are violating REST principals and may confuse users. You can see a query type API in [1] used by StackOverflow API. According to [2], it seems about 2000 characters are allowed most browsers which is a pretty large number where a normal GET request is highly

Re: [Architecture] DAS Lucene query AllowLeadingWildcard

2016-03-23 Thread Damith Wickramasinghe
Hi Gimantha, Great. One question. Does it can be any special character. ? Regards, Damith. On Thu, Mar 24, 2016 at 11:49 AM, Gimantha Bandara wrote: > Hi Damith, > > If the "roles" field contains comma separated values, you can simply > search for the specific role using the following query >

Re: [Architecture] DAS Lucene query AllowLeadingWildcard

2016-03-23 Thread Gimantha Bandara
Hi Damith, If the "roles" field contains comma separated values, you can simply search for the specific role using the following query "roles : role1" Lucene has an analysis process which takes place before indexing. So the field values will be tokenized into terms(Text fields are split removin

Re: [Architecture] Incremental Processing Support in DAS

2016-03-23 Thread Srinath Perera
Hi Sachith, Anjana, +1 for the backend model. Are we handling the case when last run was done, 25 minutes of data is processed. Basically, next run has to re-run last hour and update the value. When does one hour counting starts? is it from the moment server starts? That will be probabilistic wh

Re: [Architecture] [Dev] [REST APIs][Analytics] GET request with a payload

2016-03-23 Thread Gimantha Bandara
Thank you for your suggestions. We cannot exactly say that the number of columns/fields user will need. It depends on how many fields a table has and how many the user want to get from that table. So the url length might exceed. If so, we will have to go with a new method with POST as it seems the

[Architecture] DAS Lucene query AllowLeadingWildcard

2016-03-23 Thread Damith Wickramasinghe
Hi all, I have a column which contains roles as a comma separated string. eg:- role1,role2,role3 I need to find records which matches to specific role. As I checked theres no String contains function. But there is wildcard support[1]. To be able to work for my usecase wildcard should be of type *

Re: [Architecture] [IoTS] Is it the right thing to package "analytics" scripts with a plugin implementation?

2016-03-23 Thread Sumedha Rubasinghe
My take on this is slightly different. First @ a conceptual level when we say IoT Server can be extended by a plugin, we should have a clear definition of that plugin to have X, Y, Z capabilities. So I feel having analytics @ device type definition level (conceptually) is not a problem. Going by

Re: [Architecture] [WSO2 Docker Images] Creating Base Docker Images for WSO2 Products

2016-03-23 Thread Chamila De Alwis
Hi Imesh, Please find my comments inline. On Thu, Mar 24, 2016 at 10:11 AM, Imesh Gunaratne wrote: > May be you did not get the reason for doing this change properly. We are > not dropping support for Puppet or any other orchestration management > system for automating the configuration. > +1,

Re: [Architecture] [WSO2 Docker Images] Creating Base Docker Images for WSO2 Products

2016-03-23 Thread Imesh Gunaratne
On Thu, Mar 24, 2016 at 9:36 AM, Chamila De Alwis wrote: > > The suggested design change covers the first approach, however if we are > dropping the Puppet support (I'm seeing Puppet based configuration of the > image only in the profile specific images), we are not going to cover the > second ap

Re: [Architecture] Incremental Processing Support in DAS

2016-03-23 Thread Sachith Withana
Hi Sinthuja, Please find the comments inline. On Wed, Mar 23, 2016 at 6:50 PM, Sinthuja Ragendran wrote: > Hi Sachith, > > On Mon, Mar 21, 2016 at 5:15 PM, Sachith Withana wrote: > >> Hi all, >> >> We are adding incremental processing capability to Spark in DAS. >> As the first stage, we adde

Re: [Architecture] RFC:Security Challenges in Analytics Story

2016-03-23 Thread Srinath Perera
This thread described the authorization issue when reading data for gadgets ( as I mentioned in Dashboard server product council). When IoT server/ API manager publish events, it need to tell DAS whose data it is. ( however, server cannot login using that user, as then it will need to keep passwor

Re: [Architecture] [WSO2 Docker Images] Creating Base Docker Images for WSO2 Products

2016-03-23 Thread Chamila De Alwis
The Dockerfiles for the vanilla/base Docker images are shipped mainly because of the restriction on redistribution of the JDK. Other than that, the main function of the wso2/dockerfiles repository would be to provide tooling to ease customization and extension of the Dockerfiles. When it comes to

Re: [Architecture] [IoTS] Is it the right thing to package "analytics" scripts with a plugin implementation?

2016-03-23 Thread Sriskandarajah Suhothayan
We already have Dev Studio support for "analytics" artifacts. In this case it depends, If the plugin developer believes that he his sensor need some specialised analytics and charts then we should have the capability to provide that, this can be achieved via added the analytics in to the same cApp

Re: [Architecture] [EMM] Android OMA-DM Standardization

2016-03-23 Thread Ruwan Yatawara
+1 Thanks and Regards, Ruwan Yatawara Senior Software Engineer, WSO2 Inc. email : ruw...@wso2.com mobile : +94 77 9110413 blog : http://ruwansrants.blogspot.com/ www: :http://wso2.com On Tue, Mar 22, 2016 at 11:20 PM, Dileesha Rajapakse wrote: > Hi Ruwan, > > Yes, I strongly agree with you

Re: [Architecture] [IoTS] Is it the right thing to package "analytics" scripts with a plugin implementation?

2016-03-23 Thread Ruwan Yatawara
Hi Prabath, IoTS is basically the representation of WSO2's IoT Platform vision in the flesh (Well, not literally). The idea behind it, is to provide users with the necessary tools and components needed to get started with writing device types that will later go in actual deployments. So someone wh

Re: [Architecture] [Dev] [REST APIs][Analytics] GET request with a payload

2016-03-23 Thread Lahiru Sandaruwan
Yes, if it is possible to put the columns names in the url as Ayoma mentioned, we must use that(First i thought it is a complex payload you want to send). Unless there are limitations, like column list doesn't exceed the url length limits, we should use GET. Thanks. On Wed, Mar 23, 2016 at 3:54

Re: [Architecture] [Dev] [REST APIs][Analytics] GET request with a payload

2016-03-23 Thread Ayoma Wijethunga
Hi, It is true that using GET request with a payload is not the best option. Even though it is not strictly prohibited in specs, it can be confusing [1]. REST architecture is very open about how we use HTTP methods, but thinking in terms of REST architecture, I do not think using POST is also the

Re: [Architecture] [REST APIs][Analytics] GET request with a payload

2016-03-23 Thread Lahiru Sandaruwan
Hi, I think using a POST with a body, for retrieving information is fine considering the requirement. GET with body is not recommended. Thanks. On Wed, Mar 23, 2016 at 2:31 PM, Gimantha Bandara wrote: > Hi all, > > > We have a REST API in DAS to retrieve records in a specific table. It > suppo

[Architecture] [REST APIs][Analytics] GET request with a payload

2016-03-23 Thread Gimantha Bandara
Hi all, We have a REST API in DAS to retrieve records in a specific table. It supports GET method with the following url format. /analytics/tables/{tableName}/{from}/{to}/{start}/{count} Sending a GET request to above url will give the records between given "from", "to" time range starting from

Re: [Architecture] Incremental Processing Support in DAS

2016-03-23 Thread Sinthuja Ragendran
Hi Sachith, On Mon, Mar 21, 2016 at 5:15 PM, Sachith Withana wrote: > Hi all, > > We are adding incremental processing capability to Spark in DAS. > As the first stage, we added time slicing to Spark execution. > > Here's a quick introduction into that. > > *Execution*: > > In the first run of t

[Architecture] [IoTS] Is it the right thing to package "analytics" scripts with a plugin implementation?

2016-03-23 Thread Prabath Abeysekera
Hi Everyone, As most of you know, IoTS is a unified platform that allows users to come up with "plugins" and seamlessly add them to the framework to be able to manage different types of devices. For instance, if an organization named "Org-X" manufactures a device type "Dev-X", they can simply adop

Re: [Architecture] Session Affinity Alternatives

2016-03-23 Thread Frank Leymann
OK... But B.(ii) has disadvantages (performance, garbage collection,...). We need to argue that the benefit of platform/server supported state transfer outweighs the disadvantages. Best regards, Frank 2016-03-23 7:04 GMT+01:00 Srinath Perera : > Hi Frank > > Agreed, > > IMO, we should go for s

Re: [Architecture] [C5] Artifact deployment coordination support for C5

2016-03-23 Thread Pamod Sylvester
Would making the deployment coordination pluggable be an option (ability to use both options) ? On Wed, Mar 23, 2016 at 2:02 PM, Srinath Perera wrote: > Kasun, I would go with JMS as it gives event persistence and you will not > loose the events even if listener is restarted. > > --Srinath. > >

Re: [Architecture] [C5] Artifact deployment coordination support for C5

2016-03-23 Thread Sameera Jayasoma
I believe we need both here. In JMS case, we need to publish events to a topic rather than a queue. Thanks, Sameera. On Wed, Mar 23, 2016 at 2:02 PM, Srinath Perera wrote: > Kasun, I would go with JMS as it gives event persistence and you will not > loose the events even if listener is restart

Re: [Architecture] [C5] Artifact deployment coordination support for C5

2016-03-23 Thread Srinath Perera
Kasun, I would go with JMS as it gives event persistence and you will not loose the events even if listener is restarted. --Srinath. On Wed, Mar 23, 2016 at 10:45 AM, KasunG Gajasinghe wrote: > Hi, > > Given several issues we discovered with automatic artifact synchronization > with DepSync in

Re: [Architecture] [C5] Lifecycle listeners to listen on Artifact deployment events

2016-03-23 Thread KasunG Gajasinghe
On Wed, Mar 23, 2016 at 10:12 AM, KasunG Gajasinghe wrote: > > > On Wednesday, March 23, 2016, Aruna Karunarathna wrote: > >> >> >> On Tue, Mar 22, 2016 at 9:44 PM, KasunG Gajasinghe >> wrote: >> >>> Hi, >>> >>> To provide extensibility to the C5 deployment engine, we need to add the >>> lifecy

[Architecture] [WSO2 Docker Images] Creating Base Docker Images for WSO2 Products

2016-03-23 Thread Imesh Gunaratne
​Hi All, Recently after a discussion we had with Lakmal and the PaaS team, we reduced the size of the WSO2 Docker images by combining multiple RUN instructions, removing the puppet modules COPY command and by removing the wso2/base docker image. Afterwards we thought of redesigning the way we crea

Re: [Architecture] [Analytics for APIM] Integration Tests to cover Alerting scripts

2016-03-23 Thread Sachith Withana
Hi all, The structure is as follows, 1. You need to add a publisher ( logger) to the respective alert stream that you are testing 2. Delete the data in the respective data tables ( optional) 3. Do the publishing of simulation data ( either using a csv or programmatically. 4. Run the spark scripts