Re: [Dev] [DEV][ESB500] Invoking a secure proxy in a clustered set up

[Pubudu Priyashan]

It makes sense when you think in terms of dev cycles. Thanks for the
clarification guys.

Cheers,
Pubudu.

Pubudu D.P
Senior Software Engineer - QA Team | WSO2 inc.
Mobile : +94775464547

Linkedin: https://uk.linkedin.com/in/pubududp
Medium: https://medium.com/@pubududp


On Mon, Aug 8, 2016 at 11:19 AM, Rajith Vitharana  wrote:

> Hi Pubudu,
>
> On 8 August 2016 at 11:14, Pubudu Priyashan  wrote:
>
>> Hi Jagath,
>>
>> So when we do that, it requires the user to restart the server every time
>> they add a new https endpoint to add the keys to client-truststore. In
>> production environments, this is not
>>
> I'm not sure people do such things directly to the production envs, I
> think there are dev cycles before pushing such changes to production. Hence
> will have the chance to add required certs to the product before production.
>
> Thanks,
>
>> convenient and will have to go through a long process to add a key to the
>> truststore since it requires a server restart. Are there any better
>> approaches we can think of to overcome this without having to restart the
>> servers when we add new https endpoints?
>>
>> Cheers,
>>
>> Pubudu D.P
>> Senior Software Engineer - QA Team | WSO2 inc.
>> Mobile : +94775464547
>>
>> Linkedin: https://uk.linkedin.com/in/pubududp
>> Medium: https://medium.com/@pubududp
>>
>>
>> On Mon, Aug 8, 2016 at 10:55 AM, Jagath Sisirakumara Ariyarathne <
>> jaga...@wso2.com> wrote:
>>
>>> Hi Pubudu,
>>>
>>> Whenever you access a Https endpoint, you have to import back-end's
>>> certificate to client-truststore. That is the default requirement if you
>>> are using SSL between back-end and ESB.
>>>
>>> Thanks.
>>>
>>> On Mon, Aug 8, 2016 at 10:13 AM, Pubudu Priyashan 
>>> wrote:
>>>
 Hi ESB team,

 I have observed the following behaviour when invoking a secure proxy in
 a clustered setup which calls an external http endpoint [1] with ESB 5.0.0
 (Tested with RC1). Can you please inform if this behaviour is correct or
 are there any alternative methods?

 I have a secure proxy in my ESB cluster where it calls an external
 https endpoint [1]. When I invoked this in a single node in my local
 machine, it returned the response as expected. When I created the same
 secure proxy in a Cluster, I received this error [2]. I was able to get rid
 of this error by adding the certificate of the endpoint
 to client-truststore.jks file of each ESB node of the cluster.

 My question is, is this behaviour correct? Should we have to always add
 the https endpoint certificate every time we need to use an external
 endpoint in a secure proxy? Or is there another approach where we don't
 have to do this and not required to add the keys to client-truststore.jks
 and restart ESB nodes? This might not be convenient if we have a lot of
 https endpoints being called via a secure proxy in ESB. Can you please
 advice?

 Thanks!

 [1] https://demo5224632.mockable.io/test123

 [2] Exception in ESB worker node:-

 TID: [-1] [] [2016-08-04 05:34:33,682] DEBUG
 {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
 dispatcher-1 >> "GET /services/SecureP1 HTTP/1.1[\r][\n]"
 {org.apache.synapse.transport.http.wire}
 TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG
 {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
 dispatcher-1 >> "X-Forwarded-Host: wrk.esb500.wso2.com[\r][\n]"
 {org.apache.synapse.transport.http.wire}
 TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG
 {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
 dispatcher-1 >> "X-Forwarded-Server: wrk.esb500.wso2.com[\r][\n]"
 {org.apache.synapse.transport.http.wire}
 TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG
 {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
 dispatcher-1 >> "X-Forwarded-For: 10.100.7.95[\r][\n]"
 {org.apache.synapse.transport.http.wire}
 TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG
 {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
 dispatcher-1 >> "Connection: upgrade[\r][\n]" 
 {org.apache.synapse.transport.
 http.wire}
 TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG
 {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
 dispatcher-1 >> "Host: ssl.esb500.com[\r][\n]"
 {org.apache.synapse.transport.http.wire}
 TID: [-1] [] [2016-08-04 05:34:33,684] DEBUG
 {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
 dispatcher-1 >> "Authorization: Basic 
 YWRtaW5lc2I1MDA6ZXNiNTAwQERhcw==[\r][\n]"
 {org.apache.synapse.transport.http.wire}
 TID: [-1] [] [2016-08-04 05:34:33,684] DEBUG
 {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
 dispatcher-1 >> "Cache-Control: no-cache[\r][\n]"
 {org.apache.synapse.transport.http.wire}
 TID: [-1] [] [2016-08-04 05:34:33,684] 

Re: [Dev] [DEV][ESB500] Invoking a secure proxy in a clustered set up

[Rajith Vitharana]

Hi Pubudu,

On 8 August 2016 at 11:14, Pubudu Priyashan  wrote:

> Hi Jagath,
>
> So when we do that, it requires the user to restart the server every time
> they add a new https endpoint to add the keys to client-truststore. In
> production environments, this is not
>
I'm not sure people do such things directly to the production envs, I think
there are dev cycles before pushing such changes to production. Hence will
have the chance to add required certs to the product before production.

Thanks,

> convenient and will have to go through a long process to add a key to the
> truststore since it requires a server restart. Are there any better
> approaches we can think of to overcome this without having to restart the
> servers when we add new https endpoints?
>
> Cheers,
>
> Pubudu D.P
> Senior Software Engineer - QA Team | WSO2 inc.
> Mobile : +94775464547
>
> Linkedin: https://uk.linkedin.com/in/pubududp
> Medium: https://medium.com/@pubududp
>
>
> On Mon, Aug 8, 2016 at 10:55 AM, Jagath Sisirakumara Ariyarathne <
> jaga...@wso2.com> wrote:
>
>> Hi Pubudu,
>>
>> Whenever you access a Https endpoint, you have to import back-end's
>> certificate to client-truststore. That is the default requirement if you
>> are using SSL between back-end and ESB.
>>
>> Thanks.
>>
>> On Mon, Aug 8, 2016 at 10:13 AM, Pubudu Priyashan 
>> wrote:
>>
>>> Hi ESB team,
>>>
>>> I have observed the following behaviour when invoking a secure proxy in
>>> a clustered setup which calls an external http endpoint [1] with ESB 5.0.0
>>> (Tested with RC1). Can you please inform if this behaviour is correct or
>>> are there any alternative methods?
>>>
>>> I have a secure proxy in my ESB cluster where it calls an external https
>>> endpoint [1]. When I invoked this in a single node in my local machine, it
>>> returned the response as expected. When I created the same secure proxy in
>>> a Cluster, I received this error [2]. I was able to get rid of this error
>>> by adding the certificate of the endpoint to client-truststore.jks file of
>>> each ESB node of the cluster.
>>>
>>> My question is, is this behaviour correct? Should we have to always add
>>> the https endpoint certificate every time we need to use an external
>>> endpoint in a secure proxy? Or is there another approach where we don't
>>> have to do this and not required to add the keys to client-truststore.jks
>>> and restart ESB nodes? This might not be convenient if we have a lot of
>>> https endpoints being called via a secure proxy in ESB. Can you please
>>> advice?
>>>
>>> Thanks!
>>>
>>> [1] https://demo5224632.mockable.io/test123
>>>
>>> [2] Exception in ESB worker node:-
>>>
>>> TID: [-1] [] [2016-08-04 05:34:33,682] DEBUG
>>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>>> dispatcher-1 >> "GET /services/SecureP1 HTTP/1.1[\r][\n]"
>>> {org.apache.synapse.transport.http.wire}
>>> TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG
>>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>>> dispatcher-1 >> "X-Forwarded-Host: wrk.esb500.wso2.com[\r][\n]"
>>> {org.apache.synapse.transport.http.wire}
>>> TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG
>>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>>> dispatcher-1 >> "X-Forwarded-Server: wrk.esb500.wso2.com[\r][\n]"
>>> {org.apache.synapse.transport.http.wire}
>>> TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG
>>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>>> dispatcher-1 >> "X-Forwarded-For: 10.100.7.95[\r][\n]"
>>> {org.apache.synapse.transport.http.wire}
>>> TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG
>>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>>> dispatcher-1 >> "Connection: upgrade[\r][\n]" {org.apache.synapse.transport.
>>> http.wire}
>>> TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG
>>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>>> dispatcher-1 >> "Host: ssl.esb500.com[\r][\n]"
>>> {org.apache.synapse.transport.http.wire}
>>> TID: [-1] [] [2016-08-04 05:34:33,684] DEBUG
>>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>>> dispatcher-1 >> "Authorization: Basic 
>>> YWRtaW5lc2I1MDA6ZXNiNTAwQERhcw==[\r][\n]"
>>> {org.apache.synapse.transport.http.wire}
>>> TID: [-1] [] [2016-08-04 05:34:33,684] DEBUG
>>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>>> dispatcher-1 >> "Cache-Control: no-cache[\r][\n]"
>>> {org.apache.synapse.transport.http.wire}
>>> TID: [-1] [] [2016-08-04 05:34:33,684] DEBUG
>>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>>> dispatcher-1 >> "User-Agent: Mozilla/5.0 (X11; Linux x86_64)
>>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116
>>> Safari/537.36[\r][\n]" {org.apache.synapse.transport.http.wire}
>>> TID: [-1] [] [2016-08-04 05:34:33,684] DEBUG
>>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>>> dispatcher-1 >> "Postman-Token: 
>>> 

Re: [Dev] [DEV][ESB500] Invoking a secure proxy in a clustered set up

[Pubudu Priyashan]

Hi Jagath,

So when we do that, it requires the user to restart the server every time
they add a new https endpoint to add the keys to client-truststore. In
production environments, this is not convenient and will have to go through
a long process to add a key to the truststore since it requires a server
restart. Are there any better approaches we can think of to overcome this
without having to restart the servers when we add new https endpoints?

Cheers,

Pubudu D.P
Senior Software Engineer - QA Team | WSO2 inc.
Mobile : +94775464547

Linkedin: https://uk.linkedin.com/in/pubududp
Medium: https://medium.com/@pubududp


On Mon, Aug 8, 2016 at 10:55 AM, Jagath Sisirakumara Ariyarathne <
jaga...@wso2.com> wrote:

> Hi Pubudu,
>
> Whenever you access a Https endpoint, you have to import back-end's
> certificate to client-truststore. That is the default requirement if you
> are using SSL between back-end and ESB.
>
> Thanks.
>
> On Mon, Aug 8, 2016 at 10:13 AM, Pubudu Priyashan 
> wrote:
>
>> Hi ESB team,
>>
>> I have observed the following behaviour when invoking a secure proxy in a
>> clustered setup which calls an external http endpoint [1] with ESB 5.0.0
>> (Tested with RC1). Can you please inform if this behaviour is correct or
>> are there any alternative methods?
>>
>> I have a secure proxy in my ESB cluster where it calls an external https
>> endpoint [1]. When I invoked this in a single node in my local machine, it
>> returned the response as expected. When I created the same secure proxy in
>> a Cluster, I received this error [2]. I was able to get rid of this error
>> by adding the certificate of the endpoint to client-truststore.jks file of
>> each ESB node of the cluster.
>>
>> My question is, is this behaviour correct? Should we have to always add
>> the https endpoint certificate every time we need to use an external
>> endpoint in a secure proxy? Or is there another approach where we don't
>> have to do this and not required to add the keys to client-truststore.jks
>> and restart ESB nodes? This might not be convenient if we have a lot of
>> https endpoints being called via a secure proxy in ESB. Can you please
>> advice?
>>
>> Thanks!
>>
>> [1] https://demo5224632.mockable.io/test123
>>
>> [2] Exception in ESB worker node:-
>>
>> TID: [-1] [] [2016-08-04 05:34:33,682] DEBUG
>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>> dispatcher-1 >> "GET /services/SecureP1 HTTP/1.1[\r][\n]"
>> {org.apache.synapse.transport.http.wire}
>> TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG
>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>> dispatcher-1 >> "X-Forwarded-Host: wrk.esb500.wso2.com[\r][\n]"
>> {org.apache.synapse.transport.http.wire}
>> TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG
>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>> dispatcher-1 >> "X-Forwarded-Server: wrk.esb500.wso2.com[\r][\n]"
>> {org.apache.synapse.transport.http.wire}
>> TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG
>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>> dispatcher-1 >> "X-Forwarded-For: 10.100.7.95[\r][\n]"
>> {org.apache.synapse.transport.http.wire}
>> TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG
>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>> dispatcher-1 >> "Connection: upgrade[\r][\n]" {org.apache.synapse.transport.
>> http.wire}
>> TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG
>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>> dispatcher-1 >> "Host: ssl.esb500.com[\r][\n]"
>> {org.apache.synapse.transport.http.wire}
>> TID: [-1] [] [2016-08-04 05:34:33,684] DEBUG
>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>> dispatcher-1 >> "Authorization: Basic 
>> YWRtaW5lc2I1MDA6ZXNiNTAwQERhcw==[\r][\n]"
>> {org.apache.synapse.transport.http.wire}
>> TID: [-1] [] [2016-08-04 05:34:33,684] DEBUG
>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>> dispatcher-1 >> "Cache-Control: no-cache[\r][\n]"
>> {org.apache.synapse.transport.http.wire}
>> TID: [-1] [] [2016-08-04 05:34:33,684] DEBUG
>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>> dispatcher-1 >> "User-Agent: Mozilla/5.0 (X11; Linux x86_64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116
>> Safari/537.36[\r][\n]" {org.apache.synapse.transport.http.wire}
>> TID: [-1] [] [2016-08-04 05:34:33,684] DEBUG
>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>> dispatcher-1 >> "Postman-Token: 60dc8ef4-36b0-03bd-5841-7efad2189071[\r][\n]"
>> {org.apache.synapse.transport.http.wire}
>> TID: [-1] [] [2016-08-04 05:34:33,684] DEBUG
>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>> dispatcher-1 >> "Accept: */*[\r][\n]" {org.apache.synapse.transport.
>> http.wire}
>> TID: [-1] [] [2016-08-04 05:34:33,685] DEBUG
>> {org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O
>> dispatcher-1 >> "Accept-Encoding: gzip, deflate, sdch[\r][\n]"
>> 

Re: [Dev] Fwd: GSOC2016: [ML][CEP] [SAMOA]Predictive analytic with online data for WSO2 Machine Learner-Samoa Integration

[Mahesh Dananjaya]

Hi supun,
I think i have fixed couple of previous errors. Now my samoa extension is
working fine with cep. But i am getting a exception with my spark streaming
ml extensions. It was working fine and i did not do any changes to my
previously developed streaming extensions. This error is occured in the
line,

conf = new SparkConf().setMaster("local[*]").setAppName("Linear Regression
Example").set("spark.driver.allowMultipleContexts", "true") ;
sc = new JavaSparkContext(conf);

This what i get and cep server is crashed.
ERROR {org.apache.spark.ui.SparkUI} -  Failed to bind SparkUI
javax.servlet.UnavailableException: Servlet class
com.sun.jersey.spi.container.servlet.ServletContainer is not a
javax.servlet.Servlet
at
org.spark-project.jetty.servlet.ServletHolder.checkServletType(ServletHolder.java:438)
at
org.spark-project.jetty.servlet.ServletHolder.doStart(ServletHolder.java:316)
at
org.spark-project.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
at
org.spark-project.jetty.servlet.ServletHandler.initialize(ServletHandler.java:791)
at
org.spark-project.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:265)
at
org.spark-project.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:717)
at
org.spark-project.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
at
org.spark-project.jetty.server.handler.HandlerCollection.doStart(HandlerCollection.java:229)
at
org.spark-project.jetty.server.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:172)
at
org.spark-project.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
at
org.spark-project.jetty.server.handler.HandlerWrapper.doStart(HandlerWrapper.java:95)
at org.spark-project.jetty.server.Server.doStart(Server.java:282)
at
org.spark-project.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
at
org.apache.spark.ui.JettyUtils$.org$apache$spark$ui$JettyUtils$$connect$1(JettyUtils.scala:228)
at org.apache.spark.ui.JettyUtils$$anonfun$2.apply(JettyUtils.scala:238)
at org.apache.spark.ui.JettyUtils$$anonfun$2.apply(JettyUtils.scala:238)
at
org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:1991)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141)
at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:1982)
at
org.apache.spark.ui.JettyUtils$.startJettyServer(JettyUtils.scala:238)
at org.apache.spark.ui.WebUI.bind(WebUI.scala:117)
at
org.apache.spark.SparkContext$$anonfun$13.apply(SparkContext.scala:448)
at
org.apache.spark.SparkContext$$anonfun$13.apply(SparkContext.scala:448)
at scala.Option.foreach(Option.scala:236)
at org.apache.spark.SparkContext.(SparkContext.scala:448)
at
org.apache.spark.api.java.JavaSparkContext.(JavaSparkContext.scala:61)
at
org.gsoc.siddhi.extension.streaming.StreamingKMeansClustering.(StreamingKMeansClustering.java:60)
at
org.gsoc.siddhi.extension.streaming.StreamingKMeansClusteringStreamProcessor.init(StreamingKMeansClusteringStreamProcessor.java:68)
at
org.wso2.siddhi.core.query.processor.stream.AbstractStreamProcessor.initProcessor(AbstractStreamProcessor.java:65)
at
org.wso2.siddhi.core.util.parser.SingleInputStreamParser.generateProcessor(SingleInputStreamParser.java:182)
at
org.wso2.siddhi.core.util.parser.SingleInputStreamParser.parseInputStream(SingleInputStreamParser.java:93)
at
org.wso2.siddhi.core.util.parser.InputStreamParser.parse(InputStreamParser.java:58)
at
org.wso2.siddhi.core.util.parser.QueryParser.parse(QueryParser.java:85)
at
org.wso2.siddhi.core.util.parser.ExecutionPlanParser.parse(ExecutionPlanParser.java:145)
at
org.wso2.siddhi.core.SiddhiManager.validateExecutionPlan(SiddhiManager.java:69)
at
org.wso2.siddhi.core.SiddhiManager.validateExecutionPlan(SiddhiManager.java:75)
at
org.wso2.carbon.event.processor.core.internal.util.helper.EventProcessorHelper.validateExecutionPlan(EventProcessorHelper.java:191)
at
org.wso2.carbon.event.processor.core.internal.CarbonEventProcessorService.validateExecutionPlan(CarbonEventProcessorService.java:493)
at
org.wso2.carbon.event.processor.admin.EventProcessorAdminService.validateExecutionPlan(EventProcessorAdminService.java:329)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
org.apache.axis2.rpc.receivers.RPCUtil.invokeServiceClass(RPCUtil.java:212)
at
org.apache.axis2.rpc.receivers.RPCMessageReceiver.invokeBusinessLogic(RPCMessageReceiver.java:117)
at
org.apache.axis2.receivers.AbstractInOutMessageReceiver.invokeBusinessLogic(AbstractInOutMessageReceiver.java:40)
at

Re: [Dev] [DEV][ESB500] Invoking a secure proxy in a clustered set up

[Jagath Sisirakumara Ariyarathne]

Hi Pubudu,

Whenever you access a Https endpoint, you have to import back-end's
certificate to client-truststore. That is the default requirement if you
are using SSL between back-end and ESB.

Thanks.

On Mon, Aug 8, 2016 at 10:13 AM, Pubudu Priyashan  wrote:

> Hi ESB team,
>
> I have observed the following behaviour when invoking a secure proxy in a
> clustered setup which calls an external http endpoint [1] with ESB 5.0.0
> (Tested with RC1). Can you please inform if this behaviour is correct or
> are there any alternative methods?
>
> I have a secure proxy in my ESB cluster where it calls an external https
> endpoint [1]. When I invoked this in a single node in my local machine, it
> returned the response as expected. When I created the same secure proxy in
> a Cluster, I received this error [2]. I was able to get rid of this error
> by adding the certificate of the endpoint to client-truststore.jks file of
> each ESB node of the cluster.
>
> My question is, is this behaviour correct? Should we have to always add
> the https endpoint certificate every time we need to use an external
> endpoint in a secure proxy? Or is there another approach where we don't
> have to do this and not required to add the keys to client-truststore.jks
> and restart ESB nodes? This might not be convenient if we have a lot of
> https endpoints being called via a secure proxy in ESB. Can you please
> advice?
>
> Thanks!
>
> [1] https://demo5224632.mockable.io/test123
>
> [2] Exception in ESB worker node:-
>
> TID: [-1] [] [2016-08-04 05:34:33,682] DEBUG 
> {org.apache.synapse.transport.http.wire}
> -  HTTPS-Listener I/O dispatcher-1 >> "GET /services/SecureP1
> HTTP/1.1[\r][\n]" {org.apache.synapse.transport.http.wire}
> TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG 
> {org.apache.synapse.transport.http.wire}
> -  HTTPS-Listener I/O dispatcher-1 >> "X-Forwarded-Host:
> wrk.esb500.wso2.com[\r][\n]" {org.apache.synapse.transport.http.wire}
> TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG 
> {org.apache.synapse.transport.http.wire}
> -  HTTPS-Listener I/O dispatcher-1 >> "X-Forwarded-Server:
> wrk.esb500.wso2.com[\r][\n]" {org.apache.synapse.transport.http.wire}
> TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG 
> {org.apache.synapse.transport.http.wire}
> -  HTTPS-Listener I/O dispatcher-1 >> "X-Forwarded-For:
> 10.100.7.95[\r][\n]" {org.apache.synapse.transport.http.wire}
> TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG 
> {org.apache.synapse.transport.http.wire}
> -  HTTPS-Listener I/O dispatcher-1 >> "Connection: upgrade[\r][\n]"
> {org.apache.synapse.transport.http.wire}
> TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG 
> {org.apache.synapse.transport.http.wire}
> -  HTTPS-Listener I/O dispatcher-1 >> "Host: ssl.esb500.com[\r][\n]"
> {org.apache.synapse.transport.http.wire}
> TID: [-1] [] [2016-08-04 05:34:33,684] DEBUG 
> {org.apache.synapse.transport.http.wire}
> -  HTTPS-Listener I/O dispatcher-1 >> "Authorization: Basic
> YWRtaW5lc2I1MDA6ZXNiNTAwQERhcw==[\r][\n]" {org.apache.synapse.transport.
> http.wire}
> TID: [-1] [] [2016-08-04 05:34:33,684] DEBUG 
> {org.apache.synapse.transport.http.wire}
> -  HTTPS-Listener I/O dispatcher-1 >> "Cache-Control: no-cache[\r][\n]"
> {org.apache.synapse.transport.http.wire}
> TID: [-1] [] [2016-08-04 05:34:33,684] DEBUG 
> {org.apache.synapse.transport.http.wire}
> -  HTTPS-Listener I/O dispatcher-1 >> "User-Agent: Mozilla/5.0 (X11; Linux
> x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116
> Safari/537.36[\r][\n]" {org.apache.synapse.transport.http.wire}
> TID: [-1] [] [2016-08-04 05:34:33,684] DEBUG 
> {org.apache.synapse.transport.http.wire}
> -  HTTPS-Listener I/O dispatcher-1 >> "Postman-Token:
> 60dc8ef4-36b0-03bd-5841-7efad2189071[\r][\n]"
> {org.apache.synapse.transport.http.wire}
> TID: [-1] [] [2016-08-04 05:34:33,684] DEBUG 
> {org.apache.synapse.transport.http.wire}
> -  HTTPS-Listener I/O dispatcher-1 >> "Accept: */*[\r][\n]"
> {org.apache.synapse.transport.http.wire}
> TID: [-1] [] [2016-08-04 05:34:33,685] DEBUG 
> {org.apache.synapse.transport.http.wire}
> -  HTTPS-Listener I/O dispatcher-1 >> "Accept-Encoding: gzip, deflate,
> sdch[\r][\n]" {org.apache.synapse.transport.http.wire}
> TID: [-1] [] [2016-08-04 05:34:33,685] DEBUG 
> {org.apache.synapse.transport.http.wire}
> -  HTTPS-Listener I/O dispatcher-1 >> "Accept-Language:
> en-US,en;q=0.8[\r][\n]" {org.apache.synapse.transport.http.wire}
> TID: [-1] [] [2016-08-04 05:34:33,685] DEBUG 
> {org.apache.synapse.transport.http.wire}
> -  HTTPS-Listener I/O dispatcher-1 >> "[\r][\n]"
> {org.apache.synapse.transport.http.wire}
> TID: [-1] [] [2016-08-04 05:34:33,813] DEBUG 
> {org.apache.synapse.transport.http.wire}
> -  HTTPS-Sender I/O dispatcher-1 << "CONNECT demo5224632.mockable.io:443
> HTTP/1.1[\r][\n]" {org.apache.synapse.transport.http.wire}
> TID: [-1] [] [2016-08-04 05:34:33,814] DEBUG 
> {org.apache.synapse.transport.http.wire}
> -  HTTPS-Sender I/O dispatcher-1 << "Host: 
> 

Re: [Dev] Fwd: GSOC2016: [ML][CEP] [SAMOA]Predictive analytic with online data for WSO2 Machine Learner-Samoa Integration

[Mahesh Dananjaya]

Hi Srinath and Nirmal,
I have integrated the my samoa predictive analysis topologies with the cep
with siddhi extension developed. As you asked me, i have verified the
output of my samoa streaming clustering topology with the previously build
Spark streaming Clustering analysis. Results are almost same and is
converging. As a example if i used two algorithms spark mini-batch
clustering and samoa kernal clustering for streaming clustering analysis
outcomes are as follows in my streaming extensions.

Spark Streaming Clustering
center 0:
440.873301276124,25.477847235065635,64.20812280377294,1010.9613963380791,67.80943221749587
center 1:
470.25742216416273,12.785760940561731,42.586181145220955,1015.9444850860008,79.6946897452645

Samoa Streaming Clustering
Center :0:
442.06527377405365,25.51594930115884,60.35312500444376,1010.624677139,61.262047085828065
Center :1:
469.66365199261037,12.6577054812157,43.59210147812603,1014.3543441369945,76.81973235658958

you can see that cluster centers are almost same.This is with 5 attributes
and two clusters. As long as we have large batch size and number of
increasing number of retraining, the results are converging. Now i am
trying to integrate this with carbon-ml and trying to fix couple of issues
in the integration. I will also prepare the documentation.thank you. And i
have create new repository [1] to put essential classes.
regards,
Mahesh.
[1] https://github.com/dananjayamahesh/streaming


On Fri, Aug 5, 2016 at 6:47 PM, Nirmal Fernando  wrote:

> Mahesh,
>
> Your issue is not clear. What're you exactly trying ? carbon-ml is our git
> repo and if you mean to say about the server you're running, please use the
> term 'ML server'. Please summarize the problem and the steps you've done.
> Please use the point form.
>
> Thanks.
>
> On Fri, Aug 5, 2016 at 6:43 PM, Mahesh Dananjaya <
> dananjayamah...@gmail.com> wrote:
>
>> Hi Nirmal,
>> can those things be caused by permission requirements?
>> BR,
>> Mahesh.
>>
>> On Fri, Aug 5, 2016 at 5:39 PM, Mahesh Dananjaya <
>> dananjayamah...@gmail.com> wrote:
>>
>>> Hi Nirmal,
>>> Yes i have added samoa dependencies in carbon-ml and it is working fine.
>>> These exceptions are coming for the my classes that are inside ml. when i
>>> used classes in samoa like org.apache.samoa.tasks.ClusteringEvaluation
>>> it is working fine and run. And also same classes that i developed are
>>> running fine in my other extensions outside carbon-ml, which are developed
>>> as regular extensions. i am checking this further. thank you.
>>> regards,
>>> Mahesh.
>>>
>>> On Fri, Aug 5, 2016 at 5:31 PM, Nirmal Fernando  wrote:
>>>
 Did you add samoa jars in ML?

 On Fri, Aug 5, 2016 at 12:20 PM, Mahesh Dananjaya <
 dananjayamah...@gmail.com> wrote:

> Hi Supun,
> This is the error i am getting while run the extension in the cabon-ml
> side,
> plase reffer to link [1] for the class.
>
> ERROR 
> {org.wso2.carbon.ml.siddhi.extension.streaming.samoa.StreamingClusteringTaskBuilder}
> -  Fail to initialize the task
>
> java.lang.Exception: Class not found: StreamingClusteringTask
>
>at com.github.javacliparser.ClassOption.cliStringToObject(Class
> Option.java:136)
>
>at org.wso2.carbon.ml.siddhi.extension.streaming.samoa.Streamin
> gClusteringTaskBuilder.initClusteringTask(StreamingClusterin
> gTaskBuilder.java:129)
>
>at org.wso2.carbon.ml.siddhi.extension.streaming.samoa.Streamin
> gClusteringTaskBuilder.initTask(StreamingClusteringTaskBuild
> er.java:100)
>
>at org.wso2.carbon.ml.siddhi.extension.streaming.samoa.Streamin
> gClustering.run(StreamingClustering.java:77)
>
>at java.lang.Thread.run(Thread.java:745)
>
> [2016-08-04 16:23:07,437]  INFO {org.wso2.carbon.ml.siddhi.ext
> ension.streaming.samoa.StreamingClusteringTaskBuilder} -  Fail to
> initialize the taskjava.lang.Exception: Class not found:
> StreamingClusteringTask
>
> +++Please refeer link [1] for the StreamingClusteringTask.
>
> Then again for StreamingClusteringStream class while i bypass the
> String query in the initTask(). please refer link [2] for the class.
>
> Exception in thread "Thread-60" java.lang.IllegalArgumentException:
> Problems with option: streamTrain
>
>at com.github.javacliparser.ClassOption.setValueViaCLIString(Cl
> assOption.java:64)
>
>at com.github.javacliparser.AbstractOption.resetToDefault(Abstr
> actOption.java:90)
>
>at com.github.javacliparser.AbstractClassOption.(Abstract
> ClassOption.java:84)
>
>at com.github.javacliparser.AbstractClassOption.(Abstract
> ClassOption.java:63)
>
>at com.github.javacliparser.ClassOption.(ClassOption.java:38)
>
>at org.wso2.carbon.ml.siddhi.extension.streaming.samoa.Streamin
> 

[Dev] [DEV][ESB500] Invoking a secure proxy in a clustered set up

[Pubudu Priyashan]

Hi ESB team,

I have observed the following behaviour when invoking a secure proxy in a
clustered setup which calls an external http endpoint [1] with ESB 5.0.0
(Tested with RC1). Can you please inform if this behaviour is correct or
are there any alternative methods?

I have a secure proxy in my ESB cluster where it calls an external https
endpoint [1]. When I invoked this in a single node in my local machine, it
returned the response as expected. When I created the same secure proxy in
a Cluster, I received this error [2]. I was able to get rid of this error
by adding the certificate of the endpoint to client-truststore.jks file of
each ESB node of the cluster.

My question is, is this behaviour correct? Should we have to always add the
https endpoint certificate every time we need to use an external endpoint
in a secure proxy? Or is there another approach where we don't have to do
this and not required to add the keys to client-truststore.jks and restart
ESB nodes? This might not be convenient if we have a lot of https endpoints
being called via a secure proxy in ESB. Can you please advice?

Thanks!

[1] https://demo5224632.mockable.io/test123

[2] Exception in ESB worker node:-

TID: [-1] [] [2016-08-04 05:34:33,682] DEBUG
{org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O dispatcher-1
>> "GET /services/SecureP1 HTTP/1.1[\r][\n]"
{org.apache.synapse.transport.http.wire}
TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG
{org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O dispatcher-1
>> "X-Forwarded-Host: wrk.esb500.wso2.com[\r][\n]"
{org.apache.synapse.transport.http.wire}
TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG
{org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O dispatcher-1
>> "X-Forwarded-Server: wrk.esb500.wso2.com[\r][\n]"
{org.apache.synapse.transport.http.wire}
TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG
{org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O dispatcher-1
>> "X-Forwarded-For: 10.100.7.95[\r][\n]"
{org.apache.synapse.transport.http.wire}
TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG
{org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O dispatcher-1
>> "Connection: upgrade[\r][\n]" {org.apache.synapse.transport.http.wire}
TID: [-1] [] [2016-08-04 05:34:33,683] DEBUG
{org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O dispatcher-1
>> "Host: ssl.esb500.com[\r][\n]" {org.apache.synapse.transport.http.wire}
TID: [-1] [] [2016-08-04 05:34:33,684] DEBUG
{org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O dispatcher-1
>> "Authorization: Basic YWRtaW5lc2I1MDA6ZXNiNTAwQERhcw==[\r][\n]"
{org.apache.synapse.transport.http.wire}
TID: [-1] [] [2016-08-04 05:34:33,684] DEBUG
{org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O dispatcher-1
>> "Cache-Control: no-cache[\r][\n]"
{org.apache.synapse.transport.http.wire}
TID: [-1] [] [2016-08-04 05:34:33,684] DEBUG
{org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O dispatcher-1
>> "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML,
like Gecko) Chrome/48.0.2564.116 Safari/537.36[\r][\n]"
{org.apache.synapse.transport.http.wire}
TID: [-1] [] [2016-08-04 05:34:33,684] DEBUG
{org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O dispatcher-1
>> "Postman-Token: 60dc8ef4-36b0-03bd-5841-7efad2189071[\r][\n]"
{org.apache.synapse.transport.http.wire}
TID: [-1] [] [2016-08-04 05:34:33,684] DEBUG
{org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O dispatcher-1
>> "Accept: */*[\r][\n]" {org.apache.synapse.transport.http.wire}
TID: [-1] [] [2016-08-04 05:34:33,685] DEBUG
{org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O dispatcher-1
>> "Accept-Encoding: gzip, deflate, sdch[\r][\n]"
{org.apache.synapse.transport.http.wire}
TID: [-1] [] [2016-08-04 05:34:33,685] DEBUG
{org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O dispatcher-1
>> "Accept-Language: en-US,en;q=0.8[\r][\n]"
{org.apache.synapse.transport.http.wire}
TID: [-1] [] [2016-08-04 05:34:33,685] DEBUG
{org.apache.synapse.transport.http.wire} -  HTTPS-Listener I/O dispatcher-1
>> "[\r][\n]" {org.apache.synapse.transport.http.wire}
TID: [-1] [] [2016-08-04 05:34:33,813] DEBUG
{org.apache.synapse.transport.http.wire} -  HTTPS-Sender I/O dispatcher-1
<< "CONNECT demo5224632.mockable.io:443 HTTP/1.1[\r][\n]"
{org.apache.synapse.transport.http.wire}
TID: [-1] [] [2016-08-04 05:34:33,814] DEBUG
{org.apache.synapse.transport.http.wire} -  HTTPS-Sender I/O dispatcher-1
<< "Host: demo5224632.mockable.io:443[\r][\n]"
{org.apache.synapse.transport.http.wire}
TID: [-1] [] [2016-08-04 05:34:33,814] DEBUG
{org.apache.synapse.transport.http.wire} -  HTTPS-Sender I/O dispatcher-1
<< "Proxy-Connection: Keep-Alive[\r][\n]"
{org.apache.synapse.transport.http.wire}
TID: [-1] [] [2016-08-04 05:34:33,814] DEBUG
{org.apache.synapse.transport.http.wire} -  HTTPS-Sender I/O dispatcher-1
<< "Proxy-Authorization: Basic c3F1aWQzdTpzcXVpZDN1[\r][\n]"

Re: [Dev] GSOC 2016 - Project 21 : MongoDB Userstore Development

[Tharindu Edirisinghe]

+Mohan

Hi Asantha,

I've added Mohan to this mail thread who is working in the Analytics team
at WSO2. He will be able to give a helping hand to the user activity
analysis part that you are doing in this project.

I've scheduled a google hangout (12th Aug 2016 5.00 p.m IST) for discussing
this further where Mohan would join to clarify your concerns in this
project. I've already sent a meeting request.

Meanwhile, it is better to reply to this thread with the problems you have
faced so Mohan can get an idea on in which areas you need help.

Thanks,
TharinduE

On Mon, Aug 1, 2016 at 12:05 AM, Asantha Thilina 
wrote:

> Hi Tharindu,
>
> yes sure i am comfortable with any time and look forward for that
>
> Thanks,
> Asantha
>
> On Sun, Jul 31, 2016 at 12:15 PM, Tharindu Edirisinghe  > wrote:
>
>> Hi Asantha,
>>
>> Good work so far. Since we are reaching the final stage of GSoC, let's
>> have a hangout at the end of this week and plan the final steps to complete
>> the project.
>>
>> I will go through the deliverables and provide feedback soon.
>>
>> Thanks,
>> Tharindu
>>
>> On Mon, Jul 25, 2016 at 12:08 PM, Asantha Thilina <
>> asanthathil...@gmail.com> wrote:
>>
>>> Hi Tharindu,
>>>
>>> I have cleaned up all the bugs and removed all unnecessary codes and
>>> committed all the changes into my repository[1]
>>>
>>> [1] https://github.com/asanthamax/mongodbuserstore
>>>
>>> Thanks,
>>> Asantha
>>>
>>> On Sat, Jul 23, 2016 at 8:26 AM, Asantha Thilina <
>>> asanthathil...@gmail.com> wrote:
>>>
 Hi Tharindu,

 here i added all the commitments to spread sheet[1] and also i updated
 my blog with about java document api[2] and also i added findbug report[3]
 and also i updated my blog with the code quality analysis process i have
 done[4]

 [1]https://docs.google.com/spreadsheets/d/1jwSR_qFV-
 LfOS7ZTZ8CpiFFTYPoudDQF7zQhvblivaE/edit?usp=sharing
 [2]http://asanthamax.blogspot.com/2016/07/mongodb-user-
 store-package-hierarchy.html
 [3]https://googledrive.com/host/0B__ZE1ru1jkXZFdES3BOaHVyZ3M/
 [4]http://asanthamax.blogspot.com/2016/07/findbugs-in-
 mongodbuserstoremanager-in.html

 Thanks,
 Asantha

 On Fri, Jul 22, 2016 at 8:45 AM, Asantha Thilina <
 asanthathil...@gmail.com> wrote:

> Hi Tharindu,
>
> I have created the API documentation and hosted in my google drive[1]
> i will be preparing code inspection report as well will share it as soon 
> as
> well i also added github page also to the userstore script in here[2] as
> well
> [1] http://googledrive.com/host/0B__ZE1ru1jkXbDVoSDlveV9yNk0
> [2] https://asanthamax.github.io/mongodbuserstore/
>
> Thanks,
> Asantha
>
> On Tue, Jul 12, 2016 at 11:59 AM, Asantha Thilina <
> asanthathil...@gmail.com> wrote:
>
>> Hi Tharindu,
>> sure we will arrange a hangout on next week and sure i will refer the
>> link and do the necessary inspect in my code for any security
>> vulnerabilities and update you sooner
>>
>> Thanks,
>> Asantha
>>
>> On Tue, Jul 12, 2016 at 9:56 AM, Tharindu Edirisinghe <
>> tharin...@wso2.com> wrote:
>>
>>> Hi Asantha,
>>>
>>> Thanks for the update. I'm out of the country this week and won't be
>>> available. Let's have a hangout next week and discuss the final steps of
>>> the project.
>>>
>>> Along with find bugs, if you haven't enabled findsecuritybugs
>>> plugin, you can refer [1] and check if any security weaknesses are 
>>> there in
>>> the code.
>>>
>>> [1] http://tharindue.blogspot.in/2016/06/static-code-analysis-
>>> for-java-using.html
>>>
>>> Thanks,
>>> TharinduE
>>>
>>> On Tue, Jul 12, 2016 at 5:32 PM, Asantha Thilina <
>>> asanthathil...@gmail.com> wrote:
>>>
 Hi Tharindu,
 I have clean up the code according to find bug plugin and also for
 rollback purposes since it only concern when modifying or adding new 
 user
 or role to user store i have manually added rollback option where if 
 error
 occured all the changes done to db in code segment will be discard 
 from db
 i updated the my repository with recent changes[1] now i preparing the
 documentation and test scripts.

 [1] https://github.com/asanthamax/mongodbuserstore

 Thanks,
 Asantha

 On Mon, Jul 4, 2016 at 2:54 AM, Asantha Thilina <
 asanthathil...@gmail.com> wrote:

> Hi Tharindu,
>
> thanks for the reply ,sure i will try out that and update the
> progress as soon as possible
>
> Thanks,
> Asantha
>
> On Mon, Jul 4, 2016 at 1:38 AM, Tharindu Edirisinghe <
> tharin...@wso2.com> wrote:
>
>> Hi Asantha,
>>
>> 

[Dev] WSO2 App Cloud - Environment variables for setting credentials

[Nathiesha Maddage]

Hi all,

I deployed an application in WSO2 cloud, which requires JIRA account
credentials that need to be set easily.
Is it safe to use environment variables in WSO2 app cloud to set those
credentials? Are those environment variables visible to any external
parties?

Thanks and regards
Nathiesha
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [ESB] fn:sum is not working

[Godwin Shrimal]

Thanks for the response Rajith.

On Fri, Aug 5, 2016 at 3:58 PM, Rajith Vitharana  wrote:

> Hi Godwin,
>
> According to [1] fn:sum is used to make sum of some xpath objects, for
> example if the payload is [2], then "fn:sum(//order//item/@quantity)"
> will return you the value 11. So I think you can't use fn:sum to just add
> two numbers together.
>
>
> [1] - https://www.w3.org/TR/xpath-functions/#func-sum
> [2] - 
> 
> 
> 
> 
> 
> 
> 
>
> Thanks,
>
> On 6 August 2016 at 01:55, Godwin Shrimal  wrote:
>
>> Hi Udara,
>>
>> Yeah, we can use script mediator as an alternative, I need to know
>> weather we can use fn:sum or is it not working in ESB.
>>
>> Thanks
>> Godwin
>>
>> On Fri, Aug 5, 2016 at 3:19 PM, Udara Liyanage  wrote:
>>
>>> Hi Godwin,
>>>
>>> As a workaround following worked for me. I had to use parseInt, just
>>> using + did not work for me.
>>>
>>> 
>>> 
>>> >>   var sum = parseInt(mc.getProperty("num1"))
>>> +parseInt(mc.getProperty("num2"));
>>>   var sum = mc.setProperty("testValue",sum.toString());
>>>   ]]>
>>>
>>> On Sat, Aug 6, 2016 at 12:44 AM, Godwin Shrimal  wrote:
>>>
 Hi All,

 I have following sample config to sum two values

 
 
 http://www.w3.org/2005/xpath-functions;
 xmlns:test="http://test;  xmlns:ns="http://org.apache.synapse/xsd;
 name="testValue" expression="fn:sum($ctx:num1,$ctx:num2)"/>

 
   
 


 Total gives as 10 not 30, Is this a known bug or am I doing something
 wrong ?


 Thanks
 Godwin


 --
 *Godwin Amila Shrimal*
 Senior Software Engineer
 WSO2 Inc.; http://wso2.com
 lean.enterprise.middleware

 mobile: *+94772264165*
 linkedin: *http://lnkd.in/KUum6D *
 twitter: https://twitter.com/godwinamila
 

 ___
 Dev mailing list
 Dev@wso2.org
 http://wso2.org/cgi-bin/mailman/listinfo/dev


>>>
>>>
>>> --
>>>
>>> Udara Liyanage
>>> Software Engineer
>>> WSO2, Inc.: http://wso2.com
>>> lean. enterprise. middleware
>>>
>>> Blog: http://udaraliyanage.wordpress.com
>>>
>>
>>
>>
>> --
>> *Godwin Amila Shrimal*
>> Senior Software Engineer
>> WSO2 Inc.; http://wso2.com
>> lean.enterprise.middleware
>>
>> mobile: *+94772264165*
>> linkedin: *http://lnkd.in/KUum6D *
>> twitter: https://twitter.com/godwinamila
>> 
>>
>> ___
>> Dev mailing list
>> Dev@wso2.org
>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>
>>
>
>
> --
> Rajith Vitharana
>
> Senior Software Engineer,
> WSO2 Inc. : wso2.com
> Mobile : +94715883223
> Blog : http://lankavitharana.blogspot.com/
> 
>



-- 
*Godwin Amila Shrimal*
Senior Software Engineer
WSO2 Inc.; http://wso2.com
lean.enterprise.middleware

mobile: *+94772264165*
linkedin: *http://lnkd.in/KUum6D *
twitter: https://twitter.com/godwinamila

___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [GSoC 2016]Proposal 03: [ML]Visual(Deep) Neural Network Builder -Project Progress

[Lakini Senanayaka]

Hi,

Thank you very much.

But Upul told me about a code snippet which I can use inside the API
implementation in the last meeting.

However can I get the dataset file from calling *GET
https://localhost:9443/api/datasets/{datasetId}
 * inside the API
implementation in the backend (inside
carbon-ml\components\ml\org.wso2.carbon.ml.rest.api\src\main\java\org\wso2\carbon\ml\rest\api
)
and can I access the dataset through the response's *"sourcePath" ?*

Thank you.

On Sun, Aug 7, 2016 at 8:06 PM, Supun Sethunga  wrote:

> Hi Lakini,
>
> Please find all the APIs and their samples at [1]. Additionally, you can
> always refer the existing UI to get an idea. for eg: you can check the
> dataset page of the existing ML UI to see how the datasets are retrieved.
>
> [1] https://docs.wso2.com/display/ML110/REST+API+Guides
>
> Regards,
> Supun.
>
> On Sat, Aug 6, 2016 at 11:06 AM, Lakini Senanayaka <
> lakinisenanayak...@gmail.com> wrote:
>
>> Hi Upul,
>>
>> Could you please explain me the way to get the dataset from the carbon ML
>> to the API which I am implementing.
>>
>> Thank you.
>>
>> On Thu, Aug 4, 2016 at 4:37 PM, Lakini Senanayaka <
>> lakinisenanayak...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> This is the meeting minute for today's GSoC project review meeting.
>>>
>>> Upul reviewed the current progress of the project, "(Deep) Neural
>>> Network Builder for WSO2 Machine Learner".
>>> Below mentioned are the decisions made and the new tasks allocated at
>>> the meeting.
>>>
>>> Assigned action items:
>>> -Making the UI more user-friendly.
>>> Ex: Physical lines of connections between layers should follow
>>> the movements of the layers.
>>> -Train the feed forward network for MNIST dataset.
>>> -Testing the accuracy of the neural network model for different
>>> optimization and updater algorithms.
>>> -Handling Exceptions.
>>> -Trying to show graphs related to the model, along with the accuracy
>>> of the model in the output.
>>> -Start writing documentation.
>>> -Showing a demo for the ML team.
>>>
>>> Decisions made:
>>>
>>>  -Higher priority should be given in implementing feed-forward
>>> network than RNN.
>>>  -Upul will provide me necessary instructions on the way to access
>>> the dataset which is inserted through the WSO2 ML console as NN builder UI
>>> needs to access that dataset and use as the training dataset.
>>>
>>> Thank you.
>>>
>>> On Wed, Aug 3, 2016 at 5:59 AM, Lakini Senanayaka <
>>> lakinisenanayak...@gmail.com> wrote:
>>>
 Hi,

 Thank you very much for your kindness.

 I can schedule a meeting on this Thursday(4th-August-2016).

 Thank you.

 On Tue, Aug 2, 2016 at 10:39 AM, Upul Bandara  wrote:

> Hi,
>
> As you know we have only two weeks to complete GSOC projects.
> Unfortunately, this project is a little bit lagging behind and it looks
> like you have to put some additional effort during the last two week. In
> order to support you, we though it is better to have a small F2F meeting
> with you and it will help you to sort out any issues currently you are
> facing with.
>
> So can you please schedule a meeting with the ML team?
>
> Thanks,
> Upul
>
> On Sat, Jul 30, 2016 at 5:45 PM, Nirmal Fernando 
> wrote:
>
>>
>> On Sat, Jul 30, 2016 at 1:43 PM, Lakini Senanayaka <
>> lakinisenanayak...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I have solved the above problem.I have referred the mail thread *[Dev]
>>> "Error 403 - Forbidden" when session expires in admin console*[1] in
>>> dev mailing list.I couldn't upload  jaggery files to the console  and 
>>> when
>>> I was trying to do it, it gave the* Error 403-Forbidden.*
>>>
>>> I have copied the jaggery files to
>>> */repository/deployment/server/jaggerapps* directory as
>>> Supun instructed and it worked.
>>>
>>> Although I have implemented the back end for feed forward neural
>>> network still it has some problems.It doesn't work well.Do I need to add
>>> the  neural network algorithms to the
>>> *org.wso2.carbon.ml.core.spark.algorithms* in Carbon ML ?Do I need
>>> to follow the same pattern which is used by the Carbon ML when 
>>> implementing
>>> the back end coding and APIs?
>>>
>>
>> Yes please.
>>
>>>
>>> I will submit the demo ASAP.
>>>
>>> [1]https://mail.google.com/mail/u/0/#search/Re%3A+%5BDev%5D+
>>> %22Error+403++Forbidden%22+when+session+expires+in+admin%09c
>>> onsole/155bfcbf7e98992f?projector=1
>>>
>>> Thank you.
>>> --
>>> KIND Regards,
>>> *Lakini Senanayaka.*
>>>
>>>
>>
>>
>> --
>>
>> Thanks & regards,
>> Nirmal
>>
>> Team Lead - WSO2 Machine Learner
>> Associate Technical Lead - Data 

Re: [Dev] [GSoC 2016]Proposal 03: [ML]Visual(Deep) Neural Network Builder -Project Progress

[Supun Sethunga]

Hi Lakini,

Please find all the APIs and their samples at [1]. Additionally, you can
always refer the existing UI to get an idea. for eg: you can check the
dataset page of the existing ML UI to see how the datasets are retrieved.

[1] https://docs.wso2.com/display/ML110/REST+API+Guides

Regards,
Supun.

On Sat, Aug 6, 2016 at 11:06 AM, Lakini Senanayaka <
lakinisenanayak...@gmail.com> wrote:

> Hi Upul,
>
> Could you please explain me the way to get the dataset from the carbon ML
> to the API which I am implementing.
>
> Thank you.
>
> On Thu, Aug 4, 2016 at 4:37 PM, Lakini Senanayaka <
> lakinisenanayak...@gmail.com> wrote:
>
>> Hi,
>>
>> This is the meeting minute for today's GSoC project review meeting.
>>
>> Upul reviewed the current progress of the project, "(Deep) Neural Network
>> Builder for WSO2 Machine Learner".
>> Below mentioned are the decisions made and the new tasks allocated at the
>> meeting.
>>
>> Assigned action items:
>> -Making the UI more user-friendly.
>> Ex: Physical lines of connections between layers should follow
>> the movements of the layers.
>> -Train the feed forward network for MNIST dataset.
>> -Testing the accuracy of the neural network model for different
>> optimization and updater algorithms.
>> -Handling Exceptions.
>> -Trying to show graphs related to the model, along with the accuracy
>> of the model in the output.
>> -Start writing documentation.
>> -Showing a demo for the ML team.
>>
>> Decisions made:
>>
>>  -Higher priority should be given in implementing feed-forward
>> network than RNN.
>>  -Upul will provide me necessary instructions on the way to access
>> the dataset which is inserted through the WSO2 ML console as NN builder UI
>> needs to access that dataset and use as the training dataset.
>>
>> Thank you.
>>
>> On Wed, Aug 3, 2016 at 5:59 AM, Lakini Senanayaka <
>> lakinisenanayak...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Thank you very much for your kindness.
>>>
>>> I can schedule a meeting on this Thursday(4th-August-2016).
>>>
>>> Thank you.
>>>
>>> On Tue, Aug 2, 2016 at 10:39 AM, Upul Bandara  wrote:
>>>
 Hi,

 As you know we have only two weeks to complete GSOC projects.
 Unfortunately, this project is a little bit lagging behind and it looks
 like you have to put some additional effort during the last two week. In
 order to support you, we though it is better to have a small F2F meeting
 with you and it will help you to sort out any issues currently you are
 facing with.

 So can you please schedule a meeting with the ML team?

 Thanks,
 Upul

 On Sat, Jul 30, 2016 at 5:45 PM, Nirmal Fernando 
 wrote:

>
> On Sat, Jul 30, 2016 at 1:43 PM, Lakini Senanayaka <
> lakinisenanayak...@gmail.com> wrote:
>
>> Hi,
>>
>> I have solved the above problem.I have referred the mail thread *[Dev]
>> "Error 403 - Forbidden" when session expires in admin console*[1] in
>> dev mailing list.I couldn't upload  jaggery files to the console  and 
>> when
>> I was trying to do it, it gave the* Error 403-Forbidden.*
>>
>> I have copied the jaggery files to
>> */repository/deployment/server/jaggerapps* directory as
>> Supun instructed and it worked.
>>
>> Although I have implemented the back end for feed forward neural
>> network still it has some problems.It doesn't work well.Do I need to add
>> the  neural network algorithms to the
>> *org.wso2.carbon.ml.core.spark.algorithms* in Carbon ML ?Do I need
>> to follow the same pattern which is used by the Carbon ML when 
>> implementing
>> the back end coding and APIs?
>>
>
> Yes please.
>
>>
>> I will submit the demo ASAP.
>>
>> [1]https://mail.google.com/mail/u/0/#search/Re%3A+%5BDev%5D+
>> %22Error+403++Forbidden%22+when+session+expires+in+admin%09c
>> onsole/155bfcbf7e98992f?projector=1
>>
>> Thank you.
>> --
>> KIND Regards,
>> *Lakini Senanayaka.*
>>
>>
>
>
> --
>
> Thanks & regards,
> Nirmal
>
> Team Lead - WSO2 Machine Learner
> Associate Technical Lead - Data Technologies Team, WSO2 Inc.
> Mobile: +94715779733
> Blog: http://nirmalfdo.blogspot.com/
>
>
>


 --
 Upul Bandara,
 Associate Technical Lead, WSO2, Inc.,
 Mob: +94 715 468 345.

>>>
>>>
>>>
>>> --
>>> KIND Regards,
>>> *Lakini Senanayaka.*
>>>
>>>
>>
>>
>> --
>> KIND Regards,
>> *Lakini Senanayaka.*
>>
>>
>
>
> --
> KIND Regards,
> *Lakini Senanayaka.*
>
>


-- 
*Supun Sethunga*
Senior Software Engineer
WSO2, Inc.
http://wso2.com/
lean | enterprise | middleware
Mobile : +94 716546324
Blog: http://supunsetunga.blogspot.com
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] Updating an output stream in Siddhi Query

[Grainier Perera]

Hi Aneela,

As I pointed out in [1], you can use externalTimeBatch [2] with above
scenario. Furthermore, in CEP 4.2.0 (Siddhi 3.1.0 <), it'll introduce 2 new
optional parameters to externalTimeBatch, namely startTime and timeout.
However, you might not be needing those params with above scenario.

Moreover, CEP 4.2.0 is yet to be released, and at the time being you can
download CEP 4.2.0 RC1 [3] as @damith pointed out in reply to [4].


>- *startTime* (Optional): User defined start time. This could either
>be a constant (of type int, long or time) or an attribute of the
>corresponding stream (of type long). If an attribute is provided, initial
>value of attribute would be considered as startTime. When startTime is not
>given, initial value of timestamp is used as the default.
>
>
>- *timeout *(Optional): Time to wait for arrival of new event, before
>flushing and giving output for events belonging to a specific batch. If
>timeout is not provided, system waits till an event from next batch arrives
>to flush current batch.
>
>
[1] http://stackoverflow.com/a/38811132/1805178
[2]
https://docs.wso2.com/display/CEP410/Inbuilt+Windows#InbuiltWindows-externalTimeBatch
[3] https://github.com/wso2/product-cep/releases/download/
v4.2.0-rc1/wso2cep-4.2.0-RC1.zip
[4] [Dev] Donwloading latest wso2cep 4.2.0 release

Regards,
Grainier.

On Sat, Aug 6, 2016 at 2:31 PM, Aneela Safdar  wrote:

> Hi,
>
> I have a siddhi query to get the count of total events in one minute by
> using a timebatch window. Using an output stream I am updating a bar chart
> with constant coming values, tmestammps(date and time,till minute ) on
> x-axis and events count on y-axis.
>
> But there are sometimes when either the number of events in one minute
> take too long to be transmitted and hence query doesnot give correct
> resutls.
>
> For instance if I get total 60 events and this query first gives me the
> count of 40 which dislpayed in bar chart but then after a minute it changes
> its value to 20 which is correct according to the logic but I am concerned
> if there is a way I could update the stream as well as bar chart for any
> previous timestamps (in that case 40+20) and insert into it new values for
> the next upcoming timestamps.
>
> I have seen update function is used with tables not stream, is it so?
> And also I want two outputStreams populating two different bar charts from
> a same input stream. So is below query correct for that purpose?
>
> Query is:
>
> /* Enter a unique ExecutionPlan */
> @Plan:name('FTPExecutionPlan')
>
> /* Enter a unique description for ExecutionPlan */
> -- @Plan:description('ExecutionPlan')
>
> /* define streams/tables and write queries here ... */
>
> @Import('FTPInStream:1.0.0')
> define stream FTPInStream (ts string, uid string, id_orig_h string,
> id_orig_p int, id_resp_h string, id_resp_p int, user string, password
> string,command string, arg string, mime_type string, file_size string,
> reply_code int, reply_msg string);
>
> @Export('FTPIPOutStream:1.0.0')
> define stream FTPIPOutStream (ip_address string, ftp_requests int);
>
> @Export('FTPOutStream:1.0.0')
> define stream FTPOutStream (ts string, ftp_requests int);
>
>
> from FTPInStream
> select time:dateFormat(str:replaceAll(ts,'T',' '),'-MM-dd HH:mm',
> '-MM-dd HH:mm:ss') as ts, uid, id_orig_h, id_orig_p, id_resp_h,
> id_resp_p
> insert into intermediateStream;
>
> from intermediateStream#window.timeBatch(1 min)
> select ts, cast(count(ts), 'int') as ftp_requests
> group by ts
> insert into FTPOutStream;
>
> from intermediateStream#window.timeBatch(1 min)
> select id_orig_h as ip_address, cast(count(id_orig_h), 'int') as
> ftp_requests
> group by id_orig_h
> insert into FTPIPOutStream;
>
> Regards,
> Aneela Safdar
>
> ___
> Dev mailing list
> Dev@wso2.org
> http://wso2.org/cgi-bin/mailman/listinfo/dev
>
>


-- 
Grainier Perera
Software Engineer
Mobile : +94716122384
WSO2 Inc. | http://wso2.com
lean.enterprise.middleware
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] Donwloading latest wso2cep 4.2.0 release

[Damith Wickramasinghe]

Not a RC2 but a RC1 as specified in the download URL.

Thanks,
Damith

On Sun, Aug 7, 2016 at 6:08 PM, Damith Wickramasinghe 
wrote:

> Hi Aneela,
>
> Wso2cep 4.2.0 is not yet released. But we have released a RC2 which you
> can download using [1].
>
> [1] https://github.com/wso2/product-cep/releases/download/
> v4.2.0-rc1/wso2cep-4.2.0-RC1.zip
>
> Regards,
> Damith.
>
> On Sun, Aug 7, 2016 at 5:56 PM, Aneela Safdar  wrote:
>
>> Hi,
>>
>> I am trying to download new release of wso2cep but on download page it is
>> still giving me the older version.
>>
>> Any suggestion?
>>
>> [image: Inline image]
>>
>> Regards,
>> Aneela Safdar
>>
>> ___
>> Dev mailing list
>> Dev@wso2.org
>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>
>>
>
>
> --
> Software Engineer
> WSO2 Inc.; http://wso2.com
> 
> lean.enterprise.middleware
>
> mobile: *+94728671315 <%2B94728671315>*
>
>


-- 
Software Engineer
WSO2 Inc.; http://wso2.com

lean.enterprise.middleware

mobile: *+94728671315*
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] Donwloading latest wso2cep 4.2.0 release

[Damith Wickramasinghe]

Hi Aneela,

Wso2cep 4.2.0 is not yet released. But we have released a RC2 which you can
download using [1].

[1]
https://github.com/wso2/product-cep/releases/download/v4.2.0-rc1/wso2cep-4.2.0-RC1.zip

Regards,
Damith.

On Sun, Aug 7, 2016 at 5:56 PM, Aneela Safdar  wrote:

> Hi,
>
> I am trying to download new release of wso2cep but on download page it is
> still giving me the older version.
>
> Any suggestion?
>
> [image: Inline image]
>
> Regards,
> Aneela Safdar
>
> ___
> Dev mailing list
> Dev@wso2.org
> http://wso2.org/cgi-bin/mailman/listinfo/dev
>
>


-- 
Software Engineer
WSO2 Inc.; http://wso2.com

lean.enterprise.middleware

mobile: *+94728671315*
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Dev] Donwloading latest wso2cep 4.2.0 release

[Aneela Safdar]

Hi,
I am trying to download new release of wso2cep but on download page it is still 
giving me the older version.
Any suggestion?


 Regards, Aneela Safdar___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Dev] [CDMF] [EMM 2.1.0 Beta] Refactoring UI controller modules

[Dilan Udara Ariyaratne]

Hi All,

As part of EMM 2.1.0 Release, we are currently in the process of migrating
its UI from UUF 0.1 to
UUF 0.2, which is a much-optimized version of UUF in terms of performance.

As part of the same migration effort, many UI controller modules @
org.wso2.carbon.device.mgt.ui component layer
were subjected to update by their names and level of organization, apart
from any code-level improvements.

This mail is to update all on the corresponding changes.

Figure 1 shows the original organization of these modules in a flat
hierarchy with less emphasize
on accurately showing their correct context of relevance and intended
functionality.






*Figure 1: Original Organization of UI Controller modules- - - - - - - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -*
​
Following updates (see figure 2) have been made to most of these modules in
terms of their names and
the level of organization to overcome the mentioned issue and improve
clarity.

[1] pinch.min.js @ modules and mdm-props.js @ conf were brought into one
module
called "conf-reader" and mdm-props.js was renamed to main.js :
"conf-reader" module can be used to read application specific configuration
details @ conf/config.json

[2] api-wrapper-util.js which contained all the main OAuth token handling
functions was renamed to "token-handlers.js".
util.js which contained all the supportive functions for OAuth token
handling @ "token-handlers.js" was renamed to "token-handler-utils.js".
backend-service-invokers.js which contained all the utility functions for
invoking token-protected-backend-services was renamed to
"token-protected-service-invokers.js".
Since all these modules are related to OAuth, they were grouped as one
"oauth" module.

[3] device.js, group.js, operation.js, policy.js and user.js which
contained all the utility functions for manipulating device-mgt specific
business data
from backend services for the use of UI were grouped in as one set of
controllers called "business-controllers".

[4] invoker-request-wrapper.js and user-info.js were made obsolete and
removed as they had no usage at all by dependent sources.



​
*Figure 2: New Organization of UI Controller modules*

*- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - -*
In the meantime, rest of the modules, namely constants.js, init.js,
login.js and utility.js are still under
consideration for any improvement or refactoring as the next part of this
work.

Cheers,
Dilan.

*Dilan U. Ariyaratne*
Senior Software Engineer
WSO2 Inc. 
Mobile: +94766405580 <%2B94766405580>
lean . enterprise . middleware
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev