[Dev] Resource level monitoring dashboard for applications on AppCloud

[Malmee Weerasinghe]

Hi All,
I am working on building a dashboard for resource level monitoring of the
pods in kubernetes used by AppCloud.

In kubernetes this is the structure used for resource usage
monitoring. cAdvisor
auto-discovers all containers in a node and collects CPU, memory,
filesystem, and network usage statistics while Heapster queries all these
data from the nodes through Kubelet and Kubelet gives the data fetched from
the cAdvisor.

I have studied on connecting to the cAdvisor and the cAdvisor UI which
gives all the resource usage data of containers in a node graphically. But
accessing these data through Heapster would be a better approach as
suggested by Manjula than accessing a node seperately.

I am studing on how to access these data through Heapster and how to pump
the data received from cAdvisor API to the dashboard that is to be
implemented.

This is the approach that I will be taking and would highly appreciate your
suggestions on this.


​
Thank you
-- 
Malmee Weerasinghe
WSO2 Intern
mobile : (+94)* 71 7601905* |   email :   
mal...@wso2.com
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] Streaming File transfer with File Inbound/File Connector

[Malaka Silva]

Hi Vivekananthan,

Can we create a Jira and follow up on this please?

On Mon, Mar 14, 2016 at 10:40 AM, Kasun Indrasiri  wrote:

>
>
> On Mon, Mar 14, 2016 at 9:57 AM, Malaka Silva  wrote:
>
>> Hi Kasun,
>>
>> I don't think we have the same functionality of VFS sender in file
>> connector. We have only focused on use cases not covered with VFS transport.
>>
>
> Ok. I think we need to make sure things are consistent across the
> transport and the file connector. Can we please include this as an
> improvement for the next version of the connector.
>
>
>>
> @Vivekananthan - Please confirm this?
>>
>> On Sat, Mar 12, 2016 at 4:39 PM, Kasun Indrasiri  wrote:
>>
>>> This is great. Thanks a lot Malaka.
>>> Also, if we use the file connector as the outbound channel, will it work
>>> in the same way?
>>>
>>> On Tue, Mar 1, 2016 at 12:36 PM, Malaka Silva  wrote:
>>>
 Hi All,

 Please ignore my previous comments. This can be done with current
 implementation.

 if (builder instanceof DataSourceMessageBuilder &&
 "true".equals(streaming)) {
 dataSource = ManagedDataSourceFactory.create(new
 FileObjectDataSource(file, contentType));
 in = null;
 } else {
 in = new
 AutoCloseInputStream(file.getContent().getInputStream());
 dataSource = null;
 }
 ..
 OMElement documentElement;
 if (in != null) {
 documentElement = builder.processDocument(in,
 contentType, axis2MsgCtx);
 } else {
 documentElement =
 ((DataSourceMessageBuilder)builder).processDocument(dataSource,
 contentType, axis2MsgCtx);
 }
 ...
 if(dataSource != null) {
 dataSource.destroy();
 }

 This is
 ​because ​
 message builder able to build messages from
 ​​
 DataSource objects.
 ​
 ​
 DataSource​
  by definition the data from a DataSource can be read
 ​
 multiple times,
 ​builders that implement ​
 this interface
 ​can​
  avoid storing the message content in memory.
 ​ ​
 If a message builder implements this interface and the
 ​file/vfs
  is able to provide the message payload as a data source, then the
 method defined by this interface should be preferred over the method
 defined by Builder.
 ​ This
  helps optimizing
 ​PT​
  with
 ​vfs/file​.
 The builder will typically expose the data source directly or
 indirectly through the returned OMElement, e.g. by adding to the tree an
 OMText or OMDataSource node referencing the data source.

 ​I have checked this with inbound but there is a fix we need to do. I
 have done it in [1]. I have done several tests with and without streaming
 from 5mb to 1gb files.

 Without streaming [2] and [3] will show the memory growth. With
 streaming [4]. Without streaming most of the time ESB went OOM​.

 Find the related configs and axis2 changes in [5] and [6].

 [1] https://wso2.org/jira/browse/ESBJAVA-4458

 [2]

 [image: Inline image 1]

 [3]

 [image: Inline image 2]

 [4]

 [image: Inline image 3]

 [5]

 http://ws.apache.org/ns/synapse;  name="load"
  sequence="request"  onError="fault" protocol="file" suspend="false">

   1
   true
   >>> name="transport.vfs.ContentType">application/file
   >>> name="transport.vfs.LockReleaseSameNode">false
   false
   >>> name="transport.vfs.ActionAfterFailure">DELETE
   true
   true
   >>> name="transport.vfs.ActionAfterProcess">DELETE
   >>> name="transport.vfs.FileURI">file:///home/wso2/work/tmp/file/in
   false
   true
   enable
   true
   NONE
   false

 

 
 
 
 
 >>> value="true"/>
 
 
 
 

 [6]
 >>> class="org.apache.axis2.format.BinaryBuilder"/>

 >>>
 class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>

 On Mon, Feb 29, 2016 at 12:02 PM, Malaka Silva  wrote:

> Hi Kasun,
>
> Currently no OOB solution with ESB 4.9.0. File always gets build
> before mediation.
>
> However use case mentioned can be handled with Schedule task -> File
> connector search -> File connector copy
>
> +1 for OOB solution.
>
> We can  do this for ESB 5.0. Also there is an option to do this as a
> custom inbound. So that this can be used by previous ESB versions as well.

[Dev] Fwd: DAS analyticApi gives this error

[Dinali Dabarera]

-- Forwarded message --
From: Dinali Dabarera 
Date: Sun, Mar 13, 2016 at 12:57 PM
Subject: DAS analyticApi gives this error
To: Anjana Fernando 


Hi,

we get this error at das,
Can not figureout what this tells, Can you help us finding solutions for
this? Can you tell why this happens?



[2016-03-13 07:20:40,801] ERROR
{org.apache.catalina.core.StandardWrapperValve} -  Servlet.service() for
servlet [cxf] in context with path [/analytics] threw exception
java.lang.RuntimeException: org.apache.cxf.interceptor.Fault:
org/wso2/carbon/analytics/api/AnalyticsDataAPI
at
org.apache.cxf.interceptor.AbstractFaultChainInitiatorObserver.onMessage(AbstractFaultChainInitiatorObserver.java:116)
at
org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:336)
at
org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
at
org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:249)
at
org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:248)
at
org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:222)
at
org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:153)
at
org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:171)
at
org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:289)
at
org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:214)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
at
org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:265)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:613)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at
org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:99)
at
org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at
org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:57)
at
org.wso2.carbon.event.receiver.core.internal.tenantmgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:48)
at
org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at
org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:62)
at
org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:159)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
at
org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1739)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1698)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.cxf.interceptor.Fault:
org/wso2/carbon/analytics/api/AnalyticsDataAPI
at
org.apache.cxf.service.invoker.AbstractInvoker.createFault(AbstractInvoker.java:170)
at
org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:136)
at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:204)
at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:101)
at

Re: [Dev] Streaming File transfer with File Inbound/File Connector

[Kasun Indrasiri]

On Mon, Mar 14, 2016 at 9:57 AM, Malaka Silva  wrote:

> Hi Kasun,
>
> I don't think we have the same functionality of VFS sender in file
> connector. We have only focused on use cases not covered with VFS transport.
>

Ok. I think we need to make sure things are consistent across the transport
and the file connector. Can we please include this as an improvement for
the next version of the connector.


>
@Vivekananthan - Please confirm this?
>
> On Sat, Mar 12, 2016 at 4:39 PM, Kasun Indrasiri  wrote:
>
>> This is great. Thanks a lot Malaka.
>> Also, if we use the file connector as the outbound channel, will it work
>> in the same way?
>>
>> On Tue, Mar 1, 2016 at 12:36 PM, Malaka Silva  wrote:
>>
>>> Hi All,
>>>
>>> Please ignore my previous comments. This can be done with current
>>> implementation.
>>>
>>> if (builder instanceof DataSourceMessageBuilder &&
>>> "true".equals(streaming)) {
>>> dataSource = ManagedDataSourceFactory.create(new
>>> FileObjectDataSource(file, contentType));
>>> in = null;
>>> } else {
>>> in = new
>>> AutoCloseInputStream(file.getContent().getInputStream());
>>> dataSource = null;
>>> }
>>> ..
>>> OMElement documentElement;
>>> if (in != null) {
>>> documentElement = builder.processDocument(in,
>>> contentType, axis2MsgCtx);
>>> } else {
>>> documentElement =
>>> ((DataSourceMessageBuilder)builder).processDocument(dataSource,
>>> contentType, axis2MsgCtx);
>>> }
>>> ...
>>> if(dataSource != null) {
>>> dataSource.destroy();
>>> }
>>>
>>> This is
>>> ​because ​
>>> message builder able to build messages from
>>> ​​
>>> DataSource objects.
>>> ​
>>> ​
>>> DataSource​
>>>  by definition the data from a DataSource can be read
>>> ​
>>> multiple times,
>>> ​builders that implement ​
>>> this interface
>>> ​can​
>>>  avoid storing the message content in memory.
>>> ​ ​
>>> If a message builder implements this interface and the
>>> ​file/vfs
>>>  is able to provide the message payload as a data source, then the
>>> method defined by this interface should be preferred over the method
>>> defined by Builder.
>>> ​ This
>>>  helps optimizing
>>> ​PT​
>>>  with
>>> ​vfs/file​.
>>> The builder will typically expose the data source directly or indirectly
>>> through the returned OMElement, e.g. by adding to the tree an OMText or
>>> OMDataSource node referencing the data source.
>>>
>>> ​I have checked this with inbound but there is a fix we need to do. I
>>> have done it in [1]. I have done several tests with and without streaming
>>> from 5mb to 1gb files.
>>>
>>> Without streaming [2] and [3] will show the memory growth. With
>>> streaming [4]. Without streaming most of the time ESB went OOM​.
>>>
>>> Find the related configs and axis2 changes in [5] and [6].
>>>
>>> [1] https://wso2.org/jira/browse/ESBJAVA-4458
>>>
>>> [2]
>>>
>>> [image: Inline image 1]
>>>
>>> [3]
>>>
>>> [image: Inline image 2]
>>>
>>> [4]
>>>
>>> [image: Inline image 3]
>>>
>>> [5]
>>>
>>> http://ws.apache.org/ns/synapse;  name="load"
>>>  sequence="request"  onError="fault" protocol="file" suspend="false">
>>>
>>>   1
>>>   true
>>>   >> name="transport.vfs.ContentType">application/file
>>>   >> name="transport.vfs.LockReleaseSameNode">false
>>>   false
>>>   >> name="transport.vfs.ActionAfterFailure">DELETE
>>>   true
>>>   true
>>>   >> name="transport.vfs.ActionAfterProcess">DELETE
>>>   >> name="transport.vfs.FileURI">file:///home/wso2/work/tmp/file/in
>>>   false
>>>   true
>>>   enable
>>>   true
>>>   NONE
>>>   false
>>>
>>> 
>>>
>>> 
>>> 
>>> 
>>> 
>>> >> value="true"/>
>>> 
>>> 
>>> 
>>> 
>>>
>>> [6]
>>> >> class="org.apache.axis2.format.BinaryBuilder"/>
>>>
>>> >>
>>> class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>
>>>
>>> On Mon, Feb 29, 2016 at 12:02 PM, Malaka Silva  wrote:
>>>
 Hi Kasun,

 Currently no OOB solution with ESB 4.9.0. File always gets build before
 mediation.

 However use case mentioned can be handled with Schedule task -> File
 connector search -> File connector copy

 +1 for OOB solution.

 We can  do this for ESB 5.0. Also there is an option to do this as a
 custom inbound. So that this can be used by previous ESB versions as well.

 So default inbound can be used, if someone needs to do a mediation on
 file content and custom inbound for PT file use case. WDYT?


 On Mon, Feb 29, 2016 at 11:49 AM, Kasun Indrasiri 
 wrote:

> Hi Malaka,
>
> Do we support the $subject? Basically we use an Inbound 

Re: [Dev] Configuring load balancing in app cloud with HA Proxy

[Nishadi Kirielle]

Hi all,
+1 for going with SSL pass through approach. Once the testing with staging
is done, I will focus on this approach.

Thanks

On Mon, Mar 14, 2016 at 10:29 AM, Manjula Rathnayake 
wrote:

> Hi Imesh,
>
> On Mon, Mar 14, 2016 at 10:20 AM, Imesh Gunaratne  wrote:
>
>> Hi Manjula,
>>
>> On Mon, Mar 14, 2016 at 10:06 AM, Manjula Rathnayake 
>> wrote:
>>
>>> Hi Imesh,
>>>
>>> On Mon, Mar 14, 2016 at 9:56 AM, Imesh Gunaratne  wrote:
>>>

 On Sun, Mar 13, 2016 at 11:36 PM, Nishadi Kirielle 
 wrote:

> Hi all,
> Currently I'm working on configuring HAProxy load balancing support
> for app cloud.
> In checking the session affinity functionality in kuberenetes, I have
> verified the load balancing of http traffic with HAProxy. It could be done
> using kubernetes contribution repo, 'service loadbalancer' [1].
>
> In order to check the load balancing with https traffic the taken
> approach is SSL termination.In the scenario of app cloud, kubernetes
> cluster is not directly exposed and the load balancer exists within the
> cluster. Thus the communication between the application servers and the
> load balancer happens internally. Although SSL termination ends the secure
> connection at the load balancer, due to the above mentioned reasons, SSL
> termination seems to be a better solution. The reason for the use of SSL
> termination over SSL pass through is because of the complexity of handling
> a separate SSL certificate for each server behind the load balancer in the
> case of SSL pass through.
>
> -1 for this approach, IMO this has a major security risk.

 Let me explain the problem. If we offload SSL at the service load
 balancer, all traffic beyond the load balancer will use HTTP and the
 message content will be visible to anyone on network inside K8S. Which
 means someone can simply start a container in K8S and trace all HTTP
 traffic going through.

>>>
>>
>>> Below is from HA Proxy documentation[1]. AFAIU, HA Proxy to backend
>>> server communication happens with HTTPS enabled but not validating the
>>> server certificate.
>>>
>>
>>
>>> verify
>>> 
>>> [none|required]
>>>
>>> This setting is only available when support for OpenSSL was built in. If set
>>> to 'none', server certificate is not verified. In the other case, The
>>> certificate provided by the server is verified using CAs from 'ca-file'
>>> and optional CRLs from 'crl-file'. If 'ssl_server_verify' is not specified
>>> in global  section, this is the default. On verify failure the handshake
>>> is aborted. It is critically important to verify server certificates when
>>> using SSL to connect to servers, otherwise the communication is prone to
>>> trivial man-in-the-middle attacks rendering SSL totally useless.
>>>
>>> IMO still there is a major problem if we are not verifying the SSL
>> certificate. See the highlighted text.
>>
> +1. We will attend to this once initial end to end scenario got working in
> App Cloud. I am +1 for using a self signed cert in pods and adding it to
> truststore of HA Proxy to fix above issue.
>
> thank you.
>
>>
>> Thanks
>>
>>
>>> [1].
>>> http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#ssl%20%28Server%20and%20default-server%20options%29
>>>
>>> thank you.
>>>
>>>
 Thanks

 In configuring load balancing with SSL termination, I had to customize
> kubernetes haproxy.conf file template of service loadbalancer repo to
> support SSL termination.
>
> In order to provide SSL termination, the kubernetes services have to
> be annotated with
>   serviceloadbalancer/lb.sslTerm: "true"
>
> The default approach in load balancing with service load balancer repo
> is based on simple fan out approach which uses context path to load 
> balance
> the traffic. As we need to load balance based on the host name, we need to
> go with the name based virtual hosting approach. It can be achieved via 
> the
> following annotation.
>  serviceloadbalancer/lb.Host: ""
>
> Any suggestions on the approach taken are highly appreciated.
>
> Thank you
>
> [1].
> https://github.com/kubernetes/contrib/tree/master/service-loadbalancer
>
>



 --
 *Imesh Gunaratne*
 Senior Technical Lead
 WSO2 Inc: http://wso2.com
 T: +94 11 214 5345 M: +94 77 374 2057
 W: http://imesh.io
 Lean . Enterprise . Middleware


>>>
>>>
>>> --
>>> Manjula Rathnayaka
>>> Associate Technical Lead
>>> WSO2, Inc.
>>> Mobile:+94 77 743 1987
>>>
>>
>>
>>
>> --
>> *Imesh Gunaratne*
>> Senior Technical Lead
>> WSO2 Inc: http://wso2.com
>> T: +94 11 214 5345 M: +94 77 374 2057
>> W: http://imesh.io
>> Lean . Enterprise . Middleware
>>

Re: [Dev] Session Affinity in Kubernetes

[Nishadi Kirielle]

+1 for the proposed approach. I will stick to the other thread.

Thanks


On Mon, Mar 14, 2016 at 10:00 AM, Imesh Gunaratne  wrote:

>
>
> On Sun, Mar 13, 2016 at 11:37 PM, Nishadi Kirielle 
> wrote:
>
>> Hi Imesh,
>> The reason for choosing SSL termination over SSL pass through is due to
>> the complexity of handling separate SSL certificates for each servers
>> behind the load balancer in kubernetes cluster. As in App Cloud the
>> kubernetes cluster is not direcctly exposed and the communication between
>> the load balancer and servers happens internally, we have thought of
>> choosing SSL termination approach over SSL pass through.
>>
>
> -1 There is no way we can assume that internal networks are secure. Please
> refer the other thread "Configuring load balancing in app cloud with HA
> Proxy".
>
> FYI: I see two threads on the same topic. It might be better to keep them
> in one.
>
> Thanks
>
>>
>>
>> Thanks
>>
>> On Thu, Mar 10, 2016 at 11:21 PM, Imesh Gunaratne  wrote:
>>
>>>
>>>
>>> On Thu, Mar 10, 2016 at 1:28 PM, Nishadi Kirielle 
>>> wrote:
>>>
 Thank you for the suggestion of using the default self signed
 certificate.
 I have attempted SSL termination approach of terminating the SSL
 connection at the load balancer and sending unencrypted connections to the
 backend server via the ha proxy configuration of 'ssl verify none'. This
 approach allows https traffic to be load balanced and exposed.

 Terminating SSL at the middle of a communication flow would introduce
>>> security risks.
>>>
>>> Thanks
>>>
>>>
 Thanks


 On Thu, Mar 10, 2016 at 11:20 AM, Imesh Gunaratne 
 wrote:

>
>
> On Thu, Mar 10, 2016 at 10:49 AM, Nishadi Kirielle 
> wrote:
>
>> Hi all,
>> I have only tested for http traffic earlier. Although the kubernetes
>> service loadbalancer template has support for https, when I have deployed
>> an application ( dell/tomcat ) which has the support for https, the ha
>> proxy load balancer did not identify it as a https service in the haproxy
>> configuration file. It just identified the application as a http
>> application and updated the configuration file accordingly.
>>
>
> Yes, in our K8S services we have defined the protocol as TCP, not as
> HTTPS/SSL. Therefore there is no way for the service load balancer to find
> this information by looking at the services.
>
>
>> Thus I have manually altered the ha proxy configuration file to
>> support for https traffic with a self signed certificate specific for the
>> node ip. But it fails in accessing the application, since the application
>> needs the self signed certificate specific to the application.
>> As a solution for this I'm trying with bind option 'cert' to bind
>> several certificate files[2] of the specific applications.
>>
>
> Shall we try with the default self signed certificate distributed with
> a WSO2 product?
>
> Thanks
>
>>
>> Any suggestions on this are highly appreciated.
>> [1] .
>> https://github.com/kubernetes/contrib/tree/master/service-loadbalancer
>> [2] .
>> https://cbonte.github.io/haproxy-dconv/configuration-1.5.html#5.1-crt
>>
>> Thanks
>>
>> On Wed, Mar 9, 2016 at 10:33 AM, Imesh Gunaratne 
>> wrote:
>>
>>> Hi Deep,
>>>
>>> On Tue, Mar 8, 2016 at 8:08 PM, Deependra Ariyadewa 
>>> wrote:
>>>

 On Mon, Mar 7, 2016 at 10:30 AM, Nishadi Kirielle  wrote:

> Hi All,
> I have written the blog post on load balancing and session
> affinity in kubernetes. [1]
>

 I am going test session affinity for HTTPS triffic in Kubernetes
 following your configurations. Did you try to enable session affinity 
 for
 HTTPS triffic in Kubernetes.

 We would need to configure haproxy with relevant SSL certificates
>>> for HTTPS to work. I do not think we tested it. See [1] for the haproxy
>>> config template used by the service load balancer. This will get 
>>> packaged
>>> to the Docker service load balancer Docker image [2].
>>>
>>> [1]
>>> https://github.com/kubernetes/contrib/blob/master/service-loadbalancer/template.cfg
>>> [2]
>>> https://github.com/kubernetes/contrib/blob/master/service-loadbalancer/Dockerfile
>>>
>>> Thanks
>>>
>>>
 Thanks,
 Deependra.

>
> Thank you
>
> [1].
> http://nishadikirielle.blogspot.com/2016/03/load-balancing-kubernetes-services-and.html
>
> On Fri, Mar 4, 2016 at 8:22 PM, Nishadi Kirielle  > wrote:
>
>> Thanks a 

Re: [Dev] Configuring load balancing in app cloud with HA Proxy

[Manjula Rathnayake]

Hi Imesh,

On Mon, Mar 14, 2016 at 10:20 AM, Imesh Gunaratne  wrote:

> Hi Manjula,
>
> On Mon, Mar 14, 2016 at 10:06 AM, Manjula Rathnayake 
> wrote:
>
>> Hi Imesh,
>>
>> On Mon, Mar 14, 2016 at 9:56 AM, Imesh Gunaratne  wrote:
>>
>>>
>>> On Sun, Mar 13, 2016 at 11:36 PM, Nishadi Kirielle 
>>> wrote:
>>>
 Hi all,
 Currently I'm working on configuring HAProxy load balancing support for
 app cloud.
 In checking the session affinity functionality in kuberenetes, I have
 verified the load balancing of http traffic with HAProxy. It could be done
 using kubernetes contribution repo, 'service loadbalancer' [1].

 In order to check the load balancing with https traffic the taken
 approach is SSL termination.In the scenario of app cloud, kubernetes
 cluster is not directly exposed and the load balancer exists within the
 cluster. Thus the communication between the application servers and the
 load balancer happens internally. Although SSL termination ends the secure
 connection at the load balancer, due to the above mentioned reasons, SSL
 termination seems to be a better solution. The reason for the use of SSL
 termination over SSL pass through is because of the complexity of handling
 a separate SSL certificate for each server behind the load balancer in the
 case of SSL pass through.

 -1 for this approach, IMO this has a major security risk.
>>>
>>> Let me explain the problem. If we offload SSL at the service load
>>> balancer, all traffic beyond the load balancer will use HTTP and the
>>> message content will be visible to anyone on network inside K8S. Which
>>> means someone can simply start a container in K8S and trace all HTTP
>>> traffic going through.
>>>
>>
>
>> Below is from HA Proxy documentation[1]. AFAIU, HA Proxy to backend
>> server communication happens with HTTPS enabled but not validating the
>> server certificate.
>>
>
>
>> verify
>> 
>> [none|required]
>>
>> This setting is only available when support for OpenSSL was built in. If set
>> to 'none', server certificate is not verified. In the other case, The
>> certificate provided by the server is verified using CAs from 'ca-file'
>> and optional CRLs from 'crl-file'. If 'ssl_server_verify' is not specified
>> in global  section, this is the default. On verify failure the handshake
>> is aborted. It is critically important to verify server certificates when
>> using SSL to connect to servers, otherwise the communication is prone to
>> trivial man-in-the-middle attacks rendering SSL totally useless.
>>
>> IMO still there is a major problem if we are not verifying the SSL
> certificate. See the highlighted text.
>
+1. We will attend to this once initial end to end scenario got working in
App Cloud. I am +1 for using a self signed cert in pods and adding it to
truststore of HA Proxy to fix above issue.

thank you.

>
> Thanks
>
>
>> [1].
>> http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#ssl%20%28Server%20and%20default-server%20options%29
>>
>> thank you.
>>
>>
>>> Thanks
>>>
>>> In configuring load balancing with SSL termination, I had to customize
 kubernetes haproxy.conf file template of service loadbalancer repo to
 support SSL termination.

 In order to provide SSL termination, the kubernetes services have to be
 annotated with
   serviceloadbalancer/lb.sslTerm: "true"

 The default approach in load balancing with service load balancer repo
 is based on simple fan out approach which uses context path to load balance
 the traffic. As we need to load balance based on the host name, we need to
 go with the name based virtual hosting approach. It can be achieved via the
 following annotation.
  serviceloadbalancer/lb.Host: ""

 Any suggestions on the approach taken are highly appreciated.

 Thank you

 [1].
 https://github.com/kubernetes/contrib/tree/master/service-loadbalancer


>>>
>>>
>>>
>>> --
>>> *Imesh Gunaratne*
>>> Senior Technical Lead
>>> WSO2 Inc: http://wso2.com
>>> T: +94 11 214 5345 M: +94 77 374 2057
>>> W: http://imesh.io
>>> Lean . Enterprise . Middleware
>>>
>>>
>>
>>
>> --
>> Manjula Rathnayaka
>> Associate Technical Lead
>> WSO2, Inc.
>> Mobile:+94 77 743 1987
>>
>
>
>
> --
> *Imesh Gunaratne*
> Senior Technical Lead
> WSO2 Inc: http://wso2.com
> T: +94 11 214 5345 M: +94 77 374 2057
> W: http://imesh.io
> Lean . Enterprise . Middleware
>
>


-- 
Manjula Rathnayaka
Associate Technical Lead
WSO2, Inc.
Mobile:+94 77 743 1987
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [Architecture] [DEV] WSO2 App Manager 1.2.0 Milestone 2 Released

[Dinusha Senanayaka]

Hi Susinda,

On Mon, Mar 14, 2016 at 10:22 AM, Susinda Perera  wrote:

> Hi Lahiru
>
> Is there tooling requirement for AppManager? If so shall we have a chat
> and build a tooling plan.
>

We haven't plan tooling for AppManager immediate release (1.2.0). But, yes
we could have a chat and include it for next release.

Regards,
Dinusha.


>
> Thanks
> Susinda
>
>
> On Thu, Mar 10, 2016 at 1:18 AM, Lahiru Cooray  wrote:
>
>> Hi All,
>>
>> WSO2 App Manager team pleased to announce the WSO2 APP Manager 1.2.0 -
>> Milestone 2 release. It contains following new features, improvements and
>> bug fixes.
>>
>> You can download this distribution from below link.
>> *http://builder1.us1.wso2.org/~appm/release-1.2.0/M2/wso2appm-1.2.0-M2.zip
>> *
>>
>>
>> *New Features *
>> 1. New asset type - sites.
>> 2. Configurable subscription option for Web App and Sites asset types.
>> 3. Multiple version support for Web App and Sites  asset types.
>> 4. Java APIs for all key App Manager functionalities that need to be
>> integrated with device management functionalities.
>> 5. Role based visibility control for mobile apps.
>>
>>
>>
>> *Improvements*1. Navigation improvements in App Store.
>> 2. Add missing functionalities to Store/Publisher REST API and
>> improvements.
>> 3. Responsive store UI by Boostrap3 upgrade.
>> 4. New theme for store UI
>>
>> *Bug Fixes*
>> WSO2 App Manager 1.2.0-M2 resolved issues
>> 
>>
>>
>> Regards,
>> App Manager Team.
>>
>> --
>> *Lahiru Cooray*
>> Software Engineer
>> WSO2, Inc.;http://wso2.com/
>> lean.enterprise.middleware
>>
>> Mobile: +94 715 654154
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "WSO2 Engineering Group" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to engineering-group+unsubscr...@wso2.com.
>> For more options, visit https://groups.google.com/a/wso2.com/d/optout.
>>
>
>
>
> --
> *Susinda Perera*
> Software Engineer
> B.Sc.(Eng), M.Sc(Computer Science), AMIE(SL)
> Mobile:(+94)716049075
> Blog: susinda.blogspot.com
> WSO2 Inc. http://wso2.com/
> Tel : 94 11 214 5345 Fax :94 11 2145300
>
>
> ___
> Architecture mailing list
> architect...@wso2.org
> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>
>


-- 
Dinusha Dilrukshi
Associate Technical Lead
WSO2 Inc.: http://wso2.com/
Mobile: +94725255071
Blog: http://dinushasblog.blogspot.com/
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [DEV] WSO2 App Manager 1.2.0 Milestone 2 Released

[Susinda Perera]

Hi Lahiru

Is there tooling requirement for AppManager? If so shall we have a chat and
build a tooling plan.

Thanks
Susinda


On Thu, Mar 10, 2016 at 1:18 AM, Lahiru Cooray  wrote:

> Hi All,
>
> WSO2 App Manager team pleased to announce the WSO2 APP Manager 1.2.0 -
> Milestone 2 release. It contains following new features, improvements and
> bug fixes.
>
> You can download this distribution from below link.
> *http://builder1.us1.wso2.org/~appm/release-1.2.0/M2/wso2appm-1.2.0-M2.zip
> *
>
>
> *New Features *
> 1. New asset type - sites.
> 2. Configurable subscription option for Web App and Sites asset types.
> 3. Multiple version support for Web App and Sites  asset types.
> 4. Java APIs for all key App Manager functionalities that need to be
> integrated with device management functionalities.
> 5. Role based visibility control for mobile apps.
>
>
>
> *Improvements*1. Navigation improvements in App Store.
> 2. Add missing functionalities to Store/Publisher REST API and
> improvements.
> 3. Responsive store UI by Boostrap3 upgrade.
> 4. New theme for store UI
>
> *Bug Fixes*
> WSO2 App Manager 1.2.0-M2 resolved issues
> 
>
>
> Regards,
> App Manager Team.
>
> --
> *Lahiru Cooray*
> Software Engineer
> WSO2, Inc.;http://wso2.com/
> lean.enterprise.middleware
>
> Mobile: +94 715 654154
>
> --
> You received this message because you are subscribed to the Google Groups
> "WSO2 Engineering Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to engineering-group+unsubscr...@wso2.com.
> For more options, visit https://groups.google.com/a/wso2.com/d/optout.
>



-- 
*Susinda Perera*
Software Engineer
B.Sc.(Eng), M.Sc(Computer Science), AMIE(SL)
Mobile:(+94)716049075
Blog: susinda.blogspot.com
WSO2 Inc. http://wso2.com/
Tel : 94 11 214 5345 Fax :94 11 2145300
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] Configuring load balancing in app cloud with HA Proxy

[Imesh Gunaratne]

Hi Manjula,

On Mon, Mar 14, 2016 at 10:06 AM, Manjula Rathnayake 
wrote:

> Hi Imesh,
>
> On Mon, Mar 14, 2016 at 9:56 AM, Imesh Gunaratne  wrote:
>
>>
>> On Sun, Mar 13, 2016 at 11:36 PM, Nishadi Kirielle 
>> wrote:
>>
>>> Hi all,
>>> Currently I'm working on configuring HAProxy load balancing support for
>>> app cloud.
>>> In checking the session affinity functionality in kuberenetes, I have
>>> verified the load balancing of http traffic with HAProxy. It could be done
>>> using kubernetes contribution repo, 'service loadbalancer' [1].
>>>
>>> In order to check the load balancing with https traffic the taken
>>> approach is SSL termination.In the scenario of app cloud, kubernetes
>>> cluster is not directly exposed and the load balancer exists within the
>>> cluster. Thus the communication between the application servers and the
>>> load balancer happens internally. Although SSL termination ends the secure
>>> connection at the load balancer, due to the above mentioned reasons, SSL
>>> termination seems to be a better solution. The reason for the use of SSL
>>> termination over SSL pass through is because of the complexity of handling
>>> a separate SSL certificate for each server behind the load balancer in the
>>> case of SSL pass through.
>>>
>>> -1 for this approach, IMO this has a major security risk.
>>
>> Let me explain the problem. If we offload SSL at the service load
>> balancer, all traffic beyond the load balancer will use HTTP and the
>> message content will be visible to anyone on network inside K8S. Which
>> means someone can simply start a container in K8S and trace all HTTP
>> traffic going through.
>>
>

> Below is from HA Proxy documentation[1]. AFAIU, HA Proxy to backend server
> communication happens with HTTPS enabled but not validating the server
> certificate.
>


> verify
> 
> [none|required]
>
> This setting is only available when support for OpenSSL was built in. If set
> to 'none', server certificate is not verified. In the other case, The
> certificate provided by the server is verified using CAs from 'ca-file'
> and optional CRLs from 'crl-file'. If 'ssl_server_verify' is not specified
> in global  section, this is the default. On verify failure the handshake
> is aborted. It is critically important to verify server certificates when
> using SSL to connect to servers, otherwise the communication is prone to
> trivial man-in-the-middle attacks rendering SSL totally useless.
>
> IMO still there is a major problem if we are not verifying the SSL
certificate. See the highlighted text.

Thanks


> [1].
> http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#ssl%20%28Server%20and%20default-server%20options%29
>
> thank you.
>
>
>> Thanks
>>
>> In configuring load balancing with SSL termination, I had to customize
>>> kubernetes haproxy.conf file template of service loadbalancer repo to
>>> support SSL termination.
>>>
>>> In order to provide SSL termination, the kubernetes services have to be
>>> annotated with
>>>   serviceloadbalancer/lb.sslTerm: "true"
>>>
>>> The default approach in load balancing with service load balancer repo
>>> is based on simple fan out approach which uses context path to load balance
>>> the traffic. As we need to load balance based on the host name, we need to
>>> go with the name based virtual hosting approach. It can be achieved via the
>>> following annotation.
>>>  serviceloadbalancer/lb.Host: ""
>>>
>>> Any suggestions on the approach taken are highly appreciated.
>>>
>>> Thank you
>>>
>>> [1].
>>> https://github.com/kubernetes/contrib/tree/master/service-loadbalancer
>>>
>>>
>>
>>
>>
>> --
>> *Imesh Gunaratne*
>> Senior Technical Lead
>> WSO2 Inc: http://wso2.com
>> T: +94 11 214 5345 M: +94 77 374 2057
>> W: http://imesh.io
>> Lean . Enterprise . Middleware
>>
>>
>
>
> --
> Manjula Rathnayaka
> Associate Technical Lead
> WSO2, Inc.
> Mobile:+94 77 743 1987
>



-- 
*Imesh Gunaratne*
Senior Technical Lead
WSO2 Inc: http://wso2.com
T: +94 11 214 5345 M: +94 77 374 2057
W: http://imesh.io
Lean . Enterprise . Middleware
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] Streaming File transfer with File Inbound/File Connector

[Vivekananthan Sivanayagam]

Hi,

We don't have the same functionality of VFS sender in File Connector.




*Thank youVivekananthan Sivanayagam*

*Associate Software Engineer | WSO2*

*E:vivekanant...@wso2.com *
*M:+94752786138*

On Mon, Mar 14, 2016 at 9:57 AM, Malaka Silva  wrote:

> Hi Kasun,
>
> I don't think we have the same functionality of VFS sender in file
> connector. We have only focused on use cases not covered with VFS transport.
>
> @Vivekananthan - Please confirm this?
>
> On Sat, Mar 12, 2016 at 4:39 PM, Kasun Indrasiri  wrote:
>
>> This is great. Thanks a lot Malaka.
>> Also, if we use the file connector as the outbound channel, will it work
>> in the same way?
>>
>> On Tue, Mar 1, 2016 at 12:36 PM, Malaka Silva  wrote:
>>
>>> Hi All,
>>>
>>> Please ignore my previous comments. This can be done with current
>>> implementation.
>>>
>>> if (builder instanceof DataSourceMessageBuilder &&
>>> "true".equals(streaming)) {
>>> dataSource = ManagedDataSourceFactory.create(new
>>> FileObjectDataSource(file, contentType));
>>> in = null;
>>> } else {
>>> in = new
>>> AutoCloseInputStream(file.getContent().getInputStream());
>>> dataSource = null;
>>> }
>>> ..
>>> OMElement documentElement;
>>> if (in != null) {
>>> documentElement = builder.processDocument(in,
>>> contentType, axis2MsgCtx);
>>> } else {
>>> documentElement =
>>> ((DataSourceMessageBuilder)builder).processDocument(dataSource,
>>> contentType, axis2MsgCtx);
>>> }
>>> ...
>>> if(dataSource != null) {
>>> dataSource.destroy();
>>> }
>>>
>>> This is
>>> ​because ​
>>> message builder able to build messages from
>>> ​​
>>> DataSource objects.
>>> ​
>>> ​
>>> DataSource​
>>>  by definition the data from a DataSource can be read
>>> ​
>>> multiple times,
>>> ​builders that implement ​
>>> this interface
>>> ​can​
>>>  avoid storing the message content in memory.
>>> ​ ​
>>> If a message builder implements this interface and the
>>> ​file/vfs
>>>  is able to provide the message payload as a data source, then the
>>> method defined by this interface should be preferred over the method
>>> defined by Builder.
>>> ​ This
>>>  helps optimizing
>>> ​PT​
>>>  with
>>> ​vfs/file​.
>>> The builder will typically expose the data source directly or indirectly
>>> through the returned OMElement, e.g. by adding to the tree an OMText or
>>> OMDataSource node referencing the data source.
>>>
>>> ​I have checked this with inbound but there is a fix we need to do. I
>>> have done it in [1]. I have done several tests with and without streaming
>>> from 5mb to 1gb files.
>>>
>>> Without streaming [2] and [3] will show the memory growth. With
>>> streaming [4]. Without streaming most of the time ESB went OOM​.
>>>
>>> Find the related configs and axis2 changes in [5] and [6].
>>>
>>> [1] https://wso2.org/jira/browse/ESBJAVA-4458
>>>
>>> [2]
>>>
>>> [image: Inline image 1]
>>>
>>> [3]
>>>
>>> [image: Inline image 2]
>>>
>>> [4]
>>>
>>> [image: Inline image 3]
>>>
>>> [5]
>>>
>>> http://ws.apache.org/ns/synapse;  name="load"
>>>  sequence="request"  onError="fault" protocol="file" suspend="false">
>>>
>>>   1
>>>   true
>>>   >> name="transport.vfs.ContentType">application/file
>>>   >> name="transport.vfs.LockReleaseSameNode">false
>>>   false
>>>   >> name="transport.vfs.ActionAfterFailure">DELETE
>>>   true
>>>   true
>>>   >> name="transport.vfs.ActionAfterProcess">DELETE
>>>   >> name="transport.vfs.FileURI">file:///home/wso2/work/tmp/file/in
>>>   false
>>>   true
>>>   enable
>>>   true
>>>   NONE
>>>   false
>>>
>>> 
>>>
>>> 
>>> 
>>> 
>>> 
>>> >> value="true"/>
>>> 
>>> 
>>> 
>>> 
>>>
>>> [6]
>>> >> class="org.apache.axis2.format.BinaryBuilder"/>
>>>
>>> >>
>>> class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>
>>>
>>> On Mon, Feb 29, 2016 at 12:02 PM, Malaka Silva  wrote:
>>>
 Hi Kasun,

 Currently no OOB solution with ESB 4.9.0. File always gets build before
 mediation.

 However use case mentioned can be handled with Schedule task -> File
 connector search -> File connector copy

 +1 for OOB solution.

 We can  do this for ESB 5.0. Also there is an option to do this as a
 custom inbound. So that this can be used by previous ESB versions as well.

 So default inbound can be used, if someone needs to do a mediation on
 file content and custom inbound for PT file use case. WDYT?


 On Mon, Feb 29, 2016 at 11:49 AM, Kasun Indrasiri 
 wrote:

> Hi Malaka,
>
> Do we support 

[Dev] (no subject)

[Vivekananthan Sivanayagam]

Hi ,

We don't have the same functionality of VFS sender in File Connector.





*Thank youVivekananthan Sivanayagam*

*Associate Software Engineer | WSO2*

*E:vivekanant...@wso2.com *
*M:+94752786138*

On Mon, Mar 14, 2016 at 9:57 AM, Malaka Silva  wrote:

> Hi Kasun,
>
> I don't think we have the same functionality of VFS sender in file
> connector. We have only focused on use cases not covered with VFS transport.
>
> @Vivekananthan - Please confirm this?
>
> On Sat, Mar 12, 2016 at 4:39 PM, Kasun Indrasiri  wrote:
>
>> This is great. Thanks a lot Malaka.
>> Also, if we use the file connector as the outbound channel, will it work
>> in the same way?
>>
>> On Tue, Mar 1, 2016 at 12:36 PM, Malaka Silva  wrote:
>>
>>> Hi All,
>>>
>>> Please ignore my previous comments. This can be done with current
>>> implementation.
>>>
>>> if (builder instanceof DataSourceMessageBuilder &&
>>> "true".equals(streaming)) {
>>> dataSource = ManagedDataSourceFactory.create(new
>>> FileObjectDataSource(file, contentType));
>>> in = null;
>>> } else {
>>> in = new
>>> AutoCloseInputStream(file.getContent().getInputStream());
>>> dataSource = null;
>>> }
>>> ..
>>> OMElement documentElement;
>>> if (in != null) {
>>> documentElement = builder.processDocument(in,
>>> contentType, axis2MsgCtx);
>>> } else {
>>> documentElement =
>>> ((DataSourceMessageBuilder)builder).processDocument(dataSource,
>>> contentType, axis2MsgCtx);
>>> }
>>> ...
>>> if(dataSource != null) {
>>> dataSource.destroy();
>>> }
>>>
>>> This is
>>> ​because ​
>>> message builder able to build messages from
>>> ​​
>>> DataSource objects.
>>> ​
>>> ​
>>> DataSource​
>>>  by definition the data from a DataSource can be read
>>> ​
>>> multiple times,
>>> ​builders that implement ​
>>> this interface
>>> ​can​
>>>  avoid storing the message content in memory.
>>> ​ ​
>>> If a message builder implements this interface and the
>>> ​file/vfs
>>>  is able to provide the message payload as a data source, then the
>>> method defined by this interface should be preferred over the method
>>> defined by Builder.
>>> ​ This
>>>  helps optimizing
>>> ​PT​
>>>  with
>>> ​vfs/file​.
>>> The builder will typically expose the data source directly or indirectly
>>> through the returned OMElement, e.g. by adding to the tree an OMText or
>>> OMDataSource node referencing the data source.
>>>
>>> ​I have checked this with inbound but there is a fix we need to do. I
>>> have done it in [1]. I have done several tests with and without streaming
>>> from 5mb to 1gb files.
>>>
>>> Without streaming [2] and [3] will show the memory growth. With
>>> streaming [4]. Without streaming most of the time ESB went OOM​.
>>>
>>> Find the related configs and axis2 changes in [5] and [6].
>>>
>>> [1] https://wso2.org/jira/browse/ESBJAVA-4458
>>>
>>> [2]
>>>
>>> [image: Inline image 1]
>>>
>>> [3]
>>>
>>> [image: Inline image 2]
>>>
>>> [4]
>>>
>>> [image: Inline image 3]
>>>
>>> [5]
>>>
>>> http://ws.apache.org/ns/synapse;  name="load"
>>>  sequence="request"  onError="fault" protocol="file" suspend="false">
>>>
>>>   1
>>>   true
>>>   >> name="transport.vfs.ContentType">application/file
>>>   >> name="transport.vfs.LockReleaseSameNode">false
>>>   false
>>>   >> name="transport.vfs.ActionAfterFailure">DELETE
>>>   true
>>>   true
>>>   >> name="transport.vfs.ActionAfterProcess">DELETE
>>>   >> name="transport.vfs.FileURI">file:///home/wso2/work/tmp/file/in
>>>   false
>>>   true
>>>   enable
>>>   true
>>>   NONE
>>>   false
>>>
>>> 
>>>
>>> 
>>> 
>>> 
>>> 
>>> >> value="true"/>
>>> 
>>> 
>>> 
>>> 
>>>
>>> [6]
>>> >> class="org.apache.axis2.format.BinaryBuilder"/>
>>>
>>> >>
>>> class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>
>>>
>>> On Mon, Feb 29, 2016 at 12:02 PM, Malaka Silva  wrote:
>>>
 Hi Kasun,

 Currently no OOB solution with ESB 4.9.0. File always gets build before
 mediation.

 However use case mentioned can be handled with Schedule task -> File
 connector search -> File connector copy

 +1 for OOB solution.

 We can  do this for ESB 5.0. Also there is an option to do this as a
 custom inbound. So that this can be used by previous ESB versions as well.

 So default inbound can be used, if someone needs to do a mediation on
 file content and custom inbound for PT file use case. WDYT?


 On Mon, Feb 29, 2016 at 11:49 AM, Kasun Indrasiri 
 wrote:

> Hi Malaka,
>
> Do we 

Re: [Dev] Configuring load balancing in app cloud with HA Proxy

[Manjula Rathnayake]

Hi Imesh,

On Mon, Mar 14, 2016 at 9:56 AM, Imesh Gunaratne  wrote:

>
>
> On Sun, Mar 13, 2016 at 11:36 PM, Nishadi Kirielle 
> wrote:
>
>> Hi all,
>> Currently I'm working on configuring HAProxy load balancing support for
>> app cloud.
>> In checking the session affinity functionality in kuberenetes, I have
>> verified the load balancing of http traffic with HAProxy. It could be done
>> using kubernetes contribution repo, 'service loadbalancer' [1].
>>
>> In order to check the load balancing with https traffic the taken
>> approach is SSL termination.In the scenario of app cloud, kubernetes
>> cluster is not directly exposed and the load balancer exists within the
>> cluster. Thus the communication between the application servers and the
>> load balancer happens internally. Although SSL termination ends the secure
>> connection at the load balancer, due to the above mentioned reasons, SSL
>> termination seems to be a better solution. The reason for the use of SSL
>> termination over SSL pass through is because of the complexity of handling
>> a separate SSL certificate for each server behind the load balancer in the
>> case of SSL pass through.
>>
>> -1 for this approach, IMO this has a major security risk.
>
> Let me explain the problem. If we offload SSL at the service load
> balancer, all traffic beyond the load balancer will use HTTP and the
> message content will be visible to anyone on network inside K8S. Which
> means someone can simply start a container in K8S and trace all HTTP
> traffic going through.
>
Below is from HA Proxy documentation[1]. AFAIU, HA Proxy to backend server
communication happens with HTTPS enabled but not validating the server
certificate.
verify

[none|required]

This setting is only available when support for OpenSSL was built in. If set
to 'none', server certificate is not verified. In the other case, The
certificate provided by the server is verified using CAs from 'ca-file'
and optional CRLs from 'crl-file'. If 'ssl_server_verify' is not specified
in global  section, this is the default. On verify failure the handshake
is aborted. It is critically important to verify server certificates when
using SSL to connect to servers, otherwise the communication is prone to
trivial man-in-the-middle attacks rendering SSL totally useless.

[1].
http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#ssl%20%28Server%20and%20default-server%20options%29

thank you.


> Thanks
>
> In configuring load balancing with SSL termination, I had to customize
>> kubernetes haproxy.conf file template of service loadbalancer repo to
>> support SSL termination.
>>
>> In order to provide SSL termination, the kubernetes services have to be
>> annotated with
>>   serviceloadbalancer/lb.sslTerm: "true"
>>
>> The default approach in load balancing with service load balancer repo is
>> based on simple fan out approach which uses context path to load balance
>> the traffic. As we need to load balance based on the host name, we need to
>> go with the name based virtual hosting approach. It can be achieved via the
>> following annotation.
>>  serviceloadbalancer/lb.Host: ""
>>
>> Any suggestions on the approach taken are highly appreciated.
>>
>> Thank you
>>
>> [1].
>> https://github.com/kubernetes/contrib/tree/master/service-loadbalancer
>>
>>
>
>
>
> --
> *Imesh Gunaratne*
> Senior Technical Lead
> WSO2 Inc: http://wso2.com
> T: +94 11 214 5345 M: +94 77 374 2057
> W: http://imesh.io
> Lean . Enterprise . Middleware
>
>


-- 
Manjula Rathnayaka
Associate Technical Lead
WSO2, Inc.
Mobile:+94 77 743 1987
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] Session Affinity in Kubernetes

[Imesh Gunaratne]

On Sun, Mar 13, 2016 at 11:37 PM, Nishadi Kirielle  wrote:

> Hi Imesh,
> The reason for choosing SSL termination over SSL pass through is due to
> the complexity of handling separate SSL certificates for each servers
> behind the load balancer in kubernetes cluster. As in App Cloud the
> kubernetes cluster is not direcctly exposed and the communication between
> the load balancer and servers happens internally, we have thought of
> choosing SSL termination approach over SSL pass through.
>

-1 There is no way we can assume that internal networks are secure. Please
refer the other thread "Configuring load balancing in app cloud with HA
Proxy".

FYI: I see two threads on the same topic. It might be better to keep them
in one.

Thanks

>
>
> Thanks
>
> On Thu, Mar 10, 2016 at 11:21 PM, Imesh Gunaratne  wrote:
>
>>
>>
>> On Thu, Mar 10, 2016 at 1:28 PM, Nishadi Kirielle 
>> wrote:
>>
>>> Thank you for the suggestion of using the default self signed
>>> certificate.
>>> I have attempted SSL termination approach of terminating the SSL
>>> connection at the load balancer and sending unencrypted connections to the
>>> backend server via the ha proxy configuration of 'ssl verify none'. This
>>> approach allows https traffic to be load balanced and exposed.
>>>
>>> Terminating SSL at the middle of a communication flow would introduce
>> security risks.
>>
>> Thanks
>>
>>
>>> Thanks
>>>
>>>
>>> On Thu, Mar 10, 2016 at 11:20 AM, Imesh Gunaratne 
>>> wrote:
>>>


 On Thu, Mar 10, 2016 at 10:49 AM, Nishadi Kirielle 
 wrote:

> Hi all,
> I have only tested for http traffic earlier. Although the kubernetes
> service loadbalancer template has support for https, when I have deployed
> an application ( dell/tomcat ) which has the support for https, the ha
> proxy load balancer did not identify it as a https service in the haproxy
> configuration file. It just identified the application as a http
> application and updated the configuration file accordingly.
>

 Yes, in our K8S services we have defined the protocol as TCP, not as
 HTTPS/SSL. Therefore there is no way for the service load balancer to find
 this information by looking at the services.


> Thus I have manually altered the ha proxy configuration file to
> support for https traffic with a self signed certificate specific for the
> node ip. But it fails in accessing the application, since the application
> needs the self signed certificate specific to the application.
> As a solution for this I'm trying with bind option 'cert' to bind
> several certificate files[2] of the specific applications.
>

 Shall we try with the default self signed certificate distributed with
 a WSO2 product?

 Thanks

>
> Any suggestions on this are highly appreciated.
> [1] .
> https://github.com/kubernetes/contrib/tree/master/service-loadbalancer
> [2] .
> https://cbonte.github.io/haproxy-dconv/configuration-1.5.html#5.1-crt
>
> Thanks
>
> On Wed, Mar 9, 2016 at 10:33 AM, Imesh Gunaratne 
> wrote:
>
>> Hi Deep,
>>
>> On Tue, Mar 8, 2016 at 8:08 PM, Deependra Ariyadewa 
>> wrote:
>>
>>>
>>> On Mon, Mar 7, 2016 at 10:30 AM, Nishadi Kirielle 
>>> wrote:
>>>
 Hi All,
 I have written the blog post on load balancing and session affinity
 in kubernetes. [1]

>>>
>>> I am going test session affinity for HTTPS triffic in Kubernetes
>>> following your configurations. Did you try to enable session affinity 
>>> for
>>> HTTPS triffic in Kubernetes.
>>>
>>> We would need to configure haproxy with relevant SSL certificates
>> for HTTPS to work. I do not think we tested it. See [1] for the haproxy
>> config template used by the service load balancer. This will get packaged
>> to the Docker service load balancer Docker image [2].
>>
>> [1]
>> https://github.com/kubernetes/contrib/blob/master/service-loadbalancer/template.cfg
>> [2]
>> https://github.com/kubernetes/contrib/blob/master/service-loadbalancer/Dockerfile
>>
>> Thanks
>>
>>
>>> Thanks,
>>> Deependra.
>>>

 Thank you

 [1].
 http://nishadikirielle.blogspot.com/2016/03/load-balancing-kubernetes-services-and.html

 On Fri, Mar 4, 2016 at 8:22 PM, Nishadi Kirielle 
 wrote:

> Thanks a lot. I will write a blog post and share it.
>
> Thanks
>
>
>
> On Fri, Mar 4, 2016 at 6:07 PM, Sagara Gunathunga  > wrote:
>
>>
>> Great, it would be better if Nishadi can write a step by step
>> blog post 

Re: [Dev] Streaming File transfer with File Inbound/File Connector

[Malaka Silva]

Hi Kasun,

I don't think we have the same functionality of VFS sender in file
connector. We have only focused on use cases not covered with VFS transport.

@Vivekananthan - Please confirm this?

On Sat, Mar 12, 2016 at 4:39 PM, Kasun Indrasiri  wrote:

> This is great. Thanks a lot Malaka.
> Also, if we use the file connector as the outbound channel, will it work
> in the same way?
>
> On Tue, Mar 1, 2016 at 12:36 PM, Malaka Silva  wrote:
>
>> Hi All,
>>
>> Please ignore my previous comments. This can be done with current
>> implementation.
>>
>> if (builder instanceof DataSourceMessageBuilder &&
>> "true".equals(streaming)) {
>> dataSource = ManagedDataSourceFactory.create(new
>> FileObjectDataSource(file, contentType));
>> in = null;
>> } else {
>> in = new
>> AutoCloseInputStream(file.getContent().getInputStream());
>> dataSource = null;
>> }
>> ..
>> OMElement documentElement;
>> if (in != null) {
>> documentElement = builder.processDocument(in,
>> contentType, axis2MsgCtx);
>> } else {
>> documentElement =
>> ((DataSourceMessageBuilder)builder).processDocument(dataSource,
>> contentType, axis2MsgCtx);
>> }
>> ...
>> if(dataSource != null) {
>> dataSource.destroy();
>> }
>>
>> This is
>> ​because ​
>> message builder able to build messages from
>> ​​
>> DataSource objects.
>> ​
>> ​
>> DataSource​
>>  by definition the data from a DataSource can be read
>> ​
>> multiple times,
>> ​builders that implement ​
>> this interface
>> ​can​
>>  avoid storing the message content in memory.
>> ​ ​
>> If a message builder implements this interface and the
>> ​file/vfs
>>  is able to provide the message payload as a data source, then the method
>> defined by this interface should be preferred over the method defined by
>> Builder.
>> ​ This
>>  helps optimizing
>> ​PT​
>>  with
>> ​vfs/file​.
>> The builder will typically expose the data source directly or indirectly
>> through the returned OMElement, e.g. by adding to the tree an OMText or
>> OMDataSource node referencing the data source.
>>
>> ​I have checked this with inbound but there is a fix we need to do. I
>> have done it in [1]. I have done several tests with and without streaming
>> from 5mb to 1gb files.
>>
>> Without streaming [2] and [3] will show the memory growth. With streaming
>> [4]. Without streaming most of the time ESB went OOM​.
>>
>> Find the related configs and axis2 changes in [5] and [6].
>>
>> [1] https://wso2.org/jira/browse/ESBJAVA-4458
>>
>> [2]
>>
>> [image: Inline image 1]
>>
>> [3]
>>
>> [image: Inline image 2]
>>
>> [4]
>>
>> [image: Inline image 3]
>>
>> [5]
>>
>> http://ws.apache.org/ns/synapse;  name="load"
>>  sequence="request"  onError="fault" protocol="file" suspend="false">
>>
>>   1
>>   true
>>   > name="transport.vfs.ContentType">application/file
>>   > name="transport.vfs.LockReleaseSameNode">false
>>   false
>>   > name="transport.vfs.ActionAfterFailure">DELETE
>>   true
>>   true
>>   > name="transport.vfs.ActionAfterProcess">DELETE
>>   > name="transport.vfs.FileURI">file:///home/wso2/work/tmp/file/in
>>   false
>>   true
>>   enable
>>   true
>>   NONE
>>   false
>>
>> 
>>
>> 
>> 
>> 
>> 
>> > value="true"/>
>> 
>> 
>> 
>> 
>>
>> [6]
>> > class="org.apache.axis2.format.BinaryBuilder"/>
>>
>> >
>> class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>
>>
>> On Mon, Feb 29, 2016 at 12:02 PM, Malaka Silva  wrote:
>>
>>> Hi Kasun,
>>>
>>> Currently no OOB solution with ESB 4.9.0. File always gets build before
>>> mediation.
>>>
>>> However use case mentioned can be handled with Schedule task -> File
>>> connector search -> File connector copy
>>>
>>> +1 for OOB solution.
>>>
>>> We can  do this for ESB 5.0. Also there is an option to do this as a
>>> custom inbound. So that this can be used by previous ESB versions as well.
>>>
>>> So default inbound can be used, if someone needs to do a mediation on
>>> file content and custom inbound for PT file use case. WDYT?
>>>
>>>
>>> On Mon, Feb 29, 2016 at 11:49 AM, Kasun Indrasiri 
>>> wrote:
>>>
 Hi Malaka,

 Do we support the $subject? Basically we use an Inbound as the source
 and use file connector as the destination. If this is not supported yet, we
 got to add this to ESB 5.

 Thanks,
 Kasun

 --
 Kasun Indrasiri
 Software Architect
 WSO2, Inc.; http://wso2.com
 lean.enterprise.middleware

 cell: +94 77 556 5206
 Blog : http://kasunpanorama.blogspot.com/

>>>
>>>
>>>
>>> --
>>>
>>> Best Regards,
>>>
>>> Malaka Silva
>>> Senior 

[Dev] AppCloud Docker Registry Setup

[Nadeeshani Pathirennehelage]

Hi All,

When AppColud is setup, we need to configure docker for connecting with the
remote docker registry. I have written two shell scripts for that. [1]

   1. To create a new docker registry.
   2. To update the client machine for connecting with remote docker
   registry.

I sent a PR (#161) with these changes.

[1]
https://github.com/NShani/app-cloud/tree/master/modules/setup-scripts/tools

Thanks,

-- 
Pathirennehelage Nadeeshani
Software Engineering Intern : WSO2 Inc
Mobile : +94 (0) 716 545223
nadeesha...@wso2.com
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] Configuring load balancing in app cloud with HA Proxy

[Imesh Gunaratne]

On Sun, Mar 13, 2016 at 11:36 PM, Nishadi Kirielle  wrote:

> Hi all,
> Currently I'm working on configuring HAProxy load balancing support for
> app cloud.
> In checking the session affinity functionality in kuberenetes, I have
> verified the load balancing of http traffic with HAProxy. It could be done
> using kubernetes contribution repo, 'service loadbalancer' [1].
>
> In order to check the load balancing with https traffic the taken approach
> is SSL termination.In the scenario of app cloud, kubernetes cluster is not
> directly exposed and the load balancer exists within the cluster. Thus the
> communication between the application servers and the load balancer happens
> internally. Although SSL termination ends the secure connection at the load
> balancer, due to the above mentioned reasons, SSL termination seems to be a
> better solution. The reason for the use of SSL termination over SSL pass
> through is because of the complexity of handling a separate SSL certificate
> for each server behind the load balancer in the case of SSL pass through.
>
> -1 for this approach, IMO this has a major security risk.

Let me explain the problem. If we offload SSL at the service load balancer,
all traffic beyond the load balancer will use HTTP and the message content
will be visible to anyone on network inside K8S. Which means someone can
simply start a container in K8S and trace all HTTP traffic going through.

Thanks

In configuring load balancing with SSL termination, I had to customize
> kubernetes haproxy.conf file template of service loadbalancer repo to
> support SSL termination.
>
> In order to provide SSL termination, the kubernetes services have to be
> annotated with
>   serviceloadbalancer/lb.sslTerm: "true"
>
> The default approach in load balancing with service load balancer repo is
> based on simple fan out approach which uses context path to load balance
> the traffic. As we need to load balance based on the host name, we need to
> go with the name based virtual hosting approach. It can be achieved via the
> following annotation.
>  serviceloadbalancer/lb.Host: ""
>
> Any suggestions on the approach taken are highly appreciated.
>
> Thank you
>
> [1].
> https://github.com/kubernetes/contrib/tree/master/service-loadbalancer
>
>



-- 
*Imesh Gunaratne*
Senior Technical Lead
WSO2 Inc: http://wso2.com
T: +94 11 214 5345 M: +94 77 374 2057
W: http://imesh.io
Lean . Enterprise . Middleware
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] Fix the error message fund->found

[Mohanadarshan Vivekanandalingam]

On Mon, Mar 14, 2016 at 8:18 AM, Srinath Perera  wrote:

> Exception in thread "main"
> org.wso2.siddhi.core.exception.QueryNotExistException: No query fund with
> name: WordFreqByDay
>

We have fixed this from Siddhi 3.0.5 version.. Are you using older Siddhi
version ?

Thanks,
Mohan


>
> --
> 
> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera
> Site: http://people.apache.org/~hemapani/
> Photos: http://www.flickr.com/photos/hemapani/
> Phone: 0772360902
>



-- 
*V. Mohanadarshan*
*Senior Software Engineer,*
*Data Technologies Team,*
*WSO2, Inc. http://wso2.com  *
*lean.enterprise.middleware.*

email: mo...@wso2.com
phone:(+94) 771117673
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] GSoC Proposal #21 and #22

[Tharindu Edirisinghe]

Hi Pubudu,

We are glad to see your interest in "*21: [IS] Document Based NoSQL Support
for WSO2 Identity Server Database*" project. In order to make you familiar
with this project with regard to WSO2 platform, we would like you to
complete the following task which would provide an understanding on how the
userstore managers are used in WSO2 products.


1. Refer [1] and understand how a userstore manager can be written and used
in a product like WSO2 Identity Server [2].

2. Refer [3] and understand more about Claims and Claim Management.

3. Referring [1], extend the
org.wso2.carbon.user.core.jdbc.JDBCUserStoreManager class and write your
own JDBC userstore manaager. (MySQL is preferred).

4. Introduce a new claim called lastPasswordResetTimestamp which tracks the
timestamp of each user's last successful password reset attempt. (in WSO2
Identity Server). For this you can refer [1] and override the
doUpdateCredential, doUpdateCredentialByAdmin methods in the custom
userstore manager you wrote in previous step. Inside these methods after
successful password reset, set the current timestemp as a user claim. For
this you can refer [4].

5. In the JDBC userstore manager you have written in above step, override
the doAuthenticate method. Inside the method, after performing
authentication, compare the timestamp of the last time the user updated the
credentials with current timestamp. If it is greater than 60 days (you can
hardcode this value for the moment) return an exception with a message
saying the user has to reset the password as it is expired. (For the users
where the claim for password reset timestamp is empty, you can let the
users successfully authenticate)

When performing above tasks, if you face any difficulty, you can ask for
help from this mail thread. You can use GitHub to share your source code
and after completing a deliverable from above steps, you can share your
progress with us.

If you need further clarifications, please get back.

[1] https://docs.wso2.com/display/IS510/Writing+a+Custom+User+Store+Manager
[2] http://wso2.com/products/identity-server/
[3]
http://tharindue.blogspot.com/2015/08/claim-management-operations-in-wso2.html
[4]
http://tharindue.blogspot.com/2015/12/tracking-last-successful-login-attempt.html
[5]
http://tharindue.blogspot.com/2015/05/a-workaround-for-renaming-username-of.html

Regards,
TharinduE

On Sat, Mar 12, 2016 at 3:50 PM, Pubudu Dodangoda 
wrote:

> Hi,
>
> I am an undergraduate of Department of Computer Science and Engineering,
> University of Moratuwa. While going through the GSoC project ideas posted
> by WSO2, I found the following two very interesting.
>
> 1) Proposal 21: [IS] Document Based NoSQL Support for WSO2 Identity Server
> Database
> 2) Proposal 22: [IS] RESTful Fine Grained Authorization-as-a-Service
> (AZaaS)
>
> I found these two projects to be interesting since I have worked with
> MongoDB, JUnit, Selenium, REST and SOAP and WSO2 Products.
> I am currently studying about these two projects. Please let me know if
> there is anything that I could refer, analyze or implement before sending
> you my approach on this project. And also if possible, Can you please
> elaborate more on these two projects, so I can select one and focus more on
> that.
>
> I am sorry for being late to send this email. That was because of my
> semester exams.
>
> Thank You and Kind Regards,
>
> Pubudu
>
> --
> Pubudu Dodangoda
> BSc Engineering(Hon's) Undergraduate
> Department of Computer Science Engineering
> University of Moratuwa
> 0716053681 / 0775192994
>



-- 

Tharindu Edirisinghe
Software Engineer | WSO2 Inc
Platform Security Team
Blog : tharindue.blogspot.com
mobile : +94 775181586
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Dev] Fix the error message fund->found

[Srinath Perera]

Exception in thread "main"
org.wso2.siddhi.core.exception.QueryNotExistException: No query fund with
name: WordFreqByDay

-- 

Blog: http://srinathsview.blogspot.com twitter:@srinath_perera
Site: http://people.apache.org/~hemapani/
Photos: http://www.flickr.com/photos/hemapani/
Phone: 0772360902
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Dev] [CDMF] [IOT] IOT SDK for creating Device Agents-GSoC2016

[Harshani Perera]

Thank you. I'll read and study this.

-- 
*Iresha Perera*
Department of Computer Science Engineering
University of Moratuwa
Mobile : +94777863874

___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] Session Affinity in Kubernetes

[Nishadi Kirielle]

Hi Imesh,
The reason for choosing SSL termination over SSL pass through is due to the
complexity of handling separate SSL certificates for each servers behind
the load balancer in kubernetes cluster. As in App Cloud the kubernetes
cluster is not direcctly exposed and the communication between the load
balancer and servers happens internally, we have thought of choosing SSL
termination approach over SSL pass through.

Thanks

On Thu, Mar 10, 2016 at 11:21 PM, Imesh Gunaratne  wrote:

>
>
> On Thu, Mar 10, 2016 at 1:28 PM, Nishadi Kirielle 
> wrote:
>
>> Thank you for the suggestion of using the default self signed
>> certificate.
>> I have attempted SSL termination approach of terminating the SSL
>> connection at the load balancer and sending unencrypted connections to the
>> backend server via the ha proxy configuration of 'ssl verify none'. This
>> approach allows https traffic to be load balanced and exposed.
>>
>> Terminating SSL at the middle of a communication flow would introduce
> security risks.
>
> Thanks
>
>
>> Thanks
>>
>>
>> On Thu, Mar 10, 2016 at 11:20 AM, Imesh Gunaratne  wrote:
>>
>>>
>>>
>>> On Thu, Mar 10, 2016 at 10:49 AM, Nishadi Kirielle 
>>> wrote:
>>>
 Hi all,
 I have only tested for http traffic earlier. Although the kubernetes
 service loadbalancer template has support for https, when I have deployed
 an application ( dell/tomcat ) which has the support for https, the ha
 proxy load balancer did not identify it as a https service in the haproxy
 configuration file. It just identified the application as a http
 application and updated the configuration file accordingly.

>>>
>>> Yes, in our K8S services we have defined the protocol as TCP, not as
>>> HTTPS/SSL. Therefore there is no way for the service load balancer to find
>>> this information by looking at the services.
>>>
>>>
 Thus I have manually altered the ha proxy configuration file to support
 for https traffic with a self signed certificate specific for the node ip.
 But it fails in accessing the application, since the application needs the
 self signed certificate specific to the application.
 As a solution for this I'm trying with bind option 'cert' to bind
 several certificate files[2] of the specific applications.

>>>
>>> Shall we try with the default self signed certificate distributed with a
>>> WSO2 product?
>>>
>>> Thanks
>>>

 Any suggestions on this are highly appreciated.
 [1] .
 https://github.com/kubernetes/contrib/tree/master/service-loadbalancer
 [2] .
 https://cbonte.github.io/haproxy-dconv/configuration-1.5.html#5.1-crt

 Thanks

 On Wed, Mar 9, 2016 at 10:33 AM, Imesh Gunaratne 
 wrote:

> Hi Deep,
>
> On Tue, Mar 8, 2016 at 8:08 PM, Deependra Ariyadewa 
> wrote:
>
>>
>> On Mon, Mar 7, 2016 at 10:30 AM, Nishadi Kirielle 
>> wrote:
>>
>>> Hi All,
>>> I have written the blog post on load balancing and session affinity
>>> in kubernetes. [1]
>>>
>>
>> I am going test session affinity for HTTPS triffic in Kubernetes
>> following your configurations. Did you try to enable session affinity for
>> HTTPS triffic in Kubernetes.
>>
>> We would need to configure haproxy with relevant SSL certificates for
> HTTPS to work. I do not think we tested it. See [1] for the haproxy config
> template used by the service load balancer. This will get packaged to the
> Docker service load balancer Docker image [2].
>
> [1]
> https://github.com/kubernetes/contrib/blob/master/service-loadbalancer/template.cfg
> [2]
> https://github.com/kubernetes/contrib/blob/master/service-loadbalancer/Dockerfile
>
> Thanks
>
>
>> Thanks,
>> Deependra.
>>
>>>
>>> Thank you
>>>
>>> [1].
>>> http://nishadikirielle.blogspot.com/2016/03/load-balancing-kubernetes-services-and.html
>>>
>>> On Fri, Mar 4, 2016 at 8:22 PM, Nishadi Kirielle 
>>> wrote:
>>>
 Thanks a lot. I will write a blog post and share it.

 Thanks



 On Fri, Mar 4, 2016 at 6:07 PM, Sagara Gunathunga 
 wrote:

>
> Great, it would be better if Nishadi can write a step by step blog
> post about how to do this.  We had to do a 30 hours hackathon to 
> change
> MSF4J Pet-store sample due to this issue :)
>
> Thanks !
>
> On Fri, Mar 4, 2016 at 5:54 PM, Imesh Gunaratne 
> wrote:
>
>> Indeed! Overall great effort!!
>>
>> Thanks
>>
>> On Fri, Mar 4, 2016 at 3:36 PM, Lakmal Warusawithana <
>> lak...@wso2.com> wrote:
>>
>>> Great 

[Dev] Configuring load balancing in app cloud with HA Proxy

[Nishadi Kirielle]

Hi all,
Currently I'm working on configuring HAProxy load balancing support for app
cloud.
In checking the session affinity functionality in kuberenetes, I have
verified the load balancing of http traffic with HAProxy. It could be done
using kubernetes contribution repo, 'service loadbalancer' [1].

In order to check the load balancing with https traffic the taken approach
is SSL termination.In the scenario of app cloud, kubernetes cluster is not
directly exposed and the load balancer exists within the cluster. Thus the
communication between the application servers and the load balancer happens
internally. Although SSL termination ends the secure connection at the load
balancer, due to the above mentioned reasons, SSL termination seems to be a
better solution. The reason for the use of SSL termination over SSL pass
through is because of the complexity of handling a separate SSL certificate
for each server behind the load balancer in the case of SSL pass through.

In configuring load balancing with SSL termination, I had to customize
kubernetes haproxy.conf file template of service loadbalancer repo to
support SSL termination.

In order to provide SSL termination, the kubernetes services have to be
annotated with
  serviceloadbalancer/lb.sslTerm: "true"

The default approach in load balancing with service load balancer repo is
based on simple fan out approach which uses context path to load balance
the traffic. As we need to load balance based on the host name, we need to
go with the name based virtual hosting approach. It can be achieved via the
following annotation.
 serviceloadbalancer/lb.Host: ""

Any suggestions on the approach taken are highly appreciated.

Thank you

[1]. https://github.com/kubernetes/contrib/tree/master/service-loadbalancer
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] gsoc

[Sajith Kariyawasam]

Hi Hardik,

It would be better if you clearly explain the errors you are experiencing.
Please attach the relevant log files / configuration files which you think
would be useful for someone to figure out the issue.
If those issues are docker related, you would also need to check with the
devs/ users in docker group [1]

[1] https://groups.google.com/forum/#!forum/docker-user

On Sat, Mar 12, 2016 at 10:24 PM, Hardik Patel 
wrote:

> hi Isuruh,
>
> As your suggestion I also tried to install docker locally on ubuntu
> 14.04 for running openshift. But It gives error in installation. it
> can't find docker files.
> I have run the command, but it is unable to find the repositories.
>
> what should I do?
>
> --
>
> Thank You,
>
> Hardik Patel
> about.me/hardikprl94
>



-- 
Sajith Kariyawasam
*Committer and PMC member, Apache Stratos, *
*WSO2 Inc.; http://wso2.com *
*Mobile: 0772269575*
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [CDMF] [IOT] IOT SDK for creating Device Agents-GSoC2016

[Rasika Perera]

Hi Harshani,

Thanks for your interest on this project. The main idea is to
provide SDKs that are catered for different languages, which can be used to
design a device agent/firmware to quickly and easily connect
with IoT Server. This should hide the complexities of communicating
 authentication, encryption with the IoT Server. This SDK should also have
the capability to pick the desired transport protocols. The expected
deliverable are auto-generated code, abstract the protocol complexities and
should be easily portable for different hardware.

For instance consider a device type (eg: smart bulb) has an API that it
communicates with and this API can be either a rest api, mqtt topic or an
xmpp account. So in a device perspective it should hide the complexities by
auto generating the code that communicates with these APIs. Our focus is to
provide support for Arduino, Android, IOS and also specifically support for
languages such as C, Java, Python and Objective C++.

It would be great if you could go through the existing agent code and study
on how we can improve[1].

[1] https://github.com/wso2/carbon-device-mgt-plugins/tree/

IoTS-1.0.0-M4

On Sun, Mar 13, 2016 at 9:10 AM, Harshani Perera <
ireshaharshani...@cse.mrt.ac.lk> wrote:

> Hi all,
>
>   I am a third year Computer Science Engineering undergraduate at
> University of Moratuwa Sri Lanka. I'm very much interested in participating
> GSoC 2016 and hence, contributing to "[CDMF] [IOT] IOT SDK for creating
> Device Agents" project. I'm quite familiar with  C, C++, java, python. I
> would be grateful to you if you could provide me some tips and direct me to
> proceed further.
>
> --
> *Iresha Perera*
> Department of Computer Science Engineering
> University of Moratuwa
> Mobile : +94777863874
> 
>



-- 
With Regards,

*Rasika Perera*
Software Engineer
M: +94 71 680 9060 E: rasi...@wso2.com
LinkedIn: http://lk.linkedin.com/in/rasika90

WSO2 Inc. www.wso2.com
lean.enterprise.middleware
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [AS 6.0.0] Webapp Loader Modue - Parent First Configuration

[Miraj Abeysekara]

Hi Kishanthan,

I will remove that parent first configuration in the wso2as-web.xml.

According to the Java docs of ClassLoader[1] the child classes are only
need to register as parallel capable. Since the tomcat
WebappClassLoaderBase is registering it as parallel capable, we only need
to call the registerAsParallelCapable() in a static block of our custom
classloader (because of the condition 1 given in the Java docs.[1])

Also in the wso2as-web.xml we can add a Boolean to configure whether a web
app requires parallel capable classloader.

[1]
https://docs.oracle.com/javase/8/docs/api/java/lang/ClassLoader.html#registerAsParallelCapable--

Thanks.

On Sun, Mar 13, 2016 at 11:51 AM, Kishanthan Thangarajah <
kishant...@wso2.com> wrote:

> Did we also look into how we can use ParallelWebappClassLoader with our
> class loaded enhancements?
>
>
> https://tomcat.apache.org/tomcat-8.0-doc/config/loader.html#Standard_Implementation
>
> On Sun, Mar 13, 2016 at 11:42 AM, Kishanthan Thangarajah <
> kishant...@wso2.com> wrote:
>
>> If tomcat is already providing a way to configure the parent first
>> behaviour we should use that instead of our own. But most of the times, we
>> do not need to change this class loading behaviour to parent first as we
>> mostly use the default (child first) behaviour. This config was added
>> during OSGi based tomcat class loading time (AS 5.3.0) to make use of the
>> parent class loading behaviour, but this is no longer a valid use case with
>> plain tomcat. So let's go ahead with tomcat based config.
>>
>> On Fri, Mar 11, 2016 at 11:02 AM, Miraj Abeysekara 
>> wrote:
>>
>>> Hi all,
>>>
>>> Currently in the wso2as-web.xml contains a parent first configuration
>>> parameter per web app class loading. Also tomcat it self provide
>>> configurable parent first class loading behavior in side the context.xml.
>>>
>>> One benefit of using the tomcat configuration for controlling parent
>>> first behavior is, we can reuse their methods. But if we separate the
>>> parent first parameter from wso2as-web.xml we need to add context.xml file
>>> per web app just for changing the class loading behavior. Also for each
>>> context.xml, the Loader element must be added with specifying app-sever web
>>> app loader class if the web app requires wso2 web app loader module.
>>> Therefore it is bit complex to configure a web app if it requires to change
>>> parent first behavior.
>>>
>>> Which approach should we use for the configuration?
>>>
>>> Thanks
>>> --
>>> Miraj Abeysekara
>>> Intern (Software Engineering)
>>> Mobile: +94775690822
>>> Twitter: https://twitter.com/MiRAGECreator
>>> GooglePlus: https://plus.google.com/u/0/+MirageAbeysekara
>>>
>>
>>
>>
>> --
>> *Kishanthan Thangarajah*
>> Associate Technical Lead,
>> Platform Technologies Team,
>> WSO2, Inc.
>> lean.enterprise.middleware
>>
>> Mobile - +94773426635
>> Blog - *http://kishanthan.wordpress.com
>> *
>> Twitter - *http://twitter.com/kishanthan *
>>
>
>
>
> --
> *Kishanthan Thangarajah*
> Associate Technical Lead,
> Platform Technologies Team,
> WSO2, Inc.
> lean.enterprise.middleware
>
> Mobile - +94773426635
> Blog - *http://kishanthan.wordpress.com *
> Twitter - *http://twitter.com/kishanthan *
>



-- 
Miraj Abeysekara
Intern (Software Engineering)
Mobile: +94775690822
Twitter: https://twitter.com/MiRAGECreator
GooglePlus: https://plus.google.com/u/0/+MirageAbeysekara
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] Fwd: GSOC2016: Proposal 6: [ML]

[Maheshakya Wijewardena]

Hi Mahesh,

You don't have to look into carbon-ml.

Best regards.

On Sun, Mar 13, 2016 at 5:49 PM, Mahesh Dananjaya  wrote:

> Hi maheshakya,
> i am working on some examples related to Spark and ML.is there anything to
> do with carbon-ml. I think i dont need to look into that one.do i?
> BR,
> Mahesh
>
> On Tue, Mar 8, 2016 at 11:55 AM, Maheshakya Wijewardena <
> mahesha...@wso2.com> wrote:
>
>> Hi Mahesh,
>>
>> does that Scala API is with your current product or repo?
>>
>>
>> No, we don't have the Scala API included. What we want is to design the
>> Java implementations of those algorithms to train with mini-batches of
>> streaming data with the help of the aforementioned methods so that we can
>> include in as a CEP extension.
>>
>> As to clarify, please try to write a simple Java program using Spark
>> MLLib linear regression and k-means clustering with a sample data set (You
>> can find alot of data sets from UCI repo[1]).  You need to break the
>> dataset into several pieces and train a model repeatedly with those.
>> After each training run, save the model information (such as weights,
>> intercepts for regression and cluster centers for clustering - please check
>> the arguments of those methods I have mentioned and save the required
>> information of the model)
>> When training a model we a new piece of data, use those methods to
>> initialize and put the save values for the arguments. This way you can
>> start from where you stopped in the previous run.
>>
>> Let us know your observations and feel free to ask if you need to know
>> anything more on this.
>>
>> We'll let you know what needs to be done to include this in CEP.
>>
>> Best regards.
>>
>> On Tue, Mar 8, 2016 at 10:59 AM, Mahesh Dananjaya <
>> dananjayamah...@gmail.com> wrote:
>>
>>> Hi Maheshakya,
>>> great.thank you.i already have ML and CEP and working more towards it.
>>> does that Scala API is with your current product or repo?.  thank you.
>>> BR,
>>> Mahesh.
>>>
>>> On Sun, Mar 6, 2016 at 5:49 PM, Maheshakya Wijewardena <
>>> mahesha...@wso2.com> wrote:
>>>
 Hi Mahesh,

 Please find the comments inline.

 does data stream is taken to ML as the event publisher's format through
> event publisher. Or  we can use direct traffic that comes to event
> receiver, or else as streams
>
 We intend to use the direct data as even streams.

 1.) Those data coming from wso2 DAS to ML are coming as streams?
>
 No, WSO2 ML doesn't use any even stream. The data stored in tables in
 DAS is loaded into ML.

 2.) Are there any incremental learning algorithms currently active in
> ML?you mentioned that there are and they are with scala API. So there is a
> streaming support with that Scala API. In that API which format the data 
> is
> aquired to ML?
>
 No, there are no incremental learning algorithms in ML. The scala API
 is about Spark MLLib. MLLib supports streaming k-means and other
 generalized linear models (linear regression variants and logistic
 regression) with Scala API. What they basically do in those implementations
 is retraining the trained models with mini batches when data sequentially
 arrives. There, the breaking of streaming data into mini batches is done
 with the help of Spark Streaming. But we do not intend to use Spark
 streaming in our implementation. What we need to do is implement a similar
 behavior for event streams using the Java API.  The Java API has the
 following methods:

- *createModel

 *
(Vector

 
  weights,
double intercept) - for GLMs
- *setInitialModel

 *
(KMeansModel

 
  model)
- for K means

 With the help of these methods, we can train models again with newly
 arriving data, keeping the characteristics learned with the previous data.
 When implementing this, we need to pay attention to other parameters of
 incremental learning such as data horizon and data obsolescence (indicated
 in the project ideas page).
 We need to discuss on how to add these with CEP event streams. I have
 added Suho into the thread for more clarification.

 Best regards.


 On Sat, Mar 5, 2016 at 5:15 PM, Mahesh Dananjaya <
 dananjayamah...@gmail.com> wrote:

> Hi maheshakya,
> as we concerned to use