On Wed, Jul 22, 2009 at 02:09:27PM +0530, Hiranya Jayathilaka wrote: > Hi Folks, > > I just finished a second implementation of this feature which uses a custom > IOEventDispatch and a single IOReactor instance. I have uploaded the patch > at [1]. >
Hiranya, That is the way to go. I posted a more detailed answer to your question to the HC dev list. Cheers Oleg > However with this we are no longer able to configure SSL requirements at > endpoint level. We need to configure everything at transport level. Under > the HTTPS sender configuration we can define zero or more SSL profiles and > associate each profile with one or more servers. See the sample > configuration below. > > <transportSender name="https" > class="org.apache.synapse.transport.nhttp.HttpCoreNIOSSLSender"> > <parameter name="non-blocking" locked="false">true</parameter> > <parameter name="keystore" locked="false"> > <KeyStore> > <Location>lib/identity.jks</Location> > <Type>JKS</Type> > <Password>password</Password> > <KeyPassword>password</KeyPassword> > </KeyStore> > </parameter> > <parameter name="truststore" locked="false"> > <TrustStore> > <Location>lib/trust.jks</Location> > <Type>JKS</Type> > <Password>password</Password> > </TrustStore> > </parameter> > *<parameter name="customSSLProfiles"> > <profile> > <servers>localhost:19002, www.testserver.com:80</servers> > <KeyStore> > <Location>/home/hiranya/cert/B/service.jks</Location> > <Type>JKS</Type> > <Password>123456</Password> > <KeyPassword>123456</KeyPassword> > </KeyStore> > <TrustStore> > <Location>/home/hiranya/cert/B/client.jks</Location> > <Type>JKS</Type> > <Password>123456</Password> > </TrustStore> > </profile> > </parameter>* > </transportSender> > > The above configuration creates one SSL profile and associates it with two > destination servers. The transport sender will lookup the available SSL > contexts and pick the correct one when creating a SSL IO session. > > I believe this implementation achives our goals and gives a fair amount of > control over how certificates are used to connect to different endpoints. > Please provide your feedback. > > Thanks, > Hiranya > > > On Wed, Jul 22, 2009 at 10:56 AM, Supun Kamburugamuva > <[email protected]>wrote: > > > Hi, > > > > On Wed, Jul 22, 2009 at 8:20 AM, indika kumara <[email protected]>wrote: > > > >> Ruwan , It is not reactors that proportional to the users , but all > >> resources consumes by reactors. Did you see java doc from > >> ?AbstractMultiworkerIOReactor?. > >> > >> <javadoc> > >> *Usually it is recommended to have one worker I/O reactor per physical > >> CPU core.* > >> <javadoc> > >> > >> I believe because It is few SSL profiles this may be acceptable. But , It > >> is better to get feedback from some one using Synapse in a real production > >> environment. > >> > > > > I think this comment is about server side use of reactors. But here it is a > > client side use. Anyway these are not things that can be decided that > > easily. Better to ask some one in production. > > > > Thanks, > > Supun.. > > > > > >> > >> Indika > >> > >> On Wed, Jul 22, 2009 at 6:12 AM, Ruwan Linton > >> <[email protected]>wrote: > >> > >>> Indika, > >>> > >>> This might not be the approach that we need to do but I just wanted to > >>> get concrete in this discussion and see how we can implement this. I agree > >>> with you, this might not be a scalable solution but on the other hand we > >>> will have only a few SSL profiles, but not many in a given server.. so we > >>> are not talking about hundreds of reactors it will be like 3-5, also it is > >>> not proportional to the users in the system. > >>> > >>> Thanks for the review and I think this is very valuable. Hiranya, I > >>> suggest we wait for few other reviews and do the modifications to the code > >>> so that we gets to the sub optimal solution. You may investigate on the > >>> IOEventDispatcher approach in the mean time as well. > >>> > >>> Thanks, > >>> Ruwan > >>> > >>> > >>> On Tue, Jul 21, 2009 at 11:34 PM, indika kumara > >>> <[email protected]>wrote: > >>> > >>>> I didn?t look details at code level of patch. Patch is an implementation > >>>> of approach that uses MultiworkerIOReactor per SSLContext . > >>>> > >>>> Reactor is the heart of the Reactor pattern. It is all about > >>>> scalability. Generally, multiple reactors use for load balance between > >>>> reactors to match CPU and IO rates. By default, we use a > >>>> MultiworkerIOReactor which is a multi reactor implementation. In above > >>>> patch > >>>> we use collections of MultiworkerIOReactors. I don?t know how much > >>>> resources > >>>> it consumes. A system with n users to be scalable, the quality of > >>>> physical > >>>> resources required to support them should be at most O(n). Therefore, > >>>> without increasing in loads, if physical resources consumption is > >>>> increased, > >>>> it is not a scalable system. > >>>> > >>>> > >>>> > >>>> BTW, this will be only needed by some set of users. General users may > >>>> not be affected. Therefore, usually there will be a one > >>>> MultiworkerIOReactor. So, this approach may be an option. But, it > >>>> depends on > >>>> actual users that need this feature. Would they like to have many > >>>> MultiworkerIOReactors in a Single Server? If they like, this solution > >>>> may be > >>>> OK. > >>>> > >>>> > >>>> > >>>> Please look at following doc which is from Javadoc of > >>>> ?AbstractMultiworkerIOReactor?. > >>>> > >>>> > >>>> <javadoc> > >>>> > >>>> *Generic implementation of that can run multiple instances in separate > >>>> worker threads and distribute newly created I/O session equally across > >>>> those > >>>> I/O reactors for more optimal resource utilization and a better I/O > >>>> performance. Usually it is recommended to have one worker I/O reactor per > >>>> physical CPU core.* > >>>> > >>>> <javadoc> > >>>> > >>>> > >>>> > >>>> It is better to *ask from Oleg*?. He has written all these core codes. > >>>> If he thinks this is OK, then I believe, this approach may be OK. > >>>> > >>>> > >>>> > >>>> Indika > >>>> > >>>> > >>>> On Tue, Jul 21, 2009 at 8:56 PM, Ruwan Linton > >>>> <[email protected]>wrote: > >>>> > >>>>> If it is going to require IOEventDispatcher changes and a KeyManger > >>>>> impl that is too complex... > >>>>> > >>>>> Why don't we be concrete and have a look at the patch provided and try > >>>>> to overcome the drawbacks of the implementation (if there are any)? > >>>>> > >>>>> Asankha/Oleg, can you please have a look at the patch? > >>>>> > >>>>> Thanks, > >>>>> Ruwan > >>>>> > >>>>> > >>>>> On Tue, Jul 21, 2009 at 6:03 PM, indika kumara > >>>>> <[email protected]>wrote: > >>>>> > >>>>>> I didn?t investigate the details of any suggested solution. Just said > >>>>>> what comes in to my mind. Possible approaches for complete solution > >>>>>> may be > >>>>>> based on OEventDispatch and Keymanger. > >>>>>> You can go deep and suggest what the easy yet complete solution is. > >>>>>> Then, everyone will agree with that. > >>>>>> > >>>>>> Indika > >>>>>> > >>>>>> On Tue, Jul 21, 2009 at 5:56 PM, Hiranya Jayathilaka < > >>>>>> [email protected]> wrote: > >>>>>> > >>>>>>> > >>>>>>> > >>>>>>> On Tue, Jul 21, 2009 at 5:42 PM, indika kumara < > >>>>>>> [email protected]> wrote: > >>>>>>> > >>>>>>>> Even when using IOEventDispatch (What I suggested as my first > >>>>>>>> solution), we have to get information from IOSession to pick correct > >>>>>>>> Cert- > >>>>>>>> possibly remote domain name and ip. These things can get from > >>>>>>>> SSLSession too > >>>>>>>> - may be more. > >>>>>>>> > >>>>>>>> BTW, simple approach is best if it solves the problem. > >>>>>>>> > >>>>>>>> Any way, X509ExtendedKeyManager is in com.sun.net.ssl.internal.ssl , > >>>>>>>> it may be an issue to wrap this class. So , IOEventDispatch solution > >>>>>>>> may be > >>>>>>>> only feasible one. > >>>>>>> > >>>>>>> > >>>>>>> The X509ExtendedKeyManager abstract class comes from the > >>>>>>> javax.net.ssl package. It's implementations are JDK specific. But > >>>>>>> technically that shouldn't prevent us from wrapping it. Need to dig > >>>>>>> deeper > >>>>>>> into the API to get a clear idea on this. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> Hiranya > >>>>>>> > >>>>>>> > >>>>>>>> > >>>>>>>> Indika > >>>>>>>> > >>>>>>>> > >>>>>>>> On Tue, Jul 21, 2009 at 5:23 PM, Ruwan Linton < > >>>>>>>> [email protected]> wrote: > >>>>>>>> > >>>>>>>>> Do we have the required control of figuring out the right identity > >>>>>>>>> certificates over the endpoint in this approach? > >>>>>>>>> > >>>>>>>>> For example can we provide the certificate to be used for a > >>>>>>>>> particular endpoint (may be as an alias) in the endpoint > >>>>>>>>> configuration? I > >>>>>>>>> guess that is the initial requirement. > >>>>>>>>> > >>>>>>>>> Ruwan > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Tue, Jul 21, 2009 at 4:31 PM, indika kumara < > >>>>>>>>> [email protected]> wrote: > >>>>>>>>> > >>>>>>>>>> +1 for this approach if possible. It seems possible (cannot tell > >>>>>>>>>> exactly). Within SSLIOSessionHandler (httpcore), in the following > >>>>>>>>>> method > >>>>>>>>>> > >>>>>>>>>> *public void initialize(SSLEngine sslengine, HttpParams params) , > >>>>>>>>>> * > >>>>>>>>>> > >>>>>>>>>> we can put application specific thing into SSLSession, then access > >>>>>>>>>> within KeyManager implementation to pick the correct alias . > >>>>>>>>>> > >>>>>>>>>> BTW, the actual implementation of the X509ExtendedKeyManager is in > >>>>>>>>>> *com.sun.net.ssl.internal.ssl*. > >>>>>>>>>> > >>>>>>>>>> Indika > >>>>>>>>>> > >>>>>>>>>> On Tue, Jul 21, 2009 at 4:11 PM, Andreas Veithen< > >>>>>>>>>> [email protected]> wrote: > >>>>>>>>>> > A quick look at the Javadoc shows that X509ExtendedKeyManager > >>>>>>>>>> has > >>>>>>>>>> > access to the SSLEngine from which the SSLSession can be > >>>>>>>>>> retrieved. On > >>>>>>>>>> > the other hand, SSLSession has methods > >>>>>>>>>> getValue/putValue/removeValue > >>>>>>>>>> > to store "application layer data". This could be a proper > >>>>>>>>>> solution. To > >>>>>>>>>> > be investigated. > >>>>>>>>>> > > >>>>>>>>>> > Andreas > >>>>>>>>>> > > >>>>>>>>>> > On Tue, Jul 21, 2009 at 12:32, Paul Fremantle<[email protected]> > >>>>>>>>>> wrote: > >>>>>>>>>> >> I agree. Maybe we could wrap the key manager and then be able > >>>>>>>>>> to > >>>>>>>>>> >> implement any kind of complexity (e..g multiple key stores, > >>>>>>>>>> etc) > >>>>>>>>>> >> behind that. The only question I have is whether you can pass > >>>>>>>>>> context > >>>>>>>>>> >> when you call across the key manager API. We need to pass some > >>>>>>>>>> context > >>>>>>>>>> >> so that the wrapper can do the right thing. Maybe thread local > >>>>>>>>>> context > >>>>>>>>>> >> would allow us to pass some context over that boundary. > >>>>>>>>>> >> > >>>>>>>>>> >> Paul > >>>>>>>>>> >> > >>>>>>>>>> >> On Tue, Jul 21, 2009 at 11:22 AM, Andreas > >>>>>>>>>> >> Veithen<[email protected]> wrote: > >>>>>>>>>> >>> On Tue, Jul 21, 2009 at 12:08, Hiranya Jayathilaka< > >>>>>>>>>> [email protected]> wrote: > >>>>>>>>>> >>>> Hi Folks, > >>>>>>>>>> >>>> > >>>>>>>>>> >>>> I did some testing using two Axis2 instances each using its > >>>>>>>>>> own > >>>>>>>>>> >>>> certificates. I managed to get Synapse to connect to both > >>>>>>>>>> servers using a > >>>>>>>>>> >>>> single truststore. I had to put both client certs as well as > >>>>>>>>>> the CA cert in > >>>>>>>>>> >>>> the Synapse truststore for this scenario to work. So I guess > >>>>>>>>>> our current > >>>>>>>>>> >>>> implementation is good enough for a great extent. > >>>>>>>>>> >>>> > >>>>>>>>>> >>>> However the problem I have is the way Synapse (or rather the > >>>>>>>>>> Java SSL API) > >>>>>>>>>> >>>> selects the client cert at runtime. As Andreas has mentioned > >>>>>>>>>> it seems to be > >>>>>>>>>> >>>> using some data sent by the server in selecting the correct > >>>>>>>>>> client cert. I > >>>>>>>>>> >>>> tried to find some documentation which properly explains this > >>>>>>>>>> procedure but > >>>>>>>>>> >>>> still couldn't find something satisfactory. However according > >>>>>>>>>> to most > >>>>>>>>>> >>>> resources and explanation provided by Oleg, it seems that > >>>>>>>>>> client can decide > >>>>>>>>>> >>>> which cert to present the way it wishes. In that case the > >>>>>>>>>> above scenario > >>>>>>>>>> >>>> might fail in certain situations. > >>>>>>>>>> >>>> > >>>>>>>>>> >>>> So IMHO the best option (most scalable and robust) we got is > >>>>>>>>>> to implement a > >>>>>>>>>> >>>> custom IOEventDispatch. WDYT? > >>>>>>>>>> >>> > >>>>>>>>>> >>> Personally, I think that the HTTP(S) transport is already > >>>>>>>>>> complex > >>>>>>>>>> >>> enough and that we should only add to that complexity if there > >>>>>>>>>> is a > >>>>>>>>>> >>> good reason to do so. Assuming that the client certificate > >>>>>>>>>> selection > >>>>>>>>>> >>> algorithm doesn't give the expected results, I think that > >>>>>>>>>> implementing > >>>>>>>>>> >>> a custom IOEventDispatch is only the second best solution. We > >>>>>>>>>> should > >>>>>>>>>> >>> first investigate the possibility to change the behavior of > >>>>>>>>>> the key > >>>>>>>>>> >>> manager. Since this is the component that selects the client > >>>>>>>>>> >>> certificate, from a design perspective it would be the right > >>>>>>>>>> place do > >>>>>>>>>> >>> implement custom behavior. This approach would also have the > >>>>>>>>>> advantage > >>>>>>>>>> >>> of isolating the changes from the core of the transport. > >>>>>>>>>> >>> > >>>>>>>>>> >>> Andreas > >>>>>>>>>> >>> > >>>>>>>>>> >>>> Thanks, > >>>>>>>>>> >>>> Hiranya > >>>>>>>>>> >>>> > >>>>>>>>>> >>>> > >>>>>>>>>> >>>> > >>>>>>>>>> >>>> On Tue, Jul 21, 2009 at 3:25 PM, Andreas Veithen < > >>>>>>>>>> [email protected]> > >>>>>>>>>> >>>> wrote: > >>>>>>>>>> >>>>> > >>>>>>>>>> >>>>> On Tue, Jul 21, 2009 at 11:30, Oleg Kalnichevski< > >>>>>>>>>> [email protected]> wrote: > >>>>>>>>>> >>>>> > On Tue, Jul 21, 2009 at 11:22:58AM +0200, Andreas Veithen > >>>>>>>>>> wrote: > >>>>>>>>>> >>>>> >> > Well, if not through different stores, how can we let > >>>>>>>>>> the KeyManager > >>>>>>>>>> >>>>> >> > know > >>>>>>>>>> >>>>> >> > what cert to use for this particular endpoint? > >>>>>>>>>> >>>>> >> > >>>>>>>>>> >>>>> >> If I remember well, this is how it works: during the > >>>>>>>>>> handshake, the > >>>>>>>>>> >>>>> >> server presents a list of trusted CAs to the client. The > >>>>>>>>>> client than > >>>>>>>>>> >>>>> >> selects the certificate that is signed (directly or > >>>>>>>>>> indirectly) by > >>>>>>>>>> >>>>> >> that CA and uses that to authenticate. I'm pretty sure > >>>>>>>>>> this is what > >>>>>>>>>> >>>>> >> happens when you create a java.net.URL with the https > >>>>>>>>>> scheme and call > >>>>>>>>>> >>>>> >> openConnection on it. Since behind the scene this uses an > >>>>>>>>>> SSLContext, > >>>>>>>>>> >>>>> >> chances are high that it also works with our HTTPS > >>>>>>>>>> transport (or that > >>>>>>>>>> >>>>> >> it would be pretty easy to make it work). > >>>>>>>>>> >>>>> >> > >>>>>>>>>> >>>>> >> Of course this only satisfies the requirement if the two > >>>>>>>>>> endpoints use > >>>>>>>>>> >>>>> >> different CAs, which should be the usual case. > >>>>>>>>>> >>>>> >> > >>>>>>>>>> >>>>> >> Andreas > >>>>>>>>>> >>>>> >> > >>>>>>>>>> >>>>> > > >>>>>>>>>> >>>>> > Hi Andreas > >>>>>>>>>> >>>>> > > >>>>>>>>>> >>>>> > I may be wrong about it but I believe the client can > >>>>>>>>>> present whatever > >>>>>>>>>> >>>>> > client > >>>>>>>>>> >>>>> > cert it pleases. That cert does not _have_ to be signed by > >>>>>>>>>> one of the > >>>>>>>>>> >>>>> > trusted > >>>>>>>>>> >>>>> > CA certs sent to client by the server. For instance, > >>>>>>>>>> common browsers > >>>>>>>>>> >>>>> > simply pop > >>>>>>>>>> >>>>> > up a UI dialog and let you pick any client certificate > >>>>>>>>>> available in the > >>>>>>>>>> >>>>> > certificate store, if the server requests client > >>>>>>>>>> authentication in the > >>>>>>>>>> >>>>> > course > >>>>>>>>>> >>>>> > of SSL context negotiation. > >>>>>>>>>> >>>>> > > >>>>>>>>>> >>>>> > Oleg > >>>>>>>>>> >>>>> > > >>>>>>>>>> >>>>> > >>>>>>>>>> >>>>> That is possible, but it is only relevant for a scheme where > >>>>>>>>>> the > >>>>>>>>>> >>>>> consumer of the service creates a certificate himself > >>>>>>>>>> (typically a > >>>>>>>>>> >>>>> self-signed certificate) and somehow registers that with the > >>>>>>>>>> provider > >>>>>>>>>> >>>>> of the service. This implies that the provider has to manage > >>>>>>>>>> a list of > >>>>>>>>>> >>>>> recognized client certificates to authenticate the client. I > >>>>>>>>>> don't > >>>>>>>>>> >>>>> think that is a usual scheme for Web services (BTW, how > >>>>>>>>>> would you do > >>>>>>>>>> >>>>> that with Axis2?), but that it is more usual for the > >>>>>>>>>> provider to issue > >>>>>>>>>> >>>>> certificates to the consumer, so that authentication is > >>>>>>>>>> based on the > >>>>>>>>>> >>>>> signature on the client certificate. But again, this is a > >>>>>>>>>> question > >>>>>>>>>> >>>>> about the requirements. > >>>>>>>>>> >>>>> > >>>>>>>>>> >>>>> Andreas > >>>>>>>>>> >>>>> > >>>>>>>>>> >>>>> > > >>>>>>>>>> >>>>> > > >>>>>>>>>> >>>>> >> > >>>>>>>>>> --------------------------------------------------------------------- > >>>>>>>>>> >>>>> >> To unsubscribe, e-mail: > >>>>>>>>>> [email protected] > >>>>>>>>>> >>>>> >> For additional commands, e-mail: > >>>>>>>>>> [email protected] > >>>>>>>>>> >>>>> >> > >>>>>>>>>> >>>>> > > >>>>>>>>>> >>>>> > > >>>>>>>>>> --------------------------------------------------------------------- > >>>>>>>>>> >>>>> > To unsubscribe, e-mail: > >>>>>>>>>> [email protected] > >>>>>>>>>> >>>>> > For additional commands, e-mail: > >>>>>>>>>> [email protected] > >>>>>>>>>> >>>>> > > >>>>>>>>>> >>>>> > > >>>>>>>>>> >>>>> > >>>>>>>>>> >>>>> > >>>>>>>>>> --------------------------------------------------------------------- > >>>>>>>>>> >>>>> To unsubscribe, e-mail: [email protected] > >>>>>>>>>> >>>>> For additional commands, e-mail: > >>>>>>>>>> [email protected] > >>>>>>>>>> >>>>> > >>>>>>>>>> >>>> > >>>>>>>>>> >>>> > >>>>>>>>>> >>>> > >>>>>>>>>> >>>> -- > >>>>>>>>>> >>>> Hiranya Jayathilaka > >>>>>>>>>> >>>> Software Engineer; > >>>>>>>>>> >>>> WSO2 Inc.; http://wso2.org > >>>>>>>>>> >>>> E-mail: [email protected]; Mobile: +94 77 633 3491 > >>>>>>>>>> >>>> Blog: http://techfeast-hiranya.blogspot.com > >>>>>>>>>> >>>> > >>>>>>>>>> >>> > >>>>>>>>>> >>> > >>>>>>>>>> --------------------------------------------------------------------- > >>>>>>>>>> >>> To unsubscribe, e-mail: [email protected] > >>>>>>>>>> >>> For additional commands, e-mail: [email protected] > >>>>>>>>>> >>> > >>>>>>>>>> >>> > >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> >> -- > >>>>>>>>>> >> Paul Fremantle > >>>>>>>>>> >> Co-Founder and CTO, WSO2 > >>>>>>>>>> >> Apache Synapse PMC Chair > >>>>>>>>>> >> OASIS WS-RX TC Co-chair > >>>>>>>>>> >> > >>>>>>>>>> >> blog: http://pzf.fremantle.org > >>>>>>>>>> >> [email protected] > >>>>>>>>>> >> > >>>>>>>>>> >> "Oxygenating the Web Service Platform", www.wso2.com > >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> --------------------------------------------------------------------- > >>>>>>>>>> >> To unsubscribe, e-mail: [email protected] > >>>>>>>>>> >> For additional commands, e-mail: [email protected] > >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> --------------------------------------------------------------------- > >>>>>>>>>> > To unsubscribe, e-mail: [email protected] > >>>>>>>>>> > For additional commands, e-mail: [email protected] > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> Ruwan Linton > >>>>>>>>> Technical Lead & Product Manager; WSO2 ESB; http://wso2.org/esb > >>>>>>>>> WSO2 Inc.; http://wso2.org > >>>>>>>>> email: [email protected]; cell: +94 77 341 3097 > >>>>>>>>> blog: http://ruwansblog.blogspot.com > >>>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> Hiranya Jayathilaka > >>>>>>> Software Engineer; > >>>>>>> WSO2 Inc.; http://wso2.org > >>>>>>> E-mail: [email protected]; Mobile: +94 77 633 3491 > >>>>>>> Blog: http://techfeast-hiranya.blogspot.com > >>>>>>> > >>>>>> > >>>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> Ruwan Linton > >>>>> Technical Lead & Product Manager; WSO2 ESB; http://wso2.org/esb > >>>>> WSO2 Inc.; http://wso2.org > >>>>> email: [email protected]; cell: +94 77 341 3097 > >>>>> blog: http://ruwansblog.blogspot.com > >>>>> > >>>> > >>>> > >>> > >>> > >>> -- > >>> Ruwan Linton > >>> Technical Lead & Product Manager; WSO2 ESB; http://wso2.org/esb > >>> WSO2 Inc.; http://wso2.org > >>> email: [email protected]; cell: +94 77 341 3097 > >>> blog: http://ruwansblog.blogspot.com > >>> > >> > >> > > > > > > -- > > Software Engineer, WSO2 Inc > > http://wso2.org > > supunk.blogspot.com > > > > > > > > > -- > Hiranya Jayathilaka > Software Engineer; > WSO2 Inc.; http://wso2.org > E-mail: [email protected]; Mobile: +94 77 633 3491 > Blog: http://techfeast-hiranya.blogspot.com --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
