On Sun, Apr 3, 2016 at 6:48 AM, Sanjiva Weerawarana <[email protected]> wrote:
> :-). I think there are two things we should show: > - one can do circuit breaker (CB) with MSF4J > - when should you use it > > Wrapping of DB calls in Hystrix may indeed be appropriate in some cases, > but not commonly. Second, at that level its really not about MSF4J at all. > Yes, that is correct, MSF4J after all is just a Java library, but people ask how to do do xyz with MSF4J. So in my recent blogposts, I have been trying to make it easy for people to find the answers through Googling for some of the questions I have received from people whom I have spoken to. http://techblog.netflix.com/2011/12/making-netflix-api-more-resilient.html is a good read, and for one of the use cases, they wrap a DB select query in a circuit breaker since if that fails, it is possible to fallback to the cache. > > If we show a sample that's a service invoking another service or some > random Java code invoking a service using CB then that's better IMO. We > should mention when it is indeed appropriate to use as a fancy try catch. > > For example, in the registry code we have scenarios where there can be > transient DB issues because of concurrent access. Should we use CB there to > make that code more resilient? Almost always the transaction "failure" in > those cases is a temporary issue and a non-issue ... just bad timing. > For transient issues, the common way to try to recover is retrying, and after a few retries (we can use binary backoff with timeout), it could be possible to have some sort of fallback, perhaps send a notification about the failure & serve from a cache until it times out. > > If we can make our code better with CB we absolutely MUST do it. The key > is to provide guidance on when its appropriate to do it! > Yes, agreed. > > Sanjiva. > > On Sat, Apr 2, 2016 at 8:24 AM, Afkham Azeez <[email protected]> wrote: > >> Well one of your responses indicated that the the blogpost suggests >> wrapping all DB calls in circuit breakers, and Frank's response indicated >> that it is suggesting wrapping all calls to external systems in circuit >> breakers. If it can confuse such brilliant minds, I guess it could mislead >> an average developer. That blogpost was on how to implement the pattern in >> the context of MSF4J using Hystrix, and I think it should have been >> preceded with a blogpost introducing the pattern & explaining where it is >> applicable. Sorry for the confusion again. I will work on a separate post. >> >> Thanks >> Azeez >> >> On Fri, Apr 1, 2016 at 11:59 PM, Sanjiva Weerawarana <[email protected]> >> wrote: >> >>> Azeez, asking questions and asking to understand what the purpose of >>> some code is not a criticism. >>> >>> Of course there should be a sample! I only asked about it because to me >>> a client side sample made more sense - and then it went into a discussion >>> of when this should be used etc.. >>> >>> I would not only put the blog back but also use the opportunity to >>> explain to "lazy developers" when and where one should use a >>> circuitbreaker. That's 1000x more valuable than just the code on how to do >>> it. >>> >>> Sanjiva. >>> >>> On Thu, Mar 31, 2016 at 8:00 PM, Afkham Azeez <[email protected]> wrote: >>> >>>> The blog post has been removed. Sorry for all the confusion. This was >>>> only done as part of the agreement we had during last week's meeting to >>>> demonstrate certain features such as Spring support, JPA support, support >>>> for patterns etc. in order to help developers understand how to implement >>>> certain stuff with MSF4J. Our target community of MSF4J is primarily >>>> developers, and developers like to refer to sample code segments in order >>>> to proceed with implementing their solutions. We lazy developers love to >>>> Google search for code segments, e.g. JDBC connection example, and then >>>> copy, paste and modify those segments. What I have been trying to do with >>>> the series of blogposts is to make available such code segments developers >>>> could readily use. Since this post and mail thread has generated a lot of >>>> negative feeling and confusion, I think it is better to get rid of this >>>> controversial blog post. >>>> >>>> Thanks >>>> Azeez >>>> >>>> On Thu, Mar 31, 2016 at 6:54 PM, Afkham Azeez <[email protected]> wrote: >>>> >>>>> >>>>> >>>>> On Thu, Mar 31, 2016 at 5:03 PM, Frank Leymann <[email protected]> wrote: >>>>> >>>>>> Yes, all the stability patterns (that Nygard describes, the circuit >>>>>> breaker being just one of them) >>>>>> is not associated with microservices, but applies to all distributed >>>>>> applications. In fact, Nygard's >>>>>> book has been published in 2007, loooong before the microservice >>>>>> discussion came up ;-) >>>>>> >>>>>> Yes Frank, agreed. With the hype about microservices, people have >>>>> started talking about these a lot and during the evaluation phase, people >>>>> look at features available in frameworks. I don't understand the >>>>> excitement >>>>> here. We are not saying CircuitBreaker etc have to be used. That is as >>>>> stupid as saying every object instantiation has to be done via a Factory. >>>>> >>>>> >>>>>> Applying these patterns to each and any invocation would be a >>>>>> complete misuse. >>>>>> >>>>> >>>>> Yes, it would very stupid for someone to design a system like that or >>>>> to suggest something like that, like I said it would be like asking to >>>>> instantiate all objects using the Factory pattern! >>>>> >>>>> Patterns are just part of the toolkit of architects & developers. >>>>> Knowing to use the appropriate one at the appropriate place requires >>>>> proper >>>>> judgment. This sample nor this mail thread is not suggesting to use these >>>>> everywhere, and I don't understand what gave the impression that we are >>>>> suggesting such a thing. >>>>> >>>>> >>>>>> And it will very likely >>>>>> result in performance hits... The circuit breaker pattern, for >>>>>> example, is recommended to be applied >>>>>> to "out-of-spec errors", i.e. errors that you don't cover in the spec >>>>>> (because the errors are too unlikely ;-)) >>>>>> of the invoked function or in the spec of the program making the call >>>>>> (aka client). Often, these are errors >>>>>> that never happen during testing unless you really set up a badly >>>>>> behaving test environment. And it has >>>>>> impact on the design/implementation of the circuit breaker itself (or >>>>>> clients), for example "critical work" >>>>>> not accepted by the circuit breaker has be queued (by the client? by >>>>>> the circuit breaker?) for later use >>>>>> (automatic replay?). >>>>>> >>>>>> Thus, using one of the stability patterns is a (architecture/design) >>>>>> decision with implications on other >>>>>> components architecture/design. >>>>>> >>>>>> Documenting a sample use of the circuit breaker pattern should also >>>>>> discuss these ramifications. >>>>>> >>>>>> >>>>> Thanks. We will include these recommendations in our documentation. >>>>> >>>>> >>>>>> >>>>>> Best regards, >>>>>> Frank >>>>>> >>>>>> 2016-03-31 9:12 GMT+02:00 Sanjiva Weerawarana <[email protected]>: >>>>>> >>>>>>> Agreed. However I had understood that the circuit breaker pattern >>>>>>> was advocated primarily for service clients in MSA (and of course it has >>>>>>> nothing do with being micro). >>>>>>> >>>>>>> The general story of better failure handling applies to all code and >>>>>>> is of course not MSA specific. >>>>>>> >>>>>>> Anyway .. Sample is fine. >>>>>>> On Mar 31, 2016 9:19 AM, "Afkham Azeez" <[email protected]> wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Mar 31, 2016 at 9:04 AM, Sanjiva Weerawarana < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> That's why I said "fancy try catch" :-). >>>>>>>>> >>>>>>>>> However, are you SERIOUSLY saying that we for example should be >>>>>>>>> wrapping all our DB access code in this stuff? If not who exactly >>>>>>>>> should be >>>>>>>>> doing this? What are the perf implications? >>>>>>>>> >>>>>>>> >>>>>>>> No I am not saying that. However, there will be use cases where >>>>>>>> people want to use this pattern and this is a simplified sample that >>>>>>>> demonstrates how to use this pattern. In Nygards book about how an SQL >>>>>>>> statement execution failure resulted in an entire checking in system >>>>>>>> in an >>>>>>>> airline failing because the failure propagated is a good example of >>>>>>>> uncontrolled failure propagation (Release It, Chapter 2: Case study: >>>>>>>> The >>>>>>>> exception that grounded an airline, for those of you who have the >>>>>>>> book). So >>>>>>>> my example was somewhat inspired by that case study and is highly >>>>>>>> simplified. >>>>>>>> >>>>>>>> If a sample is too complicated, people get lost in the >>>>>>>> implementation details rather than seeing how the core concept or >>>>>>>> pattern >>>>>>>> is implemented. I certainly can implement another sample which >>>>>>>> demonstrates >>>>>>>> client->service or service->service calls, it certainly would add more >>>>>>>> code >>>>>>>> but the core concept demonstrated would be the same. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Of course wrapping remote service calls in this stuff makes sense >>>>>>>>> - great way to adjust to transient issues. In that case the overhead >>>>>>>>> is >>>>>>>>> heavily masked by the latency - I'm not so convinced that is the case >>>>>>>>> for >>>>>>>>> transactional JDBC calls but maybe it is. In that case WE must use it >>>>>>>>> internally. >>>>>>>>> >>>>>>>>> Sanjiva. >>>>>>>>> >>>>>>>>> On Thu, Mar 31, 2016 at 8:53 AM, Afkham Azeez <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Equating these fault tolerance patterns to Java 8 Optional or >>>>>>>>>> try-catch, is a highly oversimplified view. What Hystrix and these >>>>>>>>>> patterns >>>>>>>>>> provides is a framework for building fault tolerant systems. >>>>>>>>>> Something that >>>>>>>>>> is useful in the toolkit of an architect & developer. >>>>>>>>>> >>>>>>>>>> On Thu, Mar 31, 2016 at 8:36 AM, Sanjiva Weerawarana < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> This is almost kinda like that stupid new Java8 thing of "we >>>>>>>>>>> removed null by wrapping it in a fancy object" ;-). >>>>>>>>>>> >>>>>>>>>>> On Thu, Mar 31, 2016 at 8:32 AM, Sanjiva Weerawarana < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> So this is not what I expected the real use case to be ... this >>>>>>>>>>>> is basically a fancy try catch. >>>>>>>>>>>> >>>>>>>>>>>> Don't we want to show a client side example? >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Mar 31, 2016 at 6:28 AM, Afkham Azeez <[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Timeout is related to the actual operation taking more time >>>>>>>>>>>>> than anticipated. In such a case, without waiting indefinitely, >>>>>>>>>>>>> the >>>>>>>>>>>>> operation times out and the fallback of the Hystrix command will >>>>>>>>>>>>> be >>>>>>>>>>>>> invoked. The circuit will be open for a fixed period of time >>>>>>>>>>>>> configured by >>>>>>>>>>>>> https://github.com/Netflix/Hystrix/wiki/Configuration#circuitBreaker.sleepWindowInMilliseconds >>>>>>>>>>>>> >>>>>>>>>>>>> On Thu, Mar 31, 2016 at 2:53 AM, Harshan Liyanage < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Azeez, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Does this timeout in open state occurs in exponentially >>>>>>>>>>>>>> (first timeout in 10 secs, next in 20 secs etc) or linearly when >>>>>>>>>>>>>> transitioning back to half-open state? For example if the state >>>>>>>>>>>>>> is in >>>>>>>>>>>>>> "Open" and now the timeout (lets say 10secs timeout) occurs. >>>>>>>>>>>>>> Then the state >>>>>>>>>>>>>> is moved to "half-open" state. But the next request is also a >>>>>>>>>>>>>> failure and >>>>>>>>>>>>>> breaker state is moved back to "open". In this occasion the what >>>>>>>>>>>>>> will be >>>>>>>>>>>>>> the timeout value? Is it 10 secs or 20 secs? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Having an exponential timeout might be beneficiary here as it >>>>>>>>>>>>>> might save lot of resources if the service is continuously >>>>>>>>>>>>>> failing. But I >>>>>>>>>>>>>> think it would be better if we can provide both options in a >>>>>>>>>>>>>> configurable >>>>>>>>>>>>>> manner. So it is up to the developer to decide which method to >>>>>>>>>>>>>> use. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Harshan Liyanage >>>>>>>>>>>>>> Software Engineer >>>>>>>>>>>>>> Mobile: *+94724423048* >>>>>>>>>>>>>> Email: [email protected] >>>>>>>>>>>>>> Blog : http://harshanliyanage.blogspot.com/ >>>>>>>>>>>>>> *WSO2, Inc. :** wso2.com <http://wso2.com/>* >>>>>>>>>>>>>> lean.enterprise.middleware. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, Mar 30, 2016 at 5:05 AM, Afkham Azeez <[email protected] >>>>>>>>>>>>>> > wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> I have written a sample which demonstrates circuit breaker >>>>>>>>>>>>>>> in action; >>>>>>>>>>>>>>> http://blog.afkham.org/2016/03/microservices-circuit-breaker.html >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Sat, Mar 12, 2016 at 6:09 PM, Afkham Azeez < >>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> This is a feature supported by some microservices >>>>>>>>>>>>>>>> frameworks. On the server side, in this case MSF4J runtime, >>>>>>>>>>>>>>>> failure counts >>>>>>>>>>>>>>>> are kept track of and then if the failures exceed certain >>>>>>>>>>>>>>>> thresholds, the >>>>>>>>>>>>>>>> circuit trips and rather than dispatch to the service, it >>>>>>>>>>>>>>>> returns service >>>>>>>>>>>>>>>> unavailable. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Can you explain why this is not needed in a container >>>>>>>>>>>>>>>> environment? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Sat, Mar 12, 2016 at 12:56 PM, Sanjiva Weerawarana < >>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I don't understand what server side circuit breaker means. >>>>>>>>>>>>>>>>> How does the server adjust itself? Where's that bit of logic >>>>>>>>>>>>>>>>> running? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> IMO this is not needed in a container world. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Fri, Mar 11, 2016 at 4:38 PM, Afkham Azeez < >>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yes, that is client side circuit breaker. What Aruna is >>>>>>>>>>>>>>>>>> implementing is server side circuit breaker. Yes, we need >>>>>>>>>>>>>>>>>> both. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Fri, Mar 11, 2016 at 4:04 PM, Lakmal Warusawithana < >>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Did you looked at [1] >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Netflix Hystrix <https://github.com/Netflix/Hystrix> is >>>>>>>>>>>>>>>>>>> an incredibly useful library for writing code that invokes >>>>>>>>>>>>>>>>>>> remote services. >>>>>>>>>>>>>>>>>>> Hystrix times out calls that exceed the specified >>>>>>>>>>>>>>>>>>> threshold. It implements >>>>>>>>>>>>>>>>>>> a *circuit breaker* pattern, which stops the client >>>>>>>>>>>>>>>>>>> from waiting needlessly for an unresponsive service. If the >>>>>>>>>>>>>>>>>>> error rate for >>>>>>>>>>>>>>>>>>> a service exceeds a specified threshold, Hystrix trips the >>>>>>>>>>>>>>>>>>> circuit breaker >>>>>>>>>>>>>>>>>>> and all requests will fail immediately for a specified >>>>>>>>>>>>>>>>>>> period of time. >>>>>>>>>>>>>>>>>>> Hystrix lets you define a fallback action when a request >>>>>>>>>>>>>>>>>>> fails, such as >>>>>>>>>>>>>>>>>>> reading from a cache or returning a default value. If you >>>>>>>>>>>>>>>>>>> are using the JVM >>>>>>>>>>>>>>>>>>> you should definitely consider using Hystrix. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> [1] https://github.com/Netflix/Hystrix >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Fri, Mar 11, 2016 at 2:42 PM, Aruna Karunarathna < >>>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Hi Devs, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> *Scenario* >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> The deployed services in a MSF4J may fail to serve the >>>>>>>>>>>>>>>>>>>> requests due to various factors. e.g, >>>>>>>>>>>>>>>>>>>> 1. Less resources in the server. >>>>>>>>>>>>>>>>>>>> 2. High Load in the server >>>>>>>>>>>>>>>>>>>> 3, Some services take more time to respond etc. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> In this kind of a situation, if the server is getting >>>>>>>>>>>>>>>>>>>> requests though there is no resources to serve those >>>>>>>>>>>>>>>>>>>> requests, and >>>>>>>>>>>>>>>>>>>> eventually the server will get unusable. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> *Solution* >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> The Circuit Breaker design pattern can save the server >>>>>>>>>>>>>>>>>>>> from above scenarios, The typical design can be >>>>>>>>>>>>>>>>>>>> illustrated as in the >>>>>>>>>>>>>>>>>>>> following diagram. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> So as in the above diagram, when number of failures of >>>>>>>>>>>>>>>>>>>> a particular resource exceeds the Max Failure Count, then >>>>>>>>>>>>>>>>>>>> the state of that >>>>>>>>>>>>>>>>>>>> resource is moved to the open state with a timeout value >>>>>>>>>>>>>>>>>>>> (Trip Breaker). At >>>>>>>>>>>>>>>>>>>> this point the requests coming to the server is routed >>>>>>>>>>>>>>>>>>>> back without passing >>>>>>>>>>>>>>>>>>>> the internal to process further. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> After the timeout has reached, the state is moved to >>>>>>>>>>>>>>>>>>>> Half-Open state, and if the consecutive request pass to >>>>>>>>>>>>>>>>>>>> the server to >>>>>>>>>>>>>>>>>>>> process (Attempt Reset), if success then close the circuit >>>>>>>>>>>>>>>>>>>> (Reset Breaker), >>>>>>>>>>>>>>>>>>>> If fail then again move the state to the Open with a >>>>>>>>>>>>>>>>>>>> timeout value (Trip >>>>>>>>>>>>>>>>>>>> Breaker). >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Any thoughts, suggestions regarding the above approach? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> References >>>>>>>>>>>>>>>>>>>> [1]. >>>>>>>>>>>>>>>>>>>> http://www.javaworld.com/article/2824163/application-performance/stability-patterns-applied-in-a-restful-architecture.html?page=2 >>>>>>>>>>>>>>>>>>>> [2]. >>>>>>>>>>>>>>>>>>>> http://ssagara.blogspot.com/2015/05/timeout-and-circuit-breaker-pattern-in.html >>>>>>>>>>>>>>>>>>>> [3]. https://pragprog.com/book/mnee/release-it >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>>>> Aruna >>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> *Aruna Sujith Karunarathna * >>>>>>>>>>>>>>>>>>>> WSO2, Inc | lean. enterprise. middleware. >>>>>>>>>>>>>>>>>>>> #20, Palm Grove, Colombo 03, Sri Lanka >>>>>>>>>>>>>>>>>>>> Mobile: +94 71 9040362 | Work: +94 112145345 >>>>>>>>>>>>>>>>>>>> Email: [email protected] | Web: www.wso2.com >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>>>>>>> Architecture mailing list >>>>>>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>> Lakmal Warusawithana >>>>>>>>>>>>>>>>>>> Director - Cloud Architecture; WSO2 Inc. >>>>>>>>>>>>>>>>>>> Mobile : +94714289692 >>>>>>>>>>>>>>>>>>> Blog : http://lakmalsview.blogspot.com/ >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>> *Afkham Azeez* >>>>>>>>>>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>>>>>>>>>> Member; Apache Software Foundation; >>>>>>>>>>>>>>>>>> http://www.apache.org/ >>>>>>>>>>>>>>>>>> * <http://www.apache.org/>* >>>>>>>>>>>>>>>>>> *email: **[email protected]* <[email protected]> >>>>>>>>>>>>>>>>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * >>>>>>>>>>>>>>>>>> *http://blog.afkham.org* <http://blog.afkham.org> >>>>>>>>>>>>>>>>>> *twitter: **http://twitter.com/afkham_azeez* >>>>>>>>>>>>>>>>>> <http://twitter.com/afkham_azeez> >>>>>>>>>>>>>>>>>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez >>>>>>>>>>>>>>>>>> <http://lk.linkedin.com/in/afkhamazeez>* >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>>>>> Architecture mailing list >>>>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>> Sanjiva Weerawarana, Ph.D. >>>>>>>>>>>>>>>>> Founder, CEO & Chief Architect; WSO2, Inc.; >>>>>>>>>>>>>>>>> http://wso2.com/ >>>>>>>>>>>>>>>>> email: [email protected]; office: (+1 650 745 4499 | +94 >>>>>>>>>>>>>>>>> 11 214 5345) x5700; cell: +94 77 787 6880 | +1 408 466 >>>>>>>>>>>>>>>>> 5099; voip: +1 650 265 8311 >>>>>>>>>>>>>>>>> blog: http://sanjiva.weerawarana.org/; twitter: @sanjiva >>>>>>>>>>>>>>>>> Lean . Enterprise . Middleware >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>>>> Architecture mailing list >>>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> *Afkham Azeez* >>>>>>>>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>>>>>>>> * <http://www.apache.org/>* >>>>>>>>>>>>>>>> *email: **[email protected]* <[email protected]> >>>>>>>>>>>>>>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * >>>>>>>>>>>>>>>> *http://blog.afkham.org* <http://blog.afkham.org> >>>>>>>>>>>>>>>> *twitter: **http://twitter.com/afkham_azeez* >>>>>>>>>>>>>>>> <http://twitter.com/afkham_azeez> >>>>>>>>>>>>>>>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez >>>>>>>>>>>>>>>> <http://lk.linkedin.com/in/afkhamazeez>* >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> *Afkham Azeez* >>>>>>>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>>>>>>> * <http://www.apache.org/>* >>>>>>>>>>>>>>> *email: **[email protected]* <[email protected]> >>>>>>>>>>>>>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * >>>>>>>>>>>>>>> *http://blog.afkham.org* <http://blog.afkham.org> >>>>>>>>>>>>>>> *twitter: **http://twitter.com/afkham_azeez* >>>>>>>>>>>>>>> <http://twitter.com/afkham_azeez> >>>>>>>>>>>>>>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez >>>>>>>>>>>>>>> <http://lk.linkedin.com/in/afkhamazeez>* >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>> Architecture mailing list >>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>> Architecture mailing list >>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> *Afkham Azeez* >>>>>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>>>>> * <http://www.apache.org/>* >>>>>>>>>>>>> *email: **[email protected]* <[email protected]> >>>>>>>>>>>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * >>>>>>>>>>>>> *http://blog.afkham.org* <http://blog.afkham.org> >>>>>>>>>>>>> *twitter: **http://twitter.com/afkham_azeez* >>>>>>>>>>>>> <http://twitter.com/afkham_azeez> >>>>>>>>>>>>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez >>>>>>>>>>>>> <http://lk.linkedin.com/in/afkhamazeez>* >>>>>>>>>>>>> >>>>>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>>>>> >>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>> Architecture mailing list >>>>>>>>>>>>> [email protected] >>>>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Sanjiva Weerawarana, Ph.D. >>>>>>>>>>>> Founder, CEO & Chief Architect; WSO2, Inc.; http://wso2.com/ >>>>>>>>>>>> email: [email protected]; office: (+1 650 745 4499 | +94 11 >>>>>>>>>>>> 214 5345) x5700; cell: +94 77 787 6880 | +1 408 466 5099; >>>>>>>>>>>> voip: +1 650 265 8311 >>>>>>>>>>>> blog: http://sanjiva.weerawarana.org/; twitter: @sanjiva >>>>>>>>>>>> Lean . Enterprise . Middleware >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Sanjiva Weerawarana, Ph.D. >>>>>>>>>>> Founder, CEO & Chief Architect; WSO2, Inc.; http://wso2.com/ >>>>>>>>>>> email: [email protected]; office: (+1 650 745 4499 | +94 11 214 >>>>>>>>>>> 5345) x5700; cell: +94 77 787 6880 | +1 408 466 5099; voip: +1 >>>>>>>>>>> 650 265 8311 >>>>>>>>>>> blog: http://sanjiva.weerawarana.org/; twitter: @sanjiva >>>>>>>>>>> Lean . Enterprise . Middleware >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Architecture mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> *Afkham Azeez* >>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>> * <http://www.apache.org/>* >>>>>>>>>> *email: **[email protected]* <[email protected]> >>>>>>>>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * >>>>>>>>>> *http://blog.afkham.org* <http://blog.afkham.org> >>>>>>>>>> *twitter: **http://twitter.com/afkham_azeez* >>>>>>>>>> <http://twitter.com/afkham_azeez> >>>>>>>>>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez >>>>>>>>>> <http://lk.linkedin.com/in/afkhamazeez>* >>>>>>>>>> >>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Architecture mailing list >>>>>>>>>> [email protected] >>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Sanjiva Weerawarana, Ph.D. >>>>>>>>> Founder, CEO & Chief Architect; WSO2, Inc.; http://wso2.com/ >>>>>>>>> email: [email protected]; office: (+1 650 745 4499 | +94 11 214 >>>>>>>>> 5345) x5700; cell: +94 77 787 6880 | +1 408 466 5099; voip: +1 >>>>>>>>> 650 265 8311 >>>>>>>>> blog: http://sanjiva.weerawarana.org/; twitter: @sanjiva >>>>>>>>> Lean . Enterprise . Middleware >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Architecture mailing list >>>>>>>>> [email protected] >>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Afkham Azeez* >>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>> * <http://www.apache.org/>* >>>>>>>> *email: **[email protected]* <[email protected]> >>>>>>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * >>>>>>>> *http://blog.afkham.org* <http://blog.afkham.org> >>>>>>>> *twitter: **http://twitter.com/afkham_azeez* >>>>>>>> <http://twitter.com/afkham_azeez> >>>>>>>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez >>>>>>>> <http://lk.linkedin.com/in/afkhamazeez>* >>>>>>>> >>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Architecture mailing list >>>>>>>> [email protected] >>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>> >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> Architecture mailing list >>>>>>> [email protected] >>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>> >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> [email protected] >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> *Afkham Azeez* >>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>> * <http://www.apache.org/>* >>>>> *email: **[email protected]* <[email protected]> >>>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * >>>>> *http://blog.afkham.org* <http://blog.afkham.org> >>>>> *twitter: **http://twitter.com/afkham_azeez* >>>>> <http://twitter.com/afkham_azeez> >>>>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez >>>>> <http://lk.linkedin.com/in/afkhamazeez>* >>>>> >>>>> *Lean . Enterprise . Middleware* >>>>> >>>> >>>> >>>> >>>> -- >>>> *Afkham Azeez* >>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>> Member; Apache Software Foundation; http://www.apache.org/ >>>> * <http://www.apache.org/>* >>>> *email: **[email protected]* <[email protected]> >>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * >>>> *http://blog.afkham.org* <http://blog.afkham.org> >>>> *twitter: **http://twitter.com/afkham_azeez* >>>> <http://twitter.com/afkham_azeez> >>>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez >>>> <http://lk.linkedin.com/in/afkhamazeez>* >>>> >>>> *Lean . Enterprise . Middleware* >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> Sanjiva Weerawarana, Ph.D. >>> Founder, CEO & Chief Architect; WSO2, Inc.; http://wso2.com/ >>> email: [email protected]; office: (+1 650 745 4499 | +94 11 214 5345) >>> x5700; cell: +94 77 787 6880 | +1 408 466 5099; voip: +1 650 265 8311 >>> blog: http://sanjiva.weerawarana.org/; twitter: @sanjiva >>> Lean . Enterprise . Middleware >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> *Afkham Azeez* >> Director of Architecture; WSO2, Inc.; http://wso2.com >> Member; Apache Software Foundation; http://www.apache.org/ >> * <http://www.apache.org/>* >> *email: **[email protected]* <[email protected]> >> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * >> *http://blog.afkham.org* <http://blog.afkham.org> >> *twitter: **http://twitter.com/afkham_azeez* >> <http://twitter.com/afkham_azeez> >> *linked-in: **http://lk.linkedin.com/in/afkhamazeez >> <http://lk.linkedin.com/in/afkhamazeez>* >> >> *Lean . Enterprise . Middleware* >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Sanjiva Weerawarana, Ph.D. > Founder, CEO & Chief Architect; WSO2, Inc.; http://wso2.com/ > email: [email protected]; office: (+1 650 745 4499 | +94 11 214 5345) > x5700; cell: +94 77 787 6880 | +1 408 466 5099; voip: +1 650 265 8311 > blog: http://sanjiva.weerawarana.org/; twitter: @sanjiva > Lean . Enterprise . Middleware > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- *Afkham Azeez* Director of Architecture; WSO2, Inc.; http://wso2.com Member; Apache Software Foundation; http://www.apache.org/ * <http://www.apache.org/>* *email: **[email protected]* <[email protected]> * cell: +94 77 3320919blog: **http://blog.afkham.org* <http://blog.afkham.org> *twitter: **http://twitter.com/afkham_azeez* <http://twitter.com/afkham_azeez> *linked-in: **http://lk.linkedin.com/in/afkhamazeez <http://lk.linkedin.com/in/afkhamazeez>* *Lean . Enterprise . Middleware*
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
