RE: IRC chat log Thursday 23 Feb 2006

Soumadeep Fri, 24 Feb 2006 05:15:35 -0800

[Soumadeep] :-)) good thinking... what you missed out is that, at the mediator level the messages will be decrypted and also it might be the provider's intent to store the encrypted message itself to ascertain the authenticity of the message!! So you see it depends at which stage you are storing the SOAP message and for what purpose.

By the way if you want to store decrypted data then you can use the Security Mediator which will decrypt the data for you, otherwise the provider service will never be able to serve you right :-)) !

And secondly, it doesn't make any difference how you assimilate the management data as long as it can be identified by the receiver of the data, at the instrumentation level it can be separated and used appropriately. And more interestingly its mostly an aggregation of service related data!

To make it a little simpler for you , let's say the instrumentation is being done using WSDM. What a WSDM browser/App will request for is getRequestCount (mind you this will be again in a WSDM compatible request format) , now nowhere you will see that the requestCount is being sent as part of the management information. What I am trying to emphasize is that it's not Synapses' responsibility to separate, maintain or interpret management information but it's the responsibility of the manageability Endpoint. It holds true if you use JMX (MBeans)

PS: Ideas are ideas it's one's perspective that makes it meaningful or smart !

-----Original Message-----
From: Saminda Abeyruwan [mailto:[EMAIL PROTECTED]
Sent: Friday, February 24, 2006 5:21 PM
To: [email protected]
Subject: Re: IRC chat log Thursday 23 Feb 2006

Hi,

On 2/23/06, Soumadeep <[EMAIL PROTECTED]> wrote:

Proposal

-------------

2) Inferring Management related information from mediators is not possible

Going forward, monitoring of the mediators along with the service will be required to meaningfully ascertain that the system is behaving and performing as desired. This leads to the question of what information do we mandate in terms of monitoring. What are we monitoring? What is the target application that is going to process the information?

The possible technology to send events could be:
a) JMX
b) JMS
This leads us to having a Management Information Exchange object in the message context, which could gather all the relevant information from and for the mediators and would be consumed by a JMX or JMS mediator, which would finally send the event. The EventMediator would have it's own configuration file which will feed the details in regards to the target or where to send the information.
Proposed format of MIX (Management Information Exchange)

Ok

<ManagementInformationExchange>


<EndPointReference>String</EndPointReference>


<TransactionID>String</TransactionID>


<TimeStamp>2001-12-17T09:30:47.0Z</TimeStamp>


<Operation>String</Operation>


<ClientIP>String</ClientIP>


<UserAgent>String</UserAgent>


<ProviderEPR>String</ProviderEPR>


<SOAPRequest>String</SOAPRequest>


<SOAPResponse>String</SOAPResponse>


<ExecutionStatus>
<Synapse>true</Synapse>
<Provider>true</Provider>
</ExecutionStatus>


<ExecutionTime>
<TotalRoundTrip>2147483647</TotalRoundTrip>
<ProviderExecutionTime>2147483647</ProviderExecutionTime>
<MediatorExecutionTime>
   <Mediator name="M1">2147483647</Mediator>
   <Mediator name="M2">2147483647</Mediator>
   <Mediator name="M3">2147483647</Mediator>
</MediatorExecutionTime>
</ExecutionTime>
</ManagementInformationExchange>

The prior xml contains all mediator level, service level and transport level information. IMHO they should be separated, and should be managed separately. if <SOAPReques/> and <SOAPResponse/> shows the SOAP message itself, i don't think it's a smart idea. {message may be encrypted}.

Saminda

-----Original Message-----
From: ant elder [mailto:[EMAIL PROTECTED]]
Sent: Thursday, February 23, 2006 7:49 PM
To: [email protected]
Subject: IRC chat log Thursday 23 Feb 2006

Session Start: Thu Feb 23 06:22:34 2006
Session Ident: #apache-synapse
* Now talking in #apache-synapse
* saminda has joined #apache-synapse
* Hariharasudhan has joined #apache-synapse
* pvikas has joined #apache-synapse
<saminda> Hi all
* Rajesh_Koilpilla has joined #apache-synapse
<pvikas> hi
<Hariharasudhan> hi saminda
* Soumadeep has joined #apache-synapse
<Soumadeep> hi
<saminda> guys, please be kind enough to let consensus on what to expose on JMX interface
<saminda> Hi
<saminda> ex: addProcessor(). etc
<saminda> thus, we can give a management interface with m2
<ant_> hi everyone
<saminda> hi ant
<Rajesh_Koilpilla> These are some of the interfaces which we could think of Synapse to do from a management perspective
* Looking up Hariharasudhan user info...
<Rajesh_Koilpilla> JSR 77 has some generic interfaces which lets you manage any kind of Object (EJB, JMS object, JNDI resource). i.e any kind of control on these objects to modify their state
* paulfremantle has joined #apache-synapse
<ant_> morning paul
<Rajesh_Koilpilla> Synapse as a engine intends to manage mediations between multiple native provider services
<saminda> Hi Paul
<paulfremantle> hi
<paulfremantle> sorry im late
<Soumadeep> hi Paul...
<paulfremantle> didnt expect you on here Ant!
<Soumadeep> Hi everyone
<Rajesh_Koilpilla> So far each mediation in question, it could have a generic interface which could identify the resource and perform state operations on it like start, stop or the ability to pause.
<Rajesh_Koilpilla> Instead of mediation on the above sentence, I would rather say it's for a particular native endpoint
<Rajesh_Koilpilla> this way you can temporarily control the way a particular native resource is being handled
<Rajesh_Koilpilla> The other operation could be is to get health statistics on a particular managed resource (like ping operation or heart beats in a particular duration)
<Rajesh_Koilpilla> This will enable the client management console to run some health checks on the managed endpoint in question.
<pvikas> [rajesh] that means we put management as a functionality into the core of synapse and its not just another mediator...
<Rajesh_Koilpilla> yes - I think we should come to the question at the end of how the management interface can be made to sit as a mediator
<saminda> management should be expose as another mediator IMHO
<Soumadeep> Keeping in line with this... I guess there are few things which we might need to think over...
<Soumadeep> Mediators have no way of sharing data in a structured way
<Soumadeep> Inferring Management related information from mediators is not possible
<Soumadeep> Mediator related configuration data is not structured
<pvikas> isn't management more about collecting performance data, monitoring..etc, adding processors, mediators through mbeans would only equal hot-deployment....and a single mediator would not be in a position to collect data about every aspect
<Rajesh_Koilpilla> The other set of management capabilities could include attributes related to aggregate statistics on a managed resource endpoint
<Rajesh_Koilpilla> a] How many hits was processed by a particular endpoint
<Rajesh_Koilpilla> b] What was the last response time
<Rajesh_Koilpilla> c] What was the average response time
<Rajesh_Koilpilla> d] What was the fault count
<paulfremantle> i dont understand why we couldnt collect all that from just a mediator
<Soumadeep> Paul, you can't
<saminda> why?
<Rajesh_Koilpilla> Paul / Saminda - Can you explain how to route all requests to a particular mediator, which will expose management information
<paulfremantle> you can put it first in the chain
<Rajesh_Koilpilla> that will clear some of the doubts which others have
<saminda> how does <log/> works for all messages
<Soumadeep> log is not mediator specific
<paulfremantle> i dont understand that sentence soumadeep
<saminda> then what it would be :)
<Soumadeep> log is not part of management info
* Karni has joined #apache-synapse
<pvikas> log doesn't give us any data on the mediators...
<paulfremantle> but vikas
<paulfremantle> the a/b/c/d offered above
<paulfremantle> are not data on mediators
<saminda> since SynapseEnvironment is accessible with any mediator, we can get the other mediators pointer from that
<paulfremantle> but on the interactions
<pvikas> the performance statistics of a mediator, wether it allowed a request to pass or not, etc; can not be collected by a single mediator..it becomes the functionality of the core..
<ant_> aren't you going to want to gather those stats for every mediator? And if so doesn't that mean it needs to be baked into the runtime
<pvikas> and this is also management related info IMHO...
* venkat has joined #apache-synapse
<Soumadeep> are we confusing ourselves between logging and monitoring data??
<Soumadeep> Paul, as I had mentioned earlier... that we need a clocking mechanism fo rmediators...
<Soumadeep> which can't be done by a mediator... but needs to be done at the processor level...
<paulfremantle> i dont agree
<saminda> theres only one thing exist after m1, mediators not processors :)
<Soumadeep> By processor I mean the functionality that processes the mediators... invokes them...
<saminda> with the current termonology, every processor is a mediator, right paul
<saminda> so mediators can handle there timing, right
<saminda> ??
* chinthaka has joined #apache-synapse
<Soumadeep> Paul can you elaborate on what or why you don't agree...:-)
<paulfremantle> any mediator can put timing info into the context
<saminda> hi chinthaka!
<Rajesh_Koilpilla> Ant - Yes every mediator could have it's own specific metadata which needs to be manageable
<saminda> now there should be a generic way to expose those metadata of a mediator
<paulfremantle> sorry
<paulfremantle> back
<paulfremantle> i got called
<saminda> maybe a getMetada() of the Mediator interface ?
<paulfremantle> the question is simply
<paulfremantle> do we want information on the mediator
<paulfremantle> or on the final service
<paulfremantle> and the information that rajesh listed
<paulfremantle> is all about the final service
<paulfremantle> and the right way to collect that is in the mediator
<Rajesh_Koilpilla> a lot of information on the mediators
<Rajesh_Koilpilla> the native provider service is quite oblivious to some of this information
<saminda> so do we give another method for Mediator interface to collect the date
<saminda> ??
<Rajesh_Koilpilla> the round trip time after it leaves the mediators and back into the framework is what will be of interest from the native provider service perspective
<paulfremantle> so to do that
<Rajesh_Koilpilla> saminda - is it date or data?
<paulfremantle> so for round trip time
<saminda> opss sorry Data :)
<paulfremantle> we will need to store information locally
<paulfremantle> because the interactions may be asynchronous
<saminda> in a database ??
<paulfremantle> no
<paulfremantle> not for performance info
<paulfremantle> we need an in-memory cache
<Rajesh_Koilpilla> paul - you clock the time after it leaves the Synapse framework and clock it again when it enters the framework for round trip time
<saminda> runtime,
<paulfremantle> yes
<paulfremantle> exactly
<saminda> as we have SynapseEnvironment
<Rajesh_Koilpilla> it's SOAP request specific - need not in DB (ie. transient)
<paulfremantle> so one mediator on the inflow can clock the outbound time
<Soumadeep> That's one of the reason I propose a management Information exchange format... which will hold the required management information...
<Rajesh_Koilpilla> But you collate all of this information and post it back as part of the TransactionEvent
<Soumadeep>
<paulfremantle> and another can clock the inbound time
<saminda> ok
<Rajesh_Koilpilla> paul - wouldn't all of this information be part of the same Message Context?
<saminda> actually no
<paulfremantle> no because the inbound flow may be separate
<saminda> <send/> is the one who seperate inbound and outbound IMO
<Rajesh_Koilpilla> then some mechanism to correlate the flows would be required to figure out all the instrumented management data
<paulfremantle> the mechanism exists
<paulfremantle> in fact we can instrument the underlying axis2 sender
<paulfremantle> if its more efficient
<paulfremantle> but its certainly possible to do it with a mediator
<Soumadeep> Paul so what do you think is the best way to collect management information?
<Soumadeep> Let me give you some details... on what we could expect
<paulfremantle> great
<Soumadeep> <ManagementInformationExchange>
<Soumadeep> <EndPointReference>String</EndPointReference>
<Soumadeep> <TransactionID>String</TransactionID>
<Soumadeep> <TimeStamp>2001-12-17T09:30:47.0Z</TimeStamp>
<Soumadeep> <Operation>String</Operation>
<Soumadeep> <ClientIP>String</ClientIP>
<Soumadeep> <UserAgent>String</UserAgent>
<Soumadeep> <ProviderEPR>String</ProviderEPR>
<Soumadeep> <SOAPRequest>String</SOAPRequest>
<Soumadeep> <SOAPResponse>String</SOAPResponse>
<Soumadeep> <ExecutionStatus>
<Soumadeep>   <Synapse>true</Synapse>
<Soumadeep>   <Provider>true</Provider>
<Soumadeep> </ExecutionStatus>
<Soumadeep> <ExecutionTime>
<Soumadeep>   <TotalRoundTrip>2147483647</TotalRoundTrip>
<Soumadeep>   <ProviderExecutionTime>2147483647</ProviderExecutionTime>
<Soumadeep>   <MediatorExecutionTime>
<Soumadeep>    <Mediator name="M1">2147483647</Mediator>
<Soumadeep>    <Mediator name="M2">2147483647</Mediator>
<Soumadeep>    <Mediator name="M3">2147483647</Mediator>
<Soumadeep>   </MediatorExecutionTime>
<Soumadeep> </ExecutionTime>
<Soumadeep> </ManagementInformationExchange>
<Soumadeep> Sorry... if I messed by the chat window
<Soumadeep> Paul... please have a look at it...
<paulfremantle> nope thats fine
* chinthaka has quit IRC ("Chatzilla 0.9.68.5 [Firefox 1.0.7/20050915]")
<paulfremantle> looks good
<Soumadeep> This could be some data set we might be interested in dealing with
<paulfremantle> ok
<paulfremantle> seems useful
<paulfremantle> whats the purpose of collecting the mediator ex time?
<pvikas> A synapse admin might like to evaluate mediator performance...
<Soumadeep> By what you proposed... we can have a mediator send an event using the ManagementInformationExchange structure
<paulfremantle> vikas
<paulfremantle> so i imagine there are two roles here.... 1) monitoring endpoints 2) tuning synapse
<paulfremantle> but i think you are right
<Rajesh_Koilpilla> this is to identify the time spent in the mediation framework - we can do cool graphs on it ;)
<Soumadeep> So each mediator can extend maybe something like *MediatorXXX extends MIX*
<Soumadeep> So mediator is something like MIXAware...
<Soumadeep> and each mediator pushes in data...
<Soumadeep> What say Paul?
<paulfremantle> I think its a pretty cool idea
<Soumadeep> Thanks
<paulfremantle> i think there may be some ways we can do it without modifying the mediators
<paulfremantle> for example commons-modeler in apache lets you wrap an exisitng object in JMX
<Soumadeep> ok what are your thoughts on it
<paulfremantle> but i need to think :-)
<Soumadeep> But JMX notification if you remember are synchronized.. could that be an issue?
<Rajesh_Koilpilla> we wouldn't need to wrap the whole mediator interface as a manageable entity - but just the portions which need to be manageable
<Soumadeep> Yeah I agree
<paulfremantle> yes I was kind of thinking of something similar not the same
<Rajesh_Koilpilla> since JMX objects need to be Serializable
<paulfremantle> or maybe make mediator a base abstract class that collects the information
<paulfremantle> i still think we are looking at two distinct issues
<Soumadeep> Yes....
<paulfremantle> but you have definitely convinced me that its more efficient to build as part of the base framework
<Soumadeep> Paul one more thing which I think would be useful is.... should the mediators have a concept of in-data and out-data?
<paulfremantle> ?
<Soumadeep> meaning .... that way we would know what mediators take in and what does it spit out...
<paulfremantle> i still dont grok it
<paulfremantle> can you give an example
<Soumadeep> This could be used in a situation where mediators are written by different vendors...
<Rajesh_Koilpilla> soumadeep - it depends on whether you want that to be enforced as part of the mediator interface OR build that infrastructure around it
<Soumadeep> Yes.... if it's in the mediator then other vendors would know what to expect from it's previous mediator... and so on...
<Soumadeep> Paul am I making sense?
<paulfremantle> so i think there are two different issues
<paulfremantle> (im beginning to sound like a broken record)
<paulfremantle> if you are talking about the shape of messages (the schema)
<Soumadeep> :-))
<Hariharasudhan> yes ...
<Soumadeep> yeah... in-schema and out-schema
<paulfremantle> ok
<paulfremantle> so i dont like that :(
<Soumadeep> :-(
<paulfremantle> because there are already models for implementing Services
<paulfremantle> that do that
<paulfremantle> i guess it could be optional
<paulfremantle> on an XSLT it might make sense
<paulfremantle> but I'm still not sure the benefit
<paulfremantle> ON the other hand
<paulfremantle> if you are talking more generically of having some markers
<paulfremantle> this mediator must have that one run before it
<paulfremantle> is see that as very useful
<paulfremantle> for example one mediator might add contextA into the context
<paulfremantle> and mediatorB may require it
<Soumadeep> yeah... some dependency check
<paulfremantle> but I see that as something for tooling not runtime
<Soumadeep> DO we have any concept of registering a mediator... with Synapse?
<paulfremantle> because its just as easy for the mediator to check at runtime if the context is there
<paulfremantle> no....we register MediatorFactorys
<paulfremantle> but mediators are just in the classpath
<paulfremantle> at the moment
<Soumadeep> It could be helpful... if have something like getMediators at any given point of time... for a service...
<paulfremantle> guy i have to go.... (need to check out of my hotel)
<Soumadeep> ok... can we continue this over email...
<saminda> bye, Paul
<saminda> guys, i gotta go too, pls be kind enough to post the log :)
<ant_> I can do that
* paulfremantle has left #apache-synapse
<Soumadeep> Ant your thoughts?
<ant_> later everyone, see you in half an hour paul...
<saminda> it's always nice to get more choices for a subject :)
* Hariharasudhan has left #apache-synapse
* saminda has quit IRC ("Leaving")
<pvikas> great... thanks for the log ant...
<Soumadeep> [Any] your comments please?
<Soumadeep> *ANT*
<ant_> (sorry was on the phone)
<ant_> at least for the monitoring data i'm guessing its going to need some gathering mechanism thats part of the runtime not just another mediator. but ltes continue this on the ML
<ant_> i gotta go, speak to you all later...
<Soumadeep> ok
<pvikas> bye...
<Soumadeep> bye
* Soumadeep has quit IRC ("ChatZilla 0.9.61 [Mozilla rv:1.7.2/20040803]")

RE: IRC chat log Thursday 23 Feb 2006

Reply via email to