Probably we are already going too far into implementation details. I think that the project would have 3 parts:
1. Establishing the general (implementation independent) framework: - Define protocols: SOAP headers, but also mappings to HTTP headers (for REST) and JMS properties. - Define the structure of the correlation identifiers so that the entire sequence can be reconstructed in a meaningful way. - Look for strategies to detect and/or avoid duplicate logging (because of redundant interceptors or simply because a message is logged at the transmitter and receiver side). - Find solutions for cases where the message is not XML. - Think about what happens if one of the services doesn't understand the protocol. - Think about security implications. - ... 2. Build a generic Java framework that supports this style of logging (with components for remote logging, classes that represent the context, tools to browse the intercepted messages, etc.). 3. Build a set of interceptors that make use of this framework (and that log the messages and propagate the context). This can be Axis2 modules, JAX-WS handlers or AspectJ aspects. On Tue, Mar 30, 2010 at 23:28, nilupa bandara <nilu...@gmail.com> wrote: > Hello Everyone, > > First let me thank all of you for your valuable inputs. > > Now let me ask few questions to make sure that I understood the > suggested approach correctly. > > - What are the components that you meant by 'Components who call > RemoteLogger.log()' ? A Custom Message Receiver which logs the > messages that are going in (and out) to a Service Impl perhaps ? > > - Does Service Impl needs check the availability of RemoteLogger (via > a thread local variable) and log the message it sends ( and receive) > using Service Clients to other applications ? I mean is it the case > where service author needs to insert 'RemoteLogger.Log()' in > appropriate places to ensure complete traceability ? > > Using AOP - I just read few documentation on AspectJ and something > like the following could be done (But I'm not absolutely sure though > !!) > > We could use some AOP code (package in a jar) which will, just before > each Service Client invocation insert the appropriate debug info > headers to outgoing payload , plus logs the response it gets. > > This AOP could be instrumented (or weaved) into all the service > archive files in a given repository. > > We could provide a script (and AOP jar) to automate this. For instance > ./our_script AOP.jar <path-to-repository> will make all the services > that are available in the repository traceable. > > Any suggestions ? > > Nilupa > > > On Tue, Mar 30, 2010 at 3:11 PM, Glen Daniels <g...@thoughtcraft.com> wrote: >> I'm a really terribly inconsistent blogger. However, my last actual entry, >> from 2009, was about exactly this: >> >> http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html >> >> I like Andreas' suggestion below, but I think it might be a bit heavyweight >> for a GSoC project(?). Maybe we could lower the bar to something like this: >> >> * Build a distributed logging web service which can accept log messages and >> correlationIDs, logging to a file per correlationID. >> >> * Build a Module that processes incoming <debug:logInfo> headers and makes an >> appropriate RemoteLogger (or whatever) available. This could, at first, go >> in one of Axis2's contexts (OperationContext?) as a demonstration. >> Components who call RemoteLogger.log() send messages (hopefully on another >> thread) to the central log service. >> >> * The Module could automatically put the right metadata on outgoing messages, >> but only the responses, so you still have the problem Andreas points out >> about needing the app to transfer the logging info over to new >> ServiceClients. I was imagining that could be achieved by also setting a >> ThreadLocal - so if your clients run on the same thread you're all set, and >> if there are new threads it's up to the framework to copy that ThreadLocal >> over. >> >> * Regardless of the discovery mechanism, you'd want something static like >> RemoteLogger.getLogger() to hide it. >> >> This approach is definitely a bit more intrusive, as it relies on the >> components themselves to be looking for a RemoteLogger, but it seems like a >> reasonable starting point which could be evolved. However, maybe the AOP >> stuff isn't as challenging as I'm imagining.... >> >> Thoughts? >> >> --Glen >> >> On 3/30/2010 4:57 AM, Andreas Veithen wrote: >>> I was actually referring to scenarios where a service may call other >>> services, which in turn may call again other services. In these >>> scenarios, it is not sufficient to simply collect received/sent >>> messages on different hosts, unless the system is isolated >>> (development environment) or the request rate is sufficiently low so >>> that messages can be correlated based on time. Here is a concrete >>> scenario from a project in my company: >>> >>> - Request comes in on an ESB that does security and validation. >>> - Request is processed by an application server which persists the >>> received information and publishes a JMS message (pub/sub events). >>> - The event is consumed by one or more components that may in turn >>> interact with other services. >>> >>> If you want track this flow, it is not sufficient to intercept the >>> messages on the different hosts: in addition, you need to instrument >>> the services so that the outgoing requests are correlated with the >>> incoming requests (by adding SOAP and/or transport headers). What I >>> would like to be able to do in the above scenario is to enable >>> end-to-end monitoring on a per-message basis: I add a special SOAP or >>> HTTP header to the initial request to enable logging and this >>> information is propagated along the chain. Every service/component >>> then sends a copy of the incoming/outgoing requests/responses to a >>> central place where the sequence of events is reconstructed. >>> >>> One way this could be achieved is with a tool that postprocesses the >>> artifacts from the build process. For each artifact, the tool would >>> disassemble the artifact, instrument the code using a set of AspectJ >>> aspects and reassemble the artifact for deployment. The responsibility >>> of the aspects would be to intercept messages, log them and modify >>> them to ensure proper correlation. Of course this only works if the >>> artifacts are J2EE deployables (WARs or EARs). Note that we already >>> use AspectJ for a similar use case (although at a much smaller scale) >>> in the transport test suites [1]. Interestingly, this approach would >>> allow to cover interactions that are not SOAP based, e.g. one could >>> even intercept the database queries executed during the flow. >>> >>> Andreas >>> >>> [1] >>> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java >>> >>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <hemap...@gmail.com> wrote: >>>> Hi Nilupa; >>>> >>>> When we collect messages from a one location by installing proper >>>> handler that will intercept and send messages to to that one location >>>> (this one location can be a single server, pub/sub channel etc). There >>>> is many ways to make sense of those collected messages. What Andreas >>>> mentioned (following complete transaction) is a one possibility. >>>> >>>> I think you should come up with few scenarios on how you would make >>>> sense of the message. >>>> >>>> Thanks >>>> Srinath >>>> >>>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <nilu...@gmail.com> wrote: >>>>> Hi, >>>>> >>>>> First let me thank you for commenting. >>>>> >>>>> As far as I understood, what you would like to see from the proposed >>>>> tool is to view set of messages that are exchanged in reponse to a >>>>> particular input message. With the understanding that I am having at >>>>> the momnet, one way to do it is to filter out the central repository >>>>> of messages based on 'To' , 'From' headers and try to contruct the >>>>> message chain from it. We can allow the client GUI wich connects to >>>>> the central repository to provide the paramenters (For instance the >>>>> value of 'To' header) from which an intelligent filtering can be done >>>>> for the set of messages avialable at the central repository. >>>>> >>>>> Perhaps someone has an idea of a better way of doing it and is willing >>>>> to share it with us. It would be really nice to hear from them. >>>>> >>>>> Thanks in advance ..!! >>>>> >>>>> Best Regards, >>>>> Nilupa >>>>> >>>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen >>>>> <andreas.veit...@gmail.com> wrote: >>>>>> Personally, I think that the added value of extending the SOAP monitor >>>>>> module to collect messages in a central place is too limited to >>>>>> attract enough interest from the user community, so that it will be >>>>>> difficult to ensure the evolution of such a project in the future. >>>>>> However, what many people are looking for is a tool that allows to >>>>>> track the flow of a message through a distributed system, or more >>>>>> generally to track the sequence of events triggered by a given input >>>>>> message (sort of end-to-end transaction monitoring). >>>>>> >>>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <nilu...@gmail.com> wrote: >>>>>>> Hello, >>>>>>> >>>>>>> I am graduate student at Politecnica de Madrid and I am thinking of >>>>>>> proposing a GSoC project this summer. I would like to take project >>>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and >>>>>>> also should be helpful to any Java developer using Apache Axis2 Web >>>>>>> services middleware. I will submit more detailed proposal later. I >>>>>>> would really like to hear any feedback from you. Any suggestions or >>>>>>> features that you would like to see in a "Distributed TCP Monitor" are >>>>>>> most welcome. >>>>>>> >>>>>>> Best Regards, >>>>>>> Nilupa Bandara >>>>>>> >>>>>>> [1] >>>>>>> >>>>>>> Distributed TCP monitor >>>>>>> >>>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard >>>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the >>>>>>> problem by writing a Axis2 module (Handler) that intercept messages >>>>>>> comes in and goes out of a Axis2 server and send to a UI (via a >>>>>>> pub/sub channel or a message Box e.g. ) or record them so user can >>>>>>> pull those messages. Also, we can do something to turn the module on >>>>>>> and off remotely, so users can have his module deployed, but turn it >>>>>>> on only when they want to debug the system. >>>>>>> >>>>>>> Then we can take this to next level by adding this module to all >>>>>>> services in a system , configuring modules to send all collected >>>>>>> messages to a pub/sub channel, and subscribing to those messages via a >>>>>>> UI, and depicting those messages through a UI. Then users have a >>>>>>> TCPMon for a whole system. >>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: java-dev-unsubscr...@axis.apache.org >>>>>>> For additional commands, e-mail: java-dev-h...@axis.apache.org >>>>>>> >>>>>>> >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: java-dev-unsubscr...@axis.apache.org >>>>>> For additional commands, e-mail: java-dev-h...@axis.apache.org >>>>>> >>>>>> >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: java-dev-unsubscr...@axis.apache.org >>>>> For additional commands, e-mail: java-dev-h...@axis.apache.org >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> ============================ >>>> Srinath Perera, Ph.D. >>>> WSO2 Inc. http://wso2.com >>>> Blog: http://srinathsview.blogspot.com/ >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: java-dev-unsubscr...@axis.apache.org >>>> For additional commands, e-mail: java-dev-h...@axis.apache.org >>>> >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: java-dev-unsubscr...@axis.apache.org >>> For additional commands, e-mail: java-dev-h...@axis.apache.org >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-dev-unsubscr...@axis.apache.org >> For additional commands, e-mail: java-dev-h...@axis.apache.org >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-dev-unsubscr...@axis.apache.org > For additional commands, e-mail: java-dev-h...@axis.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@axis.apache.org For additional commands, e-mail: java-dev-h...@axis.apache.org
