Once again, thanks for the comments! In general you have most of the flow down, although I would do them in a different order. 1) Install the catalog editor (This is a one time activity) 2) Perform analysis. 3) Use the editor to create audit event and attribute definitions. 4) Publish those changes (this causes a JSON file to be created or updated in a git repo). 5) Run the Maven plugin against the git repo where the JSON file resides. This generates the Java interfaces. 6) Write code using the Java interfaces. https://github.com/apache/logging-log4j-audit-sample/blob/master/sample-app/src/main/java/org/apache/logging/audit/SampleApp.java <https://github.com/apache/logging-log4j-audit-sample/blob/master/sample-app/src/main/java/org/apache/logging/audit/SampleApp.java> shows how to use the LogEventFactory to get an instance of the interfaces. I’m surprised I didn’t include some example code in the web site. I meant to. 7) Configure the log4j2.xml to do something with the audit events.
As noted above Git is used to maintain the history of the catalog. Since the interfaces are generated as part of the build for the project that maintains the catalog they are not stored in git. As for the RequestContext, I consider it to be the linchpin in any web application that makes use of services, regardless of audit logging. As I discuss on that page it is used to pass request and session ids throughout all the services so that logs can be correlated. It contains key information that all the applications need to support multiple tenants transparently, without having to pass that information through every API call. The most important of these is the tenant id but things like the user’s timezone, local, and loginId can be included. Since these are always available to the application (and Log4j) there is no need to explicitly include them in the audit event definitions. Anything in the RequestContext will automatically be included in every audit event. I’ll re-read what you have below (and what I have written above) and try to figure out how to incorporate that. FYI - I have considered making the editor a Spring Boot app and will probably end up doing that so it can be started from the command line. I’ve also considered creating a JavaFX version of it since it isn’t really a typical web application. The audit service can’t be turned into a Spring boot app because it has to be modified by the customer. Ralph > On Feb 12, 2018, at 7:36 AM, Remko Popma <remko.po...@gmail.com> wrote: > > Ralph, > > very nice improvements! Especially the first page does a great job at > explaining what audit logging is and why/when you would want to use it. > > The Getting Started page then does a tutorial-style deep dive into using > Log4j Audit, but unfortunately gets a bit bogged down in setting up the > toolchain, and seems to stop half-way through. > > Would it be possible to show a diagram that visually explains the usage > workflow at a glance? For example, for diagnostic logging with Log4j2, a > common workflow is: > Step 1 (Configuration): using a text editor, create a configuration file. > Step 2 (Installation): put the configuration file and the Log4j jars in > your classpath > Step 3 (API setup): call `LogManager.getLogger()` to obtain a Logger. It is > convenient to store a reference to this logger in a private static field. > Step 4: use the API to log stuff > > With Audit Logging, the workflow is more involved. Part of that is the tool > chain, but it seems to me that there are simply more steps before users can > get to the “log stuff“ part. A diagram that allows users to grasp this > workflow at a glance would be great. It would be good if users can get an > overview-level understanding without needing to do the deep dive tutorial. > > So, what is the Audit Logging workflow? I'm guessing (correct me if I'm > wrong) something like this: > > Step 1 (Analysis): find places in your application where state > modifications occur that you want to keep a track record for > Step 2 (Install the toolchain): this is covered on the Getting Started page > Step 3 (Design): use the toolchain to create Audit Events for the state > modifications discovered in Step 1. Attributes of these Audit Events are > put in an attribute pool and can be shared by other events, so it's > important to give them good names. Audit Event attributes must also have a > data type. Description etc are optional. (From the Audit Catalog page I get > the impression that attributes must be created first and can only later be > combined into Audit Events. It may be more natural for users to start with > an Audit Event and then either pick an existing attribute or create new > attributes while building up this Audit Events definition. Such attributes > can still be put in the shared pool.) > *(from here on I'm just guessing...)* > Step 4 (Code Generation): when you're happy with the Audit Event > definition, use the toolchain to generate Java interfaces for the Audit > Events. This results in a JAR that you can add to the classpath of your > application. > Step 5: log stuff. In your application, you can use XXX (factory?, > constructor?) to instantiate an instance of the generated interface. This > interface extends AuditMessage and has setters and getters (or does it use > a builder?) for the attributes of the AuditEvent. At places in the > application where a state modification occurs that you want to keep a track > record for, populate the attributes for the appropriate AuditMessage and > log it. (Question: Should users configure a separate logger for > AuditMessages or can the diagnostic logger be used and are the > AuditMessages separated downstream with filters etc? What is the > recommended setup for this?) > > The Getting Started page covers Step 2, and the Audit Catalog page explains > Step 3, but I could not find anything for Step 4 and 5. It would be good to > explain this workflow from a user's point of view, rather than from a tool > provider's point of view. > > About the toolchain, the Getting Started page mentions that there is a WAR > file, and then asks users to install a servlet container. However, the > recommended setup is for users to run this container locally. Why not > provide this as a standalone application with an embedded servlet > container? The underlying WAR could always be deployed elsewhere but this > would give a nicer user experience out of the box. > > Some additional questions: > * Git is part of the toolchain but I'm not clear what its purpose is. The > catalog can be exported from the in-memory database in JSON format (and > imported again I assume), so is git just to version and distribute the > persisted catalog? Would another version control system also work? Would > just plain file system also work? > * Are the generated interfaces stored somewhere? Also in git? Are they > versioned also? > > About RequestContext: > I still think this page jumps in too deep too fast and I can't follow... > I gather that the use of this class is optional. > Users could use the generated interfaces to log AuditMessages directly, no? > To me, logging AuditMessages via custom (auto-generated) interfaces seems > easiest to understand. > Is it fair to say that RequestContext is only useful in web applications, > and particularly in Spring-based web applications? It may be good to > mention that explicitly in the beginning. > > You mention that RequestContext is useful to > (1) capture common elements, so the custom Audit Events can be smaller and > become easier to use, and > (2) provide a thread-safe way to pass data between several layers. > > Those features seem generally useful (also outside of web apps). In > addition to these two, I gather that RequestContext provides functionality > that makes it easy to integrate with the Spring services. I assume that > this is where the annotations come into play. I don't have enough Spring > expertise to understand the explanation for the annotations. It is unclear > to me whether these annotations would be useful (or generally how > RequestContext could be used) outside the Spring world, or even outside of > web applications. > > I like the diagram under Transporting the RequestContext though. Diagrams > are good! I like diagrams! > > It may be good to have another page showing very basic Audit logging with > the generated interfaces, without RequestContext (perhaps for a non-web > app). > > I'll stop here. Let me know what you think. > Remko > > > On Mon, Feb 12, 2018 at 11:01 Remko Popma <remko.po...@gmail.com> wrote: > >> Ok, I’ll try tonight if I can. >> >> >> >>> On Feb 12, 2018, at 9:31, Ralph Goers <ralph.go...@dslextreme.com> >> wrote: >>> >>> Remko, >>> >>> I believe I have addressed most of the feedback from these two emails, >> although I haven’t figured out how the selected component stays highlighted >> in the left hand menu. I’d appreciate you and everyone else taking another >> look at https://rgoers.github.io/log4j-audit/index.html < >> https://rgoers.github.io/log4j-audit/index.html>. >>> >>> Thanks, >>> Ralph >>> >>>> On Feb 5, 2018, at 8:04 AM, Remko Popma <remko.po...@gmail.com> wrote: >>>> >>>> About the web site, the project seems to have components, but the >> component >>>> links in the left-hand navigation menu are not very useful: >>>> If you click on "Audit API" for example, only some standard >> Maven-generated >>>> component links/pages are visible, no javadoc or sources in the >> Component >>>> Reports. >>>> Also the selected component should stay highlighted in the left-hand >> menu >>>> like we do for the Log4j 2 web site. >>>> >>>> The Javadoc page at the top of the left-hand navigation menu seems >> broken: >>>> it shows the Log4j 2 modules, not the log4j-audit modules. >>>> The submenu links under Javadoc (e.g. Javadoc/Log4j Audit API) all give >> 404 >>>> page not found errors. >>>> >>>> >>>> >>>>> On Mon, Feb 5, 2018 at 11:49 PM, Remko Popma <remko.po...@gmail.com> >> wrote: >>>>> >>>>> Some first impression feedback: >>>>> >>>>> Top page: >>>>> I think it is worth explaining the motivation/use case for audit >> logging >>>>> here. What is "audit logging"? How is audit logging different from >> "normal" >>>>> logging? What kind of applications would want to use audit logging and >> why? >>>>> What are audit events? How do audit events relate to log events? >>>>> >>>>> RequestContext page: >>>>> I had trouble following this page. The explanation is going too fast >> for >>>>> me and seems to be skipping over some steps. Is my understanding below >>>>> correct? >>>>> * Users must create a RequestContext to use audit logging (if true, >> best >>>>> to start by saying that) >>>>> * The reason users need to create a RequestContext is to have a single >>>>> container for all data points that users want to log in their audit >> log. >>>>> (Question: does this mean that RequestContext = Audit Event?) >>>>> * A recommended/convenient way to implement a RequestContext is to >> stuff >>>>> all values in the log4j ThreadContext (Question: is it really okay to >>>>> assume that the service container does not hand off requests to worker >>>>> threads?) >>>>> * The example RequestContext implementation is too long (and >> repetitive - >>>>> readers will get the point after a few attributes) - may be better to >> place >>>>> the full class in an example application and only show snippets in this >>>>> page (and perhaps link to the full example from the page) >>>>> * After the example follows some explanation about the annotations. >> Seems >>>>> pretty important stuff but is now just a wall of text. I would break >> it up >>>>> into sections with bold headers, a separate section for each >> annotation. >>>>> Current explanation of the annotations seems a bit too brief. >>>>> * RequestContextInterceptor example seems a bit long. Can you reduce >> it >>>>> to its essence or break it up? (Also formatting seems off and has >> missing >>>>> closing double quote in response.sendRedirect("/login); , but does >> this >>>>> page really need to contain a fully working example?) >>>>> * Finally, the the "passing context to service" section: are >>>>> RequestContextInterceptor and RequestContextHeaderInterceptor the >> same >>>>> thing? >>>>> * Does everyone using Spring know what " *The returned list should then >>>>> be added to the RestTemplate* " means? (I have no clue :-) but I am >>>>> Spring-ignorant.) >>>>> >>>>> Audit Catalog page: >>>>> The page mentions Products and Categories 3 times, every time saying >> "but >>>>> Log4j doesn't do anything with that". Why not just leave it out >> altogether >>>>> and not mention these? >>>>> Why is it called a Catalog? Perhaps explaining why this term is a good >>>>> name would help set the readers frame of thinking to understand the >> rest of >>>>> the page. >>>>> Also, do users need to create a catalog? Or is it something that >> emerges >>>>> automatically when one uses audit logging? What happens if you don't >> create >>>>> a catalog? >>>>> >>>>> Hope this is useful, >>>>> Remko >>>>> >>>>> >>>>> On Mon, Feb 5, 2018 at 2:35 PM, Ralph Goers < >> ralph.go...@dslextreme.com> >>>>> wrote: >>>>> >>>>>> Well I have good news and bad news. The bad news is that I forgot my >> wife >>>>>> and I were having people over for the super bowl so I didn’t have as >> much >>>>>> time as I had hoped and I wasn’t able to run the Log4j 2.11.0 release >> build. >>>>>> >>>>>> The good news is that I think Log4j Audit V1.0 is about ready for a >>>>>> release. I have published the web site at >> https://rgoers.github.io/log4j >>>>>> -audit/index.html <https://rgoers.github.io/log4j-audit/index.html>. >>>>>> Some parts of the site will have problems since it hasn’t been >> released but >>>>>> I hope you could take a look at it and review it before a release >> vote is >>>>>> attempted. >>>>>> >>>>>> You should also feel free to ask me questions here, but if it isn’t >> clear >>>>>> then I expect the web site needs more work. >>>>>> >>>>>> Ralph >>>>> >>>>> >>>>> >>> >>
