IMO metrics and auditing don’t have a lot in common. That said, I do use the
RequestContext to log elapsed time of every request. See the
RequestContextFilter class.
Ralph
> On Jun 11, 2018, at 11:56 AM, Matt Sicker <boa...@gmail.com> wrote:
>
> What about metrics gathering? Is that an orthogonal concern or can you make
> audit logs like that?
>
> On Mon, Jun 11, 2018 at 12:59, Ralph Goers <ralph.go...@dslextreme.com>
> wrote:
>
>> So for the use case you describe I would suggest you think about the
>> events that need to be audited and the data that is associated with those
>> events. Then figure out which are request context items (for web apps there
>> are a few that should be considered “universal" since every app on the
>> planet would have them) - they should be logged as part of every audit
>> event., and which are specific to the particular event or to a group of
>> related events.
>>
>> Then take that information and use the Catalog Editor (after it is
>> configured of course) to define the attributes. Once you have completed
>> that define the events. Once that is completed you should be able to
>> generate the event interfaces and start using them in code.
>>
>> Ralph
>>
>>> On Jun 11, 2018, at 9:36 AM, Remko Popma <remko.po...@gmail.com> wrote:
>>>
>>> That's why I think the site is really important.
>>> More use cases could help people imagine how they could apply this in
>> their
>>> organization.
>>>
>>> I work in finance and having an audit trail is mandatory for everything
>>> that modifies the production system.
>>> This often involves deployment scripts and GUIs (web, standalone or CLIs)
>>> that for example modify the database with configuration that drives the
>>> production system's behaviour.
>>>
>>>
>>>
>>> On Tue, Jun 12, 2018 at 1:13 AM, Matt Sicker <boa...@gmail.com> wrote:
>>>
>>>> I think this is an observability related idea which is still rather new
>> to
>>>> many companies who are just finally embracing DevOps in the first
>> place. I
>>>> love the idea though!
>>>>
>>>> On Mon, Jun 11, 2018 at 10:32, Ralph Goers <ralph.go...@dslextreme.com>
>>>> wrote:
>>>>
>>>>> Really? I would think almost everything would want to track who made
>> what
>>>>> change to the system.
>>>>>
>>>>> Ralph
>>>>>
>>>>>> On Jun 11, 2018, at 5:20 AM, Matt Sicker <boa...@gmail.com> wrote:
>>>>>>
>>>>>> Ok, thanks for the clarification. It sounded far more advanced, and
>> I’m
>>>>>> getting a better picture here. I’ve never worked in a domain that
>>>>> requires
>>>>>> auditing before.
>>>>>>
>>>>>> On Mon, Jun 11, 2018 at 08:03, Apache <ralph.go...@dslextreme.com>
>>>>> wrote:
>>>>>>
>>>>>>> No. It implements that feature through the request context but I
>>>>> wouldn’t
>>>>>>> use a catalog for simple trace logging.
>>>>>>>
>>>>>>> Ralph
>>>>>>>
>>>>>>>> On Jun 11, 2018, at 4:29 AM, Matt Sicker <boa...@gmail.com> wrote:
>>>>>>>>
>>>>>>>> One thing I didn’t notice until now is that this is a superset of
>>>>>>>> distributed trace logging. Would you say that’s accurate?
>>>>>>>>
>>>>>>>>> On Mon, Jun 11, 2018 at 00:54, Apache <ralph.go...@dslextreme.com>
>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> One thing I forgot to mention. Although Log4J audit doesn’t make
>> use
>>>>> of
>>>>>>>>> the product or category the catalog’s usefulness really comes into
>>>>> play
>>>>>>>>> when you want to create a UI to query and display the audit events.
>>>> In
>>>>>>> that
>>>>>>>>> case associating events with products and/or categories can be
>> quite
>>>>>>>>> helpful.
>>>>>>>>>
>>>>>>>>> Ralph
>>>>>>>>>
>>>>>>>>>> On Jun 10, 2018, at 1:36 AM, Apache <ralph.go...@dslextreme.com>
>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Oh. You don’t have the maven plugin. Since it hasn’t been released
>>>>> yet
>>>>>>>>> you will have to build the Log4J audit project.
>>>>>>>>>>
>>>>>>>>>> Sent from my iPad
>>>>>>>>>>
>>>>>>>>>>> On Jun 10, 2018, at 12:23 AM, Remko Popma <remko.po...@gmail.com
>>>
>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> That is with
>>>>>>>>>>> C:\Users\remko\IdeaProjects\logging-log4j-audit-sample>mvn
>>>> --version
>>>>>>>>>>> Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d;
>>>>>>>>>>> 2017-10-18T16:58:13+09:00)
>>>>>>>>>>> Maven home: C:\apps\apache-maven-3.5.2\bin\..
>>>>>>>>>>> Java version: 1.8.0_161, vendor: Oracle Corporation
>>>>>>>>>>> Java home: C:\apps\jdk1.8.0_161\jre
>>>>>>>>>>> Default locale: en_GB, platform encoding: MS932
>>>>>>>>>>> OS name: "windows 10", version: "10.0", arch: "amd64", family:
>>>>>>> "windows"
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> On Sun, Jun 10, 2018 at 4:21 PM, Remko Popma <
>>>>> remko.po...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> getting this now
>>>>>>>>>>>>
>>>>>>>>>>>> [INFO] Reactor Summary:
>>>>>>>>>>>> [INFO]
>>>>>>>>>>>> [INFO] Audit Sample Parent ................................
>>>>> SUCCESS [
>>>>>>>>>>>> 0.839 s]
>>>>>>>>>>>> [INFO] audit-service-api ..................................
>>>>> FAILURE [
>>>>>>>>>>>> 0.006 s]
>>>>>>>>>>>> [INFO] audit-service-war ..................................
>>>> SKIPPED
>>>>>>>>>>>> [INFO] audit-service ......................................
>>>> SKIPPED
>>>>>>>>>>>> [INFO] sample-app .........................................
>>>> SKIPPED
>>>>>>>>>>>> [INFO] ------------------------------
>>>> ------------------------------
>>>>>>>>>>>> ------------
>>>>>>>>>>>> [INFO] BUILD FAILURE
>>>>>>>>>>>> [INFO] ------------------------------
>>>> ------------------------------
>>>>>>>>>>>> ------------
>>>>>>>>>>>> [INFO] Total time: 1.116 s
>>>>>>>>>>>> [INFO] Finished at: 2018-06-10T16:11:08+09:00
>>>>>>>>>>>> [INFO] Final Memory: 12M/245M
>>>>>>>>>>>> [INFO] ------------------------------
>>>> ------------------------------
>>>>>>>>>>>> ------------
>>>>>>>>>>>> [ERROR] Plugin
>>>>>>>>> org.apache.logging.log4j:log4j-audit-maven-plugin:1.0.0-SNAPSHOT
>>>>>>>>>>>> or one of its dependencies could not be resolved: Could not find
>>>>>>>>> artifact
>>>>>>>>>>>>
>>>>> org.apache.logging.log4j:log4j-audit-maven-plugin:jar:1.0.0-SNAPSHOT
>>>>>>> ->
>>>>>>>>>>>> [Help 1]
>>>>>>>>>>>> [ERROR]
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Sun, Jun 10, 2018 at 2:53 PM, Ralph Goers <
>>>>>>>>> ralph.go...@dslextreme.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I finally have had time to take a breath and do something with
>>>>>>> this. I
>>>>>>>>>>>>> have tried to incorporate many of your comments in the
>>>>>>> documentation.
>>>>>>>>> I
>>>>>>>>>>>>> have updated my web site accordingly. Some comments are below.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I really would like feedback on more than just the site as I
>>>> need
>>>>> to
>>>>>>>>>>>>> release this.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On May 7, 2018, at 5:42 PM, Remko Popma <
>> remko.po...@gmail.com
>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I had time to look at this during the flight, here it is:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ----
>>>>>>>>>>>>>> index.html
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> typo: Diagnostic logs are critical in aiding in maintaining
>> the
>>>>>>>>>>>>>> servicability -> critical in maintaining?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Overall, the first three sections, "What is Audit Logging",
>>>> What
>>>>> is
>>>>>>>>> the
>>>>>>>>>>>>>> difference between audit logging and normal logging?" and
>> "What
>>>>> is
>>>>>>>>> Log4j
>>>>>>>>>>>>>> Audit?" are very good: give good overview of the purpose and
>>>>> don't
>>>>>>>>>>>>> assume
>>>>>>>>>>>>>> prior knowledge.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> From the "Features" section, the narrative changes perspective
>>>>> from
>>>>>>>>> what
>>>>>>>>>>>>>> users would want to what Log4j Audit provides.
>>>>>>>>>>>>>> I would add a few sentences to that transition, something
>> like:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> {quote}
>>>>>>>>>>>>>> (after Features)
>>>>>>>>>>>>>> Each application has its own audit events. Before using Log4j
>>>>>>> Audit,
>>>>>>>>>>>>>> applications need to define AuditMessages that capture the
>>>> exact
>>>>>>>>>>>>> attributes
>>>>>>>>>>>>>> of its audit events. The [Getting Started](link) page provides
>>>> a
>>>>>>>>>>>>> tutorial
>>>>>>>>>>>>>> that explains how to define audit events for an application.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (after Audit Event Catalog header)
>>>>>>>>>>>>>> Once audit events are defined, they need to be maintained: as
>>>> the
>>>>>>>>>>>>>> application evolves, developers will inevitably discover they
>>>>> need
>>>>>>> to
>>>>>>>>>>>>> add,
>>>>>>>>>>>>>> remove or change attributes of the audit events. Log4j Audit
>>>> can
>>>>>>>>> persist
>>>>>>>>>>>>>> the audit event definitions in a JSON file. This file becomes
>>>> the
>>>>>>>>> Audit
>>>>>>>>>>>>>> Event Catalog for the application. Log4j Audit is designed to
>>>>> store
>>>>>>>>> the
>>>>>>>>>>>>>> event definition file in a Git repository so that the
>> evolution
>>>>> of
>>>>>>>>> the
>>>>>>>>>>>>>> audit events themselves have an audit trail in the Git history
>>>> of
>>>>>>> the
>>>>>>>>>>>>> file.
>>>>>>>>>>>>>> Log4j Audit provides a web interface for editing the events.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Log4j Audit uses the catalog of events to determine ...
>>>> (continue
>>>>>>>>> with
>>>>>>>>>>>>>> current text of Audit Event Catalog)
>>>>>>>>>>>>>> {quote}
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Question about the Requirements section: it isn't clear to me
>>>>> (and
>>>>>>>>>>>>> likely
>>>>>>>>>>>>>> to other readers) why Dynamic Event Catalogs would require a
>>>>>>> database
>>>>>>>>>>>>>> instead of one or more JSON files. Is that explained
>> somewhere?
>>>>>>>>> Perhaps
>>>>>>>>>>>>>> Dynamic Audit Events need a separate page or dedicated section
>>>>>>>>>>>>> somewhere.
>>>>>>>>>>>>>> The Getting Started page mentions "manage dynamic catalogs" in
>>>>> the
>>>>>>>>>>>>>> paragraph under "What you will build" but I couldn't find
>>>>> anything
>>>>>>> on
>>>>>>>>>>>>> the
>>>>>>>>>>>>>> topic of dynamic catalogs.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ----
>>>>>>>>>>>>>> catalog.html
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> From the first paragraph, I would remove "The events may be
>>>>> grouped
>>>>>>>>> by
>>>>>>>>>>>>>> Products and/or Categories, but at this time nothing in Log4j
>>>>> Audit
>>>>>>>>>>>>> makes
>>>>>>>>>>>>>> use of the product or catalog definitions". The same sentences
>>>> is
>>>>>>>>>>>>> repeated
>>>>>>>>>>>>>> at the bottom of the page and since this feature is not used
>> it
>>>>> is
>>>>>>>>>>>>>> confusing to me that the feature is so prominently mentioned
>> in
>>>>> the
>>>>>>>>>>>>> first
>>>>>>>>>>>>>> paragraph of the page. I would consider removing this feature
>>>>>>>>>>>>> altogether.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Overall this is a very good page. Succinct but complete.
>>>> Consider
>>>>>>>>>>>>> moving it
>>>>>>>>>>>>>> above RequestContext in the left-hand navigation menu.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ----
>>>>>>>>>>>>>> gettingStarted.html
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Overall, this page is only effective for people who actually
>>>>>>> perform
>>>>>>>>> the
>>>>>>>>>>>>>> steps and execute the commands mentioned in the page.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It would be good if the page would also be useful for people
>>>> who
>>>>>>> only
>>>>>>>>>>>>> read
>>>>>>>>>>>>>> the page but don't actually perform the steps:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> * Can the page also show an example of an audit event in JSON
>>>>>>> format.
>>>>>>>>>>>>> This
>>>>>>>>>>>>>> could be a simple event with few attributes (maybe a login
>>>>> event?)
>>>>>>> or
>>>>>>>>>>>>> the
>>>>>>>>>>>>>> transfer event that is used later in the page.
>>>>>>>>>>>>>> * I would also like to see the Java interface that is
>> generated
>>>>>>> from
>>>>>>>>>>>>> this
>>>>>>>>>>>>>> JSON audit event.
>>>>>>>>>>>>>> * Finally, I would like to see how my application would use
>>>> this
>>>>>>>>>>>>> generated
>>>>>>>>>>>>>> Java interface. How do I get an instance, how do I populate
>> the
>>>>>>>>>>>>> attributes,
>>>>>>>>>>>>>> and what do I do with the instance after I populated it?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm sure the above is available in the source code of the
>>>> sample
>>>>>>>>>>>>>> application, but this page is a good place to show some of the
>>>>>>>>>>>>> highlights
>>>>>>>>>>>>>> of that source code with some explanatory text.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Secondly, the page mentions remote audit logging and how the
>>>> war
>>>>>>> file
>>>>>>>>>>>>>> provides endpoints for remove audit logging. Is it worth
>>>>>>> dedicating a
>>>>>>>>>>>>>> separate page to show how to configure end points for remote
>>>>> audit
>>>>>>>>>>>>> logging?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Finally, about the catalog screenshots: I understand that
>>>>>>> attributes
>>>>>>>>> are
>>>>>>>>>>>>>> managed separately so they can be reused. The second
>> screenshot
>>>>>>> shows
>>>>>>>>>>>>> the
>>>>>>>>>>>>>> billPay and deposit events. Are these events related to the
>>>>>>> transfer
>>>>>>>>>>>>> event
>>>>>>>>>>>>>> that is mentioned in the curl example in this page?
>>>>>>>>>>>>>
>>>>>>>>>>>>> These are events that take place in banking. billPay is paying
>> a
>>>>>>> bill,
>>>>>>>>>>>>> deposit is depositing money into an account, transfer moves
>>>> money
>>>>>>>>> from one
>>>>>>>>>>>>> account to another. I used these because many people understand
>>>>>>> these
>>>>>>>>>>>>> concepts.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I was trying to see how
>>>>>>>>>>>>>> they could be related but couldn't figure it out.
>>>>>>>>>>>>>> Also, what are the attributes for the billPay and deposit
>>>> events?
>>>>>>>>>>>>>
>>>>>>>>>>>>> The attributes for these events are those shown in the
>> “Assigned
>>>>>>>>>>>>> Attributes” column. The request context attributes are
>> available
>>>>> for
>>>>>>>>> all
>>>>>>>>>>>>> events.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> If the
>>>>>>>>>>>>>> Catalog Editor has a screen to show the attributes that are
>>>> part
>>>>> of
>>>>>>>>> an
>>>>>>>>>>>>>> event then it may be good to add a screenshot for this (I
>> guess
>>>>>>> this
>>>>>>>>>>>>> would
>>>>>>>>>>>>>> be the Edit Event screen) as well. That would tie all these
>>>>>>> concepts
>>>>>>>>>>>>>> together.
>>>>>>>>>>>>>
>>>>>>>>>>>>> As I said, the attributes are on the screen that is shown. I
>>>> would
>>>>>>>>>>>>> suggest you perform the steps in the getting started so you can
>>>>> see
>>>>>>>>> for
>>>>>>>>>>>>> yourself.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ----
>>>>>>>>>>>>>> requestContext.html
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> typo: typcial -> typical
>>>>>>>>>>>>>> typo: acrossall -> across all
>>>>>>>>>>>>>> typo: datbase -> database
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> About Mapping Annotations:
>>>>>>>>>>>>>> This is still a bit abstract to me. Would it be possible to
>>>>> provide
>>>>>>>>> some
>>>>>>>>>>>>>> more explanation on when applications should use ClientServer,
>>>>> when
>>>>>>>>>>>>> Local,
>>>>>>>>>>>>>> and when Chained annotations? Perhaps some example use cases?
>>>> Or,
>>>>>>> if
>>>>>>>>>>>>>> possible, tie this to the use case presented in the sample
>>>>>>>>> application
>>>>>>>>>>>>> (if
>>>>>>>>>>>>>> that makes sense)?
>>>>>>>>>>>>>
>>>>>>>>>>>>> I am not sure what more there is to say. Local means a value
>> in
>>>>> the
>>>>>>>>>>>>> request context is local to that server and should not be
>> passed
>>>>> to
>>>>>>>>> called
>>>>>>>>>>>>> services. ClientServer means that a value is passed from the
>>>>> current
>>>>>>>>>>>>> application to services it calls. Chained means the value is
>>>>>>>>> associated
>>>>>>>>>>>>> with one request context field on the current server but will
>> be
>>>>> in
>>>>>>> a
>>>>>>>>>>>>> different field in the called service. The example for chained
>>>> is
>>>>>>> the
>>>>>>>>>>>>> current hostname. The hostname request context field always
>>>>> contains
>>>>>>>>> the
>>>>>>>>>>>>> host name of the current server. When calling a service the
>>>>> current
>>>>>>>>> host
>>>>>>>>>>>>> name moves to the callingHost field so that the called service
>>>>> knows
>>>>>>>>> what
>>>>>>>>>>>>> server called it.
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> About Transporting the RequestContext:
>>>>>>>>>>>>>> Until now, the information was generically useful for all
>>>>>>>>> applications,
>>>>>>>>>>>>> but
>>>>>>>>>>>>>> this section is specifically useful for web applications.
>>>>>>>>>>>>>> For people who don't work on web applications this transition
>>>> may
>>>>>>> be
>>>>>>>>> a
>>>>>>>>>>>>> bit
>>>>>>>>>>>>>> jarring.
>>>>>>>>>>>>>> Would it make sense for this section and the following two
>>>>> sections
>>>>>>>>> to
>>>>>>>>>>>>> be
>>>>>>>>>>>>>> moved to a separate page? Something like "Web Applications" or
>>>>>>>>> "Remote
>>>>>>>>>>>>>> Audit Logging”?
>>>>>>>>>>>>>
>>>>>>>>>>>>> This concept applies to any kind of distributed application.
>> For
>>>>>>>>> example,
>>>>>>>>>>>>> with AMQP we do the exact same thing by copying the
>>>> RequestContext
>>>>>>>>> fields
>>>>>>>>>>>>> into AMQP headers and then repopulating the RequestContext when
>>>>> the
>>>>>>>>> message
>>>>>>>>>>>>> is processed in the consumer. I have added text to describe
>>>> this.
>>>>> I
>>>>>>>>> have
>>>>>>>>>>>>> components that do this for my day job but I have not added
>> them
>>>>> to
>>>>>>>>>>>>> log4j-audi.
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ----
>>>>>>>>>>>>>> Remko
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>> Matt Sicker <boa...@gmail.com>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>> Matt Sicker <boa...@gmail.com>
>>>>>
>>>>>
>>>>> --
>>>> Matt Sicker <boa...@gmail.com>
>>>>
>>
>>
>> --
> Matt Sicker <boa...@gmail.com>
