Re: [Spacewalk-devel] Audit Logging
On 08/29/2011 03:14 PM, Cliff Perry wrote: In short, my personal preference is not to become a toaster if audit logging has crashed (including out of space to write). Cliff, if Taskomatic has crashed, the Spacewalk will be a toaster as well. I believe the reliability is a different topic. -- Bo Maryniuk SUSE LINUX Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer HRB 16746 (AG Nürnberg) ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/30/2011 06:58 PM, Duncan Mac-Vicar P. wrote: An alternative implementation we may consider for a second iteration is to have Spacewalk send the messages via AMPQ to a broker (apache QPid seems to have a reasonable footprint), and the current daemon could fetch from there to the outputs, turning itself into a simple agent. Yes, and additionally to that, while AMQP (a *protocol*) makes more sense to integrate various distributed software rather then make components talk to each other on a single local host, still AMQP itself does not abolishes the fact that the messages are still needs to be delivered from its queue to an actual audit log analyzer in a single format. -- Bo Maryniuk SUSE LINUX Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer HRB 16746 (AG Nürnberg) ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/29/2011 04:32 PM, Ionuț Arțăriși wrote: So if the auditlogging server is down, then the XMLRPC API is down. The whole point of audit logging is to not let anything important pass through without having a record of it. You can of course turn the logging off from the server. And somehow the role Cliff mentions is done by the daemon itself, acting as a buffer/journal between the client and the outputs (syslog, database, etc). -- Duncan Mac-Vicar P. - http://www.suse.com/ SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) Maxfeldstraße 5, 90409 Nürnberg, Germany ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On Tue, Aug 23, 2011 at 06:26:18PM +0200, Bo Maryniuk wrote: The main core of the logger is a separate daemon that serves XML-RPC listener to a localhost only. Then each software component in the Spacewalk is going to send an logging event message through it. Currently, every Spacewalk component talks to one datastore, the underlying database. Why cannot that component log directly to the database to which it already has open connection, rather than trying to log to another daemon which might be down? Logging to the underlying database has the additional benefit of being transactional if you need it to be -- if the action did not take place for whatever reason and it was rolled back, so will the logging event, unless you decide to explicitly commit it, potentially in an autonomous transaction. Daemon is designed in a queue fashion, where request gets immediate response, message is accepted inside an embedded database and then processed accordingly. This way it won't impact software components performance. What if the daemon is down? Will it stop Spacewalk components from working? The API for XML-RPC are intentionally generic: they accept a login ID, a string message and a key/value map. While first two parameters are pretty obvious how to use, Nope. What is the login ID supposed to represent? Impact on a Spacewalk Entire Spacewalk components are going to be altered, where each audit-able action will perform a call. Please give me an example of three different events / code points that you would consider an auditable action. Q: Why not AMQP or other messaging? A: We considered that. But concluded that we rather want it small, straightforward and simple, available everywhere: http://goo.gl/usti8 :) So, what is the size of that daemon and its plugins, compared to AMQP? I'd rather not have yet-another-message-queue system in Spacewalk to maintain when there are already multiple open-source project achieving similar task. If the architecture is generic enough, you should be able to use AMQP instead, shouldn't you? So why not start with using AMQP as the daemon, and making your daemon optional? Q: Why not just another log appender? A: We believe it should be generic, agnostic and reliable. Hence the embedded database and thread black magic are involved among other things. :) I'd consider embedded database and thread black magic to be in direct conflict with the reliable requirement. We are going to open it under Apache Licence 2.0 and contribute back to If the patch should have any chance of being included in Spacewalk upstream, it has to be under GPLv2 as well, since according to http://www.apache.org/licenses/GPL-compatibility.html those two licences are not compatible. the community along with a mega-patch for Spacewalk 1.2 base. it the patch should have any chance of being included in Spacewalk upstream, it has to be a stream of patches again Spacewalk master, not against Spacewalk 1.2. In other words, unless the patch applied without conflicts to Spacewalk master (in which case you'd probably make it a patch against master in the first place), I doubt there will be many people willing/capable of forward porting the changes. -- Jan Pazdziora Principal Software Engineer, Satellite Engineering, Red Hat ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On Wed, Aug 24, 2011 at 01:49:27PM +0200, Bo Maryniuk wrote: On 08/24/2011 01:20 PM, Tomas Lestach wrote: what was the original use/business case for the audit logging add-on? To know who did what, when and from where. To achieve that, I'll gladly review a patch which will add logging triggers to all tables that we have in the schema, together with initial insert to a central log table with identity/timestamp/remote host/user agent/whatever information. How do you plan to present the audit information? I plan to send it where it is needed in the way the destination is asking. So it depends. Yes but for the feature to be easily consumable by the system administrators and/or auditors, in needs to have some sort of presentation in the WebUI and possibly be available over API or in reports. We already have something like that in the rhnserveraction and rhnserverhistory tables and in the /network/systems/details/history/history.pxt?sid=sid Web page -- do you plan to amend it? -- Jan Pazdziora Principal Software Engineer, Satellite Engineering, Red Hat ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On Thu, Aug 25, 2011 at 02:30:51PM +0200, Johannes Renner wrote: Well, the general 'problem' is that Spacewalk consists of a lot of different components and this audit log should gather log events coming from these sources (web ui, frontend api, backend api, taskomatic), while sending them as one 'trail' to a destination that might be whatever (syslog, database, etc.). Therefore we decided to use yet again xmlrpc for the interface, because there is at least an easy-to-use client implementation available for every scripting language that is used within Spacewalk. I'd much rather see a patch which would make the set of used technologies / stacks smaller, for example a patch which would finish the Perl-to-Java migration of the outstanding .pxt Web pages. Also, the completness and accuracy of the trail depends on thousands of new calls to be added throughout the code base. Do we really need that when we already have a place where all actions/changes end (the database) and we could do the auditing/logging from there? While the implementation of the daemon is a totally different story, I want to show you how the code might look like for the Java/Web UI part. I attached some initial patches I have written and would really like to get review and feedback on this. So (1.) there are some helper classes that I put into a new separate package, see 01-new-classes.patch: - AuditLog is a Singleton that serves as the entry point for calls to I don't like the fact that the essential information of true/false is massaged into a string -- shouldn't that be a first-class citizen fields there? The same for the user name -- what if I got and create user named 'none'? Do you really want the message to be localized? That would mean that they can change in time easily. Shouldn't the audit messages be as solid/stable as possible for automatic processing, and only be localized upon presentation? the log. The interface of the log daemon conists of only one method that is called log(). - This method is called via xmlrpc from within the AuditLogAppender. So this AuditLog is server (the daemon) side, or the client (the tomcat or taskomatic) side? - AuditLogMessage is just a wrapper object representing a log event. Don't you need to have toString there? What constructor of AuditLogMessage does the AuditLogAppender use? - LogUtil is just a class containing helper methods. The most interesting part is probably the createExtMap() method that takes an HTTPServletRequest and extracts its parameters into a HashMap while ignoring some of them using a blacklist. This map is also passed on to the logger. The LogUtil.get*String stringifies the parameters without quoting them, making it virtually impossible to process in an automatic manner later. Shouldn't the values stay as map as long as possible, so that for example the database backend can then easily and safely detect values of those parameter and store them in table columns specific for the given type, ideally with foreign keys? For the actual logging of web UI actions, I investigated into generic approaches first (using a ServletFilter, do it in RhnRequestProcessor), but I found out the result of all these approaches is not really what we want. Interesting events are not really correlating to general HTTP POST requests, unfortunately :-/ Can you be more specific about this? I further added (3.) generic audit logging of API calls, please see my attached 04-xmlrpc-api.patch: - This patch also fixes missing IP adresses in already existent logs (I can also provide this as a single patch, if I find the time..). That would be the right thing to do, obviously. - While most of the struts actions are RhnAction or RhnLookupDispatchAction, there is still some that are not derived from these classes. Therefore it's not enough to put a shortcut method to the logger in the(se) superclass(es). By all means, refactor code which needs to be refactored and submit as patch against master, instead of supporting actions coded not in the latest-greatest technology style. - Also, because sometimes an action contains the implementation of actually different things, one log message per action class will not be enough. In addition to that sometimes singular as well as pluralized messages would be needed. As mentioned above, I believe localization has no business in this side of the code. You have languages that distinguish more than just singular and plural. - Sometimes when a form is submitted it is actually hard to find out if _really_ something has changed, Why is that possible? Isn't this confirming my idea that this all should be done on the database level where you'd know for sure if something has been modified or not (because if not, the trigger would not fire)? -- Jan Pazdziora Principal Software Engineer, Satellite Engineering, Red Hat ___ Spacewalk-devel mailing
Re: [Spacewalk-devel] Audit Logging
On 08/30/2011 12:46 PM, Jan Pazdziora wrote: I'd much rather see a patch which would [...] finish the Perl-to-Java migration of the outstanding .pxt Web pages. The topic is about Audit logging right now and you have your own personal opinion on this. There will be always various ideas, of course... What if the daemon is down? That was answered multiple times already in detail. So why not start with using AMQP as the daemon, and making your daemon optional? It is optional. If the patch should have any chance of being included in Spacewalk upstream, it has to be under GPLv2 as well, since according to http://www.apache.org/licenses/GPL-compatibility.html Of course. Spacewalk patch will be GPL as it is directly linked to. -- Bo Maryniuk SUSE LINUX Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer HRB 16746 (AG Nürnberg) ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On Fri, Aug 26, 2011 at 04:10:03PM -0400, Cliff Perry wrote: Do you do it within the schema (and loose the who from where did what - unless all app code passes it down to DB). No. You do it once, at the start of the HTTP request handling, store that to the database and use identifier throughout your HTTP request processing. You have the identifier in (say) package variables in Oracle and in temporary table or environment variable in PostgreSQL, and you don't have to change a bit of application code. And if new application Y which does the same thing like X, except it does some filtering here and there and also touches objects Z gets created, you don't have to aprinkle new auditing calls into it -- you just log on the data level. -- Jan Pazdziora Principal Software Engineer, Satellite Engineering, Red Hat ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
* Jan Pazdziora jpazdzi...@redhat.com [Aug 30. 2011 12:07]: Currently, every Spacewalk component talks to one datastore, the underlying database. Why cannot that component log directly to the database to which it already has open connection, rather than trying to log to another daemon which might be down? One of the requirements we have to fulfill with the audit log is to make it tamper proof. IIRC, a Spacewalk admin has full access rights to the underlying database and could change/delete log entries. Klaus --- SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) Maxfeldstraße 5, 90409 Nürnberg, Germany ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/30/2011 12:06 PM, Jan Pazdziora wrote: To achieve that, I'll gladly review a patch which will add logging triggers to all tables that we have in the schema, together with initial insert to a central log table with identity/timestamp/remote host/user agent/whatever information. How would you log read only access? -- Miroslav Suchy Red Hat Satellite Engineering ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/30/2011 01:51 PM, Jan Pazdziora wrote: Sure. I'm just pointing out that you might save yourself some architecture and coding work if you consider the bigger picture and goals of the project and not just blindly focus on the narrow goal you have on your plate right now. The narrow goal that we have on our plate right now is basically the reason why we are implementing this feature. Of course we want this to be upstream, yes, and that is why Bo and Johannes started the discussion to get feedback of what could be improved. I have the impression the discussion is getting further from finding a common view as it progressed (the first replies where very valid suggestions). But now we are already on the topic of porting perl pages to Java. While we sure want to help with this in the future (monitoring anyone), it is not in the scope of what we need to deliver and we can't get out of our schedule in order to find the ultimate solution. If there is no agreement or the visions are too different, may be it would make sense to keep this in our tree for now and we would need to make the right decisions so that merging does not become too hard in the future. I don't find this ideal, because some of the proposals+feedback are not that intrusive, so they could be in the spacewalk tree and would be mostly optional for the user. Even the daemon is a XML-RPC interface so it can easily be replaced. Adding a logging setup to two stacks surely is less work than adding it to three, and if there still is the common goal of finishing the migration, I consider it a valid option to consider, whether to switch the order in which work is done and make things easier in total. I think Bo gave very a good explanation why this is not Logging but Auditing: more contextual/semantic data, security, tamper-proof, use cases, etc. -- Duncan Mac-Vicar P. - http://www.suse.com/ SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) Maxfeldstraße 5, 90409 Nürnberg, Germany ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/29/2011 03:14 PM, Cliff Perry wrote: In short, my personal preference is not to become a toaster if audit logging has crashed (including out of space to write). An alternative implementation we may consider for a second iteration is to have Spacewalk send the messages via AMPQ to a broker (apache QPid seems to have a reasonable footprint), and the current daemon could fetch from there to the outputs, turning itself into a simple agent. -- Duncan Mac-Vicar P. - http://www.suse.com/ SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) Maxfeldstraße 5, 90409 Nürnberg, Germany ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On Tue, Aug 30, 2011 at 05:29:48PM +0200, Miroslav Suchý wrote: On 08/30/2011 12:06 PM, Jan Pazdziora wrote: To achieve that, I'll gladly review a patch which will add logging triggers to all tables that we have in the schema, together with initial insert to a central log table with identity/timestamp/remote host/user agent/whatever information. How would you log read only access? I would not. Is that part of the requirement? So far in the proposal/patches I've only seen actions that modify something being logged. -- Jan Pazdziora Principal Software Engineer, Satellite Engineering, Red Hat ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/29/2011 08:01 AM, Ionuț Arțăriși wrote: Just to add to the discussion, I'll tell you how I thought about logging the python backend XMLRPC API calls. This needs a lot less filtering than logging the WebUI though, since we're interested in most of the methods, so it's a lot easier. We decided we only wanted to log successful API calls that are deemed important. So far this means that the api method requires authentication (and therefore the systemid XML) file to be sent with it. In the apacheRequest, after the function is called and before the response is sent back, an auditlog function is called with all the information about the XMLRPC method that was called, its arguments and the request. I think this is the only central place from which the logging function can be called. The auditlog function belongs to a separate module and it does all the processing to extract the information that we deem useful (ips/hosts/serverId etc.). It also decides if the call is important enough to be logged or just doesn't do anything if auditlogging is turned off in an rhn.conf file. It calls the auditlogging XMLRPC server with the result and also crashes the whole API request if anything goes wrong with the logging. One of the main design decisions I would consider for behaviour is... If the logging daemon is down, do all actions fail - or do you still allow successful actions, knowing that it cannot log the event. - Or do you have a fall-back temporary on-disk cache to be played back into the logger once it is running again. In short, my personal preference is not to become a toaster if audit logging has crashed (including out of space to write). Cliff Feedback appreciated :) -Ionuț ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/25/2011 06:55 PM, Johannes Renner wrote:
Well, I can count three, while only one (!) of them is public. But we
can still rename the other ones, no problem.
I count:
private static Logger log = Logger.getLogger(AuditLog.LOGGER_NAME);
as well.
- How can you distinguish between interesting and uninteresting requests?
- There is some interesting requests that are not POST requests, e.g. logouts.
- There is some POST requests that are not interesting, e.g user selects all
entries of a list.
- Log events cannot be categorized for a later filtering. At a single entry
point it is very hard to see what really has happened.
I thought that there will be configuration file, where you state what
and how will be logged. All based on URI similary to struts config file.
E.g.
/rhn/LoginSubmit.do {
key = LOGIN
value = user=${POST.username};pass=${POST.password}
}
/rhn/admin/config/GeneralConfig.do {
key = CONF
value = email=${POST.email};.
}
etc. you probably got the idea now.
And those url not specified will not be logged.
- When using an external entry point (like mod_security), you can't actually
see from the logs which user was involved since it is not possible to map
between uid, sid, ... and real world 'objects'.
I said something liek mod_security. I can imagine build something upon
existing project, but even something new written from scratch just for
Spacewalk.
And translating sid to user is not so big problem. You can have config
file where you specify how you translate sid to user. Ie.
[translate]
user = select login from web_contact join pxtsession on
web_user_id=web_contact.id where pxt.id = :sid
and in logging config have:
/rhn/admin/config/GeneralConfig.do {
key = CONF
translate[user] = sid
value = logged=${user};email=${POST.email};.
}
This way it can be even Spacewalk independent and you can use it on
different project where they have different tables.
I agree with you completely on the fact that getting the big picture is hard,
but generic logging of request data does somehow not satisfy our needs :-/
So there is place to write one :)
Just think that after some years customer will ask you and which
events/action/url are logged? Can you give me the list.
And you will have hard time to provide such list.
--
Miroslav Suchy
Red Hat Satellite Engineering
___
Spacewalk-devel mailing list
Spacewalk-devel@redhat.com
https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/25/2011 07:10 PM, Bo Maryniuk wrote: I thought the patches reviews you, guys, doing are more than just works/not-works? Not exactly. This is true for small bugs. E.g there is some bug in Spacewalk. Somebody diagnose it, create patch and sent it here (unless he is proved Spacewalk contributor and has git access). We review it and if it works, we commit it. But even then we often fix some code style. Trim trailing whitespaces, wrap lines... etc. But if it is feature, then we try to public face it as much and as soon as possible. And we try to find solution which is as generic as possible and which will be maintainable even few years from now. I have to admit that we discuss some things internally in our cubicle as RH developers sits in one room, but when we are not lazy we post it to mailing list to get feedback from others as well. Debian support and PostgreSQL support comes to my mind as good example where we discussed our progress during our work. -- Miroslav Suchy Red Hat Satellite Engineering ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/26/2011 11:02 AM, Miroslav Suchý wrote:
On 08/25/2011 06:55 PM, Johannes Renner wrote:
Well, I can count three, while only one (!) of them is public. But we
can still rename the other ones, no problem.
I count:
private static Logger log = Logger.getLogger(AuditLog.LOGGER_NAME);
as well.
While this is not a method and also not a function, however ...
- How can you distinguish between interesting and uninteresting requests?
- There is some interesting requests that are not POST requests, e.g.
logouts.
- There is some POST requests that are not interesting, e.g user selects all
entries of a list.
- Log events cannot be categorized for a later filtering. At a single entry
point it is very hard to see what really has happened.
I thought that there will be configuration file, where you state what
and how will be logged. All based on URI similary to struts config file.
E.g.
/rhn/LoginSubmit.do {
key = LOGIN
value = user=${POST.username};pass=${POST.password}
}
/rhn/admin/config/GeneralConfig.do {
key = CONF
value = email=${POST.email};.
}
etc. you probably got the idea now.
And those url not specified will not be logged.
I got the idea and I was even researching such an approach already. For the
webapp I started with a ServletFilter (see my attached patch as an example)
that simply logs all POST requests to the backend using my helper classes
from the patches I already sent. The main thing that's missing would be the
integration of a configuration file like the above.
I will now continue to investigate in this approach since I agree with you
that it will be much easier to maintain than having log statements all over
the code. However there is also some drawbacks:
- Performance might be worse, since _every_ request this filter is registered
for (e.g. all *.do) will be checked if it needs to be logged or not
- Sometimes the same URL is used for different actions, e.g. creating and
updating an object, so classification of log events might be difficult or
even not possible
- Sometimes you only want to log the request in case a certain parameter is
there, so there would need to be a something like a list of parameters
required for logging for each URL in the config
- does it make sense to have a whitelist of interesting parameters for each
URL or rather take everything and maintain a global blacklist?
- When using an external entry point (like mod_security), you can't actually
see from the logs which user was involved since it is not possible to map
between uid, sid, ... and real world 'objects'.
I said something liek mod_security. I can imagine build something upon
existing project, but even something new written from scratch just for
Spacewalk.
And translating sid to user is not so big problem. You can have config
file where you specify how you translate sid to user. Ie.
[translate]
user = select login from web_contact join pxtsession on
web_user_id=web_contact.id where pxt.id = :sid
and in logging config have:
/rhn/admin/config/GeneralConfig.do {
key = CONF
translate[user] = sid
value = logged=${user};email=${POST.email};.
}
This way it can be even Spacewalk independent and you can use it on
different project where they have different tables.
Yes, but I actually think it would make sense to do this specifically within
spacewalk-java, because there is already code to determine all the stuff from
the database. To me it would make sense to reuse this code, so we don't need
to rewrite all those queries?
I agree with you completely on the fact that getting the big picture is hard,
but generic logging of request data does somehow not satisfy our needs :-/
So there is place to write one :)
Just think that after some years customer will ask you and which
events/action/url are logged? Can you give me the list.
And you will have hard time to provide such list.
Yes, and such a configuration could even be modified by a customer itself.
--
SUSE LINUX Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg)
diff --git a/java/code/src/com/redhat/rhn/frontend/servlets/AuditLogFilter.java b/java/code/src/com/redhat/rhn/frontend/servlets/AuditLogFilter.java
new file mode 100644
index 000..df3c728
--- /dev/null
+++ b/java/code/src/com/redhat/rhn/frontend/servlets/AuditLogFilter.java
@@ -0,0 +1,70 @@
+/**
+ * Copyright (c) 2011 Novell
+ *
+ * This software is licensed to you under the GNU General Public License,
+ * version 2 (GPLv2). There is NO WARRANTY for this software, express or
+ * implied, including the implied warranties of MERCHANTABILITY or FITNESS
+ * FOR A PARTICULAR PURPOSE. You should have received a copy of GPLv2
+ * along with this software; if not, see
+ * http://www.gnu.org/licenses/old-licenses/gpl-2.0.txt.
+ *
+ * Red Hat trademarks are not licensed under GPLv2. No permission is
+ * granted to use or
Re: [Spacewalk-devel] Audit Logging
On 08/26/2011 03:10 PM, Johannes Renner wrote: I got the idea and I was even researching such an approach already. For the webapp I started with a ServletFilter (see my attached patch as an example) that simply logs all POST requests to the backend using my helper classes from the patches I already sent. The main thing that's missing would be the integration of a configuration file like the above. I will now continue to investigate in this approach since I agree with you that it will be much easier to maintain than having log statements all over the code. However there is also some drawbacks: - Performance might be worse, since _every_ request this filter is registered for (e.g. all *.do) will be checked if it needs to be logged or not *nod* - Sometimes the same URL is used for different actions, e.g. creating and updating an object, so classification of log events might be difficult or even not possible Can you provide example of such page. If we can solve such hard page and everything else will be easier, then we can continue persuade this idea. Otherwise we can scratch it and return to that mega-patch modifying all the code. - Sometimes you only want to log the request in case a certain parameter is there, so there would need to be a something like a list of parameters required for logging for each URL in the config *nod* - does it make sense to have a whitelist of interesting parameters for each URL or rather take everything and maintain a global blacklist? What about reusing idea from httpd. I.e have order deny, allow deny foo so everything having foo will be blacklisted and everything else for this url will be audited and similary order allow, deny allow foo will mean that we will not audit it unless it have foo parameter. Yes, but I actually think it would make sense to do this specifically within spacewalk-java, because there is already code to determine all the stuff from the database. To me it would make sense to reuse this code, so we don't need to rewrite all those queries? But how will you audit backend and those old perl pages we still have there? Yes, and such a configuration could even be modified by a customer itself. Indeed, I did not seen this advantage. -- Miroslav Suchy Red Hat Satellite Engineering ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/25/2011 11:43 AM, Miroslav Suchý wrote: On 08/25/2011 02:30 PM, Johannes Renner wrote: - While most of the struts actions are RhnAction or RhnLookupDispatchAction, there is still some that are not derived from these classes. Therefore it's not enough to put a shortcut method to the logger in the(se) superclass(es). In a way this high lights some of the technical hurdles that would need to be tackled in any 'auditing' capability for Spacewalk or Satellite. Do you do it within the schema (and loose the who from where did what - unless all app code passes it down to DB). Do you do it say in Apache where you have IP addresses, GET/POST and other details - but you got no idea what is in the SSM for session, so that delete POST for SSM of lots of systems still doesn't know what was deleted. Touching *lots* of code to me is likely the main way to achieve this and any implementation will be up for heated discussion on best approach. Still reading the thread, I'm very interested to see where this goes. Cliff Exactly, if you go from bottom (base classes) you can be never sure if log all childs and getting the big picture will be very hard. And you could not be really sure till you test it. ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/24/2011 01:57 PM, Bo Maryniuk wrote: Well, if your admin can do more than 2000 changes-per-second, then just let me know... :) You want to audit just frontend? I.e. if I do some changes through backend it will not be logged? -- Miroslav Suchy Red Hat Satellite Engineering ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/25/2011 04:18 PM, Miroslav Suchý wrote: You want to audit just frontend? No, everything. I.e. if I do some changes through backend it will not be logged? No, they will. -- Bo Maryniuk SUSE LINUX Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer HRB 16746 (AG Nürnberg) ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/25/2011 02:30 PM, Johannes Renner wrote: So (1.) there are some helper classes that I put into a new separate package, see 01-new-classes.patch: AuditLog.java has 4 (!) functions log() and all of them has different semantics - if I ommit that they are utilized to logging. Please consider different names. Therefore there is (2.) calls to the logger from within the existing code, see 03-rhnaction-example.patch: Hmm, what I dislike is the need to change the code. Which will be prone to coder error and you may end up in situtation where somebody commit code, which are worth auditing, but the audit call will not be there. If I would be designing such functionality I would prefer something like mod_security. So you will have one entry point and you will not care if the action is actually handled by cobbler, backend, frontend or some aliens. This would even allow you to have configuration file where you will define what and how will be audited. In your design it will be hard to say which actions are audited. Especially in 2 years from now, when other will add/remove/change your initial audit calls. - While most of the struts actions are RhnAction or RhnLookupDispatchAction, there is still some that are not derived from these classes. Therefore it's not enough to put a shortcut method to the logger in the(se) superclass(es). Exactly, if you go from bottom (base classes) you can be never sure if log all childs and getting the big picture will be very hard. And you could not be really sure till you test it. -- Miroslav Suchy Red Hat Satellite Engineering ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/25/2011 05:28 PM, Bo Maryniuk wrote: I.e. if I do some changes through backend it will not be logged? No, they will. Well we have users with tens thousands or registered machines. And they can easily create hundreds and even thousands request in the same time. So human admin is not capable of thousands request per seconds, but registred machines are capable of that. So if we accept that in Spacewalk eventually. I would definitelly like to see some benchmarks before this feature and after. And you to be willing to address potential speed issues. -- Miroslav Suchy Red Hat Satellite Engineering ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/25/2011 05:43 PM, Miroslav Suchý wrote: On 08/25/2011 02:30 PM, Johannes Renner wrote: So (1.) there are some helper classes that I put into a new separate package, see 01-new-classes.patch: AuditLog.java has 4 (!) functions log() and all of them has different semantics - if I ommit that they are utilized to logging. Please consider different names. Well, I can count three, while only one (!) of them is public. But we can still rename the other ones, no problem. Therefore there is (2.) calls to the logger from within the existing code, see 03-rhnaction-example.patch: Hmm, what I dislike is the need to change the code. Which will be prone to coder error and you may end up in situtation where somebody commit code, which are worth auditing, but the audit call will not be there. If I would be designing such functionality I would prefer something like mod_security. So you will have one entry point and you will not care if the action is actually handled by cobbler, backend, frontend or some aliens. This would even allow you to have configuration file where you will define what and how will be audited. In your design it will be hard to say which actions are audited. Especially in 2 years from now, when other will add/remove/change your initial audit calls. I completely agree and also dislike the need to change the code, but: If we wanted to log only HTTP requests and responses we could very well use something like mod_security, or alternatively use a ServletFilter or put the implementation into RhnRequestProcessor. Such an implementation would be much easier and there would be a single entry point, but there is several disadvantages as well, e.g.: - How can you distinguish between interesting and uninteresting requests? - There is some interesting requests that are not POST requests, e.g. logouts. - There is some POST requests that are not interesting, e.g user selects all entries of a list. - Log events cannot be categorized for a later filtering. At a single entry point it is very hard to see what really has happened. - When using an external entry point (like mod_security), you can't actually see from the logs which user was involved since it is not possible to map between uid, sid, ... and real world 'objects'. So, it depends a bit on how the requirements are, but to me it looks like all generic solutions will unfortunately not really meet our requirements. - While most of the struts actions are RhnAction or RhnLookupDispatchAction, there is still some that are not derived from these classes. Therefore it's not enough to put a shortcut method to the logger in the(se) superclass(es). Exactly, if you go from bottom (base classes) you can be never sure if log all childs and getting the big picture will be very hard. And you could not be really sure till you test it. It's supposed to be done in the superclass, this is just a shortcut for making it easier to put the calls in the derived classes where 'interesting' things (mostly calls to whatever *Manager class) are actually happening. I agree with you completely on the fact that getting the big picture is hard, but generic logging of request data does somehow not satisfy our needs :-/ -- SUSE LINUX Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/25/2011 05:43 PM, Miroslav Suchý wrote: AuditLog.java has 4 (!) functions log() and all of them has different semantics - if I ommit that they are utilized to logging. Please consider different names. Four? Functions?.. I see three and they are methods, where only one is public. This is called overloading and was around for a few decades... Why you dislike that part? In your design it will be hard to say which actions are audited. Especially in 2 years from now, when other will add/remove/change your initial audit calls. I thought the patches reviews you, guys, doing are more than just works/not-works? :) Generic approach is exactly what we already tried to do at the very beginning. -- Bo Maryniuk SUSE LINUX Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer HRB 16746 (AG Nürnberg) ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
Hey Bo, what was the original use/business case for the audit logging add-on? How do you plan to present the audit information? Regards, Tomas -- Tomas Lestach RHN Satellite Engineering, Red Hat On 08/23/2011 06:26 PM, Bo Maryniuk wrote: Hi, all! Here at SUSE we are working on an audit logging. As we all perfectly know, the Audit Logging is not just a logging in the sense as a syslog for like debugging or alerting etc. We developing an add-on for the Spacewalk. It should collect events from various components and then bring them all to a single point, thus send later to a various destinations (XDAS, databases, indexers etc). Hence I would like shortly introduce what this is all about. Architecture The main core of the logger is a separate daemon that serves XML-RPC listener to a localhost only. Then each software component in the Spacewalk is going to send an logging event message through it. Daemon is designed in a queue fashion, where request gets immediate response, message is accepted inside an embedded database and then processed accordingly. This way it won't impact software components performance. The API for XML-RPC are intentionally generic: they accept a login ID, a string message and a key/value map. While first two parameters are pretty obvious how to use, the key/value map is specific to the scope of audited software. The keys are validated according to a defined structure format, which can be totally different for another components or software group. The validator is a plug-in, which implements specific structure validation. In this way we have a generic auditing collector that can be reused elsewhere or integrated with just anything. There are plugins for: 1. Delivering messages. 2. Validating key/value mapping to prevent posting inconsistent nonsense. Delivering message plugins are: 1. RDBMS (PostgreSQL and Oracle). Stores data in two tables. 2. XML file. Appends data to an XML file. 3. Syslog (TCP, UDP and local). Has advantage to allow reconcile multi-line posts, once messages are too big to fit in one line. 4. STDERR and STDOUT :-) Validator plugins: 1. Spacewalk schema validator. Plugins can be reused in an unlimited combinations and instances: you can feed several databases, syslogs, XML files etc, if needed. Impact on a Spacewalk Entire Spacewalk components are going to be altered, where each audit-able action will perform a call. We already developed an log4j appender and now processing all Java part and XML-RPC backend and a frontend. This will help community to reconnect audit to something totally else, if they would really like to do so. Perl/Python-based and other components will always talk to the audit logger over XML-RPC, thus never link directly. The feature will be turned off by a default and will be able to turn it on in a Spacewalk conf, like: audit = on. You might ask... Q: Why not AMQP or other messaging? A: We considered that. But concluded that we rather want it small, straightforward and simple, available everywhere: http://goo.gl/usti8 :) Q: Why not just another log appender? A: We believe it should be generic, agnostic and reliable. Hence the embedded database and thread black magic are involved among other things. :) Q: A daemon? A: Yes. A daemon. Because if this would be a servlet on a Tomcat, so once Tomcat feels sour during various reasons, like a star wars satellite accidentally blew up WAN or endothermal recalibration caused failure converting big to little endian, for example :-), then the rest of the stack won't properly functioning. That should not happen. Q: How fast the thing is? A: 1000 messages linearly per 0.42 sec per instance should be enough. Q: Does it takes care of being tamper-proof? A: No. The software component is responsible only to collect various wild components and comb the output to a single standard that will be delivered to the enterprise archive, like EMC Centera, for example. Contribution plans We are going to open it under Apache Licence 2.0 and contribute back to the community along with a mega-patch for Spacewalk 1.2 base. If you have any feedback to this and ideas what else could be good, we welcome you to share your thoughts. Have a lot of fun! -- Bo Maryniuk SUSE LINUX Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer HRB 16746 (AG Nürnberg) ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/23/2011 06:26 PM, Bo Maryniuk wrote: Entire Spacewalk components are going to be altered, where each audit-able action will perform a call. We already developed an log4j appender and now processing all Java part and XML-RPC backend and a frontend. This will help community to reconnect audit to something So you already have something. Can you send what you have, so we can review the implementation? It would be unfortunate if you send mega patch and we will do not like several items and reject it. It does not need to be polished or apply to current HEAD. And it does not need to be complete. Just to see the code behind your ideas. The feature will be turned off by a default and will be able to turn it on in a Spacewalk conf, like: audit = on. Definitelly. Most users will not need this one, so it should be disabled by default. Q: Why not just another log appender? A: We believe it should be generic, agnostic and reliable. Hence the embedded database and thread black magic are involved among other things. :) This is where I disagree. But I would like to see your code and maybe I will change my mind. Q: A daemon? A: Yes. A daemon. Because if this would be a servlet on a Tomcat, so once Tomcat feels sour during various reasons, like a star wars satellite accidentally blew up WAN or endothermal recalibration caused failure converting big to little endian, for example :-), then the rest of the stack won't properly functioning. That should not happen. *nod* Q: How fast the thing is? A: 1000 messages linearly per 0.42 sec per instance should be enough. That is quite slow IMO for logger. We are going to open it under Apache Licence 2.0 and contribute back to the community along with a mega-patch for Spacewalk 1.2 base. Please no mega-patch. And definitelly no unless you send few iteration of what-you-have patch, so we can review and discuss whether it is good approach or not. This is thing which can be done good and be great benefit, but with only few things screwed can slow down whole Spacewalk even in disabled state. -- Miroslav Suchy Red Hat Satellite Engineering ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/24/2011 01:20 PM, Tomas Lestach wrote: what was the original use/business case for the audit logging add-on? To know who did what, when and from where. How do you plan to present the audit information? I plan to send it where it is needed in the way the destination is asking. So it depends. -- Bo ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] Audit Logging
On 08/24/2011 01:41 PM, Miroslav Suchý wrote: Q: Why not just another log appender? A: We believe it should be generic, agnostic and reliable. Hence the embedded database and thread black magic are involved among other things. :) This is where I disagree. But I would like to see your code and maybe I will change my mind. Probably I had not the best explanation here. Actually, there *are* log appenders at the Spacewalk part, so we just continue to use the same log4j the same fashion in the Java part, Python logger etc. But the appenders are actually an XML-RPC clients to the daemon. Then since you have various destinations, like a slow and remote RDBMS, like other solutions, the daemon is *separately* delivers them, after an event already been accepted. Q: How fast the thing is? A: 1000 messages linearly per 0.42 sec per instance should be enough. That is quite slow IMO for logger. Well, if your admin can do more than 2000 changes-per-second, then just let me know... :) -- Bo Maryniuk SUSE LINUX Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer HRB 16746 (AG Nürnberg) ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
