Hello Chris, As promised, here is the gist: https://gist.github.com/Jorricks/cf372012f32be21d33cbe53c8c7397e4
I need to add a couple disclaimers: 1. I removed our custom logic of what we do with the events for security reasons. 2. We do not use the `@provide_session` decorator but instead made our own session decorator. We did this because you get SQLAlchemy errors if you load in an item that is used in the same action by Airflow. 3. When you create a variable, it is currently also considered as if you change the variable. This is because we don't correctly filter out this case at this point. If you have any trouble or questions, please let me know and I'll see what I can do to help you out. Op wo 30 mrt. 2022 om 16:07 schreef [email protected] <[email protected]>: > Hi Jorrick Sleijster, if the offer still stands I wouldn't mind seeing any > code snippets you could share from your approach. Thanks :) > > > > On Mar 22, 2022, at 12:29 AM, [email protected] wrote: > > > > Thanks guys. I'll play around with it and see what I can come up with > > > > > >> On Mar 19, 2022, at 5:56 AM, Jarek Potiuk <[email protected]> wrote: > >> > >> I think it is a good idea. > >> > >> And possibly even doing it via sqlalchemy listeners (as long as this > >> is configurable) might be a good idea - this might make the > >> implementation simpler (no need to worry about API/WWW/CLI separately) > >> and if it will become part of the Airflow Core, rather than through a > >> plugin, it might be added to our CI and become "easily maintainable" > >> part of Airflow. > >> > >>>> On Sat, Mar 19, 2022 at 8:56 AM Jorrick Sleijster < > [email protected]> wrote: > >>> > >>> Hi Chris, > >>> > >>> We had the same wish around a bit more than a year ago and approached > it from a different angle. We created SQLAlchemy listeners on the Airflow > models for connections and variables (kinda hacky right?). This meant that > we were able to send notifications about who modified something and what > they modified. > >>> This was added using an Airflow plugin. If you want I can create a > minimal example to showcase how it would work. > >>> > >>> On the PR idea, even though you can already achieve the requested > behavior with the above tactic, having a less hacky interface would be nice. > >>> > >>> I think the cluster policy could be considered. However, it feels like > this serves a much different purpose than what we are looking at now and > therfore doesn't feel suitable. We could take a similar approach as the > logger config/celery config in which you specify an importable class/object. > >>> > >>> In my mind a default class could be created that has many different > functions but has 'pass' as code. This way the user can extend this class > and hopefully have a more fine tuned interface than a single function entry > point. > >>> > >>> Personally, I'd be interested in the PR and argue that this should be > added but I'd love to hear more opinions. > >>> > >>>> On Fri, 18 Mar 2022, 17:31 Chris Redekop, <[email protected]> > wrote: > >>>> > >>>> Hi all! I'm looking for some advice/direction/opinions... > >>>> > >>>> We have a need to be able to audit and/or send alerts whenever > someone performs a potentially high-impact operation in the UI. > Specifically we want to be able to audit all modifications made to > variables and connections, and we'd like the ability to send alerts > whenever someone pauses or unpauses a DAG. We're planning to do the > changes ourselves and submit a PR but I'd like to get some feedback before > starting. The general approach in my mind was to add connection+variable > mutations to the existing audit logs, and also introduce a new cluster > policy hook "on_administrative_event(object_type, action_performed, actor, > event_details)" (or something like that) which would be invoked after > (potentially) any administrative object is modified. > >>>> > >>>> So, is there value to this sort of thing for airflow in general? > Would a PR along these lines have a chance at being merged? Any advice on > direction/approach would be greatly appreciated....I've never ventured this > far into the codebase before so any code orientation tips would also be > appreciated. Thx! > >>>> > >>>> - Chris >
