[Bug 391623] Re: apport hook for Evolution
Thanks for your feedback. Isn't it better to take it in a blueprint? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/391623 Title: apport hook for Evolution To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/evolution/+bug/391623/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 391623] Re: apport hook for Evolution
Actually, this is an item for Ubuntu, not upstream -- this deals with Apport, and the way we do (or dir) things.So it should still be triaged. And... I will check how things are going, and decide on what do to. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/391623 Title: apport hook for Evolution To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/evolution/+bug/391623/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 391623] Re: apport hook for Evolution
Bug from 2009. Version not longer supported. Change status to Invalid ** Changed in: evolution (Ubuntu) Status: Triaged = Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/391623 Title: apport hook for Evolution To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/evolution/+bug/391623/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 391623] Re: apport hook for Evolution
@Jörg Frings-Fürst - What about this particular bug report leads you to believe that it is specific to a certain version of evolution? As far as I know backtraces would still benefit from sanitization and this is not being done for evolution crashes. ** Changed in: evolution (Ubuntu) Status: Invalid = Triaged -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/391623 Title: apport hook for Evolution To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/evolution/+bug/391623/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 391623] Re: apport hook for Evolution
@Brian Murray I think a bug who is not forwarded to gnome since 2009 can realy closed. CU Jörg -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/391623 Title: apport hook for Evolution To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/evolution/+bug/391623/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 391623] Re: apport hook for Evolution
Great idea C de-Avillez. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/391623 Title: apport hook for Evolution -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 391623] Re: apport hook for Evolution
Indeed. After a quite long lull, ddecator and kermiac proposed themselves to work on this, and we are moving the target a big (still being discussed): we want to provide a generic process to clean up backtraces, not something just for Evo. -- apport hook for Evolution https://bugs.launchpad.net/bugs/391623 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 391623] Re: apport hook for Evolution
Looking at the last few comments, marking this as 'patch-needswork' Please feel free to change that to a patch tag once you have a new patch. This patch was reviewed as part of operation cleansweep. Please see https://wiki.ubuntu.com/OperationCleansweep for details on how to help us review all patches in Ubuntu! ** Tags added: patch-needswork -- apport hook for Evolution https://bugs.launchpad.net/bugs/391623 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 391623] Re: apport hook for Evolution
Well, I am back on this. I lost all original bits when I had to reinstall Ubuntu (and a mis-understanding/error on debian-installer formatted the filesystem). So I will backtrack a bit, and get this done. -- apport hook for Evolution https://bugs.launchpad.net/bugs/391623 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 391623] Re: apport hook for Evolution
** Changed in: evolution (Ubuntu) Assignee: Ubuntu Desktop Bugs (desktop-bugs) = C de-Avillez (hggdh2) -- apport hook for Evolution https://bugs.launchpad.net/bugs/391623 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 391623] Re: apport hook for Evolution
** Changed in: evolution (Ubuntu) Status: New = Triaged -- apport hook for Evolution https://bugs.launchpad.net/bugs/391623 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 391623] Re: apport hook for Evolution
The Evolution hook works on the BTs, looking for known variables that
hold (potentially) private data, and -- for any other variable --, we
scan for instances of IP or email addresses, and fully-qualified server
names. All matches are replaced by the string '##MASKED##'.
Of course, this will only be fully effective when bug 387933 is resolved
for the backoffice.
Meanwhile, the hook seems to be working correctly for the list of
Evolution bugs Brian provided me with (BTW, thank you!). The hook
currently:
1. Collects Evolution GConf data ( Plugins, Junk Setup, and Prompts subkeys of
/apps/evolution); these are added in a [Miscellaneous] string;
2. for each of {Stacktrace, ThreadStacktrace): scans the lines, and replaces
any string value for following Evolution variables by the string ##MASKED##:
r'''(key|url_string|url|filename|filesave|uri|profname|user|source|username|password|server|domain|domain_name)
# variables in trace
([\s]*[=].+?[])# intermediate text (class, address, etc)
(.*?) # what we really want: the string data
([][, ]*)''' # the delimiter
3. then we search replace still-existing instances of email addresses,
fully-qualified server names, and IP addresses (in this order), in any other
variables.
4. (Currently) writes a *diff* for the changes made (creates two *new* entries
in reports[]. This was done because we were not sure of how invasive the
changes would be, and considered better to just write a diff, at least for now.
*Input needed*
For both FQSN and email addresses we use the following RE for domain names:
'(aero|arpa|asia|biz|cat|com|coop|edu|gov|info|int|jobs|mil|mobi|museum|name|net|org|pro|tel|travel|[a-z]{2})'
This RE wil match on any of the initial words, or on any two letters.
For IP addresses we use the following RE:
'([^\d])(\d{1,3}[.]\d{1,3}[.]\d{1,3}[.](\d{1,3}))([^\d])'
This RE will match on *any* dotted sequence of one to three digits, enclosed in
non-digits (for example, [1.2.3.4]). It will also match on invalid IP
addresses (since no limits are set on the range; for example it will match on
a912.513.401.12/.
For email addresses we use the following RE:
'[\w\.\...@[\w\.\-]+[.]' + DOMAIN_NAMES
This RE will match on words (plus '.' and '-', followed by an at symbol ('@')
and a DOMAIN_NAME. This is clearly not fully correct (it would allow, for
example, for an email starting with '.'), but it is enough.
For FQSN we use the following RE:
'([^\w])([\w.-]*[.]' + DOMAIN_NAMES + ')([^\w\-]|[\n])'
This RE is very similar to the email RE; the differences are (1) it is
pre/post-fixed with non-words, and has a dot instead of an at symbol.
5. Finally, we currently calculate a diff of the changes to Stacktrace
and ThreadStacktrace, and add it in the report as [Stacktrace.diff] and
[ThreadStacktrace.diff].
6. and exit.
Additional comments:
(a) although the idea is to provide a sanitised stacktrace in order to
allow for the bug to be classified Public, I was reluctant to delete the
original stacktraces: not only I may be missing something, but also
there *might* be a case where the sanitised value would be needed for a
full understanding of the issue. This is why we decided to *add* a diff
for the changes -- a sanitised stacktrace can then be easily achievable
by patching the corresponding stacktrace with its diff. Another option
would be to provide the sanitised stacktrace (removing the original) and
the, er, reverse diff, in order to get the original one.
(b) option (a) would be, in my view, the ideal scenario, but we would
depend on bug 151658 to make attachments and comments private.
I have run this hook against 753 bugs from Brian's list, and it *seems*
it is working correcly. The runs were executed by calling the hook with
the --report parameter; as currently coded, only the .diffs are printed
out.
TO BE DECIDED:
1. should we delete the original traces, and maintain only the saanitised
traces (and, perhaps, a reverse diff)?
2. should we save the original traces, and the diffs?
Note that these two options will not allow for the bug to be marked
public.
3. should we save *only* the sanitised traces, and mark the bug public?
I will provide test data, based on the runs I have.
--
apport hook for Evolution
https://bugs.launchpad.net/bugs/391623
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 391623] Re: apport hook for Evolution
Thank you for your work, the change is not trivial do you think you could add some extra details on the regular expressions and the formats used by giving some basic examples maybe and a testcase which has some values and known updated ones to make sure the code does what is expecteD? ** Changed in: evolution (Ubuntu) Importance: Undecided = Wishlist ** Changed in: evolution (Ubuntu) Assignee: (unassigned) = Ubuntu Desktop Bugs (desktop-bugs) -- apport hook for Evolution https://bugs.launchpad.net/bugs/391623 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 391623] Re: apport hook for Evolution
unsubscribing the sponsors for now so it doesn't stay on the review list while it's being discussed -- apport hook for Evolution https://bugs.launchpad.net/bugs/391623 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 391623] Re: apport hook for Evolution
** Attachment added: the Evolution apport hook http://launchpadlibrarian.net/28284831/source_evolution.py -- apport hook for Evolution https://bugs.launchpad.net/bugs/391623 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 391623] Re: apport hook for Evolution
** Attachment added: the hack to download all reports with backtraces http://launchpadlibrarian.net/28284934/evo-tester.py -- apport hook for Evolution https://bugs.launchpad.net/bugs/391623 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 391623] Re: apport hook for Evolution
** Attachment added: the CSV list of Evolution bugs http://launchpadlibrarian.net/28284840/evo-bugs-with-stacktrace.csv -- apport hook for Evolution https://bugs.launchpad.net/bugs/391623 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 391623] Re: apport hook for Evolution
** Attachment added: the apport report downloader http://launchpadlibrarian.net/28284855/apport-download.py -- apport hook for Evolution https://bugs.launchpad.net/bugs/391623 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
