Re: [cross-project-issues-dev] Automated Error Reporting for Mars Milestone Builds
Hi Martin, Am 31.08.2014 um 22:02 schrieb Oberhuber, Martin martin.oberhu...@windriver.com: Hi Marcel, I would be interested in using the tool internally at WindRiver. Could you share what is needed to set up an internal server for ourselves, and how to replace the server IP in the code before we deploy it internally ? Two changes are needed. 1. The server url is stored as preference. This needs to be adjusted. See /org.eclipse.recommenders.stacktraces.rcp/src/org/eclipse/recommenders/internal/stacktraces/rcp/PreferenceInitializer.java 2. The plugin id or at least one StackFrameElement has to start with org.eclipse (or com.codetrails. for testing at the moment). See org.eclipse.recommenders.internal.stacktraces.rcp.LogListener.startsWithRecommendersOrCodetrails(String) for details. The server-side can be any script that takes the json and simply dumps it to a (nosql) database. That’s basically what it is right now. Also what tooling / methods you used to mine the logs … There is neither a special tooling nor mining yet. The error analysis happens manually (as outlined in the slides). If there is a larger adoption of the system, I’d gonna take it to the next level and automated a few steps from what I’ve learned from the data received until then. I guess now that you've identified and filed 10-15 known bugs it would be interesting if the tooling could already identify / classify the known bugs at the client side (and no longer send duplicates). There is a feature request pending that allows to specify ignore lists for errors. Said feature would allow to disable sending error reports completely or partially based on plugin-ids, error codes, messages, or stacktrace hashes. This is yet not implemented and had a slightly different intention, but get’s close to what you are looking for. See https://bugs.eclipse.org/bugs/show_bug.cgi?id=442132 for details. As a user, I'd love to see an Advanced Errorlog in Eclipse that not only shows me the original errors + stacktraces but also the tool's assessment (known issue - bug_id, and severity). Known issues that have been identified as harmless could be shown green, unknown issues red. Interesting idea. Could you add a request to the Stacktraces Bugzilla for tracking? Here, however, I’ve to make my intension and goals clear. My goal is to spot and report issues in the milestone builds and study whether such a system can actually create added value for Eclipse. Although I like your idea, I likely won’t have the time to implement it. But if others would contribute such a feature, I’d be happy. And ideally the tool would automatically download latest assessment data from the server before uploading new data … There is a request similar to yours in Bugzilla already. See https://bugs.eclipse.org/bugs/show_bug.cgi?id=442719. What do you think ? Great input, great ideas. If you’d like to participate in the project or support it’s development in other ways, let me know... Best, Marcel Am 31.08.2014 um 22:02 schrieb Oberhuber, Martin martin.oberhu...@windriver.com: Hi Marcel, I would be interested in using the tool internally at WindRiver. Could you share what is needed to set up an internal server for ourselves, and how to replace the server IP in the code before we deploy it internally ? Also what tooling / methods you used to mine the logs ... I guess now that you've identified and filed 10-15 known bugs it would be interesting if the tooling could already identify / classify the known bugs at the client side (and no longer send duplicates). As a user, I'd love to see an Advanced Errorlog in Eclipse that not only shows me the original errors + stacktraces but also the tool's assessment (known issue - bug_id, and severity). Known issues that have been identified as harmless could be shown green, unknown issues red. And ideally the tool would automatically download latest assessment data from the server before uploading new data ... What do you think ? Thanks, Martin -- Martin Oberhuber, SMTS / Product Owner - Development Tools, Wind River direct +43.662.457915.85 fax +43.662.457915.6 -Original Message- From: cross-project-issues-dev-boun...@eclipse.org [mailto:cross-project-issues-dev-boun...@eclipse.org] On Behalf Of Marcel Bruch Sent: Saturday, August 30, 2014 11:21 AM To: Cross project issues; Recommenders developer discussions; Discussions about the IDE Subject: Re: [cross-project-issues-dev] Automated Error Reporting for Mars Milestone Builds Hi, we've just released an update of the automated error reporting tool [1] and I'd like to share a few numbers and experiences I've made in the past 1 1/2 weeks. In the meanwhile we received ~1400 error events from ~30 users [4]. Many events describe the same problem and of course most errors are not bugs. From these 1400 events roughly half of them are caused by the same class
Re: [cross-project-issues-dev] Automated Error Reporting for Mars Milestone Builds
Hi, we’ve just released an update of the automated error reporting tool“ [1] and I’d like to share a few numbers and experiences I've made in the past 1 1/2 weeks. In the meanwhile we received ~1400 error events from ~30 users [4]. Many events describe the same problem and of course most errors are not bugs. From these 1400 events roughly half of them are caused by the same class of issues. Based on the analysis of these error events, 10-15 bugs have been reported to various projects in and outside of Eclipse (PDE, Platform, JDT, Recommenders, EclEmma, Codetrails). Some of these have been fixed. Others didn’t peek interest of the project committers yet. The reasons may be manifold, but I haven’t heard back of them, and thus, can’t tell. If you have suggestions how to improve error reporting, please open a bug at [2] and let us know. See [5] for a list of bugs we’ve been working on and what else is currently the todo list – or at least in discussion. The source code is available at [7]. To get going quickly, I’d recommend to use Oomph as described in [3]. I’ve assembled a few slides describing the current state of affairs of the UI, which data is collected etc. See [6] for details. If you have questions or if you have some input I/we should be interested in, please let me know. Marcel [1] http://download.eclipse.org/recommenders/updates/milestones/ [2] https://bugs.eclipse.org/bugs/enter_bug.cgi?product=Recommenders.Incubatorcomponent=Stacktracesversion=0.1.0 [3] http://git.eclipse.org/c/recommenders/org.eclipse.recommenders.git/about/ [4] http://recommenders.eclipse.org/dashboard/index.html#/dashboard/file/stacktraces.json [5] http://eclip.se/2r [6] https://docs.google.com/a/codetrails.com/file/d/0B9Gf8BD0WA1SczNhZE43MnJ4Qzg/ [7] http://git.eclipse.org/c/recommenders/org.eclipse.recommenders.git/tree/plugins/org.eclipse.recommenders.stacktraces.rcp Am 19.08.2014 um 12:38 schrieb Marcel Bruch marcel.br...@codetrails.com: Anfang der weitergeleiteten Nachricht: Von: Marcel Bruch marcel.br...@codetrails.com Betreff: Aw: [cross-project-issues-dev] Automated Error Reporting for Mars Milestone Builds Datum: 19. August 2014 12:34:39 MESZ An: Cross project issues cross-project-issues-dev@eclipse.org Hi, I noticed a few new users are sending errors. In response, I decoupled the stacktraces error reporting plugin from the Code Recommenders. Now you should be able to install it without pulling in the complete Code Recommenders stack. Some notes for testers: • I published the feature as „Code Recommenders Stacktraces Error Reporting Utility“ on Code Recommenders’ milestones update site [1]. • Bugs and feature requests are collected and tracked in Bugzilla [2]. Please make use of it for every request. • There are already a (small) amount of people contributing. Feel free to contribute as well. Please read [3] on how to setup the project in Eclipse with Oomph. • All stacktraces are sent *without any anonymization yet* (even if the UI suggests that at the moment) to eclipse.org and can be publicly reviewed at [4]. • The plugin can be disabled temporarily by unselecting it in Preferences » General » Startup and Shutdown (see Screenshot below) and restarting your IDE Some general notes: • Feature updates happen frequently and without prior notice. Please use the update manager to automatically look for newer versions from time to time. • The code is yet not considered production-ready. It’s rather a quick-’n-dirty prototype to gather community feedback. Not all features you see today in the UI are already implemented (properly) • A releasable version is planned for Mars M2 timeframe and after we got Legal approval by the EF. • The data in the dashboard [4] may be dropped due to schema changes. We’ll keep this stable from Mars M2 but for now please consider the data sent as experimental and temporary data. Thanks to all testers and feedback, Marcel [1] http://download.eclipse.org/recommenders/updates/milestones/ [2] https://bugs.eclipse.org/bugs/enter_bug.cgi?product=Recommenders.Incubatorcomponent=Stacktracesversion=0.1.0 [3] http://git.eclipse.org/c/recommenders/org.eclipse.recommenders.git/about/ [4] http://recommenders.eclipse.org/dashboard/index.html#/dashboard/file/stacktraces.json wizard01.pngwizard02.pngstacktraces-disable.png ___ cross-project-issues-dev mailing list cross-project-issues-dev@eclipse.org To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
Re: [cross-project-issues-dev] Automated Error Reporting for Mars Milestone Builds
Hi Lars, Am 18.08.2014 um 11:13 schrieb Lars Vogel lars.vo...@gmail.com: Hi Marcel, I assume the project is based on the logger GSoC project. Can you point to the documentation and source for this remote logger plug-in. the eclipse error reporting prototype is hosted in recommenders main git repository at the moment. See [1] for instructions how to setup the project with Oomph. there is no additional documentation atm. Is it possible to use and configure it with an alternative backend server? Yes. the server url is stored as preference and may be changed by, e.g., plugin_customization.ini Best, Marcel [1] http://git.eclipse.org/c/recommenders/org.eclipse.recommenders.git/about/ Best regards, Lars 2014-08-10 12:53 GMT+02:00 Marcel Bruch marcel.br...@codetrails.com: Hi cross-projects-issues, the Mars release train is picking up speed again and I’d like to propose adding an error log reporting tool to the milestone builds that reports errors to a central logging service at eclipse.org. What’s cross-project’s take on this? Do you see a need and would you agree to add this to the milestone builds? The intension is obvious I think. Errors happen (especially during the milestone builds). The sad thing is that it usually takes weeks until a user recognizes them as such and reports them to Bugzilla – if ever. To help us to get notified about issues in the current code base and fixing them as quickly as possible, I’d like to add a log listener to the platform log which automatically sends error log entries to a server at eclipse.org. The proposed user-workflow is pretty simple: Whenever an error is logged to the Eclipse error log, that service checks whether it’s an error logged by an Eclipse plugin (by checking the plugin id starts with „org.eclipse“). If so, it opens a dialog asking the user for permission to send that error to our server (recommenders.eclipse.org) for further processing. For a brief summary of the UI, please see the wizard screenshot below. On the server side, the error is logged and we (ATM me) are notified by email that there is something new to analyze. There is also a dashboard [1] that allows committers to review which errors have been logged in the past, say, 7 days and to analyze the submitted errors. If of interest, that system can be extended to send errors by email to selected committers based on some search criteria like containing „org.eclipse.your-project“ in a stacktrace etc. For details about the submitted data, please study one of the example error log entries in [1]. It contains all data that would be sent. To pick up the question from the beginning: What’s cross-project’s take on this? Should such a feature be added to the milestone builds? I sure that EF needs to give their permission, but the general question is whether we’d like to know (and get notified in such a way) about errors our users experience. From my experience only few people discuss on cross-projects. But maybe you can cast your voice in a mini-survey [2]? Marcel [1] http://recommenders.eclipse.org/dashboard/index.html#/dashboard/file/stacktraces.json [2] https://docs.google.com/forms/d/1BLbeAtu-pvz26aXgfP2v4Sm3ysPjkUknkxEJsj909Lk/viewform error-reporting-wizard.pngBildschirmfoto 2014-08-10 um 12.24.13.png ___ cross-project-issues-dev mailing list cross-project-issues-dev@eclipse.org To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev ___ cross-project-issues-dev mailing list cross-project-issues-dev@eclipse.org To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev ___ cross-project-issues-dev mailing list cross-project-issues-dev@eclipse.org To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
Re: [cross-project-issues-dev] Automated Error Reporting for Mars Milestone Builds
Thanks Ivan. I added the obfuscate option to the list of potential improvements. Am 11.08.2014 um 10:05 schrieb Ivan Inozemtsev ivan.inozemt...@xored.com: +1, I'd love to have this feature not only for milestone builds. Several years ago at Xored we were trying to build a similar system for server-side error log collection and grouping, with additional collection of extra information like jre version, os name, memory settings, disk usage, but couldn't find enough resources to make it live. Additional note – sometimes, even during email communication, people don't send error logs because they don't want to reveal their proprietary bundle IDs (so they don't want you to know their company/product name), or send logs with those ids are replaced by asterisks. So maybe having an option like obfuscate non-eclipse bundles and class names could potentially attract such users. Ivan Inozemtsev ___ cross-project-issues-dev mailing list cross-project-issues-dev@eclipse.org To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
Re: [cross-project-issues-dev] Automated Error Reporting for Mars Milestone Builds
Just a brief summary of the current poll results [1] +1: 7 people voted for adding this feature. ±0: 0 people had no opinion -1: 0 people voted for not adding this feature Not too many, but it’s summer… If anyone still likes to cast his/her voice, please visit [2]. Wayne, Mike, I assume adding such a feature to any EPP package needs permission of the EMO. What would be required from your side? Do you need any number of votes for or against that proposal? Marcel [1] https://docs.google.com/spreadsheets/d/16JOCI1Z_cQK89eCMbPBxLp-r8kuj_oFAULCvyUZ8FIQ/edit?usp=sharing [2] https://docs.google.com/forms/d/1BLbeAtu-pvz26aXgfP2v4Sm3ysPjkUknkxEJsj909Lk/viewform ___ cross-project-issues-dev mailing list cross-project-issues-dev@eclipse.org To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
