Re: [Ohrrpgce] CrashRpt report summaries
With those things considered, maybe putting them directly in the bug tracker is a bad idea. I am fine with putting them on the wiki if that is easiest. The only other possibility I can think of is pushing them into a Google calc spreadsheet, but I don't know if doing that via python is hard or easy. On Mon, Aug 19, 2019, 11:04 AM Ralph Versteegen wrote: > I think it would be redundant to open one issue per report when most would > just immediately be marked as duplicate, since I can see at a glance that > they're a known bug. Nicer not to have hundreds of uninteresting closed > issues. But I guess that's just being picky, plenty of projects do manage > with spam-filled bug trackers. > > More importantly, I think a table (with sortable columns) is just the best > way to view large numbers of report summaries. For example I could quickly > see which bugs have identical or similar backtraces, whether all reports of > a bug are coming from the same person or the same game, which were reported > in a short period of time (we have some Game+Custom dual crashes), if > someone reported a bunch of bugs but only provided their email/name once, > what are the oldest and newest build for which a bug has been reported. And > that's something on which github's issue tracker falls flat compared to > bugzilla: it doesn't seem that you can sort by anything like that, and you > can only attach labels, not non-categorical data, so you can't tabulate. > > It is odd to use a wiki in this way though. Oh, I just remembered GH has a > wiki too. But it seems it doesn't have tables with sortable columns. > Another option would be something like a php/python/etc script that serves > a simple html page with a table generated from a text file, wiki page, or > the github api. > > Things I imagine might be hand assigned to each crash report in the table > (rather than in the issue tracker): > -bug number > -whether the bug is fixed > -whether the minidump has been inspected with a debugger (aside from > seeing local variables and threads, often you get a better backtrace) > -whether the debug logs have been sighted for interesting messages > More detail belongs in an issue tracke. > > But the privacy issue is a question regardless of whether we put them on a > wiki or elsewhere that's public. > > Oh, also CrashRpt has a sister project, CrashFix, which is web server > software for receiving and managing the crash reports, with a built-in > issue tracker, permissions/user management and its own utility for > generating backtraces (which is quite a major undertaking). But like > CrashRpt the original project isn't maintained, now there are a lot of > forks. Initially I thought it was too heavy for what we needed (it requires > a daemon running to process reports) and I didn't manage to get working, > since I don't know what I'm doing when it comes to servers. I didn't > anticipate spending quite this much time on my own solution. However, it > hasn't been a reinvention of the wheel, there's a lot of OHRRPGCE-specific > info in the report summaries from process_crashreports.py. > > > On Mon, 19 Aug 2019 at 21:50, James Paige wrote: > >> This crash reporting feature is great. I just read through a bunch of >> them. >> >> I am not sure what is best re putting the reports in the wiki. Is the >> idea that we want to be able to mark them resolved? Would it be better (if >> noisier) to put them in the issue tracker? >> >> >> >> On Sun, Aug 18, 2019, 10:11 PM Ralph Versteegen >> wrote: >> >>> I decided to added usernames to the listing and summary table because >>> it's the easiest way to figure out which reports are coming from the same >>> person, especially if they only give their email address or name once. And >>> the username is typically visible in a lot of log output or .rpg path >>> anyway. But we could, for example, instead generate a unique ID on each >>> computer. >>> >>> On Mon, 19 Aug 2019 at 14:03, Ralph Versteegen >>> wrote: >>> An update. http://tmc.castleparadox.com/ohr/reports20190819.txt We're currently up to 52 valid crash reports, and few empty or test ones. Here's a listing. There's better summary information now at the bottom of the file. There are a few interesting crashes, such as three identical ones inside in the Sell menu, with the comment "I was testing a game I was created, and the game crashed when I selected the sell menu. This also occurs in other games as well" Many people are hitting the MIDI loop crash, which causes quite a lot of different stacktraces. I'm still planning to create a page on the wiki with a table of the summaries where we can add additional notes and triage them, and links to the bug tracker (with detailed investigations of certain crashes). The wiki seems the easiest place for that, since we need to be able to edit. I need to write a script that can update it while preserving edits. But I don't
Re: [Ohrrpgce] CrashRpt report summaries
I think it would be redundant to open one issue per report when most would just immediately be marked as duplicate, since I can see at a glance that they're a known bug. Nicer not to have hundreds of uninteresting closed issues. But I guess that's just being picky, plenty of projects do manage with spam-filled bug trackers. More importantly, I think a table (with sortable columns) is just the best way to view large numbers of report summaries. For example I could quickly see which bugs have identical or similar backtraces, whether all reports of a bug are coming from the same person or the same game, which were reported in a short period of time (we have some Game+Custom dual crashes), if someone reported a bunch of bugs but only provided their email/name once, what are the oldest and newest build for which a bug has been reported. And that's something on which github's issue tracker falls flat compared to bugzilla: it doesn't seem that you can sort by anything like that, and you can only attach labels, not non-categorical data, so you can't tabulate. It is odd to use a wiki in this way though. Oh, I just remembered GH has a wiki too. But it seems it doesn't have tables with sortable columns. Another option would be something like a php/python/etc script that serves a simple html page with a table generated from a text file, wiki page, or the github api. Things I imagine might be hand assigned to each crash report in the table (rather than in the issue tracker): -bug number -whether the bug is fixed -whether the minidump has been inspected with a debugger (aside from seeing local variables and threads, often you get a better backtrace) -whether the debug logs have been sighted for interesting messages More detail belongs in an issue tracke. But the privacy issue is a question regardless of whether we put them on a wiki or elsewhere that's public. Oh, also CrashRpt has a sister project, CrashFix, which is web server software for receiving and managing the crash reports, with a built-in issue tracker, permissions/user management and its own utility for generating backtraces (which is quite a major undertaking). But like CrashRpt the original project isn't maintained, now there are a lot of forks. Initially I thought it was too heavy for what we needed (it requires a daemon running to process reports) and I didn't manage to get working, since I don't know what I'm doing when it comes to servers. I didn't anticipate spending quite this much time on my own solution. However, it hasn't been a reinvention of the wheel, there's a lot of OHRRPGCE-specific info in the report summaries from process_crashreports.py. On Mon, 19 Aug 2019 at 21:50, James Paige wrote: > This crash reporting feature is great. I just read through a bunch of them. > > I am not sure what is best re putting the reports in the wiki. Is the idea > that we want to be able to mark them resolved? Would it be better (if > noisier) to put them in the issue tracker? > > > > On Sun, Aug 18, 2019, 10:11 PM Ralph Versteegen > wrote: > >> I decided to added usernames to the listing and summary table because >> it's the easiest way to figure out which reports are coming from the same >> person, especially if they only give their email address or name once. And >> the username is typically visible in a lot of log output or .rpg path >> anyway. But we could, for example, instead generate a unique ID on each >> computer. >> >> On Mon, 19 Aug 2019 at 14:03, Ralph Versteegen >> wrote: >> >>> An update. >>> http://tmc.castleparadox.com/ohr/reports20190819.txt >>> >>> We're currently up to 52 valid crash reports, and few empty or test >>> ones. Here's a listing. There's better summary information now at the >>> bottom of the file. >>> There are a few interesting crashes, such as three identical ones inside >>> in the Sell menu, with the comment "I was testing a game I was created, and >>> the game crashed when I selected the sell menu. This also occurs in other >>> games as well" >>> Many people are hitting the MIDI loop crash, which causes quite a lot of >>> different stacktraces. >>> >>> I'm still planning to create a page on the wiki with a table of the >>> summaries where we can add additional notes and triage them, and links to >>> the bug tracker (with detailed investigations of certain crashes). The wiki >>> seems the easiest place for that, since we need to be able to edit. I need >>> to write a script that can update it while preserving edits. But I don't >>> want to put the more detailed information than summaries on the wiki, it >>> would be too much. And what about privacy, should I even be putting >>> usernames, .rpg filenames, partially-masked emails, which are in the >>> summary table, in public view? What about the detailed version, which may >>> include, for example, locales and filenames encountered while browsing? >>> >>> On Fri, 15 Mar 2019 at 16:39, Ralph Versteegen >>> wrote: >>> We've now collected a few crashes from
Re: [Ohrrpgce] CrashRpt report summaries
This crash reporting feature is great. I just read through a bunch of them. I am not sure what is best re putting the reports in the wiki. Is the idea that we want to be able to mark them resolved? Would it be better (if noisier) to put them in the issue tracker? On Sun, Aug 18, 2019, 10:11 PM Ralph Versteegen wrote: > I decided to added usernames to the listing and summary table because it's > the easiest way to figure out which reports are coming from the same > person, especially if they only give their email address or name once. And > the username is typically visible in a lot of log output or .rpg path > anyway. But we could, for example, instead generate a unique ID on each > computer. > > On Mon, 19 Aug 2019 at 14:03, Ralph Versteegen wrote: > >> An update. >> http://tmc.castleparadox.com/ohr/reports20190819.txt >> >> We're currently up to 52 valid crash reports, and few empty or test ones. >> Here's a listing. There's better summary information now at the bottom of >> the file. >> There are a few interesting crashes, such as three identical ones inside >> in the Sell menu, with the comment "I was testing a game I was created, and >> the game crashed when I selected the sell menu. This also occurs in other >> games as well" >> Many people are hitting the MIDI loop crash, which causes quite a lot of >> different stacktraces. >> >> I'm still planning to create a page on the wiki with a table of the >> summaries where we can add additional notes and triage them, and links to >> the bug tracker (with detailed investigations of certain crashes). The wiki >> seems the easiest place for that, since we need to be able to edit. I need >> to write a script that can update it while preserving edits. But I don't >> want to put the more detailed information than summaries on the wiki, it >> would be too much. And what about privacy, should I even be putting >> usernames, .rpg filenames, partially-masked emails, which are in the >> summary table, in public view? What about the detailed version, which may >> include, for example, locales and filenames encountered while browsing? >> >> On Fri, 15 Mar 2019 at 16:39, Ralph Versteegen >> wrote: >> >>> We've now collected a few crashes from CrashRpt. Currently showbug >>> doesn't generate a report, but I will add that too. There are also many >>> more errors that could be converted to showbug. >>> Even with just a 10 reports so far, it's a pain to go through them, so >>> I've written a tool, misc/process_crashreports.py, to analyse them. >>> >>> Generating a backtrace from the minidump was tricky -- I tried >>> Microsoft's windbg and cdb and CrashRpt's own crprober but they all >>> generated garbage, only Visual Studio's debugger (which is completely >>> separate) worked. So I used Breakpad to analyse the minidumps instead, >>> that's extra complexity but works great and is cross-platform and well >>> maintained (Breakpad is used by both Chrome and Firefox and has associated >>> client-side libraries to replace CrashRpt too not that I'm planning to) >>> >>> Unfortunately wine is still needed to generate the .sym files from the >>> .pdb file, so the process couldn't really be done on the server >>> automatically. We could easily generate the .sym files (eg during the >>> nightly build) and upload them with the pdbs, but still, if we can't run >>> git on the server, then we can't annotate the backtraces with source code. >>> But aside from that git isn't really needed, so I guess we could generate >>> and email nearly-full report summaries automatically with the crashrpt php >>> script. >>> >>> But otherwise, I'll just manually run the tool on the reports from time >>> to time. >>> >>> I've attached the generated summaries of those 10 reports. >>> There is a summary of the summaries at the bottom of the file. >>> Still not summarised enough. My summary of those summary-summaries is: >>> -one is a crash in DESCRIBE_TAG_AUTOSET_PLACES, which I know about but >>> have been lazy about fixing >>> -two are new to me, they are in MAPEDIT_APPEND_IMPORTED_TILEMAPS. I >>> found the cause of this. Hurrah! It was worth it! >>> -three are crashes while playing a MIDI file. So that still happens, >>> although I can't reproduce it anymore :( >>> -the last four are from Charbile's gfx_sdl2 testing >>> --one is inside SDL2.dll and is the crash he reported when fullscreening >>> at 1280*720 >>> --the other three are in switch_gfx_backend, but this manifests as two >>> different backtraces >>> >>> We're going to need a better way to keep track of all of these. A text >>> file with annotations is probably good enough, plus filing bug reports for >>> crashes so we have a bug number to refer to. >>> But crashrpt actually has an accompanying web server/app for collecting, >>> summarising and filing reports: http://crashfix.sourceforge.net/ >>> CrashFix is probably quite overwrought for this task though. >>> (To my surprise, it can actually parse minidumps and pdb files to in >>>
Re: [Ohrrpgce] CrashRpt report summaries
I decided to added usernames to the listing and summary table because it's the easiest way to figure out which reports are coming from the same person, especially if they only give their email address or name once. And the username is typically visible in a lot of log output or .rpg path anyway. But we could, for example, instead generate a unique ID on each computer. On Mon, 19 Aug 2019 at 14:03, Ralph Versteegen wrote: > An update. > http://tmc.castleparadox.com/ohr/reports20190819.txt > > We're currently up to 52 valid crash reports, and few empty or test ones. > Here's a listing. There's better summary information now at the bottom of > the file. > There are a few interesting crashes, such as three identical ones inside > in the Sell menu, with the comment "I was testing a game I was created, and > the game crashed when I selected the sell menu. This also occurs in other > games as well" > Many people are hitting the MIDI loop crash, which causes quite a lot of > different stacktraces. > > I'm still planning to create a page on the wiki with a table of the > summaries where we can add additional notes and triage them, and links to > the bug tracker (with detailed investigations of certain crashes). The wiki > seems the easiest place for that, since we need to be able to edit. I need > to write a script that can update it while preserving edits. But I don't > want to put the more detailed information than summaries on the wiki, it > would be too much. And what about privacy, should I even be putting > usernames, .rpg filenames, partially-masked emails, which are in the > summary table, in public view? What about the detailed version, which may > include, for example, locales and filenames encountered while browsing? > > On Fri, 15 Mar 2019 at 16:39, Ralph Versteegen wrote: > >> We've now collected a few crashes from CrashRpt. Currently showbug >> doesn't generate a report, but I will add that too. There are also many >> more errors that could be converted to showbug. >> Even with just a 10 reports so far, it's a pain to go through them, so >> I've written a tool, misc/process_crashreports.py, to analyse them. >> >> Generating a backtrace from the minidump was tricky -- I tried >> Microsoft's windbg and cdb and CrashRpt's own crprober but they all >> generated garbage, only Visual Studio's debugger (which is completely >> separate) worked. So I used Breakpad to analyse the minidumps instead, >> that's extra complexity but works great and is cross-platform and well >> maintained (Breakpad is used by both Chrome and Firefox and has associated >> client-side libraries to replace CrashRpt too not that I'm planning to) >> >> Unfortunately wine is still needed to generate the .sym files from the >> .pdb file, so the process couldn't really be done on the server >> automatically. We could easily generate the .sym files (eg during the >> nightly build) and upload them with the pdbs, but still, if we can't run >> git on the server, then we can't annotate the backtraces with source code. >> But aside from that git isn't really needed, so I guess we could generate >> and email nearly-full report summaries automatically with the crashrpt php >> script. >> >> But otherwise, I'll just manually run the tool on the reports from time >> to time. >> >> I've attached the generated summaries of those 10 reports. >> There is a summary of the summaries at the bottom of the file. >> Still not summarised enough. My summary of those summary-summaries is: >> -one is a crash in DESCRIBE_TAG_AUTOSET_PLACES, which I know about but >> have been lazy about fixing >> -two are new to me, they are in MAPEDIT_APPEND_IMPORTED_TILEMAPS. I found >> the cause of this. Hurrah! It was worth it! >> -three are crashes while playing a MIDI file. So that still happens, >> although I can't reproduce it anymore :( >> -the last four are from Charbile's gfx_sdl2 testing >> --one is inside SDL2.dll and is the crash he reported when fullscreening >> at 1280*720 >> --the other three are in switch_gfx_backend, but this manifests as two >> different backtraces >> >> We're going to need a better way to keep track of all of these. A text >> file with annotations is probably good enough, plus filing bug reports for >> crashes so we have a bug number to refer to. >> But crashrpt actually has an accompanying web server/app for collecting, >> summarising and filing reports: http://crashfix.sourceforge.net/ >> CrashFix is probably quite overwrought for this task though. >> (To my surprise, it can actually parse minidumps and pdb files to in >> order to produce stack traces, on linux. All other tools use Microsoft's >> libraries to read pdb files, including Breakpad.) >> >> ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] CrashRpt report summaries
An update. http://tmc.castleparadox.com/ohr/reports20190819.txt We're currently up to 52 valid crash reports, and few empty or test ones. Here's a listing. There's better summary information now at the bottom of the file. There are a few interesting crashes, such as three identical ones inside in the Sell menu, with the comment "I was testing a game I was created, and the game crashed when I selected the sell menu. This also occurs in other games as well" Many people are hitting the MIDI loop crash, which causes quite a lot of different stacktraces. I'm still planning to create a page on the wiki with a table of the summaries where we can add additional notes and triage them, and links to the bug tracker (with detailed investigations of certain crashes). The wiki seems the easiest place for that, since we need to be able to edit. I need to write a script that can update it while preserving edits. But I don't want to put the more detailed information than summaries on the wiki, it would be too much. And what about privacy, should I even be putting usernames, .rpg filenames, partially-masked emails, which are in the summary table, in public view? What about the detailed version, which may include, for example, locales and filenames encountered while browsing? On Fri, 15 Mar 2019 at 16:39, Ralph Versteegen wrote: > We've now collected a few crashes from CrashRpt. Currently showbug doesn't > generate a report, but I will add that too. There are also many more errors > that could be converted to showbug. > Even with just a 10 reports so far, it's a pain to go through them, so > I've written a tool, misc/process_crashreports.py, to analyse them. > > Generating a backtrace from the minidump was tricky -- I tried Microsoft's > windbg and cdb and CrashRpt's own crprober but they all generated garbage, > only Visual Studio's debugger (which is completely separate) worked. So I > used Breakpad to analyse the minidumps instead, that's extra complexity but > works great and is cross-platform and well maintained (Breakpad is used by > both Chrome and Firefox and has associated client-side libraries to replace > CrashRpt too not that I'm planning to) > > Unfortunately wine is still needed to generate the .sym files from the > .pdb file, so the process couldn't really be done on the server > automatically. We could easily generate the .sym files (eg during the > nightly build) and upload them with the pdbs, but still, if we can't run > git on the server, then we can't annotate the backtraces with source code. > But aside from that git isn't really needed, so I guess we could generate > and email nearly-full report summaries automatically with the crashrpt php > script. > > But otherwise, I'll just manually run the tool on the reports from time to > time. > > I've attached the generated summaries of those 10 reports. > There is a summary of the summaries at the bottom of the file. > Still not summarised enough. My summary of those summary-summaries is: > -one is a crash in DESCRIBE_TAG_AUTOSET_PLACES, which I know about but > have been lazy about fixing > -two are new to me, they are in MAPEDIT_APPEND_IMPORTED_TILEMAPS. I found > the cause of this. Hurrah! It was worth it! > -three are crashes while playing a MIDI file. So that still happens, > although I can't reproduce it anymore :( > -the last four are from Charbile's gfx_sdl2 testing > --one is inside SDL2.dll and is the crash he reported when fullscreening > at 1280*720 > --the other three are in switch_gfx_backend, but this manifests as two > different backtraces > > We're going to need a better way to keep track of all of these. A text > file with annotations is probably good enough, plus filing bug reports for > crashes so we have a bug number to refer to. > But crashrpt actually has an accompanying web server/app for collecting, > summarising and filing reports: http://crashfix.sourceforge.net/ CrashFix > is probably quite overwrought for this task though. > (To my surprise, it can actually parse minidumps and pdb files to in order > to produce stack traces, on linux. All other tools use Microsoft's > libraries to read pdb files, including Breakpad.) > > ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
