On 17 8 2003 at 2:51 pm -0400, Ben Kennedy wrote: >In any case, after several more hours of work, I am beginning to have >some luck deriving the database format! Or at least, so far, enough it >seems to be able to correlate headers and bodies. > >I will keep the list updated on my progress.
All, Losing my mail database has turned around from being a major disaster to a rewarding exercise in forensics and programming. :-D I am pleased to announce that after about 5 days of work, I have a working GUI-based application that will take a PowerMail database (optionally with severe damage), scan it, produce a navigable summary of all messages that it has found (and their condition), and then allow you to extract the ones of your choosing into a PowerMail Exchange file which can then be re-imported into a good database. Using this on my damaged database that PowerMail was unable to fix (simply reporting "DB is corrupt" and stopping cold), I have been able to recover 14566 of about 15200 messages. I also know with ease that there were 8 damaged areas in the file totalling about 186 KB of garbage. Even more fortunately, I was able to recover ALL (I believe) of the new messages since my most recent backup! Producing this application has been a result of several phases: 1. Examining a known-good Message Database file in a hex editor and making pages and pages of notes and hypotheses, repeating this for quite awhile with different messages until I was able to make conclusions about the various record formats. In essence, I have clean-room derived a good portion of the PM database structure. 2. Examining a PowerMail Exchange export of the same known-good database, and comparing with notes and results derived above, to essentially derive the PM Exchange file format. 4. Writing code to automate the parsing job at each stage of analysis, to validate the structure of the PM Database file based on acquired knowledge, then using the resultant output to further refine the analysis. 5. Bundling all of the above into a Cocoa-based GUI application to allow easy loading, viewing and exporting of an arbitrarily damaged PM database file. To make the application truly usable to the masses will require some further refinement of the user interface and so on. Furthermore, with additional technical information from CTM, I will be able to solidify and correct my assumptions/conclusions about the data and attributes and how they are stored, so that the app's proper and accurate functioning will be theoretically sound in addition to merely being empirically sound. Thus, I will be contacting CTM privately to share my results and get their feedback before making this application publically available, and will hopefully have more to offer everyone next week. In the mean time, if anyone else should happen to suffer major database damage due to filesystem corruption or the like, you can now relax with some confidence that all is not lost. On a personal note, I should mention that this has been hugely rewarding just from a research/analysis/development point of view. Furthermore, it gives me even greater confidence in using PowerMail as my mail client of choice, since I am now more familiar with its inner workings. (e.g. if PM mysteriously stops working on December 31st and CTM vanishes off the earth without a trace, I'll still be okay!) -ben -- Ben Kennedy, chief magician zygoat creative technical services 613-228-3392 | 1-866-466-4628 http://www.zygoat.ca
