I'm pleased to announce the release of Beagle 0.0.12. This version of Beagle contains a number of exciting new features, including: * libbeagle, a C library for querying the beagle daemon. * Tools for building stand-alone, static indexes that can be searched by the beagle daemon. * Many improvements to the configuration system. * A much-needed revamp of the filtering system that (among other things) has allowed up to starting indexing e-mail attachments.
We have also continued to focus on fixing bugs. This is the most stable version of Beagle yet, and should be suitable for everyday use. Please note that Beagle 0.0.12 uses inotify 0.23 when available. Earlier versions of inotify will fail in mysterious ways, and will generally sap your will to live. Also note that as of this version, Beagle requires Gmime 2.1.15. For the best possible experience, you should run Beagle with Mono 1.1.8.2 or better. OUR MANY URLS ------------- To download the 0.0.12 tarball or learn more, visit the Beagle wiki at: http://www.beagle-project.org Joe Gasiorek writes a Beagle newsletter. You can read it at: http://www.beagle-project.org/Newsletter The latest gossip is available at: http://www.planetbeagle.org Nat Friedman made some cool movies that demonstrate Beagle in action: http://nat.org/demos We still talk about Beagle on the dashboard-hackers mailing list: http://mail.gnome.org/mailman/listinfo/dashboard-hackers Fireflies are not flies. They are beetles: http://iris.biosci.ohio-state.edu/projects/FFiles WHAT IS BEAGLE? --------------- Beagle is a tool for indexing and searching your data. It is in an early stage of development and should be considered experimental. Beagle is improving rapidly on many fronts, and should work well enough for everyday use. The Beagle daemon transparently monitors your data and updates the index to reflect any changes. On an inotify-enabled system, these updates happen more-or-less in real time. So for example, * Files are immediately indexed when they are created, are re-indexed when they are modified, and are dropped from the index upon deletion. * E-mails are indexed upon arrival. * IM conversations are indexed as you chat, a line at a time. Beagle uses the Lucene indexing system from the prodigious Doug Cutting. Best is a graphical tool for searching the index that the daemon creates. Best doesn't query the index directly; it passes the search terms to the daemon and the daemon sends any matches back to Best. Best then renders the results and allows you to perform useful actions on the matching objects. Indexing your data requires a fair amount of computing power, but the Beagle daemon tries to be as unobtrusive as possible. It contains a scheduler that works to prioritize tasks and control CPU usage, based on whether or not you are actively using your workstation. DEPENDENCY HECK --------------- Beagle has many dependencies, and thus can be difficult to compile. It requires: * The full Mono stack (We all use 1.1.x, and you probably should too, but 1.0.6 will also work. 1.0.5 and earlier will NOT work.) * gtk-sharp 1.0.8 (1.9.x will NOT work) * Gecko-sharp 0.6 * Gmime 2.1.15 * Libexif 0.5 or better For the best possible Beagle experience, you should also have: * Mono 1.1.8 or better * Evolution-sharp 0.6 * wv 1.0.3 * An inotify 0.23-enabled kernel * A tall, cool glass of lemonade CHANGES SINCE 0.0.11.1 ---------------------- Daemon/Infrastructure: * Add the libbeagle C API (Imendio, Joe Shaw) * Big changes to the filtering code to make it more flexible (Joe, Fredrik Hedberg) * Match multiple filters per indexable, use the highest ranking match (Fredrik) * Match filters with a combination of Uri, Extension and/or MimeType (Fredrik) * Allow FilterFactory to fall throught the filter list in the case of an error (Fredrik) * Introducted events / callbacks when indexables are filtered (Fredrik) * Mark files in the file system backend with the Filter used (Fredrik) * Re-index files when a newer filter is available (Fredrik) * Don't instantiate a new LuceneDriver for every remote indexing request (Fredrik) * Allow us to disable Lucene locking per directory (Fredrik) * Handle more failure conditions in the message passing code (Joe, Jon Trowbridge) * Add support for running without extended attributes, and speed up the sqlite fallback by using transactions (Joe, Fredrik) * Use UTC timestamps in Lucene so that we're not boxing integers unnecessarily (Joe) * Add throttling to the scheduler when many immediate priority tasks are added, like when untarring a tarball or running rm -r on a directory (Joe) * Set the default QueryDomain to both Local and System, so that per-user and the new systemwide indexes are both searched by default (Joe) * Fix a bug whereby updates would cause hits to show up multiple times by issuing a subtract event before the add event (Joe, Daniel Drake) * Friendlier inotify error messages (Daniel) * libbeagle: Try BEAGLE_HOME and check that the bus socket exists (Daniel) * Don't try and discover the file mime-type twice (Daniel) * Small fixes to libbeagle (Lukas Lipka) * Better built-in heap-buddy support (Jon) * Avoid unnecessary boxing in the message passing code (Jon) * Sqlite FileAttributeStore clean-ups (Jon) * Added the ability to change an Inotify.Watch's event mask (Jon) Backends: * Indexing of mail attachments (Joe) * Add a mail filter, giving us Maildir support, and port the Evolution mail backend to use it (Joe) * Use IndexableGenerators for the Blam and Liferea backends so that it scales and performs better (Joe) * Configuration option to ignore files by path, pattern etc. (Fredrik) * Configuration option to ignore mail folders (Fredrik) * Added StaticQueryable for searched canned indexes (Fredrik) * Disabled the LauncherQueryable in favor of the system indexing features (Fredrik) * Fix the Evolution mail backend to not reschedule a reindex if it's already in the middle of indexing itself, which would otherwise interrupt it (Joe) * Fix a bug in the addressbook backend with older (pre-2.2) versions of Evolution in which contacts wouldn't be indexed after the addressbook index was first created (Joe) * Fix handling of gaim logs with non-ascii characters represented in names (Daniel) * Update the properties in some backends to the be correct (Lukas) * Explicitly disable filtering on gaim log indexables, since the stream we produce is already filtered (Jon) * To ensure that beagle-index-info returns correct information, get the item count from the IIndexer, not the driver we are using for querying (Jon) * Fix a problem where we were losing mails from the index if an IMAP summary was updated later (Joe) Filters: * Added .desktop filter (Fredrik) * Added .directory filter (Fredrik) * Added Monodoc filter (Fredrik) * Added Docbook filter (Fredrik) * Filter text/x-chdr (C header files) using the C source code filter (Joe) * Image filter abstraction. Created a FilterImage which all the image filters use. This lets us index F-Spot metadata information (Lukas) * Added F-Spot metadata support which we now index (Lukas) UI/Tools: * Added beagle-build-index for building static indexes (Fredrik) * Added beagle-manage-index for low-level Lucene index management (Fredrik) * Configuration UI (Fredrik) * Fix a problem with Best whereby the window wasn't being placed in the same place when F12 was pressed (Joe) * Add a "Reply" action back to the mail tile, as it's available in Evo 2.2 (Joe) * Don't show "stop words" in the snippet highlights (Joe) * Fix ImLogViewer to work with gaim's character escaping style (Daniel) * Remember selected log in the IM timeline tree, improve searching, other random optimizations to the IM viewer (Lukas) * Galago support and integration. Added Galago support to the Mail, IM log and Contact tile (Lukas) * Fixed a crasher in TileMailMessage.cs and some random cleanups. Fix a bug when the tile didnt show the subject when 'Who' was null. This was related to the email attachments (Lukas) * Added an author field to the blog template (Lukas) * Render mail attachments nicely (Lukas) Web Services: * Support for cascaded Beagle Networking (KN Vijay) * Configuration of Beagle Web-services and Beagle networking via 'beagle-config', with corresponding handlers for real-time update of configuration changes (Vijay) * Added on-demand Prefetching and Caching of snippets for network hits (Vijay) * Updates to Web interface wrt handling Actions for external accesses (Vijay) Everything Else: * Support either sqlite2 or sqlite3 (Daniel) * Build fixes (Jon) * Newsletter (Joe Gasiorek) * Added Polish translation (Michal Kastelik) * Added Turkish translation (Enver Altin) * Updated Canadian English translation (Adam Weinberger) * Updated Chinese translation (Chao-Hsiung Liao) * Updated German translation (Hendrik Brandt) * Updated Hungarian translation (Gabor Kelemen) * Updated Norwegian translation (Terance Sola) * Updated Spanish translation (Francisco Javier F. Serrador) * Updated Swedish translation (Fredrik) * All the stuff I forgot (All of the people I forgot) KNOWN ISSUES ------------ It doesn't take that much ingenuity to confuse the file system backend. In particular, the right thing doesn't always happen if a file's name changes very rapidly. (i.e. "mv foo bar; mv bar baz; mv baz foo") The amount of memory consumed by the beagle daemon tends to grows over time. All of the major leaks have been fixed, but it might need to be manually killed and restarted after running for long periods of time. Due to a bug in mono, the beagle daemon will occasionally lock up when starting the helper process. As far as we can tell, this is pretty rare and only happens on SMP systems. There are a few other relatively rare deadlocks that also seem to be mono bugs. Occasionally beagled will deadlock on start-up. Sometimes a query will throw an exception and cause the beagled to deadlock until the querying process is killed. These are hard to trigger, but have been observed on non-SMP systems. On non-inotify systems, sometimes beagle will get confused and repeatedly re-index the same directory. At this point in development, we cannot commit to stable APIs or file formats. You will almost certainly need to delete your indexes and start again at some point in the future. _______________________________________________ Dashboard-hackers mailing list [email protected] http://mail.gnome.org/mailman/listinfo/dashboard-hackers
