Joe already leaked the news of the changes in the the handling of snippets. So a brief mail about the changes is in order. For the impatient, there were changes in how the snippets were generated, how to request a snippet and how the response snippet look like. Backward compatibility has been ensured by providing wrapper methods/properties and so all the existing C#/C/python clients should work as they are.
Beagle API allows the clients to request a snippet for a particular result of some query. In response, the clients used to get an HTML string representing some occurrences of the query terms in the data; the occurrences were marked as bold and in different colours (using HTML tags). The snippet being in HTML, it could be display directly if the underlying widget understood HTML or modified to display it correctly. It was easy to get back the snippet but hard to manipulate it since it was all pre-formatted with HTML. I made two major changes: 1) Added an option "FullText" to the snippet-request to get back the _entire_ (cached) text for that result. Just like google page-cache. The entire text is currently _not_ marked with the occurrences of the query terms for performance reasons. Any client can mark the occurrences roughly by scanning the text and marking the words which match any of the query.stemmed_text words. 2) The snippet is not returned as a single HTML string. Instead it a list of SnippetLine. Each snippetline denotes a snippet containing one or more matches. The snippetline stores the line number (so that supported applications can scroll the file to the right places) and a list of fragments giving a continuous sequence of text containing one or more matches. The fragments can be concatenated together to get the matching fragment. Each fragment stores a text and a QueryTermIndex index. If the fragment is part of the sentence before, after or between the matches, it has QueryTermIndex=1; if the fragment is match, it has QueryTermIndex = the index of the matched stemmed_text in query.stemmed_text array. If there are multiple but far apart matches in a single line, there could be more than one snippetline. It is confusing but contains all the information about snippets that any client can possibly need. And more important, no formatting. It is also easy to join the fragments and snippetlines with any kind of formatting to create the complete snippet. For e.g. if the query was "good abcd 1234", and there was line 17 looked like: "abcd is a good desktop search application but it cannot return results for queries like 1234 which is pretty lame", then this would produce this: snippets = [snippetline1, snippetline2] snippetline1 = (line=17, [fragment1, fragment2, fragment3, fragment4]) fragment1=(index=1 , text="abcd") fragment2=(index=-1, text=" is a ") fragment3=(index=0, text="good") fragment4=(index=-1, text=" desktop search application") snippetline2 = (line=17, [fragment5, fragment6, fragment7]) fragment5=(index=-1, text="for queries like ") fragment6=(index=2, text="1234") fragment7=(index=-1, text=" which is pretty") If FullText is set, a snigle snippetline is returned with line=1 and a single fragment containing the entire text. Using the earlier SnippetResponse.Snippet/ beagle_snippet_response_get_snippet() methods return an HTML coloured string just like before: "<font XXX><b>abcd</b></font> is a <font XXX><b>good</b></font> desktop search application ... for queries like <font XXX><b>1234</b></font> which is pretty..." Not done yet: - The C# BeagleClient API allows clients to access the detailed structure of the snippet i.e. access to the snippetlines and fragments. But the libbeagle/python bindings weren't updated to open up the detailed structure in the API. So, as of now libbeagle/pybeagle clients can get only the coloured HTML string as snippet. - beagle-search still contains the hack where it changes the HTML snippet directly to change its formatting. - Somehow integrate "Show cached text" into the UI. Any patch is welcome. - dBera PS1: There was another change where the snippets are now generated on the fly from the textcache file right before being shipped to the client. PS2: TextCache, the major source of the snippets, is also undergoing a lot of changes. More on that later. -- ----------------------------------------------------- Debajyoti Bera @ http://dtecht.blogspot.com beagle / KDE fan Mandriva / Inspiron-1100 user _______________________________________________ Dashboard-hackers mailing list [email protected] http://mail.gnome.org/mailman/listinfo/dashboard-hackers
