Re: [Zope] substring search on zcatalog textindex
On Thu, 10 Aug 2000, Chris Withers wrote: Casey Duncan wrote: TextIndexes index individual words separately (using a vocabulary object to identify each word in the catalog). All non-alphanumeric characters (such as punctuation) are dropped so that excludes searching for "?" or "*" or any other non-alphanumerics using TextIndexes. IIRC, you can use another Vocabulary that wouldn't necessarilly behave like that. I wonder if this works yet? If it does, my initial question still remains ;-) In fact, to use substring matching on a text index, you have to set the Vocabulary to support it *when you first create the ZCatalog*. Which seems a little bit broken to me, since unless I missed something you have to choose not to add a vocabulary when you first create the ZCatalog, and then go add one with substring checked from the management screens afterwards. In the code the difference between the checkbox checked and not checked is two different lexicon implementations, not just the setting of a flag. So you can't just change the flag, you have to see to it that the whole lexicon gets rebuilt. I haven't checked into how you do that yet wry grin. As for your question about different vocabularies, punctuation, and globbing support, all of that happens at the lexicon level. That is, the lexicon implements the breaking up of strings into indexable words (calling the splitter) and it also implements the expansion of wildcards into matches. It gets to do that *before* the text index machinery parses the query, so it can decide on the syntax rules, as far as I can see. So if you want a Vocabulary that supports punctuation and globbing, you'll have to write one, and then *you* get to decide what the syntax is to handle the case you ask about grin. --RDM ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] substring search on zcatalog textindex
This is illuminating. I have a question maybe you (or somebody else) could answer: Searching TextIndexes you can use "and", "or" or "andnot" as query criteria. I also see support in the source code for near searches using "..." in the query string. I have not been able to get this to work (although the first three work great for me), and would like to. Am I missing something? -Original Message- From: R. David Murray [mailto:[EMAIL PROTECTED]] Sent: Monday, August 14, 2000 12:59 PM To: Chris Withers Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: [Zope] substring search on zcatalog textindex On Thu, 10 Aug 2000, Chris Withers wrote: Casey Duncan wrote: TextIndexes index individual words separately (using a vocabulary object to identify each word in the catalog). All non-alphanumeric characters (such as punctuation) are dropped so that excludes searching for "?" or "*" or any other non-alphanumerics using TextIndexes. IIRC, you can use another Vocabulary that wouldn't necessarilly behave like that. I wonder if this works yet? If it does, my initial question still remains ;-) In fact, to use substring matching on a text index, you have to set the Vocabulary to support it *when you first create the ZCatalog*. Which seems a little bit broken to me, since unless I missed something you have to choose not to add a vocabulary when you first create the ZCatalog, and then go add one with substring checked from the management screens afterwards. In the code the difference between the checkbox checked and not checked is two different lexicon implementations, not just the setting of a flag. So you can't just change the flag, you have to see to it that the whole lexicon gets rebuilt. I haven't checked into how you do that yet wry grin. As for your question about different vocabularies, punctuation, and globbing support, all of that happens at the lexicon level. That is, the lexicon implements the breaking up of strings into indexable words (calling the splitter) and it also implements the expansion of wildcards into matches. It gets to do that *before* the text index machinery parses the query, so it can decide on the syntax rules, as far as I can see. So if you want a Vocabulary that supports punctuation and globbing, you'll have to write one, and then *you* get to decide what the syntax is to handle the case you ask about grin. --RDM ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] substring search on zcatalog textindex
So the question is, does anyone know of a simple way to get the zcatalog to also find substring matches on a textindex? -- Peter Armstrong For Text Indexes you can use wildcards like * and ? in searches. So that searching for Foo* would find Foo, FooBar, fool, etc. I'm not sure if this works for Field or Keyword indexes though, I haven't tried it. My though is it only works for text indexes which should help you anyway. Good Luck, Casey Duncan Well, this sounded like it would do just fine, however, it doesn't seem to work. :( I'm using basically a zsearchinterface, and if I enter something like inter*, it comes up and says there was no matches, despite being able to find plenty of matches if I enter internet. Also, the and and notand don't seem to work either. Also, entering internet* also turns up nothing. I didn't setup Zope here, so I'm wondering if something could have been missed out then that could cause these things to not work? Or perhaps a version too old? Or could it be a problem with my dtml not talking to the Zcatalog in the correct manner? TIA Peter Armstrong ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] substring search on zcatalog textindex
I didn't setup Zope here, so I'm wondering if something could have been missed out then that could cause these things to not work? Or perhaps a version too old? Or could it be a problem with my dtml not talking to the Zcatalog in the correct manner? Ok, I typed to soon. Seems it was the version of Zope that didn't like using wild cards. A newer version is being prepared and wild cards work fine with it. Sorry for the trouble Peter Armstrong ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] substring search on zcatalog textindex
On Mon, 14 Aug 2000, Casey Duncan wrote: Searching TextIndexes you can use "and", "or" or "andnot" as query criteria. I also see support in the source code for near searches using "..." in the query string. I have not been able to get this to work (although the first three work great for me), and would like to. I haven't tried to get near searches to work yet; haven't had the need. I'm just reading the code grin. I think Dieter looked at this in 2.1.x and found some bugs, and I don't know if his fixes got into 2.2; I don't think he's had opportunity to test 2.2 yet. If you scan the zope.nipltd.com archives for '...' you'll probably find the relevant messages. Or 'zcatalog and near' if '...' isn't a valid search string. --RDM ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] substring search on zcatalog textindex
TextIndexes index individual words separately (using a vocabulary object to
identify each word in the catalog). All non-alphanumeric characters (such as
punctuation) are dropped so that excludes searching for "?" or "*" or any
other non-alphanumerics using TextIndexes.
Not much of this is formally documented, but you can review the source (in
Python and C) for the indexing mechanism in:
{YourZopeDir}/lib/python/SearchIndex
-Original Message-
From: Chris Withers [mailto:[EMAIL PROTECTED]]
Sent: Thursday, August 10, 2000 2:37 AM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: Re: [Zope] substring search on zcatalog textindex
Casey Duncan wrote:
So the question is, does anyone know of a simple way to get the zcatalog
to
also find substring matches on a textindex?
--
Peter Armstrong
For Text Indexes you can use wildcards like * and ? in searches. So that
searching for Foo* would find Foo, FooBar, fool, etc. I'm not sure if this
works for Field or Keyword indexes though, I haven't tried it. My though
is
it only works for text indexes which should help you anyway.
What do you do if you actually want to search for a '*' or a '?' ?
cheers,
Chris
___
Zope maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] substring search on zcatalog textindex
If I remember correctly, partial searching was implemented against text indexes. I have no idea how to use it, however, as AFAIK it was never documented. Maybe someone else can help? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Wednesday, August 09, 2000 3:29 AM To: [EMAIL PROTECTED] Subject: [Zope] substring search on zcatalog textindex Hi people, Got a slight problem. I've got a zcatalog with a heap of names in it, and I want to be able to search for and find a name, given I only know part of the name. I did have it going through the entire catalog and by using the substring function, had it only display a result if there was a match, otherwise it didn't display anything. This was ok until the catalog grew bigger. As it is displaying the results in batches of 20, if there is no match until say the 300th element in the index, then it will display a page with only the "previous 20 results" and "next 20 results" links, and nothing inbetween them, and by following the "next" link, you have to go through 15 such pages before you get to the actual matches. So the question is, does anyone know of a simple way to get the zcatalog to also find substring matches on a textindex? -- Peter Armstrong ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] substring search on zcatalog textindex
So the question is, does anyone know of a simple way to get the zcatalog to also find substring matches on a textindex? -- Peter Armstrong For Text Indexes you can use wildcards like * and ? in searches. So that searching for Foo* would find Foo, FooBar, fool, etc. I'm not sure if this works for Field or Keyword indexes though, I haven't tried it. My though is it only works for text indexes which should help you anyway. Good Luck, Casey Duncan ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
