Re: [Zope-dev] Catalog improvements
On Tuesday 20 November 2001 05:35 pm, Wolfram Kerber allegedly wrote: Hi, i'm currently working on a product that allows to attach relational information to zope-objects. It works quite well so far, but to further enhance it i need to make some changes to the Catalog. I could perhaps implement it as a separate product, but i strongly feel that those changes are best applied to the Catalog itself, as they are of general use (i think) and involve a lot of changes to the inner workings of the Catalog. In particular i need the following: - named/stored queries these are precompiled queries, so they can be executed without parsing and are easily cacheable i.e. similar to what is implemented in CMFTopic, but stored in the Catalog and a bit smarter - caching support - unions and intersections sub-queries (i.e. queries that are directed at a certain index) should be more flexibly combineable I have some code that implements this in my CatalogQuery product. It creates a query object from a string. Presently these are not persistent, but they could easily be made to be to create precompiled queries. code at: http://www.zope.org/Members/Kaivo/CatalogQuery I searched this mailing-list as well as zope.org to get an idea about what has already been discussed and requested, and there seems to be some interest in improving the Catalog. Some people even seem to have worked on this, perhaps they could give an update on this? Possibly i don't have to write everything from scratch... I would be willing to help both in coding and getting the code put into the Zope core. I would have put this into a proposal, but there already are two proposals that deal with the features i want, one is dedicated to unions/intersections, the other (TopicIndexes) to performance issues (i dont't know what's the status of these though, especially the first one is rather old), and i don't want to hijack them without asking. As so often i will need to complete my current project first, but would then like to help in improving the Catalog for a more general use. Possibly we need to rekindle discussion. I would suggest contacting the authors of those proposals to see how compatible your concepts are wth theirs. Perhaps a new proposal should be drafted with the new ideas and ty them back to the previous ones. If there is redundancy, that can be worked out. So, if there is interest, i would propose to collect some ideas and comments about how a better Catalog should look like, how it could be best implemented and how to organize this effort (with respect to the already existing proposals). I am very interested in such a discussion. Let me know what I can do to help. /---\ Casey Duncan, Sr. Web Developer National Legal Aid and Defender Association [EMAIL PROTECTED] \---/ ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Catalog improvements
Casey Duncan wrote: I have some code that implements this in my CatalogQuery product. It creates a query object from a string. Presently these are not persistent, but they could easily be made to be to create precompiled queries. code at: http://www.zope.org/Members/Kaivo/CatalogQuery Casey, did you get a chance to look at my patches for adding an extended uniqueValues method to CatalogQuery? I would be willing to help both in coding and getting the code put into the Zope core. raises hand me too! So, if there is interest, i would propose to collect some ideas and comments about how a better Catalog should look like, how it could be best implemented and how to organize this effort (with respect to the already existing proposals). I am very interested in such a discussion. Let me know what I can do to help. I'm interested in this too, and I'm keen to get a solution that will work with just the ZODB, without needing all of Zope. -- Steve Alexander Software Engineer Cat-Box limited ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Catalog improvements
Is this code available for public ? Andreas - Original Message - From: Matt Hamilton [EMAIL PROTECTED] To: Casey Duncan [EMAIL PROTECTED] Cc: Steve Alexander [EMAIL PROTECTED]; Wolfram Kerber [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Tuesday, November 27, 2001 10:06 Subject: Re: [Zope-dev] Catalog improvements On Tue, 27 Nov 2001, Casey Duncan wrote: I'm interested in this too, and I'm keen to get a solution that will work with just the ZODB, without needing all of Zope. Yes, I second, third and forth that motion. I have a bunch of ideas kicking around for ZODB-level indexing. Let's talk more. Perhaps we should arrange an indexing and catalog chat on #zope. I would like in on that too :) About a year or so ago I was working on a full-text indexing system for indexing several gigabytes of text (mailing list archives). Most of it was written in C and uses quite a lot of cool algorithms from various information retrieval papers and books. I have been hoping to have the time to take parts of it and work it into the new PluginIndex architecture. The existing code uses BerkeleyDB files to hold the index structures, but I would like to use ZODB instead to give it a bit more modularity. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Catalog improvements
On Tue, 27 Nov 2001, Andreas Jung wrote: Is this code available for public ? Sort of :) It used to be around, but the server with it on is currently offline and in need of a new disk controller, so it is not to hand. It is also poorly commented :( and written in very highly optimised (read: illegible) C. The main bits needed from it are the routines to store an retrieve compressed lists of ascending integers (ie. used in indexes). I want to write a python wrapper around them and release a list-like python data structure that will allow efficient storage of indexes. The other bit is the code for doing the cosine ranking similarity comparison in order to rank the documents in order of relevance to a query. Most of the code is taken from the book/code 'Managing Gigabytes' by Witten, Moffat Bell (http://www.cs.mu.OZ.AU/mg/) The code is quite old now (1999) and designed for quite large systems, or reletively static text (ie. doesn't do incremental indexing very well). I worked on developing a 'forward' index which could be easily updated, and then inverted quite quickly on a regular basis (since it didn't need to parse the source text again). -Matt -- Matt Hamilton [EMAIL PROTECTED] Netsight Internet Solutions, Ltd. Business Vision on the Internet http://www.netsight.co.uk +44 (0)117 9090901 Web Hosting | Web Design | Domain Names | Co-location | DB Integration ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Resize a image stored in a Oracle's BLOB Field
I'm not really sure when this was asked ( I dont think it was '99) :)
The img> tag is in the containing document, not the actual method which returns the image... Somewhere else you have an img src="myZopeUrlWhichReturnsAnImage"> which is where you slap the height and width tags.
On Friday, November 12, 1999, at 07:40 AM, Juan Javier Carrera Obrero wrote:
Hello,
Thanks, I have displayed the images which them was stored in a Oracle's BLOB field, with:
dtml-call expr="RESPONSE.setHeader('content-type','image/jpeg')">
dtml-in SQLMethod_with_a_Only_Row>
dtml-var expr="BLOB_FIELD.read()">
/dtml-in>
However, I would like to resize this image. I can not do it with the "height" and "width" attributes, because the image is obtained from a Oracle's BLOB field, and no from image object (i.e.: I can not use dtml-var "ImageObject.tag(border='5', align='left')">).
How can I do it ?
Thanks again !
Re: [Zope-dev] More signal 11 restarts....
The only real suggestion I have is to attach the debugger to a running thread and hope it hits the fault while the debugger is attached. Linux core files are difficult to impossible to debug when threading is active. The 2.4 kernels may have addressed this some but I dont know that gdb has caught up. On Monday, November 26, 2001, at 12:47 PM, Daniel Duclos wrote: I have a zope that is dumping signal 11 every 40 minutes or so. I have tried recompile python 2.1.1 with-threads without-pymalloc, recompile Zope with it, recompile ZPAtterns, recompile and instal MYSQL for Python 0.9.1, upgraded to Zope 2.4.3, all this on a Debian Linux box. Nothing changed... still restarting... Anybody, please, has any ideia on this matter? Please, let me know if there's any relevant info that I forgot to mention abot my case! Thanks in advance!! -- daniel lobato duclos -- [EMAIL PROTECTED] -- http://www.hiperlogica.com.br - ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] PluginIndexes, 2 bugs
First bug, the module 'Products/PluginIndexes/tests/testFieldIndex.py'
contains the methods 'checkEmpty', 'checkPopulated', 'checkZero',
'checkNone' and 'checkRange'; they should be named 'testEmpty',
'testPopulated', etc.. otherwise the tests doesn't run.
Last bug, The file 'Products/PluginIndexes/README.txt' from Zope 2.4.3
says that there're two ways to pass parameters:
old: dtml-in myCatalog(myindex='xx yy',myindex_usage':'blabla')
new: dtml-in myCatalog(myindex={'query':'xx yy','usage':''})
And the class 'CatalogTool' in 'Products/CMFCore/CatalogTool.py'
(CMF from the CVS) uses the new way, but it doesn't works.
Add the following test case to'PluginIndexes/tests/testFieldIndex.py':
def testNewRange(self):
Test a range search
index = FieldIndex( 'foo' )
for i in range(100):
index.index_object(i, Dummy(i%10))
r=index._apply_index({'foo': {'query': [-99, 3],
'usage': 'range:min:max'}})
assert tuple(r[1])==('foo',), r[1]
r=list(r[0].keys())
expect=[
0, 1, 2, 3, 10, 11, 12, 13, 20, 21, 22, 23, 30, 31, 32, 33,
40, 41, 42, 43, 50, 51, 52, 53, 60, 61, 62, 63, 70, 71, 72,
73, 80, 81, 82, 83, 90, 91, 92, 93
]
assert r==expect, r
This test is similar to 'testRange' but uses the new way to pass
parameters. It fails:
AssertionError: [3, 13, 23, 33, 43, 53, 63, 73, 83, 93]
--
J. David Ibáñez, Nuxeo.com
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Overriding base class method / CatalogAware mixin
[EMAIL PROTECTED] writes: I got a TypeError that I think is due to ExtensionClass when trying to call the superclasses method, even with an instance of the subclass passed as the first variable. I am not sure, I understand your problem... But it may be the problem clearly described in the ExtensionClass documentations. Look for inheritedAttribute... Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] More signal 11 restarts....
Daniel Duclos writes: I have a zope that is dumping signal 11 every 40 minutes or so. I have tried recompile python 2.1.1 with-threads without-pymalloc, recompile Zope with it, recompile ZPAtterns, recompile and instal MYSQL for Python 0.9.1, upgraded to Zope 2.4.3, all this on a Debian Linux box. Nothing changed... still restarting... Anybody, please, has any ideia on this matter? I would enable core writing (if they are not already enabled), this is done with the ulimit/limit shell command. Then, the SIG 11 will create core files that can be analysed with a debugger. This may help localize the problem... Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Catalog improvements
Hi No, i wasn't aware of your product :-( , the only one i found was ZOQL by Stephan Richter, but that didn't help much. Well, now i have written an implementation that reuses some of the code in TextIndex (for parenthesis parsing and insertion of a default operator) an then saves the query in RPN format (so the Catalog does't need to think that hard when being queried). I have taken a look at your product, and i'd say a 'new' Catalog should have sort of QueryParser plugins that know how to turn string-queries (as yours) or SQL to native Catalog queries ... I've also contacted the authors of the two proposals, just wasn't sure wether i should start this off, since i have no experience as to how the fishbowl works and i'm expected to finish my current project sometime soon. Wolfram - Original Message - From: Casey Duncan [EMAIL PROTECTED] To: Wolfram Kerber [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Tuesday, November 27, 2001 2:48 PM Subject: Re: [Zope-dev] Catalog improvements On Tuesday 20 November 2001 05:35 pm, Wolfram Kerber allegedly wrote: Hi, i'm currently working on a product that allows to attach relational information to zope-objects. It works quite well so far, but to further enhance it i need to make some changes to the Catalog. I could perhaps implement it as a separate product, but i strongly feel that those changes are best applied to the Catalog itself, as they are of general use (i think) and involve a lot of changes to the inner workings of the Catalog. In particular i need the following: - named/stored queries these are precompiled queries, so they can be executed without parsing and are easily cacheable i.e. similar to what is implemented in CMFTopic, but stored in the Catalog and a bit smarter - caching support - unions and intersections sub-queries (i.e. queries that are directed at a certain index) should be more flexibly combineable I have some code that implements this in my CatalogQuery product. It creates a query object from a string. Presently these are not persistent, but they could easily be made to be to create precompiled queries. code at: http://www.zope.org/Members/Kaivo/CatalogQuery I searched this mailing-list as well as zope.org to get an idea about what has already been discussed and requested, and there seems to be some interest in improving the Catalog. Some people even seem to have worked on this, perhaps they could give an update on this? Possibly i don't have to write everything from scratch... I would be willing to help both in coding and getting the code put into the Zope core. I would have put this into a proposal, but there already are two proposals that deal with the features i want, one is dedicated to unions/intersections, the other (TopicIndexes) to performance issues (i dont't know what's the status of these though, especially the first one is rather old), and i don't want to hijack them without asking. As so often i will need to complete my current project first, but would then like to help in improving the Catalog for a more general use. Possibly we need to rekindle discussion. I would suggest contacting the authors of those proposals to see how compatible your concepts are wth theirs. Perhaps a new proposal should be drafted with the new ideas and ty them back to the previous ones. If there is redundancy, that can be worked out. So, if there is interest, i would propose to collect some ideas and comments about how a better Catalog should look like, how it could be best implemented and how to organize this effort (with respect to the already existing proposals). I am very interested in such a discussion. Let me know what I can do to help. /---\ Casey Duncan, Sr. Web Developer National Legal Aid and Defender Association [EMAIL PROTECTED] \---/ ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
