Re: [Zope] Search on ZCatalog or ZSQL in MySQL
Nestor A. Diaz L. writes: I have to make a system to store news, so i have to migrate a old system using perl cgi's and perform a search on text files to use Zope as the search engine however i don't know if i have to use a SQL database like MySQL or use the ZCatalog, i need full text search, and ZCatalog fits on the true decision however the current database news is about 350M and i would like to extend it to store images and file, so is possible that at the final year if everyting comes ok it outperforms the 1GB data If you have a database or search engine that handles full text, I would go for it. However, most relational databases have difficulties with full text. I doubt, that MySQL can handle it well. The ZCatalog as it is now, is nice for small to medium size content. I, personally, would not use it for content bigger than 50 to 100 MB. Due to some weeknesses in the BTree implementation (which is used by ZCatalog to maintain its indexes) a considerable part of the index structure is rewritten if the index changes. This may lead to a fast growing ZODB with the need for frequent "pack"'s. Before you decide for ZODB, you should make some tests: something like this: * index about 10 MB of content * change content * search with the catalog * index another 10 MB of content (you now have 20 MB content) * change content * search with the catalog For each step: Look how the size of the ZODB changes: before and after packing Look how the main memory requirements changes (especially during indexing) Look how the timings changes (for all relevant operations: indexing, packing, searching) Try to extrapolate your results. You may need to add more 10 MB content chunks to recognize a tendancy. Dieter ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
[Zope] Search on ZCatalog or ZSQL in MySQL
Hi, I have a to take a decision to use or not to use the ZCatalog that's because i'm some kind newbie to Zope. Well the history is the following: I have to make a system to store news, so i have to migrate a old system using perl cgi's and perform a search on text files to use Zope as the search engine however i don't know if i have to use a SQL database like MySQL or use the ZCatalog, i need full text search, and ZCatalog fits on the true decision however the current database news is about 350M and i would like to extend it to store images and file, so is possible that at the final year if everyting comes ok it outperforms the 1GB data I read that CodeCatalog.com use MySQL to make their full text search engine, so i think that i can do the same with MySQL using a unique number for each registry and let ZCatalog do a Vocabulary based on the ZClass that get such data? and know how to access the database when someone request a word in the vocabulary based on this unique id? I have a made a prototype in Zope at http://repnet.engendro.com using ZCatalog and the admin console is at http://repnet.engendro.com/admin_html i got based on the ZClass Tutorial from behieve and get it working. Well, i have to make a decision so my last resource is to use a coin ;) Thank you very much (sorry for my poor english :DDD) -- Nestor A. Diaz Ingeniero de Desarrollo Engendro - Comercio Electronico sobre Linux Email: [EMAIL PROTECTED] - WWW: http://www.engendro.com ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Search on ZCatalog or ZSQL in MySQL
the true decision however the current database news is about 350M and i would like to extend it to store images and file, so is possible that at the final year if everyting comes ok it outperforms the 1GB data The limit I think you refer to is the 2GB file size limit on _some_ OSes. So you really have more room than you thought you did, and you may have more room than you can imagine depending on your platform. Some things I would suggest. Store the stories in the ZODB. Store URLs for the images in the ZODB as properties of the story. Store the images on the filesystem and serve them out of apache or some other static webserver. That'll help keep your Data.fs size down and also give you a little better performance. ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
[Zope] Search on ZCatalog or ZSQL in MySQL
Hi, I have a to take a decision to use or not to use the ZCatalog that's because i'm some kind newbie to Zope. Well the history is the following: I have to make a system to store news, so i have to migrate a old system using perl cgi's and perform a search on text files to use Zope as the search engine however i don't know if i have to use a SQL database like MySQL or use the ZCatalog, i need full text search, and ZCatalog fits on the true decision however the current database news is about 350M and i would like to extend it to store images and file, so is possible that at the final year if everyting comes ok it outperforms the 1GB data I read that CodeCatalog.com use MySQL to make their full text search engine, so i think that i can do the same with MySQL using a unique number for each registry and let ZCatalog do a Vocabulary based on the ZClass that get such data? and know how to access the database when someone request a word in the vocabulary based on this unique id? I have a made a prototype in Zope at http://repnet.engendro.com using ZCatalog and the admin console is at http://repnet.engendro.com/admin_html i got based on the ZClass Tutorial from behieve and get it working. Well, i have to make a decision so my last resource is to use a coin ;) Thank you very much (sorry for my poor english :DDD) -- Nestor A. Diaz Ingeniero de Desarrollo Engendro - Comercio Electronico sobre Linux Email: [EMAIL PROTECTED] - WWW: http://www.engendro.com ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
