Re: [Zope-dev] Small fix in Products.ZCTextIndex, how to go further?
Hi, does the current version of Products.ZCTextIndex.ZCTextIndex.ZCTextIndex.index_object method seems ok about keeping optimization? If so, what is the process to release a new version because we really need this in our projects and in Plone... And as it fixes a bug, I think everybody using it needs it ;-) How to process? Thank you very much and have a nice end of day, Gauthier Bastien Support IMIO - CommunesPlone rue de la Vieille Sambre 34 5190 Mornimont Tél: +32(0)71 780979 Disclaimer Les informations contenues dans ce courrier électronique (annexes incluses) sont confidentielles et réservées à l'usage exclusif des destinataires repris ci-dessus. Si vous n'êtes pas le destinataire, soyez informé par la présente que vous ne pouvez ni divulguer, ni reproduire, ni faire usage de ces informations pour vous-même ou toute tierce personne. Si vous avez reçu ce courrier électronique par erreur, vous êtes prié d'en avertir immédiatement l'expéditeur et d'effacer le message e-mail de votre ordinateur. De informatie in deze e-mail, bijlagen inbegrepen, is vertrouwelijk en is als dus danig voorbehouden voor exclusief gebruik door de hierboven vermelde bestemmeling(en). Indien u niet de bestemmeling bent, willen wij u erop wijzen dat u deze informatie niet mag aanwenden voor eigen gebruik noch verspreiden aan derden. Indien u deze e-mail per ongeluk hebt ontvangen, gelieve de afzender onmiddellijk te verwittigen en deze e-mail van uw computer te verwijderen. The information contained in this e-mail and the annexed documents is confidential and exclusively available to the here above mentioned addressee(s).Should you not be the addressee, please be informed that you may neither disclose nor reproduce this e-mail, nor may the information contained in this e-mail and its eventually annexed documents be used by yourself or by a third party. If you erroneously received this e-mail, could you kindly and immediately inform the addresser and delete the message on your computer. Le 14/11/12 16:26, Hanno Schlichting a écrit : Hi. The change looks ok. But I think you broke an optimization. IIRC the code compares the old and new values for the index, and skips the indexing step if they are the same. The typical item.reindexObject() call sents data for all indexes, even if just one or two them have changed. The optimization made sure to skip any extra work, if there wasn't really any change for the text index. Without that check, you end up updating and writing a bunch of internal data structures in the text index every time. Those lead to slower write performance and more conflict errors. Could you have another look, and see if you can preserve the optimization? Thanks, Hanno On Wed, Nov 14, 2012 at 4:16 PM, Gauthier Bastienwrote: Hi everybody, I made a small change in the current Products.ZCTextIndex product, the change can be saw here : http://svn.zope.org/Products.ZCTextIndex/trunk/src/Products/ZCTextIndex/ZCTextIndex.py?rev=128273&sortby=log&r2=128273&r1=113689 and I added a test here : http://svn.zope.org/Products.ZCTextIndex/trunk/src/Products/ZCTextIndex/tests/testZCTextIndex.py?rev=128273&sortby=log&r2=128273&r1=113689 This fixes ticket : https://dev.plone.org/ticket/13310 : while reindexing, if the new content is empty, the index is not reindexed and the last indexed value is kept. I just wonder if it was the way to go, and if it was and if a reviewer can confirm that changes are OK, what to do to release a new version because this is a huge bug for us... Thank you and have a nice end of day, --
Re: [Zope-dev] Small fix in Products.ZCTextIndex, how to go further?
Hi Hanno, I did not know if checking the current index value could be faster or slower than indexing anyway. I think that checking the current index length could do what we need : Keeping the current code and changing this : --- if all_texts: +++ if all_texts or self.index.length(): return self.index.index_doc(documentId, all_texts) This way, it we have all_texts or if the index already contains something, we reindex... Would this keep the optimization? I adapted here and every tests passed... Gauthier Bastien Support IMIO - CommunesPlone rue de la Vieille Sambre 34 5190 Mornimont Tél: +32(0)71 780979 Disclaimer Les informations contenues dans ce courrier électronique (annexes incluses) sont confidentielles et réservées à l'usage exclusif des destinataires repris ci-dessus. Si vous n'êtes pas le destinataire, soyez informé par la présente que vous ne pouvez ni divulguer, ni reproduire, ni faire usage de ces informations pour vous-même ou toute tierce personne. Si vous avez reçu ce courrier électronique par erreur, vous êtes prié d'en avertir immédiatement l'expéditeur et d'effacer le message e-mail de votre ordinateur. De informatie in deze e-mail, bijlagen inbegrepen, is vertrouwelijk en is als dus danig voorbehouden voor exclusief gebruik door de hierboven vermelde bestemmeling(en). Indien u niet de bestemmeling bent, willen wij u erop wijzen dat u deze informatie niet mag aanwenden voor eigen gebruik noch verspreiden aan derden. Indien u deze e-mail per ongeluk hebt ontvangen, gelieve de afzender onmiddellijk te verwittigen en deze e-mail van uw computer te verwijderen. The information contained in this e-mail and the annexed documents is confidential and exclusively available to the here above mentioned addressee(s).Should you not be the addressee, please be informed that you may neither disclose nor reproduce this e-mail, nor may the information contained in this e-mail and its eventually annexed documents be used by yourself or by a third party. If you erroneously received this e-mail, could you kindly and immediately inform the addresser and delete the message on your computer. Le 14/11/12 16:26, Hanno Schlichting a écrit : Hi. The change looks ok. But I think you broke an optimization. IIRC the code compares the old and new values for the index, and skips the indexing step if they are the same. The typical item.reindexObject() call sents data for all indexes, even if just one or two them have changed. The optimization made sure to skip any extra work, if there wasn't really any change for the text index. Without that check, you end up updating and writing a bunch of internal data structures in the text index every time. Those lead to slower write performance and more conflict errors. Could you have another look, and see if you can preserve the optimization? Thanks, Hanno On Wed, Nov 14, 2012 at 4:16 PM, Gauthier Bastienwrote: Hi everybody, I made a small change in the current Products.ZCTextIndex product, the change can be saw here : http://svn.zope.org/Products.ZCTextIndex/trunk/src/Products/ZCTextIndex/ZCTextIndex.py?rev=128273&sortby=log&r2=128273&r1=113689 and I added a test here : http://svn.zope.org/Products.ZCTextIndex/trunk/src/Products/ZCTextIndex/tests/testZCTextIndex.py?rev=128273&sortby=log&r2=128273&r1=113689 This fixes ticket : https://dev.plone.org/ticket/13310 : while reindexing, if the new content is empty, the index is not reindexed and the last indexed value is kept. I just wonder if it was the way to go, and if it was and if a reviewer can confirm that changes are OK, what to do to release a ne
Re: [Zope-dev] Small fix in Products.ZCTextIndex, how to go further?
Am 14.11.2012, 16:26 Uhr, schrieb Hanno Schlichting : Hi. The change looks ok. But I think you broke an optimization. IIRC the code compares the old and new values for the index, and skips the indexing step if they are the same. The typical item.reindexObject() call sents data for all indexes, even if just one or two them have changed. The optimization made sure to skip any extra work, if there wasn't really any change for the text index. Without that check, you end up updating and writing a bunch of internal data structures in the text index every time. Those lead to slower write performance and more conflict errors. Could you have another look, and see if you can preserve the optimization? Could we also have a clean up with a specific exception in the try: except: clause? I assume we're expecting an AttributeError? And move filter to a generator expression / list comprehension? I could say for Python 3 compatibility but that doesn't matter so much as there is no easy way for the isinstance(t, basestring) Charlie -- Charlie Clark Managing Director Clark Consulting & Research German Office Kronenstr. 27a Düsseldorf D- 40217 Tel: +49-211-600-3657 Mobile: +49-178-782-6226 ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Small fix in Products.ZCTextIndex, how to go further?
Hi. The change looks ok. But I think you broke an optimization. IIRC the code compares the old and new values for the index, and skips the indexing step if they are the same. The typical item.reindexObject() call sents data for all indexes, even if just one or two them have changed. The optimization made sure to skip any extra work, if there wasn't really any change for the text index. Without that check, you end up updating and writing a bunch of internal data structures in the text index every time. Those lead to slower write performance and more conflict errors. Could you have another look, and see if you can preserve the optimization? Thanks, Hanno On Wed, Nov 14, 2012 at 4:16 PM, Gauthier Bastien wrote: > Hi everybody, > > I made a small change in the current Products.ZCTextIndex product, the > change can be saw here : > > > http://svn.zope.org/Products.ZCTextIndex/trunk/src/Products/ZCTextIndex/ZCTextIndex.py?rev=128273&sortby=log&r2=128273&r1=113689 > > and I added a test here : > > > http://svn.zope.org/Products.ZCTextIndex/trunk/src/Products/ZCTextIndex/tests/testZCTextIndex.py?rev=128273&sortby=log&r2=128273&r1=113689 > > This fixes ticket : https://dev.plone.org/ticket/13310 : while > reindexing, if the new content is empty, the index is not reindexed and the > last indexed value is kept. > > I just wonder if it was the way to go, and if it was and if a reviewer can > confirm that changes are OK, what to do to release a new version because > this is a huge bug for us... > > Thank you and have a nice end of day, > > -- > > Gauthier Bastien > Support IMIO - CommunesPlone > rue de la Vieille Sambre 34 > 5190 Mornimont > Tél: +32(0)71 780979 > > Disclaimer > > Les informations contenues dans ce courrier électronique (annexes > incluses) sont confidentielles et réservées à l'usage exclusif des > destinataires repris ci-dessus. Si vous n'êtes pas le destinataire, soyez > informé par la présente que vous ne pouvez ni divulguer, ni reproduire, ni > faire usage de ces informations pour vous-même ou toute tierce personne. Si > vous avez reçu ce courrier électronique par erreur, vous êtes prié d'en > avertir immédiatement l'expéditeur et d'effacer le message e-mail de votre > ordinateur. > > De informatie in deze e-mail, bijlagen inbegrepen, is vertrouwelijk en is > als dus danig voorbehouden voor exclusief gebruik door de hierboven > vermelde bestemmeling(en). Indien u niet de bestemmeling bent, willen wij u > erop wijzen dat u deze informatie niet mag aanwenden voor eigen gebruik > noch verspreiden aan derden. Indien u deze e-mail per ongeluk hebt > ontvangen, gelieve de afzender onmiddellijk te verwittigen en deze e-mail > van uw computer te verwijderen. > > The information contained in this e-mail and the annexed documents is > confidential and exclusively available to the here above mentioned > addressee(s).Should you not be the addressee, please be informed that you > may neither disclose nor reproduce this e-mail, nor may the information > contained in this e-mail and its eventually annexed documents be used by > yourself or by a third party. If you erroneously received this e-mail, > could you kindly and immediately inform the addresser and delete the > message on your computer. > > ___ > Zope-Dev maillist - Zope-Dev@zope.org > https://mail.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > https://mail.zope.org/mailman/listinfo/zope-announce > https://mail.zope.org/mailman/listinfo/zope ) > > <>___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Small fix in Products.ZCTextIndex, how to go further?
Hi everybody, I made a small change in the current Products.ZCTextIndex product, the change can be saw here : http://svn.zope.org/Products.ZCTextIndex/trunk/src/Products/ZCTextIndex/ZCTextIndex.py?rev=128273&sortby=log&r2=128273&r1=113689 and I added a test here : http://svn.zope.org/Products.ZCTextIndex/trunk/src/Products/ZCTextIndex/tests/testZCTextIndex.py?rev=128273&sortby=log&r2=128273&r1=113689 This fixes ticket : https://dev.plone.org/ticket/13310 : while reindexing, if the new content is empty, the index is not reindexed and the last indexed value is kept. I just wonder if it was the way to go, and if it was and if a reviewer can confirm that changes are OK, what to do to release a new version because this is a huge bug for us... Thank you and have a nice end of day, -- Gauthier Bastien Support IMIO - CommunesPlone rue de la Vieille Sambre 34 5190 Mornimont Tél: +32(0)71 780979 Disclaimer Les informations contenues dans ce courrier électronique (annexes incluses) sont confidentielles et réservées à l'usage exclusif des destinataires repris ci-dessus. Si vous n'êtes pas le destinataire, soyez informé par la présente que vous ne pouvez ni divulguer, ni reproduire, ni faire usage de ces informations pour vous-même ou toute tierce personne. Si vous avez reçu ce courrier électronique par erreur, vous êtes prié d'en avertir immédiatement l'expéditeur et d'effacer le message e-mail de votre ordinateur. De informatie in deze e-mail, bijlagen inbegrepen, is vertrouwelijk en is als dus danig voorbehouden voor exclusief gebruik door de hierboven vermelde bestemmeling(en). Indien u niet de bestemmeling bent, willen wij u erop wijzen dat u deze informatie niet mag aanwenden voor eigen gebruik noch verspreiden aan derden. Indien u deze e-mail per ongeluk hebt ontvangen, gelieve de afzender onmiddellijk te verwittigen en deze e-mail van uw computer te verwijderen. The information contained in this e-mail and the annexed documents is confidential and exclusively available to the here above mentioned addressee(s).Should you not be the addressee, please be informed that you may neither disclose nor reproduce this e-mail, nor may the information contained in this e-mail and its eventually annexed documents be used by yourself or by a third party. If you erroneously received this e-mail, could you kindly and immediately inform the addresser and delete the message on your computer. ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )