Re: [Zope-dev] Small fix in Products.ZCTextIndex, how to go further?

2012-11-21 Thread Gauthier Bastien

  
  
Hi,
  
  does the current version of
  Products.ZCTextIndex.ZCTextIndex.ZCTextIndex.index_object method
  seems ok about keeping optimization?
  
  If so, what is the process to release a new version because we
  really need this in our projects and in Plone...  And as it fixes
  a bug, I think everybody using it needs it ;-)
  
  How to process?
  
  Thank you very much and have a nice end of day,
  
  

  

  

  
  
Gauthier Bastien
Support IMIO - CommunesPlone
rue de la Vieille Sambre 34
5190 Mornimont
Tél: +32(0)71 780979
  


  
Disclaimer


Les informations contenues dans ce courrier électronique
(annexes incluses) sont confidentielles et réservées à
l'usage exclusif des destinataires repris ci-dessus. Si
vous n'êtes pas le destinataire, soyez informé par la
présente que vous ne pouvez ni divulguer, ni reproduire,
ni faire usage de ces informations pour vous-même ou
toute tierce personne. Si vous avez reçu ce courrier
électronique par erreur, vous êtes prié d'en avertir
immédiatement l'expéditeur et d'effacer le message
e-mail de votre ordinateur.


De informatie in deze e-mail, bijlagen inbegrepen, is
vertrouwelijk en is als dus danig voorbehouden voor
exclusief gebruik door de hierboven vermelde
bestemmeling(en). Indien u niet de bestemmeling bent,
willen wij u erop wijzen dat u deze informatie niet mag
aanwenden voor eigen gebruik noch verspreiden aan
derden. Indien u deze e-mail per ongeluk hebt ontvangen,
gelieve de afzender onmiddellijk te verwittigen en deze
e-mail van uw computer te verwijderen.


The information contained in this e-mail and the annexed
documents is confidential and exclusively available to
the here above mentioned addressee(s).Should you not be
the addressee, please be informed that you may neither
disclose nor reproduce this e-mail, nor may the
information contained in this e-mail and its eventually
annexed documents be used by yourself or by a third
party. If you erroneously received this e-mail, could
you kindly and immediately inform the addresser and
delete the message on your computer.
  

  

  
  Le 14/11/12 16:26, Hanno Schlichting a écrit :

Hi.
  
  The change looks ok. But I think you broke an optimization. IIRC
  the code compares the old and new values for the index, and skips
  the indexing step if they are the same. The typical
  item.reindexObject() call sents data for all indexes, even if just
  one or two them have changed. The optimization made sure to skip
  any extra work, if there wasn't really any change for the text
  index. Without that check, you end up updating and writing a bunch
  of internal data structures in the text index every time. Those
  lead to slower write performance and more conflict errors.
  
  Could you have another look, and see if you can preserve the
  optimization?
  
  Thanks,
  Hanno
  

On Wed, Nov 14, 2012 at 4:16 PM,
  Gauthier Bastien gauth...@imio.be wrote:
  
 Hi everybody,
  
  I made a small change in the current Products.ZCTextIndex
  product, the change can be saw here :
  
  http://svn.zope.org/Products.ZCTextIndex/trunk/src/Products/ZCTextIndex/ZCTextIndex.py?rev=128273sortby=logr2=128273r1=113689
  
  and I added a test here :
  
  http://svn.zope.org/Products.ZCTextIndex/trunk/src/Products/ZCTextIndex/tests/testZCTextIndex.py?rev=128273sortby=logr2=128273r1=113689
  
  This fixes ticket : https://dev.plone.org/ticket/13310 :
  while reindexing, if the new content is empty, the index
  is not reindexed and the last indexed value is kept.
  
  I just wonder if it was the way to go, and if it was and
  if a reviewer can confirm that changes are OK, what to do
  to release a new version because this is a huge bug for
  us...
  
  Thank you and have a nice end of day,
  
  -- 

  
   

[Zope-dev] Small fix in Products.ZCTextIndex, how to go further?

2012-11-14 Thread Gauthier Bastien

  
  
Hi everybody,

I made a small change in the current Products.ZCTextIndex product,
the change can be saw here :

http://svn.zope.org/Products.ZCTextIndex/trunk/src/Products/ZCTextIndex/ZCTextIndex.py?rev=128273sortby=logr2=128273r1=113689

and I added a test here :

http://svn.zope.org/Products.ZCTextIndex/trunk/src/Products/ZCTextIndex/tests/testZCTextIndex.py?rev=128273sortby=logr2=128273r1=113689

This fixes ticket : https://dev.plone.org/ticket/13310 : while
reindexing, if the new content is empty, the index is not reindexed
and the last indexed value is kept.

I just wonder if it was the way to go, and if it was and if a
reviewer can confirm that changes are OK, what to do to release a
new version because this is a huge bug for us...

Thank you and have a nice end of day,

-- 
  

  

  


  Gauthier Bastien
Support IMIO - CommunesPlone
rue de la Vieille Sambre 34
5190 Mornimont
Tl: +32(0)71 780979

  
  

  Disclaimer
  
  
  Les informations contenues dans ce courrier lectronique
  (annexes incluses) sont confidentielles et rserves 
  l'usage exclusif des destinataires repris ci-dessus. Si
  vous n'tes pas le destinataire, soyez inform par la
  prsente que vous ne pouvez ni divulguer, ni reproduire,
  ni faire usage de ces informations pour vous-mme ou toute
  tierce personne. Si vous avez reu ce courrier
  lectronique par erreur, vous tes pri d'en avertir
  immdiatement l'expditeur et d'effacer le message e-mail
  de votre ordinateur.
  
  
  De informatie in deze e-mail, bijlagen inbegrepen, is
  vertrouwelijk en is als dus danig voorbehouden voor
  exclusief gebruik door de hierboven vermelde
  bestemmeling(en). Indien u niet de bestemmeling bent,
  willen wij u erop wijzen dat u deze informatie niet mag
  aanwenden voor eigen gebruik noch verspreiden aan derden.
  Indien u deze e-mail per ongeluk hebt ontvangen, gelieve
  de afzender onmiddellijk te verwittigen en deze e-mail van
  uw computer te verwijderen.
  
  
  The information contained in this e-mail and the annexed
  documents is confidential and exclusively available to the
  here above mentioned addressee(s).Should you not be the
  addressee, please be informed that you may neither
  disclose nor reproduce this e-mail, nor may the
  information contained in this e-mail and its eventually
  annexed documents be used by yourself or by a third party.
  If you erroneously received this e-mail, could you kindly
  and immediately inform the addresser and delete the
  message on your computer.

  

  

  

___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Small fix in Products.ZCTextIndex, how to go further?

2012-11-14 Thread Hanno Schlichting
Hi.

The change looks ok. But I think you broke an optimization. IIRC the code
compares the old and new values for the index, and skips the indexing step
if they are the same. The typical item.reindexObject() call sents data for
all indexes, even if just one or two them have changed. The optimization
made sure to skip any extra work, if there wasn't really any change for the
text index. Without that check, you end up updating and writing a bunch of
internal data structures in the text index every time. Those lead to slower
write performance and more conflict errors.

Could you have another look, and see if you can preserve the optimization?

Thanks,
Hanno


On Wed, Nov 14, 2012 at 4:16 PM, Gauthier Bastien gauth...@imio.be wrote:

  Hi everybody,

 I made a small change in the current Products.ZCTextIndex product, the
 change can be saw here :


 http://svn.zope.org/Products.ZCTextIndex/trunk/src/Products/ZCTextIndex/ZCTextIndex.py?rev=128273sortby=logr2=128273r1=113689

 and I added a test here :


 http://svn.zope.org/Products.ZCTextIndex/trunk/src/Products/ZCTextIndex/tests/testZCTextIndex.py?rev=128273sortby=logr2=128273r1=113689

 This fixes ticket : https://dev.plone.org/ticket/13310 : while
 reindexing, if the new content is empty, the index is not reindexed and the
 last indexed value is kept.

 I just wonder if it was the way to go, and if it was and if a reviewer can
 confirm that changes are OK, what to do to release a new version because
 this is a huge bug for us...

 Thank you and have a nice end of day,

 --

  Gauthier Bastien
 Support IMIO - CommunesPlone
 rue de la Vieille Sambre 34
 5190 Mornimont
 Tél: +32(0)71 780979

   Disclaimer

 Les informations contenues dans ce courrier électronique (annexes
 incluses) sont confidentielles et réservées à l'usage exclusif des
 destinataires repris ci-dessus. Si vous n'êtes pas le destinataire, soyez
 informé par la présente que vous ne pouvez ni divulguer, ni reproduire, ni
 faire usage de ces informations pour vous-même ou toute tierce personne. Si
 vous avez reçu ce courrier électronique par erreur, vous êtes prié d'en
 avertir immédiatement l'expéditeur et d'effacer le message e-mail de votre
 ordinateur.

 De informatie in deze e-mail, bijlagen inbegrepen, is vertrouwelijk en is
 als dus danig voorbehouden voor exclusief gebruik door de hierboven
 vermelde bestemmeling(en). Indien u niet de bestemmeling bent, willen wij u
 erop wijzen dat u deze informatie niet mag aanwenden voor eigen gebruik
 noch verspreiden aan derden. Indien u deze e-mail per ongeluk hebt
 ontvangen, gelieve de afzender onmiddellijk te verwittigen en deze e-mail
 van uw computer te verwijderen.

 The information contained in this e-mail and the annexed documents is
 confidential and exclusively available to the here above mentioned
 addressee(s).Should you not be the addressee, please be informed that you
 may neither disclose nor reproduce this e-mail, nor may the information
 contained in this e-mail and its eventually annexed documents be used by
 yourself or by a third party. If you erroneously received this e-mail,
 could you kindly and immediately inform the addresser and delete the
 message on your computer.

 ___
 Zope-Dev maillist  -  Zope-Dev@zope.org
 https://mail.zope.org/mailman/listinfo/zope-dev
 **  No cross posts or HTML encoding!  **
 (Related lists -
  https://mail.zope.org/mailman/listinfo/zope-announce
  https://mail.zope.org/mailman/listinfo/zope )


images1___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Small fix in Products.ZCTextIndex, how to go further?

2012-11-14 Thread Charlie Clark

Am 14.11.2012, 16:26 Uhr, schrieb Hanno Schlichting ha...@hannosch.eu:


Hi.
The change looks ok. But I think you broke an optimization. IIRC the code
compares the old and new values for the index, and skips the indexing  
step
if they are the same. The typical item.reindexObject() call sents data  
for

all indexes, even if just one or two them have changed. The optimization
made sure to skip any extra work, if there wasn't really any change for  
the
text index. Without that check, you end up updating and writing a bunch  
of
internal data structures in the text index every time. Those lead to  
slower

write performance and more conflict errors.
Could you have another look, and see if you can preserve the  
optimization?


Could we also have a clean up with a specific exception in the try:  
except: clause? I assume we're expecting an AttributeError?


And move filter to a generator expression / list comprehension? I could  
say for Python 3 compatibility but that doesn't matter so much as there is  
no easy way for the isinstance(t, basestring)


Charlie
--
Charlie Clark
Managing Director
Clark Consulting  Research
German Office
Kronenstr. 27a
Düsseldorf
D- 40217
Tel: +49-211-600-3657
Mobile: +49-178-782-6226
___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] Small fix in Products.ZCTextIndex, how to go further?

2012-11-14 Thread Gauthier Bastien

  
  
Hi Hanno,
  
  I did not know if checking the current index value could be faster
  or slower than indexing anyway.
  
  I think that checking the current index length could do what we
  need :
  
  Keeping the current code and changing this :
  
  
  --- if all_texts:
+++ if all_texts or self.index.length():
    return self.index.index_doc(documentId, all_texts)
  
  
  This way, it we have all_texts or if the index already contains
  something, we reindex...
  
  Would this keep the optimization?
  
  I adapted here and every tests passed...
  
  

  

  

  
  
Gauthier Bastien
Support IMIO - CommunesPlone
rue de la Vieille Sambre 34
5190 Mornimont
Tél: +32(0)71 780979
  


  
Disclaimer


Les informations contenues dans ce courrier électronique
(annexes incluses) sont confidentielles et réservées à
l'usage exclusif des destinataires repris ci-dessus. Si
vous n'êtes pas le destinataire, soyez informé par la
présente que vous ne pouvez ni divulguer, ni reproduire,
ni faire usage de ces informations pour vous-même ou
toute tierce personne. Si vous avez reçu ce courrier
électronique par erreur, vous êtes prié d'en avertir
immédiatement l'expéditeur et d'effacer le message
e-mail de votre ordinateur.


De informatie in deze e-mail, bijlagen inbegrepen, is
vertrouwelijk en is als dus danig voorbehouden voor
exclusief gebruik door de hierboven vermelde
bestemmeling(en). Indien u niet de bestemmeling bent,
willen wij u erop wijzen dat u deze informatie niet mag
aanwenden voor eigen gebruik noch verspreiden aan
derden. Indien u deze e-mail per ongeluk hebt ontvangen,
gelieve de afzender onmiddellijk te verwittigen en deze
e-mail van uw computer te verwijderen.


The information contained in this e-mail and the annexed
documents is confidential and exclusively available to
the here above mentioned addressee(s).Should you not be
the addressee, please be informed that you may neither
disclose nor reproduce this e-mail, nor may the
information contained in this e-mail and its eventually
annexed documents be used by yourself or by a third
party. If you erroneously received this e-mail, could
you kindly and immediately inform the addresser and
delete the message on your computer.
  

  

  
  Le 14/11/12 16:26, Hanno Schlichting a écrit :

Hi.
  
  The change looks ok. But I think you broke an optimization. IIRC
  the code compares the old and new values for the index, and skips
  the indexing step if they are the same. The typical
  item.reindexObject() call sents data for all indexes, even if just
  one or two them have changed. The optimization made sure to skip
  any extra work, if there wasn't really any change for the text
  index. Without that check, you end up updating and writing a bunch
  of internal data structures in the text index every time. Those
  lead to slower write performance and more conflict errors.
  
  Could you have another look, and see if you can preserve the
  optimization?
  
  Thanks,
  Hanno
  

On Wed, Nov 14, 2012 at 4:16 PM,
  Gauthier Bastien gauth...@imio.be wrote:
  
 Hi everybody,
  
  I made a small change in the current Products.ZCTextIndex
  product, the change can be saw here :
  
  http://svn.zope.org/Products.ZCTextIndex/trunk/src/Products/ZCTextIndex/ZCTextIndex.py?rev=128273sortby=logr2=128273r1=113689
  
  and I added a test here :
  
  http://svn.zope.org/Products.ZCTextIndex/trunk/src/Products/ZCTextIndex/tests/testZCTextIndex.py?rev=128273sortby=logr2=128273r1=113689
  
  This fixes ticket : https://dev.plone.org/ticket/13310 :
  while reindexing, if the new content is empty, the index
  is not reindexed and the last indexed value is kept.
  
  I just wonder if it was the way to go, and if it was and
  if a reviewer can confirm that changes are OK, what to do
  to release a new