All of you, thanks for your thoughts. Allow me to pick your brains a bit further :)
Later replies (including Ard's) concluded rightfully that the problem lies on the receiver side (website frontend), while Ard's remark below applies to the sender (repository). Also, I ran into those delete/recreate problem before (was discussed on this forum as well) so I made sure I am doing updates only. So the problem lies not there.
Seeing the discussion I remember seeing this problem before, when I used Dav2Disk to upload a bunch of documents.No problem when the documents did not exist before, but at times I was puzzled to see changes in those documents in the repo that were not visible at the site. Became never really a problem, so I didn't look further into it then.
OK. So we know what underlies the problem.Like I mentioned I built my frontend using HST1 which as far as I know is built on top of the standard, caching Java-adapter. So is it correct to conclude that the problem is basically that the java-adapter is too slow?
I see several options to solve/work around this: - (Dirk-Jan suggestion) to build in something like a queuing mechanism into the java-adapter. A question here is whether it is worth fixing the java-adapter. Isn't it deprecated with Hippo 7?- don't know the JMS protocol or its implementation, but maybe it is possible to extend some timeout on the sender (repo) side, giving listeners more time to process? Of course, this might be undesirable for the speed of the repo...
- slow down the process with which I update documents (sleep between documents) or try to become smarter updating documents (99% of the time nothing changes).
- invalidate (a whole section of) the cache whenever I know I've had my update or do this even time-triggered. Can this be done? Can I just kick the cache to flush everything? Or preferably everything within a given folder, as all documents go into the same folder.
Regards, Reinier Ard Schrijvers wrote:
Hello, On Mon, Sep 7, 2009 at 10:42 AM, Dirkjan van Diepen<[email protected]> wrote:Hi Reinier, When a batch of JMS events are received, not all JMs events are guaranteed to be processed. When the time to process the events take too long, there is a possibility that events, that are received in the mean time, will be lost before it's being processed. You have to make sure that the processing of the event takes as less time as possible.Are,you sure about this? This sounds really strange to me, and also, fortunately, I am pretty confident this is not the case. Afaik, a synchronized method is responsible for evicting cached items, where, even if it takes long, which is unlikely, no jms event ever would and should be lost. I think the problem is wrt writing from an application to the repository. Suppose the indexing takes place every 3 seconds. This means, document queue for indexing is build up for 3 seconds before being processed. It is a known issue that *when* within this time, you delete and recreate the document, the jms event of the delete is lost. This is an internal repository issue, which I could have fixed in the past, but, after I explained it on lists, people always chose the workaround: instead of delete and recreate, just use update. If you search the our mail archive, you'll sure find my previous explanations on the issue. Regards ArdA solution could be to process the events in a dedicated thread. In the processEvent method of the listener you will only add events to the thread. And let the thread handle the list of JMS events. For an example, you can have a look at http://svn.hippocms.org/repos/hippo/hippo-components/hippo-community-apps/tags/Release-HCA-v2_02_04/hca-notifier/src/java/nl/hippo/hca/notifier/updater/ Regards, Dirkjan ******************************************** Hippocms-dev: Hippo CMS development public mailinglist Searchable archives can be found at: MarkMail: http://hippocms-dev.markmail.org Nabble: http://www.nabble.com/Hippo-CMS-f26633.html******************************************** Hippocms-dev: Hippo CMS development public mailinglist Searchable archives can be found at: MarkMail: http://hippocms-dev.markmail.org Nabble: http://www.nabble.com/Hippo-CMS-f26633.html
-- Reinier van den Born HintTech B.V. T: +31(0)88 268 25 00 F: +31(0)88 268 25 01 M: +31(0)6 494 171 36 Delftechpark 37i | 2628 XJ Delft | The Netherlands www.hinttech.com HintTech is a specialist in eBusiness Technology ( .Net, Java platform, Tridion ) and IT-Projects. Chamber of Commerce The Hague nr. 27242282 | Sales Tax nr. NL8062.16.396.B01
<<attachment: reinier_van_den_born.vcf>>
******************************************** Hippocms-dev: Hippo CMS development public mailinglist Searchable archives can be found at: MarkMail: http://hippocms-dev.markmail.org Nabble: http://www.nabble.com/Hippo-CMS-f26633.html
