Le 07/05/2014 18:17, mark debusschere a écrit : > Was this issue related to M16 or has it been outstanding? I have had > countless > problems with indexes, a few self induced, but recently indexes just stopped > working. > This was after server failed to start due to system schema already present, > then > second attempt server started but app failed to function as all searches were > failing. (first, sorry for the delayed answer, your mail just made it to my inbox due to heavy issues in The ASF mailers which has stopped working for 5 days...)
There is not one single response to your question. Let me try to give you some insight. There are many different use cases, and potantial kind of failures : - indexes you create before starting the server and injecting data - indexes you create after having injected data into the server - we have user indexes (those you create yourself) and system indexes (those used internally by the server) The bugs we had was related to the missing initialization of indexes created after some data were injected. This has been fixed partially in M16 and in trunk. Beside this obvious bug, there is a way more critical issue we are working on for months now : the backend we are using (JDBM) is ok when it comes to deal with one single BTree (an index is just a BTree, well, a bit more), but it's not supporting transactions across btrees. What does it means, and what are the consequences ? Pretty heavy : - when some user is updating an entry, it will impact tens of indexes - if at the same time another user is searching the server, it's likely to use the very same indexes - *if* this occurs at the very same time, then the updating client might modify an idex which is used by one of the reader, leading to some inconsistancies, and an error. We have added many locks to try to workaround this issue, we modified the way we fetched the candidates to limit the time it takes to build this set of candidates - and as a consequence the time during which we are holding an index -. We also added a journal in the backend to be able to recover when a crash occurs. All those modifications have improved the situation, up to a point the server now don't crash when there are some concurrent reads and writes, but it comes to a price : all the operations are way longer than they could be. We are now working on a complete replacement for the JDBM backend : Mavibot. It will allow us to process an update as a whole, ie all the indexes will be updated in one shot, or not at all. The searches will be guaranteed to work as they will *always* proceed on an existing revision. That will take a bit of time to achieve this new backend, but we are heavily working on it. I hope I clarified some aspects of the probelm, and the solution we are working on... > >
