My bad, there seems to be Spring injection issues:
mailbox-tools.xml defines:
<bean id ="fake-reindexer"
class="org.apache.mailbox.tools.indexer.ThrowsReIndexer" lazy-init="true"/>
As the indexer is configurable, we are not able to inject the
ListeningMessageSearchIndex for doing that.
We may need aditional logic upon the indexer choice for providing the
correct reIndexer binding:
- keep the throwing one when lucene is not selected
- Use ReIndexerImpl with luceneSearchIndex injected in it when lucene
is configured.
I will write the issue, contributions are welcomed here!
See https://issues.apache.org/jira/browse/JAMES-2965
Benoit
On 05/11/2019 13:14, Jerry Malcolm wrote:
> I had already changed back to luceneIndex from lazyIndex and rebooted.
> I also removed the lazyIndex's version of the lucene folder. But I'm
> still getting the same exception.
>
>
> On 11/5/2019 12:07 AM, Tellier Benoit wrote:
>> ReIndexing is not implemented on top of lazyIndex.
>>
>> You need to configure lucene in index.xml (meaning for you, downtime in
>> search)
>>
>> On 05/11/2019 12:44, Jerry Malcolm wrote:
>>> Benoit,
>>>
>>> This all makes sense. I tried the CLI to ReindexAll and got an
>>> exception. I figured I was just doing something wrong on the CLI with
>>> JMX, etc. But every other command I tried worked fine. Just the
>>> ReindexAll and ReindexMailbox failed. (BTW.... the web page shows the
>>> command: |{cli} Reindex #private u...@domain.tld INBOX but the actual
>>> command is ReindexMailbox for a specific mailbox.)|
>>>
>>> |Before I start digging into what is happening, does this exception tell
>>> you anything? Do I need to configure something in the james-cli.sh file
>>> telling it that I'm using JPA instead of file-based mailboxes or
>>> anything like that?
>>> |
>>>
>>> |Exception:
>>> |
>>>
>>> [root@ip-172-31-32-236 james-3.4.0]# ./bin/james-cli.sh ReindexAll
>>> ERROR 23:22:54,105 | org.apache.james.cli.ServerCmd | Error while
>>> playing command
>>> org.apache.james.mailbox.exception.MailboxException: Not implemented
>>> at
>>> org.apache.mailbox.tools.indexer.ThrowsReIndexer.reIndex(ThrowsReIndexer.java:43)
>>>
>>>
>>> at
>>> org.apache.james.adapter.mailbox.ReIndexerManagement.reIndex(ReIndexerManagement.java:67)
>>>
>>>
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>
>>>
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>
>>>
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>
>>> On 11/4/2019 8:48 PM, Tellier Benoit wrote:
>>>> Hi Jerry,
>>>>
>>>> Thanks for the updated details, it now makes perfect sens!
>>>>
>>>> Long story short: Your Lucene index needs to be migrated as well. I
>>>> suspect you migrated only your database (full copy), hence issues.
>>>>
>>>> To do migration, at Linagora, we prefer using external protocols, and
>>>> leaving the inner data structures untouched (it's just too easy to make
>>>> mistakes...).
>>>>
>>>> What happens is that then you had the data in SQL database but none in
>>>> the Lucene index, hence the unaccurate search results harming the IOS
>>>> experience.
>>>>
>>>> From that point:
>>>> - You can fix that from re-indexing data from the SQL database and
>>>> rebuild the Lucene index from it. There's webadmin(guice only) [1] and
>>>> CLI [2] support for that (though the CLI experience, the only available
>>>> option for Spring is more primitive.
>>>>
>>>> lazyIndex means full scan on every search request. Expect the AWS bill
>>>> to be expansive. Expect your clients to complain about latencies. I
>>>> would be you, I'd rebuild the Lucene index at the first occasion.
>>>>
>>>> Now, I also believe cascading user flags issues might be caused by some
>>>> unappropriate table structure, there was some bugs reported regarding
>>>> this. I'm unable to retrieve corresponding information.
>>>>
>>>> [1]
>>>> https://github.com/apache/james-project/blob/master/src/site/markdown/server/manage-webadmin.md#reindexing-all-mails
>>>>
>>>>
>>>> [2]
>>>> https://github.com/apache/james-project/blob/master/src/site/markdown/server/manage-cli.md#re-indexing
>>>>
>>>>
>>>>
>>>> Cheers,
>>>>
>>>> Benoit
>>>>
>>>> On 05/11/2019 07:47, Jerry Malcolm wrote:
>>>>> I'm starting a new thread for this specific issue. One external
>>>>> symptom
>>>>> that has now been isolated to this problem is documented in my
>>>>> long-running (and hopefully completed) thread about iPhones losing all
>>>>> of their mail.
>>>>>
>>>>> Here's what I know about this so far:
>>>>>
>>>>> 1) I had no clue what 'Lucene' was or what it did. So I've never
>>>>> touched anything in any config file related to it. Whatever happened
>>>>> came right out of the box as far as lucene config is concern. (ver
>>>>> 3.4+... I extracted the master branch a day or so after 3.4 was
>>>>> released).
>>>>>
>>>>> 2) I have been in the process of migrating 17+ years of mail in a
>>>>> James
>>>>> SQL db from v3b5 on a dedicated server to ver 3.4 in Amazon Web
>>>>> Services
>>>>> environment.
>>>>>
>>>>> 3) On 21Oct, I threw the switch and went live on my AWS environment.
>>>>>
>>>>> 4) Immediately some of my clients with iPhones complained that all of
>>>>> their mail disappeared from their phone.
>>>>>
>>>>> 5) Being an iPhone user myself, I verified the problem. However, I
>>>>> noticed that mail arriving to the new James instance was showing up in
>>>>> the iPhone. Just no mail pre 21Oct.
>>>>>
>>>>> 6) Thunderbird was showing all of the old mail in the folders. But
>>>>> I've
>>>>> since determined that TBird uses a different imap command that goes
>>>>> straight to the JPA database to populate folders, bypassing the Lucene
>>>>> index.
>>>>>
>>>>> 7) After two weeks of educating myself on the entire imap component
>>>>> and
>>>>> doing a whole lot of adding log statements and trial & error debug,
>>>>> the
>>>>> Lucene index became the prime suspect
>>>>>
>>>>> 8) Today, I removed the var/store/lucene folder and rebooted. The
>>>>> iPhones now wiped out the last two weeks of email and either showed
>>>>> "No
>>>>> Mail" or mail that arrived in the few minutes after wiping the lucene
>>>>> folder. Note that it STILL didn't reindex old mail, even with an
>>>>> index
>>>>> 'restart'.
>>>>>
>>>>> 9) Finally on a whim, I changed indexer.xml to use "lazyIndex" instead
>>>>> of "luceneIndex". Finally, the indexes rebuilt successfully, and the
>>>>> iPhones now show all of the mail. Success...
>>>>>
>>>>> Summary: lazyIndex seems to have no problems. Not sure what's going
>>>>> wrong when luceneIndex is selected. But it isn't right.
>>>>>
>>>>> Do I have any mis-assumptions? Is there something else I need to
>>>>> do in
>>>>> order to get Lucene to index old mail?
>>>>>
>>>>> This has been a tough one... I think I'll take a nap now...:-)
>>>>>
>>>>> Jerry
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
>>>>> For additional commands, e-mail: server-dev-h...@james.apache.org
>>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
>>>> For additional commands, e-mail: server-dev-h...@james.apache.org
>>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
>> For additional commands, e-mail: server-dev-h...@james.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
> For additional commands, e-mail: server-dev-h...@james.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
