No problem. Is there a quick-fix / hack that perhaps I can hardcode to
bypass insertion until the actual fix is available?
Just speculating here... we've got an option (lazyIndex) that rebuilds
the index every time whether it is necessary or not. And we have an
option (luceneIndex) that won't build a full index automatically even
when no index exists for the folder. Seems to me a much better option
would be a 'build it once if it doesn't exist' option. That would be
one less thing for an overwhelmed administrator (me) to remember to do
when migrating James. Ideas?
Jerry
On 11/5/2019 1:04 AM, Tellier Benoit wrote:
My bad, there seems to be Spring injection issues:
mailbox-tools.xml defines:
<bean id ="fake-reindexer"
class="org.apache.mailbox.tools.indexer.ThrowsReIndexer" lazy-init="true"/>
As the indexer is configurable, we are not able to inject the
ListeningMessageSearchIndex for doing that.
We may need aditional logic upon the indexer choice for providing the
correct reIndexer binding:
- keep the throwing one when lucene is not selected
- Use ReIndexerImpl with luceneSearchIndex injected in it when lucene
is configured.
I will write the issue, contributions are welcomed here!
See https://issues.apache.org/jira/browse/JAMES-2965
Benoit
On 05/11/2019 13:14, Jerry Malcolm wrote:
I had already changed back to luceneIndex from lazyIndex and rebooted.
I also removed the lazyIndex's version of the lucene folder. But I'm
still getting the same exception.
On 11/5/2019 12:07 AM, Tellier Benoit wrote:
ReIndexing is not implemented on top of lazyIndex.
You need to configure lucene in index.xml (meaning for you, downtime in
search)
On 05/11/2019 12:44, Jerry Malcolm wrote:
Benoit,
This all makes sense. I tried the CLI to ReindexAll and got an
exception. I figured I was just doing something wrong on the CLI with
JMX, etc. But every other command I tried worked fine. Just the
ReindexAll and ReindexMailbox failed. (BTW.... the web page shows the
command: |{cli} Reindex #private u...@domain.tld INBOX but the actual
command is ReindexMailbox for a specific mailbox.)|
|Before I start digging into what is happening, does this exception tell
you anything? Do I need to configure something in the james-cli.sh file
telling it that I'm using JPA instead of file-based mailboxes or
anything like that?
|
|Exception:
|
[root@ip-172-31-32-236 james-3.4.0]# ./bin/james-cli.sh ReindexAll
ERROR 23:22:54,105 | org.apache.james.cli.ServerCmd | Error while
playing command
org.apache.james.mailbox.exception.MailboxException: Not implemented
at
org.apache.mailbox.tools.indexer.ThrowsReIndexer.reIndex(ThrowsReIndexer.java:43)
at
org.apache.james.adapter.mailbox.ReIndexerManagement.reIndex(ReIndexerManagement.java:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
On 11/4/2019 8:48 PM, Tellier Benoit wrote:
Hi Jerry,
Thanks for the updated details, it now makes perfect sens!
Long story short: Your Lucene index needs to be migrated as well. I
suspect you migrated only your database (full copy), hence issues.
To do migration, at Linagora, we prefer using external protocols, and
leaving the inner data structures untouched (it's just too easy to make
mistakes...).
What happens is that then you had the data in SQL database but none in
the Lucene index, hence the unaccurate search results harming the IOS
experience.
From that point:
- You can fix that from re-indexing data from the SQL database and
rebuild the Lucene index from it. There's webadmin(guice only) [1] and
CLI [2] support for that (though the CLI experience, the only available
option for Spring is more primitive.
lazyIndex means full scan on every search request. Expect the AWS bill
to be expansive. Expect your clients to complain about latencies. I
would be you, I'd rebuild the Lucene index at the first occasion.
Now, I also believe cascading user flags issues might be caused by some
unappropriate table structure, there was some bugs reported regarding
this. I'm unable to retrieve corresponding information.
[1]
https://github.com/apache/james-project/blob/master/src/site/markdown/server/manage-webadmin.md#reindexing-all-mails
[2]
https://github.com/apache/james-project/blob/master/src/site/markdown/server/manage-cli.md#re-indexing
Cheers,
Benoit
On 05/11/2019 07:47, Jerry Malcolm wrote:
I'm starting a new thread for this specific issue. One external
symptom
that has now been isolated to this problem is documented in my
long-running (and hopefully completed) thread about iPhones losing all
of their mail.
Here's what I know about this so far:
1) I had no clue what 'Lucene' was or what it did. So I've never
touched anything in any config file related to it. Whatever happened
came right out of the box as far as lucene config is concern. (ver
3.4+... I extracted the master branch a day or so after 3.4 was
released).
2) I have been in the process of migrating 17+ years of mail in a
James
SQL db from v3b5 on a dedicated server to ver 3.4 in Amazon Web
Services
environment.
3) On 21Oct, I threw the switch and went live on my AWS environment.
4) Immediately some of my clients with iPhones complained that all of
their mail disappeared from their phone.
5) Being an iPhone user myself, I verified the problem. However, I
noticed that mail arriving to the new James instance was showing up in
the iPhone. Just no mail pre 21Oct.
6) Thunderbird was showing all of the old mail in the folders. But
I've
since determined that TBird uses a different imap command that goes
straight to the JPA database to populate folders, bypassing the Lucene
index.
7) After two weeks of educating myself on the entire imap component
and
doing a whole lot of adding log statements and trial & error debug,
the
Lucene index became the prime suspect
8) Today, I removed the var/store/lucene folder and rebooted. The
iPhones now wiped out the last two weeks of email and either showed
"No
Mail" or mail that arrived in the few minutes after wiping the lucene
folder. Note that it STILL didn't reindex old mail, even with an
index
'restart'.
9) Finally on a whim, I changed indexer.xml to use "lazyIndex" instead
of "luceneIndex". Finally, the indexes rebuilt successfully, and the
iPhones now show all of the mail. Success...
Summary: lazyIndex seems to have no problems. Not sure what's going
wrong when luceneIndex is selected. But it isn't right.
Do I have any mis-assumptions? Is there something else I need to
do in
order to get Lucene to index old mail?
This has been a tough one... I think I'll take a nap now...:-)
Jerry
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
