If you can afford to, in Admin UI, you can dynamically change log level for
specific subsystem or even whole Solr (or whole Apache).
Maybe turn it to Trace for a couple of minutes in the right window of time
and see what happens. Make sure you are looking at log files and not
console output, they have different log levels.
Also solrconfig.xml had a setting about echoing lucene information to log.
Maybe that influences segment generation information, which would be one of
the things you could look for.
If desperate, you could always just get a fresh Solr install instance for
the same version and run tutorial commands and look for output there. That
would allow to compare expectations with reality and you could definitely
enable Trace debug level there.
Regards,
Alex
On Wed., Oct. 6, 2021, 12:23 p.m. Saur, Alexandre (ELS-AMS), <
a.s...@elsevier.com> wrote:
> Hi Shawn,
>
> Thanks for the considerations. I also cannot see anything in the logs,
> which is unfortunate. I'm trying to see if the autocommits have a direct
> relationship with the bad Solr 8 throughput I'm experiencing. I'll start
> experimenting with the values!
>
> Regards,
> Alex
>
>
> ________________________________
> From: Shawn Heisey <apa...@elyograg.org>
> Sent: 06 October 2021 16:59
> To: users@solr.apache.org <users@solr.apache.org>
> Subject: Re: Solr 8 autocommit confirmation
>
> *** External email: use caution ***
>
>
>
> On 10/6/21 7:00 AM, Saur, Alexandre (ELS-AMS) wrote:
> > I have a quick question regarding Solr 8 autocommit feature. This is my
> configuration:
> >
> > <autoCommit>
> > <maxTime>600000</maxTime>
> > <openSearcher>true</openSearcher>
> > </autoCommit>
> >
> > <autoSoftCommit>
> > <maxTime>${solr.autoSoftCommit.maxTime:-1}</maxTime>
> > </autoSoftCommit>
> >
> > I can check this is true using the console. However, I cannot see any
> thread doing the autocommit in the logs, nor any metric that gives me the
> time the autocommit is being done.
>
> The client (dovecot) that I have using my personal Solr server issues
> commits itself, so I do not have any automatic commits in my log, and at
> this time I do not use Solr at work. I have no idea if Solr's logfile
> has specific mention of automatic commits. I would hope that it does.
> I have done a full reindex several times and I do not see anything that
> suggests an automatic commit ever fired. I have 60000 for autoCommit
> and 120000 for autoSoftCommit.
>
> If you do a manual commit after every indexing batch, and the batch
> takes less than 10 minutes (your autoCommit time) then autoCommit will
> not actually do anything, and what I see in my own log suggests that
> autoCommit and autoSoftCommit are smart enough to not even run if a
> commit is done manually before they fire.
>
> This is what I would do if I were in your shoes, and I might further
> reduce the autoSoftCommit, depending on how long commits actually take:
>
> <autoCommit>
> <maxTime>60000</maxTime>
> <openSearcher>false</openSearcher>
> </autoCommit>
>
> <autoSoftCommit>
> <maxTime>300000</maxTime>
> </autoSoftCommit>
>
> Assuming you don't do manual commits in your indexing software, this
> autoCommit would ensure the transaction logs will have less data in them,
> which is a good thing, and the autoSoftCommit handles change visibility.
> Of course, if your indexing software does its own commits and does them
> with enough frequency, then the automatic commits may be essentially
> unused.
>
> This article is valuable reading. The title says "SolrCloud" but it is
> also applicable when NOT in cloud mode:
>
>
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flucidworks.com%2Fpost%2Funderstanding-transaction-logs-softcommit-and-commit-in-sorlcloud%2F&data=04%7C01%7Ca.saur%40elsevier.com%7Cbb90d89a7de54274fec208d988d9ffac%7C9274ee3f94254109a27f9fb15c10675d%7C0%7C0%7C637691293307772302%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=xVcTSmftEFx3yRb3q%2Bi8GtSUppT0jZx35g4y3cDXLWc%3D&reserved=0
>
> Thanks,
> Shawn
>
>
> ________________________________
>
> Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The
> Netherlands, Registration No. 33158992, Registered in The Netherlands.
>
