Re: [ext] dovecot 2.3.7.2-1~bionic: Performance issues caused by excessive IO to ~/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.tmp

[Ralf Hildebrandt via dovecot]

* Aki Tuomi via dovecot :

> 2.3.7 does not generate DH keys. It's been removed since 2.3.0

Yes, it was the only periodic process I could think/knew of.

> Is it possible for you to track and find out which process is causing the 
> peak?

Will try. Next hour :)

-- 
Ralf Hildebrandt
  Geschäftsbereich IT | Abteilung Netzwerk
  Charité - Universitätsmedizin Berlin
  Campus Benjamin Franklin
  Hindenburgdamm 30 | D-12203 Berlin
  Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
  ralf.hildebra...@charite.de | https://www.charite.de

Re: [ext] dovecot 2.3.7.2-1~bionic: Performance issues caused by excessive IO to ~/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.tmp

[Aki Tuomi via dovecot]

> On 16/10/2019 13:31 Ralf Hildebrandt via dovecot  wrote:
> 
>  
> * Ralf Hildebrandt via dovecot :
> > * Timo Sirainen :
> > 
> > > > BTW: This post is a followup to my "2.3.7 slower than 2.3.6?" post from 
> > > > back in July.
> > > 
> > > Fixed by 
> > > https://github.com/dovecot/core/commit/5e9e09a041b318025fd52db2df25052b60d0fc98
> > >  and will be in the soon-to-be-released v2.3.8.
> > 
> > I stopped 2.3.7, copied over the index files from the ramdisk into
> > the physical "realm" and restarted with a fresh 2.3.8. It probably
> > takes a few days to be absolutely sure.
> 
> So, in general the performance issues are gone.
> 
> But... 
> 
> I'm seeing odd hourly spikes almost every hour, on the hour.
> You might say: Well yes, that's a cronjob sending lots of mails. But
> it isn't. There's not more or less mail coming in at that very moment.
> 
> I suspect something in dovecot running every hour (DH key regeneration?)
> 
> -- 
> Ralf Hildebrandt

2.3.7 does not generate DH keys. It's been removed since 2.3.0

Is it possible for you to track and find out which process is causing the peak?

Aki

Re: [ext] dovecot 2.3.7.2-1~bionic: Performance issues caused by excessive IO to ~/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.tmp

[Ralf Hildebrandt via dovecot]

* Ralf Hildebrandt via dovecot :
> * Timo Sirainen :
> 
> > > BTW: This post is a followup to my "2.3.7 slower than 2.3.6?" post from 
> > > back in July.
> > 
> > Fixed by 
> > https://github.com/dovecot/core/commit/5e9e09a041b318025fd52db2df25052b60d0fc98
> >  and will be in the soon-to-be-released v2.3.8.
> 
> I stopped 2.3.7, copied over the index files from the ramdisk into
> the physical "realm" and restarted with a fresh 2.3.8. It probably
> takes a few days to be absolutely sure.

So, in general the performance issues are gone.

But... 

I'm seeing odd hourly spikes almost every hour, on the hour.
You might say: Well yes, that's a cronjob sending lots of mails. But
it isn't. There's not more or less mail coming in at that very moment.

I suspect something in dovecot running every hour (DH key regeneration?)

-- 
Ralf Hildebrandt
  Geschäftsbereich IT | Abteilung Netzwerk
  Charité - Universitätsmedizin Berlin
  Campus Benjamin Franklin
  Hindenburgdamm 30 | D-12203 Berlin
  Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
  ralf.hildebra...@charite.de | https://www.charite.de

Re: [ext] dovecot 2.3.7.2-1~bionic: Performance issues caused by excessive IO to ~/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.tmp

[Ralf Hildebrandt via dovecot]

* Timo Sirainen :

> > BTW: This post is a followup to my "2.3.7 slower than 2.3.6?" post from 
> > back in July.
> 
> Fixed by 
> https://github.com/dovecot/core/commit/5e9e09a041b318025fd52db2df25052b60d0fc98
>  and will be in the soon-to-be-released v2.3.8.

I stopped 2.3.7, copied over the index files from the ramdisk into
the physical "realm" and restarted with a fresh 2.3.8. It probably
takes a few days to be absolutely sure.

-- 
Ralf Hildebrandt
  Geschäftsbereich IT | Abteilung Netzwerk
  Charité - Universitätsmedizin Berlin
  Campus Benjamin Franklin
  Hindenburgdamm 30 | D-12203 Berlin
  Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
  ralf.hildebra...@charite.de | https://www.charite.de

Re: [ext] dovecot 2.3.7.2-1~bionic: Performance issues caused by excessive IO to ~/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.tmp

[Ralf Hildebrandt via dovecot]

* Timo Sirainen :

> >> But why is that? Why would the index file be updated so often?
> > 
> > BTW: This post is a followup to my "2.3.7 slower than 2.3.6?" post from 
> > back in July.
> 
> Fixed by 
> https://github.com/dovecot/core/commit/5e9e09a041b318025fd52db2df25052b60d0fc98
>  
> 
>  and will be in the soon-to-be-released v2.3.8.

Thanks. I will test this (once it's been released) by moving the index files 
back to
conventional storage. 

-- 
Ralf Hildebrandt
  Geschäftsbereich IT | Abteilung Netzwerk
  Charité - Universitätsmedizin Berlin
  Campus Benjamin Franklin
  Hindenburgdamm 30 | D-12203 Berlin
  Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
  ralf.hildebra...@charite.de | https://www.charite.de

Re: [ext] dovecot 2.3.7.2-1~bionic: Performance issues caused by excessive IO to ~/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.tmp

[Timo Sirainen via dovecot]

On 1 Oct 2019, at 16.45, Ralf Hildebrandt via dovecot  
wrote:
> 
> * Ralf Hildebrandt via dovecot :
> 
>> But why is that? Why would the index file be updated so often?
> 
> BTW: This post is a followup to my "2.3.7 slower than 2.3.6?" post from back 
> in July.

Fixed by 
https://github.com/dovecot/core/commit/5e9e09a041b318025fd52db2df25052b60d0fc98 

 and will be in the soon-to-be-released v2.3.8.

Re: [ext] dovecot 2.3.7.2-1~bionic: Performance issues caused by excessive IO to ~/mdbox/mailboxes/INBOX/dbox-Mails/dovecot.index.tmp

[Ralf Hildebrandt via dovecot]

* Ralf Hildebrandt via dovecot :

> But why is that? Why would the index file be updated so often?

BTW: This post is a followup to my "2.3.7 slower than 2.3.6?" post from back in 
July.