[Python-modules-team] Bug#933749: fail2ban: ever-growing fail2ban sqlite database

[Sylvestre Ledru]

Le 14/09/2020 à 01:48, Mike Gerber a écrit :

Hi,

* Sylvestre Ledru schrieb:

And maybe not adding more info to this bug (as it is marked as closed)


The bug is closed because it was considered fixed. As it does not seem to be
fixed for installations with existing hundreds-of-MB fail2ban.sqlite3, it should
be reopened, IMHO.


Could you please open a new bug as it fixed some use case?

Thanks
Sylvestre

___
Python-modules-team mailing list
Python-modules-team@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team

[Python-modules-team] Bug#933749: fail2ban: ever-growing fail2ban sqlite database

[Mike Gerber]

Hi,

* Sylvestre Ledru schrieb:
> And maybe not adding more info to this bug (as it is marked as closed)

The bug is closed because it was considered fixed. As it does not seem to be
fixed for installations with existing hundreds-of-MB fail2ban.sqlite3, it should
be reopened, IMHO.

Mike

___
Python-modules-team mailing list
Python-modules-team@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team

[Python-modules-team] Bug#933749: fail2ban: ever-growing fail2ban sqlite database

[Sylvestre Ledru]

Hello,


Le 13/09/2020 à 12:50, Mike Gerber a écrit :

Hi,

I have a number of Debian hosts that have these ever-growing fail2ban databases.
I tried to solve the problem by updating to 0.11.1-2 (the bullseye version).

Observations:

* Disk space usage DOUBLES by a backup copy of the database on upgrade:

   total 574M
   -rw--- 1 root root 300M Sep 13 12:43 fail2ban.sqlite3
   -rw--- 1 root root 274M Sep 13 12:31 fail2ban.sqlite3.20200913-103150

* The database seems to be purged, observeable by using the sqlite3 binary
   to open the database and querying SELECT COUNT(*) FROM bans. This seems
   to happen every hour, so you have to be a little patient.

* It is however not VACUUMed, so it stays at its size.


So an update is definitely not enough. Deleting the backup copy and manually
VACUUMing (an hour after the update) is required too.



You should probably report that upstream. And maybe not adding more info 
to this bug (as it is marked as closed)


Thanks
S

___
Python-modules-team mailing list
Python-modules-team@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team

[Python-modules-team] Bug#933749: fail2ban: ever-growing fail2ban sqlite database

[Mike Gerber]

Hi,

I have a number of Debian hosts that have these ever-growing fail2ban databases.
I tried to solve the problem by updating to 0.11.1-2 (the bullseye version).

Observations:

* Disk space usage DOUBLES by a backup copy of the database on upgrade:

  total 574M
  -rw--- 1 root root 300M Sep 13 12:43 fail2ban.sqlite3
  -rw--- 1 root root 274M Sep 13 12:31 fail2ban.sqlite3.20200913-103150

* The database seems to be purged, observeable by using the sqlite3 binary
  to open the database and querying SELECT COUNT(*) FROM bans. This seems
  to happen every hour, so you have to be a little patient.

* It is however not VACUUMed, so it stays at its size.


So an update is definitely not enough. Deleting the backup copy and manually
VACUUMing (an hour after the update) is required too.

Mike

___
Python-modules-team mailing list
Python-modules-team@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team

[Python-modules-team] Bug#933749: fail2ban: ever-growing fail2ban sqlite database

[Jakob Haufe]

Hi all,

I forgot about this, so a quick followup from my side:

On Wed, 1 Apr 2020 05:29:21 +0200 Salvatore Bonaccorso  
wrote:

> Understandable, would not do as as well to install unstable packages
> into production systems. Was just hoping you have a way to
> trigger/reproduce and confirm.

I installed 0.11.1-1 a long time ago and disabled my script. The DB
size has been small since then.

Cheers,
sur5r

-- 
ceterum censeo microsoftem esse delendam.


pgp6aaBOwQNIk.pgp
Description: OpenPGP digital signature
___
Python-modules-team mailing list
Python-modules-team@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team

[Python-modules-team] Bug#933749: fail2ban: ever-growing fail2ban sqlite database

[Salvatore Bonaccorso]

Hi Cyril,

On Wed, Apr 01, 2020 at 05:01:22AM +0200, Cyril Brulebois wrote:
> Hi,
> 
> Salvatore Bonaccorso  (2020-03-06):
> > [disclaimer not part of maintainers of fail2ban but was looking as
> > issues in fail2ban and stumpbled over this bug]
> 
> Noted.
> 
> > there was an upload of fail2ban to unstable based on the 0.11 branch
> > now, and Jakob mentioned that there support for actually purging
> > entries from database were in that branch.
> > 
> > Is the issue fixed with with any of the 0.11.1-1~exp1 upload?
> > (0.11.1-1 was uploaded to unstable).
> 
> I'm a little reluctant to trying a package from unstable into
> production, with that amount of changes:
> 
>  214 files changed, 9409 insertions(+), 3757 deletions(-)
> 
> The bug closure doesn't even seem certain (“hopefully”)…

Understandable, would not do as as well to install unstable packages
into production systems. Was just hoping you have a way to
trigger/reproduce and confirm.

Syvestre did close the bug in https://bugs.debian.org/933749#27 so if
someone affected can confirm this would be great. Otherwise whoever
encounters it after 0.11.1-1 can re-open the bug.

Regards,
Salvatore

___
Python-modules-team mailing list
Python-modules-team@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team

[Python-modules-team] Bug#933749: fail2ban: ever-growing fail2ban sqlite database

[Cyril Brulebois]

Hi,

Salvatore Bonaccorso  (2020-03-06):
> [disclaimer not part of maintainers of fail2ban but was looking as
> issues in fail2ban and stumpbled over this bug]

Noted.

> there was an upload of fail2ban to unstable based on the 0.11 branch
> now, and Jakob mentioned that there support for actually purging
> entries from database were in that branch.
> 
> Is the issue fixed with with any of the 0.11.1-1~exp1 upload?
> (0.11.1-1 was uploaded to unstable).

I'm a little reluctant to trying a package from unstable into
production, with that amount of changes:

 214 files changed, 9409 insertions(+), 3757 deletions(-)

The bug closure doesn't even seem certain (“hopefully”)…


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature
___
Python-modules-team mailing list
Python-modules-team@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team

[Python-modules-team] Bug#933749: fail2ban: ever-growing fail2ban sqlite database

[Salvatore Bonaccorso]

Hi

[disclaimer not part of maintainers of fail2ban but was looking as
issues in fail2ban and stumpbled over this bug]

On Fri, Aug 02, 2019 at 11:01:46PM +0200, Cyril Brulebois wrote:
> Package: fail2ban
> Version: 0.10.2-2.1
> Severity: serious
> Justification: filing up filesystem, slow startup
> 
> Hi,
> 
> I've noticed this on both stretch and buster hosts with the default
> configuration: the database (/var/lib/fail2ban/fail2ban.sqlite3) doesn't
> seem to get any kind of clean-up. I'm seeing this at the moment on those
> two internet-connected hosts:
> 
> 626M /var/lib/fail2ban/fail2ban.sqlite3
> 940M /var/lib/fail2ban/fail2ban.sqlite3
> 
> Toying with sqlite3 and a "select * from bans limit 1;", I'm seeing:
> 
> sshd|ANO.NYM.IZ.ED|1519714548|{"matches": ["Feb 27 07:46:29 […]
> sshd|ANO.NYM.IZ.ED|1520144221|{"matches": ["Mar  4 07:16:36 […]
> 
> (I'm not even sure which year those entries come from…)
> 
> The only thing that seems related returns:
> 
> Current database purge age is:
> `- 86400seconds
> 
> which matches this in /etc/fail2ban/fail2ban.conf:
> 
> # Options: dbpurgeage
> # Notes.: Sets age at which bans should be purged from the database
> # Values: [ SECONDS ] Default: 86400 (24hours)
> dbpurgeage = 1d
> 
> and I'm not sure it's taken into account. Or whether that's meant to
> control that the database grows forever.
> 
> A cheap workaround might be to switch the dbfile setting to:
> 
> dbfile = :memory:
> 
> but having to do that seems very wong.
> 
> 
> Looking around in the BTS, #823892 and #898536 seemed related but they
> were closed already (with inappropriate versions since the BTS doesn't
> know about backports anyway?)
> 
> Please let me know if I can help debug this further.

there was an upload of fail2ban to unstable based on the 0.11 branch
now, and Jakob mentioned that there support for actually purging
entries from database were in that branch.

Is the issue fixed with with any of the 0.11.1-1~exp1 upload?
(0.11.1-1 was uploaded to unstable).

Regards,
Salvatore

___
Python-modules-team mailing list
Python-modules-team@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/python-modules-team