Re: [Rpm-maint] [rpm-software-management/rpm] Increase lmdb DB size from 256M to 1G (#902)

2019-10-16 Thread Panu Matilainen
Merged #902 into master. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/902#event-2720110416___ Rpm-maint mailing list Rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Increase lmdb DB size from 256M to 1G (#902)

2019-10-16 Thread Panu Matilainen
pmatilai approved this pull request. Yeah, the need to predetermine the database size is one of the shortcomings of LMDB. On 64bit platforms that support sparse files, we could just some huge size and forget, but even 1GB is an enormous chunk of the available address space on 32bit. Anyway, 25

Re: [Rpm-maint] [rpm-software-management/rpm] Fixes for some issues on Arm platforms (#901)

2019-10-16 Thread Panu Matilainen
pmatilai requested changes on this pull request. As noted above, this needs to be worked out between various Arm maintainers to come up with solutions that work for everybody. We're not on a seesaw. -- You are receiving this because you are subscribed to this thread. Reply to this email direc

Re: [Rpm-maint] [rpm-software-management/rpm] Fixes for some issues on Arm platforms (#901)

2019-10-16 Thread Panu Matilainen
Have I ever mentioned I hate arm patches, because the only guaranteed thing is that nobody agrees on how they should be, and I can't be the judge since I don't have a clue. The commits being reversed have gone in to fix somebody elses issues, and we can't revert them just like that. You'll need

Re: [Rpm-maint] [rpm-software-management/rpm] Add an sqlite based rpmdb backend (experimental) (#899)

2019-10-16 Thread Panu Matilainen
pmatilai commented on this pull request. > @@ -59,6 +64,15 @@ dbDetectBackend(rpmdb rdb) } #endif +#if defined(WITH_SQLITE) +path = rstrscat(NULL, dbhome, "/Packages.sqlite", NULL); The thought has crossed my mind as well. The "reason" to have it resemble the others is basically to

Re: [Rpm-maint] [rpm-software-management/rpm] Add an sqlite based rpmdb backend (experimental) (#899)

2019-10-16 Thread Panu Matilainen
@mlschroe , choice is good, right? ;) There's value in having a minimal built-in solution, and there's value in having a solution based on proven standard building blocks, they just appeal to different audiences. And right now we don't really have the latter: BDB is a six years unmaintained beh

Re: [Rpm-maint] [rpm-software-management/rpm] Fixes for some issues on Arm platforms (#901)

2019-10-16 Thread ニール・ゴンパ
Note that even that fix as it stands will break OpenMandriva, who wrote that to support their distribution. If you want to alter it, work with @berolinux to fix it properly. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: h

Re: [Rpm-maint] [rpm-software-management/rpm] Fixes for some issues on Arm platforms (#901)

2019-10-16 Thread ニール・ゴンパ
@nullr0ute The only potentially legitimate commit here is https://github.com/rpm-software-management/rpm/pull/901/commits/790b0ec08027ba1676acc4091045d78e65aa8f2d, as this one is actually about fixing the problem referenced in the RH bug linked in the PR description. -- You are receiving this

Re: [Rpm-maint] [rpm-software-management/rpm] Fixes for some issues on Arm platforms (#901)

2019-10-16 Thread ニール・ゴンパ
Conan-Kudo requested changes on this pull request. This whole PR is awful. Aside from the fact that all but one of the commits reverted are mine, most of those changes were made as part of upstreaming ARM enablement from Mageia. The last commit was upstreaming changes from OpenMandriva to supp

Re: [Rpm-maint] [rpm-software-management/rpm] Add an sqlite based rpmdb backend (experimental) (#899)

2019-10-16 Thread ニール・ゴンパ
@mlschroe @pmatilai Well, @midipix and @Redfoxmoon3 are experimenting with the ndb backend for rpm-based [midipix](https://midipix.org/). It's been interesting... I'm more interested in the SQLite backend simply because the tooling is better for diagnosing and repairing SQLite databases. (cc:

Re: [Rpm-maint] [rpm-software-management/rpm] Add an sqlite based rpmdb backend (experimental) (#899)

2019-10-16 Thread ニール・ゴンパ
Conan-Kudo requested changes on this pull request. > @@ -59,6 +64,15 @@ dbDetectBackend(rpmdb rdb) } #endif +#if defined(WITH_SQLITE) +path = rstrscat(NULL, dbhome, "/Packages.sqlite", NULL); For this, we only have the single database file, right? Couldn't we just call this `rpmdb.

[Rpm-maint] [rpm-software-management/rpm] Increase lmdb DB size from 256M to 1G (#902)

2019-10-16 Thread ifel
The mapsize in LMDB is a limit of the DB size. It's set to 256M now, and if the DB reaches this size, rpm operations fails with: --- error: lmdb:rc(-30792) = mdb_cursor_put(): MDB_MAP_FULL: Environment mapsize limit reached error: rc(-30792) adding header #31959 record: MDB_MAP_FULL: Environm

[Rpm-maint] [rpm-software-management/rpm] Fixes for some issues on Arm platforms (#901)

2019-10-16 Thread Peter Robinson
We're seeing a few issues on Arm platforms [1], ARMv7 and aarch64, and review there's a few issues and clarifications needed around the way some of the Arm features work. This fixes up the issues seen and clarifies a few details of some of the Arm architecture quirks. [1] https://bugzilla.redha

Re: [Rpm-maint] Rpm database backend benchmarks

2019-10-16 Thread Panu Matilainen
On 10/16/19 4:28 PM, Michael Schroeder wrote: Hi, I wrote a little benchmarking tool to find out how the different database backends compare. The backends tested are bdb, ndb, lmdb, and the new sqlite backend. I used the 2506 packages I have on my system as data set. The benchmark tool does th

[Rpm-maint] Rpm database backend benchmarks

2019-10-16 Thread Michael Schroeder
Hi, I wrote a little benchmarking tool to find out how the different database backends compare. The backends tested are bdb, ndb, lmdb, and the new sqlite backend. I used the 2506 packages I have on my system as data set. The benchmark tool does the following: Add all packages to an empty data

Re: [Rpm-maint] [rpm-software-management/rpm] Revert "Fully shutdown DBUS on systemd_inhibit cleanup (RhBug:1714657)" (#900)

2019-10-16 Thread Colin Walters
Maybe key off `rpmsqSetInterruptSafety()`? I don't think mock calls that yet but it probably should. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/900#issuecomment-54269

Re: [Rpm-maint] [rpm-software-management/rpm] Revert "Fully shutdown DBUS on systemd_inhibit cleanup (RhBug:1714657)" (#900)

2019-10-16 Thread Colin Walters
Oh wow. Is there a way for me to unconditionally turn this off in rpm-ostree? Since our upgrades are always offline+transactional we never need it. More generally, it seems like this plugin should be a no-op if we're installing into a root != `/` for e.g. the mock case right? -- You are rece

Re: [Rpm-maint] [rpm-software-management/rpm] Revert "Fully shutdown DBUS on systemd_inhibit cleanup (RhBug:1714657)" (#900)

2019-10-16 Thread Panu Matilainen
Merged #900 into master. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/900#event-2717192640___ Rpm-maint mailing list Rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Revert "Fully shutdown DBUS on systemd_inhibit cleanup (RhBug:1714657)" (#900)

2019-10-16 Thread ニール・ゴンパ
Conan-Kudo approved this pull request. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/900#pullrequestreview-302470416___ Rpm

Re: [Rpm-maint] [rpm-software-management/rpm] Add NEVR provides for all packages that would be built into source rpms (#891)

2019-10-16 Thread Panu Matilainen
Tagging #657 for are interesting, related ideas -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/891#issuecomment-542616720___ Rpm

Re: [Rpm-maint] [rpm-software-management/rpm] Make %{buildsubdir} usable without being a side effect of %setup. (#860)

2019-10-16 Thread Panu Matilainen
#551 is relevant/related. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/860#issuecomment-542613820___ Rpm-maint mailing list Rp

Re: [Rpm-maint] [rpm-software-management/rpm] bad version comparison in all rpm (#896)

2019-10-16 Thread Michael Schroeder
(See also issue #561 ) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/896#issuecomment-542611714___ Rpm-maint mailing list Rpm

Re: [Rpm-maint] [rpm-software-management/rpm] Enable -Werror in CI builds (#898)

2019-10-16 Thread Panu Matilainen
Merged #898 into master. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/898#event-2716838543___ Rpm-maint mailing list Rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Enable -Werror in CI builds (#898)

2019-10-16 Thread Panu Matilainen
Since nobody objects... Note that rawhide introducing a new gcc might introduce new warnings that then completely block our CI, if that happens we'll just disable -Werror on rawhide until dealt with. -- You are receiving this because you are subscribed to this thread. Reply to this email direc

Re: [Rpm-maint] [rpm-software-management/rpm] Multiple ndb improvements (#895)

2019-10-16 Thread Panu Matilainen
Roger that. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/895#issuecomment-542605858___ Rpm-maint mailing list Rpm-maint@lists.

Re: [Rpm-maint] [rpm-software-management/rpm] Multiple ndb improvements (#895)

2019-10-16 Thread Panu Matilainen
Merged #895 into master. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/895#event-2716829046___ Rpm-maint mailing list Rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Add an sqlite based rpmdb backend (experimental) (#899)

2019-10-16 Thread Michael Schroeder
I wouldn't dismiss poor ndb that easily. It has not many LoC and fits rpm's needs perfectly. The packages database is similar to what rpm-3 had in the past (but written with fault tolerance in mind), and the indexes are just mmaped hash tables. -- You are receiving this because you are subscri

Re: [Rpm-maint] [rpm-software-management/rpm] Multiple ndb improvements (#895)

2019-10-16 Thread Michael Schroeder
It just needs to be removed, the MAP_SHARED was the next argument of the mmap call. Force-pushed new version. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/895#issuecomme

Re: [Rpm-maint] [rpm-software-management/rpm] Multiple ndb improvements (#895)

2019-10-16 Thread Michael Schroeder
@mlschroe pushed 1 commit. b475725264bbe776c9384a22b7c5438dfe8da33f Refactor mmap/munmap/mremap handling in ndb -- You are receiving this because you are subscribed to this thread. View it on GitHub: https://github.com/rpm-software-management/rpm/pull/895/files/e716bd973be5ed76c179077f94f629d

[Rpm-maint] [rpm-software-management/rpm] Revert "Fully shutdown DBUS on systemd_inhibit cleanup (RhBug:1714657)" (#900)

2019-10-16 Thread Panu Matilainen
Turns out this isn't a safe thing to do, as an API user could have their own dbus connections in the same process and shutting those down is a rather impolite thing to do (and causes crash, burn and other injuries, eg RhBug:1750575) This reverts commit d5f201345f6d27b6280750e5c6502f4418614fbc. You

Re: [Rpm-maint] [rpm-software-management/rpm] Add an sqlite based rpmdb backend (experimental) (#899)

2019-10-16 Thread Panu Matilainen
For further background on the topic: The world is full of key-value databases, but there are *very* few databases that support the kind of multiprocess concurrency that rpm requires, and even fewer that are license compatible. There's LMDB, but existing versions have severe limitations to key s

[Rpm-maint] [rpm-software-management/rpm] Add an sqlite based rpmdb backend (experimental) (#899)

2019-10-16 Thread Panu Matilainen
All normal functionality is expected to work. Automatic generation of missing index tables is missing, but that's not relevant at this time. Going forward, we'll also want some sort of compatibility tracking for the sql schema. The database scheme basically just mirrors what BDB does, using string