[Rpm-maint] [rpm-software-management/rpm] Generate file class dictionary after file classification for stability (#944)

2019-11-21 Thread Panu Matilainen
Store the file type strings in the classifier, and generate the dictionary and its ids serially after the parallel section completes to ensure stable order. Besides making the classifying really run in parallel again, this also moves the pool- and file-counting related constraints out of the pa

Re: [Rpm-maint] [rpm-software-management/rpm] Generate file class dictionary after file classification for stability (#944)

2019-11-21 Thread Panu Matilainen
@bmwiedemann if you can verify this works for you, I'd appreciate. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/944#issuecomment-556981828

Re: [Rpm-maint] [rpm-software-management/rpm] Rpm query causes corruption in the file-backed mmaped bdb regions (#232)

2019-11-21 Thread zhangjianwei
Recently, I have encountered similar problems in my CentOS 7 environment. `[root@controller-3 ~]# uname -a Linux controller-3 3.10.0-862.el7.x86_64 #1 SMP Fri Apr 20 16:44:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux [root@controller-3 ~]# cat /etc/redhat-release CentOS Linux release 7.5.1804 (Core

[Rpm-maint] [rpm-software-management/rpm] Fix resource leaks on zstd open error paths (#945)

2019-11-21 Thread Panu Matilainen
If zstd stream initialization fails, the opened fd and the stream itself are leaked. Handle error exit in a central label. You can view, comment on, or merge this pull request online at: https://github.com/rpm-software-management/rpm/pull/945 -- Commit Summary -- * Fix resource leaks on zstd

Re: [Rpm-maint] [rpm-software-management/rpm] Generate file class dictionary after file classification for stability (#944)

2019-11-21 Thread Bernhard M. Wiedemann
Tested it with our khmeros-fonts build. Results are still reproducible. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/944#issuecomment-557048211___

[Rpm-maint] [rpm-software-management/rpm] Fix Arm detection improvements (#946)

2019-11-21 Thread Peter Robinson
There's some issues with the detection of particular Arm hardware features. We never want to use the crypto feature as part of rpm functionality, it's deployment is widely variable, and the HW optimisation should be run time detected with fall back through various optimisation paths. The 8c3a7b8fa

Re: [Rpm-maint] [rpm-software-management/rpm] Generate file class dictionary after file classification for stability (#944)

2019-11-21 Thread Panu Matilainen
Ack, thanks for verifying. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/944#issuecomment-557076155___ Rpm-maint mailing list R

Re: [Rpm-maint] [rpm-software-management/rpm] Rpm query causes corruption in the file-backed mmaped bdb regions (#232)

2019-11-21 Thread Phil Dibowitz
@jianwei1216 Are the processes running as root? If so, the memmap() the DB read-write, and write it back out when they are done... hence if then something changes at the same time, one of those process can write back old pages. -- You are receiving this because you are subscribed to this thread

Re: [Rpm-maint] [rpm-software-management/rpm] Rpm query causes corruption in the file-backed mmaped bdb regions (#232)

2019-11-21 Thread zhangjianwei
@jaymzh Yes, the processes running as root account. `the memmap() the DB read-write, and write it back out when they are done... hence if then something changes at the same time, one of those process can write back old pages.` `# ls -lthra /var/lib/rpm total 70M -rw-r--r--. 1 root root0 Oc

Re: [Rpm-maint] [rpm-software-management/rpm] Rpm query causes corruption in the file-backed mmaped bdb regions (#232)

2019-11-21 Thread zhangjianwei
Can not avoid it? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/232#issuecomment-557374397___ Rpm-maint mailing list Rpm-main

Re: [Rpm-maint] [rpm-software-management/rpm] Rpm query causes corruption in the file-backed mmaped bdb regions (#232)

2019-11-21 Thread Phil Dibowitz
You could instead run it as nonroot: `su -c "rpm -q " nobody` (or possibly `su -c "rpm -q " -s /bin/bash nobody` if nobody's shell is not a real shell). Or you can make sure [dcrpm](https://github.com/facebookincubator/dcrpm) runs in a regular basis to detect and correct your RPM db... but cert

Re: [Rpm-maint] [rpm-software-management/rpm] Rpm query causes corruption in the file-backed mmaped bdb regions (#232)

2019-11-21 Thread zhangjianwei
@jaymzh I see that dcrpm used typing(begin python-3.5 as a new feature added) , but my project is running on centos 7.5 1804 and using the python 2.7.5 version, Is typing(dcrpm) compatible with Python 2.7.5? -- You are receiving this because you are subscribed to this thread. Reply to this e

Re: [Rpm-maint] [rpm-software-management/rpm] Rpm query causes corruption in the file-backed mmaped bdb regions (#232)

2019-11-21 Thread Davide Cavalca
Yes, it should work just fine. We actually have packaging for dcrpm at https://github.com/facebookincubator/rpm-backports/tree/master/rpms/dcrpm that we use on CentOS 7 (and macOS) with no issues. -- You are receiving this because you are subscribed to this thread. Reply to this email directly

Re: [Rpm-maint] [rpm-software-management/rpm] Rpm query causes corruption in the file-backed mmaped bdb regions (#232)

2019-11-21 Thread zhangjianwei
In the process of testing dcrpm, when I execute the command line lsof - F p /var/lib/rpm/.dbenv.lock, lsof displays the PIDs of all processes, including 1, instead of only displaying the PIDs of the .dbenv.lock file; It may have been generated after I installed typing (Python module) independen

[Rpm-maint] [rpm-software-management/rpm] Optimize sqlite database rebuild (#947)

2019-11-21 Thread Panu Matilainen
Postpone sql index creation until closing time on database rebuilds, this is more than 50% improvement on rebuilddb time on both SSD and traditional disks. You can view, comment on, or merge this pull request online at: https://github.com/rpm-software-management/rpm/pull/947 -- Commit Summary -

Re: [Rpm-maint] [rpm-software-management/rpm] Generate file class dictionary after file classification for stability (#944)

2019-11-21 Thread Panu Matilainen
Merged #944 into master. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/944#event-2822297248___ Rpm-maint mailing list Rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Fix resource leaks on zstd open error paths (#945)

2019-11-21 Thread Panu Matilainen
Merged #945 into master. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/945#event-2822298006___ Rpm-maint mailing list Rpm-maint