Re: [Sks-devel] Cleaning up KDB directory?
On 19.02.2015 14:12, Jeffrey Johnson wrote: On Feb 19, 2015, at 1:48 PM, Matt Wagner wrote: Signed PGP part Hi all, I just noticed on one of my servers that the KDB directory has grown to 26GB in size, including almost 1,200 10MB log.00 files. My other server has only 600 of the log files. I'm curious if there's a recommended practice for housekeeping here. Disk space isn't yet a problem, but I'd rather not wait until it becomes one. (Incidentally: I was looking at this while wondering if it's practical to bring up a keyserver using a snapshot of the DB, versus the .gpg exports. I was tinkering with setting up skskeyserver on a Raspberry Pi with an SD card, and each batch of 15,000 keys takes 15 to 60 minutes to import. I probably should have done fastbuild... Has anyone ever tried stopping the DB and copying the files over, versus exporting and then importing dumps?) -- Matt There are two approaches: 1) (automated) Add this line to DB_CONFIG (which you are using, correct?) set_flags DB_LOG_AUTOREMOVE 2) (manual) Run these commands on idle databases (with KDB/PTree as cwd) dbXY_recover -ev dbXY_checkpoint -1 dbXY_archive -dv (XY == version of Berkeley DB in use, e.g. XY=53 for db-5.3.x) hth 73 de Jeff I'm curious if this is a OS or distribution specific change as I took the liberty of checking on my SKS hosts and found on all 3 the sks/DB directory was only around 12GB and at most I found 3 10MB log files while none of the hosts have a DB_CONFIG other than under /usr/share/doc. ___ Sks-devel mailing list Sks-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/sks-devel
Re: [Sks-devel] Cleaning up KDB directory?
On Feb 22, 2015, at 10:19 AM, Jeremy T. Bouse jeremy.bo...@undergrid.net wrote: There are two approaches: 1) (automated) Add this line to DB_CONFIG (which you are using, correct?) set_flags DB_LOG_AUTOREMOVE 2) (manual) Run these commands on idle databases (with KDB/PTree as cwd) dbXY_recover -ev dbXY_checkpoint -1 dbXY_archive -dv (XY == version of Berkeley DB in use, e.g. XY=53 for db-5.3.x) hth 73 de Jeff I'm curious if this is a OS or distribution specific change as I took the liberty of checking on my SKS hosts and found on all 3 the sks/DB directory was only around 12GB and at most I found 3 10MB log files while none of the hosts have a DB_CONFIG other than under /usr/share/doc. Nope: bog standard Berkeley DB configuration and utilities for transactional databases. 73 de Jeff ___ Sks-devel mailing list Sks-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/sks-devel ___ Sks-devel mailing list Sks-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/sks-devel
Re: [Sks-devel] sks keeps dump files open
Brian Minton : And, related question: Will sks be okay if the dump files are modified by an external process? I have a cron job set up to fetch the latest dump files via rsync. As you built your database using sks build then you are safe. Those who use sks fastbuild MUST NOT modify the dump files. -- Kim Minh ___ Sks-devel mailing list Sks-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/sks-devel
Re: [Sks-devel] sks keeps dump files open
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 02/22/2015 10:15 PM, Kim Minh Kaplan wrote: Brian Minton writes: When I first started my keyserver, I imported the dump without the fast option. However, when running, the server process still has all the dump files open (as shown by lsof. On startup, SKS will open and keep them open all the files that are in the dump directory. Even if it does not use them. Can I convert the server's db to not read the dump files? If, and ONLY if, you built your database with sks build then you can safely remove the dump directory. To elaborate on this, if using the sks_build script, using a normal build (as opposed to fastbuild) uses this method. - -- - Kristian Fiskerstrand Blog: http://blog.sumptuouscapital.com Twitter: @krifisk - Public OpenPGP key 0xE3EDFAE3 at hkp://pool.sks-keyservers.net fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3 - Be a yardstick of quality. Some people aren't used to an environment where excellence is expected. (Steve Jobs) -BEGIN PGP SIGNATURE- iQEcBAEBCgAGBQJU6kfCAAoJEP7VAChXwav6DywH/2KNDS3qZfsPnM1vB6JOvhx5 suBXGQ/UKJ6kp5hRLGy9zKlIZeLRZpV0kVgLBU/IMgNiU7BRn0mx4KH74uXxQrv4 p5/7LCD6ug0YsYmZ5xaYZq27EFeXA1YeNqXa7GOCIKzTx4JB1JcbkucknTshgXCn vkYRiOm6s3hAfEXr/8USoGuy9WweRX2As2FaL3QM0nIjkQSpXNRy+m3QIPjTNzHG Vq9I+5Nh7Uaeh69oQLF/zyCuiCEHq9/1aMyXeXX76+9n81U/CiN1dirWuEbEXHl0 4k7m1xU6yHNfCYkykHuXthnLQiETcHmDgz/E9KIQyE3x3kMHTC1Fs3zqrVHtIdc= =I5dg -END PGP SIGNATURE- ___ Sks-devel mailing list Sks-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/sks-devel
Re: [Sks-devel] sks keeps dump files open
Brian Minton writes: When I first started my keyserver, I imported the dump without the fast option. However, when running, the server process still has all the dump files open (as shown by lsof. On startup, SKS will open and keep them open all the files that are in the dump directory. Even if it does not use them. Can I convert the server's db to not read the dump files? If, and ONLY if, you built your database with sks build then you can safely remove the dump directory. -- Kim Minh ___ Sks-devel mailing list Sks-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/sks-devel
Re: [Sks-devel] Cleaning up KDB directory?
On 22.02.2015 10:23, Jeffrey Johnson wrote: On Feb 22, 2015, at 10:19 AM, Jeremy T. Bouse jeremy.bo...@undergrid.net wrote: There are two approaches: 1) (automated) Add this line to DB_CONFIG (which you are using, correct?) set_flags DB_LOG_AUTOREMOVE 2) (manual) Run these commands on idle databases (with KDB/PTree as cwd) dbXY_recover -ev dbXY_checkpoint -1 dbXY_archive -dv (XY == version of Berkeley DB in use, e.g. XY=53 for db-5.3.x) hth 73 de Jeff I'm curious if this is a OS or distribution specific change as I took the liberty of checking on my SKS hosts and found on all 3 the sks/DB directory was only around 12GB and at most I found 3 10MB log files while none of the hosts have a DB_CONFIG other than under /usr/share/doc. Nope: bog standard Berkeley DB configuration and utilities for transactional databases. 73 de Jeff Then perhaps Debian has changed defaults or something as I've made no customization and have no DB_CONFIG nore do I have any process that runs the manual commands to maintain the database and it has never once become and issue since I've been running the servers. ___ Sks-devel mailing list Sks-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/sks-devel
Re: [Sks-devel] where are the good init files for FreeBSD?
On 2015-02-20 at 19:24 -0800, David Benfell wrote: In a recent discussion, Phil mentioned he had some more recent init files for sks for FreeBSD. I included them in that mail; part of a thread, but I changed the Subject to FreeBSD init scripts. https://lists.nongnu.org/archive/html/sks-devel/2015-02/msg00060.html Content is inline there; I PGP-signed the mail, so if you dig the original mail out of your history, you should be able to verify. I don't know why the old ones have broken. But they're dead, Jim. I am able to run sks by su'ing into the sks user; cd'ing into the home directory and running 'sks db' and 'sks recon' by hand. The init scripts I have instantly fail with no log messages whatsoever. I'm clueless. The new ones are unlikely to help. I don't know which scripts you're using, it's _possible_ that the `db_recover` invocation is not in those, in which case the new script might help after all. Also, it sounded like what Phil is doing has an additional dependency. What is it? The `runit` Port; it's a more modern implementation of some of the system management functionality; the accessory tools therein are more widely used and debugged. Don't worry about supervise; that is a different approach, which the db start script supports with `startfg` instead of `start`, so that the init script doesn't actually daemonize. If you don't want to learn about supervise or the like, then it's just additional complexity which you likely don't need. But really, until you can get sks to run when manually started from the command-line, no automatic method of running sks is really likely to help. Make sure that you have the `DB_CONFIG` file from before, or that you're setting pagesizes correctly in `sksconf`. Try setting `debuglevel` and then `debug`, then look at the logs. -Phil ___ Sks-devel mailing list Sks-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/sks-devel
