Re: Snapshots ordering on slave and pfs according to freequency for snapshot management
On Sat, Apr 24, 2010 at 11:16 PM, Matthew Dillon dil...@apollo.backplane.com wrote: So, here's an example. Lets say you have: /samba_export Your samba export hierarchy /hammer Your hammer filesystem /hammer/pfs/blah Some PFS in the hammer filesystem And now lets asy you want to make snapshots from PFS(s) in the hammer filesystem available to the export. This would work: mkdir /samba_export/fubar mount_null /hammer/pfs /samba_export/fubar If I do this then all the pfses under /hammer/pfs will be exported using samba I guess you meant mount_null /hammer/pfs/blah /samba_export/fubar ? That is what I did for the Master pfs /Backup1 - Hammer File System /Backup1/www-5mbak- Samba export Directory and mount /Backup1/pfs/www-hot /Backup1/www-5mbak/www-hotnullrw 0 0 But the problem is that a slave PFs cannot be mounted. So My problem is what I do with the slave PFS :-( You may have other issues... tracking the snapshots could present a problem since hammer cleanup only allows one snapshot directory to be specified. So you might have to come up with a little script to maintain a second set of snapsoft softlinks with the proper samba path which mirrors the master set which hammer cleanup maintains. Otherwise you will wind up with a lot of stale snapshot softlinks. I Initially planned to take the 5 mins snapshots on the Master andDaily snapshots on the slave. So I can export by Samba 2 shares. One having the links to the 5 mins backup( for 2 days ) and other having the links to the Daily backup ( 2000 days or old ) All I need is to figure out how to remove the 5mins snapshots that gets mirrored on the slave older than two days with out removing the daily snapshots. But I am a bit confused now since I dont see snapshots actually removed after a hammer cleanup. I will send the details with a new subject thanks --Siju
make buildworld error: field 'sysctl_ctx' has incomplete type
Hi, My Dragonfly system ( 2.7-DEVELOPMENT DragonFly v2.7.0.54.g0754fe-DEVELOPMENT #18: Wed Apr 7 10:09:21 IST 2010 ) paniced and dropped to ddb prompt with some hammer ioctl error in the morning. I rebooted it thinking I will get the error in the logs but it iwas not there in 'messages' or 'dmesg'. So Thinking there is something wrong with this development version I tried to upgrade. dfly-bkpsrv# git describe v2.7.2-62-g907aee5 but got this error during #make buildworld === usr.bin/kdump cc -O -pipe -I/usr/src/usr.bin/kdump/../ktrace -I/usr/src/usr.bin/kdump -I/usr/src/usr.bin/kdump/../.. -I/usr/src/usr.bin/kdump/../../sys -Wsystem-headers -Werror -Wall -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wold-style-definition -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wcast-align -Wunused-parameter -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -c /usr/src/usr.bin/kdump/kdump.c cc -O -pipe -I/usr/src/usr.bin/kdump/../ktrace -I/usr/src/usr.bin/kdump -I/usr/src/usr.bin/kdump/../.. -I/usr/src/usr.bin/kdump/../../sys -Wsystem-headers -Werror -Wall -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wold-style-definition -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wcast-align -Wunused-parameter -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -c ioctl.c In file included from ioctl.c:108: /usr/src/usr.bin/kdump/../../sys/sys/dsched.h:112: error: field 'sysctl_ctx' has incomplete type *** Error code 1 Stop in /usr/src/usr.bin/kdump. *** Error code 1 Stop in /usr/src/usr.bin. *** Error code 1 Stop in /usr/src. *** Error code 1 Stop in /usr/src. *** Error code 1 Stop in /usr/src. You have new mail. Thanks --Siju
RE: make buildworld error: field 'sysctl_ctx' has incomplete type
I'm aware of this issue and currently trying out a fix. Cheers, Alex Hornung : -Original Message- : From: users-err...@crater.dragonflybsd.org [mailto:users- : err...@crater.dragonflybsd.org] On Behalf Of Siju George : Sent: 26 April 2010 08:54 : To: users@crater.dragonflybsd.org : Subject: make buildworld error: field 'sysctl_ctx' has incomplete type : : Hi, : : My Dragonfly system ( 2.7-DEVELOPMENT DragonFly : v2.7.0.54.g0754fe-DEVELOPMENT #18: Wed Apr 7 10:09:21 IST 2010 ) : paniced and dropped to ddb prompt with some hammer ioctl error in : the morning. I rebooted it thinking I will get the error in the logs : but it iwas not there in 'messages' or 'dmesg'. : : So Thinking there is something wrong with this development version I : tried to upgrade. : : dfly-bkpsrv# git describe : v2.7.2-62-g907aee5 : : but got this error during : : #make buildworld : : : === usr.bin/kdump : cc -O -pipe -I/usr/src/usr.bin/kdump/../ktrace : -I/usr/src/usr.bin/kdump -I/usr/src/usr.bin/kdump/../.. : -I/usr/src/usr.bin/kdump/../../sys -Wsystem-headers -Werror -Wall -W : -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes : -Wpointer-arith -Wold-style-definition -Wreturn-type -Wcast-qual : -Wwrite-strings -Wswitch -Wshadow -Wcast-align -Wunused-parameter : -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -c : /usr/src/usr.bin/kdump/kdump.c : cc -O -pipe -I/usr/src/usr.bin/kdump/../ktrace : -I/usr/src/usr.bin/kdump -I/usr/src/usr.bin/kdump/../.. : -I/usr/src/usr.bin/kdump/../../sys -Wsystem-headers -Werror -Wall -W : -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes : -Wpointer-arith -Wold-style-definition -Wreturn-type -Wcast-qual : -Wwrite-strings -Wswitch -Wshadow -Wcast-align -Wunused-parameter : -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -c : ioctl.c : In file included from ioctl.c:108: : /usr/src/usr.bin/kdump/../../sys/sys/dsched.h:112: error: field : 'sysctl_ctx' has incomplete type : *** Error code 1 : : Stop in /usr/src/usr.bin/kdump. : *** Error code 1 : : Stop in /usr/src/usr.bin. : *** Error code 1 : : Stop in /usr/src. : *** Error code 1 : : Stop in /usr/src. : *** Error code 1 : : Stop in /usr/src. : You have new mail. : : Thanks : : --Siju
RE: make buildworld error: field 'sysctl_ctx' has incomplete type
: My Dragonfly system ( 2.7-DEVELOPMENT DragonFly : v2.7.0.54.g0754fe-DEVELOPMENT #18: Wed Apr 7 10:09:21 IST 2010 ) : paniced and dropped to ddb prompt with some hammer ioctl error in : the morning. I rebooted it thinking I will get the error in the logs : but it iwas not there in 'messages' or 'dmesg'. On another note it would be more useful if you could type 'call dumpsys' whenever you are dropped to the ddb prompt, so that we can take a look at the issue. Cheers, Alex Hornung
Re: make buildworld error: field 'sysctl_ctx' has incomplete type
On Mon, Apr 26, 2010 at 1:38 PM, Alex Hornung ahorn...@gmail.com wrote: : My Dragonfly system ( 2.7-DEVELOPMENT DragonFly : v2.7.0.54.g0754fe-DEVELOPMENT #18: Wed Apr 7 10:09:21 IST 2010 ) : paniced and dropped to ddb prompt with some hammer ioctl error in : the morning. I rebooted it thinking I will get the error in the logs : but it iwas not there in 'messages' or 'dmesg'. On another note it would be more useful if you could type 'call dumpsys' whenever you are dropped to the ddb prompt, so that we can take a look at the issue. Alright thanks :-) --Siju
Re: make buildworld error: field 'sysctl_ctx' has incomplete type
On Mon, Apr 26, 2010 at 1:36 PM, Alex Hornung ahorn...@gmail.com wrote: I'm aware of this issue and currently trying out a fix. Thanks :-) Could you also please tell me where to get the messages that were thrown into the console while the system dropped to ddb? --Siju
Hammer clean up doesn't actually remove snapshots?
Hi, I am a bit confused here. I thought this happened only on my slave pfs. But I tested it on my Master pfs as well. I had 3 days worth of 5 mins snapshots on my pfs. I ran dfly-bkpsrv# pwd /Backup1/www-5mbak dfly-bkpsrv# hammer config /Backup1/www-5mbak/www-hot snapshots 0d 2d prune 1d 5m rebalance 1d 5m reblock 1d 5m recopy1d 1h dfly-bkpsrv# hammer cleanup /Backup1/www-5mbak/www-hot cleanup /Backup1/www-5mbak/www-hot - handle PFS #5 using /var/hammer/Backup1/www-5mbak/www-hot snapshots - only removing old snapshots prune - skip rebalance - skip reblock - skip recopy - run..^Cdfly-bkpsrv# Pruning didn't take place this time because I ran hammer cleanup in the morning. Now the snapshots that remain according to snapls are dfly-bkpsrv# hammer snapls /Backup1/www-5mbak/www-hot Snapshots on /Backup1/www-5mbak/www-hot PFS #5 Transaction ID Timestamp Note 0x0001d45fe0f0 2010-04-26 09:01:06 IST - 0x0001d5b79500 2010-04-26 09:05:21 IST - 0x0001d5db1c40 2010-04-26 11:45:36 IST - 0x0001d5dc1e40 2010-04-26 11:50:08 IST - 0x0001d5dc9fd0 2010-04-26 11:55:19 IST - dfly-bkpsrv# but if I access the removed snapshots with their old softlinks I still get them :-( dfly-bkpsrv# ls -l snap-20100421-11* lrwxr-xr-x 1 root wheel 47 Apr 21 11:03 snap-20100421-1103 - /Backup1/www-5mbak/www-hot/@@0x0001cfb1fad0 lrwxr-xr-x 1 root wheel 47 Apr 21 11:10 snap-20100421-1110 - /Backup1/www-5mbak/www-hot/@@0x0001cfb1fcb0 lrwxr-xr-x 1 root wheel 47 Apr 21 11:15 snap-20100421-1115 - /Backup1/www-5mbak/www-hot/@@0x0001cfb1fe70 lrwxr-xr-x 1 root wheel 47 Apr 21 11:20 snap-20100421-1120 - /Backup1/www-5mbak/www-hot/@@0x0001cfb1ff70 lrwxr-xr-x 1 root wheel 47 Apr 21 11:25 snap-20100421-1125 - /Backup1/www-5mbak/www-hot/@@0x0001cfb20070 lrwxr-xr-x 1 root wheel 47 Apr 21 11:30 snap-20100421-1130 - /Backup1/www-5mbak/www-hot/@@0x0001cfb20170 lrwxr-xr-x 1 root wheel 47 Apr 21 11:35 snap-20100421-1135 - /Backup1/www-5mbak/www-hot/@@0x0001cfb20270 lrwxr-xr-x 1 root wheel 47 Apr 21 11:40 snap-20100421-1140 - /Backup1/www-5mbak/www-hot/@@0x0001cfb20370 lrwxr-xr-x 1 root wheel 47 Apr 21 11:45 snap-20100421-1145 - /Backup1/www-5mbak/www-hot/@@0x0001cfb20470 lrwxr-xr-x 1 root wheel 47 Apr 21 11:50 snap-20100421-1150 - /Backup1/www-5mbak/www-hot/@@0x0001cfb20570 lrwxr-xr-x 1 root wheel 47 Apr 21 11:55 snap-20100421-1155 - /Backup1/www-5mbak/www-hot/@@0x0001cfb20690 You have new mail. dfly-bkpsrv# cd snap-20100421-1103 dfly-bkpsrv# pwd /Backup1/www-5mbak/www-hot/@@0x0001cfb1fad0 dfly-bkpsrv# ls 2.0 dfly-bkpsrv# cd 2.0/ dfly-bkpsrv# pwd /Backup1/www-5mbak/www-hot/@@0x0001cfb1fad0/2.0 dfly-bkpsrv# cd ../../../snap-20100421-1155 dfly-bkpsrv# pwd /Backup1/www-5mbak/www-hot/@@0x0001cfb20690 dfly-bkpsrv# ls 2.0 dfly-bkpsrv# cd 2.0/ dfly-bkpsrv# pwd /Backup1/www-5mbak/www-hot/@@0x0001cfb20690/2.0 dfly-bkpsrv# Now I just entered the next synctid of a snapshot simply. dfly-bkpsrv# pwd /Backup1/www-5mbak/www-hot/@@0x0001cfb20691 dfly-bkpsrv# ls 2.0 dfly-bkpsrv# cd 2.0/ dfly-bkpsrv# pwd /Backup1/www-5mbak/www-hot/@@0x0001cfb20691/2.0 dfly-bkpsrv# I could access it also just like I access a snapshot. But the next snapshot is actually dfly-bkpsrv# cd /Backup1/www-5mbak/ dfly-bkpsrv# ls -l snap-20100421-1200 lrwxr-xr-x 1 root wheel 47 Apr 21 12:00 snap-20100421-1200 - /Backup1/www-5mbak/www-hot/@@0x0001cfb207b0 So does it mean pruning also didn't work? I am just confused please somebody help and clarify :-) Thanks --Siju
Re: Disabling snapshot symlinks during manual snapshot
On Sun, Apr 25, 2010 at 3:16 AM, Patrick Georgi patrick.geo...@coresystems.de wrote: Am 22.04.2010 07:30, schrieb Siju George: The pfs is mounted inside the snapshot Directory. For a MS Windows user it will look like the actual pic below. http://picasaweb.google.com/sgeorge.ml/OpenBSDDesktop#5462824618389365522 Clicking on the snapshot Directory they can get the snapshot contents :-) Your mail (and the mention on the digest) inspired me to try to configure samba's shadow_copy2 VFS module to match. That is great patrick :-) --Siju
RE: make buildworld error: field 'sysctl_ctx' has incomplete type
: /usr/src/usr.bin/kdump/../../sys/sys/dsched.h:112: error: field : 'sysctl_ctx' has incomplete type : *** Error code 1 This issue is fixed in commit 22befd90ce33cce626d1b4c5eafca9bc2770ceea[1]. Cheers, Alex Hornung [1]: http://gitweb.dragonflybsd.org/dragonfly.git/commit/22befd90ce33cce626d1b4c5 eafca9bc2770ceea
pfs-update code does not check if 'snapshots' directory exists
Hi, dfly-bkpsrv# hammer pfs-update mysql-hot prune-min=4d label=Daily backup of MySQL DB with histroy retained for 4 days snapshots=/Backup2/mysql-daily-bak mysql-hot sync-beg-tid=0x0001 sync-end-tid=0x0001d7bb29d0 shared-uuid=7b22b9b9-5111-11df-8e52-011617202aa6 unique-uuid=7b22b9d8-5111-11df-8e52-011617202aa6 label=Daily backup of MySQL DB with histroy retained for 4 days snapshots=/Backup2/mysql-daily-bak prune-min=4d operating as a MASTER dfly-bkpsrv# ls -l /Backup2/mysql-daily-bak ls: /Backup2/mysql-daily-bak: No such file or directory dfly-bkpsrv# mkdir /Backup2/mysql-daily-bak dfly-bkpsrv# ls -l /Backup2/mysql-daily-bak total 0 Is it a bug or was this done on purpose? Thanks -Siju
Re: pfs-update code does not check if 'snapshots' directory exists
On Mon, Apr 26, 2010 at 02:32:56PM +0530, Siju George wrote: Is it a bug or was this done on purpose? I'm not sure if it's intended, but if the specified directory doesn't exist, the snapshots phase of `hammer cleanup' happily replaces the PFS symlink with a snap-MMDD-hhmm and the subsequent phases fail. Maybe cleanup command should refuse to proceed if it doesn't have write access to the snapshot directory (or print a warning and forget about it?). Cheers.
Re: Hammer clean up doesn't actually remove snapshots?
but if I access the removed snapshots with their old softlinks I still get them :-( Accessing deleted snapshots gives undefined behavior. If you try, you might see parts of directories and files; anything between all and nothing might be 'missing'. In general accessing @@TID symlinks for pruned ranges gives undefined behavior; so if TID (Transaction ID) is not on a snapshot, or is older (lower number) than your most recent snapshot, or you are not within defined prune-min (PFS parameter) period, hammer pruning reblocking are allowed to delete historic data. This is just what hammer cleanup does, wrapping up the task. I think this is described in the manual pages hammer(8) HAMMER(5): http://leaf.dragonflybsd.org/cgi/web-man?command=HAMMER http://leaf.dragonflybsd.org/cgi/web-man?command=hammer .. prune softlink-dir Prune the file system based on previously created snapshot soft- links. Pruning is the act of deleting file system history. The prune command will delete file system history such that the file system state is retained for the given snapshots, and all history after the latest snapshot. By setting the per PFS parameter prune-min, history is guaranteed to be saved at least this time interval. All other history is deleted. If description is not clear, you are very welcome to suggest changes. -thomas
Insufficients buffers for rebalance
Hi, I got the Insufficient buffers for rebalance in 2.6.2-RELEASE #1 It was running in a virtualbox with 128MB ram and 60GB hd, i read that it need to have more than 128MB memory in an old mail, so i incresed the memory to 386MB, still get the same error everytime i run rebalance. Any suggestions? Damian -- http://dfbsd.trackbsd.org.ar
Amount of wiki spam
Hi, as you might noticed the amount of spam in our wiki increases (at least in my opinion). I thought about some possibilities to reduce the amount of spam. What do you guys think about it? - Registering a new account is no longer possible without administrator approval. Is this supported by ikiwiki or would this lead to an enormous amount of approval posts? Maybe Justin can comment on this ... - Add some captchas to the wiki. I really hate (!) captchas, but if this helps I'm fine with it. - Maybe more ... IMO the current protection of our main website is too fragile. Some weeks ago a malicious guy even managed it to remove our main site. And while I'm here: it would be nice if we could enforce commit messages for the wiki. Most people change things without explaining what they're doing and you have to look into the git changelog to figure it out. Cheers Matthias
pkgsrc libarchive issues and another sub-version release cycle
We've been having some build issues with pkgsrc recently. One of the main issues has been libarchive which moved to a new extattr ABI. Snapshots have been broken for too long because of this and other issues. The only way to solve this properly is to update our extattr ABI (basically dummy system calls in our case). I have done this on HEAD and MFCd the work into the 2.6 branch. Once we get daily snapshot builds and pkgsrc bulk builds working again I intend to do another mini-release cycle for 2.6. The next mini-release will probably occur in 1.5 to 2 weeks. -Matt
Re: Amount of wiki spam
I agree completely w/ regards to requiring registrations to be vetted. I'm kinda getting tired of the wiki getting messed up all the time too. -Matt
Re: Insufficients buffers for rebalance
:Hi, :I got the Insufficient buffers for rebalance in 2.6.2-RELEASE #1 :It was running in a virtualbox with 128MB ram and 60GB hd, i read that it :need to have more than 128MB memory in an old mail, so i incresed the memory :to 386MB, still get the same error everytime i run rebalance. :Any suggestions? :Damian Don't bother trying to rebalance. It won't actually hurt the filesystem much to not rebalance. The rebalancer needs a lot of buffer cache buffers to operate at the moment and there's no easy solution other then to add memory. -Matt
Re: Amount of wiki spam
On Mon, April 26, 2010 11:36 am, Matthias Schmidt wrote: Hi, as you might noticed the amount of spam in our wiki increases (at least in my opinion). I'm only noticing one spamming event maybe every couple of weeks. Am I missing more items? I watch page changes through RSS. The old wiki was getting spammed multiple times an hour, so this is light, relatively speaking. I think we've really benefited from the wiki free-to-edit-and-revert style; the number of people making changes has gone up significantly. zero, the number we had before, is easy to improve on - but even since it became truly wiki-like and open to editing the amount of contributions has improved. - Registering a new account is no longer possible without administrator approval. Is this supported by ikiwiki or would this lead to an enormous amount of approval posts? Maybe Justin can comment on this ... We can set it so that people need a password to create a new account; it could be a commonly known password or even something on the web page, like a lazy captcha. http://ikiwiki.info/plugins/passwordauth/ (account_creation_password option) I don't know how much difference this will make. There's also a spam filter: http://ikiwiki.info/plugins/blogspam/ This will reject changes that look spamm; I have not tried it but it won't hurt. Also, since it's stored in git, we can revert easily. If I (or someone) got around to setting the permissions right on the actual git repo, /usr/local/www/ikiwiki-srcdir/, you could even pull and revert without having to re-enter anything. - Add some captchas to the wiki. I really hate (!) captchas, but if this helps I'm fine with it. I haven't seen a single automated spam hit our site. They've all been attempts from individuals, as far as I can tell; the wiki equivalent of gold farmers. A captcha won't help with that. - Maybe more ... IMO the current protection of our main website is too fragile. Some weeks ago a malicious guy even managed it to remove our main site. And while I'm here: it would be nice if we could enforce commit messages for the wiki. Most people change things without explaining what they're doing and you have to look into the git changelog to figure it out. I'd like to see enforced messages too. I don't see an easy way to do that.
Re: Amount of wiki spam
Hmm. Well, when I think about it careful its more of a bad memory than anything recent. I do like the idea of having an easy-to-remember global password that we can just paste on irc. It might be worth trying that. -Matt
Re: Snapshots ordering on slave and pfs according to freequency for snapshot management
:All I need is to figure out how to remove the 5mins snapshots that :gets mirrored on the slave older than two days with out removing the :daily snapshots. : :But I am a bit confused now since I dont see snapshots actually :removed after a hammer cleanup. :I will send the details with a new subject : :thanks : :--Siju hammer cleanup only removes snapshots over X days old. It can't distinguish between fine-grained and coarse-grained snapshots that you explicitly tell hammer to make. You would have to remove those yourself (if you want to expire them before the X days) using hammer snaprm. You can script it fairly easily by setting the comment field for each snapshot you take, then filtering out the list based on that. See the manual page. -Matt Matthew Dillon dil...@backplane.com
Re: Amount of wiki spam
On Mon, April 26, 2010 9:01 pm, Matthew Dillon wrote: Hmm. Well, when I think about it careful its more of a bad memory than anything recent. I do like the idea of having an easy-to-remember global password that we can just paste on irc. It might be worth trying that. We could do what NetBSD does with their online bug forms. There's a last sentence that says This server runs NetBSD. To verify you are not a bot, which OS does this server run? with a little spot to type NetBSD. We could do the same, possibly.
Re: Amount of wiki spam
: :On Mon, April 26, 2010 9:01 pm, Matthew Dillon wrote: : Hmm. Well, when I think about it careful its more of a bad memory : than anything recent. I do like the idea of having an : easy-to-remember global password that we can just paste on irc. : It might be worth trying that. : :We could do what NetBSD does with their online bug forms. There's a last :sentence that says This server runs NetBSD. To verify you are not a bot, :which OS does this server run? with a little spot to type NetBSD. We :could do the same, possibly. Ooh, I like that. Here's the password XYZ, pleaes type it in. heh. -Matt Matthew Dillon dil...@backplane.com
