Re: xterm initial tty weirdness
Alexander Hall alexander at beard.se writes: Editing commands sometimes don't (seem to) work on the first shell prompt in newly opened xterm window: $ bar^A^E^Afoo ENTER Interesting. I sometimes have that after exiting a curses application (e.g. lynx) or less.
Re: What are the disadvantages of soft updates?
I was following this discussion with the great interest but without intend to participate in it until today. Namely one of my OpenBSD servers (5.6 sparc64) runs Mollify and last night I received an e-mail from an angry user who could not upload files (the upload will fail or upload the file with file size zero). After running df I noticed that /tmp was 100% full (4GB used) but the size of individual files was only 12Kb. I thought for a second and I remember seeing this with HAMMER on DF. Long story short I checked /etc/fstab and sure enough I had rw,softdep next to each partition including tmp. I removed softdep rebooted the sytem and /tmp usage dropped to 0%. More importantly users could upload files again. My questions is which partitions should be mounted with softdep option? Is there a way to prune metadata which will save me for problems like the one I encountered last night. Best, Predrag
Munich BSD meetup
Hi, stolen the idea from Reyk, maybe we can have a BSD meetup near munich. One and only cirtitcal question is where. I live about 30km North of Munich, but my job is near munich central station. I fine with everywhere reachable in 1,5 hours. Jan
Re: Hannover BSD meetup
Time it with CeBit and everyone will have a reason to come from afar :-)
Re: Munich BSD meetup
On Fri, Jan 23, 2015 at 07:58:37PM +0100, Jan Lambertz wrote: Hi, stolen the idea from Reyk, maybe we can have a BSD meetup near munich. One and only cirtitcal question is where. I live about 30km North of Munich, but my job is near munich central station. I fine with everywhere reachable in 1,5 hours. After all, someone wants to see very druns russians, bears etc?! .damn policy!
Re: Hannover BSD meetup
Challenge accepted =) I'm thinking about organizing a little meet up for some time. i thought nobody will come, but maybe its worth a try. Proposals for a location ? And Maybe we should start a new thread for munich.
Re: What are the disadvantages of soft updates?
Hi Predrag, Predrag Punosevac wrote on Fri, Jan 23, 2015 at 03:24:00PM -0500: I was following this discussion with the great interest but without intend to participate in it until today. Namely one of my OpenBSD servers (5.6 sparc64) runs Mollify and last night I received an e-mail from an angry user who could not upload files (the upload will fail or upload the file with file size zero). After running df I noticed that /tmp was 100% full (4GB used) but the size of individual files was only 12Kb. That is unlikely to be due to softdep. The usual reason for a file system to be actually full and seemingly almost empty at the same time is somebody doing the following sequence of operations: - open(2) a file for writing - writing a lot of data until the file system is full - unlink(2) the file - keep the process running that open(2)'ed it - let that process keep the file open and *not* close(2) it Specifically, in /tmp, anybody can do that. I thought for a second and I remember seeing this with HAMMER on DF. The above works with almost any file system. Long story short I checked /etc/fstab and sure enough I had rw,softdep next to each partition including tmp. I removed softdep rebooted the sytem and /tmp usage dropped to 0%. That's not likely to be related to softdep either. Chances are just rebooting would have solved the issue as well - simply because rebooting terminates all running processes, and consequently closes all open files. What you should have done instead was run fstat(1), look for processes having files open in /tmp, use ls(1) -iRa /tmp to find the inode numbers of linked files in /tmp, and kill the processes having files open that were *not* linked until you found the one having the big file open - and then have a friendly talk with the responsible user, if any, or figure out what went wrong in case some daemon process caused the issue. My questions is which partitions should be mounted with softdep option? I'm not an expert on that and hardly ever use softdep, but i'd say on file systems where file create/delete performance is *critically* important, performace has been clearly demonstrated to be insufficient without softdep, and you consider data loss harmless. Is there a way to prune metadata which will save me for problems like the one I encountered last night. I'm not convinced that's a good question to ask. Yours, Ingo
Re: What are the disadvantages of soft updates?
On Fri, Jan 23, 2015 at 2:24 PM, Predrag Punosevac punoseva...@gmail.com wrote: I was following this discussion with the great interest but without intend to participate in it until today. Namely one of my OpenBSD servers (5.6 sparc64) runs Mollify and last night I received an e-mail from an angry user who could not upload files (the upload will fail or upload the file with file size zero). After running df I noticed that /tmp was 100% full (4GB used) but the size of individual files was only 12Kb. I thought for a second and I remember seeing this with HAMMER on DF. Long story short I checked /etc/fstab and sure enough I had rw,softdep next to each partition including tmp. I removed softdep rebooted the sytem and /tmp usage dropped to 0%. More importantly users could upload files again. Unless your server is heavily loaded softdep will sync and write to disk within 30 secs to few minutes. IMHO, your /tmp being 4GB is unrelated to softdep. What is the underlying issue why /tmp fills so much?
Re: What are the disadvantages of soft updates?
Quoting Predrag Punosevac punoseva...@gmail.com: I was following this discussion with the great interest but without intend to participate in it until today. Namely one of my OpenBSD servers (5.6 sparc64) runs Mollify and last night I received an e-mail from an angry user who could not upload files (the upload will fail or upload the file with file size zero). After running df I noticed that /tmp was 100% full (4GB used) but the size of individual files was only 12Kb. I thought for a second and I remember seeing this with HAMMER on DF. Long story short I checked /etc/fstab and sure enough I had rw,softdep next to each partition including tmp. I removed softdep rebooted the sytem and /tmp usage dropped to 0%. More importantly users could upload files again. Two things: UNIX servers like OpenBSD usually clean /tmp every reboot: $ ls -la /tmp total 20 drwxrwxrwt 5 root wheel 512 Jan 23 15:00 . drwxr-xr-x 16 root wheel 512 Jan 23 14:58 .. drwxrwxrwt 2 root wheel 512 Jan 23 15:00 .ICE-unix drwxrwxrwt 2 root wheel 512 Jan 23 15:00 .X11-unix drwxr-xr-x 2 root wheel 512 Jan 23 15:00 aucat $ uptime 3:00PM up 1 min, 1 user, load averages: 1.11, 0.41, 0.16 And one thing is space available and other different but related is inodes available: $ df -i /tmp Filesystem 512-blocks Used Avail Capacity iused ifree %iused Mounted on /dev/wd0a 1920764126340 1698388 7%2439 127479 2% / If you have lots of small files you might have plenty of space available, but will be unable to create more files if there are no inodes available. -- Best regards, Jorge Lopez. This message was sent using IMP, the Internet Messaging Program.
pf queuing and dropped packets
I noticed the following when downloading a large file: queue tcp_ack parent root on fxp0 bandwidth 2M qlimit 50 [ pkts: 289461 bytes: 15631434 dropped pkts: 16 bytes:864 ] [ qlength: 0/ 50 ] [ measured: 3660.9 packets/s, 1.58Mb/s ] While the number of dropped packets is very small and probably insignificant, I would have expected zero dropped packets as little else is competing for the ~12Mbps that's available in the parent queue/circuit. I thought this might be related to qlength, but since this is, apparently, zero during the time of the download I'm not certain what would be causing this. What might I be missing here and how do I resolve (I don't want to set a min here if it can be avoided). FWIW, net.inet.ip.ifq.drops=0. Thanks.
Re: What are the disadvantages of soft updates?
On Fri, Jan 23, 2015 at 2:53 PM, Ingo Schwarze schwa...@usta.de wrote: Hi Predrag, Predrag Punosevac wrote on Fri, Jan 23, 2015 at 03:24:00PM -0500: I was following this discussion with the great interest but without intend to participate in it until today. Namely one of my OpenBSD servers (5.6 sparc64) runs Mollify and last night I received an e-mail from an angry user who could not upload files (the upload will fail or upload the file with file size zero). After running df I noticed that /tmp was 100% full (4GB used) but the size of individual files was only 12Kb. That is unlikely to be due to softdep. The usual reason for a file system to be actually full and seemingly almost empty at the same time is somebody doing the following sequence of operations: - open(2) a file for writing - writing a lot of data until the file system is full - unlink(2) the file - keep the process running that open(2)'ed it - let that process keep the file open and *not* close(2) it Specifically, in /tmp, anybody can do that. I thought for a second and I remember seeing this with HAMMER on DF. The above works with almost any file system. Long story short I checked /etc/fstab and sure enough I had rw,softdep next to each partition including tmp. I removed softdep rebooted the sytem and /tmp usage dropped to 0%. That's not likely to be related to softdep either. Chances are just rebooting would have solved the issue as well - simply because rebooting terminates all running processes, and consequently closes all open files. One more point to add to Ingo's detailed and very helpful reply. Reboot actually clears /tmp. What you should have done instead was run fstat(1), look for processes having files open in /tmp, use ls(1) -iRa /tmp to find the inode numbers of linked files in /tmp, and kill the processes having files open that were *not* linked until you found the one having the big file open - and then have a friendly talk with the responsible user, if any, or figure out what went wrong in case some daemon process caused the issue. My questions is which partitions should be mounted with softdep option? I'm not an expert on that and hardly ever use softdep, but i'd say on file systems where file create/delete performance is *critically* important, performace has been clearly demonstrated to be insufficient without softdep, and you consider data loss harmless. Is there a way to prune metadata which will save me for problems like the one I encountered last night. I'm not convinced that's a good question to ask. Yours, Ingo
Re: What are the disadvantages of soft updates?
Ingo Schwarze schwa...@usta.de wrote: Hi Predrag, Hi Ingo, Predrag Punosevac wrote on Fri, Jan 23, 2015 at 03:24:00PM -0500: I was following this discussion with the great interest but without intend to participate in it until today. Namely one of my OpenBSD servers (5.6 sparc64) runs Mollify and last night I received an e-mail from an angry user who could not upload files (the upload will fail or upload the file with file size zero). After running df I noticed that /tmp was 100% full (4GB used) but the size of individual files was only 12Kb. That is unlikely to be due to softdep. The usual reason for a file system to be actually full and seemingly almost empty at the same time is somebody doing the following sequence of operations: Now it seems obvious that I made a mistake and put the blame on metadata instead of running fstat but at the moment I had a mindset of DF user. Namely couple of months ago I had a rogue process which keeps altering log files on one of DF machines. Long story short after couple of hours my /var had only about 1GB of log files but HAMMER history almost completely filled the rest of /var due to the frequent changes of log files. I regained the space by clearing HAMMER history and learned how to tune HAMMER retention parameters. It seems from what you are saying and from what seems clear to me before last night incident that I made a fundamental mistake of thinking of soft updates as honest journaling file system while in reality they are just a way of maintaining file system meta-data integrity. Hence softdep are safe from HAMMER like history retention problems. Most Kind Regards, Predrag P.S. Since you are developer I can't resist not to ask if anybody is looking at Bitrig code which brings WAPBL essentially into OpenBSD? - open(2) a file for writing - writing a lot of data until the file system is full - unlink(2) the file - keep the process running that open(2)'ed it - let that process keep the file open and *not* close(2) it Specifically, in /tmp, anybody can do that. I thought for a second and I remember seeing this with HAMMER on DF. The above works with almost any file system. Long story short I checked /etc/fstab and sure enough I had rw,softdep next to each partition including tmp. I removed softdep rebooted the sytem and /tmp usage dropped to 0%. That's not likely to be related to softdep either. Chances are just rebooting would have solved the issue as well - simply because rebooting terminates all running processes, and consequently closes all open files. What you should have done instead was run fstat(1), look for processes having files open in /tmp, use ls(1) -iRa /tmp to find the inode numbers of linked files in /tmp, and kill the processes having files open that were *not* linked until you found the one having the big file open - and then have a friendly talk with the responsible user, if any, or figure out what went wrong in case some daemon process caused the issue. My questions is which partitions should be mounted with softdep option? I'm not an expert on that and hardly ever use softdep, but i'd say on file systems where file create/delete performance is *critically* important, performace has been clearly demonstrated to be insufficient without softdep, and you consider data loss harmless. Is there a way to prune metadata which will save me for problems like the one I encountered last night. I'm not convinced that's a good question to ask. Yours, Ingo
Re: What are the disadvantages of soft updates?
Hi, Amit Kulkarni wrote on Fri, Jan 23, 2015 at 03:05:03PM -0600: One more point to add to Ingo's detailed and very helpful reply. Reboot actually clears /tmp. The reason i didn't mention that is that it definitely doesn't have anything to do with Predrag's problem, which was that /tmp was full even though it contained almost no filename entries. Rebooting clears /tmp in the sense that it deletes some filenames. That's irrelevant here because Predrag reports that there were no filenames pointing to large files in the first place. Yours, Ingo
Question about divert-to and divert-reply with pf.conf
Hi all, I have installed a proxy server in a DMZ and I need to redirect all http traffic from my internal lan to this proxy server in my openbsd firewall. Readming pf.conf manual and squid wiki, I see that this can be accomplished using divert-to and divert-reply in pf.conf. Configuration is like this?? pass in quick on inet proto tcp from 192.0.2.0/24 to port www divert-to 172.16.1.1 port 8080 pass out quick inet from 192.0.2.0/24 divert-reply Thanks.
Re: What are the disadvantages of soft updates?
On 1/22/2015 9:13 AM, Reyk Floeter wrote: What release and what virtualized SCSI controller where you using? I found my old notes, it turns out it was on 4.6 and the crash message was: softdep_setup_freeblocks: got error 5 while accessing filesystem dev = 0x404, block = 1315, fs = /var panic: ffs_blkfree: freeing free frag Stopped at Debugger+0x4: leave I have screenshots of trace and ps I can send if interested, but I'm sure a lot has been fixed in the interim. I can also attempt to reproduce on -current if you want, but even at its worst it'd stay up for a month or two before crashing.
support new
0 C Czech Republic P Prague T Prague Z 11000 O CORE ZONE s.r.o. I Gabriel Medon A Rybna 716/24 M i...@corezone.cz U http://corezone.cz/ B +420 607 577 677 X NONE N CORE ZONE s.r.o. is an IT outsourcing and consultancy company which provides installations, administration and support for systems based on FreeBSD, OpenBSD, Linux (CentOS, Debian, Ubuntu).
Re: What are the disadvantages of soft updates?
On 1/22/2015 9:13 AM, Reyk Floeter wrote: I've personally had problems with OpenBSD panics with softupdates when running under ESXi when the back-end storage becomes high-latency (aggressive SAN backups, not enough spindles). I haven't tried recently (it was difficult to repro on demand) but I didn't really consider it an OpenBSD issue. Presumably softupdate has shorter timeouts. What release and what virtualized SCSI controller where you using? Hm... probably 5.0, with LSI Logic Parallel.
Re: Hannover BSD meetup
On Fri, Jan 23, 2015, at 03:34 AM, Jan Klemkow wrote: On Thu, Jan 22, 2015 at 08:05:13PM +0100, Jan Lambertz wrote: Hey Reyk, that sounds great. Unfortunately the Way to Hannover is 600km from here. I hope something simliar is happening soon near Munich. I was not able to find any Meeting for OpenBSD here. Jan Hi Jan, I'm living in Munich and interested to meet OpenBSD folks, too. And I know some other OpenBSD people around Munich. If you organise a place with food and coke I will join! bye, Jan You, of course, mean Coca-Cola right? ;)
Re: iked and isakmpd in parallel
On 2015-01-20, Fedor Piecka teplav...@gmail.com wrote: We need to support both IKEv1 and IKEv2 peers in our environment. At this point, you need VMs or separate hardware to do this with OpenBSD. My first question now is how to instruct iked to listen only on a selected interface. The second question is whether the 2 of them wouldn't interfere with each other. There are ways around address binding, but they would still interfere.
Re: Hannover BSD meetup
On Thu, Jan 22, 2015 at 08:05:13PM +0100, Jan Lambertz wrote: Hey Reyk, that sounds great. Unfortunately the Way to Hannover is 600km from here. I hope something simliar is happening soon near Munich. I was not able to find any Meeting for OpenBSD here. Jan Hi Jan, I'm living in Munich and interested to meet OpenBSD folks, too. And I know some other OpenBSD people around Munich. If you organise a place with food and coke I will join! bye, Jan
Re: Hannover BSD meetup
I'm amazed about the feedback on twitter and misc; it will definitively happen. Thanks! And I'm sure that people in Munich can find others to have their own OpenBSD Haxn-und-Mass-Oktoberfest every now and then ;) Reyk On Thu, Jan 22, 2015 at 03:02:30PM +0100, Reyk Floeter wrote: Hi, we figured out that there are more BSD people in the Hannover area, Germany, which seems to be a good reason to meet and get beer. We're not quite a user group, but let's give it a try. We're a few developers and users, mostly from OpenBSD but the other ones are welcome. We don't have a mailing list; just contact me directly or poke me on twitter (@reykfloeter). Save the date: Thursday, February 19th, 19:30 at GiG Linden. Thanks, Reyk --
Re: OpenBSD talk at ScotLUG, Glasgow
On 2015-01-19 Mon 22:29 PM |, Fred wrote: If anyone is in Glasgow on Thursday evening - I'm giving a talk at the Scottish Linux User Group (http://scotlug.github.io/) on Building redundant and transparent firewalls with OpenBSD. Superb teaching Fred! He came backpacked with 5 OpenBSD devices networked, which the live streaming of internet music survived the furious battle-axe attack! Like the capital city of Edinburgh, the Glasgow group has free use of a hackerspace, with broadband projector. The BBC were live broadcasting from another room in the venue, but I reckon Fred's show would've gotten better ratings. After Fred's 2 dramatic networking demos, he performed an upgrade on an Alix firewall. Ace! -- Craig Skinner | http://twitter.com/Craig_Skinner | http://linkd.in/yGqkv7
