Re: a little help with ipsec
On Tue, 1 Dec 2015 23:49:37 + (UTC) Stuart Hendersonwrote: > Neither isakmpd nor iked tracks DNS changes. This is good to know, thank you for the information. > On the central side use "passive" not "dynamic". Remove the "peer > $gw_branche" to set this for the 'default peer' (i.e. to avoid > matching on IP address). > > Do you really need the first flow? It will simplify things if you can > restrict yourself to $lan_branche addresses and just have the second > flow. (Otherwise because you want to use the 'default peer' you'll > need to collapse these into a single rule with "to any"). Also very helpful. All the examples I found, including "AUTOMATIC KEYING" section of ipsec.conf, have flow between gateways configured. I tried without them first, but I couldn't make it work. Only later I discovered it was related to the firewall rule, but forgot to retry without gateway-to-gateway flow once I fixed it. > It might be easier to get the basic setup working with psk first, but > when you have that up and running, see the PUBLIC KEY AUTHENTICATION > section in isakmpd(8) and get that setup, it is pretty simple to use > and much safer than psk. That was the idea from the beginning, didn't want to complicate further before having basic setup working. Regards, -- Before enlightenment - chop wood, draw water. After enlightenment - chop wood, draw water. Marko Cupać https://www.mimar.rs/
Re: home keys in tmux
Do you have anything in your .tmux.conf? On Wed, Dec 2, 2015 at 6:42 AM, Ted Unangstwrote: > When i push home at a ksh prompt in xterm, the cursor goes to the > beginning of > the line. When i do the same in tmux, nothing happens. > > TERM in xterm is xterm. TERM in tmux is screen. > > How do i fix this? (Why do i need to fix it?)
Re: OpenBSD + pf + DPI
I don't search an all in one solution software for DPI, but asking if there is some software on base/ports to accomplish to this purpose and if someone had configured a solution with OBSD for DPI (personal experiences). My question is malformed, sorry. Il 02/12/2015 13:25, Romain FABBRI ha scritto: I don't understant your purpose What specific protocols would you like to inspect deeply ? Because the is no base/port complete solution that I am aware of. And the idea sounds crazy. Some vendors have filters/plugins/proxies that are application aware... And it's often disabled by admins because it's making the applications which doesn't comply strictly to fail -Message d'origine- De : owner-m...@openbsd.org [mailto:owner-m...@openbsd.org] De la part de Alessandro Baggi Envoyé : mercredi 2 décembre 2015 12:45 À : misc@openbsd.org Objet : OpenBSD + pf + DPI Hi list, I don't know how to start to make Deep Packet Inspection. My interest is OpenBSD and pf related. Anyone has already used on OpenBSD? It is possibile on OpenBSD with shipped (base/ports) software? Every tips are appreciated. Thanks in advance.
Re: OpenBSD 5.8 on VMware 5.5
2 décembre 2015 13:00 "Felipe Gomes"a écrit: > I just wanted to thank everyone for their feedback. Thanks a lot! > > You guys are amazing. > > Best regards, > Felipe Gomes > > On Wed, Dec 2, 2015 at 4:03 AM, Bruno Flueckiger wrote: > >> On 01.12.2015 16:50, Felipe Gomes wrote: >> >>> Folks, >>> >>> I've been trying to search for more information on OpenBSD as a VMWare >>> guest, but I wasn't able to find much... and the information is pretty >>> much >>> outdated. >>> >>> What are the recommendations for OpenBSD 5.8 (amd64) as a guest on VMware >>> 5.5? >>> >>> Guest Operating System: should I pick "Other (64bit)" or FreeBSD? >>> >>> How does OpenBSD work with "virtual sockets" and "cores per virtual >>> socket"? >>> >>> What is the best NIC? E1000, E1000E, VMXNET2 ENHANCED or VMXNET3? >>> >>> What is the recommended SCSI Controller? LSI Logic Parallel, LSI Logic SAS >>> or VMware Paravirtual? >>> >>> I'd believe that all of these options work... I just don't know which is >>> more stable or perform better. >>> >>> Any other tips on fine tunning or special setting? >>> >>> I'm planning on migrating a few Soekris boxes to virtual machines. Is this >>> reliable? Is anyone running production OpenBSD servers on VMware? >>> >>> Thanks in advance! >> >> I run a productive SMTP server with OpenBSD 5.8-stable on VMware 5.5 for >> some >> months and so far I didn't experience any problems. Guest OS is FreeBSD, >> NIC >> is VMXNET3 and the controller is LSI Logic Parallel. >> >> There are plans for more OpenBSD servers on VMware in the company I work >> for >> due to the small footprint of the OS and the very good experience we have >> so >> far. >> >> Cheers, >> Bruno Hi, works here like a charm, on prod with OpenBSD 5.8 amd64 : Guest OS is FreeBSD 64 NIC is VMXNET3 scsi controller is paravirtual multiple openbsd VMs on vmware since 3 years without any problems. Morgan
Re: OpenBSD + pf + DPI
Em 02-12-2015 12:56, Alessandro Baggi escreveu: > I don't search an all in one solution software for DPI, but asking if > there is some software on base/ports to accomplish to this purpose and > if someone had configured a solution with OBSD for DPI (personal > experiences). My question is malformed, sorry. Take a look at bro. It's on ports. Cheers, Giancarlo Razzolini
Re: home keys in tmux
Em 02-12-2015 10:42, Ted Unangst escreveu: > How do i fix this? (Why do i need to fix it?) Coincidentally, I saw that same question asked today on IRC and it wasn't even on OpenBSD. The OP changed it by setting TERM to xterm-256 if I'm not mistaken. And he also nailed it down to the fact that the num lock switch was on (or off). At first I thought it wasn't tmux related. But now it seems otherwise. Cheers, Giancarlo Razzolini
Re: pf, anchors, and macros
Em 02-12-2015 07:56, Sarevok Anchev escreveu: > .. but I don't think it's relevant as I've tried to run the test between > pf.conf and the base anchor, and still macros defined in pf.conf are not > available from /etc/pf/anchors/base. > > Is this intended behaviour? Macros need to be present in each anchor file. Tables don't need to. I have a little script that copies all my macros after I edit /etc/pf.conf to the anchors. I use commented marks on /etc/pf.con to know where to begin copying and where to end. But you get the point. Cheers, Giancarlo Razzolini
Re: A branded USB stick as an alternative to the CD set?
I have no clue what a hackathon costs, any ballpark averages? Sent from my BlackBerry 10 smartphone on the Verizon Wireless 4G LTE network. Original Message From: Theo de Raadt Sent: Wednesday, December 2, 2015 11:11 AM To: Donald Allen Cc: Theo de Raadt; misc; Richard Thornton Subject: Re: A branded USB stick as an alternative to the CD set? > But if we lose the project leader due to lack of exercise and food, > that's not good for the project. You made it very clear in a previous > message to this thread that no Foundation money comes to you. So while > the Foundation may be doing good things with their money, we, the > community, need to be sure that you have what you need. And in the > unlikely event that the freeloader factor decreases and we send you > more than you need, couldn't you turn the excess over to the > Foundation? Easily done by paying for a hackathon or two directly.
Re: A branded USB stick as an alternative to the CD set?
> But if we lose the project leader due to lack of exercise and food, > that's not good for the project. You made it very clear in a previous > message to this thread that no Foundation money comes to you. So while > the Foundation may be doing good things with their money, we, the > community, need to be sure that you have what you need. And in the > unlikely event that the freeloader factor decreases and we send you > more than you need, couldn't you turn the excess over to the > Foundation? Easily done by paying for a hackathon or two directly.
Upgrade 5.6->5.7->5.8 broke claws-mail's GPG agent tie in
I have recently upgraded my OpenBSD server from 5.6 to 5.8 (following the appropriate procedure and not skipping upgrades, of course). Unfortunately, when I finished the 2 hops from 5.6 to 5.8, I realized that claws-mail was no longer able to access the GPG key agent (I think that's what it uses, my apologies if I'm using the wrong terminology) and had switched to using some console based passphrase dialog. This wouldn't be a problem except my OBSD machine runs headless. Most of its duties require no interaction, but I also use it for my secure communications box... This includes claws-mail and sending digitally clearsigned messages. Everything was working fine @ 5.6, the connections tunneled properly to ask for the key phrase before signing the message... Now, though, even if I don't background the process and leave it in control of the shell, typing the pass phrase isn't working. Ideally I'd like to get things back to the point where I'm able to have that tunneled GPG key agent ask me for the phrase and like actually work again... I'd settle for having to run claws-mail in the foreground and typing in the passphrase on console, too, if I absolutely had to, but I don't know what's going on with the inability to authenticate for the key there... Any helpful suggestions, pointers, intuition, anything, would be of great assistance here. Please let me know if you need any more information or facts surrounding the matter that I can supply to make this more useful. Thank you for your time and consideration on this matter! -Damo
Re: home keys in tmux
Ax0n wrote: > Do you have anything in your .tmux.conf? No, don't have one. (i don't want one)
Re: OpenBSD 5.8 on VMware 5.5
On Wed, Dec 02, 2015 at 02:40:48PM -0200, Felipe Gomes wrote: > I'm kinda worried with the performance: the host is a Dell R815 4 CPU > Opteron 6136 / 64GB. There are no other VMs at the moment, just a single > stance of OpenBSD 5.8 installed, 4 virtual CPUs, 8 GB RAM. > > I've already enabled softdep on fstab, however it's taking more than 45 > minutes to compile the kernel (no modifications, GENERIC and GENERIC.MP > aswell, and no installation -- I'm just doing this to benchmark). > > Two hours ago I've started the make build and it seems its stalled on the > cleaning phase yet. > > I don't think this is right... > > If needed, I can provide dmesg or any other information related to this. > > Once again, thanks a lot. I'm not sure how much you've worked with VMware ESXi before but does this host have a battery backed RAID controller configured? If you're using a single SATA disk or even a RAID array without battery backed cache, all caching is disabled and all disk access will be extremely slow (I forget all the exact details now). This can be mitigated for the most part by using an SSD but the best performance is definitely with a battery backed RAID controller. I've used LSI SAS9261-8i with the appropriate BBU module very successfully with my own custom ESXi servers with excellent success. As far as OpenBSD on ESXi goes, I've never personally had any issues except for an issue five years ago where a NetApp filer would run some maintenance routines at 2am which caused the NFS-backed datastore to not respond briefly and that caused OpenBSD VMs to sense something was wrong at the filesystem level and panic. Just before I arrived on the scene, the change had been made for the ESXi hosts to connect to the NetApp filer using NFS rather than Fibre Channel and I think this change made a big difference but wasn't able to fully prove this out. Otherwise, I've used OpenBSD VMs on ESXi 4, 5, 5.5, and 6 without issue using direct attached storage (LSI SAS9261-8i with BBU and 4-8 drives in RAID 10) and also with SSDs temporarily in some cases. I'm happy for the more precise information in this thread regarding some of the VM settings but I've mostly used the defaults and also found that the vmt(4) driver tends to change my OpenBSD/amd64 VMs to show as "FreeBSD 32-bit" although I normally select "Other 64-bit" although this seems to have no functional change that I have observed. Bryan
Re: OpenBSD + pf + DPI
On Wed, Dec 02, 2015 at 01:35:10PM +0100, Patrik Lundin wrote: > On Wed, Dec 02, 2015 at 12:45:26PM +0100, Alessandro Baggi wrote: > > Hi list, > > I don't know how to start to make Deep Packet Inspection. My interest is > > OpenBSD and pf related. > > > > Anyone has already used on OpenBSD? It is possibile on OpenBSD with shipped > > (base/ports) software? > > > > Every tips are appreciated. > > > > You might want to read divert(4) which describes how to pass packets > from pf to a userland application and back. Yep, maybe a way to go would be divert -> some userland app like dnsfilter[1] but using ndpi code from ntop to just filter based on detected protocol. [1] http://sha256.net/dnsfilter/ j.
Re: A branded USB stick as an alternative to the CD set?
On Wed, Dec 2, 2015 at 11:11 AM, Theo de Raadtwrote: >> But if we lose the project leader due to lack of exercise and food, >> that's not good for the project. You made it very clear in a previous >> message to this thread that no Foundation money comes to you. So while >> the Foundation may be doing good things with their money, we, the >> community, need to be sure that you have what you need. And in the >> unlikely event that the freeloader factor decreases and we send you >> more than you need, couldn't you turn the excess over to the >> Foundation? > > Easily done by paying for a hackathon or two directly. Just to be sure I'm understanding you correctly, you are saying that you can handle the case "easily" where you have more than enough money for your needs by your paying for hackathons from the excess? If I've got that right, then it seems clear that when we sit down to thank you and your team by helping financially, the money ought to go to you. That might be hard for you to say, because of the obvious awkwardness, but I can say it. This seems clear, given declining CD revenues (which provide some support for you) and the Foundation providing no support for you. That doesn't sound sustainable, unless we pitch in with sufficient direct contributions to you.
Re: OpenBSD 5.8 on VMware 5.5
Folks, I'm kinda worried with the performance: the host is a Dell R815 4 CPU Opteron 6136 / 64GB. There are no other VMs at the moment, just a single stance of OpenBSD 5.8 installed, 4 virtual CPUs, 8 GB RAM. I've already enabled softdep on fstab, however it's taking more than 45 minutes to compile the kernel (no modifications, GENERIC and GENERIC.MP aswell, and no installation -- I'm just doing this to benchmark). Two hours ago I've started the make build and it seems its stalled on the cleaning phase yet. I don't think this is right... If needed, I can provide dmesg or any other information related to this. Once again, thanks a lot. On Wed, Dec 2, 2015 at 1:14 PM, Comètewrote: > 2 décembre 2015 13:00 "Felipe Gomes" a écrit: > > I just wanted to thank everyone for their feedback. Thanks a lot! > > > > You guys are amazing. > > > > Best regards, > > Felipe Gomes > > > > On Wed, Dec 2, 2015 at 4:03 AM, Bruno Flueckiger > wrote: > > > >> On 01.12.2015 16:50, Felipe Gomes wrote: > >> > >>> Folks, > >>> > >>> I've been trying to search for more information on OpenBSD as a VMWare > >>> guest, but I wasn't able to find much... and the information is pretty > >>> much > >>> outdated. > >>> > >>> What are the recommendations for OpenBSD 5.8 (amd64) as a guest on > VMware > >>> 5.5? > >>> > >>> Guest Operating System: should I pick "Other (64bit)" or FreeBSD? > >>> > >>> How does OpenBSD work with "virtual sockets" and "cores per virtual > >>> socket"? > >>> > >>> What is the best NIC? E1000, E1000E, VMXNET2 ENHANCED or VMXNET3? > >>> > >>> What is the recommended SCSI Controller? LSI Logic Parallel, LSI Logic > SAS > >>> or VMware Paravirtual? > >>> > >>> I'd believe that all of these options work... I just don't know which > is > >>> more stable or perform better. > >>> > >>> Any other tips on fine tunning or special setting? > >>> > >>> I'm planning on migrating a few Soekris boxes to virtual machines. Is > this > >>> reliable? Is anyone running production OpenBSD servers on VMware? > >>> > >>> Thanks in advance! > >> > >> I run a productive SMTP server with OpenBSD 5.8-stable on VMware 5.5 for > >> some > >> months and so far I didn't experience any problems. Guest OS is FreeBSD, > >> NIC > >> is VMXNET3 and the controller is LSI Logic Parallel. > >> > >> There are plans for more OpenBSD servers on VMware in the company I work > >> for > >> due to the small footprint of the OS and the very good experience we > have > >> so > >> far. > >> > >> Cheers, > >> Bruno > > Hi, > > works here like a charm, on prod with OpenBSD 5.8 amd64 : > > Guest OS is FreeBSD 64 > NIC is VMXNET3 > scsi controller is paravirtual > > multiple openbsd VMs on vmware since 3 years without any problems. > > Morgan
Re: A branded USB stick as an alternative to the CD set?
> I have no clue what a hackathon costs, any ballpark averages? http://www.openbsdfoundation.org/financials/2014/IncomeStatement2014.txt http://www.openbsdfoundation.org/financials/2013/IncomeStatement2013.txt These reports can be compared against http://www.openbsd.org/hackathons.html to find events not listed in the OpenBSD Foundation report. In that case someone else stepped up to cover the costs.
Re: home keys in tmux
Giancarlo Razzolini wrote: > Em 02-12-2015 10:42, Ted Unangst escreveu: > > How do i fix this? (Why do i need to fix it?) > Coincidentally, I saw that same question asked today on IRC and it > wasn't even on OpenBSD. The OP changed it by setting TERM to xterm-256 > if I'm not mistaken. And he also nailed it down to the fact that the num > lock switch was on (or off). At first I thought it wasn't tmux related. > But now it seems otherwise. fiddling with TERM should be unnecessary imo. in any case, setting TERM=xterm in tmux doesn't work. and fwiw, setting TERM=screen in xterm continues to work.
Re: A branded USB stick as an alternative to the CD set?
On 2015-12-02 16:21, Theo de Raadt wrote: I have no clue what a hackathon costs, any ballpark averages? http://www.openbsdfoundation.org/financials/2014/IncomeStatement2014.txt http://www.openbsdfoundation.org/financials/2013/IncomeStatement2013.txt These reports can be compared against http://www.openbsd.org/hackathons.html to find events not listed in the OpenBSD Foundation report. In that case someone else stepped up to cover the costs. Those 2014 figures look a lot healthier than the 2013 ones. Hopefully that keeps up, it's a good trajectory. :) + Justin
Re: relayd ssl interception and certificate subject
On 25. Nov 8:02:17, Stuart Henderson wrote: > On 2015-11-24, Uwe Werlerwrote: > > Hello, > > > > I'm just testing ssl interception and noticed the following problem. > > Sometimes the Subject/Subject Alternative Name of the cert is altered with > > a different name than the one the original cert has: > > When relayd connects to the server to find out what names to use in > the subject/SAN, it doesn't send the requested hostname (SNI) in > the ClientHello, so it only has the information from the server's > "default" certificate to include in the new generated certificate. > > You can see this for yourself with openssl s_client -connect hostname:443 > compared with openssl s_client -connect hostname:443 -servername hostname. > Hello Stuart, thanks! Ok, got it. Only for my understanding: is there a reason (probably security related?) for not using the host name from ClientHello in relayd for fetching the target cert? And if not - is it planned to implement it in relayd? Thanks in advance! Regards Uwe
Re: relayd ssl interception and certificate subject
On 2015/12/02 14:53, Uwe Werler wrote: > On 25. Nov 8:02:17, Stuart Henderson wrote: > > On 2015-11-24, Uwe Werlerwrote: > > > Hello, > > > > > > I'm just testing ssl interception and noticed the following problem. > > > Sometimes the Subject/Subject Alternative Name of the cert is altered > > > with a different name than the one the original cert has: > > > > When relayd connects to the server to find out what names to use in > > the subject/SAN, it doesn't send the requested hostname (SNI) in > > the ClientHello, so it only has the information from the server's > > "default" certificate to include in the new generated certificate. > > > > You can see this for yourself with openssl s_client -connect hostname:443 > > compared with openssl s_client -connect hostname:443 -servername hostname. > > > > Hello Stuart, > > thanks! Ok, got it. > > Only for my understanding: is there a reason (probably security related?) for > not using the host name from ClientHello in relayd for fetching the target > cert? > > And if not - is it planned to implement it in relayd? > > Thanks in advance! > > Regards Uwe AFAIK it's just not implemented yet, I don't see a security reason for not doing this. (if you need this now, squid can do it, but config is more complex - there's also sslsplit in ports but that's not really a normal proxy). Setting a hostname in an outgoing request is pretty simple (SSL_set_tlsext_host_name) but you need to get it from the request first and that's a little more complicated, afaik you need to setup a callback (with SSL_CTX_set_tlsext_servername_callback and SSL_CTX_set_tlsext_servername_arg) to point at a function which will do a SSL_get_tlsext_host_name call to fetch the hostname and store it for later use in the request.
Re: home keys in tmux
Ax0n wrote: Do you have anything in your .tmux.conf? Ha, I have a funny problem in tmux that thwarts me. I changed the prefix key to C-a but the sequence C-a C-a doesn't work like C-b C-b, the C-a doesn't ever seem to get sent to the shell. Which means I can't jump to head-of-line Emacs-style like I'm used to. Maybe I could figure this out with a hour of study but maybe somebody on the list knows ;) -- Jack J. Woehr # Science is more than a body of knowledge. It's a way of www.well.com/~jax # thinking, a way of skeptically interrogating the universe www.softwoehr.com # with a fine understanding of human fallibility. - Carl Sagan
Re: Failure to boot install media using bootia32.efi
On 2 December 2015 at 12:11, YASUOKA Masahikowrote: > On Tue, 1 Dec 2015 20:41:15 + > Callum Davies wrote: >> I have two "devices" using IA32 UEFI firmware with 64-bit >> hardware. An Asus EeeBook X502TA and qemu-system-x86_64 with >> an IA32 TianoCore firmware. Neither of these will boot from >> snapshots/amd64/install58.fs. >> >> Attempting to run bootia32.efi from the UEFI shell of the qemu system >> simply tells me "Command Error Status: Not Found". >> >> The EeeBook is deficient, and doesn't provide an UEFI shell, but I >> suspect it fails for the same reason. > > Fixed some issues on ia32 on the CVS tree. Can you try that by > replacing BOOTIA32.EFI in install58.fs? Happy to report that the EeeBook boots bsd.rd with these changes; thank you! Less happy to report that the keyboard is unresponsive after the kernel is booted. Here are dmesgs for a boot with the EHCI controller enabled,and one with the XHCI controller enabled. Boot stuff: probing pc0 mem[572K 56K 511M 1469M 2M] disk: hd0 hd1* hd2* hd3* >> OpenBSD/amd64 EFIBOOT 3.29 boot> cannot open hd0a:/etc/random.seed: No such file or directory booting hd0a:/5.8/amd64/bsd.rd: 3264684+1395280+2409472+0+569344 [72+365040+2380 95]=0x7de6c8 entry point at 0xf000160 [7205c766, 3404, 24448b12, 1680a304] ===EHCI===: kbc: cmd word write error Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All right reserved. Copyright (c) 1995-2015 OpenBSD. All right reserved. http://www.OpenBSD.org OpenBSD 5.8-current (RAMDISK_CD) #1529: Mon Nov 30 14:08:11 MST 2015 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/RAMDISK_CD RTC BIOS diagnostic error 3f real mem = 2863134720 (1967MB) avail mem = 1998950400 (1986MB) mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.7 @ 0x7c31a010 (16 entries) bios0: vendor American Megatrends Inc. version "X205TA.208" date 12/18/2014 bios0: ASUSTeK COMPUTER INC. X205TA acpi0 at bios0: rev 2, ACPI contl unavailable cpu0 at mainbus0: (uniprocessor) cpu0: Intel(R) Atom(TM) CPU Z3735F @ 1.33GHz, 1333.50 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCE,CMOV,PAT,PSE36,CF LUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX ,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,AES,RDRAND,NXE ,LONG,LAHF,3DNOWP,PERF,ITSC,SMEP,ERMS,SENSOR,ARAT cpu0: 1MB 64b/line 16-way L2 cache cpu0: mwait min=64 max=64 C-substates=0.2.0.0.0.0.3.3, IBE pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 "Intel Bay Trail Host" rev 0x0f "Intel Bay Trail Video" rev 0x0f at pci0 dev 2 function 0 not configured "Intel Bay Trail TXE" rev 0x0f at pci0 dev 26 function 0 not configured ehci0 at pci0 dev 29 function 0 vendor "Intel", unknown product 0x0f34 rev 0x0f: couldn't map interrupt "Intel Bay Trail LPC" rev 0x0f at pci0 dev 31 function 0 not configured isa0 at mainbus0 com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo efifb0 at mainbus0 wsdisplay0 at efifb0 mux 1: console (std, vt100 emulation) softraid0 at root scsibus0 at softraid0: 256 targets root on rd0a swap on rd0b dump on rd0b ===XHCI===: kbc: cmd word write error Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All right reserved. Copyright (c) 1995-2015 OpenBSD. All right reserved. http://www.OpenBSD.org OpenBSD 5.8-current (RAMDISK_CD) #1529: Mon Nov 30 14:08:11 MST 2015 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/RAMDISK_CD RTC BIOS diagnostic error 3f real mem = 2863134720 (1967MB) avail mem = 1998950400 (1986MB) mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.7 @ 0x7c31a010 (16 entries) bios0: vendor American Megatrends Inc. version "X205TA.208" date 12/18/2014 bios0: ASUSTeK COMPUTER INC. X205TA acpi0 at bios0: rev 2, ACPI contl unavailable cpu0 at mainbus0: (uniprocessor) cpu0: Intel(R) Atom(TM) CPU Z3735F @ 1.33GHz, 1333.58 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCE,CMOV,PAT,PSE36,CF LUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX ,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,AES,RDRAND,NXE ,LONG,LAHF,3DNOWP,PERF,ITSC,SMEP,ERMS,SENSOR,ARAT cpu0: 1MB 64b/line 16-way L2 cache cpu0: mwait min=64 max=64 C-substates=0.2.0.0.0.0.3.3, IBE pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 "Intel Bay Trail Host" rev 0x0f "Intel Bay Trail Video" rev 0x0f at pci0 dev 2 function 0 not configured xhci0 at pci0 dev 20 function 0 "Intel Bay Trail xHCI" rev 0x0f: couldn't map in terrupt "Intel Bay Trail TXE" rev 0x0f at pci0 dev 26 function 0 not configured "Intel Bay Trail LPC" rev 0x0f at pci0 dev 31 function 0 not configured isa0 at mainbus0 com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo efifb0 at mainbus0 wsdisplay0 at efifb0 mux 1: console (std, vt100 emulation) softraid0
Re: serious watchdog timeout issues with em driver
On 30.11.2015 14:08, Atanas Vladimirov wrote: Hi, I'm not sure if this is related to resent em(4) changes, but after upgrade from: Hi, Just ignore my previous assumptions. I thinks that I found the real cause for this upload speed problem. I'm using ifstated to inform me when something goes wrong with my egress interface. snip from ifstated.conf ... state extif_online { init { run "echo External interface ON-line @ `date +%H:%M:%S` | mail -s 'External Interface ON-line' t...@example.com" run "/usr/sbin/arp -Ff /etc/ether.mac" } if $em2_up && ! $peer_up { set-state extif_up } if $em2_down { set-state extif_down } } ... /snip [ns]~$ cat /etc/ether.mac 95.YY.XXX.225 64:87:88:58:b2:41 permanent ^^^ this is the ip of my default gateway If I have a permanent arp entry for my gateway, then I observe 1-2mbps upload speed. After I clear the arp I get 30-40mbps as it should be. Meanwhile I updated to more resent snapshot #1696: Wed Dec 2 10:13:03 MST 2015 and the problem persist. If you need more info just ask. Best regard, Atanas dmesg: OpenBSD 5.8-current (GENERIC.MP) #1696: Wed Dec 2 10:13:03 MST 2015 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 4269342720 (4071MB) avail mem = 4135833600 (3944MB) mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.6 @ 0x9f000 (70 entries) bios0: vendor American Megatrends Inc. version "1.2a" date 06/27/2012 bios0: Supermicro X8SIL acpi0 at bios0: rev 2 acpi0: sleep states S0 S1 S4 S5 acpi0: tables DSDT FACP APIC MCFG OEMB HPET GSCI SSDT EINJ BERT ERST HEST acpi0: wakeup devices P0P1(S4) P0P3(S4) P0P4(S4) P0P5(S4) P0P6(S4) BR1E(S4) PS2K(S4) PS2M(S4) USB0(S4) USB1(S4) USB2(S4) USB3(S4) USB4(S4) USB5(S4) USB6(S4) GBE_(S4) [...] acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Xeon(R) CPU X3430 @ 2.40GHz, 2400.38 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF,PERF,ITSC,SENSOR cpu0: 256KB 64b/line 8-way L2 cache cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges cpu0: apic clock running at 133MHz cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE cpu1 at mainbus0: apid 2 (application processor) cpu1: Intel(R) Xeon(R) CPU X3430 @ 2.40GHz, 2400.00 MHz cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF,PERF,ITSC,SENSOR cpu1: 256KB 64b/line 8-way L2 cache cpu1: smt 0, core 1, package 0 cpu2 at mainbus0: apid 4 (application processor) cpu2: Intel(R) Xeon(R) CPU X3430 @ 2.40GHz, 2400.00 MHz cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF,PERF,ITSC,SENSOR cpu2: 256KB 64b/line 8-way L2 cache cpu2: smt 0, core 2, package 0 cpu3 at mainbus0: apid 6 (application processor) cpu3: Intel(R) Xeon(R) CPU X3430 @ 2.40GHz, 2400.00 MHz cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF,PERF,ITSC,SENSOR cpu3: 256KB 64b/line 8-way L2 cache cpu3: smt 0, core 3, package 0 ioapic0 at mainbus0: apid 7 pa 0xfec0, version 20, 24 pins ioapic0: misconfigured as apic 1, remapped to apid 7 acpimcfg0 at acpi0 addr 0xe000, bus 0-255 acpihpet0 at acpi0: 14318179 Hz acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus -1 (P0P1) acpiprt2 at acpi0: bus 1 (P0P3) acpiprt3 at acpi0: bus 2 (P0P5) acpiprt4 at acpi0: bus -1 (P0P6) acpiprt5 at acpi0: bus 6 (BR1E) acpiprt6 at acpi0: bus 3 (BR20) acpiprt7 at acpi0: bus 4 (BR24) acpiprt8 at acpi0: bus 5 (BR25) acpicpu0 at acpi0: !C3(350@17 mwait.1@0x20), !C2(500@17 mwait.1@0x10), C1(1000@1 mwait.1), PSS acpicpu1 at acpi0: !C3(350@17 mwait.1@0x20), !C2(500@17 mwait.1@0x10), C1(1000@1 mwait.1), PSS acpicpu2 at acpi0: !C3(350@17 mwait.1@0x20), !C2(500@17 mwait.1@0x10), C1(1000@1 mwait.1), PSS acpicpu3 at acpi0: !C3(350@17 mwait.1@0x20), !C2(500@17 mwait.1@0x10), C1(1000@1 mwait.1), PSS acpibtn0 at acpi0: SLPB acpibtn1 at acpi0: PWRB ipmi at mainbus0 not configured cpu0: Enhanced SpeedStep 2400 MHz: speeds: 2401, 2400, 2267, 2133, 2000, 1867, 1733, 1600, 1467, 1333, 1200 MHz pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 "Intel Core DMI" rev 0x11 ppb0 at pci0 dev 3 function 0 "Intel Core PCIE" rev 0x11: msi pci1 at ppb0 bus 1 ppb1 at pci0 dev 5 function 0 "Intel Core
Re: home keys in tmux
Johan Mellberg wrote: Anyway, screen steals C-a so to jump to the start of a line, hit C-a, then a again. Doesn't work :( -- Jack J. Woehr # Science is more than a body of knowledge. It's a way of www.well.com/~jax # thinking, a way of skeptically interrogating the universe www.softwoehr.com # with a fine understanding of human fallibility. - Carl Sagan
Re: python uwsgi port/package
On 2015-12-02, Christopher Sean Hiltonwrote: > Hi, > > I'm looking for a uwsgi port for use with nginx and django. Searching > the ports collection I don't find anything. I'd like to know if that's > not done because no one has needed it yet or because of some security > implication that I don't know about. > > I'd prefer a port since I don't want to use two packaging systems, pkg > and pip. If I build a port I'd also eventually add a rc-script since > under the uwsgi model of the web the backend web process gets started > seperately. > > I can take an existing python port and create something which I would > gladly share with the project. But if uwsgi is excluding because of > security issues then building a port would be silly. > > Thanks for any information, I made a start at a port, I was going to use it for something but it didn't happen in the end so I left it in openbsd-wip in case anyone wants to pick it up. The basics are there (though may need updating) and IIRC it did work, it'll want a bit of polish though - rc script, probably its own uid/gid, maybe a readme etc.
Re: python uwsgi port/package
Node.js modules have been removed also in favor of npm. I highly recommend virtualenv and pip to keep your system cleaner if not every other reason (package versions, incompatibilities, etc). Keep Python packages away from your system and into their own environment. On Dec 2, 2015 6:58 PM, "Christopher Sean Hilton"wrote: > Hi, > > I'm looking for a uwsgi port for use with nginx and django. Searching > the ports collection I don't find anything. I'd like to know if that's > not done because no one has needed it yet or because of some security > implication that I don't know about. > > I'd prefer a port since I don't want to use two packaging systems, pkg > and pip. If I build a port I'd also eventually add a rc-script since > under the uwsgi model of the web the backend web process gets started > seperately. > > I can take an existing python port and create something which I would > gladly share with the project. But if uwsgi is excluding because of > security issues then building a port would be silly. > > Thanks for any information, > -- > Chris > > __o "All I was trying to do was get home from work." > _`\<,_ -Rosa Parks > ___(*)/_(*).___o..___..o...ooO..._ > Christopher Sean Hilton[chris/at/vindaloo/dot/com]
Re: pf, anchors, and macros
On Wed, Dec 02, 2015 at 01:37:52PM -0200, Giancarlo Razzolini wrote: > Macros need to be present in each anchor file. Tables don't need to. I > have a little script that copies all my macros after I edit /etc/pf.conf > to the anchors. I use commented marks on /etc/pf.con to know where to > begin copying and where to end. But you get the point. > I think it's always been this way. This may have changed but if you specify filter conditions in your anchor definition the screening you get is combination of the screen on the anchor from the base pf.conf file and the filters specified in the anchor file itself. I use anchors on FreeBSD which is using an older version of pf but I got around the issue this way: --- /etc/pf.conf --- ... anchor imapd in on $ext_if from any to ($ext_if) load anchor imapd from "/etc/pf-anchor-home/imapd.conf" ... --- /etc/pf-anchor-home/imapd.conf - imapd_ports="{ 143, 993 }" pass in proto tcp to any port $imapd_ports keep state That's a simple example. It would honestly be better without the anchor since using the anchor divides the rule up into two places. I do it that way because I can easily split firewalling up across two puppet rules. As Giancarlo wrote, the anchor can use your tables. He didn't mention that the anchor can define it's own macros. The net result of this is: pass in on $ext_if \ from any \ to ($ext_if) port { 143, 993 } \ keep state -- Chris __o "All I was trying to do was get home from work." _`\<,_ -Rosa Parks ___(*)/_(*).___o..___..o...ooO..._ Christopher Sean Hilton[chris/at/vindaloo/dot/com]
Re: home keys in tmux
Spending a little time with 'cat -v', I ended up with the following non-.tmux.conf approach to making home/end happy in tmux with an otherwise-unmodified ksh shell: bind '^[[1~'=beginning-of-line bind '^[[4~'=end-of-line It doesn't appear to break normal xterm[-256color] use. These are still workarounds, of course. Brian Conway On Wed, Dec 2, 2015 at 10:17 AM, Ted Unangstwrote: > Ax0n wrote: >> Do you have anything in your .tmux.conf? > > No, don't have one. (i don't want one)
Re: home keys in tmux
Philip Guenther wrote: My crystal ball says that you changed the prefix but didn't change the binding of 'a'. I would verify my crystal ball against your config...but you didn't show your config... I only made the change I noted, and thank you for some helpful advice! -- Jack J. Woehr # Science is more than a body of knowledge. It's a way of www.well.com/~jax # thinking, a way of skeptically interrogating the universe www.softwoehr.com # with a fine understanding of human fallibility. - Carl Sagan
Re: python uwsgi port/package
If you have multiple apps in production with different versions of packages that break compatibility then you'll be in a world of pain. You also have supervisor to make it rc-able. On Dec 2, 2015 7:52 PM, "Christopher Sean Hilton"wrote: > On Wed, Dec 02, 2015 at 07:19:25PM +, Pedro Tender wrote: > > Node.js modules have been removed also in favor of npm. > > I highly recommend virtualenv and pip to keep your system cleaner if not > > every other reason (package versions, incompatibilities, etc). > > Keep Python packages away from your system and into their own > environment. > > While I love pip and virtualenv in development, I don't understand the > advantage they offer over the system package manager on a production > machine. In addition, I feel that a reasonable uwsgi package would > include an rc-script to start your app automatically at system boot > time. [1] Combine all of this with puppet, git and some git-hook magic for > your custom bits and you end up with an easily managed system. > > There's no doubt that all of this could be hand hacked but the way I > see it the less hand hacking on production machines, the better. It > might just be my style, but I feel that the less work I have to do on > a production system from the command line, the more reliable that > system will be. > > > > [1] As an aside, my efforts might be better spent adding an rc script > to the current gunicorn package. But, if I'm correct uwsgi is written > in C so I expect it to be a little more performant. My project is > going to run on a Soekris Net5501 at the end of the day and the whole > reason I'm going here is because apache/mod_wsgi has horrible first > time startup costs serving django applications and tuning it is a bear. > > -- > Chris > > __o "All I was trying to do was get home from work." > _`\<,_ -Rosa Parks > ___(*)/_(*).___o..___..o...ooO..._ > Christopher Sean Hilton[chris/at/vindaloo/dot/com]
Re: home keys in tmux
On Wed, Dec 2, 2015 at 9:43 AM, Jack J. Woehrwrote: > Ha, I have a funny problem in tmux that thwarts me. I changed the prefix key > to C-a but the sequence C-a C-a doesn't work like C-b C-b, > the C-a doesn't ever seem to get sent to the shell. Which means I can't jump > to head-of-line Emacs-style like I'm used to. Maybe I could > figure this out with a hour of study but maybe somebody on the list knows ;) My crystal ball says that you changed the prefix but didn't change the binding of 'a'. I would verify my crystal ball against your config...but you didn't show your config...
Re: python uwsgi port/package
On Wed, Dec 02, 2015 at 07:19:25PM +, Pedro Tender wrote: > Node.js modules have been removed also in favor of npm. > I highly recommend virtualenv and pip to keep your system cleaner if not > every other reason (package versions, incompatibilities, etc). > Keep Python packages away from your system and into their own environment. While I love pip and virtualenv in development, I don't understand the advantage they offer over the system package manager on a production machine. In addition, I feel that a reasonable uwsgi package would include an rc-script to start your app automatically at system boot time. [1] Combine all of this with puppet, git and some git-hook magic for your custom bits and you end up with an easily managed system. There's no doubt that all of this could be hand hacked but the way I see it the less hand hacking on production machines, the better. It might just be my style, but I feel that the less work I have to do on a production system from the command line, the more reliable that system will be. [1] As an aside, my efforts might be better spent adding an rc script to the current gunicorn package. But, if I'm correct uwsgi is written in C so I expect it to be a little more performant. My project is going to run on a Soekris Net5501 at the end of the day and the whole reason I'm going here is because apache/mod_wsgi has horrible first time startup costs serving django applications and tuning it is a bear. -- Chris __o "All I was trying to do was get home from work." _`\<,_ -Rosa Parks ___(*)/_(*).___o..___..o...ooO..._ Christopher Sean Hilton[chris/at/vindaloo/dot/com]
Re: home keys in tmux
We'll see if this gets to the list, sending from a phone. Anyway, screen steals C-a so to jump to the start of a line, hit C-a, then a again. Might work for you. > 2 dec. 2015 kl. 18:43 skrev Jack J. Woehr: > > Ax0n wrote: >> Do you have anything in your .tmux.conf? >> > Ha, I have a funny problem in tmux that thwarts me. I changed the prefix key to C-a but the sequence C-a C-a doesn't work like C-b C-b, > the C-a doesn't ever seem to get sent to the shell. Which means I can't jump to head-of-line Emacs-style like I'm used to. Maybe I could > figure this out with a hour of study but maybe somebody on the list knows ;) > > -- > Jack J. Woehr # Science is more than a body of knowledge. It's a way of > www.well.com/~jax # thinking, a way of skeptically interrogating the universe > www.softwoehr.com # with a fine understanding of human fallibility. - Carl Sagan
python uwsgi port/package
Hi, I'm looking for a uwsgi port for use with nginx and django. Searching the ports collection I don't find anything. I'd like to know if that's not done because no one has needed it yet or because of some security implication that I don't know about. I'd prefer a port since I don't want to use two packaging systems, pkg and pip. If I build a port I'd also eventually add a rc-script since under the uwsgi model of the web the backend web process gets started seperately. I can take an existing python port and create something which I would gladly share with the project. But if uwsgi is excluding because of security issues then building a port would be silly. Thanks for any information, -- Chris __o "All I was trying to do was get home from work." _`\<,_ -Rosa Parks ___(*)/_(*).___o..___..o...ooO..._ Christopher Sean Hilton[chris/at/vindaloo/dot/com]
Re: python uwsgi port/package
Everything boils down to whether you'd like to run more than one app on your box. > While I love pip and virtualenv in development, I don't understand the > advantage they offer over the system package manager on a production > machine. Easy: whenever you can't be bothered with proper containers. App X requires package foo version 1.2, app Y requires foo version 1.4. Docker solves this universally. You can also achieve a similar effect by building a chroot. virtualenv's advantage is it doesn't require root, and is (subjectively) easier to use. Also dev = staging = live. Every difference between the environments is a potential bug, ready to blow up in your face the moment you hit the deploy button. > In addition, I feel that a reasonable uwsgi package would include an > rc-script to start your app automatically at system boot time. I prefer to run my application servers with runit. Traditional RC scripts usually assume one package = one application instance. Usually that's a sane assumption (what would be your reason for running two instances of Apache?) but again, if you can't be bothered with containers, virtualenv+runit make it easy to just put app X in /home/x, app Y in /home/y, then run two uwsgi's. > There's no doubt that all of this could be hand hacked but the way I > see it the less hand hacking on production machines, the better. It > might just be my style, but I feel that the less work I have to do on > a production system from the command line, the more reliable that > system will be. You've mentioned Puppet. Also check out Ansible. K.
getting money to deraadt@ [Was: Re: A branded USB stick as an alternative to the CD set?]
dera...@openbsd.org (Theo de Raadt), 2015.12.02 (Wed) 02:18 (CET): > >I don't think that quite covers it. Those of us who have the choice > >can send checks or Paypal money directly to Theo, as described on the > >Donations page. I think checks are preferable, because they eliminate > >Paypal skimming its credit-card-like fees, at the cost of a stamp. The > >CDs also involve paying a middle-man. > > Completely true. Also it is a 20 minute walk each way to the bank, > and keyboard folk need to do more walks. Theo, sending money to the account below gets it to you directly, right? (though it undermines your walking training, sorry ;-) from donations.html to bank-donation.html, haven't seen this mentioned in the thread yet: >From Inside Europe (SEPA): IBAN: DE91 7007 0024 0338 1779 00 BIC: DEUTDEDBMUC Name: Theo de Raadt Address: Deutsche Bank, Marienplatz 21 80331 M?nchen, Germany >From outside Europe: SWIFT: DEUTDEDBMUC Account: 7007 0024 0338 1779 00 Name: Theo de Raadt Address: Deutsche Bank, Marienplatz 21 80331 M?nchen, Germany Bye, Marcus > !DSPAM:565e47d1289635898012334!
Re: Any news on Merkle tree-hash-based whole-disk checksums (=ZFS-style checksums) in softraid?
On Tue, Dec 1, 2015 at 11:21 PM, Tinkerwrote: > So your current solution is *NOT* data-safe toward "mis-write":s and other > write errors that go unnoticed at write time. yes, if write error is silent, drive do not know about it nor it signals to OS, then yes, this will go unnoticed. If drive signals error, then this is handled by general softraid framework by off-lining the drive. > While I agree that the probability that the writes to both disks and to > their checksum areas would fail are really low, the "hash tree"/"100% hash" > way of ZFS must be said to be a big enabler because it's an integrity > preservation/data safety scheme of a completely other, higher level: :-) I really like ZFS, but you still kind of mix what can be done on the fs level and what can be done on block-device level. > The "checksum area" for the whole tree could be located right at the end of > the disk too, meaning that the "backward compatibility" you describe would > be preserved too. > > You are right that Fletcher is just another hash function with the standard > definition i.e. hash(data) => hashvalue - > > ZFS' magic ingredient is a Merkle tree of hashes that's all. > > > The benefit I see with a hash tree is that you have in RAM always stored a > hash of the whole disk (and the first level hashes in the hash tree). IMHO not in case of ZFS. i.e. not in case of yet unread data from ZFS. > This means that protection against serious transparent write > errors/mis-write:s goes from none (although implausible) to really solid. How? I don't see engineering way how would ZFS detect mis-writes which even drive does not know at all. Yes, ZFS will detect it during the scrub time (read) or when you access data and read them. But that's what RAID1C will do too. So where is the difference? (you know I told you you need scrub for this and that it's also on my todo list) > I see that the hash-tree could be implemented in a really simple, > straightforward way: > > What about you'd introduce an "über-hash", and then a fixed size of > "first-level hashes". > > The über-hash is a hash of all the first-level hashes, and the first-level > hashes respectively are a hash of their corresponding set of bottom level > checksums. so let's consider write. RAID1C does it as read chksum, write data, write chksum. If your u"ber hash or first level hashes are smaller than drive block, then I would probably need to store more than one into one block (to save space) and this would mean that on write I will do: read uber hash, read first-level hash, read chksum, write data, write chksum, write first level hash, write uber hash. Hence 1 IO -> 7 IOs while in case of RAID1C just 1 IO -> 3 IOs. > If for performance you need more levels then so be it, in all cases it can > be contained right at the end of the disk. > > The benefit here is that the über-hash and first level always will be kept > in RAM. OK, so you cache those, but still you need to write them -- to preserve data integrity, right? Then 1 IO -> 4 IOs still. > This means that as soon as any data or bottom-level checksums go out > of the disk cache and later on are read from the physical disk, then the > checking of all that data with the RAM-stored hashes, will give us the > precious absolute fread() guarantee. "precious obsolute" -- not at all! Read something about hash collisions and you will lose your hope. :-) > (Integrity between reboots will be a slightly more sensitive point. Maybe > some sysctl could be used to extract the über-hash so you could doublecheck > it after reboot.) I've been also thinking about async chksum write as an option with possible chksum merges over several real IOs.. The problem is that this means data on drive are not well-formed for the case of sudden crash or power-outage. Yes, it may be the option, but still priority should be IMHO be consistent (data-wise). > * Really just a hashtree-based checksummed passthrough discipline would > make all sense, e.g. JBOD .. or RAID 0. > >RAID 1 is nice but if you have many nodes and you just want Absolute > fread() integrity on a single machine, hashtree-checksummed passthrough or > JBOD or RAID 0 might be a preferable "lean and mean" solution. > >In an environment where you have perfect backups, RAID 1's benefit over > passthrough is that disk degradation happens slightly more gracefully - > instead of watching for broken file access and halting immediately then, > then, as administrator you monitor those sysctl:s you introduce, that tell > if either underlying disk is broken. I must admit that indeed that's pretty > neat :) > >..But still it could always happen that both disks break at the same > time, so also still the passhtorugh usecase is really relevant also. ^ this looks like a set of your own wishes? > * Do you do any load balancing of read operations to the underlying RAID:s, > like, round robin? It's like SR-RAID1! So yes. > * About the checksum
Re: ansible openbsd_rcctl module
On Tue, Dec 01, 2015 at 08:54:25AM -, Sarevok Anchev wrote: > Hello, > > Recently I submitted openbsd_rcctl to ansible. In order to speed up the > process of having it included by default, I'm asking the community to > review/test the module and drop a comment at > https://github.com/ansible/ansible-modules-extras/pull/1296 > > Let me know if there are other OpenBSD-specific modules you'd like to see > for ansible. > I have added a comment to your pull request regarding the rcctl support already existing in the main service module. -- Patrik Lundin
Re: A branded USB stick as an alternative to the CD set?
I don't think it should be the contributors who should choose directly to who the money is "attributed". That being exactly the role of the foundation, in my opinion. But maybe : - the foundation could take into account the time given by each dev - the devs could have their own votes in regards to the importance of each others work (but not their own) - contributors could "vote" for the efforts they appreciate most (the choicelist is going to be hard to produce)... These opinions/facts *could* be taken into account by the foundation... But it might already be the case ? PS : >From now I'll stop buying the CD and convert this money into a little bigger donation :) Many thanks to Theo, and all the devs for their excellent work. -Message d'origine- De : owner-m...@openbsd.org [mailto:owner-m...@openbsd.org] De la part de Michel Behr Envoyé : mardi 1 décembre 2015 21:41 À : Theo de RaadtCc : Kevin Chadwick ; OpenBSD ; fundrais...@openbsdfoundation.org Objet : Re: A branded USB stick as an alternative to the CD set? Just one more thing: for non-developers, if you think there's any sense in this idea I just described, please, some "seconding" and/or additions would be welcomed. Also some e-mails directed to fundrais...@openbsdfoundation.org would be great in this regard too. (Again: OpenBSD developers should *NOT* need to get involved in this discussion, this is between non-developers and the OpenBSD foundation). On Tue, Dec 1, 2015 at 6:18 PM, Michel Behr wrote: > As I understand, one of the reasons for the Foundation to avoid > targeted contributions is to preserve the independence of the project > - in the current model they are accountable for allocating the > resources as they see fit. So IMHO there is value in that model for > that regard. On the other hand, the fact that none of the donations is > directed specifically to compensate for the hard work of the > developers (and more specifically > Theo's) gives the foundation the prerogative to, for example, have at > the donation page one donation account separate, specific for > developers, with a clear message that those resources would go > directly to the developers (or to one developer...), in contrast with > the "standard" donation channel, which funds only events, > infrastructure, etc. It would a reasonable exception. I think if this > done with the same transparency things have being managed so far, there's no problem. > > And by the way, this suggestion is mine, not Theo's (and I'm far from > being a developer!), so I'm cc'ing the foundations' e-mail address - I > see this as matter of interest to the foundation because it touches > directly their purpose of providing the administrative support for the > project to keep it moving forward - e.g. providing a channel through > the donations page for developers to receive direct contributions > would permit them the flexibility to dedicate even more time to the > project. It would also be one more "communication channel" for > recognition to the developers high-quality code that's been produced over the years. > > Anyway, just my $0.02... (I think this is a matter that's between the > non-developers community and the OpenBSD Foundation, Theo and the > other OpenBSD developers should not need to get involved on this discussion). > > Kind regards, > > On Tue, Dec 1, 2015 at 4:35 PM, Theo de Raadt > > wrote: > >> > > > Now to be clear Theo, are donation via the paypal on the >> > > > donations >> page >> > > > are directly to you and you can do as you see fit, and/or only >> checks >> > > > would be best? >> > > >> > > Correct, as I see fit. I try to use it for the Project for >> > > things the Foundation doesn't fund. I declared it that way on >> > > the web site. I have not used it much for my own needs. >> > >> > I'd guess this has been thought of and just throwing in lame ideas >> > on the off chance it's of any use and maybe it's just extra site >> > coding work and there are legal complications, if not then are the >> > people in charge of the foundation website/operation privvy to this >> > list? Is Bob part of that? >> > >> > >> > I wonder if it would gain any traction if there was a separate >> > donation box and cheque address with a statement along the lines of >> > The OpenBSD project leader works full time and receives no support >> > from donations to the foundation. If you would like to also support >> > The project leader directly then you can do so here or by sending a cheque to. >> > >> > ___Made up example, Don't send here >> > >> > Theo De Raadt >> > The OpenBSD project leader >> > 8101 160 Street >> > Edmonton, Alberta, Canada >> > T5R 2G9 >> > ___ >> > >> > Alternatively but perhaps more complex behind the scenes?.. a >> > percentage box so everytime someone
Re: disklabel suggestion
> I'm trying to make several changes to my disklabel at once. If I try > to do it with -R to read in a file I get disklabel: ioctl DIOCWDINFO: > Open partition would move or shrink You are attempting to change the position or size of a mounted partition. You can't do that. The filesystem will attempt to write out to the disk and scribble somewhere unhealthy. The kernel therefore refuses to perform the action.
Re: Upgrade 5.6->5.7->5.8 broke claws-mail's GPG agent tie in
On Wed, 2 Dec 2015 07:58:26 -0800, Damon Getsmanwrote: > I realized > that claws-mail was no longer able to access the GPG key agent (I > think that's what it uses, my apologies if I'm using the wrong > terminology) and had switched to using some console based passphrase > dialog. Hi, First of all, you should address this kind of email to ports@ while CC'ing the maintainer (which is me for claws so it's not needed for now). :) For your problem, claws-mail uses gpgme which itself use any gnupg when it runs but it needs gnupg2 to build. The dialog window for the passphrase (even if empty) AFAIK comes with gnupg2. So you should try to have gnupg on your system but not gnupg2 and it should work. I already had this problem and I solved it as I said. Cheers, Daniel
Re: python uwsgi port/package
On Wed, Dec 02, 2015 at 07:54:48PM +, Pedro Tender wrote: > If you have multiple apps in production with different versions of packages > that break compatibility then you'll be in a world of pain. I do see that advantage. > You also have supervisor to make it rc-able. pip/virtualenv includes a supervisor or I have to write a script that sets up virtualenv for startup and launches the app. -- Chris [demime 1.01d removed an attachment of type application/pgp-signature]
Re: python uwsgi port/package
On Wed, Dec 02, 2015 at 09:19:25PM +, Pedro Tender wrote: >You have a port http://ports.su/sysutils/supervisor > Thanks for the tip, that's exactly what I'm looking for!! I also wanted to say thanks for the input. I understand what you are saying and when I run into version incompatiblity issues I usually run to: Create a user that does this app, Create an environment for the app to run it. It's just not where I'd like to be by default. Thanks again for the tips! -- Chris [demime 1.01d removed an attachment of type application/pgp-signature]
Re: disklabel suggestion
I understand what it's saying but I can't figure out which one it's complaining about. All I have mounted is: freebie# mount /dev/wd0a on / type ffs (local) /dev/wd0i on /win_c type msdos (local) /dev/wd0l on /win_d type msdos (local) /dev/wd0m on /win_e type msdos (local) /dev/wd0n on /usr type ffs (local, nodev) fusefs on /root/.gvfs type fuse (local) freebie# I'm trying to go from: # /dev/rwd0c: type: ESDI disk: ESDI/IDE disk label: ST31000340AS duid: f1f9d8681047d339 flags: bytes/sector: 512 sectors/track: 63 tracks/cylinder: 255 sectors/cylinder: 16065 cylinders: 121601 total sectors: 1953525168 boundstart: 65545200 boundend: 196619535 drivedata: 0 16 partitions: #size offset fstype [fsize bsize cpg] a: 41945712 65545200 4.2BSD 2048 163841 # / b: 2088453107490912swap # none c: 19535251680 unused d: 87040128109579392 4.2BSD 2048 163841 i: 65545137 63 MSDOS # /win_c j:32130196619535 ext2fs # /grubpart k: 10249407262148733 unknown # none l: 65545137272398203 MSDOS # /win_d m: 65545137337943403 MSDOS # /win_e n: 1023999102403488603 4.2BSD 2048 163841 # /usr o:526032297 1427487768 ext2fs to (boundend is wrong here, it might as well be the whole drive) # /dev/rwd0c: type: ESDI disk: ESDI/IDE disk label: ST31000340AS duid: f1f9d8681047d339 flags: bytes/sector: 512 sectors/track: 63 tracks/cylinder: 255 sectors/cylinder: 16065 cylinders: 121601 total sectors: 1953525168 boundstart: 65545200 #boundend: 196619535 boundend: 1427487704 drivedata: 0 16 partitions: #size offset fstype [fsize bsize cpg] a: 41945712 65545200 4.2BSD 2048 163841 # / b: 2088453107490912swap # none c: 19535251680 unused # d: 87040128109579392 4.2BSD 2048 163841 e: 4096196671488 ext2fs # /debroot f: 24515127237633543 ext2fs # /archroot g:32130196619535 ext2fs # /grubpart i: 65545137 63 MSDOS # /win_c j: 10244096262152192swap # shared swap # k: 10249407262148733 unknown # none (?) moved # k was my /usr as installed, I moved the files to n l: 65545137272398203 MSDOS # /win_d m: 65545137337943403 MSDOS # /win_e n: 1023999102403488603 4.2BSD 2048 163841 # /usr o:327677742 1427487768 ext2fs # /debdata p:184313856 1769209856 ext2fs # /archdata I did some rearranging in gparted but I was careful to not move anything except empty partitions. Everything except the little grub partition was empty. o doesn't exist anymore, there was never anything in there, but it's not mounted either. On 12/2/15, Theo de Raadtwrote: >> I'm trying to make several changes to my disklabel at once. If I try >> to do it with -R to read in a file I get disklabel: ioctl DIOCWDINFO: >> Open partition would move or shrink > > You are attempting to change the position or size of a mounted partition. > You can't do that. The filesystem will attempt to write out to the disk > and scribble somewhere unhealthy. The kernel therefore refuses to > perform the action. > -- Credit is the root of all evil. - AB1JX
Re: python uwsgi port/package
You have a port http://ports.su/sysutils/supervisor On Dec 2, 2015 8:54 PM, "Christopher Sean Hilton"wrote: > On Wed, Dec 02, 2015 at 07:54:48PM +, Pedro Tender wrote: > > If you have multiple apps in production with different versions of > packages > > that break compatibility then you'll be in a world of pain. > > I do see that advantage. > > > You also have supervisor to make it rc-able. > > pip/virtualenv includes a supervisor or I have to write a script that > sets up virtualenv for startup and launches the app. > > -- Chris
Re: Any news on Merkle tree-hash-based whole-disk checksums (=ZFS-style checksums) in softraid?
Karel, the most important thing at the bottom of the email :) On 2015-12-02 19:10, Karel Gardas wrote: To answer your question: In that case, as soon as that invalid data would actually be read from disk, it would be caught by the checksums that are guaranteed to be kept in RAM, so that is, the first-level checkums (or the über-checksum) match would fail. Ah, ok, but then this is safe only as long as you do not switch machine off. So in a catastrophic scenario like you describe where all writes fails to all drives, you switch machine off in a hope everything's all right and when you switch it on again your read should probably return old data. Right? First, that is an extremely high data security standard. Second, by the total hash could be checked before and after reboot. Third, the very much more likely time for a disk to break would be either during ordinary operation (disk made a mis-write, and you detect that when ), and very much secondarily while being turned off. I think problems with flushing data exactly at the time when turning off a disk would be extremely rare. Btw, any checksum algorithm would work for implementing a tree like this by the way, even CRC64 I guess. So Fletcher as such is out the window. I intend to followup on your other emails in some hours. You do not realise how expensive this (I mean whole this tree chksumming business) is on fsync. Single proof of this is database benchmark. I'm using pgbench. Try to run that and see for yourself, but if you believe me and trust my numbers then on OpenBSD 1 client bench setup I'm able to get ~1190 tps on RAID1, ~950 tps on RAID1C and guess what, just ~100 tps on ZFS on Solaris 11.3. So yes, ZFS is great at caching writes and optimising writes this way, but once you insist on fsync, then bad performance happen. Side note: I'm really looking forward how hammer2 is going to solve that. Aha, point taken that fsync() would be slow. However, for any IO that not involves constant fsync():s, performance should be pretty fine, no? (And what about fsyc():s on SSD:s.. anyhow not relevant to my usecase.) Also point taken that ZFS does have some overhead. The point with me though is that I'd be happy to "pay" that.. and I believe it can be made much less than the 85% overhead seen in your example benchmark here. Another side note: even in current RAID1C I can do delay writes (like ZFS), optimize and merge chksum computation and writes this way. I can even read chksum from different chunk than actual data are read to mitigate your all writes mis-directs on bad drive scenario (avoiding all drives fail scenario) but then the result will be more complex code, with the former way much complex than with the later which is easier actually. But based on what I've seen so far adding another layer or even two of them for another chksums and properly caching this, I'm afraid the complexity would go over the roof completely here and would not be considered OpenBSD-like or OpenBSD friendly solution anymore. Anyway, as you like the scheme, please take the code and hack it together. If this is that fantastic and works well I would be your first loyal user believe me. Delaying writes would be all fine with me. What causes the code to be complex here? I would guess that a beautiful way to implement this hashing would be atop your RAID1C! :D What about a total hash, and then one level or max two levels of hashes under it? You already implemented caching of checksums and the logics to maintain their reserved area. And an individual disk's size will never change, that should help to keep complexity under control I guess. I would guess the whole total-checksum functionality could be done in 1000-2000 locs. Feels realistic?
Re: serious watchdog timeout issues with em driver
On 02.12.2015 22:25, Atanas Vladimirov wrote: On 30.11.2015 14:08, Atanas Vladimirov wrote: Hi, I'm not sure if this is related to resent em(4) changes, but after upgrade from: Hi, Just ignore my previous assumptions. Hi, Sorry for the noise! Please ignore all of my previous emails. It seems that my ISP changed a NIC port on the router which served as my default gateway and I used a wrong MAC address. I'm really sorry. Best wishes, Atanas
+1 vote for including the keydisk-on-same-hdd patch :) Re: Unable to sufficiently clean up softraid metadata (Re: crypto softraid and keydisk on same harddrive)
On 2015-12-02 08:26, Patrik Lundin wrote: Hello, I have a custom installer script which automatically creates RAID devices and assembles an sd1 CRYPTO device before the ordinary installer continues (making the installer use sd1 for the rest of the installation). This works well, other than needing this patch since the keydisk is on the same harddrive: http://marc.info/?l=openbsd-misc=141450636905550=2 http://marc.info/?l=openbsd-misc=141450636905550=2 I vote +1 for including this patch into OpenBSD! I found it to work well. Patrik do you vote +1 too? Did it work well for you? I have no idea if this is a high quality patch though, someone else would need to tell. But again it did have the desired effect.
Re: +1 vote for including the keydisk-on-same-hdd patch :) Re: Unable to sufficiently clean up softraid metadata (Re: crypto softraid and keydisk on same harddrive)
On Thu, Dec 03, 2015 at 05:52:13AM +0800, Tinker wrote: > On 2015-12-02 08:26, Patrik Lundin wrote: > > > >This works well, other than needing this patch since the keydisk is on > >the same harddrive: > >http://marc.info/?l=openbsd-misc=141450636905550=2 > > > I vote +1 for including this patch into OpenBSD! I found it to work well. > > Patrik do you vote +1 too? Did it work well for you? > > I have no idea if this is a high quality patch though, someone else would > need to tell. But again it did have the desired effect. > While I appreciate someone else pushing for this diff, I am in no position to "vote" for it. If the suitable devs find this a valuable use of their time I am sure no voting is needed. If they don't then it is up to me or someone else to present a diff that lives up to the expected code quality. I can add that it seems to work well on amd64, but that is all I have to offer for now. -- Patrik Lundin
"# systrace -c1000:1000 kate" for privilege escalated editing?
I want to be able to use systrace for privilege escalation for kompare for sysmerge diffs and kate. Why isn't systrace able to do this? -Luke
Weird cursor problem
I don't see anything in the archives about this. On 5.7 i386 with fvwm several times a day my cursor changes to the left-pointing finger arrow that Firefox uses to point to links and clicking on things has no effect. I can't change the focus, if an rxvt window has the focus I can type in it. Today it happened when Firefox hadn't been running in at least a couple hours. If I can manage to kill Firefox that usually stops it. Today I discovered that ctrl-arrow to change panes of the desktop stops it, 100% of the time so far. Repaint window, change cursor I guess. Not a huge problem, just thought I'd mention it. Just a curiosity. -- Credit is the root of all evil. - AB1JX
Re: Any news on Merkle tree-hash-based whole-disk checksums (=ZFS-style checksums) in softraid?
> You already implemented caching of checksums no, this part is not working yet. > I would guess the whole total-checksum functionality could be done in > 1000-2000 locs. Feels realistic? I don't think so, but please try it and send me your patch.
Re: Any news on Merkle tree-hash-based whole-disk checksums (=ZFS-style checksums) in softraid?
On Wed, Dec 2, 2015 at 6:06 AM, Tinkerwrote: > In comparison, Karel's RAID1C in its present form would be like downloading > the file twice, and per-block CRC32 hashes twice, and then comparing both > copies to know you got the right thing. > > That's nice as it provides some automatic healing, but, that has a > limitation in the extra space used, and yet it's not safe to misdirected > writes, not even across the time that it's mounted continuously. Seriously I do my best in order to increase data safety on OpenBSD based RAID1 system. What I see in practice looks good enough to me. > Just hashing the whole disk (and also keeping that hash in RAM fort he whole > period that it's in use) seems like a pretty inexpensive and "lean and mean" > way to data safety guarantees to me. I guess the devil is in "inexpensive". You again do your assumption based on what COW fss do. The problem here is that we're on completely different level and what's seems to be easy/inexpensive on COW fs level seems to be expensive on block-device level. At least I'm guessing from my own experience with SR-RAID. > We do know that what is happening is that disks do fail in all kinds of > ways, some less and some more incredible, we do see that ordinary > filesystems not would detect misdirected writes at the location where, and > the question I wanted to pose by this conversation was how to maximize data > safety - I think RAID1C is capable of detecting mis-directed write. I wrote about it in some of my previous email.
hotplug - blacklisting/whitelisting devices
As "hotplug pseudo-device passes device attachment and detachment events", does it mean it is too late to blacklist/whitelist hotplug devices? I was checking https://github.com/dkopecek/usbguard and I got curious if blacklisting/whitelisting of removable usb devices could be done in hotplugd. j.
Re: Any news on Merkle tree-hash-based whole-disk checksums (=ZFS-style checksums) in softraid?
On Wed, Dec 2, 2015 at 10:39 AM, Tinkerwrote: > On 2015-12-02 17:31, Karel Gardas wrote: >> >> I think RAID1C is capable of detecting mis-directed write. I wrote >> about it in some of my previous email. > > > Hi Karel, > > I'll follow up on the other things in a separate email later, but, you > clarify that you think RAID1C has protection against misdirected writes > already - I don't understand how it works, can you please explain to me > *exactly* how it can be said to be solid against misdirected writes?: > > > Let's assume the following nightmare scenario: > > * You write data to sector X. All the physical writes for that on all the > underlying disks are mis-directed (about 4 writes). I've been talking about mis-directed write (singular!) while you suddenly switch to talk about writes (plural!). So what you describe here is basically you get *all* writes to *all* drives mis-directed? Oh, what's the probability of this? Well, if your read on the drives is not mis-directed in the same way (why would we then be talking about mis-directed write right?), then you read old data from the sector silently. BOOOM! :-) But allow me to counter with the question: what will your scheme do if *all* writes to *all* your drives are mis-directed? Wouldn't it also return old data on read from X? :-)
Re: Any news on Merkle tree-hash-based whole-disk checksums (=ZFS-style checksums) in softraid?
On 2015-12-02 18:10, Karel Gardas wrote: On Wed, Dec 2, 2015 at 10:39 AM, Tinkerwrote: On 2015-12-02 17:31, Karel Gardas wrote: I think RAID1C is capable of detecting mis-directed write. I wrote about it in some of my previous email. Hi Karel, I'll follow up on the other things in a separate email later, but, you clarify that you think RAID1C has protection against misdirected writes already - I don't understand how it works, can you please explain to me *exactly* how it can be said to be solid against misdirected writes?: Let's assume the following nightmare scenario: * You write data to sector X. All the physical writes for that on all the underlying disks are mis-directed (about 4 writes). I've been talking about mis-directed write (singular!) while you suddenly switch to talk about writes (plural!). So what you describe here is basically you get *all* writes to *all* drives mis-directed? Oh, what's the probability of this? Well, if your read on the drives is not mis-directed in the same way (why would we then be talking about mis-directed write right?), then you read old data from the sector silently. BOOOM! :-) But allow me to counter with the question: what will your scheme do if *all* writes to *all* your drives are mis-directed? Wouldn't it also return old data on read from X? :-) Karel, To answer your question: In that case, as soon as that invalid data would actually be read from disk, it would be caught by the checksums that are guaranteed to be kept in RAM, so that is, the first-level checkums (or the über-checksum) match would fail. So that's what's nice about having all the disk checksummed, that data security works even if all writes fail. That moves the risk surface altogether from the whole time period when the disk is in use, to only the time between being taken out of use and being taken in use again. So that's the difference here. Thoughts? Btw, any checksum algorithm would work for implementing a tree like this by the way, even CRC64 I guess. So Fletcher as such is out the window. I intend to followup on your other emails in some hours. Tinker
Re: Unable to sufficiently clean up softraid metadata
On Tue, Dec 01, 2015 at 05:53:22PM -0800, Nathan Wheeler wrote: > I have a similar sort of setup during installs and I clear out the > first 10m before setting up the CRYPTO disk and it works for me. I > don't think you're zeroing out enough at the beginning of the disk. > > dd if=/dev/zero of=/dev/rsd0c bs=10m count=1 > Following your tip i tried the following series of commands which failed in the same way: === # dd if=/dev/zero of=/dev/rsd0d # dd if=/dev/zero of=/dev/rsd0a bs=1m count=1 # dd if=/dev/zero of=/dev/rsd0c bs=10m count=1 === I then tried the following variant which also failed: === # dd if=/dev/zero of=/dev/rsd0d # dd if=/dev/zero of=/dev/rsd0a bs=10m count=1 # dd if=/dev/zero of=/dev/rsd0c bs=1m count=1 === Finally i tried this just to cover all possibilities, which also failed: === # dd if=/dev/zero of=/dev/rsd0d # dd if=/dev/zero of=/dev/rsd0a bs=10m count=1 # dd if=/dev/zero of=/dev/rsd0c bs=10m count=1 === I might have been unclear on this point (and I am not sure if this is how you are doing it) but the above commands are executed on the running system before rebooting into the installer. Could it be that the kernel writes out some in-memory softraid information to the disk before rebooting? I have noticed that just leaving a "dd if=/dev/zero of=/dev/rsd0c bs=1m" "long enough" will work, but it feels too brittle, and my optimal situation would be that the system is able to operate after the above commands are run, only having an impact after a reboot or power outage, which the unbounded dd does not achieve (this might not be an achievable goal at all of course). -- Patrik Lundin
Re: Any news on Merkle tree-hash-based whole-disk checksums (=ZFS-style checksums) in softraid?
On 2015-12-02 17:31, Karel Gardas wrote: I think RAID1C is capable of detecting mis-directed write. I wrote about it in some of my previous email. Hi Karel, I'll follow up on the other things in a separate email later, but, you clarify that you think RAID1C has protection against misdirected writes already - I don't understand how it works, can you please explain to me *exactly* how it can be said to be solid against misdirected writes?: Let's assume the following nightmare scenario: * You write data to sector X. All the physical writes for that on all the underlying disks are mis-directed (about 4 writes). * Sector X is wiped from the filesystem cache and any other cache, except for any cached hash or checksum values that RAID1C guaranteedly stores in RAM for its whole time of operation. * An fread() of sector X is done. What happens on the fread()? Thanks! Tinker
Re: Unable to sufficiently clean up softraid metadata
On Wed, Dec 02, 2015 at 10:44:44AM +0100, Patrik Lundin wrote: > On Tue, Dec 01, 2015 at 05:53:22PM -0800, Nathan Wheeler wrote: > > I have a similar sort of setup during installs and I clear out the > > first 10m before setting up the CRYPTO disk and it works for me. I > > don't think you're zeroing out enough at the beginning of the disk. > > > > dd if=/dev/zero of=/dev/rsd0c bs=10m count=1 > > > > Following your tip i tried the following series of commands which failed in > the > same way: > === > # dd if=/dev/zero of=/dev/rsd0d > # dd if=/dev/zero of=/dev/rsd0a bs=1m count=1 > # dd if=/dev/zero of=/dev/rsd0c bs=10m count=1 > === > > I then tried the following variant which also failed: > === > # dd if=/dev/zero of=/dev/rsd0d > # dd if=/dev/zero of=/dev/rsd0a bs=10m count=1 > # dd if=/dev/zero of=/dev/rsd0c bs=1m count=1 > === > > Finally i tried this just to cover all possibilities, which also failed: > === > # dd if=/dev/zero of=/dev/rsd0d > # dd if=/dev/zero of=/dev/rsd0a bs=10m count=1 > # dd if=/dev/zero of=/dev/rsd0c bs=10m count=1 > === > > I might have been unclear on this point (and I am not sure if this is > how you are doing it) but the above commands are executed on the running > system before rebooting into the installer. Could it be that the kernel > writes out some in-memory softraid information to the disk before > rebooting? If you are zeroing the char devices under the feet of a running OS i would not dare to guess what happens. Can you try to zero the key disk and the first 1MB of the RAID partition from bsd.rd instead? > > I have noticed that just leaving a "dd if=/dev/zero of=/dev/rsd0c bs=1m" > "long enough" will work, but it feels too brittle, and my optimal > situation would be that the system is able to operate after the above > commands are run, only having an impact after a reboot or power outage, > which the unbounded dd does not achieve (this might not be an achievable > goal at all of course). > > -- > Patrik Lundin -- / Raimo Niskanen, Erlang/OTP, Ericsson AB
pf, anchors, and macros
I'm puzzled by the following. According to the documentation it should work? The example at http://www.openbsd.org/faq/pf/anchors.html indeed works, but that's an inline anchor. Here's the problem: I would like to define a macro in an anchor, and use that macro in other anchors below it -- like so: [ /etc/pf/anchors/base ] ext_if="ix0" (...) anchor ipsec load anchor ipsec from "/etc/pf/anchors/ipsec" [ /etc/pf/anchors/ipsec ] (...) pass out quick on $ext_if inet proto udp from ($ext_if:0) to port { isakmp, ipsec-nat-t } keep state (...) As far as I understand this should work. Instead the result is: # pfctl -nf /etc/pf.conf /etc/pf/anchors/ipsec:6: macro 'ext_if' not defined /etc/pf/anchors/ipsec:6: syntax error pfctl: load anchors The base anchor is loaded from /etc/pf.conf like so: anchor "base/*" load anchor base from "/etc/pf/anchors/base" .. but I don't think it's relevant as I've tried to run the test between pf.conf and the base anchor, and still macros defined in pf.conf are not available from /etc/pf/anchors/base. Is this intended behaviour? Running on 5.8 release.
Re: Any news on Merkle tree-hash-based whole-disk checksums (=ZFS-style checksums) in softraid?
> To answer your question: In that case, as soon as that invalid data would > actually be read from disk, it would be caught by the checksums that are > guaranteed to be kept in RAM, so that is, the first-level checkums (or the > über-checksum) match would fail. Ah, ok, but then this is safe only as long as you do not switch machine off. So in a catastrophic scenario like you describe where all writes fails to all drives, you switch machine off in a hope everything's all right and when you switch it on again your read should probably return old data. Right? > Btw, any checksum algorithm would work for implementing a tree like this by > the way, even CRC64 I guess. So Fletcher as such is out the window. I intend > to followup on your other emails in some hours. You do not realise how expensive this (I mean whole this tree chksumming business) is on fsync. Single proof of this is database benchmark. I'm using pgbench. Try to run that and see for yourself, but if you believe me and trust my numbers then on OpenBSD 1 client bench setup I'm able to get ~1190 tps on RAID1, ~950 tps on RAID1C and guess what, just ~100 tps on ZFS on Solaris 11.3. So yes, ZFS is great at caching writes and optimising writes this way, but once you insist on fsync, then bad performance happen. Side note: I'm really looking forward how hammer2 is going to solve that. Another side note: even in current RAID1C I can do delay writes (like ZFS), optimize and merge chksum computation and writes this way. I can even read chksum from different chunk than actual data are read to mitigate your all writes mis-directs on bad drive scenario (avoiding all drives fail scenario) but then the result will be more complex code, with the former way much complex than with the later which is easier actually. But based on what I've seen so far adding another layer or even two of them for another chksums and properly caching this, I'm afraid the complexity would go over the roof completely here and would not be considered OpenBSD-like or OpenBSD friendly solution anymore. Anyway, as you like the scheme, please take the code and hack it together. If this is that fantastic and works well I would be your first loyal user believe me.
Re: OpenBSD 5.8 on VMware 5.5
I just wanted to thank everyone for their feedback. Thanks a lot! You guys are amazing. Best regards, Felipe Gomes On Wed, Dec 2, 2015 at 4:03 AM, Bruno Flueckigerwrote: > On 01.12.2015 16:50, Felipe Gomes wrote: > >> Folks, >> >> I've been trying to search for more information on OpenBSD as a VMWare >> guest, but I wasn't able to find much... and the information is pretty >> much >> outdated. >> >> What are the recommendations for OpenBSD 5.8 (amd64) as a guest on VMware >> 5.5? >> >> Guest Operating System: should I pick "Other (64bit)" or FreeBSD? >> >> How does OpenBSD work with "virtual sockets" and "cores per virtual >> socket"? >> >> What is the best NIC? E1000, E1000E, VMXNET2 ENHANCED or VMXNET3? >> >> What is the recommended SCSI Controller? LSI Logic Parallel, LSI Logic SAS >> or VMware Paravirtual? >> >> I'd believe that all of these options work... I just don't know which is >> more stable or perform better. >> >> Any other tips on fine tunning or special setting? >> >> I'm planning on migrating a few Soekris boxes to virtual machines. Is this >> reliable? Is anyone running production OpenBSD servers on VMware? >> >> Thanks in advance! >> > > I run a productive SMTP server with OpenBSD 5.8-stable on VMware 5.5 for > some > months and so far I didn't experience any problems. Guest OS is FreeBSD, > NIC > is VMXNET3 and the controller is LSI Logic Parallel. > > There are plans for more OpenBSD servers on VMware in the company I work > for > due to the small footprint of the OS and the very good experience we have > so > far. > > Cheers, > Bruno
OpenBSD + pf + DPI
Hi list, I don't know how to start to make Deep Packet Inspection. My interest is OpenBSD and pf related. Anyone has already used on OpenBSD? It is possibile on OpenBSD with shipped (base/ports) software? Every tips are appreciated. Thanks in advance.
Re: Unable to sufficiently clean up softraid metadata
On Wed, Dec 02, 2015 at 11:34:25AM +0100, Raimo Niskanen wrote: > > If you are zeroing the char devices under the feet of a running OS i would > not dare to guess what happens. Can you try to zero the key disk and the > first 1MB of the RAID partition from bsd.rd instead? > Given that I can not be sure sd0a exists when the installer runs (since it may be a fresh machine), I would need to add some complexity inside the script to look if it exists, and if that is the case then wipe it. However, thinking some more about what I could do at install time I recalled you can add "-C force" to the bioctl invocation to make it ignore unclean data in metadata areas, turning: === bioctl -c C -l /dev/sd0a -k /dev/sd0d softraid0 === ... into: === bioctl -c C -C force -l /dev/sd0a -k /dev/sd0d softraid0 === And that indeed seems to fix the problem. I now only wipe the keydisk partition when "destroying" the disk as I did before yet the installer manages to create the sd1 device as expected. Thanks for the pointers :). -- Patrik Lundin
Re: OpenBSD + pf + DPI
On Wed, Dec 02, 2015 at 12:45:26PM +0100, Alessandro Baggi wrote: > Hi list, > I don't know how to start to make Deep Packet Inspection. My interest is > OpenBSD and pf related. > > Anyone has already used on OpenBSD? It is possibile on OpenBSD with shipped > (base/ports) software? > > Every tips are appreciated. > You might want to read divert(4) which describes how to pass packets from pf to a userland application and back. -- Patrik Lundin
Re: Failure to boot install media using bootia32.efi
On Tue, 1 Dec 2015 20:41:15 + Callum Davieswrote: > I have two "devices" using IA32 UEFI firmware with 64-bit > hardware. An Asus EeeBook X502TA and qemu-system-x86_64 with > an IA32 TianoCore firmware. Neither of these will boot from > snapshots/amd64/install58.fs. > > Attempting to run bootia32.efi from the UEFI shell of the qemu system > simply tells me "Command Error Status: Not Found". > > The EeeBook is deficient, and doesn't provide an UEFI shell, but I > suspect it fails for the same reason. Fixed some issues on ia32 on the CVS tree. Can you try that by replacing BOOTIA32.EFI in install58.fs? I put compiled one on http://yasuoka.net/~yasuoka/BOOTIA32.EFI . (On qemu, the video hangs after the boot loader switches the video mode. But this seems to be a problem of qemu or my environment.) --yasuoka
home keys in tmux
When i push home at a ksh prompt in xterm, the cursor goes to the beginning of the line. When i do the same in tmux, nothing happens. TERM in xterm is xterm. TERM in tmux is screen. How do i fix this? (Why do i need to fix it?)
Re: A branded USB stick as an alternative to the CD set?
On Tue, Dec 1, 2015 at 8:18 PM, Theo de Raadtwrote: >>"All I can do is buy the CD's and give some $ to the >>foundation. Any other suggestion is not productive." >> >>I don't think that quite covers it. Those of us who have the choice >>can send checks or Paypal money directly to Theo, as described on the >>Donations page. I think checks are preferable, because they eliminate >>Paypal skimming its credit-card-like fees, at the cost of a stamp. The >>CDs also involve paying a middle-man. > > Completely true. Also it is a 20 minute walk each way to the bank, > and keyboard folk need to do more walks. > >>Checks to Theo get the maximum amount of money to the place where it >>will do the project the most good, which includes providing Theo with >>the money he needs to continue doing what he's doing. > > On a personal "hate ramen noodles and tuna" level, I agree. > > But my good-for-project-good-for-the-world side says the OpenBSD > Foundation is more effective at growing the contribution pie and > in particular funding the hackathons where great work happens. But if we lose the project leader due to lack of exercise and food, that's not good for the project. You made it very clear in a previous message to this thread that no Foundation money comes to you. So while the Foundation may be doing good things with their money, we, the community, need to be sure that you have what you need. And in the unlikely event that the freeloader factor decreases and we send you more than you need, couldn't you turn the excess over to the Foundation?
Re: OpenBSD + pf + DPI
On 2015-12-02, Alessandro Baggiwrote: > Hi list, > I don't know how to start to make Deep Packet Inspection. My interest is > OpenBSD and pf related. > > Anyone has already used on OpenBSD? It is possibile on OpenBSD with > shipped (base/ports) software? > > Every tips are appreciated. > > Thanks in advance. > > You can inspect packets deeply with tcpdump(1)... Without more information about what you want to do, this isn't really something anyone can answer sensibly.
Re: python uwsgi port/package
On Wed, Dec 02, 2015 at 09:16:05PM +0100, Kamil CholewiÅski wrote: > Everything boils down to whether you'd like to run more than one app on > your box. > > > While I love pip and virtualenv in development, I don't understand the > > advantage they offer over the system package manager on a production > > machine. > > Easy: whenever you can't be bothered with proper containers. App X > requires package foo version 1.2, app Y requires foo version 1.4. > > Docker solves this universally. You can also achieve a similar effect by > building a chroot. virtualenv's advantage is it doesn't require root, > and is (subjectively) easier to use. > I agree with this completely but I tend to be in the one VM per app category which puts me solidly in the "one app per box" square. I like what I've heard about Docker because it commercializes and commoditizes the one app per box management philosophy. [... snip ...] > > In addition, I feel that a reasonable uwsgi package would include an > > rc-script to start your app automatically at system boot time. > > I prefer to run my application servers with runit. Traditional RC > scripts usually assume one package = one application instance. Usually > that's a sane assumption (what would be your reason for running two > instances of Apache?) but again, if you can't be bothered with > containers, virtualenv+runit make it easy to just put app X in /home/x, > app Y in /home/y, then run two uwsgi's. > I looked at runit but the documentation bills it as a replacement for init which I find to be very heavyweight. Am I missing something about runit, like a way to use it to manage a set of processes under init? > You've mentioned Puppet. Also check out Ansible. I would have said that I'm with Winston Churchill [1] on puppet but I have to say that I'm not. Right now, puppet's what I know. I'm aware of chef and I have seen Ansible in the space. If Ansible is the on that's written in python I think I want to look at that one next. Thank you very much, -- Chris [1] "Indeed it has been said that democracy is the worst form of Government except for all those other forms that have been tried from time to time." -- Winston Churchill to the House of Commons - 11-Nov, 1947 [demime 1.01d removed an attachment of type application/pgp-signature]
disklabel suggestion
I'm trying to make several changes to my disklabel at once. If I try to do it with -R to read in a file I get disklabel: ioctl DIOCWDINFO: Open partition would move or shrink So I used -E and used the interactive editor, which let me get through the same edits without complaining about anything, then when I did q gave the same error and dumped everything. It doesn't say which partition would change and I can't spot it, doesn't give you the option to stay in the editor or save to a file. I guess I have to make one change at a time or make one and write it with w before the next one. I thought I was getting away with something until the last minute. Seems like on that error it could stay in the editor, maybe even narrow down the problem better in the error message. Don't quit unless the write was successful? -- Credit is the root of all evil. - AB1JX
Re: "# systrace -c1000:1000 kate" for privilege escalated editing?
03 дек. 2015 г. 4:27 полÑзоваÑÐµÐ»Ñ "Luke Small"напиÑал: > > I want to be able to use systrace for privilege escalation for kompare for > sysmerge diffs and kate. Why isn't systrace able to do this? Because noone wrote a systrace policy for Kate and Kompare (for your installation and user) yet? That's without mentioning that it would be hard to restrict those applications in a correct manner: they do use a lot of system resources by just being nice KDE apps. That being said, I won't expect much security problems in Kompare itself. Kate is more complex, but still doesn't run in terminal. Thus Kompare and Kate likely not being hurt by some crazy escape codes in patch files. Anything else lies outside of usage profile you're talking about, if I understood you correctly. -- Vadim Zhukov