Re: linebuffering diff for tr(1)

2013-11-20 Thread Jan Klemkow
On Tue, Nov 19, 2013 at 09:34:22PM +, Stuart Henderson wrote:
 On 2013/11/19 14:10, Theo de Raadt wrote:
  In general, new non-standard options are bad.

I know and this is my own opinion to, in general.

  Basically, if we add this someone will use it in a script.  Then it will
  become non-portable.  You cannot just invent something on your own like
  this, without doing research to find out if someone else added a different
  option.  I don't see evidence of that, so the gut answer is no.

You are right, I forgot to have a look on other UNIX OSs before writing
this diff.  I just saw that -u in sed(1) is an extension to POSIX like
this diff would be.  So I think, may be it is a good extension?!?

Thanks to Stuart for doing my homework :-)
 Only FreeBSD/Dragonfly seem to use -u in tr.  How about sed -u instead?

At the moment I use sed(1) for that job, but for deleting one character
sed it to big. (Just my opinion and feeling during writing that script.)

It there a way to get the -u option in tr(1) a BSD extension like the
others?  In this case I would change my diff from line buffering to
unbuffered, like FreeBSD does.

   here is a diff that adds optional linebuffering to tr(1) with command
   line switch -u like in sed(1).  I need this to remove '\r' characters
   from a continues input steam which lines have to be there immediately.
   
   Please write me if something is wrong with this diff or the change
   itself.  I will fix it.
   
   bye,
   Jan
   
   Index: tr.1
   ===
   RCS file: /cvs/src/usr.bin/tr/tr.1,v
   retrieving revision 1.20
   diff -u -p -r1.20 tr.1
   --- tr.1  14 Aug 2013 08:39:27 -  1.20
   +++ tr.1  19 Nov 2013 20:46:33 -
   @@ -41,18 +41,18 @@
.Nd translate characters
.Sh SYNOPSIS
.Nm tr
   -.Op Fl cs
   +.Op Fl csu
.Ar string1 string2
.Nm tr
   -.Op Fl c
   +.Op Fl cu
.Fl d
.Ar string1
.Nm tr
   -.Op Fl c
   +.Op Fl cu
.Fl s
.Ar string1
.Nm tr
   -.Op Fl c
   +.Op Fl cu
.Fl ds
.Ar string1 string2
.Sh DESCRIPTION
   @@ -86,6 +86,14 @@ or
.Ar string2 )
in the input into a single instance of the character.
This occurs after all deletion and translation is completed.
   +.It Fl u
   +Force output to be line buffered,
   +printing each line as it becomes available.
   +By default, output is line buffered when standard output is a terminal
   +and block buffered otherwise.
   +See
   +.Xr setbuf 3
   +for a more detailed explanation.
.El
.Pp
In the first synopsis form, the characters in
   @@ -284,6 +292,10 @@ The
utility is compliant with the
.St -p1003.1-2008
specification.
   +.Pp
   +The flag
   +.Op Fl u
   +is an extension to that specification.
.Pp
System V has historically implemented character ranges using the syntax
.Dq [c-c]
   Index: tr.c
   ===
   RCS file: /cvs/src/usr.bin/tr/tr.c,v
   retrieving revision 1.15
   diff -u -p -r1.15 tr.c
   --- tr.c  27 Oct 2009 23:59:46 -  1.15
   +++ tr.c  19 Nov 2013 20:46:33 -
   @@ -88,7 +88,7 @@ main(int argc, char *argv[])
 int cflag, dflag, sflag, isstring2;

 cflag = dflag = sflag = 0;
   - while ((ch = getopt(argc, argv, cds)) != -1)
   + while ((ch = getopt(argc, argv, cdsu)) != -1)
 switch((char)ch) {
 case 'c':
 cflag = 1;
   @@ -99,6 +99,9 @@ main(int argc, char *argv[])
 case 's':
 sflag = 1;
 break;
   + case 'u':
   + setlinebuf(stdout);
   + break;
 case '?':
 default:
 usage();
   @@ -239,9 +242,9 @@ static void
usage(void)
{
 fprintf(stderr,
   - usage: tr [-cs] string1 string2\n
   -tr [-c] -d string1\n
   -tr [-c] -s string1\n
   -tr [-c] -ds string1 string2\n);
   + usage: tr [-csu] string1 string2\n
   +tr [-cu] -d string1\n
   +tr [-cu] -s string1\n
   +tr [-cu] -ds string1 string2\n);
 exit(1);
}
   
  
 



icmp cksums

2013-11-20 Thread Henning Brauer
make the icmp stack use the fake offload engine.
prevents double cksumming in some cases and happens to fix a bug in an
obscure, constructed case.

Index: ip_icmp.c
===
RCS file: /cvs/src/sys/netinet/ip_icmp.c,v
retrieving revision 1.110
diff -u -p -r1.110 ip_icmp.c
--- ip_icmp.c   17 Nov 2013 10:07:32 -  1.110
+++ ip_icmp.c   20 Nov 2013 07:47:49 -
@@ -830,7 +830,7 @@ icmp_send(struct mbuf *m, struct mbuf *o
hlen = ip-ip_hl  2;
icp = (struct icmp *)(mtod(m, caddr_t) + hlen);
icp-icmp_cksum = 0;
-   icp-icmp_cksum = in4_cksum(m, 0, hlen, ntohs(ip-ip_len) - hlen);
+   m-m_pkthdr.csum_flags |= M_ICMP_CSUM_OUT;
 #ifdef ICMPPRINTFS
if (icmpprintfs) {
char dst[INET_ADDRSTRLEN], src[INET_ADDRSTRLEN];



msgbuf_write audit

2013-11-20 Thread Henning Brauer
so, msgbuf_write can now (again) return EAGAIN. some daemons have been
fixed/adopted, some not. I did a full audit of the tree for all
msgbuf_write users EAGAIN handling - this is the result.

Index: usr.sbin/dvmrpd/control.c
===
RCS file: /cvs/src/usr.sbin/dvmrpd/control.c,v
retrieving revision 1.17
diff -u -p -r1.17 control.c
--- usr.sbin/dvmrpd/control.c   11 Mar 2013 17:40:11 -  1.17
+++ usr.sbin/dvmrpd/control.c   20 Nov 2013 08:18:05 -
@@ -229,7 +229,7 @@ control_dispatch_imsg(int fd, short even
}
}
if (event  EV_WRITE) {
-   if (msgbuf_write(c-iev.ibuf.w) == -1) {
+   if (msgbuf_write(c-iev.ibuf.w) == -1  errno != EAGAIN) {
control_close(fd);
return;
}
Index: usr.sbin/dvmrpd/dvmrpd.c
===
RCS file: /cvs/src/usr.sbin/dvmrpd/dvmrpd.c,v
retrieving revision 1.14
diff -u -p -r1.14 dvmrpd.c
--- usr.sbin/dvmrpd/dvmrpd.c20 Aug 2011 19:02:28 -  1.14
+++ usr.sbin/dvmrpd/dvmrpd.c20 Nov 2013 08:10:34 -
@@ -366,7 +366,7 @@ main_dispatch_dvmrpe(int fd, short event
fatalx(pipe closed);
}
if (event  EV_WRITE) {
-   if (msgbuf_write(ibuf-w) == -1)
+   if (msgbuf_write(ibuf-w) == -1  errno != EAGAIN)
fatal(msgbuf_write);
}
 
@@ -419,7 +419,7 @@ main_dispatch_rde(int fd, short event, v
fatalx(pipe closed);
}
if (event  EV_WRITE) {
-   if (msgbuf_write(ibuf-w) == -1)
+   if (msgbuf_write(ibuf-w) == -1  errno != EAGAIN)
fatal(msgbuf_write);
}
 
Index: usr.sbin/dvmrpd/dvmrpe.c
===
RCS file: /cvs/src/usr.sbin/dvmrpd/dvmrpe.c,v
retrieving revision 1.10
diff -u -p -r1.10 dvmrpe.c
--- usr.sbin/dvmrpd/dvmrpe.c4 Jul 2011 04:34:14 -   1.10
+++ usr.sbin/dvmrpd/dvmrpe.c20 Nov 2013 08:12:35 -
@@ -1,4 +1,4 @@
-/* $OpenBSD: dvmrpe.c,v 1.9 2010/05/26 13:56:07 nicm Exp $ */
+/* $OpenBSD: dvmrpe.c,v 1.10 2011/07/04 04:34:14 claudio Exp $ */
 
 /*
  * Copyright (c) 2005 Claudio Jeker clau...@openbsd.org
@@ -249,7 +249,7 @@ dvmrpe_dispatch_main(int fd, short event
fatalx(pipe closed);
}
if (event  EV_WRITE) {
-   if (msgbuf_write(ibuf-w) == -1)
+   if (msgbuf_write(ibuf-w) == -1  errno != EAGAIN)
fatal(msgbuf_write);
}
 
@@ -314,7 +314,7 @@ dvmrpe_dispatch_rde(int fd, short event,
fatalx(pipe closed);
}
if (event  EV_WRITE) {
-   if (msgbuf_write(ibuf-w) == -1)
+   if (msgbuf_write(ibuf-w) == -1  errno != EAGAIN)
fatal(msgbuf_write);
}
 
Index: usr.sbin/dvmrpd/rde.c
===
RCS file: /cvs/src/usr.sbin/dvmrpd/rde.c,v
retrieving revision 1.22
diff -u -p -r1.22 rde.c
--- usr.sbin/dvmrpd/rde.c   2 Nov 2009 20:31:50 -   1.22
+++ usr.sbin/dvmrpd/rde.c   20 Nov 2013 08:12:57 -
@@ -1,4 +1,4 @@
-/* $OpenBSD: rde.c,v 1.21 2009/09/06 09:52:14 michele Exp $ */
+/* $OpenBSD: rde.c,v 1.22 2009/11/02 20:31:50 claudio Exp $ */
 
 /*
  * Copyright (c) 2004, 2005 Claudio Jeker clau...@openbsd.org
@@ -209,7 +209,7 @@ rde_dispatch_imsg(int fd, short event, v
fatalx(pipe closed);
}
if (event  EV_WRITE) {
-   if (msgbuf_write(ibuf-w) == -1)
+   if (msgbuf_write(ibuf-w) == -1  errno != EAGAIN)
fatal(msgbuf_write);
}
 
Index: usr.sbin/ldapd/imsgev.c
===
RCS file: /cvs/src/usr.sbin/ldapd/imsgev.c,v
retrieving revision 1.2
diff -u -p -r1.2 imsgev.c
--- usr.sbin/ldapd/imsgev.c 16 Jun 2012 00:08:32 -  1.2
+++ usr.sbin/ldapd/imsgev.c 20 Nov 2013 08:38:48 -
@@ -138,7 +138,7 @@ imsgev_dispatch(int fd, short ev, void *
 * closed, or some error occured. Both case are not recoverable
 * from the imsg perspective, so we treat it as a WRITE error.
 */
-   if ((n = msgbuf_write(ibuf-w)) != 1) {
+   if ((n = msgbuf_write(ibuf-w)) != 1  errno != EAGAIN) {
imsgev_disconnect(iev, IMSGEV_EWRITE);
return;
}
Index: usr.sbin/ldpd/control.c
===
RCS file: /cvs/src/usr.sbin/ldpd/control.c,v
retrieving revision 1.12
diff -u -p -r1.12 control.c
--- usr.sbin/ldpd/control.c 4 Jun 2013 02:25:28 -   1.12
+++ usr.sbin/ldpd/control.c 20 Nov 2013 08:16:39 -
@@ -209,7 +209,7 

Kill IA_SIN()

2013-11-20 Thread Martin Pieuchot
This one is not under _KERNEL but it's used at only one place and
if a port use it, it should probably define it by itself.

ok?

Index: netinet/if_ether.c
===
RCS file: /home/ncvs/src/sys/netinet/if_ether.c,v
retrieving revision 1.111
diff -u -p -r1.111 if_ether.c
--- netinet/if_ether.c  11 Nov 2013 09:15:34 -  1.111
+++ netinet/if_ether.c  20 Nov 2013 09:02:33 -
@@ -868,8 +868,8 @@ arp_ifinit(struct arpcom *ac, struct ifa
 
/* Warn the user if another station has this IP address. */
arprequest(ac-ac_if,
-   (IA_SIN(ifa)-sin_addr.s_addr),
-   (IA_SIN(ifa)-sin_addr.s_addr),
+   satosin(ifa-ifa_addr)-sin_addr.s_addr,
+   satosin(ifa-ifa_addr)-sin_addr.s_addr,
ac-ac_enaddr);
ifa-ifa_rtrequest = arp_rtrequest;
ifa-ifa_flags |= RTF_CLONING;
Index: netinet/in_var.h
===
RCS file: /home/ncvs/src/sys/netinet/in_var.h,v
retrieving revision 1.26
diff -u -p -r1.26 in_var.h
--- netinet/in_var.h23 Oct 2013 15:12:42 -  1.26
+++ netinet/in_var.h20 Nov 2013 09:02:33 -
@@ -73,11 +73,6 @@ struct   in_aliasreq {
 #defineifra_broadaddr  ifra_dstaddr
struct  sockaddr_in ifra_mask;
 };
-/*
- * Given a pointer to an in_ifaddr (ifaddr),
- * return a pointer to the addr as a sockaddr_in.
- */
-#defineIA_SIN(ia) struct in_ifaddr *)(ia))-ia_addr))
 
 
 #ifdef _KERNEL



Unused proc argument in in6_control()

2013-11-20 Thread Martin Pieuchot
Any reason to keep this unused proc argument, in_control() does not have
it.  Ok to kill it?

Index: netinet/tcp_usrreq.c
===
RCS file: /home/ncvs/src/sys/netinet/tcp_usrreq.c,v
retrieving revision 1.116
diff -u -p -r1.116 tcp_usrreq.c
--- netinet/tcp_usrreq.c20 Oct 2013 11:03:01 -  1.116
+++ netinet/tcp_usrreq.c20 Nov 2013 09:16:48 -
@@ -144,7 +144,7 @@ tcp_usrreq(so, req, m, nam, control, p)
 #ifdef INET6
if (sotopf(so) == PF_INET6)
return in6_control(so, (u_long)m, (caddr_t)nam,
-   (struct ifnet *)control, 0);
+   (struct ifnet *)control);
else
 #endif /* INET6 */
return (in_control(so, (u_long)m, (caddr_t)nam,
Index: netinet/udp_usrreq.c
===
RCS file: /home/ncvs/src/sys/netinet/udp_usrreq.c,v
retrieving revision 1.170
diff -u -p -r1.170 udp_usrreq.c
--- netinet/udp_usrreq.c20 Oct 2013 11:03:01 -  1.170
+++ netinet/udp_usrreq.c20 Nov 2013 09:16:48 -
@@ -1128,7 +1128,7 @@ udp_usrreq(struct socket *so, int req, s
 #ifdef INET6
if (inp-inp_flags  INP_IPV6)
return (in6_control(so, (u_long)m, (caddr_t)addr,
-   (struct ifnet *)control, 0));
+   (struct ifnet *)control));
else
 #endif /* INET6 */
return (in_control(so, (u_long)m, (caddr_t)addr,
Index: netinet6/in6.c
===
RCS file: /home/ncvs/src/sys/netinet6/in6.c,v
retrieving revision 1.124
diff -u -p -r1.124 in6.c
--- netinet6/in6.c  13 Nov 2013 08:27:24 -  1.124
+++ netinet6/in6.c  20 Nov 2013 09:16:48 -
@@ -119,8 +119,7 @@ const struct in6_addr in6mask64 = IN6MAS
 const struct in6_addr in6mask96 = IN6MASK96;
 const struct in6_addr in6mask128 = IN6MASK128;
 
-int in6_lifaddr_ioctl(struct socket *, u_long, caddr_t, struct ifnet *,
-   struct proc *);
+int in6_lifaddr_ioctl(struct socket *, u_long, caddr_t, struct ifnet *);
 int in6_ifinit(struct ifnet *, struct in6_ifaddr *, int);
 void in6_unlink_ifa(struct in6_ifaddr *, struct ifnet *);
 void in6_ifloop_request(int, struct ifaddr *);
@@ -332,8 +331,7 @@ in6_mask2len(struct in6_addr *mask, u_ch
 }
 
 int
-in6_control(struct socket *so, u_long cmd, caddr_t data, struct ifnet *ifp,
-struct proc *p)
+in6_control(struct socket *so, u_long cmd, caddr_t data, struct ifnet *ifp)
 {
struct  in6_ifreq *ifr = (struct in6_ifreq *)data;
struct  in6_ifaddr *ia = NULL;
@@ -389,7 +387,7 @@ in6_control(struct socket *so, u_long cm
return (EPERM);
/* FALLTHROUGH */
case SIOCGLIFADDR:
-   return in6_lifaddr_ioctl(so, cmd, data, ifp, p);
+   return in6_lifaddr_ioctl(so, cmd, data, ifp);
}
 
/*
@@ -1301,8 +1299,8 @@ in6_unlink_ifa(struct in6_ifaddr *ia, st
  * address encoding scheme. (see figure on page 8)
  */
 int
-in6_lifaddr_ioctl(struct socket *so, u_long cmd, caddr_t data, 
-struct ifnet *ifp, struct proc *p)
+in6_lifaddr_ioctl(struct socket *so, u_long cmd, caddr_t data,
+struct ifnet *ifp)
 {
struct if_laddrreq *iflr = (struct if_laddrreq *)data;
struct ifaddr *ifa;
@@ -1409,7 +1407,7 @@ in6_lifaddr_ioctl(struct socket *so, u_l
in6_prefixlen2mask(ifra.ifra_prefixmask.sin6_addr, prefixlen);
 
ifra.ifra_flags = iflr-flags  ~IFLR_PREFIX;
-   return in6_control(so, SIOCAIFADDR_IN6, (caddr_t)ifra, ifp, p);
+   return in6_control(so, SIOCAIFADDR_IN6, (caddr_t)ifra, ifp);
}
case SIOCGLIFADDR:
case SIOCDLIFADDR:
@@ -1505,7 +1503,7 @@ in6_lifaddr_ioctl(struct socket *so, u_l
 
ifra.ifra_flags = ia-ia6_flags;
return in6_control(so, SIOCDIFADDR_IN6, (caddr_t)ifra,
-   ifp, p);
+   ifp);
}
}
}
Index: netinet6/in6_var.h
===
RCS file: /home/ncvs/src/sys/netinet6/in6_var.h,v
retrieving revision 1.44
diff -u -p -r1.44 in6_var.h
--- netinet6/in6_var.h  24 Oct 2013 11:31:43 -  1.44
+++ netinet6/in6_var.h  20 Nov 2013 09:16:48 -
@@ -528,8 +528,7 @@ struct  in6_multi *in6_addmulti(struct in
 void   in6_delmulti(struct in6_multi *);
 struct in6_multi_mship *in6_joingroup(struct ifnet *, struct in6_addr *, int 
*);
 intin6_leavegroup(struct in6_multi_mship *);
-intin6_control(struct socket *, u_long, caddr_t, struct ifnet *,
-   struct proc *);
+intin6_control(struct socket *, u_long, caddr_t, struct ifnet *);
 intin6_update_ifa(struct ifnet *, struct in6_aliasreq *,

Re: initial i217/i218 Haswell Ethernet support for em(4)

2013-11-20 Thread Mark Kettenis
 Date: Wed, 20 Nov 2013 14:45:38 +1100
 From: Jonathan Gray j...@jsg.id.au
 
 On Tue, Nov 19, 2013 at 01:09:32AM +1100, Jonathan Gray wrote:
  On Sat, Nov 16, 2013 at 10:42:05AM +0100, Markus Hennecke wrote:
   On Sat, 9 Nov 2013, Jonathan Gray wrote:
   
This adds the initial bits for the i217/i218 PHY and the
Lynx Point PCH found in Haswell systems.

Doesn't include the new workarounds yet and follows
the pch2/82579 paths for now but this seems to be enough
to make a desktop machine with I217-LM work
   
   This fails for the I217-V with an EEPROM invalid checksum error:
  
  Here is a revised diff with different eeprom bank logic
  from FreeBSD as suggested by Masanobu SAITOH:
 
 And here is a version that should fix gigabit autoneg:

Time to get this tested on all other em(4) types.

 Index: if_em.c
 ===
 RCS file: /cvs/src/sys/dev/pci/if_em.c,v
 retrieving revision 1.271
 diff -u -p -r1.271 if_em.c
 --- if_em.c   15 Nov 2013 14:53:03 -  1.271
 +++ if_em.c   16 Nov 2013 09:35:41 -
 @@ -130,6 +130,10 @@ const struct pci_matchid em_devices[] = 
   { PCI_VENDOR_INTEL, PCI_PRODUCT_INTEL_82578DM },
   { PCI_VENDOR_INTEL, PCI_PRODUCT_INTEL_82579LM },
   { PCI_VENDOR_INTEL, PCI_PRODUCT_INTEL_82579V },
 + { PCI_VENDOR_INTEL, PCI_PRODUCT_INTEL_I217_LM },
 + { PCI_VENDOR_INTEL, PCI_PRODUCT_INTEL_I217_V },
 + { PCI_VENDOR_INTEL, PCI_PRODUCT_INTEL_I218_LM },
 + { PCI_VENDOR_INTEL, PCI_PRODUCT_INTEL_I218_V },
   { PCI_VENDOR_INTEL, PCI_PRODUCT_INTEL_82580_COPPER },
   { PCI_VENDOR_INTEL, PCI_PRODUCT_INTEL_82580_FIBER },
   { PCI_VENDOR_INTEL, PCI_PRODUCT_INTEL_82580_SERDES },
 @@ -788,6 +792,7 @@ em_init(void *arg)
   break;
   case em_pchlan:
   case em_pch2lan:
 + case em_pch_lpt:
   pba = E1000_PBA_26K;
   break;
   default:
 @@ -1626,7 +1631,8 @@ em_allocate_pci_resources(struct em_soft
   sc-hw.mac_type == em_ich9lan ||
   sc-hw.mac_type == em_ich10lan ||
   sc-hw.mac_type == em_pchlan ||
 - sc-hw.mac_type == em_pch2lan) {
 + sc-hw.mac_type == em_pch2lan ||
 + sc-hw.mac_type == em_pch_lpt) {
   val = pci_conf_read(pa-pa_pc, pa-pa_tag, EM_FLASH);
   if (PCI_MAPREG_TYPE(val) != PCI_MAPREG_TYPE_MEM) {
   printf(: flash is not mem space\n);
 Index: if_em_hw.c
 ===
 RCS file: /cvs/src/sys/dev/pci/if_em_hw.c,v
 retrieving revision 1.73
 diff -u -p -r1.73 if_em_hw.c
 --- if_em_hw.c7 Aug 2013 01:06:34 -   1.73
 +++ if_em_hw.c20 Nov 2013 03:32:03 -
 @@ -179,6 +179,7 @@ int32_t   em_get_pcs_speed_and_duplex_825
  uint16_t *);
  int32_t  em_set_eee_i350(struct em_hw *);
  int32_t  em_set_eee_pchlan(struct em_hw *);
 +int32_t  em_valid_nvm_bank_detect_ich8lan(struct em_hw *, 
 uint32_t *);
  
  
  /* IGP cable length table */
 @@ -255,6 +256,9 @@ em_set_phy_type(struct em_hw *hw)
   case I82579_E_PHY_ID:
   hw-phy_type = em_phy_82579;
   break;
 + case I217_E_PHY_ID:
 + hw-phy_type = em_phy_i217;
 + break;
   case I82580_I_PHY_ID:
   case I350_I_PHY_ID:
   hw-phy_type = em_phy_82580;
 @@ -568,6 +572,12 @@ em_set_mac_type(struct em_hw *hw)
   case E1000_DEV_ID_PCH2_LV_V:
   hw-mac_type = em_pch2lan;
   break;
 + case E1000_DEV_ID_PCH_LPT_I217_LM:
 + case E1000_DEV_ID_PCH_LPT_I217_V:
 + case E1000_DEV_ID_PCH_LPTLP_I218_LM:
 + case E1000_DEV_ID_PCH_LPTLP_I218_V:
 + hw-mac_type = em_pch_lpt;
 + break;
   case E1000_DEV_ID_EP80579_LAN_1:
   hw-mac_type = em_icp_;
   hw-icp__port_num = 0;
 @@ -591,6 +601,7 @@ em_set_mac_type(struct em_hw *hw)
   case em_ich10lan:
   case em_pchlan:
   case em_pch2lan:
 + case em_pch_lpt:
   hw-swfwhw_semaphore_present = TRUE;
   hw-asf_firmware_present = TRUE;
   break;
 @@ -682,6 +693,7 @@ em_set_media_type(struct em_hw *hw)
   case em_ich10lan:
   case em_pchlan:
   case em_pch2lan:
 + case em_pch_lpt:
   case em_82573:
   case em_82574:
   /*
 @@ -838,6 +850,7 @@ em_reset_hw(struct em_hw *hw)
   case em_ich10lan:
   case em_pchlan:
   case em_pch2lan:
 + case em_pch_lpt:
   if (!hw-phy_reset_disable 
   em_check_phy_reset_block(hw) == E1000_SUCCESS) {
   /*
 @@ -973,7 +986,8 @@ em_reset_hw(struct em_hw *hw)
   hw-mac_type == em_ich9lan ||
   hw-mac_type == em_ich10lan ||
   hw-mac_type == em_pchlan ||
 - hw-mac_type == em_pch2lan) {
 + hw-mac_type == 

Re: sdmmc update

2013-11-20 Thread Stefan Sperling
On Thu, Nov 07, 2013 at 08:10:16PM +0100, Stefan Sperling wrote:
 On Thu, Nov 07, 2013 at 08:06:11PM +0100, Sylvestre Gallon wrote:
  +int
  +rtsx_bus_width(sdmmc_chipset_handle_t sch, int width)
  +{
  +   struct rtsx_softc *sc = sch;
  +
  +   return (rtsx_set_bus_width(sc, width));
 
 rtsx_set_bus_width() currently never returns an error so you'll
 need to tweak it as well.
 
 I will test your diff on rtsx and sdhc.

This diff works with sdhc and provides an increase in read performance.

Unfortunately, the diff breaks rtsx. The disk manages to attach after
a CRC error, but it cannot be read. We'll need to figure this out.

rtsx0: CRC error
sdmmc0: SD_SEND_SCR send failed.
sdmmc0: mem init failed
scsibus3 at sdmmc0: 2 targets, initiator 0
sd2 at scsibus3 targ 1 lun 0: SD/MMC, Drive #01,  SCSI2 0/direct fixed
sd2: 945MB, 512 bytes/sector, 1935360 sectors
sd2 detached
scsibus3 detached



Re: Kill IA_SIN()

2013-11-20 Thread Stuart Henderson
On 2013/11/20 10:10, Martin Pieuchot wrote:
 This one is not under _KERNEL but it's used at only one place and
 if a port use it, it should probably define it by itself.

This is used in at least kde-workspace and embedded copies of slirp
(the ones I know about so far are emulators/qemu and emulators/BasiliskII).

It's easy enough to patch existing users in ports now, but removing it
makes things more difficult for people porting something else later
which happens to use it - so I wonder what's the reasoning behind this?
(if it's a namespace issue, I don't think that's valid given that it's
widely available - xnu, solaris, free/net/open/dfbsd and derivatives).

btw if people want a quick way to search for things in common source code,
http://codesearch.debian.net/ is useful.



remove disksort()

2013-11-20 Thread David Gwynne
the subject says it all really. this is sort of inspired by 5d2ecd5224
in bitrig except this brings all the architectures and device drivers
forward (i didnt get to delete any to prepare for this), and maintains
an algorithm for trying to order io on spinning rust (nscan).

ok?

Index: kern/kern_bufq.c
===
RCS file: /cvs/src/sys/kern/kern_bufq.c,v
retrieving revision 1.25
diff -u -p -r1.25 kern_bufq.c
--- kern/kern_bufq.c10 Apr 2013 01:35:55 -  1.25
+++ kern/kern_bufq.c20 Nov 2013 10:02:47 -
@@ -42,13 +42,6 @@ struct bufq_impl {
int  (*impl_peek)(void *);
 };
 
-void   *bufq_disksort_create(void);
-voidbufq_disksort_destroy(void *);
-voidbufq_disksort_queue(void *, struct buf *);
-struct buf *bufq_disksort_dequeue(void *);
-voidbufq_disksort_requeue(void *, struct buf *);
-int bufq_disksort_peek(void *);
-
 void   *bufq_fifo_create(void);
 voidbufq_fifo_destroy(void *);
 voidbufq_fifo_queue(void *, struct buf *);
@@ -65,14 +58,6 @@ int   bufq_nscan_peek(void *);
 
 const struct bufq_impl bufq_impls[BUFQ_HOWMANY] = {
{
-   bufq_disksort_create,
-   bufq_disksort_destroy,
-   bufq_disksort_queue,
-   bufq_disksort_dequeue,
-   bufq_disksort_requeue,
-   bufq_disksort_peek
-   },
-   {
bufq_fifo_create,
bufq_fifo_destroy,
bufq_fifo_queue,
@@ -323,61 +308,6 @@ bufq_restart(void)
mtx_leave(bufqs_mtx);
 }
 
-/*
- * disksort implementation.
- */
-
-void *
-bufq_disksort_create(void)
-{
-   return (malloc(sizeof(struct buf), M_DEVBUF, M_NOWAIT | M_ZERO));
-}
-
-void
-bufq_disksort_destroy(void *data)
-{
-   free(data, M_DEVBUF);
-}
-
-void
-bufq_disksort_queue(void *data, struct buf *bp)
-{
-   disksort((struct buf *)data, bp);
-}
-
-struct buf *
-bufq_disksort_dequeue(void *data)
-{
-   struct buf  *bufq = data;
-   struct buf  *bp;
-
-   bp = bufq-b_actf;
-   if (bp != NULL)
-   bufq-b_actf = bp-b_actf;
-   if (bufq-b_actf == NULL)
-   bufq-b_actb = bufq-b_actf;
-
-   return (bp);
-}
-
-void
-bufq_disksort_requeue(void *data, struct buf *bp)
-{
-   struct buf  *bufq = data;
-
-   bp-b_actf = bufq-b_actf;
-   bufq-b_actf = bp;
-   if (bp-b_actf == NULL)
-   bufq-b_actb = bp-b_actf;
-}
-
-int
-bufq_disksort_peek(void *data)
-{
-   struct buf  *bufq = data;
-
-   return (bufq-b_actf != NULL);
-}
 
 /*
  * fifo implementation
@@ -444,9 +374,7 @@ bufq_fifo_peek(void *data)
  * nscan implementation
  */
 
-#define BUF_INORDER(ba, bb) \
-(((ba)-b_cylinder  (bb)-b_cylinder) || \
-((ba)-b_cylinder == (bb)-b_cylinder  (ba)-b_blkno  (bb)-b_blkno))
+#define BUF_INORDER(ba, bb) ((ba)-b_blkno  (bb)-b_blkno)
 
 #define dsentries b_bufq.bufq_data_nscan.bqf_entries
 
Index: kern/subr_disk.c
===
RCS file: /cvs/src/sys/kern/subr_disk.c,v
retrieving revision 1.158
diff -u -p -r1.158 subr_disk.c
--- kern/subr_disk.c18 Nov 2013 17:45:01 -  1.158
+++ kern/subr_disk.c20 Nov 2013 10:02:47 -
@@ -91,99 +91,6 @@ void sr_map_root(void);
 void disk_attach_callback(void *, void *);
 
 /*
- * Seek sort for disks.  We depend on the driver which calls us using b_resid
- * as the current cylinder number.
- *
- * The argument ap structure holds a b_actf activity chain pointer on which we
- * keep two queues, sorted in ascending cylinder order.  The first queue holds
- * those requests which are positioned after the current cylinder (in the first
- * request); the second holds requests which came in after their cylinder 
number
- * was passed.  Thus we implement a one way scan, retracting after reaching the
- * end of the drive to the first request on the second queue, at which time it
- * becomes the first queue.
- *
- * A one-way scan is natural because of the way UNIX read-ahead blocks are
- * allocated.
- */
-
-void
-disksort(struct buf *ap, struct buf *bp)
-{
-   struct buf *bq;
-
-   /* If the queue is empty, then it's easy. */
-   if (ap-b_actf == NULL) {
-   bp-b_actf = NULL;
-   ap-b_actf = bp;
-   return;
-   }
-
-   /*
-* If we lie after the first (currently active) request, then we
-* must locate the second request list and add ourselves to it.
-*/
-   bq = ap-b_actf;
-   if (bp-b_cylinder  bq-b_cylinder) {
-   while (bq-b_actf) {
-   /*
-* Check for an ``inversion'' in the normally ascending
-* cylinder numbers, indicating the start of the second
-* request list.
-  

remove the b_cylinder #define to b_resid in struct buf

2013-11-20 Thread David Gwynne
most hardware and most of the kernel has given up on cylinders
meaning anything, we generally work in logical block addresses now.
now that disksort is (almost) gone it is no longer used for anything
meaningful except in floppy disk drivers.

the #define b_cylinder b_resid alias has always grated on me, so
this removes it. the big fallout is in the fd drivers, so they now
keep track of how much of a buf theyve handled by using resid to
store a resid.

ive tested this on sys/dev/isa/fd.c, and compiled it on sparc64. a
compile on sparc would be appreciated, and any further hardware
tests.

this relies on the previous diff i sent out to remove disksort
because that previous diff removed a use of b_cylinder. it shouldnt
compile with that b_cylinder in there.

ok?

Index: arch/amd64/amd64/dkcsum.c
===
RCS file: /cvs/src/sys/arch/amd64/amd64/dkcsum.c,v
retrieving revision 1.19
diff -u -p -r1.19 dkcsum.c
--- arch/amd64/amd64/dkcsum.c   27 Jun 2011 01:14:24 -  1.19
+++ arch/amd64/amd64/dkcsum.c   20 Nov 2013 10:02:46 -
@@ -116,7 +116,6 @@ dkcsumattach(void)
bp-b_error = 0; /* B_ERROR and b_error may have stale data. */
CLR(bp-b_flags, B_READ | B_WRITE | B_DONE | B_ERROR);
SET(bp-b_flags, B_BUSY | B_READ | B_RAW);
-   bp-b_cylinder = 0;
(*bdsw-d_strategy)(bp);
if ((error = biowait(bp))) {
/* XXX What to do here? */
Index: arch/i386/i386/dkcsum.c
===
RCS file: /cvs/src/sys/arch/i386/i386/dkcsum.c,v
retrieving revision 1.30
diff -u -p -r1.30 dkcsum.c
--- arch/i386/i386/dkcsum.c 27 Jun 2011 01:14:24 -  1.30
+++ arch/i386/i386/dkcsum.c 20 Nov 2013 10:02:46 -
@@ -116,7 +116,6 @@ dkcsumattach(void)
bp-b_error = 0; /* B_ERROR and b_error may have stale data. */
CLR(bp-b_flags, B_READ | B_WRITE | B_DONE | B_ERROR);
SET(bp-b_flags, B_BUSY | B_READ | B_RAW);
-   bp-b_cylinder = 0;
(*bdsw-d_strategy)(bp);
if ((error = biowait(bp))) {
/* XXX What to do here? */
Index: arch/sparc/dev/fd.c
===
RCS file: /cvs/src/sys/arch/sparc/dev/fd.c,v
retrieving revision 1.87
diff -u -p -r1.87 fd.c
--- arch/sparc/dev/fd.c 18 Nov 2013 01:56:35 -  1.87
+++ arch/sparc/dev/fd.c 20 Nov 2013 10:02:46 -
@@ -705,6 +705,7 @@ fdstrategy(bp)
if (bp-b_bcount == 0)
goto done;
 
+   bp-b_resid = bp-b_count;
sz = howmany(bp-b_bcount, DEV_BSIZE);
 
if (bp-b_blkno + sz  (fd-sc_type-size * DEV_BSIZE) / FD_BSIZE(fd)) {
@@ -712,7 +713,6 @@ fdstrategy(bp)
- bp-b_blkno;
if (sz == 0) {
/* If exactly at end of disk, return EOF. */
-   bp-b_resid = bp-b_bcount;
goto done;
}
if (sz  0) {
@@ -724,14 +724,11 @@ fdstrategy(bp)
bp-b_bcount = sz  DEV_BSHIFT;
}
 
-   bp-b_cylinder = (bp-b_blkno * DEV_BSIZE) /
-   (FD_BSIZE(fd) * fd-sc_type-seccyl);
-
 #ifdef FD_DEBUG
if (fdc_debug  1)
-   printf(fdstrategy: b_blkno %lld b_bcount %ld blkno %lld 
-   cylin %ld\n, (long long)bp-b_blkno, bp-b_bcount,
-   (long long)fd-sc_blkno, bp-b_cylinder);
+   printf(fdstrategy: b_blkno %lld b_bcount %ld blkno %lld\n,
+   (long long)bp-b_blkno, bp-b_bcount,
+   (long long)fd-sc_blkno);
 #endif
 
/* Queue transfer */
@@ -802,7 +799,6 @@ fdfinish(fd, bp)
TAILQ_INSERT_TAIL(fdc-sc_drives, fd, sc_drivechain);
}
 
-   bp-b_resid = fd-sc_bcount;
biodone(bp);
/* turn off motor 5s from now */
timeout_add_sec(fd-fd_motor_off_to, 5);
@@ -1262,11 +1258,10 @@ fdcstate(fdc)
 
struct fd_softc *fd;
struct buf *bp;
-   int read, head, sec, nblks;
+   int read, head, sec, nblks, cylin;
struct fd_type *type;
struct fd_formb *finfo = NULL;
 
-
if (fdc-sc_istatus == FDC_ISTATUS_ERROR) {
/* Prevent loop if the reset sequence produces errors */
if (fdc-sc_state != RESETCOMPLETE 
@@ -1295,6 +1290,9 @@ loop:
goto loop;
}
 
+   cylin = ((bp-b_blkno * DEV_BSIZE) + (bp-b_bcount - bp-b_resid)) /
+   (FD_BSIZE(fd) * fd-sc_type-seccyl);
+
if (bp-b_flags  B_FORMAT)
finfo = (struct fd_formb *)bp-b_data;
 
@@ -1336,12 +1334,12 @@ loop:
doseek:
if ((fdc-sc_flags  FDC_EIS) 
(bp-b_flags  B_FORMAT) == 0) {
-   fd-sc_cylin = bp-b_cylinder;
+   fd-sc_cylin = cylin;
/* We use 

Re: Kill IA_SIN()

2013-11-20 Thread Martin Pieuchot
On 20/11/13(Wed) 10:16, Stuart Henderson wrote:
 On 2013/11/20 10:10, Martin Pieuchot wrote:
  This one is not under _KERNEL but it's used at only one place and
  if a port use it, it should probably define it by itself.
 
 This is used in at least kde-workspace and embedded copies of slirp
 (the ones I know about so far are emulators/qemu and emulators/BasiliskII).
 
 It's easy enough to patch existing users in ports now, but removing it
 makes things more difficult for people porting something else later
 which happens to use it - so I wonder what's the reasoning behind this?
 (if it's a namespace issue, I don't think that's valid given that it's
 widely available - xnu, solaris, free/net/open/dfbsd and derivatives).

Simply cleaning, to unify our code base.

 btw if people want a quick way to search for things in common source code,
 http://codesearch.debian.net/ is useful.

Nice tool, thanks for the hint.

So here's an updated version that keeps the define for userland, would
it be ok?

Index: netinet/if_ether.c
===
RCS file: /home/ncvs/src/sys/netinet/if_ether.c,v
retrieving revision 1.111
diff -u -p -r1.111 if_ether.c
--- netinet/if_ether.c  11 Nov 2013 09:15:34 -  1.111
+++ netinet/if_ether.c  20 Nov 2013 10:27:12 -
@@ -868,8 +868,8 @@ arp_ifinit(struct arpcom *ac, struct ifa
 
/* Warn the user if another station has this IP address. */
arprequest(ac-ac_if,
-   (IA_SIN(ifa)-sin_addr.s_addr),
-   (IA_SIN(ifa)-sin_addr.s_addr),
+   satosin(ifa-ifa_addr)-sin_addr.s_addr,
+   satosin(ifa-ifa_addr)-sin_addr.s_addr,
ac-ac_enaddr);
ifa-ifa_rtrequest = arp_rtrequest;
ifa-ifa_flags |= RTF_CLONING;
Index: netinet/in_var.h
===
RCS file: /home/ncvs/src/sys/netinet/in_var.h,v
retrieving revision 1.26
diff -u -p -r1.26 in_var.h
--- netinet/in_var.h23 Oct 2013 15:12:42 -  1.26
+++ netinet/in_var.h20 Nov 2013 10:27:12 -
@@ -73,12 +73,6 @@ struct   in_aliasreq {
 #defineifra_broadaddr  ifra_dstaddr
struct  sockaddr_in ifra_mask;
 };
-/*
- * Given a pointer to an in_ifaddr (ifaddr),
- * return a pointer to the addr as a sockaddr_in.
- */
-#defineIA_SIN(ia) struct in_ifaddr *)(ia))-ia_addr))
-
 
 #ifdef _KERNEL
 TAILQ_HEAD(in_ifaddrhead, in_ifaddr);
@@ -179,6 +173,14 @@ struct in_multi *in_addmulti(struct in_a
 void   in_delmulti(struct in_multi *);
 void   in_ifscrub(struct ifnet *, struct in_ifaddr *);
 intin_control(struct socket *, u_long, caddr_t, struct ifnet *);
-#endif
+#else /* _KERNEL */
+
+/*
+ * Given a pointer to an in_ifaddr (ifaddr),
+ * return a pointer to the addr as a sockaddr_in.
+ */
+#defineIA_SIN(ia) struct in_ifaddr *)(ia))-ia_addr))
+
+#endif /* _KERNEL */
 
 #endif /* _NETINET_IN_VAR_H_ */



pfsync(4) mangles prio in master/slave setup

2013-11-20 Thread Alexey Suslikov
Hi.

This is on 5.4-stable. Trivial master/slave carp(4) setup. vlan(4) is to
make picture clear wrt prio.

Test 1 (without using match).

pf.conf (BOX1 and BOX2).

ext_if=vlan101
dmz_if=vlan10
pf_sync=vlan50
block log all
pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
pass quick on $dmz_if inet proto icmp all icmp-type echoreq set prio 5
pass quick on $dmz_if
pass out quick on $ext_if inet proto icmp all icmp-type echoreq set prio 5
pass out quick on $ext_if

BOX1 is Master, BOX2 is Slave.

BOX1:
00:07:36.108948 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo request
00:07:36.109281 802.1Q vid 101 pri 5 X.X.185.145  X.X.36.14: icmp: echo request
00:07:36.110013 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo reply
00:07:36.110030 802.1Q vid 10 pri 5 X.X.36.14  X.X.185.145: icmp: echo reply

BOX1 is Slave, BOX2 is Master.

BOX2:
00:12:43.981979 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo request
00:12:43.982013 802.1Q vid 101 pri 0 X.X.185.145  X.X.36.14: icmp: echo request
00:12:43.982693 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo reply
00:12:43.982713 802.1Q vid 10 pri 0 X.X.36.14  X.X.185.145: icmp: echo reply

Test 2 (using match).

pf.conf (BOX1 and BOX2).

ext_if=vlan101
dmz_if=vlan10
pf_sync=vlan50
block log all
match quick on { $ext_if, $dmz_if } inet proto icmp all icmp-type
echoreq set prio 5
pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
pass quick on $dmz_if
pass out quick on $ext_if

BOX1 is Master, BOX2 is Slave.

BOX1:
00:27:47.442820 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo request
00:27:47.442839 802.1Q vid 101 pri 5 X.X.185.145  X.X.36.14: icmp: echo request
00:27:48.468709 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo reply
00:27:47.443523 802.1Q vid 10 pri 5 X.X.36.14  X.X.185.145: icmp: echo reply

BOX1 is Slave, BOX2 is Master.

BOX2:
00:30:35.317329 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo request
00:30:35.317354 802.1Q vid 101 pri 0 X.X.185.145  X.X.36.14: icmp: echo request
00:30:35.318065 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo reply
00:30:35.318084 802.1Q vid 10 pri 0 X.X.36.14  X.X.185.145: icmp: echo reply

Maybe ICMP is not a sort of traffic which makes difference, but think
about TCP ACKs are prioritized. Switching to Slave in production setup
makes things *REALLY* bad.

Should I configure something, or this is an issue?

(Speaking of pfsync code, I'm unable to find where prio is set inside
pfsync_state_import).

Thanks,
Alexey



Re: pfsync(4) mangles prio in master/slave setup

2013-11-20 Thread Mike Belopuhov
could you please add more description to this report since
it's very hard to follow and interpret your mail.

On 20 November 2013 12:11, Alexey Suslikov alexey.susli...@gmail.com wrote:
 Hi.

 This is on 5.4-stable. Trivial master/slave carp(4) setup. vlan(4) is to
 make picture clear wrt prio.

 Test 1 (without using match).

 pf.conf (BOX1 and BOX2).

 ext_if=vlan101
 dmz_if=vlan10
 pf_sync=vlan50
 block log all
 pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
 pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
 pass quick on $dmz_if inet proto icmp all icmp-type echoreq set prio 5
 pass quick on $dmz_if
 pass out quick on $ext_if inet proto icmp all icmp-type echoreq set prio 5
 pass out quick on $ext_if

 BOX1 is Master, BOX2 is Slave.

 BOX1:
 00:07:36.108948 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:07:36.109281 802.1Q vid 101 pri 5 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:07:36.110013 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo reply
 00:07:36.110030 802.1Q vid 10 pri 5 X.X.36.14  X.X.185.145: icmp: echo reply

 BOX1 is Slave, BOX2 is Master.

 BOX2:
 00:12:43.981979 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:12:43.982013 802.1Q vid 101 pri 0 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:12:43.982693 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo reply
 00:12:43.982713 802.1Q vid 10 pri 0 X.X.36.14  X.X.185.145: icmp: echo reply

 Test 2 (using match).

 pf.conf (BOX1 and BOX2).

 ext_if=vlan101
 dmz_if=vlan10
 pf_sync=vlan50
 block log all
 match quick on { $ext_if, $dmz_if } inet proto icmp all icmp-type
 echoreq set prio 5
 pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
 pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
 pass quick on $dmz_if
 pass out quick on $ext_if

 BOX1 is Master, BOX2 is Slave.

 BOX1:
 00:27:47.442820 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:27:47.442839 802.1Q vid 101 pri 5 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:27:48.468709 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo reply
 00:27:47.443523 802.1Q vid 10 pri 5 X.X.36.14  X.X.185.145: icmp: echo reply

 BOX1 is Slave, BOX2 is Master.

 BOX2:
 00:30:35.317329 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:30:35.317354 802.1Q vid 101 pri 0 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:30:35.318065 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo reply
 00:30:35.318084 802.1Q vid 10 pri 0 X.X.36.14  X.X.185.145: icmp: echo reply

 Maybe ICMP is not a sort of traffic which makes difference, but think
 about TCP ACKs are prioritized. Switching to Slave in production setup
 makes things *REALLY* bad.

 Should I configure something, or this is an issue?

 (Speaking of pfsync code, I'm unable to find where prio is set inside
 pfsync_state_import).

 Thanks,
 Alexey




Re: pfsync(4) mangles prio in master/slave setup

2013-11-20 Thread Alexey Suslikov
On Wed, Nov 20, 2013 at 1:32 PM, Mike Belopuhov m...@belopuhov.com wrote:
 could you please add more description to this report since
 it's very hard to follow and interpret your mail.

basically, when setup switches to slave, packets (matching
given state) have wrong prio set (wrong means they were
right when state was created on master).

I will be glad to provide more information/tests/etc - just say
what is needed.


 On 20 November 2013 12:11, Alexey Suslikov alexey.susli...@gmail.com wrote:
 Hi.

 This is on 5.4-stable. Trivial master/slave carp(4) setup. vlan(4) is to
 make picture clear wrt prio.

 Test 1 (without using match).

 pf.conf (BOX1 and BOX2).

 ext_if=vlan101
 dmz_if=vlan10
 pf_sync=vlan50
 block log all
 pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
 pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
 pass quick on $dmz_if inet proto icmp all icmp-type echoreq set prio 5
 pass quick on $dmz_if
 pass out quick on $ext_if inet proto icmp all icmp-type echoreq set prio 5
 pass out quick on $ext_if

 BOX1 is Master, BOX2 is Slave.

 BOX1:
 00:07:36.108948 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:07:36.109281 802.1Q vid 101 pri 5 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:07:36.110013 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
 reply
 00:07:36.110030 802.1Q vid 10 pri 5 X.X.36.14  X.X.185.145: icmp: echo reply

 BOX1 is Slave, BOX2 is Master.

 BOX2:
 00:12:43.981979 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:12:43.982013 802.1Q vid 101 pri 0 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:12:43.982693 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
 reply
 00:12:43.982713 802.1Q vid 10 pri 0 X.X.36.14  X.X.185.145: icmp: echo reply

 Test 2 (using match).

 pf.conf (BOX1 and BOX2).

 ext_if=vlan101
 dmz_if=vlan10
 pf_sync=vlan50
 block log all
 match quick on { $ext_if, $dmz_if } inet proto icmp all icmp-type
 echoreq set prio 5
 pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
 pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
 pass quick on $dmz_if
 pass out quick on $ext_if

 BOX1 is Master, BOX2 is Slave.

 BOX1:
 00:27:47.442820 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:27:47.442839 802.1Q vid 101 pri 5 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:27:48.468709 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
 reply
 00:27:47.443523 802.1Q vid 10 pri 5 X.X.36.14  X.X.185.145: icmp: echo reply

 BOX1 is Slave, BOX2 is Master.

 BOX2:
 00:30:35.317329 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:30:35.317354 802.1Q vid 101 pri 0 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:30:35.318065 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
 reply
 00:30:35.318084 802.1Q vid 10 pri 0 X.X.36.14  X.X.185.145: icmp: echo reply

 Maybe ICMP is not a sort of traffic which makes difference, but think
 about TCP ACKs are prioritized. Switching to Slave in production setup
 makes things *REALLY* bad.

 Should I configure something, or this is an issue?

 (Speaking of pfsync code, I'm unable to find where prio is set inside
 pfsync_state_import).

 Thanks,
 Alexey




Re: pfsync(4) mangles prio in master/slave setup

2013-11-20 Thread Alexey Suslikov
On Wed, Nov 20, 2013 at 1:38 PM, Alexey Suslikov
alexey.susli...@gmail.com wrote:
 On Wed, Nov 20, 2013 at 1:32 PM, Mike Belopuhov m...@belopuhov.com wrote:
 could you please add more description to this report since
 it's very hard to follow and interpret your mail.

 basically, when setup switches to slave, packets (matching
 given state) have wrong prio set (wrong means they were
 right when state was created on master).

 I will be glad to provide more information/tests/etc - just say
 what is needed.


 On 20 November 2013 12:11, Alexey Suslikov alexey.susli...@gmail.com wrote:
 Hi.

 This is on 5.4-stable. Trivial master/slave carp(4) setup. vlan(4) is to
 make picture clear wrt prio.

 Test 1 (without using match).

 pf.conf (BOX1 and BOX2).

 ext_if=vlan101
 dmz_if=vlan10
 pf_sync=vlan50
 block log all
 pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
 pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
 pass quick on $dmz_if inet proto icmp all icmp-type echoreq set prio 5
 pass quick on $dmz_if
 pass out quick on $ext_if inet proto icmp all icmp-type echoreq set prio 5
 pass out quick on $ext_if

 BOX1 is Master, BOX2 is Slave.

 BOX1:
 00:07:36.108948 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:07:36.109281 802.1Q vid 101 pri 5 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:07:36.110013 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
 reply
 00:07:36.110030 802.1Q vid 10 pri 5 X.X.36.14  X.X.185.145: icmp: echo 
 reply

 BOX1 is Slave, BOX2 is Master.

 BOX2:
 00:12:43.981979 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:12:43.982013 802.1Q vid 101 pri 0 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:12:43.982693 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
 reply
 00:12:43.982713 802.1Q vid 10 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
 reply

While on Slave, having all zeroes prio in output path (echo request in
vlan101 and reply in vlan10), imo, indicates a state being crafted by
pfsync_state_import without a prio took in account.

In contrast to set_tos, min_ttl and such, pf_state_export too isn't doing
anything about prio, so I think prio neither exported to pfsync packet,
nor imported from.


 Test 2 (using match).

 pf.conf (BOX1 and BOX2).

 ext_if=vlan101
 dmz_if=vlan10
 pf_sync=vlan50
 block log all
 match quick on { $ext_if, $dmz_if } inet proto icmp all icmp-type
 echoreq set prio 5
 pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
 pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
 pass quick on $dmz_if
 pass out quick on $ext_if

 BOX1 is Master, BOX2 is Slave.

 BOX1:
 00:27:47.442820 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:27:47.442839 802.1Q vid 101 pri 5 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:27:48.468709 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
 reply
 00:27:47.443523 802.1Q vid 10 pri 5 X.X.36.14  X.X.185.145: icmp: echo 
 reply

 BOX1 is Slave, BOX2 is Master.

 BOX2:
 00:30:35.317329 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:30:35.317354 802.1Q vid 101 pri 0 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:30:35.318065 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
 reply
 00:30:35.318084 802.1Q vid 10 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
 reply

 Maybe ICMP is not a sort of traffic which makes difference, but think
 about TCP ACKs are prioritized. Switching to Slave in production setup
 makes things *REALLY* bad.

 Should I configure something, or this is an issue?

 (Speaking of pfsync code, I'm unable to find where prio is set inside
 pfsync_state_import).

 Thanks,
 Alexey




Re: pfsync(4) mangles prio in master/slave setup

2013-11-20 Thread Florian Obser
On Wed, Nov 20, 2013 at 01:38:11PM +0200, Alexey Suslikov wrote:
 On Wed, Nov 20, 2013 at 1:32 PM, Mike Belopuhov m...@belopuhov.com wrote:
  could you please add more description to this report since
  it's very hard to follow and interpret your mail.
 
 basically, when setup switches to slave, packets (matching
 given state) have wrong prio set (wrong means they were
 right when state was created on master).
 
 I will be glad to provide more information/tests/etc - just say
 what is needed.

Do you have the same ruleset checksum on both machines? check with
pfctl -vs info | fgrep Checksum

 
 
  On 20 November 2013 12:11, Alexey Suslikov alexey.susli...@gmail.com 
  wrote:
  Hi.
 
  This is on 5.4-stable. Trivial master/slave carp(4) setup. vlan(4) is to
  make picture clear wrt prio.
 
  Test 1 (without using match).
 
  pf.conf (BOX1 and BOX2).
 
  ext_if=vlan101
  dmz_if=vlan10
  pf_sync=vlan50
  block log all
  pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
  pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
  pass quick on $dmz_if inet proto icmp all icmp-type echoreq set prio 5
  pass quick on $dmz_if
  pass out quick on $ext_if inet proto icmp all icmp-type echoreq set prio 5
  pass out quick on $ext_if
 
  BOX1 is Master, BOX2 is Slave.
 
  BOX1:
  00:07:36.108948 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
  request
  00:07:36.109281 802.1Q vid 101 pri 5 X.X.185.145  X.X.36.14: icmp: echo 
  request
  00:07:36.110013 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
  reply
  00:07:36.110030 802.1Q vid 10 pri 5 X.X.36.14  X.X.185.145: icmp: echo 
  reply
 
  BOX1 is Slave, BOX2 is Master.
 
  BOX2:
  00:12:43.981979 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
  request
  00:12:43.982013 802.1Q vid 101 pri 0 X.X.185.145  X.X.36.14: icmp: echo 
  request
  00:12:43.982693 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
  reply
  00:12:43.982713 802.1Q vid 10 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
  reply
 
  Test 2 (using match).
 
  pf.conf (BOX1 and BOX2).
 
  ext_if=vlan101
  dmz_if=vlan10
  pf_sync=vlan50
  block log all
  match quick on { $ext_if, $dmz_if } inet proto icmp all icmp-type
  echoreq set prio 5
  pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
  pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
  pass quick on $dmz_if
  pass out quick on $ext_if
 
  BOX1 is Master, BOX2 is Slave.
 
  BOX1:
  00:27:47.442820 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
  request
  00:27:47.442839 802.1Q vid 101 pri 5 X.X.185.145  X.X.36.14: icmp: echo 
  request
  00:27:48.468709 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
  reply
  00:27:47.443523 802.1Q vid 10 pri 5 X.X.36.14  X.X.185.145: icmp: echo 
  reply
 
  BOX1 is Slave, BOX2 is Master.
 
  BOX2:
  00:30:35.317329 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
  request
  00:30:35.317354 802.1Q vid 101 pri 0 X.X.185.145  X.X.36.14: icmp: echo 
  request
  00:30:35.318065 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
  reply
  00:30:35.318084 802.1Q vid 10 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
  reply
 
  Maybe ICMP is not a sort of traffic which makes difference, but think
  about TCP ACKs are prioritized. Switching to Slave in production setup
  makes things *REALLY* bad.
 
  Should I configure something, or this is an issue?
 
  (Speaking of pfsync code, I'm unable to find where prio is set inside
  pfsync_state_import).
 
  Thanks,
  Alexey
 
 

-- 
I'm not entirely sure you are real.



Re: pfsync(4) mangles prio in master/slave setup

2013-11-20 Thread Mike Belopuhov
On 20 November 2013 13:10, Alexey Suslikov alexey.susli...@gmail.com wrote:
 On Wed, Nov 20, 2013 at 1:38 PM, Alexey Suslikov
 alexey.susli...@gmail.com wrote:
 On Wed, Nov 20, 2013 at 1:32 PM, Mike Belopuhov m...@belopuhov.com wrote:
 could you please add more description to this report since
 it's very hard to follow and interpret your mail.

 basically, when setup switches to slave, packets (matching
 given state) have wrong prio set (wrong means they were
 right when state was created on master).

 I will be glad to provide more information/tests/etc - just say
 what is needed.


 On 20 November 2013 12:11, Alexey Suslikov alexey.susli...@gmail.com 
 wrote:
 Hi.

 This is on 5.4-stable. Trivial master/slave carp(4) setup. vlan(4) is to
 make picture clear wrt prio.

 Test 1 (without using match).

 pf.conf (BOX1 and BOX2).

 ext_if=vlan101
 dmz_if=vlan10
 pf_sync=vlan50
 block log all
 pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
 pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
 pass quick on $dmz_if inet proto icmp all icmp-type echoreq set prio 5
 pass quick on $dmz_if
 pass out quick on $ext_if inet proto icmp all icmp-type echoreq set prio 5
 pass out quick on $ext_if

 BOX1 is Master, BOX2 is Slave.

 BOX1:
 00:07:36.108948 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:07:36.109281 802.1Q vid 101 pri 5 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:07:36.110013 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
 reply
 00:07:36.110030 802.1Q vid 10 pri 5 X.X.36.14  X.X.185.145: icmp: echo 
 reply

 BOX1 is Slave, BOX2 is Master.

 BOX2:
 00:12:43.981979 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:12:43.982013 802.1Q vid 101 pri 0 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:12:43.982693 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
 reply
 00:12:43.982713 802.1Q vid 10 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
 reply

 While on Slave, having all zeroes prio in output path (echo request in
 vlan101 and reply in vlan10), imo, indicates a state being crafted by
 pfsync_state_import without a prio took in account.

 In contrast to set_tos, min_ttl and such, pf_state_export too isn't doing
 anything about prio, so I think prio neither exported to pfsync packet,
 nor imported from.


correct.  henning?


 Test 2 (using match).

 pf.conf (BOX1 and BOX2).

 ext_if=vlan101
 dmz_if=vlan10
 pf_sync=vlan50
 block log all
 match quick on { $ext_if, $dmz_if } inet proto icmp all icmp-type
 echoreq set prio 5
 pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
 pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
 pass quick on $dmz_if
 pass out quick on $ext_if

 BOX1 is Master, BOX2 is Slave.

 BOX1:
 00:27:47.442820 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:27:47.442839 802.1Q vid 101 pri 5 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:27:48.468709 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
 reply
 00:27:47.443523 802.1Q vid 10 pri 5 X.X.36.14  X.X.185.145: icmp: echo 
 reply

 BOX1 is Slave, BOX2 is Master.

 BOX2:
 00:30:35.317329 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:30:35.317354 802.1Q vid 101 pri 0 X.X.185.145  X.X.36.14: icmp: echo 
 request
 00:30:35.318065 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
 reply
 00:30:35.318084 802.1Q vid 10 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
 reply

 Maybe ICMP is not a sort of traffic which makes difference, but think
 about TCP ACKs are prioritized. Switching to Slave in production setup
 makes things *REALLY* bad.

 Should I configure something, or this is an issue?

 (Speaking of pfsync code, I'm unable to find where prio is set inside
 pfsync_state_import).

 Thanks,
 Alexey




Re: pfsync(4) mangles prio in master/slave setup

2013-11-20 Thread Alexey Suslikov
On Wed, Nov 20, 2013 at 2:15 PM, Florian Obser flor...@openbsd.org wrote:
 On Wed, Nov 20, 2013 at 01:38:11PM +0200, Alexey Suslikov wrote:
 On Wed, Nov 20, 2013 at 1:32 PM, Mike Belopuhov m...@belopuhov.com wrote:
  could you please add more description to this report since
  it's very hard to follow and interpret your mail.

 basically, when setup switches to slave, packets (matching
 given state) have wrong prio set (wrong means they were
 right when state was created on master).

 I will be glad to provide more information/tests/etc - just say
 what is needed.

 Do you have the same ruleset checksum on both machines? check with
 pfctl -vs info | fgrep Checksum

yes. checksums are same.



 
  On 20 November 2013 12:11, Alexey Suslikov alexey.susli...@gmail.com 
  wrote:
  Hi.
 
  This is on 5.4-stable. Trivial master/slave carp(4) setup. vlan(4) is to
  make picture clear wrt prio.
 
  Test 1 (without using match).
 
  pf.conf (BOX1 and BOX2).
 
  ext_if=vlan101
  dmz_if=vlan10
  pf_sync=vlan50
  block log all
  pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
  pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
  pass quick on $dmz_if inet proto icmp all icmp-type echoreq set prio 5
  pass quick on $dmz_if
  pass out quick on $ext_if inet proto icmp all icmp-type echoreq set prio 5
  pass out quick on $ext_if
 
  BOX1 is Master, BOX2 is Slave.
 
  BOX1:
  00:07:36.108948 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
  request
  00:07:36.109281 802.1Q vid 101 pri 5 X.X.185.145  X.X.36.14: icmp: echo 
  request
  00:07:36.110013 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
  reply
  00:07:36.110030 802.1Q vid 10 pri 5 X.X.36.14  X.X.185.145: icmp: echo 
  reply
 
  BOX1 is Slave, BOX2 is Master.
 
  BOX2:
  00:12:43.981979 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
  request
  00:12:43.982013 802.1Q vid 101 pri 0 X.X.185.145  X.X.36.14: icmp: echo 
  request
  00:12:43.982693 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
  reply
  00:12:43.982713 802.1Q vid 10 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
  reply
 
  Test 2 (using match).
 
  pf.conf (BOX1 and BOX2).
 
  ext_if=vlan101
  dmz_if=vlan10
  pf_sync=vlan50
  block log all
  match quick on { $ext_if, $dmz_if } inet proto icmp all icmp-type
  echoreq set prio 5
  pass quick on $pf_sync proto pfsync keep state (no-sync) set prio 7
  pass quick on { $ext_if, $dmz_if } proto carp keep state (no-sync)
  pass quick on $dmz_if
  pass out quick on $ext_if
 
  BOX1 is Master, BOX2 is Slave.
 
  BOX1:
  00:27:47.442820 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
  request
  00:27:47.442839 802.1Q vid 101 pri 5 X.X.185.145  X.X.36.14: icmp: echo 
  request
  00:27:48.468709 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
  reply
  00:27:47.443523 802.1Q vid 10 pri 5 X.X.36.14  X.X.185.145: icmp: echo 
  reply
 
  BOX1 is Slave, BOX2 is Master.
 
  BOX2:
  00:30:35.317329 802.1Q vid 10 pri 3 X.X.185.145  X.X.36.14: icmp: echo 
  request
  00:30:35.317354 802.1Q vid 101 pri 0 X.X.185.145  X.X.36.14: icmp: echo 
  request
  00:30:35.318065 802.1Q vid 101 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
  reply
  00:30:35.318084 802.1Q vid 10 pri 0 X.X.36.14  X.X.185.145: icmp: echo 
  reply
 
  Maybe ICMP is not a sort of traffic which makes difference, but think
  about TCP ACKs are prioritized. Switching to Slave in production setup
  makes things *REALLY* bad.
 
  Should I configure something, or this is an issue?
 
  (Speaking of pfsync code, I'm unable to find where prio is set inside
  pfsync_state_import).
 
  Thanks,
  Alexey
 


 --
 I'm not entirely sure you are real.



Re: linebuffering diff for tr(1)

2013-11-20 Thread Christian Weisgerber
Jan Klemkow j.klem...@wemelug.de wrote:

 here is a diff that adds optional linebuffering to tr(1) with command
 line switch -u like in sed(1).  I need this to remove '\r' characters
 from a continues input steam which lines have to be there immediately.

It's really odd to make tr output line-buffered, since tr doesn't
process lines to begin with.  FreeBSD's tr -u is unbuffered.

Maybe, instead of adding such flags to more and more utilities, we
should have a general unbuffer tool that runs things through a pty.
(And I'd be surprised if something like this wasn't already floating
around.)

-- 
Christian naddy Weisgerber  na...@mips.inka.de



Re: linebuffering diff for tr(1)

2013-11-20 Thread Shawn K. Quinn
On Tue, Nov 19, 2013, at 03:10 PM, Theo de Raadt wrote:
 In general, new non-standard options are bad.
 
 Basically, if we add this someone will use it in a script.  Then it will
 become non-portable.  You cannot just invent something on your own like
 this, without doing research to find out if someone else added a
 different
 option.  I don't see evidence of that, so the gut answer is no.

FreeBSD and Dragonfly BSD have this option in tr. So, this actually
improves portability.

-- 
  Shawn K. Quinn
  skqu...@rushpost.com



Re: linebuffering diff for tr(1)

2013-11-20 Thread Stuart Henderson
On 2013/11/20 07:40, Shawn K. Quinn wrote:
 On Tue, Nov 19, 2013, at 03:10 PM, Theo de Raadt wrote:
  In general, new non-standard options are bad.
  
  Basically, if we add this someone will use it in a script.  Then it will
  become non-portable.  You cannot just invent something on your own like
  this, without doing research to find out if someone else added a
  different
  option.  I don't see evidence of that, so the gut answer is no.
 
 FreeBSD and Dragonfly BSD have this option in tr. So, this actually
 improves portability.

People on those OS which *do* have this have to take extra care to make
sure they aren't writing unportable scripts. It isn't even supported in Linux
(instead they have a generic utility in coreutils, stdbuf).



Re: Kill IA_SIN()

2013-11-20 Thread Stuart Henderson
On 2013/11/20 11:32, Martin Pieuchot wrote:
 On 20/11/13(Wed) 10:16, Stuart Henderson wrote:
  On 2013/11/20 10:10, Martin Pieuchot wrote:
   This one is not under _KERNEL but it's used at only one place and
   if a port use it, it should probably define it by itself.
  
  This is used in at least kde-workspace and embedded copies of slirp
  (the ones I know about so far are emulators/qemu and emulators/BasiliskII).
  
  It's easy enough to patch existing users in ports now, but removing it
  makes things more difficult for people porting something else later
  which happens to use it - so I wonder what's the reasoning behind this?
  (if it's a namespace issue, I don't think that's valid given that it's
  widely available - xnu, solaris, free/net/open/dfbsd and derivatives).
 
 Simply cleaning, to unify our code base.
 
  btw if people want a quick way to search for things in common source code,
  http://codesearch.debian.net/ is useful.
 
 Nice tool, thanks for the hint.
 
 So here's an updated version that keeps the define for userland, would
 it be ok?

After some test builds and a careful look at where the #ifdef's are,
I've revised my opinion, nothing in ports does actually use it ;)
As a result, I'm OK with either version of this.



LC_MONETARY 3: regression test

2013-11-20 Thread Vladimir Támara Patiño

Tests for POSIX, en_US and spanish speaking countries.

--
Dios, gracias por tu amor infinito.
--  
 Vladimir Támara Patiño.  http://vtamara.pasosdeJesus.org/

 http://www.pasosdejesus.org/dominio_publico_colombia.html

diff -ruN -x *~ -x obj -x *orig src55-orig/regress/lib/libc/locale/Makefile 
src55-mon/regress/lib/libc/locale/Makefile
--- src55-orig/regress/lib/libc/locale/Makefile Fri Oct 18 09:22:50 2013
+++ src55-mon/regress/lib/libc/locale/Makefile  Wed Nov 20 09:12:20 2013
@@ -1,7 +1,7 @@
 # $OpenBSD: Makefile,v 1.2 2013/08/01 21:26:30 kettenis Exp $
 
 .if defined(REGRESS_FULL)
-SUBDIR+= check_isw
+SUBDIR+= check_isw check_monetary
 .endif
 
 install:
diff -ruN -x *~ -x obj -x *orig 
src55-orig/regress/lib/libc/locale/check_monetary/Makefile 
src55-mon/regress/lib/libc/locale/check_monetary/Makefile
--- src55-orig/regress/lib/libc/locale/check_monetary/Makefile  Wed Dec 31 
19:00:00 1969
+++ src55-mon/regress/lib/libc/locale/check_monetary/Makefile   Tue Nov 19 
20:34:54 2013
@@ -0,0 +1,11 @@
+
+NOMAN=
+PROG=check_monetary
+
+CFLAGS=-g
+
+
+run-regress-check_monetary: ${PROG}
+   ./${PROG} /dev/null
+
+.include bsd.regress.mk
diff -ruN -x *~ -x obj -x *orig 
src55-orig/regress/lib/libc/locale/check_monetary/check_monetary.c 
src55-mon/regress/lib/libc/locale/check_monetary/check_monetary.c
--- src55-orig/regress/lib/libc/locale/check_monetary/check_monetary.c  Wed Dec 
31 19:00:00 1969
+++ src55-mon/regress/lib/libc/locale/check_monetary/check_monetary.c   Wed Nov 
20 09:08:21 2013
@@ -0,0 +1,574 @@
+/**
+ * Public domain according to Colombian Legislation. 
+ * http://www.pasosdejesus.org/dominio_publico_colombia.html
+ * 2013. vtam...@pasosdejesus.org
+ *
+ * $adJ$
+ */
+
+#include langinfo.h
+#include limits.h
+#include locale.h
+#include monetary.h
+#include stdio.h
+#include stdlib.h
+#include string.h
+#include wchar.h
+#include wctype.h
+
+int bad;
+
+#define p(t) printf(%s:\t ,#t); \
+   if (t) { \
+   printf(\x1b[38;5;2mOK\x1b[0m\n); \
+   } else { \
+   bad++; \
+   printf(\x1b[38;5;1mERROR\x1b[0m\n); \
+   }
+
+/** Adapted from lmonetary.c */
+void
+m_monetarydebug(struct lconv *p) {
+   printf( int_curr_symbol = %s\n
+   currency_symbol = %s\n
+   mon_decimal_point = %s\n
+   mon_thousands_sep = %s\n
+   mon_grouping[0] = %d\n
+   positive_sign = %s\n
+   negative_sign = %s\n
+   int_frac_digits = %d\n
+   frac_digits = %d\n
+   p_cs_precedes = %d\n
+   p_sep_by_space = %d\n
+   n_cs_precedes = %d\n
+   n_sep_by_space = %d\n
+   p_sign_posn = %d\n
+   n_sign_posn = %d\n,
+   p-int_curr_symbol,
+   p-currency_symbol,
+   p-mon_decimal_point,
+   p-mon_thousands_sep,
+   p-mon_grouping[0],
+   p-positive_sign,
+   p-negative_sign,
+   p-int_frac_digits,
+   p-frac_digits,
+   p-p_cs_precedes,
+   p-p_sep_by_space,
+   p-n_cs_precedes,
+   p-n_sep_by_space,
+   p-p_sign_posn,
+   p-n_sign_posn
+   );
+   printf( \n\n
+   int_p_cs_precedes = %d\n
+   int_p_sep_by_space = %d\n
+   int_n_cs_precedes = %d\n
+   int_n_sep_by_space = %d\n
+   int_p_sign_posn = %d\n
+   int_n_sign_posn = %d\n,
+   p-int_p_cs_precedes,
+   p-int_p_sep_by_space,
+   p-int_n_cs_precedes,
+   p-int_n_sep_by_space,
+   p-int_p_sign_posn,
+   p-int_n_sign_posn
+   );
+}
+
+void test_posix()
+{
+   char *nl = setlocale(LC_ALL, POSIX);
+   printf(locale %s\n, nl);
+   struct lconv *p = localeconv();
+   char col[512];
+
+   // Values from 
http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap07.html#tag_07_03
+   p(strcmp(p-int_curr_symbol, ) == 0);
+   p(strcmp(p-currency_symbol, ) == 0);
+   p(strcmp(p-mon_decimal_point, ) == 0);
+   p(strcmp(p-mon_thousands_sep, ) == 0);
+   // there is like inconsistency in 
http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap07.html#tag_07_03
 
+   // Since in the locale definition it shows mon_grouping -1 for POSIX
+   // locale
+   // But in the table below it shows  as the return value for
+   // localeconv
+   p(p-mon_grouping[0] == '\0');
+   p(strcmp(p-positive_sign, ) == 0);
+   p(strcmp(p-negative_sign, ) == 0);
+   p(p-int_frac_digits == CHAR_MAX);
+   p(p-frac_digits == CHAR_MAX);
+   p(p-p_cs_precedes == CHAR_MAX);
+   p(p-p_sep_by_space == CHAR_MAX);
+   p(p-n_cs_precedes == CHAR_MAX);
+   p(p-n_sep_by_space == CHAR_MAX);
+   p(p-p_sign_posn == CHAR_MAX);
+   p(p-n_sign_posn == CHAR_MAX);
+   

Re: initial i217/i218 Haswell Ethernet support for em(4)

2013-11-20 Thread Markus Hennecke
On Wed, 20 Nov 2013, Jonathan Gray wrote:

 On Tue, Nov 19, 2013 at 01:09:32AM +1100, Jonathan Gray wrote:
  On Sat, Nov 16, 2013 at 10:42:05AM +0100, Markus Hennecke wrote:
   On Sat, 9 Nov 2013, Jonathan Gray wrote:
   
This adds the initial bits for the i217/i218 PHY and the
Lynx Point PCH found in Haswell systems.

Doesn't include the new workarounds yet and follows
the pch2/82579 paths for now but this seems to be enough
to make a desktop machine with I217-LM work
   
   This fails for the I217-V with an EEPROM invalid checksum error:
  
  Here is a revised diff with different eeprom bank logic
  from FreeBSD as suggested by Masanobu SAITOH:
 
 And here is a version that should fix gigabit autoneg:

This version works for me.

Regards,
Markus



Re: linebuffering diff for tr(1)

2013-11-20 Thread Ted Unangst
 On 2013/11/20 07:40, Shawn K. Quinn wrote:

 FreeBSD and Dragonfly BSD have this option in tr. So, this actually
 improves portability.

It's just spreading the disease. portable means it works everywhere.
Increasing the number of people who can write nonportable code is not
the same as increasing portability.



better sem_open internals

2013-11-20 Thread Ted Unangst
Fix a few things. I think I started down the sem inside a sem road
because I misunderstood some minor point of the API. Turns out it's
entirely possible to just map the semaphore and be done with it. We
only need a flag to identify shared semaphores. This makes everything
a little bit easier.

In the process of adding einval checking to sem_close, I noticed
that the inval checks are all broken because they deref a pointer
before checking for null. So fix all of those.

Index: rthread.h
===
RCS file: /cvs/src/lib/librthread/rthread.h,v
retrieving revision 1.46
diff -u -p -r1.46 rthread.h
--- rthread.h   18 Nov 2013 23:10:48 -  1.46
+++ rthread.h   20 Nov 2013 19:04:23 -
@@ -64,7 +64,7 @@ struct __sem {
struct _spinlock lock;
volatile int waitcount;
volatile int value;
-   struct __sem *shared;
+   int shared;
 };
 
 TAILQ_HEAD(pthread_queue, pthread);
Index: rthread_sem.c
===
RCS file: /cvs/src/lib/librthread/rthread_sem.c,v
retrieving revision 1.12
diff -u -p -r1.12 rthread_sem.c
--- rthread_sem.c   20 Nov 2013 03:16:39 -  1.12
+++ rthread_sem.c   20 Nov 2013 19:04:23 -
@@ -58,10 +58,8 @@ _sem_wait(sem_t sem, int tryonly, const 
void *ident = (void *)sem-waitcount;
int r;
 
-   if (sem-shared) {
-   sem = sem-shared;
+   if (sem-shared)
ident = SHARED_IDENT;
-   }
 
_spinlock(sem-lock);
if (sem-value) {
@@ -96,10 +94,8 @@ _sem_post(sem_t sem)
void *ident = (void *)sem-waitcount;
int rv = 0;
 
-   if (sem-shared) {
-   sem = sem-shared;
+   if (sem-shared)
ident = SHARED_IDENT;
-   }
 
_spinlock(sem-lock);
sem-value++;
@@ -119,7 +115,12 @@ sem_init(sem_t *semp, int pshared, unsig
 {
sem_t sem, *sempshared;
int i, oerrno;
-   char name[SEM_RANDOM_NAME_LEN];
+   char name[SEM_RANDOM_NAME_LEN];
+
+   if (!semp) {
+   errno = EINVAL;
+   return (-1);
+   }
 
if (pshared) {
while (1) {
@@ -145,9 +146,9 @@ sem_init(sem_t *semp, int pshared, unsig
}
 
sem = *sempshared;
-   free(sempshared);
sem-value = value;
*semp = sem;
+   free(sempshared);
return (0);
}
 
@@ -173,11 +174,10 @@ sem_destroy(sem_t *semp)
 {
sem_t sem;
 
-   if (!semp || !*semp) {
+   if (!semp || !(sem = *semp)) {
errno = EINVAL;
return (-1);
}
-   sem = *semp;
 
if (sem-waitcount) {
 #define MSG sem_destroy on semaphore with waiters!\n
@@ -187,11 +187,11 @@ sem_destroy(sem_t *semp)
return (-1);
}
 
-   if (sem-shared)
-   munmap(sem-shared, SEM_MMAP_SIZE);
-
*semp = NULL;
-   free(sem);
+   if (sem-shared)
+   munmap(sem, SEM_MMAP_SIZE);
+   else
+   free(sem);
 
return (0);
 }
@@ -199,16 +199,13 @@ sem_destroy(sem_t *semp)
 int
 sem_getvalue(sem_t *semp, int *sval)
 {
-   sem_t sem = *semp;
+   sem_t sem;
 
-   if (!semp || !*semp) {
+   if (!semp || !(sem = *semp)) {
errno = EINVAL;
return (-1);
}
 
-   if (sem-shared)
-   sem = sem-shared;
-
_spinlock(sem-lock);
*sval = sem-value;
_spinunlock(sem-lock);
@@ -219,9 +216,9 @@ sem_getvalue(sem_t *semp, int *sval)
 int
 sem_post(sem_t *semp)
 {
-   sem_t sem = *semp;
+   sem_t sem;
 
-   if (!semp || !*semp) {
+   if (!semp || !(sem = *semp)) {
errno = EINVAL;
return (-1);
}
@@ -234,11 +231,11 @@ sem_post(sem_t *semp)
 int
 sem_wait(sem_t *semp)
 {
-   sem_t sem = *semp;
pthread_t self = pthread_self();
+   sem_t sem;
int r;
 
-   if (!semp || !*semp) {
+   if (!semp || !(sem = *semp)) {
errno = EINVAL;
return (-1);
}
@@ -258,11 +255,11 @@ sem_wait(sem_t *semp)
 int
 sem_timedwait(sem_t *semp, const struct timespec *abstime)
 {
-   sem_t sem = *semp;
pthread_t self = pthread_self();
+   sem_t sem;
int r;
 
-   if (!semp || !*semp) {
+   if (!semp || !(sem = *semp)) {
errno = EINVAL;
return (-1);
}
@@ -282,10 +279,10 @@ sem_timedwait(sem_t *semp, const struct 
 int
 sem_trywait(sem_t *semp)
 {
-   sem_t sem = *semp;
+   sem_t sem;
int r;
 
-   if (!semp || !*semp) {
+   if (!semp || !(sem = *semp)) {
errno = EINVAL;
return (-1);
}
@@ -316,7 +313,7 @@ sem_open(const char *name, int oflag, ..
char sempath[SEM_PATH_SIZE];
struct stat sb;
int 

Re: linebuffering diff for tr(1)

2013-11-20 Thread Shawn K. Quinn
On Wed, Nov 20, 2013, at 12:49 PM, Ted Unangst wrote:
  On 2013/11/20 07:40, Shawn K. Quinn wrote:
 
  FreeBSD and Dragonfly BSD have this option in tr. So, this actually
  improves portability.
 
 It's just spreading the disease. portable means it works everywhere.
 Increasing the number of people who can write nonportable code is not
 the same as increasing portability.

How many others have to adopt it before it's considered portable, then?

Would you feel the same way if this were the -l option on ls, GNU had
it, and none of the BSD descendants did?

It's possible, as mentioned elsewhere, that simply making tr be
unbuffered by default is the better move, and ignore -u for
compatibility with FreeBSD and Dragonfly BSD.

-- 
  Shawn K. Quinn
  skqu...@rushpost.com



Re: linebuffering diff for tr(1)

2013-11-20 Thread Theo de Raadt
   FreeBSD and Dragonfly BSD have this option in tr. So, this actually
   improves portability.
  
  It's just spreading the disease. portable means it works everywhere.
  Increasing the number of people who can write nonportable code is not
  the same as increasing portability.
 
 How many others have to adopt it before it's considered portable, then?

It is portable when all of them have it.  Since you can't fix the past,
we must be very conservative in our approach.

 Would you feel the same way if this were the -l option on ls, GNU had
 it, and none of the BSD descendants did?

Uhm, that's a pretty weak argument.

 It's possible, as mentioned elsewhere, that simply making tr be
 unbuffered by default is the better move, and ignore -u for
 compatibility with FreeBSD and Dragonfly BSD.



Re: linebuffering diff for tr(1)

2013-11-20 Thread Franco Fichtner
On 20 Nov 2013, at 21:40, Theo de Raadt dera...@cvs.openbsd.org wrote:

 FreeBSD and Dragonfly BSD have this option in tr. So, this actually
 improves portability.
 
 It's just spreading the disease. portable means it works everywhere.
 Increasing the number of people who can write nonportable code is not
 the same as increasing portability.
 
 How many others have to adopt it before it's considered portable, then?
 
 It is portable when all of them have it.  Since you can't fix the past,
 we must be very conservative in our approach.

In this case `portable' simply means `unavailable'.  And that's good.  :)
DragonFly has it solely because of the shared FreeBSD history, not because
it's being used a lot.

 It's possible, as mentioned elsewhere, that simply making tr be
 unbuffered by default is the better move, and ignore -u for
 compatibility with FreeBSD and Dragonfly BSD.

How will that make things better?



Re: linebuffering diff for tr(1)

2013-11-20 Thread Theo de Raadt
  FreeBSD and Dragonfly BSD have this option in tr. So, this actually
  improves portability.
  
  It's just spreading the disease. portable means it works everywhere.
  Increasing the number of people who can write nonportable code is not
  the same as increasing portability.
  
  How many others have to adopt it before it's considered portable, then?
  
  It is portable when all of them have it.  Since you can't fix the past,
  we must be very conservative in our approach.
 
 In this case `portable' simply means `unavailable'.  And that's good.  :)
 DragonFly has it solely because of the shared FreeBSD history, not because
 it's being used a lot.

We always face a mix of goals:

   - portability (for common tools, don't diverge unless value is high enough)
   - improvements (diverge if the value is high enough)
   - standards (when portability is mandated, strongly follow that)
   - defacto standards (non-official standards also exist)

It takes a pretty big cry to start making changes.  In part that is why
this conversation isn't dead yet.

It is quite telling that FreeBSD added it a long long time ago, and it
has not been adopted elsewhere.  Pushes back against the urgency.

  It's possible, as mentioned elsewhere, that simply making tr be
  unbuffered by default is the better move, and ignore -u for
  compatibility with FreeBSD and Dragonfly BSD.
 
 How will that make things better?

Standing alone, compatibility with FOO is not a very strong
argument.  What next, compatibility with Xenix and Windows?



sem_open

2013-11-20 Thread Stuart Henderson
graphics/ilmbase hangs during autoconf; it runs something close to this

#include semaphore.h
int
main()
{
sem_t   mysem;

if (sem_init(mysem, 1, 1) == 0) {
if (sem_wait(mysem) == 0) {
sem_post(mysem);
sem_destroy(mysem);
}
}
}



Re: sem_open

2013-11-20 Thread Ted Unangst
On Wed, Nov 20, 2013 at 21:59, Stuart Henderson wrote:
 graphics/ilmbase hangs during autoconf; it runs something close to this

The diff I previously posted seems to fix this. At least, I could
repro with a snapshot and now I can't reproduce after installing the
new library.

I also spotted one more bug, but it shouldn't affect i386 or amd64.
The spinlock needs to be initialized in sem_open.

 
 #include semaphore.h
 int
 main()
 {
 sem_t mysem;
 
 if (sem_init(mysem, 1, 1) == 0) {
 if (sem_wait(mysem) == 0) {
 sem_post(mysem);
 sem_destroy(mysem);
 }
 }
 }



Re: sem_open

2013-11-20 Thread Stuart Henderson
On 2013/11/20 18:09, Ted Unangst wrote:
 On Wed, Nov 20, 2013 at 21:59, Stuart Henderson wrote:
  graphics/ilmbase hangs during autoconf; it runs something close to this
 
 The diff I previously posted seems to fix this. At least, I could
 repro with a snapshot and now I can't reproduce after installing the
 new library.

thanks, confirmed, http://marc.info/?l=openbsd-techm=138497459327445w=2 fixes 
it.



Re: sem_open

2013-11-20 Thread Ted Unangst
On Wed, Nov 20, 2013 at 23:12, Stuart Henderson wrote:
 On 2013/11/20 18:09, Ted Unangst wrote:
 On Wed, Nov 20, 2013 at 21:59, Stuart Henderson wrote:
  graphics/ilmbase hangs during autoconf; it runs something close to this

 The diff I previously posted seems to fix this. At least, I could
 repro with a snapshot and now I can't reproduce after installing the
 new library.
 
 thanks, confirmed, http://marc.info/?l=openbsd-techm=138497459327445w=2
 fixes it.

ok, i just went ahead and committed it. no sense in delaying.



sem_open value

2013-11-20 Thread Ted Unangst
Read the standard again and discovered some more missing features.

1. sem_open allows setting the value via a fourth argument. Fixed.

2. Multiple sem_open calls of the same path in the same process are
supposed to return the same pointer. Not the same semaphore, the same
pointer. This is mind boggling and hard to accomplish with this
implementation. Not fixed.

3. There are also some requirements that sem_open be atomic in
ways that it is not. We're publishing, via the filesystem, the new
semaphore before we're done initializing it. This is fixable, but
requires bizarro filesystem rename hijinks. Maybe another diff.

Index: rthread_sem.c
===
RCS file: /cvs/src/lib/librthread/rthread_sem.c,v
retrieving revision 1.13
diff -u -p -r1.13 rthread_sem.c
--- rthread_sem.c   20 Nov 2013 23:18:17 -  1.13
+++ rthread_sem.c   21 Nov 2013 03:34:03 -
@@ -24,6 +24,7 @@
 #include errno.h
 #include fcntl.h
 #include sha2.h
+#include stdarg.h
 #include stdlib.h
 #include stdio.h
 #include string.h
@@ -315,12 +316,22 @@ sem_open(const char *name, int oflag, ..
int created = 0, fd, oerrno;
sem_t sem;
sem_t *semp = SEM_FAILED;
+   mode_t unusedmode;
+   unsigned value = 0;
 
if (oflag  ~(O_CREAT | O_EXCL)) {
errno = EINVAL;
return (semp);
}
 
+   if (oflag  O_CREAT) {
+   va_list ap;
+   va_start(ap, oflag);
+   unusedmode = va_arg(ap, mode_t);
+   value = va_arg(ap, unsigned);
+   va_end(ap);
+   }
+
makesempath(name, sempath, sizeof(sempath));
fd = open(sempath, O_RDWR | O_NOFOLLOW | oflag, 0600);
if (fd == -1)
@@ -363,8 +374,10 @@ sem_open(const char *name, int oflag, ..
errno = oerrno;
return (semp);
}
-   if (created)
+   if (created) {
sem-lock = _SPINLOCK_UNLOCKED_ASSIGN;
+   sem-value = value;
+   }
sem-shared = 1;
semp = malloc(sizeof(*semp));
if (!semp) {
@@ -382,7 +395,7 @@ int
 sem_close(sem_t *semp)
 {
sem_t sem;
-   
+
if (!semp || !(sem = *semp) || !sem-shared) {
errno = EINVAL;
return (-1);