Re: httpd rewrites with Lua's pattern matching

2015-06-22 Thread Gregory Edigarov

what a nice small piece of code!
I think it should be made into library, as there are other 
projects(think smtpd), that would benefit from it too.


On 06/21/2015 05:01 PM, Reyk Floeter wrote:

On Sat, Jun 20, 2015 at 03:01:18PM +0200, Reyk Floeter wrote:

Hi,

there is some great interest in getting support for rewrites and
better matching in httpd.  I refused to implement this using regex, as
regex is extremely complicated code, there have been lots of bugs,
they allow, if not specified carefully, dangerous recursions and
ReDOS, and I would add another potential attack surface in httpd.

Thanks to tedu@'s hint at BSDCan, I stumbled across Lua's pattern
matching implementation.  It is relatively small (less than 700loc),
powerful, portable C code, MIT-licensed, and doesn't suffer from some
of regex' problems (eg., it doesn't allow recursive captures).  I
ported it on my flight back from Ottawa, KNF'ed it, and turned it into
a C API without the Lua bindings.  No, this diff does not bring the
Lua language to httpd!

Here is a diff that adds pattern matching to httpd, allowing rewrites
with redirects.  Additional use cases can be added later and, if it
works out, we can probably move it to an existing library.

 location match '^/page/(%d+)' {
 block return 302 /index.cgi?page=%1'
 }

This diff will be two commits, one is pending to fix '?' in the uri.

Comments? OK?


Here is an updated version with some fixes from Sebastien Marie (semarie).

We're continuing to work on it, but would like to do it in the tree.

OK?  Any concerns?

Reyk

diff --git httpd/Makefile httpd/Makefile
index 885ad42..69fdb5e 100644
--- httpd/Makefile
+++ httpd/Makefile
@@ -6,6 +6,9 @@ SRCS+=  config.c control.c httpd.c log.c logger.c proc.c
  SRCS+=server.c server_http.c server_file.c server_fcgi.c
  MAN=  httpd.8 httpd.conf.5
  
+SRCS+=		patterns.c

+MAN+=  patterns.7
+
  LDADD=-levent -ltls -lssl -lcrypto -lutil
  DPADD=${LIBEVENT} ${LIBTLS} ${LIBSSL} ${LIBCRYPTO} ${LIBUTIL}
  #DEBUG=   -g -DDEBUG=3 -O0
diff --git httpd/httpd.conf.5 httpd/httpd.conf.5
index 87866d2..24d92ac 100644
--- httpd/httpd.conf.5
+++ httpd/httpd.conf.5
@@ -131,14 +131,38 @@ The configured web servers.
  .Pp
  Each
  .Ic server
-must have a
-.Ar name
-and include one or more lines of the following syntax:
+section starts with a declaration of the server
+.Ar name :
+.Bl -tag -width Ds
+.It Ic server Ar name Brq ...
+Match the server name using shell globbing rules.
+This can be an explicit name,
+.Ar www.example.com ,
+or a name including wildcards,
+.Ar *.example.com .
+.It Ic server match Ar name Brq ...
+Match the server name using pattern matching,
+see
+.Xr patterns 7 .
+.El
+.Pp
+Followed by a block of options that is enclosed in curly brackets:
  .Bl -tag -width Ds
  .It Ic alias Ar name
  Specify an additional alias
  .Ar name
  for this server.
+.It Ic alias match Ar name
+Like the
+.Ic alias
+option,
+but
+.Ic match
+the
+.Ar name
+using pattern matching instead of shell globbing rules,
+see
+.Xr patterns 7 .
  .It Oo Ic no Oc Ic authenticate Oo Ar realm Oc Ic with Pa htpasswd
  Authenticate a remote user for
  .Ar realm
@@ -188,6 +212,12 @@ The configured IP address of the server.
  The configured TCP server port of the server.
  .It Ic $SERVER_NAME
  The name of the server.
+.It Ic Pf % Ar n
+The capture index
+.Ar n
+of a string that was captured by the enclosing
+.Ic location match
+option.
  .El
  .It Ic connection Ar option
  Set the specified options and limits for HTTP connections.
@@ -247,6 +277,22 @@ except
  .Ic location
  and
  .Ic tcp .
+.It Ic location match Ar path Brq ...
+Like the
+.Ic location
+option,
+but
+.Ic match
+the
+.Ar path
+using pattern matching instead of shell globbing rules,
+see
+.Xr patterns 7 .
+The pattern may contain captures that can be used in the
+.Ar uri
+of an enclosed
+.Ic block return
+option.
  .It Oo Ic no Oc Ic log Op Ar option
  Set the specified logging options.
  Logging is enabled by default using the standard
@@ -516,6 +562,7 @@ server www.example.com {
  .Ed
  .Sh SEE ALSO
  .Xr htpasswd 1 ,
+.Xr patterns 7 ,
  .Xr httpd 8 ,
  .Xr slowcgi 8
  .Sh AUTHORS
diff --git httpd/httpd.h httpd/httpd.h
index 1431eaa..ff76281 100644
--- httpd/httpd.h
+++ httpd/httpd.h
@@ -35,6 +35,8 @@
  #include imsg.h
  #include tls.h
  
+#include patterns.h

+
  #define CONF_FILE /etc/httpd.conf
  #define HTTPD_SOCKET  /var/run/httpd.sock
  #define HTTPD_USERwww
@@ -278,6 +280,7 @@ struct client {
void*clt_srv_conf;
u_int32_tclt_srv_id;
struct sockaddr_storage  clt_srv_ss;
+   struct str_match clt_srv_match;
  
  	int			 clt_s;

in_port_tclt_port;
@@ -341,12 +344,15 @@ SPLAY_HEAD(client_tree, client);
  #define SRVFLAG_NO_AUTH   0x0002
  #define SRVFLAG_BLOCK 

Re: chromium does not handle some video streams that firefox does

2015-06-22 Thread Stuart Henderson
On 2015/06/22 13:34, Marc Espie wrote:
 For instance,
 https://www.youtube.com/watch?v=J_8mdH20qTQ
 
 Youtube-dl makes the reason pretty obvious:
 https://www.youtube.com/watch?v=J_8mdH20qTQ
 [youtube] J_8mdH20qTQ: Downloading webpage
 [youtube] J_8mdH20qTQ: Extracting video information
 [youtube] J_8mdH20qTQ: Downloading DASH manifest
 [download] Destination: Remove cat before flight-J_8mdH20qTQ.f137.mp4
 [download] 100% of 58.62MiB in 01:29
 [download] Destination: Remove cat before flight-J_8mdH20qTQ.f140.m4a
 [download] 100% of 2.68MiB in 00:04
 [ffmpeg] Merging formats into Remove cat before flight-J_8mdH20qTQ.mp4
 Deleting original file Remove cat before flight-J_8mdH20qTQ.f137.mp4 (pass -k 
 to keep)
 Deleting original file Remove cat before flight-J_8mdH20qTQ.f140.m4a (pass -k 
 to keep)
 
 (e.g., split video/audio)
 
 There is definitely something different going on, because firefox can stream
 this, and chrome (on windows) has no issue as well...

The split a/v is something that newer youtube-dl does on purpose, I don't think 
it's
particularly related to why chromium doesn't handle it.. Something to do with 
how the
bundled FFmpeg is compiled perhaps?



Use-after-free in OpenBSD NTFS driver from ntnodeq

2015-06-22 Thread Aliz 'Randomdude'
Hi List.

I've stumbled across a use-after-free condition in the NTFS filesystem
driver of OpenBSD Current (HEAD, plus it's present in STABLE, and
appears to go back a few releases - I repro'ed with 5.1). I have seen
it triggered accidentally through normal use to cause a panic.

The bug revolves around the ntm_ntnodeq, a 16-entry LRU which is a
member of the ntfsmount structure. Elements are added to this by
ntfs_loadntnode, but are destroyed when removed from the LRU, without
any reference checking - so another process may have its ntnodes freed
unexectedly, causing the kernel to use memory which has been released.

The following happens:

1) Process 'foo' starts loading some data from the disk. It loads an
ntnode A, adds it to the ntnodeq (see ntfs_subr.c:353, ntfs_loadntnode
- 'TAILQ_INSERT_HEAD(ntmp-ntm_ntnodeq, ip, i_loaded);'), and then
tsleep's waiting for a subsequent disk read to complete.
2) Process 'bar' reads sixteen (or more) ntnodes from disk. The
ntnodeq gets cycled, and ntnode A, loaded by foo previously, is freed
- see ntfs_subr.c:278, ntfs_loadntnode - 'ntfs_freentvattr(vap);')
3) Process 'foo' then attempts to use its ntnode A, which points to
freed memory.

The bad access isn't very obvious (although it does happen, as
verified by hand via a debugger) and usually goes unnoticed without
any crash or panic. If you zero out the memory before freeing it, it
is much easier to reproduce. Edit ntfs_freentvattr, found in
ntfs_subr.c:

+ memset(vap, 0, sizeof(struct ntvattr));   // (add this line)
  free(vap, M_NTFSNTVATTR, 0);  // (this line already present)

An easy way to repro the issue is to create ~100 small files, plus one
heavily fragmented file. Reading the fragmented file will cause
tsleep's between ntnode reads, while the small files can be read
repeatedly in the background. Since there are only 100 of them, they
should all fit in the cache and can be served quickly, causing them to
exhaust the LRU. Here's a script that'll create such a disk layout.
Run it in the ntfs mountpoint:

--

#!/bin/sh

# Fill disk will small files
i=0
while ( true ); do
 dd if=/dev/zero count=2 of=small_$i
 if [ $? -ne 0 ]; do
  break
 fi
 i=$(( $i + 1 ))
done

# Delete some non-sequential ones
for toDel in 0 $i 100; do
 rm small_$i
done

# Create the big file
dd if=/dev/zero of=/bigfile
--

You should be able to repro by:

0) Applying the memset above (or by being lucky)
1) Optionally reboot, which will make sure bigfile isn't cached and
will up your chances of a successful repo
2) Mounting this FS
3) Repeatedly reading 100 of the small files, causing the LRU to churn:
  while ( true ); do cat /mnt/ntfs/small_??  /dev/null; done 
4) Reading the large file. By the time the OS has finished reading the
large file, the access of the small files will cause the first
clusters of the large file to be freed.
  cat /mnt/ntfs/big  /dev/null

The result will probably be a panic, since the memory area being used
by the kernel is free'd. Manual analysis verifies that the memory
being accessed is, indeed, freed. A sample panic (full logs below):

ntfs_readntvattr_plain: POSSIBLE RUN ERROR
ntfs_readattr_plain: ntfs_readntvattr_plain failed: o: 1179648, s: 65536
ntfs_readattr_plain: attrib: 0 - 0
panic: ntfs_ntrele: ino: 159 usecount: -1
Stopped at  Debugger+0x9:   leave
RUN AT LEAST 'trace' AND 'ps' AND INCLUDE OUTPUT WHEN REPORTING THIS PANIC!
DO NOT EVEN BOTHER REPORTING THIS WITHOUT INCLUDING THAT INFORMATION!
ddb trace
Debugger() at Debugger+0x9
panic() at panic+0xfe
ntfs_ntrele() at ntfs_ntrele+0x28
ntfs_ntvattrrele() at ntfs_ntvattrrele+0x11
ntfs_readattr() at ntfs_readattr+0x135
ntfs_read() at ntfs_read+0x69
VOP_READ() at VOP_READ+0x3f
vn_read() at vn_read+0xa1
dofilereadv() at dofilereadv+0x1c4
sys_read() at sys_read+0x89
syscall() at syscall+0x192
--- syscall (number 3) ---
end trace frame: 0x0, count: -11

My repros were done on a VMWare Workstation VM, configured to have 1
CPU core, 256MB RAM, and to have a 64bit guest. I have verified this
issue on physical hardware also.

I'm guessing that the call to ntfs_freentvattr is in error, and it
should be a call to ntfs_ntput, but I'm unfamilliar with OpenBSD and
don't know if this is correct (especially in regard to locking
semantics).

Hope this is helpful. Don't hesitate to contact me if there's any more
info you need, or just to keep me updated. If you end up putting out
an advisory, I'd be very grateful if you could credit me on it - it
might help me find a job!

- Aliz Hammond

Full ddb log:

rebooting...
 OpenBSD/amd64 BOOT 3.28
boot
booting hd0a:/bsd:
entry point at 0x1000160 [7205c766, 3404, 24448b12, 9060a304]
[ using 932984 bytes of bsd ELF symbol table ]
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California.  All rights reserved.
Copyright (c) 1995-2015 OpenBSD. All rights reserved.  http://www.OpenBSD.org
OpenBSD 5.8-beta (GENERIC) #0: Mon Jun 22 04:41:57 BST 2015

Re: Conver bridge(4) to if_input()

2015-06-22 Thread Martin Pieuchot
On 22/06/15(Mon) 17:12, Alexander Bluhm wrote:
 On Wed, Jun 17, 2015 at 02:08:14PM +0200, Martin Pieuchot wrote:
  diff -u -p -r1.340 if.c
  --- net/if.c16 Jun 2015 11:09:39 -  1.340
  +++ net/if.c17 Jun 2015 12:03:36 -
  @@ -530,6 +530,15 @@ if_input_process(void *xmq)
  continue;
  }
   
  +#if NBRIDGE  0
  +   if (ifp-if_bridgeport  (m-m_flags  M_PROTO1) == 0) {
  +   m = bridge_input(m);
  +   if (m == NULL)
  +   continue;
  +   }
  +   m-m_flags = ~M_PROTO1;/* Loop prevention */
  +#endif
 
 Should we reset the loop prevention only if our call to bridge_input()
 did set M_PROTO1?  Something like this
 
   if (ifp-if_bridgeport  (m-m_flags  M_PROTO1) == 0) {
   m = bridge_input(m);
   if (m == NULL)
   continue;
   m-m_flags = ~M_PROTO1;/* Loop prevention */
   }

Yes and no :)

bridge_input() will set M_PROTO1 on the mbuf copies that it enqueues on
its ports.

If you receive a packet on em0 in bridge0 with tun0, you want to call
bridge_input() only once, but you'll call if_input() in em0 and tun0.

So the first packet will enter if_input() without M_PROTO1, go through
bridge_input() then be processed by the stack.  Then the copy of this
packet created in bridge_input() will have the M_PROTO1 flag set and
when it will be dequeued by if_input() it won't be passed to 
bridge_input() again.

Does that make sense?



Re: ed(1): keep custom prompt string that was specified by -p

2015-06-22 Thread Jason McIntyre
On Thu, Jun 18, 2015 at 12:33:58PM +0200, Theo Buehler wrote:
 Here's a silly ed(1) session:
 
 $ ed -p : 
 : P
 P
 *q
 $
 
 Notice how the prompt string changed from the custom prompt :  to
 the default prompt *.
 
 This behavior seems to contradict both the man page and POSIX:
 
 From ed(1):
  -p string  Specifies a command prompt.  This may be toggled on and off
 with the P command.
 
 From POSIX:
The P command shall cause ed to prompt with an asterisk ('*') (or
string, if -p is specified) for all subsequent commands. The P command
alternatively shall turn this mode on and off; it shall be initially on
if the -p option is specified; otherwise, off.
 
 With the patch below, the above session becomes
 
 $ ed -p : 
 : P 
 P
 : q 
 $
 
 which, I believe, is closer to both the documentation and POSIX.
 
 This also matches the behavior of GNU Ed 1.10.
 

this behaviour does seem more logical i guess. no one has stepped up to
change it though, so i've added a note to our doc until such a time.

thanks for the report.
jmc

 Index: bin/ed/main.c
 ===
 RCS file: /cvs/src/bin/ed/main.c,v
 retrieving revision 1.43
 diff -u -p -r1.43 main.c
 --- bin/ed/main.c 16 Jan 2015 06:39:32 -  1.43
 +++ bin/ed/main.c 18 Jun 2015 10:33:21 -
 @@ -110,7 +110,7 @@ top:
   while ((c = getopt(argc, argv, p:sx)) != -1)
   switch (c) {
   case 'p':   /* set prompt */
 - prompt = optarg;
 + dps = prompt = optarg;
   break;
   case 's':   /* run script */
   scripted = 1;
 



Re: Conver bridge(4) to if_input()

2015-06-22 Thread Alexander Bluhm
On Wed, Jun 17, 2015 at 02:08:14PM +0200, Martin Pieuchot wrote:
 diff -u -p -r1.340 if.c
 --- net/if.c  16 Jun 2015 11:09:39 -  1.340
 +++ net/if.c  17 Jun 2015 12:03:36 -
 @@ -530,6 +530,15 @@ if_input_process(void *xmq)
   continue;
   }
  
 +#if NBRIDGE  0
 + if (ifp-if_bridgeport  (m-m_flags  M_PROTO1) == 0) {
 + m = bridge_input(m);
 + if (m == NULL)
 + continue;
 + }
 + m-m_flags = ~M_PROTO1;/* Loop prevention */
 +#endif

Should we reset the loop prevention only if our call to bridge_input()
did set M_PROTO1?  Something like this

if (ifp-if_bridgeport  (m-m_flags  M_PROTO1) == 0) {
m = bridge_input(m);
if (m == NULL)
continue;
m-m_flags = ~M_PROTO1;/* Loop prevention */
}

otherwise OK bluhm@



Re: samba4 slow startup, shared libs?

2015-06-22 Thread Ian Mcwilliam

For reference: https://bugzilla.samba.org/show_bug.cgi?id=11355



Ian McWilliam


From: Ian Mcwilliam
Sent: Friday, 19 June 2015 10:03 AM
To: Jérémie Courrèges-Anglas
Cc: Stuart Henderson; Mark Kettenis; tech@openbsd.org
Subject: RE: samba4 slow startup, shared libs?

Sweet. I'll knockup a patch and send it upstream shortly.


Ian McWilliam


From: Jérémie Courrèges-Anglas [j...@wxcvbn.org]
Sent: Thursday, 18 June 2015 8:11 PM
To: Ian Mcwilliam
Cc: Stuart Henderson; Mark Kettenis; tech@openbsd.org
Subject: Re: samba4 slow startup, shared libs?

Ian Mcwilliam i.mcwill...@uws.edu.au writes:

 Curiously from

 --- samba-4.0.3/wscript   Tue Dec  4 21:07:44 2012
 +++ samba-4.0.26/wscript  Mon Dec  8 18:46:38 2014


 -if sys.platform != openbsd4:
 +if not sys.platform.startswith(openbsd):
  conf.env.asneeded_ldflags = conf.ADD_LDFLAGS('-Wl,--as-needed', 
 testflags=True)

Here's the output of LD_DEBUG=1 time samba-tool:

ritchie /usr/ports/net/samba4$ tail before after
== before ==
doing dtors obj 0x76bdc238000 @0x76c004b0a90: [/usr/local/lib/libiconv.so.6.0]
doing dtors
doing dtors obj 0x76bb7b3b000 @0x76bfbc6e740: 
[/usr/local/lib/libpython2.7.so.0.0]
doing dtors obj 0x76bb7b3b800 @0x76b8c3331b0: [/usr/lib/libstdc++.so.57.0]
doing dtors obj 0x76c1ede3400 @0x76bbadbc950: [/usr/lib/libc.so.79.0]
doing dtors obj 0x76c1ede3000 @0x76c333c5a40: [/usr/lib/libutil.so.12.1]
doing dtors obj 0x76bdc238800 @0x76c0159b340: [/usr/lib/libm.so.9.0]
  251.92 real   251.96 user 0.19 sys
doing dtors
 doing dtors obj 0xf0ae0f94c00 @0xf0b5ec96950: [/usr/lib/libc.so.79.0]

== after ==
doing dtors obj 0xa6021425000 @0xa60bf833a90: [/usr/local/lib/libiconv.so.6.0]
doing dtors
doing dtors obj 0xa6021425800 @0xa603fca0740: 
[/usr/local/lib/libpython2.7.so.0.0]
doing dtors obj 0xa6034d7e800 @0xa60c0c911b0: [/usr/lib/libstdc++.so.57.0]
doing dtors obj 0xa60a5bcf400 @0xa604a3e0950: [/usr/lib/libc.so.79.0]
doing dtors obj 0xa60a5bcf000 @0xa60eb3eba40: [/usr/lib/libutil.so.12.1]
doing dtors obj 0xa6021425400 @0xa610ba86340: [/usr/lib/libm.so.9.0]
1.49 real 1.24 user 0.22 sys
doing dtors
doing dtors obj 0x4e7a8e11c00 @0x4e767f81950: [/usr/lib/libc.so.79.0]

(with wip samba-4.1.18)

--
jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF  DDCC 0DFA 74AE 1524 E7EE



Re: freetype-2.6

2015-06-22 Thread David Coppa
On Sun, Jun 21, 2015 at 9:26 PM, Matthieu Herrb matth...@herrb.eu wrote:
 On Sat, Jun 20, 2015 at 09:26:25PM +0200, David Coppa wrote:
 On Sat, Jun 20, 2015 at 12:24 PM, Matthieu Herrb matth...@herrb.eu wrote:
  On Wed, Jun 17, 2015 at 04:34:15AM -0600, David Coppa wrote:
 
  Hi all,
 
  Here's an update to freetype-2.6
 
  I'd like to have it in a ports bulk build.
 
  Already tested in a xenocara build: built successfully and it seems
  fine so far.
 
 
  Two nits:
 
  - from my analysis there were 2 public symbols removed from
  libfreetype.so (ft_raccess_guess_table and ft_raster5_renderer_class)
  so it should be a major shlib_version bump.

 Fixed, thanks

  - the patches to the windows and wince build files failed because of
  DOS style line separators that got mangled somehow. Be careful with
  those when doing the actual commit.

 Probably the mail transfer. It applied correctly to my local tree.

 Here's the new diff gzipped, it should be safe.

 New diff applies cleanly. ok matthieu@

Thanks.

Waiting for the bulk build's results...



if_pfsync error counter

2015-06-22 Thread Erik Lax
Hi,

I noticed by looking at the code that in sys/net/if_pfsync.c :
pfsync_sendout() a failed ip_output() does not increment
sc-sc_if.if_oerrors++ only pfsyncstats.pfsyncs_oerrors++. Similar
pattern do in eg. if_pflow.c. Should it?


chromium does not handle some video streams that firefox does

2015-06-22 Thread Marc Espie
For instance,
https://www.youtube.com/watch?v=J_8mdH20qTQ

Youtube-dl makes the reason pretty obvious:
https://www.youtube.com/watch?v=J_8mdH20qTQ
[youtube] J_8mdH20qTQ: Downloading webpage
[youtube] J_8mdH20qTQ: Extracting video information
[youtube] J_8mdH20qTQ: Downloading DASH manifest
[download] Destination: Remove cat before flight-J_8mdH20qTQ.f137.mp4
[download] 100% of 58.62MiB in 01:29
[download] Destination: Remove cat before flight-J_8mdH20qTQ.f140.m4a
[download] 100% of 2.68MiB in 00:04
[ffmpeg] Merging formats into Remove cat before flight-J_8mdH20qTQ.mp4
Deleting original file Remove cat before flight-J_8mdH20qTQ.f137.mp4 (pass -k 
to keep)
Deleting original file Remove cat before flight-J_8mdH20qTQ.f140.m4a (pass -k 
to keep)

(e.g., split video/audio)

There is definitely something different going on, because firefox can stream
this, and chrome (on windows) has no issue as well...



Re: important audio simplifications to test and review

2015-06-22 Thread Mark Kettenis
 Date: Sat, 20 Jun 2015 15:25:37 +0200
 From: Alexandre Ratchov a...@caoua.org
 
 On Thu, Jun 18, 2015 at 07:25:31PM +, Christian Weisgerber wrote:
  On 2015-06-13, Alexandre Ratchov a...@caoua.org wrote:
  
   Many thanks to all who tested, below is an updated diff with the
   missing i386 bits and working uaudio.
  
  I have run into a serious problem on sparc64 (but I haven't tested
  elsewhere).
  
  I've added -L- to the sndiod flags and I'm streaming audio from a
  remote machine (AUDIODEVICE=snd@host/0) to the sparc64.
 
 
  After a while, the network on the sparc64 locks up.  Not just sndiod,
  all the network.  I can't even ping the machine any longer.  But
  it is only the network; the machine remains fully responsive over
  the serial console.  Playing around with ping/ping6, I see that the
  machine still sends packets, but it doesn't see any incoming packets,
  including ARP and NDP replies.
  
  It appears reproducible.
  
  Details:
  Sun Blade 150 with on-board gem(4) and a cmpci(4) expansion card
  sndiod_flags=-r44100 -f rsnd/1 -L-
 
 Sorry, i couldn't make my cmpci card work on my sparc64 (pci
 version incompatibiliy). Could you check the following:
 
 - try the same setup, but with the onboard autri(4) instead of the
   cmpci(4). The devices are similar and if the bug is in the
   midlayer this may trigger it.
 
 - try the same setup with without any diffs applied.

To be honest, I think the problem naddy is seeing is in the gem(4)
driver, not the audio layer.

What does systat mbuf show when it gets into this state?

If the number of ALIVE mbufs is lower than 4, it might be worth trying
the diff below.


Index: gem.c
===
RCS file: /cvs/src/sys/dev/ic/gem.c,v
retrieving revision 1.111
diff -u -p -r1.111 gem.c
--- gem.c   14 Mar 2015 03:38:47 -  1.111
+++ gem.c   22 Jun 2015 08:18:45 -
@@ -94,6 +94,7 @@ int   gem_disable_rx(struct gem_softc *);
 intgem_disable_tx(struct gem_softc *);
 void   gem_rx_watchdog(void *);
 void   gem_rxdrain(struct gem_softc *);
+void   gem_rx_fill(void *);
 void   gem_fill_rx_ring(struct gem_softc *);
 intgem_add_rxbuf(struct gem_softc *, int idx);
 void   gem_iff(struct gem_softc *);
@@ -346,6 +347,7 @@ gem_config(struct gem_softc *sc)
ether_ifattach(ifp);
 
timeout_set(sc-sc_tick_ch, gem_tick, sc);
+   timeout_set(sc-sc_rx_fill, gem_rx_fill, sc);
timeout_set(sc-sc_rx_watchdog, gem_rx_watchdog, sc);
return;
 
@@ -554,6 +556,9 @@ gem_stop(struct ifnet *ifp, int softonly
}
sc-sc_tx_cnt = sc-sc_tx_prod = sc-sc_tx_cons = 0;
 
+   timeout_del(sc-sc_rx_fill);
+   timeout_del(sc-sc_rx_watchdog);
+
gem_rxdrain(sc);
 }
 
@@ -1017,6 +1022,18 @@ gem_rint(struct gem_softc *sc)
 }
 
 void
+gem_rx_fill(void *arg)
+{
+   struct gem_softc *sc = arg;
+   int s;
+
+   s = splnet();
+   if (if_rxr_inuse(sc-sc_rx_ring)  4)
+   gem_fill_rx_ring(sc);
+   splx(s);
+}
+
+void
 gem_fill_rx_ring(struct gem_softc *sc)
 {
u_int slots;
@@ -1027,6 +1044,9 @@ gem_fill_rx_ring(struct gem_softc *sc)
break;
}
if_rxr_put(sc-sc_rx_ring, slots);
+
+   if (if_rxr_inuse(sc-sc_rx_ring)  4)
+   timeout_add(sc-sc_rx_fill, 1);
 }
 
 /*
Index: gemvar.h
===
RCS file: /cvs/src/sys/dev/ic/gemvar.h,v
retrieving revision 1.29
diff -u -p -r1.29 gemvar.h
--- gemvar.h8 Jul 2014 05:35:18 -   1.29
+++ gemvar.h22 Jun 2015 08:18:45 -
@@ -178,6 +178,7 @@ struct gem_softc {
struct gem_rxsoft sc_rxsoft[GEM_NRXDESC];
struct if_rxring sc_rx_ring;
u_int32_t sc_rx_prod, sc_rx_cons;
+   struct timeout sc_rx_fill;
 
/*
 * Control data structures.



Re: interfaces and priorities for relayd routers

2015-06-22 Thread Reyk Floeter
Hi,

On Thu, May 14, 2015 at 09:44:22PM +1000, David Gwynne wrote:
 i want relayd to check teh availability of some services and inject
 routes when the service is available. if it is available, i want
 to advertise the routes using ospfd, but i also want the local
 machine to be able to contact the service even if it isnt the carp
 master.
 
 to do that i need to inject the routes twice, once on my real
 interface and again on my carp interfaces. the route on the real
 interface needs to be a higher priority than the carp route.
 
 this shuffles relayd to accomodate this.
 
 it mostly adds stuff to the route statements in routers, but i also
 take priorities away from hosts in tables so i can specify them on
 routes.
 

I'm generally ok with extending the use cases of router, but it seems
that you're replacing one narrow use case with another one.

The router code was mostly done for something like link balancing,
you have two or more uplink gateways and want to select the route
based on their availability.  I'm afraid that moving the priority from
hosts to routes, and changing the semantics, breaks this original use
case somehow.

table gateways {
$gw1 ip ttl 1 priority 8,
$gw2 ip ttl 1 priority 52
}
router uplinks {
route 0.0.0.0/0
forward to gateways check icmp
}

But maybe even multiple routes for something internal:

router srvlan {
route 10.10.0.0/16
route 10.40.0.0/16
forward to srvgateways check icmp
}

Sometimes different gateways can even be connected via the same interface!
(e.g. with an intermediate switch and a single cable to the OpenBSD box)

 this is an example config:
 
   rns_nogal=130.102.71.227
   rns_neem=130.102.71.229
 
   table rns { $rns_nogal ip ttl 1, $rns_neem ip ttl 1 } 
 
   router rns { 
   route 130.102.71.160/31 interface vlan888 priority 8
   route 130.102.71.160/31 interface carp40888 priority 16
   forward to rns check icmp 
   }
 

I'm wondering, how would you translate my examples above?

   redirect dns {
   listen on 130.102.71.160 tcp port 53
   listen on 130.102.71.160 udp port 53
   listen on 130.102.71.161 tcp port 53
   listen on 130.102.71.161 udp port 53
 
   match pftag rns
   forward to rns port 53 check icmp
   }
 
 i wish redirects took a prefix. maybe thats a diff for another day.


Would make sense.

 anyway, here's the diff.
 
 thoughts? tweaks? ok?
 

In addition to my general concern, see comments inline below.

Reyk

 Index: parse.y
 ===
 RCS file: /cvs/src/usr.sbin/relayd/parse.y,v
 retrieving revision 1.204
 diff -u -p -r1.204 parse.y
 --- parse.y   2 May 2015 13:15:24 -   1.204
 +++ parse.y   14 May 2015 11:30:28 -
 @@ -173,6 +173,7 @@ typedef struct {
  %token   ROUTER RTLABEL TRANSPARENT TRAP UPDATES URL VIRTUAL WITH TTL 
 RTABLE
  %token   MATCH PARAMS RANDOM LEASTSTATES SRCHASH KEY CERTIFICATE 
 PASSWORD ECDH
  %token   EDH CURVE
 +%token   NONE LOCAL CONNECTED STATIC OSPF ISIS RIP BGP DEFAULT

A lot of keywords for basically an enum.  See below.

  %token   v.string  STRING
  %token  v.number   NUMBER
  %typev.string  hostname interface table value optstring
 @@ -182,6 +183,7 @@ typedef struct {
  %typev.number  redirect_proto relay_proto match
  %typev.number  action ruleaf key_option
  %typev.number  tlsdhparams tlsecdhcurve
 +%typev.number  rtprio
  %typev.portport
  %typev.hosthost
  %typev.addraddress
 @@ -1864,8 +1866,8 @@ router  : ROUTER STRING {
   router = rt;
  
   tableport = -1;
 - } '{' optnl routeopts_l '}' {
 - if (!router-rt_conf.nroutes) {
 + } '{' optnl routeropts_l '}'{
 + if (TAILQ_EMPTY(router-rt_netroutes)) {
   yyerror(router %s without routes,
   router-rt_conf.name);
   free(router);
 @@ -1881,11 +1883,11 @@ router: ROUTER STRING {
   }
   ;
  
 -routeopts_l  : routeopts_l routeoptsl nl
 - | routeoptsl optnl
 +routeropts_l : routeropts_l routeroptsl nl
 + | routeroptsl optnl
   ;
  
 -routeoptsl   : ROUTE address '/' NUMBER {
 +routeroptsl  : ROUTE address '/' NUMBER {
   struct netroute *nr;
  
   if (router-rt_conf.af == AF_UNSPEC)
 @@ -1914,15 +1916,15 @@ routeoptsl: ROUTE address '/' NUMBER {
   YYERROR;
   }
   

Thinkpad dock only provides HDMI2 after first suspend.

2015-06-22 Thread Edd Barrett
Hi,

I use a docking station for my x240t. It's a Thinkpad Ultrabase Series 3
P/N 0B67692. It has a ton of ports including a HDMI port, which shows as
HDMI2 under X11. The problem is, this port only becomes usable after a
suspend and resume cycle. This is awkward since I move between different
monitor configurations frequently.

I've spoken to matthieu@ and jsg@ about this briefly, but we did not
find a fix. I'm posting here in case others have any ideas.

See below the output of xrandr before and after zzz, and also a full
dmesg.

Cheers

Before:

wilfred:edd xrandr
Screen 0: minimum 8 x 8, current 1280 x 1024, maximum 32767 x 32767
LVDS1 connected (normal left inverted right x axis y axis)
   1366x768  60.02 +
   1024x768  60.00  
   800x600   60.3256.25  
   640x480   59.94  
DP1 disconnected (normal left inverted right x axis y axis)
DP2 disconnected (normal left inverted right x axis y axis)
HDMI1 disconnected (normal left inverted right x axis y axis)
HDMI2 disconnected (normal left inverted right x axis y axis)
VGA1 connected 1280x1024+0+0 (normal left inverted right x axis y axis) 376mm x 
301mm
   1280x1024 60.02*+  75.02  
   1152x864  75.00  
   1024x768  75.0860.00  
   800x600   75.0060.32  
   640x480   75.0060.00  
   720x400   70.08  
VIRTUAL1 disconnected (normal left inverted right x axis y axis)

After:

wilfred:edd xrandr
Screen 0: minimum 8 x 8, current 1280 x 1024, maximum 32767 x 32767
LVDS1 connected (normal left inverted right x axis y axis)
   1366x768  60.02 +
   1024x768  60.00  
   800x600   60.3256.25  
   640x480   59.94  
DP1 disconnected (normal left inverted right x axis y axis)
DP2 disconnected (normal left inverted right x axis y axis)
HDMI1 disconnected (normal left inverted right x axis y axis)
HDMI2 connected (normal left inverted right x axis y axis)
   1920x1080 60.00 +
   1680x1050 59.88  
   1280x1024 60.02  
   1440x900  59.90  
   1280x800  59.91  
   1280x720  59.97  
   1024x768  70.0760.00  
   800x600   60.3256.25  
   640x480   66.6760.00  
   720x400   70.08  
VGA1 connected 1280x1024+0+0 (normal left inverted right x axis y axis) 376mm x 
301mm
   1280x1024 60.02*+  75.02  
   1152x864  75.00  
   1024x768  75.0860.00  
   800x600   75.0060.32  
   640x480   75.0060.00  
   720x400   70.08  
VIRTUAL1 disconnected (normal left inverted right x axis y axis)


Dmesg:

OpenBSD 5.8-beta (GENERIC.MP) #1090: Sun Jun 21 13:19:59 MDT 2015
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 16844521472 (16064MB)
avail mem = 16330137600 (15573MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xdae9d000 (68 entries)
bios0: vendor LENOVO version GCET92WW (2.52 ) date 02/25/2013
bios0: LENOVO 3437CTO
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SLIC TCPA SSDT SSDT SSDT HPET APIC MCFG ECDT FPDT ASF! 
UEFI UEFI POAT SSDT SSDT UEFI DBG2
acpi0: wakeup devices LID_(S4) SLPB(S3) IGBE(S4) EXP3(S4) XHCI(S3) EHC1(S3) 
EHC2(S3) HDEF(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i5-3320M CPU @ 2.60GHz, 2594.62 MHz
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1.2, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Core(TM) i5-3320M CPU @ 2.60GHz, 2594.11 MHz
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 1, core 0, package 0
cpu2 at mainbus0: apid 2 (application processor)
cpu2: Intel(R) Core(TM) i5-3320M CPU @ 2.60GHz, 2594.11 MHz
cpu2: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,SENSOR,ARAT
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 1, package 0
cpu3 at mainbus0: apid 3 (application processor)

sed(1) negative patterns

2015-06-22 Thread Liviu Daia
Currently, ! in sed(1) addresses acts as a toggle: /foo/!!d is
equivalent to /foo/d.  However, POSIX mandates that multiple ! should
be treated as a single negation:

: A function can be preceded by one or more '!' characters, in which
: case the function shall be applied if the addresses do not select the
: pattern space.

(cf. 
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html#tag_20_116_13_03)

Now, I can't really fathom the rationale behind that phrase (did
they reason that multiple ! make a gang bang, rather than a multiple
logical negations?), but the phrase is there.  FreeBSD has fixed this in
November 2008:

https://github.com/freebsd/freebsd/commit/83407db259b48d45da4d45d23fe20f2565f89a8b

GNU sed(1) on the other hand simply rejects the pattern:

sed: -e expression #1, char 7: multiple `!'s

Anyway, this can be fixed the same way FreeBSD did:

Index: compile.c
===
RCS file: /cvs/src/usr.bin/sed/compile.c,v
retrieving revision 1.37
diff -u -p -r1.37 compile.c
--- compile.c   12 Dec 2014 03:32:55 -  1.37
+++ compile.c   22 Jun 2015 17:18:31 -
@@ -208,7 +208,7 @@ nonsel: /* Now parse the command */
case NONSEL:/* ! */
p++;
EATSPACE();
-   cmd-nonsel = ! cmd-nonsel;
+   cmd-nonsel = 1;
goto nonsel;
case GROUP: /* { */
p++;

Regards,

Liviu Daia



Re: Conver bridge(4) to if_input()

2015-06-22 Thread Alexander Bluhm
On Mon, Jun 22, 2015 at 05:31:29PM +0200, Martin Pieuchot wrote:
 On 22/06/15(Mon) 17:12, Alexander Bluhm wrote:
  On Wed, Jun 17, 2015 at 02:08:14PM +0200, Martin Pieuchot wrote:
   diff -u -p -r1.340 if.c
   --- net/if.c  16 Jun 2015 11:09:39 -  1.340
   +++ net/if.c  17 Jun 2015 12:03:36 -
   @@ -530,6 +530,15 @@ if_input_process(void *xmq)
 continue;
 }

   +#if NBRIDGE  0
   + if (ifp-if_bridgeport  (m-m_flags  M_PROTO1) == 0) {
   + m = bridge_input(m);
   + if (m == NULL)
   + continue;
   + }
   + m-m_flags = ~M_PROTO1;/* Loop prevention */
   +#endif
  
  Should we reset the loop prevention only if our call to bridge_input()
  did set M_PROTO1?  Something like this
  
  if (ifp-if_bridgeport  (m-m_flags  M_PROTO1) == 0) {
  m = bridge_input(m);
  if (m == NULL)
  continue;
  m-m_flags = ~M_PROTO1;/* Loop prevention */
  }
 
 Yes and no :)
 
 bridge_input() will set M_PROTO1 on the mbuf copies that it enqueues on
 its ports.
 
 If you receive a packet on em0 in bridge0 with tun0, you want to call
 bridge_input() only once, but you'll call if_input() in em0 and tun0.
 
 So the first packet will enter if_input() without M_PROTO1, go through
 bridge_input() then be processed by the stack.  Then the copy of this
 packet created in bridge_input() will have the M_PROTO1 flag set and
 when it will be dequeued by if_input() it won't be passed to 
 bridge_input() again.
 
 Does that make sense?

I have tested bridge loops with vlans.  The packets run in a circle
with both implementations.  Softnet task goes to 100% but no crashes
or stack overflow.

So with your explanation and my test I am fine with your loop
prevention.

OK bluhm@



Better support for write combining mappings in libpciaccess

2015-06-22 Thread Mark Kettenis
This needs my last diff that adds MEMRANGE_WC_RANGE to become
functional.  Then it will use the PAT (on CPUs that have PAT) to
establish write combining mappings.  Modern systems don't seem to
properly set up MTRRs anymore, and this is a lot easier than setting
up mtrr's ourselves.

ok?


Index: src/openbsd_pci.c
===
RCS file: /cvs/xenocara/lib/libpciaccess/src/openbsd_pci.c,v
retrieving revision 1.23
diff -u -p -r1.23 openbsd_pci.c
--- src/openbsd_pci.c   10 May 2015 09:31:51 -  1.23
+++ src/openbsd_pci.c   22 Jun 2015 19:00:46 -
@@ -192,12 +192,18 @@ pci_device_openbsd_map_range(struct pci_
struct mem_range_desc mr;
struct mem_range_op mo;
int prot = PROT_READ;
+   off_t addr = map-base;
 
if (map-flags  PCI_DEV_MAP_FLAG_WRITABLE)
prot |= PROT_WRITE;
 
+#ifdef MEMRANGE_WC_RANGE
+   if (map-flags  PCI_DEV_MAP_FLAG_WRITABLE)
+   addr += MEMRANGE_WC_RANGE;
+#endif
+
map-memory = mmap(NULL, map-size, prot, MAP_SHARED, aperturefd,
-   map-base);
+   addr);
if (map-memory == MAP_FAILED)
return  errno;
 #if defined(__i386__) || defined(__amd64__)



Re: [patch] cleaner checksum modification for pf

2015-06-22 Thread Richard Procter

On 16/06/2015, at 1:09 PM, Richard Procter wrote:

 - I was unable to test af-to, which does a lot of packet fiddling.

I've now tested this without obvious issue. Note: a couple of 
one-line changes in icmp af-translation remain untested. Details 
attached.

 - I haven't tested modification of unaligned TCP options,
   for SACK and timestamp. 

This tested without issue, too.

Also, some points raised in off-list comment suggested I was unclear in my
patch post, which I'll address here.

- Note this diff does not revert all of Henning's checksum work, just returns
to checksum modification for pf-altered packets as in 5.3[0] but without the 
ugly nested pf_checksum_fixup() calls and without (I expect) loss of 
efficiency. I've aimed for the minimal necessary changes only.

And the patch is made much easier now that pseudo-header checksums are
calculated late in the output path, as pf now never sees a partial checksum.
Worst-case, pf does some unnecessary arithmetic when altering locally generated
packets, but no more than the original 5.3 code did, as the 5.3 modification
code didn't distinguish between local and forwarded packets either.

- The first patch renames pf_change_a() to pf_change_32_unaligned() as it was
not being used for addresses alone, and I needed to move it aside for an
address-specific pf_change_a().

- Some commented that, although the patch re-enables end-to-end verification of
transport payloads traversing pf, it is unimportant as we now have far stronger
end-to-end checks in ssh and TLS.

These protocols, however, assume a reliable transport[1]: they terminate the
connection on a failed MAC as evidence of malicious tampering[*]. If they
didn't they would need to reimplement TCP's retransmission strategies, as 
the Internet can be expected to damage packets as a matter of course. So TCP 
checksums continue to play an important role for secure streams built atop TCP
and this diff bears upon them, too. 


As usual, comment, questions, or criticism is welcome.

best, 
Richard.
 
[0] 5.3, not 5.4 as I stated
[1] for those interested in the fine print: 
rfc4253 (2006) SSH p3
rfc5246 (2008) TLSv1.2 p3
[*] IIRC someone last year reported mysterious ssh disconnects due to corrupted
MACs which Stuart Henderson pointed out could have been due to a flakey NAT
router.

Test details:

- unaligned SACK, timestamp options
- tested on i386, and I expect no issues for architectures with stronger 
  alignment constraints but aiming to test this in the weekend on socppc 
  for kicks. 

- pf_translate_icmp_af
These two remain untested due to my lack of test nodes:
nextmtu for ICMP6_PACKET_TOO_BIG 
pptr for ICMP_PARAMPROB
but low risk as these are one line substitutions with tested primitives, 
and nextmtu is always changed.

- translate quoted address family (af-to)
via pf_change_icmp_af pf_change_ap

Testing af-to 4 - 6 
pass in on ral0 inet from any to 192.168.2.2 af-to inet6 from 
  fec0:0:0:2::1 to fec0:0:0:2::2

   af-to TCP
- inet4 telnet to inet6 netcat host [TESTED]
   af-to ICMP
- unassociated with connection
 - src ICMP4 ping [TESTED]
- associated with connection
 - traceroute elicits ICMP6 port unreachable
   from dest; translated result inspected with wireshark [TESTED]

Testing af-to 6 - 4 
pass in on vr0 inet6 from any to 64:ff9b::/96 af-to inet from 192.168.2.3

   af-to TCP
- inet6 telnet to inet4 netcat host [TESTED]
   af-to ICMP
- unassociated with connection
 - src ICMP6 ping [TESTED]
- associated with connection
 - traceroute6 elicits ICMP4 port unreachable
   from dest; translated result inspected with wireshark. [TESTED]






Huawei ME909u-521 - cannot find data bulk in

2015-06-22 Thread Brian S. Vangsgaard

Hi,

(this is a repost from misc@)

Needing a backup connection to the internet, I decided to give the 
Huawei 909u-521 a try.


Hardware details: 
http://consumer.huawei.com/en/solutions/m2m-solutions/products/tech-specs/me909u-521mini-pcie-en.htm


It's mounted in a Soekris 6501 device.

During boot, the following information is given;

cdce0 at uhub1 port 1 configuration 2 interface 0 HUAWEI Technology 
HUAWEI Mobile rev 2.00/2.28 addr 2

cdce0: could not find data bulk in
ugen0 at uhub1 port 1 configuration 2 HUAWEI Technology HUAWEI Mobile 
rev 2.00/2.28 addr 2


A little more information..

$ usbdevs -dv
...
Controller /dev/usb1:
addr 1: high speed, self powered, config 1, EHCI root hub(0x), 
Intel(0x8086), rev 1.00

  uhub1
 port 1 addr 2: high speed, power 500 mA, config 2, HUAWEI 
Mobile(0x1573), HUAWEI Technology(0x12d1), rev 2.28, iSerialNumber 
0123456712ABCA17

   cdce0
   ugen0
 port 2 powered
 port 3 powered
...

So, the device seems to be detected, but some tweaking might be 
required.


 Many USB devices notoriously fail to report their class and 
interfaces
 correctly.  Undetected products might work flawlessly when their 
vendor

 and product IDs are added to the driver manually.

Can anyone tell me how to move on from this point, what steps are 
needed to get the id's added manually?



--
Regards Brian



Re: umass quirk for ignoring residue?

2015-06-22 Thread Martin Pieuchot
On 20/06/15(Sat) 18:50, frantisek holop wrote:
 trying to use a replacement usb enclosure for an old 2.5 IDE drive.
 [...] 
 Controller /dev/usb0:
 addr 1: high speed, self powered, config 1, EHCI root hub(0x), 
 Intel(0x8086), rev 1.00
  port 2 addr 6: high speed, self powered, config 1, USB 2.0  IDE 
 DEVICE(0x6600), Super Top(0x14cd), rev 2.01, iSerialNumber ??
 
 apparently the USB 2.0  IDE DEVICE(0x6600) from Super Top(0x14cd)
 is a well known offender...
 
 netbsd/dev/usb: (in usbdevs but no quirk)
   usbdevs
   504:vendor SUPERTOP  0x14cd  SuperTop
   3116:product SUPERTOP IDEBRIDGE  0x6600  SuperTop IDE Bridge
 
 
 freebsd/dev/usb:
   usbdevs
   649:vendor SUPERTOP  0x14cd  Super Top
   4316:product SUPERTOP IDE0x6600  USB-IDE
 
   quirk/usb_quirk.c
   410: USB_QUIRK(SUPERTOP, IDE, 0x, 0x, UQ_MSC_IGNORE_RESIDUE,
   411: UQ_MSC_NO_SYNC_CACHE),
 
 
 linux/drivers/usb:
   storage/unusual_devs.h
   1976:UNUSUAL_DEV(  0x14cd, 0x6600, 0x0201, 0x0201,
   1977:Super Top,
   1978:IDE DEVICE,
   1979:USB_SC_DEVICE, USB_PR_DEVICE, NULL,
   1980:US_FL_IGNORE_RESIDUE ),
 
 
 trying to compare the different kernels re: ignoring residue,
 in openbsd ADEV_NOSENSE seemed like the way to go:

I believe it's the way to go.

 Index: umass_quirks.c
 ===
 RCS file: /cvs/src/sys/dev/usb/umass_quirks.c,v
 retrieving revision 1.31
 diff -u -p -r1.31 umass_quirks.c
 --- umass_quirks.c5 Oct 2014 08:34:14 -   1.31
 +++ umass_quirks.c20 Jun 2015 16:24:11 -
 @@ -465,6 +465,14 @@ const struct umass_quirk umass_quirks[] 
UMATCH_VENDOR_PRODUCT,
NULL, NULL
   },
 +
 + { { USB_VENDOR_SUPERTOP, USB_PRODUCT_SUPERTOP_IDEBRIDGE },
 +  UMASS_WPROTO_UNSPEC, UMASS_CPROTO_UNSPEC,
 +  0,
 +  ADEV_NOSENSE,
 +  UMATCH_VENDOR_PRODUCT,
 +  NULL, NULL
 + },
  };
  
  const struct umass_quirk *
 Index: usbdevs
 ===
 RCS file: /cvs/src/sys/dev/usb/usbdevs,v
 retrieving revision 1.651
 diff -u -p -r1.651 usbdevs
 --- usbdevs   16 Jun 2015 05:07:25 -  1.651
 +++ usbdevs   20 Jun 2015 16:24:11 -
 @@ -534,6 +534,7 @@ vendor SILICOM0x1485  Silicom
  vendor RALINK0x148f  Ralink Technology
  vendor STARTECH  0x14b0  StarTech.com
  vendor CONCEPTRONIC2 0x14b2  Conceptronic
 +vendor SUPERTOP  0x14cd  SuperTop
  vendor PLANEX3   0x14ea  Planex Communications
  vendor SILICONPORTALS0x1527  Silicon Portals
  vendor UBLOX 0x1546  U-blox
 @@ -4058,6 +4059,9 @@ product SUNTAC VS10U0x0009  Slipper U
  product SUNTAC IS96U 0x000a  Ir-Trinity
  product SUNTAC AS64LX0x000b  U-Cable type A3
  product SUNTAC AS144L4   0x0011  U-Cable type A4
 +
 +/* SuperTop products */
 +product SUPERTOP IDEBRIDGE   0x6600  SuperTop IDE Bridge
  
  /* System TALKS, Inc. */
  product  SYSTEMTALKS SGCX2UL 0x1920  SGC-X2UL
 
 
 but this did not seem to help.
 
 here is the output from usb/umass debug enabled kernel with _only_
 the usbdevs part of the patch, not the quirk.
 connecting the drive, waiting a couple of seconds
 and issuing sudo fdisk sd1:
 
 Jun 20 17:55:13 hatvan /bsd: umass0 at uhub0
 Jun 20 17:55:13 hatvan /bsd:  port 2 configuration 1 interface 0 Super Top 
 USB 2.0  IDE DEVICE rev 2.00/2.01 addr 6
 Jun 20 17:55:13 hatvan /bsd: umass0: using SCSI over Bulk-Only
 Jun 20 17:55:14 hatvan /bsd: umass0: Get Max Lun
 Jun 20 17:55:14 hatvan /bsd: umass0: Max Lun 0
 Jun 20 17:55:14 hatvan /bsd: umass0: opening iface 0xd3364360 epaddr 2 for 
 BULKOUT
 Jun 20 17:55:14 hatvan /bsd: umass0: opening iface 0xd3364360 epaddr 129 for 
 BULKIN
 Jun 20 17:55:14 hatvan /bsd: umass0: umass_attach_bus: SCSI
 Jun 20 17:55:14 hatvan /bsd: sc = 0x0xd3361400, scbus = 0x0xd3362800
 Jun 20 17:55:14 hatvan /bsd: scsibus4 at umass0: 2 targets, initiator 0
 Jun 20 17:55:14 hatvan /bsd: umass0: umass_scsi_cmd: at 1434815714.000358: 
 1:0 xs=0xd9181000 cmd=0x00 datalen=0 (quirks=0x400e, poll=0)
 Jun 20 17:55:14 hatvan /bsd: umass_scsi_cmd: async dir=0, cmdlen=6 datalen=0
 Jun 20 17:55:14 hatvan /bsd: umass0: umass_bbb_transfer cmd=0x00
 Jun 20 17:55:14 hatvan /bsd: umass0: CBW 158: cmdlen=6 
 (0x), data = 0 bytes, dir = out
 Jun 20 17:55:14 hatvan /bsd: umass0: start xfer buffer=0xd3361464 buflen=31 
 flags=0x0 timeout=15000
 Jun 20 17:55:14 hatvan /bsd: umass0: Handling BBB state 1 (BBB CBW), 
 xfer=0xd9183460, NORMAL_COMPLETION
 Jun 20 17:55:14 hatvan /bsd: umass0: no data phase
 Jun 20 17:55:14 hatvan /bsd: umass0: start xfer buffer=0xd3361483 buflen=13 
 flags=0x0 timeout=15000
 Jun 20 17:55:14 hatvan /bsd: umass0: Handling BBB state 4 (BBB CSW, 1st