Re: ANONCVS MIRROR MAINTAINERS.. YOU NEED TO READ THIS!
On Wed, Jun 25, 2014 at 10:01:06PM -0700, patrick keshishian wrote: On Thu, Jun 26, 2014 at 06:37:00AM +0200, Alexander Hall wrote: On 06/25/14 20:52, Bob Beck wrote: If you or someone you love runs an anoncvs server, they need to see this. As you know we recently added commitid support to cvs, and we had you update your cvsync binary. Unfortunately, the fix wasn't quite right. We ran into problems with the synching of commitid files. naddy managed to cook a correct fix for us. anoncvs.ca (the fanout machine) has been fixed - again. What you need to do is to update your cvsync to the latest one that was just committed into ports (cvsync-0.25.0pre0 or later). Remove your scanfile (if any). Once you do that, re-run cvsync and you should be back in business. Is it known whether this requires a fixed upstream cvsync first, or if it is ok to apply the fix and your repo will become fine whenever the upstream is fixed? Anyone with insight, please feel free to chime in. The commit message suggests, at least, the server must be updated: | Add full support for commitid and bump protocol version | Old clients will receive updates with commitid stripped out. It seems, running updated cvsync against an un-updated cvsyncd (I assume) results in failure: Connecting to anoncvs1.usa.openbsd.org port Connected to 149.20.54.217 port Running... Updating (collection openbsd-src/rcs) Updater(RCS): UPDATE(delta): error Updater(RCS): UPDATE: /cvs/anoncvs/cvs/obsd/src/lib/libc/string/timingsafe_bcmp.3,v Updater: RCS Error Socket Error: recv: 2 residue 2 Receiver Error: recv Mux(SEND) Error: not running: 0 DirScan: RCS Error Mux(SEND) Error: not running: 1 FileScan(RCS): UPDATE /cvs/anoncvs/cvs/obsd/src/sbin/pfctl/parse.y,v FileScan: RCS Error Failed ouch :-) --patrick HTH, --patrick /Alexander to check a server do: cvs -d anon...@yourserver.org:/cvs log /usr/src/libexec/ld.so/malloc.c | less and see if it has r 1.3 Thanks, and sorry for the disruption, -Bob
Re: ANONCVS MIRROR MAINTAINERS.. YOU NEED TO READ THIS!
On 26 June 2014 08:53, patrick keshishian sids...@boxsoft.com wrote: On Wed, Jun 25, 2014 at 10:01:06PM -0700, patrick keshishian wrote: On Thu, Jun 26, 2014 at 06:37:00AM +0200, Alexander Hall wrote: On 06/25/14 20:52, Bob Beck wrote: If you or someone you love runs an anoncvs server, they need to see this. As you know we recently added commitid support to cvs, and we had you update your cvsync binary. Unfortunately, the fix wasn't quite right. We ran into problems with the synching of commitid files. naddy managed to cook a correct fix for us. anoncvs.ca (the fanout machine) has been fixed - again. What you need to do is to update your cvsync to the latest one that was just committed into ports (cvsync-0.25.0pre0 or later). Remove your scanfile (if any). Once you do that, re-run cvsync and you should be back in business. Is it known whether this requires a fixed upstream cvsync first, or if it is ok to apply the fix and your repo will become fine whenever the upstream is fixed? Anyone with insight, please feel free to chime in. The commit message suggests, at least, the server must be updated: | Add full support for commitid and bump protocol version | Old clients will receive updates with commitid stripped out. It seems, running updated cvsync against an un-updated cvsyncd (I assume) results in failure: Connecting to anoncvs1.usa.openbsd.org port Connected to 149.20.54.217 port Running... Updating (collection openbsd-src/rcs) Updater(RCS): UPDATE(delta): error Updater(RCS): UPDATE: /cvs/anoncvs/cvs/obsd/src/lib/libc/string/timingsafe_bcmp.3,v Updater: RCS Error Socket Error: recv: 2 residue 2 Receiver Error: recv Mux(SEND) Error: not running: 0 DirScan: RCS Error Mux(SEND) Error: not running: 1 FileScan(RCS): UPDATE /cvs/anoncvs/cvs/obsd/src/sbin/pfctl/parse.y,v FileScan: RCS Error Failed removing offending files locally and restarting cvsync helps.
Re: ANONCVS MIRROR MAINTAINERS.. YOU NEED TO READ THIS!
On 2014/06/26 11:02, Mike Belopuhov wrote: On 26 June 2014 08:53, patrick keshishian sids...@boxsoft.com wrote: On Wed, Jun 25, 2014 at 10:01:06PM -0700, patrick keshishian wrote: On Thu, Jun 26, 2014 at 06:37:00AM +0200, Alexander Hall wrote: On 06/25/14 20:52, Bob Beck wrote: If you or someone you love runs an anoncvs server, they need to see this. As you know we recently added commitid support to cvs, and we had you update your cvsync binary. Unfortunately, the fix wasn't quite right. We ran into problems with the synching of commitid files. naddy managed to cook a correct fix for us. anoncvs.ca (the fanout machine) has been fixed - again. What you need to do is to update your cvsync to the latest one that was just committed into ports (cvsync-0.25.0pre0 or later). Remove your scanfile (if any). Once you do that, re-run cvsync and you should be back in business. Is it known whether this requires a fixed upstream cvsync first, or if it is ok to apply the fix and your repo will become fine whenever the upstream is fixed? Anyone with insight, please feel free to chime in. The commit message suggests, at least, the server must be updated: | Add full support for commitid and bump protocol version | Old clients will receive updates with commitid stripped out. It seems, running updated cvsync against an un-updated cvsyncd (I assume) results in failure: Connecting to anoncvs1.usa.openbsd.org port Connected to 149.20.54.217 port Running... Updating (collection openbsd-src/rcs) Updater(RCS): UPDATE(delta): error Updater(RCS): UPDATE: /cvs/anoncvs/cvs/obsd/src/lib/libc/string/timingsafe_bcmp.3,v Updater: RCS Error Socket Error: recv: 2 residue 2 Receiver Error: recv Mux(SEND) Error: not running: 0 DirScan: RCS Error Mux(SEND) Error: not running: 1 FileScan(RCS): UPDATE /cvs/anoncvs/cvs/obsd/src/sbin/pfctl/parse.y,v FileScan: RCS Error Failed removing offending files locally and restarting cvsync helps. I suspect this may only help until they receive another commit. The server *must* be updated - though it looks like there may still be some problems.
Re: ANONCVS MIRROR MAINTAINERS.. YOU NEED TO READ THIS!
On 2014/06/26 06:37, Alexander Hall wrote: On 06/25/14 20:52, Bob Beck wrote: If you or someone you love runs an anoncvs server, they need to see this. As you know we recently added commitid support to cvs, and we had you update your cvsync binary. Unfortunately, the fix wasn't quite right. We ran into problems with the synching of commitid files. naddy managed to cook a correct fix for us. anoncvs.ca (the fanout machine) has been fixed - again. What you need to do is to update your cvsync to the latest one that was just committed into ports (cvsync-0.25.0pre0 or later). Remove your scanfile (if any). Once you do that, re-run cvsync and you should be back in business. Is it known whether this requires a fixed upstream cvsync first, or if it is ok to apply the fix and your repo will become fine whenever the upstream is fixed? If you run the client with cvsync -v, it will show you whether the server is updated (Protocol: 0.25) or not (Protocol: 0.24).
carp(4) dead code
There's no way to have a negative number of addresses configured on a
carp(4) interface.
This hack might have been required in the past when the SIOCDIFADDR
handler was doing sc-sc_naddrs-- but it is definitively dead code
now.
Ok to kill it?
Index: netinet/ip_carp.c
===
RCS file: /home/ncvs/src/sys/netinet/ip_carp.c,v
retrieving revision 1.229
diff -u -p -r1.229 ip_carp.c
--- netinet/ip_carp.c 30 Apr 2014 10:04:33 - 1.229
+++ netinet/ip_carp.c 26 Jun 2014 09:48:01 -
@@ -1697,24 +1697,6 @@ carp_set_ifp(struct carp_softc *sc, stru
if (sc-sc_carpdev != NULL)
carpdetach(sc);
- /* join multicast groups */
- if (sc-sc_naddrs 0
- (error = carp_join_multicast(sc)) != 0) {
- if (ncif != NULL)
- free(ncif, M_IFADDR);
- return (error);
- }
-
-#ifdef INET6
- if (sc-sc_naddrs6 0
- (error = carp_join_multicast6(sc)) != 0) {
- if (ncif != NULL)
- free(ncif, M_IFADDR);
- carp_multicast_cleanup(sc);
- return (error);
- }
-#endif
-
/* attach carp interface to physical interface */
if (ncif != NULL)
ifp-if_carp = (caddr_t)ncif;
Re: ANONCVS MIRROR MAINTAINERS.. YOU NEED TO READ THIS!
On 06/25/14 21:16, Nick Holland wrote: openbsd.cs.toronto.edu updated (again) and cvsyncd restarted. Unfortunately, this was long after the above note. *blush* But yes, there are still issues. Nick. On 06/25/14 14:51, Bob Beck wrote: If you or someone you love runs an anoncvs server, they need to see this. As you know we recently added commitid support to cvs, and we had you update your cvsync binary. Unfortunately, the fix wasn't quite right. We ran into problems with the synching of commitid files. naddy managed to cook a correct fix for us. anoncvs.ca (the fanout machine) has been fixed - again. What you need to do is to update your cvsync to the latest one that was just committed into ports (cvsync-0.25.0pre0 or later). Remove your scanfile (if any). Once you do that, re-run cvsync and you should be back in business. to check a server do: cvs -d anon...@yourserver.org:/cvs log /usr/src/libexec/ld.so/malloc.c | less and see if it has r 1.3 Thanks, and sorry for the disruption, -Bob
Re: increase netcat's buffer...
Hi.
Now soliciting diffs to change readwrite to a loop with two buffers
that poll()s in all four directions. :)
Good thing you made me remember I wrote just this a while ago.
This is my first OpenBSD diff, so tell me if I missed anything obvious.
Tested quite extensively originally; for this diff I only checked a
simple nc to nc hello.
Index: netcat.c
===
RCS file: /mount/cvsdev/cvs/openbsd/src/usr.bin/nc/netcat.c,v
retrieving revision 1.121
diff -u -p -r1.121 netcat.c
--- netcat.c10 Jun 2014 16:35:42 - 1.121
+++ netcat.c26 Jun 2014 11:29:45 -
@@ -65,6 +65,12 @@
#define PORT_MAX_LEN 6
#define UNIX_DG_TMP_SOCKET_SIZE19
+#define POLL_STDIN 0
+#define POLL_NETOUT 1
+#define POLL_NETIN 2
+#define POLL_STDOUT 3
+#define BUFSIZE 16384
+
/* Command Line Options */
intdflag; /* detached, no stdin */
intFflag; /* fdpass sock to stdout */
@@ -112,6 +118,8 @@ voidset_common_sockopts(int);
intmap_tos(char *, int *);
void report_connect(const struct sockaddr *, socklen_t);
void usage(int);
+ssize_t drainbuf(int, unsigned char *, size_t *);
+ssize_t fillbuf(int, unsigned char *, size_t *);
int
main(int argc, char *argv[])
@@ -608,7 +616,7 @@ remote_connect(const char *host, const c
if (bind(s, (struct sockaddr *)ares-ai_addr,
ares-ai_addrlen) 0)
- err(1, bind failed);
+ errx(1, bind failed: %s, strerror(errno));
freeaddrinfo(ares);
}
@@ -640,7 +648,7 @@ timeout_connect(int s, const struct sock
if (timeout != -1) {
flags = fcntl(s, F_GETFL, 0);
if (fcntl(s, F_SETFL, flags | O_NONBLOCK) == -1)
- err(1, set non-blocking mode);
+ warn(unable to set non-blocking mode);
}
if ((ret = connect(s, name, namelen)) != 0 errno == EINPROGRESS) {
@@ -730,67 +738,229 @@ local_listen(char *host, char *port, str
* Loop that polls on the network file descriptor and stdin.
*/
void
-readwrite(int nfd)
+readwrite(int net_fd)
{
- struct pollfd pfd[2];
- unsigned char buf[16 * 1024];
- int n, wfd = fileno(stdin);
- int lfd = fileno(stdout);
- int plen;
-
- plen = sizeof(buf);
-
- /* Setup Network FD */
- pfd[0].fd = nfd;
- pfd[0].events = POLLIN;
+ struct pollfd pfd[4];
+ int stdin_fd = STDIN_FILENO;
+ int stdout_fd = STDOUT_FILENO;
+ unsigned char netinbuf[BUFSIZE];
+ size_t netinbufpos = 0;
+ unsigned char stdinbuf[BUFSIZE];
+ size_t stdinbufpos = 0;
+ int n, num_fds, flags;
+ ssize_t ret;
+
+ /* don't read from stdin if requested */
+ if (dflag)
+ stdin_fd = -1;
+
+ /* stdin */
+ pfd[POLL_STDIN].fd = stdin_fd;
+ pfd[POLL_STDIN].events = POLLIN;
+
+ /* network out */
+ pfd[POLL_NETOUT].fd = net_fd;
+ pfd[POLL_NETOUT].events = 0;
+
+ /* network in */
+ pfd[POLL_NETIN].fd = net_fd;
+ pfd[POLL_NETIN].events = POLLIN;
+
+ /* stdout */
+ pfd[POLL_STDOUT].fd = stdout_fd;
+ pfd[POLL_STDOUT].events = 0;
+
+
+ /* make all fds non-blocking */
+ for (n = 0; n 4; n++) {
+ if (pfd[n].fd == -1)
+ continue;
+ flags = fcntl(pfd[n].fd, F_GETFL, 0);
+ /*
+* For sockets and pipes, we want non-block, but setting it
+* might fail for files or devices, so we ignore the return
+* code.
+*/
+ fcntl(pfd[n].fd, F_SETFL, flags | O_NONBLOCK);
+ }
- /* Set up STDIN FD. */
- pfd[1].fd = wfd;
- pfd[1].events = POLLIN;
+ while (1) {
+ /* both inputs are gone, buffers are empty, we are done */
+ if (pfd[POLL_STDIN].fd == -1 pfd[POLL_NETIN].fd == -1
+stdinbufpos == 0 netinbufpos == 0) {
+ close(net_fd);
+ return;
+ }
+ /* both outputs are gone, we can't continue */
+ if (pfd[POLL_NETOUT].fd == -1 pfd[POLL_STDOUT].fd == -1) {
+ close(net_fd);
+ return;
+ }
+ /* listen and net in gone, queues empty, done */
+ if (lflag pfd[POLL_NETIN].fd == -1
+stdinbufpos == 0 netinbufpos == 0) {
+ close(net_fd);
+ return;
+ }
- while (pfd[0].fd != -1) {
+ /* help says -i is for wait between lines sent. We read and
+* write arbitray amounts of data, and we don't want to start
+* scanning for newlines,
Re: ANONCVS MIRROR MAINTAINERS.. YOU NEED TO READ THIS!
2014-06-25 20:52 GMT+02:00, Bob Beck b...@obtuse.com: If you or someone you love runs an anoncvs server, they need to see this. As you know we recently added commitid support to cvs, and we had you update your cvsync binary. anoncvs.eu.openbsd.org updated and cvsyncd restarted, but I don't think I get updated repos myself from the feeds I've tried. -- May the most significant bit of your life be positive.
Re: increase netcat's buffer...
On Thu, Jun 26, 2014 at 7:43 AM, Arne Becker arne_bec...@genua.de wrote:
Hi.
Now soliciting diffs to change readwrite to a loop with two buffers
that poll()s in all four directions. :)
Good thing you made me remember I wrote just this a while ago.
This is my first OpenBSD diff, so tell me if I missed anything obvious.
Tested quite extensively originally; for this diff I only checked a
simple nc to nc hello.
Index: netcat.c
===
RCS file: /mount/cvsdev/cvs/openbsd/src/usr.bin/nc/netcat.c,v
retrieving revision 1.121
diff -u -p -r1.121 netcat.c
--- netcat.c10 Jun 2014 16:35:42 - 1.121
+++ netcat.c26 Jun 2014 11:29:45 -
@@ -65,6 +65,12 @@
#define PORT_MAX_LEN 6
#define UNIX_DG_TMP_SOCKET_SIZE19
+#define POLL_STDIN 0
+#define POLL_NETOUT 1
+#define POLL_NETIN 2
+#define POLL_STDOUT 3
+#define BUFSIZE 16384
+
/* Command Line Options */
intdflag; /* detached, no stdin */
intFflag; /* fdpass sock to stdout */
@@ -112,6 +118,8 @@ voidset_common_sockopts(int);
intmap_tos(char *, int *);
void report_connect(const struct sockaddr *, socklen_t);
void usage(int);
+ssize_t drainbuf(int, unsigned char *, size_t *);
+ssize_t fillbuf(int, unsigned char *, size_t *);
int
main(int argc, char *argv[])
@@ -608,7 +616,7 @@ remote_connect(const char *host, const c
if (bind(s, (struct sockaddr *)ares-ai_addr,
ares-ai_addrlen) 0)
- err(1, bind failed);
+ errx(1, bind failed: %s, strerror(errno));
freeaddrinfo(ares);
}
@@ -640,7 +648,7 @@ timeout_connect(int s, const struct sock
if (timeout != -1) {
flags = fcntl(s, F_GETFL, 0);
if (fcntl(s, F_SETFL, flags | O_NONBLOCK) == -1)
- err(1, set non-blocking mode);
+ warn(unable to set non-blocking mode);
}
if ((ret = connect(s, name, namelen)) != 0 errno == EINPROGRESS) {
@@ -730,67 +738,229 @@ local_listen(char *host, char *port, str
* Loop that polls on the network file descriptor and stdin.
*/
void
-readwrite(int nfd)
+readwrite(int net_fd)
{
- struct pollfd pfd[2];
- unsigned char buf[16 * 1024];
- int n, wfd = fileno(stdin);
- int lfd = fileno(stdout);
- int plen;
-
- plen = sizeof(buf);
-
- /* Setup Network FD */
- pfd[0].fd = nfd;
- pfd[0].events = POLLIN;
+ struct pollfd pfd[4];
+ int stdin_fd = STDIN_FILENO;
+ int stdout_fd = STDOUT_FILENO;
+ unsigned char netinbuf[BUFSIZE];
+ size_t netinbufpos = 0;
+ unsigned char stdinbuf[BUFSIZE];
+ size_t stdinbufpos = 0;
+ int n, num_fds, flags;
+ ssize_t ret;
+
+ /* don't read from stdin if requested */
+ if (dflag)
+ stdin_fd = -1;
+
+ /* stdin */
+ pfd[POLL_STDIN].fd = stdin_fd;
+ pfd[POLL_STDIN].events = POLLIN;
+
+ /* network out */
+ pfd[POLL_NETOUT].fd = net_fd;
+ pfd[POLL_NETOUT].events = 0;
+
+ /* network in */
+ pfd[POLL_NETIN].fd = net_fd;
+ pfd[POLL_NETIN].events = POLLIN;
+
+ /* stdout */
+ pfd[POLL_STDOUT].fd = stdout_fd;
+ pfd[POLL_STDOUT].events = 0;
+
+
+ /* make all fds non-blocking */
+ for (n = 0; n 4; n++) {
+ if (pfd[n].fd == -1)
+ continue;
+ flags = fcntl(pfd[n].fd, F_GETFL, 0);
+ /*
+* For sockets and pipes, we want non-block, but setting it
+* might fail for files or devices, so we ignore the return
+* code.
+*/
+ fcntl(pfd[n].fd, F_SETFL, flags | O_NONBLOCK);
+ }
- /* Set up STDIN FD. */
- pfd[1].fd = wfd;
- pfd[1].events = POLLIN;
+ while (1) {
+ /* both inputs are gone, buffers are empty, we are done */
+ if (pfd[POLL_STDIN].fd == -1 pfd[POLL_NETIN].fd == -1
+stdinbufpos == 0 netinbufpos == 0) {
+ close(net_fd);
+ return;
+ }
+ /* both outputs are gone, we can't continue */
+ if (pfd[POLL_NETOUT].fd == -1 pfd[POLL_STDOUT].fd == -1) {
+ close(net_fd);
+ return;
+ }
+ /* listen and net in gone, queues empty, done */
+ if (lflag pfd[POLL_NETIN].fd == -1
lflag ???
warning only one ref in the diff
+stdinbufpos == 0 netinbufpos == 0) {
+ close(net_fd);
+ return;
+
Re: increase netcat's buffer...
On 2014/06/26 08:13, sven falempin wrote: trim lots of pointless lines + close(net_fd); + return; + } + /* listen and net in gone, queues empty, done */ + if (lflag pfd[POLL_NETIN].fd == -1 lflag ??? warning only one ref in the diff lflag is not new.
Re: increase netcat's buffer...
Hi. + /* listen and net in gone, queues empty, done */ + if (lflag pfd[POLL_NETIN].fd == -1 lflag ??? warning only one ref in the diff lflag is a global, the listen flag, as in: nc -l 127.0.0.1 80 I believe this is correct. Only when we listen do we want to close when the network input is gone. - Arne
Re: increase netcat's buffer...
On Thu, Jun 26, 2014 at 8:21 AM, Arne Becker arne_bec...@genua.de wrote: Hi. + /* listen and net in gone, queues empty, done */ + if (lflag pfd[POLL_NETIN].fd == -1 lflag ??? warning only one ref in the diff lflag is a global, the listen flag, as in: nc -l 127.0.0.1 80 I believe this is correct. Only when we listen do we want to close when the network input is gone. - Arne i have Zero idea if it is right or wrong, just warn because the symbol was lonely. For testing i would do something like nc - l | nc and run something like iperf3 on both end iperf client -tcp/udp nc -l | nc client -tcp/udp iperf server For a review i dislike + unsigned char stdinbuf[BUFSIZE]; and the memmove on it: Dear tech savvy, isn it better to malloc a buffer like this instead of alloca it ? or just a static buffer would be better so it is in the program mem space ? If the buffer is fixed, dont bother memmove it, just remember the begining and the end: http://en.wikipedia.org/wiki/Circular_buffer Is there a bunch of macro like TAIL_ ??? available for this ? -- - () ascii ribbon campaign - against html e-mail /\
Re: increase netcat's buffer...
On 2014/06/26 08:33, sven falempin wrote: i have Zero idea if it is right or wrong, just warn because the symbol was lonely. A diff only tells part of the story, it is also necessary to look at the surrounding code.
Re: increase netcat's buffer...
Hi. If the buffer is fixed, dont bother memmove it, just remember the begining and the end: http://en.wikipedia.org/wiki/Circular_buffer There's a tradeoff - lots of memmove vs. lots of very small reads/writes if you get near the end of the buffer. My gut feeling told me that local things, even if they require a syscall, will be a lot faster than things over the network, so I chose the version with larger reads and writes. A BIP buffer [1] would be the best solution to this problem. If such a thing already exists in the codebase, please point me to it. For this diff, I benchmarked the binary, and found no significant change in speed versus the previous implementation. I haven't really paid attention to resource usage. [1] http://www.codeproject.com/Articles/3479/The-Bip-Buffer-The-Circular-Buffer-with-a-Twist
Re: increase netcat's buffer...
On Thu, Jun 26, 2014 at 9:37 AM, Arne Becker arne_bec...@genua.de wrote: Hi. If the buffer is fixed, dont bother memmove it, just remember the begining and the end: http://en.wikipedia.org/wiki/Circular_buffer There's a tradeoff - lots of memmove vs. lots of very small reads/writes if you get near the end of the buffer. My gut feeling told me that local things, even if they require a syscall, will be a lot faster than things over the network, so I chose the version with larger reads and writes. A BIP buffer [1] would be the best solution to this problem. If such a thing already exists in the codebase, please point me to it. For this diff, I benchmarked the binary, and found no significant change in speed versus the previous implementation. I haven't really paid attention to resource usage. [1] http://www.codeproject.com/Articles/3479/The-Bip-Buffer-The-Circular-Buffer-with-a-Twist My circular buffer never made more than 2 reads or writes per read or write call, and very often only one. this BIP is just a proper implementation as far as i am concerned. does'nt change you are on the stack with the buffer, and this is not good (right?) Only system devs may explain this accurately and i do not want to make wrong statement anyway. Moreover there 's architecture i have never used that may behave differently. -- - () ascii ribbon campaign - against html e-mail /\
Re: increase netcat's buffer...
On Thu, Jun 26, 2014 at 08:33, sven falempin wrote: For a review i dislike + unsigned char stdinbuf[BUFSIZE]; and the memmove on it: Dear tech savvy, isn it better to malloc a buffer like this instead of alloca it ? or just a static buffer would be better so it is in the program mem space ? No, there's nothing wrong with a stack buffer of moderate size in a program like this. We don't use alloca() much though.
Re: increase netcat's buffer...
On Thu, Jun 26, 2014 at 10:09 AM, Ted Unangst t...@tedunangst.com wrote: On Thu, Jun 26, 2014 at 08:33, sven falempin wrote: For a review i dislike + unsigned char stdinbuf[BUFSIZE]; and the memmove on it: Dear tech savvy, isn it better to malloc a buffer like this instead of alloca it ? or just a static buffer would be better so it is in the program mem space ? No, there's nothing wrong with a stack buffer of moderate size in a program like this. We don't use alloca() much though. Ted, What is a 'good maximum' size for a buffer like this in amd64 world and arm ? From which amount should we consider to not use the stack. (I know it depends the type of function, but lets focus on this case ) Cheers. -- - () ascii ribbon campaign - against html e-mail /\
Re: increase netcat's buffer...
On Thu, Jun 26, 2014 at 11:12, sven falempin wrote: What is a 'good maximum' size for a buffer like this in amd64 world and arm ? From which amount should we consider to not use the stack. (I know it depends the type of function, but lets focus on this case ) Stack rlimit is 4096K, so there shouldn't be anything wrong with using some number up to a few hundred K. After about a megabyte I'd probably start asking questions, but only because a fixed size buffer that large is unusual, regardless of where it's been allocated.
boot: off-by-one
Hi, in boot, we have an off-by-one error in readline. When the user ends input with enter, the string will be ended twice, like: p[1] = *p = '\0'; Comparing readline with read_conf (reading commands from file), there is no need to double end the input line. Tobias Index: cmd.c === RCS file: /cvs/src/sys/stand/boot/cmd.c,v retrieving revision 1.61 diff -u -p -r1.61 cmd.c --- cmd.c 23 Dec 2013 23:32:40 - 1.61 +++ cmd.c 26 Jun 2014 18:55:02 - @@ -283,7 +283,7 @@ readline(char *buf, size_t n, int to) continue; case '\n': case '\r': - p[1] = *p = '\0'; + *p = '\0'; break; case '\b': case '\177':
Unnecessary mmap flags?
I just reviewed our mmap(2) flags to compare them against Linux, FreeBSD, Solaris, and Darwin's flags. Of the flags listed below, none of them are specified by POSIX, and none of them do anything interesting on OpenBSD: MAP_COPY just gets rewritten to MAP_PRIVATE, and the rest are silently ignored by UVM. Linux FreeBSD Solaris Darwin MAP_COPYno YES*no YES* MAP_RENAME no YES*YES*YES* MAP_NORESERVE YES YES*YES YES* MAP_INHERIT no YES** no no MAP_NOEXTENDno no no YES* MAP_HASSEMAPHOREno YES*** no YES*** MAP_TRYFIXEDno no no no * These are defined in the OS's sys/mman.h, but are undocumented in their mmap(2) manual, and behave the same as on OpenBSD (i.e., MAP_COPY is an alias for MAP_PRIVATE; the others have no effect). ** MAP_INHERIT is documented in FreeBSD's mmap(2) manual (as This flag never operated as advertised, which is true on OpenBSD too), but not defined in their sys/mman.h. *** MAP_HASSEMAPHORE is documented in FreeBSD and Darwin's mmap(2) manuals and defined in their sys/mman.h, but has no effects on either's VM behavior. So MAP_NORESERVE is perhaps necessary/worth keeping around, but the others seem like candidates for removal if nothing in ports needs them. MAP_HASSEMAPHORE is used in rthread_sem.c, but it doesn't do anything, so I suspect it's just cargo culting based on man page misinformation? Are there architectures that actually have restrictions on semaphore memory?
Re: ANONCVS MIRROR MAINTAINERS.. YOU NEED TO READ THIS!
On Thu, Jun 26, 2014 at 10:19:09AM +0100, Stuart Henderson wrote: On 2014/06/26 11:02, Mike Belopuhov wrote: On 26 June 2014 08:53, patrick keshishian sids...@boxsoft.com wrote: On Wed, Jun 25, 2014 at 10:01:06PM -0700, patrick keshishian wrote: On Thu, Jun 26, 2014 at 06:37:00AM +0200, Alexander Hall wrote: On 06/25/14 20:52, Bob Beck wrote: If you or someone you love runs an anoncvs server, they need to see this. As you know we recently added commitid support to cvs, and we had you update your cvsync binary. Unfortunately, the fix wasn't quite right. We ran into problems with the synching of commitid files. naddy managed to cook a correct fix for us. anoncvs.ca (the fanout machine) has been fixed - again. What you need to do is to update your cvsync to the latest one that was just committed into ports (cvsync-0.25.0pre0 or later). Remove your scanfile (if any). Once you do that, re-run cvsync and you should be back in business. Is it known whether this requires a fixed upstream cvsync first, or if it is ok to apply the fix and your repo will become fine whenever the upstream is fixed? Anyone with insight, please feel free to chime in. The commit message suggests, at least, the server must be updated: | Add full support for commitid and bump protocol version | Old clients will receive updates with commitid stripped out. It seems, running updated cvsync against an un-updated cvsyncd (I assume) results in failure: Connecting to anoncvs1.usa.openbsd.org port Connected to 149.20.54.217 port Running... Updating (collection openbsd-src/rcs) Updater(RCS): UPDATE(delta): error Updater(RCS): UPDATE: /cvs/anoncvs/cvs/obsd/src/lib/libc/string/timingsafe_bcmp.3,v Updater: RCS Error Socket Error: recv: 2 residue 2 Receiver Error: recv Mux(SEND) Error: not running: 0 DirScan: RCS Error Mux(SEND) Error: not running: 1 FileScan(RCS): UPDATE /cvs/anoncvs/cvs/obsd/src/sbin/pfctl/parse.y,v FileScan: RCS Error Failed removing offending files locally and restarting cvsync helps. I suspect this may only help until they receive another commit. I was thinking it would be bit of a pain to keep running cvsync, have it fail, remove files, run it again, repeat. I stopped after three cycles. The server *must* be updated - though it looks like there may still be some problems. Run with -v option per Stuart Henderson's comment. Parsing a file /etc/obsd-cvsync... Connecting to anoncvs1.usa.openbsd.org port Connected to 149.20.54.217 port Protocol: 0.25 Hash: MD5 Exchanging collection list... collection name openbsd-src release rcs umask 002 collection name openbsd-xenocara release rcs umask 002 collection name openbsd-ports release rcs umask 002 Compression: zlib Trying to establish the multiplexed channel... Running... Updating (collection openbsd-src/rcs) Updater(RCS): UPDATE(delta): error Updater(RCS): UPDATE: /cvs/anoncvs/cvs/obsd/src/lib/libcrypto/crypto/getentropy_linux.c,v Updater: RCS Error Socket Error: recv: 1691 residue 334 Receiver(DATA) Error: recv Mux(SEND) Error: not running: 1 FileScan(RCS): UPDATE /cvs/anoncvs/cvs/obsd/src/usr.sbin/installboot/i386_installboot.h,v FileScan: RCS Error Mux(SEND) Error: not running: 0 DirScan: RCS Error Failed I suspect this is because 'there still are some problems'. Is there anther way to update one's source trees for the time being? Best, --patrick
Re: ANONCVS MIRROR MAINTAINERS.. YOU NEED TO READ THIS!
As everybody noticed, there was another problem. Please update to cvsync-0.25.0pre0p0 for the latest bug fix. Sorry for all the inconvenience. -- Christian naddy Weisgerber na...@mips.inka.de
Re: ANONCVS MIRROR MAINTAINERS.. YOU NEED TO READ THIS!
On 2014/06/26 20:20, Christian Weisgerber wrote: As everybody noticed, there was another problem. Please update to cvsync-0.25.0pre0p0 for the latest bug fix. Sorry for all the inconvenience. At least the following anoncvs mirrors have this as of now: anon...@anoncvs.spacehopper.org:/cvs anon...@anoncvs.usa.openbsd.org:/cvs anon...@anoncvs.ca.openbsd.org:/cvs anon...@anoncvs.cs.toronto.edu:/cvs
boot: overflow in bootarg on truncation
Hi,
this is of interest for amd64 and i386 only:
The bootarg code is a code collection to manage a bootarg_list, which
can be assembled into contiguous code with makebootargs().
makebootargs takes two arguments: pointer and available space. Upon
return, the available space variable will be overriden with actually
consumed space.
consumed space = previously added bootargs + size for terminating bootarg
of type BOOTARG_END.
If there are too many bootargs, they will be truncated. If DEBUG has
been defined, a warning will give information about this situation.
Unfortunately the code allows an overrun upon truncation. If
truncation occurs, the for-loop of available bootargs will add as
many bootargs as possible _ignoring the additional BOOTARG_END_, which
will be added at the end.
Therefore the fix is simple: consider space for additional bootarg.
Additionally to this, bootarg_list can be static again, that was
changed back in OpenBSD 2.3 times. Also, return the actually consumed
space if truncation occurred, not simply available space.
I wrote an example code + Makefile that triggers this overflow in userland:
http://stoeckmann.org/openbsd/bootarg.tar.gz
Two programs get created from main.c:
- crash adds a bootarg in perfect size to let BOOTARG_END overflow
into guard page on i386
- nocrash adds the largest allowed bootarg -- not crashing.
Tobias
Index: bootarg.c
===
RCS file: /cvs/src/sys/stand/boot/bootarg.c,v
retrieving revision 1.10
diff -u -p -u -p -r1.10 bootarg.c
--- bootarg.c 11 Aug 2003 06:23:07 - 1.10
+++ bootarg.c 26 Jun 2014 21:22:18 -
@@ -30,7 +30,7 @@
#include lib/libsa/stand.h
#include stand/boot/bootarg.h
-bootarg_t *bootarg_list;
+static bootarg_t *bootarg_list;
void
addbootarg(int t, size_t l, void *p)
@@ -53,18 +53,20 @@ makebootargs(caddr_t v, size_t *lenp)
/* get total size */
l = sizeof(*p);
- for (p = bootarg_list; p != NULL; p = p-ba_next)
+ for (p = bootarg_list; p != NULL; p = p-ba_next) {
l += p-ba_size;
- if (*lenp l) {
+ if (*lenp l) {
#ifdef DEBUG
- printf(makebootargs: too many args\n);
+ printf(makebootargs: too many args\n);
#endif
- l = *lenp;
+ l -= p-ba_size;
+ break;
+ }
}
*lenp = l;
/* copy them out */
for (p = bootarg_list, q = v;
-p != NULL ((q + p-ba_size) - (u_char *)v) l;
+p != NULL ((q + p-ba_size) - (u_char *)v) = l - sizeof(*p);
q += p-ba_size, p = p-ba_next) {
#ifdef DEBUG
printf(%d,%d , p-ba_type, p-ba_size);
Re: Unnecessary mmap flags?
On 06/26/2014 03:28 PM, Matthew Dempsky wrote: I just reviewed our mmap(2) flags to compare them against Linux, FreeBSD, Solaris, and Darwin's flags. Of the flags listed below, none of them are specified by POSIX, and none of them do anything interesting on OpenBSD: MAP_COPY just gets rewritten to MAP_PRIVATE, and the rest are silently ignored by UVM. Linux FreeBSD Solaris Darwin MAP_COPYno YES*no YES* MAP_RENAME no YES*YES*YES* MAP_NORESERVE YES YES*YES YES* MAP_INHERIT no YES** no no MAP_NOEXTENDno no no YES* MAP_HASSEMAPHOREno YES*** no YES*** MAP_TRYFIXEDno no no no * These are defined in the OS's sys/mman.h, but are undocumented in their mmap(2) manual, and behave the same as on OpenBSD (i.e., MAP_COPY is an alias for MAP_PRIVATE; the others have no effect). ** MAP_INHERIT is documented in FreeBSD's mmap(2) manual (as This flag never operated as advertised, which is true on OpenBSD too), but not defined in their sys/mman.h. *** MAP_HASSEMAPHORE is documented in FreeBSD and Darwin's mmap(2) manuals and defined in their sys/mman.h, but has no effects on either's VM behavior. So MAP_NORESERVE is perhaps necessary/worth keeping around, but the others seem like candidates for removal if nothing in ports needs them. MAP_HASSEMAPHORE is used in rthread_sem.c, but it doesn't do anything, so I suspect it's just cargo culting based on man page misinformation? Are there architectures that actually have restrictions on semaphore memory? Sun allowed requests for shared memory to be uncached. This was used for DMA and (IIRC) interprocessor communication since adding sufficient memory barriers was painful. I don't remember how to make that request. Geoff Steckel
Re: Unnecessary mmap flags?
On Thu, Jun 26, 2014 at 12:28:18PM -0700, Matthew Dempsky wrote: I just reviewed our mmap(2) flags to compare them against Linux, FreeBSD, Solaris, and Darwin's flags. Of the flags listed below, none of them are specified by POSIX, and none of them do anything interesting on OpenBSD: MAP_COPY just gets rewritten to MAP_PRIVATE, and the rest are silently ignored by UVM. Feedback so far is the useless flags should go away. Diff below is a first step towards this: 1. MAP_COPY is redefined as an alias for MAP_PRIVATE, and the other useless MAP_* flags are redefined to 0. They're also hidden from the kernel to make sure no kernel code accidentally depends on them still. 2. Adds COMPAT_O55_MAP_COPY so we can stay binary compatible with any OpenBSD 5.5 programs that still use MAP_COPY (probably none, but it's trivial to do), and COMPAT_O55_MAP_NOOPMASK just to keep track of which bits were previously reserved for do-nothing flags. 3. Reshuffles the defines a little bit so the order makes more sense. Followup patch will add a deprecation warning for the MAP_* flags, but I think that'll need some ports testing, whereas this should be safe to commit now. ok? Index: sys/mman.h === RCS file: /home/matthew/cvs-mirror/cvs/src/sys/sys/mman.h,v retrieving revision 1.24 diff -u -p -r1.24 mman.h --- sys/mman.h 13 Jun 2014 01:48:52 - 1.24 +++ sys/mman.h 26 Jun 2014 23:54:28 - @@ -50,32 +50,43 @@ */ #defineMAP_SHARED 0x0001 /* share changes */ #defineMAP_PRIVATE 0x0002 /* changes are private */ -#defineMAP_COPY0x0004 /* copy region at mmap time */ + +/* + * Mapping type + */ +#defineMAP_FILE0x /* map from file (default) */ +#defineMAP_ANON0x1000 /* allocated from memory, swap space */ /* * Other flags */ -#defineMAP_FIXED0x0010 /* map addr must be exactly as requested */ -#defineMAP_RENAME 0x0020 /* Sun: rename private pages to file */ -#defineMAP_NORESERVE0x0040 /* Sun: don't reserve needed swap area */ -#defineMAP_INHERIT 0x0080 /* region is retained after exec */ -#defineMAP_NOEXTEND 0x0100 /* for MAP_FILE, don't change file size */ -#defineMAP_HASSEMAPHORE 0x0200 /* region may contain semaphores */ -#defineMAP_TRYFIXED 0x0400 /* attempt hint address, even within heap */ +#defineMAP_FIXED 0x0010 /* map addr must be exactly as requested */ +#define__MAP_NOREPLACE 0x0800 /* fail if address not available */ -#define__MAP_NOREPLACE 0x0800 /* fail if address not available */ +#ifdef _KERNEL +#define COMPAT_O55_MAP_COPY0x0004 /* alias for MAP_PRIVATE */ +#define COMPAT_O55_MAP_NOOPMASK0x07e0 /* formerly reserved flag bits */ +#endif + +#defineMAP_FLAGMASK0x1ff7 +#ifndef _KERNEL /* - * Error return from mmap() + * Deprecated flags with no significant meaning on OpenBSD. */ -#define MAP_FAILED ((void *)-1) +#defineMAP_COPYMAP_PRIVATE +#defineMAP_HASSEMAPHORE0 +#defineMAP_INHERIT 0 +#defineMAP_NOEXTEND0 +#defineMAP_NORESERVE 0 +#defineMAP_RENAME 0 +#defineMAP_TRYFIXED0 +#endif /* - * Mapping type + * Error return from mmap() */ -#defineMAP_FILE0x /* map from file (default) */ -#defineMAP_ANON0x1000 /* allocated from memory, swap space */ -#defineMAP_FLAGMASK0x1ff7 +#define MAP_FAILED ((void *)-1) /* * POSIX memory advisory values. Index: uvm/uvm_mmap.c === RCS file: /home/matthew/cvs-mirror/cvs/src/sys/uvm/uvm_mmap.c,v retrieving revision 1.94 diff -u -p -r1.94 uvm_mmap.c --- uvm/uvm_mmap.c 13 Apr 2014 23:14:15 - 1.94 +++ uvm/uvm_mmap.c 26 Jun 2014 23:49:39 - @@ -345,8 +345,8 @@ sys_mmap(struct proc *p, void *v, regist return (EINVAL); if ((flags MAP_FLAGMASK) != flags) return (EINVAL); - if (flags MAP_COPY) - flags = (flags ~MAP_COPY) | MAP_PRIVATE; + if (flags COMPAT_O55_MAP_COPY) + flags = (flags ~COMPAT_O55_MAP_COPY) | MAP_PRIVATE; if ((flags (MAP_SHARED|MAP_PRIVATE)) == (MAP_SHARED|MAP_PRIVATE)) return (EINVAL); if ((flags (MAP_FIXED|__MAP_NOREPLACE)) == __MAP_NOREPLACE)
Re: ANONCVS MIRROR MAINTAINERS.. YOU NEED TO READ THIS!
On 06/26/14 17:18, Stuart Henderson wrote: On 2014/06/26 20:20, Christian Weisgerber wrote: As everybody noticed, there was another problem. Please update to cvsync-0.25.0pre0p0 for the latest bug fix. Sorry for all the inconvenience. At least the following anoncvs mirrors have this as of now: anon...@anoncvs.spacehopper.org:/cvs anon...@anoncvs.usa.openbsd.org:/cvs anon...@anoncvs.ca.openbsd.org:/cvs anon...@anoncvs.cs.toronto.edu:/cvs anon...@openbsd.cs.toronto.edu:/cvs is updated now, too, and looking MUCH happier. Thanks, Naddy and Stuart and Bob! Nick.
Re: Unnecessary mmap flags?
1. MAP_COPY is redefined as an alias for MAP_PRIVATE, and the other useless MAP_* flags are redefined to 0. They're also hidden from the kernel to make sure no kernel code accidentally depends on them still. 2. Adds COMPAT_O55_MAP_COPY so we can stay binary compatible with any OpenBSD 5.5 programs that still use MAP_COPY (probably none, but it's trivial to do), and COMPAT_O55_MAP_NOOPMASK just to keep track of which bits were previously reserved for do-nothing flags. Yes, then we can remove the kernel support soon. We need to remember to do that.. sometimes such things live far longer than the ABI requires. I suspect the new name might hurt kdump support (grep for mmapflagsname).
simple fstab.5 diff
fix missing forward-slash. Index: fstab.5 === RCS file: /cvs/obsd/src/share/man/man5/fstab.5,v retrieving revision 1.48 diff -u -p -u -p -r1.48 fstab.5 --- fstab.5 6 Jan 2014 00:52:21 - 1.48 +++ fstab.5 27 Jun 2014 05:52:08 - @@ -248,7 +248,7 @@ a value of zero is returned and will assume that the filesystem does not need to be checked. .Bd -literal #defineFSTAB_RWrw/* read/write device */ -#defineFSTAB_RQrq/* read/write with quotas * +#defineFSTAB_RQrq/* read/write with quotas */ #defineFSTAB_ROro/* read-only device */ #defineFSTAB_SWsw/* swap device */ #defineFSTAB_XXxx/* ignore totally */
