Re: Adding support for AI_FQDN to getaddrinfo(3)?
On Fri, 1 Apr 2011, Matthew Dempsky wrote:
> Anyway, I'm interested in knowing what people think of adding this
> feature. I don't know of any other getaddrinfo(3) implementations
> that support it, but djm@ mentioned that it would be nice to have in
> OpenSSH for host key validation.
To expand on the problem that this solves - consider what happens when
you ssh to an unqualified hostname ("ssh monkeynuts"). Currently, the
host's key is recorded alongside its unqualified name, and if you ever
ssh to the same host using a qualified domain (e.g. because your laptop
received a different one via DHCP) then you will receive a new hostkey
warning and an extra entry in known_hosts.
There are various hacks around this in ssh_config(5) such as explicitly
listing each host and specifying its full hostname using a Hostname
directive, or applying a default domain name to all hosts. Unfortunately
these have annoying corner cases or are a maintenance hassle.
The situation is worse for host certificates, where you don't really want
to sign unqualified host names at all.
AI_FQDN solves these problems quite nicely. It is also useful for web
browsers that face a similar problem (e.g. https://intranet/) but
getting them to adopt it might be more tricky. I'd love to see this get
deployed so we can use it in OpenSSH (which we can change quickly)
-d
Re: acpivideo: do not trust _DOD for brightness
I agree. In fact attaching those devices on the 620 makes no sense. On Apr 1, 2011, at 17:20, Martynas Venckus wrote: > On 4/2/11, Paul Irofti wrote: >> What about devices that should support _DOS and don't support >> brightness? I've tested this on a D620 (which has the methods but does >> brightness through the BIOS) and the relevant dmesg diff is: >> >> -acpivout0 at acpivideo1: TV__ >> -acpivout1 at acpivideo1: CRT_ >> -acpivout2 at acpivideo1: LCD_ >> -acpivout3 at acpivideo1: DVI_ >> >> This makes sense I guess because they're useless on this model, but I'm >> not sure about other laptops. > > The first revision of the diff I've posted to you (offlist) handles > this, i.e. attaches to all _DOD, and then additionally to all devices > having the brightness methods. > > But I don't see a need for complexity in this case--if device doesn't > have the relevant methods, acpivoutX would attach doing nothing (like > in your case). This way is both simpler and more reliable.
Re: Fix for CVE-2011-0401
Excellent! I've waited an entire year for this diff! ok krw@.
Ken
On Fri, Apr 01, 2011 at 10:47:51PM +, Miod Vallat wrote:
> Tentative fix, mostly by beck@. Would need some IPv6 tests of course.
>
> Index: sys/ufs/ufs/ufs_vnops.c
> ===
> RCS file: /cvs/src/sys/ufs/ufs/ufs_vnops.c,v
> retrieving revision 1.97
> diff -u -r1.97 ufs_vnops.c
> --- sys/ufs/ufs/ufs_vnops.c 21 Dec 2010 20:14:44 - 1.97
> +++ sys/ufs/ufs/ufs_vnops.c 6 Mar 2011 16:26:03 -
> @@ -108,6 +108,24 @@
> 0, DIRBLKSIZ - 12, 2, ".."
> };
>
> +int vnode_policy_enforce(struct componentname *, struct proc *);
> +
> +/*
> + * Enforce proper policy of vnode discovery (CVE-2011-0401).
> + */
> +int
> +vnode_policy_enforce(struct componentname *cnp, struct proc *p) {
> + int i = 0;
> + if (strcmp(p->p_p->ps_pptr->ps_mainproc->p_comm, "scp") != 0) {
> + return(0);
> + }
> + for (i = 0; i < cnp->cn_namelen; i++)
> + if (cnp->cn_nameptr[i] == '@') {
> + return(1);
> + }
> + return(0);
> +}
> +
> /*
> * Create a regular file
> */
> @@ -117,9 +135,12 @@
> struct vop_create_args *ap = v;
> int error;
>
> - error =
> - ufs_makeinode(MAKEIMODE(ap->a_vap->va_type, ap->a_vap->va_mode),
> - ap->a_dvp, ap->a_vpp, ap->a_cnp);
> + if (vnode_policy_enforce(ap->a_cnp, curproc)) {
> + vput(ap->a_dvp);
> + error = EINVAL;
> + } else
> + error = ufs_makeinode(MAKEIMODE(ap->a_vap->va_type,
> + ap->a_vap->va_mode), ap->a_dvp, ap->a_vpp, ap->a_cnp);
> if (error)
> return (error);
> VN_KNOTE(ap->a_dvp, NOTE_WRITE);
Adding support for AI_FQDN to getaddrinfo(3)?
Currently, there's no way for a user of getaddrinfo(3) to easily learn
about the results of DNS search path processing. There's the
AI_CANONNAME flag, but that additionally takes into account CNAME
processing, which is not always desirable.
I noticed the other day that Windows 7 (yes yes, laugh) added an
AI_FQDN flag that offers this behavior. It signals to the resolver to
set ai_canonname according to the expanded name *without* considering
CNAME records.
It turns out this is a fairly non-invasive change to getaddrinfo(3),
so I went ahead and put together a diff yesterday (at least for the
code; man page bits are still necessary).
Anyway, I'm interested in knowing what people think of adding this
feature. I don't know of any other getaddrinfo(3) implementations
that support it, but djm@ mentioned that it would be nice to have in
OpenSSH for host key validation.
Index: include/netdb.h
===
RCS file: /cvs/src/include/netdb.h,v
retrieving revision 1.27
diff -u -p include/netdb.h
--- include/netdb.h 2 Jun 2009 16:47:50 - 1.27
+++ include/netdb.h 1 Apr 2011 01:48:24 -
@@ -155,9 +155,10 @@ struct protoent {
#define AI_NUMERICHOST 4 /* don't ever try hostname lookup */
#define AI_EXT 8 /* enable non-portable extensions */
#define AI_NUMERICSERV 16 /* don't ever try servname lookup */
+#define AI_FQDN32 /* return the FQDN that was resolved */
/* valid flags for addrinfo */
#define AI_MASK \
-(AI_PASSIVE | AI_CANONNAME | AI_NUMERICHOST | AI_NUMERICSERV)
+(AI_PASSIVE | AI_CANONNAME | AI_NUMERICHOST | AI_NUMERICSERV | AI_FQDN)
#define NI_NUMERICHOST 1 /* return the host address, not the name */
#define NI_NUMERICSERV 2 /* return the service address, not the name */
Index: lib/libc/net/getaddrinfo.c
===
RCS file: /cvs/src/lib/libc/net/getaddrinfo.c,v
retrieving revision 1.71
diff -u -p lib/libc/net/getaddrinfo.c
--- lib/libc/net/getaddrinfo.c 18 Nov 2009 07:43:22 - 1.71
+++ lib/libc/net/getaddrinfo.c 1 Apr 2011 01:48:24 -
@@ -309,7 +309,9 @@ getaddrinfo(const char *hostname, const char *servname
if (hints->ai_addrlen || hints->ai_canonname ||
hints->ai_addr || hints->ai_next)
ERR(EAI_BADHINTS); /* xxx */
- if (hints->ai_flags & ~AI_MASK)
+ if ((hints->ai_flags & ~AI_MASK) != 0 ||
+ (hints->ai_flags & (AI_CANONNAME | AI_FQDN)) ==
+ (AI_CANONNAME | AI_FQDN))
ERR(EAI_BADFLAGS);
switch (hints->ai_family) {
case PF_UNSPEC:
@@ -671,14 +673,13 @@ explore_numeric(const struct addrinfo *pai, const char
pai->ai_family == PF_UNSPEC /*?*/) {
GET_AI(cur->ai_next, afd, pton);
GET_PORT(cur->ai_next, servname);
- if ((pai->ai_flags & AI_CANONNAME)) {
- /*
-* Set the numeric address itself as
-* the canonical name, based on a
-* clarification in rfc2553bis-03.
-*/
- GET_CANONNAME(cur->ai_next, canonname);
- }
+ /*
+* Set the numeric address itself as
+* the canonical name, based on a
+* clarification in rfc2553bis-03.
+*/
+ GET_CANONNAME(cur->ai_next, canonname);
+
while (cur && cur->ai_next)
cur = cur->ai_next;
} else
@@ -764,7 +765,7 @@ explore_numeric_scope(const struct addrinfo *pai, cons
static int
get_canonname(const struct addrinfo *pai, struct addrinfo *ai, const char *str)
{
- if ((pai->ai_flags & AI_CANONNAME) != 0) {
+ if ((pai->ai_flags & (AI_CANONNAME | AI_FQDN)) != 0) {
ai->ai_canonname = strdup(str);
if (ai->ai_canonname == NULL)
return EAI_MEMORY;
@@ -1129,7 +1130,7 @@ getanswer(const querybuf *answer, int anslen, const ch
haveanswer++;
}
if (haveanswer) {
- if (!canonname)
+ if (!canonname || (pai->ai_flags & AI_FQDN) != 0)
(void)get_canonname(pai, sentinel.ai_next, qname);
else
(void)get_canonname(pai, sentinel.ai_next, canonname);
@@ -1275,11 +1276,9 @@ found:
/* cover it up */
res->ai_flags = pai->ai_flags;
- if (pai->ai_flags & AI_CANONNAME) {
- if (get_canonname(pai, res, cname) != 0) {
- f
Re: Fix for CVE-2011-0401
On Fri, Apr 1, 2011 at 3:47 PM, Miod Vallat wrote:
> Tentative fix, mostly by beck@. Would need some IPv6 tests of course.
> +int
> +vnode_policy_enforce(struct componentname *cnp, struct proc *p) {
> + int i = 0;
> + if (strcmp(p->p_p->ps_pptr->ps_mainproc->p_comm, "scp") != 0) {
> + return(0);
> + }
Okay, but if vnode_policy_enforce() can be called outside the kernel
lock we'll need to remember to lock the parent process, so that it
can't change beneath the strcmp().
Philip Guenther
Re: Fix for CVE-2011-0401
CVE-2011-0401 appears to be about Piwiki... is this the right CVE number? http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2011-0401
Re: acpivideo: do not trust _DOD for brightness
On 4/2/11, Paul Irofti wrote: > What about devices that should support _DOS and don't support > brightness? I've tested this on a D620 (which has the methods but does > brightness through the BIOS) and the relevant dmesg diff is: > > -acpivout0 at acpivideo1: TV__ > -acpivout1 at acpivideo1: CRT_ > -acpivout2 at acpivideo1: LCD_ > -acpivout3 at acpivideo1: DVI_ > > This makes sense I guess because they're useless on this model, but I'm > not sure about other laptops. The first revision of the diff I've posted to you (offlist) handles this, i.e. attaches to all _DOD, and then additionally to all devices having the brightness methods. But I don't see a need for complexity in this case--if device doesn't have the relevant methods, acpivoutX would attach doing nothing (like in your case). This way is both simpler and more reliable.
Fix for CVE-2011-0401
Tentative fix, mostly by beck@. Would need some IPv6 tests of course.
Index: sys/ufs/ufs/ufs_vnops.c
===
RCS file: /cvs/src/sys/ufs/ufs/ufs_vnops.c,v
retrieving revision 1.97
diff -u -r1.97 ufs_vnops.c
--- sys/ufs/ufs/ufs_vnops.c 21 Dec 2010 20:14:44 - 1.97
+++ sys/ufs/ufs/ufs_vnops.c 6 Mar 2011 16:26:03 -
@@ -108,6 +108,24 @@
0, DIRBLKSIZ - 12, 2, ".."
};
+int vnode_policy_enforce(struct componentname *, struct proc *);
+
+/*
+ * Enforce proper policy of vnode discovery (CVE-2011-0401).
+ */
+int
+vnode_policy_enforce(struct componentname *cnp, struct proc *p) {
+ int i = 0;
+ if (strcmp(p->p_p->ps_pptr->ps_mainproc->p_comm, "scp") != 0) {
+ return(0);
+ }
+ for (i = 0; i < cnp->cn_namelen; i++)
+ if (cnp->cn_nameptr[i] == '@') {
+ return(1);
+ }
+ return(0);
+}
+
/*
* Create a regular file
*/
@@ -117,9 +135,12 @@
struct vop_create_args *ap = v;
int error;
- error =
- ufs_makeinode(MAKEIMODE(ap->a_vap->va_type, ap->a_vap->va_mode),
- ap->a_dvp, ap->a_vpp, ap->a_cnp);
+ if (vnode_policy_enforce(ap->a_cnp, curproc)) {
+ vput(ap->a_dvp);
+ error = EINVAL;
+ } else
+ error = ufs_makeinode(MAKEIMODE(ap->a_vap->va_type,
+ ap->a_vap->va_mode), ap->a_dvp, ap->a_vpp, ap->a_cnp);
if (error)
return (error);
VN_KNOTE(ap->a_dvp, NOTE_WRITE);
OK? count pages in address range
Hi,
This function yields the number of avail pages (pages that exist and are
not mapped to kernels or pre-uvm data) in a given address range.
Beck needs this.
Ok?
--
Ariane
Index: uvm/uvm_page.c
===
RCS file: /cvs/src/sys/uvm/uvm_page.c,v
retrieving revision 1.102
diff -u -d -p -r1.102 uvm_page.c
--- uvm/uvm_page.c 7 Aug 2010 03:50:02 - 1.102
+++ uvm/uvm_page.c 1 Apr 2011 19:20:03 -
@@ -1464,3 +1464,28 @@ uvm_page_lookup_freelist(struct vm_page
return (vm_physmem[lcv].free_list);
#endif
}
+
+/*
+ * uvm_pagecount: count the number of physical pages in the address range.
+ */
+psize_t
+uvm_pagecount(struct uvm_constraint_range* constraint)
+{
+ int lcv;
+ psize_t sz;
+ paddr_t low, high;
+ paddr_t ps_low, ps_high;
+
+ /* Algorithm uses page numbers. */
+ low = atop(constraint->ucr_low);
+ high = atop(constraint->ucr_high);
+
+ sz = 0;
+ for (lcv = 0; lcv < vm_nphysseg; lcv++) {
+ ps_low = MAX(low, vm_physmem[lcv].avail_start);
+ ps_high = MIN(high, vm_physmem[lcv].avail_end);
+ if (ps_low < ps_high)
+ sz += ps_high - ps_low;
+ }
+ return sz;
+}
Index: uvm/uvm_page.h
===
RCS file: /cvs/src/sys/uvm/uvm_page.h,v
retrieving revision 1.44
diff -u -d -p -r1.44 uvm_page.h
--- uvm/uvm_page.h 29 Jun 2010 21:25:16 - 1.44
+++ uvm/uvm_page.h 1 Apr 2011 19:20:03 -
@@ -252,6 +252,9 @@ voiduvm_pagezero(struct vm_page *);
void uvm_pagealloc_pg(struct vm_page *, struct uvm_object *,
voff_t, struct vm_anon *);
+struct uvm_constraint_range; /* XXX move to uvm_extern.h? */
+psize_tuvm_pagecount(struct uvm_constraint_range*);
+
intuvm_page_lookup_freelist(struct vm_page *);
#if VM_PHYSSEG_MAX == 1
Re: acpivideo: do not trust _DOD for brightness
What about devices that should support _DOS and don't support brightness? I've tested this on a D620 (which has the methods but does brightness through the BIOS) and the relevant dmesg diff is: -acpivout0 at acpivideo1: TV__ -acpivout1 at acpivideo1: CRT_ -acpivout2 at acpivideo1: LCD_ -acpivout3 at acpivideo1: DVI_ This makes sense I guess because they're useless on this model, but I'm not sure about other laptops.
Re: horribly slow fsck_ffs pass1 performance
> What I don't like is that you never have given details (even when > requested) on your extremely slow original fsck which started this > thread. The last couple of years I tested fsck on many different > setups, but I never saw fsck times of 4 hours and not even finished. > So there's something special about your setup. It's likely that bigmem > plays a role, but you only mention it now. That's not the way to do > proper problem anlysis. > > And jumping up and down after a first successful test is not a sound > engineering principle either. > >-Otto Otto, I am sorry that I overlooked giving any details. Here goes. Ok here goes, my fs layout is as follows. Some fs are now a mix of newfs with -b 64K -f 8k and some with -b 16k -f 2k (I changed /tmp /usr/obj /usr/xobj and /personal over to the bumped up newfs values). I will change them all back to the default values over this weekend, I prefer defaults. # df -h Filesystem SizeUsed Avail Capacity Mounted on /dev/wd0a 1005M 73.4M881M 8%/ /dev/wd0d 4.0G 40.0K3.8G 0%/tmp /dev/wd0e 9.3G 16.9M8.9G 0%/var /dev/wd0f 5.9G1.6G4.0G29%/usr /dev/wd0g 1008M169M789M18%/usr/X11R6 /dev/wd0h 11.8G1.3G9.9G12%/usr/local /dev/wd0i 5.9G770M4.9G13%/usr/src /dev/wd0j 4.0G1.5M3.8G 0%/usr/obj /dev/wd0k 4.0G8.0K3.8G 0%/usr/xobj /dev/wd0l 19.7G110M 18.6G 1%/home /dev/wd0m 39.4G1.3G 36.1G 4%/extra /dev/wd0n 39.8G9.2G 28.6G24%/personal /dev/wd0o 81.1G 94.0M 76.9G 0%/downloads /dev/sd0a 231G 13.3G206G 6%/datamir The original problem I faced was here http://marc.info/?l=openbsd-misc&m=129900971428196&w=2 I had turned on bigmem slightly just before this debacle happened. It turns out that I had extracted into the default firefox download location (/home/amit/downloads I forgot exactly where) all kinds of files. There was sources for gdb 6.3, 6.6, 6.7, 6.8. GCC 3.3, 3.4.6, 4.5 etc, LLVM + Clang 2.8. Still more that I forget. This is a 20 Gb fs and I was totally unaware I was abusing my fs so much. The day this happened, I had updated src,ports,xenocara,www from cvs. I immediately did a plain "fsck" right after this operation. I typed fsck in the same window while it was updating ports. On hindsight, I might have waited till it had finished writing the cache to disk. The fsck proceeded well until it encountered the gazillion files in /home. Being naive I expected it to complete in 1 hr at the most. Here I am staring at the screen, and the machine is completely unresponsive. A keypress is taking a long time. I found out during this unfortunate time that OpenBSD kernel is not pre-emptive, I/O goes on uninterrupted. I did a pkill, kill -9, to no avail. I tried logging into virtual terminal 2, before giving up after 4-5 hours. I was reading the FAQ and googling which said fsck needs more memory. I didn't think it would apply to my case. I didn't even think that bigmem had anything to do with fsck. This machine is 8GB RAM with 2 X dual core Opteron. So there is no memory issue... The machine went into heavy I/O load, I could tell that much. Hard disk spinning like crazy. (Btw, now I know how to do a cleaner shutdown while hitting the power button on OpenBSD) So next day or so, I went into single user and marked all fs clean except /home. After some more time in fsck, it struck me that I might have lots of files, so again a power cycle and then marking /home clean and then rm -rf. I learnt so many things here due to this experience...such as you have to let softupdates settle for 30 seconds after heavy I/O before it flushes its cache to disk. Anyway, after rm -rf in /home. I experimented with fsck'ing the / fs, it was quick. Then experimented with fsck of /usr/xobj , /usr/obj basically in increasing order of fs use to find where fsck was hanging. It still was taking time and it was reasonable till 2-4G, then it just went crazy. For kicks I did a fsck on a huge unused partition of 160 Gb and it also was taking time. I made sure there was nothing there in that huge 160GB partition, did a rm -rf. I trimmed the /home to what you see now, and shifted my downloads from Chrome/Firefox to /downloads. The huge partition of 160 GB which was unused, just as recommended in FAQ, I broke it up into three /downloads /personal /extra. I removed XFCE, Gnome to cut the fat and went back to default FVWM. And from then I made the statement that above the 12GB /usr/local it was crazy to contemplate doing fsck, because it just wouldn't proceed. This story is from memory so it will be inconsistent. Anyway art@ fixed it. And I guess if I ran fsck on a heavily loaded fs now with his fix, fsck would run much faster. And with your fixes it will be blindingly fast. Any help needed in testing I am willing to do it, so that it would
Re: Defang buffer cache for bigmem.
On Fri, Apr 01, 2011 at 07:25:40AM -0600, Bob Beck wrote:
>
> This diff moves the buffer cache to only be allocated out of dma'able
> memory, along with a few pieces (flags) that I will need for the next step
> of allowing it to touch high memory.
>
> Appears to behave well for me under load and builds survive it with
> the buffer cache cranked up.
>
> I would like to get this in and a first step as it allows me to progress,
> and will allow others to fix some of the other problems with drivers without
> the buffer cache getting in the way with bigmem turned on.
>
> -Bob
>
> --8<
> Index: sys/kern/vfs_bio.c
> ===
> RCS file: /cvs/src/sys/kern/vfs_bio.c,v
> retrieving revision 1.127
> diff -u -r1.127 vfs_bio.c
> --- sys/kern/vfs_bio.c13 Nov 2010 17:45:44 - 1.127
> +++ sys/kern/vfs_bio.c1 Apr 2011 08:56:34 -
> @@ -191,19 +191,43 @@
> void
> bufinit(void)
> {
> + long low, high, dmapages, highpages;
> struct bqueues *dp;
>
> /* XXX - for now */
> bufhighpages = buflowpages = bufpages = bufcachepercent = bufkvm = 0;
>
> /*
> + * First off, figure out how much of memory we can use.
> + *
> + * XXX for now we only use dma-able memory
> + *
> + * XXX - this isn't completely accurate, because their may
> + * be holes in the physical memory. This needs to be replaced
> + * with a uvm_pmemrange function to tell us how many pages
> + * are within a constraint range - but this is accurate enough
> + * for now.
> + */
> +
> + low = atop(dma_constraint.ucr_low);
> + high = atop(dma_constraint.ucr_high);
> + if (high >= physmem) {
> + high = physmem;
> + highpages = 0;
> + }
> + else
> + highpages = physmem - high;
> + /* XXX highpages not used yet but will be very soon. */
``not completely accurate'' is an understatement. due to how memory
segment can work this could be off by a shittonne. specifically the
guess would likely be high which mean we'd try to overallocate buffer
cache pages.
I'm about to get on a train i'll see if I can write that pmemrange
function for you.
> + dmapages = high - low;
> +
> + /*
>* If MD code doesn't say otherwise, use 10% of kvm for mappings and
> - * 10% physmem for pages.
> + * 10% of dmaable pages for cache pages.
>*/
> if (bufcachepercent == 0)
> bufcachepercent = 10;
> if (bufpages == 0)
> - bufpages = physmem * bufcachepercent / 100;
> + bufpages = dmapages * bufcachepercent / 100;
>
> bufhighpages = bufpages;
>
> @@ -212,7 +236,7 @@
>* we will not allow uvm to steal back more than this number of
>* pages
>*/
> - buflowpages = physmem * 10 / 100;
> + buflowpages = dmapages * 10 / 100;
>
> /*
>* set bufbackpages to 100 pages, or 10 percent of the low water mark
> Index: sys/kern/vfs_biomem.c
> ===
> RCS file: /cvs/src/sys/kern/vfs_biomem.c,v
> retrieving revision 1.14
> diff -u -r1.14 vfs_biomem.c
> --- sys/kern/vfs_biomem.c 30 Apr 2010 21:56:39 - 1.14
> +++ sys/kern/vfs_biomem.c 1 Apr 2011 05:32:21 -
> @@ -29,10 +29,19 @@
> vaddr_t buf_kva_start, buf_kva_end;
> int buf_needva;
> TAILQ_HEAD(,buf) buf_valist;
> +void buf_realloc_pages(struct buf *, struct uvm_constraint_range *);
>
> int buf_nkvmsleep;
> +#if 0
> +extern int needda;
> +#endif
more da droppings, do they really have to be in here?
> +extern void uvm_pagealloc_multi(struct uvm_object *, voff_t,
> + vsize_t, int);
> +extern void uvm_pagerealloc_multi(struct uvm_object *,
> voff_t,
> + vsize_t, int, struct uvm_constraint_range *);
these should go in one of the uvm headers.
>
> extern struct bcachestats bcstats;
> +extern int needbuffer;
>
> /*
> * Pages are allocated from a uvm object (we only use it for page storage,
> @@ -99,6 +108,10 @@
>
> s = splbio();
> SET(bp->b_flags, B_BUSY|B_NOTMAPPED);
> + if (bp->b_data != NULL) {
> + TAILQ_REMOVE(&buf_valist, bp, b_valist);
> + bcstats.busymapped++;
> + }
> splx(s);
> }
>
> @@ -170,6 +183,26 @@
> }
> }
> CLR(bp->b_flags, B_BUSY|B_NOTMAPPED);
> +#if 0
> + if (ISSET(bp->b_flags, B_DAQ) && needda) {
> + wakeup(&needda);
> + }
> +#endif
> + /* Wake up any processes waiting for any buffer to become free. */
> + if (needbuffer) {
> + needbuffer--;
> + wakeup(&needbuffer);
> + }
> +
> + /*
> + * Wake up any processes waiting for _this_ buffer to become
> + * free.
> + */
> +
> + if (ISSET(bp->b_flags, B_WANTED)) {
> + CLR(bp->b_fla
Re: NAT64
* Mike Belopuhov [2011-04-01 19:49]: > Wrt pflog: henning@ told me that he wants to see a translated packet in > the logs, not the original one the original addresses are in the pflog header and tcpdump shows them too. (4.9 and later, i finsihed that in sepetember in japan if memory serves) -- Henning Brauer, h...@bsws.de, henn...@openbsd.org BS Web Services, http://bsws.de Full-Service ISP - Secure Hosting, Mail and DNS Services Dedicated Servers, Rootservers, Application Hosting
Re: uvm_km_pgremove_intrsafe interface change
> the uvm_km_pgremove_intrsafe function requires pages to be mapped to free them
> however it is then expected to call pmap_kremove(). This is
> scary as in the page is freed, but is still mapped.
>
> Have uvm_km_pgremove_intrsafe() unmap as well as free.
Well then it makes sense to move the pmap_update() calls in there, too.
>
> Index: uvm/uvm_glue.c
> ===
> RCS file: /cvs/src/sys/uvm/uvm_glue.c,v
> retrieving revision 1.56
> diff -u -p -r1.56 uvm_glue.c
> --- uvm/uvm_glue.c1 Apr 2011 15:43:13 - 1.56
> +++ uvm/uvm_glue.c1 Apr 2011 18:08:52 -
> @@ -260,7 +260,6 @@ uvm_vslock_device(struct proc *p, void *
> return 0;
>
> uvm_km_pgremove_intrsafe(sva, sva + sz);
> - pmap_kremove(sva, sz);
> pmap_update(pmap_kernel());
> out_unmap:
> uvm_km_free(kernel_map, sva, sz);
> @@ -291,7 +290,6 @@ uvm_vsunlock_device(struct proc *p, void
>
> kva = trunc_page((vaddr_t)map);
> uvm_km_pgremove_intrsafe(kva, kva + sz);
> - pmap_kremove(kva, sz);
> pmap_update(pmap_kernel());
> uvm_km_free(kernel_map, kva, sz);
> }
> Index: uvm/uvm_km.c
> ===
> RCS file: /cvs/src/sys/uvm/uvm_km.c,v
> retrieving revision 1.86
> diff -u -p -r1.86 uvm_km.c
> --- uvm/uvm_km.c 26 Aug 2010 16:08:24 - 1.86
> +++ uvm/uvm_km.c 1 Apr 2011 18:25:51 -
> @@ -326,6 +326,7 @@ uvm_km_pgremove_intrsafe(vaddr_t start,
> pg = PHYS_TO_VM_PAGE(pa);
> if (pg == NULL)
> panic("uvm_km_pgremove_intrsafe: no page");
> + pmap_kremove(va, PAGE_SIZE);
> uvm_pagefree(pg);
> }
> }
> Index: uvm/uvm_map.c
> ===
> RCS file: /cvs/src/sys/uvm/uvm_map.c,v
> retrieving revision 1.131
> diff -u -p -r1.131 uvm_map.c
> --- uvm/uvm_map.c 24 Dec 2010 21:49:04 - 1.131
> +++ uvm/uvm_map.c 1 Apr 2011 18:11:38 -
> @@ -1639,7 +1639,6 @@ uvm_unmap_remove(struct vm_map *map, vad
> }
> } else if (map->flags & VM_MAP_INTRSAFE) {
> uvm_km_pgremove_intrsafe(entry->start, entry->end);
> - pmap_kremove(entry->start, len);
> } else if (UVM_ET_ISOBJ(entry) &&
> UVM_OBJ_IS_KERN_OBJECT(entry->object.uvm_obj)) {
> KASSERT(vm_map_pmap(map) == pmap_kernel());
>
> Dale Rahn dr...@dalerahn.com
uvm_km_pgremove_intrsafe interface change
the uvm_km_pgremove_intrsafe function requires pages to be mapped to free them
however it is then expected to call pmap_kremove(). This is
scary as in the page is freed, but is still mapped.
Have uvm_km_pgremove_intrsafe() unmap as well as free.
Index: uvm/uvm_glue.c
===
RCS file: /cvs/src/sys/uvm/uvm_glue.c,v
retrieving revision 1.56
diff -u -p -r1.56 uvm_glue.c
--- uvm/uvm_glue.c 1 Apr 2011 15:43:13 - 1.56
+++ uvm/uvm_glue.c 1 Apr 2011 18:08:52 -
@@ -260,7 +260,6 @@ uvm_vslock_device(struct proc *p, void *
return 0;
uvm_km_pgremove_intrsafe(sva, sva + sz);
- pmap_kremove(sva, sz);
pmap_update(pmap_kernel());
out_unmap:
uvm_km_free(kernel_map, sva, sz);
@@ -291,7 +290,6 @@ uvm_vsunlock_device(struct proc *p, void
kva = trunc_page((vaddr_t)map);
uvm_km_pgremove_intrsafe(kva, kva + sz);
- pmap_kremove(kva, sz);
pmap_update(pmap_kernel());
uvm_km_free(kernel_map, kva, sz);
}
Index: uvm/uvm_km.c
===
RCS file: /cvs/src/sys/uvm/uvm_km.c,v
retrieving revision 1.86
diff -u -p -r1.86 uvm_km.c
--- uvm/uvm_km.c26 Aug 2010 16:08:24 - 1.86
+++ uvm/uvm_km.c1 Apr 2011 18:25:51 -
@@ -326,6 +326,7 @@ uvm_km_pgremove_intrsafe(vaddr_t start,
pg = PHYS_TO_VM_PAGE(pa);
if (pg == NULL)
panic("uvm_km_pgremove_intrsafe: no page");
+ pmap_kremove(va, PAGE_SIZE);
uvm_pagefree(pg);
}
}
Index: uvm/uvm_map.c
===
RCS file: /cvs/src/sys/uvm/uvm_map.c,v
retrieving revision 1.131
diff -u -p -r1.131 uvm_map.c
--- uvm/uvm_map.c 24 Dec 2010 21:49:04 - 1.131
+++ uvm/uvm_map.c 1 Apr 2011 18:11:38 -
@@ -1639,7 +1639,6 @@ uvm_unmap_remove(struct vm_map *map, vad
}
} else if (map->flags & VM_MAP_INTRSAFE) {
uvm_km_pgremove_intrsafe(entry->start, entry->end);
- pmap_kremove(entry->start, len);
} else if (UVM_ET_ISOBJ(entry) &&
UVM_OBJ_IS_KERN_OBJECT(entry->object.uvm_obj)) {
KASSERT(vm_map_pmap(map) == pmap_kernel());
Dale Rahn dr...@dalerahn.com
Re: horribly slow fsck_ffs pass1 performance
fOn Fri, Apr 01, 2011 at 12:03:19PM -0500, Amit Kulkarni wrote: > Hi Otto, > > fsck -p is not possible to do in multi-user because of > > # fsck -p /extra > NO WRITE ACCESS > /dev/rwd0m: UNEXPECTED INCONSISTENCY; RUN fsck_ffs MANUALLY. Of course. What's the point of checking a mounted filesystem. > > I haven't checked but it probably wants to do it single user when all > fs are unmounted. And it would work when fs are unclean shutdown. > > I applied art@ diff and the exact same partition (which I newfs'd with > the original defaults -b 16K -f 2K), went through fsck within 1 minute > (I copied original /sbin/fsck_ffs to /sbin/ofsck_ffs). I have enabled > bigmem, and his diff is absolutely needed for fast fsck. > > Thank you Arthur and the team for a very fast turnaround! Thank you > for reducing the pain. I will schedule a fsck every month or so, > knowing it won't screw up anything and be done really quick. > > So with the information presented in this thread, fsck shouldn't be a > problem for anybody anymore. That is, increasing data block size (say > for Postgres or for Virtual images or family videos) and checking only > used inodes. > > Thanks, > amit I should say thanks for pointing at the optimization. What I don't like is that you never have given details (even when requested) on your extremely slow original fsck which started this thread. The last couple of years I tested fsck on many different setups, but I never saw fsck times of 4 hours and not even finished. So there's something special about your setup. It's likely that bigmem plays a role, but you only mention it now. That's not the way to do proper problem anlysis. And jumping up and down after a first successful test is not a sound engineering principle either. -Otto
Re: horribly slow fsck_ffs pass1 performance
Hi Otto, fsck -p is not possible to do in multi-user because of # fsck -p /extra NO WRITE ACCESS /dev/rwd0m: UNEXPECTED INCONSISTENCY; RUN fsck_ffs MANUALLY. I haven't checked but it probably wants to do it single user when all fs are unmounted. And it would work when fs are unclean shutdown. I applied art@ diff and the exact same partition (which I newfs'd with the original defaults -b 16K -f 2K), went through fsck within 1 minute (I copied original /sbin/fsck_ffs to /sbin/ofsck_ffs). I have enabled bigmem, and his diff is absolutely needed for fast fsck. Thank you Arthur and the team for a very fast turnaround! Thank you for reducing the pain. I will schedule a fsck every month or so, knowing it won't screw up anything and be done really quick. So with the information presented in this thread, fsck shouldn't be a problem for anybody anymore. That is, increasing data block size (say for Postgres or for Virtual images or family videos) and checking only used inodes. Thanks, amit > Please tell us more why -p does not work. What happens if you try it. > Be more exact in your description. > > So why is you system extra slow? Maybe it has too little memory and > starts swapping. Some details might come in handy.
Re: games/tetris: hide the cursor during game
ok nicm
On Thu, Mar 31, 2011 at 10:30:48AM +0200, David Coppa wrote:
> Hi,
>
> Nice improvement for the best Tetris implementation ever made.
> From NetBSD.
>
> OK?
>
> Index: screen.c
> ===
> RCS file: /cvs/src/games/tetris/screen.c,v
> retrieving revision 1.13
> diff -u -p -r1.13 screen.c
> --- screen.c 20 Apr 2006 03:25:36 - 1.13
> +++ screen.c 31 Mar 2011 08:16:26 -
> @@ -80,7 +80,9 @@ static char
> *LLstr, /* last line, first column */
> *pcstr, /* pad character */
> *TEstr, /* end cursor motion mode */
> - *TIstr; /* begin cursor motion mode */
> + *TIstr, /* begin cursor motion mode */
> + *VIstr, /* make cursor invisible */
> + *VEstr; /* make cursor appear normal */
> char
> *SEstr, /* end standout mode */
> *SOstr; /* begin standout mode */
> @@ -107,6 +109,8 @@ struct tcsinfo { /* termcap string info
> {"so", &SOstr},
> {"te", &TEstr},
> {"ti", &TIstr},
> + {"vi", &VIstr},
> + {"ve", &VEstr},
> {"up", &UP},/* cursor up */
> { {0}, NULL}
> };
> @@ -291,6 +295,8 @@ scr_set(void)
>*/
> if (TIstr)
> putstr(TIstr); /* termcap(5) says this is not padded */
> + if (VIstr)
> + putstr(VIstr); /* termcap(5) says this is not padded */
> if (tstp != SIG_IGN)
> (void) signal(SIGTSTP, scr_stop);
> if (ttou != SIG_IGN)
> @@ -321,6 +327,8 @@ scr_end(void)
> /* exit screen mode */
> if (TEstr)
> putstr(TEstr); /* termcap(5) says this is not padded */
> + if (VEstr)
> + putstr(VEstr); /* termcap(5) says this is not padded */
> (void) fflush(stdout);
> (void) tcsetattr(0, TCSADRAIN, &oldtt);
> isset = 0;
Fix physio on bigmem
There were two problems with vslock_device functions that are
used for magic page flipping for physio and bigmem.
- Fix error handling so that we free stuff on error.
- We use the mappings to keep track of which pages need to be
freed so don't unmap before freeing (this is theoretically
incorrect and will be fixed soon).
This makes fsck happy on bigmem machines (it doesn't leak all
dma:able memory anymore).
Index: uvm/uvm_glue.c
===
RCS file: /cvs/src/sys/uvm/uvm_glue.c,v
retrieving revision 1.55
diff -u -r1.55 uvm_glue.c
--- uvm/uvm_glue.c 2 Jul 2010 22:38:32 - 1.55
+++ uvm/uvm_glue.c 1 Apr 2011 15:08:40 -
@@ -222,8 +222,10 @@
paddr_t pa;
if (!pmap_extract(p->p_vmspace->vm_map.pmap,
- start + ptoa(i), &pa))
- return (EFAULT);
+ start + ptoa(i), &pa)) {
+ error = EFAULT;
+ goto out_unwire;
+ }
if (!PADDR_IS_DMA_REACHABLE(pa))
break;
}
@@ -233,13 +235,15 @@
}
if ((va = uvm_km_valloc(kernel_map, sz)) == 0) {
- return (ENOMEM);
+ error = ENOMEM;
+ goto out_unwire;
}
TAILQ_INIT(&pgl);
error = uvm_pglistalloc(npages * PAGE_SIZE, dma_constraint.ucr_low,
dma_constraint.ucr_high, 0, 0, &pgl, npages, UVM_PLA_WAITOK);
- KASSERT(error == 0);
+ if (error)
+ goto out_unmap;
sva = va;
while ((pg = TAILQ_FIRST(&pgl)) != NULL) {
@@ -252,7 +256,16 @@
KASSERT(va == sva + sz);
*retp = (void *)(sva + off);
- error = copyin(addr, *retp, len);
+ if ((error = copyin(addr, *retp, len)) == 0)
+ return 0;
+
+ uvm_km_pgremove_intrsafe(sva, sva + sz);
+ pmap_kremove(sva, sz);
+ pmap_update(pmap_kernel());
+out_unmap:
+ uvm_km_free(kernel_map, sva, sz);
+out_unwire:
+ uvm_fault_unwire(&p->p_vmspace->vm_map, start, end);
return (error);
}
@@ -277,9 +290,9 @@
return;
kva = trunc_page((vaddr_t)map);
+ uvm_km_pgremove_intrsafe(kva, kva + sz);
pmap_kremove(kva, sz);
pmap_update(pmap_kernel());
- uvm_km_pgremove_intrsafe(kva, kva + sz);
uvm_km_free(kernel_map, kva, sz);
}
Defang buffer cache for bigmem.
This diff moves the buffer cache to only be allocated out of dma'able
memory, along with a few pieces (flags) that I will need for the next step
of allowing it to touch high memory.
Appears to behave well for me under load and builds survive it with
the buffer cache cranked up.
I would like to get this in and a first step as it allows me to progress,
and will allow others to fix some of the other problems with drivers without
the buffer cache getting in the way with bigmem turned on.
-Bob
--8<
Index: sys/kern/vfs_bio.c
===
RCS file: /cvs/src/sys/kern/vfs_bio.c,v
retrieving revision 1.127
diff -u -r1.127 vfs_bio.c
--- sys/kern/vfs_bio.c 13 Nov 2010 17:45:44 - 1.127
+++ sys/kern/vfs_bio.c 1 Apr 2011 08:56:34 -
@@ -191,19 +191,43 @@
void
bufinit(void)
{
+ long low, high, dmapages, highpages;
struct bqueues *dp;
/* XXX - for now */
bufhighpages = buflowpages = bufpages = bufcachepercent = bufkvm = 0;
/*
+* First off, figure out how much of memory we can use.
+*
+* XXX for now we only use dma-able memory
+*
+* XXX - this isn't completely accurate, because their may
+* be holes in the physical memory. This needs to be replaced
+* with a uvm_pmemrange function to tell us how many pages
+* are within a constraint range - but this is accurate enough
+* for now.
+*/
+
+ low = atop(dma_constraint.ucr_low);
+ high = atop(dma_constraint.ucr_high);
+ if (high >= physmem) {
+ high = physmem;
+ highpages = 0;
+ }
+ else
+ highpages = physmem - high;
+ /* XXX highpages not used yet but will be very soon. */
+ dmapages = high - low;
+
+ /*
* If MD code doesn't say otherwise, use 10% of kvm for mappings and
-* 10% physmem for pages.
+* 10% of dmaable pages for cache pages.
*/
if (bufcachepercent == 0)
bufcachepercent = 10;
if (bufpages == 0)
- bufpages = physmem * bufcachepercent / 100;
+ bufpages = dmapages * bufcachepercent / 100;
bufhighpages = bufpages;
@@ -212,7 +236,7 @@
* we will not allow uvm to steal back more than this number of
* pages
*/
- buflowpages = physmem * 10 / 100;
+ buflowpages = dmapages * 10 / 100;
/*
* set bufbackpages to 100 pages, or 10 percent of the low water mark
Index: sys/kern/vfs_biomem.c
===
RCS file: /cvs/src/sys/kern/vfs_biomem.c,v
retrieving revision 1.14
diff -u -r1.14 vfs_biomem.c
--- sys/kern/vfs_biomem.c 30 Apr 2010 21:56:39 - 1.14
+++ sys/kern/vfs_biomem.c 1 Apr 2011 05:32:21 -
@@ -29,10 +29,19 @@
vaddr_t buf_kva_start, buf_kva_end;
int buf_needva;
TAILQ_HEAD(,buf) buf_valist;
+void buf_realloc_pages(struct buf *, struct uvm_constraint_range *);
int buf_nkvmsleep;
+#if 0
+extern int needda;
+#endif
+extern voiduvm_pagealloc_multi(struct uvm_object *, voff_t,
+ vsize_t, int);
+extern voiduvm_pagerealloc_multi(struct uvm_object *,
voff_t,
+ vsize_t, int, struct uvm_constraint_range *);
extern struct bcachestats bcstats;
+extern int needbuffer;
/*
* Pages are allocated from a uvm object (we only use it for page storage,
@@ -99,6 +108,10 @@
s = splbio();
SET(bp->b_flags, B_BUSY|B_NOTMAPPED);
+ if (bp->b_data != NULL) {
+ TAILQ_REMOVE(&buf_valist, bp, b_valist);
+ bcstats.busymapped++;
+ }
splx(s);
}
@@ -170,6 +183,26 @@
}
}
CLR(bp->b_flags, B_BUSY|B_NOTMAPPED);
+#if 0
+ if (ISSET(bp->b_flags, B_DAQ) && needda) {
+ wakeup(&needda);
+ }
+#endif
+ /* Wake up any processes waiting for any buffer to become free. */
+ if (needbuffer) {
+ needbuffer--;
+ wakeup(&needbuffer);
+ }
+
+ /*
+* Wake up any processes waiting for _this_ buffer to become
+* free.
+*/
+
+ if (ISSET(bp->b_flags, B_WANTED)) {
+ CLR(bp->b_flags, B_WANTED);
+ wakeup(bp);
+ }
splx(s);
}
@@ -259,11 +292,11 @@
return (va);
}
+/* Always allocates in Device Accessible Memory */
void
buf_alloc_pages(struct buf *bp, vsize_t size)
{
- struct vm_page *pg;
- voff_t offs, i;
+ voff_t offs;
int s;
KASSERT(size == round_page(size));
@@ -277,25 +310,51 @@
KASSERT(buf_page_offset > 0);
- for (i = 0; i < atop(size); i++) {
-#if defined(DEBUG) || 1
- if ((pg = uvm_pagelookup(buf_object, offs + ptoa(i
- panic("buf_
Fw: Segue em anexo...
[IMAGE] 1 anexo(s) Documento.doc (143,2 kb) Segue em anexo conforme solicitado o relatorio e a cotacao de precos e produtos listados a seguir. Tenha um bom dia! Agredecemos a sua preferencia!!! __
Informazioni su Italiainweekend
[IMAGE] Oggetto: Proposta dâinserimento Banner pubblicitari sul news portal www.italiainweekend.it e relativi pubbliredazionali compresi lâinserimento di comunicati stampa, informazioni di servizio e istituzionali della Sua Struttura. Egregia dottoressa, egregio dottore Ci pregiamo comunicarLe di questa nostra iniziativa editoriale riservata a Istituzioni, Enti nazionali e locali, Consorzi, Aziende Turistiche, Alberghi, Enti Parco, Comunit` Montane, Festival nazionali e internazionali di arte, cultura, cinema, spettacolo, enogastronomia. Attraverso i banner che saremo lieti di riservarLe, modulabili in base alle esigenze della Sua Struttura, sar` possibile diffondere a circa 3.000 lettori, con 9000 articoli letti giornalieri (fonte Google Analytics) i principali aspetti dellâattivit` svolta dalla Sua Azienda/Ente tramite testo e foto completamente dedicati. Il news portal www.italiainweekend.it racconta tutti gli eventi che giornalmente avvengono nel Bel Paese, spaziando da argomenti di cultura, turismo, arte, cinema, spettacolo; ai percorsi enogastronomici. H rivolto a tutti quelli che intendono promuovere iniziative tese a valorizzare e pubblicizzare i loro progetti. OFFERTA a) Pubblicazione annuale di comunicati â¬. 300,00 i.e. La tariffa comprende la possibilit` dâinserimento di fotografie e altro materiale oltre il testo fornito, fino a un massimo di un comunicato o un redazionale (firmato) al giorno(anche festivi), per tutta la durata dellâanno contrattuale. b) Banner pubblicitario annuale LATERALE 200x200 px â¬. 600,00 i.e. c) Banner pubblicitario annuale HEADER 644x100 px â¬. 3.600,00 i.e. d) Banner pubblicitario annuale SUPER HEADER 1200x75 px â¬. 7.200,00 i.e. [IMAGE] Nel ringraziarLa dell'attenzione prestataci, porgiamo i nostri piy cordiali saluti. Alfonso Balletta Business Editor Speciali Turismo Italia R.B. Comunicazioni scarl Via Picazio, 1 - CASERTA Tel. 0823 1700340 - Fax 0823 1761335 pubblic...@italiainweekend.it
Re: horribly slow fsck_ffs pass1 performance
Op 31 mrt. 2011 om 22:25 heeft Otto Moerbeek het volgende geschreven: > On Thu, Mar 31, 2011 at 10:14:46PM +0200, Otto Moerbeek wrote: > >> So here's an initial, only lightly tested diff. >> >> Beware, this very well could eat your filesystems. >> >> To note any difference, you should use the -p mode of fsck_ffs (rc >> does that) and the fs should have been mounted with softdep. I now realize speedup will also be there for non -p usage in quite a few cases. So that explains the speed differences seen without -p. But the reported original 4 hours likely means the system is swapping. >> >> I have seen very nice speedups already. > > But don't count yourself a rich man too soon: for ffs2 filesystesm, > you won't see a lot of speedup, because inode blocks are allocated > on-demand there, so a filesystem with few inodes used likely has few > inode blocks. > > Also, depending on the usage patterns, you might have a fs where high > numbered inodes are used, while the fs itself is pretty empty. Filling > up a fs with lots of files and them removing a lot of them is an > example that could lead to such a situation. This diff does not speed > things up in such cases. > >-Otto
