Re: CVS commit: src/share/man/man4
On 29.08.2011 22:27, Manuel Bouyer wrote: > On Mon, Aug 29, 2011 at 03:17:16PM +0100, Jean-Yves Migeon wrote: >> Both; usually I am using the VGA-emulated display, but sometimes >> (like I did lately) I switch to console, so I can keep a reasonable >> amount of history of the dom0 output in my terminal. > > The the dom0 kernel is using xencons, and the ddb sequence is '+'. > See the cn_set_magic() call in xencons.c Yes, I can confirm now. I'll update ddb(4) to reflect this. -- Jean-Yves Migeon jeanyves.mig...@free.fr
Re: CVS commit: src/share/man/man4
On Mon, Aug 29, 2011 at 03:17:16PM +0100, Jean-Yves Migeon wrote: > Both; usually I am using the VGA-emulated display, but sometimes > (like I did lately) I switch to console, so I can keep a reasonable > amount of history of the dom0 output in my terminal. The the dom0 kernel is using xencons, and the ddb sequence is '+'. See the cn_set_magic() call in xencons.c -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference --
Re: CVS commit: src/sys/arch/xen
On 29.08.2011 15:03, Cherry G. Mathew wrote: > I'm a bit confused now - are we assuming that the pmap lock protects the > (pte update op queue-push(es) + pmap_pte_flush()) as a single atomic > operation (ie; no invlpg/tlbflush or out-of-band-read can occur between > the update(s) and the pmap_pte_flush()) ? > > If so, I think I've slightly misunderstood the scope of the mmu queue > design - I assumed that the queue is longer-lived, and the sync points > (for the queue flush) can span across pmap locking - a sort of lazy pte > update, with the queue being flushed at out-of-band or in-band read > time ( I guess that won't work though - how does one know when the > hardware walks the page table ?) . It seems that the queue is meant for > pte updates in loops, for eg:, quickly followed by a flush. Is this > correct ? IMHO, this should be regarded this way, and nothing else. x86 and xen pmap(9) share a lot in common, low level operations (like these: PT/PD editing, TLB flushes, MMU updates...) should not leak through this abstraction. Said differently, the way Xen handles MMU must remain transparent to pmap, except in a few places. Remember, although we are adding a level of indirection through hypervisor, the calls should remain close to native x86 semantic. However, for convenience, Xen offers "multiseats" MMU hypercalls, where you can schedule more than one op at a time to avoid unneeded context switches, like in the pmap_alloc_level() function. This is our problematic part. > If so, there's just one hazard afaict - the synchronous TLB_FLUSH_MULTI > could beat the race between the queue update and the queue flush via > pmap_tlb_shootnow() (see pmap_tlb.c on the cherry-xenmp branch, and *if* > other CPUs reload their TLBs before the flush, they'll have stale info. What stale info? If a VCPU queue isn't empty while another VCPU has scheduled a TLB_FLUSH_MULTI op, the stale content of the queue will eventually be depleted later after a pmap_pte_flush() followed by a local invalidation. This is the part that should be carefully reviewed. For clarity, I would expect a queue to be empty when leaving pmap (e.g. when releasing its lock). Assertions might be necessary to catch all corner cases. > So the important question (rmind@ ?) is, is pmap_tlb_shootnow() > guaranteed to be always called with the pmap lock held ? > > In real life, I removed the global xpq_queue_lock() and the pmap was > falling apart. So a bit of debugging ahead. Did you keep the same queue for all CPUs? If that was the case, yes, this is a call for trouble. Anyway, we can't put a giant lock around xpq_queue. This doesn't make any sense in a MP system, especially for operations that are frequently called and still may take a while to complete. Just imagine: all CPUs waiting for one to finish its TLB flush before they can edit their PD/PT again... ouch. -- Jean-Yves Migeon jeanyves.mig...@free.fr
Re: CVS commit: src/share/man/man4
On Mon, 29 Aug 2011 15:25:03 +0200, Manuel Bouyer wrote: On Mon, Aug 29, 2011 at 12:45:27PM +0100, Jean-Yves Migeon wrote: Hmm, I'll have to cross-check that one this afternoon. IIRC, I am also using the the default "break" command when I am running the dom0 inside QEMU, and '+' is only used when I want to break in the domU (through xencons(4)). But when running inside qemu, you're using qemu's VGA display, not its serial port, right ? Both; usually I am using the VGA-emulated display, but sometimes (like I did lately) I switch to console, so I can keep a reasonable amount of history of the dom0 output in my terminal. in this case I guess the dom0 is using wscons, not the xencons. xencons is (should) be used by dom0 only on serial console. -- Jean-Yves Migeon jeanyves.mig...@free.fr
Re: CVS commit: src/sys/arch/xen
On Mon, Aug 29, 2011 at 03:03:37PM +0200, Cherry G. Mathew wrote: > Hi Manuel, > > > "Manuel" == Manuel Bouyer writes: > > > [...] > > >> > >> Xen's TLB_FLUSH operation is synchronous, and doesn't require an > >> IPI (within the domain), which makes the queue ordering even more > >> important (to make sure that stale ptes are not reloaded before > >> the per-cpu queue has made progress). Yes, we can implement a > >> roundabout ipi driven queueflush + tlbflush scheme(described > >> below), but that would be performance sensitive, and the basic > >> issue won't go away, imho. > >> > >> Let's stick to the xpq ops for a second, ignoring "out-of-band" > >> reads (for which I agree that your assertion, that locking needs > >> to be done at a higher level, holds true). > >> > >> The question here, really is, what are the global ordering > >> requirements of per-cpu memory op queues, given the following > >> basic "ops": > >> > >> i) write memory (via MMU_NORMAL_PT_UPDATE, MMU_MACHPHYS_UPDATE) > >> ii) read memory via: MMUEXT_PIN_L1_TABLE MMUEXT_PIN_L2_TABLE > >> MMUEXT_PIN_L3_TABLE MMUEXT_PIN_L4_TABLE MMUEXT_UNPIN_TABLE > > Manuel> This is when adding/removing a page table from a pmap. When > Manuel> this occurs, the pmap is locked, isn't it ? > > >> MMUEXT_NEW_BASEPTR MMUEXT_NEW_USER_BASEPTR > > Manuel> This is a context switch > > >> MMUEXT_TLB_FLUSH_LOCAL MMUEXT_INVLPG_LOCAL MMUEXT_TLB_FLUSH_MULTI > >> MMUEXT_INVLPG_MULTI MMUEXT_TLB_FLUSH_ALL MMUEXT_INVLPG_ALL > >> MMUEXT_FLUSH_CACHE > > Manuel> This may, or may not, cause a read. This usually happens > Manuel> after updating the pmap, and I guess this also happens with > Manuel> the pmap locked (I have not carefully checked). > > Manuel> So couldn't we just use the pmap lock for this ? I suspect > Manuel> the same lock will also be needed for out of band reads at > Manuel> some point (right now it's protected by splvm()). > > I'm a bit confused now - are we assuming that the pmap lock protects the > (pte update op queue-push(es) + pmap_pte_flush()) as a single atomic > operation (ie; no invlpg/tlbflush or out-of-band-read can occur between > the update(s) and the pmap_pte_flush()) ? out of band reads can always occurs, there's no lock which can protect against this. > > If so, I think I've slightly misunderstood the scope of the mmu queue > design - I assumed that the queue is longer-lived, and the sync points > (for the queue flush) can span across pmap locking - a sort of lazy pte > update, with the queue being flushed at out-of-band or in-band read > time ( I guess that won't work though - how does one know when the > hardware walks the page table ?) . It seems that the queue is meant for > pte updates in loops, for eg:, quickly followed by a flush. Is this > correct ? it was not explicitely designed this way, but I think that's how things are in practice, yes. Usage would need to be checked, though. There may be some special case in the kernel pmap area ... > > If so, there's just one hazard afaict - the synchronous TLB_FLUSH_MULTI > could beat the race between the queue update and the queue flush via > pmap_tlb_shootnow() (see pmap_tlb.c on the cherry-xenmp branch, and *if* > other CPUs reload their TLBs before the flush, they'll have stale info. > > So the important question (rmind@ ?) is, is pmap_tlb_shootnow() > guaranteed to be always called with the pmap lock held ? I don't think so; but I also don't think that's the problem. There shouldn't be more race with this than on native hardware. > > In real life, I removed the global xpq_queue_lock() and the pmap was > falling apart. So a bit of debugging ahead. Hum, on seocnd though, something more may be needed to protect the queue. The pmap lock won't probably work for pmap_kernel ... > > >> [...] > >> > >> >>> I'm thinking that it might be easier and more justifiable to > >> >>> nuke the current queue scheme and implement shadow page > >> tables, >>> which would fit more naturally and efficiently with > >> CAS pte >>> updates, etc. > >> >> > >> >> I'm not sure this would completely fis the issue: with shadow > >> >> page tables you can't use a CAS to assure atomic operation > >> with >> the hardware TLB, as this is, precisely, a shadow PT and > >> not the >> one used by hardware. > >> > >> Definitely worth looking into, I imho. I'm not very comfortable > >> with the queue based scheme for MP. > >> > >> the CAS doesn't provide any guarantees with the TLB on native > >> h/w, afaict. > > Manuel> What makes you think so ? I think the hw TLB also does CAS > Manuel> to update referenced and dirty bits in the PTE, otherwise we > Manuel> couldn't rely on these bits; this would be bad especially > Manuel> for the dirty bit. > > Yes, I missed that one (which i
Re: CVS commit: src/share/man/man4
On Mon, Aug 29, 2011 at 12:45:27PM +0100, Jean-Yves Migeon wrote: > Hmm, I'll have to cross-check that one this afternoon. IIRC, I am > also using the the default "break" command when I am running the > dom0 inside QEMU, and '+' is only used when I want to break in > the domU (through xencons(4)). But when running inside qemu, you're using qemu's VGA display, not its serial port, right ? in this case I guess the dom0 is using wscons, not the xencons. xencons is (should) be used by dom0 only on serial console. -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference --
Re: CVS commit: src/sys/arch/xen
Hi Manuel, > "Manuel" == Manuel Bouyer writes: [...] >> >> Xen's TLB_FLUSH operation is synchronous, and doesn't require an >> IPI (within the domain), which makes the queue ordering even more >> important (to make sure that stale ptes are not reloaded before >> the per-cpu queue has made progress). Yes, we can implement a >> roundabout ipi driven queueflush + tlbflush scheme(described >> below), but that would be performance sensitive, and the basic >> issue won't go away, imho. >> >> Let's stick to the xpq ops for a second, ignoring "out-of-band" >> reads (for which I agree that your assertion, that locking needs >> to be done at a higher level, holds true). >> >> The question here, really is, what are the global ordering >> requirements of per-cpu memory op queues, given the following >> basic "ops": >> >> i) write memory (via MMU_NORMAL_PT_UPDATE, MMU_MACHPHYS_UPDATE) >> ii) read memory via: MMUEXT_PIN_L1_TABLE MMUEXT_PIN_L2_TABLE >> MMUEXT_PIN_L3_TABLE MMUEXT_PIN_L4_TABLE MMUEXT_UNPIN_TABLE Manuel> This is when adding/removing a page table from a pmap. When Manuel> this occurs, the pmap is locked, isn't it ? >> MMUEXT_NEW_BASEPTR MMUEXT_NEW_USER_BASEPTR Manuel> This is a context switch >> MMUEXT_TLB_FLUSH_LOCAL MMUEXT_INVLPG_LOCAL MMUEXT_TLB_FLUSH_MULTI >> MMUEXT_INVLPG_MULTI MMUEXT_TLB_FLUSH_ALL MMUEXT_INVLPG_ALL >> MMUEXT_FLUSH_CACHE Manuel> This may, or may not, cause a read. This usually happens Manuel> after updating the pmap, and I guess this also happens with Manuel> the pmap locked (I have not carefully checked). Manuel> So couldn't we just use the pmap lock for this ? I suspect Manuel> the same lock will also be needed for out of band reads at Manuel> some point (right now it's protected by splvm()). I'm a bit confused now - are we assuming that the pmap lock protects the (pte update op queue-push(es) + pmap_pte_flush()) as a single atomic operation (ie; no invlpg/tlbflush or out-of-band-read can occur between the update(s) and the pmap_pte_flush()) ? If so, I think I've slightly misunderstood the scope of the mmu queue design - I assumed that the queue is longer-lived, and the sync points (for the queue flush) can span across pmap locking - a sort of lazy pte update, with the queue being flushed at out-of-band or in-band read time ( I guess that won't work though - how does one know when the hardware walks the page table ?) . It seems that the queue is meant for pte updates in loops, for eg:, quickly followed by a flush. Is this correct ? If so, there's just one hazard afaict - the synchronous TLB_FLUSH_MULTI could beat the race between the queue update and the queue flush via pmap_tlb_shootnow() (see pmap_tlb.c on the cherry-xenmp branch, and *if* other CPUs reload their TLBs before the flush, they'll have stale info. So the important question (rmind@ ?) is, is pmap_tlb_shootnow() guaranteed to be always called with the pmap lock held ? In real life, I removed the global xpq_queue_lock() and the pmap was falling apart. So a bit of debugging ahead. >> [...] >> >> >>> I'm thinking that it might be easier and more justifiable to >> >>> nuke the current queue scheme and implement shadow page >> tables, >>> which would fit more naturally and efficiently with >> CAS pte >>> updates, etc. >> >> >> >> I'm not sure this would completely fis the issue: with shadow >> >> page tables you can't use a CAS to assure atomic operation >> with >> the hardware TLB, as this is, precisely, a shadow PT and >> not the >> one used by hardware. >> >> Definitely worth looking into, I imho. I'm not very comfortable >> with the queue based scheme for MP. >> >> the CAS doesn't provide any guarantees with the TLB on native >> h/w, afaict. Manuel> What makes you think so ? I think the hw TLB also does CAS Manuel> to update referenced and dirty bits in the PTE, otherwise we Manuel> couldn't rely on these bits; this would be bad especially Manuel> for the dirty bit. Yes, I missed that one (which is much of the point of the CAS in the first place!), you're right. >> If you do a CAS pte update, and the update succeeded, it's a good >> idea to invalidate + shootdown anyway (even on baremetal). Manuel> Yes, of course inval is needed after updating the PTE. But Manuel> using a true CAS is important to get the refereced and dirty Manuel> bits right. I haven't looked into this in detail, but I thought that xen disconnects the shadow (presumably with correct update/dirty bits) and flushes the TLB when a write to the shadow occurs ? In which case these bits will be in sync until the next h/w access ? I'm speculating, I haven't looked at the xen code for this yet. -- Cherry
Re: CVS commit: src/sys/arch/i386/conf
On Mon, Aug 29, 2011 at 02:58:32PM +0200, Marc Balmer wrote: > Am 29.08.11 14:53, schrieb Jukka Ruohonen: > > On Mon, Aug 29, 2011 at 02:48:52PM +0200, Marc Balmer wrote: > >> Am 29.08.11 11:38, schrieb Jukka Ruohonen: > >>> On Sat, Aug 27, 2011 at 09:28:56AM +, Marc Balmer wrote: > Module Name: src > Committed By:mbalmer > Date:Sat Aug 27 09:28:56 UTC 2011 > > Modified Files: > src/sys/arch/i386/conf: GENERIC > > Log Message: > Enable some gpio devices. > >>> > >>> Is onewire(4) really something to be enabled by default on i386? > >> > >> Yes, why not?. It is small enough and lets you access cheap temperature > >> sensors via gpiow(4). > > > > Just that I have never seen a i386 system with onewire(4). > > with gpioow(4) you can use it over gpio(4), and means many system, e.g. > all of Soekris, Alix, etc. Ok, I stand corrected. And then I should've seen it on i386... - Jukka.
Re: CVS commit: src/sys/arch/i386/conf
Am 29.08.11 14:53, schrieb Jukka Ruohonen: > On Mon, Aug 29, 2011 at 02:48:52PM +0200, Marc Balmer wrote: >> Am 29.08.11 11:38, schrieb Jukka Ruohonen: >>> On Sat, Aug 27, 2011 at 09:28:56AM +, Marc Balmer wrote: Module Name: src Committed By: mbalmer Date: Sat Aug 27 09:28:56 UTC 2011 Modified Files: src/sys/arch/i386/conf: GENERIC Log Message: Enable some gpio devices. >>> >>> Is onewire(4) really something to be enabled by default on i386? >> >> Yes, why not?. It is small enough and lets you access cheap temperature >> sensors via gpiow(4). > > Just that I have never seen a i386 system with onewire(4). with gpioow(4) you can use it over gpio(4), and means many system, e.g. all of Soekris, Alix, etc. > > (GENERIC as in: something common or descriptive for all members.) > > - Jukka.
Re: CVS commit: src/sys/arch/i386/conf
On Mon, Aug 29, 2011 at 02:48:52PM +0200, Marc Balmer wrote: > Am 29.08.11 11:38, schrieb Jukka Ruohonen: > > On Sat, Aug 27, 2011 at 09:28:56AM +, Marc Balmer wrote: > >> Module Name: src > >> Committed By: mbalmer > >> Date: Sat Aug 27 09:28:56 UTC 2011 > >> > >> Modified Files: > >>src/sys/arch/i386/conf: GENERIC > >> > >> Log Message: > >> Enable some gpio devices. > > > > Is onewire(4) really something to be enabled by default on i386? > > Yes, why not?. It is small enough and lets you access cheap temperature > sensors via gpiow(4). Just that I have never seen a i386 system with onewire(4). (GENERIC as in: something common or descriptive for all members.) - Jukka.
Re: CVS commit: src/sys/arch/i386/conf
Am 29.08.11 11:38, schrieb Jukka Ruohonen: > On Sat, Aug 27, 2011 at 09:28:56AM +, Marc Balmer wrote: >> Module Name: src >> Committed By:mbalmer >> Date:Sat Aug 27 09:28:56 UTC 2011 >> >> Modified Files: >> src/sys/arch/i386/conf: GENERIC >> >> Log Message: >> Enable some gpio devices. > > Is onewire(4) really something to be enabled by default on i386? Yes, why not?. It is small enough and lets you access cheap temperature sensors via gpiow(4).
Re: CVS commit: src/share/man/man4
On Mon, 29 Aug 2011 11:46:06 +0200, Manuel Bouyer wrote: On Mon, Aug 29, 2011 at 09:01:47AM +0200, Jean-Yves Migeon wrote: What kind of console is attaching for you in dom0? I can't see how '+' would get wired in by default. At least when either started from bare metal, or QEMU. '+' is used for serial console (or, to be more precise, xencons which is what is used by dom0 when the hypervisor is configured to use serial console), where break won't work because the hypervisor doesn't forward it to the dom0. Hmm, I'll have to cross-check that one this afternoon. IIRC, I am also using the the default "break" command when I am running the dom0 inside QEMU, and '+' is only used when I want to break in the domU (through xencons(4)). In that case, I'll have to make adjustments. Having the same key sequence for both dom0 and domU is... problematic. -- Jean-Yves Migeon jeanyves.mig...@free.fr
Re: CVS commit: src/sys/arch/xen
> "Cherry" == Cherry G Mathew writes: [...] Cherry> Xen's TLB_FLUSH operation is synchronous, and doesn't I mean, Xen's TLB_FLUSH_MULTI operations. my current implementation of tlb shootdown is therefore synchronous wrt TLB flushes on all cpus. Cheers, -- Cherry
Re: CVS commit: src/share/man/man4
On Mon, Aug 29, 2011 at 07:19:04AM -0400, Jared McNeill wrote: > Can you put the driver xrefs back in 'SEE ALSO'? I find it useful. I prefer personally a clear list instead (particularly if we have more than 10 of these drivers some day). But sure. - Jukka. > -Original Message- > From: Jukka Ruohonen > Sent: Monday, August 29, 2011 6:41 AM Newsgroups: > gmane.os.netbsd.devel.cvs.full > To: source-changes-full-qavaossjccednm+yrof...@public.gmane.org > Subject: CVS commit: src/share/man/man4 > > Module Name: src > Committed By: jruoho > Date: Mon Aug 29 10:41:10 UTC 2011 > > Modified Files: > src/share/man/man4: dtv.4 > > Log Message: > Various improvements; e.g. primarily speak about DTV (i.e. DVB is just a > standard), split out SUPPORTED DEVICES from SEE ALSO, improve grammar, fix > references, speak about interface instead of device, fix AUTHORS, etc. > > > To generate a diff of this commit: > cvs rdiff -u -r1.2 -r1.3 src/share/man/man4/dtv.4 > > Please note that diffs are not public domain; they are subject to the > copyright notices on the relevant files. > > > > > > > > Modified files: > > Index: src/share/man/man4/dtv.4 > diff -u src/share/man/man4/dtv.4:1.2 src/share/man/man4/dtv.4:1.3 > --- src/share/man/man4/dtv.4:1.2 Sun Aug 14 20:42:09 2011 > +++ src/share/man/man4/dtv.4 Mon Aug 29 10:41:10 2011 > @@ -1,4 +1,4 @@ > -.\" $NetBSD: dtv.4,v 1.2 2011/08/14 20:42:09 wiz Exp $ > +.\" $NetBSD: dtv.4,v 1.3 2011/08/29 10:41:10 jruoho Exp $ > .\" > .\" Copyright (c) 2011 The NetBSD Foundation, Inc. > .\" All rights reserved. > @@ -32,7 +32,7 @@ > .Os > .Sh NAME > .Nm dtv > -.Nd DVB (digital video broadcasting) programming interface > +.Nd interface for digital television > .Sh SYNOPSIS > .Cd "dtv* at dtvbus?" > .Pp > @@ -40,38 +40,63 @@ > .In dev/dtv/dtvio_demux.h > .In dev/dtv/dtvio_frontend.h > .Sh DESCRIPTION > -The > +The machine-independent > +.Nm > +interface provides support for digital television > +.Pq Dv DTV . > +A subset of the Linux Digital Video Broadcasting > +.Pq Dv DVB > +.Dv APIs > +is supported. > +In particular, > .Nm > -driver provides support for a subset of the Linux DVB (digital > -video broadcasting) APIs; in particular, > +provides the DVB frontend and demodulator > +.Dv APIs . > +.Sh SUPPORTED DEVICES > +The following machine-independent devices are supported: > +.Bl -tag -width "auvitek(4) " -offset indent > +.It Xr auvitek 4 > +Auvitek AU0828-based video capture cards > +.It Xr coram 4 > +Hauppauge WinTV-HVR-1250 > +.It Xr xdtv 4 > +Conexant CX2388X-based DTV cards > +.It Xr emdtv 4 > +EM28XX-based DTV cards > +.El > +.Pp > +Note that the > .Nm > -provides the DVB Frontend API and the DVB Demux Device. > +device drivers are generally only available as kernel modules; see > +.Xr module 7 > +and > +.Xr modload 8 > +for additional details. > .Sh FILES > .Bl -tag -width 28n > .It Pa /dev/dvb/adapterN/frontend0 > The frontend device, for controlling the tuner and demodulator hardware. > .It Pa /dev/dvb/adapterN/demux0 > -The demux device, for controlling the DVB hardware's filters. > +The demux device, for controlling the hardware filters. > .It Pa /dev/dvb/adapterN/dvr0 > -Digital video recording device > +Digital video recording device. > .El > .Sh SEE ALSO > -.Xr auvitek 4 , > -.Xr coram 4 , > -.Xr cxdtv 4 , > -.Xr emdtv 4 , > +.Xr video 9 > +.Pp > .Pa pkgsrc/multimedia/dvb-apps , > .Pa pkgsrc/multimedia/mplayer , > -.Pa http://linuxtv.org/downloads/v4l-dvb-apis/dvbapi.html > +.Rs > +.%T Linux Media Infrastructure API. > +.%T Part II. Linux DVB API > +.%D 2011 > +.%U http://linuxtv.org/downloads/v4l-dvb-apis/dvbapi.html > +.Re > .Sh HISTORY > The > .Nm > -device driver first appeared in > +interface first appeared in > .Nx 6.0 . > .Sh AUTHORS > -.An -nosplit > -The > -.Nm > -driver was written by > .An Jared D. McNeill > -.Aq jmcneill-qavaossjccednm+yrof...@public.gmane.org . > +.Aq jmcneill-m2zzwtmgr3hafepfgvp...@public.gmane.org >
Re: CVS commit: src/share/man/man4
Can you put the driver xrefs back in 'SEE ALSO'? I find it useful. -Original Message- From: Jukka Ruohonen Sent: Monday, August 29, 2011 6:41 AM Newsgroups: gmane.os.netbsd.devel.cvs.full To: source-changes-full-qavaossjccednm+yrof...@public.gmane.org Subject: CVS commit: src/share/man/man4 Module Name: src Committed By: jruoho Date: Mon Aug 29 10:41:10 UTC 2011 Modified Files: src/share/man/man4: dtv.4 Log Message: Various improvements; e.g. primarily speak about DTV (i.e. DVB is just a standard), split out SUPPORTED DEVICES from SEE ALSO, improve grammar, fix references, speak about interface instead of device, fix AUTHORS, etc. To generate a diff of this commit: cvs rdiff -u -r1.2 -r1.3 src/share/man/man4/dtv.4 Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files. Modified files: Index: src/share/man/man4/dtv.4 diff -u src/share/man/man4/dtv.4:1.2 src/share/man/man4/dtv.4:1.3 --- src/share/man/man4/dtv.4:1.2 Sun Aug 14 20:42:09 2011 +++ src/share/man/man4/dtv.4 Mon Aug 29 10:41:10 2011 @@ -1,4 +1,4 @@ -.\" $NetBSD: dtv.4,v 1.2 2011/08/14 20:42:09 wiz Exp $ +.\" $NetBSD: dtv.4,v 1.3 2011/08/29 10:41:10 jruoho Exp $ .\" .\" Copyright (c) 2011 The NetBSD Foundation, Inc. .\" All rights reserved. @@ -32,7 +32,7 @@ .Os .Sh NAME .Nm dtv -.Nd DVB (digital video broadcasting) programming interface +.Nd interface for digital television .Sh SYNOPSIS .Cd "dtv* at dtvbus?" .Pp @@ -40,38 +40,63 @@ .In dev/dtv/dtvio_demux.h .In dev/dtv/dtvio_frontend.h .Sh DESCRIPTION -The +The machine-independent +.Nm +interface provides support for digital television +.Pq Dv DTV . +A subset of the Linux Digital Video Broadcasting +.Pq Dv DVB +.Dv APIs +is supported. +In particular, .Nm -driver provides support for a subset of the Linux DVB (digital -video broadcasting) APIs; in particular, +provides the DVB frontend and demodulator +.Dv APIs . +.Sh SUPPORTED DEVICES +The following machine-independent devices are supported: +.Bl -tag -width "auvitek(4) " -offset indent +.It Xr auvitek 4 +Auvitek AU0828-based video capture cards +.It Xr coram 4 +Hauppauge WinTV-HVR-1250 +.It Xr xdtv 4 +Conexant CX2388X-based DTV cards +.It Xr emdtv 4 +EM28XX-based DTV cards +.El +.Pp +Note that the .Nm -provides the DVB Frontend API and the DVB Demux Device. +device drivers are generally only available as kernel modules; see +.Xr module 7 +and +.Xr modload 8 +for additional details. .Sh FILES .Bl -tag -width 28n .It Pa /dev/dvb/adapterN/frontend0 The frontend device, for controlling the tuner and demodulator hardware. .It Pa /dev/dvb/adapterN/demux0 -The demux device, for controlling the DVB hardware's filters. +The demux device, for controlling the hardware filters. .It Pa /dev/dvb/adapterN/dvr0 -Digital video recording device +Digital video recording device. .El .Sh SEE ALSO -.Xr auvitek 4 , -.Xr coram 4 , -.Xr cxdtv 4 , -.Xr emdtv 4 , +.Xr video 9 +.Pp .Pa pkgsrc/multimedia/dvb-apps , .Pa pkgsrc/multimedia/mplayer , -.Pa http://linuxtv.org/downloads/v4l-dvb-apis/dvbapi.html +.Rs +.%T Linux Media Infrastructure API. +.%T Part II. Linux DVB API +.%D 2011 +.%U http://linuxtv.org/downloads/v4l-dvb-apis/dvbapi.html +.Re .Sh HISTORY The .Nm -device driver first appeared in +interface first appeared in .Nx 6.0 . .Sh AUTHORS -.An -nosplit -The -.Nm -driver was written by .An Jared D. McNeill -.Aq jmcneill-qavaossjccednm+yrof...@public.gmane.org . +.Aq jmcneill-m2zzwtmgr3hafepfgvp...@public.gmane.org
Re: CVS commit: src/sys/arch/xen
On Mon, Aug 29, 2011 at 12:07:05PM +0200, Cherry G. Mathew wrote: > JM> On Mon, 22 Aug 2011 12:47:40 +0200, Manuel Bouyer wrote: > >>> This is slightly more complicated than it appears. Some of the > >>> "ops" in a per-cpu queue may have ordering dependencies with > >>> other cpu queues, and I think this would be hard to express > >>> trivially. (an example would be a pte update on one queue, and > >>> reading the same pte read on another queue - these cases are > >>> quite analogous (although completely unrelated) > >> > > Hi, > > So I had a better look at this - implemented per-cpu queues and messed > with locking a bit: > > > >> read don't go through the xpq queue, don't they ? > > JM> Nope, PTE are directly obtained from the recursive mappings > JM> (vtopte/kvtopte). > > Let's call this "out of band" reads. But see below for "in-band" reads. > > JM> Content is "obviously" only writable by hypervisor (so it can > JM> keep control of his mapping alone). > >> I think this is similar to a tlb flush but the other way round, I > >> guess we could use a IPI for this too. > > JM> IIRC that's what the current native x86 code does: it uses an > JM> IPI to signal other processors that a shootdown is necessary. > > Xen's TLB_FLUSH operation is synchronous, and doesn't require an IPI > (within the domain), which makes the queue ordering even more important > (to make sure that stale ptes are not reloaded before the per-cpu queue > has made progress). Yes, we can implement a roundabout ipi driven > queueflush + tlbflush scheme(described below), but that would be > performance sensitive, and the basic issue won't go away, imho. > > Let's stick to the xpq ops for a second, ignoring "out-of-band" reads > (for which I agree that your assertion, that locking needs to be done at > a higher level, holds true). > > The question here, really is, what are the global ordering requirements > of per-cpu memory op queues, given the following basic "ops": > > i) write memory (via MMU_NORMAL_PT_UPDATE, MMU_MACHPHYS_UPDATE) > ii) read memory > via: > MMUEXT_PIN_L1_TABLE > MMUEXT_PIN_L2_TABLE > MMUEXT_PIN_L3_TABLE > MMUEXT_PIN_L4_TABLE > MMUEXT_UNPIN_TABLE This is when adding/removing a page table from a pmap. When this occurs, the pmap is locked, isn't it ? > MMUEXT_NEW_BASEPTR > MMUEXT_NEW_USER_BASEPTR This is a context switch > MMUEXT_TLB_FLUSH_LOCAL > MMUEXT_INVLPG_LOCAL > MMUEXT_TLB_FLUSH_MULTI > MMUEXT_INVLPG_MULTI > MMUEXT_TLB_FLUSH_ALL > MMUEXT_INVLPG_ALL > MMUEXT_FLUSH_CACHE This may, or may not, cause a read. This usually happens after updating the pmap, and I guess this also happens with the pmap locked (I have not carefully checked). So couldn't we just use the pmap lock for this ? I suspect the same lock will also be needed for out of band reads at some point (right now it's protected by splvm()). > [...] > > >>> I'm thinking that it might be easier and more justifiable to > >>> nuke the current queue scheme and implement shadow page tables, > >>> which would fit more naturally and efficiently with CAS pte > >>> updates, etc. > >> > >> I'm not sure this would completely fis the issue: with shadow > >> page tables you can't use a CAS to assure atomic operation with > >> the hardware TLB, as this is, precisely, a shadow PT and not the > >> one used by hardware. > > Definitely worth looking into, I imho. I'm not very comfortable with > the queue based scheme for MP. > > the CAS doesn't provide any guarantees with the TLB on native h/w, > afaict. What makes you think so ? I think the hw TLB also does CAS to update referenced and dirty bits in the PTE, otherwise we couldn't rely on these bits; this would be bad especially for the dirty bit. > If you do a CAS pte update, and the update succeeded, it's a > good idea to invalidate + shootdown anyway (even on baremetal). Yes, of course inval is needed after updating the PTE. But using a true CAS is important to get the refereced and dirty bits right. -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference --
Re: CVS commit: src/sys/arch/xen
Cc:ing tech-kern, to get wider feedback. Thread started here: http://mail-index.netbsd.org/source-changes-d/2011/08/21/msg003897.html > "JM" == Jean-Yves Migeon writes: JM> On Mon, 22 Aug 2011 12:47:40 +0200, Manuel Bouyer wrote: >>> This is slightly more complicated than it appears. Some of the >>> "ops" in a per-cpu queue may have ordering dependencies with >>> other cpu queues, and I think this would be hard to express >>> trivially. (an example would be a pte update on one queue, and >>> reading the same pte read on another queue - these cases are >>> quite analogous (although completely unrelated) >> Hi, So I had a better look at this - implemented per-cpu queues and messed with locking a bit: >> read don't go through the xpq queue, don't they ? JM> Nope, PTE are directly obtained from the recursive mappings JM> (vtopte/kvtopte). Let's call this "out of band" reads. But see below for "in-band" reads. JM> Content is "obviously" only writable by hypervisor (so it can JM> keep control of his mapping alone). >> I think this is similar to a tlb flush but the other way round, I >> guess we could use a IPI for this too. JM> IIRC that's what the current native x86 code does: it uses an JM> IPI to signal other processors that a shootdown is necessary. Xen's TLB_FLUSH operation is synchronous, and doesn't require an IPI (within the domain), which makes the queue ordering even more important (to make sure that stale ptes are not reloaded before the per-cpu queue has made progress). Yes, we can implement a roundabout ipi driven queueflush + tlbflush scheme(described below), but that would be performance sensitive, and the basic issue won't go away, imho. Let's stick to the xpq ops for a second, ignoring "out-of-band" reads (for which I agree that your assertion, that locking needs to be done at a higher level, holds true). The question here, really is, what are the global ordering requirements of per-cpu memory op queues, given the following basic "ops": i) write memory (via MMU_NORMAL_PT_UPDATE, MMU_MACHPHYS_UPDATE) ii) read memory via: MMUEXT_PIN_L1_TABLE MMUEXT_PIN_L2_TABLE MMUEXT_PIN_L3_TABLE MMUEXT_PIN_L4_TABLE MMUEXT_UNPIN_TABLE MMUEXT_NEW_BASEPTR MMUEXT_TLB_FLUSH_LOCAL MMUEXT_INVLPG_LOCAL MMUEXT_TLB_FLUSH_MULTI MMUEXT_INVLPG_MULTI MMUEXT_TLB_FLUSH_ALL MMUEXT_INVLPG_ALL MMUEXT_FLUSH_CACHE MMUEXT_NEW_USER_BASEPTR (ie; anything that will cause the processor to re-read data updated via another cpu (via, for eg: pte update with i), above) There's two ways I can think of fixing this: a) *before* queueing a local read-op, synchronously flush queues on all other CPUs via ipis. This is slightly racy, but can be done, I think. An optimisation for invlpg could be to implement a scoreboard that watches mem. locations that have been queued for update on any cpu. Scan through the scoreboard for the memory range we're invlpg-ing. If it's not there, there's no need to flush any queues on other cpus. b) read-ops wait on a global condvar. If it's set, a write-op that needs flushing is pending. Wait (with optional timeout and ipi-"nudge") until the remote queue is flushed. When flushing a queue, send a cv_broadcast to any waiters. Option "b)" is slightly better than my current scheme which is to lock any and all mmu-ops and operate the queues synchronously (via XENDEBUG_SYNC). I cannot think of anything else, other than ad-hoc locking + queue flushing, which could be hard to maintain and debug in the long run. >>> I'm thinking that it might be easier and more justifiable to >>> nuke the current queue scheme and implement shadow page tables, >>> which would fit more naturally and efficiently with CAS pte >>> updates, etc. >> >> I'm not sure this would completely fis the issue: with shadow >> page tables you can't use a CAS to assure atomic operation with >> the hardware TLB, as this is, precisely, a shadow PT and not the >> one used by hardware. Definitely worth looking into, I imho. I'm not very comfortable with the queue based scheme for MP. the CAS doesn't provide any guarantees with the TLB on native h/w, afaict. If you do a CAS pte update, and the update succeeded, it's a good idea to invalidate + shootdown anyway (even on baremetal). Do let me know your thoughts, Cheers, -- Cherry
Re: CVS commit: src/sys/kern
On Mon, Aug 29, 2011 at 08:47:51AM +0200, J. Hannken-Illjes wrote: > > On Aug 29, 2011, at 2:39 AM, Mindaugas Rasiukevicius wrote: > > > Module Name:src > > Committed By: rmind > > Date: Mon Aug 29 00:39:16 UTC 2011 > > > > Modified Files: > > src/sys/kern: sys_select.c > > > > Log Message: > > Add kern.direct_select sysctl. Default to 0 for now. > > Why is it disabled by default? And what it does? Can we have it also documented? - Jukka.
Re: CVS commit: src/share/man/man4
On Mon, Aug 29, 2011 at 09:01:47AM +0200, Jean-Yves Migeon wrote: > What kind of console is attaching for you in dom0? I can't see how > '+' would get wired in by default. At least when either started from > bare metal, or QEMU. '+' is used for serial console (or, to be more precise, xencons which is what is used by dom0 when the hypervisor is configured to use serial console), where break won't work because the hypervisor doesn't forward it to the dom0. -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference --
Re: CVS commit: src/sys/arch/i386/conf
On Sat, Aug 27, 2011 at 09:28:56AM +, Marc Balmer wrote: > Module Name: src > Committed By: mbalmer > Date: Sat Aug 27 09:28:56 UTC 2011 > > Modified Files: > src/sys/arch/i386/conf: GENERIC > > Log Message: > Enable some gpio devices. Is onewire(4) really something to be enabled by default on i386? - Jukka.
Re: CVS commit: src/share/man/man4
On 29.08.2011 06:30, Christoph Egger wrote: > On 29.08.11 06:26, Christoph Egger wrote: >> On 29.08.11 00:09, Jean-Yves Migeon wrote: >>> Module Name:src >>> Committed By: jym >>> Date: Sun Aug 28 22:09:37 UTC 2011 >>> >>> Modified Files: >>> src/share/man/man4: ddb.4 >>> >>> Log Message: >>> Be more precise for Xen: + is only valid for Xen domUs. dom0 uses >>> the same key sequences as i386/amd64. >> >> + also works for me in a dom0. > > I think I have to mention I am using serial console regularly. What kind of console is attaching for you in dom0? I can't see how '+' would get wired in by default. At least when either started from bare metal, or QEMU. -- Jean-Yves Migeon jeanyves.mig...@free.fr
