Re: CVS commit: src/sys/dev/acpi
On Mon, Jul 19, 2010 at 12:39:08PM +, Quentin Garnier wrote: > When the platform uses the APIC interrupt model, OSPM associates > processors declared in the namespace with entries in the MADT. Prior to > ACPI 3.0, this was accomplished using the processor object's ProcessorID > and the ACPI Processor ID fields in MADT entries. UID fields were added > to MADT entries in ACPI 3.0. By expanding processor declaration using > Device definitions, UID object values under a processor device are used to > associate processor devices with entries in the MADT. This removes the > previous 256 processor declaration limit. Yes, you are quite right. What I meant that the Processor opcode (section 18.5.93) just says that the ID should be unique. We can of course match against cpu_info::ci_cpuid or cpu_info::ci_initapicid, or anything that is derived from MADT, which we should always prefer. But it does not really matter what the base is. Nor does it matter how and where the ACPI CPUs are attached. We have at least the case where the MADT values are "right" (e.g. are derived from 0) and the processor object's values are "bogus" (e.g. enumerated from 1 by the BIOS writer). Arguably these are BIOS bugs, but unfortunately such flaws are not uncommon. If you have any good ideas, I am all ears. Please feel free to commit as well. It may be enlightening to go see how for instance FreeBSD struggled with this. - Jukka.
re: CVS commit: src/sys/arch
> > FWIW, having a different module load path and choosing the right set > > of modules is an incoming feature but i have no time line for it. > > > > i plan for it to be useful for ppc oea vs. 4xx, sparc32 vs sparc64 > > in 32 bitmode, and there was someone else wanting it too. > > IMHO, it makes sense for these platforms. But here, i386 and PAE are so > close in concept that it feels like a quirk (at least to me). yeah, i pretty much agree, espcially considering all the work ad has done to make x86 modules work regardless of most other options. .mrg.
Re: CVS commit: src/sys/arch
On 24.07.2010 20:47, matthew green wrote: >> On 24.07.2010 09:50, Christoph Egger wrote: >> Not necessarily, the kvm(3) API manipulates PA as u_long (see >> _kvm_kvatop in kvm_i386.c). Changing the paddr_t will need a >> modification of this API too, and basically, all ports will have to move >> to uint64_t PA (or put casts everywhere...) > > ah, i thought that it already used paddr_t for this. it should, imo. > > and i'd think that libkvm should be compiled with 64bit paddr_t (ie, > have the useland definition of paddr_t always be 64 bit) and be smart > enough to deal with the kernel you feed it. 32 bit sparc has to deal > with this a lot, since the page tables are considerably more different > from sparc v8/v9 than i386 vs amd64. That would be a solution, yes. FWIW, I am making all PAE related macros and types prefixed with "pae_" (~ some kind of namespace), and alias the relevant function with them in _KERNEL when option PAE is enabled. For kvm(3), both could be used; I could extract a value from a symbol like "pae_enabled" out of a core file through kvm_nlist, then use the relevant vatop functions. -- Jean-Yves Migeon jeanyves.mig...@free.fr
Re: CVS commit: src/sys/arch
On 24.07.2010 21:00, matthew green wrote: >> On 24.07.2010 17:54, Matthias Drochner wrote: >>> >>> c...@cubidou.net said: Kernel modules, on the other hand... >>> >>> Hmm, didn't think of that. (using them myself only for testing) >>> a gaping ABI incompatibility is completely unacceptable >>> >>> There are two ways to fix this without the 64-bit-paddr overhead, >>> a short-term and a long-term one. >>> The short-term fix would be to use another module load path. >>> This is close to calling it a different port, but it would >>> not affect userland. >> >> "i386pae" and "i386"? Huh... then we will get i386xenpae, i386xen, and >> so on? > > FWIW, having a different module load path and choosing the right set > of modules is an incoming feature but i have no time line for it. > > i plan for it to be useful for ppc oea vs. 4xx, sparc32 vs sparc64 > in 32 bitmode, and there was someone else wanting it too. IMHO, it makes sense for these platforms. But here, i386 and PAE are so close in concept that it feels like a quirk (at least to me). -- Jean-Yves Migeon jeanyves.mig...@free.fr
re: CVS commit: src/sys/arch
> On 24.07.2010 17:54, Matthias Drochner wrote: > > > > c...@cubidou.net said: > >> Kernel modules, on the other hand... > > > > Hmm, didn't think of that. (using them myself only for testing) > > > >> a gaping ABI incompatibility is completely unacceptable > > > > There are two ways to fix this without the 64-bit-paddr overhead, > > a short-term and a long-term one. > > The short-term fix would be to use another module load path. > > This is close to calling it a different port, but it would > > not affect userland. > > "i386pae" and "i386"? Huh... then we will get i386xenpae, i386xen, and > so on? FWIW, having a different module load path and choosing the right set of modules is an incoming feature but i have no time line for it. i plan for it to be useful for ppc oea vs. 4xx, sparc32 vs sparc64 in 32 bitmode, and there was someone else wanting it too. .mrg.
re: CVS commit: src/sys/arch
> On 24.07.2010 09:50, Christoph Egger wrote: > >> XXX Mixing PAE and !PAE modules may lead to unwanted/unexpected results. > >> This > >> cannot be solved easily, and needs lots of thinking before being declared > >> safe (paddr_t/bus_addr_t size handling, PD/PT macros abstractions). > > > > How about making paddr_t always 64bit? That makes it much easier to deal > > with in libkvm. > > Not necessarily, the kvm(3) API manipulates PA as u_long (see > _kvm_kvatop in kvm_i386.c). Changing the paddr_t will need a > modification of this API too, and basically, all ports will have to move > to uint64_t PA (or put casts everywhere...) ah, i thought that it already used paddr_t for this. it should, imo. and i'd think that libkvm should be compiled with 64bit paddr_t (ie, have the useland definition of paddr_t always be 64 bit) and be smart enough to deal with the kernel you feed it. 32 bit sparc has to deal with this a lot, since the page tables are considerably more different from sparc v8/v9 than i386 vs amd64. .mrg.
Re: CVS commit: src/sys/arch
On Sat Jul 24 2010 at 19:05:54 +0200, Jean-Yves Migeon wrote: > > Didn't anyone else read the mail from a week or so ago containing detailed > > measurements of the overhead? > > It measures overhead of PAE, not turning paddr_t to 64 bits on !PAE i386. Oh, right. Sun probably fried my brain today and got me confused. > I don't think that paddr_t moving to 64 bits add much overhead. I would That would be my guess too. > > (I'm not 100% sure if I believe the results without further analysis, > > but at least there are benchmarks) > > I am not sure that benchmarking is a matter of believing :) Of course I believe the numbers. That doesn't mean I believe they represent PAE, i.e. that you actually benchmarked the essence of PAE instead of some random side-effects (or implementation bugs). When facts don't match expectations, suspicions arise. (more poetry available per request)
Re: CVS commit: src/sys/arch
On 24.07.2010 18:40, Antti Kantee wrote: > On Sat Jul 24 2010 at 15:23:50 +, Quentin Garnier wrote: >>> PAE is a "bridge technology", interesting only for few systems. >>> Users of low-end i386 systems would be penalized, and most users >>> of boxes with >4G memory would gain nothing because they run >>> a 64-bit system anyway. >>> So, imho, no kernel overhead is acceptable. >> >> Well, that's one opinion. My own, probably just as humble, is that such >> a gaping ABI incompatibility is completely unacceptable, especially >> after all the work that has been done to make some kernel subsystems a >> little bit more responsible regarding ABI. >> >> I'm really curious to see some actual measurements about that alleged >> overhead. The way I see it right now, we have a known lethal ABI >> incompatibility versus an alleged overhead of unknown size. > > Didn't anyone else read the mail from a week or so ago containing detailed > measurements of the overhead? It measures overhead of PAE, not turning paddr_t to 64 bits on !PAE i386. I don't think that paddr_t moving to 64 bits add much overhead. I would rather incriminate TLB flushes and 64 bits atomic ops... > (I'm not 100% sure if I believe the results without further analysis, > but at least there are benchmarks) I am not sure that benchmarking is a matter of believing :) I could make something out of the phoronix tests [1] and track performance regressions on a revision (or week) basis, atf-style, but as always, the hardest part about benchmarks is interpretation, not running them (once everything is in place) While sysbench announces 20% less memory bandwidth, build.sh runs have no more than 3% overhead with PAE. So, hmm, between one specific measurement and real world use... there is quite a gap. -- Jean-Yves Migeon jeanyves.mig...@free.fr
Re: CVS commit: src/sys/arch
On Sat, Jul 24, 2010 at 05:54:41PM +0200, Matthias Drochner wrote: > > c...@cubidou.net said: > > Kernel modules, on the other hand... > > Hmm, didn't think of that. (using them myself only for testing) > > > a gaping ABI incompatibility is completely unacceptable > > There are two ways to fix this without the 64-bit-paddr overhead, > a short-term and a long-term one. > The short-term fix would be to use another module load path. > This is close to calling it a different port, but it would > not affect userland. > A more correct but more expensive fix would be to keep out > paddr_t from the kernel ABI relevant to modules. You won't keep bus_addr_t out of the ABI any time soon, so this is not a fix. Anyway, this is why I ask about measurements. That would at least allow a discussion based on facts instead of suppositions. -- Quentin Garnier - c...@cubidou.net - c...@netbsd.org "See the look on my face from staying too long in one place [...] every time the morning breaks I know I'm closer to falling" KT Tunstall, Saving My Face, Drastic Fantastic, 2007. pgpmgDEIJOOfL.pgp Description: PGP signature
Re: CVS commit: src/sys/arch
On Sat Jul 24 2010 at 15:23:50 +, Quentin Garnier wrote: > > PAE is a "bridge technology", interesting only for few systems. > > Users of low-end i386 systems would be penalized, and most users > > of boxes with >4G memory would gain nothing because they run > > a 64-bit system anyway. > > So, imho, no kernel overhead is acceptable. > > Well, that's one opinion. My own, probably just as humble, is that such > a gaping ABI incompatibility is completely unacceptable, especially > after all the work that has been done to make some kernel subsystems a > little bit more responsible regarding ABI. > > I'm really curious to see some actual measurements about that alleged > overhead. The way I see it right now, we have a known lethal ABI > incompatibility versus an alleged overhead of unknown size. Didn't anyone else read the mail from a week or so ago containing detailed measurements of the overhead? (I'm not 100% sure if I believe the results without further analysis, but at least there are benchmarks)
Re: CVS commit: src/sys/arch
On 24.07.2010 17:54, Matthias Drochner wrote: > > c...@cubidou.net said: >> Kernel modules, on the other hand... > > Hmm, didn't think of that. (using them myself only for testing) > >> a gaping ABI incompatibility is completely unacceptable > > There are two ways to fix this without the 64-bit-paddr overhead, > a short-term and a long-term one. > The short-term fix would be to use another module load path. > This is close to calling it a different port, but it would > not affect userland. "i386pae" and "i386"? Huh... then we will get i386xenpae, i386xen, and so on? > A more correct but more expensive fix would be to keep out > paddr_t from the kernel ABI relevant to modules. Then bus_addr_t and paddr_t should be split. I do not think that there are many modules that make use of paddr_t-bound variables; in itself, it should remain within uvm and pmap. My biggest concern is that there is no real "stable KPI" we could rely on. rmind@ is reworking the pmap/uvm code in its branch, but told me (in a private mail) that it may take some time before he merges it. So I asked if I could commit PAE first and come back to it later. A long term fix would be to reuse (and probably extend) the modular world, and have some kind of "one kernel fits all" possibility for x86, because paddr_t/PD/PT handling also differs with Xen. But making that work in a safe and consistant manner, without breaking ABI, needs more thought that I currently could put in my patch. Cheers, -- Jean-Yves Migeon jeanyves.mig...@free.fr
Re: CVS commit: src/sys/arch
c...@cubidou.net said: > Kernel modules, on the other hand... Hmm, didn't think of that. (using them myself only for testing) > a gaping ABI incompatibility is completely unacceptable There are two ways to fix this without the 64-bit-paddr overhead, a short-term and a long-term one. The short-term fix would be to use another module load path. This is close to calling it a different port, but it would not affect userland. A more correct but more expensive fix would be to keep out paddr_t from the kernel ABI relevant to modules. best regards Matthias Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: CVS commit: src/sys/arch
On Sat, Jul 24, 2010 at 04:49:28PM +0200, Matthias Drochner wrote: > > christoph_eg...@gmx.de said: > > How about making paddr_t always 64bit? That makes it much easier to > > deal with in libkvm. > > I don't think this is a good argument. The little gain of making > a rarely used library like libkvm simpler can't justify adding > overhead to the kernel. libkvm is hardly an issue. Kernel modules, on the other hand... > PAE is a "bridge technology", interesting only for few systems. > Users of low-end i386 systems would be penalized, and most users > of boxes with >4G memory would gain nothing because they run > a 64-bit system anyway. > So, imho, no kernel overhead is acceptable. Well, that's one opinion. My own, probably just as humble, is that such a gaping ABI incompatibility is completely unacceptable, especially after all the work that has been done to make some kernel subsystems a little bit more responsible regarding ABI. I'm really curious to see some actual measurements about that alleged overhead. The way I see it right now, we have a known lethal ABI incompatibility versus an alleged overhead of unknown size. -- Quentin Garnier - c...@cubidou.net - c...@netbsd.org "See the look on my face from staying too long in one place [...] every time the morning breaks I know I'm closer to falling" KT Tunstall, Saving My Face, Drastic Fantastic, 2007. pgpz9dx02Z0SE.pgp Description: PGP signature
Re: CVS commit: src/sys/arch
christoph_eg...@gmx.de said: > How about making paddr_t always 64bit? That makes it much easier to > deal with in libkvm. I don't think this is a good argument. The little gain of making a rarely used library like libkvm simpler can't justify adding overhead to the kernel. PAE is a "bridge technology", interesting only for few systems. Users of low-end i386 systems would be penalized, and most users of boxes with >4G memory would gain nothing because they run a 64-bit system anyway. So, imho, no kernel overhead is acceptable. best regards Matthias Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt
Re: CVS commit: src/sys/arch
Manuel Bouyer wrote: > On Sat, Jul 24, 2010 at 09:50:33AM +0200, Christoph Egger wrote: > > > How about making paddr_t always 64bit? That makes it much easier to deal > > with in libkvm. > > The overhead would need to be evaluated first. I don't have any figures handy, but when I first tried 64-bit paddr_t on Alchemy (MIPS) CPUs I wasn't able to notice any diffence with some simple benchmarks (a build maybe?). This would have been on a CPU in the 400MHz range. Obviously we can't compare that with modern x86, but I'd still be surprised if you could notice a diffence using 64-bit paddr_t. Cheers, Simon.
Re: CVS commit: src/sys/arch
On 24.07.2010 09:50, Christoph Egger wrote: >> XXX Mixing PAE and !PAE modules may lead to unwanted/unexpected results. This >> cannot be solved easily, and needs lots of thinking before being declared >> safe (paddr_t/bus_addr_t size handling, PD/PT macros abstractions). > > How about making paddr_t always 64bit? That makes it much easier to deal > with in libkvm. Not necessarily, the kvm(3) API manipulates PA as u_long (see _kvm_kvatop in kvm_i386.c). Changing the paddr_t will need a modification of this API too, and basically, all ports will have to move to uint64_t PA (or put casts everywhere...) > paddr_t being always 64bit wide also allows to switch PAE on and off at > runtime like Solaris does. And *off* at runtime? That's one requirement, however you would have tons of extra work to make pmap "dynamically" switch from !PAE to PAE, due to paddr_t, pd/pt, bus_addr_t modifications. That would need a modular pmap, and load that module very early. IMHO, "if it ever gets done", we should add Xen to the loop (and fix that module path issue). So that we would have only one kernel for DOM0, GENERIC and GENERIC PAE. Besides, I suspect that turning paddr_t to uint64_t will add overhead, even if the upper 32 bits are always 0 in GENERIC. > Pleaes fix the amd64 build error reported on current-us...@. > The build error is related to rump. Investigating. rumptest just finished for i386 and amd64, and no error whatsoever. Guess I'll have to dig further... -- Jean-Yves Migeon jeanyves.mig...@free.fr
Re: CVS commit: src/sys/arch
On Sat, Jul 24, 2010 at 02:03:14PM +0200, Jean-Yves Migeon wrote: > On 24.07.2010 12:30, Manuel Bouyer wrote: > >> How about making paddr_t always 64bit? That makes it much easier to deal > >> with in libkvm. > > > > The overhead would need to be evaluated first. > > Also, I'm not sure this would fix all the libkvm issues (the page table > > format is still different). > > For the page table format, the PAE types and macros could be redefined > with a prefix, and then aliased to the ones used in kernel when "options > PAE" is defined. > > kvatop/pa2off functions could have access to both macros/typedefs, by > just having pae_ prefixed in front (pae_pl*_pi, pae_pd_entry_t, and so > on). It becomes a matter of calling the proper code within kvm(3), by > checking if PAE was enabled within the kernel dump (through kvm_nlist, > for example). Is there other places where paddr_t is used in libkvm (AFAIK there's none) ? If not, then there's no point in making paddr_t 64bits for plain i386 for this issue. -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference --
Re: CVS commit: src/sys/arch
On 24.07.2010 12:30, Manuel Bouyer wrote: >> How about making paddr_t always 64bit? That makes it much easier to deal >> with in libkvm. > > The overhead would need to be evaluated first. > Also, I'm not sure this would fix all the libkvm issues (the page table > format is still different). For the page table format, the PAE types and macros could be redefined with a prefix, and then aliased to the ones used in kernel when "options PAE" is defined. kvatop/pa2off functions could have access to both macros/typedefs, by just having pae_ prefixed in front (pae_pl*_pi, pae_pd_entry_t, and so on). It becomes a matter of calling the proper code within kvm(3), by checking if PAE was enabled within the kernel dump (through kvm_nlist, for example). -- Jean-Yves Migeon jeanyves.mig...@free.fr
Re: CVS commit: src/sys/arch
On Sat, Jul 24, 2010 at 09:50:33AM +0200, Christoph Egger wrote: > > XXX kvm(3) will be fixed in another patch to properly handle both PAE and > > !PAE > > kernel dumps (VA => PA macros are slightly different, and need proper 64 > > bits > > PA support in kvm_i386). > > > > XXX Mixing PAE and !PAE modules may lead to unwanted/unexpected results. > > This > > cannot be solved easily, and needs lots of thinking before being declared > > safe (paddr_t/bus_addr_t size handling, PD/PT macros abstractions). > > > > How about making paddr_t always 64bit? That makes it much easier to deal > with in libkvm. The overhead would need to be evaluated first. Also, I'm not sure this would fix all the libkvm issues (the page table format is still different). -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference --
Re: CVS commit: src/sys/arch
On 24.07.10 02:45, Jean-Yves Migeon wrote: > Module Name: src > Committed By: jym > Date: Sat Jul 24 00:45:57 UTC 2010 > > Modified Files: > src/sys/arch/i386/conf: GENERIC > src/sys/arch/i386/i386: bioscall.S kvm86call.S locore.S machdep.c > mptramp.S multiboot.c > src/sys/arch/i386/include: pmap.h > src/sys/arch/x86/include: cpu.h pmap.h > src/sys/arch/x86/x86: cpu.c pmap.c > src/sys/arch/xen/x86: cpu.c x86_xpmap.c xenfunc.c > > Log Message: > Welcome PAE inside i386 current. > Great work! [...] > XXX kvm(3) will be fixed in another patch to properly handle both PAE and !PAE > kernel dumps (VA => PA macros are slightly different, and need proper 64 bits > PA support in kvm_i386). > > XXX Mixing PAE and !PAE modules may lead to unwanted/unexpected results. This > cannot be solved easily, and needs lots of thinking before being declared > safe (paddr_t/bus_addr_t size handling, PD/PT macros abstractions). > How about making paddr_t always 64bit? That makes it much easier to deal with in libkvm. paddr_t being always 64bit wide also allows to switch PAE on and off at runtime like Solaris does. Pleaes fix the amd64 build error reported on current-us...@. The build error is related to rump. Christoph
