Re: 8.1-PRERELEASE: CPU packages not detected correctly
on 02/09/2010 11:25 Andriy Gapon said the following: on 01/09/2010 21:26 Jung-uk Kim said the following: On Friday 27 August 2010 03:36 pm, Jung-uk Kim wrote: [3] AMD is working on an SMT-capable CPU (code-named Bulldozer) and my patch won't work on them. If anyone has a Bulldozer sample, please look into it. I checked AMD website today and found out a new CPUID Spec. Rev. 2.34 was just released: http://support.amd.com/us/Embedded_TechDocs/25481.pdf They have added CPUID 0x801d and 0x801e to detect topology, it seems. Also, CPUID 0x8001 %ecx bit 22 (TopologyExtensions) tells you whether the above CPUID functions are supported. Interesting... Yeah, I've heard that they are adding SMT capabilities in Bulldozer processors, so I guess they have to change CPU topology detection indicators. The above comment by me is almost nonsense :) I looked at the document with better attention and I think that what the new way does is better/provides proper support for discovering non-uniform topologies based on APIC IDs of the cores. BTW, I am reading that AMD Bulldozer CPU package will consist of modules, which in turn will consist of cores. When a core would be something in between current notion of core and current notion of hardware thread (SMT/HTT/whatever). The Bulldozer cores will have independent arithmetic units and L1 caches, but shared FPU, L2, fetch+decode and some other elements; L3 cache is per package (shared by all modules). -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
on 01/09/2010 21:26 Jung-uk Kim said the following: On Friday 27 August 2010 03:36 pm, Jung-uk Kim wrote: [3] AMD is working on an SMT-capable CPU (code-named Bulldozer) and my patch won't work on them. If anyone has a Bulldozer sample, please look into it. I checked AMD website today and found out a new CPUID Spec. Rev. 2.34 was just released: http://support.amd.com/us/Embedded_TechDocs/25481.pdf They have added CPUID 0x801d and 0x801e to detect topology, it seems. Also, CPUID 0x8001 %ecx bit 22 (TopologyExtensions) tells you whether the above CPUID functions are supported. Interesting... Yeah, I've heard that they are adding SMT capabilities in Bulldozer processors, so I guess they have to change CPU topology detection indicators. -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On Friday 27 August 2010 03:36 pm, Jung-uk Kim wrote: [3] AMD is working on an SMT-capable CPU (code-named Bulldozer) and my patch won't work on them. If anyone has a Bulldozer sample, please look into it. I checked AMD website today and found out a new CPUID Spec. Rev. 2.34 was just released: http://support.amd.com/us/Embedded_TechDocs/25481.pdf They have added CPUID 0x801d and 0x801e to detect topology, it seems. Also, CPUID 0x8001 %ecx bit 22 (TopologyExtensions) tells you whether the above CPUID functions are supported. Interesting... Jung-uk Kim ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
Oops, the patch had a small mistake. I've put it here now, just in case I'll want to fix/cleanup anything else: http://people.freebsd.org/~avg/intel-cpu-topo.diff on 28/08/2010 23:22 Andriy Gapon said the following: So, here is my take on the problem. The attached patch is against sources in FreeBSD tree, it should be applied either to sys/amd64/amd64/mp_machdep.c or sys/i386/i386/mp_machdep.c depending on the desired architecture. The patch is substantially based on the Junk-uk's patch, but with some changes and additions: - topo_probo_0x4() is rewritten so that it does APIC ID matching against masks as described in the Intel article. The code still heavily depends on the assumption of the uniform topology, it discovers number of cores in BSP package and number of threads in BSP core and extrapolates that to global topology. The difference with current code and Junk-uk's patch is that actual APIC ID matching is done as opposed to deriving counts purely from max. values. - added a few comments that describe uniformity assumption, plus couple other useful things. - changed final fallback code, so that each logical CPU is considered to be in its own physical package as opposed to current code placing all logical CPUs as cores of a single package. The rest is Junk-uk's work. Concerns: - about my code: ilog2_round_pow2 name is ugly; looking for suggestions on a better name or re-arranging/writing that code, so that the function is not needed. - about current code: logical_cpus variable (don't confuse with cpu_logical) doesn't seem to be consistently used; e.g. it is not set at all by topo_probo_0xb(); also, the method of using it for setting logical_cpus_mask doesn't seem to be reliable - BSP may be missed. Reviews, comments and test reports are very welcome! Please test the patch if you have any problems with how CPU topology is reported by the current code. Please test even if everything is OK, to avoid regressions. -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
on 28/08/2010 01:02 Andriy Gapon said the following:
BTW, it may be not that hard.
It seems that 0x4 topology building involves knowing the masks and we already
have that data (just interpreted differently), and APIC IDs of the CPUs and it
seems that we also have that. We don't need to bind to CPUs to learn their
IDs,
we can just iterate over cpu_apic_ids[].
The only problem is that currently topo_probe() is called before
assign_cpu_ids() which populates cpu_apic_ids.
assign_cpu_ids depends on topo_probe to know hyperthreading_cpus value.
So, either cpu_apic_ids could be split out or alternatively we could use
cpu_info[] similarly to how it's done in topo_probe_0xb (skipping !cpu_present
and cpu_disabled entries).
So, here is my take on the problem.
The attached patch is against sources in FreeBSD tree, it should be applied
either to sys/amd64/amd64/mp_machdep.c or sys/i386/i386/mp_machdep.c depending
on the desired architecture.
The patch is substantially based on the Junk-uk's patch, but with some changes
and additions:
- topo_probo_0x4() is rewritten so that it does APIC ID matching against masks
as described in the Intel article. The code still heavily depends on the
assumption of the uniform topology, it discovers number of cores in BSP package
and number of threads in BSP core and extrapolates that to global topology.
The difference with current code and Junk-uk's patch is that actual APIC ID
matching is done as opposed to deriving counts purely from max. values.
- added a few comments that describe uniformity assumption, plus couple other
useful things.
- changed final fallback code, so that each logical CPU is considered to be in
its own physical package as opposed to current code placing all logical CPUs as
cores of a single package.
The rest is Junk-uk's work.
Concerns:
- about my code: ilog2_round_pow2 name is ugly; looking for suggestions on a
better name or re-arranging/writing that code, so that the function is not
needed.
- about current code: logical_cpus variable (don't confuse with cpu_logical)
doesn't seem to be consistently used; e.g. it is not set at all by
topo_probo_0xb(); also, the method of using it for setting logical_cpus_mask
doesn't seem to be reliable - BSP may be missed.
Reviews, comments and test reports are very welcome!
Please test the patch if you have any problems with how CPU topology is reported
by the current code. Please test even if everything is OK, to avoid
regressions.
Thanks!
--
Andriy Gapon
diff --git a/sys/amd64/amd64/mp_machdep.c b/sys/amd64/amd64/mp_machdep.c
index e2f82ec..47a5c0b 100644
--- a/sys/amd64/amd64/mp_machdep.c
+++ b/sys/amd64/amd64/mp_machdep.c
@@ -177,24 +177,101 @@ mem_range_AP_init(void)
}
static void
+topo_probe_amd(void)
+{
+
+ /* AMD processors do not support HTT. */
+ cpu_cores = (amd_feature2 AMDID2_CMP) != 0 ?
+ (cpu_procinfo2 AMDID_CMP_CORES) + 1 : 1;
+ cpu_logical = 1;
+}
+
+/*
+ * Round up to the next power of two, if necessary, and then
+ * take log2.
+ */
+static __inline int
+ilog2_round_pow2(u_int x)
+{
+
+ return fls(x (1 - powerof2(x)));
+}
+
+static void
+topo_probe_0x4(void)
+{
+ u_int p[4];
+ int core_id_bits;
+ int smt_id_bits;
+ int max_cores;
+ int max_logical;
+ int id;
+
+ max_logical = (cpu_feature CPUID_HTT) != 0 ?
+ (cpu_procinfo CPUID_HTT_CORES) 16 : 1;
+ if (max_logical == 1)
+ return;
+
+ /*
+* Because of uniformity assumption we examine only
+* those logical processors that belong to the same
+* package as BSP. Further, we count number of
+* logical processors that belong to the same core
+* as BSP thus deducing number of threads per core.
+*/
+ cpuid_count(0x04, 0, p);
+ max_cores = ((p[0] 26) 0x3f) + 1;
+ smt_id_bits = ilog2_round_pow2(max_logical/max_cores) - 1;
+ if (smt_id_bits 0)
+ return;
+ core_id_bits = smt_id_bits + fls(max_cores) - 1;
+
+ for (id = 0; id = MAX_APIC_ID; id++) {
+ if (!cpu_info[id].cpu_present || cpu_info[id].cpu_disabled)
+ continue;
+ if ((id core_id_bits) != (boot_cpu_id core_id_bits))
+ continue;
+ cpu_cores++;
+ if ((id smt_id_bits) == (boot_cpu_id smt_id_bits))
+ cpu_logical++;
+ }
+
+ cpu_cores /= cpu_logical;
+ if (cpu_logical 1)
+ hyperthreading_cpus = logical_cpus = cpu_logical;
+}
+
+static void
topo_probe_0xb(void)
{
- int logical;
- int p[4];
+ u_int p[4];
int bits;
- int type;
int cnt;
int i;
+ int logical;
+ int type;
int x;
- /* We only support two levels for now. */
+ /* We only support three levels for now. */
for (i = 0; i 3; i++) {
- cpuid_count(0x0B, i, p);
Re: 8.1-PRERELEASE: CPU packages not detected correctly
on 19/08/2010 20:26 Jung-uk Kim said the following: One thing I am not sure is whether those CPUID instructions are executed on *real* CPUs or translated in HVM. On top of that, I am not even sure they will be executed on *correct* cores. I bet they won't. Hmm, I have an impression that we try to detect the topology by doing stuff only on BSP and that's why we handle properly only uniform topologies. So that would make your point about correct cores moot. Am I mistaken? -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On Friday 27 August 2010 01:33 pm, Andriy Gapon wrote: on 19/08/2010 20:26 Jung-uk Kim said the following: One thing I am not sure is whether those CPUID instructions are executed on *real* CPUs or translated in HVM. On top of that, I am not even sure they will be executed on *correct* cores. I bet they won't. Hmm, I have an impression that we try to detect the topology by doing stuff only on BSP and that's why we handle properly only uniform topologies. So that would make your point about correct cores moot. Am I mistaken? No, you're correct. Here goes little background information. Originally, Jeff tried to implement Intel's algorithm[1] as described here, if my understanding is correct: http://software.intel.com/en-us/articles/intel-64-architecture-processor-topology-enumeration/ Unfortunately, this algorithm needed to execute CPUID instruction on every core and aggregate them later. Thus, he had to compromise and made it simpler, I believe[2]. However, this compromise had some side effects and my patch was an attempt to correct them (sorry, I forgot the details). It was never committed because Jeff said I will look it over in more detail soon at the time. Then, I had to return the CPU, which was an engineering sample[3] and now in production. That's how my patch has been neglected for long time and just re-discovered from trashcan of rejected patches, thanks to this thread. ;-) Now, back to my original question. My point was, we should never trust any CPUIDs on emulated CPU if they are translated. What should happen if you have four physical cores and you assign just one for VirtualBox, for example? What should we announce if you are emulating two cores on UP host? How do we know whether it is the real BSP or not? Is it really bound to a CPU? Is CPUID leaf 11 emulated properly? If not, is it an emulator bug or a guest OS bug? Do we really care about physical topology in these cases? IMHO, the answer is no, we don't, and we should say all cores are independent. If anyone really cares and wants prettier printing, we may say N virtual cores, though. I hope it makes sense. Jung-uk Kim [1] I believe he referenced an older revision, though. [2] It was committed as r191648: http://svn.freebsd.org/viewvc/base?view=revisionrevision=191648 Also, this commit is related to kern/145385: http://www.freebsd.org/cgi/query-pr.cgi?pr=145385 [3] AMD is working on an SMT-capable CPU (code-named Bulldozer) and my patch won't work on them. If anyone has a Bulldozer sample, please look into it. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
on 27/08/2010 22:36 Jung-uk Kim said the following: Now, back to my original question. My point was, we should never trust any CPUIDs on emulated CPU if they are translated. What should happen if you have four physical cores and you assign just one for VirtualBox, for example? What should we announce if you are emulating two cores on UP host? How do we know whether it is the real BSP or not? Is it really bound to a CPU? Is CPUID leaf 11 emulated properly? If not, is it an emulator bug or a guest OS bug? Do we really care about physical topology in these cases? IMHO, the answer is no, we don't, and we should say all cores are independent. If anyone really cares and wants prettier printing, we may say N virtual cores, though. Thanks a lot for the rest of the info that I snipped, very interesting and useful! To this issue - I'd say let the developers of virtual machines worry that their machines look like real hardware, not us. More specifically, in this thread we saw that current FreeBSD code (without the patch) and Intel's code detect the same topology and that topology looks reasonable for the person who started the thread. With the patch though, detected topology looks different. So I'd rather not worry about the general case of virtual machines right now. Let's first see more evidence of whether we should trust them or not. -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On Friday 27 August 2010 03:47 pm, Andriy Gapon wrote: on 27/08/2010 22:36 Jung-uk Kim said the following: Now, back to my original question. My point was, we should never trust any CPUIDs on emulated CPU if they are translated. What should happen if you have four physical cores and you assign just one for VirtualBox, for example? What should we announce if you are emulating two cores on UP host? How do we know whether it is the real BSP or not? Is it really bound to a CPU? Is CPUID leaf 11 emulated properly? If not, is it an emulator bug or a guest OS bug? Do we really care about physical topology in these cases? IMHO, the answer is no, we don't, and we should say all cores are independent. If anyone really cares and wants prettier printing, we may say N virtual cores, though. Thanks a lot for the rest of the info that I snipped, very interesting and useful! To this issue - I'd say let the developers of virtual machines worry that their machines look like real hardware, not us. More specifically, in this thread we saw that current FreeBSD code (without the patch) and Intel's code detect the same topology and that topology looks reasonable for the person who started the thread. With the patch though, detected topology looks different. So I'd rather not worry about the general case of virtual machines right now. Let's first see more evidence of whether we should trust them or not. I quickly looked over Xen sources. It seems the CPUID instruction is translated by this code: http://lxr.xensource.com/lxr/source/tools/libxc/xc_cpuid_x86.c For HVM case, this is how the CPUID_HTT_CORES is set: 185 case 0x0001: 186 /* 187 * EBX[23:16] is Maximum Logical Processors Per Package. 188 * Update to reflect vLAPIC_ID = vCPU_ID * 2. 189 */ 190 regs[1] = (regs[1] 0xu) | ((regs[1] 0x007fu) 1); But CPUID 0x0b is not handled and falls here: 265 default: 266 regs[0] = regs[1] = regs[2] = regs[3] = 0; 267 break; I am not a Xen developer, so please don't shoot me. ;-) Jung-uk Kim ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On 19 August 2010 20:56, pluknet pluk...@gmail.com wrote:
On 19 August 2010 20:39, Andriy Gapon a...@icyb.net.ua wrote:
on 10/08/2010 19:55 pluknet said the following:
On 16 July 2010 19:47, Jung-uk Kim j...@freebsd.org wrote:
The patch should apply fine on both sys/amd64/amd64/mp_machdep.c and
sys/i386/i386/mp_machdep.c.
http://people.freebsd.org/~jkim/mp_machdep2.diff
Hi.
Just checked on Xen HVM with 3 cores.
1) 8.1 unmodified:
FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs
FreeBSD/SMP: 1 package(s) x 3 core(s)
2) 8.1 + patch
FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs
FreeBSD/SMP: 0 package(s) x 1 core(s) x 32 HTT threads
WARNING: Non-uniform processors.
WARNING: Using suboptimal topology.
Can you debug, e.g. with printfs, what exactly goes wrong?
I wonder if in this case code follows some unusual/unexpected path.
Sorry, I'm a bit busy right now.
I hope to debug this somewhere in the next week.
First, sorry for late replay, and thanks Andriy for kicking me ;)
Something really weird there .
topo_probe_0xb() falls early on 1st iteration back to topo_probe_0x4().
topo_probe_0x4() returns incorrect data as well.
topo_probe: cpu_high = b
topo_probe: cpu_vendor_id = 8086
topo_probe_0xb: i = 0, p[1] = 0
topo_probe_0x4: cpu_procinfo = 200800
topo_probe_0x4: cpu_logical = 32
topo_probe_0x4: i = 0, type = 1
topo_probe_0x4: i = 0, level = 1
topo_probe_0x4: i = 0, logical = 1
topo_probe_0x4: i = 0, cores = 16
topo_probe_0x4: i = 1, type = 2
topo_probe_0x4: i = 1, level = 1
topo_probe_0x4: i = 1, logical = 1
topo_probe_0x4: i = 1, cores = 16
topo_probe_0x4: i = 2, type = 3
topo_probe_0x4: i = 2, level = 2
topo_probe_0x4: i = 2, logical = 1
topo_probe_0x4: i = 2, cores = 16
topo_probe#1: mp_ncpus = 3
topo_probe#1: cpu_cores = 1
topo_probe#1: cpu_logical = 32
topo_probe#1: hyperthreading_cpus = 32
topo_probe#2: mp_ncpus = 3
topo_probe#2: cpu_cores = 1
topo_probe#2: cpu_logical = 32
topo_probe#2: hyperthreading_cpus = 32
%%%
static void
topo_probe_0x4(void)
{
u_int p[4];
int cores;
int i;
int level;
int logical;
int type;
cpu_logical = (cpu_feature CPUID_HTT) != 0 ?
(cpu_procinfo CPUID_HTT_CORES) 16 : 1;
printf(topo_probe_0x4: cpu_procinfo = %x\n, cpu_procinfo);
printf(topo_probe_0x4: cpu_logical = %d\n, cpu_logical);
if (cpu_logical == 1) {
cpu_cores = 1;
return;
}
/* We only support three levels for now. */
for (i = 0; i 3; i++) {
cpuid_count(0x04, i, p);
type = p[0] 0x1f;
printf(topo_probe_0x4: i = %d, type = %d\n, i, type);
level = (p[0] 5) 0x7;
printf(topo_probe_0x4: i = %d, level = %d\n, i, level);
logical = ((p[0] 14) 0xfff) + 1;
printf(topo_probe_0x4: i = %d, logical = %d\n, i, logical);
cores = ((p[0] 26) 0x3f) + 1;
printf(topo_probe_0x4: i = %d, cores = %d\n, i, cores);
if (type == 0)
break;
if (level == 1 cpu_logical == logical * cores) {
cpu_cores = cores;
cpu_logical = logical;
break;
}
}
if (cpu_cores == 0)
cpu_cores = 1;
if (cpu_logical 1)
hyperthreading_cpus = logical_cpus = cpu_logical;
}
static void
topo_probe_0xb(void)
{
u_int p[4];
int bits;
int cnt;
int i;
int logical;
int type;
int x;
/* We only support three levels for now. */
for (i = 0; i 3; i++) {
cpuid_count(0x0b, i, p);
/*
* Fall back if it is not really supported.
*/
if (i == 0 p[1] == 0) {
printf(topo_probe_0xb: i = %d, p[1] = %d\n, i, p[1]);
topo_probe_0x4();
return;
}
[...]
}
static void
topo_probe(void)
{
static int cpu_topo_probed = 0;
if (cpu_topo_probed)
return;
printf(topo_probe: cpu_high = %x\n, cpu_high);
printf(topo_probe: cpu_vendor_id = %x\n, cpu_vendor_id);
logical_cpus = logical_cpus_mask = 0;
if (cpu_vendor_id == CPU_VENDOR_AMD)
topo_probe_amd();
else if (cpu_vendor_id == CPU_VENDOR_INTEL) {
if (cpu_high = 0xb)
topo_probe_0xb();
else if (cpu_high = 0x4)
topo_probe_0x4();
}
printf(topo_probe#1: mp_ncpus = %d\n, mp_ncpus);
printf(topo_probe#1: cpu_cores = %d\n, cpu_cores);
printf(topo_probe#1: cpu_logical = %d\n, cpu_logical);
printf(topo_probe#1: hyperthreading_cpus = %d\n, hyperthreading_cpus);
if (cpu_cores == 0)
Re: 8.1-PRERELEASE: CPU packages not detected correctly
on 27/08/2010 23:15 Jung-uk Kim said the following: I quickly looked over Xen sources. It seems the CPUID instruction is translated by this code: http://lxr.xensource.com/lxr/source/tools/libxc/xc_cpuid_x86.c For HVM case, this is how the CPUID_HTT_CORES is set: 185 case 0x0001: 186 /* 187 * EBX[23:16] is Maximum Logical Processors Per Package. 188 * Update to reflect vLAPIC_ID = vCPU_ID * 2. 189 */ 190 regs[1] = (regs[1] 0xu) | ((regs[1] 0x007fu) 1); But CPUID 0x0b is not handled and falls here: Does it have to be handled? 0x4 should still work. 265 default: 266 regs[0] = regs[1] = regs[2] = regs[3] = 0; 267 break; I am not a Xen developer, so please don't shoot me. ;-) I still don't get your point. My point is that if the Intel's code gets the topology right, then the hardware is emulated properly and the problem is with the patch. What is your point? :) -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
on 27/08/2010 23:18 pluknet said the following: First, sorry for late replay, and thanks Andriy for kicking me ;) Something really weird there . topo_probe_0xb() falls early on 1st iteration back to topo_probe_0x4(). topo_probe_0x4() returns incorrect data as well. topo_probe: cpu_high = b topo_probe: cpu_vendor_id = 8086 topo_probe_0xb: i = 0, p[1] = 0 topo_probe_0x4: cpu_procinfo = 200800 topo_probe_0x4: cpu_logical = 32 topo_probe_0x4: i = 0, type = 1 topo_probe_0x4: i = 0, level = 1 topo_probe_0x4: i = 0, logical = 1 topo_probe_0x4: i = 0, cores = 16 topo_probe_0x4: i = 1, type = 2 topo_probe_0x4: i = 1, level = 1 topo_probe_0x4: i = 1, logical = 1 topo_probe_0x4: i = 1, cores = 16 topo_probe_0x4: i = 2, type = 3 topo_probe_0x4: i = 2, level = 2 topo_probe_0x4: i = 2, logical = 1 topo_probe_0x4: i = 2, cores = 16 topo_probe#1: mp_ncpus = 3 topo_probe#1: cpu_cores = 1 topo_probe#1: cpu_logical = 32 topo_probe#1: hyperthreading_cpus = 32 topo_probe#2: mp_ncpus = 3 topo_probe#2: cpu_cores = 1 topo_probe#2: cpu_logical = 32 topo_probe#2: hyperthreading_cpus = 32 My interpretation: 1. Current (unpatched) code reports correct results either by a chance or because Jeff knows some magic patterns. cpu_high (CPUID_Leaf_Max in Intel's lingo) is 0xb, but CPUID.(EAX=11, ECX=0):EBX == 0, which means that we should fallback to 0x4 method. But the code doesn't do that and instead simply sets cpu_logical to 1 and cpu_cores stays zero, which is later translated to mp_ncpus, which happens to match the reality. 2. Intel reports correct results, because it properly probes the topology. It binds to each of the logical processors available and checks their APIC IDs against masks and builds topology info. 3. Patched code works incorrectly, because essentially it only calculates masks for CPU (and cache, for some reason) topology building. Instead of checking IDs against those masks, the code assumes numbers of cores and threads are at their maximum values allowed by the masks[*]. But that is not true and the numbers do not add up and we get that strange incorrect result. Things like that probably do not happen with real hardware much, but they could. The only way to deal with this is by following the correct procedure instead of making assumptions based on BSP. But that may be hard. [*] E.g. that page says: CPUID.1:EBX[23:16] represents the maximum number of addressable IDs (initial APIC ID) that can be assigned to logical processors in a physical package. The value may not be the same as the number of logical processors that are present in the hardware of a physical package. The value of (1 + (CPUID.(EAX=4, ECX=0):EAX[31:26] )) represents the maximum number of addressable IDs (Core_ID) that can be used to enumerate different processor cores in a physical package. The value also can be different than the actual number of processor cores that are present in the hardware of a physical package. -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On Friday 27 August 2010 05:25 pm, Andriy Gapon wrote: on 27/08/2010 23:18 pluknet said the following: First, sorry for late replay, and thanks Andriy for kicking me ;) Something really weird there . topo_probe_0xb() falls early on 1st iteration back to topo_probe_0x4(). topo_probe_0x4() returns incorrect data as well. topo_probe: cpu_high = b topo_probe: cpu_vendor_id = 8086 topo_probe_0xb: i = 0, p[1] = 0 topo_probe_0x4: cpu_procinfo = 200800 topo_probe_0x4: cpu_logical = 32 topo_probe_0x4: i = 0, type = 1 topo_probe_0x4: i = 0, level = 1 topo_probe_0x4: i = 0, logical = 1 topo_probe_0x4: i = 0, cores = 16 topo_probe_0x4: i = 1, type = 2 topo_probe_0x4: i = 1, level = 1 topo_probe_0x4: i = 1, logical = 1 topo_probe_0x4: i = 1, cores = 16 topo_probe_0x4: i = 2, type = 3 topo_probe_0x4: i = 2, level = 2 topo_probe_0x4: i = 2, logical = 1 topo_probe_0x4: i = 2, cores = 16 topo_probe#1: mp_ncpus = 3 topo_probe#1: cpu_cores = 1 topo_probe#1: cpu_logical = 32 topo_probe#1: hyperthreading_cpus = 32 topo_probe#2: mp_ncpus = 3 topo_probe#2: cpu_cores = 1 topo_probe#2: cpu_logical = 32 topo_probe#2: hyperthreading_cpus = 32 My interpretation: 1. Current (unpatched) code reports correct results either by a chance or because Jeff knows some magic patterns. cpu_high (CPUID_Leaf_Max in Intel's lingo) is 0xb, but CPUID.(EAX=11, ECX=0):EBX == 0, which means that we should fallback to 0x4 method. But the code doesn't do that and instead simply sets cpu_logical to 1 and cpu_cores stays zero, which is later translated to mp_ncpus, which happens to match the reality. 2. Intel reports correct results, because it properly probes the topology. It binds to each of the logical processors available and checks their APIC IDs against masks and builds topology info. 3. Patched code works incorrectly, because essentially it only calculates masks for CPU (and cache, for some reason) topology building. Instead of checking IDs against those masks, the code assumes numbers of cores and threads are at their maximum values allowed by the masks[*]. But that is not true and the numbers do not add up and we get that strange incorrect result. Yes, your interpretation is correct, I believe. Things like that probably do not happen with real hardware much, but they could. AFAIK, it never happened on a real hardware. The only way to deal with this is by following the correct procedure instead of making assumptions based on BSP. But that may be hard. Feel free to rewrite the patch. I never intended to commit the patch, any way. If I ever did, it was a year ago. :-) [*] E.g. that page says: CPUID.1:EBX[23:16] represents the maximum number of addressable IDs (initial APIC ID) that can be assigned to logical processors in a physical package. The value may not be the same as the number of logical processors that are present in the hardware of a physical package. The value of (1 + (CPUID.(EAX=4, ECX=0):EAX[31:26] )) represents the maximum number of addressable IDs (Core_ID) that can be used to enumerate different processor cores in a physical package. The value also can be different than the actual number of processor cores that are present in the hardware of a physical package. Yes, I already know that. Jung-uk Kim ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
on 28/08/2010 00:43 Jung-uk Kim said the following: Things like that probably do not happen with real hardware much, but they could. AFAIK, it never happened on a real hardware. The only way to deal with this is by following the correct procedure instead of making assumptions based on BSP. But that may be hard. Feel free to rewrite the patch. I never intended to commit the patch, any way. If I ever did, it was a year ago. :-) :-) BTW, it may be not that hard. It seems that 0x4 topology building involves knowing the masks and we already have that data (just interpreted differently), and APIC IDs of the CPUs and it seems that we also have that. We don't need to bind to CPUs to learn their IDs, we can just iterate over cpu_apic_ids[]. The only problem is that currently topo_probe() is called before assign_cpu_ids() which populates cpu_apic_ids. assign_cpu_ids depends on topo_probe to know hyperthreading_cpus value. So, either cpu_apic_ids could be split out or alternatively we could use cpu_info[] similarly to how it's done in topo_probe_0xb (skipping !cpu_present and cpu_disabled entries). -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On Friday 27 August 2010 04:19 pm, Andriy Gapon wrote: I still don't get your point. My point is that if the Intel's code gets the topology right, then the hardware is emulated properly and the problem is with the patch. What is your point? :) My point is right topology means nothing in this context if CPU affinity of guest OS does not reflect hypervisor's point of view. Jung-uk Kim ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
on 28/08/2010 01:03 Jung-uk Kim said the following: On Friday 27 August 2010 04:19 pm, Andriy Gapon wrote: I still don't get your point. My point is that if the Intel's code gets the topology right, then the hardware is emulated properly and the problem is with the patch. What is your point? :) My point is right topology means nothing in this context if CPU affinity of guest OS does not reflect hypervisor's point of view. I see. And I just cared about the pretty message in dmesg, nothing more :-) -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On Friday 27 August 2010 06:02 pm, Andriy Gapon wrote: on 28/08/2010 00:43 Jung-uk Kim said the following: Things like that probably do not happen with real hardware much, but they could. AFAIK, it never happened on a real hardware. The only way to deal with this is by following the correct procedure instead of making assumptions based on BSP. But that may be hard. Feel free to rewrite the patch. I never intended to commit the patch, any way. If I ever did, it was a year ago. :-) :-) BTW, it may be not that hard. It seems that 0x4 topology building involves knowing the masks and we already have that data (just interpreted differently), and APIC IDs of the CPUs and it seems that we also have that. We don't need to bind to CPUs to learn their IDs, we can just iterate over cpu_apic_ids[]. The only problem is that currently topo_probe() is called before assign_cpu_ids() which populates cpu_apic_ids. assign_cpu_ids depends on topo_probe to know hyperthreading_cpus value. So, either cpu_apic_ids could be split out or alternatively we could use cpu_info[] similarly to how it's done in topo_probe_0xb (skipping !cpu_present and cpu_disabled entries). If you are really up to this, it has to be a two-pass process. Even then, the dmesg won't be pretty because the topology can only be announced after all APs have been started. I mean, nobody's going to like to see a message like this from dmesg output: ... ACPI APIC Table: FOOBAR FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 --- 8 --- Snip several hundred lines! --- 8 --- SMP: AP CPU #1 Launched! FreeBSD/SMP: 1 package(s) x 2 core(s) Root mount waiting for: usbus5 usbus2 Root mount waiting for: usbus5 usbus2 ... In fact, I implemented something like that while I was writing the patch but I discarded it for an obvious reason. ;-) Also, don't forget jhb's work based on ACPI affinity tables. Jung-uk Kim ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
on 28/08/2010 01:33 Jung-uk Kim said the following: If you are really up to this, it has to be a two-pass process. Even then, the dmesg won't be pretty because the topology can only be announced after all APs have been started. I mean, nobody's going to like to see a message like this from dmesg output: ... ACPI APIC Table: FOOBAR FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 --- 8 --- Snip several hundred lines! --- 8 --- SMP: AP CPU #1 Launched! FreeBSD/SMP: 1 package(s) x 2 core(s) Root mount waiting for: usbus5 usbus2 Root mount waiting for: usbus5 usbus2 ... In fact, I implemented something like that while I was writing the patch but I discarded it for an obvious reason. ;-) Well, I was just going to write that I would still keep the assumption that physical packages are identical :-) Not nice, but messing with APs I don't want :) Also, don't forget jhb's work based on ACPI affinity tables. Not sure how they are applicable here. -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On Friday 27 August 2010 06:46 pm, Andriy Gapon wrote: on 28/08/2010 01:33 Jung-uk Kim said the following: Also, don't forget jhb's work based on ACPI affinity tables. Not sure how they are applicable here. Only SRAT is implemented ATM but SLIT should provide CPU affinity information without messing with CPUID. In fact, I'd prefer that over messing with Intel's CPUID hacks, which gets messier whenever they release a new core. ;-) Jung-uk Kim ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
on 19/08/2010 22:15 pluknet said the following: On 19 August 2010 21:27, Andriy Gapon a...@icyb.net.ua wrote: on 19/08/2010 19:56 pluknet said the following: CPU: Intel(R) Xeon(R) CPU E5520 @ 2.27GHz (2763.12-MHz 686-class CPU) Origin = GenuineIntel Id = 0x106a5 Family = 6 Model = 1a Stepping = 5 Features=0x1781fbbfFPU,VME,DE,PSE,TSC,MSR,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,MMX,FXSR,SSE,SSE2,HTT Features2=0x80982201SSE3,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,b31 TSC: P-state invariant real memory = 4194304000 (4000 MB) avail memory = 3932786688 (3750 MB) ACPI APIC Table: Xen HVM FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs FreeBSD/SMP: 0 package(s) x 1 core(s) x 32 HTT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP/HT): APIC ID: 2 cpu2 (AP/HT): APIC ID: 4 Thanks! BTW, what does Intel's code report? Jung-uk's convenience script: http://people.freebsd.org/~jkim/cpu_topology-12212009.sh Software visible enumeration in the system: Number of logical processors visible to the OS: 3 Number of logical processors visible to this process: 3 Number of processor cores visible to this process: 3 Number of physical packages visible to this process: 1 Hierarchical counts by levels of processor topology: # of cores in package 0 visible to this process: 3 . So, original Intel code detects the topology correctly. Jung-uk, despite what you said in the parallel followup, I think that this demonstrates that there is a flaw in your patch as compared to the logic in the Intel-provided code. FWIW, I was surprised to see a loop in topo_probe_0x4 - I don't see such a loop in Intel's code. Also, (level == 1 cpu_logical == logical * cores) verification might be a suspect too. It may be OK for real hardware, but emulated hardware may stick to minimal compatibility required. Affinity masks per SMT thread, per core, per package: Individual: P:0, C:0, T:0 -- 1 Core-aggregated: P:0, C:0 -- 1 Individual: P:0, C:1, T:0 -- 2 Core-aggregated: P:0, C:1 -- 2 Individual: P:0, C:2, T:0 -- 4 Core-aggregated: P:0, C:2 -- 4 Pkg-aggregated: P:0 -- 7 APIC ID listings from affinity masks OS cpu 0, Affinity mask 01 - apic id 0 OS cpu 1, Affinity mask 02 - apic id 2 OS cpu 2, Affinity mask 04 - apic id 4 Package 0 Cache and Thread details L1D is Level 1 Data cache, size(KBytes)= 32, Cores/cache= 1, Caches/package= 3 L1I is Level 1 Instruction cache, size(KBytes)= 32, Cores/cache= 1, Caches/package= 3 L2 is Level 2 Unified cache, size(KBytes)= 256, Cores/cache= 1, Caches/package= 3 L3 is Level 3 Unified cache, size(KBytes)= 8192, Cores/cache= 1, Caches/package= 3 ++++ Cache | L1D| L1D| L1D| Size | 32K| 32K| 32K| OScpu#| 0| 1| 2| Core | c0| c1| c2| AffMsk| 1| 2| 4| ++++ Cache | L1I| L1I| L1I| Size | 32K| 32K| 32K| ++++ Cache | L2| L2| L2| Size |256K|256K|256K| ++++ Cache | L3| L3| L3| Size | 8M| 8M| 8M| ++++ Combined socket AffinityMask= 0x7 -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
on 10/08/2010 19:55 pluknet said the following: On 16 July 2010 19:47, Jung-uk Kim j...@freebsd.org wrote: The patch should apply fine on both sys/amd64/amd64/mp_machdep.c and sys/i386/i386/mp_machdep.c. http://people.freebsd.org/~jkim/mp_machdep2.diff Hi. Just checked on Xen HVM with 3 cores. 1) 8.1 unmodified: FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs FreeBSD/SMP: 1 package(s) x 3 core(s) 2) 8.1 + patch FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs FreeBSD/SMP: 0 package(s) x 1 core(s) x 32 HTT threads WARNING: Non-uniform processors. WARNING: Using suboptimal topology. Can you debug, e.g. with printfs, what exactly goes wrong? I wonder if in this case code follows some unusual/unexpected path. BTW, could you please also provide CPU name/model/features as detected by the kernel? Thanks! -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On 19 August 2010 20:39, Andriy Gapon a...@icyb.net.ua wrote: on 10/08/2010 19:55 pluknet said the following: On 16 July 2010 19:47, Jung-uk Kim j...@freebsd.org wrote: The patch should apply fine on both sys/amd64/amd64/mp_machdep.c and sys/i386/i386/mp_machdep.c. http://people.freebsd.org/~jkim/mp_machdep2.diff Hi. Just checked on Xen HVM with 3 cores. 1) 8.1 unmodified: FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs FreeBSD/SMP: 1 package(s) x 3 core(s) 2) 8.1 + patch FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs FreeBSD/SMP: 0 package(s) x 1 core(s) x 32 HTT threads WARNING: Non-uniform processors. WARNING: Using suboptimal topology. Can you debug, e.g. with printfs, what exactly goes wrong? I wonder if in this case code follows some unusual/unexpected path. Sorry, I'm a bit busy right now. I hope to debug this somewhere in the next week. BTW, could you please also provide CPU name/model/features as detected by the kernel? Sure. CPU: Intel(R) Xeon(R) CPU E5520 @ 2.27GHz (2763.12-MHz 686-class CPU) Origin = GenuineIntel Id = 0x106a5 Family = 6 Model = 1a Stepping = 5 Features=0x1781fbbfFPU,VME,DE,PSE,TSC,MSR,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,MMX,FXSR,SSE,SSE2,HTT Features2=0x80982201SSE3,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,b31 TSC: P-state invariant real memory = 4194304000 (4000 MB) avail memory = 3932786688 (3750 MB) ACPI APIC Table: Xen HVM FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs FreeBSD/SMP: 0 package(s) x 1 core(s) x 32 HTT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP/HT): APIC ID: 2 cpu2 (AP/HT): APIC ID: 4 Just a thought. # HTT might somehow correlate with current maxcpus limit (32). Thanks! -- wbr, pluknet ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On Thursday 19 August 2010 12:56 pm, pluknet wrote: On 19 August 2010 20:39, Andriy Gapon a...@icyb.net.ua wrote: on 10/08/2010 19:55 pluknet said the following: On 16 July 2010 19:47, Jung-uk Kim j...@freebsd.org wrote: The patch should apply fine on both sys/amd64/amd64/mp_machdep.c and sys/i386/i386/mp_machdep.c. http://people.freebsd.org/~jkim/mp_machdep2.diff Hi. Just checked on Xen HVM with 3 cores. 1) 8.1 unmodified: FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs FreeBSD/SMP: 1 package(s) x 3 core(s) 2) 8.1 + patch FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs FreeBSD/SMP: 0 package(s) x 1 core(s) x 32 HTT threads WARNING: Non-uniform processors. WARNING: Using suboptimal topology. Can you debug, e.g. with printfs, what exactly goes wrong? I wonder if in this case code follows some unusual/unexpected path. Sorry, I'm a bit busy right now. I hope to debug this somewhere in the next week. BTW, could you please also provide CPU name/model/features as detected by the kernel? Sure. CPU: Intel(R) Xeon(R) CPU E5520 @ 2.27GHz (2763.12-MHz 686-class CPU) Origin = GenuineIntel Id = 0x106a5 Family = 6 Model = 1a Stepping = 5 Features=0x1781fbbfFPU,VME,DE,PSE,TSC,MSR,MCE,CX8,APIC,SEP,MTRR,PG E,MCA,CMOV,PAT,MMX,FXSR,SSE,SSE2,HTT Features2=0x80982201SSE3,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,b31 TSC: P-state invariant real memory = 4194304000 (4000 MB) avail memory = 3932786688 (3750 MB) ACPI APIC Table: Xen HVM FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs FreeBSD/SMP: 0 package(s) x 1 core(s) x 32 HTT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP/HT): APIC ID: 2 cpu2 (AP/HT): APIC ID: 4 Just a thought. # HTT might somehow correlate with current maxcpus limit (32). One thing I am not sure is whether those CPUID instructions are executed on *real* CPUs or translated in HVM. On top of that, I am not even sure they will be executed on *correct* cores. I bet they won't. If that's the case, we should add exception for virtualized environment as we did for default HZ. Jung-uk Kim ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
on 19/08/2010 19:56 pluknet said the following: CPU: Intel(R) Xeon(R) CPU E5520 @ 2.27GHz (2763.12-MHz 686-class CPU) Origin = GenuineIntel Id = 0x106a5 Family = 6 Model = 1a Stepping = 5 Features=0x1781fbbfFPU,VME,DE,PSE,TSC,MSR,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,MMX,FXSR,SSE,SSE2,HTT Features2=0x80982201SSE3,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,b31 TSC: P-state invariant real memory = 4194304000 (4000 MB) avail memory = 3932786688 (3750 MB) ACPI APIC Table: Xen HVM FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs FreeBSD/SMP: 0 package(s) x 1 core(s) x 32 HTT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP/HT): APIC ID: 2 cpu2 (AP/HT): APIC ID: 4 Thanks! BTW, what does Intel's code report? Jung-uk's convenience script: http://people.freebsd.org/~jkim/cpu_topology-12212009.sh -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On 19 August 2010 21:27, Andriy Gapon a...@icyb.net.ua wrote: on 19/08/2010 19:56 pluknet said the following: CPU: Intel(R) Xeon(R) CPU E5520 @ 2.27GHz (2763.12-MHz 686-class CPU) Origin = GenuineIntel Id = 0x106a5 Family = 6 Model = 1a Stepping = 5 Features=0x1781fbbfFPU,VME,DE,PSE,TSC,MSR,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,MMX,FXSR,SSE,SSE2,HTT Features2=0x80982201SSE3,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,b31 TSC: P-state invariant real memory = 4194304000 (4000 MB) avail memory = 3932786688 (3750 MB) ACPI APIC Table: Xen HVM FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs FreeBSD/SMP: 0 package(s) x 1 core(s) x 32 HTT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP/HT): APIC ID: 2 cpu2 (AP/HT): APIC ID: 4 Thanks! BTW, what does Intel's code report? Jung-uk's convenience script: http://people.freebsd.org/~jkim/cpu_topology-12212009.sh Software visible enumeration in the system: Number of logical processors visible to the OS: 3 Number of logical processors visible to this process: 3 Number of processor cores visible to this process: 3 Number of physical packages visible to this process: 1 Hierarchical counts by levels of processor topology: # of cores in package 0 visible to this process: 3 . Affinity masks per SMT thread, per core, per package: Individual: P:0, C:0, T:0 -- 1 Core-aggregated: P:0, C:0 -- 1 Individual: P:0, C:1, T:0 -- 2 Core-aggregated: P:0, C:1 -- 2 Individual: P:0, C:2, T:0 -- 4 Core-aggregated: P:0, C:2 -- 4 Pkg-aggregated: P:0 -- 7 APIC ID listings from affinity masks OS cpu 0, Affinity mask 01 - apic id 0 OS cpu 1, Affinity mask 02 - apic id 2 OS cpu 2, Affinity mask 04 - apic id 4 Package 0 Cache and Thread details L1D is Level 1 Data cache, size(KBytes)= 32, Cores/cache= 1, Caches/package= 3 L1I is Level 1 Instruction cache, size(KBytes)= 32, Cores/cache= 1, Caches/package= 3 L2 is Level 2 Unified cache, size(KBytes)= 256, Cores/cache= 1, Caches/package= 3 L3 is Level 3 Unified cache, size(KBytes)= 8192, Cores/cache= 1, Caches/package= 3 ++++ Cache | L1D| L1D| L1D| Size | 32K| 32K| 32K| OScpu#| 0| 1| 2| Core | c0| c1| c2| AffMsk| 1| 2| 4| ++++ Cache | L1I| L1I| L1I| Size | 32K| 32K| 32K| ++++ Cache | L2| L2| L2| Size |256K|256K|256K| ++++ Cache | L3| L3| L3| Size | 8M| 8M| 8M| ++++ Combined socket AffinityMask= 0x7 -- wbr, pluknet ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On 19 August 2010 21:26, Jung-uk Kim j...@freebsd.org wrote: On Thursday 19 August 2010 12:56 pm, pluknet wrote: On 19 August 2010 20:39, Andriy Gapon a...@icyb.net.ua wrote: on 10/08/2010 19:55 pluknet said the following: On 16 July 2010 19:47, Jung-uk Kim j...@freebsd.org wrote: The patch should apply fine on both sys/amd64/amd64/mp_machdep.c and sys/i386/i386/mp_machdep.c. http://people.freebsd.org/~jkim/mp_machdep2.diff Hi. Just checked on Xen HVM with 3 cores. 1) 8.1 unmodified: FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs FreeBSD/SMP: 1 package(s) x 3 core(s) 2) 8.1 + patch FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs FreeBSD/SMP: 0 package(s) x 1 core(s) x 32 HTT threads WARNING: Non-uniform processors. WARNING: Using suboptimal topology. Can you debug, e.g. with printfs, what exactly goes wrong? I wonder if in this case code follows some unusual/unexpected path. Sorry, I'm a bit busy right now. I hope to debug this somewhere in the next week. BTW, could you please also provide CPU name/model/features as detected by the kernel? Sure. CPU: Intel(R) Xeon(R) CPU E5520 @ 2.27GHz (2763.12-MHz 686-class CPU) Origin = GenuineIntel Id = 0x106a5 Family = 6 Model = 1a Stepping = 5 Features=0x1781fbbfFPU,VME,DE,PSE,TSC,MSR,MCE,CX8,APIC,SEP,MTRR,PG E,MCA,CMOV,PAT,MMX,FXSR,SSE,SSE2,HTT Features2=0x80982201SSE3,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,b31 TSC: P-state invariant real memory = 4194304000 (4000 MB) avail memory = 3932786688 (3750 MB) ACPI APIC Table: Xen HVM FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs FreeBSD/SMP: 0 package(s) x 1 core(s) x 32 HTT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP/HT): APIC ID: 2 cpu2 (AP/HT): APIC ID: 4 Just a thought. # HTT might somehow correlate with current maxcpus limit (32). One thing I am not sure is whether those CPUID instructions are executed on *real* CPUs or translated in HVM. I may add only that b31 of Features2 presents only in Xen HVM environment, and its role is afaik to indicate a Xen guest mode. There is no any mention of this bit in the latest Intel doc (ie it's reserved/unused). Also, at least NetBSD has a special handling of this bit. See commit log for CPUID2_RAZ in sys/arch/x86/include/specialreg.h, 1.37 FWIW RAZ states for reserved and zero or so. -- wbr, pluknet ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On Thursday 19 August 2010 03:30 pm, pluknet wrote: On 19 August 2010 21:26, Jung-uk Kim j...@freebsd.org wrote: On Thursday 19 August 2010 12:56 pm, pluknet wrote: On 19 August 2010 20:39, Andriy Gapon a...@icyb.net.ua wrote: on 10/08/2010 19:55 pluknet said the following: On 16 July 2010 19:47, Jung-uk Kim j...@freebsd.org wrote: The patch should apply fine on both sys/amd64/amd64/mp_machdep.c and sys/i386/i386/mp_machdep.c. http://people.freebsd.org/~jkim/mp_machdep2.diff Hi. Just checked on Xen HVM with 3 cores. 1) 8.1 unmodified: FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs FreeBSD/SMP: 1 package(s) x 3 core(s) 2) 8.1 + patch FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs FreeBSD/SMP: 0 package(s) x 1 core(s) x 32 HTT threads WARNING: Non-uniform processors. WARNING: Using suboptimal topology. Can you debug, e.g. with printfs, what exactly goes wrong? I wonder if in this case code follows some unusual/unexpected path. Sorry, I'm a bit busy right now. I hope to debug this somewhere in the next week. BTW, could you please also provide CPU name/model/features as detected by the kernel? Sure. CPU: Intel(R) Xeon(R) CPU � � � � � E5520 �@ 2.27GHz (2763.12-MHz 686-class CPU) Origin = GenuineIntel �Id = 0x106a5 �Family = 6 Model = 1a �Stepping = 5 Features=0x1781fbbfFPU,VME,DE,PSE,TSC,MSR,MCE,CX8,APIC,SEP,MTRR ,PG E,MCA,CMOV,PAT,MMX,FXSR,SSE,SSE2,HTT Features2=0x80982201SSE3,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,b31 TSC: P-state invariant real memory �= 4194304000 (4000 MB) avail memory = 3932786688 (3750 MB) ACPI APIC Table: Xen HVM FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs FreeBSD/SMP: 0 package(s) x 1 core(s) x 32 HTT threads �cpu0 (BSP): APIC ID: �0 �cpu1 (AP/HT): APIC ID: �2 �cpu2 (AP/HT): APIC ID: �4 Just a thought. �# HTT might somehow correlate with current maxcpus limit (32). One thing I am not sure is whether those CPUID instructions are executed on *real* CPUs or translated in HVM. I may add only that b31 of Features2 presents only in Xen HVM environment, and its role is afaik to indicate a Xen guest mode. There is no any mention of this bit in the latest Intel doc (ie it's reserved/unused). Ah, that means the HVM is actually translating the instruction, not running directly on the CPU. That means, we cannot use any CPUID instructions for topology detection in HVM. And I bet all MSRs will be translated as well. It is not just HVM, but also any emulations and virtualizations in general. Actually, CPU topology detection does nothing in these environments because hypervisor or whatever will do memory translations and stuff unless some hints are given by guest or ballooning is done for virtual devices and resources. Usually, this kind of problem is handled by VM-specific device drivers, e.g., VirtualBox Guest Additions, VMware Tools, etc. In theory, Xen domU should do much better job than these tools because it is specifically modified to handle these problems without losing performance. Also, at least NetBSD has a special handling of this bit. See commit log for CPUID2_RAZ in sys/arch/x86/include/specialreg.h, 1.37 FWIW RAZ states for reserved and zero or so. Hmm... Interesting. But we should never rely on an undocumented bit to identify HVM or whatever. Thanks for the info, Jung-uk Kim ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On 16 July 2010 19:47, Jung-uk Kim j...@freebsd.org wrote: The patch should apply fine on both sys/amd64/amd64/mp_machdep.c and sys/i386/i386/mp_machdep.c. http://people.freebsd.org/~jkim/mp_machdep2.diff Hi. Just checked on Xen HVM with 3 cores. 1) 8.1 unmodified: FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs FreeBSD/SMP: 1 package(s) x 3 core(s) 2) 8.1 + patch FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs FreeBSD/SMP: 0 package(s) x 1 core(s) x 32 HTT threads WARNING: Non-uniform processors. WARNING: Using suboptimal topology. -- wbr, pluknet ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
Oliver Fromme wrote: Jung-uk Kim wrote: On Thursday 15 July 2010 01:56 pm, Andriy Gapon wrote: on 15/07/2010 19:57 Oliver Fromme said the following: I patched topo_probe() so it calls topo_probe_0x4() after topo_probe_0xb() if cpu_cores is still 0. I think this is a better fallback procedure. With this patch, cpu_cores gets the value 4 which is the correct one, finally: [...] I think that your addition achieves this effect, perhaps just not as explicitly as I would preferred. Jung-uk, what do you think? Yes, you're right. Please try new patch: http://people.freebsd.org/~jkim/mp_machdep2.diff Thank you! I will have access to that particular machine on Monday again, so testing the new patch will have to wait until Monday. But from looking at your patch it should have the same result as my simpler patch, so it should work fine. As expected, the patch works perfectly fine. The cores are detected correctly: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) kern.sched.topology_spec: groups group level=1 cache-level=0 cpu count=8 mask=0xff0, 1, 2, 3, 4, 5, 6, 7/cpu flags/flags children group level=3 cache-level=2 cpu count=4 mask=0xf0, 1, 2, 3/cpu flags/flags /group group level=3 cache-level=2 cpu count=4 mask=0xf04, 5, 6, 7/cpu flags/flags /group /children /group /groups Thanks for fixing this! Best regards Oliver -- Oliver Fromme, secnetix GmbH Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd Perl is worse than Python because people wanted it worse. -- Larry Wall ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
David Xu wrote: Do you have patch for i386 branch ? I want to test. On my Pentium-D machine: $ sysctl kern.sched.topology_spec kern.sched.topology_spec: groups group level=1 cache-level=0 cpu count=2 mask=0x30, 1/cpu flags/flags children group level=3 cache-level=2 cpu count=2 mask=0x30, 1/cpu flags/flags /group /children /group /groups it seems the kernel thinks that the Pentuim-D is sharing L2 cache, but in fact, the design of the Pentium Ds was simply two P4 cores sitting side by side. They do not share anything and they basically work independently. Same thing on my intel Atom 330 board at home: groups group level=1 cache-level=0 cpu count=4 mask=0xf0, 1, 2, 3/cpu flags/flags children group level=3 cache-level=2 cpu count=4 mask=0xf0, 1, 2, 3/cpu flags/flags children group level=5 cache-level=1 cpu count=2 mask=0x30, 1/cpu flagsflag name=HTTHTT group/flag /flags /group group level=5 cache-level=1 cpu count=2 mask=0xc2, 3/cpu flagsflag name=HTTHTT group/flag /flags /group /children /group /children /group /groups The intel Atom 330 consists of two Diamondville dies on one package, each with its own 512 KB of L2 cache. There is no L3 cache, or any other shared cache. (Or maybe I misinterpret the XML output; I think that the level and cache-level numbers are confusing.) BTW, I noticed that the indentation problem (newline before /flags) is already fixed in -current, 5 weeks ago. Any plan to MFC this? Best regards Oliver -- Oliver Fromme, secnetix GmbH Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd Python tricks is a tough one, cuz the language is so clean. E.g., C makes an art of confusing pointers with arrays and strings, which leads to lotsa neat pointer tricks; APL mistakes everything for an array, leading to neat one-liners; and Perl confuses everything period, making each line a joyous adventure wink. -- Tim Peters ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On Thu, Jul 15, 2010 at 10:01:48PM +0200, Oliver Fromme wrote: Jung-uk Kim wrote: On Thursday 15 July 2010 01:56 pm, Andriy Gapon wrote: on 15/07/2010 19:57 Oliver Fromme said the following: I patched topo_probe() so it calls topo_probe_0x4() after topo_probe_0xb() if cpu_cores is still 0. I think this is a better fallback procedure. With this patch, cpu_cores gets the value 4 which is the correct one, finally: [...] I think that your addition achieves this effect, perhaps just not as explicitly as I would preferred. Jung-uk, what do you think? Yes, you're right. Please try new patch: http://people.freebsd.org/~jkim/mp_machdep2.diff Thank you! I will have access to that particular machine on Monday again, so testing the new patch will have to wait until Monday. But from looking at your patch it should have the same result as my simpler patch, so it should work fine. I have a general question for everyone involved in this thread (which is highly educational/interesting -- thank you for all the info!): Does the problem reported affect actual performance/behaviour of FreeBSD kernel-wise at all, or is it just a cosmetical issue with regards to showing how many cores/threads there are? -- | Jeremy Chadwick j...@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On 15 July 2010 23:18, Jung-uk Kim j...@freebsd.org wrote: On Thursday 15 July 2010 03:07 pm, Jung-uk Kim wrote: On Thursday 15 July 2010 01:56 pm, Andriy Gapon wrote: on 15/07/2010 19:57 Oliver Fromme said the following: In topo_probe(), cpu_high is 0xd, so topo_probe_0xb() is called. But the cpuid 0xb instruction doesn't seem to return useful data: All values are zero already in the first level, so cpu_cores remains 0. Back in topo_probe(), there is a fallback if cpu_cores is stil 0: It assigns mp_ncpu to cpu_cores, so it gets 8 which is wrong. I patched topo_probe() so it calls topo_probe_0x4() after topo_probe_0xb() if cpu_cores is still 0. I think this is a better fallback procedure. With this patch, cpu_cores gets the value 4 which is the correct one, finally: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) Thank you for debugging this issue! Not sure if this is the best patch that there can be, but its direction is definitely correct. As the Intel document says (translated to our x86 mp_machdep.c terms): if cpu_high = 0xb then we should execute cpuid_count(0xb, 0, p) and examine EBX value (p[1]), only if it's non-zero should we proceed with topo_probe_0xb(), otherwise we should fall back to topo_probe_0x4, etc. I think that your addition achieves this effect, perhaps just not as explicitly as I would preferred. Jung-uk, what do you think? Yes, you're right. Please try new patch: http://people.freebsd.org/~jkim/mp_machdep2.diff I uploaded the patch again, it's compile-tested this now. Just tried with the patch against 8.1-rc2. 2x E5520 - OK, no changes FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) x 2 SMT threads 2x E5440 - now OK FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) 1x 5050 - OK, no changes FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 HTT threads -- wbr, pluknet ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On Thursday 15 July 2010 09:34 pm, David Xu wrote: Jung-uk Kim wrote: On Wednesday 14 July 2010 05:40 pm, Jung-uk Kim wrote: On Wednesday 14 July 2010 01:31 pm, Andriy Gapon wrote: on 14/07/2010 17:14 Oliver Fromme said the following: In a machine installed yesterday, 8.1-PRERELEASE doesn't seem to detect the number of CPU packages vs. cores per package correctly: | FreeBSD 8.1-PRERELEASE-20100713 #0: Tue Jul 13 19:51:18 UTC | 2010 [...] | CPU: Intel(R) Xeon(R) CPU L5408 @ 2.13GHz | (2133.42-MHz K8-class CPU) Origin = GenuineIntel Id = | 0x1067a Family = 6 Model = 17 Stepping = 10 | Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC |, SE | P,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE |, SSE 2,SS,HTT,TM,PBE | Features2=0x40ce3bdSSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE |3 ,C X16,xTPR,PDCM,DCA,SSE4.1,XSAVE AMD | Features=0x2800SYSCALL,LM | AMD Features2=0x1LAHF | TSC: P-state invariant | real memory = 34359738368 (32768 MB) | avail memory = 33151377408 (31615 MB) | ACPI APIC Table: IBMSERBLADE | FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs | FreeBSD/SMP: 1 package(s) x 8 core(s) | cpu0 (BSP): APIC ID: 0 | cpu1 (AP): APIC ID: 1 | cpu2 (AP): APIC ID: 2 | cpu3 (AP): APIC ID: 3 | cpu4 (AP): APIC ID: 4 | cpu5 (AP): APIC ID: 5 | cpu6 (AP): APIC ID: 6 | cpu7 (AP): APIC ID: 7 | ioapic1 Version 2.0 irqs 24-47 on motherboard | ioapic0 Version 2.0 irqs 0-23 on motherboard I'm pretty sure that this is a 2 x 4 machine (2 CPU packages with 4 cores per package), not 1 x 8. That's what the BIOS displays during POST. I'm not sure if this is just a cosmetic issue, or if this is a critical thing ... I could imagine that performance might be sub-optimal if the CPU topology isn't detected correctly, but I'm not sure if FreeBSD can take advantage of the topology. Could you please try to do the following? 1. Fetch topo-12212009.tar from the top of this page: http://software.intel.com/en-us/articles/intel-64-architecture- pr oc essor-topology-enumeration/ 2. Untar it and apply this patch to the code: http://people.freebsd.org/~avg/cpu-topology.diff 3. Compile it by running sh mk_64.sh (supposing you have amd64 system installed) 4. Run cpu_topology64.out and report back its output. It's funny that I actually wrote a convenience script (and cleaned up today): http://people.freebsd.org/~jkim/cpu_topology-12212009.sh BTW, current topology detection code is not optimal for some Intel processors if my memory serves. Surprisingly, I found a patch I wrote last year from /tmp of my desktop and it is still applied cleanly. 8-) Just in case, it is available from here: http://people.freebsd.org/~jkim/mp_machdep.diff It is applicable to both sys/amd64/amd64/mp_machdep.c and sys/i386/i386/mp_machdep.c. I have to warn you, though, it hasn't been used for more than a year, so it may not work at all. ;-) Jung-uk Kim Do you have patch for i386 branch ? I want to test. The patch should apply fine on both sys/amd64/amd64/mp_machdep.c and sys/i386/i386/mp_machdep.c. http://people.freebsd.org/~jkim/mp_machdep2.diff On my Pentium-D machine: $ sysctl kern.sched.topology_spec kern.sched.topology_spec: groups group level=1 cache-level=0 cpu count=2 mask=0x30, 1/cpu flags/flags children group level=3 cache-level=2 cpu count=2 mask=0x30, 1/cpu flags/flags /group /children /group /groups it seems the kernel thinks that the Pentuim-D is sharing L2 cache, but in fact, the design of the Pentium Ds was simply two P4 cores sitting side by side. They do not share anything and they basically work independently. cpu_topo() in mp_machdep.c is not that smart to distinguish whether two cores in one package is sharing L2 cache or not: /* * Only HTT no multi-core. */ if (cpu_logical 1 cpu_cores == 1) return (smp_topo_1level(CG_SHARE_L1, cpu_logical, cg_flags)); /* * Only multi-core no HTT. */ if (cpu_cores 1 cpu_logical == 1) return (smp_topo_1level(CG_SHARE_L2, cpu_cores, cg_flags)); /* * Both HTT and multi-core. */ return (smp_topo_2level(CG_SHARE_L2, cpu_cores, CG_SHARE_L1, cpu_logical, cg_flags)); In other words, if one package contains multiple cores, it is *assumed* sharing L2 cache. The same is true for HTT/SMT case (i.e., assumed sharing L1 cache). It does not care about L3 cache at all. Yes, we need some improvement in this area. Jung-uk Kim ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On Friday 16 July 2010 03:55 am, Jeremy Chadwick wrote: On Thu, Jul 15, 2010 at 10:01:48PM +0200, Oliver Fromme wrote: Jung-uk Kim wrote: On Thursday 15 July 2010 01:56 pm, Andriy Gapon wrote: on 15/07/2010 19:57 Oliver Fromme said the following: I patched topo_probe() so it calls topo_probe_0x4() after topo_probe_0xb() if cpu_cores is still 0. I think this is a better fallback procedure. With this patch, cpu_cores gets the value 4 which is the correct one, finally: [...] I think that your addition achieves this effect, perhaps just not as explicitly as I would preferred. Jung-uk, what do you think? Yes, you're right. Please try new patch: http://people.freebsd.org/~jkim/mp_machdep2.diff Thank you! I will have access to that particular machine on Monday again, so testing the new patch will have to wait until Monday. But from looking at your patch it should have the same result as my simpler patch, so it should work fine. I have a general question for everyone involved in this thread (which is highly educational/interesting -- thank you for all the info!): Does the problem reported affect actual performance/behaviour of FreeBSD kernel-wise at all, or is it just a cosmetical issue with regards to showing how many cores/threads there are? Theoretically there is behavioral changes from scheduler. jeff@ should be able to tell you more about this. Jung-uk Kim ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On 14 July 2010 18:14, Oliver Fromme o...@lurza.secnetix.de wrote: In a machine installed yesterday, 8.1-PRERELEASE doesn't seem to detect the number of CPU packages vs. cores per package correctly: | FreeBSD 8.1-PRERELEASE-20100713 #0: Tue Jul 13 19:51:18 UTC 2010 | [...] | CPU: Intel(R) Xeon(R) CPU L5408 @ 2.13GHz (2133.42-MHz K8-class CPU) | Origin = GenuineIntel Id = 0x1067a Family = 6 Model = 17 Stepping = 10 | Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE | Features2=0x40ce3bdSSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,XSAVE | AMD Features=0x2800SYSCALL,LM | AMD Features2=0x1LAHF | TSC: P-state invariant | real memory = 34359738368 (32768 MB) | avail memory = 33151377408 (31615 MB) | ACPI APIC Table: IBM SERBLADE | FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs | FreeBSD/SMP: 1 package(s) x 8 core(s) Just for the reference, I collected CPU detection from various branches. 6.4 Cores per package: 4 FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs 7.3 Cores per package: 4 FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs 8.1-rc FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 1 package(s) x 8 core(s) Indeed, looks like a regression. -- wbr, pluknet ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
Andriy Gapon wrote: Could you please try to do the following? 1. Fetch topo-12212009.tar from the top of this page: http://software.intel.com/en-us/articles/intel-64-architecture-processor-topology-enumeration/ 2. Untar it and apply this patch to the code: http://people.freebsd.org/~avg/cpu-topology.diff 3. Compile it by running sh mk_64.sh (supposing you have amd64 system installed) 4. Run cpu_topology64.out and report back its output. Ok, did that. This tool seems to get it right, i.e. it detects 2 packages with 4 cores each. Here's the output: Software visible enumeration in the system: Number of logical processors visible to the OS: 8 Number of logical processors visible to this process: 8 Number of processor cores visible to this process: 8 Number of physical packages visible to this process: 2 Hierarchical counts by levels of processor topology: # of cores in package 0 visible to this process: 4 . # of cores in package 1 visible to this process: 4 . Affinity masks per SMT thread, per core, per package: Individual: P:0, C:0, T:0 -- 1 Core-aggregated: P:0, C:0 -- 1 Individual: P:0, C:1, T:0 -- 2 Core-aggregated: P:0, C:1 -- 2 Individual: P:0, C:2, T:0 -- 4 Core-aggregated: P:0, C:2 -- 4 Individual: P:0, C:3, T:0 -- 8 Core-aggregated: P:0, C:3 -- 8 Pkg-aggregated: P:0 -- f Individual: P:1, C:0, T:0 -- 10 Core-aggregated: P:1, C:0 -- 10 Individual: P:1, C:1, T:0 -- 20 Core-aggregated: P:1, C:1 -- 20 Individual: P:1, C:2, T:0 -- 40 Core-aggregated: P:1, C:2 -- 40 Individual: P:1, C:3, T:0 -- 80 Core-aggregated: P:1, C:3 -- 80 Pkg-aggregated: P:1 -- f0 APIC ID listings from affinity masks OS cpu 0, Affinity mask 0001 - apic id 0 OS cpu 1, Affinity mask 0002 - apic id 1 OS cpu 2, Affinity mask 0004 - apic id 2 OS cpu 3, Affinity mask 0008 - apic id 3 OS cpu 4, Affinity mask 0010 - apic id 4 OS cpu 5, Affinity mask 0020 - apic id 5 OS cpu 6, Affinity mask 0040 - apic id 6 OS cpu 7, Affinity mask 0080 - apic id 7 Package 0 Cache and Thread details Box Description: Cache is cache level designator Size is cache size OScpu# is cpu # as seen by OS Core is core#[_thread# if 1 thread/core] inside socket AffMsk is AffinityMask(extended hex) for core and thread CmbMsk is Combined AffinityMask(extended hex) for hw threads sharing cache CmbMsk will differ from AffMsk if 1 hw_thread/cache Extended Hex replaces trailing zeroes with 'z#' where # is number of zeroes (so '8z5' is '0x80') L1D is Level 1 Data cache, size(KBytes)= 32, Cores/cache= 1, Caches/package= 4 L1I is Level 1 Instruction cache, size(KBytes)= 32, Cores/cache= 1, Caches/package= 4 L2 is Level 2 Unified cache, size(KBytes)= 6144, Cores/cache= 2, Caches/package= 2 +++++ Cache | L1D| L1D| L1D| L1D| Size | 32K| 32K| 32K| 32K| OScpu#| 0| 1| 2| 3| Core | c0| c1| c2| c3| AffMsk| 1| 2| 4| 8| +++++ Cache | L1I| L1I| L1I| L1I| Size | 32K| 32K| 32K| 32K| +++++ Cache | L2 | L2 | Size | 6M | 6M | CmbMsk| 3 | c | +-+-+ Combined socket AffinityMask= 0xf Package 1 Cache and Thread details Box Description: Cache is cache level designator Size is cache size OScpu# is cpu # as seen by OS Core is core#[_thread# if 1 thread/core] inside socket AffMsk is AffinityMask(extended hex) for core and thread CmbMsk is Combined AffinityMask(extended hex) for hw threads sharing cache CmbMsk will differ from AffMsk if 1 hw_thread/cache Extended Hex replaces trailing zeroes with 'z#' where # is number of zeroes (so '8z5' is '0x80') +++++ Cache | L1D| L1D| L1D| L1D| Size | 32K| 32K| 32K| 32K| OScpu#| 4| 5| 6| 7| Core | c0| c1| c2| c3| AffMsk| 10| 20| 40| 80| +++++ Cache | L1I| L1I| L1I| L1I| Size | 32K| 32K| 32K| 32K| +++++ Cache | L2 | L2 | Size | 6M | 6M | CmbMsk| 30 | c0 | +-+-+ Combined socket AffinityMask= 0xf0 -- Oliver Fromme, secnetix GmbH Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd Clear perl code is better than unclear awk code; but NOTHING comes close to unclear perl code (taken from comp.lang.awk FAQ) ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send
Re: 8.1-PRERELEASE: CPU packages not detected correctly
on 15/07/2010 14:58 Oliver Fromme said the following: Andriy Gapon wrote: Could you please try to do the following? 1. Fetch topo-12212009.tar from the top of this page: http://software.intel.com/en-us/articles/intel-64-architecture-processor-topology-enumeration/ 2. Untar it and apply this patch to the code: http://people.freebsd.org/~avg/cpu-topology.diff 3. Compile it by running sh mk_64.sh (supposing you have amd64 system installed) 4. Run cpu_topology64.out and report back its output. Ok, did that. This tool seems to get it right, i.e. it detects 2 packages with 4 cores each. Here's the output: Great! So perhaps you can now test jkim's patch posted in this thread? -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
Andriy Gapon wrote: on 15/07/2010 14:58 Oliver Fromme said the following: Andriy Gapon wrote: Could you please try to do the following? 1. Fetch topo-12212009.tar from the top of this page: http://software.intel.com/en-us/articles/intel-64-architecture-processor-topology-enumeration/ 2. Untar it and apply this patch to the code: http://people.freebsd.org/~avg/cpu-topology.diff 3. Compile it by running sh mk_64.sh (supposing you have amd64 system installed) 4. Run cpu_topology64.out and report back its output. Ok, did that. This tool seems to get it right, i.e. it detects 2 packages with 4 cores each. Here's the output: Great! So perhaps you can now test jkim's patch posted in this thread? Sure, the kernel is being compiled right now ... -- Oliver Fromme, secnetix GmbH Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd One Unix to rule them all, One Resolver to find them, One IP to bring them all and in the zone to bind them. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
Andriy Gapon wrote: on 15/07/2010 14:58 Oliver Fromme said the following: Andriy Gapon wrote: Could you please try to do the following? 1. Fetch topo-12212009.tar from the top of this page: http://software.intel.com/en-us/articles/intel-64-architecture-processor-topology-enumeration/ 2. Untar it and apply this patch to the code: http://people.freebsd.org/~avg/cpu-topology.diff 3. Compile it by running sh mk_64.sh (supposing you have amd64 system installed) 4. Run cpu_topology64.out and report back its output. Ok, did that. This tool seems to get it right, i.e. it detects 2 packages with 4 cores each. Here's the output: Great! So perhaps you can now test jkim's patch posted in this thread? Unfortunately, it didn't change. Kernel output during boot is still the same; it displays 1 package x 8 cores. Best regards Oliver PS: make kernel -j 10 took exactly 8 minutes on this box. -- Oliver Fromme, secnetix GmbH Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd With sufficient thrust, pigs fly just fine. However, this is not necessarily a good idea. It is hard to be sure where they are going to land, and it could be dangerous sitting under them as they fly overhead. -- RFC 1925 ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
on 15/07/2010 15:27 Oliver Fromme said the following: Unfortunately, it didn't change. Kernel output during boot is still the same; it displays 1 package x 8 cores. If you are sure that everything is done correctly (patch really applied, kernel really rebuilt and reinstalled, and reboot was to the new kernel), then perhaps jkim would be interested in this issue. -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
Andriy Gapon wrote:
on 15/07/2010 15:27 Oliver Fromme said the following:
Unfortunately, it didn't change. Kernel output during boot
is still the same; it displays 1 package x 8 cores.
If you are sure that everything is done correctly (patch really applied,
kernel
really rebuilt and reinstalled, and reboot was to the new kernel), then
perhaps
jkim would be interested in this issue.
Yes, I'm sure, I even added some printf() so I can be sure
that the patched code is actually executed.
Here is what happens in sys/amd64/amd64/mp_machdep.c:
In topo_probe(), cpu_high is 0xd, so topo_probe_0xb() is
called. But the cpuid 0xb instruction doesn't seem to
return useful data: All values are zero already in the
first level, so cpu_cores remains 0.
Back in topo_probe(), there is a fallback if cpu_cores is
stil 0: It assigns mp_ncpu to cpu_cores, so it gets 8
which is wrong.
I patched topo_probe() so it calls topo_probe_0x4() after
topo_probe_0xb() if cpu_cores is still 0. I think this
is a better fallback procedure. With this patch, cpu_cores
gets the value 4 which is the correct one, finally:
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 2 package(s) x 4 core(s)
This is the patch, it's against jkim's patched version:
@@ -265,7 +268,7 @@
else if (cpu_vendor_id == CPU_VENDOR_INTEL) {
if (cpu_high = 0xb)
topo_probe_0xb();
- else if (cpu_high = 0x4)
+ if (cpu_high = 0x4 cpu_cores == 0)
topo_probe_0x4();
}
if (cpu_cores == 0)
Best regards
Oliver
--
Oliver Fromme, secnetix GmbH Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart
FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd
With sufficient thrust, pigs fly just fine. However, this
is not necessarily a good idea. It is hard to be sure where
they are going to land, and it could be dangerous sitting
under them as they fly overhead. -- RFC 1925
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
on 15/07/2010 19:57 Oliver Fromme said the following:
In topo_probe(), cpu_high is 0xd, so topo_probe_0xb() is
called. But the cpuid 0xb instruction doesn't seem to
return useful data: All values are zero already in the
first level, so cpu_cores remains 0.
Back in topo_probe(), there is a fallback if cpu_cores is
stil 0: It assigns mp_ncpu to cpu_cores, so it gets 8
which is wrong.
I patched topo_probe() so it calls topo_probe_0x4() after
topo_probe_0xb() if cpu_cores is still 0. I think this
is a better fallback procedure. With this patch, cpu_cores
gets the value 4 which is the correct one, finally:
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 2 package(s) x 4 core(s)
Thank you for debugging this issue!
Not sure if this is the best patch that there can be, but its direction is
definitely correct.
As the Intel document says (translated to our x86 mp_machdep.c terms):
if cpu_high = 0xb then we should execute cpuid_count(0xb, 0, p) and examine EBX
value (p[1]), only if it's non-zero should we proceed with topo_probe_0xb(),
otherwise we should fall back to topo_probe_0x4, etc.
I think that your addition achieves this effect, perhaps just not as explicitly
as
I would preferred.
Jung-uk, what do you think?
This is the patch, it's against jkim's patched version:
@@ -265,7 +268,7 @@
else if (cpu_vendor_id == CPU_VENDOR_INTEL) {
if (cpu_high = 0xb)
topo_probe_0xb();
- else if (cpu_high = 0x4)
+ if (cpu_high = 0x4 cpu_cores == 0)
topo_probe_0x4();
}
if (cpu_cores == 0)
--
Andriy Gapon
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On Thursday, July 15, 2010 1:56:11 pm Andriy Gapon wrote: on 15/07/2010 19:57 Oliver Fromme said the following: In topo_probe(), cpu_high is 0xd, so topo_probe_0xb() is called. But the cpuid 0xb instruction doesn't seem to return useful data: All values are zero already in the first level, so cpu_cores remains 0. Back in topo_probe(), there is a fallback if cpu_cores is stil 0: It assigns mp_ncpu to cpu_cores, so it gets 8 which is wrong. I patched topo_probe() so it calls topo_probe_0x4() after topo_probe_0xb() if cpu_cores is still 0. I think this is a better fallback procedure. With this patch, cpu_cores gets the value 4 which is the correct one, finally: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) Thank you for debugging this issue! Not sure if this is the best patch that there can be, but its direction is definitely correct. As the Intel document says (translated to our x86 mp_machdep.c terms): if cpu_high = 0xb then we should execute cpuid_count(0xb, 0, p) and examine EBX value (p[1]), only if it's non-zero should we proceed with topo_probe_0xb(), otherwise we should fall back to topo_probe_0x4, etc. I think that your addition achieves this effect, perhaps just not as explicitly as I would preferred. Maybe have topo_probe_0xb() explicitly call topo_probe_0x4() if EBX is 0? -- John Baldwin ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On Thursday 15 July 2010 01:56 pm, Andriy Gapon wrote:
on 15/07/2010 19:57 Oliver Fromme said the following:
In topo_probe(), cpu_high is 0xd, so topo_probe_0xb() is
called. But the cpuid 0xb instruction doesn't seem to
return useful data: All values are zero already in the
first level, so cpu_cores remains 0.
Back in topo_probe(), there is a fallback if cpu_cores is
stil 0: It assigns mp_ncpu to cpu_cores, so it gets 8
which is wrong.
I patched topo_probe() so it calls topo_probe_0x4() after
topo_probe_0xb() if cpu_cores is still 0. I think this
is a better fallback procedure. With this patch, cpu_cores
gets the value 4 which is the correct one, finally:
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 2 package(s) x 4 core(s)
Thank you for debugging this issue!
Not sure if this is the best patch that there can be, but its
direction is definitely correct.
As the Intel document says (translated to our x86 mp_machdep.c
terms): if cpu_high = 0xb then we should execute cpuid_count(0xb,
0, p) and examine EBX value (p[1]), only if it's non-zero should we
proceed with topo_probe_0xb(), otherwise we should fall back to
topo_probe_0x4, etc.
I think that your addition achieves this effect, perhaps just not
as explicitly as I would preferred.
Jung-uk, what do you think?
Yes, you're right. Please try new patch:
http://people.freebsd.org/~jkim/mp_machdep2.diff
Thanks,
Jung-uk Kim
This is the patch, it's against jkim's patched version:
@@ -265,7 +268,7 @@
else if (cpu_vendor_id == CPU_VENDOR_INTEL) {
if (cpu_high = 0xb)
topo_probe_0xb();
- else if (cpu_high = 0x4)
+ if (cpu_high = 0x4 cpu_cores == 0)
topo_probe_0x4();
}
if (cpu_cores == 0)
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On Thursday 15 July 2010 03:07 pm, Jung-uk Kim wrote: On Thursday 15 July 2010 01:56 pm, Andriy Gapon wrote: on 15/07/2010 19:57 Oliver Fromme said the following: In topo_probe(), cpu_high is 0xd, so topo_probe_0xb() is called. But the cpuid 0xb instruction doesn't seem to return useful data: All values are zero already in the first level, so cpu_cores remains 0. Back in topo_probe(), there is a fallback if cpu_cores is stil 0: It assigns mp_ncpu to cpu_cores, so it gets 8 which is wrong. I patched topo_probe() so it calls topo_probe_0x4() after topo_probe_0xb() if cpu_cores is still 0. I think this is a better fallback procedure. With this patch, cpu_cores gets the value 4 which is the correct one, finally: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) Thank you for debugging this issue! Not sure if this is the best patch that there can be, but its direction is definitely correct. As the Intel document says (translated to our x86 mp_machdep.c terms): if cpu_high = 0xb then we should execute cpuid_count(0xb, 0, p) and examine EBX value (p[1]), only if it's non-zero should we proceed with topo_probe_0xb(), otherwise we should fall back to topo_probe_0x4, etc. I think that your addition achieves this effect, perhaps just not as explicitly as I would preferred. Jung-uk, what do you think? Yes, you're right. Please try new patch: http://people.freebsd.org/~jkim/mp_machdep2.diff I uploaded the patch again, it's compile-tested this now. Sorry, if anyone has downloaded it few minutes ago. Jung-uk Kim ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
Jung-uk Kim wrote: On Thursday 15 July 2010 01:56 pm, Andriy Gapon wrote: on 15/07/2010 19:57 Oliver Fromme said the following: I patched topo_probe() so it calls topo_probe_0x4() after topo_probe_0xb() if cpu_cores is still 0. I think this is a better fallback procedure. With this patch, cpu_cores gets the value 4 which is the correct one, finally: [...] I think that your addition achieves this effect, perhaps just not as explicitly as I would preferred. Jung-uk, what do you think? Yes, you're right. Please try new patch: http://people.freebsd.org/~jkim/mp_machdep2.diff Thank you! I will have access to that particular machine on Monday again, so testing the new patch will have to wait until Monday. But from looking at your patch it should have the same result as my simpler patch, so it should work fine. Best regards Oliver -- Oliver Fromme, secnetix GmbH Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd C++ is the only current language making COBOL look good. -- Bertrand Meyer ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
Jung-uk Kim wrote: On Wednesday 14 July 2010 05:40 pm, Jung-uk Kim wrote: On Wednesday 14 July 2010 01:31 pm, Andriy Gapon wrote: on 14/07/2010 17:14 Oliver Fromme said the following: In a machine installed yesterday, 8.1-PRERELEASE doesn't seem to detect the number of CPU packages vs. cores per package correctly: | FreeBSD 8.1-PRERELEASE-20100713 #0: Tue Jul 13 19:51:18 UTC | 2010 [...] | CPU: Intel(R) Xeon(R) CPU L5408 @ 2.13GHz | (2133.42-MHz K8-class CPU) Origin = GenuineIntel Id = | 0x1067a Family = 6 Model = 17 Stepping = 10 | Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC, |SE | P,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE, |SSE 2,SS,HTT,TM,PBE | Features2=0x40ce3bdSSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3 |,C X16,xTPR,PDCM,DCA,SSE4.1,XSAVE AMD | Features=0x2800SYSCALL,LM | AMD Features2=0x1LAHF | TSC: P-state invariant | real memory = 34359738368 (32768 MB) | avail memory = 33151377408 (31615 MB) | ACPI APIC Table: IBMSERBLADE | FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs | FreeBSD/SMP: 1 package(s) x 8 core(s) | cpu0 (BSP): APIC ID: 0 | cpu1 (AP): APIC ID: 1 | cpu2 (AP): APIC ID: 2 | cpu3 (AP): APIC ID: 3 | cpu4 (AP): APIC ID: 4 | cpu5 (AP): APIC ID: 5 | cpu6 (AP): APIC ID: 6 | cpu7 (AP): APIC ID: 7 | ioapic1 Version 2.0 irqs 24-47 on motherboard | ioapic0 Version 2.0 irqs 0-23 on motherboard I'm pretty sure that this is a 2 x 4 machine (2 CPU packages with 4 cores per package), not 1 x 8. That's what the BIOS displays during POST. I'm not sure if this is just a cosmetic issue, or if this is a critical thing ... I could imagine that performance might be sub-optimal if the CPU topology isn't detected correctly, but I'm not sure if FreeBSD can take advantage of the topology. Could you please try to do the following? 1. Fetch topo-12212009.tar from the top of this page: http://software.intel.com/en-us/articles/intel-64-architecture-pr oc essor-topology-enumeration/ 2. Untar it and apply this patch to the code: http://people.freebsd.org/~avg/cpu-topology.diff 3. Compile it by running sh mk_64.sh (supposing you have amd64 system installed) 4. Run cpu_topology64.out and report back its output. It's funny that I actually wrote a convenience script (and cleaned up today): http://people.freebsd.org/~jkim/cpu_topology-12212009.sh BTW, current topology detection code is not optimal for some Intel processors if my memory serves. Surprisingly, I found a patch I wrote last year from /tmp of my desktop and it is still applied cleanly. 8-) Just in case, it is available from here: http://people.freebsd.org/~jkim/mp_machdep.diff It is applicable to both sys/amd64/amd64/mp_machdep.c and sys/i386/i386/mp_machdep.c. I have to warn you, though, it hasn't been used for more than a year, so it may not work at all. ;-) Jung-uk Kim Do you have patch for i386 branch ? I want to test. On my Pentium-D machine: $ sysctl kern.sched.topology_spec kern.sched.topology_spec: groups group level=1 cache-level=0 cpu count=2 mask=0x30, 1/cpu flags/flags children group level=3 cache-level=2 cpu count=2 mask=0x30, 1/cpu flags/flags /group /children /group /groups it seems the kernel thinks that the Pentuim-D is sharing L2 cache, but in fact, the design of the Pentium Ds was simply two P4 cores sitting side by side. They do not share anything and they basically work independently. Regards, David Xu ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
8.1-PRERELEASE: CPU packages not detected correctly
In a machine installed yesterday, 8.1-PRERELEASE doesn't seem to detect the number of CPU packages vs. cores per package correctly: | FreeBSD 8.1-PRERELEASE-20100713 #0: Tue Jul 13 19:51:18 UTC 2010 | [...] | CPU: Intel(R) Xeon(R) CPU L5408 @ 2.13GHz (2133.42-MHz K8-class CPU) | Origin = GenuineIntel Id = 0x1067a Family = 6 Model = 17 Stepping = 10 | Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE | Features2=0x40ce3bdSSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,XSAVE | AMD Features=0x2800SYSCALL,LM | AMD Features2=0x1LAHF | TSC: P-state invariant | real memory = 34359738368 (32768 MB) | avail memory = 33151377408 (31615 MB) | ACPI APIC Table: IBMSERBLADE | FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs | FreeBSD/SMP: 1 package(s) x 8 core(s) | cpu0 (BSP): APIC ID: 0 | cpu1 (AP): APIC ID: 1 | cpu2 (AP): APIC ID: 2 | cpu3 (AP): APIC ID: 3 | cpu4 (AP): APIC ID: 4 | cpu5 (AP): APIC ID: 5 | cpu6 (AP): APIC ID: 6 | cpu7 (AP): APIC ID: 7 | ioapic1 Version 2.0 irqs 24-47 on motherboard | ioapic0 Version 2.0 irqs 0-23 on motherboard I'm pretty sure that this is a 2 x 4 machine (2 CPU packages with 4 cores per package), not 1 x 8. That's what the BIOS displays during POST. I'm not sure if this is just a cosmetic issue, or if this is a critical thing ... I could imagine that performance might be sub-optimal if the CPU topology isn't detected correctly, but I'm not sure if FreeBSD can take advantage of the topology. Best regards Oliver -- Oliver Fromme, secnetix GmbH Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd A misleading benchmark test can accomplish in minutes what years of good engineering can never do. -- Dilbert (2009-03-02) ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On 14 July 2010 18:14, Oliver Fromme o...@lurza.secnetix.de wrote: In a machine installed yesterday, 8.1-PRERELEASE doesn't seem to detect the number of CPU packages vs. cores per package correctly: | FreeBSD 8.1-PRERELEASE-20100713 #0: Tue Jul 13 19:51:18 UTC 2010 | [...] | CPU: Intel(R) Xeon(R) CPU L5408 @ 2.13GHz (2133.42-MHz K8-class CPU) | Origin = GenuineIntel Id = 0x1067a Family = 6 Model = 17 Stepping = 10 | Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE | Features2=0x40ce3bdSSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,XSAVE | AMD Features=0x2800SYSCALL,LM | AMD Features2=0x1LAHF | TSC: P-state invariant | real memory = 34359738368 (32768 MB) | avail memory = 33151377408 (31615 MB) | ACPI APIC Table: IBM SERBLADE | FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs | FreeBSD/SMP: 1 package(s) x 8 core(s) | cpu0 (BSP): APIC ID: 0 | cpu1 (AP): APIC ID: 1 | cpu2 (AP): APIC ID: 2 | cpu3 (AP): APIC ID: 3 | cpu4 (AP): APIC ID: 4 | cpu5 (AP): APIC ID: 5 | cpu6 (AP): APIC ID: 6 | cpu7 (AP): APIC ID: 7 | ioapic1 Version 2.0 irqs 24-47 on motherboard | ioapic0 Version 2.0 irqs 0-23 on motherboard I'm pretty sure that this is a 2 x 4 machine (2 CPU packages with 4 cores per package), not 1 x 8. That's what the BIOS displays during POST. Hi, can you show kern.sched.topology_spec ? It would clarify things a bit. -- wbr, pluknet ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
pluknet pluk...@gmail.com wrote: On 14 July 2010 18:14, Oliver Fromme o...@lurza.secnetix.de wrote: In a machine installed yesterday, 8.1-PRERELEASE doesn't seem to detect the number of CPU packages vs. cores per package correctly: | FreeBSD 8.1-PRERELEASE-20100713 #0: Tue Jul 13 19:51:18 UTC 2010 | [...] | CPU: Intel(R) Xeon(R) CPU L5408 @ 2.13GHz (2133.42-MHz K8-class CPU) | Origin = GenuineIntel Id = 0x1067a Family = 6 Model = 17 Stepping = 10 | Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE | Features2=0x40ce3bdSSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,XSAVE | AMD Features=0x2800SYSCALL,LM | AMD Features2=0x1LAHF | TSC: P-state invariant | real memory = 34359738368 (32768 MB) | avail memory = 33151377408 (31615 MB) | ACPI APIC Table: IBM SERBLADE | FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs | FreeBSD/SMP: 1 package(s) x 8 core(s) | cpu0 (BSP): APIC ID: 0 | cpu1 (AP): APIC ID: 1 | cpu2 (AP): APIC ID: 2 | cpu3 (AP): APIC ID: 3 | cpu4 (AP): APIC ID: 4 | cpu5 (AP): APIC ID: 5 | cpu6 (AP): APIC ID: 6 | cpu7 (AP): APIC ID: 7 | ioapic1 Version 2.0 irqs 24-47 on motherboard | ioapic0 Version 2.0 irqs 0-23 on motherboard I'm pretty sure that this is a 2 x 4 machine (2 CPU packages with 4 cores per package), not 1 x 8. That's what the BIOS displays during POST. Hi, can you show kern.sched.topology_spec ? It would clarify things a bit. Sure: groups group level=1 cache-level=0 cpu count=8 mask=0xff0, 1, 2, 3, 4, 5, 6, 7/cpu flags/flags children group level=3 cache-level=2 cpu count=8 mask=0xff0, 1, 2, 3, 4, 5, 6, 7/cpu flags/flags /group /children /group /groups Best regards Oliver -- Oliver Fromme, secnetix GmbH Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd Blogging: Never before have so many people with so little to say said so much to so few. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On Wed, Jul 14, 2010 at 04:44:05PM +0200, Oliver Fromme wrote: pluknet pluk...@gmail.com wrote: On 14 July 2010 18:14, Oliver Fromme o...@lurza.secnetix.de wrote: In a machine installed yesterday, 8.1-PRERELEASE doesn't seem to detect the number of CPU packages vs. cores per package correctly: | FreeBSD 8.1-PRERELEASE-20100713 #0: Tue Jul 13 19:51:18 UTC 2010 | [...] | CPU: Intel(R) Xeon(R) CPU L5408 @ 2.13GHz (2133.42-MHz K8-class CPU) | Origin = GenuineIntel Id = 0x1067a Family = 6 Model = 17 Stepping = 10 | Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE | Features2=0x40ce3bdSSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,XSAVE | AMD Features=0x2800SYSCALL,LM | AMD Features2=0x1LAHF | TSC: P-state invariant | real memory = 34359738368 (32768 MB) | avail memory = 33151377408 (31615 MB) | ACPI APIC Table: IBM SERBLADE | FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs | FreeBSD/SMP: 1 package(s) x 8 core(s) | cpu0 (BSP): APIC ID: 0 | cpu1 (AP): APIC ID: 1 | cpu2 (AP): APIC ID: 2 | cpu3 (AP): APIC ID: 3 | cpu4 (AP): APIC ID: 4 | cpu5 (AP): APIC ID: 5 | cpu6 (AP): APIC ID: 6 | cpu7 (AP): APIC ID: 7 | ioapic1 Version 2.0 irqs 24-47 on motherboard | ioapic0 Version 2.0 irqs 0-23 on motherboard I'm pretty sure that this is a 2 x 4 machine (2 CPU packages with 4 cores per package), not 1 x 8. That's what the BIOS displays during POST. Hi, can you show kern.sched.topology_spec ? It would clarify things a bit. Sure: groups group level=1 cache-level=0 cpu count=8 mask=0xff0, 1, 2, 3, 4, 5, 6, 7/cpu flags/flags children group level=3 cache-level=2 cpu count=8 mask=0xff0, 1, 2, 3, 4, 5, 6, 7/cpu flags/flags /group /children /group /groups Can you also provide the output of acpidump -dt? This will probably be quite long (possibly 300KB or more), so you may want to put it up on the web somewhere. -- | Jeremy Chadwick j...@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
Jeremy Chadwick wrote: Can you also provide the output of acpidump -dt? This will probably be quite long (possibly 300KB or more), so you may want to put it up on the web somewhere. http://www.secnetix.de/olli/tmp/acpidump-dt.txt Best regards Oliver -- Oliver Fromme, secnetix GmbH Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd 'Instead of asking why a piece of software is using 1970s technology, start asking why software is ignoring 30 years of accumulated wisdom.' ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
on 14/07/2010 17:14 Oliver Fromme said the following: In a machine installed yesterday, 8.1-PRERELEASE doesn't seem to detect the number of CPU packages vs. cores per package correctly: | FreeBSD 8.1-PRERELEASE-20100713 #0: Tue Jul 13 19:51:18 UTC 2010 | [...] | CPU: Intel(R) Xeon(R) CPU L5408 @ 2.13GHz (2133.42-MHz K8-class CPU) | Origin = GenuineIntel Id = 0x1067a Family = 6 Model = 17 Stepping = 10 | Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE | Features2=0x40ce3bdSSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,XSAVE | AMD Features=0x2800SYSCALL,LM | AMD Features2=0x1LAHF | TSC: P-state invariant | real memory = 34359738368 (32768 MB) | avail memory = 33151377408 (31615 MB) | ACPI APIC Table: IBMSERBLADE | FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs | FreeBSD/SMP: 1 package(s) x 8 core(s) | cpu0 (BSP): APIC ID: 0 | cpu1 (AP): APIC ID: 1 | cpu2 (AP): APIC ID: 2 | cpu3 (AP): APIC ID: 3 | cpu4 (AP): APIC ID: 4 | cpu5 (AP): APIC ID: 5 | cpu6 (AP): APIC ID: 6 | cpu7 (AP): APIC ID: 7 | ioapic1 Version 2.0 irqs 24-47 on motherboard | ioapic0 Version 2.0 irqs 0-23 on motherboard I'm pretty sure that this is a 2 x 4 machine (2 CPU packages with 4 cores per package), not 1 x 8. That's what the BIOS displays during POST. I'm not sure if this is just a cosmetic issue, or if this is a critical thing ... I could imagine that performance might be sub-optimal if the CPU topology isn't detected correctly, but I'm not sure if FreeBSD can take advantage of the topology. Could you please try to do the following? 1. Fetch topo-12212009.tar from the top of this page: http://software.intel.com/en-us/articles/intel-64-architecture-processor-topology-enumeration/ 2. Untar it and apply this patch to the code: http://people.freebsd.org/~avg/cpu-topology.diff 3. Compile it by running sh mk_64.sh (supposing you have amd64 system installed) 4. Run cpu_topology64.out and report back its output. Thanks! -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On Wednesday 14 July 2010 01:31 pm, Andriy Gapon wrote: on 14/07/2010 17:14 Oliver Fromme said the following: In a machine installed yesterday, 8.1-PRERELEASE doesn't seem to detect the number of CPU packages vs. cores per package correctly: | FreeBSD 8.1-PRERELEASE-20100713 #0: Tue Jul 13 19:51:18 UTC | 2010 [...] | CPU: Intel(R) Xeon(R) CPU L5408 @ 2.13GHz | (2133.42-MHz K8-class CPU) Origin = GenuineIntel Id = | 0x1067a Family = 6 Model = 17 Stepping = 10 | Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SE |P,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE |2,SS,HTT,TM,PBE | Features2=0x40ce3bdSSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,C |X16,xTPR,PDCM,DCA,SSE4.1,XSAVE AMD | Features=0x2800SYSCALL,LM | AMD Features2=0x1LAHF | TSC: P-state invariant | real memory = 34359738368 (32768 MB) | avail memory = 33151377408 (31615 MB) | ACPI APIC Table: IBMSERBLADE | FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs | FreeBSD/SMP: 1 package(s) x 8 core(s) | cpu0 (BSP): APIC ID: 0 | cpu1 (AP): APIC ID: 1 | cpu2 (AP): APIC ID: 2 | cpu3 (AP): APIC ID: 3 | cpu4 (AP): APIC ID: 4 | cpu5 (AP): APIC ID: 5 | cpu6 (AP): APIC ID: 6 | cpu7 (AP): APIC ID: 7 | ioapic1 Version 2.0 irqs 24-47 on motherboard | ioapic0 Version 2.0 irqs 0-23 on motherboard I'm pretty sure that this is a 2 x 4 machine (2 CPU packages with 4 cores per package), not 1 x 8. That's what the BIOS displays during POST. I'm not sure if this is just a cosmetic issue, or if this is a critical thing ... I could imagine that performance might be sub-optimal if the CPU topology isn't detected correctly, but I'm not sure if FreeBSD can take advantage of the topology. Could you please try to do the following? 1. Fetch topo-12212009.tar from the top of this page: http://software.intel.com/en-us/articles/intel-64-architecture-proc essor-topology-enumeration/ 2. Untar it and apply this patch to the code: http://people.freebsd.org/~avg/cpu-topology.diff 3. Compile it by running sh mk_64.sh (supposing you have amd64 system installed) 4. Run cpu_topology64.out and report back its output. It's funny that I actually wrote a convenience script (and cleaned up today): http://people.freebsd.org/~jkim/cpu_topology-12212009.sh BTW, current topology detection code is not optimal for some Intel processors if my memory serves. Jung-uk Kim ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
on 15/07/2010 00:40 Jung-uk Kim said the following: It's funny that I actually wrote a convenience script (and cleaned up today): http://people.freebsd.org/~jkim/cpu_topology-12212009.sh BTW, current topology detection code is not optimal for some Intel processors if my memory serves. Indeed. Your patch looks so much cleaner too. Thanks! -- Andriy Gapon ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: 8.1-PRERELEASE: CPU packages not detected correctly
On Wednesday 14 July 2010 05:40 pm, Jung-uk Kim wrote: On Wednesday 14 July 2010 01:31 pm, Andriy Gapon wrote: on 14/07/2010 17:14 Oliver Fromme said the following: In a machine installed yesterday, 8.1-PRERELEASE doesn't seem to detect the number of CPU packages vs. cores per package correctly: | FreeBSD 8.1-PRERELEASE-20100713 #0: Tue Jul 13 19:51:18 UTC | 2010 [...] | CPU: Intel(R) Xeon(R) CPU L5408 @ 2.13GHz | (2133.42-MHz K8-class CPU) Origin = GenuineIntel Id = | 0x1067a Family = 6 Model = 17 Stepping = 10 | Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC, |SE | P,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE, |SSE 2,SS,HTT,TM,PBE | Features2=0x40ce3bdSSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3 |,C X16,xTPR,PDCM,DCA,SSE4.1,XSAVE AMD | Features=0x2800SYSCALL,LM | AMD Features2=0x1LAHF | TSC: P-state invariant | real memory = 34359738368 (32768 MB) | avail memory = 33151377408 (31615 MB) | ACPI APIC Table: IBMSERBLADE | FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs | FreeBSD/SMP: 1 package(s) x 8 core(s) | cpu0 (BSP): APIC ID: 0 | cpu1 (AP): APIC ID: 1 | cpu2 (AP): APIC ID: 2 | cpu3 (AP): APIC ID: 3 | cpu4 (AP): APIC ID: 4 | cpu5 (AP): APIC ID: 5 | cpu6 (AP): APIC ID: 6 | cpu7 (AP): APIC ID: 7 | ioapic1 Version 2.0 irqs 24-47 on motherboard | ioapic0 Version 2.0 irqs 0-23 on motherboard I'm pretty sure that this is a 2 x 4 machine (2 CPU packages with 4 cores per package), not 1 x 8. That's what the BIOS displays during POST. I'm not sure if this is just a cosmetic issue, or if this is a critical thing ... I could imagine that performance might be sub-optimal if the CPU topology isn't detected correctly, but I'm not sure if FreeBSD can take advantage of the topology. Could you please try to do the following? 1. Fetch topo-12212009.tar from the top of this page: http://software.intel.com/en-us/articles/intel-64-architecture-pr oc essor-topology-enumeration/ 2. Untar it and apply this patch to the code: http://people.freebsd.org/~avg/cpu-topology.diff 3. Compile it by running sh mk_64.sh (supposing you have amd64 system installed) 4. Run cpu_topology64.out and report back its output. It's funny that I actually wrote a convenience script (and cleaned up today): http://people.freebsd.org/~jkim/cpu_topology-12212009.sh BTW, current topology detection code is not optimal for some Intel processors if my memory serves. Surprisingly, I found a patch I wrote last year from /tmp of my desktop and it is still applied cleanly. 8-) Just in case, it is available from here: http://people.freebsd.org/~jkim/mp_machdep.diff It is applicable to both sys/amd64/amd64/mp_machdep.c and sys/i386/i386/mp_machdep.c. I have to warn you, though, it hasn't been used for more than a year, so it may not work at all. ;-) Jung-uk Kim ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
