Re: [PATCH] fix dom0 builder TAKE 2(was Re: [Xen-ia64-devel] [PATCH] fix dom0 builder)
Hi Alex. If CONFIG_FLATMEM=y is enabled(xenLinux default is so), it seems sane that the total is about 5GB. Do disabling CONFIG_FLATMEM=n and enabling CONFIG_DISCONTIGMEM=y (or CONFIG_SPARSEMEM=y) make difference? And I also noticed that xenLinux default config disables CONFIG_VIRTUAL_MEM_MAP=n. Since dom0 may see underlying machine memory layout with the patch, should we revise default Linux config? Thanks. On Sun, Oct 15, 2006 at 10:35:42PM -0600, Alex Williamson wrote: On Mon, 2006-10-16 at 12:34 +0900, Isaku Yamahata wrote: Looks good and much better than mine. Thanks, unfortunately it doesn't behave very well. A zx1 box has memory from 0 - 1 GB, and the rest lives above 4GB. Therefore, if I boot with dom0_mem=5G, I should have 2GB of memory for dom0 (0-1G, 4-5G). Here's what the memory map looks like: (XEN) dom mem: type=13, attr=0x8008, range=[0x-0x1000) (4KB) (XEN) dom mem: type=10, attr=0x8008, range=[0x1000-0x2000) (4KB) (XEN) dom mem: type= 6, attr=0x8008, range=[0x2000-0x3000) (4KB) (XEN) dom mem: type= 7, attr=0x0008, range=[0x3000-0x000a) (628KB) (XEN) dom mem: type=11, attr=0x0003, range=[0x000a-0x000c) (128KB) (XEN) dom mem: type= 5, attr=0x8001, range=[0x000c-0x0010) (256KB) (XEN) dom mem: type= 7, attr=0x0008, range=[0x0010-0x0400) (63MB) (XEN) dom mem: type= 2, attr=0x0008, range=[0x0400-0x0813a000) (65MB) (XEN) dom mem: type= 7, attr=0x0008, range=[0x0813a000-0x3f5e4000) (884MB) (XEN) dom mem: type= 5, attr=0x8008, range=[0x3f5e4000-0x3fac) (4MB) (XEN) dom mem: type= 7, attr=0x0008, range=[0x3fb0-0x3fb08000) (32KB) (XEN) dom mem: type= 4, attr=0x0008, range=[0x3fb08000-0x3fb2c000) (144KB) (XEN) dom mem: type= 9, attr=0x0008, range=[0x3fb2c000-0x3fb38000) (48KB) (XEN) dom mem: type= 6, attr=0x8008, range=[0x3fb38000-0x4000) (4MB) (XEN) dom mem: type=11, attr=0x0001, range=[0x8000-0xfe00) (2016MB) (XEN) dom mem: type=11, attr=0x8001, range=[0xfed0-0x0001) (19MB) (XEN) dom mem: type= 7, attr=0x0008, range=[0x0001-0x00014000) (1024MB) (XEN) dom mem: type= 6, attr=0x8008, range=[0x00027fffe000-0x00028000) (8KB) (XEN) dom mem: type= 5, attr=0x8008, range=[0x0040ffd6a000-0x0040ffda6000) (240KB) (XEN) dom mem: type= 5, attr=0x8008, range=[0x0040ffe12000-0x0040ffe8) (440KB) (XEN) dom mem: type= 6, attr=0x8008, range=[0x0040fffc-0x0041) (256KB) (XEN) dom mem: type=11, attr=0x0001, range=[0x0800-0x1000) (8388608MB) (XEN) dom mem: type=12, attr=0x8001, range=[0x0003fc00-0x0004) (64MB) I see about 2GB of memory in there, we're ok so far. Dom0 boots up, and I see: Memory: 1948096k/2064384k available (10405k code, 115072k reserved, 4256k data, 288k init) Everything is still as expected. Then I login to the console and run 'free': total used free sharedbuffers cached Mem: 522780833726081855200 0 5408 31552 -/+ buffers/cache:33356481892160 Swap:0 0 0 This is the first sign that something is really wrong. When I tried 'xend start', the oom killer took over my system and never gave it back. Any thoughts? Thanks, Alex -- Alex Williamson HP Open Source Linux Org. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel -- yamahata ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
Re: [PATCH] fix dom0 builder TAKE 2(was Re: [Xen-ia64-devel] [PATCH] fix dom0 builder)
On Mon, 2006-10-16 at 15:06 +0900, Isaku Yamahata wrote: Hi Alex. If CONFIG_FLATMEM=y is enabled(xenLinux default is so), it seems sane that the total is about 5GB. Do disabling CONFIG_FLATMEM=n and enabling CONFIG_DISCONTIGMEM=y (or CONFIG_SPARSEMEM=y) make difference? And I also noticed that xenLinux default config disables CONFIG_VIRTUAL_MEM_MAP=n. Since dom0 may see underlying machine memory layout with the patch, should we revise default Linux config? Hi Isaku, Yes, I think we'll need to switch to discontig/sparsemem for the dom0 kernel if the memmory map is going to reflect the bare metal hardware memory layout. Unfortunately, just switching to discontig w/ virtual memmap does not solve the problem I'm seeing. The dom0 kernel reports the correct amount of memory early in bootup. Perhaps the balloon driver is causing this(?) Thanks, Alex -- Alex Williamson HP Open Source Linux Org. ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
[PATCH] fix dom0 builder TAKE 2(was Re: [Xen-ia64-devel] [PATCH] fix dom0 builder)
fix dom0 builder TAKE 2.
On Mon, Oct 09, 2006 at 02:01:09PM +0200, Tristan Gingold wrote:
Le Vendredi 06 Octobre 2006 09:41, Isaku Yamahata a écrit :
fix dom0 builder so that xen doesn't assign memory on I/O area.
TODO: the meaning of the xen boot option, dom0_mem, is changed.
dom0 memory is populated in the intersection of
[baremetal conventional memory] and [0, dom0_mem).
Consequently if baremetal memory is sparsely populated,
the total memory assigned to dom0 might be much less than dom0_mem.
HI,
just two remarks:
I think this is really a TODO for Altix, as the intersection may be empty :-(
Why not getting rid of dom0_align ? With dom0vp I think this is a useless
feature.
Tristan.
___
Xen-ia64-devel mailing list
Xen-ia64-devel@lists.xensource.com
http://lists.xensource.com/xen-ia64-devel
--
yamahata
# HG changeset patch
# User [EMAIL PROTECTED]
# Date 1160964212 -32400
# Node ID 94f013f5032b59553445532160143d4238fafc93
# Parent 6bf2a43d253f615beff16d2f2329c44d3099c664
fix dom0 builder so that xen doesn't assign memory on I/O area.
TODO: the meaning of the xen boot option, dom0_mem, is changed.
dom0 memory is populated in the intersection of
[baremetal conventional memory] and [0, dom0_mem).
Consequently if baremetal memory is sparsely populated,
the total memory assigned to dom0 might be much less than dom0_mem.
PATCHNAME: fix_dom0_builder
Signed-off-by: Isaku Yamahata [EMAIL PROTECTED]
diff -r 6bf2a43d253f -r 94f013f5032b xen/arch/ia64/xen/dom_fw.c
--- a/xen/arch/ia64/xen/dom_fw.cMon Oct 16 10:54:00 2006 +0900
+++ b/xen/arch/ia64/xen/dom_fw.cMon Oct 16 11:03:32 2006 +0900
@@ -486,6 +486,49 @@ struct fw_tables {
#define FW_FIELD_MPA(field) \
FW_TABLES_BASE_PADDR + offsetof(struct fw_tables, field)
+static int
+clip_dom0_mem(unsigned long* start, unsigned long* end)
+{
+ void *efi_map_start, *efi_map_end;
+ u64 efi_desc_size;
+ void *p;
+
+ efi_map_start = __va(ia64_boot_param-efi_memmap);
+ efi_map_end = efi_map_start + ia64_boot_param-efi_memmap_size;
+ efi_desc_size = ia64_boot_param-efi_memdesc_size;
+ BUG_ON(*start *end);
+
+ for (p = efi_map_start; p efi_map_end; p += efi_desc_size) {
+ efi_memory_desc_t *md = p;
+ unsigned long md_start = md-phys_addr;
+ unsigned long md_end =
+ md-phys_addr + (md-num_pages EFI_PAGE_SHIFT);
+
+ if (md_end = *start)
+ continue;
+ if (*end md_start) {
+ *end = md_start;
+ return -ENOENT;
+ }
+
+ if (md-type != EFI_CONVENTIONAL_MEMORY ||
+ !(md-attribute EFI_MEMORY_WB))
+ continue;
+
+ if (*start md_start)
+ *start = md-phys_addr;
+ if (md_end *end)
+ *end = md_end;
+ if (*end = *start) {
+ *end = *start;
+ return -ENOENT;
+ }
+ return 0;
+ }
+ *end = ~0UL;
+ return -ENOENT;
+}
+
/* Complete the dom0 memmap. */
static int
complete_dom0_memmap(struct domain *d,
@@ -588,6 +631,7 @@ complete_dom0_memmap(struct domain *d,
for (j = 0; j num_mds; j++) {
unsigned long end;
unsigned long next_start;
+ unsigned long mem_start;
md = tables-efi_memmap[j];
end = md-phys_addr + (md-num_pages EFI_PAGE_SHIFT);
@@ -622,9 +666,28 @@ complete_dom0_memmap(struct domain *d,
/* No room for memory. */
if (end = next_start)
continue;
-
- MAKE_MD(EFI_CONVENTIONAL_MEMORY, EFI_MEMORY_WB,
- end, next_start);
+
+ // dom0 needs to see all I/O devices and wants to
+ // see them at the address where mpaddr = machine physaddr.
+ // However EFI doesn't cover all I/O area.
+ // e.g. PCI brige might not be covered.
+ // So we must be carefull not to use resions used for I/O by
+ // populating only in real machine's conventional memory
+ // region.
+ mem_start = end;
+ while (mem_start next_start) {
+ unsigned long mem_end = next_start;
+ int error;
+ unsigned long next_mem_start;
+
+ error = clip_dom0_mem(mem_start, mem_end);
+ next_mem_start = mem_end;
+ if (error == 0 mem_start mem_end) {
+ MAKE_MD(EFI_CONVENTIONAL_MEMORY, EFI_MEMORY_WB,
+
Re: [PATCH] fix dom0 builder TAKE 2(was Re: [Xen-ia64-devel] [PATCH] fix dom0 builder)
On Mon, 2006-10-16 at 11:06 +0900, Isaku Yamahata wrote:
fix dom0 builder TAKE 2.
Hi Isaku,
What do you think about the patch below? I think this does
essentially the same thing, but removes a lot of the complexity. Please
let me know if I'm overlooking something in your patch. It seems like
it should be trivial to make dom0_mem= become equivalent to the Linux
mem= option. It might be nice to make something equivalent to the Linux
max_addr= option, which is essentially what dom0_mem= does now. Thanks,
Alex
Signed-off-by: Alex Williamson [EMAIL PROTECTED]
---
diff -r fcd746cf4647 xen/arch/ia64/xen/dom_fw.c
--- a/xen/arch/ia64/xen/dom_fw.c Sat Oct 14 18:10:08 2006 -0600
+++ b/xen/arch/ia64/xen/dom_fw.c Sun Oct 15 20:59:30 2006 -0600
@@ -495,7 +495,6 @@ complete_dom0_memmap(struct domain *d,
{
efi_memory_desc_t *md;
u64 addr;
- int j;
void *efi_map_start, *efi_map_end, *p;
u64 efi_desc_size;
int i;
@@ -545,25 +544,20 @@ complete_dom0_memmap(struct domain *d,
case EFI_LOADER_DATA:
case EFI_BOOT_SERVICES_CODE:
case EFI_BOOT_SERVICES_DATA:
- /* Create dom0 MDT entries for conventional memory
- below 1MB. Without this Linux will assume VGA is
- present because 0xA will always be either a hole
- in the MDT or an I/O region via the passthrough. */
-
- end = min(ONE_MB, end);
-
- /* Avoid firmware and hypercall area.
- We know they are 0-based. */
- if (end FW_END_PADDR || start = ONE_MB)
+ if (end FW_END_PADDR || start = maxmem)
break;
- if (start FW_END_PADDR)
-start = FW_END_PADDR;
-
+
+ if (!(md-attribute EFI_MEMORY_WB))
+break;
+
+ start = max(FW_END_PADDR, start);
+ end = min(end, maxmem);
+
dom_md-type = EFI_CONVENTIONAL_MEMORY;
dom_md-phys_addr = start;
dom_md-virt_addr = 0;
dom_md-num_pages = (end - start) EFI_PAGE_SHIFT;
- dom_md-attribute = md-attribute;
+ dom_md-attribute = EFI_MEMORY_WB;
num_mds++;
break;
@@ -580,57 +574,6 @@ complete_dom0_memmap(struct domain *d,
}
BUG_ON(num_mds NUM_MEM_DESCS);
- sort(tables-efi_memmap, num_mds, sizeof(efi_memory_desc_t),
- efi_mdt_cmp, NULL);
-
- /* find gaps and fill them with conventional memory */
- i = num_mds;
- for (j = 0; j num_mds; j++) {
- unsigned long end;
- unsigned long next_start;
-
- md = tables-efi_memmap[j];
- end = md-phys_addr + (md-num_pages EFI_PAGE_SHIFT);
-
- if (j + 1 num_mds) {
- efi_memory_desc_t* next_md;
- next_md = tables-efi_memmap[j + 1];
- next_start = next_md-phys_addr;
-
- /* Have just been sorted. */
- BUG_ON(end next_start);
-
- /* No room for memory! */
- if (end == next_start)
-continue;
-
- if (next_start maxmem)
-next_start = maxmem;
- }
- else
- next_start = maxmem;
-
- /* Avoid legacy low memory addresses
- and the HYPERCALL area. */
- if (end ONE_MB)
- end = ONE_MB;
-
- // clip the range and align to PAGE_SIZE
- next_start = next_start PAGE_MASK;
- end = PAGE_ALIGN(end);
-
- /* No room for memory. */
- if (end = next_start)
- continue;
-
- MAKE_MD(EFI_CONVENTIONAL_MEMORY, EFI_MEMORY_WB,
- end, next_start);
-
- if (next_start = maxmem)
- break;
- }
- num_mds = i;
- BUG_ON(num_mds NUM_MEM_DESCS);
sort(tables-efi_memmap, num_mds, sizeof(efi_memory_desc_t),
efi_mdt_cmp, NULL);
___
Xen-ia64-devel mailing list
Xen-ia64-devel@lists.xensource.com
http://lists.xensource.com/xen-ia64-devel
Re: [PATCH] fix dom0 builder TAKE 2(was Re: [Xen-ia64-devel] [PATCH] fix dom0 builder)
Looks good and much better than mine. On Sun, Oct 15, 2006 at 09:05:42PM -0600, Alex Williamson wrote: On Mon, 2006-10-16 at 11:06 +0900, Isaku Yamahata wrote: fix dom0 builder TAKE 2. Hi Isaku, What do you think about the patch below? I think this does essentially the same thing, but removes a lot of the complexity. Please let me know if I'm overlooking something in your patch. It seems like it should be trivial to make dom0_mem= become equivalent to the Linux mem= option. It might be nice to make something equivalent to the Linux max_addr= option, which is essentially what dom0_mem= does now. Thanks, Alex Signed-off-by: Alex Williamson [EMAIL PROTECTED] --- ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel -- yamahata ___ Xen-ia64-devel mailing list Xen-ia64-devel@lists.xensource.com http://lists.xensource.com/xen-ia64-devel
