Thanks, Jan.
That worked and I am able to boot non-root with the Broadcom NIC assigned. ________________________________ From: Jan Kiszka <[email protected]> Sent: Monday, January 16, 2017 3:20:04 PM To: Vaidyanath, Rajiv; [email protected] Subject: Re: Device assignment to non-root On 2017-01-16 21:17, Vaidyanath, Rajiv wrote: > > Thank you very much ! That did the trick. > > > non-root cell loads the driver but craps out on some bogus offset. > > > > > [ 0.762705] [drm] Initialized > [ 0.767062] loop: module loaded > [ 0.770444] tg3.c:v3.137 (May 11, 2014) > [ 0.774291] tg3 0000:04:00.0: enabling device (0400 -> 0402) > > FATAL: Invalid PCI config write, port: cfc, size 4, address port: 80040068 According to your config, there's a vendor-specifc cap at 0x58, likely large enough to explain that access as well (3rd byte of the cap encodes the length). Adjust the related cap entry in the config to describe the whole length and permit write access. Jan > RIP: 0xffffffff817039e8 RSP: 0xffffc900000abb60 FLAGS: 46 > RAX: 0x00000000a200009a RBX: 0x0000000000000068 RCX: 0x0000000000000068 > RDX: 0x0000000000000cfc RSI: 0x0000000000000097 RDI: 0x0000000080040068 > CS: 10 BASE: 0x0000000000000000 AR-BYTES: a09b EFER.LMA 1 > CR0: 0x0000000080050033 CR3: 0x0000000001e16000 CR4: 0x00000000001426f0 > EFER: 0x0000000000000d01 > Parking CPU 2 (Cell: "linux2") > > ------------------------------------------------------------------------ > *From:* Jan Kiszka <[email protected]> > *Sent:* Monday, January 16, 2017 2:54:38 PM > *To:* Vaidyanath, Rajiv; [email protected] > *Subject:* Re: Device assignment to non-root > > On 2017-01-16 20:42, Vaidyanath, Rajiv wrote: >> I removed MMCONFIG info for tg3 in the non-root config (mem_regions) and >> the driver does not expose any ports in /proc/ioports. The only entries >> I have in non-root config are pci_devices and pci_caps, for tg3 NIC. >> >> >> When I load the non-root after this change, I still get the invalid PCI >> config write message. >> > > From your non-root cell config: > >> .pci_devices = { >> /* PCIDevice: 04:00.0 */ >> { >> .type = JAILHOUSE_PCI_TYPE_DEVICE, >> .iommu = 1, >> .domain = 0x0, >> .bdf = 0x400, >> .bar_mask = { >> 0xffff0000, 0xffffffff, 0x00000000, >> 0x00000000, 0x00000000, 0x00000000, >> }, >> .caps_start = 58, > > You need to update this as well. It's the index into the caps array. Now > it's 0 as there are no other devices and no other caps. > >> .num_caps = 9, >> .num_msi_vectors = 1, >> .msi_64bits = 1, >> .num_msix_vectors = 0, >> .msix_region_size = 0x0, >> .msix_address = 0x0, >> }, >> >> }, > > Jan > > -- > Siemens AG, Corporate Technology, CT RDA ITP SES-DE > Corporate Competence Center Embedded Linux -- Siemens AG, Corporate Technology, CT RDA ITP SES-DE Corporate Competence Center Embedded Linux -- You received this message because you are subscribed to the Google Groups "Jailhouse" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
[ 0.000000] Linux version 4.9.0-rt1+ (root@beast) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-4) (GCC) ) #1 SMP PREEMPT RT Fri Jan 13 13:51:28 EST 2017 [ 0.000000] Command line: console=ttyS0,115200 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] x86/fpu: Using 'eager' FPU context switches. [ 0.000000] e820: BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x00000000000fffff] usable [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x0000000000100fff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000200000-0x000000000c9fffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] extended physical RAM map: [ 0.000000] reserve setup_data: [mem 0x0000000000000000-0x00000000000f5fff] usable [ 0.000000] reserve setup_data: [mem 0x00000000000f6000-0x00000000000f6112] usable [ 0.000000] reserve setup_data: [mem 0x00000000000f6113-0x00000000000fffff] usable [ 0.000000] reserve setup_data: [mem 0x0000000000100000-0x0000000000100fff] reserved [ 0.000000] reserve setup_data: [mem 0x0000000000200000-0x000000000c9fffff] usable [ 0.000000] DMI not present or invalid. [ 0.000000] Hypervisor detected: Jailhouse [ 0.000000] Jailhouse: PM-Timer IO Port: 0x408 [ 0.000000] PCI: Using configuration type 1 for base access [ 0.000000] e820: last_pfn = 0xca00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WC UC- WT [ 0.000000] x2apic: enabled by BIOS, switching to x2apic ops [ 0.000000] Scanning 1 areas for low memory corruption [ 0.000000] Using GB pages for direct mapping [ 0.000000] RAMDISK: [mem 0x02f62000-0x0314bfff] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000000c9fffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x0c9fc000-0x0c9fffff] [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x000000000c9fffff] [ 0.000000] Normal empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009ffff] [ 0.000000] node 0: [mem 0x0000000000200000-0x000000000c9fffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000000c9fffff] [ 0.000000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000f5fff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f6000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0x00100000-0x00100fff] [ 0.000000] PM: Registered nosave memory: [mem 0x00101000-0x001fffff] [ 0.000000] e820: [mem 0x0ca00000-0xffffffff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on bare hardware [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:64 nr_cpumask_bits:64 nr_cpu_ids:2 nr_node_ids:1 [ 0.000000] percpu: Embedded 35 pages/cpu @ffff88000c600000 s104896 r8192 d30272 u1048576 [ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 50534 [ 0.000000] Policy zone: DMA32 [ 0.000000] Kernel command line: console=ttyS0,115200 [ 0.000000] PID hash table entries: 1024 (order: 1, 8192 bytes) [ 0.000000] Memory: 181140K/205436K available (9331K kernel code, 1285K rwdata, 2812K rodata, 1168K init, 968K bss, 24296K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 [ 0.000000] Preemptible hierarchical RCU implementation. [ 0.000000] Build-time adjustment of leaf fanout to 64. [ 0.000000] RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2. [ 0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=2 [ 0.000000] NR_IRQS:4352 nr_irqs:32 0 [ 0.000000] Console: colour dummy device 80x25 [ 0.000000] console [ttyS0] enabled [ 0.000000] tsc: Detected 2300.013 MHz processor [ 0.000063] Calibrating delay loop (skipped), value calculated using timer frequency.. 4600.02 BogoMIPS (lpj=2300013) [ 0.000064] pid_max: default: 32768 minimum: 301 [ 0.000079] Security Framework initialized [ 0.000080] SELinux: Initializing. [ 0.000138] Dentry cache hash table entries: 32768 (order: 7, 524288 bytes) [ 0.000212] Inode-cache hash table entries: 16384 (order: 5, 131072 bytes) [ 0.000235] Mount-cache hash table entries: 512 (order: 0, 4096 bytes) [ 0.000236] Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes) [ 0.000441] CPU: Physical Processor ID: 0 [ 0.000442] CPU: Processor Core ID: 2 [ 0.000452] mce: CPU supports 22 MCE banks [ 0.000469] CPU0: Thermal monitoring enabled (TM1) [ 0.000502] process: using mwait in idle threads [ 0.000506] Last level iTLB entries: 4KB 1024, 2MB 1024, 4MB 1024 [ 0.000506] Last level dTLB entries: 4KB 1024, 2MB 1024, 4MB 1024, 1GB 4 [ 0.002003] Freeing SMP alternatives memory: 32K (ffffffff82067000 - ffffffff8206f000) [ 0.003697] smpboot: APIC(4) Converting physical 0 to logical package 0 [ 0.110282] smpboot: Max logical packages: 1 [ 0.114548] Switched APIC routing to physical x2apic. [ 0.119609] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz (family: 0x6, model: 0x3f, stepping: 0x2) [ 0.129635] Performance Events: PEBS fmt2+, Haswell events, 16-deep LBR, full-width counters, Intel PMU driver. [ 0.139889] ... version: 3 [ 0.143897] ... bit width: 48 [ 0.147985] ... generic registers: 8 [ 0.151990] ... value mask: 0000ffffffffffff [ 0.157294] ... max period: 00007fffffffffff [ 0.162597] ... fixed-purpose events: 3 [ 0.166602] ... event mask: 00000007000000ff [ 0.195798] x86: Booting SMP configuration: [ 0.199981] .... node #0, CPUs: #1CPU 3 received SIPI, vector 9a [ 0.268923] x86: Booted up 1 node, 2 CPUs [ 0.272933] smpboot: Total of 2 processors activated (9204.78 BogoMIPS) [ 0.284313] devtmpfs: initialized [ 0.287784] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.289194] kworker/u4:0 (26) used greatest stack depth: 14504 bytes left [ 0.304430] NET: Registered protocol family 16 [ 0.309212] cpuidle: using governor menu [ 0.313231] kworker/u4:0 (40) used greatest stack depth: 13992 bytes left [ 0.313232] core: PMU erratum BJ122, BV98, HSD29 workaround disabled, HT off [ 0.320268] rcgepcB upgiteerse: 200B 0 g08si00, 0re000l0 at00 0 0ag00 000 [ 0.326250] ACPI: Interpreter disabled. [ 0.326340] vgaarb: loaded [ 0.326397] SCSI subsystem initialized [ 0.328295] usbcore: registered new interface driver usbfs [ 0.328304] usbcore: registered new interface driver hub [ 0.328316] usbcore: registered new device driver usb [ 0.328329] pps_core: LinuxPPS API ver. 1 registered [ 0.328329] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <[email protected]> [ 0.328333] PTP clock support registered [ 0.330259] dmi: Firmware registration failed. [ 0.330277] Advanced Linux Sound Architecture Driver Initialized. [ 0.330290] PCI: Probing PCI hardware [ 0.330317] PCI host bridge to bus 0000:00 [ 0.330320] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] [ 0.330321] pci_bus 0000:00: root bus resource [mem 0x00000000-0x3fffffffffff] [ 0.330322] pci_bus 0000:00: No busn resource found for root bus, will use [bus 00-ff] [ 0.331620] PCI: Discovered peer bus 04 [ 0.331646] PCI host bridge to bus 0000:04 [ 0.331647] pci_bus 0000:04: root bus resource [io 0x0000-0xffff] [ 0.331648] pci_bus 0000:04: root bus resource [mem 0x00000000-0x3fffffffffff] [ 0.331649] pci_bus 0000:04: No busn resource found for root bus, will use [bus 04-ff] [ 0.403481] NetLabel: Initializing [ 0.403481] NetLabel: domain hash size = 128 [ 0.403482] NetLabel: protocols = UNLABELED CIPSOv4 [ 0.403495] NetLabel: unlabeled traffic allowed by default [ 0.403571] clocksource: Switched to clocksource refined-jiffies [ 0.412965] VFS: Disk quotas dquot_6.6.0 [ 0.412977] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.485597] pnp: PnP ACPI: disabled [ 0.495556] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.504452] clocksource: Switched to clocksource acpi_pm [ 0.509787] NET: Registered protocol family 2 [ 0.514275] TCP established hash table entries: 2048 (order: 2, 16384 bytes) [ 0.521330] TCP bind hash table entries: 2048 (order: 4, 114688 bytes) [ 0.527881] TCP: Hash tables configured (established 2048 bind 2048) [ 0.534238] UDP hash table entries: 256 (order: 3, 32768 bytes) [ 0.540162] UDP-Lite hash table entries: 256 (order: 3, 32768 bytes) [ 0.546581] NET: Registered protocol family 1 [ 0.551012] RPC: Registered named UNIX socket transport module. [ 0.556919] RPC: Registered udp transport module. [ 0.561616] RPC: Registered tcp transport module. [ 0.566316] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.572803] Unpacking initramfs... [ 0.580017] Freeing initrd memory: 1960K (ffff880002f62000 - ffff88000314c000) [ 0.587245] Intel CQM monitoring enabled [ 0.591190] RAPL PMU: API unit is 2^-32 Joules, 4 fixed counters, 655360 ms ovfl timer [ 0.599095] RAPL PMU: hw unit of domain pp0-core 2^-14 Joules [ 0.604830] RAPL PMU: hw unit of domain package 2^-14 Joules [ 0.610487] RAPL PMU: hw unit of domain dram 2^-14 Joules [ 0.615877] RAPL PMU: hw unit of domain pp1-gpu 2^-14 Joules [ 0.621575] platform rtc_cmos: registered platform RTC device (no PNP device found) [ 0.629434] Scanning for low memory corruption every 60 seconds [ 0.635638] futex hash table entries: 512 (order: 4, 65536 bytes) [ 0.641742] audit: initializing netlink subsys (disabled) [ 0.647142] audit: type=2000 audit(0.520:1): initialized [ 0.652902] workingset: timestamp_bits=56 max_order=16 bucket_order=0 [ 0.662519] NFS: Registering the id_resolver key type [ 0.667577] Key type id_resolver registered [ 0.671755] Key type id_legacy registered [ 0.677145] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251) [ 0.684535] io scheduler noop registered [ 0.688455] io scheduler deadline registered [ 0.692770] io scheduler cfq registered (default) [ 0.697570] pci_hotplug: PCI Hot Plug PCI Core version: 0.5 [ 0.703273] Serial: 8250/16550 driver, 1 ports, IRQ sharing enabled [ 0.730188] serial8250: ttyS0 at I/O 0x3f8 (irq = 0, base_baud = 115200) is a 16550A [ 0.738173] Non-volatile memory driver v1.3 [ 0.742388] Linux agpgart interface v0.103 [ 0.746568] [drm] Initialized [ 0.750921] loop: module loaded [ 0.754292] tg3.c:v3.137 (May 11, 2014) [ 0.758134] tg3 0000:04:00.0: enabling device (0400 -> 0402) [ 0.772700] tg3 0000:04:00.0 eth0: Tigon3 [partno(BCM95722) rev a200] (PCI Express) MAC address 00:0a:f7:2c:95:cc [ 0.782950] tg3 0000:04:00.0 eth0: attached PHY is 5722/5756 (10/100/1000Base-T Ethernet) (WireSpeed[1], EEE[0]) [ 0.793111] tg3 0000:04:00.0 eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] TSOcap[1] [ 0.800928] tg3 0000:04:00.0 eth0: dma_rwctrl[76180000] dma_mask[64-bit] [ 0.807702] e100: Intel(R) PRO/100 Network Driver, 3.5.24-k2-NAPI [ 0.813789] e100: Copyright(c) 1999-2006 Intel Corporation [ 0.819297] e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI [ 0.826339] e1000: Copyright (c) 1999-2006 Intel Corporation. [ 0.832105] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k [ 0.837926] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. [ 0.843863] sky2: driver version 1.30 [ 0.847682] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver [ 0.854204] ehci-pci: EHCI PCI platform driver [ 0.858666] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver [ 0.864842] ohci-pci: OHCI PCI platform driver [ 0.869308] uhci_hcd: USB Universal Host Controller Interface driver [ 0.875704] usbcore: registered new interface driver usblp [ 0.881210] usbcore: registered new interface driver usb-storage [ 0.887260] mousedev: PS/2 mouse device common for all mice [ 0.892975] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0 [ 0.899382] rtc_cmos: probe of rtc_cmos failed with error -22 [ 0.905302] device-mapper: ioctl: 4.35.0-ioctl (2016-06-23) initialised: [email protected] [ 0.913767] hidraw: raw HID events driver (C) Jiri Kosina [ 0.919573] usbcore: registered new interface driver usbhid [ 0.925141] usbhid: USB HID core driver [ 0.929244] Netfilter messages via NETLINK v0.30. [ 0.934011] nf_conntrack version 0.5.0 (1536 buckets, 6144 max) [ 0.939947] ctnetlink v0.93: registering with nfnetlink. [ 0.945375] ip_tables: (C) 2000-2006 Netfilter Core Team [ 0.950690] Initializing XFRM netlink socket [ 0.955040] NET: Registered protocol family 10 [ 0.959730] ip6_tables: (C) 2000-2006 Netfilter Core Team [ 0.965245] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver [ 0.971317] NET: Registered protocol family 17 [ 0.975778] Key type dns_resolver registered [ 0.980251] microcode: sig=0x306f2, pf=0x1, revision=0x36 [ 0.980296] microcode: Microcode Update Driver: v2.01 <[email protected]>, Peter Oruba [ 0.994621] registered taskstats version 1 [ 0.998783] console [netcon0] enabled [ 1.002446] netconsole: network logging started [ 1.006992] ALSA device list: [ 1.009958] No soundcards found. [ 1.014301] Freeing unused kernel memory: 1168K (ffffffff81f43000 - ffffffff82067000) [ 1.022117] Write protecting the kernel read-only data: 14336k [ 1.028425] Freeing unused kernel memory: 892K (ffff880001921000 - ffff880001a00000) [ 1.039290] Freeing unused kernel memory: 1284K (ffff880001cbf000 - ffff880001e00000) Starting logging[ 1.058906] random: dd: uninitialized urandom read (512 bytes read) : OK Initializi[ 1.068878] ip (867) used greatest stack depth: 13648 bytes left [ 1.074886] NOHZ: local_softirq_pending 02 [ 1.074999] NOHZ: local_softirq_pending 02 ng random number generator... done. Starting network: OK Welcome to Buildroot buildroot login: [ 1.630483] tsc: Refined TSC clocksource calibration: 2299.998 MHz [ 1.636682] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x212733415c7, max_idle_ns: 440795236380 ns [ 1.646677] NOHZ: local_softirq_pending 02 [ 2.654524] clocksource: Switched to clocksource tsc Welcome to Buildroot buildroot login: root # ifconfig lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) # ifconfig -a eth0 Link encap:Ethernet HWaddr 00:0A:F7:2C:95:CC BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) Interrupt:11 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) sit0 Link encap:IPv6-in-IPv4 NOARP MTU:1480 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
