I am trying to activate the NIC in the non-root cell and hit a snag. Note that
this is running the debut output that you wanted earlier.
Attaching, the updated config files.
buildroot login: root
# ifconfig eth0 10.134.30.231 netmask 255.255.255.0 broadcast 10.134.30.255
arch_pci_update_msi: 400: 0 00800000 00000000 00000000 00000000
arch_pci_update_msi: 400: 0 00800000 00000000 00000000 00000000
arch_pci_update_msi: 400: 0 00800000 fee04000 00000000 00000000
arch_pci_update_msi: 400: 0 00800000 fee04000 00000000 00000000
arch_pci_update_msi: 400: 0 00800000 fee04000 00000000 00004031
arch_pci_update_msi: 400: 1 00810000 fee04000 00000000 00004031
400004031 97
arch_pci_update_msi: 400: 1 00810000 fee04000 00000000 00004031
400004031 97
arch_pci_update_msi: 400: 1 00810000 fee04000 00000000 00004031
400004031 97
arch_pci_update_msi: 400: 1 00810000 fee04000 00000000 00004031
400004031 97
arch_pci_update_msi: 400: 1 00810000 fee04000 00000000 00004031
400004031 97
arch_pci_update_msi: 400: 1 00810000 fee04000 00000000 00004031
400004031 97
arch_pci_update_msi: 400: 1 00810000 fee04000 00000000 00004031
400004031 97
arch_pci_update_msi: 400: 1 00810000 fee04000 00000000 00004031
400004031 97
arch_pci_update_msi: 400: 1 00810000 fee04000 00000000 00004031
400004031 97
arch_pci_update_msi: 400: 1 00810000 fee04000 00000000 00004031
400004031 97
arch_pci_update_msi: 400: 1 00810000 00000000 00000000 00004031
4031 -1
FATAL: Invalid PCI config write, port: cfc, size 4, address port: 800400ec
RIP: 0xffffffff81702358 RSP: 0xffffc9000022b9f8 FLAGS: 46
RAX: 0x0000000000000000 RBX: 0x00000000000000ec RCX: 0x00000000000000ec
RDX: 0x0000000000000cfc RSI: 0x0000000000000097 RDI: 0x00000000800400ec
CS: 10 BASE: 0x0000000000000000 AR-BYTES: a09b EFER.LMA 1
CR0: 0x0000000080050033 CR3: 0x000000000acaf000 CR4: 0x00000000001426f0
EFER: 0x0000000000000d01
Parking CPU 2 (Cell: "linux2")
________________________________
From: Vaidyanath, Rajiv
Sent: Monday, January 16, 2017 5:58:56 PM
To: Jan Kiszka; [email protected]
Subject: Re: Device assignment to non-root
Thanks, Jan.
That worked and I am able to boot non-root with the Broadcom NIC assigned.
________________________________
From: Jan Kiszka <[email protected]>
Sent: Monday, January 16, 2017 3:20:04 PM
To: Vaidyanath, Rajiv; [email protected]
Subject: Re: Device assignment to non-root
On 2017-01-16 21:17, Vaidyanath, Rajiv wrote:
>
> Thank you very much ! That did the trick.
>
>
> non-root cell loads the driver but craps out on some bogus offset.
>
>
>
>
> [ 0.762705] [drm] Initialized
> [ 0.767062] loop: module loaded
> [ 0.770444] tg3.c:v3.137 (May 11, 2014)
> [ 0.774291] tg3 0000:04:00.0: enabling device (0400 -> 0402)
>
> FATAL: Invalid PCI config write, port: cfc, size 4, address port: 80040068
According to your config, there's a vendor-specifc cap at 0x58, likely
large enough to explain that access as well (3rd byte of the cap encodes
the length). Adjust the related cap entry in the config to describe the
whole length and permit write access.
Jan
> RIP: 0xffffffff817039e8 RSP: 0xffffc900000abb60 FLAGS: 46
> RAX: 0x00000000a200009a RBX: 0x0000000000000068 RCX: 0x0000000000000068
> RDX: 0x0000000000000cfc RSI: 0x0000000000000097 RDI: 0x0000000080040068
> CS: 10 BASE: 0x0000000000000000 AR-BYTES: a09b EFER.LMA 1
> CR0: 0x0000000080050033 CR3: 0x0000000001e16000 CR4: 0x00000000001426f0
> EFER: 0x0000000000000d01
> Parking CPU 2 (Cell: "linux2")
>
> ------------------------------------------------------------------------
> *From:* Jan Kiszka <[email protected]>
> *Sent:* Monday, January 16, 2017 2:54:38 PM
> *To:* Vaidyanath, Rajiv; [email protected]
> *Subject:* Re: Device assignment to non-root
>
> On 2017-01-16 20:42, Vaidyanath, Rajiv wrote:
>> I removed MMCONFIG info for tg3 in the non-root config (mem_regions) and
>> the driver does not expose any ports in /proc/ioports. The only entries
>> I have in non-root config are pci_devices and pci_caps, for tg3 NIC.
>>
>>
>> When I load the non-root after this change, I still get the invalid PCI
>> config write message.
>>
>
> From your non-root cell config:
>
>> .pci_devices = {
>> /* PCIDevice: 04:00.0 */
>> {
>> .type = JAILHOUSE_PCI_TYPE_DEVICE,
>> .iommu = 1,
>> .domain = 0x0,
>> .bdf = 0x400,
>> .bar_mask = {
>> 0xffff0000, 0xffffffff, 0x00000000,
>> 0x00000000, 0x00000000, 0x00000000,
>> },
>> .caps_start = 58,
>
> You need to update this as well. It's the index into the caps array. Now
> it's 0 as there are no other devices and no other caps.
>
>> .num_caps = 9,
>> .num_msi_vectors = 1,
>> .msi_64bits = 1,
>> .num_msix_vectors = 0,
>> .msix_region_size = 0x0,
>> .msix_address = 0x0,
>> },
>>
>> },
>
> Jan
>
> --
> Siemens AG, Corporate Technology, CT RDA ITP SES-DE
> Corporate Competence Center Embedded Linux
--
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
/*
* Jailhouse, a Linux-based partitioning hypervisor
*
* Configuration for linux inmate, 1 CPU, ~60 MB RAM, 1 serial port
*
* Copyright (c) Siemens AG, 2013-2015
*
* Authors:
* Jan Kiszka <[email protected]>
*
* This work is licensed under the terms of the GNU GPL, version 2. See
* the COPYING file in the top-level directory.
*/
#include <jailhouse/types.h>
#include <jailhouse/cell-config.h>
#define ARRAY_SIZE(a) sizeof(a) / sizeof(a[0])
struct {
struct jailhouse_cell_desc cell;
__u64 cpus[1];
struct jailhouse_memory mem_regions[12];
__u8 pio_bitmap[0x2000];
struct jailhouse_pci_device pci_devices[2];
struct jailhouse_pci_capability pci_caps[21];
} __attribute__((packed)) config = {
.cell = {
.signature = JAILHOUSE_CELL_DESC_SIGNATURE,
.revision = JAILHOUSE_CONFIG_REVISION,
.name = "linux2",
.flags = JAILHOUSE_CELL_PASSIVE_COMMREG,
.cpu_set_size = sizeof(config.cpus),
.num_memory_regions = ARRAY_SIZE(config.mem_regions),
.num_cache_regions = 0,
.num_irqchips = 0,
.pio_bitmap_size = ARRAY_SIZE(config.pio_bitmap),
.num_pci_devices = ARRAY_SIZE(config.pci_devices),
.num_pci_caps = ARRAY_SIZE(config.pci_caps),
},
.cpus = {
0xc,
},
.mem_regions = {
/* RAM */ {
.phys_start = 0x401000000,
.virt_start = 0,
.size = 0x00100000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE |
JAILHOUSE_MEM_EXECUTE | JAILHOUSE_MEM_DMA |
JAILHOUSE_MEM_LOADABLE,
},
/* communication region */ {
.virt_start = 0x00100000,
.size = 0x00001000,
.flags = JAILHOUSE_MEM_READ |
JAILHOUSE_MEM_WRITE | JAILHOUSE_MEM_COMM_REGION,
},
/* high RAM */
{
.phys_start = 0x401100000,
.virt_start = 0x00200000,
.size = 0xC800000,
.flags = JAILHOUSE_MEM_READ |
JAILHOUSE_MEM_WRITE | JAILHOUSE_MEM_EXECUTE | JAILHOUSE_MEM_DMA
| JAILHOUSE_MEM_LOADABLE,
},
/* MemRegion: c7200000-c723ffff : 0000:04:00.0 */
{
.phys_start = 0xc7200000,
.virt_start = 0xc7200000,
.size = 0x40000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE,
},
/* MemRegion: c7240000-c724ffff : tg3 */
{
.phys_start = 0xc7240000,
.virt_start = 0xc7240000,
.size = 0x10000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE,
},
},
.pio_bitmap = {
[ 0/8 ... 0x6f/8] = -1,
[ 0x70/8 ... 0x77/8] = 0,
[ 0x80/8 ... 0x3f7/8] = -1,
[ 0x3f8/8 ... 0x3ff/8] = 0, /* serial1 */
[ 0x400/8 ... 0xca3/8] = -1,
[ 0xcf8/8 ... 0xcff/8] = -1,
[0x1000/8 ... 0xe00f/8] = -1,
[0xe010/8 ... 0xe017/8] = -1,
[0xe018/8 ... 0xffff/8] = -1,
},
.pci_devices = {
/* PCIDevice: 04:00.0 */
{
.type = JAILHOUSE_PCI_TYPE_DEVICE,
.iommu = 1,
.domain = 0x0,
.bdf = 0x400,
.bar_mask = {
0xffff0000, 0xffffffff, 0x00000000,
0x00000000, 0x00000000, 0x00000000,
},
.caps_start = 0,
.num_caps = 9,
.num_msi_vectors = 1,
.msi_64bits = 1,
.num_msix_vectors = 0,
.msix_region_size = 0x0,
.msix_address = 0x0,
},
},
.pci_caps = {
/* PCIDevice: 04:00.0 */
{
.id = 0x1,
.start = 0x48,
.len = 8,
.flags = JAILHOUSE_PCICAPS_WRITE,
},
{
.id = 0x3,
.start = 0x50,
.len = 2,
.flags = 0,
},
{
.id = 0x9,
.start = 0x58,
.len = 78,
.flags = JAILHOUSE_PCICAPS_WRITE,
},
{
.id = 0x5,
.start = 0xe8,
.len = 14,
.flags = JAILHOUSE_PCICAPS_WRITE,
},
{
.id = 0x10,
.start = 0xd0,
.len = 20,
.flags = JAILHOUSE_PCICAPS_WRITE,
},
{
.id = 0x1 | JAILHOUSE_PCI_EXT_CAP,
.start = 0x100,
.len = 4,
.flags = 0,
},
{
.id = 0x2 | JAILHOUSE_PCI_EXT_CAP,
.start = 0x13c,
.len = 4,
.flags = 0,
},
{
.id = 0x3 | JAILHOUSE_PCI_EXT_CAP,
.start = 0x160,
.len = 4,
.flags = 0,
},
{
.id = 0x4 | JAILHOUSE_PCI_EXT_CAP,
.start = 0x16c,
.len = 4,
.flags = 0,
},
},
};
04:00.0 Ethernet controller: Broadcom Limited NetXtreme BCM5722 Gigabit
Ethernet PCI Express
Subsystem: Broadcom Limited NetXtreme BCM5722 Gigabit Ethernet PCI
Express
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 16
Region 0: Memory at fbc40000 (64-bit, non-prefetchable) [size=64K]
Expansion ROM at fbc00000 [disabled] [size=256K]
Capabilities: [48] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] Vital Product Data
Not readable
Capabilities: [58] Vendor Specific Information: Len=78 <?>
Capabilities: [e8] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: e17ffa5f4bdffbec Data: bf7a
Capabilities: [d0] Express (v1) Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <4us, L1
unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal-
Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+
TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Exit
Latency L0s <4us, L1 <64us
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+
DLActive- BWMgmt- ABWMgmt-
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt-
RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt-
RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt-
RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
Capabilities: [13c v1] Virtual Channel
Caps: LPEVC=0 RefClk=100ns PATEntryBits=1
Arb: Fixed- WRR32- WRR64- WRR128-
Ctrl: ArbSelect=Fixed
Status: InProgress-
VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=01
Status: NegoPending- InProgress-
Capabilities: [160 v1] Device Serial Number 00-0a-f7-ff-fe-2c-95-cc
Capabilities: [16c v1] Power Budgeting <?>
Kernel driver in use: tg3
