Re: [Xen-devel] Some Xen pci-passthrough questions

2017-10-27 Thread Boris Ostrovsky 



On 10/27/2017 04:16 AM, Sander Eikelenboom wrote:

On 27/10/17 00:30, Boris Ostrovsky wrote:

On 10/26/2017 04:48 PM, Sander Eikelenboom wrote:

- Enabling the xen pt logging in qemu spit out some things, i wonder if they 
are normal:

 qemu-system-i386: -serial pty: char device redirected to /dev/pts/16 
(label serial0)
 [00:05.0] xen_pt_realize: Assigning real physical device 08:00.0 to 
devfn 0x28
 [00:05.0] xen_pt_register_regions: IO region 0 registered 
(size=0x2000 base_addr=0xfe1fe000 type: 0x4)

 Are these somehow expected / benign (they also occur when pci passthrough 
is succesful) ?:
 [00:05.0] xen_pt_config_reg_init: Offset 0x000e mismatch! 
Emulated=0x0080, host=0x, syncing to 0x0080.
 [00:05.0] xen_pt_config_reg_init: Offset 0x0010 mismatch! 
Emulated=0x, host=0xfe1fe004, syncing to 0xfe1fe004.
 [00:05.0] xen_pt_config_reg_init: Offset 0x0052 mismatch! 
Emulated=0x, host=0x4803, syncing to 0x0003.
 [00:05.0] xen_pt_config_reg_init: Offset 0x0072 mismatch! 
Emulated=0x, host=0x0086, syncing to 0x0080.
 [00:05.0] xen_pt_config_reg_init: Offset 0x00a4 mismatch! 
Emulated=0x, host=0x8fc0, syncing to 0x8fc0.
 [00:05.0] xen_pt_config_reg_init: Offset 0x00b2 mismatch! 
Emulated=0x, host=0x1012, syncing to 0x1012.

 [00:05.0] xen_pt_msix_init: get MSI-X table BAR base 0xfe1fe000
 [00:05.0] xen_pt_msix_init: table_off = 0x1000, total_entries = 8
 [00:05.0] xen_pt_msix_init: table_off = 0x1000, total_entries = 8, 
PCI_MSIX_ENTRY_SIZE = 0x10,  msix->table_offset_adjust = 0,  msix->table_base = 
0xfe1fe000
 [00:05.0] xen_pt_msix_init: Error: Can't map physical MSI-X table: 
Invalid argument
That's mmap() of /dev/mem failing:

mmap(NULL,
  total_entries * PCI_MSIX_ENTRY_SIZE +
msix->table_offset_adjust,
  PROT_READ,
  MAP_SHARED | MAP_LOCKED,
  fd,
  msix->table_base + table_off - msix->table_offset_adjust);


Are you running with Craig Bergstrom's patch?

Yes sorry for not being clear (and using the log of running with the
mmap patch), i specifically meant the "mismatch!" lines, although they
also appear in the working case (without the patch) so they probably
aren't an issue, but still thought it wouldn't hurt to ask :).


I believe these warnings are expected. At least in the case of BAR 
(offset 0x10) I can see that qemu initializes emulated register to 0 
(xen_pt_bar_reg_init()) which would not match what the actual HW has. 
But I'll let Antony or Stefan confirm that. (I do think the warnings 
look a bit too alarmist so perhaps they can be softened somewhat)


And as far as Craig's patch is concerned we know it's broken and will be 
reverted.


-boris

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Some Xen pci-passthrough questions

2017-10-27 Thread Sander Eikelenboom 
On 27/10/17 00:30, Boris Ostrovsky wrote:
> On 10/26/2017 04:48 PM, Sander Eikelenboom wrote:
>> Hi Boris / Andrew,
>>
>> In the aftermath of the linux mmap path I have some questions regarding 
>> pci-passthrough:
>>
>> - Is pci-passthrough in combination with an auto-ballooning dom0 supposed to 
>> be a supported combination ?
> 
> I thought it is. I haven't done passthrough recently (and can't do it
> right now) but I when I did I am pretty sure I was running with
> auto-ballooning dom0.
> 
> Our production does passthrough but they always run with dom0's memory
> limited.
> 

Ok, at least it's not a clear "nope", so i will have a go again in a
couple of days and report back.

>>
>> I have used dom0_maxmem settings for dom0 since ages and that works fine 
>> and stable, 
>> but while doing some testing around the linux mmap patch i also thought 
>> to try an auto-ballooning dom0.
>>
>> That ended up in a crashing PV guest with pci-passthrough and a strange 
>> error on two HVM's with pci-passthrough,
>> about vcpu's (while no configurations where changed).
>>
>> So if it is supported i probably have some more testing and reporting to 
>> do ...
>>
>>
>> - While adding some extra logging and enabling the logging on xen pt in qemu,
>>   i wonder if it wouldn't be beneficial to have at least some basic logging 
>> permanently enabled ? 
>>
>> - Enabling the xen pt logging in qemu spit out some things, i wonder if they 
>> are normal:
>>
>> qemu-system-i386: -serial pty: char device redirected to /dev/pts/16 
>> (label serial0)
>> [00:05.0] xen_pt_realize: Assigning real physical device 08:00.0 to 
>> devfn 0x28
>> [00:05.0] xen_pt_register_regions: IO region 0 registered 
>> (size=0x2000 base_addr=0xfe1fe000 type: 0x4)
>>
>> Are these somehow expected / benign (they also occur when pci 
>> passthrough is succesful) ?:
>> [00:05.0] xen_pt_config_reg_init: Offset 0x000e mismatch! 
>> Emulated=0x0080, host=0x, syncing to 0x0080.
>> [00:05.0] xen_pt_config_reg_init: Offset 0x0010 mismatch! 
>> Emulated=0x, host=0xfe1fe004, syncing to 0xfe1fe004.
>> [00:05.0] xen_pt_config_reg_init: Offset 0x0052 mismatch! 
>> Emulated=0x, host=0x4803, syncing to 0x0003.
>> [00:05.0] xen_pt_config_reg_init: Offset 0x0072 mismatch! 
>> Emulated=0x, host=0x0086, syncing to 0x0080.
>> [00:05.0] xen_pt_config_reg_init: Offset 0x00a4 mismatch! 
>> Emulated=0x, host=0x8fc0, syncing to 0x8fc0.
>> [00:05.0] xen_pt_config_reg_init: Offset 0x00b2 mismatch! 
>> Emulated=0x, host=0x1012, syncing to 0x1012.
>>
>> [00:05.0] xen_pt_msix_init: get MSI-X table BAR base 0xfe1fe000
>> [00:05.0] xen_pt_msix_init: table_off = 0x1000, total_entries = 8
>> [00:05.0] xen_pt_msix_init: table_off = 0x1000, total_entries = 8, 
>> PCI_MSIX_ENTRY_SIZE = 0x10,  msix->table_offset_adjust = 0,  
>> msix->table_base = 0xfe1fe000
>> [00:05.0] xen_pt_msix_init: Error: Can't map physical MSI-X table: 
>> Invalid argument
> 
> That's mmap() of /dev/mem failing:
> 
> mmap(NULL,
>  total_entries * PCI_MSIX_ENTRY_SIZE +
> msix->table_offset_adjust,
>  PROT_READ,
>  MAP_SHARED | MAP_LOCKED,
>  fd,
>  msix->table_base + table_off - msix->table_offset_adjust);
> 
> 
> Are you running with Craig Bergstrom's patch?

Yes sorry for not being clear (and using the log of running with the
mmap patch), i specifically meant the "mismatch!" lines, although they
also appear in the working case (without the patch) so they probably
aren't an issue, but still thought it wouldn't hurt to ask :).




> -boris
> 
> 
>> [00:05.0] xen_pt_msix_size_init: Error: Internal error: Invalid 
>> xen_pt_msix_init.
>> Failed to initialize 12/15, type = 0x1, rc: -22
>> [00:05.0] xen_pt_msi_set_enable: disabling MSI.
>>
>> This crash seems to indicate the above disabling of MSI isn't handled 
>> well enough to prevent this from happening: 
>> *** Error in `/usr/local/lib/xen/bin/qemu-system-i386': corrupted 
>> size vs. prev_size: 0x55ce13565570 ***
>> === Backtrace: =
>> /lib/x86_64-linux-gnu/libc.so.6(+0x70bcb)[0x7f700ab7ebcb]
>> /lib/x86_64-linux-gnu/libc.so.6(+0x76f96)[0x7f700ab84f96]
>> 
>>
>> --
>> Sander
> 


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Some Xen pci-passthrough questions

2017-10-27 Thread Jan Beulich 
>>> On 26.10.17 at 22:48,  wrote:
> - While adding some extra logging and enabling the logging on xen pt in 
> qemu,
>   i wonder if it wouldn't be beneficial to have at least some basic logging 
> permanently enabled ? 

For this end everything further down you would better have added
the qemu maintainers. I've now put them on Cc.

Jan

> - Enabling the xen pt logging in qemu spit out some things, i wonder if they 
> are normal:
> 
> qemu-system-i386: -serial pty: char device redirected to /dev/pts/16 
> (label serial0)
> [00:05.0] xen_pt_realize: Assigning real physical device 08:00.0 to 
> devfn 0x28
> [00:05.0] xen_pt_register_regions: IO region 0 registered 
> (size=0x2000 base_addr=0xfe1fe000 type: 0x4)
> 
> Are these somehow expected / benign (they also occur when pci 
> passthrough is succesful) ?:
> [00:05.0] xen_pt_config_reg_init: Offset 0x000e mismatch! 
> Emulated=0x0080, host=0x, syncing to 0x0080.
> [00:05.0] xen_pt_config_reg_init: Offset 0x0010 mismatch! 
> Emulated=0x, host=0xfe1fe004, syncing to 0xfe1fe004.
> [00:05.0] xen_pt_config_reg_init: Offset 0x0052 mismatch! 
> Emulated=0x, host=0x4803, syncing to 0x0003.
> [00:05.0] xen_pt_config_reg_init: Offset 0x0072 mismatch! 
> Emulated=0x, host=0x0086, syncing to 0x0080.
> [00:05.0] xen_pt_config_reg_init: Offset 0x00a4 mismatch! 
> Emulated=0x, host=0x8fc0, syncing to 0x8fc0.
> [00:05.0] xen_pt_config_reg_init: Offset 0x00b2 mismatch! 
> Emulated=0x, host=0x1012, syncing to 0x1012.
> 
> [00:05.0] xen_pt_msix_init: get MSI-X table BAR base 0xfe1fe000
> [00:05.0] xen_pt_msix_init: table_off = 0x1000, total_entries = 8
> [00:05.0] xen_pt_msix_init: table_off = 0x1000, total_entries = 8, 
> PCI_MSIX_ENTRY_SIZE = 0x10,  msix->table_offset_adjust = 0,  msix->table_base 
> = 
> 0xfe1fe000
> [00:05.0] xen_pt_msix_init: Error: Can't map physical MSI-X table: 
> Invalid argument
> [00:05.0] xen_pt_msix_size_init: Error: Internal error: Invalid 
> xen_pt_msix_init.
> Failed to initialize 12/15, type = 0x1, rc: -22
> [00:05.0] xen_pt_msi_set_enable: disabling MSI.
> 
> This crash seems to indicate the above disabling of MSI isn't handled 
> well enough to prevent this from happening: 
> *** Error in `/usr/local/lib/xen/bin/qemu-system-i386': corrupted 
> size vs. prev_size: 0x55ce13565570 ***
> === Backtrace: =
> /lib/x86_64-linux-gnu/libc.so.6(+0x70bcb)[0x7f700ab7ebcb]
> /lib/x86_64-linux-gnu/libc.so.6(+0x76f96)[0x7f700ab84f96]
> 
> 
> --
> Sander
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org 
> https://lists.xen.org/xen-devel 




___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Some Xen pci-passthrough questions

2017-10-26 Thread Boris Ostrovsky 
On 10/26/2017 04:48 PM, Sander Eikelenboom wrote:
> Hi Boris / Andrew,
>
> In the aftermath of the linux mmap path I have some questions regarding 
> pci-passthrough:
>
> - Is pci-passthrough in combination with an auto-ballooning dom0 supposed to 
> be a supported combination ?

I thought it is. I haven't done passthrough recently (and can't do it
right now) but I when I did I am pretty sure I was running with
auto-ballooning dom0.

Our production does passthrough but they always run with dom0's memory
limited.

>
> I have used dom0_maxmem settings for dom0 since ages and that works fine 
> and stable, 
> but while doing some testing around the linux mmap patch i also thought 
> to try an auto-ballooning dom0.
>
> That ended up in a crashing PV guest with pci-passthrough and a strange 
> error on two HVM's with pci-passthrough,
> about vcpu's (while no configurations where changed).
>
> So if it is supported i probably have some more testing and reporting to 
> do ...
>
>
> - While adding some extra logging and enabling the logging on xen pt in qemu,
>   i wonder if it wouldn't be beneficial to have at least some basic logging 
> permanently enabled ? 
>
> - Enabling the xen pt logging in qemu spit out some things, i wonder if they 
> are normal:
>
> qemu-system-i386: -serial pty: char device redirected to /dev/pts/16 
> (label serial0)
> [00:05.0] xen_pt_realize: Assigning real physical device 08:00.0 to 
> devfn 0x28
> [00:05.0] xen_pt_register_regions: IO region 0 registered 
> (size=0x2000 base_addr=0xfe1fe000 type: 0x4)
>
> Are these somehow expected / benign (they also occur when pci passthrough 
> is succesful) ?:
> [00:05.0] xen_pt_config_reg_init: Offset 0x000e mismatch! 
> Emulated=0x0080, host=0x, syncing to 0x0080.
> [00:05.0] xen_pt_config_reg_init: Offset 0x0010 mismatch! 
> Emulated=0x, host=0xfe1fe004, syncing to 0xfe1fe004.
> [00:05.0] xen_pt_config_reg_init: Offset 0x0052 mismatch! 
> Emulated=0x, host=0x4803, syncing to 0x0003.
> [00:05.0] xen_pt_config_reg_init: Offset 0x0072 mismatch! 
> Emulated=0x, host=0x0086, syncing to 0x0080.
> [00:05.0] xen_pt_config_reg_init: Offset 0x00a4 mismatch! 
> Emulated=0x, host=0x8fc0, syncing to 0x8fc0.
> [00:05.0] xen_pt_config_reg_init: Offset 0x00b2 mismatch! 
> Emulated=0x, host=0x1012, syncing to 0x1012.
>
> [00:05.0] xen_pt_msix_init: get MSI-X table BAR base 0xfe1fe000
> [00:05.0] xen_pt_msix_init: table_off = 0x1000, total_entries = 8
> [00:05.0] xen_pt_msix_init: table_off = 0x1000, total_entries = 8, 
> PCI_MSIX_ENTRY_SIZE = 0x10,  msix->table_offset_adjust = 0,  msix->table_base 
> = 0xfe1fe000
> [00:05.0] xen_pt_msix_init: Error: Can't map physical MSI-X table: 
> Invalid argument

That's mmap() of /dev/mem failing:

mmap(NULL,
 total_entries * PCI_MSIX_ENTRY_SIZE +
msix->table_offset_adjust,
 PROT_READ,
 MAP_SHARED | MAP_LOCKED,
 fd,
 msix->table_base + table_off - msix->table_offset_adjust);


Are you running with Craig Bergstrom's patch?

-boris


> [00:05.0] xen_pt_msix_size_init: Error: Internal error: Invalid 
> xen_pt_msix_init.
> Failed to initialize 12/15, type = 0x1, rc: -22
> [00:05.0] xen_pt_msi_set_enable: disabling MSI.
>
> This crash seems to indicate the above disabling of MSI isn't handled 
> well enough to prevent this from happening: 
> *** Error in `/usr/local/lib/xen/bin/qemu-system-i386': corrupted 
> size vs. prev_size: 0x55ce13565570 ***
> === Backtrace: =
> /lib/x86_64-linux-gnu/libc.so.6(+0x70bcb)[0x7f700ab7ebcb]
> /lib/x86_64-linux-gnu/libc.so.6(+0x76f96)[0x7f700ab84f96]
> 
>
> --
> Sander


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] Some Xen pci-passthrough questions

2017-10-26 Thread Sander Eikelenboom 
Hi Boris / Andrew,

In the aftermath of the linux mmap path I have some questions regarding 
pci-passthrough:

- Is pci-passthrough in combination with an auto-ballooning dom0 supposed to be 
a supported combination ?

I have used dom0_maxmem settings for dom0 since ages and that works fine 
and stable, 
but while doing some testing around the linux mmap patch i also thought to 
try an auto-ballooning dom0.

That ended up in a crashing PV guest with pci-passthrough and a strange 
error on two HVM's with pci-passthrough,
about vcpu's (while no configurations where changed).

So if it is supported i probably have some more testing and reporting to do 
...


- While adding some extra logging and enabling the logging on xen pt in qemu,
  i wonder if it wouldn't be beneficial to have at least some basic logging 
permanently enabled ? 

- Enabling the xen pt logging in qemu spit out some things, i wonder if they 
are normal:

qemu-system-i386: -serial pty: char device redirected to /dev/pts/16 
(label serial0)
[00:05.0] xen_pt_realize: Assigning real physical device 08:00.0 to 
devfn 0x28
[00:05.0] xen_pt_register_regions: IO region 0 registered 
(size=0x2000 base_addr=0xfe1fe000 type: 0x4)

Are these somehow expected / benign (they also occur when pci passthrough 
is succesful) ?:
[00:05.0] xen_pt_config_reg_init: Offset 0x000e mismatch! 
Emulated=0x0080, host=0x, syncing to 0x0080.
[00:05.0] xen_pt_config_reg_init: Offset 0x0010 mismatch! 
Emulated=0x, host=0xfe1fe004, syncing to 0xfe1fe004.
[00:05.0] xen_pt_config_reg_init: Offset 0x0052 mismatch! 
Emulated=0x, host=0x4803, syncing to 0x0003.
[00:05.0] xen_pt_config_reg_init: Offset 0x0072 mismatch! 
Emulated=0x, host=0x0086, syncing to 0x0080.
[00:05.0] xen_pt_config_reg_init: Offset 0x00a4 mismatch! 
Emulated=0x, host=0x8fc0, syncing to 0x8fc0.
[00:05.0] xen_pt_config_reg_init: Offset 0x00b2 mismatch! 
Emulated=0x, host=0x1012, syncing to 0x1012.

[00:05.0] xen_pt_msix_init: get MSI-X table BAR base 0xfe1fe000
[00:05.0] xen_pt_msix_init: table_off = 0x1000, total_entries = 8
[00:05.0] xen_pt_msix_init: table_off = 0x1000, total_entries = 8, 
PCI_MSIX_ENTRY_SIZE = 0x10,  msix->table_offset_adjust = 0,  msix->table_base = 
0xfe1fe000
[00:05.0] xen_pt_msix_init: Error: Can't map physical MSI-X table: 
Invalid argument
[00:05.0] xen_pt_msix_size_init: Error: Internal error: Invalid 
xen_pt_msix_init.
Failed to initialize 12/15, type = 0x1, rc: -22
[00:05.0] xen_pt_msi_set_enable: disabling MSI.

This crash seems to indicate the above disabling of MSI isn't handled well 
enough to prevent this from happening: 
*** Error in `/usr/local/lib/xen/bin/qemu-system-i386': corrupted size 
vs. prev_size: 0x55ce13565570 ***
=== Backtrace: =
/lib/x86_64-linux-gnu/libc.so.6(+0x70bcb)[0x7f700ab7ebcb]
/lib/x86_64-linux-gnu/libc.so.6(+0x76f96)[0x7f700ab84f96]


--
Sander

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel