Re: problem with pass-through on amd

2017-11-16 Thread Anish 
Hi Andriy,
>After I fixed the MSI setup problem that I described in the previous email
I was
finally able to see the real problem.

Awesome! fault interrupt handler has been very helpful in debugging AMD-Vi.

>I've passed both of the PCI device to bhyve and everything started working.
Thanks!
Bhyve uses single domain for all the passthrough devices of a guest, that's
why even with aliasing it worked.

>But it appears to me that FreeBSD currently ignores those aliases.
Also, it looks like we are not making any use of IVHD entries beyond
printing
them.

We make use of of very few IVRS entries, see ivhd_dev_parse() but not much
for aliasing entries. Found out AMD IOMMU spec has some details on it:

"An Alias device type entry is used for each peripheral that does not use
its own DeviceID information

in bus transactions. For example, peripherals downstream of a bridge device
that use the DeviceID of

the bridge must have a corresponding Alias Select or Alias Start of Range
entry to inform system

software which IOMMU Device Table entry will be used for translation
information."


Thanks a lot for trying out AMD-Vi, let me know if you have any other
feedback.

Thanks and regards,

Anish





On Thu, Nov 16, 2017 at 12:51 PM, Andriy Gapon  wrote:

> On 14/11/2017 06:22, Anish wrote:
> > [root@ryzen /home/anish/FreeBSD/head]# vmstat -ia |grep ivh
> > irq256: ivhd0:fault0  0
> > irq257: ivhd1:fault0  0
>
> After I fixed the MSI setup problem that I described in the previous email
> I was
> finally able to see the real problem.
> Now I have:
>
> irq263: ivhd0:fault9  0
>
> dev.ivhd.0.event_tail: 240
> dev.ivhd.0.event_head: 240
> dev.ivhd.0.event_intr_count: 9
>
>
> I've got a bunch of messages like these in the log:
> [IO_PAGE_FAULT EVT: devId:0xa01 DomId:0x2 Addr:0x70f70400x30]
> [IO_PAGE_FAULT EVT: devId:0xa01 DomId:0x2 Addr:0x2fdf0400x30]
> (BTW, there seems to be a missing space before 0x30)
>
> Now it's obvious what the problem is.
> The controller has a RID of 0xa00 (its PCI locator is 10:0:0) but the
> requests
> are coming from 0xa01.  The card actually has another PCI device at that
> location (10:0:1), it's a vestigial IDE controller (in a sense that it is
> not
> connected to any ports, so it cannot really provide any functionality).
>
> I've passed both of the PCI device to bhyve and everything started working.
> Thanks!
>
> I've googled a little bit and it seems that this is not an uncommon
> problem.
> Linux has a bunch of quirks for this and similar kinds of problems (they
> call it
> PCI or DMA aliasing):
> http://elixir.free-electrons.com/linux/v4.8.16/source/
> drivers/pci/quirks.c#L3701
> The comments say that sometimes requests come from devices that do not
> exist at
> all (or hidden beyond non-transparent bridges).
>
> An interesting new world for me :-)
>
> By the way, my understanding is that the alias IVHD entries in IVRS are
> designed
> to report such aliasing issues when they are known to the platform.
> But it appears to me that FreeBSD currently ignores those aliases.
> Also, it looks like we are not making any use of IVHD entries beyond
> printing
> them.  Based on the specification I think that we should use IVHD flags to
> set
> up appropriate bits in the corresponding device table entries.  But we are
> not
> doing that.  That's probably okay since we do not support the pass-though
> for
> devices with complex properties anyway.
>
> --
> Andriy Gapon
>
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: problem with pass-through on amd

2017-11-16 Thread Andriy Gapon 
On 14/11/2017 06:22, Anish wrote:
> [root@ryzen /home/anish/FreeBSD/head]# vmstat -ia |grep ivh
> irq256: ivhd0:fault                    0          0
> irq257: ivhd1:fault                    0          0

After I fixed the MSI setup problem that I described in the previous email I was
finally able to see the real problem.
Now I have:

irq263: ivhd0:fault9  0

dev.ivhd.0.event_tail: 240
dev.ivhd.0.event_head: 240
dev.ivhd.0.event_intr_count: 9


I've got a bunch of messages like these in the log:
[IO_PAGE_FAULT EVT: devId:0xa01 DomId:0x2 Addr:0x70f70400x30]
[IO_PAGE_FAULT EVT: devId:0xa01 DomId:0x2 Addr:0x2fdf0400x30]
(BTW, there seems to be a missing space before 0x30)

Now it's obvious what the problem is.
The controller has a RID of 0xa00 (its PCI locator is 10:0:0) but the requests
are coming from 0xa01.  The card actually has another PCI device at that
location (10:0:1), it's a vestigial IDE controller (in a sense that it is not
connected to any ports, so it cannot really provide any functionality).

I've passed both of the PCI device to bhyve and everything started working.
Thanks!

I've googled a little bit and it seems that this is not an uncommon problem.
Linux has a bunch of quirks for this and similar kinds of problems (they call it
PCI or DMA aliasing):
http://elixir.free-electrons.com/linux/v4.8.16/source/drivers/pci/quirks.c#L3701
The comments say that sometimes requests come from devices that do not exist at
all (or hidden beyond non-transparent bridges).

An interesting new world for me :-)

By the way, my understanding is that the alias IVHD entries in IVRS are designed
to report such aliasing issues when they are known to the platform.
But it appears to me that FreeBSD currently ignores those aliases.
Also, it looks like we are not making any use of IVHD entries beyond printing
them.  Based on the specification I think that we should use IVHD flags to set
up appropriate bits in the corresponding device table entries.  But we are not
doing that.  That's probably okay since we do not support the pass-though for
devices with complex properties anyway.

-- 
Andriy Gapon
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: problem with pass-through on amd

2017-11-16 Thread Andriy Gapon 
On 14/11/2017 06:22, Anish wrote:
> Also ivhd has fault interrupt enabled which is very helpful in debugging:
> 
> [root@ryzen /home/anish/FreeBSD/head]# vmstat -ia |grep ivh
> irq256: ivhd0:fault                    0          0
> irq257: ivhd1:fault                    0          0
> 

Anish,

I have made several interesting discoveries regarding my problem.
One of them is that actually there were some IOMMU log events:

dev.ivhd.0.event_tail: 240
dev.ivhd.0.event_head: 0
dev.ivhd.0.event_intr_count: 0

But there were no interrupts and the events are unconsumed and unreported.
I examined MSI configuration of the IOMMU PCI device and the address and data
registers were zeroed out.
I looked at dmesg and at the code and I realized why that happened.

So, first of all, I pre-load vmm via loader.conf.  Probably as a result of that
the ivhd device attaches before any bridges and buses on my system.  And
amdvi_alloc_intr_resources() does a rather untypical thing, it
configures an MSI for a PCI device by directly writing to its configuration
registers.  The PCI bus code is completely unaware of those changes and it wipes
them out in pci_add_child() -> pci_cfg_restore().

Also, I think that even if ivhd attached after the root PCI bus, then what it
does would be still unsafe.  I think that, for example, a suspend-resume cycle
would wipe out the MSI configuration too.
I think that in that case we should better use pci methods to configure MSI.

Now, why does ivhd attach before the root Host-PCI bridge and what can we do to
fix the order?

ivrs_drv.c has this code:
/*
 * Load this module at the end after PCI re-probing to configure interrupt.
 */
DRIVER_MODULE_ORDERED(ivhd, acpi, ivhd_driver, ivhd_devclass, 0, 0,
  SI_ORDER_ANY);

But apparently this SI_ORDER_ANY does not help much.
It affects only the driver registration order, but not the device probe and
attachment order.

This code is far more significant:
ivhd_devs[i] = BUS_ADD_CHILD(parent, 1, "ivhd", i);

ivhd passes 1 as the order.
This is a very high order for the acpi bus.
As a comment in acpi_probe_child() says:
/*
 * Create a placeholder device for this node.  Sort the
 * placeholder so that the probe/attach passes will run
 * breadth-first.  Orders less than ACPI_DEV_BASE_ORDER
 * are reserved for special objects (i.e., system
 * resources).
 */
where ACPI_DEV_BASE_ORDER is 100.

For example, order of the Host-PCI bridge on my system is 120.

I must note that this is important only of vmm is preloaded (which is probably
not an extremely rare case).  If vmm is loaded after the system is booted then,
of course, ivhd will be probed after the PCI buses / bridges.

-- 
Andriy Gapon
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: problem with pass-through on amd

2017-11-14 Thread Andriy Gapon 
On 14/11/2017 06:22, Anish wrote:
> Hi Andriy,
> Newer AMD IOMMU has EFR capability which is missing here, something like:
> ivhd0: PCI cap 0x190b640f@0x40 feature:19
> 
> Is this an old chipset?

Yes, this is an old 900 series chipset (990X).

> Also ivhd has fault interrupt enabled which is very
> helpful in debugging:
> 
> [root@ryzen /home/anish/FreeBSD/head]# vmstat -ia |grep ivh
> irq256: ivhd0:fault                    0          0
> irq257: ivhd1:fault                    0          0

irq256: ivhd0:fault0  0

> Another thing to worth checking is PCI config space of ivhd/IOMMU if you have
not already done so.

For what should I look there?

-- 
Andriy Gapon
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: problem with pass-through on amd

2017-11-13 Thread Anish 
Hi Andriy,
Newer AMD IOMMU has EFR capability which is missing here, something like:
ivhd0: PCI cap 0x190b640f@0x40 feature:19

Is this an old chipset? Also ivhd has fault interrupt enabled which is very
helpful in debugging:

[root@ryzen /home/anish/FreeBSD/head]# vmstat -ia |grep ivh
irq256: ivhd0:fault0  0
irq257: ivhd1:fault0  0


Regards,
Anish

On Mon, Nov 13, 2017 at 12:55 PM, Andriy Gapon  wrote:

> On 13/11/2017 22:41, Anish wrote:
> > Hi Andriy,
> >
> >>What's suspicious is that there are no interrupts either for the
> > ppt device on the host or the ahci device in the guest.
> >
> > I think most likely AMD IOMMU/amdvi is not enabled, by default it is
> disabled
> > and can be confirmed by sysctl.
> >
> > [root@ryzen /usr/home/anish]# sysctl hw.vmm.amdvi
> > hw.vmm.amdvi.domain_id: 3
> > hw.vmm.amdvi.disable_io_fault: 0
> > hw.vmm.amdvi.ptp_level: 4
> > hw.vmm.amdvi.host_ptp: 1
> > hw.vmm.amdvi.enable: 1
> > hw.vmm.amdvi.count: 2
> >
> > If hw.vmm.amdvi.enable is 0, set it and reload vmm and things should
> work.
>
>
> I followed your how-to, so I didn't make this mistake :-)
>
> $ sysctl hw.vmm.amdvi
> hw.vmm.amdvi.domain_id: 3
> hw.vmm.amdvi.disable_io_fault: 0
> hw.vmm.amdvi.ptp_level: 4
> hw.vmm.amdvi.host_ptp: 1
> hw.vmm.amdvi.enable: 1
> hw.vmm.amdvi.count: 1
>
> Could it be that AMD-Vi support is broken in the hardware or BIOS?
> Are there any checks that I could do?
>
> Here is what I see in dmesg, but it is not a full dump of IVRS, of course:
> $ dmesg | egrep -i 'ivrs|ivh|amd-vi|mmu'
> Table 'IVRS' at 0xbdafaf80
> ACPI: IVRS 0xBDAFAF80 F8 (v01 AMDRD890S   00202031 AMD
> )
> AMD-Vi IVRS VAsize = 64 PAsize = 52 GVAsize = 0 flags:0
> ivhd0:  on acpi0
> ivhd0: Unknown dev entry:0xff
> ivhd0: Flag:3e
> ivhd0: max supported paging level:7 restricting to: 4
> ivhd0: device supported range [0x0 - 0xb01]
> ivhd0: device [0xa0 - 0xa0]config:d7
> ivhd0: PCI cap 0x10b540f@0x40 feature:1
> pci0:  at device 0.2 (no driver attached)
>
> > On Mon, Nov 13, 2017 at 11:02 AM, Andriy Gapon  > > wrote:
> >
> >
> > I have a FreeBSD guest on a FreeBSD host, both amd64 head.
> > I decided to experiment with PCI pass-through and used an AHCI
> controller for
> > the experiment.  It seems like that partially works.  The guest
> detects the
> > controller and attaches a driver to it, the messages (reported
> capabilities,
> > etc) look identical to those on the host.  But the guest can not
> detect any
> > disks behind the controller.  From what I can tell, the guest has no
> problems
> > accessing PCI configuration registers and a memory mapped BAR.  Not
> sure about
> > the I/O ports.  What's suspicious is that there are no interrupts
> either for the
> > ppt device on the host or the ahci device in the guest.  Maybe that
> is what
> > causes the failure to see the disks.
> >
> > When the guest driver attaches to the controller I see these
> messages on the
> > host:
> > ppt0: attempting to allocate 1 MSI vectors (1 supported)
> > ppt0: using IRQ 265 for MSI
> > and in the guest:
> > ahci0: attempting to allocate 1 MSI vectors (1 supported)
> > ahci0: using IRQ 279 for MSI
> >
> > But vmstat -i does not report any interrupts.
> >
> > Just in case, the controller has some complex topology where the
> AHCI PCI device
> > is behind two PCI-PCI bridges from the main bus.  But I think that
> that should
> > not matter and the guest needs to see only the AHCI device and the
> host handles
> > the bridges.
> >
> > Here are some more messages from the guest:
> > ahci0:  port
> > 0x21c0-0x21c7,0x21c8-0x21cb,0x21d0-0x21d7,0x21d8-0x21db,0x21e0-0x21ef
> mem
> > 0xc000e000-0xc000efff irq 17 at device 9.0 on pci0
> > ahci0: attempting to allocate 1 MSI vectors (1 supported)
> > ahci0: using IRQ 279 for MSI
> > ahci0: AHCI v1.00 with 2 6Gbps ports, Port Multiplier supported with
> FBS
> > ahci0: Caps: 64bit NCQ SNTF AL 6Gbps PM FBS PMD SSC PSC 32cmd eSATA
> 2ports
> > ahcich0:  at channel 0 on ahci0
> > ahcich0: Caps: CPD ESP FBSCP
> > ahcich1:  at channel 1 on ahci0
> > ahcich1: Caps: CPD ESP FBSCP
> > ahcich0: AHCI reset...
> > ahcich0: SATA connect time=100us status=0123
> > ahcich0: AHCI reset: device found
> > ahcich1: AHCI reset...
> > ahcich1: SATA connect time=100us status=0123
> > ahcich1: AHCI reset: device found
> > ahcich1: AHCI reset: device ready after 100ms
> > ahcich1: Poll timeout on slot 1 port 15
> > ahcich1: is  cs 0002 ss  rs 0002 tfd 150
> serr 
> > cmd 00318017
> > ahcich0: AHCI reset: device ready after 200ms
> > ahcich0: Poll timeout on slot 1 port 15
> > ahcich0: is  cs 0002 ss  rs 0002 tfd 150
> serr 
> > cmd 00318017
> > And some mo

Re: problem with pass-through on amd

2017-11-13 Thread Andriy Gapon 
On 13/11/2017 22:41, Anish wrote:
> Hi Andriy,
> 
>>What's suspicious is that there are no interrupts either for the
> ppt device on the host or the ahci device in the guest.
> 
> I think most likely AMD IOMMU/amdvi is not enabled, by default it is disabled
> and can be confirmed by sysctl.
> 
> [root@ryzen /usr/home/anish]# sysctl hw.vmm.amdvi
> hw.vmm.amdvi.domain_id: 3
> hw.vmm.amdvi.disable_io_fault: 0
> hw.vmm.amdvi.ptp_level: 4
> hw.vmm.amdvi.host_ptp: 1
> hw.vmm.amdvi.enable: 1
> hw.vmm.amdvi.count: 2 
> 
> If hw.vmm.amdvi.enable is 0, set it and reload vmm and things should work.


I followed your how-to, so I didn't make this mistake :-)

$ sysctl hw.vmm.amdvi
hw.vmm.amdvi.domain_id: 3
hw.vmm.amdvi.disable_io_fault: 0
hw.vmm.amdvi.ptp_level: 4
hw.vmm.amdvi.host_ptp: 1
hw.vmm.amdvi.enable: 1
hw.vmm.amdvi.count: 1

Could it be that AMD-Vi support is broken in the hardware or BIOS?
Are there any checks that I could do?

Here is what I see in dmesg, but it is not a full dump of IVRS, of course:
$ dmesg | egrep -i 'ivrs|ivh|amd-vi|mmu'
Table 'IVRS' at 0xbdafaf80
ACPI: IVRS 0xBDAFAF80 F8 (v01 AMDRD890S   00202031 AMD  
)
AMD-Vi IVRS VAsize = 64 PAsize = 52 GVAsize = 0 flags:0
ivhd0:  on acpi0
ivhd0: Unknown dev entry:0xff
ivhd0: Flag:3e
ivhd0: max supported paging level:7 restricting to: 4
ivhd0: device supported range [0x0 - 0xb01]
ivhd0: device [0xa0 - 0xa0]config:d7
ivhd0: PCI cap 0x10b540f@0x40 feature:1
pci0:  at device 0.2 (no driver attached)

> On Mon, Nov 13, 2017 at 11:02 AM, Andriy Gapon  > wrote:
> 
> 
> I have a FreeBSD guest on a FreeBSD host, both amd64 head.
> I decided to experiment with PCI pass-through and used an AHCI controller 
> for
> the experiment.  It seems like that partially works.  The guest detects 
> the
> controller and attaches a driver to it, the messages (reported 
> capabilities,
> etc) look identical to those on the host.  But the guest can not detect 
> any
> disks behind the controller.  From what I can tell, the guest has no 
> problems
> accessing PCI configuration registers and a memory mapped BAR.  Not sure 
> about
> the I/O ports.  What's suspicious is that there are no interrupts either 
> for the
> ppt device on the host or the ahci device in the guest.  Maybe that is 
> what
> causes the failure to see the disks.
> 
> When the guest driver attaches to the controller I see these messages on 
> the
> host:
> ppt0: attempting to allocate 1 MSI vectors (1 supported)
> ppt0: using IRQ 265 for MSI
> and in the guest:
> ahci0: attempting to allocate 1 MSI vectors (1 supported)
> ahci0: using IRQ 279 for MSI
> 
> But vmstat -i does not report any interrupts.
> 
> Just in case, the controller has some complex topology where the AHCI PCI 
> device
> is behind two PCI-PCI bridges from the main bus.  But I think that that 
> should
> not matter and the guest needs to see only the AHCI device and the host 
> handles
> the bridges.
> 
> Here are some more messages from the guest:
> ahci0:  port
> 0x21c0-0x21c7,0x21c8-0x21cb,0x21d0-0x21d7,0x21d8-0x21db,0x21e0-0x21ef mem
> 0xc000e000-0xc000efff irq 17 at device 9.0 on pci0
> ahci0: attempting to allocate 1 MSI vectors (1 supported)
> ahci0: using IRQ 279 for MSI
> ahci0: AHCI v1.00 with 2 6Gbps ports, Port Multiplier supported with FBS
> ahci0: Caps: 64bit NCQ SNTF AL 6Gbps PM FBS PMD SSC PSC 32cmd eSATA 2ports
> ahcich0:  at channel 0 on ahci0
> ahcich0: Caps: CPD ESP FBSCP
> ahcich1:  at channel 1 on ahci0
> ahcich1: Caps: CPD ESP FBSCP
> ahcich0: AHCI reset...
> ahcich0: SATA connect time=100us status=0123
> ahcich0: AHCI reset: device found
> ahcich1: AHCI reset...
> ahcich1: SATA connect time=100us status=0123
> ahcich1: AHCI reset: device found
> ahcich1: AHCI reset: device ready after 100ms
> ahcich1: Poll timeout on slot 1 port 15
> ahcich1: is  cs 0002 ss  rs 0002 tfd 150 serr 
> 
> cmd 00318017
> ahcich0: AHCI reset: device ready after 200ms
> ahcich0: Poll timeout on slot 1 port 15
> ahcich0: is  cs 0002 ss  rs 0002 tfd 150 serr 
> 
> cmd 00318017
> And some more...
> 
> Any ideas or suggestions?
> Or maybe some additional information from me?
> 
> Thank you!
> 
> --
> Andriy Gapon
> ___
> freebsd-virtualization@freebsd.org
>  mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> 
> To unsubscribe, send any mail to
> "freebsd-virtualization-unsubscr...@freebsd.org
> "
> 
> 


-- 
Andriy Gapon
___

Re: problem with pass-through on amd

2017-11-13 Thread Anish 
Hi Andriy,

>What's suspicious is that there are no interrupts either for the
ppt device on the host or the ahci device in the guest.

I think most likely AMD IOMMU/amdvi is not enabled, by default it is
disabled and can be confirmed by sysctl.

[root@ryzen /usr/home/anish]# sysctl hw.vmm.amdvi
hw.vmm.amdvi.domain_id: 3
hw.vmm.amdvi.disable_io_fault: 0
hw.vmm.amdvi.ptp_level: 4
hw.vmm.amdvi.host_ptp: 1
hw.vmm.amdvi.enable: 1
hw.vmm.amdvi.count: 2

If hw.vmm.amdvi.enable is 0, set it and reload vmm and things should work.

Regards,
Anish


On Mon, Nov 13, 2017 at 11:02 AM, Andriy Gapon  wrote:

>
> I have a FreeBSD guest on a FreeBSD host, both amd64 head.
> I decided to experiment with PCI pass-through and used an AHCI controller
> for
> the experiment.  It seems like that partially works.  The guest detects the
> controller and attaches a driver to it, the messages (reported
> capabilities,
> etc) look identical to those on the host.  But the guest can not detect any
> disks behind the controller.  From what I can tell, the guest has no
> problems
> accessing PCI configuration registers and a memory mapped BAR.  Not sure
> about
> the I/O ports.  What's suspicious is that there are no interrupts either
> for the
> ppt device on the host or the ahci device in the guest.  Maybe that is what
> causes the failure to see the disks.
>
> When the guest driver attaches to the controller I see these messages on
> the host:
> ppt0: attempting to allocate 1 MSI vectors (1 supported)
> ppt0: using IRQ 265 for MSI
> and in the guest:
> ahci0: attempting to allocate 1 MSI vectors (1 supported)
> ahci0: using IRQ 279 for MSI
>
> But vmstat -i does not report any interrupts.
>
> Just in case, the controller has some complex topology where the AHCI PCI
> device
> is behind two PCI-PCI bridges from the main bus.  But I think that that
> should
> not matter and the guest needs to see only the AHCI device and the host
> handles
> the bridges.
>
> Here are some more messages from the guest:
> ahci0:  port
> 0x21c0-0x21c7,0x21c8-0x21cb,0x21d0-0x21d7,0x21d8-0x21db,0x21e0-0x21ef mem
> 0xc000e000-0xc000efff irq 17 at device 9.0 on pci0
> ahci0: attempting to allocate 1 MSI vectors (1 supported)
> ahci0: using IRQ 279 for MSI
> ahci0: AHCI v1.00 with 2 6Gbps ports, Port Multiplier supported with FBS
> ahci0: Caps: 64bit NCQ SNTF AL 6Gbps PM FBS PMD SSC PSC 32cmd eSATA 2ports
> ahcich0:  at channel 0 on ahci0
> ahcich0: Caps: CPD ESP FBSCP
> ahcich1:  at channel 1 on ahci0
> ahcich1: Caps: CPD ESP FBSCP
> ahcich0: AHCI reset...
> ahcich0: SATA connect time=100us status=0123
> ahcich0: AHCI reset: device found
> ahcich1: AHCI reset...
> ahcich1: SATA connect time=100us status=0123
> ahcich1: AHCI reset: device found
> ahcich1: AHCI reset: device ready after 100ms
> ahcich1: Poll timeout on slot 1 port 15
> ahcich1: is  cs 0002 ss  rs 0002 tfd 150 serr
> 
> cmd 00318017
> ahcich0: AHCI reset: device ready after 200ms
> ahcich0: Poll timeout on slot 1 port 15
> ahcich0: is  cs 0002 ss  rs 0002 tfd 150 serr
> 
> cmd 00318017
> And some more...
>
> Any ideas or suggestions?
> Or maybe some additional information from me?
>
> Thank you!
>
> --
> Andriy Gapon
> ___
> freebsd-virtualization@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to "freebsd-virtualization-
> unsubscr...@freebsd.org"
>
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"