On Sat, Jun 12, 2010 at 7:33 PM, Blue Swirl <blauwir...@gmail.com> wrote: > On Sat, Jun 12, 2010 at 3:58 PM, Paul Brook <p...@codesourcery.com> wrote: >>> I think message passing interrupts >>> are only used in bus based systems, like PCI or UPA/JBUS etc. I don't >>> know how LAPIC/IOAPIC bus works, it could be similar. >> >> PCI Message Signalled Interrupts use a regular data write, and we model it >> exactly that way. Under normal circumstances you program the device to write >> to memory mapped APIC control registers, but there's no real reason why you >> couldn't write to RAM. >> >> LAPIC/IOAPIC have their own bus. On some hardware this is multiplexed over >> the >> main system bus, but logically it is a separate interconnect. >> >> The fact that these a bus based systems suggests against using qemu_irq, >> which >> is an inherently point-point system. >> >>> > How is a >>> > receiver meant to know for format of the message being delivered, and who >>> > it's intended for? >>> >>> This depends on the architecture. On Sparc64 the message is specified >>> by the guest or OF. >>>... >>> In real HW (MSI or Sparc64 mondo interrupts) a bus delivers the >>> message. But our buses only handle address and data lines. >> >> IIUC you're trying to use qemu_irq as a generic interconnect between devices. >> I think that's going to cause more problems than it solves. If a bus has >> additional message passing capabilities then this can be part of the bus >> interface. >> >> If two devices need a private communication channel then we probably want >> some >> implementation agnostic way of defining this link. Connecting LAPIC to >> IOAPIC >> is a potential example of this. However this needs to be done in a type-safe >> manner. DMA channels may be another potential use of this linking. We'd want >> to avoid connecting a DMA channel to an APIC. >> >> [*] A simple unidirectional dma request line is suitable for qmu_irq. A DMA >> system that transfers data outside of memory read/write transactions is not. >> e.g. ISA effectively defines a regular memory bus plus 8 bidirectional data >> streams (aka DMA channels). These are multiplexed over the same physical >> pins, >> but logically distinct. The DMA channels could either be bundled up into the >> ISA bus interface, or modelled as links between devices and the DMA >> controller. > > Very very interesting. There's some out of band data related to DMA > (bus errors, IOMMU faults, ISA DREQ) which hasn't been covered in > previous Generic DMA proposals. Maybe all we need is a generic side > band link, which would be usable for point to point (qemu_irq, > coalescing, messages) and bus channels?
I think we could solve all problems (well, maybe not world peace, yet) by switching to message based system for all of DMA and IRQs. Each device would have a message input port and way to output messages. Examples: Zero copy memory access from device D1 to D2 to host memory (D3) with access broken to page length units and errors occurring on the last byte: D1 send_msg(ID, MSG_MEM_WRITE, DMA address, length) -> D2 D2 send_msg(ID, MSG_MEM_WRITE, DMA address2, length) -> D3 D3 send_replymsg(ID, MSG_MEM_PTR, host address, 4096) -> D2 D2 send_replymsg(ID, MSG_MEM_PTR, host address, 4096) -> D1 D3 send_replymsg(ID, MSG_MEM_PTR, host address, length - 4096 - 1) -> D2 D2 send_replymsg(ID, MSG_MEM_PTR, host address, length - 4096 - 1) -> D1 D3 send_replymsg(ID, MSG_MEM_ERROR, DMA address2 + length - 1, 1, status) -> D2 D2 send_replymsg(ID, MSG_MEM_ERROR, DMA address + length - 1, 1, status) -> D1 IRQ delivery chain D1->D2->D3 with coalescing, messages, delivery reporting and EOI: D1 send_msg(ID, MSG_IRQ_RAISE, payload) -> D2 D2 send_msg(ID, MSG_IRQ_RAISE, payload) -> D3 D3 send_replymsg(ID, MSG_IRQ_STATUS, MI_COALESCED) -> D2 D2 send_replymsg(ID, MSG_IRQ_STATUS, MI_COALESCED) -> D1 D3 send_replymsg(ID, MSG_IRQ_STATUS, MI_DELIVERED) -> D2 D2 send_replymsg(ID, MSG_IRQ_STATUS, MI_DELIVERED) -> D1 D3 send_replymsg(ID, MSG_IRQ_STATUS, MI_EOI) -> D2 D2 send_replymsg(ID, MSG_IRQ_STATUS, MI_EOI) -> D1 What else do we want? :-)
