On Mon, 18 Jun 2018 14:44:56 +1000
Benjamin Herrenschmidt <b...@kernel.crashing.org> wrote:
> On Mon, 2018-06-18 at 12:13 +1000, Alexey Kardashevskiy wrote:
> > On Sat, 16 Jun 2018 11:05:19 +1000
> > Benjamin Herrenschmidt <b...@kernel.crashing.org> wrote:
> >
> > > On Fri, 2018-06-01 at 18:10 +1000, Alexey Kardashevskiy wrote:
> > > > These are found in POWER9 chips. Right now these PHBs have unknown type
> > > > so changing it to PHB4 won't make much of a difference except enabling
> > > > sketchy bypass for POWER9 as this does below.
> > >
> > > And that will break on multi-chip systems since P9 doesn't have the
> > > memory contiguous (it has the chip ID in the top bits).
> >
> >
> > This did not break mine and it is hard to see why it would break at all
> > if we use 1G pages and the maximum we need to cover is 48 bits (this
> > is what we are trying to support here - all these gpus, right?), or is
> > it more now? If so, I have posted v2 of tce multilevel dynamic
> > allocation which helps with enormous tce tables.
>
> The whole point of sketchy bypass is to deal with devices with small
> amount of DMA bits... most Radeon's have 40 for example. So that won't
> work terribly well.
40 with 1GB pages needs 40-30=10 bits i.e. (1<<10)*8 = 8KB for the
entire TCE table. I am definitely missing something here, what is it?
>
> Cheers,
> Ben.
>
> >
> >
> > >
> > > Russell is working on a different implementation that should be much
> > > more imune to the system physical memory layout.
> > >
> > > > Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru>
> > > > ---
> > > > arch/powerpc/platforms/powernv/pci.h | 1 +
> > > > arch/powerpc/platforms/powernv/pci-ioda.c | 5 ++++-
> > > > 2 files changed, 5 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/arch/powerpc/platforms/powernv/pci.h
> > > > b/arch/powerpc/platforms/powernv/pci.h
> > > > index eada4b6..1408247 100644
> > > > --- a/arch/powerpc/platforms/powernv/pci.h
> > > > +++ b/arch/powerpc/platforms/powernv/pci.h
> > > > @@ -23,6 +23,7 @@ enum pnv_phb_model {
> > > > PNV_PHB_MODEL_UNKNOWN,
> > > > PNV_PHB_MODEL_P7IOC,
> > > > PNV_PHB_MODEL_PHB3,
> > > > + PNV_PHB_MODEL_PHB4,
> > > > PNV_PHB_MODEL_NPU,
> > > > PNV_PHB_MODEL_NPU2,
> > > > };
> > > > diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c
> > > > b/arch/powerpc/platforms/powernv/pci-ioda.c
> > > > index 9239142..66c2804 100644
> > > > --- a/arch/powerpc/platforms/powernv/pci-ioda.c
> > > > +++ b/arch/powerpc/platforms/powernv/pci-ioda.c
> > > > @@ -1882,7 +1882,8 @@ static int pnv_pci_ioda_dma_set_mask(struct
> > > > pci_dev *pdev, u64 dma_mask)
> > > > if (dma_mask >> 32 &&
> > > > dma_mask > (memory_hotplug_max() + (1ULL << 32)) &&
> > > > pnv_pci_ioda_pe_single_vendor(pe) &&
> > > > - phb->model == PNV_PHB_MODEL_PHB3) {
> > > > + (phb->model == PNV_PHB_MODEL_PHB3 ||
> > > > + phb->model == PNV_PHB_MODEL_PHB4)) {
> > > > /* Configure the bypass mode */
> > > > rc = pnv_pci_ioda_dma_64bit_bypass(pe);
> > > > if (rc)
> > > > @@ -3930,6 +3931,8 @@ static void __init pnv_pci_init_ioda_phb(struct
> > > > device_node *np,
> > > > phb->model = PNV_PHB_MODEL_P7IOC;
> > > > else if (of_device_is_compatible(np, "ibm,power8-pciex"))
> > > > phb->model = PNV_PHB_MODEL_PHB3;
> > > > + else if (of_device_is_compatible(np, "ibm,power9-pciex"))
> > > > + phb->model = PNV_PHB_MODEL_PHB4;
> > > > else if (of_device_is_compatible(np, "ibm,power8-npu-pciex"))
> > > > phb->model = PNV_PHB_MODEL_NPU;
> > > > else if (of_device_is_compatible(np, "ibm,power9-npu-pciex"))
> > > >
> >
> >
> >
> > --
> > Alexey
--
Alexey
