Re: [net-next PATCH 0/8] configuration support for switch headers & phy

[Sunil Kovvuri]

On Thu, Mar 25, 2021 at 6:51 PM Andrew Lunn  wrote:
>
> On Thu, Mar 25, 2021 at 06:32:12PM +0530, Sunil Kovvuri wrote:
> > On Thu, Mar 25, 2021 at 6:20 PM Andrew Lunn  wrote:
> > >
> > > > > So you completely skipped how this works with mv88e6xxx or
> > > > > prestera. If you need this private flag for some out of mainline
> > > > > Marvell SDK, it is very unlikely to be accepted.
> > > > >
> > > > > Andrew
> > > >
> > > > What we are trying to do here has no dependency on DSA drivers and
> > > > neither impacts that functionality.
> > >
> > > So this is an indirect way of saying: Yes, this is for some out of
> > > mainline Marvell SDK.
> > >
> > > > Here we are just notifying the HW to parse the packets properly.
> > >
> > > But the correct way for this to happen is probably some kernel
> > > internal API between the MAC and the DSA driver. Mainline probably has
> > > no need for this private flag.
> > >
> > >Andrew
> >
> > Didn't get why you say so.
> > HW expects some info from SW to do the packet parsing properly and
> > this is specific to this hardware.
>
> Anything which needs out of mainline code is going to be rejected,
> unless you can show there is an in mainline use case as well. Which is
> why i keep pointing you to mv88e6xxx and prestira. That provides the
> necessary {E}DSA tags. You have an mv88e6xxx and prestira switch being
> controlled by Linux with DSA tagged frames flowing out of it into your
> MAC driver. What is the big picture use case that requires this
> private flag to enable DSA parsing? Why don't the MAC driver and the
> DSA driver just talk to each other, and setup the parsing?
>
> Andrew

The usecase is simple, unlike DSA tag, this 4byte FDSA tag doesn't
have a ethertype,
so HW cannot recognize this header. If such packers arise, then HW parsing will
fail and RSS will not work.

Hypothetically if we introduce some communication between MAC driver
and DSA driver,
wouldn't that also become specific to the device, what generic usecase
that communication
will have ?

Thanks,
Sunil.

Re: [net-next PATCH 0/8] configuration support for switch headers & phy

[Sunil Kovvuri]

On Thu, Mar 25, 2021 at 6:20 PM Andrew Lunn  wrote:
>
> > > So you completely skipped how this works with mv88e6xxx or
> > > prestera. If you need this private flag for some out of mainline
> > > Marvell SDK, it is very unlikely to be accepted.
> > >
> > > Andrew
> >
> > What we are trying to do here has no dependency on DSA drivers and
> > neither impacts that functionality.
>
> So this is an indirect way of saying: Yes, this is for some out of
> mainline Marvell SDK.
>
> > Here we are just notifying the HW to parse the packets properly.
>
> But the correct way for this to happen is probably some kernel
> internal API between the MAC and the DSA driver. Mainline probably has
> no need for this private flag.
>
>Andrew

Didn't get why you say so.
HW expects some info from SW to do the packet parsing properly and
this is specific to this hardware.
How can we generalize this ?
It's not just the DSA tags, the requirement is also for packets with
Higig header ie when system is connected to a switch
which appends Higig2 header to all pkts.

Thanks,
Sunil.

Re: [net-next PATCH 0/8] configuration support for switch headers & phy

[Sunil Kovvuri]

> > > Hi Hariprasad
> > >
> > > Private flags sound very wrong here. I would expect to see some 
> > > integration
> > > between the switchdev/DSA driver and the MAC driver.
> > > Please show how this works in combination with drivers/net/dsa/mv88e6xxx
> > > or drivers/net/ethernet/marvell/prestera.
> > >
> >   Octeontx2 silicon supports NPC (network parser and cam) unit , 
> > through which packet parsing and packet classification is achieved.
> >   Packet parsing extracting different fields from each layer.
> > DMAC + SMAC  --> LA
> >  VLAN ID --> LB
> >  SIP + DIP --> LC
> > TCP SPORT + 
> > DPORT --> LD
> > And packet classification is achieved through  flow identification in 
> > key extraction and mcam search key . User can install mcam rules
> > With action as
> >   forward packet to PF and to receive  queue 0
> >   forward packet to VF and  with as RSS ( Receive side scaling)
> >   drop the packet
> >   etc..
> >
> >Now with switch header ( EDSA /FDSA) and HIGIG2 appended to regular 
> > packet , NPC can not parse these
> >Ingress packets as these headers does not have fixed headers. To achieve 
> > this Special PKIND( port kind) is allocated in hardware
> >which will help NPC to parse the packets.
> >
> >  For example incase of EDSA 8 byte header which is placed right after SMAC 
> > , special PKIND reserved for EDSA helps NPC to
> >  Identify the  input packet is EDSA . Such that NPC can extract fields in 
> > this header and forward to
> >  Parse rest of the headers.
> >
> >  Same is the case with higig2 header where 16 bytes header is placed at 
> > start of the packet.
> >
> > In this case private flags helps user to configure interface in EDSA/FDSA 
> > or HIGIG2. Such that special
> > PKIND reserved for that header are assigned to the interface.  The scope of 
> > the patch series is how
> > User can configure interface mode as switch header(HIGIG2/EDSA etc) .In our 
> > case no DSA logical
> > Ports are created as these headers can be stripped by NPC.
>
> So you completely skipped how this works with mv88e6xxx or
> prestera. If you need this private flag for some out of mainline
> Marvell SDK, it is very unlikely to be accepted.
>
> Andrew

What we are trying to do here has no dependency on DSA drivers and
neither impacts that functionality.
Here we are just notifying the HW to parse the packets properly.

Thanks,
Sunil.

Re: [PATCH net-next] octeontx2: fix -Wnonnull warning

[Sunil Kovvuri]

On Tue, Mar 23, 2021 at 6:26 PM Arnd Bergmann  wrote:
>
> From: Arnd Bergmann 
>
> When compile testing this driver on a platform on which probe() is
> known to fail at compile time, gcc warns about the cgx_lmactype_string[]
> array being uninitialized:
>
> In function 'strncpy',
> inlined from 'link_status_user_format' at 
> /git/arm-soc/drivers/net/ethernet/marvell/octeontx2/af/cgx.c:838:2,
> inlined from 'cgx_link_change_handler' at 
> /git/arm-soc/drivers/net/ethernet/marvell/octeontx2/af/cgx.c:853:2:
> include/linux/fortify-string.h:27:30: error: argument 2 null where non-null 
> expected [-Werror=nonnull]
>27 | #define __underlying_strncpy __builtin_strncpy
>
> Address this by turning the runtime initialization into a fixed array,
> which should also produce better code.
>
> Signed-off-by: Arnd Bergmann 
> ---
>  .../net/ethernet/marvell/octeontx2/af/cgx.c   | 60 +--
>  1 file changed, 28 insertions(+), 32 deletions(-)
>
> diff --git a/drivers/net/ethernet/marvell/octeontx2/af/cgx.c 
> b/drivers/net/ethernet/marvell/octeontx2/af/cgx.c
> index 9caa375d01b1..ea5a033a1d0b 100644
> --- a/drivers/net/ethernet/marvell/octeontx2/af/cgx.c
> +++ b/drivers/net/ethernet/marvell/octeontx2/af/cgx.c
> @@ -30,10 +30,35 @@
>  static LIST_HEAD(cgx_list);
>
>  /* Convert firmware speed encoding to user format(Mbps) */
> -static u32 cgx_speed_mbps[CGX_LINK_SPEED_MAX];
> +static const u32 cgx_speed_mbps[CGX_LINK_SPEED_MAX] = {
> +   [CGX_LINK_NONE] = 0,
> +   [CGX_LINK_10M] = 10,
> +   [CGX_LINK_100M] = 100,
> +   [CGX_LINK_1G] = 1000,
> +   [CGX_LINK_2HG] = 2500,
> +   [CGX_LINK_5G] = 5000,
> +   [CGX_LINK_10G] = 1,
> +   [CGX_LINK_20G] = 2,
> +   [CGX_LINK_25G] = 25000,
> +   [CGX_LINK_40G] = 4,
> +   [CGX_LINK_50G] = 5,
> +   [CGX_LINK_80G] = 8,
> +   [CGX_LINK_100G] = 10,
> +};
>
>  /* Convert firmware lmac type encoding to string */
> -static char *cgx_lmactype_string[LMAC_MODE_MAX];
> +static const char *cgx_lmactype_string[LMAC_MODE_MAX] = {
> +   [LMAC_MODE_SGMII] = "SGMII",
> +   [LMAC_MODE_XAUI] = "XAUI",
> +   [LMAC_MODE_RXAUI] = "RXAUI",
> +   [LMAC_MODE_10G_R] = "10G_R",
> +   [LMAC_MODE_40G_R] = "40G_R",
> +   [LMAC_MODE_QSGMII] = "QSGMII",
> +   [LMAC_MODE_25G_R] = "25G_R",
> +   [LMAC_MODE_50G_R] = "50G_R",
> +   [LMAC_MODE_100G_R] = "100G_R",
> +   [LMAC_MODE_USXGMII] = "USXGMII",
> +};
>
>  /* CGX PHY management internal APIs */
>  static int cgx_fwi_link_change(struct cgx *cgx, int lmac_id, bool en);
> @@ -657,34 +682,6 @@ int cgx_fwi_cmd_generic(u64 req, u64 *resp, struct cgx 
> *cgx, int lmac_id)
> return err;
>  }
>
> -static inline void cgx_link_usertable_init(void)
> -{
> -   cgx_speed_mbps[CGX_LINK_NONE] = 0;
> -   cgx_speed_mbps[CGX_LINK_10M] = 10;
> -   cgx_speed_mbps[CGX_LINK_100M] = 100;
> -   cgx_speed_mbps[CGX_LINK_1G] = 1000;
> -   cgx_speed_mbps[CGX_LINK_2HG] = 2500;
> -   cgx_speed_mbps[CGX_LINK_5G] = 5000;
> -   cgx_speed_mbps[CGX_LINK_10G] = 1;
> -   cgx_speed_mbps[CGX_LINK_20G] = 2;
> -   cgx_speed_mbps[CGX_LINK_25G] = 25000;
> -   cgx_speed_mbps[CGX_LINK_40G] = 4;
> -   cgx_speed_mbps[CGX_LINK_50G] = 5;
> -   cgx_speed_mbps[CGX_LINK_80G] = 8;
> -   cgx_speed_mbps[CGX_LINK_100G] = 10;
> -
> -   cgx_lmactype_string[LMAC_MODE_SGMII] = "SGMII";
> -   cgx_lmactype_string[LMAC_MODE_XAUI] = "XAUI";
> -   cgx_lmactype_string[LMAC_MODE_RXAUI] = "RXAUI";
> -   cgx_lmactype_string[LMAC_MODE_10G_R] = "10G_R";
> -   cgx_lmactype_string[LMAC_MODE_40G_R] = "40G_R";
> -   cgx_lmactype_string[LMAC_MODE_QSGMII] = "QSGMII";
> -   cgx_lmactype_string[LMAC_MODE_25G_R] = "25G_R";
> -   cgx_lmactype_string[LMAC_MODE_50G_R] = "50G_R";
> -   cgx_lmactype_string[LMAC_MODE_100G_R] = "100G_R";
> -   cgx_lmactype_string[LMAC_MODE_USXGMII] = "USXGMII";
> -}
> -
>  static int cgx_link_usertable_index_map(int speed)
>  {
> switch (speed) {
> @@ -826,7 +823,7 @@ static inline void link_status_user_format(u64 lstat,
>struct cgx_link_user_info *linfo,
>struct cgx *cgx, u8 lmac_id)
>  {
> -   char *lmac_string;
> +   const char *lmac_string;
>
> linfo->link_up = FIELD_GET(RESP_LINKSTAT_UP, lstat);
> linfo->full_duplex = FIELD_GET(RESP_LINKSTAT_FDUPLEX, lstat);
> @@ -1375,7 +1372,6 @@ static int cgx_probe(struct pci_dev *pdev, const struct 
> pci_device_id *id)
>
> list_add(>cgx_list, _list);
>
> -   cgx_link_usertable_init();
>
> cgx_populate_features(cgx);
>
> --
> 2.29.2
>

Looks good to me, thanks for the fix.

Acked-by: Sunil Goutham 

Re: [PATCH] octeontx2-af: Fix memory leak of object buf

[Sunil Kovvuri]

On Tue, Mar 23, 2021 at 6:07 PM Colin King  wrote:
>
> From: Colin Ian King 
>
> Currently the error return path when lfs fails to allocate is not free'ing
> the memory allocated to buf. Fix this by adding the missing kfree.
>
> Addresses-Coverity: ("Resource leak")
> Fixes: f7884097141b ("octeontx2-af: Formatting debugfs entry rsrc_alloc.")
> Signed-off-by: Colin Ian King 
> ---
>  drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c 
> b/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c
> index 8ec17ee72b5d..9bf8eaabf9ab 100644
> --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c
> +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c
> @@ -253,8 +253,10 @@ static ssize_t rvu_dbg_rsrc_attach_status(struct file 
> *filp,
> return -ENOSPC;
>
> lfs = kzalloc(lf_str_size, GFP_KERNEL);
> -   if (!lfs)
> +   if (!lfs) {
> +   kfree(buf);
> return -ENOMEM;
> +   }
> off +=  scnprintf([off], buf_size - 1 - off, "%-*s", lf_str_size,
>   "pcifunc");
> for (index = 0; index < BLK_COUNT; index++)
> --
> 2.30.2
>

Thanks for the fix,
Acked-by: Sunil Goutham 

Re: [net-next PATCH] octeontx2-af: cn10k: Fixes CN10K RPM reference issue

[Sunil Kovvuri]

On Mon, Feb 15, 2021 at 11:27 PM Geetha sowjanya  wrote:
>
> This patch fixes references to uninitialized variables and
> debugfs entry name for CN10K platform and HW_TSO flag check.
>
> Signed-off-by: Geetha sowjanya 
> Signed-off-by: Sunil Goutham 
>
> This patch fixes the bug introduced by the commit
> 3ad3f8f93c81 ("octeontx2-af: cn10k: MAC internal loopback support".
> These changes are not yet merged into net branch, hence submitting
> to net-next.
>
> ---
>  drivers/net/ethernet/marvell/octeontx2/af/rvu_cgx.c   |  2 ++
>  .../net/ethernet/marvell/octeontx2/af/rvu_debugfs.c   |  2 +-
>  .../net/ethernet/marvell/octeontx2/nic/otx2_txrx.c| 11 ++-
>  3 files changed, 9 insertions(+), 6 deletions(-)
>

> diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c 
> b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c
> index 3f778fc054b5..22ec03a618b1 100644
> --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c
> +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c
> @@ -816,22 +816,23 @@ static bool is_hw_tso_supported(struct otx2_nic *pfvf,
>  {
> int payload_len, last_seg_size;
>
> +   if (test_bit(HW_TSO, >hw.cap_flag))
> +   return true;
> +
> +   /* On 96xx A0, HW TSO not supported */
> +   if (!is_96xx_B0(pfvf->pdev))
> +   return false;
>
> /* HW has an issue due to which when the payload of the last LSO
>  * segment is shorter than 16 bytes, some header fields may not
>  * be correctly modified, hence don't offload such TSO segments.
>  */
> -   if (!is_96xx_B0(pfvf->pdev))
> -   return true;
>
> payload_len = skb->len - (skb_transport_offset(skb) + 
> tcp_hdrlen(skb));
> last_seg_size = payload_len % skb_shinfo(skb)->gso_size;
> if (last_seg_size && last_seg_size < 16)
> return false;
>
> -   if (!test_bit(HW_TSO, >hw.cap_flag))
> -   return false;
> -
> return true;
>  }

The HW_TSO flag should not be set for B0 silicon as well, otherwise
the checks related to
HW issue mentioned above will not come into effect.

Thanks,
Sunil.

Re: [PATCH] octeontx2-af: fix memory leak of lmac and lmac->name

[Sunil Kovvuri]

On Thu, Jan 7, 2021 at 6:11 PM Colin King  wrote:
>
> From: Colin Ian King 
>
> Currently the error return paths don't kfree lmac and lmac->name
> leading to some memory leaks.  Fix this by adding two error return
> paths that kfree these objects
>
> Addresses-Coverity: ("Resource leak")
> Fixes: 1463f382f58d ("octeontx2-af: Add support for CGX link management")
> Signed-off-by: Colin Ian King 
> ---

Thanks for the fix, looks good to me.

Sunil.

Re: [PATCH net-next] octeontx2-af: debugfs: delete dead code

[Sunil Kovvuri]

On Wed, Dec 2, 2020 at 12:28 PM Dan Carpenter  wrote:
>
> These debugfs never return NULL so all this code will never be run.
>
> In the normal case, (and in this case particularly), the debugfs
> functions are not supposed to be checked for errors so all this error
> checking code can be safely deleted.
>
> Signed-off-by: Dan Carpenter 
> ---

Thanks for the changes.

Re: [PATCH v2 net-next 3/3] octeontx2-af: Add devlink health reporters for NIX

[Sunil Kovvuri]

On Sat, Nov 7, 2020 at 2:28 AM Saeed Mahameed  wrote:
>
> On Fri, 2020-11-06 at 00:59 +0530, Sunil Kovvuri wrote:
> > > > > > Output:
> > > > > >  # ./devlink health
> > > > > >  pci/0002:01:00.0:
> > > > > >reporter npa
> > > > > >  state healthy error 0 recover 0
> > > > > >reporter nix
> > > > > >  state healthy error 0 recover 0
> > > > > >  # ./devlink  health dump show pci/0002:01:00.0 reporter nix
> > > > > >   NIX_AF_GENERAL:
> > > > > >  Memory Fault on NIX_AQ_INST_S read: 0
> > > > > >  Memory Fault on NIX_AQ_RES_S write: 0
> > > > > >  AQ Doorbell error: 0
> > > > > >  Rx on unmapped PF_FUNC: 0
> > > > > >  Rx multicast replication error: 0
> > > > > >  Memory fault on NIX_RX_MCE_S read: 0
> > > > > >  Memory fault on multicast WQE read: 0
> > > > > >  Memory fault on mirror WQE read: 0
> > > > > >  Memory fault on mirror pkt write: 0
> > > > > >  Memory fault on multicast pkt write: 0
> > > > > >NIX_AF_RAS:
> > > > > >  Poisoned data on NIX_AQ_INST_S read: 0
> > > > > >  Poisoned data on NIX_AQ_RES_S write: 0
> > > > > >  Poisoned data on HW context read: 0
> > > > > >  Poisoned data on packet read from mirror buffer: 0
> > > > > >  Poisoned data on packet read from mcast buffer: 0
> > > > > >  Poisoned data on WQE read from mirror buffer: 0
> > > > > >  Poisoned data on WQE read from multicast buffer: 0
> > > > > >  Poisoned data on NIX_RX_MCE_S read: 0
> > > > > >NIX_AF_RVU:
> > > > > >  Unmap Slot Error: 0
> > > > > >
> > > > >
> > > > > Now i am a little bit skeptic here, devlink health reporter
> > > > > infrastructure was
> > > > > never meant to deal with dump op only, the main purpose is to
> > > > > diagnose/dump and recover.
> > > > >
> > > > > especially in your use case where you only report counters, i
> > > > > don't
> > > > > believe
> > > > > devlink health dump is a proper interface for this.
> > > > These are not counters. These are error interrupts raised by HW
> > > > blocks.
> > > > The count is provided to understand on how frequently the errors
> > > > are
> > > > seen.
> > > > Error recovery for some of the blocks happen internally. That is
> > > > the
> > > > reason,
> > > > Currently only dump op is added.
> > >
> > > So you are counting these events in driver, sounds like a counter
> > > to
> > > me, i really think this shouldn't belong to devlink, unless you
> > > really
> > > utilize devlink health ops for actual reporting and recovery.
> > >
> > > what's wrong with just dumping these counters to ethtool ?
> >
> > This driver is a administrative driver which handles all the
> > resources
> > in the system and doesn't do any IO.
> > NIX and NPA are key co-processor blocks which this driver handles.
> > With NIX and NPA, there are pieces
> > which gets attached to a PCI device to make it a networking device.
> > We
> > have netdev drivers registered to this
> > networking device. Some more information about the drivers is
> > available at
> > https://www.kernel.org/doc/html/latest/networking/device_drivers/ethernet/marvell/octeontx2.html
> >
> > So we don't have a netdev here to report these co-processor block
> > level errors over ethtool.
> >
>
> but AF driver can't be standalone to operate your hw, it must have a
> PF/VF with netdev interface to do io, so even if your model is modular,
> a common user of this driver will always see a netdev.
>

That's right, user will always see a netdev, but
The co-processor blocks are like this
- Each co-processor has two parts, AF unit and LF units (local function)
- Each of the co-processor can have multiple LFs, incase of NIX
co-processor, each of the LF provides RQ, SQ, CQs etc.
- So the AF driver handles the co-processor's AF unit and upon
receiving requests from PF/VF attaches the LFs to them, so that they
can do network IO.
- Within co-processor, AF unit specific errors (global) are reported
to AF driver and LF specific errors are reported to netdev driver.
- There can be 10s of netdev driver instances in the system, so these
AF unit global errors cannot be routed and shown in one of the
netdev's ethtool.

Thanks,
Sunil.

Re: [PATCH v2 net-next 3/3] octeontx2-af: Add devlink health reporters for NIX

[Sunil Kovvuri]

> > > > Output:
> > > >  # ./devlink health
> > > >  pci/0002:01:00.0:
> > > >reporter npa
> > > >  state healthy error 0 recover 0
> > > >reporter nix
> > > >  state healthy error 0 recover 0
> > > >  # ./devlink  health dump show pci/0002:01:00.0 reporter nix
> > > >   NIX_AF_GENERAL:
> > > >  Memory Fault on NIX_AQ_INST_S read: 0
> > > >  Memory Fault on NIX_AQ_RES_S write: 0
> > > >  AQ Doorbell error: 0
> > > >  Rx on unmapped PF_FUNC: 0
> > > >  Rx multicast replication error: 0
> > > >  Memory fault on NIX_RX_MCE_S read: 0
> > > >  Memory fault on multicast WQE read: 0
> > > >  Memory fault on mirror WQE read: 0
> > > >  Memory fault on mirror pkt write: 0
> > > >  Memory fault on multicast pkt write: 0
> > > >NIX_AF_RAS:
> > > >  Poisoned data on NIX_AQ_INST_S read: 0
> > > >  Poisoned data on NIX_AQ_RES_S write: 0
> > > >  Poisoned data on HW context read: 0
> > > >  Poisoned data on packet read from mirror buffer: 0
> > > >  Poisoned data on packet read from mcast buffer: 0
> > > >  Poisoned data on WQE read from mirror buffer: 0
> > > >  Poisoned data on WQE read from multicast buffer: 0
> > > >  Poisoned data on NIX_RX_MCE_S read: 0
> > > >NIX_AF_RVU:
> > > >  Unmap Slot Error: 0
> > > >
> > >
> > > Now i am a little bit skeptic here, devlink health reporter
> > > infrastructure was
> > > never meant to deal with dump op only, the main purpose is to
> > > diagnose/dump and recover.
> > >
> > > especially in your use case where you only report counters, i don't
> > > believe
> > > devlink health dump is a proper interface for this.
> > These are not counters. These are error interrupts raised by HW
> > blocks.
> > The count is provided to understand on how frequently the errors are
> > seen.
> > Error recovery for some of the blocks happen internally. That is the
> > reason,
> > Currently only dump op is added.
>
> So you are counting these events in driver, sounds like a counter to
> me, i really think this shouldn't belong to devlink, unless you really
> utilize devlink health ops for actual reporting and recovery.
>
> what's wrong with just dumping these counters to ethtool ?

This driver is a administrative driver which handles all the resources
in the system and doesn't do any IO.
NIX and NPA are key co-processor blocks which this driver handles.
With NIX and NPA, there are pieces
which gets attached to a PCI device to make it a networking device. We
have netdev drivers registered to this
networking device. Some more information about the drivers is available at
https://www.kernel.org/doc/html/latest/networking/device_drivers/ethernet/marvell/octeontx2.html

So we don't have a netdev here to report these co-processor block
level errors over ethtool.

Thanks,
Sunil.

Re: [PATCH v3 00/15] soc: octeontx2: Add RVU admin function driver

[Sunil Kovvuri]

On Tue, Sep 11, 2018 at 7:07 PM Arnd Bergmann  wrote:
>
> On Tue, Sep 11, 2018 at 2:37 PM Sunil Kovvuri  wrote:
> >
> > Didn't receive any feedback for the v3 patch series over a week's time.
> > Can you please pick up these patches to merge into arm-soc ?
>
> I would still prefer to see the whole thing as part of drivers/net/
> instead of drivers/soc,
> and reviewed in full on the netdev side, including the parts that are
> not ethernet specific.
>
>Arnd

Hmm.. I agree that there are many networking terms used in the driver
but it's not a
networking driver, it's just a HW configuration driver which includes
how HW should
parse the packet. This driver doesn't fit into drivers/net.

Let's say if netdev driver in drivers/net/ethernet doesn't make use of
crypto feature
then i guess netdev maintainers would reject any patches which configure crypto
block. Also as i have been saying there are other scenarios as well.
Future silicons may add support for other features into this resource
virtualization unit's domain.
An example would be compression. Any patches which do compression
related HW configuration
might be rejected by netdev maintainers, cause they are no way related
to networking.

I will keep netdev mailing list in all the patch submissions but
moving this driver into drivers/net
doesn't sound right, from it's functionality perspective.

Thanks,
Sunil.

Re: [PATCH v3 00/15] soc: octeontx2: Add RVU admin function driver

[Sunil Kovvuri]

On Tue, Sep 11, 2018 at 7:07 PM Arnd Bergmann  wrote:
>
> On Tue, Sep 11, 2018 at 2:37 PM Sunil Kovvuri  wrote:
> >
> > Didn't receive any feedback for the v3 patch series over a week's time.
> > Can you please pick up these patches to merge into arm-soc ?
>
> I would still prefer to see the whole thing as part of drivers/net/
> instead of drivers/soc,
> and reviewed in full on the netdev side, including the parts that are
> not ethernet specific.
>
>Arnd

Hmm.. I agree that there are many networking terms used in the driver
but it's not a
networking driver, it's just a HW configuration driver which includes
how HW should
parse the packet. This driver doesn't fit into drivers/net.

Let's say if netdev driver in drivers/net/ethernet doesn't make use of
crypto feature
then i guess netdev maintainers would reject any patches which configure crypto
block. Also as i have been saying there are other scenarios as well.
Future silicons may add support for other features into this resource
virtualization unit's domain.
An example would be compression. Any patches which do compression
related HW configuration
might be rejected by netdev maintainers, cause they are no way related
to networking.

I will keep netdev mailing list in all the patch submissions but
moving this driver into drivers/net
doesn't sound right, from it's functionality perspective.

Thanks,
Sunil.

Re: [PATCH v2 00/15] soc: octeontx2: Add RVU admin function driver

[Sunil Kovvuri]

On Tue, Sep 4, 2018 at 6:16 PM Andrew Lunn  wrote:
>
> On Tue, Sep 04, 2018 at 05:24:35PM +0530, sunil.kovv...@gmail.com wrote:
> > From: Sunil Goutham 
> >
> > Resource virtualization unit (RVU) on Marvell's OcteonTX2 SOC supports
> > multiple PCIe SRIOV physical functions (PFs) and virtual functions (VFs).
> > PF0 is called administrative / admin function (AF) and has privilege access
> > to registers to provision different RVU functional blocks to each of
> > PF/VF.
>
> Hi Sunil
>
> Please keep netdev in the loop. The people with most experience with
> PF/VF tend to hang out there, not arm-soc.
>
>   Andrew

Sure, will submit again with netdev in loop.

Sunil.

Re: [PATCH v2 00/15] soc: octeontx2: Add RVU admin function driver

[Sunil Kovvuri]

On Tue, Sep 4, 2018 at 6:16 PM Andrew Lunn  wrote:
>
> On Tue, Sep 04, 2018 at 05:24:35PM +0530, sunil.kovv...@gmail.com wrote:
> > From: Sunil Goutham 
> >
> > Resource virtualization unit (RVU) on Marvell's OcteonTX2 SOC supports
> > multiple PCIe SRIOV physical functions (PFs) and virtual functions (VFs).
> > PF0 is called administrative / admin function (AF) and has privilege access
> > to registers to provision different RVU functional blocks to each of
> > PF/VF.
>
> Hi Sunil
>
> Please keep netdev in the loop. The people with most experience with
> PF/VF tend to hang out there, not arm-soc.
>
>   Andrew

Sure, will submit again with netdev in loop.

Sunil.

[PATCH v2 14/15] soc: octeontx2: Register for CGX lmac events

[sunil . kovvuri]

From: Linu Cherian 

Added support in RVU AF driver to register for
CGX LMAC link status change events from firmware
and managing them. Processing part will be added
in followup patches.

- Introduced eventqueue for posting events from cgx lmac.
  Queueing mechanism will ensure that events can be posted
  and firmware can be acked immediately and hence event
  reception and processing are decoupled.
- Events gets added to the queue by notification callback.
  Notification callback is expected to be atomic, since it
  is called from interrupt context.
- Events are dequeued and processed in a worker thread.

Signed-off-by: Linu Cherian 
---
 drivers/soc/marvell/octeontx2/rvu.c |   6 +-
 drivers/soc/marvell/octeontx2/rvu.h |   5 ++
 drivers/soc/marvell/octeontx2/rvu_cgx.c | 101 +++-
 3 files changed, 108 insertions(+), 4 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index faf7d0f..282982f 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -1564,10 +1564,11 @@ static int rvu_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
 
err = rvu_register_interrupts(rvu);
if (err)
-   goto err_mbox;
+   goto err_cgx;
 
return 0;
-
+err_cgx:
+   rvu_cgx_wq_destroy(rvu);
 err_mbox:
rvu_mbox_destroy(rvu);
 err_hwsetup:
@@ -1589,6 +1590,7 @@ static void rvu_remove(struct pci_dev *pdev)
struct rvu *rvu = pci_get_drvdata(pdev);
 
rvu_unregister_interrupts(rvu);
+   rvu_cgx_wq_destroy(rvu);
rvu_mbox_destroy(rvu);
rvu_reset_all_blocks(rvu);
rvu_free_hw_resources(rvu);
diff --git a/drivers/soc/marvell/octeontx2/rvu.h 
b/drivers/soc/marvell/octeontx2/rvu.h
index 385f597..d169fa9 100644
--- a/drivers/soc/marvell/octeontx2/rvu.h
+++ b/drivers/soc/marvell/octeontx2/rvu.h
@@ -110,6 +110,10 @@ struct rvu {
  * every cgx lmac port
  */
void**cgx_idmap; /* cgx id to cgx data map table */
+   struct  work_struct cgx_evh_work;
+   struct  workqueue_struct *cgx_evh_wq;
+   spinlock_t  cgx_evq_lock; /* cgx event queue lock */
+   struct list_headcgx_evq_head; /* cgx event queue head */
 };
 
 static inline void rvu_write64(struct rvu *rvu, u64 block, u64 offset, u64 val)
@@ -150,4 +154,5 @@ int rvu_poll_reg(struct rvu *rvu, u64 block, u64 offset, 
u64 mask, bool zero);
 
 /* CGX APIs */
 int rvu_cgx_probe(struct rvu *rvu);
+void rvu_cgx_wq_destroy(struct rvu *rvu);
 #endif /* RVU_H */
diff --git a/drivers/soc/marvell/octeontx2/rvu_cgx.c 
b/drivers/soc/marvell/octeontx2/rvu_cgx.c
index bf81507..2359806e 100644
--- a/drivers/soc/marvell/octeontx2/rvu_cgx.c
+++ b/drivers/soc/marvell/octeontx2/rvu_cgx.c
@@ -15,6 +15,11 @@
 #include "rvu.h"
 #include "cgx.h"
 
+struct cgx_evq_entry {
+   struct list_head evq_node;
+   struct cgx_link_event link_event;
+};
+
 static inline u8 cgxlmac_id_to_bmap(u8 cgx_id, u8 lmac_id)
 {
return ((cgx_id & 0xF) << 4) | (lmac_id & 0xF);
@@ -72,9 +77,95 @@ static int rvu_map_cgx_lmac_pf(struct rvu *rvu)
return 0;
 }
 
+/* This is called from interrupt context and is expected to be atomic */
+static int cgx_lmac_postevent(struct cgx_link_event *event, void *data)
+{
+   struct rvu *rvu = data;
+   struct cgx_evq_entry *qentry;
+
+   /* post event to the event queue */
+   qentry = kmalloc(sizeof(*qentry), GFP_ATOMIC);
+   if (!qentry)
+   return -ENOMEM;
+   qentry->link_event = *event;
+   spin_lock(>cgx_evq_lock);
+   list_add_tail(>evq_node, >cgx_evq_head);
+   spin_unlock(>cgx_evq_lock);
+
+   /* start worker to process the events */
+   queue_work(rvu->cgx_evh_wq, >cgx_evh_work);
+
+   return 0;
+}
+
+static void cgx_evhandler_task(struct work_struct *work)
+{
+   struct rvu *rvu = container_of(work, struct rvu, cgx_evh_work);
+   struct cgx_evq_entry *qentry;
+   struct cgx_link_event *event;
+   unsigned long flags;
+
+   do {
+   /* Dequeue an event */
+   spin_lock_irqsave(>cgx_evq_lock, flags);
+   qentry = list_first_entry_or_null(>cgx_evq_head,
+ struct cgx_evq_entry,
+ evq_node);
+   if (qentry)
+   list_del(>evq_node);
+   spin_unlock_irqrestore(>cgx_evq_lock, flags);
+   if (!qentry)
+   break; /* nothing more to process */
+
+   event = >link_event;
+
+   /* Do nothing for now */
+   kfree(qentry);
+   } while (1);
+}
+
+static void cgx_lmac_event_handler_init(struct rvu *rvu)
+{
+   struct cgx_event_cb cb;
+   int 

[PATCH v2 15/15] MAINTAINERS: Add entry for Marvell OcteonTX2 Admin Function driver

[sunil . kovvuri]

From: Sunil Goutham 

Added maintainers entry for Marvell OcteonTX2 SOC's RVU
admin function driver.

Signed-off-by: Sunil Goutham 
---
 MAINTAINERS | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index e178f2b..38f874c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8748,6 +8748,16 @@ S:   Supported
 F: drivers/mmc/host/sdhci-xenon*
 F: Documentation/devicetree/bindings/mmc/marvell,xenon-sdhci.txt
 
+MARVELL OCTEONTX2 RVU ADMIN FUNCTION DRIVER
+M: Sunil Goutham 
+M: Linu Cherian 
+M: Geetha sowjanya 
+M: Jerin Jacob 
+L: linux-kernel@vger.kernel.org
+L: linux-arm-ker...@lists.infradead.org (moderated for non-subscribers)
+S: Maintained
+F: drivers/soc/marvell/octeontx2
+
 MATROX FRAMEBUFFER DRIVER
 L: linux-fb...@vger.kernel.org
 S: Orphan
-- 
2.7.4

[PATCH v2 15/15] MAINTAINERS: Add entry for Marvell OcteonTX2 Admin Function driver

[sunil . kovvuri]

From: Sunil Goutham 

Added maintainers entry for Marvell OcteonTX2 SOC's RVU
admin function driver.

Signed-off-by: Sunil Goutham 
---
 MAINTAINERS | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index e178f2b..38f874c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8748,6 +8748,16 @@ S:   Supported
 F: drivers/mmc/host/sdhci-xenon*
 F: Documentation/devicetree/bindings/mmc/marvell,xenon-sdhci.txt
 
+MARVELL OCTEONTX2 RVU ADMIN FUNCTION DRIVER
+M: Sunil Goutham 
+M: Linu Cherian 
+M: Geetha sowjanya 
+M: Jerin Jacob 
+L: linux-kernel@vger.kernel.org
+L: linux-arm-ker...@lists.infradead.org (moderated for non-subscribers)
+S: Maintained
+F: drivers/soc/marvell/octeontx2
+
 MATROX FRAMEBUFFER DRIVER
 L: linux-fb...@vger.kernel.org
 S: Orphan
-- 
2.7.4

[PATCH v2 14/15] soc: octeontx2: Register for CGX lmac events

[sunil . kovvuri]

From: Linu Cherian 

Added support in RVU AF driver to register for
CGX LMAC link status change events from firmware
and managing them. Processing part will be added
in followup patches.

- Introduced eventqueue for posting events from cgx lmac.
  Queueing mechanism will ensure that events can be posted
  and firmware can be acked immediately and hence event
  reception and processing are decoupled.
- Events gets added to the queue by notification callback.
  Notification callback is expected to be atomic, since it
  is called from interrupt context.
- Events are dequeued and processed in a worker thread.

Signed-off-by: Linu Cherian 
---
 drivers/soc/marvell/octeontx2/rvu.c |   6 +-
 drivers/soc/marvell/octeontx2/rvu.h |   5 ++
 drivers/soc/marvell/octeontx2/rvu_cgx.c | 101 +++-
 3 files changed, 108 insertions(+), 4 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index faf7d0f..282982f 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -1564,10 +1564,11 @@ static int rvu_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
 
err = rvu_register_interrupts(rvu);
if (err)
-   goto err_mbox;
+   goto err_cgx;
 
return 0;
-
+err_cgx:
+   rvu_cgx_wq_destroy(rvu);
 err_mbox:
rvu_mbox_destroy(rvu);
 err_hwsetup:
@@ -1589,6 +1590,7 @@ static void rvu_remove(struct pci_dev *pdev)
struct rvu *rvu = pci_get_drvdata(pdev);
 
rvu_unregister_interrupts(rvu);
+   rvu_cgx_wq_destroy(rvu);
rvu_mbox_destroy(rvu);
rvu_reset_all_blocks(rvu);
rvu_free_hw_resources(rvu);
diff --git a/drivers/soc/marvell/octeontx2/rvu.h 
b/drivers/soc/marvell/octeontx2/rvu.h
index 385f597..d169fa9 100644
--- a/drivers/soc/marvell/octeontx2/rvu.h
+++ b/drivers/soc/marvell/octeontx2/rvu.h
@@ -110,6 +110,10 @@ struct rvu {
  * every cgx lmac port
  */
void**cgx_idmap; /* cgx id to cgx data map table */
+   struct  work_struct cgx_evh_work;
+   struct  workqueue_struct *cgx_evh_wq;
+   spinlock_t  cgx_evq_lock; /* cgx event queue lock */
+   struct list_headcgx_evq_head; /* cgx event queue head */
 };
 
 static inline void rvu_write64(struct rvu *rvu, u64 block, u64 offset, u64 val)
@@ -150,4 +154,5 @@ int rvu_poll_reg(struct rvu *rvu, u64 block, u64 offset, 
u64 mask, bool zero);
 
 /* CGX APIs */
 int rvu_cgx_probe(struct rvu *rvu);
+void rvu_cgx_wq_destroy(struct rvu *rvu);
 #endif /* RVU_H */
diff --git a/drivers/soc/marvell/octeontx2/rvu_cgx.c 
b/drivers/soc/marvell/octeontx2/rvu_cgx.c
index bf81507..2359806e 100644
--- a/drivers/soc/marvell/octeontx2/rvu_cgx.c
+++ b/drivers/soc/marvell/octeontx2/rvu_cgx.c
@@ -15,6 +15,11 @@
 #include "rvu.h"
 #include "cgx.h"
 
+struct cgx_evq_entry {
+   struct list_head evq_node;
+   struct cgx_link_event link_event;
+};
+
 static inline u8 cgxlmac_id_to_bmap(u8 cgx_id, u8 lmac_id)
 {
return ((cgx_id & 0xF) << 4) | (lmac_id & 0xF);
@@ -72,9 +77,95 @@ static int rvu_map_cgx_lmac_pf(struct rvu *rvu)
return 0;
 }
 
+/* This is called from interrupt context and is expected to be atomic */
+static int cgx_lmac_postevent(struct cgx_link_event *event, void *data)
+{
+   struct rvu *rvu = data;
+   struct cgx_evq_entry *qentry;
+
+   /* post event to the event queue */
+   qentry = kmalloc(sizeof(*qentry), GFP_ATOMIC);
+   if (!qentry)
+   return -ENOMEM;
+   qentry->link_event = *event;
+   spin_lock(>cgx_evq_lock);
+   list_add_tail(>evq_node, >cgx_evq_head);
+   spin_unlock(>cgx_evq_lock);
+
+   /* start worker to process the events */
+   queue_work(rvu->cgx_evh_wq, >cgx_evh_work);
+
+   return 0;
+}
+
+static void cgx_evhandler_task(struct work_struct *work)
+{
+   struct rvu *rvu = container_of(work, struct rvu, cgx_evh_work);
+   struct cgx_evq_entry *qentry;
+   struct cgx_link_event *event;
+   unsigned long flags;
+
+   do {
+   /* Dequeue an event */
+   spin_lock_irqsave(>cgx_evq_lock, flags);
+   qentry = list_first_entry_or_null(>cgx_evq_head,
+ struct cgx_evq_entry,
+ evq_node);
+   if (qentry)
+   list_del(>evq_node);
+   spin_unlock_irqrestore(>cgx_evq_lock, flags);
+   if (!qentry)
+   break; /* nothing more to process */
+
+   event = >link_event;
+
+   /* Do nothing for now */
+   kfree(qentry);
+   } while (1);
+}
+
+static void cgx_lmac_event_handler_init(struct rvu *rvu)
+{
+   struct cgx_event_cb cb;
+   int 

[PATCH v2 12/15] soc: octeontx2: Set RVU PFs to CGX LMACs mapping

[sunil . kovvuri]

From: Linu Cherian 

Each of the enabled CGX LMAC is considered a physical
interface and RVU PFs are mapped to these. VFs of these
SRIOV PFs will be virtual interfaces and share CGX LMAC
along with PF.

This mapping info will be used later on for Rx/Tx pkt steering.

Signed-off-by: Linu Cherian 
Signed-off-by: Geetha sowjanya 
---
 drivers/soc/marvell/octeontx2/Makefile  |  2 +-
 drivers/soc/marvell/octeontx2/cgx.c | 59 
 drivers/soc/marvell/octeontx2/cgx.h | 15 -
 drivers/soc/marvell/octeontx2/rvu.c |  4 ++
 drivers/soc/marvell/octeontx2/rvu.h | 12 
 drivers/soc/marvell/octeontx2/rvu_cgx.c | 97 +
 6 files changed, 186 insertions(+), 3 deletions(-)
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_cgx.c

diff --git a/drivers/soc/marvell/octeontx2/Makefile 
b/drivers/soc/marvell/octeontx2/Makefile
index 8646421..eaac264 100644
--- a/drivers/soc/marvell/octeontx2/Makefile
+++ b/drivers/soc/marvell/octeontx2/Makefile
@@ -7,4 +7,4 @@ obj-$(CONFIG_OCTEONTX2_MBOX) += octeontx2_mbox.o
 obj-$(CONFIG_OCTEONTX2_AF) += octeontx2_af.o
 
 octeontx2_mbox-y := mbox.o
-octeontx2_af-y := cgx.o rvu.o
+octeontx2_af-y := cgx.o rvu.o rvu_cgx.o
diff --git a/drivers/soc/marvell/octeontx2/cgx.c 
b/drivers/soc/marvell/octeontx2/cgx.c
index 47aa4cb..c5e0ebb 100644
--- a/drivers/soc/marvell/octeontx2/cgx.c
+++ b/drivers/soc/marvell/octeontx2/cgx.c
@@ -29,8 +29,12 @@ struct cgx {
void __iomem*reg_base;
struct pci_dev  *pdev;
u8  cgx_id;
+   u8  lmac_count;
+   struct list_headcgx_list;
 };
 
+static LIST_HEAD(cgx_list);
+
 /* Supported devices */
 static const struct pci_device_id cgx_id_table[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_CGX) },
@@ -43,6 +47,53 @@ MODULE_LICENSE("GPL v2");
 MODULE_VERSION(DRV_VERSION);
 MODULE_DEVICE_TABLE(pci, cgx_id_table);
 
+static u64 cgx_read(struct cgx *cgx, u64 lmac, u64 offset)
+{
+   return readq(cgx->reg_base + (lmac << 18) + offset);
+}
+
+int cgx_get_cgx_cnt(void)
+{
+   struct cgx *cgx_dev;
+   int count = 0;
+
+   list_for_each_entry(cgx_dev, _list, cgx_list)
+   count++;
+
+   return count;
+}
+EXPORT_SYMBOL(cgx_get_cgx_cnt);
+
+int cgx_get_lmac_cnt(void *cgxd)
+{
+   struct cgx *cgx = cgxd;
+
+   if (!cgx)
+   return -ENODEV;
+
+   return cgx->lmac_count;
+}
+EXPORT_SYMBOL(cgx_get_lmac_cnt);
+
+void *cgx_get_pdata(int cgx_id)
+{
+   struct cgx *cgx_dev;
+
+   list_for_each_entry(cgx_dev, _list, cgx_list) {
+   if (cgx_dev->cgx_id == cgx_id)
+   return cgx_dev;
+   }
+   return NULL;
+}
+EXPORT_SYMBOL(cgx_get_pdata);
+
+static void cgx_lmac_init(struct cgx *cgx)
+{
+   cgx->lmac_count = cgx_read(cgx, 0, CGXX_CMRX_RX_LMACS) & 0x7;
+   if (cgx->lmac_count > MAX_LMAC_PER_CGX)
+   cgx->lmac_count = MAX_LMAC_PER_CGX;
+}
+
 static int cgx_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 {
int err;
@@ -77,9 +128,14 @@ static int cgx_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
goto err_release_regions;
}
 
+   list_add(>cgx_list, _list);
+   cgx->cgx_id = cgx_get_cgx_cnt() - 1;
+   cgx_lmac_init(cgx);
+
return 0;
 
 err_release_regions:
+   list_del(>cgx_list);
pci_release_regions(pdev);
 err_disable_device:
pci_disable_device(pdev);
@@ -89,6 +145,9 @@ static int cgx_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
 
 static void cgx_remove(struct pci_dev *pdev)
 {
+   struct cgx *cgx = pci_get_drvdata(pdev);
+
+   list_del(>cgx_list);
pci_release_regions(pdev);
pci_disable_device(pdev);
pci_set_drvdata(pdev, NULL);
diff --git a/drivers/soc/marvell/octeontx2/cgx.h 
b/drivers/soc/marvell/octeontx2/cgx.h
index a7d4b39..acdc16e 100644
--- a/drivers/soc/marvell/octeontx2/cgx.h
+++ b/drivers/soc/marvell/octeontx2/cgx.h
@@ -12,11 +12,22 @@
 #define CGX_H
 
  /* PCI device IDs */
-#definePCI_DEVID_OCTEONTX2_CGX 0xA059
+#definePCI_DEVID_OCTEONTX2_CGX 0xA059
 
 /* PCI BAR nos */
-#define PCI_CFG_REG_BAR_NUM0
+#define PCI_CFG_REG_BAR_NUM0
+
+#define MAX_CGX3
+#define MAX_LMAC_PER_CGX   4
+#define CGX_OFFSET(x)  ((x) * MAX_LMAC_PER_CGX)
+
+/* Registers */
+#define CGXX_CMRX_RX_ID_MAP0x060
+#define CGXX_CMRX_RX_LMACS 0x128
 
 extern struct pci_driver cgx_driver;
 
+int cgx_get_cgx_cnt(void);
+int cgx_get_lmac_cnt(void *cgxd);
+void *cgx_get_pdata(int cgx_id);
 #endif /* CGX_H */
diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index daa6fd3..faf7d0f 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ 

[PATCH v2 13/15] soc: octeontx2: Add support for CGX link management

[sunil . kovvuri]

From: Linu Cherian 

CGX LMAC initialization, link status polling etc is done
by low level secure firmware. For link management this patch
adds a interface or communication mechanism between firmware
and this kernel CGX driver.

- Firmware interface specification is defined in cgx_fw_if.h.
- Support to send/receive commands/events to/form firmware.
- events/commands implemented
  * link up
  * link down
  * reading firmware version

Signed-off-by: Linu Cherian 
Signed-off-by: Nithya Mani 
---
 drivers/soc/marvell/octeontx2/cgx.c   | 364 +-
 drivers/soc/marvell/octeontx2/cgx.h   |  32 +++
 drivers/soc/marvell/octeontx2/cgx_fw_if.h | 225 ++
 3 files changed, 617 insertions(+), 4 deletions(-)
 create mode 100644 drivers/soc/marvell/octeontx2/cgx_fw_if.h

diff --git a/drivers/soc/marvell/octeontx2/cgx.c 
b/drivers/soc/marvell/octeontx2/cgx.c
index c5e0ebb..26af8fa 100644
--- a/drivers/soc/marvell/octeontx2/cgx.c
+++ b/drivers/soc/marvell/octeontx2/cgx.c
@@ -25,16 +25,43 @@
 #define DRV_STRING  "Marvell OcteonTX2 CGX/MAC Driver"
 #define DRV_VERSION"1.0"
 
+/**
+ * struct lmac
+ * @wq_cmd_cmplt:  waitq to keep the process blocked until cmd completion
+ * @cmd_lock:  Lock to serialize the command interface
+ * @resp:  command response
+ * @event_cb:  callback for linkchange events
+ * @cmd_pend:  flag set before new command is started
+ * flag cleared after command response is received
+ * @cgx:   parent cgx port
+ * @lmac_id:   lmac port id
+ * @name:  lmac port name
+ */
+struct lmac {
+   wait_queue_head_t wq_cmd_cmplt;
+   struct mutex cmd_lock;
+   struct cgx_evt_sts resp;
+   struct cgx_event_cb event_cb;
+   bool cmd_pend;
+   struct cgx *cgx;
+   u8 lmac_id;
+   char *name;
+};
+
 struct cgx {
void __iomem*reg_base;
struct pci_dev  *pdev;
u8  cgx_id;
u8  lmac_count;
+   struct lmac *lmac_idmap[MAX_LMAC_PER_CGX];
struct list_headcgx_list;
 };
 
 static LIST_HEAD(cgx_list);
 
+/* CGX PHY management internal APIs */
+static int cgx_fwi_link_change(struct cgx *cgx, int lmac_id, bool en);
+
 /* Supported devices */
 static const struct pci_device_id cgx_id_table[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_CGX) },
@@ -47,11 +74,24 @@ MODULE_LICENSE("GPL v2");
 MODULE_VERSION(DRV_VERSION);
 MODULE_DEVICE_TABLE(pci, cgx_id_table);
 
+static void cgx_write(struct cgx *cgx, u64 lmac, u64 offset, u64 val)
+{
+   writeq(val, cgx->reg_base + (lmac << 18) + offset);
+}
+
 static u64 cgx_read(struct cgx *cgx, u64 lmac, u64 offset)
 {
return readq(cgx->reg_base + (lmac << 18) + offset);
 }
 
+static inline struct lmac *lmac_pdata(u8 lmac_id, struct cgx *cgx)
+{
+   if (!cgx || lmac_id >= MAX_LMAC_PER_CGX)
+   return NULL;
+
+   return cgx->lmac_idmap[lmac_id];
+}
+
 int cgx_get_cgx_cnt(void)
 {
struct cgx *cgx_dev;
@@ -87,18 +127,318 @@ void *cgx_get_pdata(int cgx_id)
 }
 EXPORT_SYMBOL(cgx_get_pdata);
 
-static void cgx_lmac_init(struct cgx *cgx)
+/* CGX Firmware interface low level support */
+static int cgx_fwi_cmd_send(struct cgx_cmd *cmd, struct cgx_evt_sts *rsp,
+   struct lmac *lmac)
+{
+   struct cgx *cgx = lmac->cgx;
+   union cgx_cmdreg creg;
+   union cgx_evtreg ereg;
+   struct device *dev;
+   int err = 0;
+
+   /* Ensure no other command is in progress */
+   err = mutex_lock_interruptible(>cmd_lock);
+   if (err)
+   return err;
+
+   /* Ensure command register is free */
+   creg.val = cgx_read(cgx, lmac->lmac_id,  CGX_COMMAND_REG);
+   if (creg.cmd.own != CGX_CMD_OWN_NS) {
+   err = -EBUSY;
+   goto unlock;
+   }
+
+   /* Update ownership in command request */
+   cmd->own = CGX_CMD_OWN_FIRMWARE;
+
+   /* Mark this lmac as pending, before we start */
+   lmac->cmd_pend = true;
+
+   /* Start command in hardware */
+   creg.cmd = *cmd;
+   cgx_write(cgx, lmac->lmac_id, CGX_COMMAND_REG, creg.val);
+   creg.val = cgx_read(cgx, lmac->lmac_id,  CGX_COMMAND_REG);
+
+   /* Ensure command is completed without errors */
+   if (!wait_event_timeout(lmac->wq_cmd_cmplt, !lmac->cmd_pend,
+   msecs_to_jiffies(CGX_CMD_TIMEOUT))) {
+   dev = >pdev->dev;
+   ereg.val = cgx_read(cgx, lmac->lmac_id,  CGX_EVENT_REG);
+   if (ereg.val) {
+   dev_err(dev, "cgx port %d:%d: No event for response\n",
+   cgx->cgx_id, lmac->lmac_id);
+   /* copy event */
+   lmac->resp = ereg.evt_sts;
+   } else {
+   dev_err(dev, "cgx port %d:%d cmd 

[PATCH v2 13/15] soc: octeontx2: Add support for CGX link management

[sunil . kovvuri]

From: Linu Cherian 

CGX LMAC initialization, link status polling etc is done
by low level secure firmware. For link management this patch
adds a interface or communication mechanism between firmware
and this kernel CGX driver.

- Firmware interface specification is defined in cgx_fw_if.h.
- Support to send/receive commands/events to/form firmware.
- events/commands implemented
  * link up
  * link down
  * reading firmware version

Signed-off-by: Linu Cherian 
Signed-off-by: Nithya Mani 
---
 drivers/soc/marvell/octeontx2/cgx.c   | 364 +-
 drivers/soc/marvell/octeontx2/cgx.h   |  32 +++
 drivers/soc/marvell/octeontx2/cgx_fw_if.h | 225 ++
 3 files changed, 617 insertions(+), 4 deletions(-)
 create mode 100644 drivers/soc/marvell/octeontx2/cgx_fw_if.h

diff --git a/drivers/soc/marvell/octeontx2/cgx.c 
b/drivers/soc/marvell/octeontx2/cgx.c
index c5e0ebb..26af8fa 100644
--- a/drivers/soc/marvell/octeontx2/cgx.c
+++ b/drivers/soc/marvell/octeontx2/cgx.c
@@ -25,16 +25,43 @@
 #define DRV_STRING  "Marvell OcteonTX2 CGX/MAC Driver"
 #define DRV_VERSION"1.0"
 
+/**
+ * struct lmac
+ * @wq_cmd_cmplt:  waitq to keep the process blocked until cmd completion
+ * @cmd_lock:  Lock to serialize the command interface
+ * @resp:  command response
+ * @event_cb:  callback for linkchange events
+ * @cmd_pend:  flag set before new command is started
+ * flag cleared after command response is received
+ * @cgx:   parent cgx port
+ * @lmac_id:   lmac port id
+ * @name:  lmac port name
+ */
+struct lmac {
+   wait_queue_head_t wq_cmd_cmplt;
+   struct mutex cmd_lock;
+   struct cgx_evt_sts resp;
+   struct cgx_event_cb event_cb;
+   bool cmd_pend;
+   struct cgx *cgx;
+   u8 lmac_id;
+   char *name;
+};
+
 struct cgx {
void __iomem*reg_base;
struct pci_dev  *pdev;
u8  cgx_id;
u8  lmac_count;
+   struct lmac *lmac_idmap[MAX_LMAC_PER_CGX];
struct list_headcgx_list;
 };
 
 static LIST_HEAD(cgx_list);
 
+/* CGX PHY management internal APIs */
+static int cgx_fwi_link_change(struct cgx *cgx, int lmac_id, bool en);
+
 /* Supported devices */
 static const struct pci_device_id cgx_id_table[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_CGX) },
@@ -47,11 +74,24 @@ MODULE_LICENSE("GPL v2");
 MODULE_VERSION(DRV_VERSION);
 MODULE_DEVICE_TABLE(pci, cgx_id_table);
 
+static void cgx_write(struct cgx *cgx, u64 lmac, u64 offset, u64 val)
+{
+   writeq(val, cgx->reg_base + (lmac << 18) + offset);
+}
+
 static u64 cgx_read(struct cgx *cgx, u64 lmac, u64 offset)
 {
return readq(cgx->reg_base + (lmac << 18) + offset);
 }
 
+static inline struct lmac *lmac_pdata(u8 lmac_id, struct cgx *cgx)
+{
+   if (!cgx || lmac_id >= MAX_LMAC_PER_CGX)
+   return NULL;
+
+   return cgx->lmac_idmap[lmac_id];
+}
+
 int cgx_get_cgx_cnt(void)
 {
struct cgx *cgx_dev;
@@ -87,18 +127,318 @@ void *cgx_get_pdata(int cgx_id)
 }
 EXPORT_SYMBOL(cgx_get_pdata);
 
-static void cgx_lmac_init(struct cgx *cgx)
+/* CGX Firmware interface low level support */
+static int cgx_fwi_cmd_send(struct cgx_cmd *cmd, struct cgx_evt_sts *rsp,
+   struct lmac *lmac)
+{
+   struct cgx *cgx = lmac->cgx;
+   union cgx_cmdreg creg;
+   union cgx_evtreg ereg;
+   struct device *dev;
+   int err = 0;
+
+   /* Ensure no other command is in progress */
+   err = mutex_lock_interruptible(>cmd_lock);
+   if (err)
+   return err;
+
+   /* Ensure command register is free */
+   creg.val = cgx_read(cgx, lmac->lmac_id,  CGX_COMMAND_REG);
+   if (creg.cmd.own != CGX_CMD_OWN_NS) {
+   err = -EBUSY;
+   goto unlock;
+   }
+
+   /* Update ownership in command request */
+   cmd->own = CGX_CMD_OWN_FIRMWARE;
+
+   /* Mark this lmac as pending, before we start */
+   lmac->cmd_pend = true;
+
+   /* Start command in hardware */
+   creg.cmd = *cmd;
+   cgx_write(cgx, lmac->lmac_id, CGX_COMMAND_REG, creg.val);
+   creg.val = cgx_read(cgx, lmac->lmac_id,  CGX_COMMAND_REG);
+
+   /* Ensure command is completed without errors */
+   if (!wait_event_timeout(lmac->wq_cmd_cmplt, !lmac->cmd_pend,
+   msecs_to_jiffies(CGX_CMD_TIMEOUT))) {
+   dev = >pdev->dev;
+   ereg.val = cgx_read(cgx, lmac->lmac_id,  CGX_EVENT_REG);
+   if (ereg.val) {
+   dev_err(dev, "cgx port %d:%d: No event for response\n",
+   cgx->cgx_id, lmac->lmac_id);
+   /* copy event */
+   lmac->resp = ereg.evt_sts;
+   } else {
+   dev_err(dev, "cgx port %d:%d cmd 

[PATCH v2 12/15] soc: octeontx2: Set RVU PFs to CGX LMACs mapping

[sunil . kovvuri]

From: Linu Cherian 

Each of the enabled CGX LMAC is considered a physical
interface and RVU PFs are mapped to these. VFs of these
SRIOV PFs will be virtual interfaces and share CGX LMAC
along with PF.

This mapping info will be used later on for Rx/Tx pkt steering.

Signed-off-by: Linu Cherian 
Signed-off-by: Geetha sowjanya 
---
 drivers/soc/marvell/octeontx2/Makefile  |  2 +-
 drivers/soc/marvell/octeontx2/cgx.c | 59 
 drivers/soc/marvell/octeontx2/cgx.h | 15 -
 drivers/soc/marvell/octeontx2/rvu.c |  4 ++
 drivers/soc/marvell/octeontx2/rvu.h | 12 
 drivers/soc/marvell/octeontx2/rvu_cgx.c | 97 +
 6 files changed, 186 insertions(+), 3 deletions(-)
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_cgx.c

diff --git a/drivers/soc/marvell/octeontx2/Makefile 
b/drivers/soc/marvell/octeontx2/Makefile
index 8646421..eaac264 100644
--- a/drivers/soc/marvell/octeontx2/Makefile
+++ b/drivers/soc/marvell/octeontx2/Makefile
@@ -7,4 +7,4 @@ obj-$(CONFIG_OCTEONTX2_MBOX) += octeontx2_mbox.o
 obj-$(CONFIG_OCTEONTX2_AF) += octeontx2_af.o
 
 octeontx2_mbox-y := mbox.o
-octeontx2_af-y := cgx.o rvu.o
+octeontx2_af-y := cgx.o rvu.o rvu_cgx.o
diff --git a/drivers/soc/marvell/octeontx2/cgx.c 
b/drivers/soc/marvell/octeontx2/cgx.c
index 47aa4cb..c5e0ebb 100644
--- a/drivers/soc/marvell/octeontx2/cgx.c
+++ b/drivers/soc/marvell/octeontx2/cgx.c
@@ -29,8 +29,12 @@ struct cgx {
void __iomem*reg_base;
struct pci_dev  *pdev;
u8  cgx_id;
+   u8  lmac_count;
+   struct list_headcgx_list;
 };
 
+static LIST_HEAD(cgx_list);
+
 /* Supported devices */
 static const struct pci_device_id cgx_id_table[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_CGX) },
@@ -43,6 +47,53 @@ MODULE_LICENSE("GPL v2");
 MODULE_VERSION(DRV_VERSION);
 MODULE_DEVICE_TABLE(pci, cgx_id_table);
 
+static u64 cgx_read(struct cgx *cgx, u64 lmac, u64 offset)
+{
+   return readq(cgx->reg_base + (lmac << 18) + offset);
+}
+
+int cgx_get_cgx_cnt(void)
+{
+   struct cgx *cgx_dev;
+   int count = 0;
+
+   list_for_each_entry(cgx_dev, _list, cgx_list)
+   count++;
+
+   return count;
+}
+EXPORT_SYMBOL(cgx_get_cgx_cnt);
+
+int cgx_get_lmac_cnt(void *cgxd)
+{
+   struct cgx *cgx = cgxd;
+
+   if (!cgx)
+   return -ENODEV;
+
+   return cgx->lmac_count;
+}
+EXPORT_SYMBOL(cgx_get_lmac_cnt);
+
+void *cgx_get_pdata(int cgx_id)
+{
+   struct cgx *cgx_dev;
+
+   list_for_each_entry(cgx_dev, _list, cgx_list) {
+   if (cgx_dev->cgx_id == cgx_id)
+   return cgx_dev;
+   }
+   return NULL;
+}
+EXPORT_SYMBOL(cgx_get_pdata);
+
+static void cgx_lmac_init(struct cgx *cgx)
+{
+   cgx->lmac_count = cgx_read(cgx, 0, CGXX_CMRX_RX_LMACS) & 0x7;
+   if (cgx->lmac_count > MAX_LMAC_PER_CGX)
+   cgx->lmac_count = MAX_LMAC_PER_CGX;
+}
+
 static int cgx_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 {
int err;
@@ -77,9 +128,14 @@ static int cgx_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
goto err_release_regions;
}
 
+   list_add(>cgx_list, _list);
+   cgx->cgx_id = cgx_get_cgx_cnt() - 1;
+   cgx_lmac_init(cgx);
+
return 0;
 
 err_release_regions:
+   list_del(>cgx_list);
pci_release_regions(pdev);
 err_disable_device:
pci_disable_device(pdev);
@@ -89,6 +145,9 @@ static int cgx_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
 
 static void cgx_remove(struct pci_dev *pdev)
 {
+   struct cgx *cgx = pci_get_drvdata(pdev);
+
+   list_del(>cgx_list);
pci_release_regions(pdev);
pci_disable_device(pdev);
pci_set_drvdata(pdev, NULL);
diff --git a/drivers/soc/marvell/octeontx2/cgx.h 
b/drivers/soc/marvell/octeontx2/cgx.h
index a7d4b39..acdc16e 100644
--- a/drivers/soc/marvell/octeontx2/cgx.h
+++ b/drivers/soc/marvell/octeontx2/cgx.h
@@ -12,11 +12,22 @@
 #define CGX_H
 
  /* PCI device IDs */
-#definePCI_DEVID_OCTEONTX2_CGX 0xA059
+#definePCI_DEVID_OCTEONTX2_CGX 0xA059
 
 /* PCI BAR nos */
-#define PCI_CFG_REG_BAR_NUM0
+#define PCI_CFG_REG_BAR_NUM0
+
+#define MAX_CGX3
+#define MAX_LMAC_PER_CGX   4
+#define CGX_OFFSET(x)  ((x) * MAX_LMAC_PER_CGX)
+
+/* Registers */
+#define CGXX_CMRX_RX_ID_MAP0x060
+#define CGXX_CMRX_RX_LMACS 0x128
 
 extern struct pci_driver cgx_driver;
 
+int cgx_get_cgx_cnt(void);
+int cgx_get_lmac_cnt(void *cgxd);
+void *cgx_get_pdata(int cgx_id);
 #endif /* CGX_H */
diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index daa6fd3..faf7d0f 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ 

[PATCH v2 11/15] soc: octeontx2: Add Marvell OcteonTX2 CGX driver

[sunil . kovvuri]

From: Sunil Goutham 

This patch adds basic template for Marvell OcteonTX2's
CGX ethernet interface driver. Just the probe.
RVU AF driver will use APIs exported by this driver
for various things like PF to physical interface mapping,
loopback mode, interface stats etc. Hence marged both
drivers into a single module.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/Kconfig|   3 +-
 drivers/soc/marvell/octeontx2/Makefile |   2 +-
 drivers/soc/marvell/octeontx2/cgx.c| 102 +
 drivers/soc/marvell/octeontx2/cgx.h|  22 +++
 drivers/soc/marvell/octeontx2/rvu.c|  14 -
 5 files changed, 140 insertions(+), 3 deletions(-)
 create mode 100644 drivers/soc/marvell/octeontx2/cgx.c
 create mode 100644 drivers/soc/marvell/octeontx2/cgx.h

diff --git a/drivers/soc/marvell/Kconfig b/drivers/soc/marvell/Kconfig
index 428d22e..8f36f3a 100644
--- a/drivers/soc/marvell/Kconfig
+++ b/drivers/soc/marvell/Kconfig
@@ -11,7 +11,8 @@ config OCTEONTX2_AF
tristate "OcteonTX2 RVU Admin Function driver"
select OCTEONTX2_MBOX
depends on ARM64 && PCI
-   help
+   ---help---
  This driver supports Marvell's OcteonTX2 Resource Virtualization
  Unit's admin function manager which manages all RVU HW resources.
+
 endmenu
diff --git a/drivers/soc/marvell/octeontx2/Makefile 
b/drivers/soc/marvell/octeontx2/Makefile
index ac17cb9..8646421 100644
--- a/drivers/soc/marvell/octeontx2/Makefile
+++ b/drivers/soc/marvell/octeontx2/Makefile
@@ -7,4 +7,4 @@ obj-$(CONFIG_OCTEONTX2_MBOX) += octeontx2_mbox.o
 obj-$(CONFIG_OCTEONTX2_AF) += octeontx2_af.o
 
 octeontx2_mbox-y := mbox.o
-octeontx2_af-y := rvu.o
+octeontx2_af-y := cgx.o rvu.o
diff --git a/drivers/soc/marvell/octeontx2/cgx.c 
b/drivers/soc/marvell/octeontx2/cgx.c
new file mode 100644
index 000..47aa4cb
--- /dev/null
+++ b/drivers/soc/marvell/octeontx2/cgx.c
@@ -0,0 +1,102 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Marvell OcteonTx2 CGX driver
+ *
+ * Copyright (C) 2018 Marvell International Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "cgx.h"
+
+#define DRV_NAME   "octeontx2-cgx"
+#define DRV_STRING  "Marvell OcteonTX2 CGX/MAC Driver"
+#define DRV_VERSION"1.0"
+
+struct cgx {
+   void __iomem*reg_base;
+   struct pci_dev  *pdev;
+   u8  cgx_id;
+};
+
+/* Supported devices */
+static const struct pci_device_id cgx_id_table[] = {
+   { PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_CGX) },
+   { 0, }  /* end of table */
+};
+
+MODULE_AUTHOR("Marvell International Ltd.");
+MODULE_DESCRIPTION(DRV_STRING);
+MODULE_LICENSE("GPL v2");
+MODULE_VERSION(DRV_VERSION);
+MODULE_DEVICE_TABLE(pci, cgx_id_table);
+
+static int cgx_probe(struct pci_dev *pdev, const struct pci_device_id *id)
+{
+   int err;
+   struct device *dev = >dev;
+   struct cgx *cgx;
+
+   cgx = devm_kzalloc(dev, sizeof(*cgx), GFP_KERNEL);
+   if (!cgx)
+   return -ENOMEM;
+   cgx->pdev = pdev;
+
+   pci_set_drvdata(pdev, cgx);
+
+   err = pci_enable_device(pdev);
+   if (err) {
+   dev_err(dev, "Failed to enable PCI device\n");
+   pci_set_drvdata(pdev, NULL);
+   return err;
+   }
+
+   err = pci_request_regions(pdev, DRV_NAME);
+   if (err) {
+   dev_err(dev, "PCI request regions failed 0x%x\n", err);
+   goto err_disable_device;
+   }
+
+   /* MAP configuration registers */
+   cgx->reg_base = pcim_iomap(pdev, PCI_CFG_REG_BAR_NUM, 0);
+   if (!cgx->reg_base) {
+   dev_err(dev, "CGX: Cannot map CSR memory space, aborting\n");
+   err = -ENOMEM;
+   goto err_release_regions;
+   }
+
+   return 0;
+
+err_release_regions:
+   pci_release_regions(pdev);
+err_disable_device:
+   pci_disable_device(pdev);
+   pci_set_drvdata(pdev, NULL);
+   return err;
+}
+
+static void cgx_remove(struct pci_dev *pdev)
+{
+   pci_release_regions(pdev);
+   pci_disable_device(pdev);
+   pci_set_drvdata(pdev, NULL);
+}
+
+struct pci_driver cgx_driver = {
+   .name = DRV_NAME,
+   .id_table = cgx_id_table,
+   .probe = cgx_probe,
+   .remove = cgx_remove,
+};
diff --git a/drivers/soc/marvell/octeontx2/cgx.h 
b/drivers/soc/marvell/octeontx2/cgx.h
new file mode 100644
index 000..a7d4b39
--- /dev/null
+++ b/drivers/soc/marvell/octeontx2/cgx.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0
+ * Marvell OcteonTx2 CGX driver
+ *
+ * Copyright (C) 2018 Marvell International Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it 

[PATCH v2 11/15] soc: octeontx2: Add Marvell OcteonTX2 CGX driver

[sunil . kovvuri]

From: Sunil Goutham 

This patch adds basic template for Marvell OcteonTX2's
CGX ethernet interface driver. Just the probe.
RVU AF driver will use APIs exported by this driver
for various things like PF to physical interface mapping,
loopback mode, interface stats etc. Hence marged both
drivers into a single module.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/Kconfig|   3 +-
 drivers/soc/marvell/octeontx2/Makefile |   2 +-
 drivers/soc/marvell/octeontx2/cgx.c| 102 +
 drivers/soc/marvell/octeontx2/cgx.h|  22 +++
 drivers/soc/marvell/octeontx2/rvu.c|  14 -
 5 files changed, 140 insertions(+), 3 deletions(-)
 create mode 100644 drivers/soc/marvell/octeontx2/cgx.c
 create mode 100644 drivers/soc/marvell/octeontx2/cgx.h

diff --git a/drivers/soc/marvell/Kconfig b/drivers/soc/marvell/Kconfig
index 428d22e..8f36f3a 100644
--- a/drivers/soc/marvell/Kconfig
+++ b/drivers/soc/marvell/Kconfig
@@ -11,7 +11,8 @@ config OCTEONTX2_AF
tristate "OcteonTX2 RVU Admin Function driver"
select OCTEONTX2_MBOX
depends on ARM64 && PCI
-   help
+   ---help---
  This driver supports Marvell's OcteonTX2 Resource Virtualization
  Unit's admin function manager which manages all RVU HW resources.
+
 endmenu
diff --git a/drivers/soc/marvell/octeontx2/Makefile 
b/drivers/soc/marvell/octeontx2/Makefile
index ac17cb9..8646421 100644
--- a/drivers/soc/marvell/octeontx2/Makefile
+++ b/drivers/soc/marvell/octeontx2/Makefile
@@ -7,4 +7,4 @@ obj-$(CONFIG_OCTEONTX2_MBOX) += octeontx2_mbox.o
 obj-$(CONFIG_OCTEONTX2_AF) += octeontx2_af.o
 
 octeontx2_mbox-y := mbox.o
-octeontx2_af-y := rvu.o
+octeontx2_af-y := cgx.o rvu.o
diff --git a/drivers/soc/marvell/octeontx2/cgx.c 
b/drivers/soc/marvell/octeontx2/cgx.c
new file mode 100644
index 000..47aa4cb
--- /dev/null
+++ b/drivers/soc/marvell/octeontx2/cgx.c
@@ -0,0 +1,102 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Marvell OcteonTx2 CGX driver
+ *
+ * Copyright (C) 2018 Marvell International Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "cgx.h"
+
+#define DRV_NAME   "octeontx2-cgx"
+#define DRV_STRING  "Marvell OcteonTX2 CGX/MAC Driver"
+#define DRV_VERSION"1.0"
+
+struct cgx {
+   void __iomem*reg_base;
+   struct pci_dev  *pdev;
+   u8  cgx_id;
+};
+
+/* Supported devices */
+static const struct pci_device_id cgx_id_table[] = {
+   { PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_CGX) },
+   { 0, }  /* end of table */
+};
+
+MODULE_AUTHOR("Marvell International Ltd.");
+MODULE_DESCRIPTION(DRV_STRING);
+MODULE_LICENSE("GPL v2");
+MODULE_VERSION(DRV_VERSION);
+MODULE_DEVICE_TABLE(pci, cgx_id_table);
+
+static int cgx_probe(struct pci_dev *pdev, const struct pci_device_id *id)
+{
+   int err;
+   struct device *dev = >dev;
+   struct cgx *cgx;
+
+   cgx = devm_kzalloc(dev, sizeof(*cgx), GFP_KERNEL);
+   if (!cgx)
+   return -ENOMEM;
+   cgx->pdev = pdev;
+
+   pci_set_drvdata(pdev, cgx);
+
+   err = pci_enable_device(pdev);
+   if (err) {
+   dev_err(dev, "Failed to enable PCI device\n");
+   pci_set_drvdata(pdev, NULL);
+   return err;
+   }
+
+   err = pci_request_regions(pdev, DRV_NAME);
+   if (err) {
+   dev_err(dev, "PCI request regions failed 0x%x\n", err);
+   goto err_disable_device;
+   }
+
+   /* MAP configuration registers */
+   cgx->reg_base = pcim_iomap(pdev, PCI_CFG_REG_BAR_NUM, 0);
+   if (!cgx->reg_base) {
+   dev_err(dev, "CGX: Cannot map CSR memory space, aborting\n");
+   err = -ENOMEM;
+   goto err_release_regions;
+   }
+
+   return 0;
+
+err_release_regions:
+   pci_release_regions(pdev);
+err_disable_device:
+   pci_disable_device(pdev);
+   pci_set_drvdata(pdev, NULL);
+   return err;
+}
+
+static void cgx_remove(struct pci_dev *pdev)
+{
+   pci_release_regions(pdev);
+   pci_disable_device(pdev);
+   pci_set_drvdata(pdev, NULL);
+}
+
+struct pci_driver cgx_driver = {
+   .name = DRV_NAME,
+   .id_table = cgx_id_table,
+   .probe = cgx_probe,
+   .remove = cgx_remove,
+};
diff --git a/drivers/soc/marvell/octeontx2/cgx.h 
b/drivers/soc/marvell/octeontx2/cgx.h
new file mode 100644
index 000..a7d4b39
--- /dev/null
+++ b/drivers/soc/marvell/octeontx2/cgx.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0
+ * Marvell OcteonTx2 CGX driver
+ *
+ * Copyright (C) 2018 Marvell International Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it 

[PATCH v2 10/15] soc: octeontx2: Reconfig MSIX base with IOVA

[sunil . kovvuri]

From: Geetha sowjanya 

HW interprets RVU_AF_MSIXTR_BASE address as an IOVA, hence
create a IOMMU mapping for the physcial address configured by
firmware and reconfig RVU_AF_MSIXTR_BASE with IOVA.

Signed-off-by: Geetha sowjanya 
Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/rvu.c | 33 ++---
 drivers/soc/marvell/octeontx2/rvu.h |  1 +
 2 files changed, 31 insertions(+), 3 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 8ac3524..40684c9 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -442,9 +442,10 @@ static int rvu_setup_msix_resources(struct rvu *rvu)
 {
struct rvu_hwinfo *hw = rvu->hw;
int pf, vf, numvfs, hwvf, err;
+   int nvecs, offset, max_msix;
struct rvu_pfvf *pfvf;
-   int nvecs, offset;
-   u64 cfg;
+   u64 cfg, phy_addr;
+   dma_addr_t iova;
 
for (pf = 0; pf < hw->total_pfs; pf++) {
cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_PFX_CFG(pf));
@@ -523,6 +524,22 @@ static int rvu_setup_msix_resources(struct rvu *rvu)
}
}
 
+   /* HW interprets RVU_AF_MSIXTR_BASE address as an IOVA, hence
+* create a IOMMU mapping for the physcial address configured by
+* firmware and reconfig RVU_AF_MSIXTR_BASE with IOVA.
+*/
+   cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_CONST);
+   max_msix = cfg & 0xF;
+   phy_addr = rvu_read64(rvu, BLKADDR_RVUM, RVU_AF_MSIXTR_BASE);
+   iova = dma_map_single(rvu->dev, (void *)phy_addr,
+ max_msix * PCI_MSIX_ENTRY_SIZE,
+ DMA_BIDIRECTIONAL);
+   if (dma_mapping_error(rvu->dev, iova))
+   return -ENOMEM;
+
+   rvu_write64(rvu, BLKADDR_RVUM, RVU_AF_MSIXTR_BASE, (u64)iova);
+   rvu->msix_base_iova = iova;
+
return 0;
 }
 
@@ -531,7 +548,8 @@ static void rvu_free_hw_resources(struct rvu *rvu)
struct rvu_hwinfo *hw = rvu->hw;
struct rvu_block *block;
struct rvu_pfvf  *pfvf;
-   int id;
+   int id, max_msix;
+   u64 cfg;
 
/* Free block LF bitmaps */
for (id = 0; id < BLK_COUNT; id++) {
@@ -549,6 +567,15 @@ static void rvu_free_hw_resources(struct rvu *rvu)
pfvf = >hwvf[id];
kfree(pfvf->msix.bmap);
}
+
+   /* Unmap MSIX vector base IOVA mapping */
+   if (!rvu->msix_base_iova)
+   return;
+   cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_CONST);
+   max_msix = cfg & 0xF;
+   dma_unmap_single(rvu->dev, rvu->msix_base_iova,
+max_msix * PCI_MSIX_ENTRY_SIZE,
+DMA_BIDIRECTIONAL);
 }
 
 static int rvu_setup_hw_resources(struct rvu *rvu)
diff --git a/drivers/soc/marvell/octeontx2/rvu.h 
b/drivers/soc/marvell/octeontx2/rvu.h
index 7435e83..92c2022 100644
--- a/drivers/soc/marvell/octeontx2/rvu.h
+++ b/drivers/soc/marvell/octeontx2/rvu.h
@@ -99,6 +99,7 @@ struct rvu {
u16 num_vec;
char*irq_name;
bool*irq_allocated;
+   dma_addr_t  msix_base_iova;
 };
 
 static inline void rvu_write64(struct rvu *rvu, u64 block, u64 offset, u64 val)
-- 
2.7.4

[PATCH v2 09/15] soc: octeontx2: Configure block LF's MSIX vector offset

[sunil . kovvuri]

From: Sunil Goutham 

Firmware configures a certain number of MSIX vectors to each of
enabled RVU PF/VF. When a block LF is attached to a PF/VF, number
of MSIX vectors needed by that LF are set aside (out of PF/VF's
total MSIX vectors) and LF's msix_offset is configured in HW.

Also added support for a RVU PF/VF to retrieve that block LF's
MSIX vector offset information from AF via mbox.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/mbox.h   |  18 ++
 drivers/soc/marvell/octeontx2/rvu.c| 333 -
 drivers/soc/marvell/octeontx2/rvu.h|   7 +
 drivers/soc/marvell/octeontx2/rvu_struct.h |   2 +
 4 files changed, 357 insertions(+), 3 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/mbox.h 
b/drivers/soc/marvell/octeontx2/mbox.h
index 7280d49..bedf0ee 100644
--- a/drivers/soc/marvell/octeontx2/mbox.h
+++ b/drivers/soc/marvell/octeontx2/mbox.h
@@ -122,6 +122,7 @@ static inline struct mbox_msghdr 
*otx2_mbox_alloc_msg(struct otx2_mbox *mbox,
 M(READY,   0x001, msg_req, ready_msg_rsp)  \
 M(ATTACH_RESOURCES,0x002, rsrc_attach, msg_rsp)\
 M(DETACH_RESOURCES,0x003, rsrc_detach, msg_rsp)\
+M(MSIX_OFFSET, 0x004, msg_req, msix_offset_rsp)\
 /* CGX mbox IDs (range 0x200 - 0x3FF) */   \
 /* NPA mbox IDs (range 0x400 - 0x5FF) */   \
 /* SSO/SSOW mbox IDs (range 0x600 - 0x7FF) */  \
@@ -190,4 +191,21 @@ struct rsrc_detach {
u8 cptlfs:1;
 };
 
+#define MSIX_VECTOR_INVALID0x
+#define MAX_RVU_BLKLF_CNT  256
+
+struct msix_offset_rsp {
+   struct mbox_msghdr hdr;
+   u16  npa_msixoff;
+   u16  nix_msixoff;
+   u8   sso;
+   u8   ssow;
+   u8   timlfs;
+   u8   cptlfs;
+   u16  sso_msixoff[MAX_RVU_BLKLF_CNT];
+   u16  ssow_msixoff[MAX_RVU_BLKLF_CNT];
+   u16  timlf_msixoff[MAX_RVU_BLKLF_CNT];
+   u16  cptlf_msixoff[MAX_RVU_BLKLF_CNT];
+};
+
 #endif /* MBOX_H */
diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 39dc45d..8ac3524 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -24,6 +24,11 @@
 
 static int rvu_get_hwvf(struct rvu *rvu, int pcifunc);
 
+static void rvu_set_msix_offset(struct rvu *rvu, struct rvu_pfvf *pfvf,
+   struct rvu_block *block, int lf);
+static void rvu_clear_msix_offset(struct rvu *rvu, struct rvu_pfvf *pfvf,
+ struct rvu_block *block, int lf);
+
 /* Supported devices */
 static const struct pci_device_id rvu_id_table[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_RVU_AF) },
@@ -75,6 +80,45 @@ int rvu_alloc_rsrc(struct rsrc_bmap *rsrc)
return id;
 }
 
+static int rvu_alloc_rsrc_contig(struct rsrc_bmap *rsrc, int nrsrc)
+{
+   int start;
+
+   if (!rsrc->bmap)
+   return -EINVAL;
+
+   start = bitmap_find_next_zero_area(rsrc->bmap, rsrc->max, 0, nrsrc, 0);
+   if (start >= rsrc->max)
+   return -ENOSPC;
+
+   bitmap_set(rsrc->bmap, start, nrsrc);
+   return start;
+}
+
+static void rvu_free_rsrc_contig(struct rsrc_bmap *rsrc, int nrsrc, int start)
+{
+   if (!rsrc->bmap)
+   return;
+   if (start >= rsrc->max)
+   return;
+
+   bitmap_clear(rsrc->bmap, start, nrsrc);
+}
+
+static bool rvu_rsrc_check_contig(struct rsrc_bmap *rsrc, int nrsrc)
+{
+   int start;
+
+   if (!rsrc->bmap)
+   return false;
+
+   start = bitmap_find_next_zero_area(rsrc->bmap, rsrc->max, 0, nrsrc, 0);
+   if (start >= rsrc->max)
+   return false;
+
+   return true;
+}
+
 void rvu_free_rsrc(struct rsrc_bmap *rsrc, int id)
 {
if (!rsrc->bmap)
@@ -103,6 +147,26 @@ int rvu_alloc_bitmap(struct rsrc_bmap *rsrc)
return 0;
 }
 
+/* Get block LF's HW index from a PF_FUNC's block slot number */
+int rvu_get_lf(struct rvu *rvu, struct rvu_block *block, u16 pcifunc, u16 slot)
+{
+   int lf;
+   u16 match = 0;
+
+   spin_lock(>rsrc_lock);
+   for (lf = 0; lf < block->lf.max; lf++) {
+   if (block->fn_map[lf] == pcifunc) {
+   if (slot == match) {
+   spin_unlock(>rsrc_lock);
+   return lf;
+   }
+   match++;
+   }
+   }
+   spin_unlock(>rsrc_lock);
+   return -ENODEV;
+}
+
 /* Convert BLOCK_TYPE_E to a BLOCK_ADDR_E.
  * Some silicon variants of OcteonTX2 supports
  * multiple blocks of same type.
@@ -237,6 +301,16 @@ inline int rvu_get_pf(u16 pcifunc)
return (pcifunc >> RVU_PFVF_PF_SHIFT) & RVU_PFVF_PF_MASK;
 }
 
+void rvu_get_pf_numvfs(struct rvu *rvu, int pf, int *numvfs, int *hwvf)
+{
+   u64 cfg;
+
+   /* Get numVFs attached to this 

[PATCH v2 10/15] soc: octeontx2: Reconfig MSIX base with IOVA

[sunil . kovvuri]

From: Geetha sowjanya 

HW interprets RVU_AF_MSIXTR_BASE address as an IOVA, hence
create a IOMMU mapping for the physcial address configured by
firmware and reconfig RVU_AF_MSIXTR_BASE with IOVA.

Signed-off-by: Geetha sowjanya 
Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/rvu.c | 33 ++---
 drivers/soc/marvell/octeontx2/rvu.h |  1 +
 2 files changed, 31 insertions(+), 3 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 8ac3524..40684c9 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -442,9 +442,10 @@ static int rvu_setup_msix_resources(struct rvu *rvu)
 {
struct rvu_hwinfo *hw = rvu->hw;
int pf, vf, numvfs, hwvf, err;
+   int nvecs, offset, max_msix;
struct rvu_pfvf *pfvf;
-   int nvecs, offset;
-   u64 cfg;
+   u64 cfg, phy_addr;
+   dma_addr_t iova;
 
for (pf = 0; pf < hw->total_pfs; pf++) {
cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_PFX_CFG(pf));
@@ -523,6 +524,22 @@ static int rvu_setup_msix_resources(struct rvu *rvu)
}
}
 
+   /* HW interprets RVU_AF_MSIXTR_BASE address as an IOVA, hence
+* create a IOMMU mapping for the physcial address configured by
+* firmware and reconfig RVU_AF_MSIXTR_BASE with IOVA.
+*/
+   cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_CONST);
+   max_msix = cfg & 0xF;
+   phy_addr = rvu_read64(rvu, BLKADDR_RVUM, RVU_AF_MSIXTR_BASE);
+   iova = dma_map_single(rvu->dev, (void *)phy_addr,
+ max_msix * PCI_MSIX_ENTRY_SIZE,
+ DMA_BIDIRECTIONAL);
+   if (dma_mapping_error(rvu->dev, iova))
+   return -ENOMEM;
+
+   rvu_write64(rvu, BLKADDR_RVUM, RVU_AF_MSIXTR_BASE, (u64)iova);
+   rvu->msix_base_iova = iova;
+
return 0;
 }
 
@@ -531,7 +548,8 @@ static void rvu_free_hw_resources(struct rvu *rvu)
struct rvu_hwinfo *hw = rvu->hw;
struct rvu_block *block;
struct rvu_pfvf  *pfvf;
-   int id;
+   int id, max_msix;
+   u64 cfg;
 
/* Free block LF bitmaps */
for (id = 0; id < BLK_COUNT; id++) {
@@ -549,6 +567,15 @@ static void rvu_free_hw_resources(struct rvu *rvu)
pfvf = >hwvf[id];
kfree(pfvf->msix.bmap);
}
+
+   /* Unmap MSIX vector base IOVA mapping */
+   if (!rvu->msix_base_iova)
+   return;
+   cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_CONST);
+   max_msix = cfg & 0xF;
+   dma_unmap_single(rvu->dev, rvu->msix_base_iova,
+max_msix * PCI_MSIX_ENTRY_SIZE,
+DMA_BIDIRECTIONAL);
 }
 
 static int rvu_setup_hw_resources(struct rvu *rvu)
diff --git a/drivers/soc/marvell/octeontx2/rvu.h 
b/drivers/soc/marvell/octeontx2/rvu.h
index 7435e83..92c2022 100644
--- a/drivers/soc/marvell/octeontx2/rvu.h
+++ b/drivers/soc/marvell/octeontx2/rvu.h
@@ -99,6 +99,7 @@ struct rvu {
u16 num_vec;
char*irq_name;
bool*irq_allocated;
+   dma_addr_t  msix_base_iova;
 };
 
 static inline void rvu_write64(struct rvu *rvu, u64 block, u64 offset, u64 val)
-- 
2.7.4

[PATCH v2 09/15] soc: octeontx2: Configure block LF's MSIX vector offset

[sunil . kovvuri]

From: Sunil Goutham 

Firmware configures a certain number of MSIX vectors to each of
enabled RVU PF/VF. When a block LF is attached to a PF/VF, number
of MSIX vectors needed by that LF are set aside (out of PF/VF's
total MSIX vectors) and LF's msix_offset is configured in HW.

Also added support for a RVU PF/VF to retrieve that block LF's
MSIX vector offset information from AF via mbox.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/mbox.h   |  18 ++
 drivers/soc/marvell/octeontx2/rvu.c| 333 -
 drivers/soc/marvell/octeontx2/rvu.h|   7 +
 drivers/soc/marvell/octeontx2/rvu_struct.h |   2 +
 4 files changed, 357 insertions(+), 3 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/mbox.h 
b/drivers/soc/marvell/octeontx2/mbox.h
index 7280d49..bedf0ee 100644
--- a/drivers/soc/marvell/octeontx2/mbox.h
+++ b/drivers/soc/marvell/octeontx2/mbox.h
@@ -122,6 +122,7 @@ static inline struct mbox_msghdr 
*otx2_mbox_alloc_msg(struct otx2_mbox *mbox,
 M(READY,   0x001, msg_req, ready_msg_rsp)  \
 M(ATTACH_RESOURCES,0x002, rsrc_attach, msg_rsp)\
 M(DETACH_RESOURCES,0x003, rsrc_detach, msg_rsp)\
+M(MSIX_OFFSET, 0x004, msg_req, msix_offset_rsp)\
 /* CGX mbox IDs (range 0x200 - 0x3FF) */   \
 /* NPA mbox IDs (range 0x400 - 0x5FF) */   \
 /* SSO/SSOW mbox IDs (range 0x600 - 0x7FF) */  \
@@ -190,4 +191,21 @@ struct rsrc_detach {
u8 cptlfs:1;
 };
 
+#define MSIX_VECTOR_INVALID0x
+#define MAX_RVU_BLKLF_CNT  256
+
+struct msix_offset_rsp {
+   struct mbox_msghdr hdr;
+   u16  npa_msixoff;
+   u16  nix_msixoff;
+   u8   sso;
+   u8   ssow;
+   u8   timlfs;
+   u8   cptlfs;
+   u16  sso_msixoff[MAX_RVU_BLKLF_CNT];
+   u16  ssow_msixoff[MAX_RVU_BLKLF_CNT];
+   u16  timlf_msixoff[MAX_RVU_BLKLF_CNT];
+   u16  cptlf_msixoff[MAX_RVU_BLKLF_CNT];
+};
+
 #endif /* MBOX_H */
diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 39dc45d..8ac3524 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -24,6 +24,11 @@
 
 static int rvu_get_hwvf(struct rvu *rvu, int pcifunc);
 
+static void rvu_set_msix_offset(struct rvu *rvu, struct rvu_pfvf *pfvf,
+   struct rvu_block *block, int lf);
+static void rvu_clear_msix_offset(struct rvu *rvu, struct rvu_pfvf *pfvf,
+ struct rvu_block *block, int lf);
+
 /* Supported devices */
 static const struct pci_device_id rvu_id_table[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_RVU_AF) },
@@ -75,6 +80,45 @@ int rvu_alloc_rsrc(struct rsrc_bmap *rsrc)
return id;
 }
 
+static int rvu_alloc_rsrc_contig(struct rsrc_bmap *rsrc, int nrsrc)
+{
+   int start;
+
+   if (!rsrc->bmap)
+   return -EINVAL;
+
+   start = bitmap_find_next_zero_area(rsrc->bmap, rsrc->max, 0, nrsrc, 0);
+   if (start >= rsrc->max)
+   return -ENOSPC;
+
+   bitmap_set(rsrc->bmap, start, nrsrc);
+   return start;
+}
+
+static void rvu_free_rsrc_contig(struct rsrc_bmap *rsrc, int nrsrc, int start)
+{
+   if (!rsrc->bmap)
+   return;
+   if (start >= rsrc->max)
+   return;
+
+   bitmap_clear(rsrc->bmap, start, nrsrc);
+}
+
+static bool rvu_rsrc_check_contig(struct rsrc_bmap *rsrc, int nrsrc)
+{
+   int start;
+
+   if (!rsrc->bmap)
+   return false;
+
+   start = bitmap_find_next_zero_area(rsrc->bmap, rsrc->max, 0, nrsrc, 0);
+   if (start >= rsrc->max)
+   return false;
+
+   return true;
+}
+
 void rvu_free_rsrc(struct rsrc_bmap *rsrc, int id)
 {
if (!rsrc->bmap)
@@ -103,6 +147,26 @@ int rvu_alloc_bitmap(struct rsrc_bmap *rsrc)
return 0;
 }
 
+/* Get block LF's HW index from a PF_FUNC's block slot number */
+int rvu_get_lf(struct rvu *rvu, struct rvu_block *block, u16 pcifunc, u16 slot)
+{
+   int lf;
+   u16 match = 0;
+
+   spin_lock(>rsrc_lock);
+   for (lf = 0; lf < block->lf.max; lf++) {
+   if (block->fn_map[lf] == pcifunc) {
+   if (slot == match) {
+   spin_unlock(>rsrc_lock);
+   return lf;
+   }
+   match++;
+   }
+   }
+   spin_unlock(>rsrc_lock);
+   return -ENODEV;
+}
+
 /* Convert BLOCK_TYPE_E to a BLOCK_ADDR_E.
  * Some silicon variants of OcteonTX2 supports
  * multiple blocks of same type.
@@ -237,6 +301,16 @@ inline int rvu_get_pf(u16 pcifunc)
return (pcifunc >> RVU_PFVF_PF_SHIFT) & RVU_PFVF_PF_MASK;
 }
 
+void rvu_get_pf_numvfs(struct rvu *rvu, int pf, int *numvfs, int *hwvf)
+{
+   u64 cfg;
+
+   /* Get numVFs attached to this 

[PATCH v2 08/15] soc: octeontx2: Add RVU block LF provisioning support

[sunil . kovvuri]

From: Sunil Goutham 

Added support for a RVU PF/VF to request AF via mailbox
to attach or detach NPA/NIX/SSO/SSOW/TIM/CPT block LFs.
Also supports partial detachment and modifying current
LF attached count of a certian block type.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/mbox.h|  45 ++-
 drivers/soc/marvell/octeontx2/rvu.c | 472 +++-
 drivers/soc/marvell/octeontx2/rvu.h |   8 +-
 drivers/soc/marvell/octeontx2/rvu_reg.h |   8 +-
 4 files changed, 523 insertions(+), 10 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/mbox.h 
b/drivers/soc/marvell/octeontx2/mbox.h
index fc593f0..7280d49 100644
--- a/drivers/soc/marvell/octeontx2/mbox.h
+++ b/drivers/soc/marvell/octeontx2/mbox.h
@@ -118,7 +118,17 @@ static inline struct mbox_msghdr 
*otx2_mbox_alloc_msg(struct otx2_mbox *mbox,
 #define MBOX_MSG_MAX   0x
 
 #define MBOX_MESSAGES  \
-M(READY,   0x001, msg_req, ready_msg_rsp)
+/* Generic mbox IDs (range 0x000 - 0x1FF) */   \
+M(READY,   0x001, msg_req, ready_msg_rsp)  \
+M(ATTACH_RESOURCES,0x002, rsrc_attach, msg_rsp)\
+M(DETACH_RESOURCES,0x003, rsrc_detach, msg_rsp)\
+/* CGX mbox IDs (range 0x200 - 0x3FF) */   \
+/* NPA mbox IDs (range 0x400 - 0x5FF) */   \
+/* SSO/SSOW mbox IDs (range 0x600 - 0x7FF) */  \
+/* TIM mbox IDs (range 0x800 - 0x9FF) */   \
+/* CPT mbox IDs (range 0xA00 - 0xBFF) */   \
+/* NPC mbox IDs (range 0x6000 - 0x7FFF) */ \
+/* NIX mbox IDs (range 0x8000 - 0x) */ \
 
 enum {
 #define M(_name, _id, _1, _2) MBOX_MSG_ ## _name = _id,
@@ -147,4 +157,37 @@ struct ready_msg_rsp {
u16sclk_feq;/* SCLK frequency */
 };
 
+/* Structure for requesting resource provisioning.
+ * 'modify' flag to be used when either requesting more
+ * or to detach partial of a cetain resource type.
+ * Rest of the fields specify how many of what type to
+ * be attached.
+ */
+struct rsrc_attach {
+   struct mbox_msghdr hdr;
+   u8   modify:1;
+   u8   npalf:1;
+   u8   nixlf:1;
+   u16  sso;
+   u16  ssow;
+   u16  timlfs;
+   u16  cptlfs;
+};
+
+/* Structure for relinquishing resources.
+ * 'partial' flag to be used when relinquishing all resources
+ * but only of a certain type. If not set, all resources of all
+ * types provisioned to the RVU function will be detached.
+ */
+struct rsrc_detach {
+   struct mbox_msghdr hdr;
+   u8 partial:1;
+   u8 npalf:1;
+   u8 nixlf:1;
+   u8 sso:1;
+   u8 ssow:1;
+   u8 timlfs:1;
+   u8 cptlfs:1;
+};
+
 #endif /* MBOX_H */
diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 9539ab9..39dc45d 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -59,6 +59,41 @@ int rvu_poll_reg(struct rvu *rvu, u64 block, u64 offset, u64 
mask, bool zero)
return -EBUSY;
 }
 
+int rvu_alloc_rsrc(struct rsrc_bmap *rsrc)
+{
+   int id;
+
+   if (!rsrc->bmap)
+   return -EINVAL;
+
+   id = find_first_zero_bit(rsrc->bmap, rsrc->max);
+   if (id >= rsrc->max)
+   return -ENOSPC;
+
+   __set_bit(id, rsrc->bmap);
+
+   return id;
+}
+
+void rvu_free_rsrc(struct rsrc_bmap *rsrc, int id)
+{
+   if (!rsrc->bmap)
+   return;
+
+   __clear_bit(id, rsrc->bmap);
+}
+
+int rvu_rsrc_free_count(struct rsrc_bmap *rsrc)
+{
+   int used;
+
+   if (!rsrc->bmap)
+   return 0;
+
+   used = bitmap_weight(rsrc->bmap, rsrc->max);
+   return (rsrc->max - used);
+}
+
 int rvu_alloc_bitmap(struct rsrc_bmap *rsrc)
 {
rsrc->bmap = kcalloc(BITS_TO_LONGS(rsrc->max),
@@ -68,6 +103,78 @@ int rvu_alloc_bitmap(struct rsrc_bmap *rsrc)
return 0;
 }
 
+/* Convert BLOCK_TYPE_E to a BLOCK_ADDR_E.
+ * Some silicon variants of OcteonTX2 supports
+ * multiple blocks of same type.
+ *
+ * @pcifunc has to be zero when no LF is yet attached.
+ */
+int rvu_get_blkaddr(struct rvu *rvu, int blktype, u16 pcifunc)
+{
+   int devnum, blkaddr = -ENODEV;
+   u64 cfg, reg;
+   bool is_pf;
+
+   switch (blktype) {
+   case BLKTYPE_NPA:
+   blkaddr = BLKADDR_NPA;
+   goto exit;
+   case BLKTYPE_NIX:
+   /* For now assume NIX0 */
+   if (!pcifunc) {
+   blkaddr = BLKADDR_NIX0;
+   goto exit;
+   }
+   break;
+   case BLKTYPE_SSO:
+   blkaddr = BLKADDR_SSO;
+   goto exit;
+   case BLKTYPE_SSOW:
+   blkaddr = BLKADDR_SSOW;
+   goto exit;
+  

[PATCH v2 08/15] soc: octeontx2: Add RVU block LF provisioning support

[sunil . kovvuri]

From: Sunil Goutham 

Added support for a RVU PF/VF to request AF via mailbox
to attach or detach NPA/NIX/SSO/SSOW/TIM/CPT block LFs.
Also supports partial detachment and modifying current
LF attached count of a certian block type.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/mbox.h|  45 ++-
 drivers/soc/marvell/octeontx2/rvu.c | 472 +++-
 drivers/soc/marvell/octeontx2/rvu.h |   8 +-
 drivers/soc/marvell/octeontx2/rvu_reg.h |   8 +-
 4 files changed, 523 insertions(+), 10 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/mbox.h 
b/drivers/soc/marvell/octeontx2/mbox.h
index fc593f0..7280d49 100644
--- a/drivers/soc/marvell/octeontx2/mbox.h
+++ b/drivers/soc/marvell/octeontx2/mbox.h
@@ -118,7 +118,17 @@ static inline struct mbox_msghdr 
*otx2_mbox_alloc_msg(struct otx2_mbox *mbox,
 #define MBOX_MSG_MAX   0x
 
 #define MBOX_MESSAGES  \
-M(READY,   0x001, msg_req, ready_msg_rsp)
+/* Generic mbox IDs (range 0x000 - 0x1FF) */   \
+M(READY,   0x001, msg_req, ready_msg_rsp)  \
+M(ATTACH_RESOURCES,0x002, rsrc_attach, msg_rsp)\
+M(DETACH_RESOURCES,0x003, rsrc_detach, msg_rsp)\
+/* CGX mbox IDs (range 0x200 - 0x3FF) */   \
+/* NPA mbox IDs (range 0x400 - 0x5FF) */   \
+/* SSO/SSOW mbox IDs (range 0x600 - 0x7FF) */  \
+/* TIM mbox IDs (range 0x800 - 0x9FF) */   \
+/* CPT mbox IDs (range 0xA00 - 0xBFF) */   \
+/* NPC mbox IDs (range 0x6000 - 0x7FFF) */ \
+/* NIX mbox IDs (range 0x8000 - 0x) */ \
 
 enum {
 #define M(_name, _id, _1, _2) MBOX_MSG_ ## _name = _id,
@@ -147,4 +157,37 @@ struct ready_msg_rsp {
u16sclk_feq;/* SCLK frequency */
 };
 
+/* Structure for requesting resource provisioning.
+ * 'modify' flag to be used when either requesting more
+ * or to detach partial of a cetain resource type.
+ * Rest of the fields specify how many of what type to
+ * be attached.
+ */
+struct rsrc_attach {
+   struct mbox_msghdr hdr;
+   u8   modify:1;
+   u8   npalf:1;
+   u8   nixlf:1;
+   u16  sso;
+   u16  ssow;
+   u16  timlfs;
+   u16  cptlfs;
+};
+
+/* Structure for relinquishing resources.
+ * 'partial' flag to be used when relinquishing all resources
+ * but only of a certain type. If not set, all resources of all
+ * types provisioned to the RVU function will be detached.
+ */
+struct rsrc_detach {
+   struct mbox_msghdr hdr;
+   u8 partial:1;
+   u8 npalf:1;
+   u8 nixlf:1;
+   u8 sso:1;
+   u8 ssow:1;
+   u8 timlfs:1;
+   u8 cptlfs:1;
+};
+
 #endif /* MBOX_H */
diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 9539ab9..39dc45d 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -59,6 +59,41 @@ int rvu_poll_reg(struct rvu *rvu, u64 block, u64 offset, u64 
mask, bool zero)
return -EBUSY;
 }
 
+int rvu_alloc_rsrc(struct rsrc_bmap *rsrc)
+{
+   int id;
+
+   if (!rsrc->bmap)
+   return -EINVAL;
+
+   id = find_first_zero_bit(rsrc->bmap, rsrc->max);
+   if (id >= rsrc->max)
+   return -ENOSPC;
+
+   __set_bit(id, rsrc->bmap);
+
+   return id;
+}
+
+void rvu_free_rsrc(struct rsrc_bmap *rsrc, int id)
+{
+   if (!rsrc->bmap)
+   return;
+
+   __clear_bit(id, rsrc->bmap);
+}
+
+int rvu_rsrc_free_count(struct rsrc_bmap *rsrc)
+{
+   int used;
+
+   if (!rsrc->bmap)
+   return 0;
+
+   used = bitmap_weight(rsrc->bmap, rsrc->max);
+   return (rsrc->max - used);
+}
+
 int rvu_alloc_bitmap(struct rsrc_bmap *rsrc)
 {
rsrc->bmap = kcalloc(BITS_TO_LONGS(rsrc->max),
@@ -68,6 +103,78 @@ int rvu_alloc_bitmap(struct rsrc_bmap *rsrc)
return 0;
 }
 
+/* Convert BLOCK_TYPE_E to a BLOCK_ADDR_E.
+ * Some silicon variants of OcteonTX2 supports
+ * multiple blocks of same type.
+ *
+ * @pcifunc has to be zero when no LF is yet attached.
+ */
+int rvu_get_blkaddr(struct rvu *rvu, int blktype, u16 pcifunc)
+{
+   int devnum, blkaddr = -ENODEV;
+   u64 cfg, reg;
+   bool is_pf;
+
+   switch (blktype) {
+   case BLKTYPE_NPA:
+   blkaddr = BLKADDR_NPA;
+   goto exit;
+   case BLKTYPE_NIX:
+   /* For now assume NIX0 */
+   if (!pcifunc) {
+   blkaddr = BLKADDR_NIX0;
+   goto exit;
+   }
+   break;
+   case BLKTYPE_SSO:
+   blkaddr = BLKADDR_SSO;
+   goto exit;
+   case BLKTYPE_SSOW:
+   blkaddr = BLKADDR_SSOW;
+   goto exit;
+  

[PATCH v2 00/15] soc: octeontx2: Add RVU admin function driver

[sunil . kovvuri]

From: Sunil Goutham 

Resource virtualization unit (RVU) on Marvell's OcteonTX2 SOC supports
multiple PCIe SRIOV physical functions (PFs) and virtual functions (VFs).
PF0 is called administrative / admin function (AF) and has privilege access
to registers to provision different RVU functional blocks to each of
PF/VF.

This admin function (AF) driver acts as a configuration / administrative
software which provisions functional blocks to a PF/VF on demand for them
to work as one of the following
 - A basic network controller (i.e NIC).
 - NIC with packet filtering, shaping and scheduling capabilities.
 - A crypto device.
 - A combination of above etc.

PF/VFs communicate with admin function via a shared memory region.
This patch series adds logic for the following
 - RVU AF driver with functional blocks provisioning support
 - Mailbox infrastructure for communication between AF and PFs.
 - CGX driver which provides information about physcial network
   interfaces which AF processes and forwards required info to
   PF/VF drivers.

This is the first set of patches out of 70 odd patches.

Note: This driver neither receives any data nor processes it i.e no I/O,
  just does the hardware configuration.

Changes from v1:
 1 Merged RVU admin function and CGX drivers into a single module
   - Suggested by Arnd Bergmann
 2 Pulled mbox communication APIs into a separate module to remove
   admin function driver dependency in a VM where AF is not attached.
   - Suggested by Arnd Bergmann

Aleksey Makarov (2):
  soc: octeontx2: Add mailbox support infra
  soc: octeontx2: Convert mbox msg id check to a macro

Geetha sowjanya (1):
  soc: octeontx2: Reconfig MSIX base with IOVA

Linu Cherian (3):
  soc: octeontx2: Set RVU PFs to CGX LMACs mapping
  soc: octeontx2: Add support for CGX link management
  soc: octeontx2: Register for CGX lmac events

Sunil Goutham (9):
  soc: octeontx2: Add Marvell OcteonTX2 RVU AF driver
  soc: octeontx2: Reset all RVU blocks
  soc: octeontx2: Gather RVU blocks HW info
  soc: octeontx2: Add mailbox IRQ and msg handlers
  soc: octeontx2: Scan blocks for LFs provisioned to PF/VF
  soc: octeontx2: Add RVU block LF provisioning support
  soc: octeontx2: Configure block LF's MSIX vector offset
  soc: octeontx2: Add Marvell OcteonTX2 CGX driver
  MAINTAINERS: Add entry for Marvell OcteonTX2 Admin Function driver

 MAINTAINERS|   10 +
 drivers/soc/Kconfig|1 +
 drivers/soc/Makefile   |1 +
 drivers/soc/marvell/Kconfig|   18 +
 drivers/soc/marvell/Makefile   |2 +
 drivers/soc/marvell/octeontx2/Makefile |   10 +
 drivers/soc/marvell/octeontx2/cgx.c|  517 +
 drivers/soc/marvell/octeontx2/cgx.h|   65 ++
 drivers/soc/marvell/octeontx2/cgx_fw_if.h  |  225 
 drivers/soc/marvell/octeontx2/mbox.c   |  303 +
 drivers/soc/marvell/octeontx2/mbox.h   |  211 
 drivers/soc/marvell/octeontx2/rvu.c| 1637 
 drivers/soc/marvell/octeontx2/rvu.h|  158 +++
 drivers/soc/marvell/octeontx2/rvu_cgx.c|  194 
 drivers/soc/marvell/octeontx2/rvu_reg.h|  442 
 drivers/soc/marvell/octeontx2/rvu_struct.h |   78 ++
 16 files changed, 3872 insertions(+)
 create mode 100644 drivers/soc/marvell/Kconfig
 create mode 100644 drivers/soc/marvell/Makefile
 create mode 100644 drivers/soc/marvell/octeontx2/Makefile
 create mode 100644 drivers/soc/marvell/octeontx2/cgx.c
 create mode 100644 drivers/soc/marvell/octeontx2/cgx.h
 create mode 100644 drivers/soc/marvell/octeontx2/cgx_fw_if.h
 create mode 100644 drivers/soc/marvell/octeontx2/mbox.c
 create mode 100644 drivers/soc/marvell/octeontx2/mbox.h
 create mode 100644 drivers/soc/marvell/octeontx2/rvu.c
 create mode 100644 drivers/soc/marvell/octeontx2/rvu.h
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_cgx.c
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_reg.h
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_struct.h

-- 
2.7.4

[PATCH v2 06/15] soc: octeontx2: Convert mbox msg id check to a macro

[sunil . kovvuri]

From: Aleksey Makarov 

With 10's of mailbox messages expected to be handled in future,
checking for message id could become a lengthy switch case. Hence
added a macro to auto generate the switch case for each msg id.

Signed-off-by: Aleksey Makarov 
---
 drivers/soc/marvell/octeontx2/rvu.c | 44 -
 1 file changed, 38 insertions(+), 6 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index e795c2f..25f79bf 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -258,6 +258,12 @@ static int rvu_setup_hw_resources(struct rvu *rvu)
return 0;
 }
 
+static int rvu_mbox_handler_READY(struct rvu *rvu, struct msg_req *req,
+ struct ready_msg_rsp *rsp)
+{
+   return 0;
+}
+
 static int rvu_process_mbox_msg(struct rvu *rvu, int devid,
struct mbox_msghdr *req)
 {
@@ -265,13 +271,39 @@ static int rvu_process_mbox_msg(struct rvu *rvu, int 
devid,
if (req->sig != OTX2_MBOX_REQ_SIG)
goto bad_message;
 
-   if (req->id == MBOX_MSG_READY)
-   return 0;
-
+   switch (req->id) {
+#define M(_name, _id, _req_type, _rsp_type)\
+   case _id: { \
+   struct _rsp_type *rsp;  \
+   int err;\
+   \
+   rsp = (struct _rsp_type *)otx2_mbox_alloc_msg(  \
+   >mbox, devid,  \
+   sizeof(struct _rsp_type));  \
+   if (rsp) {  \
+   rsp->hdr.id = _id;  \
+   rsp->hdr.sig = OTX2_MBOX_RSP_SIG;   \
+   rsp->hdr.pcifunc = req->pcifunc;\
+   rsp->hdr.rc = 0;\
+   }   \
+   \
+   err = rvu_mbox_handler_ ## _name(rvu,   \
+(struct _req_type *)req, \
+rsp);  \
+   if (rsp && err) \
+   rsp->hdr.rc = err;  \
+   \
+   return rsp ? err : -ENOMEM; \
+   }
+MBOX_MESSAGES
+#undef M
+   break;
 bad_message:
-   otx2_reply_invalid_msg(>mbox, devid, req->pcifunc,
-  req->id);
-   return -ENODEV;
+   default:
+   otx2_reply_invalid_msg(>mbox, devid, req->pcifunc,
+  req->id);
+   return -ENODEV;
+   }
 }
 
 static void rvu_mbox_handler(struct work_struct *work)
-- 
2.7.4

[PATCH v2 02/15] soc: octeontx2: Reset all RVU blocks

[sunil . kovvuri]

From: Sunil Goutham 

Go through all BLKADDRs and check which ones are implemented
on this silicon and do a HW reset of each implemented block.
Also added all RVU AF and PF register offsets.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/rvu.c|  78 
 drivers/soc/marvell/octeontx2/rvu.h|  37 ++
 drivers/soc/marvell/octeontx2/rvu_reg.h| 113 +
 drivers/soc/marvell/octeontx2/rvu_struct.h |  36 +
 4 files changed, 264 insertions(+)
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_reg.h
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_struct.h

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 5af4da6..d40fabf 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -16,6 +16,7 @@
 #include 
 
 #include "rvu.h"
+#include "rvu_reg.h"
 
 #define DRV_NAME   "octeontx2-af"
 #define DRV_STRING  "Marvell OcteonTX2 RVU Admin Function Driver"
@@ -33,6 +34,70 @@ MODULE_LICENSE("GPL v2");
 MODULE_VERSION(DRV_VERSION);
 MODULE_DEVICE_TABLE(pci, rvu_id_table);
 
+/* Poll a RVU block's register 'offset', for a 'zero'
+ * or 'nonzero' at bits specified by 'mask'
+ */
+int rvu_poll_reg(struct rvu *rvu, u64 block, u64 offset, u64 mask, bool zero)
+{
+   void __iomem *reg;
+   int timeout = 100;
+   u64 reg_val;
+
+   reg = rvu->afreg_base + ((block << 28) | offset);
+   while (timeout) {
+   reg_val = readq(reg);
+   if (zero && !(reg_val & mask))
+   return 0;
+   if (!zero && (reg_val & mask))
+   return 0;
+   udelay(1);
+   cpu_relax();
+   timeout--;
+   }
+   return -EBUSY;
+}
+
+static void rvu_check_block_implemented(struct rvu *rvu)
+{
+   struct rvu_hwinfo *hw = rvu->hw;
+   struct rvu_block *block;
+   int blkid;
+   u64 cfg;
+
+   /* For each block check if 'implemented' bit is set */
+   for (blkid = 0; blkid < BLK_COUNT; blkid++) {
+   block = >block[blkid];
+   cfg = rvupf_read64(rvu, RVU_PF_BLOCK_ADDRX_DISC(blkid));
+   if (cfg & BIT_ULL(11))
+   block->implemented = true;
+   }
+}
+
+static void rvu_block_reset(struct rvu *rvu, int blkaddr, u64 rst_reg)
+{
+   struct rvu_block *block = >hw->block[blkaddr];
+
+   if (!block->implemented)
+   return;
+
+   rvu_write64(rvu, blkaddr, rst_reg, BIT_ULL(0));
+   rvu_poll_reg(rvu, blkaddr, rst_reg, BIT_ULL(63), true);
+}
+
+static void rvu_reset_all_blocks(struct rvu *rvu)
+{
+   /* Do a HW reset of all RVU blocks */
+   rvu_block_reset(rvu, BLKADDR_NPA, NPA_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_NIX0, NIX_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_NPC, NPC_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_SSO, SSO_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_TIM, TIM_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_CPT0, CPT_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_NDC0, NDC_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_NDC1, NDC_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_NDC2, NDC_AF_BLK_RST);
+}
+
 static int rvu_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 {
struct device *dev = >dev;
@@ -43,6 +108,12 @@ static int rvu_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
if (!rvu)
return -ENOMEM;
 
+   rvu->hw = devm_kzalloc(dev, sizeof(struct rvu_hwinfo), GFP_KERNEL);
+   if (!rvu->hw) {
+   devm_kfree(dev, rvu);
+   return -ENOMEM;
+   }
+
pci_set_drvdata(pdev, rvu);
rvu->pdev = pdev;
rvu->dev = >dev;
@@ -80,6 +151,11 @@ static int rvu_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
goto err_release_regions;
}
 
+   /* Check which blocks the HW supports */
+   rvu_check_block_implemented(rvu);
+
+   rvu_reset_all_blocks(rvu);
+
return 0;
 
 err_release_regions:
@@ -88,6 +164,7 @@ static int rvu_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
pci_disable_device(pdev);
 err_freemem:
pci_set_drvdata(pdev, NULL);
+   devm_kfree(>dev, rvu->hw);
devm_kfree(dev, rvu);
return err;
 }
@@ -100,6 +177,7 @@ static void rvu_remove(struct pci_dev *pdev)
pci_disable_device(pdev);
pci_set_drvdata(pdev, NULL);
 
+   devm_kfree(>dev, rvu->hw);
devm_kfree(>dev, rvu);
 }
 
diff --git a/drivers/soc/marvell/octeontx2/rvu.h 
b/drivers/soc/marvell/octeontx2/rvu.h
index 4a4b0ad..e2c54d0 100644
--- a/drivers/soc/marvell/octeontx2/rvu.h
+++ b/drivers/soc/marvell/octeontx2/rvu.h
@@ -11,6 +11,8 @@
 #ifndef RVU_H
 #define RVU_H
 
+#include "rvu_struct.h"
+
 /* PCI device IDs */
 #definePCI_DEVID_OCTEONTX2_RVU_AF  0xA065
 
@@ 

[PATCH v2 04/15] soc: octeontx2: Add mailbox support infra

[sunil . kovvuri]

From: Aleksey Makarov 

This patch adds mailbox support infrastructure APIs.
Each RVU device has a dedicated 64KB mailbox region
shared with it's peer for communication. RVU AF has
a separate mailbox region shared with each of RVU PFs
and a RVU PF has a separate region shared with each of
it's VF.

These set of APIs are used by this driver (RVU AF) and
other RVU PF/VF drivers eg netdev, crypto e.t.c.

Signed-off-by: Aleksey Makarov 
Signed-off-by: Sunil Goutham 
Signed-off-by: Lukasz Bartosik 
---
 drivers/soc/marvell/Kconfig |   4 +
 drivers/soc/marvell/octeontx2/Makefile  |   2 +
 drivers/soc/marvell/octeontx2/mbox.c| 303 
 drivers/soc/marvell/octeontx2/mbox.h| 142 +++
 drivers/soc/marvell/octeontx2/rvu_reg.h |   4 +
 5 files changed, 455 insertions(+)
 create mode 100644 drivers/soc/marvell/octeontx2/mbox.c
 create mode 100644 drivers/soc/marvell/octeontx2/mbox.h

diff --git a/drivers/soc/marvell/Kconfig b/drivers/soc/marvell/Kconfig
index 4499caf..428d22e 100644
--- a/drivers/soc/marvell/Kconfig
+++ b/drivers/soc/marvell/Kconfig
@@ -4,8 +4,12 @@
 
 menu "Marvell SoC drivers"
 
+config OCTEONTX2_MBOX
+   tristate
+
 config OCTEONTX2_AF
tristate "OcteonTX2 RVU Admin Function driver"
+   select OCTEONTX2_MBOX
depends on ARM64 && PCI
help
  This driver supports Marvell's OcteonTX2 Resource Virtualization
diff --git a/drivers/soc/marvell/octeontx2/Makefile 
b/drivers/soc/marvell/octeontx2/Makefile
index dacbd16..ac17cb9 100644
--- a/drivers/soc/marvell/octeontx2/Makefile
+++ b/drivers/soc/marvell/octeontx2/Makefile
@@ -3,6 +3,8 @@
 # Makefile for Marvell's OcteonTX2 RVU Admin Function driver
 #
 
+obj-$(CONFIG_OCTEONTX2_MBOX) += octeontx2_mbox.o
 obj-$(CONFIG_OCTEONTX2_AF) += octeontx2_af.o
 
+octeontx2_mbox-y := mbox.o
 octeontx2_af-y := rvu.o
diff --git a/drivers/soc/marvell/octeontx2/mbox.c 
b/drivers/soc/marvell/octeontx2/mbox.c
new file mode 100644
index 000..0722fa4
--- /dev/null
+++ b/drivers/soc/marvell/octeontx2/mbox.c
@@ -0,0 +1,303 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Marvell OcteonTx2 RVU Admin Function driver
+ *
+ * Copyright (C) 2018 Marvell International Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include 
+#include 
+#include 
+
+#include "rvu_reg.h"
+#include "mbox.h"
+
+static const u16 msgs_offset = ALIGN(sizeof(struct mbox_hdr), MBOX_MSG_ALIGN);
+
+void otx2_mbox_reset(struct otx2_mbox *mbox, int devid)
+{
+   struct otx2_mbox_dev *mdev = >dev[devid];
+   struct mbox_hdr *tx_hdr =
+   (struct mbox_hdr *)(mdev->mbase  + mbox->tx_start);
+   struct mbox_hdr *rx_hdr =
+   (struct mbox_hdr *)(mdev->mbase  + mbox->rx_start);
+
+   spin_lock(>mbox_lock);
+   mdev->msg_size = 0;
+   mdev->rsp_size = 0;
+   tx_hdr->num_msgs = 0;
+   rx_hdr->num_msgs = 0;
+   spin_unlock(>mbox_lock);
+}
+EXPORT_SYMBOL(otx2_mbox_reset);
+
+void otx2_mbox_destroy(struct otx2_mbox *mbox)
+{
+   mbox->reg_base = NULL;
+   mbox->hwbase = NULL;
+
+   kfree(mbox->dev);
+   mbox->dev = NULL;
+}
+EXPORT_SYMBOL(otx2_mbox_destroy);
+
+int otx2_mbox_init(struct otx2_mbox *mbox, void *hwbase, struct pci_dev *pdev,
+  void *reg_base, int direction, int ndevs)
+{
+   int devid;
+   struct otx2_mbox_dev *mdev;
+
+   switch (direction) {
+   case MBOX_DIR_AFPF:
+   case MBOX_DIR_PFVF:
+   mbox->tx_start = MBOX_DOWN_TX_START;
+   mbox->rx_start = MBOX_DOWN_RX_START;
+   mbox->tx_size  = MBOX_DOWN_TX_SIZE;
+   mbox->rx_size  = MBOX_DOWN_RX_SIZE;
+   break;
+   case MBOX_DIR_PFAF:
+   case MBOX_DIR_VFPF:
+   mbox->tx_start = MBOX_DOWN_RX_START;
+   mbox->rx_start = MBOX_DOWN_TX_START;
+   mbox->tx_size  = MBOX_DOWN_RX_SIZE;
+   mbox->rx_size  = MBOX_DOWN_TX_SIZE;
+   break;
+   case MBOX_DIR_AFPF_UP:
+   case MBOX_DIR_PFVF_UP:
+   mbox->tx_start = MBOX_UP_TX_START;
+   mbox->rx_start = MBOX_UP_RX_START;
+   mbox->tx_size  = MBOX_UP_TX_SIZE;
+   mbox->rx_size  = MBOX_UP_RX_SIZE;
+   break;
+   case MBOX_DIR_PFAF_UP:
+   case MBOX_DIR_VFPF_UP:
+   mbox->tx_start = MBOX_UP_RX_START;
+   mbox->rx_start = MBOX_UP_TX_START;
+   mbox->tx_size  = MBOX_UP_RX_SIZE;
+   mbox->rx_size  = MBOX_UP_TX_SIZE;
+   break;
+   default:
+   return -ENODEV;
+   }
+
+   switch (direction) {
+   case MBOX_DIR_AFPF:
+   case MBOX_DIR_AFPF_UP:
+   mbox->trigger = RVU_AF_AFPF_MBOX0;
+   mbox->tr_shift = 4;
+   break;
+   case MBOX_DIR_PFAF:
+   

[PATCH v2 05/15] soc: octeontx2: Add mailbox IRQ and msg handlers

[sunil . kovvuri]

From: Sunil Goutham 

This patch adds support for mailbox interrupt and message
handling. Mapped mailbox region and registered a workqueue
for message handling. Enabled mailbox IRQ of RVU PFs
and registered a interrupt handler. When IRQ is triggered
work is added to the mbox workqueue for msgs to get processed.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/mbox.h   |  14 +-
 drivers/soc/marvell/octeontx2/rvu.c| 254 +
 drivers/soc/marvell/octeontx2/rvu.h|  22 +++
 drivers/soc/marvell/octeontx2/rvu_struct.h |  22 +++
 4 files changed, 309 insertions(+), 3 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/mbox.h 
b/drivers/soc/marvell/octeontx2/mbox.h
index 8e205fd..fc593f0 100644
--- a/drivers/soc/marvell/octeontx2/mbox.h
+++ b/drivers/soc/marvell/octeontx2/mbox.h
@@ -33,6 +33,8 @@
 # error "incorrect mailbox area sizes"
 #endif
 
+#define INTR_MASK(pfvfs) ((pfvfs < 64) ? (BIT_ULL(pfvfs) - 1) : (~0ull))
+
 #define MBOX_RSP_TIMEOUT   1000 /* in ms, Time to wait for mbox response */
 
 #define MBOX_MSG_ALIGN 16  /* Align mbox msg start to 16bytes */
@@ -90,8 +92,9 @@ struct mbox_msghdr {
 
 void otx2_mbox_reset(struct otx2_mbox *mbox, int devid);
 void otx2_mbox_destroy(struct otx2_mbox *mbox);
-int otx2_mbox_init(struct otx2_mbox *mbox, void *hwbase, struct pci_dev *pdev,
-  void *reg_base, int direction, int ndevs);
+int otx2_mbox_init(struct otx2_mbox *mbox, void __force *hwbase,
+  struct pci_dev *pdev, void __force *reg_base,
+  int direction, int ndevs);
 void otx2_mbox_msg_send(struct otx2_mbox *mbox, int devid);
 int otx2_mbox_wait_for_rsp(struct otx2_mbox *mbox, int devid);
 int otx2_mbox_busy_poll_for_rsp(struct otx2_mbox *mbox, int devid);
@@ -115,7 +118,7 @@ static inline struct mbox_msghdr 
*otx2_mbox_alloc_msg(struct otx2_mbox *mbox,
 #define MBOX_MSG_MAX   0x
 
 #define MBOX_MESSAGES  \
-M(READY,   0x001, msg_req, msg_rsp)
+M(READY,   0x001, msg_req, ready_msg_rsp)
 
 enum {
 #define M(_name, _id, _1, _2) MBOX_MSG_ ## _name = _id,
@@ -139,4 +142,9 @@ struct msg_rsp {
struct mbox_msghdr hdr;
 };
 
+struct ready_msg_rsp {
+   struct mbox_msghdr hdr;
+   u16sclk_feq;/* SCLK frequency */
+};
+
 #endif /* MBOX_H */
diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index fa5f40b..e795c2f 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -258,6 +258,245 @@ static int rvu_setup_hw_resources(struct rvu *rvu)
return 0;
 }
 
+static int rvu_process_mbox_msg(struct rvu *rvu, int devid,
+   struct mbox_msghdr *req)
+{
+   /* Check if valid, if not reply with a invalid msg */
+   if (req->sig != OTX2_MBOX_REQ_SIG)
+   goto bad_message;
+
+   if (req->id == MBOX_MSG_READY)
+   return 0;
+
+bad_message:
+   otx2_reply_invalid_msg(>mbox, devid, req->pcifunc,
+  req->id);
+   return -ENODEV;
+}
+
+static void rvu_mbox_handler(struct work_struct *work)
+{
+   struct rvu_work *mwork = container_of(work, struct rvu_work, work);
+   struct rvu *rvu = mwork->rvu;
+   struct otx2_mbox_dev *mdev;
+   struct mbox_hdr *req_hdr;
+   struct mbox_msghdr *msg;
+   struct otx2_mbox *mbox;
+   int offset, id, err;
+   u16 pf;
+
+   mbox = >mbox;
+   pf = mwork - rvu->mbox_wrk;
+   mdev = >dev[pf];
+
+   /* Process received mbox messages */
+   req_hdr = (struct mbox_hdr *)(mdev->mbase + mbox->rx_start);
+   if (req_hdr->num_msgs == 0)
+   return;
+
+   offset = mbox->rx_start + ALIGN(sizeof(*req_hdr), MBOX_MSG_ALIGN);
+
+   for (id = 0; id < req_hdr->num_msgs; id++) {
+   msg = (struct mbox_msghdr *)(mdev->mbase + offset);
+
+   /* Set which PF sent this message based on mbox IRQ */
+   msg->pcifunc &= ~(RVU_PFVF_PF_MASK << RVU_PFVF_PF_SHIFT);
+   msg->pcifunc |= (pf << RVU_PFVF_PF_SHIFT);
+   err = rvu_process_mbox_msg(rvu, pf, msg);
+   if (!err) {
+   offset = mbox->rx_start + msg->next_msgoff;
+   continue;
+   }
+
+   if (msg->pcifunc & RVU_PFVF_FUNC_MASK)
+   dev_warn(rvu->dev, "Error %d when processing message %s 
(0x%x) from PF%d:VF%d\n",
+err, otx2_mbox_id2name(msg->id), msg->id, pf,
+(msg->pcifunc & RVU_PFVF_FUNC_MASK) - 1);
+   else
+   dev_warn(rvu->dev, "Error %d when processing message %s 
(0x%x) from PF%d\n",
+err, otx2_mbox_id2name(msg->id), msg->id, pf);
+   }
+
+   /* Send mbox responses to PF */
+   

[PATCH v2 00/15] soc: octeontx2: Add RVU admin function driver

[sunil . kovvuri]

From: Sunil Goutham 

Resource virtualization unit (RVU) on Marvell's OcteonTX2 SOC supports
multiple PCIe SRIOV physical functions (PFs) and virtual functions (VFs).
PF0 is called administrative / admin function (AF) and has privilege access
to registers to provision different RVU functional blocks to each of
PF/VF.

This admin function (AF) driver acts as a configuration / administrative
software which provisions functional blocks to a PF/VF on demand for them
to work as one of the following
 - A basic network controller (i.e NIC).
 - NIC with packet filtering, shaping and scheduling capabilities.
 - A crypto device.
 - A combination of above etc.

PF/VFs communicate with admin function via a shared memory region.
This patch series adds logic for the following
 - RVU AF driver with functional blocks provisioning support
 - Mailbox infrastructure for communication between AF and PFs.
 - CGX driver which provides information about physcial network
   interfaces which AF processes and forwards required info to
   PF/VF drivers.

This is the first set of patches out of 70 odd patches.

Note: This driver neither receives any data nor processes it i.e no I/O,
  just does the hardware configuration.

Changes from v1:
 1 Merged RVU admin function and CGX drivers into a single module
   - Suggested by Arnd Bergmann
 2 Pulled mbox communication APIs into a separate module to remove
   admin function driver dependency in a VM where AF is not attached.
   - Suggested by Arnd Bergmann

Aleksey Makarov (2):
  soc: octeontx2: Add mailbox support infra
  soc: octeontx2: Convert mbox msg id check to a macro

Geetha sowjanya (1):
  soc: octeontx2: Reconfig MSIX base with IOVA

Linu Cherian (3):
  soc: octeontx2: Set RVU PFs to CGX LMACs mapping
  soc: octeontx2: Add support for CGX link management
  soc: octeontx2: Register for CGX lmac events

Sunil Goutham (9):
  soc: octeontx2: Add Marvell OcteonTX2 RVU AF driver
  soc: octeontx2: Reset all RVU blocks
  soc: octeontx2: Gather RVU blocks HW info
  soc: octeontx2: Add mailbox IRQ and msg handlers
  soc: octeontx2: Scan blocks for LFs provisioned to PF/VF
  soc: octeontx2: Add RVU block LF provisioning support
  soc: octeontx2: Configure block LF's MSIX vector offset
  soc: octeontx2: Add Marvell OcteonTX2 CGX driver
  MAINTAINERS: Add entry for Marvell OcteonTX2 Admin Function driver

 MAINTAINERS|   10 +
 drivers/soc/Kconfig|1 +
 drivers/soc/Makefile   |1 +
 drivers/soc/marvell/Kconfig|   18 +
 drivers/soc/marvell/Makefile   |2 +
 drivers/soc/marvell/octeontx2/Makefile |   10 +
 drivers/soc/marvell/octeontx2/cgx.c|  517 +
 drivers/soc/marvell/octeontx2/cgx.h|   65 ++
 drivers/soc/marvell/octeontx2/cgx_fw_if.h  |  225 
 drivers/soc/marvell/octeontx2/mbox.c   |  303 +
 drivers/soc/marvell/octeontx2/mbox.h   |  211 
 drivers/soc/marvell/octeontx2/rvu.c| 1637 
 drivers/soc/marvell/octeontx2/rvu.h|  158 +++
 drivers/soc/marvell/octeontx2/rvu_cgx.c|  194 
 drivers/soc/marvell/octeontx2/rvu_reg.h|  442 
 drivers/soc/marvell/octeontx2/rvu_struct.h |   78 ++
 16 files changed, 3872 insertions(+)
 create mode 100644 drivers/soc/marvell/Kconfig
 create mode 100644 drivers/soc/marvell/Makefile
 create mode 100644 drivers/soc/marvell/octeontx2/Makefile
 create mode 100644 drivers/soc/marvell/octeontx2/cgx.c
 create mode 100644 drivers/soc/marvell/octeontx2/cgx.h
 create mode 100644 drivers/soc/marvell/octeontx2/cgx_fw_if.h
 create mode 100644 drivers/soc/marvell/octeontx2/mbox.c
 create mode 100644 drivers/soc/marvell/octeontx2/mbox.h
 create mode 100644 drivers/soc/marvell/octeontx2/rvu.c
 create mode 100644 drivers/soc/marvell/octeontx2/rvu.h
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_cgx.c
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_reg.h
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_struct.h

-- 
2.7.4

[PATCH v2 06/15] soc: octeontx2: Convert mbox msg id check to a macro

[sunil . kovvuri]

From: Aleksey Makarov 

With 10's of mailbox messages expected to be handled in future,
checking for message id could become a lengthy switch case. Hence
added a macro to auto generate the switch case for each msg id.

Signed-off-by: Aleksey Makarov 
---
 drivers/soc/marvell/octeontx2/rvu.c | 44 -
 1 file changed, 38 insertions(+), 6 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index e795c2f..25f79bf 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -258,6 +258,12 @@ static int rvu_setup_hw_resources(struct rvu *rvu)
return 0;
 }
 
+static int rvu_mbox_handler_READY(struct rvu *rvu, struct msg_req *req,
+ struct ready_msg_rsp *rsp)
+{
+   return 0;
+}
+
 static int rvu_process_mbox_msg(struct rvu *rvu, int devid,
struct mbox_msghdr *req)
 {
@@ -265,13 +271,39 @@ static int rvu_process_mbox_msg(struct rvu *rvu, int 
devid,
if (req->sig != OTX2_MBOX_REQ_SIG)
goto bad_message;
 
-   if (req->id == MBOX_MSG_READY)
-   return 0;
-
+   switch (req->id) {
+#define M(_name, _id, _req_type, _rsp_type)\
+   case _id: { \
+   struct _rsp_type *rsp;  \
+   int err;\
+   \
+   rsp = (struct _rsp_type *)otx2_mbox_alloc_msg(  \
+   >mbox, devid,  \
+   sizeof(struct _rsp_type));  \
+   if (rsp) {  \
+   rsp->hdr.id = _id;  \
+   rsp->hdr.sig = OTX2_MBOX_RSP_SIG;   \
+   rsp->hdr.pcifunc = req->pcifunc;\
+   rsp->hdr.rc = 0;\
+   }   \
+   \
+   err = rvu_mbox_handler_ ## _name(rvu,   \
+(struct _req_type *)req, \
+rsp);  \
+   if (rsp && err) \
+   rsp->hdr.rc = err;  \
+   \
+   return rsp ? err : -ENOMEM; \
+   }
+MBOX_MESSAGES
+#undef M
+   break;
 bad_message:
-   otx2_reply_invalid_msg(>mbox, devid, req->pcifunc,
-  req->id);
-   return -ENODEV;
+   default:
+   otx2_reply_invalid_msg(>mbox, devid, req->pcifunc,
+  req->id);
+   return -ENODEV;
+   }
 }
 
 static void rvu_mbox_handler(struct work_struct *work)
-- 
2.7.4

[PATCH v2 02/15] soc: octeontx2: Reset all RVU blocks

[sunil . kovvuri]

From: Sunil Goutham 

Go through all BLKADDRs and check which ones are implemented
on this silicon and do a HW reset of each implemented block.
Also added all RVU AF and PF register offsets.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/rvu.c|  78 
 drivers/soc/marvell/octeontx2/rvu.h|  37 ++
 drivers/soc/marvell/octeontx2/rvu_reg.h| 113 +
 drivers/soc/marvell/octeontx2/rvu_struct.h |  36 +
 4 files changed, 264 insertions(+)
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_reg.h
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_struct.h

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 5af4da6..d40fabf 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -16,6 +16,7 @@
 #include 
 
 #include "rvu.h"
+#include "rvu_reg.h"
 
 #define DRV_NAME   "octeontx2-af"
 #define DRV_STRING  "Marvell OcteonTX2 RVU Admin Function Driver"
@@ -33,6 +34,70 @@ MODULE_LICENSE("GPL v2");
 MODULE_VERSION(DRV_VERSION);
 MODULE_DEVICE_TABLE(pci, rvu_id_table);
 
+/* Poll a RVU block's register 'offset', for a 'zero'
+ * or 'nonzero' at bits specified by 'mask'
+ */
+int rvu_poll_reg(struct rvu *rvu, u64 block, u64 offset, u64 mask, bool zero)
+{
+   void __iomem *reg;
+   int timeout = 100;
+   u64 reg_val;
+
+   reg = rvu->afreg_base + ((block << 28) | offset);
+   while (timeout) {
+   reg_val = readq(reg);
+   if (zero && !(reg_val & mask))
+   return 0;
+   if (!zero && (reg_val & mask))
+   return 0;
+   udelay(1);
+   cpu_relax();
+   timeout--;
+   }
+   return -EBUSY;
+}
+
+static void rvu_check_block_implemented(struct rvu *rvu)
+{
+   struct rvu_hwinfo *hw = rvu->hw;
+   struct rvu_block *block;
+   int blkid;
+   u64 cfg;
+
+   /* For each block check if 'implemented' bit is set */
+   for (blkid = 0; blkid < BLK_COUNT; blkid++) {
+   block = >block[blkid];
+   cfg = rvupf_read64(rvu, RVU_PF_BLOCK_ADDRX_DISC(blkid));
+   if (cfg & BIT_ULL(11))
+   block->implemented = true;
+   }
+}
+
+static void rvu_block_reset(struct rvu *rvu, int blkaddr, u64 rst_reg)
+{
+   struct rvu_block *block = >hw->block[blkaddr];
+
+   if (!block->implemented)
+   return;
+
+   rvu_write64(rvu, blkaddr, rst_reg, BIT_ULL(0));
+   rvu_poll_reg(rvu, blkaddr, rst_reg, BIT_ULL(63), true);
+}
+
+static void rvu_reset_all_blocks(struct rvu *rvu)
+{
+   /* Do a HW reset of all RVU blocks */
+   rvu_block_reset(rvu, BLKADDR_NPA, NPA_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_NIX0, NIX_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_NPC, NPC_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_SSO, SSO_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_TIM, TIM_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_CPT0, CPT_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_NDC0, NDC_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_NDC1, NDC_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_NDC2, NDC_AF_BLK_RST);
+}
+
 static int rvu_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 {
struct device *dev = >dev;
@@ -43,6 +108,12 @@ static int rvu_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
if (!rvu)
return -ENOMEM;
 
+   rvu->hw = devm_kzalloc(dev, sizeof(struct rvu_hwinfo), GFP_KERNEL);
+   if (!rvu->hw) {
+   devm_kfree(dev, rvu);
+   return -ENOMEM;
+   }
+
pci_set_drvdata(pdev, rvu);
rvu->pdev = pdev;
rvu->dev = >dev;
@@ -80,6 +151,11 @@ static int rvu_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
goto err_release_regions;
}
 
+   /* Check which blocks the HW supports */
+   rvu_check_block_implemented(rvu);
+
+   rvu_reset_all_blocks(rvu);
+
return 0;
 
 err_release_regions:
@@ -88,6 +164,7 @@ static int rvu_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
pci_disable_device(pdev);
 err_freemem:
pci_set_drvdata(pdev, NULL);
+   devm_kfree(>dev, rvu->hw);
devm_kfree(dev, rvu);
return err;
 }
@@ -100,6 +177,7 @@ static void rvu_remove(struct pci_dev *pdev)
pci_disable_device(pdev);
pci_set_drvdata(pdev, NULL);
 
+   devm_kfree(>dev, rvu->hw);
devm_kfree(>dev, rvu);
 }
 
diff --git a/drivers/soc/marvell/octeontx2/rvu.h 
b/drivers/soc/marvell/octeontx2/rvu.h
index 4a4b0ad..e2c54d0 100644
--- a/drivers/soc/marvell/octeontx2/rvu.h
+++ b/drivers/soc/marvell/octeontx2/rvu.h
@@ -11,6 +11,8 @@
 #ifndef RVU_H
 #define RVU_H
 
+#include "rvu_struct.h"
+
 /* PCI device IDs */
 #definePCI_DEVID_OCTEONTX2_RVU_AF  0xA065
 
@@ 

[PATCH v2 04/15] soc: octeontx2: Add mailbox support infra

[sunil . kovvuri]

From: Aleksey Makarov 

This patch adds mailbox support infrastructure APIs.
Each RVU device has a dedicated 64KB mailbox region
shared with it's peer for communication. RVU AF has
a separate mailbox region shared with each of RVU PFs
and a RVU PF has a separate region shared with each of
it's VF.

These set of APIs are used by this driver (RVU AF) and
other RVU PF/VF drivers eg netdev, crypto e.t.c.

Signed-off-by: Aleksey Makarov 
Signed-off-by: Sunil Goutham 
Signed-off-by: Lukasz Bartosik 
---
 drivers/soc/marvell/Kconfig |   4 +
 drivers/soc/marvell/octeontx2/Makefile  |   2 +
 drivers/soc/marvell/octeontx2/mbox.c| 303 
 drivers/soc/marvell/octeontx2/mbox.h| 142 +++
 drivers/soc/marvell/octeontx2/rvu_reg.h |   4 +
 5 files changed, 455 insertions(+)
 create mode 100644 drivers/soc/marvell/octeontx2/mbox.c
 create mode 100644 drivers/soc/marvell/octeontx2/mbox.h

diff --git a/drivers/soc/marvell/Kconfig b/drivers/soc/marvell/Kconfig
index 4499caf..428d22e 100644
--- a/drivers/soc/marvell/Kconfig
+++ b/drivers/soc/marvell/Kconfig
@@ -4,8 +4,12 @@
 
 menu "Marvell SoC drivers"
 
+config OCTEONTX2_MBOX
+   tristate
+
 config OCTEONTX2_AF
tristate "OcteonTX2 RVU Admin Function driver"
+   select OCTEONTX2_MBOX
depends on ARM64 && PCI
help
  This driver supports Marvell's OcteonTX2 Resource Virtualization
diff --git a/drivers/soc/marvell/octeontx2/Makefile 
b/drivers/soc/marvell/octeontx2/Makefile
index dacbd16..ac17cb9 100644
--- a/drivers/soc/marvell/octeontx2/Makefile
+++ b/drivers/soc/marvell/octeontx2/Makefile
@@ -3,6 +3,8 @@
 # Makefile for Marvell's OcteonTX2 RVU Admin Function driver
 #
 
+obj-$(CONFIG_OCTEONTX2_MBOX) += octeontx2_mbox.o
 obj-$(CONFIG_OCTEONTX2_AF) += octeontx2_af.o
 
+octeontx2_mbox-y := mbox.o
 octeontx2_af-y := rvu.o
diff --git a/drivers/soc/marvell/octeontx2/mbox.c 
b/drivers/soc/marvell/octeontx2/mbox.c
new file mode 100644
index 000..0722fa4
--- /dev/null
+++ b/drivers/soc/marvell/octeontx2/mbox.c
@@ -0,0 +1,303 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Marvell OcteonTx2 RVU Admin Function driver
+ *
+ * Copyright (C) 2018 Marvell International Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include 
+#include 
+#include 
+
+#include "rvu_reg.h"
+#include "mbox.h"
+
+static const u16 msgs_offset = ALIGN(sizeof(struct mbox_hdr), MBOX_MSG_ALIGN);
+
+void otx2_mbox_reset(struct otx2_mbox *mbox, int devid)
+{
+   struct otx2_mbox_dev *mdev = >dev[devid];
+   struct mbox_hdr *tx_hdr =
+   (struct mbox_hdr *)(mdev->mbase  + mbox->tx_start);
+   struct mbox_hdr *rx_hdr =
+   (struct mbox_hdr *)(mdev->mbase  + mbox->rx_start);
+
+   spin_lock(>mbox_lock);
+   mdev->msg_size = 0;
+   mdev->rsp_size = 0;
+   tx_hdr->num_msgs = 0;
+   rx_hdr->num_msgs = 0;
+   spin_unlock(>mbox_lock);
+}
+EXPORT_SYMBOL(otx2_mbox_reset);
+
+void otx2_mbox_destroy(struct otx2_mbox *mbox)
+{
+   mbox->reg_base = NULL;
+   mbox->hwbase = NULL;
+
+   kfree(mbox->dev);
+   mbox->dev = NULL;
+}
+EXPORT_SYMBOL(otx2_mbox_destroy);
+
+int otx2_mbox_init(struct otx2_mbox *mbox, void *hwbase, struct pci_dev *pdev,
+  void *reg_base, int direction, int ndevs)
+{
+   int devid;
+   struct otx2_mbox_dev *mdev;
+
+   switch (direction) {
+   case MBOX_DIR_AFPF:
+   case MBOX_DIR_PFVF:
+   mbox->tx_start = MBOX_DOWN_TX_START;
+   mbox->rx_start = MBOX_DOWN_RX_START;
+   mbox->tx_size  = MBOX_DOWN_TX_SIZE;
+   mbox->rx_size  = MBOX_DOWN_RX_SIZE;
+   break;
+   case MBOX_DIR_PFAF:
+   case MBOX_DIR_VFPF:
+   mbox->tx_start = MBOX_DOWN_RX_START;
+   mbox->rx_start = MBOX_DOWN_TX_START;
+   mbox->tx_size  = MBOX_DOWN_RX_SIZE;
+   mbox->rx_size  = MBOX_DOWN_TX_SIZE;
+   break;
+   case MBOX_DIR_AFPF_UP:
+   case MBOX_DIR_PFVF_UP:
+   mbox->tx_start = MBOX_UP_TX_START;
+   mbox->rx_start = MBOX_UP_RX_START;
+   mbox->tx_size  = MBOX_UP_TX_SIZE;
+   mbox->rx_size  = MBOX_UP_RX_SIZE;
+   break;
+   case MBOX_DIR_PFAF_UP:
+   case MBOX_DIR_VFPF_UP:
+   mbox->tx_start = MBOX_UP_RX_START;
+   mbox->rx_start = MBOX_UP_TX_START;
+   mbox->tx_size  = MBOX_UP_RX_SIZE;
+   mbox->rx_size  = MBOX_UP_TX_SIZE;
+   break;
+   default:
+   return -ENODEV;
+   }
+
+   switch (direction) {
+   case MBOX_DIR_AFPF:
+   case MBOX_DIR_AFPF_UP:
+   mbox->trigger = RVU_AF_AFPF_MBOX0;
+   mbox->tr_shift = 4;
+   break;
+   case MBOX_DIR_PFAF:
+   

[PATCH v2 05/15] soc: octeontx2: Add mailbox IRQ and msg handlers

[sunil . kovvuri]

From: Sunil Goutham 

This patch adds support for mailbox interrupt and message
handling. Mapped mailbox region and registered a workqueue
for message handling. Enabled mailbox IRQ of RVU PFs
and registered a interrupt handler. When IRQ is triggered
work is added to the mbox workqueue for msgs to get processed.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/mbox.h   |  14 +-
 drivers/soc/marvell/octeontx2/rvu.c| 254 +
 drivers/soc/marvell/octeontx2/rvu.h|  22 +++
 drivers/soc/marvell/octeontx2/rvu_struct.h |  22 +++
 4 files changed, 309 insertions(+), 3 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/mbox.h 
b/drivers/soc/marvell/octeontx2/mbox.h
index 8e205fd..fc593f0 100644
--- a/drivers/soc/marvell/octeontx2/mbox.h
+++ b/drivers/soc/marvell/octeontx2/mbox.h
@@ -33,6 +33,8 @@
 # error "incorrect mailbox area sizes"
 #endif
 
+#define INTR_MASK(pfvfs) ((pfvfs < 64) ? (BIT_ULL(pfvfs) - 1) : (~0ull))
+
 #define MBOX_RSP_TIMEOUT   1000 /* in ms, Time to wait for mbox response */
 
 #define MBOX_MSG_ALIGN 16  /* Align mbox msg start to 16bytes */
@@ -90,8 +92,9 @@ struct mbox_msghdr {
 
 void otx2_mbox_reset(struct otx2_mbox *mbox, int devid);
 void otx2_mbox_destroy(struct otx2_mbox *mbox);
-int otx2_mbox_init(struct otx2_mbox *mbox, void *hwbase, struct pci_dev *pdev,
-  void *reg_base, int direction, int ndevs);
+int otx2_mbox_init(struct otx2_mbox *mbox, void __force *hwbase,
+  struct pci_dev *pdev, void __force *reg_base,
+  int direction, int ndevs);
 void otx2_mbox_msg_send(struct otx2_mbox *mbox, int devid);
 int otx2_mbox_wait_for_rsp(struct otx2_mbox *mbox, int devid);
 int otx2_mbox_busy_poll_for_rsp(struct otx2_mbox *mbox, int devid);
@@ -115,7 +118,7 @@ static inline struct mbox_msghdr 
*otx2_mbox_alloc_msg(struct otx2_mbox *mbox,
 #define MBOX_MSG_MAX   0x
 
 #define MBOX_MESSAGES  \
-M(READY,   0x001, msg_req, msg_rsp)
+M(READY,   0x001, msg_req, ready_msg_rsp)
 
 enum {
 #define M(_name, _id, _1, _2) MBOX_MSG_ ## _name = _id,
@@ -139,4 +142,9 @@ struct msg_rsp {
struct mbox_msghdr hdr;
 };
 
+struct ready_msg_rsp {
+   struct mbox_msghdr hdr;
+   u16sclk_feq;/* SCLK frequency */
+};
+
 #endif /* MBOX_H */
diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index fa5f40b..e795c2f 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -258,6 +258,245 @@ static int rvu_setup_hw_resources(struct rvu *rvu)
return 0;
 }
 
+static int rvu_process_mbox_msg(struct rvu *rvu, int devid,
+   struct mbox_msghdr *req)
+{
+   /* Check if valid, if not reply with a invalid msg */
+   if (req->sig != OTX2_MBOX_REQ_SIG)
+   goto bad_message;
+
+   if (req->id == MBOX_MSG_READY)
+   return 0;
+
+bad_message:
+   otx2_reply_invalid_msg(>mbox, devid, req->pcifunc,
+  req->id);
+   return -ENODEV;
+}
+
+static void rvu_mbox_handler(struct work_struct *work)
+{
+   struct rvu_work *mwork = container_of(work, struct rvu_work, work);
+   struct rvu *rvu = mwork->rvu;
+   struct otx2_mbox_dev *mdev;
+   struct mbox_hdr *req_hdr;
+   struct mbox_msghdr *msg;
+   struct otx2_mbox *mbox;
+   int offset, id, err;
+   u16 pf;
+
+   mbox = >mbox;
+   pf = mwork - rvu->mbox_wrk;
+   mdev = >dev[pf];
+
+   /* Process received mbox messages */
+   req_hdr = (struct mbox_hdr *)(mdev->mbase + mbox->rx_start);
+   if (req_hdr->num_msgs == 0)
+   return;
+
+   offset = mbox->rx_start + ALIGN(sizeof(*req_hdr), MBOX_MSG_ALIGN);
+
+   for (id = 0; id < req_hdr->num_msgs; id++) {
+   msg = (struct mbox_msghdr *)(mdev->mbase + offset);
+
+   /* Set which PF sent this message based on mbox IRQ */
+   msg->pcifunc &= ~(RVU_PFVF_PF_MASK << RVU_PFVF_PF_SHIFT);
+   msg->pcifunc |= (pf << RVU_PFVF_PF_SHIFT);
+   err = rvu_process_mbox_msg(rvu, pf, msg);
+   if (!err) {
+   offset = mbox->rx_start + msg->next_msgoff;
+   continue;
+   }
+
+   if (msg->pcifunc & RVU_PFVF_FUNC_MASK)
+   dev_warn(rvu->dev, "Error %d when processing message %s 
(0x%x) from PF%d:VF%d\n",
+err, otx2_mbox_id2name(msg->id), msg->id, pf,
+(msg->pcifunc & RVU_PFVF_FUNC_MASK) - 1);
+   else
+   dev_warn(rvu->dev, "Error %d when processing message %s 
(0x%x) from PF%d\n",
+err, otx2_mbox_id2name(msg->id), msg->id, pf);
+   }
+
+   /* Send mbox responses to PF */
+   

[PATCH v2 07/15] soc: octeontx2: Scan blocks for LFs provisioned to PF/VF

[sunil . kovvuri]

From: Sunil Goutham 

Scan all RVU blocks to find any 'LF to RVU PF/VF' mapping done by
low level firmware. If found any, mark them as used in respective
block's LF bitmap and also save mapped PF/VF's PF_FUNC info.

This is done to avoid reattaching a block LF to a different RVU PF/VF.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/rvu.c| 148 -
 drivers/soc/marvell/octeontx2/rvu.h|  16 
 drivers/soc/marvell/octeontx2/rvu_struct.h |  18 
 3 files changed, 180 insertions(+), 2 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 25f79bf..9539ab9 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -22,6 +22,8 @@
 #define DRV_STRING  "Marvell OcteonTX2 RVU Admin Function Driver"
 #define DRV_VERSION"1.0"
 
+static int rvu_get_hwvf(struct rvu *rvu, int pcifunc);
+
 /* Supported devices */
 static const struct pci_device_id rvu_id_table[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_RVU_AF) },
@@ -66,6 +68,91 @@ int rvu_alloc_bitmap(struct rsrc_bmap *rsrc)
return 0;
 }
 
+static void rvu_update_rsrc_map(struct rvu *rvu, struct rvu_pfvf *pfvf,
+   struct rvu_block *block, u16 pcifunc,
+   u16 lf, bool attach)
+{
+   int devnum, num_lfs = 0;
+   bool is_pf;
+   u64 reg;
+
+   if (lf >= block->lf.max) {
+   dev_err(>pdev->dev,
+   "%s: FATAL: LF %d is >= %s's max lfs i.e %d\n",
+   __func__, lf, block->name, block->lf.max);
+   return;
+   }
+
+   /* Check if this is for a RVU PF or VF */
+   if (pcifunc & RVU_PFVF_FUNC_MASK) {
+   is_pf = false;
+   devnum = rvu_get_hwvf(rvu, pcifunc);
+   } else {
+   is_pf = true;
+   devnum = rvu_get_pf(pcifunc);
+   }
+
+   block->fn_map[lf] = attach ? pcifunc : 0;
+
+   switch (block->type) {
+   case BLKTYPE_NPA:
+   pfvf->npalf = attach ? true : false;
+   num_lfs = pfvf->npalf;
+   break;
+   case BLKTYPE_NIX:
+   pfvf->nixlf = attach ? true : false;
+   num_lfs = pfvf->nixlf;
+   break;
+   case BLKTYPE_SSO:
+   attach ? pfvf->sso++ : pfvf->sso--;
+   num_lfs = pfvf->sso;
+   break;
+   case BLKTYPE_SSOW:
+   attach ? pfvf->ssow++ : pfvf->ssow--;
+   num_lfs = pfvf->ssow;
+   break;
+   case BLKTYPE_TIM:
+   attach ? pfvf->timlfs++ : pfvf->timlfs--;
+   num_lfs = pfvf->timlfs;
+   break;
+   case BLKTYPE_CPT:
+   attach ? pfvf->cptlfs++ : pfvf->cptlfs--;
+   num_lfs = pfvf->cptlfs;
+   break;
+   }
+
+   reg = is_pf ? block->pf_lfcnt_reg : block->vf_lfcnt_reg;
+   rvu_write64(rvu, BLKADDR_RVUM, reg | (devnum << 16), num_lfs);
+}
+
+inline int rvu_get_pf(u16 pcifunc)
+{
+   return (pcifunc >> RVU_PFVF_PF_SHIFT) & RVU_PFVF_PF_MASK;
+}
+
+static int rvu_get_hwvf(struct rvu *rvu, int pcifunc)
+{
+   int pf, func;
+   u64 cfg;
+
+   pf = rvu_get_pf(pcifunc);
+   func = pcifunc & RVU_PFVF_FUNC_MASK;
+
+   /* Get first HWVF attached to this PF */
+   cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_PFX_CFG(pf));
+
+   return ((cfg & 0xFFF) + func - 1);
+}
+
+struct rvu_pfvf *rvu_get_pfvf(struct rvu *rvu, int pcifunc)
+{
+   /* Check if it is a PF or VF */
+   if (pcifunc & RVU_PFVF_FUNC_MASK)
+   return >hwvf[rvu_get_hwvf(rvu, pcifunc)];
+   else
+   return >pf[rvu_get_pf(pcifunc)];
+}
+
 static void rvu_check_block_implemented(struct rvu *rvu)
 {
struct rvu_hwinfo *hw = rvu->hw;
@@ -107,6 +194,28 @@ static void rvu_reset_all_blocks(struct rvu *rvu)
rvu_block_reset(rvu, BLKADDR_NDC2, NDC_AF_BLK_RST);
 }
 
+static void rvu_scan_block(struct rvu *rvu, struct rvu_block *block)
+{
+   struct rvu_pfvf *pfvf;
+   u64 cfg;
+   int lf;
+
+   for (lf = 0; lf < block->lf.max; lf++) {
+   cfg = rvu_read64(rvu, block->addr,
+block->lfcfg_reg | (lf << block->lfshift));
+   if (!(cfg & BIT_ULL(63)))
+   continue;
+
+   /* Set this resource as being used */
+   __set_bit(lf, block->lf.bmap);
+
+   /* Get, to whom this LF is attached */
+   pfvf = rvu_get_pfvf(rvu, (cfg >> 8) & 0x);
+   rvu_update_rsrc_map(rvu, pfvf, block,
+   (cfg >> 8) & 0x, lf, true);
+   }
+}
+
 static void rvu_free_hw_resources(struct rvu *rvu)
 {
struct rvu_hwinfo *hw = rvu->hw;
@@ -124,7 +233,7 @@ static int rvu_setup_hw_resources(struct rvu *rvu)
 {
struct rvu_hwinfo *hw = 

[PATCH v2 01/15] soc: octeontx2: Add Marvell OcteonTX2 RVU AF driver

[sunil . kovvuri]

From: Sunil Goutham 

This patch adds basic template for Marvell OcteonTX2's
resource virtualization unit (RVU) admin function (AF)
driver. Just the driver registration and probe.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/Kconfig|   1 +
 drivers/soc/Makefile   |   1 +
 drivers/soc/marvell/Kconfig|  13 
 drivers/soc/marvell/Makefile   |   2 +
 drivers/soc/marvell/octeontx2/Makefile |   8 +++
 drivers/soc/marvell/octeontx2/rvu.c| 126 +
 drivers/soc/marvell/octeontx2/rvu.h|  31 
 7 files changed, 182 insertions(+)
 create mode 100644 drivers/soc/marvell/Kconfig
 create mode 100644 drivers/soc/marvell/Makefile
 create mode 100644 drivers/soc/marvell/octeontx2/Makefile
 create mode 100644 drivers/soc/marvell/octeontx2/rvu.c
 create mode 100644 drivers/soc/marvell/octeontx2/rvu.h

diff --git a/drivers/soc/Kconfig b/drivers/soc/Kconfig
index c07b4a8..42f2d0b 100644
--- a/drivers/soc/Kconfig
+++ b/drivers/soc/Kconfig
@@ -6,6 +6,7 @@ source "drivers/soc/atmel/Kconfig"
 source "drivers/soc/bcm/Kconfig"
 source "drivers/soc/fsl/Kconfig"
 source "drivers/soc/imx/Kconfig"
+source "drivers/soc/marvell/Kconfig"
 source "drivers/soc/mediatek/Kconfig"
 source "drivers/soc/qcom/Kconfig"
 source "drivers/soc/renesas/Kconfig"
diff --git a/drivers/soc/Makefile b/drivers/soc/Makefile
index 113e884..5e18cbb 100644
--- a/drivers/soc/Makefile
+++ b/drivers/soc/Makefile
@@ -12,6 +12,7 @@ obj-y += fsl/
 obj-$(CONFIG_ARCH_GEMINI)  += gemini/
 obj-$(CONFIG_ARCH_MXC) += imx/
 obj-$(CONFIG_SOC_XWAY) += lantiq/
+obj-y  += marvell/
 obj-y  += mediatek/
 obj-$(CONFIG_ARCH_MESON)   += amlogic/
 obj-y  += qcom/
diff --git a/drivers/soc/marvell/Kconfig b/drivers/soc/marvell/Kconfig
new file mode 100644
index 000..4499caf
--- /dev/null
+++ b/drivers/soc/marvell/Kconfig
@@ -0,0 +1,13 @@
+#
+# MARVELL SoC drivers
+#
+
+menu "Marvell SoC drivers"
+
+config OCTEONTX2_AF
+   tristate "OcteonTX2 RVU Admin Function driver"
+   depends on ARM64 && PCI
+   help
+ This driver supports Marvell's OcteonTX2 Resource Virtualization
+ Unit's admin function manager which manages all RVU HW resources.
+endmenu
diff --git a/drivers/soc/marvell/Makefile b/drivers/soc/marvell/Makefile
new file mode 100644
index 000..16e0ca0
--- /dev/null
+++ b/drivers/soc/marvell/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0
+obj-y  += octeontx2/
diff --git a/drivers/soc/marvell/octeontx2/Makefile 
b/drivers/soc/marvell/octeontx2/Makefile
new file mode 100644
index 000..dacbd16
--- /dev/null
+++ b/drivers/soc/marvell/octeontx2/Makefile
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for Marvell's OcteonTX2 RVU Admin Function driver
+#
+
+obj-$(CONFIG_OCTEONTX2_AF) += octeontx2_af.o
+
+octeontx2_af-y := rvu.o
diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
new file mode 100644
index 000..5af4da6
--- /dev/null
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -0,0 +1,126 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Marvell OcteonTx2 RVU Admin Function driver
+ *
+ * Copyright (C) 2018 Marvell International Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "rvu.h"
+
+#define DRV_NAME   "octeontx2-af"
+#define DRV_STRING  "Marvell OcteonTX2 RVU Admin Function Driver"
+#define DRV_VERSION"1.0"
+
+/* Supported devices */
+static const struct pci_device_id rvu_id_table[] = {
+   { PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_RVU_AF) },
+   { 0, }  /* end of table */
+};
+
+MODULE_AUTHOR("Marvell International Ltd.");
+MODULE_DESCRIPTION(DRV_STRING);
+MODULE_LICENSE("GPL v2");
+MODULE_VERSION(DRV_VERSION);
+MODULE_DEVICE_TABLE(pci, rvu_id_table);
+
+static int rvu_probe(struct pci_dev *pdev, const struct pci_device_id *id)
+{
+   struct device *dev = >dev;
+   struct rvu *rvu;
+   interr;
+
+   rvu = devm_kzalloc(dev, sizeof(*rvu), GFP_KERNEL);
+   if (!rvu)
+   return -ENOMEM;
+
+   pci_set_drvdata(pdev, rvu);
+   rvu->pdev = pdev;
+   rvu->dev = >dev;
+
+   err = pci_enable_device(pdev);
+   if (err) {
+   dev_err(dev, "Failed to enable PCI device\n");
+   goto err_freemem;
+   }
+
+   err = pci_request_regions(pdev, DRV_NAME);
+   if (err) {
+   dev_err(dev, "PCI request regions failed 0x%x\n", err);
+   goto err_disable_device;
+   }
+
+   err = pci_set_dma_mask(pdev, DMA_BIT_MASK(48));
+   if (err) {
+   dev_err(dev, "Unable to set DMA 

[PATCH v2 03/15] soc: octeontx2: Gather RVU blocks HW info

[sunil . kovvuri]

From: Sunil Goutham 

This patch gathers NPA/NIX/SSO/SSOW/TIM/CPT RVU blocks's
HW info like number of LFs. Important register offsets
saved for later use to avoid code duplication for each block.
A bitmap is allocated for each of the blocks which later
on will be used to allocate a LF for a RVU PF/VF.

Also added RVU NIX/NPA block registers and few registers
of other blocks.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/rvu.c | 167 
 drivers/soc/marvell/octeontx2/rvu.h |  21 ++
 drivers/soc/marvell/octeontx2/rvu_reg.h | 335 +++-
 3 files changed, 518 insertions(+), 5 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index d40fabf..fa5f40b 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -57,6 +57,15 @@ int rvu_poll_reg(struct rvu *rvu, u64 block, u64 offset, u64 
mask, bool zero)
return -EBUSY;
 }
 
+int rvu_alloc_bitmap(struct rsrc_bmap *rsrc)
+{
+   rsrc->bmap = kcalloc(BITS_TO_LONGS(rsrc->max),
+sizeof(long), GFP_KERNEL);
+   if (!rsrc->bmap)
+   return -ENOMEM;
+   return 0;
+}
+
 static void rvu_check_block_implemented(struct rvu *rvu)
 {
struct rvu_hwinfo *hw = rvu->hw;
@@ -98,6 +107,157 @@ static void rvu_reset_all_blocks(struct rvu *rvu)
rvu_block_reset(rvu, BLKADDR_NDC2, NDC_AF_BLK_RST);
 }
 
+static void rvu_free_hw_resources(struct rvu *rvu)
+{
+   struct rvu_hwinfo *hw = rvu->hw;
+   struct rvu_block *block;
+   int id;
+
+   /* Free all bitmaps */
+   for (id = 0; id < BLK_COUNT; id++) {
+   block = >block[id];
+   kfree(block->lf.bmap);
+   }
+}
+
+static int rvu_setup_hw_resources(struct rvu *rvu)
+{
+   struct rvu_hwinfo *hw = rvu->hw;
+   struct rvu_block *block;
+   int err;
+   u64 cfg;
+
+   /* Get HW supported max RVU PF & VF count */
+   cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_CONST);
+   hw->total_pfs = (cfg >> 32) & 0xFF;
+   hw->total_vfs = (cfg >> 20) & 0xFFF;
+   hw->max_vfs_per_pf = (cfg >> 40) & 0xFF;
+
+   /* Init NPA LF's bitmap */
+   block = >block[BLKADDR_NPA];
+   if (!block->implemented)
+   goto nix;
+   cfg = rvu_read64(rvu, BLKADDR_NPA, NPA_AF_CONST);
+   block->lf.max = (cfg >> 16) & 0xFFF;
+   block->addr = BLKADDR_NPA;
+   block->lfshift = 8;
+   block->lookup_reg = NPA_AF_RVU_LF_CFG_DEBUG;
+   block->pf_lfcnt_reg = RVU_PRIV_PFX_NPA_CFG;
+   block->vf_lfcnt_reg = RVU_PRIV_HWVFX_NPA_CFG;
+   block->lfcfg_reg = NPA_PRIV_LFX_CFG;
+   block->msixcfg_reg = NPA_PRIV_LFX_INT_CFG;
+   block->lfreset_reg = NPA_AF_LF_RST;
+   sprintf(block->name, "NPA");
+   err = rvu_alloc_bitmap(>lf);
+   if (err)
+   return err;
+
+nix:
+   /* Init NIX LF's bitmap */
+   block = >block[BLKADDR_NIX0];
+   if (!block->implemented)
+   goto sso;
+   cfg = rvu_read64(rvu, BLKADDR_NIX0, NIX_AF_CONST2);
+   block->lf.max = cfg & 0xFFF;
+   block->addr = BLKADDR_NIX0;
+   block->lfshift = 8;
+   block->lookup_reg = NIX_AF_RVU_LF_CFG_DEBUG;
+   block->pf_lfcnt_reg = RVU_PRIV_PFX_NIX_CFG;
+   block->vf_lfcnt_reg = RVU_PRIV_HWVFX_NIX_CFG;
+   block->lfcfg_reg = NIX_PRIV_LFX_CFG;
+   block->msixcfg_reg = NIX_PRIV_LFX_INT_CFG;
+   block->lfreset_reg = NIX_AF_LF_RST;
+   sprintf(block->name, "NIX");
+   err = rvu_alloc_bitmap(>lf);
+   if (err)
+   return err;
+
+sso:
+   /* Init SSO group's bitmap */
+   block = >block[BLKADDR_SSO];
+   if (!block->implemented)
+   goto ssow;
+   cfg = rvu_read64(rvu, BLKADDR_SSO, SSO_AF_CONST);
+   block->lf.max = cfg & 0x;
+   block->addr = BLKADDR_SSO;
+   block->multislot = true;
+   block->lfshift = 3;
+   block->lookup_reg = SSO_AF_RVU_LF_CFG_DEBUG;
+   block->pf_lfcnt_reg = RVU_PRIV_PFX_SSO_CFG;
+   block->vf_lfcnt_reg = RVU_PRIV_HWVFX_SSO_CFG;
+   block->lfcfg_reg = SSO_PRIV_LFX_HWGRP_CFG;
+   block->msixcfg_reg = SSO_PRIV_LFX_HWGRP_INT_CFG;
+   block->lfreset_reg = SSO_AF_LF_HWGRP_RST;
+   sprintf(block->name, "SSO GROUP");
+   err = rvu_alloc_bitmap(>lf);
+   if (err)
+   return err;
+
+ssow:
+   /* Init SSO workslot's bitmap */
+   block = >block[BLKADDR_SSOW];
+   if (!block->implemented)
+   goto tim;
+   block->lf.max = (cfg >> 56) & 0xFF;
+   block->addr = BLKADDR_SSOW;
+   block->multislot = true;
+   block->lfshift = 3;
+   block->lookup_reg = SSOW_AF_RVU_LF_HWS_CFG_DEBUG;
+   block->pf_lfcnt_reg = RVU_PRIV_PFX_SSOW_CFG;
+   block->vf_lfcnt_reg = RVU_PRIV_HWVFX_SSOW_CFG;
+   block->lfcfg_reg = SSOW_PRIV_LFX_HWS_CFG;
+   block->msixcfg_reg = SSOW_PRIV_LFX_HWS_INT_CFG;
+   block->lfreset_reg = 

[PATCH v2 07/15] soc: octeontx2: Scan blocks for LFs provisioned to PF/VF

[sunil . kovvuri]

From: Sunil Goutham 

Scan all RVU blocks to find any 'LF to RVU PF/VF' mapping done by
low level firmware. If found any, mark them as used in respective
block's LF bitmap and also save mapped PF/VF's PF_FUNC info.

This is done to avoid reattaching a block LF to a different RVU PF/VF.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/rvu.c| 148 -
 drivers/soc/marvell/octeontx2/rvu.h|  16 
 drivers/soc/marvell/octeontx2/rvu_struct.h |  18 
 3 files changed, 180 insertions(+), 2 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 25f79bf..9539ab9 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -22,6 +22,8 @@
 #define DRV_STRING  "Marvell OcteonTX2 RVU Admin Function Driver"
 #define DRV_VERSION"1.0"
 
+static int rvu_get_hwvf(struct rvu *rvu, int pcifunc);
+
 /* Supported devices */
 static const struct pci_device_id rvu_id_table[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_RVU_AF) },
@@ -66,6 +68,91 @@ int rvu_alloc_bitmap(struct rsrc_bmap *rsrc)
return 0;
 }
 
+static void rvu_update_rsrc_map(struct rvu *rvu, struct rvu_pfvf *pfvf,
+   struct rvu_block *block, u16 pcifunc,
+   u16 lf, bool attach)
+{
+   int devnum, num_lfs = 0;
+   bool is_pf;
+   u64 reg;
+
+   if (lf >= block->lf.max) {
+   dev_err(>pdev->dev,
+   "%s: FATAL: LF %d is >= %s's max lfs i.e %d\n",
+   __func__, lf, block->name, block->lf.max);
+   return;
+   }
+
+   /* Check if this is for a RVU PF or VF */
+   if (pcifunc & RVU_PFVF_FUNC_MASK) {
+   is_pf = false;
+   devnum = rvu_get_hwvf(rvu, pcifunc);
+   } else {
+   is_pf = true;
+   devnum = rvu_get_pf(pcifunc);
+   }
+
+   block->fn_map[lf] = attach ? pcifunc : 0;
+
+   switch (block->type) {
+   case BLKTYPE_NPA:
+   pfvf->npalf = attach ? true : false;
+   num_lfs = pfvf->npalf;
+   break;
+   case BLKTYPE_NIX:
+   pfvf->nixlf = attach ? true : false;
+   num_lfs = pfvf->nixlf;
+   break;
+   case BLKTYPE_SSO:
+   attach ? pfvf->sso++ : pfvf->sso--;
+   num_lfs = pfvf->sso;
+   break;
+   case BLKTYPE_SSOW:
+   attach ? pfvf->ssow++ : pfvf->ssow--;
+   num_lfs = pfvf->ssow;
+   break;
+   case BLKTYPE_TIM:
+   attach ? pfvf->timlfs++ : pfvf->timlfs--;
+   num_lfs = pfvf->timlfs;
+   break;
+   case BLKTYPE_CPT:
+   attach ? pfvf->cptlfs++ : pfvf->cptlfs--;
+   num_lfs = pfvf->cptlfs;
+   break;
+   }
+
+   reg = is_pf ? block->pf_lfcnt_reg : block->vf_lfcnt_reg;
+   rvu_write64(rvu, BLKADDR_RVUM, reg | (devnum << 16), num_lfs);
+}
+
+inline int rvu_get_pf(u16 pcifunc)
+{
+   return (pcifunc >> RVU_PFVF_PF_SHIFT) & RVU_PFVF_PF_MASK;
+}
+
+static int rvu_get_hwvf(struct rvu *rvu, int pcifunc)
+{
+   int pf, func;
+   u64 cfg;
+
+   pf = rvu_get_pf(pcifunc);
+   func = pcifunc & RVU_PFVF_FUNC_MASK;
+
+   /* Get first HWVF attached to this PF */
+   cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_PFX_CFG(pf));
+
+   return ((cfg & 0xFFF) + func - 1);
+}
+
+struct rvu_pfvf *rvu_get_pfvf(struct rvu *rvu, int pcifunc)
+{
+   /* Check if it is a PF or VF */
+   if (pcifunc & RVU_PFVF_FUNC_MASK)
+   return >hwvf[rvu_get_hwvf(rvu, pcifunc)];
+   else
+   return >pf[rvu_get_pf(pcifunc)];
+}
+
 static void rvu_check_block_implemented(struct rvu *rvu)
 {
struct rvu_hwinfo *hw = rvu->hw;
@@ -107,6 +194,28 @@ static void rvu_reset_all_blocks(struct rvu *rvu)
rvu_block_reset(rvu, BLKADDR_NDC2, NDC_AF_BLK_RST);
 }
 
+static void rvu_scan_block(struct rvu *rvu, struct rvu_block *block)
+{
+   struct rvu_pfvf *pfvf;
+   u64 cfg;
+   int lf;
+
+   for (lf = 0; lf < block->lf.max; lf++) {
+   cfg = rvu_read64(rvu, block->addr,
+block->lfcfg_reg | (lf << block->lfshift));
+   if (!(cfg & BIT_ULL(63)))
+   continue;
+
+   /* Set this resource as being used */
+   __set_bit(lf, block->lf.bmap);
+
+   /* Get, to whom this LF is attached */
+   pfvf = rvu_get_pfvf(rvu, (cfg >> 8) & 0x);
+   rvu_update_rsrc_map(rvu, pfvf, block,
+   (cfg >> 8) & 0x, lf, true);
+   }
+}
+
 static void rvu_free_hw_resources(struct rvu *rvu)
 {
struct rvu_hwinfo *hw = rvu->hw;
@@ -124,7 +233,7 @@ static int rvu_setup_hw_resources(struct rvu *rvu)
 {
struct rvu_hwinfo *hw = 

[PATCH v2 01/15] soc: octeontx2: Add Marvell OcteonTX2 RVU AF driver

[sunil . kovvuri]

From: Sunil Goutham 

This patch adds basic template for Marvell OcteonTX2's
resource virtualization unit (RVU) admin function (AF)
driver. Just the driver registration and probe.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/Kconfig|   1 +
 drivers/soc/Makefile   |   1 +
 drivers/soc/marvell/Kconfig|  13 
 drivers/soc/marvell/Makefile   |   2 +
 drivers/soc/marvell/octeontx2/Makefile |   8 +++
 drivers/soc/marvell/octeontx2/rvu.c| 126 +
 drivers/soc/marvell/octeontx2/rvu.h|  31 
 7 files changed, 182 insertions(+)
 create mode 100644 drivers/soc/marvell/Kconfig
 create mode 100644 drivers/soc/marvell/Makefile
 create mode 100644 drivers/soc/marvell/octeontx2/Makefile
 create mode 100644 drivers/soc/marvell/octeontx2/rvu.c
 create mode 100644 drivers/soc/marvell/octeontx2/rvu.h

diff --git a/drivers/soc/Kconfig b/drivers/soc/Kconfig
index c07b4a8..42f2d0b 100644
--- a/drivers/soc/Kconfig
+++ b/drivers/soc/Kconfig
@@ -6,6 +6,7 @@ source "drivers/soc/atmel/Kconfig"
 source "drivers/soc/bcm/Kconfig"
 source "drivers/soc/fsl/Kconfig"
 source "drivers/soc/imx/Kconfig"
+source "drivers/soc/marvell/Kconfig"
 source "drivers/soc/mediatek/Kconfig"
 source "drivers/soc/qcom/Kconfig"
 source "drivers/soc/renesas/Kconfig"
diff --git a/drivers/soc/Makefile b/drivers/soc/Makefile
index 113e884..5e18cbb 100644
--- a/drivers/soc/Makefile
+++ b/drivers/soc/Makefile
@@ -12,6 +12,7 @@ obj-y += fsl/
 obj-$(CONFIG_ARCH_GEMINI)  += gemini/
 obj-$(CONFIG_ARCH_MXC) += imx/
 obj-$(CONFIG_SOC_XWAY) += lantiq/
+obj-y  += marvell/
 obj-y  += mediatek/
 obj-$(CONFIG_ARCH_MESON)   += amlogic/
 obj-y  += qcom/
diff --git a/drivers/soc/marvell/Kconfig b/drivers/soc/marvell/Kconfig
new file mode 100644
index 000..4499caf
--- /dev/null
+++ b/drivers/soc/marvell/Kconfig
@@ -0,0 +1,13 @@
+#
+# MARVELL SoC drivers
+#
+
+menu "Marvell SoC drivers"
+
+config OCTEONTX2_AF
+   tristate "OcteonTX2 RVU Admin Function driver"
+   depends on ARM64 && PCI
+   help
+ This driver supports Marvell's OcteonTX2 Resource Virtualization
+ Unit's admin function manager which manages all RVU HW resources.
+endmenu
diff --git a/drivers/soc/marvell/Makefile b/drivers/soc/marvell/Makefile
new file mode 100644
index 000..16e0ca0
--- /dev/null
+++ b/drivers/soc/marvell/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0
+obj-y  += octeontx2/
diff --git a/drivers/soc/marvell/octeontx2/Makefile 
b/drivers/soc/marvell/octeontx2/Makefile
new file mode 100644
index 000..dacbd16
--- /dev/null
+++ b/drivers/soc/marvell/octeontx2/Makefile
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for Marvell's OcteonTX2 RVU Admin Function driver
+#
+
+obj-$(CONFIG_OCTEONTX2_AF) += octeontx2_af.o
+
+octeontx2_af-y := rvu.o
diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
new file mode 100644
index 000..5af4da6
--- /dev/null
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -0,0 +1,126 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Marvell OcteonTx2 RVU Admin Function driver
+ *
+ * Copyright (C) 2018 Marvell International Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "rvu.h"
+
+#define DRV_NAME   "octeontx2-af"
+#define DRV_STRING  "Marvell OcteonTX2 RVU Admin Function Driver"
+#define DRV_VERSION"1.0"
+
+/* Supported devices */
+static const struct pci_device_id rvu_id_table[] = {
+   { PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_RVU_AF) },
+   { 0, }  /* end of table */
+};
+
+MODULE_AUTHOR("Marvell International Ltd.");
+MODULE_DESCRIPTION(DRV_STRING);
+MODULE_LICENSE("GPL v2");
+MODULE_VERSION(DRV_VERSION);
+MODULE_DEVICE_TABLE(pci, rvu_id_table);
+
+static int rvu_probe(struct pci_dev *pdev, const struct pci_device_id *id)
+{
+   struct device *dev = >dev;
+   struct rvu *rvu;
+   interr;
+
+   rvu = devm_kzalloc(dev, sizeof(*rvu), GFP_KERNEL);
+   if (!rvu)
+   return -ENOMEM;
+
+   pci_set_drvdata(pdev, rvu);
+   rvu->pdev = pdev;
+   rvu->dev = >dev;
+
+   err = pci_enable_device(pdev);
+   if (err) {
+   dev_err(dev, "Failed to enable PCI device\n");
+   goto err_freemem;
+   }
+
+   err = pci_request_regions(pdev, DRV_NAME);
+   if (err) {
+   dev_err(dev, "PCI request regions failed 0x%x\n", err);
+   goto err_disable_device;
+   }
+
+   err = pci_set_dma_mask(pdev, DMA_BIT_MASK(48));
+   if (err) {
+   dev_err(dev, "Unable to set DMA 

[PATCH v2 03/15] soc: octeontx2: Gather RVU blocks HW info

[sunil . kovvuri]

From: Sunil Goutham 

This patch gathers NPA/NIX/SSO/SSOW/TIM/CPT RVU blocks's
HW info like number of LFs. Important register offsets
saved for later use to avoid code duplication for each block.
A bitmap is allocated for each of the blocks which later
on will be used to allocate a LF for a RVU PF/VF.

Also added RVU NIX/NPA block registers and few registers
of other blocks.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/rvu.c | 167 
 drivers/soc/marvell/octeontx2/rvu.h |  21 ++
 drivers/soc/marvell/octeontx2/rvu_reg.h | 335 +++-
 3 files changed, 518 insertions(+), 5 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index d40fabf..fa5f40b 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -57,6 +57,15 @@ int rvu_poll_reg(struct rvu *rvu, u64 block, u64 offset, u64 
mask, bool zero)
return -EBUSY;
 }
 
+int rvu_alloc_bitmap(struct rsrc_bmap *rsrc)
+{
+   rsrc->bmap = kcalloc(BITS_TO_LONGS(rsrc->max),
+sizeof(long), GFP_KERNEL);
+   if (!rsrc->bmap)
+   return -ENOMEM;
+   return 0;
+}
+
 static void rvu_check_block_implemented(struct rvu *rvu)
 {
struct rvu_hwinfo *hw = rvu->hw;
@@ -98,6 +107,157 @@ static void rvu_reset_all_blocks(struct rvu *rvu)
rvu_block_reset(rvu, BLKADDR_NDC2, NDC_AF_BLK_RST);
 }
 
+static void rvu_free_hw_resources(struct rvu *rvu)
+{
+   struct rvu_hwinfo *hw = rvu->hw;
+   struct rvu_block *block;
+   int id;
+
+   /* Free all bitmaps */
+   for (id = 0; id < BLK_COUNT; id++) {
+   block = >block[id];
+   kfree(block->lf.bmap);
+   }
+}
+
+static int rvu_setup_hw_resources(struct rvu *rvu)
+{
+   struct rvu_hwinfo *hw = rvu->hw;
+   struct rvu_block *block;
+   int err;
+   u64 cfg;
+
+   /* Get HW supported max RVU PF & VF count */
+   cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_CONST);
+   hw->total_pfs = (cfg >> 32) & 0xFF;
+   hw->total_vfs = (cfg >> 20) & 0xFFF;
+   hw->max_vfs_per_pf = (cfg >> 40) & 0xFF;
+
+   /* Init NPA LF's bitmap */
+   block = >block[BLKADDR_NPA];
+   if (!block->implemented)
+   goto nix;
+   cfg = rvu_read64(rvu, BLKADDR_NPA, NPA_AF_CONST);
+   block->lf.max = (cfg >> 16) & 0xFFF;
+   block->addr = BLKADDR_NPA;
+   block->lfshift = 8;
+   block->lookup_reg = NPA_AF_RVU_LF_CFG_DEBUG;
+   block->pf_lfcnt_reg = RVU_PRIV_PFX_NPA_CFG;
+   block->vf_lfcnt_reg = RVU_PRIV_HWVFX_NPA_CFG;
+   block->lfcfg_reg = NPA_PRIV_LFX_CFG;
+   block->msixcfg_reg = NPA_PRIV_LFX_INT_CFG;
+   block->lfreset_reg = NPA_AF_LF_RST;
+   sprintf(block->name, "NPA");
+   err = rvu_alloc_bitmap(>lf);
+   if (err)
+   return err;
+
+nix:
+   /* Init NIX LF's bitmap */
+   block = >block[BLKADDR_NIX0];
+   if (!block->implemented)
+   goto sso;
+   cfg = rvu_read64(rvu, BLKADDR_NIX0, NIX_AF_CONST2);
+   block->lf.max = cfg & 0xFFF;
+   block->addr = BLKADDR_NIX0;
+   block->lfshift = 8;
+   block->lookup_reg = NIX_AF_RVU_LF_CFG_DEBUG;
+   block->pf_lfcnt_reg = RVU_PRIV_PFX_NIX_CFG;
+   block->vf_lfcnt_reg = RVU_PRIV_HWVFX_NIX_CFG;
+   block->lfcfg_reg = NIX_PRIV_LFX_CFG;
+   block->msixcfg_reg = NIX_PRIV_LFX_INT_CFG;
+   block->lfreset_reg = NIX_AF_LF_RST;
+   sprintf(block->name, "NIX");
+   err = rvu_alloc_bitmap(>lf);
+   if (err)
+   return err;
+
+sso:
+   /* Init SSO group's bitmap */
+   block = >block[BLKADDR_SSO];
+   if (!block->implemented)
+   goto ssow;
+   cfg = rvu_read64(rvu, BLKADDR_SSO, SSO_AF_CONST);
+   block->lf.max = cfg & 0x;
+   block->addr = BLKADDR_SSO;
+   block->multislot = true;
+   block->lfshift = 3;
+   block->lookup_reg = SSO_AF_RVU_LF_CFG_DEBUG;
+   block->pf_lfcnt_reg = RVU_PRIV_PFX_SSO_CFG;
+   block->vf_lfcnt_reg = RVU_PRIV_HWVFX_SSO_CFG;
+   block->lfcfg_reg = SSO_PRIV_LFX_HWGRP_CFG;
+   block->msixcfg_reg = SSO_PRIV_LFX_HWGRP_INT_CFG;
+   block->lfreset_reg = SSO_AF_LF_HWGRP_RST;
+   sprintf(block->name, "SSO GROUP");
+   err = rvu_alloc_bitmap(>lf);
+   if (err)
+   return err;
+
+ssow:
+   /* Init SSO workslot's bitmap */
+   block = >block[BLKADDR_SSOW];
+   if (!block->implemented)
+   goto tim;
+   block->lf.max = (cfg >> 56) & 0xFF;
+   block->addr = BLKADDR_SSOW;
+   block->multislot = true;
+   block->lfshift = 3;
+   block->lookup_reg = SSOW_AF_RVU_LF_HWS_CFG_DEBUG;
+   block->pf_lfcnt_reg = RVU_PRIV_PFX_SSOW_CFG;
+   block->vf_lfcnt_reg = RVU_PRIV_HWVFX_SSOW_CFG;
+   block->lfcfg_reg = SSOW_PRIV_LFX_HWS_CFG;
+   block->msixcfg_reg = SSOW_PRIV_LFX_HWS_INT_CFG;
+   block->lfreset_reg = 

Re: [PATCH 11/15] soc: octeontx2: Add Marvell OcteonTX2 CGX driver

[Sunil Kovvuri]

On Fri, Aug 31, 2018 at 11:59 PM Arnd Bergmann  wrote:
>
> On Fri, Aug 31, 2018 at 6:01 PM Sunil Kovvuri  wrote:
> > On Fri, Aug 31, 2018 at 7:50 PM Arnd Bergmann  wrote:
>
> >
> > Thanks for the suggestion, that does makes sense.
> > Actually i did thought about it, but i was skeptical if it would be
> > acceptable to make
> > a single module out of drivers registering for two different PCI devices.
>
> I don't think it matters much whether there is one module or two
> (others might have a strong opinion one way or the other).
>
> What is important though are these two points:
>
> - How to represent the two PCI devices to user space: You should only
>   have one interface to user space I think, and this should be similar
>   to how other drivers manage similar cases (I don't actually know what they
>   do, but I assume you've done some research here)
>
> - How you connect find the pair of devices: Generally speaking while
>   the SoC might only have one of each, you shouldn't make that assumption
>   in the code, but instead have a reliable way of having one driver wait
>   for the other driver to finish probing so you can match the pair.
>

Agreed.
I do have a patch in works for that, i.e to defer AF driver probe till
CGX driver is loaded.
https://github.com/sunilkovvuri/rvu_drivers/commit/27b449087c16224a9e93dd91d6d9f734b5210bed

> > Will wait for few more days for more feedback from anyone and port v2 
> > series.
>
> Ok.
>
>   Arnd

Re: [PATCH 11/15] soc: octeontx2: Add Marvell OcteonTX2 CGX driver

[Sunil Kovvuri]

On Fri, Aug 31, 2018 at 11:59 PM Arnd Bergmann  wrote:
>
> On Fri, Aug 31, 2018 at 6:01 PM Sunil Kovvuri  wrote:
> > On Fri, Aug 31, 2018 at 7:50 PM Arnd Bergmann  wrote:
>
> >
> > Thanks for the suggestion, that does makes sense.
> > Actually i did thought about it, but i was skeptical if it would be
> > acceptable to make
> > a single module out of drivers registering for two different PCI devices.
>
> I don't think it matters much whether there is one module or two
> (others might have a strong opinion one way or the other).
>
> What is important though are these two points:
>
> - How to represent the two PCI devices to user space: You should only
>   have one interface to user space I think, and this should be similar
>   to how other drivers manage similar cases (I don't actually know what they
>   do, but I assume you've done some research here)
>
> - How you connect find the pair of devices: Generally speaking while
>   the SoC might only have one of each, you shouldn't make that assumption
>   in the code, but instead have a reliable way of having one driver wait
>   for the other driver to finish probing so you can match the pair.
>

Agreed.
I do have a patch in works for that, i.e to defer AF driver probe till
CGX driver is loaded.
https://github.com/sunilkovvuri/rvu_drivers/commit/27b449087c16224a9e93dd91d6d9f734b5210bed

> > Will wait for few more days for more feedback from anyone and port v2 
> > series.
>
> Ok.
>
>   Arnd

Re: [PATCH 11/15] soc: octeontx2: Add Marvell OcteonTX2 CGX driver

[Sunil Kovvuri]

On Fri, Aug 31, 2018 at 7:50 PM Arnd Bergmann  wrote:
>
> On Thu, Aug 30, 2018 at 7:55 PM Sunil Kovvuri  wrote:
> > On Thu, Aug 30, 2018 at 7:37 PM Arnd Bergmann  wrote:
> > > On Tue, Aug 28, 2018 at 3:10 PM Sunil Kovvuri  
> > > wrote:
> > > Ok, I think I understand the PF/VF distinction now. One (to me)
> > > surprising aspect here is that you not just have one physical function
> > > that you can use to assign resources to multiple virtual functions,
> > > but also a second level of virtualization that is used to assign
> > > resources to "physical functions" that are less physical than the
> > > name suggests.
> >
> > Yes, PF is just for name sake, on-boot there is no difference between
> > PFs/VFs as such.
> > PF0 has privilege access to assign resources to all PFs and their VFs.
> > This admin function driver loads for PF0.
>
> ok
>
> > > The part that I have not grasped yet is what the split between
> > > the CGX and the AF is for, how they relate to one another, and
> > > what the software abstraction for the two is going to be.
> >
> > In HW, CGX is a separate PCI device which handles the serdes and
> > physical ethernet interface.
> > Ethernet driver in drivers/net/ethernet can only communicate to
> > admin function driver since they share a mailbox memory.
> > So we had to bind both CGX and admin function drivers to almost work as one,
> > inorder to provide relavent info to ethernet drivers. That's why we
> > have many functions
> > from CGX driver which AF uses.
> >
> > eg: Firmware gets to know about a physical interface status change,
> > which CGX driver gets
> > to know and it uses AF's mailbox communication to inform ethernet
> > driver about the event.
>
> Would it make sense then to combine the CGX driver and the AF
> driver into a single module? It sounds like you can never really
> use one without the other anyway, and that would make it easier
> to have a sensible abstraction to user space.
>
>   Arnd

Thanks for the suggestion, that does makes sense.
Actually i did thought about it, but i was skeptical if it would be
acceptable to make
a single module out of drivers registering for two different PCI devices.

Will wait for few more days for more feedback from anyone and port v2 series.

Sunil.

Re: [PATCH 11/15] soc: octeontx2: Add Marvell OcteonTX2 CGX driver

[Sunil Kovvuri]

On Fri, Aug 31, 2018 at 7:50 PM Arnd Bergmann  wrote:
>
> On Thu, Aug 30, 2018 at 7:55 PM Sunil Kovvuri  wrote:
> > On Thu, Aug 30, 2018 at 7:37 PM Arnd Bergmann  wrote:
> > > On Tue, Aug 28, 2018 at 3:10 PM Sunil Kovvuri  
> > > wrote:
> > > Ok, I think I understand the PF/VF distinction now. One (to me)
> > > surprising aspect here is that you not just have one physical function
> > > that you can use to assign resources to multiple virtual functions,
> > > but also a second level of virtualization that is used to assign
> > > resources to "physical functions" that are less physical than the
> > > name suggests.
> >
> > Yes, PF is just for name sake, on-boot there is no difference between
> > PFs/VFs as such.
> > PF0 has privilege access to assign resources to all PFs and their VFs.
> > This admin function driver loads for PF0.
>
> ok
>
> > > The part that I have not grasped yet is what the split between
> > > the CGX and the AF is for, how they relate to one another, and
> > > what the software abstraction for the two is going to be.
> >
> > In HW, CGX is a separate PCI device which handles the serdes and
> > physical ethernet interface.
> > Ethernet driver in drivers/net/ethernet can only communicate to
> > admin function driver since they share a mailbox memory.
> > So we had to bind both CGX and admin function drivers to almost work as one,
> > inorder to provide relavent info to ethernet drivers. That's why we
> > have many functions
> > from CGX driver which AF uses.
> >
> > eg: Firmware gets to know about a physical interface status change,
> > which CGX driver gets
> > to know and it uses AF's mailbox communication to inform ethernet
> > driver about the event.
>
> Would it make sense then to combine the CGX driver and the AF
> driver into a single module? It sounds like you can never really
> use one without the other anyway, and that would make it easier
> to have a sensible abstraction to user space.
>
>   Arnd

Thanks for the suggestion, that does makes sense.
Actually i did thought about it, but i was skeptical if it would be
acceptable to make
a single module out of drivers registering for two different PCI devices.

Will wait for few more days for more feedback from anyone and port v2 series.

Sunil.

Re: [PATCH 10/15] soc: octeontx2: Reconfig MSIX base with IOVA

[Sunil Kovvuri]

On Thu, Aug 30, 2018 at 7:23 PM Arnd Bergmann  wrote:
>
> On Tue, Aug 28, 2018 at 3:17 PM Sunil Kovvuri  wrote:
> > On Tue, Aug 28, 2018 at 6:27 PM Arnd Bergmann  wrote:
> > > On Tue, Aug 28, 2018 at 2:42 PM Sunil Kovvuri  
> > > wrote:
> > > > On Tue, Aug 28, 2018 at 5:39 PM Arnd Bergmann  wrote:
> > > > > On Tue, Aug 28, 2018 at 12:58 PM  wrote:
> > > >
> > > > This admin function is a PCI device which is capable of provisioning
> > > > HW blocks to other PCIe SRIOV devices in the system. Each HW block
> > > > (eg memory buffer pools, NIC dewscriptors, crypto engines e.t.c) needs
> > > > certain no of MSIX vectors. Admin function has a set of 32K MSIX vectors
> > > > in memory (not on-chip) which based on HW block provisioning to a PCI 
> > > > device
> > > > attaches the required number of vectors to that device. Some part of 
> > > > this
> > > > configuration is done by low level firmware.
> > > >
> > > > RVU_AF_MSIXTR_BASE points to the memory region allocated for 32K MSIX
> > > > vectors. If kernel is booted with IOMMU enabled and admin function 
> > > > device
> > > > is attached to SMMU, HW will go through translation to access this MSIX
> > > > vector memory region. Hence the mapping done in this patch.
> > >
> > > Do you mean this is not a regular PCIe MSI-X interrupt to the GIC, but
> > > something internal to your device that gets routed through the IOMMU
> > > back into the device?
> > >
> >
> > This is a regular PCIe MSI-X interrupt, the difference is that the
> > bunch of PCI devices
> > here doesn't have a fixed set of MSIX vectors. Admin function has a
> > memory region with
> > 32K MSIX vectors which it provisions to PCI devices based on the HW
> > functional blocks
> > attached to them. A PCI device which works as a ethernet device needs
> > X number of vectors
> > and a crypto device needs Y number of vectors.
> >
> > Since the admin function owns the whole MSIX vector region, HW uses
> > this device's stream ID
> > to access the vectors. Hence the IOMMU mapping. Once MSIX vectors are
> > provisioned to
> > a PCI device they work as normal MSIX interrupt like any other device.
>
> Ok, I think I got it now, just to confirm: the MSIX vectors you allocate
> in the admin device refer to memory backing the BAR that contains
> the MSI-X entries of the other functions, right?

Yes, that's correct.

>
> I was a bit confused here and assumed that you were mapping
> the MMIO area of an interrupt controller that receives the interupt
> transactions.
>
>  Arnd

Re: [PATCH 10/15] soc: octeontx2: Reconfig MSIX base with IOVA

[Sunil Kovvuri]

On Thu, Aug 30, 2018 at 7:23 PM Arnd Bergmann  wrote:
>
> On Tue, Aug 28, 2018 at 3:17 PM Sunil Kovvuri  wrote:
> > On Tue, Aug 28, 2018 at 6:27 PM Arnd Bergmann  wrote:
> > > On Tue, Aug 28, 2018 at 2:42 PM Sunil Kovvuri  
> > > wrote:
> > > > On Tue, Aug 28, 2018 at 5:39 PM Arnd Bergmann  wrote:
> > > > > On Tue, Aug 28, 2018 at 12:58 PM  wrote:
> > > >
> > > > This admin function is a PCI device which is capable of provisioning
> > > > HW blocks to other PCIe SRIOV devices in the system. Each HW block
> > > > (eg memory buffer pools, NIC dewscriptors, crypto engines e.t.c) needs
> > > > certain no of MSIX vectors. Admin function has a set of 32K MSIX vectors
> > > > in memory (not on-chip) which based on HW block provisioning to a PCI 
> > > > device
> > > > attaches the required number of vectors to that device. Some part of 
> > > > this
> > > > configuration is done by low level firmware.
> > > >
> > > > RVU_AF_MSIXTR_BASE points to the memory region allocated for 32K MSIX
> > > > vectors. If kernel is booted with IOMMU enabled and admin function 
> > > > device
> > > > is attached to SMMU, HW will go through translation to access this MSIX
> > > > vector memory region. Hence the mapping done in this patch.
> > >
> > > Do you mean this is not a regular PCIe MSI-X interrupt to the GIC, but
> > > something internal to your device that gets routed through the IOMMU
> > > back into the device?
> > >
> >
> > This is a regular PCIe MSI-X interrupt, the difference is that the
> > bunch of PCI devices
> > here doesn't have a fixed set of MSIX vectors. Admin function has a
> > memory region with
> > 32K MSIX vectors which it provisions to PCI devices based on the HW
> > functional blocks
> > attached to them. A PCI device which works as a ethernet device needs
> > X number of vectors
> > and a crypto device needs Y number of vectors.
> >
> > Since the admin function owns the whole MSIX vector region, HW uses
> > this device's stream ID
> > to access the vectors. Hence the IOMMU mapping. Once MSIX vectors are
> > provisioned to
> > a PCI device they work as normal MSIX interrupt like any other device.
>
> Ok, I think I got it now, just to confirm: the MSIX vectors you allocate
> in the admin device refer to memory backing the BAR that contains
> the MSI-X entries of the other functions, right?

Yes, that's correct.

>
> I was a bit confused here and assumed that you were mapping
> the MMIO area of an interrupt controller that receives the interupt
> transactions.
>
>  Arnd

Re: [PATCH 11/15] soc: octeontx2: Add Marvell OcteonTX2 CGX driver

[Sunil Kovvuri]

On Thu, Aug 30, 2018 at 7:37 PM Arnd Bergmann  wrote:
>
> On Tue, Aug 28, 2018 at 3:10 PM Sunil Kovvuri  wrote:
> >
> > > > > If this is a regular PCI ethernet driver, why do you put it into 
> > > > > driver/soc
> > > > > rather than drivers/net/ethernet/ ?
> > > >
> > > > No, this is not a ethernet driver, as mentioned in the cover letter
> > > > this driver and AF driver doesn't
> > > > handle any IO. There will be a separate ethernet driver (will submit
> > > > that as well in future) which will
> > > > communicate with these drivers for configuring hardware.
> > > >
> > > > The driver in question here is for a serdes controller which handles
> > > > physical ethernet interfaces.
> > > > Admin function driver gathers info w.r.t current state of physical
> > > > ethernet interfaces from this driver
> > > > and notifies actual ethernet driver about changes, if any.
> > >
> > > Ok. Can you describe the structure that the PCI devices appear
> > > in? It might help to be make the connection between the differnet
> > > patches to understand how things fit together. In the final
> > > picture, how many different pci_driver instances do you have,
> > > and what part are they for?
> >
> > List of PCI devices are CGX, RVU PF0-PFn SRIOV physical functions
> > and RVU VF0-VFn SRIOV virtual functions. No of VFs per PF is configurable
> > and done by low level firmware.
> >
> > List of PCI driver instances would be CGX driver, RVU PF0 (i.e admin
> > function) driver,
> > PF1-PFn either netdev driver or crypto driver, VF0-VFn functionality would 
> > be
> > same as their PF.
> >
> > The current plan is to have CGX driver, Admin function driver, PF
> > netdev driver, VF netdev driver and PF/VF crypto drivers.
>
> Ok, I think I understand the PF/VF distinction now. One (to me)
> surprising aspect here is that you not just have one physical function
> that you can use to assign resources to multiple virtual functions,
> but also a second level of virtualization that is used to assign
> resources to "physical functions" that are less physical than the
> name suggests.

Yes, PF is just for name sake, on-boot there is no difference between
PFs/VFs as such.
PF0 has privilege access to assign resources to all PFs and their VFs.
This admin function driver loads for PF0.

>
> The part that I have not grasped yet is what the split between
> the CGX and the AF is for, how they relate to one another, and
> what the software abstraction for the two is going to be.

In HW, CGX is a separate PCI device which handles the serdes and
physical ethernet interface.
Ethernet driver in drivers/net/ethernet can only communicate to
admin function driver since they share a mailbox memory.
So we had to bind both CGX and admin function drivers to almost work as one,
inorder to provide relavent info to ethernet drivers. That's why we
have many functions
from CGX driver which AF uses.

eg: Firmware gets to know about a physical interface status change,
which CGX driver gets
to know and it uses AF's mailbox communication to inform ethernet
driver about the event.

>
> > > Is the idea that an ethernet device driver always attaches to a
> > > virtual function that gets created by the main driver, and that
> > > the two drivers share no interfaces on the kernel side, or do
> > > you have multiple drivers linking to each other?
> >
> > Ethernet device driver can attach to both physical function and virtual 
> > function
> > whose HW resources are provisioned by admin function driver.
> >
> > Yes the PF/VF ethernet drivers and these drivers won't share any
> > kernel interfaces.
> > Physical ethernet interface is owned by ethernet driver only, this driver 
> > just
> > configures which ethernet driver instance uses which physcial interface.
>
> Ok.
>
>  Arnd

Re: [PATCH 11/15] soc: octeontx2: Add Marvell OcteonTX2 CGX driver

[Sunil Kovvuri]

On Thu, Aug 30, 2018 at 7:37 PM Arnd Bergmann  wrote:
>
> On Tue, Aug 28, 2018 at 3:10 PM Sunil Kovvuri  wrote:
> >
> > > > > If this is a regular PCI ethernet driver, why do you put it into 
> > > > > driver/soc
> > > > > rather than drivers/net/ethernet/ ?
> > > >
> > > > No, this is not a ethernet driver, as mentioned in the cover letter
> > > > this driver and AF driver doesn't
> > > > handle any IO. There will be a separate ethernet driver (will submit
> > > > that as well in future) which will
> > > > communicate with these drivers for configuring hardware.
> > > >
> > > > The driver in question here is for a serdes controller which handles
> > > > physical ethernet interfaces.
> > > > Admin function driver gathers info w.r.t current state of physical
> > > > ethernet interfaces from this driver
> > > > and notifies actual ethernet driver about changes, if any.
> > >
> > > Ok. Can you describe the structure that the PCI devices appear
> > > in? It might help to be make the connection between the differnet
> > > patches to understand how things fit together. In the final
> > > picture, how many different pci_driver instances do you have,
> > > and what part are they for?
> >
> > List of PCI devices are CGX, RVU PF0-PFn SRIOV physical functions
> > and RVU VF0-VFn SRIOV virtual functions. No of VFs per PF is configurable
> > and done by low level firmware.
> >
> > List of PCI driver instances would be CGX driver, RVU PF0 (i.e admin
> > function) driver,
> > PF1-PFn either netdev driver or crypto driver, VF0-VFn functionality would 
> > be
> > same as their PF.
> >
> > The current plan is to have CGX driver, Admin function driver, PF
> > netdev driver, VF netdev driver and PF/VF crypto drivers.
>
> Ok, I think I understand the PF/VF distinction now. One (to me)
> surprising aspect here is that you not just have one physical function
> that you can use to assign resources to multiple virtual functions,
> but also a second level of virtualization that is used to assign
> resources to "physical functions" that are less physical than the
> name suggests.

Yes, PF is just for name sake, on-boot there is no difference between
PFs/VFs as such.
PF0 has privilege access to assign resources to all PFs and their VFs.
This admin function driver loads for PF0.

>
> The part that I have not grasped yet is what the split between
> the CGX and the AF is for, how they relate to one another, and
> what the software abstraction for the two is going to be.

In HW, CGX is a separate PCI device which handles the serdes and
physical ethernet interface.
Ethernet driver in drivers/net/ethernet can only communicate to
admin function driver since they share a mailbox memory.
So we had to bind both CGX and admin function drivers to almost work as one,
inorder to provide relavent info to ethernet drivers. That's why we
have many functions
from CGX driver which AF uses.

eg: Firmware gets to know about a physical interface status change,
which CGX driver gets
to know and it uses AF's mailbox communication to inform ethernet
driver about the event.

>
> > > Is the idea that an ethernet device driver always attaches to a
> > > virtual function that gets created by the main driver, and that
> > > the two drivers share no interfaces on the kernel side, or do
> > > you have multiple drivers linking to each other?
> >
> > Ethernet device driver can attach to both physical function and virtual 
> > function
> > whose HW resources are provisioned by admin function driver.
> >
> > Yes the PF/VF ethernet drivers and these drivers won't share any
> > kernel interfaces.
> > Physical ethernet interface is owned by ethernet driver only, this driver 
> > just
> > configures which ethernet driver instance uses which physcial interface.
>
> Ok.
>
>  Arnd

Re: [PATCH 10/15] soc: octeontx2: Reconfig MSIX base with IOVA

[Sunil Kovvuri]

On Tue, Aug 28, 2018 at 6:27 PM Arnd Bergmann  wrote:
>
> On Tue, Aug 28, 2018 at 2:42 PM Sunil Kovvuri  wrote:
> >
> > On Tue, Aug 28, 2018 at 5:39 PM Arnd Bergmann  wrote:
> > >
> > > On Tue, Aug 28, 2018 at 12:58 PM  wrote:
> > > >
> > > > From: Geetha sowjanya 
> > > >
> > > > HW interprets RVU_AF_MSIXTR_BASE address as an IOVA, hence
> > > > create a IOMMU mapping for the physcial address configured by
> > > > firmware and reconfig RVU_AF_MSIXTR_BASE with IOVA.
> > > >
> > > > Signed-off-by: Geetha sowjanya 
> > > > Signed-off-by: Sunil Goutham 
> > >
> > > I think this needs some more explanation. What is the difference between
> > > the MSI-X support in this driver and every other one? Are you working
> > > around a hardware bug, or is there something odd in the implementation
> > > of your irqchip driver? Do you use a GIC to handle the MSI interrupts
> > > or something else?
> >
> > This admin function is a PCI device which is capable of provisioning
> > HW blocks to other PCIe SRIOV devices in the system. Each HW block
> > (eg memory buffer pools, NIC dewscriptors, crypto engines e.t.c) needs
> > certain no of MSIX vectors. Admin function has a set of 32K MSIX vectors
> > in memory (not on-chip) which based on HW block provisioning to a PCI device
> > attaches the required number of vectors to that device. Some part of this
> > configuration is done by low level firmware.
> >
> > RVU_AF_MSIXTR_BASE points to the memory region allocated for 32K MSIX
> > vectors. If kernel is booted with IOMMU enabled and admin function device
> > is attached to SMMU, HW will go through translation to access this MSIX
> > vector memory region. Hence the mapping done in this patch.
>
> Do you mean this is not a regular PCIe MSI-X interrupt to the GIC, but
> something internal to your device that gets routed through the IOMMU
> back into the device?
>
> I'm still confused.
>
>Arnd

This is a regular PCIe MSI-X interrupt, the difference is that the
bunch of PCI devices
here doesn't have a fixed set of MSIX vectors. Admin function has a
memory region with
32K MSIX vectors which it provisions to PCI devices based on the HW
functional blocks
attached to them. A PCI device which works as a ethernet device needs
X number of vectors
and a crypto device needs Y number of vectors.

Since the admin function owns the whole MSIX vector region, HW uses
this device's stream ID
to access the vectors. Hence the IOMMU mapping. Once MSIX vectors are
provisioned to
a PCI device they work as normal MSIX interrupt like any other device.

Thanks,
Sunil.

Re: [PATCH 10/15] soc: octeontx2: Reconfig MSIX base with IOVA

[Sunil Kovvuri]

On Tue, Aug 28, 2018 at 6:27 PM Arnd Bergmann  wrote:
>
> On Tue, Aug 28, 2018 at 2:42 PM Sunil Kovvuri  wrote:
> >
> > On Tue, Aug 28, 2018 at 5:39 PM Arnd Bergmann  wrote:
> > >
> > > On Tue, Aug 28, 2018 at 12:58 PM  wrote:
> > > >
> > > > From: Geetha sowjanya 
> > > >
> > > > HW interprets RVU_AF_MSIXTR_BASE address as an IOVA, hence
> > > > create a IOMMU mapping for the physcial address configured by
> > > > firmware and reconfig RVU_AF_MSIXTR_BASE with IOVA.
> > > >
> > > > Signed-off-by: Geetha sowjanya 
> > > > Signed-off-by: Sunil Goutham 
> > >
> > > I think this needs some more explanation. What is the difference between
> > > the MSI-X support in this driver and every other one? Are you working
> > > around a hardware bug, or is there something odd in the implementation
> > > of your irqchip driver? Do you use a GIC to handle the MSI interrupts
> > > or something else?
> >
> > This admin function is a PCI device which is capable of provisioning
> > HW blocks to other PCIe SRIOV devices in the system. Each HW block
> > (eg memory buffer pools, NIC dewscriptors, crypto engines e.t.c) needs
> > certain no of MSIX vectors. Admin function has a set of 32K MSIX vectors
> > in memory (not on-chip) which based on HW block provisioning to a PCI device
> > attaches the required number of vectors to that device. Some part of this
> > configuration is done by low level firmware.
> >
> > RVU_AF_MSIXTR_BASE points to the memory region allocated for 32K MSIX
> > vectors. If kernel is booted with IOMMU enabled and admin function device
> > is attached to SMMU, HW will go through translation to access this MSIX
> > vector memory region. Hence the mapping done in this patch.
>
> Do you mean this is not a regular PCIe MSI-X interrupt to the GIC, but
> something internal to your device that gets routed through the IOMMU
> back into the device?
>
> I'm still confused.
>
>Arnd

This is a regular PCIe MSI-X interrupt, the difference is that the
bunch of PCI devices
here doesn't have a fixed set of MSIX vectors. Admin function has a
memory region with
32K MSIX vectors which it provisions to PCI devices based on the HW
functional blocks
attached to them. A PCI device which works as a ethernet device needs
X number of vectors
and a crypto device needs Y number of vectors.

Since the admin function owns the whole MSIX vector region, HW uses
this device's stream ID
to access the vectors. Hence the IOMMU mapping. Once MSIX vectors are
provisioned to
a PCI device they work as normal MSIX interrupt like any other device.

Thanks,
Sunil.

Re: [PATCH 11/15] soc: octeontx2: Add Marvell OcteonTX2 CGX driver

[Sunil Kovvuri]

> > > If this is a regular PCI ethernet driver, why do you put it into 
> > > driver/soc
> > > rather than drivers/net/ethernet/ ?
> >
> > No, this is not a ethernet driver, as mentioned in the cover letter
> > this driver and AF driver doesn't
> > handle any IO. There will be a separate ethernet driver (will submit
> > that as well in future) which will
> > communicate with these drivers for configuring hardware.
> >
> > The driver in question here is for a serdes controller which handles
> > physical ethernet interfaces.
> > Admin function driver gathers info w.r.t current state of physical
> > ethernet interfaces from this driver
> > and notifies actual ethernet driver about changes, if any.
>
> Ok. Can you describe the structure that the PCI devices appear
> in? It might help to be make the connection between the differnet
> patches to understand how things fit together. In the final
> picture, how many different pci_driver instances do you have,
> and what part are they for?

List of PCI devices are CGX, RVU PF0-PFn SRIOV physical functions
and RVU VF0-VFn SRIOV virtual functions. No of VFs per PF is configurable
and done by low level firmware.

List of PCI driver instances would be CGX driver, RVU PF0 (i.e admin
function) driver,
PF1-PFn either netdev driver or crypto driver, VF0-VFn functionality would be
same as their PF.

The current plan is to have CGX driver, Admin function driver, PF
netdev driver,
VF netdev driver and PF/VF crypto drivers.

>
> Is the idea that an ethernet device driver always attaches to a
> virtual function that gets created by the main driver, and that
> the two drivers share no interfaces on the kernel side, or do
> you have multiple drivers linking to each other?

Ethernet device driver can attach to both physical function and virtual function
whose HW resources are provisioned by admin function driver.

Yes the PF/VF ethernet drivers and these drivers won't share any
kernel interfaces.
Physical ethernet interface is owned by ethernet driver only, this driver just
configures which ethernet driver instance uses which physcial interface.

>
>   Arnd

Re: [PATCH 11/15] soc: octeontx2: Add Marvell OcteonTX2 CGX driver

[Sunil Kovvuri]

> > > If this is a regular PCI ethernet driver, why do you put it into 
> > > driver/soc
> > > rather than drivers/net/ethernet/ ?
> >
> > No, this is not a ethernet driver, as mentioned in the cover letter
> > this driver and AF driver doesn't
> > handle any IO. There will be a separate ethernet driver (will submit
> > that as well in future) which will
> > communicate with these drivers for configuring hardware.
> >
> > The driver in question here is for a serdes controller which handles
> > physical ethernet interfaces.
> > Admin function driver gathers info w.r.t current state of physical
> > ethernet interfaces from this driver
> > and notifies actual ethernet driver about changes, if any.
>
> Ok. Can you describe the structure that the PCI devices appear
> in? It might help to be make the connection between the differnet
> patches to understand how things fit together. In the final
> picture, how many different pci_driver instances do you have,
> and what part are they for?

List of PCI devices are CGX, RVU PF0-PFn SRIOV physical functions
and RVU VF0-VFn SRIOV virtual functions. No of VFs per PF is configurable
and done by low level firmware.

List of PCI driver instances would be CGX driver, RVU PF0 (i.e admin
function) driver,
PF1-PFn either netdev driver or crypto driver, VF0-VFn functionality would be
same as their PF.

The current plan is to have CGX driver, Admin function driver, PF
netdev driver,
VF netdev driver and PF/VF crypto drivers.

>
> Is the idea that an ethernet device driver always attaches to a
> virtual function that gets created by the main driver, and that
> the two drivers share no interfaces on the kernel side, or do
> you have multiple drivers linking to each other?

Ethernet device driver can attach to both physical function and virtual function
whose HW resources are provisioned by admin function driver.

Yes the PF/VF ethernet drivers and these drivers won't share any
kernel interfaces.
Physical ethernet interface is owned by ethernet driver only, this driver just
configures which ethernet driver instance uses which physcial interface.

>
>   Arnd

Re: [PATCH 04/15] soc: octeontx2: Add mailbox support infra

[Sunil Kovvuri]

On Tue, Aug 28, 2018 at 5:33 PM Arnd Bergmann  wrote:
>
> On Tue, Aug 28, 2018 at 12:57 PM  wrote:
> >
> > From: Aleksey Makarov 
> >
> > This patch adds mailbox support infrastructure APIs.
> > Each RVU device has a dedicated 64KB mailbox region
> > shared with it's peer for communication. RVU AF has
> > a separate mailbox region shared with each of RVU PFs
> > and a RVU PF has a separate region shared with each of
> > it's VF.
> >
> > These set of APIs are used by this driver (RVU AF) and
> > other RVU PF/VF drivers eg netdev, crypto e.t.c.
> >
> > Signed-off-by: Aleksey Makarov 
> > Signed-off-by: Sunil Goutham 
> > Signed-off-by: Lukasz Bartosik 
>
> Why does this driver not use the drivers/mailbox/ infrastructure?
>
>Arnd

This is a common administrative software driver which will be handling requests
from kernel drivers and as well as drivers in userspace applications.
We had to keep
mailbox communication infrastructure same across all usages.

Thanks,
Sunil.

Re: [PATCH 04/15] soc: octeontx2: Add mailbox support infra

[Sunil Kovvuri]

On Tue, Aug 28, 2018 at 5:33 PM Arnd Bergmann  wrote:
>
> On Tue, Aug 28, 2018 at 12:57 PM  wrote:
> >
> > From: Aleksey Makarov 
> >
> > This patch adds mailbox support infrastructure APIs.
> > Each RVU device has a dedicated 64KB mailbox region
> > shared with it's peer for communication. RVU AF has
> > a separate mailbox region shared with each of RVU PFs
> > and a RVU PF has a separate region shared with each of
> > it's VF.
> >
> > These set of APIs are used by this driver (RVU AF) and
> > other RVU PF/VF drivers eg netdev, crypto e.t.c.
> >
> > Signed-off-by: Aleksey Makarov 
> > Signed-off-by: Sunil Goutham 
> > Signed-off-by: Lukasz Bartosik 
>
> Why does this driver not use the drivers/mailbox/ infrastructure?
>
>Arnd

This is a common administrative software driver which will be handling requests
from kernel drivers and as well as drivers in userspace applications.
We had to keep
mailbox communication infrastructure same across all usages.

Thanks,
Sunil.

Re: [PATCH 10/15] soc: octeontx2: Reconfig MSIX base with IOVA

[Sunil Kovvuri]

On Tue, Aug 28, 2018 at 5:39 PM Arnd Bergmann  wrote:
>
> On Tue, Aug 28, 2018 at 12:58 PM  wrote:
> >
> > From: Geetha sowjanya 
> >
> > HW interprets RVU_AF_MSIXTR_BASE address as an IOVA, hence
> > create a IOMMU mapping for the physcial address configured by
> > firmware and reconfig RVU_AF_MSIXTR_BASE with IOVA.
> >
> > Signed-off-by: Geetha sowjanya 
> > Signed-off-by: Sunil Goutham 
>
> I think this needs some more explanation. What is the difference between
> the MSI-X support in this driver and every other one? Are you working
> around a hardware bug, or is there something odd in the implementation
> of your irqchip driver? Do you use a GIC to handle the MSI interrupts
> or something else?
>
>Arnd

This admin function is a PCI device which is capable of provisioning
HW blocks to other PCIe SRIOV devices in the system. Each HW block
(eg memory buffer pools, NIC dewscriptors, crypto engines e.t.c) needs
certain no of MSIX vectors. Admin function has a set of 32K MSIX vectors
in memory (not on-chip) which based on HW block provisioning to a PCI device
attaches the required number of vectors to that device. Some part of this
configuration is done by low level firmware.

RVU_AF_MSIXTR_BASE points to the memory region allocated for 32K MSIX
vectors. If kernel is booted with IOMMU enabled and admin function device
is attached to SMMU, HW will go through translation to access this MSIX
vector memory region. Hence the mapping done in this patch.

Thanks,
Sunil.

Re: [PATCH 10/15] soc: octeontx2: Reconfig MSIX base with IOVA

[Sunil Kovvuri]

On Tue, Aug 28, 2018 at 5:39 PM Arnd Bergmann  wrote:
>
> On Tue, Aug 28, 2018 at 12:58 PM  wrote:
> >
> > From: Geetha sowjanya 
> >
> > HW interprets RVU_AF_MSIXTR_BASE address as an IOVA, hence
> > create a IOMMU mapping for the physcial address configured by
> > firmware and reconfig RVU_AF_MSIXTR_BASE with IOVA.
> >
> > Signed-off-by: Geetha sowjanya 
> > Signed-off-by: Sunil Goutham 
>
> I think this needs some more explanation. What is the difference between
> the MSI-X support in this driver and every other one? Are you working
> around a hardware bug, or is there something odd in the implementation
> of your irqchip driver? Do you use a GIC to handle the MSI interrupts
> or something else?
>
>Arnd

This admin function is a PCI device which is capable of provisioning
HW blocks to other PCIe SRIOV devices in the system. Each HW block
(eg memory buffer pools, NIC dewscriptors, crypto engines e.t.c) needs
certain no of MSIX vectors. Admin function has a set of 32K MSIX vectors
in memory (not on-chip) which based on HW block provisioning to a PCI device
attaches the required number of vectors to that device. Some part of this
configuration is done by low level firmware.

RVU_AF_MSIXTR_BASE points to the memory region allocated for 32K MSIX
vectors. If kernel is booted with IOMMU enabled and admin function device
is attached to SMMU, HW will go through translation to access this MSIX
vector memory region. Hence the mapping done in this patch.

Thanks,
Sunil.

Re: [PATCH 11/15] soc: octeontx2: Add Marvell OcteonTX2 CGX driver

[Sunil Kovvuri]

On Tue, Aug 28, 2018 at 5:40 PM Arnd Bergmann  wrote:
>
> On Tue, Aug 28, 2018 at 12:58 PM  wrote:
> >
> > From: Sunil Goutham 
> >
> > This patch adds basic template for Marvell OcteonTX2's
> > CGX ethernet interface driver. Just the probe.
> > RVU AF driver will use APIs exported by this driver
> > for various things like PF to physical interface mapping,
> > loopback mode, interface stats etc.
> >
> > Signed-off-by: Sunil Goutham 
> > ---
> >  drivers/soc/marvell/Kconfig|  10 +++
> >  drivers/soc/marvell/octeontx2/Makefile |   2 +
> >  drivers/soc/marvell/octeontx2/cgx.c| 117 
> > +
> >  drivers/soc/marvell/octeontx2/cgx.h|  20 ++
>
> If this is a regular PCI ethernet driver, why do you put it into driver/soc
> rather than drivers/net/ethernet/ ?

No, this is not a ethernet driver, as mentioned in the cover letter
this driver and AF driver doesn't
handle any IO. There will be a separate ethernet driver (will submit
that as well in future) which will
communicate with these drivers for configuring hardware.

The driver in question here is for a serdes controller which handles
physical ethernet interfaces.
Admin function driver gathers info w.r.t current state of physical
ethernet interfaces from this driver
and notifies actual ethernet driver about changes, if any.

Thanks,
Sunil.

>
>   Arnd

Re: [PATCH 11/15] soc: octeontx2: Add Marvell OcteonTX2 CGX driver

[Sunil Kovvuri]

On Tue, Aug 28, 2018 at 5:40 PM Arnd Bergmann  wrote:
>
> On Tue, Aug 28, 2018 at 12:58 PM  wrote:
> >
> > From: Sunil Goutham 
> >
> > This patch adds basic template for Marvell OcteonTX2's
> > CGX ethernet interface driver. Just the probe.
> > RVU AF driver will use APIs exported by this driver
> > for various things like PF to physical interface mapping,
> > loopback mode, interface stats etc.
> >
> > Signed-off-by: Sunil Goutham 
> > ---
> >  drivers/soc/marvell/Kconfig|  10 +++
> >  drivers/soc/marvell/octeontx2/Makefile |   2 +
> >  drivers/soc/marvell/octeontx2/cgx.c| 117 
> > +
> >  drivers/soc/marvell/octeontx2/cgx.h|  20 ++
>
> If this is a regular PCI ethernet driver, why do you put it into driver/soc
> rather than drivers/net/ethernet/ ?

No, this is not a ethernet driver, as mentioned in the cover letter
this driver and AF driver doesn't
handle any IO. There will be a separate ethernet driver (will submit
that as well in future) which will
communicate with these drivers for configuring hardware.

The driver in question here is for a serdes controller which handles
physical ethernet interfaces.
Admin function driver gathers info w.r.t current state of physical
ethernet interfaces from this driver
and notifies actual ethernet driver about changes, if any.

Thanks,
Sunil.

>
>   Arnd

[PATCH 11/15] soc: octeontx2: Add Marvell OcteonTX2 CGX driver

[sunil . kovvuri]

From: Sunil Goutham 

This patch adds basic template for Marvell OcteonTX2's
CGX ethernet interface driver. Just the probe.
RVU AF driver will use APIs exported by this driver
for various things like PF to physical interface mapping,
loopback mode, interface stats etc.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/Kconfig|  10 +++
 drivers/soc/marvell/octeontx2/Makefile |   2 +
 drivers/soc/marvell/octeontx2/cgx.c| 117 +
 drivers/soc/marvell/octeontx2/cgx.h|  20 ++
 4 files changed, 149 insertions(+)
 create mode 100644 drivers/soc/marvell/octeontx2/cgx.c
 create mode 100644 drivers/soc/marvell/octeontx2/cgx.h

diff --git a/drivers/soc/marvell/Kconfig b/drivers/soc/marvell/Kconfig
index 4499caf..73c8f8d 100644
--- a/drivers/soc/marvell/Kconfig
+++ b/drivers/soc/marvell/Kconfig
@@ -7,7 +7,17 @@ menu "Marvell SoC drivers"
 config OCTEONTX2_AF
tristate "OcteonTX2 RVU Admin Function driver"
depends on ARM64 && PCI
+   select OCTEONTX2_CGX
help
  This driver supports Marvell's OcteonTX2 Resource Virtualization
  Unit's admin function manager which manages all RVU HW resources.
+
+config OCTEONTX2_CGX
+   tristate "OcteonTX2 MAC interface (CGX) driver"
+   depends on ARM64 && PCI
+   select PHYLIB
+   help
+ This driver supports programming and controlling of MAC
+ interfaces from RVU Admin Function driver.
+
 endmenu
diff --git a/drivers/soc/marvell/octeontx2/Makefile 
b/drivers/soc/marvell/octeontx2/Makefile
index 8737ec3..50b8f74 100644
--- a/drivers/soc/marvell/octeontx2/Makefile
+++ b/drivers/soc/marvell/octeontx2/Makefile
@@ -3,6 +3,8 @@
 # Makefile for Marvell's OcteonTX2 RVU Admin Function driver
 #
 
+obj-$(CONFIG_OCTEONTX2_CGX) += octeontx2_cgx.o
 obj-$(CONFIG_OCTEONTX2_AF) += octeontx2_af.o
 
+octeontx2_cgx-y := cgx.o
 octeontx2_af-y := rvu.o mbox.o
diff --git a/drivers/soc/marvell/octeontx2/cgx.c 
b/drivers/soc/marvell/octeontx2/cgx.c
new file mode 100644
index 000..6f0b6f4
--- /dev/null
+++ b/drivers/soc/marvell/octeontx2/cgx.c
@@ -0,0 +1,117 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Marvell OcteonTx2 CGX driver
+ *
+ * Copyright (C) 2018 Marvell International Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "cgx.h"
+
+#define DRV_NAME   "octeontx2-cgx"
+#define DRV_STRING  "Marvell OcteonTX2 CGX/MAC Driver"
+#define DRV_VERSION"1.0"
+
+struct cgx {
+   void __iomem*reg_base;
+   struct pci_dev  *pdev;
+   u8  cgx_id;
+};
+
+/* Supported devices */
+static const struct pci_device_id cgx_id_table[] = {
+   { PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_CGX) },
+   { 0, }  /* end of table */
+};
+
+MODULE_AUTHOR("Marvell International Ltd.");
+MODULE_DESCRIPTION(DRV_STRING);
+MODULE_LICENSE("GPL v2");
+MODULE_VERSION(DRV_VERSION);
+MODULE_DEVICE_TABLE(pci, cgx_id_table);
+
+static int cgx_probe(struct pci_dev *pdev, const struct pci_device_id *id)
+{
+   int err;
+   struct device *dev = >dev;
+   struct cgx *cgx;
+
+   cgx = devm_kzalloc(dev, sizeof(*cgx), GFP_KERNEL);
+   if (!cgx)
+   return -ENOMEM;
+   cgx->pdev = pdev;
+
+   pci_set_drvdata(pdev, cgx);
+
+   err = pci_enable_device(pdev);
+   if (err) {
+   dev_err(dev, "Failed to enable PCI device\n");
+   pci_set_drvdata(pdev, NULL);
+   return err;
+   }
+
+   err = pci_request_regions(pdev, DRV_NAME);
+   if (err) {
+   dev_err(dev, "PCI request regions failed 0x%x\n", err);
+   goto err_disable_device;
+   }
+
+   /* MAP configuration registers */
+   cgx->reg_base = pcim_iomap(pdev, PCI_CFG_REG_BAR_NUM, 0);
+   if (!cgx->reg_base) {
+   dev_err(dev, "CGX: Cannot map CSR memory space, aborting\n");
+   err = -ENOMEM;
+   goto err_release_regions;
+   }
+
+   return 0;
+
+err_release_regions:
+   pci_release_regions(pdev);
+err_disable_device:
+   pci_disable_device(pdev);
+   pci_set_drvdata(pdev, NULL);
+   return err;
+}
+
+static void cgx_remove(struct pci_dev *pdev)
+{
+   pci_release_regions(pdev);
+   pci_disable_device(pdev);
+   pci_set_drvdata(pdev, NULL);
+}
+
+static struct pci_driver cgx_driver = {
+   .name = DRV_NAME,
+   .id_table = cgx_id_table,
+   .probe = cgx_probe,
+   .remove = cgx_remove,
+};
+
+static int __init cgx_init_module(void)
+{
+   pr_info("%s: %s\n", DRV_NAME, DRV_STRING);
+
+   return pci_register_driver(_driver);
+}
+
+static void __exit cgx_cleanup_module(void)
+{
+   

[PATCH 12/15] soc: octeontx2: Set RVU PFs to CGX LMACs mapping

[sunil . kovvuri]

From: Linu Cherian 

Each of the enabled CGX LMAC is considered a physical
interface and RVU PFs are mapped to these. VFs of these
SRIOV PFs will be virtual interfaces and share CGX LMAC
along with PF.

This mapping info will be used later on for Rx/Tx pkt steering.

Signed-off-by: Linu Cherian 
Signed-off-by: Geetha sowjanya 
---
 drivers/soc/marvell/octeontx2/Makefile  |  2 +-
 drivers/soc/marvell/octeontx2/cgx.c | 59 
 drivers/soc/marvell/octeontx2/cgx.h | 15 -
 drivers/soc/marvell/octeontx2/rvu.c |  4 ++
 drivers/soc/marvell/octeontx2/rvu.h | 12 
 drivers/soc/marvell/octeontx2/rvu_cgx.c | 97 +
 6 files changed, 186 insertions(+), 3 deletions(-)
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_cgx.c

diff --git a/drivers/soc/marvell/octeontx2/Makefile 
b/drivers/soc/marvell/octeontx2/Makefile
index 50b8f74..74c768d 100644
--- a/drivers/soc/marvell/octeontx2/Makefile
+++ b/drivers/soc/marvell/octeontx2/Makefile
@@ -7,4 +7,4 @@ obj-$(CONFIG_OCTEONTX2_CGX) += octeontx2_cgx.o
 obj-$(CONFIG_OCTEONTX2_AF) += octeontx2_af.o
 
 octeontx2_cgx-y := cgx.o
-octeontx2_af-y := rvu.o mbox.o
+octeontx2_af-y := rvu.o mbox.o rvu_cgx.o
diff --git a/drivers/soc/marvell/octeontx2/cgx.c 
b/drivers/soc/marvell/octeontx2/cgx.c
index 6f0b6f4..70ca2e2f1 100644
--- a/drivers/soc/marvell/octeontx2/cgx.c
+++ b/drivers/soc/marvell/octeontx2/cgx.c
@@ -29,8 +29,12 @@ struct cgx {
void __iomem*reg_base;
struct pci_dev  *pdev;
u8  cgx_id;
+   u8  lmac_count;
+   struct list_headcgx_list;
 };
 
+static LIST_HEAD(cgx_list);
+
 /* Supported devices */
 static const struct pci_device_id cgx_id_table[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_CGX) },
@@ -43,6 +47,53 @@ MODULE_LICENSE("GPL v2");
 MODULE_VERSION(DRV_VERSION);
 MODULE_DEVICE_TABLE(pci, cgx_id_table);
 
+static u64 cgx_read(struct cgx *cgx, u64 lmac, u64 offset)
+{
+   return readq(cgx->reg_base + (lmac << 18) + offset);
+}
+
+int cgx_get_cgx_cnt(void)
+{
+   struct cgx *cgx_dev;
+   int count = 0;
+
+   list_for_each_entry(cgx_dev, _list, cgx_list)
+   count++;
+
+   return count;
+}
+EXPORT_SYMBOL(cgx_get_cgx_cnt);
+
+int cgx_get_lmac_cnt(void *cgxd)
+{
+   struct cgx *cgx = cgxd;
+
+   if (!cgx)
+   return -ENODEV;
+
+   return cgx->lmac_count;
+}
+EXPORT_SYMBOL(cgx_get_lmac_cnt);
+
+void *cgx_get_pdata(int cgx_id)
+{
+   struct cgx *cgx_dev;
+
+   list_for_each_entry(cgx_dev, _list, cgx_list) {
+   if (cgx_dev->cgx_id == cgx_id)
+   return cgx_dev;
+   }
+   return NULL;
+}
+EXPORT_SYMBOL(cgx_get_pdata);
+
+static void cgx_lmac_init(struct cgx *cgx)
+{
+   cgx->lmac_count = cgx_read(cgx, 0, CGXX_CMRX_RX_LMACS) & 0x7;
+   if (cgx->lmac_count > MAX_LMAC_PER_CGX)
+   cgx->lmac_count = MAX_LMAC_PER_CGX;
+}
+
 static int cgx_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 {
int err;
@@ -77,9 +128,14 @@ static int cgx_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
goto err_release_regions;
}
 
+   list_add(>cgx_list, _list);
+   cgx->cgx_id = cgx_get_cgx_cnt() - 1;
+   cgx_lmac_init(cgx);
+
return 0;
 
 err_release_regions:
+   list_del(>cgx_list);
pci_release_regions(pdev);
 err_disable_device:
pci_disable_device(pdev);
@@ -89,6 +145,9 @@ static int cgx_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
 
 static void cgx_remove(struct pci_dev *pdev)
 {
+   struct cgx *cgx = pci_get_drvdata(pdev);
+
+   list_del(>cgx_list);
pci_release_regions(pdev);
pci_disable_device(pdev);
pci_set_drvdata(pdev, NULL);
diff --git a/drivers/soc/marvell/octeontx2/cgx.h 
b/drivers/soc/marvell/octeontx2/cgx.h
index 8056264..9ec72b1 100644
--- a/drivers/soc/marvell/octeontx2/cgx.h
+++ b/drivers/soc/marvell/octeontx2/cgx.h
@@ -12,9 +12,20 @@
 #define CGX_H
 
  /* PCI device IDs */
-#definePCI_DEVID_OCTEONTX2_CGX 0xA059
+#definePCI_DEVID_OCTEONTX2_CGX 0xA059
 
 /* PCI BAR nos */
-#define PCI_CFG_REG_BAR_NUM0
+#define PCI_CFG_REG_BAR_NUM0
 
+#define MAX_CGX3
+#define MAX_LMAC_PER_CGX   4
+#define CGX_OFFSET(x)  ((x) * MAX_LMAC_PER_CGX)
+
+/* Registers */
+#define CGXX_CMRX_RX_ID_MAP0x060
+#define CGXX_CMRX_RX_LMACS 0x128
+
+int cgx_get_cgx_cnt(void);
+int cgx_get_lmac_cnt(void *cgxd);
+void *cgx_get_pdata(int cgx_id);
 #endif /* CGX_H */
diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 40684c9..d7b19e0 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -1557,6 +1557,10 @@ static int rvu_probe(struct 

[PATCH 11/15] soc: octeontx2: Add Marvell OcteonTX2 CGX driver

[sunil . kovvuri]

From: Sunil Goutham 

This patch adds basic template for Marvell OcteonTX2's
CGX ethernet interface driver. Just the probe.
RVU AF driver will use APIs exported by this driver
for various things like PF to physical interface mapping,
loopback mode, interface stats etc.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/Kconfig|  10 +++
 drivers/soc/marvell/octeontx2/Makefile |   2 +
 drivers/soc/marvell/octeontx2/cgx.c| 117 +
 drivers/soc/marvell/octeontx2/cgx.h|  20 ++
 4 files changed, 149 insertions(+)
 create mode 100644 drivers/soc/marvell/octeontx2/cgx.c
 create mode 100644 drivers/soc/marvell/octeontx2/cgx.h

diff --git a/drivers/soc/marvell/Kconfig b/drivers/soc/marvell/Kconfig
index 4499caf..73c8f8d 100644
--- a/drivers/soc/marvell/Kconfig
+++ b/drivers/soc/marvell/Kconfig
@@ -7,7 +7,17 @@ menu "Marvell SoC drivers"
 config OCTEONTX2_AF
tristate "OcteonTX2 RVU Admin Function driver"
depends on ARM64 && PCI
+   select OCTEONTX2_CGX
help
  This driver supports Marvell's OcteonTX2 Resource Virtualization
  Unit's admin function manager which manages all RVU HW resources.
+
+config OCTEONTX2_CGX
+   tristate "OcteonTX2 MAC interface (CGX) driver"
+   depends on ARM64 && PCI
+   select PHYLIB
+   help
+ This driver supports programming and controlling of MAC
+ interfaces from RVU Admin Function driver.
+
 endmenu
diff --git a/drivers/soc/marvell/octeontx2/Makefile 
b/drivers/soc/marvell/octeontx2/Makefile
index 8737ec3..50b8f74 100644
--- a/drivers/soc/marvell/octeontx2/Makefile
+++ b/drivers/soc/marvell/octeontx2/Makefile
@@ -3,6 +3,8 @@
 # Makefile for Marvell's OcteonTX2 RVU Admin Function driver
 #
 
+obj-$(CONFIG_OCTEONTX2_CGX) += octeontx2_cgx.o
 obj-$(CONFIG_OCTEONTX2_AF) += octeontx2_af.o
 
+octeontx2_cgx-y := cgx.o
 octeontx2_af-y := rvu.o mbox.o
diff --git a/drivers/soc/marvell/octeontx2/cgx.c 
b/drivers/soc/marvell/octeontx2/cgx.c
new file mode 100644
index 000..6f0b6f4
--- /dev/null
+++ b/drivers/soc/marvell/octeontx2/cgx.c
@@ -0,0 +1,117 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Marvell OcteonTx2 CGX driver
+ *
+ * Copyright (C) 2018 Marvell International Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "cgx.h"
+
+#define DRV_NAME   "octeontx2-cgx"
+#define DRV_STRING  "Marvell OcteonTX2 CGX/MAC Driver"
+#define DRV_VERSION"1.0"
+
+struct cgx {
+   void __iomem*reg_base;
+   struct pci_dev  *pdev;
+   u8  cgx_id;
+};
+
+/* Supported devices */
+static const struct pci_device_id cgx_id_table[] = {
+   { PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_CGX) },
+   { 0, }  /* end of table */
+};
+
+MODULE_AUTHOR("Marvell International Ltd.");
+MODULE_DESCRIPTION(DRV_STRING);
+MODULE_LICENSE("GPL v2");
+MODULE_VERSION(DRV_VERSION);
+MODULE_DEVICE_TABLE(pci, cgx_id_table);
+
+static int cgx_probe(struct pci_dev *pdev, const struct pci_device_id *id)
+{
+   int err;
+   struct device *dev = >dev;
+   struct cgx *cgx;
+
+   cgx = devm_kzalloc(dev, sizeof(*cgx), GFP_KERNEL);
+   if (!cgx)
+   return -ENOMEM;
+   cgx->pdev = pdev;
+
+   pci_set_drvdata(pdev, cgx);
+
+   err = pci_enable_device(pdev);
+   if (err) {
+   dev_err(dev, "Failed to enable PCI device\n");
+   pci_set_drvdata(pdev, NULL);
+   return err;
+   }
+
+   err = pci_request_regions(pdev, DRV_NAME);
+   if (err) {
+   dev_err(dev, "PCI request regions failed 0x%x\n", err);
+   goto err_disable_device;
+   }
+
+   /* MAP configuration registers */
+   cgx->reg_base = pcim_iomap(pdev, PCI_CFG_REG_BAR_NUM, 0);
+   if (!cgx->reg_base) {
+   dev_err(dev, "CGX: Cannot map CSR memory space, aborting\n");
+   err = -ENOMEM;
+   goto err_release_regions;
+   }
+
+   return 0;
+
+err_release_regions:
+   pci_release_regions(pdev);
+err_disable_device:
+   pci_disable_device(pdev);
+   pci_set_drvdata(pdev, NULL);
+   return err;
+}
+
+static void cgx_remove(struct pci_dev *pdev)
+{
+   pci_release_regions(pdev);
+   pci_disable_device(pdev);
+   pci_set_drvdata(pdev, NULL);
+}
+
+static struct pci_driver cgx_driver = {
+   .name = DRV_NAME,
+   .id_table = cgx_id_table,
+   .probe = cgx_probe,
+   .remove = cgx_remove,
+};
+
+static int __init cgx_init_module(void)
+{
+   pr_info("%s: %s\n", DRV_NAME, DRV_STRING);
+
+   return pci_register_driver(_driver);
+}
+
+static void __exit cgx_cleanup_module(void)
+{
+   

[PATCH 12/15] soc: octeontx2: Set RVU PFs to CGX LMACs mapping

[sunil . kovvuri]

From: Linu Cherian 

Each of the enabled CGX LMAC is considered a physical
interface and RVU PFs are mapped to these. VFs of these
SRIOV PFs will be virtual interfaces and share CGX LMAC
along with PF.

This mapping info will be used later on for Rx/Tx pkt steering.

Signed-off-by: Linu Cherian 
Signed-off-by: Geetha sowjanya 
---
 drivers/soc/marvell/octeontx2/Makefile  |  2 +-
 drivers/soc/marvell/octeontx2/cgx.c | 59 
 drivers/soc/marvell/octeontx2/cgx.h | 15 -
 drivers/soc/marvell/octeontx2/rvu.c |  4 ++
 drivers/soc/marvell/octeontx2/rvu.h | 12 
 drivers/soc/marvell/octeontx2/rvu_cgx.c | 97 +
 6 files changed, 186 insertions(+), 3 deletions(-)
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_cgx.c

diff --git a/drivers/soc/marvell/octeontx2/Makefile 
b/drivers/soc/marvell/octeontx2/Makefile
index 50b8f74..74c768d 100644
--- a/drivers/soc/marvell/octeontx2/Makefile
+++ b/drivers/soc/marvell/octeontx2/Makefile
@@ -7,4 +7,4 @@ obj-$(CONFIG_OCTEONTX2_CGX) += octeontx2_cgx.o
 obj-$(CONFIG_OCTEONTX2_AF) += octeontx2_af.o
 
 octeontx2_cgx-y := cgx.o
-octeontx2_af-y := rvu.o mbox.o
+octeontx2_af-y := rvu.o mbox.o rvu_cgx.o
diff --git a/drivers/soc/marvell/octeontx2/cgx.c 
b/drivers/soc/marvell/octeontx2/cgx.c
index 6f0b6f4..70ca2e2f1 100644
--- a/drivers/soc/marvell/octeontx2/cgx.c
+++ b/drivers/soc/marvell/octeontx2/cgx.c
@@ -29,8 +29,12 @@ struct cgx {
void __iomem*reg_base;
struct pci_dev  *pdev;
u8  cgx_id;
+   u8  lmac_count;
+   struct list_headcgx_list;
 };
 
+static LIST_HEAD(cgx_list);
+
 /* Supported devices */
 static const struct pci_device_id cgx_id_table[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_CGX) },
@@ -43,6 +47,53 @@ MODULE_LICENSE("GPL v2");
 MODULE_VERSION(DRV_VERSION);
 MODULE_DEVICE_TABLE(pci, cgx_id_table);
 
+static u64 cgx_read(struct cgx *cgx, u64 lmac, u64 offset)
+{
+   return readq(cgx->reg_base + (lmac << 18) + offset);
+}
+
+int cgx_get_cgx_cnt(void)
+{
+   struct cgx *cgx_dev;
+   int count = 0;
+
+   list_for_each_entry(cgx_dev, _list, cgx_list)
+   count++;
+
+   return count;
+}
+EXPORT_SYMBOL(cgx_get_cgx_cnt);
+
+int cgx_get_lmac_cnt(void *cgxd)
+{
+   struct cgx *cgx = cgxd;
+
+   if (!cgx)
+   return -ENODEV;
+
+   return cgx->lmac_count;
+}
+EXPORT_SYMBOL(cgx_get_lmac_cnt);
+
+void *cgx_get_pdata(int cgx_id)
+{
+   struct cgx *cgx_dev;
+
+   list_for_each_entry(cgx_dev, _list, cgx_list) {
+   if (cgx_dev->cgx_id == cgx_id)
+   return cgx_dev;
+   }
+   return NULL;
+}
+EXPORT_SYMBOL(cgx_get_pdata);
+
+static void cgx_lmac_init(struct cgx *cgx)
+{
+   cgx->lmac_count = cgx_read(cgx, 0, CGXX_CMRX_RX_LMACS) & 0x7;
+   if (cgx->lmac_count > MAX_LMAC_PER_CGX)
+   cgx->lmac_count = MAX_LMAC_PER_CGX;
+}
+
 static int cgx_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 {
int err;
@@ -77,9 +128,14 @@ static int cgx_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
goto err_release_regions;
}
 
+   list_add(>cgx_list, _list);
+   cgx->cgx_id = cgx_get_cgx_cnt() - 1;
+   cgx_lmac_init(cgx);
+
return 0;
 
 err_release_regions:
+   list_del(>cgx_list);
pci_release_regions(pdev);
 err_disable_device:
pci_disable_device(pdev);
@@ -89,6 +145,9 @@ static int cgx_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
 
 static void cgx_remove(struct pci_dev *pdev)
 {
+   struct cgx *cgx = pci_get_drvdata(pdev);
+
+   list_del(>cgx_list);
pci_release_regions(pdev);
pci_disable_device(pdev);
pci_set_drvdata(pdev, NULL);
diff --git a/drivers/soc/marvell/octeontx2/cgx.h 
b/drivers/soc/marvell/octeontx2/cgx.h
index 8056264..9ec72b1 100644
--- a/drivers/soc/marvell/octeontx2/cgx.h
+++ b/drivers/soc/marvell/octeontx2/cgx.h
@@ -12,9 +12,20 @@
 #define CGX_H
 
  /* PCI device IDs */
-#definePCI_DEVID_OCTEONTX2_CGX 0xA059
+#definePCI_DEVID_OCTEONTX2_CGX 0xA059
 
 /* PCI BAR nos */
-#define PCI_CFG_REG_BAR_NUM0
+#define PCI_CFG_REG_BAR_NUM0
 
+#define MAX_CGX3
+#define MAX_LMAC_PER_CGX   4
+#define CGX_OFFSET(x)  ((x) * MAX_LMAC_PER_CGX)
+
+/* Registers */
+#define CGXX_CMRX_RX_ID_MAP0x060
+#define CGXX_CMRX_RX_LMACS 0x128
+
+int cgx_get_cgx_cnt(void);
+int cgx_get_lmac_cnt(void *cgxd);
+void *cgx_get_pdata(int cgx_id);
 #endif /* CGX_H */
diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 40684c9..d7b19e0 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -1557,6 +1557,10 @@ static int rvu_probe(struct 

[PATCH 09/15] soc: octeontx2: Configure block LF's MSIX vector offset

[sunil . kovvuri]

From: Sunil Goutham 

Firmware configures a certain number of MSIX vectors to each of
enabled RVU PF/VF. When a block LF is attached to a PF/VF, number
of MSIX vectors needed by that LF are set aside (out of PF/VF's
total MSIX vectors) and LF's msix_offset is configured in HW.

Also added support for a RVU PF/VF to retrieve that block LF's
MSIX vector offset information from AF via mbox.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/mbox.h   |  18 ++
 drivers/soc/marvell/octeontx2/rvu.c| 333 -
 drivers/soc/marvell/octeontx2/rvu.h|   7 +
 drivers/soc/marvell/octeontx2/rvu_struct.h |   2 +
 4 files changed, 357 insertions(+), 3 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/mbox.h 
b/drivers/soc/marvell/octeontx2/mbox.h
index 7280d49..bedf0ee 100644
--- a/drivers/soc/marvell/octeontx2/mbox.h
+++ b/drivers/soc/marvell/octeontx2/mbox.h
@@ -122,6 +122,7 @@ static inline struct mbox_msghdr 
*otx2_mbox_alloc_msg(struct otx2_mbox *mbox,
 M(READY,   0x001, msg_req, ready_msg_rsp)  \
 M(ATTACH_RESOURCES,0x002, rsrc_attach, msg_rsp)\
 M(DETACH_RESOURCES,0x003, rsrc_detach, msg_rsp)\
+M(MSIX_OFFSET, 0x004, msg_req, msix_offset_rsp)\
 /* CGX mbox IDs (range 0x200 - 0x3FF) */   \
 /* NPA mbox IDs (range 0x400 - 0x5FF) */   \
 /* SSO/SSOW mbox IDs (range 0x600 - 0x7FF) */  \
@@ -190,4 +191,21 @@ struct rsrc_detach {
u8 cptlfs:1;
 };
 
+#define MSIX_VECTOR_INVALID0x
+#define MAX_RVU_BLKLF_CNT  256
+
+struct msix_offset_rsp {
+   struct mbox_msghdr hdr;
+   u16  npa_msixoff;
+   u16  nix_msixoff;
+   u8   sso;
+   u8   ssow;
+   u8   timlfs;
+   u8   cptlfs;
+   u16  sso_msixoff[MAX_RVU_BLKLF_CNT];
+   u16  ssow_msixoff[MAX_RVU_BLKLF_CNT];
+   u16  timlf_msixoff[MAX_RVU_BLKLF_CNT];
+   u16  cptlf_msixoff[MAX_RVU_BLKLF_CNT];
+};
+
 #endif /* MBOX_H */
diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 39dc45d..8ac3524 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -24,6 +24,11 @@
 
 static int rvu_get_hwvf(struct rvu *rvu, int pcifunc);
 
+static void rvu_set_msix_offset(struct rvu *rvu, struct rvu_pfvf *pfvf,
+   struct rvu_block *block, int lf);
+static void rvu_clear_msix_offset(struct rvu *rvu, struct rvu_pfvf *pfvf,
+ struct rvu_block *block, int lf);
+
 /* Supported devices */
 static const struct pci_device_id rvu_id_table[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_RVU_AF) },
@@ -75,6 +80,45 @@ int rvu_alloc_rsrc(struct rsrc_bmap *rsrc)
return id;
 }
 
+static int rvu_alloc_rsrc_contig(struct rsrc_bmap *rsrc, int nrsrc)
+{
+   int start;
+
+   if (!rsrc->bmap)
+   return -EINVAL;
+
+   start = bitmap_find_next_zero_area(rsrc->bmap, rsrc->max, 0, nrsrc, 0);
+   if (start >= rsrc->max)
+   return -ENOSPC;
+
+   bitmap_set(rsrc->bmap, start, nrsrc);
+   return start;
+}
+
+static void rvu_free_rsrc_contig(struct rsrc_bmap *rsrc, int nrsrc, int start)
+{
+   if (!rsrc->bmap)
+   return;
+   if (start >= rsrc->max)
+   return;
+
+   bitmap_clear(rsrc->bmap, start, nrsrc);
+}
+
+static bool rvu_rsrc_check_contig(struct rsrc_bmap *rsrc, int nrsrc)
+{
+   int start;
+
+   if (!rsrc->bmap)
+   return false;
+
+   start = bitmap_find_next_zero_area(rsrc->bmap, rsrc->max, 0, nrsrc, 0);
+   if (start >= rsrc->max)
+   return false;
+
+   return true;
+}
+
 void rvu_free_rsrc(struct rsrc_bmap *rsrc, int id)
 {
if (!rsrc->bmap)
@@ -103,6 +147,26 @@ int rvu_alloc_bitmap(struct rsrc_bmap *rsrc)
return 0;
 }
 
+/* Get block LF's HW index from a PF_FUNC's block slot number */
+int rvu_get_lf(struct rvu *rvu, struct rvu_block *block, u16 pcifunc, u16 slot)
+{
+   int lf;
+   u16 match = 0;
+
+   spin_lock(>rsrc_lock);
+   for (lf = 0; lf < block->lf.max; lf++) {
+   if (block->fn_map[lf] == pcifunc) {
+   if (slot == match) {
+   spin_unlock(>rsrc_lock);
+   return lf;
+   }
+   match++;
+   }
+   }
+   spin_unlock(>rsrc_lock);
+   return -ENODEV;
+}
+
 /* Convert BLOCK_TYPE_E to a BLOCK_ADDR_E.
  * Some silicon variants of OcteonTX2 supports
  * multiple blocks of same type.
@@ -237,6 +301,16 @@ inline int rvu_get_pf(u16 pcifunc)
return (pcifunc >> RVU_PFVF_PF_SHIFT) & RVU_PFVF_PF_MASK;
 }
 
+void rvu_get_pf_numvfs(struct rvu *rvu, int pf, int *numvfs, int *hwvf)
+{
+   u64 cfg;
+
+   /* Get numVFs attached to this 

[PATCH 08/15] soc: octeontx2: Add RVU block LF provisioning support

[sunil . kovvuri]

From: Sunil Goutham 

Added support for a RVU PF/VF to request AF via mailbox
to attach or detach NPA/NIX/SSO/SSOW/TIM/CPT block LFs.
Also supports partial detachment and modifying current
LF attached count of a certian block type.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/mbox.h|  45 ++-
 drivers/soc/marvell/octeontx2/rvu.c | 472 +++-
 drivers/soc/marvell/octeontx2/rvu.h |   8 +-
 drivers/soc/marvell/octeontx2/rvu_reg.h |   8 +-
 4 files changed, 523 insertions(+), 10 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/mbox.h 
b/drivers/soc/marvell/octeontx2/mbox.h
index fc593f0..7280d49 100644
--- a/drivers/soc/marvell/octeontx2/mbox.h
+++ b/drivers/soc/marvell/octeontx2/mbox.h
@@ -118,7 +118,17 @@ static inline struct mbox_msghdr 
*otx2_mbox_alloc_msg(struct otx2_mbox *mbox,
 #define MBOX_MSG_MAX   0x
 
 #define MBOX_MESSAGES  \
-M(READY,   0x001, msg_req, ready_msg_rsp)
+/* Generic mbox IDs (range 0x000 - 0x1FF) */   \
+M(READY,   0x001, msg_req, ready_msg_rsp)  \
+M(ATTACH_RESOURCES,0x002, rsrc_attach, msg_rsp)\
+M(DETACH_RESOURCES,0x003, rsrc_detach, msg_rsp)\
+/* CGX mbox IDs (range 0x200 - 0x3FF) */   \
+/* NPA mbox IDs (range 0x400 - 0x5FF) */   \
+/* SSO/SSOW mbox IDs (range 0x600 - 0x7FF) */  \
+/* TIM mbox IDs (range 0x800 - 0x9FF) */   \
+/* CPT mbox IDs (range 0xA00 - 0xBFF) */   \
+/* NPC mbox IDs (range 0x6000 - 0x7FFF) */ \
+/* NIX mbox IDs (range 0x8000 - 0x) */ \
 
 enum {
 #define M(_name, _id, _1, _2) MBOX_MSG_ ## _name = _id,
@@ -147,4 +157,37 @@ struct ready_msg_rsp {
u16sclk_feq;/* SCLK frequency */
 };
 
+/* Structure for requesting resource provisioning.
+ * 'modify' flag to be used when either requesting more
+ * or to detach partial of a cetain resource type.
+ * Rest of the fields specify how many of what type to
+ * be attached.
+ */
+struct rsrc_attach {
+   struct mbox_msghdr hdr;
+   u8   modify:1;
+   u8   npalf:1;
+   u8   nixlf:1;
+   u16  sso;
+   u16  ssow;
+   u16  timlfs;
+   u16  cptlfs;
+};
+
+/* Structure for relinquishing resources.
+ * 'partial' flag to be used when relinquishing all resources
+ * but only of a certain type. If not set, all resources of all
+ * types provisioned to the RVU function will be detached.
+ */
+struct rsrc_detach {
+   struct mbox_msghdr hdr;
+   u8 partial:1;
+   u8 npalf:1;
+   u8 nixlf:1;
+   u8 sso:1;
+   u8 ssow:1;
+   u8 timlfs:1;
+   u8 cptlfs:1;
+};
+
 #endif /* MBOX_H */
diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 9539ab9..39dc45d 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -59,6 +59,41 @@ int rvu_poll_reg(struct rvu *rvu, u64 block, u64 offset, u64 
mask, bool zero)
return -EBUSY;
 }
 
+int rvu_alloc_rsrc(struct rsrc_bmap *rsrc)
+{
+   int id;
+
+   if (!rsrc->bmap)
+   return -EINVAL;
+
+   id = find_first_zero_bit(rsrc->bmap, rsrc->max);
+   if (id >= rsrc->max)
+   return -ENOSPC;
+
+   __set_bit(id, rsrc->bmap);
+
+   return id;
+}
+
+void rvu_free_rsrc(struct rsrc_bmap *rsrc, int id)
+{
+   if (!rsrc->bmap)
+   return;
+
+   __clear_bit(id, rsrc->bmap);
+}
+
+int rvu_rsrc_free_count(struct rsrc_bmap *rsrc)
+{
+   int used;
+
+   if (!rsrc->bmap)
+   return 0;
+
+   used = bitmap_weight(rsrc->bmap, rsrc->max);
+   return (rsrc->max - used);
+}
+
 int rvu_alloc_bitmap(struct rsrc_bmap *rsrc)
 {
rsrc->bmap = kcalloc(BITS_TO_LONGS(rsrc->max),
@@ -68,6 +103,78 @@ int rvu_alloc_bitmap(struct rsrc_bmap *rsrc)
return 0;
 }
 
+/* Convert BLOCK_TYPE_E to a BLOCK_ADDR_E.
+ * Some silicon variants of OcteonTX2 supports
+ * multiple blocks of same type.
+ *
+ * @pcifunc has to be zero when no LF is yet attached.
+ */
+int rvu_get_blkaddr(struct rvu *rvu, int blktype, u16 pcifunc)
+{
+   int devnum, blkaddr = -ENODEV;
+   u64 cfg, reg;
+   bool is_pf;
+
+   switch (blktype) {
+   case BLKTYPE_NPA:
+   blkaddr = BLKADDR_NPA;
+   goto exit;
+   case BLKTYPE_NIX:
+   /* For now assume NIX0 */
+   if (!pcifunc) {
+   blkaddr = BLKADDR_NIX0;
+   goto exit;
+   }
+   break;
+   case BLKTYPE_SSO:
+   blkaddr = BLKADDR_SSO;
+   goto exit;
+   case BLKTYPE_SSOW:
+   blkaddr = BLKADDR_SSOW;
+   goto exit;
+  

[PATCH 06/15] soc: octeontx2: Convert mbox msg id check to a macro

[sunil . kovvuri]

From: Aleksey Makarov 

With 10's of mailbox messages expected to be handled in future,
checking for message id could become a lengthy switch case. Hence
added a macro to auto generate the switch case for each msg id.

Signed-off-by: Aleksey Makarov 
---
 drivers/soc/marvell/octeontx2/rvu.c | 44 -
 1 file changed, 38 insertions(+), 6 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index e795c2f..25f79bf 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -258,6 +258,12 @@ static int rvu_setup_hw_resources(struct rvu *rvu)
return 0;
 }
 
+static int rvu_mbox_handler_READY(struct rvu *rvu, struct msg_req *req,
+ struct ready_msg_rsp *rsp)
+{
+   return 0;
+}
+
 static int rvu_process_mbox_msg(struct rvu *rvu, int devid,
struct mbox_msghdr *req)
 {
@@ -265,13 +271,39 @@ static int rvu_process_mbox_msg(struct rvu *rvu, int 
devid,
if (req->sig != OTX2_MBOX_REQ_SIG)
goto bad_message;
 
-   if (req->id == MBOX_MSG_READY)
-   return 0;
-
+   switch (req->id) {
+#define M(_name, _id, _req_type, _rsp_type)\
+   case _id: { \
+   struct _rsp_type *rsp;  \
+   int err;\
+   \
+   rsp = (struct _rsp_type *)otx2_mbox_alloc_msg(  \
+   >mbox, devid,  \
+   sizeof(struct _rsp_type));  \
+   if (rsp) {  \
+   rsp->hdr.id = _id;  \
+   rsp->hdr.sig = OTX2_MBOX_RSP_SIG;   \
+   rsp->hdr.pcifunc = req->pcifunc;\
+   rsp->hdr.rc = 0;\
+   }   \
+   \
+   err = rvu_mbox_handler_ ## _name(rvu,   \
+(struct _req_type *)req, \
+rsp);  \
+   if (rsp && err) \
+   rsp->hdr.rc = err;  \
+   \
+   return rsp ? err : -ENOMEM; \
+   }
+MBOX_MESSAGES
+#undef M
+   break;
 bad_message:
-   otx2_reply_invalid_msg(>mbox, devid, req->pcifunc,
-  req->id);
-   return -ENODEV;
+   default:
+   otx2_reply_invalid_msg(>mbox, devid, req->pcifunc,
+  req->id);
+   return -ENODEV;
+   }
 }
 
 static void rvu_mbox_handler(struct work_struct *work)
-- 
2.7.4

[PATCH 13/15] soc: octeontx2: Add support for CGX link management

[sunil . kovvuri]

From: Linu Cherian 

CGX LMAC initialization, link status polling etc is done
by low level secure firmware. For link management this patch
adds a interface or communication mechanism between firmware
and this kernel CGX driver.

- Firmware interface specification is defined in cgx_fw_if.h.
- Support to send/receive commands/events to/form firmware.
- events/commands implemented
  * link up
  * link down
  * reading firmware version

Signed-off-by: Linu Cherian 
Signed-off-by: Nithya Mani 
---
 drivers/soc/marvell/octeontx2/cgx.c   | 361 +-
 drivers/soc/marvell/octeontx2/cgx.h   |  32 +++
 drivers/soc/marvell/octeontx2/cgx_fw_if.h | 225 +++
 3 files changed, 614 insertions(+), 4 deletions(-)
 create mode 100644 drivers/soc/marvell/octeontx2/cgx_fw_if.h

diff --git a/drivers/soc/marvell/octeontx2/cgx.c 
b/drivers/soc/marvell/octeontx2/cgx.c
index 70ca2e2f1..811422f 100644
--- a/drivers/soc/marvell/octeontx2/cgx.c
+++ b/drivers/soc/marvell/octeontx2/cgx.c
@@ -25,16 +25,43 @@
 #define DRV_STRING  "Marvell OcteonTX2 CGX/MAC Driver"
 #define DRV_VERSION"1.0"
 
+/**
+ * struct lmac
+ * @wq_cmd_cmplt:  waitq to keep the process blocked until cmd completion
+ * @cmd_lock:  Lock to serialize the command interface
+ * @resp:  command response
+ * @event_cb:  callback for linkchange events
+ * @cmd_pend:  flag set before new command is started
+ * flag cleared after command response is received
+ * @cgx:   parent cgx port
+ * @lmac_id:   lmac port id
+ * @name:  lmac port name
+ */
+struct lmac {
+   wait_queue_head_t wq_cmd_cmplt;
+   struct mutex cmd_lock;
+   struct cgx_evt_sts resp;
+   struct cgx_event_cb event_cb;
+   bool cmd_pend;
+   struct cgx *cgx;
+   u8 lmac_id;
+   char *name;
+};
+
 struct cgx {
void __iomem*reg_base;
struct pci_dev  *pdev;
u8  cgx_id;
u8  lmac_count;
+   struct lmac *lmac_idmap[MAX_LMAC_PER_CGX];
struct list_headcgx_list;
 };
 
 static LIST_HEAD(cgx_list);
 
+/* CGX PHY management internal APIs */
+static int cgx_fwi_link_change(struct cgx *cgx, int lmac_id, bool en);
+
 /* Supported devices */
 static const struct pci_device_id cgx_id_table[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_CGX) },
@@ -47,11 +74,24 @@ MODULE_LICENSE("GPL v2");
 MODULE_VERSION(DRV_VERSION);
 MODULE_DEVICE_TABLE(pci, cgx_id_table);
 
+static void cgx_write(struct cgx *cgx, u64 lmac, u64 offset, u64 val)
+{
+   writeq(val, cgx->reg_base + (lmac << 18) + offset);
+}
+
 static u64 cgx_read(struct cgx *cgx, u64 lmac, u64 offset)
 {
return readq(cgx->reg_base + (lmac << 18) + offset);
 }
 
+static inline struct lmac *lmac_pdata(u8 lmac_id, struct cgx *cgx)
+{
+   if (!cgx || lmac_id >= MAX_LMAC_PER_CGX)
+   return NULL;
+
+   return cgx->lmac_idmap[lmac_id];
+}
+
 int cgx_get_cgx_cnt(void)
 {
struct cgx *cgx_dev;
@@ -87,18 +127,315 @@ void *cgx_get_pdata(int cgx_id)
 }
 EXPORT_SYMBOL(cgx_get_pdata);
 
-static void cgx_lmac_init(struct cgx *cgx)
+/* CGX Firmware interface low level support */
+static int cgx_fwi_cmd_send(struct cgx_cmd *cmd, struct cgx_evt_sts *rsp,
+   struct lmac *lmac)
+{
+   struct cgx *cgx = lmac->cgx;
+   union cgx_cmdreg creg;
+   union cgx_evtreg ereg;
+   struct device *dev;
+   int err = 0;
+
+   /* Ensure no other command is in progress */
+   err = mutex_lock_interruptible(>cmd_lock);
+   if (err)
+   return err;
+
+   /* Ensure command register is free */
+   creg.val = cgx_read(cgx, lmac->lmac_id,  CGX_COMMAND_REG);
+   if (creg.cmd.own != CGX_CMD_OWN_NS) {
+   err = -EBUSY;
+   goto unlock;
+   }
+
+   /* Update ownership in command request */
+   cmd->own = CGX_CMD_OWN_FIRMWARE;
+
+   /* Mark this lmac as pending, before we start */
+   lmac->cmd_pend = true;
+
+   /* Start command in hardware */
+   creg.cmd = *cmd;
+   cgx_write(cgx, lmac->lmac_id, CGX_COMMAND_REG, creg.val);
+   creg.val = cgx_read(cgx, lmac->lmac_id,  CGX_COMMAND_REG);
+
+   /* Ensure command is completed without errors */
+   if (!wait_event_timeout(lmac->wq_cmd_cmplt, !lmac->cmd_pend,
+   msecs_to_jiffies(CGX_CMD_TIMEOUT))) {
+   dev = >pdev->dev;
+   ereg.val = cgx_read(cgx, lmac->lmac_id,  CGX_EVENT_REG);
+   if (ereg.val) {
+   dev_err(dev, "cgx port %d:%d: No event for response\n",
+   cgx->cgx_id, lmac->lmac_id);
+   /* copy event */
+   lmac->resp = ereg.evt_sts;
+   } else {
+   dev_err(dev, "cgx port %d:%d cmd 

[PATCH 15/15] MAINTAINERS: Add entry for Marvell OcteonTX2 Admin Function driver

[sunil . kovvuri]

From: Sunil Goutham 

Added maintainers entry for Marvell OcteonTX2 SOC's RVU
admin function driver.

Signed-off-by: Sunil Goutham 
---
 MAINTAINERS | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 8bef28b..99ef6c1 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8821,6 +8821,16 @@ S:   Supported
 F: drivers/mmc/host/sdhci-xenon*
 F: Documentation/devicetree/bindings/mmc/marvell,xenon-sdhci.txt
 
+MARVELL OCTEONTX2 RVU ADMIN FUNCTION DRIVER
+M: Sunil Goutham 
+M: Linu Cherian 
+M: Geetha sowjanya 
+M: Jerin Jacob 
+L: linux-kernel@vger.kernel.org
+L: linux-arm-ker...@lists.infradead.org (moderated for non-subscribers)
+S: Maintained
+F: drivers/soc/marvell/octeontx2
+
 MATROX FRAMEBUFFER DRIVER
 L: linux-fb...@vger.kernel.org
 S: Orphan
-- 
2.7.4

[PATCH 10/15] soc: octeontx2: Reconfig MSIX base with IOVA

[sunil . kovvuri]

From: Geetha sowjanya 

HW interprets RVU_AF_MSIXTR_BASE address as an IOVA, hence
create a IOMMU mapping for the physcial address configured by
firmware and reconfig RVU_AF_MSIXTR_BASE with IOVA.

Signed-off-by: Geetha sowjanya 
Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/rvu.c | 33 ++---
 drivers/soc/marvell/octeontx2/rvu.h |  1 +
 2 files changed, 31 insertions(+), 3 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 8ac3524..40684c9 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -442,9 +442,10 @@ static int rvu_setup_msix_resources(struct rvu *rvu)
 {
struct rvu_hwinfo *hw = rvu->hw;
int pf, vf, numvfs, hwvf, err;
+   int nvecs, offset, max_msix;
struct rvu_pfvf *pfvf;
-   int nvecs, offset;
-   u64 cfg;
+   u64 cfg, phy_addr;
+   dma_addr_t iova;
 
for (pf = 0; pf < hw->total_pfs; pf++) {
cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_PFX_CFG(pf));
@@ -523,6 +524,22 @@ static int rvu_setup_msix_resources(struct rvu *rvu)
}
}
 
+   /* HW interprets RVU_AF_MSIXTR_BASE address as an IOVA, hence
+* create a IOMMU mapping for the physcial address configured by
+* firmware and reconfig RVU_AF_MSIXTR_BASE with IOVA.
+*/
+   cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_CONST);
+   max_msix = cfg & 0xF;
+   phy_addr = rvu_read64(rvu, BLKADDR_RVUM, RVU_AF_MSIXTR_BASE);
+   iova = dma_map_single(rvu->dev, (void *)phy_addr,
+ max_msix * PCI_MSIX_ENTRY_SIZE,
+ DMA_BIDIRECTIONAL);
+   if (dma_mapping_error(rvu->dev, iova))
+   return -ENOMEM;
+
+   rvu_write64(rvu, BLKADDR_RVUM, RVU_AF_MSIXTR_BASE, (u64)iova);
+   rvu->msix_base_iova = iova;
+
return 0;
 }
 
@@ -531,7 +548,8 @@ static void rvu_free_hw_resources(struct rvu *rvu)
struct rvu_hwinfo *hw = rvu->hw;
struct rvu_block *block;
struct rvu_pfvf  *pfvf;
-   int id;
+   int id, max_msix;
+   u64 cfg;
 
/* Free block LF bitmaps */
for (id = 0; id < BLK_COUNT; id++) {
@@ -549,6 +567,15 @@ static void rvu_free_hw_resources(struct rvu *rvu)
pfvf = >hwvf[id];
kfree(pfvf->msix.bmap);
}
+
+   /* Unmap MSIX vector base IOVA mapping */
+   if (!rvu->msix_base_iova)
+   return;
+   cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_CONST);
+   max_msix = cfg & 0xF;
+   dma_unmap_single(rvu->dev, rvu->msix_base_iova,
+max_msix * PCI_MSIX_ENTRY_SIZE,
+DMA_BIDIRECTIONAL);
 }
 
 static int rvu_setup_hw_resources(struct rvu *rvu)
diff --git a/drivers/soc/marvell/octeontx2/rvu.h 
b/drivers/soc/marvell/octeontx2/rvu.h
index 7435e83..92c2022 100644
--- a/drivers/soc/marvell/octeontx2/rvu.h
+++ b/drivers/soc/marvell/octeontx2/rvu.h
@@ -99,6 +99,7 @@ struct rvu {
u16 num_vec;
char*irq_name;
bool*irq_allocated;
+   dma_addr_t  msix_base_iova;
 };
 
 static inline void rvu_write64(struct rvu *rvu, u64 block, u64 offset, u64 val)
-- 
2.7.4

[PATCH 09/15] soc: octeontx2: Configure block LF's MSIX vector offset

[sunil . kovvuri]

From: Sunil Goutham 

Firmware configures a certain number of MSIX vectors to each of
enabled RVU PF/VF. When a block LF is attached to a PF/VF, number
of MSIX vectors needed by that LF are set aside (out of PF/VF's
total MSIX vectors) and LF's msix_offset is configured in HW.

Also added support for a RVU PF/VF to retrieve that block LF's
MSIX vector offset information from AF via mbox.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/mbox.h   |  18 ++
 drivers/soc/marvell/octeontx2/rvu.c| 333 -
 drivers/soc/marvell/octeontx2/rvu.h|   7 +
 drivers/soc/marvell/octeontx2/rvu_struct.h |   2 +
 4 files changed, 357 insertions(+), 3 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/mbox.h 
b/drivers/soc/marvell/octeontx2/mbox.h
index 7280d49..bedf0ee 100644
--- a/drivers/soc/marvell/octeontx2/mbox.h
+++ b/drivers/soc/marvell/octeontx2/mbox.h
@@ -122,6 +122,7 @@ static inline struct mbox_msghdr 
*otx2_mbox_alloc_msg(struct otx2_mbox *mbox,
 M(READY,   0x001, msg_req, ready_msg_rsp)  \
 M(ATTACH_RESOURCES,0x002, rsrc_attach, msg_rsp)\
 M(DETACH_RESOURCES,0x003, rsrc_detach, msg_rsp)\
+M(MSIX_OFFSET, 0x004, msg_req, msix_offset_rsp)\
 /* CGX mbox IDs (range 0x200 - 0x3FF) */   \
 /* NPA mbox IDs (range 0x400 - 0x5FF) */   \
 /* SSO/SSOW mbox IDs (range 0x600 - 0x7FF) */  \
@@ -190,4 +191,21 @@ struct rsrc_detach {
u8 cptlfs:1;
 };
 
+#define MSIX_VECTOR_INVALID0x
+#define MAX_RVU_BLKLF_CNT  256
+
+struct msix_offset_rsp {
+   struct mbox_msghdr hdr;
+   u16  npa_msixoff;
+   u16  nix_msixoff;
+   u8   sso;
+   u8   ssow;
+   u8   timlfs;
+   u8   cptlfs;
+   u16  sso_msixoff[MAX_RVU_BLKLF_CNT];
+   u16  ssow_msixoff[MAX_RVU_BLKLF_CNT];
+   u16  timlf_msixoff[MAX_RVU_BLKLF_CNT];
+   u16  cptlf_msixoff[MAX_RVU_BLKLF_CNT];
+};
+
 #endif /* MBOX_H */
diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 39dc45d..8ac3524 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -24,6 +24,11 @@
 
 static int rvu_get_hwvf(struct rvu *rvu, int pcifunc);
 
+static void rvu_set_msix_offset(struct rvu *rvu, struct rvu_pfvf *pfvf,
+   struct rvu_block *block, int lf);
+static void rvu_clear_msix_offset(struct rvu *rvu, struct rvu_pfvf *pfvf,
+ struct rvu_block *block, int lf);
+
 /* Supported devices */
 static const struct pci_device_id rvu_id_table[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_RVU_AF) },
@@ -75,6 +80,45 @@ int rvu_alloc_rsrc(struct rsrc_bmap *rsrc)
return id;
 }
 
+static int rvu_alloc_rsrc_contig(struct rsrc_bmap *rsrc, int nrsrc)
+{
+   int start;
+
+   if (!rsrc->bmap)
+   return -EINVAL;
+
+   start = bitmap_find_next_zero_area(rsrc->bmap, rsrc->max, 0, nrsrc, 0);
+   if (start >= rsrc->max)
+   return -ENOSPC;
+
+   bitmap_set(rsrc->bmap, start, nrsrc);
+   return start;
+}
+
+static void rvu_free_rsrc_contig(struct rsrc_bmap *rsrc, int nrsrc, int start)
+{
+   if (!rsrc->bmap)
+   return;
+   if (start >= rsrc->max)
+   return;
+
+   bitmap_clear(rsrc->bmap, start, nrsrc);
+}
+
+static bool rvu_rsrc_check_contig(struct rsrc_bmap *rsrc, int nrsrc)
+{
+   int start;
+
+   if (!rsrc->bmap)
+   return false;
+
+   start = bitmap_find_next_zero_area(rsrc->bmap, rsrc->max, 0, nrsrc, 0);
+   if (start >= rsrc->max)
+   return false;
+
+   return true;
+}
+
 void rvu_free_rsrc(struct rsrc_bmap *rsrc, int id)
 {
if (!rsrc->bmap)
@@ -103,6 +147,26 @@ int rvu_alloc_bitmap(struct rsrc_bmap *rsrc)
return 0;
 }
 
+/* Get block LF's HW index from a PF_FUNC's block slot number */
+int rvu_get_lf(struct rvu *rvu, struct rvu_block *block, u16 pcifunc, u16 slot)
+{
+   int lf;
+   u16 match = 0;
+
+   spin_lock(>rsrc_lock);
+   for (lf = 0; lf < block->lf.max; lf++) {
+   if (block->fn_map[lf] == pcifunc) {
+   if (slot == match) {
+   spin_unlock(>rsrc_lock);
+   return lf;
+   }
+   match++;
+   }
+   }
+   spin_unlock(>rsrc_lock);
+   return -ENODEV;
+}
+
 /* Convert BLOCK_TYPE_E to a BLOCK_ADDR_E.
  * Some silicon variants of OcteonTX2 supports
  * multiple blocks of same type.
@@ -237,6 +301,16 @@ inline int rvu_get_pf(u16 pcifunc)
return (pcifunc >> RVU_PFVF_PF_SHIFT) & RVU_PFVF_PF_MASK;
 }
 
+void rvu_get_pf_numvfs(struct rvu *rvu, int pf, int *numvfs, int *hwvf)
+{
+   u64 cfg;
+
+   /* Get numVFs attached to this 

[PATCH 08/15] soc: octeontx2: Add RVU block LF provisioning support

[sunil . kovvuri]

From: Sunil Goutham 

Added support for a RVU PF/VF to request AF via mailbox
to attach or detach NPA/NIX/SSO/SSOW/TIM/CPT block LFs.
Also supports partial detachment and modifying current
LF attached count of a certian block type.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/mbox.h|  45 ++-
 drivers/soc/marvell/octeontx2/rvu.c | 472 +++-
 drivers/soc/marvell/octeontx2/rvu.h |   8 +-
 drivers/soc/marvell/octeontx2/rvu_reg.h |   8 +-
 4 files changed, 523 insertions(+), 10 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/mbox.h 
b/drivers/soc/marvell/octeontx2/mbox.h
index fc593f0..7280d49 100644
--- a/drivers/soc/marvell/octeontx2/mbox.h
+++ b/drivers/soc/marvell/octeontx2/mbox.h
@@ -118,7 +118,17 @@ static inline struct mbox_msghdr 
*otx2_mbox_alloc_msg(struct otx2_mbox *mbox,
 #define MBOX_MSG_MAX   0x
 
 #define MBOX_MESSAGES  \
-M(READY,   0x001, msg_req, ready_msg_rsp)
+/* Generic mbox IDs (range 0x000 - 0x1FF) */   \
+M(READY,   0x001, msg_req, ready_msg_rsp)  \
+M(ATTACH_RESOURCES,0x002, rsrc_attach, msg_rsp)\
+M(DETACH_RESOURCES,0x003, rsrc_detach, msg_rsp)\
+/* CGX mbox IDs (range 0x200 - 0x3FF) */   \
+/* NPA mbox IDs (range 0x400 - 0x5FF) */   \
+/* SSO/SSOW mbox IDs (range 0x600 - 0x7FF) */  \
+/* TIM mbox IDs (range 0x800 - 0x9FF) */   \
+/* CPT mbox IDs (range 0xA00 - 0xBFF) */   \
+/* NPC mbox IDs (range 0x6000 - 0x7FFF) */ \
+/* NIX mbox IDs (range 0x8000 - 0x) */ \
 
 enum {
 #define M(_name, _id, _1, _2) MBOX_MSG_ ## _name = _id,
@@ -147,4 +157,37 @@ struct ready_msg_rsp {
u16sclk_feq;/* SCLK frequency */
 };
 
+/* Structure for requesting resource provisioning.
+ * 'modify' flag to be used when either requesting more
+ * or to detach partial of a cetain resource type.
+ * Rest of the fields specify how many of what type to
+ * be attached.
+ */
+struct rsrc_attach {
+   struct mbox_msghdr hdr;
+   u8   modify:1;
+   u8   npalf:1;
+   u8   nixlf:1;
+   u16  sso;
+   u16  ssow;
+   u16  timlfs;
+   u16  cptlfs;
+};
+
+/* Structure for relinquishing resources.
+ * 'partial' flag to be used when relinquishing all resources
+ * but only of a certain type. If not set, all resources of all
+ * types provisioned to the RVU function will be detached.
+ */
+struct rsrc_detach {
+   struct mbox_msghdr hdr;
+   u8 partial:1;
+   u8 npalf:1;
+   u8 nixlf:1;
+   u8 sso:1;
+   u8 ssow:1;
+   u8 timlfs:1;
+   u8 cptlfs:1;
+};
+
 #endif /* MBOX_H */
diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 9539ab9..39dc45d 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -59,6 +59,41 @@ int rvu_poll_reg(struct rvu *rvu, u64 block, u64 offset, u64 
mask, bool zero)
return -EBUSY;
 }
 
+int rvu_alloc_rsrc(struct rsrc_bmap *rsrc)
+{
+   int id;
+
+   if (!rsrc->bmap)
+   return -EINVAL;
+
+   id = find_first_zero_bit(rsrc->bmap, rsrc->max);
+   if (id >= rsrc->max)
+   return -ENOSPC;
+
+   __set_bit(id, rsrc->bmap);
+
+   return id;
+}
+
+void rvu_free_rsrc(struct rsrc_bmap *rsrc, int id)
+{
+   if (!rsrc->bmap)
+   return;
+
+   __clear_bit(id, rsrc->bmap);
+}
+
+int rvu_rsrc_free_count(struct rsrc_bmap *rsrc)
+{
+   int used;
+
+   if (!rsrc->bmap)
+   return 0;
+
+   used = bitmap_weight(rsrc->bmap, rsrc->max);
+   return (rsrc->max - used);
+}
+
 int rvu_alloc_bitmap(struct rsrc_bmap *rsrc)
 {
rsrc->bmap = kcalloc(BITS_TO_LONGS(rsrc->max),
@@ -68,6 +103,78 @@ int rvu_alloc_bitmap(struct rsrc_bmap *rsrc)
return 0;
 }
 
+/* Convert BLOCK_TYPE_E to a BLOCK_ADDR_E.
+ * Some silicon variants of OcteonTX2 supports
+ * multiple blocks of same type.
+ *
+ * @pcifunc has to be zero when no LF is yet attached.
+ */
+int rvu_get_blkaddr(struct rvu *rvu, int blktype, u16 pcifunc)
+{
+   int devnum, blkaddr = -ENODEV;
+   u64 cfg, reg;
+   bool is_pf;
+
+   switch (blktype) {
+   case BLKTYPE_NPA:
+   blkaddr = BLKADDR_NPA;
+   goto exit;
+   case BLKTYPE_NIX:
+   /* For now assume NIX0 */
+   if (!pcifunc) {
+   blkaddr = BLKADDR_NIX0;
+   goto exit;
+   }
+   break;
+   case BLKTYPE_SSO:
+   blkaddr = BLKADDR_SSO;
+   goto exit;
+   case BLKTYPE_SSOW:
+   blkaddr = BLKADDR_SSOW;
+   goto exit;
+  

[PATCH 06/15] soc: octeontx2: Convert mbox msg id check to a macro

[sunil . kovvuri]

From: Aleksey Makarov 

With 10's of mailbox messages expected to be handled in future,
checking for message id could become a lengthy switch case. Hence
added a macro to auto generate the switch case for each msg id.

Signed-off-by: Aleksey Makarov 
---
 drivers/soc/marvell/octeontx2/rvu.c | 44 -
 1 file changed, 38 insertions(+), 6 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index e795c2f..25f79bf 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -258,6 +258,12 @@ static int rvu_setup_hw_resources(struct rvu *rvu)
return 0;
 }
 
+static int rvu_mbox_handler_READY(struct rvu *rvu, struct msg_req *req,
+ struct ready_msg_rsp *rsp)
+{
+   return 0;
+}
+
 static int rvu_process_mbox_msg(struct rvu *rvu, int devid,
struct mbox_msghdr *req)
 {
@@ -265,13 +271,39 @@ static int rvu_process_mbox_msg(struct rvu *rvu, int 
devid,
if (req->sig != OTX2_MBOX_REQ_SIG)
goto bad_message;
 
-   if (req->id == MBOX_MSG_READY)
-   return 0;
-
+   switch (req->id) {
+#define M(_name, _id, _req_type, _rsp_type)\
+   case _id: { \
+   struct _rsp_type *rsp;  \
+   int err;\
+   \
+   rsp = (struct _rsp_type *)otx2_mbox_alloc_msg(  \
+   >mbox, devid,  \
+   sizeof(struct _rsp_type));  \
+   if (rsp) {  \
+   rsp->hdr.id = _id;  \
+   rsp->hdr.sig = OTX2_MBOX_RSP_SIG;   \
+   rsp->hdr.pcifunc = req->pcifunc;\
+   rsp->hdr.rc = 0;\
+   }   \
+   \
+   err = rvu_mbox_handler_ ## _name(rvu,   \
+(struct _req_type *)req, \
+rsp);  \
+   if (rsp && err) \
+   rsp->hdr.rc = err;  \
+   \
+   return rsp ? err : -ENOMEM; \
+   }
+MBOX_MESSAGES
+#undef M
+   break;
 bad_message:
-   otx2_reply_invalid_msg(>mbox, devid, req->pcifunc,
-  req->id);
-   return -ENODEV;
+   default:
+   otx2_reply_invalid_msg(>mbox, devid, req->pcifunc,
+  req->id);
+   return -ENODEV;
+   }
 }
 
 static void rvu_mbox_handler(struct work_struct *work)
-- 
2.7.4

[PATCH 13/15] soc: octeontx2: Add support for CGX link management

[sunil . kovvuri]

From: Linu Cherian 

CGX LMAC initialization, link status polling etc is done
by low level secure firmware. For link management this patch
adds a interface or communication mechanism between firmware
and this kernel CGX driver.

- Firmware interface specification is defined in cgx_fw_if.h.
- Support to send/receive commands/events to/form firmware.
- events/commands implemented
  * link up
  * link down
  * reading firmware version

Signed-off-by: Linu Cherian 
Signed-off-by: Nithya Mani 
---
 drivers/soc/marvell/octeontx2/cgx.c   | 361 +-
 drivers/soc/marvell/octeontx2/cgx.h   |  32 +++
 drivers/soc/marvell/octeontx2/cgx_fw_if.h | 225 +++
 3 files changed, 614 insertions(+), 4 deletions(-)
 create mode 100644 drivers/soc/marvell/octeontx2/cgx_fw_if.h

diff --git a/drivers/soc/marvell/octeontx2/cgx.c 
b/drivers/soc/marvell/octeontx2/cgx.c
index 70ca2e2f1..811422f 100644
--- a/drivers/soc/marvell/octeontx2/cgx.c
+++ b/drivers/soc/marvell/octeontx2/cgx.c
@@ -25,16 +25,43 @@
 #define DRV_STRING  "Marvell OcteonTX2 CGX/MAC Driver"
 #define DRV_VERSION"1.0"
 
+/**
+ * struct lmac
+ * @wq_cmd_cmplt:  waitq to keep the process blocked until cmd completion
+ * @cmd_lock:  Lock to serialize the command interface
+ * @resp:  command response
+ * @event_cb:  callback for linkchange events
+ * @cmd_pend:  flag set before new command is started
+ * flag cleared after command response is received
+ * @cgx:   parent cgx port
+ * @lmac_id:   lmac port id
+ * @name:  lmac port name
+ */
+struct lmac {
+   wait_queue_head_t wq_cmd_cmplt;
+   struct mutex cmd_lock;
+   struct cgx_evt_sts resp;
+   struct cgx_event_cb event_cb;
+   bool cmd_pend;
+   struct cgx *cgx;
+   u8 lmac_id;
+   char *name;
+};
+
 struct cgx {
void __iomem*reg_base;
struct pci_dev  *pdev;
u8  cgx_id;
u8  lmac_count;
+   struct lmac *lmac_idmap[MAX_LMAC_PER_CGX];
struct list_headcgx_list;
 };
 
 static LIST_HEAD(cgx_list);
 
+/* CGX PHY management internal APIs */
+static int cgx_fwi_link_change(struct cgx *cgx, int lmac_id, bool en);
+
 /* Supported devices */
 static const struct pci_device_id cgx_id_table[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_CGX) },
@@ -47,11 +74,24 @@ MODULE_LICENSE("GPL v2");
 MODULE_VERSION(DRV_VERSION);
 MODULE_DEVICE_TABLE(pci, cgx_id_table);
 
+static void cgx_write(struct cgx *cgx, u64 lmac, u64 offset, u64 val)
+{
+   writeq(val, cgx->reg_base + (lmac << 18) + offset);
+}
+
 static u64 cgx_read(struct cgx *cgx, u64 lmac, u64 offset)
 {
return readq(cgx->reg_base + (lmac << 18) + offset);
 }
 
+static inline struct lmac *lmac_pdata(u8 lmac_id, struct cgx *cgx)
+{
+   if (!cgx || lmac_id >= MAX_LMAC_PER_CGX)
+   return NULL;
+
+   return cgx->lmac_idmap[lmac_id];
+}
+
 int cgx_get_cgx_cnt(void)
 {
struct cgx *cgx_dev;
@@ -87,18 +127,315 @@ void *cgx_get_pdata(int cgx_id)
 }
 EXPORT_SYMBOL(cgx_get_pdata);
 
-static void cgx_lmac_init(struct cgx *cgx)
+/* CGX Firmware interface low level support */
+static int cgx_fwi_cmd_send(struct cgx_cmd *cmd, struct cgx_evt_sts *rsp,
+   struct lmac *lmac)
+{
+   struct cgx *cgx = lmac->cgx;
+   union cgx_cmdreg creg;
+   union cgx_evtreg ereg;
+   struct device *dev;
+   int err = 0;
+
+   /* Ensure no other command is in progress */
+   err = mutex_lock_interruptible(>cmd_lock);
+   if (err)
+   return err;
+
+   /* Ensure command register is free */
+   creg.val = cgx_read(cgx, lmac->lmac_id,  CGX_COMMAND_REG);
+   if (creg.cmd.own != CGX_CMD_OWN_NS) {
+   err = -EBUSY;
+   goto unlock;
+   }
+
+   /* Update ownership in command request */
+   cmd->own = CGX_CMD_OWN_FIRMWARE;
+
+   /* Mark this lmac as pending, before we start */
+   lmac->cmd_pend = true;
+
+   /* Start command in hardware */
+   creg.cmd = *cmd;
+   cgx_write(cgx, lmac->lmac_id, CGX_COMMAND_REG, creg.val);
+   creg.val = cgx_read(cgx, lmac->lmac_id,  CGX_COMMAND_REG);
+
+   /* Ensure command is completed without errors */
+   if (!wait_event_timeout(lmac->wq_cmd_cmplt, !lmac->cmd_pend,
+   msecs_to_jiffies(CGX_CMD_TIMEOUT))) {
+   dev = >pdev->dev;
+   ereg.val = cgx_read(cgx, lmac->lmac_id,  CGX_EVENT_REG);
+   if (ereg.val) {
+   dev_err(dev, "cgx port %d:%d: No event for response\n",
+   cgx->cgx_id, lmac->lmac_id);
+   /* copy event */
+   lmac->resp = ereg.evt_sts;
+   } else {
+   dev_err(dev, "cgx port %d:%d cmd 

[PATCH 15/15] MAINTAINERS: Add entry for Marvell OcteonTX2 Admin Function driver

[sunil . kovvuri]

From: Sunil Goutham 

Added maintainers entry for Marvell OcteonTX2 SOC's RVU
admin function driver.

Signed-off-by: Sunil Goutham 
---
 MAINTAINERS | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 8bef28b..99ef6c1 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8821,6 +8821,16 @@ S:   Supported
 F: drivers/mmc/host/sdhci-xenon*
 F: Documentation/devicetree/bindings/mmc/marvell,xenon-sdhci.txt
 
+MARVELL OCTEONTX2 RVU ADMIN FUNCTION DRIVER
+M: Sunil Goutham 
+M: Linu Cherian 
+M: Geetha sowjanya 
+M: Jerin Jacob 
+L: linux-kernel@vger.kernel.org
+L: linux-arm-ker...@lists.infradead.org (moderated for non-subscribers)
+S: Maintained
+F: drivers/soc/marvell/octeontx2
+
 MATROX FRAMEBUFFER DRIVER
 L: linux-fb...@vger.kernel.org
 S: Orphan
-- 
2.7.4

[PATCH 10/15] soc: octeontx2: Reconfig MSIX base with IOVA

[sunil . kovvuri]

From: Geetha sowjanya 

HW interprets RVU_AF_MSIXTR_BASE address as an IOVA, hence
create a IOMMU mapping for the physcial address configured by
firmware and reconfig RVU_AF_MSIXTR_BASE with IOVA.

Signed-off-by: Geetha sowjanya 
Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/rvu.c | 33 ++---
 drivers/soc/marvell/octeontx2/rvu.h |  1 +
 2 files changed, 31 insertions(+), 3 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 8ac3524..40684c9 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -442,9 +442,10 @@ static int rvu_setup_msix_resources(struct rvu *rvu)
 {
struct rvu_hwinfo *hw = rvu->hw;
int pf, vf, numvfs, hwvf, err;
+   int nvecs, offset, max_msix;
struct rvu_pfvf *pfvf;
-   int nvecs, offset;
-   u64 cfg;
+   u64 cfg, phy_addr;
+   dma_addr_t iova;
 
for (pf = 0; pf < hw->total_pfs; pf++) {
cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_PFX_CFG(pf));
@@ -523,6 +524,22 @@ static int rvu_setup_msix_resources(struct rvu *rvu)
}
}
 
+   /* HW interprets RVU_AF_MSIXTR_BASE address as an IOVA, hence
+* create a IOMMU mapping for the physcial address configured by
+* firmware and reconfig RVU_AF_MSIXTR_BASE with IOVA.
+*/
+   cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_CONST);
+   max_msix = cfg & 0xF;
+   phy_addr = rvu_read64(rvu, BLKADDR_RVUM, RVU_AF_MSIXTR_BASE);
+   iova = dma_map_single(rvu->dev, (void *)phy_addr,
+ max_msix * PCI_MSIX_ENTRY_SIZE,
+ DMA_BIDIRECTIONAL);
+   if (dma_mapping_error(rvu->dev, iova))
+   return -ENOMEM;
+
+   rvu_write64(rvu, BLKADDR_RVUM, RVU_AF_MSIXTR_BASE, (u64)iova);
+   rvu->msix_base_iova = iova;
+
return 0;
 }
 
@@ -531,7 +548,8 @@ static void rvu_free_hw_resources(struct rvu *rvu)
struct rvu_hwinfo *hw = rvu->hw;
struct rvu_block *block;
struct rvu_pfvf  *pfvf;
-   int id;
+   int id, max_msix;
+   u64 cfg;
 
/* Free block LF bitmaps */
for (id = 0; id < BLK_COUNT; id++) {
@@ -549,6 +567,15 @@ static void rvu_free_hw_resources(struct rvu *rvu)
pfvf = >hwvf[id];
kfree(pfvf->msix.bmap);
}
+
+   /* Unmap MSIX vector base IOVA mapping */
+   if (!rvu->msix_base_iova)
+   return;
+   cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_CONST);
+   max_msix = cfg & 0xF;
+   dma_unmap_single(rvu->dev, rvu->msix_base_iova,
+max_msix * PCI_MSIX_ENTRY_SIZE,
+DMA_BIDIRECTIONAL);
 }
 
 static int rvu_setup_hw_resources(struct rvu *rvu)
diff --git a/drivers/soc/marvell/octeontx2/rvu.h 
b/drivers/soc/marvell/octeontx2/rvu.h
index 7435e83..92c2022 100644
--- a/drivers/soc/marvell/octeontx2/rvu.h
+++ b/drivers/soc/marvell/octeontx2/rvu.h
@@ -99,6 +99,7 @@ struct rvu {
u16 num_vec;
char*irq_name;
bool*irq_allocated;
+   dma_addr_t  msix_base_iova;
 };
 
 static inline void rvu_write64(struct rvu *rvu, u64 block, u64 offset, u64 val)
-- 
2.7.4

[PATCH 14/15] soc: octeontx2: Register for CGX lmac events

[sunil . kovvuri]

From: Linu Cherian 

Added support in RVU AF driver to register for
CGX LMAC link status change events from firmware
and managing them. Processing part will be added
in followup patches.

- Introduced eventqueue for posting events from cgx lmac.
  Queueing mechanism will ensure that events can be posted
  and firmware can be acked immediately and hence event
  reception and processing are decoupled.
- Events gets added to the queue by notification callback.
  Notification callback is expected to be atomic, since it
  is called from interrupt context.
- Events are dequeued and processed in a worker thread.

Signed-off-by: Linu Cherian 
---
 drivers/soc/marvell/octeontx2/rvu.c |   6 +-
 drivers/soc/marvell/octeontx2/rvu.h |   5 ++
 drivers/soc/marvell/octeontx2/rvu_cgx.c | 101 +++-
 3 files changed, 108 insertions(+), 4 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index d7b19e0..3809517 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -1563,10 +1563,11 @@ static int rvu_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
 
err = rvu_register_interrupts(rvu);
if (err)
-   goto err_mbox;
+   goto err_cgx;
 
return 0;
-
+err_cgx:
+   rvu_cgx_wq_destroy(rvu);
 err_mbox:
rvu_mbox_destroy(rvu);
 err_hwsetup:
@@ -1588,6 +1589,7 @@ static void rvu_remove(struct pci_dev *pdev)
struct rvu *rvu = pci_get_drvdata(pdev);
 
rvu_unregister_interrupts(rvu);
+   rvu_cgx_wq_destroy(rvu);
rvu_mbox_destroy(rvu);
rvu_reset_all_blocks(rvu);
rvu_free_hw_resources(rvu);
diff --git a/drivers/soc/marvell/octeontx2/rvu.h 
b/drivers/soc/marvell/octeontx2/rvu.h
index 385f597..d169fa9 100644
--- a/drivers/soc/marvell/octeontx2/rvu.h
+++ b/drivers/soc/marvell/octeontx2/rvu.h
@@ -110,6 +110,10 @@ struct rvu {
  * every cgx lmac port
  */
void**cgx_idmap; /* cgx id to cgx data map table */
+   struct  work_struct cgx_evh_work;
+   struct  workqueue_struct *cgx_evh_wq;
+   spinlock_t  cgx_evq_lock; /* cgx event queue lock */
+   struct list_headcgx_evq_head; /* cgx event queue head */
 };
 
 static inline void rvu_write64(struct rvu *rvu, u64 block, u64 offset, u64 val)
@@ -150,4 +154,5 @@ int rvu_poll_reg(struct rvu *rvu, u64 block, u64 offset, 
u64 mask, bool zero);
 
 /* CGX APIs */
 int rvu_cgx_probe(struct rvu *rvu);
+void rvu_cgx_wq_destroy(struct rvu *rvu);
 #endif /* RVU_H */
diff --git a/drivers/soc/marvell/octeontx2/rvu_cgx.c 
b/drivers/soc/marvell/octeontx2/rvu_cgx.c
index bf81507..2359806e 100644
--- a/drivers/soc/marvell/octeontx2/rvu_cgx.c
+++ b/drivers/soc/marvell/octeontx2/rvu_cgx.c
@@ -15,6 +15,11 @@
 #include "rvu.h"
 #include "cgx.h"
 
+struct cgx_evq_entry {
+   struct list_head evq_node;
+   struct cgx_link_event link_event;
+};
+
 static inline u8 cgxlmac_id_to_bmap(u8 cgx_id, u8 lmac_id)
 {
return ((cgx_id & 0xF) << 4) | (lmac_id & 0xF);
@@ -72,9 +77,95 @@ static int rvu_map_cgx_lmac_pf(struct rvu *rvu)
return 0;
 }
 
+/* This is called from interrupt context and is expected to be atomic */
+static int cgx_lmac_postevent(struct cgx_link_event *event, void *data)
+{
+   struct rvu *rvu = data;
+   struct cgx_evq_entry *qentry;
+
+   /* post event to the event queue */
+   qentry = kmalloc(sizeof(*qentry), GFP_ATOMIC);
+   if (!qentry)
+   return -ENOMEM;
+   qentry->link_event = *event;
+   spin_lock(>cgx_evq_lock);
+   list_add_tail(>evq_node, >cgx_evq_head);
+   spin_unlock(>cgx_evq_lock);
+
+   /* start worker to process the events */
+   queue_work(rvu->cgx_evh_wq, >cgx_evh_work);
+
+   return 0;
+}
+
+static void cgx_evhandler_task(struct work_struct *work)
+{
+   struct rvu *rvu = container_of(work, struct rvu, cgx_evh_work);
+   struct cgx_evq_entry *qentry;
+   struct cgx_link_event *event;
+   unsigned long flags;
+
+   do {
+   /* Dequeue an event */
+   spin_lock_irqsave(>cgx_evq_lock, flags);
+   qentry = list_first_entry_or_null(>cgx_evq_head,
+ struct cgx_evq_entry,
+ evq_node);
+   if (qentry)
+   list_del(>evq_node);
+   spin_unlock_irqrestore(>cgx_evq_lock, flags);
+   if (!qentry)
+   break; /* nothing more to process */
+
+   event = >link_event;
+
+   /* Do nothing for now */
+   kfree(qentry);
+   } while (1);
+}
+
+static void cgx_lmac_event_handler_init(struct rvu *rvu)
+{
+   struct cgx_event_cb cb;
+   int 

[PATCH 14/15] soc: octeontx2: Register for CGX lmac events

[sunil . kovvuri]

From: Linu Cherian 

Added support in RVU AF driver to register for
CGX LMAC link status change events from firmware
and managing them. Processing part will be added
in followup patches.

- Introduced eventqueue for posting events from cgx lmac.
  Queueing mechanism will ensure that events can be posted
  and firmware can be acked immediately and hence event
  reception and processing are decoupled.
- Events gets added to the queue by notification callback.
  Notification callback is expected to be atomic, since it
  is called from interrupt context.
- Events are dequeued and processed in a worker thread.

Signed-off-by: Linu Cherian 
---
 drivers/soc/marvell/octeontx2/rvu.c |   6 +-
 drivers/soc/marvell/octeontx2/rvu.h |   5 ++
 drivers/soc/marvell/octeontx2/rvu_cgx.c | 101 +++-
 3 files changed, 108 insertions(+), 4 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index d7b19e0..3809517 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -1563,10 +1563,11 @@ static int rvu_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
 
err = rvu_register_interrupts(rvu);
if (err)
-   goto err_mbox;
+   goto err_cgx;
 
return 0;
-
+err_cgx:
+   rvu_cgx_wq_destroy(rvu);
 err_mbox:
rvu_mbox_destroy(rvu);
 err_hwsetup:
@@ -1588,6 +1589,7 @@ static void rvu_remove(struct pci_dev *pdev)
struct rvu *rvu = pci_get_drvdata(pdev);
 
rvu_unregister_interrupts(rvu);
+   rvu_cgx_wq_destroy(rvu);
rvu_mbox_destroy(rvu);
rvu_reset_all_blocks(rvu);
rvu_free_hw_resources(rvu);
diff --git a/drivers/soc/marvell/octeontx2/rvu.h 
b/drivers/soc/marvell/octeontx2/rvu.h
index 385f597..d169fa9 100644
--- a/drivers/soc/marvell/octeontx2/rvu.h
+++ b/drivers/soc/marvell/octeontx2/rvu.h
@@ -110,6 +110,10 @@ struct rvu {
  * every cgx lmac port
  */
void**cgx_idmap; /* cgx id to cgx data map table */
+   struct  work_struct cgx_evh_work;
+   struct  workqueue_struct *cgx_evh_wq;
+   spinlock_t  cgx_evq_lock; /* cgx event queue lock */
+   struct list_headcgx_evq_head; /* cgx event queue head */
 };
 
 static inline void rvu_write64(struct rvu *rvu, u64 block, u64 offset, u64 val)
@@ -150,4 +154,5 @@ int rvu_poll_reg(struct rvu *rvu, u64 block, u64 offset, 
u64 mask, bool zero);
 
 /* CGX APIs */
 int rvu_cgx_probe(struct rvu *rvu);
+void rvu_cgx_wq_destroy(struct rvu *rvu);
 #endif /* RVU_H */
diff --git a/drivers/soc/marvell/octeontx2/rvu_cgx.c 
b/drivers/soc/marvell/octeontx2/rvu_cgx.c
index bf81507..2359806e 100644
--- a/drivers/soc/marvell/octeontx2/rvu_cgx.c
+++ b/drivers/soc/marvell/octeontx2/rvu_cgx.c
@@ -15,6 +15,11 @@
 #include "rvu.h"
 #include "cgx.h"
 
+struct cgx_evq_entry {
+   struct list_head evq_node;
+   struct cgx_link_event link_event;
+};
+
 static inline u8 cgxlmac_id_to_bmap(u8 cgx_id, u8 lmac_id)
 {
return ((cgx_id & 0xF) << 4) | (lmac_id & 0xF);
@@ -72,9 +77,95 @@ static int rvu_map_cgx_lmac_pf(struct rvu *rvu)
return 0;
 }
 
+/* This is called from interrupt context and is expected to be atomic */
+static int cgx_lmac_postevent(struct cgx_link_event *event, void *data)
+{
+   struct rvu *rvu = data;
+   struct cgx_evq_entry *qentry;
+
+   /* post event to the event queue */
+   qentry = kmalloc(sizeof(*qentry), GFP_ATOMIC);
+   if (!qentry)
+   return -ENOMEM;
+   qentry->link_event = *event;
+   spin_lock(>cgx_evq_lock);
+   list_add_tail(>evq_node, >cgx_evq_head);
+   spin_unlock(>cgx_evq_lock);
+
+   /* start worker to process the events */
+   queue_work(rvu->cgx_evh_wq, >cgx_evh_work);
+
+   return 0;
+}
+
+static void cgx_evhandler_task(struct work_struct *work)
+{
+   struct rvu *rvu = container_of(work, struct rvu, cgx_evh_work);
+   struct cgx_evq_entry *qentry;
+   struct cgx_link_event *event;
+   unsigned long flags;
+
+   do {
+   /* Dequeue an event */
+   spin_lock_irqsave(>cgx_evq_lock, flags);
+   qentry = list_first_entry_or_null(>cgx_evq_head,
+ struct cgx_evq_entry,
+ evq_node);
+   if (qentry)
+   list_del(>evq_node);
+   spin_unlock_irqrestore(>cgx_evq_lock, flags);
+   if (!qentry)
+   break; /* nothing more to process */
+
+   event = >link_event;
+
+   /* Do nothing for now */
+   kfree(qentry);
+   } while (1);
+}
+
+static void cgx_lmac_event_handler_init(struct rvu *rvu)
+{
+   struct cgx_event_cb cb;
+   int 

[PATCH 04/15] soc: octeontx2: Add mailbox support infra

[sunil . kovvuri]

From: Aleksey Makarov 

This patch adds mailbox support infrastructure APIs.
Each RVU device has a dedicated 64KB mailbox region
shared with it's peer for communication. RVU AF has
a separate mailbox region shared with each of RVU PFs
and a RVU PF has a separate region shared with each of
it's VF.

These set of APIs are used by this driver (RVU AF) and
other RVU PF/VF drivers eg netdev, crypto e.t.c.

Signed-off-by: Aleksey Makarov 
Signed-off-by: Sunil Goutham 
Signed-off-by: Lukasz Bartosik 
---
 drivers/soc/marvell/octeontx2/Makefile  |   2 +-
 drivers/soc/marvell/octeontx2/mbox.c| 300 
 drivers/soc/marvell/octeontx2/mbox.h| 142 +++
 drivers/soc/marvell/octeontx2/rvu_reg.h |   4 +
 4 files changed, 447 insertions(+), 1 deletion(-)
 create mode 100644 drivers/soc/marvell/octeontx2/mbox.c
 create mode 100644 drivers/soc/marvell/octeontx2/mbox.h

diff --git a/drivers/soc/marvell/octeontx2/Makefile 
b/drivers/soc/marvell/octeontx2/Makefile
index dacbd16..8737ec3 100644
--- a/drivers/soc/marvell/octeontx2/Makefile
+++ b/drivers/soc/marvell/octeontx2/Makefile
@@ -5,4 +5,4 @@
 
 obj-$(CONFIG_OCTEONTX2_AF) += octeontx2_af.o
 
-octeontx2_af-y := rvu.o
+octeontx2_af-y := rvu.o mbox.o
diff --git a/drivers/soc/marvell/octeontx2/mbox.c 
b/drivers/soc/marvell/octeontx2/mbox.c
new file mode 100644
index 000..564cbf9
--- /dev/null
+++ b/drivers/soc/marvell/octeontx2/mbox.c
@@ -0,0 +1,300 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Marvell OcteonTx2 RVU Admin Function driver
+ *
+ * Copyright (C) 2018 Marvell International Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include 
+#include 
+#include 
+
+#include "rvu_reg.h"
+#include "mbox.h"
+
+static const u16 msgs_offset = ALIGN(sizeof(struct mbox_hdr), MBOX_MSG_ALIGN);
+
+void otx2_mbox_reset(struct otx2_mbox *mbox, int devid)
+{
+   struct otx2_mbox_dev *mdev = >dev[devid];
+   struct mbox_hdr *tx_hdr =
+   (struct mbox_hdr *)(mdev->mbase  + mbox->tx_start);
+   struct mbox_hdr *rx_hdr =
+   (struct mbox_hdr *)(mdev->mbase  + mbox->rx_start);
+
+   spin_lock(>mbox_lock);
+   mdev->msg_size = 0;
+   mdev->rsp_size = 0;
+   tx_hdr->num_msgs = 0;
+   rx_hdr->num_msgs = 0;
+   spin_unlock(>mbox_lock);
+}
+EXPORT_SYMBOL(otx2_mbox_reset);
+
+void otx2_mbox_destroy(struct otx2_mbox *mbox)
+{
+   mbox->reg_base = NULL;
+   mbox->hwbase = NULL;
+
+   kfree(mbox->dev);
+   mbox->dev = NULL;
+}
+EXPORT_SYMBOL(otx2_mbox_destroy);
+
+int otx2_mbox_init(struct otx2_mbox *mbox, void *hwbase, struct pci_dev *pdev,
+  void *reg_base, int direction, int ndevs)
+{
+   int devid;
+   struct otx2_mbox_dev *mdev;
+
+   switch (direction) {
+   case MBOX_DIR_AFPF:
+   case MBOX_DIR_PFVF:
+   mbox->tx_start = MBOX_DOWN_TX_START;
+   mbox->rx_start = MBOX_DOWN_RX_START;
+   mbox->tx_size  = MBOX_DOWN_TX_SIZE;
+   mbox->rx_size  = MBOX_DOWN_RX_SIZE;
+   break;
+   case MBOX_DIR_PFAF:
+   case MBOX_DIR_VFPF:
+   mbox->tx_start = MBOX_DOWN_RX_START;
+   mbox->rx_start = MBOX_DOWN_TX_START;
+   mbox->tx_size  = MBOX_DOWN_RX_SIZE;
+   mbox->rx_size  = MBOX_DOWN_TX_SIZE;
+   break;
+   case MBOX_DIR_AFPF_UP:
+   case MBOX_DIR_PFVF_UP:
+   mbox->tx_start = MBOX_UP_TX_START;
+   mbox->rx_start = MBOX_UP_RX_START;
+   mbox->tx_size  = MBOX_UP_TX_SIZE;
+   mbox->rx_size  = MBOX_UP_RX_SIZE;
+   break;
+   case MBOX_DIR_PFAF_UP:
+   case MBOX_DIR_VFPF_UP:
+   mbox->tx_start = MBOX_UP_RX_START;
+   mbox->rx_start = MBOX_UP_TX_START;
+   mbox->tx_size  = MBOX_UP_RX_SIZE;
+   mbox->rx_size  = MBOX_UP_TX_SIZE;
+   break;
+   default:
+   return -ENODEV;
+   }
+
+   switch (direction) {
+   case MBOX_DIR_AFPF:
+   case MBOX_DIR_AFPF_UP:
+   mbox->trigger = RVU_AF_AFPF_MBOX0;
+   mbox->tr_shift = 4;
+   break;
+   case MBOX_DIR_PFAF:
+   case MBOX_DIR_PFAF_UP:
+   mbox->trigger = RVU_PF_PFAF_MBOX1;
+   mbox->tr_shift = 0;
+   break;
+   case MBOX_DIR_PFVF:
+   case MBOX_DIR_PFVF_UP:
+   mbox->trigger = RVU_PF_VFX_PFVF_MBOX0;
+   mbox->tr_shift = 12;
+   break;
+   case MBOX_DIR_VFPF:
+   case MBOX_DIR_VFPF_UP:
+   mbox->trigger = RVU_VF_VFPF_MBOX1;
+   mbox->tr_shift = 0;
+   break;
+   default:
+   return -ENODEV;
+   }
+
+   mbox->reg_base = reg_base;
+   mbox->hwbase = hwbase;
+   mbox->pdev = 

[PATCH 04/15] soc: octeontx2: Add mailbox support infra

[sunil . kovvuri]

From: Aleksey Makarov 

This patch adds mailbox support infrastructure APIs.
Each RVU device has a dedicated 64KB mailbox region
shared with it's peer for communication. RVU AF has
a separate mailbox region shared with each of RVU PFs
and a RVU PF has a separate region shared with each of
it's VF.

These set of APIs are used by this driver (RVU AF) and
other RVU PF/VF drivers eg netdev, crypto e.t.c.

Signed-off-by: Aleksey Makarov 
Signed-off-by: Sunil Goutham 
Signed-off-by: Lukasz Bartosik 
---
 drivers/soc/marvell/octeontx2/Makefile  |   2 +-
 drivers/soc/marvell/octeontx2/mbox.c| 300 
 drivers/soc/marvell/octeontx2/mbox.h| 142 +++
 drivers/soc/marvell/octeontx2/rvu_reg.h |   4 +
 4 files changed, 447 insertions(+), 1 deletion(-)
 create mode 100644 drivers/soc/marvell/octeontx2/mbox.c
 create mode 100644 drivers/soc/marvell/octeontx2/mbox.h

diff --git a/drivers/soc/marvell/octeontx2/Makefile 
b/drivers/soc/marvell/octeontx2/Makefile
index dacbd16..8737ec3 100644
--- a/drivers/soc/marvell/octeontx2/Makefile
+++ b/drivers/soc/marvell/octeontx2/Makefile
@@ -5,4 +5,4 @@
 
 obj-$(CONFIG_OCTEONTX2_AF) += octeontx2_af.o
 
-octeontx2_af-y := rvu.o
+octeontx2_af-y := rvu.o mbox.o
diff --git a/drivers/soc/marvell/octeontx2/mbox.c 
b/drivers/soc/marvell/octeontx2/mbox.c
new file mode 100644
index 000..564cbf9
--- /dev/null
+++ b/drivers/soc/marvell/octeontx2/mbox.c
@@ -0,0 +1,300 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Marvell OcteonTx2 RVU Admin Function driver
+ *
+ * Copyright (C) 2018 Marvell International Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include 
+#include 
+#include 
+
+#include "rvu_reg.h"
+#include "mbox.h"
+
+static const u16 msgs_offset = ALIGN(sizeof(struct mbox_hdr), MBOX_MSG_ALIGN);
+
+void otx2_mbox_reset(struct otx2_mbox *mbox, int devid)
+{
+   struct otx2_mbox_dev *mdev = >dev[devid];
+   struct mbox_hdr *tx_hdr =
+   (struct mbox_hdr *)(mdev->mbase  + mbox->tx_start);
+   struct mbox_hdr *rx_hdr =
+   (struct mbox_hdr *)(mdev->mbase  + mbox->rx_start);
+
+   spin_lock(>mbox_lock);
+   mdev->msg_size = 0;
+   mdev->rsp_size = 0;
+   tx_hdr->num_msgs = 0;
+   rx_hdr->num_msgs = 0;
+   spin_unlock(>mbox_lock);
+}
+EXPORT_SYMBOL(otx2_mbox_reset);
+
+void otx2_mbox_destroy(struct otx2_mbox *mbox)
+{
+   mbox->reg_base = NULL;
+   mbox->hwbase = NULL;
+
+   kfree(mbox->dev);
+   mbox->dev = NULL;
+}
+EXPORT_SYMBOL(otx2_mbox_destroy);
+
+int otx2_mbox_init(struct otx2_mbox *mbox, void *hwbase, struct pci_dev *pdev,
+  void *reg_base, int direction, int ndevs)
+{
+   int devid;
+   struct otx2_mbox_dev *mdev;
+
+   switch (direction) {
+   case MBOX_DIR_AFPF:
+   case MBOX_DIR_PFVF:
+   mbox->tx_start = MBOX_DOWN_TX_START;
+   mbox->rx_start = MBOX_DOWN_RX_START;
+   mbox->tx_size  = MBOX_DOWN_TX_SIZE;
+   mbox->rx_size  = MBOX_DOWN_RX_SIZE;
+   break;
+   case MBOX_DIR_PFAF:
+   case MBOX_DIR_VFPF:
+   mbox->tx_start = MBOX_DOWN_RX_START;
+   mbox->rx_start = MBOX_DOWN_TX_START;
+   mbox->tx_size  = MBOX_DOWN_RX_SIZE;
+   mbox->rx_size  = MBOX_DOWN_TX_SIZE;
+   break;
+   case MBOX_DIR_AFPF_UP:
+   case MBOX_DIR_PFVF_UP:
+   mbox->tx_start = MBOX_UP_TX_START;
+   mbox->rx_start = MBOX_UP_RX_START;
+   mbox->tx_size  = MBOX_UP_TX_SIZE;
+   mbox->rx_size  = MBOX_UP_RX_SIZE;
+   break;
+   case MBOX_DIR_PFAF_UP:
+   case MBOX_DIR_VFPF_UP:
+   mbox->tx_start = MBOX_UP_RX_START;
+   mbox->rx_start = MBOX_UP_TX_START;
+   mbox->tx_size  = MBOX_UP_RX_SIZE;
+   mbox->rx_size  = MBOX_UP_TX_SIZE;
+   break;
+   default:
+   return -ENODEV;
+   }
+
+   switch (direction) {
+   case MBOX_DIR_AFPF:
+   case MBOX_DIR_AFPF_UP:
+   mbox->trigger = RVU_AF_AFPF_MBOX0;
+   mbox->tr_shift = 4;
+   break;
+   case MBOX_DIR_PFAF:
+   case MBOX_DIR_PFAF_UP:
+   mbox->trigger = RVU_PF_PFAF_MBOX1;
+   mbox->tr_shift = 0;
+   break;
+   case MBOX_DIR_PFVF:
+   case MBOX_DIR_PFVF_UP:
+   mbox->trigger = RVU_PF_VFX_PFVF_MBOX0;
+   mbox->tr_shift = 12;
+   break;
+   case MBOX_DIR_VFPF:
+   case MBOX_DIR_VFPF_UP:
+   mbox->trigger = RVU_VF_VFPF_MBOX1;
+   mbox->tr_shift = 0;
+   break;
+   default:
+   return -ENODEV;
+   }
+
+   mbox->reg_base = reg_base;
+   mbox->hwbase = hwbase;
+   mbox->pdev = 

[PATCH 05/15] soc: octeontx2: Add mailbox IRQ and msg handlers

[sunil . kovvuri]

From: Sunil Goutham 

This patch adds support for mailbox interrupt and message
handling. Mapped mailbox region and registered a workqueue
for message handling. Enabled mailbox IRQ of RVU PFs
and registered a interrupt handler. When IRQ is triggered
work is added to the mbox workqueue for msgs to get processed.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/mbox.h   |  14 +-
 drivers/soc/marvell/octeontx2/rvu.c| 254 +
 drivers/soc/marvell/octeontx2/rvu.h|  22 +++
 drivers/soc/marvell/octeontx2/rvu_struct.h |  22 +++
 4 files changed, 309 insertions(+), 3 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/mbox.h 
b/drivers/soc/marvell/octeontx2/mbox.h
index 8e205fd..fc593f0 100644
--- a/drivers/soc/marvell/octeontx2/mbox.h
+++ b/drivers/soc/marvell/octeontx2/mbox.h
@@ -33,6 +33,8 @@
 # error "incorrect mailbox area sizes"
 #endif
 
+#define INTR_MASK(pfvfs) ((pfvfs < 64) ? (BIT_ULL(pfvfs) - 1) : (~0ull))
+
 #define MBOX_RSP_TIMEOUT   1000 /* in ms, Time to wait for mbox response */
 
 #define MBOX_MSG_ALIGN 16  /* Align mbox msg start to 16bytes */
@@ -90,8 +92,9 @@ struct mbox_msghdr {
 
 void otx2_mbox_reset(struct otx2_mbox *mbox, int devid);
 void otx2_mbox_destroy(struct otx2_mbox *mbox);
-int otx2_mbox_init(struct otx2_mbox *mbox, void *hwbase, struct pci_dev *pdev,
-  void *reg_base, int direction, int ndevs);
+int otx2_mbox_init(struct otx2_mbox *mbox, void __force *hwbase,
+  struct pci_dev *pdev, void __force *reg_base,
+  int direction, int ndevs);
 void otx2_mbox_msg_send(struct otx2_mbox *mbox, int devid);
 int otx2_mbox_wait_for_rsp(struct otx2_mbox *mbox, int devid);
 int otx2_mbox_busy_poll_for_rsp(struct otx2_mbox *mbox, int devid);
@@ -115,7 +118,7 @@ static inline struct mbox_msghdr 
*otx2_mbox_alloc_msg(struct otx2_mbox *mbox,
 #define MBOX_MSG_MAX   0x
 
 #define MBOX_MESSAGES  \
-M(READY,   0x001, msg_req, msg_rsp)
+M(READY,   0x001, msg_req, ready_msg_rsp)
 
 enum {
 #define M(_name, _id, _1, _2) MBOX_MSG_ ## _name = _id,
@@ -139,4 +142,9 @@ struct msg_rsp {
struct mbox_msghdr hdr;
 };
 
+struct ready_msg_rsp {
+   struct mbox_msghdr hdr;
+   u16sclk_feq;/* SCLK frequency */
+};
+
 #endif /* MBOX_H */
diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index fa5f40b..e795c2f 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -258,6 +258,245 @@ static int rvu_setup_hw_resources(struct rvu *rvu)
return 0;
 }
 
+static int rvu_process_mbox_msg(struct rvu *rvu, int devid,
+   struct mbox_msghdr *req)
+{
+   /* Check if valid, if not reply with a invalid msg */
+   if (req->sig != OTX2_MBOX_REQ_SIG)
+   goto bad_message;
+
+   if (req->id == MBOX_MSG_READY)
+   return 0;
+
+bad_message:
+   otx2_reply_invalid_msg(>mbox, devid, req->pcifunc,
+  req->id);
+   return -ENODEV;
+}
+
+static void rvu_mbox_handler(struct work_struct *work)
+{
+   struct rvu_work *mwork = container_of(work, struct rvu_work, work);
+   struct rvu *rvu = mwork->rvu;
+   struct otx2_mbox_dev *mdev;
+   struct mbox_hdr *req_hdr;
+   struct mbox_msghdr *msg;
+   struct otx2_mbox *mbox;
+   int offset, id, err;
+   u16 pf;
+
+   mbox = >mbox;
+   pf = mwork - rvu->mbox_wrk;
+   mdev = >dev[pf];
+
+   /* Process received mbox messages */
+   req_hdr = (struct mbox_hdr *)(mdev->mbase + mbox->rx_start);
+   if (req_hdr->num_msgs == 0)
+   return;
+
+   offset = mbox->rx_start + ALIGN(sizeof(*req_hdr), MBOX_MSG_ALIGN);
+
+   for (id = 0; id < req_hdr->num_msgs; id++) {
+   msg = (struct mbox_msghdr *)(mdev->mbase + offset);
+
+   /* Set which PF sent this message based on mbox IRQ */
+   msg->pcifunc &= ~(RVU_PFVF_PF_MASK << RVU_PFVF_PF_SHIFT);
+   msg->pcifunc |= (pf << RVU_PFVF_PF_SHIFT);
+   err = rvu_process_mbox_msg(rvu, pf, msg);
+   if (!err) {
+   offset = mbox->rx_start + msg->next_msgoff;
+   continue;
+   }
+
+   if (msg->pcifunc & RVU_PFVF_FUNC_MASK)
+   dev_warn(rvu->dev, "Error %d when processing message %s 
(0x%x) from PF%d:VF%d\n",
+err, otx2_mbox_id2name(msg->id), msg->id, pf,
+(msg->pcifunc & RVU_PFVF_FUNC_MASK) - 1);
+   else
+   dev_warn(rvu->dev, "Error %d when processing message %s 
(0x%x) from PF%d\n",
+err, otx2_mbox_id2name(msg->id), msg->id, pf);
+   }
+
+   /* Send mbox responses to PF */
+   

[PATCH 03/15] soc: octeontx2: Gather RVU blocks HW info

[sunil . kovvuri]

From: Sunil Goutham 

This patch gathers NPA/NIX/SSO/SSOW/TIM/CPT RVU blocks's
HW info like number of LFs. Important register offsets
saved for later use to avoid code duplication for each block.
A bitmap is allocated for each of the blocks which later
on will be used to allocate a LF for a RVU PF/VF.

Also added RVU NIX/NPA block registers and few registers
of other blocks.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/rvu.c | 167 
 drivers/soc/marvell/octeontx2/rvu.h |  21 ++
 drivers/soc/marvell/octeontx2/rvu_reg.h | 335 +++-
 3 files changed, 518 insertions(+), 5 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index d40fabf..fa5f40b 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -57,6 +57,15 @@ int rvu_poll_reg(struct rvu *rvu, u64 block, u64 offset, u64 
mask, bool zero)
return -EBUSY;
 }
 
+int rvu_alloc_bitmap(struct rsrc_bmap *rsrc)
+{
+   rsrc->bmap = kcalloc(BITS_TO_LONGS(rsrc->max),
+sizeof(long), GFP_KERNEL);
+   if (!rsrc->bmap)
+   return -ENOMEM;
+   return 0;
+}
+
 static void rvu_check_block_implemented(struct rvu *rvu)
 {
struct rvu_hwinfo *hw = rvu->hw;
@@ -98,6 +107,157 @@ static void rvu_reset_all_blocks(struct rvu *rvu)
rvu_block_reset(rvu, BLKADDR_NDC2, NDC_AF_BLK_RST);
 }
 
+static void rvu_free_hw_resources(struct rvu *rvu)
+{
+   struct rvu_hwinfo *hw = rvu->hw;
+   struct rvu_block *block;
+   int id;
+
+   /* Free all bitmaps */
+   for (id = 0; id < BLK_COUNT; id++) {
+   block = >block[id];
+   kfree(block->lf.bmap);
+   }
+}
+
+static int rvu_setup_hw_resources(struct rvu *rvu)
+{
+   struct rvu_hwinfo *hw = rvu->hw;
+   struct rvu_block *block;
+   int err;
+   u64 cfg;
+
+   /* Get HW supported max RVU PF & VF count */
+   cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_CONST);
+   hw->total_pfs = (cfg >> 32) & 0xFF;
+   hw->total_vfs = (cfg >> 20) & 0xFFF;
+   hw->max_vfs_per_pf = (cfg >> 40) & 0xFF;
+
+   /* Init NPA LF's bitmap */
+   block = >block[BLKADDR_NPA];
+   if (!block->implemented)
+   goto nix;
+   cfg = rvu_read64(rvu, BLKADDR_NPA, NPA_AF_CONST);
+   block->lf.max = (cfg >> 16) & 0xFFF;
+   block->addr = BLKADDR_NPA;
+   block->lfshift = 8;
+   block->lookup_reg = NPA_AF_RVU_LF_CFG_DEBUG;
+   block->pf_lfcnt_reg = RVU_PRIV_PFX_NPA_CFG;
+   block->vf_lfcnt_reg = RVU_PRIV_HWVFX_NPA_CFG;
+   block->lfcfg_reg = NPA_PRIV_LFX_CFG;
+   block->msixcfg_reg = NPA_PRIV_LFX_INT_CFG;
+   block->lfreset_reg = NPA_AF_LF_RST;
+   sprintf(block->name, "NPA");
+   err = rvu_alloc_bitmap(>lf);
+   if (err)
+   return err;
+
+nix:
+   /* Init NIX LF's bitmap */
+   block = >block[BLKADDR_NIX0];
+   if (!block->implemented)
+   goto sso;
+   cfg = rvu_read64(rvu, BLKADDR_NIX0, NIX_AF_CONST2);
+   block->lf.max = cfg & 0xFFF;
+   block->addr = BLKADDR_NIX0;
+   block->lfshift = 8;
+   block->lookup_reg = NIX_AF_RVU_LF_CFG_DEBUG;
+   block->pf_lfcnt_reg = RVU_PRIV_PFX_NIX_CFG;
+   block->vf_lfcnt_reg = RVU_PRIV_HWVFX_NIX_CFG;
+   block->lfcfg_reg = NIX_PRIV_LFX_CFG;
+   block->msixcfg_reg = NIX_PRIV_LFX_INT_CFG;
+   block->lfreset_reg = NIX_AF_LF_RST;
+   sprintf(block->name, "NIX");
+   err = rvu_alloc_bitmap(>lf);
+   if (err)
+   return err;
+
+sso:
+   /* Init SSO group's bitmap */
+   block = >block[BLKADDR_SSO];
+   if (!block->implemented)
+   goto ssow;
+   cfg = rvu_read64(rvu, BLKADDR_SSO, SSO_AF_CONST);
+   block->lf.max = cfg & 0x;
+   block->addr = BLKADDR_SSO;
+   block->multislot = true;
+   block->lfshift = 3;
+   block->lookup_reg = SSO_AF_RVU_LF_CFG_DEBUG;
+   block->pf_lfcnt_reg = RVU_PRIV_PFX_SSO_CFG;
+   block->vf_lfcnt_reg = RVU_PRIV_HWVFX_SSO_CFG;
+   block->lfcfg_reg = SSO_PRIV_LFX_HWGRP_CFG;
+   block->msixcfg_reg = SSO_PRIV_LFX_HWGRP_INT_CFG;
+   block->lfreset_reg = SSO_AF_LF_HWGRP_RST;
+   sprintf(block->name, "SSO GROUP");
+   err = rvu_alloc_bitmap(>lf);
+   if (err)
+   return err;
+
+ssow:
+   /* Init SSO workslot's bitmap */
+   block = >block[BLKADDR_SSOW];
+   if (!block->implemented)
+   goto tim;
+   block->lf.max = (cfg >> 56) & 0xFF;
+   block->addr = BLKADDR_SSOW;
+   block->multislot = true;
+   block->lfshift = 3;
+   block->lookup_reg = SSOW_AF_RVU_LF_HWS_CFG_DEBUG;
+   block->pf_lfcnt_reg = RVU_PRIV_PFX_SSOW_CFG;
+   block->vf_lfcnt_reg = RVU_PRIV_HWVFX_SSOW_CFG;
+   block->lfcfg_reg = SSOW_PRIV_LFX_HWS_CFG;
+   block->msixcfg_reg = SSOW_PRIV_LFX_HWS_INT_CFG;
+   block->lfreset_reg = 

[PATCH 01/15] soc: octeontx2: Add Marvell OcteonTX2 RVU AF driver

[sunil . kovvuri]

From: Sunil Goutham 

This patch adds basic template for Marvell OcteonTX2's
resource virtualization unit (RVU) admin function (AF)
driver. Just the driver registration and probe.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/Kconfig|   1 +
 drivers/soc/Makefile   |   1 +
 drivers/soc/marvell/Kconfig|  13 
 drivers/soc/marvell/Makefile   |   2 +
 drivers/soc/marvell/octeontx2/Makefile |   8 +++
 drivers/soc/marvell/octeontx2/rvu.c| 126 +
 drivers/soc/marvell/octeontx2/rvu.h|  31 
 7 files changed, 182 insertions(+)
 create mode 100644 drivers/soc/marvell/Kconfig
 create mode 100644 drivers/soc/marvell/Makefile
 create mode 100644 drivers/soc/marvell/octeontx2/Makefile
 create mode 100644 drivers/soc/marvell/octeontx2/rvu.c
 create mode 100644 drivers/soc/marvell/octeontx2/rvu.h

diff --git a/drivers/soc/Kconfig b/drivers/soc/Kconfig
index c07b4a8..42f2d0b 100644
--- a/drivers/soc/Kconfig
+++ b/drivers/soc/Kconfig
@@ -6,6 +6,7 @@ source "drivers/soc/atmel/Kconfig"
 source "drivers/soc/bcm/Kconfig"
 source "drivers/soc/fsl/Kconfig"
 source "drivers/soc/imx/Kconfig"
+source "drivers/soc/marvell/Kconfig"
 source "drivers/soc/mediatek/Kconfig"
 source "drivers/soc/qcom/Kconfig"
 source "drivers/soc/renesas/Kconfig"
diff --git a/drivers/soc/Makefile b/drivers/soc/Makefile
index 113e884..5e18cbb 100644
--- a/drivers/soc/Makefile
+++ b/drivers/soc/Makefile
@@ -12,6 +12,7 @@ obj-y += fsl/
 obj-$(CONFIG_ARCH_GEMINI)  += gemini/
 obj-$(CONFIG_ARCH_MXC) += imx/
 obj-$(CONFIG_SOC_XWAY) += lantiq/
+obj-y  += marvell/
 obj-y  += mediatek/
 obj-$(CONFIG_ARCH_MESON)   += amlogic/
 obj-y  += qcom/
diff --git a/drivers/soc/marvell/Kconfig b/drivers/soc/marvell/Kconfig
new file mode 100644
index 000..4499caf
--- /dev/null
+++ b/drivers/soc/marvell/Kconfig
@@ -0,0 +1,13 @@
+#
+# MARVELL SoC drivers
+#
+
+menu "Marvell SoC drivers"
+
+config OCTEONTX2_AF
+   tristate "OcteonTX2 RVU Admin Function driver"
+   depends on ARM64 && PCI
+   help
+ This driver supports Marvell's OcteonTX2 Resource Virtualization
+ Unit's admin function manager which manages all RVU HW resources.
+endmenu
diff --git a/drivers/soc/marvell/Makefile b/drivers/soc/marvell/Makefile
new file mode 100644
index 000..16e0ca0
--- /dev/null
+++ b/drivers/soc/marvell/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0
+obj-y  += octeontx2/
diff --git a/drivers/soc/marvell/octeontx2/Makefile 
b/drivers/soc/marvell/octeontx2/Makefile
new file mode 100644
index 000..dacbd16
--- /dev/null
+++ b/drivers/soc/marvell/octeontx2/Makefile
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for Marvell's OcteonTX2 RVU Admin Function driver
+#
+
+obj-$(CONFIG_OCTEONTX2_AF) += octeontx2_af.o
+
+octeontx2_af-y := rvu.o
diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
new file mode 100644
index 000..5af4da6
--- /dev/null
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -0,0 +1,126 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Marvell OcteonTx2 RVU Admin Function driver
+ *
+ * Copyright (C) 2018 Marvell International Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "rvu.h"
+
+#define DRV_NAME   "octeontx2-af"
+#define DRV_STRING  "Marvell OcteonTX2 RVU Admin Function Driver"
+#define DRV_VERSION"1.0"
+
+/* Supported devices */
+static const struct pci_device_id rvu_id_table[] = {
+   { PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_RVU_AF) },
+   { 0, }  /* end of table */
+};
+
+MODULE_AUTHOR("Marvell International Ltd.");
+MODULE_DESCRIPTION(DRV_STRING);
+MODULE_LICENSE("GPL v2");
+MODULE_VERSION(DRV_VERSION);
+MODULE_DEVICE_TABLE(pci, rvu_id_table);
+
+static int rvu_probe(struct pci_dev *pdev, const struct pci_device_id *id)
+{
+   struct device *dev = >dev;
+   struct rvu *rvu;
+   interr;
+
+   rvu = devm_kzalloc(dev, sizeof(*rvu), GFP_KERNEL);
+   if (!rvu)
+   return -ENOMEM;
+
+   pci_set_drvdata(pdev, rvu);
+   rvu->pdev = pdev;
+   rvu->dev = >dev;
+
+   err = pci_enable_device(pdev);
+   if (err) {
+   dev_err(dev, "Failed to enable PCI device\n");
+   goto err_freemem;
+   }
+
+   err = pci_request_regions(pdev, DRV_NAME);
+   if (err) {
+   dev_err(dev, "PCI request regions failed 0x%x\n", err);
+   goto err_disable_device;
+   }
+
+   err = pci_set_dma_mask(pdev, DMA_BIT_MASK(48));
+   if (err) {
+   dev_err(dev, "Unable to set DMA 

[PATCH 07/15] soc: octeontx2: Scan blocks for LFs provisioned to PF/VF

[sunil . kovvuri]

From: Sunil Goutham 

Scan all RVU blocks to find any 'LF to RVU PF/VF' mapping done by
low level firmware. If found any, mark them as used in respective
block's LF bitmap and also save mapped PF/VF's PF_FUNC info.

This is done to avoid reattaching a block LF to a different RVU PF/VF.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/rvu.c| 148 -
 drivers/soc/marvell/octeontx2/rvu.h|  16 
 drivers/soc/marvell/octeontx2/rvu_struct.h |  18 
 3 files changed, 180 insertions(+), 2 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 25f79bf..9539ab9 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -22,6 +22,8 @@
 #define DRV_STRING  "Marvell OcteonTX2 RVU Admin Function Driver"
 #define DRV_VERSION"1.0"
 
+static int rvu_get_hwvf(struct rvu *rvu, int pcifunc);
+
 /* Supported devices */
 static const struct pci_device_id rvu_id_table[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_RVU_AF) },
@@ -66,6 +68,91 @@ int rvu_alloc_bitmap(struct rsrc_bmap *rsrc)
return 0;
 }
 
+static void rvu_update_rsrc_map(struct rvu *rvu, struct rvu_pfvf *pfvf,
+   struct rvu_block *block, u16 pcifunc,
+   u16 lf, bool attach)
+{
+   int devnum, num_lfs = 0;
+   bool is_pf;
+   u64 reg;
+
+   if (lf >= block->lf.max) {
+   dev_err(>pdev->dev,
+   "%s: FATAL: LF %d is >= %s's max lfs i.e %d\n",
+   __func__, lf, block->name, block->lf.max);
+   return;
+   }
+
+   /* Check if this is for a RVU PF or VF */
+   if (pcifunc & RVU_PFVF_FUNC_MASK) {
+   is_pf = false;
+   devnum = rvu_get_hwvf(rvu, pcifunc);
+   } else {
+   is_pf = true;
+   devnum = rvu_get_pf(pcifunc);
+   }
+
+   block->fn_map[lf] = attach ? pcifunc : 0;
+
+   switch (block->type) {
+   case BLKTYPE_NPA:
+   pfvf->npalf = attach ? true : false;
+   num_lfs = pfvf->npalf;
+   break;
+   case BLKTYPE_NIX:
+   pfvf->nixlf = attach ? true : false;
+   num_lfs = pfvf->nixlf;
+   break;
+   case BLKTYPE_SSO:
+   attach ? pfvf->sso++ : pfvf->sso--;
+   num_lfs = pfvf->sso;
+   break;
+   case BLKTYPE_SSOW:
+   attach ? pfvf->ssow++ : pfvf->ssow--;
+   num_lfs = pfvf->ssow;
+   break;
+   case BLKTYPE_TIM:
+   attach ? pfvf->timlfs++ : pfvf->timlfs--;
+   num_lfs = pfvf->timlfs;
+   break;
+   case BLKTYPE_CPT:
+   attach ? pfvf->cptlfs++ : pfvf->cptlfs--;
+   num_lfs = pfvf->cptlfs;
+   break;
+   }
+
+   reg = is_pf ? block->pf_lfcnt_reg : block->vf_lfcnt_reg;
+   rvu_write64(rvu, BLKADDR_RVUM, reg | (devnum << 16), num_lfs);
+}
+
+inline int rvu_get_pf(u16 pcifunc)
+{
+   return (pcifunc >> RVU_PFVF_PF_SHIFT) & RVU_PFVF_PF_MASK;
+}
+
+static int rvu_get_hwvf(struct rvu *rvu, int pcifunc)
+{
+   int pf, func;
+   u64 cfg;
+
+   pf = rvu_get_pf(pcifunc);
+   func = pcifunc & RVU_PFVF_FUNC_MASK;
+
+   /* Get first HWVF attached to this PF */
+   cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_PFX_CFG(pf));
+
+   return ((cfg & 0xFFF) + func - 1);
+}
+
+struct rvu_pfvf *rvu_get_pfvf(struct rvu *rvu, int pcifunc)
+{
+   /* Check if it is a PF or VF */
+   if (pcifunc & RVU_PFVF_FUNC_MASK)
+   return >hwvf[rvu_get_hwvf(rvu, pcifunc)];
+   else
+   return >pf[rvu_get_pf(pcifunc)];
+}
+
 static void rvu_check_block_implemented(struct rvu *rvu)
 {
struct rvu_hwinfo *hw = rvu->hw;
@@ -107,6 +194,28 @@ static void rvu_reset_all_blocks(struct rvu *rvu)
rvu_block_reset(rvu, BLKADDR_NDC2, NDC_AF_BLK_RST);
 }
 
+static void rvu_scan_block(struct rvu *rvu, struct rvu_block *block)
+{
+   struct rvu_pfvf *pfvf;
+   u64 cfg;
+   int lf;
+
+   for (lf = 0; lf < block->lf.max; lf++) {
+   cfg = rvu_read64(rvu, block->addr,
+block->lfcfg_reg | (lf << block->lfshift));
+   if (!(cfg & BIT_ULL(63)))
+   continue;
+
+   /* Set this resource as being used */
+   __set_bit(lf, block->lf.bmap);
+
+   /* Get, to whom this LF is attached */
+   pfvf = rvu_get_pfvf(rvu, (cfg >> 8) & 0x);
+   rvu_update_rsrc_map(rvu, pfvf, block,
+   (cfg >> 8) & 0x, lf, true);
+   }
+}
+
 static void rvu_free_hw_resources(struct rvu *rvu)
 {
struct rvu_hwinfo *hw = rvu->hw;
@@ -124,7 +233,7 @@ static int rvu_setup_hw_resources(struct rvu *rvu)
 {
struct rvu_hwinfo *hw = 

[PATCH 05/15] soc: octeontx2: Add mailbox IRQ and msg handlers

[sunil . kovvuri]

From: Sunil Goutham 

This patch adds support for mailbox interrupt and message
handling. Mapped mailbox region and registered a workqueue
for message handling. Enabled mailbox IRQ of RVU PFs
and registered a interrupt handler. When IRQ is triggered
work is added to the mbox workqueue for msgs to get processed.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/mbox.h   |  14 +-
 drivers/soc/marvell/octeontx2/rvu.c| 254 +
 drivers/soc/marvell/octeontx2/rvu.h|  22 +++
 drivers/soc/marvell/octeontx2/rvu_struct.h |  22 +++
 4 files changed, 309 insertions(+), 3 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/mbox.h 
b/drivers/soc/marvell/octeontx2/mbox.h
index 8e205fd..fc593f0 100644
--- a/drivers/soc/marvell/octeontx2/mbox.h
+++ b/drivers/soc/marvell/octeontx2/mbox.h
@@ -33,6 +33,8 @@
 # error "incorrect mailbox area sizes"
 #endif
 
+#define INTR_MASK(pfvfs) ((pfvfs < 64) ? (BIT_ULL(pfvfs) - 1) : (~0ull))
+
 #define MBOX_RSP_TIMEOUT   1000 /* in ms, Time to wait for mbox response */
 
 #define MBOX_MSG_ALIGN 16  /* Align mbox msg start to 16bytes */
@@ -90,8 +92,9 @@ struct mbox_msghdr {
 
 void otx2_mbox_reset(struct otx2_mbox *mbox, int devid);
 void otx2_mbox_destroy(struct otx2_mbox *mbox);
-int otx2_mbox_init(struct otx2_mbox *mbox, void *hwbase, struct pci_dev *pdev,
-  void *reg_base, int direction, int ndevs);
+int otx2_mbox_init(struct otx2_mbox *mbox, void __force *hwbase,
+  struct pci_dev *pdev, void __force *reg_base,
+  int direction, int ndevs);
 void otx2_mbox_msg_send(struct otx2_mbox *mbox, int devid);
 int otx2_mbox_wait_for_rsp(struct otx2_mbox *mbox, int devid);
 int otx2_mbox_busy_poll_for_rsp(struct otx2_mbox *mbox, int devid);
@@ -115,7 +118,7 @@ static inline struct mbox_msghdr 
*otx2_mbox_alloc_msg(struct otx2_mbox *mbox,
 #define MBOX_MSG_MAX   0x
 
 #define MBOX_MESSAGES  \
-M(READY,   0x001, msg_req, msg_rsp)
+M(READY,   0x001, msg_req, ready_msg_rsp)
 
 enum {
 #define M(_name, _id, _1, _2) MBOX_MSG_ ## _name = _id,
@@ -139,4 +142,9 @@ struct msg_rsp {
struct mbox_msghdr hdr;
 };
 
+struct ready_msg_rsp {
+   struct mbox_msghdr hdr;
+   u16sclk_feq;/* SCLK frequency */
+};
+
 #endif /* MBOX_H */
diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index fa5f40b..e795c2f 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -258,6 +258,245 @@ static int rvu_setup_hw_resources(struct rvu *rvu)
return 0;
 }
 
+static int rvu_process_mbox_msg(struct rvu *rvu, int devid,
+   struct mbox_msghdr *req)
+{
+   /* Check if valid, if not reply with a invalid msg */
+   if (req->sig != OTX2_MBOX_REQ_SIG)
+   goto bad_message;
+
+   if (req->id == MBOX_MSG_READY)
+   return 0;
+
+bad_message:
+   otx2_reply_invalid_msg(>mbox, devid, req->pcifunc,
+  req->id);
+   return -ENODEV;
+}
+
+static void rvu_mbox_handler(struct work_struct *work)
+{
+   struct rvu_work *mwork = container_of(work, struct rvu_work, work);
+   struct rvu *rvu = mwork->rvu;
+   struct otx2_mbox_dev *mdev;
+   struct mbox_hdr *req_hdr;
+   struct mbox_msghdr *msg;
+   struct otx2_mbox *mbox;
+   int offset, id, err;
+   u16 pf;
+
+   mbox = >mbox;
+   pf = mwork - rvu->mbox_wrk;
+   mdev = >dev[pf];
+
+   /* Process received mbox messages */
+   req_hdr = (struct mbox_hdr *)(mdev->mbase + mbox->rx_start);
+   if (req_hdr->num_msgs == 0)
+   return;
+
+   offset = mbox->rx_start + ALIGN(sizeof(*req_hdr), MBOX_MSG_ALIGN);
+
+   for (id = 0; id < req_hdr->num_msgs; id++) {
+   msg = (struct mbox_msghdr *)(mdev->mbase + offset);
+
+   /* Set which PF sent this message based on mbox IRQ */
+   msg->pcifunc &= ~(RVU_PFVF_PF_MASK << RVU_PFVF_PF_SHIFT);
+   msg->pcifunc |= (pf << RVU_PFVF_PF_SHIFT);
+   err = rvu_process_mbox_msg(rvu, pf, msg);
+   if (!err) {
+   offset = mbox->rx_start + msg->next_msgoff;
+   continue;
+   }
+
+   if (msg->pcifunc & RVU_PFVF_FUNC_MASK)
+   dev_warn(rvu->dev, "Error %d when processing message %s 
(0x%x) from PF%d:VF%d\n",
+err, otx2_mbox_id2name(msg->id), msg->id, pf,
+(msg->pcifunc & RVU_PFVF_FUNC_MASK) - 1);
+   else
+   dev_warn(rvu->dev, "Error %d when processing message %s 
(0x%x) from PF%d\n",
+err, otx2_mbox_id2name(msg->id), msg->id, pf);
+   }
+
+   /* Send mbox responses to PF */
+   

[PATCH 03/15] soc: octeontx2: Gather RVU blocks HW info

[sunil . kovvuri]

From: Sunil Goutham 

This patch gathers NPA/NIX/SSO/SSOW/TIM/CPT RVU blocks's
HW info like number of LFs. Important register offsets
saved for later use to avoid code duplication for each block.
A bitmap is allocated for each of the blocks which later
on will be used to allocate a LF for a RVU PF/VF.

Also added RVU NIX/NPA block registers and few registers
of other blocks.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/rvu.c | 167 
 drivers/soc/marvell/octeontx2/rvu.h |  21 ++
 drivers/soc/marvell/octeontx2/rvu_reg.h | 335 +++-
 3 files changed, 518 insertions(+), 5 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index d40fabf..fa5f40b 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -57,6 +57,15 @@ int rvu_poll_reg(struct rvu *rvu, u64 block, u64 offset, u64 
mask, bool zero)
return -EBUSY;
 }
 
+int rvu_alloc_bitmap(struct rsrc_bmap *rsrc)
+{
+   rsrc->bmap = kcalloc(BITS_TO_LONGS(rsrc->max),
+sizeof(long), GFP_KERNEL);
+   if (!rsrc->bmap)
+   return -ENOMEM;
+   return 0;
+}
+
 static void rvu_check_block_implemented(struct rvu *rvu)
 {
struct rvu_hwinfo *hw = rvu->hw;
@@ -98,6 +107,157 @@ static void rvu_reset_all_blocks(struct rvu *rvu)
rvu_block_reset(rvu, BLKADDR_NDC2, NDC_AF_BLK_RST);
 }
 
+static void rvu_free_hw_resources(struct rvu *rvu)
+{
+   struct rvu_hwinfo *hw = rvu->hw;
+   struct rvu_block *block;
+   int id;
+
+   /* Free all bitmaps */
+   for (id = 0; id < BLK_COUNT; id++) {
+   block = >block[id];
+   kfree(block->lf.bmap);
+   }
+}
+
+static int rvu_setup_hw_resources(struct rvu *rvu)
+{
+   struct rvu_hwinfo *hw = rvu->hw;
+   struct rvu_block *block;
+   int err;
+   u64 cfg;
+
+   /* Get HW supported max RVU PF & VF count */
+   cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_CONST);
+   hw->total_pfs = (cfg >> 32) & 0xFF;
+   hw->total_vfs = (cfg >> 20) & 0xFFF;
+   hw->max_vfs_per_pf = (cfg >> 40) & 0xFF;
+
+   /* Init NPA LF's bitmap */
+   block = >block[BLKADDR_NPA];
+   if (!block->implemented)
+   goto nix;
+   cfg = rvu_read64(rvu, BLKADDR_NPA, NPA_AF_CONST);
+   block->lf.max = (cfg >> 16) & 0xFFF;
+   block->addr = BLKADDR_NPA;
+   block->lfshift = 8;
+   block->lookup_reg = NPA_AF_RVU_LF_CFG_DEBUG;
+   block->pf_lfcnt_reg = RVU_PRIV_PFX_NPA_CFG;
+   block->vf_lfcnt_reg = RVU_PRIV_HWVFX_NPA_CFG;
+   block->lfcfg_reg = NPA_PRIV_LFX_CFG;
+   block->msixcfg_reg = NPA_PRIV_LFX_INT_CFG;
+   block->lfreset_reg = NPA_AF_LF_RST;
+   sprintf(block->name, "NPA");
+   err = rvu_alloc_bitmap(>lf);
+   if (err)
+   return err;
+
+nix:
+   /* Init NIX LF's bitmap */
+   block = >block[BLKADDR_NIX0];
+   if (!block->implemented)
+   goto sso;
+   cfg = rvu_read64(rvu, BLKADDR_NIX0, NIX_AF_CONST2);
+   block->lf.max = cfg & 0xFFF;
+   block->addr = BLKADDR_NIX0;
+   block->lfshift = 8;
+   block->lookup_reg = NIX_AF_RVU_LF_CFG_DEBUG;
+   block->pf_lfcnt_reg = RVU_PRIV_PFX_NIX_CFG;
+   block->vf_lfcnt_reg = RVU_PRIV_HWVFX_NIX_CFG;
+   block->lfcfg_reg = NIX_PRIV_LFX_CFG;
+   block->msixcfg_reg = NIX_PRIV_LFX_INT_CFG;
+   block->lfreset_reg = NIX_AF_LF_RST;
+   sprintf(block->name, "NIX");
+   err = rvu_alloc_bitmap(>lf);
+   if (err)
+   return err;
+
+sso:
+   /* Init SSO group's bitmap */
+   block = >block[BLKADDR_SSO];
+   if (!block->implemented)
+   goto ssow;
+   cfg = rvu_read64(rvu, BLKADDR_SSO, SSO_AF_CONST);
+   block->lf.max = cfg & 0x;
+   block->addr = BLKADDR_SSO;
+   block->multislot = true;
+   block->lfshift = 3;
+   block->lookup_reg = SSO_AF_RVU_LF_CFG_DEBUG;
+   block->pf_lfcnt_reg = RVU_PRIV_PFX_SSO_CFG;
+   block->vf_lfcnt_reg = RVU_PRIV_HWVFX_SSO_CFG;
+   block->lfcfg_reg = SSO_PRIV_LFX_HWGRP_CFG;
+   block->msixcfg_reg = SSO_PRIV_LFX_HWGRP_INT_CFG;
+   block->lfreset_reg = SSO_AF_LF_HWGRP_RST;
+   sprintf(block->name, "SSO GROUP");
+   err = rvu_alloc_bitmap(>lf);
+   if (err)
+   return err;
+
+ssow:
+   /* Init SSO workslot's bitmap */
+   block = >block[BLKADDR_SSOW];
+   if (!block->implemented)
+   goto tim;
+   block->lf.max = (cfg >> 56) & 0xFF;
+   block->addr = BLKADDR_SSOW;
+   block->multislot = true;
+   block->lfshift = 3;
+   block->lookup_reg = SSOW_AF_RVU_LF_HWS_CFG_DEBUG;
+   block->pf_lfcnt_reg = RVU_PRIV_PFX_SSOW_CFG;
+   block->vf_lfcnt_reg = RVU_PRIV_HWVFX_SSOW_CFG;
+   block->lfcfg_reg = SSOW_PRIV_LFX_HWS_CFG;
+   block->msixcfg_reg = SSOW_PRIV_LFX_HWS_INT_CFG;
+   block->lfreset_reg = 

[PATCH 01/15] soc: octeontx2: Add Marvell OcteonTX2 RVU AF driver

[sunil . kovvuri]

From: Sunil Goutham 

This patch adds basic template for Marvell OcteonTX2's
resource virtualization unit (RVU) admin function (AF)
driver. Just the driver registration and probe.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/Kconfig|   1 +
 drivers/soc/Makefile   |   1 +
 drivers/soc/marvell/Kconfig|  13 
 drivers/soc/marvell/Makefile   |   2 +
 drivers/soc/marvell/octeontx2/Makefile |   8 +++
 drivers/soc/marvell/octeontx2/rvu.c| 126 +
 drivers/soc/marvell/octeontx2/rvu.h|  31 
 7 files changed, 182 insertions(+)
 create mode 100644 drivers/soc/marvell/Kconfig
 create mode 100644 drivers/soc/marvell/Makefile
 create mode 100644 drivers/soc/marvell/octeontx2/Makefile
 create mode 100644 drivers/soc/marvell/octeontx2/rvu.c
 create mode 100644 drivers/soc/marvell/octeontx2/rvu.h

diff --git a/drivers/soc/Kconfig b/drivers/soc/Kconfig
index c07b4a8..42f2d0b 100644
--- a/drivers/soc/Kconfig
+++ b/drivers/soc/Kconfig
@@ -6,6 +6,7 @@ source "drivers/soc/atmel/Kconfig"
 source "drivers/soc/bcm/Kconfig"
 source "drivers/soc/fsl/Kconfig"
 source "drivers/soc/imx/Kconfig"
+source "drivers/soc/marvell/Kconfig"
 source "drivers/soc/mediatek/Kconfig"
 source "drivers/soc/qcom/Kconfig"
 source "drivers/soc/renesas/Kconfig"
diff --git a/drivers/soc/Makefile b/drivers/soc/Makefile
index 113e884..5e18cbb 100644
--- a/drivers/soc/Makefile
+++ b/drivers/soc/Makefile
@@ -12,6 +12,7 @@ obj-y += fsl/
 obj-$(CONFIG_ARCH_GEMINI)  += gemini/
 obj-$(CONFIG_ARCH_MXC) += imx/
 obj-$(CONFIG_SOC_XWAY) += lantiq/
+obj-y  += marvell/
 obj-y  += mediatek/
 obj-$(CONFIG_ARCH_MESON)   += amlogic/
 obj-y  += qcom/
diff --git a/drivers/soc/marvell/Kconfig b/drivers/soc/marvell/Kconfig
new file mode 100644
index 000..4499caf
--- /dev/null
+++ b/drivers/soc/marvell/Kconfig
@@ -0,0 +1,13 @@
+#
+# MARVELL SoC drivers
+#
+
+menu "Marvell SoC drivers"
+
+config OCTEONTX2_AF
+   tristate "OcteonTX2 RVU Admin Function driver"
+   depends on ARM64 && PCI
+   help
+ This driver supports Marvell's OcteonTX2 Resource Virtualization
+ Unit's admin function manager which manages all RVU HW resources.
+endmenu
diff --git a/drivers/soc/marvell/Makefile b/drivers/soc/marvell/Makefile
new file mode 100644
index 000..16e0ca0
--- /dev/null
+++ b/drivers/soc/marvell/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0
+obj-y  += octeontx2/
diff --git a/drivers/soc/marvell/octeontx2/Makefile 
b/drivers/soc/marvell/octeontx2/Makefile
new file mode 100644
index 000..dacbd16
--- /dev/null
+++ b/drivers/soc/marvell/octeontx2/Makefile
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for Marvell's OcteonTX2 RVU Admin Function driver
+#
+
+obj-$(CONFIG_OCTEONTX2_AF) += octeontx2_af.o
+
+octeontx2_af-y := rvu.o
diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
new file mode 100644
index 000..5af4da6
--- /dev/null
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -0,0 +1,126 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Marvell OcteonTx2 RVU Admin Function driver
+ *
+ * Copyright (C) 2018 Marvell International Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "rvu.h"
+
+#define DRV_NAME   "octeontx2-af"
+#define DRV_STRING  "Marvell OcteonTX2 RVU Admin Function Driver"
+#define DRV_VERSION"1.0"
+
+/* Supported devices */
+static const struct pci_device_id rvu_id_table[] = {
+   { PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_RVU_AF) },
+   { 0, }  /* end of table */
+};
+
+MODULE_AUTHOR("Marvell International Ltd.");
+MODULE_DESCRIPTION(DRV_STRING);
+MODULE_LICENSE("GPL v2");
+MODULE_VERSION(DRV_VERSION);
+MODULE_DEVICE_TABLE(pci, rvu_id_table);
+
+static int rvu_probe(struct pci_dev *pdev, const struct pci_device_id *id)
+{
+   struct device *dev = >dev;
+   struct rvu *rvu;
+   interr;
+
+   rvu = devm_kzalloc(dev, sizeof(*rvu), GFP_KERNEL);
+   if (!rvu)
+   return -ENOMEM;
+
+   pci_set_drvdata(pdev, rvu);
+   rvu->pdev = pdev;
+   rvu->dev = >dev;
+
+   err = pci_enable_device(pdev);
+   if (err) {
+   dev_err(dev, "Failed to enable PCI device\n");
+   goto err_freemem;
+   }
+
+   err = pci_request_regions(pdev, DRV_NAME);
+   if (err) {
+   dev_err(dev, "PCI request regions failed 0x%x\n", err);
+   goto err_disable_device;
+   }
+
+   err = pci_set_dma_mask(pdev, DMA_BIT_MASK(48));
+   if (err) {
+   dev_err(dev, "Unable to set DMA 

[PATCH 07/15] soc: octeontx2: Scan blocks for LFs provisioned to PF/VF

[sunil . kovvuri]

From: Sunil Goutham 

Scan all RVU blocks to find any 'LF to RVU PF/VF' mapping done by
low level firmware. If found any, mark them as used in respective
block's LF bitmap and also save mapped PF/VF's PF_FUNC info.

This is done to avoid reattaching a block LF to a different RVU PF/VF.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/rvu.c| 148 -
 drivers/soc/marvell/octeontx2/rvu.h|  16 
 drivers/soc/marvell/octeontx2/rvu_struct.h |  18 
 3 files changed, 180 insertions(+), 2 deletions(-)

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 25f79bf..9539ab9 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -22,6 +22,8 @@
 #define DRV_STRING  "Marvell OcteonTX2 RVU Admin Function Driver"
 #define DRV_VERSION"1.0"
 
+static int rvu_get_hwvf(struct rvu *rvu, int pcifunc);
+
 /* Supported devices */
 static const struct pci_device_id rvu_id_table[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_CAVIUM, PCI_DEVID_OCTEONTX2_RVU_AF) },
@@ -66,6 +68,91 @@ int rvu_alloc_bitmap(struct rsrc_bmap *rsrc)
return 0;
 }
 
+static void rvu_update_rsrc_map(struct rvu *rvu, struct rvu_pfvf *pfvf,
+   struct rvu_block *block, u16 pcifunc,
+   u16 lf, bool attach)
+{
+   int devnum, num_lfs = 0;
+   bool is_pf;
+   u64 reg;
+
+   if (lf >= block->lf.max) {
+   dev_err(>pdev->dev,
+   "%s: FATAL: LF %d is >= %s's max lfs i.e %d\n",
+   __func__, lf, block->name, block->lf.max);
+   return;
+   }
+
+   /* Check if this is for a RVU PF or VF */
+   if (pcifunc & RVU_PFVF_FUNC_MASK) {
+   is_pf = false;
+   devnum = rvu_get_hwvf(rvu, pcifunc);
+   } else {
+   is_pf = true;
+   devnum = rvu_get_pf(pcifunc);
+   }
+
+   block->fn_map[lf] = attach ? pcifunc : 0;
+
+   switch (block->type) {
+   case BLKTYPE_NPA:
+   pfvf->npalf = attach ? true : false;
+   num_lfs = pfvf->npalf;
+   break;
+   case BLKTYPE_NIX:
+   pfvf->nixlf = attach ? true : false;
+   num_lfs = pfvf->nixlf;
+   break;
+   case BLKTYPE_SSO:
+   attach ? pfvf->sso++ : pfvf->sso--;
+   num_lfs = pfvf->sso;
+   break;
+   case BLKTYPE_SSOW:
+   attach ? pfvf->ssow++ : pfvf->ssow--;
+   num_lfs = pfvf->ssow;
+   break;
+   case BLKTYPE_TIM:
+   attach ? pfvf->timlfs++ : pfvf->timlfs--;
+   num_lfs = pfvf->timlfs;
+   break;
+   case BLKTYPE_CPT:
+   attach ? pfvf->cptlfs++ : pfvf->cptlfs--;
+   num_lfs = pfvf->cptlfs;
+   break;
+   }
+
+   reg = is_pf ? block->pf_lfcnt_reg : block->vf_lfcnt_reg;
+   rvu_write64(rvu, BLKADDR_RVUM, reg | (devnum << 16), num_lfs);
+}
+
+inline int rvu_get_pf(u16 pcifunc)
+{
+   return (pcifunc >> RVU_PFVF_PF_SHIFT) & RVU_PFVF_PF_MASK;
+}
+
+static int rvu_get_hwvf(struct rvu *rvu, int pcifunc)
+{
+   int pf, func;
+   u64 cfg;
+
+   pf = rvu_get_pf(pcifunc);
+   func = pcifunc & RVU_PFVF_FUNC_MASK;
+
+   /* Get first HWVF attached to this PF */
+   cfg = rvu_read64(rvu, BLKADDR_RVUM, RVU_PRIV_PFX_CFG(pf));
+
+   return ((cfg & 0xFFF) + func - 1);
+}
+
+struct rvu_pfvf *rvu_get_pfvf(struct rvu *rvu, int pcifunc)
+{
+   /* Check if it is a PF or VF */
+   if (pcifunc & RVU_PFVF_FUNC_MASK)
+   return >hwvf[rvu_get_hwvf(rvu, pcifunc)];
+   else
+   return >pf[rvu_get_pf(pcifunc)];
+}
+
 static void rvu_check_block_implemented(struct rvu *rvu)
 {
struct rvu_hwinfo *hw = rvu->hw;
@@ -107,6 +194,28 @@ static void rvu_reset_all_blocks(struct rvu *rvu)
rvu_block_reset(rvu, BLKADDR_NDC2, NDC_AF_BLK_RST);
 }
 
+static void rvu_scan_block(struct rvu *rvu, struct rvu_block *block)
+{
+   struct rvu_pfvf *pfvf;
+   u64 cfg;
+   int lf;
+
+   for (lf = 0; lf < block->lf.max; lf++) {
+   cfg = rvu_read64(rvu, block->addr,
+block->lfcfg_reg | (lf << block->lfshift));
+   if (!(cfg & BIT_ULL(63)))
+   continue;
+
+   /* Set this resource as being used */
+   __set_bit(lf, block->lf.bmap);
+
+   /* Get, to whom this LF is attached */
+   pfvf = rvu_get_pfvf(rvu, (cfg >> 8) & 0x);
+   rvu_update_rsrc_map(rvu, pfvf, block,
+   (cfg >> 8) & 0x, lf, true);
+   }
+}
+
 static void rvu_free_hw_resources(struct rvu *rvu)
 {
struct rvu_hwinfo *hw = rvu->hw;
@@ -124,7 +233,7 @@ static int rvu_setup_hw_resources(struct rvu *rvu)
 {
struct rvu_hwinfo *hw = 

[PATCH 02/15] soc: octeontx2: Reset all RVU blocks

[sunil . kovvuri]

From: Sunil Goutham 

Go through all BLKADDRs and check which ones are implemented
on this silicon and do a HW reset of each implemented block.
Also added all RVU AF and PF register offsets.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/rvu.c|  78 
 drivers/soc/marvell/octeontx2/rvu.h|  37 ++
 drivers/soc/marvell/octeontx2/rvu_reg.h| 113 +
 drivers/soc/marvell/octeontx2/rvu_struct.h |  36 +
 4 files changed, 264 insertions(+)
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_reg.h
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_struct.h

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 5af4da6..d40fabf 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -16,6 +16,7 @@
 #include 
 
 #include "rvu.h"
+#include "rvu_reg.h"
 
 #define DRV_NAME   "octeontx2-af"
 #define DRV_STRING  "Marvell OcteonTX2 RVU Admin Function Driver"
@@ -33,6 +34,70 @@ MODULE_LICENSE("GPL v2");
 MODULE_VERSION(DRV_VERSION);
 MODULE_DEVICE_TABLE(pci, rvu_id_table);
 
+/* Poll a RVU block's register 'offset', for a 'zero'
+ * or 'nonzero' at bits specified by 'mask'
+ */
+int rvu_poll_reg(struct rvu *rvu, u64 block, u64 offset, u64 mask, bool zero)
+{
+   void __iomem *reg;
+   int timeout = 100;
+   u64 reg_val;
+
+   reg = rvu->afreg_base + ((block << 28) | offset);
+   while (timeout) {
+   reg_val = readq(reg);
+   if (zero && !(reg_val & mask))
+   return 0;
+   if (!zero && (reg_val & mask))
+   return 0;
+   udelay(1);
+   cpu_relax();
+   timeout--;
+   }
+   return -EBUSY;
+}
+
+static void rvu_check_block_implemented(struct rvu *rvu)
+{
+   struct rvu_hwinfo *hw = rvu->hw;
+   struct rvu_block *block;
+   int blkid;
+   u64 cfg;
+
+   /* For each block check if 'implemented' bit is set */
+   for (blkid = 0; blkid < BLK_COUNT; blkid++) {
+   block = >block[blkid];
+   cfg = rvupf_read64(rvu, RVU_PF_BLOCK_ADDRX_DISC(blkid));
+   if (cfg & BIT_ULL(11))
+   block->implemented = true;
+   }
+}
+
+static void rvu_block_reset(struct rvu *rvu, int blkaddr, u64 rst_reg)
+{
+   struct rvu_block *block = >hw->block[blkaddr];
+
+   if (!block->implemented)
+   return;
+
+   rvu_write64(rvu, blkaddr, rst_reg, BIT_ULL(0));
+   rvu_poll_reg(rvu, blkaddr, rst_reg, BIT_ULL(63), true);
+}
+
+static void rvu_reset_all_blocks(struct rvu *rvu)
+{
+   /* Do a HW reset of all RVU blocks */
+   rvu_block_reset(rvu, BLKADDR_NPA, NPA_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_NIX0, NIX_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_NPC, NPC_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_SSO, SSO_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_TIM, TIM_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_CPT0, CPT_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_NDC0, NDC_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_NDC1, NDC_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_NDC2, NDC_AF_BLK_RST);
+}
+
 static int rvu_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 {
struct device *dev = >dev;
@@ -43,6 +108,12 @@ static int rvu_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
if (!rvu)
return -ENOMEM;
 
+   rvu->hw = devm_kzalloc(dev, sizeof(struct rvu_hwinfo), GFP_KERNEL);
+   if (!rvu->hw) {
+   devm_kfree(dev, rvu);
+   return -ENOMEM;
+   }
+
pci_set_drvdata(pdev, rvu);
rvu->pdev = pdev;
rvu->dev = >dev;
@@ -80,6 +151,11 @@ static int rvu_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
goto err_release_regions;
}
 
+   /* Check which blocks the HW supports */
+   rvu_check_block_implemented(rvu);
+
+   rvu_reset_all_blocks(rvu);
+
return 0;
 
 err_release_regions:
@@ -88,6 +164,7 @@ static int rvu_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
pci_disable_device(pdev);
 err_freemem:
pci_set_drvdata(pdev, NULL);
+   devm_kfree(>dev, rvu->hw);
devm_kfree(dev, rvu);
return err;
 }
@@ -100,6 +177,7 @@ static void rvu_remove(struct pci_dev *pdev)
pci_disable_device(pdev);
pci_set_drvdata(pdev, NULL);
 
+   devm_kfree(>dev, rvu->hw);
devm_kfree(>dev, rvu);
 }
 
diff --git a/drivers/soc/marvell/octeontx2/rvu.h 
b/drivers/soc/marvell/octeontx2/rvu.h
index 4a4b0ad..e2c54d0 100644
--- a/drivers/soc/marvell/octeontx2/rvu.h
+++ b/drivers/soc/marvell/octeontx2/rvu.h
@@ -11,6 +11,8 @@
 #ifndef RVU_H
 #define RVU_H
 
+#include "rvu_struct.h"
+
 /* PCI device IDs */
 #definePCI_DEVID_OCTEONTX2_RVU_AF  0xA065
 
@@ 

[PATCH 00/15] soc: octeontx2: Add RVU admin function driver

[sunil . kovvuri]

From: Sunil Goutham 

Resource virtualization unit (RVU) on Marvell's OcteonTX2 SOC supports
multiple PCIe SRIOV physical functions (PFs) and virtual functions (VFs).
PF0 is called administrative / admin function (AF) and has privilege access
to registers to provision different RVU functional blocks to each of
PF/VF. 

This admin function (AF) driver acts as a configuration / administrative
software which provisions functional blocks to a PF/VF on demand for them
to work as one of the following
 - A basic network controller (i.e NIC).
 - NIC with packet filtering, shaping and scheduling capabilities.
 - A crypto device.
 - A combination of above etc.

PF/VFs communicate with admin function via a shared memory region.
This patch series adds logic for the following
 - RVU AF driver with functional blocks provisioning support
 - Mailbox infrastructure for communication between AF and PFs.
 - CGX driver which provides information about physcial network
   interfaces which AF processes and forwards required info to
   PF/VF drivers.

This is the first set of patches out of 70 odd patches.

Note: This driver neither receives any data nor processes it i.e no I/O,
  just does the hardware configuration.

Aleksey Makarov (2):
  soc: octeontx2: Add mailbox support infra
  soc: octeontx2: Convert mbox msg id check to a macro

Geetha sowjanya (1):
  soc: octeontx2: Reconfig MSIX base with IOVA

Linu Cherian (3):
  soc: octeontx2: Set RVU PFs to CGX LMACs mapping
  soc: octeontx2: Add support for CGX link management
  soc: octeontx2: Register for CGX lmac events

Sunil Goutham (9):
  soc: octeontx2: Add Marvell OcteonTX2 RVU AF driver
  soc: octeontx2: Reset all RVU blocks
  soc: octeontx2: Gather RVU blocks HW info
  soc: octeontx2: Add mailbox IRQ and msg handlers
  soc: octeontx2: Scan blocks for LFs provisioned to PF/VF
  soc: octeontx2: Add RVU block LF provisioning support
  soc: octeontx2: Configure block LF's MSIX vector offset
  soc: octeontx2: Add Marvell OcteonTX2 CGX driver
  MAINTAINERS: Add entry for Marvell OcteonTX2 Admin Function driver

 MAINTAINERS|   10 +
 drivers/soc/Kconfig|1 +
 drivers/soc/Makefile   |1 +
 drivers/soc/marvell/Kconfig|   23 +
 drivers/soc/marvell/Makefile   |2 +
 drivers/soc/marvell/octeontx2/Makefile |   10 +
 drivers/soc/marvell/octeontx2/cgx.c|  529 +
 drivers/soc/marvell/octeontx2/cgx.h|   63 ++
 drivers/soc/marvell/octeontx2/cgx_fw_if.h  |  225 
 drivers/soc/marvell/octeontx2/mbox.c   |  300 +
 drivers/soc/marvell/octeontx2/mbox.h   |  211 
 drivers/soc/marvell/octeontx2/rvu.c| 1625 
 drivers/soc/marvell/octeontx2/rvu.h|  158 +++
 drivers/soc/marvell/octeontx2/rvu_cgx.c|  194 
 drivers/soc/marvell/octeontx2/rvu_reg.h|  442 
 drivers/soc/marvell/octeontx2/rvu_struct.h |   78 ++
 16 files changed, 3872 insertions(+)
 create mode 100644 drivers/soc/marvell/Kconfig
 create mode 100644 drivers/soc/marvell/Makefile
 create mode 100644 drivers/soc/marvell/octeontx2/Makefile
 create mode 100644 drivers/soc/marvell/octeontx2/cgx.c
 create mode 100644 drivers/soc/marvell/octeontx2/cgx.h
 create mode 100644 drivers/soc/marvell/octeontx2/cgx_fw_if.h
 create mode 100644 drivers/soc/marvell/octeontx2/mbox.c
 create mode 100644 drivers/soc/marvell/octeontx2/mbox.h
 create mode 100644 drivers/soc/marvell/octeontx2/rvu.c
 create mode 100644 drivers/soc/marvell/octeontx2/rvu.h
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_cgx.c
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_reg.h
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_struct.h

-- 
2.7.4

[PATCH 02/15] soc: octeontx2: Reset all RVU blocks

[sunil . kovvuri]

From: Sunil Goutham 

Go through all BLKADDRs and check which ones are implemented
on this silicon and do a HW reset of each implemented block.
Also added all RVU AF and PF register offsets.

Signed-off-by: Sunil Goutham 
---
 drivers/soc/marvell/octeontx2/rvu.c|  78 
 drivers/soc/marvell/octeontx2/rvu.h|  37 ++
 drivers/soc/marvell/octeontx2/rvu_reg.h| 113 +
 drivers/soc/marvell/octeontx2/rvu_struct.h |  36 +
 4 files changed, 264 insertions(+)
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_reg.h
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_struct.h

diff --git a/drivers/soc/marvell/octeontx2/rvu.c 
b/drivers/soc/marvell/octeontx2/rvu.c
index 5af4da6..d40fabf 100644
--- a/drivers/soc/marvell/octeontx2/rvu.c
+++ b/drivers/soc/marvell/octeontx2/rvu.c
@@ -16,6 +16,7 @@
 #include 
 
 #include "rvu.h"
+#include "rvu_reg.h"
 
 #define DRV_NAME   "octeontx2-af"
 #define DRV_STRING  "Marvell OcteonTX2 RVU Admin Function Driver"
@@ -33,6 +34,70 @@ MODULE_LICENSE("GPL v2");
 MODULE_VERSION(DRV_VERSION);
 MODULE_DEVICE_TABLE(pci, rvu_id_table);
 
+/* Poll a RVU block's register 'offset', for a 'zero'
+ * or 'nonzero' at bits specified by 'mask'
+ */
+int rvu_poll_reg(struct rvu *rvu, u64 block, u64 offset, u64 mask, bool zero)
+{
+   void __iomem *reg;
+   int timeout = 100;
+   u64 reg_val;
+
+   reg = rvu->afreg_base + ((block << 28) | offset);
+   while (timeout) {
+   reg_val = readq(reg);
+   if (zero && !(reg_val & mask))
+   return 0;
+   if (!zero && (reg_val & mask))
+   return 0;
+   udelay(1);
+   cpu_relax();
+   timeout--;
+   }
+   return -EBUSY;
+}
+
+static void rvu_check_block_implemented(struct rvu *rvu)
+{
+   struct rvu_hwinfo *hw = rvu->hw;
+   struct rvu_block *block;
+   int blkid;
+   u64 cfg;
+
+   /* For each block check if 'implemented' bit is set */
+   for (blkid = 0; blkid < BLK_COUNT; blkid++) {
+   block = >block[blkid];
+   cfg = rvupf_read64(rvu, RVU_PF_BLOCK_ADDRX_DISC(blkid));
+   if (cfg & BIT_ULL(11))
+   block->implemented = true;
+   }
+}
+
+static void rvu_block_reset(struct rvu *rvu, int blkaddr, u64 rst_reg)
+{
+   struct rvu_block *block = >hw->block[blkaddr];
+
+   if (!block->implemented)
+   return;
+
+   rvu_write64(rvu, blkaddr, rst_reg, BIT_ULL(0));
+   rvu_poll_reg(rvu, blkaddr, rst_reg, BIT_ULL(63), true);
+}
+
+static void rvu_reset_all_blocks(struct rvu *rvu)
+{
+   /* Do a HW reset of all RVU blocks */
+   rvu_block_reset(rvu, BLKADDR_NPA, NPA_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_NIX0, NIX_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_NPC, NPC_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_SSO, SSO_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_TIM, TIM_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_CPT0, CPT_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_NDC0, NDC_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_NDC1, NDC_AF_BLK_RST);
+   rvu_block_reset(rvu, BLKADDR_NDC2, NDC_AF_BLK_RST);
+}
+
 static int rvu_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 {
struct device *dev = >dev;
@@ -43,6 +108,12 @@ static int rvu_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
if (!rvu)
return -ENOMEM;
 
+   rvu->hw = devm_kzalloc(dev, sizeof(struct rvu_hwinfo), GFP_KERNEL);
+   if (!rvu->hw) {
+   devm_kfree(dev, rvu);
+   return -ENOMEM;
+   }
+
pci_set_drvdata(pdev, rvu);
rvu->pdev = pdev;
rvu->dev = >dev;
@@ -80,6 +151,11 @@ static int rvu_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
goto err_release_regions;
}
 
+   /* Check which blocks the HW supports */
+   rvu_check_block_implemented(rvu);
+
+   rvu_reset_all_blocks(rvu);
+
return 0;
 
 err_release_regions:
@@ -88,6 +164,7 @@ static int rvu_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
pci_disable_device(pdev);
 err_freemem:
pci_set_drvdata(pdev, NULL);
+   devm_kfree(>dev, rvu->hw);
devm_kfree(dev, rvu);
return err;
 }
@@ -100,6 +177,7 @@ static void rvu_remove(struct pci_dev *pdev)
pci_disable_device(pdev);
pci_set_drvdata(pdev, NULL);
 
+   devm_kfree(>dev, rvu->hw);
devm_kfree(>dev, rvu);
 }
 
diff --git a/drivers/soc/marvell/octeontx2/rvu.h 
b/drivers/soc/marvell/octeontx2/rvu.h
index 4a4b0ad..e2c54d0 100644
--- a/drivers/soc/marvell/octeontx2/rvu.h
+++ b/drivers/soc/marvell/octeontx2/rvu.h
@@ -11,6 +11,8 @@
 #ifndef RVU_H
 #define RVU_H
 
+#include "rvu_struct.h"
+
 /* PCI device IDs */
 #definePCI_DEVID_OCTEONTX2_RVU_AF  0xA065
 
@@ 

[PATCH 00/15] soc: octeontx2: Add RVU admin function driver

[sunil . kovvuri]

From: Sunil Goutham 

Resource virtualization unit (RVU) on Marvell's OcteonTX2 SOC supports
multiple PCIe SRIOV physical functions (PFs) and virtual functions (VFs).
PF0 is called administrative / admin function (AF) and has privilege access
to registers to provision different RVU functional blocks to each of
PF/VF. 

This admin function (AF) driver acts as a configuration / administrative
software which provisions functional blocks to a PF/VF on demand for them
to work as one of the following
 - A basic network controller (i.e NIC).
 - NIC with packet filtering, shaping and scheduling capabilities.
 - A crypto device.
 - A combination of above etc.

PF/VFs communicate with admin function via a shared memory region.
This patch series adds logic for the following
 - RVU AF driver with functional blocks provisioning support
 - Mailbox infrastructure for communication between AF and PFs.
 - CGX driver which provides information about physcial network
   interfaces which AF processes and forwards required info to
   PF/VF drivers.

This is the first set of patches out of 70 odd patches.

Note: This driver neither receives any data nor processes it i.e no I/O,
  just does the hardware configuration.

Aleksey Makarov (2):
  soc: octeontx2: Add mailbox support infra
  soc: octeontx2: Convert mbox msg id check to a macro

Geetha sowjanya (1):
  soc: octeontx2: Reconfig MSIX base with IOVA

Linu Cherian (3):
  soc: octeontx2: Set RVU PFs to CGX LMACs mapping
  soc: octeontx2: Add support for CGX link management
  soc: octeontx2: Register for CGX lmac events

Sunil Goutham (9):
  soc: octeontx2: Add Marvell OcteonTX2 RVU AF driver
  soc: octeontx2: Reset all RVU blocks
  soc: octeontx2: Gather RVU blocks HW info
  soc: octeontx2: Add mailbox IRQ and msg handlers
  soc: octeontx2: Scan blocks for LFs provisioned to PF/VF
  soc: octeontx2: Add RVU block LF provisioning support
  soc: octeontx2: Configure block LF's MSIX vector offset
  soc: octeontx2: Add Marvell OcteonTX2 CGX driver
  MAINTAINERS: Add entry for Marvell OcteonTX2 Admin Function driver

 MAINTAINERS|   10 +
 drivers/soc/Kconfig|1 +
 drivers/soc/Makefile   |1 +
 drivers/soc/marvell/Kconfig|   23 +
 drivers/soc/marvell/Makefile   |2 +
 drivers/soc/marvell/octeontx2/Makefile |   10 +
 drivers/soc/marvell/octeontx2/cgx.c|  529 +
 drivers/soc/marvell/octeontx2/cgx.h|   63 ++
 drivers/soc/marvell/octeontx2/cgx_fw_if.h  |  225 
 drivers/soc/marvell/octeontx2/mbox.c   |  300 +
 drivers/soc/marvell/octeontx2/mbox.h   |  211 
 drivers/soc/marvell/octeontx2/rvu.c| 1625 
 drivers/soc/marvell/octeontx2/rvu.h|  158 +++
 drivers/soc/marvell/octeontx2/rvu_cgx.c|  194 
 drivers/soc/marvell/octeontx2/rvu_reg.h|  442 
 drivers/soc/marvell/octeontx2/rvu_struct.h |   78 ++
 16 files changed, 3872 insertions(+)
 create mode 100644 drivers/soc/marvell/Kconfig
 create mode 100644 drivers/soc/marvell/Makefile
 create mode 100644 drivers/soc/marvell/octeontx2/Makefile
 create mode 100644 drivers/soc/marvell/octeontx2/cgx.c
 create mode 100644 drivers/soc/marvell/octeontx2/cgx.h
 create mode 100644 drivers/soc/marvell/octeontx2/cgx_fw_if.h
 create mode 100644 drivers/soc/marvell/octeontx2/mbox.c
 create mode 100644 drivers/soc/marvell/octeontx2/mbox.h
 create mode 100644 drivers/soc/marvell/octeontx2/rvu.c
 create mode 100644 drivers/soc/marvell/octeontx2/rvu.h
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_cgx.c
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_reg.h
 create mode 100644 drivers/soc/marvell/octeontx2/rvu_struct.h

-- 
2.7.4

Re: net: thunder: change q_len's type to handle max ring size

[Sunil Kovvuri]

On Fri, Feb 9, 2018 at 3:27 AM, Dean Nelson  wrote:
> On 02/08/2018 02:34 PM, David Miller wrote:
>>
>> From: Dean Nelson 
>> Date:
>>
>>> The Cavium thunder nicvf driver supports rx/tx rings of up to 65536
>>> entries per.
>>> The number of entires are stored in the q_len member of struct
>>> q_desc_mem. The
>>> problem is that q_len being a u16, results in 65536 becoming 0.
>>>
>>> In getting pointers to descriptors in the rings, the driver uses q_len
>>> minus 1
>>> as a mask after incrementing the pointer, in order to go back to the
>>> beginning
>>> and not go past the end of the ring.
>>>
>>> With the q_len set to 0 the mask is no longer correct and the driver does
>>> go
>>> beyond the end of the ring, causing various ills. Usually the first thing
>>> that
>>> shows up is a "NETDEV WATCHDOG: enP2p1s0f1 (nicvf): transmit queue 7
>>> timed out"
>>> warning.
>>>
>>> This patch remedies the problem by changing q_len to a u32.
>>>
>>> Signed-off-by: Dean Nelson 
>>
>>
>> Applied, thanks.
>
>
> Thank you!
>
>>
>> Another way to solve this could have been to encode that length
>> as "length - 1"
>
>
> True. I had pondered that, but felt that since changing q_len's type
> didn't add any length to the structure and that it was less impactful
> from a number-of-lines of code changed perspective, I'd opt for this
> route.
>
> Cavium, if you'd prefer this goes the route that Dave just mentioned,
> please let me know and I can make a new patch against what's been
> applied?

Thanks for fixing this and i think the current patch is fine.

Thanks,
Sunil.

>
> Thanks,
> Dean
>
>
>
>
>
> ___
> linux-arm-kernel mailing list
> linux-arm-ker...@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: net: thunder: change q_len's type to handle max ring size

[Sunil Kovvuri]

On Fri, Feb 9, 2018 at 3:27 AM, Dean Nelson  wrote:
> On 02/08/2018 02:34 PM, David Miller wrote:
>>
>> From: Dean Nelson 
>> Date:
>>
>>> The Cavium thunder nicvf driver supports rx/tx rings of up to 65536
>>> entries per.
>>> The number of entires are stored in the q_len member of struct
>>> q_desc_mem. The
>>> problem is that q_len being a u16, results in 65536 becoming 0.
>>>
>>> In getting pointers to descriptors in the rings, the driver uses q_len
>>> minus 1
>>> as a mask after incrementing the pointer, in order to go back to the
>>> beginning
>>> and not go past the end of the ring.
>>>
>>> With the q_len set to 0 the mask is no longer correct and the driver does
>>> go
>>> beyond the end of the ring, causing various ills. Usually the first thing
>>> that
>>> shows up is a "NETDEV WATCHDOG: enP2p1s0f1 (nicvf): transmit queue 7
>>> timed out"
>>> warning.
>>>
>>> This patch remedies the problem by changing q_len to a u32.
>>>
>>> Signed-off-by: Dean Nelson 
>>
>>
>> Applied, thanks.
>
>
> Thank you!
>
>>
>> Another way to solve this could have been to encode that length
>> as "length - 1"
>
>
> True. I had pondered that, but felt that since changing q_len's type
> didn't add any length to the structure and that it was less impactful
> from a number-of-lines of code changed perspective, I'd opt for this
> route.
>
> Cavium, if you'd prefer this goes the route that Dave just mentioned,
> please let me know and I can make a new patch against what's been
> applied?

Thanks for fixing this and i think the current patch is fine.

Thanks,
Sunil.

>
> Thanks,
> Dean
>
>
>
>
>
> ___
> linux-arm-kernel mailing list
> linux-arm-ker...@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH net] net: thunderx: Fix TCP/UDP checksum offload for IPv6 pkts

[Sunil Kovvuri]

On Wed, Nov 22, 2017 at 9:27 PM, Eric Dumazet  wrote:
> On Wed, 2017-11-22 at 15:37 +0300, Aleksey Makarov wrote:
>> From: Sunil Goutham 
>>
>> This fixes a previous patch which missed some changes
>> and due to which L3 checksum offload was getting enabled
>> for IPv6 pkts. And HW is dropping these pkts as it assumes
>> the pkt is IPv4 when IP csum offload is set in the SQ
>> descriptor.
>>
>> Fixes: 494fd005 ("net: thunderx: Enable TSO and checksum offloads
>> for ipv6")
>> Signed-off-by: Sunil Goutham 
>> Signed-off-by: Aleksey Makarov 
>> ---
>>  drivers/net/ethernet/cavium/thunder/nicvf_queues.c | 3 ++-
>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
>> b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
>> index d4496e9afcdf..184d5bdbe7e0 100644
>> --- a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
>> +++ b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
>> @@ -1355,10 +1355,11 @@ nicvf_sq_add_hdr_subdesc(struct nicvf *nic,
>> struct snd_queue *sq, int qentry,
>>
>>   /* Offload checksum calculation to HW */
>>   if (skb->ip_summed == CHECKSUM_PARTIAL) {
>> - hdr->csum_l3 = 1; /* Enable IP csum calculation */
>>   hdr->l3_offset = skb_network_offset(skb);
>>   hdr->l4_offset = skb_transport_offset(skb);
>>
>> + /* Enable IP HDR csum calculation for V4 pkts */
>> + hdr->csum_l3 = (ip.v4->version == 4) ? 1 : 0;
>
> Have you tried to set hdr->csum_l3 to 0 regardless of version being 4
> or 6 ?
>
> This would remove the need for yet another conditional.
>
> AFAIK, linux does not offload IPv4 header checksums to NIC, it is not
> worth the trouble.

Looks like I misunderstood the IPSUM netdev feature flag.
Thanks, will check.

Sunil.

>
>
>
> ___
> linux-arm-kernel mailing list
> linux-arm-ker...@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH net] net: thunderx: Fix TCP/UDP checksum offload for IPv6 pkts

[Sunil Kovvuri]

On Wed, Nov 22, 2017 at 9:27 PM, Eric Dumazet  wrote:
> On Wed, 2017-11-22 at 15:37 +0300, Aleksey Makarov wrote:
>> From: Sunil Goutham 
>>
>> This fixes a previous patch which missed some changes
>> and due to which L3 checksum offload was getting enabled
>> for IPv6 pkts. And HW is dropping these pkts as it assumes
>> the pkt is IPv4 when IP csum offload is set in the SQ
>> descriptor.
>>
>> Fixes: 494fd005 ("net: thunderx: Enable TSO and checksum offloads
>> for ipv6")
>> Signed-off-by: Sunil Goutham 
>> Signed-off-by: Aleksey Makarov 
>> ---
>>  drivers/net/ethernet/cavium/thunder/nicvf_queues.c | 3 ++-
>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
>> b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
>> index d4496e9afcdf..184d5bdbe7e0 100644
>> --- a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
>> +++ b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
>> @@ -1355,10 +1355,11 @@ nicvf_sq_add_hdr_subdesc(struct nicvf *nic,
>> struct snd_queue *sq, int qentry,
>>
>>   /* Offload checksum calculation to HW */
>>   if (skb->ip_summed == CHECKSUM_PARTIAL) {
>> - hdr->csum_l3 = 1; /* Enable IP csum calculation */
>>   hdr->l3_offset = skb_network_offset(skb);
>>   hdr->l4_offset = skb_transport_offset(skb);
>>
>> + /* Enable IP HDR csum calculation for V4 pkts */
>> + hdr->csum_l3 = (ip.v4->version == 4) ? 1 : 0;
>
> Have you tried to set hdr->csum_l3 to 0 regardless of version being 4
> or 6 ?
>
> This would remove the need for yet another conditional.
>
> AFAIK, linux does not offload IPv4 header checksums to NIC, it is not
> worth the trouble.

Looks like I misunderstood the IPSUM netdev feature flag.
Thanks, will check.

Sunil.

>
>
>
> ___
> linux-arm-kernel mailing list
> linux-arm-ker...@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel