The Gemini ethernet has been around for years as an out-of-tree patch used with the NAS boxen and routers built on StorLink SL3512 and SL3516, later Storm Semiconductor, later Cortina Systems. These ASICs are still being deployed and brand new off-the-shelf systems using it can easily be acquired.
Gemini is the common codename used for all the SoCs using this IP. An earlier codename was "Lepus". Cc: Tobias Waldvogel <tobias.waldvo...@gmail.com> Signed-off-by: Michał Mirosław <mirq-li...@rere.qmqm.pl> Signed-off-by: Linus Walleij <linus.wall...@linaro.org> --- The latest v6 incarnation of this driver was written by Michał Mirosław and submitted for inclusion in 2011. This was the last post: https://lwn.net/Articles/437889/ DaveM ACKed it at the time: https://marc.info/?l=linux-netdev&m=130255434310315&w=2 The controversial pieces under ARM (board files) and other subsystems are now gone and replaced by DeviceTree. Michał: I hope you don't mind me picking it up and hope you can still test it on your ICYbox, I have device tree patches in my tree: https://git.kernel.org/pub/scm/linux/kernel/git/linusw/linux-nomadik.git/log/?h=gemini-ethernet Changes from v6: - Drop all arch support code using the old board files. - Adapted for device tree probing - Getting all resources using devm_* accessors where applicable - Split in parent ethernet device and two per-port devices that get spawn from the parent. This is necessary with device tree and other aspects of the PHY device model and device tree structure that requires a 1:1 mapping between a device and PHY to work properly. - Grab clocks and reset handles as resources from the clock and reset subsystems infrastructure instead of open coding access to system devices. - Let the pin control subsystem deal with setting up the multplexing and clock skew/delay settings of the RGMII lines. - A separate SoC driver was created to deal with setting up bus arbitration and will be merged separately. - Tested with the D-Link DNS-313 NAS box with a Realtek RTL8211B transciever. - Rename and move code around to fit better with the new device handling with a top level device and two children. - Order code as net vendor Cortina and adapter Gemini. We have confirmed with Faraday that this network device is not from them (which was initially suspected). - Rebased onto v4.15-rc1 Changes from v5: - merge arch setup code into the patch - move platform data include to include/linux/platform_data/gemini_gmac.h - use new hw_features instead of ethtool_ops for offload setting - add some #ifdefs for build testing on other arches - a bit of cleanups Changes from v4: - rebased on upcoming 2.6.38 (removal of page_to_dma() and per-txq stats) - removed setting last_rx and trans_start as that's handled by net core - changed __raw_read/writel() to read/writel() - added setting of AHB_WEIGHT register (didn't improve anything, I'm afraid) - fixed DMA unmapping bug - added limit of packet size for TX offload (HW checks only 13 bits of mtu_size field) - reduced RX_MAX_ALLOC_ORDER as it caused a lot of order 4 allocation failures under load - cleanups Changes from v3: - fixed remaining tx_queue_len misuse bugs - bulk RX DMA page map/unmap - whitespace changes to make checkpatch happier (please ignore remaining complaints - long lines in .c and typedefs/whitespace/long lines in .h) Changes from v2: - converted to page buffers and napi_gro_frags() - later IRQ acking and NAPI exits - larger rings by default - tx-interrupt coalescing - MTU changing - jumbo frames support - ringparam and coalesce settings via ethtool - more fixes/cleanups Changes from v1: - fixed stats (now using u64_stats_sync; no-op on UP anyway) - pre-load mdio-gpio if built as module - disable TX checksum offload by default (unreliable HW) - convert to NAPI+GRO (netperf TCP STREAM RX test: before: 156mbit/s, now: 185mbit/s) Later TODO: - netpoll (netconsole) - parse MAC address from flash settings and pass it through platform data - move TX completion to NAPI poll - implement rx copybreak - remove DMA API abuse on RX (large map, small unmaps) - better test multicast support --- MAINTAINERS | 2 + drivers/net/ethernet/Kconfig | 1 + drivers/net/ethernet/Makefile | 1 + drivers/net/ethernet/cortina/Kconfig | 24 + drivers/net/ethernet/cortina/Makefile | 4 + drivers/net/ethernet/cortina/gemini.c | 2461 +++++++++++++++++++++++++++++++++ drivers/net/ethernet/cortina/gemini.h | 1432 +++++++++++++++++++ 7 files changed, 3925 insertions(+) create mode 100644 drivers/net/ethernet/cortina/Kconfig create mode 100644 drivers/net/ethernet/cortina/Makefile create mode 100644 drivers/net/ethernet/cortina/gemini.c create mode 100644 drivers/net/ethernet/cortina/gemini.h diff --git a/MAINTAINERS b/MAINTAINERS index aa71ab52fd76..200ff7670276 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1326,8 +1326,10 @@ T: git git://github.com/ulli-kroll/linux.git S: Maintained F: Documentation/devicetree/bindings/arm/gemini.txt F: Documentation/devicetree/bindings/pinctrl/cortina,gemini-pinctrl.txt +F: Documentation/devicetree/bindings/net/cortina,gemini-ethernet.txt F: Documentation/devicetree/bindings/rtc/faraday,ftrtc010.txt F: arch/arm/mach-gemini/ +F: drivers/net/ethernet/cortina/gemini/* F: drivers/pinctrl/pinctrl-gemini.c F: drivers/rtc/rtc-ftrtc010.c diff --git a/drivers/net/ethernet/Kconfig b/drivers/net/ethernet/Kconfig index c60421339a98..f02727175857 100644 --- a/drivers/net/ethernet/Kconfig +++ b/drivers/net/ethernet/Kconfig @@ -42,6 +42,7 @@ source "drivers/net/ethernet/cavium/Kconfig" source "drivers/net/ethernet/chelsio/Kconfig" source "drivers/net/ethernet/cirrus/Kconfig" source "drivers/net/ethernet/cisco/Kconfig" +source "drivers/net/ethernet/cortina/Kconfig" config CX_ECAT tristate "Beckhoff CX5020 EtherCAT master support" diff --git a/drivers/net/ethernet/Makefile b/drivers/net/ethernet/Makefile index 39f6273358ed..1b356f6ec87a 100644 --- a/drivers/net/ethernet/Makefile +++ b/drivers/net/ethernet/Makefile @@ -29,6 +29,7 @@ obj-$(CONFIG_NET_VENDOR_CAVIUM) += cavium/ obj-$(CONFIG_NET_VENDOR_CHELSIO) += chelsio/ obj-$(CONFIG_NET_VENDOR_CIRRUS) += cirrus/ obj-$(CONFIG_NET_VENDOR_CISCO) += cisco/ +obj-$(CONFIG_NET_VENDOR_CORTINA) += cortina/ obj-$(CONFIG_CX_ECAT) += ec_bhf.o obj-$(CONFIG_DM9000) += davicom/ obj-$(CONFIG_DNET) += dnet.o diff --git a/drivers/net/ethernet/cortina/Kconfig b/drivers/net/ethernet/cortina/Kconfig new file mode 100644 index 000000000000..7d279ac4357d --- /dev/null +++ b/drivers/net/ethernet/cortina/Kconfig @@ -0,0 +1,24 @@ +# SPDX-License-Identifier: GPL-2.0 +# Cortina ethernet devices + +config NET_VENDOR_CORTINA + bool "Cortina Gemini devices" + depends on (ARM || COMPILE_TEST) + default ARCH_GEMINI + ---help--- + If you have a network (Ethernet) card belonging to this class, say Y + and read the Ethernet-HOWTO, available from + <http://www.tldp.org/docs.html#howto>. + +if NET_VENDOR_CORTINA + +config GEMINI_ETHERNET + tristate "Gemini Gigabit Ethernet support" + depends on ARCH_GEMINI + depends on OF + select PHYLIB + select CRC32 + ---help--- + This driver supports StorLink SL351x (Gemini) dual Gigabit Ethernet. + +endif # NET_VENDOR_CORTINA diff --git a/drivers/net/ethernet/cortina/Makefile b/drivers/net/ethernet/cortina/Makefile new file mode 100644 index 000000000000..4e86d398a89c --- /dev/null +++ b/drivers/net/ethernet/cortina/Makefile @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0 +# Makefile for the Cortina Gemini network device drivers. + +obj-$(CONFIG_GEMINI_ETHERNET) += gemini.o diff --git a/drivers/net/ethernet/cortina/gemini.c b/drivers/net/ethernet/cortina/gemini.c new file mode 100644 index 000000000000..49ca395248b9 --- /dev/null +++ b/drivers/net/ethernet/cortina/gemini.c @@ -0,0 +1,2461 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Ethernet device driver for Cortina Systems Gemini SoC + * Also known as the StorLink SL3512 and SL3516 (SL351x) GMAC + * + * Authors: + * Linus Walleij <linus.wall...@linaro.org> + * Tobias Waldvogel <tobias.waldvo...@gmail.com> (OpenWRT) + * MichaÅ MirosÅaw <mirq-li...@rere.qmqm.pl> + * Paulius Zaleckas <paulius.zalec...@gmail.com> + * Giuseppe De Robertis <giuseppe.derober...@ba.infn.it> + * Gary Chen & Ch Hsu Storlink Semiconductor + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + */ + +#include <linux/module.h> +#include <linux/kernel.h> +#include <linux/init.h> + +#include <linux/spinlock.h> +#include <linux/slab.h> +#include <linux/dma-mapping.h> +#include <linux/cache.h> +#include <linux/interrupt.h> +#include <linux/reset.h> +#include <linux/clk.h> +#include <linux/of.h> +#include <linux/of_mdio.h> +#include <linux/of_net.h> +#include <linux/of_platform.h> +#include <linux/platform_device.h> +#include <linux/etherdevice.h> +#include <linux/if_vlan.h> +#include <linux/skbuff.h> +#include <linux/phy.h> +#include <linux/crc32.h> +#include <linux/ethtool.h> +#include <linux/tcp.h> +#include <linux/u64_stats_sync.h> + +#include <linux/in.h> +#include <linux/ip.h> +#include <linux/ipv6.h> + +#include "gemini.h" + +#define DRV_NAME "gmac-gemini" +#define DRV_VERSION "1.0" + +#define HSIZE_8 0b00 +#define HSIZE_16 0b01 +#define HSIZE_32 0b10 + +#define HBURST_SINGLE 0b00 +#define HBURST_INCR 0b01 +#define HBURST_INCR4 0b10 +#define HBURST_INCR8 0b11 + +#define HPROT_DATA_CACHE BIT(0) +#define HPROT_PRIVILIGED BIT(1) +#define HPROT_BUFFERABLE BIT(2) +#define HPROT_CACHABLE BIT(3) + +#define DEFAULT_RX_COALESCE_NSECS 0 +#define DEFAULT_GMAC_RXQ_ORDER 9 +#define DEFAULT_GMAC_TXQ_ORDER 8 +#define DEFAULT_RX_BUF_ORDER 11 +#define DEFAULT_NAPI_WEIGHT 64 +#define TX_MAX_FRAGS 16 +#define TX_QUEUE_NUM 1 /* max: 6 */ +#define RX_MAX_ALLOC_ORDER 2 + +#define GMAC0_IRQ0_2 (GMAC0_TXDERR_INT_BIT|GMAC0_TXPERR_INT_BIT| \ + GMAC0_RXDERR_INT_BIT|GMAC0_RXPERR_INT_BIT) +#define GMAC0_IRQ0_TXQ0_INTS (GMAC0_SWTQ00_EOF_INT_BIT| \ + GMAC0_SWTQ00_FIN_INT_BIT) +#define GMAC0_IRQ4_8 (GMAC0_MIB_INT_BIT|GMAC0_RX_OVERRUN_INT_BIT) + +#define GMAC_OFFLOAD_FEATURES (NETIF_F_SG | NETIF_F_IP_CSUM | \ + NETIF_F_IPV6_CSUM | NETIF_F_RXCSUM | \ + NETIF_F_TSO | NETIF_F_TSO_ECN | NETIF_F_TSO6) + +struct gmac_txq { + GMAC_TXDESC_T *ring; + struct sk_buff **skb; + unsigned int cptr; + unsigned int noirq_packets; +}; + +struct gemini_ethernet; + +struct gemini_ethernet_port { + unsigned int id; /* 0 or 1 */ + + struct gemini_ethernet *geth; + struct net_device *netdev; + struct device *dev; + void __iomem *dma_base; + void __iomem *gmac_base; + struct clk *pclk; + struct reset_control *reset; + int irq; + __le32 mac_addr[3]; + + void __iomem *rxq_rwptr; + GMAC_RXDESC_T *rxq_ring; + unsigned int rxq_order; + + struct napi_struct napi; + struct hrtimer rx_coalesce_timer; + unsigned int rx_coalesce_nsecs; + unsigned int freeq_refill; + struct gmac_txq txq[TX_QUEUE_NUM]; + unsigned int txq_order; + unsigned int irq_every_tx_packets; + + dma_addr_t rxq_dma_base; + dma_addr_t txq_dma_base; + + unsigned int msg_enable; + spinlock_t config_lock; + + struct u64_stats_sync tx_stats_syncp; + struct u64_stats_sync rx_stats_syncp; + struct u64_stats_sync ir_stats_syncp; + + struct rtnl_link_stats64 stats; + u64 hw_stats[RX_STATS_NUM]; + u64 rx_stats[RX_STATUS_NUM]; + u64 rx_csum_stats[RX_CHKSUM_NUM]; + u64 rx_napi_exits; + u64 tx_frag_stats[TX_MAX_FRAGS]; + u64 tx_frags_linearized; + u64 tx_hw_csummed; +}; + +struct gemini_ethernet { + struct device *dev; + void __iomem *base; + struct gemini_ethernet_port *port0; + struct gemini_ethernet_port *port1; + + spinlock_t irq_lock; + unsigned int freeq_order; + unsigned int freeq_frag_order; + GMAC_RXDESC_T *freeq_ring; + dma_addr_t freeq_dma_base; + struct page **freeq_page_tab; + spinlock_t freeq_lock; +}; + +#define GMAC_STATS_NUM ( \ + RX_STATS_NUM + RX_STATUS_NUM + RX_CHKSUM_NUM + 1 + \ + TX_MAX_FRAGS + 2) + +static const char gmac_stats_strings[GMAC_STATS_NUM][ETH_GSTRING_LEN] = { + "GMAC_IN_DISCARDS", + "GMAC_IN_ERRORS", + "GMAC_IN_MCAST", + "GMAC_IN_BCAST", + "GMAC_IN_MAC1", + "GMAC_IN_MAC2", + "RX_STATUS_GOOD_FRAME", + "RX_STATUS_TOO_LONG_GOOD_CRC", + "RX_STATUS_RUNT_FRAME", + "RX_STATUS_SFD_NOT_FOUND", + "RX_STATUS_CRC_ERROR", + "RX_STATUS_TOO_LONG_BAD_CRC", + "RX_STATUS_ALIGNMENT_ERROR", + "RX_STATUS_TOO_LONG_BAD_ALIGN", + "RX_STATUS_RX_ERR", + "RX_STATUS_DA_FILTERED", + "RX_STATUS_BUFFER_FULL", + "RX_STATUS_11", + "RX_STATUS_12", + "RX_STATUS_13", + "RX_STATUS_14", + "RX_STATUS_15", + "RX_CHKSUM_IP_UDP_TCP_OK", + "RX_CHKSUM_IP_OK_ONLY", + "RX_CHKSUM_NONE", + "RX_CHKSUM_3", + "RX_CHKSUM_IP_ERR_UNKNOWN", + "RX_CHKSUM_IP_ERR", + "RX_CHKSUM_TCP_UDP_ERR", + "RX_CHKSUM_7", + "RX_NAPI_EXITS", + "TX_FRAGS[1]", + "TX_FRAGS[2]", + "TX_FRAGS[3]", + "TX_FRAGS[4]", + "TX_FRAGS[5]", + "TX_FRAGS[6]", + "TX_FRAGS[7]", + "TX_FRAGS[8]", + "TX_FRAGS[9]", + "TX_FRAGS[10]", + "TX_FRAGS[11]", + "TX_FRAGS[12]", + "TX_FRAGS[13]", + "TX_FRAGS[14]", + "TX_FRAGS[15]", + "TX_FRAGS[16+]", + "TX_FRAGS_LINEARIZED", + "TX_HW_CSUMMED", +}; + +static void gmac_dump_dma_state(struct net_device *netdev); + +static void gmac_update_config0_reg(struct net_device *netdev, u32 val, u32 vmask) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + unsigned long flags; + u32 reg; + + spin_lock_irqsave(&port->config_lock, flags); + + reg = readl(port->gmac_base + GMAC_CONFIG0); + reg = (reg & ~vmask) | val; + writel(reg, port->gmac_base + GMAC_CONFIG0); + + spin_unlock_irqrestore(&port->config_lock, flags); +} + +static void gmac_enable_tx_rx(struct net_device *netdev) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + void __iomem *config0 = port->gmac_base + GMAC_CONFIG0; + unsigned long flags; + u32 reg; + + spin_lock_irqsave(&port->config_lock, flags); + + reg = readl(config0); + reg &= ~CONFIG0_TX_RX_DISABLE; + writel(reg, config0); + + spin_unlock_irqrestore(&port->config_lock, flags); +} + +static void gmac_disable_tx_rx(struct net_device *netdev) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + void __iomem *config0 = port->gmac_base + GMAC_CONFIG0; + unsigned long flags; + u32 reg; + + spin_lock_irqsave(&port->config_lock, flags); + + reg = readl(config0); + reg |= CONFIG0_TX_RX_DISABLE; + writel(reg, config0); + + spin_unlock_irqrestore(&port->config_lock, flags); + + mdelay(10); /* let GMAC consume packet */ +} + +static void gmac_set_flow_control(struct net_device *netdev, bool tx, bool rx) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + void __iomem *config0 = port->gmac_base + GMAC_CONFIG0; + unsigned long flags; + u32 reg; + + spin_lock_irqsave(&port->config_lock, flags); + + reg = readl(config0); + reg &= ~CONFIG0_FLOW_CTL; + if (tx) + reg |= CONFIG0_FLOW_TX; + if (rx) + reg |= CONFIG0_FLOW_RX; + writel(reg, config0); + + spin_unlock_irqrestore(&port->config_lock, flags); +} + +static void gmac_speed_set(struct net_device *netdev) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + void __iomem *status_reg = port->gmac_base + GMAC_STATUS; + struct phy_device *phydev = netdev->phydev; + GMAC_STATUS_T status, old_status; + int pause_tx=0, pause_rx=0; + + old_status.bits32 = status.bits32 = readl(status_reg); + + status.bits.link = phydev->link; + status.bits.duplex = phydev->duplex; + + switch (phydev->speed) { + case 1000: + status.bits.speed = GMAC_SPEED_1000; + if (phydev->interface == PHY_INTERFACE_MODE_RGMII) + status.bits.mii_rmii = GMAC_PHY_RGMII_1000; + netdev_info(netdev, "connect to RGMII @ 1Gbit\n"); + break; + case 100: + status.bits.speed = GMAC_SPEED_100; + if (phydev->interface == PHY_INTERFACE_MODE_RGMII) + status.bits.mii_rmii = GMAC_PHY_RGMII_100_10; + netdev_info(netdev, "connect to RGMII @ 100 Mbit\n"); + break; + case 10: + status.bits.speed = GMAC_SPEED_10; + if (phydev->interface == PHY_INTERFACE_MODE_RGMII) + status.bits.mii_rmii = GMAC_PHY_RGMII_100_10; + netdev_info(netdev, "connect to RGMII @ 10 Mbit\n"); + break; + default: + netdev_warn(netdev, "Not supported PHY speed (%d)\n", + phydev->speed); + } + + if (phydev->duplex == DUPLEX_FULL) { + u16 lcladv = phy_read(phydev, MII_ADVERTISE); + u16 rmtadv = phy_read(phydev, MII_LPA); + u8 cap = mii_resolve_flowctrl_fdx(lcladv, rmtadv); + + if (cap & FLOW_CTRL_RX) + pause_rx=1; + if (cap & FLOW_CTRL_TX) + pause_tx=1; + } + + gmac_set_flow_control(netdev, pause_tx, pause_rx); + + if (old_status.bits32 == status.bits32) + return; + + if (netif_msg_link(port)) { + phy_print_status(phydev); + netdev_info(netdev, "link flow control: %s\n", + phydev->pause + ? (phydev->asym_pause ? "tx" : "both") + : (phydev->asym_pause ? "rx" : "none") + ); + } + + gmac_disable_tx_rx(netdev); + writel(status.bits32, status_reg); + gmac_enable_tx_rx(netdev); +} + +static int gmac_setup_phy(struct net_device *netdev) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + struct device *dev = port->dev; + struct phy_device *phy; + GMAC_STATUS_T status = { .bits32 = 0 }; + + phy = of_phy_get_and_connect(netdev, + dev->of_node, + gmac_speed_set); + if (!phy) + return -ENODEV; + netdev->phydev = phy; + + netdev_info(netdev, "connected to PHY \"%s\"\n", + phydev_name(phy)); + phy_attached_print(phy, "phy_id=0x%.8lx, phy_mode=%s\n", + (unsigned long)phy->phy_id, + phy_modes(phy->interface)); + + phy->supported &= PHY_GBIT_FEATURES; + phy->supported |= SUPPORTED_Asym_Pause | SUPPORTED_Pause; + phy->advertising = phy->supported; + + /* set PHY interface type */ + switch (phy->interface) { + case PHY_INTERFACE_MODE_MII: + netdev_info(netdev, "set GMAC0 to GMII mode, GMAC1 disabled\n"); + status.bits.mii_rmii = GMAC_PHY_MII; + netdev_info(netdev, "connect to MII\n"); + break; + case PHY_INTERFACE_MODE_GMII: + netdev_info(netdev, "set GMAC0 to GMII mode, GMAC1 disabled\n"); + status.bits.mii_rmii = GMAC_PHY_GMII; + netdev_info(netdev, "connect to GMII\n"); + break; + case PHY_INTERFACE_MODE_RGMII: + dev_info(dev, "set GMAC0 and GMAC1 to MII/RGMII mode\n"); + status.bits.mii_rmii = GMAC_PHY_RGMII_100_10; + netdev_info(netdev, "connect to RGMII\n"); + break; + default: + netdev_err(netdev, "Unsupported MII interface\n"); + phy_disconnect(phy); + netdev->phydev = NULL; + return -EINVAL; + } + writel(status.bits32, port->gmac_base + GMAC_STATUS); + + return 0; +} + +static int gmac_pick_rx_max_len(int max_l3_len) +{ + /* index = CONFIG_MAXLEN_XXX values */ + static const int max_len[8] = { + 1536, 1518, 1522, 1542, + 9212, 10236, 1518, 1518 + }; + int i, n = 5; + + max_l3_len += ETH_HLEN + VLAN_HLEN; + + if (max_l3_len > max_len[n]) + return -1; + + for (i = 0; i < 5; ++i) { + if (max_len[i] >= max_l3_len && max_len[i] < max_len[n]) + n = i; + } + + return n; +} + +static int gmac_init(struct net_device *netdev) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + u32 val; + + GMAC_CONFIG0_T config0 = { .bits = { + .dis_tx = 1, + .dis_rx = 1, + .ipv4_rx_chksum = 1, + .ipv6_rx_chksum = 1, + .rx_err_detect = 1, + .rgmm_edge = 1, + .port0_chk_hwq = 1, + .port1_chk_hwq = 1, + .port0_chk_toeq = 1, + .port1_chk_toeq = 1, + .port0_chk_classq = 1, + .port1_chk_classq = 1, + } }; + GMAC_AHB_WEIGHT_T ahb_weight = { .bits = { + .rx_weight = 1, + .tx_weight = 1, + .hash_weight = 1, + .pre_req = 0x1f, + .tqDV_threshold = 0, + } }; + GMAC_TX_WCR0_T hw_weigh = { .bits = { + .hw_tq3 = 1, + .hw_tq2 = 1, + .hw_tq1 = 1, + .hw_tq0 = 1, + } }; + GMAC_TX_WCR1_T sw_weigh = { .bits = { + .sw_tq5 = 1, + .sw_tq4 = 1, + .sw_tq3 = 1, + .sw_tq2 = 1, + .sw_tq1 = 1, + .sw_tq0 = 1, + } }; + GMAC_CONFIG1_T config1 = { .bits = { + .set_threshold = 16, + .rel_threshold = 24, + } }; + GMAC_CONFIG2_T config2 = { .bits = { + .set_threshold = 16, + .rel_threshold = 32, + } }; + GMAC_CONFIG3_T config3 = { .bits = { + .set_threshold = 0, + .rel_threshold = 0, + } }; + + config0.bits.max_len = gmac_pick_rx_max_len(netdev->mtu); + + val = readl(port->gmac_base + GMAC_CONFIG0); + config0.bits.reserved = ((GMAC_CONFIG0_T)val).bits.reserved; + writel(config0.bits32, port->gmac_base + GMAC_CONFIG0); + writel(config1.bits32, port->gmac_base + GMAC_CONFIG1); + writel(config2.bits32, port->gmac_base + GMAC_CONFIG2); + writel(config3.bits32, port->gmac_base + GMAC_CONFIG3); + + val = readl(port->dma_base + GMAC_AHB_WEIGHT_REG); + writel(ahb_weight.bits32, port->dma_base + GMAC_AHB_WEIGHT_REG); + + writel(hw_weigh.bits32, + port->dma_base + GMAC_TX_WEIGHTING_CTRL_0_REG); + writel(sw_weigh.bits32, + port->dma_base + GMAC_TX_WEIGHTING_CTRL_1_REG); + + port->rxq_order = DEFAULT_GMAC_RXQ_ORDER; + port->txq_order = DEFAULT_GMAC_TXQ_ORDER; + port->rx_coalesce_nsecs = DEFAULT_RX_COALESCE_NSECS; + + /* Mark every quarter of the queue a packet for interrupt + in order to be able to wake up the queue if it was stopped */ + port->irq_every_tx_packets = 1 << (port->txq_order - 2); + + return 0; +} + +static void gmac_uninit(struct net_device *netdev) +{ + if (netdev->phydev) + phy_disconnect(netdev->phydev); +} + +static int gmac_setup_txqs(struct net_device *netdev) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + struct gemini_ethernet *geth = port->geth; + void __iomem *rwptr_reg = port->dma_base + GMAC_SW_TX_QUEUE0_PTR_REG; + void __iomem *base_reg = port->dma_base + GMAC_SW_TX_QUEUE_BASE_REG; + + unsigned int n_txq = netdev->num_tx_queues; + size_t entries = 1 << port->txq_order; + size_t len = n_txq * entries; + struct gmac_txq *txq = port->txq; + GMAC_TXDESC_T *desc_ring; + struct sk_buff **skb_tab; + unsigned int r; + int i; + + skb_tab = kzalloc(len * sizeof(*skb_tab), GFP_KERNEL); + if (!skb_tab) + return -ENOMEM; + + desc_ring = dma_alloc_coherent(geth->dev, len * sizeof(*desc_ring), + &port->txq_dma_base, GFP_KERNEL); + + if (!desc_ring) { + kfree(skb_tab); + return -ENOMEM; + } + + BUG_ON(port->txq_dma_base & ~DMA_Q_BASE_MASK); + + writel(port->txq_dma_base | port->txq_order, base_reg); + + for (i = 0; i < n_txq; i++) { + txq->ring = desc_ring; + txq->skb = skb_tab; + txq->noirq_packets = 0; + + r = readw(rwptr_reg); + rwptr_reg += 2; + writew(r, rwptr_reg); + rwptr_reg +=2; + txq->cptr = r; + + txq++; + desc_ring += entries; + skb_tab += entries; + } + + return 0; +} + +static void gmac_clean_txq(struct net_device *netdev, struct gmac_txq *txq, + unsigned int r) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + struct gemini_ethernet *geth = port->geth; + unsigned int errs = 0; + unsigned int pkts = 0; + unsigned int hwchksum = 0; + unsigned long bytes = 0; + unsigned int m = (1 << port->txq_order) - 1; + unsigned int c = txq->cptr; + GMAC_TXDESC_0_T word0; + GMAC_TXDESC_1_T word1; + unsigned int word3; + dma_addr_t mapping; + GMAC_TXDESC_T *txd; + unsigned short nfrags; + + if (unlikely(c == r)) + return; + + rmb(); + while (c != r) { + txd = txq->ring + c; + word0 = txd->word0; + word1 = txd->word1; + mapping = txd->word2.buf_adr; + word3 = txd->word3.bits32; + + dma_unmap_single(geth->dev, mapping, + word0.bits.buffer_size, DMA_TO_DEVICE); + + if (word3 & EOF_BIT) + dev_kfree_skb(txq->skb[c]); + + c++; + c &= m; + + if (!(word3 & SOF_BIT)) + continue; + + if (!word0.bits.status_tx_ok) { + errs++; + continue; + } + + pkts++; + bytes += txd->word1.bits.byte_count; + + if (word1.bits32 & TSS_CHECKUM_ENABLE) + hwchksum++; + + nfrags = word0.bits.desc_count - 1; + if (nfrags) { + if (nfrags >= TX_MAX_FRAGS) + nfrags = TX_MAX_FRAGS - 1; + + u64_stats_update_begin(&port->tx_stats_syncp); + port->tx_frag_stats[nfrags]++; + u64_stats_update_end(&port->ir_stats_syncp); + } + } + + u64_stats_update_begin(&port->ir_stats_syncp); + port->stats.tx_errors += errs; + port->stats.tx_packets += pkts; + port->stats.tx_bytes += bytes; + port->tx_hw_csummed += hwchksum; + u64_stats_update_end(&port->ir_stats_syncp); + + txq->cptr = c; +} + +static void gmac_cleanup_txqs(struct net_device *netdev) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + struct gemini_ethernet *geth = port->geth; + void __iomem *rwptr_reg = port->dma_base + GMAC_SW_TX_QUEUE0_PTR_REG; + void __iomem *base_reg = port->dma_base + GMAC_SW_TX_QUEUE_BASE_REG; + + unsigned n_txq = netdev->num_tx_queues; + unsigned int r, i; + + for (i = 0; i < n_txq; i++) { + r = readw(rwptr_reg); + rwptr_reg += 2; + writew(r, rwptr_reg); + rwptr_reg += 2; + + gmac_clean_txq(netdev, port->txq + i, r); + } + writel(0, base_reg); + + kfree(port->txq->skb); + dma_free_coherent(geth->dev, + n_txq * sizeof(*port->txq->ring) << port->txq_order, + port->txq->ring, port->txq_dma_base); +} + +static int gmac_setup_rxq(struct net_device *netdev) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + struct gemini_ethernet *geth = port->geth; + NONTOE_QHDR_T __iomem *qhdr = geth->base + + TOE_DEFAULT_Q_HDR_BASE(netdev->dev_id); + + port->rxq_rwptr = &qhdr->word1; + port->rxq_ring = dma_alloc_coherent(geth->dev, + sizeof(*port->rxq_ring) << port->rxq_order, + &port->rxq_dma_base, GFP_KERNEL); + if (!port->rxq_ring) + return -ENOMEM; + + BUG_ON(port->rxq_dma_base & ~NONTOE_QHDR0_BASE_MASK); + + writel(port->rxq_dma_base | port->rxq_order, &qhdr->word0); + writel(0, port->rxq_rwptr); + return 0; +} + +static void gmac_cleanup_rxq(struct net_device *netdev) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + struct gemini_ethernet *geth = port->geth; + + NONTOE_QHDR_T __iomem *qhdr = geth->base + + TOE_DEFAULT_Q_HDR_BASE(netdev->dev_id); + void __iomem *dma_reg = &qhdr->word0; + void __iomem *ptr_reg = &qhdr->word1; + GMAC_RXDESC_T *rxd = port->rxq_ring; + DMA_RWPTR_T rw; + unsigned int r, w; + unsigned int m = (1 <<port->rxq_order) - 1; + struct page *page; + dma_addr_t mapping; + + rw.bits32 = readl(ptr_reg); + r = rw.bits.rptr; + w = rw.bits.wptr; + writew(r, ptr_reg + 2); + + writel(0, dma_reg); + + rmb(); + while (r != w) { + mapping = rxd[r].word2.buf_adr; + r++; + r &= m; + + if (!mapping) + continue; + + page = pfn_to_page(dma_to_pfn(geth->dev, mapping)); + put_page(page); + } + + dma_free_coherent(geth->dev, sizeof(*port->rxq_ring) << port->rxq_order, + port->rxq_ring, port->rxq_dma_base); +} + +static struct page *geth_freeq_alloc_map_page(struct gemini_ethernet *geth, + int pn) +{ + unsigned int fpp_order = PAGE_SHIFT - geth->freeq_frag_order; + unsigned int frag_len = 1 << geth->freeq_frag_order; + GMAC_RXDESC_T *freeq_entry; + dma_addr_t mapping; + struct page *page; + int i; + + page = alloc_page(GFP_ATOMIC); + if (!page) + return NULL; + + mapping = dma_map_single(geth->dev, page_address(page), + PAGE_SIZE, DMA_FROM_DEVICE); + + if (unlikely(dma_mapping_error(geth->dev, mapping) || !mapping)) { + put_page(page); + return NULL; + } + + freeq_entry = geth->freeq_ring + (pn << fpp_order); + for (i = 1 << fpp_order; i > 0; --i) { + freeq_entry->word2.buf_adr = mapping; + freeq_entry++; + mapping += frag_len; + } + + if (geth->freeq_page_tab[pn]) { + mapping = geth->freeq_ring[pn << fpp_order].word2.buf_adr; + dma_unmap_single(geth->dev, mapping, frag_len, DMA_FROM_DEVICE); + put_page(geth->freeq_page_tab[pn]); + } + + geth->freeq_page_tab[pn] = page; + return page; +} + +static unsigned int geth_fill_freeq(struct gemini_ethernet *geth, int reset) +{ + void __iomem *rwptr_reg = geth->base + GLOBAL_SWFQ_RWPTR_REG; + + DMA_RWPTR_T rw; + unsigned int pn, epn; + unsigned int fpp_order = PAGE_SHIFT - geth->freeq_frag_order; + unsigned int m_pn = (1 << (geth->freeq_order - fpp_order)) - 1; + struct page *page; + unsigned int count = 0; + unsigned long flags; + + spin_lock_irqsave(&geth->freeq_lock, flags); + + rw.bits32 = readl(rwptr_reg); + pn = (reset ? rw.bits.rptr : rw.bits.wptr) >> fpp_order; + epn = (rw.bits.rptr >> fpp_order) - 1; + epn &= m_pn; + + while (pn != epn) { + page = geth->freeq_page_tab[pn]; + + if (page_ref_count(page) > 1) { + unsigned int fl = (pn -epn) & m_pn; + + if (fl > 64 >> fpp_order) + break; + + page = geth_freeq_alloc_map_page(geth, pn); + if (!page) + break; + } + + page_ref_add(page, 1 << fpp_order); + count += 1 << fpp_order; + pn++; + pn &= m_pn; + } + + wmb(); + writew(pn << fpp_order, rwptr_reg+2); + + spin_unlock_irqrestore(&geth->freeq_lock, flags); + return count; +} + +static int geth_setup_freeq(struct gemini_ethernet *geth) +{ + void __iomem *dma_reg = geth->base + GLOBAL_SW_FREEQ_BASE_SIZE_REG; + QUEUE_THRESHOLD_T qt; + DMA_SKB_SIZE_T skbsz; + unsigned int filled; + unsigned int frag_len = 1 << geth->freeq_frag_order; + unsigned int len = 1 << geth->freeq_order; + unsigned int fpp_order = PAGE_SHIFT - geth->freeq_frag_order; + unsigned int pages = len >> fpp_order; + dma_addr_t mapping; + unsigned int pn; + + geth->freeq_ring = dma_alloc_coherent(geth->dev, + sizeof(*geth->freeq_ring) << geth->freeq_order, + &geth->freeq_dma_base, GFP_KERNEL); + if (!geth->freeq_ring) + return -ENOMEM; + + BUG_ON(geth->freeq_dma_base & ~DMA_Q_BASE_MASK); + + geth->freeq_page_tab = kzalloc(pages * sizeof(*geth->freeq_page_tab), + GFP_KERNEL); + if (!geth->freeq_page_tab) + goto err_freeq; + + dev_dbg(geth->dev, "allocate %d pages for queue\n", pages); + for (pn = 0; pn < pages; pn++) + if (!geth_freeq_alloc_map_page(geth, pn)) + goto err_freeq_alloc; + + filled = geth_fill_freeq(geth, 1); + if (!filled) + goto err_freeq_alloc; + + qt.bits32 = readl(geth->base + GLOBAL_QUEUE_THRESHOLD_REG); + qt.bits.swfq_empty = 32; + writel(qt.bits32, geth->base + GLOBAL_QUEUE_THRESHOLD_REG); + + skbsz.bits.sw_skb_size = 1 << geth->freeq_frag_order; + writel(skbsz.bits32, geth->base + GLOBAL_DMA_SKB_SIZE_REG); + writel(geth->freeq_dma_base | geth->freeq_order, dma_reg); + + return 0; + +err_freeq_alloc: + while (pn > 0) { + --pn; + mapping = geth->freeq_ring[pn << fpp_order].word2.buf_adr; + dma_unmap_single(geth->dev, mapping, frag_len, DMA_FROM_DEVICE); + put_page(geth->freeq_page_tab[pn]); + } + +err_freeq: + dma_free_coherent(geth->dev, + sizeof(*geth->freeq_ring) << geth->freeq_order, + geth->freeq_ring, geth->freeq_dma_base); + geth->freeq_ring = NULL; + return -ENOMEM; +} + +/** + * geth_cleanup_freeq() - cleanup the DMA mappings and free the queue + * @geth: the Gemini global ethernet state + */ +static void geth_cleanup_freeq(struct gemini_ethernet *geth) +{ + void __iomem *dma_reg = geth->base + GLOBAL_SW_FREEQ_BASE_SIZE_REG; + void __iomem *ptr_reg = geth->base + GLOBAL_SWFQ_RWPTR_REG; + + unsigned int frag_len = 1 << geth->freeq_frag_order; + unsigned int len = 1 << geth->freeq_order; + unsigned int fpp_order = PAGE_SHIFT - geth->freeq_frag_order; + unsigned int pages = len >> fpp_order; + struct page *page; + dma_addr_t mapping; + unsigned int pn; + + writew(readw(ptr_reg), ptr_reg + 2); + writel(0, dma_reg); + + for (pn = 0; pn < pages; pn++) { + mapping = geth->freeq_ring[pn << fpp_order].word2.buf_adr; + dma_unmap_single(geth->dev, mapping, frag_len, DMA_FROM_DEVICE); + + page = geth->freeq_page_tab[pn]; + while (page_ref_count(page) > 0) + put_page(page); + } + + kfree(geth->freeq_page_tab); + + dma_free_coherent(geth->dev, + sizeof(*geth->freeq_ring) << geth->freeq_order, + geth->freeq_ring, geth->freeq_dma_base); +} + +/** + * geth_resize_freeq() - resize the software queue depth + * @port: the port requesting the change + * + * This gets called at least once during probe() so the device queue gets + * "resized" from the hardware defaults. Since both ports/net devices share + * the same hardware queue, some synchronization between the ports is + * needed. + */ +static int geth_resize_freeq(struct gemini_ethernet_port *port) +{ + struct gemini_ethernet *geth = port->geth; + struct net_device *netdev = port->netdev; + struct net_device *other_netdev; + struct gemini_ethernet_port *other_port; + unsigned new_size = 0; + unsigned new_order; + unsigned long flags; + u32 en; + int ret; + + if (netdev->dev_id == 0) + other_netdev = geth->port1->netdev; + else + other_netdev = geth->port0->netdev; + + if (other_netdev && netif_running(other_netdev)) + return -EBUSY; + + new_size = 1 << (port->rxq_order + 1); + netdev_dbg(netdev, "port %d size: %d order %d\n", + netdev->dev_id, + new_size, + port->rxq_order); + if (other_netdev) { + other_port = netdev_priv(other_netdev); + new_size += 1 << (other_port->rxq_order + 1); + netdev_dbg(other_netdev, "port %d size: %d order %d\n", + other_netdev->dev_id, + (1 << (other_port->rxq_order + 1)), + other_port->rxq_order); + } + + new_order = min(15, ilog2(new_size - 1) + 1); + dev_dbg(geth->dev, "set shared queue to size %d order %d\n", + new_size, new_order); + if (geth->freeq_order == new_order) + return 0; + + spin_lock_irqsave(&geth->irq_lock, flags); + + /* Disable the software queue IRQs */ + en = readl(geth->base + GLOBAL_INTERRUPT_ENABLE_4_REG); + en &= ~SWFQ_EMPTY_INT_BIT; + writel(en, geth->base + GLOBAL_INTERRUPT_ENABLE_4_REG); + + /* Drop the old queue */ + if (geth->freeq_ring) + geth_cleanup_freeq(geth); + + /* Allocate a new queue with the desired order */ + geth->freeq_order = new_order; + ret = geth_setup_freeq(geth); + + /* + * Restart the interrupts - NOTE if this is the first resize + * after probe(), this is where the interrupts get turned on + * in the first place. + */ + en |= SWFQ_EMPTY_INT_BIT; + writel(en, geth->base + GLOBAL_INTERRUPT_ENABLE_4_REG); + spin_unlock_irqrestore(&geth->irq_lock, flags); + + return ret; +} + +static void gmac_tx_irq_enable(struct net_device *netdev, unsigned txq, int en) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + struct gemini_ethernet *geth = port->geth; + unsigned val, mask; + + netdev_info(netdev, "%s device %d\n", __func__, netdev->dev_id); + + mask = GMAC0_IRQ0_TXQ0_INTS << (6 * netdev->dev_id + txq); + + if (en) + writel(mask, geth->base + GLOBAL_INTERRUPT_STATUS_0_REG); + + val = readl(geth->base + GLOBAL_INTERRUPT_ENABLE_0_REG); + val = en ? val | mask : val & ~mask; + writel(val, geth->base + GLOBAL_INTERRUPT_ENABLE_0_REG); +} + + +static void gmac_tx_irq(struct net_device *netdev, unsigned txq_num) +{ + struct netdev_queue *ntxq = netdev_get_tx_queue(netdev, txq_num); + + gmac_tx_irq_enable(netdev, txq_num, 0); + netif_tx_wake_queue(ntxq); +} + +static int gmac_map_tx_bufs(struct net_device *netdev, struct sk_buff *skb, + struct gmac_txq *txq, unsigned short *desc) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + struct gemini_ethernet *geth = port->geth; + struct skb_shared_info *skb_si = skb_shinfo(skb); + skb_frag_t *skb_frag; + short frag, last_frag = skb_si->nr_frags - 1; + unsigned short m = (1 << port->txq_order) -1; + unsigned short w = *desc; + unsigned word1, word3, buflen; + dma_addr_t mapping; + void *buffer; + unsigned short mtu; + GMAC_TXDESC_T *txd; + + mtu = ETH_HLEN; + mtu += netdev->mtu; + if (skb->protocol == htons(ETH_P_8021Q)) + mtu += VLAN_HLEN; + + word1 = skb->len; + word3 = SOF_BIT; + + if (word1 > mtu) { + word1 |= TSS_MTU_ENABLE_BIT; + word3 += mtu; + } + + if (skb->ip_summed != CHECKSUM_NONE) { + int tcp = 0; + if (skb->protocol == htons(ETH_P_IP)) { + word1 |= TSS_IP_CHKSUM_BIT; + tcp = ip_hdr(skb)->protocol == IPPROTO_TCP; + } else { /* IPv6 */ + word1 |= TSS_IPV6_ENABLE_BIT; + tcp = ipv6_hdr(skb)->nexthdr == IPPROTO_TCP; + } + + word1 |= tcp ? TSS_TCP_CHKSUM_BIT : TSS_UDP_CHKSUM_BIT; + } + + frag = -1; + while (frag <= last_frag) { + if (frag == -1) { + buffer = skb->data; + buflen = skb_headlen(skb); + } else { + skb_frag = skb_si->frags + frag; + buffer = page_address(skb_frag_page(skb_frag)) + + skb_frag->page_offset; + buflen = skb_frag->size; + } + + if (frag == last_frag) { + word3 |= EOF_BIT; + txq->skb[w] = skb; + } + + mapping = dma_map_single(geth->dev, buffer, buflen, + DMA_TO_DEVICE); + if (dma_mapping_error(geth->dev, mapping) || + !(mapping & PAGE_MASK)) + goto map_error; + + txd = txq->ring + w; + txd->word0.bits32 = buflen; + txd->word1.bits32 = word1; + txd->word2.buf_adr = mapping; + txd->word3.bits32 = word3; + + word3 &= MTU_SIZE_BIT_MASK; + w++; + w &= m; + frag++; + } + + *desc = w; + return 0; + +map_error: + while (w != *desc) { + w--; + w &= m; + + dma_unmap_page(geth->dev, txq->ring[w].word2.buf_adr, + txq->ring[w].word0.bits.buffer_size, DMA_TO_DEVICE); + } + return ENOMEM; +} + +static int gmac_start_xmit(struct sk_buff *skb, struct net_device *netdev) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + + void __iomem *ptr_reg; + struct gmac_txq *txq; + struct netdev_queue *ntxq; + int txq_num, nfrags; + DMA_RWPTR_T rw; + unsigned short r, w, d; + unsigned short m = (1 << port->txq_order) - 1; + + SKB_FRAG_ASSERT(skb); + + if (unlikely(skb->len >= 0x10000)) + goto out_drop_free; + + txq_num = skb_get_queue_mapping(skb); + ptr_reg = port->dma_base + GMAC_SW_TX_QUEUE_PTR_REG(txq_num); + txq = &port->txq[txq_num]; + ntxq = netdev_get_tx_queue(netdev, txq_num); + nfrags = skb_shinfo(skb)->nr_frags; + + rw.bits32 = readl(ptr_reg); + r = rw.bits.rptr; + w = rw.bits.wptr; + + d = txq->cptr - w - 1; + d &= m; + + if (unlikely(d < nfrags+2)) + { + gmac_clean_txq(netdev, txq, r); + d = txq->cptr - w - 1; + d &= m; + + if (unlikely(d < nfrags+2)) { + netif_tx_stop_queue(ntxq); + + d = txq->cptr + nfrags + 16; + d &= m; + txq->ring[d].word3.bits.eofie = 1; + gmac_tx_irq_enable(netdev, txq_num, 1); + + u64_stats_update_begin(&port->tx_stats_syncp); + netdev->stats.tx_fifo_errors++; + u64_stats_update_end(&port->tx_stats_syncp); + return NETDEV_TX_BUSY; + } + } + + if (unlikely(gmac_map_tx_bufs(netdev, skb, txq, &w))) { + if (skb_linearize(skb)) + goto out_drop; + + if (unlikely(gmac_map_tx_bufs(netdev, skb, txq, &w))) + goto out_drop_free; + + u64_stats_update_begin(&port->tx_stats_syncp); + port->tx_frags_linearized++; + u64_stats_update_end(&port->tx_stats_syncp); + } + + writew(w, ptr_reg+2); + + gmac_clean_txq(netdev, txq, r); + return NETDEV_TX_OK; + +out_drop_free: + dev_kfree_skb(skb); +out_drop: + u64_stats_update_begin(&port->tx_stats_syncp); + port->stats.tx_dropped++; + u64_stats_update_end(&port->tx_stats_syncp); + return NETDEV_TX_OK; +} + +static void gmac_tx_timeout(struct net_device *netdev) +{ + netdev_err(netdev, "Tx timeout\n"); + gmac_dump_dma_state(netdev); +} + +static void gmac_enable_irq(struct net_device *netdev, int enable) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + struct gemini_ethernet *geth = port->geth; + unsigned long flags; + unsigned val, mask; + + netdev_info(netdev, "%s device %d %s\n", __func__, + netdev->dev_id, enable ? "enable" : "disable"); + spin_lock_irqsave(&geth->irq_lock, flags); + + mask = GMAC0_IRQ0_2 << (netdev->dev_id * 2); + val = readl(geth->base + GLOBAL_INTERRUPT_ENABLE_0_REG); + val = enable ? (val | mask) : (val & ~mask); + writel(val, geth->base + GLOBAL_INTERRUPT_ENABLE_0_REG); + + mask = DEFAULT_Q0_INT_BIT << netdev->dev_id; + val = readl(geth->base + GLOBAL_INTERRUPT_ENABLE_1_REG); + val = enable ? (val | mask) : (val & ~mask); + writel(val, geth->base + GLOBAL_INTERRUPT_ENABLE_1_REG); + + mask = GMAC0_IRQ4_8 << (netdev->dev_id * 8); + val = readl(geth->base + GLOBAL_INTERRUPT_ENABLE_4_REG); + val = enable ? (val | mask) : (val & ~mask); + writel(val, geth->base + GLOBAL_INTERRUPT_ENABLE_4_REG); + + spin_unlock_irqrestore(&geth->irq_lock, flags); +} + +static void gmac_enable_rx_irq(struct net_device *netdev, int enable) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + struct gemini_ethernet *geth = port->geth; + unsigned long flags; + u32 val, mask; + + netdev_dbg(netdev, "%s device %d %s\n", __func__, netdev->dev_id, + enable ? "enable" : "disable"); + spin_lock_irqsave(&geth->irq_lock, flags); + mask = DEFAULT_Q0_INT_BIT << netdev->dev_id; + + val = readl(geth->base + GLOBAL_INTERRUPT_ENABLE_1_REG); + val = enable ? (val | mask) : (val & ~mask); + writel(val, geth->base + GLOBAL_INTERRUPT_ENABLE_1_REG); + + spin_unlock_irqrestore(&geth->irq_lock, flags); +} + +static struct sk_buff *gmac_skb_if_good_frame(struct gemini_ethernet_port *port, + GMAC_RXDESC_0_T word0, unsigned frame_len) +{ + struct sk_buff *skb = NULL; + unsigned rx_status = word0.bits.status; + unsigned rx_csum = word0.bits.chksum_status; + + port->rx_stats[rx_status]++; + port->rx_csum_stats[rx_csum]++; + + if (word0.bits.derr || word0.bits.perr || + rx_status || frame_len < ETH_ZLEN || + rx_csum >= RX_CHKSUM_IP_ERR_UNKNOWN) { + port->stats.rx_errors++; + + if (frame_len < ETH_ZLEN || RX_ERROR_LENGTH(rx_status)) + port->stats.rx_length_errors++; + if (RX_ERROR_OVER(rx_status)) + port->stats.rx_over_errors++; + if (RX_ERROR_CRC(rx_status)) + port->stats.rx_crc_errors++; + if (RX_ERROR_FRAME(rx_status)) + port->stats.rx_frame_errors++; + + return NULL; + } + + skb = napi_get_frags(&port->napi); + if (!skb) + return NULL; + + if (rx_csum == RX_CHKSUM_IP_UDP_TCP_OK) + skb->ip_summed = CHECKSUM_UNNECESSARY; + + port->stats.rx_bytes += frame_len; + port->stats.rx_packets++; + return skb; +} + +static unsigned int gmac_rx(struct net_device *netdev, unsigned budget) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + struct gemini_ethernet *geth = port->geth; + void __iomem *ptr_reg = port->rxq_rwptr; + + static struct sk_buff *skb; + + DMA_RWPTR_T rw; + unsigned short r, w; + unsigned short m = (1 << port->rxq_order) -1; + GMAC_RXDESC_T *rx = NULL; + struct page* page = NULL; + unsigned page_offs; + unsigned int frame_len, frag_len; + int frag_nr = 0; + + GMAC_RXDESC_0_T word0; + GMAC_RXDESC_1_T word1; + dma_addr_t mapping; + GMAC_RXDESC_3_T word3; + + rw.bits32 = readl(ptr_reg); + /* Reset interrupt as all packages until here are taken into account */ + writel(DEFAULT_Q0_INT_BIT << netdev->dev_id, + geth->base + GLOBAL_INTERRUPT_STATUS_1_REG); + r = rw.bits.rptr; + w = rw.bits.wptr; + + while (budget && w != r) { + rx = port->rxq_ring + r; + word0 = rx->word0; + word1 = rx->word1; + mapping = rx->word2.buf_adr; + word3 = rx->word3; + + r++; + r &= m; + + frag_len = word0.bits.buffer_size; + frame_len =word1.bits.byte_count; + page_offs = mapping & ~PAGE_MASK; + + if (unlikely(!mapping)) { + netdev_err(netdev, + "rxq[%u]: HW BUG: zero DMA desc\n", r); + goto err_drop; + } + + page = pfn_to_page(dma_to_pfn(geth->dev, mapping)); + + if (word3.bits32 & SOF_BIT) { + if (unlikely(skb)) { + napi_free_frags(&port->napi); + port->stats.rx_dropped++; + } + + skb = gmac_skb_if_good_frame(port, word0, frame_len); + if (unlikely(!skb)) + goto err_drop; + + page_offs += NET_IP_ALIGN; + frag_len -= NET_IP_ALIGN; + frag_nr = 0; + + } else if (!skb) { + put_page(page); + continue; + } + + if (word3.bits32 & EOF_BIT) + frag_len = frame_len - skb->len; + + /* append page frag to skb */ + if (unlikely(frag_nr == MAX_SKB_FRAGS)) + goto err_drop; + + if (frag_len == 0) + netdev_err(netdev, "Received fragment with len = 0\n"); + + skb_fill_page_desc(skb, frag_nr, page, page_offs, frag_len); + skb->len += frag_len; + skb->data_len += frag_len; + skb->truesize += frag_len; + frag_nr++; + + if (word3.bits32 & EOF_BIT) { + napi_gro_frags(&port->napi); + skb = NULL; + --budget; + } + continue; + +err_drop: + if (skb) { + napi_free_frags(&port->napi); + skb = NULL; + } + + if (mapping) + put_page(page); + + port->stats.rx_dropped++; + } + + writew(r, ptr_reg); + return budget; +} + +static int gmac_napi_poll(struct napi_struct *napi, int budget) +{ + struct gemini_ethernet_port *port = netdev_priv(napi->dev); + struct gemini_ethernet *geth = port->geth; + unsigned int freeq_threshold = 1 << (geth->freeq_order - 1); + unsigned int received; + + u64_stats_update_begin(&port->rx_stats_syncp); + + received = gmac_rx(napi->dev, budget); + if (received < budget) { + napi_gro_flush(napi, false); + napi_complete_done(napi, received); + gmac_enable_rx_irq(napi->dev, 1); + ++port->rx_napi_exits; + } + + port->freeq_refill += (budget - received); + if (port->freeq_refill > freeq_threshold) { + port->freeq_refill -= freeq_threshold; + geth_fill_freeq(geth, 0); + } + + u64_stats_update_end(&port->rx_stats_syncp); + return received; +} + +static void gmac_dump_dma_state(struct net_device *netdev) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + struct gemini_ethernet *geth = port->geth; + void __iomem *ptr_reg; + unsigned reg[5]; + + /* Interrupt status */ + reg[0] = readl(geth->base + GLOBAL_INTERRUPT_STATUS_0_REG); + reg[1] = readl(geth->base + GLOBAL_INTERRUPT_STATUS_1_REG); + reg[2] = readl(geth->base + GLOBAL_INTERRUPT_STATUS_2_REG); + reg[3] = readl(geth->base + GLOBAL_INTERRUPT_STATUS_3_REG); + reg[4] = readl(geth->base + GLOBAL_INTERRUPT_STATUS_4_REG); + netdev_err(netdev, "IRQ status: 0x%08x 0x%08x 0x%08x 0x%08x 0x%08x\n", + reg[0], reg[1], reg[2], reg[3], reg[4]); + + /* Interrupt enable */ + reg[0] = readl(geth->base + GLOBAL_INTERRUPT_ENABLE_0_REG); + reg[1] = readl(geth->base + GLOBAL_INTERRUPT_ENABLE_1_REG); + reg[2] = readl(geth->base + GLOBAL_INTERRUPT_ENABLE_2_REG); + reg[3] = readl(geth->base + GLOBAL_INTERRUPT_ENABLE_3_REG); + reg[4] = readl(geth->base + GLOBAL_INTERRUPT_ENABLE_4_REG); + netdev_err(netdev, "IRQ enable: 0x%08x 0x%08x 0x%08x 0x%08x 0x%08x\n", + reg[0], reg[1], reg[2], reg[3], reg[4]); + + /* RX DMA status */ + reg[0] = readl(port->dma_base + GMAC_DMA_RX_FIRST_DESC_REG); + reg[1] = readl(port->dma_base + GMAC_DMA_RX_CURR_DESC_REG); + reg[2] = GET_RPTR(port->rxq_rwptr); + reg[3] = GET_WPTR(port->rxq_rwptr); + netdev_err(netdev, "RX DMA regs: 0x%08x 0x%08x, ptr: %u %u\n", + reg[0], reg[1], reg[2], reg[3]); + + reg[0] = readl(port->dma_base + GMAC_DMA_RX_DESC_WORD0_REG); + reg[1] = readl(port->dma_base + GMAC_DMA_RX_DESC_WORD1_REG); + reg[2] = readl(port->dma_base + GMAC_DMA_RX_DESC_WORD2_REG); + reg[3] = readl(port->dma_base + GMAC_DMA_RX_DESC_WORD3_REG); + netdev_err(netdev, "RX DMA descriptor: 0x%08x 0x%08x 0x%08x 0x%08x\n", + reg[0], reg[1], reg[2], reg[3]); + + /* TX DMA status */ + ptr_reg = port->dma_base + GMAC_SW_TX_QUEUE0_PTR_REG; + + reg[0] = readl(port->dma_base + GMAC_DMA_TX_FIRST_DESC_REG); + reg[1] = readl(port->dma_base + GMAC_DMA_TX_CURR_DESC_REG); + reg[2] = GET_RPTR(ptr_reg); + reg[3] = GET_WPTR(ptr_reg); + netdev_err(netdev, "TX DMA regs: 0x%08x 0x%08x, ptr: %u %u\n", + reg[0], reg[1], reg[2], reg[3]); + + reg[0] = readl(port->dma_base + GMAC_DMA_TX_DESC_WORD0_REG); + reg[1] = readl(port->dma_base + GMAC_DMA_TX_DESC_WORD1_REG); + reg[2] = readl(port->dma_base + GMAC_DMA_TX_DESC_WORD2_REG); + reg[3] = readl(port->dma_base + GMAC_DMA_TX_DESC_WORD3_REG); + netdev_err(netdev, "TX DMA descriptor: 0x%08x 0x%08x 0x%08x 0x%08x\n", + reg[0], reg[1], reg[2], reg[3]); + + /* FREE queues status */ + ptr_reg = geth->base + GLOBAL_SWFQ_RWPTR_REG; + + reg[0] = GET_RPTR(ptr_reg); + reg[1] = GET_WPTR(ptr_reg); + + ptr_reg = geth->base + GLOBAL_HWFQ_RWPTR_REG; + + reg[2] = GET_RPTR(ptr_reg); + reg[3] = GET_WPTR(ptr_reg); + netdev_err(netdev, "FQ SW ptr: %u %u, HW ptr: %u %u\n", + reg[0], reg[1], reg[2], reg[3]); +} + +static void gmac_update_hw_stats(struct net_device *netdev) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + struct gemini_ethernet *geth = port->geth; + unsigned long flags; + unsigned int rx_discards, rx_mcast, rx_bcast; + + spin_lock_irqsave(&geth->irq_lock, flags); + u64_stats_update_begin(&port->ir_stats_syncp); + + rx_discards = readl(port->gmac_base + GMAC_IN_DISCARDS); + port->hw_stats[0] += rx_discards; + port->hw_stats[1] += readl(port->gmac_base + GMAC_IN_ERRORS); + rx_mcast = readl(port->gmac_base + GMAC_IN_MCAST); + port->hw_stats[2] += rx_mcast; + rx_bcast = readl(port->gmac_base + GMAC_IN_BCAST); + port->hw_stats[3] += rx_bcast; + port->hw_stats[4] += readl(port->gmac_base + GMAC_IN_MAC1); + port->hw_stats[5] += readl(port->gmac_base + GMAC_IN_MAC2); + + port->stats.rx_missed_errors += rx_discards; + port->stats.multicast += rx_mcast; + port->stats.multicast += rx_bcast; + + writel(GMAC0_MIB_INT_BIT << (netdev->dev_id * 8), + geth->base + GLOBAL_INTERRUPT_STATUS_4_REG); + + u64_stats_update_end(&port->ir_stats_syncp); + spin_unlock_irqrestore(&geth->irq_lock, flags); +} + +/** + * gmac_get_intr_flags() - get interrupt status flags for a port from + * @netdev: the net device for the port to get flags from + * @i: the interrupt status register 0..4 + */ +static inline unsigned gmac_get_intr_flags(struct net_device *netdev, int i) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + struct gemini_ethernet *geth = port->geth; + void __iomem *irqif_reg, *irqen_reg; + unsigned offs, val; + + /* Calculate the offset using the stride of the status registers */ + offs = i * (GLOBAL_INTERRUPT_STATUS_1_REG - + GLOBAL_INTERRUPT_STATUS_0_REG); + + irqif_reg = geth->base + GLOBAL_INTERRUPT_STATUS_0_REG + offs; + irqen_reg = geth->base + GLOBAL_INTERRUPT_ENABLE_0_REG + offs; + + val = readl(irqif_reg) & readl(irqen_reg); + return val; +} + +enum hrtimer_restart gmac_coalesce_delay_expired(struct hrtimer *timer) +{ + struct gemini_ethernet_port *port = + container_of(timer, struct gemini_ethernet_port, + rx_coalesce_timer); + + napi_schedule(&port->napi); + return HRTIMER_NORESTART; +} + +static irqreturn_t gmac_irq(int irq, void *data) +{ + struct net_device *netdev = data; + struct gemini_ethernet_port *port = netdev_priv(netdev); + struct gemini_ethernet *geth = port->geth; + unsigned val, orr = 0; + + val = gmac_get_intr_flags(netdev, 0); + orr |= val; + + if (val & (GMAC0_IRQ0_2 << (netdev->dev_id * 2))) { + /* Oh, crap */ + netdev_err(netdev, "hw failure/sw bug\n"); + gmac_dump_dma_state(netdev); + + /* don't know how to recover, just reduce losses */ + gmac_enable_irq(netdev, 0); + return IRQ_HANDLED; + } + + if (val & (GMAC0_IRQ0_TXQ0_INTS << (netdev->dev_id * 6))) + gmac_tx_irq(netdev, 0); + + val = gmac_get_intr_flags(netdev, 1); + orr |= val; + + if (val & (DEFAULT_Q0_INT_BIT << netdev->dev_id)) { + + gmac_enable_rx_irq(netdev, 0); + + if (!port->rx_coalesce_nsecs) + napi_schedule(&port->napi); + else { + ktime_t ktime; + ktime = ktime_set(0, port->rx_coalesce_nsecs); + hrtimer_start(&port->rx_coalesce_timer, ktime, + HRTIMER_MODE_REL); + } + } + + val = gmac_get_intr_flags(netdev, 4); + orr |= val; + + if (val & (GMAC0_MIB_INT_BIT << (netdev->dev_id * 8))) + gmac_update_hw_stats(netdev); + + if (val & (GMAC0_RX_OVERRUN_INT_BIT << (netdev->dev_id * 8))) { + writel(GMAC0_RXDERR_INT_BIT << (netdev->dev_id * 8), + geth->base + GLOBAL_INTERRUPT_STATUS_4_REG); + + spin_lock(&geth->irq_lock); + u64_stats_update_begin(&port->ir_stats_syncp); + ++port->stats.rx_fifo_errors; + u64_stats_update_end(&port->ir_stats_syncp); + spin_unlock(&geth->irq_lock); + } + + return orr ? IRQ_HANDLED : IRQ_NONE; +} + +static void gmac_start_dma(struct gemini_ethernet_port *port) +{ + void __iomem *dma_ctrl_reg = port->dma_base + GMAC_DMA_CTRL_REG; + GMAC_DMA_CTRL_T dma_ctrl; + + dma_ctrl.bits32 = readl(dma_ctrl_reg); + dma_ctrl.bits.rd_enable = 1; + dma_ctrl.bits.td_enable = 1; + dma_ctrl.bits.loopback = 0; + dma_ctrl.bits.drop_small_ack = 0; + dma_ctrl.bits.rd_insert_bytes = NET_IP_ALIGN; + dma_ctrl.bits.rd_prot = HPROT_DATA_CACHE | HPROT_PRIVILIGED; + dma_ctrl.bits.rd_burst_size = HBURST_INCR8; + dma_ctrl.bits.rd_bus = HSIZE_8; + dma_ctrl.bits.td_prot = HPROT_DATA_CACHE; + dma_ctrl.bits.td_burst_size = HBURST_INCR8; + dma_ctrl.bits.td_bus = HSIZE_8; + + writel(dma_ctrl.bits32, dma_ctrl_reg); +} + +static void gmac_stop_dma(struct gemini_ethernet_port *port) +{ + void __iomem *dma_ctrl_reg = port->dma_base + GMAC_DMA_CTRL_REG; + GMAC_DMA_CTRL_T dma_ctrl; + + dma_ctrl.bits32 = readl(dma_ctrl_reg); + dma_ctrl.bits.rd_enable = 0; + dma_ctrl.bits.td_enable = 0; + writel(dma_ctrl.bits32, dma_ctrl_reg); +} + +static int gmac_open(struct net_device *netdev) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + int err; + + if (!netdev->phydev) { + err = gmac_setup_phy(netdev); + if (err) { + netif_err(port, ifup, netdev, + "PHY init failed: %d\n", err); + return err; + } + } + + err = request_irq(netdev->irq, gmac_irq, + IRQF_SHARED, netdev->name, netdev); + if (err) { + netdev_err(netdev, "no IRQ\n"); + return err; + } + + netif_carrier_off(netdev); + phy_start(netdev->phydev); + + err = geth_resize_freeq(port); + if (err) { + netdev_err(netdev, "could not resize freeq\n"); + goto err_stop_phy; + } + + err = gmac_setup_rxq(netdev); + if (err) { + netdev_err(netdev, "could not setup RXQ\n"); + goto err_stop_phy; + } + + err = gmac_setup_txqs(netdev); + if (err) { + netdev_err(netdev, "could not setup TXQs\n"); + gmac_cleanup_rxq(netdev); + goto err_stop_phy; + } + + napi_enable(&port->napi); + + gmac_start_dma(port); + gmac_enable_irq(netdev, 1); + gmac_enable_tx_rx(netdev); + netif_tx_start_all_queues(netdev); + + hrtimer_init(&port->rx_coalesce_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL); + port->rx_coalesce_timer.function = &gmac_coalesce_delay_expired; + + netdev_info(netdev, "opened\n"); + + return 0; + +err_stop_phy: + phy_stop(netdev->phydev); + free_irq(netdev->irq, netdev); + return err; +} + +static int gmac_stop(struct net_device *netdev) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + + hrtimer_cancel(&port->rx_coalesce_timer); + netif_tx_stop_all_queues(netdev); + gmac_disable_tx_rx(netdev); + gmac_stop_dma(port); + napi_disable(&port->napi); + + gmac_enable_irq(netdev, 0); + gmac_cleanup_rxq(netdev); + gmac_cleanup_txqs(netdev); + + phy_stop(netdev->phydev); + free_irq(netdev->irq, netdev); + + gmac_update_hw_stats(netdev); + return 0; +} + +static void gmac_set_rx_mode(struct net_device *netdev) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + struct netdev_hw_addr *ha; + __u32 mc_filter[2]; + unsigned bit_nr; + GMAC_RX_FLTR_T filter = { .bits = { + .broadcast = 1, + .multicast = 1, + .unicast = 1, + } }; + + mc_filter[1] = mc_filter[0] = 0; + + if (netdev->flags & IFF_PROMISC) { + filter.bits.error = 1; + filter.bits.promiscuous = 1; + } else if (!(netdev->flags & IFF_ALLMULTI)) { + mc_filter[1] = mc_filter[0] = 0; + netdev_for_each_mc_addr(ha, netdev) { + bit_nr = ~crc32_le(~0, ha->addr, ETH_ALEN) & 0x3f; + mc_filter[bit_nr >> 5] |= 1 << (bit_nr & 0x1f); + } + } + + writel(mc_filter[0], port->gmac_base + GMAC_MCAST_FIL0); + writel(mc_filter[1], port->gmac_base + GMAC_MCAST_FIL1); + writel(filter.bits32, port->gmac_base + GMAC_RX_FLTR); +} + +static void gmac_write_mac_address(struct net_device *netdev) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + __le32 addr[3]; + + memset(addr, 0, sizeof(addr)); + memcpy(addr, netdev->dev_addr, ETH_ALEN); + + writel(le32_to_cpu(addr[0]), port->gmac_base + GMAC_STA_ADD0); + writel(le32_to_cpu(addr[1]), port->gmac_base + GMAC_STA_ADD1); + writel(le32_to_cpu(addr[2]), port->gmac_base + GMAC_STA_ADD2); +} + +static int gmac_set_mac_address(struct net_device *netdev, void *addr) +{ + struct sockaddr *sa = addr; + + memcpy(netdev->dev_addr, sa->sa_data, ETH_ALEN); + gmac_write_mac_address(netdev); + + return 0; +} + +static void gmac_clear_hw_stats(struct net_device *netdev) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + + readl(port->gmac_base + GMAC_IN_DISCARDS); + readl(port->gmac_base + GMAC_IN_ERRORS); + readl(port->gmac_base + GMAC_IN_MCAST); + readl(port->gmac_base + GMAC_IN_BCAST); + readl(port->gmac_base + GMAC_IN_MAC1); + readl(port->gmac_base + GMAC_IN_MAC2); +} + +static void gmac_get_stats64(struct net_device *netdev, + struct rtnl_link_stats64 *stats) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + unsigned int start; + + gmac_update_hw_stats(netdev); + + /* Racing with RX NAPI */ + do { + start = u64_stats_fetch_begin(&port->rx_stats_syncp); + + stats->rx_packets = port->stats.rx_packets; + stats->rx_bytes = port->stats.rx_bytes; + stats->rx_errors = port->stats.rx_errors; + stats->rx_dropped = port->stats.rx_dropped; + + stats->rx_length_errors = port->stats.rx_length_errors; + stats->rx_over_errors = port->stats.rx_over_errors; + stats->rx_crc_errors = port->stats.rx_crc_errors; + stats->rx_frame_errors = port->stats.rx_frame_errors; + + } while (u64_stats_fetch_retry(&port->rx_stats_syncp, start)); + + /* Racing with MIB and TX completion interrupts */ + do { + start = u64_stats_fetch_begin(&port->ir_stats_syncp); + + stats->tx_errors = port->stats.tx_errors; + stats->tx_packets = port->stats.tx_packets; + stats->tx_bytes = port->stats.tx_bytes; + + stats->multicast = port->stats.multicast; + stats->rx_missed_errors = port->stats.rx_missed_errors; + stats->rx_fifo_errors = port->stats.rx_fifo_errors; + + } while (u64_stats_fetch_retry(&port->ir_stats_syncp, start)); + + /* Racing with hard_start_xmit */ + do { + start = u64_stats_fetch_begin(&port->tx_stats_syncp); + + stats->tx_dropped = port->stats.tx_dropped; + + } while (u64_stats_fetch_retry(&port->tx_stats_syncp, start)); + + stats->rx_dropped += stats->rx_missed_errors; +} + +static int gmac_change_mtu(struct net_device *netdev, int new_mtu) +{ + int max_len = gmac_pick_rx_max_len(new_mtu); + + if (max_len < 0) + return -EINVAL; + + gmac_disable_tx_rx(netdev); + + netdev->mtu = new_mtu; + gmac_update_config0_reg(netdev, + max_len << CONFIG0_MAXLEN_SHIFT, + CONFIG0_MAXLEN_MASK); + + netdev_update_features(netdev); + + gmac_enable_tx_rx(netdev); + + return 0; +} + +static netdev_features_t gmac_fix_features(struct net_device *netdev, netdev_features_t features) +{ + if (netdev->mtu + ETH_HLEN + VLAN_HLEN > MTU_SIZE_BIT_MASK) + features &= ~GMAC_OFFLOAD_FEATURES; + + return features; +} + +static int gmac_set_features(struct net_device *netdev, netdev_features_t features) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + int enable = features & NETIF_F_RXCSUM; + unsigned long flags; + u32 reg; + + spin_lock_irqsave(&port->config_lock, flags); + + reg = readl(port->gmac_base + GMAC_CONFIG0); + reg = enable ? reg | CONFIG0_RX_CHKSUM : reg & ~CONFIG0_RX_CHKSUM; + writel(reg, port->gmac_base + GMAC_CONFIG0); + + spin_unlock_irqrestore(&port->config_lock, flags); + return 0; +} + +static int gmac_get_sset_count(struct net_device *netdev, int sset) +{ + return sset == ETH_SS_STATS ? GMAC_STATS_NUM : 0; +} + +static void gmac_get_strings(struct net_device *netdev, u32 stringset, u8 *data) +{ + if (stringset != ETH_SS_STATS) + return; + + memcpy(data, gmac_stats_strings, sizeof(gmac_stats_strings)); +} + +static void gmac_get_ethtool_stats(struct net_device *netdev, + struct ethtool_stats *estats, u64 *values) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + unsigned int start; + u64 *p; + int i; + + gmac_update_hw_stats(netdev); + + /* racing with MIB interrupt */ + do { + p = values; + start = u64_stats_fetch_begin(&port->ir_stats_syncp); + + for (i = 0; i < RX_STATS_NUM; ++i) + *p++ = port->hw_stats[i]; + + } while (u64_stats_fetch_retry(&port->ir_stats_syncp, start)); + values = p; + + /* racing with RX NAPI */ + do { + p = values; + start = u64_stats_fetch_begin(&port->rx_stats_syncp); + + for (i = 0; i < RX_STATUS_NUM; ++i) + *p++ = port->rx_stats[i]; + for (i = 0; i < RX_CHKSUM_NUM; ++i) + *p++ = port->rx_csum_stats[i]; + *p++ = port->rx_napi_exits; + + } while (u64_stats_fetch_retry(&port->rx_stats_syncp, start)); + values = p; + + /* racing with TX start_xmit */ + do { + p = values; + start = u64_stats_fetch_begin(&port->tx_stats_syncp); + + for (i = 0; i < TX_MAX_FRAGS; ++i) { + *values++ = port->tx_frag_stats[i]; + port->tx_frag_stats[i] = 0; + } + *values++ = port->tx_frags_linearized; + *values++ = port->tx_hw_csummed; + + } while (u64_stats_fetch_retry(&port->tx_stats_syncp, start)); +} + +static int gmac_get_ksettings(struct net_device *netdev, + struct ethtool_link_ksettings *cmd) +{ + if (!netdev->phydev) + return -ENXIO; + phy_ethtool_ksettings_get(netdev->phydev, cmd); + + return 0; +} + +static int gmac_set_ksettings(struct net_device *netdev, + const struct ethtool_link_ksettings *cmd) +{ + if (!netdev->phydev) + return -ENXIO; + return phy_ethtool_ksettings_set(netdev->phydev, cmd); +} + +static int gmac_nway_reset(struct net_device *netdev) +{ + if (!netdev->phydev) + return -ENXIO; + return phy_start_aneg(netdev->phydev); +} + +static void gmac_get_pauseparam(struct net_device *netdev, + struct ethtool_pauseparam *pparam) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + GMAC_CONFIG0_T config0; + + config0.bits32 = readl(port->gmac_base + GMAC_CONFIG0); + + pparam->rx_pause = config0.bits.rx_fc_en; + pparam->tx_pause = config0.bits.tx_fc_en; + pparam->autoneg = true; +} + +static void gmac_get_ringparam(struct net_device *netdev, + struct ethtool_ringparam *rp) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + GMAC_CONFIG0_T config0; + + config0.bits32 = readl(port->gmac_base + GMAC_CONFIG0); + + rp->rx_max_pending = 1 << 15; + rp->rx_mini_max_pending = 0; + rp->rx_jumbo_max_pending = 0; + rp->tx_max_pending = 1 << 15; + + rp->rx_pending = 1 << port->rxq_order; + rp->rx_mini_pending = 0; + rp->rx_jumbo_pending = 0; + rp->tx_pending = 1 << port->txq_order; +} + +static int gmac_set_ringparam(struct net_device *netdev, + struct ethtool_ringparam *rp) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + int err = 0; + + if (netif_running(netdev)) + return -EBUSY; + + if (rp->rx_pending) { + port->rxq_order = min(15, ilog2(rp->rx_pending - 1) + 1); + err = geth_resize_freeq(port); + } + if (rp->tx_pending) { + port->txq_order = min(15, ilog2(rp->tx_pending - 1) + 1); + port->irq_every_tx_packets = 1 << (port->txq_order - 2); + } + + return err; +} + +static int gmac_get_coalesce(struct net_device *netdev, + struct ethtool_coalesce *ecmd) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + + ecmd->rx_max_coalesced_frames = 1; + ecmd->tx_max_coalesced_frames = port->irq_every_tx_packets; + ecmd->rx_coalesce_usecs = port->rx_coalesce_nsecs/1000; + + return 0; +} + +static int gmac_set_coalesce(struct net_device *netdev, + struct ethtool_coalesce *ecmd) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + + if (ecmd->tx_max_coalesced_frames < 1) + return -EINVAL; + if (ecmd->tx_max_coalesced_frames >= 1 << port->txq_order) + return -EINVAL; + + port->irq_every_tx_packets = ecmd->tx_max_coalesced_frames; + port->rx_coalesce_nsecs = ecmd->rx_coalesce_usecs * 1000; + + return 0; +} + +static u32 gmac_get_msglevel(struct net_device *netdev) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + return port->msg_enable; +} + +static void gmac_set_msglevel(struct net_device *netdev, u32 level) +{ + struct gemini_ethernet_port *port = netdev_priv(netdev); + port->msg_enable = level; +} + +static void gmac_get_drvinfo(struct net_device *netdev, + struct ethtool_drvinfo *info) +{ + strcpy(info->driver, DRV_NAME); + strcpy(info->version, DRV_VERSION); + strcpy(info->bus_info, netdev->dev_id ? "1" : "0"); +} + +static const struct net_device_ops gmac_351x_ops = { + .ndo_init = gmac_init, + .ndo_uninit = gmac_uninit, + .ndo_open = gmac_open, + .ndo_stop = gmac_stop, + .ndo_start_xmit = gmac_start_xmit, + .ndo_tx_timeout = gmac_tx_timeout, + .ndo_set_rx_mode = gmac_set_rx_mode, + .ndo_set_mac_address = gmac_set_mac_address, + .ndo_get_stats64 = gmac_get_stats64, + .ndo_change_mtu = gmac_change_mtu, + .ndo_fix_features = gmac_fix_features, + .ndo_set_features = gmac_set_features, +}; + +static const struct ethtool_ops gmac_351x_ethtool_ops = { + .get_sset_count = gmac_get_sset_count, + .get_strings = gmac_get_strings, + .get_ethtool_stats = gmac_get_ethtool_stats, + .get_link = ethtool_op_get_link, + .get_link_ksettings = gmac_get_ksettings, + .set_link_ksettings = gmac_set_ksettings, + .nway_reset = gmac_nway_reset, + .get_pauseparam = gmac_get_pauseparam, + .get_ringparam = gmac_get_ringparam, + .set_ringparam = gmac_set_ringparam, + .get_coalesce = gmac_get_coalesce, + .set_coalesce = gmac_set_coalesce, + .get_msglevel = gmac_get_msglevel, + .set_msglevel = gmac_set_msglevel, + .get_drvinfo = gmac_get_drvinfo, +}; + +static irqreturn_t gemini_port_irq_thread(int irq, void *data) +{ + struct gemini_ethernet_port *port = data; + struct gemini_ethernet *geth = port->geth; + unsigned long irqmask = SWFQ_EMPTY_INT_BIT; + unsigned long flags; + + geth_fill_freeq(geth, 0); + + spin_lock_irqsave(&geth->irq_lock, flags); + /* ACK queue interrupt */ + writel(irqmask, geth->base + GLOBAL_INTERRUPT_STATUS_4_REG); + /* Enable queue interrupt again */ + irqmask |= readl(geth->base + GLOBAL_INTERRUPT_ENABLE_4_REG); + writel(irqmask, geth->base + GLOBAL_INTERRUPT_ENABLE_4_REG); + spin_unlock_irqrestore(&geth->irq_lock, flags); + + return IRQ_HANDLED; +} + +static irqreturn_t gemini_port_irq(int irq, void *data) +{ + struct gemini_ethernet_port *port = data; + struct gemini_ethernet *geth = port->geth; + u32 val, en; + irqreturn_t ret = IRQ_NONE; + + spin_lock(&geth->irq_lock); + + val = readl(geth->base + GLOBAL_INTERRUPT_STATUS_4_REG); + en = readl(geth->base + GLOBAL_INTERRUPT_ENABLE_4_REG); + + if (val & en & SWFQ_EMPTY_INT_BIT) { + /* + * Disable the queue empty interrupt while we work on + * processing the queue. Also disable overrun interrupts + * as there is not much we can do about it here. + */ + en &= ~(SWFQ_EMPTY_INT_BIT | GMAC0_RX_OVERRUN_INT_BIT + | GMAC1_RX_OVERRUN_INT_BIT); + writel(en, geth->base + GLOBAL_INTERRUPT_ENABLE_4_REG); + ret = IRQ_WAKE_THREAD; + } + + spin_unlock(&geth->irq_lock); + + return ret; +} + +static void gemini_port_remove(struct gemini_ethernet_port *port) +{ + if (port->netdev) + unregister_netdev(port->netdev); + clk_disable_unprepare(port->pclk); + geth_cleanup_freeq(port->geth); +} + +static void gemini_ethernet_init(struct gemini_ethernet *geth) +{ + writel(0, geth->base + GLOBAL_INTERRUPT_ENABLE_0_REG); + writel(0, geth->base + GLOBAL_INTERRUPT_ENABLE_1_REG); + writel(0, geth->base + GLOBAL_INTERRUPT_ENABLE_2_REG); + writel(0, geth->base + GLOBAL_INTERRUPT_ENABLE_3_REG); + writel(0, geth->base + GLOBAL_INTERRUPT_ENABLE_4_REG); + + /* + * Interrupt config: + * + * GMAC0 intr bits ------> int0 ----> eth0 + * GMAC1 intr bits ------> int1 ----> eth1 + * TOE intr -------------> int1 ----> eth1 + * Classification Intr --> int0 ----> eth0 + * Default Q0 -----------> int0 ----> eth0 + * Default Q1 -----------> int1 ----> eth1 + * FreeQ intr -----------> int1 ----> eth1 + */ + writel(0xCCFC0FC0, geth->base + GLOBAL_INTERRUPT_SELECT_0_REG); + writel(0x00F00002, geth->base + GLOBAL_INTERRUPT_SELECT_1_REG); + writel(0xFFFFFFFF, geth->base + GLOBAL_INTERRUPT_SELECT_2_REG); + writel(0xFFFFFFFF, geth->base + GLOBAL_INTERRUPT_SELECT_3_REG); + writel(0xFF000003, geth->base + GLOBAL_INTERRUPT_SELECT_4_REG); + + /* edge-triggered interrupts packed to level-triggered one... */ + writel(~0, geth->base + GLOBAL_INTERRUPT_STATUS_0_REG); + writel(~0, geth->base + GLOBAL_INTERRUPT_STATUS_1_REG); + writel(~0, geth->base + GLOBAL_INTERRUPT_STATUS_2_REG); + writel(~0, geth->base + GLOBAL_INTERRUPT_STATUS_3_REG); + writel(~0, geth->base + GLOBAL_INTERRUPT_STATUS_4_REG); + + /* Set up queue */ + writel(0, geth->base + GLOBAL_SW_FREEQ_BASE_SIZE_REG); + writel(0, geth->base + GLOBAL_HW_FREEQ_BASE_SIZE_REG); + writel(0, geth->base + GLOBAL_SWFQ_RWPTR_REG); + writel(0, geth->base + GLOBAL_HWFQ_RWPTR_REG); + + geth->freeq_frag_order = DEFAULT_RX_BUF_ORDER; + /* + * This makes the queue resize on probe() so that we + * set up and enable the queue IRQ. FIXME: fragile. + */ + geth->freeq_order = 1; +} + +static void gemini_port_save_mac_addr(struct gemini_ethernet_port *port) +{ + port->mac_addr[0] = + cpu_to_le32(readl(port->gmac_base + GMAC_STA_ADD0)); + port->mac_addr[1] = + cpu_to_le32(readl(port->gmac_base + GMAC_STA_ADD1)); + port->mac_addr[2] = + cpu_to_le32(readl(port->gmac_base + GMAC_STA_ADD2)); +} + +static int gemini_ethernet_port_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct device *parent = dev->parent; + struct gemini_ethernet *geth = dev_get_drvdata(parent); + struct gemini_ethernet_port *port; + struct net_device *netdev; + struct resource *dmares, *gmacres; + char *port_names[2] = { "ethernet0", "ethernet1" }; + unsigned int id; + int irq; + int ret; + + if (!strcmp(dev_name(dev), "60008000.port0")) + id = 0; + else if (!strcmp(dev_name(dev), "6000c000.port1")) + id = 1; + else + return -ENODEV; + + dev_info(dev, "probe %s ID %d\n", dev_name(dev), id); + + netdev = alloc_etherdev_mq(sizeof(*port), TX_QUEUE_NUM); + if (!netdev) { + dev_err(dev, "Can't allocate ethernet device #%d\n", id); + return -ENOMEM; + } + + port = netdev_priv(netdev); + SET_NETDEV_DEV(netdev, dev); + port->netdev = netdev; + port->id = id; + port->geth = geth; + port->dev = dev; + + /* DMA memory */ + dmares = platform_get_resource(pdev, IORESOURCE_MEM, 0); + if (!dmares) { + dev_err(dev, "no DMA resource\n"); + return -ENODEV; + } + port->dma_base = devm_ioremap_resource(dev, dmares); + if (IS_ERR(port->dma_base)) + return PTR_ERR(port->dma_base); + + /* GMAC config memory */ + gmacres = platform_get_resource(pdev, IORESOURCE_MEM, 1); + if (!gmacres) { + dev_err(dev, "no GMAC resource\n"); + return -ENODEV; + } + port->gmac_base = devm_ioremap_resource(dev, gmacres); + if (IS_ERR(port->gmac_base)) + return PTR_ERR(port->gmac_base); + + /* Interrupt */ + irq = platform_get_irq(pdev, 0); + if (irq <= 0) { + dev_err(dev, "no IRQ\n"); + return irq ? irq : -ENODEV; + } + port->irq = irq; + + /* Clock the port */ + port->pclk = devm_clk_get(dev, "PCLK"); + if (IS_ERR(port->pclk)) { + dev_err(dev, "no PCLK\n"); + return PTR_ERR(port->pclk); + } + ret = clk_prepare_enable(port->pclk); + if (ret) + return ret; + + /* Maybe there is a nice ethernet address we should use */ + gemini_port_save_mac_addr(port); + + /* Reset the port */ + port->reset = devm_reset_control_get_exclusive(dev, NULL); + if (IS_ERR(port->reset)) { + dev_err(dev, "no reset\n"); + return PTR_ERR(port->reset); + } + reset_control_reset(port->reset); + udelay(100); + + /* Assign pointer in the main state container */ + if (!id) + geth->port0 = port; + else + geth->port1 = port; + platform_set_drvdata(pdev, port); + + /* Set up and register the netdev */ + netdev->dev_id = port->id; + netdev->irq = irq; + netdev->netdev_ops = &gmac_351x_ops; + netdev->ethtool_ops = &gmac_351x_ethtool_ops; + + spin_lock_init(&port->config_lock); + gmac_clear_hw_stats(netdev); + + netdev->hw_features = GMAC_OFFLOAD_FEATURES; + netdev->features |= GMAC_OFFLOAD_FEATURES | NETIF_F_GRO; + + port->freeq_refill = 0; + netif_napi_add(netdev, &port->napi, gmac_napi_poll, + DEFAULT_NAPI_WEIGHT); + + if (is_valid_ether_addr((void *)port->mac_addr)) { + memcpy(netdev->dev_addr, port->mac_addr, ETH_ALEN); + } else { + dev_dbg(dev, "ethernet address 0x%08x%08x%08x invalid\n", + port->mac_addr[0], port->mac_addr[1], + port->mac_addr[2]); + dev_info(dev, "using a random ethernet address\n"); + random_ether_addr(netdev->dev_addr); + } + gmac_write_mac_address(netdev); + + ret = devm_request_threaded_irq(port->dev, + port->irq, + gemini_port_irq, + gemini_port_irq_thread, + IRQF_SHARED, + port_names[port->id], + port); + if (ret) + return ret; + + ret = register_netdev(netdev); + if (!ret) { + netdev_info(netdev, + "irq %d, DMA @ 0x%08x, GMAC @ 0x%08x\n", + port->irq, dmares->start, + gmacres->start); + ret = gmac_setup_phy(netdev); + if (ret) + netdev_info(netdev, + "PHY init failed, deferring to ifup time\n"); + return 0; + } + + port->netdev = NULL; + free_netdev(netdev); + return ret; +} + +static int gemini_ethernet_port_remove(struct platform_device *pdev) +{ + struct gemini_ethernet_port *port = platform_get_drvdata(pdev); + + gemini_port_remove(port); + return 0; +} + +static const struct of_device_id gemini_ethernet_port_of_match[] = { + { + .compatible = "cortina,gemini-ethernet-port", + }, + {}, +}; +MODULE_DEVICE_TABLE(of, gemini_ethernet_port_of_match); + +static struct platform_driver gemini_ethernet_port_driver = { + .driver = { + .name = "gemini-ethernet-port", + .of_match_table = of_match_ptr(gemini_ethernet_port_of_match), + }, + .probe = gemini_ethernet_port_probe, + .remove = gemini_ethernet_port_remove, +}; +module_platform_driver(gemini_ethernet_port_driver); + +static int gemini_ethernet_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct device_node *np = dev->of_node; + struct resource *res; + struct gemini_ethernet *geth; + unsigned int retry = 5; + u32 val; + + /* Global registers */ + geth = devm_kzalloc(dev, sizeof(*geth), GFP_KERNEL); + if (!geth) + return -ENOMEM; + res = platform_get_resource(pdev, IORESOURCE_MEM, 0); + if (!res) + return -ENODEV; + geth->base = devm_ioremap_resource(dev, res); + if (IS_ERR(geth->base)) + return PTR_ERR(geth->base); + geth->dev = dev; + + /* + * Wait for ports to stabilize. + */ + do { + udelay(2); + val = readl(geth->base + GLOBAL_TOE_VERSION_REG); + barrier(); + } while (!val && --retry); + if (!retry) { + dev_err(dev, "failed to reset ethernet\n"); + return -EIO; + } + dev_info(dev, "Ethernet device ID: 0x%03x, revision 0x%01x\n", + (val >> 4) & 0xFFFU, val & 0xFU); + + spin_lock_init(&geth->irq_lock); + spin_lock_init(&geth->freeq_lock); + gemini_ethernet_init(geth); + + /* The children will use this */ + platform_set_drvdata(pdev, geth); + + /* Spawn child devices for the two ports */ + return devm_of_platform_populate(dev); +} + +static int gemini_ethernet_remove(struct platform_device *pdev) +{ + struct gemini_ethernet *geth = platform_get_drvdata(pdev); + + gemini_ethernet_init(geth); + geth_cleanup_freeq(geth); + + return 0; +} + +static const struct of_device_id gemini_ethernet_of_match[] = { + { + .compatible = "cortina,gemini-ethernet", + }, + {}, +}; +MODULE_DEVICE_TABLE(of, gemini_ethernet_of_match); + +static struct platform_driver gemini_ethernet_driver = { + .driver = { + .name = DRV_NAME, + .of_match_table = of_match_ptr(gemini_ethernet_of_match), + }, + .probe = gemini_ethernet_probe, + .remove = gemini_ethernet_remove, +}; +module_platform_driver(gemini_ethernet_driver); + +MODULE_AUTHOR("Linus Walleij <linus.wall...@linaro.org>"); +MODULE_DESCRIPTION("StorLink SL351x (Gemini) ethernet driver"); +MODULE_LICENSE("GPL"); +MODULE_ALIAS("platform:" DRV_NAME); diff --git a/drivers/net/ethernet/cortina/gemini.h b/drivers/net/ethernet/cortina/gemini.h new file mode 100644 index 000000000000..b8ab79cb3752 --- /dev/null +++ b/drivers/net/ethernet/cortina/gemini.h @@ -0,0 +1,1432 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Register definitions for Gemini LEPUS GMAC Ethernet device driver. + * + * Copyright (C) 2006, Storlink, Corp. + * Copyright (C) 2008-2009, Paulius Zaleckas <paulius.zalec...@teltonika.lt> + * Copyright (C) 2010, MichaÅ MirosÅaw <mirq-li...@rere.qmqm.pl> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + */ +#ifndef _GEMINI_ETHERNET_H +#define _GEMINI_ETHERNET_H + +#include <linux/bitops.h> + +/* + * Base Registers + */ +#define TOE_NONTOE_QUE_HDR_BASE 0x2000 +#define TOE_TOE_QUE_HDR_BASE 0x3000 + +/* + * Queue ID + */ +#define TOE_SW_FREE_QID 0x00 +#define TOE_HW_FREE_QID 0x01 +#define TOE_GMAC0_SW_TXQ0_QID 0x02 +#define TOE_GMAC0_SW_TXQ1_QID 0x03 +#define TOE_GMAC0_SW_TXQ2_QID 0x04 +#define TOE_GMAC0_SW_TXQ3_QID 0x05 +#define TOE_GMAC0_SW_TXQ4_QID 0x06 +#define TOE_GMAC0_SW_TXQ5_QID 0x07 +#define TOE_GMAC0_HW_TXQ0_QID 0x08 +#define TOE_GMAC0_HW_TXQ1_QID 0x09 +#define TOE_GMAC0_HW_TXQ2_QID 0x0A +#define TOE_GMAC0_HW_TXQ3_QID 0x0B +#define TOE_GMAC1_SW_TXQ0_QID 0x12 +#define TOE_GMAC1_SW_TXQ1_QID 0x13 +#define TOE_GMAC1_SW_TXQ2_QID 0x14 +#define TOE_GMAC1_SW_TXQ3_QID 0x15 +#define TOE_GMAC1_SW_TXQ4_QID 0x16 +#define TOE_GMAC1_SW_TXQ5_QID 0x17 +#define TOE_GMAC1_HW_TXQ0_QID 0x18 +#define TOE_GMAC1_HW_TXQ1_QID 0x19 +#define TOE_GMAC1_HW_TXQ2_QID 0x1A +#define TOE_GMAC1_HW_TXQ3_QID 0x1B +#define TOE_GMAC0_DEFAULT_QID 0x20 +#define TOE_GMAC1_DEFAULT_QID 0x21 +#define TOE_CLASSIFICATION_QID(x) (0x22 + x) /* 0x22 ~ 0x2F */ +#define TOE_TOE_QID(x) (0x40 + x) /* 0x40 ~ 0x7F */ + +/* + * old info: + * TOE DMA Queue Size should be 2^n, n = 6...12 + * TOE DMA Queues are the following queue types: + * SW Free Queue, HW Free Queue, + * GMAC 0/1 SW TX Q0-5, and GMAC 0/1 HW TX Q0-5 + * The base address and descriptor number are configured at + * DMA Queues Descriptor Ring Base Address/Size Register (offset 0x0004) + */ + +#define GET_WPTR(addr) readw((addr) + 2) +#define GET_RPTR(addr) readw((addr)) +#define SET_WPTR(addr, data) writew((data), (addr) + 2) +#define SET_RPTR(addr, data) writew((data), (addr)) +#define __RWPTR_NEXT(x, mask) (((unsigned int)(x) + 1) & (mask)) +#define __RWPTR_PREV(x, mask) (((unsigned int)(x) - 1) & (mask)) +#define __RWPTR_DISTANCE(r, w, mask) (((unsigned int)(w) - (r)) & (mask)) +#define __RWPTR_MASK(order) ((1 << (order)) - 1) +#define RWPTR_NEXT(x, order) __RWPTR_NEXT((x), __RWPTR_MASK((order))) +#define RWPTR_PREV(x, order) __RWPTR_PREV((x), __RWPTR_MASK((order))) +#define RWPTR_DISTANCE(r, w, order) __RWPTR_DISTANCE((r), (w), \ + __RWPTR_MASK((order))) + +/* + * Global registers + * #define TOE_GLOBAL_BASE (TOE_BASE + 0x0000) + */ +#define GLOBAL_TOE_VERSION_REG 0x0000 +#define GLOBAL_SW_FREEQ_BASE_SIZE_REG 0x0004 +#define GLOBAL_HW_FREEQ_BASE_SIZE_REG 0x0008 +#define GLOBAL_DMA_SKB_SIZE_REG 0x0010 +#define GLOBAL_SWFQ_RWPTR_REG 0x0014 +#define GLOBAL_HWFQ_RWPTR_REG 0x0018 +#define GLOBAL_INTERRUPT_STATUS_0_REG 0x0020 +#define GLOBAL_INTERRUPT_ENABLE_0_REG 0x0024 +#define GLOBAL_INTERRUPT_SELECT_0_REG 0x0028 +#define GLOBAL_INTERRUPT_STATUS_1_REG 0x0030 +#define GLOBAL_INTERRUPT_ENABLE_1_REG 0x0034 +#define GLOBAL_INTERRUPT_SELECT_1_REG 0x0038 +#define GLOBAL_INTERRUPT_STATUS_2_REG 0x0040 +#define GLOBAL_INTERRUPT_ENABLE_2_REG 0x0044 +#define GLOBAL_INTERRUPT_SELECT_2_REG 0x0048 +#define GLOBAL_INTERRUPT_STATUS_3_REG 0x0050 +#define GLOBAL_INTERRUPT_ENABLE_3_REG 0x0054 +#define GLOBAL_INTERRUPT_SELECT_3_REG 0x0058 +#define GLOBAL_INTERRUPT_STATUS_4_REG 0x0060 +#define GLOBAL_INTERRUPT_ENABLE_4_REG 0x0064 +#define GLOBAL_INTERRUPT_SELECT_4_REG 0x0068 +#define GLOBAL_HASH_TABLE_BASE_REG 0x006C +#define GLOBAL_QUEUE_THRESHOLD_REG 0x0070 + +/* + * GMAC 0/1 DMA/TOE register + * #define TOE_GMAC0_DMA_BASE (TOE_BASE + 0x8000) + * #define TOE_GMAC1_DMA_BASE (TOE_BASE + 0xC000) + * Base 0x60008000 or 0x6000C000 + */ +#define GMAC_DMA_CTRL_REG 0x0000 +#define GMAC_TX_WEIGHTING_CTRL_0_REG 0x0004 +#define GMAC_TX_WEIGHTING_CTRL_1_REG 0x0008 +#define GMAC_SW_TX_QUEUE0_PTR_REG 0x000C +#define GMAC_SW_TX_QUEUE1_PTR_REG 0x0010 +#define GMAC_SW_TX_QUEUE2_PTR_REG 0x0014 +#define GMAC_SW_TX_QUEUE3_PTR_REG 0x0018 +#define GMAC_SW_TX_QUEUE4_PTR_REG 0x001C +#define GMAC_SW_TX_QUEUE5_PTR_REG 0x0020 +#define GMAC_SW_TX_QUEUE_PTR_REG(i) (GMAC_SW_TX_QUEUE0_PTR_REG + 4 * (i)) +#define GMAC_HW_TX_QUEUE0_PTR_REG 0x0024 +#define GMAC_HW_TX_QUEUE1_PTR_REG 0x0028 +#define GMAC_HW_TX_QUEUE2_PTR_REG 0x002C +#define GMAC_HW_TX_QUEUE3_PTR_REG 0x0030 +#define GMAC_HW_TX_QUEUE_PTR_REG(i) (GMAC_HW_TX_QUEUE0_PTR_REG + 4 * (i)) +#define GMAC_DMA_TX_FIRST_DESC_REG 0x0038 +#define GMAC_DMA_TX_CURR_DESC_REG 0x003C +#define GMAC_DMA_TX_DESC_WORD0_REG 0x0040 +#define GMAC_DMA_TX_DESC_WORD1_REG 0x0044 +#define GMAC_DMA_TX_DESC_WORD2_REG 0x0048 +#define GMAC_DMA_TX_DESC_WORD3_REG 0x004C +#define GMAC_SW_TX_QUEUE_BASE_REG 0x0050 +#define GMAC_HW_TX_QUEUE_BASE_REG 0x0054 +#define GMAC_DMA_RX_FIRST_DESC_REG 0x0058 +#define GMAC_DMA_RX_CURR_DESC_REG 0x005C +#define GMAC_DMA_RX_DESC_WORD0_REG 0x0060 +#define GMAC_DMA_RX_DESC_WORD1_REG 0x0064 +#define GMAC_DMA_RX_DESC_WORD2_REG 0x0068 +#define GMAC_DMA_RX_DESC_WORD3_REG 0x006C +#define GMAC_HASH_ENGINE_REG0 0x0070 +#define GMAC_HASH_ENGINE_REG1 0x0074 +/* matching rule 0 Control register 0 */ +#define GMAC_MR0CR0 0x0078 +#define GMAC_MR0CR1 0x007C +#define GMAC_MR0CR2 0x0080 +#define GMAC_MR1CR0 0x0084 +#define GMAC_MR1CR1 0x0088 +#define GMAC_MR1CR2 0x008C +#define GMAC_MR2CR0 0x0090 +#define GMAC_MR2CR1 0x0094 +#define GMAC_MR2CR2 0x0098 +#define GMAC_MR3CR0 0x009C +#define GMAC_MR3CR1 0x00A0 +#define GMAC_MR3CR2 0x00A4 +/* Support Protocol Regsister 0 */ +#define GMAC_SPR0 0x00A8 +#define GMAC_SPR1 0x00AC +#define GMAC_SPR2 0x00B0 +#define GMAC_SPR3 0x00B4 +#define GMAC_SPR4 0x00B8 +#define GMAC_SPR5 0x00BC +#define GMAC_SPR6 0x00C0 +#define GMAC_SPR7 0x00C4 +/* GMAC Hash/Rx/Tx AHB Weighting register */ +#define GMAC_AHB_WEIGHT_REG 0x00C8 + +/* + * TOE GMAC 0/1 register + * #define TOE_GMAC0_BASE (TOE_BASE + 0xA000) + * #define TOE_GMAC1_BASE (TOE_BASE + 0xE000) + * Base 0x6000A000 or 0x6000E000 + */ +enum GMAC_REGISTER { + GMAC_STA_ADD0 = 0x0000, + GMAC_STA_ADD1 = 0x0004, + GMAC_STA_ADD2 = 0x0008, + GMAC_RX_FLTR = 0x000c, + GMAC_MCAST_FIL0 = 0x0010, + GMAC_MCAST_FIL1 = 0x0014, + GMAC_CONFIG0 = 0x0018, + GMAC_CONFIG1 = 0x001c, + GMAC_CONFIG2 = 0x0020, + GMAC_CONFIG3 = 0x0024, + GMAC_RESERVED = 0x0028, + GMAC_STATUS = 0x002c, + GMAC_IN_DISCARDS= 0x0030, + GMAC_IN_ERRORS = 0x0034, + GMAC_IN_MCAST = 0x0038, + GMAC_IN_BCAST = 0x003c, + GMAC_IN_MAC1 = 0x0040, /* for STA 1 MAC Address */ + GMAC_IN_MAC2 = 0x0044 /* for STA 2 MAC Address */ +}; + +#define RX_STATS_NUM 6 + +/* + * DMA Queues description Ring Base Address/Size Register (offset 0x0004) + */ +typedef union { + unsigned int bits32; + unsigned int base_size; +} DMA_Q_BASE_SIZE_T; +#define DMA_Q_BASE_MASK (~0x0f) + +/* + * DMA SKB Buffer register (offset 0x0008) + */ +typedef union { + unsigned int bits32; + struct bit_0008 { + unsigned int sw_skb_size : 16; /* SW Free poll SKB Size */ + unsigned int hw_skb_size : 16; /* HW Free poll SKB Size */ + } bits; +} DMA_SKB_SIZE_T; + +/* + * DMA SW Free Queue Read/Write Pointer Register (offset 0x000C) + */ +typedef union { + unsigned int bits32; + struct bit_000c { + unsigned int rptr : 16; /* Read Ptr, RO */ + unsigned int wptr : 16; /* Write Ptr, RW */ + } bits; +} DMA_RWPTR_T; + +/* + * DMA HW Free Queue Read/Write Pointer Register (offset 0x0010) + * see DMA_RWPTR_T structure + */ + +/* + * Interrupt Status Register 0 (offset 0x0020) + * Interrupt Mask Register 0 (offset 0x0024) + * Interrupt Select Register 0 (offset 0x0028) + */ +typedef union { + unsigned int bits32; + struct bit_0020 { + /* GMAC0 SW Tx Queue 0 EOF Interrupt */ + unsigned int swtq00_eof : 1; + unsigned int swtq01_eof : 1; + unsigned int swtq02_eof : 1; + unsigned int swtq03_eof : 1; + unsigned int swtq04_eof : 1; + unsigned int swtq05_eof : 1; + /* GMAC1 SW Tx Queue 0 EOF Interrupt */ + unsigned int swtq10_eof : 1; + unsigned int swtq11_eof : 1; + unsigned int swtq12_eof : 1; + unsigned int swtq13_eof : 1; + unsigned int swtq14_eof : 1; + unsigned int swtq15_eof : 1; + /* GMAC0 SW Tx Queue 0 Finish Interrupt */ + unsigned int swtq00_fin : 1; + unsigned int swtq01_fin : 1; + unsigned int swtq02_fin : 1; + unsigned int swtq03_fin : 1; + unsigned int swtq04_fin : 1; + unsigned int swtq05_fin : 1; + /* GMAC1 SW Tx Queue 0 Finish Interrupt */ + unsigned int swtq10_fin : 1; + unsigned int swtq11_fin : 1; + unsigned int swtq12_fin : 1; + unsigned int swtq13_fin : 1; + unsigned int swtq14_fin : 1; + unsigned int swtq15_fin : 1; + /* GMAC0 Rx Descriptor Protocol Error */ + unsigned int rxPerr0 : 1; + /* GMAC0 AHB Bus Error while Rx */ + unsigned int rxDerr0 : 1; + /* GMAC1 Rx Descriptor Protocol Error */ + unsigned int rxPerr1 : 1; + /* GMAC1 AHB Bus Error while Rx */ + unsigned int rxDerr1 : 1; + /* GMAC0 Tx Descriptor Protocol Error */ + unsigned int txPerr0 : 1; + /* GMAC0 AHB Bus Error while Tx */ + unsigned int txDerr0 : 1; + /* GMAC1 Tx Descriptor Protocol Error */ + unsigned int txPerr1 : 1; + /* GMAC1 AHB Bus Error while Tx */ + unsigned int txDerr1 : 1; + } bits; +} INTR_REG0_T; + +#define GMAC1_TXDERR_INT_BIT BIT(31) +#define GMAC1_TXPERR_INT_BIT BIT(30) +#define GMAC0_TXDERR_INT_BIT BIT(29) +#define GMAC0_TXPERR_INT_BIT BIT(28) +#define GMAC1_RXDERR_INT_BIT BIT(27) +#define GMAC1_RXPERR_INT_BIT BIT(26) +#define GMAC0_RXDERR_INT_BIT BIT(25) +#define GMAC0_RXPERR_INT_BIT BIT(24) +#define GMAC1_SWTQ15_FIN_INT_BIT BIT(23) +#define GMAC1_SWTQ14_FIN_INT_BIT BIT(22) +#define GMAC1_SWTQ13_FIN_INT_BIT BIT(21) +#define GMAC1_SWTQ12_FIN_INT_BIT BIT(20) +#define GMAC1_SWTQ11_FIN_INT_BIT BIT(19) +#define GMAC1_SWTQ10_FIN_INT_BIT BIT(18) +#define GMAC0_SWTQ05_FIN_INT_BIT BIT(17) +#define GMAC0_SWTQ04_FIN_INT_BIT BIT(16) +#define GMAC0_SWTQ03_FIN_INT_BIT BIT(15) +#define GMAC0_SWTQ02_FIN_INT_BIT BIT(14) +#define GMAC0_SWTQ01_FIN_INT_BIT BIT(13) +#define GMAC0_SWTQ00_FIN_INT_BIT BIT(12) +#define GMAC1_SWTQ15_EOF_INT_BIT BIT(11) +#define GMAC1_SWTQ14_EOF_INT_BIT BIT(10) +#define GMAC1_SWTQ13_EOF_INT_BIT BIT(9) +#define GMAC1_SWTQ12_EOF_INT_BIT BIT(8) +#define GMAC1_SWTQ11_EOF_INT_BIT BIT(7) +#define GMAC1_SWTQ10_EOF_INT_BIT BIT(6) +#define GMAC0_SWTQ05_EOF_INT_BIT BIT(5) +#define GMAC0_SWTQ04_EOF_INT_BIT BIT(4) +#define GMAC0_SWTQ03_EOF_INT_BIT BIT(3) +#define GMAC0_SWTQ02_EOF_INT_BIT BIT(2) +#define GMAC0_SWTQ01_EOF_INT_BIT BIT(1) +#define GMAC0_SWTQ00_EOF_INT_BIT BIT(0) + +/* + * Interrupt Status Register 1 (offset 0x0030) + * Interrupt Mask Register 1 (offset 0x0034) + * Interrupt Select Register 1 (offset 0x0038) + */ +typedef union { + unsigned int bits32; + struct bit_0030 { + unsigned int default_q0_eof : 1; /* Default Queue 0 EOF Interrupt */ + unsigned int default_q1_eof : 1; /* Default Queue 1 EOF Interrupt */ + unsigned int class_rx : 14; /* Classification Queue Rx Interrupt */ + unsigned int hwtq00_eof : 1; /* GMAC0 HW Tx Queue0 EOF Interrupt */ + unsigned int hwtq01_eof : 1; /* GMAC0 HW Tx Queue1 EOF Interrupt */ + unsigned int hwtq02_eof : 1; /* GMAC0 HW Tx Queue2 EOF Interrupt */ + unsigned int hwtq03_eof : 1; /* GMAC0 HW Tx Queue3 EOF Interrupt */ + unsigned int hwtq10_eof : 1; /* GMAC1 HW Tx Queue0 EOF Interrupt */ + unsigned int hwtq11_eof : 1; /* GMAC1 HW Tx Queue1 EOF Interrupt */ + unsigned int hwtq12_eof : 1; /* GMAC1 HW Tx Queue2 EOF Interrupt */ + unsigned int hwtq13_eof : 1; /* GMAC1 HW Tx Queue3 EOF Interrupt */ + unsigned int toe_iq0_intr : 1; /* TOE Interrupt Queue 0 with Interrupts */ + unsigned int toe_iq1_intr : 1; /* TOE Interrupt Queue 1 with Interrupts */ + unsigned int toe_iq2_intr : 1; /* TOE Interrupt Queue 2 with Interrupts */ + unsigned int toe_iq3_intr : 1; /* TOE Interrupt Queue 3 with Interrupts */ + unsigned int toe_iq0_full : 1; /* TOE Interrupt Queue 0 Full Interrupt */ + unsigned int toe_iq1_full : 1; /* TOE Interrupt Queue 1 Full Interrupt */ + unsigned int toe_iq2_full : 1; /* TOE Interrupt Queue 2 Full Interrupt */ + unsigned int toe_iq3_full : 1; /* TOE Interrupt Queue 3 Full Interrupt */ + } bits; +} INTR_REG1_T; + +#define TOE_IQ3_FULL_INT_BIT BIT(31) +#define TOE_IQ2_FULL_INT_BIT BIT(30) +#define TOE_IQ1_FULL_INT_BIT BIT(29) +#define TOE_IQ0_FULL_INT_BIT BIT(28) +#define TOE_IQ3_INT_BIT BIT(27) +#define TOE_IQ2_INT_BIT BIT(26) +#define TOE_IQ1_INT_BIT BIT(25) +#define TOE_IQ0_INT_BIT BIT(24) +#define GMAC1_HWTQ13_EOF_INT_BIT BIT(23) +#define GMAC1_HWTQ12_EOF_INT_BIT BIT(22) +#define GMAC1_HWTQ11_EOF_INT_BIT BIT(21) +#define GMAC1_HWTQ10_EOF_INT_BIT BIT(20) +#define GMAC0_HWTQ03_EOF_INT_BIT BIT(19) +#define GMAC0_HWTQ02_EOF_INT_BIT BIT(18) +#define GMAC0_HWTQ01_EOF_INT_BIT BIT(17) +#define GMAC0_HWTQ00_EOF_INT_BIT BIT(16) +#define CLASS_RX_INT_BIT(x) BIT((x + 2)) +#define DEFAULT_Q1_INT_BIT BIT(1) +#define DEFAULT_Q0_INT_BIT BIT(0) + +#define TOE_IQ_INT_BITS (TOE_IQ0_INT_BIT | TOE_IQ1_INT_BIT | \ + TOE_IQ2_INT_BIT | TOE_IQ3_INT_BIT) +#define TOE_IQ_FULL_BITS (TOE_IQ0_FULL_INT_BIT | TOE_IQ1_FULL_INT_BIT | \ + TOE_IQ2_FULL_INT_BIT | TOE_IQ3_FULL_INT_BIT) +#define TOE_IQ_ALL_BITS (TOE_IQ_INT_BITS | TOE_IQ_FULL_BITS) +#define TOE_CLASS_RX_INT_BITS 0xfffc + +/* + * Interrupt Status Register 2 (offset 0x0040) + * Interrupt Mask Register 2 (offset 0x0044) + * Interrupt Select Register 2 (offset 0x0048) + */ +typedef union { + unsigned int bits32; + struct bit_0040 { + unsigned int toe_q0_full : 1; /* bit 0 TOE Queue 0 Full Interrupt */ + unsigned int toe_q1_full : 1; /* bit 1 TOE Queue 1 Full Interrupt */ + unsigned int toe_q2_full : 1; /* bit 2 TOE Queue 2 Full Interrupt */ + unsigned int toe_q3_full : 1; /* bit 3 TOE Queue 3 Full Interrupt */ + unsigned int toe_q4_full : 1; /* bit 4 TOE Queue 4 Full Interrupt */ + unsigned int toe_q5_full : 1; /* bit 5 TOE Queue 5 Full Interrupt */ + unsigned int toe_q6_full : 1; /* bit 6 TOE Queue 6 Full Interrupt */ + unsigned int toe_q7_full : 1; /* bit 7 TOE Queue 7 Full Interrupt */ + unsigned int toe_q8_full : 1; /* bit 8 TOE Queue 8 Full Interrupt */ + unsigned int toe_q9_full : 1; /* bit 9 TOE Queue 9 Full Interrupt */ + unsigned int toe_q10_full : 1; /* bit 10 TOE Queue 10 Full Interrupt */ + unsigned int toe_q11_full : 1; /* bit 11 TOE Queue 11 Full Interrupt */ + unsigned int toe_q12_full : 1; /* bit 12 TOE Queue 12 Full Interrupt */ + unsigned int toe_q13_full : 1; /* bit 13 TOE Queue 13 Full Interrupt */ + unsigned int toe_q14_full : 1; /* bit 14 TOE Queue 14 Full Interrupt */ + unsigned int toe_q15_full : 1; /* bit 15 TOE Queue 15 Full Interrupt */ + unsigned int toe_q16_full : 1; /* bit 16 TOE Queue 16 Full Interrupt */ + unsigned int toe_q17_full : 1; /* bit 17 TOE Queue 17 Full Interrupt */ + unsigned int toe_q18_full : 1; /* bit 18 TOE Queue 18 Full Interrupt */ + unsigned int toe_q19_full : 1; /* bit 19 TOE Queue 19 Full Interrupt */ + unsigned int toe_q20_full : 1; /* bit 20 TOE Queue 20 Full Interrupt */ + unsigned int toe_q21_full : 1; /* bit 21 TOE Queue 21 Full Interrupt */ + unsigned int toe_q22_full : 1; /* bit 22 TOE Queue 22 Full Interrupt */ + unsigned int toe_q23_full : 1; /* bit 23 TOE Queue 23 Full Interrupt */ + unsigned int toe_q24_full : 1; /* bit 24 TOE Queue 24 Full Interrupt */ + unsigned int toe_q25_full : 1; /* bit 25 TOE Queue 25 Full Interrupt */ + unsigned int toe_q26_full : 1; /* bit 26 TOE Queue 26 Full Interrupt */ + unsigned int toe_q27_full : 1; /* bit 27 TOE Queue 27 Full Interrupt */ + unsigned int toe_q28_full : 1; /* bit 28 TOE Queue 28 Full Interrupt */ + unsigned int toe_q29_full : 1; /* bit 29 TOE Queue 29 Full Interrupt */ + unsigned int toe_q30_full : 1; /* bit 30 TOE Queue 30 Full Interrupt */ + unsigned int toe_q31_full : 1; /* bit 31 TOE Queue 31 Full Interrupt */ + } bits; +} INTR_REG2_T; + +#define TOE_QL_FULL_INT_BIT(x) BIT(x) + +/* + * Interrupt Status Register 3 (offset 0x0050) + * Interrupt Mask Register 3 (offset 0x0054) + * Interrupt Select Register 3 (offset 0x0058) + */ +typedef union { + unsigned int bits32; + struct bit_0050 { + unsigned int toe_q32_full : 1; /* bit 32 TOE Queue 32 Full Interrupt */ + unsigned int toe_q33_full : 1; /* bit 33 TOE Queue 33 Full Interrupt */ + unsigned int toe_q34_full : 1; /* bit 34 TOE Queue 34 Full Interrupt */ + unsigned int toe_q35_full : 1; /* bit 35 TOE Queue 35 Full Interrupt */ + unsigned int toe_q36_full : 1; /* bit 36 TOE Queue 36 Full Interrupt */ + unsigned int toe_q37_full : 1; /* bit 37 TOE Queue 37 Full Interrupt */ + unsigned int toe_q38_full : 1; /* bit 38 TOE Queue 38 Full Interrupt */ + unsigned int toe_q39_full : 1; /* bit 39 TOE Queue 39 Full Interrupt */ + unsigned int toe_q40_full : 1; /* bit 40 TOE Queue 40 Full Interrupt */ + unsigned int toe_q41_full : 1; /* bit 41 TOE Queue 41 Full Interrupt */ + unsigned int toe_q42_full : 1; /* bit 42 TOE Queue 42 Full Interrupt */ + unsigned int toe_q43_full : 1; /* bit 43 TOE Queue 43 Full Interrupt */ + unsigned int toe_q44_full : 1; /* bit 44 TOE Queue 44 Full Interrupt */ + unsigned int toe_q45_full : 1; /* bit 45 TOE Queue 45 Full Interrupt */ + unsigned int toe_q46_full : 1; /* bit 46 TOE Queue 46 Full Interrupt */ + unsigned int toe_q47_full : 1; /* bit 47 TOE Queue 47 Full Interrupt */ + unsigned int toe_q48_full : 1; /* bit 48 TOE Queue 48 Full Interrupt */ + unsigned int toe_q49_full : 1; /* bit 49 TOE Queue 49 Full Interrupt */ + unsigned int toe_q50_full : 1; /* bit 50 TOE Queue 50 Full Interrupt */ + unsigned int toe_q51_full : 1; /* bit 51 TOE Queue 51 Full Interrupt */ + unsigned int toe_q52_full : 1; /* bit 52 TOE Queue 52 Full Interrupt */ + unsigned int toe_q53_full : 1; /* bit 53 TOE Queue 53 Full Interrupt */ + unsigned int toe_q54_full : 1; /* bit 54 TOE Queue 54 Full Interrupt */ + unsigned int toe_q55_full : 1; /* bit 55 TOE Queue 55 Full Interrupt */ + unsigned int toe_q56_full : 1; /* bit 56 TOE Queue 56 Full Interrupt */ + unsigned int toe_q57_full : 1; /* bit 57 TOE Queue 57 Full Interrupt */ + unsigned int toe_q58_full : 1; /* bit 58 TOE Queue 58 Full Interrupt */ + unsigned int toe_q59_full : 1; /* bit 59 TOE Queue 59 Full Interrupt */ + unsigned int toe_q60_full : 1; /* bit 60 TOE Queue 60 Full Interrupt */ + unsigned int toe_q61_full : 1; /* bit 61 TOE Queue 61 Full Interrupt */ + unsigned int toe_q62_full : 1; /* bit 62 TOE Queue 62 Full Interrupt */ + unsigned int toe_q63_full : 1; /* bit 63 TOE Queue 63 Full Interrupt */ + } bits; +} INTR_REG3_T; + +#define TOE_QH_FULL_INT_BIT(x) BIT(x-32) + +/* + * Interrupt Status Register 4 (offset 0x0060) + * Interrupt Mask Register 4 (offset 0x0064) + * Interrupt Select Register 4 (offset 0x0068) + */ +typedef union { + unsigned char byte; + struct bit_0060 { + unsigned char status_changed : 1; /* Status Changed Intr for RGMII Mode */ + unsigned char rx_overrun : 1; /* GMAC Rx FIFO overrun interrupt */ + unsigned char tx_pause_off : 1; /* received pause off frame interrupt */ + unsigned char rx_pause_off : 1; /* received pause off frame interrupt */ + unsigned char tx_pause_on : 1; /* transmit pause on frame interrupt */ + unsigned char rx_pause_on : 1; /* received pause on frame interrupt */ + unsigned char cnt_full : 1; /* MIB counters half full interrupt */ + unsigned char reserved : 1; /* */ + } __packed bits; +} __packed GMAC_INTR_T; + +typedef union { + unsigned int bits32; + struct bit_0060_2 { + unsigned int swfq_empty : 1; /* bit 0 Software Free Queue Empty Intr. */ + unsigned int hwfq_empty : 1; /* bit 1 Hardware Free Queue Empty Intr. */ + unsigned int class_qf_int : 14; /* bit 15:2 Classification Rx Queue13-0 Full Intr. */ + GMAC_INTR_T gmac0; + GMAC_INTR_T gmac1; + } bits; +} INTR_REG4_T; + +#define GMAC1_RESERVED_INT_BIT BIT(31) +#define GMAC1_MIB_INT_BIT BIT(30) +#define GMAC1_RX_PAUSE_ON_INT_BIT BIT(29) +#define GMAC1_TX_PAUSE_ON_INT_BIT BIT(28) +#define GMAC1_RX_PAUSE_OFF_INT_BIT BIT(27) +#define GMAC1_TX_PAUSE_OFF_INT_BIT BIT(26) +#define GMAC1_RX_OVERRUN_INT_BIT BIT(25) +#define GMAC1_STATUS_CHANGE_INT_BIT BIT(24) +#define GMAC0_RESERVED_INT_BIT BIT(23) +#define GMAC0_MIB_INT_BIT BIT(22) +#define GMAC0_RX_PAUSE_ON_INT_BIT BIT(21) +#define GMAC0_TX_PAUSE_ON_INT_BIT BIT(20) +#define GMAC0_RX_PAUSE_OFF_INT_BIT BIT(19) +#define GMAC0_TX_PAUSE_OFF_INT_BIT BIT(18) +#define GMAC0_RX_OVERRUN_INT_BIT BIT(17) +#define GMAC0_STATUS_CHANGE_INT_BIT BIT(16) +#define CLASS_RX_FULL_INT_BIT(x) BIT((x+2)) +#define HWFQ_EMPTY_INT_BIT BIT(1) +#define SWFQ_EMPTY_INT_BIT BIT(0) + +#define GMAC0_INT_BITS (GMAC0_RESERVED_INT_BIT | GMAC0_MIB_INT_BIT | \ + GMAC0_RX_PAUSE_ON_INT_BIT | GMAC0_TX_PAUSE_ON_INT_BIT | \ + GMAC0_RX_PAUSE_OFF_INT_BIT | GMAC0_TX_PAUSE_OFF_INT_BIT | \ + GMAC0_RX_OVERRUN_INT_BIT | GMAC0_STATUS_CHANGE_INT_BIT) +#define GMAC1_INT_BITS (GMAC1_RESERVED_INT_BIT | GMAC1_MIB_INT_BIT | \ + GMAC1_RX_PAUSE_ON_INT_BIT | GMAC1_TX_PAUSE_ON_INT_BIT | \ + GMAC1_RX_PAUSE_OFF_INT_BIT | GMAC1_TX_PAUSE_OFF_INT_BIT | \ + GMAC1_RX_OVERRUN_INT_BIT | GMAC1_STATUS_CHANGE_INT_BIT) + +#define CLASS_RX_FULL_INT_BITS 0xfffc + +/* + * GLOBAL_QUEUE_THRESHOLD_REG (offset 0x0070) + */ +typedef union { + unsigned int bits32; + struct bit_0070_2 { + unsigned int swfq_empty : 8; /* 7:0 Software Free Queue Empty Threshold */ + unsigned int hwfq_empty : 8; /* 15:8 Hardware Free Queue Empty Threshold */ + unsigned int intrq : 8; /* 23:16 */ + unsigned int toe_class : 8; /* 31:24 */ + } bits; +} QUEUE_THRESHOLD_T; + + +/* + * GMAC DMA Control Register + * GMAC0 offset 0x8000 + * GMAC1 offset 0xC000 + */ +typedef union { + unsigned int bits32; + struct bit_8000 { + unsigned int td_bus : 2; /* bit 1:0 Peripheral Bus Width */ + unsigned int td_burst_size : 2; /* bit 3:2 TxDMA max burst size for every AHB request */ + unsigned int td_prot : 4; /* bit 7:4 TxDMA protection control */ + unsigned int rd_bus : 2; /* bit 9:8 Peripheral Bus Width */ + unsigned int rd_burst_size : 2; /* bit 11:10 DMA max burst size for every AHB request */ + unsigned int rd_prot : 4; /* bit 15:12 DMA Protection Control */ + unsigned int rd_insert_bytes : 2; /* bit 17:16 */ + unsigned int reserved : 10; /* bit 27:18 */ + unsigned int drop_small_ack : 1; /* bit 28 1: Drop, 0: Accept */ + unsigned int loopback : 1; /* bit 29 Loopback TxDMA to RxDMA */ + unsigned int td_enable : 1; /* bit 30 Tx DMA Enable */ + unsigned int rd_enable : 1; /* bit 31 Rx DMA Enable */ + } bits; +} GMAC_DMA_CTRL_T; + +/* + * GMAC Tx Weighting Control Register 0 + * GMAC0 offset 0x8004 + * GMAC1 offset 0xC004 + */ +typedef union { + unsigned int bits32; + struct bit_8004 { + unsigned int hw_tq0 : 6; /* bit 5:0 HW TX Queue 3 */ + unsigned int hw_tq1 : 6; /* bit 11:6 HW TX Queue 2 */ + unsigned int hw_tq2 : 6; /* bit 17:12 HW TX Queue 1 */ + unsigned int hw_tq3 : 6; /* bit 23:18 HW TX Queue 0 */ + unsigned int reserved : 8; /* bit 31:24 */ + } bits; +} GMAC_TX_WCR0_T; /* Weighting Control Register 0 */ + +/* + * GMAC Tx Weighting Control Register 1 + * GMAC0 offset 0x8008 + * GMAC1 offset 0xC008 + */ +typedef union { + unsigned int bits32; + struct bit_8008 { + unsigned int sw_tq0 : 5; /* bit 4:0 SW TX Queue 0 */ + unsigned int sw_tq1 : 5; /* bit 9:5 SW TX Queue 1 */ + unsigned int sw_tq2 : 5; /* bit 14:10 SW TX Queue 2 */ + unsigned int sw_tq3 : 5; /* bit 19:15 SW TX Queue 3 */ + unsigned int sw_tq4 : 5; /* bit 24:20 SW TX Queue 4 */ + unsigned int sw_tq5 : 5; /* bit 29:25 SW TX Queue 5 */ + unsigned int reserved : 2; /* bit 31:30 */ + } bits; +} GMAC_TX_WCR1_T; /* Weighting Control Register 1 */ + +/* + * Queue Read/Write Pointer + * GMAC SW TX Queue 0~5 Read/Write Pointer register + * GMAC0 offset 0x800C ~ 0x8020 + * GMAC1 offset 0xC00C ~ 0xC020 + * GMAC HW TX Queue 0~3 Read/Write Pointer register + * GMAC0 offset 0x8024 ~ 0x8030 + * GMAC1 offset 0xC024 ~ 0xC030 + * + * see DMA_RWPTR_T structure + */ + +/* + * GMAC DMA Tx First Description Address Register + * GMAC0 offset 0x8038 + * GMAC1 offset 0xC038 + */ +typedef union { + unsigned int bits32; + struct bit_8038 { + unsigned int reserved : 3; + unsigned int td_busy : 1; /* bit 3 1: TxDMA busy; 0: TxDMA idle */ + unsigned int td_first_des_ptr : 28; /* bit 31:4 first descriptor address */ + } bits; +} GMAC_TXDMA_FIRST_DESC_T; + +/* + * GMAC DMA Tx Current Description Address Register + * GMAC0 offset 0x803C + * GMAC1 offset 0xC03C + */ +typedef union { + unsigned int bits32; + struct bit_803C { + unsigned int reserved : 4; + unsigned int td_curr_desc_ptr : 28; /* bit 31:4 current descriptor address */ + } bits; +} GMAC_TXDMA_CURR_DESC_T; + +/* + * GMAC DMA Tx Description Word 0 Register + * GMAC0 offset 0x8040 + * GMAC1 offset 0xC040 + */ +typedef union { + unsigned int bits32; + struct bit_8040 { + unsigned int buffer_size : 16; /* bit 15:0 Transfer size */ + unsigned int desc_count : 6; /* bit 21:16 number of descriptors used for the current frame */ + unsigned int status_tx_ok : 1; /* bit 22 Tx Status, 1: Successful 0: Failed */ + unsigned int status_rvd : 6; /* bit 28:23 Tx Status, Reserved bits */ + unsigned int perr : 1; /* bit 29 protocol error during processing this descriptor */ + unsigned int derr : 1; /* bit 30 data error during processing this descriptor */ + unsigned int reserved : 1; /* bit 31 */ + } bits; +} GMAC_TXDESC_0_T; + +/* + * GMAC DMA Tx Description Word 1 Register + * GMAC0 offset 0x8044 + * GMAC1 offset 0xC044 + */ +typedef union { + unsigned int bits32; + struct txdesc_word1 { + unsigned int byte_count : 16; /* bit 15: 0 Tx Frame Byte Count */ + unsigned int mtu_enable : 1; /* bit 16 TSS segmentation use MTU setting */ + unsigned int ip_chksum : 1; /* bit 17 IPV4 Header Checksum Enable */ + unsigned int ipv6_enable : 1; /* bit 18 IPV6 Tx Enable */ + unsigned int tcp_chksum : 1; /* bit 19 TCP Checksum Enable */ + unsigned int udp_chksum : 1; /* bit 20 UDP Checksum Enable */ + unsigned int bypass_tss : 1; /* bit 21 Bypass HW offload engine */ + unsigned int ip_fixed_len : 1; /* bit 22 Don't update IP length field */ + unsigned int reserved : 9; /* bit 31:23 Tx Flag, Reserved */ + } bits; +} GMAC_TXDESC_1_T; + +#define TSS_IP_FIXED_LEN_BIT BIT(22) +#define TSS_BYPASS_BIT BIT(21) +#define TSS_UDP_CHKSUM_BIT BIT(20) +#define TSS_TCP_CHKSUM_BIT BIT(19) +#define TSS_IPV6_ENABLE_BIT BIT(18) +#define TSS_IP_CHKSUM_BIT BIT(17) +#define TSS_MTU_ENABLE_BIT BIT(16) + +#define TSS_CHECKUM_ENABLE \ + (TSS_IP_CHKSUM_BIT|TSS_IPV6_ENABLE_BIT| \ + TSS_TCP_CHKSUM_BIT|TSS_UDP_CHKSUM_BIT) + +/* + * GMAC DMA Tx Description Word 2 Register + * GMAC0 offset 0x8048 + * GMAC1 offset 0xC048 + */ +typedef union { + unsigned int bits32; + unsigned int buf_adr; +} GMAC_TXDESC_2_T; + +/* + * GMAC DMA Tx Description Word 3 Register + * GMAC0 offset 0x804C + * GMAC1 offset 0xC04C + */ +typedef union { + unsigned int bits32; + struct txdesc_word3 { + unsigned int mtu_size : 13; /* bit 12: 0 Tx Frame Byte Count */ + unsigned int reserved : 16; /* bit 28:13 */ + unsigned int eofie : 1; /* bit 29 End of frame interrupt enable */ + unsigned int sof_eof : 2; /* bit 31:30 11: only one, 10: first, 01: last, 00: linking */ + } bits; +} GMAC_TXDESC_3_T; +#define SOF_EOF_BIT_MASK 0x3fffffff +#define SOF_BIT 0x80000000 +#define EOF_BIT 0x40000000 +#define EOFIE_BIT BIT(29) +#define MTU_SIZE_BIT_MASK 0x1fff + +/* + * GMAC Tx Descriptor + */ +typedef struct { + GMAC_TXDESC_0_T word0; + GMAC_TXDESC_1_T word1; + GMAC_TXDESC_2_T word2; + GMAC_TXDESC_3_T word3; +} GMAC_TXDESC_T; + +/* + * GMAC DMA Rx First Description Address Register + * GMAC0 offset 0x8058 + * GMAC1 offset 0xC058 + */ +typedef union { + unsigned int bits32; + struct bit_8058 { + unsigned int reserved : 3; /* bit 2:0 */ + unsigned int rd_busy : 1; /* bit 3 1-RxDMA busy; 0-RxDMA idle */ + unsigned int rd_first_des_ptr : 28; /* bit 31:4 first descriptor address */ + } bits; +} GMAC_RXDMA_FIRST_DESC_T; + +/* + * GMAC DMA Rx Current Description Address Register + * GMAC0 offset 0x805C + * GMAC1 offset 0xC05C + */ +typedef union { + unsigned int bits32; + struct bit_805C { + unsigned int reserved : 4; /* bit 3:0 */ + unsigned int rd_curr_des_ptr : 28; /* bit 31:4 current descriptor address */ + } bits; +} GMAC_RXDMA_CURR_DESC_T; + +/* + * GMAC DMA Rx Description Word 0 Register + * GMAC0 offset 0x8060 + * GMAC1 offset 0xC060 + */ +typedef union { + unsigned int bits32; + struct bit_8060 { + unsigned int buffer_size : 16; /* bit 15:0 number of descriptors used for the current frame */ + unsigned int desc_count : 6; /* bit 21:16 number of descriptors used for the current frame */ + unsigned int status : 4; /* bit 24:22 Status of rx frame */ + unsigned int chksum_status : 3; /* bit 28:26 Check Sum Status */ + unsigned int perr : 1; /* bit 29 protocol error during processing this descriptor */ + unsigned int derr : 1; /* bit 30 data error during processing this descriptor */ + unsigned int drop : 1; /* bit 31 TOE/CIS Queue Full dropped packet to default queue */ + } bits; +} GMAC_RXDESC_0_T; + +#define GMAC_RXDESC_0_T_derr BIT(30) +#define GMAC_RXDESC_0_T_perr BIT(29) +#define GMAC_RXDESC_0_T_chksum_status(x) BIT((x+26)) +#define GMAC_RXDESC_0_T_status(x) BIT((x+22)) +#define GMAC_RXDESC_0_T_desc_count(x) BIT((x+16)) + +#define RX_CHKSUM_IP_UDP_TCP_OK 0 +#define RX_CHKSUM_IP_OK_ONLY 1 +#define RX_CHKSUM_NONE 2 +#define RX_CHKSUM_IP_ERR_UNKNOWN 4 +#define RX_CHKSUM_IP_ERR 5 +#define RX_CHKSUM_TCP_UDP_ERR 6 +#define RX_CHKSUM_NUM 8 + +#define RX_STATUS_GOOD_FRAME 0 +#define RX_STATUS_TOO_LONG_GOOD_CRC 1 +#define RX_STATUS_RUNT_FRAME 2 +#define RX_STATUS_SFD_NOT_FOUND 3 +#define RX_STATUS_CRC_ERROR 4 +#define RX_STATUS_TOO_LONG_BAD_CRC 5 +#define RX_STATUS_ALIGNMENT_ERROR 6 +#define RX_STATUS_TOO_LONG_BAD_ALIGN 7 +#define RX_STATUS_RX_ERR 8 +#define RX_STATUS_DA_FILTERED 9 +#define RX_STATUS_BUFFER_FULL 10 +#define RX_STATUS_NUM 16 + +#define RX_ERROR_LENGTH(s) \ + ((s) == RX_STATUS_TOO_LONG_GOOD_CRC || \ + (s) == RX_STATUS_TOO_LONG_BAD_CRC || \ + (s) == RX_STATUS_TOO_LONG_BAD_ALIGN) +#define RX_ERROR_OVER(s) \ + ((s) == RX_STATUS_BUFFER_FULL) +#define RX_ERROR_CRC(s) \ + ((s) == RX_STATUS_CRC_ERROR || \ + (s) == RX_STATUS_TOO_LONG_BAD_CRC) +#define RX_ERROR_FRAME(s) \ + ((s) == RX_STATUS_ALIGNMENT_ERROR || \ + (s) == RX_STATUS_TOO_LONG_BAD_ALIGN) +#define RX_ERROR_FIFO(s) \ + (0) + +/* + * GMAC DMA Rx Description Word 1 Register + * GMAC0 offset 0x8064 + * GMAC1 offset 0xC064 + */ +typedef union { + unsigned int bits32; + struct rxdesc_word1 { + unsigned int byte_count : 16; /* bit 15: 0 Rx Frame Byte Count */ + unsigned int sw_id : 16; /* bit 31:16 Software ID */ + } bits; +} GMAC_RXDESC_1_T; + +/* + * GMAC DMA Rx Description Word 2 Register + * GMAC0 offset 0x8068 + * GMAC1 offset 0xC068 + */ +typedef union { + unsigned int bits32; + unsigned int buf_adr; +} GMAC_RXDESC_2_T; + +#define RX_INSERT_NONE 0 +#define RX_INSERT_1_BYTE 1 +#define RX_INSERT_2_BYTE 2 +#define RX_INSERT_3_BYTE 3 + +/* + * GMAC DMA Rx Description Word 3 Register + * GMAC0 offset 0x806C + * GMAC1 offset 0xC06C + */ +typedef union { + unsigned int bits32; + struct rxdesc_word3 { + unsigned int l3_offset : 8; /* bit 7: 0 L3 data offset */ + unsigned int l4_offset : 8; /* bit 15: 8 L4 data offset */ + unsigned int l7_offset : 8; /* bit 23: 16 L7 data offset */ + unsigned int dup_ack : 1; /* bit 24 Duplicated ACK detected */ + unsigned int abnormal : 1; /* bit 25 abnormal case found */ + unsigned int option : 1; /* bit 26 IPV4 option or IPV6 extension header */ + unsigned int out_of_seq : 1; /* bit 27 Out of Sequence packet */ + unsigned int ctrl_flag : 1; /* bit 28 Control Flag is present */ + unsigned int eofie : 1; /* bit 29 End of frame interrupt enable */ + unsigned int sof_eof : 2; /* bit 31:30 11: only one, 10: first, 01: last, 00: linking */ + } bits; +} GMAC_RXDESC_3_T; + +/* + * GMAC Rx Descriptor + */ +typedef struct { + GMAC_RXDESC_0_T word0; + GMAC_RXDESC_1_T word1; + GMAC_RXDESC_2_T word2; + GMAC_RXDESC_3_T word3; +} GMAC_RXDESC_T; + +/* + * GMAC Hash Engine Enable/Action Register 0 Offset Register + * GMAC0 offset 0x8070 + * GMAC1 offset 0xC070 + */ +typedef union { + unsigned int bits32; + struct bit_8070 { + unsigned int mr0hel : 6; /* bit 5:0 match rule 0 hash entry size */ + unsigned int mr0_action : 5; /* bit 10:6 Matching Rule 0 action offset */ + unsigned int reserved0 : 4; /* bit 14:11 */ + unsigned int mr0en : 1; /* bit 15 Enable Matching Rule 0 */ + unsigned int mr1hel : 6; /* bit 21:16 match rule 1 hash entry size */ + unsigned int mr1_action : 5; /* bit 26:22 Matching Rule 1 action offset */ + unsigned int timing : 3; /* bit 29:27 */ + unsigned int reserved1 : 1; /* bit 30 */ + unsigned int mr1en : 1; /* bit 31 Enable Matching Rule 1 */ + } bits; +} GMAC_HASH_ENABLE_REG0_T; + +/* + * GMAC Hash Engine Enable/Action Register 1 Offset Register + * GMAC0 offset 0x8074 + * GMAC1 offset 0xC074 + */ +typedef union { + unsigned int bits32; + struct bit_8074 { + unsigned int mr2hel : 6; /* bit 5:0 match rule 2 hash entry size */ + unsigned int mr2_action : 5; /* bit 10:6 Matching Rule 2 action offset */ + unsigned int reserved2 : 4; /* bit 14:11 */ + unsigned int mr2en : 1; /* bit 15 Enable Matching Rule 2 */ + unsigned int mr3hel : 6; /* bit 21:16 match rule 3 hash entry size */ + unsigned int mr3_action : 5; /* bit 26:22 Matching Rule 3 action offset */ + unsigned int reserved1 : 4; /* bit 30:27 */ + unsigned int mr3en : 1; /* bit 31 Enable Matching Rule 3 */ + } bits; +} GMAC_HASH_ENABLE_REG1_T; + +/* + * GMAC Matching Rule Control Register 0 + * GMAC0 offset 0x8078 + * GMAC1 offset 0xC078 + */ +typedef union { + unsigned int bits32; + struct bit_8078 { + unsigned int sprx : 8; /* bit 7:0 Support Protocol Register 7:0 */ + unsigned int reserved2 : 4; /* bit 11:8 */ + unsigned int tos_traffic : 1; /* bit 12 IPV4 TOS or IPV6 Traffice Class */ + unsigned int flow_lable : 1; /* bit 13 IPV6 Flow label */ + unsigned int ip_hdr_len : 1; /* bit 14 IPV4 Header length */ + unsigned int ip_version : 1; /* bit 15 0: IPV4, 1: IPV6 */ + unsigned int reserved1 : 3; /* bit 18:16 */ + unsigned int pppoe : 1; /* bit 19 PPPoE Session ID enable */ + unsigned int vlan : 1; /* bit 20 VLAN ID enable */ + unsigned int ether_type : 1; /* bit 21 Ethernet type enable */ + unsigned int sa : 1; /* bit 22 MAC SA enable */ + unsigned int da : 1; /* bit 23 MAC DA enable */ + unsigned int priority : 3; /* bit 26:24 priority if multi-rules matched */ + unsigned int port : 1; /* bit 27 PORT ID matching enable */ + unsigned int l7 : 1; /* bit 28 L7 matching enable */ + unsigned int l4 : 1; /* bit 29 L4 matching enable */ + unsigned int l3 : 1; /* bit 30 L3 matching enable */ + unsigned int l2 : 1; /* bit 31 L2 matching enable */ + } bits; +} GMAC_MRxCR0_T; + +#define MR_L2_BIT BIT(31) +#define MR_L3_BIT BIT(30) +#define MR_L4_BIT BIT(29) +#define MR_L7_BIT BIT(28) +#define MR_PORT_BIT BIT(27) +#define MR_PRIORITY_BIT BIT(26) +#define MR_DA_BIT BIT(23) +#define MR_SA_BIT BIT(22) +#define MR_ETHER_TYPE_BIT BIT(21) +#define MR_VLAN_BIT BIT(20) +#define MR_PPPOE_BIT BIT(19) +#define MR_IP_VER_BIT BIT(15) +#define MR_IP_HDR_LEN_BIT BIT(14) +#define MR_FLOW_LABLE_BIT BIT(13) +#define MR_TOS_TRAFFIC_BIT BIT(12) +#define MR_SPR_BIT(x) BIT(x) +#define MR_SPR_BITS 0xff + +/* + * GMAC Matching Rule Control Register 1 + * GMAC0 offset 0x807C + * GMAC1 offset 0xC07C + */ +typedef union { + unsigned int bits32; + struct bit_807C { + unsigned int l4_byte0_15 : 16; /* bit 15: 0 */ + unsigned int dip_netmask : 7; /* bit 22:16 Dest IP net mask, number of mask bits */ + unsigned int dip : 1; /* bit 23 Dest IP */ + unsigned int sip_netmask : 7; /* bit 30:24 Srce IP net mask, number of mask bits */ + unsigned int sip : 1; /* bit 31 Srce IP */ + } bits; +} GMAC_MRxCR1_T; + +/* + * GMAC Matching Rule Control Register 2 + * GMAC0 offset 0x8080 + * GMAC1 offset 0xC080 + */ +typedef union { + unsigned int bits32; + struct bit_8080 { + unsigned int l7_byte0_23 : 24; /* bit 23:0 */ + unsigned int l4_byte16_24 : 8; /* bit 31: 24 */ + } bits; +} GMAC_MRxCR2_T; + +/* + * GMAC Support registers + * GMAC0 offset 0x80A8 + * GMAC1 offset 0xC0A8 + */ +typedef union { + unsigned int bits32; + struct bit_80A8 { + unsigned int protocol : 8; /* bit 7:0 Supported protocol */ + unsigned int swap : 3; /* bit 10:8 Swap */ + unsigned int reserved : 21; /* bit 31:11 */ + } bits; +} GMAC_SPR_T; + +/* + * GMAC_AHB_WEIGHT registers + * GMAC0 offset 0x80C8 + * GMAC1 offset 0xC0C8 + */ +typedef union { + unsigned int bits32; + struct bit_80C8 { + unsigned int hash_weight : 5; /* 4:0 */ + unsigned int rx_weight : 5; /* 9:5 */ + unsigned int tx_weight : 5; /* 14:10 */ + unsigned int pre_req : 5; /* 19:15 Rx Data Pre Request FIFO Threshold */ + unsigned int tqDV_threshold : 5; /* 24:20 DMA TqCtrl to Start tqDV FIFO Threshold */ + unsigned int reserved : 7; /* 31:25 */ + } bits; +} GMAC_AHB_WEIGHT_T; + +/* + * the register structure of GMAC + */ + +/* + * GMAC RX FLTR + * GMAC0 Offset 0xA00C + * GMAC1 Offset 0xE00C + */ +typedef union { + unsigned int bits32; + struct bit1_000c { + unsigned int unicast : 1; /* enable receive of unicast frames that are sent to STA address */ + unsigned int multicast : 1; /* enable receive of multicast frames that pass multicast filter */ + unsigned int broadcast : 1; /* enable receive of broadcast frames */ + unsigned int promiscuous : 1; /* enable receive of all frames */ + unsigned int error : 1; /* enable receive of all error frames */ + unsigned int : 27; + } bits; +} GMAC_RX_FLTR_T; + +/* + * GMAC Configuration 0 + * GMAC0 Offset 0xA018 + * GMAC1 Offset 0xE018 + */ +typedef union { + unsigned int bits32; + struct bit1_0018 { + unsigned int dis_tx : 1; /* 0: disable transmit */ + unsigned int dis_rx : 1; /* 1: disable receive */ + unsigned int loop_back : 1; /* 2: transmit data loopback enable */ + unsigned int flow_ctrl : 1; /* 3: flow control also trigged by Rx queues */ + unsigned int adj_ifg : 4; /* 4-7: adjust IFG from 96+/-56 */ + unsigned int max_len : 3; /* 8-10 maximum receive frame length allowed */ + unsigned int dis_bkoff : 1; /* 11: disable back-off function */ + unsigned int dis_col : 1; /* 12: disable 16 collisions abort function */ + unsigned int sim_test : 1; /* 13: speed up timers in simulation */ + unsigned int rx_fc_en : 1; /* 14: RX flow control enable */ + unsigned int tx_fc_en : 1; /* 15: TX flow control enable */ + unsigned int rgmii_en : 1; /* 16: RGMII in-band status enable */ + unsigned int ipv4_rx_chksum : 1; /* 17: IPv4 RX Checksum enable */ + unsigned int ipv6_rx_chksum : 1; /* 18: IPv6 RX Checksum enable */ + unsigned int rx_tag_remove : 1; /* 19: Remove Rx VLAN tag */ + unsigned int rgmm_edge : 1; /* 20 */ + unsigned int rxc_inv : 1; /* 21 */ + unsigned int ipv6_exthdr_order : 1; /* 22 */ + unsigned int rx_err_detect : 1; /* 23 */ + unsigned int port0_chk_hwq : 1; /* 24 */ + unsigned int port1_chk_hwq : 1; /* 25 */ + unsigned int port0_chk_toeq : 1; /* 26 */ + unsigned int port1_chk_toeq : 1; /* 27 */ + unsigned int port0_chk_classq : 1; /* 28 */ + unsigned int port1_chk_classq : 1; /* 29 */ + unsigned int reserved : 2; /* 31 */ + } bits; +} GMAC_CONFIG0_T; + +#define CONFIG0_TX_RX_DISABLE (BIT(1)|BIT(0)) +#define CONFIG0_RX_CHKSUM (BIT(18)|BIT(17)) +#define CONFIG0_FLOW_RX (BIT(14)) +#define CONFIG0_FLOW_TX (BIT(15)) +#define CONFIG0_FLOW_TX_RX (BIT(14)|BIT(15)) +#define CONFIG0_FLOW_CTL (BIT(14)|BIT(15)) + +#define CONFIG0_MAXLEN_SHIFT 8 +#define CONFIG0_MAXLEN_MASK (7 << CONFIG0_MAXLEN_SHIFT) +#define CONFIG0_MAXLEN_1536 0 +#define CONFIG0_MAXLEN_1518 1 +#define CONFIG0_MAXLEN_1522 2 +#define CONFIG0_MAXLEN_1542 3 +#define CONFIG0_MAXLEN_9k 4 /* 9212 */ +#define CONFIG0_MAXLEN_10k 5 /* 10236 */ +#define CONFIG0_MAXLEN_1518__6 6 +#define CONFIG0_MAXLEN_1518__7 7 + +/* + * GMAC Configuration 1 + * GMAC0 Offset 0xA01C + * GMAC1 Offset 0xE01C + */ +typedef union { + unsigned int bits32; + struct bit1_001c { + unsigned int set_threshold : 8; /* flow control set threshold */ + unsigned int rel_threshold : 8; /* flow control release threshold */ + unsigned int reserved : 16; + } bits; +} GMAC_CONFIG1_T; + +#define GMAC_FLOWCTRL_SET_MAX 32 +#define GMAC_FLOWCTRL_SET_MIN 0 +#define GMAC_FLOWCTRL_RELEASE_MAX 32 +#define GMAC_FLOWCTRL_RELEASE_MIN 0 + +/* + * GMAC Configuration 2 + * GMAC0 Offset 0xA020 + * GMAC1 Offset 0xE020 + */ +typedef union { + unsigned int bits32; + struct bit1_0020 { + unsigned int set_threshold : 16; /* flow control set threshold */ + unsigned int rel_threshold : 16; /* flow control release threshold */ + } bits; +} GMAC_CONFIG2_T; + +/* + * GMAC Configuration 3 + * GMAC0 Offset 0xA024 + * GMAC1 Offset 0xE024 + */ +typedef union { + unsigned int bits32; + struct bit1_0024 { + unsigned int set_threshold : 16; /* flow control set threshold */ + unsigned int rel_threshold : 16; /* flow control release threshold */ + } bits; +} GMAC_CONFIG3_T; + + +/* + * GMAC STATUS + * GMAC0 Offset 0xA02C + * GMAC1 Offset 0xE02C + */ +typedef union { + unsigned int bits32; + struct bit1_002c { + unsigned int link : 1; /* link status */ + unsigned int speed : 2; /* link speed(00->2.5M 01->25M 10->125M) */ + unsigned int duplex : 1; /* duplex mode */ + unsigned int reserved : 1; + unsigned int mii_rmii : 2; /* PHY interface type */ + unsigned int : 25; + } bits; +} GMAC_STATUS_T; + +#define GMAC_SPEED_10 0 +#define GMAC_SPEED_100 1 +#define GMAC_SPEED_1000 2 + +#define GMAC_PHY_MII 0 +#define GMAC_PHY_GMII 1 +#define GMAC_PHY_RGMII_100_10 2 +#define GMAC_PHY_RGMII_1000 3 + +/* + * Queue Header + * (1) TOE Queue Header + * (2) Non-TOE Queue Header + * (3) Interrupt Queue Header + * + * memory Layout + * TOE Queue Header + * 0x60003000 +---------------------------+ 0x0000 + * | TOE Queue 0 Header | + * | 8 * 4 Bytes | + * +---------------------------+ 0x0020 + * | TOE Queue 1 Header | + * | 8 * 4 Bytes | + * +---------------------------+ 0x0040 + * | ...... | + * | | + * +---------------------------+ + * + * Non TOE Queue Header + * 0x60002000 +---------------------------+ 0x0000 + * | Default Queue 0 Header | + * | 2 * 4 Bytes | + * +---------------------------+ 0x0008 + * | Default Queue 1 Header | + * | 2 * 4 Bytes | + * +---------------------------+ 0x0010 + * | Classification Queue 0 | + * | 2 * 4 Bytes | + * +---------------------------+ + * | Classification Queue 1 | + * | 2 * 4 Bytes | + * +---------------------------+ (n * 8 + 0x10) + * | ... | + * | 2 * 4 Bytes | + * +---------------------------+ (13 * 8 + 0x10) + * | Classification Queue 13 | + * | 2 * 4 Bytes | + * +---------------------------+ 0x80 + * | Interrupt Queue 0 | + * | 2 * 4 Bytes | + * +---------------------------+ + * | Interrupt Queue 1 | + * | 2 * 4 Bytes | + * +---------------------------+ + * | Interrupt Queue 2 | + * | 2 * 4 Bytes | + * +---------------------------+ + * | Interrupt Queue 3 | + * | 2 * 4 Bytes | + * +---------------------------+ + * + */ +#define TOE_QUEUE_HDR_ADDR(n) (TOE_TOE_QUE_HDR_BASE + n * 32) +#define TOE_Q_HDR_AREA_END (TOE_QUEUE_HDR_ADDR(TOE_TOE_QUEUE_MAX + 1)) +#define TOE_DEFAULT_Q_HDR_BASE(x) (TOE_NONTOE_QUE_HDR_BASE + 0x08 * (x)) +#define TOE_CLASS_Q_HDR_BASE (TOE_NONTOE_QUE_HDR_BASE + 0x10) +#define TOE_INTR_Q_HDR_BASE (TOE_NONTOE_QUE_HDR_BASE + 0x80) +#define INTERRUPT_QUEUE_HDR_ADDR(n) (TOE_INTR_Q_HDR_BASE + n * 8) +#define NONTOE_Q_HDR_AREA_END (INTERRUPT_QUEUE_HDR_ADDR(TOE_INTR_QUEUE_MAX + 1)) +/* + * TOE Queue Header Word 0 + */ +typedef union { + unsigned int bits32; + unsigned int base_size; +} TOE_QHDR0_T; + +#define TOE_QHDR0_BASE_MASK (~0x0f) + +/* + * TOE Queue Header Word 1 + */ +typedef union { + unsigned int bits32; + struct bit_qhdr1 { + unsigned int rptr : 16; /* bit 15:0 */ + unsigned int wptr : 16; /* bit 31:16 */ + } bits; +} TOE_QHDR1_T; + +/* + * TOE Queue Header Word 2 + */ +typedef union { + unsigned int bits32; + struct bit_qhdr2 { + unsigned int TotalPktSize : 17; /* bit 16: 0 Total packet size */ + unsigned int reserved : 7; /* bit 23:17 */ + unsigned int dack : 1; /* bit 24 1: Duplicated ACK */ + unsigned int abn : 1; /* bit 25 1: Abnormal case Found */ + unsigned int tcp_opt : 1; /* bit 26 1: Have TCP option */ + unsigned int ip_opt : 1; /* bit 27 1: have IPV4 option or IPV6 Extension header */ + unsigned int sat : 1; /* bit 28 1: SeqCnt > SeqThreshold, or AckCnt > AckThreshold */ + unsigned int osq : 1; /* bit 29 1: out of sequence */ + unsigned int ctl : 1; /* bit 30 1: have control flag bits (except ack) */ + unsigned int usd : 1; /* bit 31 0: if no data assembled yet */ + } bits; +} TOE_QHDR2_T; + +/* + * TOE Queue Header Word 3 + */ +typedef union { + unsigned int bits32; + unsigned int seq_num; +} TOE_QHDR3_T; + +/* + * TOE Queue Header Word 4 + */ +typedef union { + unsigned int bits32; + unsigned int ack_num; +} TOE_QHDR4_T; + +/* + * TOE Queue Header Word 5 + */ +typedef union { + unsigned int bits32; + struct bit_qhdr5 { + unsigned int AckCnt : 16; /* bit 15:0 */ + unsigned int SeqCnt : 16; /* bit 31:16 */ + } bits; +} TOE_QHDR5_T; + +/* + * TOE Queue Header Word 6 + */ +typedef union { + unsigned int bits32; + struct bit_qhdr6 { + unsigned int WinSize : 16; /* bit 15:0 */ + unsigned int iq_num : 2; /* bit 17:16 */ + unsigned int MaxPktSize : 14; /* bit 31:18 */ + } bits; +} TOE_QHDR6_T; + +/* + * TOE Queue Header Word 7 + */ +typedef union { + unsigned int bits32; + struct bit_qhdr7 { + unsigned int AckThreshold : 16; /* bit 15:0 */ + unsigned int SeqThreshold : 16; /* bit 31:16 */ + } bits; +} TOE_QHDR7_T; + +/* + * TOE Queue Header + */ +typedef struct { + TOE_QHDR0_T word0; + TOE_QHDR1_T word1; + TOE_QHDR2_T word2; + TOE_QHDR3_T word3; + TOE_QHDR4_T word4; + TOE_QHDR5_T word5; + TOE_QHDR6_T word6; + TOE_QHDR7_T word7; +} TOE_QHDR_T; + +/* + * NONTOE Queue Header Word 0 + */ +typedef union { + unsigned int bits32; + unsigned int base_size; +} NONTOE_QHDR0_T; + +#define NONTOE_QHDR0_BASE_MASK (~0x0f) + +/* + * NONTOE Queue Header Word 1 + */ +typedef union { + unsigned int bits32; + struct bit_nonqhdr1 { + unsigned int rptr : 16; /* bit 15:0 */ + unsigned int wptr : 16; /* bit 31:16 */ + } bits; +} NONTOE_QHDR1_T; + +/* + * Non-TOE Queue Header + */ +typedef struct { + NONTOE_QHDR0_T word0; + NONTOE_QHDR1_T word1; +} NONTOE_QHDR_T; + +/* + * Interrupt Queue Header Word 0 + */ +typedef union { + unsigned int bits32; + struct bit_intrqhdr0 { + unsigned int win_size : 16; /* bit 15:0 Descriptor Ring Size */ + unsigned int wptr : 16; /* bit 31:16 Write Pointer where hw stopped */ + } bits; +} INTR_QHDR0_T; + +/* + * Interrupt Queue Header Word 1 + */ +typedef union { + unsigned int bits32; + struct bit_intrqhdr1 { + unsigned int TotalPktSize : 17; /* bit 16: 0 Total packet size */ + unsigned int tcp_qid : 8; /* bit 24:17 TCP Queue ID */ + unsigned int dack : 1; /* bit 25 1: Duplicated ACK */ + unsigned int abn : 1; /* bit 26 1: Abnormal case Found */ + unsigned int tcp_opt : 1; /* bit 27 1: Have TCP option */ + unsigned int ip_opt : 1; /* bit 28 1: have IPV4 option or IPV6 Extension header */ + unsigned int sat : 1; /* bit 29 1: SeqCnt > SeqThreshold, or AckCnt > AckThreshold */ + unsigned int osq : 1; /* bit 30 1: out of sequence */ + unsigned int ctl : 1; /* bit 31 1: have control flag bits (except ack) */ + } bits; +} INTR_QHDR1_T; + +/* + * Interrupt Queue Header Word 2 + */ +typedef union { + unsigned int bits32; + unsigned int seq_num; +} INTR_QHDR2_T; + +/* + * Interrupt Queue Header Word 3 + */ +typedef union { + unsigned int bits32; + unsigned int ack_num; +} INTR_QHDR3_T; + +/* + * Interrupt Queue Header Word 4 + */ +typedef union { + unsigned int bits32; + struct bit_intrqhdr4 { + unsigned int AckCnt : 16; /* bit 15:0 Ack# change since last ack# intr. */ + unsigned int SeqCnt : 16; /* bit 31:16 Seq# change since last seq# intr. */ + } bits; +} INTR_QHDR4_T; + +/* + * Interrupt Queue Header + */ +typedef struct { + INTR_QHDR0_T word0; + INTR_QHDR1_T word1; + INTR_QHDR2_T word2; + INTR_QHDR3_T word3; + INTR_QHDR4_T word4; + unsigned int word5; + unsigned int word6; + unsigned int word7; +} INTR_QHDR_T; + +#endif /* _GEMINI_ETHERNET_H */ -- 2.14.3
