[PATCH net-next v6 0/2] net: thunderx: add support for PTP clock

[Aleksey Makarov]

This series adds support for IEEE 1588 Precision Time Protocol
to Cavium ethernet driver.

The first patch adds support for the Precision Time Protocol Clocks and
Timestamping coprocessor (PTP) found on Cavium processors.
It registers a new PTP clock in the PTP core and provides functions
to use the counter in BGX, TNS, GTI, and NIC blocks.

The second patch introduces support for the PTP protocol to the
Cavium ThunderX ethernet driver.

v6:
- check if ptp_clock_register() returns NULL (Richard Cochran)
- fix doc comment for cavium_ptp_enable() (Richard Cochran)
- fix a function call formatting; use defined constant (Richard Cochran)
- add comments for `tx_ptp_skbs` and `ptp_skb` (Richard Cochran)
- use adjfine() instead of adjfreq() (Richard Cochran)
- add Acked-by: Philippe Ombredanne 

v5: https://lkml.kernel.org/r/20171211141435.2915-1-aleksey.maka...@cavium.com
- fix the file headers (add SPDX tags, remove advertisment) (Philippe 
Ombredanne)
- use "imply" instead of "select" (Richard Cochran)
- add some code in cavium_ptp_get() for the case when the PTP driver has not 
been
  registered with the PTP core

v4: https://lkml.kernel.org/r/20171208103442.19354-1-aleksey.maka...@cavium.com
- use IS_ENABLED. This fixes compilation of the ptp as a module (David Miller)
- select PTP_1588_CLOCK, not depend on it.  This fixes a build warning.
- change u64 to __be64.  This fixes the sparse warning
  "warning: cast to restricted __be64"
- make nicvf_config_hwtstamp() static. This fixes the sparse warning
  "warning: symbol 'nicvf_config_hwtstamp' was not declared. Should it be 
static?"

v3: https://lkml.kernel.org/r/20171206133100.26436-1-aleksey.maka...@cavium.com
- rebase to net-next

v2: https://lkml.kernel.org/r/20171117134909.8954-1-aleksey.maka...@cavium.com
- use readq()/writeq() in place of cavium_ptp_reg_read()/cavium_ptp_reg_write(),
  don't use readq_relaxed()/writeq_relaxed() (David Daney)

v1: https://lkml.kernel.org/r/20171107190704.15458-1-aleksey.maka...@cavium.com

Radoslaw Biernacki (1):
  net: add support for Cavium PTP coprocessor

Sunil Goutham (1):
  net: thunderx: add timestamping support

 drivers/net/ethernet/cavium/Kconfig|  13 +
 drivers/net/ethernet/cavium/Makefile   |   1 +
 drivers/net/ethernet/cavium/common/Makefile|   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c| 353 +
 drivers/net/ethernet/cavium/common/cavium_ptp.h|  70 
 drivers/net/ethernet/cavium/thunder/nic.h  |  36 +++
 drivers/net/ethernet/cavium/thunder/nic_main.c |  56 +++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 169 +-
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 13 files changed, 782 insertions(+), 6 deletions(-)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

-- 
2.15.1

[PATCH net-next v6 0/2] net: thunderx: add support for PTP clock

[Aleksey Makarov]

This series adds support for IEEE 1588 Precision Time Protocol
to Cavium ethernet driver.

The first patch adds support for the Precision Time Protocol Clocks and
Timestamping coprocessor (PTP) found on Cavium processors.
It registers a new PTP clock in the PTP core and provides functions
to use the counter in BGX, TNS, GTI, and NIC blocks.

The second patch introduces support for the PTP protocol to the
Cavium ThunderX ethernet driver.

v6:
- check if ptp_clock_register() returns NULL (Richard Cochran)
- fix doc comment for cavium_ptp_enable() (Richard Cochran)
- fix a function call formatting; use defined constant (Richard Cochran)
- add comments for `tx_ptp_skbs` and `ptp_skb` (Richard Cochran)
- use adjfine() instead of adjfreq() (Richard Cochran)
- add Acked-by: Philippe Ombredanne 

v5: https://lkml.kernel.org/r/20171211141435.2915-1-aleksey.maka...@cavium.com
- fix the file headers (add SPDX tags, remove advertisment) (Philippe 
Ombredanne)
- use "imply" instead of "select" (Richard Cochran)
- add some code in cavium_ptp_get() for the case when the PTP driver has not 
been
  registered with the PTP core

v4: https://lkml.kernel.org/r/20171208103442.19354-1-aleksey.maka...@cavium.com
- use IS_ENABLED. This fixes compilation of the ptp as a module (David Miller)
- select PTP_1588_CLOCK, not depend on it.  This fixes a build warning.
- change u64 to __be64.  This fixes the sparse warning
  "warning: cast to restricted __be64"
- make nicvf_config_hwtstamp() static. This fixes the sparse warning
  "warning: symbol 'nicvf_config_hwtstamp' was not declared. Should it be 
static?"

v3: https://lkml.kernel.org/r/20171206133100.26436-1-aleksey.maka...@cavium.com
- rebase to net-next

v2: https://lkml.kernel.org/r/20171117134909.8954-1-aleksey.maka...@cavium.com
- use readq()/writeq() in place of cavium_ptp_reg_read()/cavium_ptp_reg_write(),
  don't use readq_relaxed()/writeq_relaxed() (David Daney)

v1: https://lkml.kernel.org/r/20171107190704.15458-1-aleksey.maka...@cavium.com

Radoslaw Biernacki (1):
  net: add support for Cavium PTP coprocessor

Sunil Goutham (1):
  net: thunderx: add timestamping support

 drivers/net/ethernet/cavium/Kconfig|  13 +
 drivers/net/ethernet/cavium/Makefile   |   1 +
 drivers/net/ethernet/cavium/common/Makefile|   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c| 353 +
 drivers/net/ethernet/cavium/common/cavium_ptp.h|  70 
 drivers/net/ethernet/cavium/thunder/nic.h  |  36 +++
 drivers/net/ethernet/cavium/thunder/nic_main.c |  56 +++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 169 +-
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 13 files changed, 782 insertions(+), 6 deletions(-)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

-- 
2.15.1

[PATCH net-next v6 2/2] net: thunderx: add timestamping support

[Aleksey Makarov]

From: Sunil Goutham <sgout...@cavium.com>

This adds timestamping support for both receive and transmit
paths. On the receive side no filters are supported i.e either
all pkts will get a timestamp appended infront of the packet or none.
On the transmit side HW doesn't support timestamp insertion but
only generates a separate CQE with transmitted packet's timestamp.
Also HW supports only one packet at a time for timestamping on the
transmit side.

Signed-off-by: Sunil Goutham <sgout...@cavium.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@cavium.com>
Acked-by: Philippe Ombredanne <pombreda...@nexb.com>
---
 drivers/net/ethernet/cavium/Kconfig|   1 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  36 +
 drivers/net/ethernet/cavium/thunder/nic_main.c |  56 ++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +++-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 169 -
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 9 files changed, 345 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index 96586c0b4490..043e3c11c42b 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -27,6 +27,7 @@ config THUNDER_NIC_PF
 
 config THUNDER_NIC_VF
tristate "Thunder Virtual function driver"
+   imply CAVIUM_PTP
depends on 64BIT
---help---
  This driver supports Thunder's NIC virtual function
diff --git a/drivers/net/ethernet/cavium/thunder/nic.h 
b/drivers/net/ethernet/cavium/thunder/nic.h
index 4a02e618e318..4cacce5d2b16 100644
--- a/drivers/net/ethernet/cavium/thunder/nic.h
+++ b/drivers/net/ethernet/cavium/thunder/nic.h
@@ -263,6 +263,8 @@ struct nicvf_drv_stats {
struct u64_stats_sync   syncp;
 };
 
+struct cavium_ptp;
+
 struct nicvf {
struct nicvf*pnicvf;
struct net_device   *netdev;
@@ -312,6 +314,33 @@ struct nicvf {
struct tasklet_struct   qs_err_task;
struct work_struct  reset_task;
 
+   /* PTP timestamp */
+   struct cavium_ptp   *ptp_clock;
+   /* Inbound timestamping is on */
+   boolhw_rx_tstamp;
+   /* When the packet that requires timestamping is sent, hardware inserts
+* two entries to the completion queue.  First is the regular
+* CQE_TYPE_SEND entry that signals that the packet was sent.
+* The second is CQE_TYPE_SEND_PTP that contains the actual timestamp
+* for that packet.
+* `ptp_skb` is initialized in the handler for the CQE_TYPE_SEND
+* entry and is used and zeroed in the handler for the CQE_TYPE_SEND_PTP
+* entry.
+* So `ptp_skb` is used to hold the pointer to the packet between
+* the calls to CQE_TYPE_SEND and CQE_TYPE_SEND_PTP handlers.
+*/
+   struct sk_buff  *ptp_skb;
+   /* `tx_ptp_skbs` is set when the hardware is sending a packet that
+* requires timestamping.  Cavium hardware can not process more than one
+* such packet at once so this is set each time the driver submits
+* a packet that requires timestamping to the send queue and clears
+* each time it receives the entry on the completion queue saying
+* that such packet was sent.
+* So `tx_ptp_skbs` prevents driver from submitting more than one
+* packet that requires timestamping to the hardware for transmitting.
+*/
+   atomic_ttx_ptp_skbs;
+
/* Interrupt coalescing settings */
u32 cq_coalesce_usecs;
u32 msg_enable;
@@ -371,6 +400,7 @@ struct nicvf {
 #defineNIC_MBOX_MSG_LOOPBACK   0x16/* Set interface in 
loopback */
 #defineNIC_MBOX_MSG_RESET_STAT_COUNTER 0x17/* Reset statistics 
counters */
 #defineNIC_MBOX_MSG_PFC0x18/* Pause frame control 
*/
+#defineNIC_MBOX_MSG_PTP_CFG0x19/* HW packet timestamp 
*/
 #defineNIC_MBOX_MSG_CFG_DONE   0xF0/* VF configuration 
done */
 #defineNIC_MBOX_MSG_SHUTDOWN   0xF1/* VF is being shutdown 
*/
 
@@ -521,6 +551,11 @@ struct pfc {
u8fc_tx;
 };
 
+struct set_ptp {
+   u8msg;
+   bool  enable;
+};
+
 /* 128 bit shared memory between PF and each VF */
 union nic_mbx {
struct { u8 msg; }  msg;
@@ -540,6 +575,7 @@ union nic_mbx {
struct set_loopback lbk;
struct reset_stat_cfg   reset_stat;
struct pfc  pfc;
+   struct set_ptp  ptp;
 };
 
 #define NIC_NODE_ID_MASK   0x03
diff --git a/drivers/net/ethernet/cavium

[PATCH net-next v6 2/2] net: thunderx: add timestamping support

[Aleksey Makarov]

From: Sunil Goutham 

This adds timestamping support for both receive and transmit
paths. On the receive side no filters are supported i.e either
all pkts will get a timestamp appended infront of the packet or none.
On the transmit side HW doesn't support timestamp insertion but
only generates a separate CQE with transmitted packet's timestamp.
Also HW supports only one packet at a time for timestamping on the
transmit side.

Signed-off-by: Sunil Goutham 
Signed-off-by: Aleksey Makarov 
Acked-by: Philippe Ombredanne 
---
 drivers/net/ethernet/cavium/Kconfig|   1 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  36 +
 drivers/net/ethernet/cavium/thunder/nic_main.c |  56 ++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +++-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 169 -
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 9 files changed, 345 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index 96586c0b4490..043e3c11c42b 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -27,6 +27,7 @@ config THUNDER_NIC_PF
 
 config THUNDER_NIC_VF
tristate "Thunder Virtual function driver"
+   imply CAVIUM_PTP
depends on 64BIT
---help---
  This driver supports Thunder's NIC virtual function
diff --git a/drivers/net/ethernet/cavium/thunder/nic.h 
b/drivers/net/ethernet/cavium/thunder/nic.h
index 4a02e618e318..4cacce5d2b16 100644
--- a/drivers/net/ethernet/cavium/thunder/nic.h
+++ b/drivers/net/ethernet/cavium/thunder/nic.h
@@ -263,6 +263,8 @@ struct nicvf_drv_stats {
struct u64_stats_sync   syncp;
 };
 
+struct cavium_ptp;
+
 struct nicvf {
struct nicvf*pnicvf;
struct net_device   *netdev;
@@ -312,6 +314,33 @@ struct nicvf {
struct tasklet_struct   qs_err_task;
struct work_struct  reset_task;
 
+   /* PTP timestamp */
+   struct cavium_ptp   *ptp_clock;
+   /* Inbound timestamping is on */
+   boolhw_rx_tstamp;
+   /* When the packet that requires timestamping is sent, hardware inserts
+* two entries to the completion queue.  First is the regular
+* CQE_TYPE_SEND entry that signals that the packet was sent.
+* The second is CQE_TYPE_SEND_PTP that contains the actual timestamp
+* for that packet.
+* `ptp_skb` is initialized in the handler for the CQE_TYPE_SEND
+* entry and is used and zeroed in the handler for the CQE_TYPE_SEND_PTP
+* entry.
+* So `ptp_skb` is used to hold the pointer to the packet between
+* the calls to CQE_TYPE_SEND and CQE_TYPE_SEND_PTP handlers.
+*/
+   struct sk_buff  *ptp_skb;
+   /* `tx_ptp_skbs` is set when the hardware is sending a packet that
+* requires timestamping.  Cavium hardware can not process more than one
+* such packet at once so this is set each time the driver submits
+* a packet that requires timestamping to the send queue and clears
+* each time it receives the entry on the completion queue saying
+* that such packet was sent.
+* So `tx_ptp_skbs` prevents driver from submitting more than one
+* packet that requires timestamping to the hardware for transmitting.
+*/
+   atomic_ttx_ptp_skbs;
+
/* Interrupt coalescing settings */
u32 cq_coalesce_usecs;
u32 msg_enable;
@@ -371,6 +400,7 @@ struct nicvf {
 #defineNIC_MBOX_MSG_LOOPBACK   0x16/* Set interface in 
loopback */
 #defineNIC_MBOX_MSG_RESET_STAT_COUNTER 0x17/* Reset statistics 
counters */
 #defineNIC_MBOX_MSG_PFC0x18/* Pause frame control 
*/
+#defineNIC_MBOX_MSG_PTP_CFG0x19/* HW packet timestamp 
*/
 #defineNIC_MBOX_MSG_CFG_DONE   0xF0/* VF configuration 
done */
 #defineNIC_MBOX_MSG_SHUTDOWN   0xF1/* VF is being shutdown 
*/
 
@@ -521,6 +551,11 @@ struct pfc {
u8fc_tx;
 };
 
+struct set_ptp {
+   u8msg;
+   bool  enable;
+};
+
 /* 128 bit shared memory between PF and each VF */
 union nic_mbx {
struct { u8 msg; }  msg;
@@ -540,6 +575,7 @@ union nic_mbx {
struct set_loopback lbk;
struct reset_stat_cfg   reset_stat;
struct pfc  pfc;
+   struct set_ptp  ptp;
 };
 
 #define NIC_NODE_ID_MASK   0x03
diff --git a/drivers/net/ethernet/cavium/thunder/nic_main.c 
b/drivers/net/ethernet/cavium/thunder/nic_main.c
index 8f1dd55b3e08..8325577d7442 100644
---

[PATCH net-next v6 1/2] net: add support for Cavium PTP coprocessor

[Aleksey Makarov]

From: Radoslaw Biernacki <r...@semihalf.com>

This patch adds support for the Precision Time Protocol
Clocks and Timestamping hardware found on Cavium ThunderX
processors.

Signed-off-by: Radoslaw Biernacki <r...@semihalf.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@cavium.com>
Acked-by: Philippe Ombredanne <pombreda...@nexb.com>
---
 drivers/net/ethernet/cavium/Kconfig |  12 +
 drivers/net/ethernet/cavium/Makefile|   1 +
 drivers/net/ethernet/cavium/common/Makefile |   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c | 353 
 drivers/net/ethernet/cavium/common/cavium_ptp.h |  70 +
 5 files changed, 437 insertions(+)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index 63be75eb34d2..96586c0b4490 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -50,6 +50,18 @@ config   THUNDER_NIC_RGX
  This driver supports configuring XCV block of RGX interface
  present on CN81XX chip.
 
+config CAVIUM_PTP
+   tristate "Cavium PTP coprocessor as PTP clock"
+   depends on 64BIT
+   imply PTP_1588_CLOCK
+   default y
+   ---help---
+ This driver adds support for the Precision Time Protocol Clocks and
+ Timestamping coprocessor (PTP) found on Cavium processors.
+ PTP provides timestamping mechanism that is suitable for use in IEEE 
1588
+ Precision Time Protocol or other purposes.  Timestamps can be used in
+ BGX, TNS, GTI, and NIC blocks.
+
 config LIQUIDIO
tristate "Cavium LiquidIO support"
depends on 64BIT
diff --git a/drivers/net/ethernet/cavium/Makefile 
b/drivers/net/ethernet/cavium/Makefile
index 872da9f7c31a..946bba84e81d 100644
--- a/drivers/net/ethernet/cavium/Makefile
+++ b/drivers/net/ethernet/cavium/Makefile
@@ -1,6 +1,7 @@
 #
 # Makefile for the Cavium ethernet device drivers.
 #
+obj-$(CONFIG_NET_VENDOR_CAVIUM) += common/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += thunder/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += liquidio/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += octeon/
diff --git a/drivers/net/ethernet/cavium/common/Makefile 
b/drivers/net/ethernet/cavium/common/Makefile
new file mode 100644
index ..dd8561b8060b
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_CAVIUM_PTP) += cavium_ptp.o
diff --git a/drivers/net/ethernet/cavium/common/cavium_ptp.c 
b/drivers/net/ethernet/cavium/common/cavium_ptp.c
new file mode 100644
index ..c87c9c684a33
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/cavium_ptp.c
@@ -0,0 +1,353 @@
+// SPDX-License-Identifier: GPL-2.0
+/* cavium_ptp.c - PTP 1588 clock on Cavium hardware
+ * Copyright (c) 2003-2015, 2017 Cavium, Inc.
+ */
+
+#include 
+#include 
+#include 
+#include 
+
+#include "cavium_ptp.h"
+
+#define DRV_NAME   "Cavium PTP Driver"
+
+#define PCI_DEVICE_ID_CAVIUM_PTP   0xA00C
+#define PCI_DEVICE_ID_CAVIUM_RST   0xA00E
+
+#define PCI_PTP_BAR_NO 0
+#define PCI_RST_BAR_NO 0
+
+#define PTP_CLOCK_CFG  0xF00ULL
+#define  PTP_CLOCK_CFG_PTP_EN  BIT(0)
+#define PTP_CLOCK_LO   0xF08ULL
+#define PTP_CLOCK_HI   0xF10ULL
+#define PTP_CLOCK_COMP 0xF18ULL
+
+#define RST_BOOT   0x1600ULL
+#define CLOCK_BASE_RATE5000ULL
+
+static u64 ptp_cavium_clock_get(void)
+{
+   struct pci_dev *pdev;
+   void __iomem *base;
+   u64 ret = CLOCK_BASE_RATE * 16;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_RST, NULL);
+   if (!pdev)
+   goto error;
+
+   base = pci_ioremap_bar(pdev, PCI_RST_BAR_NO);
+   if (!base)
+   goto error_put_pdev;
+
+   ret = CLOCK_BASE_RATE * ((readq(base + RST_BOOT) >> 33) & 0x3f);
+
+   iounmap(base);
+
+error_put_pdev:
+   pci_dev_put(pdev);
+
+error:
+   return ret;
+}
+
+struct cavium_ptp *cavium_ptp_get(void)
+{
+   struct cavium_ptp *ptp;
+   struct pci_dev *pdev;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_PTP, NULL);
+   if (!pdev)
+   return ERR_PTR(-ENODEV);
+
+   ptp = pci_get_drvdata(pdev);
+   if (!ptp)
+   ptp = ERR_PTR(-EPROBE_DEFER);
+   if (IS_ERR(ptp))
+   pci_dev_put(pdev);
+
+   return ptp;
+}
+EXPORT_SYMBOL(cavium_ptp_get);
+
+void cavium_ptp_put(struct cavium_ptp *ptp)
+{
+   pci_dev_put(ptp->pdev);
+}
+EXPORT_SYMBOL(cavium_ptp_put);
+
+/**
+ * cavium_ptp_adjfine() - Adjust ptp frequency
+ * @ptp: PTP clock info
+ * @scaled_ppm: how much to adjust by, in parts per million

[PATCH net-next v6 1/2] net: add support for Cavium PTP coprocessor

[Aleksey Makarov]

From: Radoslaw Biernacki 

This patch adds support for the Precision Time Protocol
Clocks and Timestamping hardware found on Cavium ThunderX
processors.

Signed-off-by: Radoslaw Biernacki 
Signed-off-by: Aleksey Makarov 
Acked-by: Philippe Ombredanne 
---
 drivers/net/ethernet/cavium/Kconfig |  12 +
 drivers/net/ethernet/cavium/Makefile|   1 +
 drivers/net/ethernet/cavium/common/Makefile |   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c | 353 
 drivers/net/ethernet/cavium/common/cavium_ptp.h |  70 +
 5 files changed, 437 insertions(+)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index 63be75eb34d2..96586c0b4490 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -50,6 +50,18 @@ config   THUNDER_NIC_RGX
  This driver supports configuring XCV block of RGX interface
  present on CN81XX chip.
 
+config CAVIUM_PTP
+   tristate "Cavium PTP coprocessor as PTP clock"
+   depends on 64BIT
+   imply PTP_1588_CLOCK
+   default y
+   ---help---
+ This driver adds support for the Precision Time Protocol Clocks and
+ Timestamping coprocessor (PTP) found on Cavium processors.
+ PTP provides timestamping mechanism that is suitable for use in IEEE 
1588
+ Precision Time Protocol or other purposes.  Timestamps can be used in
+ BGX, TNS, GTI, and NIC blocks.
+
 config LIQUIDIO
tristate "Cavium LiquidIO support"
depends on 64BIT
diff --git a/drivers/net/ethernet/cavium/Makefile 
b/drivers/net/ethernet/cavium/Makefile
index 872da9f7c31a..946bba84e81d 100644
--- a/drivers/net/ethernet/cavium/Makefile
+++ b/drivers/net/ethernet/cavium/Makefile
@@ -1,6 +1,7 @@
 #
 # Makefile for the Cavium ethernet device drivers.
 #
+obj-$(CONFIG_NET_VENDOR_CAVIUM) += common/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += thunder/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += liquidio/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += octeon/
diff --git a/drivers/net/ethernet/cavium/common/Makefile 
b/drivers/net/ethernet/cavium/common/Makefile
new file mode 100644
index ..dd8561b8060b
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_CAVIUM_PTP) += cavium_ptp.o
diff --git a/drivers/net/ethernet/cavium/common/cavium_ptp.c 
b/drivers/net/ethernet/cavium/common/cavium_ptp.c
new file mode 100644
index ..c87c9c684a33
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/cavium_ptp.c
@@ -0,0 +1,353 @@
+// SPDX-License-Identifier: GPL-2.0
+/* cavium_ptp.c - PTP 1588 clock on Cavium hardware
+ * Copyright (c) 2003-2015, 2017 Cavium, Inc.
+ */
+
+#include 
+#include 
+#include 
+#include 
+
+#include "cavium_ptp.h"
+
+#define DRV_NAME   "Cavium PTP Driver"
+
+#define PCI_DEVICE_ID_CAVIUM_PTP   0xA00C
+#define PCI_DEVICE_ID_CAVIUM_RST   0xA00E
+
+#define PCI_PTP_BAR_NO 0
+#define PCI_RST_BAR_NO 0
+
+#define PTP_CLOCK_CFG  0xF00ULL
+#define  PTP_CLOCK_CFG_PTP_EN  BIT(0)
+#define PTP_CLOCK_LO   0xF08ULL
+#define PTP_CLOCK_HI   0xF10ULL
+#define PTP_CLOCK_COMP 0xF18ULL
+
+#define RST_BOOT   0x1600ULL
+#define CLOCK_BASE_RATE5000ULL
+
+static u64 ptp_cavium_clock_get(void)
+{
+   struct pci_dev *pdev;
+   void __iomem *base;
+   u64 ret = CLOCK_BASE_RATE * 16;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_RST, NULL);
+   if (!pdev)
+   goto error;
+
+   base = pci_ioremap_bar(pdev, PCI_RST_BAR_NO);
+   if (!base)
+   goto error_put_pdev;
+
+   ret = CLOCK_BASE_RATE * ((readq(base + RST_BOOT) >> 33) & 0x3f);
+
+   iounmap(base);
+
+error_put_pdev:
+   pci_dev_put(pdev);
+
+error:
+   return ret;
+}
+
+struct cavium_ptp *cavium_ptp_get(void)
+{
+   struct cavium_ptp *ptp;
+   struct pci_dev *pdev;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_PTP, NULL);
+   if (!pdev)
+   return ERR_PTR(-ENODEV);
+
+   ptp = pci_get_drvdata(pdev);
+   if (!ptp)
+   ptp = ERR_PTR(-EPROBE_DEFER);
+   if (IS_ERR(ptp))
+   pci_dev_put(pdev);
+
+   return ptp;
+}
+EXPORT_SYMBOL(cavium_ptp_get);
+
+void cavium_ptp_put(struct cavium_ptp *ptp)
+{
+   pci_dev_put(ptp->pdev);
+}
+EXPORT_SYMBOL(cavium_ptp_put);
+
+/**
+ * cavium_ptp_adjfine() - Adjust ptp frequency
+ * @ptp: PTP clock info
+ * @scaled_ppm: how much to adjust by, in parts per million, but with a
+ *  16 bit binary fractional field
+ */
+static int cavium_ptp_adjfine(struct ptp_clo

Re: [PATCH net-next v5 2/2] net: thunderx: add timestamping support

[Aleksey Makarov]

On 12.12.2017 05:32, Richard Cochran wrote:
> On Mon, Dec 11, 2017 at 05:14:31PM +0300, Aleksey Makarov wrote:
>> diff --git a/drivers/net/ethernet/cavium/thunder/nic.h 
>> b/drivers/net/ethernet/cavium/thunder/nic.h
>> index 4a02e618e318..204b234beb9d 100644
>> --- a/drivers/net/ethernet/cavium/thunder/nic.h
>> +++ b/drivers/net/ethernet/cavium/thunder/nic.h
>> @@ -263,6 +263,8 @@ struct nicvf_drv_stats {
>>  struct u64_stats_sync   syncp;
>>  };
>>  
>> +struct cavium_ptp;
>> +
>>  struct nicvf {
>>  struct nicvf*pnicvf;
>>  struct net_device   *netdev;
>> @@ -312,6 +314,12 @@ struct nicvf {
>>  struct tasklet_struct   qs_err_task;
>>  struct work_struct  reset_task;
>>  
>> +/* PTP timestamp */
>> +struct cavium_ptp   *ptp_clock;
>> +boolhw_rx_tstamp;
>> +struct sk_buff  *ptp_skb;
>> +atomic_ttx_ptp_skbs;
> 
> It is disturbing that the above two fields are set in different
> places.  Shouldn't they be unified into one logical lock?

No, they should not as they are not quite related.

`tx_ptp_skbs` is set when the hardware is sending a packet that requires
timestamping.  Cavium hardware can not process more than one
such packet at once so this is set each time the driver submits
a packet that requires timestamping to the send queue and clears
each time it receives the entry on the completion queue saying
that such packet was sent.

So `tx_ptp_skbs` prevents driver from submitting more than one
packet that requires timestamping to the hardware for transmitting.

When that packet is sent, hardware inserts two entries to
the completion queue.  First is the regular CQE_TYPE_SEND entry
that signals that the packet was sent.  The second is CQE_TYPE_SEND_PTP
that contains the actual timestamp for that packet.

`ptp_skb` is initialized in the handler for the CQE_TYPE_SEND
entry and is used and zeroed in the handler for the CQE_TYPE_SEND_PTP
entry.

So `ptp_skb` is used to hold the pointer to the packet between
the calls to CQE_TYPE_SEND and CQE_TYPE_SEND_PTP handlers.

I am going to add those comments to the sources, fix other issues and
send v6 in short time.

Thank you
Aleksey Makarov

> Here you clear them together:
> 
>> +static void nicvf_snd_ptp_handler(struct net_device *netdev,
>> +  struct cqe_send_t *cqe_tx)
>> +{
>> +struct nicvf *nic = netdev_priv(netdev);
>> +struct skb_shared_hwtstamps ts;
>> +u64 ns;
>> +
>> +nic = nic->pnicvf;
>> +
>> +/* Sync for 'ptp_skb' */
>> +smp_rmb();
>> +
>> +/* New timestamp request can be queued now */
>> +atomic_set(>tx_ptp_skbs, 0);
>> +
>> +/* Check for timestamp requested skb */
>> +if (!nic->ptp_skb)
>> +return;
>> +
>> +/* Check if timestamping is timedout, which is set to 10us */
>> +if (cqe_tx->send_status == CQ_TX_ERROP_TSTMP_TIMEOUT ||
>> +cqe_tx->send_status == CQ_TX_ERROP_TSTMP_CONFLICT)
>> +goto no_tstamp;
>> +
>> +/* Get the timestamp */
>> +memset(, 0, sizeof(ts));
>> +ns = cavium_ptp_tstamp2time(nic->ptp_clock, cqe_tx->ptp_timestamp);
>> +ts.hwtstamp = ns_to_ktime(ns);
>> +skb_tstamp_tx(nic->ptp_skb, );
>> +
>> +no_tstamp:
>> +/* Free the original skb */
>> +dev_kfree_skb_any(nic->ptp_skb);
>> +nic->ptp_skb = NULL;
>> +/* Sync 'ptp_skb' */
>> +smp_wmb();
>> +}
>> +
> 
> but here you set the one:
> 
>> @@ -657,7 +697,12 @@ static void nicvf_snd_pkt_handler(struct net_device 
>> *netdev,
>>  prefetch(skb);
>>  (*tx_pkts)++;
>>  *tx_bytes += skb->len;
>> -napi_consume_skb(skb, budget);
>> +/* If timestamp is requested for this skb, don't free it */
>> +if (skb_shinfo(skb)->tx_flags & SKBTX_IN_PROGRESS &&
>> +!nic->pnicvf->ptp_skb)
>> +nic->pnicvf->ptp_skb = skb;
>> +else
>> +napi_consume_skb(skb, budget);
>>  sq->skbuff[cqe_tx->sqe_ptr] = (u64)NULL;
>>  } else {
>>  /* In case of SW TSO on 88xx, only last segment will have
> 
> here you clear one:
> 
>> @@ -1319,12 +1382,28 @@ int nicvf_stop(struct net_device *netdev)
>>  
>>  nicvf_free_cq_poll(nic);
>>  
>> +/* Free any pending SKB saved to receive timestamp */
>> +if (nic->ptp_skb) 

Re: [PATCH net-next v5 2/2] net: thunderx: add timestamping support

[Aleksey Makarov]

On 12.12.2017 05:32, Richard Cochran wrote:
> On Mon, Dec 11, 2017 at 05:14:31PM +0300, Aleksey Makarov wrote:
>> diff --git a/drivers/net/ethernet/cavium/thunder/nic.h 
>> b/drivers/net/ethernet/cavium/thunder/nic.h
>> index 4a02e618e318..204b234beb9d 100644
>> --- a/drivers/net/ethernet/cavium/thunder/nic.h
>> +++ b/drivers/net/ethernet/cavium/thunder/nic.h
>> @@ -263,6 +263,8 @@ struct nicvf_drv_stats {
>>  struct u64_stats_sync   syncp;
>>  };
>>  
>> +struct cavium_ptp;
>> +
>>  struct nicvf {
>>  struct nicvf*pnicvf;
>>  struct net_device   *netdev;
>> @@ -312,6 +314,12 @@ struct nicvf {
>>  struct tasklet_struct   qs_err_task;
>>  struct work_struct  reset_task;
>>  
>> +/* PTP timestamp */
>> +struct cavium_ptp   *ptp_clock;
>> +boolhw_rx_tstamp;
>> +struct sk_buff  *ptp_skb;
>> +atomic_ttx_ptp_skbs;
> 
> It is disturbing that the above two fields are set in different
> places.  Shouldn't they be unified into one logical lock?

No, they should not as they are not quite related.

`tx_ptp_skbs` is set when the hardware is sending a packet that requires
timestamping.  Cavium hardware can not process more than one
such packet at once so this is set each time the driver submits
a packet that requires timestamping to the send queue and clears
each time it receives the entry on the completion queue saying
that such packet was sent.

So `tx_ptp_skbs` prevents driver from submitting more than one
packet that requires timestamping to the hardware for transmitting.

When that packet is sent, hardware inserts two entries to
the completion queue.  First is the regular CQE_TYPE_SEND entry
that signals that the packet was sent.  The second is CQE_TYPE_SEND_PTP
that contains the actual timestamp for that packet.

`ptp_skb` is initialized in the handler for the CQE_TYPE_SEND
entry and is used and zeroed in the handler for the CQE_TYPE_SEND_PTP
entry.

So `ptp_skb` is used to hold the pointer to the packet between
the calls to CQE_TYPE_SEND and CQE_TYPE_SEND_PTP handlers.

I am going to add those comments to the sources, fix other issues and
send v6 in short time.

Thank you
Aleksey Makarov

> Here you clear them together:
> 
>> +static void nicvf_snd_ptp_handler(struct net_device *netdev,
>> +  struct cqe_send_t *cqe_tx)
>> +{
>> +struct nicvf *nic = netdev_priv(netdev);
>> +struct skb_shared_hwtstamps ts;
>> +u64 ns;
>> +
>> +nic = nic->pnicvf;
>> +
>> +/* Sync for 'ptp_skb' */
>> +smp_rmb();
>> +
>> +/* New timestamp request can be queued now */
>> +atomic_set(>tx_ptp_skbs, 0);
>> +
>> +/* Check for timestamp requested skb */
>> +if (!nic->ptp_skb)
>> +return;
>> +
>> +/* Check if timestamping is timedout, which is set to 10us */
>> +if (cqe_tx->send_status == CQ_TX_ERROP_TSTMP_TIMEOUT ||
>> +cqe_tx->send_status == CQ_TX_ERROP_TSTMP_CONFLICT)
>> +goto no_tstamp;
>> +
>> +/* Get the timestamp */
>> +memset(, 0, sizeof(ts));
>> +ns = cavium_ptp_tstamp2time(nic->ptp_clock, cqe_tx->ptp_timestamp);
>> +ts.hwtstamp = ns_to_ktime(ns);
>> +skb_tstamp_tx(nic->ptp_skb, );
>> +
>> +no_tstamp:
>> +/* Free the original skb */
>> +dev_kfree_skb_any(nic->ptp_skb);
>> +nic->ptp_skb = NULL;
>> +/* Sync 'ptp_skb' */
>> +smp_wmb();
>> +}
>> +
> 
> but here you set the one:
> 
>> @@ -657,7 +697,12 @@ static void nicvf_snd_pkt_handler(struct net_device 
>> *netdev,
>>  prefetch(skb);
>>  (*tx_pkts)++;
>>  *tx_bytes += skb->len;
>> -napi_consume_skb(skb, budget);
>> +/* If timestamp is requested for this skb, don't free it */
>> +if (skb_shinfo(skb)->tx_flags & SKBTX_IN_PROGRESS &&
>> +!nic->pnicvf->ptp_skb)
>> +nic->pnicvf->ptp_skb = skb;
>> +else
>> +napi_consume_skb(skb, budget);
>>  sq->skbuff[cqe_tx->sqe_ptr] = (u64)NULL;
>>  } else {
>>  /* In case of SW TSO on 88xx, only last segment will have
> 
> here you clear one:
> 
>> @@ -1319,12 +1382,28 @@ int nicvf_stop(struct net_device *netdev)
>>  
>>  nicvf_free_cq_poll(nic);
>>  
>> +/* Free any pending SKB saved to receive timestamp */
>> +if (nic->ptp_skb) 

Re: [PATCH net-next v5 1/2] net: add support for Cavium PTP coprocessor

[Aleksey Makarov]

Hi Richard,

On 12/12/2017 01:59 AM, Richard Cochran wrote:


Sorry I didn't finish reviewing before...

On Mon, Dec 11, 2017 at 05:14:30PM +0300, Aleksey Makarov wrote:


[ ... ]


+static int cavium_ptp_probe(struct pci_dev *pdev,
+   const struct pci_device_id *ent)
+{
+   struct device *dev = >dev;
+   struct cavium_ptp *clock;
+   struct cyclecounter *cc;
+   u64 clock_cfg;
+   u64 clock_comp;
+   int err;
+
+   clock = devm_kzalloc(dev, sizeof(*clock), GFP_KERNEL);
+   if (!clock)
+   return -ENOMEM;
+
+   clock->pdev = pdev;
+
+   err = pcim_enable_device(pdev);
+   if (err)
+   return err;
+
+   err = pcim_iomap_regions(pdev, 1 << PCI_PTP_BAR_NO, pci_name(pdev));
+   if (err)
+   return err;
+
+   clock->reg_base = pcim_iomap_table(pdev)[PCI_PTP_BAR_NO];
+
+   spin_lock_init(>spin_lock);
+
+   cc = >cycle_counter;
+   cc->read = cavium_ptp_cc_read;
+   cc->mask = CYCLECOUNTER_MASK(64);
+   cc->mult = 1;
+   cc->shift = 0;
+
+   timecounter_init(>time_counter, >cycle_counter,
+ktime_to_ns(ktime_get_real()));
+
+   clock->clock_rate = ptp_cavium_clock_get();
+
+   clock->ptp_info = (struct ptp_clock_info) {
+   .owner  = THIS_MODULE,
+   .name   = "ThunderX PTP",
+   .max_adj= 10ull,
+   .n_ext_ts   = 0,
+   .n_pins = 0,
+   .pps= 0,
+   .adjfreq= cavium_ptp_adjfreq,
+   .adjtime= cavium_ptp_adjtime,
+   .gettime64  = cavium_ptp_gettime,
+   .settime64  = cavium_ptp_settime,
+   .enable = cavium_ptp_enable,
+   };
+
+   clock_cfg = readq(clock->reg_base + PTP_CLOCK_CFG);
+   clock_cfg |= PTP_CLOCK_CFG_PTP_EN;
+   writeq(clock_cfg, clock->reg_base + PTP_CLOCK_CFG);
+
+   clock_comp = ((u64)10ull << 32) / clock->clock_rate;
+   writeq(clock_comp, clock->reg_base + PTP_CLOCK_COMP);
+
+   clock->ptp_clock = ptp_clock_register(>ptp_info, dev);
+   if (IS_ERR(clock->ptp_clock)) {


You need to handle the case when ptp_clock_register() returns NULL.

from ptp_clock_kernel.h:

/**
  * ptp_clock_register() - register a PTP hardware clock driver
  *
  * @info:   Structure describing the new clock.
  * @parent: Pointer to the parent device of the new clock.
  *
  * Returns a valid pointer on success or PTR_ERR on failure.  If PHC
  * support is missing at the configuration level, this function
  * returns NULL, and drivers are expected to gracefully handle that
  * case separately.
  */


If ptp_clock_register() returns NULL, the device is still paired with the 
driver,
but the driver is not registered in the PTP core.  When ethernet driver needs
the reference to this cavium PTP driver, it calls cavium_ptp_get() that checks
if ptp->ptp_clock is NULL and, if so, returns -ENODEV.

I need this behavior because I need to differentiate between two cases:

- the state when the driver is not initialized for the device because of PTP 
core
  has not registered it.  In this case function cavium_ptp_get() returns -ENODEV
  and ethernet driver proceeds without PTP device.

- the state when the driver is not initialized because kernel has not tired
  to initialize it yet.  In this case function cavium_ptp_get() returns 
-EPROBE_DEFER
  that is used in ethernet driver to defer initialization.

If you know how to do the same in more smoothly please share it.  Or else I 
would
prefer to insert a comment about it and leave it as is.

Richard, thank you for review.  I am going to address your comments in my next 
series.

Thank you
Aleksey Makarov
 

+   clock_cfg = readq(clock->reg_base + PTP_CLOCK_CFG);
+   clock_cfg &= ~PTP_CLOCK_CFG_PTP_EN;
+   writeq(clock_cfg, clock->reg_base + PTP_CLOCK_CFG);
+   return PTR_ERR(clock->ptp_clock);
+   }
+
+   pci_set_drvdata(pdev, clock);
+   return 0;
+}


Thanks,
Richard

Re: [PATCH net-next v5 1/2] net: add support for Cavium PTP coprocessor

[Aleksey Makarov]

Hi Richard,

On 12/12/2017 01:59 AM, Richard Cochran wrote:


Sorry I didn't finish reviewing before...

On Mon, Dec 11, 2017 at 05:14:30PM +0300, Aleksey Makarov wrote:


[ ... ]


+static int cavium_ptp_probe(struct pci_dev *pdev,
+   const struct pci_device_id *ent)
+{
+   struct device *dev = >dev;
+   struct cavium_ptp *clock;
+   struct cyclecounter *cc;
+   u64 clock_cfg;
+   u64 clock_comp;
+   int err;
+
+   clock = devm_kzalloc(dev, sizeof(*clock), GFP_KERNEL);
+   if (!clock)
+   return -ENOMEM;
+
+   clock->pdev = pdev;
+
+   err = pcim_enable_device(pdev);
+   if (err)
+   return err;
+
+   err = pcim_iomap_regions(pdev, 1 << PCI_PTP_BAR_NO, pci_name(pdev));
+   if (err)
+   return err;
+
+   clock->reg_base = pcim_iomap_table(pdev)[PCI_PTP_BAR_NO];
+
+   spin_lock_init(>spin_lock);
+
+   cc = >cycle_counter;
+   cc->read = cavium_ptp_cc_read;
+   cc->mask = CYCLECOUNTER_MASK(64);
+   cc->mult = 1;
+   cc->shift = 0;
+
+   timecounter_init(>time_counter, >cycle_counter,
+ktime_to_ns(ktime_get_real()));
+
+   clock->clock_rate = ptp_cavium_clock_get();
+
+   clock->ptp_info = (struct ptp_clock_info) {
+   .owner  = THIS_MODULE,
+   .name   = "ThunderX PTP",
+   .max_adj= 10ull,
+   .n_ext_ts   = 0,
+   .n_pins = 0,
+   .pps= 0,
+   .adjfreq= cavium_ptp_adjfreq,
+   .adjtime= cavium_ptp_adjtime,
+   .gettime64  = cavium_ptp_gettime,
+   .settime64  = cavium_ptp_settime,
+   .enable = cavium_ptp_enable,
+   };
+
+   clock_cfg = readq(clock->reg_base + PTP_CLOCK_CFG);
+   clock_cfg |= PTP_CLOCK_CFG_PTP_EN;
+   writeq(clock_cfg, clock->reg_base + PTP_CLOCK_CFG);
+
+   clock_comp = ((u64)10ull << 32) / clock->clock_rate;
+   writeq(clock_comp, clock->reg_base + PTP_CLOCK_COMP);
+
+   clock->ptp_clock = ptp_clock_register(>ptp_info, dev);
+   if (IS_ERR(clock->ptp_clock)) {


You need to handle the case when ptp_clock_register() returns NULL.

from ptp_clock_kernel.h:

/**
  * ptp_clock_register() - register a PTP hardware clock driver
  *
  * @info:   Structure describing the new clock.
  * @parent: Pointer to the parent device of the new clock.
  *
  * Returns a valid pointer on success or PTR_ERR on failure.  If PHC
  * support is missing at the configuration level, this function
  * returns NULL, and drivers are expected to gracefully handle that
  * case separately.
  */


If ptp_clock_register() returns NULL, the device is still paired with the 
driver,
but the driver is not registered in the PTP core.  When ethernet driver needs
the reference to this cavium PTP driver, it calls cavium_ptp_get() that checks
if ptp->ptp_clock is NULL and, if so, returns -ENODEV.

I need this behavior because I need to differentiate between two cases:

- the state when the driver is not initialized for the device because of PTP 
core
  has not registered it.  In this case function cavium_ptp_get() returns -ENODEV
  and ethernet driver proceeds without PTP device.

- the state when the driver is not initialized because kernel has not tired
  to initialize it yet.  In this case function cavium_ptp_get() returns 
-EPROBE_DEFER
  that is used in ethernet driver to defer initialization.

If you know how to do the same in more smoothly please share it.  Or else I 
would
prefer to insert a comment about it and leave it as is.

Richard, thank you for review.  I am going to address your comments in my next 
series.

Thank you
Aleksey Makarov
 

+   clock_cfg = readq(clock->reg_base + PTP_CLOCK_CFG);
+   clock_cfg &= ~PTP_CLOCK_CFG_PTP_EN;
+   writeq(clock_cfg, clock->reg_base + PTP_CLOCK_CFG);
+   return PTR_ERR(clock->ptp_clock);
+   }
+
+   pci_set_drvdata(pdev, clock);
+   return 0;
+}


Thanks,
Richard

[PATCH net-next v5 2/2] net: thunderx: add timestamping support

[Aleksey Makarov]

From: Sunil Goutham <sgout...@cavium.com>

This adds timestamping support for both receive and transmit
paths. On the receive side no filters are supported i.e either
all pkts will get a timestamp appended infront of the packet or none.
On the transmit side HW doesn't support timestamp insertion but
only generates a separate CQE with transmitted packet's timestamp.
Also HW supports only one packet at a time for timestamping on the
transmit side.

Signed-off-by: Sunil Goutham <sgout...@cavium.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@cavium.com>
---
 drivers/net/ethernet/cavium/Kconfig|   1 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  15 ++
 drivers/net/ethernet/cavium/thunder/nic_main.c |  58 ++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +++-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 173 -
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 9 files changed, 330 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index 96586c0b4490..043e3c11c42b 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -27,6 +27,7 @@ config THUNDER_NIC_PF
 
 config THUNDER_NIC_VF
tristate "Thunder Virtual function driver"
+   imply CAVIUM_PTP
depends on 64BIT
---help---
  This driver supports Thunder's NIC virtual function
diff --git a/drivers/net/ethernet/cavium/thunder/nic.h 
b/drivers/net/ethernet/cavium/thunder/nic.h
index 4a02e618e318..204b234beb9d 100644
--- a/drivers/net/ethernet/cavium/thunder/nic.h
+++ b/drivers/net/ethernet/cavium/thunder/nic.h
@@ -263,6 +263,8 @@ struct nicvf_drv_stats {
struct u64_stats_sync   syncp;
 };
 
+struct cavium_ptp;
+
 struct nicvf {
struct nicvf*pnicvf;
struct net_device   *netdev;
@@ -312,6 +314,12 @@ struct nicvf {
struct tasklet_struct   qs_err_task;
struct work_struct  reset_task;
 
+   /* PTP timestamp */
+   struct cavium_ptp   *ptp_clock;
+   boolhw_rx_tstamp;
+   struct sk_buff  *ptp_skb;
+   atomic_ttx_ptp_skbs;
+
/* Interrupt coalescing settings */
u32 cq_coalesce_usecs;
u32 msg_enable;
@@ -371,6 +379,7 @@ struct nicvf {
 #defineNIC_MBOX_MSG_LOOPBACK   0x16/* Set interface in 
loopback */
 #defineNIC_MBOX_MSG_RESET_STAT_COUNTER 0x17/* Reset statistics 
counters */
 #defineNIC_MBOX_MSG_PFC0x18/* Pause frame control 
*/
+#defineNIC_MBOX_MSG_PTP_CFG0x19/* HW packet timestamp 
*/
 #defineNIC_MBOX_MSG_CFG_DONE   0xF0/* VF configuration 
done */
 #defineNIC_MBOX_MSG_SHUTDOWN   0xF1/* VF is being shutdown 
*/
 
@@ -521,6 +530,11 @@ struct pfc {
u8fc_tx;
 };
 
+struct set_ptp {
+   u8msg;
+   bool  enable;
+};
+
 /* 128 bit shared memory between PF and each VF */
 union nic_mbx {
struct { u8 msg; }  msg;
@@ -540,6 +554,7 @@ union nic_mbx {
struct set_loopback lbk;
struct reset_stat_cfg   reset_stat;
struct pfc  pfc;
+   struct set_ptp  ptp;
 };
 
 #define NIC_NODE_ID_MASK   0x03
diff --git a/drivers/net/ethernet/cavium/thunder/nic_main.c 
b/drivers/net/ethernet/cavium/thunder/nic_main.c
index 8f1dd55b3e08..4c1c5414a162 100644
--- a/drivers/net/ethernet/cavium/thunder/nic_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nic_main.c
@@ -426,13 +426,22 @@ static void nic_init_hw(struct nicpf *nic)
/* Enable backpressure */
nic_reg_write(nic, NIC_PF_BP_CFG, (1ULL << 6) | 0x03);
 
-   /* TNS and TNS bypass modes are present only on 88xx */
+   /* TNS and TNS bypass modes are present only on 88xx
+* Also offset of this CSR has changed in 81xx and 83xx.
+*/
if (nic->pdev->subsystem_device == PCI_SUBSYS_DEVID_88XX_NIC_PF) {
/* Disable TNS mode on both interfaces */
nic_reg_write(nic, NIC_PF_INTF_0_1_SEND_CFG,
- (NIC_TNS_BYPASS_MODE << 7) | BGX0_BLOCK);
+ (NIC_TNS_BYPASS_MODE << 7) |
+ BGX0_BLOCK | (1ULL << 16));
nic_reg_write(nic, NIC_PF_INTF_0_1_SEND_CFG | (1 << 8),
- (NIC_TNS_BYPASS_MODE << 7) | BGX1_BLOCK);
+ (NIC_TNS_BYPASS_MODE << 7) |
+ BGX1_BLOCK | (1ULL << 16));
+   } else {
+   /* Configure timestamp

[PATCH net-next v5 2/2] net: thunderx: add timestamping support

[Aleksey Makarov]

From: Sunil Goutham 

This adds timestamping support for both receive and transmit
paths. On the receive side no filters are supported i.e either
all pkts will get a timestamp appended infront of the packet or none.
On the transmit side HW doesn't support timestamp insertion but
only generates a separate CQE with transmitted packet's timestamp.
Also HW supports only one packet at a time for timestamping on the
transmit side.

Signed-off-by: Sunil Goutham 
Signed-off-by: Aleksey Makarov 
---
 drivers/net/ethernet/cavium/Kconfig|   1 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  15 ++
 drivers/net/ethernet/cavium/thunder/nic_main.c |  58 ++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +++-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 173 -
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 9 files changed, 330 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index 96586c0b4490..043e3c11c42b 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -27,6 +27,7 @@ config THUNDER_NIC_PF
 
 config THUNDER_NIC_VF
tristate "Thunder Virtual function driver"
+   imply CAVIUM_PTP
depends on 64BIT
---help---
  This driver supports Thunder's NIC virtual function
diff --git a/drivers/net/ethernet/cavium/thunder/nic.h 
b/drivers/net/ethernet/cavium/thunder/nic.h
index 4a02e618e318..204b234beb9d 100644
--- a/drivers/net/ethernet/cavium/thunder/nic.h
+++ b/drivers/net/ethernet/cavium/thunder/nic.h
@@ -263,6 +263,8 @@ struct nicvf_drv_stats {
struct u64_stats_sync   syncp;
 };
 
+struct cavium_ptp;
+
 struct nicvf {
struct nicvf*pnicvf;
struct net_device   *netdev;
@@ -312,6 +314,12 @@ struct nicvf {
struct tasklet_struct   qs_err_task;
struct work_struct  reset_task;
 
+   /* PTP timestamp */
+   struct cavium_ptp   *ptp_clock;
+   boolhw_rx_tstamp;
+   struct sk_buff  *ptp_skb;
+   atomic_ttx_ptp_skbs;
+
/* Interrupt coalescing settings */
u32 cq_coalesce_usecs;
u32 msg_enable;
@@ -371,6 +379,7 @@ struct nicvf {
 #defineNIC_MBOX_MSG_LOOPBACK   0x16/* Set interface in 
loopback */
 #defineNIC_MBOX_MSG_RESET_STAT_COUNTER 0x17/* Reset statistics 
counters */
 #defineNIC_MBOX_MSG_PFC0x18/* Pause frame control 
*/
+#defineNIC_MBOX_MSG_PTP_CFG0x19/* HW packet timestamp 
*/
 #defineNIC_MBOX_MSG_CFG_DONE   0xF0/* VF configuration 
done */
 #defineNIC_MBOX_MSG_SHUTDOWN   0xF1/* VF is being shutdown 
*/
 
@@ -521,6 +530,11 @@ struct pfc {
u8fc_tx;
 };
 
+struct set_ptp {
+   u8msg;
+   bool  enable;
+};
+
 /* 128 bit shared memory between PF and each VF */
 union nic_mbx {
struct { u8 msg; }  msg;
@@ -540,6 +554,7 @@ union nic_mbx {
struct set_loopback lbk;
struct reset_stat_cfg   reset_stat;
struct pfc  pfc;
+   struct set_ptp  ptp;
 };
 
 #define NIC_NODE_ID_MASK   0x03
diff --git a/drivers/net/ethernet/cavium/thunder/nic_main.c 
b/drivers/net/ethernet/cavium/thunder/nic_main.c
index 8f1dd55b3e08..4c1c5414a162 100644
--- a/drivers/net/ethernet/cavium/thunder/nic_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nic_main.c
@@ -426,13 +426,22 @@ static void nic_init_hw(struct nicpf *nic)
/* Enable backpressure */
nic_reg_write(nic, NIC_PF_BP_CFG, (1ULL << 6) | 0x03);
 
-   /* TNS and TNS bypass modes are present only on 88xx */
+   /* TNS and TNS bypass modes are present only on 88xx
+* Also offset of this CSR has changed in 81xx and 83xx.
+*/
if (nic->pdev->subsystem_device == PCI_SUBSYS_DEVID_88XX_NIC_PF) {
/* Disable TNS mode on both interfaces */
nic_reg_write(nic, NIC_PF_INTF_0_1_SEND_CFG,
- (NIC_TNS_BYPASS_MODE << 7) | BGX0_BLOCK);
+ (NIC_TNS_BYPASS_MODE << 7) |
+ BGX0_BLOCK | (1ULL << 16));
nic_reg_write(nic, NIC_PF_INTF_0_1_SEND_CFG | (1 << 8),
- (NIC_TNS_BYPASS_MODE << 7) | BGX1_BLOCK);
+ (NIC_TNS_BYPASS_MODE << 7) |
+ BGX1_BLOCK | (1ULL << 16));
+   } else {
+   /* Configure timestamp generation timeout to 10us */
+   for (i = 0; i < nic->hw->bgx_cnt; i

[PATCH net-next v5 1/2] net: add support for Cavium PTP coprocessor

[Aleksey Makarov]

From: Radoslaw Biernacki <r...@semihalf.com>

This patch adds support for the Precision Time Protocol
Clocks and Timestamping hardware found on Cavium ThunderX
processors.

Signed-off-by: Radoslaw Biernacki <r...@semihalf.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@cavium.com>
---
 drivers/net/ethernet/cavium/Kconfig |  12 +
 drivers/net/ethernet/cavium/Makefile|   1 +
 drivers/net/ethernet/cavium/common/Makefile |   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c | 329 
 drivers/net/ethernet/cavium/common/cavium_ptp.h |  70 +
 5 files changed, 413 insertions(+)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index 63be75eb34d2..96586c0b4490 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -50,6 +50,18 @@ config   THUNDER_NIC_RGX
  This driver supports configuring XCV block of RGX interface
  present on CN81XX chip.
 
+config CAVIUM_PTP
+   tristate "Cavium PTP coprocessor as PTP clock"
+   depends on 64BIT
+   imply PTP_1588_CLOCK
+   default y
+   ---help---
+ This driver adds support for the Precision Time Protocol Clocks and
+ Timestamping coprocessor (PTP) found on Cavium processors.
+ PTP provides timestamping mechanism that is suitable for use in IEEE 
1588
+ Precision Time Protocol or other purposes.  Timestamps can be used in
+ BGX, TNS, GTI, and NIC blocks.
+
 config LIQUIDIO
tristate "Cavium LiquidIO support"
depends on 64BIT
diff --git a/drivers/net/ethernet/cavium/Makefile 
b/drivers/net/ethernet/cavium/Makefile
index 872da9f7c31a..946bba84e81d 100644
--- a/drivers/net/ethernet/cavium/Makefile
+++ b/drivers/net/ethernet/cavium/Makefile
@@ -1,6 +1,7 @@
 #
 # Makefile for the Cavium ethernet device drivers.
 #
+obj-$(CONFIG_NET_VENDOR_CAVIUM) += common/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += thunder/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += liquidio/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += octeon/
diff --git a/drivers/net/ethernet/cavium/common/Makefile 
b/drivers/net/ethernet/cavium/common/Makefile
new file mode 100644
index ..dd8561b8060b
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_CAVIUM_PTP) += cavium_ptp.o
diff --git a/drivers/net/ethernet/cavium/common/cavium_ptp.c 
b/drivers/net/ethernet/cavium/common/cavium_ptp.c
new file mode 100644
index ..8b48863bfdec
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/cavium_ptp.c
@@ -0,0 +1,329 @@
+// SPDX-License-Identifier: GPL-2.0
+/* cavium_ptp.c - PTP 1588 clock on Cavium hardware
+ * Copyright (c) 2003-2015, 2017 Cavium, Inc.
+ */
+
+#include 
+#include 
+#include 
+#include 
+
+#include "cavium_ptp.h"
+
+#define DRV_NAME   "Cavium PTP Driver"
+
+#define PCI_DEVICE_ID_CAVIUM_PTP   0xA00C
+#define PCI_DEVICE_ID_CAVIUM_RST   0xA00E
+
+#define PCI_PTP_BAR_NO 0
+#define PCI_RST_BAR_NO 0
+
+#define PTP_CLOCK_CFG  0xF00ULL
+#define  PTP_CLOCK_CFG_PTP_EN  BIT(0)
+#define PTP_CLOCK_LO   0xF08ULL
+#define PTP_CLOCK_HI   0xF10ULL
+#define PTP_CLOCK_COMP 0xF18ULL
+
+#define RST_BOOT   0x1600ULL
+#define CLOCK_BASE_RATE5000ULL
+
+static u64 ptp_cavium_clock_get(void)
+{
+   struct pci_dev *pdev;
+   void __iomem *base;
+   u64 ret = CLOCK_BASE_RATE * 16;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_RST, NULL);
+   if (!pdev)
+   goto error;
+
+   base = pci_ioremap_bar(pdev, PCI_RST_BAR_NO);
+   if (!base)
+   goto error_put_pdev;
+
+   ret = CLOCK_BASE_RATE * ((readq(base + RST_BOOT) >> 33) & 0x3f);
+
+   iounmap(base);
+
+error_put_pdev:
+   pci_dev_put(pdev);
+
+error:
+   return ret;
+}
+
+struct cavium_ptp *cavium_ptp_get(void)
+{
+   struct cavium_ptp *ptp;
+   struct pci_dev *pdev;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_PTP, NULL);
+   if (!pdev)
+   return ERR_PTR(-ENODEV);
+
+   ptp = pci_get_drvdata(pdev);
+   if (!ptp) {
+   pci_dev_put(pdev);
+   ptp = ERR_PTR(-EPROBE_DEFER);
+   } else if (!ptp->ptp_clock) {
+   pci_dev_put(pdev);
+   ptp = ERR_PTR(-ENODEV);
+   }
+
+   return ptp;
+}
+EXPORT_SYMBOL(cavium_ptp_get);
+
+void cavium_ptp_put(struct cavium_ptp *ptp)
+{
+   pci_dev_put(ptp->pdev);
+}
+EXPORT_SYMBOL(cavium_ptp_put);
+
+/**
+ * cavium_ptp_adjfreq() - Adjust ptp frequency
+ * @ptp: PTP clock info
+ * @ppb: how m

[PATCH net-next v5 1/2] net: add support for Cavium PTP coprocessor

[Aleksey Makarov]

From: Radoslaw Biernacki 

This patch adds support for the Precision Time Protocol
Clocks and Timestamping hardware found on Cavium ThunderX
processors.

Signed-off-by: Radoslaw Biernacki 
Signed-off-by: Aleksey Makarov 
---
 drivers/net/ethernet/cavium/Kconfig |  12 +
 drivers/net/ethernet/cavium/Makefile|   1 +
 drivers/net/ethernet/cavium/common/Makefile |   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c | 329 
 drivers/net/ethernet/cavium/common/cavium_ptp.h |  70 +
 5 files changed, 413 insertions(+)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index 63be75eb34d2..96586c0b4490 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -50,6 +50,18 @@ config   THUNDER_NIC_RGX
  This driver supports configuring XCV block of RGX interface
  present on CN81XX chip.
 
+config CAVIUM_PTP
+   tristate "Cavium PTP coprocessor as PTP clock"
+   depends on 64BIT
+   imply PTP_1588_CLOCK
+   default y
+   ---help---
+ This driver adds support for the Precision Time Protocol Clocks and
+ Timestamping coprocessor (PTP) found on Cavium processors.
+ PTP provides timestamping mechanism that is suitable for use in IEEE 
1588
+ Precision Time Protocol or other purposes.  Timestamps can be used in
+ BGX, TNS, GTI, and NIC blocks.
+
 config LIQUIDIO
tristate "Cavium LiquidIO support"
depends on 64BIT
diff --git a/drivers/net/ethernet/cavium/Makefile 
b/drivers/net/ethernet/cavium/Makefile
index 872da9f7c31a..946bba84e81d 100644
--- a/drivers/net/ethernet/cavium/Makefile
+++ b/drivers/net/ethernet/cavium/Makefile
@@ -1,6 +1,7 @@
 #
 # Makefile for the Cavium ethernet device drivers.
 #
+obj-$(CONFIG_NET_VENDOR_CAVIUM) += common/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += thunder/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += liquidio/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += octeon/
diff --git a/drivers/net/ethernet/cavium/common/Makefile 
b/drivers/net/ethernet/cavium/common/Makefile
new file mode 100644
index ..dd8561b8060b
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_CAVIUM_PTP) += cavium_ptp.o
diff --git a/drivers/net/ethernet/cavium/common/cavium_ptp.c 
b/drivers/net/ethernet/cavium/common/cavium_ptp.c
new file mode 100644
index ..8b48863bfdec
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/cavium_ptp.c
@@ -0,0 +1,329 @@
+// SPDX-License-Identifier: GPL-2.0
+/* cavium_ptp.c - PTP 1588 clock on Cavium hardware
+ * Copyright (c) 2003-2015, 2017 Cavium, Inc.
+ */
+
+#include 
+#include 
+#include 
+#include 
+
+#include "cavium_ptp.h"
+
+#define DRV_NAME   "Cavium PTP Driver"
+
+#define PCI_DEVICE_ID_CAVIUM_PTP   0xA00C
+#define PCI_DEVICE_ID_CAVIUM_RST   0xA00E
+
+#define PCI_PTP_BAR_NO 0
+#define PCI_RST_BAR_NO 0
+
+#define PTP_CLOCK_CFG  0xF00ULL
+#define  PTP_CLOCK_CFG_PTP_EN  BIT(0)
+#define PTP_CLOCK_LO   0xF08ULL
+#define PTP_CLOCK_HI   0xF10ULL
+#define PTP_CLOCK_COMP 0xF18ULL
+
+#define RST_BOOT   0x1600ULL
+#define CLOCK_BASE_RATE5000ULL
+
+static u64 ptp_cavium_clock_get(void)
+{
+   struct pci_dev *pdev;
+   void __iomem *base;
+   u64 ret = CLOCK_BASE_RATE * 16;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_RST, NULL);
+   if (!pdev)
+   goto error;
+
+   base = pci_ioremap_bar(pdev, PCI_RST_BAR_NO);
+   if (!base)
+   goto error_put_pdev;
+
+   ret = CLOCK_BASE_RATE * ((readq(base + RST_BOOT) >> 33) & 0x3f);
+
+   iounmap(base);
+
+error_put_pdev:
+   pci_dev_put(pdev);
+
+error:
+   return ret;
+}
+
+struct cavium_ptp *cavium_ptp_get(void)
+{
+   struct cavium_ptp *ptp;
+   struct pci_dev *pdev;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_PTP, NULL);
+   if (!pdev)
+   return ERR_PTR(-ENODEV);
+
+   ptp = pci_get_drvdata(pdev);
+   if (!ptp) {
+   pci_dev_put(pdev);
+   ptp = ERR_PTR(-EPROBE_DEFER);
+   } else if (!ptp->ptp_clock) {
+   pci_dev_put(pdev);
+   ptp = ERR_PTR(-ENODEV);
+   }
+
+   return ptp;
+}
+EXPORT_SYMBOL(cavium_ptp_get);
+
+void cavium_ptp_put(struct cavium_ptp *ptp)
+{
+   pci_dev_put(ptp->pdev);
+}
+EXPORT_SYMBOL(cavium_ptp_put);
+
+/**
+ * cavium_ptp_adjfreq() - Adjust ptp frequency
+ * @ptp: PTP clock info
+ * @ppb: how much to adjust by, in parts-per-billion
+ */
+static int cavium_ptp_adjfreq(str

[PATCH net-next v5 0/2] net: thunderx: add support for PTP clock

[Aleksey Makarov]

This series adds support for IEEE 1588 Precision Time Protocol
to Cavium ethernet driver.

The first patch adds support for the Precision Time Protocol Clocks and
Timestamping coprocessor (PTP) found on Cavium processors.
It registers a new PTP clock in the PTP core and provides functions
to use the counter in BGX, TNS, GTI, and NIC blocks.

The second patch introduces support for the PTP protocol to the
Cavium ThunderX ethernet driver.

v5:
- fix the file headers (add SPDX tags, remove advertisment) (Philippe 
Ombredanne)
- use "imply" instead of "select" (Richard Cochran)
- add some code in cavium_ptp_get() for the case when the PTP driver has not 
been
  registered with the PTP core

v4: https://lkml.kernel.org/r/20171208103442.19354-1-aleksey.maka...@cavium.com
- use IS_ENABLED. This fixes compilation of the ptp as a module (David Miller)
- select PTP_1588_CLOCK, not depend on it.  This fixes a build warning.
- change u64 to __be64.  This fixes the sparse warning
  "warning: cast to restricted __be64"
- make nicvf_config_hwtstamp() static. This fixes the sparse warning
  "warning: symbol 'nicvf_config_hwtstamp' was not declared. Should it be 
static?"

v3: https://lkml.kernel.org/r/20171206133100.26436-1-aleksey.maka...@cavium.com
- rebase to net-next

v2: https://lkml.kernel.org/r/20171117134909.8954-1-aleksey.maka...@cavium.com
- use readq()/writeq() in place of cavium_ptp_reg_read()/cavium_ptp_reg_write(),
  don't use readq_relaxed()/writeq_relaxed() (David Daney)

v1: https://lkml.kernel.org/r/20171107190704.15458-1-aleksey.maka...@cavium.com

Radoslaw Biernacki (1):
  net: add support for Cavium PTP coprocessor

Sunil Goutham (1):
  net: thunderx: add timestamping support

 drivers/net/ethernet/cavium/Kconfig|  13 +
 drivers/net/ethernet/cavium/Makefile   |   1 +
 drivers/net/ethernet/cavium/common/Makefile|   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c| 329 +
 drivers/net/ethernet/cavium/common/cavium_ptp.h|  70 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  15 +
 drivers/net/ethernet/cavium/thunder/nic_main.c |  58 +++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 173 ++-
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 13 files changed, 743 insertions(+), 6 deletions(-)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

-- 
2.15.1

[PATCH net-next v5 0/2] net: thunderx: add support for PTP clock

[Aleksey Makarov]

This series adds support for IEEE 1588 Precision Time Protocol
to Cavium ethernet driver.

The first patch adds support for the Precision Time Protocol Clocks and
Timestamping coprocessor (PTP) found on Cavium processors.
It registers a new PTP clock in the PTP core and provides functions
to use the counter in BGX, TNS, GTI, and NIC blocks.

The second patch introduces support for the PTP protocol to the
Cavium ThunderX ethernet driver.

v5:
- fix the file headers (add SPDX tags, remove advertisment) (Philippe 
Ombredanne)
- use "imply" instead of "select" (Richard Cochran)
- add some code in cavium_ptp_get() for the case when the PTP driver has not 
been
  registered with the PTP core

v4: https://lkml.kernel.org/r/20171208103442.19354-1-aleksey.maka...@cavium.com
- use IS_ENABLED. This fixes compilation of the ptp as a module (David Miller)
- select PTP_1588_CLOCK, not depend on it.  This fixes a build warning.
- change u64 to __be64.  This fixes the sparse warning
  "warning: cast to restricted __be64"
- make nicvf_config_hwtstamp() static. This fixes the sparse warning
  "warning: symbol 'nicvf_config_hwtstamp' was not declared. Should it be 
static?"

v3: https://lkml.kernel.org/r/20171206133100.26436-1-aleksey.maka...@cavium.com
- rebase to net-next

v2: https://lkml.kernel.org/r/20171117134909.8954-1-aleksey.maka...@cavium.com
- use readq()/writeq() in place of cavium_ptp_reg_read()/cavium_ptp_reg_write(),
  don't use readq_relaxed()/writeq_relaxed() (David Daney)

v1: https://lkml.kernel.org/r/20171107190704.15458-1-aleksey.maka...@cavium.com

Radoslaw Biernacki (1):
  net: add support for Cavium PTP coprocessor

Sunil Goutham (1):
  net: thunderx: add timestamping support

 drivers/net/ethernet/cavium/Kconfig|  13 +
 drivers/net/ethernet/cavium/Makefile   |   1 +
 drivers/net/ethernet/cavium/common/Makefile|   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c| 329 +
 drivers/net/ethernet/cavium/common/cavium_ptp.h|  70 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  15 +
 drivers/net/ethernet/cavium/thunder/nic_main.c |  58 +++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 173 ++-
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 13 files changed, 743 insertions(+), 6 deletions(-)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

-- 
2.15.1

[PATCH net-next v4 2/2] net: thunderx: add timestamping support

[Aleksey Makarov]

From: Sunil Goutham <sgout...@cavium.com>

This adds timestamping support for both receive and transmit
paths. On the receive side no filters are supported i.e either
all pkts will get a timestamp appended infront of the packet or none.
On the transmit side HW doesn't support timestamp insertion but
only generates a separate CQE with transmitted packet's timestamp.
Also HW supports only one packet at a time for timestamping on the
transmit side.

Signed-off-by: Sunil Goutham <sgout...@cavium.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@cavium.com>
---
 drivers/net/ethernet/cavium/Kconfig|   1 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  15 ++
 drivers/net/ethernet/cavium/thunder/nic_main.c |  58 ++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +++-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 173 -
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 9 files changed, 330 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index 2380e9834007..d163c5bdbbcb 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -27,6 +27,7 @@ config THUNDER_NIC_PF
 
 config THUNDER_NIC_VF
tristate "Thunder Virtual function driver"
+   select CAVIUM_PTP
depends on 64BIT
---help---
  This driver supports Thunder's NIC virtual function
diff --git a/drivers/net/ethernet/cavium/thunder/nic.h 
b/drivers/net/ethernet/cavium/thunder/nic.h
index 4a02e618e318..204b234beb9d 100644
--- a/drivers/net/ethernet/cavium/thunder/nic.h
+++ b/drivers/net/ethernet/cavium/thunder/nic.h
@@ -263,6 +263,8 @@ struct nicvf_drv_stats {
struct u64_stats_sync   syncp;
 };
 
+struct cavium_ptp;
+
 struct nicvf {
struct nicvf*pnicvf;
struct net_device   *netdev;
@@ -312,6 +314,12 @@ struct nicvf {
struct tasklet_struct   qs_err_task;
struct work_struct  reset_task;
 
+   /* PTP timestamp */
+   struct cavium_ptp   *ptp_clock;
+   boolhw_rx_tstamp;
+   struct sk_buff  *ptp_skb;
+   atomic_ttx_ptp_skbs;
+
/* Interrupt coalescing settings */
u32 cq_coalesce_usecs;
u32 msg_enable;
@@ -371,6 +379,7 @@ struct nicvf {
 #defineNIC_MBOX_MSG_LOOPBACK   0x16/* Set interface in 
loopback */
 #defineNIC_MBOX_MSG_RESET_STAT_COUNTER 0x17/* Reset statistics 
counters */
 #defineNIC_MBOX_MSG_PFC0x18/* Pause frame control 
*/
+#defineNIC_MBOX_MSG_PTP_CFG0x19/* HW packet timestamp 
*/
 #defineNIC_MBOX_MSG_CFG_DONE   0xF0/* VF configuration 
done */
 #defineNIC_MBOX_MSG_SHUTDOWN   0xF1/* VF is being shutdown 
*/
 
@@ -521,6 +530,11 @@ struct pfc {
u8fc_tx;
 };
 
+struct set_ptp {
+   u8msg;
+   bool  enable;
+};
+
 /* 128 bit shared memory between PF and each VF */
 union nic_mbx {
struct { u8 msg; }  msg;
@@ -540,6 +554,7 @@ union nic_mbx {
struct set_loopback lbk;
struct reset_stat_cfg   reset_stat;
struct pfc  pfc;
+   struct set_ptp  ptp;
 };
 
 #define NIC_NODE_ID_MASK   0x03
diff --git a/drivers/net/ethernet/cavium/thunder/nic_main.c 
b/drivers/net/ethernet/cavium/thunder/nic_main.c
index 8f1dd55b3e08..4c1c5414a162 100644
--- a/drivers/net/ethernet/cavium/thunder/nic_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nic_main.c
@@ -426,13 +426,22 @@ static void nic_init_hw(struct nicpf *nic)
/* Enable backpressure */
nic_reg_write(nic, NIC_PF_BP_CFG, (1ULL << 6) | 0x03);
 
-   /* TNS and TNS bypass modes are present only on 88xx */
+   /* TNS and TNS bypass modes are present only on 88xx
+* Also offset of this CSR has changed in 81xx and 83xx.
+*/
if (nic->pdev->subsystem_device == PCI_SUBSYS_DEVID_88XX_NIC_PF) {
/* Disable TNS mode on both interfaces */
nic_reg_write(nic, NIC_PF_INTF_0_1_SEND_CFG,
- (NIC_TNS_BYPASS_MODE << 7) | BGX0_BLOCK);
+ (NIC_TNS_BYPASS_MODE << 7) |
+ BGX0_BLOCK | (1ULL << 16));
nic_reg_write(nic, NIC_PF_INTF_0_1_SEND_CFG | (1 << 8),
- (NIC_TNS_BYPASS_MODE << 7) | BGX1_BLOCK);
+ (NIC_TNS_BYPASS_MODE << 7) |
+ BGX1_BLOCK | (1ULL << 16));
+   } else {
+   /* Configure timestamp

[PATCH net-next v4 2/2] net: thunderx: add timestamping support

[Aleksey Makarov]

From: Sunil Goutham 

This adds timestamping support for both receive and transmit
paths. On the receive side no filters are supported i.e either
all pkts will get a timestamp appended infront of the packet or none.
On the transmit side HW doesn't support timestamp insertion but
only generates a separate CQE with transmitted packet's timestamp.
Also HW supports only one packet at a time for timestamping on the
transmit side.

Signed-off-by: Sunil Goutham 
Signed-off-by: Aleksey Makarov 
---
 drivers/net/ethernet/cavium/Kconfig|   1 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  15 ++
 drivers/net/ethernet/cavium/thunder/nic_main.c |  58 ++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +++-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 173 -
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 9 files changed, 330 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index 2380e9834007..d163c5bdbbcb 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -27,6 +27,7 @@ config THUNDER_NIC_PF
 
 config THUNDER_NIC_VF
tristate "Thunder Virtual function driver"
+   select CAVIUM_PTP
depends on 64BIT
---help---
  This driver supports Thunder's NIC virtual function
diff --git a/drivers/net/ethernet/cavium/thunder/nic.h 
b/drivers/net/ethernet/cavium/thunder/nic.h
index 4a02e618e318..204b234beb9d 100644
--- a/drivers/net/ethernet/cavium/thunder/nic.h
+++ b/drivers/net/ethernet/cavium/thunder/nic.h
@@ -263,6 +263,8 @@ struct nicvf_drv_stats {
struct u64_stats_sync   syncp;
 };
 
+struct cavium_ptp;
+
 struct nicvf {
struct nicvf*pnicvf;
struct net_device   *netdev;
@@ -312,6 +314,12 @@ struct nicvf {
struct tasklet_struct   qs_err_task;
struct work_struct  reset_task;
 
+   /* PTP timestamp */
+   struct cavium_ptp   *ptp_clock;
+   boolhw_rx_tstamp;
+   struct sk_buff  *ptp_skb;
+   atomic_ttx_ptp_skbs;
+
/* Interrupt coalescing settings */
u32 cq_coalesce_usecs;
u32 msg_enable;
@@ -371,6 +379,7 @@ struct nicvf {
 #defineNIC_MBOX_MSG_LOOPBACK   0x16/* Set interface in 
loopback */
 #defineNIC_MBOX_MSG_RESET_STAT_COUNTER 0x17/* Reset statistics 
counters */
 #defineNIC_MBOX_MSG_PFC0x18/* Pause frame control 
*/
+#defineNIC_MBOX_MSG_PTP_CFG0x19/* HW packet timestamp 
*/
 #defineNIC_MBOX_MSG_CFG_DONE   0xF0/* VF configuration 
done */
 #defineNIC_MBOX_MSG_SHUTDOWN   0xF1/* VF is being shutdown 
*/
 
@@ -521,6 +530,11 @@ struct pfc {
u8fc_tx;
 };
 
+struct set_ptp {
+   u8msg;
+   bool  enable;
+};
+
 /* 128 bit shared memory between PF and each VF */
 union nic_mbx {
struct { u8 msg; }  msg;
@@ -540,6 +554,7 @@ union nic_mbx {
struct set_loopback lbk;
struct reset_stat_cfg   reset_stat;
struct pfc  pfc;
+   struct set_ptp  ptp;
 };
 
 #define NIC_NODE_ID_MASK   0x03
diff --git a/drivers/net/ethernet/cavium/thunder/nic_main.c 
b/drivers/net/ethernet/cavium/thunder/nic_main.c
index 8f1dd55b3e08..4c1c5414a162 100644
--- a/drivers/net/ethernet/cavium/thunder/nic_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nic_main.c
@@ -426,13 +426,22 @@ static void nic_init_hw(struct nicpf *nic)
/* Enable backpressure */
nic_reg_write(nic, NIC_PF_BP_CFG, (1ULL << 6) | 0x03);
 
-   /* TNS and TNS bypass modes are present only on 88xx */
+   /* TNS and TNS bypass modes are present only on 88xx
+* Also offset of this CSR has changed in 81xx and 83xx.
+*/
if (nic->pdev->subsystem_device == PCI_SUBSYS_DEVID_88XX_NIC_PF) {
/* Disable TNS mode on both interfaces */
nic_reg_write(nic, NIC_PF_INTF_0_1_SEND_CFG,
- (NIC_TNS_BYPASS_MODE << 7) | BGX0_BLOCK);
+ (NIC_TNS_BYPASS_MODE << 7) |
+ BGX0_BLOCK | (1ULL << 16));
nic_reg_write(nic, NIC_PF_INTF_0_1_SEND_CFG | (1 << 8),
- (NIC_TNS_BYPASS_MODE << 7) | BGX1_BLOCK);
+ (NIC_TNS_BYPASS_MODE << 7) |
+ BGX1_BLOCK | (1ULL << 16));
+   } else {
+   /* Configure timestamp generation timeout to 10us */
+   for (i = 0; i < nic->hw->bgx_cnt; i

[PATCH net-next v4 0/2] net: thunderx: add support for PTP clock

[Aleksey Makarov]

This series adds support for IEEE 1588 Precision Time Protocol
to Cavium ethernet driver.

The first patch adds support for the Precision Time Protocol Clocks and
Timestamping coprocessor (PTP) found on Cavium processors.
It registers a new PTP clock in the PTP core and provides functions
to use the counter in BGX, TNS, GTI, and NIC blocks.

The second patch introduces support for the PTP protocol to the
Cavium ThunderX ethernet driver.

v4:
- use IS_ENABLED. This fixes compilation of the ptp as a module (David Miller)
- select PTP_1588_CLOCK, not depend on it.  This fixes a build warning.
- change u64 to __be64.  This fixes the sparse warning
  "warning: cast to restricted __be64"
- make nicvf_config_hwtstamp() static. This fixes the sparse warning
  "warning: symbol 'nicvf_config_hwtstamp' was not declared. Should it be 
static?"

v3: https://lkml.kernel.org/r/20171206133100.26436-1-aleksey.maka...@cavium.com
- rebase to net-next

v2: https://lkml.kernel.org/r/20171117134909.8954-1-aleksey.maka...@cavium.com
- use readq()/writeq() in place of cavium_ptp_reg_read()/cavium_ptp_reg_write(),
  don't use readq_relaxed()/writeq_relaxed() (David Daney)

v1: https://lkml.kernel.org/r/20171107190704.15458-1-aleksey.maka...@cavium.com

Radoslaw Biernacki (1):
  net: add support for Cavium PTP coprocessor

Sunil Goutham (1):
  net: thunderx: add timestamping support

 drivers/net/ethernet/cavium/Kconfig|  13 +
 drivers/net/ethernet/cavium/Makefile   |   1 +
 drivers/net/ethernet/cavium/common/Makefile|   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c| 334 +
 drivers/net/ethernet/cavium/common/cavium_ptp.h|  78 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  15 +
 drivers/net/ethernet/cavium/thunder/nic_main.c |  58 +++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 173 ++-
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 13 files changed, 756 insertions(+), 6 deletions(-)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

-- 
2.15.1

[PATCH net-next v4 1/2] net: add support for Cavium PTP coprocessor

[Aleksey Makarov]

From: Radoslaw Biernacki <r...@semihalf.com>

This patch adds support for the Precision Time Protocol
Clocks and Timestamping hardware found on Cavium ThunderX
processors.

Signed-off-by: Radoslaw Biernacki <r...@semihalf.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@cavium.com>
---
 drivers/net/ethernet/cavium/Kconfig |  12 +
 drivers/net/ethernet/cavium/Makefile|   1 +
 drivers/net/ethernet/cavium/common/Makefile |   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c | 334 
 drivers/net/ethernet/cavium/common/cavium_ptp.h |  78 ++
 5 files changed, 426 insertions(+)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index 63be75eb34d2..2380e9834007 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -50,6 +50,18 @@ config   THUNDER_NIC_RGX
  This driver supports configuring XCV block of RGX interface
  present on CN81XX chip.
 
+config CAVIUM_PTP
+   tristate "Cavium PTP coprocessor as PTP clock"
+   depends on 64BIT
+   select PTP_1588_CLOCK
+   default y
+   ---help---
+ This driver adds support for the Precision Time Protocol Clocks and
+ Timestamping coprocessor (PTP) found on Cavium processors.
+ PTP provides timestamping mechanism that is suitable for use in IEEE 
1588
+ Precision Time Protocol or other purposes.  Timestamps can be used in
+ BGX, TNS, GTI, and NIC blocks.
+
 config LIQUIDIO
tristate "Cavium LiquidIO support"
depends on 64BIT
diff --git a/drivers/net/ethernet/cavium/Makefile 
b/drivers/net/ethernet/cavium/Makefile
index 872da9f7c31a..946bba84e81d 100644
--- a/drivers/net/ethernet/cavium/Makefile
+++ b/drivers/net/ethernet/cavium/Makefile
@@ -1,6 +1,7 @@
 #
 # Makefile for the Cavium ethernet device drivers.
 #
+obj-$(CONFIG_NET_VENDOR_CAVIUM) += common/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += thunder/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += liquidio/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += octeon/
diff --git a/drivers/net/ethernet/cavium/common/Makefile 
b/drivers/net/ethernet/cavium/common/Makefile
new file mode 100644
index ..dd8561b8060b
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_CAVIUM_PTP) += cavium_ptp.o
diff --git a/drivers/net/ethernet/cavium/common/cavium_ptp.c 
b/drivers/net/ethernet/cavium/common/cavium_ptp.c
new file mode 100644
index ..f4c738db27fd
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/cavium_ptp.c
@@ -0,0 +1,334 @@
+/*
+ * cavium_ptp.c - PTP 1588 clock on Cavium hardware
+ *
+ * Copyright (c) 2003-2015, 2017 Cavium, Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License.
+ *
+ * This file may also be available under a different license from Cavium.
+ * Contact Cavium, Inc. for more information
+ */
+
+#include 
+#include 
+#include 
+#include 
+
+#include "cavium_ptp.h"
+
+#define DRV_NAME   "Cavium PTP Driver"
+
+#define PCI_DEVICE_ID_CAVIUM_PTP   0xA00C
+#define PCI_DEVICE_ID_CAVIUM_RST   0xA00E
+
+#define PCI_PTP_BAR_NO 0
+#define PCI_RST_BAR_NO 0
+
+#define PTP_CLOCK_CFG  0xF00ULL
+#define  PTP_CLOCK_CFG_PTP_EN  BIT(0)
+#define PTP_CLOCK_LO   0xF08ULL
+#define PTP_CLOCK_HI   0xF10ULL
+#define PTP_CLOCK_COMP 0xF18ULL
+
+#define RST_BOOT   0x1600ULL
+#define CLOCK_BASE_RATE5000ULL
+
+static u64 ptp_cavium_clock_get(void)
+{
+   struct pci_dev *pdev;
+   void __iomem *base;
+   u64 ret = CLOCK_BASE_RATE * 16;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_RST, NULL);
+   if (!pdev)
+   goto error;
+
+   base = pci_ioremap_bar(pdev, PCI_RST_BAR_NO);
+   if (!base)
+   goto error_put_pdev;
+
+   ret = CLOCK_BASE_RATE * ((readq(base + RST_BOOT) >> 33) & 0x3f);
+
+   iounmap(base);
+
+error_put_pdev:
+   pci_dev_put(pdev);
+
+error:
+   return ret;
+}
+
+struct cavium_ptp *cavium_ptp_get(void)
+{
+   struct cavium_ptp *ptp;
+   struct pci_dev *pdev;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_PTP, NULL);
+   if (!pdev)
+   return ERR_PTR(-ENODEV);
+
+   ptp = pci_get_drvdata(pdev);
+   if (!ptp) {
+   pci_dev_put(pdev);
+   ptp = ERR_PTR(-EPROBE_DEFER);
+   }
+
+   return ptp;
+}
+EXPORT_SYMBOL(cavium_ptp_get);
+
+void cav

[PATCH net-next v4 0/2] net: thunderx: add support for PTP clock

[Aleksey Makarov]

This series adds support for IEEE 1588 Precision Time Protocol
to Cavium ethernet driver.

The first patch adds support for the Precision Time Protocol Clocks and
Timestamping coprocessor (PTP) found on Cavium processors.
It registers a new PTP clock in the PTP core and provides functions
to use the counter in BGX, TNS, GTI, and NIC blocks.

The second patch introduces support for the PTP protocol to the
Cavium ThunderX ethernet driver.

v4:
- use IS_ENABLED. This fixes compilation of the ptp as a module (David Miller)
- select PTP_1588_CLOCK, not depend on it.  This fixes a build warning.
- change u64 to __be64.  This fixes the sparse warning
  "warning: cast to restricted __be64"
- make nicvf_config_hwtstamp() static. This fixes the sparse warning
  "warning: symbol 'nicvf_config_hwtstamp' was not declared. Should it be 
static?"

v3: https://lkml.kernel.org/r/20171206133100.26436-1-aleksey.maka...@cavium.com
- rebase to net-next

v2: https://lkml.kernel.org/r/20171117134909.8954-1-aleksey.maka...@cavium.com
- use readq()/writeq() in place of cavium_ptp_reg_read()/cavium_ptp_reg_write(),
  don't use readq_relaxed()/writeq_relaxed() (David Daney)

v1: https://lkml.kernel.org/r/20171107190704.15458-1-aleksey.maka...@cavium.com

Radoslaw Biernacki (1):
  net: add support for Cavium PTP coprocessor

Sunil Goutham (1):
  net: thunderx: add timestamping support

 drivers/net/ethernet/cavium/Kconfig|  13 +
 drivers/net/ethernet/cavium/Makefile   |   1 +
 drivers/net/ethernet/cavium/common/Makefile|   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c| 334 +
 drivers/net/ethernet/cavium/common/cavium_ptp.h|  78 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  15 +
 drivers/net/ethernet/cavium/thunder/nic_main.c |  58 +++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 173 ++-
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 13 files changed, 756 insertions(+), 6 deletions(-)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

-- 
2.15.1

[PATCH net-next v4 1/2] net: add support for Cavium PTP coprocessor

[Aleksey Makarov]

From: Radoslaw Biernacki 

This patch adds support for the Precision Time Protocol
Clocks and Timestamping hardware found on Cavium ThunderX
processors.

Signed-off-by: Radoslaw Biernacki 
Signed-off-by: Aleksey Makarov 
---
 drivers/net/ethernet/cavium/Kconfig |  12 +
 drivers/net/ethernet/cavium/Makefile|   1 +
 drivers/net/ethernet/cavium/common/Makefile |   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c | 334 
 drivers/net/ethernet/cavium/common/cavium_ptp.h |  78 ++
 5 files changed, 426 insertions(+)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index 63be75eb34d2..2380e9834007 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -50,6 +50,18 @@ config   THUNDER_NIC_RGX
  This driver supports configuring XCV block of RGX interface
  present on CN81XX chip.
 
+config CAVIUM_PTP
+   tristate "Cavium PTP coprocessor as PTP clock"
+   depends on 64BIT
+   select PTP_1588_CLOCK
+   default y
+   ---help---
+ This driver adds support for the Precision Time Protocol Clocks and
+ Timestamping coprocessor (PTP) found on Cavium processors.
+ PTP provides timestamping mechanism that is suitable for use in IEEE 
1588
+ Precision Time Protocol or other purposes.  Timestamps can be used in
+ BGX, TNS, GTI, and NIC blocks.
+
 config LIQUIDIO
tristate "Cavium LiquidIO support"
depends on 64BIT
diff --git a/drivers/net/ethernet/cavium/Makefile 
b/drivers/net/ethernet/cavium/Makefile
index 872da9f7c31a..946bba84e81d 100644
--- a/drivers/net/ethernet/cavium/Makefile
+++ b/drivers/net/ethernet/cavium/Makefile
@@ -1,6 +1,7 @@
 #
 # Makefile for the Cavium ethernet device drivers.
 #
+obj-$(CONFIG_NET_VENDOR_CAVIUM) += common/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += thunder/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += liquidio/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += octeon/
diff --git a/drivers/net/ethernet/cavium/common/Makefile 
b/drivers/net/ethernet/cavium/common/Makefile
new file mode 100644
index ..dd8561b8060b
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_CAVIUM_PTP) += cavium_ptp.o
diff --git a/drivers/net/ethernet/cavium/common/cavium_ptp.c 
b/drivers/net/ethernet/cavium/common/cavium_ptp.c
new file mode 100644
index ..f4c738db27fd
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/cavium_ptp.c
@@ -0,0 +1,334 @@
+/*
+ * cavium_ptp.c - PTP 1588 clock on Cavium hardware
+ *
+ * Copyright (c) 2003-2015, 2017 Cavium, Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License.
+ *
+ * This file may also be available under a different license from Cavium.
+ * Contact Cavium, Inc. for more information
+ */
+
+#include 
+#include 
+#include 
+#include 
+
+#include "cavium_ptp.h"
+
+#define DRV_NAME   "Cavium PTP Driver"
+
+#define PCI_DEVICE_ID_CAVIUM_PTP   0xA00C
+#define PCI_DEVICE_ID_CAVIUM_RST   0xA00E
+
+#define PCI_PTP_BAR_NO 0
+#define PCI_RST_BAR_NO 0
+
+#define PTP_CLOCK_CFG  0xF00ULL
+#define  PTP_CLOCK_CFG_PTP_EN  BIT(0)
+#define PTP_CLOCK_LO   0xF08ULL
+#define PTP_CLOCK_HI   0xF10ULL
+#define PTP_CLOCK_COMP 0xF18ULL
+
+#define RST_BOOT   0x1600ULL
+#define CLOCK_BASE_RATE5000ULL
+
+static u64 ptp_cavium_clock_get(void)
+{
+   struct pci_dev *pdev;
+   void __iomem *base;
+   u64 ret = CLOCK_BASE_RATE * 16;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_RST, NULL);
+   if (!pdev)
+   goto error;
+
+   base = pci_ioremap_bar(pdev, PCI_RST_BAR_NO);
+   if (!base)
+   goto error_put_pdev;
+
+   ret = CLOCK_BASE_RATE * ((readq(base + RST_BOOT) >> 33) & 0x3f);
+
+   iounmap(base);
+
+error_put_pdev:
+   pci_dev_put(pdev);
+
+error:
+   return ret;
+}
+
+struct cavium_ptp *cavium_ptp_get(void)
+{
+   struct cavium_ptp *ptp;
+   struct pci_dev *pdev;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_PTP, NULL);
+   if (!pdev)
+   return ERR_PTR(-ENODEV);
+
+   ptp = pci_get_drvdata(pdev);
+   if (!ptp) {
+   pci_dev_put(pdev);
+   ptp = ERR_PTR(-EPROBE_DEFER);
+   }
+
+   return ptp;
+}
+EXPORT_SYMBOL(cavium_ptp_get);
+
+void cavium_ptp_put(struct cavium_ptp *ptp)
+{
+   pci_dev_put(ptp->pdev);
+}
+EXPORT_

[PATCH net-next v3 2/2] net: thunderx: add timestamping support

[Aleksey Makarov]

From: Sunil Goutham <sgout...@cavium.com>

This adds timestamping support for both receive and transmit
paths. On the receive side no filters are supported i.e either
all pkts will get a timestamp appended infront of the packet or none.
On the transmit side HW doesn't support timestamp insertion but
only generates a separate CQE with transmitted packet's timestamp.
Also HW supports only one packet at a time for timestamping on the
transmit side.

Signed-off-by: Sunil Goutham <sgout...@cavium.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@cavium.com>
---
 drivers/net/ethernet/cavium/Kconfig|   1 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  15 ++
 drivers/net/ethernet/cavium/thunder/nic_main.c |  58 ++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +++-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 173 -
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 9 files changed, 330 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index fabe0ffcc2d4..6d003cbe197c 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -27,6 +27,7 @@ config THUNDER_NIC_PF
 
 config THUNDER_NIC_VF
tristate "Thunder Virtual function driver"
+   select CAVIUM_PTP
depends on 64BIT
---help---
  This driver supports Thunder's NIC virtual function
diff --git a/drivers/net/ethernet/cavium/thunder/nic.h 
b/drivers/net/ethernet/cavium/thunder/nic.h
index 4a02e618e318..204b234beb9d 100644
--- a/drivers/net/ethernet/cavium/thunder/nic.h
+++ b/drivers/net/ethernet/cavium/thunder/nic.h
@@ -263,6 +263,8 @@ struct nicvf_drv_stats {
struct u64_stats_sync   syncp;
 };
 
+struct cavium_ptp;
+
 struct nicvf {
struct nicvf*pnicvf;
struct net_device   *netdev;
@@ -312,6 +314,12 @@ struct nicvf {
struct tasklet_struct   qs_err_task;
struct work_struct  reset_task;
 
+   /* PTP timestamp */
+   struct cavium_ptp   *ptp_clock;
+   boolhw_rx_tstamp;
+   struct sk_buff  *ptp_skb;
+   atomic_ttx_ptp_skbs;
+
/* Interrupt coalescing settings */
u32 cq_coalesce_usecs;
u32 msg_enable;
@@ -371,6 +379,7 @@ struct nicvf {
 #defineNIC_MBOX_MSG_LOOPBACK   0x16/* Set interface in 
loopback */
 #defineNIC_MBOX_MSG_RESET_STAT_COUNTER 0x17/* Reset statistics 
counters */
 #defineNIC_MBOX_MSG_PFC0x18/* Pause frame control 
*/
+#defineNIC_MBOX_MSG_PTP_CFG0x19/* HW packet timestamp 
*/
 #defineNIC_MBOX_MSG_CFG_DONE   0xF0/* VF configuration 
done */
 #defineNIC_MBOX_MSG_SHUTDOWN   0xF1/* VF is being shutdown 
*/
 
@@ -521,6 +530,11 @@ struct pfc {
u8fc_tx;
 };
 
+struct set_ptp {
+   u8msg;
+   bool  enable;
+};
+
 /* 128 bit shared memory between PF and each VF */
 union nic_mbx {
struct { u8 msg; }  msg;
@@ -540,6 +554,7 @@ union nic_mbx {
struct set_loopback lbk;
struct reset_stat_cfg   reset_stat;
struct pfc  pfc;
+   struct set_ptp  ptp;
 };
 
 #define NIC_NODE_ID_MASK   0x03
diff --git a/drivers/net/ethernet/cavium/thunder/nic_main.c 
b/drivers/net/ethernet/cavium/thunder/nic_main.c
index 8f1dd55b3e08..4c1c5414a162 100644
--- a/drivers/net/ethernet/cavium/thunder/nic_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nic_main.c
@@ -426,13 +426,22 @@ static void nic_init_hw(struct nicpf *nic)
/* Enable backpressure */
nic_reg_write(nic, NIC_PF_BP_CFG, (1ULL << 6) | 0x03);
 
-   /* TNS and TNS bypass modes are present only on 88xx */
+   /* TNS and TNS bypass modes are present only on 88xx
+* Also offset of this CSR has changed in 81xx and 83xx.
+*/
if (nic->pdev->subsystem_device == PCI_SUBSYS_DEVID_88XX_NIC_PF) {
/* Disable TNS mode on both interfaces */
nic_reg_write(nic, NIC_PF_INTF_0_1_SEND_CFG,
- (NIC_TNS_BYPASS_MODE << 7) | BGX0_BLOCK);
+ (NIC_TNS_BYPASS_MODE << 7) |
+ BGX0_BLOCK | (1ULL << 16));
nic_reg_write(nic, NIC_PF_INTF_0_1_SEND_CFG | (1 << 8),
- (NIC_TNS_BYPASS_MODE << 7) | BGX1_BLOCK);
+ (NIC_TNS_BYPASS_MODE << 7) |
+ BGX1_BLOCK | (1ULL << 16));
+   } else {
+   /* Configure timestamp

[PATCH net-next v3 0/2] net: thunderx: add support for PTP clock

[Aleksey Makarov]

This series adds support for IEEE 1588 Precision Time Protocol
to Cavium ethernet driver.

The first patch adds support for the Precision Time Protocol Clocks and
Timestamping coprocessor (PTP) found on Cavium processors.
It registers a new PTP clock in the PTP core and provides functions
to use the counter in BGX, TNS, GTI, and NIC blocks.

The second patch introduces support for the PTP protocol to the
Cavium ThunderX ethernet driver.

v3:
- rebase to net-next

v2: https://lkml.kernel.org/r/20171117134909.8954-1-aleksey.maka...@cavium.com
- use readq()/writeq() in place of cavium_ptp_reg_read()/cavium_ptp_reg_write(),
  don't use readq_relaxed()/writeq_relaxed() (David Daney)

v1: https://lkml.kernel.org/r/20171107190704.15458-1-aleksey.maka...@cavium.com

Radoslaw Biernacki (1):
  net: add support for Cavium PTP coprocessor

Sunil Goutham (1):
  net: thunderx: add timestamping support

 drivers/net/ethernet/cavium/Kconfig|  14 +
 drivers/net/ethernet/cavium/Makefile   |   1 +
 drivers/net/ethernet/cavium/common/Makefile|   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c| 334 +
 drivers/net/ethernet/cavium/common/cavium_ptp.h|  78 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  15 +
 drivers/net/ethernet/cavium/thunder/nic_main.c |  58 +++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 173 ++-
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 13 files changed, 757 insertions(+), 6 deletions(-)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

-- 
2.15.1

[PATCH net-next v3 1/2] net: add support for Cavium PTP coprocessor

[Aleksey Makarov]

From: Radoslaw Biernacki <r...@semihalf.com>

This patch adds support for the Precision Time Protocol
Clocks and Timestamping hardware found on Cavium ThunderX
processors.

Signed-off-by: Radoslaw Biernacki <r...@semihalf.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@cavium.com>
---
 drivers/net/ethernet/cavium/Kconfig |  13 +
 drivers/net/ethernet/cavium/Makefile|   1 +
 drivers/net/ethernet/cavium/common/Makefile |   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c | 334 
 drivers/net/ethernet/cavium/common/cavium_ptp.h |  78 ++
 5 files changed, 427 insertions(+)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index 63be75eb34d2..fabe0ffcc2d4 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -50,6 +50,19 @@ config   THUNDER_NIC_RGX
  This driver supports configuring XCV block of RGX interface
  present on CN81XX chip.
 
+config CAVIUM_PTP
+   tristate "Cavium PTP coprocessor as PTP clock"
+   depends on 64BIT
+   depends on PTP_1588_CLOCK
+   select CAVIUM_RST
+   default y
+   ---help---
+ This driver adds support for the Precision Time Protocol Clocks and
+ Timestamping coprocessor (PTP) found on Cavium processors.
+ PTP provides timestamping mechanism that is suitable for use in IEEE 
1588
+ Precision Time Protocol or other purposes.  Timestamps can be used in
+ BGX, TNS, GTI, and NIC blocks.
+
 config LIQUIDIO
tristate "Cavium LiquidIO support"
depends on 64BIT
diff --git a/drivers/net/ethernet/cavium/Makefile 
b/drivers/net/ethernet/cavium/Makefile
index 872da9f7c31a..946bba84e81d 100644
--- a/drivers/net/ethernet/cavium/Makefile
+++ b/drivers/net/ethernet/cavium/Makefile
@@ -1,6 +1,7 @@
 #
 # Makefile for the Cavium ethernet device drivers.
 #
+obj-$(CONFIG_NET_VENDOR_CAVIUM) += common/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += thunder/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += liquidio/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += octeon/
diff --git a/drivers/net/ethernet/cavium/common/Makefile 
b/drivers/net/ethernet/cavium/common/Makefile
new file mode 100644
index ..dd8561b8060b
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_CAVIUM_PTP) += cavium_ptp.o
diff --git a/drivers/net/ethernet/cavium/common/cavium_ptp.c 
b/drivers/net/ethernet/cavium/common/cavium_ptp.c
new file mode 100644
index ..f4c738db27fd
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/cavium_ptp.c
@@ -0,0 +1,334 @@
+/*
+ * cavium_ptp.c - PTP 1588 clock on Cavium hardware
+ *
+ * Copyright (c) 2003-2015, 2017 Cavium, Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License.
+ *
+ * This file may also be available under a different license from Cavium.
+ * Contact Cavium, Inc. for more information
+ */
+
+#include 
+#include 
+#include 
+#include 
+
+#include "cavium_ptp.h"
+
+#define DRV_NAME   "Cavium PTP Driver"
+
+#define PCI_DEVICE_ID_CAVIUM_PTP   0xA00C
+#define PCI_DEVICE_ID_CAVIUM_RST   0xA00E
+
+#define PCI_PTP_BAR_NO 0
+#define PCI_RST_BAR_NO 0
+
+#define PTP_CLOCK_CFG  0xF00ULL
+#define  PTP_CLOCK_CFG_PTP_EN  BIT(0)
+#define PTP_CLOCK_LO   0xF08ULL
+#define PTP_CLOCK_HI   0xF10ULL
+#define PTP_CLOCK_COMP 0xF18ULL
+
+#define RST_BOOT   0x1600ULL
+#define CLOCK_BASE_RATE5000ULL
+
+static u64 ptp_cavium_clock_get(void)
+{
+   struct pci_dev *pdev;
+   void __iomem *base;
+   u64 ret = CLOCK_BASE_RATE * 16;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_RST, NULL);
+   if (!pdev)
+   goto error;
+
+   base = pci_ioremap_bar(pdev, PCI_RST_BAR_NO);
+   if (!base)
+   goto error_put_pdev;
+
+   ret = CLOCK_BASE_RATE * ((readq(base + RST_BOOT) >> 33) & 0x3f);
+
+   iounmap(base);
+
+error_put_pdev:
+   pci_dev_put(pdev);
+
+error:
+   return ret;
+}
+
+struct cavium_ptp *cavium_ptp_get(void)
+{
+   struct cavium_ptp *ptp;
+   struct pci_dev *pdev;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_PTP, NULL);
+   if (!pdev)
+   return ERR_PTR(-ENODEV);
+
+   ptp = pci_get_drvdata(pdev);
+   if (!ptp) {
+   pci_dev_put(pdev);
+   ptp = ERR_PTR(-EPROBE_DEFER);
+   }
+
+   return ptp;
+}
+EXPORT_S

[PATCH net-next v3 2/2] net: thunderx: add timestamping support

[Aleksey Makarov]

From: Sunil Goutham 

This adds timestamping support for both receive and transmit
paths. On the receive side no filters are supported i.e either
all pkts will get a timestamp appended infront of the packet or none.
On the transmit side HW doesn't support timestamp insertion but
only generates a separate CQE with transmitted packet's timestamp.
Also HW supports only one packet at a time for timestamping on the
transmit side.

Signed-off-by: Sunil Goutham 
Signed-off-by: Aleksey Makarov 
---
 drivers/net/ethernet/cavium/Kconfig|   1 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  15 ++
 drivers/net/ethernet/cavium/thunder/nic_main.c |  58 ++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +++-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 173 -
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 9 files changed, 330 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index fabe0ffcc2d4..6d003cbe197c 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -27,6 +27,7 @@ config THUNDER_NIC_PF
 
 config THUNDER_NIC_VF
tristate "Thunder Virtual function driver"
+   select CAVIUM_PTP
depends on 64BIT
---help---
  This driver supports Thunder's NIC virtual function
diff --git a/drivers/net/ethernet/cavium/thunder/nic.h 
b/drivers/net/ethernet/cavium/thunder/nic.h
index 4a02e618e318..204b234beb9d 100644
--- a/drivers/net/ethernet/cavium/thunder/nic.h
+++ b/drivers/net/ethernet/cavium/thunder/nic.h
@@ -263,6 +263,8 @@ struct nicvf_drv_stats {
struct u64_stats_sync   syncp;
 };
 
+struct cavium_ptp;
+
 struct nicvf {
struct nicvf*pnicvf;
struct net_device   *netdev;
@@ -312,6 +314,12 @@ struct nicvf {
struct tasklet_struct   qs_err_task;
struct work_struct  reset_task;
 
+   /* PTP timestamp */
+   struct cavium_ptp   *ptp_clock;
+   boolhw_rx_tstamp;
+   struct sk_buff  *ptp_skb;
+   atomic_ttx_ptp_skbs;
+
/* Interrupt coalescing settings */
u32 cq_coalesce_usecs;
u32 msg_enable;
@@ -371,6 +379,7 @@ struct nicvf {
 #defineNIC_MBOX_MSG_LOOPBACK   0x16/* Set interface in 
loopback */
 #defineNIC_MBOX_MSG_RESET_STAT_COUNTER 0x17/* Reset statistics 
counters */
 #defineNIC_MBOX_MSG_PFC0x18/* Pause frame control 
*/
+#defineNIC_MBOX_MSG_PTP_CFG0x19/* HW packet timestamp 
*/
 #defineNIC_MBOX_MSG_CFG_DONE   0xF0/* VF configuration 
done */
 #defineNIC_MBOX_MSG_SHUTDOWN   0xF1/* VF is being shutdown 
*/
 
@@ -521,6 +530,11 @@ struct pfc {
u8fc_tx;
 };
 
+struct set_ptp {
+   u8msg;
+   bool  enable;
+};
+
 /* 128 bit shared memory between PF and each VF */
 union nic_mbx {
struct { u8 msg; }  msg;
@@ -540,6 +554,7 @@ union nic_mbx {
struct set_loopback lbk;
struct reset_stat_cfg   reset_stat;
struct pfc  pfc;
+   struct set_ptp  ptp;
 };
 
 #define NIC_NODE_ID_MASK   0x03
diff --git a/drivers/net/ethernet/cavium/thunder/nic_main.c 
b/drivers/net/ethernet/cavium/thunder/nic_main.c
index 8f1dd55b3e08..4c1c5414a162 100644
--- a/drivers/net/ethernet/cavium/thunder/nic_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nic_main.c
@@ -426,13 +426,22 @@ static void nic_init_hw(struct nicpf *nic)
/* Enable backpressure */
nic_reg_write(nic, NIC_PF_BP_CFG, (1ULL << 6) | 0x03);
 
-   /* TNS and TNS bypass modes are present only on 88xx */
+   /* TNS and TNS bypass modes are present only on 88xx
+* Also offset of this CSR has changed in 81xx and 83xx.
+*/
if (nic->pdev->subsystem_device == PCI_SUBSYS_DEVID_88XX_NIC_PF) {
/* Disable TNS mode on both interfaces */
nic_reg_write(nic, NIC_PF_INTF_0_1_SEND_CFG,
- (NIC_TNS_BYPASS_MODE << 7) | BGX0_BLOCK);
+ (NIC_TNS_BYPASS_MODE << 7) |
+ BGX0_BLOCK | (1ULL << 16));
nic_reg_write(nic, NIC_PF_INTF_0_1_SEND_CFG | (1 << 8),
- (NIC_TNS_BYPASS_MODE << 7) | BGX1_BLOCK);
+ (NIC_TNS_BYPASS_MODE << 7) |
+ BGX1_BLOCK | (1ULL << 16));
+   } else {
+   /* Configure timestamp generation timeout to 10us */
+   for (i = 0; i < nic->hw->bgx_cnt; i

[PATCH net-next v3 0/2] net: thunderx: add support for PTP clock

[Aleksey Makarov]

This series adds support for IEEE 1588 Precision Time Protocol
to Cavium ethernet driver.

The first patch adds support for the Precision Time Protocol Clocks and
Timestamping coprocessor (PTP) found on Cavium processors.
It registers a new PTP clock in the PTP core and provides functions
to use the counter in BGX, TNS, GTI, and NIC blocks.

The second patch introduces support for the PTP protocol to the
Cavium ThunderX ethernet driver.

v3:
- rebase to net-next

v2: https://lkml.kernel.org/r/20171117134909.8954-1-aleksey.maka...@cavium.com
- use readq()/writeq() in place of cavium_ptp_reg_read()/cavium_ptp_reg_write(),
  don't use readq_relaxed()/writeq_relaxed() (David Daney)

v1: https://lkml.kernel.org/r/20171107190704.15458-1-aleksey.maka...@cavium.com

Radoslaw Biernacki (1):
  net: add support for Cavium PTP coprocessor

Sunil Goutham (1):
  net: thunderx: add timestamping support

 drivers/net/ethernet/cavium/Kconfig|  14 +
 drivers/net/ethernet/cavium/Makefile   |   1 +
 drivers/net/ethernet/cavium/common/Makefile|   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c| 334 +
 drivers/net/ethernet/cavium/common/cavium_ptp.h|  78 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  15 +
 drivers/net/ethernet/cavium/thunder/nic_main.c |  58 +++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 173 ++-
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 13 files changed, 757 insertions(+), 6 deletions(-)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

-- 
2.15.1

[PATCH net-next v3 1/2] net: add support for Cavium PTP coprocessor

[Aleksey Makarov]

From: Radoslaw Biernacki 

This patch adds support for the Precision Time Protocol
Clocks and Timestamping hardware found on Cavium ThunderX
processors.

Signed-off-by: Radoslaw Biernacki 
Signed-off-by: Aleksey Makarov 
---
 drivers/net/ethernet/cavium/Kconfig |  13 +
 drivers/net/ethernet/cavium/Makefile|   1 +
 drivers/net/ethernet/cavium/common/Makefile |   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c | 334 
 drivers/net/ethernet/cavium/common/cavium_ptp.h |  78 ++
 5 files changed, 427 insertions(+)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index 63be75eb34d2..fabe0ffcc2d4 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -50,6 +50,19 @@ config   THUNDER_NIC_RGX
  This driver supports configuring XCV block of RGX interface
  present on CN81XX chip.
 
+config CAVIUM_PTP
+   tristate "Cavium PTP coprocessor as PTP clock"
+   depends on 64BIT
+   depends on PTP_1588_CLOCK
+   select CAVIUM_RST
+   default y
+   ---help---
+ This driver adds support for the Precision Time Protocol Clocks and
+ Timestamping coprocessor (PTP) found on Cavium processors.
+ PTP provides timestamping mechanism that is suitable for use in IEEE 
1588
+ Precision Time Protocol or other purposes.  Timestamps can be used in
+ BGX, TNS, GTI, and NIC blocks.
+
 config LIQUIDIO
tristate "Cavium LiquidIO support"
depends on 64BIT
diff --git a/drivers/net/ethernet/cavium/Makefile 
b/drivers/net/ethernet/cavium/Makefile
index 872da9f7c31a..946bba84e81d 100644
--- a/drivers/net/ethernet/cavium/Makefile
+++ b/drivers/net/ethernet/cavium/Makefile
@@ -1,6 +1,7 @@
 #
 # Makefile for the Cavium ethernet device drivers.
 #
+obj-$(CONFIG_NET_VENDOR_CAVIUM) += common/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += thunder/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += liquidio/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += octeon/
diff --git a/drivers/net/ethernet/cavium/common/Makefile 
b/drivers/net/ethernet/cavium/common/Makefile
new file mode 100644
index ..dd8561b8060b
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_CAVIUM_PTP) += cavium_ptp.o
diff --git a/drivers/net/ethernet/cavium/common/cavium_ptp.c 
b/drivers/net/ethernet/cavium/common/cavium_ptp.c
new file mode 100644
index ..f4c738db27fd
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/cavium_ptp.c
@@ -0,0 +1,334 @@
+/*
+ * cavium_ptp.c - PTP 1588 clock on Cavium hardware
+ *
+ * Copyright (c) 2003-2015, 2017 Cavium, Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License.
+ *
+ * This file may also be available under a different license from Cavium.
+ * Contact Cavium, Inc. for more information
+ */
+
+#include 
+#include 
+#include 
+#include 
+
+#include "cavium_ptp.h"
+
+#define DRV_NAME   "Cavium PTP Driver"
+
+#define PCI_DEVICE_ID_CAVIUM_PTP   0xA00C
+#define PCI_DEVICE_ID_CAVIUM_RST   0xA00E
+
+#define PCI_PTP_BAR_NO 0
+#define PCI_RST_BAR_NO 0
+
+#define PTP_CLOCK_CFG  0xF00ULL
+#define  PTP_CLOCK_CFG_PTP_EN  BIT(0)
+#define PTP_CLOCK_LO   0xF08ULL
+#define PTP_CLOCK_HI   0xF10ULL
+#define PTP_CLOCK_COMP 0xF18ULL
+
+#define RST_BOOT   0x1600ULL
+#define CLOCK_BASE_RATE5000ULL
+
+static u64 ptp_cavium_clock_get(void)
+{
+   struct pci_dev *pdev;
+   void __iomem *base;
+   u64 ret = CLOCK_BASE_RATE * 16;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_RST, NULL);
+   if (!pdev)
+   goto error;
+
+   base = pci_ioremap_bar(pdev, PCI_RST_BAR_NO);
+   if (!base)
+   goto error_put_pdev;
+
+   ret = CLOCK_BASE_RATE * ((readq(base + RST_BOOT) >> 33) & 0x3f);
+
+   iounmap(base);
+
+error_put_pdev:
+   pci_dev_put(pdev);
+
+error:
+   return ret;
+}
+
+struct cavium_ptp *cavium_ptp_get(void)
+{
+   struct cavium_ptp *ptp;
+   struct pci_dev *pdev;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_PTP, NULL);
+   if (!pdev)
+   return ERR_PTR(-ENODEV);
+
+   ptp = pci_get_drvdata(pdev);
+   if (!ptp) {
+   pci_dev_put(pdev);
+   ptp = ERR_PTR(-EPROBE_DEFER);
+   }
+
+   return ptp;
+}
+EXPORT_SYMBOL(cavium_ptp_get);
+
+void cavium_ptp_put(struct cavium_ptp *ptp)
+{
+   pci_de

[PATCH net-next] net: thunderx: Set max queue count taking XDP_TX into account

[Aleksey Makarov]

From: Sunil Goutham <sgout...@cavium.com>

on T81 there are only 4 cores, hence setting max queue count to 4
would leave nothing for XDP_TX. This patch fixes this by doubling
max queue count in above scenarios.

Signed-off-by: Sunil Goutham <sgout...@cavium.com>
Signed-off-by: cjacob <cja...@caviumnetworks.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@cavium.com>
---
 drivers/net/ethernet/cavium/thunder/nicvf_main.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_main.c 
b/drivers/net/ethernet/cavium/thunder/nicvf_main.c
index b82e28262c57..52b3a6044f85 100644
--- a/drivers/net/ethernet/cavium/thunder/nicvf_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nicvf_main.c
@@ -1891,6 +1891,11 @@ static int nicvf_probe(struct pci_dev *pdev, const 
struct pci_device_id *ent)
nic->pdev = pdev;
nic->pnicvf = nic;
nic->max_queues = qcount;
+   /* If no of CPUs are too low, there won't be any queues left
+* for XDP_TX, hence double it.
+*/
+   if (!nic->t88)
+   nic->max_queues *= 2;
 
/* MAP VF's configuration registers */
nic->reg_base = pcim_iomap(pdev, PCI_CFG_REG_BAR_NUM, 0);
-- 
2.15.0

[PATCH net-next] net: thunderx: Set max queue count taking XDP_TX into account

[Aleksey Makarov]

From: Sunil Goutham 

on T81 there are only 4 cores, hence setting max queue count to 4
would leave nothing for XDP_TX. This patch fixes this by doubling
max queue count in above scenarios.

Signed-off-by: Sunil Goutham 
Signed-off-by: cjacob 
Signed-off-by: Aleksey Makarov 
---
 drivers/net/ethernet/cavium/thunder/nicvf_main.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_main.c 
b/drivers/net/ethernet/cavium/thunder/nicvf_main.c
index b82e28262c57..52b3a6044f85 100644
--- a/drivers/net/ethernet/cavium/thunder/nicvf_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nicvf_main.c
@@ -1891,6 +1891,11 @@ static int nicvf_probe(struct pci_dev *pdev, const 
struct pci_device_id *ent)
nic->pdev = pdev;
nic->pnicvf = nic;
nic->max_queues = qcount;
+   /* If no of CPUs are too low, there won't be any queues left
+* for XDP_TX, hence double it.
+*/
+   if (!nic->t88)
+   nic->max_queues *= 2;
 
/* MAP VF's configuration registers */
nic->reg_base = pcim_iomap(pdev, PCI_CFG_REG_BAR_NUM, 0);
-- 
2.15.0

[PATCH net-next] net: thunderx: Add support for xdp redirect

[Aleksey Makarov]

From: Sunil Goutham <sgout...@cavium.com>

This patch adds support for XDP_REDIRECT. Flush is not
yet supported.

Signed-off-by: Sunil Goutham <sgout...@cavium.com>
Signed-off-by: cjacob <cja...@caviumnetworks.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@cavium.com>
---
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 110 -
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  11 ++-
 drivers/net/ethernet/cavium/thunder/nicvf_queues.h |   4 +
 3 files changed, 94 insertions(+), 31 deletions(-)

diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_main.c 
b/drivers/net/ethernet/cavium/thunder/nicvf_main.c
index a063c36c4c58..b82e28262c57 100644
--- a/drivers/net/ethernet/cavium/thunder/nicvf_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nicvf_main.c
@@ -65,6 +65,11 @@ module_param(cpi_alg, int, S_IRUGO);
 MODULE_PARM_DESC(cpi_alg,
 "PFC algorithm (0=none, 1=VLAN, 2=VLAN16, 3=IP Diffserv)");
 
+struct nicvf_xdp_tx {
+   u64 dma_addr;
+   u8  qidx;
+};
+
 static inline u8 nicvf_netdev_qidx(struct nicvf *nic, u8 qidx)
 {
if (nic->sqs_mode)
@@ -500,14 +505,29 @@ static int nicvf_init_resources(struct nicvf *nic)
return 0;
 }
 
+static void nicvf_unmap_page(struct nicvf *nic, struct page *page, u64 
dma_addr)
+{
+   /* Check if it's a recycled page, if not unmap the DMA mapping.
+* Recycled page holds an extra reference.
+*/
+   if (page_ref_count(page) == 1) {
+   dma_addr &= PAGE_MASK;
+   dma_unmap_page_attrs(>pdev->dev, dma_addr,
+RCV_FRAG_LEN + XDP_HEADROOM,
+DMA_FROM_DEVICE,
+DMA_ATTR_SKIP_CPU_SYNC);
+   }
+}
+
 static inline bool nicvf_xdp_rx(struct nicvf *nic, struct bpf_prog *prog,
struct cqe_rx_t *cqe_rx, struct snd_queue *sq,
struct sk_buff **skb)
 {
struct xdp_buff xdp;
struct page *page;
+   struct nicvf_xdp_tx *xdp_tx = NULL;
u32 action;
-   u16 len, offset = 0;
+   u16 len, err, offset = 0;
u64 dma_addr, cpu_addr;
void *orig_data;
 
@@ -521,7 +541,7 @@ static inline bool nicvf_xdp_rx(struct nicvf *nic, struct 
bpf_prog *prog,
cpu_addr = (u64)phys_to_virt(cpu_addr);
page = virt_to_page((void *)cpu_addr);
 
-   xdp.data_hard_start = page_address(page);
+   xdp.data_hard_start = page_address(page) + RCV_BUF_HEADROOM;
xdp.data = (void *)cpu_addr;
xdp_set_data_meta_invalid();
xdp.data_end = xdp.data + len;
@@ -540,18 +560,7 @@ static inline bool nicvf_xdp_rx(struct nicvf *nic, struct 
bpf_prog *prog,
 
switch (action) {
case XDP_PASS:
-   /* Check if it's a recycled page, if not
-* unmap the DMA mapping.
-*
-* Recycled page holds an extra reference.
-*/
-   if (page_ref_count(page) == 1) {
-   dma_addr &= PAGE_MASK;
-   dma_unmap_page_attrs(>pdev->dev, dma_addr,
-RCV_FRAG_LEN + XDP_PACKET_HEADROOM,
-DMA_FROM_DEVICE,
-DMA_ATTR_SKIP_CPU_SYNC);
-   }
+   nicvf_unmap_page(nic, page, dma_addr);
 
/* Build SKB and pass on packet to network stack */
*skb = build_skb(xdp.data,
@@ -564,6 +573,20 @@ static inline bool nicvf_xdp_rx(struct nicvf *nic, struct 
bpf_prog *prog,
case XDP_TX:
nicvf_xdp_sq_append_pkt(nic, sq, (u64)xdp.data, dma_addr, len);
return true;
+   case XDP_REDIRECT:
+   /* Save DMA address for use while transmitting */
+   xdp_tx = (struct nicvf_xdp_tx *)page_address(page);
+   xdp_tx->dma_addr = dma_addr;
+   xdp_tx->qidx = nicvf_netdev_qidx(nic, cqe_rx->rq_idx);
+
+   err = xdp_do_redirect(nic->pnicvf->netdev, , prog);
+   if (!err)
+   return true;
+
+   /* Free the page on error */
+   nicvf_unmap_page(nic, page, dma_addr);
+   put_page(page);
+   break;
default:
bpf_warn_invalid_xdp_action(action);
/* fall through */
@@ -571,18 +594,7 @@ static inline bool nicvf_xdp_rx(struct nicvf *nic, struct 
bpf_prog *prog,
trace_xdp_exception(nic->netdev, prog, action);
/* fall through */
case XDP_DROP:
-   /* Check if it's a recycled page, if not
-* unmap the DMA mapping.
-*
-* Recycled page holds an extra reference.
-*/
-   if 

[PATCH net-next] net: thunderx: Add support for xdp redirect

[Aleksey Makarov]

From: Sunil Goutham 

This patch adds support for XDP_REDIRECT. Flush is not
yet supported.

Signed-off-by: Sunil Goutham 
Signed-off-by: cjacob 
Signed-off-by: Aleksey Makarov 
---
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 110 -
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  11 ++-
 drivers/net/ethernet/cavium/thunder/nicvf_queues.h |   4 +
 3 files changed, 94 insertions(+), 31 deletions(-)

diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_main.c 
b/drivers/net/ethernet/cavium/thunder/nicvf_main.c
index a063c36c4c58..b82e28262c57 100644
--- a/drivers/net/ethernet/cavium/thunder/nicvf_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nicvf_main.c
@@ -65,6 +65,11 @@ module_param(cpi_alg, int, S_IRUGO);
 MODULE_PARM_DESC(cpi_alg,
 "PFC algorithm (0=none, 1=VLAN, 2=VLAN16, 3=IP Diffserv)");
 
+struct nicvf_xdp_tx {
+   u64 dma_addr;
+   u8  qidx;
+};
+
 static inline u8 nicvf_netdev_qidx(struct nicvf *nic, u8 qidx)
 {
if (nic->sqs_mode)
@@ -500,14 +505,29 @@ static int nicvf_init_resources(struct nicvf *nic)
return 0;
 }
 
+static void nicvf_unmap_page(struct nicvf *nic, struct page *page, u64 
dma_addr)
+{
+   /* Check if it's a recycled page, if not unmap the DMA mapping.
+* Recycled page holds an extra reference.
+*/
+   if (page_ref_count(page) == 1) {
+   dma_addr &= PAGE_MASK;
+   dma_unmap_page_attrs(>pdev->dev, dma_addr,
+RCV_FRAG_LEN + XDP_HEADROOM,
+DMA_FROM_DEVICE,
+DMA_ATTR_SKIP_CPU_SYNC);
+   }
+}
+
 static inline bool nicvf_xdp_rx(struct nicvf *nic, struct bpf_prog *prog,
struct cqe_rx_t *cqe_rx, struct snd_queue *sq,
struct sk_buff **skb)
 {
struct xdp_buff xdp;
struct page *page;
+   struct nicvf_xdp_tx *xdp_tx = NULL;
u32 action;
-   u16 len, offset = 0;
+   u16 len, err, offset = 0;
u64 dma_addr, cpu_addr;
void *orig_data;
 
@@ -521,7 +541,7 @@ static inline bool nicvf_xdp_rx(struct nicvf *nic, struct 
bpf_prog *prog,
cpu_addr = (u64)phys_to_virt(cpu_addr);
page = virt_to_page((void *)cpu_addr);
 
-   xdp.data_hard_start = page_address(page);
+   xdp.data_hard_start = page_address(page) + RCV_BUF_HEADROOM;
xdp.data = (void *)cpu_addr;
xdp_set_data_meta_invalid();
xdp.data_end = xdp.data + len;
@@ -540,18 +560,7 @@ static inline bool nicvf_xdp_rx(struct nicvf *nic, struct 
bpf_prog *prog,
 
switch (action) {
case XDP_PASS:
-   /* Check if it's a recycled page, if not
-* unmap the DMA mapping.
-*
-* Recycled page holds an extra reference.
-*/
-   if (page_ref_count(page) == 1) {
-   dma_addr &= PAGE_MASK;
-   dma_unmap_page_attrs(>pdev->dev, dma_addr,
-RCV_FRAG_LEN + XDP_PACKET_HEADROOM,
-DMA_FROM_DEVICE,
-DMA_ATTR_SKIP_CPU_SYNC);
-   }
+   nicvf_unmap_page(nic, page, dma_addr);
 
/* Build SKB and pass on packet to network stack */
*skb = build_skb(xdp.data,
@@ -564,6 +573,20 @@ static inline bool nicvf_xdp_rx(struct nicvf *nic, struct 
bpf_prog *prog,
case XDP_TX:
nicvf_xdp_sq_append_pkt(nic, sq, (u64)xdp.data, dma_addr, len);
return true;
+   case XDP_REDIRECT:
+   /* Save DMA address for use while transmitting */
+   xdp_tx = (struct nicvf_xdp_tx *)page_address(page);
+   xdp_tx->dma_addr = dma_addr;
+   xdp_tx->qidx = nicvf_netdev_qidx(nic, cqe_rx->rq_idx);
+
+   err = xdp_do_redirect(nic->pnicvf->netdev, , prog);
+   if (!err)
+   return true;
+
+   /* Free the page on error */
+   nicvf_unmap_page(nic, page, dma_addr);
+   put_page(page);
+   break;
default:
bpf_warn_invalid_xdp_action(action);
/* fall through */
@@ -571,18 +594,7 @@ static inline bool nicvf_xdp_rx(struct nicvf *nic, struct 
bpf_prog *prog,
trace_xdp_exception(nic->netdev, prog, action);
/* fall through */
case XDP_DROP:
-   /* Check if it's a recycled page, if not
-* unmap the DMA mapping.
-*
-* Recycled page holds an extra reference.
-*/
-   if (page_ref_count(page) == 1) {
-   dma_addr &= PAGE_MASK;
-   dma_unmap_page_attrs(>pdev->dev, dma_ad

[PATCH net v3] net: thunderx: Fix TCP/UDP checksum offload for IPv6 pkts

[Aleksey Makarov]

From: Sunil Goutham <sgout...@cavium.com>

Don't offload IP header checksum to NIC.

This fixes a previous patch which enabled checksum offloading
for both IPv4 and IPv6 packets.  So L3 checksum offload was
getting enabled for IPv6 pkts.  And HW is dropping these pkts
as it assumes the pkt is IPv4 when IP csum offload is set
in the SQ descriptor.

Fixes:  3a9024f52c2e ("net: thunderx: Enable TSO and checksum offloads for 
ipv6")
Signed-off-by: Sunil Goutham <sgout...@cavium.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@auriga.com>
Reviewed-by: Eric Dumazet <eduma...@google.com>
---
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c | 1 -
 1 file changed, 1 deletion(-)

v3:
- fix the Fixes: tag (David Miller)
- add Reviewed-by Eric Dumazet

v2: https://lkml.kernel.org/r/20171123144158.26111-1-aleksey.maka...@auriga.com
- don't enable checksum offloading both for IPv4 and IPv6 (Eric Dumazet)

v1: https://lkml.kernel.org/r/20171122123727.23580-1-aleksey.maka...@auriga.com


diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c 
b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
index d4496e9afcdf..8b2c31e2a2b0 100644
--- a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
+++ b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
@@ -1355,7 +1355,6 @@ nicvf_sq_add_hdr_subdesc(struct nicvf *nic, struct 
snd_queue *sq, int qentry,
 
/* Offload checksum calculation to HW */
if (skb->ip_summed == CHECKSUM_PARTIAL) {
-   hdr->csum_l3 = 1; /* Enable IP csum calculation */
hdr->l3_offset = skb_network_offset(skb);
hdr->l4_offset = skb_transport_offset(skb);
 
-- 
2.15.0

[PATCH net v3] net: thunderx: Fix TCP/UDP checksum offload for IPv6 pkts

[Aleksey Makarov]

From: Sunil Goutham 

Don't offload IP header checksum to NIC.

This fixes a previous patch which enabled checksum offloading
for both IPv4 and IPv6 packets.  So L3 checksum offload was
getting enabled for IPv6 pkts.  And HW is dropping these pkts
as it assumes the pkt is IPv4 when IP csum offload is set
in the SQ descriptor.

Fixes:  3a9024f52c2e ("net: thunderx: Enable TSO and checksum offloads for 
ipv6")
Signed-off-by: Sunil Goutham 
Signed-off-by: Aleksey Makarov 
Reviewed-by: Eric Dumazet 
---
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c | 1 -
 1 file changed, 1 deletion(-)

v3:
- fix the Fixes: tag (David Miller)
- add Reviewed-by Eric Dumazet

v2: https://lkml.kernel.org/r/20171123144158.26111-1-aleksey.maka...@auriga.com
- don't enable checksum offloading both for IPv4 and IPv6 (Eric Dumazet)

v1: https://lkml.kernel.org/r/20171122123727.23580-1-aleksey.maka...@auriga.com


diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c 
b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
index d4496e9afcdf..8b2c31e2a2b0 100644
--- a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
+++ b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
@@ -1355,7 +1355,6 @@ nicvf_sq_add_hdr_subdesc(struct nicvf *nic, struct 
snd_queue *sq, int qentry,
 
/* Offload checksum calculation to HW */
if (skb->ip_summed == CHECKSUM_PARTIAL) {
-   hdr->csum_l3 = 1; /* Enable IP csum calculation */
hdr->l3_offset = skb_network_offset(skb);
hdr->l4_offset = skb_transport_offset(skb);
 
-- 
2.15.0

[PATCH net v3] net: thunderx: Fix TCP/UDP checksum offload for IPv6 pkts

[Aleksey Makarov]

From: Sunil Goutham <sgout...@cavium.com>

Don't offload IP header checksum to NIC.

This fixes a previous patch which enabled checksum offloading
for both IPv4 and IPv6 packets.  So L3 checksum offload was
getting enabled for IPv6 pkts.  And HW is dropping these pkts
as it assumes the pkt is IPv4 when IP csum offload is set
in the SQ descriptor.

Fixes:  3a9024f52c2e ("net: thunderx: Enable TSO and checksum offloads for 
ipv6")
Signed-off-by: Sunil Goutham <sgout...@cavium.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@auriga.com>
---
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c | 1 -
 1 file changed, 1 deletion(-)

v3:
- fix the Fixes: tag (David Miller)

v2: https://lkml.kernel.org/r/20171123144158.26111-1-aleksey.maka...@auriga.com
- don't enable checksum offloading both for IPv4 and IPv6 (Eric Dumazet)

v1: https://lkml.kernel.org/r/20171122123727.23580-1-aleksey.maka...@auriga.com


diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c 
b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
index d4496e9afcdf..8b2c31e2a2b0 100644
--- a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
+++ b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
@@ -1355,7 +1355,6 @@ nicvf_sq_add_hdr_subdesc(struct nicvf *nic, struct 
snd_queue *sq, int qentry,
 
/* Offload checksum calculation to HW */
if (skb->ip_summed == CHECKSUM_PARTIAL) {
-   hdr->csum_l3 = 1; /* Enable IP csum calculation */
hdr->l3_offset = skb_network_offset(skb);
hdr->l4_offset = skb_transport_offset(skb);
 
-- 
2.15.0

[PATCH net v3] net: thunderx: Fix TCP/UDP checksum offload for IPv6 pkts

[Aleksey Makarov]

From: Sunil Goutham 

Don't offload IP header checksum to NIC.

This fixes a previous patch which enabled checksum offloading
for both IPv4 and IPv6 packets.  So L3 checksum offload was
getting enabled for IPv6 pkts.  And HW is dropping these pkts
as it assumes the pkt is IPv4 when IP csum offload is set
in the SQ descriptor.

Fixes:  3a9024f52c2e ("net: thunderx: Enable TSO and checksum offloads for 
ipv6")
Signed-off-by: Sunil Goutham 
Signed-off-by: Aleksey Makarov 
---
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c | 1 -
 1 file changed, 1 deletion(-)

v3:
- fix the Fixes: tag (David Miller)

v2: https://lkml.kernel.org/r/20171123144158.26111-1-aleksey.maka...@auriga.com
- don't enable checksum offloading both for IPv4 and IPv6 (Eric Dumazet)

v1: https://lkml.kernel.org/r/20171122123727.23580-1-aleksey.maka...@auriga.com


diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c 
b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
index d4496e9afcdf..8b2c31e2a2b0 100644
--- a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
+++ b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
@@ -1355,7 +1355,6 @@ nicvf_sq_add_hdr_subdesc(struct nicvf *nic, struct 
snd_queue *sq, int qentry,
 
/* Offload checksum calculation to HW */
if (skb->ip_summed == CHECKSUM_PARTIAL) {
-   hdr->csum_l3 = 1; /* Enable IP csum calculation */
hdr->l3_offset = skb_network_offset(skb);
hdr->l4_offset = skb_transport_offset(skb);
 
-- 
2.15.0

[PATCH net v2] net: thunderx: Fix TCP/UDP checksum offload for IPv6 pkts

[Aleksey Makarov]

From: Sunil Goutham <sgout...@cavium.com>

Don't offload IP header checksum to NIC.

This fixes a previous patch which enabled checksum offloading
for both IPv4 and IPv6 packets.  So L3 checksum offload was
getting enabled for IPv6 pkts.  And HW is dropping these pkts
as it assumes the pkt is IPv4 when IP csum offload is set
in the SQ descriptor.

Fixes: 494fd005 ("net: thunderx: Enable TSO and checksum offloads for ipv6")
Signed-off-by: Sunil Goutham <sgout...@cavium.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@auriga.com>
---
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c | 1 -
 1 file changed, 1 deletion(-)

v2:
- Don't enable checksum offloading both for IPv4 and IPv6 (Eric Dumazet)

v1:
  https://lkml.kernel.org/r/20171122123727.23580-1-aleksey.maka...@auriga.com

diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c 
b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
index d4496e9afcdf..8b2c31e2a2b0 100644
--- a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
+++ b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
@@ -1355,7 +1355,6 @@ nicvf_sq_add_hdr_subdesc(struct nicvf *nic, struct 
snd_queue *sq, int qentry,
 
/* Offload checksum calculation to HW */
if (skb->ip_summed == CHECKSUM_PARTIAL) {
-   hdr->csum_l3 = 1; /* Enable IP csum calculation */
hdr->l3_offset = skb_network_offset(skb);
hdr->l4_offset = skb_transport_offset(skb);
 
-- 
2.15.0

[PATCH net v2] net: thunderx: Fix TCP/UDP checksum offload for IPv6 pkts

[Aleksey Makarov]

From: Sunil Goutham 

Don't offload IP header checksum to NIC.

This fixes a previous patch which enabled checksum offloading
for both IPv4 and IPv6 packets.  So L3 checksum offload was
getting enabled for IPv6 pkts.  And HW is dropping these pkts
as it assumes the pkt is IPv4 when IP csum offload is set
in the SQ descriptor.

Fixes: 494fd005 ("net: thunderx: Enable TSO and checksum offloads for ipv6")
Signed-off-by: Sunil Goutham 
Signed-off-by: Aleksey Makarov 
---
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c | 1 -
 1 file changed, 1 deletion(-)

v2:
- Don't enable checksum offloading both for IPv4 and IPv6 (Eric Dumazet)

v1:
  https://lkml.kernel.org/r/20171122123727.23580-1-aleksey.maka...@auriga.com

diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c 
b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
index d4496e9afcdf..8b2c31e2a2b0 100644
--- a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
+++ b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
@@ -1355,7 +1355,6 @@ nicvf_sq_add_hdr_subdesc(struct nicvf *nic, struct 
snd_queue *sq, int qentry,
 
/* Offload checksum calculation to HW */
if (skb->ip_summed == CHECKSUM_PARTIAL) {
-   hdr->csum_l3 = 1; /* Enable IP csum calculation */
hdr->l3_offset = skb_network_offset(skb);
hdr->l4_offset = skb_transport_offset(skb);
 
-- 
2.15.0

[PATCH net] net: thunderx: Fix TCP/UDP checksum offload for IPv6 pkts

[Aleksey Makarov]

From: Sunil Goutham <sgout...@cavium.com>

This fixes a previous patch which missed some changes
and due to which L3 checksum offload was getting enabled
for IPv6 pkts. And HW is dropping these pkts as it assumes
the pkt is IPv4 when IP csum offload is set in the SQ
descriptor.

Fixes: 494fd005 ("net: thunderx: Enable TSO and checksum offloads for ipv6")
Signed-off-by: Sunil Goutham <sgout...@cavium.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@auriga.com>
---
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c 
b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
index d4496e9afcdf..184d5bdbe7e0 100644
--- a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
+++ b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
@@ -1355,10 +1355,11 @@ nicvf_sq_add_hdr_subdesc(struct nicvf *nic, struct 
snd_queue *sq, int qentry,
 
/* Offload checksum calculation to HW */
if (skb->ip_summed == CHECKSUM_PARTIAL) {
-   hdr->csum_l3 = 1; /* Enable IP csum calculation */
hdr->l3_offset = skb_network_offset(skb);
hdr->l4_offset = skb_transport_offset(skb);
 
+   /* Enable IP HDR csum calculation for V4 pkts */
+   hdr->csum_l3 = (ip.v4->version == 4) ? 1 : 0;
proto = (ip.v4->version == 4) ? ip.v4->protocol :
ip.v6->nexthdr;
 
-- 
2.15.0

[PATCH net] net: thunderx: Fix TCP/UDP checksum offload for IPv6 pkts

[Aleksey Makarov]

From: Sunil Goutham 

This fixes a previous patch which missed some changes
and due to which L3 checksum offload was getting enabled
for IPv6 pkts. And HW is dropping these pkts as it assumes
the pkt is IPv4 when IP csum offload is set in the SQ
descriptor.

Fixes: 494fd005 ("net: thunderx: Enable TSO and checksum offloads for ipv6")
Signed-off-by: Sunil Goutham 
Signed-off-by: Aleksey Makarov 
---
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c 
b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
index d4496e9afcdf..184d5bdbe7e0 100644
--- a/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
+++ b/drivers/net/ethernet/cavium/thunder/nicvf_queues.c
@@ -1355,10 +1355,11 @@ nicvf_sq_add_hdr_subdesc(struct nicvf *nic, struct 
snd_queue *sq, int qentry,
 
/* Offload checksum calculation to HW */
if (skb->ip_summed == CHECKSUM_PARTIAL) {
-   hdr->csum_l3 = 1; /* Enable IP csum calculation */
hdr->l3_offset = skb_network_offset(skb);
hdr->l4_offset = skb_transport_offset(skb);
 
+   /* Enable IP HDR csum calculation for V4 pkts */
+   hdr->csum_l3 = (ip.v4->version == 4) ? 1 : 0;
proto = (ip.v4->version == 4) ? ip.v4->protocol :
ip.v6->nexthdr;
 
-- 
2.15.0

[PATCH net-next v2 2/2] net: thunderx: add timestamping support

[Aleksey Makarov]

From: Sunil Goutham <sgout...@cavium.com>

This adds timestamping support for both receive and transmit
paths. On the receive side no filters are supported i.e either
all pkts will get a timestamp appended infront of the packet or none.
On the transmit side HW doesn't support timestamp insertion but
only generates a separate CQE with transmitted packet's timestamp.
Also HW supports only one packet at a time for timestamping on the
transmit side.

Signed-off-by: Sunil Goutham <sgout...@cavium.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@cavium.com>
---
 drivers/net/ethernet/cavium/Kconfig|   1 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  15 ++
 drivers/net/ethernet/cavium/thunder/nic_main.c |  58 ++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +++-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 173 -
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 9 files changed, 330 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index fabe0ffcc2d4..6d003cbe197c 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -27,6 +27,7 @@ config THUNDER_NIC_PF
 
 config THUNDER_NIC_VF
tristate "Thunder Virtual function driver"
+   select CAVIUM_PTP
depends on 64BIT
---help---
  This driver supports Thunder's NIC virtual function
diff --git a/drivers/net/ethernet/cavium/thunder/nic.h 
b/drivers/net/ethernet/cavium/thunder/nic.h
index 4a02e618e318..204b234beb9d 100644
--- a/drivers/net/ethernet/cavium/thunder/nic.h
+++ b/drivers/net/ethernet/cavium/thunder/nic.h
@@ -263,6 +263,8 @@ struct nicvf_drv_stats {
struct u64_stats_sync   syncp;
 };
 
+struct cavium_ptp;
+
 struct nicvf {
struct nicvf*pnicvf;
struct net_device   *netdev;
@@ -312,6 +314,12 @@ struct nicvf {
struct tasklet_struct   qs_err_task;
struct work_struct  reset_task;
 
+   /* PTP timestamp */
+   struct cavium_ptp   *ptp_clock;
+   boolhw_rx_tstamp;
+   struct sk_buff  *ptp_skb;
+   atomic_ttx_ptp_skbs;
+
/* Interrupt coalescing settings */
u32 cq_coalesce_usecs;
u32 msg_enable;
@@ -371,6 +379,7 @@ struct nicvf {
 #defineNIC_MBOX_MSG_LOOPBACK   0x16/* Set interface in 
loopback */
 #defineNIC_MBOX_MSG_RESET_STAT_COUNTER 0x17/* Reset statistics 
counters */
 #defineNIC_MBOX_MSG_PFC0x18/* Pause frame control 
*/
+#defineNIC_MBOX_MSG_PTP_CFG0x19/* HW packet timestamp 
*/
 #defineNIC_MBOX_MSG_CFG_DONE   0xF0/* VF configuration 
done */
 #defineNIC_MBOX_MSG_SHUTDOWN   0xF1/* VF is being shutdown 
*/
 
@@ -521,6 +530,11 @@ struct pfc {
u8fc_tx;
 };
 
+struct set_ptp {
+   u8msg;
+   bool  enable;
+};
+
 /* 128 bit shared memory between PF and each VF */
 union nic_mbx {
struct { u8 msg; }  msg;
@@ -540,6 +554,7 @@ union nic_mbx {
struct set_loopback lbk;
struct reset_stat_cfg   reset_stat;
struct pfc  pfc;
+   struct set_ptp  ptp;
 };
 
 #define NIC_NODE_ID_MASK   0x03
diff --git a/drivers/net/ethernet/cavium/thunder/nic_main.c 
b/drivers/net/ethernet/cavium/thunder/nic_main.c
index 8f1dd55b3e08..4c1c5414a162 100644
--- a/drivers/net/ethernet/cavium/thunder/nic_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nic_main.c
@@ -426,13 +426,22 @@ static void nic_init_hw(struct nicpf *nic)
/* Enable backpressure */
nic_reg_write(nic, NIC_PF_BP_CFG, (1ULL << 6) | 0x03);
 
-   /* TNS and TNS bypass modes are present only on 88xx */
+   /* TNS and TNS bypass modes are present only on 88xx
+* Also offset of this CSR has changed in 81xx and 83xx.
+*/
if (nic->pdev->subsystem_device == PCI_SUBSYS_DEVID_88XX_NIC_PF) {
/* Disable TNS mode on both interfaces */
nic_reg_write(nic, NIC_PF_INTF_0_1_SEND_CFG,
- (NIC_TNS_BYPASS_MODE << 7) | BGX0_BLOCK);
+ (NIC_TNS_BYPASS_MODE << 7) |
+ BGX0_BLOCK | (1ULL << 16));
nic_reg_write(nic, NIC_PF_INTF_0_1_SEND_CFG | (1 << 8),
- (NIC_TNS_BYPASS_MODE << 7) | BGX1_BLOCK);
+ (NIC_TNS_BYPASS_MODE << 7) |
+ BGX1_BLOCK | (1ULL << 16));
+   } else {
+   /* Configure timestamp

[PATCH net-next v2 2/2] net: thunderx: add timestamping support

[Aleksey Makarov]

From: Sunil Goutham 

This adds timestamping support for both receive and transmit
paths. On the receive side no filters are supported i.e either
all pkts will get a timestamp appended infront of the packet or none.
On the transmit side HW doesn't support timestamp insertion but
only generates a separate CQE with transmitted packet's timestamp.
Also HW supports only one packet at a time for timestamping on the
transmit side.

Signed-off-by: Sunil Goutham 
Signed-off-by: Aleksey Makarov 
---
 drivers/net/ethernet/cavium/Kconfig|   1 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  15 ++
 drivers/net/ethernet/cavium/thunder/nic_main.c |  58 ++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +++-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 173 -
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 9 files changed, 330 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index fabe0ffcc2d4..6d003cbe197c 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -27,6 +27,7 @@ config THUNDER_NIC_PF
 
 config THUNDER_NIC_VF
tristate "Thunder Virtual function driver"
+   select CAVIUM_PTP
depends on 64BIT
---help---
  This driver supports Thunder's NIC virtual function
diff --git a/drivers/net/ethernet/cavium/thunder/nic.h 
b/drivers/net/ethernet/cavium/thunder/nic.h
index 4a02e618e318..204b234beb9d 100644
--- a/drivers/net/ethernet/cavium/thunder/nic.h
+++ b/drivers/net/ethernet/cavium/thunder/nic.h
@@ -263,6 +263,8 @@ struct nicvf_drv_stats {
struct u64_stats_sync   syncp;
 };
 
+struct cavium_ptp;
+
 struct nicvf {
struct nicvf*pnicvf;
struct net_device   *netdev;
@@ -312,6 +314,12 @@ struct nicvf {
struct tasklet_struct   qs_err_task;
struct work_struct  reset_task;
 
+   /* PTP timestamp */
+   struct cavium_ptp   *ptp_clock;
+   boolhw_rx_tstamp;
+   struct sk_buff  *ptp_skb;
+   atomic_ttx_ptp_skbs;
+
/* Interrupt coalescing settings */
u32 cq_coalesce_usecs;
u32 msg_enable;
@@ -371,6 +379,7 @@ struct nicvf {
 #defineNIC_MBOX_MSG_LOOPBACK   0x16/* Set interface in 
loopback */
 #defineNIC_MBOX_MSG_RESET_STAT_COUNTER 0x17/* Reset statistics 
counters */
 #defineNIC_MBOX_MSG_PFC0x18/* Pause frame control 
*/
+#defineNIC_MBOX_MSG_PTP_CFG0x19/* HW packet timestamp 
*/
 #defineNIC_MBOX_MSG_CFG_DONE   0xF0/* VF configuration 
done */
 #defineNIC_MBOX_MSG_SHUTDOWN   0xF1/* VF is being shutdown 
*/
 
@@ -521,6 +530,11 @@ struct pfc {
u8fc_tx;
 };
 
+struct set_ptp {
+   u8msg;
+   bool  enable;
+};
+
 /* 128 bit shared memory between PF and each VF */
 union nic_mbx {
struct { u8 msg; }  msg;
@@ -540,6 +554,7 @@ union nic_mbx {
struct set_loopback lbk;
struct reset_stat_cfg   reset_stat;
struct pfc  pfc;
+   struct set_ptp  ptp;
 };
 
 #define NIC_NODE_ID_MASK   0x03
diff --git a/drivers/net/ethernet/cavium/thunder/nic_main.c 
b/drivers/net/ethernet/cavium/thunder/nic_main.c
index 8f1dd55b3e08..4c1c5414a162 100644
--- a/drivers/net/ethernet/cavium/thunder/nic_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nic_main.c
@@ -426,13 +426,22 @@ static void nic_init_hw(struct nicpf *nic)
/* Enable backpressure */
nic_reg_write(nic, NIC_PF_BP_CFG, (1ULL << 6) | 0x03);
 
-   /* TNS and TNS bypass modes are present only on 88xx */
+   /* TNS and TNS bypass modes are present only on 88xx
+* Also offset of this CSR has changed in 81xx and 83xx.
+*/
if (nic->pdev->subsystem_device == PCI_SUBSYS_DEVID_88XX_NIC_PF) {
/* Disable TNS mode on both interfaces */
nic_reg_write(nic, NIC_PF_INTF_0_1_SEND_CFG,
- (NIC_TNS_BYPASS_MODE << 7) | BGX0_BLOCK);
+ (NIC_TNS_BYPASS_MODE << 7) |
+ BGX0_BLOCK | (1ULL << 16));
nic_reg_write(nic, NIC_PF_INTF_0_1_SEND_CFG | (1 << 8),
- (NIC_TNS_BYPASS_MODE << 7) | BGX1_BLOCK);
+ (NIC_TNS_BYPASS_MODE << 7) |
+ BGX1_BLOCK | (1ULL << 16));
+   } else {
+   /* Configure timestamp generation timeout to 10us */
+   for (i = 0; i < nic->hw->bgx_cnt; i

[PATCH net-next v2 1/2] net: add support for Cavium PTP coprocessor

[Aleksey Makarov]

From: Radoslaw Biernacki <r...@semihalf.com>

This patch adds support for the Precision Time Protocol
Clocks and Timestamping hardware found on Cavium ThunderX
processors.

Signed-off-by: Radoslaw Biernacki <r...@semihalf.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@cavium.com>
---
 drivers/net/ethernet/cavium/Kconfig |  13 +
 drivers/net/ethernet/cavium/Makefile|   1 +
 drivers/net/ethernet/cavium/common/Makefile |   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c | 334 
 drivers/net/ethernet/cavium/common/cavium_ptp.h |  78 ++
 5 files changed, 427 insertions(+)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index 63be75eb34d2..fabe0ffcc2d4 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -50,6 +50,19 @@ config   THUNDER_NIC_RGX
  This driver supports configuring XCV block of RGX interface
  present on CN81XX chip.
 
+config CAVIUM_PTP
+   tristate "Cavium PTP coprocessor as PTP clock"
+   depends on 64BIT
+   depends on PTP_1588_CLOCK
+   select CAVIUM_RST
+   default y
+   ---help---
+ This driver adds support for the Precision Time Protocol Clocks and
+ Timestamping coprocessor (PTP) found on Cavium processors.
+ PTP provides timestamping mechanism that is suitable for use in IEEE 
1588
+ Precision Time Protocol or other purposes.  Timestamps can be used in
+ BGX, TNS, GTI, and NIC blocks.
+
 config LIQUIDIO
tristate "Cavium LiquidIO support"
depends on 64BIT
diff --git a/drivers/net/ethernet/cavium/Makefile 
b/drivers/net/ethernet/cavium/Makefile
index 872da9f7c31a..946bba84e81d 100644
--- a/drivers/net/ethernet/cavium/Makefile
+++ b/drivers/net/ethernet/cavium/Makefile
@@ -1,6 +1,7 @@
 #
 # Makefile for the Cavium ethernet device drivers.
 #
+obj-$(CONFIG_NET_VENDOR_CAVIUM) += common/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += thunder/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += liquidio/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += octeon/
diff --git a/drivers/net/ethernet/cavium/common/Makefile 
b/drivers/net/ethernet/cavium/common/Makefile
new file mode 100644
index ..dd8561b8060b
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_CAVIUM_PTP) += cavium_ptp.o
diff --git a/drivers/net/ethernet/cavium/common/cavium_ptp.c 
b/drivers/net/ethernet/cavium/common/cavium_ptp.c
new file mode 100644
index ..f4c738db27fd
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/cavium_ptp.c
@@ -0,0 +1,334 @@
+/*
+ * cavium_ptp.c - PTP 1588 clock on Cavium hardware
+ *
+ * Copyright (c) 2003-2015, 2017 Cavium, Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License.
+ *
+ * This file may also be available under a different license from Cavium.
+ * Contact Cavium, Inc. for more information
+ */
+
+#include 
+#include 
+#include 
+#include 
+
+#include "cavium_ptp.h"
+
+#define DRV_NAME   "Cavium PTP Driver"
+
+#define PCI_DEVICE_ID_CAVIUM_PTP   0xA00C
+#define PCI_DEVICE_ID_CAVIUM_RST   0xA00E
+
+#define PCI_PTP_BAR_NO 0
+#define PCI_RST_BAR_NO 0
+
+#define PTP_CLOCK_CFG  0xF00ULL
+#define  PTP_CLOCK_CFG_PTP_EN  BIT(0)
+#define PTP_CLOCK_LO   0xF08ULL
+#define PTP_CLOCK_HI   0xF10ULL
+#define PTP_CLOCK_COMP 0xF18ULL
+
+#define RST_BOOT   0x1600ULL
+#define CLOCK_BASE_RATE5000ULL
+
+static u64 ptp_cavium_clock_get(void)
+{
+   struct pci_dev *pdev;
+   void __iomem *base;
+   u64 ret = CLOCK_BASE_RATE * 16;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_RST, NULL);
+   if (!pdev)
+   goto error;
+
+   base = pci_ioremap_bar(pdev, PCI_RST_BAR_NO);
+   if (!base)
+   goto error_put_pdev;
+
+   ret = CLOCK_BASE_RATE * ((readq(base + RST_BOOT) >> 33) & 0x3f);
+
+   iounmap(base);
+
+error_put_pdev:
+   pci_dev_put(pdev);
+
+error:
+   return ret;
+}
+
+struct cavium_ptp *cavium_ptp_get(void)
+{
+   struct cavium_ptp *ptp;
+   struct pci_dev *pdev;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_PTP, NULL);
+   if (!pdev)
+   return ERR_PTR(-ENODEV);
+
+   ptp = pci_get_drvdata(pdev);
+   if (!ptp) {
+   pci_dev_put(pdev);
+   ptp = ERR_PTR(-EPROBE_DEFER);
+   }
+
+   return ptp;
+}
+EXPORT_S

[PATCH net-next v2 0/2] net: thunderx: add support for PTP clock

[Aleksey Makarov]

This series adds support for IEEE 1588 Precision Time Protocol
to Cavium ethernet driver.

The first patch adds support for the Precision Time Protocol Clocks and
Timestamping coprocessor (PTP) found on Cavium processors.
It registers a new PTP clock in the PTP core and provides functions
to use the counter in BGX, TNS, GTI, and NIC blocks.

The second patch introduces support for the PTP protocol to the
Cavium ThunderX ethernet driver.

v2:
- use readq()/writeq() in place of cavium_ptp_reg_read()/cavium_ptp_reg_write(),
  don't use readq_relaxed()/writeq_relaxed() (David Daney)

v1:
  https://lkml.kernel.org/r/20171107190704.15458-1-aleksey.maka...@cavium.com

Radoslaw Biernacki (1):
  net: add support for Cavium PTP coprocessor

Sunil Goutham (1):
  net: thunderx: add timestamping support

 drivers/net/ethernet/cavium/Kconfig|  14 +
 drivers/net/ethernet/cavium/Makefile   |   1 +
 drivers/net/ethernet/cavium/common/Makefile|   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c| 334 +
 drivers/net/ethernet/cavium/common/cavium_ptp.h|  78 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  15 +
 drivers/net/ethernet/cavium/thunder/nic_main.c |  58 +++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 173 ++-
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 13 files changed, 757 insertions(+), 6 deletions(-)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

-- 
2.15.0

[PATCH net-next v2 1/2] net: add support for Cavium PTP coprocessor

[Aleksey Makarov]

From: Radoslaw Biernacki 

This patch adds support for the Precision Time Protocol
Clocks and Timestamping hardware found on Cavium ThunderX
processors.

Signed-off-by: Radoslaw Biernacki 
Signed-off-by: Aleksey Makarov 
---
 drivers/net/ethernet/cavium/Kconfig |  13 +
 drivers/net/ethernet/cavium/Makefile|   1 +
 drivers/net/ethernet/cavium/common/Makefile |   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c | 334 
 drivers/net/ethernet/cavium/common/cavium_ptp.h |  78 ++
 5 files changed, 427 insertions(+)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index 63be75eb34d2..fabe0ffcc2d4 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -50,6 +50,19 @@ config   THUNDER_NIC_RGX
  This driver supports configuring XCV block of RGX interface
  present on CN81XX chip.
 
+config CAVIUM_PTP
+   tristate "Cavium PTP coprocessor as PTP clock"
+   depends on 64BIT
+   depends on PTP_1588_CLOCK
+   select CAVIUM_RST
+   default y
+   ---help---
+ This driver adds support for the Precision Time Protocol Clocks and
+ Timestamping coprocessor (PTP) found on Cavium processors.
+ PTP provides timestamping mechanism that is suitable for use in IEEE 
1588
+ Precision Time Protocol or other purposes.  Timestamps can be used in
+ BGX, TNS, GTI, and NIC blocks.
+
 config LIQUIDIO
tristate "Cavium LiquidIO support"
depends on 64BIT
diff --git a/drivers/net/ethernet/cavium/Makefile 
b/drivers/net/ethernet/cavium/Makefile
index 872da9f7c31a..946bba84e81d 100644
--- a/drivers/net/ethernet/cavium/Makefile
+++ b/drivers/net/ethernet/cavium/Makefile
@@ -1,6 +1,7 @@
 #
 # Makefile for the Cavium ethernet device drivers.
 #
+obj-$(CONFIG_NET_VENDOR_CAVIUM) += common/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += thunder/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += liquidio/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += octeon/
diff --git a/drivers/net/ethernet/cavium/common/Makefile 
b/drivers/net/ethernet/cavium/common/Makefile
new file mode 100644
index ..dd8561b8060b
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_CAVIUM_PTP) += cavium_ptp.o
diff --git a/drivers/net/ethernet/cavium/common/cavium_ptp.c 
b/drivers/net/ethernet/cavium/common/cavium_ptp.c
new file mode 100644
index ..f4c738db27fd
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/cavium_ptp.c
@@ -0,0 +1,334 @@
+/*
+ * cavium_ptp.c - PTP 1588 clock on Cavium hardware
+ *
+ * Copyright (c) 2003-2015, 2017 Cavium, Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License.
+ *
+ * This file may also be available under a different license from Cavium.
+ * Contact Cavium, Inc. for more information
+ */
+
+#include 
+#include 
+#include 
+#include 
+
+#include "cavium_ptp.h"
+
+#define DRV_NAME   "Cavium PTP Driver"
+
+#define PCI_DEVICE_ID_CAVIUM_PTP   0xA00C
+#define PCI_DEVICE_ID_CAVIUM_RST   0xA00E
+
+#define PCI_PTP_BAR_NO 0
+#define PCI_RST_BAR_NO 0
+
+#define PTP_CLOCK_CFG  0xF00ULL
+#define  PTP_CLOCK_CFG_PTP_EN  BIT(0)
+#define PTP_CLOCK_LO   0xF08ULL
+#define PTP_CLOCK_HI   0xF10ULL
+#define PTP_CLOCK_COMP 0xF18ULL
+
+#define RST_BOOT   0x1600ULL
+#define CLOCK_BASE_RATE5000ULL
+
+static u64 ptp_cavium_clock_get(void)
+{
+   struct pci_dev *pdev;
+   void __iomem *base;
+   u64 ret = CLOCK_BASE_RATE * 16;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_RST, NULL);
+   if (!pdev)
+   goto error;
+
+   base = pci_ioremap_bar(pdev, PCI_RST_BAR_NO);
+   if (!base)
+   goto error_put_pdev;
+
+   ret = CLOCK_BASE_RATE * ((readq(base + RST_BOOT) >> 33) & 0x3f);
+
+   iounmap(base);
+
+error_put_pdev:
+   pci_dev_put(pdev);
+
+error:
+   return ret;
+}
+
+struct cavium_ptp *cavium_ptp_get(void)
+{
+   struct cavium_ptp *ptp;
+   struct pci_dev *pdev;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_PTP, NULL);
+   if (!pdev)
+   return ERR_PTR(-ENODEV);
+
+   ptp = pci_get_drvdata(pdev);
+   if (!ptp) {
+   pci_dev_put(pdev);
+   ptp = ERR_PTR(-EPROBE_DEFER);
+   }
+
+   return ptp;
+}
+EXPORT_SYMBOL(cavium_ptp_get);
+
+void cavium_ptp_put(struct cavium_ptp *ptp)
+{
+   pci_de

[PATCH net-next v2 0/2] net: thunderx: add support for PTP clock

[Aleksey Makarov]

This series adds support for IEEE 1588 Precision Time Protocol
to Cavium ethernet driver.

The first patch adds support for the Precision Time Protocol Clocks and
Timestamping coprocessor (PTP) found on Cavium processors.
It registers a new PTP clock in the PTP core and provides functions
to use the counter in BGX, TNS, GTI, and NIC blocks.

The second patch introduces support for the PTP protocol to the
Cavium ThunderX ethernet driver.

v2:
- use readq()/writeq() in place of cavium_ptp_reg_read()/cavium_ptp_reg_write(),
  don't use readq_relaxed()/writeq_relaxed() (David Daney)

v1:
  https://lkml.kernel.org/r/20171107190704.15458-1-aleksey.maka...@cavium.com

Radoslaw Biernacki (1):
  net: add support for Cavium PTP coprocessor

Sunil Goutham (1):
  net: thunderx: add timestamping support

 drivers/net/ethernet/cavium/Kconfig|  14 +
 drivers/net/ethernet/cavium/Makefile   |   1 +
 drivers/net/ethernet/cavium/common/Makefile|   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c| 334 +
 drivers/net/ethernet/cavium/common/cavium_ptp.h|  78 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  15 +
 drivers/net/ethernet/cavium/thunder/nic_main.c |  58 +++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 173 ++-
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 13 files changed, 757 insertions(+), 6 deletions(-)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

-- 
2.15.0

[PATCH net-next] net: thunderx: fix double free error

[Aleksey Makarov]

This patch fixes an error in memory allocation/freeing in
ThunderX PF driver.

I moved the allocation to the probe() function and made it managed.

>From the Colin's email:

While running static analysis on linux-next with CoverityScan I found 3
double free errors in the Cavium thunder driver.

The issue occurs on the err_disable_device: label of function nic_probe
when nic_free_lmacmem(nic) is called and a double free occurs on
nic->duplex, nic->link and nic->speed.  This occurs when nic_init_hw()
fails:

/* Initialize hardware */
err = nic_init_hw(nic);
if (err)
goto err_release_regions;

nic_init_hw() calls nic_get_hw_info() and this calls nic_free_lmacmem()
if any of the allocations fail. This free'ing occurs again by the call
to nic_free_lmacmem() on the err_release_regions exit path in nic_probe().

Reported-by: Colin Ian King <colin.k...@canonical.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@cavium.com>
---
 drivers/net/ethernet/cavium/thunder/nic_main.c | 82 ++
 1 file changed, 30 insertions(+), 52 deletions(-)

diff --git a/drivers/net/ethernet/cavium/thunder/nic_main.c 
b/drivers/net/ethernet/cavium/thunder/nic_main.c
index c12ab7f1adb9..26309e209217 100644
--- a/drivers/net/ethernet/cavium/thunder/nic_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nic_main.c
@@ -361,17 +361,8 @@ static void nic_set_lmac_vf_mapping(struct nicpf *nic)
}
 }
 
-static void nic_free_lmacmem(struct nicpf *nic)
+static void nic_get_hw_info(struct nicpf *nic)
 {
-   kfree(nic->vf_lmac_map);
-   kfree(nic->link);
-   kfree(nic->duplex);
-   kfree(nic->speed);
-}
-
-static int nic_get_hw_info(struct nicpf *nic)
-{
-   u8 max_lmac;
u16 sdevid;
struct hw_info *hw = nic->hw;
 
@@ -419,41 +410,16 @@ static int nic_get_hw_info(struct nicpf *nic)
break;
}
hw->tl4_cnt = MAX_QUEUES_PER_QSET * pci_sriov_get_totalvfs(nic->pdev);
-
-   /* Allocate memory for LMAC tracking elements */
-   max_lmac = hw->bgx_cnt * MAX_LMAC_PER_BGX;
-   nic->vf_lmac_map = kmalloc_array(max_lmac, sizeof(u8), GFP_KERNEL);
-   if (!nic->vf_lmac_map)
-   goto error;
-   nic->link = kmalloc_array(max_lmac, sizeof(u8), GFP_KERNEL);
-   if (!nic->link)
-   goto error;
-   nic->duplex = kmalloc_array(max_lmac, sizeof(u8), GFP_KERNEL);
-   if (!nic->duplex)
-   goto error;
-   nic->speed = kmalloc_array(max_lmac, sizeof(u32), GFP_KERNEL);
-   if (!nic->speed)
-   goto error;
-   return 0;
-
-error:
-   nic_free_lmacmem(nic);
-   return -ENOMEM;
 }
 
 #define BGX0_BLOCK 8
 #define BGX1_BLOCK 9
 
-static int nic_init_hw(struct nicpf *nic)
+static void nic_init_hw(struct nicpf *nic)
 {
-   int i, err;
+   int i;
u64 cqm_cfg;
 
-   /* Get HW capability info */
-   err = nic_get_hw_info(nic);
-   if (err)
-   return err;
-
/* Enable NIC HW block */
nic_reg_write(nic, NIC_PF_CFG, 0x3);
 
@@ -507,8 +473,6 @@ static int nic_init_hw(struct nicpf *nic)
cqm_cfg = nic_reg_read(nic, NIC_PF_CQM_CFG);
if (cqm_cfg < NICPF_CQM_MIN_DROP_LEVEL)
nic_reg_write(nic, NIC_PF_CQM_CFG, NICPF_CQM_MIN_DROP_LEVEL);
-
-   return 0;
 }
 
 /* Channel parse index configuration */
@@ -1325,6 +1289,7 @@ static int nic_probe(struct pci_dev *pdev, const struct 
pci_device_id *ent)
 {
struct device *dev = >dev;
struct nicpf *nic;
+   u8 max_lmac;
interr;
 
BUILD_BUG_ON(sizeof(union nic_mbx) > 16);
@@ -1334,10 +1299,8 @@ static int nic_probe(struct pci_dev *pdev, const struct 
pci_device_id *ent)
return -ENOMEM;
 
nic->hw = devm_kzalloc(dev, sizeof(struct hw_info), GFP_KERNEL);
-   if (!nic->hw) {
-   devm_kfree(dev, nic);
+   if (!nic->hw)
return -ENOMEM;
-   }
 
pci_set_drvdata(pdev, nic);
 
@@ -1378,11 +1341,33 @@ static int nic_probe(struct pci_dev *pdev, const struct 
pci_device_id *ent)
 
nic->node = nic_get_node_id(pdev);
 
-   /* Initialize hardware */
-   err = nic_init_hw(nic);
-   if (err)
+   /* Get HW capability info */
+   nic_get_hw_info(nic);
+
+   /* Allocate memory for LMAC tracking elements */
+   err = -ENOMEM;
+   max_lmac = nic->hw->bgx_cnt * MAX_LMAC_PER_BGX;
+
+   nic->vf_lmac_map = devm_kmalloc_array(dev, max_lmac, sizeof(u8),
+ GFP_KERNEL);
+   if (!nic->vf_lmac_map)
+   goto err_release_regions;
+
+   nic->link = devm_kmalloc_array(dev, max_lmac, sizeof(u8), GFP_KERNEL);
+   if (!nic->link)
goto err_release_regions;
 
+   nic->duplex = devm_kmalloc_array(dev, max_lmac,

[PATCH net-next] net: thunderx: fix double free error

[Aleksey Makarov]

This patch fixes an error in memory allocation/freeing in
ThunderX PF driver.

I moved the allocation to the probe() function and made it managed.

>From the Colin's email:

While running static analysis on linux-next with CoverityScan I found 3
double free errors in the Cavium thunder driver.

The issue occurs on the err_disable_device: label of function nic_probe
when nic_free_lmacmem(nic) is called and a double free occurs on
nic->duplex, nic->link and nic->speed.  This occurs when nic_init_hw()
fails:

/* Initialize hardware */
err = nic_init_hw(nic);
if (err)
goto err_release_regions;

nic_init_hw() calls nic_get_hw_info() and this calls nic_free_lmacmem()
if any of the allocations fail. This free'ing occurs again by the call
to nic_free_lmacmem() on the err_release_regions exit path in nic_probe().

Reported-by: Colin Ian King 
Signed-off-by: Aleksey Makarov 
---
 drivers/net/ethernet/cavium/thunder/nic_main.c | 82 ++
 1 file changed, 30 insertions(+), 52 deletions(-)

diff --git a/drivers/net/ethernet/cavium/thunder/nic_main.c 
b/drivers/net/ethernet/cavium/thunder/nic_main.c
index c12ab7f1adb9..26309e209217 100644
--- a/drivers/net/ethernet/cavium/thunder/nic_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nic_main.c
@@ -361,17 +361,8 @@ static void nic_set_lmac_vf_mapping(struct nicpf *nic)
}
 }
 
-static void nic_free_lmacmem(struct nicpf *nic)
+static void nic_get_hw_info(struct nicpf *nic)
 {
-   kfree(nic->vf_lmac_map);
-   kfree(nic->link);
-   kfree(nic->duplex);
-   kfree(nic->speed);
-}
-
-static int nic_get_hw_info(struct nicpf *nic)
-{
-   u8 max_lmac;
u16 sdevid;
struct hw_info *hw = nic->hw;
 
@@ -419,41 +410,16 @@ static int nic_get_hw_info(struct nicpf *nic)
break;
}
hw->tl4_cnt = MAX_QUEUES_PER_QSET * pci_sriov_get_totalvfs(nic->pdev);
-
-   /* Allocate memory for LMAC tracking elements */
-   max_lmac = hw->bgx_cnt * MAX_LMAC_PER_BGX;
-   nic->vf_lmac_map = kmalloc_array(max_lmac, sizeof(u8), GFP_KERNEL);
-   if (!nic->vf_lmac_map)
-   goto error;
-   nic->link = kmalloc_array(max_lmac, sizeof(u8), GFP_KERNEL);
-   if (!nic->link)
-   goto error;
-   nic->duplex = kmalloc_array(max_lmac, sizeof(u8), GFP_KERNEL);
-   if (!nic->duplex)
-   goto error;
-   nic->speed = kmalloc_array(max_lmac, sizeof(u32), GFP_KERNEL);
-   if (!nic->speed)
-   goto error;
-   return 0;
-
-error:
-   nic_free_lmacmem(nic);
-   return -ENOMEM;
 }
 
 #define BGX0_BLOCK 8
 #define BGX1_BLOCK 9
 
-static int nic_init_hw(struct nicpf *nic)
+static void nic_init_hw(struct nicpf *nic)
 {
-   int i, err;
+   int i;
u64 cqm_cfg;
 
-   /* Get HW capability info */
-   err = nic_get_hw_info(nic);
-   if (err)
-   return err;
-
/* Enable NIC HW block */
nic_reg_write(nic, NIC_PF_CFG, 0x3);
 
@@ -507,8 +473,6 @@ static int nic_init_hw(struct nicpf *nic)
cqm_cfg = nic_reg_read(nic, NIC_PF_CQM_CFG);
if (cqm_cfg < NICPF_CQM_MIN_DROP_LEVEL)
nic_reg_write(nic, NIC_PF_CQM_CFG, NICPF_CQM_MIN_DROP_LEVEL);
-
-   return 0;
 }
 
 /* Channel parse index configuration */
@@ -1325,6 +1289,7 @@ static int nic_probe(struct pci_dev *pdev, const struct 
pci_device_id *ent)
 {
struct device *dev = >dev;
struct nicpf *nic;
+   u8 max_lmac;
interr;
 
BUILD_BUG_ON(sizeof(union nic_mbx) > 16);
@@ -1334,10 +1299,8 @@ static int nic_probe(struct pci_dev *pdev, const struct 
pci_device_id *ent)
return -ENOMEM;
 
nic->hw = devm_kzalloc(dev, sizeof(struct hw_info), GFP_KERNEL);
-   if (!nic->hw) {
-   devm_kfree(dev, nic);
+   if (!nic->hw)
return -ENOMEM;
-   }
 
pci_set_drvdata(pdev, nic);
 
@@ -1378,11 +1341,33 @@ static int nic_probe(struct pci_dev *pdev, const struct 
pci_device_id *ent)
 
nic->node = nic_get_node_id(pdev);
 
-   /* Initialize hardware */
-   err = nic_init_hw(nic);
-   if (err)
+   /* Get HW capability info */
+   nic_get_hw_info(nic);
+
+   /* Allocate memory for LMAC tracking elements */
+   err = -ENOMEM;
+   max_lmac = nic->hw->bgx_cnt * MAX_LMAC_PER_BGX;
+
+   nic->vf_lmac_map = devm_kmalloc_array(dev, max_lmac, sizeof(u8),
+ GFP_KERNEL);
+   if (!nic->vf_lmac_map)
+   goto err_release_regions;
+
+   nic->link = devm_kmalloc_array(dev, max_lmac, sizeof(u8), GFP_KERNEL);
+   if (!nic->link)
goto err_release_regions;
 
+   nic->duplex = devm_kmalloc_array(dev, max_lmac, sizeof(u8), GFP_KERNEL);
+   if (!nic->duplex)
+   g

Re: [PATCH net-next 1/2] net: add support for Cavium PTP coprocessor

[Aleksey Makarov]

On 11/07/2017 10:49 PM, David Daney wrote:

On 11/07/2017 11:07 AM, Aleksey Makarov wrote:

From: Radoslaw Biernacki <r...@semihalf.com>

This patch adds support for the Precision Time Protocol
Clocks and Timestamping hardware found on Cavium ThunderX
processors.

Signed-off-by: Radoslaw Biernacki <r...@semihalf.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@cavium.com>
---
  drivers/net/ethernet/cavium/Kconfig |  13 +
  drivers/net/ethernet/cavium/Makefile    |   1 +
  drivers/net/ethernet/cavium/common/Makefile |   1 +
  drivers/net/ethernet/cavium/common/cavium_ptp.c | 353 
  drivers/net/ethernet/cavium/common/cavium_ptp.h |  78 ++
  5 files changed, 446 insertions(+)
  create mode 100644 drivers/net/ethernet/cavium/common/Makefile
  create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
  create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h


[...]

+
+/* The Cavium PTP can *only* be found in SoCs containing the ThunderX
+ * ARM64 CPU implementation.  All accesses to the device registers on this
+ * platform are implicitly strongly ordered with respect to memory
+ * accesses.


I believe that is not correct.  I/O register accesses are implicitly
ordered with respect to other I/O register accesses.  However, with
respect to memory accesses, no ordering is imposed.  Therefore, one
must be very careful not to introduce subtile memory ordering bugs
with these things when using the unordered versions.


I will fix it in the next version.

Thank you
Aleksey Makarov


+ * So writeq_relaxed() and readq_relaxed() are safe to use with
+ * no memory barriers in this driver.  The readq()/writeq() functions add
+ * explicit ordering operation which in this case are redundant, and only
+ * add overhead.



Also it should be noted that on production silicon, the performance difference between 
the "relaxed" variant and the normal variant of read*/write* is often 
negligible.



+ */
+
+static u64 cavium_ptp_reg_read(struct cavium_ptp *clock, u64 offset)
+{
+    return readq_relaxed(clock->reg_base + offset);
+}
+
+static void cavium_ptp_reg_write(struct cavium_ptp *clock, u64 offset, u64 val)
+{
+    writeq_relaxed(val, clock->reg_base + offset);
+}
+


Are the PTP register access really so much in the hot path that using the 
relaxed variants can be measured here?  If not, would it make the driver look 
cleaner to remove these and just use readq/writeq calls directly  in the body 
of the driver?

David.

Re: [PATCH net-next 1/2] net: add support for Cavium PTP coprocessor

[Aleksey Makarov]

On 11/07/2017 10:49 PM, David Daney wrote:

On 11/07/2017 11:07 AM, Aleksey Makarov wrote:

From: Radoslaw Biernacki 

This patch adds support for the Precision Time Protocol
Clocks and Timestamping hardware found on Cavium ThunderX
processors.

Signed-off-by: Radoslaw Biernacki 
Signed-off-by: Aleksey Makarov 
---
  drivers/net/ethernet/cavium/Kconfig |  13 +
  drivers/net/ethernet/cavium/Makefile    |   1 +
  drivers/net/ethernet/cavium/common/Makefile |   1 +
  drivers/net/ethernet/cavium/common/cavium_ptp.c | 353 
  drivers/net/ethernet/cavium/common/cavium_ptp.h |  78 ++
  5 files changed, 446 insertions(+)
  create mode 100644 drivers/net/ethernet/cavium/common/Makefile
  create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
  create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h


[...]

+
+/* The Cavium PTP can *only* be found in SoCs containing the ThunderX
+ * ARM64 CPU implementation.  All accesses to the device registers on this
+ * platform are implicitly strongly ordered with respect to memory
+ * accesses.


I believe that is not correct.  I/O register accesses are implicitly
ordered with respect to other I/O register accesses.  However, with
respect to memory accesses, no ordering is imposed.  Therefore, one
must be very careful not to introduce subtile memory ordering bugs
with these things when using the unordered versions.


I will fix it in the next version.

Thank you
Aleksey Makarov


+ * So writeq_relaxed() and readq_relaxed() are safe to use with
+ * no memory barriers in this driver.  The readq()/writeq() functions add
+ * explicit ordering operation which in this case are redundant, and only
+ * add overhead.



Also it should be noted that on production silicon, the performance difference between 
the "relaxed" variant and the normal variant of read*/write* is often 
negligible.



+ */
+
+static u64 cavium_ptp_reg_read(struct cavium_ptp *clock, u64 offset)
+{
+    return readq_relaxed(clock->reg_base + offset);
+}
+
+static void cavium_ptp_reg_write(struct cavium_ptp *clock, u64 offset, u64 val)
+{
+    writeq_relaxed(val, clock->reg_base + offset);
+}
+


Are the PTP register access really so much in the hot path that using the 
relaxed variants can be measured here?  If not, would it make the driver look 
cleaner to remove these and just use readq/writeq calls directly  in the body 
of the driver?

David.

[PATCH net-next 2/2] net: thunderx: add timestamping support

[Aleksey Makarov]

From: Sunil Goutham <sgout...@cavium.com>

This adds timestamping support for both receive and transmit
paths. On the receive side no filters are supported i.e either
all pkts will get a timestamp appended infront of the packet or none.
On the transmit side HW doesn't support timestamp insertion but
only generates a separate CQE with transmitted packet's timestamp.
Also HW supports only one packet at a time for timestamping on the
transmit side.

Signed-off-by: Sunil Goutham <sgout...@cavium.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@cavium.com>
---
 drivers/net/ethernet/cavium/Kconfig|   1 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  15 ++
 drivers/net/ethernet/cavium/thunder/nic_main.c |  58 ++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +++-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 173 -
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 9 files changed, 330 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index 634c79a8c812..b61be4afc9b8 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -27,6 +27,7 @@ config THUNDER_NIC_PF
 
 config THUNDER_NIC_VF
tristate "Thunder Virtual function driver"
+   select CAVIUM_PTP
depends on 64BIT
---help---
  This driver supports Thunder's NIC virtual function
diff --git a/drivers/net/ethernet/cavium/thunder/nic.h 
b/drivers/net/ethernet/cavium/thunder/nic.h
index 4a02e618e318..204b234beb9d 100644
--- a/drivers/net/ethernet/cavium/thunder/nic.h
+++ b/drivers/net/ethernet/cavium/thunder/nic.h
@@ -263,6 +263,8 @@ struct nicvf_drv_stats {
struct u64_stats_sync   syncp;
 };
 
+struct cavium_ptp;
+
 struct nicvf {
struct nicvf*pnicvf;
struct net_device   *netdev;
@@ -312,6 +314,12 @@ struct nicvf {
struct tasklet_struct   qs_err_task;
struct work_struct  reset_task;
 
+   /* PTP timestamp */
+   struct cavium_ptp   *ptp_clock;
+   boolhw_rx_tstamp;
+   struct sk_buff  *ptp_skb;
+   atomic_ttx_ptp_skbs;
+
/* Interrupt coalescing settings */
u32 cq_coalesce_usecs;
u32 msg_enable;
@@ -371,6 +379,7 @@ struct nicvf {
 #defineNIC_MBOX_MSG_LOOPBACK   0x16/* Set interface in 
loopback */
 #defineNIC_MBOX_MSG_RESET_STAT_COUNTER 0x17/* Reset statistics 
counters */
 #defineNIC_MBOX_MSG_PFC0x18/* Pause frame control 
*/
+#defineNIC_MBOX_MSG_PTP_CFG0x19/* HW packet timestamp 
*/
 #defineNIC_MBOX_MSG_CFG_DONE   0xF0/* VF configuration 
done */
 #defineNIC_MBOX_MSG_SHUTDOWN   0xF1/* VF is being shutdown 
*/
 
@@ -521,6 +530,11 @@ struct pfc {
u8fc_tx;
 };
 
+struct set_ptp {
+   u8msg;
+   bool  enable;
+};
+
 /* 128 bit shared memory between PF and each VF */
 union nic_mbx {
struct { u8 msg; }  msg;
@@ -540,6 +554,7 @@ union nic_mbx {
struct set_loopback lbk;
struct reset_stat_cfg   reset_stat;
struct pfc  pfc;
+   struct set_ptp  ptp;
 };
 
 #define NIC_NODE_ID_MASK   0x03
diff --git a/drivers/net/ethernet/cavium/thunder/nic_main.c 
b/drivers/net/ethernet/cavium/thunder/nic_main.c
index fb770b0182d3..c12ab7f1adb9 100644
--- a/drivers/net/ethernet/cavium/thunder/nic_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nic_main.c
@@ -460,13 +460,22 @@ static int nic_init_hw(struct nicpf *nic)
/* Enable backpressure */
nic_reg_write(nic, NIC_PF_BP_CFG, (1ULL << 6) | 0x03);
 
-   /* TNS and TNS bypass modes are present only on 88xx */
+   /* TNS and TNS bypass modes are present only on 88xx
+* Also offset of this CSR has changed in 81xx and 83xx.
+*/
if (nic->pdev->subsystem_device == PCI_SUBSYS_DEVID_88XX_NIC_PF) {
/* Disable TNS mode on both interfaces */
nic_reg_write(nic, NIC_PF_INTF_0_1_SEND_CFG,
- (NIC_TNS_BYPASS_MODE << 7) | BGX0_BLOCK);
+ (NIC_TNS_BYPASS_MODE << 7) |
+ BGX0_BLOCK | (1ULL << 16));
nic_reg_write(nic, NIC_PF_INTF_0_1_SEND_CFG | (1 << 8),
- (NIC_TNS_BYPASS_MODE << 7) | BGX1_BLOCK);
+ (NIC_TNS_BYPASS_MODE << 7) |
+ BGX1_BLOCK | (1ULL << 16));
+   } else {
+   /* Configure timestamp

[PATCH net-next 2/2] net: thunderx: add timestamping support

[Aleksey Makarov]

From: Sunil Goutham 

This adds timestamping support for both receive and transmit
paths. On the receive side no filters are supported i.e either
all pkts will get a timestamp appended infront of the packet or none.
On the transmit side HW doesn't support timestamp insertion but
only generates a separate CQE with transmitted packet's timestamp.
Also HW supports only one packet at a time for timestamping on the
transmit side.

Signed-off-by: Sunil Goutham 
Signed-off-by: Aleksey Makarov 
---
 drivers/net/ethernet/cavium/Kconfig|   1 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  15 ++
 drivers/net/ethernet/cavium/thunder/nic_main.c |  58 ++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +++-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 173 -
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 9 files changed, 330 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index 634c79a8c812..b61be4afc9b8 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -27,6 +27,7 @@ config THUNDER_NIC_PF
 
 config THUNDER_NIC_VF
tristate "Thunder Virtual function driver"
+   select CAVIUM_PTP
depends on 64BIT
---help---
  This driver supports Thunder's NIC virtual function
diff --git a/drivers/net/ethernet/cavium/thunder/nic.h 
b/drivers/net/ethernet/cavium/thunder/nic.h
index 4a02e618e318..204b234beb9d 100644
--- a/drivers/net/ethernet/cavium/thunder/nic.h
+++ b/drivers/net/ethernet/cavium/thunder/nic.h
@@ -263,6 +263,8 @@ struct nicvf_drv_stats {
struct u64_stats_sync   syncp;
 };
 
+struct cavium_ptp;
+
 struct nicvf {
struct nicvf*pnicvf;
struct net_device   *netdev;
@@ -312,6 +314,12 @@ struct nicvf {
struct tasklet_struct   qs_err_task;
struct work_struct  reset_task;
 
+   /* PTP timestamp */
+   struct cavium_ptp   *ptp_clock;
+   boolhw_rx_tstamp;
+   struct sk_buff  *ptp_skb;
+   atomic_ttx_ptp_skbs;
+
/* Interrupt coalescing settings */
u32 cq_coalesce_usecs;
u32 msg_enable;
@@ -371,6 +379,7 @@ struct nicvf {
 #defineNIC_MBOX_MSG_LOOPBACK   0x16/* Set interface in 
loopback */
 #defineNIC_MBOX_MSG_RESET_STAT_COUNTER 0x17/* Reset statistics 
counters */
 #defineNIC_MBOX_MSG_PFC0x18/* Pause frame control 
*/
+#defineNIC_MBOX_MSG_PTP_CFG0x19/* HW packet timestamp 
*/
 #defineNIC_MBOX_MSG_CFG_DONE   0xF0/* VF configuration 
done */
 #defineNIC_MBOX_MSG_SHUTDOWN   0xF1/* VF is being shutdown 
*/
 
@@ -521,6 +530,11 @@ struct pfc {
u8fc_tx;
 };
 
+struct set_ptp {
+   u8msg;
+   bool  enable;
+};
+
 /* 128 bit shared memory between PF and each VF */
 union nic_mbx {
struct { u8 msg; }  msg;
@@ -540,6 +554,7 @@ union nic_mbx {
struct set_loopback lbk;
struct reset_stat_cfg   reset_stat;
struct pfc  pfc;
+   struct set_ptp  ptp;
 };
 
 #define NIC_NODE_ID_MASK   0x03
diff --git a/drivers/net/ethernet/cavium/thunder/nic_main.c 
b/drivers/net/ethernet/cavium/thunder/nic_main.c
index fb770b0182d3..c12ab7f1adb9 100644
--- a/drivers/net/ethernet/cavium/thunder/nic_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nic_main.c
@@ -460,13 +460,22 @@ static int nic_init_hw(struct nicpf *nic)
/* Enable backpressure */
nic_reg_write(nic, NIC_PF_BP_CFG, (1ULL << 6) | 0x03);
 
-   /* TNS and TNS bypass modes are present only on 88xx */
+   /* TNS and TNS bypass modes are present only on 88xx
+* Also offset of this CSR has changed in 81xx and 83xx.
+*/
if (nic->pdev->subsystem_device == PCI_SUBSYS_DEVID_88XX_NIC_PF) {
/* Disable TNS mode on both interfaces */
nic_reg_write(nic, NIC_PF_INTF_0_1_SEND_CFG,
- (NIC_TNS_BYPASS_MODE << 7) | BGX0_BLOCK);
+ (NIC_TNS_BYPASS_MODE << 7) |
+ BGX0_BLOCK | (1ULL << 16));
nic_reg_write(nic, NIC_PF_INTF_0_1_SEND_CFG | (1 << 8),
- (NIC_TNS_BYPASS_MODE << 7) | BGX1_BLOCK);
+ (NIC_TNS_BYPASS_MODE << 7) |
+ BGX1_BLOCK | (1ULL << 16));
+   } else {
+   /* Configure timestamp generation timeout to 10us */
+   for (i = 0; i < nic->hw->bgx_cnt; i

[PATCH net-next 1/2] net: add support for Cavium PTP coprocessor

[Aleksey Makarov]

From: Radoslaw Biernacki <r...@semihalf.com>

This patch adds support for the Precision Time Protocol
Clocks and Timestamping hardware found on Cavium ThunderX
processors.

Signed-off-by: Radoslaw Biernacki <r...@semihalf.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@cavium.com>
---
 drivers/net/ethernet/cavium/Kconfig |  13 +
 drivers/net/ethernet/cavium/Makefile|   1 +
 drivers/net/ethernet/cavium/common/Makefile |   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c | 353 
 drivers/net/ethernet/cavium/common/cavium_ptp.h |  78 ++
 5 files changed, 446 insertions(+)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index dcbce6cac63e..634c79a8c812 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -50,6 +50,19 @@ config   THUNDER_NIC_RGX
  This driver supports configuring XCV block of RGX interface
  present on CN81XX chip.
 
+config CAVIUM_PTP
+   tristate "Cavium PTP coprocessor as PTP clock"
+   depends on 64BIT
+   depends on PTP_1588_CLOCK
+   select CAVIUM_RST
+   default y
+   ---help---
+ This driver adds support for the Precision Time Protocol Clocks and
+ Timestamping coprocessor (PTP) found on Cavium processors.
+ PTP provides timestamping mechanism that is suitable for use in IEEE 
1588
+ Precision Time Protocol or other purposes.  Timestamps can be used in
+ BGX, TNS, GTI, and NIC blocks.
+
 config LIQUIDIO
tristate "Cavium LiquidIO support"
depends on 64BIT
diff --git a/drivers/net/ethernet/cavium/Makefile 
b/drivers/net/ethernet/cavium/Makefile
index 872da9f7c31a..946bba84e81d 100644
--- a/drivers/net/ethernet/cavium/Makefile
+++ b/drivers/net/ethernet/cavium/Makefile
@@ -1,6 +1,7 @@
 #
 # Makefile for the Cavium ethernet device drivers.
 #
+obj-$(CONFIG_NET_VENDOR_CAVIUM) += common/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += thunder/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += liquidio/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += octeon/
diff --git a/drivers/net/ethernet/cavium/common/Makefile 
b/drivers/net/ethernet/cavium/common/Makefile
new file mode 100644
index ..dd8561b8060b
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_CAVIUM_PTP) += cavium_ptp.o
diff --git a/drivers/net/ethernet/cavium/common/cavium_ptp.c 
b/drivers/net/ethernet/cavium/common/cavium_ptp.c
new file mode 100644
index ..6c6f162bedbf
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/cavium_ptp.c
@@ -0,0 +1,353 @@
+/*
+ * cavium_ptp.c - PTP 1588 clock on Cavium hardware
+ *
+ * Copyright (c) 2003-2015, 2017 Cavium, Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License.
+ *
+ * This file may also be available under a different license from Cavium.
+ * Contact Cavium, Inc. for more information
+ */
+
+#include 
+#include 
+#include 
+#include 
+
+#include "cavium_ptp.h"
+
+#define DRV_NAME   "Cavium PTP Driver"
+
+#define PCI_DEVICE_ID_CAVIUM_PTP   0xA00C
+#define PCI_DEVICE_ID_CAVIUM_RST   0xA00E
+
+#define PCI_PTP_BAR_NO 0
+#define PCI_RST_BAR_NO 0
+
+#define PTP_CLOCK_CFG  0xF00ULL
+#define  PTP_CLOCK_CFG_PTP_EN  BIT(0)
+#define PTP_CLOCK_LO   0xF08ULL
+#define PTP_CLOCK_HI   0xF10ULL
+#define PTP_CLOCK_COMP 0xF18ULL
+
+#define RST_BOOT   0x1600ULL
+#define CLOCK_BASE_RATE5000ULL
+
+static u64 ptp_cavium_clock_get(void)
+{
+   struct pci_dev *pdev;
+   void __iomem *base;
+   u64 ret = CLOCK_BASE_RATE * 16;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_RST, NULL);
+   if (!pdev)
+   goto error;
+
+   base = pci_ioremap_bar(pdev, PCI_RST_BAR_NO);
+   if (!base)
+   goto error_put_pdev;
+
+   ret = CLOCK_BASE_RATE * ((readq_relaxed(base + RST_BOOT) >> 33) & 0x3f);
+
+   iounmap(base);
+
+error_put_pdev:
+   pci_dev_put(pdev);
+
+error:
+   return ret;
+}
+
+/* The Cavium PTP can *only* be found in SoCs containing the ThunderX
+ * ARM64 CPU implementation.  All accesses to the device registers on this
+ * platform are implicitly strongly ordered with respect to memory
+ * accesses. So writeq_relaxed() and readq_relaxed() are safe to use with
+ * no memory barriers in this driver.  The readq()/writeq() functions add
+ * explicit ordering operation which in this case are redundant, and only
+ * add overhead.
+ */
+

[PATCH net-next 1/2] net: add support for Cavium PTP coprocessor

[Aleksey Makarov]

From: Radoslaw Biernacki 

This patch adds support for the Precision Time Protocol
Clocks and Timestamping hardware found on Cavium ThunderX
processors.

Signed-off-by: Radoslaw Biernacki 
Signed-off-by: Aleksey Makarov 
---
 drivers/net/ethernet/cavium/Kconfig |  13 +
 drivers/net/ethernet/cavium/Makefile|   1 +
 drivers/net/ethernet/cavium/common/Makefile |   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c | 353 
 drivers/net/ethernet/cavium/common/cavium_ptp.h |  78 ++
 5 files changed, 446 insertions(+)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

diff --git a/drivers/net/ethernet/cavium/Kconfig 
b/drivers/net/ethernet/cavium/Kconfig
index dcbce6cac63e..634c79a8c812 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -50,6 +50,19 @@ config   THUNDER_NIC_RGX
  This driver supports configuring XCV block of RGX interface
  present on CN81XX chip.
 
+config CAVIUM_PTP
+   tristate "Cavium PTP coprocessor as PTP clock"
+   depends on 64BIT
+   depends on PTP_1588_CLOCK
+   select CAVIUM_RST
+   default y
+   ---help---
+ This driver adds support for the Precision Time Protocol Clocks and
+ Timestamping coprocessor (PTP) found on Cavium processors.
+ PTP provides timestamping mechanism that is suitable for use in IEEE 
1588
+ Precision Time Protocol or other purposes.  Timestamps can be used in
+ BGX, TNS, GTI, and NIC blocks.
+
 config LIQUIDIO
tristate "Cavium LiquidIO support"
depends on 64BIT
diff --git a/drivers/net/ethernet/cavium/Makefile 
b/drivers/net/ethernet/cavium/Makefile
index 872da9f7c31a..946bba84e81d 100644
--- a/drivers/net/ethernet/cavium/Makefile
+++ b/drivers/net/ethernet/cavium/Makefile
@@ -1,6 +1,7 @@
 #
 # Makefile for the Cavium ethernet device drivers.
 #
+obj-$(CONFIG_NET_VENDOR_CAVIUM) += common/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += thunder/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += liquidio/
 obj-$(CONFIG_NET_VENDOR_CAVIUM) += octeon/
diff --git a/drivers/net/ethernet/cavium/common/Makefile 
b/drivers/net/ethernet/cavium/common/Makefile
new file mode 100644
index ..dd8561b8060b
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_CAVIUM_PTP) += cavium_ptp.o
diff --git a/drivers/net/ethernet/cavium/common/cavium_ptp.c 
b/drivers/net/ethernet/cavium/common/cavium_ptp.c
new file mode 100644
index ..6c6f162bedbf
--- /dev/null
+++ b/drivers/net/ethernet/cavium/common/cavium_ptp.c
@@ -0,0 +1,353 @@
+/*
+ * cavium_ptp.c - PTP 1588 clock on Cavium hardware
+ *
+ * Copyright (c) 2003-2015, 2017 Cavium, Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License.
+ *
+ * This file may also be available under a different license from Cavium.
+ * Contact Cavium, Inc. for more information
+ */
+
+#include 
+#include 
+#include 
+#include 
+
+#include "cavium_ptp.h"
+
+#define DRV_NAME   "Cavium PTP Driver"
+
+#define PCI_DEVICE_ID_CAVIUM_PTP   0xA00C
+#define PCI_DEVICE_ID_CAVIUM_RST   0xA00E
+
+#define PCI_PTP_BAR_NO 0
+#define PCI_RST_BAR_NO 0
+
+#define PTP_CLOCK_CFG  0xF00ULL
+#define  PTP_CLOCK_CFG_PTP_EN  BIT(0)
+#define PTP_CLOCK_LO   0xF08ULL
+#define PTP_CLOCK_HI   0xF10ULL
+#define PTP_CLOCK_COMP 0xF18ULL
+
+#define RST_BOOT   0x1600ULL
+#define CLOCK_BASE_RATE5000ULL
+
+static u64 ptp_cavium_clock_get(void)
+{
+   struct pci_dev *pdev;
+   void __iomem *base;
+   u64 ret = CLOCK_BASE_RATE * 16;
+
+   pdev = pci_get_device(PCI_VENDOR_ID_CAVIUM,
+ PCI_DEVICE_ID_CAVIUM_RST, NULL);
+   if (!pdev)
+   goto error;
+
+   base = pci_ioremap_bar(pdev, PCI_RST_BAR_NO);
+   if (!base)
+   goto error_put_pdev;
+
+   ret = CLOCK_BASE_RATE * ((readq_relaxed(base + RST_BOOT) >> 33) & 0x3f);
+
+   iounmap(base);
+
+error_put_pdev:
+   pci_dev_put(pdev);
+
+error:
+   return ret;
+}
+
+/* The Cavium PTP can *only* be found in SoCs containing the ThunderX
+ * ARM64 CPU implementation.  All accesses to the device registers on this
+ * platform are implicitly strongly ordered with respect to memory
+ * accesses. So writeq_relaxed() and readq_relaxed() are safe to use with
+ * no memory barriers in this driver.  The readq()/writeq() functions add
+ * explicit ordering operation which in this case are redundant, and only
+ * add overhead.
+ */
+
+static u64 cavium_ptp_reg_read(struct cavium_ptp *clock, u64 offset)
+{
+   return rea

[PATCH net-next 0/2] net: thunderx: add support for PTP clock

[Aleksey Makarov]

This series adds support for IEEE 1588 Precision Time Protocol
to Cavium ethernet driver.

The first patch adds support for the Precision Time Protocol Clocks and
Timestamping coprocessor (PTP) found on Cavium processors.
It registers a new PTP clock in the PTP core and provides functions
to use the counter in BGX, TNS, GTI, and NIC blocks.

The second patch introduces support for the PTP protocol to the
Cavium ThunderX ethernet driver.

Radoslaw Biernacki (1):
  net: add support for Cavium PTP coprocessor

Sunil Goutham (1):
  net: thunderx: add timestamping support

 drivers/net/ethernet/cavium/Kconfig|  14 +
 drivers/net/ethernet/cavium/Makefile   |   1 +
 drivers/net/ethernet/cavium/common/Makefile|   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c| 353 +
 drivers/net/ethernet/cavium/common/cavium_ptp.h|  78 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  15 +
 drivers/net/ethernet/cavium/thunder/nic_main.c |  58 +++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 173 +-
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 13 files changed, 776 insertions(+), 6 deletions(-)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

-- 
2.15.0

[PATCH net-next 0/2] net: thunderx: add support for PTP clock

[Aleksey Makarov]

This series adds support for IEEE 1588 Precision Time Protocol
to Cavium ethernet driver.

The first patch adds support for the Precision Time Protocol Clocks and
Timestamping coprocessor (PTP) found on Cavium processors.
It registers a new PTP clock in the PTP core and provides functions
to use the counter in BGX, TNS, GTI, and NIC blocks.

The second patch introduces support for the PTP protocol to the
Cavium ThunderX ethernet driver.

Radoslaw Biernacki (1):
  net: add support for Cavium PTP coprocessor

Sunil Goutham (1):
  net: thunderx: add timestamping support

 drivers/net/ethernet/cavium/Kconfig|  14 +
 drivers/net/ethernet/cavium/Makefile   |   1 +
 drivers/net/ethernet/cavium/common/Makefile|   1 +
 drivers/net/ethernet/cavium/common/cavium_ptp.c| 353 +
 drivers/net/ethernet/cavium/common/cavium_ptp.h|  78 +
 drivers/net/ethernet/cavium/thunder/nic.h  |  15 +
 drivers/net/ethernet/cavium/thunder/nic_main.c |  58 +++-
 drivers/net/ethernet/cavium/thunder/nic_reg.h  |   1 +
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c|  29 +-
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 173 +-
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c |  26 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  29 ++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   4 +
 13 files changed, 776 insertions(+), 6 deletions(-)
 create mode 100644 drivers/net/ethernet/cavium/common/Makefile
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.c
 create mode 100644 drivers/net/ethernet/cavium/common/cavium_ptp.h

-- 
2.15.0

Re: [PATCH v9 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 05/18/2017 06:49 PM, Petr Mladek wrote:

On Sun 2017-05-14 23:37:50, Aleksey Makarov wrote:



On 05/12/2017 03:57 PM, Petr Mladek wrote:

On Thu 2017-05-11 17:41:58, Sergey Senozhatsky wrote:

On (05/11/17 17:24), Sergey Senozhatsky wrote:

On (05/09/17 10:29), Sabrina Dubroca wrote: [..]

That's caused a change of behavior in my qemu setup, with
this cmdline

root=/dev/sda1 console=ttyS1 console=ttyS0

Before, the kernel logs appeared on ttyS1, and I logged in
with ttyS0 (with my setup, ttyS1 is a file and ttyS0 is
unix socket). Now, the kernel logs go to ttyS0. I need to
swap the two console= parameters to restore behavior.

There might be some other problem (in qemu?) though,
because adding console=tty0 anywhere on that cmdline makes
the logs appear on both tty0 and one ttyS* (but only one of
them, and the ordering of the ttyS* matters).




The problem is that when we are registering a new console, we walk
over the `console_cmdline` list and match _only one_ of the
specified consoles, contrary to what stated in 
Documentation/admin-guide/serial-console.rst:


You can specify multiple console= options on the kernel command
line. Output will appear on *all* of them.


and from the other mail:

Last mentioned 'console=' (preferred console) is the console that 
should become /dev/console.  Its driver is returned by

console_device(). In other respects the last mentioned console is
not special, so I believe it is irrelevant to the report.


Thanks a lot for explanation. I missed these pieces.

But this also means that your commit cf39bf58afdaabc0b ("printk: fix
double printing with earlycon") helps only when the duplicate of the
boot console is defined as the preferred one.

Well, the reverse order of searching the console_cmdline might help
also in other cases but it is weird approach.


I emphasized all here.  Moreover, it is impossible to fix this 
without deep reworking of all the console framework.


IMHO, the same is true also for fixing the bug with double printing
correctly. The current fix helps in some situations but it might
break others. The question is how many people will see the good and
"bad" effects.


I meant that it is difficult to fix the bug that only one line
(first matched) of registered console device receives kernel logs.
That is because of 'index' field of struct console can refer to only one
tty line.

I re-evaluated the "printk: fix double printing with earlycon"
patch and still think it is ok as it breaks only situations
that should not happen, when different lines of the same
console are registered in console_cmdline. But I agree that
someone else with more experience should decide.

- The reason why logs appear twice is that earlycon is not
deregistered after non-early consoles are available.

- Earlycon is not deregistered because in some cases
flag CON_CONSDEV is not set for any of the new non-early
consoles.

- That flag is not set when two entries of console_cmdline
array refer to the same console.  We match the first entry,
it is not preferred and the flag CON_CONSDEV is set only
for preferred console.

- So I changed the order of array walk so that if there
are two entries referring to the same console, last one is matched.
If we specify a console as prefered (last entry in command line)
that means that we expect a driver will eventually register
its console and for that driver we will match the last
entry and the earlycon will be deregistered.

- The subtle question is why we deregister earlycon only
when CON_CONSDEV is set (i. e. only for preferred console).
That check for CON_CONSDEV was initially introduced in

d37bf60de0b4 console: console handover to preferred console

and then refactored in

4d09161196c9 printk: Enable the use of more than one CON_BOOT (early console)
8259cf434202 printk: Ensure that "console enabled" messages are printed on the 
console

The reason is that some consoles can be registered later or earlier
and if we deregister earlycon when the first non-early console
is registered, we can lose some lines of log.  To prevent that
we specify the console *which is being watched* as preferred
(the last console in the console_cmdline array) and
deregister earlycon only when kernel registers its driver.


BTW: I wonder if we really need to add consoles defined by ACPI SPCR
table into the console_cmdline array.


The console_cmdline array is the place where we add descriptions
of consoles that should receive log output.  ACPI SPCR specifies exactly
that information, and I believe it should go to that array.
I don't quite follow your concern.


I am even more curious when
seeing the following code in drivers/acpi/spcr.c:

int __init parse_spcr(bool earlycon) { [...] if (earlycon) 
setup_earlycon(opts);


err = add_preferred_console(uart, 0, opts + strlen(uart) + 1); }

It seems that we setup (register) the early console before we add it
to the console_cmdline array. Do we really need to call
add_preferred_c

Re: [PATCH v9 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 05/18/2017 06:49 PM, Petr Mladek wrote:

On Sun 2017-05-14 23:37:50, Aleksey Makarov wrote:



On 05/12/2017 03:57 PM, Petr Mladek wrote:

On Thu 2017-05-11 17:41:58, Sergey Senozhatsky wrote:

On (05/11/17 17:24), Sergey Senozhatsky wrote:

On (05/09/17 10:29), Sabrina Dubroca wrote: [..]

That's caused a change of behavior in my qemu setup, with
this cmdline

root=/dev/sda1 console=ttyS1 console=ttyS0

Before, the kernel logs appeared on ttyS1, and I logged in
with ttyS0 (with my setup, ttyS1 is a file and ttyS0 is
unix socket). Now, the kernel logs go to ttyS0. I need to
swap the two console= parameters to restore behavior.

There might be some other problem (in qemu?) though,
because adding console=tty0 anywhere on that cmdline makes
the logs appear on both tty0 and one ttyS* (but only one of
them, and the ordering of the ttyS* matters).




The problem is that when we are registering a new console, we walk
over the `console_cmdline` list and match _only one_ of the
specified consoles, contrary to what stated in 
Documentation/admin-guide/serial-console.rst:


You can specify multiple console= options on the kernel command
line. Output will appear on *all* of them.


and from the other mail:

Last mentioned 'console=' (preferred console) is the console that 
should become /dev/console.  Its driver is returned by

console_device(). In other respects the last mentioned console is
not special, so I believe it is irrelevant to the report.


Thanks a lot for explanation. I missed these pieces.

But this also means that your commit cf39bf58afdaabc0b ("printk: fix
double printing with earlycon") helps only when the duplicate of the
boot console is defined as the preferred one.

Well, the reverse order of searching the console_cmdline might help
also in other cases but it is weird approach.


I emphasized all here.  Moreover, it is impossible to fix this 
without deep reworking of all the console framework.


IMHO, the same is true also for fixing the bug with double printing
correctly. The current fix helps in some situations but it might
break others. The question is how many people will see the good and
"bad" effects.


I meant that it is difficult to fix the bug that only one line
(first matched) of registered console device receives kernel logs.
That is because of 'index' field of struct console can refer to only one
tty line.

I re-evaluated the "printk: fix double printing with earlycon"
patch and still think it is ok as it breaks only situations
that should not happen, when different lines of the same
console are registered in console_cmdline. But I agree that
someone else with more experience should decide.

- The reason why logs appear twice is that earlycon is not
deregistered after non-early consoles are available.

- Earlycon is not deregistered because in some cases
flag CON_CONSDEV is not set for any of the new non-early
consoles.

- That flag is not set when two entries of console_cmdline
array refer to the same console.  We match the first entry,
it is not preferred and the flag CON_CONSDEV is set only
for preferred console.

- So I changed the order of array walk so that if there
are two entries referring to the same console, last one is matched.
If we specify a console as prefered (last entry in command line)
that means that we expect a driver will eventually register
its console and for that driver we will match the last
entry and the earlycon will be deregistered.

- The subtle question is why we deregister earlycon only
when CON_CONSDEV is set (i. e. only for preferred console).
That check for CON_CONSDEV was initially introduced in

d37bf60de0b4 console: console handover to preferred console

and then refactored in

4d09161196c9 printk: Enable the use of more than one CON_BOOT (early console)
8259cf434202 printk: Ensure that "console enabled" messages are printed on the 
console

The reason is that some consoles can be registered later or earlier
and if we deregister earlycon when the first non-early console
is registered, we can lose some lines of log.  To prevent that
we specify the console *which is being watched* as preferred
(the last console in the console_cmdline array) and
deregister earlycon only when kernel registers its driver.


BTW: I wonder if we really need to add consoles defined by ACPI SPCR
table into the console_cmdline array.


The console_cmdline array is the place where we add descriptions
of consoles that should receive log output.  ACPI SPCR specifies exactly
that information, and I believe it should go to that array.
I don't quite follow your concern.


I am even more curious when
seeing the following code in drivers/acpi/spcr.c:

int __init parse_spcr(bool earlycon) { [...] if (earlycon) 
setup_earlycon(opts);


err = add_preferred_console(uart, 0, opts + strlen(uart) + 1); }

It seems that we setup (register) the early console before we add it
to the console_cmdline array. Do we really need to call
add_preferred_c

Re: [PATCH v9 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 05/12/2017 04:46 PM, Petr Mladek wrote:

On Fri 2017-05-12 14:57:29, Petr Mladek wrote:

On Thu 2017-05-11 17:41:58, Sergey Senozhatsky wrote:

On (05/11/17 17:24), Sergey Senozhatsky wrote:

On (05/09/17 10:29), Sabrina Dubroca wrote:
[..]

That's caused a change of behavior in my qemu setup, with this cmdline

 root=/dev/sda1 console=ttyS1 console=ttyS0

Before, the kernel logs appeared on ttyS1, and I logged in with ttyS0
(with my setup, ttyS1 is a file and ttyS0 is unix socket). Now, the
kernel logs go to ttyS0. I need to swap the two console= parameters to
restore behavior.


Do you actually need to define console=ttyS0 on the cmdline?


Exactly.  You should specify a console on the command line only
if you want kernel logs on it.  It's kernel bug that one of this
consoles does not receive kernel logs, see
Documentation/admin-guide/serial-console.rst


IMHO, if register_console() was called for the unix socket, it
would make logs appear on both ttyS0 and ttyS1. It seems
that register_console() is called only for the console that
stores logs into the file.


It's not quite accurate sentence.  It's qemu who deals with file/socket and
it is transparent to kernel.


adding
console=tty0 anywhere on that cmdline makes the logs appear on both
tty0 and one ttyS* (but only one of them, and the ordering of the
ttyS* matters).


I guess that it worked this way before. I mean that the logs appeared
on both tty0 and one of ttyS*. The only difference should be that
the patch changed the selected ttyS*. So this is still the same problem.



Hmm, I have no idea how to fix this. This is the case where
a registered console matches more entries from the command line.
The fix that caused this regression fixed exactly this situation
and we wanted to make the preferred console first.

I am not sure if we broke some backward compatibility or actually made
it more predictable in the long term.


I think that we actually fixed a very old bug. The last mentioned
console= should be the preferred one and the logs are finally
printed there. Or do I miss anything?


Last mentioned 'console=' (preferred console) is the console that
should become /dev/console.  Its driver is returned by console_device().
In other respects the last mentioned console is not special,
so I believe it is irrelevant to the report.

Thank you
Aleksey Makarov

Re: [PATCH v9 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 05/12/2017 04:46 PM, Petr Mladek wrote:

On Fri 2017-05-12 14:57:29, Petr Mladek wrote:

On Thu 2017-05-11 17:41:58, Sergey Senozhatsky wrote:

On (05/11/17 17:24), Sergey Senozhatsky wrote:

On (05/09/17 10:29), Sabrina Dubroca wrote:
[..]

That's caused a change of behavior in my qemu setup, with this cmdline

 root=/dev/sda1 console=ttyS1 console=ttyS0

Before, the kernel logs appeared on ttyS1, and I logged in with ttyS0
(with my setup, ttyS1 is a file and ttyS0 is unix socket). Now, the
kernel logs go to ttyS0. I need to swap the two console= parameters to
restore behavior.


Do you actually need to define console=ttyS0 on the cmdline?


Exactly.  You should specify a console on the command line only
if you want kernel logs on it.  It's kernel bug that one of this
consoles does not receive kernel logs, see
Documentation/admin-guide/serial-console.rst


IMHO, if register_console() was called for the unix socket, it
would make logs appear on both ttyS0 and ttyS1. It seems
that register_console() is called only for the console that
stores logs into the file.


It's not quite accurate sentence.  It's qemu who deals with file/socket and
it is transparent to kernel.


adding
console=tty0 anywhere on that cmdline makes the logs appear on both
tty0 and one ttyS* (but only one of them, and the ordering of the
ttyS* matters).


I guess that it worked this way before. I mean that the logs appeared
on both tty0 and one of ttyS*. The only difference should be that
the patch changed the selected ttyS*. So this is still the same problem.



Hmm, I have no idea how to fix this. This is the case where
a registered console matches more entries from the command line.
The fix that caused this regression fixed exactly this situation
and we wanted to make the preferred console first.

I am not sure if we broke some backward compatibility or actually made
it more predictable in the long term.


I think that we actually fixed a very old bug. The last mentioned
console= should be the preferred one and the logs are finally
printed there. Or do I miss anything?


Last mentioned 'console=' (preferred console) is the console that
should become /dev/console.  Its driver is returned by console_device().
In other respects the last mentioned console is not special,
so I believe it is irrelevant to the report.

Thank you
Aleksey Makarov

Re: [PATCH v9 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

old unfixed bug shows itself.

But reproducing this bug requires to specify the same console
in command line twice (with different lines), which is pointless
so I would say it is irrelevant.

Thank you
Aleksey Makarov

Re: [PATCH v9 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

old unfixed bug shows itself.

But reproducing this bug requires to specify the same console
in command line twice (with different lines), which is pointless
so I would say it is irrelevant.

Thank you
Aleksey Makarov

Re: [PATCH v9 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 05/11/2017 02:32 PM, Sergey Senozhatsky wrote:

On (05/11/17 17:41), Sergey Senozhatsky wrote:
[..]

Alexey,
can we have preferred console at offset 0 (not at console_cmdline_cnt - 1)
and restore the previous register_console() iteration order?


btw, what if someone has configured the system as
console=   non-braille non-braille braille non-braille?
"The last non-braille console is always the preferred one"
is not true in this case.


I don't quite follow what you think is problem here.
The invariant keeps here, the code makes it always true.
In this case the last console is non-braille and it is preferred.
What is the problem?

Thank you
Aleksey Makarov

Re: [PATCH v9 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 05/11/2017 02:32 PM, Sergey Senozhatsky wrote:

On (05/11/17 17:41), Sergey Senozhatsky wrote:
[..]

Alexey,
can we have preferred console at offset 0 (not at console_cmdline_cnt - 1)
and restore the previous register_console() iteration order?


btw, what if someone has configured the system as
console=   non-braille non-braille braille non-braille?
"The last non-braille console is always the preferred one"
is not true in this case.


I don't quite follow what you think is problem here.
The invariant keeps here, the code makes it always true.
In this case the last console is non-braille and it is preferred.
What is the problem?

Thank you
Aleksey Makarov

Re: [PATCH v9 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 05/11/2017 11:41 AM, Sergey Senozhatsky wrote:

On (05/11/17 17:24), Sergey Senozhatsky wrote:

On (05/09/17 10:29), Sabrina Dubroca wrote:
[..]

That's caused a change of behavior in my qemu setup, with this cmdline

 root=/dev/sda1 console=ttyS1 console=ttyS0

Before, the kernel logs appeared on ttyS1, and I logged in with ttyS0
(with my setup, ttyS1 is a file and ttyS0 is unix socket). Now, the
kernel logs go to ttyS0. I need to swap the two console= parameters to
restore behavior.

There might be some other problem (in qemu?) though, because adding
console=tty0 anywhere on that cmdline makes the logs appear on both
tty0 and one ttyS* (but only one of them, and the ordering of the
ttyS* matters).


thanks for the report.

so we have ttyS1 first and ttyS0 last.
after commit in question, register_console() iterates console_cmdline
in reverse order so we see ttyS0 first, then we hit `if (newcon->index < 0)'
condition, set newcon to ttyS0, because we iterate in reverse order now, and
break out. so we enable ttyS0, instead of ttyS1.

previously, we iterated console_cmdline from index 0 and saw ttyS1 first.
so the same `if (newcon->index < 0)' condition would set newcone to ttyS1,
and, thus, we would enable ttyS1, not ttyS0.


Alexey,
can we have preferred console at offset 0 (not at console_cmdline_cnt - 1)
and restore the previous register_console() iteration order?


I don't quite understand what is the problem.  Give me more time please.
I hope I will be able to look at this on the weekend.

Thank you
Aleksey Makarov

Re: [PATCH v9 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 05/11/2017 11:41 AM, Sergey Senozhatsky wrote:

On (05/11/17 17:24), Sergey Senozhatsky wrote:

On (05/09/17 10:29), Sabrina Dubroca wrote:
[..]

That's caused a change of behavior in my qemu setup, with this cmdline

 root=/dev/sda1 console=ttyS1 console=ttyS0

Before, the kernel logs appeared on ttyS1, and I logged in with ttyS0
(with my setup, ttyS1 is a file and ttyS0 is unix socket). Now, the
kernel logs go to ttyS0. I need to swap the two console= parameters to
restore behavior.

There might be some other problem (in qemu?) though, because adding
console=tty0 anywhere on that cmdline makes the logs appear on both
tty0 and one ttyS* (but only one of them, and the ordering of the
ttyS* matters).


thanks for the report.

so we have ttyS1 first and ttyS0 last.
after commit in question, register_console() iterates console_cmdline
in reverse order so we see ttyS0 first, then we hit `if (newcon->index < 0)'
condition, set newcon to ttyS0, because we iterate in reverse order now, and
break out. so we enable ttyS0, instead of ttyS1.

previously, we iterated console_cmdline from index 0 and saw ttyS1 first.
so the same `if (newcon->index < 0)' condition would set newcone to ttyS1,
and, thus, we would enable ttyS1, not ttyS0.


Alexey,
can we have preferred console at offset 0 (not at console_cmdline_cnt - 1)
and restore the previous register_console() iteration order?


I don't quite understand what is the problem.  Give me more time please.
I hope I will be able to look at this on the weekend.

Thank you
Aleksey Makarov

Re: [PATCH v9 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 04/11/2017 10:43 AM, Petr Mladek wrote:

On Mon 2017-04-10 21:00:35, Aleksey Makarov wrote:



On 04/10/2017 05:22 PM, Petr Mladek wrote:

On Wed 2017-04-05 23:20:00, Aleksey Makarov wrote:


[..]


Thank you for review.  Can you (or anybody else) ACK it?
I am going to resend the whole series without those empty lines.
May I add your Acked-by:?


Sure. Feel free to use:

Acked-by: Petr Mladek <pmla...@suse.com>

The meaning of the tags is a bit unclear. Acked-by means that
the maintainer agrees with the idea. But it does not necessarily
means that she reviewed the code in details. I agree with the idea
and did the review, so I used the Reviewed-by tag.

Also you do not need to resend the patchset just because the two
empty lines. Sergey agrees. I will wait day or two and push
all three patches into the printk.git if nobody complains
in the meantime. I could remove the two empty lines when doing so.


Ok, then I will not resend it.


Thanks a lot for the fix and patience.


Thank you for reviewing and acking it.
Aleksey Makarov

Re: [PATCH v9 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 04/11/2017 10:43 AM, Petr Mladek wrote:

On Mon 2017-04-10 21:00:35, Aleksey Makarov wrote:



On 04/10/2017 05:22 PM, Petr Mladek wrote:

On Wed 2017-04-05 23:20:00, Aleksey Makarov wrote:


[..]


Thank you for review.  Can you (or anybody else) ACK it?
I am going to resend the whole series without those empty lines.
May I add your Acked-by:?


Sure. Feel free to use:

Acked-by: Petr Mladek 

The meaning of the tags is a bit unclear. Acked-by means that
the maintainer agrees with the idea. But it does not necessarily
means that she reviewed the code in details. I agree with the idea
and did the review, so I used the Reviewed-by tag.

Also you do not need to resend the patchset just because the two
empty lines. Sergey agrees. I will wait day or two and push
all three patches into the printk.git if nobody complains
in the meantime. I could remove the two empty lines when doing so.


Ok, then I will not resend it.


Thanks a lot for the fix and patience.


Thank you for reviewing and acking it.
Aleksey Makarov

Re: [PATCH v9 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 04/10/2017 05:22 PM, Petr Mladek wrote:

On Wed 2017-04-05 23:20:00, Aleksey Makarov wrote:

If a console was specified by ACPI SPCR table _and_ command line
parameters like "console=ttyAMA0" _and_ "earlycon" were specified,
then log messages appear twice.

The root cause is that the code traverses the list of specified
consoles (the `console_cmdline` array) and stops at the first match.
But it may happen that the same console is referred by the elements
of this array twice:

pl011,mmio,0x87e02400,115200 -- from SPCR
ttyAMA0 -- from command line

but in this case `preferred_console` points to the second entry and
the flag CON_CONSDEV is not set, so bootconsole is not deregistered.

To fix that, introduce an invariant "The last non-braille console
is always the preferred one" on the entries of the console_cmdline
array.  Then traverse it in reverse order to be sure that if
the console is preferred then it will be the first matching entry.
Introduce variable console_cmdline_cnt that keeps the number
of elements of the console_cmdline array (Petr Mladek).  It helps
to get rid of the loop that searches for the end of this array.

Reported-by: Sudeep Holla <sudeep.ho...@arm.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@linaro.org>


This version looks fine to me. Just a small nitpick below.
Anyway:

Reviewed-by: Petr Mladek <pmla...@suse.com>


Thank you for review.  Can you (or anybody else) ACK it?
I am going to resend the whole series without those empty lines.
May I add your Acked-by:?

Thank you
Aleksey Makarov


diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index fd752f0c8ef1..be657af45758 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -269,6 +269,7 @@ static struct console *exclusive_console;
  #define MAX_CMDLINECONSOLES 8
  
  static struct console_cmdline console_cmdline[MAX_CMDLINECONSOLES];

+static int console_cmdline_cnt;
  
  static int preferred_console = -1;

  int console_set_on_cmdline;
@@ -1905,12 +1906,26 @@ static int __add_preferred_console(char *name, int idx, 
char *options,
 *  See if this tty is not yet registered, and
 *  if we have a slot free.
 */
-   for (i = 0, c = console_cmdline;
-i < MAX_CMDLINECONSOLES && c->name[0];
-i++, c++) {
+   for (i = 0, c = console_cmdline; i < console_cmdline_cnt; i++, c++) {
if (strcmp(c->name, name) == 0 && c->index == idx) {
-   if (!brl_options)
-   preferred_console = i;
+


This extra new line is non-standard and looks slightly weird to me.
I just point it out. I personally do not mind ;-)



+   if (brl_options)
+   return 0;
+
+   /*
+* Maintain an invariant that will help to find if
+* the matching console is preferred, see
+* register_console():
+*
+* The last non-braille console is always
+* the preferred one.
+*/
+   if (i != console_cmdline_cnt - 1)
+   swap(console_cmdline[i],
+console_cmdline[console_cmdline_cnt - 1]);
+
+   preferred_console = console_cmdline_cnt - 1;
+
return 0;
}
}
@@ -2457,12 +2473,24 @@ void register_console(struct console *newcon)
}
  
  	/*

-*  See if this console matches one we selected on
-*  the command line.
+* See if this console matches one we selected on the command line.
+*
+* There may be several entries in the console_cmdline array matching
+* with the same console, one with newcon->match(), another by
+* name/index:
+*
+*  pl011,mmio,0x87e02400,115200 -- added from SPCR
+*  ttyAMA0 -- added from command line
+*
+* Traverse the console_cmdline array in reverse order to be
+* sure that if this console is preferred then it will be the first
+* matching entry.  We use the invariant that is maintained in
+* __add_preferred_console().
 */
-   for (i = 0, c = console_cmdline;
-i < MAX_CMDLINECONSOLES && c->name[0];
-i++, c++) {
+   for (i = console_cmdline_cnt - 1; i >= 0; i--) {
+


Same here.


+   c = console_cmdline + i;
+
if (!newcon->match ||
newcon->match(newcon, c->name, c->index, c->options) != 0) {
/* default matching */


Best Regards,
Petr

Re: [PATCH v9 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 04/10/2017 05:22 PM, Petr Mladek wrote:

On Wed 2017-04-05 23:20:00, Aleksey Makarov wrote:

If a console was specified by ACPI SPCR table _and_ command line
parameters like "console=ttyAMA0" _and_ "earlycon" were specified,
then log messages appear twice.

The root cause is that the code traverses the list of specified
consoles (the `console_cmdline` array) and stops at the first match.
But it may happen that the same console is referred by the elements
of this array twice:

pl011,mmio,0x87e02400,115200 -- from SPCR
ttyAMA0 -- from command line

but in this case `preferred_console` points to the second entry and
the flag CON_CONSDEV is not set, so bootconsole is not deregistered.

To fix that, introduce an invariant "The last non-braille console
is always the preferred one" on the entries of the console_cmdline
array.  Then traverse it in reverse order to be sure that if
the console is preferred then it will be the first matching entry.
Introduce variable console_cmdline_cnt that keeps the number
of elements of the console_cmdline array (Petr Mladek).  It helps
to get rid of the loop that searches for the end of this array.

Reported-by: Sudeep Holla 
Signed-off-by: Aleksey Makarov 


This version looks fine to me. Just a small nitpick below.
Anyway:

Reviewed-by: Petr Mladek 


Thank you for review.  Can you (or anybody else) ACK it?
I am going to resend the whole series without those empty lines.
May I add your Acked-by:?

Thank you
Aleksey Makarov


diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index fd752f0c8ef1..be657af45758 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -269,6 +269,7 @@ static struct console *exclusive_console;
  #define MAX_CMDLINECONSOLES 8
  
  static struct console_cmdline console_cmdline[MAX_CMDLINECONSOLES];

+static int console_cmdline_cnt;
  
  static int preferred_console = -1;

  int console_set_on_cmdline;
@@ -1905,12 +1906,26 @@ static int __add_preferred_console(char *name, int idx, 
char *options,
 *  See if this tty is not yet registered, and
 *  if we have a slot free.
 */
-   for (i = 0, c = console_cmdline;
-i < MAX_CMDLINECONSOLES && c->name[0];
-i++, c++) {
+   for (i = 0, c = console_cmdline; i < console_cmdline_cnt; i++, c++) {
if (strcmp(c->name, name) == 0 && c->index == idx) {
-   if (!brl_options)
-   preferred_console = i;
+


This extra new line is non-standard and looks slightly weird to me.
I just point it out. I personally do not mind ;-)



+   if (brl_options)
+   return 0;
+
+   /*
+* Maintain an invariant that will help to find if
+* the matching console is preferred, see
+* register_console():
+*
+* The last non-braille console is always
+* the preferred one.
+*/
+   if (i != console_cmdline_cnt - 1)
+   swap(console_cmdline[i],
+console_cmdline[console_cmdline_cnt - 1]);
+
+   preferred_console = console_cmdline_cnt - 1;
+
return 0;
}
}
@@ -2457,12 +2473,24 @@ void register_console(struct console *newcon)
}
  
  	/*

-*  See if this console matches one we selected on
-*  the command line.
+* See if this console matches one we selected on the command line.
+*
+* There may be several entries in the console_cmdline array matching
+* with the same console, one with newcon->match(), another by
+* name/index:
+*
+*  pl011,mmio,0x87e02400,115200 -- added from SPCR
+*  ttyAMA0 -- added from command line
+*
+* Traverse the console_cmdline array in reverse order to be
+* sure that if this console is preferred then it will be the first
+* matching entry.  We use the invariant that is maintained in
+* __add_preferred_console().
 */
-   for (i = 0, c = console_cmdline;
-i < MAX_CMDLINECONSOLES && c->name[0];
-i++, c++) {
+   for (i = console_cmdline_cnt - 1; i >= 0; i--) {
+


Same here.


+   c = console_cmdline + i;
+
if (!newcon->match ||
newcon->match(newcon, c->name, c->index, c->options) != 0) {
/* default matching */


Best Regards,
Petr

Re: [PATCH v9 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 04/06/2017 12:57 AM, Andy Shevchenko wrote:

On Wed, Apr 5, 2017 at 11:20 PM, Aleksey Makarov
<aleksey.maka...@linaro.org> wrote:

If a console was specified by ACPI SPCR table _and_ command line
parameters like "console=ttyAMA0" _and_ "earlycon" were specified,
then log messages appear twice.

The root cause is that the code traverses the list of specified
consoles (the `console_cmdline` array) and stops at the first match.
But it may happen that the same console is referred by the elements
of this array twice:

pl011,mmio,0x87e02400,115200 -- from SPCR
ttyAMA0 -- from command line

but in this case `preferred_console` points to the second entry and
the flag CON_CONSDEV is not set, so bootconsole is not deregistered.

To fix that, introduce an invariant "The last non-braille console
is always the preferred one" on the entries of the console_cmdline
array.  Then traverse it in reverse order to be sure that if
the console is preferred then it will be the first matching entry.
Introduce variable console_cmdline_cnt that keeps the number
of elements of the console_cmdline array (Petr Mladek).  It helps
to get rid of the loop that searches for the end of this array.



 #define MAX_CMDLINECONSOLES 8

 static struct console_cmdline console_cmdline[MAX_CMDLINECONSOLES];
+static int console_cmdline_cnt;


This should be equal to -1 at the beginning, am I right?


No, this is not an index of the last element, this is count of
elements of cmdline_console array.  So it is 0 initially.


 static int preferred_console = -1;
 int console_set_on_cmdline;
@@ -1905,12 +1906,26 @@ static int __add_preferred_console(char *name, int idx, 
char *options,
 *  See if this tty is not yet registered, and
 *  if we have a slot free.
 */
-   for (i = 0, c = console_cmdline;
-i < MAX_CMDLINECONSOLES && c->name[0];
-i++, c++) {
+   for (i = 0, c = console_cmdline; i < console_cmdline_cnt; i++, c++) {
if (strcmp(c->name, name) == 0 && c->index == idx) {
-   if (!brl_options)
-   preferred_console = i;
+



+   if (brl_options)
+   return 0;


Is it invariant or brl_options may appear while looping?


I am not sure I understand your question.
If we find that we are registering a braille console that
has already been registered, we just return without updating
preferred console (it is only about regular consoles) and
without swapping it with the last element of the array (because it
is explicitly mentioned in the invariant:  The last
*non-braille* console is always the preferred one)


+
+   /*
+* Maintain an invariant that will help to find if
+* the matching console is preferred, see
+* register_console():
+*
+* The last non-braille console is always
+* the preferred one.
+*/
+   if (i != console_cmdline_cnt - 1)
+   swap(console_cmdline[i],
+console_cmdline[console_cmdline_cnt - 1]);


i'm wondering if you can iterate from the end to beginning as you do below.
It would simplify things.


You mean iterate to find the last element?
Yes I can and it is how this was implemented in v8,
Petr Mladek asked to introduce console_cmdline_cnt.

Thank you for review
Aleksey Makarov


+
+   preferred_console = console_cmdline_cnt - 1;
+
return 0;
}
}
@@ -1923,6 +1938,7 @@ static int __add_preferred_console(char *name, int idx, 
char *options,
braille_set_options(c, brl_options);

c->index = idx;
+   console_cmdline_cnt++;
return 0;
 }
 /*
@@ -2457,12 +2473,24 @@ void register_console(struct console *newcon)
}

/*
-*  See if this console matches one we selected on
-*  the command line.
+* See if this console matches one we selected on the command line.
+*
+* There may be several entries in the console_cmdline array matching
+* with the same console, one with newcon->match(), another by
+* name/index:
+*
+*  pl011,mmio,0x87e02400,115200 -- added from SPCR
+*  ttyAMA0 -- added from command line
+*
+* Traverse the console_cmdline array in reverse order to be
+* sure that if this console is preferred then it will be the first
+* matching entry.  We use the invariant that is maintained in
+* __add_preferred_console().
 */
-   for (i = 0, c = console_cmdline;
-i < MAX_CMDLINECONSOLES && c->name[0];
-i++, c++)

Re: [PATCH v9 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 04/06/2017 12:57 AM, Andy Shevchenko wrote:

On Wed, Apr 5, 2017 at 11:20 PM, Aleksey Makarov
 wrote:

If a console was specified by ACPI SPCR table _and_ command line
parameters like "console=ttyAMA0" _and_ "earlycon" were specified,
then log messages appear twice.

The root cause is that the code traverses the list of specified
consoles (the `console_cmdline` array) and stops at the first match.
But it may happen that the same console is referred by the elements
of this array twice:

pl011,mmio,0x87e02400,115200 -- from SPCR
ttyAMA0 -- from command line

but in this case `preferred_console` points to the second entry and
the flag CON_CONSDEV is not set, so bootconsole is not deregistered.

To fix that, introduce an invariant "The last non-braille console
is always the preferred one" on the entries of the console_cmdline
array.  Then traverse it in reverse order to be sure that if
the console is preferred then it will be the first matching entry.
Introduce variable console_cmdline_cnt that keeps the number
of elements of the console_cmdline array (Petr Mladek).  It helps
to get rid of the loop that searches for the end of this array.



 #define MAX_CMDLINECONSOLES 8

 static struct console_cmdline console_cmdline[MAX_CMDLINECONSOLES];
+static int console_cmdline_cnt;


This should be equal to -1 at the beginning, am I right?


No, this is not an index of the last element, this is count of
elements of cmdline_console array.  So it is 0 initially.


 static int preferred_console = -1;
 int console_set_on_cmdline;
@@ -1905,12 +1906,26 @@ static int __add_preferred_console(char *name, int idx, 
char *options,
 *  See if this tty is not yet registered, and
 *  if we have a slot free.
 */
-   for (i = 0, c = console_cmdline;
-i < MAX_CMDLINECONSOLES && c->name[0];
-i++, c++) {
+   for (i = 0, c = console_cmdline; i < console_cmdline_cnt; i++, c++) {
if (strcmp(c->name, name) == 0 && c->index == idx) {
-   if (!brl_options)
-   preferred_console = i;
+



+   if (brl_options)
+   return 0;


Is it invariant or brl_options may appear while looping?


I am not sure I understand your question.
If we find that we are registering a braille console that
has already been registered, we just return without updating
preferred console (it is only about regular consoles) and
without swapping it with the last element of the array (because it
is explicitly mentioned in the invariant:  The last
*non-braille* console is always the preferred one)


+
+   /*
+* Maintain an invariant that will help to find if
+* the matching console is preferred, see
+* register_console():
+*
+* The last non-braille console is always
+* the preferred one.
+*/
+   if (i != console_cmdline_cnt - 1)
+   swap(console_cmdline[i],
+console_cmdline[console_cmdline_cnt - 1]);


i'm wondering if you can iterate from the end to beginning as you do below.
It would simplify things.


You mean iterate to find the last element?
Yes I can and it is how this was implemented in v8,
Petr Mladek asked to introduce console_cmdline_cnt.

Thank you for review
Aleksey Makarov


+
+   preferred_console = console_cmdline_cnt - 1;
+
return 0;
}
}
@@ -1923,6 +1938,7 @@ static int __add_preferred_console(char *name, int idx, 
char *options,
braille_set_options(c, brl_options);

c->index = idx;
+   console_cmdline_cnt++;
return 0;
 }
 /*
@@ -2457,12 +2473,24 @@ void register_console(struct console *newcon)
}

/*
-*  See if this console matches one we selected on
-*  the command line.
+* See if this console matches one we selected on the command line.
+*
+* There may be several entries in the console_cmdline array matching
+* with the same console, one with newcon->match(), another by
+* name/index:
+*
+*  pl011,mmio,0x87e02400,115200 -- added from SPCR
+*  ttyAMA0 -- added from command line
+*
+* Traverse the console_cmdline array in reverse order to be
+* sure that if this console is preferred then it will be the first
+* matching entry.  We use the invariant that is maintained in
+* __add_preferred_console().
 */
-   for (i = 0, c = console_cmdline;
-i < MAX_CMDLINECONSOLES && c->name[0];
-i++, c++) {



+   for (i = conso

[PATCH v9 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

If a console was specified by ACPI SPCR table _and_ command line
parameters like "console=ttyAMA0" _and_ "earlycon" were specified,
then log messages appear twice.

The root cause is that the code traverses the list of specified
consoles (the `console_cmdline` array) and stops at the first match.
But it may happen that the same console is referred by the elements
of this array twice:

pl011,mmio,0x87e02400,115200 -- from SPCR
ttyAMA0 -- from command line

but in this case `preferred_console` points to the second entry and
the flag CON_CONSDEV is not set, so bootconsole is not deregistered.

To fix that, introduce an invariant "The last non-braille console
is always the preferred one" on the entries of the console_cmdline
array.  Then traverse it in reverse order to be sure that if
the console is preferred then it will be the first matching entry.
Introduce variable console_cmdline_cnt that keeps the number
of elements of the console_cmdline array (Petr Mladek).  It helps
to get rid of the loop that searches for the end of this array.

Reported-by: Sudeep Holla <sudeep.ho...@arm.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@linaro.org>
---

v8 -> v9:
- Introduce variable console_cmdline_cnt that keeps the number
  of elements of the console_cmdline array (Petr Mladek).  It helps
  to get rid of the loop that searches for the end of this array.
For the record: I think that this console_cmdline_cnt implementation
is worse than just a loop that finds the end of the array because 
1) we have to support consistency between console_cmdline_cnt and
  console_cmdline_cnt
2) it makes patch a bit more intrusive

v7 -> v8:
- add an explanation to the comment how console_cmdline can contain entries
  referring to the same console
- move the body of the function introduced in the previous version to cycle
- don't panic() (Sergey Senozhatsky).  Don't check this condition because
  the loop condition guarantees that it holds.
- use swap() (Sergey Senozhatsky)

v6 -> v7:
- return back to v5 
- leave the check for already registered entries and add a function that
  maintains the invariant.

v5 -> v6:
- drop v5 and continue to work on v4:
- introduce _braille_is_braille_console(). It helps to split original loop
  into three parts: 1) search for braille console, 2) check for
  preferred_console, 3) match other entries so that these three parts do not
  intersect.
- introduce for_each_console_cmdline() macros to traverse console_cmdline
  (Petr Mladek)

 kernel/printk/printk.c | 48 ++--
 1 file changed, 38 insertions(+), 10 deletions(-)

diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index fd752f0c8ef1..be657af45758 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -269,6 +269,7 @@ static struct console *exclusive_console;
 #define MAX_CMDLINECONSOLES 8
 
 static struct console_cmdline console_cmdline[MAX_CMDLINECONSOLES];
+static int console_cmdline_cnt;
 
 static int preferred_console = -1;
 int console_set_on_cmdline;
@@ -1905,12 +1906,26 @@ static int __add_preferred_console(char *name, int idx, 
char *options,
 *  See if this tty is not yet registered, and
 *  if we have a slot free.
 */
-   for (i = 0, c = console_cmdline;
-i < MAX_CMDLINECONSOLES && c->name[0];
-i++, c++) {
+   for (i = 0, c = console_cmdline; i < console_cmdline_cnt; i++, c++) {
if (strcmp(c->name, name) == 0 && c->index == idx) {
-   if (!brl_options)
-   preferred_console = i;
+
+   if (brl_options)
+   return 0;
+
+   /*
+* Maintain an invariant that will help to find if
+* the matching console is preferred, see
+* register_console():
+*
+* The last non-braille console is always
+* the preferred one.
+*/
+   if (i != console_cmdline_cnt - 1)
+   swap(console_cmdline[i],
+console_cmdline[console_cmdline_cnt - 1]);
+
+   preferred_console = console_cmdline_cnt - 1;
+
return 0;
}
}
@@ -1923,6 +1938,7 @@ static int __add_preferred_console(char *name, int idx, 
char *options,
braille_set_options(c, brl_options);
 
c->index = idx;
+   console_cmdline_cnt++;
return 0;
 }
 /*
@@ -2457,12 +2473,24 @@ void register_console(struct console *newcon)
}
 
/*
-*  See if this console matches one we selected on
-*  the command line.
+* See if this console

[PATCH v9 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

If a console was specified by ACPI SPCR table _and_ command line
parameters like "console=ttyAMA0" _and_ "earlycon" were specified,
then log messages appear twice.

The root cause is that the code traverses the list of specified
consoles (the `console_cmdline` array) and stops at the first match.
But it may happen that the same console is referred by the elements
of this array twice:

pl011,mmio,0x87e02400,115200 -- from SPCR
ttyAMA0 -- from command line

but in this case `preferred_console` points to the second entry and
the flag CON_CONSDEV is not set, so bootconsole is not deregistered.

To fix that, introduce an invariant "The last non-braille console
is always the preferred one" on the entries of the console_cmdline
array.  Then traverse it in reverse order to be sure that if
the console is preferred then it will be the first matching entry.
Introduce variable console_cmdline_cnt that keeps the number
of elements of the console_cmdline array (Petr Mladek).  It helps
to get rid of the loop that searches for the end of this array.

Reported-by: Sudeep Holla 
Signed-off-by: Aleksey Makarov 
---

v8 -> v9:
- Introduce variable console_cmdline_cnt that keeps the number
  of elements of the console_cmdline array (Petr Mladek).  It helps
  to get rid of the loop that searches for the end of this array.
For the record: I think that this console_cmdline_cnt implementation
is worse than just a loop that finds the end of the array because 
1) we have to support consistency between console_cmdline_cnt and
  console_cmdline_cnt
2) it makes patch a bit more intrusive

v7 -> v8:
- add an explanation to the comment how console_cmdline can contain entries
  referring to the same console
- move the body of the function introduced in the previous version to cycle
- don't panic() (Sergey Senozhatsky).  Don't check this condition because
  the loop condition guarantees that it holds.
- use swap() (Sergey Senozhatsky)

v6 -> v7:
- return back to v5 
- leave the check for already registered entries and add a function that
  maintains the invariant.

v5 -> v6:
- drop v5 and continue to work on v4:
- introduce _braille_is_braille_console(). It helps to split original loop
  into three parts: 1) search for braille console, 2) check for
  preferred_console, 3) match other entries so that these three parts do not
  intersect.
- introduce for_each_console_cmdline() macros to traverse console_cmdline
  (Petr Mladek)

 kernel/printk/printk.c | 48 ++--
 1 file changed, 38 insertions(+), 10 deletions(-)

diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index fd752f0c8ef1..be657af45758 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -269,6 +269,7 @@ static struct console *exclusive_console;
 #define MAX_CMDLINECONSOLES 8
 
 static struct console_cmdline console_cmdline[MAX_CMDLINECONSOLES];
+static int console_cmdline_cnt;
 
 static int preferred_console = -1;
 int console_set_on_cmdline;
@@ -1905,12 +1906,26 @@ static int __add_preferred_console(char *name, int idx, 
char *options,
 *  See if this tty is not yet registered, and
 *  if we have a slot free.
 */
-   for (i = 0, c = console_cmdline;
-i < MAX_CMDLINECONSOLES && c->name[0];
-i++, c++) {
+   for (i = 0, c = console_cmdline; i < console_cmdline_cnt; i++, c++) {
if (strcmp(c->name, name) == 0 && c->index == idx) {
-   if (!brl_options)
-   preferred_console = i;
+
+   if (brl_options)
+   return 0;
+
+   /*
+* Maintain an invariant that will help to find if
+* the matching console is preferred, see
+* register_console():
+*
+* The last non-braille console is always
+* the preferred one.
+*/
+   if (i != console_cmdline_cnt - 1)
+   swap(console_cmdline[i],
+console_cmdline[console_cmdline_cnt - 1]);
+
+   preferred_console = console_cmdline_cnt - 1;
+
return 0;
}
}
@@ -1923,6 +1938,7 @@ static int __add_preferred_console(char *name, int idx, 
char *options,
braille_set_options(c, brl_options);
 
c->index = idx;
+   console_cmdline_cnt++;
return 0;
 }
 /*
@@ -2457,12 +2473,24 @@ void register_console(struct console *newcon)
}
 
/*
-*  See if this console matches one we selected on
-*  the command line.
+* See if this console matches one we selected on the command line.
+*
+

Re: [PATCH v8 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 04/04/2017 02:12 PM, Petr Mladek wrote:

On Thu 2017-03-30 14:55:46, Sergey Senozhatsky wrote:

On (03/28/17 14:56), Petr Mladek wrote:


[..]


Alekesey, any chance to use the global variable to count used or point
to the last element?

I know that you have already spent a lot of time with it. It was great
work. But the current solution of the cycle looks weird to me.


Sorry for the delay.  I will send next version soon.

Thank you
Aleksey Makarov

Re: [PATCH v8 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 04/04/2017 02:12 PM, Petr Mladek wrote:

On Thu 2017-03-30 14:55:46, Sergey Senozhatsky wrote:

On (03/28/17 14:56), Petr Mladek wrote:


[..]


Alekesey, any chance to use the global variable to count used or point
to the last element?

I know that you have already spent a lot of time with it. It was great
work. But the current solution of the cycle looks weird to me.


Sorry for the delay.  I will send next version soon.

Thank you
Aleksey Makarov

Re: [PATCH v8 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 03/27/2017 05:14 PM, Petr Mladek wrote:

On Mon 2017-03-20 13:03:00, Aleksey Makarov wrote:


[..]


diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index fd752f0c8ef1..462036e7a767 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -1909,8 +1909,28 @@ static int __add_preferred_console(char *name, int idx, 
char *options,
 i < MAX_CMDLINECONSOLES && c->name[0];
 i++, c++) {
if (strcmp(c->name, name) == 0 && c->index == idx) {
-   if (!brl_options)
-   preferred_console = i;
+   int last;
+
+   if (brl_options)
+   return 0;
+
+   /*
+* Maintain an invariant that will help to find if
+* the matching console is preferred, see
+* register_console():
+*
+* The last non-braille console is always
+* the preferred one.
+*/
+   for (last = MAX_CMDLINECONSOLES - 1;
+last >= 0 && !console_cmdline[last].name[0];
+last--)
+   ;


This is a rather non-trivial code to find the last element.
I might make sense to count it in a global variable.
Then we might remove the check for console_cmdline[i].name[0]
also in the other for cycles and make them better readable.


Having an additional variable console_cmdline_last pointing to the last element
would require maintaining consistency between this variable and
contents of console_cmdline.  For the code we have it is not hard, but when code
is changed we need to check this.  Also there exists preferred_console that
has almost the same meaning but it points not to the last element, but to the
last non-braille element.  Also we need to have a special value (-1) for this
variable for empty array. So I personally would instead try to rewrite this:

for (last = MAX_CMDLINECONSOLES - 1; last >= 0; last--)
if (console_cmdline[last].name[0])
break;

Is it better?  If not, I will send a version with console_cmdline_last.


+
+   if (i != last)
+   swap(console_cmdline[i], console_cmdline[last]);


I was not aware of the swap() function. It is great to know ;-)


Yes, same for me.  Thanks to Sergey Senozhatsky.

Thank you for review
Aleksey Makarov

Re: [PATCH v8 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 03/27/2017 05:14 PM, Petr Mladek wrote:

On Mon 2017-03-20 13:03:00, Aleksey Makarov wrote:


[..]


diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index fd752f0c8ef1..462036e7a767 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -1909,8 +1909,28 @@ static int __add_preferred_console(char *name, int idx, 
char *options,
 i < MAX_CMDLINECONSOLES && c->name[0];
 i++, c++) {
if (strcmp(c->name, name) == 0 && c->index == idx) {
-   if (!brl_options)
-   preferred_console = i;
+   int last;
+
+   if (brl_options)
+   return 0;
+
+   /*
+* Maintain an invariant that will help to find if
+* the matching console is preferred, see
+* register_console():
+*
+* The last non-braille console is always
+* the preferred one.
+*/
+   for (last = MAX_CMDLINECONSOLES - 1;
+last >= 0 && !console_cmdline[last].name[0];
+last--)
+   ;


This is a rather non-trivial code to find the last element.
I might make sense to count it in a global variable.
Then we might remove the check for console_cmdline[i].name[0]
also in the other for cycles and make them better readable.


Having an additional variable console_cmdline_last pointing to the last element
would require maintaining consistency between this variable and
contents of console_cmdline.  For the code we have it is not hard, but when code
is changed we need to check this.  Also there exists preferred_console that
has almost the same meaning but it points not to the last element, but to the
last non-braille element.  Also we need to have a special value (-1) for this
variable for empty array. So I personally would instead try to rewrite this:

for (last = MAX_CMDLINECONSOLES - 1; last >= 0; last--)
if (console_cmdline[last].name[0])
break;

Is it better?  If not, I will send a version with console_cmdline_last.


+
+   if (i != last)
+   swap(console_cmdline[i], console_cmdline[last]);


I was not aware of the swap() function. It is great to know ;-)


Yes, same for me.  Thanks to Sergey Senozhatsky.

Thank you for review
Aleksey Makarov

[PATCH v8 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

If a console was specified by ACPI SPCR table _and_ command line
parameters like "console=ttyAMA0" _and_ "earlycon" were specified,
then log messages appear twice.

The root cause is that the code traverses the list of specified
consoles (the `console_cmdline` array) and stops at the first match.
But it may happen that the same console is referred by the elements
of this array twice:

pl011,mmio,0x87e02400,115200 -- from SPCR
ttyAMA0 -- from command line

but in this case `preferred_console` points to the second entry and
the flag CON_CONSDEV is not set, so bootconsole is not deregistered.

To fix that, introduce an invariant "The last non-braille console
is always the preferred one" on the entries of the console_cmdline
array.  Then traverse it in reverse order to be sure that if
the console is preferred then it will be the first matching entry.

Reported-by: Sudeep Holla <sudeep.ho...@arm.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@linaro.org>
---

v7 -> v8:
- add an explanation to the comment how console_cmdline can contain entries
  referring to the same console
- move the body of the function introduced in the previous version to cycle
- don't panic() (Sergey Senozhatsky).  Don't check this condition because
  the loop condition guarantees that it holds.
- use swap() (Sergey Senozhatsky)

v6 -> v7:
- return back to v5 
- leave the check for already registered entries and add a function that
  maintains the invariant.

v5 -> v6:
- drop v5 and continue to work on v4:
- introduce _braille_is_braille_console(). It helps to split original loop
  into three parts: 1) search for braille console, 2) check for
  preferred_console, 3) match other entries so that these three parts do not
  intersect.
- introduce for_each_console_cmdline() macros to traverse console_cmdline
  (Petr Mladek)

 kernel/printk/printk.c | 49 ++---
 1 file changed, 42 insertions(+), 7 deletions(-)

diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index fd752f0c8ef1..462036e7a767 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -1909,8 +1909,28 @@ static int __add_preferred_console(char *name, int idx, 
char *options,
 i < MAX_CMDLINECONSOLES && c->name[0];
 i++, c++) {
if (strcmp(c->name, name) == 0 && c->index == idx) {
-   if (!brl_options)
-   preferred_console = i;
+   int last;
+
+   if (brl_options)
+   return 0;
+
+   /*
+* Maintain an invariant that will help to find if
+* the matching console is preferred, see
+* register_console():
+*
+* The last non-braille console is always
+* the preferred one.
+*/
+   for (last = MAX_CMDLINECONSOLES - 1;
+last >= 0 && !console_cmdline[last].name[0];
+last--)
+   ;
+
+   if (i != last)
+   swap(console_cmdline[i], console_cmdline[last]);
+
+   preferred_console = last;
return 0;
}
}
@@ -2457,12 +2477,27 @@ void register_console(struct console *newcon)
}
 
/*
-*  See if this console matches one we selected on
-*  the command line.
+* See if this console matches one we selected on the command line.
+*
+* There may be several entries in the console_cmdline array matching
+* with the same console, one with newcon->match(), another by
+* name/index:
+*
+*  pl011,mmio,0x87e02400,115200 -- added from SPCR
+*  ttyAMA0 -- added from command line
+*
+* Traverse the console_cmdline array in reverse order to be
+* sure that if this console is preferred then it will be the first
+* matching entry.  We use the invariant that is maintained in
+* __add_preferred_console().
 */
-   for (i = 0, c = console_cmdline;
-i < MAX_CMDLINECONSOLES && c->name[0];
-i++, c++) {
+   for (i = MAX_CMDLINECONSOLES - 1; i >= 0; i--) {
+
+   if (!console_cmdline[i].name[0])
+   continue;
+
+   c = console_cmdline + i;
+
if (!newcon->match ||
newcon->match(newcon, c->name, c->index, c->options) != 0) {
/* default matching */
-- 
2.12.0

[PATCH v8 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

If a console was specified by ACPI SPCR table _and_ command line
parameters like "console=ttyAMA0" _and_ "earlycon" were specified,
then log messages appear twice.

The root cause is that the code traverses the list of specified
consoles (the `console_cmdline` array) and stops at the first match.
But it may happen that the same console is referred by the elements
of this array twice:

pl011,mmio,0x87e02400,115200 -- from SPCR
ttyAMA0 -- from command line

but in this case `preferred_console` points to the second entry and
the flag CON_CONSDEV is not set, so bootconsole is not deregistered.

To fix that, introduce an invariant "The last non-braille console
is always the preferred one" on the entries of the console_cmdline
array.  Then traverse it in reverse order to be sure that if
the console is preferred then it will be the first matching entry.

Reported-by: Sudeep Holla 
Signed-off-by: Aleksey Makarov 
---

v7 -> v8:
- add an explanation to the comment how console_cmdline can contain entries
  referring to the same console
- move the body of the function introduced in the previous version to cycle
- don't panic() (Sergey Senozhatsky).  Don't check this condition because
  the loop condition guarantees that it holds.
- use swap() (Sergey Senozhatsky)

v6 -> v7:
- return back to v5 
- leave the check for already registered entries and add a function that
  maintains the invariant.

v5 -> v6:
- drop v5 and continue to work on v4:
- introduce _braille_is_braille_console(). It helps to split original loop
  into three parts: 1) search for braille console, 2) check for
  preferred_console, 3) match other entries so that these three parts do not
  intersect.
- introduce for_each_console_cmdline() macros to traverse console_cmdline
  (Petr Mladek)

 kernel/printk/printk.c | 49 ++---
 1 file changed, 42 insertions(+), 7 deletions(-)

diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index fd752f0c8ef1..462036e7a767 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -1909,8 +1909,28 @@ static int __add_preferred_console(char *name, int idx, 
char *options,
 i < MAX_CMDLINECONSOLES && c->name[0];
 i++, c++) {
if (strcmp(c->name, name) == 0 && c->index == idx) {
-   if (!brl_options)
-   preferred_console = i;
+   int last;
+
+   if (brl_options)
+   return 0;
+
+   /*
+* Maintain an invariant that will help to find if
+* the matching console is preferred, see
+* register_console():
+*
+* The last non-braille console is always
+* the preferred one.
+*/
+   for (last = MAX_CMDLINECONSOLES - 1;
+last >= 0 && !console_cmdline[last].name[0];
+last--)
+   ;
+
+   if (i != last)
+   swap(console_cmdline[i], console_cmdline[last]);
+
+   preferred_console = last;
return 0;
}
}
@@ -2457,12 +2477,27 @@ void register_console(struct console *newcon)
}
 
/*
-*  See if this console matches one we selected on
-*  the command line.
+* See if this console matches one we selected on the command line.
+*
+* There may be several entries in the console_cmdline array matching
+* with the same console, one with newcon->match(), another by
+* name/index:
+*
+*  pl011,mmio,0x87e02400,115200 -- added from SPCR
+*  ttyAMA0 -- added from command line
+*
+* Traverse the console_cmdline array in reverse order to be
+* sure that if this console is preferred then it will be the first
+* matching entry.  We use the invariant that is maintained in
+* __add_preferred_console().
 */
-   for (i = 0, c = console_cmdline;
-i < MAX_CMDLINECONSOLES && c->name[0];
-i++, c++) {
+   for (i = MAX_CMDLINECONSOLES - 1; i >= 0; i--) {
+
+   if (!console_cmdline[i].name[0])
+   continue;
+
+   c = console_cmdline + i;
+
if (!newcon->match ||
newcon->match(newcon, c->name, c->index, c->options) != 0) {
/* default matching */
-- 
2.12.0

Re: [PATCH] braille-console: Fix value returned by _braille_console_setup

[Aleksey Makarov]

On 03/19/2017 05:37 PM, Samuel Thibault wrote:

commit bbeddf52adc1 ("printk: move braille console support into
separate braille.[ch] files") introduced _braille_console_setup()
to outline the braille initialization code.  There was however some
confusion over the value it was supposed to return. commit 2cfe6c4ac7ee
("printk: Fix return of braille_register_console()") tried to fix it
but failed to.

This fixes and documents the returned value according to the use
in printk.c: non-zero return means a parsing error, and thus this
console configuration should be ignored.


Did you test the code on real braille hardware?
I am asking because there could also be other problems with the code
that have not been run for 3 years since it was changed.

Thank you
Aleksey Makarov


Signed-off-by: Samuel Thibault <samuel.thiba...@ens-lyon.org>
Cc: Aleksey Makarov <aleksey.maka...@linaro.org>
Cc: Joe Perches <j...@perches.com>
Cc: Ming Lei <ming@canonical.com>
Cc: Steven Rostedt <rost...@goodmis.org>
Cc: Petr Mladek <pmla...@suse.com>

Index: linux-4.10/kernel/printk/braille.c
===
--- linux-4.10.orig/kernel/printk/braille.c
+++ linux-4.10/kernel/printk/braille.c
@@ -2,12 +2,13 @@

 #include 
 #include 
+#include 
 #include 

 #include "console_cmdline.h"
 #include "braille.h"

-char *_braille_console_setup(char **str, char **brl_options)
+int _braille_console_setup(char **str, char **brl_options)
 {
if (!strncmp(*str, "brl,", 4)) {
*brl_options = "";
@@ -15,14 +16,15 @@ char *_braille_console_setup(char **str,
} else if (!strncmp(*str, "brl=", 4)) {
*brl_options = *str + 4;
*str = strchr(*brl_options, ',');
-   if (!*str)
+   if (!*str) {
pr_err("need port name after brl=\n");
+   return -EINVAL;
+   }
else
*((*str)++) = 0;
-   } else
-   return NULL;
+   }

-   return *str;
+   return 0;
 }

 int
Index: linux-4.10/kernel/printk/braille.h
===
--- linux-4.10.orig/kernel/printk/braille.h
+++ linux-4.10/kernel/printk/braille.h
@@ -9,7 +9,14 @@ braille_set_options(struct console_cmdli
c->brl_options = brl_options;
 }

-char *
+/*
+ * Setup console according to braille options.
+ * Return -EINVAL on syntax error, 0 on success (or no braille option was
+ * actually given).
+ * Modifies str to point to the serial options
+ * Sets brl_options to the parsed braille options.
+ */
+int
 _braille_console_setup(char **str, char **brl_options);

 int
@@ -25,10 +32,10 @@ braille_set_options(struct console_cmdli
 {
 }

-static inline char *
+static inline int
 _braille_console_setup(char **str, char **brl_options)
 {
-   return NULL;
+   return 0;
 }

 static inline int

Re: [PATCH] braille-console: Fix value returned by _braille_console_setup

[Aleksey Makarov]

On 03/19/2017 05:37 PM, Samuel Thibault wrote:

commit bbeddf52adc1 ("printk: move braille console support into
separate braille.[ch] files") introduced _braille_console_setup()
to outline the braille initialization code.  There was however some
confusion over the value it was supposed to return. commit 2cfe6c4ac7ee
("printk: Fix return of braille_register_console()") tried to fix it
but failed to.

This fixes and documents the returned value according to the use
in printk.c: non-zero return means a parsing error, and thus this
console configuration should be ignored.


Did you test the code on real braille hardware?
I am asking because there could also be other problems with the code
that have not been run for 3 years since it was changed.

Thank you
Aleksey Makarov


Signed-off-by: Samuel Thibault 
Cc: Aleksey Makarov 
Cc: Joe Perches 
Cc: Ming Lei 
Cc: Steven Rostedt 
Cc: Petr Mladek 

Index: linux-4.10/kernel/printk/braille.c
===
--- linux-4.10.orig/kernel/printk/braille.c
+++ linux-4.10/kernel/printk/braille.c
@@ -2,12 +2,13 @@

 #include 
 #include 
+#include 
 #include 

 #include "console_cmdline.h"
 #include "braille.h"

-char *_braille_console_setup(char **str, char **brl_options)
+int _braille_console_setup(char **str, char **brl_options)
 {
if (!strncmp(*str, "brl,", 4)) {
*brl_options = "";
@@ -15,14 +16,15 @@ char *_braille_console_setup(char **str,
} else if (!strncmp(*str, "brl=", 4)) {
*brl_options = *str + 4;
*str = strchr(*brl_options, ',');
-   if (!*str)
+   if (!*str) {
pr_err("need port name after brl=\n");
+   return -EINVAL;
+   }
else
*((*str)++) = 0;
-   } else
-   return NULL;
+   }

-   return *str;
+   return 0;
 }

 int
Index: linux-4.10/kernel/printk/braille.h
===
--- linux-4.10.orig/kernel/printk/braille.h
+++ linux-4.10/kernel/printk/braille.h
@@ -9,7 +9,14 @@ braille_set_options(struct console_cmdli
c->brl_options = brl_options;
 }

-char *
+/*
+ * Setup console according to braille options.
+ * Return -EINVAL on syntax error, 0 on success (or no braille option was
+ * actually given).
+ * Modifies str to point to the serial options
+ * Sets brl_options to the parsed braille options.
+ */
+int
 _braille_console_setup(char **str, char **brl_options);

 int
@@ -25,10 +32,10 @@ braille_set_options(struct console_cmdli
 {
 }

-static inline char *
+static inline int
 _braille_console_setup(char **str, char **brl_options)
 {
-   return NULL;
+   return 0;
 }

 static inline int

[PATCH v7 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

If a console was specified by ACPI SPCR table _and_ command line
parameters like "console=ttyAMA0" _and_ "earlycon" were specified,
then log messages appear twice.

The root cause is that the code traverses the list of specified
consoles (the `console_cmdline` array) and stops at the first match.
But it may happen that the same console is referred by the elements
of this array twice:

pl011,mmio,0x87e02400,115200 -- from SPCR
ttyAMA0 -- from command line

but in this case `preferred_console` points to the second entry and
the flag CON_CONSDEV is not set, so bootconsole is not deregistered.

To fix that, introduce an invariant "The last non-braille console
is always the preferred one" on the entries of the console_cmdline
array.  Then traverse it in reverse order to be sure that if
the console is preferred then it will be the first matching entry.

Reported-by: Sudeep Holla <sudeep.ho...@arm.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@linaro.org>
---

v6 -> v7:
- return back to v5 :)
- leave the check for already registered entries and add a function that
  maintains the invariant.

v5 -> v6:
- drop v5 and continue to work on v4:
- introduce _braille_is_braille_console(). It helps to split original loop
  into three parts: 1) search for braille console, 2) check for
  preferred_console, 3) match other entries so that these three parts do not
  intersect.
- introduce for_each_console_cmdline() macros to traverse console_cmdline
  (Petr Mladek)

 kernel/printk/printk.c | 48 ++--
 1 file changed, 42 insertions(+), 6 deletions(-)

diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index fd752f0c8ef1..88f86fb23bc4 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -1895,6 +1895,34 @@ asmlinkage __visible void early_printk(const char *fmt, 
...)
 }
 #endif
 
+/*
+ * This function maintains an invariant that will help to find if
+ * the matching console is preferred, see register_console():
+ *
+ *  The last non-braille console is always the preferred one.
+ */
+static void ensure_preferred_is_last(int i)
+{
+   int last;
+
+   for (last = MAX_CMDLINECONSOLES - 1;
+last >= 0 && !console_cmdline[last].name[0];
+last--)
+   ;
+
+   BUG_ON(last < 0);
+
+   if (i != last) {
+   struct console_cmdline t;
+
+   t = console_cmdline[i];
+   console_cmdline[i] = console_cmdline[last];
+   console_cmdline[last] = t;
+   }
+
+   preferred_console = last;
+}
+
 static int __add_preferred_console(char *name, int idx, char *options,
   char *brl_options)
 {
@@ -1910,7 +1938,7 @@ static int __add_preferred_console(char *name, int idx, 
char *options,
 i++, c++) {
if (strcmp(c->name, name) == 0 && c->index == idx) {
if (!brl_options)
-   preferred_console = i;
+   ensure_preferred_is_last(i);
return 0;
}
}
@@ -2457,12 +2485,20 @@ void register_console(struct console *newcon)
}
 
/*
-*  See if this console matches one we selected on
-*  the command line.
+* See if this console matches one we selected on the command line.
+*
+* The console_cmdline array is traversed in the reverse order because
+* we want to be sure that if this console is preferred then it will be
+* the first matching entry.  We use the invariant that is maintained in
+* __add_preferred_console().
 */
-   for (i = 0, c = console_cmdline;
-i < MAX_CMDLINECONSOLES && c->name[0];
-i++, c++) {
+   for (i = MAX_CMDLINECONSOLES - 1; i >= 0; i--) {
+
+   if (!console_cmdline[i].name[0])
+   continue;
+
+   c = console_cmdline + i;
+
if (!newcon->match ||
newcon->match(newcon, c->name, c->index, c->options) != 0) {
/* default matching */
-- 
2.12.0

[PATCH v7 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

If a console was specified by ACPI SPCR table _and_ command line
parameters like "console=ttyAMA0" _and_ "earlycon" were specified,
then log messages appear twice.

The root cause is that the code traverses the list of specified
consoles (the `console_cmdline` array) and stops at the first match.
But it may happen that the same console is referred by the elements
of this array twice:

pl011,mmio,0x87e02400,115200 -- from SPCR
ttyAMA0 -- from command line

but in this case `preferred_console` points to the second entry and
the flag CON_CONSDEV is not set, so bootconsole is not deregistered.

To fix that, introduce an invariant "The last non-braille console
is always the preferred one" on the entries of the console_cmdline
array.  Then traverse it in reverse order to be sure that if
the console is preferred then it will be the first matching entry.

Reported-by: Sudeep Holla 
Signed-off-by: Aleksey Makarov 
---

v6 -> v7:
- return back to v5 :)
- leave the check for already registered entries and add a function that
  maintains the invariant.

v5 -> v6:
- drop v5 and continue to work on v4:
- introduce _braille_is_braille_console(). It helps to split original loop
  into three parts: 1) search for braille console, 2) check for
  preferred_console, 3) match other entries so that these three parts do not
  intersect.
- introduce for_each_console_cmdline() macros to traverse console_cmdline
  (Petr Mladek)

 kernel/printk/printk.c | 48 ++--
 1 file changed, 42 insertions(+), 6 deletions(-)

diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index fd752f0c8ef1..88f86fb23bc4 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -1895,6 +1895,34 @@ asmlinkage __visible void early_printk(const char *fmt, 
...)
 }
 #endif
 
+/*
+ * This function maintains an invariant that will help to find if
+ * the matching console is preferred, see register_console():
+ *
+ *  The last non-braille console is always the preferred one.
+ */
+static void ensure_preferred_is_last(int i)
+{
+   int last;
+
+   for (last = MAX_CMDLINECONSOLES - 1;
+last >= 0 && !console_cmdline[last].name[0];
+last--)
+   ;
+
+   BUG_ON(last < 0);
+
+   if (i != last) {
+   struct console_cmdline t;
+
+   t = console_cmdline[i];
+   console_cmdline[i] = console_cmdline[last];
+   console_cmdline[last] = t;
+   }
+
+   preferred_console = last;
+}
+
 static int __add_preferred_console(char *name, int idx, char *options,
   char *brl_options)
 {
@@ -1910,7 +1938,7 @@ static int __add_preferred_console(char *name, int idx, 
char *options,
 i++, c++) {
if (strcmp(c->name, name) == 0 && c->index == idx) {
if (!brl_options)
-   preferred_console = i;
+   ensure_preferred_is_last(i);
return 0;
}
}
@@ -2457,12 +2485,20 @@ void register_console(struct console *newcon)
}
 
/*
-*  See if this console matches one we selected on
-*  the command line.
+* See if this console matches one we selected on the command line.
+*
+* The console_cmdline array is traversed in the reverse order because
+* we want to be sure that if this console is preferred then it will be
+* the first matching entry.  We use the invariant that is maintained in
+* __add_preferred_console().
 */
-   for (i = 0, c = console_cmdline;
-i < MAX_CMDLINECONSOLES && c->name[0];
-i++, c++) {
+   for (i = MAX_CMDLINECONSOLES - 1; i >= 0; i--) {
+
+   if (!console_cmdline[i].name[0])
+   continue;
+
+   c = console_cmdline + i;
+
if (!newcon->match ||
newcon->match(newcon, c->name, c->index, c->options) != 0) {
/* default matching */
-- 
2.12.0

Re: [PATCH v6 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 03/17/2017 02:43 PM, Aleksey Makarov wrote:

[..]

> @@ -2457,40 +2491,50 @@ void register_console(struct console *newcon)
>   }
>  
>   /*
> -  *  See if this console matches one we selected on
> -  *  the command line.
> +  * See if this console matches one we selected on the command line.
> +  * Do it in three steps:
> +  *
> +  * 1) check if it is a braille console..
>*/
> - for (i = 0, c = console_cmdline;
> -  i < MAX_CMDLINECONSOLES && c->name[0];
> -  i++, c++) {
> - if (!newcon->match ||
> - newcon->match(newcon, c->name, c->index, c->options) != 0) {
> - /* default matching */
> - BUILD_BUG_ON(sizeof(c->name) != sizeof(newcon->name));
> - if (strcmp(c->name, newcon->name) != 0)
> - continue;
> - if (newcon->index >= 0 &&
> - newcon->index != c->index)
> - continue;
> - if (newcon->index < 0)
> - newcon->index = c->index;
> -
> - if (_braille_register_console(newcon, c))
> - return;
> -
> - if (newcon->setup &&
> - newcon->setup(newcon, c->options) != 0)
> - break;
> - }
> + for_each_console_cmdline(i, c)
> + if (_braille_is_braille_console(c) &&
> + match_console_name(newcon, c) &&
> + _braille_register_console(newcon, c))
> + return;

I am sorry to say that, but it looks like this does not work either.
newcon->index still can be changed here in match_console_name(), 
but (correctly implemented) _braille_register_console() may refuse
to register it, and the changed newcon is passed to newcon->match() below.

I tried the shuffle (i. e. v5) approach again and it seems I managed to
write quite nice code.  I am going to send it in a minute.

Thank you
Aleksey Makarov

> - newcon->flags |= CON_ENABLED;
> - if (i == preferred_console) {
> + /*
> +  * 2) check if this console was set as preferred by command line
> +  * parameters or by call to add_preferred_console().  There may be
> +  * several entries in the console_cmdline array matching with the same
> +  * console, one with newcon->match(), another by name/index:
> +  *
> +  *  pl011,mmio,0x87e02400,115200 -- added from SPCR
> +  *  ttyAMA0 -- added from command line
> +  *
> +  * so we can not use the first match.  Instead check the
> +  * entry pointed by preferred_console and then all other entries.
> +  */
> + if (preferred_console >= 0 &&
> + match_console(newcon, console_cmdline + preferred_console)) {
> + if (newcon->flags & CON_ENABLED) {
>   newcon->flags |= CON_CONSDEV;
>   has_preferred = true;
>   }
> - break;
> + goto match;
> + }
> +
> + /*
> +  * 3) check other entries
> +  */
> + for_each_console_cmdline(i, c) {
> + if (preferred_console == i || _braille_is_braille_console(c))
> + continue;
> +
> + if (match_console(newcon, c))
> + goto match;
>   }
>  
> +match:
>   if (!(newcon->flags & CON_ENABLED))
>   return;
>  
> 

Re: [PATCH v6 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 03/17/2017 02:43 PM, Aleksey Makarov wrote:

[..]

> @@ -2457,40 +2491,50 @@ void register_console(struct console *newcon)
>   }
>  
>   /*
> -  *  See if this console matches one we selected on
> -  *  the command line.
> +  * See if this console matches one we selected on the command line.
> +  * Do it in three steps:
> +  *
> +  * 1) check if it is a braille console..
>*/
> - for (i = 0, c = console_cmdline;
> -  i < MAX_CMDLINECONSOLES && c->name[0];
> -  i++, c++) {
> - if (!newcon->match ||
> - newcon->match(newcon, c->name, c->index, c->options) != 0) {
> - /* default matching */
> - BUILD_BUG_ON(sizeof(c->name) != sizeof(newcon->name));
> - if (strcmp(c->name, newcon->name) != 0)
> - continue;
> - if (newcon->index >= 0 &&
> - newcon->index != c->index)
> - continue;
> - if (newcon->index < 0)
> - newcon->index = c->index;
> -
> - if (_braille_register_console(newcon, c))
> - return;
> -
> - if (newcon->setup &&
> - newcon->setup(newcon, c->options) != 0)
> - break;
> - }
> + for_each_console_cmdline(i, c)
> + if (_braille_is_braille_console(c) &&
> + match_console_name(newcon, c) &&
> + _braille_register_console(newcon, c))
> + return;

I am sorry to say that, but it looks like this does not work either.
newcon->index still can be changed here in match_console_name(), 
but (correctly implemented) _braille_register_console() may refuse
to register it, and the changed newcon is passed to newcon->match() below.

I tried the shuffle (i. e. v5) approach again and it seems I managed to
write quite nice code.  I am going to send it in a minute.

Thank you
Aleksey Makarov

> - newcon->flags |= CON_ENABLED;
> - if (i == preferred_console) {
> + /*
> +  * 2) check if this console was set as preferred by command line
> +  * parameters or by call to add_preferred_console().  There may be
> +  * several entries in the console_cmdline array matching with the same
> +  * console, one with newcon->match(), another by name/index:
> +  *
> +  *  pl011,mmio,0x87e02400,115200 -- added from SPCR
> +  *  ttyAMA0 -- added from command line
> +  *
> +  * so we can not use the first match.  Instead check the
> +  * entry pointed by preferred_console and then all other entries.
> +  */
> + if (preferred_console >= 0 &&
> + match_console(newcon, console_cmdline + preferred_console)) {
> + if (newcon->flags & CON_ENABLED) {
>   newcon->flags |= CON_CONSDEV;
>   has_preferred = true;
>   }
> - break;
> + goto match;
> + }
> +
> + /*
> +  * 3) check other entries
> +  */
> + for_each_console_cmdline(i, c) {
> + if (preferred_console == i || _braille_is_braille_console(c))
> + continue;
> +
> + if (match_console(newcon, c))
> + goto match;
>   }
>  
> +match:
>   if (!(newcon->flags & CON_ENABLED))
>   return;
>  
> 

[PATCH v6 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

If a console was specified by ACPI SPCR table _and_ command line
parameters like "console=ttyAMA0" _and_ "earlycon" were specified,
then log messages appear twice.

The root cause is that the code traverses the list of specified
consoles (the `console_cmdline` array) and stops at the first match.
But it may happen that the same console is referred by the elements
of this array twice:

pl011,mmio,0x87e02400,115200 -- from SPCR
ttyAMA0 -- from command line

but in this case `preferred_console` points to the second entry and
the flag CON_CONSDEV is not set, so bootconsole is not deregistered.

To fix that, split the loop where we search for matching entry of
console_cmdline into three parts that do not intersect:
1) search for braille console
2) check for preferred_console
3) match other entries so that these three parts do not

To to that introduce predicate _braille_is_braille_console() that checks if
its argument is an entry describing a braille console.

Also introduce a macro for_each_console_cmdline() to traverse
the console_cmdline array.

Reported-by: Sudeep Holla <sudeep.ho...@arm.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@linaro.org>
---

v5 -> v6:
- drop v5 and continue to work on v4:
- introduce _braille_is_braille_console(). It helps to split original loop
  into three parts: 1) search for braille console, 2) check for
  preferred_console, 3) match other entries so that these three parts do not
  intersect.
- introduce for_each_console_cmdline() macros to traverse console_cmdline
  (Petr Mladek)

 kernel/printk/braille.h |  12 ++
 kernel/printk/printk.c  | 104 ++--
 2 files changed, 86 insertions(+), 30 deletions(-)

diff --git a/kernel/printk/braille.h b/kernel/printk/braille.h
index 769d771145c8..183aebf6e1dc 100644
--- a/kernel/printk/braille.h
+++ b/kernel/printk/braille.h
@@ -18,6 +18,12 @@ _braille_register_console(struct console *console, struct 
console_cmdline *c);
 int
 _braille_unregister_console(struct console *console);
 
+static inline int
+_braille_is_braille_console(struct console_cmdline *c)
+{
+   return !!c->brl_options;
+}
+
 #else
 
 static inline void
@@ -43,6 +49,12 @@ _braille_unregister_console(struct console *console)
return 0;
 }
 
+static inline int
+_braille_is_braille_console(struct console_cmdline *c)
+{
+   return 0;
+}
+
 #endif
 
 #endif
diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index fd752f0c8ef1..ab2433681ca5 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -270,6 +270,11 @@ static struct console *exclusive_console;
 
 static struct console_cmdline console_cmdline[MAX_CMDLINECONSOLES];
 
+#define for_each_console_cmdline(i, c) \
+   for (i = 0, c = console_cmdline;\
+i < MAX_CMDLINECONSOLES && c->name[0]; \
+i++, c++)
+
 static int preferred_console = -1;
 int console_set_on_cmdline;
 EXPORT_SYMBOL(console_set_on_cmdline);
@@ -1905,9 +1910,7 @@ static int __add_preferred_console(char *name, int idx, 
char *options,
 *  See if this tty is not yet registered, and
 *  if we have a slot free.
 */
-   for (i = 0, c = console_cmdline;
-i < MAX_CMDLINECONSOLES && c->name[0];
-i++, c++) {
+   for_each_console_cmdline(i, c) {
if (strcmp(c->name, name) == 0 && c->index == idx) {
if (!brl_options)
preferred_console = i;
@@ -2383,6 +2386,37 @@ static int __init keep_bootcon_setup(char *str)
 
 early_param("keep_bootcon", keep_bootcon_setup);
 
+static bool match_console_name(struct console *newcon,
+  struct console_cmdline *c)
+{
+   BUILD_BUG_ON(sizeof(c->name) != sizeof(newcon->name));
+   if (strcmp(c->name, newcon->name) != 0)
+   return false;
+   if (newcon->index >= 0 && newcon->index != c->index)
+   return false;
+   if (newcon->index < 0)
+   newcon->index = c->index;
+   return true;
+}
+
+static bool match_console(struct console *newcon, struct console_cmdline *c)
+{
+   if (newcon->match &&
+   newcon->match(newcon, c->name, c->index, c->options) == 0) {
+   newcon->flags |= CON_ENABLED;
+   return true;
+   }
+
+   if (match_console_name(newcon, c)) {
+   if (!newcon->setup || newcon->setup(newcon, c->options) == 0)
+   newcon->flags |= CON_ENABLED;
+
+   return true;
+   }
+
+   return false;
+}
+
 /*
  * The console driver calls this routine during kernel initialization
  * to register the console printing procedure with printk() and to
@@ -2457,40 +2491,50 @@ void register_console(st

[PATCH v6 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

If a console was specified by ACPI SPCR table _and_ command line
parameters like "console=ttyAMA0" _and_ "earlycon" were specified,
then log messages appear twice.

The root cause is that the code traverses the list of specified
consoles (the `console_cmdline` array) and stops at the first match.
But it may happen that the same console is referred by the elements
of this array twice:

pl011,mmio,0x87e02400,115200 -- from SPCR
ttyAMA0 -- from command line

but in this case `preferred_console` points to the second entry and
the flag CON_CONSDEV is not set, so bootconsole is not deregistered.

To fix that, split the loop where we search for matching entry of
console_cmdline into three parts that do not intersect:
1) search for braille console
2) check for preferred_console
3) match other entries so that these three parts do not

To to that introduce predicate _braille_is_braille_console() that checks if
its argument is an entry describing a braille console.

Also introduce a macro for_each_console_cmdline() to traverse
the console_cmdline array.

Reported-by: Sudeep Holla 
Signed-off-by: Aleksey Makarov 
---

v5 -> v6:
- drop v5 and continue to work on v4:
- introduce _braille_is_braille_console(). It helps to split original loop
  into three parts: 1) search for braille console, 2) check for
  preferred_console, 3) match other entries so that these three parts do not
  intersect.
- introduce for_each_console_cmdline() macros to traverse console_cmdline
  (Petr Mladek)

 kernel/printk/braille.h |  12 ++
 kernel/printk/printk.c  | 104 ++--
 2 files changed, 86 insertions(+), 30 deletions(-)

diff --git a/kernel/printk/braille.h b/kernel/printk/braille.h
index 769d771145c8..183aebf6e1dc 100644
--- a/kernel/printk/braille.h
+++ b/kernel/printk/braille.h
@@ -18,6 +18,12 @@ _braille_register_console(struct console *console, struct 
console_cmdline *c);
 int
 _braille_unregister_console(struct console *console);
 
+static inline int
+_braille_is_braille_console(struct console_cmdline *c)
+{
+   return !!c->brl_options;
+}
+
 #else
 
 static inline void
@@ -43,6 +49,12 @@ _braille_unregister_console(struct console *console)
return 0;
 }
 
+static inline int
+_braille_is_braille_console(struct console_cmdline *c)
+{
+   return 0;
+}
+
 #endif
 
 #endif
diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index fd752f0c8ef1..ab2433681ca5 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -270,6 +270,11 @@ static struct console *exclusive_console;
 
 static struct console_cmdline console_cmdline[MAX_CMDLINECONSOLES];
 
+#define for_each_console_cmdline(i, c) \
+   for (i = 0, c = console_cmdline;\
+i < MAX_CMDLINECONSOLES && c->name[0]; \
+i++, c++)
+
 static int preferred_console = -1;
 int console_set_on_cmdline;
 EXPORT_SYMBOL(console_set_on_cmdline);
@@ -1905,9 +1910,7 @@ static int __add_preferred_console(char *name, int idx, 
char *options,
 *  See if this tty is not yet registered, and
 *  if we have a slot free.
 */
-   for (i = 0, c = console_cmdline;
-i < MAX_CMDLINECONSOLES && c->name[0];
-i++, c++) {
+   for_each_console_cmdline(i, c) {
if (strcmp(c->name, name) == 0 && c->index == idx) {
if (!brl_options)
preferred_console = i;
@@ -2383,6 +2386,37 @@ static int __init keep_bootcon_setup(char *str)
 
 early_param("keep_bootcon", keep_bootcon_setup);
 
+static bool match_console_name(struct console *newcon,
+  struct console_cmdline *c)
+{
+   BUILD_BUG_ON(sizeof(c->name) != sizeof(newcon->name));
+   if (strcmp(c->name, newcon->name) != 0)
+   return false;
+   if (newcon->index >= 0 && newcon->index != c->index)
+   return false;
+   if (newcon->index < 0)
+   newcon->index = c->index;
+   return true;
+}
+
+static bool match_console(struct console *newcon, struct console_cmdline *c)
+{
+   if (newcon->match &&
+   newcon->match(newcon, c->name, c->index, c->options) == 0) {
+   newcon->flags |= CON_ENABLED;
+   return true;
+   }
+
+   if (match_console_name(newcon, c)) {
+   if (!newcon->setup || newcon->setup(newcon, c->options) == 0)
+   newcon->flags |= CON_ENABLED;
+
+   return true;
+   }
+
+   return false;
+}
+
 /*
  * The console driver calls this routine during kernel initialization
  * to register the console printing procedure with printk() and to
@@ -2457,40 +2491,50 @@ void register_console(struct console *newcon)
}
 
/*
-* 

Re: Does braille console work?

[Aleksey Makarov]

On 03/17/2017 03:53 AM, Samuel Thibault wrote:
> Hello,
> 
> Aleksey Makarov, on jeu. 16 mars 2017 17:02:53 +0300, wrote:
>> There can be 3 outcomes from this function:
>> 1) it returns NULL and does not set brl_options
>> 2) it returns NULL and set the variable pointed by str to NULL
>> 3) it returns non-NULL
> 
> Uh, that's odd indeed, the intent was that it'd return an error code
> only.
> 
>> To register a braille console (i. e. to call __add_preferred_console()
>> with non-NULL brl_options) function _braille_console_setup() should
>> return NULL and initialize brl_options. 
> 
> return 0 (as in no error), actually, in the intent.
> 
>> 1) in this case brl_options is NULL and non-braille console is registered
>> 2) kernel produces oops later in console_setup().  I reproduced it
>>passing "console=brl=aaa" parameter to kernel, see below.
>> 3) no console is registered
>>
>> So braille console registration should not work.  What do I miss?
> 
> Nothing, the code transformation was broken :/

So the kernel contains seemingly unmaintained code that does not work
for at least 3 years.  Its parts in printk.c considerably complicate
code support.

>> So it looks like braille console has not been used for more than 3 years.
>> Should we remote it?
> 
> Please, no :)
> 
> It is indeed not often useful, since one would usually use a userland
> daemon to access the system, but when one's system does not boot
> properly or something similar, it's useful to have the braille
> console. That's just very not often useful.

Why should we keep the code which does not work?  And which is used
more than 3 years ago and usage requires patching.
I think removing it is a good idea

Thank you
Aleksey Makarov

> Samuel
> --
> To unsubscribe from this list: send the line "unsubscribe linux-serial" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: Does braille console work?

[Aleksey Makarov]

On 03/17/2017 03:53 AM, Samuel Thibault wrote:
> Hello,
> 
> Aleksey Makarov, on jeu. 16 mars 2017 17:02:53 +0300, wrote:
>> There can be 3 outcomes from this function:
>> 1) it returns NULL and does not set brl_options
>> 2) it returns NULL and set the variable pointed by str to NULL
>> 3) it returns non-NULL
> 
> Uh, that's odd indeed, the intent was that it'd return an error code
> only.
> 
>> To register a braille console (i. e. to call __add_preferred_console()
>> with non-NULL brl_options) function _braille_console_setup() should
>> return NULL and initialize brl_options. 
> 
> return 0 (as in no error), actually, in the intent.
> 
>> 1) in this case brl_options is NULL and non-braille console is registered
>> 2) kernel produces oops later in console_setup().  I reproduced it
>>passing "console=brl=aaa" parameter to kernel, see below.
>> 3) no console is registered
>>
>> So braille console registration should not work.  What do I miss?
> 
> Nothing, the code transformation was broken :/

So the kernel contains seemingly unmaintained code that does not work
for at least 3 years.  Its parts in printk.c considerably complicate
code support.

>> So it looks like braille console has not been used for more than 3 years.
>> Should we remote it?
> 
> Please, no :)
> 
> It is indeed not often useful, since one would usually use a userland
> daemon to access the system, but when one's system does not boot
> properly or something similar, it's useful to have the braille
> console. That's just very not often useful.

Why should we keep the code which does not work?  And which is used
more than 3 years ago and usage requires patching.
I think removing it is a good idea

Thank you
Aleksey Makarov

> Samuel
> --
> To unsubscribe from this list: send the line "unsubscribe linux-serial" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH v5 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 03/16/2017 04:54 PM, Petr Mladek wrote:
> On Thu 2017-03-16 13:36:35, Aleksey Makarov wrote:
>>
>>
>> On 03/15/2017 07:58 PM, Petr Mladek wrote:
>>> On Wed 2017-03-15 13:28:52, Aleksey Makarov wrote:

[..]

> I personally prefer v4. The braille console adds an unexpected
> twist into the code because it skips the rest of the function.
> IMHO, it is better when it is is tested separately with clear
> conditions and comments.
> 
> I would personally add the following into
> kernel/printk/console_cmdline.h
> 
> #define for_each_console_cmdline(c, i) \
>   for (i = 0, c = console_cmdline;\
>i < MAX_CMDLINECONSOLES && c->name[0]; \
>i++, c++)
> 
> It can be used in both __add_preferred_console()
> and register_console().

Ok

> Also I would add the following into kernel/printk/braille.h
> 
> static inline bool
> is_braille_console(struct console_cmdline *c)
> {
>   return !!c->brl_options;
> }
> 
> Then the braille-specific code in register_console() might
> look like:
> 
>   /*
>* Braille console is handled a very special way.
>* Is is not listed in the console_drivers list.
>* Instead it registers its own keyboard and vt notifiers.
>*
>* Be careful and avoid calling c->match() here
>* because it might have side effects!
>*/
>   if (is_braille_console(c) &&
>   match_console_name(newcon, c) &&
>   _braille_register_console(newcon, c))
>   return;

Yes, this is exactly what I was going to do.

> Finally, I would prefer to move
> 
>   newcon->flags |= CON_ENABLED;
> 
> outside the match_console() function. The function will still
> have side effects because of the c->match() calls. But we should
> not make it worse. In fact, it would be great to remove side effects
> from the c->match() functions in the long term (not in this patch set).

I see the motivation but I am afraid this is not possible until
side effects are removed from newcon->match() and match_console().
The point is that match_console() also calls newcon->setup() (side effect)
and this CON_ENABLED flag indicates if this call was successful.

>> I am going to fix v5 preserving both the check for duplicates
>> and the invariant, but tell me please if you prefer the v4 approach.
> 
> I guess that you planed to shuffle console_cmdline entries to keep
> the preferred console first/last. I am afraid that it won't be
> a nice code either. But I might be wrong.

You are right, I tried that, the code was awful.

Thank you
Aleksey Makarov

>>> The
>>> newcon->setup() call is called only when the console matches.
>>> AFAIK, there is only one braille console. We should be on
>>> the safe side if this one does not implement the match()
>>> callback. Or is it even more complicated?
>>
>> As you can see from the original code, the check for braille console
>> was performed in that branch of code where we missed newcon->match(),
>> so yes, it looks like braille console(s) does not have the match() method.
>> I used that in v4 to factor out matching for braille from the loop.
> 
> The check for blr_options is sufficient and better.
> 
> I suggest to wait at least two days until you spend to much time
> on reshuffling the code. It is possible that others would prefer
> v5 or suggest even other solution.
> 
> Best Regards,
> Petr
> --
> To unsubscribe from this list: send the line "unsubscribe linux-serial" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH v5 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 03/16/2017 04:54 PM, Petr Mladek wrote:
> On Thu 2017-03-16 13:36:35, Aleksey Makarov wrote:
>>
>>
>> On 03/15/2017 07:58 PM, Petr Mladek wrote:
>>> On Wed 2017-03-15 13:28:52, Aleksey Makarov wrote:

[..]

> I personally prefer v4. The braille console adds an unexpected
> twist into the code because it skips the rest of the function.
> IMHO, it is better when it is is tested separately with clear
> conditions and comments.
> 
> I would personally add the following into
> kernel/printk/console_cmdline.h
> 
> #define for_each_console_cmdline(c, i) \
>   for (i = 0, c = console_cmdline;\
>i < MAX_CMDLINECONSOLES && c->name[0]; \
>i++, c++)
> 
> It can be used in both __add_preferred_console()
> and register_console().

Ok

> Also I would add the following into kernel/printk/braille.h
> 
> static inline bool
> is_braille_console(struct console_cmdline *c)
> {
>   return !!c->brl_options;
> }
> 
> Then the braille-specific code in register_console() might
> look like:
> 
>   /*
>* Braille console is handled a very special way.
>* Is is not listed in the console_drivers list.
>* Instead it registers its own keyboard and vt notifiers.
>*
>* Be careful and avoid calling c->match() here
>* because it might have side effects!
>*/
>   if (is_braille_console(c) &&
>   match_console_name(newcon, c) &&
>   _braille_register_console(newcon, c))
>   return;

Yes, this is exactly what I was going to do.

> Finally, I would prefer to move
> 
>   newcon->flags |= CON_ENABLED;
> 
> outside the match_console() function. The function will still
> have side effects because of the c->match() calls. But we should
> not make it worse. In fact, it would be great to remove side effects
> from the c->match() functions in the long term (not in this patch set).

I see the motivation but I am afraid this is not possible until
side effects are removed from newcon->match() and match_console().
The point is that match_console() also calls newcon->setup() (side effect)
and this CON_ENABLED flag indicates if this call was successful.

>> I am going to fix v5 preserving both the check for duplicates
>> and the invariant, but tell me please if you prefer the v4 approach.
> 
> I guess that you planed to shuffle console_cmdline entries to keep
> the preferred console first/last. I am afraid that it won't be
> a nice code either. But I might be wrong.

You are right, I tried that, the code was awful.

Thank you
Aleksey Makarov

>>> The
>>> newcon->setup() call is called only when the console matches.
>>> AFAIK, there is only one braille console. We should be on
>>> the safe side if this one does not implement the match()
>>> callback. Or is it even more complicated?
>>
>> As you can see from the original code, the check for braille console
>> was performed in that branch of code where we missed newcon->match(),
>> so yes, it looks like braille console(s) does not have the match() method.
>> I used that in v4 to factor out matching for braille from the loop.
> 
> The check for blr_options is sufficient and better.
> 
> I suggest to wait at least two days until you spend to much time
> on reshuffling the code. It is possible that others would prefer
> v5 or suggest even other solution.
> 
> Best Regards,
> Petr
> --
> To unsubscribe from this list: send the line "unsubscribe linux-serial" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Does braille console work?

[Aleksey Makarov]

Hi,

I am looking at the braille console code and don't understand how its
initialization works.

This is from kernel/printk/braille.c:

char *_braille_console_setup(char **str, char **brl_options)
{
if (!strncmp(*str, "brl,", 4)) {
*brl_options = "";
*str += 4; (3)
} else if (!strncmp(*str, "brl=", 4)) {
*brl_options = *str + 4;
*str = strchr(*brl_options, ',');
if (!*str)
pr_err("need port name after brl=\n"); (2)
else
*((*str)++) = 0;   (3)
} else
return NULL;   (1)

return *str;
}

There can be 3 outcomes from this function:
1) it returns NULL and does not set brl_options
2) it returns NULL and set the variable pointed by str to NULL
3) it returns non-NULL

And this is how it is called in a __setup() function from 
kernel/printk/printk.c:

static int __init console_setup(char *str)
{
char buf[sizeof(console_cmdline[0].name) + 4]; /* 4 for "ttyS" */
char *s, *options, *brl_options = NULL;
int idx;

if (_braille_console_setup(, _options))
return 1;

/*
 * Decode str into name, index, options.
 */
if (str[0] >= '0' && str[0] <= '9') {
strcpy(buf, "ttyS");
strncpy(buf + 4, str, sizeof(buf) - 5);
} else {
strncpy(buf, str, sizeof(buf) - 1);
}

[... Parse other console options here ...]


__add_preferred_console(buf, idx, options, brl_options);
console_set_on_cmdline = 1;
return 1;
}
__setup("console=", console_setup);

To register a braille console (i. e. to call __add_preferred_console()
with non-NULL brl_options) function _braille_console_setup() should
return NULL and initialize brl_options. 

1) in this case brl_options is NULL and non-braille console is registered
2) kernel produces oops later in console_setup().  I reproduced it
   passing "console=brl=aaa" parameter to kernel, see below.
3) no console is registered

So braille console registration should not work.  What do I miss?

The code was changed in July/August 2013 in commits

commit bbeddf52adc1 ("printk: move braille console support into separate 
braille.[ch] files")
commit 2cfe6c4ac7ee ("printk: Fix return of braille_register_console()")

So it looks like braille console has not been used for more than 3 years.
Should we remote it?



Kernel command line: systemd.show_status=no rootwait rw root=/dev/sda7 
acpi=force earlycon console=brl=aaa
braille: need port name after brl=
Unable to handle kernel NULL pointer dereference at virtual address 
pgd = fc00089b
[] *pgd=001e0003, *pud=001e0003, *pmd=001e0003, 
*pte=
Internal error: Oops: 9606 [#1] PREEMPT SMP
Modules linked in:
CPU: 0 PID: 0 Comm: swapper Not tainted 4.11.0-rc2-07-00033-g417d6c6d1a16-dirty 
#727
Hardware name: Cavium ThunderX CN88XX board (DT)
task: fc0008868000 task.stack: fc000884
PC is at console_setup+0x28/0x108
LR is at console_setup+0x20/0x108
pc : [] lr : [] pstate: 80c5
sp : fc0008843e70
x29: fc0008843e70 x28:  
x27: fc00087a7b18 x26: 0001 
x25: fc000880ccb9 x24: 0008 
x23: fc0008812b40 x22:  
x21: fe1afece x20: fe1afec6 
x19:  x18: 0001 
x17:  x16: 0018 
x15: 0002 x14: fff0 
x13: fc0008873158 x12: fc00088fc82a 
x11:  x10: 002d 
x9 : 0023 x8 : 7262207265746661 
x7 : 20656d616e207472 x6 : 002e 
x5 : 0400 x4 :  
x3 : 0002 x2 : 0002 
x1 : 0001 x0 :  

Process swapper (pid: 0, stack limit = 0xfc000884)
Stack: (0xfc0008843e70 to 0xfc0008844000)
3e60:   fc0008843ec0 fc00087c0600
3e80: fc0008812330 fe1afec6 fc0008812060 
3ea0: fe1afed2 fc00087c0598 fc0008811f40 fe1afec6
3ec0: fc0008843f10 fc00080c816c fc0008623558 fe1afec6
3ee0: fc00087a7b18 fe1afece 0080 fc0008707130
3f00: 00bc fe1afed5 fc0008843fa0 fc00087c094c
3f20: fc00087f3948 fc00088d fc0008863000 fc00088d
3f40: fe1afe80 fc00087f3948 0174 001fffaf51a0
3f60: 001fffa38d00 01bc0018 fc000886e6f0 fc000870e560
3f80:  fc00087c0558  fc00087a5db8
3fa0: fc0008843ff0 fc00087c01e8 001ffa3d0a18 2120
3fc0: 

Does braille console work?

[Aleksey Makarov]

Hi,

I am looking at the braille console code and don't understand how its
initialization works.

This is from kernel/printk/braille.c:

char *_braille_console_setup(char **str, char **brl_options)
{
if (!strncmp(*str, "brl,", 4)) {
*brl_options = "";
*str += 4; (3)
} else if (!strncmp(*str, "brl=", 4)) {
*brl_options = *str + 4;
*str = strchr(*brl_options, ',');
if (!*str)
pr_err("need port name after brl=\n"); (2)
else
*((*str)++) = 0;   (3)
} else
return NULL;   (1)

return *str;
}

There can be 3 outcomes from this function:
1) it returns NULL and does not set brl_options
2) it returns NULL and set the variable pointed by str to NULL
3) it returns non-NULL

And this is how it is called in a __setup() function from 
kernel/printk/printk.c:

static int __init console_setup(char *str)
{
char buf[sizeof(console_cmdline[0].name) + 4]; /* 4 for "ttyS" */
char *s, *options, *brl_options = NULL;
int idx;

if (_braille_console_setup(, _options))
return 1;

/*
 * Decode str into name, index, options.
 */
if (str[0] >= '0' && str[0] <= '9') {
strcpy(buf, "ttyS");
strncpy(buf + 4, str, sizeof(buf) - 5);
} else {
strncpy(buf, str, sizeof(buf) - 1);
}

[... Parse other console options here ...]


__add_preferred_console(buf, idx, options, brl_options);
console_set_on_cmdline = 1;
return 1;
}
__setup("console=", console_setup);

To register a braille console (i. e. to call __add_preferred_console()
with non-NULL brl_options) function _braille_console_setup() should
return NULL and initialize brl_options. 

1) in this case brl_options is NULL and non-braille console is registered
2) kernel produces oops later in console_setup().  I reproduced it
   passing "console=brl=aaa" parameter to kernel, see below.
3) no console is registered

So braille console registration should not work.  What do I miss?

The code was changed in July/August 2013 in commits

commit bbeddf52adc1 ("printk: move braille console support into separate 
braille.[ch] files")
commit 2cfe6c4ac7ee ("printk: Fix return of braille_register_console()")

So it looks like braille console has not been used for more than 3 years.
Should we remote it?



Kernel command line: systemd.show_status=no rootwait rw root=/dev/sda7 
acpi=force earlycon console=brl=aaa
braille: need port name after brl=
Unable to handle kernel NULL pointer dereference at virtual address 
pgd = fc00089b
[] *pgd=001e0003, *pud=001e0003, *pmd=001e0003, 
*pte=
Internal error: Oops: 9606 [#1] PREEMPT SMP
Modules linked in:
CPU: 0 PID: 0 Comm: swapper Not tainted 4.11.0-rc2-07-00033-g417d6c6d1a16-dirty 
#727
Hardware name: Cavium ThunderX CN88XX board (DT)
task: fc0008868000 task.stack: fc000884
PC is at console_setup+0x28/0x108
LR is at console_setup+0x20/0x108
pc : [] lr : [] pstate: 80c5
sp : fc0008843e70
x29: fc0008843e70 x28:  
x27: fc00087a7b18 x26: 0001 
x25: fc000880ccb9 x24: 0008 
x23: fc0008812b40 x22:  
x21: fe1afece x20: fe1afec6 
x19:  x18: 0001 
x17:  x16: 0018 
x15: 0002 x14: fff0 
x13: fc0008873158 x12: fc00088fc82a 
x11:  x10: 002d 
x9 : 0023 x8 : 7262207265746661 
x7 : 20656d616e207472 x6 : 002e 
x5 : 0400 x4 :  
x3 : 0002 x2 : 0002 
x1 : 0001 x0 :  

Process swapper (pid: 0, stack limit = 0xfc000884)
Stack: (0xfc0008843e70 to 0xfc0008844000)
3e60:   fc0008843ec0 fc00087c0600
3e80: fc0008812330 fe1afec6 fc0008812060 
3ea0: fe1afed2 fc00087c0598 fc0008811f40 fe1afec6
3ec0: fc0008843f10 fc00080c816c fc0008623558 fe1afec6
3ee0: fc00087a7b18 fe1afece 0080 fc0008707130
3f00: 00bc fe1afed5 fc0008843fa0 fc00087c094c
3f20: fc00087f3948 fc00088d fc0008863000 fc00088d
3f40: fe1afe80 fc00087f3948 0174 001fffaf51a0
3f60: 001fffa38d00 01bc0018 fc000886e6f0 fc000870e560
3f80:  fc00087c0558  fc00087a5db8
3fa0: fc0008843ff0 fc00087c01e8 001ffa3d0a18 2120
3fc0: 

Re: [PATCH v5 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 03/15/2017 07:58 PM, Petr Mladek wrote:
> On Wed 2017-03-15 13:28:52, Aleksey Makarov wrote:
>> If a console was specified by ACPI SPCR table _and_ command line
>> parameters like "console=ttyAMA0" _and_ "earlycon" were specified,
>> then log messages appear twice.
>>
>> The root cause is that the code traverses the list of specified
>> consoles (the `console_cmdline` array) and stops at the first match.
>> But it may happen that the same console is referred by the elements
>> of this array twice:
>>
>>  pl011,mmio,0x87e02400,115200 -- from SPCR
>>  ttyAMA0 -- from command line
>>
>> but in this case `preferred_console` points to the second entry and
>> the flag CON_CONSDEV is not set, so bootconsole is not deregistered.
>>
>> To fix that, introduce an invariant "The last non-braille console
>> is always the preferred one" on the entries of the console_cmdline
>> array and don't try to check for double entries.  Then traverse it
>> in reverse order to be sure that if the console is preferred then
>> it will be the first matching entry.
>>
>> Reported-by: Sudeep Holla <sudeep.ho...@arm.com>
>> Signed-off-by: Aleksey Makarov <aleksey.maka...@linaro.org>
>> ---
>>  kernel/printk/printk.c | 45 +
>>  1 file changed, 29 insertions(+), 16 deletions(-)
>>
>> diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
>> index fd752f0c8ef1..7dc53b2831fb 100644
>> --- a/kernel/printk/printk.c
>> +++ b/kernel/printk/printk.c
>> @@ -1902,20 +1902,25 @@ static int __add_preferred_console(char *name, int 
>> idx, char *options,
>>  int i;
>>  
>>  /*
>> - *  See if this tty is not yet registered, and
>> - *  if we have a slot free.
>> + * Don't check if the console has already been registered, because it is
>> + * pointless.  After all, we can not check if two entries refer to
>> + * the same console if one is matched with console->match(), and another
>> + * by name/index:
>> + *
>> + *  pl011,mmio,0x87e02400,115200 -- added from SPCR
>> + *  ttyAMA0 -- added from command line
>> + *
>> + * Also this allows to maintain an invariant that will help to find if
>> + * the matching console is preferred, see register_console():
> 
> It is an interesting idea.
> 
> I just wonder if the check for duplicates was there for a serious
> reason. It is hard to say because it was already in the initial git
> commit. In each case, MAX_CMDLINECONSOLES is 8. There is not much
> space for duplicates.
> 
> Note that add_preferred_console() is called also from _probe() calls,
> see
> 
>uart_add_one_port() -> of_console_check()
>sunserial_console_match() -> add_preferred_console()
> 
> I wonder they might be called repeatedly, for example because
> of suspend/restore, hotplug, module load/unload.
> 
> I would feel more comfortable if we keep the check for
> duplicates here.

Now I see the problem, thank you.

> It is a pity that the console->match() calls have side effects.
> I still wonder if the 4th version might be more safe. 

I pushed v4 to the linaro git server:

https://git.linaro.org/people/aleksey.makarov/linux.git/commit/?h=amakarov/console2.19.v4=47a8227e37ca54d9cc7051abe9b3c2d072f4f75f

The problem with that approach is again a side effect.
Function match_console_name() can change newcon->index.
But it can be called in the very first pass that looks for braille console
and if the call to _braille_register_console() fails,
this newcon with changed index is passed to newcon->match().

This can be fixed by introducing a predicate that checks if the
console_cmdline entry has braille options and calling match_console_name()
only for such consoles, but I think that the code is too convoluted
and the v5 approach is better.

I am going to fix v5 preserving both the check for duplicates
and the invariant, but tell me please if you prefer the v4 approach.

> The
> newcon->setup() call is called only when the console matches.
> AFAIK, there is only one braille console. We should be on
> the safe side if this one does not implement the match()
> callback. Or is it even more complicated?

As you can see from the original code, the check for braille console
was performed in that branch of code where we missed newcon->match(),
so yes, it looks like braille console(s) does not have the match() method.
I used that in v4 to factor out matching for braille from the loop.

Thank you
Aleksey Makarov

> To be honest, I am not much familiar with the console registration
> code. I am still trying to get a better picture. It is pity that
> many function have unexpected side effects.
> 
> Best Regards,
> Petr
> --
> To unsubscribe from this list: send the line "unsubscribe linux-serial" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH v5 3/3] printk: fix double printing with earlycon

[Aleksey Makarov]

On 03/15/2017 07:58 PM, Petr Mladek wrote:
> On Wed 2017-03-15 13:28:52, Aleksey Makarov wrote:
>> If a console was specified by ACPI SPCR table _and_ command line
>> parameters like "console=ttyAMA0" _and_ "earlycon" were specified,
>> then log messages appear twice.
>>
>> The root cause is that the code traverses the list of specified
>> consoles (the `console_cmdline` array) and stops at the first match.
>> But it may happen that the same console is referred by the elements
>> of this array twice:
>>
>>  pl011,mmio,0x87e02400,115200 -- from SPCR
>>  ttyAMA0 -- from command line
>>
>> but in this case `preferred_console` points to the second entry and
>> the flag CON_CONSDEV is not set, so bootconsole is not deregistered.
>>
>> To fix that, introduce an invariant "The last non-braille console
>> is always the preferred one" on the entries of the console_cmdline
>> array and don't try to check for double entries.  Then traverse it
>> in reverse order to be sure that if the console is preferred then
>> it will be the first matching entry.
>>
>> Reported-by: Sudeep Holla 
>> Signed-off-by: Aleksey Makarov 
>> ---
>>  kernel/printk/printk.c | 45 +
>>  1 file changed, 29 insertions(+), 16 deletions(-)
>>
>> diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
>> index fd752f0c8ef1..7dc53b2831fb 100644
>> --- a/kernel/printk/printk.c
>> +++ b/kernel/printk/printk.c
>> @@ -1902,20 +1902,25 @@ static int __add_preferred_console(char *name, int 
>> idx, char *options,
>>  int i;
>>  
>>  /*
>> - *  See if this tty is not yet registered, and
>> - *  if we have a slot free.
>> + * Don't check if the console has already been registered, because it is
>> + * pointless.  After all, we can not check if two entries refer to
>> + * the same console if one is matched with console->match(), and another
>> + * by name/index:
>> + *
>> + *  pl011,mmio,0x87e02400,115200 -- added from SPCR
>> + *  ttyAMA0 -- added from command line
>> + *
>> + * Also this allows to maintain an invariant that will help to find if
>> + * the matching console is preferred, see register_console():
> 
> It is an interesting idea.
> 
> I just wonder if the check for duplicates was there for a serious
> reason. It is hard to say because it was already in the initial git
> commit. In each case, MAX_CMDLINECONSOLES is 8. There is not much
> space for duplicates.
> 
> Note that add_preferred_console() is called also from _probe() calls,
> see
> 
>uart_add_one_port() -> of_console_check()
>sunserial_console_match() -> add_preferred_console()
> 
> I wonder they might be called repeatedly, for example because
> of suspend/restore, hotplug, module load/unload.
> 
> I would feel more comfortable if we keep the check for
> duplicates here.

Now I see the problem, thank you.

> It is a pity that the console->match() calls have side effects.
> I still wonder if the 4th version might be more safe. 

I pushed v4 to the linaro git server:

https://git.linaro.org/people/aleksey.makarov/linux.git/commit/?h=amakarov/console2.19.v4=47a8227e37ca54d9cc7051abe9b3c2d072f4f75f

The problem with that approach is again a side effect.
Function match_console_name() can change newcon->index.
But it can be called in the very first pass that looks for braille console
and if the call to _braille_register_console() fails,
this newcon with changed index is passed to newcon->match().

This can be fixed by introducing a predicate that checks if the
console_cmdline entry has braille options and calling match_console_name()
only for such consoles, but I think that the code is too convoluted
and the v5 approach is better.

I am going to fix v5 preserving both the check for duplicates
and the invariant, but tell me please if you prefer the v4 approach.

> The
> newcon->setup() call is called only when the console matches.
> AFAIK, there is only one braille console. We should be on
> the safe side if this one does not implement the match()
> callback. Or is it even more complicated?

As you can see from the original code, the check for braille console
was performed in that branch of code where we missed newcon->match(),
so yes, it looks like braille console(s) does not have the match() method.
I used that in v4 to factor out matching for braille from the loop.

Thank you
Aleksey Makarov

> To be honest, I am not much familiar with the console registration
> code. I am still trying to get a better picture. It is pity that
> many function have unexpected side effects.
> 
> Best Regards,
> Petr
> --
> To unsubscribe from this list: send the line "unsubscribe linux-serial" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH v3] Revert "tty: serial: pl011: add ttyAMA for matching pl011 console"

[Aleksey Makarov]

On 03/16/2017 10:11 AM, Jayachandran C. wrote:
> Hi Greg,
> 
> On Tue, Mar 14, 2017 at 9:44 PM, Sudeep Holla <sudeep.ho...@arm.com> wrote:
>>
>>
>> On 01/03/17 15:23, Aleksey Makarov wrote:
>>> The original patch makes the condition always true, so it is wrong.
>>>
>>> It masks (but not fixes) the bug described in the commit message
>>> but introduces a regression (no console is selected by SPCR)
>>> in regular (no 'console=ttyAMA') case.
>>>
>>> s/||/&&/ would not fix the problem as the root cause was identified
>>> incorrectly.
>>>
>>> This reverts commit aea9a80ba98a0c9b4de88850260e9fbdcc98360b.
>>>
>>
>> Sorry for that, I will test your patches and respond to that. For this
>> patch:
>>
>> Acked-by: Sudeep Holla <sudeep.ho...@arm.com>
>>
> 
> This fixes a regression I see in v4.11-rc2
> 
> Tested-by: Jayachandran C <jn...@caviumnetworks.com>
> 
> I don't see it in the tty/serial tree yet

It's commit 713b93f1b849 from tty-next branch of 
https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty.git

So it looks like it is scheduled for 4.12

Greg, this is a fix for regression.  Can it be applied to 4.11-rcX?

Thank you
Aleksey Makarov

> 
> JC.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-serial" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH v3] Revert "tty: serial: pl011: add ttyAMA for matching pl011 console"

[Aleksey Makarov]

On 03/16/2017 10:11 AM, Jayachandran C. wrote:
> Hi Greg,
> 
> On Tue, Mar 14, 2017 at 9:44 PM, Sudeep Holla  wrote:
>>
>>
>> On 01/03/17 15:23, Aleksey Makarov wrote:
>>> The original patch makes the condition always true, so it is wrong.
>>>
>>> It masks (but not fixes) the bug described in the commit message
>>> but introduces a regression (no console is selected by SPCR)
>>> in regular (no 'console=ttyAMA') case.
>>>
>>> s/||/&&/ would not fix the problem as the root cause was identified
>>> incorrectly.
>>>
>>> This reverts commit aea9a80ba98a0c9b4de88850260e9fbdcc98360b.
>>>
>>
>> Sorry for that, I will test your patches and respond to that. For this
>> patch:
>>
>> Acked-by: Sudeep Holla 
>>
> 
> This fixes a regression I see in v4.11-rc2
> 
> Tested-by: Jayachandran C 
> 
> I don't see it in the tty/serial tree yet

It's commit 713b93f1b849 from tty-next branch of 
https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty.git

So it looks like it is scheduled for 4.12

Greg, this is a fix for regression.  Can it be applied to 4.11-rcX?

Thank you
Aleksey Makarov

> 
> JC.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-serial" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

[PATCH v5 2/3] printk: rename selected_console -> preferred_console

[Aleksey Makarov]

The variable selected_console is set in __add_preferred_console()
to point to the last console parameter that was added to the
console_cmdline array.

Rename it to preferred_console so that the name reflects the usage.

Petr Mladek:
"[..] the selected_console/preferred_console
value is used to keep the console first in the console_drivers list.
IMHO, the main effect is that each line will first appear on this
console, see call_console_drivers(). But the message will still
appear also on all other enabled consoles. From this point,
the name "preferred" sounds better to me. More consoles
are selected (enabled) and only one is preferred (first)."

Acked-by: Steven Rostedt (VMware) <rost...@goodmis.org>
Acked-by: Petr Mladek <pmla...@suse.com>
Reviewed-by: Sergey Senozhatsky <sergey.senozhat...@gmail.com>
Suggested-by: Peter Hurley <pe...@hurleysoftware.com>
Signed-off-by: Aleksey Makarov <aleksey.maka...@linaro.org>
---
 kernel/printk/printk.c | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index 80bc1b72d03d..fd752f0c8ef1 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -270,7 +270,7 @@ static struct console *exclusive_console;
 
 static struct console_cmdline console_cmdline[MAX_CMDLINECONSOLES];
 
-static int selected_console = -1;
+static int preferred_console = -1;
 int console_set_on_cmdline;
 EXPORT_SYMBOL(console_set_on_cmdline);
 
@@ -1910,14 +1910,14 @@ static int __add_preferred_console(char *name, int idx, 
char *options,
 i++, c++) {
if (strcmp(c->name, name) == 0 && c->index == idx) {
if (!brl_options)
-   selected_console = i;
+   preferred_console = i;
return 0;
}
}
if (i == MAX_CMDLINECONSOLES)
return -E2BIG;
if (!brl_options)
-   selected_console = i;
+   preferred_console = i;
strlcpy(c->name, name, sizeof(c->name));
c->options = options;
braille_set_options(c, brl_options);
@@ -2436,7 +2436,7 @@ void register_console(struct console *newcon)
bcon = console_drivers;
 
if (!has_preferred || bcon || !console_drivers)
-   has_preferred = selected_console >= 0;
+   has_preferred = preferred_console >= 0;
 
/*
 *  See if we want to use this console driver. If we
@@ -2484,7 +2484,7 @@ void register_console(struct console *newcon)
}
 
newcon->flags |= CON_ENABLED;
-   if (i == selected_console) {
+   if (i == preferred_console) {
newcon->flags |= CON_CONSDEV;
has_preferred = true;
}
-- 
2.12.0

[PATCH v5 2/3] printk: rename selected_console -> preferred_console

[Aleksey Makarov]

The variable selected_console is set in __add_preferred_console()
to point to the last console parameter that was added to the
console_cmdline array.

Rename it to preferred_console so that the name reflects the usage.

Petr Mladek:
"[..] the selected_console/preferred_console
value is used to keep the console first in the console_drivers list.
IMHO, the main effect is that each line will first appear on this
console, see call_console_drivers(). But the message will still
appear also on all other enabled consoles. From this point,
the name "preferred" sounds better to me. More consoles
are selected (enabled) and only one is preferred (first)."

Acked-by: Steven Rostedt (VMware) 
Acked-by: Petr Mladek 
Reviewed-by: Sergey Senozhatsky 
Suggested-by: Peter Hurley 
Signed-off-by: Aleksey Makarov 
---
 kernel/printk/printk.c | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index 80bc1b72d03d..fd752f0c8ef1 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -270,7 +270,7 @@ static struct console *exclusive_console;
 
 static struct console_cmdline console_cmdline[MAX_CMDLINECONSOLES];
 
-static int selected_console = -1;
+static int preferred_console = -1;
 int console_set_on_cmdline;
 EXPORT_SYMBOL(console_set_on_cmdline);
 
@@ -1910,14 +1910,14 @@ static int __add_preferred_console(char *name, int idx, 
char *options,
 i++, c++) {
if (strcmp(c->name, name) == 0 && c->index == idx) {
if (!brl_options)
-   selected_console = i;
+   preferred_console = i;
return 0;
}
}
if (i == MAX_CMDLINECONSOLES)
return -E2BIG;
if (!brl_options)
-   selected_console = i;
+   preferred_console = i;
strlcpy(c->name, name, sizeof(c->name));
c->options = options;
braille_set_options(c, brl_options);
@@ -2436,7 +2436,7 @@ void register_console(struct console *newcon)
bcon = console_drivers;
 
if (!has_preferred || bcon || !console_drivers)
-   has_preferred = selected_console >= 0;
+   has_preferred = preferred_console >= 0;
 
/*
 *  See if we want to use this console driver. If we
@@ -2484,7 +2484,7 @@ void register_console(struct console *newcon)
}
 
newcon->flags |= CON_ENABLED;
-   if (i == selected_console) {
+   if (i == preferred_console) {
newcon->flags |= CON_CONSDEV;
has_preferred = true;
}
-- 
2.12.0