Hi, I think I managed to get around this problem.
in function init_hw_timer():
----
case e1000_i350:
case e1000_82580:
/*
* The 82580 timesync updates the system timer every 8ns by 8ns
* and the value cannot be shifted. Instead we need to shift
* the registers to generate a 64bit timer value. As a result
* SYSTIMR/L/H, TXSTMPL/H, RXSTMPL/H all have to be shifted by
* 24 in order to generate a larger value for synchronization.
*/
adapter->cycles.shift = IGB_82580_TSYNC_SHIFT;
----
I have the I350 and I dont think this cycles shift was meant for my adapter.
IGB_82580_TSYNC_SHIFT is 24.
Also when doing igb_read_clock,
this shifting is applied only for the 82580 and I guess it should not have
been set in the first place for the I350:
--------
/*
* The timestamp latches on lowest register read. For the 82580
* the lowest register is SYSTIMR instead of SYSTIML. However we never
* adjusted TIMINCA so SYSTIMR will just read as all 0s so ignore it.
*/
if (hw->mac.type == e1000_82580) {
stamp = rd32(E1000_SYSTIMR) >> 8;
shift = IGB_82580_TSYNC_SHIFT;
}
stamp |= (u64)rd32(E1000_SYSTIML) << shift;
stamp |= (u64)rd32(E1000_SYSTIMH) << (shift + 32);
-------------
Setting adapter->cycles.shift to 0 makes the clock run at about the same
speed as the system clock.
Next problem I have is that the syststamp is not syncing correctly against
the system clock.
Any help is much appreciated.
Regards,
Mattias
On Mon, Oct 12, 2015 at 5:04 PM, Mattias Barthel <[email protected]>
wrote:
> Hi again,
>
> Started to try and timestamp all packets in the I350.
>
> Driver version:
>
> driver: igb
> version: 3.0.6-k2
> firmware-version: 0.147-0
>
> Doing an ioctl that turns on timestamping for all packets.
>
> hwconfig.tx_type = HWTSTAMP_TX_OFF;
> hwconfig.rx_filter = HWTSTAMP_FILTER_ALL;
> hwconfig_requested = hwconfig;
> ioctl(sock, SIOCSHWTSTAMP, &hwtstamp)
>
> The strangest thing is that although it works to read the timestamps the
> clock seems to advance very slowly and/or incorrectly.
>
> I just printout the HW timestamps alongside the systemclock and the hw
> stamping is advancing the nanoseconds when the systemclock is advancing us
> and seconds.
>
> Is there a know bug in this version of the driver that maybe initializes
> the clock incorrectly or maybe reads incorrectly from the clock?
>
>
> [ 63.004116] igb_rx_hwtstamp
> [ 63.004684] hwtstamp: 1444656871372825302, syststamp:
> 1444656871372845807
> [ 63.005721] system timestamp: tv_sec: 1444656793, tv_usec: 683825
> [ 63.006631] sys hw timestamp: tstamp: 1444656871372845807, tv_sec:
> 1444656871, tv_usec: 372845
> [ 63.008044] hw timestamp: tstamp: 1444656871372825302, tv_sec:
> 1444656871, tv_usec: 372825
> [ 63.009395]
> [ 63.104111] igb_rx_hwtstamp
> [ 63.104704] hwtstamp: 1444656871372825308, syststamp:
> 1444656871372845813
> [ 63.105747] system timestamp: tv_sec: 1444656793, tv_usec: 783851
> [ 63.106669] sys hw timestamp: tstamp: 1444656871372845813, tv_sec:
> 1444656871, tv_usec: 372845
> [ 63.108075] hw timestamp: tstamp: 1444656871372825308, tv_sec:
> 1444656871, tv_usec: 372825
> [ 63.109428]
> [ 63.204109] igb_rx_hwtstamp
> [ 63.204683] hwtstamp: 1444656871372825314, syststamp:
> 1444656871372845819
> [ 63.206018] system timestamp: tv_sec: 1444656793, tv_usec: 884122
> [ 63.207242] sys hw timestamp: tstamp: 1444656871372845819, tv_sec:
> 1444656871, tv_usec: 372845
> [ 63.209096] hw timestamp: tstamp: 1444656871372825314, tv_sec:
> 1444656871, tv_usec: 372825
> [ 63.210897]
> [ 63.304068] igb_rx_hwtstamp
> [ 63.304849] hwtstamp: 1444656871372825320, syststamp:
> 1444656871372845825
> [ 63.306176] system timestamp: tv_sec: 1444656793, tv_usec: 984280
> [ 63.307391] sys hw timestamp: tstamp: 1444656871372845825, tv_sec:
> 1444656871, tv_usec: 372845
> [ 63.309252] hw timestamp: tstamp: 1444656871372825320, tv_sec:
> 1444656871, tv_usec: 372825
> [ 63.311032]
> [ 63.404107] igb_rx_hwtstamp
> [ 63.404659] hwtstamp: 1444656871372825326, syststamp:
> 1444656871372845831
> [ 63.405695] system timestamp: tv_sec: 1444656794, tv_usec: 83798
> [ 63.406635] sys hw timestamp: tstamp: 1444656871372845831, tv_sec:
> 1444656871, tv_usec: 372845
> [ 63.408044] hw timestamp: tstamp: 1444656871372825326, tv_sec:
> 1444656871, tv_usec: 372825
> [ 63.409405]
> [ 63.504084] igb_rx_hwtstamp
> [ 63.504670] hwtstamp: 1444656871372825332, syststamp:
> 1444656871372845837
> [ 63.505715] system timestamp: tv_sec: 1444656794, tv_usec: 183819
> [ 63.506657] sys hw timestamp: tstamp: 1444656871372845837, tv_sec:
> 1444656871, tv_usec: 372845
> [ 63.508090] hw timestamp: tstamp: 1444656871372825332, tv_sec:
> 1444656871, tv_usec: 372825
>
>
> Any help will be very much appreciated.
>
> Best regards,
>
> Mattias
>
>
>
>
>
>
>
>
> On Thu, Oct 8, 2015 at 6:01 PM, Alexander Duyck <[email protected]
> > wrote:
>
>> The i350 should support timestamping all packets as I recall. You should
>> probably look into getting the linuxptp package and seeing if you can make
>> use of the hwstamp_ctl command to enable timestamping for all incoming
>> packets on the adapter.
>>
>> - Alex
>>
>> On 10/08/2015 12:33 AM, Mattias Barthel wrote:
>>
>>> Hi
>>> thanks for answer.
>>>
>>> Unfortunately, the udp/ip stream just as the cfm stream is against only
>>> one peer.
>>> Furthemore on the guest side I have only one vcpu tied to two physical
>>> siblings to minimize L2 cache misses on ISRs due to cache coherence.
>>>
>>> I had an idea this morning, maybe I can make use of the ptp hw
>>> timestaming for this? This should give better accuracy.
>>> Is this possible or does it only work for ethertype ptp?
>>>
>>> On Thursday, October 8, 2015, Alexander Duyck <[email protected]
>>> <mailto:[email protected]>> wrote:
>>>
>>> The difference could be Receive Side Scaling (RSS). The UDP/IP
>>> packets
>>> can be load balanced across all queues via RSS based on a hash of the
>>> source and destination addresses. So if you have multiple sources
>>> generating these packets, or they are being received on multiple IP
>>> addresses then it is possible the load is being spread out over
>>> multiple
>>> CPUs. Your CFM packets are on an Ethertype other than IPv4 or IPv6 so
>>> all packets will end up on queue 0 if I am not mistaken.
>>>
>>> - Alex
>>>
>>> On 10/07/2015 10:21 PM, Mattias Barthel wrote:
>>> > Hi again!
>>> >
>>> > I realise my message might not have been so clear.
>>> >
>>> > I will try to clarify. I am doing PM with software timestamping.
>>> >
>>> > igb versions:
>>> > version: 3.0.6-k2
>>> > firmware-version: 0.147-0
>>> >
>>> > I do timestamping (NTP format) of the packets in the same place
>>> for all
>>> > protocols. (in igb_clean_rx_irq_adv()).
>>> >
>>> > For these non ip packets like IEEE 801.2ag CFM ETH-DM
>>> (ethertype 0x8902) I
>>> > get quite a lot worse accuracy in the timestamping.
>>> >
>>> > For UDP/IP TWAMP I get for 10k pps a max delay of around 140us
>>> back to back.
>>> > For CFM ETH DM I get for the same packet rate and packet size a
>>> max delay
>>> > of around 900us.
>>> >
>>> > So my hypothesis was:
>>> > Could FW be putting these L2 packets in another queue with
>>> different
>>> > characteristics?
>>> >
>>> > I tried to add a ET-type filter as written in my previous mail
>>> but it
>>> > showed no difference.
>>> >
>>> >
>>> > Thanks for any help given!
>>> >
>>> >
>>> >
>>> > On Mon, Oct 5, 2015 at 2:19 PM, Mattias Barthel
>>> <[email protected] <javascript:;>>
>>>
>>> > wrote:
>>> >
>>> >> Greetings,
>>> >>
>>> >> Im getting quite bad latency when using the igb driver on i350
>>> on linux
>>> >> regarding
>>> >> ETH CFM packets. This compared to TWAMP (IPv4 UDP packets).
>>> >> The environment is KVM with the i350 devices in PCI-passthrough.
>>> >>
>>> >> So i figured I add an own rx-queue (filter) for those types of
>>> ethernet
>>> >> protocol packets.
>>> >>
>>> >> This is what i added:
>>> >>
>>> >>
>>> >> wr32(E1000_ETQF(4),
>>> >> (1 << 31) | /* queue enable */
>>> >> (E1000_ETQF_FILTER_ENABLE) | /* enable filter */
>>> >> (1 << 29) | /* enable immediate interrupt */
>>> >> (0x4 << 16) | /* queue no. 4 */
>>> >> (ETH_P_8021AG)); /* 0x8902 CFM eth protocol type */
>>> >>
>>> >>
>>> >> ETH_P_8021A is 0x8902
>>> >>
>>> >>
>>> >>
>>> >> Is this a correct/good approach?
>>> >>
>>> >> Thanks in advance!
>>> >>
>>> >> --
>>> >> Mattias Barthel
>>> >>
>>> >
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> _______________________________________________
>>> E1000-devel mailing list
>>> [email protected] <javascript:;>
>>> https://lists.sourceforge.net/lists/listinfo/e1000-devel
>>> To learn more about Intel® Ethernet, visit
>>> http://communities.intel.com/community/wired
>>>
>>>
>>> Avis de confidentialité
>>>
>>> Les informations contenues dans le présent message et dans toute pièce
>>> qui lui est jointe sont confidentielles et peuvent être protégées par le
>>> secret professionnel. Ces informations sont à l’usage exclusif de son ou de
>>> ses destinataires. Si vous recevez ce message par erreur, veuillez s’il
>>> vous plait communiquer immédiatement avec l’expéditeur et en détruire tout
>>> exemplaire. De plus, il vous est strictement interdit de le divulguer, de
>>> le distribuer ou de le reproduire sans l’autorisation de l’expéditeur.
>>> Merci.
>>>
>>> Confidentiality notice
>>>
>>> This e-mail message and any attachment hereto contain confidential
>>> information which may be privileged and which is intended for the exclusive
>>> use of its addressee(s). If you receive this message in error, please
>>> inform sender immediately and destroy any copy thereof. Furthermore, any
>>> disclosure, distribution or copying of this message and/or any attachment
>>> hereto without the consent of the sender is strictly prohibited. Thank you.
>>>
>>>
>>
>
>
> --
> Mattias Barthel
>
--
Mattias Barthel
------------------------------------------------------------------------------
_______________________________________________
E1000-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit
http://communities.intel.com/community/wired
