You can probably just download and use the latest driver from:
http://sourceforge.net/projects/e1000/files/igb%20stable/
It should be able to build on a 3.0.8 kernel provided nothing has been
back-ported that would change the APIs.
- Alex
On 10/13/2015 10:26 AM, Mattias Barthel wrote:
> Thanks a lot Alex, there might still be some hope then.
> Unfortunately Im bound to using the linux kernel 3.0.8. What
> out-of-tree driver can I use with that, please?
>
> On Tuesday, October 13, 2015, Alexander Duyck
> <[email protected] <mailto:[email protected]>> wrote:
>
> I don't know if it is available in that driver or not, but there
> is an option in later igb drivers to support a time stamp being
> written with the packet data. Based on the fact that you are
> already seeing issues, you might want to try getting the
> out-of-tree driver from e1000.sf.net <http://e1000.sf.net> and
> seeing if you can use that with the time stamp in packet option
> that is enabled when you select to timestamp all incoming Rx packets.
>
> - Alex
>
> On 10/13/2015 10:01 AM, Mattias Barthel wrote:
>
> Yes, setting the cycles.shift to the same as on the 82580 and
> also shifting the reads accordingly everything works. The sync
> of shhwtstamps->syststamp too.
>
> My driver version is 3.0.6-k2 and linux is 3.0.8.
>
> Although this might not be very suited to my needs.
> The incoming buffers wont be timestamped if you cant keep up
> with reading the timestamping registers as I understand it.
> This might be perfectly fine in the case of PTP where the
> packet rate is fairly low. In my case Im looking at some
> 80kpps that all would need to be timestamped.
> There is a comment about this in the driver:
>
> *//* If this bit is set, then the RX registers contain the
> time stamp. No/**/ other packet will be time stamped until we
> read these registers, so/* */read the registers to make them
> available again. Because only one/**/ packet can be time
> stamped at a time, we know that the register/* */values must
> belong to this one here and therefore we don't need to/**/
> compare any of the additional attributes stored for it./* */If
> nothing went wrong, then it should have a shared tx_flags that
> we/**/ can turn into a skb_shared_hwtstamps. *//*
> *//*
> *//*
>
> On Tuesday, October 13, 2015, Alexander Duyck
> <[email protected] <mailto:[email protected]>>
> wrote:
>
> I'm assuming you are using either an older kernel or an
> out-of-tree driver do I have that right? I ask because
> this code
> doesn't resemble what is is currently in the latest Linux
> kernel.
>
> Comments on what you have found are inline below.
>
> - Alex
>
> On 10/13/2015 12:53 AM, Mattias Barthel wrote:
>
> Hi, I think I managed to get around this problem.
>
>
> in function init_hw_timer():
> ----
> case e1000_i350:
> case e1000_82580:
>
> /*
> * The 82580 timesync updates the system timer every
> 8ns by 8ns
> * and the value cannot be shifted. Instead we need to
> shift
> * the registers to generate a 64bit timer value. As a
> result
> * SYSTIMR/L/H, TXSTMPL/H, RXSTMPL/H all have to be
> shifted by
> * 24 in order to generate a larger value for
> synchronization.
> */
> adapter->cycles.shift = IGB_82580_TSYNC_SHIFT;
>
> ----
>
> I have the I350 and I dont think this cycles shift was
> meant
> for my adapter.
> IGB_82580_TSYNC_SHIFT is 24.
>
> Also when doing igb_read_clock,
> this shifting is applied only for the 82580 and I guess it
> should not have been set in the first place for the I350:
> --------
> /*
> * The timestamp latches on lowest register read. For
> the 82580
> * the lowest register is SYSTIMR instead of SYSTIML.
> However
> we never
> * adjusted TIMINCA so SYSTIMR will just read as all 0s so
> ignore it.
> */
> if (hw->mac.type == e1000_82580) {
> stamp = rd32(E1000_SYSTIMR) >> 8;
> shift = IGB_82580_TSYNC_SHIFT;
> }
>
> stamp |= (u64)rd32(E1000_SYSTIML) << shift;
> stamp |= (u64)rd32(E1000_SYSTIMH) << (shift + 32);
> -------------
>
>
> This is a bug in the code. The i350 should be shifted by
> the same
> value as the 82580. Then the result would be the same as
> if you
> had set the shift to 0. As I recall the reasoning behind the
> change was to make use of as many bits of the time as
> possible,
> and on 82580 and newer the SYSTIMH portion of the clock only
> contained 8 bits of actual clock data.
>
> Without that change I believe there is a risk of the
> counter being
> considered to have cycled too fast and causing errors as
> the cycle
> counter will roll over at 40 bits instead of 64. The
> alternative
> would be to update the cycle counter you are using so that
> it is
> aware that it is limited to 40 bits and then use an shift
> of 0.
>
> Setting adapter->cycles.shift to 0 makes the clock run at
> about the same speed as the system clock.
>
> Next problem I have is that the syststamp is not syncing
> correctly against the system clock.
>
> Any help is much appreciated.
>
> Regards,
>
> Mattias
>
>
>
>
>
>
>
>
>
>
>
>
> On Mon, Oct 12, 2015 at 5:04 PM, Mattias Barthel
> <[email protected]
> <mailto:[email protected]>>
> wrote:
>
> Hi again,
>
> Started to try and timestamp all packets in the I350.
>
> Driver version:
>
> driver: igb
> version: 3.0.6-k2
> firmware-version: 0.147-0
>
> Doing an ioctl that turns on timestamping for all
> packets.
>
> hwconfig.tx_type = HWTSTAMP_TX_OFF;
> hwconfig.rx_filter = HWTSTAMP_FILTER_ALL;
> hwconfig_requested = hwconfig;
> ioctl(sock, SIOCSHWTSTAMP, &hwtstamp)
>
> The strangest thing is that although it works to
> read the
> timestamps the clock seems to advance very slowly
> and/or
> incorrectly.
>
> I just printout the HW timestamps alongside the
> systemclock and
> the hw stamping is advancing the nanoseconds when the
> systemclock
> is advancing us and seconds.
>
> Is there a know bug in this version of the driver
> that maybe
> initializes the clock incorrectly or maybe reads
> incorrectly from
> the clock?
>
>
> [ 63.004116] igb_rx_hwtstamp
> [ 63.004684] hwtstamp: 1444656871372825302, syststamp:
> 1444656871372845807
> [ 63.005721] system timestamp: tv_sec: 1444656793,
> tv_usec: 683825
> [ 63.006631] sys hw timestamp: tstamp:
> 1444656871372845807,
> tv_sec: 1444656871, tv_usec: 372845
> [ 63.008044] hw timestamp: tstamp:
> 1444656871372825302,
> tv_sec:
> 1444656871, tv_usec: 372825
> [ 63.009395]
> [ 63.104111] igb_rx_hwtstamp
> [ 63.104704] hwtstamp: 1444656871372825308, syststamp:
> 1444656871372845813
> [ 63.105747] system timestamp: tv_sec: 1444656793,
> tv_usec: 783851
> [ 63.106669] sys hw timestamp: tstamp:
> 1444656871372845813,
> tv_sec: 1444656871, tv_usec: 372845
> [ 63.108075] hw timestamp: tstamp:
> 1444656871372825308,
> tv_sec:
> 1444656871, tv_usec: 372825
> [ 63.109428]
> [ 63.204109] igb_rx_hwtstamp
> [ 63.204683] hwtstamp: 1444656871372825314, syststamp:
> 1444656871372845819
> [ 63.206018] system timestamp: tv_sec: 1444656793,
> tv_usec: 884122
> [ 63.207242] sys hw timestamp: tstamp:
> 1444656871372845819,
> tv_sec: 1444656871, tv_usec: 372845
> [ 63.209096] hw timestamp: tstamp:
> 1444656871372825314,
> tv_sec:
> 1444656871, tv_usec: 372825
> [ 63.210897]
> [ 63.304068] igb_rx_hwtstamp
> [ 63.304849] hwtstamp: 1444656871372825320, syststamp:
> 1444656871372845825
> [ 63.306176] system timestamp: tv_sec: 1444656793,
> tv_usec: 984280
> [ 63.307391] sys hw timestamp: tstamp:
> 1444656871372845825,
> tv_sec: 1444656871, tv_usec: 372845
> [ 63.309252] hw timestamp: tstamp:
> 1444656871372825320,
> tv_sec:
> 1444656871, tv_usec: 372825
> [ 63.311032]
> [ 63.404107] igb_rx_hwtstamp
> [ 63.404659] hwtstamp: 1444656871372825326, syststamp:
> 1444656871372845831
> [ 63.405695] system timestamp: tv_sec: 1444656794,
> tv_usec: 83798
> [ 63.406635] sys hw timestamp: tstamp:
> 1444656871372845831,
> tv_sec: 1444656871, tv_usec: 372845
> [ 63.408044] hw timestamp: tstamp:
> 1444656871372825326,
> tv_sec:
> 1444656871, tv_usec: 372825
> [ 63.409405]
> [ 63.504084] igb_rx_hwtstamp
> [ 63.504670] hwtstamp: 1444656871372825332, syststamp:
> 1444656871372845837
> [ 63.505715] system timestamp: tv_sec: 1444656794,
> tv_usec: 183819
> [ 63.506657] sys hw timestamp: tstamp:
> 1444656871372845837,
> tv_sec: 1444656871, tv_usec: 372845
> [ 63.508090] hw timestamp: tstamp:
> 1444656871372825332,
> tv_sec:
> 1444656871, tv_usec: 372825
>
>
> Any help will be very much appreciated.
>
> Best regards,
>
> Mattias
>
>
>
>
>
>
>
>
> On Thu, Oct 8, 2015 at 6:01 PM, Alexander Duyck
> <[email protected]
> <mailto:[email protected]>> wrote:
>
> The i350 should support timestamping all
> packets as I
> recall. You should probably look into getting the
> linuxptp package and
> seeing if you can make use of the hwstamp_ctl
> command to
> enable timestamping for all incoming packets
> on the
> adapter.
>
> - Alex
>
> On 10/08/2015 12:33 AM, Mattias Barthel wrote:
>
> Hi
> thanks for answer.
>
> Unfortunately, the udp/ip stream just as
> the cfm
> stream is
> against only one peer.
> Furthemore on the guest side I have only
> one vcpu
> tied to
> two physical siblings to minimize L2 cache
> misses
> on ISRs
> due to cache coherence.
>
> I had an idea this morning, maybe I can
> make use
> of the
> ptp hw timestaming for this? This should give
> better accuracy.
> Is this possible or does it only work for
> ethertype ptp?
>
> On Thursday, October 8, 2015, Alexander Duyck
> <[email protected]
> <mailto:[email protected]>
> <mailto:[email protected]
> <mailto:[email protected]>>> wrote:
>
> The difference could be Receive Side
> Scaling
> (RSS). The UDP/IP
> packets
> can be load balanced across all queues
> via RSS
> based
> on a hash of the
> source and destination addresses. So
> if you have
> multiple sources
> generating these packets, or they are
> being
> received
> on multiple IP
> addresses then it is possible the load is
> being spread
> out over
> multiple
> CPUs. Your CFM packets are on an Ethertype
> other than
> IPv4 or IPv6 so
> all packets will end up on queue 0 if
> I am not
> mistaken.
>
> - Alex
>
> On 10/07/2015 10:21 PM, Mattias
> Barthel wrote:
> > Hi again!
> >
> > I realise my message might not have
> been so
> clear.
> >
> > I will try to clarify. I am doing PM
> with
> software
> timestamping.
> >
> > igb versions:
> > version: 3.0.6-k2
> > firmware-version: 0.147-0
> >
> > I do timestamping (NTP format) of the
> packets in the
> same place
> for all
> > protocols. (in igb_clean_rx_irq_adv()).
> >
> > For these non ip packets like IEEE
> 801.2ag CFM ETH-DM
> (ethertype 0x8902) I
> > get quite a lot worse accuracy in the
> timestamping.
> >
> > For UDP/IP TWAMP I get for 10k pps a max
> delay of
> around 140us
> back to back.
> > For CFM ETH DM I get for the same packet
> rate and
> packet size a
> max delay
> > of around 900us.
> >
> > So my hypothesis was:
> > Could FW be putting these L2 packets
> in another
> queue with different
> > characteristics?
> >
> > I tried to add a ET-type filter as
> written in my
> previous mail
> but it
> > showed no difference.
> >
> >
> > Thanks for any help given!
> >
> >
> >
> > On Mon, Oct 5, 2015 at 2:19 PM,
> Mattias Barthel
> <[email protected]
> <mailto:[email protected]>
> <javascript:;>>
>
> > wrote:
> >
> >> Greetings,
> >>
> >> Im getting quite bad latency when
> using the igb
> driver on i350
> on linux
> >> regarding
> >> ETH CFM packets. This compared to TWAMP
> (IPv4 UDP
> packets).
> >> The environment is KVM with the
> i350 devices in
> PCI-passthrough.
> >>
> >> So i figured I add an own rx-queue
> (filter) for
> those types of
> ethernet
> >> protocol packets.
> >>
> >> This is what i added:
> >>
> >>
> >> wr32(E1000_ETQF(4),
> >> (1 << 31) | /* queue enable */
> >> (E1000_ETQF_FILTER_ENABLE) | /* enable
> filter */
> >> (1 << 29) | /* enable immediate
> interrupt */
> >> (0x4 << 16) | /* queue no. 4 */
> >> (ETH_P_8021AG)); /* 0x8902
> CFM eth
> protocol type */
> >>
> >>
> >> ETH_P_8021A is 0x8902
> >>
> >>
> >>
> >> Is this a correct/good approach?
> >>
> >> Thanks in advance!
> >>
> >> --
> >> Mattias Barthel
> >>
> >
>
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> E1000-devel mailing list
> [email protected]
> <mailto:[email protected]>
> <javascript:;>
> https://lists.sourceforge.net/lists/listinfo/e1000-devel
> To learn more about Intel®
> Ethernet, visit
> http://communities.intel.com/community/wired
>
>
> Avis de confidentialité
>
> Les informations contenues dans le présent
> message
> et dans
> toute pièce qui lui est jointe sont
> confidentielles et
> peuvent être protégées par le secret
> professionnel. Ces
> informations sont à l’usage exclusif de
> son ou de ses
> destinataires. Si vous recevez ce message
> par erreur,
> veuillez s’il vous plait communiquer
> immédiatement
> avec
> l’expéditeur et en détruire tout
> exemplaire. De
> plus, il
> vous est strictement interdit de le
> divulguer, de le
> distribuer ou de le reproduire sans
> l’autorisation de
> l’expéditeur. Merci.
>
> Confidentiality notice
>
> This e-mail message and any attachment
> hereto contain
> confidential information which may be
> privileged
> and which
> is intended for the exclusive use of its
> addressee(s). If
> you receive this message in error, please
> inform
> sender
> immediately and destroy any copy thereof.
> Furthermore, any
> disclosure, distribution or copying of this
> message and/or
> any attachment hereto without the consent
> of the
> sender is
> strictly prohibited. Thank you.
>
>
>
>
>
> -- Mattias Barthel
>
>
>
>
> -- Mattias Barthel
>
>
>
> Avis de confidentialité
>
> Les informations contenues dans le présent message et dans
> toute pièce qui lui est jointe sont confidentielles et peuvent
> être protégées par le secret professionnel. Ces informations
> sont à l’usage exclusif de son ou de ses destinataires. Si
> vous recevez ce message par erreur, veuillez s’il vous plait
> communiquer immédiatement avec l’expéditeur et en détruire
> tout exemplaire. De plus, il vous est strictement interdit de
> le divulguer, de le distribuer ou de le reproduire sans
> l’autorisation de l’expéditeur. Merci.
>
> Confidentiality notice
>
> This e-mail message and any attachment hereto contain
> confidential information which may be privileged and which is
> intended for the exclusive use of its addressee(s). If you
> receive this message in error, please inform sender
> immediately and destroy any copy thereof. Furthermore, any
> disclosure, distribution or copying of this message and/or any
> attachment hereto without the consent of the sender is
> strictly prohibited. Thank you.
>
>
>
> Avis de confidentialité
>
> Les informations contenues dans le présent message et dans toute pièce
> qui lui est jointe sont confidentielles et peuvent être protégées par
> le secret professionnel. Ces informations sont à l’usage exclusif de
> son ou de ses destinataires. Si vous recevez ce message par erreur,
> veuillez s’il vous plait communiquer immédiatement avec l’expéditeur
> et en détruire tout exemplaire. De plus, il vous est strictement
> interdit de le divulguer, de le distribuer ou de le reproduire sans
> l’autorisation de l’expéditeur. Merci.
>
> Confidentiality notice
>
> This e-mail message and any attachment hereto contain confidential
> information which may be privileged and which is intended for the
> exclusive use of its addressee(s). If you receive this message in
> error, please inform sender immediately and destroy any copy thereof.
> Furthermore, any disclosure, distribution or copying of this message
> and/or any attachment hereto without the consent of the sender is
> strictly prohibited. Thank you.
>
------------------------------------------------------------------------------
_______________________________________________
E1000-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit
http://communities.intel.com/community/wired
