Re: [very-RFC 0/8] TSN driver for the kernel
On 2016年06月20日 18:06, Henrik Austad wrote: On Sun, Jun 19, 2016 at 11:45:47PM +0900, Takashi Sakamoto wrote: (remove C.C. to lkml. This is not so major feature.) On Jun 19 2916 07:45, Henrik Austad wrote: snip 802.1Q gives you low latency through the network, but more importantly, no dropped frames. gPTP gives you a central reference to time. When such a long message is required, it means that we don't have enough premises for this discussion. Isn't a discussion part of how information is conveyed and finding parts that require more knowledge? You have just interests in gPTP and transferring AVTPDUs, while no interests in the others such as "what the basic ideas of TSN come from" and "the reason that IEEE 1722 refers to IEC 61883 series which is originally designed for IEEE 1394 bus" and "the reason that I was motivated to join in this discussion even though not a netdev developer". I'm sorry, I'm not sure I follow you here. What do you mean I don't have any interest in where TSN comes from? or the reason why 1722 use IEC 61883? What about "they picked 61883 because it made sense?" gPTP itself is *not* about transffering audio-data, it is about agreeing on a common time so that when you *do* transfer audio-data, the samplerate actually means something. Let me ask you this; if you have 2 sound-cards in your computer and you want to attach a mic to one and speakers to the other, how do you solve streaming of audio from the mic to the speaker If you answer does not contain something akin to "different timing-domain issues", I'd be very surprised. If you are interested in TSN for transferring *anything*, _including_ audio, you *have* to take gPTP into consideration. Just as you have to think about stream reservation, compliant hardware and all the different subsystems you are going to run into, either via kernel or via userspace. Here, could I ask you a question? Do you know a role of cycle start packet of IEEE Std 1394? No, I do not. I have only passing knowledge of the firewire standard, I've looked at the encoding described in 1722 and added that to the alsa shim as an example of how to use TSN. As I stated, this was a *very* early version and I would like to use TSN for audio - and more work is needed. If you think it's not related to this discussion, please tell it to me. Then I'll drop out from this thread. There are tons of details left and right, and as I said, I'm not all to familiar with firewire. I know that one of the authors behind the firewire standard happened to be part of 1722 standard. I am currently working my way through the firewire-stak paper you've written, and I have gotten a lot of pointers to other areas I need to dig into so I should be busy for a while. That being said, Richard's point about a way to find sample-rate of a hardware device and ways to influence that, is important for AVB/TSN. History Repeats itself. ? OK. Bye. Takashi Sakamoto
Re: [very-RFC 0/8] TSN driver for the kernel
(remove C.C. to lkml. This is not so major feature.) On Jun 19 2916 07:45, Henrik Austad wrote: snip 802.1Q gives you low latency through the network, but more importantly, no dropped frames. gPTP gives you a central reference to time. When such a long message is required, it means that we don't have enough premises for this discussion. You have just interests in gPTP and transferring AVTPDUs, while no interests in the others such as "what the basic ideas of TSN come from" and "the reason that IEEE 1722 refers to IEC 61883 series which is originally designed for IEEE 1394 bus" and "the reason that I was motivated to join in this discussion even though not a netdev developer". Here, could I ask you a question? Do you know a role of cycle start packet of IEEE Std 1394? If you think it's not related to this discussion, please tell it to me. Then I'll drop out from this thread. History Repeats itself. Takashi Sakamoto
Re: [very-RFC 0/8] TSN driver for the kernel
Hi, Sorry to be late. In this weekday, I have little time for this thread because working for alsa-lib[1]. Besides, I'm not full-time developer for this kind of work. In short, I use my limited private time for this discussion. On Jun 15 2016 17:06, Richard Cochran wrote: > On Wed, Jun 15, 2016 at 12:15:24PM +0900, Takashi Sakamoto wrote: >>> On Mon, Jun 13, 2016 at 01:47:13PM +0200, Richard Cochran wrote: >>>> I have seen audio PLL/multiplier chips that will take, for example, a >>>> 10 kHz input and produce your 48 kHz media clock. With the right HW >>>> design, you can tell your PTP Hardware Clock to produce a 1 PPS, >>>> and you will have a synchronized AVB endpoint. The software is all >>>> there already. Somebody should tell the ALSA guys about it. >> >> Just from my curiosity, could I ask you more explanation for it in ALSA >> side? > > (Disclaimer: I really don't know too much about ALSA, expect that is > fairly big and complex ;) In this morning, I read IEEE 1722:2011 and realized that it quite roughly refers to IEC 61883-1/6 and includes much ambiguities to end applications. (In my opinion, the author just focuses on packet with timestamps, without enough considering about how to implement endpoint applications which perform semi-real sampling, fetching and queueing and so on, so as you. They're satisfied just by handling packet with timestamp, without enough consideration about actual hardware/software applications.) > Here is what I think ALSA should provide: > > - The DA and AD clocks should appear as attributes of the HW device. > > - There should be a method for measuring the DA/AD clock rate with > respect to both the system time and the PTP Hardware Clock (PHC) > time. > > - There should be a method for adjusting the DA/AD clock rate if > possible. If not, then ALSA should fall back to sample rate > conversion. > > - There should be a method to determine the time delay from the point > when the audio data are enqueued into ALSA until they pass through > the D/A converter. If this cannot be known precisely, then the > library should provide an estimate with an error bound. > > - I think some AVB use cases will need to know the time delay from A/D > until the data are available to the local application. (Distributed > microphones? I'm not too sure about that.) > > - If the DA/AD clocks are connected to other clock devices in HW, > there should be a way to find this out in SW. For example, if SW > can see the PTP-PHC-PLL-DA relationship from the above example, then > it knows how to synchronize the DA clock using the network. > > [ Implementing this point involves other subsystems beyond ALSA. It > isn't really necessary for people designing AVB systems, since > they know their designs, but it would be nice to have for writing > generic applications that can deal with any kind of HW setup. ] Depends on which subsystem decides "AVTP presentation time"[3]. This value is dominant to the number of events included in an IEC 61883-1 packet. If this TSN subsystem decides it, most of these items don't need to be in ALSA. As long as I know, the number of AVTPDU per second seems not to be fixed. So each application is not allowed to calculate the timestamp by its own way unless TSN implementation gives the information to each applications. For your information, in current ALSA implementation of IEC 61883-1/6 on IEEE 1394 bus, the presentation timestamp is decided in ALSA side. The number of isochronous packet transmitted per second is fixed by 8,000 in IEEE 1394, and the number of data blocks in an IEC 61883-1 packet is deterministic according to 'sampling transfer frequency' in IEC 61883-6 and isochronous cycle count passed from Linux FireWire subsystem. In the TSN subsystem, like FireWire subsystem, callback for filling payload should have information of 'when the packet is scheduled to be transmitted'. With the information, each application can calculate the number of event in the packet and presentation timestamp. Of cource, this timestamp should be handled as 'avtp_timestamp' in packet queueing. >> In ALSA, sampling rate conversion should be in userspace, not in kernel >> land. In alsa-lib, sampling rate conversion is implemented in shared object. >> When userspace applications start playbacking/capturing, depending on PCM >> node to access, these applications load the shared object and convert PCM >> frames from buffer in userspace to mmapped DMA-buffer, then commit them. > > The AVB use case places an additional requirement on the rate > conversion. You will need to adjust the frequency on the fly, as the > stream is playing. I would guess that ALSA doesn't have that option? In ALSA
Re: [very-RFC 0/8] TSN driver for the kernel
Hi Richard, On Tue, 14 Jun 2016 19:04:44 +0200, Richard Cochran write: >> Well, I guess I should have said, I am not too familiar with the >> breadth of current audio hardware, high end or low end. Of course I >> would like to see even consumer devices work with AVB, but it is up to >> the ALSA people to make that happen. So far, nothing has been done, >> afaict. In OSS world, there's few developers for this kind of devices, even if it's alsa-project. Furthermore, manufacturerer for recording equipments have no interests in OSS. In short, what we can do for these devices is just to reverse-engineering. For models of Ethernet-AVB, it might be just to transfer or receive packets, and read them. The devices are still black-boxes and we have no ways to reveal their details. So when you require the details to implement something in your side, few developers can tell you, I think. Regards Takashi Sakamoto
Re: [very-RFC 0/8] TSN driver for the kernel
Hi Richard, > On Mon, Jun 13, 2016 at 01:47:13PM +0200, Richard Cochran wrote: >> 3. ALSA support for tunable AD/DA clocks. The rate of the Listener's >>DA clock must match that of the Talker and the other Listeners. >>Either you adjust it in HW using a VCO or similar, or you do >>adaptive sample rate conversion in the application. (And that is >>another reason for *not* having a shared kernel buffer.) For the >>Talker, either you adjust the AD clock to match the PTP time, or >>you measure the frequency offset. >> >> I have seen audio PLL/multiplier chips that will take, for example, a >> 10 kHz input and produce your 48 kHz media clock. With the right HW >> design, you can tell your PTP Hardware Clock to produce a 1 PPS, >> and you will have a synchronized AVB endpoint. The software is all >> there already. Somebody should tell the ALSA guys about it. Just from my curiosity, could I ask you more explanation for it in ALSA side? The similar mechanism to synchronize endpoints was also applied to audio and music unit on IEEE 1394 bus. According to IEC 61883-1/6, some of these actual units can generate presentation-timestamp from header information of 8,000 packet per sec, and utilize the signal as sampling clock[1]. There's much differences between IEC 61883-1/6 on IEEE 1394 bus and Audio and Video Bridge on Ethernet[2], especially for synchronization, but in this point of transferring synchnization signal and time-based data, we have the similar requirements of software implementations, I think. My motivation to join in this discussion is to consider about to make it clear to implement packet-oriented drivers in ALSA kernel-land, and enhance my work for drivers to handle IEC 61883-1/6 on IEEE 1394 bus. >> I don't know if ALSA has anything for sample rate conversion or not, >> but haven't seen anything that addresses distributed synchronized >> audio applications. In ALSA, sampling rate conversion should be in userspace, not in kernel land. In alsa-lib, sampling rate conversion is implemented in shared object. When userspace applications start playbacking/capturing, depending on PCM node to access, these applications load the shared object and convert PCM frames from buffer in userspace to mmapped DMA-buffer, then commit them. Before establishing a PCM substream, userspace applications and in-kernel drivers communicate to decide sampling rate, PCM frame format, the size of PCM buffer, and so on. (see snd_pcm_hw_params() and ioctl(SNDRV_PCM_IOCTL_HW_PARAMS)). Thus, as long as in-kernel drivers know specifications of endpoints, userspace applications can start PCM substreams correctly. [1] In detail, please refer to specification of 1394TA I introduced: http://www.spinics.net/lists/netdev/msg381259.html [2] I guess that IEC 61883-1/6 packet for Ethernet-AVB is a mutant from original specifications. Regards Takashi Sakamoto
Re: [very-RFC 0/8] TSN driver for the kernel
On Jun 12 2016 17:31, Henrik Austad wrote: > On Sun, Jun 12, 2016 at 01:30:24PM +0900, Takashi Sakamoto wrote: >> On Jun 12 2016 12:38, Takashi Sakamoto wrote: >>> In your patcset, there's no actual codes about how to handle any >>> interrupt contexts (software / hardware), how to handle packet payload, >>> and so on. Especially, for recent sound subsystem, the timing of >>> generating interrupts and which context does what works are important to >>> reduce playback/capture latency and power consumption. >>> >>> Of source, your intention of this patchset is to show your early concept >>> of TSN feature. Nevertheless, both of explaination and codes are >>> important to the other developers who have little knowledges about TSN, >>> AVB and AES-64 such as me. >> >> Oops. Your 5th patch was skipped by alsa-project.org. I guess that size >> of the patch is too large to the list service. I can see it: >> http://marc.info/?l=linux-netdev=146568672728661=2 >> >> As long as seeing the patch, packets are queueing in hrtimer callbacks >> every 1 seconds. > > Actually, the hrtimer fires every 1ms, and that part is something I have to > do something about, also because it sends of the same number of frames > every time, regardless of how accurate the internal timer is to the rest of > the network (there's no backpressure from the networking layer). > >> (This is a high level discussion and it's OK to ignore it for the >> moment. When writing packet-oriented drivers for sound subsystem, you >> need to pay special attention to accuracy of the number of PCM frames >> transferred currently, and granularity of the number of PCM frames >> transferred by one operation. In this case, snd_avb_hw, >> snd_avb_pcm_pointer(), tsn_buffer_write_net() and tsn_buffer_read_net() >> are involved in this discussion. You can see ALSA developers' struggle >> in USB audio device class drivers and (of cource) IEC 61883-1/6 drivers.) > > Ah, good point. Any particular parts of the USB-subsystem I should start > looking at? I don't think it's a beter way for you to study USB Audio Device Class driver unless you're interested in ALSA or USB subsystem. (But for your information, snd-usb-audio is in sound/usb/* of Linux kernel. IEC 61883-1/6 driver is in sound/firewire/*.) We need different strategy to achieve it on different transmission backend. > Knowing where to start looking is a tremendous help It's not well-documented, and not well-generalized for packet-oriented drivers. Most of developers who have enough knowledge about it work for DMA-oriented drivers in mobile platforms and have little interests in packet-oriented drivers. You need to find your own way. Currently I have few advices to you, because I'm also on the way for drivers to process IEC 61883-1/6 packets on IEEE 1394 bus with enough accuracy and granularity. The paper I introduced is for the way (but not mature). I wish you get more helps from the other developers. Your work is more significant to Linux system, than mine. (And I hope your future work get no ignorance and no unreasonable hostility from coarse users.) Regards Takashi Sakamoto signature.asc Description: OpenPGP digital signature
Re: [very-RFC 0/8] TSN driver for the kernel
On Jun 12 2016 17:28, Henrik Austad wrote: > On Sun, Jun 12, 2016 at 12:38:36PM +0900, Takashi Sakamoto wrote: >> I'm one of maintainers for ALSA firewire stack, which handles IEC >> 61883-1/6 and vendor-unique packets on IEEE 1394 bus for consumer >> recording equipments. >> (I'm not in MAINTAINERS because I'm a shy boy.) >> >> IEC 61883-6 describes that one packet can multiplex several types of >> data in its data channels; i.e. Multi Bit Linear Audio data (PCM >> samples), One Bit Audio Data (DSD), MIDI messages and so on. > > Hmm, that I did not know, not sure how that applies to AVB, but definately > something I have to look into. For your information, I describe more about it. You can see pre-standardized specification for IEC 61883-6 in website of 1394 Trade Association. Let's look for 'Audio and Music Data Transmission Protocol 2.3 (October 13, 2010, 1394TA)' http://1394ta.org/specifications/ In 'clause 12. AM824 SEQUENCE ADAPTATION LAYERS', you can see that one data block includes several types of data. But I can imagine that joint group for AVB loosely refers to IEC 61883-6. In this case, AVB specification might describe one data block transfers one type of data, to drop unreasonable complexities. >> If you handles packet payload in 'struct snd_pcm_ops.copy', a process >> context of an ALSA PCM applications performs the work. Thus, no chances >> to multiplex data with the other types. > > The driver is not adhering fully to any standards right now, the amount of > detail is quite high - but I'm slowly improving as I go through the > standards. Getting on top of all the standards and all the different > subsystems are definately a work in progress (it's a lot to digest!) In my taste, the driver is not necessarily compliant to any standards. It's enough just to work its task, without bad side-effects to Linux system. Based on this concept, current ALSA firewire stack just support PCM frames and MIDI messages. Here, I tell you that actual devices tend not to be compliant to any standards and lost inter-operability. (Especially, most of audio and music units on IEEE 1394 bus ignores some of items in standards. In short, they already lost inter-operability.) So here, we just consider about what actual devices do, instead of following any standards. Regards Takashi Sakamoto signature.asc Description: OpenPGP digital signature
Re: [very-RFC 0/8] TSN driver for the kernel
On Jun 12 2016 12:38, Takashi Sakamoto wrote: > In your patcset, there's no actual codes about how to handle any > interrupt contexts (software / hardware), how to handle packet payload, > and so on. Especially, for recent sound subsystem, the timing of > generating interrupts and which context does what works are important to > reduce playback/capture latency and power consumption. > > Of source, your intention of this patchset is to show your early concept > of TSN feature. Nevertheless, both of explaination and codes are > important to the other developers who have little knowledges about TSN, > AVB and AES-64 such as me. Oops. Your 5th patch was skipped by alsa-project.org. I guess that size of the patch is too large to the list service. I can see it: http://marc.info/?l=linux-netdev=146568672728661=2 As long as seeing the patch, packets are queueing in hrtimer callbacks every 1 seconds. (This is a high level discussion and it's OK to ignore it for the moment. When writing packet-oriented drivers for sound subsystem, you need to pay special attention to accuracy of the number of PCM frames transferred currently, and granularity of the number of PCM frames transferred by one operation. In this case, snd_avb_hw, snd_avb_pcm_pointer(), tsn_buffer_write_net() and tsn_buffer_read_net() are involved in this discussion. You can see ALSA developers' struggle in USB audio device class drivers and (of cource) IEC 61883-1/6 drivers.) Regards Takashi Sakamoto
Re: [very-RFC 0/8] TSN driver for the kernel
Hi, I'm one of maintainers for ALSA firewire stack, which handles IEC 61883-1/6 and vendor-unique packets on IEEE 1394 bus for consumer recording equipments. (I'm not in MAINTAINERS because I'm a shy boy.) IEC 61883-6 describes that one packet can multiplex several types of data in its data channels; i.e. Multi Bit Linear Audio data (PCM samples), One Bit Audio Data (DSD), MIDI messages and so on. If you handles packet payload in 'struct snd_pcm_ops.copy', a process context of an ALSA PCM applications performs the work. Thus, no chances to multiplex data with the other types. To prevent this situation, current ALSA firewire stack handles packet payload in software interrupt context of isochronous context of OHCI 1394. As a result of this, the software stack supports PCM substreams and MIDI substreams. In your patcset, there's no actual codes about how to handle any interrupt contexts (software / hardware), how to handle packet payload, and so on. Especially, for recent sound subsystem, the timing of generating interrupts and which context does what works are important to reduce playback/capture latency and power consumption. Of source, your intention of this patchset is to show your early concept of TSN feature. Nevertheless, both of explaination and codes are important to the other developers who have little knowledges about TSN, AVB and AES-64 such as me. And, I might cooperate to prepare for common IEC 61883 layer. For actual codes of ALSA firewire stack, please see mainline kernel code. For actual devices of IEC 61883-1/6 and IEEE 1394 bus, please refer to my report in 2014. At least, you can get to know what to consider about developing upper drivers near ALSA userspace applications. https://github.com/takaswie/alsa-firewire-report (But I confirm that the report includes my misunderstandings in clause 3.4 and 6.2. need more time...) Regards Takashi Sakamoto On Jun 12 2016 08:01, Henrik Austad wrote: > Hi all > (series based on v4.7-rc2, now with the correct netdev) > > This is a *very* early RFC for a TSN-driver in the kernel. It has been > floating around in my repo for a while and I would appreciate some > feedback on the overall design to avoid doing some major blunders. > > TSN: Time Sensitive Networking, formely known as AVB (Audio/Video > Bridging). > > There are at least one AVB-driver (the AV-part of TSN) in the kernel > already, however this driver aims to solve a wider scope as TSN can do > much more than just audio. A very basic ALSA-driver is added to the end > that allows you to play music between 2 machines using aplay in one end > and arecord | aplay on the other (some fiddling required) We have plans > for doing the same for v4l2 eventually (but there are other fishes to > fry first). The same goes for a TSN_SOCK type approach as well. > > TSN is all about providing infrastructure. Allthough there are a few > very interesting uses for TSN (reliable, deterministic network for audio > and video), once you have that reliable link, you can do a lot more. > > Some notes on the design: > > The driver is directed via ConfigFS as we need userspace to handle > stream-reservation (MSRP), discovery and enumeration (IEEE 1722.1) and > whatever other management is needed. Once we have all the required > attributes, we can create link using mkdir, and use write() to set the > attributes. Once ready, specify the 'shim' (basically a thin wrapper > between TSN and another subsystem) and we start pushing out frames. > > The network part: it ties directly into the rx-handler for receive and > writes skb's using netdev_start_xmit(). This could probably be > improved. 2 new fields in netdev_ops have been introduced, and the Intel > igb-driver has been updated (as this is available as a PCI-e card). The > igb-driver works-ish > > > What remains > - tie to (g)PTP properly, currently using ktime_get() for presentation > time > - get time from shim into TSN and vice versa > - let shim create/manage buffer > > Henrik Austad (8): > TSN: add documentation > TSN: Add the standard formerly known as AVB to the kernel > Adding TSN-driver to Intel I210 controller > Add TSN header for the driver > Add TSN machinery to drive the traffic from a shim over the network > Add TSN event-tracing > AVB ALSA - Add ALSA shim for TSN > MAINTAINERS: add TSN/AVB-entries > > Documentation/TSN/tsn.txt | 147 + > MAINTAINERS | 14 + > drivers/media/Kconfig | 15 + > drivers/media/Makefile| 3 +- > drivers/media/avb/Makefile| 5 + > drivers/media/avb/avb_alsa.c | 742 +++ > drivers/media/avb/tsn_iec61883.h | 124 > drivers/net/ethernet/intel/Kconfig