Re: write method for audio driver?
On 8/27/2023 7:49 PM, Nathan Hartman wrote: I don't know whether you've had a chance to read [1] yet, but if not, it might help visualize the two-part structure of device drivers in NuttX. (If you've already read it, then never mind, and apologies for the noise. :-) The audio subsystem is different from typical character drivers (I used the name lower half, but that really isn't really accurate). It is a chain of audio processing nodes; audio buffers are passed from one node to the next. Two special nodes are the source and the sink. The Class D driver is a sink node. It receives audio buffers and disposes of them... usually by interacting with hardware to "make sounds". The source node is the one that receives unprocessed audio data. It is audio/audio.c and it registers itself as character driver. It received unprocessed audio data from the application and passes the audio buffers down the chain. The board level logic can string together the audio processing nodes in most any order. But only the source node is registered by calling audio_register() in audio/audio.c which registers the character driver in /dev/audio/XXX where only XXX is provided by the board logic. None of the other nodes are capable of being registered as character devices. Audio applications like nxplayer may, for example, open a file, read audio data, and pass the audio data to the audio/audio.c driver which will transfer it down the chain, eventually, "playing" the file. Only the audio/audio.c is registered and it is the only character driver. It has no lower half.
Re: write method for audio driver?
On Sun, Aug 27, 2023 at 6:24 PM Tim Hardisty wrote: > SAMA5D2 specific, I should add. > > > > On 27 Aug 2023, at 23:23, Tim Hardisty wrote: > > > > The classD driver DOES register itself as /dev/audio/pcm0, and works > correctly as such with nx_player. > > > > The conversions are wav (or mp3 etc) to pcm. > > > > Don’t if we’re talking cross purposes or the classd driver works in a > way it shouldn’t!!! I don't know whether you've had a chance to read [1] yet, but if not, it might help visualize the two-part structure of device drivers in NuttX. (If you've already read it, then never mind, and apologies for the noise. :-) [1] https://nuttx.apache.org/docs/latest/components/drivers/index.html Cheers, Nathan
Re: write method for audio driver?
On 8/27/2023 4:24 PM, Tim Hardisty wrote: SAMA5D2 specific, I should add. There is no support for the Class D driver for any SAMA5D2 board in the source tree.
Re: write method for audio driver?
On 8/27/2023 4:22 PM, Tim Hardisty wrote: The classD driver DOES register itself as /dev/audio/pcm0, and works correctly as such with nx_player. You are mistaken. The Class D driver does not register itself as a character driver. grep -r register_driver arch/arm/src/sama5/ proves that is true. The logic that registers /dev/audio/pcm0 is in audio/audio.c. audio/Kconfig defines the directory. Nothing is the code base except files in audio/ knows about that directory or is capable of registering a driver there. config AUDIO_DEV_PATH string "Base path for Audio devices" default "/dev/audio" audio/audio.c: audio/audio.c: FAR const char *devname = CONFIG_AUDIO_DEV_PATH; Most certainly the class D lower half cannot be register as a driver. That is totally impossible. It is not a character driver; it is only the lower half of a driver. It contains no usable driver logic; only the low level hardware interface. You must be using audio_register() in your board start up logic. That is part of the configuration of the audio chain. It does NOT produce a PCM driver. The driver is the audio chain that starts in audio/audio.c. Class D is the terminus of the audio chain and cannot be accessed using /dev/audio/pcm0. It has that name only because that is the name that you gave it when you called audio_register. The entire audio chain supports PCM, but the driver is the audio.c driver. he conversions are wav (or mp3 etc) to pcm. Don’t if we’re talking cross purposes or the classd driver works in a way it shouldn’t!!! In the previous email I listed the only place in the code base where a driver is registered at /dev/audio/pcm0 for any SAMA5 board. The only cases in the code base do PCM to WAV conversion. That is done by audio/pcm_decode.c. You may have some other audio logic that uses the Class D to handle PCM directly, I don't know. If so, it is not in the source tree. Certainly that driver is not sam_classd.c. sam_classd.c is not a driver in the POSIX sense. It is only a hardware lower half. It could never possibly be registered as a driver. It doesn't support the standard driver calls. In the terminology of the audio subsystem, it is called an audio driver. It is the sink at the end of the audio chain managed by the logic in audio/.
Re: write method for audio driver?
SAMA5D2 specific, I should add.
> On 27 Aug 2023, at 23:23, Tim Hardisty wrote:
>
> The classD driver DOES register itself as /dev/audio/pcm0, and works
> correctly as such with nx_player.
>
> The conversions are wav (or mp3 etc) to pcm.
>
> Don’t if we’re talking cross purposes or the classd driver works in a way it
> shouldn’t!!!
>
>> On 27 Aug 2023, at 23:00, Gregory Nutt wrote:
>>
>> Also, I don't think the /dev/audio/pcm0 device you are talking about is
>> what you think it is. It is a character driver but not the Class D lower
>> half. So, yes it can be opened.
>>
>> /Caveat: It has been ages since I worked with the audio subsystem so I
>> might be completely wrong./
>>
>> /dev/audio/pcm0 is the audio subsystem interface device. The "pcm"
>> indicates that it used the PCM software decoder (that will convert PCM file
>> data to WAV). It gets set up like:
>>
>> boards/arm/sama5/sama5d4-ek/src/sam_wm8904.c:
>>
>> pcm = pcm_decode_initialize(wm8904);
>> snprintf(devname, 12, "pcm%d", minor);
>> ret = audio_register(devname, pcm);
>>
>> And a character driver is registered by the audio subsystem in audio/audio.c:
>>
>> int audio_register(FAR const char *name, FAR struct
>> audio_lowerhalf_s *dev)
>> {
>> ...
>>
>> audinfo("Registering %s\n", path);
>> return register_driver(path, &g_audioops, 0666, upper);
>> }
>>
>> Where g_audioops is the character driver operations structure:
>>
>> static const struct file_operations g_audioops =
>> {
>> audio_open, /* open */
>> audio_close, /* close */
>> audio_read, /* read */
>> audio_write, /* write */
>> NULL,/* seek */
>> audio_ioctl, /* ioctl */
>> };
>>
>> So the registered pcm0 is the standard audio buffer chain configured for PCM
>> file to WAV conversion and terminating with a wm8904 DAC.
>>
>> I don't believe that there is any way to get the Class D audio_ops_s as a
>> driver under /dev. Nothing like that is supported.
>>
Re: write method for audio driver?
The classD driver DOES register itself as /dev/audio/pcm0, and works correctly
as such with nx_player.
The conversions are wav (or mp3 etc) to pcm.
Don’t if we’re talking cross purposes or the classd driver works in a way it
shouldn’t!!!
> On 27 Aug 2023, at 23:00, Gregory Nutt wrote:
>
> Also, I don't think the /dev/audio/pcm0 device you are talking about is what
> you think it is. It is a character driver but not the Class D lower half.
> So, yes it can be opened.
>
> /Caveat: It has been ages since I worked with the audio subsystem so I might
> be completely wrong./
>
> /dev/audio/pcm0 is the audio subsystem interface device. The "pcm"
> indicates that it used the PCM software decoder (that will convert PCM file
> data to WAV). It gets set up like:
>
> boards/arm/sama5/sama5d4-ek/src/sam_wm8904.c:
>
> pcm = pcm_decode_initialize(wm8904);
> snprintf(devname, 12, "pcm%d", minor);
> ret = audio_register(devname, pcm);
>
> And a character driver is registered by the audio subsystem in audio/audio.c:
>
> int audio_register(FAR const char *name, FAR struct
> audio_lowerhalf_s *dev)
> {
> ...
>
> audinfo("Registering %s\n", path);
> return register_driver(path, &g_audioops, 0666, upper);
> }
>
> Where g_audioops is the character driver operations structure:
>
> static const struct file_operations g_audioops =
> {
> audio_open, /* open */
> audio_close, /* close */
> audio_read, /* read */
> audio_write, /* write */
> NULL,/* seek */
> audio_ioctl, /* ioctl */
> };
>
> So the registered pcm0 is the standard audio buffer chain configured for PCM
> file to WAV conversion and terminating with a wm8904 DAC.
>
> I don't believe that there is any way to get the Class D audio_ops_s as a
> driver under /dev. Nothing like that is supported.
>
Re: write method for audio driver?
Also, I don't think the /dev/audio/pcm0 device you are talking about is
what you think it is. It is a character driver but not the Class D
lower half. So, yes it can be opened.
/Caveat: It has been ages since I worked with the audio subsystem so I
might be completely wrong./
/dev/audio/pcm0 is the audio subsystem interface device. The "pcm"
indicates that it used the PCM software decoder (that will convert PCM
file data to WAV). It gets set up like:
boards/arm/sama5/sama5d4-ek/src/sam_wm8904.c:
pcm = pcm_decode_initialize(wm8904);
snprintf(devname, 12, "pcm%d", minor);
ret = audio_register(devname, pcm);
And a character driver is registered by the audio subsystem in
audio/audio.c:
int audio_register(FAR const char *name, FAR struct
audio_lowerhalf_s *dev)
{
...
audinfo("Registering %s\n", path);
return register_driver(path, &g_audioops, 0666, upper);
}
Where g_audioops is the character driver operations structure:
static const struct file_operations g_audioops =
{
audio_open, /* open */
audio_close, /* close */
audio_read, /* read */
audio_write, /* write */
NULL, /* seek */
audio_ioctl, /* ioctl */
};
So the registered pcm0 is the standard audio buffer chain configured for
PCM file to WAV conversion and terminating with a wm8904 DAC.
I don't believe that there is any way to get the Class D audio_ops_s as
a driver under /dev. Nothing like that is supported.
Re: write method for audio driver?
I suppose that the easiest thing of all would be to put the tones in small PCM files in a built-in ROMFS file system On 8/27/2023 1:47 PM, Gregory Nutt wrote: On 8/27/2023 5:41 AM, Tim Hardisty wrote: Or am I completely missing the point and I just need to use the nxaudio system!!?? I think you have been missing point. You have two options: 1. As you suggest, extend nxaudio to be a source of tone data (just like audio file data) and uses the audio subsystem to pas that laong the audio chain as any other audio. Or 2. Create a character driver that wraps the audio_ops_s as a character driver that can be directly accessed from applications. You could use tone.c and a model to get started. This would not be part of the OS audio system but a kludgy "bad" hanging to the side of the audio subsystem. But simpler. One thing that is certain is that you cannot open and use the audio device interface directly from the application. Only via the audio subsystem or via some hack custom logic. That is equivalent to saying that you want to use the Ethernet driver directly without going though the network layer. For the network layer, you would use the raw socket interface to accomplish the same thing without using the Ethernet driver directly. We had the same kind of discussion about radios a few years back. Capable radios are treated as network devices and support full TCP/UDP communications. Less capable radios use simpler character drivers. And, worse, some radios use both network and character drivers which ends up being quite a mess.
Re: write method for audio driver?
On 8/27/2023 5:41 AM, Tim Hardisty wrote: Or am I completely missing the point and I just need to use the nxaudio system!!?? I think you have been missing point. You have two options: 1. As you suggest, extend nxaudio to be a source of tone data (just like audio file data) and uses the audio subsystem to pas that laong the audio chain as any other audio. Or 2. Create a character driver that wraps the audio_ops_s as a character driver that can be directly accessed from applications. You could use tone.c and a model to get started. This would not be part of the OS audio system but a kludgy "bad" hanging to the side of the audio subsystem. But simpler. One thing that is certain is that you cannot open and use the audio device interface directly from the application. Only via the audio subsystem or via some hack custom logic. That is equivalent to saying that you want to use the Ethernet driver directly without going though the network layer. For the network layer, you would use the raw socket interface to accomplish the same thing without using the Ethernet driver directly. We had the same kind of discussion about radios a few years back. Capable radios are treated as network devices and support full TCP/UDP communications. Less capable radios use simpler character drivers. And, worse, some radios use both network and character drivers which ends up being quite a mess.
Re: write method for audio driver?
On 8/27/2023 5:25 AM, Tim Hardisty wrote: Looking a little more at the write function of audio_ops_s it is not used by any existing driver and is documented to be for device-specific information...so it is not portable in terms of playing tones by simply writing PCM data word by word. I get that now. So I'm thinking the right way is possibly either: 1) Complete the "tone" support of nxplayer - it exists as a command parameter but has no underlying code/functions 2) create a tone player as a standalone (example) app that opens /dev/audio/pcm%d and makes use of the audio_ops_s functions and ioctls to set up a NuttX sound system compliant device and play tones. That is then portable and can be used with any supported audio device without having to use nxplayer in its entirety. e.g you could run, from nsh something like: "pcmtone " so "pcm_tone /dev/pcm0 48 1000 500" to play a 1kHz tone using 48 sampling for 500ms. Looks like there are similar functions "out there" I could reference. You cannot open or access any audio driver from an application. It is type struct audio_ops_s. That cannot be opened. Only character devices can be opened (or block and MTD devices which have a character device proxy later. In order to open any charter driver, it must export a struct file_operations which the class D driver does not. But you can add a character driver proxy similar to tone.c that will allow you do do exactly what you want
Re: write method for audio driver?
On 8/27/2023 4:17 AM, Tim Hardisty wrote: Thanks Gregory. The existing ClassD driver is registered as /dev/audio/pcm0 and is a PCM device not PWM. It has the "usual" audio ops to configure it, start/stop/pause/enqueue etc. It is exposed via ioctls so does have a user interface; that is how buffers of audio are passed to it. Is that an OS-function only, hence your comments that it should not be exposed to the user and I'm muddling things up? It is impossible to expose the audio lower-half interface to applications without trashing the POSIX standard. You could, however, create a POSIX compliant character driver similar to drivers/audio/tone.c to interact with it from applications via standard read/write. When I mentioned tone.c, I was referring to the interface model of tone.c. It is irrelevant that one is PWM and one is PCM. The point is that the interface model provides a POSIX application interface that mediates calls to the lower level interface. In your case sam_classd.c would contain the lower level interface. The declaration of the audio_ops_s struct also includes a read and write function that are typically defined as NULL in these audio drivers. That's what I was thinking of adding, but perhaps I'm am not understanding that /dev/audio/pcm0 can't (shouldn't) actually be opened as a character driver? These read/write methods cannot be called from applications. You cannot register the driver, you cannot open the driver, you cannot write to it (unless someone has changed to code in some non-compliant way). But you can do all of those things via a standard character driver similar to tone.c that provides struct file_operations, but interfacing to a different architecture-specific lower half. Maybe audio_op_s could be that interface? Or maybe a new interface? Using the buffer interface would be more consistent with other usage of audio drivers in the system. It would be better to avoid the write method if possible. The character driver could also forward IOCTLs from the application to the audio lower half. This would all have to be generalized, nothing under drivers can to tailored for any specific MCU, but must be generic OS code suitable for any PCM driver.
Re: write method for audio driver?
Or am I completely missing the point and I just need to use the nxaudio system!!?? On 27/08/2023 12:25, Tim Hardisty wrote: Looking a little more at the write function of audio_ops_s it is not used by any existing driver and is documented to be for device-specific information...so it is not portable in terms of playing tones by simply writing PCM data word by word. I get that now. So I'm thinking the right way is possibly either: 1) Complete the "tone" support of nxplayer - it exists as a command parameter but has no underlying code/functions 2) create a tone player as a standalone (example) app that opens /dev/audio/pcm%d and makes use of the audio_ops_s functions and ioctls to set up a NuttX sound system compliant device and play tones. That is then portable and can be used with any supported audio device without having to use nxplayer in its entirety. e.g you could run, from nsh something like: "pcmtone " so "pcm_tone /dev/pcm0 48 1000 500" to play a 1kHz tone using 48 sampling for 500ms. Looks like there are similar functions "out there" I could reference. Is that more along the right lines perhaps? On 27/08/2023 11:17, Tim Hardisty wrote: Thanks Gregory. The existing ClassD driver is registered as /dev/audio/pcm0 and is a PCM device not PWM. It has the "usual" audio ops to configure it, start/stop/pause/enqueue etc. It is exposed via ioctls so does have a user interface; that is how buffers of audio are passed to it. Is that an OS-function only, hence your comments that it should not be exposed to the user and I'm muddling things up? The declaration of the audio_ops_s struct also includes a read and write function that are typically defined as NULL in these audio drivers. That's what I was thinking of adding, but perhaps I'm am not understanding that /dev/audio/pcm0 can't (shouldn't) actually be opened as a character driver? Sorry for the dumb questions! On 26/08/2023 23:10, Gregory Nutt wrote: On 8/26/2023 12:18 PM, Tim Hardisty wrote: This is, I'm sure, more a generic POSIX question than NuttX-specific but I am still not that familiar with either! When I wrote the SAMA5D2 ClassD audio driver I followed the methods appropriate for using "apb" so it works well with nxplayer (for example). But I want to play simple audio tones, from a sine look-up table, and filling a buffer and enqueuing it seems over the top. Is there any reason - from a "correctness" point of view - I couldn't add a basic file write function to the driver that allows you to simply fwrite to the opened device and have it dump data that way? A simple piece of code that simply writes to the processor audio sample buffer, word at a time, and polls the "ready" bit works 100% fine but this is not portable! So if such an interface is not "right" I won't add it :) I don't think it makes sense to add a write method to the audio driver interface. That is not driver is not POSIX and cannot be exposed as a user interface. I suggest thinking about this in a different way. How about an audio source driver that exposes two interfaces: (1) The audio interface that is (ONLY) used inside of the OS and certain audio applications (which it really shouldn't). An audio driver can be a source of audio data, a sink of audio data, or can intercept and modify audio data in the audio chain. And (2) a standard character driver that accepts audio input from application space. The character driver is POSIX and does support the write method. So any data written via the character driver interface would be treated as audio source data.
Re: write method for audio driver?
Looking a little more at the write function of audio_ops_s it is not used by any existing driver and is documented to be for device-specific information...so it is not portable in terms of playing tones by simply writing PCM data word by word. I get that now. So I'm thinking the right way is possibly either: 1) Complete the "tone" support of nxplayer - it exists as a command parameter but has no underlying code/functions 2) create a tone player as a standalone (example) app that opens /dev/audio/pcm%d and makes use of the audio_ops_s functions and ioctls to set up a NuttX sound system compliant device and play tones. That is then portable and can be used with any supported audio device without having to use nxplayer in its entirety. e.g you could run, from nsh something like: "pcmtone " so "pcm_tone /dev/pcm0 48 1000 500" to play a 1kHz tone using 48 sampling for 500ms. Looks like there are similar functions "out there" I could reference. Is that more along the right lines perhaps? On 27/08/2023 11:17, Tim Hardisty wrote: Thanks Gregory. The existing ClassD driver is registered as /dev/audio/pcm0 and is a PCM device not PWM. It has the "usual" audio ops to configure it, start/stop/pause/enqueue etc. It is exposed via ioctls so does have a user interface; that is how buffers of audio are passed to it. Is that an OS-function only, hence your comments that it should not be exposed to the user and I'm muddling things up? The declaration of the audio_ops_s struct also includes a read and write function that are typically defined as NULL in these audio drivers. That's what I was thinking of adding, but perhaps I'm am not understanding that /dev/audio/pcm0 can't (shouldn't) actually be opened as a character driver? Sorry for the dumb questions! On 26/08/2023 23:10, Gregory Nutt wrote: On 8/26/2023 12:18 PM, Tim Hardisty wrote: This is, I'm sure, more a generic POSIX question than NuttX-specific but I am still not that familiar with either! When I wrote the SAMA5D2 ClassD audio driver I followed the methods appropriate for using "apb" so it works well with nxplayer (for example). But I want to play simple audio tones, from a sine look-up table, and filling a buffer and enqueuing it seems over the top. Is there any reason - from a "correctness" point of view - I couldn't add a basic file write function to the driver that allows you to simply fwrite to the opened device and have it dump data that way? A simple piece of code that simply writes to the processor audio sample buffer, word at a time, and polls the "ready" bit works 100% fine but this is not portable! So if such an interface is not "right" I won't add it :) I don't think it makes sense to add a write method to the audio driver interface. That is not driver is not POSIX and cannot be exposed as a user interface. I suggest thinking about this in a different way. How about an audio source driver that exposes two interfaces: (1) The audio interface that is (ONLY) used inside of the OS and certain audio applications (which it really shouldn't). An audio driver can be a source of audio data, a sink of audio data, or can intercept and modify audio data in the audio chain. And (2) a standard character driver that accepts audio input from application space. The character driver is POSIX and does support the write method. So any data written via the character driver interface would be treated as audio source data.
Re: write method for audio driver?
Thanks Gregory. The existing ClassD driver is registered as /dev/audio/pcm0 and is a PCM device not PWM. It has the "usual" audio ops to configure it, start/stop/pause/enqueue etc. It is exposed via ioctls so does have a user interface; that is how buffers of audio are passed to it. Is that an OS-function only, hence your comments that it should not be exposed to the user and I'm muddling things up? The declaration of the audio_ops_s struct also includes a read and write function that are typically defined as NULL in these audio drivers. That's what I was thinking of adding, but perhaps I'm am not understanding that /dev/audio/pcm0 can't (shouldn't) actually be opened as a character driver? Sorry for the dumb questions! On 26/08/2023 23:10, Gregory Nutt wrote: On 8/26/2023 12:18 PM, Tim Hardisty wrote: This is, I'm sure, more a generic POSIX question than NuttX-specific but I am still not that familiar with either! When I wrote the SAMA5D2 ClassD audio driver I followed the methods appropriate for using "apb" so it works well with nxplayer (for example). But I want to play simple audio tones, from a sine look-up table, and filling a buffer and enqueuing it seems over the top. Is there any reason - from a "correctness" point of view - I couldn't add a basic file write function to the driver that allows you to simply fwrite to the opened device and have it dump data that way? A simple piece of code that simply writes to the processor audio sample buffer, word at a time, and polls the "ready" bit works 100% fine but this is not portable! So if such an interface is not "right" I won't add it :) I don't think it makes sense to add a write method to the audio driver interface. That is not driver is not POSIX and cannot be exposed as a user interface. I suggest thinking about this in a different way. How about an audio source driver that exposes two interfaces: (1) The audio interface that is (ONLY) used inside of the OS and certain audio applications (which it really shouldn't). An audio driver can be a source of audio data, a sink of audio data, or can intercept and modify audio data in the audio chain. And (2) a standard character driver that accepts audio input from application space. The character driver is POSIX and does support the write method. So any data written via the character driver interface would be treated as audio source data.
Re: write method for audio driver?
Sort of perhaps. The SAMA5D@ needs lots of setting up and the existing driver does that with all the usual audio device ioctls and enqueue methods: it works just fine using nxplayer and can playback .wav files etc. But it it is based on creating buffers of audio - the assumption being that you are playing back long(ish) streams of audio, so the buffer/queue methods make sense. To play a brief tone it just seems like a lot of hassle to generate a 0.5s (say) buffer of sinewave pcm date when it is really only necessary to write it sample by sample to the (already configured) ClassD device at /dev/audio/pcm0 and have the driver manage the buffer and ready flags etc. I will add other notes in an answer to Gregory's reply to this. On 27/08/2023 05:59, Tomek CEDRO wrote: On Sat, Aug 26, 2023, 20:18 Tim Hardisty wrote: This is, I'm sure, more a generic POSIX question than NuttX-specific but I am still not that familiar with either! When I wrote the SAMA5D2 ClassD audio driver I followed the methods appropriate for using "apb" so it works well with nxplayer (for example). But I want to play simple audio tones, from a sine look-up table, and filling a buffer and enqueuing it seems over the top. Is there any reason - from a "correctness" point of view - I couldn't add a basic file write function to the driver that allows you to simply fwrite to the opened device and have it dump data that way? A simple piece of code that simply writes to the processor audio sample buffer, word at a time, and polls the "ready" bit works 100% fine but this is not portable! So if such an interface is not "right" I won't add it :) you mean /dev/dsp ? https://man.freebsd.org/cgi/man.cgi?query=sound&apropos=0&manpath=FreeBSD+13.2-RELEASE+and+Ports -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
Re: write method for audio driver?
Thanks Gregory. I looked at that but it produces tones using PWM. The ClassD of the SAMA5D2 is a PCM device, so not an immediately obvious match? On 26/08/2023 23:17, Gregory Nutt wrote: > >>> But I want to play simple audio tones, from a sine look-up table, >>> and filling a buffer and enqueuing it seems over the top. > > If you just want to plane simple audio tones, maybe something > drivers/audio/tone.c is what you need? That is a character driver > with a custom tone interface. >
