Re: [PATCH 2/4] ASoC: dt-bindings: fsl,xcvr: Add two PLL clock sources
On Fri, May 10, 2024 at 10:27 AM Shengjiu Wang wrote: > > On Fri, May 10, 2024 at 1:14 AM Conor Dooley wrote: > > > > On Thu, May 09, 2024 at 10:57:38AM +0800, Shengjiu Wang wrote: > > > Add two PLL clock sources, they are the parent clocks of the root clock > > > one is for 8kHz series rates, named as 'pll8k', another one is for > > > 11kHz series rates, named as 'pll11k'. They are optional clocks, > > > if there are such clocks, then the driver can switch between them to > > > support more accurate sample rates. > > > > > > As 'pll8k' and 'pll11k' are optional, then add 'minItems: 4' for > > > clocks and clock-names properties. > > > > Despite the detail given here in the commit message, the series this is > > appearing in and one of the driver patches makes me a bit "suspicious" > > of this patch. Are these newly added clocks available on all devices, or > > just on the imx95, or? > > These newly added clocks are only available for the imx95 XCVR. > Looks like I should merge patch1 & 2 together, patch 3 & 3 together. right? Best regards Shengjiu Wang > Best regards > Shengjiu Wang > > > > Thanks, > > Conor. > > > > > > > > Signed-off-by: Shengjiu Wang > > > --- > > > Documentation/devicetree/bindings/sound/fsl,xcvr.yaml | 6 ++ > > > 1 file changed, 6 insertions(+) > > > > > > diff --git a/Documentation/devicetree/bindings/sound/fsl,xcvr.yaml > > > b/Documentation/devicetree/bindings/sound/fsl,xcvr.yaml > > > index 1c74a32def09..c4660faed404 100644 > > > --- a/Documentation/devicetree/bindings/sound/fsl,xcvr.yaml > > > +++ b/Documentation/devicetree/bindings/sound/fsl,xcvr.yaml > > > @@ -50,6 +50,9 @@ properties: > > >- description: PHY clock > > >- description: SPBA clock > > >- description: PLL clock > > > + - description: PLL clock source for 8kHz series > > > + - description: PLL clock source for 11kHz series > > > +minItems: 4 > > > > > >clock-names: > > > items: > > > @@ -57,6 +60,9 @@ properties: > > >- const: phy > > >- const: spba > > >- const: pll_ipg > > > + - const: pll8k > > > + - const: pll11k > > > +minItems: 4 > > > > > >dmas: > > > items: > > > -- > > > 2.34.1 > > >
Re: [PATCH 1/4] ASoC: dt-bindings: fsl,xcvr: Add compatible string for i.MX95
On Fri, May 10, 2024 at 1:11 AM Conor Dooley wrote: > > On Thu, May 09, 2024 at 10:57:37AM +0800, Shengjiu Wang wrote: > > Add compatible string "fsl,imx95-xcvr" for i.MX95 platform. > > That's apparent from the diff. Why is it not compatible with existing > devices? i.MX8MP: There is PHY and support eARC, ARC, SPDIF i.MX93: There is no PHY and support SPDIF only i.MX95: There is PHY (PHY is different with i.MX8MP), only support SPDIF. Will add such info in the commit message. Best regards Shengjiu > > Cheers, > Conor. > > > > > Signed-off-by: Shengjiu Wang > > --- > > Documentation/devicetree/bindings/sound/fsl,xcvr.yaml | 1 + > > 1 file changed, 1 insertion(+) > > > > diff --git a/Documentation/devicetree/bindings/sound/fsl,xcvr.yaml > > b/Documentation/devicetree/bindings/sound/fsl,xcvr.yaml > > index 0eb0c1ba8710..1c74a32def09 100644 > > --- a/Documentation/devicetree/bindings/sound/fsl,xcvr.yaml > > +++ b/Documentation/devicetree/bindings/sound/fsl,xcvr.yaml > > @@ -22,6 +22,7 @@ properties: > > enum: > >- fsl,imx8mp-xcvr > >- fsl,imx93-xcvr > > + - fsl,imx95-xcvr > > > >reg: > > items: > > -- > > 2.34.1 > >
Re: [PATCH 2/4] ASoC: dt-bindings: fsl,xcvr: Add two PLL clock sources
On Fri, May 10, 2024 at 1:14 AM Conor Dooley wrote: > > On Thu, May 09, 2024 at 10:57:38AM +0800, Shengjiu Wang wrote: > > Add two PLL clock sources, they are the parent clocks of the root clock > > one is for 8kHz series rates, named as 'pll8k', another one is for > > 11kHz series rates, named as 'pll11k'. They are optional clocks, > > if there are such clocks, then the driver can switch between them to > > support more accurate sample rates. > > > > As 'pll8k' and 'pll11k' are optional, then add 'minItems: 4' for > > clocks and clock-names properties. > > Despite the detail given here in the commit message, the series this is > appearing in and one of the driver patches makes me a bit "suspicious" > of this patch. Are these newly added clocks available on all devices, or > just on the imx95, or? These newly added clocks are only available for the imx95 XCVR. Best regards Shengjiu Wang > > Thanks, > Conor. > > > > > Signed-off-by: Shengjiu Wang > > --- > > Documentation/devicetree/bindings/sound/fsl,xcvr.yaml | 6 ++ > > 1 file changed, 6 insertions(+) > > > > diff --git a/Documentation/devicetree/bindings/sound/fsl,xcvr.yaml > > b/Documentation/devicetree/bindings/sound/fsl,xcvr.yaml > > index 1c74a32def09..c4660faed404 100644 > > --- a/Documentation/devicetree/bindings/sound/fsl,xcvr.yaml > > +++ b/Documentation/devicetree/bindings/sound/fsl,xcvr.yaml > > @@ -50,6 +50,9 @@ properties: > >- description: PHY clock > >- description: SPBA clock > >- description: PLL clock > > + - description: PLL clock source for 8kHz series > > + - description: PLL clock source for 11kHz series > > +minItems: 4 > > > >clock-names: > > items: > > @@ -57,6 +60,9 @@ properties: > >- const: phy > >- const: spba > >- const: pll_ipg > > + - const: pll8k > > + - const: pll11k > > +minItems: 4 > > > >dmas: > > items: > > -- > > 2.34.1 > >
Re: [PATCH v15 00/16] Add audio support in v4l2 framework
On Thu, May 9, 2024 at 6:28 PM Amadeusz Sławiński wrote: > > On 5/9/2024 12:12 PM, Shengjiu Wang wrote: > > On Thu, May 9, 2024 at 5:50 PM Amadeusz Sławiński > > wrote: > >> > >> On 5/9/2024 11:36 AM, Shengjiu Wang wrote: > >>> On Wed, May 8, 2024 at 4:14 PM Amadeusz Sławiński > >>> wrote: > >>>> > >>>> On 5/8/2024 10:00 AM, Hans Verkuil wrote: > >>>>> On 06/05/2024 10:49, Shengjiu Wang wrote: > >>>>>> On Fri, May 3, 2024 at 4:42 PM Mauro Carvalho Chehab > >>>>>> wrote: > >>>>>>> > >>>>>>> Em Fri, 3 May 2024 10:47:19 +0900 > >>>>>>> Mark Brown escreveu: > >>>>>>> > >>>>>>>> On Thu, May 02, 2024 at 10:26:43AM +0100, Mauro Carvalho Chehab > >>>>>>>> wrote: > >>>>>>>>> Mauro Carvalho Chehab escreveu: > >>>>>>>> > >>>>>>>>>> There are still time control associated with it, as audio and video > >>>>>>>>>> needs to be in sync. This is done by controlling the buffers size > >>>>>>>>>> and could be fine-tuned by checking when the buffer transfer is > >>>>>>>>>> done. > >>>>>>>> > >>>>>>>> ... > >>>>>>>> > >>>>>>>>> Just complementing: on media, we do this per video buffer (or > >>>>>>>>> per half video buffer). A typical use case on cameras is to have > >>>>>>>>> buffers transferred 30 times per second, if the video was streamed > >>>>>>>>> at 30 frames per second. > >>>>>>>> > >>>>>>>> IIRC some big use case for this hardware was transcoding so there > >>>>>>>> was a > >>>>>>>> desire to just go at whatever rate the hardware could support as > >>>>>>>> there > >>>>>>>> is no interactive user consuming the output as it is generated. > >>>>>>> > >>>>>>> Indeed, codecs could be used to just do transcoding, but I would > >>>>>>> expect it to be a border use case. See, as the chipsets implementing > >>>>>>> codecs are typically the ones used on mobiles, I would expect that > >>>>>>> the major use cases to be to watch audio and video and to participate > >>>>>>> on audio/video conferences. > >>>>>>> > >>>>>>> Going further, the codec API may end supporting not only transcoding > >>>>>>> (which is something that CPU can usually handle without too much > >>>>>>> processing) but also audio processing that may require more > >>>>>>> complex algorithms - even deep learning ones - like background noise > >>>>>>> removal, echo detection/removal, volume auto-gain, audio enhancement > >>>>>>> and such. > >>>>>>> > >>>>>>> On other words, the typical use cases will either have input > >>>>>>> or output being a physical hardware (microphone or speaker). > >>>>>>> > >>>>>> > >>>>>> All, thanks for spending time to discuss, it seems we go back to > >>>>>> the start point of this topic again. > >>>>>> > >>>>>> Our main request is that there is a hardware sample rate converter > >>>>>> on the chip, so users can use it in user space as a component like > >>>>>> software sample rate converter. It mostly may run as a gstreamer > >>>>>> plugin. > >>>>>> so it is a memory to memory component. > >>>>>> > >>>>>> I didn't find such API in ALSA for such purpose, the best option for > >>>>>> this > >>>>>> in the kernel is the V4L2 memory to memory framework I found. > >>>>>> As Hans said it is well designed for memory to memory. > >>>>>> > >>>>>> And I think audio is one of 'media'. As I can see that part of Radio > >>>>>> function is in ALSA, part of Radio function is in V4L2. part of HDMI > >>>>>> function is in DRM, pa
Re: [PATCH v15 00/16] Add audio support in v4l2 framework
On Thu, May 9, 2024 at 5:50 PM Amadeusz Sławiński wrote: > > On 5/9/2024 11:36 AM, Shengjiu Wang wrote: > > On Wed, May 8, 2024 at 4:14 PM Amadeusz Sławiński > > wrote: > >> > >> On 5/8/2024 10:00 AM, Hans Verkuil wrote: > >>> On 06/05/2024 10:49, Shengjiu Wang wrote: > >>>> On Fri, May 3, 2024 at 4:42 PM Mauro Carvalho Chehab > >>>> wrote: > >>>>> > >>>>> Em Fri, 3 May 2024 10:47:19 +0900 > >>>>> Mark Brown escreveu: > >>>>> > >>>>>> On Thu, May 02, 2024 at 10:26:43AM +0100, Mauro Carvalho Chehab wrote: > >>>>>>> Mauro Carvalho Chehab escreveu: > >>>>>> > >>>>>>>> There are still time control associated with it, as audio and video > >>>>>>>> needs to be in sync. This is done by controlling the buffers size > >>>>>>>> and could be fine-tuned by checking when the buffer transfer is done. > >>>>>> > >>>>>> ... > >>>>>> > >>>>>>> Just complementing: on media, we do this per video buffer (or > >>>>>>> per half video buffer). A typical use case on cameras is to have > >>>>>>> buffers transferred 30 times per second, if the video was streamed > >>>>>>> at 30 frames per second. > >>>>>> > >>>>>> IIRC some big use case for this hardware was transcoding so there was a > >>>>>> desire to just go at whatever rate the hardware could support as there > >>>>>> is no interactive user consuming the output as it is generated. > >>>>> > >>>>> Indeed, codecs could be used to just do transcoding, but I would > >>>>> expect it to be a border use case. See, as the chipsets implementing > >>>>> codecs are typically the ones used on mobiles, I would expect that > >>>>> the major use cases to be to watch audio and video and to participate > >>>>> on audio/video conferences. > >>>>> > >>>>> Going further, the codec API may end supporting not only transcoding > >>>>> (which is something that CPU can usually handle without too much > >>>>> processing) but also audio processing that may require more > >>>>> complex algorithms - even deep learning ones - like background noise > >>>>> removal, echo detection/removal, volume auto-gain, audio enhancement > >>>>> and such. > >>>>> > >>>>> On other words, the typical use cases will either have input > >>>>> or output being a physical hardware (microphone or speaker). > >>>>> > >>>> > >>>> All, thanks for spending time to discuss, it seems we go back to > >>>> the start point of this topic again. > >>>> > >>>> Our main request is that there is a hardware sample rate converter > >>>> on the chip, so users can use it in user space as a component like > >>>> software sample rate converter. It mostly may run as a gstreamer plugin. > >>>> so it is a memory to memory component. > >>>> > >>>> I didn't find such API in ALSA for such purpose, the best option for this > >>>> in the kernel is the V4L2 memory to memory framework I found. > >>>> As Hans said it is well designed for memory to memory. > >>>> > >>>> And I think audio is one of 'media'. As I can see that part of Radio > >>>> function is in ALSA, part of Radio function is in V4L2. part of HDMI > >>>> function is in DRM, part of HDMI function is in ALSA... > >>>> So using V4L2 for audio is not new from this point of view. > >>>> > >>>> Even now I still think V4L2 is the best option, but it looks like there > >>>> are a lot of rejects. If develop a new ALSA-mem2mem, it is also > >>>> a duplication of code (bigger duplication that just add audio support > >>>> in V4L2 I think). > >>> > >>> After reading this thread I still believe that the mem2mem framework is > >>> a reasonable option, unless someone can come up with a method that is > >>> easy to implement in the alsa subsystem. From what I can tell from this > >>> discussion no such method exists. > >>> > >> > >> Hi, > >> > >
Re: [PATCH v15 00/16] Add audio support in v4l2 framework
On Wed, May 8, 2024 at 4:14 PM Amadeusz Sławiński wrote: > > On 5/8/2024 10:00 AM, Hans Verkuil wrote: > > On 06/05/2024 10:49, Shengjiu Wang wrote: > >> On Fri, May 3, 2024 at 4:42 PM Mauro Carvalho Chehab > >> wrote: > >>> > >>> Em Fri, 3 May 2024 10:47:19 +0900 > >>> Mark Brown escreveu: > >>> > >>>> On Thu, May 02, 2024 at 10:26:43AM +0100, Mauro Carvalho Chehab wrote: > >>>>> Mauro Carvalho Chehab escreveu: > >>>> > >>>>>> There are still time control associated with it, as audio and video > >>>>>> needs to be in sync. This is done by controlling the buffers size > >>>>>> and could be fine-tuned by checking when the buffer transfer is done. > >>>> > >>>> ... > >>>> > >>>>> Just complementing: on media, we do this per video buffer (or > >>>>> per half video buffer). A typical use case on cameras is to have > >>>>> buffers transferred 30 times per second, if the video was streamed > >>>>> at 30 frames per second. > >>>> > >>>> IIRC some big use case for this hardware was transcoding so there was a > >>>> desire to just go at whatever rate the hardware could support as there > >>>> is no interactive user consuming the output as it is generated. > >>> > >>> Indeed, codecs could be used to just do transcoding, but I would > >>> expect it to be a border use case. See, as the chipsets implementing > >>> codecs are typically the ones used on mobiles, I would expect that > >>> the major use cases to be to watch audio and video and to participate > >>> on audio/video conferences. > >>> > >>> Going further, the codec API may end supporting not only transcoding > >>> (which is something that CPU can usually handle without too much > >>> processing) but also audio processing that may require more > >>> complex algorithms - even deep learning ones - like background noise > >>> removal, echo detection/removal, volume auto-gain, audio enhancement > >>> and such. > >>> > >>> On other words, the typical use cases will either have input > >>> or output being a physical hardware (microphone or speaker). > >>> > >> > >> All, thanks for spending time to discuss, it seems we go back to > >> the start point of this topic again. > >> > >> Our main request is that there is a hardware sample rate converter > >> on the chip, so users can use it in user space as a component like > >> software sample rate converter. It mostly may run as a gstreamer plugin. > >> so it is a memory to memory component. > >> > >> I didn't find such API in ALSA for such purpose, the best option for this > >> in the kernel is the V4L2 memory to memory framework I found. > >> As Hans said it is well designed for memory to memory. > >> > >> And I think audio is one of 'media'. As I can see that part of Radio > >> function is in ALSA, part of Radio function is in V4L2. part of HDMI > >> function is in DRM, part of HDMI function is in ALSA... > >> So using V4L2 for audio is not new from this point of view. > >> > >> Even now I still think V4L2 is the best option, but it looks like there > >> are a lot of rejects. If develop a new ALSA-mem2mem, it is also > >> a duplication of code (bigger duplication that just add audio support > >> in V4L2 I think). > > > > After reading this thread I still believe that the mem2mem framework is > > a reasonable option, unless someone can come up with a method that is > > easy to implement in the alsa subsystem. From what I can tell from this > > discussion no such method exists. > > > > Hi, > > my main question would be how is mem2mem use case different from > loopback exposing playback and capture frontends in user space with DSP > (or other piece of HW) in the middle? > I think loopback has a timing control, user need to feed data to playback at a fixed time and get data from capture at a fixed time. Otherwise there is xrun in playback and capture. mem2mem case: there is no such timing control, user feeds data to it then it generates output, if user doesn't feed data, there is no xrun. but mem2mem is just one of the components in the playback or capture pipeline, overall there is time control for whole pipeline, Best regards Shengjiu Wang > Amadeusz >
[PATCH 4/4] ASoC: fsl_xcvr: Add support for i.MX95 platform
On i.MX95, the XCVR uses a new PLL in the PHY, which is
General Purpose (GP) PLL. Add GP PLL configuration support
in the driver and add the 'pll_ver' flag to distinguish
different PLL on different platforms.
The XCVR also use PHY but limited for SPDIF only case
Add 'use_phy' flag to distinguish these platforms.
Signed-off-by: Shengjiu Wang
Reviewed-by: Chancel Liu
---
sound/soc/fsl/fsl_xcvr.c | 120 +--
sound/soc/fsl/fsl_xcvr.h | 91 +
2 files changed, 168 insertions(+), 43 deletions(-)
diff --git a/sound/soc/fsl/fsl_xcvr.c b/sound/soc/fsl/fsl_xcvr.c
index 0ffa10e924ef..6b1715ac67c5 100644
--- a/sound/soc/fsl/fsl_xcvr.c
+++ b/sound/soc/fsl/fsl_xcvr.c
@@ -20,10 +20,17 @@
#define FSL_XCVR_CAPDS_SIZE256
+enum fsl_xcvr_pll_verison {
+ PLL_MX8MP,
+ PLL_MX95,
+};
+
struct fsl_xcvr_soc_data {
const char *fw_name;
bool spdif_only;
bool use_edma;
+ bool use_phy;
+ enum fsl_xcvr_pll_verison pll_ver;
};
struct fsl_xcvr {
@@ -265,10 +272,10 @@ static int fsl_xcvr_ai_write(struct fsl_xcvr *xcvr, u8
reg, u32 data, bool phy)
static int fsl_xcvr_en_phy_pll(struct fsl_xcvr *xcvr, u32 freq, bool tx)
{
struct device *dev = >pdev->dev;
- u32 i, div = 0, log2;
+ u32 i, div = 0, log2, val;
int ret;
- if (xcvr->soc_data->spdif_only)
+ if (!xcvr->soc_data->use_phy)
return 0;
for (i = 0; i < ARRAY_SIZE(fsl_xcvr_pll_cfg); i++) {
@@ -291,45 +298,62 @@ static int fsl_xcvr_en_phy_pll(struct fsl_xcvr *xcvr, u32
freq, bool tx)
return ret;
}
- /* PLL: BANDGAP_SET: EN_VBG (enable bandgap) */
- fsl_xcvr_ai_write(xcvr, FSL_XCVR_PLL_BANDGAP_SET,
- FSL_XCVR_PLL_BANDGAP_EN_VBG, 0);
-
- /* PLL: CTRL0: DIV_INTEGER */
- fsl_xcvr_ai_write(xcvr, FSL_XCVR_PLL_CTRL0, fsl_xcvr_pll_cfg[i].mfi, 0);
- /* PLL: NUMERATOR: MFN */
- fsl_xcvr_ai_write(xcvr, FSL_XCVR_PLL_NUM, fsl_xcvr_pll_cfg[i].mfn, 0);
- /* PLL: DENOMINATOR: MFD */
- fsl_xcvr_ai_write(xcvr, FSL_XCVR_PLL_DEN, fsl_xcvr_pll_cfg[i].mfd, 0);
- /* PLL: CTRL0_SET: HOLD_RING_OFF, POWER_UP */
- fsl_xcvr_ai_write(xcvr, FSL_XCVR_PLL_CTRL0_SET,
- FSL_XCVR_PLL_CTRL0_HROFF | FSL_XCVR_PLL_CTRL0_PWP, 0);
- udelay(25);
- /* PLL: CTRL0: Clear Hold Ring Off */
- fsl_xcvr_ai_write(xcvr, FSL_XCVR_PLL_CTRL0_CLR,
- FSL_XCVR_PLL_CTRL0_HROFF, 0);
- udelay(100);
- if (tx) { /* TX is enabled for SPDIF only */
- /* PLL: POSTDIV: PDIV0 */
- fsl_xcvr_ai_write(xcvr, FSL_XCVR_PLL_PDIV,
- FSL_XCVR_PLL_PDIVx(log2, 0), 0);
- /* PLL: CTRL_SET: CLKMUX0_EN */
- fsl_xcvr_ai_write(xcvr, FSL_XCVR_PLL_CTRL0_SET,
- FSL_XCVR_PLL_CTRL0_CM0_EN, 0);
- } else if (xcvr->mode == FSL_XCVR_MODE_EARC) { /* eARC RX */
- /* PLL: POSTDIV: PDIV1 */
- fsl_xcvr_ai_write(xcvr, FSL_XCVR_PLL_PDIV,
- FSL_XCVR_PLL_PDIVx(log2, 1), 0);
- /* PLL: CTRL_SET: CLKMUX1_EN */
- fsl_xcvr_ai_write(xcvr, FSL_XCVR_PLL_CTRL0_SET,
- FSL_XCVR_PLL_CTRL0_CM1_EN, 0);
- } else { /* SPDIF / ARC RX */
- /* PLL: POSTDIV: PDIV2 */
- fsl_xcvr_ai_write(xcvr, FSL_XCVR_PLL_PDIV,
- FSL_XCVR_PLL_PDIVx(log2, 2), 0);
- /* PLL: CTRL_SET: CLKMUX2_EN */
+ switch (xcvr->soc_data->pll_ver) {
+ case PLL_MX8MP:
+ /* PLL: BANDGAP_SET: EN_VBG (enable bandgap) */
+ fsl_xcvr_ai_write(xcvr, FSL_XCVR_PLL_BANDGAP_SET,
+ FSL_XCVR_PLL_BANDGAP_EN_VBG, 0);
+
+ /* PLL: CTRL0: DIV_INTEGER */
+ fsl_xcvr_ai_write(xcvr, FSL_XCVR_PLL_CTRL0,
fsl_xcvr_pll_cfg[i].mfi, 0);
+ /* PLL: NUMERATOR: MFN */
+ fsl_xcvr_ai_write(xcvr, FSL_XCVR_PLL_NUM,
fsl_xcvr_pll_cfg[i].mfn, 0);
+ /* PLL: DENOMINATOR: MFD */
+ fsl_xcvr_ai_write(xcvr, FSL_XCVR_PLL_DEN,
fsl_xcvr_pll_cfg[i].mfd, 0);
+ /* PLL: CTRL0_SET: HOLD_RING_OFF, POWER_UP */
fsl_xcvr_ai_write(xcvr, FSL_XCVR_PLL_CTRL0_SET,
- FSL_XCVR_PLL_CTRL0_CM2_EN, 0);
+ FSL_XCVR_PLL_CTRL0_HROFF |
FSL_XCVR_PLL_CTRL0_PWP, 0);
+ udelay(25);
+ /* PLL: CTRL0: Clear Hold Ring Off */
+ fsl_xcvr_ai_write(xcvr, FSL_XCVR_PLL_CTRL0_CLR,
+ FSL_XCVR_PLL_CTRL0_HROFF, 0);
+ udelay(100);
+ if (tx) { /* TX is enabled for SPDIF only */
+ /* PLL: POSTDIV: PDIV0 */
+
[PATCH 3/4] ASoC: fsl_xcvr: Support reparent pll clocks for phy_clk
When there are 'pll8k' and 'pll11k' clock existing, the clock
source of 'phy_clk' can be changed for different sample rate
requirement.
Signed-off-by: Shengjiu Wang
---
sound/soc/fsl/fsl_xcvr.c | 8
1 file changed, 8 insertions(+)
diff --git a/sound/soc/fsl/fsl_xcvr.c b/sound/soc/fsl/fsl_xcvr.c
index c46f64557a7f..0ffa10e924ef 100644
--- a/sound/soc/fsl/fsl_xcvr.c
+++ b/sound/soc/fsl/fsl_xcvr.c
@@ -15,6 +15,7 @@
#include
#include "fsl_xcvr.h"
+#include "fsl_utils.h"
#include "imx-pcm.h"
#define FSL_XCVR_CAPDS_SIZE256
@@ -33,6 +34,8 @@ struct fsl_xcvr {
struct clk *pll_ipg_clk;
struct clk *phy_clk;
struct clk *spba_clk;
+ struct clk *pll8k_clk;
+ struct clk *pll11k_clk;
struct reset_control *reset;
u8 streams;
u32 mode;
@@ -362,6 +365,8 @@ static int fsl_xcvr_en_aud_pll(struct fsl_xcvr *xcvr, u32
freq)
freq = xcvr->soc_data->spdif_only ? freq / 5 : freq;
clk_disable_unprepare(xcvr->phy_clk);
+ fsl_asoc_reparent_pll_clocks(dev, xcvr->phy_clk,
+xcvr->pll8k_clk, xcvr->pll11k_clk, freq);
ret = clk_set_rate(xcvr->phy_clk, freq);
if (ret < 0) {
dev_err(dev, "Error while setting AUD PLL rate: %d\n", ret);
@@ -1287,6 +1292,9 @@ static int fsl_xcvr_probe(struct platform_device *pdev)
return PTR_ERR(xcvr->pll_ipg_clk);
}
+ fsl_asoc_get_pll_clocks(dev, >pll8k_clk,
+ >pll11k_clk);
+
xcvr->ram_addr = devm_platform_ioremap_resource_byname(pdev, "ram");
if (IS_ERR(xcvr->ram_addr))
return PTR_ERR(xcvr->ram_addr);
--
2.34.1
[PATCH 0/4] ASoC: fsl_xcvr: Support i.MX95 platform
On i.MX95 wakeup domain, there is one instance of Audio XCVR supporting SPDIF mode with a connection to the Audio XCVR physical interface. Shengjiu Wang (4): ASoC: dt-bindings: fsl,xcvr: Add compatible string for i.MX95 ASoC: dt-bindings: fsl,xcvr: Add two PLL clock sources ASoC: fsl_xcvr: Support reparent pll clocks for phy_clk ASoC: fsl_xcvr: Add support for i.MX95 platform .../devicetree/bindings/sound/fsl,xcvr.yaml | 7 + sound/soc/fsl/fsl_xcvr.c | 128 -- sound/soc/fsl/fsl_xcvr.h | 91 + 3 files changed, 183 insertions(+), 43 deletions(-) -- 2.34.1
[PATCH 2/4] ASoC: dt-bindings: fsl,xcvr: Add two PLL clock sources
Add two PLL clock sources, they are the parent clocks of the root clock one is for 8kHz series rates, named as 'pll8k', another one is for 11kHz series rates, named as 'pll11k'. They are optional clocks, if there are such clocks, then the driver can switch between them to support more accurate sample rates. As 'pll8k' and 'pll11k' are optional, then add 'minItems: 4' for clocks and clock-names properties. Signed-off-by: Shengjiu Wang --- Documentation/devicetree/bindings/sound/fsl,xcvr.yaml | 6 ++ 1 file changed, 6 insertions(+) diff --git a/Documentation/devicetree/bindings/sound/fsl,xcvr.yaml b/Documentation/devicetree/bindings/sound/fsl,xcvr.yaml index 1c74a32def09..c4660faed404 100644 --- a/Documentation/devicetree/bindings/sound/fsl,xcvr.yaml +++ b/Documentation/devicetree/bindings/sound/fsl,xcvr.yaml @@ -50,6 +50,9 @@ properties: - description: PHY clock - description: SPBA clock - description: PLL clock + - description: PLL clock source for 8kHz series + - description: PLL clock source for 11kHz series +minItems: 4 clock-names: items: @@ -57,6 +60,9 @@ properties: - const: phy - const: spba - const: pll_ipg + - const: pll8k + - const: pll11k +minItems: 4 dmas: items: -- 2.34.1
[PATCH 1/4] ASoC: dt-bindings: fsl,xcvr: Add compatible string for i.MX95
Add compatible string "fsl,imx95-xcvr" for i.MX95 platform. Signed-off-by: Shengjiu Wang --- Documentation/devicetree/bindings/sound/fsl,xcvr.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/Documentation/devicetree/bindings/sound/fsl,xcvr.yaml b/Documentation/devicetree/bindings/sound/fsl,xcvr.yaml index 0eb0c1ba8710..1c74a32def09 100644 --- a/Documentation/devicetree/bindings/sound/fsl,xcvr.yaml +++ b/Documentation/devicetree/bindings/sound/fsl,xcvr.yaml @@ -22,6 +22,7 @@ properties: enum: - fsl,imx8mp-xcvr - fsl,imx93-xcvr + - fsl,imx95-xcvr reg: items: -- 2.34.1
Re: [PATCH v15 00/16] Add audio support in v4l2 framework
On Fri, May 3, 2024 at 4:42 PM Mauro Carvalho Chehab wrote: > > Em Fri, 3 May 2024 10:47:19 +0900 > Mark Brown escreveu: > > > On Thu, May 02, 2024 at 10:26:43AM +0100, Mauro Carvalho Chehab wrote: > > > Mauro Carvalho Chehab escreveu: > > > > > > There are still time control associated with it, as audio and video > > > > needs to be in sync. This is done by controlling the buffers size > > > > and could be fine-tuned by checking when the buffer transfer is done. > > > > ... > > > > > Just complementing: on media, we do this per video buffer (or > > > per half video buffer). A typical use case on cameras is to have > > > buffers transferred 30 times per second, if the video was streamed > > > at 30 frames per second. > > > > IIRC some big use case for this hardware was transcoding so there was a > > desire to just go at whatever rate the hardware could support as there > > is no interactive user consuming the output as it is generated. > > Indeed, codecs could be used to just do transcoding, but I would > expect it to be a border use case. See, as the chipsets implementing > codecs are typically the ones used on mobiles, I would expect that > the major use cases to be to watch audio and video and to participate > on audio/video conferences. > > Going further, the codec API may end supporting not only transcoding > (which is something that CPU can usually handle without too much > processing) but also audio processing that may require more > complex algorithms - even deep learning ones - like background noise > removal, echo detection/removal, volume auto-gain, audio enhancement > and such. > > On other words, the typical use cases will either have input > or output being a physical hardware (microphone or speaker). > All, thanks for spending time to discuss, it seems we go back to the start point of this topic again. Our main request is that there is a hardware sample rate converter on the chip, so users can use it in user space as a component like software sample rate converter. It mostly may run as a gstreamer plugin. so it is a memory to memory component. I didn't find such API in ALSA for such purpose, the best option for this in the kernel is the V4L2 memory to memory framework I found. As Hans said it is well designed for memory to memory. And I think audio is one of 'media'. As I can see that part of Radio function is in ALSA, part of Radio function is in V4L2. part of HDMI function is in DRM, part of HDMI function is in ALSA... So using V4L2 for audio is not new from this point of view. Even now I still think V4L2 is the best option, but it looks like there are a lot of rejects. If develop a new ALSA-mem2mem, it is also a duplication of code (bigger duplication that just add audio support in V4L2 I think). Best regards Shengjiu Wang. > > > I would assume that, on an audio/video stream, the audio data > > > transfer will be programmed to also happen on a regular interval. > > > > With audio the API is very much "wake userspace every Xms".
Re: [PATCH v2] ASoC: dt-bindings: fsl,ssi: Convert to YAML
On Tue, Apr 23, 2024 at 10:24 PM Rob Herring wrote: > > On Mon, Apr 22, 2024 at 01:48:14PM +0800, Shengjiu Wang wrote: > > Convert the fsl,ssi binding to YAML. > > > > Add below compatible strings which were not listed > > in document: > > > > fsl,imx50-ssi > > fsl,imx53-ssi > > fsl,imx25-ssi > > fsl,imx27-ssi > > fsl,imx6q-ssi > > fsl,imx6sl-ssi > > fsl,imx6sx-ssi > > > > Add below fsl,mode strings which were not listed. > > > > i2s-slave > > i2s-master > > lj-slave > > lj-master > > rj-slave > > rj-master > > > > Add 'ac97-gpios' property which were not listed. > > Then dtbs_check can pass. > > > > And remove the 'codec' description which should be > > in the 'codec' binding doc. > > > > Signed-off-by: Shengjiu Wang > > --- > > changes in v2: > > - change fallback string to const. > > - add dai-common.yaml > > - add ac97-gpios property > > > > .../devicetree/bindings/sound/fsl,ssi.txt | 87 > > .../devicetree/bindings/sound/fsl,ssi.yaml| 192 ++ > > 2 files changed, 192 insertions(+), 87 deletions(-) > > delete mode 100644 Documentation/devicetree/bindings/sound/fsl,ssi.txt > > create mode 100644 Documentation/devicetree/bindings/sound/fsl,ssi.yaml > > > > diff --git a/Documentation/devicetree/bindings/sound/fsl,ssi.txt > > b/Documentation/devicetree/bindings/sound/fsl,ssi.txt > > deleted file mode 100644 > > index 7e15a85cecd2.. > > --- a/Documentation/devicetree/bindings/sound/fsl,ssi.txt > > +++ /dev/null > > @@ -1,87 +0,0 @@ > > -Freescale Synchronous Serial Interface > > - > > -The SSI is a serial device that communicates with audio codecs. It can > > -be programmed in AC97, I2S, left-justified, or right-justified modes. > > - > > -Required properties: > > -- compatible: Compatible list, should contain one of the following > > -compatibles: > > - fsl,mpc8610-ssi > > - fsl,imx51-ssi > > - fsl,imx35-ssi > > - fsl,imx21-ssi > > -- cell-index: The SSI, <0> = SSI1, <1> = SSI2, and so on. > > -- reg: Offset and length of the register set for the device. > > -- interrupts:where a is the interrupt number and b is a > > -field that represents an encoding of the sense and > > -level information for the interrupt. This should be > > -encoded based on the information in section 2) > > -depending on the type of interrupt controller you > > -have. > > -- fsl,fifo-depth: The number of elements in the transmit and receive > > FIFOs. > > -This number is the maximum allowed value for > > SFCSR[TFWM0]. > > - - clocks: "ipg" - Required clock for the SSI unit > > -"baud" - Required clock for SSI master mode. Otherwise > > this > > - clock is not used > > - > > -Required are also ac97 link bindings if ac97 is used. See > > -Documentation/devicetree/bindings/sound/soc-ac97link.txt for the necessary > > -bindings. > > - > > -Optional properties: > > -- codec-handle: Phandle to a 'codec' node that defines an audio > > -codec connected to this SSI. This node is typically > > -a child of an I2C or other control node. > > -- fsl,fiq-stream-filter: Bool property. Disabled DMA and use FIQ instead to > > - filter the codec stream. This is necessary for some boards > > - where an incompatible codec is connected to this SSI, e.g. > > - on pca100 and pcm043. > > -- dmas: Generic dma devicetree binding as described in > > - Documentation/devicetree/bindings/dma/dma.txt. > > -- dma-names: Two dmas have to be defined, "tx" and "rx", if fsl,imx-fiq > > - is not defined. > > -- fsl,mode: The operating mode for the AC97 interface only. > > -"ac97-slave" - AC97 mode, SSI is clock slave > > -"ac97-master" - AC97 mode, SSI is clock master > > -- fsl,ssi-asynchronous: > > -If specified, the SSI is to be programmed in > > asynchronous > > -mode. In this mode, pins SRCK, STCK, SRFS, and ST
[PATCH v2] ASoC: dt-bindings: fsl,ssi: Convert to YAML
Convert the fsl,ssi binding to YAML. Add below compatible strings which were not listed in document: fsl,imx50-ssi fsl,imx53-ssi fsl,imx25-ssi fsl,imx27-ssi fsl,imx6q-ssi fsl,imx6sl-ssi fsl,imx6sx-ssi Add below fsl,mode strings which were not listed. i2s-slave i2s-master lj-slave lj-master rj-slave rj-master Add 'ac97-gpios' property which were not listed. Then dtbs_check can pass. And remove the 'codec' description which should be in the 'codec' binding doc. Signed-off-by: Shengjiu Wang --- changes in v2: - change fallback string to const. - add dai-common.yaml - add ac97-gpios property .../devicetree/bindings/sound/fsl,ssi.txt | 87 .../devicetree/bindings/sound/fsl,ssi.yaml| 192 ++ 2 files changed, 192 insertions(+), 87 deletions(-) delete mode 100644 Documentation/devicetree/bindings/sound/fsl,ssi.txt create mode 100644 Documentation/devicetree/bindings/sound/fsl,ssi.yaml diff --git a/Documentation/devicetree/bindings/sound/fsl,ssi.txt b/Documentation/devicetree/bindings/sound/fsl,ssi.txt deleted file mode 100644 index 7e15a85cecd2.. --- a/Documentation/devicetree/bindings/sound/fsl,ssi.txt +++ /dev/null @@ -1,87 +0,0 @@ -Freescale Synchronous Serial Interface - -The SSI is a serial device that communicates with audio codecs. It can -be programmed in AC97, I2S, left-justified, or right-justified modes. - -Required properties: -- compatible: Compatible list, should contain one of the following -compatibles: - fsl,mpc8610-ssi - fsl,imx51-ssi - fsl,imx35-ssi - fsl,imx21-ssi -- cell-index: The SSI, <0> = SSI1, <1> = SSI2, and so on. -- reg: Offset and length of the register set for the device. -- interrupts:where a is the interrupt number and b is a -field that represents an encoding of the sense and -level information for the interrupt. This should be -encoded based on the information in section 2) -depending on the type of interrupt controller you -have. -- fsl,fifo-depth: The number of elements in the transmit and receive FIFOs. -This number is the maximum allowed value for SFCSR[TFWM0]. - - clocks: "ipg" - Required clock for the SSI unit -"baud" - Required clock for SSI master mode. Otherwise this - clock is not used - -Required are also ac97 link bindings if ac97 is used. See -Documentation/devicetree/bindings/sound/soc-ac97link.txt for the necessary -bindings. - -Optional properties: -- codec-handle: Phandle to a 'codec' node that defines an audio -codec connected to this SSI. This node is typically -a child of an I2C or other control node. -- fsl,fiq-stream-filter: Bool property. Disabled DMA and use FIQ instead to - filter the codec stream. This is necessary for some boards - where an incompatible codec is connected to this SSI, e.g. - on pca100 and pcm043. -- dmas:Generic dma devicetree binding as described in - Documentation/devicetree/bindings/dma/dma.txt. -- dma-names: Two dmas have to be defined, "tx" and "rx", if fsl,imx-fiq - is not defined. -- fsl,mode: The operating mode for the AC97 interface only. -"ac97-slave" - AC97 mode, SSI is clock slave -"ac97-master" - AC97 mode, SSI is clock master -- fsl,ssi-asynchronous: -If specified, the SSI is to be programmed in asynchronous -mode. In this mode, pins SRCK, STCK, SRFS, and STFS must -all be connected to valid signals. In synchronous mode, -SRCK and SRFS are ignored. Asynchronous mode allows -playback and capture to use different sample sizes and -sample rates. Some drivers may require that SRCK and STCK -be connected together, and SRFS and STFS be connected -together. This would still allow different sample sizes, -but not different sample rates. -- fsl,playback-dma: Phandle to a node for the DMA channel to use for -playback of audio. This is typically dictated by SOC -design. See the notes below. -Only used on Power Architecture. -- fsl,capture-dma: Phandle to a node for the DMA channel to use for -capture (recording) of audio. This is typically dictated -by SOC design. See the notes below. -Only used on Power Architecture. - -Child 'codec' node required properties: -- compatible: Compat
[PATCH 2/2] ASoC: dt-bindings: fsl-asoc-card: Add compatbile string for wm8904 codec
The wm8904 codec is used on an i.MX95 Toradex board. Signed-off-by: Shengjiu Wang --- Documentation/devicetree/bindings/sound/fsl-asoc-card.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/Documentation/devicetree/bindings/sound/fsl-asoc-card.yaml b/Documentation/devicetree/bindings/sound/fsl-asoc-card.yaml index 42ca39eebd49..16093b48823a 100644 --- a/Documentation/devicetree/bindings/sound/fsl-asoc-card.yaml +++ b/Documentation/devicetree/bindings/sound/fsl-asoc-card.yaml @@ -76,6 +76,7 @@ properties: - fsl,imx-audio-tlv320aic31xx - fsl,imx-audio-tlv320aic32x4 - fsl,imx-audio-wm8524 + - fsl,imx-audio-wm8904 - fsl,imx-audio-wm8960 - fsl,imx-audio-wm8962 - fsl,imx-audio-wm8958 -- 2.34.1
[PATCH 1/2] ASoC: fsl-asoc-card: add wm8904 codec support
wm8904 codec is used on i.MX95 Toradex board
Signed-off-by: Shengjiu Wang
---
sound/soc/fsl/fsl-asoc-card.c | 8
1 file changed, 8 insertions(+)
diff --git a/sound/soc/fsl/fsl-asoc-card.c b/sound/soc/fsl/fsl-asoc-card.c
index bc07f26ba303..f6d2564864c6 100644
--- a/sound/soc/fsl/fsl-asoc-card.c
+++ b/sound/soc/fsl/fsl-asoc-card.c
@@ -28,6 +28,7 @@
#include "../codecs/wm8994.h"
#include "../codecs/tlv320aic31xx.h"
#include "../codecs/nau8822.h"
+#include "../codecs/wm8904.h"
#define DRIVER_NAME "fsl-asoc-card"
@@ -709,6 +710,12 @@ static int fsl_asoc_card_probe(struct platform_device
*pdev)
priv->dai_fmt |= SND_SOC_DAIFMT_CBM_CFM;
if (codec_dev)
priv->codec_priv.mclk = devm_clk_get(codec_dev, NULL);
+ } else if (of_device_is_compatible(np, "fsl,imx-audio-wm8904")) {
+ codec_dai_name = "wm8904-hifi";
+ priv->codec_priv.mclk_id = WM8904_FLL_MCLK;
+ priv->codec_priv.fll_id = WM8904_CLK_FLL;
+ priv->codec_priv.pll_id = WM8904_FLL_MCLK;
+ priv->dai_fmt |= SND_SOC_DAIFMT_CBP_CFP;
} else {
dev_err(>dev, "unknown Device Tree compatible\n");
ret = -EINVAL;
@@ -935,6 +942,7 @@ static const struct of_device_id fsl_asoc_card_dt_ids[] = {
{ .compatible = "fsl,imx-audio-si476x", },
{ .compatible = "fsl,imx-audio-wm8958", },
{ .compatible = "fsl,imx-audio-nau8822", },
+ { .compatible = "fsl,imx-audio-wm8904", },
{}
};
MODULE_DEVICE_TABLE(of, fsl_asoc_card_dt_ids);
--
2.34.1
[PATCH 0/2] ASoC: fsl-asoc-card: add wm8904 codec support
add wm8904 codec support in fsl-asoc-card. Shengjiu Wang (2): ASoC: fsl-asoc-card: add wm8904 codec support ASoC: dt-bindings: fsl-asoc-card: Add compatbile string for wm8904 codec .../devicetree/bindings/sound/fsl-asoc-card.yaml | 1 + sound/soc/fsl/fsl-asoc-card.c | 8 2 files changed, 9 insertions(+) -- 2.34.1
Re: [PATCH 1/1] ASoC: fsl: fsl_ssi: Add dev_err_probe if PCM DMA init fails
On Thu, Mar 14, 2024 at 10:16 PM Alexander Stein
wrote:
>
> This happens especially if this driver is built-in, but SDMA driver
> is configured as module.
>
> Signed-off-by: Alexander Stein
Acked-by: Shengjiu Wang
Best Regards
Shengjiu Wang
> ---
> sound/soc/fsl/fsl_ssi.c | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/sound/soc/fsl/fsl_ssi.c b/sound/soc/fsl/fsl_ssi.c
> index ab6ec1974807..4ca3a16f7ac0 100644
> --- a/sound/soc/fsl/fsl_ssi.c
> +++ b/sound/soc/fsl/fsl_ssi.c
> @@ -1401,8 +1401,10 @@ static int fsl_ssi_imx_probe(struct platform_device
> *pdev,
> goto error_pcm;
> } else {
> ret = imx_pcm_dma_init(pdev);
> - if (ret)
> + if (ret) {
> + dev_err_probe(dev, ret, "Failed to init PCM DMA\n");
> goto error_pcm;
> + }
> }
>
> return 0;
> --
> 2.34.1
>
Re: [PATCH v3 0/5] ASoC: fsl: Support register and unregister rpmsg sound card through remoteproc
On Mon, Mar 11, 2024 at 7:14 PM Chancel Liu wrote: > > echo /lib/firmware/fw.elf > /sys/class/remoteproc/remoteproc0/firmware > (A) echo start > /sys/class/remoteproc/remoteproc0/state > (B) echo stop > /sys/class/remoteproc/remoteproc0/state > > The rpmsg sound card is registered in (A) and unregistered in (B). > After "start", imx-audio-rpmsg registers devices for ASoC platform driver > and machine driver. Then sound card is registered. After "stop", > imx-audio-rpmsg unregisters devices for ASoC platform driver and machine > driver. Then sound card is unregistered. Acked-by: Shengjiu Wang Best regards Shengjiu Wang > > changes in v2 > - Fix build errors reported by kernel test robot > > changes in v3 > - Add a new patch for fsl_rpmsg to register CPU DAI with rpmsg channel > name > - Update imx-rpmsg.c to get DT node of ASoC CPU DAI device with rpmsg > channel name instead of using undocumented bindings > > Chancel Liu (5): > ASoC: fsl: imx-pcm-rpmsg: Register component with rpmsg channel name > ASoC: fsl: imx-audio-rpmsg: Register device with rpmsg channel name > ASoC: fsl: Let imx-audio-rpmsg register platform device for card > ASoC: fsl: fsl_rpmsg: Register CPU DAI with name of rpmsg channel > ASoC: fsl: imx-rpmsg: Update to correct DT node > > sound/soc/fsl/fsl_rpmsg.c | 43 - > sound/soc/fsl/imx-audio-rpmsg.c | 21 +--- > sound/soc/fsl/imx-pcm-rpmsg.c | 11 ++--- > sound/soc/fsl/imx-rpmsg.c | 28 ++--- > 4 files changed, 71 insertions(+), 32 deletions(-) > > -- > 2.43.0 >
[PATCH v15 16/16] media: vim2m-audio: add virtual driver for audio memory to memory
Audio memory to memory virtual driver use video memory to memory
virtual driver vim2m.c as example. The main difference is
device type is VFL_TYPE_AUDIO and device cap type is V4L2_CAP_AUDIO_M2M.
The device_run function is a dummy function, which is simply
copy the data from input buffer to output buffer.
Signed-off-by: Shengjiu Wang
---
MAINTAINERS | 9 +
drivers/media/test-drivers/Kconfig | 10 +
drivers/media/test-drivers/Makefile | 1 +
drivers/media/test-drivers/vim2m-audio.c | 793 +++
4 files changed, 813 insertions(+)
create mode 100644 drivers/media/test-drivers/vim2m-audio.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 7b8b9ee65c61..215d40d80508 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -23575,6 +23575,15 @@ L: linux-fsde...@vger.kernel.org
S: Maintained
F: fs/vboxsf/*
+VIRTUAL MEM2MEM DRIVER FOR AUDIO
+M: Hans Verkuil
+M: Shengjiu Wang
+L: linux-me...@vger.kernel.org
+S: Maintained
+W: https://linuxtv.org
+T: git git://linuxtv.org/media_tree.git
+F: drivers/media/test-drivers/vim2m-audio.c
+
VIRTUAL PCM TEST DRIVER
M: Ivan Orlov
L: linux-so...@vger.kernel.org
diff --git a/drivers/media/test-drivers/Kconfig
b/drivers/media/test-drivers/Kconfig
index 5a5379524bde..b6b52a7ca042 100644
--- a/drivers/media/test-drivers/Kconfig
+++ b/drivers/media/test-drivers/Kconfig
@@ -16,6 +16,16 @@ config VIDEO_VIM2M
This is a virtual test device for the memory-to-memory driver
framework.
+config VIDEO_VIM2M_AUDIO
+ tristate "Virtual Memory-to-Memory Driver For Audio"
+ depends on VIDEO_DEV
+ select VIDEOBUF2_VMALLOC
+ select V4L2_MEM2MEM_DEV
+ select MEDIA_CONTROLLER
+ help
+ This is a virtual audio test device for the memory-to-memory driver
+ framework.
+
source "drivers/media/test-drivers/vicodec/Kconfig"
source "drivers/media/test-drivers/vimc/Kconfig"
source "drivers/media/test-drivers/vivid/Kconfig"
diff --git a/drivers/media/test-drivers/Makefile
b/drivers/media/test-drivers/Makefile
index 740714a4584d..0c61c9ada3e1 100644
--- a/drivers/media/test-drivers/Makefile
+++ b/drivers/media/test-drivers/Makefile
@@ -10,6 +10,7 @@ obj-$(CONFIG_DVB_VIDTV) += vidtv/
obj-$(CONFIG_VIDEO_VICODEC) += vicodec/
obj-$(CONFIG_VIDEO_VIM2M) += vim2m.o
+obj-$(CONFIG_VIDEO_VIM2M_AUDIO) += vim2m-audio.o
obj-$(CONFIG_VIDEO_VIMC) += vimc/
obj-$(CONFIG_VIDEO_VIVID) += vivid/
obj-$(CONFIG_VIDEO_VISL) += visl/
diff --git a/drivers/media/test-drivers/vim2m-audio.c
b/drivers/media/test-drivers/vim2m-audio.c
new file mode 100644
index ..6361df6320b3
--- /dev/null
+++ b/drivers/media/test-drivers/vim2m-audio.c
@@ -0,0 +1,793 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * A virtual v4l2-mem2mem example for audio device.
+ */
+
+#include
+#include
+#include
+#include
+#include
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+MODULE_DESCRIPTION("Virtual device for audio mem2mem testing");
+MODULE_LICENSE("GPL");
+
+static unsigned int debug;
+module_param(debug, uint, 0644);
+MODULE_PARM_DESC(debug, "debug level");
+
+#define MEM2MEM_NAME "vim2m-audio"
+
+#define dprintk(dev, lvl, fmt, arg...) \
+ v4l2_dbg(lvl, debug, &(dev)->v4l2_dev, "%s: " fmt, __func__, ## arg)
+
+#define SAMPLE_NUM 4096
+
+static void audm2m_dev_release(struct device *dev)
+{}
+
+static struct platform_device audm2m_pdev = {
+ .name = MEM2MEM_NAME,
+ .dev.release= audm2m_dev_release,
+};
+
+static u32 formats[] = {
+ V4L2_AUDIO_FMT_S16_LE,
+};
+
+#define NUM_FORMATS ARRAY_SIZE(formats)
+
+/* Per-queue, driver-specific private data */
+struct audm2m_q_data {
+ unsigned intrate;
+ unsigned intchannels;
+ unsigned intbuffersize;
+ unsigned intsequence;
+ u32 fourcc;
+};
+
+enum {
+ V4L2_M2M_SRC = 0,
+ V4L2_M2M_DST = 1,
+};
+
+static snd_pcm_format_t find_format(u32 fourcc)
+{
+ snd_pcm_format_t fmt;
+ unsigned int k;
+
+ for (k = 0; k < NUM_FORMATS; k++) {
+ if (formats[k] == fourcc)
+ break;
+ }
+
+ if (k == NUM_FORMATS)
+ return 0;
+
+ fmt = v4l2_fourcc_to_audfmt(formats[k]);
+
+ return fmt;
+}
+
+struct audm2m_dev {
+ struct v4l2_device v4l2_dev;
+ struct video_device vfd;
+
+ struct mutexdev_mutex;
+
+ struct v4l2_m2m_dev *m2m_dev;
+#ifdef CONFIG_MEDIA_CONTROLLER
+ struct media_device mdev;
+#endif
+};
+
+struct audm2m_ctx {
+ struct v4l2_fh fh;
+ struct v4l2_ctrl_handlerctrl_handler;
+ struct audm2m_dev *dev;
+
+ struct mutex
[PATCH v15 15/16] media: imx-asrc: Add memory to memory driver
Implement the ASRC memory to memory function using
the v4l2 framework, user can use this function with
v4l2 ioctl interface.
User send the output and capture buffer to driver and
driver store the converted data to the capture buffer.
This feature can be shared by ASRC and EASRC drivers
Signed-off-by: Shengjiu Wang
---
MAINTAINERS |8 +
drivers/media/platform/nxp/Kconfig| 13 +
drivers/media/platform/nxp/Makefile |1 +
drivers/media/platform/nxp/imx-asrc.c | 1256 +
4 files changed, 1278 insertions(+)
create mode 100644 drivers/media/platform/nxp/imx-asrc.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 375d34363777..7b8b9ee65c61 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -15821,6 +15821,14 @@ F: drivers/nvmem/
F: include/linux/nvmem-consumer.h
F: include/linux/nvmem-provider.h
+NXP ASRC V4L2 MEM2MEM DRIVERS
+M: Shengjiu Wang
+L: linux-me...@vger.kernel.org
+S: Maintained
+W: https://linuxtv.org
+T: git git://linuxtv.org/media_tree.git
+F: drivers/media/platform/nxp/imx-asrc.c
+
NXP BLUETOOTH WIRELESS DRIVERS
M: Amitkumar Karwar
M: Neeraj Kale
diff --git a/drivers/media/platform/nxp/Kconfig
b/drivers/media/platform/nxp/Kconfig
index 40e3436669e2..8d0ca335601f 100644
--- a/drivers/media/platform/nxp/Kconfig
+++ b/drivers/media/platform/nxp/Kconfig
@@ -67,3 +67,16 @@ config VIDEO_MX2_EMMAPRP
source "drivers/media/platform/nxp/dw100/Kconfig"
source "drivers/media/platform/nxp/imx-jpeg/Kconfig"
+
+config VIDEO_IMX_ASRC
+ tristate "NXP i.MX ASRC M2M support"
+ depends on V4L_MEM2MEM_DRIVERS
+ depends on MEDIA_SUPPORT
+ select VIDEOBUF2_DMA_CONTIG
+ select V4L2_MEM2MEM_DEV
+ select MEDIA_CONTROLLER
+ help
+ Say Y if you want to add ASRC M2M support for NXP CPUs.
+ It is a complement for ASRC M2P and ASRC P2M features.
+ This option is only useful for out-of-tree drivers since
+ in-tree drivers select it automatically.
diff --git a/drivers/media/platform/nxp/Makefile
b/drivers/media/platform/nxp/Makefile
index 4d90eb713652..1325675e34f5 100644
--- a/drivers/media/platform/nxp/Makefile
+++ b/drivers/media/platform/nxp/Makefile
@@ -9,3 +9,4 @@ obj-$(CONFIG_VIDEO_IMX8MQ_MIPI_CSI2) += imx8mq-mipi-csi2.o
obj-$(CONFIG_VIDEO_IMX_MIPI_CSIS) += imx-mipi-csis.o
obj-$(CONFIG_VIDEO_IMX_PXP) += imx-pxp.o
obj-$(CONFIG_VIDEO_MX2_EMMAPRP) += mx2_emmaprp.o
+obj-$(CONFIG_VIDEO_IMX_ASRC) += imx-asrc.o
diff --git a/drivers/media/platform/nxp/imx-asrc.c
b/drivers/media/platform/nxp/imx-asrc.c
new file mode 100644
index ..0c25a36199b1
--- /dev/null
+++ b/drivers/media/platform/nxp/imx-asrc.c
@@ -0,0 +1,1256 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (C) 2014-2016 Freescale Semiconductor, Inc.
+// Copyright (C) 2019-2023 NXP
+//
+// Freescale ASRC Memory to Memory (M2M) driver
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#define V4L_CAP OUT
+#define V4L_OUT IN
+
+#define ASRC_xPUT_DMA_CALLBACK(dir) \
+ (((dir) == V4L_OUT) ? asrc_input_dma_callback \
+ : asrc_output_dma_callback)
+
+#define DIR_STR(dir) (dir) == V4L_OUT ? "out" : "cap"
+
+/* Maximum output and capture buffer size */
+#define ASRC_M2M_BUFFER_SIZE (512 * 1024)
+
+/* Maximum output and capture period size */
+#define ASRC_M2M_PERIOD_SIZE (48 * 1024)
+
+struct asrc_pair_m2m {
+ struct fsl_asrc_pair *pair;
+ struct asrc_m2m *m2m;
+ struct v4l2_fh fh;
+ struct v4l2_ctrl_handler ctrl_handler;
+ int channels[2];
+ unsigned int sequence[2];
+ s64 src_rate_off_prev; /* Q31.32 */
+ s64 dst_rate_off_prev; /* Q31.32 */
+ s64 src_rate_off_cur; /* Q31.32 */
+ s64 dst_rate_off_cur; /* Q31.32 */
+};
+
+struct asrc_m2m {
+ struct fsl_asrc_m2m_pdata pdata;
+ struct v4l2_device v4l2_dev;
+ struct v4l2_m2m_dev *m2m_dev;
+ struct video_device *dec_vdev;
+ struct mutex mlock; /* v4l2 ioctls serialization */
+ struct platform_device *pdev;
+#ifdef CONFIG_MEDIA_CONTROLLER
+ struct media_device mdev;
+#endif
+};
+
+static u32 formats[] = {
+ V4L2_AUDIO_FMT_S8,
+ V4L2_AUDIO_FMT_S16_LE,
+ V4L2_AUDIO_FMT_U16_LE,
+ V4L2_AUDIO_FMT_S24_LE,
+ V4L2_AUDIO_FMT_S24_3LE,
+ V4L2_AUDIO_FMT_U24_LE,
+ V4L2_AUDIO_FMT_U24_3LE,
+ V4L2_AUDIO_FMT_S32_LE,
+ V4L2_AUDIO_FMT_U32_LE,
+ V4L2_AUDIO_FMT_S20_3LE,
+ V4L2_AUDIO_FMT_U20_3LE,
+ V4L2_AUDIO_FMT_FLOAT_LE,
+ V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE,
+};
+
+#define NUM_FORMATS ARRAY_SIZE(formats)
+
+static const s64 asrc_v1_m2m_rates[] = {
+ 5512, 8000, 11025, 12000, 16000,
+ 22050, 24000, 32000, 44100,
+ 48000, 64000, 88200, 96000,
+ 128000, 176400,
[PATCH v15 14/16] media: vivid: add fixed point test controls
Add fixed point test controls, one is for Q4.16 format
another one is for Q63 format.
Signed-off-by: Shengjiu Wang
---
drivers/media/test-drivers/vivid/vivid-core.h | 2 ++
.../media/test-drivers/vivid/vivid-ctrls.c| 26 +++
include/media/v4l2-ctrls.h| 6 +
3 files changed, 34 insertions(+)
diff --git a/drivers/media/test-drivers/vivid/vivid-core.h
b/drivers/media/test-drivers/vivid/vivid-core.h
index cfb8e66083f6..f65465191bc9 100644
--- a/drivers/media/test-drivers/vivid/vivid-core.h
+++ b/drivers/media/test-drivers/vivid/vivid-core.h
@@ -222,6 +222,8 @@ struct vivid_dev {
struct v4l2_ctrl*boolean;
struct v4l2_ctrl*int32;
struct v4l2_ctrl*int64;
+ struct v4l2_ctrl*int32_q16;
+ struct v4l2_ctrl*int64_q63;
struct v4l2_ctrl*menu;
struct v4l2_ctrl*string;
struct v4l2_ctrl*bitmask;
diff --git a/drivers/media/test-drivers/vivid/vivid-ctrls.c
b/drivers/media/test-drivers/vivid/vivid-ctrls.c
index f2b20e25a7a4..2444ea95b285 100644
--- a/drivers/media/test-drivers/vivid/vivid-ctrls.c
+++ b/drivers/media/test-drivers/vivid/vivid-ctrls.c
@@ -38,6 +38,8 @@
#define VIVID_CID_U8_PIXEL_ARRAY (VIVID_CID_CUSTOM_BASE + 14)
#define VIVID_CID_S32_ARRAY(VIVID_CID_CUSTOM_BASE + 15)
#define VIVID_CID_S64_ARRAY(VIVID_CID_CUSTOM_BASE + 16)
+#define VIVID_CID_INT_Q4_16(VIVID_CID_CUSTOM_BASE + 17)
+#define VIVID_CID_INT64_Q63(VIVID_CID_CUSTOM_BASE + 18)
#define VIVID_CID_VIVID_BASE (0x00f0 | 0xf000)
#define VIVID_CID_VIVID_CLASS (0x00f0 | 1)
@@ -182,6 +184,28 @@ static const struct v4l2_ctrl_config vivid_ctrl_int64 = {
.step = 1,
};
+static const struct v4l2_ctrl_config vivid_ctrl_int32_q16 = {
+ .ops = _user_gen_ctrl_ops,
+ .id = VIVID_CID_INT_Q4_16,
+ .name = "Integer 32 Bits Q4.16",
+ .type = V4L2_CTRL_TYPE_INTEGER,
+ .min = v4l2_ctrl_fp_compose(-16, 0, 16),
+ .max = v4l2_ctrl_fp_compose(15, 0x, 16),
+ .step = 1,
+ .fraction_bits = 16,
+};
+
+static const struct v4l2_ctrl_config vivid_ctrl_int64_q63 = {
+ .ops = _user_gen_ctrl_ops,
+ .id = VIVID_CID_INT64_Q63,
+ .name = "Integer 64 Bits Q63",
+ .type = V4L2_CTRL_TYPE_INTEGER64,
+ .min = v4l2_ctrl_fp_compose(-1, 0, 63),
+ .max = v4l2_ctrl_fp_compose(0, LLONG_MAX, 63),
+ .step = 1,
+ .fraction_bits = 63,
+};
+
static const struct v4l2_ctrl_config vivid_ctrl_u32_array = {
.ops = _user_gen_ctrl_ops,
.id = VIVID_CID_U32_ARRAY,
@@ -1670,6 +1694,8 @@ int vivid_create_controls(struct vivid_dev *dev, bool
show_ccs_cap,
dev->button = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_button,
NULL);
dev->int32 = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_int32,
NULL);
dev->int64 = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_int64,
NULL);
+ dev->int32_q16 = v4l2_ctrl_new_custom(hdl_user_gen,
_ctrl_int32_q16, NULL);
+ dev->int64_q63 = v4l2_ctrl_new_custom(hdl_user_gen,
_ctrl_int64_q63, NULL);
dev->boolean = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_boolean,
NULL);
dev->menu = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_menu, NULL);
dev->string = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_string,
NULL);
diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
index c35514c5bf88..197d8b67ac13 100644
--- a/include/media/v4l2-ctrls.h
+++ b/include/media/v4l2-ctrls.h
@@ -1593,4 +1593,10 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl);
*/
int v4l2_ctrl_type_op_validate(const struct v4l2_ctrl *ctrl, union
v4l2_ctrl_ptr ptr);
+/*
+ * Fixed point compose helper define. This helper maps to the value
+ * i + f / (1 << fraction_bits).
+ */
+#define v4l2_ctrl_fp_compose(i, f, fraction_bits) (((s64)(i) << fraction_bits)
+ (f))
+
#endif
--
2.34.1
[PATCH v15 13/16] media: uapi: Add an entity type for audio resampler
Add and document a media entity type for an audio resampler.
It is MEDIA_ENT_F_PROC_AUDIO_RESAMPLER.
Signed-off-by: Shengjiu Wang
---
Documentation/userspace-api/media/mediactl/media-types.rst | 6 ++
include/uapi/linux/media.h | 1 +
2 files changed, 7 insertions(+)
diff --git a/Documentation/userspace-api/media/mediactl/media-types.rst
b/Documentation/userspace-api/media/mediactl/media-types.rst
index adfb37430f8e..d353f17c3344 100644
--- a/Documentation/userspace-api/media/mediactl/media-types.rst
+++ b/Documentation/userspace-api/media/mediactl/media-types.rst
@@ -40,6 +40,7 @@ Types and flags used to represent the media graph elements
.. _MEDIA-ENT-F-PROC-VIDEO-ENCODER:
.. _MEDIA-ENT-F-PROC-VIDEO-DECODER:
.. _MEDIA-ENT-F-PROC-VIDEO-ISP:
+.. _MEDIA-ENT-F-PROC-AUDIO-RESAMPLER:
.. _MEDIA-ENT-F-VID-MUX:
.. _MEDIA-ENT-F-VID-IF-BRIDGE:
.. _MEDIA-ENT-F-DV-DECODER:
@@ -208,6 +209,11 @@ Types and flags used to represent the media graph elements
combination of custom V4L2 controls and IOCTLs, and parameters
supplied in a metadata buffer.
+* - ``MEDIA_ENT_F_PROC_AUDIO_RESAMPLER``
+ - An Audio Resampler device. An entity capable of
+ resampling an audio stream from one sample rate to another sample
+ rate. Must have one sink pad and at least one source pad.
+
* - ``MEDIA_ENT_F_VID_MUX``
- Video multiplexer. An entity capable of multiplexing must have at
least two sink pads and one source pad, and must pass the video
diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
index 9ff6dec7393a..a8266eaa8042 100644
--- a/include/uapi/linux/media.h
+++ b/include/uapi/linux/media.h
@@ -125,6 +125,7 @@ struct media_device_info {
#define MEDIA_ENT_F_PROC_VIDEO_ENCODER (MEDIA_ENT_F_BASE + 0x4007)
#define MEDIA_ENT_F_PROC_VIDEO_DECODER (MEDIA_ENT_F_BASE + 0x4008)
#define MEDIA_ENT_F_PROC_VIDEO_ISP (MEDIA_ENT_F_BASE + 0x4009)
+#define MEDIA_ENT_F_PROC_AUDIO_RESAMPLER (MEDIA_ENT_F_BASE + 0x400a)
/*
* Switch and bridge entity functions
--
2.34.1
[PATCH v15 12/16] media: uapi: Declare interface types for Audio
Declare the interface types that will be used by Audio.
The type is MEDIA_INTF_T_V4L_AUDIO.
Signed-off-by: Shengjiu Wang
---
.../userspace-api/media/mediactl/media-types.rst| 5 +
drivers/media/v4l2-core/v4l2-dev.c | 4
drivers/media/v4l2-core/v4l2-mem2mem.c | 13 +
include/uapi/linux/media.h | 1 +
4 files changed, 19 insertions(+), 4 deletions(-)
diff --git a/Documentation/userspace-api/media/mediactl/media-types.rst
b/Documentation/userspace-api/media/mediactl/media-types.rst
index 6332e8395263..adfb37430f8e 100644
--- a/Documentation/userspace-api/media/mediactl/media-types.rst
+++ b/Documentation/userspace-api/media/mediactl/media-types.rst
@@ -265,6 +265,7 @@ Types and flags used to represent the media graph elements
.. _MEDIA-INTF-T-V4L-SUBDEV:
.. _MEDIA-INTF-T-V4L-SWRADIO:
.. _MEDIA-INTF-T-V4L-TOUCH:
+.. _MEDIA-INTF-T-V4L-AUDIO:
.. _MEDIA-INTF-T-ALSA-PCM-CAPTURE:
.. _MEDIA-INTF-T-ALSA-PCM-PLAYBACK:
.. _MEDIA-INTF-T-ALSA-CONTROL:
@@ -322,6 +323,10 @@ Types and flags used to represent the media graph elements
- Device node interface for Touch device (V4L)
- typically, /dev/v4l-touch?
+* - ``MEDIA_INTF_T_V4L_AUDIO``
+ - Device node interface for Audio device (V4L)
+ - typically, /dev/v4l-audio?
+
* - ``MEDIA_INTF_T_ALSA_PCM_CAPTURE``
- Device node interface for ALSA PCM Capture
- typically, /dev/snd/pcmC?D?c
diff --git a/drivers/media/v4l2-core/v4l2-dev.c
b/drivers/media/v4l2-core/v4l2-dev.c
index bac008fcedc6..ca8462a61e1f 100644
--- a/drivers/media/v4l2-core/v4l2-dev.c
+++ b/drivers/media/v4l2-core/v4l2-dev.c
@@ -844,6 +844,10 @@ static int video_register_media_controller(struct
video_device *vdev)
intf_type = MEDIA_INTF_T_V4L_SUBDEV;
/* Entity will be created via v4l2_device_register_subdev() */
break;
+ case VFL_TYPE_AUDIO:
+ intf_type = MEDIA_INTF_T_V4L_AUDIO;
+ /* Entity will be created via v4l2_device_register_subdev() */
+ break;
default:
return 0;
}
diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c
b/drivers/media/v4l2-core/v4l2-mem2mem.c
index 75517134a5e9..cda5e255305f 100644
--- a/drivers/media/v4l2-core/v4l2-mem2mem.c
+++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
@@ -1143,10 +1143,15 @@ int v4l2_m2m_register_media_controller(struct
v4l2_m2m_dev *m2m_dev,
if (ret)
goto err_rm_links0;
- /* Create video interface */
- m2m_dev->intf_devnode = media_devnode_create(mdev,
- MEDIA_INTF_T_V4L_VIDEO, 0,
- VIDEO_MAJOR, vdev->minor);
+ if (vdev->vfl_type == VFL_TYPE_AUDIO)
+ m2m_dev->intf_devnode = media_devnode_create(mdev,
+ MEDIA_INTF_T_V4L_AUDIO, 0,
+ VIDEO_MAJOR, vdev->minor);
+ else
+ /* Create video interface */
+ m2m_dev->intf_devnode = media_devnode_create(mdev,
+ MEDIA_INTF_T_V4L_VIDEO, 0,
+ VIDEO_MAJOR, vdev->minor);
if (!m2m_dev->intf_devnode) {
ret = -ENOMEM;
goto err_rm_links1;
diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
index 1c80b1d6bbaf..9ff6dec7393a 100644
--- a/include/uapi/linux/media.h
+++ b/include/uapi/linux/media.h
@@ -260,6 +260,7 @@ struct media_links_enum {
#define MEDIA_INTF_T_V4L_SUBDEV(MEDIA_INTF_T_V4L_BASE
+ 3)
#define MEDIA_INTF_T_V4L_SWRADIO (MEDIA_INTF_T_V4L_BASE + 4)
#define MEDIA_INTF_T_V4L_TOUCH (MEDIA_INTF_T_V4L_BASE + 5)
+#define MEDIA_INTF_T_V4L_AUDIO (MEDIA_INTF_T_V4L_BASE + 6)
#define MEDIA_INTF_T_ALSA_BASE 0x0300
#define MEDIA_INTF_T_ALSA_PCM_CAPTURE (MEDIA_INTF_T_ALSA_BASE)
--
2.34.1
[PATCH v15 11/16] media: uapi: Add audio rate controls support
Add V4L2_CID_M2M_AUDIO_SOURCE_RATE and V4L2_CID_M2M_AUDIO_DEST_RATE
new IDs for rate control.
Add V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET and
V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET for clock drift.
Signed-off-by: Shengjiu Wang
---
.../media/v4l/ext-ctrls-audio-m2m.rst | 38 +++
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 6 +++
include/uapi/linux/v4l2-controls.h| 5 +++
3 files changed, 49 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
index 82d2ecedbfee..b137b7c442e6 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
@@ -19,3 +19,41 @@ Audio M2M Control IDs
The Audio M2M class descriptor. Calling
:ref:`VIDIOC_QUERYCTRL` for this control will
return a description of this control class.
+
+.. _v4l2-audio-asrc:
+
+``V4L2_CID_M2M_AUDIO_SOURCE_RATE (integer menu)``
+This control specifies the audio source sample rate, unit is Hz
+
+``V4L2_CID_M2M_AUDIO_DEST_RATE (integer menu)``
+This control specifies the audio destination sample rate, unit is Hz
+
+``V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET (fixed point)``
+This control specifies the offset from the audio source sample rate,
+unit is Hz.
+
+The offset compensates for any clock drift. The actual source audio
+sample rate is the ideal source audio sample rate from
+``V4L2_CID_M2M_AUDIO_SOURCE_RATE`` plus this fixed point offset.
+
+The audio source clock may have some drift. Reducing or increasing the
+audio sample rate dynamically to ensure that Sample Rate Converter is
+working on the real sample rate, this feature is for the Asynchronous
+Sample Rate Converter module.
+So, userspace would be expected to be monitoring such drift
+and increasing/decreasing the sample frequency as needed by this control.
+
+``V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET (fixed point)``
+This control specifies the offset from the audio destination sample rate,
+unit is Hz.
+
+The offset compensates for any clock drift. The actual destination audio
+sample rate is the ideal source audio sample rate from
+``V4L2_CID_M2M_AUDIO_DEST_RATE`` plus this fixed point offset.
+
+The audio destination clock may have some drift. Reducing or increasing
+the audio sample rate dynamically to ensure that sample rate converter
+is working on the real sample rate, this feature is for the Asynchronous
+Sample Rate Converter module.
+So, userspace would be expected to be monitoring such drift
+and increasing/decreasing the sample frequency as needed by this control.
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index 2a85ea3dc92f..91e1f5348c23 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -1245,6 +1245,8 @@ const char *v4l2_ctrl_get_name(u32 id)
/* Audio M2M controls */
case V4L2_CID_M2M_AUDIO_CLASS: return "Audio M2M Controls";
+ case V4L2_CID_M2M_AUDIO_SOURCE_RATE:return "Audio Source Sample
Rate";
+ case V4L2_CID_M2M_AUDIO_DEST_RATE: return "Audio Destination
Sample Rate";
default:
return NULL;
}
@@ -1606,6 +1608,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum
v4l2_ctrl_type *type,
case V4L2_CID_COLORIMETRY_HDR10_MASTERING_DISPLAY:
*type = V4L2_CTRL_TYPE_HDR10_MASTERING_DISPLAY;
break;
+ case V4L2_CID_M2M_AUDIO_SOURCE_RATE:
+ case V4L2_CID_M2M_AUDIO_DEST_RATE:
+ *type = V4L2_CTRL_TYPE_INTEGER_MENU;
+ break;
default:
*type = V4L2_CTRL_TYPE_INTEGER;
break;
diff --git a/include/uapi/linux/v4l2-controls.h
b/include/uapi/linux/v4l2-controls.h
index a8b4b830c757..30129ccdc282 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -3495,6 +3495,11 @@ struct v4l2_ctrl_av1_film_grain {
#define V4L2_CID_M2M_AUDIO_CLASS_BASE (V4L2_CTRL_CLASS_M2M_AUDIO | 0x900)
#define V4L2_CID_M2M_AUDIO_CLASS (V4L2_CTRL_CLASS_M2M_AUDIO | 1)
+#define V4L2_CID_M2M_AUDIO_SOURCE_RATE (V4L2_CID_M2M_AUDIO_CLASS_BASE + 0)
+#define V4L2_CID_M2M_AUDIO_DEST_RATE (V4L2_CID_M2M_AUDIO_CLASS_BASE + 1)
+#define V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET (V4L2_CID_M2M_AUDIO_CLASS_BASE
+ 2)
+#define V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET(V4L2_CID_M2M_AUDIO_CLASS_BASE
+ 3)
+
/* MPEG-compression definitions kept for backwards compatibility */
#ifndef __KERNEL__
#define V4L2_CTRL_CLASS_MPEGV4L2_CTRL_CLASS_CODEC
--
2.34.1
[PATCH v15 10/16] media: uapi: Add V4L2_CTRL_CLASS_M2M_AUDIO
The Audio M2M class includes controls for audio memory-to-memory
use cases. The controls can be used for audio codecs, audio
preprocessing, audio postprocessing.
Signed-off-by: Shengjiu Wang
---
.../userspace-api/media/v4l/common.rst| 1 +
.../media/v4l/ext-ctrls-audio-m2m.rst | 21 +++
.../media/v4l/vidioc-g-ext-ctrls.rst | 4
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 4
include/uapi/linux/v4l2-controls.h| 4
5 files changed, 34 insertions(+)
create mode 100644
Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
diff --git a/Documentation/userspace-api/media/v4l/common.rst
b/Documentation/userspace-api/media/v4l/common.rst
index ea0435182e44..d5366e96a596 100644
--- a/Documentation/userspace-api/media/v4l/common.rst
+++ b/Documentation/userspace-api/media/v4l/common.rst
@@ -52,6 +52,7 @@ applicable to all devices.
ext-ctrls-fm-rx
ext-ctrls-detect
ext-ctrls-colorimetry
+ext-ctrls-audio-m2m
fourcc
format
planar-apis
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
new file mode 100644
index ..82d2ecedbfee
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
@@ -0,0 +1,21 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+
+.. _audiom2m-controls:
+
+***
+Audio M2M Control Reference
+***
+
+The Audio M2M class includes controls for audio memory-to-memory
+use cases. The controls can be used for audio codecs, audio
+preprocessing, audio postprocessing.
+
+Audio M2M Control IDs
+---
+
+.. _audiom2m-control-id:
+
+``V4L2_CID_M2M_AUDIO_CLASS (class)``
+The Audio M2M class descriptor. Calling
+:ref:`VIDIOC_QUERYCTRL` for this control will
+return a description of this control class.
diff --git a/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
b/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
index 4d56c0528ad7..aeb1ad8e7d29 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
@@ -488,6 +488,10 @@ still cause this situation.
- 0xa5
- The class containing colorimetry controls. These controls are
described in :ref:`colorimetry-controls`.
+* - ``V4L2_CTRL_CLASS_M2M_AUDIO``
+ - 0xa6
+ - The class containing audio m2m controls. These controls are
+ described in :ref:`audiom2m-controls`.
Return Value
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index 8696eb1cdd61..2a85ea3dc92f 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -1242,6 +1242,9 @@ const char *v4l2_ctrl_get_name(u32 id)
case V4L2_CID_COLORIMETRY_CLASS:return "Colorimetry Controls";
case V4L2_CID_COLORIMETRY_HDR10_CLL_INFO: return "HDR10
Content Light Info";
case V4L2_CID_COLORIMETRY_HDR10_MASTERING_DISPLAY: return "HDR10
Mastering Display";
+
+ /* Audio M2M controls */
+ case V4L2_CID_M2M_AUDIO_CLASS: return "Audio M2M Controls";
default:
return NULL;
}
@@ -1451,6 +1454,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum
v4l2_ctrl_type *type,
case V4L2_CID_DETECT_CLASS:
case V4L2_CID_CODEC_STATELESS_CLASS:
case V4L2_CID_COLORIMETRY_CLASS:
+ case V4L2_CID_M2M_AUDIO_CLASS:
*type = V4L2_CTRL_TYPE_CTRL_CLASS;
/* You can neither read nor write these */
*flags |= V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_WRITE_ONLY;
diff --git a/include/uapi/linux/v4l2-controls.h
b/include/uapi/linux/v4l2-controls.h
index 99c3f5e99da7..a8b4b830c757 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -30,6 +30,7 @@
#define V4L2_CTRL_CLASS_DETECT 0x00a3 /* Detection controls */
#define V4L2_CTRL_CLASS_CODEC_STATELESS 0x00a4 /* Stateless codecs
controls */
#define V4L2_CTRL_CLASS_COLORIMETRY0x00a5 /* Colorimetry controls
*/
+#define V4L2_CTRL_CLASS_M2M_AUDIO 0x00a6 /* Audio M2M controls */
/* User-class control IDs */
@@ -3491,6 +3492,9 @@ struct v4l2_ctrl_av1_film_grain {
__u8 reserved[4];
};
+#define V4L2_CID_M2M_AUDIO_CLASS_BASE (V4L2_CTRL_CLASS_M2M_AUDIO | 0x900)
+#define V4L2_CID_M2M_AUDIO_CLASS (V4L2_CTRL_CLASS_M2M_AUDIO | 1)
+
/* MPEG-compression definitions kept for backwards compatibility */
#ifndef __KERNEL__
#define V4L2_CTRL_CLASS_MPEGV4L2_CTRL_CLASS_CODEC
--
2.34.1
[PATCH v15 09/16] media: uapi: Define audio sample format fourcc type
The audio sample format definition is from alsa,
the header file is include/uapi/sound/asound.h, but
don't include this header file directly, because in
user space, there is another copy in alsa-lib.
There will be conflict in userspace for include
videodev2.h & asound.h and asoundlib.h
Here still use the fourcc format.
Signed-off-by: Shengjiu Wang
---
.../userspace-api/media/v4l/pixfmt-audio.rst | 100 ++
.../userspace-api/media/v4l/pixfmt.rst| 1 +
drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++
include/uapi/linux/videodev2.h| 29 +
4 files changed, 143 insertions(+)
create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
new file mode 100644
index ..a66db9a19936
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
@@ -0,0 +1,100 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+
+.. _pixfmt-audio:
+
+*
+Audio Formats
+*
+
+These formats are used for :ref:`audiomem2mem` interface only.
+
+All FourCCs starting with 'AU' are reserved for mappings
+of the snd_pcm_format_t type.
+
+The v4l2_audfmt_to_fourcc() is defined to convert the snd_pcm_format_t
+type to a FourCC. The first character is 'A', the second character
+is 'U', and the remaining two characters is the snd_pcm_format_t
+value in ASCII. Example: SNDRV_PCM_FORMAT_S16_LE (with value 2)
+maps to 'AU02' and SNDRV_PCM_FORMAT_S24_3LE (with value 32) maps
+to 'AU32'."
+
+The v4l2_fourcc_to_audfmt() is defined to convert these FourCCs to
+snd_pcm_format_t type.
+
+.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}|
+
+.. cssclass:: longtable
+
+.. flat-table:: Audio Format
+:header-rows: 1
+:stub-columns: 0
+:widths: 3 1 4
+
+* - Identifier
+ - Code
+ - Details
+* .. _V4L2-AUDIO-FMT-S8:
+
+ - ``V4L2_AUDIO_FMT_S8``
+ - 'S8'
+ - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA
+* .. _V4L2-AUDIO-FMT-S16-LE:
+
+ - ``V4L2_AUDIO_FMT_S16_LE``
+ - 'S16_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA
+* .. _V4L2-AUDIO-FMT-U16-LE:
+
+ - ``V4L2_AUDIO_FMT_U16_LE``
+ - 'U16_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA
+* .. _V4L2-AUDIO-FMT-S24-LE:
+
+ - ``V4L2_AUDIO_FMT_S24_LE``
+ - 'S24_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA
+* .. _V4L2-AUDIO-FMT-U24-LE:
+
+ - ``V4L2_AUDIO_FMT_U24_LE``
+ - 'U24_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA
+* .. _V4L2-AUDIO-FMT-S32-LE:
+
+ - ``V4L2_AUDIO_FMT_S32_LE``
+ - 'S32_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA
+* .. _V4L2-AUDIO-FMT-U32-LE:
+
+ - ``V4L2_AUDIO_FMT_U32_LE``
+ - 'U32_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA
+* .. _V4L2-AUDIO-FMT-FLOAT-LE:
+
+ - ``V4L2_AUDIO_FMT_FLOAT_LE``
+ - 'FLOAT_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_FLOAT_LE in ALSA
+* .. _V4L2-AUDIO-FMT-IEC958-SUBFRAME-LE:
+
+ - ``V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE``
+ - 'IEC958_SUBFRAME_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_IEC958_SUBFRAME_LE in ALSA
+* .. _V4L2-AUDIO-FMT-S24-3LE:
+
+ - ``V4L2_AUDIO_FMT_S24_3LE``
+ - 'S24_3LE'
+ - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
+* .. _V4L2-AUDIO-FMT-U24-3LE:
+
+ - ``V4L2_AUDIO_FMT_U24_3LE``
+ - 'U24_3LE'
+ - Corresponds to SNDRV_PCM_FORMAT_U24_3LE in ALSA
+* .. _V4L2-AUDIO-FMT-S20-3LE:
+
+ - ``V4L2_AUDIO_FMT_S20_3LE``
+ - 'S20_3LE'
+ - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
+* .. _V4L2-AUDIO-FMT-U20-3LE:
+
+ - ``V4L2_AUDIO_FMT_U20_3LE``
+ - 'U20_3LE'
+ - Corresponds to SNDRV_PCM_FORMAT_U20_3LE in ALSA
diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst
b/Documentation/userspace-api/media/v4l/pixfmt.rst
index 11dab4a90630..2eb6fdd3b43d 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt.rst
@@ -36,3 +36,4 @@ see also :ref:`VIDIOC_G_FBUF `.)
colorspaces
colorspaces-defs
colorspaces-details
+pixfmt-audio
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
b/drivers/media/v4l2-core/v4l2-ioctl.c
index 961abcdf7290..be229c69e991 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1471,6 +1471,19 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_Y210: descr = "10-bit YUYV Packed"; break;
case V4L2_PIX_FMT_Y212: descr = "12-bit YUYV Packed"; break;
case V4L2_PIX_FMT_Y216: descr = "16-bit YUYV Packed"; break;
+ case V4L2_AUDIO_FMT_S8: descr = "8-bit Signed"; break;
+ case V4L2_AUDIO_FMT_S16_LE: descr = "16-bit
[PATCH v15 08/16] media: v4l2: Add audio capture and output support
Audio signal processing has the requirement for memory to
memory similar as Video.
This patch is to add this support in v4l2 framework, defined
new buffer type V4L2_BUF_TYPE_AUDIO_CAPTURE and
V4L2_BUF_TYPE_AUDIO_OUTPUT, defined new format v4l2_audio_format
for audio case usage.
The created audio device is named "/dev/v4l-audioX".
Signed-off-by: Shengjiu Wang
---
.../userspace-api/media/v4l/buffer.rst| 6 ++
.../media/v4l/dev-audio-mem2mem.rst | 71 +++
.../userspace-api/media/v4l/devices.rst | 1 +
.../media/v4l/vidioc-enum-fmt.rst | 2 +
.../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 ++
.../media/videodev2.h.rst.exceptions | 2 +
.../media/common/videobuf2/videobuf2-v4l2.c | 4 ++
drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 9 +++
drivers/media/v4l2-core/v4l2-dev.c| 17 +
drivers/media/v4l2-core/v4l2-ioctl.c | 53 ++
include/media/v4l2-dev.h | 2 +
include/media/v4l2-ioctl.h| 34 +
include/uapi/linux/videodev2.h| 17 +
13 files changed, 222 insertions(+)
create mode 100644 Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
diff --git a/Documentation/userspace-api/media/v4l/buffer.rst
b/Documentation/userspace-api/media/v4l/buffer.rst
index 52bbee81c080..a3754ca6f0d6 100644
--- a/Documentation/userspace-api/media/v4l/buffer.rst
+++ b/Documentation/userspace-api/media/v4l/buffer.rst
@@ -438,6 +438,12 @@ enum v4l2_buf_type
* - ``V4L2_BUF_TYPE_META_OUTPUT``
- 14
- Buffer for metadata output, see :ref:`metadata`.
+* - ``V4L2_BUF_TYPE_AUDIO_CAPTURE``
+ - 15
+ - Buffer for audio capture, see :ref:`audio`.
+* - ``V4L2_BUF_TYPE_AUDIO_OUTPUT``
+ - 16
+ - Buffer for audio output, see :ref:`audio`.
.. _buffer-flags:
diff --git a/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
new file mode 100644
index ..54cc2abb6c04
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
@@ -0,0 +1,71 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+
+.. _audiomem2mem:
+
+
+Audio Memory-To-Memory Interface
+
+
+An audio memory-to-memory device can compress, decompress, transform, or
+otherwise convert audio data from one format into another format, in memory.
+Such memory-to-memory devices set the ``V4L2_CAP_AUDIO_M2M`` capability.
+Examples of memory-to-memory devices are audio codecs, audio preprocessing,
+audio postprocessing.
+
+A memory-to-memory audio node supports both output (sending audio frames from
+memory to the hardware) and capture (receiving the processed audio frames
+from the hardware into memory) stream I/O. An application will have to
+setup the stream I/O for both sides and finally call
+:ref:`VIDIOC_STREAMON ` for both capture and output to
+start the hardware.
+
+Memory-to-memory devices function as a shared resource: you can
+open the audio node multiple times, each application setting up their
+own properties that are local to the file handle, and each can use
+it independently from the others. The driver will arbitrate access to
+the hardware and reprogram it whenever another file handler gets access.
+
+Audio memory-to-memory devices are accessed through character device
+special files named ``/dev/v4l-audio``
+
+Querying Capabilities
+=
+
+Device nodes supporting the audio memory-to-memory interface set the
+``V4L2_CAP_AUDIO_M2M`` flag in the ``device_caps`` field of the
+:c:type:`v4l2_capability` structure returned by the :c:func:`VIDIOC_QUERYCAP`
+ioctl.
+
+Data Format Negotiation
+===
+
+The audio device uses the :ref:`format` ioctls to select the capture format.
+The audio buffer content format is bound to that selected format. In addition
+to the basic :ref:`format` ioctls, the :c:func:`VIDIOC_ENUM_FMT` ioctl must be
+supported as well.
+
+To use the :ref:`format` ioctls applications set the ``type`` field of the
+:c:type:`v4l2_format` structure to ``V4L2_BUF_TYPE_AUDIO_CAPTURE`` or to
+``V4L2_BUF_TYPE_AUDIO_OUTPUT``. Both drivers and applications must set the
+remainder of the :c:type:`v4l2_format` structure to 0.
+
+.. c:type:: v4l2_audio_format
+
+.. tabularcolumns:: |p{1.4cm}|p{2.4cm}|p{13.5cm}|
+
+.. flat-table:: struct v4l2_audio_format
+:header-rows: 0
+:stub-columns: 0
+:widths: 1 1 2
+
+* - __u32
+ - ``audioformat``
+ - The sample format, set by the application. see :ref:`pixfmt-audio`
+* - __u32
+ - ``channels``
+ - The channel number, set by the application. channel number range is
+[1, 32].
+* - __u32
+ - ``buffersize``
+ - Maximum buffer size in bytes required for data. The value is set by the
+driver.
diff --git a/Doc
[PATCH v15 07/16] media: uapi: Add V4L2_CAP_AUDIO_M2M capability flag
V4L2_CAP_AUDIO_M2M is similar to V4L2_CAP_VIDEO_M2M flag.
It is used for audio memory to memory case.
Signed-off-by: Shengjiu Wang
---
Documentation/userspace-api/media/v4l/vidioc-querycap.rst| 3 +++
Documentation/userspace-api/media/videodev2.h.rst.exceptions | 1 +
include/uapi/linux/videodev2.h | 1 +
3 files changed, 5 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/vidioc-querycap.rst
b/Documentation/userspace-api/media/v4l/vidioc-querycap.rst
index 6c57b8428356..1c0d97bf192a 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-querycap.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-querycap.rst
@@ -173,6 +173,9 @@ specification the ioctl returns an ``EINVAL`` error code.
interface. A video overlay device typically stores captured images
directly in the video memory of a graphics card, with hardware
clipping and scaling.
+* - ``V4L2_CAP_AUDIO_M2M``
+ - 0x0008
+ - The device supports the audio Memory-To-Memory interface.
* - ``V4L2_CAP_VBI_CAPTURE``
- 0x0010
- The device supports the :ref:`Raw VBI Capture `
diff --git a/Documentation/userspace-api/media/videodev2.h.rst.exceptions
b/Documentation/userspace-api/media/videodev2.h.rst.exceptions
index 3e58aac4ef0b..da6d0b8e4c2c 100644
--- a/Documentation/userspace-api/media/videodev2.h.rst.exceptions
+++ b/Documentation/userspace-api/media/videodev2.h.rst.exceptions
@@ -197,6 +197,7 @@ replace define V4L2_CAP_META_OUTPUT device-capabilities
replace define V4L2_CAP_DEVICE_CAPS device-capabilities
replace define V4L2_CAP_TOUCH device-capabilities
replace define V4L2_CAP_IO_MC device-capabilities
+replace define V4L2_CAP_AUDIO_M2M device-capabilities
# V4L2 pix flags
replace define V4L2_PIX_FMT_PRIV_MAGIC :c:type:`v4l2_pix_format`
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index b8573e9ccde6..5cc2a978fd9c 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -473,6 +473,7 @@ struct v4l2_capability {
#define V4L2_CAP_VIDEO_CAPTURE 0x0001 /* Is a video capture
device */
#define V4L2_CAP_VIDEO_OUTPUT 0x0002 /* Is a video output device
*/
#define V4L2_CAP_VIDEO_OVERLAY 0x0004 /* Can do video overlay */
+#define V4L2_CAP_AUDIO_M2M 0x0008 /* audio memory to memory */
#define V4L2_CAP_VBI_CAPTURE 0x0010 /* Is a raw VBI capture
device */
#define V4L2_CAP_VBI_OUTPUT0x0020 /* Is a raw VBI output
device */
#define V4L2_CAP_SLICED_VBI_CAPTURE0x0040 /* Is a sliced VBI capture
device */
--
2.34.1
[PATCH v15 06/16] ASoC: fsl_easrc: register m2m platform device
Register m2m platform device,that user can
use M2M feature.
Signed-off-by: Shengjiu Wang
Acked-by: Mark Brown
---
sound/soc/fsl/fsl_easrc.c | 19 +++
1 file changed, 19 insertions(+)
diff --git a/sound/soc/fsl/fsl_easrc.c b/sound/soc/fsl/fsl_easrc.c
index cf7ad30a323b..ccbf45c7abf4 100644
--- a/sound/soc/fsl/fsl_easrc.c
+++ b/sound/soc/fsl/fsl_easrc.c
@@ -2075,6 +2075,7 @@ MODULE_DEVICE_TABLE(of, fsl_easrc_dt_ids);
static int fsl_easrc_probe(struct platform_device *pdev)
{
struct fsl_easrc_priv *easrc_priv;
+ struct fsl_asrc_m2m_pdata m2m_pdata;
struct device *dev = >dev;
struct fsl_asrc *easrc;
struct resource *res;
@@ -2190,6 +2191,19 @@ static int fsl_easrc_probe(struct platform_device *pdev)
goto err_pm_disable;
}
+ m2m_pdata.asrc = easrc;
+ m2m_pdata.fmt_in = FSL_EASRC_FORMATS;
+ m2m_pdata.fmt_out = FSL_EASRC_FORMATS |
SNDRV_PCM_FMTBIT_IEC958_SUBFRAME_LE;
+ m2m_pdata.rate_min = 8000;
+ m2m_pdata.rate_max = 768000;
+ m2m_pdata.chan_min = 1;
+ m2m_pdata.chan_max = 32;
+ easrc->m2m_pdev = platform_device_register_data(>dev,
+ M2M_DRV_NAME,
+ PLATFORM_DEVID_AUTO,
+ _pdata,
+ sizeof(m2m_pdata));
+
return 0;
err_pm_disable:
@@ -2199,6 +2213,11 @@ static int fsl_easrc_probe(struct platform_device *pdev)
static void fsl_easrc_remove(struct platform_device *pdev)
{
+ struct fsl_asrc *easrc = dev_get_drvdata(>dev);
+
+ if (easrc->m2m_pdev && !IS_ERR(easrc->m2m_pdev))
+ platform_device_unregister(easrc->m2m_pdev);
+
pm_runtime_disable(>dev);
}
--
2.34.1
[PATCH v15 05/16] ASoC: fsl_asrc: register m2m platform device
Register m2m platform device, that user can
use M2M feature.
Defined platform data structure and platform
driver name.
Signed-off-by: Shengjiu Wang
Acked-by: Mark Brown
---
include/sound/fsl_asrc_common.h | 23 +++
sound/soc/fsl/fsl_asrc.c| 18 ++
2 files changed, 41 insertions(+)
diff --git a/include/sound/fsl_asrc_common.h b/include/sound/fsl_asrc_common.h
index 3b53d366182f..c709b8906929 100644
--- a/include/sound/fsl_asrc_common.h
+++ b/include/sound/fsl_asrc_common.h
@@ -71,6 +71,7 @@ struct fsl_asrc_pair {
* @dma_params_rx: DMA parameters for receive channel
* @dma_params_tx: DMA parameters for transmit channel
* @pdev: platform device pointer
+ * @m2m_pdev: m2m platform device pointer
* @regmap: regmap handler
* @paddr: physical address to the base address of registers
* @mem_clk: clock source to access register
@@ -103,6 +104,7 @@ struct fsl_asrc {
struct snd_dmaengine_dai_dma_data dma_params_rx;
struct snd_dmaengine_dai_dma_data dma_params_tx;
struct platform_device *pdev;
+ struct platform_device *m2m_pdev;
struct regmap *regmap;
unsigned long paddr;
struct clk *mem_clk;
@@ -139,6 +141,27 @@ struct fsl_asrc {
void *private;
};
+/**
+ * struct fsl_asrc_m2m_pdata - platform data
+ * @asrc: pointer to struct fsl_asrc
+ * @fmt_in: input sample format
+ * @fmt_out: output sample format
+ * @chan_min: minimum channel number
+ * @chan_max: maximum channel number
+ * @rate_min: minimum rate
+ * @rate_max: maximum rete
+ */
+struct fsl_asrc_m2m_pdata {
+ struct fsl_asrc *asrc;
+ u64 fmt_in;
+ u64 fmt_out;
+ int chan_min;
+ int chan_max;
+ int rate_min;
+ int rate_max;
+};
+
+#define M2M_DRV_NAME "fsl_asrc_m2m"
#define DRV_NAME "fsl-asrc-dai"
extern struct snd_soc_component_driver fsl_asrc_component;
diff --git a/sound/soc/fsl/fsl_asrc.c b/sound/soc/fsl/fsl_asrc.c
index 7d8643ee0ba0..5ecb5d869607 100644
--- a/sound/soc/fsl/fsl_asrc.c
+++ b/sound/soc/fsl/fsl_asrc.c
@@ -1187,6 +1187,7 @@ static int fsl_asrc_runtime_suspend(struct device *dev);
static int fsl_asrc_probe(struct platform_device *pdev)
{
struct device_node *np = pdev->dev.of_node;
+ struct fsl_asrc_m2m_pdata m2m_pdata;
struct fsl_asrc_priv *asrc_priv;
struct fsl_asrc *asrc;
struct resource *res;
@@ -1368,6 +1369,18 @@ static int fsl_asrc_probe(struct platform_device *pdev)
goto err_pm_get_sync;
}
+ m2m_pdata.asrc = asrc;
+ m2m_pdata.fmt_in = FSL_ASRC_FORMATS;
+ m2m_pdata.fmt_out = FSL_ASRC_FORMATS | SNDRV_PCM_FMTBIT_S8;
+ m2m_pdata.rate_min = 5512;
+ m2m_pdata.rate_max = 192000;
+ m2m_pdata.chan_min = 1;
+ m2m_pdata.chan_max = 10;
+ asrc->m2m_pdev = platform_device_register_data(>dev,
+ M2M_DRV_NAME,
+ PLATFORM_DEVID_AUTO,
+ _pdata,
+ sizeof(m2m_pdata));
return 0;
err_pm_get_sync:
@@ -1380,6 +1393,11 @@ static int fsl_asrc_probe(struct platform_device *pdev)
static void fsl_asrc_remove(struct platform_device *pdev)
{
+ struct fsl_asrc *asrc = dev_get_drvdata(>dev);
+
+ if (asrc->m2m_pdev && !IS_ERR(asrc->m2m_pdev))
+ platform_device_unregister(asrc->m2m_pdev);
+
pm_runtime_disable(>dev);
if (!pm_runtime_status_suspended(>dev))
fsl_asrc_runtime_suspend(>dev);
--
2.34.1
[PATCH v15 04/16] ASoC: fsl_asrc: move fsl_asrc_common.h to include/sound
Move fsl_asrc_common.h to include/sound that it can be
included from other drivers.
Signed-off-by: Shengjiu Wang
Acked-by: Mark Brown
---
{sound/soc/fsl => include/sound}/fsl_asrc_common.h | 0
sound/soc/fsl/fsl_asrc.h | 2 +-
sound/soc/fsl/fsl_asrc_dma.c | 2 +-
sound/soc/fsl/fsl_easrc.h | 2 +-
4 files changed, 3 insertions(+), 3 deletions(-)
rename {sound/soc/fsl => include/sound}/fsl_asrc_common.h (100%)
diff --git a/sound/soc/fsl/fsl_asrc_common.h b/include/sound/fsl_asrc_common.h
similarity index 100%
rename from sound/soc/fsl/fsl_asrc_common.h
rename to include/sound/fsl_asrc_common.h
diff --git a/sound/soc/fsl/fsl_asrc.h b/sound/soc/fsl/fsl_asrc.h
index 1c492eb237f5..66544624de7b 100644
--- a/sound/soc/fsl/fsl_asrc.h
+++ b/sound/soc/fsl/fsl_asrc.h
@@ -10,7 +10,7 @@
#ifndef _FSL_ASRC_H
#define _FSL_ASRC_H
-#include "fsl_asrc_common.h"
+#include
#define ASRC_M2M_INPUTFIFO_WML 0x4
#define ASRC_M2M_OUTPUTFIFO_WML0x2
diff --git a/sound/soc/fsl/fsl_asrc_dma.c b/sound/soc/fsl/fsl_asrc_dma.c
index f501f47242fb..f067bf1ecea7 100644
--- a/sound/soc/fsl/fsl_asrc_dma.c
+++ b/sound/soc/fsl/fsl_asrc_dma.c
@@ -12,7 +12,7 @@
#include
#include
-#include "fsl_asrc_common.h"
+#include
#define FSL_ASRC_DMABUF_SIZE (256 * 1024)
diff --git a/sound/soc/fsl/fsl_easrc.h b/sound/soc/fsl/fsl_easrc.h
index c9f770862662..a24e540876a4 100644
--- a/sound/soc/fsl/fsl_easrc.h
+++ b/sound/soc/fsl/fsl_easrc.h
@@ -9,7 +9,7 @@
#include
#include
-#include "fsl_asrc_common.h"
+#include
/* EASRC Register Map */
--
2.34.1
[PATCH v15 03/16] ASoC: fsl_easrc: define functions for memory to memory usage
ASRC can be used on memory to memory case, define several
functions for m2m usage and export them as function pointer.
Signed-off-by: Shengjiu Wang
Acked-by: Mark Brown
---
sound/soc/fsl/fsl_easrc.c | 214 ++
sound/soc/fsl/fsl_easrc.h | 4 +
2 files changed, 218 insertions(+)
diff --git a/sound/soc/fsl/fsl_easrc.c b/sound/soc/fsl/fsl_easrc.c
index ec53bda46a46..cf7ad30a323b 100644
--- a/sound/soc/fsl/fsl_easrc.c
+++ b/sound/soc/fsl/fsl_easrc.c
@@ -1861,6 +1861,211 @@ static int fsl_easrc_get_fifo_addr(u8 dir, enum
asrc_pair_index index)
return REG_EASRC_FIFO(dir, index);
}
+/* Get sample numbers in FIFO */
+static unsigned int fsl_easrc_get_output_fifo_size(struct fsl_asrc_pair *pair)
+{
+ struct fsl_asrc *asrc = pair->asrc;
+ enum asrc_pair_index index = pair->index;
+ u32 val;
+
+ regmap_read(asrc->regmap, REG_EASRC_SFS(index), );
+ val &= EASRC_SFS_NSGO_MASK;
+
+ return val >> EASRC_SFS_NSGO_SHIFT;
+}
+
+static int fsl_easrc_m2m_prepare(struct fsl_asrc_pair *pair)
+{
+ struct fsl_easrc_ctx_priv *ctx_priv = pair->private;
+ struct fsl_asrc *asrc = pair->asrc;
+ struct device *dev = >pdev->dev;
+ int ret;
+
+ ctx_priv->in_params.sample_rate = pair->rate[IN];
+ ctx_priv->in_params.sample_format = pair->sample_format[IN];
+ ctx_priv->out_params.sample_rate = pair->rate[OUT];
+ ctx_priv->out_params.sample_format = pair->sample_format[OUT];
+
+ ctx_priv->in_params.fifo_wtmk = FSL_EASRC_INPUTFIFO_WML;
+ ctx_priv->out_params.fifo_wtmk = FSL_EASRC_OUTPUTFIFO_WML;
+ /* Fill the right half of the re-sampler with zeros */
+ ctx_priv->rs_init_mode = 0x2;
+ /* Zero fill the right half of the prefilter */
+ ctx_priv->pf_init_mode = 0x2;
+
+ ret = fsl_easrc_set_ctx_format(pair,
+ _priv->in_params.sample_format,
+ _priv->out_params.sample_format);
+ if (ret) {
+ dev_err(dev, "failed to set context format: %d\n", ret);
+ return ret;
+ }
+
+ ret = fsl_easrc_config_context(asrc, pair->index);
+ if (ret) {
+ dev_err(dev, "failed to config context %d\n", ret);
+ return ret;
+ }
+
+ ctx_priv->in_params.iterations = 1;
+ ctx_priv->in_params.group_len = pair->channels;
+ ctx_priv->in_params.access_len = pair->channels;
+ ctx_priv->out_params.iterations = 1;
+ ctx_priv->out_params.group_len = pair->channels;
+ ctx_priv->out_params.access_len = pair->channels;
+
+ ret = fsl_easrc_set_ctx_organziation(pair);
+ if (ret) {
+ dev_err(dev, "failed to set fifo organization\n");
+ return ret;
+ }
+
+ /* The context start flag */
+ pair->first_convert = 1;
+ return 0;
+}
+
+static int fsl_easrc_m2m_start(struct fsl_asrc_pair *pair)
+{
+ /* start context once */
+ if (pair->first_convert) {
+ fsl_easrc_start_context(pair);
+ pair->first_convert = 0;
+ }
+
+ return 0;
+}
+
+static int fsl_easrc_m2m_stop(struct fsl_asrc_pair *pair)
+{
+ /* Stop pair/context */
+ if (!pair->first_convert) {
+ fsl_easrc_stop_context(pair);
+ pair->first_convert = 1;
+ }
+
+ return 0;
+}
+
+/* calculate capture data length according to output data length and sample
rate */
+static int fsl_easrc_m2m_calc_out_len(struct fsl_asrc_pair *pair, int
input_buffer_length)
+{
+ struct fsl_asrc *easrc = pair->asrc;
+ struct fsl_easrc_priv *easrc_priv = easrc->private;
+ struct fsl_easrc_ctx_priv *ctx_priv = pair->private;
+ unsigned int in_rate = ctx_priv->in_params.norm_rate;
+ unsigned int out_rate = ctx_priv->out_params.norm_rate;
+ unsigned int channels = pair->channels;
+ unsigned int in_samples, out_samples;
+ unsigned int in_width, out_width;
+ unsigned int out_length;
+ unsigned int frac_bits;
+ u64 val1, val2;
+
+ switch (easrc_priv->rs_num_taps) {
+ case EASRC_RS_32_TAPS:
+ /* integer bits = 5; */
+ frac_bits = 39;
+ break;
+ case EASRC_RS_64_TAPS:
+ /* integer bits = 6; */
+ frac_bits = 38;
+ break;
+ case EASRC_RS_128_TAPS:
+ /* integer bits = 7; */
+ frac_bits = 37;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ val1 = (u64)in_rate << frac_bits;
+ do_div(val1, out_rate);
+ val1 += (s64)ctx_priv->ratio_mod << (frac_bits - 31);
+
+ in_width =
snd_pcm_format_physical_width(c
[PATCH v15 02/16] ASoC: fsl_asrc: define functions for memory to memory usage
ASRC can be used on memory to memory case, define several
functions for m2m usage.
m2m_prepare: prepare for the start step
m2m_start: the start step
m2m_unprepare: unprepare for stop step, optional
m2m_stop: stop step
m2m_check_format: check format is supported or not
m2m_calc_out_len: calculate output length according to input length
m2m_get_maxburst: burst size for dma
m2m_pair_suspend: suspend function of pair, optional.
m2m_pair_resume: resume function of pair
get_output_fifo_size: get remaining data size in FIFO
Signed-off-by: Shengjiu Wang
Acked-by: Mark Brown
---
sound/soc/fsl/fsl_asrc.c| 126
sound/soc/fsl/fsl_asrc.h| 2 +
sound/soc/fsl/fsl_asrc_common.h | 37 ++
3 files changed, 165 insertions(+)
diff --git a/sound/soc/fsl/fsl_asrc.c b/sound/soc/fsl/fsl_asrc.c
index b793263291dc..7d8643ee0ba0 100644
--- a/sound/soc/fsl/fsl_asrc.c
+++ b/sound/soc/fsl/fsl_asrc.c
@@ -1063,6 +1063,124 @@ static int fsl_asrc_get_fifo_addr(u8 dir, enum
asrc_pair_index index)
return REG_ASRDx(dir, index);
}
+/* Get sample numbers in FIFO */
+static unsigned int fsl_asrc_get_output_fifo_size(struct fsl_asrc_pair *pair)
+{
+ struct fsl_asrc *asrc = pair->asrc;
+ enum asrc_pair_index index = pair->index;
+ u32 val;
+
+ regmap_read(asrc->regmap, REG_ASRFST(index), );
+
+ val &= ASRFSTi_OUTPUT_FIFO_MASK;
+
+ return val >> ASRFSTi_OUTPUT_FIFO_SHIFT;
+}
+
+static int fsl_asrc_m2m_prepare(struct fsl_asrc_pair *pair)
+{
+ struct fsl_asrc_pair_priv *pair_priv = pair->private;
+ struct fsl_asrc *asrc = pair->asrc;
+ struct device *dev = >pdev->dev;
+ struct asrc_config config;
+ int ret;
+
+ /* fill config */
+ config.pair = pair->index;
+ config.channel_num = pair->channels;
+ config.input_sample_rate = pair->rate[IN];
+ config.output_sample_rate = pair->rate[OUT];
+ config.input_format = pair->sample_format[IN];
+ config.output_format = pair->sample_format[OUT];
+ config.inclk = INCLK_NONE;
+ config.outclk = OUTCLK_ASRCK1_CLK;
+
+ pair_priv->config =
+ ret = fsl_asrc_config_pair(pair, true);
+ if (ret) {
+ dev_err(dev, "failed to config pair: %d\n", ret);
+ return ret;
+ }
+
+ pair->first_convert = 1;
+
+ return 0;
+}
+
+static int fsl_asrc_m2m_start(struct fsl_asrc_pair *pair)
+{
+ if (pair->first_convert) {
+ fsl_asrc_start_pair(pair);
+ pair->first_convert = 0;
+ }
+ /*
+* Clear DMA request during the stall state of ASRC:
+* During STALL state, the remaining in input fifo would never be
+* smaller than the input threshold while the output fifo would not
+* be bigger than output one. Thus the DMA request would be cleared.
+*/
+ fsl_asrc_set_watermarks(pair, ASRC_FIFO_THRESHOLD_MIN,
+ ASRC_FIFO_THRESHOLD_MAX);
+
+ /* Update the real input threshold to raise DMA request */
+ fsl_asrc_set_watermarks(pair, ASRC_M2M_INPUTFIFO_WML,
+ ASRC_M2M_OUTPUTFIFO_WML);
+
+ return 0;
+}
+
+static int fsl_asrc_m2m_stop(struct fsl_asrc_pair *pair)
+{
+ if (!pair->first_convert) {
+ fsl_asrc_stop_pair(pair);
+ pair->first_convert = 1;
+ }
+
+ return 0;
+}
+
+/* calculate capture data length according to output data length and sample
rate */
+static int fsl_asrc_m2m_calc_out_len(struct fsl_asrc_pair *pair, int
input_buffer_length)
+{
+ unsigned int in_width, out_width;
+ unsigned int channels = pair->channels;
+ unsigned int in_samples, out_samples;
+ unsigned int out_length;
+
+ in_width = snd_pcm_format_physical_width(pair->sample_format[IN]) / 8;
+ out_width = snd_pcm_format_physical_width(pair->sample_format[OUT]) / 8;
+
+ in_samples = input_buffer_length / in_width / channels;
+ out_samples = pair->rate[OUT] * in_samples / pair->rate[IN];
+ out_length = (out_samples - ASRC_OUTPUT_LAST_SAMPLE) * out_width *
channels;
+
+ return out_length;
+}
+
+static int fsl_asrc_m2m_get_maxburst(u8 dir, struct fsl_asrc_pair *pair)
+{
+ struct fsl_asrc *asrc = pair->asrc;
+ struct fsl_asrc_priv *asrc_priv = asrc->private;
+ int wml = (dir == IN) ? ASRC_M2M_INPUTFIFO_WML :
ASRC_M2M_OUTPUTFIFO_WML;
+
+ if (!asrc_priv->soc->use_edma)
+ return wml * pair->channels;
+ else
+ return 1;
+}
+
+static int fsl_asrc_m2m_pair_resume(struct fsl_asrc_pair *pair)
+{
+ struct fsl_asrc *asrc = pair->asrc;
+ int i;
+
+ for (i = 0; i < pair->channels * 4; i++)
+ regmap_write(asrc->regmap, REG_ASRDI(pair->index), 0);
+
+
[PATCH v15 01/16] media: v4l2-ctrls: add support for fraction_bits
From: Hans Verkuil
This adds support for the fraction_bits field, used with integer controls.
This allows fixed point formats to be described.
The fraction_bits field is only exposed through VIDIOC_QUERY_EXT_CTRL.
For a given signed two's complement Qf fixed point value 'f' equals
fraction_bits.
Signed-off-by: Hans Verkuil
---
.../media/v4l/vidioc-queryctrl.rst| 11 ++-
drivers/media/v4l2-core/v4l2-ctrls-api.c | 1 +
drivers/media/v4l2-core/v4l2-ctrls-core.c | 93 +++
include/media/v4l2-ctrls.h| 7 +-
include/uapi/linux/videodev2.h| 3 +-
5 files changed, 95 insertions(+), 20 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
index 4d38acafe8e1..e65c7e5d78ec 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
@@ -267,7 +267,16 @@ See also the examples in :ref:`control`.
- The size of each dimension. The first ``nr_of_dims`` elements of
this array must be non-zero, all remaining elements must be zero.
* - __u32
- - ``reserved``\ [32]
+ - ``fraction_bits``
+ - The number of least significant bits of the control value that
+form the fraction of the fixed point value. This is 0 if the value
+ is a regular integer. This can be used with all integer control types
+ (``INTEGER``, ``INTEGER64``, ``U8``, ``U16`` and ``U32``).
+ For the signed types the signed two's complement representation is used.
+ This field applies to the control value as well as the ``minimum``,
+ ``maximum``, ``step`` and ``default_value`` fields.
+* - __u32
+ - ``reserved``\ [31]
- Reserved for future extensions. Applications and drivers must set
the array to zero.
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-api.c
b/drivers/media/v4l2-core/v4l2-ctrls-api.c
index d9a422017bd9..ef16b00421ec 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-api.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-api.c
@@ -1101,6 +1101,7 @@ int v4l2_query_ext_ctrl(struct v4l2_ctrl_handler *hdl,
struct v4l2_query_ext_ctr
qc->elems = ctrl->elems;
qc->nr_of_dims = ctrl->nr_of_dims;
memcpy(qc->dims, ctrl->dims, qc->nr_of_dims * sizeof(qc->dims[0]));
+ qc->fraction_bits = ctrl->fraction_bits;
qc->minimum = ctrl->minimum;
qc->maximum = ctrl->maximum;
qc->default_value = ctrl->default_value;
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c
b/drivers/media/v4l2-core/v4l2-ctrls-core.c
index c4d995f32191..d83a37198bb5 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
@@ -252,12 +252,61 @@ void v4l2_ctrl_type_op_init(const struct v4l2_ctrl *ctrl,
u32 from_idx,
}
EXPORT_SYMBOL(v4l2_ctrl_type_op_init);
+static void v4l2_ctrl_log_fp(s64 v, unsigned int fraction_bits)
+{
+ s64 i, f, mask;
+
+ if (!fraction_bits) {
+ pr_cont("%lld", v);
+ return;
+ }
+
+ mask = (1ULL << fraction_bits) - 1;
+
+ /*
+* Note: this function does not support fixed point u64 with
+* fraction_bits set to 64. At the moment there is no U64
+* control type, but if that is added, then this code will have
+* to add support for it.
+*/
+ if (fraction_bits >= 63)
+ i = v < 0 ? -1 : 0;
+ else
+ i = div64_s64(v, 1LL << fraction_bits);
+
+ f = v < 0 ? -((-v) & mask) : (v & mask);
+
+ if (!f) {
+ pr_cont("%lld", i);
+ } else if (fraction_bits < 20) {
+ u64 div = 1ULL << fraction_bits;
+
+ if (!i && f < 0)
+ pr_cont("-%lld/%llu", -f, div);
+ else if (!i)
+ pr_cont("%lld/%llu", f, div);
+ else if (i < 0 || f < 0)
+ pr_cont("-%lld-%llu/%llu", -i, -f, div);
+ else
+ pr_cont("%lld+%llu/%llu", i, f, div);
+ } else {
+ if (!i && f < 0)
+ pr_cont("-%lld/(2^%u)", -f, fraction_bits);
+ else if (!i)
+ pr_cont("%lld/(2^%u)", f, fraction_bits);
+ else if (i < 0 || f < 0)
+ pr_cont("-%lld-%llu/(2^%u)", -i, -f, fraction_bits);
+ else
+ pr_cont("%lld+%llu/(2^%u)", i, f, fraction_bits);
+ }
+}
+
void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl)
{
union v4l2_ctrl_ptr ptr = ctrl->p_cur;
if (ctrl->is_array) {
- unsigned i;
+ unsigned int i;
for (i = 0; i < ctrl->nr_of_dims; i++)
pr_cont("[%u]", ctrl->dims[i]);
@@ -266,7 +315,7 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl)
[PATCH v15 00/16] Add audio support in v4l2 framework
Audio signal processing also has the requirement for memory to memory similar as Video. This asrc memory to memory (memory ->asrc->memory) case is a non real time use case. User fills the input buffer to the asrc module, after conversion, then asrc sends back the output buffer to user. So it is not a traditional ALSA playback and capture case. It is a specific use case, there is no reference in current kernel. v4l2 memory to memory is the closed implementation, v4l2 current support video, image, radio, tuner, touch devices, so it is not complicated to add support for this specific audio case. Because we had implemented the "memory -> asrc ->i2s device-> codec" use case in ALSA. Now the "memory->asrc->memory" needs to reuse the code in asrc driver, so the first 3 patches is for refining the code to make it can be shared by the "memory->asrc->memory" driver. The main change is in the v4l2 side, A /dev/vl4-audioX will be created, user applications only use the ioctl of v4l2 framework. Other change is to add memory to memory support for two kinds of i.MX ASRC module. changes in v15: - update MAINTAINERS for imx-asrc.c and vim2m-audio.c changes in v14: - document the reservation of 'AUXX' fourcc format. - add v4l2_audfmt_to_fourcc() definition. changes in v13 - change 'pixelformat' to 'audioformat' in dev-audio-mem2mem.rst - add more description for clock drift in ext-ctrls-audio-m2m.rst - Add "media: v4l2-ctrls: add support for fraction_bits" from Hans to avoid build issue for kernel test robot changes in v12 - minor changes according to comments - drop min_buffers_needed = 1 and V4L2_CTRL_FLAG_UPDATE flag - drop bus_info changes in v11 - add add-fixed-point-test-controls in vivid. - add v4l2_ctrl_fp_compose() helper function for min and max changes in v10 - remove FIXED_POINT type - change code base on media: v4l2-ctrls: add support for fraction_bits - fix issue reported by kernel test robot - remove module_alias changes in v9: - add MEDIA_ENT_F_PROC_AUDIO_RESAMPLER. - add MEDIA_INTF_T_V4L_AUDIO - add media controller support - refine the vim2m-audio to support 8k<->16k conversion. changes in v8: - refine V4L2_CAP_AUDIO_M2M to be 0x0008 - update doc for FIXED_POINT - address comments for imx-asrc changes in v7: - add acked-by from Mark - separate commit for fixed point, m2m audio class, audio rate controls - use INTEGER_MENU for rate, FIXED_POINT for rate offset - remove used fmts - address other comments for Hans changes in v6: - use m2m_prepare/m2m_unprepare/m2m_start/m2m_stop to replace m2m_start_part_one/m2m_stop_part_one, m2m_start_part_two/m2m_stop_part_two. - change V4L2_CTRL_TYPE_ASRC_RATE to V4L2_CTRL_TYPE_FIXED_POINT - fix warning by kernel test rebot - remove some unused format V4L2_AUDIO_FMT_XX - Get SNDRV_PCM_FORMAT from V4L2_AUDIO_FMT in driver. - rename audm2m to viaudm2m. changes in v5: - remove V4L2_AUDIO_FMT_LPCM - define audio pixel format like V4L2_AUDIO_FMT_S8... - remove rate and format in struct v4l2_audio_format. - Add V4L2_CID_ASRC_SOURCE_RATE and V4L2_CID_ASRC_DEST_RATE controls - updata document accordingly. changes in v4: - update document style - separate V4L2_AUDIO_FMT_LPCM and V4L2_CAP_AUDIO_M2M in separate commit changes in v3: - Modify documents for adding audio m2m support - Add audio virtual m2m driver - Defined V4L2_AUDIO_FMT_LPCM format type for audio. - Defined V4L2_CAP_AUDIO_M2M capability type for audio m2m case. - with modification in v4l-utils, pass v4l2-compliance test. changes in v2: - decouple the implementation in v4l2 and ALSA - implement the memory to memory driver as a platfrom driver and move it to driver/media - move fsl_asrc_common.h to include/sound folder Hans Verkuil (1): media: v4l2-ctrls: add support for fraction_bits Shengjiu Wang (15): ASoC: fsl_asrc: define functions for memory to memory usage ASoC: fsl_easrc: define functions for memory to memory usage ASoC: fsl_asrc: move fsl_asrc_common.h to include/sound ASoC: fsl_asrc: register m2m platform device ASoC: fsl_easrc: register m2m platform device media: uapi: Add V4L2_CAP_AUDIO_M2M capability flag media: v4l2: Add audio capture and output support media: uapi: Define audio sample format fourcc type media: uapi: Add V4L2_CTRL_CLASS_M2M_AUDIO media: uapi: Add audio rate controls support media: uapi: Declare interface types for Audio media: uapi: Add an entity type for audio resampler media: vivid: add fixed point test controls media: imx-asrc: Add memory to memory driver media: vim2m-audio: add virtual driver for audio memory to memory .../media/mediactl/media-types.rst| 11 + .../userspace-api/media/v4l/buffer.rst|6 + .../userspace-api/media/v4l/common.rst|1 + .../media/v4l/dev-audio-mem2mem.rst | 71 + .../userspace-api/media/v4l/devices.rst |1 + .../media/v4l/ext-ctrls-audio-m2m.r
Re: [PATCH v14 00/16] Add audio support in v4l2 framework
On Tue, Mar 19, 2024 at 10:24 AM Shengjiu Wang wrote: > > On Mon, Mar 18, 2024 at 11:56 PM Hans Verkuil wrote: > > > > Hi Shengjiu, > > > > I was about to prepare a pull request for this when I realized that > > entries for the new drivers need to be added to the MAINTAINERS file. > > > > Can you post patches 'v14 17/16' and 'v14/18/16' adding entries for each > > new driver? > > Ok, How about the change below? > > VIRTUAL MEM2MEM DRIVER FOR AUDIO > M: Hans Verkuil > M: Shengjiu Wang > L: linux-me...@vger.kernel.org > S: Maintained > W: https://linuxtv.org > T: git git://linuxtv.org/media_tree.git > F: drivers/media/test-drivers/vim2m-audio.c > > NXP ASRC V4L2 MEM2MEM DRIVERS > M: Shengjiu Wang > L: linux-me...@vger.kernel.org > S: Maintained > W: https://linuxtv.org > T: git git://linuxtv.org/media_tree.git > F: drivers/media/platform/nxp/imx-asrc.c > BTW, should I merge this change to PATCH 15/16 and PATCH 16/16? That I will resend the whole patch series. Seems this is a better way. Best regrads Shengjiu Wang > best regards > wang shengjiu > > > > > Thank you, > > > > Hans > > > > On 11/03/2024 10:53 am, Shengjiu Wang wrote: > > > Audio signal processing also has the requirement for memory to > > > memory similar as Video. > > > > > > This asrc memory to memory (memory ->asrc->memory) case is a non > > > real time use case. > > > > > > User fills the input buffer to the asrc module, after conversion, then > > > asrc > > > sends back the output buffer to user. So it is not a traditional ALSA > > > playback > > > and capture case. > > > > > > It is a specific use case, there is no reference in current kernel. > > > v4l2 memory to memory is the closed implementation, v4l2 current > > > support video, image, radio, tuner, touch devices, so it is not > > > complicated to add support for this specific audio case. > > > > > > Because we had implemented the "memory -> asrc ->i2s device-> codec" > > > use case in ALSA. Now the "memory->asrc->memory" needs > > > to reuse the code in asrc driver, so the first 3 patches is for refining > > > the code to make it can be shared by the "memory->asrc->memory" > > > driver. > > > > > > The main change is in the v4l2 side, A /dev/vl4-audioX will be created, > > > user applications only use the ioctl of v4l2 framework. > > > > > > Other change is to add memory to memory support for two kinds of i.MX ASRC > > > module. > > > > > > changes in v14: > > > - document the reservation of 'AUXX' fourcc format. > > > - add v4l2_audfmt_to_fourcc() definition. > > > > > > changes in v13 > > > - change 'pixelformat' to 'audioformat' in dev-audio-mem2mem.rst > > > - add more description for clock drift in ext-ctrls-audio-m2m.rst > > > - Add "media: v4l2-ctrls: add support for fraction_bits" from Hans > > > to avoid build issue for kernel test robot > > > > > > changes in v12 > > > - minor changes according to comments > > > - drop min_buffers_needed = 1 and V4L2_CTRL_FLAG_UPDATE flag > > > - drop bus_info > > > > > > changes in v11 > > > - add add-fixed-point-test-controls in vivid. > > > - add v4l2_ctrl_fp_compose() helper function for min and max > > > > > > changes in v10 > > > - remove FIXED_POINT type > > > - change code base on media: v4l2-ctrls: add support for fraction_bits > > > - fix issue reported by kernel test robot > > > - remove module_alias > > > > > > changes in v9: > > > - add MEDIA_ENT_F_PROC_AUDIO_RESAMPLER. > > > - add MEDIA_INTF_T_V4L_AUDIO > > > - add media controller support > > > - refine the vim2m-audio to support 8k<->16k conversion. > > > > > > changes in v8: > > > - refine V4L2_CAP_AUDIO_M2M to be 0x0008 > > > - update doc for FIXED_POINT > > > - address comments for imx-asrc > > > > > > changes in v7: > > > - add acked-by from Mark > > > - separate commit for fixed point, m2m audio class, audio rate controls > > > - use INTEGER_MENU for rate, FIXED_POINT for rate offset > > > - remove used fmts > > > - address other comments for Hans > > > > > > changes in v6: > > > - use m2m_prepa
Re: [PATCH v14 00/16] Add audio support in v4l2 framework
On Mon, Mar 18, 2024 at 11:56 PM Hans Verkuil wrote: > > Hi Shengjiu, > > I was about to prepare a pull request for this when I realized that > entries for the new drivers need to be added to the MAINTAINERS file. > > Can you post patches 'v14 17/16' and 'v14/18/16' adding entries for each > new driver? Ok, How about the change below? VIRTUAL MEM2MEM DRIVER FOR AUDIO M: Hans Verkuil M: Shengjiu Wang L: linux-me...@vger.kernel.org S: Maintained W: https://linuxtv.org T: git git://linuxtv.org/media_tree.git F: drivers/media/test-drivers/vim2m-audio.c NXP ASRC V4L2 MEM2MEM DRIVERS M: Shengjiu Wang L: linux-me...@vger.kernel.org S: Maintained W: https://linuxtv.org T: git git://linuxtv.org/media_tree.git F: drivers/media/platform/nxp/imx-asrc.c best regards wang shengjiu > > Thank you, > > Hans > > On 11/03/2024 10:53 am, Shengjiu Wang wrote: > > Audio signal processing also has the requirement for memory to > > memory similar as Video. > > > > This asrc memory to memory (memory ->asrc->memory) case is a non > > real time use case. > > > > User fills the input buffer to the asrc module, after conversion, then asrc > > sends back the output buffer to user. So it is not a traditional ALSA > > playback > > and capture case. > > > > It is a specific use case, there is no reference in current kernel. > > v4l2 memory to memory is the closed implementation, v4l2 current > > support video, image, radio, tuner, touch devices, so it is not > > complicated to add support for this specific audio case. > > > > Because we had implemented the "memory -> asrc ->i2s device-> codec" > > use case in ALSA. Now the "memory->asrc->memory" needs > > to reuse the code in asrc driver, so the first 3 patches is for refining > > the code to make it can be shared by the "memory->asrc->memory" > > driver. > > > > The main change is in the v4l2 side, A /dev/vl4-audioX will be created, > > user applications only use the ioctl of v4l2 framework. > > > > Other change is to add memory to memory support for two kinds of i.MX ASRC > > module. > > > > changes in v14: > > - document the reservation of 'AUXX' fourcc format. > > - add v4l2_audfmt_to_fourcc() definition. > > > > changes in v13 > > - change 'pixelformat' to 'audioformat' in dev-audio-mem2mem.rst > > - add more description for clock drift in ext-ctrls-audio-m2m.rst > > - Add "media: v4l2-ctrls: add support for fraction_bits" from Hans > > to avoid build issue for kernel test robot > > > > changes in v12 > > - minor changes according to comments > > - drop min_buffers_needed = 1 and V4L2_CTRL_FLAG_UPDATE flag > > - drop bus_info > > > > changes in v11 > > - add add-fixed-point-test-controls in vivid. > > - add v4l2_ctrl_fp_compose() helper function for min and max > > > > changes in v10 > > - remove FIXED_POINT type > > - change code base on media: v4l2-ctrls: add support for fraction_bits > > - fix issue reported by kernel test robot > > - remove module_alias > > > > changes in v9: > > - add MEDIA_ENT_F_PROC_AUDIO_RESAMPLER. > > - add MEDIA_INTF_T_V4L_AUDIO > > - add media controller support > > - refine the vim2m-audio to support 8k<->16k conversion. > > > > changes in v8: > > - refine V4L2_CAP_AUDIO_M2M to be 0x0008 > > - update doc for FIXED_POINT > > - address comments for imx-asrc > > > > changes in v7: > > - add acked-by from Mark > > - separate commit for fixed point, m2m audio class, audio rate controls > > - use INTEGER_MENU for rate, FIXED_POINT for rate offset > > - remove used fmts > > - address other comments for Hans > > > > changes in v6: > > - use m2m_prepare/m2m_unprepare/m2m_start/m2m_stop to replace > > m2m_start_part_one/m2m_stop_part_one, > > m2m_start_part_two/m2m_stop_part_two. > > - change V4L2_CTRL_TYPE_ASRC_RATE to V4L2_CTRL_TYPE_FIXED_POINT > > - fix warning by kernel test rebot > > - remove some unused format V4L2_AUDIO_FMT_XX > > - Get SNDRV_PCM_FORMAT from V4L2_AUDIO_FMT in driver. > > - rename audm2m to viaudm2m. > > > > changes in v5: > > - remove V4L2_AUDIO_FMT_LPCM > > - define audio pixel format like V4L2_AUDIO_FMT_S8... > > - remove rate and format in struct v4l2_audio_format. > > - Add V4L2_CID_ASRC_SOURCE_RATE and V4L2_CID_ASRC_DEST_RATE controls > > - updata document accordingly. > > > > changes in v4: > &
[PATCH v14 16/16] media: vim2m-audio: add virtual driver for audio memory to memory
Audio memory to memory virtual driver use video memory to memory
virtual driver vim2m.c as example. The main difference is
device type is VFL_TYPE_AUDIO and device cap type is V4L2_CAP_AUDIO_M2M.
The device_run function is a dummy function, which is simply
copy the data from input buffer to output buffer.
Signed-off-by: Shengjiu Wang
---
drivers/media/test-drivers/Kconfig | 10 +
drivers/media/test-drivers/Makefile | 1 +
drivers/media/test-drivers/vim2m-audio.c | 793 +++
3 files changed, 804 insertions(+)
create mode 100644 drivers/media/test-drivers/vim2m-audio.c
diff --git a/drivers/media/test-drivers/Kconfig
b/drivers/media/test-drivers/Kconfig
index 5a5379524bde..b6b52a7ca042 100644
--- a/drivers/media/test-drivers/Kconfig
+++ b/drivers/media/test-drivers/Kconfig
@@ -16,6 +16,16 @@ config VIDEO_VIM2M
This is a virtual test device for the memory-to-memory driver
framework.
+config VIDEO_VIM2M_AUDIO
+ tristate "Virtual Memory-to-Memory Driver For Audio"
+ depends on VIDEO_DEV
+ select VIDEOBUF2_VMALLOC
+ select V4L2_MEM2MEM_DEV
+ select MEDIA_CONTROLLER
+ help
+ This is a virtual audio test device for the memory-to-memory driver
+ framework.
+
source "drivers/media/test-drivers/vicodec/Kconfig"
source "drivers/media/test-drivers/vimc/Kconfig"
source "drivers/media/test-drivers/vivid/Kconfig"
diff --git a/drivers/media/test-drivers/Makefile
b/drivers/media/test-drivers/Makefile
index 740714a4584d..0c61c9ada3e1 100644
--- a/drivers/media/test-drivers/Makefile
+++ b/drivers/media/test-drivers/Makefile
@@ -10,6 +10,7 @@ obj-$(CONFIG_DVB_VIDTV) += vidtv/
obj-$(CONFIG_VIDEO_VICODEC) += vicodec/
obj-$(CONFIG_VIDEO_VIM2M) += vim2m.o
+obj-$(CONFIG_VIDEO_VIM2M_AUDIO) += vim2m-audio.o
obj-$(CONFIG_VIDEO_VIMC) += vimc/
obj-$(CONFIG_VIDEO_VIVID) += vivid/
obj-$(CONFIG_VIDEO_VISL) += visl/
diff --git a/drivers/media/test-drivers/vim2m-audio.c
b/drivers/media/test-drivers/vim2m-audio.c
new file mode 100644
index ..6361df6320b3
--- /dev/null
+++ b/drivers/media/test-drivers/vim2m-audio.c
@@ -0,0 +1,793 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * A virtual v4l2-mem2mem example for audio device.
+ */
+
+#include
+#include
+#include
+#include
+#include
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+MODULE_DESCRIPTION("Virtual device for audio mem2mem testing");
+MODULE_LICENSE("GPL");
+
+static unsigned int debug;
+module_param(debug, uint, 0644);
+MODULE_PARM_DESC(debug, "debug level");
+
+#define MEM2MEM_NAME "vim2m-audio"
+
+#define dprintk(dev, lvl, fmt, arg...) \
+ v4l2_dbg(lvl, debug, &(dev)->v4l2_dev, "%s: " fmt, __func__, ## arg)
+
+#define SAMPLE_NUM 4096
+
+static void audm2m_dev_release(struct device *dev)
+{}
+
+static struct platform_device audm2m_pdev = {
+ .name = MEM2MEM_NAME,
+ .dev.release= audm2m_dev_release,
+};
+
+static u32 formats[] = {
+ V4L2_AUDIO_FMT_S16_LE,
+};
+
+#define NUM_FORMATS ARRAY_SIZE(formats)
+
+/* Per-queue, driver-specific private data */
+struct audm2m_q_data {
+ unsigned intrate;
+ unsigned intchannels;
+ unsigned intbuffersize;
+ unsigned intsequence;
+ u32 fourcc;
+};
+
+enum {
+ V4L2_M2M_SRC = 0,
+ V4L2_M2M_DST = 1,
+};
+
+static snd_pcm_format_t find_format(u32 fourcc)
+{
+ snd_pcm_format_t fmt;
+ unsigned int k;
+
+ for (k = 0; k < NUM_FORMATS; k++) {
+ if (formats[k] == fourcc)
+ break;
+ }
+
+ if (k == NUM_FORMATS)
+ return 0;
+
+ fmt = v4l2_fourcc_to_audfmt(formats[k]);
+
+ return fmt;
+}
+
+struct audm2m_dev {
+ struct v4l2_device v4l2_dev;
+ struct video_device vfd;
+
+ struct mutexdev_mutex;
+
+ struct v4l2_m2m_dev *m2m_dev;
+#ifdef CONFIG_MEDIA_CONTROLLER
+ struct media_device mdev;
+#endif
+};
+
+struct audm2m_ctx {
+ struct v4l2_fh fh;
+ struct v4l2_ctrl_handlerctrl_handler;
+ struct audm2m_dev *dev;
+
+ struct mutexvb_mutex;
+
+ /* Source and destination queue data */
+ struct audm2m_q_data q_data[2];
+};
+
+static inline struct audm2m_ctx *file2ctx(struct file *file)
+{
+ return container_of(file->private_data, struct audm2m_ctx, fh);
+}
+
+static struct audm2m_q_data *get_q_data(struct audm2m_ctx *ctx,
+ enum v4l2_buf_type type)
+{
+ if (type == V4L2_BUF_TYPE_AUDIO_OUTPUT)
+ return >q_data[V4L2_M2M_SRC];
+ return >q_data[V4L2_M2M_DST];
+}
+
+static const char *type_name(enum v4l2_buf_type type)
+{
+ if (type == V4L2_BUF
[PATCH v14 15/16] media: imx-asrc: Add memory to memory driver
Implement the ASRC memory to memory function using
the v4l2 framework, user can use this function with
v4l2 ioctl interface.
User send the output and capture buffer to driver and
driver store the converted data to the capture buffer.
This feature can be shared by ASRC and EASRC drivers
Signed-off-by: Shengjiu Wang
---
drivers/media/platform/nxp/Kconfig| 13 +
drivers/media/platform/nxp/Makefile |1 +
drivers/media/platform/nxp/imx-asrc.c | 1256 +
3 files changed, 1270 insertions(+)
create mode 100644 drivers/media/platform/nxp/imx-asrc.c
diff --git a/drivers/media/platform/nxp/Kconfig
b/drivers/media/platform/nxp/Kconfig
index 40e3436669e2..8d0ca335601f 100644
--- a/drivers/media/platform/nxp/Kconfig
+++ b/drivers/media/platform/nxp/Kconfig
@@ -67,3 +67,16 @@ config VIDEO_MX2_EMMAPRP
source "drivers/media/platform/nxp/dw100/Kconfig"
source "drivers/media/platform/nxp/imx-jpeg/Kconfig"
+
+config VIDEO_IMX_ASRC
+ tristate "NXP i.MX ASRC M2M support"
+ depends on V4L_MEM2MEM_DRIVERS
+ depends on MEDIA_SUPPORT
+ select VIDEOBUF2_DMA_CONTIG
+ select V4L2_MEM2MEM_DEV
+ select MEDIA_CONTROLLER
+ help
+ Say Y if you want to add ASRC M2M support for NXP CPUs.
+ It is a complement for ASRC M2P and ASRC P2M features.
+ This option is only useful for out-of-tree drivers since
+ in-tree drivers select it automatically.
diff --git a/drivers/media/platform/nxp/Makefile
b/drivers/media/platform/nxp/Makefile
index 4d90eb713652..1325675e34f5 100644
--- a/drivers/media/platform/nxp/Makefile
+++ b/drivers/media/platform/nxp/Makefile
@@ -9,3 +9,4 @@ obj-$(CONFIG_VIDEO_IMX8MQ_MIPI_CSI2) += imx8mq-mipi-csi2.o
obj-$(CONFIG_VIDEO_IMX_MIPI_CSIS) += imx-mipi-csis.o
obj-$(CONFIG_VIDEO_IMX_PXP) += imx-pxp.o
obj-$(CONFIG_VIDEO_MX2_EMMAPRP) += mx2_emmaprp.o
+obj-$(CONFIG_VIDEO_IMX_ASRC) += imx-asrc.o
diff --git a/drivers/media/platform/nxp/imx-asrc.c
b/drivers/media/platform/nxp/imx-asrc.c
new file mode 100644
index ..0c25a36199b1
--- /dev/null
+++ b/drivers/media/platform/nxp/imx-asrc.c
@@ -0,0 +1,1256 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (C) 2014-2016 Freescale Semiconductor, Inc.
+// Copyright (C) 2019-2023 NXP
+//
+// Freescale ASRC Memory to Memory (M2M) driver
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#define V4L_CAP OUT
+#define V4L_OUT IN
+
+#define ASRC_xPUT_DMA_CALLBACK(dir) \
+ (((dir) == V4L_OUT) ? asrc_input_dma_callback \
+ : asrc_output_dma_callback)
+
+#define DIR_STR(dir) (dir) == V4L_OUT ? "out" : "cap"
+
+/* Maximum output and capture buffer size */
+#define ASRC_M2M_BUFFER_SIZE (512 * 1024)
+
+/* Maximum output and capture period size */
+#define ASRC_M2M_PERIOD_SIZE (48 * 1024)
+
+struct asrc_pair_m2m {
+ struct fsl_asrc_pair *pair;
+ struct asrc_m2m *m2m;
+ struct v4l2_fh fh;
+ struct v4l2_ctrl_handler ctrl_handler;
+ int channels[2];
+ unsigned int sequence[2];
+ s64 src_rate_off_prev; /* Q31.32 */
+ s64 dst_rate_off_prev; /* Q31.32 */
+ s64 src_rate_off_cur; /* Q31.32 */
+ s64 dst_rate_off_cur; /* Q31.32 */
+};
+
+struct asrc_m2m {
+ struct fsl_asrc_m2m_pdata pdata;
+ struct v4l2_device v4l2_dev;
+ struct v4l2_m2m_dev *m2m_dev;
+ struct video_device *dec_vdev;
+ struct mutex mlock; /* v4l2 ioctls serialization */
+ struct platform_device *pdev;
+#ifdef CONFIG_MEDIA_CONTROLLER
+ struct media_device mdev;
+#endif
+};
+
+static u32 formats[] = {
+ V4L2_AUDIO_FMT_S8,
+ V4L2_AUDIO_FMT_S16_LE,
+ V4L2_AUDIO_FMT_U16_LE,
+ V4L2_AUDIO_FMT_S24_LE,
+ V4L2_AUDIO_FMT_S24_3LE,
+ V4L2_AUDIO_FMT_U24_LE,
+ V4L2_AUDIO_FMT_U24_3LE,
+ V4L2_AUDIO_FMT_S32_LE,
+ V4L2_AUDIO_FMT_U32_LE,
+ V4L2_AUDIO_FMT_S20_3LE,
+ V4L2_AUDIO_FMT_U20_3LE,
+ V4L2_AUDIO_FMT_FLOAT_LE,
+ V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE,
+};
+
+#define NUM_FORMATS ARRAY_SIZE(formats)
+
+static const s64 asrc_v1_m2m_rates[] = {
+ 5512, 8000, 11025, 12000, 16000,
+ 22050, 24000, 32000, 44100,
+ 48000, 64000, 88200, 96000,
+ 128000, 176400, 192000,
+};
+
+static const s64 asrc_v2_m2m_rates[] = {
+ 8000, 11025, 12000, 16000,
+ 22050, 24000, 32000, 44100,
+ 48000, 64000, 88200, 96000,
+ 128000, 176400, 192000, 256000,
+ 352800, 384000, 705600, 768000,
+};
+
+static u32 find_fourcc(snd_pcm_format_t format)
+{
+ snd_pcm_format_t fmt;
+ unsigned int k;
+
+ for (k = 0; k < NUM_FORMATS; k++) {
+ fmt = v4l2_fourcc_to_audfmt(formats[k]);
+ if (fmt == format)
+ return formats[k];
+ }
+
+ return 0;
+}
+
+static snd_pcm_forma
[PATCH v14 14/16] media: vivid: add fixed point test controls
Add fixed point test controls, one is for Q4.16 format
another one is for Q63 format.
Signed-off-by: Shengjiu Wang
---
drivers/media/test-drivers/vivid/vivid-core.h | 2 ++
.../media/test-drivers/vivid/vivid-ctrls.c| 26 +++
include/media/v4l2-ctrls.h| 6 +
3 files changed, 34 insertions(+)
diff --git a/drivers/media/test-drivers/vivid/vivid-core.h
b/drivers/media/test-drivers/vivid/vivid-core.h
index cfb8e66083f6..f65465191bc9 100644
--- a/drivers/media/test-drivers/vivid/vivid-core.h
+++ b/drivers/media/test-drivers/vivid/vivid-core.h
@@ -222,6 +222,8 @@ struct vivid_dev {
struct v4l2_ctrl*boolean;
struct v4l2_ctrl*int32;
struct v4l2_ctrl*int64;
+ struct v4l2_ctrl*int32_q16;
+ struct v4l2_ctrl*int64_q63;
struct v4l2_ctrl*menu;
struct v4l2_ctrl*string;
struct v4l2_ctrl*bitmask;
diff --git a/drivers/media/test-drivers/vivid/vivid-ctrls.c
b/drivers/media/test-drivers/vivid/vivid-ctrls.c
index f2b20e25a7a4..2444ea95b285 100644
--- a/drivers/media/test-drivers/vivid/vivid-ctrls.c
+++ b/drivers/media/test-drivers/vivid/vivid-ctrls.c
@@ -38,6 +38,8 @@
#define VIVID_CID_U8_PIXEL_ARRAY (VIVID_CID_CUSTOM_BASE + 14)
#define VIVID_CID_S32_ARRAY(VIVID_CID_CUSTOM_BASE + 15)
#define VIVID_CID_S64_ARRAY(VIVID_CID_CUSTOM_BASE + 16)
+#define VIVID_CID_INT_Q4_16(VIVID_CID_CUSTOM_BASE + 17)
+#define VIVID_CID_INT64_Q63(VIVID_CID_CUSTOM_BASE + 18)
#define VIVID_CID_VIVID_BASE (0x00f0 | 0xf000)
#define VIVID_CID_VIVID_CLASS (0x00f0 | 1)
@@ -182,6 +184,28 @@ static const struct v4l2_ctrl_config vivid_ctrl_int64 = {
.step = 1,
};
+static const struct v4l2_ctrl_config vivid_ctrl_int32_q16 = {
+ .ops = _user_gen_ctrl_ops,
+ .id = VIVID_CID_INT_Q4_16,
+ .name = "Integer 32 Bits Q4.16",
+ .type = V4L2_CTRL_TYPE_INTEGER,
+ .min = v4l2_ctrl_fp_compose(-16, 0, 16),
+ .max = v4l2_ctrl_fp_compose(15, 0x, 16),
+ .step = 1,
+ .fraction_bits = 16,
+};
+
+static const struct v4l2_ctrl_config vivid_ctrl_int64_q63 = {
+ .ops = _user_gen_ctrl_ops,
+ .id = VIVID_CID_INT64_Q63,
+ .name = "Integer 64 Bits Q63",
+ .type = V4L2_CTRL_TYPE_INTEGER64,
+ .min = v4l2_ctrl_fp_compose(-1, 0, 63),
+ .max = v4l2_ctrl_fp_compose(0, LLONG_MAX, 63),
+ .step = 1,
+ .fraction_bits = 63,
+};
+
static const struct v4l2_ctrl_config vivid_ctrl_u32_array = {
.ops = _user_gen_ctrl_ops,
.id = VIVID_CID_U32_ARRAY,
@@ -1670,6 +1694,8 @@ int vivid_create_controls(struct vivid_dev *dev, bool
show_ccs_cap,
dev->button = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_button,
NULL);
dev->int32 = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_int32,
NULL);
dev->int64 = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_int64,
NULL);
+ dev->int32_q16 = v4l2_ctrl_new_custom(hdl_user_gen,
_ctrl_int32_q16, NULL);
+ dev->int64_q63 = v4l2_ctrl_new_custom(hdl_user_gen,
_ctrl_int64_q63, NULL);
dev->boolean = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_boolean,
NULL);
dev->menu = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_menu, NULL);
dev->string = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_string,
NULL);
diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
index c35514c5bf88..197d8b67ac13 100644
--- a/include/media/v4l2-ctrls.h
+++ b/include/media/v4l2-ctrls.h
@@ -1593,4 +1593,10 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl);
*/
int v4l2_ctrl_type_op_validate(const struct v4l2_ctrl *ctrl, union
v4l2_ctrl_ptr ptr);
+/*
+ * Fixed point compose helper define. This helper maps to the value
+ * i + f / (1 << fraction_bits).
+ */
+#define v4l2_ctrl_fp_compose(i, f, fraction_bits) (((s64)(i) << fraction_bits)
+ (f))
+
#endif
--
2.34.1
[PATCH v14 13/16] media: uapi: Add an entity type for audio resampler
Add and document a media entity type for an audio resampler.
It is MEDIA_ENT_F_PROC_AUDIO_RESAMPLER.
Signed-off-by: Shengjiu Wang
---
Documentation/userspace-api/media/mediactl/media-types.rst | 6 ++
include/uapi/linux/media.h | 1 +
2 files changed, 7 insertions(+)
diff --git a/Documentation/userspace-api/media/mediactl/media-types.rst
b/Documentation/userspace-api/media/mediactl/media-types.rst
index adfb37430f8e..d353f17c3344 100644
--- a/Documentation/userspace-api/media/mediactl/media-types.rst
+++ b/Documentation/userspace-api/media/mediactl/media-types.rst
@@ -40,6 +40,7 @@ Types and flags used to represent the media graph elements
.. _MEDIA-ENT-F-PROC-VIDEO-ENCODER:
.. _MEDIA-ENT-F-PROC-VIDEO-DECODER:
.. _MEDIA-ENT-F-PROC-VIDEO-ISP:
+.. _MEDIA-ENT-F-PROC-AUDIO-RESAMPLER:
.. _MEDIA-ENT-F-VID-MUX:
.. _MEDIA-ENT-F-VID-IF-BRIDGE:
.. _MEDIA-ENT-F-DV-DECODER:
@@ -208,6 +209,11 @@ Types and flags used to represent the media graph elements
combination of custom V4L2 controls and IOCTLs, and parameters
supplied in a metadata buffer.
+* - ``MEDIA_ENT_F_PROC_AUDIO_RESAMPLER``
+ - An Audio Resampler device. An entity capable of
+ resampling an audio stream from one sample rate to another sample
+ rate. Must have one sink pad and at least one source pad.
+
* - ``MEDIA_ENT_F_VID_MUX``
- Video multiplexer. An entity capable of multiplexing must have at
least two sink pads and one source pad, and must pass the video
diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
index 9ff6dec7393a..a8266eaa8042 100644
--- a/include/uapi/linux/media.h
+++ b/include/uapi/linux/media.h
@@ -125,6 +125,7 @@ struct media_device_info {
#define MEDIA_ENT_F_PROC_VIDEO_ENCODER (MEDIA_ENT_F_BASE + 0x4007)
#define MEDIA_ENT_F_PROC_VIDEO_DECODER (MEDIA_ENT_F_BASE + 0x4008)
#define MEDIA_ENT_F_PROC_VIDEO_ISP (MEDIA_ENT_F_BASE + 0x4009)
+#define MEDIA_ENT_F_PROC_AUDIO_RESAMPLER (MEDIA_ENT_F_BASE + 0x400a)
/*
* Switch and bridge entity functions
--
2.34.1
[PATCH v14 12/16] media: uapi: Declare interface types for Audio
Declare the interface types that will be used by Audio.
The type is MEDIA_INTF_T_V4L_AUDIO.
Signed-off-by: Shengjiu Wang
---
.../userspace-api/media/mediactl/media-types.rst| 5 +
drivers/media/v4l2-core/v4l2-dev.c | 4
drivers/media/v4l2-core/v4l2-mem2mem.c | 13 +
include/uapi/linux/media.h | 1 +
4 files changed, 19 insertions(+), 4 deletions(-)
diff --git a/Documentation/userspace-api/media/mediactl/media-types.rst
b/Documentation/userspace-api/media/mediactl/media-types.rst
index 6332e8395263..adfb37430f8e 100644
--- a/Documentation/userspace-api/media/mediactl/media-types.rst
+++ b/Documentation/userspace-api/media/mediactl/media-types.rst
@@ -265,6 +265,7 @@ Types and flags used to represent the media graph elements
.. _MEDIA-INTF-T-V4L-SUBDEV:
.. _MEDIA-INTF-T-V4L-SWRADIO:
.. _MEDIA-INTF-T-V4L-TOUCH:
+.. _MEDIA-INTF-T-V4L-AUDIO:
.. _MEDIA-INTF-T-ALSA-PCM-CAPTURE:
.. _MEDIA-INTF-T-ALSA-PCM-PLAYBACK:
.. _MEDIA-INTF-T-ALSA-CONTROL:
@@ -322,6 +323,10 @@ Types and flags used to represent the media graph elements
- Device node interface for Touch device (V4L)
- typically, /dev/v4l-touch?
+* - ``MEDIA_INTF_T_V4L_AUDIO``
+ - Device node interface for Audio device (V4L)
+ - typically, /dev/v4l-audio?
+
* - ``MEDIA_INTF_T_ALSA_PCM_CAPTURE``
- Device node interface for ALSA PCM Capture
- typically, /dev/snd/pcmC?D?c
diff --git a/drivers/media/v4l2-core/v4l2-dev.c
b/drivers/media/v4l2-core/v4l2-dev.c
index bac008fcedc6..ca8462a61e1f 100644
--- a/drivers/media/v4l2-core/v4l2-dev.c
+++ b/drivers/media/v4l2-core/v4l2-dev.c
@@ -844,6 +844,10 @@ static int video_register_media_controller(struct
video_device *vdev)
intf_type = MEDIA_INTF_T_V4L_SUBDEV;
/* Entity will be created via v4l2_device_register_subdev() */
break;
+ case VFL_TYPE_AUDIO:
+ intf_type = MEDIA_INTF_T_V4L_AUDIO;
+ /* Entity will be created via v4l2_device_register_subdev() */
+ break;
default:
return 0;
}
diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c
b/drivers/media/v4l2-core/v4l2-mem2mem.c
index 75517134a5e9..cda5e255305f 100644
--- a/drivers/media/v4l2-core/v4l2-mem2mem.c
+++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
@@ -1143,10 +1143,15 @@ int v4l2_m2m_register_media_controller(struct
v4l2_m2m_dev *m2m_dev,
if (ret)
goto err_rm_links0;
- /* Create video interface */
- m2m_dev->intf_devnode = media_devnode_create(mdev,
- MEDIA_INTF_T_V4L_VIDEO, 0,
- VIDEO_MAJOR, vdev->minor);
+ if (vdev->vfl_type == VFL_TYPE_AUDIO)
+ m2m_dev->intf_devnode = media_devnode_create(mdev,
+ MEDIA_INTF_T_V4L_AUDIO, 0,
+ VIDEO_MAJOR, vdev->minor);
+ else
+ /* Create video interface */
+ m2m_dev->intf_devnode = media_devnode_create(mdev,
+ MEDIA_INTF_T_V4L_VIDEO, 0,
+ VIDEO_MAJOR, vdev->minor);
if (!m2m_dev->intf_devnode) {
ret = -ENOMEM;
goto err_rm_links1;
diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
index 1c80b1d6bbaf..9ff6dec7393a 100644
--- a/include/uapi/linux/media.h
+++ b/include/uapi/linux/media.h
@@ -260,6 +260,7 @@ struct media_links_enum {
#define MEDIA_INTF_T_V4L_SUBDEV(MEDIA_INTF_T_V4L_BASE
+ 3)
#define MEDIA_INTF_T_V4L_SWRADIO (MEDIA_INTF_T_V4L_BASE + 4)
#define MEDIA_INTF_T_V4L_TOUCH (MEDIA_INTF_T_V4L_BASE + 5)
+#define MEDIA_INTF_T_V4L_AUDIO (MEDIA_INTF_T_V4L_BASE + 6)
#define MEDIA_INTF_T_ALSA_BASE 0x0300
#define MEDIA_INTF_T_ALSA_PCM_CAPTURE (MEDIA_INTF_T_ALSA_BASE)
--
2.34.1
[PATCH v14 11/16] media: uapi: Add audio rate controls support
Add V4L2_CID_M2M_AUDIO_SOURCE_RATE and V4L2_CID_M2M_AUDIO_DEST_RATE
new IDs for rate control.
Add V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET and
V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET for clock drift.
Signed-off-by: Shengjiu Wang
---
.../media/v4l/ext-ctrls-audio-m2m.rst | 38 +++
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 6 +++
include/uapi/linux/v4l2-controls.h| 5 +++
3 files changed, 49 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
index 82d2ecedbfee..b137b7c442e6 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
@@ -19,3 +19,41 @@ Audio M2M Control IDs
The Audio M2M class descriptor. Calling
:ref:`VIDIOC_QUERYCTRL` for this control will
return a description of this control class.
+
+.. _v4l2-audio-asrc:
+
+``V4L2_CID_M2M_AUDIO_SOURCE_RATE (integer menu)``
+This control specifies the audio source sample rate, unit is Hz
+
+``V4L2_CID_M2M_AUDIO_DEST_RATE (integer menu)``
+This control specifies the audio destination sample rate, unit is Hz
+
+``V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET (fixed point)``
+This control specifies the offset from the audio source sample rate,
+unit is Hz.
+
+The offset compensates for any clock drift. The actual source audio
+sample rate is the ideal source audio sample rate from
+``V4L2_CID_M2M_AUDIO_SOURCE_RATE`` plus this fixed point offset.
+
+The audio source clock may have some drift. Reducing or increasing the
+audio sample rate dynamically to ensure that Sample Rate Converter is
+working on the real sample rate, this feature is for the Asynchronous
+Sample Rate Converter module.
+So, userspace would be expected to be monitoring such drift
+and increasing/decreasing the sample frequency as needed by this control.
+
+``V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET (fixed point)``
+This control specifies the offset from the audio destination sample rate,
+unit is Hz.
+
+The offset compensates for any clock drift. The actual destination audio
+sample rate is the ideal source audio sample rate from
+``V4L2_CID_M2M_AUDIO_DEST_RATE`` plus this fixed point offset.
+
+The audio destination clock may have some drift. Reducing or increasing
+the audio sample rate dynamically to ensure that sample rate converter
+is working on the real sample rate, this feature is for the Asynchronous
+Sample Rate Converter module.
+So, userspace would be expected to be monitoring such drift
+and increasing/decreasing the sample frequency as needed by this control.
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index 2a85ea3dc92f..91e1f5348c23 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -1245,6 +1245,8 @@ const char *v4l2_ctrl_get_name(u32 id)
/* Audio M2M controls */
case V4L2_CID_M2M_AUDIO_CLASS: return "Audio M2M Controls";
+ case V4L2_CID_M2M_AUDIO_SOURCE_RATE:return "Audio Source Sample
Rate";
+ case V4L2_CID_M2M_AUDIO_DEST_RATE: return "Audio Destination
Sample Rate";
default:
return NULL;
}
@@ -1606,6 +1608,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum
v4l2_ctrl_type *type,
case V4L2_CID_COLORIMETRY_HDR10_MASTERING_DISPLAY:
*type = V4L2_CTRL_TYPE_HDR10_MASTERING_DISPLAY;
break;
+ case V4L2_CID_M2M_AUDIO_SOURCE_RATE:
+ case V4L2_CID_M2M_AUDIO_DEST_RATE:
+ *type = V4L2_CTRL_TYPE_INTEGER_MENU;
+ break;
default:
*type = V4L2_CTRL_TYPE_INTEGER;
break;
diff --git a/include/uapi/linux/v4l2-controls.h
b/include/uapi/linux/v4l2-controls.h
index a8b4b830c757..30129ccdc282 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -3495,6 +3495,11 @@ struct v4l2_ctrl_av1_film_grain {
#define V4L2_CID_M2M_AUDIO_CLASS_BASE (V4L2_CTRL_CLASS_M2M_AUDIO | 0x900)
#define V4L2_CID_M2M_AUDIO_CLASS (V4L2_CTRL_CLASS_M2M_AUDIO | 1)
+#define V4L2_CID_M2M_AUDIO_SOURCE_RATE (V4L2_CID_M2M_AUDIO_CLASS_BASE + 0)
+#define V4L2_CID_M2M_AUDIO_DEST_RATE (V4L2_CID_M2M_AUDIO_CLASS_BASE + 1)
+#define V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET (V4L2_CID_M2M_AUDIO_CLASS_BASE
+ 2)
+#define V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET(V4L2_CID_M2M_AUDIO_CLASS_BASE
+ 3)
+
/* MPEG-compression definitions kept for backwards compatibility */
#ifndef __KERNEL__
#define V4L2_CTRL_CLASS_MPEGV4L2_CTRL_CLASS_CODEC
--
2.34.1
[PATCH v14 10/16] media: uapi: Add V4L2_CTRL_CLASS_M2M_AUDIO
The Audio M2M class includes controls for audio memory-to-memory
use cases. The controls can be used for audio codecs, audio
preprocessing, audio postprocessing.
Signed-off-by: Shengjiu Wang
---
.../userspace-api/media/v4l/common.rst| 1 +
.../media/v4l/ext-ctrls-audio-m2m.rst | 21 +++
.../media/v4l/vidioc-g-ext-ctrls.rst | 4
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 4
include/uapi/linux/v4l2-controls.h| 4
5 files changed, 34 insertions(+)
create mode 100644
Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
diff --git a/Documentation/userspace-api/media/v4l/common.rst
b/Documentation/userspace-api/media/v4l/common.rst
index ea0435182e44..d5366e96a596 100644
--- a/Documentation/userspace-api/media/v4l/common.rst
+++ b/Documentation/userspace-api/media/v4l/common.rst
@@ -52,6 +52,7 @@ applicable to all devices.
ext-ctrls-fm-rx
ext-ctrls-detect
ext-ctrls-colorimetry
+ext-ctrls-audio-m2m
fourcc
format
planar-apis
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
new file mode 100644
index ..82d2ecedbfee
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
@@ -0,0 +1,21 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+
+.. _audiom2m-controls:
+
+***
+Audio M2M Control Reference
+***
+
+The Audio M2M class includes controls for audio memory-to-memory
+use cases. The controls can be used for audio codecs, audio
+preprocessing, audio postprocessing.
+
+Audio M2M Control IDs
+---
+
+.. _audiom2m-control-id:
+
+``V4L2_CID_M2M_AUDIO_CLASS (class)``
+The Audio M2M class descriptor. Calling
+:ref:`VIDIOC_QUERYCTRL` for this control will
+return a description of this control class.
diff --git a/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
b/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
index 4d56c0528ad7..aeb1ad8e7d29 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
@@ -488,6 +488,10 @@ still cause this situation.
- 0xa5
- The class containing colorimetry controls. These controls are
described in :ref:`colorimetry-controls`.
+* - ``V4L2_CTRL_CLASS_M2M_AUDIO``
+ - 0xa6
+ - The class containing audio m2m controls. These controls are
+ described in :ref:`audiom2m-controls`.
Return Value
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index 8696eb1cdd61..2a85ea3dc92f 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -1242,6 +1242,9 @@ const char *v4l2_ctrl_get_name(u32 id)
case V4L2_CID_COLORIMETRY_CLASS:return "Colorimetry Controls";
case V4L2_CID_COLORIMETRY_HDR10_CLL_INFO: return "HDR10
Content Light Info";
case V4L2_CID_COLORIMETRY_HDR10_MASTERING_DISPLAY: return "HDR10
Mastering Display";
+
+ /* Audio M2M controls */
+ case V4L2_CID_M2M_AUDIO_CLASS: return "Audio M2M Controls";
default:
return NULL;
}
@@ -1451,6 +1454,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum
v4l2_ctrl_type *type,
case V4L2_CID_DETECT_CLASS:
case V4L2_CID_CODEC_STATELESS_CLASS:
case V4L2_CID_COLORIMETRY_CLASS:
+ case V4L2_CID_M2M_AUDIO_CLASS:
*type = V4L2_CTRL_TYPE_CTRL_CLASS;
/* You can neither read nor write these */
*flags |= V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_WRITE_ONLY;
diff --git a/include/uapi/linux/v4l2-controls.h
b/include/uapi/linux/v4l2-controls.h
index 99c3f5e99da7..a8b4b830c757 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -30,6 +30,7 @@
#define V4L2_CTRL_CLASS_DETECT 0x00a3 /* Detection controls */
#define V4L2_CTRL_CLASS_CODEC_STATELESS 0x00a4 /* Stateless codecs
controls */
#define V4L2_CTRL_CLASS_COLORIMETRY0x00a5 /* Colorimetry controls
*/
+#define V4L2_CTRL_CLASS_M2M_AUDIO 0x00a6 /* Audio M2M controls */
/* User-class control IDs */
@@ -3491,6 +3492,9 @@ struct v4l2_ctrl_av1_film_grain {
__u8 reserved[4];
};
+#define V4L2_CID_M2M_AUDIO_CLASS_BASE (V4L2_CTRL_CLASS_M2M_AUDIO | 0x900)
+#define V4L2_CID_M2M_AUDIO_CLASS (V4L2_CTRL_CLASS_M2M_AUDIO | 1)
+
/* MPEG-compression definitions kept for backwards compatibility */
#ifndef __KERNEL__
#define V4L2_CTRL_CLASS_MPEGV4L2_CTRL_CLASS_CODEC
--
2.34.1
[PATCH v14 09/16] media: uapi: Define audio sample format fourcc type
The audio sample format definition is from alsa,
the header file is include/uapi/sound/asound.h, but
don't include this header file directly, because in
user space, there is another copy in alsa-lib.
There will be conflict in userspace for include
videodev2.h & asound.h and asoundlib.h
Here still use the fourcc format.
Signed-off-by: Shengjiu Wang
---
.../userspace-api/media/v4l/pixfmt-audio.rst | 100 ++
.../userspace-api/media/v4l/pixfmt.rst| 1 +
drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++
include/uapi/linux/videodev2.h| 29 +
4 files changed, 143 insertions(+)
create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
new file mode 100644
index ..a66db9a19936
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
@@ -0,0 +1,100 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+
+.. _pixfmt-audio:
+
+*
+Audio Formats
+*
+
+These formats are used for :ref:`audiomem2mem` interface only.
+
+All FourCCs starting with 'AU' are reserved for mappings
+of the snd_pcm_format_t type.
+
+The v4l2_audfmt_to_fourcc() is defined to convert the snd_pcm_format_t
+type to a FourCC. The first character is 'A', the second character
+is 'U', and the remaining two characters is the snd_pcm_format_t
+value in ASCII. Example: SNDRV_PCM_FORMAT_S16_LE (with value 2)
+maps to 'AU02' and SNDRV_PCM_FORMAT_S24_3LE (with value 32) maps
+to 'AU32'."
+
+The v4l2_fourcc_to_audfmt() is defined to convert these FourCCs to
+snd_pcm_format_t type.
+
+.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}|
+
+.. cssclass:: longtable
+
+.. flat-table:: Audio Format
+:header-rows: 1
+:stub-columns: 0
+:widths: 3 1 4
+
+* - Identifier
+ - Code
+ - Details
+* .. _V4L2-AUDIO-FMT-S8:
+
+ - ``V4L2_AUDIO_FMT_S8``
+ - 'S8'
+ - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA
+* .. _V4L2-AUDIO-FMT-S16-LE:
+
+ - ``V4L2_AUDIO_FMT_S16_LE``
+ - 'S16_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA
+* .. _V4L2-AUDIO-FMT-U16-LE:
+
+ - ``V4L2_AUDIO_FMT_U16_LE``
+ - 'U16_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA
+* .. _V4L2-AUDIO-FMT-S24-LE:
+
+ - ``V4L2_AUDIO_FMT_S24_LE``
+ - 'S24_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA
+* .. _V4L2-AUDIO-FMT-U24-LE:
+
+ - ``V4L2_AUDIO_FMT_U24_LE``
+ - 'U24_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA
+* .. _V4L2-AUDIO-FMT-S32-LE:
+
+ - ``V4L2_AUDIO_FMT_S32_LE``
+ - 'S32_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA
+* .. _V4L2-AUDIO-FMT-U32-LE:
+
+ - ``V4L2_AUDIO_FMT_U32_LE``
+ - 'U32_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA
+* .. _V4L2-AUDIO-FMT-FLOAT-LE:
+
+ - ``V4L2_AUDIO_FMT_FLOAT_LE``
+ - 'FLOAT_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_FLOAT_LE in ALSA
+* .. _V4L2-AUDIO-FMT-IEC958-SUBFRAME-LE:
+
+ - ``V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE``
+ - 'IEC958_SUBFRAME_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_IEC958_SUBFRAME_LE in ALSA
+* .. _V4L2-AUDIO-FMT-S24-3LE:
+
+ - ``V4L2_AUDIO_FMT_S24_3LE``
+ - 'S24_3LE'
+ - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
+* .. _V4L2-AUDIO-FMT-U24-3LE:
+
+ - ``V4L2_AUDIO_FMT_U24_3LE``
+ - 'U24_3LE'
+ - Corresponds to SNDRV_PCM_FORMAT_U24_3LE in ALSA
+* .. _V4L2-AUDIO-FMT-S20-3LE:
+
+ - ``V4L2_AUDIO_FMT_S20_3LE``
+ - 'S20_3LE'
+ - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
+* .. _V4L2-AUDIO-FMT-U20-3LE:
+
+ - ``V4L2_AUDIO_FMT_U20_3LE``
+ - 'U20_3LE'
+ - Corresponds to SNDRV_PCM_FORMAT_U20_3LE in ALSA
diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst
b/Documentation/userspace-api/media/v4l/pixfmt.rst
index 11dab4a90630..2eb6fdd3b43d 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt.rst
@@ -36,3 +36,4 @@ see also :ref:`VIDIOC_G_FBUF `.)
colorspaces
colorspaces-defs
colorspaces-details
+pixfmt-audio
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
b/drivers/media/v4l2-core/v4l2-ioctl.c
index 961abcdf7290..be229c69e991 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1471,6 +1471,19 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_Y210: descr = "10-bit YUYV Packed"; break;
case V4L2_PIX_FMT_Y212: descr = "12-bit YUYV Packed"; break;
case V4L2_PIX_FMT_Y216: descr = "16-bit YUYV Packed"; break;
+ case V4L2_AUDIO_FMT_S8: descr = "8-bit Signed"; break;
+ case V4L2_AUDIO_FMT_S16_LE: descr = "16-bit
[PATCH v14 08/16] media: v4l2: Add audio capture and output support
Audio signal processing has the requirement for memory to
memory similar as Video.
This patch is to add this support in v4l2 framework, defined
new buffer type V4L2_BUF_TYPE_AUDIO_CAPTURE and
V4L2_BUF_TYPE_AUDIO_OUTPUT, defined new format v4l2_audio_format
for audio case usage.
The created audio device is named "/dev/v4l-audioX".
Signed-off-by: Shengjiu Wang
---
.../userspace-api/media/v4l/buffer.rst| 6 ++
.../media/v4l/dev-audio-mem2mem.rst | 71 +++
.../userspace-api/media/v4l/devices.rst | 1 +
.../media/v4l/vidioc-enum-fmt.rst | 2 +
.../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 ++
.../media/videodev2.h.rst.exceptions | 2 +
.../media/common/videobuf2/videobuf2-v4l2.c | 4 ++
drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 9 +++
drivers/media/v4l2-core/v4l2-dev.c| 17 +
drivers/media/v4l2-core/v4l2-ioctl.c | 53 ++
include/media/v4l2-dev.h | 2 +
include/media/v4l2-ioctl.h| 34 +
include/uapi/linux/videodev2.h| 17 +
13 files changed, 222 insertions(+)
create mode 100644 Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
diff --git a/Documentation/userspace-api/media/v4l/buffer.rst
b/Documentation/userspace-api/media/v4l/buffer.rst
index 52bbee81c080..a3754ca6f0d6 100644
--- a/Documentation/userspace-api/media/v4l/buffer.rst
+++ b/Documentation/userspace-api/media/v4l/buffer.rst
@@ -438,6 +438,12 @@ enum v4l2_buf_type
* - ``V4L2_BUF_TYPE_META_OUTPUT``
- 14
- Buffer for metadata output, see :ref:`metadata`.
+* - ``V4L2_BUF_TYPE_AUDIO_CAPTURE``
+ - 15
+ - Buffer for audio capture, see :ref:`audio`.
+* - ``V4L2_BUF_TYPE_AUDIO_OUTPUT``
+ - 16
+ - Buffer for audio output, see :ref:`audio`.
.. _buffer-flags:
diff --git a/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
new file mode 100644
index ..54cc2abb6c04
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
@@ -0,0 +1,71 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+
+.. _audiomem2mem:
+
+
+Audio Memory-To-Memory Interface
+
+
+An audio memory-to-memory device can compress, decompress, transform, or
+otherwise convert audio data from one format into another format, in memory.
+Such memory-to-memory devices set the ``V4L2_CAP_AUDIO_M2M`` capability.
+Examples of memory-to-memory devices are audio codecs, audio preprocessing,
+audio postprocessing.
+
+A memory-to-memory audio node supports both output (sending audio frames from
+memory to the hardware) and capture (receiving the processed audio frames
+from the hardware into memory) stream I/O. An application will have to
+setup the stream I/O for both sides and finally call
+:ref:`VIDIOC_STREAMON ` for both capture and output to
+start the hardware.
+
+Memory-to-memory devices function as a shared resource: you can
+open the audio node multiple times, each application setting up their
+own properties that are local to the file handle, and each can use
+it independently from the others. The driver will arbitrate access to
+the hardware and reprogram it whenever another file handler gets access.
+
+Audio memory-to-memory devices are accessed through character device
+special files named ``/dev/v4l-audio``
+
+Querying Capabilities
+=
+
+Device nodes supporting the audio memory-to-memory interface set the
+``V4L2_CAP_AUDIO_M2M`` flag in the ``device_caps`` field of the
+:c:type:`v4l2_capability` structure returned by the :c:func:`VIDIOC_QUERYCAP`
+ioctl.
+
+Data Format Negotiation
+===
+
+The audio device uses the :ref:`format` ioctls to select the capture format.
+The audio buffer content format is bound to that selected format. In addition
+to the basic :ref:`format` ioctls, the :c:func:`VIDIOC_ENUM_FMT` ioctl must be
+supported as well.
+
+To use the :ref:`format` ioctls applications set the ``type`` field of the
+:c:type:`v4l2_format` structure to ``V4L2_BUF_TYPE_AUDIO_CAPTURE`` or to
+``V4L2_BUF_TYPE_AUDIO_OUTPUT``. Both drivers and applications must set the
+remainder of the :c:type:`v4l2_format` structure to 0.
+
+.. c:type:: v4l2_audio_format
+
+.. tabularcolumns:: |p{1.4cm}|p{2.4cm}|p{13.5cm}|
+
+.. flat-table:: struct v4l2_audio_format
+:header-rows: 0
+:stub-columns: 0
+:widths: 1 1 2
+
+* - __u32
+ - ``audioformat``
+ - The sample format, set by the application. see :ref:`pixfmt-audio`
+* - __u32
+ - ``channels``
+ - The channel number, set by the application. channel number range is
+[1, 32].
+* - __u32
+ - ``buffersize``
+ - Maximum buffer size in bytes required for data. The value is set by the
+driver.
diff --git a/Doc
[PATCH v14 07/16] media: uapi: Add V4L2_CAP_AUDIO_M2M capability flag
V4L2_CAP_AUDIO_M2M is similar to V4L2_CAP_VIDEO_M2M flag.
It is used for audio memory to memory case.
Signed-off-by: Shengjiu Wang
---
Documentation/userspace-api/media/v4l/vidioc-querycap.rst| 3 +++
Documentation/userspace-api/media/videodev2.h.rst.exceptions | 1 +
include/uapi/linux/videodev2.h | 1 +
3 files changed, 5 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/vidioc-querycap.rst
b/Documentation/userspace-api/media/v4l/vidioc-querycap.rst
index 6c57b8428356..1c0d97bf192a 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-querycap.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-querycap.rst
@@ -173,6 +173,9 @@ specification the ioctl returns an ``EINVAL`` error code.
interface. A video overlay device typically stores captured images
directly in the video memory of a graphics card, with hardware
clipping and scaling.
+* - ``V4L2_CAP_AUDIO_M2M``
+ - 0x0008
+ - The device supports the audio Memory-To-Memory interface.
* - ``V4L2_CAP_VBI_CAPTURE``
- 0x0010
- The device supports the :ref:`Raw VBI Capture `
diff --git a/Documentation/userspace-api/media/videodev2.h.rst.exceptions
b/Documentation/userspace-api/media/videodev2.h.rst.exceptions
index 3e58aac4ef0b..da6d0b8e4c2c 100644
--- a/Documentation/userspace-api/media/videodev2.h.rst.exceptions
+++ b/Documentation/userspace-api/media/videodev2.h.rst.exceptions
@@ -197,6 +197,7 @@ replace define V4L2_CAP_META_OUTPUT device-capabilities
replace define V4L2_CAP_DEVICE_CAPS device-capabilities
replace define V4L2_CAP_TOUCH device-capabilities
replace define V4L2_CAP_IO_MC device-capabilities
+replace define V4L2_CAP_AUDIO_M2M device-capabilities
# V4L2 pix flags
replace define V4L2_PIX_FMT_PRIV_MAGIC :c:type:`v4l2_pix_format`
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index b8573e9ccde6..5cc2a978fd9c 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -473,6 +473,7 @@ struct v4l2_capability {
#define V4L2_CAP_VIDEO_CAPTURE 0x0001 /* Is a video capture
device */
#define V4L2_CAP_VIDEO_OUTPUT 0x0002 /* Is a video output device
*/
#define V4L2_CAP_VIDEO_OVERLAY 0x0004 /* Can do video overlay */
+#define V4L2_CAP_AUDIO_M2M 0x0008 /* audio memory to memory */
#define V4L2_CAP_VBI_CAPTURE 0x0010 /* Is a raw VBI capture
device */
#define V4L2_CAP_VBI_OUTPUT0x0020 /* Is a raw VBI output
device */
#define V4L2_CAP_SLICED_VBI_CAPTURE0x0040 /* Is a sliced VBI capture
device */
--
2.34.1
[PATCH v14 06/16] ASoC: fsl_easrc: register m2m platform device
Register m2m platform device,that user can
use M2M feature.
Signed-off-by: Shengjiu Wang
Acked-by: Mark Brown
---
sound/soc/fsl/fsl_easrc.c | 19 +++
1 file changed, 19 insertions(+)
diff --git a/sound/soc/fsl/fsl_easrc.c b/sound/soc/fsl/fsl_easrc.c
index cf7ad30a323b..ccbf45c7abf4 100644
--- a/sound/soc/fsl/fsl_easrc.c
+++ b/sound/soc/fsl/fsl_easrc.c
@@ -2075,6 +2075,7 @@ MODULE_DEVICE_TABLE(of, fsl_easrc_dt_ids);
static int fsl_easrc_probe(struct platform_device *pdev)
{
struct fsl_easrc_priv *easrc_priv;
+ struct fsl_asrc_m2m_pdata m2m_pdata;
struct device *dev = >dev;
struct fsl_asrc *easrc;
struct resource *res;
@@ -2190,6 +2191,19 @@ static int fsl_easrc_probe(struct platform_device *pdev)
goto err_pm_disable;
}
+ m2m_pdata.asrc = easrc;
+ m2m_pdata.fmt_in = FSL_EASRC_FORMATS;
+ m2m_pdata.fmt_out = FSL_EASRC_FORMATS |
SNDRV_PCM_FMTBIT_IEC958_SUBFRAME_LE;
+ m2m_pdata.rate_min = 8000;
+ m2m_pdata.rate_max = 768000;
+ m2m_pdata.chan_min = 1;
+ m2m_pdata.chan_max = 32;
+ easrc->m2m_pdev = platform_device_register_data(>dev,
+ M2M_DRV_NAME,
+ PLATFORM_DEVID_AUTO,
+ _pdata,
+ sizeof(m2m_pdata));
+
return 0;
err_pm_disable:
@@ -2199,6 +2213,11 @@ static int fsl_easrc_probe(struct platform_device *pdev)
static void fsl_easrc_remove(struct platform_device *pdev)
{
+ struct fsl_asrc *easrc = dev_get_drvdata(>dev);
+
+ if (easrc->m2m_pdev && !IS_ERR(easrc->m2m_pdev))
+ platform_device_unregister(easrc->m2m_pdev);
+
pm_runtime_disable(>dev);
}
--
2.34.1
[PATCH v14 05/16] ASoC: fsl_asrc: register m2m platform device
Register m2m platform device, that user can
use M2M feature.
Defined platform data structure and platform
driver name.
Signed-off-by: Shengjiu Wang
Acked-by: Mark Brown
---
include/sound/fsl_asrc_common.h | 23 +++
sound/soc/fsl/fsl_asrc.c| 18 ++
2 files changed, 41 insertions(+)
diff --git a/include/sound/fsl_asrc_common.h b/include/sound/fsl_asrc_common.h
index 3b53d366182f..c709b8906929 100644
--- a/include/sound/fsl_asrc_common.h
+++ b/include/sound/fsl_asrc_common.h
@@ -71,6 +71,7 @@ struct fsl_asrc_pair {
* @dma_params_rx: DMA parameters for receive channel
* @dma_params_tx: DMA parameters for transmit channel
* @pdev: platform device pointer
+ * @m2m_pdev: m2m platform device pointer
* @regmap: regmap handler
* @paddr: physical address to the base address of registers
* @mem_clk: clock source to access register
@@ -103,6 +104,7 @@ struct fsl_asrc {
struct snd_dmaengine_dai_dma_data dma_params_rx;
struct snd_dmaengine_dai_dma_data dma_params_tx;
struct platform_device *pdev;
+ struct platform_device *m2m_pdev;
struct regmap *regmap;
unsigned long paddr;
struct clk *mem_clk;
@@ -139,6 +141,27 @@ struct fsl_asrc {
void *private;
};
+/**
+ * struct fsl_asrc_m2m_pdata - platform data
+ * @asrc: pointer to struct fsl_asrc
+ * @fmt_in: input sample format
+ * @fmt_out: output sample format
+ * @chan_min: minimum channel number
+ * @chan_max: maximum channel number
+ * @rate_min: minimum rate
+ * @rate_max: maximum rete
+ */
+struct fsl_asrc_m2m_pdata {
+ struct fsl_asrc *asrc;
+ u64 fmt_in;
+ u64 fmt_out;
+ int chan_min;
+ int chan_max;
+ int rate_min;
+ int rate_max;
+};
+
+#define M2M_DRV_NAME "fsl_asrc_m2m"
#define DRV_NAME "fsl-asrc-dai"
extern struct snd_soc_component_driver fsl_asrc_component;
diff --git a/sound/soc/fsl/fsl_asrc.c b/sound/soc/fsl/fsl_asrc.c
index 7d8643ee0ba0..5ecb5d869607 100644
--- a/sound/soc/fsl/fsl_asrc.c
+++ b/sound/soc/fsl/fsl_asrc.c
@@ -1187,6 +1187,7 @@ static int fsl_asrc_runtime_suspend(struct device *dev);
static int fsl_asrc_probe(struct platform_device *pdev)
{
struct device_node *np = pdev->dev.of_node;
+ struct fsl_asrc_m2m_pdata m2m_pdata;
struct fsl_asrc_priv *asrc_priv;
struct fsl_asrc *asrc;
struct resource *res;
@@ -1368,6 +1369,18 @@ static int fsl_asrc_probe(struct platform_device *pdev)
goto err_pm_get_sync;
}
+ m2m_pdata.asrc = asrc;
+ m2m_pdata.fmt_in = FSL_ASRC_FORMATS;
+ m2m_pdata.fmt_out = FSL_ASRC_FORMATS | SNDRV_PCM_FMTBIT_S8;
+ m2m_pdata.rate_min = 5512;
+ m2m_pdata.rate_max = 192000;
+ m2m_pdata.chan_min = 1;
+ m2m_pdata.chan_max = 10;
+ asrc->m2m_pdev = platform_device_register_data(>dev,
+ M2M_DRV_NAME,
+ PLATFORM_DEVID_AUTO,
+ _pdata,
+ sizeof(m2m_pdata));
return 0;
err_pm_get_sync:
@@ -1380,6 +1393,11 @@ static int fsl_asrc_probe(struct platform_device *pdev)
static void fsl_asrc_remove(struct platform_device *pdev)
{
+ struct fsl_asrc *asrc = dev_get_drvdata(>dev);
+
+ if (asrc->m2m_pdev && !IS_ERR(asrc->m2m_pdev))
+ platform_device_unregister(asrc->m2m_pdev);
+
pm_runtime_disable(>dev);
if (!pm_runtime_status_suspended(>dev))
fsl_asrc_runtime_suspend(>dev);
--
2.34.1
[PATCH v14 04/16] ASoC: fsl_asrc: move fsl_asrc_common.h to include/sound
Move fsl_asrc_common.h to include/sound that it can be
included from other drivers.
Signed-off-by: Shengjiu Wang
Acked-by: Mark Brown
---
{sound/soc/fsl => include/sound}/fsl_asrc_common.h | 0
sound/soc/fsl/fsl_asrc.h | 2 +-
sound/soc/fsl/fsl_asrc_dma.c | 2 +-
sound/soc/fsl/fsl_easrc.h | 2 +-
4 files changed, 3 insertions(+), 3 deletions(-)
rename {sound/soc/fsl => include/sound}/fsl_asrc_common.h (100%)
diff --git a/sound/soc/fsl/fsl_asrc_common.h b/include/sound/fsl_asrc_common.h
similarity index 100%
rename from sound/soc/fsl/fsl_asrc_common.h
rename to include/sound/fsl_asrc_common.h
diff --git a/sound/soc/fsl/fsl_asrc.h b/sound/soc/fsl/fsl_asrc.h
index 1c492eb237f5..66544624de7b 100644
--- a/sound/soc/fsl/fsl_asrc.h
+++ b/sound/soc/fsl/fsl_asrc.h
@@ -10,7 +10,7 @@
#ifndef _FSL_ASRC_H
#define _FSL_ASRC_H
-#include "fsl_asrc_common.h"
+#include
#define ASRC_M2M_INPUTFIFO_WML 0x4
#define ASRC_M2M_OUTPUTFIFO_WML0x2
diff --git a/sound/soc/fsl/fsl_asrc_dma.c b/sound/soc/fsl/fsl_asrc_dma.c
index f501f47242fb..f067bf1ecea7 100644
--- a/sound/soc/fsl/fsl_asrc_dma.c
+++ b/sound/soc/fsl/fsl_asrc_dma.c
@@ -12,7 +12,7 @@
#include
#include
-#include "fsl_asrc_common.h"
+#include
#define FSL_ASRC_DMABUF_SIZE (256 * 1024)
diff --git a/sound/soc/fsl/fsl_easrc.h b/sound/soc/fsl/fsl_easrc.h
index c9f770862662..a24e540876a4 100644
--- a/sound/soc/fsl/fsl_easrc.h
+++ b/sound/soc/fsl/fsl_easrc.h
@@ -9,7 +9,7 @@
#include
#include
-#include "fsl_asrc_common.h"
+#include
/* EASRC Register Map */
--
2.34.1
[PATCH v14 03/16] ASoC: fsl_easrc: define functions for memory to memory usage
ASRC can be used on memory to memory case, define several
functions for m2m usage and export them as function pointer.
Signed-off-by: Shengjiu Wang
Acked-by: Mark Brown
---
sound/soc/fsl/fsl_easrc.c | 214 ++
sound/soc/fsl/fsl_easrc.h | 4 +
2 files changed, 218 insertions(+)
diff --git a/sound/soc/fsl/fsl_easrc.c b/sound/soc/fsl/fsl_easrc.c
index ec53bda46a46..cf7ad30a323b 100644
--- a/sound/soc/fsl/fsl_easrc.c
+++ b/sound/soc/fsl/fsl_easrc.c
@@ -1861,6 +1861,211 @@ static int fsl_easrc_get_fifo_addr(u8 dir, enum
asrc_pair_index index)
return REG_EASRC_FIFO(dir, index);
}
+/* Get sample numbers in FIFO */
+static unsigned int fsl_easrc_get_output_fifo_size(struct fsl_asrc_pair *pair)
+{
+ struct fsl_asrc *asrc = pair->asrc;
+ enum asrc_pair_index index = pair->index;
+ u32 val;
+
+ regmap_read(asrc->regmap, REG_EASRC_SFS(index), );
+ val &= EASRC_SFS_NSGO_MASK;
+
+ return val >> EASRC_SFS_NSGO_SHIFT;
+}
+
+static int fsl_easrc_m2m_prepare(struct fsl_asrc_pair *pair)
+{
+ struct fsl_easrc_ctx_priv *ctx_priv = pair->private;
+ struct fsl_asrc *asrc = pair->asrc;
+ struct device *dev = >pdev->dev;
+ int ret;
+
+ ctx_priv->in_params.sample_rate = pair->rate[IN];
+ ctx_priv->in_params.sample_format = pair->sample_format[IN];
+ ctx_priv->out_params.sample_rate = pair->rate[OUT];
+ ctx_priv->out_params.sample_format = pair->sample_format[OUT];
+
+ ctx_priv->in_params.fifo_wtmk = FSL_EASRC_INPUTFIFO_WML;
+ ctx_priv->out_params.fifo_wtmk = FSL_EASRC_OUTPUTFIFO_WML;
+ /* Fill the right half of the re-sampler with zeros */
+ ctx_priv->rs_init_mode = 0x2;
+ /* Zero fill the right half of the prefilter */
+ ctx_priv->pf_init_mode = 0x2;
+
+ ret = fsl_easrc_set_ctx_format(pair,
+ _priv->in_params.sample_format,
+ _priv->out_params.sample_format);
+ if (ret) {
+ dev_err(dev, "failed to set context format: %d\n", ret);
+ return ret;
+ }
+
+ ret = fsl_easrc_config_context(asrc, pair->index);
+ if (ret) {
+ dev_err(dev, "failed to config context %d\n", ret);
+ return ret;
+ }
+
+ ctx_priv->in_params.iterations = 1;
+ ctx_priv->in_params.group_len = pair->channels;
+ ctx_priv->in_params.access_len = pair->channels;
+ ctx_priv->out_params.iterations = 1;
+ ctx_priv->out_params.group_len = pair->channels;
+ ctx_priv->out_params.access_len = pair->channels;
+
+ ret = fsl_easrc_set_ctx_organziation(pair);
+ if (ret) {
+ dev_err(dev, "failed to set fifo organization\n");
+ return ret;
+ }
+
+ /* The context start flag */
+ pair->first_convert = 1;
+ return 0;
+}
+
+static int fsl_easrc_m2m_start(struct fsl_asrc_pair *pair)
+{
+ /* start context once */
+ if (pair->first_convert) {
+ fsl_easrc_start_context(pair);
+ pair->first_convert = 0;
+ }
+
+ return 0;
+}
+
+static int fsl_easrc_m2m_stop(struct fsl_asrc_pair *pair)
+{
+ /* Stop pair/context */
+ if (!pair->first_convert) {
+ fsl_easrc_stop_context(pair);
+ pair->first_convert = 1;
+ }
+
+ return 0;
+}
+
+/* calculate capture data length according to output data length and sample
rate */
+static int fsl_easrc_m2m_calc_out_len(struct fsl_asrc_pair *pair, int
input_buffer_length)
+{
+ struct fsl_asrc *easrc = pair->asrc;
+ struct fsl_easrc_priv *easrc_priv = easrc->private;
+ struct fsl_easrc_ctx_priv *ctx_priv = pair->private;
+ unsigned int in_rate = ctx_priv->in_params.norm_rate;
+ unsigned int out_rate = ctx_priv->out_params.norm_rate;
+ unsigned int channels = pair->channels;
+ unsigned int in_samples, out_samples;
+ unsigned int in_width, out_width;
+ unsigned int out_length;
+ unsigned int frac_bits;
+ u64 val1, val2;
+
+ switch (easrc_priv->rs_num_taps) {
+ case EASRC_RS_32_TAPS:
+ /* integer bits = 5; */
+ frac_bits = 39;
+ break;
+ case EASRC_RS_64_TAPS:
+ /* integer bits = 6; */
+ frac_bits = 38;
+ break;
+ case EASRC_RS_128_TAPS:
+ /* integer bits = 7; */
+ frac_bits = 37;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ val1 = (u64)in_rate << frac_bits;
+ do_div(val1, out_rate);
+ val1 += (s64)ctx_priv->ratio_mod << (frac_bits - 31);
+
+ in_width =
snd_pcm_format_physical_width(c
[PATCH v14 02/16] ASoC: fsl_asrc: define functions for memory to memory usage
ASRC can be used on memory to memory case, define several
functions for m2m usage.
m2m_prepare: prepare for the start step
m2m_start: the start step
m2m_unprepare: unprepare for stop step, optional
m2m_stop: stop step
m2m_check_format: check format is supported or not
m2m_calc_out_len: calculate output length according to input length
m2m_get_maxburst: burst size for dma
m2m_pair_suspend: suspend function of pair, optional.
m2m_pair_resume: resume function of pair
get_output_fifo_size: get remaining data size in FIFO
Signed-off-by: Shengjiu Wang
Acked-by: Mark Brown
---
sound/soc/fsl/fsl_asrc.c| 126
sound/soc/fsl/fsl_asrc.h| 2 +
sound/soc/fsl/fsl_asrc_common.h | 37 ++
3 files changed, 165 insertions(+)
diff --git a/sound/soc/fsl/fsl_asrc.c b/sound/soc/fsl/fsl_asrc.c
index b793263291dc..7d8643ee0ba0 100644
--- a/sound/soc/fsl/fsl_asrc.c
+++ b/sound/soc/fsl/fsl_asrc.c
@@ -1063,6 +1063,124 @@ static int fsl_asrc_get_fifo_addr(u8 dir, enum
asrc_pair_index index)
return REG_ASRDx(dir, index);
}
+/* Get sample numbers in FIFO */
+static unsigned int fsl_asrc_get_output_fifo_size(struct fsl_asrc_pair *pair)
+{
+ struct fsl_asrc *asrc = pair->asrc;
+ enum asrc_pair_index index = pair->index;
+ u32 val;
+
+ regmap_read(asrc->regmap, REG_ASRFST(index), );
+
+ val &= ASRFSTi_OUTPUT_FIFO_MASK;
+
+ return val >> ASRFSTi_OUTPUT_FIFO_SHIFT;
+}
+
+static int fsl_asrc_m2m_prepare(struct fsl_asrc_pair *pair)
+{
+ struct fsl_asrc_pair_priv *pair_priv = pair->private;
+ struct fsl_asrc *asrc = pair->asrc;
+ struct device *dev = >pdev->dev;
+ struct asrc_config config;
+ int ret;
+
+ /* fill config */
+ config.pair = pair->index;
+ config.channel_num = pair->channels;
+ config.input_sample_rate = pair->rate[IN];
+ config.output_sample_rate = pair->rate[OUT];
+ config.input_format = pair->sample_format[IN];
+ config.output_format = pair->sample_format[OUT];
+ config.inclk = INCLK_NONE;
+ config.outclk = OUTCLK_ASRCK1_CLK;
+
+ pair_priv->config =
+ ret = fsl_asrc_config_pair(pair, true);
+ if (ret) {
+ dev_err(dev, "failed to config pair: %d\n", ret);
+ return ret;
+ }
+
+ pair->first_convert = 1;
+
+ return 0;
+}
+
+static int fsl_asrc_m2m_start(struct fsl_asrc_pair *pair)
+{
+ if (pair->first_convert) {
+ fsl_asrc_start_pair(pair);
+ pair->first_convert = 0;
+ }
+ /*
+* Clear DMA request during the stall state of ASRC:
+* During STALL state, the remaining in input fifo would never be
+* smaller than the input threshold while the output fifo would not
+* be bigger than output one. Thus the DMA request would be cleared.
+*/
+ fsl_asrc_set_watermarks(pair, ASRC_FIFO_THRESHOLD_MIN,
+ ASRC_FIFO_THRESHOLD_MAX);
+
+ /* Update the real input threshold to raise DMA request */
+ fsl_asrc_set_watermarks(pair, ASRC_M2M_INPUTFIFO_WML,
+ ASRC_M2M_OUTPUTFIFO_WML);
+
+ return 0;
+}
+
+static int fsl_asrc_m2m_stop(struct fsl_asrc_pair *pair)
+{
+ if (!pair->first_convert) {
+ fsl_asrc_stop_pair(pair);
+ pair->first_convert = 1;
+ }
+
+ return 0;
+}
+
+/* calculate capture data length according to output data length and sample
rate */
+static int fsl_asrc_m2m_calc_out_len(struct fsl_asrc_pair *pair, int
input_buffer_length)
+{
+ unsigned int in_width, out_width;
+ unsigned int channels = pair->channels;
+ unsigned int in_samples, out_samples;
+ unsigned int out_length;
+
+ in_width = snd_pcm_format_physical_width(pair->sample_format[IN]) / 8;
+ out_width = snd_pcm_format_physical_width(pair->sample_format[OUT]) / 8;
+
+ in_samples = input_buffer_length / in_width / channels;
+ out_samples = pair->rate[OUT] * in_samples / pair->rate[IN];
+ out_length = (out_samples - ASRC_OUTPUT_LAST_SAMPLE) * out_width *
channels;
+
+ return out_length;
+}
+
+static int fsl_asrc_m2m_get_maxburst(u8 dir, struct fsl_asrc_pair *pair)
+{
+ struct fsl_asrc *asrc = pair->asrc;
+ struct fsl_asrc_priv *asrc_priv = asrc->private;
+ int wml = (dir == IN) ? ASRC_M2M_INPUTFIFO_WML :
ASRC_M2M_OUTPUTFIFO_WML;
+
+ if (!asrc_priv->soc->use_edma)
+ return wml * pair->channels;
+ else
+ return 1;
+}
+
+static int fsl_asrc_m2m_pair_resume(struct fsl_asrc_pair *pair)
+{
+ struct fsl_asrc *asrc = pair->asrc;
+ int i;
+
+ for (i = 0; i < pair->channels * 4; i++)
+ regmap_write(asrc->regmap, REG_ASRDI(pair->index), 0);
+
+
[PATCH v14 00/16] Add audio support in v4l2 framework
Audio signal processing also has the requirement for memory to memory similar as Video. This asrc memory to memory (memory ->asrc->memory) case is a non real time use case. User fills the input buffer to the asrc module, after conversion, then asrc sends back the output buffer to user. So it is not a traditional ALSA playback and capture case. It is a specific use case, there is no reference in current kernel. v4l2 memory to memory is the closed implementation, v4l2 current support video, image, radio, tuner, touch devices, so it is not complicated to add support for this specific audio case. Because we had implemented the "memory -> asrc ->i2s device-> codec" use case in ALSA. Now the "memory->asrc->memory" needs to reuse the code in asrc driver, so the first 3 patches is for refining the code to make it can be shared by the "memory->asrc->memory" driver. The main change is in the v4l2 side, A /dev/vl4-audioX will be created, user applications only use the ioctl of v4l2 framework. Other change is to add memory to memory support for two kinds of i.MX ASRC module. changes in v14: - document the reservation of 'AUXX' fourcc format. - add v4l2_audfmt_to_fourcc() definition. changes in v13 - change 'pixelformat' to 'audioformat' in dev-audio-mem2mem.rst - add more description for clock drift in ext-ctrls-audio-m2m.rst - Add "media: v4l2-ctrls: add support for fraction_bits" from Hans to avoid build issue for kernel test robot changes in v12 - minor changes according to comments - drop min_buffers_needed = 1 and V4L2_CTRL_FLAG_UPDATE flag - drop bus_info changes in v11 - add add-fixed-point-test-controls in vivid. - add v4l2_ctrl_fp_compose() helper function for min and max changes in v10 - remove FIXED_POINT type - change code base on media: v4l2-ctrls: add support for fraction_bits - fix issue reported by kernel test robot - remove module_alias changes in v9: - add MEDIA_ENT_F_PROC_AUDIO_RESAMPLER. - add MEDIA_INTF_T_V4L_AUDIO - add media controller support - refine the vim2m-audio to support 8k<->16k conversion. changes in v8: - refine V4L2_CAP_AUDIO_M2M to be 0x0008 - update doc for FIXED_POINT - address comments for imx-asrc changes in v7: - add acked-by from Mark - separate commit for fixed point, m2m audio class, audio rate controls - use INTEGER_MENU for rate, FIXED_POINT for rate offset - remove used fmts - address other comments for Hans changes in v6: - use m2m_prepare/m2m_unprepare/m2m_start/m2m_stop to replace m2m_start_part_one/m2m_stop_part_one, m2m_start_part_two/m2m_stop_part_two. - change V4L2_CTRL_TYPE_ASRC_RATE to V4L2_CTRL_TYPE_FIXED_POINT - fix warning by kernel test rebot - remove some unused format V4L2_AUDIO_FMT_XX - Get SNDRV_PCM_FORMAT from V4L2_AUDIO_FMT in driver. - rename audm2m to viaudm2m. changes in v5: - remove V4L2_AUDIO_FMT_LPCM - define audio pixel format like V4L2_AUDIO_FMT_S8... - remove rate and format in struct v4l2_audio_format. - Add V4L2_CID_ASRC_SOURCE_RATE and V4L2_CID_ASRC_DEST_RATE controls - updata document accordingly. changes in v4: - update document style - separate V4L2_AUDIO_FMT_LPCM and V4L2_CAP_AUDIO_M2M in separate commit changes in v3: - Modify documents for adding audio m2m support - Add audio virtual m2m driver - Defined V4L2_AUDIO_FMT_LPCM format type for audio. - Defined V4L2_CAP_AUDIO_M2M capability type for audio m2m case. - with modification in v4l-utils, pass v4l2-compliance test. changes in v2: - decouple the implementation in v4l2 and ALSA - implement the memory to memory driver as a platfrom driver and move it to driver/media - move fsl_asrc_common.h to include/sound folder Hans Verkuil (1): media: v4l2-ctrls: add support for fraction_bits Shengjiu Wang (15): ASoC: fsl_asrc: define functions for memory to memory usage ASoC: fsl_easrc: define functions for memory to memory usage ASoC: fsl_asrc: move fsl_asrc_common.h to include/sound ASoC: fsl_asrc: register m2m platform device ASoC: fsl_easrc: register m2m platform device media: uapi: Add V4L2_CAP_AUDIO_M2M capability flag media: v4l2: Add audio capture and output support media: uapi: Define audio sample format fourcc type media: uapi: Add V4L2_CTRL_CLASS_M2M_AUDIO media: uapi: Add audio rate controls support media: uapi: Declare interface types for Audio media: uapi: Add an entity type for audio resampler media: vivid: add fixed point test controls media: imx-asrc: Add memory to memory driver media: vim2m-audio: add virtual driver for audio memory to memory .../media/mediactl/media-types.rst| 11 + .../userspace-api/media/v4l/buffer.rst|6 + .../userspace-api/media/v4l/common.rst|1 + .../media/v4l/dev-audio-mem2mem.rst | 71 + .../userspace-api/media/v4l/devices.rst |1 + .../media/v4l/ext-ctrls-audio-m2m.rst | 59 + .../userspace-api/media/v4l/pixfmt-audio.rst | 100 ++ .../user
[PATCH v14 01/16] media: v4l2-ctrls: add support for fraction_bits
From: Hans Verkuil
This adds support for the fraction_bits field, used with integer controls.
This allows fixed point formats to be described.
The fraction_bits field is only exposed through VIDIOC_QUERY_EXT_CTRL.
For a given signed two's complement Qf fixed point value 'f' equals
fraction_bits.
Signed-off-by: Hans Verkuil
---
.../media/v4l/vidioc-queryctrl.rst| 11 ++-
drivers/media/v4l2-core/v4l2-ctrls-api.c | 1 +
drivers/media/v4l2-core/v4l2-ctrls-core.c | 93 +++
include/media/v4l2-ctrls.h| 7 +-
include/uapi/linux/videodev2.h| 3 +-
5 files changed, 95 insertions(+), 20 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
index 4d38acafe8e1..e65c7e5d78ec 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
@@ -267,7 +267,16 @@ See also the examples in :ref:`control`.
- The size of each dimension. The first ``nr_of_dims`` elements of
this array must be non-zero, all remaining elements must be zero.
* - __u32
- - ``reserved``\ [32]
+ - ``fraction_bits``
+ - The number of least significant bits of the control value that
+form the fraction of the fixed point value. This is 0 if the value
+ is a regular integer. This can be used with all integer control types
+ (``INTEGER``, ``INTEGER64``, ``U8``, ``U16`` and ``U32``).
+ For the signed types the signed two's complement representation is used.
+ This field applies to the control value as well as the ``minimum``,
+ ``maximum``, ``step`` and ``default_value`` fields.
+* - __u32
+ - ``reserved``\ [31]
- Reserved for future extensions. Applications and drivers must set
the array to zero.
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-api.c
b/drivers/media/v4l2-core/v4l2-ctrls-api.c
index d9a422017bd9..ef16b00421ec 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-api.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-api.c
@@ -1101,6 +1101,7 @@ int v4l2_query_ext_ctrl(struct v4l2_ctrl_handler *hdl,
struct v4l2_query_ext_ctr
qc->elems = ctrl->elems;
qc->nr_of_dims = ctrl->nr_of_dims;
memcpy(qc->dims, ctrl->dims, qc->nr_of_dims * sizeof(qc->dims[0]));
+ qc->fraction_bits = ctrl->fraction_bits;
qc->minimum = ctrl->minimum;
qc->maximum = ctrl->maximum;
qc->default_value = ctrl->default_value;
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c
b/drivers/media/v4l2-core/v4l2-ctrls-core.c
index c4d995f32191..d83a37198bb5 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
@@ -252,12 +252,61 @@ void v4l2_ctrl_type_op_init(const struct v4l2_ctrl *ctrl,
u32 from_idx,
}
EXPORT_SYMBOL(v4l2_ctrl_type_op_init);
+static void v4l2_ctrl_log_fp(s64 v, unsigned int fraction_bits)
+{
+ s64 i, f, mask;
+
+ if (!fraction_bits) {
+ pr_cont("%lld", v);
+ return;
+ }
+
+ mask = (1ULL << fraction_bits) - 1;
+
+ /*
+* Note: this function does not support fixed point u64 with
+* fraction_bits set to 64. At the moment there is no U64
+* control type, but if that is added, then this code will have
+* to add support for it.
+*/
+ if (fraction_bits >= 63)
+ i = v < 0 ? -1 : 0;
+ else
+ i = div64_s64(v, 1LL << fraction_bits);
+
+ f = v < 0 ? -((-v) & mask) : (v & mask);
+
+ if (!f) {
+ pr_cont("%lld", i);
+ } else if (fraction_bits < 20) {
+ u64 div = 1ULL << fraction_bits;
+
+ if (!i && f < 0)
+ pr_cont("-%lld/%llu", -f, div);
+ else if (!i)
+ pr_cont("%lld/%llu", f, div);
+ else if (i < 0 || f < 0)
+ pr_cont("-%lld-%llu/%llu", -i, -f, div);
+ else
+ pr_cont("%lld+%llu/%llu", i, f, div);
+ } else {
+ if (!i && f < 0)
+ pr_cont("-%lld/(2^%u)", -f, fraction_bits);
+ else if (!i)
+ pr_cont("%lld/(2^%u)", f, fraction_bits);
+ else if (i < 0 || f < 0)
+ pr_cont("-%lld-%llu/(2^%u)", -i, -f, fraction_bits);
+ else
+ pr_cont("%lld+%llu/(2^%u)", i, f, fraction_bits);
+ }
+}
+
void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl)
{
union v4l2_ctrl_ptr ptr = ctrl->p_cur;
if (ctrl->is_array) {
- unsigned i;
+ unsigned int i;
for (i = 0; i < ctrl->nr_of_dims; i++)
pr_cont("[%u]", ctrl->dims[i]);
@@ -266,7 +315,7 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl)
Re: [PATCH v13 09/16] media: uapi: Define audio sample format fourcc type
On Fri, Mar 8, 2024 at 10:04 PM Hans Verkuil wrote:
>
> On 08/03/2024 2:52 pm, Shengjiu Wang wrote:
> > On Fri, Mar 8, 2024 at 8:06 PM Hans Verkuil wrote:
> >>
> >> On 08/03/2024 12:52 pm, Shengjiu Wang wrote:
> >>> On Fri, Mar 8, 2024 at 3:34 PM Hans Verkuil wrote:
> >>>>
> >>>> Hi Shengjiu,
> >>>>
> >>>> After thinking it over I think this patch can be improved:
> >>>>
> >>>> On 26/02/2024 9:28 am, Shengjiu Wang wrote:
> >>>>> The audio sample format definition is from alsa,
> >>>>> the header file is include/uapi/sound/asound.h, but
> >>>>> don't include this header file directly, because in
> >>>>> user space, there is another copy in alsa-lib.
> >>>>> There will be conflict in userspace for include
> >>>>> videodev2.h & asound.h and asoundlib.h
> >>>>>
> >>>>> Here still use the fourcc format.
> >>>>>
> >>>>> Signed-off-by: Shengjiu Wang
> >>>>> ---
> >>>>> .../userspace-api/media/v4l/pixfmt-audio.rst | 87 +++
> >>>>> .../userspace-api/media/v4l/pixfmt.rst| 1 +
> >>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++
> >>>>> include/uapi/linux/videodev2.h| 23 +
> >>>>> 4 files changed, 124 insertions(+)
> >>>>> create mode 100644
> >>>>> Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >>>>>
> >>>>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >>>>> b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >>>>> new file mode 100644
> >>>>> index ..04b4a7fbd8f4
> >>>>> --- /dev/null
> >>>>> +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >>>>> @@ -0,0 +1,87 @@
> >>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> >>>>> +
> >>>>> +.. _pixfmt-audio:
> >>>>> +
> >>>>> +*
> >>>>> +Audio Formats
> >>>>> +*
> >>>>> +
> >>>>> +These formats are used for :ref:`audiomem2mem` interface only.
> >>>>
> >>>> Here you should also document that all these fourccs start with 'AU' and
> >>>> are
> >>>> reserved for mappings of the snd_pcm_format_t type.
> >>>>
> >>>> Also document the v4l2_fourcc_to_audfmt define and the
> >>>> v4l2_audfmt_to_fourcc
> >>>> define (see also below).
> >>>
> >>> How about below description?
> >>> '
> >>> All these fourccs starting with 'AU' are reserved for mappings
> >>
> >> All these fourccs -> All FourCCs
> >>
> >>> of the snd_pcm_format_t type.
> >>>
> >>> The v4l2_audfmt_to_fourcc() is defined to convert snd_pcm_format_t
> >>
> >> convert -> convert the
> >>
> >>> type to fourcc. The first character is 'A', the second character
> >>
> >> fourcc -> a FourCC
> >>
> >>> is 'U', the third character is ten's digit of snd_pcm_format_t,
> >>> the fourth character is unit's digit of snd_pcm_format_t.
> >>
> >> "is 'U', and the remaining two characters is the snd_pcm_format_t
> >> value in ASCII. Example: SNDRV_PCM_FORMAT_S16_LE (with value 2)
> >> maps to 'AU02' and SNDRV_PCM_FORMAT_S24_3LE (with value 32) maps
> >> to 'AU32'."
> >>
> >>>
> >>> The v4l2_fourcc_to_audfmt() is defined to convert these fourccs to
> >>
> >> fourccs -> FourCCs
> >>
> >>> snd_pcm_format_t type.
> >>
> >> BTW, given the way snd_pcm_format_t is defined I am fairly certain
> >> some casts are needed for the v4l2_audfmt_to_fourcc define.
> >>
> >> Regards,
> >>
> >> Hans
> >>
> >>> '
> >>> Best regards
> >>> Shengjiu Wang
> >>>>
> >>>>> +
> >>>>> +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}|
> >>>>> +
> >>>>> +.. cssclass:: longtable
> >>>>> +
> >>>>> +.. flat-table:: Audio Format
> >>>>> +
Re: [PATCH v13 09/16] media: uapi: Define audio sample format fourcc type
On Fri, Mar 8, 2024 at 8:06 PM Hans Verkuil wrote:
>
> On 08/03/2024 12:52 pm, Shengjiu Wang wrote:
> > On Fri, Mar 8, 2024 at 3:34 PM Hans Verkuil wrote:
> >>
> >> Hi Shengjiu,
> >>
> >> After thinking it over I think this patch can be improved:
> >>
> >> On 26/02/2024 9:28 am, Shengjiu Wang wrote:
> >>> The audio sample format definition is from alsa,
> >>> the header file is include/uapi/sound/asound.h, but
> >>> don't include this header file directly, because in
> >>> user space, there is another copy in alsa-lib.
> >>> There will be conflict in userspace for include
> >>> videodev2.h & asound.h and asoundlib.h
> >>>
> >>> Here still use the fourcc format.
> >>>
> >>> Signed-off-by: Shengjiu Wang
> >>> ---
> >>> .../userspace-api/media/v4l/pixfmt-audio.rst | 87 +++
> >>> .../userspace-api/media/v4l/pixfmt.rst| 1 +
> >>> drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++
> >>> include/uapi/linux/videodev2.h| 23 +
> >>> 4 files changed, 124 insertions(+)
> >>> create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >>>
> >>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >>> b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >>> new file mode 100644
> >>> index ..04b4a7fbd8f4
> >>> --- /dev/null
> >>> +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >>> @@ -0,0 +1,87 @@
> >>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> >>> +
> >>> +.. _pixfmt-audio:
> >>> +
> >>> +*
> >>> +Audio Formats
> >>> +*
> >>> +
> >>> +These formats are used for :ref:`audiomem2mem` interface only.
> >>
> >> Here you should also document that all these fourccs start with 'AU' and
> >> are
> >> reserved for mappings of the snd_pcm_format_t type.
> >>
> >> Also document the v4l2_fourcc_to_audfmt define and the
> >> v4l2_audfmt_to_fourcc
> >> define (see also below).
> >
> > How about below description?
> > '
> > All these fourccs starting with 'AU' are reserved for mappings
>
> All these fourccs -> All FourCCs
>
> > of the snd_pcm_format_t type.
> >
> > The v4l2_audfmt_to_fourcc() is defined to convert snd_pcm_format_t
>
> convert -> convert the
>
> > type to fourcc. The first character is 'A', the second character
>
> fourcc -> a FourCC
>
> > is 'U', the third character is ten's digit of snd_pcm_format_t,
> > the fourth character is unit's digit of snd_pcm_format_t.
>
> "is 'U', and the remaining two characters is the snd_pcm_format_t
> value in ASCII. Example: SNDRV_PCM_FORMAT_S16_LE (with value 2)
> maps to 'AU02' and SNDRV_PCM_FORMAT_S24_3LE (with value 32) maps
> to 'AU32'."
>
> >
> > The v4l2_fourcc_to_audfmt() is defined to convert these fourccs to
>
> fourccs -> FourCCs
>
> > snd_pcm_format_t type.
>
> BTW, given the way snd_pcm_format_t is defined I am fairly certain
> some casts are needed for the v4l2_audfmt_to_fourcc define.
>
> Regards,
>
> Hans
>
> > '
> > Best regards
> > Shengjiu Wang
> >>
> >>> +
> >>> +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}|
> >>> +
> >>> +.. cssclass:: longtable
> >>> +
> >>> +.. flat-table:: Audio Format
> >>> +:header-rows: 1
> >>> +:stub-columns: 0
> >>> +:widths: 3 1 4
> >>> +
> >>> +* - Identifier
> >>> + - Code
> >>> + - Details
> >>> +* .. _V4L2-AUDIO-FMT-S8:
> >>> +
> >>> + - ``V4L2_AUDIO_FMT_S8``
> >>> + - 'S8'
> >>> + - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA
> >>> +* .. _V4L2-AUDIO-FMT-S16-LE:
> >>> +
> >>> + - ``V4L2_AUDIO_FMT_S16_LE``
> >>> + - 'S16_LE'
> >>> + - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA
> >>> +* .. _V4L2-AUDIO-FMT-U16-LE:
> >>> +
> >>> + - ``V4L2_AUDIO_FMT_U16_LE``
> >>> + - 'U16_LE'
> >>> + - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA
> >>> +* .. _V4L2-AUDIO-FMT-S24-LE:
&g
Re: [PATCH v13 09/16] media: uapi: Define audio sample format fourcc type
On Fri, Mar 8, 2024 at 3:34 PM Hans Verkuil wrote:
>
> Hi Shengjiu,
>
> After thinking it over I think this patch can be improved:
>
> On 26/02/2024 9:28 am, Shengjiu Wang wrote:
> > The audio sample format definition is from alsa,
> > the header file is include/uapi/sound/asound.h, but
> > don't include this header file directly, because in
> > user space, there is another copy in alsa-lib.
> > There will be conflict in userspace for include
> > videodev2.h & asound.h and asoundlib.h
> >
> > Here still use the fourcc format.
> >
> > Signed-off-by: Shengjiu Wang
> > ---
> > .../userspace-api/media/v4l/pixfmt-audio.rst | 87 +++
> > .../userspace-api/media/v4l/pixfmt.rst| 1 +
> > drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++
> > include/uapi/linux/videodev2.h| 23 +
> > 4 files changed, 124 insertions(+)
> > create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >
> > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> > b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> > new file mode 100644
> > index ..04b4a7fbd8f4
> > --- /dev/null
> > +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> > @@ -0,0 +1,87 @@
> > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> > +
> > +.. _pixfmt-audio:
> > +
> > +*
> > +Audio Formats
> > +*
> > +
> > +These formats are used for :ref:`audiomem2mem` interface only.
>
> Here you should also document that all these fourccs start with 'AU' and are
> reserved for mappings of the snd_pcm_format_t type.
>
> Also document the v4l2_fourcc_to_audfmt define and the v4l2_audfmt_to_fourcc
> define (see also below).
How about below description?
'
All these fourccs starting with 'AU' are reserved for mappings
of the snd_pcm_format_t type.
The v4l2_audfmt_to_fourcc() is defined to convert snd_pcm_format_t
type to fourcc. The first character is 'A', the second character
is 'U', the third character is ten's digit of snd_pcm_format_t,
the fourth character is unit's digit of snd_pcm_format_t.
The v4l2_fourcc_to_audfmt() is defined to convert these fourccs to
snd_pcm_format_t type.
'
Best regards
Shengjiu Wang
>
> > +
> > +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}|
> > +
> > +.. cssclass:: longtable
> > +
> > +.. flat-table:: Audio Format
> > +:header-rows: 1
> > +:stub-columns: 0
> > +:widths: 3 1 4
> > +
> > +* - Identifier
> > + - Code
> > + - Details
> > +* .. _V4L2-AUDIO-FMT-S8:
> > +
> > + - ``V4L2_AUDIO_FMT_S8``
> > + - 'S8'
> > + - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA
> > +* .. _V4L2-AUDIO-FMT-S16-LE:
> > +
> > + - ``V4L2_AUDIO_FMT_S16_LE``
> > + - 'S16_LE'
> > + - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA
> > +* .. _V4L2-AUDIO-FMT-U16-LE:
> > +
> > + - ``V4L2_AUDIO_FMT_U16_LE``
> > + - 'U16_LE'
> > + - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA
> > +* .. _V4L2-AUDIO-FMT-S24-LE:
> > +
> > + - ``V4L2_AUDIO_FMT_S24_LE``
> > + - 'S24_LE'
> > + - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA
> > +* .. _V4L2-AUDIO-FMT-U24-LE:
> > +
> > + - ``V4L2_AUDIO_FMT_U24_LE``
> > + - 'U24_LE'
> > + - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA
> > +* .. _V4L2-AUDIO-FMT-S32-LE:
> > +
> > + - ``V4L2_AUDIO_FMT_S32_LE``
> > + - 'S32_LE'
> > + - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA
> > +* .. _V4L2-AUDIO-FMT-U32-LE:
> > +
> > + - ``V4L2_AUDIO_FMT_U32_LE``
> > + - 'U32_LE'
> > + - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA
> > +* .. _V4L2-AUDIO-FMT-FLOAT-LE:
> > +
> > + - ``V4L2_AUDIO_FMT_FLOAT_LE``
> > + - 'FLOAT_LE'
> > + - Corresponds to SNDRV_PCM_FORMAT_FLOAT_LE in ALSA
> > +* .. _V4L2-AUDIO-FMT-IEC958-SUBFRAME-LE:
> > +
> > + - ``V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE``
> > + - 'IEC958_SUBFRAME_LE'
> > + - Corresponds to SNDRV_PCM_FORMAT_IEC958_SUBFRAME_LE in ALSA
> > +* .. _V4L2-AUDIO-FMT-S24-3LE:
> > +
> > + - ``V4L2_AUDIO_FMT_S24_3LE``
> > + - 'S24_3LE'
> > + - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
> > +* .. _V4L2-AUDIO-FMT-U24-3LE:
> > +
> > + - ``V4L2_AUDIO_FMT_U24_3LE``
> > + - 'U24_3LE'
> > + - Cor
Re: [PATCH v13 09/16] media: uapi: Define audio sample format fourcc type
On Mon, Feb 26, 2024 at 9:55 PM Nicolas Dufresne
wrote:
>
> Le lundi 26 février 2024 à 16:28 +0800, Shengjiu Wang a écrit :
> > The audio sample format definition is from alsa,
> > the header file is include/uapi/sound/asound.h, but
> > don't include this header file directly, because in
> > user space, there is another copy in alsa-lib.
> > There will be conflict in userspace for include
> > videodev2.h & asound.h and asoundlib.h
> >
> > Here still use the fourcc format.
>
> I'd like to join Mauro's voice that duplicating the audio formats is a
bad idea.
> We have the same issues with video formats when you look at V4L2 vs DRM.
You're
> rationale is that videodev2.h will be ambiguous if it includes asound.h,
but
> looking at this change, there is no reason why you need to include
asound.h in
> videodev2.h at all. The format type can be abstracted out with a uint32
in the
> API, and then it would be up to the users to include and use the
appropriate
> formats IDs.
>
Thanks.
There is another reason mentioned by Hans:
"
*The problem is that within V4L2 we use fourcc consistently to describe
aformat, including in VIDIOC_ENUM_FMT. And the expectation is that the
fourcccan be printed to a human readable string (there is even a printk
format forthat these days).But the pcm values are all small integers (and
can even be 0!), andprinting the fourcc will give garbage. It doesn't work
well at allwith the V4L2 API. But by having a straightforward conversion
between thepcm identifier and a fourcc it was really easy to deal with
this.There might even be applications today that call VIDIOC_ENUM_FMT to
seewhat is supported and fail if it is not a proper fourcc is returned.It
will certainly report nonsense in v4l_print_fmtdesc() (v4l2-ioctl.c).One of
the early versions of this patch series did precisely what you request,but
it just doesn't work well within the V4L2 uAPI.*
*"*
Best regards
Shengjiu Wang
> Nicolas
>
> >
> > Signed-off-by: Shengjiu Wang
> > ---
> > .../userspace-api/media/v4l/pixfmt-audio.rst | 87 +++
> > .../userspace-api/media/v4l/pixfmt.rst| 1 +
> > drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++
> > include/uapi/linux/videodev2.h| 23 +
> > 4 files changed, 124 insertions(+)
> > create mode 100644
Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >
> > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> > new file mode 100644
> > index ..04b4a7fbd8f4
> > --- /dev/null
> > +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> > @@ -0,0 +1,87 @@
> > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> > +
> > +.. _pixfmt-audio:
> > +
> > +*
> > +Audio Formats
> > +*
> > +
> > +These formats are used for :ref:`audiomem2mem` interface only.
> > +
> > +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}|
> > +
> > +.. cssclass:: longtable
> > +
> > +.. flat-table:: Audio Format
> > +:header-rows: 1
> > +:stub-columns: 0
> > +:widths: 3 1 4
> > +
> > +* - Identifier
> > + - Code
> > + - Details
> > +* .. _V4L2-AUDIO-FMT-S8:
> > +
> > + - ``V4L2_AUDIO_FMT_S8``
> > + - 'S8'
> > + - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA
> > +* .. _V4L2-AUDIO-FMT-S16-LE:
> > +
> > + - ``V4L2_AUDIO_FMT_S16_LE``
> > + - 'S16_LE'
> > + - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA
> > +* .. _V4L2-AUDIO-FMT-U16-LE:
> > +
> > + - ``V4L2_AUDIO_FMT_U16_LE``
> > + - 'U16_LE'
> > + - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA
> > +* .. _V4L2-AUDIO-FMT-S24-LE:
> > +
> > + - ``V4L2_AUDIO_FMT_S24_LE``
> > + - 'S24_LE'
> > + - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA
> > +* .. _V4L2-AUDIO-FMT-U24-LE:
> > +
> > + - ``V4L2_AUDIO_FMT_U24_LE``
> > + - 'U24_LE'
> > + - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA
> > +* .. _V4L2-AUDIO-FMT-S32-LE:
> > +
> > + - ``V4L2_AUDIO_FMT_S32_LE``
> > + - 'S32_LE'
> > + - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA
> > +* .. _V4L2-AUDIO-FMT-U32-LE:
> > +
> > + - ``V4L2_AUDIO_FMT_U32_LE``
> > + - 'U32_LE'
> > + - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA
> > +* .. _V4L2-AUDIO-FMT-FLOAT-LE:
> > +
> > + - ``V4L2_AUDIO_FMT_FLOAT_LE``
> > +
Re: [PATCH v13 09/16] media: uapi: Define audio sample format fourcc type
On Mon, Feb 26, 2024 at 9:55 PM Nicolas Dufresne wrote:
>
> Le lundi 26 février 2024 à 16:28 +0800, Shengjiu Wang a écrit :
> > The audio sample format definition is from alsa,
> > the header file is include/uapi/sound/asound.h, but
> > don't include this header file directly, because in
> > user space, there is another copy in alsa-lib.
> > There will be conflict in userspace for include
> > videodev2.h & asound.h and asoundlib.h
> >
> > Here still use the fourcc format.
>
> I'd like to join Mauro's voice that duplicating the audio formats is a bad
> idea.
> We have the same issues with video formats when you look at V4L2 vs DRM.
> You're
> rationale is that videodev2.h will be ambiguous if it includes asound.h, but
> looking at this change, there is no reason why you need to include asound.h in
> videodev2.h at all. The format type can be abstracted out with a uint32 in the
> API, and then it would be up to the users to include and use the appropriate
> formats IDs.
>
Resend for the plain text issue
Thanks.
There is another reason mentioned by Hans:
"
The problem is that within V4L2 we use fourcc consistently to describe a
format, including in VIDIOC_ENUM_FMT. And the expectation is that the fourcc
can be printed to a human readable string (there is even a printk format for
that these days).
But the pcm values are all small integers (and can even be 0!), and
printing the fourcc will give garbage. It doesn't work well at all
with the V4L2 API. But by having a straightforward conversion between the
pcm identifier and a fourcc it was really easy to deal with this.
There might even be applications today that call VIDIOC_ENUM_FMT to see
what is supported and fail if it is not a proper fourcc is returned.
It will certainly report nonsense in v4l_print_fmtdesc() (v4l2-ioctl.c).
One of the early versions of this patch series did precisely what you request,
but it just doesn't work well within the V4L2 uAPI.
"
Best regards
Shengjiu Wang
> Nicolas
>
> >
> > Signed-off-by: Shengjiu Wang
> > ---
> > .../userspace-api/media/v4l/pixfmt-audio.rst | 87 +++
> > .../userspace-api/media/v4l/pixfmt.rst| 1 +
> > drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++
> > include/uapi/linux/videodev2.h| 23 +
> > 4 files changed, 124 insertions(+)
> > create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >
> > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> > b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> > new file mode 100644
> > index ..04b4a7fbd8f4
> > --- /dev/null
> > +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> > @@ -0,0 +1,87 @@
> > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> > +
> > +.. _pixfmt-audio:
> > +
> > +*
> > +Audio Formats
> > +*
> > +
> > +These formats are used for :ref:`audiomem2mem` interface only.
> > +
> > +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}|
> > +
> > +.. cssclass:: longtable
> > +
> > +.. flat-table:: Audio Format
> > +:header-rows: 1
> > +:stub-columns: 0
> > +:widths: 3 1 4
> > +
> > +* - Identifier
> > + - Code
> > + - Details
> > +* .. _V4L2-AUDIO-FMT-S8:
> > +
> > + - ``V4L2_AUDIO_FMT_S8``
> > + - 'S8'
> > + - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA
> > +* .. _V4L2-AUDIO-FMT-S16-LE:
> > +
> > + - ``V4L2_AUDIO_FMT_S16_LE``
> > + - 'S16_LE'
> > + - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA
> > +* .. _V4L2-AUDIO-FMT-U16-LE:
> > +
> > + - ``V4L2_AUDIO_FMT_U16_LE``
> > + - 'U16_LE'
> > + - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA
> > +* .. _V4L2-AUDIO-FMT-S24-LE:
> > +
> > + - ``V4L2_AUDIO_FMT_S24_LE``
> > + - 'S24_LE'
> > + - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA
> > +* .. _V4L2-AUDIO-FMT-U24-LE:
> > +
> > + - ``V4L2_AUDIO_FMT_U24_LE``
> > + - 'U24_LE'
> > + - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA
> > +* .. _V4L2-AUDIO-FMT-S32-LE:
> > +
> > + - ``V4L2_AUDIO_FMT_S32_LE``
> > + - 'S32_LE'
> > + - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA
> > +* .. _V4L2-AUDIO-FMT-U32-LE:
> > +
> > + - ``V4L2_AUDIO_FMT_U32_LE``
> > + - 'U32_LE'
> > + - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA
> > +* .. _V4L2-AUDIO-FMT-FLOAT
[PATCH v13 14/16] media: vivid: add fixed point test controls
Add fixed point test controls, one is for Q4.16 format
another one is for Q63 format.
Signed-off-by: Shengjiu Wang
---
drivers/media/test-drivers/vivid/vivid-core.h | 2 ++
.../media/test-drivers/vivid/vivid-ctrls.c| 26 +++
include/media/v4l2-ctrls.h| 6 +
3 files changed, 34 insertions(+)
diff --git a/drivers/media/test-drivers/vivid/vivid-core.h
b/drivers/media/test-drivers/vivid/vivid-core.h
index cfb8e66083f6..f65465191bc9 100644
--- a/drivers/media/test-drivers/vivid/vivid-core.h
+++ b/drivers/media/test-drivers/vivid/vivid-core.h
@@ -222,6 +222,8 @@ struct vivid_dev {
struct v4l2_ctrl*boolean;
struct v4l2_ctrl*int32;
struct v4l2_ctrl*int64;
+ struct v4l2_ctrl*int32_q16;
+ struct v4l2_ctrl*int64_q63;
struct v4l2_ctrl*menu;
struct v4l2_ctrl*string;
struct v4l2_ctrl*bitmask;
diff --git a/drivers/media/test-drivers/vivid/vivid-ctrls.c
b/drivers/media/test-drivers/vivid/vivid-ctrls.c
index f2b20e25a7a4..2444ea95b285 100644
--- a/drivers/media/test-drivers/vivid/vivid-ctrls.c
+++ b/drivers/media/test-drivers/vivid/vivid-ctrls.c
@@ -38,6 +38,8 @@
#define VIVID_CID_U8_PIXEL_ARRAY (VIVID_CID_CUSTOM_BASE + 14)
#define VIVID_CID_S32_ARRAY(VIVID_CID_CUSTOM_BASE + 15)
#define VIVID_CID_S64_ARRAY(VIVID_CID_CUSTOM_BASE + 16)
+#define VIVID_CID_INT_Q4_16(VIVID_CID_CUSTOM_BASE + 17)
+#define VIVID_CID_INT64_Q63(VIVID_CID_CUSTOM_BASE + 18)
#define VIVID_CID_VIVID_BASE (0x00f0 | 0xf000)
#define VIVID_CID_VIVID_CLASS (0x00f0 | 1)
@@ -182,6 +184,28 @@ static const struct v4l2_ctrl_config vivid_ctrl_int64 = {
.step = 1,
};
+static const struct v4l2_ctrl_config vivid_ctrl_int32_q16 = {
+ .ops = _user_gen_ctrl_ops,
+ .id = VIVID_CID_INT_Q4_16,
+ .name = "Integer 32 Bits Q4.16",
+ .type = V4L2_CTRL_TYPE_INTEGER,
+ .min = v4l2_ctrl_fp_compose(-16, 0, 16),
+ .max = v4l2_ctrl_fp_compose(15, 0x, 16),
+ .step = 1,
+ .fraction_bits = 16,
+};
+
+static const struct v4l2_ctrl_config vivid_ctrl_int64_q63 = {
+ .ops = _user_gen_ctrl_ops,
+ .id = VIVID_CID_INT64_Q63,
+ .name = "Integer 64 Bits Q63",
+ .type = V4L2_CTRL_TYPE_INTEGER64,
+ .min = v4l2_ctrl_fp_compose(-1, 0, 63),
+ .max = v4l2_ctrl_fp_compose(0, LLONG_MAX, 63),
+ .step = 1,
+ .fraction_bits = 63,
+};
+
static const struct v4l2_ctrl_config vivid_ctrl_u32_array = {
.ops = _user_gen_ctrl_ops,
.id = VIVID_CID_U32_ARRAY,
@@ -1670,6 +1694,8 @@ int vivid_create_controls(struct vivid_dev *dev, bool
show_ccs_cap,
dev->button = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_button,
NULL);
dev->int32 = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_int32,
NULL);
dev->int64 = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_int64,
NULL);
+ dev->int32_q16 = v4l2_ctrl_new_custom(hdl_user_gen,
_ctrl_int32_q16, NULL);
+ dev->int64_q63 = v4l2_ctrl_new_custom(hdl_user_gen,
_ctrl_int64_q63, NULL);
dev->boolean = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_boolean,
NULL);
dev->menu = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_menu, NULL);
dev->string = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_string,
NULL);
diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
index c35514c5bf88..197d8b67ac13 100644
--- a/include/media/v4l2-ctrls.h
+++ b/include/media/v4l2-ctrls.h
@@ -1593,4 +1593,10 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl);
*/
int v4l2_ctrl_type_op_validate(const struct v4l2_ctrl *ctrl, union
v4l2_ctrl_ptr ptr);
+/*
+ * Fixed point compose helper define. This helper maps to the value
+ * i + f / (1 << fraction_bits).
+ */
+#define v4l2_ctrl_fp_compose(i, f, fraction_bits) (((s64)(i) << fraction_bits)
+ (f))
+
#endif
--
2.34.1
[PATCH v13 09/16] media: uapi: Define audio sample format fourcc type
The audio sample format definition is from alsa,
the header file is include/uapi/sound/asound.h, but
don't include this header file directly, because in
user space, there is another copy in alsa-lib.
There will be conflict in userspace for include
videodev2.h & asound.h and asoundlib.h
Here still use the fourcc format.
Signed-off-by: Shengjiu Wang
---
.../userspace-api/media/v4l/pixfmt-audio.rst | 87 +++
.../userspace-api/media/v4l/pixfmt.rst| 1 +
drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++
include/uapi/linux/videodev2.h| 23 +
4 files changed, 124 insertions(+)
create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
new file mode 100644
index ..04b4a7fbd8f4
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
@@ -0,0 +1,87 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+
+.. _pixfmt-audio:
+
+*
+Audio Formats
+*
+
+These formats are used for :ref:`audiomem2mem` interface only.
+
+.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}|
+
+.. cssclass:: longtable
+
+.. flat-table:: Audio Format
+:header-rows: 1
+:stub-columns: 0
+:widths: 3 1 4
+
+* - Identifier
+ - Code
+ - Details
+* .. _V4L2-AUDIO-FMT-S8:
+
+ - ``V4L2_AUDIO_FMT_S8``
+ - 'S8'
+ - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA
+* .. _V4L2-AUDIO-FMT-S16-LE:
+
+ - ``V4L2_AUDIO_FMT_S16_LE``
+ - 'S16_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA
+* .. _V4L2-AUDIO-FMT-U16-LE:
+
+ - ``V4L2_AUDIO_FMT_U16_LE``
+ - 'U16_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA
+* .. _V4L2-AUDIO-FMT-S24-LE:
+
+ - ``V4L2_AUDIO_FMT_S24_LE``
+ - 'S24_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA
+* .. _V4L2-AUDIO-FMT-U24-LE:
+
+ - ``V4L2_AUDIO_FMT_U24_LE``
+ - 'U24_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA
+* .. _V4L2-AUDIO-FMT-S32-LE:
+
+ - ``V4L2_AUDIO_FMT_S32_LE``
+ - 'S32_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA
+* .. _V4L2-AUDIO-FMT-U32-LE:
+
+ - ``V4L2_AUDIO_FMT_U32_LE``
+ - 'U32_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA
+* .. _V4L2-AUDIO-FMT-FLOAT-LE:
+
+ - ``V4L2_AUDIO_FMT_FLOAT_LE``
+ - 'FLOAT_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_FLOAT_LE in ALSA
+* .. _V4L2-AUDIO-FMT-IEC958-SUBFRAME-LE:
+
+ - ``V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE``
+ - 'IEC958_SUBFRAME_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_IEC958_SUBFRAME_LE in ALSA
+* .. _V4L2-AUDIO-FMT-S24-3LE:
+
+ - ``V4L2_AUDIO_FMT_S24_3LE``
+ - 'S24_3LE'
+ - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
+* .. _V4L2-AUDIO-FMT-U24-3LE:
+
+ - ``V4L2_AUDIO_FMT_U24_3LE``
+ - 'U24_3LE'
+ - Corresponds to SNDRV_PCM_FORMAT_U24_3LE in ALSA
+* .. _V4L2-AUDIO-FMT-S20-3LE:
+
+ - ``V4L2_AUDIO_FMT_S20_3LE``
+ - 'S20_3LE'
+ - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
+* .. _V4L2-AUDIO-FMT-U20-3LE:
+
+ - ``V4L2_AUDIO_FMT_U20_3LE``
+ - 'U20_3LE'
+ - Corresponds to SNDRV_PCM_FORMAT_U20_3LE in ALSA
diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst
b/Documentation/userspace-api/media/v4l/pixfmt.rst
index 11dab4a90630..2eb6fdd3b43d 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt.rst
@@ -36,3 +36,4 @@ see also :ref:`VIDIOC_G_FBUF `.)
colorspaces
colorspaces-defs
colorspaces-details
+pixfmt-audio
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
b/drivers/media/v4l2-core/v4l2-ioctl.c
index 961abcdf7290..be229c69e991 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1471,6 +1471,19 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_Y210: descr = "10-bit YUYV Packed"; break;
case V4L2_PIX_FMT_Y212: descr = "12-bit YUYV Packed"; break;
case V4L2_PIX_FMT_Y216: descr = "16-bit YUYV Packed"; break;
+ case V4L2_AUDIO_FMT_S8: descr = "8-bit Signed"; break;
+ case V4L2_AUDIO_FMT_S16_LE: descr = "16-bit Signed LE"; break;
+ case V4L2_AUDIO_FMT_U16_LE: descr = "16-bit Unsigned LE";
break;
+ case V4L2_AUDIO_FMT_S24_LE: descr = "24(32)-bit Signed LE";
break;
+ case V4L2_AUDIO_FMT_U24_LE: descr = "24(32)-bit Unsigned
LE"; break;
+ case V4L2_AUDIO_FMT_S32_LE: descr = "32-bit Signed LE";
break;
+ case V4L2_AUDIO_FMT_U32_LE: descr = "32-bit Unsigned LE";
break;
+ case V4L2_AUDIO_FMT_FLOAT_LE:
[PATCH v13 16/16] media: vim2m-audio: add virtual driver for audio memory to memory
Audio memory to memory virtual driver use video memory to memory
virtual driver vim2m.c as example. The main difference is
device type is VFL_TYPE_AUDIO and device cap type is V4L2_CAP_AUDIO_M2M.
The device_run function is a dummy function, which is simply
copy the data from input buffer to output buffer.
Signed-off-by: Shengjiu Wang
---
drivers/media/test-drivers/Kconfig | 10 +
drivers/media/test-drivers/Makefile | 1 +
drivers/media/test-drivers/vim2m-audio.c | 793 +++
3 files changed, 804 insertions(+)
create mode 100644 drivers/media/test-drivers/vim2m-audio.c
diff --git a/drivers/media/test-drivers/Kconfig
b/drivers/media/test-drivers/Kconfig
index 5a5379524bde..b6b52a7ca042 100644
--- a/drivers/media/test-drivers/Kconfig
+++ b/drivers/media/test-drivers/Kconfig
@@ -16,6 +16,16 @@ config VIDEO_VIM2M
This is a virtual test device for the memory-to-memory driver
framework.
+config VIDEO_VIM2M_AUDIO
+ tristate "Virtual Memory-to-Memory Driver For Audio"
+ depends on VIDEO_DEV
+ select VIDEOBUF2_VMALLOC
+ select V4L2_MEM2MEM_DEV
+ select MEDIA_CONTROLLER
+ help
+ This is a virtual audio test device for the memory-to-memory driver
+ framework.
+
source "drivers/media/test-drivers/vicodec/Kconfig"
source "drivers/media/test-drivers/vimc/Kconfig"
source "drivers/media/test-drivers/vivid/Kconfig"
diff --git a/drivers/media/test-drivers/Makefile
b/drivers/media/test-drivers/Makefile
index 740714a4584d..0c61c9ada3e1 100644
--- a/drivers/media/test-drivers/Makefile
+++ b/drivers/media/test-drivers/Makefile
@@ -10,6 +10,7 @@ obj-$(CONFIG_DVB_VIDTV) += vidtv/
obj-$(CONFIG_VIDEO_VICODEC) += vicodec/
obj-$(CONFIG_VIDEO_VIM2M) += vim2m.o
+obj-$(CONFIG_VIDEO_VIM2M_AUDIO) += vim2m-audio.o
obj-$(CONFIG_VIDEO_VIMC) += vimc/
obj-$(CONFIG_VIDEO_VIVID) += vivid/
obj-$(CONFIG_VIDEO_VISL) += visl/
diff --git a/drivers/media/test-drivers/vim2m-audio.c
b/drivers/media/test-drivers/vim2m-audio.c
new file mode 100644
index ..6361df6320b3
--- /dev/null
+++ b/drivers/media/test-drivers/vim2m-audio.c
@@ -0,0 +1,793 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * A virtual v4l2-mem2mem example for audio device.
+ */
+
+#include
+#include
+#include
+#include
+#include
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+MODULE_DESCRIPTION("Virtual device for audio mem2mem testing");
+MODULE_LICENSE("GPL");
+
+static unsigned int debug;
+module_param(debug, uint, 0644);
+MODULE_PARM_DESC(debug, "debug level");
+
+#define MEM2MEM_NAME "vim2m-audio"
+
+#define dprintk(dev, lvl, fmt, arg...) \
+ v4l2_dbg(lvl, debug, &(dev)->v4l2_dev, "%s: " fmt, __func__, ## arg)
+
+#define SAMPLE_NUM 4096
+
+static void audm2m_dev_release(struct device *dev)
+{}
+
+static struct platform_device audm2m_pdev = {
+ .name = MEM2MEM_NAME,
+ .dev.release= audm2m_dev_release,
+};
+
+static u32 formats[] = {
+ V4L2_AUDIO_FMT_S16_LE,
+};
+
+#define NUM_FORMATS ARRAY_SIZE(formats)
+
+/* Per-queue, driver-specific private data */
+struct audm2m_q_data {
+ unsigned intrate;
+ unsigned intchannels;
+ unsigned intbuffersize;
+ unsigned intsequence;
+ u32 fourcc;
+};
+
+enum {
+ V4L2_M2M_SRC = 0,
+ V4L2_M2M_DST = 1,
+};
+
+static snd_pcm_format_t find_format(u32 fourcc)
+{
+ snd_pcm_format_t fmt;
+ unsigned int k;
+
+ for (k = 0; k < NUM_FORMATS; k++) {
+ if (formats[k] == fourcc)
+ break;
+ }
+
+ if (k == NUM_FORMATS)
+ return 0;
+
+ fmt = v4l2_fourcc_to_audfmt(formats[k]);
+
+ return fmt;
+}
+
+struct audm2m_dev {
+ struct v4l2_device v4l2_dev;
+ struct video_device vfd;
+
+ struct mutexdev_mutex;
+
+ struct v4l2_m2m_dev *m2m_dev;
+#ifdef CONFIG_MEDIA_CONTROLLER
+ struct media_device mdev;
+#endif
+};
+
+struct audm2m_ctx {
+ struct v4l2_fh fh;
+ struct v4l2_ctrl_handlerctrl_handler;
+ struct audm2m_dev *dev;
+
+ struct mutexvb_mutex;
+
+ /* Source and destination queue data */
+ struct audm2m_q_data q_data[2];
+};
+
+static inline struct audm2m_ctx *file2ctx(struct file *file)
+{
+ return container_of(file->private_data, struct audm2m_ctx, fh);
+}
+
+static struct audm2m_q_data *get_q_data(struct audm2m_ctx *ctx,
+ enum v4l2_buf_type type)
+{
+ if (type == V4L2_BUF_TYPE_AUDIO_OUTPUT)
+ return >q_data[V4L2_M2M_SRC];
+ return >q_data[V4L2_M2M_DST];
+}
+
+static const char *type_name(enum v4l2_buf_type type)
+{
+ if (type == V4L2_BUF
[PATCH v13 15/16] media: imx-asrc: Add memory to memory driver
Implement the ASRC memory to memory function using
the v4l2 framework, user can use this function with
v4l2 ioctl interface.
User send the output and capture buffer to driver and
driver store the converted data to the capture buffer.
This feature can be shared by ASRC and EASRC drivers
Signed-off-by: Shengjiu Wang
---
drivers/media/platform/nxp/Kconfig| 13 +
drivers/media/platform/nxp/Makefile |1 +
drivers/media/platform/nxp/imx-asrc.c | 1256 +
3 files changed, 1270 insertions(+)
create mode 100644 drivers/media/platform/nxp/imx-asrc.c
diff --git a/drivers/media/platform/nxp/Kconfig
b/drivers/media/platform/nxp/Kconfig
index 40e3436669e2..8d0ca335601f 100644
--- a/drivers/media/platform/nxp/Kconfig
+++ b/drivers/media/platform/nxp/Kconfig
@@ -67,3 +67,16 @@ config VIDEO_MX2_EMMAPRP
source "drivers/media/platform/nxp/dw100/Kconfig"
source "drivers/media/platform/nxp/imx-jpeg/Kconfig"
+
+config VIDEO_IMX_ASRC
+ tristate "NXP i.MX ASRC M2M support"
+ depends on V4L_MEM2MEM_DRIVERS
+ depends on MEDIA_SUPPORT
+ select VIDEOBUF2_DMA_CONTIG
+ select V4L2_MEM2MEM_DEV
+ select MEDIA_CONTROLLER
+ help
+ Say Y if you want to add ASRC M2M support for NXP CPUs.
+ It is a complement for ASRC M2P and ASRC P2M features.
+ This option is only useful for out-of-tree drivers since
+ in-tree drivers select it automatically.
diff --git a/drivers/media/platform/nxp/Makefile
b/drivers/media/platform/nxp/Makefile
index 4d90eb713652..1325675e34f5 100644
--- a/drivers/media/platform/nxp/Makefile
+++ b/drivers/media/platform/nxp/Makefile
@@ -9,3 +9,4 @@ obj-$(CONFIG_VIDEO_IMX8MQ_MIPI_CSI2) += imx8mq-mipi-csi2.o
obj-$(CONFIG_VIDEO_IMX_MIPI_CSIS) += imx-mipi-csis.o
obj-$(CONFIG_VIDEO_IMX_PXP) += imx-pxp.o
obj-$(CONFIG_VIDEO_MX2_EMMAPRP) += mx2_emmaprp.o
+obj-$(CONFIG_VIDEO_IMX_ASRC) += imx-asrc.o
diff --git a/drivers/media/platform/nxp/imx-asrc.c
b/drivers/media/platform/nxp/imx-asrc.c
new file mode 100644
index ..0c25a36199b1
--- /dev/null
+++ b/drivers/media/platform/nxp/imx-asrc.c
@@ -0,0 +1,1256 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (C) 2014-2016 Freescale Semiconductor, Inc.
+// Copyright (C) 2019-2023 NXP
+//
+// Freescale ASRC Memory to Memory (M2M) driver
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#define V4L_CAP OUT
+#define V4L_OUT IN
+
+#define ASRC_xPUT_DMA_CALLBACK(dir) \
+ (((dir) == V4L_OUT) ? asrc_input_dma_callback \
+ : asrc_output_dma_callback)
+
+#define DIR_STR(dir) (dir) == V4L_OUT ? "out" : "cap"
+
+/* Maximum output and capture buffer size */
+#define ASRC_M2M_BUFFER_SIZE (512 * 1024)
+
+/* Maximum output and capture period size */
+#define ASRC_M2M_PERIOD_SIZE (48 * 1024)
+
+struct asrc_pair_m2m {
+ struct fsl_asrc_pair *pair;
+ struct asrc_m2m *m2m;
+ struct v4l2_fh fh;
+ struct v4l2_ctrl_handler ctrl_handler;
+ int channels[2];
+ unsigned int sequence[2];
+ s64 src_rate_off_prev; /* Q31.32 */
+ s64 dst_rate_off_prev; /* Q31.32 */
+ s64 src_rate_off_cur; /* Q31.32 */
+ s64 dst_rate_off_cur; /* Q31.32 */
+};
+
+struct asrc_m2m {
+ struct fsl_asrc_m2m_pdata pdata;
+ struct v4l2_device v4l2_dev;
+ struct v4l2_m2m_dev *m2m_dev;
+ struct video_device *dec_vdev;
+ struct mutex mlock; /* v4l2 ioctls serialization */
+ struct platform_device *pdev;
+#ifdef CONFIG_MEDIA_CONTROLLER
+ struct media_device mdev;
+#endif
+};
+
+static u32 formats[] = {
+ V4L2_AUDIO_FMT_S8,
+ V4L2_AUDIO_FMT_S16_LE,
+ V4L2_AUDIO_FMT_U16_LE,
+ V4L2_AUDIO_FMT_S24_LE,
+ V4L2_AUDIO_FMT_S24_3LE,
+ V4L2_AUDIO_FMT_U24_LE,
+ V4L2_AUDIO_FMT_U24_3LE,
+ V4L2_AUDIO_FMT_S32_LE,
+ V4L2_AUDIO_FMT_U32_LE,
+ V4L2_AUDIO_FMT_S20_3LE,
+ V4L2_AUDIO_FMT_U20_3LE,
+ V4L2_AUDIO_FMT_FLOAT_LE,
+ V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE,
+};
+
+#define NUM_FORMATS ARRAY_SIZE(formats)
+
+static const s64 asrc_v1_m2m_rates[] = {
+ 5512, 8000, 11025, 12000, 16000,
+ 22050, 24000, 32000, 44100,
+ 48000, 64000, 88200, 96000,
+ 128000, 176400, 192000,
+};
+
+static const s64 asrc_v2_m2m_rates[] = {
+ 8000, 11025, 12000, 16000,
+ 22050, 24000, 32000, 44100,
+ 48000, 64000, 88200, 96000,
+ 128000, 176400, 192000, 256000,
+ 352800, 384000, 705600, 768000,
+};
+
+static u32 find_fourcc(snd_pcm_format_t format)
+{
+ snd_pcm_format_t fmt;
+ unsigned int k;
+
+ for (k = 0; k < NUM_FORMATS; k++) {
+ fmt = v4l2_fourcc_to_audfmt(formats[k]);
+ if (fmt == format)
+ return formats[k];
+ }
+
+ return 0;
+}
+
+static snd_pcm_forma
[PATCH v13 00/16] Add audio support in v4l2 framework
Audio signal processing also has the requirement for memory to memory similar as Video. This asrc memory to memory (memory ->asrc->memory) case is a non real time use case. User fills the input buffer to the asrc module, after conversion, then asrc sends back the output buffer to user. So it is not a traditional ALSA playback and capture case. It is a specific use case, there is no reference in current kernel. v4l2 memory to memory is the closed implementation, v4l2 current support video, image, radio, tuner, touch devices, so it is not complicated to add support for this specific audio case. Because we had implemented the "memory -> asrc ->i2s device-> codec" use case in ALSA. Now the "memory->asrc->memory" needs to reuse the code in asrc driver, so the first 3 patches is for refining the code to make it can be shared by the "memory->asrc->memory" driver. The main change is in the v4l2 side, A /dev/vl4-audioX will be created, user applications only use the ioctl of v4l2 framework. Other change is to add memory to memory support for two kinds of i.MX ASRC module. changes in v13 - change 'pixelformat' to 'audioformat' in dev-audio-mem2mem.rst - add more description for clock drift in ext-ctrls-audio-m2m.rst - Add "media: v4l2-ctrls: add support for fraction_bits" from Hans to avoid build issue for kernel test robot changes in v12 - minor changes according to comments - drop min_buffers_needed = 1 and V4L2_CTRL_FLAG_UPDATE flag - drop bus_info changes in v11 - add add-fixed-point-test-controls in vivid. - add v4l2_ctrl_fp_compose() helper function for min and max changes in v10 - remove FIXED_POINT type - change code base on media: v4l2-ctrls: add support for fraction_bits - fix issue reported by kernel test robot - remove module_alias changes in v9: - add MEDIA_ENT_F_PROC_AUDIO_RESAMPLER. - add MEDIA_INTF_T_V4L_AUDIO - add media controller support - refine the vim2m-audio to support 8k<->16k conversion. changes in v8: - refine V4L2_CAP_AUDIO_M2M to be 0x0008 - update doc for FIXED_POINT - address comments for imx-asrc changes in v7: - add acked-by from Mark - separate commit for fixed point, m2m audio class, audio rate controls - use INTEGER_MENU for rate, FIXED_POINT for rate offset - remove used fmts - address other comments for Hans changes in v6: - use m2m_prepare/m2m_unprepare/m2m_start/m2m_stop to replace m2m_start_part_one/m2m_stop_part_one, m2m_start_part_two/m2m_stop_part_two. - change V4L2_CTRL_TYPE_ASRC_RATE to V4L2_CTRL_TYPE_FIXED_POINT - fix warning by kernel test rebot - remove some unused format V4L2_AUDIO_FMT_XX - Get SNDRV_PCM_FORMAT from V4L2_AUDIO_FMT in driver. - rename audm2m to viaudm2m. changes in v5: - remove V4L2_AUDIO_FMT_LPCM - define audio pixel format like V4L2_AUDIO_FMT_S8... - remove rate and format in struct v4l2_audio_format. - Add V4L2_CID_ASRC_SOURCE_RATE and V4L2_CID_ASRC_DEST_RATE controls - updata document accordingly. changes in v4: - update document style - separate V4L2_AUDIO_FMT_LPCM and V4L2_CAP_AUDIO_M2M in separate commit changes in v3: - Modify documents for adding audio m2m support - Add audio virtual m2m driver - Defined V4L2_AUDIO_FMT_LPCM format type for audio. - Defined V4L2_CAP_AUDIO_M2M capability type for audio m2m case. - with modification in v4l-utils, pass v4l2-compliance test. changes in v2: - decouple the implementation in v4l2 and ALSA - implement the memory to memory driver as a platfrom driver and move it to driver/media - move fsl_asrc_common.h to include/sound folder Hans Verkuil (1): media: v4l2-ctrls: add support for fraction_bits Shengjiu Wang (15): ASoC: fsl_asrc: define functions for memory to memory usage ASoC: fsl_easrc: define functions for memory to memory usage ASoC: fsl_asrc: move fsl_asrc_common.h to include/sound ASoC: fsl_asrc: register m2m platform device ASoC: fsl_easrc: register m2m platform device media: uapi: Add V4L2_CAP_AUDIO_M2M capability flag media: v4l2: Add audio capture and output support media: uapi: Define audio sample format fourcc type media: uapi: Add V4L2_CTRL_CLASS_M2M_AUDIO media: uapi: Add audio rate controls support media: uapi: Declare interface types for Audio media: uapi: Add an entity type for audio resampler media: vivid: add fixed point test controls media: imx-asrc: Add memory to memory driver media: vim2m-audio: add virtual driver for audio memory to memory .../media/mediactl/media-types.rst| 11 + .../userspace-api/media/v4l/buffer.rst|6 + .../userspace-api/media/v4l/common.rst|1 + .../media/v4l/dev-audio-mem2mem.rst | 71 + .../userspace-api/media/v4l/devices.rst |1 + .../media/v4l/ext-ctrls-audio-m2m.rst | 59 + .../userspace-api/media/v4l/pixfmt-audio.rst | 87 ++ .../userspace-api/media/v4l/pixfmt.rst|1 + .../media/v4l/vidioc-enum-fmt.rst |2 + .../me
[PATCH v13 04/16] ASoC: fsl_asrc: move fsl_asrc_common.h to include/sound
Move fsl_asrc_common.h to include/sound that it can be
included from other drivers.
Signed-off-by: Shengjiu Wang
Acked-by: Mark Brown
---
{sound/soc/fsl => include/sound}/fsl_asrc_common.h | 0
sound/soc/fsl/fsl_asrc.h | 2 +-
sound/soc/fsl/fsl_asrc_dma.c | 2 +-
sound/soc/fsl/fsl_easrc.h | 2 +-
4 files changed, 3 insertions(+), 3 deletions(-)
rename {sound/soc/fsl => include/sound}/fsl_asrc_common.h (100%)
diff --git a/sound/soc/fsl/fsl_asrc_common.h b/include/sound/fsl_asrc_common.h
similarity index 100%
rename from sound/soc/fsl/fsl_asrc_common.h
rename to include/sound/fsl_asrc_common.h
diff --git a/sound/soc/fsl/fsl_asrc.h b/sound/soc/fsl/fsl_asrc.h
index 1c492eb237f5..66544624de7b 100644
--- a/sound/soc/fsl/fsl_asrc.h
+++ b/sound/soc/fsl/fsl_asrc.h
@@ -10,7 +10,7 @@
#ifndef _FSL_ASRC_H
#define _FSL_ASRC_H
-#include "fsl_asrc_common.h"
+#include
#define ASRC_M2M_INPUTFIFO_WML 0x4
#define ASRC_M2M_OUTPUTFIFO_WML0x2
diff --git a/sound/soc/fsl/fsl_asrc_dma.c b/sound/soc/fsl/fsl_asrc_dma.c
index f501f47242fb..f067bf1ecea7 100644
--- a/sound/soc/fsl/fsl_asrc_dma.c
+++ b/sound/soc/fsl/fsl_asrc_dma.c
@@ -12,7 +12,7 @@
#include
#include
-#include "fsl_asrc_common.h"
+#include
#define FSL_ASRC_DMABUF_SIZE (256 * 1024)
diff --git a/sound/soc/fsl/fsl_easrc.h b/sound/soc/fsl/fsl_easrc.h
index c9f770862662..a24e540876a4 100644
--- a/sound/soc/fsl/fsl_easrc.h
+++ b/sound/soc/fsl/fsl_easrc.h
@@ -9,7 +9,7 @@
#include
#include
-#include "fsl_asrc_common.h"
+#include
/* EASRC Register Map */
--
2.34.1
[PATCH v13 13/16] media: uapi: Add an entity type for audio resampler
Add and document a media entity type for an audio resampler.
It is MEDIA_ENT_F_PROC_AUDIO_RESAMPLER.
Signed-off-by: Shengjiu Wang
---
Documentation/userspace-api/media/mediactl/media-types.rst | 6 ++
include/uapi/linux/media.h | 1 +
2 files changed, 7 insertions(+)
diff --git a/Documentation/userspace-api/media/mediactl/media-types.rst
b/Documentation/userspace-api/media/mediactl/media-types.rst
index adfb37430f8e..d353f17c3344 100644
--- a/Documentation/userspace-api/media/mediactl/media-types.rst
+++ b/Documentation/userspace-api/media/mediactl/media-types.rst
@@ -40,6 +40,7 @@ Types and flags used to represent the media graph elements
.. _MEDIA-ENT-F-PROC-VIDEO-ENCODER:
.. _MEDIA-ENT-F-PROC-VIDEO-DECODER:
.. _MEDIA-ENT-F-PROC-VIDEO-ISP:
+.. _MEDIA-ENT-F-PROC-AUDIO-RESAMPLER:
.. _MEDIA-ENT-F-VID-MUX:
.. _MEDIA-ENT-F-VID-IF-BRIDGE:
.. _MEDIA-ENT-F-DV-DECODER:
@@ -208,6 +209,11 @@ Types and flags used to represent the media graph elements
combination of custom V4L2 controls and IOCTLs, and parameters
supplied in a metadata buffer.
+* - ``MEDIA_ENT_F_PROC_AUDIO_RESAMPLER``
+ - An Audio Resampler device. An entity capable of
+ resampling an audio stream from one sample rate to another sample
+ rate. Must have one sink pad and at least one source pad.
+
* - ``MEDIA_ENT_F_VID_MUX``
- Video multiplexer. An entity capable of multiplexing must have at
least two sink pads and one source pad, and must pass the video
diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
index 9ff6dec7393a..a8266eaa8042 100644
--- a/include/uapi/linux/media.h
+++ b/include/uapi/linux/media.h
@@ -125,6 +125,7 @@ struct media_device_info {
#define MEDIA_ENT_F_PROC_VIDEO_ENCODER (MEDIA_ENT_F_BASE + 0x4007)
#define MEDIA_ENT_F_PROC_VIDEO_DECODER (MEDIA_ENT_F_BASE + 0x4008)
#define MEDIA_ENT_F_PROC_VIDEO_ISP (MEDIA_ENT_F_BASE + 0x4009)
+#define MEDIA_ENT_F_PROC_AUDIO_RESAMPLER (MEDIA_ENT_F_BASE + 0x400a)
/*
* Switch and bridge entity functions
--
2.34.1
[PATCH v13 08/16] media: v4l2: Add audio capture and output support
Audio signal processing has the requirement for memory to
memory similar as Video.
This patch is to add this support in v4l2 framework, defined
new buffer type V4L2_BUF_TYPE_AUDIO_CAPTURE and
V4L2_BUF_TYPE_AUDIO_OUTPUT, defined new format v4l2_audio_format
for audio case usage.
The created audio device is named "/dev/v4l-audioX".
Signed-off-by: Shengjiu Wang
---
.../userspace-api/media/v4l/buffer.rst| 6 ++
.../media/v4l/dev-audio-mem2mem.rst | 71 +++
.../userspace-api/media/v4l/devices.rst | 1 +
.../media/v4l/vidioc-enum-fmt.rst | 2 +
.../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 ++
.../media/videodev2.h.rst.exceptions | 2 +
.../media/common/videobuf2/videobuf2-v4l2.c | 4 ++
drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 9 +++
drivers/media/v4l2-core/v4l2-dev.c| 17 +
drivers/media/v4l2-core/v4l2-ioctl.c | 53 ++
include/media/v4l2-dev.h | 2 +
include/media/v4l2-ioctl.h| 34 +
include/uapi/linux/videodev2.h| 17 +
13 files changed, 222 insertions(+)
create mode 100644 Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
diff --git a/Documentation/userspace-api/media/v4l/buffer.rst
b/Documentation/userspace-api/media/v4l/buffer.rst
index 52bbee81c080..a3754ca6f0d6 100644
--- a/Documentation/userspace-api/media/v4l/buffer.rst
+++ b/Documentation/userspace-api/media/v4l/buffer.rst
@@ -438,6 +438,12 @@ enum v4l2_buf_type
* - ``V4L2_BUF_TYPE_META_OUTPUT``
- 14
- Buffer for metadata output, see :ref:`metadata`.
+* - ``V4L2_BUF_TYPE_AUDIO_CAPTURE``
+ - 15
+ - Buffer for audio capture, see :ref:`audio`.
+* - ``V4L2_BUF_TYPE_AUDIO_OUTPUT``
+ - 16
+ - Buffer for audio output, see :ref:`audio`.
.. _buffer-flags:
diff --git a/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
new file mode 100644
index ..54cc2abb6c04
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
@@ -0,0 +1,71 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+
+.. _audiomem2mem:
+
+
+Audio Memory-To-Memory Interface
+
+
+An audio memory-to-memory device can compress, decompress, transform, or
+otherwise convert audio data from one format into another format, in memory.
+Such memory-to-memory devices set the ``V4L2_CAP_AUDIO_M2M`` capability.
+Examples of memory-to-memory devices are audio codecs, audio preprocessing,
+audio postprocessing.
+
+A memory-to-memory audio node supports both output (sending audio frames from
+memory to the hardware) and capture (receiving the processed audio frames
+from the hardware into memory) stream I/O. An application will have to
+setup the stream I/O for both sides and finally call
+:ref:`VIDIOC_STREAMON ` for both capture and output to
+start the hardware.
+
+Memory-to-memory devices function as a shared resource: you can
+open the audio node multiple times, each application setting up their
+own properties that are local to the file handle, and each can use
+it independently from the others. The driver will arbitrate access to
+the hardware and reprogram it whenever another file handler gets access.
+
+Audio memory-to-memory devices are accessed through character device
+special files named ``/dev/v4l-audio``
+
+Querying Capabilities
+=
+
+Device nodes supporting the audio memory-to-memory interface set the
+``V4L2_CAP_AUDIO_M2M`` flag in the ``device_caps`` field of the
+:c:type:`v4l2_capability` structure returned by the :c:func:`VIDIOC_QUERYCAP`
+ioctl.
+
+Data Format Negotiation
+===
+
+The audio device uses the :ref:`format` ioctls to select the capture format.
+The audio buffer content format is bound to that selected format. In addition
+to the basic :ref:`format` ioctls, the :c:func:`VIDIOC_ENUM_FMT` ioctl must be
+supported as well.
+
+To use the :ref:`format` ioctls applications set the ``type`` field of the
+:c:type:`v4l2_format` structure to ``V4L2_BUF_TYPE_AUDIO_CAPTURE`` or to
+``V4L2_BUF_TYPE_AUDIO_OUTPUT``. Both drivers and applications must set the
+remainder of the :c:type:`v4l2_format` structure to 0.
+
+.. c:type:: v4l2_audio_format
+
+.. tabularcolumns:: |p{1.4cm}|p{2.4cm}|p{13.5cm}|
+
+.. flat-table:: struct v4l2_audio_format
+:header-rows: 0
+:stub-columns: 0
+:widths: 1 1 2
+
+* - __u32
+ - ``audioformat``
+ - The sample format, set by the application. see :ref:`pixfmt-audio`
+* - __u32
+ - ``channels``
+ - The channel number, set by the application. channel number range is
+[1, 32].
+* - __u32
+ - ``buffersize``
+ - Maximum buffer size in bytes required for data. The value is set by the
+driver.
diff --git a/Doc
[PATCH v13 10/16] media: uapi: Add V4L2_CTRL_CLASS_M2M_AUDIO
The Audio M2M class includes controls for audio memory-to-memory
use cases. The controls can be used for audio codecs, audio
preprocessing, audio postprocessing.
Signed-off-by: Shengjiu Wang
---
.../userspace-api/media/v4l/common.rst| 1 +
.../media/v4l/ext-ctrls-audio-m2m.rst | 21 +++
.../media/v4l/vidioc-g-ext-ctrls.rst | 4
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 4
include/uapi/linux/v4l2-controls.h| 4
5 files changed, 34 insertions(+)
create mode 100644
Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
diff --git a/Documentation/userspace-api/media/v4l/common.rst
b/Documentation/userspace-api/media/v4l/common.rst
index ea0435182e44..d5366e96a596 100644
--- a/Documentation/userspace-api/media/v4l/common.rst
+++ b/Documentation/userspace-api/media/v4l/common.rst
@@ -52,6 +52,7 @@ applicable to all devices.
ext-ctrls-fm-rx
ext-ctrls-detect
ext-ctrls-colorimetry
+ext-ctrls-audio-m2m
fourcc
format
planar-apis
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
new file mode 100644
index ..82d2ecedbfee
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
@@ -0,0 +1,21 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+
+.. _audiom2m-controls:
+
+***
+Audio M2M Control Reference
+***
+
+The Audio M2M class includes controls for audio memory-to-memory
+use cases. The controls can be used for audio codecs, audio
+preprocessing, audio postprocessing.
+
+Audio M2M Control IDs
+---
+
+.. _audiom2m-control-id:
+
+``V4L2_CID_M2M_AUDIO_CLASS (class)``
+The Audio M2M class descriptor. Calling
+:ref:`VIDIOC_QUERYCTRL` for this control will
+return a description of this control class.
diff --git a/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
b/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
index 4d56c0528ad7..aeb1ad8e7d29 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
@@ -488,6 +488,10 @@ still cause this situation.
- 0xa5
- The class containing colorimetry controls. These controls are
described in :ref:`colorimetry-controls`.
+* - ``V4L2_CTRL_CLASS_M2M_AUDIO``
+ - 0xa6
+ - The class containing audio m2m controls. These controls are
+ described in :ref:`audiom2m-controls`.
Return Value
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index 8696eb1cdd61..2a85ea3dc92f 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -1242,6 +1242,9 @@ const char *v4l2_ctrl_get_name(u32 id)
case V4L2_CID_COLORIMETRY_CLASS:return "Colorimetry Controls";
case V4L2_CID_COLORIMETRY_HDR10_CLL_INFO: return "HDR10
Content Light Info";
case V4L2_CID_COLORIMETRY_HDR10_MASTERING_DISPLAY: return "HDR10
Mastering Display";
+
+ /* Audio M2M controls */
+ case V4L2_CID_M2M_AUDIO_CLASS: return "Audio M2M Controls";
default:
return NULL;
}
@@ -1451,6 +1454,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum
v4l2_ctrl_type *type,
case V4L2_CID_DETECT_CLASS:
case V4L2_CID_CODEC_STATELESS_CLASS:
case V4L2_CID_COLORIMETRY_CLASS:
+ case V4L2_CID_M2M_AUDIO_CLASS:
*type = V4L2_CTRL_TYPE_CTRL_CLASS;
/* You can neither read nor write these */
*flags |= V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_WRITE_ONLY;
diff --git a/include/uapi/linux/v4l2-controls.h
b/include/uapi/linux/v4l2-controls.h
index 99c3f5e99da7..a8b4b830c757 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -30,6 +30,7 @@
#define V4L2_CTRL_CLASS_DETECT 0x00a3 /* Detection controls */
#define V4L2_CTRL_CLASS_CODEC_STATELESS 0x00a4 /* Stateless codecs
controls */
#define V4L2_CTRL_CLASS_COLORIMETRY0x00a5 /* Colorimetry controls
*/
+#define V4L2_CTRL_CLASS_M2M_AUDIO 0x00a6 /* Audio M2M controls */
/* User-class control IDs */
@@ -3491,6 +3492,9 @@ struct v4l2_ctrl_av1_film_grain {
__u8 reserved[4];
};
+#define V4L2_CID_M2M_AUDIO_CLASS_BASE (V4L2_CTRL_CLASS_M2M_AUDIO | 0x900)
+#define V4L2_CID_M2M_AUDIO_CLASS (V4L2_CTRL_CLASS_M2M_AUDIO | 1)
+
/* MPEG-compression definitions kept for backwards compatibility */
#ifndef __KERNEL__
#define V4L2_CTRL_CLASS_MPEGV4L2_CTRL_CLASS_CODEC
--
2.34.1
[PATCH v13 11/16] media: uapi: Add audio rate controls support
Add V4L2_CID_M2M_AUDIO_SOURCE_RATE and V4L2_CID_M2M_AUDIO_DEST_RATE
new IDs for rate control.
Add V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET and
V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET for clock drift.
Signed-off-by: Shengjiu Wang
---
.../media/v4l/ext-ctrls-audio-m2m.rst | 38 +++
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 6 +++
include/uapi/linux/v4l2-controls.h| 5 +++
3 files changed, 49 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
index 82d2ecedbfee..b137b7c442e6 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
@@ -19,3 +19,41 @@ Audio M2M Control IDs
The Audio M2M class descriptor. Calling
:ref:`VIDIOC_QUERYCTRL` for this control will
return a description of this control class.
+
+.. _v4l2-audio-asrc:
+
+``V4L2_CID_M2M_AUDIO_SOURCE_RATE (integer menu)``
+This control specifies the audio source sample rate, unit is Hz
+
+``V4L2_CID_M2M_AUDIO_DEST_RATE (integer menu)``
+This control specifies the audio destination sample rate, unit is Hz
+
+``V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET (fixed point)``
+This control specifies the offset from the audio source sample rate,
+unit is Hz.
+
+The offset compensates for any clock drift. The actual source audio
+sample rate is the ideal source audio sample rate from
+``V4L2_CID_M2M_AUDIO_SOURCE_RATE`` plus this fixed point offset.
+
+The audio source clock may have some drift. Reducing or increasing the
+audio sample rate dynamically to ensure that Sample Rate Converter is
+working on the real sample rate, this feature is for the Asynchronous
+Sample Rate Converter module.
+So, userspace would be expected to be monitoring such drift
+and increasing/decreasing the sample frequency as needed by this control.
+
+``V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET (fixed point)``
+This control specifies the offset from the audio destination sample rate,
+unit is Hz.
+
+The offset compensates for any clock drift. The actual destination audio
+sample rate is the ideal source audio sample rate from
+``V4L2_CID_M2M_AUDIO_DEST_RATE`` plus this fixed point offset.
+
+The audio destination clock may have some drift. Reducing or increasing
+the audio sample rate dynamically to ensure that sample rate converter
+is working on the real sample rate, this feature is for the Asynchronous
+Sample Rate Converter module.
+So, userspace would be expected to be monitoring such drift
+and increasing/decreasing the sample frequency as needed by this control.
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index 2a85ea3dc92f..91e1f5348c23 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -1245,6 +1245,8 @@ const char *v4l2_ctrl_get_name(u32 id)
/* Audio M2M controls */
case V4L2_CID_M2M_AUDIO_CLASS: return "Audio M2M Controls";
+ case V4L2_CID_M2M_AUDIO_SOURCE_RATE:return "Audio Source Sample
Rate";
+ case V4L2_CID_M2M_AUDIO_DEST_RATE: return "Audio Destination
Sample Rate";
default:
return NULL;
}
@@ -1606,6 +1608,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum
v4l2_ctrl_type *type,
case V4L2_CID_COLORIMETRY_HDR10_MASTERING_DISPLAY:
*type = V4L2_CTRL_TYPE_HDR10_MASTERING_DISPLAY;
break;
+ case V4L2_CID_M2M_AUDIO_SOURCE_RATE:
+ case V4L2_CID_M2M_AUDIO_DEST_RATE:
+ *type = V4L2_CTRL_TYPE_INTEGER_MENU;
+ break;
default:
*type = V4L2_CTRL_TYPE_INTEGER;
break;
diff --git a/include/uapi/linux/v4l2-controls.h
b/include/uapi/linux/v4l2-controls.h
index a8b4b830c757..30129ccdc282 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -3495,6 +3495,11 @@ struct v4l2_ctrl_av1_film_grain {
#define V4L2_CID_M2M_AUDIO_CLASS_BASE (V4L2_CTRL_CLASS_M2M_AUDIO | 0x900)
#define V4L2_CID_M2M_AUDIO_CLASS (V4L2_CTRL_CLASS_M2M_AUDIO | 1)
+#define V4L2_CID_M2M_AUDIO_SOURCE_RATE (V4L2_CID_M2M_AUDIO_CLASS_BASE + 0)
+#define V4L2_CID_M2M_AUDIO_DEST_RATE (V4L2_CID_M2M_AUDIO_CLASS_BASE + 1)
+#define V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET (V4L2_CID_M2M_AUDIO_CLASS_BASE
+ 2)
+#define V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET(V4L2_CID_M2M_AUDIO_CLASS_BASE
+ 3)
+
/* MPEG-compression definitions kept for backwards compatibility */
#ifndef __KERNEL__
#define V4L2_CTRL_CLASS_MPEGV4L2_CTRL_CLASS_CODEC
--
2.34.1
[PATCH v13 05/16] ASoC: fsl_asrc: register m2m platform device
Register m2m platform device, that user can
use M2M feature.
Defined platform data structure and platform
driver name.
Signed-off-by: Shengjiu Wang
Acked-by: Mark Brown
---
include/sound/fsl_asrc_common.h | 23 +++
sound/soc/fsl/fsl_asrc.c| 18 ++
2 files changed, 41 insertions(+)
diff --git a/include/sound/fsl_asrc_common.h b/include/sound/fsl_asrc_common.h
index 3b53d366182f..c709b8906929 100644
--- a/include/sound/fsl_asrc_common.h
+++ b/include/sound/fsl_asrc_common.h
@@ -71,6 +71,7 @@ struct fsl_asrc_pair {
* @dma_params_rx: DMA parameters for receive channel
* @dma_params_tx: DMA parameters for transmit channel
* @pdev: platform device pointer
+ * @m2m_pdev: m2m platform device pointer
* @regmap: regmap handler
* @paddr: physical address to the base address of registers
* @mem_clk: clock source to access register
@@ -103,6 +104,7 @@ struct fsl_asrc {
struct snd_dmaengine_dai_dma_data dma_params_rx;
struct snd_dmaengine_dai_dma_data dma_params_tx;
struct platform_device *pdev;
+ struct platform_device *m2m_pdev;
struct regmap *regmap;
unsigned long paddr;
struct clk *mem_clk;
@@ -139,6 +141,27 @@ struct fsl_asrc {
void *private;
};
+/**
+ * struct fsl_asrc_m2m_pdata - platform data
+ * @asrc: pointer to struct fsl_asrc
+ * @fmt_in: input sample format
+ * @fmt_out: output sample format
+ * @chan_min: minimum channel number
+ * @chan_max: maximum channel number
+ * @rate_min: minimum rate
+ * @rate_max: maximum rete
+ */
+struct fsl_asrc_m2m_pdata {
+ struct fsl_asrc *asrc;
+ u64 fmt_in;
+ u64 fmt_out;
+ int chan_min;
+ int chan_max;
+ int rate_min;
+ int rate_max;
+};
+
+#define M2M_DRV_NAME "fsl_asrc_m2m"
#define DRV_NAME "fsl-asrc-dai"
extern struct snd_soc_component_driver fsl_asrc_component;
diff --git a/sound/soc/fsl/fsl_asrc.c b/sound/soc/fsl/fsl_asrc.c
index 7d8643ee0ba0..5ecb5d869607 100644
--- a/sound/soc/fsl/fsl_asrc.c
+++ b/sound/soc/fsl/fsl_asrc.c
@@ -1187,6 +1187,7 @@ static int fsl_asrc_runtime_suspend(struct device *dev);
static int fsl_asrc_probe(struct platform_device *pdev)
{
struct device_node *np = pdev->dev.of_node;
+ struct fsl_asrc_m2m_pdata m2m_pdata;
struct fsl_asrc_priv *asrc_priv;
struct fsl_asrc *asrc;
struct resource *res;
@@ -1368,6 +1369,18 @@ static int fsl_asrc_probe(struct platform_device *pdev)
goto err_pm_get_sync;
}
+ m2m_pdata.asrc = asrc;
+ m2m_pdata.fmt_in = FSL_ASRC_FORMATS;
+ m2m_pdata.fmt_out = FSL_ASRC_FORMATS | SNDRV_PCM_FMTBIT_S8;
+ m2m_pdata.rate_min = 5512;
+ m2m_pdata.rate_max = 192000;
+ m2m_pdata.chan_min = 1;
+ m2m_pdata.chan_max = 10;
+ asrc->m2m_pdev = platform_device_register_data(>dev,
+ M2M_DRV_NAME,
+ PLATFORM_DEVID_AUTO,
+ _pdata,
+ sizeof(m2m_pdata));
return 0;
err_pm_get_sync:
@@ -1380,6 +1393,11 @@ static int fsl_asrc_probe(struct platform_device *pdev)
static void fsl_asrc_remove(struct platform_device *pdev)
{
+ struct fsl_asrc *asrc = dev_get_drvdata(>dev);
+
+ if (asrc->m2m_pdev && !IS_ERR(asrc->m2m_pdev))
+ platform_device_unregister(asrc->m2m_pdev);
+
pm_runtime_disable(>dev);
if (!pm_runtime_status_suspended(>dev))
fsl_asrc_runtime_suspend(>dev);
--
2.34.1
[PATCH v13 06/16] ASoC: fsl_easrc: register m2m platform device
Register m2m platform device,that user can
use M2M feature.
Signed-off-by: Shengjiu Wang
Acked-by: Mark Brown
---
sound/soc/fsl/fsl_easrc.c | 19 +++
1 file changed, 19 insertions(+)
diff --git a/sound/soc/fsl/fsl_easrc.c b/sound/soc/fsl/fsl_easrc.c
index cf7ad30a323b..ccbf45c7abf4 100644
--- a/sound/soc/fsl/fsl_easrc.c
+++ b/sound/soc/fsl/fsl_easrc.c
@@ -2075,6 +2075,7 @@ MODULE_DEVICE_TABLE(of, fsl_easrc_dt_ids);
static int fsl_easrc_probe(struct platform_device *pdev)
{
struct fsl_easrc_priv *easrc_priv;
+ struct fsl_asrc_m2m_pdata m2m_pdata;
struct device *dev = >dev;
struct fsl_asrc *easrc;
struct resource *res;
@@ -2190,6 +2191,19 @@ static int fsl_easrc_probe(struct platform_device *pdev)
goto err_pm_disable;
}
+ m2m_pdata.asrc = easrc;
+ m2m_pdata.fmt_in = FSL_EASRC_FORMATS;
+ m2m_pdata.fmt_out = FSL_EASRC_FORMATS |
SNDRV_PCM_FMTBIT_IEC958_SUBFRAME_LE;
+ m2m_pdata.rate_min = 8000;
+ m2m_pdata.rate_max = 768000;
+ m2m_pdata.chan_min = 1;
+ m2m_pdata.chan_max = 32;
+ easrc->m2m_pdev = platform_device_register_data(>dev,
+ M2M_DRV_NAME,
+ PLATFORM_DEVID_AUTO,
+ _pdata,
+ sizeof(m2m_pdata));
+
return 0;
err_pm_disable:
@@ -2199,6 +2213,11 @@ static int fsl_easrc_probe(struct platform_device *pdev)
static void fsl_easrc_remove(struct platform_device *pdev)
{
+ struct fsl_asrc *easrc = dev_get_drvdata(>dev);
+
+ if (easrc->m2m_pdev && !IS_ERR(easrc->m2m_pdev))
+ platform_device_unregister(easrc->m2m_pdev);
+
pm_runtime_disable(>dev);
}
--
2.34.1
[PATCH v13 01/16] media: v4l2-ctrls: add support for fraction_bits
From: Hans Verkuil
This adds support for the fraction_bits field, used with integer controls.
This allows fixed point formats to be described.
The fraction_bits field is only exposed through VIDIOC_QUERY_EXT_CTRL.
For a given signed two's complement Qf fixed point value 'f' equals
fraction_bits.
Signed-off-by: Hans Verkuil
---
.../media/v4l/vidioc-queryctrl.rst| 11 ++-
drivers/media/v4l2-core/v4l2-ctrls-api.c | 1 +
drivers/media/v4l2-core/v4l2-ctrls-core.c | 93 +++
include/media/v4l2-ctrls.h| 7 +-
include/uapi/linux/videodev2.h| 3 +-
5 files changed, 95 insertions(+), 20 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
index 4d38acafe8e1..e65c7e5d78ec 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
@@ -267,7 +267,16 @@ See also the examples in :ref:`control`.
- The size of each dimension. The first ``nr_of_dims`` elements of
this array must be non-zero, all remaining elements must be zero.
* - __u32
- - ``reserved``\ [32]
+ - ``fraction_bits``
+ - The number of least significant bits of the control value that
+form the fraction of the fixed point value. This is 0 if the value
+ is a regular integer. This can be used with all integer control types
+ (``INTEGER``, ``INTEGER64``, ``U8``, ``U16`` and ``U32``).
+ For the signed types the signed two's complement representation is used.
+ This field applies to the control value as well as the ``minimum``,
+ ``maximum``, ``step`` and ``default_value`` fields.
+* - __u32
+ - ``reserved``\ [31]
- Reserved for future extensions. Applications and drivers must set
the array to zero.
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-api.c
b/drivers/media/v4l2-core/v4l2-ctrls-api.c
index d9a422017bd9..ef16b00421ec 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-api.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-api.c
@@ -1101,6 +1101,7 @@ int v4l2_query_ext_ctrl(struct v4l2_ctrl_handler *hdl,
struct v4l2_query_ext_ctr
qc->elems = ctrl->elems;
qc->nr_of_dims = ctrl->nr_of_dims;
memcpy(qc->dims, ctrl->dims, qc->nr_of_dims * sizeof(qc->dims[0]));
+ qc->fraction_bits = ctrl->fraction_bits;
qc->minimum = ctrl->minimum;
qc->maximum = ctrl->maximum;
qc->default_value = ctrl->default_value;
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c
b/drivers/media/v4l2-core/v4l2-ctrls-core.c
index c4d995f32191..d83a37198bb5 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
@@ -252,12 +252,61 @@ void v4l2_ctrl_type_op_init(const struct v4l2_ctrl *ctrl,
u32 from_idx,
}
EXPORT_SYMBOL(v4l2_ctrl_type_op_init);
+static void v4l2_ctrl_log_fp(s64 v, unsigned int fraction_bits)
+{
+ s64 i, f, mask;
+
+ if (!fraction_bits) {
+ pr_cont("%lld", v);
+ return;
+ }
+
+ mask = (1ULL << fraction_bits) - 1;
+
+ /*
+* Note: this function does not support fixed point u64 with
+* fraction_bits set to 64. At the moment there is no U64
+* control type, but if that is added, then this code will have
+* to add support for it.
+*/
+ if (fraction_bits >= 63)
+ i = v < 0 ? -1 : 0;
+ else
+ i = div64_s64(v, 1LL << fraction_bits);
+
+ f = v < 0 ? -((-v) & mask) : (v & mask);
+
+ if (!f) {
+ pr_cont("%lld", i);
+ } else if (fraction_bits < 20) {
+ u64 div = 1ULL << fraction_bits;
+
+ if (!i && f < 0)
+ pr_cont("-%lld/%llu", -f, div);
+ else if (!i)
+ pr_cont("%lld/%llu", f, div);
+ else if (i < 0 || f < 0)
+ pr_cont("-%lld-%llu/%llu", -i, -f, div);
+ else
+ pr_cont("%lld+%llu/%llu", i, f, div);
+ } else {
+ if (!i && f < 0)
+ pr_cont("-%lld/(2^%u)", -f, fraction_bits);
+ else if (!i)
+ pr_cont("%lld/(2^%u)", f, fraction_bits);
+ else if (i < 0 || f < 0)
+ pr_cont("-%lld-%llu/(2^%u)", -i, -f, fraction_bits);
+ else
+ pr_cont("%lld+%llu/(2^%u)", i, f, fraction_bits);
+ }
+}
+
void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl)
{
union v4l2_ctrl_ptr ptr = ctrl->p_cur;
if (ctrl->is_array) {
- unsigned i;
+ unsigned int i;
for (i = 0; i < ctrl->nr_of_dims; i++)
pr_cont("[%u]", ctrl->dims[i]);
@@ -266,7 +315,7 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl)
[PATCH v13 07/16] media: uapi: Add V4L2_CAP_AUDIO_M2M capability flag
V4L2_CAP_AUDIO_M2M is similar to V4L2_CAP_VIDEO_M2M flag.
It is used for audio memory to memory case.
Signed-off-by: Shengjiu Wang
---
Documentation/userspace-api/media/v4l/vidioc-querycap.rst| 3 +++
Documentation/userspace-api/media/videodev2.h.rst.exceptions | 1 +
include/uapi/linux/videodev2.h | 1 +
3 files changed, 5 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/vidioc-querycap.rst
b/Documentation/userspace-api/media/v4l/vidioc-querycap.rst
index 6c57b8428356..1c0d97bf192a 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-querycap.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-querycap.rst
@@ -173,6 +173,9 @@ specification the ioctl returns an ``EINVAL`` error code.
interface. A video overlay device typically stores captured images
directly in the video memory of a graphics card, with hardware
clipping and scaling.
+* - ``V4L2_CAP_AUDIO_M2M``
+ - 0x0008
+ - The device supports the audio Memory-To-Memory interface.
* - ``V4L2_CAP_VBI_CAPTURE``
- 0x0010
- The device supports the :ref:`Raw VBI Capture `
diff --git a/Documentation/userspace-api/media/videodev2.h.rst.exceptions
b/Documentation/userspace-api/media/videodev2.h.rst.exceptions
index 3e58aac4ef0b..da6d0b8e4c2c 100644
--- a/Documentation/userspace-api/media/videodev2.h.rst.exceptions
+++ b/Documentation/userspace-api/media/videodev2.h.rst.exceptions
@@ -197,6 +197,7 @@ replace define V4L2_CAP_META_OUTPUT device-capabilities
replace define V4L2_CAP_DEVICE_CAPS device-capabilities
replace define V4L2_CAP_TOUCH device-capabilities
replace define V4L2_CAP_IO_MC device-capabilities
+replace define V4L2_CAP_AUDIO_M2M device-capabilities
# V4L2 pix flags
replace define V4L2_PIX_FMT_PRIV_MAGIC :c:type:`v4l2_pix_format`
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index b8573e9ccde6..5cc2a978fd9c 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -473,6 +473,7 @@ struct v4l2_capability {
#define V4L2_CAP_VIDEO_CAPTURE 0x0001 /* Is a video capture
device */
#define V4L2_CAP_VIDEO_OUTPUT 0x0002 /* Is a video output device
*/
#define V4L2_CAP_VIDEO_OVERLAY 0x0004 /* Can do video overlay */
+#define V4L2_CAP_AUDIO_M2M 0x0008 /* audio memory to memory */
#define V4L2_CAP_VBI_CAPTURE 0x0010 /* Is a raw VBI capture
device */
#define V4L2_CAP_VBI_OUTPUT0x0020 /* Is a raw VBI output
device */
#define V4L2_CAP_SLICED_VBI_CAPTURE0x0040 /* Is a sliced VBI capture
device */
--
2.34.1
[PATCH v13 02/16] ASoC: fsl_asrc: define functions for memory to memory usage
ASRC can be used on memory to memory case, define several
functions for m2m usage.
m2m_prepare: prepare for the start step
m2m_start: the start step
m2m_unprepare: unprepare for stop step, optional
m2m_stop: stop step
m2m_check_format: check format is supported or not
m2m_calc_out_len: calculate output length according to input length
m2m_get_maxburst: burst size for dma
m2m_pair_suspend: suspend function of pair, optional.
m2m_pair_resume: resume function of pair
get_output_fifo_size: get remaining data size in FIFO
Signed-off-by: Shengjiu Wang
Acked-by: Mark Brown
---
sound/soc/fsl/fsl_asrc.c| 126
sound/soc/fsl/fsl_asrc.h| 2 +
sound/soc/fsl/fsl_asrc_common.h | 37 ++
3 files changed, 165 insertions(+)
diff --git a/sound/soc/fsl/fsl_asrc.c b/sound/soc/fsl/fsl_asrc.c
index b793263291dc..7d8643ee0ba0 100644
--- a/sound/soc/fsl/fsl_asrc.c
+++ b/sound/soc/fsl/fsl_asrc.c
@@ -1063,6 +1063,124 @@ static int fsl_asrc_get_fifo_addr(u8 dir, enum
asrc_pair_index index)
return REG_ASRDx(dir, index);
}
+/* Get sample numbers in FIFO */
+static unsigned int fsl_asrc_get_output_fifo_size(struct fsl_asrc_pair *pair)
+{
+ struct fsl_asrc *asrc = pair->asrc;
+ enum asrc_pair_index index = pair->index;
+ u32 val;
+
+ regmap_read(asrc->regmap, REG_ASRFST(index), );
+
+ val &= ASRFSTi_OUTPUT_FIFO_MASK;
+
+ return val >> ASRFSTi_OUTPUT_FIFO_SHIFT;
+}
+
+static int fsl_asrc_m2m_prepare(struct fsl_asrc_pair *pair)
+{
+ struct fsl_asrc_pair_priv *pair_priv = pair->private;
+ struct fsl_asrc *asrc = pair->asrc;
+ struct device *dev = >pdev->dev;
+ struct asrc_config config;
+ int ret;
+
+ /* fill config */
+ config.pair = pair->index;
+ config.channel_num = pair->channels;
+ config.input_sample_rate = pair->rate[IN];
+ config.output_sample_rate = pair->rate[OUT];
+ config.input_format = pair->sample_format[IN];
+ config.output_format = pair->sample_format[OUT];
+ config.inclk = INCLK_NONE;
+ config.outclk = OUTCLK_ASRCK1_CLK;
+
+ pair_priv->config =
+ ret = fsl_asrc_config_pair(pair, true);
+ if (ret) {
+ dev_err(dev, "failed to config pair: %d\n", ret);
+ return ret;
+ }
+
+ pair->first_convert = 1;
+
+ return 0;
+}
+
+static int fsl_asrc_m2m_start(struct fsl_asrc_pair *pair)
+{
+ if (pair->first_convert) {
+ fsl_asrc_start_pair(pair);
+ pair->first_convert = 0;
+ }
+ /*
+* Clear DMA request during the stall state of ASRC:
+* During STALL state, the remaining in input fifo would never be
+* smaller than the input threshold while the output fifo would not
+* be bigger than output one. Thus the DMA request would be cleared.
+*/
+ fsl_asrc_set_watermarks(pair, ASRC_FIFO_THRESHOLD_MIN,
+ ASRC_FIFO_THRESHOLD_MAX);
+
+ /* Update the real input threshold to raise DMA request */
+ fsl_asrc_set_watermarks(pair, ASRC_M2M_INPUTFIFO_WML,
+ ASRC_M2M_OUTPUTFIFO_WML);
+
+ return 0;
+}
+
+static int fsl_asrc_m2m_stop(struct fsl_asrc_pair *pair)
+{
+ if (!pair->first_convert) {
+ fsl_asrc_stop_pair(pair);
+ pair->first_convert = 1;
+ }
+
+ return 0;
+}
+
+/* calculate capture data length according to output data length and sample
rate */
+static int fsl_asrc_m2m_calc_out_len(struct fsl_asrc_pair *pair, int
input_buffer_length)
+{
+ unsigned int in_width, out_width;
+ unsigned int channels = pair->channels;
+ unsigned int in_samples, out_samples;
+ unsigned int out_length;
+
+ in_width = snd_pcm_format_physical_width(pair->sample_format[IN]) / 8;
+ out_width = snd_pcm_format_physical_width(pair->sample_format[OUT]) / 8;
+
+ in_samples = input_buffer_length / in_width / channels;
+ out_samples = pair->rate[OUT] * in_samples / pair->rate[IN];
+ out_length = (out_samples - ASRC_OUTPUT_LAST_SAMPLE) * out_width *
channels;
+
+ return out_length;
+}
+
+static int fsl_asrc_m2m_get_maxburst(u8 dir, struct fsl_asrc_pair *pair)
+{
+ struct fsl_asrc *asrc = pair->asrc;
+ struct fsl_asrc_priv *asrc_priv = asrc->private;
+ int wml = (dir == IN) ? ASRC_M2M_INPUTFIFO_WML :
ASRC_M2M_OUTPUTFIFO_WML;
+
+ if (!asrc_priv->soc->use_edma)
+ return wml * pair->channels;
+ else
+ return 1;
+}
+
+static int fsl_asrc_m2m_pair_resume(struct fsl_asrc_pair *pair)
+{
+ struct fsl_asrc *asrc = pair->asrc;
+ int i;
+
+ for (i = 0; i < pair->channels * 4; i++)
+ regmap_write(asrc->regmap, REG_ASRDI(pair->index), 0);
+
+
[PATCH v13 03/16] ASoC: fsl_easrc: define functions for memory to memory usage
ASRC can be used on memory to memory case, define several
functions for m2m usage and export them as function pointer.
Signed-off-by: Shengjiu Wang
Acked-by: Mark Brown
---
sound/soc/fsl/fsl_easrc.c | 214 ++
sound/soc/fsl/fsl_easrc.h | 4 +
2 files changed, 218 insertions(+)
diff --git a/sound/soc/fsl/fsl_easrc.c b/sound/soc/fsl/fsl_easrc.c
index ec53bda46a46..cf7ad30a323b 100644
--- a/sound/soc/fsl/fsl_easrc.c
+++ b/sound/soc/fsl/fsl_easrc.c
@@ -1861,6 +1861,211 @@ static int fsl_easrc_get_fifo_addr(u8 dir, enum
asrc_pair_index index)
return REG_EASRC_FIFO(dir, index);
}
+/* Get sample numbers in FIFO */
+static unsigned int fsl_easrc_get_output_fifo_size(struct fsl_asrc_pair *pair)
+{
+ struct fsl_asrc *asrc = pair->asrc;
+ enum asrc_pair_index index = pair->index;
+ u32 val;
+
+ regmap_read(asrc->regmap, REG_EASRC_SFS(index), );
+ val &= EASRC_SFS_NSGO_MASK;
+
+ return val >> EASRC_SFS_NSGO_SHIFT;
+}
+
+static int fsl_easrc_m2m_prepare(struct fsl_asrc_pair *pair)
+{
+ struct fsl_easrc_ctx_priv *ctx_priv = pair->private;
+ struct fsl_asrc *asrc = pair->asrc;
+ struct device *dev = >pdev->dev;
+ int ret;
+
+ ctx_priv->in_params.sample_rate = pair->rate[IN];
+ ctx_priv->in_params.sample_format = pair->sample_format[IN];
+ ctx_priv->out_params.sample_rate = pair->rate[OUT];
+ ctx_priv->out_params.sample_format = pair->sample_format[OUT];
+
+ ctx_priv->in_params.fifo_wtmk = FSL_EASRC_INPUTFIFO_WML;
+ ctx_priv->out_params.fifo_wtmk = FSL_EASRC_OUTPUTFIFO_WML;
+ /* Fill the right half of the re-sampler with zeros */
+ ctx_priv->rs_init_mode = 0x2;
+ /* Zero fill the right half of the prefilter */
+ ctx_priv->pf_init_mode = 0x2;
+
+ ret = fsl_easrc_set_ctx_format(pair,
+ _priv->in_params.sample_format,
+ _priv->out_params.sample_format);
+ if (ret) {
+ dev_err(dev, "failed to set context format: %d\n", ret);
+ return ret;
+ }
+
+ ret = fsl_easrc_config_context(asrc, pair->index);
+ if (ret) {
+ dev_err(dev, "failed to config context %d\n", ret);
+ return ret;
+ }
+
+ ctx_priv->in_params.iterations = 1;
+ ctx_priv->in_params.group_len = pair->channels;
+ ctx_priv->in_params.access_len = pair->channels;
+ ctx_priv->out_params.iterations = 1;
+ ctx_priv->out_params.group_len = pair->channels;
+ ctx_priv->out_params.access_len = pair->channels;
+
+ ret = fsl_easrc_set_ctx_organziation(pair);
+ if (ret) {
+ dev_err(dev, "failed to set fifo organization\n");
+ return ret;
+ }
+
+ /* The context start flag */
+ pair->first_convert = 1;
+ return 0;
+}
+
+static int fsl_easrc_m2m_start(struct fsl_asrc_pair *pair)
+{
+ /* start context once */
+ if (pair->first_convert) {
+ fsl_easrc_start_context(pair);
+ pair->first_convert = 0;
+ }
+
+ return 0;
+}
+
+static int fsl_easrc_m2m_stop(struct fsl_asrc_pair *pair)
+{
+ /* Stop pair/context */
+ if (!pair->first_convert) {
+ fsl_easrc_stop_context(pair);
+ pair->first_convert = 1;
+ }
+
+ return 0;
+}
+
+/* calculate capture data length according to output data length and sample
rate */
+static int fsl_easrc_m2m_calc_out_len(struct fsl_asrc_pair *pair, int
input_buffer_length)
+{
+ struct fsl_asrc *easrc = pair->asrc;
+ struct fsl_easrc_priv *easrc_priv = easrc->private;
+ struct fsl_easrc_ctx_priv *ctx_priv = pair->private;
+ unsigned int in_rate = ctx_priv->in_params.norm_rate;
+ unsigned int out_rate = ctx_priv->out_params.norm_rate;
+ unsigned int channels = pair->channels;
+ unsigned int in_samples, out_samples;
+ unsigned int in_width, out_width;
+ unsigned int out_length;
+ unsigned int frac_bits;
+ u64 val1, val2;
+
+ switch (easrc_priv->rs_num_taps) {
+ case EASRC_RS_32_TAPS:
+ /* integer bits = 5; */
+ frac_bits = 39;
+ break;
+ case EASRC_RS_64_TAPS:
+ /* integer bits = 6; */
+ frac_bits = 38;
+ break;
+ case EASRC_RS_128_TAPS:
+ /* integer bits = 7; */
+ frac_bits = 37;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ val1 = (u64)in_rate << frac_bits;
+ do_div(val1, out_rate);
+ val1 += (s64)ctx_priv->ratio_mod << (frac_bits - 31);
+
+ in_width =
snd_pcm_format_physical_width(c
[PATCH v13 12/16] media: uapi: Declare interface types for Audio
Declare the interface types that will be used by Audio.
The type is MEDIA_INTF_T_V4L_AUDIO.
Signed-off-by: Shengjiu Wang
---
.../userspace-api/media/mediactl/media-types.rst| 5 +
drivers/media/v4l2-core/v4l2-dev.c | 4
drivers/media/v4l2-core/v4l2-mem2mem.c | 13 +
include/uapi/linux/media.h | 1 +
4 files changed, 19 insertions(+), 4 deletions(-)
diff --git a/Documentation/userspace-api/media/mediactl/media-types.rst
b/Documentation/userspace-api/media/mediactl/media-types.rst
index 6332e8395263..adfb37430f8e 100644
--- a/Documentation/userspace-api/media/mediactl/media-types.rst
+++ b/Documentation/userspace-api/media/mediactl/media-types.rst
@@ -265,6 +265,7 @@ Types and flags used to represent the media graph elements
.. _MEDIA-INTF-T-V4L-SUBDEV:
.. _MEDIA-INTF-T-V4L-SWRADIO:
.. _MEDIA-INTF-T-V4L-TOUCH:
+.. _MEDIA-INTF-T-V4L-AUDIO:
.. _MEDIA-INTF-T-ALSA-PCM-CAPTURE:
.. _MEDIA-INTF-T-ALSA-PCM-PLAYBACK:
.. _MEDIA-INTF-T-ALSA-CONTROL:
@@ -322,6 +323,10 @@ Types and flags used to represent the media graph elements
- Device node interface for Touch device (V4L)
- typically, /dev/v4l-touch?
+* - ``MEDIA_INTF_T_V4L_AUDIO``
+ - Device node interface for Audio device (V4L)
+ - typically, /dev/v4l-audio?
+
* - ``MEDIA_INTF_T_ALSA_PCM_CAPTURE``
- Device node interface for ALSA PCM Capture
- typically, /dev/snd/pcmC?D?c
diff --git a/drivers/media/v4l2-core/v4l2-dev.c
b/drivers/media/v4l2-core/v4l2-dev.c
index bac008fcedc6..ca8462a61e1f 100644
--- a/drivers/media/v4l2-core/v4l2-dev.c
+++ b/drivers/media/v4l2-core/v4l2-dev.c
@@ -844,6 +844,10 @@ static int video_register_media_controller(struct
video_device *vdev)
intf_type = MEDIA_INTF_T_V4L_SUBDEV;
/* Entity will be created via v4l2_device_register_subdev() */
break;
+ case VFL_TYPE_AUDIO:
+ intf_type = MEDIA_INTF_T_V4L_AUDIO;
+ /* Entity will be created via v4l2_device_register_subdev() */
+ break;
default:
return 0;
}
diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c
b/drivers/media/v4l2-core/v4l2-mem2mem.c
index 75517134a5e9..cda5e255305f 100644
--- a/drivers/media/v4l2-core/v4l2-mem2mem.c
+++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
@@ -1143,10 +1143,15 @@ int v4l2_m2m_register_media_controller(struct
v4l2_m2m_dev *m2m_dev,
if (ret)
goto err_rm_links0;
- /* Create video interface */
- m2m_dev->intf_devnode = media_devnode_create(mdev,
- MEDIA_INTF_T_V4L_VIDEO, 0,
- VIDEO_MAJOR, vdev->minor);
+ if (vdev->vfl_type == VFL_TYPE_AUDIO)
+ m2m_dev->intf_devnode = media_devnode_create(mdev,
+ MEDIA_INTF_T_V4L_AUDIO, 0,
+ VIDEO_MAJOR, vdev->minor);
+ else
+ /* Create video interface */
+ m2m_dev->intf_devnode = media_devnode_create(mdev,
+ MEDIA_INTF_T_V4L_VIDEO, 0,
+ VIDEO_MAJOR, vdev->minor);
if (!m2m_dev->intf_devnode) {
ret = -ENOMEM;
goto err_rm_links1;
diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
index 1c80b1d6bbaf..9ff6dec7393a 100644
--- a/include/uapi/linux/media.h
+++ b/include/uapi/linux/media.h
@@ -260,6 +260,7 @@ struct media_links_enum {
#define MEDIA_INTF_T_V4L_SUBDEV(MEDIA_INTF_T_V4L_BASE
+ 3)
#define MEDIA_INTF_T_V4L_SWRADIO (MEDIA_INTF_T_V4L_BASE + 4)
#define MEDIA_INTF_T_V4L_TOUCH (MEDIA_INTF_T_V4L_BASE + 5)
+#define MEDIA_INTF_T_V4L_AUDIO (MEDIA_INTF_T_V4L_BASE + 6)
#define MEDIA_INTF_T_ALSA_BASE 0x0300
#define MEDIA_INTF_T_ALSA_PCM_CAPTURE (MEDIA_INTF_T_ALSA_BASE)
--
2.34.1
Re: [PATCH v12 08/15] media: uapi: Define audio sample format fourcc type
On Wed, Feb 21, 2024 at 7:10 PM Hans Verkuil wrote:
>
> On 19/02/2024 13:56, Mauro Carvalho Chehab wrote:
> > Em Mon, 19 Feb 2024 12:05:02 +0800
> > Shengjiu Wang escreveu:
> >
> >> Hi Mauro
> >>
> >> On Sat, Feb 17, 2024 at 5:19 PM Mauro Carvalho Chehab
> >> wrote:
> >>>
> >>> Em Thu, 18 Jan 2024 20:32:01 +0800
> >>> Shengjiu Wang escreveu:
> >>>
> >>>> The audio sample format definition is from alsa,
> >>>> the header file is include/uapi/sound/asound.h, but
> >>>> don't include this header file directly, because in
> >>>> user space, there is another copy in alsa-lib.
> >>>> There will be conflict in userspace for include
> >>>> videodev2.h & asound.h and asoundlib.h
> >>>>
> >>>> Here still use the fourcc format.
> >>>>
> >>>> Signed-off-by: Shengjiu Wang
> >>>> ---
> >>>> .../userspace-api/media/v4l/pixfmt-audio.rst | 87 +++
> >>>> .../userspace-api/media/v4l/pixfmt.rst| 1 +
> >>>> drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++
> >>>> include/uapi/linux/videodev2.h| 23 +
> >>>> 4 files changed, 124 insertions(+)
> >>>> create mode 100644
> >>>> Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >>>>
> >>>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >>>> b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >>>> new file mode 100644
> >>>> index ..04b4a7fbd8f4
> >>>> --- /dev/null
> >>>> +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >>>> @@ -0,0 +1,87 @@
> >>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> >>>> +
> >>>> +.. _pixfmt-audio:
> >>>> +
> >>>> +*
> >>>> +Audio Formats
> >>>> +*
> >>>> +
> >>>> +These formats are used for :ref:`audiomem2mem` interface only.
> >>>> +
> >>>> +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}|
> >>>> +
> >>>> +.. cssclass:: longtable
> >>>> +
> >>>> +.. flat-table:: Audio Format
> >>>> +:header-rows: 1
> >>>> +:stub-columns: 0
> >>>> +:widths: 3 1 4
> >>>> +
> >>>> +* - Identifier
> >>>> + - Code
> >>>> + - Details
> >>>> +* .. _V4L2-AUDIO-FMT-S8:
> >>>> +
> >>>> + - ``V4L2_AUDIO_FMT_S8``
> >>>> + - 'S8'
> >>>> + - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA
> >>>> +* .. _V4L2-AUDIO-FMT-S16-LE:
> >>>
> >>> Hmm... why can't we just use SNDRV_*_FORMAT_*? Those are already part of
> >>> an uAPI header. No need to add any abstraction here and/or redefine
> >>> what is there already at include/uapi/sound/asound.h.
> >>>
> >> Actually I try to avoid including the include/uapi/sound/asound.h.
> >> Because in user space, there is another copy in alsa-lib (asoundlib.h).
> >> There will be conflict in userspace when including videodev2.h and
> >> asoundlib.h.
> >
> > Well, alsasoundlib.h seems to be using the same definitions:
> > https://github.com/michaelwu/alsa-lib/blob/master/include/pcm.h
> >
> > So, I can't see what would be the actual issue, as both userspace library
> > and ALSA internal headers use the same magic numbers.
> >
> > You can still do things like:
> >
> > #ifdef __KERNEL__
> > # include
> > #else
> > # include
> > #endif
> >
> > To avoid such kind of conflicts, if you need to have it included on
> > some header file. Yet, I can't see why you would need that.
> >
> > IMO, at uAPI headers, you just need to declare the uAPI audiofmt field
> > to be either __u32 or __u64, pointing it to where this value comes from
> > (on both userspace and Kernelspace. E. g.:
> >
> > /**
> > * struct v4l2_audio_format - audio data format definition
> > * @audioformat:
> > *an integer number matching the fields inside
> > *enum snd_pcm_format_t (e. g. `SNDRV_PCM_FORMAT_*`), as defined
> > *in include/uapi/sound/asound.h
Re: [PATCH v12 07/15] media: v4l2: Add audio capture and output support
On Wed, Feb 21, 2024 at 12:30 PM Tomasz Figa wrote:
>
> On Sat, Feb 17, 2024 at 6:42 PM Mauro Carvalho Chehab
> wrote:
> >
> > Em Thu, 18 Jan 2024 20:32:00 +0800
> > Shengjiu Wang escreveu:
> >
> > > Audio signal processing has the requirement for memory to
> > > memory similar as Video.
> > >
> > > This patch is to add this support in v4l2 framework, defined
> > > new buffer type V4L2_BUF_TYPE_AUDIO_CAPTURE and
> > > V4L2_BUF_TYPE_AUDIO_OUTPUT, defined new format v4l2_audio_format
> > > for audio case usage.
> > >
> > > The created audio device is named "/dev/v4l-audioX".
> > >
> > > Signed-off-by: Shengjiu Wang
> > > ---
> > > .../userspace-api/media/v4l/buffer.rst| 6 ++
> > > .../media/v4l/dev-audio-mem2mem.rst | 71 +++
> > > .../userspace-api/media/v4l/devices.rst | 1 +
> > > .../media/v4l/vidioc-enum-fmt.rst | 2 +
> > > .../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 ++
> > > .../media/videodev2.h.rst.exceptions | 2 +
> > > .../media/common/videobuf2/videobuf2-v4l2.c | 4 ++
> > > drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 9 +++
> > > drivers/media/v4l2-core/v4l2-dev.c| 17 +
> > > drivers/media/v4l2-core/v4l2-ioctl.c | 53 ++
> > > include/media/v4l2-dev.h | 2 +
> > > include/media/v4l2-ioctl.h| 34 +
> > > include/uapi/linux/videodev2.h| 17 +
> > > 13 files changed, 222 insertions(+)
> > > create mode 100644
> > > Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> > >
> > > diff --git a/Documentation/userspace-api/media/v4l/buffer.rst
> > > b/Documentation/userspace-api/media/v4l/buffer.rst
> > > index 52bbee81c080..a3754ca6f0d6 100644
> > > --- a/Documentation/userspace-api/media/v4l/buffer.rst
> > > +++ b/Documentation/userspace-api/media/v4l/buffer.rst
> > > @@ -438,6 +438,12 @@ enum v4l2_buf_type
> > > * - ``V4L2_BUF_TYPE_META_OUTPUT``
> > >- 14
> >
> > >- Buffer for metadata output, see :ref:`metadata`.
> > > +* - ``V4L2_BUF_TYPE_AUDIO_CAPTURE``
> > > + - 15
> > > + - Buffer for audio capture, see :ref:`audio`.
> > > +* - ``V4L2_BUF_TYPE_AUDIO_OUTPUT``
> > > + - 16
> >
> > Hmm... alsa APi define input/output as:
> > enum {
> > SNDRV_PCM_STREAM_PLAYBACK = 0,
> > SNDRV_PCM_STREAM_CAPTURE,
> > SNDRV_PCM_STREAM_LAST = SNDRV_PCM_STREAM_CAPTURE,
> > };
> >
> >
> > I would use a namespace as close as possible to the
> > ALSA API. Also, we're not talking about V4L2, but, instead
> > audio. so, not sure if I like the prefix to start with
> > V4L2_. Maybe ALSA_?
> >
> > So, a better namespace would be:
> >
> > ${prefix}_BUF_TYPE_PCM_STREAM_PLAYBACK
> > and
> > ${prefix}_BUF_TYPE_PCM_STREAM_CAPTURE
> >
>
> The API is still V4L2, and all the other non-video buf types also use
> the V4L2_ prefix, so perhaps that's good here as well?
>
> Whether AUDIO or PCM_STREAM makes more sense goes outside of my
> expertise. Subjectively, a PCM stream sounds more specific than an
> audio stream. Do those buf types also support non-PCM audio streams?
Currently I use it for PCM, but I think it can also be used for non-PCM.
So use the below name?
V4L2_BUF_TYPE_AUDIO_CAPTURE
V4L2_BUF_TYPE_AUDIO_PLAYBACK
>
> > > + - Buffer for audio output, see :ref:`audio`.
> > >
> > >
> > > .. _buffer-flags:
> > > diff --git a/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> > > b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> > > new file mode 100644
> > > index ..68faecfe3a02
> > > --- /dev/null
> > > +++ b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> > > @@ -0,0 +1,71 @@
> > > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> > > +
> > > +.. _audiomem2mem:
> > > +
> > > +
> > > +Audio Memory-To-Memory Interface
> > > +
> > > +
> > > +An audio memory-to-memory device can compress, decompress, transform, or
> > > +otherwise convert audio data from one format into another format, in
> > > memory.
> &
Re: [PATCH v12 07/15] media: v4l2: Add audio capture and output support
Hi Mauro, Hans
On Sat, Feb 17, 2024 at 5:42 PM Mauro Carvalho Chehab
wrote:
>
> Em Thu, 18 Jan 2024 20:32:00 +0800
> Shengjiu Wang escreveu:
>
> > Audio signal processing has the requirement for memory to
> > memory similar as Video.
> >
> > This patch is to add this support in v4l2 framework, defined
> > new buffer type V4L2_BUF_TYPE_AUDIO_CAPTURE and
> > V4L2_BUF_TYPE_AUDIO_OUTPUT, defined new format v4l2_audio_format
> > for audio case usage.
> >
> > The created audio device is named "/dev/v4l-audioX".
> >
> > Signed-off-by: Shengjiu Wang
> > ---
> > .../userspace-api/media/v4l/buffer.rst| 6 ++
> > .../media/v4l/dev-audio-mem2mem.rst | 71 +++
> > .../userspace-api/media/v4l/devices.rst | 1 +
> > .../media/v4l/vidioc-enum-fmt.rst | 2 +
> > .../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 ++
> > .../media/videodev2.h.rst.exceptions | 2 +
> > .../media/common/videobuf2/videobuf2-v4l2.c | 4 ++
> > drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 9 +++
> > drivers/media/v4l2-core/v4l2-dev.c| 17 +
> > drivers/media/v4l2-core/v4l2-ioctl.c | 53 ++
> > include/media/v4l2-dev.h | 2 +
> > include/media/v4l2-ioctl.h| 34 +
> > include/uapi/linux/videodev2.h| 17 +
> > 13 files changed, 222 insertions(+)
> > create mode 100644
> > Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> >
> > diff --git a/Documentation/userspace-api/media/v4l/buffer.rst
> > b/Documentation/userspace-api/media/v4l/buffer.rst
> > index 52bbee81c080..a3754ca6f0d6 100644
> > --- a/Documentation/userspace-api/media/v4l/buffer.rst
> > +++ b/Documentation/userspace-api/media/v4l/buffer.rst
> > @@ -438,6 +438,12 @@ enum v4l2_buf_type
> > * - ``V4L2_BUF_TYPE_META_OUTPUT``
> >- 14
>
> >- Buffer for metadata output, see :ref:`metadata`.
> > +* - ``V4L2_BUF_TYPE_AUDIO_CAPTURE``
> > + - 15
> > + - Buffer for audio capture, see :ref:`audio`.
> > +* - ``V4L2_BUF_TYPE_AUDIO_OUTPUT``
> > + - 16
>
> Hmm... alsa APi define input/output as:
> enum {
> SNDRV_PCM_STREAM_PLAYBACK = 0,
> SNDRV_PCM_STREAM_CAPTURE,
> SNDRV_PCM_STREAM_LAST = SNDRV_PCM_STREAM_CAPTURE,
> };
>
>
> I would use a namespace as close as possible to the
> ALSA API. Also, we're not talking about V4L2, but, instead
> audio. so, not sure if I like the prefix to start with
> V4L2_. Maybe ALSA_?
>
> So, a better namespace would be:
>
> ${prefix}_BUF_TYPE_PCM_STREAM_PLAYBACK
> and
> ${prefix}_BUF_TYPE_PCM_STREAM_CAPTURE
>
Ok, So can I use the name below?
V4L2_BUF_TYPE_PCM_STREAM_PLAYBACK
V4L2_BUF_TYPE_PCM_STREAM_CAPTURE
> > + - Buffer for audio output, see :ref:`audio`.
> >
> >
> > .. _buffer-flags:
> > diff --git a/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> > b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> > new file mode 100644
> > index ..68faecfe3a02
> > --- /dev/null
> > +++ b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> > @@ -0,0 +1,71 @@
> > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> > +
> > +.. _audiomem2mem:
> > +
> > +
> > +Audio Memory-To-Memory Interface
> > +
> > +
> > +An audio memory-to-memory device can compress, decompress, transform, or
> > +otherwise convert audio data from one format into another format, in
> > memory.
> > +Such memory-to-memory devices set the ``V4L2_CAP_AUDIO_M2M`` capability.
> > +Examples of memory-to-memory devices are audio codecs, audio preprocessing,
> > +audio postprocessing.
> > +
> > +A memory-to-memory audio node supports both output (sending audio frames
> > from
> > +memory to the hardware) and capture (receiving the processed audio frames
> > +from the hardware into memory) stream I/O. An application will have to
> > +setup the stream I/O for both sides and finally call
> > +:ref:`VIDIOC_STREAMON ` for both capture and output to
> > +start the hardware.
> > +
> > +Memory-to-memory devices function as a shared resource: you can
> > +open the audio node multiple times, each application setting up their
> > +own properties that are local to the file handle, and each can use
> > +it independently from the
Re: [PATCH v12 10/15] media: uapi: Add audio rate controls support
On Sat, Feb 17, 2024 at 5:57 PM Mauro Carvalho Chehab wrote: > > Em Thu, 18 Jan 2024 20:32:03 +0800 > Shengjiu Wang escreveu: > > > Add V4L2_CID_M2M_AUDIO_SOURCE_RATE and V4L2_CID_M2M_AUDIO_DEST_RATE > > new IDs for rate control. > > > > Add V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET and > > V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET for clock drift. > > > > Signed-off-by: Shengjiu Wang > > --- > > .../media/v4l/ext-ctrls-audio-m2m.rst | 20 +++ > > drivers/media/v4l2-core/v4l2-ctrls-defs.c | 6 ++ > > include/uapi/linux/v4l2-controls.h| 5 + > > 3 files changed, 31 insertions(+) > > > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst > > b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst > > index 82d2ecedbfee..de579ab8fb94 100644 > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst > > @@ -19,3 +19,23 @@ Audio M2M Control IDs > > The Audio M2M class descriptor. Calling > > :ref:`VIDIOC_QUERYCTRL` for this control will > > return a description of this control class. > > + > > +.. _v4l2-audio-asrc: > > + > > +``V4L2_CID_M2M_AUDIO_SOURCE_RATE (integer menu)`` > > +Sets the audio source sample rate, unit is Hz > > + > > +``V4L2_CID_M2M_AUDIO_DEST_RATE (integer menu)`` > > +Sets the audio destination sample rate, unit is Hz > > + > > +``V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET (fixed point)`` > > +Sets the offset from the audio source sample rate, unit is Hz. > > +The offset compensates for any clock drift. The actual source audio > > +sample rate is the ideal source audio sample rate from > > +``V4L2_CID_M2M_AUDIO_SOURCE_RATE`` plus this fixed point offset. > > + > > +``V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET (fixed point)`` > > +Sets the offset from the audio destination sample rate, unit is Hz. > > +The offset compensates for any clock drift. The actual destination > > audio > > +sample rate is the ideal source audio sample rate from > > +``V4L2_CID_M2M_AUDIO_DEST_RATE`` plus this fixed point offset. > > Hmm... first of all, controls on V4L2 API can either be get or set. > So, starting the sentence with "Set" sounds an assumption that may > be wrong. Ok, will update the description. > > Also, I would explain a little bit more about the frequency offset values, > as clock drift adjustment on PCM streams is something that can be done > using different approaches. > > I'm assuming that what you wanted here is to use it to check if the > video and audio clocks have some drift, and reducing or increasing > the audio sample rate dynamically to ensure that such drift will > stay constraint to a maximum allowed drift measured in mili or nano > seconds. So, userspace would be expected to be monitoring such drift > and increasing/decreasing the sample frequency as needed to maintain > such constraint. > > Is that the way such uAPI is expected to work? Yes. Userspace should monitor the drift, get the offset based on the common sample rate (8k, 44.1k, 48k...) then send it to the driver. The offset is a fixed point. it is base on the patch: https://patchwork.kernel.org/project/linux-media/patch/cec82507-ced9-4e7d-802c-04a40f84a...@xs4all.nl/ Best regards Shengjiu Wang > > > diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c > > b/drivers/media/v4l2-core/v4l2-ctrls-defs.c > > index 2a85ea3dc92f..91e1f5348c23 100644 > > --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c > > +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c > > @@ -1245,6 +1245,8 @@ const char *v4l2_ctrl_get_name(u32 id) > > > > /* Audio M2M controls */ > > case V4L2_CID_M2M_AUDIO_CLASS: return "Audio M2M Controls"; > > + case V4L2_CID_M2M_AUDIO_SOURCE_RATE:return "Audio Source Sample > > Rate"; > > + case V4L2_CID_M2M_AUDIO_DEST_RATE: return "Audio Destination > > Sample Rate"; > > default: > > return NULL; > > } > > @@ -1606,6 +1608,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum > > v4l2_ctrl_type *type, > > case V4L2_CID_COLORIMETRY_HDR10_MASTERING_DISPLAY: > > *type = V4L2_CTRL_TYPE_HDR10_MASTERING_DISPLAY; > > break; > > + case V4L2_CID_M2M_AUDIO_SOURCE_RATE: > > + case V4L2_CID_M2M_AUDIO_DEST_RATE: > > + *type = V4L2_CTRL_TYPE_INTEGER_MENU; > > + break; > > defa
Re: [PATCH v12 08/15] media: uapi: Define audio sample format fourcc type
Hi Mauro
On Sat, Feb 17, 2024 at 5:19 PM Mauro Carvalho Chehab
wrote:
>
> Em Thu, 18 Jan 2024 20:32:01 +0800
> Shengjiu Wang escreveu:
>
> > The audio sample format definition is from alsa,
> > the header file is include/uapi/sound/asound.h, but
> > don't include this header file directly, because in
> > user space, there is another copy in alsa-lib.
> > There will be conflict in userspace for include
> > videodev2.h & asound.h and asoundlib.h
> >
> > Here still use the fourcc format.
> >
> > Signed-off-by: Shengjiu Wang
> > ---
> > .../userspace-api/media/v4l/pixfmt-audio.rst | 87 +++
> > .../userspace-api/media/v4l/pixfmt.rst| 1 +
> > drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++
> > include/uapi/linux/videodev2.h| 23 +
> > 4 files changed, 124 insertions(+)
> > create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >
> > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> > b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> > new file mode 100644
> > index ..04b4a7fbd8f4
> > --- /dev/null
> > +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> > @@ -0,0 +1,87 @@
> > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> > +
> > +.. _pixfmt-audio:
> > +
> > +*
> > +Audio Formats
> > +*
> > +
> > +These formats are used for :ref:`audiomem2mem` interface only.
> > +
> > +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}|
> > +
> > +.. cssclass:: longtable
> > +
> > +.. flat-table:: Audio Format
> > +:header-rows: 1
> > +:stub-columns: 0
> > +:widths: 3 1 4
> > +
> > +* - Identifier
> > + - Code
> > + - Details
> > +* .. _V4L2-AUDIO-FMT-S8:
> > +
> > + - ``V4L2_AUDIO_FMT_S8``
> > + - 'S8'
> > + - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA
> > +* .. _V4L2-AUDIO-FMT-S16-LE:
>
> Hmm... why can't we just use SNDRV_*_FORMAT_*? Those are already part of
> an uAPI header. No need to add any abstraction here and/or redefine
> what is there already at include/uapi/sound/asound.h.
>
Actually I try to avoid including the include/uapi/sound/asound.h.
Because in user space, there is another copy in alsa-lib (asoundlib.h).
There will be conflict in userspace when including videodev2.h and
asoundlib.h.
And in the V4l framework, the fourcc type is commonly used in other
cases (video, radio, touch, meta), to avoid changing common code
a lot, so I think using fourcc definition for audio may be simpler.
Best regards
Shengjiu Wang
[PATCH v12 15/15] media: vim2m-audio: add virtual driver for audio memory to memory
Audio memory to memory virtual driver use video memory to memory
virtual driver vim2m.c as example. The main difference is
device type is VFL_TYPE_AUDIO and device cap type is V4L2_CAP_AUDIO_M2M.
The device_run function is a dummy function, which is simply
copy the data from input buffer to output buffer.
Signed-off-by: Shengjiu Wang
---
drivers/media/test-drivers/Kconfig | 10 +
drivers/media/test-drivers/Makefile | 1 +
drivers/media/test-drivers/vim2m-audio.c | 793 +++
3 files changed, 804 insertions(+)
create mode 100644 drivers/media/test-drivers/vim2m-audio.c
diff --git a/drivers/media/test-drivers/Kconfig
b/drivers/media/test-drivers/Kconfig
index 5a5379524bde..b6b52a7ca042 100644
--- a/drivers/media/test-drivers/Kconfig
+++ b/drivers/media/test-drivers/Kconfig
@@ -16,6 +16,16 @@ config VIDEO_VIM2M
This is a virtual test device for the memory-to-memory driver
framework.
+config VIDEO_VIM2M_AUDIO
+ tristate "Virtual Memory-to-Memory Driver For Audio"
+ depends on VIDEO_DEV
+ select VIDEOBUF2_VMALLOC
+ select V4L2_MEM2MEM_DEV
+ select MEDIA_CONTROLLER
+ help
+ This is a virtual audio test device for the memory-to-memory driver
+ framework.
+
source "drivers/media/test-drivers/vicodec/Kconfig"
source "drivers/media/test-drivers/vimc/Kconfig"
source "drivers/media/test-drivers/vivid/Kconfig"
diff --git a/drivers/media/test-drivers/Makefile
b/drivers/media/test-drivers/Makefile
index 740714a4584d..0c61c9ada3e1 100644
--- a/drivers/media/test-drivers/Makefile
+++ b/drivers/media/test-drivers/Makefile
@@ -10,6 +10,7 @@ obj-$(CONFIG_DVB_VIDTV) += vidtv/
obj-$(CONFIG_VIDEO_VICODEC) += vicodec/
obj-$(CONFIG_VIDEO_VIM2M) += vim2m.o
+obj-$(CONFIG_VIDEO_VIM2M_AUDIO) += vim2m-audio.o
obj-$(CONFIG_VIDEO_VIMC) += vimc/
obj-$(CONFIG_VIDEO_VIVID) += vivid/
obj-$(CONFIG_VIDEO_VISL) += visl/
diff --git a/drivers/media/test-drivers/vim2m-audio.c
b/drivers/media/test-drivers/vim2m-audio.c
new file mode 100644
index ..6361df6320b3
--- /dev/null
+++ b/drivers/media/test-drivers/vim2m-audio.c
@@ -0,0 +1,793 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * A virtual v4l2-mem2mem example for audio device.
+ */
+
+#include
+#include
+#include
+#include
+#include
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+MODULE_DESCRIPTION("Virtual device for audio mem2mem testing");
+MODULE_LICENSE("GPL");
+
+static unsigned int debug;
+module_param(debug, uint, 0644);
+MODULE_PARM_DESC(debug, "debug level");
+
+#define MEM2MEM_NAME "vim2m-audio"
+
+#define dprintk(dev, lvl, fmt, arg...) \
+ v4l2_dbg(lvl, debug, &(dev)->v4l2_dev, "%s: " fmt, __func__, ## arg)
+
+#define SAMPLE_NUM 4096
+
+static void audm2m_dev_release(struct device *dev)
+{}
+
+static struct platform_device audm2m_pdev = {
+ .name = MEM2MEM_NAME,
+ .dev.release= audm2m_dev_release,
+};
+
+static u32 formats[] = {
+ V4L2_AUDIO_FMT_S16_LE,
+};
+
+#define NUM_FORMATS ARRAY_SIZE(formats)
+
+/* Per-queue, driver-specific private data */
+struct audm2m_q_data {
+ unsigned intrate;
+ unsigned intchannels;
+ unsigned intbuffersize;
+ unsigned intsequence;
+ u32 fourcc;
+};
+
+enum {
+ V4L2_M2M_SRC = 0,
+ V4L2_M2M_DST = 1,
+};
+
+static snd_pcm_format_t find_format(u32 fourcc)
+{
+ snd_pcm_format_t fmt;
+ unsigned int k;
+
+ for (k = 0; k < NUM_FORMATS; k++) {
+ if (formats[k] == fourcc)
+ break;
+ }
+
+ if (k == NUM_FORMATS)
+ return 0;
+
+ fmt = v4l2_fourcc_to_audfmt(formats[k]);
+
+ return fmt;
+}
+
+struct audm2m_dev {
+ struct v4l2_device v4l2_dev;
+ struct video_device vfd;
+
+ struct mutexdev_mutex;
+
+ struct v4l2_m2m_dev *m2m_dev;
+#ifdef CONFIG_MEDIA_CONTROLLER
+ struct media_device mdev;
+#endif
+};
+
+struct audm2m_ctx {
+ struct v4l2_fh fh;
+ struct v4l2_ctrl_handlerctrl_handler;
+ struct audm2m_dev *dev;
+
+ struct mutexvb_mutex;
+
+ /* Source and destination queue data */
+ struct audm2m_q_data q_data[2];
+};
+
+static inline struct audm2m_ctx *file2ctx(struct file *file)
+{
+ return container_of(file->private_data, struct audm2m_ctx, fh);
+}
+
+static struct audm2m_q_data *get_q_data(struct audm2m_ctx *ctx,
+ enum v4l2_buf_type type)
+{
+ if (type == V4L2_BUF_TYPE_AUDIO_OUTPUT)
+ return >q_data[V4L2_M2M_SRC];
+ return >q_data[V4L2_M2M_DST];
+}
+
+static const char *type_name(enum v4l2_buf_type type)
+{
+ if (type == V4L2_BUF
[PATCH v12 14/15] media: imx-asrc: Add memory to memory driver
Implement the ASRC memory to memory function using
the v4l2 framework, user can use this function with
v4l2 ioctl interface.
User send the output and capture buffer to driver and
driver store the converted data to the capture buffer.
This feature can be shared by ASRC and EASRC drivers
Signed-off-by: Shengjiu Wang
---
drivers/media/platform/nxp/Kconfig| 13 +
drivers/media/platform/nxp/Makefile |1 +
drivers/media/platform/nxp/imx-asrc.c | 1256 +
3 files changed, 1270 insertions(+)
create mode 100644 drivers/media/platform/nxp/imx-asrc.c
diff --git a/drivers/media/platform/nxp/Kconfig
b/drivers/media/platform/nxp/Kconfig
index 40e3436669e2..8d0ca335601f 100644
--- a/drivers/media/platform/nxp/Kconfig
+++ b/drivers/media/platform/nxp/Kconfig
@@ -67,3 +67,16 @@ config VIDEO_MX2_EMMAPRP
source "drivers/media/platform/nxp/dw100/Kconfig"
source "drivers/media/platform/nxp/imx-jpeg/Kconfig"
+
+config VIDEO_IMX_ASRC
+ tristate "NXP i.MX ASRC M2M support"
+ depends on V4L_MEM2MEM_DRIVERS
+ depends on MEDIA_SUPPORT
+ select VIDEOBUF2_DMA_CONTIG
+ select V4L2_MEM2MEM_DEV
+ select MEDIA_CONTROLLER
+ help
+ Say Y if you want to add ASRC M2M support for NXP CPUs.
+ It is a complement for ASRC M2P and ASRC P2M features.
+ This option is only useful for out-of-tree drivers since
+ in-tree drivers select it automatically.
diff --git a/drivers/media/platform/nxp/Makefile
b/drivers/media/platform/nxp/Makefile
index 4d90eb713652..1325675e34f5 100644
--- a/drivers/media/platform/nxp/Makefile
+++ b/drivers/media/platform/nxp/Makefile
@@ -9,3 +9,4 @@ obj-$(CONFIG_VIDEO_IMX8MQ_MIPI_CSI2) += imx8mq-mipi-csi2.o
obj-$(CONFIG_VIDEO_IMX_MIPI_CSIS) += imx-mipi-csis.o
obj-$(CONFIG_VIDEO_IMX_PXP) += imx-pxp.o
obj-$(CONFIG_VIDEO_MX2_EMMAPRP) += mx2_emmaprp.o
+obj-$(CONFIG_VIDEO_IMX_ASRC) += imx-asrc.o
diff --git a/drivers/media/platform/nxp/imx-asrc.c
b/drivers/media/platform/nxp/imx-asrc.c
new file mode 100644
index ..0c25a36199b1
--- /dev/null
+++ b/drivers/media/platform/nxp/imx-asrc.c
@@ -0,0 +1,1256 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Copyright (C) 2014-2016 Freescale Semiconductor, Inc.
+// Copyright (C) 2019-2023 NXP
+//
+// Freescale ASRC Memory to Memory (M2M) driver
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#define V4L_CAP OUT
+#define V4L_OUT IN
+
+#define ASRC_xPUT_DMA_CALLBACK(dir) \
+ (((dir) == V4L_OUT) ? asrc_input_dma_callback \
+ : asrc_output_dma_callback)
+
+#define DIR_STR(dir) (dir) == V4L_OUT ? "out" : "cap"
+
+/* Maximum output and capture buffer size */
+#define ASRC_M2M_BUFFER_SIZE (512 * 1024)
+
+/* Maximum output and capture period size */
+#define ASRC_M2M_PERIOD_SIZE (48 * 1024)
+
+struct asrc_pair_m2m {
+ struct fsl_asrc_pair *pair;
+ struct asrc_m2m *m2m;
+ struct v4l2_fh fh;
+ struct v4l2_ctrl_handler ctrl_handler;
+ int channels[2];
+ unsigned int sequence[2];
+ s64 src_rate_off_prev; /* Q31.32 */
+ s64 dst_rate_off_prev; /* Q31.32 */
+ s64 src_rate_off_cur; /* Q31.32 */
+ s64 dst_rate_off_cur; /* Q31.32 */
+};
+
+struct asrc_m2m {
+ struct fsl_asrc_m2m_pdata pdata;
+ struct v4l2_device v4l2_dev;
+ struct v4l2_m2m_dev *m2m_dev;
+ struct video_device *dec_vdev;
+ struct mutex mlock; /* v4l2 ioctls serialization */
+ struct platform_device *pdev;
+#ifdef CONFIG_MEDIA_CONTROLLER
+ struct media_device mdev;
+#endif
+};
+
+static u32 formats[] = {
+ V4L2_AUDIO_FMT_S8,
+ V4L2_AUDIO_FMT_S16_LE,
+ V4L2_AUDIO_FMT_U16_LE,
+ V4L2_AUDIO_FMT_S24_LE,
+ V4L2_AUDIO_FMT_S24_3LE,
+ V4L2_AUDIO_FMT_U24_LE,
+ V4L2_AUDIO_FMT_U24_3LE,
+ V4L2_AUDIO_FMT_S32_LE,
+ V4L2_AUDIO_FMT_U32_LE,
+ V4L2_AUDIO_FMT_S20_3LE,
+ V4L2_AUDIO_FMT_U20_3LE,
+ V4L2_AUDIO_FMT_FLOAT_LE,
+ V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE,
+};
+
+#define NUM_FORMATS ARRAY_SIZE(formats)
+
+static const s64 asrc_v1_m2m_rates[] = {
+ 5512, 8000, 11025, 12000, 16000,
+ 22050, 24000, 32000, 44100,
+ 48000, 64000, 88200, 96000,
+ 128000, 176400, 192000,
+};
+
+static const s64 asrc_v2_m2m_rates[] = {
+ 8000, 11025, 12000, 16000,
+ 22050, 24000, 32000, 44100,
+ 48000, 64000, 88200, 96000,
+ 128000, 176400, 192000, 256000,
+ 352800, 384000, 705600, 768000,
+};
+
+static u32 find_fourcc(snd_pcm_format_t format)
+{
+ snd_pcm_format_t fmt;
+ unsigned int k;
+
+ for (k = 0; k < NUM_FORMATS; k++) {
+ fmt = v4l2_fourcc_to_audfmt(formats[k]);
+ if (fmt == format)
+ return formats[k];
+ }
+
+ return 0;
+}
+
+static snd_pcm_forma
[PATCH v12 13/15] media: vivid: add fixed point test controls
Add fixed point test controls, one is for Q4.16 format
another one is for Q63 format.
Signed-off-by: Shengjiu Wang
---
drivers/media/test-drivers/vivid/vivid-core.h | 2 ++
.../media/test-drivers/vivid/vivid-ctrls.c| 26 +++
include/media/v4l2-ctrls.h| 6 +
3 files changed, 34 insertions(+)
diff --git a/drivers/media/test-drivers/vivid/vivid-core.h
b/drivers/media/test-drivers/vivid/vivid-core.h
index cfb8e66083f6..f65465191bc9 100644
--- a/drivers/media/test-drivers/vivid/vivid-core.h
+++ b/drivers/media/test-drivers/vivid/vivid-core.h
@@ -222,6 +222,8 @@ struct vivid_dev {
struct v4l2_ctrl*boolean;
struct v4l2_ctrl*int32;
struct v4l2_ctrl*int64;
+ struct v4l2_ctrl*int32_q16;
+ struct v4l2_ctrl*int64_q63;
struct v4l2_ctrl*menu;
struct v4l2_ctrl*string;
struct v4l2_ctrl*bitmask;
diff --git a/drivers/media/test-drivers/vivid/vivid-ctrls.c
b/drivers/media/test-drivers/vivid/vivid-ctrls.c
index f2b20e25a7a4..2444ea95b285 100644
--- a/drivers/media/test-drivers/vivid/vivid-ctrls.c
+++ b/drivers/media/test-drivers/vivid/vivid-ctrls.c
@@ -38,6 +38,8 @@
#define VIVID_CID_U8_PIXEL_ARRAY (VIVID_CID_CUSTOM_BASE + 14)
#define VIVID_CID_S32_ARRAY(VIVID_CID_CUSTOM_BASE + 15)
#define VIVID_CID_S64_ARRAY(VIVID_CID_CUSTOM_BASE + 16)
+#define VIVID_CID_INT_Q4_16(VIVID_CID_CUSTOM_BASE + 17)
+#define VIVID_CID_INT64_Q63(VIVID_CID_CUSTOM_BASE + 18)
#define VIVID_CID_VIVID_BASE (0x00f0 | 0xf000)
#define VIVID_CID_VIVID_CLASS (0x00f0 | 1)
@@ -182,6 +184,28 @@ static const struct v4l2_ctrl_config vivid_ctrl_int64 = {
.step = 1,
};
+static const struct v4l2_ctrl_config vivid_ctrl_int32_q16 = {
+ .ops = _user_gen_ctrl_ops,
+ .id = VIVID_CID_INT_Q4_16,
+ .name = "Integer 32 Bits Q4.16",
+ .type = V4L2_CTRL_TYPE_INTEGER,
+ .min = v4l2_ctrl_fp_compose(-16, 0, 16),
+ .max = v4l2_ctrl_fp_compose(15, 0x, 16),
+ .step = 1,
+ .fraction_bits = 16,
+};
+
+static const struct v4l2_ctrl_config vivid_ctrl_int64_q63 = {
+ .ops = _user_gen_ctrl_ops,
+ .id = VIVID_CID_INT64_Q63,
+ .name = "Integer 64 Bits Q63",
+ .type = V4L2_CTRL_TYPE_INTEGER64,
+ .min = v4l2_ctrl_fp_compose(-1, 0, 63),
+ .max = v4l2_ctrl_fp_compose(0, LLONG_MAX, 63),
+ .step = 1,
+ .fraction_bits = 63,
+};
+
static const struct v4l2_ctrl_config vivid_ctrl_u32_array = {
.ops = _user_gen_ctrl_ops,
.id = VIVID_CID_U32_ARRAY,
@@ -1670,6 +1694,8 @@ int vivid_create_controls(struct vivid_dev *dev, bool
show_ccs_cap,
dev->button = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_button,
NULL);
dev->int32 = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_int32,
NULL);
dev->int64 = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_int64,
NULL);
+ dev->int32_q16 = v4l2_ctrl_new_custom(hdl_user_gen,
_ctrl_int32_q16, NULL);
+ dev->int64_q63 = v4l2_ctrl_new_custom(hdl_user_gen,
_ctrl_int64_q63, NULL);
dev->boolean = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_boolean,
NULL);
dev->menu = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_menu, NULL);
dev->string = v4l2_ctrl_new_custom(hdl_user_gen, _ctrl_string,
NULL);
diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
index c35514c5bf88..197d8b67ac13 100644
--- a/include/media/v4l2-ctrls.h
+++ b/include/media/v4l2-ctrls.h
@@ -1593,4 +1593,10 @@ void v4l2_ctrl_type_op_log(const struct v4l2_ctrl *ctrl);
*/
int v4l2_ctrl_type_op_validate(const struct v4l2_ctrl *ctrl, union
v4l2_ctrl_ptr ptr);
+/*
+ * Fixed point compose helper define. This helper maps to the value
+ * i + f / (1 << fraction_bits).
+ */
+#define v4l2_ctrl_fp_compose(i, f, fraction_bits) (((s64)(i) << fraction_bits)
+ (f))
+
#endif
--
2.34.1
[PATCH v12 12/15] media: uapi: Add an entity type for audio resampler
Add and document a media entity type for an audio resampler.
It is MEDIA_ENT_F_PROC_AUDIO_RESAMPLER.
Signed-off-by: Shengjiu Wang
---
Documentation/userspace-api/media/mediactl/media-types.rst | 6 ++
include/uapi/linux/media.h | 1 +
2 files changed, 7 insertions(+)
diff --git a/Documentation/userspace-api/media/mediactl/media-types.rst
b/Documentation/userspace-api/media/mediactl/media-types.rst
index f0880aea41d6..9ca6542517d5 100644
--- a/Documentation/userspace-api/media/mediactl/media-types.rst
+++ b/Documentation/userspace-api/media/mediactl/media-types.rst
@@ -40,6 +40,7 @@ Types and flags used to represent the media graph elements
.. _MEDIA-ENT-F-PROC-VIDEO-ENCODER:
.. _MEDIA-ENT-F-PROC-VIDEO-DECODER:
.. _MEDIA-ENT-F-PROC-VIDEO-ISP:
+.. _MEDIA-ENT-F-PROC-AUDIO-RESAMPLER:
.. _MEDIA-ENT-F-VID-MUX:
.. _MEDIA-ENT-F-VID-IF-BRIDGE:
.. _MEDIA-ENT-F-DV-DECODER:
@@ -208,6 +209,11 @@ Types and flags used to represent the media graph elements
combination of custom V4L2 controls and IOCTLs, and parameters
supplied in a metadata buffer.
+* - ``MEDIA_ENT_F_PROC_AUDIO_RESAMPLER``
+ - An Audio Resampler device. An entity capable of
+ resampling an audio stream from one sample rate to another sample
+ rate. Must have one sink pad and at least one source pad.
+
* - ``MEDIA_ENT_F_VID_MUX``
- Video multiplexer. An entity capable of multiplexing must have at
least two sink pads and one source pad, and must pass the video
diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
index 9ff6dec7393a..a8266eaa8042 100644
--- a/include/uapi/linux/media.h
+++ b/include/uapi/linux/media.h
@@ -125,6 +125,7 @@ struct media_device_info {
#define MEDIA_ENT_F_PROC_VIDEO_ENCODER (MEDIA_ENT_F_BASE + 0x4007)
#define MEDIA_ENT_F_PROC_VIDEO_DECODER (MEDIA_ENT_F_BASE + 0x4008)
#define MEDIA_ENT_F_PROC_VIDEO_ISP (MEDIA_ENT_F_BASE + 0x4009)
+#define MEDIA_ENT_F_PROC_AUDIO_RESAMPLER (MEDIA_ENT_F_BASE + 0x400a)
/*
* Switch and bridge entity functions
--
2.34.1
[PATCH v12 11/15] media: uapi: Declare interface types for Audio
Declare the interface types that will be used by Audio.
The type is MEDIA_INTF_T_V4L_AUDIO.
Signed-off-by: Shengjiu Wang
---
.../userspace-api/media/mediactl/media-types.rst| 5 +
drivers/media/v4l2-core/v4l2-dev.c | 4
drivers/media/v4l2-core/v4l2-mem2mem.c | 13 +
include/uapi/linux/media.h | 1 +
4 files changed, 19 insertions(+), 4 deletions(-)
diff --git a/Documentation/userspace-api/media/mediactl/media-types.rst
b/Documentation/userspace-api/media/mediactl/media-types.rst
index 0ffeece1e0c8..f0880aea41d6 100644
--- a/Documentation/userspace-api/media/mediactl/media-types.rst
+++ b/Documentation/userspace-api/media/mediactl/media-types.rst
@@ -265,6 +265,7 @@ Types and flags used to represent the media graph elements
.. _MEDIA-INTF-T-V4L-SUBDEV:
.. _MEDIA-INTF-T-V4L-SWRADIO:
.. _MEDIA-INTF-T-V4L-TOUCH:
+.. _MEDIA-INTF-T-V4L-AUDIO:
.. _MEDIA-INTF-T-ALSA-PCM-CAPTURE:
.. _MEDIA-INTF-T-ALSA-PCM-PLAYBACK:
.. _MEDIA-INTF-T-ALSA-CONTROL:
@@ -322,6 +323,10 @@ Types and flags used to represent the media graph elements
- Device node interface for Touch device (V4L)
- typically, /dev/v4l-touch?
+* - ``MEDIA_INTF_T_V4L_AUDIO``
+ - Device node interface for Audio device (V4L)
+ - typically, /dev/v4l-audio?
+
* - ``MEDIA_INTF_T_ALSA_PCM_CAPTURE``
- Device node interface for ALSA PCM Capture
- typically, /dev/snd/pcmC?D?c
diff --git a/drivers/media/v4l2-core/v4l2-dev.c
b/drivers/media/v4l2-core/v4l2-dev.c
index bac008fcedc6..ca8462a61e1f 100644
--- a/drivers/media/v4l2-core/v4l2-dev.c
+++ b/drivers/media/v4l2-core/v4l2-dev.c
@@ -844,6 +844,10 @@ static int video_register_media_controller(struct
video_device *vdev)
intf_type = MEDIA_INTF_T_V4L_SUBDEV;
/* Entity will be created via v4l2_device_register_subdev() */
break;
+ case VFL_TYPE_AUDIO:
+ intf_type = MEDIA_INTF_T_V4L_AUDIO;
+ /* Entity will be created via v4l2_device_register_subdev() */
+ break;
default:
return 0;
}
diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c
b/drivers/media/v4l2-core/v4l2-mem2mem.c
index 9e983176542b..e899674c7d22 100644
--- a/drivers/media/v4l2-core/v4l2-mem2mem.c
+++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
@@ -1137,10 +1137,15 @@ int v4l2_m2m_register_media_controller(struct
v4l2_m2m_dev *m2m_dev,
if (ret)
goto err_rm_links0;
- /* Create video interface */
- m2m_dev->intf_devnode = media_devnode_create(mdev,
- MEDIA_INTF_T_V4L_VIDEO, 0,
- VIDEO_MAJOR, vdev->minor);
+ if (vdev->vfl_type == VFL_TYPE_AUDIO)
+ m2m_dev->intf_devnode = media_devnode_create(mdev,
+ MEDIA_INTF_T_V4L_AUDIO, 0,
+ VIDEO_MAJOR, vdev->minor);
+ else
+ /* Create video interface */
+ m2m_dev->intf_devnode = media_devnode_create(mdev,
+ MEDIA_INTF_T_V4L_VIDEO, 0,
+ VIDEO_MAJOR, vdev->minor);
if (!m2m_dev->intf_devnode) {
ret = -ENOMEM;
goto err_rm_links1;
diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
index 1c80b1d6bbaf..9ff6dec7393a 100644
--- a/include/uapi/linux/media.h
+++ b/include/uapi/linux/media.h
@@ -260,6 +260,7 @@ struct media_links_enum {
#define MEDIA_INTF_T_V4L_SUBDEV(MEDIA_INTF_T_V4L_BASE
+ 3)
#define MEDIA_INTF_T_V4L_SWRADIO (MEDIA_INTF_T_V4L_BASE + 4)
#define MEDIA_INTF_T_V4L_TOUCH (MEDIA_INTF_T_V4L_BASE + 5)
+#define MEDIA_INTF_T_V4L_AUDIO (MEDIA_INTF_T_V4L_BASE + 6)
#define MEDIA_INTF_T_ALSA_BASE 0x0300
#define MEDIA_INTF_T_ALSA_PCM_CAPTURE (MEDIA_INTF_T_ALSA_BASE)
--
2.34.1
[PATCH v12 10/15] media: uapi: Add audio rate controls support
Add V4L2_CID_M2M_AUDIO_SOURCE_RATE and V4L2_CID_M2M_AUDIO_DEST_RATE
new IDs for rate control.
Add V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET and
V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET for clock drift.
Signed-off-by: Shengjiu Wang
---
.../media/v4l/ext-ctrls-audio-m2m.rst | 20 +++
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 6 ++
include/uapi/linux/v4l2-controls.h| 5 +
3 files changed, 31 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
index 82d2ecedbfee..de579ab8fb94 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
@@ -19,3 +19,23 @@ Audio M2M Control IDs
The Audio M2M class descriptor. Calling
:ref:`VIDIOC_QUERYCTRL` for this control will
return a description of this control class.
+
+.. _v4l2-audio-asrc:
+
+``V4L2_CID_M2M_AUDIO_SOURCE_RATE (integer menu)``
+Sets the audio source sample rate, unit is Hz
+
+``V4L2_CID_M2M_AUDIO_DEST_RATE (integer menu)``
+Sets the audio destination sample rate, unit is Hz
+
+``V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET (fixed point)``
+Sets the offset from the audio source sample rate, unit is Hz.
+The offset compensates for any clock drift. The actual source audio
+sample rate is the ideal source audio sample rate from
+``V4L2_CID_M2M_AUDIO_SOURCE_RATE`` plus this fixed point offset.
+
+``V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET (fixed point)``
+Sets the offset from the audio destination sample rate, unit is Hz.
+The offset compensates for any clock drift. The actual destination audio
+sample rate is the ideal source audio sample rate from
+``V4L2_CID_M2M_AUDIO_DEST_RATE`` plus this fixed point offset.
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index 2a85ea3dc92f..91e1f5348c23 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -1245,6 +1245,8 @@ const char *v4l2_ctrl_get_name(u32 id)
/* Audio M2M controls */
case V4L2_CID_M2M_AUDIO_CLASS: return "Audio M2M Controls";
+ case V4L2_CID_M2M_AUDIO_SOURCE_RATE:return "Audio Source Sample
Rate";
+ case V4L2_CID_M2M_AUDIO_DEST_RATE: return "Audio Destination
Sample Rate";
default:
return NULL;
}
@@ -1606,6 +1608,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum
v4l2_ctrl_type *type,
case V4L2_CID_COLORIMETRY_HDR10_MASTERING_DISPLAY:
*type = V4L2_CTRL_TYPE_HDR10_MASTERING_DISPLAY;
break;
+ case V4L2_CID_M2M_AUDIO_SOURCE_RATE:
+ case V4L2_CID_M2M_AUDIO_DEST_RATE:
+ *type = V4L2_CTRL_TYPE_INTEGER_MENU;
+ break;
default:
*type = V4L2_CTRL_TYPE_INTEGER;
break;
diff --git a/include/uapi/linux/v4l2-controls.h
b/include/uapi/linux/v4l2-controls.h
index a8b4b830c757..30129ccdc282 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -3495,6 +3495,11 @@ struct v4l2_ctrl_av1_film_grain {
#define V4L2_CID_M2M_AUDIO_CLASS_BASE (V4L2_CTRL_CLASS_M2M_AUDIO | 0x900)
#define V4L2_CID_M2M_AUDIO_CLASS (V4L2_CTRL_CLASS_M2M_AUDIO | 1)
+#define V4L2_CID_M2M_AUDIO_SOURCE_RATE (V4L2_CID_M2M_AUDIO_CLASS_BASE + 0)
+#define V4L2_CID_M2M_AUDIO_DEST_RATE (V4L2_CID_M2M_AUDIO_CLASS_BASE + 1)
+#define V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET (V4L2_CID_M2M_AUDIO_CLASS_BASE
+ 2)
+#define V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET(V4L2_CID_M2M_AUDIO_CLASS_BASE
+ 3)
+
/* MPEG-compression definitions kept for backwards compatibility */
#ifndef __KERNEL__
#define V4L2_CTRL_CLASS_MPEGV4L2_CTRL_CLASS_CODEC
--
2.34.1
[PATCH v12 09/15] media: uapi: Add V4L2_CTRL_CLASS_M2M_AUDIO
The Audio M2M class includes controls for audio memory-to-memory
use cases. The controls can be used for audio codecs, audio
preprocessing, audio postprocessing.
Signed-off-by: Shengjiu Wang
---
.../userspace-api/media/v4l/common.rst| 1 +
.../media/v4l/ext-ctrls-audio-m2m.rst | 21 +++
.../media/v4l/vidioc-g-ext-ctrls.rst | 4
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 4
include/uapi/linux/v4l2-controls.h| 4
5 files changed, 34 insertions(+)
create mode 100644
Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
diff --git a/Documentation/userspace-api/media/v4l/common.rst
b/Documentation/userspace-api/media/v4l/common.rst
index ea0435182e44..d5366e96a596 100644
--- a/Documentation/userspace-api/media/v4l/common.rst
+++ b/Documentation/userspace-api/media/v4l/common.rst
@@ -52,6 +52,7 @@ applicable to all devices.
ext-ctrls-fm-rx
ext-ctrls-detect
ext-ctrls-colorimetry
+ext-ctrls-audio-m2m
fourcc
format
planar-apis
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
new file mode 100644
index ..82d2ecedbfee
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
@@ -0,0 +1,21 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+
+.. _audiom2m-controls:
+
+***
+Audio M2M Control Reference
+***
+
+The Audio M2M class includes controls for audio memory-to-memory
+use cases. The controls can be used for audio codecs, audio
+preprocessing, audio postprocessing.
+
+Audio M2M Control IDs
+---
+
+.. _audiom2m-control-id:
+
+``V4L2_CID_M2M_AUDIO_CLASS (class)``
+The Audio M2M class descriptor. Calling
+:ref:`VIDIOC_QUERYCTRL` for this control will
+return a description of this control class.
diff --git a/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
b/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
index 4d56c0528ad7..aeb1ad8e7d29 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-g-ext-ctrls.rst
@@ -488,6 +488,10 @@ still cause this situation.
- 0xa5
- The class containing colorimetry controls. These controls are
described in :ref:`colorimetry-controls`.
+* - ``V4L2_CTRL_CLASS_M2M_AUDIO``
+ - 0xa6
+ - The class containing audio m2m controls. These controls are
+ described in :ref:`audiom2m-controls`.
Return Value
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index 8696eb1cdd61..2a85ea3dc92f 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -1242,6 +1242,9 @@ const char *v4l2_ctrl_get_name(u32 id)
case V4L2_CID_COLORIMETRY_CLASS:return "Colorimetry Controls";
case V4L2_CID_COLORIMETRY_HDR10_CLL_INFO: return "HDR10
Content Light Info";
case V4L2_CID_COLORIMETRY_HDR10_MASTERING_DISPLAY: return "HDR10
Mastering Display";
+
+ /* Audio M2M controls */
+ case V4L2_CID_M2M_AUDIO_CLASS: return "Audio M2M Controls";
default:
return NULL;
}
@@ -1451,6 +1454,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum
v4l2_ctrl_type *type,
case V4L2_CID_DETECT_CLASS:
case V4L2_CID_CODEC_STATELESS_CLASS:
case V4L2_CID_COLORIMETRY_CLASS:
+ case V4L2_CID_M2M_AUDIO_CLASS:
*type = V4L2_CTRL_TYPE_CTRL_CLASS;
/* You can neither read nor write these */
*flags |= V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_WRITE_ONLY;
diff --git a/include/uapi/linux/v4l2-controls.h
b/include/uapi/linux/v4l2-controls.h
index 99c3f5e99da7..a8b4b830c757 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -30,6 +30,7 @@
#define V4L2_CTRL_CLASS_DETECT 0x00a3 /* Detection controls */
#define V4L2_CTRL_CLASS_CODEC_STATELESS 0x00a4 /* Stateless codecs
controls */
#define V4L2_CTRL_CLASS_COLORIMETRY0x00a5 /* Colorimetry controls
*/
+#define V4L2_CTRL_CLASS_M2M_AUDIO 0x00a6 /* Audio M2M controls */
/* User-class control IDs */
@@ -3491,6 +3492,9 @@ struct v4l2_ctrl_av1_film_grain {
__u8 reserved[4];
};
+#define V4L2_CID_M2M_AUDIO_CLASS_BASE (V4L2_CTRL_CLASS_M2M_AUDIO | 0x900)
+#define V4L2_CID_M2M_AUDIO_CLASS (V4L2_CTRL_CLASS_M2M_AUDIO | 1)
+
/* MPEG-compression definitions kept for backwards compatibility */
#ifndef __KERNEL__
#define V4L2_CTRL_CLASS_MPEGV4L2_CTRL_CLASS_CODEC
--
2.34.1
[PATCH v12 08/15] media: uapi: Define audio sample format fourcc type
The audio sample format definition is from alsa,
the header file is include/uapi/sound/asound.h, but
don't include this header file directly, because in
user space, there is another copy in alsa-lib.
There will be conflict in userspace for include
videodev2.h & asound.h and asoundlib.h
Here still use the fourcc format.
Signed-off-by: Shengjiu Wang
---
.../userspace-api/media/v4l/pixfmt-audio.rst | 87 +++
.../userspace-api/media/v4l/pixfmt.rst| 1 +
drivers/media/v4l2-core/v4l2-ioctl.c | 13 +++
include/uapi/linux/videodev2.h| 23 +
4 files changed, 124 insertions(+)
create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
new file mode 100644
index ..04b4a7fbd8f4
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
@@ -0,0 +1,87 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+
+.. _pixfmt-audio:
+
+*
+Audio Formats
+*
+
+These formats are used for :ref:`audiomem2mem` interface only.
+
+.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}|
+
+.. cssclass:: longtable
+
+.. flat-table:: Audio Format
+:header-rows: 1
+:stub-columns: 0
+:widths: 3 1 4
+
+* - Identifier
+ - Code
+ - Details
+* .. _V4L2-AUDIO-FMT-S8:
+
+ - ``V4L2_AUDIO_FMT_S8``
+ - 'S8'
+ - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA
+* .. _V4L2-AUDIO-FMT-S16-LE:
+
+ - ``V4L2_AUDIO_FMT_S16_LE``
+ - 'S16_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA
+* .. _V4L2-AUDIO-FMT-U16-LE:
+
+ - ``V4L2_AUDIO_FMT_U16_LE``
+ - 'U16_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA
+* .. _V4L2-AUDIO-FMT-S24-LE:
+
+ - ``V4L2_AUDIO_FMT_S24_LE``
+ - 'S24_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA
+* .. _V4L2-AUDIO-FMT-U24-LE:
+
+ - ``V4L2_AUDIO_FMT_U24_LE``
+ - 'U24_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA
+* .. _V4L2-AUDIO-FMT-S32-LE:
+
+ - ``V4L2_AUDIO_FMT_S32_LE``
+ - 'S32_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA
+* .. _V4L2-AUDIO-FMT-U32-LE:
+
+ - ``V4L2_AUDIO_FMT_U32_LE``
+ - 'U32_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA
+* .. _V4L2-AUDIO-FMT-FLOAT-LE:
+
+ - ``V4L2_AUDIO_FMT_FLOAT_LE``
+ - 'FLOAT_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_FLOAT_LE in ALSA
+* .. _V4L2-AUDIO-FMT-IEC958-SUBFRAME-LE:
+
+ - ``V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE``
+ - 'IEC958_SUBFRAME_LE'
+ - Corresponds to SNDRV_PCM_FORMAT_IEC958_SUBFRAME_LE in ALSA
+* .. _V4L2-AUDIO-FMT-S24-3LE:
+
+ - ``V4L2_AUDIO_FMT_S24_3LE``
+ - 'S24_3LE'
+ - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
+* .. _V4L2-AUDIO-FMT-U24-3LE:
+
+ - ``V4L2_AUDIO_FMT_U24_3LE``
+ - 'U24_3LE'
+ - Corresponds to SNDRV_PCM_FORMAT_U24_3LE in ALSA
+* .. _V4L2-AUDIO-FMT-S20-3LE:
+
+ - ``V4L2_AUDIO_FMT_S20_3LE``
+ - 'S20_3LE'
+ - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
+* .. _V4L2-AUDIO-FMT-U20-3LE:
+
+ - ``V4L2_AUDIO_FMT_U20_3LE``
+ - 'U20_3LE'
+ - Corresponds to SNDRV_PCM_FORMAT_U20_3LE in ALSA
diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst
b/Documentation/userspace-api/media/v4l/pixfmt.rst
index 11dab4a90630..2eb6fdd3b43d 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt.rst
@@ -36,3 +36,4 @@ see also :ref:`VIDIOC_G_FBUF `.)
colorspaces
colorspaces-defs
colorspaces-details
+pixfmt-audio
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
b/drivers/media/v4l2-core/v4l2-ioctl.c
index e7be7c2f302d..e5094f0e75c1 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1471,6 +1471,19 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_Y210: descr = "10-bit YUYV Packed"; break;
case V4L2_PIX_FMT_Y212: descr = "12-bit YUYV Packed"; break;
case V4L2_PIX_FMT_Y216: descr = "16-bit YUYV Packed"; break;
+ case V4L2_AUDIO_FMT_S8: descr = "8-bit Signed"; break;
+ case V4L2_AUDIO_FMT_S16_LE: descr = "16-bit Signed LE"; break;
+ case V4L2_AUDIO_FMT_U16_LE: descr = "16-bit Unsigned LE";
break;
+ case V4L2_AUDIO_FMT_S24_LE: descr = "24(32)-bit Signed LE";
break;
+ case V4L2_AUDIO_FMT_U24_LE: descr = "24(32)-bit Unsigned
LE"; break;
+ case V4L2_AUDIO_FMT_S32_LE: descr = "32-bit Signed LE";
break;
+ case V4L2_AUDIO_FMT_U32_LE: descr = "32-bit Unsigned LE";
break;
+ case V4L2_AUDIO_FMT_FLOAT_LE:
[PATCH v12 07/15] media: v4l2: Add audio capture and output support
Audio signal processing has the requirement for memory to
memory similar as Video.
This patch is to add this support in v4l2 framework, defined
new buffer type V4L2_BUF_TYPE_AUDIO_CAPTURE and
V4L2_BUF_TYPE_AUDIO_OUTPUT, defined new format v4l2_audio_format
for audio case usage.
The created audio device is named "/dev/v4l-audioX".
Signed-off-by: Shengjiu Wang
---
.../userspace-api/media/v4l/buffer.rst| 6 ++
.../media/v4l/dev-audio-mem2mem.rst | 71 +++
.../userspace-api/media/v4l/devices.rst | 1 +
.../media/v4l/vidioc-enum-fmt.rst | 2 +
.../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 ++
.../media/videodev2.h.rst.exceptions | 2 +
.../media/common/videobuf2/videobuf2-v4l2.c | 4 ++
drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 9 +++
drivers/media/v4l2-core/v4l2-dev.c| 17 +
drivers/media/v4l2-core/v4l2-ioctl.c | 53 ++
include/media/v4l2-dev.h | 2 +
include/media/v4l2-ioctl.h| 34 +
include/uapi/linux/videodev2.h| 17 +
13 files changed, 222 insertions(+)
create mode 100644 Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
diff --git a/Documentation/userspace-api/media/v4l/buffer.rst
b/Documentation/userspace-api/media/v4l/buffer.rst
index 52bbee81c080..a3754ca6f0d6 100644
--- a/Documentation/userspace-api/media/v4l/buffer.rst
+++ b/Documentation/userspace-api/media/v4l/buffer.rst
@@ -438,6 +438,12 @@ enum v4l2_buf_type
* - ``V4L2_BUF_TYPE_META_OUTPUT``
- 14
- Buffer for metadata output, see :ref:`metadata`.
+* - ``V4L2_BUF_TYPE_AUDIO_CAPTURE``
+ - 15
+ - Buffer for audio capture, see :ref:`audio`.
+* - ``V4L2_BUF_TYPE_AUDIO_OUTPUT``
+ - 16
+ - Buffer for audio output, see :ref:`audio`.
.. _buffer-flags:
diff --git a/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
new file mode 100644
index ..68faecfe3a02
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
@@ -0,0 +1,71 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+
+.. _audiomem2mem:
+
+
+Audio Memory-To-Memory Interface
+
+
+An audio memory-to-memory device can compress, decompress, transform, or
+otherwise convert audio data from one format into another format, in memory.
+Such memory-to-memory devices set the ``V4L2_CAP_AUDIO_M2M`` capability.
+Examples of memory-to-memory devices are audio codecs, audio preprocessing,
+audio postprocessing.
+
+A memory-to-memory audio node supports both output (sending audio frames from
+memory to the hardware) and capture (receiving the processed audio frames
+from the hardware into memory) stream I/O. An application will have to
+setup the stream I/O for both sides and finally call
+:ref:`VIDIOC_STREAMON ` for both capture and output to
+start the hardware.
+
+Memory-to-memory devices function as a shared resource: you can
+open the audio node multiple times, each application setting up their
+own properties that are local to the file handle, and each can use
+it independently from the others. The driver will arbitrate access to
+the hardware and reprogram it whenever another file handler gets access.
+
+Audio memory-to-memory devices are accessed through character device
+special files named ``/dev/v4l-audio``
+
+Querying Capabilities
+=
+
+Device nodes supporting the audio memory-to-memory interface set the
+``V4L2_CAP_AUDIO_M2M`` flag in the ``device_caps`` field of the
+:c:type:`v4l2_capability` structure returned by the :c:func:`VIDIOC_QUERYCAP`
+ioctl.
+
+Data Format Negotiation
+===
+
+The audio device uses the :ref:`format` ioctls to select the capture format.
+The audio buffer content format is bound to that selected format. In addition
+to the basic :ref:`format` ioctls, the :c:func:`VIDIOC_ENUM_FMT` ioctl must be
+supported as well.
+
+To use the :ref:`format` ioctls applications set the ``type`` field of the
+:c:type:`v4l2_format` structure to ``V4L2_BUF_TYPE_AUDIO_CAPTURE`` or to
+``V4L2_BUF_TYPE_AUDIO_OUTPUT``. Both drivers and applications must set the
+remainder of the :c:type:`v4l2_format` structure to 0.
+
+.. c:type:: v4l2_audio_format
+
+.. tabularcolumns:: |p{1.4cm}|p{2.4cm}|p{13.5cm}|
+
+.. flat-table:: struct v4l2_audio_format
+:header-rows: 0
+:stub-columns: 0
+:widths: 1 1 2
+
+* - __u32
+ - ``pixelformat``
+ - The sample format, set by the application. see :ref:`pixfmt-audio`
+* - __u32
+ - ``channels``
+ - The channel number, set by the application. channel number range is
+[1, 32].
+* - __u32
+ - ``buffersize``
+ - Maximum buffer size in bytes required for data. The value is set by the
+driver.
diff --git a/Doc
[PATCH v12 06/15] media: uapi: Add V4L2_CAP_AUDIO_M2M capability flag
V4L2_CAP_AUDIO_M2M is similar to V4L2_CAP_VIDEO_M2M flag.
It is used for audio memory to memory case.
Signed-off-by: Shengjiu Wang
---
Documentation/userspace-api/media/v4l/vidioc-querycap.rst| 3 +++
Documentation/userspace-api/media/videodev2.h.rst.exceptions | 1 +
include/uapi/linux/videodev2.h | 1 +
3 files changed, 5 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/vidioc-querycap.rst
b/Documentation/userspace-api/media/v4l/vidioc-querycap.rst
index 6c57b8428356..1c0d97bf192a 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-querycap.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-querycap.rst
@@ -173,6 +173,9 @@ specification the ioctl returns an ``EINVAL`` error code.
interface. A video overlay device typically stores captured images
directly in the video memory of a graphics card, with hardware
clipping and scaling.
+* - ``V4L2_CAP_AUDIO_M2M``
+ - 0x0008
+ - The device supports the audio Memory-To-Memory interface.
* - ``V4L2_CAP_VBI_CAPTURE``
- 0x0010
- The device supports the :ref:`Raw VBI Capture `
diff --git a/Documentation/userspace-api/media/videodev2.h.rst.exceptions
b/Documentation/userspace-api/media/videodev2.h.rst.exceptions
index 3e58aac4ef0b..da6d0b8e4c2c 100644
--- a/Documentation/userspace-api/media/videodev2.h.rst.exceptions
+++ b/Documentation/userspace-api/media/videodev2.h.rst.exceptions
@@ -197,6 +197,7 @@ replace define V4L2_CAP_META_OUTPUT device-capabilities
replace define V4L2_CAP_DEVICE_CAPS device-capabilities
replace define V4L2_CAP_TOUCH device-capabilities
replace define V4L2_CAP_IO_MC device-capabilities
+replace define V4L2_CAP_AUDIO_M2M device-capabilities
# V4L2 pix flags
replace define V4L2_PIX_FMT_PRIV_MAGIC :c:type:`v4l2_pix_format`
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 2c23f0b369e4..6cd65969c2b5 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -473,6 +473,7 @@ struct v4l2_capability {
#define V4L2_CAP_VIDEO_CAPTURE 0x0001 /* Is a video capture
device */
#define V4L2_CAP_VIDEO_OUTPUT 0x0002 /* Is a video output device
*/
#define V4L2_CAP_VIDEO_OVERLAY 0x0004 /* Can do video overlay */
+#define V4L2_CAP_AUDIO_M2M 0x0008 /* audio memory to memory */
#define V4L2_CAP_VBI_CAPTURE 0x0010 /* Is a raw VBI capture
device */
#define V4L2_CAP_VBI_OUTPUT0x0020 /* Is a raw VBI output
device */
#define V4L2_CAP_SLICED_VBI_CAPTURE0x0040 /* Is a sliced VBI capture
device */
--
2.34.1
[PATCH v12 05/15] ASoC: fsl_easrc: register m2m platform device
Register m2m platform device,that user can
use M2M feature.
Signed-off-by: Shengjiu Wang
Acked-by: Mark Brown
---
sound/soc/fsl/fsl_easrc.c | 19 +++
1 file changed, 19 insertions(+)
diff --git a/sound/soc/fsl/fsl_easrc.c b/sound/soc/fsl/fsl_easrc.c
index cf7ad30a323b..ccbf45c7abf4 100644
--- a/sound/soc/fsl/fsl_easrc.c
+++ b/sound/soc/fsl/fsl_easrc.c
@@ -2075,6 +2075,7 @@ MODULE_DEVICE_TABLE(of, fsl_easrc_dt_ids);
static int fsl_easrc_probe(struct platform_device *pdev)
{
struct fsl_easrc_priv *easrc_priv;
+ struct fsl_asrc_m2m_pdata m2m_pdata;
struct device *dev = >dev;
struct fsl_asrc *easrc;
struct resource *res;
@@ -2190,6 +2191,19 @@ static int fsl_easrc_probe(struct platform_device *pdev)
goto err_pm_disable;
}
+ m2m_pdata.asrc = easrc;
+ m2m_pdata.fmt_in = FSL_EASRC_FORMATS;
+ m2m_pdata.fmt_out = FSL_EASRC_FORMATS |
SNDRV_PCM_FMTBIT_IEC958_SUBFRAME_LE;
+ m2m_pdata.rate_min = 8000;
+ m2m_pdata.rate_max = 768000;
+ m2m_pdata.chan_min = 1;
+ m2m_pdata.chan_max = 32;
+ easrc->m2m_pdev = platform_device_register_data(>dev,
+ M2M_DRV_NAME,
+ PLATFORM_DEVID_AUTO,
+ _pdata,
+ sizeof(m2m_pdata));
+
return 0;
err_pm_disable:
@@ -2199,6 +2213,11 @@ static int fsl_easrc_probe(struct platform_device *pdev)
static void fsl_easrc_remove(struct platform_device *pdev)
{
+ struct fsl_asrc *easrc = dev_get_drvdata(>dev);
+
+ if (easrc->m2m_pdev && !IS_ERR(easrc->m2m_pdev))
+ platform_device_unregister(easrc->m2m_pdev);
+
pm_runtime_disable(>dev);
}
--
2.34.1
[PATCH v12 04/15] ASoC: fsl_asrc: register m2m platform device
Register m2m platform device, that user can
use M2M feature.
Defined platform data structure and platform
driver name.
Signed-off-by: Shengjiu Wang
Acked-by: Mark Brown
---
include/sound/fsl_asrc_common.h | 23 +++
sound/soc/fsl/fsl_asrc.c| 18 ++
2 files changed, 41 insertions(+)
diff --git a/include/sound/fsl_asrc_common.h b/include/sound/fsl_asrc_common.h
index 3b53d366182f..c709b8906929 100644
--- a/include/sound/fsl_asrc_common.h
+++ b/include/sound/fsl_asrc_common.h
@@ -71,6 +71,7 @@ struct fsl_asrc_pair {
* @dma_params_rx: DMA parameters for receive channel
* @dma_params_tx: DMA parameters for transmit channel
* @pdev: platform device pointer
+ * @m2m_pdev: m2m platform device pointer
* @regmap: regmap handler
* @paddr: physical address to the base address of registers
* @mem_clk: clock source to access register
@@ -103,6 +104,7 @@ struct fsl_asrc {
struct snd_dmaengine_dai_dma_data dma_params_rx;
struct snd_dmaengine_dai_dma_data dma_params_tx;
struct platform_device *pdev;
+ struct platform_device *m2m_pdev;
struct regmap *regmap;
unsigned long paddr;
struct clk *mem_clk;
@@ -139,6 +141,27 @@ struct fsl_asrc {
void *private;
};
+/**
+ * struct fsl_asrc_m2m_pdata - platform data
+ * @asrc: pointer to struct fsl_asrc
+ * @fmt_in: input sample format
+ * @fmt_out: output sample format
+ * @chan_min: minimum channel number
+ * @chan_max: maximum channel number
+ * @rate_min: minimum rate
+ * @rate_max: maximum rete
+ */
+struct fsl_asrc_m2m_pdata {
+ struct fsl_asrc *asrc;
+ u64 fmt_in;
+ u64 fmt_out;
+ int chan_min;
+ int chan_max;
+ int rate_min;
+ int rate_max;
+};
+
+#define M2M_DRV_NAME "fsl_asrc_m2m"
#define DRV_NAME "fsl-asrc-dai"
extern struct snd_soc_component_driver fsl_asrc_component;
diff --git a/sound/soc/fsl/fsl_asrc.c b/sound/soc/fsl/fsl_asrc.c
index 7d8643ee0ba0..5ecb5d869607 100644
--- a/sound/soc/fsl/fsl_asrc.c
+++ b/sound/soc/fsl/fsl_asrc.c
@@ -1187,6 +1187,7 @@ static int fsl_asrc_runtime_suspend(struct device *dev);
static int fsl_asrc_probe(struct platform_device *pdev)
{
struct device_node *np = pdev->dev.of_node;
+ struct fsl_asrc_m2m_pdata m2m_pdata;
struct fsl_asrc_priv *asrc_priv;
struct fsl_asrc *asrc;
struct resource *res;
@@ -1368,6 +1369,18 @@ static int fsl_asrc_probe(struct platform_device *pdev)
goto err_pm_get_sync;
}
+ m2m_pdata.asrc = asrc;
+ m2m_pdata.fmt_in = FSL_ASRC_FORMATS;
+ m2m_pdata.fmt_out = FSL_ASRC_FORMATS | SNDRV_PCM_FMTBIT_S8;
+ m2m_pdata.rate_min = 5512;
+ m2m_pdata.rate_max = 192000;
+ m2m_pdata.chan_min = 1;
+ m2m_pdata.chan_max = 10;
+ asrc->m2m_pdev = platform_device_register_data(>dev,
+ M2M_DRV_NAME,
+ PLATFORM_DEVID_AUTO,
+ _pdata,
+ sizeof(m2m_pdata));
return 0;
err_pm_get_sync:
@@ -1380,6 +1393,11 @@ static int fsl_asrc_probe(struct platform_device *pdev)
static void fsl_asrc_remove(struct platform_device *pdev)
{
+ struct fsl_asrc *asrc = dev_get_drvdata(>dev);
+
+ if (asrc->m2m_pdev && !IS_ERR(asrc->m2m_pdev))
+ platform_device_unregister(asrc->m2m_pdev);
+
pm_runtime_disable(>dev);
if (!pm_runtime_status_suspended(>dev))
fsl_asrc_runtime_suspend(>dev);
--
2.34.1
[PATCH v12 03/15] ASoC: fsl_asrc: move fsl_asrc_common.h to include/sound
Move fsl_asrc_common.h to include/sound that it can be
included from other drivers.
Signed-off-by: Shengjiu Wang
Acked-by: Mark Brown
---
{sound/soc/fsl => include/sound}/fsl_asrc_common.h | 0
sound/soc/fsl/fsl_asrc.h | 2 +-
sound/soc/fsl/fsl_asrc_dma.c | 2 +-
sound/soc/fsl/fsl_easrc.h | 2 +-
4 files changed, 3 insertions(+), 3 deletions(-)
rename {sound/soc/fsl => include/sound}/fsl_asrc_common.h (100%)
diff --git a/sound/soc/fsl/fsl_asrc_common.h b/include/sound/fsl_asrc_common.h
similarity index 100%
rename from sound/soc/fsl/fsl_asrc_common.h
rename to include/sound/fsl_asrc_common.h
diff --git a/sound/soc/fsl/fsl_asrc.h b/sound/soc/fsl/fsl_asrc.h
index 1c492eb237f5..66544624de7b 100644
--- a/sound/soc/fsl/fsl_asrc.h
+++ b/sound/soc/fsl/fsl_asrc.h
@@ -10,7 +10,7 @@
#ifndef _FSL_ASRC_H
#define _FSL_ASRC_H
-#include "fsl_asrc_common.h"
+#include
#define ASRC_M2M_INPUTFIFO_WML 0x4
#define ASRC_M2M_OUTPUTFIFO_WML0x2
diff --git a/sound/soc/fsl/fsl_asrc_dma.c b/sound/soc/fsl/fsl_asrc_dma.c
index f501f47242fb..f067bf1ecea7 100644
--- a/sound/soc/fsl/fsl_asrc_dma.c
+++ b/sound/soc/fsl/fsl_asrc_dma.c
@@ -12,7 +12,7 @@
#include
#include
-#include "fsl_asrc_common.h"
+#include
#define FSL_ASRC_DMABUF_SIZE (256 * 1024)
diff --git a/sound/soc/fsl/fsl_easrc.h b/sound/soc/fsl/fsl_easrc.h
index c9f770862662..a24e540876a4 100644
--- a/sound/soc/fsl/fsl_easrc.h
+++ b/sound/soc/fsl/fsl_easrc.h
@@ -9,7 +9,7 @@
#include
#include
-#include "fsl_asrc_common.h"
+#include
/* EASRC Register Map */
--
2.34.1
[PATCH v12 02/15] ASoC: fsl_easrc: define functions for memory to memory usage
ASRC can be used on memory to memory case, define several
functions for m2m usage and export them as function pointer.
Signed-off-by: Shengjiu Wang
Acked-by: Mark Brown
---
sound/soc/fsl/fsl_easrc.c | 214 ++
sound/soc/fsl/fsl_easrc.h | 4 +
2 files changed, 218 insertions(+)
diff --git a/sound/soc/fsl/fsl_easrc.c b/sound/soc/fsl/fsl_easrc.c
index ec53bda46a46..cf7ad30a323b 100644
--- a/sound/soc/fsl/fsl_easrc.c
+++ b/sound/soc/fsl/fsl_easrc.c
@@ -1861,6 +1861,211 @@ static int fsl_easrc_get_fifo_addr(u8 dir, enum
asrc_pair_index index)
return REG_EASRC_FIFO(dir, index);
}
+/* Get sample numbers in FIFO */
+static unsigned int fsl_easrc_get_output_fifo_size(struct fsl_asrc_pair *pair)
+{
+ struct fsl_asrc *asrc = pair->asrc;
+ enum asrc_pair_index index = pair->index;
+ u32 val;
+
+ regmap_read(asrc->regmap, REG_EASRC_SFS(index), );
+ val &= EASRC_SFS_NSGO_MASK;
+
+ return val >> EASRC_SFS_NSGO_SHIFT;
+}
+
+static int fsl_easrc_m2m_prepare(struct fsl_asrc_pair *pair)
+{
+ struct fsl_easrc_ctx_priv *ctx_priv = pair->private;
+ struct fsl_asrc *asrc = pair->asrc;
+ struct device *dev = >pdev->dev;
+ int ret;
+
+ ctx_priv->in_params.sample_rate = pair->rate[IN];
+ ctx_priv->in_params.sample_format = pair->sample_format[IN];
+ ctx_priv->out_params.sample_rate = pair->rate[OUT];
+ ctx_priv->out_params.sample_format = pair->sample_format[OUT];
+
+ ctx_priv->in_params.fifo_wtmk = FSL_EASRC_INPUTFIFO_WML;
+ ctx_priv->out_params.fifo_wtmk = FSL_EASRC_OUTPUTFIFO_WML;
+ /* Fill the right half of the re-sampler with zeros */
+ ctx_priv->rs_init_mode = 0x2;
+ /* Zero fill the right half of the prefilter */
+ ctx_priv->pf_init_mode = 0x2;
+
+ ret = fsl_easrc_set_ctx_format(pair,
+ _priv->in_params.sample_format,
+ _priv->out_params.sample_format);
+ if (ret) {
+ dev_err(dev, "failed to set context format: %d\n", ret);
+ return ret;
+ }
+
+ ret = fsl_easrc_config_context(asrc, pair->index);
+ if (ret) {
+ dev_err(dev, "failed to config context %d\n", ret);
+ return ret;
+ }
+
+ ctx_priv->in_params.iterations = 1;
+ ctx_priv->in_params.group_len = pair->channels;
+ ctx_priv->in_params.access_len = pair->channels;
+ ctx_priv->out_params.iterations = 1;
+ ctx_priv->out_params.group_len = pair->channels;
+ ctx_priv->out_params.access_len = pair->channels;
+
+ ret = fsl_easrc_set_ctx_organziation(pair);
+ if (ret) {
+ dev_err(dev, "failed to set fifo organization\n");
+ return ret;
+ }
+
+ /* The context start flag */
+ pair->first_convert = 1;
+ return 0;
+}
+
+static int fsl_easrc_m2m_start(struct fsl_asrc_pair *pair)
+{
+ /* start context once */
+ if (pair->first_convert) {
+ fsl_easrc_start_context(pair);
+ pair->first_convert = 0;
+ }
+
+ return 0;
+}
+
+static int fsl_easrc_m2m_stop(struct fsl_asrc_pair *pair)
+{
+ /* Stop pair/context */
+ if (!pair->first_convert) {
+ fsl_easrc_stop_context(pair);
+ pair->first_convert = 1;
+ }
+
+ return 0;
+}
+
+/* calculate capture data length according to output data length and sample
rate */
+static int fsl_easrc_m2m_calc_out_len(struct fsl_asrc_pair *pair, int
input_buffer_length)
+{
+ struct fsl_asrc *easrc = pair->asrc;
+ struct fsl_easrc_priv *easrc_priv = easrc->private;
+ struct fsl_easrc_ctx_priv *ctx_priv = pair->private;
+ unsigned int in_rate = ctx_priv->in_params.norm_rate;
+ unsigned int out_rate = ctx_priv->out_params.norm_rate;
+ unsigned int channels = pair->channels;
+ unsigned int in_samples, out_samples;
+ unsigned int in_width, out_width;
+ unsigned int out_length;
+ unsigned int frac_bits;
+ u64 val1, val2;
+
+ switch (easrc_priv->rs_num_taps) {
+ case EASRC_RS_32_TAPS:
+ /* integer bits = 5; */
+ frac_bits = 39;
+ break;
+ case EASRC_RS_64_TAPS:
+ /* integer bits = 6; */
+ frac_bits = 38;
+ break;
+ case EASRC_RS_128_TAPS:
+ /* integer bits = 7; */
+ frac_bits = 37;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ val1 = (u64)in_rate << frac_bits;
+ do_div(val1, out_rate);
+ val1 += (s64)ctx_priv->ratio_mod << (frac_bits - 31);
+
+ in_width =
snd_pcm_format_physical_width(c
[PATCH v12 00/15] Add audio support in v4l2 framework
Audio signal processing also has the requirement for memory to memory similar as Video. This asrc memory to memory (memory ->asrc->memory) case is a non real time use case. User fills the input buffer to the asrc module, after conversion, then asrc sends back the output buffer to user. So it is not a traditional ALSA playback and capture case. It is a specific use case, there is no reference in current kernel. v4l2 memory to memory is the closed implementation, v4l2 current support video, image, radio, tuner, touch devices, so it is not complicated to add support for this specific audio case. Because we had implemented the "memory -> asrc ->i2s device-> codec" use case in ALSA. Now the "memory->asrc->memory" needs to reuse the code in asrc driver, so the first 3 patches is for refining the code to make it can be shared by the "memory->asrc->memory" driver. The main change is in the v4l2 side, A /dev/vl4-audioX will be created, user applications only use the ioctl of v4l2 framework. Other change is to add memory to memory support for two kinds of i.MX ASRC module. changes in v12 - minor changes according to comments - drop min_buffers_needed = 1 and V4L2_CTRL_FLAG_UPDATE flag - drop bus_info changes in v11 - add add-fixed-point-test-controls in vivid. - add v4l2_ctrl_fp_compose() helper function for min and max changes in v10 - remove FIXED_POINT type - change code base on media: v4l2-ctrls: add support for fraction_bits - fix issue reported by kernel test robot - remove module_alias changes in v9: - add MEDIA_ENT_F_PROC_AUDIO_RESAMPLER. - add MEDIA_INTF_T_V4L_AUDIO - add media controller support - refine the vim2m-audio to support 8k<->16k conversion. changes in v8: - refine V4L2_CAP_AUDIO_M2M to be 0x0008 - update doc for FIXED_POINT - address comments for imx-asrc changes in v7: - add acked-by from Mark - separate commit for fixed point, m2m audio class, audio rate controls - use INTEGER_MENU for rate, FIXED_POINT for rate offset - remove used fmts - address other comments for Hans changes in v6: - use m2m_prepare/m2m_unprepare/m2m_start/m2m_stop to replace m2m_start_part_one/m2m_stop_part_one, m2m_start_part_two/m2m_stop_part_two. - change V4L2_CTRL_TYPE_ASRC_RATE to V4L2_CTRL_TYPE_FIXED_POINT - fix warning by kernel test rebot - remove some unused format V4L2_AUDIO_FMT_XX - Get SNDRV_PCM_FORMAT from V4L2_AUDIO_FMT in driver. - rename audm2m to viaudm2m. changes in v5: - remove V4L2_AUDIO_FMT_LPCM - define audio pixel format like V4L2_AUDIO_FMT_S8... - remove rate and format in struct v4l2_audio_format. - Add V4L2_CID_ASRC_SOURCE_RATE and V4L2_CID_ASRC_DEST_RATE controls - updata document accordingly. changes in v4: - update document style - separate V4L2_AUDIO_FMT_LPCM and V4L2_CAP_AUDIO_M2M in separate commit changes in v3: - Modify documents for adding audio m2m support - Add audio virtual m2m driver - Defined V4L2_AUDIO_FMT_LPCM format type for audio. - Defined V4L2_CAP_AUDIO_M2M capability type for audio m2m case. - with modification in v4l-utils, pass v4l2-compliance test. changes in v2: - decouple the implementation in v4l2 and ALSA - implement the memory to memory driver as a platfrom driver and move it to driver/media - move fsl_asrc_common.h to include/sound folder Shengjiu Wang (15): ASoC: fsl_asrc: define functions for memory to memory usage ASoC: fsl_easrc: define functions for memory to memory usage ASoC: fsl_asrc: move fsl_asrc_common.h to include/sound ASoC: fsl_asrc: register m2m platform device ASoC: fsl_easrc: register m2m platform device media: uapi: Add V4L2_CAP_AUDIO_M2M capability flag media: v4l2: Add audio capture and output support media: uapi: Define audio sample format fourcc type media: uapi: Add V4L2_CTRL_CLASS_M2M_AUDIO media: uapi: Add audio rate controls support media: uapi: Declare interface types for Audio media: uapi: Add an entity type for audio resampler media: vivid: add fixed point test controls media: imx-asrc: Add memory to memory driver media: vim2m-audio: add virtual driver for audio memory to memory .../media/mediactl/media-types.rst| 11 + .../userspace-api/media/v4l/buffer.rst|6 + .../userspace-api/media/v4l/common.rst|1 + .../media/v4l/dev-audio-mem2mem.rst | 71 + .../userspace-api/media/v4l/devices.rst |1 + .../media/v4l/ext-ctrls-audio-m2m.rst | 41 + .../userspace-api/media/v4l/pixfmt-audio.rst | 87 ++ .../userspace-api/media/v4l/pixfmt.rst|1 + .../media/v4l/vidioc-enum-fmt.rst |2 + .../media/v4l/vidioc-g-ext-ctrls.rst |4 + .../userspace-api/media/v4l/vidioc-g-fmt.rst |4 + .../media/v4l/vidioc-querycap.rst |3 + .../media/videodev2.h.rst.exceptions |3 + .../media/common/videobuf2/videobuf2-v4l2.c |4 + drivers/media/platform/nxp/Kconfig| 13 + driver
[PATCH v12 01/15] ASoC: fsl_asrc: define functions for memory to memory usage
ASRC can be used on memory to memory case, define several
functions for m2m usage.
m2m_prepare: prepare for the start step
m2m_start: the start step
m2m_unprepare: unprepare for stop step, optional
m2m_stop: stop step
m2m_check_format: check format is supported or not
m2m_calc_out_len: calculate output length according to input length
m2m_get_maxburst: burst size for dma
m2m_pair_suspend: suspend function of pair, optional.
m2m_pair_resume: resume function of pair
get_output_fifo_size: get remaining data size in FIFO
Signed-off-by: Shengjiu Wang
Acked-by: Mark Brown
---
sound/soc/fsl/fsl_asrc.c| 126
sound/soc/fsl/fsl_asrc.h| 2 +
sound/soc/fsl/fsl_asrc_common.h | 37 ++
3 files changed, 165 insertions(+)
diff --git a/sound/soc/fsl/fsl_asrc.c b/sound/soc/fsl/fsl_asrc.c
index b793263291dc..7d8643ee0ba0 100644
--- a/sound/soc/fsl/fsl_asrc.c
+++ b/sound/soc/fsl/fsl_asrc.c
@@ -1063,6 +1063,124 @@ static int fsl_asrc_get_fifo_addr(u8 dir, enum
asrc_pair_index index)
return REG_ASRDx(dir, index);
}
+/* Get sample numbers in FIFO */
+static unsigned int fsl_asrc_get_output_fifo_size(struct fsl_asrc_pair *pair)
+{
+ struct fsl_asrc *asrc = pair->asrc;
+ enum asrc_pair_index index = pair->index;
+ u32 val;
+
+ regmap_read(asrc->regmap, REG_ASRFST(index), );
+
+ val &= ASRFSTi_OUTPUT_FIFO_MASK;
+
+ return val >> ASRFSTi_OUTPUT_FIFO_SHIFT;
+}
+
+static int fsl_asrc_m2m_prepare(struct fsl_asrc_pair *pair)
+{
+ struct fsl_asrc_pair_priv *pair_priv = pair->private;
+ struct fsl_asrc *asrc = pair->asrc;
+ struct device *dev = >pdev->dev;
+ struct asrc_config config;
+ int ret;
+
+ /* fill config */
+ config.pair = pair->index;
+ config.channel_num = pair->channels;
+ config.input_sample_rate = pair->rate[IN];
+ config.output_sample_rate = pair->rate[OUT];
+ config.input_format = pair->sample_format[IN];
+ config.output_format = pair->sample_format[OUT];
+ config.inclk = INCLK_NONE;
+ config.outclk = OUTCLK_ASRCK1_CLK;
+
+ pair_priv->config =
+ ret = fsl_asrc_config_pair(pair, true);
+ if (ret) {
+ dev_err(dev, "failed to config pair: %d\n", ret);
+ return ret;
+ }
+
+ pair->first_convert = 1;
+
+ return 0;
+}
+
+static int fsl_asrc_m2m_start(struct fsl_asrc_pair *pair)
+{
+ if (pair->first_convert) {
+ fsl_asrc_start_pair(pair);
+ pair->first_convert = 0;
+ }
+ /*
+* Clear DMA request during the stall state of ASRC:
+* During STALL state, the remaining in input fifo would never be
+* smaller than the input threshold while the output fifo would not
+* be bigger than output one. Thus the DMA request would be cleared.
+*/
+ fsl_asrc_set_watermarks(pair, ASRC_FIFO_THRESHOLD_MIN,
+ ASRC_FIFO_THRESHOLD_MAX);
+
+ /* Update the real input threshold to raise DMA request */
+ fsl_asrc_set_watermarks(pair, ASRC_M2M_INPUTFIFO_WML,
+ ASRC_M2M_OUTPUTFIFO_WML);
+
+ return 0;
+}
+
+static int fsl_asrc_m2m_stop(struct fsl_asrc_pair *pair)
+{
+ if (!pair->first_convert) {
+ fsl_asrc_stop_pair(pair);
+ pair->first_convert = 1;
+ }
+
+ return 0;
+}
+
+/* calculate capture data length according to output data length and sample
rate */
+static int fsl_asrc_m2m_calc_out_len(struct fsl_asrc_pair *pair, int
input_buffer_length)
+{
+ unsigned int in_width, out_width;
+ unsigned int channels = pair->channels;
+ unsigned int in_samples, out_samples;
+ unsigned int out_length;
+
+ in_width = snd_pcm_format_physical_width(pair->sample_format[IN]) / 8;
+ out_width = snd_pcm_format_physical_width(pair->sample_format[OUT]) / 8;
+
+ in_samples = input_buffer_length / in_width / channels;
+ out_samples = pair->rate[OUT] * in_samples / pair->rate[IN];
+ out_length = (out_samples - ASRC_OUTPUT_LAST_SAMPLE) * out_width *
channels;
+
+ return out_length;
+}
+
+static int fsl_asrc_m2m_get_maxburst(u8 dir, struct fsl_asrc_pair *pair)
+{
+ struct fsl_asrc *asrc = pair->asrc;
+ struct fsl_asrc_priv *asrc_priv = asrc->private;
+ int wml = (dir == IN) ? ASRC_M2M_INPUTFIFO_WML :
ASRC_M2M_OUTPUTFIFO_WML;
+
+ if (!asrc_priv->soc->use_edma)
+ return wml * pair->channels;
+ else
+ return 1;
+}
+
+static int fsl_asrc_m2m_pair_resume(struct fsl_asrc_pair *pair)
+{
+ struct fsl_asrc *asrc = pair->asrc;
+ int i;
+
+ for (i = 0; i < pair->channels * 4; i++)
+ regmap_write(asrc->regmap, REG_ASRDI(pair->index), 0);
+
+
Re: [PATCH v11 15/15] media: vim2m-audio: add virtual driver for audio memory to memory
On Thu, Jan 18, 2024 at 3:56 PM Hans Verkuil wrote:
>
> On 18/01/2024 07:13, Shengjiu Wang wrote:
> > On Wed, Jan 17, 2024 at 6:32 PM Hans Verkuil wrote:
> >>
> >> On 22/11/2023 08:23, Shengjiu Wang wrote:
> >>> Audio memory to memory virtual driver use video memory to memory
> >>> virtual driver vim2m.c as example. The main difference is
> >>> device type is VFL_TYPE_AUDIO and device cap type is V4L2_CAP_AUDIO_M2M.
> >>>
> >>> The device_run function is a dummy function, which is simply
> >>> copy the data from input buffer to output buffer.
> >>>
> >>> Signed-off-by: Shengjiu Wang
> >>> ---
> >>> drivers/media/test-drivers/Kconfig | 11 +
> >>> drivers/media/test-drivers/Makefile | 1 +
> >>> drivers/media/test-drivers/vim2m-audio.c | 799 +++
> >>> 3 files changed, 811 insertions(+)
> >>> create mode 100644 drivers/media/test-drivers/vim2m-audio.c
> >>>
> >>> diff --git a/drivers/media/test-drivers/Kconfig
> >>> b/drivers/media/test-drivers/Kconfig
> >>> index 459b433e9fae..55f8af6ee4e2 100644
> >>> --- a/drivers/media/test-drivers/Kconfig
> >>> +++ b/drivers/media/test-drivers/Kconfig
> >>> @@ -17,6 +17,17 @@ config VIDEO_VIM2M
> >>> This is a virtual test device for the memory-to-memory driver
> >>> framework.
> >>>
> >>> +config VIDEO_VIM2M_AUDIO
> >>> + tristate "Virtual Memory-to-Memory Driver For Audio"
> >>> + depends on VIDEO_DEV
> >>> + select VIDEOBUF2_VMALLOC
> >>> + select V4L2_MEM2MEM_DEV
> >>> + select MEDIA_CONTROLLER
> >>> + select MEDIA_CONTROLLER_REQUEST_API
> >>
> >> Drop this. This option has been removed.
> >>
> >>> + help
> >>> + This is a virtual audio test device for the memory-to-memory
> >>> driver
> >>> + framework.
> >>> +
> >>> source "drivers/media/test-drivers/vicodec/Kconfig"
> >>> source "drivers/media/test-drivers/vimc/Kconfig"
> >>> source "drivers/media/test-drivers/vivid/Kconfig"
> >>> diff --git a/drivers/media/test-drivers/Makefile
> >>> b/drivers/media/test-drivers/Makefile
> >>> index 740714a4584d..0c61c9ada3e1 100644
> >>> --- a/drivers/media/test-drivers/Makefile
> >>> +++ b/drivers/media/test-drivers/Makefile
> >>> @@ -10,6 +10,7 @@ obj-$(CONFIG_DVB_VIDTV) += vidtv/
> >>>
> >>> obj-$(CONFIG_VIDEO_VICODEC) += vicodec/
> >>> obj-$(CONFIG_VIDEO_VIM2M) += vim2m.o
> >>> +obj-$(CONFIG_VIDEO_VIM2M_AUDIO) += vim2m-audio.o
> >>> obj-$(CONFIG_VIDEO_VIMC) += vimc/
> >>> obj-$(CONFIG_VIDEO_VIVID) += vivid/
> >>> obj-$(CONFIG_VIDEO_VISL) += visl/
> >>> diff --git a/drivers/media/test-drivers/vim2m-audio.c
> >>> b/drivers/media/test-drivers/vim2m-audio.c
> >>> new file mode 100644
> >>> index ..72806ada8628
> >>> --- /dev/null
> >>> +++ b/drivers/media/test-drivers/vim2m-audio.c
> >>> @@ -0,0 +1,799 @@
> >>> +// SPDX-License-Identifier: GPL-2.0+
> >>> +/*
> >>> + * A virtual v4l2-mem2mem example for audio device.
> >>> + */
> >>> +
> >>> +#include
> >>> +#include
> >>> +#include
> >>> +#include
> >>> +#include
> >>> +
> >>> +#include
> >>> +#include
> >>> +#include
> >>> +#include
> >>> +#include
> >>> +#include
> >>> +#include
> >>> +#include
> >>> +
> >>> +MODULE_DESCRIPTION("Virtual device for audio mem2mem testing");
> >>> +MODULE_LICENSE("GPL");
> >>> +
> >>> +static unsigned int debug;
> >>> +module_param(debug, uint, 0644);
> >>> +MODULE_PARM_DESC(debug, "debug level");
> >>> +
> >>> +#define MEM2MEM_NAME "vim2m-audio"
> >>> +
> >>> +#define dprintk(dev, lvl, fmt, arg...) \
> >>> + v4l2_dbg(lvl, debug, &(dev)->v4l2_dev, "%s: " fmt, __func__, ## arg)
> >>> +
> >>> +#define SAMPLE_NUM 4096
> >>> +
> >>> +static void audm2m_dev_release(struct device *dev)
> >>> +{}
&
Re: [PATCH v11 15/15] media: vim2m-audio: add virtual driver for audio memory to memory
On Wed, Jan 17, 2024 at 6:32 PM Hans Verkuil wrote:
>
> On 22/11/2023 08:23, Shengjiu Wang wrote:
> > Audio memory to memory virtual driver use video memory to memory
> > virtual driver vim2m.c as example. The main difference is
> > device type is VFL_TYPE_AUDIO and device cap type is V4L2_CAP_AUDIO_M2M.
> >
> > The device_run function is a dummy function, which is simply
> > copy the data from input buffer to output buffer.
> >
> > Signed-off-by: Shengjiu Wang
> > ---
> > drivers/media/test-drivers/Kconfig | 11 +
> > drivers/media/test-drivers/Makefile | 1 +
> > drivers/media/test-drivers/vim2m-audio.c | 799 +++
> > 3 files changed, 811 insertions(+)
> > create mode 100644 drivers/media/test-drivers/vim2m-audio.c
> >
> > diff --git a/drivers/media/test-drivers/Kconfig
> > b/drivers/media/test-drivers/Kconfig
> > index 459b433e9fae..55f8af6ee4e2 100644
> > --- a/drivers/media/test-drivers/Kconfig
> > +++ b/drivers/media/test-drivers/Kconfig
> > @@ -17,6 +17,17 @@ config VIDEO_VIM2M
> > This is a virtual test device for the memory-to-memory driver
> > framework.
> >
> > +config VIDEO_VIM2M_AUDIO
> > + tristate "Virtual Memory-to-Memory Driver For Audio"
> > + depends on VIDEO_DEV
> > + select VIDEOBUF2_VMALLOC
> > + select V4L2_MEM2MEM_DEV
> > + select MEDIA_CONTROLLER
> > + select MEDIA_CONTROLLER_REQUEST_API
>
> Drop this. This option has been removed.
>
> > + help
> > + This is a virtual audio test device for the memory-to-memory driver
> > + framework.
> > +
> > source "drivers/media/test-drivers/vicodec/Kconfig"
> > source "drivers/media/test-drivers/vimc/Kconfig"
> > source "drivers/media/test-drivers/vivid/Kconfig"
> > diff --git a/drivers/media/test-drivers/Makefile
> > b/drivers/media/test-drivers/Makefile
> > index 740714a4584d..0c61c9ada3e1 100644
> > --- a/drivers/media/test-drivers/Makefile
> > +++ b/drivers/media/test-drivers/Makefile
> > @@ -10,6 +10,7 @@ obj-$(CONFIG_DVB_VIDTV) += vidtv/
> >
> > obj-$(CONFIG_VIDEO_VICODEC) += vicodec/
> > obj-$(CONFIG_VIDEO_VIM2M) += vim2m.o
> > +obj-$(CONFIG_VIDEO_VIM2M_AUDIO) += vim2m-audio.o
> > obj-$(CONFIG_VIDEO_VIMC) += vimc/
> > obj-$(CONFIG_VIDEO_VIVID) += vivid/
> > obj-$(CONFIG_VIDEO_VISL) += visl/
> > diff --git a/drivers/media/test-drivers/vim2m-audio.c
> > b/drivers/media/test-drivers/vim2m-audio.c
> > new file mode 100644
> > index ..72806ada8628
> > --- /dev/null
> > +++ b/drivers/media/test-drivers/vim2m-audio.c
> > @@ -0,0 +1,799 @@
> > +// SPDX-License-Identifier: GPL-2.0+
> > +/*
> > + * A virtual v4l2-mem2mem example for audio device.
> > + */
> > +
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +
> > +MODULE_DESCRIPTION("Virtual device for audio mem2mem testing");
> > +MODULE_LICENSE("GPL");
> > +
> > +static unsigned int debug;
> > +module_param(debug, uint, 0644);
> > +MODULE_PARM_DESC(debug, "debug level");
> > +
> > +#define MEM2MEM_NAME "vim2m-audio"
> > +
> > +#define dprintk(dev, lvl, fmt, arg...) \
> > + v4l2_dbg(lvl, debug, &(dev)->v4l2_dev, "%s: " fmt, __func__, ## arg)
> > +
> > +#define SAMPLE_NUM 4096
> > +
> > +static void audm2m_dev_release(struct device *dev)
> > +{}
> > +
> > +static struct platform_device audm2m_pdev = {
> > + .name = MEM2MEM_NAME,
> > + .dev.release= audm2m_dev_release,
> > +};
> > +
> > +static u32 formats[] = {
> > + V4L2_AUDIO_FMT_S16_LE,
> > +};
> > +
> > +#define NUM_FORMATS ARRAY_SIZE(formats)
> > +
> > +/* Per-queue, driver-specific private data */
> > +struct audm2m_q_data {
> > + unsigned intrate;
> > + unsigned intchannels;
> > + unsigned intbuffersize;
> > + unsigned intsequence;
> > + u32 fourcc;
> > +};
> > +
> > +enum {
> > + V4L2_M2M_SRC = 0,
> > + V4L2_M2M_DST = 1,
> > +};
> > +
> > +static snd_pcm_format_t find_format(u32 fourcc)
> > +{
> > + s
