Re: [Freetel-codec2] Updating the libcodec2 binding in FFmpeg, fixed-point decoding
lör 2023-12-30 klockan 12:44 +1100 skrev glen english LIST: > It's an interesting idea. For real time work, taking it to the > extreme, > putting a bit of latency in there- > > the encoder and decoder pair, could over time , generate shortcut > tokens > / lookup for whole words > > IE encoder continuously stores the encoded words. (it doesnt know the > word, only the sequence ) and puts marker tags on them, and > transmits > the marker tag with the encoded sequence > > at the other end, the decoder continously stores the sequences and > the > associated tag. > > WHen the encoder finds a fitting sequence that's already been encoded > and a close fit to work already down, it only sends the token tag to > the decoder to look up and reproduce. > > IE a form of dynamic lookup table. There must be a name for doing > this > in computer science ??? This is more or less how Lempel-Ziv works /Tomas ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Updating the libcodec2 binding in FFmpeg, fixed-point decoding
I'm wondering how much of a data size difference there is between this proposed sound encoding, the International Phonetic Alphabet, and ASCII of spelled words? Brian On Friday, December 29, 2023 at 08:46:43 PM EST, glen english LIST wrote: It's an interesting idea. For real time work, taking it to the extreme, putting a bit of latency in there- the encoder and decoder pair, could over time , generate shortcut tokens / lookup for whole words IE encoder continuously stores the encoded words. (it doesnt know the word, only the sequence ) and puts marker tags on them, and transmits the marker tag with the encoded sequence at the other end, the decoder continously stores the sequences and the associated tag. WHen the encoder finds a fitting sequence that's already been encoded and a close fit to work already down, it only sends the token tag to the decoder to look up and reproduce. IE a form of dynamic lookup table. There must be a name for doing this in computer science ??? glen On 29/12/2023 11:54 pm, Tomas Härdin wrote: > fre 2023-12-29 klockan 10:44 +1100 skrev glen english LIST: >> Hi Tomas >> "codec2 + zip makes for even smaller files" >> >> implying that there is still redundancy to remove in the codec2 >> encoded voice files >> That sounds like low hanging fruit to pick. > That is not surprising. In speech you can lower the signal rate by > talking more slowly. This and repetition happens often over the air. > Silence doesn't require more bits to encode than the length of the > silence. > > I think Dave posted on his blog a while back that the effective data > rate of human speech is on the order of 1-10 bit/s or so > > As for it being low-hanging, variable bitrate may be something to > consider for future modes. If there is silence, perhaps it is enough to > just transmit a carrier? If 14 bits are enough for a frame rather then > 28 because reasons, then maybe Manchester code those bits. Or use a > lower baudrate for those bits, but I'm not sure how that interacts with > modem stuff. Oh and that would require a breaking change to the .c2 > file format, unless you pad with zeroes > > For files this isn't a huge issue I think. Might be fun to cram an > entire 10 hour audio book onto a floppy though > > /Tomas > > > ___ > Freetel-codec2 mailing list > Freetel-codec2@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/freetel-codec2 ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2 ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Updating the libcodec2 binding in FFmpeg, fixed-point decoding
It's an interesting idea. For real time work, taking it to the extreme, putting a bit of latency in there- the encoder and decoder pair, could over time , generate shortcut tokens / lookup for whole words IE encoder continuously stores the encoded words. (it doesnt know the word, only the sequence ) and puts marker tags on them, and transmits the marker tag with the encoded sequence at the other end, the decoder continously stores the sequences and the associated tag. WHen the encoder finds a fitting sequence that's already been encoded and a close fit to work already down, it only sends the token tag to the decoder to look up and reproduce. IE a form of dynamic lookup table. There must be a name for doing this in computer science ??? glen On 29/12/2023 11:54 pm, Tomas Härdin wrote: fre 2023-12-29 klockan 10:44 +1100 skrev glen english LIST: Hi Tomas "codec2 + zip makes for even smaller files" implying that there is still redundancy to remove in the codec2 encoded voice files That sounds like low hanging fruit to pick. That is not surprising. In speech you can lower the signal rate by talking more slowly. This and repetition happens often over the air. Silence doesn't require more bits to encode than the length of the silence. I think Dave posted on his blog a while back that the effective data rate of human speech is on the order of 1-10 bit/s or so As for it being low-hanging, variable bitrate may be something to consider for future modes. If there is silence, perhaps it is enough to just transmit a carrier? If 14 bits are enough for a frame rather then 28 because reasons, then maybe Manchester code those bits. Or use a lower baudrate for those bits, but I'm not sure how that interacts with modem stuff. Oh and that would require a breaking change to the .c2 file format, unless you pad with zeroes For files this isn't a huge issue I think. Might be fun to cram an entire 10 hour audio book onto a floppy though /Tomas ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2 ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Updating the libcodec2 binding in FFmpeg, fixed-point decoding
I personally very much doubt that. You can get a higher symbol rate by tapping a CW key. On 29 December 2023 12:54:02 UTC, "Tomas Härdin" wrote: > >I think Dave posted on his blog a while back that the effective data >rate of human speech is on the order of 1-10 bit/s or so ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Updating the libcodec2 binding in FFmpeg, fixed-point decoding
fre 2023-12-29 klockan 10:44 +1100 skrev glen english LIST: > Hi Tomas > "codec2 + zip makes for even smaller files" > > implying that there is still redundancy to remove in the codec2 > encoded voice files > That sounds like low hanging fruit to pick. That is not surprising. In speech you can lower the signal rate by talking more slowly. This and repetition happens often over the air. Silence doesn't require more bits to encode than the length of the silence. I think Dave posted on his blog a while back that the effective data rate of human speech is on the order of 1-10 bit/s or so As for it being low-hanging, variable bitrate may be something to consider for future modes. If there is silence, perhaps it is enough to just transmit a carrier? If 14 bits are enough for a frame rather then 28 because reasons, then maybe Manchester code those bits. Or use a lower baudrate for those bits, but I'm not sure how that interacts with modem stuff. Oh and that would require a breaking change to the .c2 file format, unless you pad with zeroes For files this isn't a huge issue I think. Might be fun to cram an entire 10 hour audio book onto a floppy though /Tomas ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Updating the libcodec2 binding in FFmpeg, fixed-point decoding
Hi Glen, you're right, there is still some redundant information in time domain between frames. This is on purpose, as Codec2 is meant to be a real-time speech codec for wireless transmissions, which generelly speaking has to be considered to have bad channels... The main idea is that you can lose one frame (e.g. 40 ms) and can still decode the next frame without prior knowledge. That is one major advantage compared to closed speech codecs: with codec2 we have full control over all layers from physical (modulation) to coding (error correction/detection INCLUDING source encoding!). That enables tuning between all layers for best overall performance, which is in the end legibility (not necessarily a minimum BER). I agree that if you want to encode speech on a PC in non-realtime, you may apply an additional compression on top. That is if you want to save some Kbytes unless your speech file is of multi-hour duration... :-D Regards, Stefan Am 29. Dezember 2023 00:44:35 MEZ schrieb glen english LIST : >Hi Tomas >"codec2 + zip makes for even smaller files" > >implying that there is still redundancy to remove in the codec2 encoded voice >files >That sounds like low hanging fruit to pick. > >I guess though that is across a large number of frames (an audio book), where >there may be redundancy / repeats of common codewords. IEnot just a >single frame. speech pauses , and for the same speaker, phonemes and other >speech components. > >Interesting ! > >On 28/12/2023 9:09 pm, Tomas Härdin wrote: >> codec2 + zip makes for even smaller files > > >___ >Freetel-codec2 mailing list >Freetel-codec2@lists.sourceforge.net >https://lists.sourceforge.net/lists/listinfo/freetel-codec2 ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Updating the libcodec2 binding in FFmpeg, fixed-point decoding
Hi Tomas "codec2 + zip makes for even smaller files" implying that there is still redundancy to remove in the codec2 encoded voice files That sounds like low hanging fruit to pick. I guess though that is across a large number of frames (an audio book), where there may be redundancy / repeats of common codewords. IEnot just a single frame. speech pauses , and for the same speaker, phonemes and other speech components. Interesting ! On 28/12/2023 9:09 pm, Tomas Härdin wrote: codec2 + zip makes for even smaller files ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Updating the libcodec2 binding in FFmpeg, fixed-point decoding
tor 2023-12-28 klockan 07:20 +1030 skrev david: > Hi Tomas, > > Thanks for that work. 1.2 was clean up of the Git repo, so nothing > much has changed in terms of functionality. We have no plans for a > fixed point version. The .c2 file format is a corner use-case (we > focus mainly on the HF radio use case), but we have no plans to > change > it. Good. One compatibility thing that springs to mind for the future: if the header needs to be extended, try to extend it in multiples of LCM() of all block sizes. lcm(4,6,7,8) = 168. That way, while there may be garbage for the first couple of frames, the bitstream is properly framed after that. > A road map of what is currently placed for the entire FreeDV project > (which includes Codec 2) is here: > > https://freedv.org/2023/11/30/ardc-grant-project-plan/ > > Do you have any way of measuring how many people are using the Codec > 2 > option in FFmpeg? I do not, but I was contacted by someone who has been encoding audiobooks using it. codec2 + zip makes for even smaller files There may also be interest in the open telephony sphere. > We have been looking into the use of the freedv-gui application > (which > uses libcodec2) in various repos and found no evidence of the > packaged > versions of freedv-gui actually being used. The number of package > maintainers seems to to outnumber the end users. This is surprising > We are interested in auditing the use of freedv-gui and Codec 2 to > best > direct our efforts. For example there is no point supporting package > maintainers if no one is using the packages - best to direct our > limited resources elsewhere. Sounds reasonable > I find it interesting that package maintenance is such a popular > hobby > - we get more correspondence/requests for help on this topic than say > the DSP side of Codec 2 (where I like to play). Probably because DSP is harder :) /Tomas ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Updating the libcodec2 binding in FFmpeg, fixed-point decoding
I would think, that a floating pt implementation on a soft FPGA processor core with floating point would take less area that a purely FPGA fixed point HDL implementation. The reason is that the amount of work required to do (in terms of computations per cycle) does not really take advantage of the parallelism of an FPGA . and a basic soft core with an FPU can easily run fast enough on even the slowest FPGA fabric around However, I suspect that codec2 could easily work within small fractional integer space (32 to 40 bit) without changes, and not have any scaling issues and be completely scaling free in a 64 bit fractional integer space. My usual rule - if it can be run in a high level language in software, then don't do it in fpga fabric as HDL ... probably applies. I'm currently implementing codec2 for RISC-V (F) in floating point. In many RISC-V implementations, there is the facility to generate in the HDL custom instructions that can be inserted into the code which can provide , for example, vector speedups. In my opinion, There's a good reason though not to go too far from 'reference C code' if possible, unless, for example there are struct power consumption limits- as changes to the mainline become slower and more difficult to propagate into different platforms if someone needs to spend the time to (heavily) optimize. If the use of fixed point requires regular scaling, it probably becomes a pain to be linked to the mainline reference code unless the scaling requirements are scripted and automatic. This breathes light for me into thinking about a purely 64bit fractional ALU fractional implementation. In the RISCV -custom instruction support , one could add fractional ALU instructions that MACRO out the std asm to hide the shifts from the programmer and compiler. -glen On 28/12/2023 7:36 am, Mooneer Salem wrote: Hi Tomas, Someone did a FPGA implementation a while back for a university project. That's the closest we've gotten to a fixed-point implementation. Right now, there are no plans to port it officially but this may be revisited later. Thanks, -Mooneer K6AQ ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Updating the libcodec2 binding in FFmpeg, fixed-point decoding
Hi Tomas, Thanks for that work. 1.2 was clean up of the Git repo, so nothing much has changed in terms of functionality. We have no plans for a fixed point version. The .c2 file format is a corner use-case (we focus mainly on the HF radio use case), but we have no plans to change it. A road map of what is currently placed for the entire FreeDV project (which includes Codec 2) is here: https://freedv.org/2023/11/30/ardc-grant-project-plan/ Do you have any way of measuring how many people are using the Codec 2 option in FFmpeg? We have been looking into the use of the freedv-gui application (which uses libcodec2) in various repos and found no evidence of the packaged versions of freedv-gui actually being used. The number of package maintainers seems to to outnumber the end users. We are interested in auditing the use of freedv-gui and Codec 2 to best direct our efforts. For example there is no point supporting package maintainers if no one is using the packages - best to direct our limited resources elsewhere. I find it interesting that package maintenance is such a popular hobby - we get more correspondence/requests for help on this topic than say the DSP side of Codec 2 (where I like to play). Thanks, David On Wed, 2023-12-27 at 18:48 +0100, Tomas Härdin wrote: > Hi > > I see codec2 has now reached version 1.2.0. I'm working on updating > the > libcodec2 binding in libavformat in FFmpeg accordingly. > > So far I've been conservative, only allowing major version 0 and > requiring version to be >= 0.8. So in effect versions 0.8 to 0.255. > Now > that 1.X is out I'm updating libavformat accordingly. Fortunately the > .c2 format hasn't changed, so this is more of a heads up to be > careful > not to make breaking changes to the format :) > > In effect what I'm doing is allowing versions 0.8 through 1.255. > > I'm also adding tests to FFmpeg's testing system FATE. So far these > tests are quite basic, only checking metadata rather than the actual > decoded audio. This because libcodec2 uses floating point arithmetic. > At some time I'd like to improve that. Are there any plans to > implement > a fixed-point decoder? I remember seeing some chatter about that. > > /Tomas > > > ___ > Freetel-codec2 mailing list > Freetel-codec2@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/freetel-codec2 ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Updating the libcodec2 binding in FFmpeg, fixed-point decoding
Hi Tomas, Someone did a FPGA implementation a while back for a university project. That's the closest we've gotten to a fixed-point implementation. Right now, there are no plans to port it officially but this may be revisited later. Thanks, -Mooneer K6AQ On Wed, Dec 27, 2023 at 10:11 AM Tomas Härdin wrote: > Hi > > I see codec2 has now reached version 1.2.0. I'm working on updating the > libcodec2 binding in libavformat in FFmpeg accordingly. > > So far I've been conservative, only allowing major version 0 and > requiring version to be >= 0.8. So in effect versions 0.8 to 0.255. Now > that 1.X is out I'm updating libavformat accordingly. Fortunately the > .c2 format hasn't changed, so this is more of a heads up to be careful > not to make breaking changes to the format :) > > In effect what I'm doing is allowing versions 0.8 through 1.255. > > I'm also adding tests to FFmpeg's testing system FATE. So far these > tests are quite basic, only checking metadata rather than the actual > decoded audio. This because libcodec2 uses floating point arithmetic. > At some time I'd like to improve that. Are there any plans to implement > a fixed-point decoder? I remember seeing some chatter about that. > > /Tomas > > > ___ > Freetel-codec2 mailing list > Freetel-codec2@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/freetel-codec2 > ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
[Freetel-codec2] Updating the libcodec2 binding in FFmpeg, fixed-point decoding
Hi I see codec2 has now reached version 1.2.0. I'm working on updating the libcodec2 binding in libavformat in FFmpeg accordingly. So far I've been conservative, only allowing major version 0 and requiring version to be >= 0.8. So in effect versions 0.8 to 0.255. Now that 1.X is out I'm updating libavformat accordingly. Fortunately the .c2 format hasn't changed, so this is more of a heads up to be careful not to make breaking changes to the format :) In effect what I'm doing is allowing versions 0.8 through 1.255. I'm also adding tests to FFmpeg's testing system FATE. So far these tests are quite basic, only checking metadata rather than the actual decoded audio. This because libcodec2 uses floating point arithmetic. At some time I'd like to improve that. Are there any plans to implement a fixed-point decoder? I remember seeing some chatter about that. /Tomas ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
