Re: [Freetel-codec2] Codec2 and OpenGD77

[Danilo Beuche]

Hi,

I'd like to share my thoughts on this, as I also have some GD77 and love
the OpenGD77 FW (main reason for getting these). I have some experience
with porting FreeDV to the STM32 arm processor as I was one of the
developers who integrated Codec2 into the UHSDR Firmware a couple of
years ago.

The hardware in the OpenGD77 GD77 is surely not powerful enough for
700D, it may be possible to get it to work with 1600D. FreeDV also eats
a big slice of your RAM (~64k at least) of which the GD77's
MK22FN512VLL12R has only 128k. 120 Mhz is also less than the 168Mhz of
the SM1000 or the MCHF with an STM32F4xx processor. As both processor
belong to the CORTEX-M4 family, they should have similar performance per
MHZ.

In measurements for 1600D on the MCHF with the UHSDR software, this
takes roughly 70% load in the signal processing path from 48khz IQ to
audio for 1600D. Doing the math 168/120 * 70% load = 98% load. 2% cycles
left. SSB decoding takes about 10%, the remaining 60% went to FreeDV.
Not a problem in SM1600 as it does less signal processing and doesn't
have to run a graphical UI in addition (which the MCHF does).

700D runs ok on the STM32F7 processor with 216 Mhz and processor caches
(CORTEX-M7 design), speeding up code execution quite significantly. The
STM32H7 with 480 Mhz is not concerned about decoding 700D at all, there
is plenty of cpu power left. But those are unfortunately not in those
handhelds.

But for a different FreeDV mode better suited for VHF/UHF usage or a
different signal processing chain it might be an entirely different
story. Thats is where I have to hand over to David and the other experts.
And just for the record: I would love to have a "completely" open source
GD77 with digital voice.

Danilo



Can't say anything about the DM1801

On 02/02/2021 18:21, Daniel Mundall wrote:

Hi Eric,
Funny that you mention OpenGD77 as last month I discovered these
radios (Baofeng DM-1801 in my case) and picked up two of them off of
aliexpress with codec2 being the primary reason for my purchase.
There are 4 reasons why I think they would be great fit for codec2:

  * They're cheap ($65-75) & available which means you could get a
larger adoption.
  * They run on an ARM platform either STM32 or similar processor.
  * And there's already an open source firmware without inventing that.
  * The audio encoding isn't a separate chip (at least on the DM1801)
it's software encoding.

Interestingly enough original portion of the memory containing the
software audio codec is retained in OpenGD77 to avoid patent issues.

Another cool possibility that I haven't felt out yet but in theory
because of the two slot TDMA setup I think either a repeater mode or
full duplex audio should be possible.

Anyway, I just got the radios so soon as I have a chance I'm going to
get OpenGD77 loaded up and play around with it.

One thing that David or the others might be better suited to answer is
how much hardware functions of the STM32 have been used in the codec2
port. Because there could be hardware limitations with what is even
available as far a hardware functions on the MCU.

Regards,

Daniel Mundall VA7DRM


On Tue, Feb 2, 2021 at 5:59 AM Eric Jacksch mailto:e...@jacksch.com>> wrote:

I don't know enough about CODECS to know the right answer (thus must
post here), but right now the OpenGD77 source basically includes an
bit of assembly language to call some binary code lifted out of the
vendor's original firmware. My interest is to drop in a different
CODEC, but I don't know enough about them to understand exactly what
needs to be done.

On Tue, 2 Feb 2021 at 02:48, Mooneer Salem mailto:moon...@gmail.com>> wrote:
>
> I'm thinking M17 (based on Codec2) might be better to
incorporate since it's already building in routing type features
that DMR, D-STAR, etc. already have. I'm not really familiar with
OpenGD77 but it seems like a neat project. Unfortunately I don't
have a compatible HT (I use a MD-380 for DMR) but perhaps it'll
gain more support in the future.
>
> -Mooneer K6AQ
>
> On Mon, Feb 1, 2021 at 3:21 PM Eric Jacksch mailto:e...@jacksch.com>> wrote:
>>
>> Greetings,
>>
>> Has anyone looked at dropping Codec2 into OpenGD77?
>>
>> In case anyone is not aware, OpenGD77 is a (mostly) open source
>> firmware replacement for several low cost handheld two-way radios,
>> including the GD-77 (aka TYT MD-760).
>>
>> To work around intellectual property issues, the firmware currently
>> links in the DMR CODEC from the original vendor's binary.
>>
>> While it would break compatibility with other DMR radios, I'm
>> interested in seeing a fork of OpenGD77 that is fully open
source and
>> Codec 2 seems like an obvious candidate. A truly open-source
amateur
>> VHF/UHF digital mode.
>>
>> The IDE (MCUExpresso) and complete toolchain for OpenGD77 is
free and
>> relativ

Re: [Freetel-codec2] The LPCNet packaging problem...

[Danilo Beuche]

Hi Richard,

maybe two builds (one with, one without AVX2 etc.) and a wrapper program
do the job. The wrapper does nothing more than detecting what is
available and starts one or the other binary.
Similarly we could put all the application code in two differently
builds of a dynamic library and a customer wrapper just loads and pass
control to the right one.

First solution is easier but less elegant. Don't ask me how to implement
the second one, but 20 years ago I did something like this, guess it is
still doable on modern Linux&Windows.

Regards,
Danilo

On 22 Apr 2020 17:27, Richard Shaw wrote:
> So LPCNet requires AVX, AVX2, or NEON, but in most Distro's those
> instruction sets can not be assumed. There's still hardware being
> produced that don't have any of those.
>
> For instance, in Fedora x86_64, we only assume up to SSE2 because
> assuming anything higher would prevent install on some processors
> still in production (and may more still in use).
>
> Between all the architectures on Fedora (x86_64, i686, armv7hl,
> aarch64, ppc64le, and s390x) there is only one baseline that meets the
> requirements of LCPNet:
>
> aarch64 because NEON is "baked in". 
>
> There are ways to detect the available instruction sets during runtime:
>
> https://github.com/google/cpu_features
>
> But I know I don't have the knowledge how to implement this
> dynamically (at run time instead of build time).
>
> There's also a problem we've briefly discussed before, should we keep
> the name LPCNet for our library? From a packaging perspective it's an
> issue because someone may want to package the canonical LPCNet:
>
> https://github.com/mozilla/LPCNet
>
> And two packages can't have the same name for obvious reasons...
>
> Thoughts?
>
> Thanks,
> Richard
>
>
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] Problems with freedv_tx in 700D mode, my parrot rpt

[Danilo Beuche]

Hi Al,

On 19 Aug 2019 01:38, Al Beard wrote:
> Hi all,
>
> I need to bet back to basics:
>
> At the command line, do I have the tools for a "parrot" repeater?
>
> Let's start:
>
> 1) a silence "raw" file
>dd if=/dev/zero of=blank count=1
> 2) generate a Codec2 mode 700D file
>c2enc 700D blank blank.c2
> 3) generate a RAW audio file
>freedv_tx 700D blank.c2 blank.raw --codectx

3) is not working at the moment. It calls freedv_tx which does not
handle the (interleave_frames == 1) case for 700D properly as far as I
can see. The function freedv_comptx handles it correctly though.

Can't say anything for 2) but 1) is working, but creates only 512 bytes.
Which is a too short sample I think. 700D wants at least 160ms of input,
at 8 khz/16bit mono  512 bytes translate to  32 ms.

Regards,
Danilo

___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] OSX Build Help

[Danilo Beuche]

Hi David,

sure. I now have travis compiling the whole thing on MacO X using
build_linux.sh (10 min) using some precompile brew packages OR using the
build_osx.sh from Moneer which takes very long to compile (>35 min, and
it finally failed but this can be easily fixed).

It would be interesting to find out whether the build made using
precompiled brew packages actually works or if the full build from
source is required.

To have this working on the official freedv-gui repository, we have to
merge the changes in the codec2 PR   #66   to codec2 master branch
first. Then freedv-gui PR #26 can be merged and based on this I could
add the travis stuff I already have working on my repo.

Regards,
Danilo



On 11 Aug 2019 22:02, David Rowe wrote:
> Nice work guys.
>
> Mooneer and Danilo - would it be possible to set up Travis so it tests
> the OSX build when we push any freedv-gui changes?  I'd like to make
> sure we keep the OSX build working as freedv-gui develops.
>
> I'm not sure if this is practical - Danilo you mentioned the build
> process on Travis is quite lengthy. Another option would be kicking off
> the process manually, as we do for the stm32.
>
> Thanks,
> David
>
> On 12/08/19 01:03, Mooneer Salem wrote:
>> Yep, the pull request also has the changes needed for dylibbundler to
>> find Codec2 and LPCNet (thus no prompts during the build process).
>>
>> -Mooneer
>>
>> On Sun, Aug 11, 2019 at 5:40 AM Mark Jessop > <mailto:lenniethelemm...@gmail.com>> wrote:
>>
>> Nice job Mooneer!
>>
>> Were you able to produce an appbundle from this?
>>
>> Cheers,
>> Mark
>>
>> On Sun, Aug 11, 2019 at 8:23 PM Mooneer Salem > <mailto:moon...@gmail.com>> wrote:
>>
>> I was able to figure out why the build was crashing. It looks
>> like wxWidgets needs to be built with the same C++ standard
>> library as FreeDV itself (which is libc++ on OSX); once that was
>> done (by tweaking various CMake files to add compiler, etc.
>> flags), the application appeared to load fine.
>>
>> Anyway, I went ahead and created a pull request for this portion
>> of the fix: https://github.com/drowe67/freedv-gui/pull/26. In
>> effect, build_osx.sh will have everything except for Codec2 and
>> LPCNet built and linked statically.
>>
>> -Mooneer K6AQ
>>
>> On Sun, Aug 11, 2019 at 3:36 AM Danilo Beuche
>> mailto:danilo.beu...@gmx.net>> wrote:
>>
>> Hi,
>>
>> I can confirm that the fixes  by Mooneer resolve the compile
>> errors. I was able to get travis to almost finish the build:
>>
>> 
>> https://travis-ci.org/db4ple/freedv-gui/builds/570441603?utm_source=github_status&utm_medium=notification
>>
>> Compile is okay, just the program which then builds the
>> MacOS X app package fails, but this is difficult to fix for
>> me with each Travis build taking more than 10 mins.
>> But for those with a Mac, the .travis.yml found here:
>> https://github.com/db4ple/freedv-gui/blob/macosx/.travis.yml
>> can serve as a blueprint. If you find a solution for the
>> final problem, let me know and I integrate this in .travis.yml
>>
>> FYI: Travis uses homebrew instead of Mac Ports.
>> I have no idea (and no way to test) if the resulting program
>> made by Travis does work at all or if it crashes like the
>> build made by Mooneer.
>>
>> Regards,
>> Danilo
>>
>>
>> On 11 Aug 2019 09:02, Mooneer Salem wrote:
>>> Hi all,
>>>
>>> It turns out that the placement of extern "C" in some of
>>> the Codec2 include files was causing FreeDV build issues
>>> for me on 10.14, in addition to the previously mentioned
>>> ofdm_stack issue. I created a pull request for those
>>> changes: https://github.com/drowe67/codec2/pull/66
>>>
>>> Also, I have the freedv binary itself building in my
>>> forked version (https://github.com/tmiw/freedv-gui) but
>>> the app bundle task for some reason is needing me to
>>> manually enter library names. The binary is additionally
>>> segfaulting on startup with the following backtrace:
>>>
>>> 

Re: [Freetel-codec2] OSX Build Help

[Danilo Beuche]

Hi,

I can confirm that the fixes  by Mooneer resolve the compile errors. I
was able to get travis to almost finish the build:

https://travis-ci.org/db4ple/freedv-gui/builds/570441603?utm_source=github_status&utm_medium=notification

Compile is okay, just the program which then builds the MacOS X app
package fails, but this is difficult to fix for me with each Travis
build taking more than 10 mins.
But for those with a Mac, the .travis.yml found here:
https://github.com/db4ple/freedv-gui/blob/macosx/.travis.yml can serve
as a blueprint. If you find a solution for the final problem, let me
know and I integrate this in .travis.yml

FYI: Travis uses homebrew instead of Mac Ports.
I have no idea (and no way to test) if the resulting program made by
Travis does work at all or if it crashes like the build made by Mooneer.

Regards,
Danilo


On 11 Aug 2019 09:02, Mooneer Salem wrote:
> Hi all,
>
> It turns out that the placement of extern "C" in some of the Codec2
> include files was causing FreeDV build issues for me on 10.14, in
> addition to the previously mentioned ofdm_stack issue. I created a
> pull request for those changes: https://github.com/drowe67/codec2/pull/66
>
> Also, I have the freedv binary itself building in my forked version
> (https://github.com/tmiw/freedv-gui) but the app bundle task for some
> reason is needing me to manually enter library names. The binary is
> additionally segfaulting on startup with the following backtrace:
>
> * thread #1, queue = 'com.apple.main-thread', stop reason =
> EXC_BAD_ACCESS (code=1, address=0x2816820)
>
>   * frame #0: 0x000100680d21
> libwx_osx_cocoau_core-3.0.dylib`wxApp::CallOnInit() + 149
>
>     frame #1: 0x000100b80068 libwx_baseu-3.0.dylib`wxEntry(int&,
> wchar_t**) + 37
>
>     frame #2: 0x000100039593 freedv`main(argc=1,
> argv=0x7ffeefbff938) at fdmdv2_main.cpp:171:1
>
>     frame #3: 0x7fff68b5c3d5 libdyld.dylib`start + 1
>
>     frame #4: 0x7fff68b5c3d5 libdyld.dylib`start + 1
>
>
>  I'll have to do additional investigation for those issues but if
> anyone wants to try it, I adapted build_linux.sh for OSX (and named
> build_osx_macports.sh in my repo.
>
> Thanks,
>
> -Mooneer K6AQ
>
> On Sat, Aug 10, 2019 at 10:35 PM Danilo Beuche  <mailto:danilo.beu...@gmx.net>> wrote:
>
> Hi Mark,
>
> I realized that now, my „mistake“. Nevertheless,   I learned
> through that exercise how to use Travis with MacOS. Will try (!)
> to get FreeDV-GUI going as well through Travis but this is a
> fairly tedious process without access to a Mac so I won’t promise
> that I‘ll achieve anything.
>
> Regards
> Danilo
>
> Am 11.08.2019 um 01:14 schrieb Mark Jessop
> mailto:lenniethelemm...@gmail.com>>:
>
>> Hi Danilo, 
>>
>> I can build codec2 just fine (with the ofdm_stack changes) - It's
>> really freedv-gui where the build issues raise their head. 
>>
>> Cheers,
>> Mark
>>
>> On Sun, Aug 11, 2019 at 4:15 AM Danilo Beuche
>> mailto:danilo.beu...@gmx.net>> wrote:
>>
>> Hi Mark,
>>
>> I am experimenting right now with a MacOS X build via travis
>> for codec2 in my GitHub clone of codec2. So far it looks
>> mostly good. The build works, all tests are built and being
>> run. 5 tests fail. Out of these 5 failing tests are 4
>> valgrind tests for memory leaks, these need to be looked at
>> but for this one needs are real MacOS machine (which I don't
>> have, I just let Travis run one machine for me.). The last
>> tests probably needs also only a minor tweak in the build
>> configuration. So generally the build works if minor
>> modifications are made and the environment is similar to the
>> one created for the Travis build:
>>
>> https://github.com/db4ple/codec2/pull/2/files
>>
>> You can also checkout the last build log from Travis:
>>
>> 
>> https://travis-ci.org/db4ple/codec2/builds/570275451?utm_source=github_status&utm_medium=notification
>>
>> The STM32 fails on Travis as well, but this is not a concern
>> at the moment.
>>
>> Regards,
>> Danilo
>>
>>
>> On 10 Aug 2019 05:37, Mark Jessop wrote:
>>> Is there anyone out there who has been able to successfully
>>> build the latest git freedv / codec2 master branches under OSX?
>>>
>>> The instructions
>>> here: https://github.com/drowe67/fr

Re: [Freetel-codec2] OSX Build Help

[Danilo Beuche]

Hi Mark,

I realized that now, my „mistake“. Nevertheless,   I learned through that 
exercise how to use Travis with MacOS. Will try (!) to get FreeDV-GUI going as 
well through Travis but this is a fairly tedious process without access to a 
Mac so I won’t promise that I‘ll achieve anything.

Regards
Danilo

> Am 11.08.2019 um 01:14 schrieb Mark Jessop :
> 
> Hi Danilo, 
> 
> I can build codec2 just fine (with the ofdm_stack changes) - It's really 
> freedv-gui where the build issues raise their head. 
> 
> Cheers,
> Mark
> 
>> On Sun, Aug 11, 2019 at 4:15 AM Danilo Beuche  wrote:
>> Hi Mark,
>> 
>> I am experimenting right now with a MacOS X build via travis for codec2 in 
>> my GitHub clone of codec2. So far it looks mostly good. The build works, all 
>> tests are built and being run. 5 tests fail. Out of these 5 failing tests 
>> are 4 valgrind tests for memory leaks, these need to be looked at but for 
>> this one needs are real   MacOS machine (which I don't have, I just let 
>> Travis run one machine for me.). The last tests probably needs also only a 
>> minor tweak in the build configuration. So generally the build works if 
>> minor modifications are made and the environment is similar to the one 
>> created for the Travis build:
>> 
>> https://github.com/db4ple/codec2/pull/2/files
>> 
>> You can also checkout the last build log from Travis:
>> 
>> https://travis-ci.org/db4ple/codec2/builds/570275451?utm_source=github_status&utm_medium=notification
>> 
>> The STM32 fails on Travis as well, but this is not a concern at the moment. 
>> 
>> Regards,
>> Danilo
>> 
>> 
>> 
>>> On 10 Aug 2019 05:37, Mark Jessop wrote:
>>> Is there anyone out there who has been able to successfully build the 
>>> latest git freedv / codec2 master branches under OSX?
>>> 
>>> The instructions here: 
>>> https://github.com/drowe67/freedv-gui/blob/master/README.osx
>>> are now of course 100% out of date with the recent updates, and probably 
>>> need to be marked as such so others don't try and follow them and fall into 
>>> a trap.
>>> 
>>> A few notes so far:
>>> 
>>> - LPCNet builds fine with the recent-ish changes to look for AVX extensions 
>>> under OSX.
>>> 
>>> - The ofdm_stack target in codec2 won't compile using clang, as it doesn't 
>>> have the -no-pie and -Map options. I had to wrap the ofdm_stack target in 
>>> unittest/CMakeLists.txt in a if (NOT APPLE) directive so it wouldn't be 
>>> built. Once this is done codec2 builds fine. 
>>> 
>>> The cmake command for freedv ends up being: cmake -DCMAKE_BUILD_TYPE=Debug 
>>> -DCODEC2_BUILD_DIR=/Users/darkside/Dev/codec2-git/build 
>>> -DLPCNET_BUILD_DIR=/Users/darkside/Dev/LPCNet_test/build 
>>> -DWXCONFIG=/opt/local/Library/Frameworks/wxWidgets.framework/Versions/wxWidgets/3.0/lib/wx/config/osx_cocoa-unicode-3.0
>>>  ..
>>> 
>>> Compiling freedv itself is where I run into all sorts of fun errors, which 
>>> i've pasted here:
>>> https://slexy.org/view/s2r4g9MkjG
>>> 
>>> I suspect these are related to clang's supplied libc++. Looking at the 
>>> CMakeLists.txt file for freedv, it looks like someone has encountered 
>>> something similar in the past: 
>>> https://github.com/drowe67/freedv-gui/blob/master/CMakeLists.txt#L98
>>> 
>>> Not really sure where to go from here, short of attempting to completely 
>>> recompile macports using gcc instead of clang, which is something I suspect 
>>> will result in my sanity   degrading further than it already has 
>>> this morning.
>>> 
>>> Hopefully there's someone else out there that has looked into this, or has 
>>> actually been successful in getting this to work!
>>> 
>>> 73
>>> Mark VK5QI
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> ___
>>> Freetel-codec2 mailing list
>>> Freetel-codec2@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
>> ___
>> Freetel-codec2 mailing list
>> Freetel-codec2@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] OSX Build Help

[Danilo Beuche]

Hi Mark,

I am experimenting right now with a MacOS X build via travis for codec2
in my GitHub clone of codec2. So far it looks mostly good. The build
works, all tests are built and being run. 5 tests fail. Out of these 5
failing tests are 4 valgrind tests for memory leaks, these need to be
looked at but for this one needs are real MacOS machine (which I don't
have, I just let Travis run one machine for me.). The last tests
probably needs also only a minor tweak in the build configuration. So
generally the build works if minor modifications are made and the
environment is similar to the one created for the Travis build:

https://github.com/db4ple/codec2/pull/2/files

You can also checkout the last build log from Travis:

https://travis-ci.org/db4ple/codec2/builds/570275451?utm_source=github_status&utm_medium=notification

The STM32 fails on Travis as well, but this is not a concern at the moment.

Regards,
Danilo


On 10 Aug 2019 05:37, Mark Jessop wrote:
> Is there anyone out there who has been able to successfully build the
> latest git freedv / codec2 master branches under OSX?
>
> The instructions
> here: https://github.com/drowe67/freedv-gui/blob/master/README.osx
> are now of course 100% out of date with the recent updates, and
> probably need to be marked as such so others don't try and follow them
> and fall into a trap.
>
> A few notes so far:
>
> - LPCNet builds fine with the recent-ish changes to look for AVX
> extensions under OSX.
>
> - The ofdm_stack target in codec2 won't compile using clang, as it
> doesn't have the -no-pie and -Map options. I had to wrap the
> ofdm_stack target in unittest/CMakeLists.txt in a if (NOT APPLE)
> directive so it wouldn't be built. Once this is done codec2 builds fine. 
>
> The cmake command for freedv ends up being: cmake
> -DCMAKE_BUILD_TYPE=Debug
> -DCODEC2_BUILD_DIR=/Users/darkside/Dev/codec2-git/build
> -DLPCNET_BUILD_DIR=/Users/darkside/Dev/LPCNet_test/build
> -DWXCONFIG=/opt/local/Library/Frameworks/wxWidgets.framework/Versions/wxWidgets/3.0/lib/wx/config/osx_cocoa-unicode-3.0
> ..
>
> Compiling freedv itself is where I run into all sorts of fun errors,
> which i've pasted here:
> https://slexy.org/view/s2r4g9MkjG
>
> I suspect these are related to clang's supplied libc++. Looking at the
> CMakeLists.txt file for freedv, it looks like someone has encountered
> something similar in the
> past: https://github.com/drowe67/freedv-gui/blob/master/CMakeLists.txt#L98
>
> Not really sure where to go from here, short of attempting to
> completely recompile macports using gcc instead of clang, which is
> something I suspect will result in my sanity degrading further than it
> already has this morning.
>
> Hopefully there's someone else out there that has looked into this, or
> has actually been successful in getting this to work!
>
> 73
> Mark VK5QI
>
>
>
>
>
>
>
>
>
>
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] sm1000 and FreeDV 700D

[Danilo Beuche]

Hi,

I am not sure that I understand Walter's procedure as it is documented
in the sm1000_manual.md

This procedure will first dump the existing flash of the sm1000 to a
.dfu file. But it will not produce a new dfu file from the sm1000v2.bin
file.
If one flashes the dfu file created in the first step as described with
the "Upgrade" procedure, you have exactly what you had before on the device.

To create a sm1000v2.dfu file on Windows you may use the "DFU File
Manager" which is installed in addition to the DFUSeDemo tool from the
ST Package linked. However, it is not trivial to operate without
instructions.

But I think, the easiest way is if David provides us with a dfu file in
addition to the bin file. There is a small python script in the UHSDR
project which does just that.

https://github.com/df8oe/UHSDR/tree/active-devel/mchf-eclipse/support/hex2dfu

and the respective call  is

hex2dfu.py sm1000v2.hex sm1000v2.dfu 

There are probably more ways to create a dfu file on Linux but this one
works.

Regards,
Danilo



But it would be

On 16 Jul 2019 22:16, David Rowe wrote:
> Thank Steve,
>
> Could you please add that as a Pull Request for the sm1000 manual?
>
> https://github.com/drowe67/codec2/blob/master/stm32/doc/sm1000_manual.md
>
> Walter's method is already in there, but it would be useful to have
> your's as well.
>
> I had no idea the Windows world was so confusing!
>
> Cheers,
> David
>
> On 17/07/19 02:39, Steve wrote:
>> Here's how I did it on my Windows 8.1 Pro laptop machine for a friend.
>> Seemed to work, haven't tested sm1000 yet.
>>
>> Your Experience May Vary
>>
>> 73, Steve/k5okc
>>
>>
>>
>>
>>
>> ___
>> Freetel-codec2 mailing list
>> Freetel-codec2@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
>>
>
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2


___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] Building smallDV with Codec2-dev, latest SVN

[Danilo Beuche]

Hi,

On 20.02.2019 06:17, Alan Beard wrote:
>
> *"SmallDV" uses only one audio device so, cannot get into buffer
> problems *
> *because of different sample rates, in and out.*
> *
> *

this is unfortunately not exactly true. SmallDV reads audio data from
the USB device into the computer, and both have different clock domains,
i.e. there is already a (small) mismatch. And the same applies in the
opposite direction (computer to audio). USB audio has different modes
for handling this issue but all of them require both sides (USB device
and USB host) to do it properly together, see for instance
https://www.electronicdesign.com/embedded/achieving-bit-perfect-usb-audio .

The UHSDR firmware suffers from this problem, since we have not
implemented the clock synchronization (properly) and depending on the
clock differences this creates audible issues in the audio streams in
regular intervals. However, as far as  I have seen it, it does not cause
real problems, for instance if used with FT8 or FreeDV.

Regards,
Danilo

___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] Adding FreeDV700D to UHSDR firmware

[Danilo Beuche]

Hi David,

thanks for the response. I already got in touch with Don. Looking at the
much longer frames for 700D (160ms vs 40ms) I am considering to go for
the short interface in order to save some memory so that we may benefit
from Dons work on getting 700D to work on STM32F4xx. While some users
have STM32F7 or STM32H7 processors which have 512k of RAM at least, the
majority of the UHSDR users is still using the STM32F407 with 192k RAM.
To have any chance of 700D on these machines, we have to keep the memory
usage as low as possible. For the moment I decided to go with the
existing interfaces and see what we can get out of it.

The question remaining for me: Is there an technical advantage in using
IQ directly (better reception)?

Regards,
Danilo


On 10.02.2019 01:41, David Rowe wrote:
> Hi Danilo,
>
> Don has been doing some fine work on porting 700D to the stm32, and is
> nearly finished.  This has involved optimisation to reduce run-time
> memory usage. One of the changes was to use shorts rather than floats to
> save memory, and the current version just supports real valued signals
> for 700D.
>
> Danilo - Don has also developed an extensive, automated unit test system
> for the work he has done, under stm32/unittest.  The tests run on a
> Discovery board.
>
> Don - Danilo's SDR runs entirely on a stm32 (no external SSB radio).
> Previously, when using FreeDV 1600, his team passed complex valued
> samples to the FreeDV API.  IIRC Danilo is using a more modern stm32
> variant with more memory.
>
> So - we need complex valued support for 700D.  These could be complex
> shorts or floats.  One possibility:
>
> (i) modify freedv_shortrx(), freedv_comprx_700d() and the ofdm.c
> functions called to work with complex short samples (hopefully without a
> big impact on memory).
> (ii) modify freedv_comprx() to do a float to short conversion and call
> freedv_shortrx()
>
> However I'm open to suggestions (and patches) :-)
>
> Cheers,
> David
>
> On 09/02/19 19:43, Danilo Beuche wrote:
>> Hi,
>>
>> as you may know we have FreeDV 1600 support in the STM32 based TRX since
>> a while. Now I am finally aiming at integrating the 700D mode.
>>
>> For FreeDV 1600 we did not use the freedv_rx / freedv_tx calls as we
>> have float i/q data at hand due to the internal signal processing in the
>> rest of the SDR. So the freedv_comprx/freedv_comptx are used.
>>
>> I noticed that the 700D is not integrated in the
>> freedv_comprx/freedv_comptx functions. Now there are a number of questions:
>>
>> 1.I think instead we have to call freedv_shortrx which expects audio
>> samples instead of IQ. Is that correct?
>> 2. Should we go for all modes to do a demodulation first and then pass
>> audio data to the freedv_rx/freedv_tx which involves back converting the
>> data to float data?
>> 3. Or is there a benefit to pass IQ data directly to those modes which
>> support it?
>>
>>
>> In addition, are the improvements 700D for low performance processors
>> mentioned in http://www.rowetel.com/?p=6413 already integrated in the
>> source code? I guess so, but just want to be sure, since I could not
>> find support for the 700D mode directly in the SM1000 source.
>>
>>
>> Regards,
>> Danilo
>>
>>
>>
>>
>>
>>  
>>
>>
>>
>>
>> ___
>> Freetel-codec2 mailing list
>> Freetel-codec2@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
>>
>
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2


___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

[Freetel-codec2] Adding FreeDV700D to UHSDR firmware

[Danilo Beuche]

Hi,

as you may know we have FreeDV 1600 support in the STM32 based TRX since
a while. Now I am finally aiming at integrating the 700D mode.

For FreeDV 1600 we did not use the freedv_rx / freedv_tx calls as we
have float i/q data at hand due to the internal signal processing in the
rest of the SDR. So the freedv_comprx/freedv_comptx are used.

I noticed that the 700D is not integrated in the
freedv_comprx/freedv_comptx functions. Now there are a number of questions:

1.I think instead we have to call freedv_shortrx which expects audio
samples instead of IQ. Is that correct?
2. Should we go for all modes to do a demodulation first and then pass
audio data to the freedv_rx/freedv_tx which involves back converting the
data to float data?
3. Or is there a benefit to pass IQ data directly to those modes which
support it?


In addition, are the improvements 700D for low performance processors
mentioned in http://www.rowetel.com/?p=6413 already integrated in the
source code? I guess so, but just want to be sure, since I could not
find support for the 700D mode directly in the SM1000 source.


Regards,
Danilo





 




___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] Designing an SM1000 "module" for use in equipment

[Danilo Beuche]

Hi,

@Glen: why is 216 Mhz BS? The UHSDR UI runs happily with 216 Mhz.  Would
be interested to learn about your experiences.


The F7 indeed is roughly twice as fast due to extensive caches (which
also requires more careful handling of certain operations) and a much
better FPU.

The 100LQFP case pins between STM32F405 and STM32F765  are bascially on
one side shifted by one pin but this not a big deal since you have to
design a new board anyways.

But if you want fast, really fast, go with the STM32H7: basically same
porting effort and either uses much less power (at same frequency) or
runs much faster (at 400 Mhz).

H7 and F7 have roughly the same pinout.

Danilo


On 04.02.2019 22:37, glen english wrote:
> Hi Richard
>
> if you dont think you can stomach the changes of the RT1020 , suggest
> an F7. Choose the one with lots of cache.
>
> there are 4k, 8k and 16k cache parts. Use the 16k cache parts.
>
> unless necessary, be sure that your compiler treats doubles as floats.
>
> they are mostly pin compatible.  the FPU is abotu 2x the speed of the F4.
>
> My advice is dont use an F4, unless you are building a consumer
> product that is down to the dollar...
>
> F7s really only run at the same speed as F4s.  (168 MHz) .The 216 MHz
> is BS really.
>
> -glen
>
>
>
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2


___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] Designing an SM1000 "module" for use in equipment

[Danilo Beuche]

Hi,

the STM32F765 is depending how you see it relatively close to the
STM32F4 (that is mainly the viewpoint of some marketing material from
STM) but in reality the processors are fairly different to setup. First
of all the supporting libraries are completely different from the ones
used in the SM1000 build (StdPeriph vs. HAL) so this part needs to be
rewritten/exchanged. The Codec2 code is not a problem, that will work
without problems.  Even a HAL based STM32F4 program has in some areas
some subtle differences to virtual the  same code supposed to run on a
STM32F7. 

The SM1000 on the other hand is not too complex.

In any case, it is not just a simple recompile.

Danilo

On 04.02.2019 22:13, Richard wrote:
> Hi All,
>
> As I said I don't have the software skills to jump to a significantly
> different device. 
>
> Having had a better look at the F7 range it seems that to get more
> FLASH and RAM than the STM32F405VGT6 whilst keeping to the "small" 100
> pin package only actually leaves three options, differing only by
> irrelevant peripherals. That makes the STM32F765VIT6 seem good, and
> it's about £11 vs £9 for the STM32F405VGT6. So far so good.
>
> Physical pin mapping is subtly incompatible, but that can be handled
> as I haven't built any hardware yet. How much difficulty should I
> expect getting the existing code to build for it?
>
> Of course I could just proceed with the F4 like the SM1000, and just
> accept that 700D may not end up being possible. Decisions. :-)
>
> Thanks
> Richard
>
> 
>   Virus-free. www.avg.com
> 
>
>
>
> On Mon, 4 Feb 2019 at 20:25, Richard  > wrote:
>
> Hi All,
>
> Understood about the future modes, I was hoping to support 700D
> (when released) so perhaps switching processor is desirable. My
> skillset is 90%/10% hardware/software so both a complete rewrite
> and porting to a totally different platform are out for me. I
> might take a look at other ST options, several of you have
> mentioned the F7 and a quick look seems promising. Does anyone
> with more experience have a specific part number to recommend?
>
> Low cost, whilst nice, isn't really a goal for me, but physical
> size and power consumption are.
>
> Thanks
> Richard
>
> 
> 
>   Virus-free. www.avg.com
> 
> 
>
>
>
> On Mon, 4 Feb 2019 at 20:02, glen english  > wrote:
>
> Hi Don
>
> I got codec2 working on F7 a few years ago, and I was getting
> 2x the
> performance than the guys were getting on the F4. for the same
> clock
> speed...- 2 reasons- 1) the F4 code is completely unoptimized
> and really
> hurts the F4, and 2) the F7 is quite a bit faster if the cache
> access is
> optimal..
>
> IMO the attempt to keep the code base portable between PC and
> microcontroller is problematic and a limit.
>
> suggest The ORANGE PI zero is a tiny inch square PCB and has
> onboard
> audio. It's what I use quad core A7 etc and NEON.
>
> H7 is th new kid ont he block , but STM32 is expensive !!!
>
> Suggest Rt1050 series from NXP for low cost (couple of bucks) 
> and 400
> meg F7 perfromance
>
> -glen
>
> On 5/02/2019 5:24 AM, Don wrote:
> > I am currently working on getting the 700D mode to work with
> the SM1000.
> > It is not certain that this will be possible because of the
> memory
> > needed.
> >
> > Also Dave is looking at new compression modes with different
> performance
> > and bandwidths.
> >
> > I see the current firmware as best suited for experimenting
> and developing
> > on a desktop PC.  To make a small, low cost embedded version
> one would
> > likely want to do a major re-write of the code.  Lack
>
>
>
>
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> 
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
>
>
>
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
___
Freetel-codec2 mailing list
Freetel-codec2@lists.s

[Freetel-codec2] FreeDV @ UHSDR: Technical findings

[Danilo Beuche]

Hello codec2 developers,

I wonder if we  could/should move codec2_fifo.h and fifo.c to the stm32
directory, as it is used exclusively there and clashes with some library
functions called fifo_read and fifo_write in the gcc library we are
using in UHSDR. Since the fifo functions are not used right in the
normal code, we simply don't include them in the builds. So not a direct
issue.

But to be able to use the fifo function in the UHSDR firmware we would
alternatively have to rename the fifo functions in order to avoid the
name clash. I did that, but unfortunately I am right now not able to
build the stm32 code since cmake is not happy, (and I am not an cmake
expert). So I decided this as to risky since I have no way to test my
changes. I dropped that idea for now but in the long run this might be
the best idea.

Any thoughts on that? Keep as is, Move to stm32/src or change function
names to be a little more specific?

Anyway, not a direct problem for us in UHSDR.


But what I get might when compiling for the UHSDR firmware as warning
while linking could be a real problem:

In function 'ofdm_demod',
    inlined from 'freedv_comprx_700d.constprop' at
../drivers/freedv/freedv_api.c:1808:9,
    inlined from 'freedv_comprx.constprop' at
../drivers/freedv/freedv_api.c:1983:14:
../drivers/freedv/ofdm.c:837:23: warning: argument 1 value '4294967288'
exceeds maximum object size 2147483647 [-Walloc-size-larger-than=]
 complex float work[(en - st)];
   ^
../drivers/freedv/ofdm.c: In function 'freedv_comprx.constprop':
../drivers/freedv/ofdm.c:837:23: note: in a call to built-in allocation
function '__builtin_alloca_with_align'

This error may only be visible when compiling with higher optimization
levels and a fixed mode (which we do, I compiled for mode 700D only), so
that the compiler can do all the math when compiling, and it gets 
4294967288 which translates in two-complement to -8 btw.
I don't get this, when I compile the codec2-dev source code on Linux for
the x86 architecture.

Regards,
Danilo




___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] blog post on 700D port to SM1000

[Danilo Beuche]

Hi David, Don,

awesome. It now seems to be possible to get FreeDV 700D in the UHSDR
firmware to run on the mcHF as well. The OVI40 hardware (STM32F7 and
STM32H7 processors) was able to run the "original" code since these have
double precision floating point units.
Thanks Don, for putting so much and so good work into this!   

Regards,

Danilo


On 14.09.2018 23:26, David Rowe wrote:
> Don Reid, W7DMR, is doing some fine work on porting 700D to the SM1000.
>
>   http://www.rowetel.com/?p=6413
>
> - David
>
> On 14/09/18 19:02, Tomas Härdin wrote:
>> https://en.wikipedia.org/wiki/Echo_suppression_and_cancellation
>>
>> "Echo control on voice-frequency data calls that use dial-up modems may
>> cause data corruption. Some telephone devices disable echo suppression
>> or echo cancellation when they detect the 2100 or 2225 Hz "answer"
>> tones associated with such calls, in accordance with ITU-T
>> recommendation G.164 or G.165. "
>>
>> https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/voice/voiceport/confi
>> guration/15-mt/vp-15-mt-book/vp-cfg-echo-canc.pdf
>>
>> "Additionally, G.165 defines the disabling of echo cancellers in the
>> presence of 2100 Hz signals with periodic phase reversals in order to
>> support echo cancelling modem technology (V.34, for example), which
>> does not work if line echo cancellation is performed in the connection"
>>
>> /Tomas
>>
>> mån 2018-09-10 klockan 11:04 +0700 skrev Đích Kiều:
>>> Hi everyone,
>>>
>>> Anyone working on Freedv on PSTN network ? I’m have a problem about
>>> hybrid echo cancel in PSTN network. Half-duplex freedv work well. But
>>> when I run full-duplex freedv in PSTN, hybrid echo cancel from PABX
>>> make demodulator don’t work. Anyone can show me how to resolve my
>>> problem?
>>>
>>> Thank you.
>>>
>>> Sent from my iPhone
>>>
>>> ___
>>> Freetel-codec2 mailing list
>>> Freetel-codec2@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
>>
>> ___
>> Freetel-codec2 mailing list
>> Freetel-codec2@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
>>
>
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2



___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] ST-Link

[Danilo Beuche]

Hi Jacob,

if you own a "real" ST Link, you have to connect the target voltage pin
on the debugger (or to some other source of 3.3V). For a cheap ST-Link
V2 clone you typically need to connect  SWDIO, SWCLK and GND only.
Sometimes you may need to connect NRST as well, but usually not.

The most common mistake I do is to swap CLK and DIO cables. Next is the
software, I did not check, but the pins can be configured to be used as
GPIO. In this case debugging works only if coming from reset.

For testing if the connection works I often use the STM32 Link Utility
on windows (Target->Connect). Then I get my debugger configuration to
work after knowing the basic cabling works. 

However, I since I don't own a SM1000 I never used a debugger on this
specific target.

Regards,
Danilo



On 03.08.2018 13:58, Jacob Falzon wrote:
> Hey there,
> recently been trying to debug an SM1000-D with an ST-link/V2 ICD.
>
> I connected the SW and GND pins, tried in both flash and system memory
> yet can't seem to connect to target. 
>
> Any ideas?
>
>
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>
>
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] noob in voice coding asking for help about Codec2 decoder.

[Danilo Beuche]

Hi,

btw, using the gcc's -flto during compiling and linking does a great job
of removing unused data and functions. Basically the gcc breaks up every
file into individual linkable elements (each global variable and
functions becomes an independent linkage element) and anything not
referenced anywhere simply disappears from the final binary. Of course
there is a lot more going on with -flto (LTO stands for link time
optimization) but one of the side effects of using LTO is this removal
of unreferenced elements from binary code.

Other compilers have similar "tricks".

Regards,
Danilo

On 11.06.2018 03:19, Bruce Perens wrote:
> Hi Sumek,
>
> I am curious about your memory saving strategy. Specifically, why
> using the stack vs. malloc() would result in a significant memory
> savings, and how removing structs would result in a significant memory
> savings.
>
> Perhaps you mean you are removing /unused /ones?
>
> Or is there something about the Windows compiler I don't understand?
>
>     Thanks
>
>     Bruce
>
> On Sun, Jun 10, 2018 at 1:40 PM, David Rowe  > wrote:
>
> Hi Sumek,
>
> Oh I have one question, do you know if we can reduce the size
> of codebook especially lsp_cbjvm
>
>
> Many of those tables aren't used, so just link in the ones you
> need to support the Codec 2 modes you would like to use.
>
> Cheers,
>
> David
>
>
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> 
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
> 
>
>
>
>
> -- 
> Bruce Perens K6BP - CEO, Legal Engineering
> Standards committee chair, license review committee member,
> co-founder, Open Source Initiative
> President, Open Research Institute; Board Member, Fashion Freedom
> Initiative.
>
>
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>
>
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] FreeDV 700D @ STM32F7

[Danilo Beuche]

Hi all,

further investigation on the UHSDR F7 shows an interesting problem: in
each (!) frame first part (roughly 20ms or so) of the 160ms frame is
distorted and shows extremely high I and Q values. If I ran the same
simple code both on PC and UHSDR Firmware without an other stuff, I get
very different frames for silence. Which is weird. They should be
similar if not identical.  The unwanted noise is created inside the
functions freedv_comptx. I have not had the time to track the problem
down but it could be an uninitialized memory area being used somewhere
(malloced but not cleared, on PC you often still have zeroed out memory
if if you don't explicitly clear it, on the STM32F7 this is not the
case, it will have random value but they tend to be stable, i.e. after
reboot you get similar or identical values in the memory). Since
codec2_encode function is used in all of the modes, I assume the problem
is not there. I will try to track this down maybe next weekend but if
someone else has an idea where to look for issues, I'd appreciate that. 
You'll need an F7 or H7 to run the code, it is #ifndef CORTEX_M4 so not
so easy to run it on a F4 target. Memory could be an issue too on F4.

Rough outline of the relevant parts of code I am running on the PC
(modified freedv_tx.c). In the UHSDR firmware it is  identical minus the
fprintf, there I use the ST Link debugger to check values in memory.

...

    struct freedv* freedv = freedv_open(FREEDV_MODE_700D );

    freedv_set_snr_squelch_thresh(freedv, -100.0);
    freedv_set_squelch_en(freedv, 1);
    freedv_set_clip(freedv, use_clip);
    freedv_set_tx_bpf(freedv, use_txbpf);


    memset(speech_in,0,sizeof(short)*n_speech_samples);
    COMP mod_out_c[1280];

     freedv_comptx(freedv, mod_out_c, speech_in);

  for (int i = 0; i < n_nom_modem_samples;i+=4)
   {
    for (int j=0; j < 4; j++)
    {
    fprintf(stderr,"%f %f
",mod_out_c[i+j].real,mod_out_c[i+j].imag);
    }
    fprintf(stderr,"\n");
   }

    exit(1);

...

73
Danilo


On 07.06.2018 08:22, Danilo Beuche wrote:
> Hi David,
>
> further investigation showed that the UHSDR firmware used the wrong
> buffer size (which caused the crashes...). Now this works and we can see
> that we produce 160ms frames back to back (so performance is probably
> more or less sufficient on the F7) but each frame has some unwanted
> noise of exactly 20ms in it instead of the FreeDV signal. This happens
> in always exactly the same position in each frame so I assume it is a
> fault on our UHSDR buffer handling side. We'll see.
>
> 73
> Danilo
> On 06.06.2018 22:53, David Rowe wrote:
>> Hi Danilo,
>>
>> I'm surprised that (especially) the tx is too slow for that class of
>> CPU. The modem (especially the modulator) is quite simple in terms of
>> CPU compared to the early parallel tone modems, e.g. for FreeDV 1600.
>>
>> On a x86 I get similar CPU load for 1600 and 700D - and 700D has
>> powerful FEC.
>>
>> Do you ave any profiling information you can share?
>>
>> Thanks,
>>
>> David
>>
>> On 07/06/18 06:08, Danilo Beuche wrote:
>>> Hi,
>>>
>>> I have been working to get FreeDV 700D running on the STM32F743 as part
>>> of the UHSDR firmware. After being stopped for while by a bug in the STM
>>> supplied data cache init code I was able to get the OVI40 TRX to
>>> transmit for a brief moment the FreeDV sound to my RX. But unfortunately
>>> the STM32F7 is not fast enough to transmit 700D continuously. It is
>>> simply not able to produce a 160ms frame fast enough. Can't say anything
>>> about receive yet, but I can see also a serious load when in RX.
>>>
>>> Next up is to run the same code on a STM32H7, I have some hope that this
>>> one is fast enough with the code as it is (it is about twice as fast the
>>> STM32F7 clockwise).
>>>
>>> 73
>>> Danilo
>>>
>>>
>>> --
>>>
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>> ___
>>> Freetel-codec2 mailing list
>>> Freetel-codec2@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
>>>
>> --
>>
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>

Re: [Freetel-codec2] FreeDV 700D @ STM32F7

[Danilo Beuche]

Hi David,

further investigation showed that the UHSDR firmware used the wrong
buffer size (which caused the crashes...). Now this works and we can see
that we produce 160ms frames back to back (so performance is probably
more or less sufficient on the F7) but each frame has some unwanted
noise of exactly 20ms in it instead of the FreeDV signal. This happens
in always exactly the same position in each frame so I assume it is a
fault on our UHSDR buffer handling side. We'll see.

73
Danilo
On 06.06.2018 22:53, David Rowe wrote:
> Hi Danilo,
>
> I'm surprised that (especially) the tx is too slow for that class of
> CPU. The modem (especially the modulator) is quite simple in terms of
> CPU compared to the early parallel tone modems, e.g. for FreeDV 1600.
>
> On a x86 I get similar CPU load for 1600 and 700D - and 700D has
> powerful FEC.
>
> Do you ave any profiling information you can share?
>
> Thanks,
>
> David
>
> On 07/06/18 06:08, Danilo Beuche wrote:
>> Hi,
>>
>> I have been working to get FreeDV 700D running on the STM32F743 as part
>> of the UHSDR firmware. After being stopped for while by a bug in the STM
>> supplied data cache init code I was able to get the OVI40 TRX to
>> transmit for a brief moment the FreeDV sound to my RX. But unfortunately
>> the STM32F7 is not fast enough to transmit 700D continuously. It is
>> simply not able to produce a 160ms frame fast enough. Can't say anything
>> about receive yet, but I can see also a serious load when in RX.
>>
>> Next up is to run the same code on a STM32H7, I have some hope that this
>> one is fast enough with the code as it is (it is about twice as fast the
>> STM32F7 clockwise).
>>
>> 73
>> Danilo
>>
>>
>> --
>>
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> ___
>> Freetel-codec2 mailing list
>> Freetel-codec2@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
>>
>
> --
>
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] FreeDV 700D @ STM32F7

[Danilo Beuche]

Hi David,


On 06.06.2018 22:53, David Rowe wrote:
> Hi Danilo,
>
> I'm surprised that (especially) the tx is too slow for that class of
> CPU. The modem (especially the modulator) is quite simple in terms of
> CPU compared to the early parallel tone modems, e.g. for FreeDV 1600.
>
> On a x86 I get similar CPU load for 1600 and 700D - and 700D has
> powerful FEC.
>
> Do you ave any profiling information you can share?
>
No, not yet. Right now I am running with data cache disabled, that may
slow down things a lot (Not sure about the impact of an disabled data
cache to be honest).  Well, I'll try to look into this, but it might
take a while to get down to the root cause. It may also be caused by the
very long processing time due to the 160ms frames (vs. 20ms frames for
FreeDV 1600). I did not mention this, but the UHSDR firmware crashes
quite soon after starting TX for 700D (which it does not do for any
other mode), this is clearly a sign for a serious issue ;-)
But before this I can hear at least some frames coming with some silence
in between.
 
73
Danilo


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

[Freetel-codec2] FreeDV 700D @ STM32F7

[Danilo Beuche]

Hi,

I have been working to get FreeDV 700D running on the STM32F743 as part
of the UHSDR firmware. After being stopped for while by a bug in the STM
supplied data cache init code I was able to get the OVI40 TRX to
transmit for a brief moment the FreeDV sound to my RX. But unfortunately
the STM32F7 is not fast enough to transmit 700D continuously. It is
simply not able to produce a 160ms frame fast enough. Can't say anything
about receive yet, but I can see also a serious load when in RX.

Next up is to run the same code on a STM32H7, I have some hope that this
one is fast enough with the code as it is (it is about twice as fast the
STM32F7 clockwise).

73
Danilo   


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] stm32 memory

[Danilo Beuche]

Hi Dana,


On 31.05.2018 09:01, Dana Myers wrote:
> On 5/30/2018 11:41 PM, Steve wrote:
>> It just dawned on me, that the stm32 normally executes its programs
>> out of flash.
>>
>> I was thinking the flash stored the software, and the first thing it
>> did, was move the code to ram for execution. I was thinking the flash
>> access time was too slow.
>
> It depends on the specific STM32 part you have. STM32F4-series parts have
> 128-bit wide flash that effectively runs at 0-wait. I haven't verified
> the
> bigger STM32F7-series but would be surprised if this wasn't also the
> case there.
>
The STM32F4 needs 6 or 7 wait states configuration for random flash
access if running at 168/180 Mhz top speed according to my memory. Since
the processor fetches 4 32 bit values at time from flash, for linear
access it is not as bad most of the time but still slower than RAM.
Reducing flash data access was one of the  tricks I used to speed up 
the codec2 code so that we could run it as part of the UHSDR firmware on
STM32F4 (the UHSDR has to run full SDR TRX DSP as opposed to the SM1000
which just does audio processing part).
> Some of the smaller STM32F3-series parts have 'CCM' RAM which can
> be executed from, but it's a bit of a hassle to set up.
>
Also the STM32F4 has 64k of CCM RAM, which can be used to run some
highspeed code (or data but no DMA there). The various STM32 devices
have different memories with funny names like ITCM, CCM, DTCM, ... all
of these have special properties (faster for code, for data, not
disturbable by DMA, ... ).

73
Danilo
> Cheers,
> Dana  K6JQ
>
>
> --
>
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] stm32 memory

[Danilo Beuche]

Hi,


On 31.05.2018 08:41, Steve wrote:
> It just dawned on me, that the stm32 normally executes its programs
> out of flash.
>
Yes, this is often the case. See below.
> I was thinking the flash stored the software, and the first thing it
> did, was move the code to ram for execution. I was thinking the flash
> access time was too slow.
>
It is slower, indeed but due to the fairly low amount of internal RAM,
for the STM32F4 the code is in general executed straight out of flash.
Of course it is possible to move some or all code to RAM if you have
enough spare. However, I can tell you from experience, that FreeDV as a
whole cannot be moved into RAM since the math (RAM requirements + Code
size > available RAM) is against you.  Moving the most time critical
portion of the code to RAM is a lot more tricky than to move all code to
RAM but doable. That involves in case of GCC usage link script writing
and also some additional runtime/OS support for the code move. Don't
know how to achieve this in Nuttx.
> I've been playing with the NUTTX OS and it has some options about
> where to run the program and my clue light just came on.
>
See above.
> By the way, with Nuttx I was able to talk to the development board
> using a usb cable and minicom, and play with the "nut" shell. I tried
> the ram disk application and it worked as advertised.
>
BTW, newer STM32 processor such as the STM32F7 and STM32H7 families have
real caches for data and instructions, which to a certain degree reduce
the flash speed problem. They at least execute the same code much faster
if caches are turned on. No such thing on the STM32F4, though.
> Steve
Danilo


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] New Codec2-0.8 Build Failure

[Danilo Beuche]

Hi Matt,

okay. Error message is from me, indicating that the CMake compiler test
failed to detect the C99 / C11 capabilities, it does not technically
mean the compiler cannot do it.  And not being an CMake expert is no
help here, but I think I got it now almost right:

Index: CMakeLists.txt
===
--- ../CMakeLists.txt   (revision 3592)
+++ ../CMakeLists.txt   (working copy)
@@ -55,9 +55,16 @@
 # Set default C++ flags.
 include(CheckCCompilerFlag)
 set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wall -g -O3")
+
 CHECK_C_COMPILER_FLAG("-std=gnu11" COMPILER_SUPPORTS_GNU11)
-if(CMAKE_C_STANDARD_COMPUTED_DEFAULT EQUAL "90")
-set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -std=gnu99")
+CHECK_C_COMPILER_FLAG("-std=gnu99" COMPILER_SUPPORTS_GNU99)
+
+if(COMPILER_SUPPORTS_GNU11)
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -std=gnu11")
+elseif(COMPILER_SUPPORTS_GNU99)
+   set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -std=gnu99")
+else()
+   message( SEND_ERROR "Compiler doesm't seem to support at least gnu99, might 
cause problems" )
 endif()
 # set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-unused-function")

Key to success is to delete CMakeCache.txt in the build directory, as it
seems it caches the result of the variable even if you remove the
calculation of the variable (which I did, but I did not knew it was the
script itself which calculated and cached it).

Give it a try.

Regards,
Danilo



On 27.05.2018 20:12, Matt Roberts wrote:
> Hi Danilo,
>
> When I try your patch on Ubuntu Trusty or Raspbian Stretch, it fails
> trying to get either C11 or C99 support during the initial 'cmake' run.
>
> The error during 'cmake' is just the 'else' message:  "Compiler
> doesm't seem to support at least gnu99, might cause problems"
>
> On Ubuntu, the gcc is: gcc (Ubuntu 4.8.4-2ubuntu1~14.04.4) 4.8.4
>
> On Raspbian: gcc (Raspbian 6.3.0-18+rpi1+deb9u1) 6.3.0 20170516
>
> According to the man page, C99 and C11 support on gcc is still
> considered "incomplete" and "experimental."  FWIW.
>
> 73,
> Matt
>
>
> --
>
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] New Codec2-0.8 Build Failure

[Danilo Beuche]

Hi,

I just checked what goes on in the codec2-dev/CMakelist.txt and I
changed the (useless) GNU11/GNU99 "detection" to do something useful. 

I tested this on Ubuntu 16.04 with 0.8.

Index: CMakeLists.txt
===
--- CMakeLists.txt  (revision 3592)
+++ CMakeLists.txt  (working copy)
@@ -55,9 +55,13 @@
 # Set default C++ flags.
 include(CheckCCompilerFlag)
 set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wall -g -O3")
-CHECK_C_COMPILER_FLAG("-std=gnu11" COMPILER_SUPPORTS_GNU11)
-if(CMAKE_C_STANDARD_COMPUTED_DEFAULT EQUAL "90")
-set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -std=gnu99")
+
+if(COMPILER_SUPPORTS_GNU11)
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -std=gnu11")
+elseif(COMPILER_SUPPORTS_GNU99)
+   set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -std=gnu99")
+else()
+   message( SEND_ERROR "Compiler doesm't seem to support at least gnu99, might 
cause problems" )
 endif()
 # set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-unused-function")


On 27.05.2018 18:44, Matt Roberts wrote:
> When building the new codec2-0.8 library, I ran into a build error
> that I don't see in the 0.7 branch, even when the build environments
> are identical.
>
> A small 'diff' is below which will allow the library to build
> completely, along with a short explanation.
>
> --
>
> $ cmake -DCMAKE_BUILD_TYPE=Release ../codec2-0.8
>
> -- The C compiler identification is GNU 4.8.4
> -- Check for working C compiler: /usr/bin/cc
> -- Check for working C compiler: /usr/bin/cc -- works
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- codec2 version: 0.8
> -- Performing Test COMPILER_SUPPORTS_GNU11
> -- Performing Test COMPILER_SUPPORTS_GNU11 - Success
> -- Build type is: Release
> -- Compiler Flags: -O2 -march=native -mtune=native -Wall -g -O3
> -- Looking for include file stdlib.h
> -- Looking for include file stdlib.h - found
> -- Looking for include file string.h
> -- Looking for include file string.h - found
> -- Looking for floor
> -- Looking for floor - found
> -- Looking for ceil
> -- Looking for ceil - found
> -- Looking for pow
> -- Looking for pow - found
> -- Looking for sqrt
> -- Looking for sqrt - found
> -- Looking for sin
> -- Looking for sin - found
> -- Looking for cos
> -- Looking for cos - found
> -- Looking for atan2
> -- Looking for atan2 - found
> -- Looking for log10
> -- Looking for log10 - found
> -- Looking for round
> -- Looking for round - found
> -- Looking for getopt
> -- Looking for getopt - found
> -- Configuring done
> -- Generating done
> -- Build files have been written to:
> /home/mattro/Projects/freedv/build-codec2
>
> $ make
> -- snip --
> [ 35%] Building C object src/CMakeFiles/codec2.dir/freedv_api.c.o
> /home/mattro/Projects/freedv/codec2-0.8/src/freedv_api.c: In function
> ‘freedv_open_advanced’:
> /home/mattro/Projects/freedv/codec2-0.8/src/freedv_api.c:209:9: error:
> ‘for’ loop initial declarations are only allowed in C99 mode
> for (int i=0; iinterleave_frames*coded_syms_per_frame; i++) {
> ^
> /home/mattro/Projects/freedv/codec2-0.8/src/freedv_api.c:209:9: note:
> use option -std=c99 or -std=gnu99 to compile your code
> /home/mattro/Projects/freedv/codec2-0.8/src/freedv_api.c:227:18:
> error: redefinition of ‘i’
> for (int i=0; iinterleave_frames*f->n_nat_modem_samples; i++) {
> ^
> /home/mattro/Projects/freedv/codec2-0.8/src/freedv_api.c:209:18: note:
> previous definition of ‘i’ was here
> for (int i=0; iinterleave_frames*coded_syms_per_frame; i++) {
> ^
> /home/mattro/Projects/freedv/codec2-0.8/src/freedv_api.c:227:9: error:
> ‘for’ loop initial declarations are only allowed in C99 mode
> for (int i=0; iinterleave_frames*f->n_nat_modem_samples; i++) {
> ^
> make[2]: *** [src/CMakeFiles/codec2.dir/freedv_api.c.o] Error 1
> make[1]: *** [src/CMakeFiles/codec2.dir/all] Error 2
> make: *** [all] Error 2
>
> --
>
> The equivalent build steps for 0.7 work fine. Perhaps the "int i"
> could be moved out of the 'for' statement for maximum compiler
> compatibility?
>
> When I make that small change to lines 209 and 227, the 0.8 library
> builds fine.
>
> --
>
> diff -ur codec2-0.8/src/freedv_api.c codec2-0.8a/src/freedv_api.c
> --- codec2-0.8/src/freedv_api.c    2018-05-24 04:41:22.0 -0500
> +++ codec2-0.8a/src/freedv_api.c    2018-05-27 11:38:20.986569633 -0500
> @@ -206,7 +206,8 @@
>  if (f->codeword_symbols == NULL) {return NULL;}
>  f->codeword_amps =
> (float*)malloc(sizeof(float)*f->interleave_frames*coded_syms_per_frame);
>  if (f->codeword_amps == NULL) {return NULL;}
> -    for (int i=0; iinterleave_frames*coded_syms_per_frame;
> i++) {
> +        int i = 0;
> +    for (i=0; iinterleave_frames*coded_syms_per_frame; i++) {
>  f->codeword_symbols[i].real = 0.0;
>  f->codeword_symbols[i].imag = 0.0;
>  f->codeword_amps[i] = 0.0;
> @@ -224,7 +225,7 @@
>
>  f

Re: [Freetel-codec2] New Codec2-0.8 Build Failure

[Danilo Beuche]

Hi,

instead of going back to the C syntax from the 1960's I would rather
suggest to adjust the build settings to use the C99 mode, as suggested
by the compiler error message. C99 is as the name suggests, from around
1999, so all C compilers from within the last 10 years or so should have
no problem whatsoever with it. I haven't checked since which version C99
is supported by gcc, but gcc 4.0 from 2007 does for sure.

The command line option is -std=c99

Just my opinion. 

Regards,
Danilo



On 27.05.2018 18:44, Matt Roberts wrote:
> When building the new codec2-0.8 library, I ran into a build error
> that I don't see in the 0.7 branch, even when the build environments
> are identical.
>
> A small 'diff' is below which will allow the library to build
> completely, along with a short explanation.
>
> --
>
> $ cmake -DCMAKE_BUILD_TYPE=Release ../codec2-0.8
>
> -- The C compiler identification is GNU 4.8.4
> -- Check for working C compiler: /usr/bin/cc
> -- Check for working C compiler: /usr/bin/cc -- works
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- codec2 version: 0.8
> -- Performing Test COMPILER_SUPPORTS_GNU11
> -- Performing Test COMPILER_SUPPORTS_GNU11 - Success
> -- Build type is: Release
> -- Compiler Flags: -O2 -march=native -mtune=native -Wall -g -O3
> -- Looking for include file stdlib.h
> -- Looking for include file stdlib.h - found
> -- Looking for include file string.h
> -- Looking for include file string.h - found
> -- Looking for floor
> -- Looking for floor - found
> -- Looking for ceil
> -- Looking for ceil - found
> -- Looking for pow
> -- Looking for pow - found
> -- Looking for sqrt
> -- Looking for sqrt - found
> -- Looking for sin
> -- Looking for sin - found
> -- Looking for cos
> -- Looking for cos - found
> -- Looking for atan2
> -- Looking for atan2 - found
> -- Looking for log10
> -- Looking for log10 - found
> -- Looking for round
> -- Looking for round - found
> -- Looking for getopt
> -- Looking for getopt - found
> -- Configuring done
> -- Generating done
> -- Build files have been written to:
> /home/mattro/Projects/freedv/build-codec2
>
> $ make
> -- snip --
> [ 35%] Building C object src/CMakeFiles/codec2.dir/freedv_api.c.o
> /home/mattro/Projects/freedv/codec2-0.8/src/freedv_api.c: In function
> ‘freedv_open_advanced’:
> /home/mattro/Projects/freedv/codec2-0.8/src/freedv_api.c:209:9: error:
> ‘for’ loop initial declarations are only allowed in C99 mode
> for (int i=0; iinterleave_frames*coded_syms_per_frame; i++) {
> ^
> /home/mattro/Projects/freedv/codec2-0.8/src/freedv_api.c:209:9: note:
> use option -std=c99 or -std=gnu99 to compile your code
> /home/mattro/Projects/freedv/codec2-0.8/src/freedv_api.c:227:18:
> error: redefinition of ‘i’
> for (int i=0; iinterleave_frames*f->n_nat_modem_samples; i++) {
> ^
> /home/mattro/Projects/freedv/codec2-0.8/src/freedv_api.c:209:18: note:
> previous definition of ‘i’ was here
> for (int i=0; iinterleave_frames*coded_syms_per_frame; i++) {
> ^
> /home/mattro/Projects/freedv/codec2-0.8/src/freedv_api.c:227:9: error:
> ‘for’ loop initial declarations are only allowed in C99 mode
> for (int i=0; iinterleave_frames*f->n_nat_modem_samples; i++) {
> ^
> make[2]: *** [src/CMakeFiles/codec2.dir/freedv_api.c.o] Error 1
> make[1]: *** [src/CMakeFiles/codec2.dir/all] Error 2
> make: *** [all] Error 2
>
> --
>
> The equivalent build steps for 0.7 work fine. Perhaps the "int i"
> could be moved out of the 'for' statement for maximum compiler
> compatibility?
>
> When I make that small change to lines 209 and 227, the 0.8 library
> builds fine.
>
> --
>
> diff -ur codec2-0.8/src/freedv_api.c codec2-0.8a/src/freedv_api.c
> --- codec2-0.8/src/freedv_api.c    2018-05-24 04:41:22.0 -0500
> +++ codec2-0.8a/src/freedv_api.c    2018-05-27 11:38:20.986569633 -0500
> @@ -206,7 +206,8 @@
>  if (f->codeword_symbols == NULL) {return NULL;}
>  f->codeword_amps =
> (float*)malloc(sizeof(float)*f->interleave_frames*coded_syms_per_frame);
>  if (f->codeword_amps == NULL) {return NULL;}
> -    for (int i=0; iinterleave_frames*coded_syms_per_frame;
> i++) {
> +        int i = 0;
> +    for (i=0; iinterleave_frames*coded_syms_per_frame; i++) {
>  f->codeword_symbols[i].real = 0.0;
>  f->codeword_symbols[i].imag = 0.0;
>  f->codeword_amps[i] = 0.0;
> @@ -224,7 +225,7 @@
>
>  f->mod_out =
> (COMP*)malloc(sizeof(COMP)*f->interleave_frames*f->n_nat_modem_samples);
>  if (f->mod_out == NULL) { return NULL; }
> -    for (int i=0; iinterleave_frames*f->n_nat_modem_samples;
> i++) {
> +    for (i=0; iinterleave_frames*f->n_nat_modem_samples; i++) {
>  f->mod_out[i].real = 0.0;
>  f->mod_out[i].imag = 0.0;
>  }
>
> --
>
> -

Re: [Freetel-codec2] FreeDV 700D CPU usage, good

[Danilo Beuche]

Hi David,

the STM32F7 has about 512K of usable RAM, and one can get close to 1MB
on the STM32H7. It is not much compared to a 10$ Raspberry Pi but way
more than what we have on the SM1000 or STM32F4 UHSDR (aka mcHF).
So I have hopes of getting this to run w/o problems even without
modification, but we'll see. I'll report back (in a couple of days) when
I was able to use the real hardware. In the meantime I may (or may not)
instrument the linux test code to get an idea of memory usage.

Regards,
Danilo

On 04.05.2018 18:21, David Rowe wrote:
> Hi Danilo,
>
> 700D might need some refactoring to run on a small machine, in
> particular the LDPC codec uses some large arrays and doubles.
>
> For example one "modem frame" is 119 complex QPSK symbols (119 * 2
> values/complex symbol * 4 bytes/float * 2 bits/symbol = about 2kbyte
> soft decisions values per frame).  A 16 frame interleaver will need
> arrays of 2kbyte x 16.  Although the interleaving can be limited if
> necessary, I haven't settled on a fixed value for it.
>
> The OFDM modem port should be straight forward, and after a little
> optimisation perhaps use less memory and CPU than the FDMDV modem used
> for 1600.
>
> Codec 2 700C will also need to be ported.
>
> I can give you some tips on refactoring for small machines and porting
> 700D if you are interested on working on this.
>
> Cheers,
>
> David
>
> On 04/05/18 22:02, Danilo Beuche wrote:
>> Hi,
>>
>> we'll shortly be able to test how a CORTEX-M7 single core architecture
>> holds up against 700D performance demands in the UHSDR firmware
>> (STM32F7@216 MHz and STM32H7@400Mhz). And for sake of completeness, we
>> can test it with a STM32F4 @ 168 Mhz but I don't have high hopes here...
>>
>> For the 800 Mhz for the A4-5000, I think this is current frequency when
>> running the cpuinfo command, not necessarily the frequency when the
>> processor is doing 700D (unless the command was called at the right
>> time). A4-5000 ha 1.5Ghz max clock.
>>   Regards,
>> Danilo
>>
>>> model    : 0
>>> model name    : AMD A4-5000 APU with Radeon(TM) HD Graphics
>>> stepping    : 1
>>> microcode    : 0x700010b
>>> cpu MHz    : 800.000
>>> cache size    : 2048 KB
>>> fpu    : yes
>>> fpu_exception    : yes
>>> bogomips    : 2994.54
>>>
>>> top - 18:32:54 up 115 days,  4:43,  4 users,  load average: 1.46,
>>> 1.13, 0.70
>>> Tasks: 291 total,   3 running, 288 sleeping,   0 stopped,   0 zombie
>>> %Cpu(s): 31.8 us,  4.2 sy,  0.0 ni, 63.9 id,  0.0 wa,  0.0 hi,  0.1
>>> si,  0.0
>>> st
>>> KiB Mem :  7592440 total,   205480 free,  1813748 used,  5573212
>>> buff/cache
>>> KiB Swap:  8191996 total,  6313456 free,  1878540 used.  5352648
>>> avail Mem
>>>
>>>    PID USER  PR  NI    VIRT    RES    SHR S  %CPU %MEM TIME+
>>> COMMAND
>>> 28410 alanb 20   0 1205880  56316  38656 R  96.4  0.7   5:33.78
>>> freedv
>>> 11081 alanb 20   0  368284  43696  27040 S  14.9  0.6 282:15.78
>>> Xorg
>>>
>>> This should be indicative of a modern laptop.
>>>
>>> Keep smiling
>>>
>>> Alan VK2ZIW
>>>
>>>
>>> On Fri, 4 May 2018 14:55:21 +0930, David Rowe wrote
>>>> No need for alarm  it's consistent with previous versions of
>>>> FreeDV GUI that people have been running since 2012.
>>>>
>>>> Optimisation of the DSP code won't help, as it doesn't appear to be
>>>> in the modem/codec/FEC, e.g. running the FreeDV stack from the
>>>> command line over 60 second samples:
>>>>
>>>> $ time ./freedv_rx 1600 ~/Desktop/ve9qrp_1600_8010.raw /dev/null
>>>>     real    0m0.748s
>>>> $ time ./freedv_rx 700D ~/Desktop/ve9qrp_700d.wav /dev/null
>>>>     real    0m0.766s
>>>>
>>>> Perhaps 1% CPU.
>>>>
>>>> It's most likely in the GUI code.  If anyone would like to take a
>>>> look at this, I'd be interested in the results of profiling and will
>>>> happily accept patches to reduce CPU.  This is a job anyone with
>>>> general C/C++ coding experience could do - you don't need to
>>>> understand the DSP.
>>>>
>>>> - David
>>>>
>>>> On 04/05/18 12:51, glen english wrote:
>>>>> or rather either that's alot of CPU or the cache is thrashing
>>>>&g

Re: [Freetel-codec2] FreeDV 700D CPU usage, good

[Danilo Beuche]

Hi,

we'll shortly be able to test how a CORTEX-M7 single core architecture
holds up against 700D performance demands in the UHSDR firmware
(STM32F7@216 MHz and STM32H7@400Mhz). And for sake of completeness, we
can test it with a STM32F4 @ 168 Mhz but I don't have high hopes here...

For the 800 Mhz for the A4-5000, I think this is current frequency when
running the cpuinfo command, not necessarily the frequency when the
processor is doing 700D (unless the command was called at the right
time). A4-5000 ha 1.5Ghz max clock.
 
Regards,
Danilo

> model : 0
> model name: AMD A4-5000 APU with Radeon(TM) HD Graphics
> stepping  : 1
> microcode : 0x700010b
> cpu MHz   : 800.000
> cache size: 2048 KB
> fpu   : yes
> fpu_exception : yes
> bogomips  : 2994.54
>
> top - 18:32:54 up 115 days,  4:43,  4 users,  load average: 1.46, 1.13, 0.70
> Tasks: 291 total,   3 running, 288 sleeping,   0 stopped,   0 zombie
> %Cpu(s): 31.8 us,  4.2 sy,  0.0 ni, 63.9 id,  0.0 wa,  0.0 hi,  0.1 si,  0.0 
> st
> KiB Mem :  7592440 total,   205480 free,  1813748 used,  5573212 buff/cache
> KiB Swap:  8191996 total,  6313456 free,  1878540 used.  5352648 avail Mem 
>
>   PID USER  PR  NIVIRTRESSHR S  %CPU %MEM TIME+ COMMAND   
>   
> 28410 alanb 20   0 1205880  56316  38656 R  96.4  0.7   5:33.78 freedv
>   
> 11081 alanb 20   0  368284  43696  27040 S  14.9  0.6 282:15.78 Xorg 
>
> This should be indicative of a modern laptop.
>
> Keep smiling
>
> Alan VK2ZIW
>
>
> On Fri, 4 May 2018 14:55:21 +0930, David Rowe wrote
>> No need for alarm  it's consistent with previous versions of 
>> FreeDV GUI that people have been running since 2012.
>>
>> Optimisation of the DSP code won't help, as it doesn't appear to be 
>> in the modem/codec/FEC, e.g. running the FreeDV stack from the 
>> command line over 60 second samples:
>>
>> $ time ./freedv_rx 1600 ~/Desktop/ve9qrp_1600_8010.raw /dev/null
>>real  0m0.748s
>> $ time ./freedv_rx 700D ~/Desktop/ve9qrp_700d.wav /dev/null
>>real  0m0.766s
>>
>> Perhaps 1% CPU.
>>
>> It's most likely in the GUI code.  If anyone would like to take a 
>> look at this, I'd be interested in the results of profiling and will 
>> happily accept patches to reduce CPU.  This is a job anyone with 
>> general C/C++ coding experience could do - you don't need to 
>> understand the DSP.
>>
>> - David
>>
>> On 04/05/18 12:51, glen english wrote:
>>> or rather either that's alot of CPU or the cache is thrashing
>>>
>>> Is it using SSD / NEON vector instructions ?
>>>
>>>
>>> On 4/05/2018 1:11 PM, Steve wrote:
 The lights aren't dimming in the office, the UPS isn't beeping, and 
 crows aren't falling from the sky :-)

 On Thu, May 3, 2018 at 10:07 PM, glen english >>> > wrote:


     How many watts is that taking?



 -
> - 
 Check out the vibrant tech community on one of the world's most
 engaging tech sites, Slashdot.org! http://sdm.link/slashdot


 ___
 Freetel-codec2 mailing list
 Freetel-codec2@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/freetel-codec2
>>>
>>>
>>> --
>  
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>> ___
>>> Freetel-codec2 mailing list
>>> Freetel-codec2@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
>>>
>> 
> --
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> ___
>> Freetel-codec2 mailing list
>> Freetel-codec2@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
>
> Alan
>
> Evil flourishes when good men do nothing.
> Consider the Christmas child.
> ---
> Alan Beard   Unix Support Technician from 1984 to today
> 70 Wedmore Rd.   Sun Solaris, AIX, HP/UX, Linux, SCO, MIPS
> Emu Heights N.S.W. 2750  Routers, terminal servers, printers, terminals etc..
> +61 2 47353013 (h)   Support Programming, shell scripting, "C", assembler
> 0414 353013 (mobile) After uni, electronics tech
>
>
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.so

Re: [Freetel-codec2] UART, SPI Pins on SM1000

[Danilo Beuche]

Hi,


fyi: storing configuration data in stm32f4 flash is already being
supported by stm32f4_vrom.c/.h in the sm1000 code.


Regards,
Danilo


On 13.03.2018 13:56, Andy Wood wrote:
>
> www.st.com/resource/en/application_note/dm00036065.pdf
> 
>
>
> Application note for using flash as emulated eeprom.
>
>
>
> 
> *From:* Jacob Falzon 
> *Sent:* 13 March 2018 12:48
> *To:* freetel-codec2@lists.sourceforge.net
> *Subject:* Re: [Freetel-codec2] UART, SPI Pins on SM1000
>  
> Thanks a lot for the help.
>
> I'll be sure to take a look at it.
>
> I was afraid that the system was going to run out of memory if I
> expanded on the program.
>
> Ran into something else I couldn't understand on the schematic.
>
> Not all the VDD and VSS pins of the F4 are connected externally
> together. Is there a reason for that?
>  
>
> On Fri, Mar 9, 2018 at 10:43 PM, glen english  > wrote:
>
> The F4 flash is organised into sectors,
>
> small, medium, large
>
> the small sectors are organised into small pages, can be erased on
> their own while the micro is running, and are useful in
> conjunction with a wear levelling algorthm for NV memory, I
> describe below.
>
> I would suggest say choosing a small sector/page for NV.
>
> if you need 64 bytes, use an area of say 4kbytes and store your
> data in consecutive 128 byte lumps.
>
> IE
>
> erase the whole lot
>
> when you need to write settings, write your 64 bytes.
>
> when you need to write again, write into the next 64 bytes
>
> when you need to write again, write into the next 64 bytes
>
> and so on
>
> when you run out of 64 byte chunks (IE you reach the end of the
> page or the sector) and THEN do you trigger a page or sector erase
>
> this minimises  flash wear.
>
> for reading, the software just reads the chunks up to where it
> finds erased (0xff) cells , it knows which is most recent
>
> or you can compute a CRC for each say 60 bytes in the top 4 bytes
> of  the 64 bytes. if it fails crc check, it is probably an unused
> block (or a bad block where if it is your block you need to read
> then you'd read out of main flash defaults and write them in)
>
>
>
>
>
>
>
> On 10/03/2018 8:31 AM, David Rowe wrote:
>
> The EEPROM is not used - plenty of storage on board the
> stm32f4.  Not sure how much memory is being used at run time,
> but its usually pretty tight.  Less than half the flash is
> being used I think.
>
> - David
>
> On 10/03/18 02:11, Jacob Falzon wrote:
>
> Thanks!
>
> If you don't mind, I would also like to ask you regarding
> the EEPROM.
>
> Is it currently being used by the board or is it also
> implemented for expansion purposes? I thought it might
> have been used for storing prefences to be loaded but
> apparently from the comments its eems to be done from
> Flash rather than the external EEPROM.
>
> If it is used was it implemented because of a lack of
> on-board memory on the STM32F4? Might you know how much
> memory is consumed as the SM1000 is running and how I
> could verify it?
>
> Kindest regards and sorry for any bother.
>
>
>
> On Wed, Mar 7, 2018 at 9:43 PM, David Rowe
> mailto:da...@rowetel.com>
> >> wrote:
>
>     Hi Jacob,
>
>     1/ Test is an unused feature at present, you can treat
> it like a GPIO
>
>     2/ We use an opto-isolator so we don't have to provide
> a switch to
>     the external radios PTT circuit.  An
> electro-mechanical relay would
>     have been an alternative.  It's usually a mechanical
> switch on your
>     PTT mic.
>     - David
>
>     On 08/03/18 00:22, Jacob Falzon wrote:
>
>     Hello again,
>
>     there are another two pins for the SM1000 that I
> wanted to know
>     more about.
>
>     The Test pin connects to a Self-Test circuit. What
> exactly is
>     this used to test for please?
>     Also there is another pin named CPPT that connects
> to an
>     optoisolator. I understand that the CPPT signal is
> used to
>     trigger the PTT for the radio but why is the
> opto-isolator
>     implemented please?
>
>     Thank you very much for your attention.
>
>     Kindest regards,
>
>
>
>   

Re: [Freetel-codec2] Help Wanted

[Danilo Beuche]

Hi Glen,

On 09.03.2018 09:44, glen english wrote:
> Agreed.
>
> skip the F4 and go straight to the F7 , unless cost is an issue.
>
Porting the software is an issue as well. F7 needs some more care (it
has real caches for code and data), unless you stick to GPIO, a lot of
higher complexity peripherals have changed more or less slightly. It is
far from a simple recompile. I did the port of the UHSDR F4 to F7 (and
H7) so I gained some insights here. Using the STM32 HAL was helpful, but
even then it takes some effort. BTW, the CODEC2 part of the UHSDR
firmware was just a simple recompile :-)  . So YMMV... If the software
is based on the MMDVM_HS code, then you have the old StdPeripheral lib
code in, which is not available for the STM32F7xx
> CODEC2 was almost 1.7x the speed on the F7 compared to F4 for same
> clock .
Fits with my numbers
>
> The H7s are expensive, don't you think Danilo ?  I am going RT1050 for
> next design.. $5. k
Well, since there is more demand than supply, what do you expect? I see
prices from 14 Euros to more than 21 Euros for the same STM32H743ZIT6
part here in Europe (and most distributors have zero stock). Next
revision will be the first mass produced one and come much later this
year. Then prices will (hopefully) drop to be close to the F7 and then
there is no reason not to use the H7. Peripherals between H7 and F7 are
a lot closer then F4 and F7, at least from the HAL/CubeMX perspective.

Regards,
Danilo


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] Help Wanted

[Danilo Beuche]

Hi Leon,

if you design with a STM32F4xx and 100 pin, I would recommend to choose
one of the newer types (read STM32F469/STM32F479). They have a pinout
compatible to the 100 pin STM32F7xx line (and it is only slightly
different compared to the STM32F40x so a relayout shouldn't be hard).
With this it would be trivial to drop in a much higher performance
STM32F7xx without PCB redesign. The F7xx gives about twice as much
performance as the F4xx with about the same amount of power used (that
is  our experience with the signal processing in the UHSDR/mcHF vs.
UHSDR/OVI40 firmware builds). This restriction does apply for 100pin 
devices only. So if you used a 144 pin variant of the STM32F405, nothing
to worry about.

BTW, once the STM32H7 is really readily available, you get either even
more processing power at 400 Mhz clock for about the same power usage or
at the same clock rate similar performance but less power consumption
due to shrinked MCU design (90 -> 40nm AFAIR).

Danilo



On 09.03.2018 07:00, Leon Lessing wrote:
> Hi all,
>
> Just bear with me here,
> My experience with some of the brandmeister guys has been less than
> spectacular. There is no reason we cannot use "dmr framing" and use
> codec2 to transmit stuff, effectively splitting a dmr channel into 4
> potential codec2 slots.
>
> I have revamped the adf7021 hotspot thing so it is more rf friendly,
> the design is unfortunately in eagle 7.7, mail me for a copy.
>
> This week I have been on holiday and struggling with the lora issue,
> but in the back of my mind I have been thinking mmdvm, hotspot
> hardware adf7021, sm1000 and 1296. I am about 90% finished with a
> mmdvm/sm1000/sdcard hybrid using a stm32f405.
>
> The code from mmdvm_hs uses bit banging to talk to the adf7021, I will
> add an adf7021 to the mix and maybe a rda1846, a gps and esp8266.
> Another idea is to break out the extra control lines and spi bus from
> the stm32f405 to a connector so we can build a stacked system. I'll
> publish the pcbs, I have no idea how we can use git or svn to share
> the pcb work.
>
> There is issues with the adf7021 and harmonics, this is why the
> commercial market uses R5000 (dedicated dmr framing) and the CML data
> pump devices. Dual band matching seems to be an issue with this device
> as well, I am trying to get the hotspot hardware to work on 145Mhz
> without harmonics on 290Mhz.
>
> Please mail me if you need a copy of the files for a peer design
> review, I will use dirtypcbs to do the dev run and give you guys the
> links and you can order pcbs, shipping from South Africa is terribly
> expensive.
>
> Look at the pine64 as an alternative to raspberries. The idea of doing
> a sdr radio is excellent, but mixer bleed through is a massive issue
> for the transmitting side. (most mixers gives 60db local suppression,
> but 60db down on +5dbm local inject is still a very strong signal)
>
> Regards,
> Leon
>
>
>
>
> Leon Lessing
> 雷立安
> ZS6LMG/AC9GU
>
>
>
> On Fri, Mar 9, 2018 at 1:31 AM, Adrian Musceac  > wrote:
>
> Hi Alan,
>
> >
> > I ask, how far are we in being able to completely replace the
> firmware
> > in the lowly MD-380 and clone radios? Or, could we beef up the
> power in
> > the 7021, already used in the project!!
> >
> That (as in running Codec2 2400A) will never happen for technical
> reasons.
>
> Using an SDR it's possible to have Codec2, C4FM, D-Star, DMR and
> whatever else on a single device.
>
> There exist handheld full SDR terminals with a price point of
> $800-$1000, capable of running almost everything you want up to high
> speed video. Documentation is close to zero, writing software for them
> would cost in the high 6 digits figures (not going to happen soon).
> Said terminals will go EOL in 4-5 years as public services transition
> to LTE.
>
> If you want to try some of the new digital modes without spending a
> fortune in equipment and DSP training, I suggest to invest some time
> in learning GNU radio. It's the only viable alternative. I managed to
> make for myself 4 types of Codec2 VHF modems using it. BPSK, QPSK,
> 2FSK and 4FSK. Enough to play with for a while. OP25 (a GNU radio
> project) has had P25, C4FM, D-Star and DMR for a while now. Combining
> them into a single, easy to use application is just a matter of time.
>
> Cheers,
> Adrian
>
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> 
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
> 
>
>
>
>

Re: [Freetel-codec2] Using FDMDV_1600 to transmit data

[Danilo Beuche]

Hi,

when porting the SM1000 code to the UHSDR firmware for the mcHF QRP
receiver (STM32F407V CPU), I ran into problems (read crashes) due to the
large amount of memory the fdmv_create() mallocs (AFAIR, in the area of
40kb now) in comparison to the available memory. With the typical linker
script, you only have 128k RAM available.

Although advertised as 192k RAM the STM3240x MCUs only have 128 k
"normal" RAM typically used in the linker scripts. There is additionally
64 kByte CCM RAM on a different address space with restricted usability
(no DMA for instance). Unless you make extra provision both in code and
in the linker script, these are not used by the GCC linker. So a little
more memory used may bring the system out of balance easily.

Danilo





On 04.01.2018 09:16, David Rowe wrote:
> Very Very common problem to get the "ring of death" due to running out
> of memory somewhere.  At that point I start commenting things out.
>
> - David
>
> On 04/01/18 16:53, Steve wrote:
>> Bruce, I thought about valgrind, I use it on local code. I'm not aware
>> of anything that would work for the stlink remote device. It would be
>> neat to have an sm1000 simulator :-) I've decided to just bite the
>> bullet and go through the code with a look at memory management. I
>> don't think I'm out of memory, or a heap/stack crash, as it's too
>> early in the code. I may run into memory problems, as I'm hoping to
>> fit FDM, 4FSK, Analog, and Tone into the selection. If push comes to
>> shove, I'll toss the FDM, as that mode is already available.
>>
>> On Wed, Jan 3, 2018 at 4:10 PM, Bruce Perens  wrote:
>>> Use Valgrind or Electric Fence to find your memory overrun. It's not
>>> where
>>> that malloc is, but it is overwriting the malloc memory, thus GDB
>>> blows up
>>> in malloc().
>>>
>>> On Wed, Jan 3, 2018 at 1:52 PM, Steve  wrote:

 If anyone clicked the link, it took you to a movie that showed my
 Discovery board with the ring of death. Just some new years humour...

 Mysteriously, when I step through the code with GDB it blows up on a
 malloc() in fdmdv_create().

 This is strange, as the exact same code exists on the standard sm1000
 code, and after stepping through the malloc() it does fine.

 Obviously, my code must not be (really) the same, and I gimped the
 initialization somewhere.

 Happy New Year


 --

 Check out the vibrant tech community on one of the world's most
 engaging tech sites, Slashdot.org! http://sdm.link/slashdot
 ___
 Freetel-codec2 mailing list
 Freetel-codec2@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/freetel-codec2
>>>
>>>
>>>
>>> --
>>>
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>> ___
>>> Freetel-codec2 mailing list
>>> Freetel-codec2@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
>>>
>>
>> --
>>
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> ___
>> Freetel-codec2 mailing list
>> Freetel-codec2@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
>>
>
> --
>
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] arm fft F4 runs 6WS and 5WS

[Danilo Beuche]

Hi Glen,

I just verified, the cycle counts for true 16Mhz are within 1% of the 
8Mhz operations (but measurements now take half as long :-) )

Danilo


Am 18.09.2016 um 07:15 schrieb glen english:
> Hi
> OK.
> well, anyway, decode 1200 (40mS) takes  12.34mS on my kit, and 19.86
> using kiss-fft.
> I think you approximated about 14.4mS for a decode 1300 on your kit
>
> so, I will be interested to see what you come up with using cfft
>
> My codec 2 codebase is AUGUST 2015
>
> cheers
>
>
>
>
>
> On 18/09/2016 2:58 PM, Danilo Beuche wrote:
>> Hi Glen, I would not worry to much:
>>
>> - Maybe gcc 5.4 vs 4.9: difference is ~-20% (depending from which end
>> you are looking).  It is a lot but not unexplainable.
>>
>> - Maybe it is my test data. I don't know how much jitter in the kiss_fft
>> algorithm is, when different data is presented. I am running
>> "artificially" generated audio input (digitally captured codec2 frames
>> from a single 750Hz sine way also generated digitally).-
>>
>> - Maybe it is my strange way of running the mcHF firmware: the mcHF
>> Hardware has a 16Mhz XO,  but the discovery board which I have here for
>> testing has a 8Mhz XO. I didn't bother to reconfigure the PLL. So
>> everything takes twice the time. If the flash would asynchronously
>> coupled, which I doubt (otherwise no need for explicit wait state
>> settings),  it would have an influence.  But here I am quite sure, this
>> is not the case. If the caches are asynchronous: Maybe. Maybe I should
>> remeasure with fixed PLL setup so that the processor runs at true
>> 168Mhz. Will do that later and get back with updated numbers.
>>
>> Danilo
>>
>>
>>
>>
>>
>> On 18.09.2016 06:35, glen english wrote:
>>> Using environment Rowley CrossStudio for ARM 3.6.4 . GCC 4.9
>>>
>>> using cycle counter (yes)
>>>
>>> interrupt overhead : (irrelevant, most likely in my setup) (asm) irqs
>>> only set off flags...
>>>
>>> for kissfft 5ws F4, I wonder why you have 112500 cycles and I have
>>> 141000. Something for me to look at .
>>>
>>> hmm
>>>
>>> -O2 but I also have a bunch of debug symbol stuff in there dunno I think
>>> it is only symbol data at DB2 which pushes up the image size.
>>>
>>>
>>>
>>>
>>> --
>>> ___
>>> Freetel-codec2 mailing list
>>> Freetel-codec2@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
>> --
>> ___
>> Freetel-codec2 mailing list
>> Freetel-codec2@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
>>
>
>
> --
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2



--
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] arm fft F4 runs 6WS and 5WS

[Danilo Beuche]

Hi Glen, I would not worry to much:

- Maybe gcc 5.4 vs 4.9: difference is ~-20% (depending from which end
you are looking).  It is a lot but not unexplainable.

- Maybe it is my test data. I don't know how much jitter in the kiss_fft
algorithm is, when different data is presented. I am running
"artificially" generated audio input (digitally captured codec2 frames
from a single 750Hz sine way also generated digitally).-

- Maybe it is my strange way of running the mcHF firmware: the mcHF
Hardware has a 16Mhz XO,  but the discovery board which I have here for
testing has a 8Mhz XO. I didn't bother to reconfigure the PLL. So
everything takes twice the time. If the flash would asynchronously
coupled, which I doubt (otherwise no need for explicit wait state
settings),  it would have an influence.  But here I am quite sure, this
is not the case. If the caches are asynchronous: Maybe. Maybe I should
remeasure with fixed PLL setup so that the processor runs at true
168Mhz. Will do that later and get back with updated numbers.

Danilo





On 18.09.2016 06:35, glen english wrote:
> Using environment Rowley CrossStudio for ARM 3.6.4 . GCC 4.9
>
> using cycle counter (yes)
>
> interrupt overhead : (irrelevant, most likely in my setup) (asm) irqs 
> only set off flags...
>
> for kissfft 5ws F4, I wonder why you have 112500 cycles and I have 
> 141000. Something for me to look at .
>
> hmm
>
> -O2 but I also have a bunch of debug symbol stuff in there dunno I think 
> it is only symbol data at DB2 which pushes up the image size.
>
>
>
>
> --
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2


--
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] arm fft F4 runs 6WS and 5WS

[Danilo Beuche]

Hi Glen,

what and how are you measuring, in order to repeat measurements in my
environment.

We use the cycle counter of the CORTEX processor (seems you do the
same), interrupt overhead is removed (counter is stopped during the
major interrupt, which is the audio irq).

I have some prerecorded "frames" stored in flash which are feed into the
freedv_comprx routine. Measurements are taken every 50 frames, since
initially the decoder needs to lock on the data.

After 50 frames I get stable measurements.

I did some hotspot analysis and a single kiss_fft 512 fft call was
taking around 670uS  (*168) = 112500 cycles.

So this is in line with your numbers.

My results for kiss_fft vs arm_fft may not be correct, since in this
case fact I was running kiss_fftr vs. arm_rfft_fast_f32.

Maybe there is a huge penalty for the arm_rfft_fast_f32 code.

Let me try the kiss_fft vs. arm_cfft case  to confirm your measurements.

Danilo



On 18.09.2016 06:07, glen english wrote:
> arm fft, stm32F405RGT6, 6WS
> -O2, debug level2.
>
> encode 1200 (40mS frame)
> 1745799 : 10.39mS
> decode
> 2292497 : 13.64mS
>
> so, 2x speed of my other runs
>
> let's run 5WS
>
> encode 1200 (40mS frame)
> 1579922 : 9.4mS
> decode
> 2073979 : 12.34mS
>
> WOW the wait states hurt on the M4 
>
>
>
>
>
>
> --
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2


--
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] F4 runs.....

[Danilo Beuche]

Hi Glen,

difficult to say:

I have data for freedv_comprx() for FreeDV 1600 (translates to decode
1300 because of use of some bits for data transmission)  which does
modem decode every 20ms and voice decode very 40ms.

On average I have 12.2 ms -> 24.4ms, 2x5ms are the fdmdv_demod part, so
decode1300 is 14.4ms.

I am using gcc 5.4, O2 mostly (some files have O3 enable, but none of
the codec2 files.

And I run more or less newest SVN code (codec2-dev 2875)

If you run somewhat older code, the numbers you gave have been the
performance a few days ago. As David mentioned, we gained about 40%
since r2842 (or so). So they could be right.

Danilo




On 18.09.2016 05:57, glen english wrote:
> kiss fft per standard codec2 code...
>
> -O2, debug level2.
> encode 3200 (20mS) frame
> encode : 1388964 cycles: 8.26mS
> decode : 1807440 cycles : 10.75mS
>
> encode 1200 (40mS frame)
> 3006497  (17.89mS)
> decode
> 3669321 (21.8mS)
>
> hmm seems pretty slow
> I wonder what I am doing wrong ?
> Or is that inline with other's measurments ?
>
> kissFFT512 : 155,483 cycles (versus 70,000 on the M7)
>
> Next... arm asm on F4 (STM32F405RGT6)
>
>
>
>
> --
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2


--
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] more benching and thoughts

[Danilo Beuche]

Hi Glen,

Am 18.09.2016 um 05:22 schrieb glen english:
> Hi Danilo
>
> Yes, there is most-certainly a penalty for const access from flash, at
> least on the M4.
>
> and of course instruction cache is no use, is just that, only for
> instructions
Hence the name :-)
> I wonder what the bus matrix penalty is for data fetches from flash.
> There is an app-note about it somewhere I once read. It depends on what
> else is going on. The processor is pretty smart and interleaving the
> accesses as not to stall the pipeline or bus matrix.
>
> As Danilo I am sure you know : (pointed out for others)
> You can force variables into sections (like forcing a static const ) by
> using
>
> __attribute__((section("name"))) to assign say into data.
Yes,  indeed. We use that extensively at the mcHF to move certain parts 
to the CCM memory which is otherwise not so easily accessible (and 
should be used with care as it does not support DMA to/from peripherals).
Works great.
> I will some time run up the code on an M4 and see what kiss-fft does.
>
> I am very very very surprised , and do not really believe that kissFFT
> is as fast as the arm assembler  on the M4 -  my immediate thoughts are
> "you are doing it wrong". so, I will investigate.
Would be happy if I am wrong. Which would mean we get even faster fft on 
the M4 almost for free (minus investigation time that is). No problem at 
all with me :-)


Danilo



--
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] more benching and thoughts

[Danilo Beuche]

Hi Glen,

just checked, it seems to me that we have all the caches running in the 
mcHF:

https://github.com/df8oe/mchf-github/blob/670f94a2e69a55a03f099ad25390925c84c09201/mchf-eclipse/cmsis_boot/system_stm32f4xx.c#L424

So even with these caches/buffers enabled, the M4 looses performance  by 
data reads from flash. Haven't checked the manual/internet but maybe the 
flash caching works well for code but not in the same way for data.


Danilo

Am 18.09.2016 um 03:12 schrieb glen english:
> Hi Danilo
>
> Good thoughts and points.
>
> while on the RAM subject : require reading for every serious programmer :
> "Memory"
> https://lwn.net/Archives/GuestIndex/#Drepper_Ulrich
>
> read all 7 parts, 100 pages, but if you only have an hour, just read
> "part 2 - cache"
>
> On the fft:
>
> I am not surprised that the ARM lib  hand optimized assembler is that
> much faster.
> more that 2x faster in fact.
>
> I don't think kiss-fft is particular suitable for this sort of platform,
> either, I'll hold back what I really think :-) .
>
> The 5WS on flash (actually 6WS I am running @ 168M) does not really
> affect the performance too much.  In fact I can vary the WS count +/- 2
> without much change- the ART and the prefetch and the instruction and
> data caches are doing their job, so there is very little difference with
> the const values in ram or cache.
>
> In fact, most  FFT implementations are very tough on a machine with cache .
> Have you read the paper on how FFTW works ? It is very cache aware- and
> adaptive to the architecture- that is why it does trial runs and picks
> the best.
>
> The M7 is very impressive. It is certainly impressive work by ARM.
>
> However, the M4 is what all of you have to work with so we can stay
> focussed on that.
>
> I think also the ram usage will be significantly less with the arm FFT
> because of the re-entrant Kiss-fft behaviour.
>
> The m4 is quite a different beast, and no D-cache can improve
> performance over the M7 for some (inaptly) written applications (not
> this one- but as a generalization for applications grabbing a byte from
> memory randomly and all over a large dataset)
>
> Large matrix operations are where cache machines fall over- that is once
> the dataset is bigger than the cache
>
> The question is how much optimization is enough. I am tempted NOT to
> optimize any more, although I feel (just by looking at it )  I can get
> another 2x out of it. Why- well there is no real pressing need.
> Going too far away from the reference code will island the code a bit.
> However, if you run out of modem cycles/ modem ram, then we can probably
> get a bit more...
>
> cheers
>
>
>
>
>
>
>
>
>
> --
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2



--
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] more benching and thoughts

[Danilo Beuche]

Hi Glen,


On 18.09.2016 03:12, glen english wrote:
> Hi Danilo
>
> Good thoughts and points.
>
> while on the RAM subject : require reading for every serious programmer :
> "Memory"
> https://lwn.net/Archives/GuestIndex/#Drepper_Ulrich
>
> read all 7 parts, 100 pages, but if you only have an hour, just read 
> "part 2 - cache"
>
> On the fft:
>
> I am not surprised that the ARM lib  hand optimized assembler is that 
> much faster.
> more that 2x faster in fact.
As I said on the m4 kiss fft and arm dsp are on par, not much difference.
And using RAM vs. flash makes a lot of difference on the mcHF M4 Code
for code which uses tables heavily. We gained a lot from removing the
need to go to flash twice in the fir_filter vs. fir_filter2.
Maybe the mcHF startup configuration is not enabling all the caches.
Will check that.
> I don't think kiss-fft is particular suitable for this sort of platform, 
> either, I'll hold back what I really think :-) .
>
> The 5WS on flash (actually 6WS I am running @ 168M) does not really 
> affect the performance too much.  In fact I can vary the WS count +/- 2 
> without much change- the ART and the prefetch and the instruction and 
> data caches are doing their job, so there is very little difference with 
> the const values in ram or cache.
>
> In fact, most  FFT implementations are very tough on a machine with cache .
> Have you read the paper on how FFTW works ? It is very cache aware- and 
> adaptive to the architecture- that is why it does trial runs and picks 
> the best.
>
> The M7 is very impressive. It is certainly impressive work by ARM.
>
> However, the M4 is what all of you have to work with so we can stay 
> focussed on that.
>
> I think also the ram usage will be significantly less with the arm FFT 
> because of the re-entrant Kiss-fft behaviour.
>
> The m4 is quite a different beast, and no D-cache can improve 
> performance over the M7 for some (inaptly) written applications (not 
> this one- but as a generalization for applications grabbing a byte from 
> memory randomly and all over a large dataset)
>
> Large matrix operations are where cache machines fall over- that is once 
> the dataset is bigger than the cache
>
> The question is how much optimization is enough. I am tempted NOT to 
> optimize any more, although I feel (just by looking at it )  I can get 
> another 2x out of it. Why- well there is no real pressing need. 
> Going too far away from the reference code will island the code a bit.
> However, if you run out of modem cycles/ modem ram, then we can probably 
> get a bit more...
>
> cheers
Yes, unfortunately we have a M4 at hand, so a little more RAM would be
nice :-(

Regards,
Danilo
>
>
>
>
>
>
>
>
> --
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2


--
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] more benching and thoughts

[Danilo Beuche]

as I'm careful to avoid blowing out past the +/- 1.0 range, 
it's probably
every bit as good as single-precision floating point. The PSoC 5LP has 
a Cortex-M3,

I'm running it at 80MHz.

 My dynamic buffer implementation uses ...


***
Take a look at the memory management routine  heap2.c in freertos.c
   (in fact, there are heap1,2,3,4,5 .c - a few options... try heap4, also)
-this is a much smarter memory alloc and dealloc routine that is fairly
cheap.
much better than usual brain dead malloc.

I'd recommend using that. It looks for blocks same size, existing used etc


heap_4.c and, while I've never explicitly profiled it, I've never had 
a reason to
suspect the allocator is misbehaving. I commit 32KB to the heap and 
currently

never use more than about 3KB.

I would expect the same improvements on the F4 as the F7 using the CMSIS
library. The F7 is much faster on that sort of code.

I only got rid of the FFT malloc stuff the huge stack additions are
still in there
and you could save 50% there ...


Without knowing the details of this application (I'm new here), I am quite
impressed with the quality of CMSIS-DSP, particularly in terms of 
exploiting

the ARM extensions.

Cheers,
Dana


-glen


On 16/09/2016 12:06 PM, Danilo Beuche wrote:
> Hi Glen,
>
> nice, would be interesting to see how much the STM32F4 gains by use of
> CMSIS FFT routines.
>
> BTW, I am not sure, but I think you mentioned the removal of malloc as
> one of your changes. For us with the mcHF it would not be good to have
> the memory for FreeDV code statically allocated since FreeDV is just one
> operation mode of the mcHF, and we need the memory at other times for
> other stuff, especially since it really eats a lot of memory (in
> relation to the STM32F4 RAM sizes). Even half of it is still a lot.
>
> Looking forward to gain some more free cycles with your work.
>
> Regards,
> Danilo
>
>
> Am 16.09.2016 um 03:53 schrieb glen english:
>> Hi Danilo
>>
>> yeah, you have plenty in hand.
>>
>> OK so M7 and CMSIS FFT,  about 2 x speed (same clock) 7.74mS (1200bps)
>> for decode.
>>
>> On 16/09/2016 11:49 AM, Danilo Beuche wrote:
>>> H
>>>
>>> regarding the times @mcHF (STM32F4, 168Mhz) some clarifications: We
>>> measured 17.3ms per 40ms interval for the voice decode part only (this
>>> is only happening once the modem is synced) and roughly 5ms of
>>> fdmdv_demod per 20ms interval (happens all the time). Which gives us in
>>> total some 27ms per 40ms once synced. This is about 68% load.





--


___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2


/*
 * codec2_fft.h
 *
 *  Created on: 17.09.2016
 *  Author: danilo
 */

#ifndef DRIVERS_FREEDV_CODEC2_FFT_H_
#define DRIVERS_FREEDV_CODEC2_FFT_H_

#include 
#include 
#include 
#include 
#include 

#ifdef ARM_MATH_CM4
  #include "stm32f4xx.h"
  #include "core_cm4.h"
  #include "arm_math.h"
  #include "arm_const_structs.h"
#endif

#include "defines.h"
#include "kiss_fft.h"
#include "kiss_fftr.h"
#include "comp.h"

#ifndef ARM_MATH_CM4
typedef kiss_fftr_cfg codec2_fftr_cfg;
typedef kiss_fft_cfg codec2_fft_cfg;
#else
  typedef struct {
  arm_rfft_fast_instance_f32* instance;
  int inverse;
  } codec2_fftr_struct;

  typedef codec2_fftr_struct* codec2_fftr_cfg;

  typedef struct {
const arm_cfft_instance_f32* instance;
int inverse;
} codec2_fft_struct;
  typedef codec2_fft_struct* codec2_fft_cfg;
#endif

inline codec2_fftr_cfg codec2_fftr_alloc(int nfft, int inverse_fft, void* mem, 
unsigned int* lenmem)
{
codec2_fftr_cfg retval;
#ifndef ARM_MATH_CM4
retval = kiss_fftr_alloc(nfft, inverse_fft, mem, lenmem);
#else
retval = malloc(sizeof(codec2_fftr_struct));
retval->inverse  = inverse_fft;
retval->instance = malloc(sizeof(arm_rfft_fast_instance_f32));
arm_rfft_fast_init_f32(retval->instance,nfft);
#endif
return retval;
}
typedef kiss_fft_scalar codec2_fft_scalar;
typedef COMPcodec2_fft_cpx;

inline void codec2_fftr(codec2_fftr_cfg cfg, codec2_fft_scalar* in, 
codec2_fft_cpx* out)
{

#ifndef ARM_MATH_CM4
  kiss_fftr(cfg, in, (kiss_fft_cpx*)out);
#else
arm_rfft_fast_f32(cfg->instance,in,(float*)out,cfg->inverse);
out->imag = 0; // remove out[FFT_ENC/2]->real stored in out[0].imag
#endif
}

inline void codec2_fftr_free(codec2_fftr_cfg cfg)
{
#ifndef ARM_MATH_CM4
   

Re: [Freetel-codec2] on verification of performance

[Danilo Beuche]

Hi,

of course it is

"import struct". This somehow got lost when I pasted it in.

Danilo


Am 17.09.2016 um 03:21 schrieb Danilo Beuche:


Hi,

I solved my "difference analysis" problem with a simple python script, 
which takes 2 files (left.out and right.out) as input and does 16 bit 
value compares. It expects both files to be of same length. Difference 
is written to diff.out


It compares against an absolute error ( for me set to 1, i.e. least 
significant bit difference is permitted) and 1%. For my change it 
turns out, it creates only LSB difference, so not a problem here.


Danilo

--


import sruct

# word counter
count = 0

# absolute difference before seen as relevant difference
# 1 -> single bit difference
marginAbsolute = 1

# from which (difference divded by reference value (left.out))
# it is considered to be an serious error
marginFactor = 0.01
errorMax = 0.0

# how many differences
differenceCount = 0

# how many differences are higher than errorAbsolute
errorCount = 0

# how many of these differences are considered to cross the margin
errorMarginCount = 0

with open('diff.out', 'wbt') as out:
with open('left.out', 'r') as left:
with open('right.out', 'rb') as right:
try:
lw = struct.unpack('h',left.read(2))[0]
rw = struct.unpack('h',right.read(2))[0]
while (True):
out.write(struct.pack('h',lw-rw))
if abs(lw - rw) > 0:
differenceCount = differenceCount + 1
if abs(lw - rw) > marginAbsolute:
errorCount = errorCount+1
errorValue = float(abs(lw-rw))/float(abs(lw))
errorMax = max(errorMax,errorValue)
if errorValue >= marginFactor:
errorMarginCount = errorMarginCount+1
print "{},{} -> {}".format(lw,rw,errorValue)
lw = struct.unpack('h',left.read(2))[0]
rw = struct.unpack('h',right.read(2))[0]
count = count + 1
except:
pass
print "Differences total (in %): {} 
({}%)\n".format(differenceCount,100.0*differenceCount/float(count))
print "Differences of more than {}: 
{}\n".format(marginAbsolute,errorCount)
print "Differences above {}%: 
{}\n".format(marginFactor*100,errorMarginCount)

print "Highest difference in %: {}".format(errorMax*100)


out.close()



 Am 17.09.2016 um 01:41 schrieb glen english:
it's fairly easy to just use a sliding correlator  and sync up the 
audio for the compare/ MSE computation etc


you might want to oversample it say 4x inside the PC before the 
correlation- up to 32ksps from 8ksps


assumption is it is all done in the digital domain.

if done with SSE instructions, can do 128 bit worth of compare for 
the correlator per clock cycle.. (or 256 for the new AVX inst set)


On 17/09/2016 9:36 AM, Danilo Beuche wrote:


Hi,

to extended my own post:
On 17.09.2016 01:13, Danilo Beuche wrote:




--


___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2




--


___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2


--
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] on verification of performance

[Danilo Beuche]

Hi,

of course it is

"import struct". This somehow got lost when I pasted it in.

Danilo


Am 17.09.2016 um 03:21 schrieb Danilo Beuche:


Hi,

I solved my "difference analysis" problem with a simple python script, 
which takes 2 files (left.out and right.out) as input and does 16 bit 
value compares. It expects both files to be of same length. Difference 
is written to diff.out


It compares against an absolute error ( for me set to 1, i.e. least 
significant bit difference is permitted) and 1%. For my change it 
turns out, it creates only LSB difference, so not a problem here.


Danilo

--


import sruct

# word counter
count = 0

# absolute difference before seen as relevant difference
# 1 -> single bit difference
marginAbsolute = 1

# from which (difference divded by reference value (left.out))
# it is considered to be an serious error
marginFactor = 0.01
errorMax = 0.0

# how many differences
differenceCount = 0

# how many differences are higher than errorAbsolute
errorCount = 0

# how many of these differences are considered to cross the margin
errorMarginCount = 0

with open('diff.out', 'wbt') as out:
with open('left.out', 'r') as left:
with open('right.out', 'rb') as right:
try:
lw = struct.unpack('h',left.read(2))[0]
rw = struct.unpack('h',right.read(2))[0]
while (True):
out.write(struct.pack('h',lw-rw))
if abs(lw - rw) > 0:
differenceCount = differenceCount + 1
if abs(lw - rw) > marginAbsolute:
errorCount = errorCount+1
errorValue = float(abs(lw-rw))/float(abs(lw))
errorMax = max(errorMax,errorValue)
if errorValue >= marginFactor:
errorMarginCount = errorMarginCount+1
print "{},{} -> {}".format(lw,rw,errorValue)
lw = struct.unpack('h',left.read(2))[0]
rw = struct.unpack('h',right.read(2))[0]
count = count + 1
except:
pass
print "Differences total (in %): {} 
({}%)\n".format(differenceCount,100.0*differenceCount/float(count))
print "Differences of more than {}: 
{}\n".format(marginAbsolute,errorCount)
print "Differences above {}%: 
{}\n".format(marginFactor*100,errorMarginCount)

print "Highest difference in %: {}".format(errorMax*100)


out.close()



 Am 17.09.2016 um 01:41 schrieb glen english:
it's fairly easy to just use a sliding correlator  and sync up the 
audio for the compare/ MSE computation etc


you might want to oversample it say 4x inside the PC before the 
correlation- up to 32ksps from 8ksps


assumption is it is all done in the digital domain.

if done with SSE instructions, can do 128 bit worth of compare for 
the correlator per clock cycle.. (or 256 for the new AVX inst set)


On 17/09/2016 9:36 AM, Danilo Beuche wrote:


Hi,

to extended my own post:
On 17.09.2016 01:13, Danilo Beuche wrote:




--


___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2




--


___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2


--
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] on verification of performance

[Danilo Beuche]

Hi,

I solved my "difference analysis" problem with a simple python script, 
which takes 2 files (left.out and right.out) as input and does 16 bit 
value compares. It expects both files to be of same length. Difference 
is written to diff.out


It compares against an absolute error ( for me set to 1, i.e. least 
significant bit difference is permitted) and 1%. For my change it turns 
out, it creates only LSB difference, so not a problem here.


Danilo

--


import sruct

# word counter
count = 0

# absolute difference before seen as relevant difference
# 1 -> single bit difference
marginAbsolute = 1

# from which (difference divded by reference value (left.out))
# it is considered to be an serious error
marginFactor = 0.01
errorMax = 0.0

# how many differences
differenceCount = 0

# how many differences are higher than errorAbsolute
errorCount = 0

# how many of these differences are considered to cross the margin
errorMarginCount = 0

with open('diff.out', 'wbt') as out:
with open('left.out', 'r') as left:
with open('right.out', 'rb') as right:
try:
lw = struct.unpack('h',left.read(2))[0]
rw = struct.unpack('h',right.read(2))[0]
while (True):
out.write(struct.pack('h',lw-rw))
if abs(lw - rw) > 0:
differenceCount = differenceCount + 1
if abs(lw - rw) > marginAbsolute:
errorCount = errorCount+1
errorValue = float(abs(lw-rw))/float(abs(lw))
errorMax = max(errorMax,errorValue)
if errorValue >= marginFactor:
errorMarginCount = errorMarginCount+1
print "{},{} -> {}".format(lw,rw,errorValue)
lw = struct.unpack('h',left.read(2))[0]
rw = struct.unpack('h',right.read(2))[0]
count = count + 1
except:
pass
print "Differences total (in %): {} 
({}%)\n".format(differenceCount,100.0*differenceCount/float(count))
print "Differences of more than {}: 
{}\n".format(marginAbsolute,errorCount)
print "Differences above {}%: 
{}\n".format(marginFactor*100,errorMarginCount)

print "Highest difference in %: {}".format(errorMax*100)


out.close()



 Am 17.09.2016 um 01:41 schrieb glen english:
it's fairly easy to just use a sliding correlator  and sync up the 
audio for the compare/ MSE computation etc


you might want to oversample it say 4x inside the PC before the 
correlation- up to 32ksps from 8ksps


assumption is it is all done in the digital domain.

if done with SSE instructions, can do 128 bit worth of compare for the 
correlator per clock cycle.. (or 256 for the new AVX inst set)


On 17/09/2016 9:36 AM, Danilo Beuche wrote:


Hi,

to extended my own post:
On 17.09.2016 01:13, Danilo Beuche wrote:




--


___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2


--
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] on verification of performance

[Danilo Beuche]

Hi,

to extended my own post:
On 17.09.2016 01:13, Danilo Beuche wrote:
> Hi,
>
> If the goal is to identify regressions, comparing the previously
> generated output data to the output of the changed code is sufficient.
> And for this we can simply compare the value by value difference since
> beside rounding errors the output should be very close. For that a
> simple value by value difference should be good enough. 
>
Of course the assumption is that the output signal data is captured
directly, i.e. not via an audio interface. E.g. freedv_rx / freedv_tx or
similar approaches. Otherwise all sorts of timing errors may occur and
frame identification is necessary.
> For general work a quality check by comparing input and output
> properties like Glen suggested would be nice.
Danilo
>
>
> Danilo
>
> Am 17.09.2016 um 07:53 schrieb glen english  <mailto:g...@cortexrf.com.au>>:
>
>> My video compression work tells me we almost need something else rather
>> than a bit for bit compares.
>>
>> something like a SN  or difference merit
>>
>> Anyone familiar with video compression will understand this concept .
>> We would do an encode- decode and then look at the RMS, peak, and
>> various other stats  difference between the input and output .
>>
>> The tools would line up / correlate the frames to get the compare
>> occurring at the right temporal location.
>>
>> On 17/09
>>
>>
>> --
>> ___
>> Freetel-codec2 mailing list
>> Freetel-codec2@lists.sourceforge.net
>> <mailto:Freetel-codec2@lists.sourceforge.net>
>> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
>
>
> --
>
>
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2

--
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] on verification of performance

[Danilo Beuche]

Hi,

If the goal is to identify regressions, comparing the previously generated 
output data to the output of the changed code is sufficient. And for this we 
can simply compare the value by value difference since beside rounding errors 
the output should be very close. For that a simple value by value difference 
should be good enough. 

For general work a quality check by comparing input and output properties like 
Glen suggested would be nice.


Danilo

> Am 17.09.2016 um 07:53 schrieb glen english :
> 
> My video compression work tells me we almost need something else rather 
> than a bit for bit compares.
> 
> something like a SN  or difference merit
> 
> Anyone familiar with video compression will understand this concept .
> We would do an encode- decode and then look at the RMS, peak, and 
> various other stats  difference between the input and output .
> 
> The tools would line up / correlate the frames to get the compare 
> occurring at the right temporal location.
> 
> On 17/09
> 
> 
> --
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
--
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] more benching and thoughts

[Danilo Beuche]

Hi Glen,


Am 16.09.2016 um 04:13 schrieb glen english:
> Hi Danilo
> Yeah, I guess being a very bare metal programmer from the old 128 byte
> RAM days, , I dislike MALLOCs in embedded code on principal.
Well. Although I also remember that are and in general do not like 
malloc being used by someone not knowing what she/he does,
it does have its fair use cases.
> However, because the heap usage would be deterministic, it should be
> fairly safe.
However, using stack is okay by me, makes it a little harder to monitor 
but the codec2 already uses a fair share of stack in some places.
My concern was a migration to memory allocated statically which sits 
unused most of the time. This would be a true waste.
> ***
> Take a look at the memory management routine  heap2.c in freertos.c
>(in fact, there are heap1,2,3,4,5 .c - a few options... try heap4, also)
> -this is a much smarter memory alloc and dealloc routine that is fairly
> cheap.
> much better than usual brain dead malloc.
> 
> I'd recommend using that. It looks for blocks same size, existing used etc
>
> I would expect the same improvements on the F4 as the F7 using the CMSIS
> library. The F7 is much faster on that sort of code.
>
> I only got rid of the FFT malloc stuff the huge stack additions are
> still in there
> and you could save 50% there ...
Hope to see this soon. In order to not do double work here, we'll wait 
for your changes to materialize. The mcHF is now in a state where we can 
play with FreeDV. While this currently has some impact on the time 
available for task like the spectrum display, this is not critical since 
we can RX and TX without issues AFAIK.

Regards,
Danilo


--
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

Re: [Freetel-codec2] more benching and thoughts

[Danilo Beuche]

Hi Glen,

nice, would be interesting to see how much the STM32F4 gains by use of 
CMSIS FFT routines.

BTW, I am not sure, but I think you mentioned the removal of malloc as 
one of your changes. For us with the mcHF it would not be good to have 
the memory for FreeDV code statically allocated since FreeDV is just one 
operation mode of the mcHF, and we need the memory at other times for 
other stuff, especially since it really eats a lot of memory (in 
relation to the STM32F4 RAM sizes). Even half of it is still a lot.

Looking forward to gain some more free cycles with your work.

Regards,
Danilo


Am 16.09.2016 um 03:53 schrieb glen english:
> Hi Danilo
>
> yeah, you have plenty in hand.
>   
> OK so M7 and CMSIS FFT,  about 2 x speed (same clock) 7.74mS (1200bps)
> for decode.
>
> On 16/09/2016 11:49 AM, Danilo Beuche wrote:
>> H
>>
>> regarding the times @mcHF (STM32F4, 168Mhz) some clarifications: We
>> measured 17.3ms per 40ms interval for the voice decode part only (this
>> is only happening once the modem is synced) and roughly 5ms of
>> fdmdv_demod per 20ms interval (happens all the time). Which gives us in
>> total some 27ms per 40ms once synced. This is about 68% load.
>>
>> Regards,
>>
>> Danilo
>>
>>
>>
>>
>>
>> --
>> ___
>> Freetel-codec2 mailing list
>> Freetel-codec2@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/freetel-codec2
>>
>
>
> --
> ___
> Freetel-codec2 mailing list
> Freetel-codec2@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/freetel-codec2



--
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2

[Freetel-codec2] more benching and thoughts

[Danilo Beuche]

H

regarding the times @mcHF (STM32F4, 168Mhz) some clarifications: We 
measured 17.3ms per 40ms interval for the voice decode part only (this 
is only happening once the modem is synced) and roughly 5ms of 
fdmdv_demod per 20ms interval (happens all the time). Which gives us in 
total some 27ms per 40ms once synced. This is about 68% load.

Regards,

Danilo





--
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2