Re: [Freetel-codec2] Modulation Woes on SM1000
Hi,
Just want to point out that, last time I tested them, the 4FSK waveforms
(FreeDV 2400A, and 800XA) work just fine - unmodified and without a
scrambler.
800XA is being used by people over the air - in fact I'm working with
some UK Hams on exciting high power experiments using simple and compact
Class E amplifiers. There is a lot of link margin possible compared to
OFDM waveforms through linear PAs.
Now 800XA is a FreeDV GUI mode that anyone can run . pls feel free
to try it.
Two years ago Brady ran 2400A (using a 4FSK modem) on the SM2000
prototypes which use the stm32 - Glen IIRC you were in the Gipstech
audience :-)
So what we have is:
i) Code that works on the x86 platform
ii) Code that doesn't on the stm32
This suggests a porting issue to me. Steve - happy to work with you on
a series of unit tests to take us from x86 FreeDV API cmd line test
programs to working code on the stm32. It is possible there is some
corner case that we are missing, if so some careful unit tests will show
that up.
Cheers,
David
On 14/06/18 13:20, Steve wrote:
Yes. You can't see it on the FreeDV spectrum display, as that display is
filtered and averaging. But I'm sure an oscillator or two disappears.
This particular scrambler resets itself every frame. It has a starting
bit pattern. So every modem frame starts-off with the same value, and is
only 64 bits long (two codec frames). So errors may only be 80 ms in
length, before the next modem frame.
The additive scrambler for HORUS was taken out of wikipedia
1+x^{{-14}}+x^{{-15}}
On Wed, Jun 13, 2018 at 10:37 PM glen english > wrote:
The downside of most scramblers (I dont know what is used here) is that
you can get error propagtion in a scrambler.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Modulation Woes on SM1000
The simplist explanation for this is that if there was only one tone present (out of the four) , IE in the extreme case, the input to the modulator was stuck on one symbol value, there demodulator would not know what tone that was with respect to any of the others, not be able to derive any relative frequency error, and not be able to track the symbol clock. The downside of most scramblers (I dont know what is used here) is that you can get error propagtion in a scrambler. One bit in error will often produce multiple bit errors until the de scrambling shift register is cleaned out. To reduce this problem, many modems that seek to prevent long runs on a particular value will force a few transitions every frame or so, this is built into the scheme, so there is no error propagation, but it does cost some overhead. There are a few other techniques but they are the basics Glen On 14/06/2018 1:20 PM, Steve wrote: I played around a bit with the bit scrambler, first to make sure it was a two-way function, and then by running the sm1000 scrambled result into FreeDV on my x86. Without the scrambler, after I was done talking, the frequency error would waver around +/- and then slide off almost -600 Hz. The SNR would then drop down and become un-synced. Then when I started talking again it would correct itself, and the SNR would go back to max and sync. With the scrambler on however, the frequency error toggled around +/- 7.5 Hz off. The SNR never dropped down. FYI Steve -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2 -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Modulation Woes on SM1000
Yes. You can't see it on the FreeDV spectrum display, as that display is
filtered and averaging. But I'm sure an oscillator or two disappears.
This particular scrambler resets itself every frame. It has a starting bit
pattern. So every modem frame starts-off with the same value, and is only
64 bits long (two codec frames). So errors may only be 80 ms in length,
before the next modem frame.
The additive scrambler for HORUS was taken out of wikipedia
[image: 1+x^{{-14}}+x^{{-15}}]
On Wed, Jun 13, 2018 at 10:37 PM glen english wrote:
>
> The downside of most scramblers (I dont know what is used here) is that
> you can get error propagtion in a scrambler.
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Modulation Woes on SM1000
The simplist explanation for this is that if there was only one tone present (out of the four) , IE in the extreme case, the input to the modulator was stuck on one symbol value, there demodulator would not know what tone that was with respect to any of the others, not be able to derive any relative frequency error, and not be able to track the symbol clock. The downside of most scramblers (I dont know what is used here) is that you can get error propagtion in a scrambler. One bit in error will often produce multiple bit errors until the de scrambling shift register is cleaned out. To reduce this problem, many modems that seek to prevent long runs on a particular value will force a few transitions every frame or so, this is built into the scheme, so there is no error propagation, but it does cost some overhead. There are a few other techniques but they are the basics Glen On 14/06/2018 1:20 PM, Steve wrote: I played around a bit with the bit scrambler, first to make sure it was a two-way function, and then by running the sm1000 scrambled result into FreeDV on my x86. Without the scrambler, after I was done talking, the frequency error would waver around +/- and then slide off almost -600 Hz. The SNR would then drop down and become un-synced. Then when I started talking again it would correct itself, and the SNR would go back to max and sync. With the scrambler on however, the frequency error toggled around +/- 7.5 Hz off. The SNR never dropped down. FYI Steve -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2 -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Modulation Woes on SM1000
I never gave much thought to the raw vocoder bits, so it was an interesting development. On Wed, Jun 13, 2018 at 10:27 PM Bruce Perens wrote: > > Probably anything that avoids long streams one symbol is going to help here. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Modulation Woes on SM1000
Probably anything that avoids long streams one symbol is going to help here. On Wed, Jun 13, 2018 at 8:20 PM, Steve wrote: > I played around a bit with the bit scrambler, first to make sure it > was a two-way function, and then by running the sm1000 scrambled > result into FreeDV on my x86. > > Without the scrambler, after I was done talking, the frequency error > would waver around +/- and then slide off almost -600 Hz. The SNR > would then drop down and become un-synced. Then when I started talking > again it would correct itself, and the SNR would go back to max and > sync. > > With the scrambler on however, the frequency error toggled around +/- > 7.5 Hz off. The SNR never dropped down. > > FYI > > Steve > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > Freetel-codec2 mailing list > Freetel-codec2@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/freetel-codec2 > -- Bruce Perens K6BP - CEO, Legal Engineering Standards committee chair, license review committee member, co-founder, Open Source Initiative President, Open Research Institute; Board Member, Fashion Freedom Initiative. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Modulation Woes on SM1000
I played around a bit with the bit scrambler, first to make sure it was a two-way function, and then by running the sm1000 scrambled result into FreeDV on my x86. Without the scrambler, after I was done talking, the frequency error would waver around +/- and then slide off almost -600 Hz. The SNR would then drop down and become un-synced. Then when I started talking again it would correct itself, and the SNR would go back to max and sync. With the scrambler on however, the frequency error toggled around +/- 7.5 Hz off. The SNR never dropped down. FYI Steve -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] SM1000 / STM32 CMake build system ready for testing!
I've tried Richards new stm32 cmake system and it's very nice - thank you so much Richard! I haven't had a chance to try the SM1000.bin image yet, would be great if some one here could please try that with an image built using the new stm32 cmake system. Re MSVC generator, a developer from the Winmor/Ardop project had problems building the ofdm modem due to lack of support of MSVC of the complex type. Not sure if there is much we can do about that, but it would be nice to see if this code can be built using Microsoft tools, as quite a few people like those tools. A friend of mine is working on a FreeDV Iphone app, and mentioned cmake -g produced files suitable for building the FreeDV API on that platform - that's quite impressive. Cheers, David 14/06/18 08:14, Richard Shaw wrote: For those of you that would like to help with the SM1000 / STM32 development I have implemented a CMake build system that will replace the custom Makefile. To setup a build system read both the README.txt (just as to which packages that need to be installed) and README.cmake for CMake specific instructions. See what you can break and give David and I feedback so we can improve the instructions. Also, it's theoretically possible to build on MSVC now using the MSVC generator from CMake but it probably needs some tweaking. As a reminder, the SM1000 / STM32 stuff is in the stm32 subdirectory of codec2-dev. Thanks, Richard KF5OIM -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2 -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
[Freetel-codec2] SM1000 / STM32 CMake build system ready for testing!
For those of you that would like to help with the SM1000 / STM32 development I have implemented a CMake build system that will replace the custom Makefile. To setup a build system read both the README.txt (just as to which packages that need to be installed) and README.cmake for CMake specific instructions. See what you can break and give David and I feedback so we can improve the instructions. Also, it's theoretically possible to build on MSVC now using the MSVC generator from CMake but it probably needs some tweaking. As a reminder, the SM1000 / STM32 stuff is in the stm32 subdirectory of codec2-dev. Thanks, Richard KF5OIM -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
[Freetel-codec2] freedv_tx with an external VCO
I have modified freedv_tx to output VCO frequencies for the 2400A and 800XA modes: Example 1: ./freedv_tx 2400A ../../raw/hts1a.raw - --extvco | hexdump -v -e '/2 "%04d\n"' 3600 3600 1200 3600 1200 1200 1200 4800 Example 2: ./freedv_tx 800XA ../../raw/hts1a.raw - --extvco | hexdump -v -e '/2 "%04d\n"' 1200 1600 1200 2000 1200 2000 1200 0800 The output is 16 bit shorts with the frequency in Hz of each tone. The output sample rate is the symbol rate, so 1200 symbols/s for 2400A, and 400 symbols/sec for 800XA. The sample rate needs to be throttled by the "data sink" DAC. The use case is transmitting the FreeDV FSK modes with external FM transmitters, for example off the shelf VHF/UHF radio with "data port" input. The DAC could be an Arduino or similar. I've messed around a bit sending 2400A from my FT817, I used a stm32 Discovery board as the DAC. - David -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Modulation Woes on SM1000
I used a bit version of the horus byte scrambler, and got this:
[image: 4fsk-scrambled.png]
/**
* In-Place 15 bit additive scrambler with 0x4a80 Frame Sync
*
* Sync1 0 0 1 0 1 0 1 0 0 0 0 0 0 0
*+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
*| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10| 11| 12| 13| 14| 15|
*+-+-+---+---+---+---+---+---+---+---+---+---+---+---+-+-+-+-+
* ^ | |
* | v v
* | +---+
* +<| + |
* | +---+
* v
*+---+
* in --->| + |---> out
*+---+
*
* inout - input and output data as bits
*
* Returns void.
*/
void scramble(uint8_t inout[]) {
int i, ibit;
uint16_t scrambler = 0x4A80; /* initialize scrambler every frame */
uint16_t scrambler_out;
/* in place modification of each bit */
for (i = 0; i < BITS_PER_FRAME; i++) {
scrambler_out = ((scrambler & 0x2) >> 1) ^ (scrambler & 0x1);
ibit = inout[i] & 0x1;
inout[i] = ibit ^ scrambler_out;
/* update scrambler */
scrambler >>= 1;
scrambler |= (scrambler_out << 14);
}
}
Have fun!
On Wed, Jun 13, 2018 at 4:02 PM Adrian Musceac wrote:
>
> Hi Steve,
> You could try scrambling the output of the vocoder.
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Freetel-codec2 mailing list
Freetel-codec2@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Modulation Woes on SM1000
Thanks Adrian, It wouldn't be compatible, but I am interested in that idea. I'm going to whip one up and see what it does. Thanks On Wed, Jun 13, 2018 at 4:02 PM Adrian Musceac wrote: > > Hi Steve, > You could try scrambling the output of the vocoder. I use a scrambler > block from GNU radio which is described quite well here: > http://destevez.net/2016/05/scramblers-and-their-implementation-in-gnuradio/ > Of course you would need to write it yourself and add it at the vocoder > output. > > Regards, > Adrian -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Modulation Woes on SM1000
> > By the way those views are via the freedv spectrum display. > > What I was thinking was the random numbers are of course hitting all > the tones equally, whereas the vocoder bits are not as pure, and may > in fact favor bit combinations over another. Maybe a sequence of the > same bit patterns. > Hi Steve, You could try scrambling the output of the vocoder. I use a scrambler block from GNU radio which is described quite well here: http://destevez.net/2016/05/scramblers-and-their-implementation-in-gnuradio/ Of course you would need to write it yourself and add it at the vocoder output. Regards, Adrian -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Modulation Woes on SM1000
Thanks David, I wasn't too concerned, except playing into Freedv the modem seems to lose sync, and the SNR drops away. I'm also not getting a very loud decoded voice, and if I raise the SM1000 gain pot it becomes distorted. I might need to test more on the x86 as you suggest. Thanks! -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Modulation Woes on SM1000
the spectrum looks as I would expect. are your random numbers uniformly distributed ? I dont know how the spectrum display is generated in the freeDV IE if it is just taking slabs of samples with some arbritrary FFT boundary, or it is aligned to the symbol clocks etc. or how many grabs are used to generate the display, or if they are time averaged, and if they are voltage or power averaged or log averaged etc etc looks fine to me- the OUT OF BAND is still clean. that's what you need to look for for this case, rubbish more than a couple of tone steps away from the end tones. Run it through SPECTRAN or SPECTRUM LAB. see how it looks there are various averaging/ analysis options glen On 14/06/2018 6:44 AM, Steve wrote: Thanks Glen, By the way those views are via the freedv spectrum display. What I was thinking was the random numbers are of course hitting all the tones equally, whereas the vocoder bits are not as pure, and may in fact favor bit combinations over another. Maybe a sequence of the same bit patterns. I'm wondering if I need to create a transmit filter before the D/A. It being a lot less sophisticated than a sound card. Steve -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2 -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Modulation Woes on SM1000
Thanks Glen, By the way those views are via the freedv spectrum display. What I was thinking was the random numbers are of course hitting all the tones equally, whereas the vocoder bits are not as pure, and may in fact favor bit combinations over another. Maybe a sequence of the same bit patterns. I'm wondering if I need to create a transmit filter before the D/A. It being a lot less sophisticated than a sound card. Steve -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Modulation Woes on SM1000
Hi Steve, I wouldn't be too concerned, spectrums move in mysterious ways. Couple of ideas: 1/ Try the same test on an x86 2/ Measure the BER of the transmitted signal with calibrated noise, that's the ultimate test of any modem. Cheers, David On 14/06/18 05:57, Steve wrote: In modulating the 4FSK using a random number (rand() % 4) I get this spectrum: 4fsk-rand.png Looks good. Now if I modulate it with the vocoder bits instead: 4fsk-voice.png Well, isn't that special. I have no idea where those harmonics comes from, but there is always a pip at 1400 (center frequency). Plus the garbage on the right. Heck, there are only 4 frequencies possible in the modulator. I guess swinging from say F1 in one instant to F4 in the next, must be a dirty process. More than likely, it's just a well hidden bug. It's a puzzle so far :-) Steve -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2 -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
Re: [Freetel-codec2] Modulation Woes on SM1000
Hi STeve With all modulation schemes the components are never single spectral lines, as of course the tones are modulated and thus the modulation will show depending how the spectrum analysis is captured. You are right it is a dirty process switching from one to another. However it is a controlled process- the data is applied to a low pass filter before heading to the modulator (or should be) . So the dirt is controlled. Various filter schemes are available depending on whether lowest SNR, lowest bandwidth , minimum ISI etc is desired. If the tones were directly switched at the datarate, ie no low pass filter, this is the same as the individual tones being modulated with a square wave. How it will look on spectrum is highly dependent with how the spectral analysis is performed . IE time averaged periodic, sync (IE is the sample and sweep rate synchronous with the data clock and sample clocks of the modulator etc ?) But the dirt can also indicate a modulator bug, for sure, certainly a chance of that cheers On 14/06/2018 6:27 AM, Steve wrote: In modulating the 4FSK using a random number (rand() % 4) I get this spectrum: 4fsk-rand.png Looks good. Now if I modulate it with the vocoder bits instead: 4fsk-voice.png Well, isn't that special. I have no idea where those harmonics comes from, but there is always a pip at 1400 (center frequency). Plus the garbage on the right. Heck, there are only 4 frequencies possible in the modulator. I guess swinging from say F1 in one instant to F4 in the next, must be a dirty process. More than likely, it's just a well hidden bug. It's a puzzle so far :-) Steve -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2 -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
[Freetel-codec2] Modulation Woes on SM1000
In modulating the 4FSK using a random number (rand() % 4) I get this spectrum: [image: 4fsk-rand.png] Looks good. Now if I modulate it with the vocoder bits instead: [image: 4fsk-voice.png] Well, isn't that special. I have no idea where those harmonics comes from, but there is always a pip at 1400 (center frequency). Plus the garbage on the right. Heck, there are only 4 frequencies possible in the modulator. I guess swinging from say F1 in one instant to F4 in the next, must be a dirty process. More than likely, it's just a well hidden bug. It's a puzzle so far :-) Steve -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
