Re: Topband: FT-8 performance
K5ESW said: "The difference in the FT8 reported SNR and how most hams think of SNR seems explained well by Jim, KC5RUO. http://www.arrl.org/forum/topics/view/1957 https://tapr.org/pdf/DCC2018-KC5RUO-TheReal-FT8-JT65-JT9=SNR.pdf Thanks Paul. I had read one of those articles before. I didn't find the more detailed one earlier when I was looking. He is relating the S/N reported by the digital modes to what it really should be. Without digging thru all those numbers and just listening to FT8 it was pretty obvious that the reported S/N didn't have much meaning for the average person. That was very obvious by considering the example I gave previously of a S9+40 dB signal being reported as 1 dB below the noise floor when my receiver was reading the noise floor at S1. That is a huge difference. It just means the reported S/N numbers need to be ignored. The really important feature that needs to be measured is not the reported S/N but the minimum S/N that FT8 will decode. That's when using a real noise number that hams can relate to, like what your receiver reads on narrow bandwidth on a frequency with no signals present. (That would be useful for comparing CW to FT8.) That's usually atmosphere noise for those that don't have local QRN problems. I'm afraid that FT8 doesn't use that real noise number when describing its operation. It uses something else that most people can't relate to. So what does decoding a signal 24 dB below the noise floor mean? Who's definition of noise floor? Certainly not mine. The results of my experiments were that FT8 can decode signals approximately 24 dB below the S meter reading of the whole 1.5 KHs band (signal plus noise included). That conclusion was based totally on measurements, which I repeated many times. It was never exactly the same number every time (but close) but this was over the air testing with QSB not something in a lab setup. You won't be able to read S meters exactly. That's not a difficult measurement to make but it helps a lot if your receiver has S meters for both the main and sub receivers. Set the main receiver on 1.5 KHz bandwidth and the sub receiver on about 100-200 Hz bandwidth and measure the signal strength of the signal being decoded. You have to be sure that you isolate the right signal. From those measurement you can see that the minimum signal decoded is going to be a function of how many stations there are on the band and their strength. I did verify that happening. I also confirmed that when narrowing the bandwidth on the receiver doing the decoding that lower level signals can be decoded. You would think that decoding would continue down below the noise floor if there weren't any strong signals on the band, however that didn't happen with any testing I did. FT8 pooped out at a S/N number of about +15 dB. Still waiting for someone else to make similar measurements. I get the feeling that I am the only person to ever make these measurements. Jerry, K4SAV _ Searchable Archives: http://www.contesting.com/_topband - Topband Reflector
Re: Topband: FT-8 performance
Jerry, The difference in the FT8 reported SNR and how most hams think of SNR seems explained well by Jim, KC5RUO. http://www.arrl.org/forum/topics/view/1957 https://tapr.org/pdf/DCC2018-KC5RUO-TheReal-FT8-JT65-JT9=SNR.pdf He says for FT8, the noise bandwidth that impacts FT8 software´s ability to decode data is 6.250 Hz. WSJT calculates the total power (all signals and noise) in the 2500 Hz bandwith, and uses it as the denominator in SNR calculations. Jim says "it would make more sense to measure the SNR in the bandwidth that's really used by the receiver; but it may be hard to determine or define that "true" receive bandwidth. So in short, your eyes and ears are not deceiving you. Those JT65/JT9/FT8 signals are very much so well above the noise." The following shows if you focus on a single FT8 signal in a 6.250 Hz bandwidth, the WSJT reported SNR needs to have 26 dB added: SNRFSKFT8 = SNRreportedFT8 + (10 x LOG (2500 Hz/6.250 Hz)) SNRFSKFT8 = SNRreportedFT8 + (10 x LOG (400)) SNRFSKFT8 = SNRreportedFT8 + 26 dB 73, Paul K5ESW Raleigh, NC > I'm not sure how FT-8 calculates the reported S/N number. I found very > little information on the subject and what I did find was not easily > understandable. What I did was an experiment in which I was able to > get > close to the same number being reported. According to what I have > read > about FT-8, it does not implement the same method as I was using in my > testing. > > It was very obvious to me that the number being reported was useless. > Example: How would you be able to report a S/N of -1 dB when the > station is S9+40 db on the S meter and the receiver reads S1 when > tuned > to a spot with no stations. (Actual measurement) > > I made a guess that the number being reported was actually a signal to > noise plus signal ratio S1(S1 + N), where N is the sum of everything > else in the passband. The S9+40 db station in the example would be > the > main contributor to the overall level of the total stuff in the > passband > and that total is just a little more than his signal alone, so -1 dB > now > makes sense. This seems to work and it works on other FT-8 signals as > well. > > Jerry, K4SAV > _ > Searchable Archives: http://www.contesting.com/_topband - Topband > Reflector _ Searchable Archives: http://www.contesting.com/_topband - Topband Reflector
Re: Topband: FT-8 performance
Hello Jerry When FT8 came out in 2017, I tried it. Once past the gee whiz factor of the technology, I did not care for it. Being a traditional CW DX chaser on all bands, making FT8 QSO’s did nothing for me. Like kissing your sister. As the activity picked up on FT8, some soul searching was done. To continue building Challenge points, it was either adopt FT8 or quit working new ones, primarily on 6 meters, where most of my Challenge opportunity exists. So, I bit the bullet and moved to FT8, so far only for new ones, and primarily on 6. FT8 has transformed 6 Meters. The activity has nearly all moved to FT8. If you start from scratch today on 6 with CW or SSB only, I don’t know if you could make DXCC in a lifetime. Will Topband become like this? I hope not, but don’t know. The FT8 mode is very effective on 6, QSO’s are in my log which would not be there except for FT8. Lots of these QSO’s. Once can argue whether these are QSO’s in the same sense as a CW QSO, but the ARRL says they are, and they are the sponsors/keeper of the awards. I don’t personally like FT8, but it is a useful tool for making Challenge points. The needed slots are not available on conventional modes, at least not on 6 meters. I have spent the entire 6M E skip season in 2018 and 2019 on 6M FT8, and have tried to find meaning in the dB report. Strictly by observation, I can’t correlate the SNR report with an S Meter reading at all. If my noise level is low, there are several layers of signal audible below S-1 on my IC7610. It is very common to see a +10 to +15 SNR reading on a sub S-1 signal that does not even move the S meter at all. I sometimes get lower SNR reports from locals, who do move the S meter to S9 or more. It seems to have more to do with my baseline powerline noise level, and the number of signals in the passband. What I do now is pay no attention at all to the SNR report. None. It certainly seems to have no bearing on whether I could make a CW QSO or not with the signal. Actually, what myself and other experienced CW ops have noticed is we frequently can hear very weak signals on FT8 that do not decode at all. These would be workable on CW for sure. However, if the QSB during a transmit cycle drops the signal below the detection threshold for 3 or 4 seconds during a TX cycle, it is not going to decode. Those are the types of signals which might be workable by good CW ops. That said, I can frequently decode and work signals that do not meet my hearing threshold. They just have to hang in for the entire TX cycle and be above the threshold for FT8. This summer, I ran side by side comparisons with JTDX and FT8 on very weak signals. This went on for 3 weeks or so, and I became convinced JTDX had superior weak signal decoding and switched to it. JTDX has multiple decoders and other features which result in more sensitive decodes. These are predictable. On a moderately filled band, WSJTX misses decodes that are -20 or below and JTDX typically gets these. The decodes are pretty accurate; I don’t get very many garbage decodes. JTDX was directly responsible for a QSO with 6W1TA which WSJTX missed decoding. So far, I have worked no new ones on FT8 Topband or 80M, but have listened With a steady S9 plus 20 to 40 dB roar of NA to NA signals, I am a bit skeptical about how effective it will be contrasted to 6. However, this year my plan is to pay attention to the low bands on FT8 and see how it works. I will use JTDX for my testing. Again, my only interest in FT8 is to work new ones. I could care less about making hundreds of routine QSO’s on the mode. Lots of folks do enjoy it, witness the huge amount of activity. FT8 has been around since mid 2017. Imagine all the Challenge points, Digital and Mixed DXCC credits that have been awarded. I don’t see the ARRL backtracking on their initial approach to FT8/FT4 etc. For myself, I had an attitude adjustment and moved on. 73 Charlie N8RR Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10 From: Topband on behalf of K4SAV Sent: Thursday, August 1, 2019 5:09:55 PM To: topband@contesting.com Subject: Re: Topband: FT-8 performance I'm not sure how FT-8 calculates the reported S/N number. I found very little information on the subject and what I did find was not easily understandable. What I did was an experiment in which I was able to get close to the same number being reported. According to what I have read about FT-8, it does not implement the same method as I was using in my testing. It was very obvious to me that the number being reported was useless. .Example: How would you be able to report a S/N of -1 dB when the station is S9+40 db on the S meter and the receiver reads S1 when tuned to a spot with no stations. (Actual measurement) I made a guess that the number being re
Re: Topband: FT-8 performance
On 8/1/2019 1:27 PM, James Wolf wrote: I think there is some misunderstanding of bandwidth using FT-8. The power limited Shannon limit I posted about today is independent of equivalent noise bandwidth The equivalent noise bandwidth of an FT8 detected tone is only 6.25 Hz. So -26 dB S/N ratio in a 2500 Hz bandwidth is 0 dB in a 6.25 Hz bandwidth. Thus using the conventional version* of the Shannon limit, the maximum theoretical bit rate is 6.25 bits/second. FT8 supposedly can do about this rate at -24 dB S/N in 2500 Hz. IOW, it is within 2 dB of the Shannon limit. This is all self consistent. *At a SNR of 0 dB (Signal power = Noise power) the Capacity in bits/s is equal to the bandwidth in hertz, according to Wikipedia. Rick N6RK Jim Wolf, KR9U _ Searchable Archives: http://www.contesting.com/_topband - Topband Reflector
Re: Topband: FT-8 performance
I'm not sure how FT-8 calculates the reported S/N number. I found very little information on the subject and what I did find was not easily understandable. What I did was an experiment in which I was able to get close to the same number being reported. According to what I have read about FT-8, it does not implement the same method as I was using in my testing. It was very obvious to me that the number being reported was useless. .Example: How would you be able to report a S/N of -1 dB when the station is S9+40 db on the S meter and the receiver reads S1 when tuned to a spot with no stations. (Actual measurement) I made a guess that the number being reported was actually a signal to noise plus signal ratio S1(S1 + N), where N is the sum of everything else in the passband. The S9+40 db station in the example would be the main contributor to the overall level of the total stuff in the passband and that total is just a little more than his signal alone, so -1 dB now makes sense. This seems to work and it works on other FT-8 signals as well. Jerry, K4SAV _ Searchable Archives: http://www.contesting.com/_topband - Topband Reflector
Re: Topband: FT-8 performance
If anybody's interested in trying my version of the python FT8 decoder with a more sensible SNR implementation just go here: https://github.com/mcogoni/weakmon I modified the code to look for the lowest power bin within the input bandwidth and each individual signal is compared to this. To obtain better SNR values avoid multiple passes for decoding since it implies subtracting the strongest signals, bin by bin, from the waterfall so the relative powers get all modified for the "hidden" signals. marco / IS0KYB Il giorno gio 1 ago 2019 alle ore 21:46 Richard (Rick) Karlquist < rich...@karlquist.com> ha scritto: > It is instructive to calculate the Shannon > maximum theoretical data rate (power limited case) > (refer to wikipedia page for Shannon-Hartley theorem). > > If S/N ratio (BW=2,500 Hz) = -24 dB, > then S/N ratio (BW=1Hz) = -24 + 10 log 2,500 = -24 +34 > = +10 dB. 10 dB converted to a dimensionless ratio is 10. > > Now, channel capacity = C <= 1.44 X 10 = 14.4 bits/second. > > This rate is a little more than twice the FT-8 rate. > > Now a days, achieving 1/2 of the Shannon limit is > possible for AWGN. If your noise isn't AWGN, well > then that is another source of error. > > Thus the claim of -24 dB sensitivity seems plausible, > where the S/N is the true signal vs AWGN, as opposed > to whatever random number FT-8 reports. > > So I think the beef with FT-8 is in the way it > calculates the displayed S/N. We used to call > those "marketing specs". > > It is also notable that FT-8 uses at least twice > the average power compared to CW. If you compared > them on an average power basis (vs PEP) the FT-8 > advantage, if any, would drop 3 dB. > > On CW, you could send your call many times > in 15 seconds for "error correction" and take advantage > of QSB peaks. That tends to level the playing field. > More playing field leveling is using Super Check Partial > analogous to what FT-8 does. > > Rick N6RK > _ > Searchable Archives: http://www.contesting.com/_topband - Topband > Reflector > -- marco cogoni CRS4 http://sibamanna DOT duckdns DOT org _ Searchable Archives: http://www.contesting.com/_topband - Topband Reflector
Re: Topband: FT-8 performance
It is instructive to calculate the Shannon maximum theoretical data rate (power limited case) (refer to wikipedia page for Shannon-Hartley theorem). If S/N ratio (BW=2,500 Hz) = -24 dB, then S/N ratio (BW=1Hz) = -24 + 10 log 2,500 = -24 +34 = +10 dB. 10 dB converted to a dimensionless ratio is 10. Now, channel capacity = C <= 1.44 X 10 = 14.4 bits/second. This rate is a little more than twice the FT-8 rate. Now a days, achieving 1/2 of the Shannon limit is possible for AWGN. If your noise isn't AWGN, well then that is another source of error. Thus the claim of -24 dB sensitivity seems plausible, where the S/N is the true signal vs AWGN, as opposed to whatever random number FT-8 reports. So I think the beef with FT-8 is in the way it calculates the displayed S/N. We used to call those "marketing specs". It is also notable that FT-8 uses at least twice the average power compared to CW. If you compared them on an average power basis (vs PEP) the FT-8 advantage, if any, would drop 3 dB. On CW, you could send your call many times in 15 seconds for "error correction" and take advantage of QSB peaks. That tends to level the playing field. More playing field leveling is using Super Check Partial analogous to what FT-8 does. Rick N6RK _ Searchable Archives: http://www.contesting.com/_topband - Topband Reflector
Re: Topband: FT-8 performance
Do either JT9 or FT8 *really* need a wide SSB filter? What happens if we use a good narrow CW filter instead? 73, Mike www.w0btu.com On Thu, Aug 1, 2019, 2:03 PM K4SAV wrote: > W0BTU directed a question to me about JT-9. I have never tested JT-9 so > I don't have any information to supply on that subject. > > The S/N number supplied by FT-8 was only a curiosity to me because I > could see a huge disparity between what was being reported versus what I > was observing on my receiver. The reported S/N seems to be about > useless, but the really important question was what is the lowest signal > to noise FT-8 can decode,(real S/N, not the reported number). That was > the main question I wanted to answer with testing. > > A little off subject but people also need to stop using S/N numbers as a > measure of antenna gain. > > Jerry, K4SAV > > _ Searchable Archives: http://www.contesting.com/_topband - Topband Reflector
Re: Topband: FT-8 performance
W0BTU directed a question to me about JT-9. I have never tested JT-9 so I don't have any information to supply on that subject. The S/N number supplied by FT-8 was only a curiosity to me because I could see a huge disparity between what was being reported versus what I was observing on my receiver. The reported S/N seems to be about useless, but the really important question was what is the lowest signal to noise FT-8 can decode,(real S/N, not the reported number). That was the main question I wanted to answer with testing. A little off subject but people also need to stop using S/N numbers as a measure of antenna gain. Jerry, K4SAV _ Searchable Archives: http://www.contesting.com/_topband - Topband Reflector
Re: Topband: FT-8 performance
You might want to listen to this TAPR presentation on Noise and Noise calculations. https://youtu.be/xXXj1Ko4ZXg I found it pretty interesting. Mike va3mw On Thu, Aug 1, 2019 at 2:12 PM uy0zg wrote: > Is the main problem here in efficiency? > > The main thing here - the signal is not heard by a person! > --- > Nick, UY0ZG > http://www.topband.in.ua > > Tim Shoppa 2019-08-01 20:40: > > > > The "work signals way down in the noise you could never work otherwise" > > myth, is just part of the myth that FT8 is an effective operating mode, > > when really it's the least efficient mode for a good op to work Q's or > > DXCC's. See my detailed 2018 statistics by mode here: > > http://n3qe.org/n3qe2018.jpg > > > > Tim N3QE > > > _ > Searchable Archives: http://www.contesting.com/_topband - Topband > Reflector > _ Searchable Archives: http://www.contesting.com/_topband - Topband Reflector
Re: Topband: FT-8 performance
Is the main problem here in efficiency? The main thing here - the signal is not heard by a person! --- Nick, UY0ZG http://www.topband.in.ua Tim Shoppa 2019-08-01 20:40: The "work signals way down in the noise you could never work otherwise" myth, is just part of the myth that FT8 is an effective operating mode, when really it's the least efficient mode for a good op to work Q's or DXCC's. See my detailed 2018 statistics by mode here: http://n3qe.org/n3qe2018.jpg Tim N3QE _ Searchable Archives: http://www.contesting.com/_topband - Topband Reflector
Re: Topband: FT-8 performance
We went thorugh a similar discussion here a year ago about the "cooked" S/N statistics. Or at least they are cooked in a way that no CW operator would cook them, by considering a bandwidth 50 times wider than the FT8 signal. On a quiet WARC FT8 band (no interfering carriers) signals that are -18dB according to FT8 S/N would have been easily copied by any CW operator. And signals that are -22dB or lower probably could've been copied maybe with a few repeats. I know I often have a waterfall spectrum display running, and I work CW signals all the time that I don't see at all on the waterfall. The "work signals way down in the noise you could never work otherwise" myth, is just part of the myth that FT8 is an effective operating mode, when really it's the least efficient mode for a good op to work Q's or DXCC's. See my detailed 2018 statistics by mode here: http://n3qe.org/n3qe2018.jpg Tim N3QE On Thu, Aug 1, 2019 at 9:17 AM K4SAV wrote: > I get the feeling that I must be the only person that has ever tested > FT-8 to the extreme to see what it can do. It seems that everyone else > just assumes it will do what the published information says. It will > not. Below is a summary of my testing. > > First I did a bunch of testing to see if I could figure out how the > software determined the S/N number. I measured the strength of a > station calling CQ and subtracted that from the S meter reading for the > 2500 Hz USB band and that resulted in a number very close to what FT8 > reports. I repeated the test multiple times. > > Official documentation says FT8 can decode signals 24 dB below the noise > floor. That's an interesting comment because FT8 has no way of > determining the level of the noise floor. Even during the off period > when no one is transmitting it can't determine the noise floor because > the noise floor is not represented in the audio signal. The receiver > ACG brings the noise back up when no one is transmitting. Actually for > FT8 the amount of audio is greater in the off period than it is when > stations are transmitting. The only way to measure the noise floor is > by making the measurement in the RF world, which FT8 can't do. > > Actually what the program does is count everything in the 2500 Hz > bandwidth as noise. (That is comprised mainly of strong signals, not > noise.) It is limited to decoding signals 24 dB below that level. From > that info you can guess that if you narrow the bandwidth to eliminate > most of the strong signals FT8 will decode weaker signals. Yes that > works. Verified it myself and others have also found this to be true. > You guys that like to operate FT-8 should take note of that comment. > > I thought maybe it could decode stuff below the noise floor if all the > signals were also below the noise floor, so I tested for that. I found > out that the program poops out long before the noise floor is reached. > With my tests where I decreased the gain of signals from the NE (just > after sunset in the NE) by pointing my antenna west and increasing the > gain such that the biggest signals were 15 dB above the noise floor, FT8 > just about quits. There were probably 50 or more stations on the band > and it was making one decode about every 5 or 10 minutes. > > FT8 has a deep search mode where it uses stored data to make guesses at > call signs some times. I tested that too. Before I started WSJTX I set > up a poor signal to noise ratio test. Started WSJTX in the normal mode > and it did very poorly. Then I turned on deep search, increased the S/N > and let the program look at the band for a little while. Then I went > back to the poor S/N condition without turning FT-8 off and turned on > deep search and it made a lot more decodes. Nearly all of those decodes > were reported at -24 dB. I think those were guesses and it just assigns > -24 dB for guesses. > > I was fooled by the results of a test I did when adding enough noise to > the audio to cover up the signals and FT8 continued to decode the > signals. However I had previously had deep search on and it had already > memorized the band when I did that. It was just guessing that the same > station is on the same frequency as previously. > > I keep hearing reports from people that claim they are getting decodes > without hearing anything in the audio. I set up conditions where that > should have happened, but it never decoded anything. > > In summary. it appears that on an almost dead band, CW (with narrow > bandwidth) has about a 15 dB advantage at decoding weak signals. On a > very crowded band if FT-8 is using a 2500 Hz bandwidth, CW has a huge > (many dBs) advantage over FT-8 because FT-8 can only decode about 24 dB > below whatever your S meter is reading. At 2500 Hz bandwidth in the > FT-8 band on 160 my radio usually hangs at 20 to 30 dB over S9. > > I was using a TS-990S receiver with no audio processing or noise > limiting or blanking. > > If anyone else has run similar tests. I would
Re: Topband: FT-8 performance
Good morning Do we know if the SNR is calculated over the RF passband filter width, or is is calculated over AF filter bandwidth in the WSJTx engine, which it knows? There is a big difference. Mike va3mw On Thu, Aug 1, 2019 at 11:09 AM Marco Cogoni wrote: > Hi, > > I agree with Jerry. I spent a few weeks trying to use FT8 to obtain > antenna radiation patterns and I discovered how the SNR is computed: > it's totally flawed. Basically WSJTX computes the number in two steps: > the first one estimates how strong the adjacent frequency bins are with > respect to the bin of interest. Then this value is corrected by > estimating the baseline noise in a 2.5kHz bandwidth. > > The bad is that in this way the program is trying to unify a QRM concept > (how strong adjacent noise is) with band noise (due to QRN or whatever > wide band there is). > > I found out the problem because using WSPR SNR estimations lead to > antenna patterns that are very similar to NEC simulated antennas but FT8 > data produced WORSE patterns for the BEST antennas. This is due to the > fact that best antennas receive far more signals so WSJTX gives a worse > SNR. > > I tried to talk with Joe Taylor about this, but he said that this method > is what he thinks it is best for a general use, if I have a better one, > just go and modify the code yourself. > > I have a feeling that this way of computing the SNR was chosen because > it makes you think, as Jerry pointed out, that FT8 can magically decode > signals that are absolutely not hearable. That's highly dubious. > > What FT8 does achieve is an extreme overlapping of signals over 2.5kHz, > but at the expense of not really being able to tell the SNR... > > If you want to dig deeper in the FT8 implementation have a look at AB1HL > Robert Morris' FT8 Python implementation that is heavily commented and > very very educational: https://github.com/rtmrtmrtmrtm/basicft8 > > He also wrote a full code/decode high performance software (not so easy > to understand...): https://github.com/rtmrtmrtmrtm/weakmon > > Hope it helps! > > > 73, > > marco / IS0KYB > > > On 01/08/19 15:17, K4SAV wrote: > > I get the feeling that I must be the only person that has ever tested > > FT-8 to the extreme to see what it can do. It seems that everyone else > > just assumes it will do what the published information says. It will > > not. Below is a summary of my testing. > > > > First I did a bunch of testing to see if I could figure out how the > > software determined the S/N number. I measured the strength of a > > station calling CQ and subtracted that from the S meter reading for > > the 2500 Hz USB band and that resulted in a number very close to what > > FT8 reports. I repeated the test multiple times. > > > > Official documentation says FT8 can decode signals 24 dB below the > > noise floor. That's an interesting comment because FT8 has no way of > > determining the level of the noise floor. Even during the off period > > when no one is transmitting it can't determine the noise floor because > > the noise floor is not represented in the audio signal. The receiver > > ACG brings the noise back up when no one is transmitting. Actually > > for FT8 the amount of audio is greater in the off period than it is > > when stations are transmitting. The only way to measure the noise > > floor is by making the measurement in the RF world, which FT8 can't do. > > > > Actually what the program does is count everything in the 2500 Hz > > bandwidth as noise. (That is comprised mainly of strong signals, not > > noise.) It is limited to decoding signals 24 dB below that level. > > From that info you can guess that if you narrow the bandwidth to > > eliminate most of the strong signals FT8 will decode weaker signals. > > Yes that works. Verified it myself and others have also found this to > > be true. You guys that like to operate FT-8 should take note of that > > comment. > > > > I thought maybe it could decode stuff below the noise floor if all the > > signals were also below the noise floor, so I tested for that. I > > found out that the program poops out long before the noise floor is > > reached. With my tests where I decreased the gain of signals from the > > NE (just after sunset in the NE) by pointing my antenna west and > > increasing the gain such that the biggest signals were 15 dB above the > > noise floor, FT8 just about quits. There were probably 50 or more > > stations on the band and it was making one decode about every 5 or 10 > > minutes. > > > > FT8 has a deep search mode where it uses stored data to make guesses > > at call signs some times. I tested that too. Before I started WSJTX > > I set up a poor signal to noise ratio test. Started WSJTX in the > > normal mode and it did very poorly. Then I turned on deep search, > > increased the S/N and let the program look at the band for a little > > while. Then I went back to the poor S/N condition without turning > > FT-8 off and turned on
Re: Topband: FT-8 performance
Very interesting. How does JT9 compare, especially in regards to the noise floor issue that Mark raised? 73, Mike www.w0btu.com _ Searchable Archives: http://www.contesting.com/_topband - Topband Reflector
Re: Topband: FT-8 performance
I meant Jerry, not Mark. Sorry. On Thu, Aug 1, 2019, 10:15 AM Mike Waters wrote: > Very interesting. How does JT9 compare, especially in regards to the noise > floor issue that Mark raised? > > 73, Mike > www.w0btu.com > _ Searchable Archives: http://www.contesting.com/_topband - Topband Reflector
Re: Topband: FT-8 performance
Hi, I agree with Jerry. I spent a few weeks trying to use FT8 to obtain antenna radiation patterns and I discovered how the SNR is computed: it's totally flawed. Basically WSJTX computes the number in two steps: the first one estimates how strong the adjacent frequency bins are with respect to the bin of interest. Then this value is corrected by estimating the baseline noise in a 2.5kHz bandwidth. The bad is that in this way the program is trying to unify a QRM concept (how strong adjacent noise is) with band noise (due to QRN or whatever wide band there is). I found out the problem because using WSPR SNR estimations lead to antenna patterns that are very similar to NEC simulated antennas but FT8 data produced WORSE patterns for the BEST antennas. This is due to the fact that best antennas receive far more signals so WSJTX gives a worse SNR. I tried to talk with Joe Taylor about this, but he said that this method is what he thinks it is best for a general use, if I have a better one, just go and modify the code yourself. I have a feeling that this way of computing the SNR was chosen because it makes you think, as Jerry pointed out, that FT8 can magically decode signals that are absolutely not hearable. That's highly dubious. What FT8 does achieve is an extreme overlapping of signals over 2.5kHz, but at the expense of not really being able to tell the SNR... If you want to dig deeper in the FT8 implementation have a look at AB1HL Robert Morris' FT8 Python implementation that is heavily commented and very very educational: https://github.com/rtmrtmrtmrtm/basicft8 He also wrote a full code/decode high performance software (not so easy to understand...): https://github.com/rtmrtmrtmrtm/weakmon Hope it helps! 73, marco / IS0KYB On 01/08/19 15:17, K4SAV wrote: I get the feeling that I must be the only person that has ever tested FT-8 to the extreme to see what it can do. It seems that everyone else just assumes it will do what the published information says. It will not. Below is a summary of my testing. First I did a bunch of testing to see if I could figure out how the software determined the S/N number. I measured the strength of a station calling CQ and subtracted that from the S meter reading for the 2500 Hz USB band and that resulted in a number very close to what FT8 reports. I repeated the test multiple times. Official documentation says FT8 can decode signals 24 dB below the noise floor. That's an interesting comment because FT8 has no way of determining the level of the noise floor. Even during the off period when no one is transmitting it can't determine the noise floor because the noise floor is not represented in the audio signal. The receiver ACG brings the noise back up when no one is transmitting. Actually for FT8 the amount of audio is greater in the off period than it is when stations are transmitting. The only way to measure the noise floor is by making the measurement in the RF world, which FT8 can't do. Actually what the program does is count everything in the 2500 Hz bandwidth as noise. (That is comprised mainly of strong signals, not noise.) It is limited to decoding signals 24 dB below that level. From that info you can guess that if you narrow the bandwidth to eliminate most of the strong signals FT8 will decode weaker signals. Yes that works. Verified it myself and others have also found this to be true. You guys that like to operate FT-8 should take note of that comment. I thought maybe it could decode stuff below the noise floor if all the signals were also below the noise floor, so I tested for that. I found out that the program poops out long before the noise floor is reached. With my tests where I decreased the gain of signals from the NE (just after sunset in the NE) by pointing my antenna west and increasing the gain such that the biggest signals were 15 dB above the noise floor, FT8 just about quits. There were probably 50 or more stations on the band and it was making one decode about every 5 or 10 minutes. FT8 has a deep search mode where it uses stored data to make guesses at call signs some times. I tested that too. Before I started WSJTX I set up a poor signal to noise ratio test. Started WSJTX in the normal mode and it did very poorly. Then I turned on deep search, increased the S/N and let the program look at the band for a little while. Then I went back to the poor S/N condition without turning FT-8 off and turned on deep search and it made a lot more decodes. Nearly all of those decodes were reported at -24 dB. I think those were guesses and it just assigns -24 dB for guesses. I was fooled by the results of a test I did when adding enough noise to the audio to cover up the signals and FT8 continued to decode the signals. However I had previously had deep search on and it had already memorized the band when I did that. It was just guessing that the same
Topband: FT-8 performance
I get the feeling that I must be the only person that has ever tested FT-8 to the extreme to see what it can do. It seems that everyone else just assumes it will do what the published information says. It will not. Below is a summary of my testing. First I did a bunch of testing to see if I could figure out how the software determined the S/N number. I measured the strength of a station calling CQ and subtracted that from the S meter reading for the 2500 Hz USB band and that resulted in a number very close to what FT8 reports. I repeated the test multiple times. Official documentation says FT8 can decode signals 24 dB below the noise floor. That's an interesting comment because FT8 has no way of determining the level of the noise floor. Even during the off period when no one is transmitting it can't determine the noise floor because the noise floor is not represented in the audio signal. The receiver ACG brings the noise back up when no one is transmitting. Actually for FT8 the amount of audio is greater in the off period than it is when stations are transmitting. The only way to measure the noise floor is by making the measurement in the RF world, which FT8 can't do. Actually what the program does is count everything in the 2500 Hz bandwidth as noise. (That is comprised mainly of strong signals, not noise.) It is limited to decoding signals 24 dB below that level. From that info you can guess that if you narrow the bandwidth to eliminate most of the strong signals FT8 will decode weaker signals. Yes that works. Verified it myself and others have also found this to be true. You guys that like to operate FT-8 should take note of that comment. I thought maybe it could decode stuff below the noise floor if all the signals were also below the noise floor, so I tested for that. I found out that the program poops out long before the noise floor is reached. With my tests where I decreased the gain of signals from the NE (just after sunset in the NE) by pointing my antenna west and increasing the gain such that the biggest signals were 15 dB above the noise floor, FT8 just about quits. There were probably 50 or more stations on the band and it was making one decode about every 5 or 10 minutes. FT8 has a deep search mode where it uses stored data to make guesses at call signs some times. I tested that too. Before I started WSJTX I set up a poor signal to noise ratio test. Started WSJTX in the normal mode and it did very poorly. Then I turned on deep search, increased the S/N and let the program look at the band for a little while. Then I went back to the poor S/N condition without turning FT-8 off and turned on deep search and it made a lot more decodes. Nearly all of those decodes were reported at -24 dB. I think those were guesses and it just assigns -24 dB for guesses. I was fooled by the results of a test I did when adding enough noise to the audio to cover up the signals and FT8 continued to decode the signals. However I had previously had deep search on and it had already memorized the band when I did that. It was just guessing that the same station is on the same frequency as previously. I keep hearing reports from people that claim they are getting decodes without hearing anything in the audio. I set up conditions where that should have happened, but it never decoded anything. In summary. it appears that on an almost dead band, CW (with narrow bandwidth) has about a 15 dB advantage at decoding weak signals. On a very crowded band if FT-8 is using a 2500 Hz bandwidth, CW has a huge (many dBs) advantage over FT-8 because FT-8 can only decode about 24 dB below whatever your S meter is reading. At 2500 Hz bandwidth in the FT-8 band on 160 my radio usually hangs at 20 to 30 dB over S9. I was using a TS-990S receiver with no audio processing or noise limiting or blanking. If anyone else has run similar tests. I would love to hear about it. Jerry, K4SAV _ Searchable Archives: http://www.contesting.com/_topband - Topband Reflector
