Hi Joe, Thanks. I saw your last data run report and the options used, that's what I needed.
I would think, using consistent parameters between testers ( or at least for those of us that don't really know what is going on ), would be important in order to help achieve consistent results . That, coupled with the same data set "should" produce repeatable results. Unless of course, something in the users HW/SW setup is skewing / affecting the results. 73's Greg, KI7MT On 7/31/2015 11:01 AM, Joe Taylor wrote: > Hi Greg, > > I did my tests using batch files. For example, the ones for the tests I > reported an hour or so ago, based on the 15 files that had produced a > false decode, look something like this: > > ######################### > rm ALL_WSPR.TXT > wsprd 150426_0114.wav > wsprd 150426_0148.wav > wsprd 150426_0342.wav > wsprd 150426_0512.wav > wsprd 150426_0526.wav > wsprd 150426_0530.wav > wsprd 150426_0534.wav > wsprd 150426_0540.wav > wsprd 150426_0614.wav > wsprd 150426_0616.wav > wsprd 150426_0630.wav > wsprd 150426_0654.wav > wsprd 150426_0704.wav > wsprd 150426_0746.wav > wsprd 150426_0856.wav > wsprd 150426_1400.wav > mv ALL_WSPR.TXT all.1 > mv wspr_timer.out timer.1 > ######################### > > -- Joe > > On 7/31/2015 11:38 AM, KI7MT wrote: >> Hi Steve, Joe, >> >> Can you post the invocations and options you used for the three cases below? >> >> Case 1. wsprd >> Case 2. wsprd_exp Fano >> Case 3. wsprd_exp Jelink >> >> I realize the source file<-Input locations will be different. >> >> I want to play with this a bit and try to come up with a way to use >> GnuPlot to display results with various input parameters. >> >> Thanks >> >> 73's >> Greg, KI7MT >> >> On 07/30/2015 09:25 PM, Steven Franke wrote: >>> Joe, >>> Here are my results for your data set. I ran 3 cases. The execution times >>> are the average of two runs. >>> >>> Cases >>> 1. wsprd >>> 2. wsprd_exp (Fano, 10000 cycles) >>> 3. wsprd_exp (Jelinek, 5000 cycles) >>> >>> Results >>> 1. 2657 (2) decodes in 359s >>> 2. 2760 (13) decodes in 359s >>> 3. 2749 (3) decodes in 346s >>> >>> The interesting part is the number in parentheses. This time, I paid >>> attention to the number of obviously bad decodes. It’s not easy to find the >>> bad decodes that show up as type 1 callsigns - but it is easy to find and >>> count the ones that show up as type 2 or 3 callsigns with a forward slash >>> “/“. The number in parentheses is the number of bad decodes with a slash in >>> the callsign. It needs to be said that we see only the bad decodes that >>> aren’t trapped by a sanity check in the unpacking routines. >>> >>> There is something funny going on with the Fano decoder in case 2. Here is >>> the result of doing a grep for “/“ in the ALL_WSPR results from the three >>> cases: >>> >>> Case 1. wsprd >>> $ grep / Results_wsprd >>> 0342 -28 0.5 0.001523 0 PH6/OK1SCE 10 >>> 0630 -14 -0.8 0.001518 0 M0N/BX6IJG 30 >>> >>> Case 2. wsprd_exp Fano >>> $ grep / Results_Fano.txt >>> 0114 -16 -0.3 0.001524 0 88Y/9E3XMR 33 >>> 0148 -22 -0.9 0.001523 0 C4S/U23 27 >>> 0512 -12 -1.4 0.001544 -1 EYJ/BD3OWF 43 >>> 0526 -5 -1.1 0.001515 -1 J28/JH9VOA 10 >>> 0530 -10 -1.3 0.001527 -1 XIR/L12IRI 57 >>> 0534 -21 0.1 0.001451 0 5EY/588TIB 53 >>> 0540 -9 -1.3 0.001498 -1 286/CI7RCI 13 >>> 0614 -8 -1.2 0.001523 -1 W64/CZ9IYO 13 >>> 0616 -31 -1.0 0.001491 0 M2Q/ZG4VPX 13 >>> 0704 -10 -1.3 0.001550 -1 KN4OHP/44 53 >>> 0746 -8 -1.4 0.001525 -1 ATD/012KCR 27 >>> 0856 -19 -0.8 0.001523 0 I02/VK3PNP 20 >>> 1400 -17 -0.5 0.001459 0 P2INE/2 53 >>> >>> Case 3. wsprd_exp Jelinek >>> $ grep / Results_Jelinek5000.txt >>> 0114 -16 -0.3 0.001524 0 88Y/9E3XMR 33 >>> 0616 -31 -1.0 0.001491 0 M2Q/ZG4VPX 13 >>> 0654 -12 -1.3 0.001523 -1 1LY/GH4 40 >>> >>> Note the large number of bad decodes coming out of the Fano decoder in case >>> 2. There is only one bad decode that is common to cases 2 and 3. If you >>> look at the times, it appears that the bad decodes in case 2 are coming in >>> bursts. I have to wonder if this corresponds to special noise conditions, >>> e.g. lightning storm. >>> >>> It’s hard to reconcile the large difference in bad decodes between pairs >>> 1-2 and 2-3. In 1-2 the decoding algorithm is the same and in 2-3 the >>> candidates are the same. Strange, eh? >>> >>> I’ve just gone back and looked at bad decodes using the same >>> “forward-slash” criterion on two groups of my own wav files and in each >>> case I see either 2 or 3 bad decodes out of about 2000 for Fano and >>> Jelinek. There are no big differences between the number of bad decodes in >>> cases 1-3 for my test data. Still strange. >>> >>> Steve k9an >>> >>>> On Jul 30, 2015, at 8:01 AM, Joe Taylor<[email protected]> wrote: >>>> >>>> Hi Greg, >>>> >>>> I have updated Makefile.win32 in ^/branches/wsjtx so that it builds >>>> Steve's new wsprd_exp.exe correctly. >>>> >>>> I have made a tarfile with the set of WSPR *.wav files I used most >>>> recently. It is now posted at >>>> >>>> http://physics.princeton.edu/pulsar/K1JT/wspr_data.tgz >>>> >>>> It's about 1 GB in size. >>>> >>>> -- Joe, K1JT >>>> >>>> On 7/29/2015 11:52 PM, KI7MT wrote: >>>>> Hi Steve, Joe, >>>>> >>>>> I hit another show stopper error also. I'll look at what Joe is using in >>>>> ^/branches/wsjtx_exp and see what the diff's are from the main devel >>>>> branch. >>>>> >>>>> By chance, do you all have a standard set of WSPR .wav files that your >>>>> using to compare with? Would be nice to be able to use a standard set >>>>> for testing. >>>>> >>>>> 73's >>>>> Greg, KI7MT >>>>> >>>>> >>>>> >>>>> On 07/29/2015 07:36 PM, Steven Franke wrote: >>>>>> Greg - >>>>>> The windows Makefile has not been updated for the stack decoder. I’ve >>>>>> only tested it on linux and osx. It looks like the Makefile in Joe’s >>>>>> experimental branch is close to what would be needed - though it >>>>>> shouldn’t need the extended stacksize anymore since we moved the big >>>>>> arrays to heap storage. >>>>>> Steve k9an >>>>>> >>>>>>> On Jul 29, 2015, at 1:22 AM, Greg Beam<[email protected]> wrote: >>>>>>> >>>>>>> Hi Joe, Steve, >>>>>>> >>>>>>> This is off list. >>>>>>> >>>>>>> I tried to build wsprd_exp on Windows (using Qt 5.2.1 Tool-Chain, not >>>>>>> the 5.5 Tool-Chain) and ran into an error. I'm using >>>>>>> ^/branches/wsjtx/lib/wsprd folder, and Makefile.win32, is that the >>>>>>> correct location and Makefile file? >>>>>>> >>>>>>> Here's the error I'm getting: >>>>>>> ----- >>>>>>> wsprd_exp.o:wsprd_exp.c:(.text.startup+0x244c): undefined reference to >>>>>>> `jelinek' >>>>>>> collect2.exe: error: ld returned 1 exit status >>>>>>> Makefile.win32:34: recipe for target 'wsprd_exp' failed >>>>>>> mingw32-make: *** [wsprd_exp] Error 1 >>>>>>> ---- >>>>>>> >>>>>>> Any Ideas? >>>>>>> >>>>>>> 73's >>>>>>> Greg, KI7MT >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 7/28/2015 1:49 PM, Steven Franke wrote: >>>>>>>> Hi Greg and Joe, >>>>>>>> >>>>>>>> As Joe said, the stack decoder is only in wsprd_exp and it requires a >>>>>>>> command-line argument (-J) to activate it, as the default algorithm in >>>>>>>> wsprd_exp is the Fano algorithm. >>>>>>>> >>>>>>>> The tests conducted by me and Joe show that my implementation of >>>>>>>> Jelinek’s stack-bucket algorithm doesn’t seem to provide any >>>>>>>> significant advantage over the Fano decoder in the wspr application. I >>>>>>>> am still inclined to replace the current wsprd with the Fano version >>>>>>>> of wsprd_exp. All of my tests indicate that the default configuration >>>>>>>> of wsprd_exp produces more decodes in less time than wsprd does. This >>>>>>>> is due to improvements in the sync algorithm and not due to anything >>>>>>>> related to the sequential decoder. However --- when Joe compared wsprd >>>>>>>> to wsprd_exp (Fano) he didn’t find any significant difference between >>>>>>>> the two. If you decide to do some tests, Greg, it’d be interesting to >>>>>>>> see if you see any difference between wsprd and wsprd_exp (with the >>>>>>>> default settings). >>>>>>>> >>>>>>>> Steve >>>>>>>> >>>>>>>>> On Jul 28, 2015, at 2:22 PM, Joe Taylor<[email protected]> wrote: >>>>>>>>> >>>>>>>>> Hi Greg, >>>>>>>>> >>>>>>>>> The experimental ("Jelinek") decoder is currently being built only as >>>>>>>>> wsprd_exp. >>>>>>>>> -- Joe >>>>>>>>> >>>>>>>>> On 7/28/2015 3:08 PM, Greg Beam wrote: >>>>>>>>>> Hi Joe, Steve, >>>>>>>>>> >>>>>>>>>> Are these updates in the main wsprd binary, or do we still need to >>>>>>>>>> build >>>>>>>>>> the wsprd_exp binary and copy it over to wsprd to test the changes? >>>>>>>>>> >>>>>>>>>> 73's >>>>>>>>>> Greg, KI7MT >>>>>>>>>> >>>>>>>>>> On 7/28/2015 12:22 PM, Joe Taylor wrote: >>>>>>>>>>> Hi Steve, >>>>>>>>>>> >>>>>>>>>>> Nice job with implementation of a sequential decoder using the >>>>>>>>>>> Jelinek >>>>>>>>>>> Stack Algorithm! >>>>>>>>>>> >>>>>>>>>>> I have now run some reasonably thorough tests of it, comparing >>>>>>>>>>> results >>>>>>>>>>> with the default Fano decoder in wsprd. I confirm essentially all of >>>>>>>>>>> your basic conclusions. Jelinek with maxcycles=5000 produces >>>>>>>>>>> nearly the >>>>>>>>>>> same results, in the same execution time, as Fano with >>>>>>>>>>> maxcycles=10000. >>>>>>>>>>> >>>>>>>>>>> In my tests the command-line "-d" option produced about 7-8% more >>>>>>>>>>> decodes, at the cost of roughly 5 x longer execution time. Again, >>>>>>>>>>> this >>>>>>>>>>> was true for both Fano and Jelinek. >>>>>>>>>>> >>>>>>>>>>> Now we know... which is good! >>>>>>>>>>> >>>>>>>>>>> -- Joe, K1JT >>>>>>>>>>> >>>>>>>>>>> ------------------------------------------------------------------------------ >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> wsjt-devel mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/wsjt-devel >>>>>>>>>> >>>>>>>>>> ------------------------------------------------------------------------------ >>>>>>>>>> _______________________________________________ >>>>>>>>>> wsjt-devel mailing list >>>>>>>>>> [email protected] >>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/wsjt-devel >>>>>>>>> ------------------------------------------------------------------------------ >>>>>>>>> _______________________________________________ >>>>>>>>> wsjt-devel mailing list >>>>>>>>> [email protected] >>>>>>>>> https://lists.sourceforge.net/lists/listinfo/wsjt-devel >>>>>>>> >>>>>>>> ------------------------------------------------------------------------------ >>>>>>>> _______________________________________________ >>>>>>>> wsjt-devel mailing list >>>>>>>> [email protected] >>>>>>>> https://lists.sourceforge.net/lists/listinfo/wsjt-devel >>>>>>> >>>>>> >>>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> _______________________________________________ >>>> wsjt-devel mailing list >>>> [email protected] >>>> https://lists.sourceforge.net/lists/listinfo/wsjt-devel >>> >>> >>> ------------------------------------------------------------------------------ >>> _______________________________________________ >>> wsjt-devel mailing list >>> [email protected] >>> https://lists.sourceforge.net/lists/listinfo/wsjt-devel >>> >> > > ------------------------------------------------------------------------------ > _______________________________________________ > wsjt-devel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/wsjt-devel > ------------------------------------------------------------------------------ _______________________________________________ wsjt-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/wsjt-devel
