[wsjt-devel] All.txt file
Jim, I use editpad or notepad++ and cut out every line that doesn't have my call in it. This cuts it down to usable size. Bobby/N4AU -- n...@outlook.com n...@arrl.net ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT time missing
Hi Mike, It was reported very early on. Had to update my Alltext.exe program because of that problem. 73, Sam W2JDB -Original Message- From: Black Michael via wsjt-devel To: WSJT Software Development Cc: Black Michael Sent: Thu, Dec 3, 2020 9:35 am Subject: [wsjt-devel] ALL.TXT time missing RC2 seems to missing date/time from ALL.TXT -- I don't recall seeing this reported. 7.074 Rx FT8 -13 0.4 1160 K4DKW K0KC R+007.074 Rx FT8 -11 0.5 1577 JA6RCH KG5HKC EM137.074 Rx FT8 -18 0.4 1082 W9FFF KC4UCT -167.074 Rx FT8 -16 0.3 1497 VE3CCD KI5IPM EM227.074 Rx FT8 -17 0.4 400 K5VJZ KF4LXS +037.074 Rx FT8 -6 0.5 1252 KI7SKT K5ENG EL29 ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT time missing
Hi Mike, 18.11.2020: Hi Walter, we miss it too ;) I can confirm this is a known regression defect in WSJT-X v2.3.0 RC2, it is fixed for the next release. Thanks for the issue report. 73 Bill G4WJS. 73, Reino OH3mA From: Black Michael via wsjt-devel [mailto:wsjt-devel@lists.sourceforge.net] Sent: 3. joulukuuta 2020 16:36 To: WSJT Software Development Cc: Black Michael Subject: [wsjt-devel] ALL.TXT time missing RC2 seems to missing date/time from ALL.TXT -- I don't recall seeing this reported. 7.074 Rx FT8-13 0.4 1160 K4DKW K0KC R+00 7.074 Rx FT8-11 0.5 1577 JA6RCH KG5HKC EM13 7.074 Rx FT8-18 0.4 1082 W9FFF KC4UCT -16 7.074 Rx FT8-16 0.3 1497 VE3CCD KI5IPM EM22 7.074 Rx FT8-17 0.4 400 K5VJZ KF4LXS +03 7.074 Rx FT8 -6 0.5 1252 KI7SKT K5ENG EL29 ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT time missing
Mike, This has been reported… — John G4KLA smime.p7s Description: S/MIME cryptographic signature ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
[wsjt-devel] ALL.TXT time missing
RC2 seems to missing date/time from ALL.TXT -- I don't recall seeing this reported. 7.074 Rx FT8 -13 0.4 1160 K4DKW K0KC R+007.074 Rx FT8 -11 0.5 1577 JA6RCH KG5HKC EM137.074 Rx FT8 -18 0.4 1082 W9FFF KC4UCT -167.074 Rx FT8 -16 0.3 1497 VE3CCD KI5IPM EM227.074 Rx FT8 -17 0.4 400 K5VJZ KF4LXS +037.074 Rx FT8 -6 0.5 1252 KI7SKT K5ENG EL29 ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] All.txt
TextEdit, a standard part of MacOS, did a fine job on my 50+MB 937492 line ALL.TXT. 73 Bill AE6JV On 6/1/20 at 5:00 PM, bob...@bellsouth.net (Bobby Chandler) wrote: > Editpad Lite does an excellent job. > > Bobby/N4AU --- Bill Frantz|The nice thing about standards| Periwinkle (408)348-7900 |is there are so many to choose| 150 Rivermead Rd #235 www.pwpconsult.com |from. - Andrew Tanenbaum| Peterborough, NH 03458 ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
[wsjt-devel] All.txt
Editpad Lite does an excellent job. Bobby/N4AU -- bob...@bellsouth.net n...@arrl.net ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT and DT
Hi Claude Il 10/07/2019 08:08, Claude Frantz ha scritto: On 7/9/19 10:20 PM, Alessandro Gorobey via wsjt-devel wrote: Hi Alessandro & all, This give me a peek around 0.1 not at 0.2 or more as expected. Cannot understand as the DT is not centered on 0.2 as configured by default. See also sbTxDelay or txDelay_ variables. Some time ago, I have also analysed the dT in the ALL.TXT file available here. I have found that dT is spreading over a rather large range. I'm suspecting the time difference at the TX side. It seems difficult to me to get a valuable result when we are not sure about the clock time offset at the TX side. But I'm aware about such situations of stations having no access to NTP servers available via Internet and having no view to the GPS satellites on the sky. Best wishes, Claude (DJ0OT) ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel See the complete list. 84 -2.6 358 -2.5 484 -2.4 689 -2.3 697 -2.2 745 -2.1 957 -2.0 1089 -1.9 1324 -1.8 1984 -1.7 1670 -1.6 2342 -1.5 2492 -1.4 3211 -1.3 3417 -1.2 4180 -1.1 3940 -1.0 5692 -0.9 5809 -0.8 7728 -0.7 9415 -0.6 10542 -0.5 13313 -0.4 16610 -0.3 31296 -0.2 57826 -0.1 97438 0.0 572250 0.1 212942 0.2 94977 0.3 54855 0.4 40266 0.5 28636 0.6 21884 0.7 19747 0.8 16941 0.9 14093 1.0 12092 1.1 10942 1.2 8675 1.3 7452 1.4 5846 1.5 4878 1.6 4470 1.7 4291 1.8 3345 1.9 3528 2.0 3110 2.1 1988 2.2 1933 2.3 1751 2.4 552 2.5 OK, there is a lot that have poor time sync, but the question whose as the maximum is not centered at 0.2 as the default delay from TX and start of audio Thanks, -- 73 Sandro IW3RAB ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT and DT
On 7/9/19 10:20 PM, Alessandro Gorobey via wsjt-devel wrote: Hi Alessandro & all, This give me a peek around 0.1 not at 0.2 or more as expected. Cannot understand as the DT is not centered on 0.2 as configured by default. See also sbTxDelay or txDelay_ variables. Some time ago, I have also analysed the dT in the ALL.TXT file available here. I have found that dT is spreading over a rather large range. I'm suspecting the time difference at the TX side. It seems difficult to me to get a valuable result when we are not sure about the clock time offset at the TX side. But I'm aware about such situations of stations having no access to NTP servers available via Internet and having no view to the GPS satellites on the sky. Best wishes, Claude (DJ0OT) ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
[wsjt-devel] ALL.TXT and DT
Hi All,
I found several message on DT and ALL.TXT
I try to analyze ALL.TXT of a couple of months:
awk '{ if ($3=="Rx") printf "% .1f\n", $6+0.0}' ALL.TXT | sort -n | uniq
-c > dt.dat
DT can be '0.0' or '-0.0' by fortran round so the trick '$6+0.0'
the graph dt.png is by gnuplot.
This give me a peek around 0.1 not at 0.2 or more as expected.
See only this data:
57826 -0.1
97438 0.0
572250 0.1
212942 0.2
94977 0.3
My clock is in 5 mS range and is in this range all days by ntp as in
example.png
I have no subtract some mS for the travel at light speed from transmitter.
Cannot understand as the DT is not centered on 0.2 as configured by
default. See also sbTxDelay or txDelay_ variables.
--
73
Sandro
IW3RAB
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT (again)
On 7/9/19 12:46 PM, Greg Beam wrote: Hi Greg, Personally, I don't have a need for it either. If all I am after is QSO validation: grep + awk or a good quality text editor is all that's needed :-) Exactly my opinion ! However, if one wants to do any sort of data analysis, the flat file format is less than ideal. Normalizing the data would go a long way toward shrinking the footprint, however, it also adds a level of complexity some may not find very pleasant. It's depending of what we want to analyse. As an example, if we want to analyse the dT, a little parsing and extraction program followed by a statistics program like R are sufficient. Having the data stored in a database, would give us no advantage, as long as the database management system is not coupled itself to such a statistics software. I am aware, and have done a bit of parsing in the past regarding the varying data structures of the file. It's changed a a good bit over the last few versions. Exactly ! I think that a little program in perl is sufficient to extract the needed information. Of course, other languages are possible too. Best wishes, Claude (DJ0OT) ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT (again)
Hi Claude, Personally, I don't have a need for it either. If all I am after is QSO validation: grep + awk or a good quality text editor is all that's needed :-) However, if one wants to do any sort of data analysis, the flat file format is less than ideal. Normalizing the data would go a long way toward shrinking the footprint, however, it also adds a level of complexity some may not find very pleasant. I am aware, and have done a bit of parsing in the past regarding the varying data structures of the file. It's changed a a good bit over the last few versions. In any case, it's a doable thing if one wants to store their history / data (for whatever reason). IoT devices often use these variable-structure data sets with great success. The variance in the ALL.TXT file is minimal compared to some I've seen. 73's Greg, KI7MT On 7/9/19 4:11 AM, Claude Frantz wrote: On 7/9/19 10:58 AM, Greg Beam wrote: Hi Greg, I think parsing the lines into fields is the best long-term solution for storage (allows for much better indexing). This make only sense if you can assign a clear definition to every field. In the case of ALL.TXT, not all lines have the same structure. Especially, the date and the time can be in different lines, depending on the used WSJTX release. However, we'd need to do a fair bit of checking on each line to determine its structure first; This is essential. To be honest, I cannot see the advantage of having the data, stored in ALL.TXT, in a database. Myself, I'm using ALL.TXT only to verify strange situations or to verify a situation where I get a QSO confirmation for a QSO not in the log. A database is very useful when we often need access to the data and when we need rather complex queries. These requirements do not apply to ALL.TXT. Remember, the database management software needs disk space and processing time. Best wishes, Claude (DJ0OT) ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT (again)
On 7/9/19 10:58 AM, Greg Beam wrote: Hi Greg, I think parsing the lines into fields is the best long-term solution for storage (allows for much better indexing). This make only sense if you can assign a clear definition to every field. In the case of ALL.TXT, not all lines have the same structure. Especially, the date and the time can be in different lines, depending on the used WSJTX release. However, we'd need to do a fair bit of checking on each line to determine its structure first; This is essential. To be honest, I cannot see the advantage of having the data, stored in ALL.TXT, in a database. Myself, I'm using ALL.TXT only to verify strange situations or to verify a situation where I get a QSO confirmation for a QSO not in the log. A database is very useful when we often need access to the data and when we need rather complex queries. These requirements do not apply to ALL.TXT. Remember, the database management software needs disk space and processing time. Best wishes, Claude (DJ0OT) ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT (again)
Hi Mike,
I finally had some time to tinker with this some. On my Win laptop using
WSL, MSYS2, or MonaXterm (Cygwin port), import performance is not
impressive by any means. I suspect files system interoperability is the
root cause. Searches, once imported, are fine. On my Linux workstation
imports are extremely fast; note: I'm running MongoDB on an XFS
partition with an abundance of Ram (MongoDB is Ram piggy).
I tinkered around with (3) scripts:
Import Script (atimport.sh)
Link: http://paste.ubuntu.com/p/bh5vGB5GPW/
Call Sign Lookup (atcheck.sh)
Link: http://paste.ubuntu.com/p/NHv9tztx2Z/
Parse line into fields (retest.py)
Link: https://paste.ubuntu.com/p/8HdPcTVXpN/
I think parsing the lines into fields is the best long-term solution for
storage (allows for much better indexing). However, we'd need to do a
fair bit of checking on each line to determine its structure first;
check == "slower imports". Once the main (first) bulk import is done,
incremental updates would be snappy.
I'm sure there are far better solutions; like a UDP service that just
pushes the decodes straight into a database (of any kind) :-)
73's
Greg, KI7MT
On 7/1/19 3:05 PM, Black Michael via wsjt-devel wrote:
Please do post the code.
Thanks Greg.
de Mike W9MDB
On Monday, July 1, 2019, 03:53:15 PM CDT, Greg Beam
wrote:
Hello All,
Here's an example from today's log:
Results: https://paste.ubuntu.com/p/382VVMth4S/
The query takes about (2) seconds or so using a $regex search on
7,390,224 logged events matching two callsigns; this is without being
indexed nor field splitting. It is one string per line imported to a one
field document in MongDB
I can post the script I used as a gist, if anyone is interested:
- Copies the ALL.TXT to $temp_file
- Converts it to CSV
- Generates two helper JS scripts
- Generates one example JS query
- Drops, then re-imports the alltext collection
- Runs the a sample Query.
Note: for incremental (daily) updates, there is no need to drop the
collection (alltext) before inserting new events. I just do that for
performance testing.
It's a simple one-line query command that would work on Win/Linux/Mac:
mongo < example.js
You can, if desired, write any number of commands to perform stats,
lookup(s), whatever, and use the same easy method to query the DB.
However, as this is a single sting entry, much of the analytical value
would be missing, as the fields / categories are not captured.
73's
Greg, KI7MT
On 7/1/19 1:27 AM, Claude Frantz wrote:
> On 7/1/19 7:59 AM, Claude Frantz wrote:
>
> Just as an example of code extract in perl:
>
> if ($line =~ m/^(\d{4})-([A-Z][a-z]{2})-(\d{2})\b/ ) {
> $day = $3 ;
> $month_alpha = $2 ;
> $year = $1 ;
> }
> elsif ($line =~ m/^(\d\d)(\d\d)(\d\d)_\d{6}\b/ ) {
> $day = $3 ;
> $month_num = $2 ;
> $year = 2000 + $1 ;
> }
> elsif ($line =~ m/^(\d{4})-(\d\d)-(\d\d)\b/ ) {
> $day = $3 ;
> $month_num = $2 ;
> $year = $1 ;
> }
>
> I have not tested it, I hope there is no error. This allow to decode the
> 3 formats of ALL.TXT about which ones I remember about. Please note that
> the month can be numeric or alpha. If alpha, you have to convert to
> numeric, if you want to compare to a numeric value. Please note also,
> that the mode switching was an extra line in previous formats.
>
> Best wishes,
> Claude (DJ0OT)
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT (again)
Hi Dan, The example's I spoke of aren't running through a JS engine, they are being ran through Mongo itself to perform the queries. If you want to run native JS server side, you'd be better off install NodeJs or similar. There are a ton of PGP/MongoDB tutorials out there. The main difference being, you'd be using MongoClient through an ODM most likely. There's nothing to say one could not have multiple collections; one with a full string such as the example I did, and another that has each field broken out by field types. Breaking out the fields has a number of advantages, particularly for analytics. I'll tidy up the Bash script I use, and post it as a Gist. You could run it through MSYS2, Cygwin, or WSL even, as all should have access to Windows Local Servers. It would take a few changes for Path variables, but nothing major. I have Apache running on my server as it hosts some of testing environment. I'll may play around with a PHP form or two and see how that goes. 73's Greg, KI7MT On 7/1/19 3:45 PM, Dan Malcolm wrote: Because I want to search via my webserver, I have a separate PHP script that does the search. Probably not as fast as MongoDB. I do get good cohesive reports though. I get a report for example that will show me just one of my QSO's, and will store results in a text file. That makes it useful to refer to later, or to include in an email to my QSO partner. All that said I would like to explore MongoDB. The idea that query via js script may mean that I can still have private web access. I use IIS on Win10 and just the local machine can access it. __ Dan – K4SHQ -Original Message- From: Greg Beam [mailto:ki7m...@gmail.com] Sent: Monday, July 1, 2019 3:47 PM To: wsjt-devel@lists.sourceforge.net Subject: Re: [wsjt-devel] ALL.TXT (again) Hello All, Here's an example from today's log: Results: https://paste.ubuntu.com/p/382VVMth4S/ The query takes about (2) seconds or so using a $regex search on 7,390,224 logged events matching two callsigns; this is without being indexed nor field splitting. It is one string per line imported to a one field document in MongDB I can post the script I used as a gist, if anyone is interested: - Copies the ALL.TXT to $temp_file - Converts it to CSV - Generates two helper JS scripts - Generates one example JS query - Drops, then re-imports the alltext collection - Runs the a sample Query. Note: for incremental (daily) updates, there is no need to drop the collection (alltext) before inserting new events. I just do that for performance testing. It's a simple one-line query command that would work on Win/Linux/Mac: mongo < example.js You can, if desired, write any number of commands to perform stats, lookup(s), whatever, and use the same easy method to query the DB. However, as this is a single sting entry, much of the analytical value would be missing, as the fields / categories are not captured. 73's Greg, KI7MT ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT (again)
Because I want to search via my webserver, I have a separate PHP script that
does the search. Probably not as fast as MongoDB. I do get good cohesive
reports though. I get a report for example that will show me just one of my
QSO's, and will store results in a text file. That makes it useful to refer to
later, or to include in an email to my QSO partner.
All that said I would like to explore MongoDB. The idea that query via js
script may mean that I can still have private web access. I use IIS on Win10
and just the local machine can access it.
__
Dan – K4SHQ
-Original Message-
From: Greg Beam [mailto:ki7m...@gmail.com]
Sent: Monday, July 1, 2019 3:47 PM
To: wsjt-devel@lists.sourceforge.net
Subject: Re: [wsjt-devel] ALL.TXT (again)
Hello All,
Here's an example from today's log:
Results: https://paste.ubuntu.com/p/382VVMth4S/
The query takes about (2) seconds or so using a $regex search on
7,390,224 logged events matching two callsigns; this is without being indexed
nor field splitting. It is one string per line imported to a one field document
in MongDB
I can post the script I used as a gist, if anyone is interested:
- Copies the ALL.TXT to $temp_file
- Converts it to CSV
- Generates two helper JS scripts
- Generates one example JS query
- Drops, then re-imports the alltext collection
- Runs the a sample Query.
Note: for incremental (daily) updates, there is no need to drop the collection
(alltext) before inserting new events. I just do that for performance testing.
It's a simple one-line query command that would work on Win/Linux/Mac:
mongo < example.js
You can, if desired, write any number of commands to perform stats, lookup(s),
whatever, and use the same easy method to query the DB.
However, as this is a single sting entry, much of the analytical value would be
missing, as the fields / categories are not captured.
73's
Greg, KI7MT
On 7/1/19 1:27 AM, Claude Frantz wrote:
> On 7/1/19 7:59 AM, Claude Frantz wrote:
>
> Just as an example of code extract in perl:
>
> if ($line =~ m/^(\d{4})-([A-Z][a-z]{2})-(\d{2})\b/ ) {
> $day = $3 ;
> $month_alpha = $2 ;
> $year = $1 ;
> }
> elsif ($line =~ m/^(\d\d)(\d\d)(\d\d)_\d{6}\b/ ) {
> $day = $3 ;
> $month_num = $2 ;
> $year = 2000 + $1 ;
> }
> elsif ($line =~ m/^(\d{4})-(\d\d)-(\d\d)\b/ ) {
> $day = $3 ;
> $month_num = $2 ;
> $year = $1 ;
> }
>
> I have not tested it, I hope there is no error. This allow to decode
> the
> 3 formats of ALL.TXT about which ones I remember about. Please note
> that the month can be numeric or alpha. If alpha, you have to convert
> to numeric, if you want to compare to a numeric value. Please note
> also, that the mode switching was an extra line in previous formats.
>
> Best wishes,
> Claude (DJ0OT)
>
>
> ___
> wsjt-devel mailing list
> wsjt-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/wsjt-devel
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT (again)
Please do post the code.
Thanks Greg.
de Mike W9MDB
On Monday, July 1, 2019, 03:53:15 PM CDT, Greg Beam
wrote:
Hello All,
Here's an example from today's log:
Results: https://paste.ubuntu.com/p/382VVMth4S/
The query takes about (2) seconds or so using a $regex search on
7,390,224 logged events matching two callsigns; this is without being
indexed nor field splitting. It is one string per line imported to a one
field document in MongDB
I can post the script I used as a gist, if anyone is interested:
- Copies the ALL.TXT to $temp_file
- Converts it to CSV
- Generates two helper JS scripts
- Generates one example JS query
- Drops, then re-imports the alltext collection
- Runs the a sample Query.
Note: for incremental (daily) updates, there is no need to drop the
collection (alltext) before inserting new events. I just do that for
performance testing.
It's a simple one-line query command that would work on Win/Linux/Mac:
mongo < example.js
You can, if desired, write any number of commands to perform stats,
lookup(s), whatever, and use the same easy method to query the DB.
However, as this is a single sting entry, much of the analytical value
would be missing, as the fields / categories are not captured.
73's
Greg, KI7MT
On 7/1/19 1:27 AM, Claude Frantz wrote:
> On 7/1/19 7:59 AM, Claude Frantz wrote:
>
> Just as an example of code extract in perl:
>
> if ($line =~ m/^(\d{4})-([A-Z][a-z]{2})-(\d{2})\b/ ) {
> $day = $3 ;
> $month_alpha = $2 ;
> $year = $1 ;
> }
> elsif ($line =~ m/^(\d\d)(\d\d)(\d\d)_\d{6}\b/ ) {
> $day = $3 ;
> $month_num = $2 ;
> $year = 2000 + $1 ;
> }
> elsif ($line =~ m/^(\d{4})-(\d\d)-(\d\d)\b/ ) {
> $day = $3 ;
> $month_num = $2 ;
> $year = $1 ;
> }
>
> I have not tested it, I hope there is no error. This allow to decode the
> 3 formats of ALL.TXT about which ones I remember about. Please note that
> the month can be numeric or alpha. If alpha, you have to convert to
> numeric, if you want to compare to a numeric value. Please note also,
> that the mode switching was an extra line in previous formats.
>
> Best wishes,
> Claude (DJ0OT)
>
>
> ___
> wsjt-devel mailing list
> wsjt-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/wsjt-devel
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT (again)
Hello All,
Here's an example from today's log:
Results: https://paste.ubuntu.com/p/382VVMth4S/
The query takes about (2) seconds or so using a $regex search on
7,390,224 logged events matching two callsigns; this is without being
indexed nor field splitting. It is one string per line imported to a one
field document in MongDB
I can post the script I used as a gist, if anyone is interested:
- Copies the ALL.TXT to $temp_file
- Converts it to CSV
- Generates two helper JS scripts
- Generates one example JS query
- Drops, then re-imports the alltext collection
- Runs the a sample Query.
Note: for incremental (daily) updates, there is no need to drop the
collection (alltext) before inserting new events. I just do that for
performance testing.
It's a simple one-line query command that would work on Win/Linux/Mac:
mongo < example.js
You can, if desired, write any number of commands to perform stats,
lookup(s), whatever, and use the same easy method to query the DB.
However, as this is a single sting entry, much of the analytical value
would be missing, as the fields / categories are not captured.
73's
Greg, KI7MT
On 7/1/19 1:27 AM, Claude Frantz wrote:
On 7/1/19 7:59 AM, Claude Frantz wrote:
Just as an example of code extract in perl:
if ($line =~ m/^(\d{4})-([A-Z][a-z]{2})-(\d{2})\b/ ) {
$day = $3 ;
$month_alpha = $2 ;
$year = $1 ;
}
elsif ($line =~ m/^(\d\d)(\d\d)(\d\d)_\d{6}\b/ ) {
$day = $3 ;
$month_num = $2 ;
$year = 2000 + $1 ;
}
elsif ($line =~ m/^(\d{4})-(\d\d)-(\d\d)\b/ ) {
$day = $3 ;
$month_num = $2 ;
$year = $1 ;
}
I have not tested it, I hope there is no error. This allow to decode the
3 formats of ALL.TXT about which ones I remember about. Please note that
the month can be numeric or alpha. If alpha, you have to convert to
numeric, if you want to compare to a numeric value. Please note also,
that the mode switching was an extra line in previous formats.
Best wishes,
Claude (DJ0OT)
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT (again)
Hello All,
This is similar to how I parse the file also; read / split the line and
check line[0], then do what's needed based on checking the first string.
At present, my ALL.TXT is over 400MB. What I've been doing to prevent
read lock issues is creating a daily diff file between a copy and the
active ALL.TXT file; sticking each diff-file in folder to process at
whatever time I wish without affecting WSJT-X operations:
alltxt-diff-20190629-0300.txt
alltxt-diff-20190630-0300.txt
etc, etc, etc
After each diff run, I update the copy so it's ready for the next day.
There are hundreds of ways to accomplish the same thing, but, I found
this to be easy and fairly painless (disk space is cheap these days :-) )
What to do with the data after has been my focus of late. I've been
playing around with MongoDB (a schema-less JSON/BSON Document storage
database) to sick the decoded lines in. You can either split the lines,
or just stick the entire line in as a new document for long term
storage/later-date access.
The $regex processing capability of MongoDB is extensive, and very fast!
One can easily parse a multitude of string combinations, even with the
entire line in one field, for example:
use wsjtx;
db.alltxt.find( { $and: [
{event:{$regex:'MY-CALL'}},
{event:{$regex:'HIS-CALL'}}
]
});
That would print the lines (documents) that contains both 'my-call' and
'his-call'.
You could add ..'DATE_STRING' or any combination you wish to further
refine the search without having to split the lines at all.
In case folks are worried about the number of documents in each
collection, I've added the entire WSPR Decode Archive (from WSPRnet) to
a MongoDB Database/collection set (one collection for each year, 2008
thru 2019, at just over 95GB on disk size). Later collections have
"millions" of decodes in them. Single collection Query Times are =< 1 to
2 seconds. With added indexing, times are in the Millisecond range :-)
Aggregate queries, those spanning multiple collections/years, vary in
time depending on the data being sought but are well within an
acceptable time limit for most use cases I've had.
73's
Greg, KI7MT
On 7/1/19 1:27 AM, Claude Frantz wrote:
On 7/1/19 7:59 AM, Claude Frantz wrote:
Just as an example of code extract in perl:
if ($line =~ m/^(\d{4})-([A-Z][a-z]{2})-(\d{2})\b/ ) {
$day = $3 ;
$month_alpha = $2 ;
$year = $1 ;
}
elsif ($line =~ m/^(\d\d)(\d\d)(\d\d)_\d{6}\b/ ) {
$day = $3 ;
$month_num = $2 ;
$year = 2000 + $1 ;
}
elsif ($line =~ m/^(\d{4})-(\d\d)-(\d\d)\b/ ) {
$day = $3 ;
$month_num = $2 ;
$year = $1 ;
}
I have not tested it, I hope there is no error. This allow to decode the
3 formats of ALL.TXT about which ones I remember about. Please note that
the month can be numeric or alpha. If alpha, you have to convert to
numeric, if you want to compare to a numeric value. Please note also,
that the mode switching was an extra line in previous formats.
Best wishes,
Claude (DJ0OT)
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT (again)
On 7/1/19 7:59 AM, Claude Frantz wrote:
Just as an example of code extract in perl:
if ($line =~ m/^(\d{4})-([A-Z][a-z]{2})-(\d{2})\b/ ) {
$day = $3 ;
$month_alpha = $2 ;
$year = $1 ;
}
elsif ($line =~ m/^(\d\d)(\d\d)(\d\d)_\d{6}\b/ ) {
$day = $3 ;
$month_num = $2 ;
$year = 2000 + $1 ;
}
elsif ($line =~ m/^(\d{4})-(\d\d)-(\d\d)\b/ ) {
$day = $3 ;
$month_num = $2 ;
$year = $1 ;
}
I have not tested it, I hope there is no error. This allow to decode the
3 formats of ALL.TXT about which ones I remember about. Please note that
the month can be numeric or alpha. If alpha, you have to convert to
numeric, if you want to compare to a numeric value. Please note also,
that the mode switching was an extra line in previous formats.
Best wishes,
Claude (DJ0OT)
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT (again)
On 6/30/19 10:18 PM, Dan Malcolm wrote: That’s correct Claude. But my PHP program has to deal with both formats in 2019. Given that one of the formats will be found, all I have to detect is a change in month, which comes after the date is harvested from the line (string). I suggest to try to match both formats, in sequence. When the one matches, you decode the date. When not, you try to match to the second format. When it matches, you extract the date. When there is no match, then the line contains another data and you should ignore it. Be sure to match at the beginning of the line. Note the "^" as the first character of the regex. Best wishes, Claude (DJ0OT) ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT (again)
That’s correct Claude.
But my PHP program has to deal with both formats in 2019. Given that one of
the formats will be found, all I have to detect is a change in month, which
comes after the date is harvested from the line (string).
__
Dan – K4SHQ
-Original Message-
From: Claude Frantz [mailto:claude.fra...@bayern-mail.de]
Sent: Sunday, June 30, 2019 12:50 PM
To: wsjt-devel@lists.sourceforge.net
Subject: Re: [wsjt-devel] ALL.TXT (again)
On 6/30/19 5:41 PM, Dan Malcolm wrote:
Hi Dan,
> Good point Mike. Right now I’m using the PHP regex function and
> “(\d{4})-(\d{2})-(\d{2})”. That worked until the format change. The
> function returns a T/F status and sticks the result into an array.
This doesn't match with the current format.
> Claude recommended a regex '^\d{6}_\d{6}\b' to detect the new format.
> I think both will work.
The regex, which I have mentioned, works fine with the current format, but not
with previous ones.
Best wishes,
Claude (DJ0OT)
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT (again)
On 6/30/19 5:41 PM, Dan Malcolm wrote:
Hi Dan,
Good point Mike. Right now I’m using the PHP regex function and
“(\d{4})-(\d{2})-(\d{2})”. That worked until the format change. The
function returns a T/F status and sticks the result into an array.
This doesn't match with the current format.
Claude recommended a regex '^\d{6}_\d{6}\b' to detect the new format. I
think both will work.
The regex, which I have mentioned, works fine with the current format,
but not with previous ones.
Best wishes,
Claude (DJ0OT)
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT (again)
Good point Mike. Right now I’m using the PHP regex function and
“(\d{4})-(\d{2})-(\d{2})”. That worked until the format change. The function
returns a T/F status and sticks the result into an array.
Claude recommended a regex '^\d{6}_\d{6}\b' to detect the new format. I think
both will work. I’ll do some research to see which method would be quickest
and to see if PHP has a ‘sscanf’ comparable function. I’ll have to use both
for the time being as 2019’s ALL.TXT will have both formats.
Thanks to both of you.
__
Dan – K4SHQ
From: Black Michael [mailto:mdblac...@yahoo.com]
Sent: Sunday, June 30, 2019 7:53 AM
To: 'WSJT software development' ; Dan Malcolm
Subject: Re: [wsjt-devel] ALL.TXT (again)
This logic should work until the format changes in 2099 or so unless it's just
allowed to roll over in which case it will still work.
You just need to get the current yymm instead of a "saved" date.
If the yymm changes that will allow you to track any idle period correctly.
#include
#include
int main(int nargs, char *argv[])
{
int yymmsave = 0;
char buf[256];
FILE *fp = fopen(argv[1], "r");
if (fp == NULL)
{
perror(argv[1]);
exit(1);
}
while (fgets(buf, sizeof(buf), fp))
{
int yymm, dd, time;
int n = sscanf(buf, "%4d%2d_%6d", , , );
if (n == 3 && yymm != yymmsave)
{
printf("%04d\n", yymm);
yymmsave = yymm;
}
}
fclose(fp);
}
de Mike W9MDB
On Saturday, June 29, 2019, 11:32:47 PM CDT, Dan Malcolm
mailto:k4...@outlook.com>> wrote:
I’m thinking that if I know I’m in year ‘19’ and the month changes from ’02 to
GT ‘02’ then I can count that as a month change. Not likely but I can envision
where I wounn’t be on the air for a month or more.
__
Dan – K4SHQ
From: Black Michael via wsjt-devel [mailto:wsjt-devel@lists.sourceforge.net]
Sent: Saturday, June 29, 2019 10:39 PM
To: wsjtx-devel
mailto:wsjt-devel@lists.sourceforge.net>>
Cc: Black Michael mailto:mdblac...@yahoo.com>>
Subject: Re: [wsjt-devel] ALL.TXT (again)
There's nothing special about the month rollover
The format is YYMMDD so here's the Feb-Mar rollover in my file for example.
190228_23594514.074 Rx FT8 -3 0.9 2255 WB4HMA HC2AO -11
190301_0014.074 Rx FT8-13 0.9 2598 VE1DBM LU1JAO -08
de Mike W9MDB
On Saturday, June 29, 2019, 09:35:19 PM CDT, Dan Malcolm
mailto:k4...@outlook.com>> wrote:
I am aware that ALL.TXT data formatting changed in late February this year. I
am trying to write a PHP program to split All.TXT in monthly text files. I
have program that does this for 2018, but it only finds January and February of
2019. The problem is probably a format change. Can anyone help me find the
first entry of the month format? Is it MMDD_”time”? or something similar?
__
Dan – K4SHQ
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net<mailto:wsjt-devel@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT (again)
Yup...sscanf exists in php.
Most people don't know about the power of sscanf to parse fixed format data.
The old format in ALL.TXT was like this
2017-01-01 00:00 7.076 MHz JT9+JT65
So this should work.int n = sscanf(buf,"%4d-%2d-%2d
%2d:%2d",,);if (n == 5) parsed_ok..otherwise fall
through.
On Sunday, June 30, 2019, 10:41:24 AM CDT, Dan Malcolm
wrote:
Good point Mike. Right now I’m using the PHP regex function and
“(\d{4})-(\d{2})-(\d{2})”. That worked until the format change. The function
returns a T/F status and sticks the result into an array.
Claude recommended aregex '^\d{6}_\d{6}\b' to detect the new format. I think
both will work. I’ll do some research to see which method would be quickest
and to see if PHP has a ‘sscanf’ comparable function. I’ll have to use both
for the time being as 2019’s ALL.TXT will have both formats.
Thanks to both of you.
__
Dan – K4SHQ
From: Black Michael [mailto:mdblac...@yahoo.com]
Sent: Sunday, June 30, 2019 7:53 AM
To: 'WSJT software development' ; Dan Malcolm
Subject: Re: [wsjt-devel] ALL.TXT (again)
This logic should work until the format changes in 2099 or so unless it's just
allowed to roll over in which case it will still work.
You just need to get the current yymm instead of a "saved" date.
If the yymm changes that will allow you to track any idle period correctly.
#include
#include
int main(int nargs, char *argv[])
{
int yymmsave = 0;
char buf[256];
FILE *fp = fopen(argv[1], "r");
if (fp == NULL)
{
perror(argv[1]);
exit(1);
}
while (fgets(buf, sizeof(buf), fp))
{
int yymm, dd, time;
int n = sscanf(buf, "%4d%2d_%6d", , , );
if (n == 3 && yymm != yymmsave)
{
printf("%04d\n", yymm);
yymmsave = yymm;
}
}
fclose(fp);
}
de Mike W9MDB
On Saturday, June 29, 2019, 11:32:47 PM CDT, Dan Malcolm
wrote:
I’m thinking that if I know I’m in year ‘19’ and the month changes from ’02 to
GT ‘02’ then I can count that as a month change. Not likely but I can envision
where I wounn’t be on the air for a month or more.
__
Dan – K4SHQ
From: Black Michael via wsjt-devel [mailto:wsjt-devel@lists.sourceforge.net]
Sent: Saturday, June 29, 2019 10:39 PM
To: wsjtx-devel
Cc: Black Michael
Subject: Re: [wsjt-devel] ALL.TXT (again)
There's nothing special about the month rollover
The format is YYMMDD so here's the Feb-Mar rollover in my file for example.
190228_235945 14.074 Rx FT8 -3 0.9 2255 WB4HMA HC2AO -11
190301_00 14.074 Rx FT8 -13 0.9 2598 VE1DBM LU1JAO -08
de Mike W9MDB
On Saturday, June 29, 2019, 09:35:19 PM CDT, Dan Malcolm
wrote:
I am aware that ALL.TXT data formatting changed in late February this year. I
am trying to write a PHP program to split All.TXT in monthly text files. I
have program that does this for 2018, but it only finds January and February of
2019. The problem is probably a format change. Can anyone help me find the
first entry of the month format? Is it MMDD_”time”? or something similar?
__
Dan – K4SHQ
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT (again)
This logic should work until the format changes in 2099 or so unless it's just
allowed to roll over in which case it will still work.You just need to get the
current yymm instead of a "saved" date.If the yymm changes that will allow you
to track any idle period correctly.
#include #include
int main(int nargs, char *argv[]){ int yymmsave = 0; char buf[256];
FILE *fp = fopen(argv[1], "r"); if (fp == NULL) { perror(argv[1]);
exit(1); }
while (fgets(buf, sizeof(buf), fp)) { int yymm, dd, time; int n =
sscanf(buf, "%4d%2d_%6d", , , );
if (n == 3 && yymm != yymmsave) { printf("%04d\n", yymm);
yymmsave = yymm; } }
fclose(fp);}
de Mike W9MDB
On Saturday, June 29, 2019, 11:32:47 PM CDT, Dan Malcolm
wrote:
I’m thinking that if I know I’m in year ‘19’ and the month changes from ’02 to
GT ‘02’ then I can count that as a month change. Not likely but I can envision
where I wounn’t be on the air for a month or more.
__
Dan – K4SHQ
From: Black Michael via wsjt-devel [mailto:wsjt-devel@lists.sourceforge.net]
Sent: Saturday, June 29, 2019 10:39 PM
To: wsjtx-devel
Cc: Black Michael
Subject: Re: [wsjt-devel] ALL.TXT (again)
There's nothing special about the month rollover
The format is YYMMDD so here's the Feb-Mar rollover in my file for example.
190228_235945 14.074 Rx FT8 -3 0.9 2255 WB4HMA HC2AO -11
190301_00 14.074 Rx FT8 -13 0.9 2598 VE1DBM LU1JAO -08
de Mike W9MDB
On Saturday, June 29, 2019, 09:35:19 PM CDT, Dan Malcolm
wrote:
I am aware that ALL.TXT data formatting changed in late February this year. I
am trying to write a PHP program to split All.TXT in monthly text files. I
have program that does this for 2018, but it only finds January and February of
2019. The problem is probably a format change. Can anyone help me find the
first entry of the month format? Is it MMDD_”time”? or something similar?
__
Dan – K4SHQ
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT (again)
On 6/30/19 6:32 AM, Dan Malcolm wrote:
Hi Dan,
I’m thinking that if I know I’m in year ‘19’ and the month changes from
’02 to GT ‘02’ then I can count that as a month change. Not likely but
I can envision where I wounn’t be on the air for a month or more.
I'm not sure, but I think that lines with different formats can still
occur in ALL.TXT. I suggest to make sure that you only examine the lines
matching the regex '^\d{6}_\d{6}\b' .
Best wishes,
Claude (DJ0OT)
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT (again)
I’m thinking that if I know I’m in year ‘19’ and the month changes from ’02 to GT ‘02’ then I can count that as a month change. Not likely but I can envision where I wounn’t be on the air for a month or more. __ Dan – K4SHQ From: Black Michael via wsjt-devel [mailto:wsjt-devel@lists.sourceforge.net] Sent: Saturday, June 29, 2019 10:39 PM To: wsjtx-devel Cc: Black Michael Subject: Re: [wsjt-devel] ALL.TXT (again) There's nothing special about the month rollover The format is YYMMDD so here's the Feb-Mar rollover in my file for example. 190228_23594514.074 Rx FT8 -3 0.9 2255 WB4HMA HC2AO -11 190301_0014.074 Rx FT8-13 0.9 2598 VE1DBM LU1JAO -08 de Mike W9MDB On Saturday, June 29, 2019, 09:35:19 PM CDT, Dan Malcolm mailto:k4...@outlook.com>> wrote: I am aware that ALL.TXT data formatting changed in late February this year. I am trying to write a PHP program to split All.TXT in monthly text files. I have program that does this for 2018, but it only finds January and February of 2019. The problem is probably a format change. Can anyone help me find the first entry of the month format? Is it MMDD_”time”? or something similar? __ Dan – K4SHQ ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net<mailto:wsjt-devel@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/wsjt-devel ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT (again)
There's nothing special about the month rollover The format is YYMMDD so here's the Feb-Mar rollover in my file for example. 190228_235945 14.074 Rx FT8 -3 0.9 2255 WB4HMA HC2AO -11190301_00 14.074 Rx FT8 -13 0.9 2598 VE1DBM LU1JAO -08 de Mike W9MDB On Saturday, June 29, 2019, 09:35:19 PM CDT, Dan Malcolm wrote: I am aware that ALL.TXT data formatting changed in late February this year. I am trying to write a PHP program to split All.TXT in monthly text files. I have program that does this for 2018, but it only finds January and February of 2019. The problem is probably a format change. Can anyone help me find the first entry of the month format? Is it MMDD_”time”? or something similar? __ Dan – K4SHQ ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
[wsjt-devel] ALL.TXT (again)
I am aware that ALL.TXT data formatting changed in late February this year. I am trying to write a PHP program to split All.TXT in monthly text files. I have program that does this for 2018, but it only finds January and February of 2019. The problem is probably a format change. Can anyone help me find the first entry of the month format? Is it MMDD_”time”? or something similar? __ Dan – K4SHQ ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT
On 5/15/2019 1:13 PM, donald rossman KC3SCZ wrote: Joe Taylor how I can get ft4 I am looking forward to it I like it up in Firefox only I see Is ft8 I am trying found it only I see is wsjt one I see is ft8 not ft4 are you still working on ft4 Available downloads are always posted on the WSJT web site. WSJT-X 2.1.0-rc5 is a beta release. It expires on June 7, 2019. If you want to try it, scroll down near the bottom on this page: http://physics.princeton.edu/pulsar/k1jt/wsjtx.html ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT
Joe Taylor how I can get ft4 I am looking forward to it I like it up in Firefox only I see Is ft8 I am trying found it only I see is wsjt one I see is ft8 not ft4 are you still working on ft4 Thank you Joe Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10 From: Joe Taylor Sent: Wednesday, May 15, 2019 1:03:02 PM To: Black Michael via wsjt-devel Subject: Re: [wsjt-devel] ALL.TXT Tnx Mike! On 5/15/2019 11:36 AM, Black Michael via wsjt-devel wrote: > I'm using an ANAN 100 and Thetis. > One thing I have to do to avoid audio dropouts under CPU load is > increase the buffer latency which affects the dt values. > > I think I did tell you guys that I used a modified version of WSJT-X to > subtract 300ms from the audio data and I was able to decode things that > were not decodable without that offset. Simply people with > 0.5 > seconds of dt. > > Data from the test attached. The beginning of the file (30 minues or > so) should be "normal"...later on in the test I started fudging the time > as I recall to see it's affect so you may see some more odd-ball numbers > in the last 30 minutes as opposed to the first 30. I used my TimeFudge > program to tweak the time to reduce the dt value but didn't take notes > on when I started that. > > It appears I started messing with my computer time around then 2100th > sample. I'm pretty sure I didn't message with time during the first > half of the test. I can probably find another Apache owner who > participate if you need more stable data. > Inline image > > > You can see the effect the buffer latency has on dt values on > hamspots.net. I had to increase my latency this morning as I was doing > some dropout testing...higher than what it was during the FT4 test. > This is closer to the default settings in Thetis of 120ms. I would > imagine I'd have a lot of trouble on FT4 now. > Inline image > > > Mike > > > > > > > > > On Wednesday, May 15, 2019, 9:50:00 AM CDT, Joe Taylor > wrote: > > > Hi Mike, > > I'm looking at some timing issues that are critical for FT4. You're > using a Flex (and I suppose, PowerSDR/Thetis), right? Could you please > send me an extract from your ALL.TXT file for the recent FT4 practice > session? That's May 8, - 0100 UTC (190509_00 through > 190509_01). Many thanks! > > -- Joe > > > ___ > wsjt-devel mailing list > wsjt-devel@lists.sourceforge.net <mailto:wsjt-devel@lists.sourceforge.net> > https://lists.sourceforge.net/lists/listinfo/wsjt-devel > > > ___ > wsjt-devel mailing list > wsjt-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/wsjt-devel > ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
[wsjt-devel] ALL.TXT
Hi Mike, I'm looking at some timing issues that are critical for FT4. You're using a Flex (and I suppose, PowerSDR/Thetis), right? Could you please send me an extract from your ALL.TXT file for the recent FT4 practice session? That's May 8, - 0100 UTC (190509_00 through 190509_01). Many thanks! -- Joe ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT - a suggestion
Thanks Claude, Most changes won't matter, because my utility will still find what I'm looking for. What I'm having to correct is my reporting format. __ Dan – K4SHQ -Original Message- From: Claude Frantz [mailto:claude.fra...@bayern-mail.de] Sent: Thursday, April 11, 2019 11:04 AM To: wsjt-devel@lists.sourceforge.net Subject: Re: [wsjt-devel] ALL.TXT - a suggestion On 4/11/19 3:43 PM, Dan Malcolm wrote: > Missed this in the release notes, but I see it now. I found: > > “Better formatting for the files ALL.TXT and ALL_WSPR.TXT” > > Since I’ve written an ALL.TXT utility in PHP, the new format will > require some coding on my part. Not a bad thing really. I’m sure a > review of my code will reveal better ways of performing some things. > It looks like now the ‘df’ offset is recorded in ALL.TXT for both ‘Tx’ and > ‘Rx. Hi Dan, The format of the ALL.TXT file has changed more that once. Be prepared to encounter more changes in the future. Best wishes, Claude (DJ0OT) ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT - a suggestion
On 4/11/19 3:43 PM, Dan Malcolm wrote: Missed this in the release notes, but I see it now. I found: “Better formatting for the files ALL.TXT and ALL_WSPR.TXT” Since I’ve written an ALL.TXT utility in PHP, the new format will require some coding on my part. Not a bad thing really. I’m sure a review of my code will reveal better ways of performing some things. It looks like now the ‘df’ offset is recorded in ALL.TXT for both ‘Tx’ and ‘Rx. Hi Dan, The format of the ALL.TXT file has changed more that once. Be prepared to encounter more changes in the future. Best wishes, Claude (DJ0OT) ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT - a suggestion
Missed this in the release notes, but I see it now. I found: “Better formatting for the files ALL.TXT and ALL_WSPR.TXT” Since I’ve written an ALL.TXT utility in PHP, the new format will require some coding on my part. Not a bad thing really. I’m sure a review of my code will reveal better ways of performing some things. It looks like now the ‘df’ offset is recorded in ALL.TXT for both ‘Tx’ and ‘Rx. __ Dan – K4SHQ From: Black Michael via wsjt-devel [mailto:wsjt-devel@lists.sourceforge.net] Sent: Thursday, April 11, 2019 7:38 AM To: WSJT software development Cc: Black Michael Subject: Re: [wsjt-devel] ALL.TXT - a suggestion If you are perusing you can use Windows "find" or Unix's "grep" For example, to get all messages with your callsign and redirect the output to a file. find "G0HDB" all.txt >g0hdb.txt Or all Tx messages find "Tx" all.txt >mytx.txt On Unix just substitute "grep" for "find". de Mike W9MDB On Thursday, April 11, 2019, 4:02:42 AM CDT, Martin Davies G0HDB mailto:marting0...@gmail.com>> wrote: Running WSJT-X v2.0.1 7ddcb7 on a Win 7 Pro (32-bit) Dell Optiplex 780. Although I like much of the new format for the information in the ALL.TXT file, it's now relatively difficult to quickly identify my outgoing transmissions amidst the mass of 'Rx' records in the file. In the previous version(s) of ALL.TXT, a transmission was very clearly identified by the 'Transmitting' label, which resulted in much-extended lines of text that were immediately apparent when perusing the file, whereas in the new file format the transmissions are now designated solely by the 'Tx' label which are extremely difficult to spot amongst all the 'Rx' labels. Could consideration please be given to reverting to using the 'Transmitting' label, or some other means (eg. colour-coding?) of more clearly differentiating the transmissions in the ALL.TXT file. 73, and thanks in advance, --- Martin, G0HDB --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net<mailto:wsjt-devel@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/wsjt-devel ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT - a suggestion
If you are perusing you can use Windows "find" or Unix's "grep" For example, to get all messages with your callsign and redirect the output to a file.find "G0HDB" all.txt >g0hdb.txt Or all Tx messagesfind "Tx" all.txt >mytx.txt On Unix just substitute "grep" for "find".de Mike W9MDB On Thursday, April 11, 2019, 4:02:42 AM CDT, Martin Davies G0HDB wrote: Running WSJT-X v2.0.1 7ddcb7 on a Win 7 Pro (32-bit) Dell Optiplex 780. Although I like much of the new format for the information in the ALL.TXT file, it's now relatively difficult to quickly identify my outgoing transmissions amidst the mass of 'Rx' records in the file. In the previous version(s) of ALL.TXT, a transmission was very clearly identified by the 'Transmitting' label, which resulted in much-extended lines of text that were immediately apparent when perusing the file, whereas in the new file format the transmissions are now designated solely by the 'Tx' label which are extremely difficult to spot amongst all the 'Rx' labels. Could consideration please be given to reverting to using the 'Transmitting' label, or some other means (eg. colour-coding?) of more clearly differentiating the transmissions in the ALL.TXT file. 73, and thanks in advance, --- Martin, G0HDB --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT - a suggestion
Use your text editor's search function to search for any text string that you want. 73 -Jim NU0C On Thu, 11 Apr 2019 09:59:16 +0100 "Martin Davies G0HDB" wrote: > Could consideration please be given to reverting to using the 'Transmitting' > label, or some > other means (eg. colour-coding?) of more clearly differentiating the > transmissions in the > ALL.TXT file. ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
[wsjt-devel] ALL.TXT - a suggestion
Running WSJT-X v2.0.1 7ddcb7 on a Win 7 Pro (32-bit) Dell Optiplex 780. Although I like much of the new format for the information in the ALL.TXT file, it's now relatively difficult to quickly identify my outgoing transmissions amidst the mass of 'Rx' records in the file. In the previous version(s) of ALL.TXT, a transmission was very clearly identified by the 'Transmitting' label, which resulted in much-extended lines of text that were immediately apparent when perusing the file, whereas in the new file format the transmissions are now designated solely by the 'Tx' label which are extremely difficult to spot amongst all the 'Rx' labels. Could consideration please be given to reverting to using the 'Transmitting' label, or some other means (eg. colour-coding?) of more clearly differentiating the transmissions in the ALL.TXT file. 73, and thanks in advance, --- Martin, G0HDB --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT does not show VHF band changes
+1 for me, been lucky enough to QRV in HF bands only. The kind of analysis I am preparing though would be highly usable in all bands. 73 de SV1CDN, Dennis! On Sun, Jul 8, 2018 at 15:32, Denley Barnette wrote: The ALL.TXT file generated by WSJT-X v1.9.1 has entries for HF band changes but not for 6 meters and above. Can that feature be added? 73 Denley W3XY -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
[wsjt-devel] ALL.TXT does not show VHF band changes
The ALL.TXT file generated by WSJT-X v1.9.1 has entries for HF band changes but not for 6 meters and above. Can that feature be added? 73 Denley W3XY -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
[wsjt-devel] ALL.TXT date line
Colleagues, gud morning fm Athens!Third time within a week of running into this issue. My setup consists of WSJT-X release 1.9.1 and W10 Home release 1803. Both are QRV 24x7 for abt a year now.Issue faced is that (third time now) WSJT-X did not print out to ALL.TXT the line of changing date, thus I can not differentiate spots in my further analysis operations. Spots do continue to get stored in ALL.TXT with newer time stamps. So far next date line gets printed. I am available fer further debugging. 73 de SV1CDN, Dennis!-- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT in MSK144 mode
On 09/05/2017 15:32, Bill Somerville wrote: > The last two fields on an MSK144 decode line are the the number of bit > errors corrected using the message FEC and the "eyopening". The latter > is a measure of signal quality, its maximum value is 2.0 and less than > that represents a reduced signal quality. HI Claude, sorry, I forgot to mention the third from last field. It is the number of frames averaged to achieve a decode. The MSK144 decoder attempts decoding single frames, then 2 and 3 frame averages in a three frame window. If that does not recover a decode and the decode depth is better than "Fast" it will try 4 frame averages. With decode depth of "Deep" it will follow up with an attempt at 5 and 7 frame averages. 73 Bill G4WJS. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT in MSK144 mode
On 01/05/2017 08:19, Claude Frantz wrote: > The first few fields at the beginning of the lines are clear to me > because they are nearly the same as in modes like JT65. But I cannot > decode the fields at the end of the lines. Hi Claude, sorry for the delay providing this information. The last two fields on an MSK144 decode line are the the number of bit errors corrected using the message FEC and the "eyopening". The latter is a measure of signal quality, its maximum value is 2.0 and less than that represents a reduced signal quality. 73 Bill G4WJS. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT in MSK144 mode
On 04/30/2017 08:32 PM, Rich Griffiths wrote: Hi Rich & all, > I hope someone has already answered you directly - I haven't seen an > answer posted to the list server. In any case, here's what I know of > the messages. Your answer is the first one I have got. Thanks ! > 4 12 - I forget the exact meaning of these, but they have to do with the > number of pings received and the number of successful decodes. Remember > that MSK144 tries a decode every 72 milliseconds, so it does a lot of > decodes. > > -1.1 - I don't remember this one, but it's also diagnostic information. The first few fields at the beginning of the lines are clear to me because they are nearly the same as in modes like JT65. But I cannot decode the fields at the end of the lines. Many thanks for your help ! Best wishes, Claude (DJ0OT) -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
[wsjt-devel] ALL.TXT in MSK144 mode
Hello all ! Please explain me all the fields in such messages found in ALL.TXT, while in MSK144 mode: 094745 -4 0.7 1543 & S51AT EI7BMB R+00 4 12 -1.1 094745 -2 1.6 1543 & S51AT EI7BMB R+00 3 12 -1.2 094745 -1 2.3 1542 & S51AT EI7BMB R+00 3 4 -0.5 094745 0 2.8 1543 & S51AT EI7BMB R+00 4 3 0.2 094745 1 4.2 1542 & S51AT EI7BMB R+00 1 10 -1.1 094745 2 12.6 1544 & S51AT EI7BMB R+00 1 11 -1.3 094745 3 14.1 1544 & S51AT EI7BMB R+00 1 6 -0.8 094945 -3 0.9 1543 & S51AT EI7BMB R+00 4 7 -0.8 Thanks ! Best wishes, Claude (DJ0OT) -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
Hi Sandro, Thanks for the suggestion about using the ISO 8601 format for Date/Time labels in ALL.TXT. I have made this change in code revision 7312. -- Joe, K1JT On 11/8/2016 4:12 PM, Alessandro Gorobey wrote: > Hi Claude and All, > > ALL.TXT is borne to dump all the lines in receive window. > > I think that insert in a database is very expansive as space and > processor and lost the simplicity of search with preferred programs. > > The format of file is changed in years and actually use some more > characters after introduction of new decoders. > > Sincerely I not like the months names in the markers and a ISO 8601 > format will be preferable, using machines with different languages. > > 2013-lug-06 16:24 14.078 MHz JT9 > 2013-jul-06 16:24 14.078 MHz JT9 > 2013-07-06 16:24 14.078 MHz JT9 <== ISO > > Sincerely I think that ALL.TXT without the nationalization strings is > good for the work. > -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
The date is thereonly when the time rolls over or you restart WSJT-X. Another reason CSV won't work too well.2359 -3 0.1 1209 # KF6LYF W7KR -15 2016-Jan-26 00:00 14.076 MHz JT9+JT65 -25 0.2 2630 @ CQ WB7CTI DN06 I use the date and keep it around for displaying with the message info. de Mike W9MDB From: Dan Malcolm <dmalcol...@mchsi.com> To: 'WSJT software development' <wsjt-devel@lists.sourceforge.net> Sent: Tuesday, November 8, 2016 5:21 PM Subject: Re: [wsjt-devel] ALL.txt idea Am I missing something? I don't get any date indicators in ALL.TXT. I get time in 24 hour format, and of course whatever else appears in the "Band Activity" window, but no date. IMHO the date (in ISO format) would be a useful piece of information, but so far I have gotten along without it. _ Dan Malcolm CFI/II K4SHQ -Original Message- From: Alessandro Gorobey [mailto:algi...@libero.it] Sent: Tuesday, November 08, 2016 3:12 PM To: wsjt-devel@lists.sourceforge.net Subject: Re: [wsjt-devel] ALL.txt idea Hi Claude and All, ALL.TXT is borne to dump all the lines in receive window. I think that insert in a database is very expansive as space and processor and lost the simplicity of search with preferred programs. The format of file is changed in years and actually use some more characters after introduction of new decoders. Sincerely I not like the months names in the markers and a ISO 8601 format will be preferable, using machines with different languages. 2013-lug-06 16:24 14.078 MHz JT9 2013-jul-06 16:24 14.078 MHz JT9 2013-07-06 16:24 14.078 MHz JT9 <== ISO Sincerely I think that ALL.TXT without the nationalization strings is good for the work. -- 73 Sandro IW3RAB Il 08/11/2016 16:49, Claude Frantz ha scritto: > On 11/07/2016 06:11 PM, Roger Rehr W3SZ wrote: > > Hi Roger, > > Writing such simple structures in a SQL database is overkill, in my > humble opinion. Concerning the ADIF log, there is no general > definition of a SQL Database mapping outside of specific logging > programs. The new alternative XML definition of ADIF 3.* is not of > great help because XML based databases are a very special matter. > > Previously, I have pointed to the problem of the ADIF TIME_ON field > which contains the data which should be in the TIME_OFF and to the > problems around this matter. Up to now, I have found no response. > > Best 88 de Claude (DJ0OT) > > -- > Developer Access Program for Intel Xeon Phi Processors Access > to Intel Xeon Phi processor-based developer platforms. > With one year of Intel Parallel Studio XE. > Training and support from Colfax. > Order your platform today. http://sdm.link/xeonphi > ___ > wsjt-devel mailing list > wsjt-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/wsjt-devel > -- 73 Sandro IW3RAB -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
Am I missing something? I don't get any date indicators in ALL.TXT. I get time in 24 hour format, and of course whatever else appears in the "Band Activity" window, but no date. IMHO the date (in ISO format) would be a useful piece of information, but so far I have gotten along without it. _ Dan Malcolm CFI/II K4SHQ -Original Message- From: Alessandro Gorobey [mailto:algi...@libero.it] Sent: Tuesday, November 08, 2016 3:12 PM To: wsjt-devel@lists.sourceforge.net Subject: Re: [wsjt-devel] ALL.txt idea Hi Claude and All, ALL.TXT is borne to dump all the lines in receive window. I think that insert in a database is very expansive as space and processor and lost the simplicity of search with preferred programs. The format of file is changed in years and actually use some more characters after introduction of new decoders. Sincerely I not like the months names in the markers and a ISO 8601 format will be preferable, using machines with different languages. 2013-lug-06 16:24 14.078 MHz JT9 2013-jul-06 16:24 14.078 MHz JT9 2013-07-06 16:24 14.078 MHz JT9 <== ISO Sincerely I think that ALL.TXT without the nationalization strings is good for the work. -- 73 Sandro IW3RAB Il 08/11/2016 16:49, Claude Frantz ha scritto: > On 11/07/2016 06:11 PM, Roger Rehr W3SZ wrote: > > Hi Roger, > > Writing such simple structures in a SQL database is overkill, in my > humble opinion. Concerning the ADIF log, there is no general > definition of a SQL Database mapping outside of specific logging > programs. The new alternative XML definition of ADIF 3.* is not of > great help because XML based databases are a very special matter. > > Previously, I have pointed to the problem of the ADIF TIME_ON field > which contains the data which should be in the TIME_OFF and to the > problems around this matter. Up to now, I have found no response. > > Best 88 de Claude (DJ0OT) > > -- > Developer Access Program for Intel Xeon Phi Processors Access > to Intel Xeon Phi processor-based developer platforms. > With one year of Intel Parallel Studio XE. > Training and support from Colfax. > Order your platform today. http://sdm.link/xeonphi > ___ > wsjt-devel mailing list > wsjt-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/wsjt-devel > -- 73 Sandro IW3RAB -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
On 11/07/2016 06:11 PM, Roger Rehr W3SZ wrote: Hi Roger, Writing such simple structures in a SQL database is overkill, in my humble opinion. Concerning the ADIF log, there is no general definition of a SQL Database mapping outside of specific logging programs. The new alternative XML definition of ADIF 3.* is not of great help because XML based databases are a very special matter. Previously, I have pointed to the problem of the ADIF TIME_ON field which contains the data which should be in the TIME_OFF and to the problems around this matter. Up to now, I have found no response. Best 88 de Claude (DJ0OT) -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
_I_ personally do not use ALL.txt or any other artifact of WSJT-X, but I do use WSJT-X often and am a software engineer that would rather err on the side of KISS than a monolithic app with more and more library dependencies. Emitting CSV isn't too much extra effort to add to WSJT-X, but adding a vendor-specific database library is, IMHO. If the rows are not 'consistently formatted' enough for CSV, then they're likely unsuitable for a structured relational DB as well. -- David Tiller Sr. Architect/Lead Consultant | CapTech (804) 304-0638 | dtil...@captechconsulting.com<mailto:dtil...@captechconsulting.com> On Nov 7, 2016, at 4:12 PM, Black Michael <mdblac...@yahoo.com<mailto:mdblac...@yahoo.com>> wrote: Would be more true if the records were consistently formatted...but they aren't. What would you use a database for? de Mike W9MDB From: David Tiller <dtil...@captechconsulting.com<mailto:dtil...@captechconsulting.com>> To: WSJT software development <wsjt-devel@lists.sourceforge.net<mailto:wsjt-devel@lists.sourceforge.net>> Sent: Monday, November 7, 2016 3:08 PM Subject: Re: [wsjt-devel] ALL.txt idea I think Dan's idea of using comma- or tab-separated fields is a nice compromise between locking users into a particular database format (regardless of how ubiquitous) and unstructured rows. CSV can be imported/parsed/chunked/scattered/gathered 100 ways to Sunday. -- David Tiller Sr. Architect/Lead Consultant | CapTech (804) 304-0638 | dtil...@captechconsulting.com<mailto:dtil...@captechconsulting.com> On Nov 7, 2016, at 4:01 PM, Dan Malcolm <dmalcol...@mchsi.com<mailto:dmalcol...@mchsi.com>> wrote: > I too use ALL.TXT. But I believe a move towards some DB functionality could > be made use comma or tab delimited fields (CSV file) instead of a space. > The resulting text file could easily be imported into any DB management > system that I am familiar with. I don't believe it would add much to the > code. > > Just my $.02 worth. > > _ > Dan Malcolm CFI/II > K4SHQ > > > -Original Message- > From: Greg Beam [mailto:ki7m...@gmail.com<mailto:ki7m...@gmail.com>] > Sent: Monday, November 07, 2016 10:00 AM > To: Black Michael <mdblac...@yahoo.com<mailto:mdblac...@yahoo.com>>; WSJT > software development > <wsjt-devel@lists.sourceforge.net<mailto:wsjt-devel@lists.sourceforge.net>> > Subject: Re: [wsjt-devel] ALL.txt idea > > Hi Mike, > > I am not sure if the ALL.txt discussion has come up before (probably has at > some point), but certainly CALL3.txt has made the rounds several times. I > was able to get CALL3 Database functionality working with SQLite for WSJT, > and have a stand alone application for WSPR to process millions of spots the > archives produce monthly, however, it does add a level of complexity to the > application that may be more than is needed for minimal operation. > > It's hard to dismiss the simplicity of flat text files. On the other hand, > having data in tables lends itself to many positive benefits. For example, > the queries in this thread would be a fairly simple SQL query rather than > requiring a Black Belt in command-line foo to extract it. > Not to mention, the SQL language is cross-platform, and generally, no > additional tools are required, but many exit at the users option. > > I am not suggesting that WSJT-X integrate post processing tools, rather, I > am merely suggesting that the data could be stored in a different format ( > DB tables) that allows easy access by third party developers or primary > users, in a more structured manner. > > > 73's > Greg, KI7MT > > On 11/7/2016 7:18 AM, Black Michael wrote: >> I wouldn't be inclined to add database support when the most common >> operations are achievable with the existing ALL.TXT. Haven't we >> discussed this before? Somebody said something about putting the >> messages in an sqlite database as I recall and had implemented something. >> Though I'm sure we could make it portable it's adding a fair bit of >> new stuff and overhead for doing this simple task. Right now my logic >> is about 100 lines of code and can pick up non-standard end-of-qso >> messages to some degree right now. >> >> Is there some other function desirable from a database that would >> justify it? I can see where adding a field for "Rx Frequency" window >> message presence would help in processing QSOs. Non-standard QSO >> messages would be an SQL challenge. >> >> >> de Mike W9MDB >> >> >> >> ---------- >> -- >> *From:* Greg Beam <k
Re: [wsjt-devel] ALL.txt idea
Probably could add a 2nd file rather than changing ALL.txt -- backwards compatible ya' know.Or make an option. Then use the UDP packet elements to generate the CSV file since that should be fully fleshed plus auto-updated when protocols are added and such. de Mike W9MDB From: David Tiller <dtil...@captechconsulting.com> To: WSJT software development <wsjt-devel@lists.sourceforge.net> Sent: Monday, November 7, 2016 3:08 PM Subject: Re: [wsjt-devel] ALL.txt idea I think Dan's idea of using comma- or tab-separated fields is a nice compromise between locking users into a particular database format (regardless of how ubiquitous) and unstructured rows. CSV can be imported/parsed/chunked/scattered/gathered 100 ways to Sunday. -- David Tiller Sr. Architect/Lead Consultant | CapTech (804) 304-0638 | dtil...@captechconsulting.com On Nov 7, 2016, at 4:01 PM, Dan Malcolm <dmalcol...@mchsi.com> wrote: > I too use ALL.TXT. But I believe a move towards some DB functionality could > be made use comma or tab delimited fields (CSV file) instead of a space. > The resulting text file could easily be imported into any DB management > system that I am familiar with. I don't believe it would add much to the > code. > > Just my $.02 worth. > > _ > Dan Malcolm CFI/II > K4SHQ > > > -Original Message- > From: Greg Beam [mailto:ki7m...@gmail.com] > Sent: Monday, November 07, 2016 10:00 AM > To: Black Michael <mdblac...@yahoo.com>; WSJT software development > <wsjt-devel@lists.sourceforge.net> > Subject: Re: [wsjt-devel] ALL.txt idea > > Hi Mike, > > I am not sure if the ALL.txt discussion has come up before (probably has at > some point), but certainly CALL3.txt has made the rounds several times. I > was able to get CALL3 Database functionality working with SQLite for WSJT, > and have a stand alone application for WSPR to process millions of spots the > archives produce monthly, however, it does add a level of complexity to the > application that may be more than is needed for minimal operation. > > It's hard to dismiss the simplicity of flat text files. On the other hand, > having data in tables lends itself to many positive benefits. For example, > the queries in this thread would be a fairly simple SQL query rather than > requiring a Black Belt in command-line foo to extract it. > Not to mention, the SQL language is cross-platform, and generally, no > additional tools are required, but many exit at the users option. > > I am not suggesting that WSJT-X integrate post processing tools, rather, I > am merely suggesting that the data could be stored in a different format ( > DB tables) that allows easy access by third party developers or primary > users, in a more structured manner. > > > 73's > Greg, KI7MT > > On 11/7/2016 7:18 AM, Black Michael wrote: >> I wouldn't be inclined to add database support when the most common >> operations are achievable with the existing ALL.TXT. Haven't we >> discussed this before? Somebody said something about putting the >> messages in an sqlite database as I recall and had implemented something. >> Though I'm sure we could make it portable it's adding a fair bit of >> new stuff and overhead for doing this simple task. Right now my logic >> is about 100 lines of code and can pick up non-standard end-of-qso >> messages to some degree right now. >> >> Is there some other function desirable from a database that would >> justify it? I can see where adding a field for "Rx Frequency" window >> message presence would help in processing QSOs. Non-standard QSO >> messages would be an SQL challenge. >> >> >> de Mike W9MDB >> >> >> >> ------ >> -- >> *From:* Greg Beam <ki7m...@gmail.com> >> *To:* Black Michael <mdblac...@yahoo.com>; WSJT software development >> <wsjt-devel@lists.sourceforge.net> >> *Sent:* Monday, November 7, 2016 12:37 AM >> *Subject:* Re: [wsjt-devel] ALL.txt idea >> >> Hi Mike, >> >> This is another email I just pulled out of the spam box. No wonder I >> was missing the other half of the conversation. >> >> Ive been thinking about these text files for some time now. There has >> to be a better way than exercising command-line foo to get the data >> folks need. Not that I mind the command line, I spend most of my time >> there, but that's not very helpful to the masses. >> >> There seems to be allot of different uses for CALL3.txt and the >> ALL.txt files. I keep circling back to database tables
Re: [wsjt-devel] ALL.txt idea
Would be more true if the records were consistently formatted...but they aren't. What would you use a database for? de Mike W9MDB From: David Tiller <dtil...@captechconsulting.com> To: WSJT software development <wsjt-devel@lists.sourceforge.net> Sent: Monday, November 7, 2016 3:08 PM Subject: Re: [wsjt-devel] ALL.txt idea I think Dan's idea of using comma- or tab-separated fields is a nice compromise between locking users into a particular database format (regardless of how ubiquitous) and unstructured rows. CSV can be imported/parsed/chunked/scattered/gathered 100 ways to Sunday. -- David Tiller Sr. Architect/Lead Consultant | CapTech (804) 304-0638 | dtil...@captechconsulting.com On Nov 7, 2016, at 4:01 PM, Dan Malcolm <dmalcol...@mchsi.com> wrote: > I too use ALL.TXT. But I believe a move towards some DB functionality could > be made use comma or tab delimited fields (CSV file) instead of a space. > The resulting text file could easily be imported into any DB management > system that I am familiar with. I don't believe it would add much to the > code. > > Just my $.02 worth. > > _ > Dan Malcolm CFI/II > K4SHQ > > > -Original Message- > From: Greg Beam [mailto:ki7m...@gmail.com] > Sent: Monday, November 07, 2016 10:00 AM > To: Black Michael <mdblac...@yahoo.com>; WSJT software development > <wsjt-devel@lists.sourceforge.net> > Subject: Re: [wsjt-devel] ALL.txt idea > > Hi Mike, > > I am not sure if the ALL.txt discussion has come up before (probably has at > some point), but certainly CALL3.txt has made the rounds several times. I > was able to get CALL3 Database functionality working with SQLite for WSJT, > and have a stand alone application for WSPR to process millions of spots the > archives produce monthly, however, it does add a level of complexity to the > application that may be more than is needed for minimal operation. > > It's hard to dismiss the simplicity of flat text files. On the other hand, > having data in tables lends itself to many positive benefits. For example, > the queries in this thread would be a fairly simple SQL query rather than > requiring a Black Belt in command-line foo to extract it. > Not to mention, the SQL language is cross-platform, and generally, no > additional tools are required, but many exit at the users option. > > I am not suggesting that WSJT-X integrate post processing tools, rather, I > am merely suggesting that the data could be stored in a different format ( > DB tables) that allows easy access by third party developers or primary > users, in a more structured manner. > > > 73's > Greg, KI7MT > > On 11/7/2016 7:18 AM, Black Michael wrote: >> I wouldn't be inclined to add database support when the most common >> operations are achievable with the existing ALL.TXT. Haven't we >> discussed this before? Somebody said something about putting the >> messages in an sqlite database as I recall and had implemented something. >> Though I'm sure we could make it portable it's adding a fair bit of >> new stuff and overhead for doing this simple task. Right now my logic >> is about 100 lines of code and can pick up non-standard end-of-qso >> messages to some degree right now. >> >> Is there some other function desirable from a database that would >> justify it? I can see where adding a field for "Rx Frequency" window >> message presence would help in processing QSOs. Non-standard QSO >> messages would be an SQL challenge. >> >> >> de Mike W9MDB >> >> >> >> ------ >> -- >> *From:* Greg Beam <ki7m...@gmail.com> >> *To:* Black Michael <mdblac...@yahoo.com>; WSJT software development >> <wsjt-devel@lists.sourceforge.net> >> *Sent:* Monday, November 7, 2016 12:37 AM >> *Subject:* Re: [wsjt-devel] ALL.txt idea >> >> Hi Mike, >> >> This is another email I just pulled out of the spam box. No wonder I >> was missing the other half of the conversation. >> >> Ive been thinking about these text files for some time now. There has >> to be a better way than exercising command-line foo to get the data >> folks need. Not that I mind the command line, I spend most of my time >> there, but that's not very helpful to the masses. >> >> There seems to be allot of different uses for CALL3.txt and the >> ALL.txt files. I keep circling back to database tables and I've played >> with the CALL.txt file a bit with WSJT, but the Qt / C++ deal with >> WSJT-X I've not gotten through yet. >>
Re: [wsjt-devel] ALL.txt idea
I think Dan's idea of using comma- or tab-separated fields is a nice compromise between locking users into a particular database format (regardless of how ubiquitous) and unstructured rows. CSV can be imported/parsed/chunked/scattered/gathered 100 ways to Sunday. -- David Tiller Sr. Architect/Lead Consultant | CapTech (804) 304-0638 | dtil...@captechconsulting.com On Nov 7, 2016, at 4:01 PM, Dan Malcolm <dmalcol...@mchsi.com> wrote: > I too use ALL.TXT. But I believe a move towards some DB functionality could > be made use comma or tab delimited fields (CSV file) instead of a space. > The resulting text file could easily be imported into any DB management > system that I am familiar with. I don't believe it would add much to the > code. > > Just my $.02 worth. > > _ > Dan Malcolm CFI/II > K4SHQ > > > -Original Message- > From: Greg Beam [mailto:ki7m...@gmail.com] > Sent: Monday, November 07, 2016 10:00 AM > To: Black Michael <mdblac...@yahoo.com>; WSJT software development > <wsjt-devel@lists.sourceforge.net> > Subject: Re: [wsjt-devel] ALL.txt idea > > Hi Mike, > > I am not sure if the ALL.txt discussion has come up before (probably has at > some point), but certainly CALL3.txt has made the rounds several times. I > was able to get CALL3 Database functionality working with SQLite for WSJT, > and have a stand alone application for WSPR to process millions of spots the > archives produce monthly, however, it does add a level of complexity to the > application that may be more than is needed for minimal operation. > > It's hard to dismiss the simplicity of flat text files. On the other hand, > having data in tables lends itself to many positive benefits. For example, > the queries in this thread would be a fairly simple SQL query rather than > requiring a Black Belt in command-line foo to extract it. > Not to mention, the SQL language is cross-platform, and generally, no > additional tools are required, but many exit at the users option. > > I am not suggesting that WSJT-X integrate post processing tools, rather, I > am merely suggesting that the data could be stored in a different format ( > DB tables) that allows easy access by third party developers or primary > users, in a more structured manner. > > > 73's > Greg, KI7MT > > On 11/7/2016 7:18 AM, Black Michael wrote: >> I wouldn't be inclined to add database support when the most common >> operations are achievable with the existing ALL.TXT. Haven't we >> discussed this before? Somebody said something about putting the >> messages in an sqlite database as I recall and had implemented something. >> Though I'm sure we could make it portable it's adding a fair bit of >> new stuff and overhead for doing this simple task. Right now my logic >> is about 100 lines of code and can pick up non-standard end-of-qso >> messages to some degree right now. >> >> Is there some other function desirable from a database that would >> justify it? I can see where adding a field for "Rx Frequency" window >> message presence would help in processing QSOs. Non-standard QSO >> messages would be an SQL challenge. >> >> >> de Mike W9MDB >> >> >> >> ---------- >> -- >> *From:* Greg Beam <ki7m...@gmail.com> >> *To:* Black Michael <mdblac...@yahoo.com>; WSJT software development >> <wsjt-devel@lists.sourceforge.net> >> *Sent:* Monday, November 7, 2016 12:37 AM >> *Subject:* Re: [wsjt-devel] ALL.txt idea >> >> Hi Mike, >> >> This is another email I just pulled out of the spam box. No wonder I >> was missing the other half of the conversation. >> >> Ive been thinking about these text files for some time now. There has >> to be a better way than exercising command-line foo to get the data >> folks need. Not that I mind the command line, I spend most of my time >> there, but that's not very helpful to the masses. >> >> There seems to be allot of different uses for CALL3.txt and the >> ALL.txt files. I keep circling back to database tables and I've played >> with the CALL.txt file a bit with WSJT, but the Qt / C++ deal with >> WSJT-X I've not gotten through yet. >> >> Maybe if we sit down and try to capture most or at least some of the >> requirements, we could start kicking around some DB integration design >> concepts. >> >> 73's >> Greg, KI7MT >> >> >> On 11/6/2016 11:03 AM, Black Michael wrote: >>> Pretty simplewhen, for examp
Re: [wsjt-devel] ALL.txt idea
I too use ALL.TXT. But I believe a move towards some DB functionality could be made use comma or tab delimited fields (CSV file) instead of a space. The resulting text file could easily be imported into any DB management system that I am familiar with. I don't believe it would add much to the code. Just my $.02 worth. _ Dan Malcolm CFI/II K4SHQ -Original Message- From: Greg Beam [mailto:ki7m...@gmail.com] Sent: Monday, November 07, 2016 10:00 AM To: Black Michael <mdblac...@yahoo.com>; WSJT software development <wsjt-devel@lists.sourceforge.net> Subject: Re: [wsjt-devel] ALL.txt idea Hi Mike, I am not sure if the ALL.txt discussion has come up before (probably has at some point), but certainly CALL3.txt has made the rounds several times. I was able to get CALL3 Database functionality working with SQLite for WSJT, and have a stand alone application for WSPR to process millions of spots the archives produce monthly, however, it does add a level of complexity to the application that may be more than is needed for minimal operation. It's hard to dismiss the simplicity of flat text files. On the other hand, having data in tables lends itself to many positive benefits. For example, the queries in this thread would be a fairly simple SQL query rather than requiring a Black Belt in command-line foo to extract it. Not to mention, the SQL language is cross-platform, and generally, no additional tools are required, but many exit at the users option. I am not suggesting that WSJT-X integrate post processing tools, rather, I am merely suggesting that the data could be stored in a different format ( DB tables) that allows easy access by third party developers or primary users, in a more structured manner. 73's Greg, KI7MT On 11/7/2016 7:18 AM, Black Michael wrote: > I wouldn't be inclined to add database support when the most common > operations are achievable with the existing ALL.TXT. Haven't we > discussed this before? Somebody said something about putting the > messages in an sqlite database as I recall and had implemented something. > Though I'm sure we could make it portable it's adding a fair bit of > new stuff and overhead for doing this simple task. Right now my logic > is about 100 lines of code and can pick up non-standard end-of-qso > messages to some degree right now. > > Is there some other function desirable from a database that would > justify it? I can see where adding a field for "Rx Frequency" window > message presence would help in processing QSOs. Non-standard QSO > messages would be an SQL challenge. > > > de Mike W9MDB > > > > -- > -- > *From:* Greg Beam <ki7m...@gmail.com> > *To:* Black Michael <mdblac...@yahoo.com>; WSJT software development > <wsjt-devel@lists.sourceforge.net> > *Sent:* Monday, November 7, 2016 12:37 AM > *Subject:* Re: [wsjt-devel] ALL.txt idea > > Hi Mike, > > This is another email I just pulled out of the spam box. No wonder I > was missing the other half of the conversation. > > Ive been thinking about these text files for some time now. There has > to be a better way than exercising command-line foo to get the data > folks need. Not that I mind the command line, I spend most of my time > there, but that's not very helpful to the masses. > > There seems to be allot of different uses for CALL3.txt and the > ALL.txt files. I keep circling back to database tables and I've played > with the CALL.txt file a bit with WSJT, but the Qt / C++ deal with > WSJT-X I've not gotten through yet. > > Maybe if we sit down and try to capture most or at least some of the > requirements, we could start kicking around some DB integration design > concepts. > > 73's > Greg, KI7MT > > > On 11/6/2016 11:03 AM, Black Michael wrote: >> Pretty simplewhen, for example, on eQSL, you get a mismatched QSL >> you can look at your messages to see if they got it wrong or you did. >> And you can submit the evidence to them so they can correct or you >> correct your own. >> >> I've also used it when noticing in JTAlert that a band isn't >> confirmed...I can look them up and see the evidence again and either >> correct my log or ask them to correct theirs. >> >> With LOTW you don't have that luxury.since there's no visibility of >> mismatched QSOs >> >> And yes...this is a WSJT Dev list item as I'm proposing submitting a >> patch to make this easy for everyone instead of just those with grep, >> EMACS, or whatever they may be currently using. >> >> de Mike W9MDB >> >> >> ---------
Re: [wsjt-devel] ALL.txt idea
I use the All.txt now for research by parsing it and feeding it into an SQLite database. I think that having WSJTX write to an SQLite or equivalent database would be a good solution, and as David says, separate purpose-built utilities accessing that database could serve a wide range of end users with varying needs without needlessly increasing the complexity of WSJTX. 73, Roger Rehr W3SZ On 11/7/2016 10:22 AM, David Tiller wrote: > > Michael, > > > My suggestion would be to follow the Unix design philosophy - do one > thing, and do it well. I vote for any of the ALL.TXT > searching/manipulation/modification be handled in a separate utility > that's decoupled from the WSJT-X codebase. > > > -- > > *David Tiller | *Senior Manager > dtil...@captechconsulting.com > c 804.304.0638 / o 804.355.0511 > > <http://www.captechconsulting.com/> > > <http://www.facebook.com/CapTechCareers> > <http://www.twitter.com/CapTechListens> > <http://www.linkedin.com/company/captech-ventures> > <http://www.stackoverflow.com/jobs/companies/captech-consulting> > <http://www.youtube.com/user/CapTechConsulting> > <https://www.instagram.com/lifeatcaptech/> > > *Best Firms 2016 #2* in Information Technology > > > > > *From:* Black Michael <mdblac...@yahoo.com> > *Sent:* Monday, November 7, 2016 9:18 AM > *To:* Greg Beam; WSJT software development > *Subject:* Re: [wsjt-devel] ALL.txt idea > > I wouldn't be inclined to add database support when the most common > operations are achievable with the existing ALL.TXT. Haven't we > discussed this before? Somebody said something about putting the > messages in an sqlite database as I recall and had implemented something. > Though I'm sure we could make it portable it's adding a fair bit of > new stuff and overhead for doing this simple task. Right now my logic > is about 100 lines of code and can pick up non-standard end-of-qso > messages to some degree right now. > > Is there some other function desirable from a database that would > justify it? I can see where adding a field for "Rx Frequency" window > message presence would help in processing QSOs. Non-standard QSO > messages would be an SQL challenge. > > > de Mike W9MDB > > > > ---- > *From:* Greg Beam <ki7m...@gmail.com> > *To:* Black Michael <mdblac...@yahoo.com>; WSJT software development > <wsjt-devel@lists.sourceforge.net> > *Sent:* Monday, November 7, 2016 12:37 AM > *Subject:* Re: [wsjt-devel] ALL.txt idea > > Hi Mike, > > This is another email I just pulled out of the spam box. No wonder I was > missing the other half of the conversation. > > Ive been thinking about these text files for some time now. There has to > be a better way than exercising command-line foo to get the data folks > need. Not that I mind the command line, I spend most of my time there, > but that's not very helpful to the masses. > > There seems to be allot of different uses for CALL3.txt and the ALL.txt > files. I keep circling back to database tables and I've played with the > CALL.txt file a bit with WSJT, but the Qt / C++ deal with WSJT-X I've > not gotten through yet. > > Maybe if we sit down and try to capture most or at least some of the > requirements, we could start kicking around some DB integration design > concepts. > > 73's > Greg, KI7MT > > > On 11/6/2016 11:03 AM, Black Michael wrote: > > Pretty simplewhen, for example, on eQSL, you get a mismatched QSL > > you can look at your messages to see if they got it wrong or you did. > > And you can submit the evidence to them so they can correct or you > > correct your own. > > > > I've also used it when noticing in JTAlert that a band isn't > > confirmed...I can look them up and see the evidence again and either > > correct my log or ask them to correct theirs. > > > > With LOTW you don't have that luxury.since there's no visibility of > > mismatched QSOs > > > > And yes...this is a WSJT Dev list item as I'm proposing submitting a > > patch to make this easy for everyone instead of just those with grep, > > EMACS, or whatever they may be currently using. > > > > de Mike W9MDB > > > > > > > > *From:* Greg Beam <ki7m...@gmail.com <mailto:ki7m...@gmail.com>> > > *To:* WSJT software development <wsjt-devel@lists.sourceforge.net > <mailto:wsjt-devel@lists.sourceforge.net>> > > *Se
Re: [wsjt-devel] ALL.txt idea
Why would you want to do that? Far too complex, requires internet, and what advantage do you gain? de Mike W9MDB From: Stephen Treubig (K4WBF) <k4...@marauderlabs.co> To: WSJT software development <wsjt-devel@lists.sourceforge.net>; Black Michael <mdblac...@yahoo.com> Sent: Monday, November 7, 2016 10:54 AM Subject: RE: [wsjt-devel] ALL.txt idea What about putting the database in the cloud and doing a API call from WJST to it? C/T K4WBF -Original Message- From: Greg Beam [mailto:ki7m...@gmail.com] Sent: Monday, November 7, 2016 10:00 AM To: Black Michael <mdblac...@yahoo.com>; WSJT software development <wsjt-devel@lists.sourceforge.net> Subject: Re: [wsjt-devel] ALL.txt idea Hi Mike, I am not sure if the ALL.txt discussion has come up before (probably has at some point), but certainly CALL3.txt has made the rounds several times. I was able to get CALL3 Database functionality working with SQLite for WSJT, and have a stand alone application for WSPR to process millions of spots the archives produce monthly, however, it does add a level of complexity to the application that may be more than is needed for minimal operation. It's hard to dismiss the simplicity of flat text files. On the other hand, having data in tables lends itself to many positive benefits. For example, the queries in this thread would be a fairly simple SQL query rather than requiring a Black Belt in command-line foo to extract it. Not to mention, the SQL language is cross-platform, and generally, no additional tools are required, but many exit at the users option. I am not suggesting that WSJT-X integrate post processing tools, rather, I am merely suggesting that the data could be stored in a different format ( DB tables) that allows easy access by third party developers or primary users, in a more structured manner. 73's Greg, KI7MT On 11/7/2016 7:18 AM, Black Michael wrote: > I wouldn't be inclined to add database support when the most common > operations are achievable with the existing ALL.TXT. Haven't we > discussed this before? Somebody said something about putting the > messages in an sqlite database as I recall and had implemented something. > Though I'm sure we could make it portable it's adding a fair bit of > new stuff and overhead for doing this simple task. Right now my logic > is about 100 lines of code and can pick up non-standard end-of-qso > messages to some degree right now. > > Is there some other function desirable from a database that would > justify it? I can see where adding a field for "Rx Frequency" window > message presence would help in processing QSOs. Non-standard QSO > messages would be an SQL challenge. > > > de Mike W9MDB > > > > -- > -- > *From:* Greg Beam <ki7m...@gmail.com> > *To:* Black Michael <mdblac...@yahoo.com>; WSJT software development > <wsjt-devel@lists.sourceforge.net> > *Sent:* Monday, November 7, 2016 12:37 AM > *Subject:* Re: [wsjt-devel] ALL.txt idea > > Hi Mike, > > This is another email I just pulled out of the spam box. No wonder I > was missing the other half of the conversation. > > Ive been thinking about these text files for some time now. There has > to be a better way than exercising command-line foo to get the data > folks need. Not that I mind the command line, I spend most of my time > there, but that's not very helpful to the masses. > > There seems to be allot of different uses for CALL3.txt and the > ALL.txt files. I keep circling back to database tables and I've played > with the CALL.txt file a bit with WSJT, but the Qt / C++ deal with > WSJT-X I've not gotten through yet. > > Maybe if we sit down and try to capture most or at least some of the > requirements, we could start kicking around some DB integration design > concepts. > > 73's > Greg, KI7MT > > > On 11/6/2016 11:03 AM, Black Michael wrote: >> Pretty simplewhen, for example, on eQSL, you get a mismatched QSL >> you can look at your messages to see if they got it wrong or you did. >> And you can submit the evidence to them so they can correct or you >> correct your own. >> >> I've also used it when noticing in JTAlert that a band isn't >> confirmed...I can look them up and see the evidence again and either >> correct my log or ask them to correct theirs. >> >> With LOTW you don't have that luxury.since there's no visibility of >> mismatched QSOs >> >> And yes...this is a WSJT Dev list item as I'm proposing submitting a >> patch to make this easy for everyone instead of just those with grep, >> EMACS, or whatever they may be currently using. >> >> de Mike W9
Re: [wsjt-devel] ALL.txt idea
Michael, My suggestion would be to follow the Unix design philosophy - do one thing, and do it well. I vote for any of the ALL.TXT searching/manipulation/modification be handled in a separate utility that's decoupled from the WSJT-X codebase. -- David Tiller | Senior Manager dtil...@captechconsulting.com c 804.304.0638 / o 804.355.0511 [http://www.captechconsulting.com/siggen/logo.png]<http://www.captechconsulting.com/> [http://www.captechconsulting.com/siggen/Dkblue_Facebook.png]<http://www.facebook.com/CapTechCareers> [http://www.captechconsulting.com/siggen/Dkblue_Twitter.png] <http://www.twitter.com/CapTechListens> [http://www.captechconsulting.com/siggen/Dkblue_LinkedIn.png] <http://www.linkedin.com/company/captech-ventures> [http://www.captechconsulting.com/siggen/Dkblue_StackOverflow.png] <http://www.stackoverflow.com/jobs/companies/captech-consulting> [http://www.captechconsulting.com/siggen/Dkblue_YouTube.png] <http://www.youtube.com/user/CapTechConsulting> [http://www.captechconsulting.com/siggen/Dkblue_Instagram.png] <https://www.instagram.com/lifeatcaptech/> Best Firms 2016 #2 in Information Technology From: Black Michael <mdblac...@yahoo.com> Sent: Monday, November 7, 2016 9:18 AM To: Greg Beam; WSJT software development Subject: Re: [wsjt-devel] ALL.txt idea I wouldn't be inclined to add database support when the most common operations are achievable with the existing ALL.TXT. Haven't we discussed this before? Somebody said something about putting the messages in an sqlite database as I recall and had implemented something. Though I'm sure we could make it portable it's adding a fair bit of new stuff and overhead for doing this simple task. Right now my logic is about 100 lines of code and can pick up non-standard end-of-qso messages to some degree right now. Is there some other function desirable from a database that would justify it? I can see where adding a field for "Rx Frequency" window message presence would help in processing QSOs. Non-standard QSO messages would be an SQL challenge. de Mike W9MDB From: Greg Beam <ki7m...@gmail.com> To: Black Michael <mdblac...@yahoo.com>; WSJT software development <wsjt-devel@lists.sourceforge.net> Sent: Monday, November 7, 2016 12:37 AM Subject: Re: [wsjt-devel] ALL.txt idea Hi Mike, This is another email I just pulled out of the spam box. No wonder I was missing the other half of the conversation. Ive been thinking about these text files for some time now. There has to be a better way than exercising command-line foo to get the data folks need. Not that I mind the command line, I spend most of my time there, but that's not very helpful to the masses. There seems to be allot of different uses for CALL3.txt and the ALL.txt files. I keep circling back to database tables and I've played with the CALL.txt file a bit with WSJT, but the Qt / C++ deal with WSJT-X I've not gotten through yet. Maybe if we sit down and try to capture most or at least some of the requirements, we could start kicking around some DB integration design concepts. 73's Greg, KI7MT On 11/6/2016 11:03 AM, Black Michael wrote: > Pretty simplewhen, for example, on eQSL, you get a mismatched QSL > you can look at your messages to see if they got it wrong or you did. > And you can submit the evidence to them so they can correct or you > correct your own. > > I've also used it when noticing in JTAlert that a band isn't > confirmed...I can look them up and see the evidence again and either > correct my log or ask them to correct theirs. > > With LOTW you don't have that luxury.since there's no visibility of > mismatched QSOs > > And yes...this is a WSJT Dev list item as I'm proposing submitting a > patch to make this easy for everyone instead of just those with grep, > EMACS, or whatever they may be currently using. > > de Mike W9MDB > > > > *From:* Greg Beam <ki7m...@gmail.com<mailto:ki7m...@gmail.com>> > *To:* WSJT software development > <wsjt-devel@lists.sourceforge.net<mailto:wsjt-devel@lists.sourceforge.net>> > *Sent:* Sunday, November 6, 2016 11:40 AM > *Subject:* Re: [wsjt-devel] ALL.txt idea > > Hello All, > > I'm not sure this is a WSJT Dev list item, and, I arrived late to the > party on this is seems, but: > > What is the end goal here? What are you trying accomplish with the > ALL.txt file? > > 73's > Greg, KI7MT > > > On 11/6/2016 10:05 AM, Claude Frantz wrote: >> On 11/05/2016 03:42 PM, Black Michael wrote: >> >> Hi Michael, >> >>> And how are you doing that? I wrote a program so all I have to do is >>> "qsos dj0ot&q
Re: [wsjt-devel] ALL.txt idea
Hi Mike, I am not sure if the ALL.txt discussion has come up before (probably has at some point), but certainly CALL3.txt has made the rounds several times. I was able to get CALL3 Database functionality working with SQLite for WSJT, and have a stand alone application for WSPR to process millions of spots the archives produce monthly, however, it does add a level of complexity to the application that may be more than is needed for minimal operation. It's hard to dismiss the simplicity of flat text files. On the other hand, having data in tables lends itself to many positive benefits. For example, the queries in this thread would be a fairly simple SQL query rather than requiring a Black Belt in command-line foo to extract it. Not to mention, the SQL language is cross-platform, and generally, no additional tools are required, but many exit at the users option. I am not suggesting that WSJT-X integrate post processing tools, rather, I am merely suggesting that the data could be stored in a different format ( DB tables) that allows easy access by third party developers or primary users, in a more structured manner. 73's Greg, KI7MT On 11/7/2016 7:18 AM, Black Michael wrote: > I wouldn't be inclined to add database support when the most common > operations are achievable with the existing ALL.TXT. Haven't we > discussed this before? Somebody said something about putting the > messages in an sqlite database as I recall and had implemented something. > Though I'm sure we could make it portable it's adding a fair bit of new > stuff and overhead for doing this simple task. Right now my logic is > about 100 lines of code and can pick up non-standard end-of-qso messages > to some degree right now. > > Is there some other function desirable from a database that would > justify it? I can see where adding a field for "Rx Frequency" window > message presence would help in processing QSOs. Non-standard QSO > messages would be an SQL challenge. > > > de Mike W9MDB > > > > > *From:* Greg Beam <ki7m...@gmail.com> > *To:* Black Michael <mdblac...@yahoo.com>; WSJT software development > <wsjt-devel@lists.sourceforge.net> > *Sent:* Monday, November 7, 2016 12:37 AM > *Subject:* Re: [wsjt-devel] ALL.txt idea > > Hi Mike, > > This is another email I just pulled out of the spam box. No wonder I was > missing the other half of the conversation. > > Ive been thinking about these text files for some time now. There has to > be a better way than exercising command-line foo to get the data folks > need. Not that I mind the command line, I spend most of my time there, > but that's not very helpful to the masses. > > There seems to be allot of different uses for CALL3.txt and the ALL.txt > files. I keep circling back to database tables and I've played with the > CALL.txt file a bit with WSJT, but the Qt / C++ deal with WSJT-X I've > not gotten through yet. > > Maybe if we sit down and try to capture most or at least some of the > requirements, we could start kicking around some DB integration design > concepts. > > 73's > Greg, KI7MT > > > On 11/6/2016 11:03 AM, Black Michael wrote: >> Pretty simplewhen, for example, on eQSL, you get a mismatched QSL >> you can look at your messages to see if they got it wrong or you did. >> And you can submit the evidence to them so they can correct or you >> correct your own. >> >> I've also used it when noticing in JTAlert that a band isn't >> confirmed...I can look them up and see the evidence again and either >> correct my log or ask them to correct theirs. >> >> With LOTW you don't have that luxury.since there's no visibility of >> mismatched QSOs >> >> And yes...this is a WSJT Dev list item as I'm proposing submitting a >> patch to make this easy for everyone instead of just those with grep, >> EMACS, or whatever they may be currently using. >> >> de Mike W9MDB >> >> >> -------- >> *From:* Greg Beam <ki7m...@gmail.com <mailto:ki7m...@gmail.com>> >> *To:* WSJT software development <wsjt-devel@lists.sourceforge.net > <mailto:wsjt-devel@lists.sourceforge.net>> >> *Sent:* Sunday, November 6, 2016 11:40 AM >> *Subject:* Re: [wsjt-devel] ALL.txt idea >> >> Hello All, >> >> I'm not sure this is a WSJT Dev list item, and, I arrived late to the >> party on this is seems, but: >> >> What is the end goal here? What are you trying accomplish with the >> ALL.txt file? >> >> 73's >> Greg, KI7MT >> >> >> On 11
Re: [wsjt-devel] ALL.txt idea
Here's what I have now. It's able to pick out non-standard messages at the end of a QSO. Should pick out other non-standards in the middle too. So this example QSO pops out when searching my files. 2014-Nov-21 2136 Transmitting 28.0758 MHz JT65: WB6BNE W9MDB EM492014-Nov-21 2137 -1 -0.2 1334 # W9MDB WB6BNE -102014-Nov-21 2138 Transmitting 28.0758 MHz JT65: WB6BNE W9MDB R-012014-Nov-21 2139 -1 -0.2 1334 # W9MDB RRR 732014-Nov-21 2140 Transmitting 28.0758 MHz JT65: TU 5 BANDS 73 I use a one line batch file called qsos.bat to run it. I backup my ALL.TXT regularly plus keep archived prior years in Dropbox so I have 3 files to search right now. C:\Users\%USERNAME%\Dropbox\Ham\WSJT-X\alltxt w9mdb %1 c:\Users\%USERNAME%\Dropbox\Ham\WSJT-X So running "qsos wb6bne" will find every "ALL*.[txt|TXT]" file in the path and search for the requested QSO partner. https://www.dropbox.com/s/c8ysex4z7wbb2wl/alltxt.exe?dl=1 https://www.dropbox.com/s/vdugalwbsz36xkm/alltxt.c?dl=1 This is a standalone program right now so Windows needs the dirent function. In WSJT-X would use a Qt function to do that me thinkst. https://www.dropbox.com/s/84vv2cyufehssz1/dirent.c?dl=1 https://www.dropbox.com/s/hszext4ap13vph2/dirent.h?dl=1 de Mike W9MDB From: Greg Beam <ki7m...@gmail.com> To: Black Michael <mdblac...@yahoo.com>; WSJT software development <wsjt-devel@lists.sourceforge.net> Sent: Monday, November 7, 2016 12:37 AM Subject: Re: [wsjt-devel] ALL.txt idea Hi Mike, This is another email I just pulled out of the spam box. No wonder I was missing the other half of the conversation. Ive been thinking about these text files for some time now. There has to be a better way than exercising command-line foo to get the data folks need. Not that I mind the command line, I spend most of my time there, but that's not very helpful to the masses. There seems to be allot of different uses for CALL3.txt and the ALL.txt files. I keep circling back to database tables and I've played with the CALL.txt file a bit with WSJT, but the Qt / C++ deal with WSJT-X I've not gotten through yet. Maybe if we sit down and try to capture most or at least some of the requirements, we could start kicking around some DB integration design concepts. 73's Greg, KI7MT On 11/6/2016 11:03 AM, Black Michael wrote: > Pretty simplewhen, for example, on eQSL, you get a mismatched QSL > you can look at your messages to see if they got it wrong or you did. > And you can submit the evidence to them so they can correct or you > correct your own. > > I've also used it when noticing in JTAlert that a band isn't > confirmed...I can look them up and see the evidence again and either > correct my log or ask them to correct theirs. > > With LOTW you don't have that luxury.since there's no visibility of > mismatched QSOs > > And yes...this is a WSJT Dev list item as I'm proposing submitting a > patch to make this easy for everyone instead of just those with grep, > EMACS, or whatever they may be currently using. > > de Mike W9MDB > > > > *From:* Greg Beam <ki7m...@gmail.com> > *To:* WSJT software development <wsjt-devel@lists.sourceforge.net> > *Sent:* Sunday, November 6, 2016 11:40 AM > *Subject:* Re: [wsjt-devel] ALL.txt idea > > Hello All, > > I'm not sure this is a WSJT Dev list item, and, I arrived late to the > party on this is seems, but: > > What is the end goal here? What are you trying accomplish with the > ALL.txt file? > > 73's > Greg, KI7MT > > > On 11/6/2016 10:05 AM, Claude Frantz wrote: >> On 11/05/2016 03:42 PM, Black Michael wrote: >> >> Hi Michael, >> >>> And how are you doing that? I wrote a program so all I have to do is >>> "qsos dj0ot" >> >> I'm simply using the emacs editor, which has fine search functions. >> >>> Don't have any with you but here's what the output for me looks like for >>> DJ0TP -- it looks at two files (actually looks at every "ALL*.txt" >>> wildcard match) >> >> OK ! The same result is possible using the grep utility, which is >> available in nearly all Unix and Linux distributions. It's really a >> standard tool. >> >> Best 88 de Claude (DJ0OT) >> > > -- > Developer Access Program for Intel Xeon Phi Processors > Access to Intel Xeon Phi processor-based developer platforms. > With one year of Intel Parallel Studio XE. > Training and support from Colfax. > Order your platform today. http://sdm.link/xeonphi > ___ > w
Re: [wsjt-devel] ALL.txt idea
I wouldn't be inclined to add database support when the most common operations are achievable with the existing ALL.TXT. Haven't we discussed this before? Somebody said something about putting the messages in an sqlite database as I recall and had implemented something.Though I'm sure we could make it portable it's adding a fair bit of new stuff and overhead for doing this simple task. Right now my logic is about 100 lines of code and can pick up non-standard end-of-qso messages to some degree right now. Is there some other function desirable from a database that would justify it? I can see where adding a field for "Rx Frequency" window message presence would help in processing QSOs. Non-standard QSO messages would be an SQL challenge. de Mike W9MDB From: Greg Beam <ki7m...@gmail.com> To: Black Michael <mdblac...@yahoo.com>; WSJT software development <wsjt-devel@lists.sourceforge.net> Sent: Monday, November 7, 2016 12:37 AM Subject: Re: [wsjt-devel] ALL.txt idea Hi Mike, This is another email I just pulled out of the spam box. No wonder I was missing the other half of the conversation. Ive been thinking about these text files for some time now. There has to be a better way than exercising command-line foo to get the data folks need. Not that I mind the command line, I spend most of my time there, but that's not very helpful to the masses. There seems to be allot of different uses for CALL3.txt and the ALL.txt files. I keep circling back to database tables and I've played with the CALL.txt file a bit with WSJT, but the Qt / C++ deal with WSJT-X I've not gotten through yet. Maybe if we sit down and try to capture most or at least some of the requirements, we could start kicking around some DB integration design concepts. 73's Greg, KI7MT On 11/6/2016 11:03 AM, Black Michael wrote: > Pretty simplewhen, for example, on eQSL, you get a mismatched QSL > you can look at your messages to see if they got it wrong or you did. > And you can submit the evidence to them so they can correct or you > correct your own. > > I've also used it when noticing in JTAlert that a band isn't > confirmed...I can look them up and see the evidence again and either > correct my log or ask them to correct theirs. > > With LOTW you don't have that luxury.since there's no visibility of > mismatched QSOs > > And yes...this is a WSJT Dev list item as I'm proposing submitting a > patch to make this easy for everyone instead of just those with grep, > EMACS, or whatever they may be currently using. > > de Mike W9MDB > > > > *From:* Greg Beam <ki7m...@gmail.com> > *To:* WSJT software development <wsjt-devel@lists.sourceforge.net> > *Sent:* Sunday, November 6, 2016 11:40 AM > *Subject:* Re: [wsjt-devel] ALL.txt idea > > Hello All, > > I'm not sure this is a WSJT Dev list item, and, I arrived late to the > party on this is seems, but: > > What is the end goal here? What are you trying accomplish with the > ALL.txt file? > > 73's > Greg, KI7MT > > > On 11/6/2016 10:05 AM, Claude Frantz wrote: >> On 11/05/2016 03:42 PM, Black Michael wrote: >> >> Hi Michael, >> >>> And how are you doing that? I wrote a program so all I have to do is >>> "qsos dj0ot" >> >> I'm simply using the emacs editor, which has fine search functions. >> >>> Don't have any with you but here's what the output for me looks like for >>> DJ0TP -- it looks at two files (actually looks at every "ALL*.txt" >>> wildcard match) >> >> OK ! The same result is possible using the grep utility, which is >> available in nearly all Unix and Linux distributions. It's really a >> standard tool. >> >> Best 88 de Claude (DJ0OT) >> > > -- > Developer Access Program for Intel Xeon Phi Processors > Access to Intel Xeon Phi processor-based developer platforms. > With one year of Intel Parallel Studio XE. > Training and support from Colfax. > Order your platform today. http://sdm.link/xeonphi > ___ > wsjt-devel mailing list > wsjt-devel@lists.sourceforge.net <mailto:wsjt-devel@lists.sourceforge.net> > https://lists.sourceforge.net/lists/listinfo/wsjt-devel > > > > > -- > Developer Access Program for Intel Xeon Phi Processors > Access to Intel Xeon Phi processor-based developer platforms. > With one year of Intel Parallel Studio XE. > Training and support from Colfax. > Order your platform today. http://sdm.link/xe
Re: [wsjt-devel] ALL.txt idea
Hi Mike, This is another email I just pulled out of the spam box. No wonder I was missing the other half of the conversation. Ive been thinking about these text files for some time now. There has to be a better way than exercising command-line foo to get the data folks need. Not that I mind the command line, I spend most of my time there, but that's not very helpful to the masses. There seems to be allot of different uses for CALL3.txt and the ALL.txt files. I keep circling back to database tables and I've played with the CALL.txt file a bit with WSJT, but the Qt / C++ deal with WSJT-X I've not gotten through yet. Maybe if we sit down and try to capture most or at least some of the requirements, we could start kicking around some DB integration design concepts. 73's Greg, KI7MT On 11/6/2016 11:03 AM, Black Michael wrote: > Pretty simplewhen, for example, on eQSL, you get a mismatched QSL > you can look at your messages to see if they got it wrong or you did. > And you can submit the evidence to them so they can correct or you > correct your own. > > I've also used it when noticing in JTAlert that a band isn't > confirmed...I can look them up and see the evidence again and either > correct my log or ask them to correct theirs. > > With LOTW you don't have that luxury.since there's no visibility of > mismatched QSOs > > And yes...this is a WSJT Dev list item as I'm proposing submitting a > patch to make this easy for everyone instead of just those with grep, > EMACS, or whatever they may be currently using. > > de Mike W9MDB > > > > *From:* Greg Beam <ki7m...@gmail.com> > *To:* WSJT software development <wsjt-devel@lists.sourceforge.net> > *Sent:* Sunday, November 6, 2016 11:40 AM > *Subject:* Re: [wsjt-devel] ALL.txt idea > > Hello All, > > I'm not sure this is a WSJT Dev list item, and, I arrived late to the > party on this is seems, but: > > What is the end goal here? What are you trying accomplish with the > ALL.txt file? > > 73's > Greg, KI7MT > > > On 11/6/2016 10:05 AM, Claude Frantz wrote: >> On 11/05/2016 03:42 PM, Black Michael wrote: >> >> Hi Michael, >> >>> And how are you doing that? I wrote a program so all I have to do is >>> "qsos dj0ot" >> >> I'm simply using the emacs editor, which has fine search functions. >> >>> Don't have any with you but here's what the output for me looks like for >>> DJ0TP -- it looks at two files (actually looks at every "ALL*.txt" >>> wildcard match) >> >> OK ! The same result is possible using the grep utility, which is >> available in nearly all Unix and Linux distributions. It's really a >> standard tool. >> >> Best 88 de Claude (DJ0OT) >> > > -- > Developer Access Program for Intel Xeon Phi Processors > Access to Intel Xeon Phi processor-based developer platforms. > With one year of Intel Parallel Studio XE. > Training and support from Colfax. > Order your platform today. http://sdm.link/xeonphi > ___ > wsjt-devel mailing list > wsjt-devel@lists.sourceforge.net <mailto:wsjt-devel@lists.sourceforge.net> > https://lists.sourceforge.net/lists/listinfo/wsjt-devel > > > > > -- > Developer Access Program for Intel Xeon Phi Processors > Access to Intel Xeon Phi processor-based developer platforms. > With one year of Intel Parallel Studio XE. > Training and support from Colfax. > Order your platform today. http://sdm.link/xeonphi > > > > ___ > wsjt-devel mailing list > wsjt-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/wsjt-devel > -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
Installing Git for windows (Git Bash) will give all sorts of powerful *Nix tools; Grep, Awk, Sed, Cat, Cut, TR and so on. Additionally, If you have JTSDK-Win installed, you have all those tools and many more available through any of the JTSDK-ENV windows. 73's Greg, KI7MT > Hi Michael, > >> You're making my point that most can't do this...most don't have either >> of those tools. > > They are available for a MS-Windows, but not per default. > -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
With my utility:Searching
c:\Users\Black\Dropbox\Ham\WSJT-X\ALL-ICOM.txtSearching
c:\Users\Black\Dropbox\Ham\WSJT-X\ALL.TXT2016-Jul-26 2201 Transmitting 14.076
MHz JT9 : NO7T W9MDB EM492016-Jul-26 2202 -12 -0.1 2846
@ W9MDB NO7T -192016-Jul-26 2203 Transmitting 14.076 MHz JT9 : NO7T W9MDB
EM492016-Jul-26 2203 Transmitting 14.076 MHz JT9 : NO7T W9MDB
R-142016-Jul-26 2205 Transmitting 14.076 MHz JT9 : NO7T W9MDB
732016-Aug-27 1344 Transmitting 7.076 MHz JT65 : NO7T W9MDB
EM492016-Oct-11 1325 Transmitting 7.076 MHz JT9 : NO7T W9MDB
EM492016-Oct-11 1327 Transmitting 7.076 MHz JT9 : NO7T W9MDB
EM49Searching c:\Users\Black\Dropbox\Ham\WSJT-X\ALL2015.txt
With your method i get 1,559 records.
If you stack the greps you get closer...but no date.
C:\Users\Black\Dropbox\Ham\WSJT-X>grep -i NO7T all*.txt| grep W9MDBALL.TXT:2201
Transmitting 14.076 MHz JT9: NO7T W9MDB EM49ALL.TXT:2202 -12 -0.1 2846 @
W9MDB NO7T -19ALL.TXT:2203 Transmitting 14.076 MHz JT9: NO7T W9MDB
EM49ALL.TXT:2203 Transmitting 14.076 MHz JT9: NO7T W9MDB R-14ALL.TXT:2205
Transmitting 14.076 MHz JT9: NO7T W9MDB 73ALL.TXT:1344 Transmitting 7.076
MHz JT65: NO7T W9MDB EM49ALL.TXT:1325 Transmitting 7.076 MHz JT9: NO7T
W9MDB EM49ALL.TXT:1327 Transmitting 7.076 MHz JT9: NO7T W9MDB EM49
Of over 4,000 users how many do you think even know how to do this?For those of
us on the developers list more likely to know how.
But we shouldn't be writing software for "experts" alone.
de Mike W9MDB
From: Claude Frantz <claude.fra...@bayern-mail.de>
To: wsjt-devel@lists.sourceforge.net
Sent: Sunday, November 6, 2016 1:10 PM
Subject: Re: [wsjt-devel] ALL.txt idea
On 11/06/2016 07:06 PM, Black Michael wrote:
Hi Michael,
> You're making my point that most can't do this...most don't have either
> of those tools.
They are available for a MS-Windows, but not per default.
> I did use grep for a while and found it inadequate.
You get exactly the same result using "grep -i DJ0OT ALL*.TXT", e.g.
> My util, for example, reformats the line to include the date on every line.
> You can then pump that through grep again to look for a year or month of
> interest if you have too many results.
grep is a very flexible tool. Entering "grep -P
'(^2[0-9]{3}-[A-Z][a-z]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}|DJ0OT)' ALL.TXT"
and you get the date too.
In fact, you can add numerous other addition features, if you really
want, but do you think that this is really necessary to solve the
mentioned problem ?
Best 88 de Claude
--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
On 11/06/2016 07:06 PM, Black Michael wrote:
Hi Michael,
> You're making my point that most can't do this...most don't have either
> of those tools.
They are available for a MS-Windows, but not per default.
> I did use grep for a while and found it inadequate.
You get exactly the same result using "grep -i DJ0OT ALL*.TXT", e.g.
> My util, for example, reformats the line to include the date on every line.
> You can then pump that through grep again to look for a year or month of
> interest if you have too many results.
grep is a very flexible tool. Entering "grep -P
'(^2[0-9]{3}-[A-Z][a-z]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}|DJ0OT)' ALL.TXT"
and you get the date too.
In fact, you can add numerous other addition features, if you really
want, but do you think that this is really necessary to solve the
mentioned problem ?
Best 88 de Claude
--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
You're making my point that most can't do this...most don't have either of those tools.I did use grep for a while and found it inadequate.My util, for example, reformats the line to include the date on every line.You can then pump that through grep again to look for a year or month of interest if you have too many results.If I added it to WSJT-X I'd add that capability. de Mike W9MDB From: Claude Frantz <claude.fra...@bayern-mail.de> To: wsjt-devel@lists.sourceforge.net Sent: Sunday, November 6, 2016 11:05 AM Subject: Re: [wsjt-devel] ALL.txt idea On 11/05/2016 03:42 PM, Black Michael wrote: Hi Michael, > And how are you doing that? I wrote a program so all I have to do is > "qsos dj0ot" I'm simply using the emacs editor, which has fine search functions. > Don't have any with you but here's what the output for me looks like for > DJ0TP -- it looks at two files (actually looks at every "ALL*.txt" > wildcard match) OK ! The same result is possible using the grep utility, which is available in nearly all Unix and Linux distributions. It's really a standard tool. Best 88 de Claude (DJ0OT) -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
Pretty simplewhen, for example, on eQSL, you get a mismatched QSL you can look at your messages to see if they got it wrong or you did.And you can submit the evidence to them so they can correct or you correct your own. I've also used it when noticing in JTAlert that a band isn't confirmed...I can look them up and see the evidence again and either correct my log or ask them to correct theirs. With LOTW you don't have that luxury.since there's no visibility of mismatched QSOs And yes...this is a WSJT Dev list item as I'm proposing submitting a patch to make this easy for everyone instead of just those with grep, EMACS, or whatever they may be currently using. de Mike W9MDB From: Greg Beam <ki7m...@gmail.com> To: WSJT software development <wsjt-devel@lists.sourceforge.net> Sent: Sunday, November 6, 2016 11:40 AM Subject: Re: [wsjt-devel] ALL.txt idea Hello All, I'm not sure this is a WSJT Dev list item, and, I arrived late to the party on this is seems, but: What is the end goal here? What are you trying accomplish with the ALL.txt file? 73's Greg, KI7MT On 11/6/2016 10:05 AM, Claude Frantz wrote: > On 11/05/2016 03:42 PM, Black Michael wrote: > > Hi Michael, > >> And how are you doing that? I wrote a program so all I have to do is >> "qsos dj0ot" > > I'm simply using the emacs editor, which has fine search functions. > >> Don't have any with you but here's what the output for me looks like for >> DJ0TP -- it looks at two files (actually looks at every "ALL*.txt" >> wildcard match) > > OK ! The same result is possible using the grep utility, which is > available in nearly all Unix and Linux distributions. It's really a > standard tool. > > Best 88 de Claude (DJ0OT) > -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
And how are you doing that? I wrote a program so all I have to do is "qsos dj0ot" Don't have any with you but here's what the output for me looks like for DJ0TP -- it looks at two files (actually looks at every "ALL*.txt" wildcard match) I doubt many ops have the ability to do this easily. Searching C:\Users\mike/Dropbox/Ham/WSJT-X/ALL.TXT2014-Oct-15 1624 -1 0.9 1512 # W9MDB DJ0TP JO432014-Oct-15 1625 Transmitting 21.0755 MHz JT65: DJ0TP W9MDB -012014-Oct-15 1626 -2 0.9 1512 # W9MDB DJ0TP R-052014-Oct-15 1627 Transmitting 21.0755 MHz JT65: DJ0TP W9MDB RRR2014-Oct-15 1628 -4 0.9 1512 # W9MDB DJ0TP 732014-Oct-15 1629 Transmitting 21.0755 MHz JT65: DJ0TP W9MDB 73Searching C:\Users\mike/Dropbox/Ham/WSJT-X/ALL-ICOM.TXT de Mike W9MDB From: Claude Frantz <claude.fra...@bayern-mail.de> To: wsjt-devel@lists.sourceforge.net Sent: Saturday, November 5, 2016 4:47 AM Subject: Re: [wsjt-devel] ALL.txt idea On 11/05/2016 07:14 AM, Black Michael wrote: Hi Michael, > I use the ALL.txt to answer questions about QSOs when needed (mostly > from eQSL). > So when I see a reject I can check the ALL.txt to see what happened. > So I can search for a callsign and pull up the history of that callsign > with me. That is exactly what I'm doing too. > I can write an addition to WSJT-X that I think should appear under the > File menu and replace the "Erase ALL.txt" with a dialog with several > options. > > #1 Erase > #2 Rename > #3 Search > > Would such an addition be OK? In my opinion, it's not necessary. Best 88 de Claude (DJ0OT) -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
On 11/05/2016 07:14 AM, Black Michael wrote: Hi Michael, > I use the ALL.txt to answer questions about QSOs when needed (mostly > from eQSL). > So when I see a reject I can check the ALL.txt to see what happened. > So I can search for a callsign and pull up the history of that callsign > with me. That is exactly what I'm doing too. > I can write an addition to WSJT-X that I think should appear under the > File menu and replace the "Erase ALL.txt" with a dialog with several > options. > > #1 Erase > #2 Rename > #3 Search > > Would such an addition be OK? In my opinion, it's not necessary. Best 88 de Claude (DJ0OT) -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
[wsjt-devel] ALL.txt idea
I use the ALL.txt to answer questions about QSOs when needed (mostly from eQSL). So when I see a reject I can check the ALL.txt to see what happened.So I can search for a callsign and pull up the history of that callsign with me. In addition, I keep ALL.txt files around and archive them off by year so I can search back more than one year. I can write an addition to WSJT-X that I think should appear under the File menu and replace the "Erase ALL.txt" with a dialog with several options. #1 Erase#2 Rename#3 Search Would such an addition be OK? de Mike W9MDB -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.TXT
On 18/05/2016 21:31, Alessandro Gorobey wrote: > JT65 49 columns, 22 fixed, 27 for message Hi Sandro, JT65 and JT4 (probably JT9 eventually as well) when used in VHF & up mode with deep search and message averaging now have some quality information appended after the decoded message text. 73 BIll G4WJS. -- Mobile security can be enabling, not merely restricting. Employees who bring their own devices (BYOD) to work are irked by the imposition of MDM restrictions. Mobile Device Manager Plus allows you to control only the apps on BYO-devices by containerizing them, leaving personal data untouched! https://ad.doubleclick.net/ddm/clk/304595813;131938128;j ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
[wsjt-devel] ALL.TXT
Hi All, I notice that trailing spaces in file ALL.TXT have changed in time. This is more evident in actual HF mixed mode (JT65+JT9) Seem as JT65 message can be longer than a JT9 JT9 44 columns, 22 fixed, 22 for message JT65 49 columns, 22 fixed, 27 for message Is not a problem, I only ask for reason or should we expect long messages ? Thank in advance -- 73 Sandro IW3RAB -- Mobile security can be enabling, not merely restricting. Employees who bring their own devices (BYOD) to work are irked by the imposition of MDM restrictions. Mobile Device Manager Plus allows you to control only the apps on BYO-devices by containerizing them, leaving personal data untouched! https://ad.doubleclick.net/ddm/clk/304595813;131938128;j ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
