Hi Claude,
Personally, I don't have a need for it either. If all I am after is QSO
validation: grep + awk or a good quality text editor is all that's
needed :-)
However, if one wants to do any sort of data analysis, the flat file
format is less than ideal. Normalizing the data would go a long way
toward shrinking the footprint, however, it also adds a level of
complexity some may not find very pleasant.
I am aware, and have done a bit of parsing in the past regarding the
varying data structures of the file. It's changed a a good bit over the
last few versions.
In any case, it's a doable thing if one wants to store their history /
data (for whatever reason). IoT devices often use these
variable-structure data sets with great success. The variance in the
ALL.TXT file is minimal compared to some I've seen.
73's
Greg, KI7MT
On 7/9/19 4:11 AM, Claude Frantz wrote:
On 7/9/19 10:58 AM, Greg Beam wrote:
Hi Greg,
I think parsing the lines into fields is the best long-term solution
for storage (allows for much better indexing).
This make only sense if you can assign a clear definition to every
field. In the case of ALL.TXT, not all lines have the same structure.
Especially, the date and the time can be in different lines, depending
on the used WSJTX release.
However, we'd need to do a fair bit of checking on each line to
determine its structure first;
This is essential.
To be honest, I cannot see the advantage of having the data, stored in
ALL.TXT, in a database. Myself, I'm using ALL.TXT only to verify strange
situations or to verify a situation where I get a QSO confirmation for a
QSO not in the log. A database is very useful when we often need access
to the data and when we need rather complex queries. These requirements
do not apply to ALL.TXT. Remember, the database management software
needs disk space and processing time.
Best wishes,
Claude (DJ0OT)
_______________________________________________
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
