I think Dan's idea of using comma- or tab-separated fields is a nice compromise between locking users into a particular database format (regardless of how ubiquitous) and unstructured rows.
CSV can be imported/parsed/chunked/scattered/gathered 100 ways to Sunday. -- David Tiller Sr. Architect/Lead Consultant | CapTech (804) 304-0638 | dtil...@captechconsulting.com On Nov 7, 2016, at 4:01 PM, Dan Malcolm <dmalcol...@mchsi.com> wrote: > I too use ALL.TXT. But I believe a move towards some DB functionality could > be made use comma or tab delimited fields (CSV file) instead of a space. > The resulting text file could easily be imported into any DB management > system that I am familiar with. I don't believe it would add much to the > code. > > Just my $.02 worth. > > _____________________________ > Dan Malcolm CFI/II > K4SHQ > > > -----Original Message----- > From: Greg Beam [mailto:ki7m...@gmail.com] > Sent: Monday, November 07, 2016 10:00 AM > To: Black Michael <mdblac...@yahoo.com>; WSJT software development > <wsjt-devel@lists.sourceforge.net> > Subject: Re: [wsjt-devel] ALL.txt idea > > Hi Mike, > > I am not sure if the ALL.txt discussion has come up before (probably has at > some point), but certainly CALL3.txt has made the rounds several times. I > was able to get CALL3 Database functionality working with SQLite for WSJT, > and have a stand alone application for WSPR to process millions of spots the > archives produce monthly, however, it does add a level of complexity to the > application that may be more than is needed for minimal operation. > > It's hard to dismiss the simplicity of flat text files. On the other hand, > having data in tables lends itself to many positive benefits. For example, > the queries in this thread would be a fairly simple SQL query rather than > requiring a Black Belt in command-line foo to extract it. > Not to mention, the SQL language is cross-platform, and generally, no > additional tools are required, but many exit at the users option. > > I am not suggesting that WSJT-X integrate post processing tools, rather, I > am merely suggesting that the data could be stored in a different format ( > DB tables) that allows easy access by third party developers or primary > users, in a more structured manner. > > > 73's > Greg, KI7MT > > On 11/7/2016 7:18 AM, Black Michael wrote: >> I wouldn't be inclined to add database support when the most common >> operations are achievable with the existing ALL.TXT. Haven't we >> discussed this before? Somebody said something about putting the >> messages in an sqlite database as I recall and had implemented something. >> Though I'm sure we could make it portable it's adding a fair bit of >> new stuff and overhead for doing this simple task. Right now my logic >> is about 100 lines of code and can pick up non-standard end-of-qso >> messages to some degree right now. >> >> Is there some other function desirable from a database that would >> justify it? I can see where adding a field for "Rx Frequency" window >> message presence would help in processing QSOs. Non-standard QSO >> messages would be an SQL challenge. >> >> >> de Mike W9MDB >> >> >> >> ---------------------------------------------------------------------- >> -- >> *From:* Greg Beam <ki7m...@gmail.com> >> *To:* Black Michael <mdblac...@yahoo.com>; WSJT software development >> <wsjt-devel@lists.sourceforge.net> >> *Sent:* Monday, November 7, 2016 12:37 AM >> *Subject:* Re: [wsjt-devel] ALL.txt idea >> >> Hi Mike, >> >> This is another email I just pulled out of the spam box. No wonder I >> was missing the other half of the conversation. >> >> Ive been thinking about these text files for some time now. There has >> to be a better way than exercising command-line foo to get the data >> folks need. Not that I mind the command line, I spend most of my time >> there, but that's not very helpful to the masses. >> >> There seems to be allot of different uses for CALL3.txt and the >> ALL.txt files. I keep circling back to database tables and I've played >> with the CALL.txt file a bit with WSJT, but the Qt / C++ deal with >> WSJT-X I've not gotten through yet. >> >> Maybe if we sit down and try to capture most or at least some of the >> requirements, we could start kicking around some DB integration design >> concepts. >> >> 73's >> Greg, KI7MT >> >> >> On 11/6/2016 11:03 AM, Black Michael wrote: >>> Pretty simple....when, for example, on eQSL, you get a mismatched QSL >>> you can look at your messages to see if they got it wrong or you did. >>> And you can submit the evidence to them so they can correct or you >>> correct your own. >>> >>> I've also used it when noticing in JTAlert that a band isn't >>> confirmed...I can look them up and see the evidence again and either >>> correct my log or ask them to correct theirs. >>> >>> With LOTW you don't have that luxury.since there's no visibility of >>> mismatched QSOs >>> >>> And yes...this is a WSJT Dev list item as I'm proposing submitting a >>> patch to make this easy for everyone instead of just those with grep, >>> EMACS, or whatever they may be currently using. >>> >>> de Mike W9MDB >>> >>> >>> --------------------------------------------------------------------- >>> --- >>> *From:* Greg Beam <ki7m...@gmail.com <mailto:ki7m...@gmail.com>> >>> *To:* WSJT software development <wsjt-devel@lists.sourceforge.net >> <mailto:wsjt-devel@lists.sourceforge.net>> >>> *Sent:* Sunday, November 6, 2016 11:40 AM >>> *Subject:* Re: [wsjt-devel] ALL.txt idea >>> >>> Hello All, >>> >>> I'm not sure this is a WSJT Dev list item, and, I arrived late to the >>> party on this is seems, but: >>> >>> What is the end goal here? What are you trying accomplish with the >>> ALL.txt file? >>> >>> 73's >>> Greg, KI7MT >>> >>> >>> On 11/6/2016 10:05 AM, Claude Frantz wrote: >>>> On 11/05/2016 03:42 PM, Black Michael wrote: >>>> >>>> Hi Michael, >>>> >>>>> And how are you doing that? I wrote a program so all I have to do >>>>> is "qsos dj0ot" >>>> >>>> I'm simply using the emacs editor, which has fine search functions. >>>> >>>>> Don't have any with you but here's what the output for me looks >>>>> like for DJ0TP -- it looks at two files (actually looks at every > "ALL*.txt" >>>>> wildcard match) >>>> >>>> OK ! The same result is possible using the grep utility, which is >>>> available in nearly all Unix and Linux distributions. It's really a >>>> standard tool. >>>> >>>> Best 88 de Claude (DJ0OT) >>>> >>> >>> >> ---------------------------------------------------------------------- >> -------- >>> Developer Access Program for Intel Xeon Phi Processors Access to >>> Intel Xeon Phi processor-based developer platforms. >>> With one year of Intel Parallel Studio XE. >>> Training and support from Colfax. >>> Order your platform today. http://sdm.link/xeonphi >>> _______________________________________________ >>> wsjt-devel mailing list >>> wsjt-devel@lists.sourceforge.net >> <mailto:wsjt-devel@lists.sourceforge.net> >> <mailto:wsjt-devel@lists.sourceforge.net >> <mailto:wsjt-devel@lists.sourceforge.net>> >>> https://lists.sourceforge.net/lists/listinfo/wsjt-devel >> >>> >>> >>> >>> >>> >> ---------------------------------------------------------------------- >> -------- >>> Developer Access Program for Intel Xeon Phi Processors Access to >>> Intel Xeon Phi processor-based developer platforms. >>> With one year of Intel Parallel Studio XE. >>> Training and support from Colfax. >>> Order your platform today. http://sdm.link/xeonphi >>> >>> >>> >>> _______________________________________________ >>> wsjt-devel mailing list >>> wsjt-devel@lists.sourceforge.net >>> <mailto:wsjt-devel@lists.sourceforge.net> >>> https://lists.sourceforge.net/lists/listinfo/wsjt-devel >>> >> >> > > ---------------------------------------------------------------------------- > -- > Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon > Phi processor-based developer platforms. > With one year of Intel Parallel Studio XE. > Training and support from Colfax. > Order your platform today. http://sdm.link/xeonphi > _______________________________________________ > wsjt-devel mailing list > wsjt-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/wsjt-devel > > > ------------------------------------------------------------------------------ > Developer Access Program for Intel Xeon Phi Processors > Access to Intel Xeon Phi processor-based developer platforms. > With one year of Intel Parallel Studio XE. > Training and support from Colfax. > Order your platform today. http://sdm.link/xeonphi > _______________________________________________ > wsjt-devel mailing list > wsjt-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/wsjt-devel ------------------------------------------------------------------------------ Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi _______________________________________________ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
