Re: [DX-CHAT] LOTW cannot possibly catch up
Remember the days when you mailed in cards to the league and it took 3 months. to process? I'll take 10 days over that anytime. It used to take 6 years to get your cards back from Box 88. Now I see even our Russian friends are on LOTW. And I had my PT0S 160 Qso confirmed in days. LOTW even at 20 days is a tremendous tool. I'm confident they'll figure this out, it's probably not a hardware issue only, but the database bogging down under the mountain of data. These things aren't as easily managed and scaleable as they appear to people not in the IT world. I don't understand the angst over a backlog of processing data during the end of the year rush. Unless the world really is going to end in a week like the Mayans predicted. 73 Don N1DG At 12:24 PM 12/14/2012, Joe Subich, W4TV wrote: Go to the LOTW site and read carefully. There is a discussion that tells, in part, that new hardware is on order, and is expected to arrive in 6-8 weeks. THEY ARE WORKING ON IT!!! That is if you *believe* the claim that the hardware will fix it. With a very large database, decreases in throughput can just as easily be due to reaching a tipping point in the database structure (the point at which the indexing fails). Since there has been no diagnostic data reported to support the hardware hypothesis, users are left in a trust me position while the processing delay vaults past ten days as of approximately 16:30z today - on its way to 20 days by the end of the month. 73, ... Joe, W4TV On 12/14/2012 11:42 AM, Mike(W5UC) Kathy (K5MWH) wrote: Go to the LOTW site and read carefully. There is a discussion that tells, in part, that new hardware is on order, and is expected to arrive in 6-8 weeks. THEY ARE WORKING ON IT!!! RELAX!!! Constantly getting your shorts all in a wad is uncomfortable, and bad for your health. It raises your blood pressure. 73, Mike, W5UC On 12/14/2012 9:08 AM, Charlie Wooten NF4A wrote: I wish you guys would quitcherbitchin about LOTW.give em a chance to fix the problem.don't sit there and criticize something that is allowing you to get DXCC confirmations in lightning speed compared to 10 years agotake off your computer expert hat and let the league's staff deal with the problem.. NF4A --- To unsubscribe or subscribe to this list. Please send a message to imail...@njdxa.org In the message body put either unsubscribe dx-chat or subscribe dx-chat This is the DX-CHAT reflector sponsored by the NJDXA http://njdxa.org --- --- To unsubscribe or subscribe to this list. Please send a message to imail...@njdxa.org In the message body put either unsubscribe dx-chat or subscribe dx-chat This is the DX-CHAT reflector sponsored by the NJDXA http://njdxa.org --- - N1DG--Licensed since 1962 EX-WB2DND, A61AD (GUEST OP, QSL MGR), A52DG, /KH9, /BV, /VS6, /4X, /9V Pilot: VU7RG, 3Y0X, VK9ML, D68C, VK0IR, K8XP/KH9, 9M0C, ZK1XXP, ZL9CI Webmaster: VP8ORK, K4M, BS7H, K5K, A52A, VK0IR, 9M0C, ZK1XXP, WB2DND/KH9, BQ9P, ZL9CI QSL Manager: A61AD, A61AO, A61X, A61AQ, KH7Z Member: CWops, ARRL, DDXA, YCCC 2006 Inductee into the CQ DX Hall of Fame DON GREENBAUM , DUXBURY, MA, USA AIM: aurumtel --- To unsubscribe or subscribe to this list. Please send a message to imail...@njdxa.org In the message body put either unsubscribe dx-chat or subscribe dx-chat This is the DX-CHAT reflector sponsored by the NJDXA http://njdxa.org ---
Re: [DX-CHAT] LOTW cannot possibly catch up
LOTW even at 20 days is a tremendous tool. I'm not concerned about 20 days for a confirmation. My concern is 20 days (or more) even before the initial transaction is processed and I get confirmation that the QSOs are safely in the database. I'm confident they'll figure this out, it's probably not a hardware issue only, but the database bogging down under the mountain of data. I'm confident that it's not a hardware issue - the same hardware could process 100 QSOs/minute when LotW first went online according to info I've received privately from those who were there at the time. I would bet the storage hardware has not been properly maintained and tuned as the application changed. New storage hardware is a band-aid ... nothing more. LotW will have gone from 200 million QSO records in January 2009 to more than 470 million by the end of this year - that's over 3,000 QSOs/per hour. In the last 19 days more than 4.6 million new QSO records have been added to the database - that's more than 191,000 per day (3,200/hr). Now if LotW did not have a problem with 3,000 QSO/s per hour average (other than an occasional one or two day backlog around CQWW and ARRL DX) why does it suddenly have a 20 day delay? The issue is that ARRL's IT department and management have ignored the warning signs. Work should have been in full swing for some time to upgrade the entire program - not just a storage subsystem on an emergency basis and patch the tQSL interface - with something designed to handle the size and scope of a modern LotW. As it is, Newington still seems to have its collective head buried in the sand. 73, ... Joe, W4TV On 12/15/2012 8:35 AM, Donald Greenbaum wrote: Remember the days when you mailed in cards to the league and it took 3 months. to process? I'll take 10 days over that anytime. It used to take 6 years to get your cards back from Box 88. Now I see even our Russian friends are on LOTW. And I had my PT0S 160 Qso confirmed in days. LOTW even at 20 days is a tremendous tool. I'm confident they'll figure this out, it's probably not a hardware issue only, but the database bogging down under the mountain of data. These things aren't as easily managed and scaleable as they appear to people not in the IT world. I don't understand the angst over a backlog of processing data during the end of the year rush. Unless the world really is going to end in a week like the Mayans predicted. 73 Don N1DG At 12:24 PM 12/14/2012, Joe Subich, W4TV wrote: Go to the LOTW site and read carefully. There is a discussion that tells, in part, that new hardware is on order, and is expected to arrive in 6-8 weeks. THEY ARE WORKING ON IT!!! That is if you *believe* the claim that the hardware will fix it. With a very large database, decreases in throughput can just as easily be due to reaching a tipping point in the database structure (the point at which the indexing fails). Since there has been no diagnostic data reported to support the hardware hypothesis, users are left in a trust me position while the processing delay vaults past ten days as of approximately 16:30z today - on its way to 20 days by the end of the month. 73, ... Joe, W4TV On 12/14/2012 11:42 AM, Mike(W5UC) Kathy (K5MWH) wrote: Go to the LOTW site and read carefully. There is a discussion that tells, in part, that new hardware is on order, and is expected to arrive in 6-8 weeks. THEY ARE WORKING ON IT!!! RELAX!!! Constantly getting your shorts all in a wad is uncomfortable, and bad for your health. It raises your blood pressure. 73, Mike, W5UC On 12/14/2012 9:08 AM, Charlie Wooten NF4A wrote: I wish you guys would quitcherbitchin about LOTW.give em a chance to fix the problem.don't sit there and criticize something that is allowing you to get DXCC confirmations in lightning speed compared to 10 years agotake off your computer expert hat and let the league's staff deal with the problem.. NF4A --- To unsubscribe or subscribe to this list. Please send a message to imail...@njdxa.org In the message body put either unsubscribe dx-chat or subscribe dx-chat This is the DX-CHAT reflector sponsored by the NJDXA http://njdxa.org --- --- To unsubscribe or subscribe to this list. Please send a message to imail...@njdxa.org In the message body put either unsubscribe dx-chat or subscribe dx-chat This is the DX-CHAT reflector sponsored by the NJDXA http://njdxa.org --- - N1DG--Licensed since 1962 EX-WB2DND, A61AD (GUEST OP, QSL MGR), A52DG, /KH9, /BV, /VS6, /4X, /9V Pilot: VU7RG, 3Y0X, VK9ML, D68C, VK0IR, K8XP/KH9, 9M0C, ZK1XXP, ZL9CI Webmaster: VP8ORK, K4M, BS7H, K5K, A52A, VK0IR, 9M0C, ZK1XXP, WB2DND/KH9, BQ9P,
Re: [DX-CHAT] Re: LOTW cannot possibly catch up
Having read most of the comments critical of the time lag, I am reminded of the time when it took forever to get confirmations via the mail and some bureaus. This whole issue seems to me to be centered among those who either can't recall those days or the younger folks who suffer from what is hopefully not a communicable form of the disease knows as instant gratification-itis. Don K1VSK --- To unsubscribe or subscribe to this list. Please send a message to imail...@njdxa.org In the message body put either unsubscribe dx-chat or subscribe dx-chat This is the DX-CHAT reflector sponsored by the NJDXA http://njdxa.org ---
Re: [DX-CHAT] LOTW cannot possibly catch up
All of us on this reflector are painfully aware that LoTW has a problem, Additionally we all have seen this discussion go on ad nausium. All of us are concerned about the problem. None of this is new NEWS. So far I have told you nothing new. The complaining goes on on, and accomplishes nothing. SO, please let me ask; what do the complainers expect to accomplish by continuing to kick this horse? If you just like to hear yourself complain, please, just send the e-mails to yourself, and spare the rest of us from having to deal with all of the superfluous BS. 73, Mike, W5UC --- To unsubscribe or subscribe to this list. Please send a message to imail...@njdxa.org In the message body put either unsubscribe dx-chat or subscribe dx-chat This is the DX-CHAT reflector sponsored by the NJDXA http://njdxa.org ---
Re: [DX-CHAT] LOTW cannot possibly catch up
On 12/15/2012 Donald Greenbaum wrote: I don't understand the angst over a backlog of processing data during the end of the year rush. Unless the world really is going to end in a week like the Mayans predicted. 73 Don N1DG Partly because those of us who have been watching have come to realize that LOTW was 1)Losing files - which the league finally admitted and 2)the processing rate is getting MUCH slower, to the point that the delay is getting longer and longer, on the rough order of 20-23 hours/day (most days). The are currently at 10+ days - at THAT rate, we will be at 18 days for a file that is uploaded today, by that time, we'll be at 36 days. If things are not fixed, in 2-3 months, the backlog will be LONGER than your 3 months to get a card back The issue is not even that. I don't know if you are in the software business, I am, and a lot of people who are worried are also. We are seeing the symptoms of a system that is in distress and in danger of failure. We know that these kinds of problems are NOT usually a simple fix, and even if it is, there is significant testing needed. The time to call for the ARRL to fix the problem is before the system fails. Think of it like the STS Challenger - the engineers were watching the O rings, and were saying 'there are problems, that seem to get worse at low temps, but management kept saying 'no problem'. The trick is to SEE the problem BEFORE the system totally melts down This is the system showing a cough, and maybe if we do something now, we can save it before it dies, instead of waiting for it to turn to pneumonia and dyeing --- To unsubscribe or subscribe to this list. Please send a message to imail...@njdxa.org In the message body put either unsubscribe dx-chat or subscribe dx-chat This is the DX-CHAT reflector sponsored by the NJDXA http://njdxa.org ---
Re: [DX-CHAT] Re: LOTW cannot possibly catch up
Those days are irrelevant to the subject at hand and it has nothing to do with instant gratification. It has to do with the current state of technology, what it can provide, and what it should provide. There's also no need to talk about spark transmitters or paper logs, either. Barry W2UP On 12/15/2012 12:47, Don Berger wrote: Having read most of the comments critical of the time lag, I am reminded of the time when it took forever to get confirmations via the mail and some bureaus. This whole issue seems to me to be centered among those who either can't recall those days or the younger folks who suffer from what is hopefully not a communicable form of the disease knows as instant gratification-itis. Don K1VSK --- To unsubscribe or subscribe to this list. Please send a message to imail...@njdxa.org In the message body put either unsubscribe dx-chat or subscribe dx-chat This is the DX-CHAT reflector sponsored by the NJDXA http://njdxa.org --- --- To unsubscribe or subscribe to this list. Please send a message to imail...@njdxa.org In the message body put either unsubscribe dx-chat or subscribe dx-chat This is the DX-CHAT reflector sponsored by the NJDXA http://njdxa.org ---
Re: [DX-CHAT] LOTW cannot possibly catch up
Might want to consider creating a filter for the thread? Maybe filter out any messages with LoTW in the subject field ;) 73, Mike, K2CD Mike(W5UC) Kathy (K5MWH) wrote: All of us on this reflector are painfully aware that LoTW has a problem, Additionally we all have seen this discussion go on ad nausium. All of us are concerned about the problem. None of this is new NEWS. So far I have told you nothing new. The complaining goes on on, and accomplishes nothing. SO, please let me ask; what do the complainers expect to accomplish by continuing to kick this horse? If you just like to hear yourself complain, please, just send the e-mails to yourself, and spare the rest of us from having to deal with all of the superfluous BS. 73, Mike, W5UC --- To unsubscribe or subscribe to this list. Please send a message to imail...@njdxa.org In the message body put either unsubscribe dx-chat or subscribe dx-chat This is the DX-CHAT reflector sponsored by the NJDXA http://njdxa.org ---
Re: [DX-CHAT] Re: LOTW cannot possibly catch up
On 12/15/2012 Don Berger wrote: Having read most of the comments critical of the time lag, I am reminded of the time when it took forever to get confirmations via the mail and some bureaus. This whole issue seems to me to be centered among those who either can't recall those days or the younger folks who suffer from what is hopefully not a communicable form of the disease knows as instant gratification-itis. Don K1VSK No, looking at it, the biggest noise seems to be among DBAs, programmers, system designers and the like who are like the engineers at Morton Thiokol who were saying 'There is a problem, we need to look into this' and 'I don't think we should launch', when the folks a bit more removed were saying 'Oh, we haven't had a failure yet, so what if it isn't working exactly as designed' The time to FIX things is BEFORE they fail. Now the ARRL MAY have plans to do so, but so far, they aren't telling us -- 73 de KG2V - Charles Gallo Quality Custom Machine-shop work for the radio amateur (sm) --- To unsubscribe or subscribe to this list. Please send a message to imail...@njdxa.org In the message body put either unsubscribe dx-chat or subscribe dx-chat This is the DX-CHAT reflector sponsored by the NJDXA http://njdxa.org ---
RE: [DX-CHAT] LOTW cannot possibly catch up
Every transaction-oriented software system has a performance bottleneck that limits its throughput. Eliminating that bottleneck -- either by improving the software or upgrading the hardware -- always reveals the next bottleneck. The ARRL has determined that LotW's throughput is limited by the rate at which new QSOs can be inserted into the database, and has ordered a new storage subsystem to eliminate this bottleneck. This step is necessary, but may or may not be sufficient; we should know in 4-6 weeks. At present, ~50% of the QSOs uploaded to LotW are duplicates. There are several practical ways to reduce the load that these duplicates impose on LotW without imposing onerous requirements on users; at least one of them will be implemented. The ARRL is now providing daily LotW status updates and hourly LotW Queue Length updates, a level of transparency we have long sought and should further encourage. Cynical, sarcastic, non-constructive posts second-guessing the ARRL's course of action are not the best way to accomplish this. I am not suggesting that anyone back off on expressing their interest in seeing LotW restored to reliable service. The constructive message to send to ARRL management and representatives is this: I'm glad that ARRL management is finally attending to LotW and hope that this first step is sufficient; but if it's not, I expect them to waste no time in taking the follow-on actions required to bring LotW to an acceptable level of throughput and reliability, keeping the user community abreast of their efforts. 73, Dave, AA6YQ --- To unsubscribe or subscribe to this list. Please send a message to imail...@njdxa.org In the message body put either unsubscribe dx-chat or subscribe dx-chat This is the DX-CHAT reflector sponsored by the NJDXA http://njdxa.org ---
RE: [DX-CHAT] Re: LOTW cannot possibly catch up
Don, I've been reading comments on the LotW situation for several weeks now -- on this reflector, on other email reflectors, and in the online forums for several web sites. Based on that overall picture, I think we can group the critical comments into two categories: The first category concerns comments from those who are generally interested in getting the situation resolved. That includes users with and without an IT background, many of whom understand the issues regarding a worldwide database application of this magnitude, and how difficult (and expensive) the management of such a system can be -- to say nothing of the fixes when something goes kaflooey. I consider this category the constructive comments. The second category concerns comments from those who actually don't care about LotW. Most of these are from people who have a grudge against the ARRL in particular and/or other amateur radio groups or societies in general. Some of these people are simply being ornery curmudgeons just they can be (or because they like, for some twisted reason, to stir up trouble for others). Many of the comments from this group, when you step back and look at them overall, show a lack of understanding (or an unwillingness to care) about the situation in general or in specifics. Many of the comments become contradictory; that is to say, they answer another's comment solely to further their agenda at the moment, and/or smear LotW and/or ARRL, regardless of what they said in the past -- be it a month ago, a week ago, or 10 minutes ago. I consider this category destructive comments. I have no problem with the constructive comments. I may not agree with someone, but I respect them for saying it. These comments are from people who genuinely want to get the situation fixed -- both for the moment and for good. Now as far as the destructive comments, I am sure that I'm not the only one who would like to tell this small group to Shut The Front Door. These comments are not helping. They are sowing confusion, misleading conclusions, and in general are raising the background noise without improving the Signal to Noise ratio. So... those who truly wish to assist or are truly concerned about fixing the problem, please continue to post. As far as the other group goes... your displeasure has been duly noted. Thank you. Now: let the rest of us work on the issues without further distractions. 73, ron w3wn Note: Just speaking for myself. Not putting on my administrative cap on... yet. -Original Message- From: kf...@njdxa.org [mailto:kf...@njdxa.org] On Behalf Of Don Berger Sent: Saturday, December 15, 2012 2:47 PM To: Dx-Chat Subject: Re: [DX-CHAT] Re: LOTW cannot possibly catch up Having read most of the comments critical of the time lag, I am reminded of the time when it took forever to get confirmations via the mail and some bureaus. This whole issue seems to me to be centered among those who either can't recall those days or the younger folks who suffer from what is hopefully not a communicable form of the disease knows as instant gratification-itis. Don K1VSK --- To unsubscribe or subscribe to this list. Please send a message to imail...@njdxa.org In the message body put either unsubscribe dx-chat or subscribe dx-chat This is the DX-CHAT reflector sponsored by the NJDXA http://njdxa.org --- --- To unsubscribe or subscribe to this list. Please send a message to imail...@njdxa.org In the message body put either unsubscribe dx-chat or subscribe dx-chat This is the DX-CHAT reflector sponsored by the NJDXA http://njdxa.org ---
RE: [DX-CHAT] LOTW cannot possibly catch up
Blaming the ARRL IT staff for the problem is pointless. I've dealt with them in the past on issues. They are doing the best they can with what resources they have been given. To use a poor analogy: This is akin to blaming the ditch digger for using a shovel, when a backhoe would be faster. The ditch digger would rather have a backhoe as well, but it's all he was issued. Or, it's akin to blaming the line crew working on a downed power line for taking 2 hours to arrive instead of 15 minutes, when it was the power company's management that consolidated the local maintenance garages into one central one 25 miles away, and then reduced the staff levels due to redundancy and efficiency. As I have said here and elsewhere: If you don't like the situation -- and I don't like it either -- complaining here is pointless. If you are an ARRL member, complain to your Division Director. The Division Directors make up the ARRL Board, and they are SUPPOSED to listen to the membership -- that's us, and remember, WE are the League, not the Board, not the HQ staff, but US, the members. If we don't tell them we're concerned, and that we want something done... nothing will happen. Further, considering that in the recent past... -- the design bug has been discovered and fixed, so no more mysterious loss of logs -- the update process is now more transparent, as new tools are available to track upload processing -- new hardware is on order -- the Trusted QSL open source project is being reopened (that's the programming that is at least part of the LotW system), and C++ developers are being sought to help -- and if you don't believe me on that or you didn't get the email, just ask AA6YQ ... I'd say that the Directors ARE hearing our concerns, and that they ARE telling the HQ staff to devote more resources to the system, and that things ARE moving along. Maybe not as fast as we want, but they ARE in progress. It may not be ideal. It's a start. So insulting the people in the trenches working on the system does little to no good. Kvetching about things here may blow off steam make you feel better, but does little good. You want something done? Contact your Division Director. And keep contacting him (or her). 73, ron w3wn -Original Message- From: kf...@njdxa.org [mailto:kf...@njdxa.org] On Behalf Of Joe Subich, W4TV Sent: Saturday, December 15, 2012 4:18 PM To: w...@suddenlink.net Cc: Dx-Chat Subject: Re: [DX-CHAT] LOTW cannot possibly catch up SO, please let me ask; what do the complainers expect to accomplish by continuing to kick this horse? How about getting others involved in *doing something* about the problem? Since the ARRL IT department seems to want to bury its head in the sand and claims a new storage system will magically fix all the ills of LotW after doing nothing for three years ... since ARRL Management wants to spend as little as possible on LotW and has basically ignored it for six years, how about screaming at your Director to provide real resources for LotW and DX programs for a change? In case folks have forgotten, the DXCC Yearbook is no longer being mailed to each participant for free. DXCC fees went up this year, LotW fees went up this year (for those who generally purchased in bulk), etc. DX and contest related programs have been getting the short end of the stick for a lot of years ... this is one place that the line needs to be drawn. Fix LotW - stop giving it lip service. Instead of a clear and well researched diagnosis after appropriate tests and consultation, this patient who is slipping into a coma is being told take this expensive pill and call me in six months if you don't improve. 73, ... Joe, W4TV On 12/15/2012 2:55 PM, Mike(W5UC) Kathy (K5MWH) wrote: All of us on this reflector are painfully aware that LoTW has a problem, Additionally we all have seen this discussion go on ad nausium. All of us are concerned about the problem. None of this is new NEWS. So far I have told you nothing new. The complaining goes on on, and accomplishes nothing. SO, please let me ask; what do the complainers expect to accomplish by continuing to kick this horse? If you just like to hear yourself complain, please, just send the e-mails to yourself, and spare the rest of us from having to deal with all of the superfluous BS. 73, Mike, W5UC --- To unsubscribe or subscribe to this list. Please send a message to imail...@njdxa.org In the message body put either unsubscribe dx-chat or subscribe dx-chat This is the DX-CHAT reflector sponsored by the NJDXA http://njdxa.org --- --- To unsubscribe or subscribe to this list. Please send a message to imail...@njdxa.org In the message body put either unsubscribe dx-chat or subscribe dx-chat This is the DX-CHAT
