Your elapsed times in g oing from 100,000 records to 107 million looks like linear scaling. That's the best one can hope for. Working "fine" means running in less than two minutes. It did "work" for 107 million records. But it didn't "work fine" because it took longer than two minutes. I suppose this developer also expects it to take less than two minutes to process 100 billion records. The application developer needs to go to remedial multiplication class. I learned how to multiply in the third grade. Sixty years later I still remember how to multiply. It's also important to know when, why, and what to multiply.
Bill Fairchild Franklin, TN ----- Original Message ----- From: "Rob van der Heij" <[email protected]> To: [email protected] Sent: Wednesday, April 17, 2013 3:12:09 AM Subject: Re: Good Performing Code (Was: Millicode Instructions) On 17 April 2013 07:34, Ed Jaffe <[email protected]> wrote: > On 4/16/2013 3:55 PM, Scott Ford wrote: > >> I want to ask a question, in this day/age and processing power is it >> really worth >> being concerned about Assembler instructions speed ? >> > > I am not unbiased. My answer is exactly what one would expect from the > CTO of a software company that has been authoring far-better-performing > code since 1978. Am I proud of slides 67-74 in this SHARE presentation? > https://share.confex.com/**share/120/webprogram/Handout/** > Session13319/%28E%29JES%**20Update_SHARE%20120.pdf<https://share.confex.com/share/120/webprogram/Handout/Session13319/%28E%29JES%20Update_SHARE%20120.pdf> > > You bet I am! > You may! :-) I think that most of the people on the list realize that much of this type of discussions is to hone your skills, understand what challenges your code offers to the machine, and be able to diagnose issues with code fragments where it is relevant. My experience is that those who don't appreciate the low level concepts often don't see the big picture either. Much of what I learn here while lurking provides background information for when I need to address an issue in the code base that I inherited. I recently found the code spending a lot of time in sequentially searching a linked list, comparing each key with an EX of a CLC instruction (which I understand is not a good idea anymore). Since only the search argument was variable length, I could copy it to a fixed length field and do plain CLC instead. While the big improvements are in the algorithms, some understanding of the machine architecture is helpful when thinking about those issues as well. I don't know which part helped most to write the next release that added a lot of new function, reduced the size of code by 30% and reduced CPU usage by factor of 5-10. While it is true that CPUs have gotten faster, the volume of data we operate on has often increased as well. And even when algorithms are O(N) the volume of data can still surprise you. My favorite quote from an application developer is: "Rob, we know this is not effecient. But it works fine for 100,000 records. Why would it not work for 107 million?" (hint, 100K records took less than 2 minutes to run, the nightly batch took 27 hrs). Rob
