There are standards for dealing with missing data. See "Imputation (statistics) <http://en.wikipedia.org/wiki/Imputation_%28statistics%29>". Unfortunately, if one deals with missing data properly it takes one beyond those standards into the realm of universal artificial intelligence <http://www.hutter1.net/ai/uaibook.htm> -- a relatively new field where few statisticians have overcome their fear to tread. Google is heading full bore into UAI with its recent $500M purchase of Deep Mind (although you won't find any open admission that they are pursuing Hutter's AIXI -- it is apparent they are doing so from Shane Legg's Singularity Summit presentation he gave prior to founding Deep Mind <https://www.youtube.com/watch?v=V6umr1OP8uo>).
On Wed, May 27, 2015 at 4:10 PM, Orionworks - Steven Vincent Johnson < [email protected]> wrote: > I have to agree in principle with a lot of what Jed and Dave have to say > on the matter. It baffles me that anyone would insert clippings of > previously recorded data over missing gaps. I’m assuming Parkhomov > performed this action because he didn’t think it was that big of a deal, > but that’s just a guess on my part. It’s my understanding that Parkhomov > inserted previously recorded data that he believed would more or less > represent with what the actual data readings would have revealed if he had > been able to record data during the missing moments. Unfortunately > performing such an action pretty much violates one of the most fundamental > principles of collecting data scientifically & objectively, without > personal bias involved. Inserting token data serves no useful purpose. The > lessor evil of performing such an action is that it obfuscates the accuracy > of the real data. The greater evil is that it immediately calls into deep > question the integrity of the individual as well as his data. > > > > Parkhomov does appear to understand the fact that what he did was not a > smart thing to have done. He states ""...It, of course a great sin and I > sincerely repent." > > > > https://drive.google.com/file/d/0Bz7lTfqkED9WWEtKZHVvczhsakk/view > > > > Parkhomov further states: > > > > > I send you the Excel file with the data obtained during experiment on > which pauses > > > in registration of temperature are designated by admissions of rows. > > > Once again I admire your sharp observation and high professionalism. > > > I hope that this incident won't make the attitude towards me and my > researches hostile. > > > > > > Alexander Parkhomov > > > > I think we should not lose site of the fact that Parkhomov seems to want > to make amends. Offering to send the unedited Excel file was the right > thing to do. A charlatan or quack would have initiated evasive maneuvers. I > certainly don’t get the impression Parkhomov attempted to dodge the > confrontation once the duplicated artifacts were brought to his attention. > Nor do I get the impression Parkhomov was at any time deliberately trying > to manipulate the outcome of his results. The whole affair strikes me as an > innocent action. Stupid, but an innocent action nevertheless. > > > > Regards, > > Steven Vincent Johnson > > svjart.orionworks.com > > zazzle.com/orionworks >
