Now that the holiday rush has subsided I have taken some time to read this thread and do some investigation beyond the email content, as I expect some others have. Several points that are made in the response*, supposedly from David, seem unwarranted and/or out of place. What purpose does mentioning highly public names such as Kevin Mitnick and Robert Morris serve? Likewise, what purpose does mentioning smpCTF serve? smpCTF, WTF is that? Google easily helps, but really? Why even mention it, simply to mislead those of us that have only seen CTF at DEFCON?**
One rebuttal comment brought up the longstanding argument that academics typically don't solve real world problems and those of us that work in the real world are typically shunned by academia. This paper specifically states that they target a practical problem space. If this is indeed true, why does the response not even address Dave's simple classroom example? They have acknowledged existence of the thread - why are they not addressing any of the concerns raised? Two whole 0-days! Wow! Are they even remotely useful? Are they ... never mind, not even worth enumerating. Forward symbolic execution? Really? At some point you really need to stop regurgitating your own thesis work, and increasing your publication and citation count by having students re-tool said work for conferences[1]. Finally, releases are mentioned. Where is this software? Are these releases in the common academic vernacular: as in a commit exists in some SVN repo somewhere with a the comment containing the string "release", and the public will likely never see them. I applaud Dave, Sean and others that have similarly called BS on this paper. Now if only academia would accept some of you into their circles maybe we would see a truly technical program committee that would reject such a paper. Alas, this publication and later presentation will surely be a large win for the students who will use the publication as a requirement for degrees and for the junior faculty member who is certainly targeting a quantity over quality approach toward tenure. Do list members think that this research group[2] is ill intentioned, or do they honestly believe that they are making a positive impact ... or even forward progress? Gave up on academic research long ago, Kevin * Calling it a response is fairly gracious. If the authors of the paper are acknowledging questions that have arisen on this mailing list, surely CMU faculty and students are capable of figuring out how to join the list, if needed, and reply instead of creating a small html page that will never have a decent page rank and essentially has no public audit. Offering to address concerns in person in Pittsburgh? Really? I am certain this is some sort of humor, I can't figure out exactly how it is supposed to be funny, but it surely isn't how a CMU professor cops out of addressing public criticism. ** at first glance the group seems to have a solid track record in CTF competitions. Until you dig a little deeper and find out that the ones they have one are basically substandard and many of them are small competitions run in Korea. Is it mere coincidence that David's group includes several Korean exchange students? To be fair, the group apparently did recently win iCTF in December which I gather actually is an accomplishment. [1] http://oakland31.cs.virginia.edu/slides/thanassis_oakland10.pdf [2] http://security.ece.cmu.edu/people.html On Thu, Dec 16, 2010 at 5:02 PM, Sean Heelan <[email protected]> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I think 'lets create a list of real world problems for academics to > consider' is missing the point somewhat. The problem here isn't that AEG > isn't a worthy or difficult problem, it is. The problem is that in order > to work realistically on AEG (and follow up with claims of changing > threat models etc) you need a good working knowledge of real world > exploit writing. The general impression I get from some of the research > groups working on the problem is they are unwilling to invest the time > required to gain this knowledge. (To kick a dead horse some more, one > should feel free to play around in a sandbox of vulnerabilities and > protection mechanisms from the past decade. It's obviously necessary for > tool development and trying out new ideas. The problem starts when you > forget half-way through that you're playing within a sandbox and pretend > it's the real world in your paper). > > Instead of a list of problems to attack how about list of real world > tasks that the authors should be able to complete manually before > deciding to automate the process? I would have thought this was a pretty > sensible thing that most people would do but apparently not. Trying to > automate a process that you only have a vague idea of doesn't sound like > something that is every going to go too well. > > Its very easy to run off down the path of an under-explored research > problem (I should know, I've done it :P), it's a lot less glamorous in > academia to spend weeks/months sitting in front of a debugger in order > to figure out the subtleties of the problem you are actually addressing. > This can hardly be considered an excessive request if one is working at > a well funded research group at a respected university though. Without > putting in this effort then the research output and paper quality have a > ceiling in terms of real-world applicability that will never be broken > through and the disparity between claims and facts will continue to > induce ong-winded and 'venomous' (chuckle) blog posts. > > (Btw, at some point during the discussion a few people began to assume I > was criticising all of academia. This isn't the case. I was pretty > specific in my initial blog post (http://bit.ly/ikvR0y) where my issues > lay and they are with small proportion of the overall research output of > academia and industry. There is no 'us vs them' here. I would expect > anyone writing a paper to at least have a cursory understanding of > everything they discuss. Furthermore, I wasn't criticising the people > cited in the paper when I suggested less nepotism in citations would be > useful. I was suggesting that the papers authors perhaps read something > like Phrack, or Uninformed a little more extensively than 'Smashing the > Stack for Fun and Profit'. > > My general opinion is that academia is both necessary and useful. That's > partially why I wrote the blog post to begin with - the paper is a > perfect example of the stereotype many have of academics as people with > their heads in the clouds dictating to those with their feet on the > ground. I know this isn't true for many so it's annoying when someone > comes along and proves it correct. > > It's also worth mentioning that CMU's response "If Mr. Heelan feels > there are real scientific issues to discuss, he is welcome to call or > visit us at CMU to discuss them." conveniently ignores the fact that I > did send them an email outlining (yet again) both my issues with the > paper and some technical issues. I received one reply requesting a phone > call instead of email, which I declined as real-time conversations on > technical matters tend to miss a lot IMO, and then never heard anything > back. No feedback on their heap claims, nothing on their stack fix-ups, > nothing on their plans to scale to modern bugs/exploits and no response > to any of the valid complaints raised here. The only point of their > response.html [1] seems to be to foster the image that my complaints > stem from an anti-academia sentiment instead of engaging on the issues > raised here and elsewhere. Hardly the most common way for a research > group to deal with questions and comments from a pretty sizeable > proportion of their target audience.) > > In hopeful expectation of a productive discussion (or failing that a > link or two to some funny cat videos), > > Sean > > [1] http://security.ece.cmu.edu/aeg/response.html > > On 12/15/2010 08:54 AM, Miles Fidelman wrote: > > Anton Chuvakin wrote: > >>> I would love to see a resource for real-world problems that the > academic > >>> community could consider... or even a resource for other up-and-coming > >>> researchers to examine at for ideas. Such a site might not be relevant > >>> enough for PhD thesis work (which thrives on originality as I > understand > >>> it?) > >>> > >> Well, if it is created by the industry, the academics will ignore it. > >> And if created by academics, well, see discussion in this thread. > >> > > Call me cynical but.... > > > > If it has serious commercial potential, academics may be doing the > > research, but saving the results for their side/spinout companies. > > > > The really interesting research (or least the well funded-research) gets > > funded by DoD, with classified results, and never gets published. > > > > And folks who have serious countermeasures to large spambot networks > > might just not want their names visible to the unsavory characters who > > run large spambot networks. > > > > Now a list of relevant problems to research would be interesting, but I > > expect there will be little feedback as to which problems people end up > > taking on. > > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.11 (GNU/Linux) > Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ > > iQEcBAEBAgAGBQJNCkYoAAoJEMW6jFWLazyQbSMH/3azWDftzsCwVs3H3xvO8YW9 > OII+v+fa20Jpkqh/KtSx9g4AjvootsxahTXv5e0pqqOIsRwQkP+eemC9xcDs/Kk/ > BhGnIyvz54tANy2/TgKQZwLTPvkbICfbtyP7gQCr9rKk9DJaC7SyEcKjBDdaDEGF > jBFXFufjQZqpcF8kYOE7c5sLqYp2Lsfy/Kzroa4lKeoQFDyp5MjMTWzLqzULcRFl > zfdt8jNbZR3iAGYJdzbhPSFRsfseI69UOKsXLuZwGJUvNThDkyOpvlguqjkYwJ8J > THQ6ULvNuPLiSLbFLYKPI2KDYMWgjF1DXjswjmfSN8/Zv9RF10K5+n0hBlfp3r8= > =Wfg6 > -----END PGP SIGNATURE----- > _______________________________________________ > Dailydave mailing list > [email protected] > https://lists.immunityinc.com/mailman/listinfo/dailydave >
_______________________________________________ Dailydave mailing list [email protected] https://lists.immunityinc.com/mailman/listinfo/dailydave
