Machine Learning Journal: Special Issue on Data Mining Lessons Learned http://www.hpl.hp.com/personal/Tom_Fawcett/DMLL-MLJ-CFP.html
Guest editors: Nada Lavrac, Hiroshi Motoda and Tom Fawcett Submission deadline: Monday, 7 April, 2003. Call for Papers Data mining is concerned with finding interesting or valuable patterns in data. Many techniques have emerged for analyzing and visualizing large volumes of data, and what we see in the technical literature are mostly success stories of these techniques. We rarely hear of steps leading to success, failed attempts, or critical representation choices made; and rarely do papers include expert evaluations of achieved results. Insightful analyses of successful and unsuccessful applications are crucial for increasing our understanding of machine learning techniques and their limitations. Challenge problems (such as the KDD Cup, COIL and PTE challenges) have become popular in recent years and have attracted numerous participants. These challenge problems usually involve a single difficult problem domain, and participants are evaluated by how well their entries satisfy a domain expert. The results of such challenges can be a useful source of feedback to the research community. At ICML-2002 a workshop on Data Mining Lessons Learned was held and (http://www.hpl.hp.com/personal/Tom_Fawcett/DMLL-workshop.html) and was well attended. This special issue of the Machine Learning journal follows the main goals of that workshop, which are to gather experience from successful and unsuccessful data mining endeavors, and to extract the lessons learned from them. Goals The aim of this special issue is to collect the experience gained from data mining applications and challenge competitions. We are interested in lessons learned both from successes and from failures. Authors are invited to report on experiences with challenge problems, experiences in engineering representations for practical problems, and in interacting with experts evaluating solutions. We are also interested in why some particular solutions - despite good performance - were not used in practice, or required additional treatment before they could be used. An ideal contribution to this special issue would describe in sufficient detail one problem domain, either an application or a challenge problem. Contributions not desired for this special issue would be papers that report on marginal improvement over existing methods using artificial synthetic data or UCI data involving no expert evaluation. We offer the following content guidelines to authors. 1. For applications studies, we expect a description of the attempts that succeeded or failed, an analysis of the success or failure, and any steps that had to be taken to make the results practically useful (if they were). Ideally an article should support lessons with evidence, experimental or otherwise; and the lessons should generalize to a class of problems. 2. For challenge problems, we will accept either experiences preparing an individual entry or an analysis of a collection of entries. A collective study might analyze factors such as the features of successful approaches that made them appealing to experts. As with applications studies, such articles should support lessons with evidence, and preferably should generalize to a class of problems. Analyses should preferably shed light on why a certain class of method is best applicable to the type of problem addressed. 3. A submission may analyze methodological aspects from individual developments, or may analyze a subfield of machine learning or a set of data mining methods to uncover important and unknown properties of a class of methods or a field as a whole. Again, a paper should support lessons learned with appropriate evidence. We emphasize that articles to appear in this special issue must satisfy the high standards of the Machine Learning journal. Submissions will be evaluated on the following criteria: Novelty: How original is this lesson? Is this the first time this observation has been made, or has it appeared before? Generality: How widely applicable are the observations or conclusions made by this paper? Are they specific to a single project, a single domain, a class of domains, or much of data mining? Significance: How important are the lessons learned? Are they actionable? To what extent could they influence the directions of work in data mining? Support: How strong is the experimental evidence? Are the lessons drawn from a single project, a group of projects, or a thread of work in the community? Clarity: How clear is the paper? How clearly are the lessons expressed? The criteria for novelty, significance and clarity apply not only to the lessons but also to the paper as a whole. Submission Instructions Manuscripts for submission should be prepared according to the instructions at http://www.cs.ualberta.ca/~holte/mlj/ In preparing submissions, authors should follow the standard instructions for the Machine Learning journal at http://www.cs.ualberta.ca/~holte/mlj/initialsubmission.pdf Submissions should be sent via email to Hiroshi Motoda ([EMAIL PROTECTED]), as well as to Kluwer Academic Publishers ([EMAIL PROTECTED]). In the email please state very clearly that the submission is for the special issue on Data Mining Lessons Learned.
