With the recent removal of all the clustering Job classes we have
introduced a fatal bug in all of the synthetic control examples. In the
original implementation, the Jobs were responsible for deleting their
output directory prior to running the various Drivers which did not
delete output. In removing the clustering Jobs this deletion
responsibility was moved to the Drivers. Problem is, the synthetic
control examples transform the input file to the output/data directory
before calling the clustering Driver, which now also zaps output (and
thus it's input in this case) causing file not found errors. I see 4
possible solutions:
1. Reinstate the Job files, giving them the responsibility to delete
their output directory and removing that responsibility from all
Drivers.This will involve some code duplication in the Job and
Driver main methods which can be addressed by refactoring.
2. Leave the Drivers as-is and just remove their output deletion.
This puts a bit more burden on the user but makes constructing job
chains with clustering computations possible.
3. Modify synthetic control examples to use a different, non-output,
directory for the converted data and leave the Drivers alone.
4. Finally, since chains of clustering jobs usually call the driver's
static methods and not main, just move output deletion to main.
The rub here is that sequences of command-line invocations would
be problematic.
I'd like to move towards a consistent pattern across Mahout (the reason
for removing Jobs in the first place). I'm leaning towards #1 but would
like some feedback esp. from Sean and Robin who (iirc) started this ball
rolling.
