That's completely right. The use case is more for restarting a failed job rather than configuring the pipeline. You "really" want to do something different like piece together your own job.
This could be as complex as we want -- it could be its own project, defining a slightly-higher-level definition language for MR. In fact there are already one or two like that. I like the idea... somehow I think you'll find it hard to implement across all the jobs since they're not even all in the same "format" at this point! On Thu, Sep 1, 2011 at 7:43 PM, Grant Ingersoll <[email protected]> wrote: > Other than opening the code and looking, is there a way we register our > phases such that one could, via the command line, know what they are? For > instance, I think, for now, I can skip, in my application, the first two > phases of the RecommenderJob, but it seems a bit awkward to say --startPhase > 2 given that at some point in a new release a new phase could be added in > and I would then have to go check the code. Not the end of the world, but > it seems error prone and not readily maintainable. I suppose as a bonus, > it would be nice if one could also know where each phase expects things to > be and in what format. Would it make sense to have the equivalent of > prepareJob that does registerJob up front and can then be dumped out so that > one could see the phases and their inputs, etc? > > -Grant > > -------------------------------------------- > Grant Ingersoll > http://www.lucidimagination.com > Lucene Eurocon 2011: http://www.lucene-eurocon.com > >
