Doug Cutting wrote:
Frédéric Bertin wrote:
*// Set the user's name and working directory*
String user = System.getProperty("user.name");
job.setUser(user != null ? user : "Dr Who");
if (job.getWorkingDirectory() == null) {
job.setWorkingDirectory(fs.getWorkingDirectory());
}
This should run clientside, since it depends on the username, which is
different on the server.
then, what about passing the username as a parameter to the
JobSubmissionProtocol.submitJob(...) ? This avoids loading the whole
JobConf clientside just to set the username.
FileSystem userFileSys = FileSystem.get(job);
Path[] inputDirs = job.getInputPaths();
boolean[] validDirs =
job.getInputFormat().areValidInputDirectories(userFileSys, inputDirs);
for(int i=0; i < validDirs.length; ++i) {
if (!validDirs[i]) {
String msg = "Input directory " + inputDirs[i] +
" in " + userFileSys.getName() + " is
invalid.";
LOG.error(msg);
throw new IOException(msg);
}
}
*// Check the output specification*
job.getOutputFormat().checkOutputSpecs(fs, job);
Why not moving it in the JobSubmissionProtocol (JobTracker's
submitJob method) ?
These could probably run on the server. They're currently run on the
client in an attempt to return errors as quickly as possible when jobs
are misconfigured.
Is it really quicker to make all those checkings remotely than remotely
asking the JobTracker to make them locally? (just a question, I really
have no idea of the answer)
Fred
