wonook commented on a change in pull request #163: [NEMO-294] Beam-Runner
URL: https://github.com/apache/incubator-nemo/pull/163#discussion_r237151236
##########
File path: client/src/main/java/org/apache/nemo/client/JobLauncher.java
##########
@@ -101,84 +101,100 @@ private JobLauncher() {
* @throws Exception exception on the way.
*/
public static void main(final String[] args) throws Exception {
+ try {
+ setup(args);
+ // Launch client main. The shutdown() method is called inside the
launchDAG() method.
+ runUserProgramMain(builtJobConf);
+ } catch (final InjectionException e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ /**
+ * Set up the driver, etc. before the actual execution.
+ * @param args arguments.
+ * @throws InjectionException injection exception from REEF.
+ * @throws ClassNotFoundException class not found exception.
+ * @throws IOException IO exception.
+ */
+ public static void setup(final String[] args) throws InjectionException,
ClassNotFoundException, IOException {
// Get Job and Driver Confs
builtJobConf = getJobConf(args);
// Registers actions for launching the DAG.
LOG.info("Launching RPC Server");
driverRPCServer = new DriverRPCServer();
driverRPCServer
-
.registerHandler(ControlMessage.DriverToClientMessageType.DriverStarted, event
-> {
- })
- .registerHandler(ControlMessage.DriverToClientMessageType.DriverReady,
event -> driverReadyLatch.countDown())
-
.registerHandler(ControlMessage.DriverToClientMessageType.ExecutionDone, event
-> jobDoneLatch.countDown())
-
.registerHandler(ControlMessage.DriverToClientMessageType.DataCollected,
message -> COLLECTED_DATA.addAll(
-
SerializationUtils.deserialize(Base64.getDecoder().decode(message.getDataCollected().getData()))))
- .run();
+ .registerHandler(ControlMessage.DriverToClientMessageType.DriverStarted,
event -> {
+ })
+ .registerHandler(ControlMessage.DriverToClientMessageType.DriverReady,
event -> driverReadyLatch.countDown())
+ .registerHandler(ControlMessage.DriverToClientMessageType.ExecutionDone,
event -> jobDoneLatch.countDown())
+ .registerHandler(ControlMessage.DriverToClientMessageType.DataCollected,
message -> COLLECTED_DATA.addAll(
+
SerializationUtils.deserialize(Base64.getDecoder().decode(message.getDataCollected().getData()))))
+ .run();
final Configuration driverConf = getDriverConf(builtJobConf);
final Configuration driverNcsConf = getDriverNcsConf();
- final Configuration driverMessageConfg = getDriverMessageConf();
+ final Configuration driverMessageConfig = getDriverMessageConf();
+ final String defaultExecutorResourceConfig =
"[{\"type\":\"Transient\",\"memory_mb\":512,\"capacity\":5},"
Review comment:
When making a JAR file, and executing the JAR, sadly the json file is
doesn't get included in the JAR. I've tried refactoring the code into
`src/main/resources` directory, but it didn't work. This change was made to
make it possible to apply a default `executor.json` even when it is not
explicitly specified.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
