[
https://issues.apache.org/jira/browse/BEAM-8015?focusedWorklogId=301530&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-301530
]
ASF GitHub Bot logged work on BEAM-8015:
----------------------------------------
Author: ASF GitHub Bot
Created on: 26/Aug/19 21:37
Start Date: 26/Aug/19 21:37
Worklog Time Spent: 10m
Work Description: ibzib commented on pull request #9389: [BEAM-8015] Get
logs from Docker containers
URL: https://github.com/apache/beam/pull/9389#discussion_r317810021
##########
File path:
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/environment/DockerCommand.java
##########
@@ -134,10 +148,41 @@ public void killContainer(String containerId)
runShortCommand(Arrays.asList(dockerExecutable, "kill", containerId));
}
- /** Run the given command invocation and return stdout as a String. */
+ /**
+ * Removes docker container with container id.
+ *
+ * @throws IOException if an IOException occurs, or if the given container
id either does not
+ * exist or is still running
+ */
+ public void removeContainer(String containerId)
+ throws IOException, TimeoutException, InterruptedException {
+ checkArgument(containerId != null);
+ checkArgument(
+ CONTAINER_ID_PATTERN.matcher(containerId).matches(),
+ "Container ID must be a 64-character hexadecimal string");
+ runShortCommand(Arrays.asList(dockerExecutable, "rm", containerId));
+ }
+
private String runShortCommand(List<String> invocation)
throws IOException, TimeoutException, InterruptedException {
+ return runShortCommand(invocation, false, "");
+ }
+
+ /**
+ * Runs a command, blocks until {@link DockerCommand#commandTimeout} has
elapsed, then returns the
+ * command's output.
+ *
+ * @param invocation command and arguments to be run
+ * @param redirectErrorStream if true, redirect stderr of the process to its
stdout
+ * @param delimiter used for separating output lines
+ * @return stdout of the command, including stderr if {@code
redirectErrorStream} is true
+ * @throws TimeoutException if command has not finished by {@link
DockerCommand#commandTimeout}
+ */
+ private String runShortCommand(
+ List<String> invocation, boolean redirectErrorStream, CharSequence
delimiter)
+ throws IOException, TimeoutException, InterruptedException {
ProcessBuilder pb = new ProcessBuilder(invocation);
+ pb.redirectErrorStream(redirectErrorStream);
Review comment:
> I suppose my question is, how will we capture the output of the
runShortCommand when redirectErrorStream is set to true?
Here are all the cases:
- `!redirectErrorStream && exitCode == 0` Return only the stdout of the
command.
- `!redirectErrorStream && exitCode != 0` Throw an exception that includes
only the stderr of the command.
- `redirectErrorStream && exitCode == 0` Return the stdout and stderr of the
command.
- `redirectErrorStream && exitCode != 0` Throw an exception that includes
both the stdout and stderr of the command.
It's not as simple as I would like it to be, but I think this is the
behavior we want.
> If this simply redirects stderr to stdout, why couldn't we capture stderr
before?
Before, we were only using stderr if there was an error. I guess we wouldn't
want to include it in the return value of `runShortCommand` because irrelevant
warnings, etc. could potentially break the parsing of the result for some
commands.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 301530)
Time Spent: 2h 10m (was: 2h)
> Get logs for SDK worker Docker containers
> -----------------------------------------
>
> Key: BEAM-8015
> URL: https://issues.apache.org/jira/browse/BEAM-8015
> Project: Beam
> Issue Type: Improvement
> Components: java-fn-execution
> Reporter: Kyle Weaver
> Assignee: Kyle Weaver
> Priority: Major
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Currently, when SDK worker containers fail to start up properly, an exception
> is thrown that provides no information about what happened. We can improve
> debugging by keeping containers around long enough to log their logs before
> removing them.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
