[jira] [Commented] (BEAM-5288) Modify Environment to support non-dockerized SDK harness deployments
[
https://issues.apache.org/jira/browse/BEAM-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16614240#comment-16614240
]
Ankur Goenka commented on BEAM-5288:
We are thinking of removing the args as of now to avoid any confusion and
potential name collision. All the relevant information can be conveyed to the
SDKHarness using the Environment.
> Modify Environment to support non-dockerized SDK harness deployments
> -
>
> Key: BEAM-5288
> URL: https://issues.apache.org/jira/browse/BEAM-5288
> Project: Beam
> Issue Type: New Feature
> Components: beam-model
>Reporter: Maximilian Michels
>Assignee: Ankur Goenka
>Priority: Major
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> As of mailing discussions and BEAM-5187, it has become clear that we need to
> extend the Environment information. In addition to the Docker environment,
> the extended environment holds deployment options for 1) a process-based
> environment, 2) an externally managed environment.
> The proto definition, as of now, looks as follows:
> {noformat}
> message Environment {
>// (Required) The URN of the payload
>string urn = 1;
>// (Optional) The data specifying any parameters to the URN. If
>// the URN does not require any arguments, this may be omitted.
>bytes payload = 2;
> }
> message StandardEnvironments {
>enum Environments {
> DOCKER = 0 [(beam_urn) = "beam:env:docker:v1"];
> PROCESS = 1 [(beam_urn) = "beam:env:process:v1"];
> EXTERNAL = 2 [(beam_urn) = "beam:env:external:v1"];
>}
> }
> // The payload of a Docker image
> message DockerPayload {
>string container_image = 1; // implicitly linux_amd64.
> }
> message ProcessPayload {
>string os = 1; // "linux", "darwin", ..
>string arch = 2; // "amd64", ..
>string command = 3; // process to execute
>map env = 4; // environment variables
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (BEAM-5288) Modify Environment to support non-dockerized SDK harness deployments
[
https://issues.apache.org/jira/browse/BEAM-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16614237#comment-16614237
]
Henning Rohde commented on BEAM-5288:
-
If we're reusing the container contract (which I think is reasonable), then
these extra proto args would have to come either before or after the
runner-provided args. Having just the environment avoids that potential
semantic confusion. I was originally thinking that the program would be
executed in a (OS dependent) '/bin/bash -c ""' fashion and defer any
escaping to the sdk, but I have no problem with more explicit args as long as
the semantics is clear.
> Modify Environment to support non-dockerized SDK harness deployments
> -
>
> Key: BEAM-5288
> URL: https://issues.apache.org/jira/browse/BEAM-5288
> Project: Beam
> Issue Type: New Feature
> Components: beam-model
>Reporter: Maximilian Michels
>Assignee: Ankur Goenka
>Priority: Major
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> As of mailing discussions and BEAM-5187, it has become clear that we need to
> extend the Environment information. In addition to the Docker environment,
> the extended environment holds deployment options for 1) a process-based
> environment, 2) an externally managed environment.
> The proto definition, as of now, looks as follows:
> {noformat}
> message Environment {
>// (Required) The URN of the payload
>string urn = 1;
>// (Optional) The data specifying any parameters to the URN. If
>// the URN does not require any arguments, this may be omitted.
>bytes payload = 2;
> }
> message StandardEnvironments {
>enum Environments {
> DOCKER = 0 [(beam_urn) = "beam:env:docker:v1"];
> PROCESS = 1 [(beam_urn) = "beam:env:process:v1"];
> EXTERNAL = 2 [(beam_urn) = "beam:env:external:v1"];
>}
> }
> // The payload of a Docker image
> message DockerPayload {
>string container_image = 1; // implicitly linux_amd64.
> }
> message ProcessPayload {
>string os = 1; // "linux", "darwin", ..
>string arch = 2; // "amd64", ..
>string command = 3; // process to execute
>map env = 4; // environment variables
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (BEAM-5288) Modify Environment to support non-dockerized SDK harness deployments
[
https://issues.apache.org/jira/browse/BEAM-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16611413#comment-16611413
]
Ankur Goenka commented on BEAM-5288:
It will be good to expose args separately in Process Environment to make sure
we escape them correctly when we issue the command.
message ProcessPayload {
string os = 1; // "linux", "darwin", ..
string arch = 2; // "amd64", ..
string command = 3; // process to execute
map env = 4; // environment variables
}
==>>
message ProcessPayload {
string os = 1; // "linux", "darwin", ..
string arch = 2; // "amd64", ..
string command = 3; // process to execute
repeated string args = 4; // Arguments to the command
map env = 5; // Environment variables
}
> Modify Environment to support non-dockerized SDK harness deployments
> -
>
> Key: BEAM-5288
> URL: https://issues.apache.org/jira/browse/BEAM-5288
> Project: Beam
> Issue Type: New Feature
> Components: beam-model
>Reporter: Maximilian Michels
>Assignee: Maximilian Michels
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> As of mailing discussions and BEAM-5187, it has become clear that we need to
> extend the Environment information. In addition to the Docker environment,
> the extended environment holds deployment options for 1) a process-based
> environment, 2) an externally managed environment.
> The proto definition, as of now, looks as follows:
> {noformat}
> message Environment {
>// (Required) The URN of the payload
>string urn = 1;
>// (Optional) The data specifying any parameters to the URN. If
>// the URN does not require any arguments, this may be omitted.
>bytes payload = 2;
> }
> message StandardEnvironments {
>enum Environments {
> DOCKER = 0 [(beam_urn) = "beam:env:docker:v1"];
> PROCESS = 1 [(beam_urn) = "beam:env:process:v1"];
> EXTERNAL = 2 [(beam_urn) = "beam:env:external:v1"];
>}
> }
> // The payload of a Docker image
> message DockerPayload {
>string container_image = 1; // implicitly linux_amd64.
> }
> message ProcessPayload {
>string os = 1; // "linux", "darwin", ..
>string arch = 2; // "amd64", ..
>string command = 3; // process to execute
>map env = 4; // environment variables
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (BEAM-5288) Modify Environment to support non-dockerized SDK harness deployments
[
https://issues.apache.org/jira/browse/BEAM-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605603#comment-16605603
]
Maximilian Michels commented on BEAM-5288:
--
[~lcwik] +1 {{SupportedEnvironments}} is better. Thanks for the adjustments.
[~herohde] Params would be additional arguments after the container contract
parameters. IMHO can be removed since the semantics are not clear, e.g. doesn't
work if the process expects positional parameters. We could add environment
variables instead as a way to pass parameters to the process.
> Modify Environment to support non-dockerized SDK harness deployments
> -
>
> Key: BEAM-5288
> URL: https://issues.apache.org/jira/browse/BEAM-5288
> Project: Beam
> Issue Type: New Feature
> Components: beam-model
>Reporter: Maximilian Michels
>Priority: Major
>
> As of mailing discussions and BEAM-5187, it has become clear that we need to
> extend the Environment information. In addition to the Docker environment,
> the extended environment holds deployment options for 1) a process-based
> environment, 2) an externally managed environment.
> The proto definition, as of now, looks as follows:
> {noformat}
> message Environment {
>// (Required) The URN of the payload
>string urn = 1;
>// (Optional) The data specifying any parameters to the URN. If
>// the URN does not require any arguments, this may be omitted.
>bytes payload = 2;
> }
> message StandardEnvironments {
>enum Environments {
> DOCKER = 0 [(beam_urn) = "beam:env:docker:v1"];
> PROCESS = 1 [(beam_urn) = "beam:env:process:v1"];
> EXTERNAL = 2 [(beam_urn) = "beam:env:external:v1"];
>}
> }
> // The payload of a Docker image
> message DockerPayload {
>string container_image = 1; // implicitly linux_amd64.
> }
> message ProcessPayload {
>string os = 1; // "linux", "darwin", ..
>string arch = 2; // "amd64", ..
>string command = 3; // process to execute
>repeated string params = 4; // parameters
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (BEAM-5288) Modify Environment to support non-dockerized SDK harness deployments
[
https://issues.apache.org/jira/browse/BEAM-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603454#comment-16603454
]
Henning Rohde commented on BEAM-5288:
-
Clarifying question [~mxm] [~angoenka]: what is the semantics of the params
field in ProcessPayload? The SDK harness will need to know the id and endpoints
from the runner, which is why I suggested to re-use the container contract
invocation here.
> Modify Environment to support non-dockerized SDK harness deployments
> -
>
> Key: BEAM-5288
> URL: https://issues.apache.org/jira/browse/BEAM-5288
> Project: Beam
> Issue Type: New Feature
> Components: beam-model
>Reporter: Maximilian Michels
>Priority: Major
>
> As of mailing discussions and BEAM-5187, it has become clear that we need to
> extend the Environment information. In addition to the Docker environment,
> the extended environment holds deployment options for 1) a process-based
> environment, 2) an externally managed environment.
> The proto definition, as of now, looks as follows:
> {noformat}
> message Environment {
>// (Required) The URN of the payload
>string urn = 1;
>// (Optional) The data specifying any parameters to the URN. If
>// the URN does not require any arguments, this may be omitted.
>bytes payload = 2;
> }
> message StandardEnvironments {
>enum Environments {
> DOCKER = 0 [(beam_urn) = "beam:env:docker:v1"];
> PROCESS = 1 [(beam_urn) = "beam:env:process:v1"];
> EXTERNAL = 2 [(beam_urn) = "beam:env:external:v1"];
>}
> }
> // The payload of a Docker image
> message DockerPayload {
>string container_image = 1; // implicitly linux_amd64.
> }
> message ProcessPayload {
>string os = 1; // "linux", "darwin", ..
>string arch = 2; // "amd64", ..
>string command = 3; // process to execute
>repeated string params = 4; // parameters
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (BEAM-5288) Modify Environment to support non-dockerized SDK harness deployments
[
https://issues.apache.org/jira/browse/BEAM-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603274#comment-16603274
]
Luke Cwik commented on BEAM-5288:
-
I would suggest `StandardEnvironments` instead of `SupportedEnvironments`
because these URNs say that there is a standard that is/will be documented and
associated with it.
`Primitives` is also the wrong term. I would go with `Environments` unless
something better is suggested.
> Modify Environment to support non-dockerized SDK harness deployments
> -
>
> Key: BEAM-5288
> URL: https://issues.apache.org/jira/browse/BEAM-5288
> Project: Beam
> Issue Type: New Feature
> Components: beam-model
>Reporter: Maximilian Michels
>Priority: Major
>
> As of mailing discussions and BEAM-5187, it has become clear that we need to
> extend the Environment information. In addition to the Docker environment,
> the extended environment holds deployment options for 1) a process-based
> environment, 2) an externally managed environment.
> The proto definition, as of now, looks as follows:
> {noformat}
> message Environment {
>// (Required) The URN of the payload
>string urn = 1;
>// (Optional) The data specifying any parameters to the URN. If
>// the URN does not require any arguments, this may be omitted.
>bytes payload = 2;
> }
> message SupportedEnvironments {
>enum Primitives {
> DOCKER = 0 [(beam_urn) = "beam:env:docker:v1"];
> PROCESS = 1 [(beam_urn) = "beam:env:process:v1"];
> EXTERNAL = 2 [(beam_urn) = "beam:env:external:v1"];
>}
> }
> // The payload of a Docker image
> message DockerPayload {
>string container_image = 1; // implicitly linux_amd64.
> }
> message ProcessPayload {
>string os = 1; // "linux", "darwin", ..
>string arch = 2; // "amd64", ..
>string command = 3; // process to execute
>repeated string params = 4; // parameters
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
