Ah didn't see your pull request yet Thomas. Will take a look later.
On Mon, Nov 25, 2019 at 10:23 AM Thomas Weise wrote:
> Thanks, I would prefer to solve this in a way where the user does not need
> to configure anything extra though.
>
>
> On Mon, Nov 25, 2019 at 10:21 AM Kyle Weaver wrote:
Thanks, I would prefer to solve this in a way where the user does not need
to configure anything extra though.
On Mon, Nov 25, 2019 at 10:21 AM Kyle Weaver wrote:
> When we added the class loader artifact stager, we introduced artifact
> retrieval service type as a pipeline option. It would
When we added the class loader artifact stager, we introduced artifact
retrieval service type as a pipeline option. It would make sense to put a
"none" option there.
boot.go could be updated to recognize NO_ARTIFACTS_STAGED_TOKEN as
well. (Should this constant be put in a common location?)
On Sat, Nov 23, 2019 at 9:16 AM Thomas Weise wrote:
>
> JIRA: https://issues.apache.org/jira/browse/BEAM-8815
>
>
> On Fri, Nov 22, 2019 at 5:31 PM Thomas Weise wrote:
>>
JIRA: https://issues.apache.org/jira/browse/BEAM-8815
On Fri, Nov 22, 2019 at 5:31 PM Thomas Weise wrote:
> I'm running into the issue Kyle points out when I try to run a pipeline
> that does not use artifact staging:
>
> 2019-11-23 01:09:18,442 WARN
>
I'm running into the issue Kyle points out when I try to run a pipeline
that does not use artifact staging:
2019-11-23 01:09:18,442 WARN
org.apache.beam.runners.fnexecution.artifact.AbstractArtifactRetrievalService
- GetManifest for
FWIW, there are also discussions of adding a preparation phase for sdk
harness (docker) images, such that artifacts could be staged (and
installed, compiled etc.) ahead of time and shipped as part of the sdk
image rather than via a side channel (and on every worker). Anyone not
using these images
Certainly there's a lot to be re-thought in terms of artifact staging,
especially when it comes to cross-langauge pipelines. I think it would
makes sense to have a special retrieval token for the "empty"
manifest, which would mean a staging directory would never have to be
set up if no artifacts
