Hi all,
I'm working to make the CI experience with python a bit better, and my
current initiative is splitting up the giant Python PreCommit job into 5
separate jobs into separate jobs for Lint, Py2, Py3.5, Py3.6, and Py3.7.
Around 11am Pacific time tomorrow I'm going to initiate the seed jobs,
Agreeing with many things here and my own flavor to the points:
1. User's privacy is more important than anything else
2. The goal should be to make things better for users
3. Trading user's opt-in for functionality (like Gradle scans) is not
acceptable
4. It should be effectively invisible to
Hi Hannah,
I believe this is unexpected from the developer perspective. When building
something locally, we do expect that to be used. We may need to change to
not pull when the image is available locally, at least when it is a
snapshot/master branch. Release images should be immutable anyways.
Yeah, that's useful. I was asking about getting things at the jenkins
job level. E.g. are our PostCommits taking up all the time, or our
Precommits?
On Tue, Sep 24, 2019 at 1:23 PM Lukasz Cwik wrote:
>
> We can get the per gradle task profile with the --profile flag:
>
Creating a central place for collecting Beam usage sounds compelling, but
we'd have to be careful about several aspects:
- It goes without saying that this can never be on-by-default, even for a
tiny fraction of pipelines.
- For further privacy protection, including the user's PipelineOptions is
A minor update, with custom container, the pipeline would not fail, it
throws out warning and moves on to `docker run` command.
On Tue, Sep 24, 2019 at 4:05 PM Hannah Jiang wrote:
> Hi Brian
>
> If we pull docker images, it always downloads from remote repository,
> which is expected behavior.
Hi Brian
If we pull docker images, it always downloads from remote repository, which
is expected behavior.
In case we want to run a local image and pull it only when the image is not
available at local, we can use `docker run` command directly, without
pulling it in advance. [1]
In case we want
Looks like this is Docker's expected behavior [1]. For Python, this isn't a
problem because the default tag is `2.17.0.dev`, not `latest`. Not sure why
that's different for Java? Anyway, according to this comment:
'Note that docker pull will not *overwrite* your local image, it will
"untag" it
One of the options could be to just display the URL and not to phone home.
I would like it so that users can integrate this into their deployment
solution so we get regular stats instead of only when a user decides to run
a pipeline manually.
On Tue, Sep 24, 2019 at 11:13 AM Robert Bradshaw
Not to my knowledge. Maybe something is down.
Have you tried running a gradle build locally with --scan?
On Tue, Sep 24, 2019 at 1:03 PM Valentyn Tymofieiev
wrote:
> For example, https://gradle.com/s/mpfu3wpz2xfwe says: Your build scan
> could not be displayed.
>
We can get the per gradle task profile with the --profile flag:
https://jakewharton.com/static/files/trace/profile.html
This information also appears within the build scans that are sent to
Gradle.
Integrating with either of these sources of information would allow us to
figure out whether its
For example, https://gradle.com/s/mpfu3wpz2xfwe says: Your build scan
could not be displayed.
Does anyone know how to gather stats on where the time is being spent?
Several times the idea of consolidating many of the (expensive)
validates runner integration tests into a single pipeline, and then
running things individually only if that fails, has come up. I think
that'd be a big win if
Those ideas all sound good. I especially agree with trying to reduce tests
first and then if we've done all we can there and latency is still too
high, it means we need more workers. Also in addition to reducing the
amount of tests, there's also running less important tests less frequently,
I'm working on a demo cross-language pipeline on a local flink cluster that
relies on my python row coder PR [1]. The PR includes some changes to the
Java worker code, so I need to build a Java SDK container locally and use
that in the pipeline.
Unfortunately, whenever I run the pipeline, the
I'm with Luke on this. We can add a set of flags to send home stats and
crash dumps if user agrees. If we keep code isolated, it will be easy
enough for user to check what is being sent.
One more heavy-weight option is to also allow user configure and persist
what information he is ok with
Why not add a flag to the SDK that would do the phone home when specified?
>From a support perspective it would be useful to know:
* SDK version
* Runner
* SDK provided PTransforms that are used
* Features like user state/timers/side inputs/splittable dofns/...
* Graph complexity (# nodes, #
I don’t see any updates on my calendar. Does it work for others?
> On 19 Sep 2019, at 17:16, Ismaël Mejía wrote:
>
> 25/09 looks ok. I just updated the meeting invitation to the new
> date.I will prepare a mini agenda in the shared minute document in the
> meantime.
> I cannot see the old
Done, Welcome!
On Tue, Sep 24, 2019 at 11:17 AM Kishor Joshi wrote:
>
> Hi,
>
> This is kishor joshi.
> Can someone add me as a contributor for Beam's Jira issue tracker? I would
> like to create/assign tickets for my work.
>
> My JIRA id is kishorjoshi.
>
> Thanks & regards,
> Kishor
>
Hi Ismael,
Can you also add me to this meeting. I would also like to contribute.
Regards,
Vishwas
On 2019/09/19 12:13:40, Etienne Chauchot wrote:
> Hi Rahul and Xinyu,I just added you to the list of guests in the meeting.
> Time is 5pm GMT +2. That being said, for some
> reason last meeting
Hi,
This is kishor joshi. Can someone add me as a contributor for Beam's Jira issue
tracker? I would like to create/assign tickets for my work.
My JIRA id is kishorjoshi.
Thanks & regards,Kishor
21 matches
Mail list logo
