Jenkins has no understanding of this. Those are just different job names
that our scripts create. Is it a bug in PreCommitJobBuilder?
Kenn
On Thu, Jan 24, 2019 at 11:22 AM Ankur Goenka wrote:
> It seems that the cron is not deleted if the job is deleted.
> I ran the seed job manually to test
My mistake - the problem was that the Dataflow container image was not yet
built, but I did not dig deep enough into StackDriver to find that message.
All issues are resolved and all PRs against the release branch are now
merged. I am now in process of building those containers and RC1.
Kenn
On
It seems that the cron is not deleted if the job is deleted.
I ran the seed job manually to test the new job which would have created
the cron.
I am not aware of jenkins internal but it will be great if we can clean up
the cron on job deletion.
On Thu, Jan 24, 2019 at 7:40 AM Thomas Weise
On Thu, Jan 24, 2019 at 2:38 PM Robert Bradshaw wrote:
> On Thu, Jan 24, 2019 at 6:43 PM Reuven Lax wrote:
> >
> > Keep in mind that these user-supplied lambdas are commonly used in our
> IOs. One common usage is in Sink IOs, to allow dynamic destinations. e.g.
> in BigQueryIO.Write, a
You have the permissions, please self assign the issue and welcome!
On Thu, Jan 24, 2019 at 5:22 PM Robert Collins
wrote:
>
> Hi,
>
> This is Robert from Fluidly. We are setting up a streaming pipeline which
> outputs to Redis. Can someone add me as a contributor for Beam's Jira issue
>
I wanted to give a heads-up that PR#7615 [1] replaces the 'visteg' Gradle
plugin used in our build to produce .dot-file representation of the task
build graph. The plugin [2] appears to be abandoned and is not compatible
with Gradle 5.0 [3]
The PR replaces it with a different plugin,
On Fri, Jan 25, 2019 at 12:18 AM Reuven Lax wrote:
>
> On Thu, Jan 24, 2019 at 2:38 PM Robert Bradshaw wrote:
>>
>> On Thu, Jan 24, 2019 at 6:43 PM Reuven Lax wrote:
>> >
>> > Keep in mind that these user-supplied lambdas are commonly used in our
>> > IOs. One common usage is in Sink IOs, to
I'm seeing every ≈third `./gradlew compileJava` fail locally due to this;
re-running the commit has always succeeded, so far.
Sounds like there is not an immediate fix in the works / no one assigned on
the JIRA?
On Wed, Jan 23, 2019 at 3:17 PM Kenneth Knowles wrote:
> This might connect to
job_PreCommit_Python_ValidatesRunner_Flink.groovy has [1]:
previousNames('beam_PostCommit_Python_VR_Flink')
This is used by Jenkin to rename an existing job to the new name in order
to retain history. In this case, both old a new Job names already exist, so
it's failing:
*
That's a good point that this "IO" time should be tracked differently.
For a single level, a wrapper/utility that correctly and completely
(and transparently) implements the "naive" bit I sketched above under
the hood may be sufficient and implementable purely in user-space, and
quite useful.
On
On Thu, Jan 24, 2019 at 6:43 PM Reuven Lax wrote:
>
> Keep in mind that these user-supplied lambdas are commonly used in our IOs.
> One common usage is in Sink IOs, to allow dynamic destinations. e.g. in
> BigQueryIO.Write, a user-supplied lambda determines what table a record
> should be
The subject line is a quote from BEAM-6324*
This makes me sad. I hope/expect it is a failure to route a pull request to
the right reviewer. I am less sad about the functionality than the
sentiment and how a contributor is being discouraged.
Does anyone have ideas that could help?
Kenn
Hi all,
Please join me and the rest of the Beam PMC in welcoming a new committer: Gleb
Kanterov
Gleb started contributing to Beam and quickly dove deep, doing some
sensitive fixes to schemas, also general build issues, Beam SQL, Avro, and
more. In consideration of Gleb's technical and community
Thanks Scott.
I removed beam_PreCommit_Python_PVR_Flink_XXX and hopefully that will
unblock the seed job.
On Thu, Jan 24, 2019 at 1:05 PM Scott Wegner wrote:
> job_PreCommit_Python_ValidatesRunner_Flink.groovy has [1]:
>
>previousNames('beam_PostCommit_Python_VR_Flink')
>
> This is used
Please try rebasing from master, I believe this issue has been resolved.
On Thu, Jan 24, 2019 at 3:29 PM Ryan Williams wrote:
> I'm seeing every ≈third `./gradlew compileJava` fail locally due to this;
> re-running the commit has always succeeded, so far.
>
> Sounds like there is not an
Hi all,
I’d like to take on BEAM-6207 issue, but I lack JIRA permissions to assign
tickets to myself
Can anyone add me as a contributor?
My username is mwalenia.
Thanks and have a great day
Michal
@Gleb, I'll also take a look at ExpressionEncoder thanks for the pointer to
typelevel/frameless.
Etienne
Le mercredi 23 janvier 2019 à 17:06 +0100, Etienne Chauchot a écrit :
> Hi all ,Thanks for your feedback! I was indeed thinking about Reuven's work
> around Schema PCollections, hence my
If I understand correctly, the end goal is to process input elements
of a DoFn asynchronously. Were I to do this naively, I would implement
DoFns that simply take and receive [Serializable?]CompletionStages as
element types, followed by a DoFn that adds a callback to emit on
completion (possibly
Done, welcome and enjoy!
On Thu, Jan 24, 2019 at 1:24 PM Michał Walenia
wrote:
>
> Hi all,
> I’d like to take on BEAM-6207 issue, but I lack JIRA permissions to assign
> tickets to myself
> Can anyone add me as a contributor?
> My username is mwalenia.
> Thanks and have a great day
>
> Michal
Hi,
This is Robert from Fluidly. We are setting up a streaming pipeline which
outputs to Redis. Can someone add me as a contributor for Beam's Jira issue
tracker? I'd like submit a pull request for an issue we had with flushing
bundles.
My Jira username is rob...@fluidly.com
Thanks
Robert
I think I agree with a lot of what you said here, I'm just going to restate
my initial use-case to try to make it more clear as well.
>From my usage of beam, I feel like the big benefit of async DoFns would be
to allow batched IO to be implemented more simply inside a DoFn. Even in
the Beam SDK
Processing DSL script job_PreCommit_Python.groovy
Processing DSL script job_PreCommit_Python_ValidatesRunner_Flink.groovy
java.lang.IllegalArgumentException:
beam_PreCommit_Python_PVR_Flink_Cron already exists
at hudson.model.Items.verifyItemDoesNotAlreadyExist(Items.java:641)
at
I'm wondering if anybody can reproduce this issue. The build has failed
once because testcontainers didn't pull docker image. If we use caching
proxy for docker, it could be a reason for that. I didn't find any similar
known issue in testcontainers fixed recently, but just in case, I bumped
Exciting to see the cross-language train gathering steam :)
It may be useful to flesh out the user facing aspects a bit more before
going too deep on the service / expansion side or maybe that was done
elsewhere?
A few examples (of varying complexity) of how the shim/proxy transforms
would look
On Thu, Jan 24, 2019 at 5:08 PM Thomas Weise wrote:
>
> Exciting to see the cross-language train gathering steam :)
>
> It may be useful to flesh out the user facing aspects a bit more before going
> too deep on the service / expansion side or maybe that was done elsewhere?
It's been discussed,
Yes, you're correct that PR#7571 [1] had a checkstyle violation when
merged. I didn't notice the checkstyle failure and I hit the merge button.
Sorry about that.
Here's where I went wrong:
* The precommits showed one failing: Java. GitHub shows the status as a
green check or red X on the head
I have just seen it randomly occur on presubmits. So I don't have a
reliable repro, unfortunately.
It may be a specific environmental issue to the beam1 machine the tests ran
on?
https://builds.apache.org/job/beam_PreCommit_Java_Commit/3722/
On Thu, Jan 24, 2019 at 8:16 AM Gleb Kanterov wrote:
Keep in mind that these user-supplied lambdas are commonly used in our IOs.
One common usage is in Sink IOs, to allow dynamic destinations. e.g. in
BigQueryIO.Write, a user-supplied lambda determines what table a record
should be written to.
Given that IOs are one of the big selling points of
As Steve said, the main rationale for this is so that asynchronous IOs (or
in general, asynchronous remote calls) call be made. To some degree this
addresses Scott's concern: the asynchronous threads should be, for the most
part, simply waiting for IOs to complete; the reason to do the waiting
I strongly support making flakes more painful to get the incentives right.
We can control the "blast radius" of precommit flakes by more focused test
suites. Postcommit they still need triage and deflake.
Kenn
On Thu, Jan 24, 2019 at 8:56 AM Scott Wegner wrote:
> Yes, you're correct that
It's also important to note that in many (most?) IO frameworks (gRPC,
finagle, etc), asynchronous IO is typically completely non-blocking, so
there generally won't be a large number of threads waiting for IO to
complete. (netty uses a small pool of threads for the Event Loop Group for
example).
Makes sense to me. We should make it easier to write DoFn's in this pattern
that has emerged as common among I/O connectors.
Enabling asynchronous task chaining across a fusion tree is more
complicated but not necessary for this scenario.
On Thu, Jan 24, 2019 at 10:13 AM Steve Niemitz wrote:
>
32 matches
Mail list logo
