Jenkins build is back to stable : beam_Release_NightlySnapshot #332

2017-02-17 Thread Apache Jenkins Server
See

Performance Testing Next Steps

2017-02-17 Thread Jason Kuster
Hi all, I've written up a doc on next steps for getting performance testing up and running for Beam. I'd love to hear from people -- there's a fair amount of work encapsulated in here, but the end result is that we have a performance testing system which we can use for benchmarking all aspects of

Re: Metrics for Beam IOs.

2017-02-17 Thread Amit Sela
@JB I think what you're suggesting is that Beam should provide a "Metrics Reporting" API as well, and I used to think like you, but the more I thought of that the more I tend to disagree now. The SDK is for users to author pipelines, so Metrics are for user-defined metrics (in contrast to runner

RE: Merge HadoopInputFormatIO and HDFSIO in a single module

2017-02-17 Thread Dipti Kulkarni
Thank you all for your inputs! -Original Message- From: Dan Halperin [mailto:dhalp...@google.com.INVALID] Sent: Friday, February 17, 2017 12:17 PM To: dev@beam.apache.org Subject: Re: Merge HadoopInputFormatIO and HDFSIO in a single module Raghu, Amit -- +1 to your expertise :) On

Re: Metrics for Beam IOs.

2017-02-17 Thread Jean-Baptiste Onofré
Hi guys, As I'm back from vacation, I'm back on this topic ;) It's a great discussion, and I think about the Metric IO coverage, it's good. However, there's a point that we discussed very fast in the thread and I think it's an important one (maybe more important than the provided metrics