Thanks everyone. I was able to collect a lot of good feedback from everyone who contributed. I am going to wrap it up for now and label the design as "Design Finalized (Unimplemented)".
I really believe we have made a much better design than I initially wrote up. I couldn't have done it without the help of everyone who offered their time, energy and viewpoints. :) Thanks again, please let me know if you see any major issues with the design still. I think I have enough information to begin some implementation as soon as I have some time in the coming weeks. Alex https://s.apache.org/beam-gcp-debuggability https://s.apache.org/beam-histogram-metrics On Thu, May 14, 2020 at 5:22 PM Alex Amato <ajam...@google.com> wrote: > Thanks to all who have spent their time on this, there were many great > suggestions, just another reminder that tomorrow I will be finalizing the > documents, unless there are any major objections left. Please take a look > at it if you are interested. > > I will still welcome feedback at any time :). > > But I believe we have gathered enough information to produce a good > design, which I will start to work on soon. > I will begin to build the necessary subset of the new features proposed to > support the BigQueryIO metrics use case, proposed. > I will likely start with the python SDK first. > > https://s.apache.org/beam-gcp-debuggability > https://s.apache.org/beam-histogram-metrics > > > On Wed, May 13, 2020 at 3:07 PM Alex Amato <ajam...@google.com> wrote: > >> Thanks again for more feedback :). I have iterated on things again. I'll >> report back at the end of the week. If there are no major disagreements >> still, I'll close the discussion, believe it to be in a good enough state >> to start some implementation. But welcome feedback. >> >> Latest changes are changing the exponential format to allow denser >> buckets. Using only two MonitoringInfoSpec now for all of the IOs to use. >> Requiring some labels, but allowing optional >> ones for specific IOs to provide more contents. >> >> https://s.apache.org/beam-gcp-debuggability >> https://s.apache.org/beam-histogram-metrics >> >> On Mon, May 11, 2020 at 4:24 PM Alex Amato <ajam...@google.com> wrote: >> >>> Thanks for the great feedback so far :). I've included many new ideas, >>> and made some revisions. Both docs have changed a fair bit since the >>> initial mail out. >>> >>> https://s.apache.org/beam-gcp-debuggability >>> https://s.apache.org/beam-histogram-metrics >>> >>> PTAL and let me know what you think, and hopefully we can resolve major >>> issues by the end of the week. I'll try to finalize things by then, but of >>> course always stay open to your great ideas. :) >>> >>> On Wed, May 6, 2020 at 6:19 PM Alex Amato <ajam...@google.com> wrote: >>> >>>> Thanks everyone so far for taking a look so far :). >>>> >>>> I am hoping to have this finalize the two reviews by the end of next >>>> week, May 15th. >>>> >>>> I'll continue to follow up on feedback and make changes, and I will add >>>> some more mentions to the documents to draw attention >>>> >>>> https://s.apache.org/beam-gcp-debuggability >>>> https://s.apache.org/beam-histogram-metrics >>>> >>>> On Wed, May 6, 2020 at 10:00 AM Luke Cwik <lc...@google.com> wrote: >>>> >>>>> Thanks, also took a look and left some comments. >>>>> >>>>> On Tue, May 5, 2020 at 6:24 PM Alex Amato <ajam...@google.com> wrote: >>>>> >>>>>> Hello, >>>>>> >>>>>> I created another design document. This time for GCP IO Debuggability >>>>>> Metrics. Which defines some new metrics to collect in the GCP IO >>>>>> libraries. >>>>>> This is for monitoring request counts and request latencies. >>>>>> >>>>>> Please take a look and let me know what you think: >>>>>> https://s.apache.org/beam-gcp-debuggability >>>>>> >>>>>> I also sent out a separate design yesterday ( >>>>>> https://s.apache.org/beam-histogram-metrics) which is related as >>>>>> this document uses a Histogram style metric :). >>>>>> >>>>>> I would love some feedback to make this feature the best possible :D, >>>>>> Alex >>>>>> >>>>>
