[GitHub] flink pull request #4647: [FLINK-7575] [WEB-DASHBOARD] Display "Fetching..."...
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/4647 ---
[GitHub] flink pull request #4647: [FLINK-7575] [WEB-DASHBOARD] Display "Fetching..."...
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/4647#discussion_r138133547 --- Diff: flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/utils/MutableIOMetrics.java --- @@ -72,13 +72,54 @@ public void addIOMetrics(AccessExecution attempt, @Nullable MetricFetcher fetche } else { // execAttempt is still running, use MetricQueryService instead if (fetcher != null) { fetcher.update(); - MetricStore.SubtaskMetricStore metrics = fetcher.getMetricStore().getSubtaskMetricStore(jobID, taskID, attempt.getParallelSubtaskIndex()); - if (metrics != null) { - this.numBytesInLocal += Long.valueOf(metrics.getMetric(MetricNames.IO_NUM_BYTES_IN_LOCAL, "0")); - this.numBytesInRemote += Long.valueOf(metrics.getMetric(MetricNames.IO_NUM_BYTES_IN_REMOTE, "0")); - this.numBytesOut += Long.valueOf(metrics.getMetric(MetricNames.IO_NUM_BYTES_OUT, "0")); - this.numRecordsIn += Long.valueOf(metrics.getMetric(MetricNames.IO_NUM_RECORDS_IN, "0")); - this.numRecordsOut += Long.valueOf(metrics.getMetric(MetricNames.IO_NUM_RECORDS_OUT, "0")); + MetricStore metricStore = fetcher.getMetricStore(); + synchronized (metricStore) { + MetricStore.SubtaskMetricStore metrics = metricStore.getSubtaskMetricStore(jobID, taskID, attempt.getParallelSubtaskIndex()); + if (metrics != null) { + /** +* We want to keep track of missing metrics to be able to make a difference between 0 as a value +* and a missing value. +* In case a metric is missing for a parallel instance of a task, we initialize if with -1 and +* will be considered as incomplete --- End diff -- missing period, and the sentence flow is a bit weird, how about "and consider it as incomplete."? ---
[GitHub] flink pull request #4647: [FLINK-7575] [WEB-DASHBOARD] Display "Fetching..."...
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/4647#discussion_r138133369 --- Diff: flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/utils/MutableIOMetrics.java --- @@ -72,13 +72,54 @@ public void addIOMetrics(AccessExecution attempt, @Nullable MetricFetcher fetche } else { // execAttempt is still running, use MetricQueryService instead if (fetcher != null) { fetcher.update(); - MetricStore.SubtaskMetricStore metrics = fetcher.getMetricStore().getSubtaskMetricStore(jobID, taskID, attempt.getParallelSubtaskIndex()); - if (metrics != null) { - this.numBytesInLocal += Long.valueOf(metrics.getMetric(MetricNames.IO_NUM_BYTES_IN_LOCAL, "0")); - this.numBytesInRemote += Long.valueOf(metrics.getMetric(MetricNames.IO_NUM_BYTES_IN_REMOTE, "0")); - this.numBytesOut += Long.valueOf(metrics.getMetric(MetricNames.IO_NUM_BYTES_OUT, "0")); - this.numRecordsIn += Long.valueOf(metrics.getMetric(MetricNames.IO_NUM_RECORDS_IN, "0")); - this.numRecordsOut += Long.valueOf(metrics.getMetric(MetricNames.IO_NUM_RECORDS_OUT, "0")); + MetricStore metricStore = fetcher.getMetricStore(); + synchronized (metricStore) { + MetricStore.SubtaskMetricStore metrics = metricStore.getSubtaskMetricStore(jobID, taskID, attempt.getParallelSubtaskIndex()); + if (metrics != null) { + /** +* We want to keep track of missing metrics to be able to make a difference between 0 as a value +* and a missing value. +* In case a metric is missing for a parallel instance of a task, we initialize if with -1 and --- End diff -- typo: if -> it ---
[GitHub] flink pull request #4647: [FLINK-7575] [WEB-DASHBOARD] Display "Fetching..."...
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/4647#discussion_r138133763 --- Diff: flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/utils/MutableIOMetrics.java --- @@ -99,11 +140,35 @@ public void addIOMetrics(AccessExecution attempt, @Nullable MetricFetcher fetche * @throws IOException */ public void writeIOMetricsAsJson(JsonGenerator gen) throws IOException { + /** +* Ask describe in the addIOMetrics, we want to distinguish incomplete values from 0. --- End diff -- typo: ask -> as remote "the" before addIOMetrics could use ```@link annotation for the method``` ---
[GitHub] flink pull request #4647: [FLINK-7575] [WEB-DASHBOARD] Display "Fetching..."...
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/4647#discussion_r137093260 --- Diff: flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/utils/MutableIOMetrics.java --- @@ -79,6 +85,7 @@ public void addIOMetrics(AccessExecution attempt, @Nullable MetricFetcher fetche this.numBytesOut += Long.valueOf(metrics.getMetric(MetricNames.IO_NUM_BYTES_OUT, "0")); this.numRecordsIn += Long.valueOf(metrics.getMetric(MetricNames.IO_NUM_RECORDS_IN, "0")); this.numRecordsOut += Long.valueOf(metrics.getMetric(MetricNames.IO_NUM_RECORDS_OUT, "0")); + this.metricsFetched = true; --- End diff -- Unfortunately, if #4472 goes through that neat property of the synchronization goes away. ---
[GitHub] flink pull request #4647: [FLINK-7575] [WEB-DASHBOARD] Display "Fetching..."...
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/4647#discussion_r137091495 --- Diff: flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/utils/MutableIOMetrics.java --- @@ -79,6 +85,7 @@ public void addIOMetrics(AccessExecution attempt, @Nullable MetricFetcher fetche this.numBytesOut += Long.valueOf(metrics.getMetric(MetricNames.IO_NUM_BYTES_OUT, "0")); this.numRecordsIn += Long.valueOf(metrics.getMetric(MetricNames.IO_NUM_RECORDS_IN, "0")); this.numRecordsOut += Long.valueOf(metrics.getMetric(MetricNames.IO_NUM_RECORDS_OUT, "0")); + this.metricsFetched = true; --- End diff -- luckily this is only the cause because of a bug in this class. Access to the metric store must be guarded by a synchronization block on the metric store, which would then guarantee that the update from a taskmanager is applied in a all-or-nothing fashion. ---
[GitHub] flink pull request #4647: [FLINK-7575] [WEB-DASHBOARD] Display "Fetching..."...
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/4647#discussion_r137067050 --- Diff: flink-runtime-web/web-dashboard/web/js/vendor.js --- @@ -15,11 +15,11 @@ ua.range=function(t,e,n){if(arguments.length<3&&(n=1,arguments.length<2&&(e=t,t= return n(null,null,1,{point:function(t,n){i.push(t=e(t,n)),t[0]*=Ba,t[1]*=Ba}}),{type:"Polygon",coordinates:[i]}}var e,n,r=[0,0],i=6;return t.origin=function(e){return arguments.length?(r=e,t):r},t.angle=function(r){return arguments.length?(n=gn((e=+r)*Ra,i*Ra),t):e},t.precision=function(r){return arguments.length?(n=gn(e*Ra,(i=+r)*Ra),t):i},t.angle(90)},ua.geo.distance=function(t,e){var n,r=(e[0]-t[0])*Ra,i=t[1]*Ra,o=e[1]*Ra,a=Math.sin(r),s=Math.cos(r),u=Math.sin(i),l=Math.cos(i),c=Math.sin(o),f=Math.cos(o);return Math.atan2(Math.sqrt((n=f*a)*n+(n=l*c-u*f*s)*n),u*c+l*f*s)},ua.geo.graticule=function(){function t(){return{type:"MultiLineString",coordinates:e()}}function e(){return ua.range(Math.ceil(o/v)*v,i,v).map(d).concat(ua.range(Math.ceil(l/m)*m,u,m).map(h)).concat(ua.range(Math.ceil(r/p)*p,n,p).filter(function(t){return xa(t%v)>La}).map(c)).concat(ua.range(Math.ceil(s/g)*g,a,g).filter(function(t){return xa(t%m)>La}).map(f))}var n,r,i,o,a,s,u,l,c,f,d,h,p=10,g=p,v=90,m=360,y= 2.5;return t.lines=function(){return e().map(function(t){return{type:"LineString",coordinates:t}})},t.outline=function(){return{type:"Polygon",coordinates:[d(o).concat(h(u).slice(1),d(i).reverse().slice(1),h(l).reverse().slice(1))]}},t.extent=function(e){return arguments.length?t.majorExtent(e).minorExtent(e):t.minorExtent()},t.majorExtent=function(e){return arguments.length?(o=+e[0][0],i=+e[1][0],l=+e[0][1],u=+e[1][1],o>i&&(e=o,o=i,i=e),l>u&&(e=l,l=u,u=e),t.precision(y)):[[o,l],[i,u]]},t.minorExtent=function(e){return arguments.length?(r=+e[0][0],n=+e[1][0],s=+e[0][1],a=+e[1][1],r>n&&(e=r,r=n,n=e),s>a&&(e=s,s=a,a=e),t.precision(y)):[[r,s],[n,a]]},t.step=function(e){return arguments.length?t.majorStep(e).minorStep(e):t.minorStep()},t.majorStep=function(e){return arguments.length?(v=+e[0],m=+e[1],t):[v,m]},t.minorStep=function(e){return arguments.length?(p=+e[0],g=+e[1],t):[p,g]},t.precision=function(e){return arguments.length?(y=+e,c=mn(s,a,90),f=yn(r,n,y),d=mn(l,u,90),h=yn(o,i,y),t ):y},t.majorExtent([[-180,-90+La],[180,90-La]]).minorExtent([[-180,-80-La],[180,80+La]])},ua.geo.greatArc=function(){function t(){return{type:"LineString",coordinates:[e||r.apply(this,arguments),n||i.apply(this,arguments)]}}var e,n,r=bn,i=xn;return t.distance=function(){return ua.geo.distance(e||r.apply(this,arguments),n||i.apply(this,arguments))},t.source=function(n){return arguments.length?(r=n,e="function"==typeof n?null:n,t):r},t.target=function(e){return arguments.length?(i=e,n="function"==typeof e?null:e,t):i},t.precision=function(){return arguments.length?t:0},t},ua.geo.interpolate=function(t,e){return wn(t[0]*Ra,t[1]*Ra,e[0]*Ra,e[1]*Ra)},ua.geo.length=function(t){return Xs=0,ua.geo.stream(t,Zs),Xs};var Xs,Zs={sphere:w,point:w,lineStart:$n,lineEnd:w,polygonStart:w,polygonEnd:w},Qs=kn(function(t){return Math.sqrt(2/(1+t))},function(t){return 2*Math.asin(t/2)});(ua.geo.azimuthalEqualArea=function(){return an(Qs)}).raw=Qs;var Ks=kn(function(t){var e=Math.acos(t);return e&/Math .sin(e)},y);(ua.geo.azimuthalEquidistant=function(){return an(Ks)}).raw=Ks,(ua.geo.conicConformal=function(){return Ve(_n)}).raw=_n,(ua.geo.conicEquidistant=function(){return Ve(Mn)}).raw=Mn;var Js=kn(function(t){return 1/t},Math.atan);(ua.geo.gnomonic=function(){return an(Js)}).raw=Js,Cn.invert=function(t,e){return[t,2*Math.atan(Math.exp(e))-Wa]},(ua.geo.mercator=function(){return Sn(Cn)}).raw=Cn;var tu=kn(function(){return 1},Math.asin);(ua.geo.orthographic=function(){return an(tu)}).raw=tu;var eu=kn(function(t){return 1/(1+t)},function(t){return 2*Math.atan(t)});(ua.geo.stereographic=function(){return an(eu)}).raw=eu,En.invert=function(t,e){return[-e,2*Math.atan(Math.exp(t))-Wa]},(ua.geo.transverseMercator=function(){var t=Sn(En),e=t.center,n=t.rotate;return t.center=function(t){return t?e([-t[1],t[0]]):(t=e(),[t[1],-t[0]])},t.rotate=function(t){return t?n([t[0],t[1],t.length>2?t[2]+90:90]):(t=n(),[t[0],t[1],t[2]-90])},n([0,0,90])}).raw=En,ua.geom={},ua.geom.hull=function(t){func tion e(t){if(t.length<3)return[];var e,i=Ct(n),o=Ct(r),a=t.length,s=[],u=[];for(e=0;e=0;--e)h.push(t[s[l[e]][2]]);for(e=+f;e
[GitHub] flink pull request #4647: [FLINK-7575] [WEB-DASHBOARD] Display "Fetching..."...
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/4647#discussion_r137067579 --- Diff: flink-runtime-web/web-dashboard/web/js/hs/index.js --- @@ -1,2 +1,2 @@ -angular.module("flinkApp",["ui.router","angularMoment","dndLists"]).run(["$rootScope",function(e){return e.sidebarVisible=!1,e.showSidebar=function(){return e.sidebarVisible=!e.sidebarVisible,e.sidebarClass="force-show"}}]).value("flinkConfig",{jobServer:"","refresh-interval":1e4}).value("watermarksConfig",{noWatermark:-0x8000}).run(["JobsService","MainService","flinkConfig","$interval",function(e,t,r,n){return t.loadConfig().then(function(t){return angular.extend(r,t),e.listJobs(),n(function(){return e.listJobs()},r["refresh-interval"])})}]).config(["$uiViewScrollProvider",function(e){return e.useAnchorScroll()}]).run(["$rootScope","$state",function(e,t){return e.$on("$stateChangeStart",function(e,r,n,i){if(r.redirectTo)return e.preventDefault(),t.go(r.redirectTo,n)})}]).config(["$stateProvider","$urlRouterProvider",function(e,t){return e.state("completed-jobs",{url:"/completed-jobs",views:{main:{templateUrl:"partials/jobs/completed-jobs.html",controller:"CompletedJ obsController"}}}).state("single-job",{url:"/jobs/{jobid}","abstract":!0,views:{main:{templateUrl:"partials/jobs/job.html",controller:"SingleJobController"}}}).state("single-job.plan",{url:"",redirectTo:"single-job.plan.subtasks",views:{details:{templateUrl:"partials/jobs/job.plan.html",controller:"JobPlanController"}}}).state("single-job.plan.subtasks",{url:"",views:{"node-details":{templateUrl:"partials/jobs/job.plan.node-list.subtasks.html",controller:"JobPlanSubtasksController"}}}).state("single-job.plan.metrics",{url:"/metrics",views:{"node-details":{templateUrl:"partials/jobs/job.plan.node-list.metrics.html",controller:"JobPlanMetricsController"}}}).state("single-job.plan.watermarks",{url:"/watermarks",views:{"node-details":{templateUrl:"partials/jobs/job.plan.node-list.watermarks.html"}}}).state("single-job.plan.taskmanagers",{url:"/taskmanagers",views:{"node-details":{templateUrl:"partials/jobs/job.plan.node-list.taskmanagers.html",controller:"JobPlanTaskManagersController"} }}).state("single-job.plan.accumulators",{url:"/accumulators",views:{"node-details":{templateUrl:"partials/jobs/job.plan.node-list.accumulators.html",controller:"JobPlanAccumulatorsController"}}}).state("single-job.plan.checkpoints",{url:"/checkpoints",redirectTo:"single-job.plan.checkpoints.overview",views:{"node-details":{templateUrl:"partials/jobs/job.plan.node-list.checkpoints.html",controller:"JobPlanCheckpointsController"}}}).state("single-job.plan.checkpoints.overview",{url:"/overview",views:{"checkpoints-view":{templateUrl:"partials/jobs/job.plan.node.checkpoints.overview.html",controller:"JobPlanCheckpointsController"}}}).state("single-job.plan.checkpoints.summary",{url:"/summary",views:{"checkpoints-view":{templateUrl:"partials/jobs/job.plan.node.checkpoints.summary.html",controller:"JobPlanCheckpointsController"}}}).state("single-job.plan.checkpoints.history",{url:"/history",views:{"checkpoints-view":{templateUrl:"partials/jobs/job.plan.node.checkpoints.history.html",cont roller:"JobPlanCheckpointsController"}}}).state("single-job.plan.checkpoints.config",{url:"/config",views:{"checkpoints-view":{templateUrl:"partials/jobs/job.plan.node.checkpoints.config.html",controller:"JobPlanCheckpointsController"}}}).state("single-job.plan.checkpoints.details",{url:"/details/{checkpointId}",views:{"checkpoints-view":{templateUrl:"partials/jobs/job.plan.node.checkpoints.details.html",controller:"JobPlanCheckpointDetailsController"}}}).state("single-job.plan.backpressure",{url:"/backpressure",views:{"node-details":{templateUrl:"partials/jobs/job.plan.node-list.backpressure.html",controller:"JobPlanBackPressureController"}}}).state("single-job.timeline",{url:"/timeline",views:{details:{templateUrl:"partials/jobs/job.timeline.html"}}}).state("single-job.timeline.vertex",{url:"/{vertexId}",views:{vertex:{templateUrl:"partials/jobs/job.timeline.vertex.html",controller:"JobTimelineVertexController"}}}).state("single-job.exceptions",{url:"/exceptions",views:{details:{t emplateUrl:"partials/jobs/job.exceptions.html",controller:"JobExceptionsController"}}}).state("single-job.config",{url:"/config",views:{details:{templateUrl:"partials/jobs/job.config.html"}}}),t.otherwise("/completed-jobs")}]),angular.module("flinkApp").directive("bsLabel",["JobsService",function(e){return{transclude:!0,replace:!0,scope:{getLabelClass:"&",status:"@"},template:"",link:function(t,r,n){return t.getLabelClass=function(){return"label label-"+e.translateLabelState(n.status)]).directive("bpLabel",["JobsService",function(e){return{transclude:!0,replace:!0,scope:{getBackPressureLabelClass:"&",status:"@"},template:"",link:function(t,r,n){return t.getBackPressureLabelClass=function(){return"label
[GitHub] flink pull request #4647: [FLINK-7575] [WEB-DASHBOARD] Display "Fetching..."...
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/4647#discussion_r137066868 --- Diff: flink-runtime-web/web-dashboard/web/css/vendor.css --- @@ -1,4 +1,4 @@ -.nvd3 .nv-axis line,.nvd3 .nv-axis path{fill:none;shape-rendering:crispEdges}.nv-brush .extent,.nvd3 .background path,.nvd3 .nv-axis line,.nvd3 .nv-axis path{shape-rendering:crispEdges}.nvtooltip,.text-left{text-align:left}.nvtooltip,svg.nvd3-svg{display:block;-webkit-touch-callout:none;-webkit-user-select:none;-khtml-user-select:none}.nvtooltip table td,img{vertical-align:middle}.nvtooltip,.text-nowrap{white-space:nowrap}.btn-group>.btn-group,.btn-toolbar .btn,.btn-toolbar .btn-group,.btn-toolbar .input-group,.col-xs-1,.col-xs-10,.col-xs-11,.col-xs-12,.col-xs-2,.col-xs-3,.col-xs-4,.col-xs-5,.col-xs-6,.col-xs-7,.col-xs-8,.col-xs-9,.dropdown-menu{float:left}.navbar-fixed-bottom .navbar-collapse,.navbar-fixed-top .navbar-collapse,.pre-scrollable{max-height:340px}.fa,.glyphicon{-moz-osx-font-smoothing:grayscale}.nvd3 .nv-axis{pointer-events:none;opacity:1}.nvd3 .nv-axis.nv-disabled,.nvd3 .nv-controlsWrap .nv-legend .nv-check-box .nv-check{opacity:0}.nvd3 .nv-axis path{stroke:#000;s troke-opacity:.75}.nvd3 .nv-axis path.domain{stroke-opacity:.75}.nvd3 .nv-axis.nv-x path.domain{stroke-opacity:0}.nvd3 .nv-axis line{stroke:#e5e5e5}.nvd3 .nv-axis .zero line, .nvd3 .nv-axis line.zero{stroke-opacity:.75}.nvd3 .nv-axis .nv-axisMaxMin text{font-weight:700}.nvd3 .x .nv-axis .nv-axisMaxMin text,.nvd3 .x2 .nv-axis .nv-axisMaxMin text,.nvd3 .x3 .nv-axis .nv-axisMaxMin text{text-anchor:middle}.nvd3 .nv-bars rect{fill-opacity:.75;transition:fill-opacity 250ms linear;-moz-transition:fill-opacity 250ms linear;-webkit-transition:fill-opacity 250ms linear}.nvd3 .nv-bars rect.hover{fill-opacity:1}.nvd3 .nv-bars .hover rect{fill:#add8e6}.nvd3 .nv-bars text{fill:transparent}.nvd3 .nv-bars .hover text{fill:rgba(0,0,0,1)}.nvd3 .nv-discretebar .nv-groups rect,.nvd3 .nv-multibar .nv-groups rect,.nvd3 .nv-multibarHorizontal .nv-groups rect{stroke-opacity:0;transition:fill-opacity 250ms linear;-moz-transition:fill-opacity 250ms linear;-webkit-transition:fill-opacity 250ms linear}.nvd3 .n v-candlestickBar .nv-ticks rect:hover,.nvd3 .nv-discretebar .nv-groups rect:hover,.nvd3 .nv-multibar .nv-groups rect:hover,.nvd3 .nv-multibarHorizontal .nv-groups rect:hover{fill-opacity:1}.nvd3 .nv-discretebar .nv-groups text,.nvd3 .nv-multibarHorizontal .nv-groups text{font-weight:700;fill:rgba(0,0,0,1);stroke:transparent}.nvd3 .nv-boxplot circle{fill-opacity:.5}.nvd3 .nv-boxplot circle:hover,.nvd3 .nv-boxplot rect:hover{fill-opacity:1}.nvd3 line.nv-boxplot-median{stroke:#000}.nv-boxplot-tick:hover{stroke-width:2.5px}.nvd3.nv-bullet{font:10px sans-serif}.nvd3.nv-bullet .nv-measure{fill-opacity:.8}.nvd3.nv-bullet .nv-measure:hover{fill-opacity:1}.nvd3.nv-bullet .nv-marker{stroke:#000;stroke-width:2px}.nvd3.nv-bullet .nv-markerTriangle{stroke:#000;fill:#fff;stroke-width:1.5px}.nvd3.nv-bullet .nv-markerLine{stroke:#000;stroke-width:1.5px}.nvd3.nv-bullet .nv-tick line{stroke:#666;stroke-width:.5px}.nvd3.nv-bullet .nv-range.nv-s0{fill:#eee}.nvd3.nv-bullet .nv-range.nv-s1{fill:#ddd}.nvd 3.nv-bullet .nv-range.nv-s2{fill:#ccc}.nvd3.nv-bullet .nv-title{font-size:14px;font-weight:700}.nvd3.nv-bullet .nv-subtitle{fill:#999}.nvd3.nv-bullet .nv-range{fill:#bababa;fill-opacity:.4}.nvd3.nv-bullet .nv-range:hover{fill-opacity:.7}.nvd3.nv-candlestickBar .nv-ticks .nv-tick{stroke-width:1px}.nvd3.nv-candlestickBar .nv-ticks .nv-tick.hover{stroke-width:2px}.nvd3.nv-candlestickBar .nv-ticks .nv-tick.positive rect{stroke:#2ca02c;fill:#2ca02c}.nvd3.nv-candlestickBar .nv-ticks .nv-tick.negative rect{stroke:#d62728;fill:#d62728}.with-transitions .nv-candlestickBar .nv-ticks .nv-tick{transition:stroke-width 250ms linear,stroke-opacity 250ms linear;-moz-transition:stroke-width 250ms linear,stroke-opacity 250ms linear;-webkit-transition:stroke-width 250ms linear,stroke-opacity 250ms linear}.nvd3.nv-candlestickBar .nv-ticks line{stroke:#333}.nv-force-node{stroke:#fff;stroke-width:1.5px}.nv-force-link{stroke:#999;stroke-opacity:.6}.nv-force-node text{stroke-width:0}.nvd3 .nv-check-box .nv -box{fill-opacity:0;stroke-width:2}.nvd3 .nv-check-box .nv-check{fill-opacity:0;stroke-width:4}.nvd3 .nv-series.nv-disabled .nv-check-box .nv-check{fill-opacity:0;stroke-opacity:0}.nvd3.nv-linePlusBar .nv-bar rect{fill-opacity:.75}.nvd3.nv-linePlusBar .nv-bar rect:hover{fill-opacity:1}.nvd3 .nv-groups path.nv-line{fill:none}.nvd3 .nv-groups path.nv-area{stroke:none}.nvd3.nv-line .nvd3.nv-scatter .nv-groups .nv-point{fill-opacity:0;stroke-opacity:0}.nvd3.nv-scatter.nv-single-point .nv-groups .nv-point{fill-opacity:.5!important;stroke-opacity:.5!important}.with-transitions .nvd3 .nv-groups .nv-point{transition:stroke-width 250ms linear,stroke-opacity 250ms linear;-moz-transition:stroke-width 250ms
[GitHub] flink pull request #4647: [FLINK-7575] [WEB-DASHBOARD] Display "Fetching..."...
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/4647#discussion_r137068223 --- Diff: flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/utils/MutableIOMetrics.java --- @@ -79,6 +85,7 @@ public void addIOMetrics(AccessExecution attempt, @Nullable MetricFetcher fetche this.numBytesOut += Long.valueOf(metrics.getMetric(MetricNames.IO_NUM_BYTES_OUT, "0")); this.numRecordsIn += Long.valueOf(metrics.getMetric(MetricNames.IO_NUM_RECORDS_IN, "0")); this.numRecordsOut += Long.valueOf(metrics.getMetric(MetricNames.IO_NUM_RECORDS_OUT, "0")); + this.metricsFetched = true; --- End diff -- Note that this does not necessarily mean that the num/bytes IO metrics have been added yet; it only means that _some_ metric for the given subtask has already arrived and was inserted into the store. ---
[GitHub] flink pull request #4647: [FLINK-7575] [WEB-DASHBOARD] Display "Fetching..."...
Github user zentol commented on a diff in the pull request: https://github.com/apache/flink/pull/4647#discussion_r137066790 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/IOMetrics.java --- @@ -42,6 +42,7 @@ protected double numBytesInRemotePerSecond; protected double numBytesOutPerSecond; + --- End diff -- this should be reverted (goes against checkstyle) ---
[GitHub] flink pull request #4647: [FLINK-7575] [WEB-DASHBOARD] Display "Fetching..."...
GitHub user jameslafa opened a pull request: https://github.com/apache/flink/pull/4647 [FLINK-7575] [WEB-DASHBOARD] Display "Fetching..." instead of 0 when IO metrics are not yet retrieved ## Problem IO metrics is the web-dashboard are retrieved asynchronously. When we display a job's IO metrics for the first time, the API return "0" because values are not yet available. It's misleading. ## Fix There is now a new flag is the API called *metrics-fetched*. If its value is **false**, the dashboard displays "Fetching..." instead of 0. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jameslafa/flink 7575-io_metrics Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/4647.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4647 commit b14f5865967bb42207b6580ae9d719c02a9ec388 Author: James LafaDate: 2017-09-05T09:33:34Z [FLINK-7575] [webui] Show "Fetching" instead of 0 when IO metrics are not yet retrieved commit 1ab8ef42b3fd624aa495fd009a4a2f212484eb2e Author: James Lafa Date: 2017-09-05T10:25:40Z [FLINK-7575] [webui] Rebuild the dashboard commit be71dd93503b64c11e49ca6e8ee01a687770c3e6 Author: James Lafa Date: 2017-09-05T12:30:35Z [FLINK-7575] [webui] Move metricsFetched to MutableIOMetrics ---