[kudu-CR] KUDU-1571. Generate "deprecated" metrics in MDL

[Todd Lipcon (Code Review)]

Todd Lipcon has posted comments on this change.

Change subject: KUDU-1571. Generate "deprecated" metrics in MDL
..


Patch Set 2:

Oops, just realized I missed Mike's code review comments above. Will address 
these in a follow-on

-- 
To view, visit http://gerrit.cloudera.org:8080/4067
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I121203d781ab593cb94d7248d164cdad7bf26e78
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-HasComments: No

[kudu-CR] KUDU-1571. Generate "deprecated" metrics in MDL

[Todd Lipcon (Code Review)]

Todd Lipcon has submitted this change and it was merged.

Change subject: KUDU-1571. Generate "deprecated" metrics in MDL
..


KUDU-1571. Generate "deprecated" metrics in MDL

This checks in the "--dump_metrics_json" output from already-released
versions, and changes the generate_mdl.py script so that any metrics
which were ever present in an earlier version are carried through to the
current version of the MDL, but marked as deprecated.

This works around an issue in which Cloudera Manager doesn't properly
handle the removal of metrics. Namely, if a dashboard (user-generated or
built-in) references a removed metric, it will produce an NPE.

This will now become part of the release process: after a release, we
should check in the output of 'dump_metrics_json' so that the metrics in
that release will be carried forward.

I verified the results using both the newly included
'check_csd_compatibility' script as well as installing a newly built
0.10.x CSD on a cluster. Whereas the tablet server status page was
previously broken, now it shows the remote bootstrap metrics as
DEPRECATED.

Change-Id: I121203d781ab593cb94d7248d164cdad7bf26e78
Reviewed-on: http://gerrit.cloudera.org:8080/4067
Tested-by: Kudu Jenkins
Reviewed-by: Mike Percy 
---
M build-support/release/rat_exclude_files.txt
A java/kudu-csd/check_csd_compatibility.py
M java/kudu-csd/generate_mdl.py
A java/kudu-csd/old-version-metrics/0.10.0-master.json
A java/kudu-csd/old-version-metrics/0.10.0-tserver.json
A java/kudu-csd/old-version-metrics/0.9.1-master.json
A java/kudu-csd/old-version-metrics/0.9.1-tserver.json
7 files changed, 4,831 insertions(+), 17 deletions(-)

Approvals:
  Mike Percy: Looks good to me, approved
  Kudu Jenkins: Verified



-- 
To view, visit http://gerrit.cloudera.org:8080/4067
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I121203d781ab593cb94d7248d164cdad7bf26e78
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Todd Lipcon 

[kudu-CR] KUDU-1571. Generate "deprecated" metrics in MDL

[Todd Lipcon (Code Review)]

Todd Lipcon has posted comments on this change.

Change subject: KUDU-1571. Generate "deprecated" metrics in MDL
..


Patch Set 1:

To fill in anyone following along, adar and I had an offline discussion about 
this over the weekend (he was mostly AFK). The gist is that he doesn't like 
this change but willing to take it as a temporary workaround until CM has some 
releases which avoid this problem.

-- 
To view, visit http://gerrit.cloudera.org:8080/4067
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I121203d781ab593cb94d7248d164cdad7bf26e78
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-HasComments: No

[kudu-CR] KUDU-1571. Generate "deprecated" metrics in MDL

[Adar Dembo (Code Review)]

Adar Dembo has posted comments on this change.

Change subject: KUDU-1571. Generate "deprecated" metrics in MDL
..


Patch Set 1: -Code-Review

Removing my -1 based on our discussion.

-- 
To view, visit http://gerrit.cloudera.org:8080/4067
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I121203d781ab593cb94d7248d164cdad7bf26e78
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-HasComments: No

[kudu-CR] KUDU-1571. Generate "deprecated" metrics in MDL

[Adar Dembo (Code Review)]

Adar Dembo has posted comments on this change.

Change subject: KUDU-1571. Generate "deprecated" metrics in MDL
..


Patch Set 1: Code-Review-1

I would prefer we don't go down this route. It's adding complexity to our 
release process. I'd rather we do a one-off fix for 0.10.0 (such as manually 
adding the deprecated metrics to the generated MDL), and fix the underlying 
tight coupling in Cloudera Manager. I can volunteer to do that if need be. Then 
we document the limitation for previously released versions of CM and call it a 
day.

It's not a great solution, but it's far cheaper (since it places no onus on 
future Kudu releases), and it improves the quality of CM too.

-- 
To view, visit http://gerrit.cloudera.org:8080/4067
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I121203d781ab593cb94d7248d164cdad7bf26e78
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-HasComments: No

[kudu-CR] KUDU-1571. Generate "deprecated" metrics in MDL

[Todd Lipcon (Code Review)]

Hello Mike Percy, Adar Dembo,

I'd like you to do a code review.  Please visit

http://gerrit.cloudera.org:8080/4067

to review the following change.

Change subject: KUDU-1571. Generate "deprecated" metrics in MDL
..

KUDU-1571. Generate "deprecated" metrics in MDL

This checks in the "--dump_metrics_json" output from already-released
versions, and changes the generate_mdl.py script so that any metrics
which were ever present in an earlier version are carried through to the
current version of the MDL, but marked as deprecated.

This works around an issue in which Cloudera Manager doesn't properly
handle the removal of metrics. Namely, if a dashboard (user-generated or
built-in) references a removed metric, it will produce an NPE.

This will now become part of the release process: after a release, we
should check in the output of 'dump_metrics_json' so that the metrics in
that release will be carried forward.

I verified the results using both the newly included
'check_csd_compatibility' script as well as installing a newly built
0.10.x CSD on a cluster. Whereas the tablet server status page was
previously broken, now it shows the remote bootstrap metrics as
DEPRECATED.

Change-Id: I121203d781ab593cb94d7248d164cdad7bf26e78
---
M build-support/release/rat_exclude_files.txt
A java/kudu-csd/check_csd_compatibility.py
M java/kudu-csd/generate_mdl.py
A java/kudu-csd/old-version-metrics/0.10.0-master.json
A java/kudu-csd/old-version-metrics/0.10.0-tserver.json
A java/kudu-csd/old-version-metrics/0.9.1-master.json
A java/kudu-csd/old-version-metrics/0.9.1-tserver.json
7 files changed, 4,831 insertions(+), 17 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/67/4067/1
-- 
To view, visit http://gerrit.cloudera.org:8080/4067
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I121203d781ab593cb94d7248d164cdad7bf26e78
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Mike Percy 

[kudu-CR] KUDU-1571. Generate "deprecated" metrics in MDL

[Kudu Jenkins (Code Review)]

Kudu Jenkins has posted comments on this change.

Change subject: KUDU-1571. Generate "deprecated" metrics in MDL
..


Patch Set 1:

Build Started http://104.196.14.100/job/kudu-gerrit/3001/

-- 
To view, visit http://gerrit.cloudera.org:8080/4067
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I121203d781ab593cb94d7248d164cdad7bf26e78
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon 
Gerrit-Reviewer: Adar Dembo 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-HasComments: No