villebro commented on PR #19659: URL: https://github.com/apache/superset/pull/19659#issuecomment-1099911031
> villebro zhaoyongjie this fixes a fairly major regression (#19116) in my opinion, i.e., we have users reporting incorrect results which triggers a SEV-0 at Airbnb given that the most egregious error for a BI tool is actually reporting misinformation which is far worse than no information and/or an error. > > I was wondering whether Preset et al. were planning to do a post mortem (or similar) for this? The topic of post mortems might be a good topic to raise during the Superset Operational Model—Quality track. @john-bodley it would be great to get guidelines for what types of regressions require more formal post mortems. If we had these types of guidelines in place, it would be easy to take that into consideration during code review. For instance, anything that affects calculation logic requires sign off from more committers or similar. Also, @zhaoyongjie raised a good point about the previous tests not being specific enough. I think if we strive to make as isolated tests as possible that test for "obvious" cases (in this case that a change is +100, not -100, and that a going from 100 to 200 is +100 % and not, say, +50 % due to using an incorrect denominator in the calculation), the risk for these types of flipped sign regressions would not occur as easily. I'm happy to jump on a call to discuss this or assist in formalizing guidelines for this so we avoid similar problems going forward. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
