Interesting idea. I think it's something to be considered by our explorations like modularize. For each change of exported package we could just trigger some jobs in Thanos and Cortex CI which will try to upgrade dep and run tests. But still, it will be pain to maintain in terms of interface change, etc.
But yes, tests which you can ignore are rather an anti pattern, just noise. Kind Regards, Bartek On Sun, 22 Mar 2020, 12:06 'Matthias Rampke' via Prometheus Developers, < [email protected]> wrote: > I am a little concerned about "tests that are okay to fail" … especially > since merging one PR with that means we have no signal for subsequent PRs. > Meanwhile corthanos might adjust their usage, breaking older branches that > don't even touch this (solvable with rebasing but adds additional round > trips). > > How could we mark these tests as less important? How could we mark > expected failures? > > Are there other testing strategies that would maintain the "this breaks > interfaces" signal without coupling things too closely? > > /MR > > On Sat, 21 Mar 2020, 22:02 Julien Pivotto, <[email protected]> wrote: > >> Hi there, >> >> Would you think that it would be valuable to test the pull requests with >> the third parties that use our codebase (corthanos)? >> >> What I imagine would be that those tests would run tests against >> corthanos master, update the prometheus dependencies to the commit and >> the PR, and report back test results. >> >> I don't think failures over interface changes should prevent us to >> merge, but would it bring extra coverage and information regarding to >> our code? Would that be valuable? >> >> -- >> (o- Julien Pivotto >> //\ Open-Source Consultant >> V_/_ Inuits - https://www.inuits.eu >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Prometheus Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-developers/20200321210234.GA18497%40oxygen >> . >> > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/CAFU3N5UTSTNXp7sadvZc8%3D8tuwd5utWdG4t19%3DRtce1x4OJ73w%40mail.gmail.com > <https://groups.google.com/d/msgid/prometheus-developers/CAFU3N5UTSTNXp7sadvZc8%3D8tuwd5utWdG4t19%3DRtce1x4OJ73w%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CAMssQwbNxBE9-7iU5_MrEh2N2sGj5iKG_OOqY_rFvFxds1oyEQ%40mail.gmail.com.
