I am a little concerned about "tests that are okay to fail" … especially since merging one PR with that means we have no signal for subsequent PRs. Meanwhile corthanos might adjust their usage, breaking older branches that don't even touch this (solvable with rebasing but adds additional round trips).
How could we mark these tests as less important? How could we mark expected failures? Are there other testing strategies that would maintain the "this breaks interfaces" signal without coupling things too closely? /MR On Sat, 21 Mar 2020, 22:02 Julien Pivotto, <[email protected]> wrote: > Hi there, > > Would you think that it would be valuable to test the pull requests with > the third parties that use our codebase (corthanos)? > > What I imagine would be that those tests would run tests against > corthanos master, update the prometheus dependencies to the commit and > the PR, and report back test results. > > I don't think failures over interface changes should prevent us to > merge, but would it bring extra coverage and information regarding to > our code? Would that be valuable? > > -- > (o- Julien Pivotto > //\ Open-Source Consultant > V_/_ Inuits - https://www.inuits.eu > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/20200321210234.GA18497%40oxygen > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CAFU3N5UTSTNXp7sadvZc8%3D8tuwd5utWdG4t19%3DRtce1x4OJ73w%40mail.gmail.com.
