Thanks everyone. Upgrading to Prometheus 2.18.2 fixes the issue.
On Saturday, June 20, 2020 at 11:42:37 AM UTC-4, Christian Hoffmann wrote: > > On 6/20/20 5:31 PM, Johny wrote: > > If it is non-compliant endpoint, the problem should appear in both > > versions, isn't it? It is effecting more than one series. The set up is > > in corporate org so I cannot expose end points publicly. > Maybe you can build a small reproducer: Grab your metrics via curl, set > up a webserver to serve the file and let a fresh Prometheus instance > scrape it. If the problem no longer occurs, this would be a chance to > look for differences. > If it does occur, try obfuscating the data as needed and providing the > obfuscated data points so that someone can look into it. > > > I have an prometheus front end instance that remote reads from multiple > > prometheus backends. the time series is sharded across multiple > > backends. The results are also inconsistent in 2.18.1. Sometimes I get > > fewer time series back but what is consistent is the last data point is > > duplicated on all time series. Just switching front end to 2.4 with same > > configuration file fixes the problem. > As you are using remote read, try updating to at least 2.18.2 as Brian > suggested. > > Kind regards, > Christian > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/ad12f71a-ade3-48de-bd78-3d534c41cc43o%40googlegroups.com.
