Hej, For Thanos questions it's best to visit #thanos channel on CNCF slack.
You can find a link to the slack here https://slack.cncf.io/
TL;DR It would be nice to know if sidecar connects to correct Promethus
server, otherwise it takes labels driectly from config API, so we did not
see any issue on that side. Best way is to also try newest version of
Thanos (v0.13.0)
Bartek
W dniu środa, 1 lipca 2020 00:46:38 UTC+2 użytkownik Thomas Will napisał:
>
> Can anyone please help me out on this issue. It's a blocker for me.
>
> Thanks,
>
> On Tuesday, June 30, 2020 at 11:18:18 PM UTC+5:30, Thomas Will wrote:
>>
>> [image: Screenshot 1942-04-09 at 8.00.08 PM.png]
>> I can see external_labels on prometheus UI.
>>
>> On Tuesday, June 30, 2020 at 8:09:06 AM UTC+5:30, Thomas Will wrote:
>>>
>>>
>>>
>>> "ulid": "01EC1E4WSJVFQWCFH0J0CKRQTQ",
>>> "minTime": 1593480811780,
>>> "maxTime": 1593480900000,
>>> "stats": {
>>> "numSamples": 25450,
>>> "numSeries": 5046,
>>> "numChunks": 5046
>>> },
>>> "compaction": {
>>> "level": 1,
>>> "sources": [
>>> "01EC1E4WSJVFQWCFH0J0CKRQTQ"
>>> ]
>>> },
>>> "version": 1
>>> This is the Prometheus meta.json file. I think external_labels should be
>>> present in this?
>>>
>>> On Tuesday, June 30, 2020 at 6:39:00 AM UTC+5:30, Thomas Will wrote:
>>>>
>>>> Hello guys, while setting up sidecar, I am getting this error.
>>>>
>>>> level=warn ts=2020-06-30T00:59:28.739042462Z caller=sidecar.go:131
>>>> msg="failed
>>>> to fetch initial external labels. Is Prometheus running? Retrying"
>>>> err="request
>>>> config against http://ec2-54-166-121-208.compute-
>>>> level=info ts=2020-06-30T00:59:28.786279597Z caller=shipper.go:200
>>>> msg="upload
>>>> new block" id=01EC194NMFNVXJV3P3ZJV711FX
>>>> level=error ts=2020-06-30T00:59:28.788272247Z caller=shipper.go:165
>>>> msg="shipping failed" block=01EC194NMFNVXJV3P3ZJV711FX err="empty
>>>> external labels are not allowed for Thanos block."
>>>> level=info ts=2020-06-30T00:59:28.797210843Z caller=shipper.go:200
>>>> msg="upload
>>>> new block" id=01EC19C00DSNBYDF0QY6SJ0RCJ
>>>> level=error ts=2020-06-30T00:59:28.798609749Z caller=shipper.go:165
>>>> msg="shipping failed" block=01EC19C00DSNBYDF0QY6SJ0RCJ err="empty
>>>> external labels are not allowed for Thanos block."
>>>> level=info ts=2020-06-30T00:59:28.80781908Z caller=shipper.go:200
>>>> msg="upload
>>>> new block" id=01EC19N4ZDHV3A9ZAV4CW0W50Z
>>>> level=error ts=2020-06-30T00:59:28.809984074Z caller=shipper.go:165
>>>> msg="shipping failed" block=01EC19N4ZDHV3A9ZAV4CW0W50Z err="empty
>>>> external labels are not allowed for Thanos block."
>>>> level=info ts=2020-06-30T00:59:28.871277989Z caller=shipper.go:200
>>>> msg="upload
>>>> new block" id=01EC19Y9XM4EVA6C5XQB6QHA1H
>>>> level=error ts=2020-06-30T00:59:28.873172562Z caller=shipper.go:165
>>>> msg="shipping failed" block=01EC19Y9XM4EVA6C5XQB6QHA1H err="empty
>>>> external labels are not allowed for Thanos block."
>>>> level=info ts=2020-06-30T00:59:28.941585234Z caller=shipper.go:200
>>>> msg="upload
>>>> new block" id=01EC1A7EXED6X31J8AVMME60N2
>>>> level=error ts=2020-06-30T00:59:28.943638781Z caller=shipper.go:165
>>>> msg="shipping failed" block=01EC1A7EXED6X31J8AVMME60N2 err="empty
>>>> external labels are not allowed for Thanos block."
>>>> level=info ts=2020-06-30T00:59:28.959223519Z caller=shipper.go:200
>>>> msg="upload
>>>> new block" id=01EC1AGKWE20FEFYPG2EVQZ973
>>>> level=error ts=2020-06-30T00:59:28.960929593Z caller=shipper.go:165
>>>> msg="shipping failed" block=01EC1AGKWE20FEFYPG2EVQZ973 err="empty
>>>> external labels are not allowed for Thanos block."
>>>> level=info ts=2020-06-30T00:59:28.984357407Z caller=shipper.go:200
>>>> msg="upload
>>>> new block" id=01EC1ASRVEPMF6YDVHWCZ3F9HG
>>>> level=error ts=2020-06-30T00:59:28.986786419Z caller=shipper.go:165
>>>> msg="shipping failed" block=01EC1ASRVEPMF6YDVHWCZ3F9HG err="empty
>>>> external labels are not allowed for Thanos block."
>>>> level=info ts=2020-06-30T00:59:28.999918447Z caller=shipper.go:200
>>>> msg="upload
>>>> new block" id=01EC1B2XTFF24K5QZZCW4FECRW
>>>> level=error ts=2020-06-30T00:59:29.00133826Z caller=shipper.go:165
>>>> msg="shipping
>>>> failed" block=01EC1B2XTFF24K5QZZCW4FECRW err="empty external labels
>>>> are not allowed for Thanos block."
>>>> level=info ts=2020-06-30T00:59:29.020650171Z caller=shipper.go:200
>>>> msg="upload
>>>> new block" id=01EC1BC2SHK8Z9HX3DNAAAPCVM
>>>> level=error ts=2020-06-30T00:59:29.024475442Z caller=shipper.go:165
>>>> msg="shipping failed" block=01EC1BC2SHK8Z9HX3DNAAAPCVM err="empty
>>>> external labels are not allowed for Thanos block."
>>>> level=info ts=2020-06-30T00:59:29.032014206Z caller=shipper.go:200
>>>> msg="upload
>>>> new block" id=01EC1BN7RFWV6C93ZNG9BAYZ5M
>>>> level=error ts=2020-06-30T00:59:29.033591482Z caller=shipper.go:165
>>>> msg="shipping failed" block=01EC1BN7RFWV6C93ZNG9BAYZ5M err="empty
>>>> external labels are not allowed for Thanos block."
>>>>
>>>> sidecar config :-
>>>>
>>>> # thanos sidecar
>>>> ./thanos sidecar \
>>>> --prometheus.url
>>>> "http://ec2-42-685-242-86.compute-1.amazonaws.com:9090"; \
>>>> --tsdb.path "data" \
>>>> --cluster.address "localhost:20056" \
>>>> --http-address "localhost:20057" \
>>>> --grpc-address "localhost:20058" \
>>>> --log.level=debug \
>>>> >> log/sidecar.log 2>&1 &
>>>>
>>>>
>>>>
>>>> prometheus config file :-
>>>>
>>>> global:
>>>> scrape_interval: 15s
>>>> evaluation_interval: 1m
>>>> external_labels:
>>>> monitor: prometheus_monitor
>>>> cluster: 102323
>>>>
>>>>
>>>>
>>>> Can anyone please point out my mistake?
>>>>
>>>> Thanks,
>>>>
>>>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/2d6a445c-5d2b-49c4-b8a6-4675f6319819o%40googlegroups.com.
