Hi There ,
We have integrated AWS timestream to prometheus via the remote write & read
settings. We have the below setup,
1. Prometheus is a pod in EKS cluster
2. We have prometheus-timestream-adapter as well deployed in a pod
3. Prometheus remote read & write is configured to this adapter.
4. AWS timestream has a table setup with 24hrs memory & 7 days magnetic
retention
All connectivity, setup etc, looks good apart from the prometheus seems to
be sending old record sets. Hence we get this below message in the adapter
and it fails to store the data in timestream
{"level":"warn","ts":1610368736.6555033,"caller":"prometheus-timestream-adapter/timestream.go:131","msg":"Error
sending samples to remote storage","err":"RejectedRecordsException: One or
more records have been rejected. See RejectedRecords for details.\n{\n
RespMetadata: {\n StatusCode: 419,\n RequestID:
\"AHHJ7P7BBH4TIUEGMKOPO42FQQ\"\n },\n Message_: \"One or more records
have been rejected. See RejectedRecords for details.\",\n RejectedRecords:
[\n {\n Reason: \"*The record timestamp is outside the time range
*[2021-01-10T12:33:56.651Z,
2021-01-11T13:13:56.651Z) of the memory store.\",\n RecordIndex:
0\n }
Looking forward for some suggestions ...
Regards,
Vinod.
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/44e83e0d-e8c8-47f5-82c8-d3b8a9b9fd9bn%40googlegroups.com.
