It's only taken 18 months to get to this answer from the devs. Perhaps someone should be less woke and fucking DOCUMENT THIS SHIT.
On Monday, January 25, 2021 at 8:42:29 AM UTC-7 prommyonapi wrote: > Let me get this straight, poor software design choices result in > destroying any local tmpfs IoT device use case for Prometheus and anyone > that points this out is banned and given some bullshit code of conduct > reason. Anyone using Prometheus on flash memory is writing hundreds of MB > of data an hour or a day, or worse to stop kilobytes of data thus > obliterating the majority of IoT storage in the process. > > Got it. > > Well done. Well done. > > I guess there's no need to use Prometheus at this point for local IoT. > > Good job. > > > > > On Monday, January 25, 2021 at 7:39:10 AM UTC-7 [email protected] > wrote: > >> Hi, >> >> The experimental flag to allow limiting disk usage was mostly meant for >> larger disk usages than a couple of hundred MB, but there's an opportunity >> to clarify that in the flag description. Prometheus will need to store at >> least a couple of hundred MB due to the way it segments and preallocates >> data (current TSDB devs would be able to provide more detail here). Indeed >> this behavior changed with 2.x, as the entire storage engine got swapped >> out for a much better one (except for tiny use cases like this). >> >> Please refrain from personal insults, as it's against our code of conduct >> (https://github.com/cncf/foundation/blob/master/code-of-conduct.md). >> >> Regards, >> Julius >> >> On Mon, Jan 25, 2021 at 2:59 PM prommyonapi <[email protected]> wrote: >> >>> https://github.com/prometheus/prometheus/issues/8406 >>> >>> pi@raspberrypi:~ $ cat /etc/fstab >>> proc /proc proc defaults 0 0 >>> PARTUUID=6c586e13-01 /boot vfat defaults,noatime 0 2 >>> PARTUUID=6c586e13-02 / ext4 >>> defaults,noatime,commit=600,errors=remount-ro 0 1 >>> tmpfs /tmp tmpfs defaults,noatime,nosuid 0 0 >>> tmpfs /var/log tmpfs defaults,noatime,nosuid,size=50M 0 0 >>> tmpfs /prometheus-tmpfs tmpfs defaults,nosuid,size=200M >>> >>> *What did you do?* >>> Prometheus + node_exporter on Buster >>> saving prometheus in TMPFS to save SD writes >>> >>> *What did you expect to see?* >>> It working >>> >>> *What did you see instead? Under which circumstances?* >>> faliures: >>> >>> Jan 25 03:12:23 raspberrypi prometheus[6999]: level=error >>> ts=2021-01-25T03:12:23.027Z caller=db.go:745 component=tsdb msg="compaction >>> failed" err="compact head: persist head block: populate block: write >>> chunks: preallocate: no space left on device" >>> Jan 25 03:13:23 raspberrypi prometheus[6999]: level=error >>> ts=2021-01-25T03:13:23.069Z caller=db.go:745 component=tsdb msg="compaction >>> failed" err="compact head: persist head block: populate block: write >>> chunks: preallocate: no space left on device" >>> Jan 25 03:14:23 raspberrypi prometheus[6999]: level=error >>> ts=2021-01-25T03:14:23.150Z caller=db.go:745 component=tsdb msg="compaction >>> failed" err="compact head: persist head block: populate block: write >>> chunks: preallocate: no space left on device" >>> Jan 25 03:15:23 raspberrypi prometheus[6999]: level=error >>> ts=2021-01-25T03:15:23.186Z caller=db.go:745 component=tsdb msg="compaction >>> failed" err="compact head: persist head block: populate block: write >>> chunks: preallocate: no space left on device" >>> Jan 25 03:16:23 raspberrypi prometheus[6999]: level=error >>> ts=2021-01-25T03:16:23.217Z caller=db.go:745 component=tsdb msg="compaction >>> failed" err="compact head: persist head block: populate block: write >>> chunks: preallocate: no space left on device" >>> Jan 25 03:17:23 raspberrypi prometheus[6999]: level=error >>> ts=2021-01-25T03:17:23.254Z caller=db.go:745 component=tsdb msg="compaction >>> failed" err="compact head: persist head block: populate block: write >>> chunks: preallocate: no space left on device" >>> Jan 25 03:18:23 raspberrypi prometheus[6999]: level=error >>> ts=2021-01-25T03:18:23.316Z caller=db.go:745 component=tsdb msg="compaction >>> failed" err="compact head: persist head block: populate block: write >>> chunks: preallocate: no space left on device" >>> Jan 25 03:19:23 raspberrypi prometheus[6999]: level=error >>> ts=2021-01-25T03:19:23.391Z caller=db.go:745 component=tsdb msg="compaction >>> failed" err="compact head: persist head block: populate block: write >>> chunks: preallocate: no space left on device" >>> Jan 25 03:20:23 raspberrypi prometheus[6999]: level=error >>> ts=2021-01-25T03:20:23.414Z caller=db.go:745 component=tsdb msg="compaction >>> failed" err="compact head: persist head block: populate block: write >>> chunks: preallocate: no space left on device" >>> Jan 25 03:21:23 raspberrypi prometheus[6999]: level=error >>> ts=2021-01-25T03:21:23.430Z caller=db.go:745 component=tsdb msg="compaction >>> failed" err="compact head: persist head block: populate block: write >>> chunks: preallocate: no space left on device" >>> >>> *Environment* >>> Raspbian Buster >>> >>> - System information: >>> >>> Linux 5.4.83-v7l+ armv7l >>> >>> - Prometheus version: >>> >>> prometheus, version 2.24.1 (branch: HEAD, revision: e448727 >>> <https://github.com/prometheus/prometheus/commit/e4487274853c587717006eeda8804e597d120340> >>> ) >>> build user: root@0b5231a0de0f >>> build date: 20210120-01:04:17 >>> go version: go1.15.6 >>> platform: linux/arm >>> >>> - Prometheus configuration file: >>> >>> - job_name: 'node' scrape_interval: 60s scrape_timeout: 10s >>> static_configs: - targets: ['localhost:9100'] - job_name: 'dump1090' >>> scrape_interval: 60s scrape_timeout: 10s static_configs: - targets: >>> ['localhost:9105'] =/usr/bin/prometheus >>> --config.file=/etc/prometheus/prometheus.yml \ >>> --storage.tsdb.path="/prometheus-tmpfs/prometheus" \ >>> --storage.tsdb.min-block-duration=30m --storage.tsdb.max-block-duration=1h >>> \ --storage.tsdb.wal-compression \ --storage.tsdb.retention.time=1d >>> --storage.tsdb.retention.size=50MB \ --storage.tsdb.no-lockfile \ >>> --web.enable-admin-api [image: Screenshot_2021-01-24_20-22-42.png] >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Prometheus Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/prometheus-users/510fc30b-c42f-4884-b9e7-b3257540db5dn%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/prometheus-users/510fc30b-c42f-4884-b9e7-b3257540db5dn%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> >> >> -- >> Julius Volz >> PromLabs - promlabs.com >> > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/6a7836ef-f19c-44ca-9b9a-ac78f2b74706n%40googlegroups.com.
