To gain a bit better understanding of influx: what is influx writing into the wal files when it is restarted and not receiving any requests yet? I thought that the wal files are populated with data coming into influx before they are stored in the data directory.
On Monday, August 1, 2016 at 5:50:07 PM UTC+2, Jan Kis wrote: > > Removing the latest shard from > directory /data/influxdb/data/mydb/default/15 solved the issue but I am > still interested in whether there is a bug or whether we have done > something stupid. > > On Monday, August 1, 2016 at 1:25:49 PM UTC+2, Jan Kis wrote: >> >> I tried to move away the whole wal directory. That did not help I still >> see the same exception. >> >> On Monday, August 1, 2016 at 12:58:31 PM UTC+2, Jan Kis wrote: >>> >>> Hi Guys, >>> >>> today we experienced the "panic: runtime error: slice bounds out of >>> range" in influxdb 0.13. This happened while we were dropping some >>> measurements. Currently, we can not get influx up and running. We see the >>> same error on restart. I include a more detailed snippet from the logs >>> below. >>> >>> Any help is appreciated, >>> Thank you >>> >>> [tsm1] 2016/08/01 12:45:40 beginning level 2 compaction of group 0, 1 >>> TSM files >>> [tsm1] 2016/08/01 12:45:40 compacting level 2 group (0) >>> /data/influxdb/data/mydb/default/13/000000164-000000002.tsm (#0) >>> [shard] 2016/08/01 12:45:40 /data/influxdb/data/mydb/default/126 >>> database index loaded in 66.51752ms >>> [store] 2016/08/01 12:45:40 /data/influxdb/data/mydb/default/126 opened >>> in 70.212921ms >>> [tsm1wal] 2016/08/01 12:45:40 tsm1 WAL starting with 10485760 segment >>> size >>> [tsm1wal] 2016/08/01 12:45:40 tsm1 WAL writing to >>> /data/influxdb/wal/mydb/default/134 >>> [filestore] 2016/08/01 12:45:40 >>> /data/influxdb/data/mydb/default/134/000000090-000000003.tsm (#0) opened in >>> 1.669208ms >>> [cacheloader] 2016/08/01 12:45:40 reading file >>> /data/influxdb/wal/mydb/default/134/_00266.wal, size 144183 >>> [cacheloader] 2016/08/01 12:45:40 reading file >>> /data/influxdb/wal/mydb/default/134/_00296.wal, size 0 >>> [shard] 2016/08/01 12:45:40 /data/influxdb/data/mydb/default/118 >>> database index loaded in 76.444773ms >>> [store] 2016/08/01 12:45:40 /data/influxdb/data/mydb/default/118 opened >>> in 95.187655ms >>> [tsm1wal] 2016/08/01 12:45:40 tsm1 WAL starting with 10485760 segment >>> size >>> [tsm1wal] 2016/08/01 12:45:40 tsm1 WAL writing to >>> /data/influxdb/wal/mydb/default/14 >>> [filestore] 2016/08/01 12:45:40 >>> /data/influxdb/data/mydb/default/14/000000148-000000004.tsm (#1) opened in >>> 67.574µs >>> [filestore] 2016/08/01 12:45:40 >>> /data/influxdb/data/mydb/default/14/000000147-000000004.tsm (#0) opened in >>> 1.611877ms >>> [cacheloader] 2016/08/01 12:45:40 reading file >>> /data/influxdb/wal/mydb/default/14/_00302.wal, size 0 >>> [shard] 2016/08/01 12:45:40 /data/influxdb/data/mydb/default/13 database >>> index loaded in 66.423537ms >>> [store] 2016/08/01 12:45:40 /data/influxdb/data/mydb/default/13 opened >>> in 70.154698ms >>> [tsm1wal] 2016/08/01 12:45:40 tsm1 WAL starting with 10485760 segment >>> size >>> [tsm1wal] 2016/08/01 12:45:40 tsm1 WAL writing to >>> /data/influxdb/wal/mydb/default/142 >>> [filestore] 2016/08/01 12:45:40 >>> /data/influxdb/data/mydb/default/142/000000086-000000003.tsm (#0) opened in >>> 2.154482ms >>> [cacheloader] 2016/08/01 12:45:40 reading file >>> /data/influxdb/wal/mydb/default/142/_00325.wal, size 116400 >>> [cacheloader] 2016/08/01 12:45:40 reading file >>> /data/influxdb/wal/mydb/default/142/_00340.wal, size 0 >>> [shard] 2016/08/01 12:45:40 /data/influxdb/data/mydb/default/14 database >>> index loaded in 66.356981ms >>> [store] 2016/08/01 12:45:40 /data/influxdb/data/mydb/default/14 opened >>> in 68.922787ms >>> [tsm1wal] 2016/08/01 12:45:40 tsm1 WAL starting with 10485760 segment >>> size >>> [tsm1wal] 2016/08/01 12:45:40 tsm1 WAL writing to >>> /data/influxdb/wal/mydb/default/15 >>> panic: runtime error: slice bounds out of range >>> >>> goroutine 496 [running]: >>> panic(0xb5b240, 0xc82000e040) >>> /usr/local/go/src/runtime/panic.go:481 +0x3e6 >>> >>> github.com/influxdata/influxdb/tsdb/engine/tsm1.(*mmapAccessor).init(0xc8235f78c0, >>> >>> 0x0, 0x0, 0x0) >>> /root/go/src/ >>> github.com/influxdata/influxdb/tsdb/engine/tsm1/reader.go:879 +0x5a8 >>> github.com/influxdata/influxdb/tsdb/engine/tsm1.NewTSMReader(0xc825842030, >>> 0xc80b8156de, 0x0, 0x0) >>> /root/go/src/ >>> github.com/influxdata/influxdb/tsdb/engine/tsm1/reader.go:182 +0x204 >>> >>> github.com/influxdata/influxdb/tsdb/engine/tsm1.(*FileStore).Open.func1(0xc824c1c180, >>> >>> 0xc8236cb080, 0x2, 0xc825842030) >>> /root/go/src/ >>> github.com/influxdata/influxdb/tsdb/engine/tsm1/file_store.go:332 +0x77 >>> created by >>> github.com/influxdata/influxdb/tsdb/engine/tsm1.(*FileStore).Open >>> /root/go/src/ >>> github.com/influxdata/influxdb/tsdb/engine/tsm1/file_store.go:342 >>> +0x749 >>> >> -- Remember to include the InfluxDB version number with all issue reports --- You received this message because you are subscribed to the Google Groups "InfluxDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/5f0edf0a-227b-436d-bc82-05332f53c191%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
