Adrian, we've just merged https://camlistore-review.googlesource.com/c/camlistore/+/15627 Does it work for you now?
Also, if it does, and if you know where that blob comes from (which file), could please try to upload it to a dummy server with an empty index? As Brad said, I think we already now reject bogus GPS (and you probably got that blob in before we did), but it would be nice to know for sure. Better yet, if you find that file and you don't mind sharing it with me, I could try that out for you. On 24 April 2018 at 17:19, Brad Fitzpatrick <[email protected]> wrote: > Hah. I think we should try to parse the value (even if it's in 1e+02 > form), and then ignore it if it's out of range. > > > On Tue, Apr 24, 2018 at 5:18 PM, Adrian Tritschler <[email protected]> wrote: >> Digging into it a bit further. >> >> Something looks unpleasant in the blob that's reported in the panic, take a >> look at the longitude! >> >> /opt/camlistore/blobs/sha1$ sudo exiftool -s >> 42/c3/sha1-42c3170acbd238e446bbe291683d356937799a02.dat|grep GPS >> GPSVersionID : 2.2.0.0 >> GPSLatitudeRef : North >> GPSLongitudeRef : West >> GPSAltitudeRef : Above Sea Level >> GPSLatitude : 45 deg 32' 38.27" N >> GPSLongitude : 4294966352 deg 0' 0.00" W >> GPSPosition : 45 deg 32' 38.27" N, 4294966352 deg 0' >> 0.00" W >> >> Adrian >> >> >> On Wed, 25 Apr 2018 at 03:13 Brad Fitzpatrick <[email protected]> wrote: >>> >>> Adrian, >>> >>> Like Mathieu said, the panic is fixed. And don't worry about the >>> "blobpacked: 0 large blobs found in index, 3608 missing from index" >>> part. That's just because we added a new thing to the index to make >>> start-up integrity checks faster. You didn't lose anything. Let it do >>> its recommended full repair once and you'll have the index populated >>> for future start-ups. >>> >>> >>> >>> On Mon, Apr 23, 2018 at 11:03 PM, Adrian Tritschler >>> <[email protected]> wrote: >>> > Following a power outage at home, my long-running camlistored was killed >>> > and >>> > has (probably) written all over its indexes. >>> > >>> > After I moved $HOME/.config/camlistore to perkeep and changed reference >>> > to >>> > gpg keyring, starting the current perkeepd starts its self checking and >>> > then >>> > panics: >>> > >>> > 2018/04/24 15:49:57 Starting perkeepd version 2018-04-23-2286b6442b; Go >>> > go1.10 (linux/amd64) >>> > 2018/04/24 15:49:57 Starting to listen on http://0.0.0.0:80 (for Let's >>> > Encrypt challenges) >>> > 2018/04/24 15:49:57 Starting to listen on https://localhost:443 >>> > 2018/04/24 15:50:09 blobpacked: checking integrity of packed blobs >>> > against >>> > index... >>> > 2018/04/24 15:56:56 blobpacked: 0 large blobs found in index, 3608 >>> > missing >>> > from index >>> > 2018/04/24 15:56:56 sample missing large blob: >>> > sha1-0004d482fff38bbdff6c4263b2d70e5602957b15 >>> > 2018/04/24 15:56:56 sample missing large blob: >>> > sha1-0017a08d36a4a146dde89cb2130f469837726f9a >>> > 2018/04/24 15:56:56 sample missing large blob: >>> > sha1-0019a4797e3b483e34c52a6f9331873097f3b761 >>> > 2018/04/24 15:56:56 sample missing large blob: >>> > sha1-0035597bf6ae264bd89a15943e11b8b15d2178d1 >>> > 2018/04/24 15:56:56 sample missing large blob: >>> > sha1-004b8239d7e912838274f17388280dd1a5348744 >>> > 2018/04/24 15:56:56 sample missing large blob: >>> > sha1-005f3c8765f0891a5ca60cd354c445bdd72f3571 >>> > 2018/04/24 15:56:56 sample missing large blob: >>> > sha1-00746496ab7228460ec7d1e9fb244e906ed84d1b >>> > 2018/04/24 15:56:56 sample missing large blob: >>> > sha1-007feccb1fc84966e0a162d3e648de5cb584a69d >>> > 2018/04/24 15:56:56 sample missing large blob: >>> > sha1-0089cba2bb906a8558082fa94a5f9bce186905bb >>> > 2018/04/24 15:56:56 sample missing large blob: >>> > sha1-008b6ecaaddc8e62ee510af43a9d7f31879adb52 >>> > 2018/04/24 15:56:56 Caught panic installer handlers: Done called more >>> > than >>> > Start >>> > goroutine 1 [running]: >>> > runtime/debug.Stack(0xc4200c2190, 0x2, 0xc420969a80) >>> > /usr/local/go/src/runtime/debug/stack.go:24 +0xa7 >>> > runtime/debug.PrintStack() >>> > /usr/local/go/src/runtime/debug/stack.go:16 +0x22 >>> > perkeep.org/pkg/serverinit.(*Config).InstallHandlers.func1(0xc4207a9c80) >>> > >>> > /home/ajft/src/go/src/perkeep.org/pkg/serverinit/serverinit.go:548 >>> > +0x9e >>> > panic(0xf101e0, 0x1796ef0) >>> > /usr/local/go/src/runtime/panic.go:505 +0x229 >>> > >>> > perkeep.org/pkg/serverinit.(*handlerLoader).setupHandler.func1(0xc420de5050, >>> > 0xc420664500, 0x11600fd, 0xe) >>> > >>> > /home/ajft/src/go/src/perkeep.org/pkg/serverinit/serverinit.go:309 >>> > +0x198 >>> > panic(0xf101e0, 0x1796ef0) >>> > /usr/local/go/src/runtime/panic.go:505 +0x229 >>> > >>> > perkeep.org/pkg/serverinit.(*handlerLoader).setupHandler.func1(0xc420de51d0, >>> > 0xc420664500, 0x1155125, 0x4) >>> > >>> > /home/ajft/src/go/src/perkeep.org/pkg/serverinit/serverinit.go:309 >>> > +0x198 >>> > panic(0xf101e0, 0x1796ef0) >>> > /usr/local/go/src/runtime/panic.go:505 +0x229 >>> > perkeep.org/vendor/go4.org/syncutil.(*Gate).Done(0xc4202d8480) >>> > >>> > /home/ajft/src/go/src/perkeep.org/vendor/go4.org/syncutil/gate.go:39 >>> > +0x66 >>> > perkeep.org/vendor/go4.org/syncutil.(*Gate).Done-fm() >>> > /home/ajft/src/go/src/perkeep.org/pkg/sorted/sqlkv/sqlkv.go:212 >>> > +0x2a >>> > perkeep.org/pkg/sorted/sqlkv.(*iter).Close(0xc4209da000, 0x11549a3, 0x2) >>> > /home/ajft/src/go/src/perkeep.org/pkg/sorted/sqlkv/sqlkv.go:279 >>> > +0xad >>> > >>> > perkeep.org/pkg/blobserver/blobpacked.(*storage).checkLargeIntegrity(0xc42067aa80, >>> > 0x1, 0x1799680, 0xc420af9150) >>> > >>> > >>> > /home/ajft/src/go/src/perkeep.org/pkg/blobserver/blobpacked/blobpacked.go:428 >>> > +0x533 >>> > perkeep.org/pkg/blobserver/blobpacked.newFromConfig(0x17ab240, >>> > 0xc420664500, >>> > 0xc420de49c0, 0x0, 0x0, 0x0, 0x0) >>> > >>> > >>> > /home/ajft/src/go/src/perkeep.org/pkg/blobserver/blobpacked/blobpacked.go:353 >>> > +0x96e >>> > perkeep.org/pkg/blobserver.CreateStorage(0x1165f6b, 0xa, 0x17ab240, >>> > 0xc420664500, 0xc420de49c0, 0x4, 0x1, 0x2, 0x1a) >>> > /home/ajft/src/go/src/perkeep.org/pkg/blobserver/registry.go:111 >>> > +0xc3 >>> > perkeep.org/pkg/serverinit.(*handlerLoader).setupHandler(0xc420664500, >>> > 0x1155125, 0x4) >>> > >>> > /home/ajft/src/go/src/perkeep.org/pkg/serverinit/serverinit.go:322 >>> > +0x25c >>> > perkeep.org/pkg/serverinit.(*handlerLoader).GetStorage(0xc420664500, >>> > 0x1155125, 0x4, 0x2, 0x2, 0x0, 0x40) >>> > >>> > /home/ajft/src/go/src/perkeep.org/pkg/serverinit/serverinit.go:252 >>> > +0x43 >>> > perkeep.org/pkg/blobserver/replica.newFromConfig(0x17ab240, >>> > 0xc420664500, >>> > 0xc420de4b10, 0x7, 0xc4204e0870, 0xc422d71901, 0x0) >>> > >>> > /home/ajft/src/go/src/perkeep.org/pkg/blobserver/replica/replica.go:115 >>> > +0x26c >>> > perkeep.org/pkg/blobserver.CreateStorage(0x116209f, 0x7, 0x17ab240, >>> > 0xc420664500, 0xc420de4b10, 0xe, 0x0, 0x1, 0x30) >>> > /home/ajft/src/go/src/perkeep.org/pkg/blobserver/registry.go:111 >>> > +0xc3 >>> > perkeep.org/pkg/serverinit.(*handlerLoader).setupHandler(0xc420664500, >>> > 0x11600fd, 0xe) >>> > >>> > /home/ajft/src/go/src/perkeep.org/pkg/serverinit/serverinit.go:322 >>> > +0x25c >>> > perkeep.org/pkg/serverinit.(*handlerLoader).setupAll(0xc420664500) >>> > >>> > /home/ajft/src/go/src/perkeep.org/pkg/serverinit/serverinit.go:232 >>> > +0x92 >>> > perkeep.org/pkg/serverinit.(*Config).InstallHandlers(0xc420977f80, >>> > 0x179a5e0, 0xc422d4cfa0, 0xc420de2d20, 0x1f, 0x0, 0x0, 0x0, 0x0, 0x0, >>> > ...) >>> > >>> > /home/ajft/src/go/src/perkeep.org/pkg/serverinit/serverinit.go:617 >>> > +0xa0e >>> > main.Main(0x0, 0x0) >>> > >>> > /home/ajft/src/go/src/perkeep.org/server/perkeepd/camlistored.go:765 >>> > +0x4b5 >>> > main.main() >>> > >>> > /home/ajft/src/go/src/perkeep.org/server/perkeepd/camlistored.go:708 >>> > +0x29 >>> > Error parsing config: Caught panic: Done called more than Start >>> > >>> > -- >>> > You received this message because you are subscribed to the Google >>> > Groups >>> > "Perkeep" group. >>> > To unsubscribe from this group and stop receiving emails from it, send >>> > an >>> > email to [email protected]. >>> > For more options, visit https://groups.google.com/d/optout. >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Perkeep" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Perkeep" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "Perkeep" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Perkeep" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
