It is indeed the presence of non-blob files in the bucket. In pkg/blobserver/s3/enumerate.go line 76 there's a comment asking whether we should error out if unexpected files are encountered.
I assume somewhere there's an assumption that if you ask for 1000 items and get back less than 1000, you've read all of them; I suspect that's a better place to fix the issue. - Daniel On Wednesday, November 2, 2016 at 12:07:06 PM UTC+11, Daniel Heath wrote: > > That's been running for a few days and will take some time yet ( 30+gb on > a 0.2mb/s uplink ). > > The blobs being uploaded are already present in S3, and have been for some > time. > > Separately, when I run: > > ./camtool sync --verbose --src http://localhost:3179/sto-s3/ --dest > http://localhost:3179/bs > > I get: > > 2016/10/30 21:51:14 At source blob > sha1-00003e60f806230ce742737809021a427867ebf1 (1 blobs, 65682 bytes) > 2016/10/30 21:51:14 Total blobs: 987, 57103774 bytes > > I have 13 'other' files in that bucket which makes me think it's only > finding the first 1000 blobs in the s3 bucket and not synching any after > that. > > On Thursday, October 20, 2016 at 3:43:22 AM UTC+11, mpl wrote: >> >> Hi, >> >> To rule out out of order indexing errors, could you please try just >> syncing from one bs to another, instead of using --all ? >> >> To help with that, you could define the targets in your client config >> file, e.g.: >> >> "servers": { >> "src": { >> "server": "http://ec2server:3179/bs/";, >> "auth": "userpass:foo:bar" >> }, >> "dest": { >> "server": "http://localhost:3179/bs/";, >> "auth": "userpass:foo:bar" >> } >> }, >> >> (/bs-packed/ instead of /bs/ if using blobpacked) >> >> And then >> >> camtool sync -src src -dest dest >> >> If that goes well, you can then start your dest server with a -reindex >> and see if things look ok. >> >> On 8 October 2016 at 12:28, Daniel Heath <[email protected]> wrote: >> > So taking one failure as an example: >> > >> > error fetching public key blob >> > sha1-65682c91f928bcf81741ccc8ab4b0efac33a986e: blob was not fully >> indexed >> > because of a missing dependency >> > >> > sha1-65682c91f928bcf81741ccc8ab4b0efac33a986e is a public key blob; the >> > camlistore version that created it was: 2016-08-22-0367de7 >> > >> > Could syncing old -> new affect it? >> > >> > From the web interface for the blob: >> > >> > Blob content >> > >> > No data >> > >> > Indexer metadata >> > >> > { >> > "blobRef": "sha1-65682c91f928bcf81741ccc8ab4b0efac33a986e", >> > "size": 449 >> > } >> > >> > Mutation claims >> > >> > { >> > "claims": null >> > } >> > >> > Referenced by >> > No references >> > >> > >> > >> > >> > On Oct 8, 2016, at 2:00 PM, Daniel Heath <[email protected]> wrote: >> > >> > >> > On Oct 8, 2016, at 1:44 AM, Mathieu Lonjaret <[email protected]> >> > wrote: >> > >> > On 7 October 2016 at 07:58, Daniel Heath <[email protected]> wrote: >> > >> > I have a camlistore instance at home which syncs to an s3 target for >> > backups. >> > >> > I decided to try out the backups and make sure they still work. >> > >> > On a new VPS, I installed camlistore and setup the server config: >> > { >> > "listen": ":3179", >> > "identity": "00ABA6E4", >> > "identitySecretRing": >> > "/home/ec2-user/.config/camlistore/identity-secring.gpg", >> > "blobPath": "/home/ec2-user/var/camlistore/blobs", >> > "packRelated": true, >> > "levelDB": "/home/ec2-user/var/camlistore/index.leveldb", >> > "auth": "<redacted>", >> > "s3": "<redacted>", >> > "dbNames": null >> > } >> > >> > Then I ran: >> > >> > ./camtool sync --all >> > >> > >> > Let me make sure I understand the config above, regarding what you're >> > trying to do. >> > You start with an empty /home/ec2-user/var/camlistore/blobs , "s3" is >> > configured to where your backup blobs are, and you're expecting >> > camtool sync to fill "/home/ec2-user/var/camlistore/blobs" as a >> > destination, from s3 as the source, is that it? >> > >> > >> > Yes, that's correct. >> > >> > ... >> > Destination needs blob: [sha1-000e795be15c30a04b8f51b8e92e62b8bd064e1d; >> 1424 >> > bytes] >> > 2016/10/07 05:47:34 Upload of >> sha1-000e795be15c30a04b8f51b8e92e62b8bd064e1d >> > to destination blobserver failed: Server didn't receive blob. >> > Destination needs blob: [sha1-0010c726455975f0bd4cd7da20b9defc73932dd7; >> 740 >> > bytes] >> > ... >> > Error: sync all failed: 2 errors during sync >> > >> > >> > Just to be sure, have you checked that the problem isn't related to >> > https://github.com/camlistore/camlistore/issues/681 ? i.e. have you >> > tried setting "packRelated": false in the new VPS configuraton? >> > >> > >> > I had not; wiping the server and retrying with packRelated: false >> resulted >> > in the same error. >> > >> > Over in the window running camlistored, I get: >> > >> > ./camlistored >> > ... >> > ... >> > ... >> > 2016/10/07 05:47:34 error fetching public key blob >> > sha1-65682c91f928bcf81741ccc8ab4b0efac33a986e: blob was not fully >> indexed >> > because of a missing dependency >> > ... >> > >> > >> > At this point, I stopped my home camlistore server, copied the leveldb >> > directory and blobs onto the VPS, pointed the VPS instance of >> camlistore at >> > the copy and restarted it. >> > >> > This resulted in a completely blank 'no results found' UI. >> > >> > Running camtool sync --all again just gave me hundreds of pages of >> 'error >> > fetching <sha>: file does not exist' and 'Destination needs blob: >> <sha>'. >> > >> > I am running the latest release from github: >> > [ec2-user@ip-172-31-39-15 camlistore]$ ./camtool -version >> > ./camtool version: 7b78c50007 >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups >> > "Camlistore" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an >> > email to [email protected]. >> > For more options, visit https://groups.google.com/d/optout. >> > >> > >> > -- >> > You received this message because you are subscribed to a topic in the >> > Google Groups "Camlistore" group. >> > To unsubscribe from this topic, visit >> > https://groups.google.com/d/topic/camlistore/_eCFdGJFluA/unsubscribe. >> > To unsubscribe from this group and all its topics, send an email to >> > [email protected]. >> > For more options, visit https://groups.google.com/d/optout. >> > >> > >> > >> > -- >> > You received this message because you are subscribed to a topic in the >> > Google Groups "Camlistore" group. >> > To unsubscribe from this topic, visit >> > https://groups.google.com/d/topic/camlistore/_eCFdGJFluA/unsubscribe. >> > To unsubscribe from this group and all its topics, send an email to >> > [email protected]. >> > For more options, visit https://groups.google.com/d/optout. >> > >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups >> > "Camlistore" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an >> > email to [email protected]. >> > For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "Camlistore" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
