Hi,
To rule out out of order indexing errors, could you please try just
syncing from one bs to another, instead of using --all ?
To help with that, you could define the targets in your client config
file, e.g.:
"servers": {
"src": {
"server": "http://ec2server:3179/bs/";,
"auth": "userpass:foo:bar"
},
"dest": {
"server": "http://localhost:3179/bs/";,
"auth": "userpass:foo:bar"
}
},
(/bs-packed/ instead of /bs/ if using blobpacked)
And then
camtool sync -src src -dest dest
If that goes well, you can then start your dest server with a -reindex
and see if things look ok.
On 8 October 2016 at 12:28, Daniel Heath <[email protected]> wrote:
> So taking one failure as an example:
>
> error fetching public key blob
> sha1-65682c91f928bcf81741ccc8ab4b0efac33a986e: blob was not fully indexed
> because of a missing dependency
>
> sha1-65682c91f928bcf81741ccc8ab4b0efac33a986e is a public key blob; the
> camlistore version that created it was: 2016-08-22-0367de7
>
> Could syncing old -> new affect it?
>
> From the web interface for the blob:
>
> Blob content
>
> No data
>
> Indexer metadata
>
> {
> "blobRef": "sha1-65682c91f928bcf81741ccc8ab4b0efac33a986e",
> "size": 449
> }
>
> Mutation claims
>
> {
> "claims": null
> }
>
> Referenced by
> No references
>
>
>
>
> On Oct 8, 2016, at 2:00 PM, Daniel Heath <[email protected]> wrote:
>
>
> On Oct 8, 2016, at 1:44 AM, Mathieu Lonjaret <[email protected]>
> wrote:
>
> On 7 October 2016 at 07:58, Daniel Heath <[email protected]> wrote:
>
> I have a camlistore instance at home which syncs to an s3 target for
> backups.
>
> I decided to try out the backups and make sure they still work.
>
> On a new VPS, I installed camlistore and setup the server config:
> {
> "listen": ":3179",
> "identity": "00ABA6E4",
> "identitySecretRing":
> "/home/ec2-user/.config/camlistore/identity-secring.gpg",
> "blobPath": "/home/ec2-user/var/camlistore/blobs",
> "packRelated": true,
> "levelDB": "/home/ec2-user/var/camlistore/index.leveldb",
> "auth": "<redacted>",
> "s3": "<redacted>",
> "dbNames": null
> }
>
> Then I ran:
>
> ./camtool sync --all
>
>
> Let me make sure I understand the config above, regarding what you're
> trying to do.
> You start with an empty /home/ec2-user/var/camlistore/blobs , "s3" is
> configured to where your backup blobs are, and you're expecting
> camtool sync to fill "/home/ec2-user/var/camlistore/blobs" as a
> destination, from s3 as the source, is that it?
>
>
> Yes, that's correct.
>
> ...
> Destination needs blob: [sha1-000e795be15c30a04b8f51b8e92e62b8bd064e1d; 1424
> bytes]
> 2016/10/07 05:47:34 Upload of sha1-000e795be15c30a04b8f51b8e92e62b8bd064e1d
> to destination blobserver failed: Server didn't receive blob.
> Destination needs blob: [sha1-0010c726455975f0bd4cd7da20b9defc73932dd7; 740
> bytes]
> ...
> Error: sync all failed: 2 errors during sync
>
>
> Just to be sure, have you checked that the problem isn't related to
> https://github.com/camlistore/camlistore/issues/681 ? i.e. have you
> tried setting "packRelated": false in the new VPS configuraton?
>
>
> I had not; wiping the server and retrying with packRelated: false resulted
> in the same error.
>
> Over in the window running camlistored, I get:
>
> ./camlistored
> ...
> ...
> ...
> 2016/10/07 05:47:34 error fetching public key blob
> sha1-65682c91f928bcf81741ccc8ab4b0efac33a986e: blob was not fully indexed
> because of a missing dependency
> ...
>
>
> At this point, I stopped my home camlistore server, copied the leveldb
> directory and blobs onto the VPS, pointed the VPS instance of camlistore at
> the copy and restarted it.
>
> This resulted in a completely blank 'no results found' UI.
>
> Running camtool sync --all again just gave me hundreds of pages of 'error
> fetching <sha>: file does not exist' and 'Destination needs blob: <sha>'.
>
> I am running the latest release from github:
> [ec2-user@ip-172-31-39-15 camlistore]$ ./camtool -version
> ./camtool version: 7b78c50007
>
> --
> You received this message because you are subscribed to the Google Groups
> "Camlistore" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Camlistore" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/camlistore/_eCFdGJFluA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> [email protected].
> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Camlistore" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/camlistore/_eCFdGJFluA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> [email protected].
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Camlistore" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"Camlistore" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
