Re: Huge cache safe to delete?

You can nuke the cache directory. It's just a cache. We should probably have some automatic size management for it. File a bug? https://github.com/perkeep/perkeep/issues (Btw, we use the perkeep@ mailing list now, not camlistore@) On Mon, Aug 20, 2018 at 1:40 PM Viktor Ogeman wrote: > > Hi agai

Re: Which importer is the best example to follow?

I guess it depends on what type of content you want to import, and which auth protocol (if any) the site uses. On Thu, May 24, 2018 at 1:36 PM, Joshua Poehls wrote: > If I were to play around with creating an importer, which existing importer > should I reference as a representation of the curre

Re: OT: new mails

We're working on a Gerrit migration to rename it. Maybe something went wrong. I'll ask the people working on it. Sorry for any spam. On Thu, Mar 15, 2018 at 11:52 PM, Gerard wrote: > I mean from Gerrit. > > > On Friday, March 16, 2018 at 7:51:26 AM UTC+1, Gerard wrote: >> >> Why is it that I am

Perkeep status update (2018-03)

Keepers, This is your somewhat-regular status update on what's new in Perkeep. * the mailing list has been moved from camlistore@googlegroups.com to perk...@googlegroups.com. We're cross-posting this status update, but be sure to move to https://groups.google.com/group/perkeep if you haven't alre

Re: Hello and multi-stage Docker build

Nice! I can look more tomorrow or Friday, but sounds great. We've wanted this for awhile now (e.g. https://github.com/perkeep/perkeep/issues/986) Thanks! On Wed, Feb 21, 2018 at 7:40 PM, James Hillyerd wrote: > Hi! Long time gopher first time keeper. > > I wanted to run Perkeep on my NAS (in

Re: State of the Keep: Feb 2018

Oh, we also applied to present at Linuxfest Northwest again, but they haven't announced who's accepted yet. If approved, we'd also do a hackathon around the same time. Or we might anyway. On Fri, Feb 2, 2018 at 6:36 PM, Brad Fitzpatrick wrote: > Keepers, > > It's

State of the Keep: Feb 2018

Keepers, It's been awhile since we posted a Perkeep status update. (Actually, we never have, because the last time we did a status update it was called Camlistore :)) Anyway, here are some notable things recently done or in progress or on the horizon: *# Rename from Camlistore to Perkeep* ( htt

Re: Migrate mailing list?

iscourse.org/ > > > On Jan 30, 2018, at 12:56, Brad Fitzpatrick wrote: > > Or do we want to take this opportunity to decide whether mailing lists are > still the right answer? > > I see a lot of people use this forum software; > > https://forum.golangbridge.org/ >

Re: Migrate mailing list?

Or do we want to take this opportunity to decide whether mailing lists are still the right answer? I see a lot of people use this forum software; https://forum.golangbridge.org/ https://community.home-assistant.io/ Not sure what the software is called, though. Or maybe those are different. Do w

Re: Web UI broken with sha224

Could you start on web tests? On Wed, Jan 17, 2018 at 7:00 AM, Eric Drechsel wrote: > But yeah, webdriver tests, let's do that :D > > On Wed, Jan 17, 2018 at 6:59 AM, Eric Drechsel wrote: > >> Actually, realized Atilla is already WIP on this with that CL, don't want >> to duplicate work >> >>

Re: Web UI broken with sha224

Sorry about that. I did run tests, but we don't have Selenium/WebDriver/ChromeDriver tests for the UI. I'm traveling this week meeting family, so I only have pockets of free time. Any help debugging this would be great. I'd rather roll forward (just debug & fix this) than roll back (revert back t

Re: Per-contributor focuses for next major release, hack sessions

Rather than explicitly track generations, I'd just do consistent hashing, where each disk has a weight proportional to its size (by default) or explicit configuration. On read, try its ideal location. If you miss, search N shards in parallel looking for it. Optionally enqueue a migration task. On

Re: Per-contributor focuses for next major release, hack sessions

Also, just doing lots of cleanups, fixing ancient & embarrassing things in the code. :) On Sun, Jan 7, 2018 at 9:45 AM, Brad Fitzpatrick wrote: > Good idea! > > First off, let's agree on the case of the new name: Perkeep, without a > capital 'K'. :-) >

Re: Per-contributor focuses for next major release, hack sessions

Good idea! First off, let's agree on the case of the new name: Perkeep, without a capital 'K'. :-) My priorities in the short term: -- finish the project rename -- be more aggressive at unblocking people and reviewing code, or fixing it up when it's stalled but close -- work on the website, espe

Re: heads-up: are you using MySQL (or other DBMS)?

We should at least fix the launcher case, which should be easy. We can special-case that, detecting we're a launcher config and changing the automatic config it builds at startup to use the old db names. On Thu, Jan 4, 2018 at 7:47 AM, Mathieu Lonjaret wrote: > Hi, > > as of rev 7eda9fd5027fda8

Re: Custom attribute collisions

; permanode to keep > attribute keys namespaced as well as values. > > On Wednesday, January 3, 2018 at 12:23:25 PM UTC-5, Brad Fitzpatrick wrote: >> >> Best practice is to prefix custom attributes with a domain name, like: >> >> https://github.com/camlistore/camlistor

Re: Custom attribute collisions

Best practice is to prefix custom attributes with a domain name, like: https://github.com/camlistore/camlistore/blob/master/pkg/importer/foursquare/foursquare.go (search for "foursquare.com:") On Wed, Jan 3, 2018 at 9:15 AM, Angel wrote: > Hello, > > I have a few ideas for apps I'd like to se

Re: website: update logo/mascot/colors

On Tue, Jan 2, 2018 at 3:33 PM, Mathieu Lonjaret wrote: > you mean, as in something like > https://camlistore-review.googlesource.com/c/camlistore/+/10006 , but > more finely-grained? Something like that. But that one could use "Keep" for now. Later I imagine something like Google Photo's shari

Re: website: update logo/mascot/colors

aret wrote: > And once we have an animated Keepy, we can use it to replace pudgy in > the web UI. > Actually there probably are a couple of non-animated places in the web > UI where we can already do that. > > On 3 January 2018 at 00:24, Brad Fitzpatrick wrote: > > I sent yo

Re: website: update logo/mascot/colors

g > was coming soon. Do you have additional changes I should hold off for, or > do you mind me jumping in at this point? > > On Tue, Jan 2, 2018 at 12:48 PM, wrote: > >> >> >> https://camlistore.googlesource.com/camlistore/+/9427d6eef40 >> 018f465ca3309df5

Re: Camlistore is being renamed Perkeep

c 5, 2017 at 5:24 PM, Brad Fitzpatrick wrote: > Keepers, > > We're renaming Camlistore to Perkeep. > > See https://github.com/camlistore/camlistore/issues/981 for details and > background. > > Happy keeping! > > > -- You received this message because you a

Re: Reducing blobserver fsync() calls?

Following up a year later, ... I filed https://github.com/camlistore/camlistore/issues/999 I'd like to work on this soon-ish. On Sun, Oct 2, 2016 at 11:48 AM, Theodore Ts'o wrote: > Hi, > > I was noticing that it was taking "a while" to do a test backup of 76 GB, > so I started digging into w

Re: Proposal: Use grpc-web for GopherJS UI <-> backend interactions

On Fri, Dec 22, 2017 at 8:15 AM, Johan Brandhorst < johan.brandho...@gmail.com> wrote: > Hi! > > I'm the author of the GopherJS gRPC-Web bindings > . I was introduced to > Perkeep by Paul Jolly, who is the author of the GopherJS React bindings >

Re: Debugging and authentication

On Wed, Dec 27, 2017 at 12:21 AM, Norman Walsh wrote: > Brad Fitzpatrick writes: > > When you said you waited to write a "backend", I assumed you meant a > > blob storage backend, but perhaps you meant something else. Which > > part do you want to write, and for w

Re: [Go src] why forcing nil to err variable and not using

On Tue, Dec 26, 2017 at 9:36 PM, Timothy Quinn wrote: > I am completely a N00b for Go so please pardon my ignorance. I'm reverse > engineering some code > Reverse engineering or reading? :) If you have questions about how something works, please ask. The code is there also, of course, but don't

Re: Debugging and authentication

On Tue, Dec 26, 2017 at 2:45 AM, Norman Walsh wrote: > Brad Fitzpatrick writes: > > Norman, implementing by mimicking can get you pretty far, but be > > sure to actually read the protocol docs as well. > > Absolutely. I am, however, finding the documentation difficult t

Re: Perkeep on Windows - Is LevelDB stable on Windows or do I need to switch to MySQL or MariaDB as suggested?

I use LevelDB myself. I used to use MySQL but I moved my Perkeep server recently and found it too tedious to setup MySQL again so I tried LevelDB. Haven't had any problems. I'm not running on Windows, but I see that there are Windows-specific files like ./vendor/ github.com/syndtr/goleveldb/level

Re: Debugging and authentication

On Sat, Dec 23, 2017 at 10:25 AM, Mathieu Lonjaret < mathieu.lonja...@gmail.com> wrote: > On 23 December 2017 at 11:09, Norman Walsh wrote: > > Hi, > > > > I worked out, by grovelling through the source, that setting CAMLI_DEBUG > to > > true would enable the -proxy_local option by which I could

Re: Backend API?

On Thu, Dec 21, 2017 at 7:57 AM, Norman Walsh wrote: > Mathieu Lonjaret writes: > > The minimum is to implement a blob server: > > https://perkeep.org/doc/terms#blobserver > > but what you probably want is to implement a full storage interface: > > https://perkeep.org/pkg/blobserver#Storage > >

Funding Perkeep via OpenCollective

Keepers, As announced on Twitter, we've started using OpenCollective for funding Perkeep: https://opencollective.com/perkeep Currently my wife & I have been doing the majority of the funding over the past 5 years, because we really

Re: Does Perkeep have built in protection from silent corruption?

In Perkeep, every chunk of data and all metadata (which is just more data blobs) is named by its cryptographic digest, which is way stronger than simple filesystem checksum, or no checksums (like many filesystems). The digests are validated when uploading new data to Perkeep, during replication, a

Re: Location history

There have been discussions in the past, but no agreement on the schema for storing it. Daily dumps is one way, but kinda gross. Also, how do you define a day? UTC is one obvious choice, but would break many people's daily activity up across two days. It'd be nice to make sure the cut point is whe

Camlistore is being renamed Perkeep

Keepers, We're renaming Camlistore to Perkeep. See https://github.com/camlistore/camlistore/issues/981 for details and background. Happy keeping! -- You received this message because you are subscribed to the Google Groups "Camlistore" group. To unsubscribe from this group and stop receiving

Re: Status of appengine?

That felt nice. :) Thanks! On Thu, Nov 30, 2017 at 7:14 PM, Paul Lindner wrote: > Okay, change her > > https://camlistore-review.googlesource.com/c/camlistore/+/11846 > > > On Thu, Nov 30, 2017 at 6:56 PM Brad Fitzpatrick wrote: > >> Let's just delete it f

Re: Status of appengine?

later if needed. >> >> >> On 30 November 2017 at 03:14, Brad Fitzpatrick wrote: >> > We should delete all that code. >> > >> > I no longer have interest in App Engine and that code's rotten too many >> > times to count. It's not worth

Re: Status of appengine?

We should delete all that code. I no longer have interest in App Engine and that code's rotten too many times to count. It's not worth updating all the time. If people want to use App Engine, they can use App Engine Flexible Environment(tm) or whatever the App Engine mode is where you can run arb

Re: Custom Importers

Either way. The advantage of being in the main repo is code review & ongoing maintenance if we change things. On Mon, Oct 30, 2017 at 10:32 AM, Koushik Roy wrote: > If I wanted to write importers for other services (like Slack > conversations), what's the process? Right now it seems that all t

Re: backblaze b2 configuration

Yup. On Thu, Oct 26, 2017 at 1:25 PM Ty Ler wrote: > For creating a low level config do i just `camtool dumpconfig` and then > paste that file in place of current server config and manipulate? > > > On Thursday, October 26, 2017 at 6:44:10 AM UTC-7, mpl wrote: > >> Yes, it looks like the bits n

Re: Moving from diskpacked to blobpacked

I think your email's subject line is the closest thing to a problem statement in your email. Is the problem that you want an efficient way to move from diskpacked to blobpacked? On Sat, May 20, 2017 at 12:41 AM, Tamás Gulácsi wrote: > Hi, > > I'm using diskpacked storage engine, as I don't want

Re: Financial transaction importer

When I go to https://plaid.com/ it's not clear what thing I want to sign up for to get started. Can you add a README of sorts if/when you send a change? On Thu, Jan 5, 2017 at 8:56 AM, Michael Morrissey wrote: > Hi all, > > As an experiment to learn the flow of importers, I put together a quic

Re: Financial transaction importer

Hell yeah! That's awesome. I've been wanting this too. Adding plaid-go to the vendor directory SGTM. On Thu, Jan 5, 2017 at 8:56 AM, Michael Morrissey wrote: > Hi all, > > As an experiment to learn the flow of importers, I put together a quick > PoC of importing my credit card transactions usi

Re: converting from self-signed cert to letsencrypt

A typo isn't an error? On Mon, Dec 12, 2016 at 2:56 PM, Mathieu Lonjaret < mathieu.lonja...@gmail.com> wrote: > you have a typo in the config, s/httpCert/httpsCert/ :-) > > > On 12 December 2016 at 23:54, Adrian Tritschler < > adrian.tritsch...@gmail.com> wrote: > >> Matthieu, >> >> Yes, I seem

Re: converting from self-signed cert to letsencrypt

Does LetsEncrypt support non-standard port numbers? I think it might have to be 443. On Mon, Dec 12, 2016 at 2:02 PM, Adrian Tritschler < adrian.tritsch...@gmail.com> wrote: > > My config file already has (and has had for months): > > "baseURL": "https://millpond.dyndns.org:3179";, > "listen":

Re: MySQL Permissions?

In the default config, it creates databases as needed for various indexes. I guess that's what you're hitting. (yes, ideally it could use one database with different tables. The current arrangement fell out naturally from the abstraction, but isn't great for tidiness reasons.) On Sat, Dec 10, 20

Re: Default sign-in for GCE Camlistore launcher version?

There is no default. I random password is auto generated at creation time. It should be on your instance's GCE metadata, visible in the GCE VM details page. On Dec 9, 2016 11:09 AM, "clive boulton" wrote: > Please can anyone remind me the default user name and password for the > Camlistore lau

Re: Web UI error when showing other than permanodes

Was this ever resolved? (cleaning inbox) Was there a bug filed? On Mon, Oct 10, 2016 at 2:15 AM, Attila Tajti wrote: > The web UI cannot show non-permanodes (eg. image files) anymore. I suspect > commit 95c4c90591b7570c460cf03fbb98c9380fbf8170 is the culprit. > > -- > You received this messag

Re: blob server on AWS Lambda

That's definitely possible, but it's not currently supported. You'd have to do the work to wire it all up. On Wed, Nov 23, 2016 at 6:28 PM, Raffaele Sena wrote: > Well, the idea is that the blob server would run as a "Lambda" function > and would store the blobs in S3 (and possibly use DynamoDB

Re: Integrity Check and Blob location

I thought there was also a way to use camtool sync to a dummy /dev/null-ish check-hash-only target. Or maybe I'm thinking of the web interface to sync, which does have an integrity check for whether a configured sync pair (e.g. my home machine to AWS S3) is properly in sync. In any case, CLI tool

Re: Reducing blobserver fsync() calls?

I've actually been thinking that sync should be an explicit part of the protocol so higher levels can decide the atomicity that they require. Then we make everything async by default, but all blob storage implementations must support a sync (or "Flush"?) operation. And then camput and other tools

Re: schema: Files and their Contents

I don't quite understand. Can you be more concrete? Which fields would be in file vs. "object"? Which fields currently in file do you feel are too specific? Also, it's the permanode that gets tagged, not the file, yet in your second scenario you still have the permanode at the top referencing the

Re: trunk VS latest release for production

I use trunk always for all my data, fwiw. On Jul 25, 2016 3:37 PM, "Markus Peröbner" wrote: > Hi, > > > I just wanted to ask you if you are using a camlistore trunk build for you > production data or if you stick with v0.9 release for your production data? > > Do you consider it safe to use trun

Re: command line tools show http2 issues

On Fri, Jul 15, 2016 at 8:48 PM, Gergely Imreh wrote: > On 16 July 2016 at 01:50, Mathieu Lonjaret > wrote: > > ACK, will do. > > Thanks, I've tried it out, and with the patchset 2 from your link, and > I can use the CLI tools. > > > > > On 15 Ju

Re: command line tools show http2 issues

Instead of auto-detection, let's make this very opt-in and explicit and add a new client config option to "disabledHTTP2": true. On Fri, Jul 15, 2016 at 10:00 AM, Mathieu Lonjaret < mathieu.lonja...@gmail.com> wrote: > To be clear, you only need to rebuild your CLI tools (not the server) > with

Re: Signed the "Individual Contributor License Agreement" for camlistore

Now that our Gerrit (https://camlistore-review.googlesource.com) has CLA-checking enabled, the docs should probably be updated. I no longer need to be involved in this step. Just: 1) sign the CLA 2) upload the real change to Gerrit Once the real change is accepted and merged after its code review

My availability

I've spent the past few weeks with visiting family & getting married. I'm now spending the next few weeks with visiting family & going on a honeymoon. Mathieu is taking care of things while I'm away (as he usually does anyway). I'll be back to hacking on Camlistore around June 23rd. The good news

Re: access logs for running instance on GCE

It runs under systemd, so watch it like any other systemd unit: $ sudo journalctl -f -u camlistored.service (-f is follow, and -u selects which "unit") On Sun, May 22, 2016 at 8:40 AM, jason gessner wrote: > aside from the log links on /status/ how does one tail the logs on an > instance run

Re: letsencrypt cert kinda working?

Did you include the intermediate certs? You shouldn't need to configure the "trustedCerts" at all. On Sun, May 22, 2016 at 7:56 AM, jason gessner wrote: > hello! > > I am playing around with the excellent camlistore.org/launch image. I > have things up and running and managed to get a letsenc

Re: SDHC card as dumb blob storage?

Yeah, it's no different than using any other file system type. I'd use the blobpacked ("packBlobs": true) storage method to be nicest to the SD card. But I'd use blobpacked on any filesystem type, so it's not really different. On Fri, May 13, 2016 at 4:15 AM, Simon B. wrote: > Is it doable / p

Re: Camlistore Enhancement Proposals

Consider that CEP754. :-) On Tue, May 3, 2016 at 5:39 PM, Eric Drechsel wrote: > I'd like to propose introducing a new issue tag for feature > specifications. The idea is to provide a format for developing a feature > proposal, and getting feedback. > > I was thinking to call them "Camlistore

Re: location info in DescribeResponse

Cool. I also want to see all my stuff on a map! On Fri, Apr 29, 2016 at 6:34 AM, Attila Tajti wrote: > I am playing with google maps and trying to show my photos on a map. I can > search for specific locations or list blobs having location data, but see > no way of getting the location data its