Re: [Sks-devel] "SKS is effectively running as end-of-life software at this point"?

[Robert J. Hansen]

> It is no use to wait a great consensus.

Without exception, I have never met someone who was reluctant to
volunteer someone else's time.  I think this is unfortunate.  I'd rather
we were as reluctant to urge other people to commit their time and
effort to a project as we are to commit our own.



signature.asc
Description: OpenPGP digital signature
___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] A brief recap

[Tobias Frei]

Additional note: Even when restricting append-only mode to the email field,
someone could upload keys for krypton...@domain.org to permanently store
the word "kryptonite" in the database. Also, one could use the first
characters of key IDs to store information, linking the keys together as
signatures or by alphabetical sorting.

00D...
01E...
02A...
03D...
04B...
05E...
06E...
07F...

You couldn't even blacklist them without storing the information in your
blacklist.

Best regards
Tobias Frei

On Thu, Feb 7, 2019, 01:58 Robert J. Hansen  wrote:

> > I disagree that we have to do a trade off, mostly for technical
> > reasons.
>
> Let's call forbidden information 'kryptonite'.  Kryptonite is bad stuff.
>  We don't want it on moral grounds or legal grounds.  We would rather
> shut down keyservers than have kryptonite on our systems.  We then have
> three choices:
>
> * Keep it from entering the system (vetted users, approved submitters)
> * Find a way to purge it from the system (ending append-only)
> * Shut down keyservers
>
> Saying "we can use blacklists to avoid serving up data" leaves you still
> in possession of the data.  This has bad consequences for certain kinds
> of kryptonite.  And the moment you say, "well, if you're not going to
> serve it up then you don't need to store it, either" you've just agreed
> to waive the append-only property.
>
> ___
> Sks-devel mailing list
> Sks-devel@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/sks-devel
>
___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] "SKS is effectively running as end-of-life software at this point"?

[Gabor Kiss]

> There are a handful of people with the background and skills to write a
> next-generation keyserver.  I looked into it a dozen years ago and wrote
> up a whitepaper on it.  I know Phil Pennock has put a lot of thought
> into it.  Andrew, likewise.  There are easily five or six people on this
> list who have the time and ability.
> 
> What we don't have is *consensus* -- not only among ourselves, but in
> the larger community.
> 
> Let's say I decide to implement my whitepaper from 2007.  It would take,
> oh, call it 200 hours of work to implement.  So I write it, put it out
> there, and nothing happens because there's no consensus my idea is the
> direction we should go.
> 
> The problem here is not a lack of manpower or skill.
> 
> It's a lack of community consensus on what a redesign should look like.

My 2 cents:
It is no use to wait a great consensus.

The working model of open source development is:

10 publishing a formal protocol description[1]
20 some peoples implement it
30 collecting experiments
40 years later other peoples write a different implemantation that
  is compatible with the original 
50 GOTO 30

[1]: The famous paper describes the _algorithm_. A protocol description
writes bit-by-bit what is transferred on the wire, what to in case
of any errors, what must to do and what is optional etc. See RFCs.

If there would be _competing_ implementations operators could choose
which one they run. This is an evolution.
Remember Sendmail. 30-40 years ago it was virtually the only (non proprietary)
mail transfer agent (MTA). Eric Allman did a great job. But later other guys
had different ideas. Now there is a dozen MTAs on the market and almost
nobody uses Sendmail. But some do.
And all these programs can talk each to other due to RFC 821 (1982).

Gabor

___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] Excessive use of /var/lib/sks/DB/log.*

[Todd Fleisher]

This sounds like you are missing the recommended DB_CONFIG values to prevent 
your server from holding into those log files when an issue is encountered. As 
I recall, the fix is to start over from scratch and rebuild after first putting 
that file in place. It is covered in the list archives and I believe the source 
and packages ship with an example file that had the needed options. 

Sent from the Fleishphone

> On Feb 6, 2019, at 19:13, Gunnar Wolf  wrote:
> 
> Hello,
> 
> Since a couple of days ago, I have noticed an incredible growth in
> space usage (so much it has killed my server twice, hitting partition
> limits). Since February 3rd, I have over 4000 files with binary-only
> contents (could find no meaningful information in them) called
> /var/lib/sks/DB/log. - I have just
> checked, and it *seems* removing them causes no ill effects.
> 
> But... TTBOMK, I have not modified anything in my configuration. I
> removed them knowing I might be doing something stupid, in the know
> that BDB does not like external processes touching its turf, and was
> happy not to lose my keyserver - but I was ready to rebuild it.
> 
> I haven't been monitoring this list as much as I should. Is there
> anything I should be on the look for? I removed files dated ≤ Feb 3,
> so there is still a lot of information if somebody wants to debug the
> issue.
> 
> Should I just point a "logrotate" or such to the directory? What did I
> do before that I'm not correctly doing now?
> 
> Thanks!
> 
> ___
> Sks-devel mailing list
> Sks-devel@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/sks-devel


___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] Quick and dirty test

[Todd Fleisher]

FYI - that site generates an untrusted ssl certificate warning and after 
acknowledging that I get an error that the site couldn't be found on dreamboat. 

Sent from the Fleishphone

> On Feb 6, 2019, at 19:15, Gunnar Wolf  wrote:
> 
> Kiss Gabor (Bitman) dijo [Tue, Jan 29, 2019 at 07:56:32PM +0100]:
>> Hi folks,
>> 
>> It is funny but one of my peer partners did not notice that his server
>> is dead since a few months. :-)
>> 
>> So I just show anyone who is interested in it how a simple but effective
>> cron job warns me if my server is not OK:
>> 
>> 42 5-8,15-20 * * *   test "$(curl -s 
>> https://sks-keyservers.net/status/ks-status-json.php?server=keys.niif.hu | 
>> jq '[.IPv6, .Port80, .Last_status]' | tee /tmp/sksstatus | md5sum)" = 
>> 'f2a95d496447b4bd81314f4a550a247c  -' || ( echo 'Something went 
>> wrong:\\nhttps://sks-keyservers.net/status/' ; cat /tmp/sksstatus)
>> 
>> Of course hostname and actual MD5 sum value must be tailored
>> according to checked fields of JSON output from curl.
> 
> FWIW, I have been playing with some data that might be connectable to
> this. I did not *yet* intend to go public with this information, but
> it might help use cases as yours:
> 
> https://sks-status.gwolf.org/
> 
> Give me a couple of days and I'll share more complete information. In
> the mean time, you can see where your server is as part of the mesh :-]
> 
> ___
> Sks-devel mailing list
> Sks-devel@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/sks-devel
> 


___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] Quick and dirty test

[Gunnar Wolf]

Kiss Gabor (Bitman) dijo [Tue, Jan 29, 2019 at 07:56:32PM +0100]:
> Hi folks,
> 
> It is funny but one of my peer partners did not notice that his server
> is dead since a few months. :-)
> 
> So I just show anyone who is interested in it how a simple but effective
> cron job warns me if my server is not OK:
> 
> 42 5-8,15-20 * * *   test "$(curl -s 
> https://sks-keyservers.net/status/ks-status-json.php?server=keys.niif.hu | jq 
> '[.IPv6, .Port80, .Last_status]' | tee /tmp/sksstatus | md5sum)" = 
> 'f2a95d496447b4bd81314f4a550a247c  -' || ( echo 'Something went 
> wrong:\\nhttps://sks-keyservers.net/status/' ; cat /tmp/sksstatus)
> 
> Of course hostname and actual MD5 sum value must be tailored
> according to checked fields of JSON output from curl.

FWIW, I have been playing with some data that might be connectable to
this. I did not *yet* intend to go public with this information, but
it might help use cases as yours:

https://sks-status.gwolf.org/

Give me a couple of days and I'll share more complete information. In
the mean time, you can see where your server is as part of the mesh :-]

___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

[Sks-devel] Excessive use of /var/lib/sks/DB/log.*

[Gunnar Wolf]

Hello,

Since a couple of days ago, I have noticed an incredible growth in
space usage (so much it has killed my server twice, hitting partition
limits). Since February 3rd, I have over 4000 files with binary-only
contents (could find no meaningful information in them) called
/var/lib/sks/DB/log. - I have just
checked, and it *seems* removing them causes no ill effects.

But... TTBOMK, I have not modified anything in my configuration. I
removed them knowing I might be doing something stupid, in the know
that BDB does not like external processes touching its turf, and was
happy not to lose my keyserver - but I was ready to rebuild it.

I haven't been monitoring this list as much as I should. Is there
anything I should be on the look for? I removed files dated ≤ Feb 3,
so there is still a lot of information if somebody wants to debug the
issue.

Should I just point a "logrotate" or such to the directory? What did I
do before that I'm not correctly doing now?

Thanks!



signature.asc
Description: PGP signature
___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] A brief recap

[Robert J. Hansen]

> I disagree that we have to do a trade off, mostly for technical
> reasons.

Let's call forbidden information 'kryptonite'.  Kryptonite is bad stuff.
 We don't want it on moral grounds or legal grounds.  We would rather
shut down keyservers than have kryptonite on our systems.  We then have
three choices:

* Keep it from entering the system (vetted users, approved submitters)
* Find a way to purge it from the system (ending append-only)
* Shut down keyservers

Saying "we can use blacklists to avoid serving up data" leaves you still
in possession of the data.  This has bad consequences for certain kinds
of kryptonite.  And the moment you say, "well, if you're not going to
serve it up then you don't need to store it, either" you've just agreed
to waive the append-only property.



signature.asc
Description: OpenPGP digital signature
___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] A brief recap

[Daniel Roesler]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

I disagree that we have to do a trade off, mostly for technical
reasons.

The append-only database and gossip protocol only cares about
public key data, not additional key packet data (e.g. emails,
signatures, photos, etc.). So, it's entirely possible to
participate in the pool and keep in sync with your public key
database, but also not serve up key packet material when
requested.

For example, I'm envisioning a keyserver that has a local
blacklist of data packets hashes (e.g. the spam/troll data),
that is just silently dropped when gossiping/syncing with other
keyservers. That way, the public keys keep in sync (so they
won't be dropped out of the pool), but when a user requests that
key from their specific server (either through the DNS round
robin, or directly through their web interface), the server only
sends the non-blacklisted packets.

This of course raises the risk of censorship for key packets,
but again these blacklists are server-specific, so it's entirely
possible to create multiple pools of troll-resistent keyservers
and free-as-in-freedom keyservers that still stay in sync with
each other because they PTree database and gossip only cares
about the public key content.

So, I think it's totally possible to keep the append-only,
global-write, distributed syncing of public keys. The only thing
that's needed is software features to be able to locally
blacklist key packets.

Daniel

-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEcBAEBCgAGBQJcW385AAoJEMtwcDcM6wt6wlcH/jozUV/bzu1Q74X8hhq5OzYK
m2XvFNQRQdsUc5MRqwqs0zacJIqnxEDTUJsk976mivAJQMiTlB4m75CRTPCAul5H
MaBMm2G7Cv1kiARRFhG17V57Re3wBxDGALqNrJZBsLlVXt1dHa8+OVeAb93gGe31
2eJiMdUD8MLt8Ed6BbZ+4CAV47nXE2Tyy5mMH6yshp39MnGtzBZ7aMLk165Iz8Rc
TK1Rjl7oCl2qu04fabG+Q2ul81h9uYd/4ceCAXggRYp80JBAe4qzogwUTXYrir6d
Mkx2pILof5Ua+2dws3Tun9VjHvzMCRL2CI6S7S/TY+HTchdlgc/DDAGWn6BPngY=
=YYjt
-END PGP SIGNATURE-


On Wed, Feb 6, 2019 at 6:19 PM Robert J. Hansen  wrote:
>
> To spare us all diving through list archives:
>
> The keyserver network is in a lot of ways like a blockchain.  In both
> cases they are distributed ledgers where any change to the ledger is
> propagated through to everyone with a copy of the ledger.  (Blockchain
> differs by adding more cryptographic verification, but in the broad
> strokes they're very similar.)
>
> Why did keyservers evolve in such a way?  Because in the early 1990s it
> seemed like a good idea.  The idea was the distributed redundant ledger
> of cryptographic certificates would make it impossible for a corrupt
> government to force the removal of a dissident's certificates.  During
> the Clinton-era crypto wars this was a very real concern.
>
> It has also never happened in practice.  To the best of my knowledge --
> and I've been watching keyserver operations for literally more than a
> quarter-century -- no keyserver operator anywhere has ever been coerced
> by a government to even try to remove a certificate.  The attack we were
> concerned about never materialized.  It's reasonable to ask if, a
> quarter-century later, it's time to stop defending against it.
>
> Further: in the intervening time we've learned that append-only
> world-writable distributed databases are inherently unstable.  Trolls,
> hooligans, and criminals will poison it with information which is
> irrelevant to the database's purpose (spam), offensive to many of the
> maintainers (hardcore pornography), or flat out criminal (child
> pornography).
>
> So we have a few basic choices: *which condition do we waive?* being
> foremost.
>
> * Append-only?  Reconciliation just got unspeakably harder.
> * World-writable?  This means restricting keyservers to vetted users.
> * Distributed?  Then there is no more keyserver network.
>
> Waiving the "distributed" is technically easiest but it ends the era of
> keyserver networks.  Keyservers become completely balkanized.  Waiving
> the "append-only" criterion sounds nice, because if we can figure out
> how to do that then we get to keep the keyserver network while gaining
> GDPR compliance and ending spam and porn in the network.  Unfortunately,
> we have basically fuck-all zero idea of how to actually do it: the
> engineering challenges are significant.
>
> ___
> Sks-devel mailing list
> Sks-devel@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/sks-devel

___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] Unusual traffic for key 0x69D2EAD9 and 0xB33B4659

[Todd Fleisher]

I also applied these configuration options earlier today to all the servers in 
1 of my pools that was experiencing high IO load and repeated SigAlarms:
command_timeout: 600
wserver_timeout: 30
max_recover: 150

And since then, everything has been quiet:

IO on the main node that gossips externally: https://i.imgur.com/ERgz0Xo.jpg 


IO from another node in the same pool that gossips internally with the above 
node: https://i.imgur.com/wsaxrJ5.jpg 

Hopefully this can help other operators keep things in better shape for the 
time being.

-T


> On Feb 6, 2019, at 3:22 AM, Rolf Wuerdemann  wrote:
> 
> With your suggestions:
> 
> load average below 1
> Traffic: ~150G/day
> 
> Best,
> 
>   Rolf
> 
> Am 2019-02-04 12:52, schrieb Martin Dobrev:
>> Hi,
>> I've spent last week trying to optimize configuration as much as
>> possible. Following advise from a previous mail I've added:
>>> command_timeout: 600
>>> wserver_timeout: 30
>>> max_recover: 150
>> to my sksconf and it seems this fixed majority of the EventLoop
>> failures. I've added DB_CONFIG in KDB/PTree folders to get rid of DB
>> archive logs that were causing plenty of IO load too.
>> My clusters are now happily responding to queries and load-average is
>> bellow one. Traffic wise things look better too, ~20GB/day.
>> Kind regards,
>> Martin Dobrev
>> P.S. Adding/changing DB_CONFIG might cause an error in the databases
>> that you can easily fix by running
>> db_recover -e -v -h /{KDB,PTree}
>> On 04/02/2019 09:49, Rolf Wuerdemann wrote:
>>> Hi,
>>> Don't get me wrong, but within three days I've got 450G traffic
>>> which can be assigned to sks by 99.9%. Estimated to 30 days this
>>> means 4.5T (which is in good agreement of your 2+T/Key for these
>>> two poison keys).
>>> With this amount of traffic and the possibility to get
>>> more of this keys (thus more traffic) every moment, I think it's
>>> only a question of time until the network with the current
>>> implementation will vanish. Traffic increased roughly a factor of
>>> 300 (15G->4.5T) within twelve months, nodes within the network
>>> decreased by a factor of two at least for the same time.
>>> So: where to go and how?
>>> Just my 2ct,
>>> rowue
>>> Am 2019-01-30 22:09, schrieb Martin Dobrev:
>>> Hi,
>>> My observations so far show that both keys generate  2+ TB/month
>>> traffic on average for all my clustered nodes. I'm running nginx +
>>> Varnish in-memory cache tuned at 5 minutes TTL which gives plenty of
>>> CPU cycles for the never-ending EventLoop alarm loops. The latter
>>> cause load-average spikes of up to 10 with just 4 Docker containers
>>> running on a 12 core system.
>>> Don't get me wrong. The throttling penalty is something I'd
>>> swallow-up
>>> as long as we keep the network running.
>>> Regards,
>>> Martin
>>> keyserver.dobrev.eu | pgp.dobrev.it
>>>  Original message 
>>> From: Kristian Fiskerstrand
>>> 
>>> Date: 30/01/2019 20:18 (GMT+00:00)
>>> To: Shengjing Zhu , sks-devel@nongnu.org
>>> Subject: Re: [Sks-devel] Unusual traffic for key 0x69D2EAD9 and
>>> 0xB33B4659
>>> On 1/12/19 8:15 PM, Shengjing Zhu wrote:
>>> I think these requests are quite unusual.
>>> Does anyone know what happens to these two keys?
>>> Just to add a comment on this, adding a cache on the load-balancer
>>> is
>>> really a nice way to slow down hits on the underlying SKS nodes, I
>>> keep
>>> cache for 10 minutes in nginx, which really makes life more
>>> pleasant.
>>> --
>>> 
>>> Kristian Fiskerstrand
>>> Blog: https://blog.sumptuouscapital.com
>>> Twitter: @krifisk
>>> 
>>> Public OpenPGP keyblock at hkp://pool.sks-keyservers.net
>>> fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3
>>> 
>>> "Action is the foundational key to all success"
>>> (Pablo Picasso)
>>> ___
>>> Sks-devel mailing list
>>> Sks-devel@nongnu.org
>>> https://lists.nongnu.org/mailman/listinfo/sks-devel
>> ___
>> Sks-devel mailing list
>> Sks-devel@nongnu.org
>> https://lists.nongnu.org/mailman/listinfo/sks-devel
> 
> --
> Security is an illusion - Datasecurity twice
> Rolf Würdemann  -  ro...@digitalis.org  -  DL9ROW
> GnuPG fingerprint:EEDC BEA9 EFEA 54A9 E1A9  2D54 69CC 9F31 6C64 206A
> xmpp: ro...@digitalis.org E1189573 6B4A150C A0C2BF5A 5553F865 0B9CBF7A
>  ro...@jabber.ccc.de 64CBBB68 0A3514A4 026FC1E7 5328CE87 AEE2185F
> 
> ___
> Sks-devel mailing list
> Sks-devel@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/sks-devel



signature.asc
Description: Message signed with OpenPGP
___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

[Sks-devel] A brief recap

[Robert J. Hansen]

To spare us all diving through list archives:

The keyserver network is in a lot of ways like a blockchain.  In both
cases they are distributed ledgers where any change to the ledger is
propagated through to everyone with a copy of the ledger.  (Blockchain
differs by adding more cryptographic verification, but in the broad
strokes they're very similar.)

Why did keyservers evolve in such a way?  Because in the early 1990s it
seemed like a good idea.  The idea was the distributed redundant ledger
of cryptographic certificates would make it impossible for a corrupt
government to force the removal of a dissident's certificates.  During
the Clinton-era crypto wars this was a very real concern.

It has also never happened in practice.  To the best of my knowledge --
and I've been watching keyserver operations for literally more than a
quarter-century -- no keyserver operator anywhere has ever been coerced
by a government to even try to remove a certificate.  The attack we were
concerned about never materialized.  It's reasonable to ask if, a
quarter-century later, it's time to stop defending against it.

Further: in the intervening time we've learned that append-only
world-writable distributed databases are inherently unstable.  Trolls,
hooligans, and criminals will poison it with information which is
irrelevant to the database's purpose (spam), offensive to many of the
maintainers (hardcore pornography), or flat out criminal (child
pornography).

So we have a few basic choices: *which condition do we waive?* being
foremost.

* Append-only?  Reconciliation just got unspeakably harder.
* World-writable?  This means restricting keyservers to vetted users.
* Distributed?  Then there is no more keyserver network.

Waiving the "distributed" is technically easiest but it ends the era of
keyserver networks.  Keyservers become completely balkanized.  Waiving
the "append-only" criterion sounds nice, because if we can figure out
how to do that then we get to keep the keyserver network while gaining
GDPR compliance and ending spam and porn in the network.  Unfortunately,
we have basically fuck-all zero idea of how to actually do it: the
engineering challenges are significant.



signature.asc
Description: OpenPGP digital signature
___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] "SKS is effectively running as end-of-life software at this point"?

[Robert J. Hansen]

> Is it possible to develop a keyserver thar uses the same interface as
> the current one?

Sort of.

> Meaning that GnuPG-Clients don't need to change

Yes.

> and current keyservers can recon with the new keyservers (since they
> are not all upgraded simultaniously)?

No.  Keyserver reconciliation is 90% of the problem.  Fixing this would
make it impossible for older keyservers to reconcile with a next-gen design.

> Are there any more problems that need to be fixed? Like seriously,
> everyone please write the problems they have with SKS.

Please, *don't*.  We have had this discussion so many times over the
past ~15 years that I can't stand to go through it again.  Go through
the list archives, read those past discussions, and then if you come up
with anything new post it to the list.

> To answer my first question, I guess that it is possible to implement
> a keyserver with the same interface for GPG users that can still
> recon with older servers.

Let's not make this situation worse by guessing.



signature.asc
Description: OpenPGP digital signature
___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] "SKS is effectively running as end-of-life software at this point"?

[Martin Dobrev]

Hi

On 06/02/2019 19:28, Robert J. Hansen wrote:
>> If only it were open-source software and individuals with the extra time
>> and talent to work on those design flaws were able to do so. Wouldn't
>> that be a great world to live in?
> Wouldn't it be great if people were to think before snarking?
>
> There are a handful of people with the background and skills to write a
> next-generation keyserver.  I looked into it a dozen years ago and wrote
> up a whitepaper on it.  I know Phil Pennock has put a lot of thought
> into it.  Andrew, likewise.  There are easily five or six people on this
> list who have the time and ability.
I'm glad to see from server operator perspective that there is a will to
invest time in developing the server.
>
> What we don't have is *consensus* -- not only among ourselves, but in
> the larger community.
>
> Let's say I decide to implement my whitepaper from 2007.  It would take,
> oh, call it 200 hours of work to implement.  So I write it, put it out
> there, and nothing happens because there's no consensus my idea is the
> direction we should go.
>
> The problem here is not a lack of manpower or skill.
>
> It's a lack of community consensus on what a redesign should look like.
>
Is there a place where white-papers and draft proposals can be found? I
remember seeing proposals to change the backend database with something
that supports transactions, the likes of postgresql or mysql. Without it
multi-threaded server is not possible. If I had the knowledge of OCAML
I'd start like immediately a fork and implement it in order to save disk
space and more important cut on redundant Iops. Nearly 21K keys have
been added for the past month. Looking at the graph again I don't see a
single plunge just steady growth. As it stands I'm running two physical
servers with four Docker containers each that take approximately 95GB.
Even with a single-thread server implementation I can still run multiple
containers with a shared database, isn't it?

I agree redesign is inevitable if we want to address legislation changes
like GDPR for example. A lot of servers in Europe ceased operation
because of it. Today I read the news that criminals are using
block-chain network to upload child abuse photos. According to UK law
hosting such content can lead to large convictions. I don't want to be
forced to stop my clusters if criminals exploit the photo field in the
protocol. In my opinion there must be a way to remove content that's not
legal from the network.

The way I see this happening is by introducing blacklists that a)
prevent a key from being fetched during recon and b) delete the local
copy of it. Is this a censorship?! For what I know some of us are
already using blacklists to mitigate the poison key issues, the very
least I am.

Building upon the previous suggestion I'm actually envisioning different
types of blacklists in terms of legislation framework they comply with.
Subdomains like .pool.sks-keyservers.net can be introduced as
well. Then individual keyserver operators can configure what blacklists
to use. A public register will keep the audit trail justifying every
blacklist entry and used as a seed for the servers in the network. And
before you ask who's going to be responsible for keeping that register
up-to-date and accurate I have an answer for you - the very same law
enforcement agencies that in the very first place demanded that we
remove content from the network.

That being said I believe these changes will make the network a safer
place not just for the operators but the users as well. That's
something, I think, we as community can easily achieve consensus upon.

Sadly I don't have a  solution for the recent poison key issues that
opened this thread in first place. We eventually need a brand new RFC
that will define the next-gen server architecture.


>
> ___
> Sks-devel mailing list
> Sks-devel@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/sks-devel


signature.asc
Description: OpenPGP digital signature
___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] "SKS is effectively running as end-of-life software at this point"?

[Andrew Gallagher]

> On 6 Feb 2019, at 23:15, robots.txt fan  wrote:
> 
> To answer my first question, I guess that it is possible to implement a 
> keyserver with the same interface for GPG users that can still recon with 
> older servers. The older servers might try to send them keys that are on the 
> blacklist or are large, but the new server can reject those keys of course.

Easier said than done. There is plenty of discussion in the archives about how 
difficult this would be technically. Because you can reject a key, but then 
what happens is it just keeps trying to come back. Pretty soon there are so 
many rejected keys floating around that the network stops reconciling. Also, 
what happens if I reject certain keys and you don’t, but your only connection 
to the rest of the network is through me? Once nodes start implementing 
different policies you can go split-brain surprisingly easily. 

It’s not a simple matter of just coding it up.

A

___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

[Sks-devel] sks.powdarrmonkey.net is not returning

[Jonathan Wiltshire]

Hi,

sks.powdarrmonkey.net has not been gossipping properly for a while because
of resource constraints. I have decided that it's not worth my while
reviving it every few weeks while the network find itself in such turmoil.

I only regret that I'm not in a position to offer a solution to some of the
problems at the moment.


-- 
Jonathan Wiltshire  j...@debian.org
Debian Developer http://people.debian.org/~jmw

4096R: 0xD3524C51 / 0A55 B7C5 1223 3942 86EC  74C3 5394 479D D352 4C51


___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] "SKS is effectively running as end-of-life software at this point"?

[Matthew Walster]

On Wed, 6 Feb 2019, 20:28 Robert J. Hansen 
> It's a lack of community consensus on what a redesign should look like.
>

And, might I add, the desire for our efforts not to be ruined by others
"proving a point" rather than actors actually seeking malicious intent.

I've abandoned my keyserver, and only advertise my keys as a pointer in DNS
to download it via http.

M

>
___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] "SKS is effectively running as end-of-life software at this point"?

[Jeremy T. Bouse]

On 2/6/2019 11:26 AM, Andrew Gallagher wrote:

> On 06/02/2019 13:11, Steffen Kaiser wrote:
>> Is it meant litterally? The current SKS project is end of life and,
>> effectively, we have to look into another direction? Other software, new
>> fork with rewrite?
> I said "effectively", not literally. And I was in a grumpy mood that
> day. But I stand by my point about design flaws not getting the
> attention they deserve. You don't need to fork the codebase to make
> improvements, but you do need to redesign the protocol to be
> abuse-resistant, which is a Very Hard Problem. If we had some consensus
> on the way forward it would be a good start.

If only it were open-source software and individuals with the extra time
and talent to work on those design flaws were able to do so. Wouldn't
that be a great world to live in?


___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] "SKS is effectively running as end-of-life software at this point"?

[Andrew Gallagher]

On 06/02/2019 13:11, Steffen Kaiser wrote:
> Is it meant litterally? The current SKS project is end of life and,
> effectively, we have to look into another direction? Other software, new
> fork with rewrite?

I said "effectively", not literally. And I was in a grumpy mood that
day. But I stand by my point about design flaws not getting the
attention they deserve. You don't need to fork the codebase to make
improvements, but you do need to redesign the protocol to be
abuse-resistant, which is a Very Hard Problem. If we had some consensus
on the way forward it would be a good start.

-- 
Andrew Gallagher



signature.asc
Description: OpenPGP digital signature
___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] "SKS is effectively running as end-of-life software at this point"?

[Tobias Frei]

Hi,

Isn't Mozilla Thunderbird in a similar state? I'm still happily using it
*because* it has reached a point where further updates are rarely needed.

Best regards
Tobias Frei

On Wed, Feb 6, 2019, 14:22 Steffen Kaiser  wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> I picked up this sentence here on the list in a thread about poison
> key(s):
>
> "> Any chance that sks will be fixed some day?
>
> "Short answer, no. SKS is effectively running as end-of-life software at
> this point. It gets emergency bugfixes but little else. The improvements
> you are talking about would require an enormous refactoring of the
> codebase, likely requiring migration to a different database engine.
> Given that there are fundamental design flaws (poison keys) that aren't
> getting fixed, performance issues just aren't on the radar. Sorry."
>
> Is it meant litterally? The current SKS project is end of life and,
> effectively, we have to look into another direction? Other software, new
> fork with rewrite?
>
> Kind regards,
>
> - --
> Steffen Kaiser
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1
>
> iQEVAwUBXFrc6iOjcd6avHZPAQI8iAgAhMVSiTvPdMARX20Kqkt7BQAIFwcnqi/K
> GncegVdHZqf6PkyogVwXxMDxExOm+dJmkyJDD8IOsc6SQW+Tx7YDOcTynLsvFPQ9
> NjehfxC6+O0Himxr1AAm75HZ3oU82e9GMteebbbyLc2pr0ggx7BNtIf81YjCwzxy
> clLLCLZOCb8HAuXBYGi0X7z0iD9511nWQ6vbbpNEuTdz4rv/Fsvgre6SMQ8blYS6
> mYlCsSv8AoL3vFux3E+8gsDUn/uOFzdvL+W6Ei5ETMd4zt4OVfyuGX/P1SBEGl+5
> 7Wfo3ev7F7ZaY7cJy1aOfF9lTKNF8Fuweq+wvBCnhL1B3dLuZ8Gqjw==
> =Id3h
> -END PGP SIGNATURE-
>
> ___
> Sks-devel mailing list
> Sks-devel@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/sks-devel
>
___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

[Sks-devel] "SKS is effectively running as end-of-life software at this point"?

[Steffen Kaiser]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

I picked up this sentence here on the list in a thread about poison 
key(s):


"> Any chance that sks will be fixed some day?

"Short answer, no. SKS is effectively running as end-of-life software at
this point. It gets emergency bugfixes but little else. The improvements
you are talking about would require an enormous refactoring of the
codebase, likely requiring migration to a different database engine.
Given that there are fundamental design flaws (poison keys) that aren't
getting fixed, performance issues just aren't on the radar. Sorry."

Is it meant litterally? The current SKS project is end of life and, 
effectively, we have to look into another direction? Other software, new 
fork with rewrite?


Kind regards,

- -- 
Steffen Kaiser

-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEVAwUBXFrc6iOjcd6avHZPAQI8iAgAhMVSiTvPdMARX20Kqkt7BQAIFwcnqi/K
GncegVdHZqf6PkyogVwXxMDxExOm+dJmkyJDD8IOsc6SQW+Tx7YDOcTynLsvFPQ9
NjehfxC6+O0Himxr1AAm75HZ3oU82e9GMteebbbyLc2pr0ggx7BNtIf81YjCwzxy
clLLCLZOCb8HAuXBYGi0X7z0iD9511nWQ6vbbpNEuTdz4rv/Fsvgre6SMQ8blYS6
mYlCsSv8AoL3vFux3E+8gsDUn/uOFzdvL+W6Ei5ETMd4zt4OVfyuGX/P1SBEGl+5
7Wfo3ev7F7ZaY7cJy1aOfF9lTKNF8Fuweq+wvBCnhL1B3dLuZ8Gqjw==
=Id3h
-END PGP SIGNATURE-

___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

Re: [Sks-devel] Unusual traffic for key 0x69D2EAD9 and 0xB33B4659

[Rolf Wuerdemann]

With your suggestions:

load average below 1
Traffic: ~150G/day

Best,

   Rolf

Am 2019-02-04 12:52, schrieb Martin Dobrev:

Hi,

I've spent last week trying to optimize configuration as much as
possible. Following advise from a previous mail I've added:


command_timeout: 600
wserver_timeout: 30
max_recover: 150


to my sksconf and it seems this fixed majority of the EventLoop
failures. I've added DB_CONFIG in KDB/PTree folders to get rid of DB
archive logs that were causing plenty of IO load too.

My clusters are now happily responding to queries and load-average is
bellow one. Traffic wise things look better too, ~20GB/day.

Kind regards,
Martin Dobrev

P.S. Adding/changing DB_CONFIG might cause an error in the databases
that you can easily fix by running

db_recover -e -v -h /{KDB,PTree}

On 04/02/2019 09:49, Rolf Wuerdemann wrote:


Hi,

Don't get me wrong, but within three days I've got 450G traffic
which can be assigned to sks by 99.9%. Estimated to 30 days this
means 4.5T (which is in good agreement of your 2+T/Key for these
two poison keys).

With this amount of traffic and the possibility to get
more of this keys (thus more traffic) every moment, I think it's
only a question of time until the network with the current
implementation will vanish. Traffic increased roughly a factor of
300 (15G->4.5T) within twelve months, nodes within the network
decreased by a factor of two at least for the same time.

So: where to go and how?

Just my 2ct,

rowue

Am 2019-01-30 22:09, schrieb Martin Dobrev:
Hi,

My observations so far show that both keys generate  2+ TB/month
traffic on average for all my clustered nodes. I'm running nginx +
Varnish in-memory cache tuned at 5 minutes TTL which gives plenty of

CPU cycles for the never-ending EventLoop alarm loops. The latter
cause load-average spikes of up to 10 with just 4 Docker containers
running on a 12 core system.
Don't get me wrong. The throttling penalty is something I'd
swallow-up
as long as we keep the network running.

Regards,
Martin

keyserver.dobrev.eu | pgp.dobrev.it

 Original message 
From: Kristian Fiskerstrand

Date: 30/01/2019 20:18 (GMT+00:00)
To: Shengjing Zhu , sks-devel@nongnu.org
Subject: Re: [Sks-devel] Unusual traffic for key 0x69D2EAD9 and
0xB33B4659

On 1/12/19 8:15 PM, Shengjing Zhu wrote:
I think these requests are quite unusual.
Does anyone know what happens to these two keys?

Just to add a comment on this, adding a cache on the load-balancer
is
really a nice way to slow down hits on the underlying SKS nodes, I
keep
cache for 10 minutes in nginx, which really makes life more
pleasant.

--

Kristian Fiskerstrand
Blog: https://blog.sumptuouscapital.com
Twitter: @krifisk

Public OpenPGP keyblock at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3

"Action is the foundational key to all success"
(Pablo Picasso)
___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel

___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel


--
Security is an illusion - Datasecurity twice
Rolf Würdemann  -  ro...@digitalis.org  -  DL9ROW
GnuPG fingerprint:EEDC BEA9 EFEA 54A9 E1A9  2D54 69CC 9F31 6C64 206A
xmpp: ro...@digitalis.org E1189573 6B4A150C A0C2BF5A 5553F865 0B9CBF7A
  ro...@jabber.ccc.de 64CBBB68 0A3514A4 026FC1E7 5328CE87 AEE2185F

___
Sks-devel mailing list
Sks-devel@nongnu.org
https://lists.nongnu.org/mailman/listinfo/sks-devel