Re: Gradle et al

[Matthew Toseland]

On 10/01/18 21:51, Florent Daigniere wrote:
> On Wed, 2018-01-10 at 21:36 +0000, Matthew Toseland wrote:
>> On 10/01/18 21:15, Florent Daigniere wrote:
>>> On Wed, 2018-01-10 at 21:10 +, Matthew Toseland wrote:
>>>> So what is going on, and why?
>>>>
>>>>
>>> What's happening is that Arne is refusing to move forward... and
>>> keeps
>>> releasing off the old release tools and Ant.
>>>
>>> The rest of the team has been working on next (I've done most of the
>>> current gradle support, including deterministic builds, ... steve
>>> has
>>> been working on the release tools, ...) 
>> So you are checking the hashes of the downloaded components?
>>
>> I thought Gradle was just an evolution of Maven, with all the problems
>> that implies: Recursively pulling random JAR files, with very little
>> authentication, pay-for-only signature checking, and a guarantee that
>> everyone who uploaded those JARs hasn't paid for that feature. In
>> other
>> words, malware galore.
>>
>> If that's the world that Gradle takes Freenet into, then I can
>> entirely
>> understand why Arne would have a problem with it.
> We do check the hashes of downloaded components... and produce
> reproducible jars by default.
> https://github.com/freenet/fred/blob/next/build.gradle#L227
>
> Security is clearly not the concern here.

Annoying that it can't easily work over Freenet... but not a serious
concern, any HTTP(S) fetches can easily go via Tor etc.

What about the deployment side of the question?

I recall somebody arguing for getting rid of the installer and using
some third party packaging system instead? What is the status of that?



signature.asc
Description: OpenPGP digital signature

Re: a test release for 1480

[Matthew Toseland]

On 08/01/18 19:14, Arne Babenhauserheide wrote:
> Hi,
>
> I uploaded an archive with the files for 1480: jar, installer, ….
>
> CHK@MvoTOkbcT--mONuKwklk7CtunQcta1vIxLd9Abr~6Ck,rXO7Nj2L0-Bytn8WorayXo9EvERPz0SS169YdIolndA,AAMC--8/freenet-1480.tbz
>
> mkdir freenet-1480
> mv freenet-1480.tbz freenet-1480
> cd freenet-1480
> tar xf freenet-1480.tbz
>
> The main change is shipping the new windows installer. Please give it
> some testing.
>
> To check:
>
> - does it install cleanly?
> - does it offer the new installer when someone generates an installation
>   bundle?
> - can you stop freenet, drop in the jar and start to update?
>
> Is there anything I missed?
>
> This does *not* fix the last resort updater over the clearnet because I
> cannot test whether what I did would work (and am not that fond of
> coding blind folded, especially not for batch where I don't even know
> the intricacies of the language), and because no one stepped up who
> can. I see this as suboptimal, but compared to not releasing at all it’s
> the lesser of two evils. Sorry for that :/

Use some of the vast unspent funds to pay for a Windows 10 license, and
run it in a VM.
>
> Best wishes,
> Arne



signature.asc
Description: OpenPGP digital signature

Re: Gradle et al

[Matthew Toseland]

On 10/01/18 21:15, Florent Daigniere wrote:
> On Wed, 2018-01-10 at 21:10 +0000, Matthew Toseland wrote:
>> Could somebody summarise what the plans are, and what the reasons
>> behind
>> them are for:
>>
>> 1. Gradle,
>>
>> 2. Deployment/updating of JARs etc.
>>
>> I get the impression that this has been a major factor preventing
>> forward progress for over a year.
>>
>> This is particularly depressing given that more than five years ago I
>> implemented good auto-update support in fred, including the ability to
>> update different files separately, add new jars and so on, all from
>> the
>> simple file dependencies.properties. It would be very easy to generate
>> the last-resort update scripts from that, provided we can provide a
>> URL
>> to get each component from. Any third-party deployment system will
>> likely not work with Freenet, and even if it does, it won't work
>> reliably with Freenet when nearly anything except the transport layer
>> breaks (unlike the current system).
>>
>> If the goal is to split up freenet-ext.jar that's great, but again
>> that
>> is fundamentally very straightforward and does not require rewriting
>> all
>> the installers and depending on polling a third party web server.
>>
>> So what is going on, and why?
>>
>>
> What's happening is that Arne is refusing to move forward... and keeps
> releasing off the old release tools and Ant.
>
> The rest of the team has been working on next (I've done most of the
> current gradle support, including deterministic builds, ... steve has
> been working on the release tools, ...) 

So you are checking the hashes of the downloaded components?

I thought Gradle was just an evolution of Maven, with all the problems
that implies: Recursively pulling random JAR files, with very little
authentication, pay-for-only signature checking, and a guarantee that
everyone who uploaded those JARs hasn't paid for that feature. In other
words, malware galore.

If that's the world that Gradle takes Freenet into, then I can entirely
understand why Arne would have a problem with it.
> On a personal level, I've given up. I'm not enjoying contributing as
> much as I used to (mostly because I waste my time argueing with Xor and
> Arne)... so nothing is moving forward. Have a look to the diff of the
> last few released builds; it's depressing.
>
> Florent



signature.asc
Description: OpenPGP digital signature

Gradle et al

[Matthew Toseland]

Could somebody summarise what the plans are, and what the reasons behind
them are for:

1. Gradle,

2. Deployment/updating of JARs etc.

I get the impression that this has been a major factor preventing
forward progress for over a year.

This is particularly depressing given that more than five years ago I
implemented good auto-update support in fred, including the ability to
update different files separately, add new jars and so on, all from the
simple file dependencies.properties. It would be very easy to generate
the last-resort update scripts from that, provided we can provide a URL
to get each component from. Any third-party deployment system will
likely not work with Freenet, and even if it does, it won't work
reliably with Freenet when nearly anything except the transport layer
breaks (unlike the current system).

If the goal is to split up freenet-ext.jar that's great, but again that
is fundamentally very straightforward and does not require rewriting all
the installers and depending on polling a third party web server.

So what is going on, and why?




signature.asc
Description: OpenPGP digital signature

Re: We need a fix for update.cmd

[Matthew Toseland]

On 08/12/17 21:04, Arne Babenhauserheide wrote:
> Hi,
>
> The new windows installer is prepared, I can release any day, but
> there's still one critical piece missing: we need update.cmd fixed to at
> least download freenet-stable-latest.jar from github instead of trying
> to use the defunct downloads.freenetproject.org.
>
> This is the file:
> https://github.com/freenet/wininstaller-innosetup/blob/master/install_node/updater/update.cmd
>
> We need this updated to download from github releases, similar to what
> update.sh now does:
> https://github.com/freenet/java_installer/blob/next/scripts/update.sh#L271
>
> But if I were to do that myself, it would be idiotic: I have almost no
> Windows experience (I never used a Windows computer as my home system)
> and most certainly do not know about intricacies of Windows systems.
>
> Therefore we need you to step up and go and get update.cmd to work
> again:
>
> - Access https://github.com/freenet/fred/releases/latest
> - Get the forwarding URL to grab the LATEST_TAG parameter.
> - download 
> https://github.com/freenet/fred/releases/download/${LATEST_TAG}/freenet-${LATEST_TAG}.jar;
>
> If we can at least get the newest freenet jar-file, we can go and fix
> other things from there, if the switch to gradle creates unexpected
> problems.
>
> So if you have some experience with Windows command line shell, please
> step up and fix update.cmd. This is the last real blocker before we can
> go forward towards releasing next.
>
> Best wishes,
> Arne

I still think this is going to break badly whenever you have an
incompatible change to the dependencies.

update.cmd should be auto-generated from dependencies.properties.




signature.asc
Description: OpenPGP digital signature

Re: DDG Tasks Bug Bounty Proposal

[Matthew Toseland]

On 08/05/17 18:21, Steve Dougherty wrote:
>  Original Message 
> Subject: Re: DDG Tasks Bug Bounty Proposal
> Local Time: May 8, 2017 1:09 PM
> UTC Time: May 8, 2017 5:09 PM
> From: free...@nullvoid.me
> To: devl@freenetproject.org
>
> Can you provide the minimum identification requirements to be able to
> get a bug bounty from FPI? If you have to report to the IRS does that
> mean only citizens of the United States are eligible to work on Freenet
> for pay?

No, FPI can pay foreign developers, and has done in the past.
> As for access to the source code, is it not open source? If you mean
> push access to the repo, I thought most of the bug bounties are to fix
> bugs and submit code, not review and merge code. There is no security
> concern regarding anonymous vs known developers submitting code. At the
> end of the day the code should be reviewed line for line, whether it's
> by a "trusted" name or not.
>
> Right - I propose paying someone to write code which is then reviewed and 
> merged by existing community members with push access.

This is the correct approach - if somebody goes to the lengths to craft
some subtle vulnerability (Heartbleed!) they are not going to be
deterred by needing a name and address.

Having said that, review capacity has been a problem in the past. My
purge-db4o work was delayed for an entire year, for example. How can we
minimise this?




signature.asc
Description: OpenPGP digital signature

Re: DDG Tasks Bug Bounty Proposal

[Matthew Toseland]

On 06/05/17 14:11, Freenet wrote:
> Could this be solved by paying a known third party? Such as bountysource
> or something?
>
> And from there the developer who creates the patch could still remain
> anonymous and gain the funds?

Bountysource FAQ:

> As part of the cash out process we require a full name, address, and
email address. We may also require that you fill out a W-8/W-9 form for
tax obligations (see "Do I have to pay taxes on bounties I collect?") below.


If you want to avoid this, you need to raise the funds anonymously, and
build an infrastructure for managing them anonymously. This is quite
feasible and people have tried; the catch is people will use it for
other things too. But it doesn't apply to money given publicly to FPI, a
registered non-profit. The basic concept of money laundering is using
"black" money to pay for "white" services. Doing the opposite is still a
bad idea if you're a charity. If you want to spend it anonymously, you
need to raise it anonymously.



signature.asc
Description: OpenPGP digital signature

Re: DDG Tasks Bug Bounty Proposal

[Matthew Toseland]

On 06/05/17 12:36, Matthew Toseland wrote:
> On 06/05/17 10:53, Steve Dougherty wrote:
>> Hi everyone,
>>
>> To my understanding, at least currently xor does not want FPI to pay him for 
>> his work. Some developers on FMS have proposed bug bounties - say, $1000 - 
>> for completing a task like "fix Windows tray / installer to work with 64-bit 
>> Java." This would be in a "first to get reviewed and merged gets paid" 
>> fashion, the idea being we can pay people not yet familiar with the project 
>> to familiarize themselves and not have to commit to paying an unknown 
>> developer hourly. At least one developer has asked that payment be available 
>> in crypto currency; this seems reasonable to me.
> Legally you have to be able to identify them for tax purposes (since
> you're doing this as FPI, not building some anonymous infrastructure for
> it). Or you end up liable for their taxes.
>
> Apart from that, bounties sound like a good idea. But IANAFD.

IANAL too :)




signature.asc
Description: OpenPGP digital signature

Re: DDG Tasks Bug Bounty Proposal

[Matthew Toseland]

On 06/05/17 10:53, Steve Dougherty wrote:
> Hi everyone,
>
> To my understanding, at least currently xor does not want FPI to pay him for 
> his work. Some developers on FMS have proposed bug bounties - say, $1000 - 
> for completing a task like "fix Windows tray / installer to work with 64-bit 
> Java." This would be in a "first to get reviewed and merged gets paid" 
> fashion, the idea being we can pay people not yet familiar with the project 
> to familiarize themselves and not have to commit to paying an unknown 
> developer hourly. At least one developer has asked that payment be available 
> in crypto currency; this seems reasonable to me.

Legally you have to be able to identify them for tax purposes (since
you're doing this as FPI, not building some anonymous infrastructure for
it). Or you end up liable for their taxes.

Apart from that, bounties sound like a good idea. But IANAFD.



signature.asc
Description: OpenPGP digital signature

Re: [freenet-dev] poll: start writing offers

[Matthew Toseland]

On 28/01/17 15:57, Arne Babenhauserheide wrote:
> Hi,
>
>
> The poll concluded a few weeks back and exceeded its hopstolive.¹ We
> should adhere to it and spend the money. I think we should start by
> writing offers in Freenet forums to pay people to do the highest ranking
> tasks and move to later tasks when the initial tasks are done.
>
>
> These are the two top tasks (they came out on top in every evaluation):
>
>
> (1) Title: *Darknet invitation bundles*
> Description: Adding a single use node reference to an installer
>   executable. People could hand out the installer executable to
>   their friends to allow them to connect by Darknet instantly. See  
>   - https://bugs.freenetproject.org/view.php?id=166#c11290
>   (an overview of the requirements and options for invites) and
>   - https://bugs.freenetproject.org/view.php?id=1342
>   ("create a bundle installer")
> Estimated cost: 1 to 1.125 weeks (~ 1000$ to 1250$)

IMHO this is more work because making it work well pulls in a lot of
other stuff, e.g. FOAF connections, using your well-connected peer to
answer your invites, etc. Obviously we should do it but somebody needs
to look carefully at the scope of the problem. We can do it in pieces,
but you've already proposed at least two of the components above.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Alternate evaluations to get a robust top 20; was: Poll results available; Temporary quit as potential employee

[Matthew Toseland]

On 14/12/16 12:28, Arne Babenhauserheide wrote:
> Arne Babenhauserheide writes:
>
>> Arne Babenhauserheide writes:
>>> The most robust result of the poll is: we should definitely do these
>>> five tasks:
>>>
>>> - Darknet invitation bundles (requires single use references)
>>> - Improve FProxy CSS3 support to allow better Freesite UI
>>> - Friend requests, like in Facebook
>>> - Short node references
>>> - Keepalive
>> …
>>> - Finishing the first iteration of Web of Trust speed fixes (1)
>>> - Fixing the installers (2)
>>>
>>> These 7 tasks together are already estimated as 17 person-weeks, which
>>> would leave us 15% buffer for unforseen problems.
> …
>> Also the first two items in xors evaluation are also in this short-list,
>> so we can just start with finding people who can do these two — *Darknet*
>> *invitation bundles* and *fixing the installers* — in parallel to any
>> discussions about which other tasks we want to do.
> (1) Darknet invitation bundles: Feature for adding a single use node
> reference to an installer executable. People could hand out the
> installer executable to their friends to allow them to connect by
> Darknet instantly.
> See https://bugs.freenetproject.org/view.php?id=166#c11290
> (an overview of the requirements and options for invites)
> and https://bugs.freenetproject.org/view.php?id=1342
> ("create a bundle installer")

There's a whole thicket of bugs here. Doing this right is quite a lot of
work. For example, if Alice is not port forwarded, she should still be
able to send an invite to Bob, using her port-forwarded peers. This
interacts with FOAF. If you chase the links between bugs you should find
some of my ideas on this.
>
> (2) Fix the installers (windows first, then packages for linux)
> See https://bugs.freenetproject.org/view.php?id=1276
> and https://bugs.freenetproject.org/view.php?id=1275
> and https://bugs.freenetproject.org/view.php?id=5892
> and linked issues.
> …
>> Ian has been very good at ensuring that coders get their pay, so that
>> should not be a problem (thank you, Ian!), once we have a system
>> in place with which we have some quality control of the results.
>>
>> So, how do we proceed to find people who will accept money for realizing
>> these milestones?
> Ideas? Can we start by putting up an offer in Freenet-internal channels?
>
> How much money should we offer?
>
> Best wishes,
> Arne
>
> PS: hopstolive





signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Poll results available; Temporary quit as potential employee

[Matthew Toseland]

On 09/12/16 07:07, x...@freenetproject.org wrote:
> Ian and Florent have voiced concerns about whether my involvement in the poll 
> could be influenced by the prospect of me benefiting financially by 
> potentially being able to resume my job for Freenet.
>
> I want to provide you with a solid proof that this is not the case, which is 
> why I decided I am *NOT* available for hire for the year of a roadmap which 
> this poll yields.
> I will instead finish the most important remaining WoT/Freetalk fixes for 
> free, as a volunteer.
>
> In return I would be very thankful if our team's atmosphere could become less 
> toxic in the future - not only towards me, but among *everyone* on the 
> project. The bad climate between all of us *was* one of the reasons for me to 
> quit my offer; and I'm not the only one who thinks we have a problem with our 
> work climate, someone else even left as a volunteer because of it.
>
> It made me feel very bad that I was being accused of money being my 
> motivation 
> after I have already volunteered for *more* years than I was paid for, on the 
> very same stuff I was voting for. Doesn't me writing WoT/FT code 6 years 
> before money was involved show that I merely truly have been believing the 
> stuff is what Freenet needs for a long time?
> But I acknowledge you were trying to protect the project and acting with good 
> intent - just please remember that there are real human beings on the other 
> end of the Internet who are just as passionate about the project as you are.
> Constantly putting a gun on their chest is unlikely to make them able to 
> provide precision engineering. The stress is rather likely to make them act 
> emotionally and thus show their worst qualities, not their best ones.
> I am of course aware that I also produced big rants which I should be and am 
> ashamed of, and I will continue to try to not do that anymore!
>
> So anyway, you'll get another large bunch of code for free as compensation, 
> which I hope provides enough proof so we can restart our relationship as a 
> team - which would make me very happy! :)
> The sole reason why I am on this project is because I want to make the world 
> a 
> better place; and I trust you that's your intention as well.
> Thus there's no reason for us to argue, we're on the same side.
>
> Alright, so now that all hypothetical benefits in manipulating it are 
> invalidated hopefully, here's my proposal on how to evaluate the poll results:
> https://git.io/v1ah4
>
> Feedback is welcome!
>
> README:
>
> - The central output is the list of tasks sorted upon average votes divided 
> by 
> average cost estimate. This is in the file Main_Results.ods, on the sheet 
> "RANKING".
> You should open this with LibreOffice - I haven't tried with Microsoft and 
> the 
> sheet is rather complex so there may be incompatibilities.
> The ZIP includes screenshots so you can check whether it renders correctly.
>
> - The foundation for the main output consists of the evaluations of the sub-
> stages: Stage3_Results.ods and Stage4_Results.ods.
> They're also included as subsheets of the Main_Results.ods.
>
> - At the stage 3 evaluation, I've excluded the votes of anonymous voters 
> whose 
> user identities have been created within a month of the poll's announcement. 
> The project has existed for 16 years, so creating an anonymous user account 
> within 1 month of the poll is very suspicious of being a sybil attack. It 
> only 
> takes solving a captcha to create a FMS identity, you don't need an email or 
> anything, and you can create as many identities as you like to.
> Also someone on IRC *had* threatened to vote with sybils!
>
> - Stage 3 says that nextgens and toad_ have voted 1 point more than the 1000 
> available. This is likely a rounding error: The ballot template showed a 
> default vote of 15 points, but that was actually 14.925 rounded to 15 in 
> display (to make the defaults add up to 1000 votes without remainder). The 
> voters having exported their votes to CSV likely replaced the non-rounded 
> value with the rounded one.
> As I haven't exhausted all of my 1000 votes and they are important 
> contributors, they can just have the 2 additional ones from me, no need to 
> correct this :)
>
> - At stage 4, I've excluded estimates which were at least 800% off from the 
> minimum or maximum estimate. I've chosen whether to exclude the min or max 
> individually, e.g. depending on which side the majority of the other votes 
> was 
> on. I've notably also applied this to some of my *own* estimates, and to 
> values whose exclusion *worsens* the average for my subprojects :)
> This stage was not about *choosing* what you like, it was about providing a 
> scientific estimate of how long development will take. So if we exclude some 
> estimates of people, we're not hurting their right to vote because this stage 
> was *not* about voting; stage 3 was for that.
>
> - I've included nextgens' vote for fixing the installers 

[freenet-dev] Fwd: [announce-crypto] BC Security Advisory (was: Strange result with modular math functions)

[Matthew Toseland]

I think this doesn't affect us as we use ephemeral DH and then sign it
with ECDSA? Florent?



 Forwarded Message 
Subject:[announce-crypto] BC Security Advisory (was: Strange result
with modular math functions)
Date:   Tue, 29 Nov 2016 13:55:55 +1100
From:   Peter Dettman 
Reply-To:   peter.dett...@bouncycastle.org
To: BouncyCastle ,
'dev-crypto-csh...@bouncycastle.org'
,
announce-crypto-csh...@bouncycastle.org, announce-cry...@bouncycastle.org
CC: joseph.fr...@intel.com



Hi Joe,
Yes, the BC result is in error. We are aware of the issue, but were
holding off announcing it until fixed in our next release. Since it is
now public anyway, I will give a few details, and we will commit the
fixes shortly.

The problem is actually a carry propagation bug in the Nat192.square
method. The same basic issue also affects Nat128, Nat160, Nat256, and so
other curves are likewise affected, not just secp384r1. It affects both
our Java and C# APIs.

These methods are presently only used (internally) by the custom
elliptic curve implementations. The bug(s) can lead to an erroneous
scalar multiplication (SM), by our estimate the probability is less than
2^-48 per SM for randomly selected input. All scalar multipliers perform
a point validation before returning their result, so if the bug actually
occurs in normal usage, the SM should fail. We consider the probability
of an undetected fault in a single SM to be (very conservatively) less
than 2^-100.

It is however possible to choose an input that will fault or not
depending on some leading bits of the scalar, which can incrementally
reveal a long-term key. ECDH with static keys (i.e. re-using the same
key for many ECDH computations, as e.g. with the non-ephemeral ECDH
ciphersuites in TLS) is therefore expected to be exploitable as
described in https://eprint.iacr.org/2011/633, although with a
substantially higher attack cost.

Note that ECDH ciphersuites are not enabled by default in our TLS/JSSE
implementations, nor recommended in general, but they can be enabled
explicitly by users.

We recommend that users DO NOT enable static ECDH ciphersuites in TLS/JSSE.

New releases with fixes for both Java and C# are expected next week.

Regards,
Pete Dettman

On 29/11/2016 5:17 AM, Friel, Joseph wrote:
> Hi,
> 
> I've been using Bouncy Castle to do some elliptic curve/modular math
> computations, and I've come across some strange results.
> Specifically, the SecP384R1Field.square() function does not appear to
> always return a correct result. Here is a short test case where the
> BouncyCastle modular square differs from Java's BigInteger modular
> square:
> 
> https://gist.github.com/anonymous/e6322f0a7bf5a76a888dd1708f2510ca
> 
> The BigInteger result was cross-checked against an independent
> implementation (Python), so I believe it is correct and the
> BouncyCastle result is in error. Can anyone point out anything I
> might be doing wrong that would explain this discrepancy?
> 
> 
> 
> Thanks,
> 
> Joe Friel
> 
> 
> 
> 




signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Ian et al: Poll conclusions

[Matthew Toseland]

On 27/11/16 21:08, x...@freenetproject.org wrote:
> On Sunday, November 27, 2016 09:29:45 PM Arne Babenhauserheide wrote:
>> x...@freenetproject.org writes:
>>> There are some tough administrative decisions remaining to make about
>>> stage 3, namely which voters to exclude from stage 3 because they look
>>> like a sybil attack.
>> I don’t like this behind-closed-doors-guessing-about-sybil.
> That's OK, here's what happened:
> While most participants of the poll from IRC have been known in the community 
> for years, 70% of the anonymous FMS participants created their identity 
> within 
> a month of the poll's announcement, as measured by my instance of FMS which 
> has been running for years.
>
> Now one might say that this is conclusive enough and I should just publish 
> the 
> results without the sybils.

Yes, you should. Give the process a chance.
> But the initial problem which this thread is about doesn't stop there:
> There will most certainly be some controversy about what to do first because 
> a 
> poll of something like 70 items will certainly not cause any winner with a 
> big 
> majority, it will be a close call.
>
> So someone has to make a decision then. And then we're again left with the 
> initial problem: Ian doesn't seem to have the time to make decisions.
>
> And even if he does:
> Then we decided for *one* thing to do first. So we start it, and it is 
> finished some day.
> And then we *again* need to decide what to do next.
> And then Ian again has no time.
> So we repeat this 70 times now or what?

I'd be very surprised if $25K got that far...



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Internationalization on the new site

[Matthew Toseland]

On 03/11/16 05:52, Dan Roberts wrote:
> On Tue, Nov 1, 2016 at 2:26 AM, Florent Daigniere <
> nextg...@freenetproject.org> wrote:
>
>>
>> Do you have any plan to handle the bank-balance stuff? Or is that gone
>> in the new design?
> I'm doing my best to avoid any "design" decisions, so I had no plan to
> include the bank balance, but it definitely has value.
Currently this is updated hourly, requiring a sed script on the static
site, from a script running on my home server. Any thoughts on this
welcome. I'm guessing we could use either server- or client- side
includes to eliminate the sed script?



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Fake GPG key attack on a Freenet developer

[Matthew Toseland]

On 24/09/16 03:45, x...@freenetproject.org wrote:
> http://www.draketo.de/english/gnupg-attack
>
> This happened at an interesting point in time:
> The financial allocation poll was finished last Sunday and I wanted to 
> publish 
> the results - but the GPG signatures of at least 4 participants were invalid 
> and I luckily was paranoid enough to postpone the publishing because of that.
>
> I had requested the contributors to re-sign the attachments with a detached 
> signature, i.e. not embedded into the mail headers but a plain file 
> attachment 
> instead. I could validate 3 of the original attachments to not be tampered 
> with. So likely the invalid sigs were due to bugs in the mailservers.
>
> Still, I am waiting for one signature of a core developer to be validated and 
> considering this event, I will not publish the results until I have a 
> validation.
> His case is also the most concerning one: The mail with the invalid signature 
> did NOT embed it into the mail headers but shipped it as a file attachment. 
> This should be much less likely to be a mailserver bug, so I'd really rather 
> wait for the participant to find time to give me a new sig. He's aware of it.
>
> As consequences, I would request the following:
>
> - I've seen invalid signatures on devl rather frequently in the past and 
> shrugged it off because the contents were not security-critical discussion 
> and 
> mailservers frequently seem to damage the headers in a way which causes 
> invalid sigs.
> Can any of our server admins reproduce this = is this a bug of our server, 
> not 
> my mail client? I had commented on the mails with invalid sigs at the 
> "Financial allocation poll stage 3" thread.
> If yes, can you please investigate the reason? You could ask the senders of 
> those mails for copies from their "Sent mail" dir and diff against what devl 
> received.
> It would be good to fix this: Invalid signatures happening frequently teaches 
> people to ignore it.
>
> - Anyone who is not signing their mails yet should please start doing so.
> The same applies to Git commits.

I thought these were generally caused by configuration errors, with GPG
getting confused due to the hash algorithm configured in the local
config being different to that in the (older) key?



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Financial allocation poll stage 4

[Matthew Toseland]

On 31/08/16 17:11, x...@freenetproject.org wrote:
> Hereby we begin the 4th stage of the financial allocation poll.

How's this going? Do you have any votes from anyone else yet?



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Financial allocation poll stage 4

[Matthew Toseland]

On 31/08/16 17:11, x...@freenetproject.org wrote:
> Hereby we begin the 4th stage of the financial allocation poll.
>
> You may participate by filling your estimates in to this spreadsheet and
> mailing it back:
> https://github.com/xor-freenet/freenet-money-poll/archive/2016-stage4.zip
>
> It has been created with LibreOffice but may hopefully as well work with
> OpenOffice and any Microsoft Office version greater than or equal to 2007 SP2.
>
> With complex file formats like spreadsheets, it may be possible that private
> data such for example as your username or hostname leaks into the file. If
> you are concerned about that, you can:
> 1. Open the ODS file with a ZIP tool and review the plain text files it
> actually consists of.
> 2. Send back a CSV file instead. CSV is a plain text
> file format so you can review it with any text editor. For LibreOffice,
> instructions on how to export a CSV can be found at:
> https://help.libreoffice.org/Calc/Importing_and_Exporting_CSV_Files
>
> As almost 4 months have passed since the DuckDuckGo donation, we would like
> to continue to the next stage quickly - so please reply within 1 week.
>
> Thank you!

Again, many of these include too much inaccurate detail.

I have made my estimates according to what I would do for each goal, not
what the person proposing it thought.

For example, for tunnels I'd expect to implement something along the
lines of ShadowWalker. Solving the problems associated with this would
be a major undertaking. Similarly with search. So they're down as 16
weeks. Dynamic content is technically possible - there are things we
could do with javascript and tunnels - but really hard, so that's down
at 36 weeks. PSKs are surprisingly difficult not only because they
involve implementing a programmable key type, but because they more or
less require ECC SSKs, and datastore changes too, which need to be done
safely even if we run out of disk space, and have knock-on effects on
the network protocols too. Also, the stuff about FOAF is wrong, I've
estimated based on what we really want from it.

I sent my answers to Xor, but I hope there can be some discussion, so I
attach mine. Feel free to make your own mind up before looking at mine!
They weren't considered in very much detail, just quick guesstimates.
Task category / Task,Estimated time
,in 1-man-weeks
,
Instructions,
"The goal of this stage is to estimate how much time the development of individual tasks would cost us.
We will measure in how many weeks a single developer would have to spend.

The goal is NOT to vote how much time you think we should limit spending on a task to!
This stage is only about total developer time cost estimation as if we had infinite money.

The cells for the week estimate should provide you with a dropdown, please only use the values there.
For tasks which you estimate to be below or above the min/max choices in the dropdown, just choose the min/max ones.
You do not need to provide an estimate for tasks whose codebase you don't feel familiar enough with.

Non-developers were allowed to add tasks to the list so some tasks may be vague, unclear, or under-specified (""I want a pony!""). To be fair we cannot just remove them - but you may consider increasing your cost estimate accordingly to account for this uncertainty.

For tasks which depend on other tasks, do not add the cost of the dependency if the dependency has a separate entry in the spreadsheet as well. Estimate the cost of the dependency separately then.",
,
Speed - Improve performance of Freenet or Freenet applications,
Web of Trust: Finish first iteration of most critical speed fixes (1 bugtracker entry). Was subject of previous 2 years of paid work. Ensures this work is not left unfinished. Needed for Sone / Freetalk / filesharing / … (see all other sections except Technical Debt),2
Freetalk: Make Freetalk usable again by using the new WoT event-notifications API to fix its most severe performance issues. The API had been implemented as previous paid work to specifically allow repairing Freetalk. Thus we'd reap what we sowed there.,8
"WoT: Second iteration of less critical performance fixes: Reduce transaction size from O(512) to O(1): Split up ""import trustlist"" transaction into 1 transaction for each trust.",1
"WoT: Second iteration of less critical performance fixes: Identity garbage collection: Stats show that less than 1000 of the 14000 WoT identities are still active (thanks to ArneBab). Thus, we need to GC abandoned identities.",1
WoT: Second iteration of less critical performance fixes: Process results of peer review of xor’s  Web of Trust bachelor’s thesis (which was about last year’s performance improvements).,
"Fred: does it provide sufficient benefits to re-add native acceleration for FEC, or crypto? Especially JVMs without gmp acceleration - Oracle on Windows.",2
,
User Experience - make things easier and more enjoyable to use,
"Single use node references with authentication token: 

Re: [freenet-dev] Financial allocation poll stage 3

[Matthew Toseland]

On 28/08/16 19:48, Ian Clarke wrote:
> On Sun, Aug 28, 2016 1:32 PM, Matthew Toseland mj...@cam.ac.uk wrote:
> On 28/08/16 19:29, Ian wrote:
>
>> On Sun, Aug 28, 2016 at 1:23 PM, Matthew Toseland <mj...@cam.ac.uk>
>> wrote:> That's not what was asked. My prioritization proposal
>> separates the
>
>> estimation of the value of completing a task from the estimation of the
>
>> task's cost. These can then be combined later to come up with a
>
>> prioritization.
>
>
>
>
> Which will inherently prefer "easy wins". Which is probably a good
>
> thing.
> I agree. The principle is very simple, maximize value relative to
> cost. Then all
> you need is a way to estimate value and cost, these are hard problems
> in the
> general case but I think our approaches will work well.
> So, if anyone disagrees with the approach, either they disagree that
> we must
> maximize value relative to cost (I'd love to hear that argument), or
> they need
> to propose a better approach to estimating value and cost.

It doesn't always work. Sometimes it is necessary to make a start on the
big-ticket items, even though they're costly.

The classic example of this is market failure in carbon credits: Carbon
trading finds the "easy wins" very quickly, but it's very poor at things
like encouraging building new wind farms rather than coal plants. In
fact it's so bad at this that an entirely separate mechanism is needed
in the electricity sector.

Does this reasoning apply to Freenet? Well, there are some fundamental
problems that should bear a high priority even though their immediate
visible impact to users may not be that spectacular. Security
improvements are an example (e.g. deploying Arne's darknet security
fix). Although here, darknet usability/performance enhancements are the
big one. Not only do they improve security, they may actually help to
get us some viral growth. Winning all around...

Really that sort of thing just means you've under-estimated the value of
a big task. Which is bound to happen, but I'm not sure how to avoid it...

> Ian.
> Ian Clarke
> Founder, The Freenet Project
> Email: i...@freenetproject.org



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Migrating the wikis and bugtracker: please keep the bugs!

[Matthew Toseland]

On 28/08/16 19:35, Ian Clarke wrote:
> On Sun, Aug 28, 2016 12:22 PM, Matthew Toseland mj...@cam.ac.uk wrote:
> What matters more is the bug tracker: The poll is problematic because
>
> half of the suggestions are technically illiterate.
>
>
> There was more than enough time for the technically literate to
> comment on and
> modify these suggestions.
> In most of these
>
> cases detailed design has been done and is on the bug tracker.
>
> Occasionally the records are on one of the wiki's. While there may be
>
> good financial reasons for moving services off Osprey, IMHO we need to
>
> keep that information *in some form*, even if it's only a static archive
>
> (in which case note the distinction between public and private
>
> bugs/comments).
>
>
>
>
> When I looked into it many years ago, migrating the bug tracker to a
>
> third party (while keeping data) didn't look feasible, but I believe the
>
> situation has improved since then - *if* we're just moving to another
>
> Mantis instance.
> I wasn't specifically thinking about moving away from Mantis, even
> though nobody
> seems to use it any more, although perhaps that is implied by shutting
> down
> osprey. 

It looks like it. But maybe we can migrate it. I *think*  there is now
an import/export facility. But even if we use a completely new bug
tracker, IMHO it is important to have the old data.

> I have found over the years that bugtrackers often just act as a way to
> record ideas and bugs, never to be seen again.
> I wonder whether the number of open bugs in our bugtracker is increasing,
> staying the same, or decreasing over time? What is the current process
> for
> deciding which issue should be tackled next? Can anyone describe it in
> a clear
> way? Somehow I suspect not.

7 bugs were modified in August. So lightly used, but still used.

And yes, the way we use the bug tracker, many "bugs" are feature
requests. IMHO this is legitimate.

Many of them contain design work. This will be lost if we dump the bug
tracker. In particular the set of suggestions in the poll around FOAF
and darknet enhancements corresponds roughly to a web of bugs around
that that I filed a few years ago. But the bugs give much more detail
and include some variants that are much preferable to the
half-thought-through stuff in the poll. Some of this work is mine, much
of it is lifted from discussions on FMS. It has a non-zero value,
especially if it can be searched. The temptation is to say it's all
rotten. That's just not true: a lot of it is still useful, one way or
another. Some of it - mostly actual bugs - is outdated. Steve and others
have been reasonably diligent in closing/suspending old bugs where the
reporter does not respond, so this is not that big a problem today.

Maybe we should use a different system for keeping track of feature
requests versus actual bugs. A wiki may be a reasonable solution.
However summarily executing it all would be unwise IMHO, resulting in
duplication of work at least. We should either try to migrate to another
Mantis instance, or at least take a snapshot for future reference.

And IMHO the same applies to the wiki: Before we shut it down, we should
take a snapshot, even if we're not going to actively host it anywhere.
We can't rely on the Internet Archive.

We could do this today for the "old" wiki (old-wiki.freenetproject.org),
since it's already read-only. I don't think we need the full history;
spam is not such a big problem any more thanks to work by Florent et al.

> Ian.
> Ian Clarke
> Founder, The Freenet Project
> Email: i...@freenetproject.org



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Financial allocation poll stage 3

[Matthew Toseland]

I have in private advised you on your career goals on several occasions,
though I'm hardly qualified to give such advice.

The serious, non-personal point here is that we need to plan to raise
more money if we want to have more paid employees.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Financial allocation poll stage 3

[Matthew Toseland]

On 28/08/16 19:29, Ian wrote:
> On Sun, Aug 28, 2016 at 1:23 PM, Matthew Toseland <mj...@cam.ac.uk> wrote:
>
>> Asking us not to take into account the technical
>> difficulty of each task when prioritizing arguably
> That's not what was asked.  My prioritization proposal separates the
> estimation of the value of completing a task from the estimation of the
> task's cost.  These can then be combined later to come up with a
> prioritization.

Which will inherently prefer "easy wins". Which is probably a good
thing. Some of the proposals are "WTF, why didn't you just send a patch"
though. :)
> Ian.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Financial allocation poll stage 3

[Matthew Toseland]

On 28/08/16 19:23, Matthew Toseland wrote:
> On 02/08/16 23:47, x...@freenetproject.org wrote:
>> Hereby we begin the 3rd stage of the financial allocation poll.
>>
>> You may participate by filling your votes in to this spreadsheet and
>> mailing it back:
>> https://github.com/xor-freenet/freenet-money-poll/archive/2016-stage3.zip
>>
>> It has been created with LibreOffice but may hopefully as
>> well work with OpenOffice and any Microsoft Office version greater than or
>> equal to 2007 SP2.
>>
>> With complex file formats like spreadsheets, it may be possible that private
>> data such for example as your username or hostname leaks into the file. If
>> you are concerned about that, you can:
>> 1. Open the ODS file with a ZIP tool and review the plain text files it
>> actually consists of.
>> 2. Send back a CSV file instead. CSV is a plain text
>> file format so you can review it with any text editor. For LibreOffice,
>> instructions on how to export a CSV can be found at:
>> https://help.libreoffice.org/Calc/Importing_and_Exporting_CSV_Files
>>
>> As almost 3 months have passed since the DuckDuckGo donation, we would like
>> to continue to the next stage quickly - so please vote within 1 week.
>>
>> Thank you!
> I believe the voting was extended to today, so I attach my votes.
>
> I am currently somewhat detached from the project, partly due to legal
> issues I am seeking to clarify, partly because I believe we have missed
> our opportunity. But another may arise in a few decades' time and it's
> still worth trying to build something.
>
> As I have mentioned in another thread, many of the suggestions are
> technically illiterate, some are already accomplished, and the relative
> sizes vary enormously. Asking us not to take into account the technical
> difficulty of each task when prioritizing arguably makes it more
> difficult rather than less difficult. E.g. search is a major research
> project - but approximations that are usable for limited sizes, e.g. for
> sub-communities within Freenet, are much more feasible.
>
> Plus tasks overlap or appear in multiple categories... But I was busy
> when this was being discussed... My point is take my answers with a
> large pinch of salt as you would anyone else's. Right now I'm not
> contributing, though I hope to do so in future. Also please don't stick
> rigidly to the over-precise descriptions of the tasks, very often a
> better way to do it is already on the bug tracker!
>
> There are also several important missing points: There's lots we could
> do on performance (e.g. bloom filter sharing and equivalents), and other
> stuff, e.g. deploying ArneBab's fix for the fundamental security issues
> in darknet.
>
> In particular there's lots of code that just hasn't been merged, or that
> needs finishing. For example, what's Icicle? Yet another mobile app to
> make darknet connections easier? How many does that make now? Are either
> of them usable? Do either of them have a reasonable security model?
> Personally I dislike the idea of using texts to set up connections -
> bumping is much preferable. So I've given that one 0. But we DO need to
> get this sorted out somehow. Yet another example is FOAF - it's not just
> about *finding* friends, it's about darknet performance, connectivity
> for invites, *and* finding friends... All the stuff on FOAF and invites
> is on the bug tracker in great detail.
>
> I also disagree with the implications of the language-related
> suggestions. Sure, we always need to make things clearer. But we can't
> entirely avoid jargon. Twitter and Facebook have lots of jargon. Jargon
> isn't necessarily a bad thing per se. Of course you need to be careful
> with it - define it, but don't force people to look stuff up!
>
> Fewer words is not always clearer. And unfortunately one of Freenet's
> fundamental problems is the way its developers and its users see the
> world are irreconcilably different. This makes things like bootstrapping
> hard. But we've worked on that over the years and come up with
> reasonable compromises; there is space to do useful stuff.
>
> Re legal stuff, do we know that "some users were completely innocent"?
> If there is a genuine issue here we can probably get help for free by
> working with people like EFF...
>
> My two cents, in a rather scattered "everything" form to offset some
> other people's contributions... :)
>
> Hope it works out!
One point about protocol here:

Clearly we can't *do* "everything" for $20K. Ultimately if there is no
clear victor, somebody will have to choose a set of stuff for our paid
developer(s) to work on, based on the votes but also informed by some
estimate of difficulty. I guess that's the next stage...



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Financial allocation poll stage 3

[Matthew Toseland]

On 02/08/16 23:47, x...@freenetproject.org wrote:
> Hereby we begin the 3rd stage of the financial allocation poll.
>
> You may participate by filling your votes in to this spreadsheet and
> mailing it back:
> https://github.com/xor-freenet/freenet-money-poll/archive/2016-stage3.zip
>
> It has been created with LibreOffice but may hopefully as
> well work with OpenOffice and any Microsoft Office version greater than or
> equal to 2007 SP2.
>
> With complex file formats like spreadsheets, it may be possible that private
> data such for example as your username or hostname leaks into the file. If
> you are concerned about that, you can:
> 1. Open the ODS file with a ZIP tool and review the plain text files it
> actually consists of.
> 2. Send back a CSV file instead. CSV is a plain text
> file format so you can review it with any text editor. For LibreOffice,
> instructions on how to export a CSV can be found at:
> https://help.libreoffice.org/Calc/Importing_and_Exporting_CSV_Files
>
> As almost 3 months have passed since the DuckDuckGo donation, we would like
> to continue to the next stage quickly - so please vote within 1 week.
>
> Thank you!
I believe the voting was extended to today, so I attach my votes.

I am currently somewhat detached from the project, partly due to legal
issues I am seeking to clarify, partly because I believe we have missed
our opportunity. But another may arise in a few decades' time and it's
still worth trying to build something.

As I have mentioned in another thread, many of the suggestions are
technically illiterate, some are already accomplished, and the relative
sizes vary enormously. Asking us not to take into account the technical
difficulty of each task when prioritizing arguably makes it more
difficult rather than less difficult. E.g. search is a major research
project - but approximations that are usable for limited sizes, e.g. for
sub-communities within Freenet, are much more feasible.

Plus tasks overlap or appear in multiple categories... But I was busy
when this was being discussed... My point is take my answers with a
large pinch of salt as you would anyone else's. Right now I'm not
contributing, though I hope to do so in future. Also please don't stick
rigidly to the over-precise descriptions of the tasks, very often a
better way to do it is already on the bug tracker!

There are also several important missing points: There's lots we could
do on performance (e.g. bloom filter sharing and equivalents), and other
stuff, e.g. deploying ArneBab's fix for the fundamental security issues
in darknet.

In particular there's lots of code that just hasn't been merged, or that
needs finishing. For example, what's Icicle? Yet another mobile app to
make darknet connections easier? How many does that make now? Are either
of them usable? Do either of them have a reasonable security model?
Personally I dislike the idea of using texts to set up connections -
bumping is much preferable. So I've given that one 0. But we DO need to
get this sorted out somehow. Yet another example is FOAF - it's not just
about *finding* friends, it's about darknet performance, connectivity
for invites, *and* finding friends... All the stuff on FOAF and invites
is on the bug tracker in great detail.

I also disagree with the implications of the language-related
suggestions. Sure, we always need to make things clearer. But we can't
entirely avoid jargon. Twitter and Facebook have lots of jargon. Jargon
isn't necessarily a bad thing per se. Of course you need to be careful
with it - define it, but don't force people to look stuff up!

Fewer words is not always clearer. And unfortunately one of Freenet's
fundamental problems is the way its developers and its users see the
world are irreconcilably different. This makes things like bootstrapping
hard. But we've worked on that over the years and come up with
reasonable compromises; there is space to do useful stuff.

Re legal stuff, do we know that "some users were completely innocent"?
If there is a genuine issue here we can probably get help for free by
working with people like EFF...

My two cents, in a rather scattered "everything" form to offset some
other people's contributions... :)

Hope it works out!
Task category / Task,Value,% of total
Remaining,0,0%
,,
Instructions,,
"The goal of this stage of the funding allocation procedure is to estimate ""value"" of individual tasks.
Value shall measure how useful each task would be to our users, to developers, to health of the project, etc.
Every participant gets 1000 units of value to allocate between the tasks, starting from an equal distribution of units.
So the only thing you should edit is to allocate those 1000 units in the ""Value"" column of each task. Everything else will be computed automatically.
LibreOffice / OpenOffice can highlight the cells you should fill in: Go to ""View"" and select ""Value highlighting"" (CTRL+F8).

The tasks are grouped into different areas such as ""Speed"" and ""User 

[freenet-dev] Migrating the wikis and bugtracker: please keep the bugs!

[Matthew Toseland]

On 20/08/16 07:01, Florent Daigniere wrote:
> On Fri, 2016-08-19 at 22:31 -0400, Steve Dougherty wrote:
>> On 08/12/2016 04:00 PM, Florent Daigniere wrote:
>>> On Thu, 2016-08-11 at 14:28 +, Ian Clarke wrote:
 Why don't you set a date since you're the one that would do it?
>>>
>>> I have created https://github.com/freenet/wiki/wiki ... and will
>>> remove the old wikis as soon as we deploy the new website design.
>> Huh? Why? 
> Because that's what the boss said we should do... and it took me 30s.
>
>> There's a bunch of documentation on the wikis already - what's
>> the reasoning for
>>
>> 1. changing wiki platforms (making administration GitHub's problem
>> and responsibility?)
> Ian has used another argument: github's syntax is trendier.
>
> But yes, you have made my points.
>
>> 2. not transitioning existing documentation (licensing?)
> I have made it clear that I don't have the will-power to do
> it. mrsteveman1 apparently does, so a lot of progress has been made in
> that area already, thanks to his efforts.
>
> I did mention to Ian that moving the wiki to another engine is yet-
> another-recurring thread (just like "the website design sucks"), where
> in last iteration concerns were raised regarding the licensing of some
> of the content. Apparently no one cares, so why should I?
>
>> This seems like discarding lots of work - especially about FCP and
>> GSoC projects.
>>
> It doesn't have to be; help with the migration if you care.
>
> Florent
Please could we keep a static copy of the content of the old wikis? Even
if we don't host it, somebody should spider/export a tarball for reference.

Just because the Internet Archive hosts a few pages doesn't mean it has
everything; often it only has part of a mirrored site.

What matters more is the bug tracker: The poll is problematic because
half of the suggestions are technically illiterate. In most of these
cases detailed design has been done and is on the bug tracker.
Occasionally the records are on one of the wiki's. While there may be
good financial reasons for moving services off Osprey, IMHO we need to
keep that information *in some form*, even if it's only a static archive
(in which case note the distinction between public and private
bugs/comments).

When I looked into it many years ago, migrating the bug tracker to a
third party (while keeping data) didn't look feasible, but I believe the
situation has improved since then - *if* we're just moving to another
Mantis instance.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Which #FOSS project should get a free, EU-funded audit? Vote now:

[Matthew Toseland]

On 04/07/16 21:29, hyazin...@emailn.de wrote:
> All reading this who are living in an EU member state can participate at this 
> survey - briefing with links to survey link: 
> https://juliareda.eu/2016/06/eu-free-software-security-audits/

Be careful what you wish for. The technical feasibility of auditing
Freenet is highly doubtful. We simply have too much code and not enough
of a defined threat model.

There are other candidates which make more sense at this stage.
> Greetings,
> Torben Lechner



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Reducing the peer count (2 pull requests)

[Matthew Toseland]

On 13/06/16 17:09, Arne Babenhauserheide wrote:
> Hi,
>
>
> I’m writing for two reasons:
>
> - The new 10KiB/s minimum bandwidth is too high for some users.
> - With 1474 few users are able to keep more than 70 connections.¹
...
> ¹: See https://asksteved.com/stats/plot_peer_count.png — this used to
>have a peak around 100 before 1474, and people report that their
>nodes cannot keep their connections anymore.
>
I thought this occurred suddenly in 1474? That suggests something fishy
is going on, the long term issues you try to address here may not be
relevant?

One possible theory is that the peer count patcher suddenly changed from
accepting probe requests to ignoring them. This would explain both the
reduction in average peer counts and the odd probe statistics. Is there
any way we can eliminate this hypothesis, for example by asking people
using it?



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Freenet 0.7.5 build 1474 is finally released!

[Matthew Toseland]

If your node does not update automatically, you may need to update it manually 
as described in my previous message. The simplest way to deal with the problem 
is to shut down Frost, FMS and Web of Trust and reboot and the auto-update 
should work given an hour or two.

Thanks to everyone involved in getting this build out in difficult 
circumstances!

=

Freenet 0.7.5 build 1474 is now available. This is an emergency bugfix release, 
hence I am releasing it rather than Steve while he is incapacitated. It fixes 
some important bugs, one of which is involved in the current attacks on Frost 
and Sone.

Summary of changes:

* Fix the Frostbite bug: if the node downloads a malicious key, this would 
cause the whole client layer to break. This is currently being actively used to 
attack Frost and Sone.
* Automatically upgrade nodes to use the minimum bandwidth limit if necessary. 
Some nodes were unable to start up because their bandwidth limit was too low. 
Apologies to anyone affected by this. Also, improve the logic that sets the 
per-second bandwidth limits from a monthly setting. Obviously, you should be 
very careful if using Freenet on a connection with a monthly transfer limit.
* Minor security improvements to the web interface.

If your node is unable to update because of the Frostbite bug, please turn off 
the affected applications (unload the Web of Trust and Sone plugins and shut 
down Frost), and then restart the node. It should pick up the update within a 
few hours. If it still doesn't work, the update.cmd or update.sh scripts may 
fix the problem, but they will access our website in a traceable manner.

Thank you for using Freenet!

- Matthew Toseland

Git shortlog:

Bert Massop (4):
  BloomFilter: additional sanity checking of length and hash count
  Add more splitfile sanity checks
  Make KeyListenerTracker more resilient
  Fix a corner case in BloomFilter length

Florent Daigniere (7):
  Set rel='noreferrer noopener' where appropriate
  Merge branch 'do-not-die-on-too-low-bandwidth' of 
https://github.com/ArneBab/fred-staging-1 into 
ArneBab-do-not-die-on-too-low-bandwidth
  Merge branch 'ArneBab-do-not-die-on-too-low-bandwidth' into next
  Merge branch 'frostbite-hotfix' of https://github.com/bertm/fred-staging 
into bertm-frostbite-hotfix
  Merge branch 'bertm-frostbite-hotfix' into next
  Merge branch 'avoid-claiming-magic' of 
https://github.com/Thynix/fred-staging into Thynix-avoid-claiming-magic
  Merge branch 'Thynix-avoid-claiming-magic' into next

Matthew Toseland (1):
  Build 1474, mandatory in a week but crucial bugfixes

Steve Dougherty (3):
  Merge remote-tracking branch 'ArneBab/do-not-die-on-too-low-bandwidth' 
into next
  Merge remote-tracking branch 'nextgens/use-noreferrer' into next
  l10n: avoid suggesting tracing is impossible

drak@kaverne (5):
  FIX: on too low bandwidth, use min bandwidth
  node init: log increase of bandwidth to minimum
  fixed bandwidth selection per month
  whitespace (tabify)
  use asymptoticDlFraction + fix whitespace
gpg: Signature made Sun 05 Jun 2016 15:37:50 BST using RSA key ID 1946AA94
gpg: Good signature from "Matthew Toseland (2013-2018 key, higher key length) 
<matt...@toselandcs.co.uk>"
gpg:         aka "Matthew Toseland (2013-2018 key, higher key length) 
<t...@amphibian.dyndns.org>"




signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Build 1474 status and the Frostbite attack

[Matthew Toseland]

Freenet build 1474 has been partially released. It includes a critical bugfix
for the "Frostbite" bug: if you visit a malicious key, downloads can stop
working. This is being actively exploited on Frost and Sone/WoT. Unloading
WoT / turning off Frost and restarting the node should make it work again.
Unfortunately the official release manager Steve is indisposed, and nextgens
and I are still working on completing the release; it should be inserted
into the auto-update system tomorrow, and thus "officially" released.

The build has been released as far as the website, so new installs will be
1474. Sorry about this, we will try to make our recovery procedures more 
robust in future... Please relay this message to FMS etc if possible, thanks!

You can manually update to the new build as follows:

Option 1: Use update.sh or update.cmd

This is not anonymous, in that it downloads the jar file over HTTPS from our
server, so anyone listening could tell that you run Freenet. Also there
have been some reported bugs (one problem with update.sh was fixed just by
running it again, otherwise contact #freenet on irc.freenode.net - note that
this is also not anonymous).

Option 2: Download the jar from Freenet on a working node and replace them.

First, download the following key:
CHK@jCVrtDlDgly4kqNRPAH4o7j16I5Fcy39ka6Qz2~NOko,kbxLQ-wHV4S8YgPUme3y9HKYopc1FWeEMUaZ~zDeSqM,AAMC--8/freenet-build01474.jar

Signature is here (if you have gnupg and my pubkey):
CHK@~gTf8kWGI0X34XOI~pI-hagrddxzUJdbPIk20nYoLjs,a7SNdm2Sq7u3sOwjX5GyVkitpR8J92~VGdO~tSynLto,AAMC--8/freenet-build01474.jar.sig

Now shut down the node.

Open wrapper.conf in a text editor. 
There should be a line that says something like:
wrapper.java.classpath.1=freenet.jar.new

If it says freenet.jar.new then change it to freenet.jar. 
If it says freenet.jar then don't do anything.

Now replace freenet.jar with the jar you just downloaded from Freenet.



Official release notes:

$ git tag -v build01474
object ced0ba20a7ffba7fdf05466d00bf6cb585c28bc9
type commit
tag build01474
tagger Matthew Toseland <matt...@toselandcs.co.uk> 1465137470 +0100

2016-06-05

Freenet 0.7.5 build 1474 is now available. This is an emergency bugfix release, 
hence I am releasing it rather than Steve while he is incapacitated. It fixes 
some important bugs, one of which is involved in the current attacks on Frost 
and Sone.

Summary of changes:

* Fix the Frostbite bug: if the node downloads a malicious key, this would 
cause the whole client layer to break. This is currently being actively used to 
attack Frost and Sone.
* Automatically upgrade nodes to use the minimum bandwidth limit if necessary. 
Some nodes were unable to start up because their bandwidth limit was too low. 
Apologies to anyone affected by this. Also, improve the logic that sets the 
per-second bandwidth limits from a monthly setting. Obviously, you should be 
very careful if using Freenet on a connection with a monthly transfer limit.
* Minor security improvements to the web interface.

If your node is unable to update because of the Frostbite bug, please turn off 
the affected applications (unload the Web of Trust and Sone plugins and shut 
down Frost), and then restart the node. It should pick up the update within a 
few hours. If it still doesn't work, the update.cmd or update.sh scripts may 
fix the problem, but they will access our website in a traceable manner.

Thank you for using Freenet!

- Matthew Toseland

Git shortlog:

Bert Massop (4):
  BloomFilter: additional sanity checking of length and hash count
  Add more splitfile sanity checks
  Make KeyListenerTracker more resilient
  Fix a corner case in BloomFilter length

Florent Daigniere (7):
  Set rel='noreferrer noopener' where appropriate
  Merge branch 'do-not-die-on-too-low-bandwidth' of 
https://github.com/ArneBab/fred-staging-1 into 
ArneBab-do-not-die-on-too-low-bandwidth
  Merge branch 'ArneBab-do-not-die-on-too-low-bandwidth' into next
  Merge branch 'frostbite-hotfix' of https://github.com/bertm/fred-staging 
into bertm-frostbite-hotfix
  Merge branch 'bertm-frostbite-hotfix' into next
  Merge branch 'avoid-claiming-magic' of 
https://github.com/Thynix/fred-staging into Thynix-avoid-claiming-magic
  Merge branch 'Thynix-avoid-claiming-magic' into next

Matthew Toseland (1):
  Build 1474, mandatory in a week but crucial bugfixes

Steve Dougherty (3):
  Merge remote-tracking branch 'ArneBab/do-not-die-on-too-low-bandwidth' 
into next
  Merge remote-tracking branch 'nextgens/use-noreferrer' into next
  l10n: avoid suggesting tracing is impossible

drak@kaverne (5):
  FIX: on too low bandwidth, use min bandwidth
  node init: log increase of bandwidth to minimum
  fixed bandwidth selection per month
  whitespace (tabify)
  use asymptoticDlFraction + fix whitespace
gpg: Signature made Sun 05 Jun 2016 15:37:50 BST using RSA key ID 1946AA94
gpg: Go

Re: [freenet-dev] Proposal to make unit-tests in Freenet easier to implement

[Matthew Toseland]

On 07/05/16 11:35, Ian Clarke wrote:
> Hey Martin,
> This sounds like a great idea. Classes should only require the dependencies 
> they
> actually need, partially because it makes unit testing much easier, as you 
> point
> out.
> So if a large node object is being passed to classes that only use a small 
> part
> of that object, then it's definitely an architectural problem (it's been a 
> very
> long time since I delved into the codebase). The node option is basically
> functioning as a static dependency.
> If the solution is a publish-subscribe architecture, then one option you might
> look at is Guava's EventBus, it is designed for exactly this purpose:
> https://github.com/google/guava/wiki/EventBusExplained
> Another way to avoid this kind of architectural problem is to use a dependency
> injection framework like Guice (older but more widely used) or Daggar (newer 
> and with faster startup time and better on mobile).
> For anyone interested in avoiding this kind of software architecture mistake,
> and many other mistakes, I highly recommend the book “Clean Code” by Robert C.
> Martin. It's full of great ideas and best practices for architecting code (not
> Java-specific, but most of the examples are Java).
> Ian.

I am concerned that if we use Martin's approach directly we could end up
with another layer of indirection for little purpose. That is, DMT
Message's come in, and then we convert them to events on a 1:1 basis,
and then... If so, it would be better just to subscribe to Message's in
the first place! And furthermore the existing infrastructure *does do
that*...

At the moment, most of that level of the node *does* subscribe to
Message's via MessageFilter's. The MessageCore matches incoming messages
to MessageFilter's. NodeDispatcher deals with what's left.
Unfortunately, "what's left" is quite big!

Maybe we should replace NodeDispatcher with a series of more general
subscriptions in the relevant parts of Freenet? There would be some
redundant checking. There would still need to be a class which hooks
incoming requests and starts them. Announcements could indeed be handled
by OpennetManager directly.

On the other hand, if Martin's approach is taken carefully, with only
the relatively obscure, rare events such as IP re-detection, that may
make more sense.

His two examples were:

1) IP re-detection after setting a PeerNode's incoming detected IP
address. This is unnecessary; the method on the PeerNode could just as
easily call the redetect method.

2) Starting an announcement. OpennetManager could subscribe directly?
>
>
>
>
>
> On Sat, May 7, 2016 5:10 AM, Martin Byrenheid martin.byrenh...@tu-dresden.de 
> wrote:
> Hello everyone,
>
>
>
>
> I've spend some time thinking about how to make it easier to test Freenet's
>
> different subsystems, especially without having to instantiate the whole
>
> Freenet Node class for almost every test. One possibly helpful idea that came
>
> to my mind is to decouple classes by using a publish-subscribe mechanism,
>
> where each instance can subscribe to events (e.g. received a new announcement
>
> request) and publish other events together with corresponding data (e.g. the
>
> message and the neighbor node where the request came from). This way, many
>
> subsystems could then trigger methods in other subsystems without having to
>
> know them directly and also might not need a reference to the Node class
>
> anymore, making them much easier to test.
>
>
>
>
> I've integrated some examples within the NodeDispatcher-class and pushed it
>
> into my Github repository [1]. Due to its rather high level of abstraction,
>
> the publish-subscribe mechanism handles all attached data just as objects,
>
> which is not nice regarding type safety. However, I haven't yet found a better
>
> solution since I don't have much experience with Java and I first want to hear
>
> your opinion about this approach.
>
>
>
>
> Martin
>
>
>
>
> [1]
>
> https://github.com/yadevel/fred/commit/f4ce1b066ad673b995ea2a729ba21b2a7e932e5b
>
> ___
>
> Devl mailing list
>
> Devl@freenetproject.org
>
> https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
>
>
>
> Ian Clarke
> Founder, The Freenet Project
> Email: i...@freenetproject.org
> ___
> Devl mailing list
> Devl@freenetproject.org
> https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl




signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Proposal to make unit-tests in Freenet easier to implement

[Matthew Toseland]

On 07/05/16 11:10, Martin Byrenheid wrote:
> Hello everyone,
>
> I've spend some time thinking about how to make it easier to test Freenet's 
> different subsystems, especially without having to instantiate the whole 
> Freenet Node class for almost every test. One possibly helpful idea that came 
> to my mind is to decouple classes by using a publish-subscribe mechanism, 
> where each instance can subscribe to events (e.g. received a new announcement 
> request) and publish other events together with corresponding data (e.g. the 
> message and the neighbor node where the request came from). This way, many 
> subsystems could then trigger methods in other subsystems without having to 
> know them directly and also might not need a reference to the Node class 
> anymore, making them much easier to test.
>
> I've integrated some examples within the NodeDispatcher-class and pushed it 
> into my Github repository [1]. Due to its rather high level of abstraction, 
> the publish-subscribe mechanism handles all attached data just as objects, 
> which is not nice regarding type safety. However, I haven't yet found a 
> better 
> solution since I don't have much experience with Java and I first want to 
> hear 
> your opinion about this approach.

Some people have talked about Mockito, though it means another
dependency. Personally I find that objectionable unless a usable version
is packaged in common linux distributions, for reasons of code integrity.

It's not actually that hard to instantiate a node. My work on
simulations does it, and so do the existing multi-node tests in
freenet/node/simulator/. That code also involves bypassing the transport
layer, bypassing the message layer, using a proxy NodeDispatcher to keep
track of where messages go between nodes, and a subclass of
RequestTracker. Also, you don't have to _start_ the node.

The other approach is just to use more interfaces (I have done some of
this)... Your events interface is a relatively extreme form of this,
which is interesting, although I'm not sure if it's the right approach
(it might be). Mockito has a big advantage in that we can just create a
no-op Node or IPAddressDetector as needed, without needing to add lots
more interfaces that only have one implementation at run-time.

There is the more specific question of how much do we want
NodeDispatcher to do anyway? Arguably a lot of its methods - even stuff
like starting an announcement - are very similar, reading fields from a
message, doing sanity checks etc. Perhaps it should unwrap the Message's
rather than passing them elsewhere, to keep this sort of code all in one
place?

Some things could usefully be made static. The reason that more aren't
is the need to simulate multiple nodes in a single JVM. However things
like IPAddressDetector are, or should be, per-system not per-node.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Simulations was Re: Planning process step #1: Broad resource areas

[Matthew Toseland]

On 06/05/16 03:48, x...@freenetproject.org wrote:
> This mail shall contain only my ideas about your proposals.
> I've posted my proposals in a reply of its own.
> It's easy to discuss the both of them in separate threads.
>
>
> On Friday, May 06, 2016 12:02:08 AM Ian Clarke wrote:
>> * Speed -
>> Make Freenet requests and responses faster
> Working on the core network algorithms is one of the most complex tasks we 
> have. Only the "src/freenet/node" directory alone is 74 000 lines of code 
> (with only 2000 lines of unit tests for that :|).
> Given that Matthew said he's not available with the current level of funding, 
> we would have to spend very significant education costs on anyone else doing 
> this.
> Only the education might easily exceed the 6 months of planning you suggested.
> And before any attempts at changing things are released, the whole 12 months 
> of funding could expire:
> A new developer would first have to understand the codebase. Then write 
> simulations / measurements. And only then he could start modifying the actual 
> network. And then he might still have to wait for the half a year release 
> cycle of Freenet to allow his first experiments to hit the network.
> And then as there's usually more than one cycle of doing changes and 
> measuring 
> the results, he'd have to wait for another release afterwards...
>
> But there are very good news about the network performance:
>
> - AFAIK, Matthew is doing Freenet simulations as his bachelor's thesis. He 
> talked about the simulations on IRC, they also seem to include load 
> management 
> stuff. He did send quite a few pull requests from that as well.
> So given that he is *the* core network expert, I would say we should wait for 
> the results of his work to be finished before we spend any time on educating 
> someone else. Maybe his improvements will be enough already! :)

My thesis will go in on Monday. I don't want to publish it until after
the whole formal process is completed and I have my final grade, some
time in June. The code is already available though.

It is indeed about simulations, and in particular:
- Fast simulations for routing changes (and I mean fast enough to
possibly include in JUnit).
- Load management simulations. Somewhat limited, but sufficient for
basic sanity checking.

I have not implemented any substantial new load management algorithm.
But the new simulations should be sufficient to sanity check one when
and if somebody does. There are several proposals, and some of them are
relatively easy to implement.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Re darknet enhancements

[Matthew Toseland]

On 06/05/16 03:04, x...@freenetproject.org wrote:
> Thanks!
> I'll start with my proposals. I'll put ideas about your proposals in a 
> separate reply, it's easier to discuss different people's proposals in a 
> thread of their own.

> USE FRIENDLINESS
>
> Darknet enhancements.
>
> These are smaller pieces of work, so I will suggest a few:
>
> - Single use node references with authentication token: Currently, to create 
> a 
> darknet connection, *both* users have to add the node reference of each 
> other. 
> Tokenized node references would allow one person to use your reference to add 
> himself as your peer *without* you having to add his node reference manually.
> I think this is a major usability improvement, as the general workflow of 
> other stuff such as phones / WhatsApp is that you do NOT have to both add a 
> "reference" of each other. People just aren't used to this.
>
> - Darknet invitation bundles: Feature for adding a single use node reference 
> to an installer executable. People could hand out the installer executable to 
> their friends to allow them to connect by darknet instantly.
> Thanks to ArneBab for this idea!
>
> - Short node references: Currently, node references fill almost half a page 
> of 
> paper. This doesn't fit into a Facebook chat window for example.
> As most users are likely to not only use darknet but also opennet, we could 
> upload node references to Freenet itself as a random KSK, with for example 
> 128 
> bit entropy to be ~ 25 letters.
> This would also make sense to combine with the aforementioned single use node 
> references.
>
> - Friend-of-a-friend connection suggestions ("FOAF"). Like the Facebook 
> friend 
> finder, Freenet could be improved to tell you about darknet peers of your 
> peers. You could then chose to add them as your peers. Part of this codebase 
> already exists.
>
> - Friend requests, like in Facebook: With primitive FOAF, both peers would 
> still have to add each other. With friend requests, peers of your peers could 
> just request to connect to you.
> Together with the aforementioned FOAF connections, this could have a very 
> similar UI to how adding friends on Facebook works. This should thus be a 
> huge 
> usability improvement.
>
> - Darknet chat improvements: Freenet allows you to send messages to your 
> darknet peers. The UI of that is very primitive. It should be improved to be 
> similar to e.g. the Facebook chat. There is also a very high probability of 
> losing messages: Messages are not queued to disk, so restarting before a 
> message is sent results in its loss. This should be fixed.
>
> We've discussed how to implement these ideas, so I'm aware of how it would 
> work and feel capable of doing this.

Darknet enhancements are usability, security *and*  performance. The
whole bundle also includes:
- FOAF *connections*. Greatly improve performance for darknet.
- Using FOAF connections to improve connectivity.
- Using FOAF connections for invites (invites as described above will
mostly not work in the real world, using a port forwarded peer as an
intermediary will).

Hence I see them as high priority, but it also makes it awkward to fit
them into a high-level points allocation.

Having said that it's possible volunteers may do some of this.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Proposal for a democratic process to efficiently allocate resources (including the $25k)

[Matthew Toseland]

On 06/05/16 00:10, x...@freenetproject.org wrote:
> On Friday, May 06, 2016 12:33:12 AM x...@freenetproject.org wrote:
>> At the current exchange rate, it would be 23.6 hours/week.
>> This is the average of what I had delivered during the past few months of
>> work. In other words, the $27500 was chosen from that average as:
>> 23.6 hours/week * 52 weeks * total all-inclusive cost for me per hour.
> In case this wasn't clear enough:
>
> The 23.6 hours/week are not precisely the invoice average, but already 
> adjusted to the current currency exchange rates.
> I.e. my payment per hour would be less due to worse exchange rates, so this 
> is 
> an improvement to Freenet's benefit! :)
>
> (And no, I don't want adjustment so I get the same as before. I shall first 
> finish fixing WoT performance before I wish for any positive change of my 
> payment.)
IMHO it's important for FPI to pay you a fixed rate in your local currency.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Freenet New User Usecases

[Matthew Toseland]

On 02/05/16 23:05, Arne Babenhauserheide wrote:
> Hi,
>
> Florent asked me today what Freenet usecases can be done easily.
>
> I gathered some, most usable first.¹
>
>
> The first few are also shown in an example video session ☺
> https://www.youtube.com/watch?v=U8a8em-4m0w

Should this be linked from somewhere?
>
>
> * No Requirements
> This is what we can do without installing anything else:
>
> ** read what other users wrote and download what they share 
Only in terms of freesites, can't read forums without installing
WoT/Sone/FMS.
>
> hard to discover due to missing activelinks in default config: the links look 
> boring
Waiting for them to load is bad too. Javascript? More aggressive
prefetching?
>
> ** follow updates of others
>
> requires clicking on "Messages"
>
> -> Need update notifications per bookmarks like Winterface
>
> ** upload files confidentially and send them to friends by other means 
>
> This is a multistep process: It should be drag and drop -> a pastebin like 
> https://share.riseup.net/
>
> But backed by Freenet, so no one could censor or trace it. No central control.
There's a plugin for that isn't there? Or was it an app?
>
>
>
> * need to activate official plugin
> all of these need WoT
>
> ** write a flog entry
> multistep, prone to breakage, spills upload time
>
> ** write a freemail (whistleblower stuff!)
>
> breaks by "other ID not yet known".

That's bad. :(
I believe solutions were discussed? One option would just be to wait and
retry automatically?

In any case if you want them to see your post, you need to announce your
WoT identity. Fortunately FSNG and other docs help with this.

Using CAPTCHAs directly rather than WoT is an option (extend Freemail
0.1?), may be faster, but more vulnerable.
>
> * need plugin in default bookmarks
> ** write and comment in Sone
> also needs WoT setup
>
>
>
> * need inofficial plugin
> ** publish with Sharesite
> spills upload time
> could be made official (mostly reviewed)
>
>
>
> * need external application
> ** write in FLIP (Chat)
> spills activity time
>
> ** write in FMS (Forums)
> ** upload complex website
> - jSite or pyFreenet

"Upload a directory" is easy, only need basic UI. I thought that was
what Sharesite did?
>
>
>
> * need plugin + external application
> ** run inproxy to host a site over Freenet (SCGIPublisher or what 
> [[https://bluishcoder.co.nz/2015/09/14/using-freenet-for-static-websites.html][what
>  bluishcoder does]])
>
> https://github.com/saces/SCGIPublisher
Same thing with wget???
>
>
>
> ¹: Ideas for estimating the cost: clicks (.) + decisions (|) + wait time 
> (mm:ss)
>
>
> Best wishes,
> Arne



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Law-enforcement lying to courts about how Freenet works

[Matthew Toseland]

On 02/05/16 13:34, Arne Babenhauserheide wrote:
> Matthew Toseland writes:
>
>> You can still do a classic correlation attack: Connect to the node for
>> the whole duration of the request and count the proportion of the file
>> they've fetched from you…
> This might not work as well as simply theory says, due to FOAF
> routing. The first step is essentially random routing, since it routes
> to your peers, not only to you, and one third of these are long
> distance. And if you’re the best target for the content, then the
> requests could still come from far off. And the second step still uses
> FOAF routing, so it might radically change direction (probability arount
> 10% for 10 long distance peers).

For the benefit of others reading this thread, there is a pull request
to use true random routing for the first 2-3 hops. This should improve
anonymity somewhat, and may improve performance in some cases (inserts),
but likely reduce it for others (requests).
https://github.com/freenet/fred/pull/529

With pure random routing on the first hop, or pure non-FOAF routing (at
least if we know the node's peers), a correlation attack is
straightforward (maintaining connections long term to a large part of
the network being the hard part).

FOAF introduces some uncertainty, but IMHO the difference between the
number of requests seen at 1 hop from the originator and the number seen
at 2 hops away is large enough that you could still implement the
attack. This is viable even on darknet - but getting a connection to
lots of nodes on darknet is hard.
> However we know that it is possible to devise attacks against
> Opennet. It won’t be trivial, but it is possible in theory and I am sure
> that it can be implemented.
Yes. There are other attacks. The one we can't fix is "connect to
everyone all the time and do a correlation attack". Granted that takes
some resources to implement - but not a lot of resources.
>
> So what’s bothering here is not that they have an attack against
> Opennet, but that they lie to court about the capabilities of their
> attack (regardless whether its intentional or by misunderstanding
> Freenet).
Absolutely.
>
> An attack against Opennet is also not a problem for the goals of
> Freenet. An extremely cheap attack, however, is a problem, because that
> would allow for surveillance of all users. If all actions in Freenet can
> be tracked with a rack of 10-20 Computers (0.1% of the active Freenet
> users), that’s a problem, because it allows recording what people
> do. 
Connecting long-term to every node on opennet is cheap, in that sort of
ballpark. It requires some hardware and some bandwidth and some geeks.
But it's well within the reach of a government contractor. You can run
large numbers of (possibly virtual) nodes on modest hardware.

But as you say, what they are doing is little more than a fishing
expedition, if it's as they describe. They will quickly discover that
very few people who send 3 requests from an illegal file at HTL 18
actually have any illegal material on their computers ... So they'll
probably implement some sort of filtering eventually, e.g. a higher
threshold. But the reasoning given is clearly bogus.

> Insert the Kafkaeske response to the strong nothing to hide argument
> here: 
>
> Solove, Daniel J., 'I've Got Nothing to Hide' and Other
> Misunderstandings of Privastrong . San Diego Law Review, Vol. 44,
> p. 745, 2007; GWU Law School Public Law Research Paper
> No. 289. Available at SSRN: http://ssrn.com/abstract=998565
Must have a look at that, thanks!
>
> Best wishes,
> Arne



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Law-enforcement lying to courts about how Freenet works

[Matthew Toseland]

On 01/05/16 22:32, x...@freenetproject.org wrote:
> On Sunday, May 01, 2016 11:20:33 PM Arne Babenhauserheide wrote:
>> https://www.ncjtc.org/ICAC/Courses/trngres/Freenet%20Investigations%20White
>> %20Paper%20-Black%20Ice%20%20%28090413%29.pdf
> This link is dead nowadays.
>
> Archive.org has this though, including another PDF about Freenet, so make 
> sure 
> to search for "Freenet" twice here:
>
> https://web.archive.org/web/*/https://www.ncjtc.org/ICAC/Courses/trngres/*
If they take it down from archive.org there might be a public interest
argument for mirroring it regardless of nominal ownership. Make sure you
and all your journalist friends have a copy anyway. :) Hopefully they
wouldn't be that foolish since it'd blow up the press interest. :)



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Law-enforcement lying to courts about how Freenet works

[Matthew Toseland]

On 01/05/16 22:20, Arne Babenhauserheide wrote:
> Not quite a paper, just some calculations which show their math is
> wrong:
>
> http://127.0.0.1:/freenet:USK@sUm3oJISSEU4pl2Is9qa1eRoCLyz6r2LPkEqlXc3~oc,yBEbf-IJrcB8Pe~gAd53DEEHgbugUkFSHtzzLqnYlbs,AQACAAE/random_babcom/210/#Iamdarknetonlyagain
>
> There is now a detailed report how law enforcement tracks opennet
> downloaders (though the statistics are flawed pretty badly¹)
>
> I’m not allowed to upload the report here, so I can only give a clearnet link 
> to the white paper: 
> https://www.ncjtc.org/ICAC/Courses/trngres/Freenet%20Investigations%20White%20Paper%20-Black%20Ice%20%20%28090413%29.pdf
>
> ¹: the vulnerability to HTL18 they use has already been addressed in
> 2008, so any probability they claim using that is false. For every
> connection there is a 50% chance that all the requests (not only a
> single one) did not originate from the node from which we received them
> but were forwarded one step. So for 10 connection (the lowest value),
> there are 5 other nodes whose HTL18 requests are forwarded with HTL18,
> so the probability that a given HTL18 request originated at the node
> From which we received it is only about 17% (1 in 6). And this
> probability does not get better when gathering more requests of chunks
> From a specific file or a specific kind of files, because they can
> reasonably all be forwarded from a different node — the one which really
> sent them. The only way to get good statistics would be to connect to
> this node over and over again at different times when the peers of the
> node changed (that requires waiting at least 2 hours to change a
> significant number of peers — the only way to be sure would be to wait
> for the other node to go offline for more than 5 minutes and then to
> connect to it again). However screening out every node which ever sent a
> HTL17 or HTL16 request could improve the reliability a lot, though with
> significant cost. That doesn’t change that their probabilities are
> calculated incorrectly, but could give them a pretty good hit rate on
> people downloading a large volume of material.
>
> - Code: 
> https://github.com/freenet/fred/blob/next/src/freenet/node/PeerNode.java#L1603
> - Commit: 
> https://github.com/freenet/fred/commit/4aaa08f11656af1dd857e45612763c9bd2d89fc2

You can still do a classic correlation attack: Connect to the node for
the whole duration of the request and count the proportion of the file
they've fetched from you. There are probably others, but this is *the*
reason why we need darknet. And yes there's some plausible deniability
re leaf darknet peers, though there may be ways around that. For the
threat models I care about (China, whistleblowers etc), that's not a
serious problem; and leaf nodes suck, very few people use them. It may
not even be a problem for LEAs who play by the rules; they're good at
the find/squeeze-his-friends game. But it does take much more
(electronic!) resources than the "just search anyone who ever requests
an illegal block at HTL 18" attack described above.

Nonetheless, we should do something with this. The paper is clearly
wrong, and if they're using that argument in court, something is very
rotten, even if it's only their maths.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Mitigate the Pitch Black attack (the simulation works)

[Matthew Toseland]

On 22/04/16 21:31, Arne Babenhauserheide wrote:
> Matthew Toseland writes:
>
>> On 09/02/16 08:58, Arne Babenhauserheide wrote:
>>> Because in normal swapping, as soon as the network settled a bit, the
>>> changes in location should be small (though my nodestats look different:
>>> too large changes in location for my taste…). So the node data should
>>> still be reachable. When randomizing the position, however, the step is
>>> large and the node might settle into a new part of the keyspace, so the
>>> store might not be reachable anymore.
>> Not in all cases. E.g. merging several growing darknets? Is this slow
>> enough that we don't care?
>>
>> OTOH: Can this be used as some sort of DoS? Is 2 hops enough? Etc.
> Maybe this should simply be regular insertion, which however would limit
> the convergence speed.
>
>> This was likely discussed way back when darknet was first proposed,
>> maybe Oskar has an opinion about it ...
> It would be great to recover some of this, given that now more people
> seem willing to actually invest in building friend-to-friend connections
> (though I’m nut sure whether I just have that impression because it
> works for me now).
>
>>> We can’t keep nodes from leaving, but we can keep swapping which spans
>>> large parts of the keyspace from making parts of the datastore
>>> inaccessible.
>> On a hybrid network we still have the aristocracy problem: Because
>> opennet is meritocratic, fast nodes tend to connect to fast nodes. Hence
>> the distribution is likely to be non-uniform - slow nodes will be out on
>> the edge and have poor connectivity i.e. possibly a different mean
>> distance??
> This is not connected to losing store content due to darknet swapping, right?
No, but does it affect the Pitch Black fix? No, because hybrid nodes
don't swap?



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Possibly obsolete code in AnnounceSender and reordering problem during opennet announcements

[Matthew Toseland]

On 01/03/16 13:52, Martin Byrenheid wrote:
> On Wednesday, February 24, 2016 05:38:09 PM Matthew Toseland wrote:
>>>> The message should have been matched anyway: Unmatched messages (in
>>>> MessageCore are added to a data structure and we check before timing out
>>>> the waitFor()). Possibly a concurrency issue, I thought I'd debugged all
>>>> such problems ages ago. :(
>> It seems unlikely that a race condition like that would happen reliably,
>> so I don't know what is happening here.
> I looked at the log output again and found that the problem might be that an 
> unmatched message from a source that is not routable gets still marked as 
> matched.  In my case, it's the FNPBulkPacketSend-message that gets fed to the 
> NodeDispatcher before the actual noderef-receiver thread starts. And since it 
> gets marked as matched, it won't be passed to the receiver thread later on. 
> This behavior was introduced near the end of last year [1]. However, I'm not 
> sure if the problem lies in the fact that the message gets marked as matched 
> although it actually doesn't seem to be or if the source (in my case, it is a 
> SeedServerPeerNode) should normally be considered as routable. What do you 
> think? In my case, reverting the commit mentioned above solved the problem. 
Yes, we fixed this in the most recent build. Please confirm.
>
> Martin
>
> [1] 
> https://github.com/freenet/fred/commit/f3e3be45698272db462ae45f784ff4049c692e92



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Streaming audio over Freenet

[Matthew Toseland]

On 09/03/16 10:11, Arne Babenhauserheide wrote:
> Am Dienstag, 8. März 2016, 22:18:22 schrieb Bert Massop:
>> On 08-03-16 21:29, Arne Babenhauserheide wrote:
>>> So I see two ways forward:
>>>
>>>
>>> 1. Just live with the limitation of browsers and limit m3u to external
>>>applications.
>>>
>>> 2. Do content-filter-magic and conjure up some javascript during
>>>filtering which makes audio-tags with m3u work as if the browser
>>>support existed.
>>> What do you think? Should I go for the first or auto-enhance the
>>> audio-tag in freesites?
>> For now, I'd suggest to go for the first. Both HTML audio and basic
>> streaming would work, just not together.
> Since no one else wrote and this is the path which is much easier for me and 
> allows merging the feature more easily, I’ll chose the first then :)

I think we should go down the javascript route eventually, provided that
Freenet continues to work acceptably without it. Whether to use
javascript is already configurable, and Ian periodically starts a
flamewar by suggesting we rewrite fproxy in whatever the latest
Javascript-only framework is. However IMHO we should use Javascript
where it will significantly improve things, which is true of embedded
audio/video.

I believe there is some unmerged work that goes some way towards this,
some sort of framed video player thingy? Possibly from sajack?
>> I don't think we need more Javascript in core FProxy at the moment. If
>> auto-enhancing anything, please do so in a separate plugin.
> Plugins for filter-enhancements sound interesting.
>
>> It might be
>> an idea to remux the separate files (assuming they are of the same kind)
>> and present them to the browser as a single file. Note that for MP3,
>> this is a simple matter of concatenation.
> That would not allow sharing data in multiple playlists, so I think it’s much 
> weaker than actually using playlists. Future ideas for the m3u filter could 
> be to allow any Freenet key, so multiple sites could use the same files 
> without having to encode redirects in the manifest.

It would also be slow and unreliable, although we could probably avoid
fetching the whole playlist before starting. A javascript front-end is
preferable in a lot of ways. For example, for video, even if segments
are lost we can present the segments that are available.
> But it’s a possibility which Freesite authors could use right now, which 
> gives it a big advantage. So that’s OK for me for now.
>
> Best wishes,
> Arne
> --
> singing a part of the history of free software: 
>
> - http://infinite-hands.draketo.de



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] freenetproject.org traffic: Pretty good

[Matthew Toseland]

On 10/03/16 20:40, hyazin...@emailn.de wrote:
> Have a look at this: https://www.similarweb.com/website/freenetproject.org
> Looks pretty good. It's new that China is in the TOP 5 of visitors splitted 
> by origins...

How is that possible? We've been blocked for over a decade - both the
website and the protocol (at least 0.5). And the current Chinese
government is moving in the direction of more censorship rather than less.

Also how does this work, I thought we got rid of third party analytics?



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Possibly obsolete code in AnnounceSender and reordering problem during opennet announcements

[Matthew Toseland]

On Wed, Feb 24, 2016 at 03:39:05PM +0100, Martin Byrenheid wrote:
> On Tuesday, February 23, 2016 09:18:17 PM Martin Byrenheid wrote:
> > On Tuesday, February 23, 2016 06:45:46 PM Matthew Toseland wrote:
> > 
> > > Bandwidth limiting, message priorities, something of that nature. The
> > > point is it shouldn't actually complete the AnnounceSender and close the
> > > connection until existing transfers have completed. Which is why
> > > complete() waits for all transfers to finish. Unfortunately we don't
> > > call complete() in the case of an RNF! AFAICS the solution is to copy
> > > the wait loop from complete() into rnf(). Patches welcome. :)
> > 
> > Okay, I will try it out! Thanks!
> 
> I've changed the code but came across another similar problem :-/ 
> 
> The scenario again is that an Opennet node O sent an announce request to a 
> seed node S. S wants O as an opennet peer and therefore sends an announce 
> reply followed by three packets carrying its noderef. However, the first 
> packet of the noderef-transfer gets processed too early so that it gets fed 
> to the dispatcher and dropped as not routable. Although the rest of the 
> packets gets received correctly, the noderef-transfer fails. Excerpts from 
> the corresponding log file of the opennet node O:
> 
> O receives the announce reply and the first packet from the noderef-transfer:
> 
> Feb 24, 2016 14:02:47:295 (freenet.node.NewPacketFormat, UdpSocketHandler for 
> port 14834(1), MINOR): Decoded messages: 3
> Feb 24, 2016 14:02:47:295 (freenet.io.comm.Message, UdpSocketHandler for port 
> 14834(1), MINOR): Returning message: FNPBulkReceivedAll 
> {uid=5291246903919677492} from 
> freenet.node.SeedServerPeerNode@7dd5c52b@141.76.45.91:14833@9d09a2e706d48e25b685cc5413ebd3ee47638c20d62a0817a70d7c3323321e01@29b23591
> Feb 24, 2016 14:02:47:296 (freenet.io.comm.Message, UdpSocketHandler for port 
> 14834(1), MINOR): Returning message: FNPOpennetAnnounceReply 
> {uid=77823964067048
> 33260, noderefLength=503, paddedLength=3072, transferUID=8684549569896827466} 
> from 
> freenet.node.SeedServerPeerNode@7dd5c52b@141.76.45.91:14833@9d09a2e706d48e2
> 5b685cc5413ebd3ee47638c20d62a0817a70d7c3323321e01@29b23591
> Feb 24, 2016 14:02:47:296 (freenet.io.comm.Message, UdpSocketHandler for port 
> 14834(1), MINOR): Returning message: FNPBulkPacketSend 
> {uid=8684549569896827466,
>  data=Buffer {1024}, packetNo=0} from 
> freenet.node.SeedServerPeerNode@7dd5c52b@141.76.45.91:14833@9d09a2e706d48e25b685cc5413ebd3ee47638c20d62a0817a70d7c332332
> 1e01@29b23591
> 
> O starts to process the announce reply:
> 
> Feb 24, 2016 14:02:47:296 (freenet.io.comm.MessageCore, Announcer to 
> freenet.node.SeedServerPeerNode@7dd5c52b@141.76.45.91:14833@9d09a2e706d48e25b685cc5413ebd
> 3ee47638c20d62a0817a70d7c3323321e01@29b23591(7), MINOR): Returning 
> FNPOpennetAnnounceReply {uid=7782396406704833260, noderefLength=503, 
> paddedLength=3072, tra
> nsferUID=8684549569896827466} from 
> freenet.io.comm.MessageFilter@4d392d4e:FNPOpennetAnnounceCompleted
> 
> Shortly before O finished processing the reply, the first noderef-packet is 
> fed to the dispatcher and dropped:
> 
> Feb 24, 2016 14:02:47:296 (freenet.io.comm.MessageCore, UdpSocketHandler for 
> port 14834(1), MINOR): Feeding to dispatcher: FNPBulkPacketSend 
> {uid=8684549569896827466, data=Buffer {1024}, packetNo=0}
> Feb 24, 2016 14:02:47:296 (freenet.node.AnnounceSender, Announcer to 
> freenet.node.SeedServerPeerNode@7dd5c52b@141.76.45.91:14833@9d09a2e706d48e25b685cc5413ebd3ee47638c20d62a0817a70d7c3323321e01@29b23591(7),
>  DUD): second part got FNPOpennetAnnounceReply {uid=7782396406704833260, 
> noderefLength=503, paddedLength=3072, transferUID=8684549569896827466}Feb 24, 
> 2016 14:02:47:296 (freenet.node.NodeDispatcher, UdpSocketHandler for port 
> 14834(1), MINOR): Dispatching FNPBulkPacketSend {uid=8684549569896827466, 
> data=Buffer {1024}, packetNo=0} from 
> freenet.node.SeedServerPeerNode@7dd5c52b@141.76.45.91:14833@9d09a2e706d48e25b685cc5413ebd3ee47638c20d62a0817a70d7c3323321e01@29b23591
> Feb 24, 2016 14:02:47:296 (freenet.node.NodeDispatcher, UdpSocketHandler for 
> port 14834(1), DEBUG): Not routable
> 
> O then actually starts waiting for noderef packets:
> 
> Feb 24, 2016 14:02:47:296 (freenet.node.OpennetManager, (16), MINOR): 
> Receiving noderef (reply=false) as bulk transfer for request uid 
> 7782396406704833260 with transfer 8684549569896827466 from 
> freenet.node.SeedServerPeerNode@7dd5c52b@141.76.45.91:14833@9d09a2e706d48e25b685cc5413ebd3ee47638c20d62a0817a70d7c3323321e01@29b23591
> 
> ... receives the second and third packet:
> 
> Feb 24, 2016 14:02:49:659 (freenet.io.comm.MessageCore, (16), MINOR): 

Re: [freenet-dev] Possibly obsolete code in AnnounceSender and reordering problem during opennet announcements

[Matthew Toseland]

On 23/02/16 17:53, Martin Byrenheid wrote:
> Hello again,
>
> in the current Freenet-code, the realRun-method of the AnnounceSender class 
> contains a check whether opennet is disabled [1]. However, (if I understood 
> the code correctly and) if opennet is disabled, no Instance of the 
> AnnounceSender class will ever be created at this node (incoming announce 
> requests are already rejected in the handleAnnounceRequest-method of the 
> NodeDispatcher-class [2]), which renders the check mentioned above obsolete.
> Therefore it may  be removed for the sake of code simplicity. I know, it's 
> just 4 lines but I think it's worth mentioning. 

Hmmm. I thought we relayed opennet announcements through darknet nodes?
We probably do want to send them through hybrid nodes, so we can't just
say "never forward an announcement to a darknet peer"? Or can we? As
discussed previously I don't think the security issue is relevant one
way or the other, e.g. swapping and FOAF data give away a lot. So
currently we will send it to a darknet peer, but if it's a pure darknet
node it will reject it, whereas if it's a hybrid it may accept the
announcement?

Path folding, on the other hand, *is* relayed through darknet IIRC.

Anyway, the apparently redundant check is probably there to deal with
race conditions - whether opennet is enabled changes occasionally. IMHO
that is legitimate.
> While trying to run Freenet in a local testbed without connection to the 
> "real" Opennet, I encountered a problem which keeps nodes in opennet-mode 
> from 
> connecting to each other. More precisely, my testbed currently consists of 3
> nodes: a seed node S and two opennet nodes O1,O2. When O1 sends an 
> announce request (with HTL 18) to S while O2 is not connected to S, S accepts 
> and also wants O1 as a peer, thus S sends an AnnouncementReply together with 
> its noderef to O1. After S has sent its ref, it sends a FNPRouteNotFound to 
> O1 
> as no more peers are available. However, the packets received by O1 are 
> handled in a different order than they were sent by S, so that the 
> FNPRouteNotFound-message gets processed earlier than the last message of the 
> noderef-transfer, thereby closing the connection to S and causing the noderef-
> transfer to fail. I encountered an analogous case when O2 was also connected 
> to S, where a noderef-transmission from O2 to O1 via S was aborted because 
> the 
> FNPRouteNotFound-message from O2 was processed earlier than the last bulk 
> transfer packet. 
> Do you know what might lead to this reordering? Both S, O1 and O2 are running 
> on the same machine but with different ports, so I wouldn't expect the OS to 
> reorder packets. If you need more information (logfiles etc.), I would be 
> glad 
> to help.

Bandwidth limiting, message priorities, something of that nature. The
point is it shouldn't actually complete the AnnounceSender and close the
connection until existing transfers have completed. Which is why
complete() waits for all transfers to finish. Unfortunately we don't
call complete() in the case of an RNF! AFAICS the solution is to copy
the wait loop from complete() into rnf(). Patches welcome. :)

Thanks for looking into this stuff!
> Martin
>
>
> [1] 
> https://github.com/freenet/fred/blob/71a788164a896d6e5b6af0dfe0f35da5a6927633/src/freenet/node/AnnounceSender.java#L139
>
> [2] 
> https://github.com/freenet/fred/blob/71a788164a896d6e5b6af0dfe0f35da5a6927633/src/freenet/node/NodeDispatcher.java#L648
>



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Request for improvement node behavior because of ISP NAT translation timeout now is less than 5 seconds

[Matthew Toseland]

On 20/02/16 12:31, vastik_spbm wrote:
>> On 19/02/16 14:53, vastik.spbm@yahoo wrote:
>>> Dear all,
>>>
>>> I will explain my problem.
>>> My ISP decided to fight with torrents DHT and changed udp nat
>>> translation timeout to 5 seconds.
>>> What does it mean. It means that if udp packet won't return from
>>> destination within 5 seconds NAT wont create a new rule and wont allow
>>> any other packets goes back to source. To create nat rule the first
>>> packet has to return with in 5 seconds, and it doesn't matter if other
>>> packet will follow the first one, the rest can return much later, but
>>> first one has to return within 5 seconds.
>>>
>>> Right now my client cant connect to any nodes at all. This is initial
>>> connection, when I start my client.
>>> And this can be done by any of ISPs, it is not very difficult and ISP
>>> doesn't brake anything, because UDP exists and enabled.
>>> Freenet wont work this way.
>>>
>>>
>>> Therefore, please:
>>> You need to improve node behavior - it has to replay nearly
>>> immediately after a node receives an attempt to initiate a connection
>>> from a client.
>>> It doesn't mean that node will keep it up, node can reject this
>>> connection late, or just ignore any other packets from client, but not
>>> the first one.
>>>
>>> Sorry, English is not my native language to me, so if you need more
>>> details or log file from tcpdump, just ask.
>>> I use freenet a lot and would like to continue to use it.
>>>
>> I believe Freenet does reply more or less immediately to a connection
>> request. Did your node manage to connect to any seednodes? Have you
>> tried connecting to darknet nodes outside the problematic network?
>>
>> It might be losing connections to seednodes once it's established them.
>> Currently on an idle connection we send keepalive packets only every
>> 7-14 seconds (see the constant KEEPALIVE_INTERVAL in Node.java, you
>> could just reduce this to say 2 seconds). Another possibility for
>> darknet would be to ensure there is always traffic e.g. by sending files
>> using the node-to-node messaging.
>>
>> However, there are two further problems:
>>
>> 1. Some NATs will break Freenet no matter what we do. E.g. if they use a
>> different port number for each connection, UDP hole punching usually
>> won't work, at least not if the other side is also NATed.
>>
>> 2. It's actually fairly straightforward to block Freenet at present. The
>> trivial attack is to block large UDP packets, but there are other
>> options. In the long run with steganographic transports this would be
>> harder, but still easy with traffic flow analysis (depending on how much
>> of a problem false positives are i.e. blocking other traffic by
>> accident).
>>
> Hello,
>
> I would like to apologize for the noise.
>
> I found access to a different ISP and check response time, it is less
> then 3 seconds.
> If necessary I can provide log, but I do not think.
>
> It looks like my ISP did some nasty stuff to block some udp (sip
> actually works).
> I am not sure that ISP uses different port number, because sip stun
> tells that nat is symmetric.

IIRC symmetric means exactly the problem above: Different port number
for each pair of IPs, can only connect to nodes that aren't NATed. :(

> Yes my node can ONLY connect to seednodes:
>
> 15:18:51.567601 IP 192.168.1.2.35394 > 100.xx.xx.xxx.14416: UDP,
> length 272
> 15:18:51.750478 IP 192.168.1.2.35394 > 84.xx.xx.xxx.1131: UDP, length 231
> 15:18:53.617665 IP 192.168.1.2.35394 > 149.xxx.xxx.xxx.61078: UDP,
> length 285< this is request to seednodes, packet size
> correspond with others
> 15:18:53.662938 IP 149.xxx.xxx.xxx.61078 > 192.168.1.2.35394: UDP,
> length 325<--- replay, almost immediately
>
> As long as I cant check the remote side I think I just say thank you
> for your help.
>
>
> Best regards
> Vasilii Tikhonov



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Request for improvement node behavior because of ISP NAT translation timeout now is less than 5 seconds

[Matthew Toseland]

On 19/02/16 14:53, vastik.spbm@yahoo wrote:
> Dear all,
>
> I will explain my problem.
> My ISP decided to fight with torrents DHT and changed udp nat
> translation timeout to 5 seconds.
> What does it mean. It means that if udp packet won't return from
> destination within 5 seconds NAT wont create a new rule and wont allow
> any other packets goes back to source. To create nat rule the first
> packet has to return with in 5 seconds, and it doesn't matter if other
> packet will follow the first one, the rest can return much later, but
> first one has to return within 5 seconds.
>
> Right now my client cant connect to any nodes at all. This is initial
> connection, when I start my client.
> And this can be done by any of ISPs, it is not very difficult and ISP
> doesn't brake anything, because UDP exists and enabled.
> Freenet wont work this way.
>
>
> Therefore, please:
> You need to improve node behavior - it has to replay nearly
> immediately after a node receives an attempt to initiate a connection
> from a client.
> It doesn't mean that node will keep it up, node can reject this
> connection late, or just ignore any other packets from client, but not
> the first one.
>
> Sorry, English is not my native language to me, so if you need more
> details or log file from tcpdump, just ask.
> I use freenet a lot and would like to continue to use it.
>
I believe Freenet does reply more or less immediately to a connection
request. Did your node manage to connect to any seednodes? Have you
tried connecting to darknet nodes outside the problematic network?

It might be losing connections to seednodes once it's established them.
Currently on an idle connection we send keepalive packets only every
7-14 seconds (see the constant KEEPALIVE_INTERVAL in Node.java, you
could just reduce this to say 2 seconds). Another possibility for
darknet would be to ensure there is always traffic e.g. by sending files
using the node-to-node messaging.

However, there are two further problems:

1. Some NATs will break Freenet no matter what we do. E.g. if they use a
different port number for each connection, UDP hole punching usually
won't work, at least not if the other side is also NATed.

2. It's actually fairly straightforward to block Freenet at present. The
trivial attack is to block large UDP packets, but there are other
options. In the long run with steganographic transports this would be
harder, but still easy with traffic flow analysis (depending on how much
of a problem false positives are i.e. blocking other traffic by accident).



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Problem with handling of Opennet Announce Requests

[Matthew Toseland]

On 18/02/16 13:43, Steve Dougherty wrote:
> On Thu, Feb 18, 2016, 6:30 AM Martin Byrenheid <
> martin.byrenh...@tu-dresden.de> wrote:
>
>> Hi,
>>
>> while working with Freenet, I discovered that whenever a seed node
>> received an
>> OpennetAnnounceRequest-message for a target location X, it forwards the
>> request to another opennet peer node, but always for target location 0.0
>> instead.
>
> Yikes. That sure sounds like a bug.
>
> This behavior results from the fact that the constructor of the
>> AnnounceSender class (line 55 [1])  does not copy the given target location
>> into the "target" member variable. Is this an implementation bug or is
>> there a
>> good reason why the original target location should be ignored?
>>
> Matthew likely wrote it, but he's busy at university, so instead of asking
> him my impulse would be to check if that behavior is in the initial
> AnnounceSender implementation. If it was intentionally removed later there
> should be reasoning in the commit message. If - as I'd expect - it's a bug,
> it very well may exist in the initial implementation or be introduced by a
> refactor.

Argh. Yup:

https://github.com/freenet/fred/commit/5d21c855655c1f974a8e9333d74c1d564224bf4c

Please send a patch, and make target final while you're at it. Thanks!



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Swap requests over opennet a bad thing? was Fwd: Re: Swap requests in Freenet

[Matthew Toseland]

See also
https://bugs.freenetproject.org/view.php?id=6820


 Forwarded Message 
Subject:Re: Swap requests in Freenet
Date:   Tue, 9 Feb 2016 09:46:52 +0100
From:   Stefanie Roos <stefanie.r...@tu-dresden.de>
To: Matthew Toseland <mj...@cam.ac.uk>, Martin Byrenheid
<martin.byrenh...@tu-dresden.de>, Arne Babenhauserheide <arne_...@web.de>



Hi Matthew,

just to (maybe) clarify:
Our measurements raised the question if it makes sense to forward
SwapRequests into the Opennet.
The motivation seems to be that different Darknet "Pockets" can exchange
locations but we never saw
that happen, so it might be that forwarding a SwapRequest into the
Opennet just i) generates unnecessary traffic
as all requests are rejected (should not be significant as the payload
of a swap request is comparably low) and
ii) allows an Opennet node to attack the Darknet by answering a
SwapRequest with fake locations to either perform the PitchBlack Attack
(the commitments provide some protection, you are right, but my guess
would be that the attack remains possible just more costly) or simply
infer information about the structure of the Darknet (which might or
might not be a problem, I was thinking in the direction of graph
(de-)anonymization [1] to identify participants of the Darknet but the
available information might be insufficient for that to be a real issue,
hard to tell).
So, only forwarding swap requests to Darknet peers might slightly
increase the protection under the assumption that joining a Darknet is
somewhat hard for an attacker, especially since forwarding to Opennet
peers does seemingly not produce any noticeable advantage.
Anyways, I think Arne is currently working on the swapping algorithm, so
I just add him as CC.

Thanks for the fast reply,

Stef


[1] Narayanan, Arvind, and Vitaly Shmatikov. "De-anonymizing social
networks." /Security and Privacy, 2009 30th IEEE Symposium on/. IEEE, 2009.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Mitigate the Pitch Black attack (the simulation works)

[Matthew Toseland]

On 09/02/16 08:58, Arne Babenhauserheide wrote:
> Matthew Toseland writes:
>
>> Awesome! We need to:
>> 1) Implement this.
> Yepp.
>
>> 2) Publish it.
> We should… Stefanie and Michael might be able to gain something from
> that. Maybe you, too?
>
>> Why is reinserting the whole datastore important in the case of an attack?
> Because in normal swapping, as soon as the network settled a bit, the
> changes in location should be small (though my nodestats look different:
> too large changes in location for my taste…). So the node data should
> still be reachable. When randomizing the position, however, the step is
> large and the node might settle into a new part of the keyspace, so the
> store might not be reachable anymore.

Not in all cases. E.g. merging several growing darknets? Is this slow
enough that we don't care?

OTOH: Can this be used as some sort of DoS? Is 2 hops enough? Etc.

This was likely discussed way back when darknet was first proposed,
maybe Oskar has an opinion about it ...
>> AFAICS we are looking for a gap much larger than the node's local
>> average peer distance? In practice this is likely to vary a lot because
>> of different node degrees etc?
> Yes. Node degree is the core variable for that. Thanks to FOAF
> information we should be able to use an average degree of all friends
> for the calculation.
>
>> On opennet, performance has a big impact;
>> on darknet, a node's location on the graph and its peer count? I'm
>> thinking of the problems we had around the time Pitch Black was
>> published - at least some of it was due to nodes with few peers taking
>> locations in big gaps and then leaving the network.
> We can’t keep nodes from leaving, but we can keep swapping which spans
> large parts of the keyspace from making parts of the datastore
> inaccessible.

On a hybrid network we still have the aristocracy problem: Because
opennet is meritocratic, fast nodes tend to connect to fast nodes. Hence
the distribution is likely to be non-uniform - slow nodes will be out on
the edge and have poor connectivity i.e. possibly a different mean
distance??

Does this affect this?
>
> Best wishes,
> Arne



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Mitigate the Pitch Black attack (the simulation works)

[Matthew Toseland]

On 09/02/16 14:12, Michael Grube wrote:
> All,
>
> Please excuse my absence. I have been very preoccupied. I am OK with the
> idea that we should publish and am willing to spend time assisting with
> that effort.
>
> However I should disclose that I am doing research on creating a method
> very similar to Snadberg's original work, except that the network would sit
> on top of a fractal lattice.
>
> I am investigating methods to exploit the assumption of a fractal network
> to detect swap proposals that are inappropriate or very out of place.
>
> The work is shoddy and incomplete at the moment but all of my free time is
> going into it. I will make people aware of results as I get them.

That sounds great. I do think we should try to publish what we have
already though, it may have ramifications beyond Freenet itself as the
authors pointed out. Thanks for all your work making this happen!
> Thanks



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Mitigate the Pitch Black attack (the simulation works)

[Matthew Toseland]

Awesome! We need to:
1) Implement this.
2) Publish it.

Why is reinserting the whole datastore important in the case of an attack?

AFAICS we are looking for a gap much larger than the node's local
average peer distance? In practice this is likely to vary a lot because
of different node degrees etc? On opennet, performance has a big impact;
on darknet, a node's location on the graph and its peer count? I'm
thinking of the problems we had around the time Pitch Black was
published - at least some of it was due to nodes with few peers taking
locations in big gaps and then leaving the network.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] What blocks Freenet adoption?

[Matthew Toseland]

On 03/02/16 22:28, Arne Babenhauserheide wrote:
> Matthew Toseland writes:
>
>> A proper documented
>> plugin API will help - and there has been some work on documentation.
>> Getting WoT working well will help, and better deployment of our
>> existing tools e.g. Sone...
> Not to forget tutorials which are easy to follow.
>
> To cut it short: This is what we need to make it easy to join:
>
> http://stevelosh.com/blog/2013/09/teach-dont-tell/

Right. I believe we already have a plugin writing tutorial?
>
> Best wishes,
> Arne



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Similarity between original SSK proposal and Bitcoin contracts

[Matthew Toseland]

On 04/02/16 20:32, Ian Clarke wrote:
> I've been reading about Bitcoin Contracts
> , and I'm surprised by the
> similarity between these and, not just SSKs, but particularly the original
> proposal for SSKs  from way back
> in June 2000, which involved a stack based language with cryptographic
> primitives, just like the language used for Bitcoin contracts.
>
> I don't know if the Bitcoin approach was inspired by SSKs at all, I suspect
> more likely independent reinvention to solve a similar problem.

Perhaps. It's relevant to the PSKs discussion.

In practice what you can do with Bitcoin script is severely restricted
by the miners... but there is no obvious reason for this that would also
apply to Freenet key verification.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] What blocks Freenet adoption?

[Matthew Toseland]

On 03/02/16 10:36, Travis Wellman wrote:
>
> On Mon, 2016-01-04 at 02:26 +0100, Arne Babenhauserheide wrote:
>>  
>> What blocks Freenet adoption?
> I mention Freenet in conversations often.
>
> I think #1 is that it's slow. Slow means that people who don't have to
> use it won't. Our users who are relatively technically savvy have less
> time/patience than ever.
Agreed, performance matters.
> Developers, developers, developers, developers. It could be easier to
> create applications backed by Freenet. Something akin to an app store
> would unlock huge potential growth IMO. A UI that helps new users
> navigate cool things that they can do with Freenet once it's installed.

Much harder. Unfortunately Freenet makes it exceptionally difficult to
sandbox untrusted code. Plus you have to rewrite everything anyway
because it's fully distributed - it's a completely different model to
just about everything else. So that means it's down to plugins and apps.
We have quite a few of them, but we need more. A proper documented
plugin API will help - and there has been some work on documentation.
Getting WoT working well will help, and better deployment of our
existing tools e.g. Sone...



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] PSK keys

[Matthew Toseland]

On 02/02/16 17:24, x...@freenetproject.org wrote:
> Hi,
>
> I'm the maintainer of Web of Trust [2] and Freetalk [3], which are the first 
> systems which will likely be deployed as installed by default for purposes 
> subject to what you proposed with PSKs.

As I understand it, Sadao is interested primarily in writing his own
incompatible forums system with centralised moderation (i.e. the ability
to remove users), at least as a first step?
>
> While I admittedly didn't take the time to fully understand all details PSKs, 
> I think I can nevertheless speculate the following about their general 
> concept:
>
> There are a lot of systems based on WoT already: Freetalk, Freemail, Sone, 
> FlogHelper, WoTNS, Infocalypse, and the ones I forgot about at [1].
> They share a common concept: If content is downloaded, it is downloaded from 
> the very same person responsible for it. This is to allow WoT to fulfill its 
> purpose of spam filtering by saying "person X is not trustworthy anymore" and 
> causing the content to be deleted then. I.e. we know who is to blame for 
> spam, 
> so we can just delete his stuff.

We still know who posted something with PSKs. That's why we need a new
key type: You can post if your key is signed by the owner's key (at
least in Sadao's original proposal).
> With PSKs, that principle wouldn't apply anymore, there would be many people 
> who have access to one keyspace, there wouldn't just one person be to blame 
> for spam.
> This is a very disruptive change to architecture and thus would likely not 
> work with the existing foundation of all the existing WoT systems and cause 
> their 'core' to have to be rewritten.
> So what I'm trying to say is: This would be *years* of work.

I agree it would be a lot of work to adapt WoT to use PSKs. Mainly
because of the complexity of creating groups of identities, detecting
misuse of owner powers etc, all automatically.
> And we've already spent years on WoT-based systems and they still aren't 
> finished to the point where we can ship them as default with the installer.
> So I am preferring to finish WoT and its client applications with their 
> architecture as is, because the users want to finally see results.
> Rewriting everything for yet another half a decade without having any 
> officially-installed-by-default forum system is not an option :(
> And especially: The bugtracker contains sufficient ideas to make WoT and the 
> apps based on it scalable, so there doesn't seem to be a need to pull out the 
> optimization which PSKs are just yet. It is a lot more complex than the known 
> optimizations, since it requires throwing everything we have away. The known 
> ones would just improve upon systems as they are, they don't require as much 
> work.

I don't see why it would require "throwing everything away". In any
case, Sadao is perfectly entitled to build his own forums system. But as
I've explained I don't have time to implement PSKs at the moment; maybe
he will be able to. Of course he could work on something else, and that
might be more productive, but if he wants to implement PSKs, that's a
good thing.
> Sorry.
>
> I'm nevertheless glad for your idea of PSKs,and it can for sure be an option 
> some day. Just please not now.
> I'd rather wait for this until:
> 1) WoT and a few apps based on it are finished and deployed.
> 2) it turnes out in benchmarks that the simple optimizations did not help 
> enough.
>
> Meanwhile, if you do want to contribute some code to Freenet, notice this:
> Nowadays, at [1], we've gathered a *huge* list of all known subprojects. This 
> should be a good tool to find something to work on! :)
> Once you've found something, it would be nice if you could join us on 
> Freenode 
> IRC at #freenet. Most developer communication happens there.
>
>
> Greetings and thanks for your returning! :)
>
>
> [1] https://wiki.freenetproject.org/Projects
> [2] https://wiki.freenetproject.org/Web_of_Trust
> [3] https://wiki.freenetproject.org/Freetalk



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] PSK keys

[Matthew Toseland]

On 31/01/16 04:06, Sadao wrote:
> Hi all.
>
>
> Three years ago I started a thread on FMS with the topic "Efficiency of 
> various freenet message systems", where I proposed to implement a new key 
> type (PSK) in order to make a base for creating spam-protected moderated 
> messaging systems in freenet like usenet groups. Toad liked the idea and he 
> was going to implement PSK keys in freenet while I was going to write a new 
> client app (a Frost-like message system with moderation). But eventually Toad 
> switched his attention to WoT and other things and I completely lost interest 
> to freenet and left it.
>
>
> Now I returned again just to see that there is no progress. I still have 
> spare time and willingness to write a new app, but there is no support of PSK 
> keys. In theory, I could try to implement them myself, but it would take me 
> years for that. On the other hand, it’s not so difficult for a person like 
> Toad who knows freenet code very well. So I’d like to ask again: is there any 
> chance that Toad could add the support of PSK keys to freenet in the near 
> future?

No. It'd be a fairly big project, and I'm busy until at least June.
After which time I may volunteer a little for Freenet in between work;
we no longer have any paid staff.

IMHO it would be best to resolve the existing issues with keys first.
SSKs are based on 1024-bit DSA, which is severely outdated and likely
factorable by at least NSA, but it also has humongous keys compared to
modern ECC-based asymmetric crypto.

So the first step would be to implement modern ECC-based keys. This
would IMHO include:
- Merging the pubkey store and the SSK store into a single ~2KB/slot
datastore.
-- This should happen automatically. It should be well-tested and not
cause a wrapper timeout for big stores (there are methods on
WrapperManager that can help with it). I think it is reasonable to
require that there be enough disk space to do a straightforward copy.
- ECC-based SSKs.
- Different sizes:
-- ~ 800 bytes (an insert with full metadata etc fits in a single
packet, great for FLIP)
-- ~ 2KB (better for most purposes)
-- 32KB (put it in the main datastore, so lifetime is limited, but
carries more data; ideal for FMS)
- Request level changes to support the above:
-- We should always send the pubkey, rather than asking whether the node
has it.
-- Small ECC-SSKs an interesting low-latency special case.
-- Big ECC-SSKs another interesting special case which will need more code.

This is all documented in a reasonable degree of detail on the bug
tracker. Different sized ECC-SSKs is a bonus really; but please try not
to make it impossible, I suggest the basic ECC-SSKs should have a 2KB
payload. You might want to think about how you're going to trade off
space for PSK metadata as well...

Of course, now I'm volunteering you for lots more work than you had at
first anticipated. However, that's more or less what you were doing. :)

I do think PSKs could be useful for a fairly wide range of tools on
Freenet, including moderated forums (as you suggested), optimising
WoT-like forums (I haven't actually done much work on WoT),
collaboratively maintained search indexes etc. IMHO they should support
arbitrary verification operations, with some built in crypto tools, but
with the usual block size limits and severe limits on CPU time for a
verification; we should not be afraid of Turing completeness, provided
we can bound the runtime.

IMHO the stuff about PSKs was actually rather disruptive. I felt I was
being pulled in lots of different directions at once trying to keep up
with volunteers. I guess that's not intended as a criticism, merely
tactical advice: If a volunteer offers to do something after you
implement a huge feature, which will take months, and you have lots of
other more urgent stuff, explain why you can't do it and suggest that
they get started on it instead...

Please don't take this as hostile. I will try to help you if you have
specific questions. But I don't have time to do substantial work on
Freenet right now apart from my project (which is related to
simulations). I am also occasionally helping out Steve with updater
issues...



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Separation of freenet-ext component parts

[Matthew Toseland]

On 10/01/16 16:31, Florent Daigniere wrote:
> On Sun, 2016-01-10 at 16:13 +0000, Matthew Toseland wrote:
>> On 10/01/16 15:33, Ben Green wrote:
>>> Hello freenet-dev,
>>>
>>> In a recent comment on a pull request:
>>>
>>> https://github.com/freenet/fred/pull/29
>>>
>>> Steve said:
>>>
>>> 1471 isn't going to split freenet-ext either. Hrm.
>>>
>>> I would like to help with this as I have recently had the dubious
>>> pleasure of deciphering the build scripts from
>>> contrib/NativeBigInteger. I had initially wondered if it is still
>>> worth it as it has been some time since the repository was updated,
>>> I found that with the version of GMP that is "supposed" to be used,
>>> 5.0.1 (by that I mean the one referenced in the build scripts in
>>> contrib on github) the speed is:
>>>
>>> native run time: 66ms (0ms each)
>>> java run time: 329ms (3ms each)
>>> native = 20.060790273556233% of pure java time
>>> native run time: 60ms (0ms each)
>>> java run time: 310ms (3ms each)
>>> native = 19.35483870967742% of pure java time
>>>
>>> Running the same thing with the most recent GMP, 6.1.0:
>>>
>>> native run time: 47ms (0ms each)
>>> java run time: 315ms (3ms each)
>>> native = 14.920634920634921% of pure java time
>>> native run time: 49ms (0ms each)
>>> java run time: 315ms (3ms each)
>>> native = 15.555% of pure java time
>>>
>>> I can only guess what version is used in the official release but
>>> from the dates on the i2p 
>>> repository and the dates in freenet-ext.jar (09-05-2011) and
>>> TommyD's Gentoo ebuild I am guessing 4.1.4 - 4.2.2, in any case the
>>> runtime is:
>>>
>>> native run time: 101ms (1ms each)
>>> java run time: 375ms (3ms each)
>>> native = 26.934% of pure java time
>>> native run time: 89ms (0ms each)
>>> java run time: 316ms (3ms each)
>>> native = 28.164556962025316% of pure java time
>>>
>>> These are the tests included in the i2p NativeBigInteger.class file
>>> and I them several times, they seem quite consistent. These tests
>>> were performed on my laptop, of 2012 vintage: Intel(R) Core(TM) i7-
>>> 2720QM CPU @ 2.20GHz. If you happen to be running Gentoo, and let's
>>> face it why wouldn't you, TommyD's ebuild will create a library
>>> that uses whichever gmp is installed on your local machine so that
>>> is good :-). From the above, less than rigorous results, it seems
>>> that it is still worth using NativeBigInteger, at least on my
>>> machine.
>>>
>>> I also notice that we have two NativeBigInteger.class files, one in
>>> freenet-ext.jar and one in freenet.jar, they are different and if
>>> you run freenet on a Raspberry Pi 2 with freenet-ext.jar first in
>>> your classpath libjbigi-linux-arm.so is not loaded even if it is
>>> present, I can only assume that the source to that version does not
>>> support arm.
>>>
>>> I realise this is probably not news to many and so before I go off
>>> and begin re-writing the build scripts I thought I would see if
>>> anyone else is currently engaged in re-packaging freenet-ext.jar. I
>>> was thinking to use Autotools, I know that Gradle is getting a lot
>>> of attention but as jbigi and jcpuid are native components I
>>> thought it may be better to keep with Autotools (naturally, I would
>>> create and test scripts that would work in MinGW/Cygwin too but
>>> might need a little help from anyone running FreeBSD or OS X).
>>>
>>> Kind regards,
>>>
>>> Benjamin.
>> My 4p:
>>
>> 1. Splitting jars out of freenet-ext.jar is a good idea, and
>> reasonably
>> easy to deploy (just add more jars to dependencies.properties).
>>
> It requires every single installer to be updated... as well as the
> run/update scripts for each platform.

Good point, although we can safely add the new jars to the auto-update
before dealing with the rest (if they're the same or compatible versions).
>> 2. It probably requires solving the problems with Maven/Gradle (for
>> Contrib), since the current Contrib uses Maven for some components.
>> However for many components we could ship the official pre-built
>> jars,
>> especially if they are signed.
>>
> Odds are we can get rid of db4o and Maven... I think that we want
> Gradle to be used as much as possible for 

Re: [freenet-dev] Separation of freenet-ext component parts

[Matthew Toseland]

On 10/01/16 15:33, Ben Green wrote:
> Hello freenet-dev,
>
> In a recent comment on a pull request:
>
> https://github.com/freenet/fred/pull/29
>
> Steve said:
>
> 1471 isn't going to split freenet-ext either. Hrm.
>
> I would like to help with this as I have recently had the dubious pleasure of 
> deciphering the build scripts from contrib/NativeBigInteger. I had initially 
> wondered if it is still worth it as it has been some time since the 
> repository was updated, I found that with the version of GMP that is 
> "supposed" to be used, 5.0.1 (by that I mean the one referenced in the build 
> scripts in contrib on github) the speed is:
>
> native run time: 66ms (0ms each)
> java run time: 329ms (3ms each)
> native = 20.060790273556233% of pure java time
> native run time: 60ms (0ms each)
> java run time: 310ms (3ms each)
> native = 19.35483870967742% of pure java time
>
> Running the same thing with the most recent GMP, 6.1.0:
>
> native run time: 47ms (0ms each)
> java run time: 315ms (3ms each)
> native = 14.920634920634921% of pure java time
> native run time: 49ms (0ms each)
> java run time: 315ms (3ms each)
> native = 15.555% of pure java time
>
> I can only guess what version is used in the official release but from the 
> dates on the i2p 
> repository and the dates in freenet-ext.jar (09-05-2011) and TommyD's Gentoo 
> ebuild I am guessing 4.1.4 - 4.2.2, in any case the runtime is:
>
> native run time: 101ms (1ms each)
> java run time: 375ms (3ms each)
> native = 26.934% of pure java time
> native run time: 89ms (0ms each)
> java run time: 316ms (3ms each)
> native = 28.164556962025316% of pure java time
>
> These are the tests included in the i2p NativeBigInteger.class file and I 
> them several times, they seem quite consistent. These tests were performed on 
> my laptop, of 2012 vintage: Intel(R) Core(TM) i7-2720QM CPU @ 2.20GHz. If you 
> happen to be running Gentoo, and let's face it why wouldn't you, TommyD's 
> ebuild will create a library that uses whichever gmp is installed on your 
> local machine so that is good :-). From the above, less than rigorous 
> results, it seems that it is still worth using NativeBigInteger, at least on 
> my machine.
>
> I also notice that we have two NativeBigInteger.class files, one in 
> freenet-ext.jar and one in freenet.jar, they are different and if you run 
> freenet on a Raspberry Pi 2 with freenet-ext.jar first in your classpath 
> libjbigi-linux-arm.so is not loaded even if it is present, I can only assume 
> that the source to that version does not support arm.
>
> I realise this is probably not news to many and so before I go off and begin 
> re-writing the build scripts I thought I would see if anyone else is 
> currently engaged in re-packaging freenet-ext.jar. I was thinking to use 
> Autotools, I know that Gradle is getting a lot of attention but as jbigi and 
> jcpuid are native components I thought it may be better to keep with 
> Autotools (naturally, I would create and test scripts that would work in 
> MinGW/Cygwin too but might need a little help from anyone running FreeBSD or 
> OS X).
>
> Kind regards,
>
> Benjamin.

My 4p:

1. Splitting jars out of freenet-ext.jar is a good idea, and reasonably
easy to deploy (just add more jars to dependencies.properties).

2. It probably requires solving the problems with Maven/Gradle (for
Contrib), since the current Contrib uses Maven for some components.
However for many components we could ship the official pre-built jars,
especially if they are signed.

3. NativeBigInteger is only relevant for DSA. At the moment we only use
this for SSKs; in future hopefully we will have ECC-based SSKs, but we
will need to support old SSKs for some time. So it may still be worth
having native code for it.

4. The native FEC code is currently disabled (in fred), because it has
exploitable weaknesses. This is pending review by somebody really good
at C and JNI. Arguably it's not worth the risk and we should get rid of
it. Having said that IIRC there was a ~ 4x improvement in CPU usage,
although decoding is often disk-limited for bigger files... It would be
worth re-running the benchmarks for FEC.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] What blocks Freenet adoption?

[Matthew Toseland]

On 05/01/16 16:32, x...@freenetproject.org wrote:
> Overall comment about your mail: I think such threads are *asking* to cause a 
> flamewar. Pick lots of random, unrelated stuff, and complain about its state 
> - 
> the "lots of" guarantees that you'll make as many developers feel affected 
> negatively as possible. So you'll have lots of stressed people arguing, which 
> is the recipe for a flamewar.
>
> I do realize that you do this out of a positive intent, but IMHO it is the 
> same pattern as all the other flamewars we've had some weeks ago, and they 
> were highly destructive and demotivating.
>
> Do you realize that we've had 30 pull requests during the past week?
> We don't need big motivational mails, people are churning out insane amounts 
> of code :)

This is a really important point. Freenet is actually doing pretty well
right now. Of course there's a long way to go.

As for the rest, the latest changes to WoT haven't even been deployed
yet, and should make a big difference. We should deploy what we have
before arguing for getting rid of large chunks of working code in favour
of code that hasn't been maintained and doesn't support modern APIs.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Issues in BaseL10n and in BaseL10nTest in the current code

[Matthew Toseland]

On 02/01/16 16:24, Rostislav Krasny wrote:
>> Hi Rostislav,
>>
>> Thanks for the patch! A few thoughts:
>>
>> - this mixes formatting, style, and functional changes, which should be
>>  avoided.
> The formatting and style changes are minimal. I just don't like mixing
> several formatting styles in the same unit of code. And if you look at
> my changes of the the BaseL10n class you could see that I've changed
> only one method - addHTMLSubstitutions(). This change is quite small,
> even if you take formatting into account. Also instead of looking at
> the patch file you can use a comparison software that usually shows
> the changes by a more handy way and also may have several modes,
> including modes that ignore whitespaces and other unimportant
> differences. But If you want I can explain each line of my changes in
> the addHTMLSubstitutions(). method.
>
> Other java class that was changed is BaseL10nTest and there is no
> formatting changes. All changes are "painfully obvious" and this is a
> test class.
>
> All other changes relate to properties files that were just renamed
> without any content change.
>
>> - the commit message should enumerate the behavior changes and explain
>>  why they were made if it's not painfully obvious.
> The commit message was written for myself. I just used 'git
> format-patch' instead of 'git diff' to make the patch. Since I use
> Maven and all code was rearranged according to the Maven directories
> structure my patch probably can't be applied to the original Fred
> right away. At least that wasn't my intention. I just wanted to show
> what I've changed in the code extracted from the original Fred. And
> again, I can explain each code line I've changed.
>
>> - after 1471 is released we're planning to move to Gradle (and use
>>  gradle-witness), but the directory structure changes will be
>>  equivalent. At that point we will certainly merge something similar
>>  to your changes (if not your changes outright) to restore sanity to
>>  the BaseL10nTest resource path stuff.
> Why not to do so now? At least in the "next" branch that is what you
> use instead of the "master" one.
>
>> We usually do code review on GitHub; could you please open this as a
>> pull request? [0]
>>
>> - Steve
>>
>> [0] https://github.com/freenet/fred/pulls
> I never did github pull requests before. Do I need a github account?
Yes.
> Do I need to make another patch that is compatible to the Ant
> directories structure used in Fred? 
Yes.

> Why the bug report with my
> original patch isn't enough?
Because the release manager is an overworked volunteer.

Please stick to the project standards to make life easier for volunteers
dealing with your contributions. Thanks.
https://wiki.freenetproject.org/Coding_standards



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] New UPnP2 plugin

[Matthew Toseland]

On 26/12/15 16:03, Ian wrote:
> Best practice is to use the logging system, using System.out.println() for
> logging is almost universally regarded as bad practice, many static
> analysis tools will automatically flag it as a problem.
>
> The example you gave in NodeCrypto is not best practice, that should
> probably just be something like:
>
> Logger.error(this, "Could not use port: "+bindto+ ':'+portNo+": "+e, e)
>
> Freenet has it's own logging system (see here
> ),
> possibly because Freenet predates a lot of the logging frameworks, however
> we should probably be using one of the numerous logging libraries like
> Logback, as they are more flexible.
>
> Ian.

In this case it is appropriate, because the above is only shown if the
node is about to fail to start up (at least in the darknet case). See my
message explaining the current policy. Changing the policy may be
appropriate, I'm just documenting the current status quo.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Logging policy

[Matthew Toseland]

On 26/12/15 13:05, Xiaoyu Huang wrote:
> System.out.println() is for easier debugging as I can see the results
> directly in wrapper.log.
>
> I see many examples in Fred code that uses both Logger and println(). An
> example in NodeCrypto.java line around 136:
>
> Logger.normal(this, "Could not use port: "+bindto+ ':'
> +portNo+": "+e, e);
> System.err.println("Could not use port: "+bindto+ ':'
> +portNo+": "+e);
>
> I'm not quite sure what's the best practice.
>
> -- Xiaoyu
Only critical stuff should be logged to stderr. E.g. if it's likely to
break the node completely, as in the above case: Freenet will fail to
start up if it fails to bind the darknet UDP port. Then if the user
comes to us we can quickly see what went wrong. Similarly with major
milestones in a normal startup, for the same reason, and generally
anything we'll need to know when a user comes to us saying their node is
totally broken, or where we need to tell the user but haven't yet got
around to implementing a user-alert to show it on the actual fred user
interface.

General debugging should use Logger. Then you can filter it using the
logger settings in the configuration, which allow e.g. to show different
priority levels for different subsystems, change the log rotation
interval etc.

Anything that indicates an actual error should be ERROR. Something that
is suspicious but only a problem if it happens a lot should be WARNING.
Important stuff that happens normally but it's useful to know about if
debugging should be NORMAL.

MINOR and DEBUG are used for really expensive, chatty logging of details
(including potentially incriminating details e.g. keys). These should
always be conditional on logMINOR/logDEBUG, which means you need the
following boilerplate setup:

private static volatile logMINOR;
private static volatile logDEBUG;
static {
Logger.registerClass(MyClassName.this);
}

This is repeated in many classes in fred.

This should be on a wiki page ...
>
> On Sat, Dec 26, 2015 at 1:39 AM, Ian  wrote:
>
>> This looks great!
>>
>> I notice a lot of System.out.println()s in the code, do we not have a
>> logging mechanism in Fred available for use by plugins?
>>
>> On Fri, Dec 25, 2015 at 10:14 AM, Xiaoyu Huang <007...@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> As nextgens suggests, I write this plugin which is based on the newer and
>>> more stable UPnP lib named Cling (http://4thline.org/projects/cling/).
>>>
>>> Currently It only supports FredPluginIPDetector and
>> FredPluginPortForward.
>>> I will add FredPluginBandwidthIndicator and Web UI support when I have
>>> time.
>>>
>>> Code review, suggestions and tests are required. The source code can be
>>> accessed here:
>>>
>>> https://github.com/007pig/plugin-UPnP2
>>>
>>> I uses Gradle as the build system. So to build the plugin and package it
>> as
>>> plugin jar, you need:
>>>
>>>1. Copy freenet.jar, freenet-ext.jar and bcprov-jdk15on-152.jar to
>>>"[project root]/libs" dir.
>>>2. Run "./gradlew build shadowjar".
>>>3. After build, you can find the plugin in "[project root]/build/libs"
>>>with file name "plugin-UPnP2-1.0-SNAPSHOT-all.jar".
>>>
>>> Thanks,
>>> Xiaoyu



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] New UPnP2 plugin

[Matthew Toseland]

On 27/12/15 19:55, Ian wrote:
> On Sun, Dec 27, 2015 at 1:20 PM, Matthew Toseland <t...@amphibian.dyndns.org
>> wrote:
>> On 26/12/15 16:03, Ian wrote:
>>> Best practice is to use the logging system, using System.out.println()
>> for
>>> logging is almost universally regarded as bad practice, many static
>>> analysis tools will automatically flag it as a problem.
>>>
>>> The example you gave in NodeCrypto is not best practice, that should
>>> probably just be something like:
>>>
>>> Logger.error(this, "Could not use port: "+bindto+ ':'+portNo+": "+e, e)
>> In this case it is appropriate, because the above is only shown if the
>> node is about to fail to start up (at least in the darknet case). See my
>> message explaining the current policy. Changing the policy may be
>> appropriate, I'm just documenting the current status quo.
>
> Ok, but in that case this "policy" deviates from standard practice in Java
> programming, which is to use a logging framework for all log messages, even
> "fatal" messages.
>
> The exception might be in Freenet was a command-line app, but that's not
> how most people use Freenet.  A fatal error should ideally be reported via
> FProxy, since most users aren't going to see stuff written to the console.
In this particular case, it doesn't complete startup, and exits. I
believe it will try to bind fproxy and continue, but if it fails to bind
the darknet port it dies - after which fproxy is not running.

I agree that serious errors that don't prevent the node from running
should be handled as user-alerts via fproxy. For debugging they also
need to be logged, of course.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Uptime stats

[Matthew Toseland]

http://127.0.0.1:/USK@pxtehd-TmfJwyNUAW2Clk4pwv7Nshyg21NNfXcqzFv4,LTjcTWqvsq3ju6pMGe9Cqb3scvQgECG81hRdgj5WO4s,AQACAAE/statistics/1057/plot_week_uptime.png

According to ArneBab, this has changed significantly recently due to a
bug fix: We were over-counting high uptime nodes. So we have fewer high
uptime nodes than we thought.

It's hard to read though. Steve, could you add a cumulative distribution
line or something, so we can say e.g. X% of nodes have uptime > 50%?

It has significant consequences for long-term strategy: If most nodes
have low uptime then darknet is very much harder. While the times when
your node is operational may correlate strongly with those for your
friends, when we get a few hops in any random direction this will run
into problems. At best we get dramatically different available data
depending on the time of day; at worst, even with friend-of-a-friend
connections (and maybe for more hops than that!), we could get a
fragmentary network, i.e. no global connectivity at certain times of day.

Please don't take this as a Council of Doom - there are things we can do
about this, e.g. making it very easy to run Freenet on a Pi. But first
we need to clearly quantify the problem.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Freenet Canary

[Matthew Toseland]

On 04/12/15 03:37, xor wrote:
> On Thursday, December 03, 2015 05:12:37 PM Arne Babenhauserheide wrote:
>> Am Mittwoch, 2. Dezember 2015, 14:34:41 schrieb Ian:
>>> That being said, I do think the project would significantly benefit from a
>>> new and much more engaged leader, ideally with project management
>>> experience, but unfortunately such people do not grow on trees when you
>>> need them to work voluntarily.  Should we find such a person I would
>>> support them in a heartbeat.
>> I think that the project is moving nicely at the moment. Yes, we have
>> less releases per year than back when we still had a paid core
>> developer and maintainer, but there is great work done in many areas —
>> and this month will see the first ever Freenet Hackathon.
>>
>> It feels like the Freenet developer community is more and more finding
>> its rhythm after having to completely reshape its workflows 2 years
>> ago when Matthew went to university. It feels like not a week passes
>> without anyone announcing a new improvement on IRC, pull-requests get
>> active reviews and are improved until they get merged, and the focus
>> is on doing things which improve Freenet for its users.
> I fully agree!
>
> We *have* a good leader with Steve: He does review code whenever it is 
> necessary, and he does make clear decisions. And people do follow his 
> decisions (which I admit was initially bad at, but I hope to have improved 
> and 
> continue improving).
I agree, our volunteers are brilliant, especially Steve. However, the
reason that stuff hasn't been merged is that we don't have the capacity
to review it. This is because Steve is a volunteer. In the past it was
because our previous leader (Ian) regarded code review as unimportant,
but the wider community - at least me - refused to accept unreviewed
code - and because a lot of the volunteer stuff was unfinished,
especially GSoC projects. So I got on with the stuff believed to be
important... Later on I made the opposite mistake of being too often
distracted by would-be-maybe volunteers making complex technical demands.

Maybe we need to have a discussion about how to do code review. "Let
Steve do everything" appears to be something of a bottleneck!
>
> The huge rants on the mailing list during the past few months are a very poor 
> representation of the state of the project.

The fact that we have architectural problems which have been known about
for years, warned about by qualified volunteer developers, and are now
being actively exploited (unless the newspaper article is a plant), is
also a very poor representation of the project. The reason this
particular flamewar came up is that:
- mrsteveman (different to Steve, but another established contributor)
found the article (which we assume to be accurate) about US cops
exploiting Freenet
- I drafted an FAQ update about this, which was accepted, in the
interests of honesty.
- A relatively unknown poster complained about this.
- I tried to find ways to resolve this, but as you say we've been over
the whole opennet discussion many times before.

I agree that this is a problem we've discussed before. Volunteers have
warned about it, especially Florent. Our previous leader overruled all
the warnings. There are good pragmatic reasons to have opennet, but
unfortunately it is insecure against any reasonable threat model. This
is a serious problem and we need some sort of strategy for dealing with it.

Simultaneously we have another two crises:
- We don't have the review capacity to deploy existing contributions, as
mentioned above.
- We have run out of money and had to drop our remaining paid developer.

All in all there are good reasons to discuss project priorities, and
especially how we can get away from opennet, and whether it can be
saved. As long as such discussions eventually lead to action. Of course
they don't, because the priorities and opinions of various key players
are fundamentally incompatible, short of a major conceptual or technical
breakthrough.

> While the observation of Ximin and Arne that this could be a psy-ops attack 
> [1] does sound sort of paranoid, it is nevertheless an actual possibility: It 
> would be a lot easier and cheaper than any other attack. It doesn't require 
> the ability to write or understand code, you just need to be capable of 
> offense :)

You think the newspaper article is a fabrication then? Maybe - but we do
have another source for this, albeit vague.
> As a productive alternate thing to do than those discussions, maybe 
> participate in code review - there are many pending pull requests, for 
> example 
> 32 fred ones:
> https://github.com/freenet/fred/pulls

Agreed. But it can't be *purely* collaborative - somebody does have to
go over the final diff and apply it. That is potentially a bottleneck. ???



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org

Re: [freenet-dev] Freenet showstoppers

[Matthew Toseland]

On 03/12/15 00:27, Psalle wrote:
> On 03/12/15 00:44, Matthew Toseland wrote:
>> On 02/12/15 22:34, Psalle wrote:
>>> Let me hijack the topic at this point (following Victor Denisov trail)
>>> into another though experiment: instead of arguing what Freenet needs
>>> or could do, I'll contribute my *chief* reason not to use it (I've
>>> been running it from time to time since around 2005).
>>>
>>> Although I just said I'll say one chief reason, I'll cheat and offer
>>> one using three different hats.
>>>
>>> As a /user/, freenet is extraordinarily heavy for the computers I've
>>> used it on (some of them not that low spec). Disk trashing in some of
>>> them was so interfering with normal use to make it unbearable.
>> Unfortunately we've done nearly everything we can about this. The client
>> layer is *far* more efficient than it was on disk I/O, and the datastore
>> too.
>>
>> Maybe it needs some careful profiling.
>>
>> One thing we could do is use I/O priorities though, especially on
>> Windows. This might help significantly.
> Curiously, my subjective impression is that I/O load is more
> noticeable on linux than on windows (not for freenet but in general).
> I have had problems with very large dropbox folders in recent years
> but only when using ubuntu.

Well, Linux also has system calls for I/O priorities. There is a library
for calling syscalls which we could bundle and use, and there is a pull
request for using this.

>>> As a /programmer/, the two times I've tried to catch a bug or
>>> contribute some code, I found it exceedingly difficult to dive into
>>> it. Arguably I could have chosen too difficult points of entry...
>> Probably. :|
>> You should ask for some pointers?
> Yep. I don't like nagging people but I guess there's a thing as being
> too shy.
>>> As a /donor/ circa 2007 IIRC, I felt my monthly contribution was being
>>> wasted on features that led nowhere.
>> Any particular features? Ancient history I guess...
> Well, at that time my sensation was that there was a piling of
> tentative ideas on top of ideas in the hope that some of them would
> cause a breakthrough, but I felt that even a good idea would get
> drowned in the general noise. Mind you, this was a non-programmer
> perception, so things were probably different from the inside.

Not that different.
>
> Put another way, I felt that there was not a clear understanding of
> why/how well the network as a whole was working, so all the coding
> without a solid analysis discouraged me. I loved the stats and
> simulations, not surprisingly, so in that regard I was happy.

Yep. We try to be a bit more empirical. My project this year should help
with this.
>
> I understand this is a very hard problem from a scientific point of
> view. I guess I miss more developments in the theoretical side of
> things. It's not that I am against experimental algorithms, but that I
> see the freenet codebase too large and complex to really be an
> efficient platform for experimenting right now. In that regard, I
> admire how much effort you and others have devoted to such a
> frustrating problem.

Yes, exactly, it is hard to evaluate changes so we can't just try them
and see what happens. We need to be able to simulate changes "in vitro"
first, and then, if possible, have micro-benchmarks so that we can
evaluate specific consequences of our changes on the real network.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Disk thrashing was Re: Freenet showstoppers

[Matthew Toseland]

On 03/12/15 07:20, Victor Denisov wrote:
>> As a /user/, freenet is extraordinarily heavy for the computers I've 
>> used it on (some of them not that low spec). Disk trashing in some of 
>> them was so interfering with normal use to make it unbearable.
> It's gotten from "computer is unusable when Freenet is running" to
> "computer is barely usable when Freenet is running" (which absolutely
> *is* an improvement, btw). 

Umm... good? :|

> However, Freenet is still extremely
> IO-limited (to the point that I have to limit bandwidth to around
> 200-300 KB/s, or the node is backed off almost 100% of the time) 
This is surprising. Are you sure it is actually I/O bound and not just
having an impact on the rest of the system? Is there any way you could
test this? We need to know whether it is actually disk limited or
whether it is merely unfair on the rest of the system, since the fixes
may be different.

Backoff should not be related to I/O *at all*, if you mean the number of
peers listed as "BACKED OFF" (or "BUSY"). Unless you mean something
else? It is quite possible that there are multiple simultaneous problems.

Is this all datastore, or do you have active uploads, active downloads?
The client layer is responsible for a significant fraction of disk I/O
nowadays, and uploads are much less efficient than downloads even after
we got rid of db4o.

Datastore reads are constant (but should actually be fairly infrequent),
but datastore writes are periodic bursts, thanks to the caching layer.
There are tunables you could play with - IIRC there is a buffer size
config variable? Also, it's not actually reconstructing right now? If it
is it would have much heavier disk access but there'd be a message about it.

> - which
> is something I don't get, as torrent clients on ARM-based NAS boxes
> (arguably, about 10% of my desktop's computing power) have no issues
> with sustaining 4 MB/s 100 GB+ torrents for hours, and it's basically
> the same workload (small-block multithreaded random reads/writes), IO-wise.

Right. We ought to be able to handle this. Although I believe BitTorrent
uses bigger blocks, which might be a factor here - and hard to fix.
> Regards,
> Victor Denisov.
>
> P.S. I knew I've seen this discussion before, and that I've made the
> same point about performance before, so this is what I've found - from
> 2.5 years
> https://emu.freenetproject.org/pipermail/devl/2013-July/037145.html.
Yes, the pay-for-opennet thing, like many other ideas on Freenet, has
been refined and discussed periodically over a period of years. This is
normal, the details evolve and the context changes; many such ideas
eventually get implemented. If they are posted on the mailing list then
we have some hope of attributing them properly. Interestingly, MAST
first came up on Frost, i.e. anonymously and without proper archives.
And it caused no end of worry and disruption for people like me, in
spite of turning out to be bogus. Psy-ops perhaps? Who knows.

Re your old message, I agree that we will probably have growing darknet
cells glued together by opennet for some while, at least if we keep the
goal of a single coherent network. I do have ideas for improving
performance, which I might get to over the coming year. I believe
corporate leak hunters may have access to intelligence level resources,
at least to the tools that the regular police have access to (one of the
key resources of a good PI is a network of bent cops willing to do
favours!). And we could run tunnels over darknet if people *really*
don't trust their friends, although it would take quite a bit of
additional research (PISCES is rather unclear on some key points); the
price of making this fast would be less invisibility, but people like
China and Iran can block Freenet even if it's pure darknet.

I think sorting out system overhead issues is probably more
important/urgent than general downloading performance though. Because it
causes people to uninstall Freenet, or (occasionally) to run it with
poor uptime.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Disk thrashing was Re: Freenet showstoppers

[Matthew Toseland]

On 03/12/15 17:48, Matthew Toseland wrote:
> On 03/12/15 07:20, Victor Denisov wrote:
>>> As a /user/, freenet is extraordinarily heavy for the computers I've 
>>> used it on (some of them not that low spec). Disk trashing in some of 
>>> them was so interfering with normal use to make it unbearable.
>> It's gotten from "computer is unusable when Freenet is running" to
>> "computer is barely usable when Freenet is running" (which absolutely
>> *is* an improvement, btw). 
> Umm... good? :|
>
>> However, Freenet is still extremely
>> IO-limited (to the point that I have to limit bandwidth to around
>> 200-300 KB/s, or the node is backed off almost 100% of the time) 
> This is surprising. Are you sure it is actually I/O bound and not just
> having an impact on the rest of the system? Is there any way you could
> test this? We need to know whether it is actually disk limited or
> whether it is merely unfair on the rest of the system, since the fixes
> may be different.
>
> Backoff should not be related to I/O *at all*, if you mean the number of
> peers listed as "BACKED OFF" (or "BUSY"). Unless you mean something
> else? It is quite possible that there are multiple simultaneous problems.

It *is* possible that other nodes could backoff your node because of
really slow disk I/O, because we have to check the datastore when
accepting a request. This would result in fewer/slower peers. It's worth
testing this empirically. Possibly we should keep some statistics, e.g.
average time taken checking the datastore (separately for hits and
misses; most misses won't do any disk reads because they're served from
the in-memory structures).



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] First hop over Tor

[Matthew Toseland]

This has been suggested a few times but there were various worries. I
think the following is the most coherent proposal evolution so far:

Freenet: Tor first hop:
- Different protocol, TCP based. Don't need transport plugins.
-- "Do requests/inserts" - closer to client layer, not FNP. May return
new gateway nodes too, via path folding.
- Gateway nodes are not anonymous.
-- Even if we tried to make them anonymous, they'd have normal
connections, could start a request through a tunnel and trace it.
-- Having all connections over Tor would be way too slow and upset Tor
people too.
- Advertise in noderef, proxy.tor=... [ NOTE NOT physical.tor=... !! ]
-- Client nodes only accept proxy.tor= entries through tunnels, not from
direct connections.
-- But they DO relay them.
-- Hence need some initial "anonymous seeds".
-- Note: Capture is possible because of this, countermeasures maybe
similar to opennet seed capture...
- Which requests use which tunnel?
-- Long term issue, will want to label traffic generators / FCP clients
/ etc.

Downsides:
- Cheap denial of service attacks. Asymmetrical. Maybe we could make it
bandwidth-symmetrical, e.g. by requiring bogus data transfers to balance
both directions, but is that enough?
- Tor is much more likely to be blocked than Freenet. :(



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Freenet Canary

[Matthew Toseland]

On 01/12/15 21:59, salutarydiacritica...@ruggedinbox.com wrote:
>
> You picked the one thing agencies have boat loads of and made it a
> requirement for operating critical parts of the network and you
> alienate the honest users left. Have you thought of any better ways to
> kill Freenet? That's got to be the dumbest idea I've heard. Its better
> to hear my criticism than see the failed results in the newspapers.
>
> Social capital is a better way to limit evil nodes. A centralized
> model of authority servers run by trusted project members is the way
> to go. Freenet users have to trust you to not backdoor the program.
> Having to rely on infrastructure that you run isn't more risky. You
> are starting to see the limitations of a completely decentralized
> network being safe.
>
> According to you a whistleblower should connect to a journalist's node
> in Darknet mode if they want to steer clear from Opennet Sybil? The
> metadata is OK to share because Big Brother already has the address
> book... What about the political dissident who wants their blogs to
> reach more people than they know IRL? Sorry Opennet is out of order
> because Sybil.
>
> Darknet is private not anonymous and lacks the quality of data
> availability after the publisher is offline.
>
> My observation is Freenet has a leadership crisis. There is no
> agreement on how to reach the goal of anonymous communication. Every
> developer has contradicting opinions and pulls the project in a
> different direction. There is no formal process to reach a consensus
> or draft technical proposals others can review and comment on. No
> coordinated development effort. No new developers because no modern
> build process or documentation. No protocol documentation because its
> a work in progress. Its a work in progress because of rewrites and
> because no attempt to implement tried and true models like mixnets
> researched for 40 years. No users because no progress in security and
> UX. No donations and academic attention because no users.
>
> Fix the organization and your processes and there is hope. If you're
> not capitalizing on awareness of mass surveillance now you never will.
>
> Otherwise you should officially admit its a hobbyist research project
> so people don't have huge expectations and not over trust.
>
> Shoot the messenger but I doubt it will help.

Centralized solutions are not acceptable. You don't have to fully trust
the developers because you have the source code. We strongly encourage
you to review it and compile your node from source. No doubt there are
subtle bugs, most likely introduced by mistake, but if there were
obvious back-doors you could find them. And maybe the subtle bugs too.
Developers are volunteers, with limited resources.

Anything that needs big central supernodes is likely to be run by the
bad guys, and would be highly vulnerable. We know this because it has
happened before - there is evidence of Mixmaster remailers being
compromised. Equally important, we would like the network Freenet to
survive the end of the organization Freenet Project Inc.

Right now opennet needs seednodes, but darknet is fully decentralized.
Some proposals for improving opennet require the seednodes to do a bit
more work.

We need darknet because the *one and only thing* that genuine users have
that attackers do not have is friends. If you only connect to your
friends and maybe their friends, it is a great deal more difficult for
anyone to locate the source of a controversial blog. Ideally we would
have a global friend-to-friend darknet, but this will take time; in the
short run we have darknet pockets connected via opennet. This means that
attackers may be able to trace back to a single darknet pocket.

Publishing on darknet works in exactly the same way as it does on
opennet: You upload your site, e.g. with jSite, and the data is stored
on other people's nodes. It does not depend on your node being online,
it depends on whether people access the content.

We are well aware of the possibilities of mixnets, and in fact I was
trying to discuss how to implement one. Using Tor for the first hop is a
possibility, but Tor is more likely to be blocked than Freenet, and any
such scheme likely has easy denial of service attacks. Also we'd like to
provide high latency tunnels for inserts, something that Tor doesn't do.
And we'd like whatever we build to be scalable, which Tor isn't; from
the beginning scalability has been an important Freenet goal.

If we implement our own, scalable solution then we have to deal with
distributed tunnel setup, which is a well-studied hard problem. The
nominal O(c^2/n^2) security you might get in theory with a fixed list of
nodes is hard to achieve in practice, especially in a scalable way. Tor
reduces the problem somewhat by not scaling, i.e. relying on a global
consensus of nodes. ShadowWalker is one (published) distributed
solution, which tolerates up to 20% Sybil nodes. However to make this
work on opennet we 

Re: [freenet-dev] First hop over Tor

[Matthew Toseland]

On 02/12/15 17:29, Ian wrote:
> On Wed, Dec 2, 2015 at 11:13 AM, Matthew Toseland <mj...@cam.ac.uk> wrote:
>> Downsides:
>> - Cheap denial of service attacks. Asymmetrical. Maybe we could make it
>> bandwidth-symmetrical, e.g. by requiring bogus data transfers to balance
>> both directions, but is that enough?
>> - Tor is much more likely to be blocked than Freenet. :(
> The most obvious downside to me is that we'd basically be combining all of
> the disadvantages of Freenet with all of the disadvantages of Tor.  It
> doesn't make sense to me.
>
> Imagine if you could only run Linux if you were using it in an emulator on
> top of Windows.  I doubt it would be very popular.
Yeah, the idea is that it's an option for the paranoid. People keep
asking for it so AFAICS there is demand.

Just getting it out there.

Also, given that for now the best we can hope for (for the global
network) is a network of darknet pockets linked by opennet, this is a
relatively cheap way to provide something approaching acceptable
security. I believe it could be implemented relatively quickly. But I
agree that it probably shouldn't be the default.

Unless you think the focus should be on enabling specific
sub-communities to create their own darknets which are not connected to
the rest of the world, with the main network and opennet just being a
"shop front", a demo version? Maybe we should work more directly with
organizations as opposed to individual users, who might be able to
deploy substantial disconnected darknets?



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Freenet Canary

[Matthew Toseland]

On 02/12/15 20:34, Ian wrote:
> On Wed, Dec 2, 2015 at 2:24 PM, Victor Denisov  wrote:
>
>> This is very interesting, it is - but I'm afraid the true reason Freenet
>> is struggling as a software development project is much, much simpler.
>> If I can put my two kopecks' worth out there, my *personal* feeling is
>> that the project has been lacking some sort of a "steel hand",
>> Stalin-style, ever since Ian had stepped down.
>>
>> Basically, what Freenet had shown in that regard is that management "by
>> committee" doesn't work for open-source projects just like it doesn't
>> work in the world of commercial applications. You (developers) and we
>> (users) desperately need a real *leader*, a person who will listen to
>> different points of view and then make quick, binding and final
>> decisions - which everyone will respect and adhere to, and which will
>> end any and all discussions of the topic in question. You/we should
>> choose *one* person and willfully grant him/her authority to make final
>> unilateral decisions on all aspects of the project.
>>
>> I'm afraid that before that is done, Freenet is destined to be stuck in
>> the development mire it currently is.
> I'm not sure, I've definitely been very hands-off in recent years, but even
> at the peak of development around 2000-2003ish I don't think anyone would
> describe me as "Stalin-like", I always tried to act more as a
> "facilitator", trying to encourage people to play nice and be an
> independent arbiter.  For a while that worked pretty well, perhaps
> partially because the constant flow of favorable publicity provided a
> constant flow of new, enthusiastic, and (often) competent developers
> willing to devote their time.

IMHO we need to play the publicity game again. Maybe it will take a
little while to get our pieces in place e.g. sort out the website. But
we need to do something to get some attention, even if it's just calling
it 0.8.
> I don't think a "Stalin-like" approach will work for most open source
> projects because unlike Stalinist Russia, people voluntarily need to want
> to be part of your project or they'll just walk away.

I agree, being too heavy handed means people will fork the project at
best (which has happened to Freenet at least twice). At worst you lose
volunteers, or at least they are less effective. This has certainly been
a problem too.

In particular, there are long-running disagreements about things like
whether we should have opennet support. The iron fist approach is that
people like Florent who don't support Ian's view (that we should have
opennet) should just leave. That would not be helpful; Florent has
contributed lots to Freenet since opennet went in.

Or even worse you go down a blind alley and lose several years of paid
development time (this also happened, but in retrospect "db4o is wrong"
was far from obvious at the time).

Of course it doesn't help if your few paid staff are obstructive and
disobedient (no, I'm not talking about Xor). Especially when they're
right. ;)

We suck at keeping volunteers. That's a serious problem. I don't think
documenting *the protocol* will fix it, but documentation and cleanup in
general should help. Beyond that I don't know...
> That being said, I do think the project would significantly benefit from a
> new and much more engaged leader, ideally with project management
> experience, but unfortunately such people do not grow on trees when you
> need them to work voluntarily.  Should we find such a person I would
> support them in a heartbeat.
>
> Ian.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Darknets (One more way to look at opennet weakness)

[Matthew Toseland]

On 02/12/15 21:40, Matthew Toseland wrote:
> There's one more way to look at this:
>
> The global network, "Freenet" as an entity, is our shop front. The real
> business in the medium term is helping existing communities and
> organisations to build their own, disconnected darknets.
>
> This might have positive consequences for funding - working more
> directly with people who can use Freenet, or with people who provide
> funding related to such things. It might work around some of the
> political difficulties even - what's on your darknet is up to your
> users. It might result in faster deployment, and larger darknet pockets
> - seed communities from which it can grow. It might make invites to
> specific community darknets valuable, which is potentially a good thing
> for growth. It would give us a really convincing answer to "opennet is
> busted, why should we take you seriously?". And the long term plan would
> still be to connect up most of the individual darknets to a global network.
>
> Thoughts?

There are some technical consequences. Apart from the badly needed
enhancements to darknet usability and performance, we'd also need to
look into sorting out the problems with small darknets. Most of the
organisational branding etc can be run through darknet features e.g.
including custom bookmarks in an invitation bundle. However we might
want to allow some sort of network ID too.

There are a few other related issues e.g. should we search the local
darknet before going out onto opennet, possibly on the level of whole files.

But really it's a different way of looking at growth.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Darknets (One more way to look at opennet weakness)

[Matthew Toseland]

There's one more way to look at this:

The global network, "Freenet" as an entity, is our shop front. The real
business in the medium term is helping existing communities and
organisations to build their own, disconnected darknets.

This might have positive consequences for funding - working more
directly with people who can use Freenet, or with people who provide
funding related to such things. It might work around some of the
political difficulties even - what's on your darknet is up to your
users. It might result in faster deployment, and larger darknet pockets
- seed communities from which it can grow. It might make invites to
specific community darknets valuable, which is potentially a good thing
for growth. It would give us a really convincing answer to "opennet is
busted, why should we take you seriously?". And the long term plan would
still be to connect up most of the individual darknets to a global network.

Thoughts?



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Freenet showstoppers

[Matthew Toseland]

On 02/12/15 22:34, Psalle wrote:
> Let me hijack the topic at this point (following Victor Denisov trail)
> into another though experiment: instead of arguing what Freenet needs
> or could do, I'll contribute my *chief* reason not to use it (I've
> been running it from time to time since around 2005).
>
> Although I just said I'll say one chief reason, I'll cheat and offer
> one using three different hats.
>
> As a /user/, freenet is extraordinarily heavy for the computers I've
> used it on (some of them not that low spec). Disk trashing in some of
> them was so interfering with normal use to make it unbearable.

Unfortunately we've done nearly everything we can about this. The client
layer is *far* more efficient than it was on disk I/O, and the datastore
too.

Maybe it needs some careful profiling.

One thing we could do is use I/O priorities though, especially on
Windows. This might help significantly.
>
> As a /programmer/, the two times I've tried to catch a bug or
> contribute some code, I found it exceedingly difficult to dive into
> it. Arguably I could have chosen too difficult points of entry...

Probably. :|
You should ask for some pointers?
>
> As a /donor/ circa 2007 IIRC, I felt my monthly contribution was being
> wasted on features that led nowhere.

Any particular features? Ancient history I guess...
>
> I understand that this post might be interpreted as a very negative
> criticism (and become quite depressing), but that's not my intention
> at all. I simply had to prioritize money, time and resources, and
> those where the reasons I can remember instinctively (I won't argue my
> instinct and biases are perfect). I'm frustrated many times by not
> having something useful to contribute, but since these are things I'm
> sure of, I think this perspective can be a positive negative ;-)
>
> Cheers.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Fwd: Promiscuous opennet

[Matthew Toseland]

On 01/12/15 18:35, Juiceman wrote:
> I originally sent this email 3 and a half years ago.
> Would it help to limit seednodes acceptance of announcement attempts to
> once per hour? Would slow harvesting down and encourage uptime from users...

There are some throttling measures already. But they only apply on each
individual seednode. Also, path folding - normal opennet nodes exchange
noderefs constantly.

> -- Forwarded message --
> From: "Juiceman" 
> Date: Apr 10, 2012 2:32 PM
> Subject: Promiscuous opennet
> To: "Discussion of development issues" 
> Cc:
>
> Should we do something about the amount of announcements/refs handed
> out?  Seems a good way to harvest opennet (known problem, but perhaps
> we could slow it down...) or perhaps monopolize/monitor the key-space
> somehow?
>
> opennetSizeEstimateSession: 18985 nodes
> opennetSizeEstimate24h:12435 nodes
> opennetSizeEstimate48h:14855 nodes
> nodeUptimeSession:   4d23h
>
>
> Seed stats
> IPConnected  Announced  Accepted  Completed  Sent
> refsVersion
> 24.126.77.671766352352352
>   5201406
> 202.134.203.35 1785279279279
>  1812  1377
> 213.134.163.3   1792275275275
>   1819  1388
> 68.227.101.68   1793293293293
>   1747  1389
> 62.107.42.7   1803257257257
> 1774  1389
> 67.61.122.2461840267267267
>1809 1397
> 82.66.7.51 1843247247247
>  1400  1397
> 78.105.55.2061846290290290
>2053  1388
> 68.144.7.7 1853438438438
>  1481406
> 80.217.175.104  1868314314314
>   2053  1388
> 67.169.243.331919317317317
>4161406
> 208.76.88.92  1939327327327
> 1980  1376
> 98.21.244.1131951317317317
>2238  1376
> 50.4.40.1461979308308308
>  2106  1385
> 71.204.143.231   2041292292292
>1629  1385
> 81.56.65.622043278278278
>  2209  1384
> 88.174.162.112   2071297297297
>2104  1378
> 88.90.64.247  2110278278278
> 1658  1397
> 88.172.49.91  2120305305305
> 1730  1372
> 69.228.173.126   2146318318318
>2247  1364
>
>
> --
> I may disagree with what you have to say, but I shall defend, to the
> death, your right to say it. - Voltaire
> Those who would give up Liberty, to purchase temporary Safety, deserve
> neither Liberty nor Safety. - Ben Franklin
> ___
> Devl mailing list
> Devl@freenetproject.org
> https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl




signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] My project

[Matthew Toseland]

On 01/12/15 01:13, Michael Grube wrote:
> Hey toad,
>
> It's nice to see you back in the community. You know, NS2 can actually
> simulate more or less anything you can think of with quite a bit of
> accuracy. I know there are also extensions to add parallelism. You may want
> to consider writing a Freenet library for ns2 as on option because it would
> make developing simulations easier for others as well.
Yes this has been discussed. IMHO it is important to be able to test
proposed changes to the Java fred code efficiently, especially for load
management changes. There have been *many* hacked-up simulations, all of
which are out of date and no longer useful.
> I can appreciate that Java is nicer in some ways because you can build it
> out into production code more easily, but I'd encourage you to weigh the
> pros and cons.
Running Java inside NS2 might be possible but would be dramatically
slower. It only makes sense for really low level simulations where the
detailed properties of the underlying IP network matter. That's not my
main concern here. I want to show that load management works, that The
Patch breaks it unfairly, and ideally demonstrate a proposed solution. I
want to make it possible to test changes to routing and load management
efficiently but with the actual code.
> On Nov 30, 2015 4:52 PM, "Matthew Toseland" <mj...@cam.ac.uk> wrote:
>
>> My project for university this year involves improving the efficiency of
>> simulations of Freenet (by configurably bypassing the lower layers) and
>> using that to test load management. Ideally I'd like to simulate The
>> Patch and show that it causes problems, and simulate some of the
>> proposed improvements to the current load management. It has to be my
>> own work but I will be posting the branches on my Github repository,
>> since I'd like it to be merged; comments are welcome. I hope to make
>> substantial progress on it over the Christmas vacation, that is, from
>> now until January...



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Darknet uptimes are the key issue

[Matthew Toseland]

On 01/12/15 00:25, Arne Babenhauserheide wrote:
> Am Montag, 30. November 2015, 16:45:43 schrieb Matthew Toseland:
>>>> 2. Freenet needs an always-on always-connected device, especially on
>>>> darknet. Most people don't have one, the costs are significant.
>>> This is not true. 2-12 hours runtime are completely OK. We would have
>>> this using mobile phones which run Freenet only while plugged into
>>> power and already mostly charged and connected over WiFi.
>> Darknet needs high uptime, or at least strongly correlated uptimes. 2
>> hours is definitely not enough - even with FOAF connections, you'll be
>> lucky to find enough peers.
> It’s OK when some only have 2 hours, as long as others have 12 or 24
> hours.

This is the key issue, so I'll try to reply to every aspect raised here.

Darknet only works efficiently when you have a reasonable number of
peers online simultaneously. FOAF makes the set of possible peers
bigger, but low uptime dramatically reduces that. If you know 3 people
on Freenet, and your friends are likewise, then you might have a total
of 9 peers - but it's probably less than that in practice because we
expect darknets to be highly clustered, i.e. there is overlap between
friend 1's friends and friend 2's friends.

If you then have low and non-intersecting uptimes, you'd be lucky to
have 5 actually online peers at a time. And this is not enough to give
good performance.

The possible fixes for this are:

1) Connect to friends of a friend of a friend etc. Tradeoffs worth
thinking about. Arguably the invisibility we get from darknet is bogus
anyway, even darknet can be blocked; what matters more is that friends
are less likely to be Sybil. This is worth seriously considering: How
many hops can we safely go with FOAFOAFOAF...? Can we restrict what we
use more distant peers for, e.g. only relaying low HTL traffic? What
effect would that have on routing effectiveness and performance?

2) Connect to opennet. This is much more tolerant of low uptimes: It
still costs us storage, but it doesn't break connectivity completely.

3) Improve load management so you get more out of your 5 peers, even
though fast opennet nodes have 150 peers. Maybe this is possible. I'm
not sure how to make the maths add up!

4) Long term requests / delay tolerant networking. This is really hard,
and doesn't match user expectations. UI can help with the latter. In the
long run it may allow for new forms of steganography and new transports.

5) Hardware nodes to make it easier for people to have high uptimes. But
there are still significant costs associated with them: Buying them
(including a storage device with a limited lifespan), storage, noise,
fire risk associated with leaving equipment on, instantaneous load on
the internet connection (often a shared policy issue which you have
limited input into), monthly traffic limits / effect of higher monthly
traffic on upstream contention policy on other users. If you rent you
may not have the option of getting a new provider, other than expensive,
capped, p2p-blocking mobile carriers, and guess what, in our brave new
world everyone under 40 rents. Energy and noise too - but hardware nodes
greatly reduce these factors. Privacy/performance tradeoffs for e.g.
long term downloads (e.g. do you want to turn off the client layer when
you're not physically present). How to advertise the device to your
local clients without advertising it to other people on the LAN. Etc.
>>>> 3. Darknet is slow.
>>> This is not true. 5-10 Darknet connections are enough to get good
>>> performance.
>> Right, and with FOAF we could have tens of peers. But you do need the 5+
>> friends to start with. That's hard.
> with FOAS 3+ would suffice.

No, it won't. Certainly not with poor uptimes.
>
>> I agree that this part is fixable and we must fix it: There are lots of
>> technical things we can do to make darknet work better, easier and faster.
> Yes, and those are the things we should do before discussing to death
> how we could fix opennet if that would prove to not work.
>
>> We need an opennet to link up all the slowly expanding darknet pockets.
>> For now.
> I think Opennet is already good enough for that. Let’s focus on
> improving Darknet.

That depends on your threat model. One proposed recently is
"corporations can't data mine my browsing on Freenet". The problem with
that is the classic chicken and egg: There isn't much content, there
aren't many users, so only people who have something to hide or are
interested in the politics and technology use it.

Avoiding corporate data mining only makes sense if there is something
you actually want to do that you could do on both Freenet and the
corporate data mined "free" internet. Is the answer to that simply that
we need more content and services such as Sone etc? May

Re: [freenet-dev] Security quibbles was Re: Freenet Canary

[Matthew Toseland]

On 30/11/15 23:58, Arne Babenhauserheide wrote:
> For my personal threat model hybrid works. Whistleblowers definitely
> need darknet with FOAF.
>
> What security properties would you assume for someone who has a
> darknet connection with FOAF (explicitly: routing over all darknet
> peers of the darknet peer) when the darknet peer has 5 other darknet
> friends? Would that suffice for a whistleblower who logs into Freenet
> 5 times, communicates over Freemail and logs off after less than an
> hour? (the journalist-whistleblower workflow)

Whistleblowers need a public service they can connect to from a cafe
that still provides reasonable protection and isn't blocked. They
probably don't have existing darknet connections.

My opennet proposal actually provides this: Opennet with tunnels, but
enough scarcity that it can't be easily subverted. Having said that, a
centralised solution may be simpler - provided it isn't blocked.

Also they need a plane ticket to Ecuador. You will be found out, you
have to plan on the assumption that your whole life will be permanently
altered. The days of Deep Throat remaining embedded and leaking
information over a period are long gone. Freenet probably can't change
this. So really all they need technically is GPG, or the dropbox systems
we're starting to see.
>> If not, what needs implementing to get it there?
> For whistleblowers: Darknet invitations, Darknet FOAF, transport
> plugins (to hide the connections from ISP-level monitoring), WoT with
> faster bootstrapping (getting the initial IDs).

Transport plugins to hide connections? Transport plugins can't prevent
blocking, even of darknet, because of traffic flow analysis. Freenet in
general won't prevent targeted surveillance.

If WoT bootstrapping becomes the bottleneck in connecting to
whistleblowers then it will be DoS'ed. This could be done very cheaply
(although not for free).

Freenet will only work for whistleblowers when everyone else is using it
too.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Freenet Rebooted (without rewriting everything, pay for opennet)

[Matthew Toseland]

On 01/12/15 04:13, charles wrote:
> So based on what I've read from the rest of this thread, that is not
> stated clearly in this pitch, is that a Full Opennet Node is similar to
> Tor's Relays or Directory Authorities (I believe you'r saying the later,
> but not sure, maybe some combo?). So the only thing that people would
> pay for was the right to run one of these Relay/Authorities. The average
> Freenet user would never need to be aware that this was happening in
> order to use opennet as they normally do, for free. Is that correct?

Pretty much.

The idea is that Full Opennet Nodes would participate in tunnels, and we
might also restrict routing high HTL traffic to them. Ordinary nodes
would still store data and relay traffic. Or ordinary transient nodes
would just connect and send requests (including tunneled requests).
Users who run an FON and run local requests on it get better performance
than an ordinary opennet node would.

Right now there are no tunnels on Freenet. There are ways to add
tunnels, but if we want to remain scalable/near-fully decentralised,
they only protect against up to 20% malicious nodes. And we would
certainly have to avoid using slow nodes in tunnels if we want
reasonable performance.
>
> -Charles
>
> On 11/30/15 10:29 AM, Matthew Toseland wrote:
>> We have several major problems:
>> 1. We need a major injection of cash.
>> 2. We will not have a big connected darknet any time soon.
>> 3. Opennet is not secure unless users pay for introduction.
>> 4. Opennet is slow because of lowest common denominator load.
>>
>> I propose: Freenet Rebooted.
>>
>> A Kickstarter, but based on extending the current code, not a full
>> rewrite. A lot of it actually works reasonably well.
>>
>> MAJOR CHANGES:
>> 1. Darknet enhancements, but we recognise that we will need a large,
>> fast opennet backbone to connect the darknet pockets for the time being.
>> 2. You can only run a Full Opennet Node if you have an Opennet Invite
>> and meet bandwidth/performance requirements.
>> 3. Only Full Opennet Nodes route tunnels and/or high HTL traffic.
>> 4. There may be further restrictions for security reasons, if so we will
>> ensure that an OI still gives performance benefits (even if you are not
>> routing traffic).
>> 5. Opennet tunnels via ShadowWalker.
>> 6. Better seednodes.
>> 7. Most of the enhancements to other areas we've previously discussed.
>> 8. Transient mode reintroduced, so opennet Freenet is still free as in
>> beer, and secure with tunnels. Great for uploading on the run! But
>> transient nodes don't route traffic/tunnels and get lower performance.
>> 9. Investigate hardware partners and home-server UI issues. Long term we
>> need cheap, convenient hardware nodes, because we need uptime.
>>
>> Initially we aim to raise $1M. Anyone who donates $100 gets an Opennet
>> Invite, so this is 10,000 users. Hardware nodes might be a good donor
>> perk too. In future we anticipate charging for OI's, but expect an
>> increasing proportion to be provably given to other worthwhile,
>> respected and relevant charities e.g. EFF: The price paid to become part
>> of the network infrastructure is mainly a deterrent to large scale
>> attacks, rather than a means of raising revenue.
>>
>> Thoughts?
>>
>> Obviously there is a risk of people running parallel networks for $99 or
>> whatever, or #freenet-refs style auto-darknet meshes. Some of them will
>> be scammers and we have a good web presence to start with; I don't think
>> we should worry about this.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Freenet Canary

[Matthew Toseland]

On 01/12/15 00:32, Arne Babenhauserheide wrote:
> Am Montag, 30. November 2015, 17:02:26 schrieb Matthew Toseland:
>>> It’s not a zeroday. According to the article, they used a known
>>> vulnerability which takes time and careless behavior of the users to
>>> exploit.
>> Careless behaviour of the users? I didn't see any details? Clearly once
>> they identify a suspect they'll seize their computer and convict them
>> for possession.
> Downloading highly illegal material while not at their computer. Using
> Opennet despite doing something which is among the most despised acts
> in the country. Leaving the Laptop open and running while not at home.
This is partly uptime issues again: Leaving Freenet running is what we
want people to do, for routing purposes. And if they want to download
big files on slow connections, this is very useful for big downloads too.

Against any competent physical adversary being at home when they bust
the door in isn't going to save you unless you have complicated
expensive technical measures in place. They are quite capable of
freezing your RAM etc.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Freenet Rebooted (without rewriting everything, pay for opennet)

[Matthew Toseland]

On 30/11/15 21:05, Arne Babenhauserheide wrote:
> Am Montag, 30. November 2015, 19:36:43 schrieb Matthew Toseland:
>> How much of this is due to default settings where it didn't manage to
>> autodetect via UPnP? How much to users not making informed choices?
> And why do we have such low default settings?

Because we want to support lowest common denominator users, and people
who forgot that they had a traffic limit etc. I agree we might be able
to increase it.
> Best wishes,
> Arne



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] My project

[Matthew Toseland]

My project for university this year involves improving the efficiency of
simulations of Freenet (by configurably bypassing the lower layers) and
using that to test load management. Ideally I'd like to simulate The
Patch and show that it causes problems, and simulate some of the
proposed improvements to the current load management. It has to be my
own work but I will be posting the branches on my Github repository,
since I'd like it to be merged; comments are welcome. I hope to make
substantial progress on it over the Christmas vacation, that is, from
now until January...



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Freenet Rebooted (without rewriting everything, pay for opennet)

[Matthew Toseland]

On 30/11/15 21:10, Arne Babenhauserheide wrote:
> Am Montag, 30. November 2015, 19:58:38 schrieb Matthew Toseland:
>>> Even regular E-Mail providers, G+ and Facebook did not find a way to
>>> get a significant number of users to pay — for a service which is
>>> clearly essential for todays communication. Why do you think people
>>> would pay for Freenet?
>> They don't need them to.
> They are trying and trying and trying. My E-Mail provider is spamming
> me every month with a new deal to join its (paid) club.
>
> So they obviously do need them. They just don’t get them without first
> giving them free service for a very long time.

Depends on your email provider. Google charge for very little, they're
too busy with other revenue streams.
> Best wishes,
> Arne



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Freenet Rebooted (without rewriting everything, pay for opennet)

[Matthew Toseland]

On 30/11/15 21:12, Arne Babenhauserheide wrote:
> Am Montag, 30. November 2015, 20:54:38 schrieb Matthew Toseland:
>> 1) Stick our heads in the sand and sing the glories of opennet, in spite
>> of clear evidence of it being irredeemably broken, or
>> 2) Hope that more people use darknet.
> Binary choices are almost always (self-) deception.
>
> 3) Improve darknet usability so the default mode for new users is
>being invited by an existing Freenet user.

Even if that was true it would still take a long time to have large
enough darknet pockets to provide meaningful protection against an
attacker connected to all opennet nodes. And making darknet easier will
not overcome the other problems: Most people can't run Freenet 24x7 even
if they wanted to, or can't afford to, and most people don't want to run
Freenet at all, even if their friend urges them to.

It's a goal we should strive for, because if we get viral growth then we
probably get exponential growth. But it requires a very high density at
least within specific sub-communities.
> 4) Talk about the things which already work well.
Definitely a good thing, and my congratulations for your doing so on
your blog etc.

But this all started with a thread complaining that we'd added an FAQ
entry saying at least one police force can break opennet Freenet and
we're not doing anything about it and darknet isn't an acceptable
answer. I hope we achieve a global f2f darknet. I doubt it's possible
without at least a long intermediate stage during which mostly smallish
darknet pockets are linked by opennet.
>
> Best wishes,
> Arne



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Freenet Canary

[Matthew Toseland]

On 30/11/15 16:40, Arne Babenhauserheide wrote:
> Hi,
>
> Am Freitag, 27. November 2015, 18:07:50 schrieb 
> salutarydiacritica...@ruggedinbox.com:
>> There was a Sybil attack for 4 years. The Freenet 0day has been around 
>> for so long that LE contractors have built a kit around it.
> It’s not a zeroday. According to the article, they used a known
> vulnerability which takes time and careless behavior of the users to
> exploit.
Careless behaviour of the users? I didn't see any details? Clearly once
they identify a suspect they'll seize their computer and convict them
for possession. But the article implied they have software for
identifying anonymous users on the network from their downloads etc.
Which also fits with the leaked presentation.
> I just had a funding proposal turned down which included transport
> plugins and easy darknet introductions with one-time tokens. Florent
> is constantly working on keeping the crypto state-of-the-art. 
This is great, so is your fix to messaging! One real problem is we need
more volunteers, and we suck at retaining them.
> So, no,
> we’re not inactive. We’re just restricted in our available volunteer
> work time.
>
> Some of the time is spent on stuff others could do. For example work
> on fixing the wording on the website does not require any programming
> ability. We’re doing it, because there isn’t anyone else doing it. If
> you improve the wording and spend the time to really polish it, bit by
> bit (and send small and clean enough pull-requests that we can review
> it with little effort), then we have more time for other parts you
> might not be able to do.
>
> There are lots of things which can be done by dedicated people without
> programming skills. All of them are hard work, but they are very
> important to improve Freenet. For example the translators are pretty
> active and doing a great job making Freenet accessible to more people.
>
> So consider this an invitation.
>
> Best wishes,
> Arne
>
> --
> A man in the streets faces a knife.
> Two policemen are there it once. They raise a sign:
>
> “Illegal Scene! Noone may watch this!”
>
> The man gets robbed and stabbed and bleeds to death.
> The police had to hold the sign.
>
> …Welcome to Europe, citizen. Censorship is beautiful.
>
>( http://draketo.de/stichwort/censorship )
>



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Security quibbles was Re: Freenet Canary

[Matthew Toseland]

On 30/11/15 16:23, Arne Babenhauserheide wrote:
> Am Montag, 30. November 2015, 14:58:39 schrieb Matthew Toseland:
>> On 30/11/15 13:40, Arne Babenhauserheide wrote:
>>> Am Samstag, 28. November 2015, 14:52:23 schrieb Matthew Toseland:
>>>>  - a research project really.
>>> I don’t think people contributed or donated for that. Also, and I
>>> agree with earlier complaints about that, a research project does not
>>> need an auto-updater, content-filters, support for websites, forums, a
>>> full-fledged client-protocol, and so forth.
>> That depends on the nature of your research. I think we have benefited
>> considerably from having actual users testing the network. Even security
>> testing it, if they do it in such a way that we can make use of it (e.g.
>> Frost!).
>>
>> We signal this tentative status in the first-time wizard, in the logs,
>> in the FAQ, and in the version number being less than 1.0. We do not
>> provide any guarantees of security. If your life depends on Freenet's
>> security, either you're a fool, or you're in a really dark place.
> We’re saying “We SUCK” instead of saying for whom Freenet already works.
>> I do not approve of the hand-wavy simulations without source code school
>> of research. Lots of papers are not only not implemented but probably
>> not implementable. Such as PISCES. :(
> Research is that, unfinished, often only partially working, only
> applicable for the explicitly stated goal.
Freenet is also only partially working. But I agree it's a bit closer to
a working tool.
>>> Either we’re a research project, then we can strip out most of the
>>> features in Freenet, tell our users that we don’t care about them and
>>> let Freenet be replaced by the newest results of sensor network
>>> research, or we’re a project which aims at providing the technical
>>> foundation for freedom of the press, then we need to make Freenet easy
>>> to use und robust, and we need to know and communicate for whom it can
>>> already provide reasonable security.
>> Is there a group of people for whom it can provide reasonable security?
> If you want to write a blog on some specialized topic without
> connecting it to your own identity, the security is pretty good.
>
> If you want to communicate confidentially with your friends, you can
> do so over darknet connections.
>> What is your threat model? If it doesn't include at least one state, it
>> should: They usually are out to get you if you're doing anything at all
>> controversial, as we've seen fairly frequently even in western
>> countries!
> Let’s say you write slash fanfiction. The legal status of that is
> unclear, and you might not want your colleagues to know about the
> stories you write. Aragorn/Legolas anyone? If you do it on the regular
> internet (or, even worse, via Facebook), it’s only a matter of time
> until some profile pages connect your online-ID to your real ID. And
> then that information is out there.
>
> Assume that you like to write horror songs in the Star Trek
> universe. 20 years ago you would have published that under a Pseudonym
> in specialized journals, like Let’s Filk About.
If it upsets a corporation then they can find you, e.g. by asking the
police for a favour (yes they do do favours for corporations!). So this
is just about preventing your friends from discovering that you write
slash fiction? Better not add them as darknet peers then...
>> There are lots of reasons why it's hard to get darknet peers.
>> 1. Freenet is uncensorable. Most people find this offensive.
> We cannot fix that. We could reduce that, though, by only providing
> indexes in the default bookmarks which are created by anonymous people
> who don’t include offensive content.
That depends on the volunteers, as it always has.
>> 2. Freenet needs an always-on always-connected device, especially on
>> darknet. Most people don't have one, the costs are significant.
> This is not true. 2-12 hours runtime are completely OK. We would have
> this using mobile phones which run Freenet only while plugged into
> power and already mostly charged and connected over WiFi.
Darknet needs high uptime, or at least strongly correlated uptimes. 2
hours is definitely not enough - even with FOAF connections, you'll be
lucky to find enough peers.
>> 3. Darknet is slow.
> This is not true. 5-10 Darknet connections are enough to get good
> performance.
Right, and with FOAF we could have tens of peers. But you do need the 5+
friends to start with. That's hard.

I agree that this part is fixable and we must fix it: There are lots of
technical things we can do to make darknet work better, easier and faster.

But it's not the only or

[freenet-dev] Freenet Rebooted (without rewriting everything, pay for opennet)

[Matthew Toseland]

We have several major problems:
1. We need a major injection of cash.
2. We will not have a big connected darknet any time soon.
3. Opennet is not secure unless users pay for introduction.
4. Opennet is slow because of lowest common denominator load.

I propose: Freenet Rebooted.

A Kickstarter, but based on extending the current code, not a full
rewrite. A lot of it actually works reasonably well.

MAJOR CHANGES:
1. Darknet enhancements, but we recognise that we will need a large,
fast opennet backbone to connect the darknet pockets for the time being.
2. You can only run a Full Opennet Node if you have an Opennet Invite
and meet bandwidth/performance requirements.
3. Only Full Opennet Nodes route tunnels and/or high HTL traffic.
4. There may be further restrictions for security reasons, if so we will
ensure that an OI still gives performance benefits (even if you are not
routing traffic).
5. Opennet tunnels via ShadowWalker.
6. Better seednodes.
7. Most of the enhancements to other areas we've previously discussed.
8. Transient mode reintroduced, so opennet Freenet is still free as in
beer, and secure with tunnels. Great for uploading on the run! But
transient nodes don't route traffic/tunnels and get lower performance.
9. Investigate hardware partners and home-server UI issues. Long term we
need cheap, convenient hardware nodes, because we need uptime.

Initially we aim to raise $1M. Anyone who donates $100 gets an Opennet
Invite, so this is 10,000 users. Hardware nodes might be a good donor
perk too. In future we anticipate charging for OI's, but expect an
increasing proportion to be provably given to other worthwhile,
respected and relevant charities e.g. EFF: The price paid to become part
of the network infrastructure is mainly a deterrent to large scale
attacks, rather than a means of raising revenue.

Thoughts?

Obviously there is a risk of people running parallel networks for $99 or
whatever, or #freenet-refs style auto-darknet meshes. Some of them will
be scammers and we have a good web presence to start with; I don't think
we should worry about this.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Freenet Rebooted (without rewriting everything, pay for opennet)

[Matthew Toseland]

On 30/11/15 15:55, Matthew Toseland wrote:
> On 30/11/15 15:48, Ian Clarke wrote:
>> On Mon, Nov 30, 2015 at 9:29 AM, Matthew Toseland <mj...@cam.ac.uk> wrote:
>>> 3. Opennet is not secure unless users pay for introduction.
>> Who would they pay, and how would this be implemented in a decentralized
>> way?
> In the first instance they would pay *us* via a Kickstarter. In the long
> run there might be other options e.g. provable sacrifice of Bitcoins.
>> Given that we already have a shrinking userbase despite Freenet being free,
>> why do you think people will be willing to pay to use it?  Won't this
>> dramatically shrink our userbase to a tiny core of enthusiasts, which will
>> provide far less cover traffic and thus reduce security?
> No, because it won't happen unless we get at least 10,000 users/donors.
> Isn't there a "we won't charge you unless we raise the minimum" thing
> for Kickstarter?
>
> And people could still use Freenet in transient mode. In fact we might
> even allow an intermediate mode: Routes traffic but not tunnels and high
> priority traffic. 
Sorry, I mean high-HTL traffic here. To answer the rest of your
question, I believe a Kickstarter, some substantial software
improvements and publicity, and a substantial improvement in both
security and performance, could have significant benefits for our
userbase. Publicity always gets us new users. But please read the rest
of my previous response.
> But they wouldn't get good performance unless they
> become a core infrastructure opennet node.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Freenet Rebooted (without rewriting everything, pay for opennet)

[Matthew Toseland]

On 30/11/15 15:54, Bert Massop wrote:
> On Mon, Nov 30, 2015 at 4:29 PM, Matthew Toseland <mj...@cam.ac.uk> wrote:
>> The price paid to become part
>> of the network infrastructure is mainly a deterrent to large scale
>> attacks, rather than a means of raising revenue.
>>
>> Thoughts?
> I read this as "The price paid to become part of the network is mainly
> a deterrent to actual users, thinning the network until three-letter
> agencies with nine-figure budgets don't even need large-scale attacks
> to succeed."
I don't see why it would deter users. It might deter people from running
core nodes, but it might also help to get such users as I've tried to
explain, especially if it also improves performance and security and
gives us the funds to solve a lot of the remaining software problems.
> The problem lies in your assumptions:
>> 3. Opennet is not secure unless users pay for introduction.
> Money is easy for attackers (e.g. groups or organizations), and hard
> for individuals. I fail to see how Opennet would become safer with
> payments.
This is true of everything that money can buy. Which is everything, with
the possible (and slightly dubious) exception of social capital /
friends. A big global friend-to-friend darknet is a good long term
solution but the problem is how to get to that point. For the time
being, it is unlikely that one will grow organically - pockets of
darknet will hopefully grow organically, but it will take time for them
to get connected. Hence we need opennet for now, and we'd like it to
offer meaningful security. Even with tunnels, at a considerable cost in
development time and performance, the security provided is nowhere near
sufficient.

I'm simply trying to find a model that actually works and provides some
approximation of hope.
> That said, I'll be happy to fork the code and reinstate a free network
> (free as both libre and gratis) once tunnels are implemented.
> Insecure? Maybe. But still as secure as Opennet with payments, yet
> free.
No, it would be dramatically less secure than paid-for opennet, because
any attacker can cheaply add lots of opennet nodes. Which is exactly
what a contractor to the US police presumably does - this is not a
hypothetical attack any more.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Security quibbles was Re: Freenet Canary

[Matthew Toseland]

On 30/11/15 13:40, Arne Babenhauserheide wrote:
> Am Samstag, 28. November 2015, 14:52:23 schrieb Matthew Toseland:
>> But then Freenet was always just one piece of the puzzle
Okay, first, can we agree on this bit? "Freenet is one piece of the
puzzle". It doesn't provide a secure operating system to run it on, good
disk encryption, or an internet backbone!
>>  - a research project really.
> I don’t think people contributed or donated for that. Also, and I
> agree with earlier complaints about that, a research project does not
> need an auto-updater, content-filters, support for websites, forums, a
> full-fledged client-protocol, and so forth.
That depends on the nature of your research. I think we have benefited
considerably from having actual users testing the network. Even security
testing it, if they do it in such a way that we can make use of it (e.g.
Frost!).

We signal this tentative status in the first-time wizard, in the logs,
in the FAQ, and in the version number being less than 1.0. We do not
provide any guarantees of security. If your life depends on Freenet's
security, either you're a fool, or you're in a really dark place.

I do not approve of the hand-wavy simulations without source code school
of research. Lots of papers are not only not implemented but probably
not implementable. Such as PISCES. :(
> I say it so clearly, because I think that calling Freenet a research
> project is harming the project.
>
> Either we’re a research project, then we can strip out most of the
> features in Freenet, tell our users that we don’t care about them and
> let Freenet be replaced by the newest results of sensor network
> research, or we’re a project which aims at providing the technical
> foundation for freedom of the press, then we need to make Freenet easy
> to use und robust, and we need to know and communicate for whom it can
> already provide reasonable security.
Is there a group of people for whom it can provide reasonable security?
What is your threat model? If it doesn't include at least one state, it
should: They usually are out to get you if you're doing anything at all
controversial, as we've seen fairly frequently even in western
countries! We don't provide protection against petty malware - that's
somebody else's problem. And we don't provide a warm fuzzy feeling when
accessing Facebook either, because we don't proxy!
>> The problem with darknet is that Freenet is not socially acceptable
>> and is technically challenging (it effectively requires a dedicated
>> always on system), and currently is slow and inconvenient. There are
>> lots of relatively easy ways we can improve the last point. The
>> first is hard.
> Technically challenging can be solved by making it easier to join —
> all that is documented in the bugtracker. For *more* socially
> acceptable we need more actively spidering indizes which only include
> what the creator deems acceptable (nerdageddon does not count since it
> does not spider itself). However, Freenet as “if you want to avoid
> censorship, no one must be able to censor” is pretty acceptable.
There are lots of reasons why it's hard to get darknet peers.
1. Freenet is uncensorable. Most people find this offensive.
2. Freenet needs an always-on always-connected device, especially on
darknet. Most people don't have one, the costs are significant.
3. Darknet is slow.

We can fix #3. Fixing #2 is hard; even if Freenet is popular enough that
somebody can sell boxes for it, it will still slow down your internet.
Fixing #1 is impossible.

Also, #2 is likely to become more and more of a problem over time, until
civilisation collapses under the weight of the housing market bubble. :)
Unfortunately this is not exclusively a UK problem. More seriously the
technical trend to everything mobile and cloud dependent is driven at
least in part by wider social trends.
>> I have no idea what you mean by "node pinning".
> I guess it could be either reconnecting through old opennet peers, or
> reusing the same seednode. Both would make it harder to start new
> attacks against opennet users (as in “it would make it slower”).
Marginally. Old opennet peer connections don't often work because when
you want to reconnect your old peer probably doesn't - even if it hasn't
changed its IP address.

What would make a difference would be centralised restrictions on
bootstrapping new identities. But this would break a lot of the
decentralised pseudo-solutions, it would be a lot of work, and doesn't
provide meaningful protection unless we can charge real money for
joining opennet - or unless it's really hard for attackers to get widely
distributed IP addresses. Is it? The attack on Tor last year used a
single prefix. But then that was a university. Again it depends on your
threat model.

There may be distributed solutions, provided the attacker hasn't already
t

Re: [freenet-dev] Freenet Rebooted (without rewriting everything, pay for opennet)

[Matthew Toseland]

On 30/11/15 15:44, Florent Daigniere wrote:
> On Mon, 2015-11-30 at 15:29 +0000, Matthew Toseland wrote:
>> We have several major problems:
>> 1. We need a major injection of cash.
>> 2. We will not have a big connected darknet any time soon.
>> 3. Opennet is not secure unless users pay for introduction.
>> 4. Opennet is slow because of lowest common denominator load.
>>
>> I propose: Freenet Rebooted.
>>
>> A Kickstarter, but based on extending the current code, not a full
>> rewrite. A lot of it actually works reasonably well.
>>
>> MAJOR CHANGES:
>> 1. Darknet enhancements, but we recognise that we will need a large,
>> fast opennet backbone to connect the darknet pockets for the time
>> being.
>> 2. You can only run a Full Opennet Node if you have an Opennet Invite
>> and meet bandwidth/performance requirements.
>> 3. Only Full Opennet Nodes route tunnels and/or high HTL traffic.
>> 4. There may be further restrictions for security reasons, if so we
>> will
>> ensure that an OI still gives performance benefits (even if you are
>> not
>> routing traffic).
>> 5. Opennet tunnels via ShadowWalker.
>> 6. Better seednodes.
>> 7. Most of the enhancements to other areas we've previously
>> discussed.
>> 8. Transient mode reintroduced, so opennet Freenet is still free as
>> in
>> beer, and secure with tunnels. Great for uploading on the run! But
>> transient nodes don't route traffic/tunnels and get lower
>> performance.
>> 9. Investigate hardware partners and home-server UI issues. Long term
>> we
>> need cheap, convenient hardware nodes, because we need uptime.
>>
>> Initially we aim to raise $1M. Anyone who donates $100 gets an
>> Opennet
>> Invite, so this is 10,000 users. Hardware nodes might be a good donor
>> perk too. In future we anticipate charging for OI's, but expect an
>> increasing proportion to be provably given to other worthwhile,
>> respected and relevant charities e.g. EFF: The price paid to become
>> part
>> of the network infrastructure is mainly a deterrent to large scale
>> attacks, rather than a means of raising revenue.
>>
>> Thoughts?
> This assumes that Sybil is the only attack against opennet... which is
> clearly misleading. Sybil is the obvious, cheap attack; the nastier
> ones are all those related to "open" topologies and protocols:
> partitioning attacks, correlation attacks, ... for which we don't have
> solutions either.
>
> Florent
You mean for denial of service? Or for identifying users?

If we have scarcity then we can use ShadowWalker tunnels to prevent
identifying users (on arguably naive but quantified assumptions - it
works up to 20%), although granted there may be possibilities for active
attacks. Direct DoS attacks against opennet announcement are also a lot
easier to deal with.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Freenet Rebooted (without rewriting everything, pay for opennet)

[Matthew Toseland]

On 30/11/15 16:09, Bert Massop wrote:
> On Mon, Nov 30, 2015 at 5:08 PM, Michael Grube  
> wrote:
>> This is true of everything that money can buy. Which is everything, with
>>> the possible (and slightly dubious) exception of social capital /
>>> friends. A big global friend-to-friend darknet is a good long term
>>> solution but the problem is how to get to that point.
>> Please, please PLEASE don't murder me for suggesting this, but what if we
>> used social media to bootstrap network connectivity?
> How is that different from Darknet?
There's a lot we can do to make it easier to connect to your friends on
darknet and make darknet more efficient. Advertising your connectivity
on Facebook may be part of that, subject to the whims of massive
corporations we have no influence over. I believe somebody wrote an app
a long time ago. However as I've explained there are a lot of other
barriers, notably politics (will take time to change) and the need to
run a node with reasonable uptime. IMHO it is reasonable to assume that
a connected global darknet is some way off: Most of our users don't know
anyone else who uses Freenet, and certainly don't know the 5 or so
friends they'd need for acceptable performance/security (even with
friend-of-a-friend connections).



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Freenet Rebooted (without rewriting everything, pay for opennet)

[Matthew Toseland]

On 30/11/15 19:17, Arne Babenhauserheide wrote:
> Am Montag, 30. November 2015, 15:55:13 schrieb Matthew Toseland:
>> Not if we jettison the slower opennet nodes, which is also part of the
>> proposal. A lot of our performance issues are actually because we target
>> an outdated lowest common denominator.
> Sadly this isn’t true: Most of our users have low bandwidth, as shown
> by the stats from Steve:
>
> http://127.0.0.1:/USK@pxtehd-TmfJwyNUAW2Clk4pwv7Nshyg21NNfXcqzFv4,LTjcTWqvsq3ju6pMGe9Cqb3scvQgECG81hRdgj5WO4s,AQACAAE/statistics/1048/plot_peer_count.png
>
> Best wishes,
> Arne
How much of this is due to default settings where it didn't manage to
autodetect via UPnP? How much to users not making informed choices?



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Freenet Rebooted (without rewriting everything, pay for opennet)

[Matthew Toseland]

On 30/11/15 18:11, xor wrote:
> This mail is split in 2 parts:
> 1. A summary of part 2, which also includes stuff which is not in part 2.
> 2. A copy of a previous reply of mine to a similar proposal. Most of what's 
> said there applies to this as well.
>
>
> Part 1 follows:
>
> I think we shouldn't randomly change our strategy from what it was to 
> something which invalidates all its work by postponing to finish whats half-
> finished into the far future: You're proposing yet another half a decade of 
> rewriting parts of fred over and over again instead of finally giving people 
> new client apps, which was the goal of my work but isn't finished yet (sorry, 
> it is a complex task :|)... 

Client apps are important. If we can get funding for them then it'd be
great to have somebody working full time on them.

> Because let's be honest: If we now installed a fred of 7 years ago, it would 
> by default still ship the same core applications as today: I joined back then 
> to get the WoT-stuff finished to the point where we can enable it by default.
> So I think we spent more than enough years on only providing fred work.
> (Yes, it sucks that I still haven't finished WoT+Freetalk, and I'm ashamed of 
> that, but I've been a volunteer and thus had limited time to contribute for 5 
> of those years, and Freetalk+WoT are major projects. I think they're over 40 
> 000 lines of code already...)
And for much of that time you've been a paid developer and still made
limited progress. These things are hard. We need enough funding that we
can improve several different areas of Freenet simultaneously IMHO.
> This needs to change, and it won't change if we only acquire funding for 
> minor 
> fred features. We need new things such as forums, filesharing, social 
> networking, mail etc. bundled *and* enabled by default; not new minor fred 
> features.
>
> Yes, your features are major security enhancements, not minor ones. But to 
> the 
> users, a feature is something which "does" something for the user. Security 
> is 
> merely self-servicing, not serving the user. They wouldn't recognize it as a 
> major new feature.
I include the rest under item 7. I'm certainly not arguing that we
should only ask for money for improving Fred's security! I do think it
should be part of what we are looking for.

One advantage of my proposal is if it worked it would generate
sufficient funds to hire several developers for a year. I believe we
estimate $100K/year/dev including costs, so in fact it would be 10
person-years. IMHO that's the sort of scale that we should be aiming at.
I appreciate that for funding body applications we may need to start lower.
> Further, I think people will not pay for Opennet. We cannot call something 
> "Free"net if it costs money. We'd be ridiculed for that.
English sucks (libre vs gratis, free software versus free spyware
services). Most languages don't have this problem.

However, as I have repeatedly explained, people only need to pay if they
want to run a core opennet node. Non-core (possibly transient) opennet
nodes can run, and so can darknet nodes. And they will have better
security because they can tunnel through the core nodes.
> With regards to hardware development: We haven't even ported to Android yet, 
> which is > 1 billion devices. Before we re-invent the wheel by custom 
> embedded 
> hardware, we should maybe first port to the standard embedded hardware 
> everyone uses :) I would support doing that, but not as a mandatory goal of 
> fundraising please. It should be something we do if we get more money than we 
> need. It is a nice goal though: An operating system with 1/8 of all humans 
> using it is not something which can be ignored.
An operating system used exclusively on mobile devices which are
specifically designed to store everything in the cloud. P2P simply
doesn't work on mobile, because of battery life, even if we ignore all
the other problems (such as carriers blocking it).

There are good reasons to have some Freenet code on Android however. We
have an app for node reference exchange, and it would be a good idea to
extend it to interface to a fixed node. And porting to Android is
relatively easy because the non-GUI parts are very similar to Java.
> I'm thankful for your proposal, and I feel sorry for having to give this 
> strong criticism, but I fear it's necessary: We have only asked 3 entities 
> for 
> money (see the Wiki page [1]). Just because we have temporarily run out of 
> money because we *did not ask for money* doesn't mean we should randomly 
> throw 
> away parts of our work and do stuff which we wouldn't have considered a good 
> idea before. Before we have bothered to try to ask lets say 50, there is no 
> reason to change what we planned to develop anyway.

Funding agencies will only give us money if we have a track record. So
we will have to start small or get very lucky. And that limits the range
of bodies we can apply to. And so on. It's worth trying, 

Re: [freenet-dev] Freenet Rebooted (without rewriting everything, pay for opennet)

[Matthew Toseland]

On 30/11/15 19:34, Arne Babenhauserheide wrote:
> Am Montag, 30. November 2015, 15:29:25 schrieb Matthew Toseland:
>> 3. Opennet is not secure unless users pay for introduction.
> Even regular E-Mail providers, G+ and Facebook did not find a way to
> get a significant number of users to pay — for a service which is
> clearly essential for todays communication. Why do you think people
> would pay for Freenet?

They don't need them to. They harvest all their data and sell it to
advertisers etc. If you're not paying for the product you are the
product. If you are paying for the product we're probably selling your
personal data anyway!

In the first case, people contribute enormous sums to Kickstarter
projects. Many of them are very successful in fundraising. The
Kickstarter itself could raise significant funds - but only if we do it
right. In particular, you need Stuff to give to donors. IMHO such a
campaign would bring us significant publicity (a good thing in itself),
and would have some chance of succeeding in delivering significant funds
- probably a greater chance than if we just posted a pitch with some
vague goals and goodies that they can get anyway from our cafepress store.

In the second case, people don't pay for USING Freenet. They pay for the
rights to be part of the core opennet infrastructure. They pay to keep
spammers out and above all for fast performance.
> People pay for VPNs because VPNs promise them faster, anonymous
> copyright infringement — I’ve seen the ads on torrent sites. The
> Freenet Project cannot promise that without encouraging copyright
> infringement — which we don’t.
>
> And our communication sucks — with this thread a perfect example of
> why it sucks. As much as I’m irked by the often toxic behavior of
> niqnaq: this is something he’s right on. We have an existing
> userbase. These users are our greatest asset. We might not like all of
> them, but at the same time there are many awesome people using
> Freenet. We’re neglecting them. We’re not doing the easy fixes.

We don't have the resources. That's half of the problem. The other half
is that opennet is irredeemably broken and being actively exploited. We
can fix both problems simultaneously.
> Instead we’re saying “let’s make you pay to keep using Freenet”.
>
> We need more people running Darknet, so why don’t we think of a way to
> secure Opennet via Darknet? Darknet connections are the only thing at
> which attackers don’t win trivially.

Because 90% of the network at any given time has NO darknet connections,
and that's likely to remain the case for a long time. Which means we
can't use the social topology to secure anything.
> And it might turn out that for funding, this pull request is the most
> important of them all: https://github.com/freenet/website/pull/28

As I said, improving the website is important.

But getting some major publicity is important too. And even with that
it's doubtful that we can raise significant sums.
>
> Best wishes,
> Arne
> --
> Celebrate with ye beauty and gather yer friends for a Pirate Party!
> → http://1w6.org/english/flyerbook-rules#pirate-party ←



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Freenet Rebooted (without rewriting everything, pay for opennet)

[Matthew Toseland]

On 30/11/15 19:21, Florent Daigniere wrote:
> On Mon, 2015-11-30 at 15:50 +0000, Matthew Toseland wrote:
>> On 30/11/15 15:44, Florent Daigniere wrote:
>>> On Mon, 2015-11-30 at 15:29 +, Matthew Toseland wrote:
>>>> Thoughts?
>>> This assumes that Sybil is the only attack against opennet... which
>>> is
>>> clearly misleading. Sybil is the obvious, cheap attack; the nastier
>>> ones are all those related to "open" topologies and protocols:
>>> partitioning attacks, correlation attacks, ... for which we don't
>>> have
>>> solutions either.
>>>
>>> Florent
>> You mean for denial of service? Or for identifying users?
>>
>> If we have scarcity then we can use ShadowWalker tunnels to prevent
>> identifying users (on arguably naive but quantified assumptions - it
>> works up to 20%), although granted there may be possibilities for
>> active
>> attacks. Direct DoS attacks against opennet announcement are also a
>> lot
>> easier to deal with.
> Yes, active attacks is what I'm talking about here; If you knock off
> parts of the network (or make them unreachable for your target) you're
> doing a partitioning attack... and tunnels don't help you (because even
> if you manage to detect it you won't accept hard-fail - the secure
> behaviour).

Not in every case. E.g. a seednode attempting to capture new announcees
is a classic partition attack, but it's fixable by using other seeds and
some consensus protocols etc. For which making identity generation
expensive is very useful.
> This is a problem that doesn't have any real-solution, just bad trade-
> offs. For the sake of giving an example: Bitcoin has the same problem.
>
> Florent
> PS: correlation attacks are way easier on a partitioned network for
> obvious reasons



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: [freenet-dev] Freenet Rebooted (without rewriting everything, pay for opennet)

[Matthew Toseland]

On 30/11/15 19:58, Matthew Toseland wrote:
> On 30/11/15 19:34, Arne Babenhauserheide wrote:
>> Am Montag, 30. November 2015, 15:29:25 schrieb Matthew Toseland:
>>> 3. Opennet is not secure unless users pay for introduction.
>> Even regular E-Mail providers, G+ and Facebook did not find a way to
>> get a significant number of users to pay — for a service which is
>> clearly essential for todays communication. Why do you think people
>> would pay for Freenet?
> They don't need them to. They harvest all their data and sell it to
> advertisers etc. If you're not paying for the product you are the
> product. If you are paying for the product we're probably selling your
> personal data anyway!

Also, a significant minority do pay for email still, e.g. Fastmail.



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

[freenet-dev] Security quibbles was Re: Freenet Canary

[Matthew Toseland]

On 28/11/15 10:34, Florent Daigniere wrote:
> On Fri, 2015-11-27 at 18:07 -0500,
> salutarydiacritica...@ruggedinbox.com wrote:
>> Let's talk about the bad news and the way forward.
>>
>> There was a Sybil attack for 4 years. The Freenet 0day has been
>> around 
>> for so long that LE contractors have built a kit around it. Forget 
>> global adversaries or nation states, its so bad that local police 
>> stations with shoelace budgets can attack the network. My guess,
>> Frost's 
>> spam issues make traffic tagging easy.
> What's your source on this?
> Do you understand what Sybil is about?
> What makes it qualify as 0day (it's not documented on https://wiki.free
> netproject.org/Opennet_attacks ?)
Agreed, do you have any more information than the two sources we've seen
- one state-level newspaper article and a very vague leaked law
enforcement presentation?

As far as I can see, MAST-style attacks don't work, so we are talking
about network-scale Sybil here. There is a cost to this; it's not a huge
cost because the network is small and computing power and bandwidth is
cheap in bulk, but it's not something you could do as an individual
without some funding. Tools vendors presumably amortise the cost by
selling to many users (many different police stations), or big users
(NSA and agencies of other governments).
>> Before anyone gets started: "But, but.. Tor was also attacked!"
>>
>> Yes, but responses are very different from what's going on here. They
>> immediately fixed the hole and evicted the Sybil nodes. They are 
>> implementing code that will make future attempts much more difficult.
>> They did not add a line to the FAQ that said "shit happens" and shrug
>> their shoulders.
>>
>> More on what you can do later.
An attack on Tor was detected in practice, and has been mitigated
somewhat, however there are also cheap (and harder to detect) published
attacks, e.g. the one published last year that could be conducted by
anyone controlling two Autonomous Systems. So it's the tip of the
iceberg IMHO.

IMHO all of the current anonymity networks are vulnerable both to state
level attackers and to well-funded research projects.

Having said that, *any* practical system is vulnerable to NSA-level
attackers, and that's likely to remain true for the foreseeable future.
For example, the Investigatory Powers Bill (re)introduces a power for
"bulk equipment interference", in other words mass hacking. In other
words they are welcome and able to hack a majority of the network, if
they can't get the information more cheaply some other way. And we know
they introduce vulnerabilities when they need them. The war in
cyberspace has been won conclusively. But then Freenet was always just
one piece of the puzzle - a research project really. Historically we
know that lots of the anonymous email remailers have been compromised.

Can we defend against slightly weaker adversaries? Maybe - but even Tor
has cheap attacks. Does it make any difference? That's not clear either:
Intelligence agencies do pass on information when they feel it won't
jeopardise their operations and methods - and they may be more willing
to do that now that everyone knows they can break everything. Are the
Chinese intelligence agencies hunting down political dissidents much
less capable than the NSA? I wouldn't bet on it.
>> "Securing Opennet is impossible, go Darknet mode or shut up!"
>>
>> Taking your defeatist attitude to conclusion we can say anonymous 
>> communication is a very hard problem so no point trying. Let's all
>> use 
>> the surveiled network and take our chance?
>>
>> Of course not. You can raise costs to make it hard for any attack and
>> other projects proved it.
>>
>> I understand you need more resources to turn things round. That can 
>> change, but carrying a defeatist attitude can never improve anything.
>>
>> Going Darknet mode only is not a real fix.
> Can you define what is the attack and its real-fix then?
Agreed, opennet is really, really hard to fix. Unless we can limit Sybil
by requiring an up front cash payment to join the network. And even then
it becomes vulnerable because it's likely more centralised.
>>  Its like suggesting to people 
>> to limit internet access only to their LAN to stay safe. The value of
>> the network becomes diminished. Darknet mode also exposes people's 
>> social network to anyone watching enough of the internet. Its a 
>> dangerous liability.
> The idea is that you're already exposing your social network regardless
> of whether you are using darknet or not... so on the contrary, you do
> *not* leak any information by connecting to your real-life social
> contacts.
>
> What needs changing is the terminology; "friends" might not be the
> adequate word to describe darknet peers.
Yep. They (the cops, the agencies, the corporations) already have your
address book, deal with it.

And darknet is not fundamentally limiting. The problem with darknet is
that Freenet is not socially acceptable and is 

[freenet-dev] Update FAQ entry

[Matthew Toseland]

Q: Has anyone got into trouble for their anonymous activities while
running Freenet? A: US law enforcement can identify anonymous users of
Freenet[1] and Tor[2]. It is reasonable to assume that other governments
have access to the same technology, which is provided by private
contractors. If you are concerned about governments, you should use
darknet mode, and bear in mind that no current anonymity technology
provides perfect protection.

And possibly:
While we applaud the US police's apparent success in apprehending
suspects allegedly sharing child abuse images, many governments
persecute and prosecute political dissidents for legitimate speech
published online. Therefore we hope to further improve Freenet's
security in future, to protect its legitimate users. We do not have the
ability to easily identify users of Freenet downloading specific files.

[1] =
http://www.thedickinsonpress.com/news/north-dakota/3885239-predators-police-online-struggle
[2] =
http://motherboard.vice.com/read/court-docs-show-a-university-helped-fbi-bust-silk-road-2-child-porn-suspects

Also reference this in the attacks section.
I will update the wiki.

Sorry I haven't posted a pull request; I'm nowhere near synchronized
with the website updates...



signature.asc
Description: OpenPGP digital signature
___
Devl mailing list
Devl@freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl