Hi,
I did alternate evaluations of the poll to test the robustness of the results. These evaluations build on the work by xor and add several diagnostics, but do not exclude values, except where they investigate a specific group of voters. ## Executive Summary The most robust result of the poll is: we should definitely do these five tasks: - Darknet invitation bundles (requires single use references) - Improve FProxy CSS3 support to allow better Freesite UI - Friend requests, like in Facebook - Short node references - Keepalive All four groups I identified in the respondents to the poll (developers, fms-users, all-but-xor, all) agree that these tasks should be among the tasks financed with the money we got from DuckDuckGo. Two more should be added (explained in the next subsection): - Finishing the first iteration of Web of Trust speed fixes (1) - Fixing the installers (2) These 7 tasks together are already estimated as 17 person-weeks, which would leave us 15% buffer for unforseen problems. These tasks differ from the result by xor, because they are a consensus between different evaluation methods. They are number 1, 9, 12, 16 and 25 in the evaluation by xor. ### Why the two additional tasks (1) and (2) should be added (1) If either (a) we ask not only users but also developers, or (b) we only ask users but use arithmetic mean for the cost estimate, finishing the first iteration of Web of Trust speed fixes is on the list. This stays true when excluding all input from xor, so there is no indication of poll-skewing on his part. (2) Fixing the installers does not make it to the main tasks, because in a Condorcet method nothing can win which is put into the last position by the majority of people. So I would add fixing the installers to fix a mistake we made early on in the process. There are further details in the remainder of my mail, but this is the most robust result I could get, and it should not get lost in a wall of text. ## The method x...@freenetproject.org writes: > here's my proposal on how to evaluate the poll results: > https://git.io/v1ah4 I added to this: - a condorcet evaluation method (which ironically was only possible because Ian did *not* follow my suggestion of distributing only 2 points per task because using condorcet here relies on finegrained voting to provide a ranking instead of a value ☺). - using geometric mean for the values - using geometric mean for the cost - evaluating the votes for 4 different groups: full-data, no-fms, only-users (essentially that’s "only fms" ☺) and without-employees. Using the different evaluation methods (arthmetic mean, condorcet, geometric mean and median) I calculated robust top 20 lists. If a task appears in such a list, it is in the top 20 for each of the evaluation methods. To get the most robust result, I took the intersection of all these top 20 lists. The following tasks are in the top 20, regardless of the user group or the evaluation method, so they are the consensous result. ## Most robust result The most robust result of the poll is: we should definitely do these five tasks: - Darknet invitation bundles - Improve FProxy CSS3 support to allow better Freesite UI - Friend requests, like in Facebook - Short node references - Keepalive All four groups I identified in the respondents to the poll agree that these tasks should be among the tasks financed with the money we got From DuckDuckGo. (web of trust first iteration is only excluded due to the top 20 when excluding value estimates fro mdevelopers while using geometric mean for the cost estimate) ## Top 20 for different groups The following are the intersection of the top 20 when using aritmetic mean or geometric mean for the cost estimate. If something appears here, it should be robust against outliers in the cost estimate. Each of the top 20 lists included the following three entries, so I won’t repeat them there: - Darknet invitation bundles (requires single use references) - Improve FProxy CSS3 support to allow better Freesite UI - Friend requests, like in Facebook - Short node references - Keepalive ### All voters - Web of Trust first iteration - Single use node references - Document how Freenet works with infographic - Friend-of-a-friend connection suggestions - Legal docs, link to Tor docs - Language: working on how to install Freenet - Migrate to gradle build tool - Do not reject Freemail messages for unknown recipients ### Developers ("without-fms") - Web of Trust first iteration - Single use node references - Document how Freenet works with infographic - Friend-of-a-friend connection suggestions - Language: working on how to install Freenet - Darknet chat improvements - Blogging enabled by default ### Only users (essentially just FMS) - Legal docs, link to Tor docs - check re-adding native crypto for speedup - FCP calls for Freemail setup and settings - Do not reject Freemail messages for unknown recipients - Blogging enabled by default - Plugin data encryption ### All not employed here (to remove worries of skewing the poll) - Web of Trust first iteration - Single use node references - Document how Freenet works with infographic - Language: working on how to install Freenet - Migrate to gradle build tool - Legal docs, link to Tor docs - Do not reject Freemail messages for unknown recipients ## Complete results merged Merging these lists to ensure that all voices are heard gives the following list of tasks. I added the arithmetic (a) and geometric (g) mean of the cost estimates in fulltime person-weeks: - Consensus - Darknet invitation bundles (a: 1.125, g: 1.0) - Single use node references (a: 2.4, g: 2.0) (required for implementation) - Improve FProxy CSS3 support to allow better Freesite UI (a: 1.75, g: 1.681) - Friend requests, like in Facebook (a: 1.375, g: 2.0) - Short node references (a: 0.8, g: 0.757) - Keepalive (a: 1.75, g: 1.681) - Multiple groups - Web of Trust first iteration (a: 4.1, g: 2.861) - Blogging enabled by default (a: 1.7, g: 1.319) - Do not reject Freemail messages for unknown recipients (a: 1.0, g: 0.870) - Document how Freenet works with infographic (a: 1.625, g: 1.414) - Friend-of-a-friend connection suggestions (a: 2.25, g: 2.0) - Language: working on how to install Freenet (a: 0.9, g: 0.870) - Legal docs, link to Tor docs (a: 0.5, g: 0.5) - Migrate to gradle build tool (a: 1.166, g: 1.0) - Only developers - Darknet chat improvements (a: 2.2, g: 2.0) - Only users - FCP calls for Freemail setup and settings (a: 0.6, g: 0.574) - Plugin data encryption (a: 1.8, g: 1.319) - check re-adding native crypto for speedup (a: 1.166, g: 1.0) These add up to a total cost of 25 to 28 person-weeks, depending on whether I use geometric mean or arithmetic mean to average the cost estimates. However keep in mind that this is an optimistic estimate. The consensus items alone are estimated as about 10 person-weeks (Darknet invitation bundles need single use node references). When adding Web of Trust first iteration to this, we already reach three quarter of our resources — and it is likely a good idea to keep 25% buffer (better more…). ## Data and scripts I uploaded the individual top 20 lists, so you can see the differences for yourself (the links need a running Freenet fproxy on port 8888): http://127.0.0.1:8888/CHK@DDUi0KlJg-y8e58bifGthoXW0HaRB8JJBK3eDq~mNcs,i0Y65Q18mSq6SPE0Yg9XaqFryRbeT58YN27-P2~odWk,AAMC--8/eval-full-data-top-N-arithmetic.html http://127.0.0.1:8888/CHK@XBKhExvgbL37v212C~EEwNY9hUtaAuQD8t2vFS170sc,ZxBUVsJ-UDTdpQ8tQ1Y3n02fMk7ISEHdWmOtNCIfXTg,AAMC--8/eval-full-data-top-N-geometric.html http://127.0.0.1:8888/CHK@XKOREJbiJ5hF8qbomIvapa6pzHeQom0AcqL9EQXGAYY,nLcWD2IQvoNRdb5SynNmcdor4RWjr7YCXN8dVqfT4vU,AAMC--8/eval-no-fms-top-N-arithmetic.html http://127.0.0.1:8888/CHK@2uNcrN9embwRRQszrOu~nxYBnWVB-VPS7detOwjojGY,7TN34uHaYJVqkz7QUqQ3dQ~6Z98rajeRqa3RbOxUfmY,AAMC--8/eval-no-fms-top-N-geometric.html http://127.0.0.1:8888/CHK@y0PYGQLZgGWJach2vZJ4qsAehkiuWvBZ7anMWJGrYek,S9teGATzx4eWDEUaqU52JDivf3tfn8msRr3zAgTjQ-w,AAMC--8/eval-only-users-top-N-arithmetic.html http://127.0.0.1:8888/CHK@cfRoW4ZJEBF2cMhoRac54zC6Kir3R4JrwirX1W9cwMA,T84N721BebHp17s4cDxUxbKOe2E897aGDFMsh94uGRk,AAMC--8/eval-only-users-top-N-geometric.html http://127.0.0.1:8888/CHK@9vlMTx~tYVDtwgR0PfyG0krAAMBxxFeQL4csWe0set8,bMkWfJy1ZG~BPgAWjPNYyDZucISnClAJCFAdmEkMdkI,AAMC--8/eval-without-employees-top-N-arithmetic.html http://127.0.0.1:8888/CHK@5o46iBgTpTvV~aGBxGZITNhh~5EqasAuclrkSVbCC6Q,fzYvfjtjSZr8AcPNzgvG0XRsFG6xybSKX5C~piwc-mY,AAMC--8/eval-without-employees-top-N-geometric.html Data used for each of the groups and the evaluation script are available in the following zip-files (these include the top 20 and the full ranking): http://127.0.0.1:8888/CHK@Vfp8YwK0ThGPQ3Fi0TylxsfaxLrxwZtUK-hx2NhqXok,UPmek4ZQ44qZ~4Iji7MSyItejD6MQvmGpgp-s~KEoRU,AAMC--8/eval-full-data.zip http://127.0.0.1:8888/CHK@XCR38geH4T~FoKXuAGItxKgjTxLhwZjOda9ANmhxRF0,tUZBMRW~a0lnp2eDyM4oQRAHJtVAi0nYjh5UmJs7YdI,AAMC--8/eval-no-fms.zip http://127.0.0.1:8888/CHK@DsBhI6evcEBrW4B31Ch670ARQ8sUyjWwryPZA-56VKo,vLDi-~qqlI8Q-nJBx4Fxin78p0K4w0bYzTX1NuPgZJw,AAMC--8/eval-only-users.zip http://127.0.0.1:8888/CHK@NKuZ~eu4oiWYU1aLobDUKOgoXEqtjeYvbe7YFULIp6c,rwdokY2dUsWAjqGVzmrBSCsqa-L2vw0OuXG~Rq0ZNMU,AAMC--8/eval-without-employees.zip To redo the full evaluation, download the zip files, unpack them and execute the following command (needs Python3 and numpy): for i in eval-*; do (cd $i; ./evaluation.py; mv ranks.html ranks-geometric-mean.html; mv top_N.html top_N-geometric-mean.html; ./evaluation.py --arithmetic-mean-cost; mv ranks.html ranks-arithmetic-mean.html; mv top_N.html top_N-arithmetic-mean.html; cd -) & done ## Summary I extracted a robust subset of top 20. Five items are in the top 20 whatever evaluation I do: - Darknet invitation bundles (needs single use node references with token) - Improve FProxy CSS3 support to allow better Freesite UI - Friend requests, like in Facebook - Short node references - Keepalive In addition, all evaluations but one (only geometric mean cost when counting only votes from FMS) add - Web of Trust first iteration Fixing the installers does not make it here, because in a Condorcet method nothing can win which is put into the last position by the majority of people. So I would add fixing the installers to fix a mistake we made early on in the process. - Fixing the installers These 7 tasks are already estimated as 17 person-weeks. Thank you for reading this far, and happy hacking! Best wishes, Arne PS: I saw this appear in one top 20: Icicle or similar for setting up Darknet connections via smartphones (see https://f-droid.org/repository/browse/?fdid=co.loubo.icicle) -- Unpolitisch sein heißt politisch sein ohne es zu merken
signature.asc
Description: PGP signature
_______________________________________________ Devl mailing list Devl@freenetproject.org https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
