Bug#773663: Updated patch for popcon tor support
Bill Allombert: > On Thu, Aug 09, 2018 at 07:55:00AM +, Niels Thykier wrote: >> [...] >> >> Excellent, I am looking forward to it. :) Do you have an estimate of >> when 1.67 will be in unstable? > > Now. Sorry I prepared the package one month ago, tested it (which is > slow) and then forgot to upload it. Thanks for the remainder! > > Cheers, > Bill. > Thanks, :) ~Niels
Bug#773663: Updated patch for popcon tor support
On Thu, Aug 09, 2018 at 07:55:00AM +, Niels Thykier wrote: > On Sun, 20 May 2018 15:50:16 +0200 Bill Allombert > wrote: > > [...] > > > > Thanks! (and apologies to Tim to have missed the opportunuity to check Tim > > original popularity-contest.gpg report). > > > > I have received it just one (via http) on Sun May 20 08:02:01 UTC 2018 > > (however popcon cron.daily has a mechanism to prevent multiple submissions > > in the same week). > > > > Great, thanks for verifying. :) > > > > I have attached the .gpg file. I used > > > the ".new" file because it seems that the popularity-contest.new.gpg is > > > not renamed to popularity-contest.gpg any longer. > > > > Yes this was changed in 1.65 (maybe by mistake) to fix bug 850568. > > > > The only drawback I see is that tor has a number of dependencies that > > will always appear as 'recently used' in popcon report. However if tor > > is used regularly anyway this is not an issue. > > > > I will apply the patch for popularity-contest 1.67 > > > > Cheers, > > -- > > Bill. > > > > Imagine a large red swirl here. > > > > > > Excellent, I am looking forward to it. :) Do you have an estimate of > when 1.67 will be in unstable? Now. Sorry I prepared the package one month ago, tested it (which is slow) and then forgot to upload it. Thanks for the remainder! Cheers, Bill.
Bug#773663: Updated patch for popcon tor support
On Sun, 20 May 2018 15:50:16 +0200 Bill Allombert wrote: > [...] > > Thanks! (and apologies to Tim to have missed the opportunuity to check Tim > original popularity-contest.gpg report). > > I have received it just one (via http) on Sun May 20 08:02:01 UTC 2018 > (however popcon cron.daily has a mechanism to prevent multiple submissions > in the same week). > Great, thanks for verifying. :) > > I have attached the .gpg file. I used > > the ".new" file because it seems that the popularity-contest.new.gpg is > > not renamed to popularity-contest.gpg any longer. > > Yes this was changed in 1.65 (maybe by mistake) to fix bug 850568. > > The only drawback I see is that tor has a number of dependencies that > will always appear as 'recently used' in popcon report. However if tor > is used regularly anyway this is not an issue. > > I will apply the patch for popularity-contest 1.67 > > Cheers, > -- > Bill. > > Imagine a large red swirl here. > > Excellent, I am looking forward to it. :) Do you have an estimate of when 1.67 will be in unstable? Thanks, ~Niels
Bug#773663: Updated patch for popcon tor support
On Sun, May 20, 2018 at 08:06:00AM +, Niels Thykier wrote: > On Wed, 7 Sep 2016 16:51:32 +0200 Bill Allombert > wrote: > > On Sat, Aug 27, 2016 at 12:03:52AM +0100, Tim Retout wrote: > > > A few more things before you consider shipping this (sorry for not > > > thinking of them before my previous email): > > > > > > - torify is just a wrapper around torsocks. The tor package might be > > > installed but not running, or some people might have machines with > > > torsocks configured to talk to a remote Tor daemon. We should fall > > > back at runtime if connecting via tor fails - this would probably even > > > make the code clearer? > > > - It would make sense to call the "--isolate" option in torsocks, > > > otherwise this potentially identifies the tor circuit which the rest > > > of your traffic is using, via e.g. the time of the cronjob, or at > > > least highlights that it's a Debian system > > > - I think the suggestion to have a separate default URL for tor > > > submissions is a good one - if the HTTP default SUBMITURLS has not > > > been changed, maybe switch to the tor one by default? And then insert > > > the .onion URL when DSA kindly set it up. > > > > Thanks for moving forward with this. > > > > Woud you mind sending a popcon report through TOR and send me a > > copy of /var/log/popularity-contest.gpg so that I can check the report > > was received correctly ? > > Hi, > > I have applied the patch to my installed system, set USETOR to "yes" and > generated a report plus submitted it today (it should have been > submitted a few times actually). Thanks! (and apologies to Tim to have missed the opportunuity to check Tim original popularity-contest.gpg report). I have received it just one (via http) on Sun May 20 08:02:01 UTC 2018 (however popcon cron.daily has a mechanism to prevent multiple submissions in the same week). > I have attached the .gpg file. I used > the ".new" file because it seems that the popularity-contest.new.gpg is > not renamed to popularity-contest.gpg any longer. Yes this was changed in 1.65 (maybe by mistake) to fix bug 850568. The only drawback I see is that tor has a number of dependencies that will always appear as 'recently used' in popcon report. However if tor is used regularly anyway this is not an issue. I will apply the patch for popularity-contest 1.67 Cheers, -- Bill. Imagine a large red swirl here.
Bug#773663: Updated patch for popcon tor support
On Wed, 7 Sep 2016 16:51:32 +0200 Bill Allombert wrote: > On Sat, Aug 27, 2016 at 12:03:52AM +0100, Tim Retout wrote: > > A few more things before you consider shipping this (sorry for not > > thinking of them before my previous email): > > > > - torify is just a wrapper around torsocks. The tor package might be > > installed but not running, or some people might have machines with > > torsocks configured to talk to a remote Tor daemon. We should fall > > back at runtime if connecting via tor fails - this would probably even > > make the code clearer? > > - It would make sense to call the "--isolate" option in torsocks, > > otherwise this potentially identifies the tor circuit which the rest > > of your traffic is using, via e.g. the time of the cronjob, or at > > least highlights that it's a Debian system > > - I think the suggestion to have a separate default URL for tor > > submissions is a good one - if the HTTP default SUBMITURLS has not > > been changed, maybe switch to the tor one by default? And then insert > > the .onion URL when DSA kindly set it up. > > Thanks for moving forward with this. > > Woud you mind sending a popcon report through TOR and send me a > copy of /var/log/popularity-contest.gpg so that I can check the report > was received correctly ? > > Cheers, > Bill. > > Hi, I have applied the patch to my installed system, set USETOR to "yes" and generated a report plus submitted it today (it should have been submitted a few times actually). I have attached the .gpg file. I used the ".new" file because it seems that the popularity-contest.new.gpg is not renamed to popularity-contest.gpg any longer. Running the crontab with bash -x shows that it is never attempted[1] and I see nothing in the patch that would affect that, so I assume it is an unrelated issue. Thanks, ~Niels [1] The only mv I can see is for the unencrypted log """ # grep mv /etc/cron.daily/popularity-contest mv $POPCONNEW $POPCONOLD """ popularity-contest.new.gpg Description: application/pgp-encrypted
Bug#773663: Updated patch for popcon tor support
On Sat, Aug 27, 2016 at 12:03:52AM +0100, Tim Retout wrote: > A few more things before you consider shipping this (sorry for not > thinking of them before my previous email): > > - torify is just a wrapper around torsocks. The tor package might be > installed but not running, or some people might have machines with > torsocks configured to talk to a remote Tor daemon. We should fall > back at runtime if connecting via tor fails - this would probably even > make the code clearer? > - It would make sense to call the "--isolate" option in torsocks, > otherwise this potentially identifies the tor circuit which the rest > of your traffic is using, via e.g. the time of the cronjob, or at > least highlights that it's a Debian system > - I think the suggestion to have a separate default URL for tor > submissions is a good one - if the HTTP default SUBMITURLS has not > been changed, maybe switch to the tor one by default? And then insert > the .onion URL when DSA kindly set it up. Thanks for moving forward with this. Woud you mind sending a popcon report through TOR and send me a copy of /var/log/popularity-contest.gpg so that I can check the report was received correctly ? Cheers, Bill.
Bug#773663: Updated patch for popcon tor support
A few more things before you consider shipping this (sorry for not thinking of them before my previous email): - torify is just a wrapper around torsocks. The tor package might be installed but not running, or some people might have machines with torsocks configured to talk to a remote Tor daemon. We should fall back at runtime if connecting via tor fails - this would probably even make the code clearer? - It would make sense to call the "--isolate" option in torsocks, otherwise this potentially identifies the tor circuit which the rest of your traffic is using, via e.g. the time of the cronjob, or at least highlights that it's a Debian system - I think the suggestion to have a separate default URL for tor submissions is a good one - if the HTTP default SUBMITURLS has not been changed, maybe switch to the tor one by default? And then insert the .onion URL when DSA kindly set it up. Kind regards, On 26 August 2016 at 23:13, Tim Retout wrote: > Hi! I stumbled across this bug, and it looked like there were a > couple of issues with the previous version of the patch: > > - mentioned "sockproxy" instead of "torsocks" in one place > - disabling SMTP relied on USETOR=yes, but the torify function assumed > "always/auto/no" > > So I've updated the patch against the latest git, and changed to use > the "yes/maybe/no" convention. For completeness, I've added a FAQ > entry mentioning version 1.65, and attach /var/log/popularity-contest > after (hopefully) submitting it through tor (there was no gpg > extension, because it got moved to POPCONOLD). > > Technically I've not done further testing on this beyond running it a > few times, and I haven't checked for DNS leaks etc. via wireshark, so > "caveat emp-tor", so to speak? > > Kind regards, > > -- > Tim Retout -- Tim Retout
Bug#773663: Updated patch for popcon tor support
Hi! I stumbled across this bug, and it looked like there were a
couple of issues with the previous version of the patch:
- mentioned "sockproxy" instead of "torsocks" in one place
- disabling SMTP relied on USETOR=yes, but the torify function assumed
"always/auto/no"
So I've updated the patch against the latest git, and changed to use
the "yes/maybe/no" convention. For completeness, I've added a FAQ
entry mentioning version 1.65, and attach /var/log/popularity-contest
after (hopefully) submitting it through tor (there was no gpg
extension, because it got moved to POPCONOLD).
Technically I've not done further testing on this beyond running it a
few times, and I haven't checked for DNS leaks etc. via wireshark, so
"caveat emp-tor", so to speak?
Kind regards,
--
Tim Retout
diff --git a/FAQ b/FAQ
index bc76ffa..021717d 100644
--- a/FAQ
+++ b/FAQ
@@ -25,6 +25,14 @@ A) Yes, however if the package gnupg is installed and ENCRYPT is set to 'maybe'
public key cryptography, so the eavesdropper should not be able to decrypt
them. The default is 'maybe' with popularity-contest 1.60.
+ Encrypting popcon submissions still reveals that a particular host
+ is running Debian or a derivative; and the administrators of the
+ popcon server can associate submissions with a source IP address.
+ To prevent this, if USETOR is set to 'maybe' or 'yes', and if the
+ 'tor' and 'torsocks' packages are installed, the submission will be
+ made over Tor. The default is 'maybe' with popularity-contest
+ 1.65, so Tor will be used if available.
+
Q) What are the privacy considerations for popularity-contest ?
A) Each popularity-contest host is identified by a random 128bit uuid
diff --git a/debian/control b/debian/control
index a300056..c30eb23 100644
--- a/debian/control
+++ b/debian/control
@@ -14,7 +14,7 @@ Architecture: all
Pre-Depends: debconf (>= 1.5.34) | cdebconf (>= 0.106)
Depends: ${misc:Depends}, ${perl:Depends}, dpkg (>= 1.10)
Recommends: gnupg, cron | cron-daemon, exim4 | mail-transport-agent
-Suggests: anacron
+Suggests: anacron, tor, torsocks
Provides: popcon
Description: Vote for your favourite packages automatically
The popularity-contest package sets up a cron job that will
diff --git a/debian/cron.daily b/debian/cron.daily
index a3a92ef..4e19240 100644
--- a/debian/cron.daily
+++ b/debian/cron.daily
@@ -14,8 +14,39 @@ unset MY_HOSTID
unset PARTICIPATE
unset SUBMITURLS
unset USEHTTP
+unset USETOR
unset MTAOPS
+TORIFY_PATH=/usr/bin/torify
+
+torify_enabled() {
+# Return 1 to enable torify for HTTP submission, otherwise 0; exit on error
+TORSOCKS_PATH=/usr/bin/torsocks
+[ -f "$TORIFY_PATH" ] && [ -f "$TORSOCKS_PATH" ] && TOR_AVAILABLE=1
+
+case "$USETOR" in
+"yes")
+if [ -z $TOR_AVAILABLE ]; then
+echo "popularity-contest: USETOR is set but torify is not available." 2>&1
+echo "popularity-contest: Please install the tor and torsocks packages." 2>&1
+exit 1
+fi
+if [ "yes" != "$USEHTTP" ]; then
+echo "popularity-contest: when USETOR is set USEHTTP must be set as well" 2>&1
+exit 1
+fi
+return 0
+;;
+"maybe")
+[ "yes" = "$USEHTTP" ] && [ ! -z $TOR_AVAILABLE ] && return 0
+return 1
+;;
+"no")
+return 1
+;;
+esac
+}
+
# get configuration information
. /usr/share/popularity-contest/default.conf
. /etc/popularity-contest.conf
@@ -33,6 +64,14 @@ if [ -z "$MAILTO" ] && [ "yes" != "$USEHTTP" ]; then exit 0; fi
# don't run if PARTICIPATE is "no" or unset!
if [ "$PARTICIPATE" = "no" ] || [ -z "$PARTICIPATE" ]; then exit 0; fi
+# enable torify
+if torify_enabled; then
+TORIFY=$TORIFY_PATH
+else
+TORIFY=''
+fi
+
+
if [ -n "$HTTP_PROXY" ]; then
export http_proxy="$HTTP_PROXY";
fi
@@ -106,7 +145,7 @@ SUBMITTED=no
# try to post the report through http POST
if [ "$SUBMITURLS" ] && [ "yes" = "$USEHTTP" ]; then
for URL in $SUBMITURLS ; do
- if setsid /usr/share/popularity-contest/popcon-upload \
+ if setsid $TORIFY /usr/share/popularity-contest/popcon-upload \
-u $URL -f $POPCON 2>/dev/null ; then
SUBMITTED=yes
else
@@ -116,8 +155,9 @@ if [ "$SUBMITURLS" ] && [ "yes" = "$USEHTTP" ]; then
fi
# try to email the popularity contest data
+# skip emailing if USETOR is set
-if [ "$MODE" = "--crond" ] && [ yes != "$SUBMITTED" ] && [ "$MAILTO" ]; then
+if [ "$MODE" = "--crond" ] && [ yes != "$SUBMITTED" ] && [ yes != "$USETOR" ] && [ "$MAILTO" ]; then
if [ -x "`which sendmail 2>/dev/null`" ]; then
(
if [ -n "$MAILFROM" ]; then
diff --git a/default.conf b/default.conf
index 773913d..ce87699 100644
--- a/default.conf
+++ b/default.conf
@@ -51,6 +51,16 @@ SUBMITURLS="http://popcon.debian.org/cgi-bin/popcon.cgi";
# USEHTTP enables http reporting. Set this to 'yes' to enable it.
USEHTTP="yes"
+# USETOR enab
