It occurred to me that there might already be a Python-based caching proxy, and
there is (kinda/sorta):
https://pypicache.readthedocs.org/en/latest/
It's specific to PyPi, but nothing we couldn't hacksaw out of the way.
--Hardy
________________________________
From: Pottinger, Hardy J.
Sent: Friday, July 10, 2015 11:05 AM
To: Donovan Baarda
Cc: polipo-users@lists.sourceforge.net
Subject: Re: [Polipo-users] Offline mode + HTTPS does not seem possible
Oh, great, I followed that link to mitmproxy.org... I'm sure I'm on some
government list now. ;-)
OK, seriously, that sounds like a fun hack. I'm not much of a Pythonista, but
I'd love an excuse to play with it a bit. And this sounds like fun. If you need
a co-conspirator, let me know.
--Hardy
________________________________
From: dbaa...@gmail.com [dbaa...@gmail.com] on behalf of Donovan Baarda
[a...@minkirri.apana.org.au]
Sent: Thursday, July 09, 2015 8:00 PM
To: Pottinger, Hardy J.
Cc: polipo-users@lists.sourceforge.net
Subject: Re: [Polipo-users] Offline mode + HTTPS does not seem possible
Unfortunately, the enthusiastic push for better internet security with https is
breaking cacheability. None of the people serving this content care, because
traffic is one of their metrics for success, and they can cache right up to
their boundary by terminating the ssl at their edge, so they don't care if it's
non-cachable beyond there. Companies like Google "solve" the caching problem by
just pushing their "edge" further and further out, owning more and more of the
internet infrustructure between customers and their servers.
I've been considering trying to create an unholy chimera by duct-taping polipo
and https://mitmproxy.org/ together with Python to fix this.
I'm getting sick of the 5 different android devices in the house insisting on
downloading all the firmware and app updates 5x over https. This was
particularly bad when I was briefly connected using stupidly expensive 4G and
found the android devices had significantly contributed to exhausting my 5GB
pre-paid quota in less than 2 days.
Things like mitmproxy are pretty bad from a security perspective, but
unfortunately that's the only way to work around this. Unfortunately the push
for more security is working on breaking them too. Cert-pinning is the "magic
solution" to breaking on-the-fly cert generation used by mitmproxy, and it's
already starting to appear in chrome etc. Not sure how easy it's going to be to
work around.
On 10 July 2015 at 04:38, Pottinger, Hardy J.
<pottinge...@missouri.edu<mailto:pottinge...@missouri.edu>> wrote:
Hi, I'm trying to use Polipo as a way to facilitate offline access to code
repositories (Maven central, github, etc) for my Vagrant-managed Virtualbox
development environment. Our build process uses Grunt and Bower to pull in
various pieces of front-end "stuff" and many of these URLs are https URLs to
github, for example:
https://github.com/Modernizr/Modernizr.git
Now, I can have Polipo running, and the network on, on my guest machine I can
successfully run
wget https://github.com/Modernizr/Modernizr.git
Which, I would assume, is caching the content. I then disable my network
connection on my host machine, put Polipo in offline mode, and then re-attempt
a wget of Modernizer... but I get the following:
Connecting to 10.0.2.2:8123... connected.
Proxy tunneling failed: Couldn't connect: Network is unreachableUnable to
establish SSL connection
Running a tail -f on /var/log/polipo/polipo.log does not show me anything
during this error.
Here is the only additions I've made to my /etc/polipo/config file:
disableIndexing = false
disableServersList = false
dnsUseGethostbyname = reluctantly
dnsQueryIPv6 = no
dnsQueryIPv6 = reluctantly
proxyOffline = yes
Any advice would be much appreciated, thanks!
On a positive note, despite not getting offline caching to work, I *have*
noticed a considerable bump in compile performance. Down from 5 minutes to 2
minutes, with tests turned on. So, I'm moderately pleased about that. But, i
would like to figure out this offline proxy stuff. :-)
--Hardy
------------------------------------------------------------------------------
Don't Limit Your Business. Reach for the Cloud.
GigeNET's Cloud Solutions provide you with the tools and support that
you need to offload your IT needs and focus on growing your business.
Configured For All Businesses. Start Your Cloud Today.
https://www.gigenetcloud.com/
_______________________________________________
Polipo-users mailing list
Polipo-users@lists.sourceforge.net<mailto:Polipo-users@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/polipo-users
--
Donovan Baarda <a...@minkirri.apana.org.au<mailto:a...@minkirri.apana.org.au>>
------------------------------------------------------------------------------
Don't Limit Your Business. Reach for the Cloud.
GigeNET's Cloud Solutions provide you with the tools and support that
you need to offload your IT needs and focus on growing your business.
Configured For All Businesses. Start Your Cloud Today.
https://www.gigenetcloud.com/
_______________________________________________
Polipo-users mailing list
Polipo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/polipo-users
