** Description changed:
+ == Begin SRU Info ==
+ [Impact]
+ maas-import-ephemerals is the program that is executed to download images
from http://maas.ubuntu.com which are then used for the commissioning,
enlistment, and curtin install environment.
+
+ This bug causes that download to fail in most cases with an http proxy
+ between the user and maas.ubuntu.com. It is also reported to fail under
+ cases where there is no http_proxy set (but may be a proxy in between).
+
+ [Test Case]
+ The bug could be reproduced by simply doing:
+ PROXY=http://localhost:3128 # or some valid proxy
+ http_proxy=$PROXY python /usr/bin/sstream-mirror -vvv
http://maas.ubuntu.com/images/ephemeral/daily/streams/v1/index.json out.d
arch=amd64 release=quantal
+
+ You would see the download start and then hang. A suitable proxy can be
+ installed just by 'apt-get install squid3' and then using
+ http://localhost:3128.
+
+ [Regression Potential]
+ Regression potential should be very small. Previously each connection we
made was doing an open and close and then open again. This was wreaking havoc
with urllib3 as used by python-requests. Now, we're only opening and closing a
single time during the mirror, and even then,
+ a.) the url that is being downloaded will change (by having a '/' inserted
into its path) where previously it would be the same url
+ b.) step 'a' will only occur if the base url provided by the user did not
already end in '/'.
+ == End SRU Info ==
+
Since 2013-10-11 we started seeing errors running the MAAS integration
tests for saucy[1].
The checksum of the downloaded image wasn't matching the checksum
expected[2]. maas-import-pxe-files failed with the following traceback:
Traceback (most recent call last):
- File "/usr/sbin/maas-import-ephemerals", line 26, in <module>
- main(args)
- File
"/usr/lib/python2.7/dist-packages/provisioningserver/import_images/ephemerals_script.py",
line 450, in main
- target.sync(source, args.path)
- File "/usr/lib/python2.7/dist-packages/simplestreams/mirrors/__init__.py",
line 85, in sync
- return self.sync_index(reader, path, data, content)
- File "/usr/lib/python2.7/dist-packages/simplestreams/mirrors/__init__.py",
line 230, in sync_index
- self.sync(reader, path=epath)
- File "/usr/lib/python2.7/dist-packages/simplestreams/mirrors/__init__.py",
line 83, in sync
- return self.sync_products(reader, path, data, content)
- File "/usr/lib/python2.7/dist-packages/simplestreams/mirrors/__init__.py",
line 304, in sync_products
- self.insert_item(item, src, target, pgree, ipath_cs)
- File
"/usr/lib/python2.7/dist-packages/provisioningserver/import_images/ephemerals_script.py",
line 249, in insert_item
- data, src, target, pedigree, contentsource)
- File "/usr/lib/python2.7/dist-packages/simplestreams/mirrors/__init__.py",
line 388, in insert_item
- size=data.get('size'))
- File
"/usr/lib/python2.7/dist-packages/simplestreams/objectstores/__init__.py", line
147, in insert
- cksum.hexdigest(), cksum.expected))
+ File "/usr/sbin/maas-import-ephemerals", line 26, in <module>
+ main(args)
+ File
"/usr/lib/python2.7/dist-packages/provisioningserver/import_images/ephemerals_script.py",
line 450, in main
+ target.sync(source, args.path)
+ File "/usr/lib/python2.7/dist-packages/simplestreams/mirrors/__init__.py",
line 85, in sync
+ return self.sync_index(reader, path, data, content)
+ File "/usr/lib/python2.7/dist-packages/simplestreams/mirrors/__init__.py",
line 230, in sync_index
+ self.sync(reader, path=epath)
+ File "/usr/lib/python2.7/dist-packages/simplestreams/mirrors/__init__.py",
line 83, in sync
+ return self.sync_products(reader, path, data, content)
+ File "/usr/lib/python2.7/dist-packages/simplestreams/mirrors/__init__.py",
line 304, in sync_products
+ self.insert_item(item, src, target, pgree, ipath_cs)
+ File
"/usr/lib/python2.7/dist-packages/provisioningserver/import_images/ephemerals_script.py",
line 249, in insert_item
+ data, src, target, pedigree, contentsource)
+ File "/usr/lib/python2.7/dist-packages/simplestreams/mirrors/__init__.py",
line 388, in insert_item
+ size=data.get('size'))
+ File
"/usr/lib/python2.7/dist-packages/simplestreams/objectstores/__init__.py", line
147, in insert
+ cksum.hexdigest(), cksum.expected))
Exception: unexpected checksum 'sha256' on
precise/release-20131010/precise-daily-maas-amd64.tar.gz (found:
3d99e069e34d031f924689a3c28ea43d4046d8e9a79a2ea9014d35bcbcf57a16 expected:
2c7ab801ffd7e8673b77be828335bac29538f102dbc6baacd4ccc4c0d309a11b)
}}}
Initially it was thought this could be caused by a misconfigured cache
in the QA lab, but after re-installing and reconfiguring (and checking
with wget that the cache was returning the correct objects for a given
URL, as well as checking the checksum of the file matched the one in the
json file), the error was still present.
Further investigation[3] raised the suspicion that the downloaded image
is being handled by multiple processes. Turns out the code for
simplestreams and import_ephemerals.py is not multi process safe, and
multiple import-ephemerals processes working with the same datadir can
cause problems like this.
[1]
http://10.189.74.2:8080/view/MAAS/job/saucy-adt-maas-daily/220/ARCH=amd64,label=lenovo-RD230-01/console
[2]
http://10.189.74.2:8080/view/MAAS/job/saucy-adt-maas-manual/95/ARCH=amd64,label=lenovo-RD230-01/console
[3] http://pastebin.ubuntu.com/6247125/
** Also affects: simplestreams (Ubuntu Saucy)
Importance: High
Status: In Progress
** Also affects: simplestreams (Ubuntu T-series)
Importance: Undecided
Status: New
** Changed in: simplestreams (Ubuntu T-series)
Status: New => In Progress
** Changed in: simplestreams (Ubuntu T-series)
Importance: Undecided => High
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1240652
Title:
maas-import-ephemerals crashes with "unexpected checksum 'sha256'"
when using a proxy
To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1240652/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
