Greetings, The following webrev contains fixes for the following issues:
2691 ability to publish packages to local disk repository 850 pkgsend traceback if invalid or unavailable repository url specified 894 pkgsend should do more action verification 1236 pkgsend open should not allow both -e and -n 2430 pkgsend add of non-existent file causes traceback 5088 server can still create a repo when in --readonly mode 5294 pkgsend doesn't provide server response when operations fail 5944 pkgsend traceback if invalid or no action type specified webrev: http://cr.opensolaris.org/~swalker/pkg-2691/ Notes ===== I apologise in advance for the size of this changeset. The perhaps not so surprising thing is that there is still much to do, but I wanted to keep this changeset as small as possible. I would also like to apologise for not having this changeset done sooner, but as you might imagine, I wanted to do a lot of testing. I've run a redist_import more times than I care to count. Many tests have been added as well, so we should be able to be a lot more confident in our publishing operations now. I'm going to go collapse now... Timings ======= My Version ---------- REPO=file://repo time make -e 105/redist_import real 43m2.937s user 38m33.199s sys 3m6.462s REPO=http://localhost:8009 time make -e 105/redist_import real 84m53.761s user 15m17.978s sys 3m19.965s Gate Tip -------- REPO=http://localhost:8009 time make -e 105/redist_import real 88m26.626s user 15m1.740s sys 5m29.212s Change Summary ============== * the depot server will now only create a repository if not started with --readonly. See tests/cli/t_pkg_depotd.py:test_repo_create for details. * the depot server will now print error information about the exception that occurred in case an unknown error (shouldn't happen, but...) occurs when starting the depot server cherrypy process * actions now check to see the their key attribute has been provided during initialisation and raise an appropriate exception if not * both fromstr and fromlist now support an additional data keyword parameter so that file data can be provided instead of being attached manually * verify now uses less memory as hash calculation is done in chunks and discarded instead of reading the entire file into memory; it should also be less likely to fail on some platforms * actions now determine the size of the specified data payload if the pkg.size attribute is not provided * license files now have their hash calculated in chunks to avoid memory footprint blowout * added as_lines method to common catalog class * dropped now unused modules/config.py * Reworked publishing transaction class into a "class factory" that, based on repository url, returns an object that can communicate with the repository appropriate to the scheme's associated protocol * Added a refresh_index publishing operation for local publishing * Added ability to disable search indices updating to publishing close operation * Fixed pkgsend so that publishing to a remote server now returns useful error information (instead of just a 400, etc.) * unified error handling interface for publishing such that a consistent set of exceptions are raised regardless of the type of repository being published to (file, http, etc.) * moved all of the current transport layer logic for the repository into a separate DepotHTTP class in modules/server/depot.py * moved existing DepotResponse class from modules/server/depot.py to modules/server/depotresponse.py * made depot manifest operation streaming * the depot is now much more resilient against malformed actions and/or failed publishing operations * fixed a case where an elferror exception could be raised without being caught when adding a file action * fixed pkgsend to handle invalid or unavailable (connection refused, etc.) repository url * fixed pkgsend to do more action verification before sending to repository (this was mostly done in the action class itself and changing pkgsend to handle any related exceptions) * fixed pkgsend to not allow -e and -n options simultaneously * fixed pkgsend to not traceback when providing a non-existent file for the file action * fixed depot server and pkgsend to handle invalid or missing action types * fixed possible bug in server/transaction:add_content that would have caused hashes to not be correctly calculated; i don't believe it would have caused a problem with remote publishing since successive reads would have worked correctly since the rfile object isn't seekable, but this would have broken local publishing Cheers, -- Shawn Walker _______________________________________________ pkg-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
