Joshua Hesketh wrote: > Great overview and plan James, thanks for that :-). > > So it seems to me that we're duplicating the job of swift a little bit > by writing a program to accept an object over http and store it on disk. > If our end-game is logs stored in swift then why not make jenkins (and > other workers) push the logs straight to swift?
I suspect the additional benefit is the token-based security which means the slaves can have limited access to storage (they basically get a token that can only be used for one very limited - and one-time - task). That said I agree that reinventing HTTP POST storage sounds overkill and storing directly to swift would be a lot simpler. Does Swift include complex access rules that we could leverage to implement the same kind of security that James described in his original post ? If not, would it make sense to actually add what we need to Swift itself ? (after all, it's, you know, an OpenStack project) > [...] > *For example, depending on the size of the logs (and therefore the > job/worker) we could actually use javascript to serve up the swift > object with CORS further reducing the infrastructure requirement and > utilising the powerful CPU's and javascript engines we all have. I > already started doing that as a crude way to serve and add formatting to > my logs[1]. Basically javascript grabs a file and runs some regex for > highlighting. So my worker only reports > http://laughing-spice/logviewer/?q=http://worker/job/index.html where > the index.html contains links to logs like > http://laughing-spice/logviewer/?q=http://worker/job/mysqllog.txt etc. > [...] Sidenote: I like that idea too, but I'm not sure how well that would work with our huge logs. -- Thierry Carrez (ttx) _______________________________________________ OpenStack-Infra mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
