On Friday, August 26, 2016 at 5:41:37 PM UTC-4, William Stein wrote: > > On Fri, Aug 26, 2016 at 2:22 PM, Robert Lowe <png...@gmail.com > <javascript:>> wrote: > > I am pleased to report that i have successfully installed sage math > cloud on > > an AWS server, an that I am all geared up to use it with around 60 > students > > this coming fall semester (which begins on Wednesday for me). I thought > I > > would let everyone in on the customization that I made and give a few > notes > > on the experience of setting everything up. > > > > First, I went through the setup instructions contained in the git > > repository. These worked for the most part, with the occasional little > > tweak here and there to make my version of Ubuntu happy. I am running > on > > xenial, which is not an official AWS OS, but rather was upgraded from > their > > LTS image (which I believe runs Trusty). There have been some changes > to > > those instructions already, I noticed one other little thing in the > section > > about the ~/.bashrc file. It makes reference to the ~/smc/smc folder, > where > > it should be the ~/smc/src folder. > > > > Now fixed: > https://github.com/sagemathinc/smc/commit/68e504906bb251231f5c82d05b51e1eddfdfad92 > > > Anyhow, after a little bit of messing around with that, I was able to > make a > > fully functional single server setup run. The modified instructions > should > > get you there quite easily. > > Are these the modified instructions, or ? > > https://github.com/pngwen/smc/blob/master/INSTALL.md > > I was referring to the changes you made about about the hostname setup. That was the big gotcha for me when I was installing. I need to merge in the upstream changes that include those. So for anyone reading this, I'm referring to these instructions: https://github.com/pngwen/smc/blob/master/INSTALL.md Not the ones presently in my repository.
> > I did not set up ZFS, but I may do that in the > > future. I integrated everything into my existing server by setting up > > haproxy to be my fron-end, with some subdomains proxy out to apache. I > run > > a few websites on this machine, which includes mediawiki and student > > websites for a web development course. I'm happy to say that I found no > > real issues doing this, apart from needing to set up remoteip on apache > so > > that haproxy's IP address pass thrus would work. So now I have haproxy, > > nginx, and apache all living in harmony. > > > > Now here's where things get a little interesting. After looking around > the > > code, I realized there are a lot of places where smc has hardcoded > things > > that get in the way of running it in an institutional setup. First and > > foremost, it has an extensive interface geared toward buying > subscriptions > > and selling upgrades. My usecase was such that I didn't want to do > that. I > > will be giving my students accounts, and that's all they get! (I > certainly > > don't want to get in the business of selling them stuff.) Also, emails > had > > hardcoded references to sendgrid templates, and the help email address > > defaults to he...@sagemath.com <javascript:>. Also, there is the > zendesk interface, which > > I may put back in but for now I didn't really want to learn zendesk. > > > > So, to make this all work, I decided to fork the smc code on gtihub. > You > > can access my fork at http://www.github.com/pngwen/smc. The default > branch > > is just the code as it existed at the time I pulled it over around 2 > weeks > > ago. The interesting stuff is in my working branch, titled "mc", which > is > > the exact code that I am running at Maryville College. In this branch, > > you'll find what I am starting to develop into a sort of "institutional" > > version of smc. For the time being though, a lot of the hardcoded smc > stuff > > has been replaced by hardcoded Maryville College stuff. The big changes > > that I have made are: > > > > I took out all of the references I could find to "free" and "no-network" > > accounts. > > I removed links to billing policies, and also the upgrade interface. > > I removed the upgrades on the project quota page. I also took out the > > reference to "member servers" > > I modified the signup system. Now, instead of letting users directly > > register, they get sent an email which takes them into the reset > password > > system. Their password is initialized to a random uuid, so they can't > log > > in until the verify their email. > > I restricted signups to people with maryvillecollege.edu email > addresses. > > This coupled with change number 4 should make this an MC only server. > > I created a little smcd script, which starts both hub and compute on my > > server. This was just a convenience measure, nothing major. > > I removed the interfaces for the zendesk system. There are no trouble > > tickets available in my version at this time, though there are plenty of > > places where my email dress is recommended as a place of support. Also, > I > > will be seeing all of my users in person at least twice a week, and so > it > > should be easy for me to manage my load! > > I had to make a few little tweaks here and there to make everything > work. > > One that I don't like is my most recent change where I had to make > compute > > run rsync with sudo in order to get classroom distributions to work. > I'm > > not sure if I've missed something there but for now that was the only > way I > > managed to make it fly. > > Since different UNIX users are involved, and no special service is > implemented for this (and there is no ssh exchange directly between > projects), this is currently the simplest approach. Of course > something that doesn't involve sudo could somebody be implemented. > > How is this done on the sagemath.com setup? Does it work by virtue of ssh among the different servers? > > > In the coming weeks I am going to continue using SMC and will probably > find > > a few more billing references here and there to take out. Eventually, > I'm > > going to remove that code altogether from my branch. I'm also going to > move > > a lot of these changes that I've been making into a settings based > scheme. > > I would have done that this time around, but the semester was looming > large > > for me and I wanted to make sure that I had a system up and running for > my > > students! > > > > On the whole, I've found the process to be relatively straightforward, > > especially with the recent documentation updates. So to summarize, my > setup > > consists of the following recipe: > > > > AWS Medium Instance with 200GB of disk space allocated > > a secondary private address, which is where I run apache, nginx, hub, > > rethinkdb, and compute. This interface has no public facing route, it's > > completely internal > > haproxy on the frontend proxying out to apache, nginx, and hub > > let's encrypt certbot free ssl sertificate (they have a nice haproxy > script > > which I adapted to get the certs all automated in terms of renewals) > > My firewall is setup to only allow: DNS, SSH, port 80 (which haproxy > sends > > to 443), port 443 > > The mc branch of my fork of SMC > > > > After I have the standalone SMC stuff squared away, I'm going to start > > developing a few ideas for the webapp. Before I do that though, I have > to > > get more up to speed with react programming. You can see an instance of > > something I tried, which did not work, by looking at my landing page. > I'll > > figure it out though, once I do, I'm going to work on a few pet > projects: > > > > a plugin interface, allowing a user to install a per-project > "application" > > of sorts > > We definitely want that. Last week a lot of work was done on > finishing the react rewrite of the SMC frontend, which dramatically > change most code that would be relevant to this. > > Once the semester starts I'll pull in your upstream changes and then try to merge that with my mc branch. I'll do that on a dev server though! > > A general graphical client, for instance using vnc from within smc > > https://github.com/sagemathinc/smc/issues/135 > > > A logic circuit drawing tool for use in computer architecture classes. > > A schematic capture tool for use with spice. > > Contribute to some bugfixes for the core webapp! > > > > Those latter changes will be the ones that I hope most people will find > > useful. I will likely not get around to those for quite some time. For > the > > moment, I just want to keep a close watch on my own SMC installation and > > make sure my students get a positive computing experience out of it! > > > > If you have any questions or thoughts on my setup, please let me know! > > Are you planning to redo your changes in a more generic way, which we > can merge into the main SMC codebase when/if you get the time? You > don't make that clear above. Or, do you do just plan to maintain a > hardcoded fork? Or do you want us to try with your blessing? > > I plan on redoing what I've done with configuration options. My intention is to make a general purpose standalone version. Basically an SMC without the billing and upgrades. Most of what I have done is trivial to do with configuration options (like the emails and such). I was thinking for the domain restrictions for signup that I may set that up as a list stored in the db. Another thought I had was allowing people who were invited to projects to sign up. Kind of a "vampiric" authentication scheme where insiders can invite outsiders in. I'm still trying to decide if I want to hand my students that kind of power though! (after all, this server allows full internet access!) It would be great if you want to merge those into the main codebase. Ultimately, I'd like to contribute some nice frontend modules for possible inclusion in your main code. It will probably take me a bit to get to that point though. This is my first real-world exercise with coffeescript and csjx. It's proven to be pretty easy to get into though. I love functional programming! > The two main development directions right now are: > 1. finishing the react rewrite of the frontend > 2. rewriting the backend to optionally run on Kubernetes completely > The plan is that these should be done, well tested, and generally live > by the end of September. > > Note that 2 involves completely deleting a huge amount of backend > code, which replicated some chunks of what kubernetes does, but are > also necessary for the way you're running SMC now... E.g., the > compute server daemon, scripts like smc-compute, etc., will be gone. > Don't worry -- Kubernetes has very, very good AWS support. > > -- william > I should be able to migrate over to this once it's done in the upstream code. The only thing I changed in compute was adding "sudo" and "-lroot" to that one python module. My changes to the hub are relatively self contained, with there being some changes to create_account and one function which is basically a modified version of the forgot_password function. I think the changes that you are doing that will affect me the most is on the front end. Of course, a lot of what I did was with the static content. -Bob -- You received this message because you are subscribed to the Google Groups "sage-cloud" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-cloud+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-cloud/2916e24b-99a6-453d-8235-baeeaac098c7%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
