Re: [sage-cloud] Installation Problems
What I'm really after is running my own SMC installation for my students. The commercial servers are far too bogged down, even with paid tiers. Ultimately, I will be running a customized version which will only allow maryvillecollege.edu addresses to sign up, and have a few other changes that I have in mind. I can run the laptop one just fine, I want to set up a real installation though. On Thursday, August 18, 2016 at 7:55:38 PM UTC-4, Harald Schilly wrote: > > Hi, if you want to do dev work, then you should run it via the > instructions in src/dev/laptop > > The projects are all isolated, such that you do not get any > information like processes and stuff for who on the compute nodes. > There is nothing special going on, except for cgroups. The real > problem might be, that you have to do additional configuration in the > database, but I'm not sure what you really did. > > One day in the future, there will be configurations for kubernetes. > You can check that progress in the k8s subdirectory. > > -- harald > -- You received this message because you are subscribed to the Google Groups "sage-cloud" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-cloud+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-cloud/02439746-9a64-4d6e-823a-b0423ca3feed%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [sage-cloud] Installation Problems
Another part of the puzzle has emerged. I can get access to raw files through the proxy if they are in a publicly shared folder. However, if I'm doing a raw pull within a project in a non-public folder I get a bad gateway. This tells me something is going on inside smc, probably with an authentication token of some kind. I noticed in proxy.coffee that it looks like that is how it determines if it serves a file or not. The section in question reads: if not misc.path_is_in_public_paths(path, public_paths) # The requested path is not public, so nothing to do. cb() else # The requested path *is* public, so we get the file # from one (of the potentially many) compute servers # that has the file -- (right now this is implemented # via sending/receiving JSON messages and using base64 # encoding, but that could change). It's non-obvious whether the earlier code in proxy.coffee can handle an authenticated user's files. I assume it can, as this capability exists in the live cloud.sagemath.org server. Alternatively, perhaps the master branch code is not actually the code currently running on SMC. When I try manually talking to the proxy, and I send the same header sent by my web browser, I get a blank response. The proxy doesn't even send a header back, which tells me that we are likely visiting this segment of the code even while authenticated. I'm also beginning to think I'm alone in this, and once I do figure it out I will join an elite group of individuals who have managed to install SMC. On Friday, August 19, 2016 at 9:52:41 AM UTC-4, Robert Lowe wrote: > > I am making progress! I have discovered the secret handshake which starts > the hub on a single server, and how to run the compute daemon. I'm > basically just running them as they are in the dev/single directory. > > So project starts, I can use the terminal, and I can create and edit > files. Now, though, I can't view latex previews. It builds the latex > documents just fine, but preview does not work. Looking under the hood a > bit, I see that the rendered png files cannot download over the /raw links. > I did a little more checking and discovered that raw download does not > work for any file, regardless of where it is! The error I get when I go to > one of these raw links is: > > 502 Bad Gateway > > I looked in the nginx and haproxy logs and saw no errors. The hub and > compute similarly report what looks like should be successful. Has anyone > got any ideas where I should look? > > On Thursday, August 18, 2016 at 8:00:07 PM UTC-4, Robert Lowe wrote: >> >> What I'm really after is running my own SMC installation for my students. >> The commercial servers are far too bogged down, even with paid tiers. >> Ultimately, I will be running a customized version which will only allow >> maryvillecollege.edu addresses to sign up, and have a few other changes >> that I have in mind. >> >> I can run the laptop one just fine, I want to set up a real installation >> though. >> >> On Thursday, August 18, 2016 at 7:55:38 PM UTC-4, Harald Schilly wrote: >>> >>> Hi, if you want to do dev work, then you should run it via the >>> instructions in src/dev/laptop >>> >>> The projects are all isolated, such that you do not get any >>> information like processes and stuff for who on the compute nodes. >>> There is nothing special going on, except for cgroups. The real >>> problem might be, that you have to do additional configuration in the >>> database, but I'm not sure what you really did. >>> >>> One day in the future, there will be configurations for kubernetes. >>> You can check that progress in the k8s subdirectory. >>> >>> -- harald >>> >> -- You received this message because you are subscribed to the Google Groups "sage-cloud" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-cloud+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-cloud/43ed5d13-ed11-4011-91f1-4d57921f2734%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [sage-cloud] Some thoughts on my customized institutional setup
On Friday, August 26, 2016 at 5:41:37 PM UTC-4, William Stein wrote: > > On Fri, Aug 26, 2016 at 2:22 PM, Robert Lowe <png...@gmail.com > > wrote: > > I am pleased to report that i have successfully installed sage math > cloud on > > an AWS server, an that I am all geared up to use it with around 60 > students > > this coming fall semester (which begins on Wednesday for me). I thought > I > > would let everyone in on the customization that I made and give a few > notes > > on the experience of setting everything up. > > > > First, I went through the setup instructions contained in the git > > repository. These worked for the most part, with the occasional little > > tweak here and there to make my version of Ubuntu happy. I am running > on > > xenial, which is not an official AWS OS, but rather was upgraded from > their > > LTS image (which I believe runs Trusty).There have been some changes > to > > those instructions already, I noticed one other little thing in the > section > > about the ~/.bashrc file. It makes reference to the ~/smc/smc folder, > where > > it should be the ~/smc/src folder. > > > > Now fixed: > https://github.com/sagemathinc/smc/commit/68e504906bb251231f5c82d05b51e1eddfdfad92 > > > Anyhow, after a little bit of messing around with that, I was able to > make a > > fully functional single server setup run. The modified instructions > should > > get you there quite easily. > > Are these the modified instructions, or ? > > https://github.com/pngwen/smc/blob/master/INSTALL.md > > I was referring to the changes you made about about the hostname setup. That was the big gotcha for me when I was installing. I need to merge in the upstream changes that include those. So for anyone reading this, I'm referring to these instructions: https://github.com/pngwen/smc/blob/master/INSTALL.md Not the ones presently in my repository. > > I did not set up ZFS, but I may do that in the > > future. I integrated everything into my existing server by setting up > > haproxy to be my fron-end, with some subdomains proxy out to apache. I > run > > a few websites on this machine, which includes mediawiki and student > > websites for a web development course. I'm happy to say that I found no > > real issues doing this, apart from needing to set up remoteip on apache > so > > that haproxy's IP address pass thrus would work. So now I have haproxy, > > nginx, and apache all living in harmony. > > > > Now here's where things get a little interesting. After looking around > the > > code, I realized there are a lot of places where smc has hardcoded > things > > that get in the way of running it in an institutional setup. First and > > foremost, it has an extensive interface geared toward buying > subscriptions > > and selling upgrades. My usecase was such that I didn't want to do > that. I > > will be giving my students accounts, and that's all they get! (I > certainly > > don't want to get in the business of selling them stuff.) Also, emails > had > > hardcoded references to sendgrid templates, and the help email address > > defaults to he...@sagemath.com . Also, there is the > zendesk interface, which > > I may put back in but for now I didn't really want to learn zendesk. > > > > So, to make this all work, I decided to fork the smc code on gtihub. > You > > can access my fork at http://www.github.com/pngwen/smc. The default > branch > > is just the code as it existed at the time I pulled it over around 2 > weeks > > ago. The interesting stuff is in my working branch, titled "mc", which > is > > the exact code that I am running at Maryville College. In this branch, > > you'll find what I am starting to develop into a sort of "institutional" > > version of smc. For the time being though, a lot of the hardcoded smc > stuff > > has been replaced by hardcoded Maryville College stuff. The big changes > > that I have made are: > > > > I took out all of the references I could find to "free" and "no-network" > > accounts. > > I removed links to billing policies, and also the upgrade interface. > > I removed the upgrades on the project quota page. I also took out the > > reference to "member servers" > > I modified the signup system. Now, instead of letting users directly > > register, they get sent an email which takes them into the reset > password > > system. Their password is initialized t
[sage-cloud] Re: RethinkDB to PostgreSQL Migration Question
Yes! That's exactly what I am looking for. I'm not sure how I missed that. Somehow I had found the k8s migration scripts but not this one. Thank you for your help! I think I'm going to make some modifications and have my local version of the newer SMC up and running during our spring break (which starts on March 20). My students will come back to a shiny new SMC :-D -Bob On Sunday, March 5, 2017 at 7:39:27 AM UTC-5, Harald Schilly wrote: > > > > On Sunday, March 5, 2017 at 2:02:26 AM UTC+1, Robert Lowe wrote: >> >> >> I couldn't find anything that looked like a migration script. Is there >> one in the code base? >> > > I think you're looking for > > https://github.com/sagemathinc/smc/tree/master/src/scripts/postgresql/migrate > > It's some one-off code william wrote and I don't know if it would also > work for your case. There were a couple of issues with the json encoding, > too, so maybe there is even some manual work involved. > > -- harald > > -- You received this message because you are subscribed to the Google Groups "sage-cloud" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-cloud+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-cloud/9299300a-bbbc-410d-a41d-7695acee6d4a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [sage-cloud] Re: RethinkDB to PostgreSQL Migration Question
Thanks for the heads up. That's good to know! I will be sure to alert all my uses to the edit history reset. I don't think that will be a big deal for them as most of them use other revision control mechanisms (like rcs or git). Though I have heard that the time travel feature is a big hit when deadlines get close! -Bob On Sunday, March 5, 2017 at 12:02:11 PM UTC-5, William Stein wrote: > > Bob, > > I recommend not migrating the syncstrings and patches tables. Just leave > them empty. There's other stuff you would have to do to properly migrate > those.. Edit history will get reset but everything will be fine otherwise. > > William > > On Sun, Mar 5, 2017 at 7:12 AM Robert Lowe <png...@gmail.com > > wrote: > >> Yes! That's exactly what I am looking for. I'm not sure how I missed >> that. Somehow I had found the k8s migration scripts but not this one. >> Thank you for your help! I think I'm going to make some modifications and >> have my local version of the newer SMC up and running during our spring >> break (which starts on March 20). My students will come back to a shiny new >> SMC :-D >> >> -Bob >> >> >> On Sunday, March 5, 2017 at 7:39:27 AM UTC-5, Harald Schilly wrote: >>> >>> >>> >>> On Sunday, March 5, 2017 at 2:02:26 AM UTC+1, Robert Lowe wrote: >>>> >>>> >>>> I couldn't find anything that looked like a migration script. Is there >>>> one in the code base? >>>> >>> >>> I think you're looking for >>> >>> https://github.com/sagemathinc/smc/tree/master/src/scripts/postgresql/migrate >>> >>> It's some one-off code william wrote and I don't know if it would also >>> work for your case. There were a couple of issues with the json encoding, >>> too, so maybe there is even some manual work involved. >>> >>> -- harald >>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "sage-cloud" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to sage-cloud+...@googlegroups.com . >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/sage-cloud/9299300a-bbbc-410d-a41d-7695acee6d4a%40googlegroups.com >> >> <https://groups.google.com/d/msgid/sage-cloud/9299300a-bbbc-410d-a41d-7695acee6d4a%40googlegroups.com?utm_medium=email_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- > > Best Regards, > William Stein > > CEO, SageMath, Inc. > -- You received this message because you are subscribed to the Google Groups "sage-cloud" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-cloud+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-cloud/1229738e-8d1e-4a01-8329-cc0b629ee5c0%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
