Re: [sage-cloud] Installation Problems

[William Stein]

Rob,

Thanks for everything.   I made a tiny doc change just now to address
the first issue you had:

https://github.com/sagemathinc/smc/commit/ecfa2b66b8b602ff7fd98c3a686a03e985d06880


On Fri, Aug 19, 2016 at 10:36 AM, Robert Lowe  wrote:
> Ahahaha!  I figured it out!
>
> My server's hostname is read by compute and that is the hostname it uses.
> However, when the localhub runs, it binds to the first ip address on my
> server, but the hostname resolved to 127.0.0.1.  So I had to change
> /etc/hosts so that the hostname (mccs) resolved to that domain and then
> bingo I have the ability to pull raw files from within running projects.
> The reason why it worked with public folders when I was logged out is that
> in those instances the traffic passes from compute to hub, whereas with a
> running project it goes from localhub to hub.
>
> The upshot of all this is, I have now read like 90% of the hub source code
> and I feel a lot more confident in administering the system now.  Perhaps if
> time permits I may contribute some more detailed documentation to the
> project.  The code itself is reasonably well organized, but without details
> of its design it can be a bit like unraveling a cable knit sweater when it
> comes to diagnosing issues.  (Such is the reality of all distributed
> applications!)
>
> Now I just have to to get jupyter going and then I can spend the weekend
> writing custom code!
>
>
> On Friday, August 19, 2016 at 1:09:47 PM UTC-4, Robert Lowe wrote:
>>
>> Another part of the puzzle has emerged.  I can get access to raw files
>> through the proxy if they are in a publicly shared folder.  However, if I'm
>> doing a raw pull within a project in a non-public folder I get a bad
>> gateway.
>>
>> This tells me something is going on inside smc, probably with an
>> authentication token of some kind.  I noticed in proxy.coffee that it looks
>> like that is how it determines if it serves a file or not.  The section in
>> question reads:
>>
>> if not misc.path_is_in_public_paths(path, public_paths)
>> # The requested path is not public, so nothing to do.
>> cb()
>> else
>> # The requested path *is* public, so we get the file
>> # from one (of the potentially many) compute servers
>> # that has the file -- (right now this is implemented
>> # via sending/receiving JSON messages and using base64
>> # encoding, but that could change).
>> 
>>
>> It's non-obvious whether the earlier code in proxy.coffee can handle an
>> authenticated user's files.  I assume it can, as this capability exists in
>> the live cloud.sagemath.org server.  Alternatively, perhaps the master
>> branch code is not actually the code currently running on SMC.  When I try
>> manually talking to the proxy, and I send the same header sent by my web
>> browser, I get a blank response.  The proxy doesn't even send a header back,
>> which tells me that we are likely visiting this segment of the code even
>> while authenticated.
>>
>> I'm also beginning to think I'm alone in this, and once I do figure it out
>> I will join an elite group of individuals who have managed to install SMC.
>>
>>
>> On Friday, August 19, 2016 at 9:52:41 AM UTC-4, Robert Lowe wrote:
>>>
>>> I am making progress!  I have discovered the secret handshake which
>>> starts the hub on a single server, and how to run the compute daemon.  I'm
>>> basically just running them as they are in the dev/single directory.
>>>
>>> So project starts, I can use the terminal, and I can create and edit
>>> files.  Now, though, I can't view latex previews.  It builds the latex
>>> documents just fine, but preview does not work.  Looking under the hood a
>>> bit, I see that the rendered png files cannot download over the /raw links.
>>> I did a little more checking and discovered that raw download does not work
>>> for any file, regardless of where it is!  The error I get when I go to one
>>> of these raw links is:
>>>
>>> 502 Bad Gateway
>>>
>>> I looked in the nginx and haproxy logs and saw no errors.  The hub and
>>> compute similarly report what looks like should be successful.  Has anyone
>>> got any ideas where I should look?
>>>
>>> On Thursday, August 18, 2016 at 8:00:07 PM UTC-4, Robert Lowe wrote:

 What I'm really after is running my own SMC installation for my
 students.  The commercial servers are far too bogged down, even with paid
 tiers.  Ultimately, I will be running a customized version which will only
 allow maryvillecollege.edu addresses to sign up, and have a few other
 changes that I have in mind.

 I can run the laptop one just fine, I want to set up a real installation
 though.

 On Thursday, August 18, 2016 at 7:55:38 PM UTC-4, Harald Schilly wrote:
>
> Hi, if you want to do dev work, then you should 

Re: [sage-cloud] Installation Problems

[Robert Lowe]

Another part of the puzzle has emerged.  I can get access to raw files 
through the proxy if they are in a publicly shared folder.  However, if I'm 
doing a raw pull within a project in a non-public folder I get a bad 
gateway.  

This tells me something is going on inside smc, probably with an 
authentication token of some kind.  I noticed in proxy.coffee that it looks 
like that is how it determines if it serves a file or not.  The section in 
question reads:

if not misc.path_is_in_public_paths(path, public_paths)
# The requested path is not public, so nothing to do.
cb()
else
# The requested path *is* public, so we get the file
# from one (of the potentially many) compute servers
# that has the file -- (right now this is implemented
# via sending/receiving JSON messages and using base64
# encoding, but that could change).


It's non-obvious whether the earlier code in proxy.coffee can handle an 
authenticated user's files.  I assume it can, as this capability exists in 
the live cloud.sagemath.org server.  Alternatively, perhaps the master 
branch code is not actually the code currently running on SMC.  When I try 
manually talking to the proxy, and I send the same header sent by my web 
browser, I get a blank response.  The proxy doesn't even send a header 
back, which tells me that we are likely visiting this segment of the code 
even while authenticated.

I'm also beginning to think I'm alone in this, and once I do figure it out 
I will join an elite group of individuals who have managed to install SMC.  


On Friday, August 19, 2016 at 9:52:41 AM UTC-4, Robert Lowe wrote:
>
> I am making progress!  I have discovered the secret handshake which starts 
> the hub on a single server, and how to run the compute daemon.  I'm 
> basically just running them as they are in the dev/single directory.
>
> So project starts, I can use the terminal, and I can create and edit 
> files.  Now, though, I can't view latex previews.  It builds the latex 
> documents just fine, but preview does not work.  Looking under the hood a 
> bit, I see that the rendered png files cannot download over the /raw links. 
>  I did a little more checking and discovered that raw download does not 
> work for any file, regardless of where it is!  The error I get when I go to 
> one of these raw links is:
>
> 502 Bad Gateway
>
> I looked in the nginx and haproxy logs and saw no errors.  The hub and 
> compute similarly report what looks like should be successful.  Has anyone 
> got any ideas where I should look?
>
> On Thursday, August 18, 2016 at 8:00:07 PM UTC-4, Robert Lowe wrote:
>>
>> What I'm really after is running my own SMC installation for my students. 
>>  The commercial servers are far too bogged down, even with paid tiers. 
>>  Ultimately, I will be running a customized version which will only allow 
>> maryvillecollege.edu addresses to sign up, and have a few other changes 
>> that I have in mind.
>>
>> I can run the laptop one just fine, I want to set up a real installation 
>> though.
>>
>> On Thursday, August 18, 2016 at 7:55:38 PM UTC-4, Harald Schilly wrote:
>>>
>>> Hi, if you want to do dev work, then you should run it via the 
>>> instructions in src/dev/laptop 
>>>
>>> The projects are all isolated, such that you do not get any 
>>> information like processes and stuff for who on the compute nodes. 
>>> There is nothing special going on, except for cgroups. The real 
>>> problem might be, that you have to do additional configuration in the 
>>> database, but I'm not sure what you really did. 
>>>
>>> One day in the future, there will be configurations for kubernetes. 
>>> You can check that progress in the k8s subdirectory. 
>>>
>>> -- harald 
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-cloud" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-cloud+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-cloud/43ed5d13-ed11-4011-91f1-4d57921f2734%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-cloud] Installation Problems

[Robert Lowe]

What I'm really after is running my own SMC installation for my students. 
 The commercial servers are far too bogged down, even with paid tiers. 
 Ultimately, I will be running a customized version which will only allow 
maryvillecollege.edu addresses to sign up, and have a few other changes 
that I have in mind.

I can run the laptop one just fine, I want to set up a real installation 
though.

On Thursday, August 18, 2016 at 7:55:38 PM UTC-4, Harald Schilly wrote:
>
> Hi, if you want to do dev work, then you should run it via the 
> instructions in src/dev/laptop 
>
> The projects are all isolated, such that you do not get any 
> information like processes and stuff for who on the compute nodes. 
> There is nothing special going on, except for cgroups. The real 
> problem might be, that you have to do additional configuration in the 
> database, but I'm not sure what you really did. 
>
> One day in the future, there will be configurations for kubernetes. 
> You can check that progress in the k8s subdirectory. 
>
> -- harald 
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-cloud" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-cloud+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-cloud/02439746-9a64-4d6e-823a-b0423ca3feed%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-cloud] Installation Problems

[Harald Schilly]

Hi, if you want to do dev work, then you should run it via the
instructions in src/dev/laptop

The projects are all isolated, such that you do not get any
information like processes and stuff for who on the compute nodes.
There is nothing special going on, except for cgroups. The real
problem might be, that you have to do additional configuration in the
database, but I'm not sure what you really did.

One day in the future, there will be configurations for kubernetes.
You can check that progress in the k8s subdirectory.

-- harald

-- 
You received this message because you are subscribed to the Google Groups 
"sage-cloud" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-cloud+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-cloud/CAGG4CB755Dynxm-7%3D_%2Bv92JoEqaTdrKfc_jo7pZ%3DiddqAF5gwA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.