Re: [web2py] Re: Redirecting to a URL, but with changed html
So test() is going to have to grab the cookie and then include it with the urlopen(). Seems kind of weird.
[web2py] Re: Redirecting to a URL, but with changed html
Well this has all been a good education to me about how cookies work. I had not previously thought too hard about cookies, but just imagined that somehow the website communicated with the browser behind the scenes to get the cookie content when it needed it. From the responses and a little exploring I now realise that when the browser sends a url request to a website, it sends the contents of all the cookies associated with that website along with the request. Clearly as this happens on every url request it is expensive. However it is completely secure. If there were behind the scenes communication it could be spoofed. So in answer to one of my original questions there is no way that a redirect(some website) done from within web2py code cannot be the same as 'some website' being input in the users browser because the users browser version will include the cookie for the website. From the web2py code there is absolutely no way that the cookie for 'some website' can be retrieved. Thanks for all the input on this. Peter On Dec 14, 8:31 am, pbreit pbreitenb...@gmail.com wrote: So test() is going to have to grab the cookie and then include it with the urlopen(). Seems kind of weird.
[web2py] Re: Redirecting to a URL, but with changed html
Thanks for your answers. I guess editing the html and replacing the relative URLs with absolute ones is not too bad. The problem I am left with is this: Can one 'call' a URL from web2py as if the URL were called from the users browser?, but capture the html result? I imagine the answer to this question is no you cannot, because if you could it would open up a Pandora's box wrt to security etc. Peter On Dec 13, 1:05 am, Constantine Vasil thst...@gmail.com wrote: yes, redirect is little tricky ;) my solution is for ajax.
[web2py] Re: Redirecting to a URL, but with changed html
On Tuesday, December 13, 2011 4:15:22 AM UTC-5, peter wrote: Thanks for your answers. I guess editing the html and replacing the relative URLs with absolute ones is not too bad. The problem I am left with is this: Can one 'call' a URL from web2py as if the URL were called from the users browser?, but capture the html result? Not sure what you mean. Are you talking about something other than urllib.urlopen(url)?
[web2py] Re: Redirecting to a URL, but with changed html
Here is a simple example of what I mean: If I type http://127.0.0.1:8000/admin/default/site at my browser when I am logged in, I get the installed applications page. However if I do it by calling the test routine below def test(): import urllib url=http://127.0.0.1:8000/admin/default/site; f = urllib.urlopen(url) s = f.read() return s I have to login again even though I am logged in on my computer. So the question is, can I somehow add the 'context' to s so that it recognises that I am already logged in? I am using web2py's admin as an example, but the solution should work for any website. Thanks Peter On Dec 13, 1:51 pm, Anthony abasta...@gmail.com wrote: On Tuesday, December 13, 2011 4:15:22 AM UTC-5, peter wrote: Thanks for your answers. I guess editing the html and replacing the relative URLs with absolute ones is not too bad. The problem I am left with is this: Can one 'call' a URL from web2py as if the URL were called from the users browser?, but capture the html result? Not sure what you mean. Are you talking about something other than urllib.urlopen(url)?
Re: [web2py] Re: Redirecting to a URL, but with changed html
On Dec 13, 2011, at 11:09 AM, peter wrote: Here is a simple example of what I mean: If I type http://127.0.0.1:8000/admin/default/site at my browser when I am logged in, I get the installed applications page. However if I do it by calling the test routine below def test(): import urllib url=http://127.0.0.1:8000/admin/default/site; f = urllib.urlopen(url) s = f.read() return s I have to login again even though I am logged in on my computer. So the question is, can I somehow add the 'context' to s so that it recognises that I am already logged in? I am using web2py's admin as an example, but the solution should work for any website. Generally speaking, you need to include a cookie with your request so that the server (web2py in this case) can associate the request with the proper session. I can't help you much with the coding details, but you might want to look at using urllib2 instead of urllib. The cookie you want is somewhere in request, I think.
[web2py] Re: Redirecting to a URL, but with changed html
On Tuesday, December 13, 2011 2:09:31 PM UTC-5, peter wrote: def test(): import urllib url=http://127.0.0.1:8000/admin/default/site; f = urllib.urlopen(url) s = f.read() return s I have to login again even though I am logged in on my computer. So the question is, can I somehow add the 'context' to s so that it recognises that I am already logged in? I am using web2py's admin as an example, but the solution should work for any website. Are you saying you want to programmatically login to a website and then retrieve a particular page that requires a login on that site? Look into urllib2: http://docs.python.org/howto/urllib2.html. I think it's pretty easy if the site supports basic authentication, otherwise a little more complex (requires posting form data and managing cookies). Anthony
[web2py] Re: Redirecting to a URL, but with changed html
So in my example above. How does web2py's admin know I am logged in? Does it do this by accessing a cookie on my computer, if so how does it do that behind the scenes? Could urllib2.urlopen really handle this? Thanks Peter On Dec 13, 7:27 pm, Anthony abasta...@gmail.com wrote: On Tuesday, December 13, 2011 2:09:31 PM UTC-5, peter wrote: def test(): import urllib url=http://127.0.0.1:8000/admin/default/site; f = urllib.urlopen(url) s = f.read() return s I have to login again even though I am logged in on my computer. So the question is, can I somehow add the 'context' to s so that it recognises that I am already logged in? I am using web2py's admin as an example, but the solution should work for any website. Are you saying you want to programmatically login to a website and then retrieve a particular page that requires a login on that site? Look into urllib2:http://docs.python.org/howto/urllib2.html. I think it's pretty easy if the site supports basic authentication, otherwise a little more complex (requires posting form data and managing cookies). Anthony
Re: [web2py] Re: Redirecting to a URL, but with changed html
On Dec 13, 2011, at 12:28 PM, peter wrote: So in my example above. How does web2py's admin know I am logged in? Does it do this by accessing a cookie on my computer, if so how does it do that behind the scenes? Could urllib2.urlopen really handle this? Ordinarily it knows you're logged in because: 1. When you first log in, remembers it in a session and returns a cookie to your browser. 2. On subsequent requests, your browser includes that cookie, and web2py uses it to find the session (and its login state). (That's why you can find the cookie in request.cookies.) So you need to take the session cookie and (like the browser) send it along with the urllib2 request. (I could be missing something, but I don't think you need basic auth here.) Thanks Peter On Dec 13, 7:27 pm, Anthony abasta...@gmail.com wrote: On Tuesday, December 13, 2011 2:09:31 PM UTC-5, peter wrote: def test(): import urllib url=http://127.0.0.1:8000/admin/default/site; f = urllib.urlopen(url) s = f.read() return s I have to login again even though I am logged in on my computer. So the question is, can I somehow add the 'context' to s so that it recognises that I am already logged in? I am using web2py's admin as an example, but the solution should work for any website. Are you saying you want to programmatically login to a website and then retrieve a particular page that requires a login on that site? Look into urllib2:http://docs.python.org/howto/urllib2.html. I think it's pretty easy if the site supports basic authentication, otherwise a little more complex (requires posting form data and managing cookies). Anthony
Re: [web2py] Re: Redirecting to a URL, but with changed html
On Tuesday, December 13, 2011 3:45:34 PM UTC-5, Jonathan Lundell wrote: On Dec 13, 2011, at 12:28 PM, peter wrote: So in my example above. How does web2py's admin know I am logged in? Does it do this by accessing a cookie on my computer, if so how does it do that behind the scenes? Could urllib2.urlopen really handle this? Ordinarily it knows you're logged in because: 1. When you first log in, remembers it in a session and returns a cookie to your browser. 2. On subsequent requests, your browser includes that cookie, and web2py uses it to find the session (and its login state). (That's why you can find the cookie in request.cookies.) So you need to take the session cookie and (like the browser) send it along with the urllib2 request. (I could be missing something, but I don't think you need basic auth here.) Sounds about right. So, basically, you would post the login form values to the login URL, get back the session cookie, and then send the session cookie back with the subsequent page request so the app would recognize you as authenticated.
Re: [web2py] Re: Redirecting to a URL, but with changed html
On Dec 13, 2011, at 12:54 PM, Anthony wrote: On Tuesday, December 13, 2011 3:45:34 PM UTC-5, Jonathan Lundell wrote: On Dec 13, 2011, at 12:28 PM, peter wrote: So in my example above. How does web2py's admin know I am logged in? Does it do this by accessing a cookie on my computer, if so how does it do that behind the scenes? Could urllib2.urlopen really handle this? Ordinarily it knows you're logged in because: 1. When you first log in, remembers it in a session and returns a cookie to your browser. 2. On subsequent requests, your browser includes that cookie, and web2py uses it to find the session (and its login state). (That's why you can find the cookie in request.cookies.) So you need to take the session cookie and (like the browser) send it along with the urllib2 request. (I could be missing something, but I don't think you need basic auth here.) Sounds about right. So, basically, you would post the login form values to the login URL, get back the session cookie, and then send the session cookie back with the subsequent page request so the app would recognize you as authenticated. I think it's easier than that, if you're already logged in, which I think is the case here. The session cookie is already in request.cookies.
Re: [web2py] Re: Redirecting to a URL, but with changed html
I think it's easier than that, if you're already logged in, which I think is the case here. The session cookie is already in request.cookies. It sounded like he wanted to do this with external websites, not necessarily the current app (or even a web2py app).
Re: [web2py] Re: Redirecting to a URL, but with changed html
On Dec 13, 2011, at 1:40 PM, Anthony wrote: I think it's easier than that, if you're already logged in, which I think is the case here. The session cookie is already in request.cookies. It sounded like he wanted to do this with external websites, not necessarily the current app (or even a web2py app). If that's the case, then yes, he'd have to imitate the browser sequence for a login.
Re: [web2py] Re: Redirecting to a URL, but with changed html
Yeah, he's going to need to first script a login so he can grab and store the cookie which he is then going to need to present on each urllib.urlopen(). Being logged in on your browser is not going to help since your browser and your web2py code are totally separate. He could possibly make it easier by not protecting the functions with Web2py's authentication and instead inserting a secret header or cookie variable or arg/var.
Re: [web2py] Re: Redirecting to a URL, but with changed html
On Dec 13, 2011, at 5:22 PM, pbreit wrote: Yeah, he's going to need to first script a login so he can grab and store the cookie which he is then going to need to present on each urllib.urlopen(). Being logged in on your browser is not going to help since your browser and your web2py code are totally separate. Is that true? I was thinking that if the urllib2 request is being made off a browser request, the existing cookie could be used to effectively run the urllib2 request under the existing authentication umbrella. If the whole sequence is triggered from a post-authentication browser request... He could possibly make it easier by not protecting the functions with Web2py's authentication and instead inserting a secret header or cookie variable or arg/var.
Re: [web2py] Re: Redirecting to a URL, but with changed html
Oh, I see. If the person logs in to a web2py app and then hits a page which is going to do the urlopen that calls the same app, then I suppose you could grab the cookie and send it along. Is that really what id being contemplated? I wonder if a LOAD() would suffice?
Re: [web2py] Re: Redirecting to a URL, but with changed html
On Tuesday, December 13, 2011 8:36:47 PM UTC-5, pbreit wrote: Oh, I see. If the person logs in to a web2py app and then hits a page which is going to do the urlopen that calls the same app, then I suppose you could grab the cookie and send it along. Is that really what id being contemplated? The original example was for url of some web page, which I assume is not in the same app (or necessarily even a web2py app at all). If it's in the same app, or any web2py app under your control, there should be better ways to return the output of a particular function/view.
Re: [web2py] Re: Redirecting to a URL, but with changed html
On Dec 13, 2011, at 6:20 PM, Anthony wrote: The original example was for url of some web page, which I assume is not in the same app (or necessarily even a web2py app at all). If it's in the same app, or any web2py app under your control, there should be better ways to return the output of a particular function/view. Here's Peter's example: Here is a simple example of what I mean: If I type http://127.0.0.1:8000/admin/default/site at my browser when I am logged in, I get the installed applications page. However if I do it by calling the test routine below def test(): import urllib url=http://127.0.0.1:8000/admin/default/site; f = urllib.urlopen(url) s = f.read() return s I have to login again even though I am logged in on my computer. So the question is, can I somehow add the 'context' to s so that it recognises that I am already logged in? I am using web2py's admin as an example, but the solution should work for any website. If the urllib access is to the current (web2py) app, then the caller (test) is in a special position WRT the browser: it has possession of the relevant cookies. If the urllib access is to some arbitrary (third-party) website that the client browser happens to be logged in to, then this does not apply.
[web2py] Re: Redirecting to a URL, but with changed html
I guess that's not technically a redirect -- it's actually copying the other page and delivering the copy. What do you mean it's no longer connected to the session? The difficulty with copying a web page and delivering the copy is that it might contain some relative URL references (links as well as static resources) that will no longer work when being delivered from your server. You would have to rewrite the URLs to make them absolute (including URLs within any static files linked in the page) and/or copy the static resources to your own server. I'm assuming the other URL isn't another web2py app that you control, right? Anyway, Massimo just released Plasmid, which does exactly this (clones an external page, converts relative to absolute references): https://github.com/mdipierro/Plasmid. You might be able to make use of that. Anthony On Monday, December 12, 2011 5:10:40 PM UTC-5, peter wrote: If I have a function as follows in default.py def test(): redirect(url of some web page) Then app/default/test does open up the web page as if it was entered in the url bar of the browser. If the function is def test(): import urllib url=url of some web page f = urllib.urlopen(url) s = f.read() return s Then it does kind of redirect, but it is no longer connected to the session, and the defaults in the html are no longer correct. Is it possible to alter the routine above to have the same functionality as redirect? The reason I want to be able to do this is that I want to edit s. So I want to do a redirect, but with slightly edited html. Any ideas Thanks Peter
[web2py] Re: Redirecting to a URL, but with changed html
You would have to rewrite the URLs to make them absolute (including URLs within any static files linked in the page) Correction -- you probably don't have to worry about relative URLs within linked CSS files because they will be loaded relative to the CSS file URL (so only that URL needs to be converted to absolute).
[web2py] Re: Redirecting to a URL, but with changed html
Spend all week struggling with the same.
Basically you have to use browsers'
window.location.replace and initialize
the location via Python from your function.
Let say you are here:
http://www.mycoolapp.com/old_location
in the html you place the JavaScript
at the bottom before /body:
relocation_url = http://www.mycoolapp.com/old_location;
window.location.replace({{=relocation_url}})
When you want to redirect and the new html to replace the old one:
you initialize:
relocation_url = http://www.mycoolapp.com/new_location;
When the browser loads the page, close to the /body (end of page)
it sees actually window.location.replace
(http://www.mycoolapp.com/new_location;)
And it will reload the page with the new location in the browser bar
and also the new html.
If you use just the web2py redirect, it loads the html but you see
the old url in the browser address bar. If the new page is a form,
you fill out the form, hit the Submit button and nothing happens
because the browser sees the old address bar. Struggled all week with that
and only solution I found is that.
Good luck and give back to community ;)
[web2py] Re: Redirecting to a URL, but with changed html
web2py redirect issues a standard http redirect, so the address will change
in the browser address bar. I believe you were using jQuery Mobile and
Ajax, which apparently introduces some complications (not specific to
web2py). For a regular application, a standard redirect should work fine.
In any case, I don't think peter is looking for just a redirect -- he wants
to grab a copy of a page, edit it, and return the edited version -- quite a
different task.
Anthony
On Monday, December 12, 2011 6:44:01 PM UTC-5, Constantine Vasil wrote:
Spend all week struggling with the same.
Basically you have to use browsers'
window.location.replace and initialize
the location via Python from your function.
Let say you are here:
http://www.mycoolapp.com/old_location
in the html you place the JavaScript
at the bottom before /body:
relocation_url = http://www.mycoolapp.com/old_location;
window.location.replace({{=relocation_url}})
When you want to redirect and the new html to replace the old one:
you initialize:
relocation_url = http://www.mycoolapp.com/new_location;
When the browser loads the page, close to the /body (end of page)
it sees actually window.location.replace(
http://www.mycoolapp.com/new_location;)
And it will reload the page with the new location in the browser bar
and also the new html.
If you use just the web2py redirect, it loads the html but you see
the old url in the browser address bar. If the new page is a form,
you fill out the form, hit the Submit button and nothing happens
because the browser sees the old address bar. Struggled all week with that
and only solution I found is that.
Good luck and give back to community ;)
[web2py] Re: Redirecting to a URL, but with changed html
yes, redirect is little tricky ;) my solution is for ajax.
