[web2py] auth.user_group_role() method
This method seems to be still in trunk (http://code.google.com/p/web2py/source/browse/gluon/tools.py#2832), as I cannot find it in the latest stable. It takes a user id and return the name/role of the group that user is in. However, it doesn't seem to be able to work with custom groups. For example, if I do auth.settings.create_user_groups = False and create two custom groups (e.g. 'students' and 'teachers') to accomodate all the users. This auth.user_group_role() method probably will fail because it assume that every group follows the naming convention which is set by: http://code.google.com/p/web2py/source/browse/gluon/tools.py#963. Can someone confirm whether this is indeed the case. Thanks! --
[web2py] Auth.register() questions
I have a couple of questions about the provided Auth.register() function (I suppose they also apply to other Auth functions in general): 1. In a controller, if I do: def register(): return dict(form=auth.register()) with the following settings in a model: auth.settings.registration_requires_verification = True auth.settings.register_next = URL (...) Everything works as expected, i.e. the verification email is sent after the registration form is processed, and the user gets re-directed to a different page. However, as soon as I do sth like: def register(): form = auth.register() # then do sth with the form return dict(form=form) Both verification email and re-direction stop working even though the form processed successfully. I wonder what causes this breakdown? I am using the trunk version of web2py. 2. How much customization can we do with the form returned by auth.register()? I know we can add add fields to it (a few different ways), but can we modify its class? For example, I am able to do sth like: form.element(_type='submit')['_class'] = '...' to change the class of its elements. How do we change the class for the whole form? How do we position each field? How do we position the whole form on the page? etc. Thanks in advance!
[web2py] Re: A page served by two different backends
I see your point. Perhaps I should revise my definition of 'static' and 'dynamic': when I generate an HTML using a view of web2py, everything returned by the server will stay the same and will potentially change only after I refresh the page - this is what I mean by static. On the other hand, on the same page I have a websockets connection to a tornado server, so that some contents of the page will update themselves (via Javascript) without having to refresh the page - this is what I mean by dynamic. My question now is that, having these two backends (web2py and tornado), how should I setup this page on which the majority of the content are served by web2py and stay the same once they're loaded, while some content constantly update themselves through websockets to tornado? Thanks. On Monday, July 2, 2012 12:14:15 PM UTC-4, Wikus van de Merwe wrote: Your description is very confusing. Static content are the files that are not dynamically generated but simply served over the network, e.g. images, javascript, css. If you generate an HTML page using a view (template) and filling it with data from DB, that is a dynamic page generation. Now, you want to use tornado + javascript to have a user side generated page. And you want access to DB from the javascript? If so, try the REST approach and expose the data as JSON. See this book chapter for details: http://web2py.com/books/default/chapter/29/10#HTML,-XML,-and-JSON But then the question is, what is the benefit of using tornado in the first place if at the end your requests go to web2py anyway?
[web2py] A page served by two different backends
I want to implement a page that is served by two backends, one is tornado and the other is web2py, and this concerns two questions below: 1. The tornado serves the dynamic part of the content on the page (together with some help from frontend Javascript), while the web2py handles the rest of all static content. Is this something that can be easily configured and handled by web2py? If so, how should I setup it up the page? 2. By using a single user action (e.g. clicking on a button), I want to retrieve some static content (e.g. some parameter values stored in the db behind web2py) from web2py, and pass on these values to the tornado server. What is the appropriate workflow/logic for implementing this within web2py? Is it simple as send a request to web2py, get the values back and then programmatically generate another POST request to tornado? Thanks.
[web2py] Re: A page served by two different backends
By 'static content', I mean things like the layout, links, labels, buttons, input fields, tables, etc. On the other hand, values of the labels and those of the table are dynamic, such that they refreshes themselves as new values arrive via websockets. Every other pages of the app will be served using web2py, which is why I want to keep things consistent and have web2py serve all the static content of this page too.
[web2py] Two questions about models
Hi, 1. In the web2py book, there is an example for Auth settings, which presumably resides at top-level inside a model: auth.settings.register_onaccept.append(lambda form: mail.send(to= 'y...@example.com', subject='new user', message=new user email is %s' % form.vars.email)) and I wonder where exactly do these 'form' and 'form.vars.email' variables come from and how do we know which specific variables are available for which functions in web2py, such information seems to be poorly documented in the manual. 2. Is it advisable to organize models into different folders named by their related controllers? For example, for all account-related functionalities, I create a model called 'account.py' and put it at 'my_app/models/account/account.py', so that only this model will be executed for all account-related requests. Similarly, a model named 'content.py' is created in 'my_app/models/content/content.py', so that only this model will be executed for all the requests relating to user-generated content. By doing this (as opposed to put all the models in 'my_app/models/'), we can avoid executing models we don't need for certain requests, but we may need to duplicate some code (e.g. create db connections) for different models. Is this a good practice in web2py? --
[web2py] Re: Two questions about models
Thanks guys! Very helpful comments and explanations!
[web2py] Re: The state of web2py + bootstrap 2
Hi Anthony, I was looking at Carousel JS plugin. For example, I've seen a recent thread (https://groups.google.com/forum/?fromgroups#!searchin/web2py/carousel/web2py/LZqq06VAUZs/zyQIOgqZKnQJ) where people encounter conflicts between web2py.css and bootstrap.min.css, when using Carousel plugin. Another thread (https://groups.google.com/forum/?fromgroups#!searchin/web2py/bootstrap/web2py/YqdwW3jIZ_I/WSLcKvTULz8J) seems to suggest that the top menu bar from Bootstrap 2 doesn't work properly with web2py. To clarify, I don't need to use ready-made themes such as those from Bootswatch, and will just use provided components from Bootstrap 2, and customize the details such as colors, icons, etc myself. Does it make things easier? Just to confirm, do I need the latest trunk version to have Bootstrap 2 support, or the latest stable 1.99.7 is fine? Thanks! --
[web2py] Re: The state of web2py + bootstrap 2
I see. Note, web2py.css is designed primarily to work with the welcome app. You don't have to use web2py.css or the welcome app at all, though. You can instead create the entire front end with pure Bootstrap, and that should work fine with web2py (if you're using the web2py grid, you might want to retain the grid-specific CSS from web2py.css). Is this the case even if I just use the latest stable instead of trunk? If I ditch web2py.css completely and don't base my front-end on the 'welcome' app, and use purely Bootstrap (bootstrap.min.css), then there shouldn't be any restriction imposed by web2py? I see many people seem to prefer to work with web2py.css as a base, and customize stuff as they go, I wonder if there is any specific advantage to this approach (better supported by web2py? but I doubt so considering web2py is supposed to be a generic enough framework). Thanks! On Sunday, June 24, 2012 2:41:41 PM UTC-4, Anthony wrote: Anyway, trunk is based on Bootstrap, but I'm not sure web2py.css has been tested to ensure there are no conflicts with any Bootstrap components. If you find a conflict, you can of course change the CSS (and report it so it can be fixed). Anthony --
[web2py] The state of web2py + bootstrap 2
Hi, I've seen a couple of threads in the group discussing bringing bootstrap 2 together with web2py, and someone mentioned that this will become true in the coming web2py release. I just wonder what state this development is in now, are we finally going to be able to use all the features offered by bootstrap 2 with web2py, if not, what are the limitations? Many thanks. --
[web2py] Deployment on VPS with virtualenv
I intend to deploy web2py using Nginx and uwsgi on a VPS. On the same
server, I also installed virtualenv to manage other python packages needed
for my application.
At the end of this this setup
scripthttp://code.google.com/p/web2py/source/browse/scripts/setup-web2py-nginx-uwsgi-ubuntu.sh,
these lines seem to deploy web2py:
chown -R www-data:www-data web2py
cd /home/www-data/web2py
sudo -u www-data python -c from gluon.main import save_password;
save_password('$PW',443)
/etc/init.d/uwsgi restart
/etc/init.d/nginx restart
which brings up the first couple of questions I have:
1. what is the 'save_password' line for? It seems that it is executing some
python code to save down some pwd, but what is the pwd for?
2. does the above deployment method deploys every single app inside
'/applications' of web2py directory? if so, how do I selectively deploy
app? For example, I would only want to deploy '/applications/my_app/'
My next questions concern the involvement of virtualenv during deployment:
3. how exactly do I tell the Nginx and uwsgi to use the web2py code as well
as related packages inside a particular virtualenv? Is it done
programatically by some code, or by some config file(s)?
4. again, related to (2), how do I tell uwsgi to only deploy one specific
app in the web2py directory inside virtualenv?
Thanks a lot.
[web2py] Re: Questions about background process using homemade task queue
I strongly advicee the use of the scheduler because your requirements will be fullfilled best from that than a homemade task queue, at least, if not trying to use celery.anyway, just my 2 cents: SQLITE write operations are locking the entire database. one of 2 controllers (or modules, or scripts, or anything) trying to write into a single database will likely to have to wait the other one to finish. 1. with every other database that is not sqlite, it's safe because how most relational db work, transactions make you have always a consistent set of results (if you use them correctly!!) Thanks for all the info! I use PostgreSQL as the db, does that make any difference in terms of robustness and performance in this case? Are you suggesting that a beefier db like PostgreSQL *is* going to handle concurrent reads and writes comfortably? 2.you can, but installing rabbitmq just to manage a queue seems an overkill. at that point, use celery directly. I choose rabbitmq as the messages containing the updates to the db come from a separate server (tornado), so the update tasks are originated remotely but *not* on web2py server. I am not familiar with celery, does it do the same thing better here?
[web2py] Re: Questions about background process using homemade task queue
I think I'm not mistaken by saying that celery is the most used library for this kind of operations, it's written in python and founded on rabbitMQ (also on others, but primarily rabbit) to handle queues. Seems huge, but it's fairly easy to setup (especially if you planned to use rabbitmq anyway) and can be used with a few statements to make very simple tasks, but can be extremely fine-tuned for most of the requirements out there. A follow-up question, I'm a bit puzzled by the differences between Celery and RabbitMQ. Is Celery the same idea as the built-in scheduler of web2py? In particular, in my specific case, I just need to a channel which transports data from one server to the other, and receiving end will save these data into a database. For that, I guess RabbitMQ alone is sufficient. Why do I need an extra Celery task queue? Is it for some scaling consideration - that is, when I have multiple instances of server on each end? Thanks.
[web2py] Questions about background process using homemade task queue
Hi group, I am going to implement a homemade task queue as a background process for updating database, and I have some questions as below: 1. does a background process updating a database interfere with controller functions that operate on the same database? For example, if I have a task queue filled with a stream of database updates, will this background process block any other database actions performed through controller functions in web2py? Moreover, Is it recommended to run several background processes which will all read/write from/to the database? 2. given the mail queue example in the manual, it seems that a background process is usually a forever loop that processes things upon their 'arrivals'. In the mail example, this is done by checking records in the db so the queue is effectively a table of records. I wonder if I can simply replace this 'table queue' with a 'message queue' using RabbitMQ and Pika library, or there is more complication involved for such implementation? In addition, in web2py how should we detect and restart a background process when it dies unexpectedly. 3. according to the manual, a background process is run as a separate python module: python web2py.py -S app -M -N -R applications/app/private/task_queue.py Does the argument '-S app' have to match the name of the web2py application this module is running for? Second, in a production environment, what is the proper way of initiating a background process? In other words, is there a better way than manually invoking this from command line as we deploy the application? Any suggestion and comment is welcome. Thanks!
[web2py] Re: How to keep and manipulate data in memory with web2py
I've looked deeper into this, and in web2py doc: http://www.web2py.com/examples/static/epydoc/web2py.gluon.cache.CacheInRam-class.html, it mentions: This is implemented as global (per process, shared by all threads) dictionary. A mutex-lock mechanism avoid conflicts. Does this mean that when each request thread is accessing and modifying the content (e.g. a dictionary in my case) of the cache, every other cache is blocked and has to wait till the current request thread finishes with it. If so, it seems to me that the race condition we fear as above should not happen? Please correct me if I get this wrong. Thanks. Well, because cache.ram returns a reference to the object rather than a copy, I think you're OK in terms of avoiding conflicts when updating the dict (if you're just adding new keys). However, you might have to think about what happens when you hit the required number of entries. What if one request comes in with the final entry, but while that request is still processing (before it has cleared the cache), another request comes in -- what do you do with that new item? It may be workable, but you'll have to think carefully about how it should work. Thanks for confirming this. The only kind of updates would be adding new key-value pairs. Having confirmed that this sort of updates will be consistent on the cache, I am now a little worried about the performance of doing so, given the mention of the mutex-lock design of the cache. If I understand this correctly, each thread (from a request) will lock the cache so that all other threads (requests) will have to wait. I intend to store multiple dictionaries (say 10) in the cache, and each dictionary will handle the data from a fixed set of users (say 30 of them) for a given period of time. If the cache truly behaves as above, then when one thread is updating the cache, all the other 10 * 30 - 1 = 299 threads will be blocked and will have to wait. This might drag the efficiency of the server-side.
[web2py] Re: How to keep and manipulate data in memory with web2py
Thanks for confirming this. The only kind of updates would be adding new key-value pairs. Having confirmed that this sort of updates will be consistent on the cache, I am now a little worried about the performance of doing so, given the mention of the mutex-lock design of the cache. If I understand this correctly, each thread (from a request) will lock the cache so that all other threads (requests) will have to wait. I intend to store multiple dictionaries (say 10) in the cache, and each dictionary will handle the data from a fixed set of users (say 30 of them) for a given period of time. If the cache truly behaves as above, then when one thread is updating the cache, all the other 10 * 30 - 1 = 299 threads will be blocked and will have to wait. This might drag the efficiency of the server-side. As far as I can tell, the ram cache is not locked for the entire duration of the request -- it is only locked very briefly to delete keys, update access statistics, etc. So, I don't necessarily think this will pose a performance problem. Anthony That's great news! May I ask where you found these details in the source code? I'd just want to double-check and make sure it is the case, as this is important to my design and implementation. Many thanks again!
[web2py] Re: How to keep and manipulate data in memory with web2py
Very nice. Thank you!
[web2py] Re: How to keep and manipulate data in memory with web2py
Regarding the pros and cons of this approach (in comparison to David's database approach), I wonder what are the potential pitfalls/risks of the cache approach. For examples, but not limited to, 1. Is there any consistency issue for the data stored in web2py cache? Yes, this could be a problem. The nice thing about using a db with transactions is that you can ensure any operations get rolled back if an error occurs. In web2py, each request is wrapped in a transaction, so if there is an error during the request, any db operations during that request are rolled back. The other issues is volatility -- if your server goes down, you lose the contents of RAM, but not what's stored in the db (or written to a file). For each request, as long as it reaches the action without problem, then I suppose the subsequent updating to cache should be fairly straightforward. For the requests which carries the last piece of data to meet the trigger value, it will perform the processing as well as writing to the db, but as you said, this would be wrapped in a transaction. Regarding volatility, in my case, it doesn't matter if the data are written to the db at the point of crash - if it crashes, the whole process would have to start from scratch. 3. Is it thread-safe? For instance, if I have two threads A and B (two requests from different users) trying to access the same object (e.g. 'user_data' dict) stored in the cache at the same time, would that cause any problem? This especially concerns the corner case where A and B bear the very last two pieces of data expected to meet ' some_pre_defined_number'. That's a good point -- your current design would introduce a potential race condition problem (I was originally thinking each request would add separate entries to the cache, not update a single object). Of course, a db could have a similar problem if you were just repeatedly updating a single record (rather than inserting new records each time). I've looked deeper into this, and in web2py doc: http://www.web2py.com/examples/static/epydoc/web2py.gluon.cache.CacheInRam-class.html, it mentions: This is implemented as global (per process, shared by all threads) dictionary. A mutex-lock mechanism avoid conflicts. Does this mean that when each request thread is accessing and modifying the content (e.g. a dictionary in my case) of the cache, every other cache is blocked and has to wait till the current request thread finishes with it. If so, it seems to me that the race condition we fear as above should not happen? Please correct me if I get this wrong. Thanks.
[web2py] How to keep and manipulate data in memory with web2py
Hi group, Is it possible for one to store incoming user data (e.g. those submitted by users on a page) in memory, manipulate them in memory, and send them back with web2py? Or do I need to external modules/libraries to do that? So far, it seems by default all user submitted data are written to database in web2py. More specifically, I want to implement this sort of logic on the server side: the server waits for a pre-defined number of pieces data from different users, and once all the data are in, the server processes the set of data, saves the results in db and sends them back to respective users. The most obvious way I can think of is to keep (and track) the incoming data in memory and only write to the db after they're processed. Is this something that we can do using the existing functionalities of web2py? Thanks!
Re: [web2py] Re: How to keep and manipulate data in memory with web2py
That might be just slightly unsafe, don't you think, if a piece of data gets evicted before it's needed? I'd prefer to limit caching to data that can be recreated when it's not found in the cache. An alternative might be a memory-based SQLite database, taking care not to let it leak (a consideration regardless of the implementation) Thanks for the suggestions. But would a piece of data get removed from the cache arbitrarily/automatically? In other words, if we're sure of the size of data coming in and make sufficiently large cache for storing them, then would it be 100% safe to use cache to keep the data for my case here? Or am I missing something else? Your second suggestion sounds similar to using Redis, in which case would there still be a risk of leaking? Thank you.
[web2py] Re: How to keep and manipulate data in memory with web2py
If you just need to store data for a single user across requests, you can
store it in the session (each user has a separate session). But if you need
to store data from multiple users and then process it all together, you can
store it in the cache: http://web2py.com/books/default/chapter/29/4#cache.
Anthony
Thanks Anthony. The data come from multiple users in my case.
After looking at the caching mechanism provided by web2py, I have a rough
design in pseudo-code as below.
in controller:
user_data = cache.ram('user_data', lambda:dict(), time_expire=None)
# add the data from this user, this should also update the cached dict?
user_data[this_user_id] = submitted_data
if len(user_data) == some_pre_defined_number:
# get data out of the dictionary 'user_data' by user Ids
...
# process them and persist results to DB
...
# reset 'user_data' dict by removing it from cache
cache.ram('user_data', None)
The above implies that the 'if' check is made upon every request by
individual users, which is somewhat inefficient, but I am not sure if there
is other better way(s) to implement this logic within web2py. If so, please
enlighten me!
In addition, I have some specific questions regarding the cache mechanism
in web2py:
1. when exactly does an object expire in cache? My understanding is that,
as long as the 'time_expire' value of *the current request* is greater than
the gap between *the time of current request* and *the time at which the
requested object was last updated in cache*, then the object behaves as if
it never expires in cache. This is illustrated by an example in the book,
if I understand it correctly. Therefore, the only perceivable way (to me)
for an object to expire is to supply a 'time_expire' value in a request
that is smaller than the gap between the time of this request and the time
when the requested object was updated in cache. In other words, if I do:
something = cache.ram('something', lambda:'blah blah', time_expire=1)
effectively, 'something' never expires in cache until I call something like
the following after t seconds (with t make_it_expire):
something = cache.ram('something', lambda:'blah blah',
time_expire=make_it_expire
)
2. An follow-up question is that, when an objects eventually expires in
cache, does it mean it is completely removed from the cache? i.e.
equivalent to:
cache.ram(key, None)
or something else happens?
3. The manual says that by setting 'time_expire = 0', it forces the cached
object to be refreshed. Am I correct in understanding this as the cached
object will be immediately set to whatever is supplied by the second
argument of the 'cache.ram' call? Thus 'refreshed' here means 'updated'.
Your thoughts and comments are much appreciated. Thanks!
[web2py] Re: How to keep and manipulate data in memory with web2py
Databases are specially designed for keeping persistent data - so there is your answer! :) I suggest: 1. Write the data initially to a transitional Sqlite DB on disk. 2. Once all the data pieces have arrived, migrate the completed data to your main DB and delete all the transitional records. 3. It is much safer than that which you have proposed. Web2py can easily handle the two DBs. 4. If you need any housekeeping, set up a scheduled job to purge all the old, incomplete stuff once in a while. Regards, David Thanks David. This approach ensures better data consistency, but I have two concerns: 1. If we store everything in database, how should we track whether all the pieces of data have arrived. For example, if I am expecting 10 pieces of data from 10 users, do I then need to constantly poll the db to check this? Is there any other tracking/trigger mechanism available? 2. It seems to me that the following database operations are needed: 1. write to the transitional db; 2. check for data arrival; 3. once all data arrived, read from transitional db so we can process them; 4. write the results to the main db; 5. delete all data from the transitional db. Added together, all these db operations may be substantial, not to mention that this process may need to be repeated for many number of times. In this aspect, the cached solution seems to shine performance wise. Would love to hear your thoughts on the above. Thanks!
[web2py] Re: How to keep and manipulate data in memory with web2py
In general, the approach David suggests (
https://groups.google.com/d/msg/web2py/4cBbis_B1i0/KkKnNwUw8lcJ) is
probably preferable, but below are answers to your questions...
user_data = cache.ram('user_data', lambda:dict(), time_expire=None)
# add the data from this user, this should also update the cached dict?
user_data[this_user_id] = submitted_data
The above would not update the dict in the cache -- you'd have to do that
explicitly:
cache.ram('user_data', lambda: user_data, time_expire=[some positive value
]
Thank you for your insights, Anthony.
Regarding the pros and cons of this approach (in comparison to David's
database approach), I wonder what are the potential pitfalls/risks of the
cache approach. For examples, but not limited to,
1. Is there any consistency issue for the data stored in web2py cache?
2. Is there any size limit on the data stored in web2py cache?
3. Is it thread-safe? For instance, if I have two threads A and B (two
requests from different users) trying to access the same object (e.g.
'user_data' dict) stored in the cache at the same time, would that cause
any problem? This especially concerns the corner case where A and B bear
the very last two pieces of data expected to meet 'some_pre_defined_number'.
Thanks!
[web2py] Re: [video] websockets com web2py e tornado
Hi Bruno, Any chance of adding English subtitles to this video? Would be a great resource for everyone! Thanks!
[web2py] Re: Programmatically log in
That works! CRYPT() validator is temporarily removed from and added back to the list of db.auth_user.password.requires inside the same function. Thanks!
[web2py] Re: Adding permissions upon registration
Actually, looking at the code, auth.user.id should work as well, unless you have auth.settings.registration_requires_verification = True, in which case, session.auth.user.id won't work either (but form.vars.id will work in either case). I was going to update this post to reflect this, after some testing myself. I am indeed using email verification for registration, and I've noticed another potential gotcha: the verification email is sent before the register_onaccept callback, so if anything goes wrong during the callback (which I would still consider part of the registration process), the verification email will be sent regardless. I wonder if this would cause problem in some applications...
[web2py] Programmatically log in
I am trying to define a function which programmatically logs user in, e.g. def program_login: # log-in code this may be useful for some callback situations, e.g. auth.settings.verify_email_onaccept.append(program_login) in which case, user gets automatically logged in only after he/she has verified his/her email. so far, I've only found so-called manual authentication in the manual: user = auth.login_bare(username, password) but I am not so sure if this is what I need for the case above. In particular, how could I obtain the corresponding password for the user here? And I suppose it is encrypted?
[web2py] Adding permissions upon registration
Hi Group, I have something like the following in a model: def grant_permissions(form): group_id = auth.id_group(role=request.args(0)) auth.add_membership(group_id, auth.user.id) auth.settings.register_onaccept.append(grant_permissions) Basically, I want to assign a user to a group upon his/her successful registration. However, when I execute the above, I get an AttributeError: 'NoneType' object has no attribute 'id' . I tried to follow the code example on page 414 of the manual, but referencing using auth.user.id doesn't seem to work. So how can I get hold of the user id here for the user who has just successfully registered? Thanks. * *
[web2py] Re: Adding permissions upon registration
The new user id should be in form.vars.id as well as session.auth.user.id at that point. Thanks Anthony. But is this documented anywhere in the manual or in any doc of Web2py? Otherwise, it is really difficult for users to figure this out without consulting experts like you. You second point regarding security is well taken, and is taken care of in the logic of other parts of the code. Thanks.
[web2py] Re: auth.settings.create_user_groups and auth.settings.registration_requires_verification in Auth
Hi Anthony,
I've done more tests on my machine, and it appears that the first issue
does *not* lie with
auth.settings.create_user_groups = False
, but rather with the interaction between Web2py and PostgreSQL in general.
It seems to me that Web2py behaves inconsistently with respect to
auth.define_tables()
, in that I sometimes get all those (renamed) Auth tables created but other
times not.
The way I setup my database is as follows.
Use PostgreSQL command line (with username: cyan) to create the db:
createdb mydb
, and create the db object in the account.py model file:
db = DAL(postgres://cyan:123456@localhost:5432/mydb)
After these two steps and before starting Web2py, I check on the PostgreSQL
server running on the localhost and make sure that an empty 'mydb' indeed
exists. Then I start Web2py server, and navigate
to http://127.0.0.1:8000/myapp/account/register, where I type in email,
pwds, and hit 'register' button, upon which if Web2py fails to create those
Auth tables by executing
auth.define_tables()
in the model file 'account.py', I will get an error saying that the table
'user_login' (effectively 'auth_user') does not exist. But strangely
enough, sometimes tables do get created and this go through.
Regarding the second problem, I did the test you suggested above in Web2py
shell while my application is running, and I have no problem with sending
an email using
mail.send(to='someaddr...@gmail.com', message='test')
, however, it simply fails to send anything when I test it in the browser
(obviously, only when Web2py somehow creates all the necessary Auth
tables). Again, after I hit the 'register' button, I get re-directed to the
desired next page and a new record is created in 'user_login' ('auth_user')
table, but no email is sent. Thus I suspect that something might went wrong
with the code that is affected by
auth.settings.registration_requires_verification = True
, rather than the the code which actually handles sending the email. Or, in
fact, could it be related to the code which sets the content of the
verification email in the model file 'account.py':
## set the content of the verification email
auth.messages.verify_email = 'Please click on the link: http://' + \
request.env.http_host + \
URL(r=request,c='account',f='verify_email') + \
'/%(key)s to verify your email address'
Any suggestion from you would be extremely welcome. Thanks!
[web2py] Re: auth.settings.create_user_groups and auth.settings.registration_requires_verification in Auth
Hi Anthony, When I say 'sometimes Web2py fails to create Auth tables and other times it does', I mean running a test procedure like this: 1. if 'mydb' already exists, it is dropped from PostgreSQL server using 'dropdb mydb' 2. a fresh 'mydb' is created using 'createdb mydb' 3. then run the Web2py In other words, every time when I run the test there is a fresh database 'mydb' in place with no table. So I would expect Web2py create all the necessary Auth tables upon the execution of 'auth.define_tables()'. I just tried another test, I delete the whole app using Web2py appadmin, and re-create it from scratch. And it worked for auth.settings.create_user_groups = False part with all the Auth tables created for the first test, though verification email is still not sent. However, after I drop and re-create the database 'mydb' as above, and re-run the same Web2py app, the error comes back and none of the Auth tables gets created. FYI, I am running PostgreSQL 9.1 and latest stable Web2py 1.99.7 with the following db drivers available: SQLite3, pymysql, psycopg2, pg8000, IMAP. The syntax error in the email message code is due to post editing, whereas in the code I check that there is no extra space after '\'. Does it matter where I put the chunk of email-related code inside my model file, by which I mean: ## configure email mail = auth.settings.mailer mail.settings.server = 'smtp.gmail.com:587' mail.settings.sender = 'some.em...@gmail.com' mail.settings.login = 'some.email:123456' ## set the content of the verification email auth.messages.verify_email = 'Please click on the link: http://' + \ request.env.http_host + \ URL(r=request,c='account',f='verify_email') + \ '/%(key)s to verify your email address' I put them before 'auth.define_tables()', not sure if that would affect anything or not. Thanks a lot!
[web2py] Re: auth.settings.create_user_groups and auth.settings.registration_requires_verification in Auth
Hi Anthony,
Great insight! The removal of manual re-direction fixes the email failure!
One follow-up questions:
1. The Web2py manual says that
auth.settings.register_onaccept[]
should be called *before* the re-direction. For my purpose here (i.e.
re-directing the user to a new page after auth.register() is processed),
shouldn't I just do
auth.settings.register_next = URL('registered')
?
Again, thanks very much for your patience and help!
[web2py] Re: auth.settings.create_user_groups and auth.settings.registration_requires_verification in Auth
Thank you!
[web2py] auth.settings.create_user_groups and auth.settings.registration_requires_verification in Auth
Hi group, Two questions relating to two Auth settings: 1. when I set *auth.settings.create_user_groups = False*, I suspect that 'auth_group' table is not created at all after calling 'auth.define_tables()'. Because if I do a 'auth.add_group(...)' after 'auth.define_tables()' in a model file, web2py will throw error saying that 'auth_group' table does not exist when trying to do the insert. However, if I leave *auth.settings.create_user_groups = True *(by default), no such error is thrown. If this is the case, I wonder why. Surely, I would still want the table 'auth_group' created even if I don't want a new group created for each new user. 2. according to the manual, if we set *auth.settings.registration_requires_verification = True*, then an email will be sent to the user when she registers. I set it True, but nothing is sent after a user is registered. By registration, I mean the user fills out the register form exposed by Auth, clicks on the 'register' button, and gets re-directed to a separate page. I've checked in the database and the user details are indeed in 'auth_user' table. However, no email is sent, yet no error appears to have been thrown. Do I need to do something extra in model or controller? I also wonder if I should look deeper to make sure that this service not blocked by firewall on my machine etc, but it's not clear to me where I should start looking. Does Web2py retain a log for each running application somewhere, or this is done by inserting debugging code manually? If the latter case, where should I put the debugging/logging code to track down what went wrong when sending the verification emails? Many thanks!
[web2py] Re: auth.settings.create_user_groups and auth.settings.registration_requires_verification in Auth
Hi Anthony,
Thanks for looking into this.
The model code I am using is as follows:
## use PostgreSQL on localhost
db = DAL(postgres://cyan:123456@localhost:5432/mydb)
#
## web2py authentication utilities ##
#
from gluon.tools import Auth
auth = Auth(db, hmac_key=Auth.get_or_create_key())
## configure email
mail = auth.settings.mailer
mail.settings.server = 'smtp.gmail.com:587'
mail.settings.sender = 'login.n...@gmail.com'
mail.settings.login = 'login.name:123456'
## for the use of reCAPTCHA (with publicly visible site only)
#auth.settings.captcha = Recaptcha(request, 'PUBLIC_KEY', 'PRIVATE_KEY',
option=theme:'clean', lang:'en')
## re-name the authentication tables
auth.settings.table_user_name = user_login
auth.settings.table_group_name = user_group
auth.settings.table_membership_name = user_membership
auth.settings.table_permission_name = user_permission
auth.settings.table_event_name = user_event
## disable the default group creation behavior upon each user creation
auth.settings.create_user_groups = False
## turn on email verification upon registration and pwd reset
auth.settings.registration_requires_verification = True
auth.settings.reset_password_requires_verification = True
auth.settings.registration_requires_approval = False
## set the content of the verification email
auth.messages.verify_email = 'Please click on the link: http://' + \
request.env.http_host + \ URL(r=request,c='account',f='verify_email') + \
'/%(key)s to verify your email address'
## create all tables needed by Auth
auth.define_tables()
## customize the Auth tables
## abandon (but not remove) 'first_name' and 'last_name' from user_login
table
db.user_login.first_name.writable = db.user_login.last_name.writable = False
db.user_login.first_name.readable = db.user_login.last_name.readable = False
## set the email requirement
db.user_login.email.requires = [IS_NOT_EMPTY(), IS_EMAIL(forced='^.*\.edu
(|\..*)$', error_message='Email must be .edu address'),IS_NOT_IN_DB(db,
'user_login.email')]
## set the password requirement
db.user_login.password.requires = [IS_NOT_EMPTY(), IS_STRONG(min=7, special=
0, upper=0, number=0, error_message='Minimum 7 characters'), CRYPT()]
## setup the user group structure
auth.add_group('students', 'students')
auth.add_group('teachers', 'teachers')
##
## Custom DB tables ##
##
db.define_table('student', Field('login_id', db.user_login,
readable=False,writable
=False), Field('first_name', length=128, required=True),
Field('last_name',length
=128, required=True), Field('sex', required=True), Field('dob', 'date'),
Field('country'), format='%(first_name)s %(last_name)s')
db.participant.sex.requires = IS_IN_SET(['male','female'])
db.participant.country.requires = IS_IN_SET(['United States','United
Kingdom'])
db.participant.login_id.requires = IS_IN_DB(db, db.user_login.id,
'%(email)s')
You may comment out the line: auth.settings.create_user_groups = False if
you're getting the table-not-exist error and that should fix it.
As you can see above, I also setup the email account credentials (though
dummy ones in the code above) in the model, and have tested those
credentials against actual Gmail login. I have not explicitly tested it
using Web2py, but expect them to work given the description in the manual.
Thanks!
On Monday, April 16, 2012 6:39:43 PM UTC-4, Anthony wrote:
1. when I set *auth.settings.create_user_groups = False*, I suspect that
'auth_group' table is not created at all after calling
'auth.define_tables()'. Because if I do a 'auth.add_group(...)' after
'auth.define_tables()' in a model file, web2py will throw error saying that
'auth_group' table does not exist when trying to do the insert. However, if
I leave *auth.settings.create_user_groups = True *(by default), no such
error is thrown. If this is the case, I wonder why. Surely, I would still
want the table 'auth_group' created even if I don't want a new group
created for each new user.
I can't reproduce this problem with a fresh app using SQLite. Can you show
the exact code you're using?
2. according to the manual, if we set
*auth.settings.registration_requires_verification
= True*, then an email will be sent to the user when she registers. I
set it True, but nothing is sent after a user is registered.
Have you configured the mailer with your mail account credentials and
confirmed that it works?
Anthony
Re: [web2py] Eclipse and autocomplete
I've just tried it out. Unzip the latest stable source code of web2py to a folder, say, web2py_src. Go to Eclipse - Preferences - PyDev - 'Interpreter - Python' In the 'System PYTHONPATH' window, add 'web2py_src' to the path. So far, it seems autocomplete is working without doing all the import statements... Hope it helps. On Monday, April 9, 2012 3:04:30 PM UTC-4, ikdme wrote: Tang, please how do i implement this your suggestion? On Thu, Apr 5, 2012 at 3:54 PM, Xiaofeng Tang fengcli...@gmail.comwrote: I use PyDev and add web2py src into PyDev - PythonPath to resolve the incorrect refs and enable autocompletion. Alternatively, u could install the path of web2py src into systeml envirnoment vairable pythonpath. --- on 2012/4/5 22:51:41, Rod Watkins rwatk...@live.ca wrote: I know this has been asked before. I don't mean to be an annoyance. I've done the searches are read numerous posts about how to handle this. But the advice conflicts and I've yet to actually fix the problem. What is the best way to handle imports for eclipse to get autocomplete to work properly (as well as rid eclipse of all the incorrect error messages? I rather like having autocomplete functioning since, as a beginner, it helps me discover functionality that I might otherwise miss (besides saving some typing). Please accept my thanks in advance for any help on this. Cordially, Rod Watkins
Re: [web2py] Design of a minimal chat-like application in Web2py
This looks promising, but any chance of translating into English to server a larger communities of developers? (even some subtitle would help too!) Thanks. On Friday, April 13, 2012 6:43:54 PM UTC-4, rochacbruno wrote: Simple example with comet + tornado http://vimeo.com/38972256 (in Portuguese, but you can follow the code) On Fri, Apr 13, 2012 at 7:30 PM, cyan wrote: Hi group, I am trying to make a minimal multiuser chat-like app using Web2py. Nothing too fancy here, I just need the app to be able to do the following: 1. client browsers can send the receive text msgs from a server 2. server may manipulate the msgs received and selectively routes those msgs to different client browsers 3. server pushes msgs to client browsers using either comet or websocket or (something else? please enlighten me here). 4. it would be nice if the server can remember the client so that if a client drops and comes back in, all the previous sent/received msgs are remembered and shown, but this is not essential. So how should I design this given the MVC model of Web2py? What extra pieces (besides Javascript on the client side) do I need? Thanks! -- Bruno Rocha [http://rochacbruno.com.br]
[web2py] Re: Default table schema for the Auth tables
I think I will just make those fields invisible for now. Defining custom
auth tables seem also a little risky, as it is unclear that which fields
must be kept (except for auth_user, which is used as an example in the
manual) for both current and future releases.
Thanks all!
On Friday, April 13, 2012 9:30:22 AM UTC-4, Massimo Di Pierro wrote:
web2py's auth is take it or leave it.
I disagree. You can find the definition as Anthony suggests in the source
of gluon/tools.py.
You can add fields with
auth.settings.extra_fields['auth_user']=[Field('address')]
You should not remove fields. because they are the minimum necessary to
make things work. You can remove first_name and last_name but life is
easier if you simply make them invisible.
You do not have to use {{=auth.navbar(...)}} in your layout.html.
[web2py] Design of a minimal chat-like application in Web2py
Hi group, I am trying to make a minimal multiuser chat-like app using Web2py. Nothing too fancy here, I just need the app to be able to do the following: 1. client browsers can send the receive text msgs from a server 2. server may manipulate the msgs received and selectively routes those msgs to different client browsers 3. server pushes msgs to client browsers using either comet or websocket or (something else? please enlighten me here). 4. it would be nice if the server can remember the client so that if a client drops and comes back in, all the previous sent/received msgs are remembered and shown, but this is not essential. So how should I design this given the MVC model of Web2py? What extra pieces (besides Javascript on the client side) do I need? Thanks!
[web2py] Default table schema for the Auth tables
Hi group, Is there anywhere I can find the default schema for the tables involved in Auth provided by Web2py, namely, 'auth_user' 'auth_group' 'auth_membership' 'auth_permission' 'auth_event' They're described in the the manual but I would like to see the exact field definitions for each of them, if possible. In addition, can we remove fields from these tables? I know we can add extra fields or re-define the whole table from scratch to replace them. Thanks!
[web2py] redis support in Web2py
Hi group, Could someone please let me know how much support exists for redis in web2py? I've come across this: http://code.google.com/p/web2py/source/browse/gluon/contrib/redis_cache.py, but not sure if redis is officially supported by web2py. If not, any timeline available? Many thanks.
[web2py] Email verification for user registration
Hi Group, I am trying to implement a fairly standard procedure for user registration: after a user submits his/her email address, an email is sent, and by clicking on the link contained in that email, the user is registered. How do we implement this without writing anything to the DB until the user has clicked on the link in the email? This should work even if the user closes the browser and confirms his/her registration a few days later. I have a custom user account/authentication table, so would like to use that for this. Is there any example I can follow? Thanks.
[web2py] Re: Form validation on passwords
My view code is fairly minimal, but I suspect it is where the bug lies:
{{extend 'layout.html'}}
form
Email address: input type=text name=email /br /
Password: input type=password name=pwd /br /
Confirm password: input type=password name=re_pwd /br /
input type=submit /
/form
Do I need to kinda specify the error message here as well? Thanks again.
On Wednesday, March 28, 2012 8:54:55 AM UTC-4, Anthony wrote:
I tried your code, and it works fine for me. However, the password
Testpwd should actually result in an error because the IS_STRONG()
validator defaults to number=1 (i.e., at least one number in the password),
and you didn't change that default. So, you should get an error on the
password field. If the error isn't showing up, perhaps there's a problem
with your view code. Have you created a custom form in the HTML?
Anthony
On Tuesday, March 27, 2012 11:53:37 PM UTC-4, cyan wrote:
Thanks for the pointer. Anthony.
So now I have the following in my controller:
def register():
form = SQLFORM.factory(
Field('email', requires=[IS_NOT_EMPTY(), IS_EMAIL(forced=
'^.*\.edu(|\..*)$',
error_message='email must be .edu address')]),
Field('pwd', requires=[IS_NOT_EMPTY(), IS_STRONG(min=6, special=0, upper
=1,
error_message='minimum 6 characters, and at least 1 uppercase character'
), CRYPT()]),
Field('re_pwd', requires=IS_EXPR('value==%s' % repr(request.vars.get(
'pwd', None)),
error_message='passwords do not match')))
if form.process().accepted:
session.email = form.vars.email
session.pwd = form.vars.pwd
redirect(URL('registered'))
return dict(form=form)
And, for testing, I input the following:
a...@abc.edu
Testpwd
Testpwd
I didn't get any error messages, so presumably all three validations went
fine. However, using a debugger revealed that form.accepted is still None
after calling process() on the form. I wonder what went wrong here. Thanks.
On Tuesday, March 27, 2012 5:54:02 PM UTC-4, Anthony wrote:
Here's how auth.register() does it (slightly edited):
requires=IS_EXPR('value==%s' % repr(request.vars.get('password', None)),
error_message=Password fields don't match)
Anthony
On Tuesday, March 27, 2012 5:40:29 PM UTC-4, cyan wrote:
Hi group,
How do I enforce some simple validation for passwords matching on a
form in a controller? All I want to do is to check the second password is
the same as the first one, and here is some code I've come up so far:
form = SQLFORM.factory(
Field('email', requires=IS_NOT_EMPTY()),
Field('pwd', requires=[IS_NOT_EMPTY(), IS_STRONG(min=6, special=0,upper
=1), CRYPT()]),
Field('re_pwd', requires=IS_MATCH(???)))
I am attempting to use the validator 'IS_MATCH()' for the second
password, but not sure how I reference the the input in the first password
field of the same form. Any suggestion would be welcome. Thanks.
[web2py] Form validation on passwords
Hi group,
How do I enforce some simple validation for passwords matching on a form in
a controller? All I want to do is to check the second password is the same
as the first one, and here is some code I've come up so far:
form = SQLFORM.factory(
Field('email', requires=IS_NOT_EMPTY()),
Field('pwd', requires=[IS_NOT_EMPTY(), IS_STRONG(min=6, special=0, upper=1
), CRYPT()]),
Field('re_pwd', requires=IS_MATCH(???)))
I am attempting to use the validator 'IS_MATCH()' for the second password,
but not sure how I reference the the input in the first password field of
the same form. Any suggestion would be welcome. Thanks.
[web2py] Re: Form validation on passwords
Thanks for the pointer. Anthony.
So now I have the following in my controller:
def register():
form = SQLFORM.factory(
Field('email', requires=[IS_NOT_EMPTY(), IS_EMAIL(forced='^.*\.edu(|\..*)$'
,
error_message='email must be .edu address')]),
Field('pwd', requires=[IS_NOT_EMPTY(), IS_STRONG(min=6, special=0, upper=1,
error_message='minimum 6 characters, and at least 1 uppercase character'),CRYPT
()]),
Field('re_pwd', requires=IS_EXPR('value==%s' % repr(request.vars.get('pwd',
None)),
error_message='passwords do not match')))
if form.process().accepted:
session.email = form.vars.email
session.pwd = form.vars.pwd
redirect(URL('registered'))
return dict(form=form)
And, for testing, I input the following:
a...@abc.edu
Testpwd
Testpwd
I didn't get any error messages, so presumably all three validations went
fine. However, using a debugger revealed that form.accepted is still None
after calling process() on the form. I wonder what went wrong here. Thanks.
On Tuesday, March 27, 2012 5:54:02 PM UTC-4, Anthony wrote:
Here's how auth.register() does it (slightly edited):
requires=IS_EXPR('value==%s' % repr(request.vars.get('password', None)),
error_message=Password fields don't match)
Anthony
On Tuesday, March 27, 2012 5:40:29 PM UTC-4, cyan wrote:
Hi group,
How do I enforce some simple validation for passwords matching on a form
in a controller? All I want to do is to check the second password is the
same as the first one, and here is some code I've come up so far:
form = SQLFORM.factory(
Field('email', requires=IS_NOT_EMPTY()),
Field('pwd', requires=[IS_NOT_EMPTY(), IS_STRONG(min=6, special=0, upper
=1), CRYPT()]),
Field('re_pwd', requires=IS_MATCH(???)))
I am attempting to use the validator 'IS_MATCH()' for the second
password, but not sure how I reference the the input in the first password
field of the same form. Any suggestion would be welcome. Thanks.
[web2py] web2py hosting on dotcloud
Hi everyone, I'm new to web2py framework, and would like to host some app on dotcloud (dotcloud.com). Apparently, people have managed to done this successfully, and I wonder if someone could share their experience/procedure to deploy a web2py app on dotcloud. I've been trying to use dotcloud's CLI 0.4 to deploy just the plain off-the-shelf web2py, but keep getting uWSGI Error wsgi application not found when I go to the app url. I have the following setup: funback/ backend/ web2py/ wsgi.py - ./web2py/wsgihandler.py dotcloud.yml README and inside dotcloud.yml: funback: approot: backend type: python db: type: postgresql Can anyone suggest a possible cause of the problem and the remedy to fix it? Thanks a lot.
