Re: [web2py] Re: Web2py with python 3.8.5

[Joe Barnhart]

Just for the record, because it's not clear from this thread, the Python 
3,8 problem seems to have been taken care of.  I've been running web2py on 
Python 3.8 for the last couple of weeks at least and never noticed a 
problem.


On Sunday, January 3, 2021 at 2:26:52 PM UTC-6 jimka...@yahoo.com wrote:

> While investigating 3.8 issues, should also be aware of the following - 
> "3.9.1 is the first version of Python to support macOS 11 Big Sur."  as per 
> https://www.python.org/downloads/release/python-391/
>
> On Sunday, January 3, 2021 at 1:20:45 AM UTC-8 serge.bo...@gmail.com 
> wrote:
>
>> Hello,
>> I'm trying to convert my web2py apps to Ubuntu 20.4 / apache2/wsgi using 
>> python 3.8.5, and as I'm not an IT expert, the only way I found was to 
>> change the import instructions in a series of modules. I am aware I 
>> shouldn't do this, but I couldn't find any alternative... 
>> I would apreciate getting any feedback with another solution, and/or 
>> corrections of my 'durty coding'.
>>
>> Action 1:
>> - usr/lib/python3/dist-packages/yaml___init.py - line 399: 
>>  replace def class YAMLObject(metaclass=YAMLObjectMetaclass):
>> by
>> def class YAMLObject:
>> __metaclass__=YAMLObjectMetaclass
>> Action 2:
>> create /etc/securetty file (there was an example of content of this file 
>> in /usr/share/doc/util-linux/examples/securetty)
>>
>> Action 3:
>> add following line in 
>> /homr/www-data/web2py/gulon/packages/dal/pydal/adapters/base.py, line 16:
>> if not '/usr/lib/python3/dist-packages' in sys.path:
>> sys.path.append(''/usr/lib/python3/dist-packages')
>>
>> Action 4:
>> This 'import' instrunction of python is incompatible with the previous 
>> version. In each module (for instance gluon), the import of 'brother 
>> modules' (modules in the same folder than gluon) are done with the 
>> following type of instruction: 'from gluon import xxx'
>> This doesn't work in my configuration, because import instructions 
>> starting with a 'from' are supposed to refer to the relative path of the 
>> module, and not the absolute path (which scans sys.path).
>> => this instrunction must be changed (for instance by 'from .gluon import 
>> xxx')
>>
>> I am still working to get my system running, but at least, I can start 
>> web2py...
>>
>> I do not want to enter into phylosophical discussion about what I am 
>> authorized to do or not, and I hope that in early 2021, the evolution of 
>> ubuntu, python, web2py and other usefull packages will take care to keep 
>> harmony in the strategy of their evolution.
>>
>>
>> Le vendredi 29 novembre 2019 à 21:20:38 UTC+1, kell...@gmail.com a 
>> écrit :
>>
>>> Maybe, it wasn't on the radar of the django developers either and had to 
>>> be fixed in the framework after python3 was released.
>>>
>>> Happens sometimes I suppose, hard to keep up with depreciation of 
>>> functions and the impact on a framework. 
>>>
>>> Anyways, I suppose we juar have to file the bug reports and fix each 
>>> function step by step until all deprecated functions are removed and 
>>> replaced with the ones that need to be used for python 3.8 and forward.
>>>
>>>
>>>
>>>
>>>
>>> On Fri, 29 Nov 2019, 20:44 Woody,  wrote:
>>>
 Shouldn't that be done before the release rather than afterward?

 -- 
 Resources:
 - http://web2py.com
 - http://web2py.com/book (Documentation)
 - http://github.com/web2py/web2py (Source code)
 - https://code.google.com/p/web2py/issues/list (Report Issues)
 --- 
 You received this message because you are subscribed to the Google 
 Groups "web2py-users" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to web2py+un...@googlegroups.com.

>>> To view this discussion on the web visit 
 https://groups.google.com/d/msgid/web2py/cbb87866-f170-4338-ba3f-e93dfc3861f3%40googlegroups.com
 .

>>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/web2py/aa9f5f61-8a7e-4f81-b48d-55b40f72c49cn%40googlegroups.com.

[web2py] Re: Happy New Year

[Joe Barnhart]

And whatever "lucky food" you ate LAST year on New Year's -- DON'T TOUCH 
THAT STUFF!!  It didn't work.

-- Joe

On Thursday, December 31, 2020 at 4:33:09 PM UTC-6 snide...@gmail.com wrote:

> On Thursday, December 31, 2020 at 11:37:24 AM UTC-8 Massimo Di Pierro 
> wrote:
>
>>
>> I wish you all a happy new year.
>>
>
> Let's all be virtually and virtuously full of good cheer!
>
> I'm dusting off my wishes to send the best of them to you.
>
> /dps
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/web2py/4707e5c2-1014-4d16-b8df-58a451163eban%40googlegroups.com.

[web2py] Re: Redis and Python3?

[Joe Barnhart]

Here is a patch that restores redis3 sessions for my system.  I'm using the 
latest web2py 2.21.1 and redis stable 6.0.9 (via brew)

*diff --git a/gluon/contrib/redis_session.py 
b/gluon/contrib/redis_session.py*
*index 6413b52a..73c20422 100644*
*--- a/gluon/contrib/redis_session.py*
*+++ b/gluon/contrib/redis_session.py*
*@@ -13,7 +13,7 @@ from gluon import current*
* from gluon.storage import Storage*
* from gluon.contrib.redis_utils import acquire_lock, release_lock*
* from gluon.contrib.redis_utils import register_release_lock*
*-from gluon._compat import to_native*
*+from gluon._compat import to_native,to_bytes*
* from datetime import datetime*
 
* logger = logging.getLogger("web2py.session.redis")*
*@@ -208,7 +208,7 @@ class MockQuery(object):*
* if rtn:*
* if self.unique_key:*
* # make sure the id and unique_key are correct*
*-if rtn['unique_key'] == to_native(self.unique_key):*
*+if rtn['unique_key'] == to_bytes(self.unique_key):*
* rtn['update_record'] = self.update  # update 
record support*
* else:*
* rtn = None*


Does anyone know how to make code blocks in this new stupid Google Groups 
window?  Sheesh.  One step sideways and three steps back.

-- Joe

On Wednesday, September 11, 2019 at 12:11:13 AM UTC-5 Massimo Di Pierro 
wrote:

> :-)
>
>
> On Monday, 9 September 2019 22:40:36 UTC-7, Joe Barnhart wrote:
>>
>> ROFL!  I don't think it's quite ready for my prime-time cash-producing 
>> website yet.  I'll definitely keep an eye on it and start playing around 
>> with it.  I need to keep you from making any decisions I don't like, after 
>> all!
>>
>> -- Joe
>>
>>
>> On Monday, September 9, 2019 at 8:48:54 PM UTC-7, Massimo Di Pierro wrote:
>>>
>>> py4web supports redis. Just saying. ;-)
>>>
>>> On Monday, 9 September 2019 09:27:20 UTC-7, Joe Barnhart wrote:
>>>>
>>>> I sure wish somebody could fix this.  It's a real bottleneck to Python3 
>>>> adoption for my site.  I tried using local storage for sessions but the 
>>>> limitation on data size makes it a no-go for my site.  If I can't use 
>>>> redis 
>>>> I'll have to drop back to storing sessions in files and that really sucks 
>>>> for a larger deployment.
>>>>
>>>> -- Joe
>>>>
>>>> On Monday, April 1, 2019 at 11:31:10 PM UTC-7, Massimo Di Pierro wrote:
>>>>>
>>>>> we could change True/False with 1/0 but a better approach would be to 
>>>>> remove the value within the redis adapter. The value of locked does not 
>>>>> do 
>>>>> anything anyway on redis since it is not a relational database with 
>>>>> transactions.
>>>>>
>>>>> On Sunday, 31 March 2019 10:19:20 UTC-7, Jim S wrote:
>>>>>>
>>>>>> I changed the following in gluon/globals.py and mine is working now.  
>>>>>> But, I'm unclear on how to test to see if it is handling the locking 
>>>>>> properly:
>>>>>>
>>>>>> dd = dict(locked=False,
>>>>>>   client_ip=response.session_client,
>>>>>>   modified_datetime=request.now,
>>>>>>   session_data=session_pickled,
>>>>>>   unique_key=unique_key)
>>>>>>
>>>>>> to 
>>>>>>
>>>>>> dd = dict(locked='False',
>>>>>>   client_ip=response.session_client,
>>>>>>   modified_datetime=str(request.now),
>>>>>>   session_data=session_pickled,
>>>>>>   unique_key=unique_key)
>>>>>>
>>>>>> This makes all my stuff work (I'm still on Python 2.7 but I think the 
>>>>>> problem has to do with the python redis client moving to version 3).
>>>>>>
>>>>>> Leonel - I think this relates back to an issue you commented on late 
>>>>>> last year.   
>>>>>> https://groups.google.com/forum/?pli=1#!searchin/web2py/redis%7Csort:date/web2py/PdquGF_9a2E/6VJpLqsnBgAJ
>>>>>>
>>>>>> At that time I just continued using python redis 2.10.6.  But, can't 
>>>>>> do that forever.  Anyone able to test or improve upon the change I made 
>>>>>> above?  Like I said, it works for me, but I don't know how to see if I'm 
>>>>>> causing any other unforeseen damage.
>>>&

[web2py] Re: Is it time to take redis off the features list?

[Joe Barnhart]

I think I found a workaround for redis3 sessions.  Here is a mod that works 
for me:

diff --git a/gluon/contrib/redis_session.py b/gluon/contrib/redis_session.py
index 6413b52a..73c20422 100644
--- a/gluon/contrib/redis_session.py
+++ b/gluon/contrib/redis_session.py
@@ -13,7 +13,7 @@ from gluon import current
 from gluon.storage import Storage
 from gluon.contrib.redis_utils import acquire_lock, release_lock
 from gluon.contrib.redis_utils import register_release_lock
-from gluon._compat import to_native
+from gluon._compat import to_native,to_bytes
 from datetime import datetime
 
 logger = logging.getLogger("web2py.session.redis")
@@ -208,7 +208,7 @@ class MockQuery(object):
 if rtn:
 if self.unique_key:
 # make sure the id and unique_key are correct
-if rtn['unique_key'] == to_native(self.unique_key):
+if rtn['unique_key'] == to_bytes(self.unique_key):
 rtn['update_record'] = self.update  # update 
record support
 else:
 rtn = None


On Friday, November 13, 2020 at 4:25:13 AM UTC-6 Jose C wrote:

> I have it working with python3 but only as far as web2py 2.18.5.  
> Thereafter I cannot get it to work either.
>
> Unfortunately it does begin to appear that web2py has reached end of life 
> status. 
>
> On Friday, 13 November 2020 at 03:13:18 UTC Joe Barnhart wrote:
>
>> I know this is really old news, but redis no longer works under python 3 
>> and web2py.  I'm just thinking it's time to take out all mention of redis 
>> from the book and list of features of web2py.  Yes, it technically works 
>> with python 2, but since that platform is now dropped in the wake of the 
>> good ship SS Python, it's time to either bite the bullet and fix the 
>> feature or deleted it entirely from the website, book, and promotion of 
>> web2py.
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/web2py/b9ca2ee3-dfc8-4caa-9aab-344746775c70n%40googlegroups.com.

[web2py] Is it time to take redis off the features list?

[Joe Barnhart]

I know this is really old news, but redis no longer works under python 3 
and web2py.  I'm just thinking it's time to take out all mention of redis 
from the book and list of features of web2py.  Yes, it technically works 
with python 2, but since that platform is now dropped in the wake of the 
good ship SS Python, it's time to either bite the bullet and fix the 
feature or deleted it entirely from the website, book, and promotion of 
web2py.

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/web2py/d6a3ff41-1b11-4fcf-a968-9feeb30b5c9fn%40googlegroups.com.

[web2py] Using LOAD and template blocks

[Joe Barnhart]

I'm using LOAD functionality to break my webpage into bite-sized "cards" or 
"widgets" each of which is loaded separately when the page is called.  My 
overall website looks much like a "dashboard" with several of these widgets 
on a single page.

I'm trying to understand the LOAD process and how I can control the loading 
of Javascript snippets along with the HTML of the widget.  The Javascript 
added relies on other JS libraries already loaded -- only they aren't.  Yet.

My regular Javascript libraries (including jQuery, Datatables, etc.) are 
loaded at the end of the process, as usually recommended for better loading 
performance.  But the scripts loaded by LOAD are loaded before any of the 
standard Javascript libraries.  This causes JS idioms like "$(document)" to 
fail because "$" isn't defined yet.

I've created a thread-safe var called "scripts" and made it a list.  I 
added JS scripts to this list inside the LOAD template:

{{=body}}
{{
  xtra = current.scripts
  if isinstance(script,list):
xtra.extend(xtra)
  else:
xtra.append(xtra)
  pass
}}

(Apologies on the look of the fragment but we seem to have lost 
code-formatting ability in the latest "improvement" to Google Groups.)

Then at the end of my layout, I try to render these blocks at the end of 
the main view in layout.html:

  
  {{block extra_scripts}}
  {{
for ea in current.scripts or []:
  =ea
pass
  }}
  {{end extra_scripts}}

Surprisingly, this does not work as expected.  The page is rendered with 
all widgets and each Javascript segment is rendered along with the HTML of 
the widget -- NOT at the bottom of the page as commanded by the layout.html.

I'm mystified at how this occurs.  I could understand if layout.html is 
used at every invocation of LOAD, but that makes no sense whatever.  How 
would it "know" to skip all parts of layout.html except the script 
rendering part at the bottom?

If there's a better solution to my conundrum I'd like to know about it. 
 The goal again is to defer the scripts from a LOAD component until AFTER 
the main Javascript libraries are loaded.  I can't use response.files 
because these snippets are not *.js files, they are inline scripts.  They 
can't really be *.js files because they are dynamic and include 
customization for each widget individually.  So how to defer the scripts 
until after the response.files have been written?

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/web2py/4bd7d370-84bb-4d2c-9773-e7621cd0e4d8n%40googlegroups.com.

[web2py] Re: pyDAL looks abandoned?

[Joe Barnhart]

FWIW there's nothing stopping you from using peewee inside web2py instead 
of DAL, if you desire.  It's kind of an advanced topic, but it's possible. 
 I notice peewee relies on cascaded message sends.  Being an old 
Smalltalker I actually like cascaded message sends, but I recall that pypy 
really does not like them.  They can leave chains of objects that are 
harder for pypy's storage reclamation to deal with (unless that's changed 
in pypy -- It moves pretty fast).

On Monday, May 11, 2020 at 3:27:14 PM UTC-5, Val K wrote:
>
>
> There are 131 issues on pyDAL-GitHub, but the problem is not in the amount 
> but in that more than half of them without any response. Look at peewee 
> that has 7.5k stars and only 2 open issues!... yes, this is impolite 
> comparision - sorry.
> I've just need to implement CTE + WITH RECURSIVE (like this 
> http://docs.peewee-orm.com/en/latest/peewee/querying.html#recursive-ctes) 
> and I wasted 2 days to figure out how I can do very simple thing like  '1 
> AS foo'  in right way that is (as I suppose):
> Expression(db, '1', type='integer').with_alias('foo')
>
> So, the question is: why is there nothing about that in the book?
> Why does pyDAL have no that very simple but very necessary method  which 
> allows to do cool things like:
>  db(...).select(
> ...,
> db.exp(
>'max(product.price) OVER(PARTITION BY product.category)', 
>type = 'decimal(n, m)' # the same as Field(..., type = 'decimal(n, 
> m)')
> ).with_alias('most_expensive_in_category')
> )
>
> here is used WINDOW-function that allows to get products list with 
> 'most_expensive_in_category' in *one* plain query 
> *without using subquery* with GRUOP BY product.category + join
>
> Also there is my PR on web2py book 
> https://github.com/web2py/web2py-book/pull/424
> it hangs for almost a year - it is just a book, not code
> I'm sad
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/web2py/ce42fcb5-0e14-459a-af80-da635d5ef173o%40googlegroups.com.

[web2py] Re: Tip - Using multiple web2py templates

[Joe Barnhart]

Doesn't this have an unfortunate negative effect on "compiled" views?  To 
be compiled, everything about the view needs to be known ahead of time and 
your dynamic use of templates seems to defeat that.  I suspect when you 
choose to "compile" the application from the web2py admin interface, these 
views will remain interpreted, defeating the purpose of compiling the app.

But I could be wrong.

On Monday, June 1, 2020 at 4:44:16 PM UTC-5, Rodrigo Attique wrote:
>
> After a lot of brainstorming I found a way to allow web2py to use multiple 
> templates. Perhaps for the elderly it is not a novelty, but for the 
> youngest ... This will be a hand in the wheel.
>
> Link:
>
> https://web2pymania.blogspot.com/2020/06/using-multiple-web2py-templates.html
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/web2py/d067224a-e081-432b-b1cc-a8806b163eb2o%40googlegroups.com.

[web2py] Re: Auth - overlapping permissions problem

[Joe Barnhart]

Having more than one group with the same permissions is no problem at all. 
 In fact, it's kind of the purpose of role-based authentication.

I have a web site that features different classes of users who need access 
to pages.  Some groups can access every single page in the site (e.g. 
'root' user) while others can access only a tiny subset.  The key is to 
make the permissions granular enough.  If a secretary has permission for 
some kinds of reports but not others, there needs to be a permission to 
handle that subset of reports so it can be added to the secretary group.

Here is the code I use to build permissions when the database is being 
initialized.  Other permissions and groups can be added through web 
actions, but his list populates the basic site.  This code runs in a 
"module" which is why It uses the thread-safe "settings" variable.

from gluon import *
class UserRole():

@staticmethod
def permissions():
T,settings = current.T,current.settings
return [
('club_page', T('View club information page'), '', 0),
('lsc_page', T('View LSC information page'), '', 0),
('support_page', T('View %(title) support page', settings), '', 
0),
('admin_page', T('View %(title) admin page', settings), '', 0),
('root_page', T('View special root-access page'), '', 0),
('impersonate', T('Impersonate other users'), 'auth_user', 0),
]

@staticmethod
def roles():
T,settings = current.T,current.settings
return [
('root', T('Superuser access to all'), 
('club_page','lsc_page','support_page','admin_page','root_page','impersonate')),
('ss_admin', T('%(title) administration', settings), 
('club_page','lsc_page','support_page','admin_page','impersonate')),
('ss_support', T('%(title) support', settings), 
('club_page','lsc_page','support_page','impersonate')),
('lsc_admin', T('LSC staff'), ('lsc_page',)),
('club_admin', T('Club staff'), ('club_page',)),
('coach', T('Swim coach '), ('club_page',)),
('user', T('Parent or swimmer'), tuple()),
('vendor', T('Vendor or advertiser'), tuple()),
('impersonate',T('Impersonate other users'), ('impersonate',))
]

@classmethod
def define_roles(cls):
auth = current.auth
perms = {nam:(nam,obj,col) for nam,dsc,obj,col in cls.permissions()}
for ea in cls.roles():
grp,descr,plist = ea
gid = auth.add_group(grp, descr)
for p in plist:
auth.add_permission(gid,*(perms[p]))




On Thursday, July 16, 2020 at 5:30:47 PM UTC-5, Paul Ellis wrote:
>
> the situation is that I have permissions associated with auth_groups. 
>
> eg. user, team leader, business leader, oversight
>
> now I need a "secretary" or "accounts" group which has access to some 
> business leader features i.e reports.
> Some team leader features ie. price maintainence.
> But does not have access to the basic features of normal users.
>
> I can't see how to make an auth_group which has permissions which are 
> already associated with another group.
>
> I am hoping for a solution which does not involve hard coding access to 
> these features using @auth.has_membership('accounts') as I don't expect 
> this to be last case of overlapping permissions.
>
> Ideally I can make an interface where business leaders can make a custom 
> permission group and assign it to their employees. Without creating a 
> situation where each new employee needs to have a permission allocated for 
> every little thing.
> i.e. customer maintenence, customer delete, product related permissions, 
> product permissions where special knowledge is required...
>
> Is this possible with the Web2py Auth System?
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/web2py/ab7d164b-4a24-47b0-b846-0a5858ee4144o%40googlegroups.com.

[web2py] Re: 2.19.1 redis sessions problem

[Joe Barnhart]

Yep.  I just confirmed the errant behavior with web2py 2.20.4 and python 
3.8.2.  If reds sessions are enabled the logon never completes.

-- Joe B.

On Sunday, March 29, 2020 at 11:39:10 PM UTC-5, Massimo Di Pierro wrote:
>
> Sorry. We should fix this. Will look into it asap.
>
> On Sunday, 29 March 2020 15:38:27 UTC-7, Jim S wrote:
>>
>> FWIW - I've abandoned redis for sessions.  I can't get it to work 
>> reliably and no one seems interested in fixing it.  I submitted some 
>> patches but they were rejected.  So, I've moved my sessions into my 
>> database.  Not what I wanted to do, but at least that works reliably.  
>> Sorry to be negative about this, it just seems it isn't a priority for the 
>> project.
>>
>> -Jim
>>
>>
>> On Saturday, March 28, 2020 at 5:30:11 AM UTC-5, Jose C wrote:
>>>
>>> Just started testing version 2.19.1 (prior to upgrading) with py3.6 and 
>>> it appears that when sessions are stored in redis, a new session is being 
>>> created each time the login is submitted which then prevents a session from 
>>> logging in.  
>>>
>>> This issue was discussed in 
>>> https://groups.google.com/forum/#!msg/web2py/z1ee025ghRU/qByNuEjGAgAJ  
>>> and Jim S found a workaround with the .decode function which worked on py3 
>>> but not on py2 (and which I've been using live).  It looks like the redis 
>>> session code has been refactored in 2.19.1 to remove all the decodes but I 
>>> can't get it to work correctly.
>>>
>>> Can anyone confirm whether they have this issue when trying to login 
>>> using redis sessions in 2.19.1 (with py3)?  If confirmed, I'll file a bug 
>>> report.
>>>
>>> Thanks,
>>>
>>> Jose
>>>
>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/web2py/fe268b34-a931-45d2-872c-a5e73d803b43o%40googlegroups.com.

[web2py] Re: Redis and Python3?

[Joe Barnhart]

ROFL!  I don't think it's quite ready for my prime-time cash-producing 
website yet.  I'll definitely keep an eye on it and start playing around 
with it.  I need to keep you from making any decisions I don't like, after 
all!

-- Joe


On Monday, September 9, 2019 at 8:48:54 PM UTC-7, Massimo Di Pierro wrote:
>
> py4web supports redis. Just saying. ;-)
>
> On Monday, 9 September 2019 09:27:20 UTC-7, Joe Barnhart wrote:
>>
>> I sure wish somebody could fix this.  It's a real bottleneck to Python3 
>> adoption for my site.  I tried using local storage for sessions but the 
>> limitation on data size makes it a no-go for my site.  If I can't use redis 
>> I'll have to drop back to storing sessions in files and that really sucks 
>> for a larger deployment.
>>
>> -- Joe
>>
>> On Monday, April 1, 2019 at 11:31:10 PM UTC-7, Massimo Di Pierro wrote:
>>>
>>> we could change True/False with 1/0 but a better approach would be to 
>>> remove the value within the redis adapter. The value of locked does not do 
>>> anything anyway on redis since it is not a relational database with 
>>> transactions.
>>>
>>> On Sunday, 31 March 2019 10:19:20 UTC-7, Jim S wrote:
>>>>
>>>> I changed the following in gluon/globals.py and mine is working now.  
>>>> But, I'm unclear on how to test to see if it is handling the locking 
>>>> properly:
>>>>
>>>> dd = dict(locked=False,
>>>>   client_ip=response.session_client,
>>>>   modified_datetime=request.now,
>>>>   session_data=session_pickled,
>>>>   unique_key=unique_key)
>>>>
>>>> to 
>>>>
>>>> dd = dict(locked='False',
>>>>   client_ip=response.session_client,
>>>>   modified_datetime=str(request.now),
>>>>   session_data=session_pickled,
>>>>   unique_key=unique_key)
>>>>
>>>> This makes all my stuff work (I'm still on Python 2.7 but I think the 
>>>> problem has to do with the python redis client moving to version 3).
>>>>
>>>> Leonel - I think this relates back to an issue you commented on late 
>>>> last year.   
>>>> https://groups.google.com/forum/?pli=1#!searchin/web2py/redis%7Csort:date/web2py/PdquGF_9a2E/6VJpLqsnBgAJ
>>>>
>>>> At that time I just continued using python redis 2.10.6.  But, can't do 
>>>> that forever.  Anyone able to test or improve upon the change I made 
>>>> above?  Like I said, it works for me, but I don't know how to see if I'm 
>>>> causing any other unforeseen damage.
>>>>
>>>> -Jim
>>>>
>>>> On Friday, March 29, 2019 at 7:49:03 PM UTC-5, Joe Barnhart wrote:
>>>>>
>>>>> Hi Leonel --
>>>>>
>>>>> My brain refuses to put together the words "session locked field".  I 
>>>>> understand the part of pickle smashing the values and encoding them as 
>>>>> one 
>>>>> string.  The bool has something to do with session locking?  
>>>>>
>>>>> I've only looked at the web2py redis code in the most scant way as I 
>>>>> wasn't planning to become a redis expert.  Sessions in redis could be 
>>>>> important to me, tho, so I may need to set aside some time to dig into 
>>>>> it. 
>>>>>  (Storing sessions in cookies has been problematic due to the limited 
>>>>> space 
>>>>> for cookies in browsers.)
>>>>>
>>>>> -- Joe
>>>>>
>>>>>
>>>>> On Wednesday, March 27, 2019 at 4:49:25 PM UTC-7, Leonel Câmara wrote:
>>>>>>
>>>>>> It's not your values Joe, pickle serializes them as a string so they 
>>>>>> would be fine. It's web2py session locked field which is True or False. 
>>>>>> It's probably easy to fix this in redis_session.py
>>>>>>
>>>>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/web2py/18298a60-e604-4bb9-b9a8-ac9cb7469be1%40googlegroups.com.

Re: [web2py] Re: Redis and Python3?

[Joe Barnhart]

If you're asking if it runs unmodified under Python2 I do not know as of 
yet.  I would need to dust off a Python2 site, apply the patch, configure 
it for redis, and test.  

That might happen in a week or two as I don't have a Python2/redis site 
right now so I need to do some work to create a test bench for this. 

If someone else has a Python2/redis site already they may be able to test 
it quicker.

-- Joe

On Monday, September 9, 2019 at 3:00:50 PM UTC-7, Jim S wrote:
>
> Is it backward compatible as well?
>
> -Jim
>
> On Mon, Sep 9, 2019 at 4:54 PM Joe Barnhart  > wrote:
>
>> YES.  This seems to work fine in my case.  It permits me to save both 
>> sessions and cache data to redis using Python3.  Maybe we can get it 
>> accepted and blessed for the next release.
>>
>> Joe
>>
>> On Monday, September 9, 2019 at 1:03:51 PM UTC-7, rastafarien wrote:
>>>
>>> Hello
>>>
>>> I have also proposed a patch to make redis work with my applications 
>>> under wep2py latest ( 2.18.5-stable+timestamp.2019.04.08.04.22.03) and 
>>> python3
>>> it is attached to this post.
>>> "onetwomany" also released a patch.
>>>
>>> Basically
>>>
>>> 2 files are to be adjusted : redis_session.py and global.py
>>>
>>> 1/ Redis does not support bool so I replaced False by 0
>>> 2/ Redis does not support datetime so I replaced datetime by 
>>> str(datetime)
>>>
>>> Under linux and python3, I found out that it was the only way to have 
>>> the "welcome" application ( and also my own applications) to work in a 
>>> multinode environment accessing a distributed redis server for session 
>>> caching. 
>>> Let us know if it solves your problem.
>>> Regards
>>> Stephane
>>>
>>>
>>> Le lundi 9 septembre 2019 18:27:20 UTC+2, Joe Barnhart a écrit :
>>>>
>>>> I sure wish somebody could fix this.  It's a real bottleneck to Python3 
>>>> adoption for my site.  I tried using local storage for sessions but the 
>>>> limitation on data size makes it a no-go for my site.  If I can't use 
>>>> redis 
>>>> I'll have to drop back to storing sessions in files and that really sucks 
>>>> for a larger deployment.
>>>>
>>>> -- Joe
>>>>
>>>> On Monday, April 1, 2019 at 11:31:10 PM UTC-7, Massimo Di Pierro wrote:
>>>>>
>>>>> we could change True/False with 1/0 but a better approach would be to 
>>>>> remove the value within the redis adapter. The value of locked does not 
>>>>> do 
>>>>> anything anyway on redis since it is not a relational database with 
>>>>> transactions.
>>>>>
>>>>> On Sunday, 31 March 2019 10:19:20 UTC-7, Jim S wrote:
>>>>>>
>>>>>> I changed the following in gluon/globals.py and mine is working now.  
>>>>>> But, I'm unclear on how to test to see if it is handling the locking 
>>>>>> properly:
>>>>>>
>>>>>> dd = dict(locked=False,
>>>>>>   client_ip=response.session_client,
>>>>>>   modified_datetime=request.now,
>>>>>>   session_data=session_pickled,
>>>>>>   unique_key=unique_key)
>>>>>>
>>>>>> to 
>>>>>>
>>>>>> dd = dict(locked='False',
>>>>>>   client_ip=response.session_client,
>>>>>>   modified_datetime=str(request.now),
>>>>>>   session_data=session_pickled,
>>>>>>   unique_key=unique_key)
>>>>>>
>>>>>> This makes all my stuff work (I'm still on Python 2.7 but I think the 
>>>>>> problem has to do with the python redis client moving to version 3).
>>>>>>
>>>>>> Leonel - I think this relates back to an issue you commented on late 
>>>>>> last year.   
>>>>>> https://groups.google.com/forum/?pli=1#!searchin/web2py/redis%7Csort:date/web2py/PdquGF_9a2E/6VJpLqsnBgAJ
>>>>>>
>>>>>> At that time I just continued using python redis 2.10.6.  But, can't 
>>>>>> do that forever.  Anyone able to test or improve upon the change I made 
>>>>>> above?  Like I said, it works for me, but I don't know how to see if I'm 
>>>>>> causing any other unforeseen damage.

[web2py] Re: Redis and Python3?

[Joe Barnhart]

YES.  This seems to work fine in my case.  It permits me to save both 
sessions and cache data to redis using Python3.  Maybe we can get it 
accepted and blessed for the next release.

Joe

On Monday, September 9, 2019 at 1:03:51 PM UTC-7, rastafarien wrote:
>
> Hello
>
> I have also proposed a patch to make redis work with my applications under 
> wep2py latest ( 2.18.5-stable+timestamp.2019.04.08.04.22.03) and python3
> it is attached to this post.
> "onetwomany" also released a patch.
>
> Basically
>
> 2 files are to be adjusted : redis_session.py and global.py
>
> 1/ Redis does not support bool so I replaced False by 0
> 2/ Redis does not support datetime so I replaced datetime by str(datetime)
>
> Under linux and python3, I found out that it was the only way to have the 
> "welcome" application ( and also my own applications) to work in a 
> multinode environment accessing a distributed redis server for session 
> caching. 
> Let us know if it solves your problem.
> Regards
> Stephane
>
>
> Le lundi 9 septembre 2019 18:27:20 UTC+2, Joe Barnhart a écrit :
>>
>> I sure wish somebody could fix this.  It's a real bottleneck to Python3 
>> adoption for my site.  I tried using local storage for sessions but the 
>> limitation on data size makes it a no-go for my site.  If I can't use redis 
>> I'll have to drop back to storing sessions in files and that really sucks 
>> for a larger deployment.
>>
>> -- Joe
>>
>> On Monday, April 1, 2019 at 11:31:10 PM UTC-7, Massimo Di Pierro wrote:
>>>
>>> we could change True/False with 1/0 but a better approach would be to 
>>> remove the value within the redis adapter. The value of locked does not do 
>>> anything anyway on redis since it is not a relational database with 
>>> transactions.
>>>
>>> On Sunday, 31 March 2019 10:19:20 UTC-7, Jim S wrote:
>>>>
>>>> I changed the following in gluon/globals.py and mine is working now.  
>>>> But, I'm unclear on how to test to see if it is handling the locking 
>>>> properly:
>>>>
>>>> dd = dict(locked=False,
>>>>   client_ip=response.session_client,
>>>>   modified_datetime=request.now,
>>>>   session_data=session_pickled,
>>>>   unique_key=unique_key)
>>>>
>>>> to 
>>>>
>>>> dd = dict(locked='False',
>>>>   client_ip=response.session_client,
>>>>   modified_datetime=str(request.now),
>>>>   session_data=session_pickled,
>>>>   unique_key=unique_key)
>>>>
>>>> This makes all my stuff work (I'm still on Python 2.7 but I think the 
>>>> problem has to do with the python redis client moving to version 3).
>>>>
>>>> Leonel - I think this relates back to an issue you commented on late 
>>>> last year.   
>>>> https://groups.google.com/forum/?pli=1#!searchin/web2py/redis%7Csort:date/web2py/PdquGF_9a2E/6VJpLqsnBgAJ
>>>>
>>>> At that time I just continued using python redis 2.10.6.  But, can't do 
>>>> that forever.  Anyone able to test or improve upon the change I made 
>>>> above?  Like I said, it works for me, but I don't know how to see if I'm 
>>>> causing any other unforeseen damage.
>>>>
>>>> -Jim
>>>>
>>>> On Friday, March 29, 2019 at 7:49:03 PM UTC-5, Joe Barnhart wrote:
>>>>>
>>>>> Hi Leonel --
>>>>>
>>>>> My brain refuses to put together the words "session locked field".  I 
>>>>> understand the part of pickle smashing the values and encoding them as 
>>>>> one 
>>>>> string.  The bool has something to do with session locking?  
>>>>>
>>>>> I've only looked at the web2py redis code in the most scant way as I 
>>>>> wasn't planning to become a redis expert.  Sessions in redis could be 
>>>>> important to me, tho, so I may need to set aside some time to dig into 
>>>>> it. 
>>>>>  (Storing sessions in cookies has been problematic due to the limited 
>>>>> space 
>>>>> for cookies in browsers.)
>>>>>
>>>>> -- Joe
>>>>>
>>>>>
>>>>> On Wednesday, March 27, 2019 at 4:49:25 PM UTC-7, Leonel Câmara wrote:
>>>>>>
>>>>>> It's not your values Joe, pickle serializes them as a string so they 
>>>>>> would be fine. It's web2py session locked field which is True or False. 
>>>>>> It's probably easy to fix this in redis_session.py
>>>>>>
>>>>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/web2py/9ed9f575-1764-4dab-a93c-d7ec68e57638%40googlegroups.com.

[web2py] Re: Redis and Python3?

[Joe Barnhart]

I sure wish somebody could fix this.  It's a real bottleneck to Python3 
adoption for my site.  I tried using local storage for sessions but the 
limitation on data size makes it a no-go for my site.  If I can't use redis 
I'll have to drop back to storing sessions in files and that really sucks 
for a larger deployment.

-- Joe

On Monday, April 1, 2019 at 11:31:10 PM UTC-7, Massimo Di Pierro wrote:
>
> we could change True/False with 1/0 but a better approach would be to 
> remove the value within the redis adapter. The value of locked does not do 
> anything anyway on redis since it is not a relational database with 
> transactions.
>
> On Sunday, 31 March 2019 10:19:20 UTC-7, Jim S wrote:
>>
>> I changed the following in gluon/globals.py and mine is working now.  
>> But, I'm unclear on how to test to see if it is handling the locking 
>> properly:
>>
>> dd = dict(locked=False,
>>   client_ip=response.session_client,
>>   modified_datetime=request.now,
>>   session_data=session_pickled,
>>   unique_key=unique_key)
>>
>> to 
>>
>> dd = dict(locked='False',
>>   client_ip=response.session_client,
>>   modified_datetime=str(request.now),
>>   session_data=session_pickled,
>>   unique_key=unique_key)
>>
>> This makes all my stuff work (I'm still on Python 2.7 but I think the 
>> problem has to do with the python redis client moving to version 3).
>>
>> Leonel - I think this relates back to an issue you commented on late last 
>> year.   
>> https://groups.google.com/forum/?pli=1#!searchin/web2py/redis%7Csort:date/web2py/PdquGF_9a2E/6VJpLqsnBgAJ
>>
>> At that time I just continued using python redis 2.10.6.  But, can't do 
>> that forever.  Anyone able to test or improve upon the change I made 
>> above?  Like I said, it works for me, but I don't know how to see if I'm 
>> causing any other unforeseen damage.
>>
>> -Jim
>>
>> On Friday, March 29, 2019 at 7:49:03 PM UTC-5, Joe Barnhart wrote:
>>>
>>> Hi Leonel --
>>>
>>> My brain refuses to put together the words "session locked field".  I 
>>> understand the part of pickle smashing the values and encoding them as one 
>>> string.  The bool has something to do with session locking?  
>>>
>>> I've only looked at the web2py redis code in the most scant way as I 
>>> wasn't planning to become a redis expert.  Sessions in redis could be 
>>> important to me, tho, so I may need to set aside some time to dig into it. 
>>>  (Storing sessions in cookies has been problematic due to the limited space 
>>> for cookies in browsers.)
>>>
>>> -- Joe
>>>
>>>
>>> On Wednesday, March 27, 2019 at 4:49:25 PM UTC-7, Leonel Câmara wrote:
>>>>
>>>> It's not your values Joe, pickle serializes them as a string so they 
>>>> would be fine. It's web2py session locked field which is True or False. 
>>>> It's probably easy to fix this in redis_session.py
>>>>
>>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/web2py/a8c8cb9d-590a-43ff-9f1b-b757eabf8537%40googlegroups.com.

[web2py] Re: Redis and Python3?

[Joe Barnhart]

Looking at the rest of globals.py I don't think this approach will work. 
 For one, it affects storing sessions in any database, not just redis. 
 Since the table is declared with a "boolean" field I don't think its gonna 
like the string version of "False".

I wonder why we don't have a simple minded dialect in DAL for redis. 
 Admittedly, it's not a RDBMS so a lot of DAL just won't ever work on 
redis.  But it would have provided a clean place to map "booleans" to 
strings instead of bools in this case, making it easy to special-case redis.

But maybe that would encourage people to complain that the implementation 
of the redis DAL dialect is too deficient.  Putting it in DAL would raise 
people's expectations of what it can handle.

-- Joe


On Sunday, March 31, 2019 at 10:19:20 AM UTC-7, Jim S wrote:
>
> I changed the following in gluon/globals.py and mine is working now.  But, 
> I'm unclear on how to test to see if it is handling the locking properly:
>
> dd = dict(locked=False,
>   client_ip=response.session_client,
>   modified_datetime=request.now,
>   session_data=session_pickled,
>   unique_key=unique_key)
>
> to 
>
> dd = dict(locked='False',
>   client_ip=response.session_client,
>   modified_datetime=str(request.now),
>   session_data=session_pickled,
>   unique_key=unique_key)
>
> This makes all my stuff work (I'm still on Python 2.7 but I think the 
> problem has to do with the python redis client moving to version 3).
>
> Leonel - I think this relates back to an issue you commented on late last 
> year.   
> https://groups.google.com/forum/?pli=1#!searchin/web2py/redis%7Csort:date/web2py/PdquGF_9a2E/6VJpLqsnBgAJ
>
> At that time I just continued using python redis 2.10.6.  But, can't do 
> that forever.  Anyone able to test or improve upon the change I made 
> above?  Like I said, it works for me, but I don't know how to see if I'm 
> causing any other unforeseen damage.
>
> -Jim
>
> On Friday, March 29, 2019 at 7:49:03 PM UTC-5, Joe Barnhart wrote:
>>
>> Hi Leonel --
>>
>> My brain refuses to put together the words "session locked field".  I 
>> understand the part of pickle smashing the values and encoding them as one 
>> string.  The bool has something to do with session locking?  
>>
>> I've only looked at the web2py redis code in the most scant way as I 
>> wasn't planning to become a redis expert.  Sessions in redis could be 
>> important to me, tho, so I may need to set aside some time to dig into it. 
>>  (Storing sessions in cookies has been problematic due to the limited space 
>> for cookies in browsers.)
>>
>> -- Joe
>>
>>
>> On Wednesday, March 27, 2019 at 4:49:25 PM UTC-7, Leonel Câmara wrote:
>>>
>>> It's not your values Joe, pickle serializes them as a string so they 
>>> would be fine. It's web2py session locked field which is True or False. 
>>> It's probably easy to fix this in redis_session.py
>>>
>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: In praise of the lowly "nested_select"

[Joe Barnhart]

Sorry for the delay in replying.

The first step is to create a Select object, rather than an Expression. 
 Look again at the code:
subq = db(dbts.id>0).nested_select(dbts.ALL,...etc)

This creates a Select object.  The Select object implements "on".  (Table 
also implements "on", but not Expression)

I should come up with a simpler example.  This one was just grabbed 
directly out of my site so I knew it worked but i's too complex to just 
show the concept.

-- Joe



On Wednesday, February 13, 2019 at 11:22:46 AM UTC-8, Skiros wrote:
>
> Hello !
> Very interesting !!!
>
> I'm trying to reproduce this behaviour but I'm getting this error 
> "Expression object has no attribute on". What version of w2p are you using ?
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: In praise of the lowly "nested_select"

[Joe Barnhart]

Sorry I didn't see this question earlier.

The "on" attribute is on the Table object.  If you look at the code, I use 
variables like

dbt = db.times

This allows me to use "dbt.xxx" to access the Table object directly.  I 
find it a convenient syntactic shorthand because I'm always using it to get 
to Fields and other table-related items.

So, to reiterate, the "on" function is in the Table, not Expression.

-- Joe


On Wednesday, February 13, 2019 at 11:22:46 AM UTC-8, Skiros wrote:
>
> Hello !
> Very interesting !!!
>
> I'm trying to reproduce this behaviour but I'm getting this error 
> "Expression object has no attribute on". What version of w2p are you using ?
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Redis and Python3?

[Joe Barnhart]

Hi Leonel --

My brain refuses to put together the words "session locked field".  I 
understand the part of pickle smashing the values and encoding them as one 
string.  The bool has something to do with session locking?  

I've only looked at the web2py redis code in the most scant way as I wasn't 
planning to become a redis expert.  Sessions in redis could be important to 
me, tho, so I may need to set aside some time to dig into it.  (Storing 
sessions in cookies has been problematic due to the limited space for 
cookies in browsers.)

-- Joe


On Wednesday, March 27, 2019 at 4:49:25 PM UTC-7, Leonel Câmara wrote:
>
> It's not your values Joe, pickle serializes them as a string so they would 
> be fine. It's web2py session locked field which is True or False. It's 
> probably easy to fix this in redis_session.py
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [web2py] Re: Not my type

[Joe Barnhart]

I suspect you were actually running Python 3.x in which case "print xxx" 
doesn't work because print has been mad into a function (requiring 
parentheses).  When I start each on my Mac, this is the display I get:

Python 2.7:

ssmain:~ jbarnhart$ python

Python 2.7.10 (default, Feb 22 2019, 21:17:52)

[GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14)] on darwin

Type "help", "copyright", "credits" or "license" for more information.

>>> 


Python 3.7:

ssmain:~ jbarnhart$ 

ssmain:~ jbarnhart$ python3

Python 3.7.2 (default, Feb 12 2019, 08:15:36) 

[Clang 10.0.0 (clang-1000.11.45.5)] on darwin

Type "help", "copyright", "credits" or "license" for more information.

>>> 

You should be able to tell the version by the startup message.  If you are 
actually running Python 2 and the code fails, copy and paste the actual 
error message you get and we'll puzzle it out.

Warm regards,
Joe




On Friday, March 29, 2019 at 7:32:52 AM UTC-7, Al Hart wrote:
>
> Hi Joe,
>
> Thanks so much for responding. Let me be more specific. I was referring to a 
> section in the book, chapter two, on types 
> <http://web2py.com/books/default/chapter/29/02/the-python-language#Types>.
> In there it suggests
>
> >>> a = 3>>> print type(a)
>>
>> But that didn't work for me, I got error messages (don't recall what they 
> were at this moment).
> However, when I tried it using two other methods...
> 1.
>
>> >>> a = 3 
>
> >>> type(a)
>
> or
> 2.
>
>> >>> a = 3 
>
> >>> print(type(a))
>
>
> Both of those worked for me. Kind of weird because when I checked which 
> version of python I had running  it said 2.7 but perhaps I installed w2p 
> with python3.
>
> Anyway, at the end I was suggesting that maybe we could update the book 
> some making reference to how things might look with python3. I wasn't sure 
> if that sort of thing is just done here, by mentioning it in the forum and 
> someone will get to it, or if it might be done by pull request.
>
> Best,
>
> Al
>
> On Thu, Mar 28, 2019 at 9:30 PM Joe Barnhart  > wrote:
>
>> Hi Al --
>>
>> First off, welcome to the web2py group.  Next, it's not clear from your 
>> message just what the question is.  If you can elaborate on (a) what you 
>> did, (b) what you expected, and (c) what you got, I'm sure someone here can 
>> help
>>
>> Warm regards,
>>
>> Joe B.
>>
>> On Thursday, March 28, 2019 at 2:10:50 AM UTC-7, Al Hart wrote:
>>>
>>> Hi folks, if you'll pardon the corny title, I am brand new to web2py (so 
>>> excited to discover it) and I'm just working may way through the book. In 
>>> Chapter two, the section on types, I tried to run the examples, but I got 
>>> error messages. Googling around it seemed to work better if I went one of 
>>> two ways:
>>>
>>>
>>>1. a = 3
>>>type(a)
>>>2. a = 4 
>>>print(type(a))
>>>
>>>
>>> I'm on Ubuntu 18.04. Not sure if the example is based on python 3? If 
>>> not, is this the best way to suggest updates to the book or should we just 
>>> try a pull request? 
>>>
>> -- 
>> Resources:
>> - http://web2py.com
>> - http://web2py.com/book (Documentation)
>> - http://github.com/web2py/web2py (Source code)
>> - https://code.google.com/p/web2py/issues/list (Report Issues)
>> --- 
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "web2py-users" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/web2py/VityJ20AV2M/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to 
>> web...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: web3py again....

[Joe Barnhart]

I agree that back porting web2py code should be straightforward (after all, 
it's not rocket science :-D).  But maybe we should revisit those modules 
and pare them down to minimalist proportions.  I really need Auth, complete 
with user roles and groups, but there's probably a better way to 
minimize/abstract and provide a growth path.

Plus, with py3 and web2py in hand, its not like there's a fire that needs 
putting out.

-- Joe


On Thursday, March 28, 2019 at 2:16:15 AM UTC-7, Massimo Di Pierro wrote:
>
> I know this comes up once in a while but ...
>
> https://github.com/web2py/web3py
>
> what do you think? 
>
> It is very preliminary but I will have an admin soon.
> Backporting SQLFORM, grid and Auth, should not be that complicated, 
> although I think we can do better.
>
> Massimo
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Not my type

[Joe Barnhart]

Hi Al --

First off, welcome to the web2py group.  Next, it's not clear from your 
message just what the question is.  If you can elaborate on (a) what you 
did, (b) what you expected, and (c) what you got, I'm sure someone here can 
help

Warm regards,

Joe B.

On Thursday, March 28, 2019 at 2:10:50 AM UTC-7, Al Hart wrote:
>
> Hi folks, if you'll pardon the corny title, I am brand new to web2py (so 
> excited to discover it) and I'm just working may way through the book. In 
> Chapter two, the section on types, I tried to run the examples, but I got 
> error messages. Googling around it seemed to work better if I went one of 
> two ways:
>
>
>1. a = 3
>type(a)
>2. a = 4 
>print(type(a))
>
>
> I'm on Ubuntu 18.04. Not sure if the example is based on python 3? If not, 
> is this the best way to suggest updates to the book or should we just try a 
> pull request? 
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Redis and Python3?

[Joe Barnhart]

Hi Massimo --

I think I understand your point.  But from where I sit it appears that 
Redis cannot handle even VALUES as "bool".  They now say it must be mapped 
to an integer or a string.  I'm pretty sure I'm not using a bool as a key 
in any dictionaries (they make lousy keys) but I AM using numerous bools as 
VALUES in dictionaries.

Do you think there is some kind of workaround to convert bools to strings 
for the purpose of storing sessions in Redis?

Warm regards,

Joe

On Sunday, March 24, 2019 at 11:45:07 AM UTC-7, Massimo Di Pierro wrote:
>
> Hello Joe,
> based on this 
> https://github.com/andymccurdy/redis-py/issues/1071#issuecomment-439199774
> looks like Redis had a backward incompatible changes and keys in 
> dictionaries cannot be bool (values can).
> I reviewed the web2py code and I can confirm no change on our side.
>
> Random thoughts: It was a design mistake on my side to allow pickle to 
> store sessions. I should have used json. This would have restricted what 
> can go in a session but would have made the serialized session more 
> portable and the whole logic faster. In that case only str keys would have 
> been allowed. Web3py will use json for sessions or move away from sessions 
> completely.
>
> Massimo
>
> On Tuesday, 19 March 2019 21:07:43 UTC-7, Joe Barnhart wrote:
>>
>> Has something changed in the operation of redis to store sessions in 
>> web2py when we use python3 instead of python2?  I'm hitting an error in the 
>> py-redis library that complains I can't store a boolean in redis 
>> (unsupported data type).  Since this seems to be a well known design 
>> feature of redis I'm wondering how it seemed to work before.  It's been 
>> awhile since I used it, but I swear it worked before.  Didn't it?  Maybe I 
>> dreamed it.
>>
>> -- Joe
>>
>> Message:
>>
>> 1.
>> 2.
>> 3.
>> 4.
>> 5.
>> 6.
>> 7.
>> 8.
>> 9.
>> 10.
>> 11.
>> 12.
>> 13.
>> 14.
>> 15.
>> 16.
>> 17.
>> 18.
>> 19.
>> 20.
>> 21.
>> 22.
>> 23.
>> 24.
>> 25.
>> 26.
>> 27.
>> 28.
>>
>> Traceback (most recent call last):
>>   File "/Users/jbarnhart/Work/w2p/w2p_ss_new/gluon/main.py", line 444, in 
>> wsgibase
>> serve_controller(request, response, session)
>>   File "/Users/jbarnhart/Work/w2p/w2p_ss_new/gluon/main.py", line 206, in 
>> serve_controller
>> raise HTTP(response.status, page, **response.headers)
>> gluon.http.HTTP: 200 OK
>>
>> During handling of the above exception, another exception occurred:
>>
>> Traceback (most recent call last):
>>   File "/Users/jbarnhart/Work/w2p/w2p_ss_new/gluon/main.py", line 460, in 
>> wsgibase
>> session._try_store_in_db(request, response)
>>   File "/Users/jbarnhart/Work/w2p/w2p_ss_new/gluon/globals.py", line 1239, 
>> in _try_store_in_db
>> record_id = table.insert(**dd)
>>   File 
>> "/Users/jbarnhart/Work/w2p/w2p_ss_new/gluon/contrib/redis_session.py", line 
>> 150, in insert
>> pipe.execute()
>>   File 
>> "/Users/jbarnhart/.virtualenv/py3/lib/python3.7/site-packages/redis/client.py",
>>  line 3520, in execute
>> return execute(conn, stack, raise_on_error)
>>   File 
>> "/Users/jbarnhart/.virtualenv/py3/lib/python3.7/site-packages/redis/client.py",
>>  line 3385, in _execute_transaction
>> all_cmds = connection.pack_commands([args for args, options in cmds
>>   File 
>> "/Users/jbarnhart/.virtualenv/py3/lib/python3.7/site-packages/redis/connection.py",
>>  line 687, in pack_commands
>> for chunk in self.pack_command(*cmd):
>>   File 
>> "/Users/jbarnhart/.virtualenv/py3/lib/python3.7/site-packages/redis/connection.py",
>>  line 663, in pack_command
>> for arg in imap(self.encoder.encode, args):
>>   File 
>> "/Users/jbarnhart/.virtualenv/py3/lib/python3.7/site-packages/redis/connection.py",
>>  line 114, in encode
>> raise DataError("Invalid input of type: 'bool'. Convert to a "
>> redis.exceptions.DataError: Invalid input of type: 'bool'. Convert to a 
>> byte, string or number first.
>>
>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Redis and Python3?

[Joe Barnhart]

Has something changed in the operation of redis to store sessions in web2py 
when we use python3 instead of python2?  I'm hitting an error in the 
py-redis library that complains I can't store a boolean in redis 
(unsupported data type).  Since this seems to be a well known design 
feature of redis I'm wondering how it seemed to work before.  It's been 
awhile since I used it, but I swear it worked before.  Didn't it?  Maybe I 
dreamed it.

-- Joe

Message:

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.

Traceback (most recent call last):
  File "/Users/jbarnhart/Work/w2p/w2p_ss_new/gluon/main.py", line 444, in 
wsgibase
serve_controller(request, response, session)
  File "/Users/jbarnhart/Work/w2p/w2p_ss_new/gluon/main.py", line 206, in 
serve_controller
raise HTTP(response.status, page, **response.headers)
gluon.http.HTTP: 200 OK

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jbarnhart/Work/w2p/w2p_ss_new/gluon/main.py", line 460, in 
wsgibase
session._try_store_in_db(request, response)
  File "/Users/jbarnhart/Work/w2p/w2p_ss_new/gluon/globals.py", line 1239, in 
_try_store_in_db
record_id = table.insert(**dd)
  File "/Users/jbarnhart/Work/w2p/w2p_ss_new/gluon/contrib/redis_session.py", 
line 150, in insert
pipe.execute()
  File 
"/Users/jbarnhart/.virtualenv/py3/lib/python3.7/site-packages/redis/client.py", 
line 3520, in execute
return execute(conn, stack, raise_on_error)
  File 
"/Users/jbarnhart/.virtualenv/py3/lib/python3.7/site-packages/redis/client.py", 
line 3385, in _execute_transaction
all_cmds = connection.pack_commands([args for args, options in cmds
  File 
"/Users/jbarnhart/.virtualenv/py3/lib/python3.7/site-packages/redis/connection.py",
 line 687, in pack_commands
for chunk in self.pack_command(*cmd):
  File 
"/Users/jbarnhart/.virtualenv/py3/lib/python3.7/site-packages/redis/connection.py",
 line 663, in pack_command
for arg in imap(self.encoder.encode, args):
  File 
"/Users/jbarnhart/.virtualenv/py3/lib/python3.7/site-packages/redis/connection.py",
 line 114, in encode
raise DataError("Invalid input of type: 'bool'. Convert to a "
redis.exceptions.DataError: Invalid input of type: 'bool'. Convert to a byte, 
string or number first.

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: how can I get row representation set in 'format' table option

[Joe Barnhart]

Here's an example from my site:

Field("sender", "reference auth_user",
  default=lambda: current.auth.user.id,
  represent=db.auth_user._format,
  requires=IS_NULL_OR(IS_IN_DB(db,'auth_user.id'))),

As you can see, it can be accessed as the attribute _format on any table.

-- Joe

On Tuesday, October 23, 2018 at 5:53:01 AM UTC-7, Shawn Michaels wrote:
>
>
> Hi, I have this minimal example:
>
> db.define_table('entry_type',
> Field('label'),
> format='%(label)s'
> )
>
> how can I get the row represented by format?
>
> record = db['entry_type'](id)
>
> gives me requested row and I need to get it's representation set by format.
>
> Thank you in advance!
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Limits of storing session in browser cookie?

[Joe Barnhart]

Ah.  Thanks for that.  I don't have full control over what goes into 
session.  I was suspicious because something which worked normally did not 
work when I was impersonating a user.  I did notice the rather large amount 
of info stored in the session while impersonating, which got me wondering.

I suppose Redis is probably a more reliable solution at this point.  Some 
day we may be able to divide our session data into multiple cookies and 
take advantage of the 4K per cookie limit.  (At least I assume it is "per 
cookie").  It's probably a crutch that we've been able to ignore the amount 
of session data all these years with web2py.  It made my session fat and 
lazy!

-- Joe

On Thursday, September 27, 2018 at 3:51:37 PM UTC-7, Anthony wrote:
>
> You'll be limited to 4KB (post-encryption). I suppose a reasonable 
> alternative would be to use sessionStorage or localStorage in the browser 
> (depending on your use case).
>
> Also, there is a bug if using cookie sessions with AuthAPI (as opposed to 
> Auth) -- on logout, if the session is empty (i.e., nothing left after 
> auth.user is removed), then web2py does not send a replacement cookie 
> (because there is no data), so the browser simply sends the original cookie 
> right back on the next request, and the user is still logged in. The 
> workaround is on logout, add some dummy data to the session just to force 
> web2py to send a new cookie to the browser to replace the old one. Note, 
> quite incidentally, this bug does not affect Auth because the logout method 
> in Auth adds "flash" to the session after logout, which forces a new cookie 
> to be sent to the browser.
>
> Anthony
>
> On Thursday, September 27, 2018 at 12:17:28 PM UTC-4, Joe Barnhart wrote:
>>
>> I love the new(?) capability of storing the session information in a 
>> browser cookie.  Mainly because I'm lazy and don't want the responsibility 
>> of determining when to throw out sessions and manage them on my end.
>>
>> I added this to my app:
>>
>> session.connect(request, response, 
>> cookie_key="mylittleponytasteslikechicken")
>>
>> Everything works mostly.  I have seen some corner cases not working 
>> correctly, such as a form that doesn't have a "formkey" stored in the 
>> session when I am impersonating a user.  Paging thru the session variable 
>> in this last case I noticed there's a LOT of stuff in the session, and I 
>> wonder if the size of the cookie might be causing its truncation?
>>
>> What are the practical limits of using browser cookies to hold session 
>> data, and are there workaround for large-ish sessions?
>>
>> Joe
>>
>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [web2py] Setting writable and readable=True on all fields

[Joe Barnhart]

Yep.

I'm pretty sure 'fields' is a list so it doesn't need function parentheses. 
 And since "my_table" and "db['my_table']" are the same:

  for f in my_table.fields:
 my_table[f].writable=True

Same thing, but sweetened with syntactic sugar.  (low calorie)

-- Joe

On Thursday, September 27, 2018 at 3:04:04 PM UTC-7, Javier Pepe wrote:
>
> You need use the follow code:
> for f in my_table.fields():
> db['my_table'][f].writable=True
>
>
>
> El jue., 27 de sep. de 2018 a la(s) 19:01, icodk (i...@alpiron.com 
> ) escribió:
>
>> At a certain point, I need to allow read and write on all fields of a 
>> specific table
>>
>> How can I iterate through all fields in a table and set the writable, 
>> readable =true
>>
>> This does not work:
>> for f in my_table.fields():
>>f.writable=f.readable=True
>>
>>
>>
>> -- 
>> Resources:
>> - http://web2py.com
>> - http://web2py.com/book (Documentation)
>> - http://github.com/web2py/web2py (Source code)
>> - https://code.google.com/p/web2py/issues/list (Report Issues)
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "web2py-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to web2py+un...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: How to capture auth_user login time & logout time

[Joe Barnhart]

What Anthony said.  Only sessions rarely log out.  They mostly time out, 
which does not show up as an event.

Joe

On Wednesday, September 26, 2018 at 12:13:25 PM UTC-7, mostwanted wrote:
>
> Hi guys, i was wondering if there is a way for me to capture user login 
> time and user logout time
>
> Regards
>
> Moatwanted
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Limits of storing session in browser cookie?

[Joe Barnhart]

I love the new(?) capability of storing the session information in a 
browser cookie.  Mainly because I'm lazy and don't want the responsibility 
of determining when to throw out sessions and manage them on my end.

I added this to my app:

session.connect(request, response, 
cookie_key="mylittleponytasteslikechicken")

Everything works mostly.  I have seen some corner cases not working 
correctly, such as a form that doesn't have a "formkey" stored in the 
session when I am impersonating a user.  Paging thru the session variable 
in this last case I noticed there's a LOT of stuff in the session, and I 
wonder if the size of the cookie might be causing its truncation?

What are the practical limits of using browser cookies to hold session 
data, and are there workaround for large-ish sessions?

Joe

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: How to add variable in function which called inside ajax function

[Joe Barnhart]

$.ajax({
  data: {a:2, b:3},
  url: "{{=URL('loadAdditonalInfo')}}",
  success: my_success_fcn(ret)
)};


Something like this, from memory...

You can get more details at the jQuery website.  It has excellent docs.

Joe

On Tuesday, September 4, 2018 at 6:23:03 AM UTC-7, Константин Комков wrote:
>
> Hello, can you take advice can I send variables in web2py function - ajax 
> or I need to use jquery function.
> Can somebody show example how to send variables with jQuery.ajax and get 
> answer from server (controller and js in view).
> For example we have a = 2 and b = 3. I want to get from server a+b.
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: pydal metadata in db

[Joe Barnhart]

I'm worried about losing control over the db migration process.  Using 
MySql it is fairly common for the migration process to go wrong, as MySql 
doesn't do will with adding and deleting multiple columns.  I often need to 
drop a few tables, delete the corresponding file in 'databases' and then 
force recreating the tables.  It's easier than manually editing the tables 
in a MySql session and doing a "fake" migrate.

Will this change make it harder to recover from migrations that don't go 
well, turning it into an "all or nothing" approach?  That would create some 
difficulty for me unless I have good workarounds.

Joe

On Sunday, September 2, 2018 at 11:08:50 AM UTC-7, Massimo Di Pierro wrote:
>
> We you may know web2py has the ability to store table metadata in DB:
>
> from gluon.dal import InDBMigrator
> db = DAL(myconf.get('db.uri'), adapter_args=dict(migrator=InDBMigrator))
>
> This is better for scalability as there is no filsystem IO.
>
> How do people feel with the following:
> - make the above the default behavior (no more metadata/*.table)
> - eliminate logging in databases/sql.log as use log.info instead
> - make migration always on on localhost and appadmin and always off 
> otherwise
> - create a script and an appadmin endpoint for fake_migration (partial 
> database repair)
>
> This can be done with a change to the welcome app in a backward compatible 
> manner.
> Hardest part is update the docs.
>
> Massimo
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Hidden fields in SQLFORMs are one-way?

[Joe Barnhart]

Hi Anthony --

My issue was that I wanted to change the value of the hidden field 
dynamically, i.e. after its value was set in SQLFORM.

Nosing around in the code, I did find my ultimate answer.  The FORM object 
holds the hidden attributes in an attribute... called "hidden". I could go 
edit the hidden field value directly and my new value is rendered when the 
xml method is called to build the view.


form.attributes['hidden']['a'] = 'new value'


-- Joe

On Monday, August 27, 2018 at 6:39:50 AM UTC-7, Anthony wrote:
>
> From the book:
>
> form = SQLFORM(...,hidden=...)
>
> causes the hidden fields to be passed with the submission, no more, no 
> less. form.accepts(...) is not intended to read the received hidden 
> fields and move them into form.vars. The reason is security. Hidden fields 
> can be tampered with. So you have to explicitly move hidden fields from the 
> request to the form:
>
> 1
> 2
>
> form.vars.a = request.vars.aform = SQLFORM(..., hidden=dict(a='b'))
>
>
>
> Unfortunately, the code in that last example is incorrect -- line 1 should 
> go *after* line 2.
>
> So, you can add values from request.vars to form.vars before calling 
> .validate() or .process(), and the values will be available in any 
> onvalidation callback (of course, you can also refer directly to 
> request.vars anywhere).
>
> Anthony
>
>
> On Monday, August 27, 2018 at 12:37:57 AM UTC-4, Joe Barnhart wrote:
>>
>> So I'm trying to use a hidden field to create some unique behavior in a 
>> form.  I want to send out the field when the form is first created (using 
>> the hidden=dict(...) keyword on SQLFORM).  But later, while processing the 
>> form through the error phase, I want to update that value to a different 
>> value using a "form.vars" or "request.post.vars" kind of mechanism.  But 
>> form.vars does not have hidden fields, and putting in a matching value for 
>> my hidden field is ignored when the form is passed back.
>>
>> I can't change its value in the SQLFORM() process because the new value 
>> of the field is only known AFTER form creation and submission.  When it 
>> comes back, THEN I know what I want the value to be, but by this time the 
>> SQLFORM method has re-run and the value of the hidden field is fixed.
>>
>> I'd like something like:
>>
>> form = SQLFORM( hidden=dict(a='b')
>> . . . 
>> if form.validate():
>> . . .
>> elif form.errors:
>> form.var.a = 'c'
>>
>> What am I missing?  Are hidden fields dumb, one-way-trip only fields? 
>>  Are they not modifiable along with the other fields?
>>
>> -- Joe
>>
>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Hidden fields in SQLFORMs are one-way?

[Joe Barnhart]

So I'm trying to use a hidden field to create some unique behavior in a 
form.  I want to send out the field when the form is first created (using 
the hidden=dict(...) keyword on SQLFORM).  But later, while processing the 
form through the error phase, I want to update that value to a different 
value using a "form.vars" or "request.post.vars" kind of mechanism.  But 
form.vars does not have hidden fields, and putting in a matching value for 
my hidden field is ignored when the form is passed back.

I can't change its value in the SQLFORM() process because the new value of 
the field is only known AFTER form creation and submission.  When it comes 
back, THEN I know what I want the value to be, but by this time the SQLFORM 
method has re-run and the value of the hidden field is fixed.

I'd like something like:

form = SQLFORM( hidden=dict(a='b')
. . . 
if form.validate():
. . .
elif form.errors:
form.var.a = 'c'

What am I missing?  Are hidden fields dumb, one-way-trip only fields?  Are 
they not modifiable along with the other fields?

-- Joe

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Need to add form "warnings" in addition to "errors"

[Joe Barnhart]

Well, this idea works... basically.  But I need to change the DIV enclosing 
my "warning" message from class "error" to "warning" to get my preferred 
yellow color.  Now here's where it gets interesting...

I can change the div using jQuery on the view.  BUT that still shows me a 
"flash" of red before the box turns to yellow.  No problem, I thought, I'll 
use web2py "elements" to do the change before the page is rendered.

NOPE!

In a bizarre twist, the DIV containing the error block is never rendered 
until the last possible moment -- during the "xml" method.  Which means, 
there is no point at which you can use "elements" on the form to find the 
error DIV because it doesn't exist yet!

There is nothing "wrong" with any of this. Every design choice is a 
compromise at some level.  This is just the unfortunate collision of design 
choices to limit the usefulness of "elements" in this one corner case.  The 
red flash isn't that bad.  I'll just say I'm doing it deliberately to get 
the user's attention.  

"A feature is a bug as described by Marketing..."

On Wednesday, August 22, 2018 at 4:09:49 PM UTC-7, Joe Barnhart wrote:
>
> I hate to answer my own post, but what if I did something in a customer 
> Validator?  What if the error returned by my validator was something like 
> SPAN("blah blah", _class="warning").  Presumably I could search the FORM 
> using elements() finding my SPANs and then modify the enclosing DIVs to be 
> class "warning" also.  I'd provide different CSS for the yellow background. 
>  Just thinking out loud here.
>
> On Wednesday, August 22, 2018 at 3:31:14 PM UTC-7, Joe Barnhart wrote:
>>
>> I'm looking to add "warnings" to forms using (/hijacking) as much of the 
>> existing form "errors" mechanism as possible.  The idea is to provide a 
>> second level check that gives advice rather than drop-dead errors.
>>
>> I would like it to operate like this:
>>
>> 1.  The form is submitted and checked for errors as usual. 
>>
>> 2.  After the error check another set of criteria, "warnings",  are 
>> checked.  Where the errors are added as red elements below the fields, I'd 
>> like these to be added in yellow to delineate the difference.
>>
>> 3.  The user gets a chance to fix the errors and warnings (as in standard 
>> practice).
>>
>> 4.  If the error check passes but there are still warnings, the warnings 
>> are displayed.
>>
>> 5.  Whatever the user submits after the "warnings only" check is accepted.
>>
>> I can figure out the two-phase form submit.  I was having a little 
>> difficulty trying to imagine how to hijack the error message code that adds 
>> the error wrapper.  Currently that's done in the xml method for the INPUT, 
>> for example, and I don't see an easy way to extend that without subclassing 
>> each of the controls and adding my stuff.  Even then, the Fields and such 
>> would want to add the original INPUT and not my subclassed version, leading 
>> to more fun and hilarity. 
>>
>> Any simple ideas from the web2py blackbelts?  I'm a brownbelt and can 
>> probably implement it if you give me a hint.
>>
>> -- Joe
>>
>> P.S. An example of a warning would be if a user fills in his parent's 
>> last name as his first name.  It's possible, but much more likely they just 
>> switched the first name / last name fields.  (Happens a lot, for some 
>> reason.)
>>
>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Need to add form "warnings" in addition to "errors"

[Joe Barnhart]

I hate to answer my own post, but what if I did something in a customer 
Validator?  What if the error returned by my validator was something like 
SPAN("blah blah", _class="warning").  Presumably I could search the FORM 
using elements() finding my SPANs and then modify the enclosing DIVs to be 
class "warning" also.  I'd provide different CSS for the yellow background. 
 Just thinking out loud here.

On Wednesday, August 22, 2018 at 3:31:14 PM UTC-7, Joe Barnhart wrote:
>
> I'm looking to add "warnings" to forms using (/hijacking) as much of the 
> existing form "errors" mechanism as possible.  The idea is to provide a 
> second level check that gives advice rather than drop-dead errors.
>
> I would like it to operate like this:
>
> 1.  The form is submitted and checked for errors as usual. 
>
> 2.  After the error check another set of criteria, "warnings",  are 
> checked.  Where the errors are added as red elements below the fields, I'd 
> like these to be added in yellow to delineate the difference.
>
> 3.  The user gets a chance to fix the errors and warnings (as in standard 
> practice).
>
> 4.  If the error check passes but there are still warnings, the warnings 
> are displayed.
>
> 5.  Whatever the user submits after the "warnings only" check is accepted.
>
> I can figure out the two-phase form submit.  I was having a little 
> difficulty trying to imagine how to hijack the error message code that adds 
> the error wrapper.  Currently that's done in the xml method for the INPUT, 
> for example, and I don't see an easy way to extend that without subclassing 
> each of the controls and adding my stuff.  Even then, the Fields and such 
> would want to add the original INPUT and not my subclassed version, leading 
> to more fun and hilarity. 
>
> Any simple ideas from the web2py blackbelts?  I'm a brownbelt and can 
> probably implement it if you give me a hint.
>
> -- Joe
>
> P.S. An example of a warning would be if a user fills in his parent's last 
> name as his first name.  It's possible, but much more likely they just 
> switched the first name / last name fields.  (Happens a lot, for some 
> reason.)
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: session data type changed after upgrade to 2.17.1

[Joe Barnhart]

So "session" stuff gets pickled and unpicked which makes sense that dates 
might come back as strings.  What puzzles me is that it "used to work" so 
did pickled date objects used to come back as date objects again?  That 
would seem like a much bigger change than just this example.

-- Joe

On Tuesday, August 21, 2018 at 1:28:44 PM UTC-7, icodk wrote:
>
> Turned out that after the upgrade  the   form.vars.end_date and 
> form.vars.end_date(both 
> defined as type 'date') returns a data type str instead of a 'date' type
>
> The form definition is:
>
> form = SQLFORM.factory(
> Field('start_date', 'date', label=T('Date range: '))
> ,Field('end_date', 'date', label=T(''))
> ,Field('amount',label=T('Total'),default=def_amount, widget=lambda f, v: 
> SQLFORM.widgets.string.widget(f, v, _disabled=True))
> ,Field('online_order','boolean',label=T('Online 
> orders'),default=session.online_order)
> ,Field('manual_order','boolean',label=T('Manual 
> orders'),default=session.manual_order)
>
>
> And then later the session vars gets 'infected'  as  str by the form.vars 
> that should return 'date'
>
> if form.process().accepted:
> session.start_date=*form.vars.start_date *# here the form.vars.start_date 
> is of type str
> session.end_date = *form.vars.end_date   *# here the form.vars.end_date 
> is of type str
> session.online_order=form.vars.online_order
> session.manual_order = form.vars.manual_order
>
>
>
>
>
>
>
> On Tuesday, August 21, 2018 at 12:56:34 PM UTC+2, icodk wrote:
>>
>> Have:
>>
>> session.end_date=request.now.date()
>>
>> and then retrieve it as:
>>
>> datetimeEnd=datetime.datetime.combine(session.end_date, datetime.time.max)
>>
>> this all worked fine in 2.14.6
>>
>> but after upgrade to 2.17.1 I get:
>>
>> datetimeEnd=datetime.datetime.combine(session.end_date, datetime.time.max)
>>
>> TypeError: combine() argument 1 must be datetime.date, not str
>>
>>
>>
>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Need to add form "warnings" in addition to "errors"

[Joe Barnhart]

I'm looking to add "warnings" to forms using (/hijacking) as much of the 
existing form "errors" mechanism as possible.  The idea is to provide a 
second level check that gives advice rather than drop-dead errors.

I would like it to operate like this:

1.  The form is submitted and checked for errors as usual. 

2.  After the error check another set of criteria, "warnings",  are 
checked.  Where the errors are added as red elements below the fields, I'd 
like these to be added in yellow to delineate the difference.

3.  The user gets a chance to fix the errors and warnings (as in standard 
practice).

4.  If the error check passes but there are still warnings, the warnings 
are displayed.

5.  Whatever the user submits after the "warnings only" check is accepted.

I can figure out the two-phase form submit.  I was having a little 
difficulty trying to imagine how to hijack the error message code that adds 
the error wrapper.  Currently that's done in the xml method for the INPUT, 
for example, and I don't see an easy way to extend that without subclassing 
each of the controls and adding my stuff.  Even then, the Fields and such 
would want to add the original INPUT and not my subclassed version, leading 
to more fun and hilarity. 

Any simple ideas from the web2py blackbelts?  I'm a brownbelt and can 
probably implement it if you give me a hint.

-- Joe

P.S. An example of a warning would be if a user fills in his parent's last 
name as his first name.  It's possible, but much more likely they just 
switched the first name / last name fields.  (Happens a lot, for some 
reason.)

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: New web2py/pydal feautures for scalability

[Joe Barnhart]

It took me a minute to decipher this post!  For the elucidation of others 
let me rephrase what I think you wrote...

Some users may not like STORING (not "strong") data in the filesystem. 
 Storing filesystem data is a bad idea when deploying the same website 
across multiple servers, i.e. it hinders "scalability" in web deployment.

You can now store the table definitions (currently in the "databases" 
directory of your web2py application) into the database itself.  This eases 
the problem of scaling the application at the cost of a little complexity. 
 First off, store "session" data on the user's web client by putting 
session data directly in the cookie.  Then configure DAL to store the 
"metadata" (i.e. "*.table" files) in the database itself, as shown below...

If I got it wrong, please let me know.

-- Joe

On Sunday, August 5, 2018 at 6:43:00 PM UTC-7, Massimo Di Pierro wrote:
>
> Some of you may not like using the filesystem for strong data. In 
> particular that is not a good solution for scalability. Now you can do:
>
> myconf = AppConfig(reload=True)
> # always store sessions in cookies
> session.connect(request, response, cookie_key=myconf.get('app.cookie_key'))
> # store metadata tables in db
> db = DAL(myconf.get('db.uri'),
>  pool_size=myconf.get('db.pool_size'),
>  migrate_enabled=request.is_local,  # trigger a migration when 
> called locally
>  fake_migrate_all=request.is_local and request.vars._fake, # 
> trigger a fake migration when passing ?_fake=1
>  adapter_args=dict(migrator=InDBMigrator), # store metadata tables 
> in db itself
>  check_reserved=['all'])
>
> Notice that if you change your code to do this in an exiting app, you must 
> run a fake migration.
>
>
>
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: How do you change memcache session expiry?

[Joe Barnhart]

Yes, I agree.  This needs to be REMOVED FROM THE BOOK.  I've just wasted 
most of a day before realizing memcache is useless for sessions.

-- Joe

On Friday, January 18, 2013 at 1:03:01 AM UTC-8, Paolo wrote:
>
> Hi all, I have just tried the solution proposed on the issue but it doen't 
> work. 
> This is a really annoying bug not only because I have to login several 
> times in a day but also because I can lose my edits if I don't save my 
> changes in time (i.e, editing a wiki page).
> It might be better to remove from the book the advise of storing the 
> session in memcached unless we find a solution
>
> Paolo
>
> Il giorno domenica 30 settembre 2012 00:54:03 UTC+2, Robert Clark ha 
> scritto:
>>
>> Thanks everyone, have added an issue
>> http://code.google.com/p/web2py/issues/detail?id=1049
>>
>> On Sunday, September 30, 2012 3:51:10 AM UTC+13, Massimo Di Pierro wrote:
>>>
>>> It also looks to me memdb should not be implemented as it is. It should 
>>> be implemented as a plugin_adapter for DAL.
>>>
>>> On Saturday, 29 September 2012 09:50:37 UTC-5, Massimo Di Pierro wrote:

 I would prefer the syntax:

 session.connect(request, response, db=MEMDB(cache.memcache, 
 session_expiry=3600))



 On Saturday, 29 September 2012 07:11:10 UTC-5, Niphlod wrote:
>
> yep, open a bug on http://code.google.com/p/web2py/issues/list
>
> On Saturday, September 29, 2012 5:24:07 AM UTC+2, Robert Clark wrote:
>>
>> Thanks, I am not having any problems with the memcached api & python 
>> interface, that part all works as advertised.
>>
>> The problem is that if you follow the deployment recipe for storing 
>> sessions in Memcached, then they always expire after 300s and there's no 
>> way to provide an expiry.  Here's what's the code from deployment recipe 
>> chapter of web2py book:
>>
>> from gluon.contrib.memcache import MemcacheClientmemcache_servers = 
>> ['127.0.0.1:11211']
>> cache.memcache = MemcacheClient(request, memcache_servers)
>> cache.ram = cache.disk = cache.memcache
>>
>> ..and..
>>
>> from gluon.contrib.memdb import MEMDB
>> session.connect(request,response,db=MEMDB(cache.memcache))
>>
>>
>> If you do this and connect to memcached with e.g. "-vv" you can see 
>> that session data is passed in with 300s expiry.  I may be missing 
>> something obvious.  Can I suggest altering the API to be something like 
>> this:
>>
>> session.connect(request, response, db=MEMDB(cache.memcache), 
>> session_expiry=3600)
>>
>> Thanks!
>>
>>
>> On Saturday, September 29, 2012 12:27:55 AM UTC+12, Jose C wrote:
>>>
>>> The only way I've found to change this is to explicitly modify the 
 default value from this source file.

 Is there a better way to configure this value for session expiry 
 from within application code?  Cheers.

 Far as I know it's done like this:  
>>>
>>> set(key=key, value=value, time=) 
>>>
>>> where (from the source):
>>> @param time: Tells memcached the time which this value should expire
>>> , either
>>> as a delta number of seconds, or an absolute unix time-since
>>> -the-epoch
>>> value. See the memcached protocol docs section "Storage 
>>> Commands"
>>> for more info on . We default to 0 == cache forever
>>> .
>>>
>>>  Are you saying that doesn't work?
>>>
>>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] In praise of the lowly "nested_select"

[Joe Barnhart]

How did I not know about this gem earlier?

I've been resorting to executesql() for my heavy joins-within-joins queries 
because, well, I thought I had to.  The select join syntax doesn't do 
nested joins directly and I'd never heard of nested_select.

Wow, what a great tool!  I just make my sub-join its own select, make an 
alias for it (required to use it with "on") and then put it in another 
query.  It is a thing of beauty.  (Well, maybe I have low standards of 
beauty!)

This result is the equivalent of a very complex executesql() query:

def gettimes(self, mid, eid, stage):
assert mid and eid and stage
db = current.db
dbt,dbc,dbs,dbei,dbts,dbti = 
db.times,db.club,db.swimmer,db.event_index,db.time_std,db.time_std_info 
cols = [dbt[c] for c in 
['ord','rank','evt','heat','lane','age','stime','stage','id']]
cols += [dbc[c] for c in ['code','name','id']]
cols += [dbei[c] for c in 
['course','sex','distance','stroke','lower_age','upper_age']]
cols += [dbs[c] for c in ['id','first','last','mi','preferred']]
subq = db(dbts.id>0).nested_select(dbts.ALL,

join=dbti.on((dbti.id==dbts.id_info)&(dbti.id_lsc=='51')&(dbti.code=='Default')))
dbtt = subq.with_alias('dbtt')
cols += [dbtt[c] for c in ['name','next','time']]
q = (dbt.id_meet==mid)&(dbt.id_event==eid)&(dbt.stage==stage)
rows = db(q).select(*cols,cacheable=True,processor=self.as_list,
   join=[dbei.on(dbei.id==dbt.id_event),
 dbc.on(dbc.id==dbt.id_club),
 dbs.on(dbs.id==dbt.id_swim)],
   left=dbtt.on
((dbtt.event_code==dbei.code2)&(dbt.stime>0)

&(dbtt.lower_age<=dbt.age)&(dbtt.upper_age>=dbt.age)

&(dbtt.next=dbt.stime)))
return rows

To save creation of Rows I used the "processor" keyword and made a little 
processor that basically returns a list of nested dicts, just as the normal 
"as_list" would to to a set of Rows.  But it does it without creating the 
Rows first, saving time and memory.

@staticmethod
def as_list(rows, fields, colnames, blob_decode=True, cacheable = 
False):
from collections import defaultdict
rtn = list()
cols = [(f.tablename,f.name) for f in fields]
for row in rows:
d = defaultdict(dict)
for r,v in zip(cols,row):
d[c[0]][c[1]] = r
rtn.append(d)
return rtn

I'm pleased to get really good performance and still keep the power and 
generality of DAL.  This feature might deserve additional explanation in 
the book.

-- Joe

If you want to see the executesql version, hold your nose and peek below. 
 I'm sure it is ghastly and could be done much better.  There are two 
variants I was playing with, one that uses Rows and as_list and another 
which makes the list of nested dicts directly.

def gettimes2(self, mid, eid, stage):
db = current.db
dbt, dbc, dbs, dbei, dbts, dbti = db.times, db.club, db.swimmer, 
db.event_index, db.time_std, db.time_std_info
assert mid and eid and stage
table_alias = 
{'times':'dbt','club':'dbc','swimmer':'dbs','event_index':'dbei','time_std':'dbtt'}
colnames = 
['times.ord','times.rank','times.evt','times.heat','times.lane','times.age','times.stime','times.stage','times.id',
'club.code','club.name','club.id',

'event_index.course','event_index.sex','event_index.distance','event_index.stroke',
'event_index.lower_age','event_index.upper_age',

'swimmer.id','swimmer.first','swimmer.last','swimmer.mi','swimmer.preferred',
'time_std.name','time_std.next','time_std.time']
split_colnames = [fld.split('.') for fld in colnames]
rnames=['.'.join([table_alias[t],db[t][f]._rname]) for t,f in 
split_colnames]
sqlvars = dict(eid=eid, mid=mid, stage=stage, 
fields=','.join(rnames))
stmt = """
SELECT %(fields)s FROM sstimes.times dbt
JOIN ssdata.event_index dbei ON dbei.id=dbt.id_event
JOIN ssdata.club dbc ON dbc.id=dbt.id_club
JOIN sstimes.swimmer dbs ON dbs.id=dbt.id_swim
LEFT JOIN (
SELECT dbts.*,dbti.id_lsc FROM ssdata.time_std dbts 
JOIN ssdata.time_std_info dbti 
ON dbti.id=dbts.id_info AND dbti.id_lsc=51 AND 
dbti.code='Default'
) dbtt ON (
dbtt.event_code=dbei.code2 AND dbt.stime>0
AND dbtt.lower_age<=dbt.age AND dbtt.upper_age>=dbt.age
AND dbtt.next=dbt.stime
) WHERE dbt.id_meet=%(mid)s AND dbt.id_event=%(eid)s AND 
dbt.stage='%(stage)s'
"""
if False:
fields = [db[t][f] for t,f in split_colnames]
rtn = db.executesql(stmt%sqlvars, fields=fields).as_list()

[web2py] Re: Using cache.action in a module

[Joe Barnhart]

I developed a widget pattern that resembles the Auth class.  It returns a 
dictionary when you create at the controller level.  During the first 
creation of the widget (which is in a module) it builds structure and 
passes it out in a dict() to make the page.  Later when Ajax is called it 
bypasses the creation of the structure and just goes straight to the bit 
that returns json.

I guess I could cache it at the controller level.  But the incoming url is 
like "scheme://app/control/func/subfunc?vars...".  So I guess I'd parse it 
at the controller level and then apply caching only for 
"func/subfunc?vars..." which is the json-returning part.  I've not really 
used cache.action much and it has a ton of options.

But maybe I should bite the bullet and plan on using something like 
memcache or redis for this.  I realized that these json call backs are 
ideal to cache because their data is (a) relatively large (~50k), (b) 
relatively costly to get (~300ms), and (c) unchanging for all time and all 
users.  They depend only on the supplied vars for uniqueness.  When I cache 
them the app really pops!

-- Joe


On Wednesday, August 1, 2018 at 11:28:39 AM UTC-7, Anthony wrote:
>
> By the way, why do you want to use cache.action for a non-action? Even if 
> this is an Ajax request, can't you still decorate the action that receives 
> the request? If you only want to cache part of the generated response, then 
> why not just use current.cache.ram, etc.?
>
> Anthony
>
> On Wednesday, August 1, 2018 at 11:17:55 AM UTC-4, Anthony wrote:
>>
>> Sorry, didn't realize you were trying to decorate a method. cache.action 
>> is designed to cache controller actions, so it is not expecting to decorate 
>> any functions that take arguments (methods always take at least the self 
>> argument).
>>
>> On Wednesday, August 1, 2018 at 4:27:02 AM UTC-4, Joe Barnhart wrote:
>>>
>>> Ah, well.  It seems a bit beyond me.  I think it's failing because I'm 
>>> caching a bound instance method (i.e. has "self").  I get "wrapped+_f takes 
>>> no arguments, 1 given)".
>>>
>>> def lazy_cache_action(time_expire=DEFAULT_TIME_EXPIRE, cache_model=None,
>>>prefix=None, session=False, vars=True, lang=True,
>>>user_agent=False, public=True, valid_statuses=None,
>>>quick=None):
>>> def decorator(f, time_expire=time_expire, cache_model=cache_model, 
>>> prefix=prefix, 
>>> session=session, vars=vars, lang=lang, 
>>> user_agent=user_agent, public=public, 
>>> valid_statuses=valid_statuses, quick=quick):
>>> def g(*c, **d):
>>> from gluon import current
>>> return current.cache.action(time_expire, cache_model, 
>>> prefix, session, 
>>> vars, lang, user_agent, public, valid_statuses, 
>>> quick)(f)(*c, **d)
>>> g.__name__ = f.__name__
>>> return g
>>> return decorator
>>>
>>>
>>>
>>>
>>> On Wednesday, August 1, 2018 at 12:50:12 AM UTC-7, Joe Barnhart wrote:
>>>>
>>>> Oops, I meant of course;
>>>>
>>>> current.cache.action
>>>>
>>>> instead of
>>>>
>>>> current.cache
>>>>
>>>>
>>>>
>>>>
>>>> On Wednesday, August 1, 2018 at 12:48:00 AM UTC-7, Joe Barnhart wrote:
>>>>>
>>>>> You're a fountain of ideas!  I missed that one in the book.
>>>>>
>>>>> I wonder if this would work.  Off to go try it...
>>>>>
>>>>> def lazy_cache_action(self, time_expire=DEFAULT_TIME_EXPIRE, 
>>>>> cache_model=None,
>>>>>prefix=None, session=False, vars=True, lang=True,
>>>>>user_agent=False, public=True, valid_statuses=None,
>>>>>    quick=None):
>>>>> def decorator(f, time_expire, cache_model, prefix, session, vars, 
>>>>> lang,
>>>>>user_agent, public, valid_statuses, quick):
>>>>> def g(*c, **d):
>>>>> from gluon import current
>>>>> return current.cache(f, time_expire, cache_model, prefix, 
>>>>> session, vars,
>>>>> lang, user_agent, public, valid_statuses, 
>>>>> quick)(f)(*c, **d)
>>>>> g.__name__ = f.__name__
>>>>> return g
>>>>> return decorator
>>>>>
>>>>>
&

[web2py] Re: Using cache.action in a module

[Joe Barnhart]

Ah, well.  It seems a bit beyond me.  I think it's failing because I'm 
caching a bound instance method (i.e. has "self").  I get "wrapped+_f takes 
no arguments, 1 given)".

def lazy_cache_action(time_expire=DEFAULT_TIME_EXPIRE, cache_model=None,
   prefix=None, session=False, vars=True, lang=True,
   user_agent=False, public=True, valid_statuses=None,
   quick=None):
def decorator(f, time_expire=time_expire, cache_model=cache_model, 
prefix=prefix, 
session=session, vars=vars, lang=lang, user_agent=user_agent, 
public=public, 
valid_statuses=valid_statuses, quick=quick):
def g(*c, **d):
from gluon import current
return current.cache.action(time_expire, cache_model, prefix, 
session, 
vars, lang, user_agent, public, valid_statuses, 
quick)(f)(*c, **d)
g.__name__ = f.__name__
return g
return decorator




On Wednesday, August 1, 2018 at 12:50:12 AM UTC-7, Joe Barnhart wrote:
>
> Oops, I meant of course;
>
> current.cache.action
>
> instead of
>
> current.cache
>
>
>
>
> On Wednesday, August 1, 2018 at 12:48:00 AM UTC-7, Joe Barnhart wrote:
>>
>> You're a fountain of ideas!  I missed that one in the book.
>>
>> I wonder if this would work.  Off to go try it...
>>
>> def lazy_cache_action(self, time_expire=DEFAULT_TIME_EXPIRE, 
>> cache_model=None,
>>prefix=None, session=False, vars=True, lang=True,
>>user_agent=False, public=True, valid_statuses=None,
>>quick=None):
>> def decorator(f, time_expire, cache_model, prefix, session, vars, 
>> lang,
>>user_agent, public, valid_statuses, quick):
>> def g(*c, **d):
>> from gluon import current
>> return current.cache(f, time_expire, cache_model, prefix, 
>> session, vars,
>> lang, user_agent, public, valid_statuses, quick)(f)(*c, 
>> **d)
>> g.__name__ = f.__name__
>> return g
>> return decorator
>>
>>
>>
>> On Tuesday, July 31, 2018 at 11:25:10 AM UTC-7, Anthony wrote:
>>>
>>> On Tuesday, July 31, 2018 at 1:57:46 AM UTC-4, Joe Barnhart wrote:
>>>>
>>>> I was wondering about this.  I tried to search the group but didn't 
>>>> find anything relevant.  Took a look at the source code and it seemed like 
>>>> I could use in a module which is called to produce a string of Javascript 
>>>> on demand of an Ajax routine.
>>>>
>>>> Beforehand, I save the global "cache" var in my "current" object.  Then 
>>>> I rename my method "content" to "__content__", and last I do this:
>>>>
>>>> def content(self):
>>>> c = current.cache
>>>> return c.action(cache_model=c.disk, 
>>>> quick="VP")(self.__content__)()
>>>>
>>>> Seems to work.  Am I asking for trouble?  Is there anything I should 
>>>> watch for?
>>>>
>>>
>>> Seems reasonable. You could also create a custom decorator, similar to 
>>> lazy_cache 
>>> <https://github.com/web2py/web2py/blob/master/gluon/cache.py#L728-L746> 
>>> (see the end of this section: 
>>> http://web2py.com/books/default/chapter/29/04/the-core#Warning--Do-not-use-the-current-object-in-global-scope-in-a-module
>>> ).
>>>
>>> Anthony
>>>
>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Using cache.action in a module

[Joe Barnhart]

Oops, I meant of course;

current.cache.action

instead of

current.cache




On Wednesday, August 1, 2018 at 12:48:00 AM UTC-7, Joe Barnhart wrote:
>
> You're a fountain of ideas!  I missed that one in the book.
>
> I wonder if this would work.  Off to go try it...
>
> def lazy_cache_action(self, time_expire=DEFAULT_TIME_EXPIRE, 
> cache_model=None,
>prefix=None, session=False, vars=True, lang=True,
>user_agent=False, public=True, valid_statuses=None,
>quick=None):
> def decorator(f, time_expire, cache_model, prefix, session, vars, lang,
>user_agent, public, valid_statuses, quick):
> def g(*c, **d):
> from gluon import current
> return current.cache(f, time_expire, cache_model, prefix, 
> session, vars,
> lang, user_agent, public, valid_statuses, quick)(f)(*c, 
> **d)
> g.__name__ = f.__name__
> return g
> return decorator
>
>
>
> On Tuesday, July 31, 2018 at 11:25:10 AM UTC-7, Anthony wrote:
>>
>> On Tuesday, July 31, 2018 at 1:57:46 AM UTC-4, Joe Barnhart wrote:
>>>
>>> I was wondering about this.  I tried to search the group but didn't find 
>>> anything relevant.  Took a look at the source code and it seemed like I 
>>> could use in a module which is called to produce a string of Javascript on 
>>> demand of an Ajax routine.
>>>
>>> Beforehand, I save the global "cache" var in my "current" object.  Then 
>>> I rename my method "content" to "__content__", and last I do this:
>>>
>>> def content(self):
>>> c = current.cache
>>> return c.action(cache_model=c.disk, 
>>> quick="VP")(self.__content__)()
>>>
>>> Seems to work.  Am I asking for trouble?  Is there anything I should 
>>> watch for?
>>>
>>
>> Seems reasonable. You could also create a custom decorator, similar to 
>> lazy_cache 
>> <https://github.com/web2py/web2py/blob/master/gluon/cache.py#L728-L746> 
>> (see the end of this section: 
>> http://web2py.com/books/default/chapter/29/04/the-core#Warning--Do-not-use-the-current-object-in-global-scope-in-a-module
>> ).
>>
>> Anthony
>>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Using cache.action in a module

[Joe Barnhart]

You're a fountain of ideas!  I missed that one in the book.

I wonder if this would work.  Off to go try it...

def lazy_cache_action(self, time_expire=DEFAULT_TIME_EXPIRE, 
cache_model=None,
   prefix=None, session=False, vars=True, lang=True,
   user_agent=False, public=True, valid_statuses=None,
   quick=None):
def decorator(f, time_expire, cache_model, prefix, session, vars, lang,
   user_agent, public, valid_statuses, quick):
def g(*c, **d):
from gluon import current
return current.cache(f, time_expire, cache_model, prefix, 
session, vars,
lang, user_agent, public, valid_statuses, quick)(f)(*c, **d)
g.__name__ = f.__name__
return g
return decorator



On Tuesday, July 31, 2018 at 11:25:10 AM UTC-7, Anthony wrote:
>
> On Tuesday, July 31, 2018 at 1:57:46 AM UTC-4, Joe Barnhart wrote:
>>
>> I was wondering about this.  I tried to search the group but didn't find 
>> anything relevant.  Took a look at the source code and it seemed like I 
>> could use in a module which is called to produce a string of Javascript on 
>> demand of an Ajax routine.
>>
>> Beforehand, I save the global "cache" var in my "current" object.  Then I 
>> rename my method "content" to "__content__", and last I do this:
>>
>> def content(self):
>> c = current.cache
>> return c.action(cache_model=c.disk, 
>> quick="VP")(self.__content__)()
>>
>> Seems to work.  Am I asking for trouble?  Is there anything I should 
>> watch for?
>>
>
> Seems reasonable. You could also create a custom decorator, similar to 
> lazy_cache 
> <https://github.com/web2py/web2py/blob/master/gluon/cache.py#L728-L746> 
> (see the end of this section: 
> http://web2py.com/books/default/chapter/29/04/the-core#Warning--Do-not-use-the-current-object-in-global-scope-in-a-module
> ).
>
> Anthony
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Using cache.action in a module

[Joe Barnhart]

I was wondering about this.  I tried to search the group but didn't find 
anything relevant.  Took a look at the source code and it seemed like I 
could use in a module which is called to produce a string of Javascript on 
demand of an Ajax routine.

Beforehand, I save the global "cache" var in my "current" object.  Then I 
rename my method "content" to "__content__", and last I do this:

def content(self):
c = current.cache
return c.action(cache_model=c.disk, quick="VP")(self.__content__)()

Seems to work.  Am I asking for trouble?  Is there anything I should watch 
for?

-- Joe

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Why does cache.disk cause recursion error?

[Joe Barnhart]

Yes, of course you're right.  The actual code is generated by this 
monstrosity:

@staticmethod
def make_toolbar(buttons, id, opt):
#tb = CAT()
tb = DIV(_id=id, _class=opt.w_toolbar_cls,_style="padding:0 0 0 5px;")
for title, action, href_fcn, key in buttons:
if isinstance(action, list):
menu = DIV(_class="widget-menu")
items = list()
for title2, val, active in action:
items.append(A(title2,
   _href=href_fcn(val),
   _class=('' if active else 'disabled'),
   data={key:val}))
btn = TAG.button(" %s "%title,SPAN(_class="caret"),
 data={'toggle':'dropdown'},
 _class=opt.w_toolbar_btn_cls+'dropdown-toggle 
button')
menu.append(btn)
menu.append(UL(*items,_class="dropdown-menu dropdown-caret 
dropdown-close"))
tb.append(menu)
else:
tb.append(TAG.button(" %s "%title,_class=opt.w_toolbar_btn_cls))
return tb

Kind of ugly but it has the advantage that it does what I want.  Won't win 
any contest for prettiness, tho.

-- Joe

On Monday, July 30, 2018 at 12:13:42 PM UTC-7, Anthony wrote:
>
> The particular code being pickled, if it matters, looks like this:
>>>
>>> >> data-toggle="dropdown">
>>>   Male 
>>> 
>>>
>>
> Also, just to clarify, the above is not the code being pickled. The above 
> is the HTML generated by executing the view -- it is just a string and 
> would not invoke gluon.html.TAG_pickler. Rather, the controller action must 
> be returning a dictionary containing a FORM object, which contains a 
> TAG['button'] object, which is causing the pickle recursion error. When the 
> TAG['button'] object is rendered, it generates the HTML shown above.
>
> Anthony
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Unicode strings play havoc with JSON

[Joe Barnhart]

I'm sure this is a consequence of Py3 compatibility, but I"ve noticed 
several places where strings are now unicode and it plays havoc with JSON 
translation.  I'm trying to convert the output of database queries into 
JSON and the resulting strings have the shape of:

u'this is a unicode string'

The default json encoder keeps the "u" making the string unreadable to the 
web page.

I can smash everything with str() before encoding to get rid of the unicode 
flag but it's tedious.

-- Joe

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Why does cache.disk cause recursion error?

[Joe Barnhart]

I'm using cache.action with cache_model=cache.disk to play with 
optimization of my site.  To be sure, this page has a lot going on -- 
complicated menus using "button" objects among the issues.  But I don't 
have any circular references, at least as far as I can tell.  Still, I get 
the error:

Traceback (most recent call last):
  File "/Users/jbarnhart/Work/w2p_env/ss/gluon/restricted.py", line 219, in 
restricted
exec(ccode, environment)
  File 
"/Users/jbarnhart/Work/w2p_env/ss/applications/swim_smarter/controllers/results.py",
 
line 29, in 
  File "/Users/jbarnhart/Work/w2p_env/ss/gluon/globals.py", line 419, in 

self._caller = lambda f: f()
  File "/Users/jbarnhart/Work/w2p_env/ss/gluon/cache.py", line 632, in 
wrapped_f
rtn = cache_model(cache_key, lambda: func(), time_expire=time_expire)
  File "/Users/jbarnhart/Work/w2p_env/ss/gluon/cache.py", line 467, in 
__call__
self.storage[key] = (now, value)
  File "/Users/jbarnhart/Work/w2p_env/ss/gluon/cache.py", line 339, in 
__setitem__
pickle.dump(value, val_file, pickle.HIGHEST_PROTOCOL)
  File "/Users/jbarnhart/Work/w2p_env/ss/gluon/html.py", line 1256, in 
TAG_pickler
marshal_dump = pickle.dumps(d, pickle.HIGHEST_PROTOCOL)
  File "/Users/jbarnhart/Work/w2p_env/ss/gluon/html.py", line 1256, in 
TAG_pickler
marshal_dump = pickle.dumps(d, pickle.HIGHEST_PROTOCOL)
  
  ... some duplication omitted ...
  
  File "/Users/jbarnhart/Work/w2p_env/ss/gluon/html.py", line 1256, in 
TAG_pickler
marshal_dump = pickle.dumps(d, pickle.HIGHEST_PROTOCOL)
  File "/Users/jbarnhart/Work/w2p_env/ss/gluon/html.py", line 1256, in 
TAG_pickler
marshal_dump = pickle.dumps(d, pickle.HIGHEST_PROTOCOL)
  File 
"/Applications/WingIDE.app/Contents/Resources/patches/6.0.12/04-fix-logging-exc/bin/dbg/src/debug/tserver/dbgutils.py",
 
line 3409, in wrapper
  File 
"/Applications/WingIDE.app/Contents/Resources/patches/6.0.12/04-fix-logging-exc/bin/dbg/src/debug/tserver/dbgutils.py",
 
line 2955, in Log
RuntimeError: maximum recursion depth exceeded

The particular code being pickled, if it matters, looks like this:


  Male 


Followed by:


  100M Back 


Over and over.

The full HTML sequence is this.  Note the pair of problem buttons are not 
adjacent.  But I've never studied the pickler and don't know how it works, 
so maybe it's not an issue.


 
  2018 Phoenix Summer Invitation
  
   

 Male 


 Female
 Male
 Mixed

   
   

 100M Back 


 50M 
Free
 100M 
Free
 200M 
Free
 400M 
Free
 800M 
Free
 1500M 
Free
 50M 
Back
 100M Back
 200M 
Back
 50M 
Breast
 100M 
Breast
 200M 
Breast
 50M 
Fly
 100M 
Fly
 200M 
Fly
 200M 
Medley
 400M 
Medley

   
   

 11-12 


 10-Un
 11-12
 13-14
 15-Ov
 13-Ov
 Open

   
   

 Finals 


 Finals
 Prelims
 Swim-offs

   
  
 
 
  
   
   
  
 


-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Postgres migration errors

[Joe Barnhart]

I’d get rid of the idiom:

db.colname==“%s”%pythonvar

Just replace it with:

db.colname==pythonvar

— Joe

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Using Scheduler without "dict" to define functions

[Joe Barnhart]

Thank you for pulling back some of the mystery surrounding the scheduler 
for me.  The module import idea is fabulous and will completely solve my 
problem!

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Using Scheduler without "dict" to define functions

[Joe Barnhart]

So I'm expanding my use of the scheduler -- one of the best features of 
web2py -- and I'm running into a "scaling" problem.

I'm going to have perhaps a hundred little functions that the scheduler 
inserts into the queue.  Now, if I define my little functions in the same 
Model page as the scheduler, everything works as expected.  Each of my 
functions is like a stub -- it generally provides a call point for the 
scheduler but then accesses code in a module somewhere.  Although each 
function is kept as small as possible, I'm still loading a large number of 
little functions with every web page access, activity I would like to avoid.

There is a dictionary I can pass to the scheduler which names each of the 
functions to be used as entries, but then I have to create a dictionary 
with a large number of entries, each time a web page is called.  Again, 
activity I'd rather avoid in the 99.% of web page accesses that don't 
need or use the scheduler.  Plus I'm not sure I can create the dictionary 
if its functions are scattered across controllers.  That would seem 
unlikely to compile since only one controller is visible at any one pass 
through the model.

What is this dictionary for, and how can I avoid needing it at all?  There 
are only a few comments in the book regarding this topic.  Mostly a warning 
that the "workers must be restarted" if the dictionary changes.  (Has 
anyone noticed that stopping or restarting the workers is never described? 
 I presume this is a console process which makes it outsides the bounds of 
what I'm looking for.)

No matter what I try, unless the function is defined in the same model page 
as the scheduler, all I get from the scheduler is "NameError: name 'x' 
not found in scheduler's environment" even though I specifically test to 
make sure it is visible when invoking web2py -M -S app/controller/x

How do I scale this up without needing 100 stub functions in a model file 
along with the scheduler?

-- Joe

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [web2py] Re: Scheduler and far-away tasks

[Joe Barnhart]

Hi Richard --

I heartily recommend you take another look at the scheduler old friend. 
 Once you wrap your mind around it, the things it can do will simply amaze 
you.  There are a few key principles you need to grok first, and I still 
struggle a bit here and there.

As far as knowing the result of a scheduled task, the scheduler shines 
there.  It persists in the database every failed run of the scheduler, so 
you can easily find the details in the tables if your critical task ran or 
failed.  There is also a timing granularity to mind -- the scheduler does 
everything on a fixed heartbeat which defaults to around 5 seconds.  In my 
case, it doesn't matter if a swim meet becomes visible 5 seconds late or 
not.  But in some cases, pinpoint timing is called for.  In those cases, 
the scheduler heartbeat can be tightened.

I can't say enough about the scheduler.  Yes, it adds a layer of 
complication to the site, but it also has the ability to simplify things 
mightily.  Like this swim meet catalog I described to Anthony.  Yes, I 
could do it other ways but OMG this makes it so simple and the rules can be 
arbitrarily complex.  I can imagine a team saying they want their best 
swimmers to get access before their worst -- the idea is easily extended to 
any level of granularity down to individual swimmers if necessary.

-- Joe

On Monday, June 11, 2018 at 11:30:30 AM UTC-7, Richard wrote:
>
> Hello Joe,
>
> I am not using the scheduler primarilly because I never had time to 
> implemented it. But I miss so much all the things you mention... Let says 
> just configure a task to lock a user account on a specific date in the 
> future when you have employee termination in advance and you have no other 
> proper mean to recall that and don't want to pollute your calendar with a 
> bunch of todo thing...
>
> The only flaw I see is relying on the tool and having some task not 
> performed as expect for some reason and not get properly inform that a 
> given task didn't succeed as expect... Depending of the criticallity of the 
> task this may have bad consequences...
>
> Richard
>
> On Mon, Jun 11, 2018 at 11:16 AM, Anthony 
> > wrote:
>
>> I suppose that should be fine. Tasks are stored in a database, so should 
>> be as durable as any other data you are storing.
>>
>> Alternatively, you might consider storing the reveal date with the item 
>> and running a regular task that simply selects all items with reveal dates 
>> equal to or earlier than the current day and updates those items to become 
>> visible. Or even just add a common filter that automatically filters out 
>> any records with a future reveal date.
>>
>> Anthony
>>
>>
>> On Sunday, June 10, 2018 at 7:20:19 PM UTC-4, Joe Barnhart wrote:
>>>
>>> Are there any reasons I should not schedule tasks days, weeks, or months 
>>> in advance in the web2py scheduler?  It seems such a fantastic, 
>>> general-purpose tool that I'm using it for more and more things in the 
>>> site.  
>>>
>>> Currently I'm considering using it to "reveal" embargo'ed items that 
>>> should not be seen until a certain date and time.  The reveal would involve 
>>> putting a task in the scheduler for the far-away date and having it create 
>>> a permission on a table at the moment of truth.
>>>
>>> Is there some reason this might be considered bad design, or poor use of 
>>> the scheduler?  Reliability is of high importance in my site, so design 
>>> that decreases reliability is to be avoided.
>>>
>>> -- Joe
>>>
>>> -- 
>> Resources:
>> - http://web2py.com
>> - http://web2py.com/book (Documentation)
>> - http://github.com/web2py/web2py (Source code)
>> - https://code.google.com/p/web2py/issues/list (Report Issues)
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "web2py-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to web2py+un...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Scheduler and far-away tasks

[Joe Barnhart]

Hi Anthony --

I thought the DB would provide enough persistence.  My task model is, as 
you might expect, a little more complicated than the simple scenario I 
described.  In my case the catalog of swim meets are only viewable by 
specific subsets of people at given times. My product is a reservation in a 
swim meet, and only certain teams are able to see the meet.  On top of 
that, the host of the meet wants to reveal the meet to certain teams in a 
certain order, to allow a favored team to get as many swimmers in the meet 
as they want before a less-favored team has a chance to register.

So the catalog visibility is controlled by the web2py permissions 
structure, where swimmers belong to a team group and those groups get a 
permission link to "read" a line in the catalog table.  Controlling when 
these permission links are created and destroyed would be under the 
scheduler.  Believe it or not, on the old site this is all manual 
operation.  There are swim meet hosts who stay up until midnight to control 
their meet's visibility now.

-- Joe

On Monday, June 11, 2018 at 8:16:04 AM UTC-7, Anthony wrote:
>
> I suppose that should be fine. Tasks are stored in a database, so should 
> be as durable as any other data you are storing.
>
> Alternatively, you might consider storing the reveal date with the item 
> and running a regular task that simply selects all items with reveal dates 
> equal to or earlier than the current day and updates those items to become 
> visible. Or even just add a common filter that automatically filters out 
> any records with a future reveal date.
>
> Anthony
>
> On Sunday, June 10, 2018 at 7:20:19 PM UTC-4, Joe Barnhart wrote:
>>
>> Are there any reasons I should not schedule tasks days, weeks, or months 
>> in advance in the web2py scheduler?  It seems such a fantastic, 
>> general-purpose tool that I'm using it for more and more things in the 
>> site.  
>>
>> Currently I'm considering using it to "reveal" embargo'ed items that 
>> should not be seen until a certain date and time.  The reveal would involve 
>> putting a task in the scheduler for the far-away date and having it create 
>> a permission on a table at the moment of truth.
>>
>> Is there some reason this might be considered bad design, or poor use of 
>> the scheduler?  Reliability is of high importance in my site, so design 
>> that decreases reliability is to be avoided.
>>
>> -- Joe
>>
>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Scheduler and far-away tasks

[Joe Barnhart]

Are there any reasons I should not schedule tasks days, weeks, or months in 
advance in the web2py scheduler?  It seems such a fantastic, 
general-purpose tool that I'm using it for more and more things in the 
site.  

Currently I'm considering using it to "reveal" embargo'ed items that should 
not be seen until a certain date and time.  The reveal would involve 
putting a task in the scheduler for the far-away date and having it create 
a permission on a table at the moment of truth.

Is there some reason this might be considered bad design, or poor use of 
the scheduler?  Reliability is of high importance in my site, so design 
that decreases reliability is to be avoided.

-- Joe

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] AppConfig in JSON returns unicode

[Joe Barnhart]

This is sort of a bug, sort of a feature.  I just converted to the JSON 
version of AppConfig (in the gluon/contrib) and discovered the strings it 
produces are all unicode -- even in Python 2.x environments.

I'm not sure of ALL of the ramifications, but I discovered it causes crypt 
fields to fail when a key is provided.  I'm using a known key because I 
need to keep compatibility with past users who have accounts, but as soon 
as I tried to use the crypt field to enter the first logon when 
initializing the new website, it failed. It tested the "digest_alg" to see 
if it was a "str" and failed because it was "unicode".

Anyway, the workaround is to use "str()" on any fields that are critical. 
 Longer term, it may be useful for the maintainer of AppConfig to return 
strings instead of unicode from JSON format files.

-- Joe

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Customizing Fields the lazy way

[Joe Barnhart]

Actually, I borrowed Anthony's excellent idea and made a "factory" class 
for my field definitions.

First I defined a factory class:

class Field_Factory(object):
from gluon import Field
@staticmethod
def new(**kwargs):
default = dict(**kwargs)
def inner(name, **kwargs):
args = dict(default)
args.update(**kwargs)
if args.pop('hidden',False):
args.update(dict(readable=False,writable=False))
req = args.get('requires', None)
if req and (args.get('represent',None)=='formatter'):
args['represent'] = req.formatter
if args.pop('optional',False) and req:
args['requires'] = IS_EMPTY_OR(req)
rend = args.pop('render',None)
rtn = Field(name, **args)
if rend:
rtn.render = rend
return rtn
return inner

factory = Field_Factory()

Then I used it to create a bunch of field generators:

phone_field = factory.new(
type='string', length=20,
requires=IS_PHONE(), optional=True,
widget=lambda fld,val: SQLFORM.widgets.string.widget(
fld, val, _type='tel', _class='form-control')
)

email_field = factory.new(
type='string', length=50,
requires=IS_EMAIL(), optional=True,
widget=lambda fld,val: SQLFORM.widgets.string.widget(
fld, val, _type='email', _class='form-control')
)

date_field = factory.new(
type='date',
requires=IS_DATE(format='%m-%d-%Y'), optional=True,
represent=lambda v,r: v.strftime('%m-%d-%Y'),
widget=lambda fld,val:SQLFORM.widgets.date.widget(
fld, val, _class="date form-control")
)

datetime_field = factory.new(
type='datetime',
requires=IS_DATETIME(format='%m-%d-%Y %I:%M:%S %p'), optional=True,
represent=lambda v,r: v.strftime('%m-%d-%Y %I:%M:%S %p'),
widget=lambda fld,val: SQLFORM.widgets.datetime.widget(
fld, val, _class="datetime form-control")
)

zipcode_field = factory.new(
type='string', length=10,
requires=IS_ZIPCODE(),
widget=lambda fld,val: SQLFORM.widgets.string.widget(
fld, val, _type="zip", _class='zipcode form-control')
)

Finally, when I use the field generators in any table definitions, I can 
further customize them and the changes are passed through.  

define_table('joes_table',
. . .
date_field("birth", label=T("Birth date")),
. . .


It all works really well and gives me the single point of control I want. 
 It requires no changes in web2py and works with current, past, and future 
versions.  And the lazy programmer in me marvels at all the code I don't 
have to type.

-- Joe

On Thursday, March 23, 2017 at 12:54:58 PM UTC-7, Joe Barnhart wrote:
>
> Here is a bit of syntactic sugar I use for creating fields with less 
> typing.  And better consistency, of course -- there always has to be a good 
> reason for my lazy keyboard-saving shortcuts!
>
> I have a number of fields which are for specific data types, such as phone 
> numbers.  I gather the "common" parts of the Field definition into a 
> dictionary:
>
> phone_fld = {'requires':IS_EMPTY_OR(IS_PHONE()),
>  'widget':lambda fld,val: SQLFORM.widgets.string.widget(fld,
> val,_type='tel',_class='form-control')}
>
> When defining the field in a table, I invoke the dictionary at the end of 
> the Field definition:
>
> Field("homephone","string",length=20,label=T("Home phone"),**phone_fld),
> Field("workphone","string",length=20,label=T("Work phone"),**phone_fld),
> Field("cellphone","string",length=20,label=T("Cell phone"),**phone_fld),
>
> Now the field is created exactly as I want.  I keep a list of these 
> "helpers" in a module which I can call out when creating tables.  It really 
> saves the typing and allow me to control how my fields are rendered on one 
> place instead of scattered among all the tables.
>
> -- Joe
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] MySql deprecation of NO_AUTO_CREATE_USER

[Joe Barnhart]

Anybody else seeing this?

.../gluon/contrib/pymysql/cursors.py:166: Warning: (3090, u"Changing sql 
mode 'NO_AUTO_CREATE_USER' is deprecated. It will be removed in a future 
release.")

This Is showing up on the console while running version 
2.16.1-stable+timestamp.2017.11.13.23.50.07

Just curious...

-- Joe

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Decrypt password from auth_user

[Joe Barnhart]

Actually you don’t want to show the correct number of characters. That gives an 
attacker a lot of information. 

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: How to use HTML escapes in Field titles

[Joe Barnhart]

Thank you for your help, Anthony.

-- Joe

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: How to use HTML escapes in Field titles

[Joe Barnhart]

Actually I've thought a little more about it and I think this construction 
is better.

elapsed_time_field("lcm_gt",label=XML(T("LCM %s",("",,
elapsed_time_field("lcm_le",label=XML(T("LCM %s",("",,
elapsed_time_field("scm_gt",label=XML(T("SCM %s",("",,
elapsed_time_field("scm_le",label=XML(T("SCM %s",("",,
elapsed_time_field("scy_gt",label=XML(T("SCY %s",("",,
elapsed_time_field("scy_le",label=XML(T("SCY %s",("",,

-- Joe


On Wednesday, July 26, 2017 at 5:28:01 PM UTC-7, Joe Barnhart wrote:
>
> Hi Anthony --
>
> The problem is that I don't want "LT" but rather "LE", i.e. "less than or 
> equal to."
>
> As far as my table comment, I meant that when I used the SQL table and its 
> Fields to create an SQLTABLE, the labels "just worked" and produced a 
> column header with the desired symbol instead of printing "" in the 
> column heading.  So SQLTABLE behavior differed from SQLFORM in this manner.
>
> I have found a workaround, finally, which lets me have symbols in both 
> forms and tables:
>
> elapsed_time_field("lcm_gt",label=CAT(T("LCM"),XML(" "))),
> elapsed_time_field("lcm_le",label=CAT(T("LCM"),XML(" "))),
> elapsed_time_field("scm_gt",label=CAT(T("SCM"),XML(" "))),
> elapsed_time_field("scm_le",label=CAT(T("SCM"),XML(" "))),
> elapsed_time_field("scy_gt",label=CAT(T("SCY"),XML(" "))),
> elapsed_time_field("scy_le",label=CAT(T("SCY"),XML(" "))),
>
> I first tried the obvious, letting T() handle the substitution but again, 
> that doesn't work.  The  character escapes get printed literally in the 
> form label instead of creating the symbol I intended.  So I got around the 
> problem with CAT().  Still, an awful amount of work and hassle to 
> reverse-engineer and make a work around for something that just should have 
> worked.
>
> I'm not sure why  is "sanitized" to begin with.  It seems like an 
> extreme form of sanitizing, to eliminate any and all special characters 
> from form labels.
>
> -- Joe
>
> On Wednesday, July 26, 2017 at 5:16:55 PM UTC-7, Anthony wrote:
>>
>> On Wednesday, July 26, 2017 at 7:43:14 PM UTC-4, Joe Barnhart wrote:
>>>
>>> It's even worse than I imagined.
>>>
>>> Leaving off the T() operation, I find that my field labeled "LCM >" is 
>>> actually sanitized at some point into:
>>>
>>> >> id="event_join_lcm_gt__label">LCM : 
>>>
>>> Yes, something in the process has *recognized* the character ">" and 
>>> changed it to ""  But the field "LCM " was sanitized into:
>>>
>>> >> id="event_join_lcm_le__label">LCM le;: 
>>>
>>> In this case, not only was the  *not recognized* by the sanitizer, 
>>> it actually DE-SANITZED it by removing the ampersand and sanitizing it 
>>> separately.
>>>
>>
>> Both of the above are encoded as expected -- the ">" character is 
>> converted to "", and the "&" character in "" is converted to 
>> "". This is consistent and expected behavior. If you want to end up 
>> with "", then why not just start with "<"?
>>
>> What is your ultimate goal? Do you not want the final HTML to include the 
>> "" and "" HTML entities so they display as ">" and "<" on the page?
>>
>> Also, what do you mean by, "If I use the Field titles directly in a table 
>> on my own, they work as expected?" In each case, what do you want to 
>> display on the page, and what are you expecting in the raw HTML?
>>
>> Anthony
>>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: How to use HTML escapes in Field titles

[Joe Barnhart]

Hi Anthony --

The problem is that I don't want "LT" but rather "LE", i.e. "less than or 
equal to."

As far as my table comment, I meant that when I used the SQL table and its 
Fields to create an SQLTABLE, the labels "just worked" and produced a 
column header with the desired symbol instead of printing "" in the 
column heading.  So SQLTABLE behavior differed from SQLFORM in this manner.

I have found a workaround, finally, which lets me have symbols in both 
forms and tables:

elapsed_time_field("lcm_gt",label=CAT(T("LCM"),XML(" "))),
elapsed_time_field("lcm_le",label=CAT(T("LCM"),XML(" "))),
elapsed_time_field("scm_gt",label=CAT(T("SCM"),XML(" "))),
elapsed_time_field("scm_le",label=CAT(T("SCM"),XML(" "))),
elapsed_time_field("scy_gt",label=CAT(T("SCY"),XML(" "))),
elapsed_time_field("scy_le",label=CAT(T("SCY"),XML(" "))),

I first tried the obvious, letting T() handle the substitution but again, 
that doesn't work.  The  character escapes get printed literally in the 
form label instead of creating the symbol I intended.  So I got around the 
problem with CAT().  Still, an awful amount of work and hassle to 
reverse-engineer and make a work around for something that just should have 
worked.

I'm not sure why  is "sanitized" to begin with.  It seems like an 
extreme form of sanitizing, to eliminate any and all special characters 
from form labels.

-- Joe

On Wednesday, July 26, 2017 at 5:16:55 PM UTC-7, Anthony wrote:
>
> On Wednesday, July 26, 2017 at 7:43:14 PM UTC-4, Joe Barnhart wrote:
>>
>> It's even worse than I imagined.
>>
>> Leaving off the T() operation, I find that my field labeled "LCM >" is 
>> actually sanitized at some point into:
>>
>> > id="event_join_lcm_gt__label">LCM : 
>>
>> Yes, something in the process has *recognized* the character ">" and 
>> changed it to ""  But the field "LCM " was sanitized into:
>>
>> > id="event_join_lcm_le__label">LCM le;: 
>>
>> In this case, not only was the  *not recognized* by the sanitizer, 
>> it actually DE-SANITZED it by removing the ampersand and sanitizing it 
>> separately.
>>
>
> Both of the above are encoded as expected -- the ">" character is 
> converted to "", and the "&" character in "" is converted to 
> "". This is consistent and expected behavior. If you want to end up 
> with "", then why not just start with "<"?
>
> What is your ultimate goal? Do you not want the final HTML to include the 
> "" and "" HTML entities so they display as ">" and "<" on the page?
>
> Also, what do you mean by, "If I use the Field titles directly in a table 
> on my own, they work as expected?" In each case, what do you want to 
> display on the page, and what are you expecting in the raw HTML?
>
> Anthony
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: How to use HTML escapes in Field titles

[Joe Barnhart]

It's even worse than I imagined.

Leaving off the T() operation, I find that my field labeled "LCM >" is 
actually sanitized at some point into:

LCM : 

Yes, something in the process has *recognized* the character ">" and 
changed it to ""  But the field "LCM " was sanitized into:

LCM le;: 

In this case, not only was the  *not recognized* by the sanitizer, it 
actually DE-SANITZED it by removing the ampersand and sanitizing it 
separately.

This is the problem with automatic "stuff" -- such as hidden magic 
sanitization.  When it goes wrong, it goes very wrong.  And you need to 
spend hours with the source trying to figure out where it went wrong and if 
there's an easy fix without modifying the distro.

-- Joe

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: How to use HTML escapes in Field titles

[Joe Barnhart]

Ugh.  Replace the word "title" with "label" everywhere.  Global search and 
replace.

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] How to use HTML escapes in Field titles

[Joe Barnhart]

I have field titles which need to indicate "less than or equal" or "greater 
than" symbols.  HTML provides such escapes with the character sequences 
 and  respectively.  But when I use these characters in the "title" 
of a Field, the displayed SQLFORM changes these back into "" for 
example instead of displaying the "less than or equal" symbol.  If I use 
the Field titles directly in a table on my own, they work as expected.

Is there a reason why the "title" of a Field is afforded this extra 
protection from me, the hapless programmer?

Here's a table definition, cuz Anthony always asks for one.  (And he's 
right to.)

db.define_table("event",
Field("id_meet", "reference meet", label=T("Meet")),
Field("id_session", "reference session", label=T("Session")),
Field("id_event", "reference event_index", label=T("Event")),
#Field("day", "integer", notnull=True, label=T("Day")),
Field("ord", "integer", notnull=True, label=T("Order")),
Field("swim_num", "string", notnull=True, label=T("Number")),
elapsed_time_field("lcm_gt",label=T("LCM >")),
elapsed_time_field("lcm_le",label=T("LCM ")),
elapsed_time_field("scm_gt",label=T("SCM >")),
elapsed_time_field("scm_le",label=T("SCM ")),
elapsed_time_field("scy_gt",label=T("SCY >")),
elapsed_time_field("scy_le",label=T("SCY ")),
Field("nt_ok", "boolean", label=T("Allow NT")),
format=event_list,
migrate=current.settings.migrate)


You will notice I'm using a Field factory "elapsed_time_field" to create 
the Field objects.  It just defines each as an integer field and presets 
the validator and formatter.  Before you ask Yes, I tried it with and 
without the T() operator.  There was no change on the SQLFORM and the table 
still works as expected.

-- Joe

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: ValueError: Field is already bound to a table <-- need a fix for this

[Joe Barnhart]

Yes.  I just tested with 2.15.2 and all works perfectly as expected.

-- Joe


On Thursday, July 20, 2017 at 2:58:08 PM UTC-7, Massimo Di Pierro wrote:
>
> Can you confirm all is good now with 2.15.2?
> Thanks to you and Leonel for looking into this so quickly.
>
> On Sunday, 16 July 2017 21:34:40 UTC-5, Joe Barnhart wrote:
>>
>> Yes that fix looks exactly on point.
>>
>> -- Joe
>>
>> On Sunday, July 16, 2017 at 3:28:22 PM UTC-7, Leonel Câmara wrote:
>>>
>>> Joe the fix on the DAL wasn't for your problem, Your problem was fixed 
>>> in web2py here:
>>>
>>> https://github.com/web2py/web2py/pull/1686/commits/ce0b2557470ecb7c92719645293cb00785fe48bc
>>>
>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Sacramento-area web2py programmers?

[Joe Barnhart]

Government interoperability?  Isn't that an oxymoron?  You mean, like use 
the same frequency radios and stuff?

Sure, you can have me for dinner.  Maybe we can scare up some others.  I 
keep looking under rocks for more Python programmers up here.  You'd think 
with all this 100+ weather that indoor programming jobs would be highly 
preferred...

-- Joe


On Monday, July 17, 2017 at 8:45:36 AM UTC-7, Alex Glaros wrote:
>
> who else lives around Sacramento besides Joe?
>
> You're invited for dinner and talk some weekend at my house in Winters, CA 
> (near Davis)
>
> contact info: http://gov-ideas.com/contact.htm
>
> thanks,
>
> Alex Glaros
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: ValueError: Field is already bound to a table <-- need a fix for this

[Joe Barnhart]

Yes that fix looks exactly on point.

-- Joe

On Sunday, July 16, 2017 at 3:28:22 PM UTC-7, Leonel Câmara wrote:
>
> Joe the fix on the DAL wasn't for your problem, Your problem was fixed in 
> web2py here:
>
> https://github.com/web2py/web2py/pull/1686/commits/ce0b2557470ecb7c92719645293cb00785fe48bc
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: ValueError: Field is already bound to a table <-- need a fix for this

[Joe Barnhart]

You are correct.  By using the SQLFORM.factory function the table was, 
indeed, different.  I blame the lapse on the inability of neurons to 
function in the 106 degree heat we're having in the Sacramento area today.

-- Joe

On Sunday, July 16, 2017 at 2:53:10 PM UTC-7, Anthony wrote:
>
> You mentioned the field was being re-bound to the same table, which does 
> not appear to be the case below, as SQLFORM. factory creates its own dummy 
> table. Was this causing another problem in some other context? 
>
> Anthony 
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: ValueError: Field is already bound to a table <-- need a fix for this

[Joe Barnhart]

I think I tried your fix but it didn't work. None of my fields were "common".  
Massimo had pointed out the fix on Github. 

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: ValueError: Field is already bound to a table <-- need a fix for this

[Joe Barnhart]

Found the issue and a workaround see below.

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: ValueError: Field is already bound to a table <-- need a fix for this

[Joe Barnhart]

Yes... Massimo helped me come up with a workaround for my case.

I am using SQLFORM.factory to create a form to edit two tables joined 
together.  One table is used to index entries in a bunch of other tables by 
a 6-tuple.  Since these are all swimming related, the tuple is (sex, 
upper_age, lower_age, stroke, distance, course).  This is turned into an 
index and used in a bunch of other tables which all need this same 
information.

Here is the outline of the problematic method, with the fix in RED:

def edit_event():
. . . 
(dbe, dbei) = (db.event, db.event_index)
. . . 
fields = [dbei[f].clone() for f in dbei.fields if f not in ['id','code'
]]
fields += [dbe[f].clone() for f in dbe.fields if f not in ['id',
'id_meet','id_session','id_event','ord']]
rec = db(dbe.id==event_id).select(dbe.ALL, dbei.ALL, cacheable=True, 
limitby=(0,1),
  join=dbei.on(dbei.id==dbe.id_event)).
first()
d = dict(rec.event_index)
d.update(rec.event)
. . . 
form = modal_form(SQLFORM.factory, *fields, record=d, table_name=
"event_join",showid=False,
  title=H4("Editing Event Settings", _class="center”))
. . . 

Cloning the fields before using them in SQLFORM.factory avoids the error.

-- Joe



On Sunday, July 16, 2017 at 10:15:18 AM UTC-7, Leonel Câmara wrote:
>
> Joe can you show us some code? I've already fixed two bugs related to 
> this, one on the DAL and another in web2py, I want to make sure yours isn't 
> a new one.
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: ValueError: Field is already bound to a table <-- need a fix for this

[Joe Barnhart]

Hmm... That's gonna be a problem.

This is occurring in an SQLFORM... which is in a modal pop-up box... over a 
tabbed structure with about 5 more SQLFORMS...  plus 5 or 6 jQuery 
DataTables... all powered by the same basic set of 4 or 5 tables.  It will 
take a solid day to boil it down to a simpler example which could be posted 
successfully.

I'm hoping to get a sense of the philosophy of the new code, so I can debug 
it myself and figure out if it could be fixed for this case.  It seems that 
each Field is "bound" to a Table when a query is performed.  I presume this 
binding has always occurred, yet now it is causing this obscure error.  I'm 
not sure why it would ever be an error to bind the same field to the same 
table, and it seems that this "case" should be the most frequently 
encountered.  Now I'll admit it DOES bother me A LOT to place a database 
instance INSIDE the Field and the Table objects.  I'd think that would be a 
very poor idea if you wanted your design to be a clean and free from hidden 
side effects.

My biggest fear is that DAL has taken a direction which will make it 
incompatible with my use, and that I'll never be able to use 2.15.1 or 
later versions.  I'm weeks away from deploying the biggest site I've ever 
done. As of this second, my best short-term option is to revert to the 
previous version of web2py and finish.  But that has severe long-term 
repercussions if I can't update my web platform in the future.

-- Joe


On Saturday, July 15, 2017 at 3:53:40 AM UTC-7, Anthony wrote:
>
> As usual, I think we need to see some code and a traceback.

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Web2py like clone in Java?

[Joe Barnhart]

As long as we're adding interesting but obscure and little-used web 
platforms, I'd like to submit  Seaside, the Smalltalk web platform.

Just... well... because Smalltalk!

-- Joe

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Grails for web2py - anyone interested ?

[Joe Barnhart]

But Grails borrowed heavily from Rails.  And you, Massimo, mentioned that 
Rails also provided some "inspiration" for web2py  

So by the commutative property of web platforms, web2py and Grails share 
some family similarities. 

-- Joe

On Thursday, July 13, 2017 at 10:05:54 AM UTC-7, Massimo Di Pierro wrote:
>
> Actually no. I never looked into Grails.
>
> On Thursday, 13 July 2017 02:29:15 UTC-5, pbreit wrote:
>>
>> I'm curious what ever happened to this effort?
>>
>> Personally I much prefer the CLI-less approach of web2py. I would much 
>> rather just create a file here and there or add a function in a controller 
>> file. You get your default view automatically and then can add a view file 
>> when appropriate.
>>
>> But I was just looking at Grails and from a very first glance saw quite 
>> some similarities to Web2py.
>>
>> Massimo, did Grails provide any influence in web2py?
>>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] ValueError: Field is already bound to a table <-- need a fix for this

[Joe Barnhart]

I just switched to Version 2.15.1-stable+timestamp.2017.07.10.16.17.24 and 
I'm getting a new and very unwelcome error.

ValueError:  Field  is already bound to a table

Yeah.  I knew that.  In fact, the field is already bound to exactly the 
SAME table in other queries.  So what?  Why is that now an error?  I 
double-checked and sure enough the table its bound to is exactly the table 
that it would be bound to if the "bind" function completes.  (Although the 
table._db of the already bound table has a "real" URI and the not-yet-bound 
table has a table._db lacking a real URI.)

Why have I never seen this error before?  I've been developing this app on 
and off for a couple of years and this is the first time I've seen this 
problem.  Is it by chance new with 2.15.1?

What is the purpose of this error?  What binding is occurring?  (I'm 
guessing this is the binding in the driver immediately before a query.) 
 Why can this binding only happen once?  Is it happening because I do the 
heavy lifting in modules instead of controllers?  I often use the same 
tables more than once in a function, so this could be problematic (read: 
fatal) in my adoption of 2.15.1.

-- Joe

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: DAL sqlite vs postgresql: Row Orders are different?

[Joe Barnhart]

What he said.  But in addition, consider the following:

1. The entire purpose of SQL is to treat data as unordered tables and let 
you establish a "relationship" between data in different tables, or within 
a single table.  To add "order by" to every SQL select statement is a 
serious de-optimization and is fighting the way SQL is intended to work. 
 You are supposed to order the data only at the last step, and then only if 
actually necessary.  Un-ordered retrieval is always faster than ordered 
retrieval.

2. The select statement may have many tables involved (i.e. a join).  Which 
table would you "default" the order-by clause to?

-- Joe

On Friday, June 30, 2017 at 3:53:17 PM UTC-7, lyn2py wrote:
>
> I ran my app on *sqlite* and things are working well, so I ported it over 
> to *postgresql*... however I noticed one glaring difference in the 
> results retrieved immediately...
>
> On sqlite, no matter how many times I edit an entry, for example, an entry 
> with id==1, it will always be the first row on the list of rows retrieved 
> from the DB.
>
> On postgresql, after editing an entry, for example the same entry with 
> id==1, that row is no longer on the first row, but instead it falls to the 
> last row retrieved. This is ok if the behavior is consistent with other 
> rows, but I cannot seem to replicate it... different rows just gives 
> different results.
>
> After looking around for answers, I found that this is a characteristic of 
> postgresql:
>
> https://dba.stackexchange.com/questions/48989/postgresql-indeterminate-ordering-of-results
>
> Hence:
> I would like to add a default to all my db().select()'s *orderby* to 
> order by *table.id *. I am wondering what is the correct 
> code for it so that I don't have to go back to edit every single 
> db().select() that I have.
>
> Or do you know of another solution for this?
>
> Thank you!
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: cpython and web2py

[Joe Barnhart]

My experiments are all pretty old.  Too old to be useful.

Currently I have commercial websites using web2py and I don't need to do 
anything remarkable to make it faster.  I just used good design techniques 
and didn't do "premature optimization."  I'm now re-writing a Rails, site, 
a Java site, and an older web2py site into a combined site that will do all 
of the above.  My database is MySql and has about 50 tables, with 20 to 100 
columns in each.  My largest table has about 3M rows in it.  I expect 30M 
when I'm done.

I chose indexes carefully and I used "modules" (i.e. compiled python) for 
most of the model/controller details.  The beta site beats the pants off 
the sites it is replacing.  No pypy needed.  No cython either.

If you're looking at platform "benchmarks" to make a decision of what is 
fast enough, just don't do that.  The benchmarks I looked at turned out to 
be worse than useless in practice.  Most benchmarks pare down the task into 
something small and atomic that can be done by every platform.  Which means 
if you're using the services of a full-service platform like web2py the 
benchmark is already slanted against you.  It will compare the platforms 
assuming you are not using DAL, templating, routing, and all of the other 
feature that make web2py useful.  Basically they just make every app look 
like "hello world" for web servers.

The only way to compare a platform is to use a workload like the one you 
will deploy.  Mine is VERY HEAVY on database usage so the SQL database 
dominates the page load time in most cases.  I also use some heavy 
Javascript libraries (like Datatables.net) which affects load times.  The 
speed problems are never where you think they'll be.  And clever caching 
and other techniques can fix just about anything.

-- Joe


On Sunday, June 25, 2017 at 12:10:56 PM UTC-7, Ron Chatterjee wrote:
>
> Ok. I buy that.
>
> Speed is always a plus. 
>
>  Whats are the calls to run web2py with pypy? Can you share a link or an 
> example? I cant seem to find that in the book.
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: cpython and web2py

[Joe Barnhart]

It's hard to take these questions seriously when you haven't described your 
speed problem.  How is your website performing, and what are the 
bottlenecks you've found?

Yes, many people, including myself, have experimented with pypy and web2py. 
 If you're observant, you'll notice there are many messages on this forum, 
spanning years, which discuss web2py running on pypy.  Like cython, it is 
not a panacea.

Without sounding repetitive, there are many techniques for speeding up 
web2py.  Different techniques are employed for different problems.  There 
is no single "magic bullet" which just speeds up all web2py websites for 
all occasions.

-- Joe

On Sunday, June 18, 2017 at 3:27:19 PM UTC-7, Ron Chatterjee wrote:
>
> Have anyone tried this?  Wondering if this is a way to gain some speed.
>
>
> https://micropyramid.com/blog/running-django-with-pypy-to-boost-performance/
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: cpython and web2py

[Joe Barnhart]

No.

On Sunday, May 28, 2017 at 8:50:15 AM UTC-7, Ron Chatterjee wrote:
>
> I understand that. No one is disputing better coding better performance. 
> But the question is:
>
> Cython wrapper (
>
> Default. Py
>
> )
> Is it possible.  If so how? 
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Error in handling long ints in gluon.serializers json()

[Joe Barnhart]

It's me again...

And, while we're at it, shouldn't the test of "isinstance(o, XmlComponent)" 
actually be "issubclass(o.__class__, XmlComponent)?"  The former fails on 
all of the helpers which are subclasses of DIV because "isinstance" only 
checks the direct parent, it does not check up the "tree".  And all of the 
helpers are subclasses of DIV, which itself is a subclass of XmlComponent.

Just askin'...

On Friday, May 26, 2017 at 12:05:25 PM UTC-7, Joe Barnhart wrote:
>
> I dunno how this affects me and not everyone else who uses this tool.  It 
> looks like the default JSON serializer for web2py has a huge flaw.  But if 
> that were true, it would have been noticed by lots 'o people before.  So it 
> must be impure thinking on my part.
>
> My objects are being serialized with long ints.  The long ints are 
> rendered as Python long ints with an "L" at the end, such as "110L".  This 
> is illegal in Javascript and results in a fatal error.
>
> I examined the code behind the gluon.serializers and found the default 
> serializer used:
>
> def custom_json(o):
> if hasattr(o, 'custom_json') and callable(o.custom_json):
> return o.custom_json()
> if isinstance(o, (datetime.date,
>   datetime.datetime,
>   datetime.time)):
> return o.isoformat()[:19].replace('T', ' ')
> elif isinstance(o, (int, long)):
> return int(o)
> elif isinstance(o, decimal.Decimal):
> return str(o)
> elif isinstance(o, lazyT):
> return str(o)
> elif isinstance(o, XmlComponent):
> return str(o)
> elif isinstance(o, set):
> return list(o)
> elif hasattr(o, 'as_list') and callable(o.as_list):
> return o.as_list()
> elif hasattr(o, 'as_dict') and callable(o.as_dict):
> return o.as_dict()
> else:
> raise TypeError(repr(o) + " is not JSON serializable")
>
> Yep.  There it is.  Give it a long instance of 110 and it returns 110L 
> every time.  Javascript responds with "SyntaxError: No identifiers allowed 
> directly after numeric literal" every time.
>
> Where am I going wrong?
>
> -- Joe
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Error in handling long ints in gluon.serializers json()

[Joe Barnhart]

I dunno how this affects me and not everyone else who uses this tool.  It 
looks like the default JSON serializer for web2py has a huge flaw.  But if 
that were true, it would have been noticed by lots 'o people before.  So it 
must be impure thinking on my part.

My objects are being serialized with long ints.  The long ints are rendered 
as Python long ints with an "L" at the end, such as "110L".  This is 
illegal in Javascript and results in a fatal error.

I examined the code behind the gluon.serializers and found the default 
serializer used:

def custom_json(o):
if hasattr(o, 'custom_json') and callable(o.custom_json):
return o.custom_json()
if isinstance(o, (datetime.date,
  datetime.datetime,
  datetime.time)):
return o.isoformat()[:19].replace('T', ' ')
elif isinstance(o, (int, long)):
return int(o)
elif isinstance(o, decimal.Decimal):
return str(o)
elif isinstance(o, lazyT):
return str(o)
elif isinstance(o, XmlComponent):
return str(o)
elif isinstance(o, set):
return list(o)
elif hasattr(o, 'as_list') and callable(o.as_list):
return o.as_list()
elif hasattr(o, 'as_dict') and callable(o.as_dict):
return o.as_dict()
else:
raise TypeError(repr(o) + " is not JSON serializable")

Yep.  There it is.  Give it a long instance of 110 and it returns 110L 
every time.  Javascript responds with "SyntaxError: No identifiers allowed 
directly after numeric literal" every time.

Where am I going wrong?

-- Joe

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: cpython and web2py

[Joe Barnhart]

"Make it faster" is a nice goal, but it has many different components, and just 
compiling everything with cython is not an easy way out.

The path to take, like with any software project, is to implement the function 
that you need first.  If, and only if, there is a performance problem, you 
carefully profile the system to find out exactly where the problem lies.  Then 
you determine the best way to fix the problem.

I'll give you a peek in the back of the book -- born from 40+ years of 
programming experience.  The problem is never where you thought it would be.

-- Joe

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Manual error regarding "hidden" fields

[Joe Barnhart]

"Invalidate".was supposed to be "onvalidation" before autospell worked its 
wonder...

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Manual error regarding "hidden" fields

[Joe Barnhart]

It looks like just swapping the lines works.  It's the simplest example of 
using hidden fields and it works even in my more complicated case.  I was 
hoping to be "clever" and doing something in "invalidate" after the form was 
validated but before it was stored.  But the logic in the "validate" and 
"accept" functions effectively block that.

So just swap the lines and the manual is back to being perfect!

-- Joe

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Manual error regarding "hidden" fields

[Joe Barnhart]

It's embarrassing to admit, but I'm really clueless when it comes to 
collaborating using git.  I had barely mastered hg before the change and 
I'm only able to figure out basic checkout-checkin stuff with git or I'd 
contribute more.  I'm more of an old-school programmer who cut his teeth on 
cvs.

-- Joe

On Tuesday, May 9, 2017 at 7:35:08 AM UTC-7, Anthony wrote:
>
> Good point. Maybe submit a PR to the book repo.
>
> On Monday, May 8, 2017 at 5:07:01 PM UTC-4, Joe Barnhart wrote:
>>
>> In the section about SQLFORM, the description of hidden fields is 
>> demonstrably wrong.  The code example is:
>>
>> form.vars.a = request.vars.a
>> form = SQLFORM(..., hidden=dict(a='b'))
>>
>>
>> Since the variable "form" is created on the second line, it is not 
>> available on the first as the target of an assignment.
>>
>> I do not know what the documentation should say, but I suspect its a bit 
>> more complex than this simple two-line example.  I've been struggling with 
>> this most of the day and saving the values from hidden fields still eludes 
>> me.  I can't assign the fields in an "onvalidation" method because the 
>> accepts() logic specifically looks for fields which are in the request.vars 
>> and excludes them from the saved fields.  I've been tracing thru "validate" 
>> and "accept" in FORM and SQLFORM most of the day looking for a crack in the 
>> armor to get my hidden fields saved.
>>
>> Joe
>>
>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Validate_and_insert fails because "id" has no value

[Joe Barnhart]

You are correct sir.  

The table has three "reference" fields and their default validators were 
the unforgiving type.  Since this was a special case in which it was OK if 
the references were missing, I just nuked the validators before doing the 
"validate_and_insert" by setting the "requires=None" for the three 
reference fields.

-- Joe


On Tuesday, May 9, 2017 at 7:34:15 AM UTC-7, Anthony wrote:
>
> id fields do not get any validators by default, and if they did, it 
> certainly would not be an IS_IN_DB validator (if anything, it would be 
> IS_NOT_IN_DB, as id values must be unique). Can we see your table 
> definition?
>
> Is it possible the problem is instead with a reference field, which by 
> default would have an IS_IN_DB validator and would therefore be required 
> for an insert?
>
> Anthony
>
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Validate_and_insert fails because "id" has no value

[Joe Barnhart]

So I'm using "validate_and_insert" to, well, validate and insert some 
values.  Its a simple function and easily understood:

def validate_and_insert(self, **fields):
response, new_fields = self._validate_fields(fields)
if not response.errors:
response.id = self.insert(**new_fields)
return response

It relies on "_validate" to check the validators:

def _validate_fields(self, fields, defattr='default'):
response = Row()
response.id, response.errors = None, Row()
new_fields = copy.copy(fields)
for fieldname in self.fields:
default = getattr(self[fieldname], defattr)
if callable(default):
default = default()
raw_value = fields.get(fieldname, default)
value, error = self[fieldname].validate(raw_value)
if error:
response.errors[fieldname] = "%s" % error
elif value is not None:
new_fields[fieldname] = value
return response, new_fields

And here lies the problem.  The field "id" was NOT provided in the "fields" 
argument since we are inserting values, i.e. we don't have an id yet.  But 
in the loop "for fieldname in self.fields" we are looping through ALL of 
the fields in the able, including "id".  It's clear the reason is to 
process the "default" values for each field -- a good idea.  What's NOT a 
good idea is processing the validator for "id" which defines "id" can't be 
None.

The default validator for "id" is an IS_IN_DB validator.  It fails when the 
argument (value) is "None" in the following fragment:

else:
if field.type in ('id', 'integer'):
if isinstance(value, (int, long)) or value.isdigit():
value = int(value)

The failure happens on if isinstance(value, (int, long)) or value.isdigit():  
The value "None" fails the first test, but "or" requires both arguments be 
evaluated if the first test fails, so it goes on to the second test, which 
gives an exception.

I'm not sure how this ever worked, but I'm sure it did at one point.  Maybe 
there was a regression.  This was done in web2py 2.14.6-stable.

-- Joe

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Manual error regarding "hidden" fields

[Joe Barnhart]

In the section about SQLFORM, the description of hidden fields is 
demonstrably wrong.  The code example is:

form.vars.a = request.vars.a
form = SQLFORM(..., hidden=dict(a='b'))


Since the variable "form" is created on the second line, it is not 
available on the first as the target of an assignment.

I do not know what the documentation should say, but I suspect its a bit 
more complex than this simple two-line example.  I've been struggling with 
this most of the day and saving the values from hidden fields still eludes 
me.  I can't assign the fields in an "onvalidation" method because the 
accepts() logic specifically looks for fields which are in the request.vars 
and excludes them from the saved fields.  I've been tracing thru "validate" 
and "accept" in FORM and SQLFORM most of the day looking for a crack in the 
armor to get my hidden fields saved.

Joe

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: UPDATE BOOTSTRAP VERSION FROM 3 TO 4

[Joe Barnhart]

But are you sure that's the right approach?  I force problems with 
lightweight clients (i.e. phones) doing complicated layout generation.

-- Joe


On Friday, April 21, 2017 at 8:19:15 AM UTC-7, Massimo Di Pierro wrote:
>
> Which is why I hate bootstrap and we have to move to client side 
> generation of forms and grid since server side generation does not know 
> about css.
>
> On Sunday, 16 April 2017 16:42:12 UTC-5, Anthony wrote:
>>
>> Different versions of Bootstrap involve not just changes to CSS and JS 
>> files -- the structure of the HTML and the class names use also change. So 
>> you have to make changes to layout.html and other templates, formstyles, 
>> etc.
>>
>> Anthony
>>
>> On Wednesday, April 12, 2017 at 8:46:50 AM UTC-4, Áureo Dias Neto wrote:
>>>
>>> Hello guys,
>>>
>>> How to update version 3 bootstrap to 4 on web2py?
>>>
>>> I tried to replace the .css and .js files, however I got a lot of bug 
>>> and unrecognized classes ..
>>>
>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Forms with readonly fields

[Joe Barnhart]

Hi Anthony --

After some digging, I think I understand the flow and why this is required.

When the form is built, one of the byproducts is to create a widget for 
each field and preset that widget with the information it needs to do 
validation (i.e. the 'requires' of the field is copied to the widget).  By 
the time the "form = SQLFORM(...)" is executed, the widgets are built and 
they contain the requirements of the fields.

The selector fields using the IS_IN_SET() validator will fail if the field 
doesn't contain a valid selection -- irrespective of whether that field is 
"read only" or not.  This check is done at the XML level and does not care 
about any of the settings of the Field objects at the time the check is 
done.  So nothing done to the fields with respect to making them writable 
of not can have any effect at this level.

By putting the "requires=[]" or even "requires=None" in the custom widgets 
AFTER they are built by SQLFORM, we turn off this unwanted behavior and 
ensure that readonly fields cannot cause a validation failure.  I've 
stepped through it both with and without this change, and the change is 
definitely essential for SELECT objects.  Now, it could be limited to only 
SELECT objects, but there is no harm in broadening it to all widgets.

Warm regards,

Joe


On Saturday, April 1, 2017 at 12:51:55 PM UTC-7, Anthony wrote:
>
> On Friday, March 31, 2017 at 5:41:49 PM UTC-4, Joe Barnhart wrote:
>>
>> Anthony --
>>
>> One more tiny but not insignificant detail...
>>
>> I found I had to add "requires=[]" to the custom.widget:
>>
>> [form.custom.widget[f.name].update(_readonly=True, requires=[]) for 
>> f in readonly_fields]
>>
>> Otherwise, the field keeps the 'requires' of the original Field and the 
>> check fails.  The above solution is working very well!
>>
>
> Hmm, I didn't have that problem. Do you have an example model that 
> exhibited that behavior?
>
> Anthony
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [web2py] Re: Making application looks better.

[Joe Barnhart]

Something's wrong!  The text came out garbled on the screen shots.  I can't 
read a thing!

;-)


On Sunday, April 16, 2017 at 11:16:27 AM UTC-7, Ramos wrote:
>
> My humble doc management app still in testing, with workflows... 
> I have email templates looking very nice
>
> 2017-04-16 15:37 GMT+01:00 Massimo Di Pierro  >:
>
>> screenshots? :-)
>>
>>
>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: "Show_if" and Field Validation

[Joe Barnhart]

Oh boy.  And it just keeps growing hair -- in this case to spoof the 
"options" portion of the validator.  Well, this worked for me...

class OR(Validator):
def __init__(self, first, second, multiple=False):
self.first = first if isinstance(first,(list,tuple)) else [first]
self.second = second if isinstance(second,(list,tuple)) else 
[second]
self.options = False
self.multiple = False
for ea in self.first:
if hasattr(ea,'options'):
self.options = ea.options
self.multiple = ea.multiple
break
for ea in self.second:
if hasattr(ea,'options'):
self.options = ea.options
self.multiple = ea.multiple
break

def __call__(self, value):
for ea in self.first:
retval,reterr = ea(value)
if reterr is not None:
break
if reterr is not None:
for ea in self.second:
retval,reterr = ea(value)
if reterr is not None:
break
return (retval, reterr)



On Monday, April 10, 2017 at 3:41:36 PM UTC-7, Joe Barnhart wrote:
>
> Well, I'm not sure there's a nice general solution to this one.  Since 
> "show_if" is kind of an add-on to Field and isn't really a part of 
> Expression or Validator or anything else.  Here's what I came up with...
>
> First I created my set of "show_if" fields and set their validators to the 
> existing ones with a new "OR" validator added:
>
> expr = IS_EXPR('use_qual_times==True',environment=current.request.vars)
> for ea in ['qual_course','standard1','standard2','standard3']:
> field = table[ea]
> field.show_if = (table.use_qual_times==True)
> requires = field.requires
> if requires:
> field.requires = OR(expr, requires)
>
> Then I created a validator "OR" that takes a "first" and "second" 
> argument.  If "first" passes then "second" is never evaluated.  I also 
> allowed for "second" being a list of validators.  But not "first", since 
> I'm lazy and I knew it would only be the IS_EXPR validator I was adding:
>
> from gluon.validators import Validator
> class OR(Validator):
> def __init__(self, first, second):
> self.first, self.second = first, second
> if not isinstance(second, (list,tuple)):
> self.second = [second]
>
> def __call__(self, value):
> retval, reterr = self.first(value)
> if reterr is not None:
> for ea in self.second:
> retval,reterr = ea(value)
> if reterr is not None:
> break
> return (retval, reterr)
>
> Not as general case as I'd like.  Mostly because I see no graceful way to 
> automatically generate the IS_EXPR() from the argument supplied to show_if. 
>  If that could be automated, then this processing could be pushed into the 
> FORM or SQLFORM mechanics.  (But it's getting a little crowded in there, 
> I'll admit!)
>
> -- Joe
>
> P.S.  I can't stand being a lazy slug so I fixed it:
>
> from gluon.validators import Validator
> class OR(Validator):
> def __init__(self, first, second):
> self.first = first if isinstance(first,(list,tuple)) else [first]
> self.second = second if isinstance(second,(list,tuple)) else 
> [second]
>
> def __call__(self, value):
> for ea in self.first:
> retval,reterr = ea(value)
> if reterr is not None:
> break
> if reterr is not None:
> for ea in self.second:
> retval,reterr = ea(value)
> if reterr is not None:
> break
> return (retval, reterr)
>
>
>
> On Monday, April 10, 2017 at 11:23:37 AM UTC-7, Anthony wrote:
>>
>> This might be as simple as emptying out the "requires" attribute if the 
>>> "show_if" causes the field to be visible.  But I don't change core code -- 
>>> I don't want the hassle of departing from the stock distro of web2py.
>>>
>>
>> You could change the core code and submit a pull request. ;-)
>>
>>
>>
>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: "Show_if" and Field Validation

[Joe Barnhart]

This is one of those head-scratchers Marcelo.  Note the "onvalidation" 
function is called AFTER the validation is done on the form.  By then it's 
too late to prevent the validators from being run on the hidden fields.  I 
could monkey around with the added "error" stuff after the fact, but that's 
too kludgy for even me!

-- Joe


On Monday, April 10, 2017 at 8:05:12 AM UTC-7, Marcelo Huerta wrote:
>
> Maybe creating a validation function which checks the different values and 
> depending on them fills the form.errors values when a validation error 
> happens? Take a look to 
> http://web2py.com/books/default/chapter/29/07/forms-and-validators#onvalidation
>  
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: "Show_if" and Field Validation

[Joe Barnhart]

Well, I'm not sure there's a nice general solution to this one.  Since 
"show_if" is kind of an add-on to Field and isn't really a part of 
Expression or Validator or anything else.  Here's what I came up with...

First I created my set of "show_if" fields and set their validators to the 
existing ones with a new "OR" validator added:

expr = IS_EXPR('use_qual_times==True',environment=current.request.vars)
for ea in ['qual_course','standard1','standard2','standard3']:
field = table[ea]
field.show_if = (table.use_qual_times==True)
requires = field.requires
if requires:
field.requires = OR(expr, requires)

Then I created a validator "OR" that takes a "first" and "second" argument. 
 If "first" passes then "second" is never evaluated.  I also allowed for 
"second" being a list of validators.  But not "first", since I'm lazy and I 
knew it would only be the IS_EXPR validator I was adding:

from gluon.validators import Validator
class OR(Validator):
def __init__(self, first, second):
self.first, self.second = first, second
if not isinstance(second, (list,tuple)):
self.second = [second]

def __call__(self, value):
retval, reterr = self.first(value)
if reterr is not None:
for ea in self.second:
retval,reterr = ea(value)
if reterr is not None:
break
return (retval, reterr)

Not as general case as I'd like.  Mostly because I see no graceful way to 
automatically generate the IS_EXPR() from the argument supplied to show_if. 
 If that could be automated, then this processing could be pushed into the 
FORM or SQLFORM mechanics.  (But it's getting a little crowded in there, 
I'll admit!)

-- Joe


On Monday, April 10, 2017 at 11:23:37 AM UTC-7, Anthony wrote:
>
> This might be as simple as emptying out the "requires" attribute if the 
>> "show_if" causes the field to be visible.  But I don't change core code -- 
>> I don't want the hassle of departing from the stock distro of web2py.
>>
>
> You could change the core code and submit a pull request. ;-)
>
>
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] "Show_if" and Field Validation

[Joe Barnhart]

I appreciate the new "show_if" attribute for Fields but I wonder if its 
action could be enhanced by one change...

When I have a form with optional fields, I would like to set a validator on 
the field's value should it be supplied.  I can always set it to 
IS_EMPTY_OR(blah) but this isn't strictly correct for my validation.  I 
want to REQUIRE this validation if the field is displayed, but I want the 
validation IGNORED if the field is not displayed.  In fact, the validation 
should always be ignored if the field is empty, since the user will never 
even see the error if it should fail validation!

This might be as simple as emptying out the "requires" attribute if the 
"show_if" causes the field to be visible.  But I don't change core code -- 
I don't want the hassle of departing from the stock distro of web2py.

Other ideas?

-- Joe

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Forms with readonly fields

[Joe Barnhart]

Anthony --

One more tiny but not insignificant detail...

I found I had to add "requires=[]" to the custom.widget:

[form.custom.widget[f.name].update(_readonly=True, requires=[]) for f in 
readonly_fields]

Otherwise, the field keeps the 'requires' of the original Field and the 
check fails.  The above solution is working very well!

-- Joe



On Friday, March 31, 2017 at 8:49:51 AM UTC-7, Anthony wrote:
>
> We should probably make what you are doing a built-in option (maybe even 
> the default), but for now, you can try something like this:
>
> def sqlform2(*args, **kwargs):
> table = kwargs.get('table', args[0])
> fields = kwargs.get('fields', [f for f in table])
> readonly_fields = [f for f in fields if not f.writable]
> [setattr(f, 'writable', True) for f in readonly_fields]
> form = SQLFORM(*args, **kwargs)
> [form.custom.widget[f.name].update(_readonly=True) for f in 
> readonly_fields]
> [setattr(f, 'writable', False) for f in readonly_fields]
> return form
>
> def myform():
> form = sqlform2(db.mytable, request.args(0)).process()
> return dict(form=form)
>
> The above sets the readonly fields to writable before creating the form, 
> and then sets the _readonly HTML attribute of the widgets after creating 
> the form (you could also set _disabled=True if desired). It then resets all 
> the readonly fields so they are not writable before returning, so when the 
> form is processed, those fields will not be included in the database write.
>
> Anthony
>
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Forms with readonly fields

[Joe Barnhart]

You are a frickin' genius.

Here are some epiphanies I got from your solution:

1. SQLFORM builds the XML components of the forms immediately.  I didn't 
know if that was deferred or immediate.  Now I know.

2. form.custom.widgets allows access to each XML widget in a dictionary. 
 We can screw with them before they are passed to the view.  I knew about 
custom forms but did not grasp the greater generality of the 
form.custom.widget container.

3. Since the form is materialized into XML components, we can indeed screw 
with the form after it is built but before it's processed.

Very cool, Anthony!

On Friday, March 31, 2017 at 8:49:51 AM UTC-7, Anthony wrote:
>
> We should probably make what you are doing a built-in option (maybe even 
> the default), but for now, you can try something like this:
>
> def sqlform2(*args, **kwargs):
> table = kwargs.get('table', args[0])
> fields = kwargs.get('fields', [f for f in table])
> readonly_fields = [f for f in fields if not f.writable]
> [setattr(f, 'writable', True) for f in readonly_fields]
> form = SQLFORM(*args, **kwargs)
> [form.custom.widget[f.name].update(_readonly=True) for f in 
> readonly_fields]
> [setattr(f, 'writable', False) for f in readonly_fields]
> return form
>
> def myform():
> form = sqlform2(db.mytable, request.args(0)).process()
> return dict(form=form)
>
> The above sets the readonly fields to writable before creating the form, 
> and then sets the _readonly HTML attribute of the widgets after creating 
> the form (you could also set _disabled=True if desired). It then resets all 
> the readonly fields so they are not writable before returning, so when the 
> form is processed, those fields will not be included in the database write.
>
> Anthony
>
> On Friday, March 31, 2017 at 5:25:59 AM UTC-4, Joe Barnhart wrote:
>>
>> I'm going a little nuts with forms that have readonly fields.  Examples 
>> are forms where some fields are shown to the user so they can see the 
>> contents, but they aren't allowed to change them.  For example, an 
>> "expiration date" for a subscription.  They can see when it expires, but 
>> they can't just edit it.  They have to go thru an ordering process for that.
>>
>> The Field flag of writable=False has not proved useful.  It changes the 
>> field to a simple piece of text and thus breaks any custom formstyle I'm 
>> using.  The simple text without an input field looks ugly and doesn't match 
>> spacing, fonts, or any of the other myriad styles used in the read/write 
>> fields.  Sad to say, writable=False just is not useful.
>>
>> I have created an alternative that creates the fields as usual, but marks 
>> them with a "readonly" attribute.  Javascript can then be used to ensure 
>> the contents are not modified, even for tricky fields like date selectors 
>> and list box widgets.  But now we get to the underlying problem -- the form 
>> validator fails when I take this approach.
>>
>> Because the readonly fields are readonly, they do not show up in the 
>> request.vars when the form is submitted.  But since the SQLFORM knows 
>> nothing of this, it just sees null values for the fields and throws a 
>> validation error for elements like list boxes (which can't be empty).
>>
>> It's almost like I want to change the SQLFORM definition after the form 
>> is created (with the readonly list boxes) and then remove those fields from 
>> the SQLFORM when it does its validate processing, so it would just ignore 
>> the missing fields and not try to update the record with empty values.
>>
>> Here's an example.  You can see the cursor which tells the user the field 
>> is not editable.  On this view the expire date, the season, the age, and 
>> the US Swim ID are all non-editable fields.  But the season is an example 
>> of an option list which causes the form processing to throw an error and 
>> fail.
>>
>> Just looking for some ideas.
>>
>> Joe
>>
>>
>> <https://lh3.googleusercontent.com/-BqVuGZtmcZw/WN4frOAVqLI/Aoc/Hd37dU2uKB89UIRmwZTI5bfx9q2wqDN-QCLcB/s1600/Form_with_readonly.png>
>>
>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Forms with readonly fields

[Joe Barnhart]

I'm going a little nuts with forms that have readonly fields.  Examples are 
forms where some fields are shown to the user so they can see the contents, 
but they aren't allowed to change them.  For example, an "expiration date" 
for a subscription.  They can see when it expires, but they can't just edit 
it.  They have to go thru an ordering process for that.

The Field flag of writable=False has not proved useful.  It changes the 
field to a simple piece of text and thus breaks any custom formstyle I'm 
using.  The simple text without an input field looks ugly and doesn't match 
spacing, fonts, or any of the other myriad styles used in the read/write 
fields.  Sad to say, writable=False just is not useful.

I have created an alternative that creates the fields as usual, but marks 
them with a "readonly" attribute.  Javascript can then be used to ensure 
the contents are not modified, even for tricky fields like date selectors 
and list box widgets.  But now we get to the underlying problem -- the form 
validator fails when I take this approach.

Because the readonly fields are readonly, they do not show up in the 
request.vars when the form is submitted.  But since the SQLFORM knows 
nothing of this, it just sees null values for the fields and throws a 
validation error for elements like list boxes (which can't be empty).

It's almost like I want to change the SQLFORM definition after the form is 
created (with the readonly list boxes) and then remove those fields from 
the SQLFORM when it does its validate processing, so it would just ignore 
the missing fields and not try to update the record with empty values.

Here's an example.  You can see the cursor which tells the user the field 
is not editable.  On this view the expire date, the season, the age, and 
the US Swim ID are all non-editable fields.  But the season is an example 
of an option list which causes the form processing to throw an error and 
fail.

Just looking for some ideas.

Joe



-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Customizing Fields the lazy way

[Joe Barnhart]

That's kinda clever.  I may think of that!

-- Joe


On Thursday, March 23, 2017 at 4:19:05 PM UTC-7, Anthony wrote:
>
> Note, you might as well also add 'type': 'string' to your dictionary, and 
> maybe 'length': 20. You can also give yourself some flexibility by creating 
> a function:
>
> def phone_field(name, **kwargs):
> defaults = {'type': 'string',
> 'length': 20,
> 'requires': IS_EMPTY_OR(IS_PHONE()),
> 'widget': lambda fld,val: SQLFORM.widgets.string.widget(
> fld, val, _type='tel', _class='form-control')}
> defaults.update(**kwargs)
> return Field(name, **defaults)
>
> Anthony
>
>
>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: DAL executesql and escaping apostrophe/single quote

[Joe Barnhart]

I'm not sure what you're doing with a single-quote table name (?) but I 
observed your statements have different results after string substitution.

In the first case, substituting the "Single' Quote" string results in:

"SELECT * FROM table1 WHERE name = Single' Quote LIMIT 1"

Whereas in your "reference" case there are extra single quotes around the 
argument of the name:

"SELECT * FROM table1 WHERE name = 'Single'' Quote' LIMIT 1"

Perhaps this is the reason for the difference in behavior.  I know various 
versions of SQL are persnickety (technical term) with respect to quoting 
arguments.  Your earlier statement can be changed to add single quotes to 
the name:

"SELECT * FROM table1 WHERE name = '%s' LIMIT 1"%("Single' Quote",)

Which yields a single-quoted string with a single quote inside:

"SELECT * FROM table1 WHERE name = 'Single' Quote' LIMIT 1"

If that doesn't work, you can double up on the substitution argument and 
get exactly the string as your "reference"

"SELECT * FROM table1 WHERE name = '%s' LIMIT 1"%("Single'' Quote",)

Which yields:

"SELECT * FROM table1 WHERE name = 'Single'' Quote' LIMIT 1"




On Tuesday, March 21, 2017 at 4:13:32 AM UTC-7, Travis Smith wrote:
>
> Hi guys,
>
> I can't seem to find anything anywhere because everything just says "it 
> works".
>
> I'm using the latest version of web2py and mysql (maria) as my db, and I'm 
> trying to run executesql with the following query:
>
> db.executesql("SELECT * FROM table1 WHERE name = %s LIMIT 1", ("Single' 
> Quote",), as_dict=True)
>
> If I run that without the single quote, it works as expected, returning 
> the value I have without the single quote. I also have another value with 
> the single quote for testing.
>
> However, when I run the above, I get 
>
> "You have an error in your SQL syntax; check the manual that corresponds 
> to your MySQL server version for the right syntax to use near 'Quote' limit 
> 1"
>
> Is this not supposed to escape everything for me and treat them as 
> parameters? If not, what is the alternative? I have other, pre-existing 
> tables I need to query data from, so using the db.table1 syntax won't work 
> without re-creating them entirely (I think).
>
> For reference, if I run
>
> db.executesql("SELECT * FROM table1 WHERE name = 'Single'' Quote' LIMIT 
> 1", as_dict=True)
>
> and manually escape the quote, everything works as expected.
>
> Any help would be appreciated.
>
>
> Thanks,
>
> Travis
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [web2py] lazy_tables + 'reference tablename' = referenced table not lazy

[Joe Barnhart]

Just a couple of philosophical comments...

Whenever I find that the web2py platform is "forcing" me to code in large 
volumes and produce ugly, quirky code which I *know* will be hard to 
maintain, I stop myself and take another look at how I'm trying to 
accomplish my task.  I have this philosophy that using a platform like 
web2py (or Python, or Linux, or anything) involves a certain mindset. 
 Those who created and maintain the platform envisioned working with it in 
a certain way.  I call that the "swimming downstream" way of working.  When 
I find I am "swimming upstream" I realize that I'm not using the platform 
as intended.

In this case, for instance, you want to instantiate one table but NOT 
another table which is linked to this table.  That seems like a logic error 
to me.  It looks like a FEATURE that lazy table instantiation will create 
BOTH tables in this case -- certainly the linked table and its definition 
should be available wherever its linker table exists.  Standing on your 
head to get around this is "swimming upstream".

There are numerous ways to make your site more efficient,* if efficiency 
turns out to be a problem*.  Believe me, I get it.  I, too, am guilty of 
"premature optimization" more often that I like to admit.  But the best 
course of action is to steam ahead and get your application working the 
easiest way possible (downstream swimming).  Then, and only then, locate 
any performance trouble spots and apply optimization.  Lazy tables are only 
one such optimization.

You heard mention of putting tables in a module.  I have about 50 tables in 
my app, some with 50 fields.  To instantiate all of them all of the time 
would be somewhat of a burden.  So I have opted to put table creation in 
modules.  If that isn't enough, I'll worry about that when I have a problem.

Swim downstream.  It's the only way!

-- Joe


On Monday, March 13, 2017 at 11:03:46 AM UTC-7, Richard wrote:
>
> Hello,
>
> I am analysing my app and found that many tables that I thought were lazy 
> (lazy_tables = True) are actually not... Investigating why was that... I 
> found (I think) that 'reference table_name' cause a table to be not be lazy 
> anymore as mostly all table that are referenced are not lazy... There is no 
> insight about that in the book, or I didn't find any... 
>
> Is referenced tables can be lazy?
>
> I use web2py 2.14.6 build...
>
> There is old issue in pyDAL repo about reference and lazyness but they are 
> closed.
>
> Thanks
>
> Richard
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.