[web2py] Scheduler workers die on python 3 + Ubuntu 18.04

[Antonio Salazar]

Using python 2 on the same system, the scheduler is rock-solid, but using 
python 3 the workers die after processing a task. The rest of the tasks are 
left in queue and the worker entry is deleted from the scheduler_worker 
table.

Aug 28 20:22:03 ubuntu1806 sudo[16121]: Exception in thread Thread-1:
Aug 28 20:22:03 ubuntu1806 sudo[16121]: Traceback (most recent call last):
Aug 28 20:22:03 ubuntu1806 sudo[16121]:   File 
"/var/www/web2py/gluon/scheduler.py", line 1168, in send_heartbeat
Aug 28 20:22:03 ubuntu1806 sudo[16121]: mybackedstatus = db(sw.worker_name 
== self.worker_name).select().first()
Aug 28 20:22:03 ubuntu1806 sudo[16121]:   File 
"/var/www/web2py/gluon/packages/dal/pydal/objects.py", line 2277, in select
Aug 28 20:22:03 ubuntu1806 sudo[16121]: return adapter.select(self.query
, fields, attributes)
Aug 28 20:22:03 ubuntu1806 sudo[16121]:   File 
"/var/www/web2py/gluon/packages/dal/pydal/adapters/base.py", line 763, in 
select
Aug 28 20:22:03 ubuntu1806 sudo[16121]: return self._select_aux(sql, 
fields, attributes, colnames)
Aug 28 20:22:03 ubuntu1806 sudo[16121]:   File 
"/var/www/web2py/gluon/packages/dal/pydal/adapters/base.py", line 719, in 
_select_aux
Aug 28 20:22:03 ubuntu1806 sudo[16121]: rows = self._select_aux_execute(
sql)
Aug 28 20:22:03 ubuntu1806 sudo[16121]:   File 
"/var/www/web2py/gluon/packages/dal/pydal/adapters/base.py", line 713, in 
_select_aux_execute
Aug 28 20:22:03 ubuntu1806 sudo[16121]: self.execute(sql)
Aug 28 20:22:03 ubuntu1806 sudo[16121]:   File 
"/var/www/web2py/gluon/packages/dal/pydal/adapters/__init__.py", line 67, in 
wrap
Aug 28 20:22:03 ubuntu1806 sudo[16121]: return f(*args, **kwargs)
Aug 28 20:22:03 ubuntu1806 sudo[16121]:   File 
"/var/www/web2py/gluon/packages/dal/pydal/adapters/base.py", line 413, in 
execute
Aug 28 20:22:03 ubuntu1806 sudo[16121]: rv = self.cursor.execute(command
, *args[1:], **kwargs)
Aug 28 20:22:03 ubuntu1806 sudo[16121]: psycopg2.OperationalError: SSL 
connection has been closed unexpectedly
Aug 28 20:22:03 ubuntu1806 sudo[16121]: During handling of the above 
exception, another exception occurred:
Aug 28 20:22:03 ubuntu1806 sudo[16121]: Traceback (most recent call last):
Aug 28 20:22:03 ubuntu1806 sudo[16121]:   File 
"/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
Aug 28 20:22:03 ubuntu1806 sudo[16121]: self.run()
Aug 28 20:22:03 ubuntu1806 sudo[16121]:   File 
"/var/www/web2py/gluon/scheduler.py", line 637, in run
Aug 28 20:22:03 ubuntu1806 sudo[16121]: self.send_heartbeat(counter)
Aug 28 20:22:03 ubuntu1806 sudo[16121]:   File 
"/var/www/web2py/gluon/scheduler.py", line 1239, in send_heartbeat
Aug 28 20:22:03 ubuntu1806 sudo[16121]: db.rollback()
Aug 28 20:22:03 ubuntu1806 sudo[16121]:   File 
"/var/www/web2py/gluon/packages/dal/pydal/base.py", line 708, in rollback
Aug 28 20:22:03 ubuntu1806 sudo[16121]: self._adapter.rollback()
Aug 28 20:22:03 ubuntu1806 sudo[16121]:   File 
"/var/www/web2py/gluon/packages/dal/pydal/adapters/__init__.py", line 56, in 
wrap
Aug 28 20:22:03 ubuntu1806 sudo[16121]: return f(*args, **kwargs)
Aug 28 20:22:03 ubuntu1806 sudo[16121]:   File 
"/var/www/web2py/gluon/packages/dal/pydal/adapters/base.py", line 866, in 
rollback
Aug 28 20:22:03 ubuntu1806 sudo[16121]: return self.connection.rollback
()
Aug 28 20:22:03 ubuntu1806 sudo[16121]: psycopg2.InterfaceError: connection 
already closed

OS is Ubuntu 18.04 server
web2py is 2.17.1
postgresql is 10.5
python-psycopg2 and python3-psycopg2 both are 2.7.4-1.

The scheduler runs as a systemd service like this:
/usr/bin/sudo -u www-data  /var/www/web2py/web2py.py -K 
myapp,myapp:slow,myapp:slowest
where  is "python" or "python3"

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: Allowing users to delete their profiles

[Donald McClymont]

I have opened issue #1996 to cover this and can confirm works correclty 
once I remove the computed field from the auth_user table.

Many thanks
Donald

On Wednesday, August 29, 2018 at 12:58:21 AM UTC+1, Anthony wrote:
>
> Looks like a bug here: 
> https://github.com/web2py/web2py/blob/63972386c23961360d99c5ce49240b0aed5c820f/gluon/tools.py#L3657-L3661
>
> Presumably you have defined one or more extra_fields on db.auth_user that 
> are computed fields. That triggers the above code, but because the user 
> record has been deleted, None gets passed to self._update_session_user, 
> which generates an error.
>
> Please open a Github issue and refer back to this post.
>
> Anthony
>
> On Tuesday, August 28, 2018 at 6:13:53 PM UTC-4, Donald McClymont wrote:
>>
>>
>>
>> On Tuesday, August 28, 2018 at 12:16:59 PM UTC+1, Anthony wrote:
>>>
>>> Please post the traceback here. The error ticket fil
>>
>> Error ticket for "gdms"Ticket ID
>>
>> 127.0.0.1.2018-08-27.21-31-21.fbba2e6a-7e7d-4081-b191-b07d009d01c4
>>  'NoneType' object is not iterableVersion
>> web2py™ Version 2.17.1-stable+timestamp.2018.08.06.01.02.56
>> Python Python 2.7.15: C:\Python27\python.exe (prefix: C:\Python27)
>> Traceback
>>
>> 1.
>> 2.
>> 3.
>> 4.
>> 5.
>> 6.
>> 7.
>> 8.
>> 9.
>> 10.
>> 11.
>> 12.
>> 13.
>> 14.
>> 15.
>> 16.
>> 17.
>> 18.
>>
>> Traceback (most recent call last):
>>   File "C:\web2py\gluon\restricted.py", line 219, in restricted
>> exec(ccode, environment)
>>   File "C:/web2py/applications/gdms/controllers/default.py" 
>> , line 
>> 451, in 
>>   File "C:\web2py\gluon\globals.py", line 421, in 
>> self._caller = lambda f: f()
>>   File "C:/web2py/applications/gdms/controllers/default.py" 
>> , line 
>> 449, in user
>> return dict(form=auth())
>>   File "C:\web2py\gluon\tools.py", line 1799, in __call__
>> return getattr(self, args[0])()
>>   File "C:\web2py\gluon\tools.py", line 3660, in profile
>> self._update_session_user(user)
>>   File "C:\web2py\gluon\authapi.py", line 720, in _update_session_user
>> user = Row(user)
>>   File "C:\web2py\gluon\packages\dal\pydal\helpers\classes.py", line 34, in 
>> __init__
>> return self.__dict__.__init__(*args, **kwargs)
>> TypeError: 'NoneType' object is not iterable
>>
>> Error snapshot [image: help] 
>> 
>>
>> ('NoneType' object is not iterable)
>>
>> inspect attributes
>> Frames
>>
>>- 
>>
>>*File C:\web2py\gluon\restricted.py in restricted at line 219* code 
>>arguments variables
>>- 
>>
>>*File C:\web2py\applications\gdms\controllers\default.py in  
>>at line 451* code arguments variables
>>- 
>>
>>*File C:\web2py\gluon\globals.py in  at line 421* code 
>>arguments variables
>>- 
>>
>>*File C:\web2py\applications\gdms\controllers\default.py in user at 
>>line 449* code arguments variables
>>- 
>>
>>*File C:\web2py\gluon\tools.py in __call__ at line 1799* code 
>>arguments variables
>>- 
>>
>>*File C:\web2py\gluon\tools.py in profile at line 3660* code arguments
>> variables
>>- 
>>
>>*File C:\web2py\gluon\authapi.py in _update_session_user at line 720* 
>>code arguments variables
>>- 
>>
>>*File C:\web2py\gluon\packages\dal\pydal\helpers\classes.py in 
>>__init__ at line 34* code arguments variables
>>Function argument list
>>
>>(self=, *args=(None,), **kwargs={})
>>Code listing
>>
>>29.
>>30.
>>31.
>>32.
>>33.
>>34.
>>35.
>>36.
>>37.
>>38.
>>
>>
>>
>>@implements_bool
>>class BasicStorage(object):
>>def __init__(self, *args, **kwargs):
>>return self.__dict__.__init__(*args, **kwargs)
>>
>>def __getitem__(self, key):
>>return self.__dict__.__getitem__(str(key))
>>
>>Variables
>>self.__dict__ {}
>>self 
>>args (None,)
>>self.__dict__.__init__ 
>>kwargs {}
>>
>> Context
>>
>> locals request session response
>> In file: C:\web2py\applications\gdms\controllers/default.py
>>
>> 1.
>>
>>  at 04BED830, file 
>> "C:\web2py\applications\gdms\controllers/default.py", line 37>
>>
>> e is not easily readable, as it is a pickled object.
>>
>>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [web2py] web2py hangs uploading file

[Carlos Cesar Caballero Díaz]

I have found the issue, there was a typo (that's the bad ting about 
copying and pasting) in my "resource" field of "dar_resource_file" 
table, is was declared as:


    Field("file", "list:refrence dar_resource")

instead of:

    Field("file", "refrence dar_resource")

Anyway, I think that it should rise an exception instead of the current 
behavior.



Greetings.


El 29/08/18 a las 13:45, Carlos Cesar Caballero Díaz escribió:

Hi guys, I am trying to manually upload a file, but this code:

    file_store = 
db.dar_resource_file.file.store(request.vars.file.file, 
request.vars.file.filename)


db.dar_resource_file.update_or_insert(db.dar_resource_file.resorce=resource_id, 
resource=resource_id, name=resource_name, file=file_store)


completely hangs web2py, when update_or_insert is called, some kind of 
infinite loop starts, and the cpu usage goes to 100% and never finish.


I am using the last web2py version and Python 3.5

Greetings.





--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups "web2py-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] web2py hangs uploading file

[Carlos Cesar Caballero Díaz]

Hi guys, I am trying to manually upload a file, but this code:

    file_store = 
db.dar_resource_file.file.store(request.vars.file.file, 
request.vars.file.filename)


db.dar_resource_file.update_or_insert(db.dar_resource_file.resorce=resource_id, 
resource=resource_id, name=resource_name, file=file_store)


completely hangs web2py, when update_or_insert is called, some kind of 
infinite loop starts, and the cpu usage goes to 100% and never finish.


I am using the last web2py version and Python 3.5

Greetings.


--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups "web2py-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [web2py] Re: Form submit takes too long

[Maurice Waka]

Answers is a small table, can have up to 100 rows. I'll introduce the
limitby(). There are several functions that an item loops through, almost
100, but with a filter to limit the number of functions, with the final
result returned by report()


On Wed, 29 Aug 2018, 18:30 Anthony  wrote:

> db.answers.insert(quest=names, message=report())
>>
>
> What does report() do? Is that an expensive function?
>
>
>> replies = db(db.answers.author == auth.user.id).select(db.answers.ALL
>> )[-10:-1]
>>
>
> Above you are selecting every record in the db.answers table and
> converting them to Python objects -- is that a big table? You should
> instead use limitby to select only the records you want.
>
> Anthony
>
> --
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/OJRaYk57ZPQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: URL function in admin app adds "/_2.17.1/" to path variable

[Anthony]

The admin app uses the built-in static versioning functionality, as 
described 
here: 
http://web2py.com/books/default/chapter/29/04/the-core#Static-asset-management. 
You may need to configure your web server properly to handle those URLs, as 
described in the above link.

Anthony

On Wednesday, August 29, 2018 at 10:08:32 AM UTC-4, Jörg Schneider wrote:
>
>
> After downloading 2.17.1 from Github I get wrong static URLs in the admin 
> application:
>
> response.files.insert(0,URL('static','js/jquery.js')) in web2py_ajax.js 
> resolves the URL to 

[web2py] Re: Form submit takes too long

[Anthony]

>
> db.answers.insert(quest=names, message=report())
>

What does report() do? Is that an expensive function?
 

> replies = db(db.answers.author == auth.user.id).select(db.answers.ALL
> )[-10:-1]
>

Above you are selecting every record in the db.answers table and converting 
them to Python objects -- is that a big table? You should instead use 
limitby to select only the records you want.

Anthony

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Re: URL function in admin app adds "/_2.17.1/" to path variable

[Jörg Schneider]

You made my day!!! It were this lines in 0.py

Hahaha, RTFM! I'm always finding new things in this book even I thought 
I've seen all chapters... 

Am Mittwoch, 29. August 2018 16:37:48 UTC+2 schrieb fiubarc:
>
> Hello, look at db.py ... probably you have uncommented these lines:
>
> # response.static_version = myconf.get('app.version')
> # response.static_version_urls = True
>
>
> http://www.web2py.com/books/default/chapter/29/04/the-core#Static-asset-management
>
>
>
> El miércoles, 29 de agosto de 2018, 11:08:32 (UTC-3), Jörg Schneider 
> escribió:
>>
>>
>> After downloading 2.17.1 from Github I get wrong static URLs in the admin 
>> application:
>>
>> response.files.insert(0,URL('static','js/jquery.js')) in web2py_ajax.js 
>> resolves the URL to 

[web2py] Re: URL function in admin app adds "/_2.17.1/" to path variable

[fiubarc]

Hello, look at db.py ... probably you have uncommented these lines:

# response.static_version = myconf.get('app.version')
# response.static_version_urls = True

http://www.web2py.com/books/default/chapter/29/04/the-core#Static-asset-management



El miércoles, 29 de agosto de 2018, 11:08:32 (UTC-3), Jörg Schneider 
escribió:
>
>
> After downloading 2.17.1 from Github I get wrong static URLs in the admin 
> application:
>
> response.files.insert(0,URL('static','js/jquery.js')) in web2py_ajax.js 
> resolves the URL to 

[web2py] URL function in admin app adds "/_2.17.1/" to path variable

[Jörg Schneider]

After downloading 2.17.1 from Github I get wrong static URLs in the admin 
application:

response.files.insert(0,URL('static','js/jquery.js')) in web2py_ajax.js 
resolves the URL to 

[web2py] CAS and registration_id field

[fiubarc]

Hello, please, I need help with the following:
I have the login with CAS configured and when logging in, the user's registry 
is modified adding registration_id as  /cas/[user_email]... It works fine 
but,

when the user does not have an email, then the registration_id field ends as 
/cas/None and it is inserted as a new user in auth_user with all fields in 
null. 

There will be some way that the username will be used instead of the email?

Thanks!!

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[web2py] Form submit takes too long

[Maurice Waka]

I noticed that my form submit takes too long, almost 45 seconds sometimes. 
The more data I add to the database and I need to fetch it, the more longer 
it takes to submit.
This is my code



{{=form.custom.begin}}

send
{{=form.custom.end}}











Controller function

def c_function():
form = SQLFORM(db.post).process()
row = db(db.post.author== auth.user.id).select(db.post.id, db.post.
message, orderby=~db.post.id, limitby=(0,1)).first()
names = row.message if row else None
db.answers.insert(quest=names, message=report())
replies = db(db.answers.author == auth.user.id).select(db.answers.ALL)[-
10:-1]
return dict(form=form,names=names,replies=replies)

-- 
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
--- 
You received this message because you are subscribed to the Google Groups 
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.