Re: [sqlalchemy] SAWarning shows not immediately, but after some time of app execution

[TomS.]

On 07/22/2016 05:13 PM, Simon King wrote:

On Fri, Jul 22, 2016 at 3:25 PM, TomS.  wrote:

On 07/19/2016 06:41 PM, Mike Bayer wrote:



On 07/19/2016 11:51 AM, TomS. wrote:

Hi,

We have Flask app which uses SQLAlchemy. Weird error started to happen
recently. The difficulty is that we can't reproduce the error (/figure
out conditions causing issue) - maybe someone could help. Any hints/tips
would be appreciated.

There is a part in the code which constructs IN in SQL:

MyModel.id.in_(my_ids)

For some cases my_ids is an empty list. It works without any problems,
but after some time the same query (using empty list) starts to raise an
exception:

SAWarning: The IN-predicate on "MyModel.id" was invoked with an empty
sequence. This results in a contradiction, which nonetheless can be
expensive to evaluate.  Consider alternative strategies for improved
performance.

After restarting app, everything works again.

The question is - why this exception is not risen always (although we
tried to run app with empty list directly), but after some time of app
execution (~1 day)?


It's not an exception, it's a warning.  Python warnings by default emit only
once, see:
https://docs.python.org/2/library/warnings.html#the-warnings-filter

Ok, this is the explanation why it doesn't show up regularly. Thank you.

I don't know why SAWarning is treated as error. Here is the log:

2016-07-21 13:58:14,108 ERROR: Exception on /own [GET]
Traceback (most recent call last):
[...]
   File
"/home/developer/.virtualenvs/rest/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py",
line 1297, in warn
 warnings.warn(msg, exc.SAWarning, stacklevel=2)
SAWarning: The IN-predicate on "MyModel.id" was invoked with an empty
sequence. This results in a contradiction, which nonetheless can be
expensive to evaluate.

I will also ask on Flask group, but maybe you know the reason - Flask config
regarding logging is as follow:

import logging

from logging.handlers import RotatingFileHandler

logger_file_handler = RotatingFileHandler('my.log', maxBytes=1024 * 1024 *
100, backupCount=20)
logger_file_handler.setLevel(logging.DEBUG)
logger_formatter = logging.Formatter(u'%(asctime)s %(levelname)s:
%(message)s')

logger_file_handler.setFormatter(logger_formatter)

logging.captureWarnings(True)

app.logger.addHandler(logger_file_handler)
app.logger.setLevel(logging.DEBUG)

There is no code which change behavior (by  filter) of SAWarning to 'error'
...

The python warnings system is completely separate from the logging
system. Whether or not a particular warning is turned into an
exception is driven by the "warnings filter":

https://docs.python.org/2/library/warnings.html#the-warnings-filter

I think you must be configuring the warnings filter somewhere, because
by default this shouldn't raise an exception.
Yes, you were right. I finally found the source of the issue. There was 
link in the part of the code which set warnings to behave like the 
exceptions. But this code was not executed always but under certain 
conditions. That was the first issue with debuging. The second one - the 
fact that it is web app which is run using many processes. So on some 
"process" code was already executed while on others not.


Thank you for the ideas!


Simon


Cheers!

--
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

Re: [sqlalchemy] SAWarning shows not immediately, but after some time of app execution

[Simon King]

On Fri, Jul 22, 2016 at 3:25 PM, TomS.  wrote:
> On 07/19/2016 06:41 PM, Mike Bayer wrote:
>
>
>
> On 07/19/2016 11:51 AM, TomS. wrote:
>
> Hi,
>
> We have Flask app which uses SQLAlchemy. Weird error started to happen
> recently. The difficulty is that we can't reproduce the error (/figure
> out conditions causing issue) - maybe someone could help. Any hints/tips
> would be appreciated.
>
> There is a part in the code which constructs IN in SQL:
>
> MyModel.id.in_(my_ids)
>
> For some cases my_ids is an empty list. It works without any problems,
> but after some time the same query (using empty list) starts to raise an
> exception:
>
> SAWarning: The IN-predicate on "MyModel.id" was invoked with an empty
> sequence. This results in a contradiction, which nonetheless can be
> expensive to evaluate.  Consider alternative strategies for improved
> performance.
>
> After restarting app, everything works again.
>
> The question is - why this exception is not risen always (although we
> tried to run app with empty list directly), but after some time of app
> execution (~1 day)?
>
>
> It's not an exception, it's a warning.  Python warnings by default emit only
> once, see:
> https://docs.python.org/2/library/warnings.html#the-warnings-filter
>
> Ok, this is the explanation why it doesn't show up regularly. Thank you.
>
> I don't know why SAWarning is treated as error. Here is the log:
>
> 2016-07-21 13:58:14,108 ERROR: Exception on /own [GET]
> Traceback (most recent call last):
> [...]
>   File
> "/home/developer/.virtualenvs/rest/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py",
> line 1297, in warn
> warnings.warn(msg, exc.SAWarning, stacklevel=2)
> SAWarning: The IN-predicate on "MyModel.id" was invoked with an empty
> sequence. This results in a contradiction, which nonetheless can be
> expensive to evaluate.
>
> I will also ask on Flask group, but maybe you know the reason - Flask config
> regarding logging is as follow:
>
> import logging
>
> from logging.handlers import RotatingFileHandler
>
> logger_file_handler = RotatingFileHandler('my.log', maxBytes=1024 * 1024 *
> 100, backupCount=20)
> logger_file_handler.setLevel(logging.DEBUG)
> logger_formatter = logging.Formatter(u'%(asctime)s %(levelname)s:
> %(message)s')
>
> logger_file_handler.setFormatter(logger_formatter)
>
> logging.captureWarnings(True)
>
> app.logger.addHandler(logger_file_handler)
> app.logger.setLevel(logging.DEBUG)
>
> There is no code which change behavior (by  filter) of SAWarning to 'error'
> ...

The python warnings system is completely separate from the logging
system. Whether or not a particular warning is turned into an
exception is driven by the "warnings filter":

https://docs.python.org/2/library/warnings.html#the-warnings-filter

I think you must be configuring the warnings filter somewhere, because
by default this shouldn't raise an exception.

Simon

-- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

Re: [sqlalchemy] SAWarning shows not immediately, but after some time of app execution

[TomS.]

On 07/19/2016 06:41 PM, Mike Bayer wrote:



On 07/19/2016 11:51 AM, TomS. wrote:

Hi,

We have Flask app which uses SQLAlchemy. Weird error started to happen
recently. The difficulty is that we can't reproduce the error (/figure
out conditions causing issue) - maybe someone could help. Any hints/tips
would be appreciated.

There is a part in the code which constructs IN in SQL:

MyModel.id.in_(my_ids)

For some cases my_ids is an empty list. It works without any problems,
but after some time the same query (using empty list) starts to raise an
exception:

SAWarning: The IN-predicate on "MyModel.id" was invoked with an empty
sequence. This results in a contradiction, which nonetheless can be
expensive to evaluate.  Consider alternative strategies for improved
performance.

After restarting app, everything works again.

The question is - why this exception is not risen always (although we
tried to run app with empty list directly), but after some time of app
execution (~1 day)?


It's not an exception, it's a warning.  Python warnings by default 
emit only once, see: 
https://docs.python.org/2/library/warnings.html#the-warnings-filter

Ok, this is the explanation why it doesn't show up regularly. Thank you.

I don't know why SAWarning is treated as error. Here is the log:

2016-07-21 13:58:14,108 ERROR: Exception on /own [GET]
Traceback (most recent call last):
[...]
  File 
"/home/developer/.virtualenvs/rest/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", 
line 1297, in warn

warnings.warn(msg, exc.SAWarning, stacklevel=2)
SAWarning: The IN-predicate on "MyModel.id" was invoked with an empty 
sequence. This results in a contradiction, which nonetheless can be 
expensive to evaluate.


I will also ask on Flask group, but maybe you know the reason - Flask 
config regarding logging is as follow:


import logging

from logging.handlers import RotatingFileHandler

logger_file_handler = RotatingFileHandler('my.log', maxBytes=1024 * 1024 
* 100, backupCount=20)

logger_file_handler.setLevel(logging.DEBUG)
logger_formatter = logging.Formatter(u'%(asctime)s %(levelname)s: 
%(message)s')


logger_file_handler.setFormatter(logger_formatter)

logging.captureWarnings(True)

app.logger.addHandler(logger_file_handler)
app.logger.setLevel(logging.DEBUG)

There is no code which change behavior (by filter) of SAWarning to 
'error' ...


Cheers






Details:
Flask==0.10.1
Flask-SQLAlchemy==2.1
SQLAlchemy==1.0.14
MySQL DB

Cheers






--
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

Re: [sqlalchemy] SAWarning shows not immediately, but after some time of app execution

[Mike Bayer]

On 07/19/2016 11:51 AM, TomS. wrote:

Hi,

We have Flask app which uses SQLAlchemy. Weird error started to happen
recently. The difficulty is that we can't reproduce the error (/figure
out conditions causing issue) - maybe someone could help. Any hints/tips
would be appreciated.

There is a part in the code which constructs IN in SQL:

MyModel.id.in_(my_ids)

For some cases my_ids is an empty list. It works without any problems,
but after some time the same query (using empty list) starts to raise an
exception:

SAWarning: The IN-predicate on "MyModel.id" was invoked with an empty
sequence. This results in a contradiction, which nonetheless can be
expensive to evaluate.  Consider alternative strategies for improved
performance.

After restarting app, everything works again.

The question is - why this exception is not risen always (although we
tried to run app with empty list directly), but after some time of app
execution (~1 day)?


It's not an exception, it's a warning.  Python warnings by default emit 
only once, see: 
https://docs.python.org/2/library/warnings.html#the-warnings-filter






Details:
Flask==0.10.1
Flask-SQLAlchemy==2.1
SQLAlchemy==1.0.14
MySQL DB

Cheers




--
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

Re: [sqlalchemy] SAWarning shows not immediately, but after some time of app execution

[TomS.]

my_ids = [] is initialized in the body of static method (on the top of 
it) which is run via Flask's route decorator.


But I can't understand why following code:

my_ids = []
MyModel.id.in_(my_ids)

doesn't raise SAWarning if 'an empty sequence' causes warning.

Cheers


On 07/19/2016 06:14 PM, Антонио Антуан wrote:


Looks like `my_ids` become empty 'after some time of app execution'. 
How do you initialize the variable?



вт, 19 июля 2016 г., 18:51 TomS. >:


Hi,

We have Flask app which uses SQLAlchemy. Weird error started to happen
recently. The difficulty is that we can't reproduce the error (/figure
out conditions causing issue) - maybe someone could help. Any
hints/tips
would be appreciated.

There is a part in the code which constructs IN in SQL:

MyModel.id.in_(my_ids)

For some cases my_ids is an empty list. It works without any problems,
but after some time the same query (using empty list) starts to
raise an
exception:

SAWarning: The IN-predicate on "MyModel.id" was invoked with an empty
sequence. This results in a contradiction, which nonetheless can be
expensive to evaluate.  Consider alternative strategies for improved
performance.

After restarting app, everything works again.

The question is - why this exception is not risen always (although we
tried to run app with empty list directly), but after some time of app
execution (~1 day)?

Details:
Flask==0.10.1
Flask-SQLAlchemy==2.1
SQLAlchemy==1.0.14
MySQL DB

Cheers


--
You received this message because you are subscribed to the Google
Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to sqlalchemy+unsubscr...@googlegroups.com
.
To post to this group, send email to sqlalchemy@googlegroups.com
.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

--

Антон

--
You received this message because you are subscribed to the Google 
Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to sqlalchemy+unsubscr...@googlegroups.com 
.
To post to this group, send email to sqlalchemy@googlegroups.com 
.

Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

Re: [sqlalchemy] SAWarning shows not immediately, but after some time of app execution

[Антонио Антуан]

Looks like `my_ids` become empty 'after some time of app execution'. How do
you initialize the variable?

вт, 19 июля 2016 г., 18:51 TomS. :

> Hi,
>
> We have Flask app which uses SQLAlchemy. Weird error started to happen
> recently. The difficulty is that we can't reproduce the error (/figure
> out conditions causing issue) - maybe someone could help. Any hints/tips
> would be appreciated.
>
> There is a part in the code which constructs IN in SQL:
>
> MyModel.id.in_(my_ids)
>
> For some cases my_ids is an empty list. It works without any problems,
> but after some time the same query (using empty list) starts to raise an
> exception:
>
> SAWarning: The IN-predicate on "MyModel.id" was invoked with an empty
> sequence. This results in a contradiction, which nonetheless can be
> expensive to evaluate.  Consider alternative strategies for improved
> performance.
>
> After restarting app, everything works again.
>
> The question is - why this exception is not risen always (although we
> tried to run app with empty list directly), but after some time of app
> execution (~1 day)?
>
> Details:
> Flask==0.10.1
> Flask-SQLAlchemy==2.1
> SQLAlchemy==1.0.14
> MySQL DB
>
> Cheers
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To post to this group, send email to sqlalchemy@googlegroups.com.
> Visit this group at https://groups.google.com/group/sqlalchemy.
> For more options, visit https://groups.google.com/d/optout.
>
-- 

Антон

-- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

[sqlalchemy] SAWarning shows not immediately, but after some time of app execution

[TomS.]

Hi,

We have Flask app which uses SQLAlchemy. Weird error started to happen 
recently. The difficulty is that we can't reproduce the error (/figure 
out conditions causing issue) - maybe someone could help. Any hints/tips 
would be appreciated.


There is a part in the code which constructs IN in SQL:

MyModel.id.in_(my_ids)

For some cases my_ids is an empty list. It works without any problems, 
but after some time the same query (using empty list) starts to raise an 
exception:


SAWarning: The IN-predicate on "MyModel.id" was invoked with an empty 
sequence. This results in a contradiction, which nonetheless can be 
expensive to evaluate.  Consider alternative strategies for improved 
performance.


After restarting app, everything works again.

The question is - why this exception is not risen always (although we 
tried to run app with empty list directly), but after some time of app 
execution (~1 day)?


Details:
Flask==0.10.1
Flask-SQLAlchemy==2.1
SQLAlchemy==1.0.14
MySQL DB

Cheers


--
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.