New submission from David Bieber <dbie...@google.com>: # Overview ast.literal_eval supports some non-literals in Python 3. The behavior of this function runs contrary to the documented behavior.
# The Issue The [documentation](https://docs.python.org/3/library/ast.html#ast.literal_eval) says of the function "It is not capable of evaluating arbitrarily complex expressions, for example involving operators or indexing." However, literal_eval is capable of evaluating expressions with certain operators, particular the operators "+" and "-". As has been explained previously, the reason for this is to support "complex" literals such as 2+3j. However, this has unintended consequences which I believe to be indicative of a bug. Examples of the unintended consequences include `ast.literal_eval('1+1') == 2` `ast.literal_eval('2017-10-10') == 1997`. I would expect each of these calls to literal_eval to throw a ValueError, as the input string is not a literal. Instead, literal_eval successfully evaluates the input string, in the latter case giving an unexpected result (since the intent of the string is to represent a 21st century date.) Since issue arose as a [Python Fire issue](https://github.com/google/python-fire/issues/97), where the behavior of Python Fire was unexpected for inputs such as those described above (1+1 and 2017-10-10) only in Python 3. For context, Python Fire is a CLI library which uses literal_eval as part of its command line argument parsing procedure. I think the resolution to this issue is having literal_eval raise a ValueError if the ast of the input represents anything other than a Python literal, as described in the documentation. That is, "The string or node provided may only consist of the following Python literal structures: strings, bytes, numbers, tuples, lists, dicts, sets, booleans, and None." Additional operations, such as the binary operations "+" and "-", unless they explicitly create a complex number, should produce a ValueError. If that resolution is not the direction we take, I also would appreciate knowing if there is another built in approach for determining if a string or ast node represents a literal. # Reproducing The following code snippets produce different behaviors in Python 2 and Python 3. ```python import ast ast.literal_eval('1+1') ``` ```python import ast ast.literal_eval('2017-10-10') ``` # References - The Python Fire issue is here: https://github.com/google/python-fire/issues/97 - Prior discussion of a similar issue: https://bugs.python.org/issue22525 - I think is where complex number support was originally added: https://bugs.python.org/issue4907 - In this thread, https://bugs.python.org/issue24146, one commenter explains that literal_eval's support for "2+3" is an unintentional side effect of supporting complex literals. ---------- messages: 304294 nosy: David Bieber priority: normal severity: normal status: open title: ast.literal_eval supports non-literals in Python 3 versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue31778> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
