Re: Aw: Re: mypy question
On 31/12/23 8:05 am, Chris Angelico wrote: Ah, I think you've hit on the problem there. Consider this: def add_item(stuff: dict[str: str | int]): stuff["spam"] = "ham" stuff["vooom"] = 1_000_000 Yep, that's it exactly. It's not the union itself that's the problem, but the fact that there's a *mutable container* containing that type. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: Aw: Re: mypy question
On Sun, 31 Dec 2023 at 03:38, Thomas Passin via Python-list wrote: > I am not very expert in Python type hints. In working up the example > program I just posted, I got an error message from mypy that remarked > that "list" is invariant, and to try Sequence which is "covariant". I > don't know what that means (and I haven't looked into it yet), but when > I changed from list to Sequence as suggested, mypy stopped complaining. > Ah, I think you've hit on the problem there. Consider this: def add_item(stuff: dict[str: str | int]): stuff["spam"] = "ham" stuff["vooom"] = 1_000_000 Is it valid to pass this function a dict[str: str]? No, because it's going to add an integer into it. Hopefully that helps explain what's going on a bit. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Aw: Re: mypy question
On 12/30/2023 10:08 AM, Karsten Hilbert via Python-list wrote: Dear Thomas, thanks for taking the time to look into my issue. Maybe it helps if I explain what I want (sorry that my web mailer does not respect indentation, I will insert dots). I want a function to run SQL queries: run_queries(conn, queries): ...for q in queries: ..conn.execute(q) I now want to add type hints such that my large codebase can be checked for silly doings. First, queries is to be a list of the SQL to run: run_queries(conn, queries:list): Then, each list entry can be ...a string holding simple, complete SQL (say "SELECT 1") run_queries(conn, queries:list[str]): It occurs to me that you could simplify things if you converted those plain query strings to dicts: 'SELECT 1' --> {'SQL': 'SELECT 1'} I'm fairly sure your database queries don't actually give you strings or dicts, right? You probably get lists (or iterators) of tuples and somewhere you convert them to the arguments you are feeding to run_queries(). At least, that is how the standard Python db adapters work. If you change that conversion step, your arguments to run_queries() will all be lists of dicts, making your code simpler and reducing the complexity of the type hints. or ...a dict holding the SQL and arguments for parameters run_queries(conn, queries:list[dict]): So, taken together: run_queries(conn, queries:list[str|dict]): (yes, this is in Python 3.11/3.12) Now, when it is a list of dicts I want to further constrain the dicts. Each is to contain the keys "SQL" and "args". So the keys are of type str. The values for the keys will be of various types, such that I chose Any as pseudo-type, so that each list entry that is of type dict should be dict[str, Any], hence: queries = [{'SQL': 'SELECT %(value)s', 'args': {'value': 1}}] and run_queries(conn, queries:list[str|dict[str, Any]]): If I now call this function with a simple SQL query: SQL_query = 'SELECT 1' # should be type str ? queries = [SQL_query] # should be type list[str] ? run_queries(conn, queries = queries) and run mypy over that (at least inside my complex codebase) I will get a type mismatch being hinted at. So far I don't grasp at which point my reasoning above is faulty. Karsten -- https://mail.python.org/mailman/listinfo/python-list
Re: Aw: Re: mypy question
On 12/30/2023 10:08 AM, Karsten Hilbert via Python-list wrote: Dear Thomas, thanks for taking the time to look into my issue. Maybe it helps if I explain what I want (sorry that my web mailer does not respect indentation, I will insert dots). I want a function to run SQL queries: run_queries(conn, queries): ...for q in queries: ..conn.execute(q) I now want to add type hints such that my large codebase can be checked for silly doings. First, queries is to be a list of the SQL to run: run_queries(conn, queries:list): Then, each list entry can be ...a string holding simple, complete SQL (say "SELECT 1") run_queries(conn, queries:list[str]): or ...a dict holding the SQL and arguments for parameters run_queries(conn, queries:list[dict]): So, taken together: run_queries(conn, queries:list[str|dict]): (yes, this is in Python 3.11/3.12) Now, when it is a list of dicts I want to further constrain the dicts. Each is to contain the keys "SQL" and "args". So the keys are of type str. The values for the keys will be of various types, such that I chose Any as pseudo-type, so that each list entry that is of type dict should be dict[str, Any], hence: queries = [{'SQL': 'SELECT %(value)s', 'args': {'value': 1}}] and run_queries(conn, queries:list[str|dict[str, Any]]): If I now call this function with a simple SQL query: SQL_query = 'SELECT 1' # should be type str ? queries = [SQL_query] # should be type list[str] ? run_queries(conn, queries = queries) and run mypy over that (at least inside my complex codebase) I will get a type mismatch being hinted at. So far I don't grasp at which point my reasoning above is faulty. Karsten I am not very expert in Python type hints. In working up the example program I just posted, I got an error message from mypy that remarked that "list" is invariant, and to try Sequence which is "covariant". I don't know what that means (and I haven't looked into it yet), but when I changed from list to Sequence as suggested, mypy stopped complaining. Here is the exact error message, and it has a reference you might want to follow up with: c:\temp\python\typing_test.py:16: note: "List" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance c:\temp\python\typing_test.py:16: note: Consider using "Sequence" instead, which is covariant Before that, mypy insisted that str and Union[str, list] were incompatible argument types, which is something you are seeing, too. I suggest that you build up your types as in my example, so that it's very clear what they are and very easy to change them, and use Sequence instead of List (or list). See if that will do the job. -- https://mail.python.org/mailman/listinfo/python-list
Aw: Re: mypy question
Dear Thomas, thanks for taking the time to look into my issue. Maybe it helps if I explain what I want (sorry that my web mailer does not respect indentation, I will insert dots). I want a function to run SQL queries: run_queries(conn, queries): ...for q in queries: ..conn.execute(q) I now want to add type hints such that my large codebase can be checked for silly doings. First, queries is to be a list of the SQL to run: run_queries(conn, queries:list): Then, each list entry can be ...a string holding simple, complete SQL (say "SELECT 1") run_queries(conn, queries:list[str]): or ...a dict holding the SQL and arguments for parameters run_queries(conn, queries:list[dict]): So, taken together: run_queries(conn, queries:list[str|dict]): (yes, this is in Python 3.11/3.12) Now, when it is a list of dicts I want to further constrain the dicts. Each is to contain the keys "SQL" and "args". So the keys are of type str. The values for the keys will be of various types, such that I chose Any as pseudo-type, so that each list entry that is of type dict should be dict[str, Any], hence: queries = [{'SQL': 'SELECT %(value)s', 'args': {'value': 1}}] and run_queries(conn, queries:list[str|dict[str, Any]]): If I now call this function with a simple SQL query: SQL_query = 'SELECT 1' # should be type str ? queries = [SQL_query] # should be type list[str] ? run_queries(conn, queries = queries) and run mypy over that (at least inside my complex codebase) I will get a type mismatch being hinted at. So far I don't grasp at which point my reasoning above is faulty. Karsten -- https://mail.python.org/mailman/listinfo/python-list
Aw: Re: mypy question
Hi Greg, > dict[str, str] is not a subtype of dict[str, str | something_else] > because you can assign a value of type something_else to the latter > but not the former. I understand what you are saying but I do not yet understand why this applies to my situation. I don't have Python at hand currently, so I'll write untested pseudocode: def print_greeting(greeting:int|str): print(greeting) print_greeting('hello') The above snippet should be equivalent to my more complicated code over which mypy complains to the equivalent of "input" is of type "str" but expected type "Union[str,int] I do understand that "str" is formally more narrow than "Union [str,int]" and the type system has every right to not consider them equivalent. However, this seems like a very common use case: "allow passing in either str or int and have type checking pass either input as valid" -- yet mypy doesn't seem to share that idea. Or else there's something I haven't wrapped my head around yet. But what ? Karsten -- https://mail.python.org/mailman/listinfo/python-list