[Python-ideas] Re: Operator as first class citizens -- like in scala -- or yet another new operator?
On Wed, Jun 12, 2019 at 7:56 PM Yanghao Hua wrote: > On Wed, Jun 12, 2019 at 11:27 PM Chris Angelico wrote: > > Yes, you would need some sort of syntactic parser. There are a couple > > of ways to go about it. One is to make use of Python's own tools, like > > the ast module; the other is to mandate that your specific syntax be > > "tidier" than the rest of the Python code, which would permit you to > > use a more naive and simplistic parser (even a regex). > > Yep ... I just tried to use MacroPy3 to handle this but failed, seems > MacroPy3 does expect valid Python syntax in the first place for > anything else to happen. I also tried the raw ast module which seems > also the case. So it means if one want to use python ast module to > parse user input, the user input has to be valid python syntax (e.g. > no <==) at the first place. Seems this is a chicken-egg problem. > As I mentioned very early on in this seemingly-never-ending discussion, there is a way to do things similar to what you want to do, by doing transformations on the source code prior to execution (prior to constructing an AST). Here's the link to the set of examples which I have done as demonstrations of this technique: https://github.com/aroberge/experimental/tree/master/experimental/transformers André Roberge > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/Z3FVILBIN7NZGDZP2ROUMEOPT3SSKWWB/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/XQAU4A4QTDBD5FFEGRTM64SOIKVWIUAJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Operator as first class citizens -- like in scala -- or yet another new operator?
Barry the reason I use python and don't parse syntax directly as I want to have python as meta programming environment for my DSLs. I can mostly work within the python syntax (with some pretty heavy metaclasses) I rarely have to touch the AST. Their only two places where I ever have to touch the AST, in assignment statements and in control flow. Theres no easy way around needing to rewrite control flow but an assignment operator would drastically decrease the amount of AST manipulation I do. In class bodies it is easy to redefine what assignment means, in every other context its very annoying, I don't see why that must be the case. Caleb On Wed, Jun 12, 2019 at 2:28 PM Chris Angelico wrote: > On Thu, Jun 13, 2019 at 6:51 AM Yanghao Hua wrote: > > > > On Wed, Jun 12, 2019 at 9:39 AM Chris Angelico wrote: > > > If Python is really THAT close, then devise two syntaxes: an abstract > > > syntax for your actual source code, and then a concrete syntax that > > > can be executed. It's okay for things to be a little bit ugly (like > > > "signal[:] = 42") in the concrete form, because you won't actually be > > > editing that. Then your program just has to transform one into the > > > other, and then run the program. > > > > Thought about that too but as you can imagine, you can write: > > > > x <== 3 # or > > x \ > > <== 3 # or > > x \ > > \ > > ... > > \ <== 3 # This is crazy but valid python syntax! > > # more crazy ones are skipped ... > > > > so this is not a simple text replacement problem, eventually you end > > up writing a python parser? Or a HDL parser. > > Yes, you would need some sort of syntactic parser. There are a couple > of ways to go about it. One is to make use of Python's own tools, like > the ast module; the other is to mandate that your specific syntax be > "tidier" than the rest of the Python code, which would permit you to > use a more naive and simplistic parser (even a regex). > > ChrisA > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/BCGYTPPODZCFLFSETZOXPAQFKPGJ6ZVT/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/MGOCHF4YT37637BBS7U6PSCWKFHXP4IN/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Operator as first class citizens -- like in scala -- or yet another new operator?
On Thu, Jun 13, 2019 at 12:52 AM Yanghao Hua wrote: > > On Wed, Jun 12, 2019 at 11:27 PM Chris Angelico wrote: > > Yes, you would need some sort of syntactic parser. There are a couple > > of ways to go about it. One is to make use of Python's own tools, like > > the ast module; the other is to mandate that your specific syntax be > > "tidier" than the rest of the Python code, which would permit you to > > use a more naive and simplistic parser (even a regex). > > Yep ... I just tried to use MacroPy3 to handle this but failed, seems > MacroPy3 does expect valid Python syntax in the first place for > anything else to happen. I also tried the raw ast module which seems > also the case. So it means if one want to use python ast module to > parse user input, the user input has to be valid python syntax (e.g. > no <==) at the first place. Seems this is a chicken-egg problem. Attaching the trace: Traceback (most recent call last): File "tt.py", line 34, in main() File "tt.py", line 8, in main tree = ast.parse(source.read()) File "/usr/lib/python3.6/ast.py", line 35, in parse return compile(source, filename, mode, PyCF_ONLY_AST) File "", line 1 x <== 3 ^ SyntaxError: invalid syntax ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/RTAAWQZ74VTP244FBK5K2WFNJTFTDJSW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Operator as first class citizens -- like in scala -- or yet another new operator?
On Wed, Jun 12, 2019 at 11:27 PM Chris Angelico wrote: > Yes, you would need some sort of syntactic parser. There are a couple > of ways to go about it. One is to make use of Python's own tools, like > the ast module; the other is to mandate that your specific syntax be > "tidier" than the rest of the Python code, which would permit you to > use a more naive and simplistic parser (even a regex). Yep ... I just tried to use MacroPy3 to handle this but failed, seems MacroPy3 does expect valid Python syntax in the first place for anything else to happen. I also tried the raw ast module which seems also the case. So it means if one want to use python ast module to parse user input, the user input has to be valid python syntax (e.g. no <==) at the first place. Seems this is a chicken-egg problem. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/Z3FVILBIN7NZGDZP2ROUMEOPT3SSKWWB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Operator as first class citizens -- like in scala -- or yet another new operator?
On Thu, Jun 13, 2019 at 6:51 AM Yanghao Hua wrote: > > On Wed, Jun 12, 2019 at 9:39 AM Chris Angelico wrote: > > If Python is really THAT close, then devise two syntaxes: an abstract > > syntax for your actual source code, and then a concrete syntax that > > can be executed. It's okay for things to be a little bit ugly (like > > "signal[:] = 42") in the concrete form, because you won't actually be > > editing that. Then your program just has to transform one into the > > other, and then run the program. > > Thought about that too but as you can imagine, you can write: > > x <== 3 # or > x \ > <== 3 # or > x \ > \ > ... > \ <== 3 # This is crazy but valid python syntax! > # more crazy ones are skipped ... > > so this is not a simple text replacement problem, eventually you end > up writing a python parser? Or a HDL parser. Yes, you would need some sort of syntactic parser. There are a couple of ways to go about it. One is to make use of Python's own tools, like the ast module; the other is to mandate that your specific syntax be "tidier" than the rest of the Python code, which would permit you to use a more naive and simplistic parser (even a regex). ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/BCGYTPPODZCFLFSETZOXPAQFKPGJ6ZVT/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Operator as first class citizens -- like in scala -- or yet another new operator?
On Wed, Jun 12, 2019 at 10:10 AM Greg Ewing wrote: > > Yanghao Hua wrote: > > You are absolutely right in that writing a new parser > > might not be a bad idea, but it is just s close for enabling > > Python to be used as the DSL language. > > You seem to be arguing for this as an enabler for using Python > for DSLs in general, but you're really only talking about one > particular DSL, i.e. your HDL. > > Do you have any evidence that this a single missing piece that > will be of great benefit to other DSLs? Or will the next person > who wants a DSL be asking for yet another piece of new syntax > to support their particular application, etc? I believe the "<==" operator has more use than just for HDL design (e.g. for replacing descriptors as described earlier ... but it also has its cons, e.g. it is no longer transparent for the normal assignment "="). This might help other DSLs but definitely not making Python feature complete for all DSLs. Just like @ operator pushed into Python, once there are proven use cases, I believe Python will keep adding more? This feels painful but ... what can we do? Waiting more operators to come or try to allow users to define new operators in python itself? ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/KW32IXFQC3DJZWGE5TVZF72TUFCIXQOM/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Operator as first class citizens -- like in scala -- or yet another new operator?
On Wed, Jun 12, 2019 at 9:39 AM Chris Angelico wrote: > If Python is really THAT close, then devise two syntaxes: an abstract > syntax for your actual source code, and then a concrete syntax that > can be executed. It's okay for things to be a little bit ugly (like > "signal[:] = 42") in the concrete form, because you won't actually be > editing that. Then your program just has to transform one into the > other, and then run the program. Thought about that too but as you can imagine, you can write: x <== 3 # or x \ <== 3 # or x \ \ ... \ <== 3 # This is crazy but valid python syntax! # more crazy ones are skipped ... so this is not a simple text replacement problem, eventually you end up writing a python parser? Or a HDL parser. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/C6RQCO4RCJXFT2F2K2TDTR3W2IWAHGS3/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Operator as first class citizens -- like in scala -- or yet another new operator?
Yanghao Hua wrote: You are absolutely right in that writing a new parser might not be a bad idea, but it is just s close for enabling Python to be used as the DSL language. You seem to be arguing for this as an enabler for using Python for DSLs in general, but you're really only talking about one particular DSL, i.e. your HDL. Do you have any evidence that this a single missing piece that will be of great benefit to other DSLs? Or will the next person who wants a DSL be asking for yet another piece of new syntax to support their particular application, etc? -- Greg ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/KRIUO6UDXLWW3FGQGGYJI6CLPWZ3JUEJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Operator as first class citizens -- like in scala -- or yet another new operator?
On Wed, Jun 12, 2019 at 5:26 PM Yanghao Hua wrote: > > On Tue, Jun 11, 2019 at 11:35 PM Barry Scott wrote: > > Some times a DSL is usable within the python syntax and that is great. > > I have use python for a number of DSL's. > > > > But when the DSL is beyond what python can help with directly I'm wondering > > why you do not parse the DSL with python and execute the results. > > > > In that way you can have any semantics that you want from any syntax that > > you > > wish to have. However you do not need the python language to be changed at > > all. > > > > And it must be clear that you are making little to no progress on > > convincing people > > that changing python is a good idea. > > > > Barry > > Hi Barry, realized that. Please do allow me sometime for formalize > everything. You are absolutely right in that writing a new parser > might not be a bad idea, but it is just s close for enabling > Python to be used as the DSL language. We do not even need a DSL for > HDL in this case, as Python can co-simulate with any HDL simulator > that supports VPI/DPI/etc., but writing a full featured HDL is really > just a few hundred of lines of code which enables you to throw away > the entire HDL simulator (hundreds of thousands of lines, plus another > hundreds of lines of glue code in between python and HDL simulator. If Python is really THAT close, then devise two syntaxes: an abstract syntax for your actual source code, and then a concrete syntax that can be executed. It's okay for things to be a little bit ugly (like "signal[:] = 42") in the concrete form, because you won't actually be editing that. Then your program just has to transform one into the other, and then run the program. ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/FZF7UJLKUOLRVUH6EBHATFQAGKH4V7SS/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Operator as first class citizens -- like in scala -- or yet another new operator?
On Tue, Jun 11, 2019 at 11:35 PM Barry Scott wrote: > Some times a DSL is usable within the python syntax and that is great. > I have use python for a number of DSL's. > > But when the DSL is beyond what python can help with directly I'm wondering > why you do not parse the DSL with python and execute the results. > > In that way you can have any semantics that you want from any syntax that you > wish to have. However you do not need the python language to be changed at > all. > > And it must be clear that you are making little to no progress on convincing > people > that changing python is a good idea. > > Barry Hi Barry, realized that. Please do allow me sometime for formalize everything. You are absolutely right in that writing a new parser might not be a bad idea, but it is just s close for enabling Python to be used as the DSL language. We do not even need a DSL for HDL in this case, as Python can co-simulate with any HDL simulator that supports VPI/DPI/etc., but writing a full featured HDL is really just a few hundred of lines of code which enables you to throw away the entire HDL simulator (hundreds of thousands of lines, plus another hundreds of lines of glue code in between python and HDL simulator. Develop a dedicated parser eventually is NOT saving much more effort than using existing ones. And more importantly is the integration of Python and the DSL, as we want to use Python not only the write hardware, but write test for hardware which can be far more complex than the hardware itself. And python's ability to allow develop new test scenarios fast is the key here. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/TBZOIXY6GT53DVS2KSIG2CLVHWK3TSZK/ Code of Conduct: http://python.org/psf/codeofconduct/
