Hi Aaron, Many thanks for your response. I think it is clear how to create the parser transformer. I just followed the transformations already written inside sympy's parser module. Thanks for sharing the documentation for the python tokenizer, as well :).
BW Giorgino On Sat, Nov 13, 2021 at 12:53 AM Aaron Meurer <[email protected]> wrote: > Yes, you will need to write a custom parser transformer that merges colon > tokens into name tokens. This should be straightforward because colon > tokens wouldn't appear in any other context in an expression (only in a > larger block of code, which parse_expr() cannot handle). So you can > unconditionally merge colons in your transformer. > > This is actually something that would be nice to document better. For now, > Oscar's suggestion gives the basic idea of what you need to do. If you need > help with the Python tokenizer, I have written a guide here that goes over > it https://www.asmeurer.com/brown-water-python/ > > Aaron Meurer > > On Thu, Nov 11, 2021 at 3:32 PM Giorgino R <[email protected]> wrote: > >> Hi Oscar, >> >> Many thanks for this, I see. I just went through the documentation and I >> found the transformations. I guess I will have to add this to the standard >> transformations (e.g. (merge_double_colon, ) + standard_transformations) so >> as to work. >> >> BW >> Giorgino >> >> >> >> On Thu, Nov 11, 2021 at 10:22 AM Oscar Gustafsson < >> [email protected]> wrote: >> >>> This behavior is from the Python tokenizer. >>> >>> >>> In [13]: input_code = StringIO('l_::p+3') >>> >>> In [14]: list(generate_tokens(input_code.readline)) >>> >>> Out[14]: >>> >>> [TokenInfo(type=1 (NAME), string='l_', start=(1, 0), end=(1, 2), >>> line='l_::p+3'), >>> >>> TokenInfo(type=54 (OP), string=':', start=(1, 2), end=(1, 3), >>> line='l_::p+3'), >>> >>> TokenInfo(type=54 (OP), string=':', start=(1, 3), end=(1, 4), >>> line='l_::p+3'), >>> >>> TokenInfo(type=1 (NAME), string='p', start=(1, 4), end=(1, 5), >>> line='l_::p+3'), >>> >>> TokenInfo(type=54 (OP), string='+', start=(1, 5), end=(1, 6), >>> line='l_::p+3'), >>> >>> TokenInfo(type=2 (NUMBER), string='3', start=(1, 6), end=(1, 7), >>> line='l_::p+3'), >>> >>> TokenInfo(type=4 (NEWLINE), string='', start=(1, 7), end=(1, 8), >>> line=''), >>> >>> TokenInfo(type=0 (ENDMARKER), string='', start=(2, 0), end=(2, 0), >>> line='')] >>> >>> >>> I guess you can add a transformation that merges NAME, OP=':', OP=':', >>> NAME sequences into a single name token. >>> >>> >>> BR Oscar >>> >>> >>> >>> Den tors 11 nov. 2021 kl 10:43 skrev Giorgino R <[email protected]>: >>> >>>> Hi all, >>>> >>>> I am using `parse_exr` to parse some strings as sympy symbols and then >>>> generate cxx code with `cxxcode`. When I use an expression with double >>>> colon, parse_expr is trying to split it. In contrast, if I define this >>>> expression with Sympy Symbol, I can create cxx code (see below). >>>> >>>> [image: image.png] >>>> >>>> Any idea how I could overwrite this behavior in `parse_expr`? >>>> Many thanks in advance, >>>> Giorgino >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "sympy" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/sympy/CA%2Bc1-%2B-iMn8BiYWU3T23yb2_Yj3ASTW3LDiy6DWr149TJRGJkg%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/sympy/CA%2Bc1-%2B-iMn8BiYWU3T23yb2_Yj3ASTW3LDiy6DWr149TJRGJkg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "sympy" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/sympy/CAFjzj-%2Bh7bXZMpp0n4OKFK%2B9Jegj9J%2BX9xhC7Lo7YomrDZ1mDg%40mail.gmail.com >>> <https://groups.google.com/d/msgid/sympy/CAFjzj-%2Bh7bXZMpp0n4OKFK%2B9Jegj9J%2BX9xhC7Lo7YomrDZ1mDg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "sympy" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/sympy/CA%2Bc1-%2B_4nJb%3D3POZ5L5X0LEndj0YHyndJTCmc0FkPBLH29h6ig%40mail.gmail.com >> <https://groups.google.com/d/msgid/sympy/CA%2Bc1-%2B_4nJb%3D3POZ5L5X0LEndj0YHyndJTCmc0FkPBLH29h6ig%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sympy/CAKgW%3D6JoAcMDO2JeLNqv-m0XWNODvz%3DWTUH%3Dg8sEVfRv4brEqA%40mail.gmail.com > <https://groups.google.com/d/msgid/sympy/CAKgW%3D6JoAcMDO2JeLNqv-m0XWNODvz%3DWTUH%3Dg8sEVfRv4brEqA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CA%2Bc1-%2B9-OPrk%2BTF_L4Wv3i-8zx1ewTLTssvkNd7TvE55xZ6A1A%40mail.gmail.com.
