I've been thinking that the import operation sounds like it could be viewed as a transformation from a python file to a python file with sentinals (and altered indentation). The node construction would be replaced by nesting of node sentinals. When the transform is done, the file could be loaded like any other file with sentinals.
It's not easy to tell if this would be a simpler approach without actually trying it, though! On Tuesday, November 30, 2021 at 4:53:00 PM UTC-5 Edward K. Ream wrote: > On Tuesday, November 30, 2021 at 9:41:39 AM UTC-6 Edward K. Ream wrote: > >> On Tuesday, November 30, 2021 at 8:21:35 AM UTC-6 [email protected] >> wrote: >> >>> Would it be feasible to use Python's tokenizer? That would eliminate >>> the problem of unusual whitespace, as well as continuation lines, in a >>> perfectly Python-compatible way. >>> >> >> The short answer is: feasible, yes, useful, no. >> > > A slightly longer answer. The front end of Leo's importers are solid. > gen_lines calls scan_line to determine the scan state. For python, the line > ends is "in a state" if the *end *of the line is in a string or > docstring. In those case, the python importer simply adds the line to the > present node. > > The difficulties lie in the back end, not the front end. gen_lines assigns > lines to nodes (creating new nodes as needed), based on a combination of: > > - The indentation and 'kind' fields node p and all its ancestors. > - The indentation of the to-be-assigned line. > > The difficulty is that of an explosion of cases. In this respect (and > others) the python importer resembles a compiler's code generator. > > I think I have discovered a way to drastically reduce the number of "code > generation" cases. Perhaps later today. > > Edward > -- You received this message because you are subscribed to the Google Groups "leo-editor" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/leo-editor/e823494f-227f-4217-836f-4bdf209e06b8n%40googlegroups.com.
