I've been building a PL/I lexer for Pygments, the Python syntax-
highlighting library used by a lot of tools (and, via a chain of
dependencies described below, by Wikipedia):

    https://github.com/shmuelmetz/pygments-extensions

Some background on why this matters beyond "another syntax
highlighter": MediaWiki's `<syntaxhighlight>` tag -- the thing that
renders colored code blocks on Wikipedia and other wikis -- is powered
by Pygments under the hood, via MediaWiki's SyntaxHighlight extension.
`<syntaxhighlight lang="pli">` doesn't work today because Pygments has
never shipped a PL/I lexer. But PL/I highlighting actually worked on
Wikipedia once, just not through Pygments: SyntaxHighlight used to run
on GeSHi, which did have a PL/I definition, before switching to a
Pygments backend during which PL/I support was lost. So this isn't
"brand new, nobody asked" -- it's restoring a capability that existed
and disappeared. Getting `lang="pli"` working on Wikipedia again is a
multi-stage pipeline (submit to Pygments -> get merged -> Pygments
cuts a release -> Wikimedia picks up that release -> a Phabricator
request to enable the language tag -> Wikimedia deploys it), so this
is a longer-term effort, not something arriving next week -- but the
lexer itself is real and working today, and useful on its own for
anyone using Pygments locally or in other tools.

The same repo also has an ooRexx lexer, for anyone here who cares
about that side of things too. Wikimedia already supports
`lang="rexx"` today (classic Rexx, via Pygments' existing lexer), but
not `lang="oorexx"` for the object-oriented dialect -- so that one's a
new addition rather than a restoration, but the same pipeline, and
`lang="rexx"` already working is a real precedent that the pipeline
above actually succeeds end to end.

What I'm asking for:

  1. Real-world PL/I source to test against. The lexer has been
     validated against synthetic samples and against a real-world
     corpus, but I'd like more, and more varied, sources -- especially
     anything that exercises:
       - preprocessor directives (%IF/%DO/%INCLUDE/%PROCESS and the
         rest of the %-statement family)
       - embedded EXEC SQL, EXEC CICS, or IMS (EXEC DLI) blocks
       - fixed-format source, not just free-format
     If you have PL/I source you can share (or can point me at public
     samples), or you're willing to run the lexer against your own
     code and tell me what it gets wrong, that's exactly the kind of
     feedback that matters most right now -- "here's what's
     mis-highlighted" is more useful to me than a general look at the
     project.

  2. Access to the actual PL/I language standards, if anyone has it.
     The lexer's keyword/attribute/built-in-function vocabulary is
     currently sourced from IBM's current Enterprise PL/I
     documentation, which is authoritative for IBM's compiler but
     isn't the same thing as the language standard. I'd like to
     cross-check against ANSI X3.53-1976 (ISO 6160:1979) and both
     ANSI Subset G editions -- X3.74-1981 and the later, distinct
     X3.74-1987 (ISO 6522:1985 and ISO/IEC 6522:1992) -- none of which
     I've found readily accessible online. If you have a personal
     copy, university library access, or know of an archive, I'd
     appreciate a pointer.

  3. Reviewers/collaborators generally, if this is useful to you.

Happy to answer questions about the approach (it's a Pygments
RegexLexer, state-based, not a full parser) or anything else. Thanks
for reading this far.


-- 
Shmuel (Seymour J.) Metz
http://mason.gmu.edu/~smetz3
עַם יִשְׂרָאֵל חַי
נֵ֣צַח יִשְׂרָאֵ֔ל לֹ֥א יְשַׁקֵּ֖ר



----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [email protected] with the message: INFO IBM-MAIN

Reply via email to