On Mon, Jul 9, 2012 at 8:13 AM, Gregory Szorc <[email protected]> wrote: > On 7/1/12 11:48 PM, Manuel Klimek wrote: > > + lib.clang_tokenize(tu, extent, byref(tokens_memory), > + byref(tokens_count)) > > Strange indent. Hm, after reading further I see that that kind of indent is > quite common. Is there a guideline for how to indent this? > > It appears my editor is indenting 8 spaces when wrapping lines. I'm pretty > sure that's provided by a popular "macro" settings file to handle Python. > I'll change it to 4 spaces, as that seems more conventional. > > > > --- /dev/null > +++ b/bindings/python/clang/enumerations.py > > +TokenKinds = [ > + ('PUNCTUATION', 0), > + ('KEYWORD', 1), > + ('IDENTIFIER', 2), > + ('LITERAL', 3), > + ('COMMENT', 4), > +] > > This doesn't seem to match the comment. Those "TokenKinds" are different > from what I see in clang/Basic/TokenKinds.def. Am I missing something? > > What doesn't match the comment? > > Also, these come from > http://clang.llvm.org/doxygen/group__CINDEX__LEX.html#gaf63e37eee4280e2c039829af24bbc201, > not the low-level tokens from TokenKinds.def.
Oh, wow. Can you add a comment on where these TokenKinds come from. With the overloaded meaning it's really confusing. Thanks! /Manuel > > > On Sat, Jun 30, 2012 at 4:16 AM, Gregory Szorc <[email protected]> > wrote: >> >> Updated patch attached. >> >> * Added Cursor.get_tokens() >> * Refactored TokenGroup out of TranslationUnit to support above >> * Refactored with TranslationUnit.get_* API changes from last patch. >> >> On Fri, Jun 29, 2012 at 12:18 AM, Gregory Szorc <[email protected]> >> wrote: >> > This exposes all of libclang's token functions in the Python bindings. >> > >> > I'm putting the TokenKind enumerations in a new module, >> > clang.enumerations. I plan to eventually move all existing >> > enumerations there so they are all consolidated. And, maybe one day, >> > we can even generate that file automatically by parsing the libclang >> > header files. I know Anders has code around somewhere that does >> > this... >> > >> > --- >> > bindings/python/clang/cindex.py | 177 >> > +++++++++++++++++++-- >> > bindings/python/clang/enumerations.py | 32 ++++ >> > bindings/python/tests/cindex/test_token_kind.py | 43 +++++ >> > bindings/python/tests/cindex/test_tokens.py | 52 ++++++ >> > .../python/tests/cindex/test_translation_unit.py | 24 ++- >> > 5 files changed, 312 insertions(+), 16 deletions(-) >> > create mode 100644 bindings/python/clang/enumerations.py >> > create mode 100644 bindings/python/tests/cindex/test_token_kind.py >> > create mode 100644 bindings/python/tests/cindex/test_tokens.py >> >> _______________________________________________ >> cfe-commits mailing list >> [email protected] >> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits >> > > _______________________________________________ cfe-commits mailing list [email protected] http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
