* Index.parse() and Index.read() now raise a TanslationUnitLoadException
instead of returning None if a TranslationUnit could not be
instantiated. This is backwards incompatible.
* Ability to save TranslationUnits via TranslationUnit.save().
* TranslationUnit now holds onto Index instance that created. This means
the Index can't be GC'd until the TranslationUnit is itself GC'd,
making memory management thoughtless.
* Richer TranslationUnit.__init__ support. e.g. can now create a
TranslationUnit without explicitly creating an Index.
* Implemented Index.read() through TranslationUnit.__init__ and
documented API as deprecated.
* Don't use [] as a default argument value, as the initial value used is
reused for the duration of the program.
---
bindings/python/clang/cindex.py | 193 +++++++++++++++++---
.../python/tests/cindex/test_translation_unit.py | 75 ++++++++-
bindings/python/tests/cindex/util.py | 4 +-
3 files changed, 246 insertions(+), 26 deletions(-)
----
I realize this patch may be a little big. I tried keeping it split
locally, but management of all the patches was getting to be quite
cumbersome. I gave up. I apologize for my laziness.
The most controversial part of this patch will be the
backwards-incompatible change to Index.parse() and Index.read(), which
now raise an exception instead of returning None. I feel the API is
more pleasant because (and I have no empirical evidence to back this
up) that the overwhelming majority of consumers assume a TU will be
properly created when they try to create one. These consumers are
currently penalized because [good programmers] will add the necessary
"if tu is not None" check every time they try to obtain a TU. This
patch does away with that penalty. Of course, the exception may still
be handled explicitly if so chosen and this will result in roughly the
same number of lines for error checking. But, I have a hunch most
consumers won't want to deal with it and will have the exception
bubble up.
I held off adding exception throwing to TranslationUnit.reparse(). I
can certainly add that if we want things to be consistent everywhere.
Or, I can revert the exception throwing altogether. Or, maybe I could
make the new TranslationUnit.__init__ raise and have the old Index
APIs swallow the exception and keep returning None. Choices.
I'd eventually like to roll Index.parse's main implementation into
TranslationUnit.__init__, effectively deprecating Index.parse. I've
already done this with Index.read() because it was trivial. I held off
because I wanted to get blessing first. In the long term I'd like to
move away from APIs on Index and towards the use of constructors on
other objects. If Index had more direct APIs, I could be convinced
otherwise. But, as it stands, it just feels that Index is unnecessary
baggage in Python land. If we can make its existence invisible, I
think that's a net win. This is why I've marked Index.read() as
deprecated. I can certainly remove that comment if people don't agree
with me.
diff --git a/bindings/python/clang/cindex.py b/bindings/python/clang/cindex.py
index 6f0d25f..0835c54 100644
--- a/bindings/python/clang/cindex.py
+++ b/bindings/python/clang/cindex.py
@@ -81,16 +81,62 @@ def get_cindex_library():
# ctypes doesn't implicitly convert c_void_p to the appropriate wrapper
# object. This is a problem, because it means that from_parameter will see an
# integer and pass the wrong value on platforms where int != void*. Work around
# this by marshalling object arguments as void**.
c_object_p = POINTER(c_void_p)
lib = get_cindex_library()
+### Exception Classes ###
+
+class TranslationUnitLoadError(Exception):
+ """Represents an error that occurred when loading a TranslationUnit.
+
+ This is raised in the case where a TranslationUnit could not be
+ instantiated due to failure in the libclang library.
+
+ Unfortunately, the libclang library doesn't expose any additional error
+ information in this scenario.
+ """
+ pass
+
+class TranslationUnitSaveError(Exception):
+ """Represents an error that occurred when saving a TranslationUnit.
+
+ Each error has associated with it an enumerated value, accessible under
+ e.save_error. Consumers can compare the value with one of the ERROR_
+ constants in this class.
+ """
+
+ # No error occurred.
+ ERROR_OK = 0
+
+ # Indicates that an unknown error occurred. This typically indicates that
+ # I/O failed during save.
+ ERROR_UNKNOWN = 1
+
+ # Indicates that errors during translation prevented saving. The errors
+ # should be available via the TranslationUnit's diagnostics.
+ ERROR_TRANSLATION_ERRORS = 2
+
+ # Indicates that the translation unit was somehow invalid.
+ ERROR_INVALID_TU = 3
+
+ def __init__(self, enumeration, message):
+ assert isinstance(enumeration, int)
+
+ if enumeration < 1 or enumeration > 3:
+ raise Exception("Encountered undefined TranslationUnit save error "
+ "constant: %d. Please file a bug to have this "
+ "value supported." % enumeration)
+
+ self.save_error = enumeration
+ Exception.__init__(self, message)
+
### Structures and Utility Classes ###
class _CXString(Structure):
"""Helper for transforming CXString results."""
_fields_ = [("spelling", c_char_p), ("free", c_int)]
def __del__(self):
@@ -1576,33 +1622,41 @@ class Index(ClangObject):
excludeDecls -- Exclude local declarations from translation units.
"""
return Index(Index_create(excludeDecls, 0))
def __del__(self):
Index_dispose(self)
def read(self, path):
- """Load the translation unit from the given AST file."""
- ptr = TranslationUnit_read(self, path)
- if ptr:
- return TranslationUnit(ptr)
- return None
+ """Load the translation unit from the given AST file.
- def parse(self, path, args = [], unsaved_files = [], options = 0):
+ DEPRECATED: Use TranslationUnit's constructor instead.
"""
- Load the translation unit from the given source code file by running
+ return TranslationUnit(filename=path, index=self)
+
+ def parse(self, path, args=None, unsaved_files=None, options = 0):
+ """Load the translation unit from the given source code file by running
clang and generating the AST before loading. Additional command line
parameters can be passed to clang via the args parameter.
In-memory contents for files can be provided by passing a list of pairs
to as unsaved_files, the first item should be the filenames to be mapped
and the second should be the contents to be substituted for the
file. The contents may be passed as strings or file objects.
+
+ If an error was encountered during parsing, a TranslationUnitLoadError
+ will be raised.
"""
+ if args is None:
+ args = []
+
+ if unsaved_files is None:
+ unsaved_files = []
+
arg_array = 0
if len(args):
arg_array = (c_char_p * len(args))(* args)
unsaved_files_array = 0
if len(unsaved_files):
unsaved_files_array = (_CXUnsavedFile * len(unsaved_files))()
for i,(name,value) in enumerate(unsaved_files):
if not isinstance(value, str):
@@ -1613,29 +1667,78 @@ class Index(ClangObject):
if not isinstance(value, str):
raise TypeError,'Unexpected unsaved file contents.'
unsaved_files_array[i].name = name
unsaved_files_array[i].contents = value
unsaved_files_array[i].length = len(value)
ptr = TranslationUnit_parse(self, path, arg_array, len(args),
unsaved_files_array, len(unsaved_files),
options)
- if ptr:
- return TranslationUnit(ptr)
- return None
+ if ptr is None:
+ raise TranslationUnitLoadError("Error parsing translation unit.")
+ return TranslationUnit(ptr, index=self)
class TranslationUnit(ClangObject):
- """
- The TranslationUnit class represents a source code translation unit and
- provides read-only access to its top-level declarations.
+ """Represents a source code translation unit.
+
+ This is one of the main types in the API. Any time you wish to interact
+ with Clang's representation of a source file, you typically start with a
+ translation unit.
"""
- def __init__(self, ptr):
- ClangObject.__init__(self, ptr)
+ def __init__(self, ptr=None, filename=None, index=None):
+ """Create a TranslationUnit instance.
+
+ Instances can be created in the following ways:
+
+ * By passing a pointer to a c_object_p instance via ptr. This is
+ an internal mechanism and should not be used outside of this
+ module.
+ * By passing a filename of a previously-saved TranslationUnit
+ instance. This file would have been produced via -emit-ast or by
+ saving the TranslationUnit using the API.
+
+ If arguments specifying multiple sources are defined, behavior is
+ undefined.
+
+ FIXME: Index.parse() logic should be moved to this constructor.
+
+ Arguments:
+
+ ptr -- c_object_p instance returned from a libclang call.
+ filename -- String path to file of previously saved TranslationUnit to
+ load.
+ index -- Index instance associated with this TranslationUnit. It must
+ be provided if the TranslationUnit is being created from a
+ c_object_p. If creating from another source and it isn't provided, a
+ new Index will be created automatically.
+ """
+
+ if ptr is not None:
+ assert isinstance(index, Index)
+ ClangObject.__init__(self, ptr)
+ self._index = index
+ return
+
+ if filename is not None:
+ if index is not None:
+ assert isinstance(index, Index)
+ else:
+ index = Index.create()
+
+ ptr = TranslationUnit_read(index, filename)
+ if ptr is None:
+ raise TranslationUnitLoadError(filename)
+
+ ClangObject.__init__(self, ptr)
+ self._index = index
+ return
+
+ raise Exception("Invalid arguments to create TranslationUnit.")
def __del__(self):
TranslationUnit_dispose(self)
@property
def cursor(self):
"""Retrieve the cursor that represents the given translation unit."""
return TranslationUnit_cursor(self)
@@ -1680,25 +1783,28 @@ class TranslationUnit(ClangObject):
def __getitem__(self, key):
diag = _clang_getDiagnostic(self.tu, key)
if not diag:
raise IndexError
return Diagnostic(diag)
return DiagIterator(self)
- def reparse(self, unsaved_files = [], options = 0):
+ def reparse(self, unsaved_files=None, options=0):
"""
Reparse an already parsed translation unit.
In-memory contents for files can be provided by passing a list of pairs
as unsaved_files, the first items should be the filenames to be mapped
and the second should be the contents to be substituted for the
file. The contents may be passed as strings or file objects.
"""
+ if unsaved_files is None:
+ unsaved_files = []
+
unsaved_files_array = 0
if len(unsaved_files):
unsaved_files_array = (_CXUnsavedFile * len(unsaved_files))()
for i,(name,value) in enumerate(unsaved_files):
if not isinstance(value, str):
# FIXME: It would be great to support an efficient version
# of this, one day.
value = value.read()
@@ -1706,25 +1812,50 @@ class TranslationUnit(ClangObject):
if not isinstance(value, str):
raise TypeError,'Unexpected unsaved file contents.'
unsaved_files_array[i].name = name
unsaved_files_array[i].contents = value
unsaved_files_array[i].length = len(value)
ptr = TranslationUnit_reparse(self, len(unsaved_files),
unsaved_files_array,
options)
- def codeComplete(self, path, line, column, unsaved_files = [], options = 0):
+
+ def save(self, filename):
+ """Saves the TranslationUnit to a file.
+
+ This is equivalent to passing -emit-ast to the clang frontend. The
+ saved file can be loaded back into a TranslationUnit. Or, if it
+ corresponds to a header, it can be used as a pre-compiled header file.
+
+ If an error occurs while saving, a TranslationUnitSaveError is raised.
+ If the error was TranslationUnitSaveError.ERROR_INVALID_TU, this means
+ the constructed TranslationUnit was not valid at time of save. In this
+ case, the reason(s) why should be available via
+ TranslationUnit.diagnostics().
+
+ filename -- The path to save the translation unit to.
+ """
+ options = TranslationUnit_defaultSaveOptions(self)
+ result = int(TranslationUnit_save(self, filename, options))
+ if result != 0:
+ raise TranslationUnitSaveError(result,
+ 'Error saving TranslationUnit.')
+
+ def codeComplete(self, path, line, column, unsaved_files=[], options=0):
"""
Code complete in this translation unit.
In-memory contents for files can be provided by passing a list of pairs
as unsaved_files, the first items should be the filenames to be mapped
and the second should be the contents to be substituted for the
file. The contents may be passed as strings or file objects.
"""
+ if unsaved_files is None:
+ unsaved_files = []
+
unsaved_files_array = 0
if len(unsaved_files):
unsaved_files_array = (_CXUnsavedFile * len(unsaved_files))()
for i,(name,value) in enumerate(unsaved_files):
if not isinstance(value, str):
# FIXME: It would be great to support an efficient version
# of this, one day.
value = value.read()
@@ -2063,16 +2194,24 @@ TranslationUnit_includes_callback = CFUNCTYPE(None,
c_object_p,
POINTER(SourceLocation),
c_uint, py_object)
TranslationUnit_includes = lib.clang_getInclusions
TranslationUnit_includes.argtypes = [TranslationUnit,
TranslationUnit_includes_callback,
py_object]
+TranslationUnit_defaultSaveOptions = lib.clang_defaultSaveOptions
+TranslationUnit_defaultSaveOptions.argtypes = [TranslationUnit]
+TranslationUnit_defaultSaveOptions.restype = c_uint
+
+TranslationUnit_save = lib.clang_saveTranslationUnit
+TranslationUnit_save.argtypes = [TranslationUnit, c_char_p, c_uint]
+TranslationUnit_save.restype = c_int
+
# File Functions
File_getFile = lib.clang_getFile
File_getFile.argtypes = [TranslationUnit, c_char_p]
File_getFile.restype = c_object_p
File_name = lib.clang_getFileName
File_name.argtypes = [File]
File_name.restype = _CXString
@@ -2114,13 +2253,23 @@ _clang_getCompletionAvailability = lib.clang_getCompletionAvailability
_clang_getCompletionAvailability.argtypes = [c_void_p]
_clang_getCompletionAvailability.restype = c_int
_clang_getCompletionPriority = lib.clang_getCompletionPriority
_clang_getCompletionPriority.argtypes = [c_void_p]
_clang_getCompletionPriority.restype = c_int
-###
-
-__all__ = ['Index', 'TranslationUnit', 'Cursor', 'CursorKind', 'Type', 'TypeKind',
- 'Diagnostic', 'FixIt', 'CodeCompletionResults', 'SourceRange',
- 'SourceLocation', 'File']
+__all__ = [
+ 'CodeCompletionResults',
+ 'CursorKind',
+ 'Cursor',
+ 'Diagnostic',
+ 'File',
+ 'FixIt',
+ 'Index',
+ 'SourceLocation',
+ 'SourceRange',
+ 'TranslationUnitLoadError',
+ 'TranslationUnit',
+ 'TypeKind',
+ 'Type',
+]
diff --git a/bindings/python/tests/cindex/test_translation_unit.py b/bindings/python/tests/cindex/test_translation_unit.py
index 2e65d95..579dba0 100644
--- a/bindings/python/tests/cindex/test_translation_unit.py
+++ b/bindings/python/tests/cindex/test_translation_unit.py
@@ -1,9 +1,15 @@
-from clang.cindex import *
+from clang.cindex import CursorKind
+from clang.cindex import Cursor
+from clang.cindex import Index
+from clang.cindex import TranslationUnitSaveError
+from clang.cindex import TranslationUnit
+from .util import get_cursor
+from .util import get_tu
import os
kInputsDir = os.path.join(os.path.dirname(__file__), 'INPUTS')
def test_spelling():
path = os.path.join(kInputsDir, 'hello.cpp')
index = Index.create()
tu = index.parse(path)
@@ -77,8 +83,75 @@ def test_includes():
h2 = os.path.join(kInputsDir, "header2.h")
h3 = os.path.join(kInputsDir, "header3.h")
inc = [(src, h1), (h1, h3), (src, h2), (h2, h3)]
index = Index.create()
tu = index.parse(src)
for i in zip(inc, tu.get_includes()):
assert eq(i[0], i[1])
+
+def save_tu(tu):
+ """Convenience API to save a TranslationUnit to a file.
+
+ Returns the filename it was saved to.
+ """
+
+ # FIXME Generate a temp file path using system APIs.
+ base = 'TEMP_FOR_TRANSLATIONUNIT_SAVE.c'
+ path = os.path.join(kInputsDir, base)
+
+ # Just in case.
+ if os.path.exists(path):
+ os.unlink(path)
+
+ tu.save(path)
+
+ return path
+
+def test_save():
+ """Ensure TranslationUnit.save() works."""
+
+ tu = get_tu('int foo();')
+
+ path = save_tu(tu)
+ assert os.path.exists(path)
+ assert os.path.getsize(path) > 0
+ os.unlink(path)
+
+def test_save_translation_errors():
+ """Ensures that saving an invalid TranslationUnit raises properly."""
+
+ tu = get_tu('fd23f23 < f2g3h')
+ assert len(tu.diagnostics) > 0
+
+ path = None
+
+ try:
+ path = save_tu(tu)
+ assert False
+ except TranslationUnitSaveError as ex:
+ expected = TranslationUnitSaveError.ERROR_TRANSLATION_ERRORS
+ assert ex.save_error == expected
+
+ if path is not None:
+ assert not os.path.exists(path)
+
+def test_load():
+ """Ensure TranslationUnits can be constructed from saved files."""
+
+ tu = get_tu('int foo();')
+ assert len(tu.diagnostics) == 0
+ path = save_tu(tu)
+
+ assert os.path.exists(path)
+ assert os.path.getsize(path) > 0
+
+ tu2 = TranslationUnit(filename=path)
+ assert len(tu2.diagnostics) == 0
+
+ foo = get_cursor(tu2, 'foo')
+ assert foo is not None
+
+ # Just in case there is an open file descriptor somewhere.
+ del tu2
+
+ os.unlink(path)
diff --git a/bindings/python/tests/cindex/util.py b/bindings/python/tests/cindex/util.py
index 388b269..e911ba4 100644
--- a/bindings/python/tests/cindex/util.py
+++ b/bindings/python/tests/cindex/util.py
@@ -22,19 +22,17 @@ def get_tu(source, lang='c', all_warnings=False):
elif lang != 'c':
raise Exception('Unknown language: %s' % lang)
args = []
if all_warnings:
args = ['-Wall', '-Wextra']
index = Index.create()
- tu = index.parse(name, args=args, unsaved_files=[(name, source)])
- assert tu is not None
- return tu
+ return index.parse(name, args=args, unsaved_files=[(name, source)])
def get_cursor(source, spelling):
"""Obtain a cursor from a source object.
This provides a convenient search mechanism to find a cursor with specific
spelling within a source. The first argument can be either a
TranslationUnit or Cursor instance.
_______________________________________________
cfe-commits mailing list
[email protected]
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
