This looks good and, btw, is totally awesome! -Argiris
On Jul 2, 2009, at 10:08 AM, Douglas Gregor wrote: > Author: dgregor > Date: Thu Jul 2 12:08:52 2009 > New Revision: 74704 > > URL: http://llvm.org/viewvc/llvm-project?rev=74704&view=rev > Log: > Add support for retrieving the Doxygen comment associated with a given > declaration in the AST. > > The new ASTContext::getCommentForDecl function searches for a comment > that is attached to the given declaration, and returns that comment, > which may be composed of several comment blocks. > > Comments are always available in an AST. However, to avoid harming > performance, we don't actually parse the comments. Rather, we keep the > source ranges of all of the comments within a large, sorted vector, > then lazily extract comments via a binary search in that vector only > when needed (which never occurs in a "normal" compile). > > Comments are written to a precompiled header/AST file as a blob of > source ranges. That blob is only lazily loaded when one requests a > comment for a declaration (this never occurs in a "normal" compile). > > The indexer testbed now supports comment extraction. When the > -point-at location points to a declaration with a Doxygen-style > comment, the indexer testbed prints the associated comment > block(s). See test/Index/comments.c for an example. > > Some notes: > - We don't actually attempt to parse the comment blocks themselves, > beyond identifying them as Doxygen comment blocks to associate them > with a declaration. > - We won't find comment blocks that aren't adjacent to the > declaration, because we start our search based on the location of > the declaration. > - We don't go through the necessary hops to find, for example, > whether some redeclaration of a declaration has comments when our > current declaration does not. Similarly, we don't attempt to > associate a \param Foo marker in a function body comment with the > parameter named Foo (although that is certainly possible). > - Verification of my "no performance impact" claims is still "to be > done". > > > Added: > cfe/trunk/test/Index/comments.c (with props) > Modified: > cfe/trunk/include/clang/AST/ASTContext.h > cfe/trunk/include/clang/AST/ExternalASTSource.h > cfe/trunk/include/clang/Frontend/PCHBitCodes.h > cfe/trunk/include/clang/Frontend/PCHReader.h > cfe/trunk/include/clang/Frontend/PCHWriter.h > cfe/trunk/include/clang/Lex/Preprocessor.h > cfe/trunk/include/clang/Parse/Action.h > cfe/trunk/include/clang/Parse/Parser.h > cfe/trunk/lib/AST/ASTContext.cpp > cfe/trunk/lib/Frontend/PCHReader.cpp > cfe/trunk/lib/Frontend/PCHWriter.cpp > cfe/trunk/lib/Lex/Lexer.cpp > cfe/trunk/lib/Lex/Pragma.cpp > cfe/trunk/lib/Lex/Preprocessor.cpp > cfe/trunk/lib/Parse/Parser.cpp > cfe/trunk/lib/Sema/Sema.cpp > cfe/trunk/lib/Sema/Sema.h > cfe/trunk/test/CMakeLists.txt > cfe/trunk/tools/index-test/index-test.cpp > > Modified: cfe/trunk/include/clang/AST/ASTContext.h > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/AST/ASTContext.h?rev=74704&r1=74703&r2=74704&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- cfe/trunk/include/clang/AST/ASTContext.h (original) > +++ cfe/trunk/include/clang/AST/ASTContext.h Thu Jul 2 12:08:52 2009 > @@ -141,10 +141,19 @@ > /// this ASTContext object. > LangOptions LangOpts; > > + /// \brief Whether we have already loaded comment source ranges > from an > + /// external source. > + bool LoadedExternalComments; > + > /// MallocAlloc/BumpAlloc - The allocator objects used to create > AST objects. > bool FreeMemory; > llvm::MallocAllocator MallocAlloc; > llvm::BumpPtrAllocator BumpAlloc; > + > + /// \brief Mapping from declarations to their comments, once we > have > + /// already looked up the comment associated with a given > declaration. > + llvm::DenseMap<const Decl *, std::string> DeclComments; > + > public: > TargetInfo &Target; > IdentifierTable &Idents; > @@ -154,6 +163,10 @@ > llvm::OwningPtr<ExternalASTSource> ExternalSource; > clang::PrintingPolicy PrintingPolicy; > > + /// \brief Source ranges for all of the comments in the source > file, > + /// sorted in order of appearance in the translation unit. > + std::vector<SourceRange> Comments; > + > SourceManager& getSourceManager() { return SourceMgr; } > const SourceManager& getSourceManager() const { return SourceMgr; } > void *Allocate(unsigned Size, unsigned Align = 8) { > @@ -178,7 +191,8 @@ > > TranslationUnitDecl *getTranslationUnitDecl() const { return > TUDecl; } > > - > + const char *getCommentForDecl(const Decl *D); > + > // Builtin Types. > QualType VoidTy; > QualType BoolTy; > > Modified: cfe/trunk/include/clang/AST/ExternalASTSource.h > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/AST/ExternalASTSource.h?rev=74704&r1=74703&r2=74704&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- cfe/trunk/include/clang/AST/ExternalASTSource.h (original) > +++ cfe/trunk/include/clang/AST/ExternalASTSource.h Thu Jul 2 > 12:08:52 2009 > @@ -18,6 +18,7 @@ > #include "clang/AST/Type.h" > #include "llvm/ADT/SmallVector.h" > #include <cassert> > +#include <vector> > namespace clang { > > class ASTConsumer; > @@ -57,6 +58,14 @@ > > virtual ~ExternalASTSource(); > > + /// \brief Reads the source ranges that correspond to comments from > + /// an external AST source. > + /// > + /// \param Comments the contents of this vector will be > + /// replaced with the sorted set of source ranges corresponding to > + /// comments in the source code. > + virtual void ReadComments(std::vector<SourceRange> &Comments) = 0; > + > /// \brief Resolve a type ID into a type, potentially building a new > /// type. > virtual QualType GetType(uint32_t ID) = 0; > > Modified: cfe/trunk/include/clang/Frontend/PCHBitCodes.h > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/PCHBitCodes.h?rev=74704&r1=74703&r2=74704&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- cfe/trunk/include/clang/Frontend/PCHBitCodes.h (original) > +++ cfe/trunk/include/clang/Frontend/PCHBitCodes.h Thu Jul 2 > 12:08:52 2009 > @@ -218,7 +218,11 @@ > > /// \brief Record code for the original file that was used to > /// generate the precompiled header. > - ORIGINAL_FILE_NAME = 20 > + ORIGINAL_FILE_NAME = 20, > + > + /// \brief Record code for the sorted array of source ranges > where > + /// comments were encountered in the source code. > + COMMENT_RANGES = 21 > }; > > /// \brief Record types used within a source manager block. > > Modified: cfe/trunk/include/clang/Frontend/PCHReader.h > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/PCHReader.h?rev=74704&r1=74703&r2=74704&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- cfe/trunk/include/clang/Frontend/PCHReader.h (original) > +++ cfe/trunk/include/clang/Frontend/PCHReader.h Thu Jul 2 12:08:52 > 2009 > @@ -279,6 +279,12 @@ > /// been loaded. > llvm::SmallVector<Selector, 16> SelectorsLoaded; > > + /// \brief A sorted array of source ranges containing comments. > + SourceRange *Comments; > + > + /// \brief The number of source ranges in the Comments array. > + unsigned NumComments; > + > /// \brief The set of external definitions stored in the the PCH > /// file. > llvm::SmallVector<uint64_t, 16> ExternalDefinitions; > @@ -452,6 +458,14 @@ > /// build prior to including the precompiled header. > const std::string &getSuggestedPredefines() { return > SuggestedPredefines; } > > + /// \brief Reads the source ranges that correspond to comments from > + /// an external AST source. > + /// > + /// \param Comments the contents of this vector will be > + /// replaced with the sorted set of source ranges corresponding to > + /// comments in the source code. > + virtual void ReadComments(std::vector<SourceRange> &Comments); > + > /// \brief Resolve a type ID into a type, potentially building a new > /// type. > virtual QualType GetType(pch::TypeID ID); > > Modified: cfe/trunk/include/clang/Frontend/PCHWriter.h > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Frontend/PCHWriter.h?rev=74704&r1=74703&r2=74704&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- cfe/trunk/include/clang/Frontend/PCHWriter.h (original) > +++ cfe/trunk/include/clang/Frontend/PCHWriter.h Thu Jul 2 12:08:52 > 2009 > @@ -166,6 +166,7 @@ > void WriteSourceManagerBlock(SourceManager &SourceMgr, > const Preprocessor &PP); > void WritePreprocessor(const Preprocessor &PP); > + void WriteComments(ASTContext &Context); > void WriteType(const Type *T); > void WriteTypesBlock(ASTContext &Context); > uint64_t WriteDeclContextLexicalBlock(ASTContext &Context, > DeclContext *DC); > > Modified: cfe/trunk/include/clang/Lex/Preprocessor.h > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Lex/Preprocessor.h?rev=74704&r1=74703&r2=74704&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- cfe/trunk/include/clang/Lex/Preprocessor.h (original) > +++ cfe/trunk/include/clang/Lex/Preprocessor.h Thu Jul 2 12:08:52 > 2009 > @@ -26,6 +26,7 @@ > #include "llvm/ADT/DenseMap.h" > #include "llvm/ADT/OwningPtr.h" > #include "llvm/Support/Allocator.h" > +#include <vector> > > namespace clang { > > @@ -35,6 +36,7 @@ > class HeaderSearch; > class PragmaNamespace; > class PragmaHandler; > +class CommentHandler; > class ScratchBuffer; > class TargetInfo; > class PPCallbacks; > @@ -109,6 +111,10 @@ > /// with this preprocessor. > PragmaNamespace *PragmaHandlers; > > + /// \brief Tracks all of the comment handlers that the client > registered > + /// with this preprocessor. > + std::vector<CommentHandler *> CommentHandlers; > + > /// CurLexer - This is the current top of the stack that we're > lexing from if > /// not expanding a macro and we are lexing directly from source > code. > /// Only one of CurLexer, CurPTHLexer, or CurTokenLexer will be > non-null. > @@ -301,6 +307,14 @@ > /// to remove a handler that has not been registered. > void RemovePragmaHandler(const char *Namespace, PragmaHandler > *Handler); > > + /// \brief Add the specified comment handler to the preprocessor. > + void AddCommentHandler(CommentHandler *Handler); > + > + /// \brief Remove the specified comment handler. > + /// > + /// It is an error to remove a handler that has not been > registered. > + void RemoveCommentHandler(CommentHandler *Handler); > + > /// EnterMainSourceFile - Enter the specified FileID as the main > source file, > /// which implicitly adds the builtin defines etc. > void EnterMainSourceFile(); > @@ -791,6 +805,7 @@ > void HandlePragmaSystemHeader(Token &SysHeaderTok); > void HandlePragmaDependency(Token &DependencyTok); > void HandlePragmaComment(Token &CommentTok); > + void HandleComment(SourceRange Comment); > }; > > /// PreprocessorFactory - A generic factory interface for lazily > creating > @@ -801,6 +816,15 @@ > virtual Preprocessor* CreatePreprocessor() = 0; > }; > > +/// \brief Abstract base class that describes a handler that will > receive > +/// source ranges for each of the comments encountered in the > source file. > +class CommentHandler { > +public: > + virtual ~CommentHandler(); > + > + virtual void HandleComment(Preprocessor &PP, SourceRange Comment) > = 0; > +}; > + > } // end namespace clang > > #endif > > Modified: cfe/trunk/include/clang/Parse/Action.h > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Parse/Action.h?rev=74704&r1=74703&r2=74704&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- cfe/trunk/include/clang/Parse/Action.h (original) > +++ cfe/trunk/include/clang/Parse/Action.h Thu Jul 2 12:08:52 2009 > @@ -152,6 +152,10 @@ > /// an empty string if not. This is used for pretty crash > reporting. > virtual std::string getDeclName(DeclPtrTy D) { return ""; } > > + /// \brief Invoked for each comment in the source code, providing > the source > + /// range that contains the comment. > + virtual void ActOnComment(SourceRange Comment) { } > + > // > = > = > =-------------------------------------------------------------------- > ===// > // Declaration Tracking Callbacks. > // > = > = > =-------------------------------------------------------------------- > ===// > > Modified: cfe/trunk/include/clang/Parse/Parser.h > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Parse/Parser.h?rev=74704&r1=74703&r2=74704&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- cfe/trunk/include/clang/Parse/Parser.h (original) > +++ cfe/trunk/include/clang/Parse/Parser.h Thu Jul 2 12:08:52 2009 > @@ -82,7 +82,8 @@ > llvm::OwningPtr<PragmaHandler> PackHandler; > llvm::OwningPtr<PragmaHandler> UnusedHandler; > llvm::OwningPtr<PragmaHandler> WeakHandler; > - > + llvm::OwningPtr<clang::CommentHandler> CommentHandler; > + > /// Whether the '>' token acts as an operator or not. This will be > /// true except when we are parsing an expression within a C++ > /// template argument list, where the '>' closes the template > > Modified: cfe/trunk/lib/AST/ASTContext.cpp > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/AST/ASTContext.cpp?rev=74704&r1=74703&r2=74704&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- cfe/trunk/lib/AST/ASTContext.cpp (original) > +++ cfe/trunk/lib/AST/ASTContext.cpp Thu Jul 2 12:08:52 2009 > @@ -37,7 +37,8 @@ > bool FreeMem, unsigned size_reserve) : > GlobalNestedNameSpecifier(0), CFConstantStringTypeDecl(0), > ObjCFastEnumerationStateTypeDecl(0), SourceMgr(SM), LangOpts(LOpts), > - FreeMemory(FreeMem), Target(t), Idents(idents), Selectors(sels), > + LoadedExternalComments(false), FreeMemory(FreeMem), Target(t), > + Idents(idents), Selectors(sels), > BuiltinInfo(builtins), ExternalSource(0), PrintingPolicy(LOpts) { > if (size_reserve > 0) Types.reserve(size_reserve); > InitBuiltinTypes(); > @@ -202,6 +203,207 @@ > InitBuiltinType(NullPtrTy, BuiltinType::NullPtr); > } > > +namespace { > + class BeforeInTranslationUnit > + : std::binary_function<SourceRange, SourceRange, bool> { > + SourceManager *SourceMgr; > + > + public: > + explicit BeforeInTranslationUnit(SourceManager *SM) : SourceMgr > (SM) { } > + > + bool operator()(SourceRange X, SourceRange Y) { > + return SourceMgr->isBeforeInTranslationUnit(X.getBegin(), > Y.getBegin()); > + } > + }; > +} > + > +/// \brief Determine whether the given comment is a Doxygen-style > comment. > +/// > +/// \param Start the start of the comment text. > +/// > +/// \param End the end of the comment text. > +/// > +/// \param Member whether we want to check whether this is a member > comment > +/// (which requires a < after the Doxygen-comment delimiter). > Otherwise, > +/// we only return true when we find a non-member comment. > +static bool > +isDoxygenComment(SourceManager &SourceMgr, SourceRange Comment, > + bool Member = false) { > + const char *BufferStart > + = SourceMgr.getBufferData(SourceMgr.getFileID(Comment.getBegin > ())).first; > + const char *Start = BufferStart + SourceMgr.getFileOffset > (Comment.getBegin()); > + const char* End = BufferStart + SourceMgr.getFileOffset > (Comment.getEnd()); > + > + if (End - Start < 4) > + return false; > + > + assert(Start[0] == '/' && "Not a comment?"); > + if (Start[1] == '*' && !(Start[2] == '!' || Start[2] == '*')) > + return false; > + if (Start[1] == '/' && !(Start[2] == '!' || Start[2] == '/')) > + return false; > + > + return (Start[3] == '<') == Member; > +} > + > +/// \brief Retrieve the comment associated with the given > declaration, if > +/// it has one. > +const char *ASTContext::getCommentForDecl(const Decl *D) { > + if (!D) > + return 0; > + > + // Check whether we have cached a comment string for this > declaration > + // already. > + llvm::DenseMap<const Decl *, std::string>::iterator Pos > + = DeclComments.find(D); > + if (Pos != DeclComments.end()) > + return Pos->second.c_str(); > + > + // If we have an external AST source and have not yet loaded > comments from > + // that source, do so now. > + if (ExternalSource && !LoadedExternalComments) { > + std::vector<SourceRange> LoadedComments; > + ExternalSource->ReadComments(LoadedComments); > + > + if (!LoadedComments.empty()) > + Comments.insert(Comments.begin(), LoadedComments.begin(), > + LoadedComments.end()); > + > + LoadedExternalComments = true; > + } > + > + // If there are no comments anywhere, we won't find anything. > + if (Comments.empty()) > + return 0; > + > + // If the declaration doesn't map directly to a location in a > file, we > + // can't find the comment. > + SourceLocation DeclStartLoc = D->getLocStart(); > + if (DeclStartLoc.isInvalid() || !DeclStartLoc.isFileID()) > + return 0; > + > + // Find the comment that occurs just before this declaration. > + std::vector<SourceRange>::iterator LastComment > + = std::lower_bound(Comments.begin(), Comments.end(), > + SourceRange(DeclStartLoc), > + BeforeInTranslationUnit(&SourceMgr)); > + > + // Decompose the location for the start of the declaration and > find the > + // beginning of the file buffer. > + std::pair<FileID, unsigned> DeclStartDecomp > + = SourceMgr.getDecomposedLoc(DeclStartLoc); > + const char *FileBufferStart > + = SourceMgr.getBufferData(DeclStartDecomp.first).first; > + > + // First check whether we have a comment for a member. > + if (LastComment != Comments.end() && > + !isa<TagDecl>(D) && !isa<NamespaceDecl>(D) && > + isDoxygenComment(SourceMgr, *LastComment, true)) { > + std::pair<FileID, unsigned> LastCommentEndDecomp > + = SourceMgr.getDecomposedLoc(LastComment->getEnd()); > + if (DeclStartDecomp.first == LastCommentEndDecomp.first && > + SourceMgr.getLineNumber(DeclStartDecomp.first, > DeclStartDecomp.second) > + == SourceMgr.getLineNumber(LastCommentEndDecomp.first, > + LastCommentEndDecomp.second)) { > + // The Doxygen member comment comes after the declaration > starts and > + // is on the same line and in the same file as the > declaration. This > + // is the comment we want. > + std::string &Result = DeclComments[D]; > + Result.append(FileBufferStart + > + SourceMgr.getFileOffset(LastComment->getBegin > ()), > + FileBufferStart + LastCommentEndDecomp.second + > 1); > + return Result.c_str(); > + } > + } > + > + if (LastComment == Comments.begin()) > + return 0; > + --LastComment; > + > + // Decompose the end of the comment. > + std::pair<FileID, unsigned> LastCommentEndDecomp > + = SourceMgr.getDecomposedLoc(LastComment->getEnd()); > + > + // If the comment and the declaration aren't in the same file, > then they > + // aren't related. > + if (DeclStartDecomp.first != LastCommentEndDecomp.first) > + return 0; > + > + // Check that we actually have a Doxygen comment. > + if (!isDoxygenComment(SourceMgr, *LastComment)) > + return 0; > + > + // Compute the starting line for the declaration and for the end > of the > + // comment (this is expensive). > + unsigned DeclStartLine > + = SourceMgr.getLineNumber(DeclStartDecomp.first, > DeclStartDecomp.second); > + unsigned CommentEndLine > + = SourceMgr.getLineNumber(LastCommentEndDecomp.first, > + LastCommentEndDecomp.second); > + > + // If the comment does not end on the line prior to the > declaration, then > + // the comment is not associated with the declaration at all. > + if (CommentEndLine + 1 != DeclStartLine) > + return 0; > + > + // We have a comment, but there may be more comments on the > previous lines. > + // Keep looking so long as the comments are still Doxygen > comments and are > + // still adjacent. > + unsigned ExpectedLine > + = SourceMgr.getSpellingLineNumber(LastComment->getBegin()) - 1; > + std::vector<SourceRange>::iterator FirstComment = LastComment; > + while (FirstComment != Comments.begin()) { > + // Look at the previous comment > + --FirstComment; > + std::pair<FileID, unsigned> Decomp > + = SourceMgr.getDecomposedLoc(FirstComment->getEnd()); > + > + // If this previous comment is in a different file, we're done. > + if (Decomp.first != DeclStartDecomp.first) { > + ++FirstComment; > + break; > + } > + > + // If this comment is not a Doxygen comment, we're done. > + if (!isDoxygenComment(SourceMgr, *FirstComment)) { > + ++FirstComment; > + break; > + } > + > + // If the line number is not what we expected, we're done. > + unsigned Line = SourceMgr.getLineNumber(Decomp.first, > Decomp.second); > + if (Line != ExpectedLine) { > + ++FirstComment; > + break; > + } > + > + // Set the next expected line number. > + ExpectedLine > + = SourceMgr.getSpellingLineNumber(FirstComment->getBegin()) - > 1; > + } > + > + // The iterator range [FirstComment, LastComment] contains all of > the > + // BCPL comments that, together, are associated with this > declaration. > + // Form a single comment block string for this declaration that > concatenates > + // all of these comments. > + std::string &Result = DeclComments[D]; > + while (FirstComment != LastComment) { > + std::pair<FileID, unsigned> DecompStart > + = SourceMgr.getDecomposedLoc(FirstComment->getBegin()); > + std::pair<FileID, unsigned> DecompEnd > + = SourceMgr.getDecomposedLoc(FirstComment->getEnd()); > + Result.append(FileBufferStart + DecompStart.second, > + FileBufferStart + DecompEnd.second + 1); > + ++FirstComment; > + } > + > + // Append the last comment line. > + Result.append(FileBufferStart + > + SourceMgr.getFileOffset(LastComment->getBegin()), > + FileBufferStart + LastCommentEndDecomp.second + 1); > + return Result.c_str(); > +} > + > // > = > = > = > ----------------------------------------------------------------------= > ==// > // Type Sizing and Analysis > // > = > = > = > ----------------------------------------------------------------------= > ==// > > Modified: cfe/trunk/lib/Frontend/PCHReader.cpp > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/PCHReader.cpp?rev=74704&r1=74703&r2=74704&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- cfe/trunk/lib/Frontend/PCHReader.cpp (original) > +++ cfe/trunk/lib/Frontend/PCHReader.cpp Thu Jul 2 12:08:52 2009 > @@ -344,7 +344,8 @@ > IdentifierOffsets(0), > MethodPoolLookupTable(0), MethodPoolLookupTableData(0), > TotalSelectorsInMethodPool(0), SelectorOffsets(0), > - TotalNumSelectors(0), NumStatHits(0), NumStatMisses(0), > + TotalNumSelectors(0), Comments(0), NumComments(0), > + NumStatHits(0), NumStatMisses(0), > NumSLocEntriesRead(0), NumStatementsRead(0), > NumMacrosRead(0), NumMethodPoolSelectorsRead(0), > NumMethodPoolMisses(0), > NumLexicalDeclContextsRead(0), NumVisibleDeclContextsRead(0) { } > @@ -1350,6 +1351,11 @@ > case pch::ORIGINAL_FILE_NAME: > OriginalFileName.assign(BlobStart, BlobLen); > break; > + > + case pch::COMMENT_RANGES: > + Comments = (SourceRange *)BlobStart; > + NumComments = BlobLen / sizeof(SourceRange); > + break; > } > } > Error("premature end of bitstream in PCH file"); > @@ -1664,6 +1670,12 @@ > return false; > } > > +void PCHReader::ReadComments(std::vector<SourceRange> &Comments) { > + Comments.resize(NumComments); > + std::copy(this->Comments, this->Comments + NumComments, > + Comments.begin()); > +} > + > /// \brief Read and return the type at the given offset. > /// > /// This routine actually reads the record corresponding to the type > > Modified: cfe/trunk/lib/Frontend/PCHWriter.cpp > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/PCHWriter.cpp?rev=74704&r1=74703&r2=74704&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- cfe/trunk/lib/Frontend/PCHWriter.cpp (original) > +++ cfe/trunk/lib/Frontend/PCHWriter.cpp Thu Jul 2 12:08:52 2009 > @@ -374,7 +374,8 @@ > RECORD(STAT_CACHE); > RECORD(EXT_VECTOR_DECLS); > RECORD(OBJC_CATEGORY_IMPLEMENTATIONS); > - > + RECORD(COMMENT_RANGES); > + > // SourceManager Block. > BLOCK(SOURCE_MANAGER_BLOCK); > RECORD(SM_SLOC_FILE_ENTRY); > @@ -989,6 +990,24 @@ > Stream.ExitBlock(); > } > > +void PCHWriter::WriteComments(ASTContext &Context) { > + using namespace llvm; > + > + if (Context.Comments.empty()) > + return; > + > + BitCodeAbbrev *CommentAbbrev = new BitCodeAbbrev(); > + CommentAbbrev->Add(BitCodeAbbrevOp(pch::COMMENT_RANGES)); > + CommentAbbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Blob)); > + unsigned CommentCode = Stream.EmitAbbrev(CommentAbbrev); > + > + RecordData Record; > + Record.push_back(pch::COMMENT_RANGES); > + Stream.EmitRecordWithBlob(CommentCode, Record, > + (const char*)&Context.Comments[0], > + Context.Comments.size() * sizeof > (SourceRange)); > +} > + > // > = > = > = > ----------------------------------------------------------------------= > ==// > // Type Serialization > // > = > = > = > ----------------------------------------------------------------------= > ==// > @@ -1746,7 +1765,8 @@ > WriteStatCache(*StatCalls); > WriteSourceManagerBlock(Context.getSourceManager(), PP); > WritePreprocessor(PP); > - > + WriteComments(Context); > + > // Keep writing types and declarations until all types and > // declarations have been written. > do { > > Modified: cfe/trunk/lib/Lex/Lexer.cpp > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/Lexer.cpp?rev=74704&r1=74703&r2=74704&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- cfe/trunk/lib/Lex/Lexer.cpp (original) > +++ cfe/trunk/lib/Lex/Lexer.cpp Thu Jul 2 12:08:52 2009 > @@ -903,7 +903,10 @@ > } while (C != '\n' && C != '\r'); > > // Found but did not consume the newline. > - > + if (PP) > + PP->HandleComment(SourceRange(getSourceLocation(BufferPtr), > + getSourceLocation(CurPtr))); > + > // If we are returning comments as tokens, return this comment as > a token. > if (inKeepCommentMode()) > return SaveBCPLComment(Result, CurPtr); > @@ -1146,6 +1149,10 @@ > C = *CurPtr++; > } > > + if (PP) > + PP->HandleComment(SourceRange(getSourceLocation(BufferPtr), > + getSourceLocation(CurPtr))); > + > // If we are returning comments as tokens, return this comment as > a token. > if (inKeepCommentMode()) { > FormTokenWithChars(Result, CurPtr, tok::comment); > > Modified: cfe/trunk/lib/Lex/Pragma.cpp > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/Pragma.cpp?rev=74704&r1=74703&r2=74704&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- cfe/trunk/lib/Lex/Pragma.cpp (original) > +++ cfe/trunk/lib/Lex/Pragma.cpp Thu Jul 2 12:08:52 2009 > @@ -19,6 +19,7 @@ > #include "clang/Lex/LexDiagnostic.h" > #include "clang/Basic/FileManager.h" > #include "clang/Basic/SourceManager.h" > +#include <algorithm> > using namespace clang; > > // Out-of-line destructor to provide a home for the class. > > Modified: cfe/trunk/lib/Lex/Preprocessor.cpp > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/Preprocessor.cpp?rev=74704&r1=74703&r2=74704&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- cfe/trunk/lib/Lex/Preprocessor.cpp (original) > +++ cfe/trunk/lib/Lex/Preprocessor.cpp Thu Jul 2 12:08:52 2009 > @@ -476,3 +476,26 @@ > if (II.isExtensionToken() && !DisableMacroExpansion) > Diag(Identifier, diag::ext_token_used); > } > + > +void Preprocessor::AddCommentHandler(CommentHandler *Handler) { > + assert(Handler && "NULL comment handler"); > + assert(std::find(CommentHandlers.begin(), CommentHandlers.end(), > Handler) == > + CommentHandlers.end() && "Comment handler already > registered"); > + CommentHandlers.push_back(Handler); > +} > + > +void Preprocessor::RemoveCommentHandler(CommentHandler *Handler) { > + std::vector<CommentHandler *>::iterator Pos > + = std::find(CommentHandlers.begin(), CommentHandlers.end(), > Handler); > + assert(Pos != CommentHandlers.end() && "Comment handler not > registered"); > + CommentHandlers.erase(Pos); > +} > + > +void Preprocessor::HandleComment(SourceRange Comment) { > + for (std::vector<CommentHandler *>::iterator H = > CommentHandlers.begin(), > + HEnd = CommentHandlers.end(); > + H != HEnd; ++H) > + (*H)->HandleComment(*this, Comment); > +} > + > +CommentHandler::~CommentHandler() { } > > Modified: cfe/trunk/lib/Parse/Parser.cpp > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Parse/Parser.cpp?rev=74704&r1=74703&r2=74704&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- cfe/trunk/lib/Parse/Parser.cpp (original) > +++ cfe/trunk/lib/Parse/Parser.cpp Thu Jul 2 12:08:52 2009 > @@ -20,6 +20,19 @@ > #include "ParsePragma.h" > using namespace clang; > > +/// \brief A comment handler that passes comments found by the > preprocessor > +/// to the parser action. > +class ActionCommentHandler : public CommentHandler { > + Action &Actions; > + > +public: > + explicit ActionCommentHandler(Action &Actions) : Actions(Actions) > { } > + > + virtual void HandleComment(Preprocessor &PP, SourceRange Comment) { > + Actions.ActOnComment(Comment); > + } > +}; > + > Parser::Parser(Preprocessor &pp, Action &actions) > : CrashInfo(*this), PP(pp), Actions(actions), Diags > (PP.getDiagnostics()), > GreaterThanIsOperator(true) { > @@ -43,6 +56,9 @@ > WeakHandler.reset(new > PragmaWeakHandler(&PP.getIdentifierTable().get("weak"), > actions)); > PP.AddPragmaHandler(0, WeakHandler.get()); > + > + CommentHandler.reset(new ActionCommentHandler(actions)); > + PP.AddCommentHandler(CommentHandler.get()); > } > > /// If a crash happens while the parser is active, print out a line > indicating > @@ -294,6 +310,7 @@ > UnusedHandler.reset(); > PP.RemovePragmaHandler(0, WeakHandler.get()); > WeakHandler.reset(); > + PP.RemoveCommentHandler(CommentHandler.get()); > } > > /// Initialize - Warm up the parser. > > Modified: cfe/trunk/lib/Sema/Sema.cpp > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Sema/Sema.cpp?rev=74704&r1=74703&r2=74704&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- cfe/trunk/lib/Sema/Sema.cpp (original) > +++ cfe/trunk/lib/Sema/Sema.cpp Thu Jul 2 12:08:52 2009 > @@ -345,3 +345,7 @@ > = SemaRef.ActiveTemplateInstantiations.back(); > } > } > + > +void Sema::ActOnComment(SourceRange Comment) { > + Context.Comments.push_back(Comment); > +} > > Modified: cfe/trunk/lib/Sema/Sema.h > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Sema/Sema.h?rev=74704&r1=74703&r2=74704&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- cfe/trunk/lib/Sema/Sema.h (original) > +++ cfe/trunk/lib/Sema/Sema.h Thu Jul 2 12:08:52 2009 > @@ -362,6 +362,8 @@ > return CurBlock ? CurBlock->SwitchStack : FunctionSwitchStack; > } > > + virtual void ActOnComment(SourceRange Comment); > + > // > = > = > =-------------------------------------------------------------------- > ===// > // Type Analysis / Processing: SemaType.cpp. > // > > Modified: cfe/trunk/test/CMakeLists.txt > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CMakeLists.txt?rev=74704&r1=74703&r2=74704&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- cfe/trunk/test/CMakeLists.txt (original) > +++ cfe/trunk/test/CMakeLists.txt Thu Jul 2 12:08:52 2009 > @@ -8,6 +8,7 @@ > "Driver" > "FixIt" > "Frontend" > + "Index" > "Lexer" > "Misc" > "PCH" > @@ -54,6 +55,6 @@ > --clang-cc=${LLVM_TOOLS_PATH}/${CMAKE_CFG_INTDIR}/clang-cc > ${all_testdirs} > WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} > - DEPENDS clang clang-cc > + DEPENDS clang clang-cc index-test > COMMENT "Running Clang regression tests") > endif() > > Added: cfe/trunk/test/Index/comments.c > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Index/comments.c?rev=74704&view=auto > > = > = > = > = > = > = > = > = > ====================================================================== > --- cfe/trunk/test/Index/comments.c (added) > +++ cfe/trunk/test/Index/comments.c Thu Jul 2 12:08:52 2009 > @@ -0,0 +1,30 @@ > +// RUN: clang-cc -emit-pch -o %t.ast %s && > +// RUN: index-test %t.ast -point-at %s:22:6 | grep "starts here" && > +// RUN: index-test %t.ast -point-at %s:22:6 | grep "block comment" && > +// RUN: index-test %t.ast -point-at %s:28:6 | grep "BCPL" && > +// RUN: index-test %t.ast -point-at %s:28:6 | grep "But" && > +// RUN: index-test %t.ast -point-at %s:28:6 | grep "NOT" | count 0 && > +// RUN: index-test %t.ast -point-at %s:30:6 | grep "member" > + > + > + > + > + > + > +//! It all starts here. > +/*! It's a little odd to continue line this, > + * > + * but we need more multi-line comments. */ > +/// This comment comes before my other comments > +/** This is a block comment that is associated with the function f. > It > + * runs for three lines. > + */ > +void f(int, int); > + > +// NOT IN THE COMMENT > +/// This is a BCPL comment that is associated with the function g. > +/// It has only two lines. > +/** But there are other blocks that are part of the comment, too. */ > +void g(int); > + > +void h(int); ///< This is a member comment. > \ No newline at end of file > > Propchange: cfe/trunk/test/Index/comments.c > > ------------------------------------------------------------------------------ > svn:eol-style = native > > Propchange: cfe/trunk/test/Index/comments.c > > ------------------------------------------------------------------------------ > svn:keywords = Id > > Propchange: cfe/trunk/test/Index/comments.c > > ------------------------------------------------------------------------------ > svn:mime-type = text/plain > > Modified: cfe/trunk/tools/index-test/index-test.cpp > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/index-test/index-test.cpp?rev=74704&r1=74703&r2=74704&view=diff > > = > = > = > = > = > = > = > = > ====================================================================== > --- cfe/trunk/tools/index-test/index-test.cpp (original) > +++ cfe/trunk/tools/index-test/index-test.cpp Thu Jul 2 12:08:52 2009 > @@ -130,6 +130,9 @@ > OS << ND->getNameAsString(); > OS << "\n"; > > + if (const char *Comment = AST->getASTContext().getCommentForDecl > (Point.D)) > + OS << "Comment associated with this declaration:\n" << > Comment << "\n"; > + > if (Point.Node) { > OS << "Statement node at point: " << Point.Node- > >getStmtClassName() > << " "; > > > _______________________________________________ > cfe-commits mailing list > [email protected] > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits _______________________________________________ cfe-commits mailing list [email protected] http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
