Found this nasty surprise in my inbox and git fetch this morning. On IRC, chandlerc indicated there was no particular technical problem. It was just "too complex". Looking over the new solution I do really like the idea of using the lit script to do more work instead of a build-system-initiated task. I too want the tests to be easier to debug and 'poke at' but I think relying on the preprocessor is a step backward in that regard. The python scripts at least generated a single source file that could be poked at easily. The only piece of advice necessary was where to find that auto-generated file in the build directory. When test failures happened, the reported line number would exactly identify the failing test. With the current approach, it's impossible to tell which of the many runs of cpp11-migrate with their various -D options is causing the problem. As for debugging, I've never had much luck getting gdb to look through macros.
So my proposed solution: resurrect the python scripts for generating the files and call them from the lit script. Have the files generated directly to the lit output directory (i.e. what the lit scripts refer to as %T). Run the 'CHECK:'-stripping grep on the generated files, and then just run filecheck on those. This would be a pretty trivial change to the lit script. Now we have easy-to-read monolithic files again failing tests are easy to identify and the lit errors will even include the location of the files in the build directory for easy reference. As for the hand-written tests that need access to auto-generated headers, we can add the python call to those lit scripts too to generate the required headers. Does this sound like a workable compromise? -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Chandler Carruth Sent: Thursday, March 07, 2013 5:10 AM To: [email protected] Subject: [clang-tools-extra] r176627 - Switch from autogenerating tests to using the preprocessor. Author: chandlerc Date: Thu Mar 7 04:09:47 2013 New Revision: 176627 URL: http://llvm.org/viewvc/llvm-project?rev=176627&view=rev Log: Switch from autogenerating tests to using the preprocessor. NOTE: You may need to run 'make clean' or 'ninja -t clean' etc!!! This is due to really nasty bug/interactions between CMake/configure/make/Ninja/LIT... This commit tries to back out the support for generating test cases as part of the build system due to the issues I brought up in post-commit review: 1) It adds a *lot* of complexity and fragility to the build system. See the number of commits required to try to get all the bots happy. 2) It isn't really necessary -- we can already run scripts to generate things with the RUN lines of a test. 3) It makes the tests somewhat harder to debug as they cross between more domains. 4) In almost all cases it isn't really needed or it can be done directly using the preprocessor. I should have been more proactive reviewing this, and I'm really sorry about the churn here. =/ To help keep track of what commits are going where, this backs out most of the non-test-changes from these revisions: r176397 r176373 r176293 r176184 r175744 r175624 r175545 r175544 There were several trivial or cleanup changes to the lit files or other files. Some of these looked ok, but I didn't try to tease them apart... Edwin, if you know what to look for, please carry on with the cleanups there, and sorry for hosing stuff here but I'm not much of a Python person, and so I was erring on the side of cautiously backing out the change. I've tried to preserve the test changes everywhere I could, but review is appreciated here in case I missed some. I then re-wrote the tests to use the preprocessor rather than python to expand to the various bits of code. The nicest part of this is that now all the files are just C++ code. They edit and behave like C++ code, etc. RUN lines with different -D flags are used to run the same test over multiple different configurations, and includes bracketed in special defines are used to flesh out a collection of standard interface stubs to test interactions between pieces. These probably aren't perfect yet, but I think its an improvement (at least in terms of build system complexity) and will hopefully be a useful demonstration of the technique I prefer for these types of tests. Added: clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/Inputs/test_std_container.h clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/basic_iterator_tests.cpp Removed: clang-tools-extra/trunk/test/cpp11-migrate/CMakeLists.txt clang-tools-extra/trunk/test/cpp11-migrate/Makefile clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/Inputs/gen_my_std.h.py clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/gen_basic_std_iterator_tests.cpp.py clang-tools-extra/trunk/test/cpp11-migrate/lit.site.cfg.in Modified: clang-tools-extra/trunk/Makefile clang-tools-extra/trunk/test/CMakeLists.txt clang-tools-extra/trunk/test/Makefile clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/iterator.cpp clang-tools-extra/trunk/test/lit.cfg clang-tools-extra/trunk/test/lit.site.cfg.in Modified: clang-tools-extra/trunk/Makefile URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/Makefile?rev=176627&r1=176626&r2=176627&view=diff ============================================================================== --- clang-tools-extra/trunk/Makefile (original) +++ clang-tools-extra/trunk/Makefile Thu Mar 7 04:09:47 2013 @@ -12,15 +12,29 @@ CLANG_LEVEL := ../.. include $(CLANG_LEVEL)/../../Makefile.config PARALLEL_DIRS := remove-cstr-calls tool-template clang-format cpp11-migrate -DIRS := test include $(CLANG_LEVEL)/Makefile -# Custom target. Pass request to test/Makefile that knows what to do. To access -# this target you'd issue: -# -# make -C <build_dir>/tools/clang/tools/extra test +### +# Handle the nested test suite. + +ifneq ($(PROJ_SRC_ROOT),$(PROJ_OBJ_ROOT)) +$(RecursiveTargets):: + $(Verb) for dir in test; do \ + if [ -f $(PROJ_SRC_DIR)/$${dir}/Makefile ] && [ ! -f $${dir}/Makefile ]; then \ + $(MKDIR) $${dir}; \ + $(CP) $(PROJ_SRC_DIR)/$${dir}/Makefile $${dir}/Makefile; \ + fi \ + done +endif + test:: - @ $(MAKE) -C test test + @ $(MAKE) -C test + +report:: + @ $(MAKE) -C test report + +clean:: + @ $(MAKE) -C test clean -.PHONY: test +.PHONY: test report clean Modified: clang-tools-extra/trunk/test/CMakeLists.txt URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/CMakeLists.txt?rev=176627&r1=176626&r2=176627&view=diff ============================================================================== --- clang-tools-extra/trunk/test/CMakeLists.txt (original) +++ clang-tools-extra/trunk/test/CMakeLists.txt Thu Mar 7 04:09:47 2013 @@ -7,37 +7,28 @@ set(CLANG_TOOLS_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/..") set(CLANG_TOOLS_BINARY_DIR "${CMAKE_CURRENT_BINARY_DIR}/..") -option(CLANG_TOOLS_TEST_USE_VG "Run Clang tools' tests under Valgrind" OFF) -if(CLANG_TOOLS_TEST_USE_VG) - set(CLANG_TOOLS_TEST_EXTRA_ARGS ${CLANG_TEST_EXTRA_ARGS} "--vg") -endif() - -add_subdirectory(cpp11-migrate) - -set(TEST_SOURCE_ROOT ${CMAKE_CURRENT_SOURCE_DIR}) -set(TEST_EXEC_ROOT ${CMAKE_CURRENT_BINARY_DIR}) -set(TESTSUITE_NAME "Clang Tools") configure_lit_site_cfg( ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.in ${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg ) +option(CLANG_TOOLS_TEST_USE_VG "Run Clang tools' tests under Valgrind" +OFF) +if(CLANG_TOOLS_TEST_USE_VG) + set(CLANG_TOOLS_TEST_EXTRA_ARGS ${CLANG_TEST_EXTRA_ARGS} "--vg") +endif() + set(CLANG_TOOLS_TEST_DEPS # Base line deps. clang clang-headers FileCheck count not - remove-cstr-calls clang-format - - cpp11-migrate cpp11-migrate-autogen + # Individual tools we test. + remove-cstr-calls cpp11-migrate clang-format ) -add_lit_testsuite(check-clang-tools "Running regression tests for Clang extra tools" +add_lit_testsuite(check-clang-tools "Running the Clang extra tools' regression tests" ${CMAKE_CURRENT_BINARY_DIR} - # cpp11-migrate's auto-generated tests need to be mentioned explicitly since - # the sources live in ${CMAKE_CURRENT_BINARY_DIR} and won't get discovered - # otherwise. - ${CMAKE_CURRENT_BINARY_DIR}/cpp11-migrate/generated_tests DEPENDS ${CLANG_TOOLS_TEST_DEPS} ARGS ${CLANG_TOOLS_TEST_EXTRA_ARGS} ) -set_target_properties(check-clang-tools PROPERTIES FOLDER "Clang extra tools tests") +set_target_properties(check-clang-tools PROPERTIES FOLDER "Clang extra +tools' tests") + Modified: clang-tools-extra/trunk/test/Makefile URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/Makefile?rev=176627&r1=176626&r2=176627&view=diff ============================================================================== --- clang-tools-extra/trunk/test/Makefile (original) +++ clang-tools-extra/trunk/test/Makefile Thu Mar 7 04:09:47 2013 @@ -8,8 +8,6 @@ ##===----------------------------------------------------------------------===## CLANG_LEVEL := ../../.. -include $(CLANG_LEVEL)/../../Makefile.config -DIRS := cpp11-migrate include $(CLANG_LEVEL)/Makefile # Test in all immediate subdirectories if unset. @@ -25,11 +23,6 @@ TESTDIRS := $(TESTDIRS:$(PROJ_SRC_DIR)%= # Allow EXTRA_TESTDIRS to provide additional test directories. TESTDIRS += $(EXTRA_TESTDIRS) -# We'd like cpp11-migrate's auto-generated tests to be included in the LIT run -# below. Since the auto-generated test sources live in PROJ_OBJ_DIR they won't -# get discovered without specifying them explicitly. -TESTDIRS += $(PROJ_OBJ_DIR)/cpp11-migrate/generated_tests - ifndef TESTARGS ifdef VERBOSE TESTARGS = -v @@ -38,16 +31,14 @@ TESTARGS = -s -v endif endif +# Make sure any extra test suites can find the main site config. +LIT_ARGS := --param clang_site_config=$(PROJ_OBJ_DIR)/lit.site.cfg + ifdef VG LIT_ARGS += "--vg" endif all:: lit.site.cfg - -# Run just the Clang extra tools tests. This target assumes 'make (all)' has -# been run previously as that target is responsible for generating lit config -# files and auto-generated tests. -test:: @ echo '--- Running the Clang extra tools tests for $(TARGET_TRIPLE) ---' @ $(PYTHON) $(LLVM_SRC_ROOT)/utils/lit/lit.py \ $(LIT_ARGS) $(TESTARGS) $(TESTDIRS) @@ -55,7 +46,7 @@ test:: FORCE: lit.site.cfg: FORCE - @echo "Making lit.site.cfg for Clang extra tools..." + @echo "Making Clang extra tools' 'lit.site.cfg' file..." @$(ECHOPATH) s=@LLVM_SOURCE_DIR@=$(LLVM_SRC_ROOT)=g > lit.tmp @$(ECHOPATH) s=@LLVM_BINARY_DIR@=$(LLVM_OBJ_ROOT)=g >> lit.tmp @$(ECHOPATH) s=@LLVM_TOOLS_DIR@=$(ToolDir)=g >> lit.tmp @@ -63,13 +54,10 @@ lit.site.cfg: FORCE @$(ECHOPATH) s=@CLANG_TOOLS_SOURCE_DIR@=$(PROJ_SRC_DIR)/..=g >> lit.tmp @$(ECHOPATH) s=@CLANG_TOOLS_BINARY_DIR@=$(PROJ_OBJ_DIR)/..=g >> lit.tmp @$(ECHOPATH) s=@TARGET_TRIPLE@=$(TARGET_TRIPLE)=g >> lit.tmp - @$(ECHOPATH) s=@TEST_SOURCE_ROOT@=$(PROJ_SRC_DIR)=g >> lit.tmp - @$(ECHOPATH) s=@TEST_EXEC_ROOT@=$(PROJ_OBJ_DIR)=g >> lit.tmp - @$(ECHOPATH) s=@TESTSUITE_NAME@=Clang Tools=g >> lit.tmp @sed -f lit.tmp $(PROJ_SRC_DIR)/lit.site.cfg.in > $@ @-rm -f lit.tmp clean:: @ find . -name Output | xargs rm -fr -.PHONY: all test clean +.PHONY: all report clean Removed: clang-tools-extra/trunk/test/cpp11-migrate/CMakeLists.txt URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/cpp11-migrate/CMakeLists.txt?rev=176626&view=auto ============================================================================== --- clang-tools-extra/trunk/test/cpp11-migrate/CMakeLists.txt (original) +++ clang-tools-extra/trunk/test/cpp11-migrate/CMakeLists.txt (removed) @@ -1,71 +0,0 @@ - -# List of generator scripts. Generator scripts must: -# * Be written in python -# * Output their result to standard out. -# * Be named as gen_X.py where X will be the name of the auto-generated file. -set(generator_scripts - UseAuto/gen_basic_std_iterator_tests.cpp.py - UseAuto/Inputs/gen_my_std.h.py - ) - -# macro that runs a generator script to produce an auto-generated file. -# Generator scripts must follow scheme above. The resulting file is placed in: -# ${CMAKE_CURRENT_BINARY_DIR})/generated_tests/dirname(<script name>). -# -# Two arguments are required: -# script - The generator script (relative to ${CMAKE_CURRENT_SOURCE_DIR}) -# output - Name of the variable to store the name of the generated file in. -macro(autogenerate_file script output) - get_filename_component(dir ${script} PATH) - file(MAKE_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/generated_tests/${dir}") - string(REGEX REPLACE "gen_(.*).py" "\\1" output_name ${script}) - set(script_file - "${CMAKE_CURRENT_SOURCE_DIR}/${script}" - ) - set(output_file - "${CMAKE_CURRENT_BINARY_DIR}/generated_tests/${output_name}" - ) - set(gencmd - ${PYTHON_EXECUTABLE} - ${script_file} > ${output_file} - ) - add_custom_command( - OUTPUT ${output_file} - COMMAND ${gencmd} - DEPENDS ${script_file} - ) - set(${output} ${output_file}) -endmacro(autogenerate_file) - -# Define rules to run generator scripts. -set(depends) -foreach(script ${generator_scripts}) - autogenerate_file(${script} output) - list(APPEND depends ${output}) -endforeach() - -# Target to perform all generation. -add_custom_target(cpp11-migrate-autogen - DEPENDS ${depends} - ) - -# Create lit.site.cfg for regular regression tests. Provide access to location -# of generated tests so regular tests can use headers. -set(TEST_SOURCE_ROOT ${CMAKE_CURRENT_SOURCE_DIR}) -set(TEST_EXEC_ROOT ${CMAKE_CURRENT_BINARY_DIR}) -set(TESTSUITE_NAME "cpp11-migrate Tests") -set(GENERATED_TESTS_ROOT ${CMAKE_CURRENT_BINARY_DIR}/generated_tests) -configure_lit_site_cfg( - ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.in - ${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg - ) - -# Create lit.site.cfg for auto-generated tests. Sources for these tests live -# in the build directory. -set(TEST_SOURCE_ROOT ${GENERATED_TESTS_ROOT}) -set(TEST_EXEC_ROOT ${CMAKE_CURRENT_BINARY_DIR}) -set(TESTSUITE_NAME "cpp11-migrate Auto-Generated Tests") -configure_lit_site_cfg( - ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.in - ${GENERATED_TESTS_ROOT}/lit.site.cfg - ) Removed: clang-tools-extra/trunk/test/cpp11-migrate/Makefile URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/cpp11-migrate/Makefile?rev=176626&view=auto ============================================================================== --- clang-tools-extra/trunk/test/cpp11-migrate/Makefile (original) +++ clang-tools-extra/trunk/test/cpp11-migrate/Makefile (removed) @@ -1,88 +0,0 @@ -##===- tools/extra/test/cpp11-migrate/Makefile -------------*- Makefile -*-===## -# -# The LLVM Compiler Infrastructure -# -# This file is distributed under the University of Illinois Open Source -# License. See LICENSE.TXT for details. -# -##===----------------------------------------------------------------------===## - -# This Makefile is responsible for creating a LIT testsuite for auto-generated -# tests. The recipe that runs llvm-lit needs to provide the following path of -# the lit.site.cfg for the generated testsuite: $(PROJ_OBJ_DIR)/generated_tests. - -CLANG_LEVEL := ../../../.. -include $(CLANG_LEVEL)/Makefile - -# List of generator scripts. Generator scripts must: -# * Be written in python -# * Output their result to standard out. -# * Be named as gen_X.py where X will be the name of the auto-generated file. -GENERATOR_SCRIPTS := \ - UseAuto/gen_basic_std_iterator_tests.cpp.py \ - UseAuto/Inputs/gen_my_std.h.py - -# macro to be used with $(call) that generates a rule and recipe that causes a -# file to be auto-generated from a generator script. Generator scripts must -# follow scheme above. The resulting file is placed in: -# $(PROJ_OBJ_DIR)/generated_tests/$(dir <script name>). -# -# One argument required: the name of generator script relative to -# $(PROJ_SRC_DIR). -define autogenerate-file -OUTFILE := $(addprefix generated_tests/$(dir $1), $(patsubst gen_%.py,%,$(notdir $1))) -GENERATED_FILES := $$(GENERATED_FILES) $$(OUTFILE) -$$(OUTFILE): $1 - @echo "Autogenerating $$@" - @$(MKDIR) $(addprefix generated_tests/, $(dir $1)) - @$(PYTHON) $$< > $$@ -endef -$(foreach script, $(GENERATOR_SCRIPTS), $(eval $(call autogenerate-file,$(script)))) - -FORCE: - -GENERATED_TESTS_ROOT := $(PROJ_OBJ_DIR)/generated_tests - -# Recipe to create lit.site.cfg for the auto-generated tests suite. Sources -# for these tests are auto-generated into the build directory. -GEN_LIT_TMP := generated_tests/lit.tmp -generated_tests/lit.site.cfg: FORCE - @$(MKDIR) $(GENERATED_TESTS_ROOT) - @echo "Making lit.site.cfg for cpp11-migrate Auto-Generated Tests..." - @$(ECHOPATH) s=@LLVM_SOURCE_DIR@=$(LLVM_SRC_ROOT)=g > $(GEN_LIT_TMP) - @$(ECHOPATH) s=@LLVM_BINARY_DIR@=$(LLVM_OBJ_ROOT)=g >> $(GEN_LIT_TMP) - @$(ECHOPATH) s=@LLVM_TOOLS_DIR@=$(ToolDir)=g >> $(GEN_LIT_TMP) - @$(ECHOPATH) s=@LLVM_LIBS_DIR@=$(LibDir)=g >> $(GEN_LIT_TMP) - @$(ECHOPATH) s=@CLANG_TOOLS_SOURCE_DIR@=$(PROJ_SRC_DIR)/../..=g >> $(GEN_LIT_TMP) - @$(ECHOPATH) s=@CLANG_TOOLS_BINARY_DIR@=$(PROJ_OBJ_DIR)/../..=g >> $(GEN_LIT_TMP) - @$(ECHOPATH) s=@TARGET_TRIPLE@=$(TARGET_TRIPLE)=g >> $(GEN_LIT_TMP) - @$(ECHOPATH) s=@TEST_SOURCE_ROOT@=$(GENERATED_TESTS_ROOT)=g >> $(GEN_LIT_TMP) - @$(ECHOPATH) s=@TEST_EXEC_ROOT@=$(PROJ_OBJ_DIR)=g >> $(GEN_LIT_TMP) - @$(ECHOPATH) s=@TESTSUITE_NAME@=cpp11-migrate Auto-Generated Tests=g >> $(GEN_LIT_TMP) - @$(ECHOPATH) s=@GENERATED_TESTS_ROOT@=$(GENERATED_TESTS_ROOT)=g >> $(GEN_LIT_TMP) - @sed -f $(GEN_LIT_TMP) $(PROJ_SRC_DIR)/lit.site.cfg.in > $@ - @-rm -f $(GEN_LIT_TMP) - -# Recipe to create the lit.site.cfg for the regular cpp11-migrate test suite. -lit.site.cfg: FORCE - @echo "Making lit.site.cfg for cpp11-migrate Tests..." - @$(ECHOPATH) s=@LLVM_SOURCE_DIR@=$(LLVM_SRC_ROOT)=g > lit.tmp - @$(ECHOPATH) s=@LLVM_BINARY_DIR@=$(LLVM_OBJ_ROOT)=g >> lit.tmp - @$(ECHOPATH) s=@LLVM_TOOLS_DIR@=$(ToolDir)=g >> lit.tmp - @$(ECHOPATH) s=@LLVM_LIBS_DIR@=$(LibDir)=g >> lit.tmp - @$(ECHOPATH) s=@CLANG_TOOLS_SOURCE_DIR@=$(PROJ_SRC_DIR)/../..=g >> lit.tmp - @$(ECHOPATH) s=@CLANG_TOOLS_BINARY_DIR@=$(PROJ_OBJ_DIR)/../..=g >> lit.tmp - @$(ECHOPATH) s=@TARGET_TRIPLE@=$(TARGET_TRIPLE)=g >> lit.tmp - @$(ECHOPATH) s=@TEST_SOURCE_ROOT@=$(PROJ_SRC_DIR)=g >> lit.tmp - @$(ECHOPATH) s=@TEST_EXEC_ROOT@=$(PROJ_OBJ_DIR)=g >> lit.tmp - @$(ECHOPATH) s=@TESTSUITE_NAME@=cpp11-migrate Tests=g >> lit.tmp - @$(ECHOPATH) s=@GENERATED_TESTS_ROOT@=$(GENERATED_TESTS_ROOT)=g >> lit.tmp - @sed -f lit.tmp $(PROJ_SRC_DIR)/lit.site.cfg.in > $@ - @-rm -f lit.tmp - -all:: $(GENERATED_FILES) lit.site.cfg generated_tests/lit.site.cfg - -clean:: - @-rm -r generated_tests - -.PHONY: all clean Removed: clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/Inputs/gen_my_std.h.py URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/Inputs/gen_my_std.h.py?rev=176626&view=auto ============================================================================== --- clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/Inputs/gen_my_std.h.py (original) +++ clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/Inputs/gen_my_std +++ .h.py (removed) @@ -1,188 +0,0 @@ -#!/usr/bin/python - -# Each std container is represented below. To test the various ways in which -# a type may be defined, the containers are split into categories: -# * Define iterator types with typedefs -# * Define iterator types as nested classes -# * Define iterator types with using declarations -# -# Further, one class in each category is chosen to be defined in a way mimicing -# libc++: The container is actually defined in a different namespace (std::_1 -# is used here) and then imported into the std namespace with a using -# declaration. This is controlled with the 'using' key in the dictionary -# describing each container. -typedef_containers = [ - {"name" : "array", - "using" : True}, - {"name" : "deque", - "using" : False}, - {"name" : "forward_list", - "using" : False}, - {"name" : "list", - "using" : False}, - {"name" : "vector", - "using" : False} -] -subclass_containers = [ - {"name" : "map", - "using" : True}, - {"name" : "multimap", - "using" : False}, - {"name" : "set", - "using" : False}, - {"name" : "multiset", - "using" : False}, -] -using_containers = [ - {"name" : "unordered_map", - "using" : True}, - {"name" : "unordered_multimap", - "using" : False}, - {"name" : "unordered_set", - "using" : False}, - {"name" : "unordered_multiset", - "using" : False}, - {"name" : "queue", - "using" : False}, - {"name" : "priority_queue", - "using" : False}, - {"name" : "stack", - "using" : False} -] - - -# Every class requires these functions. -iterator_generators = """ - iterator begin() { return iterator(); } - iterator end() { return iterator(); } - - const_iterator begin() const { return const_iterator(); } - const_iterator end() const { return const_iterator(); } - - reverse_iterator rbegin() { return reverse_iterator(); } - reverse_iterator rend() { return reverse_iterator(); } - - const_reverse_iterator rbegin() const { return const_reverse_iterator(); } - const_reverse_iterator rend() const { return const_reverse_iterator(); } -""" - - -# Convenience function for nested class definition within a special namespace -# to mimic libc++ style std container definitions. -def outputClassDef(Definition, ClassName, Import): - if Import: - print "namespace _1 {" - - print Definition - - if Import: - print """ -} // namespace _1 -using _1::%s;""" % ClassName - - -# Output preamble and common functionality -print """ -//===-----------------------------------------------------------*- C++ -*--===// -// -// This file was automatically generated from gen_my_std.h.py by the build -// system as a dependency for cpp11-migrate's test suite. -// -// This file contains a shell implementation of std containers and iterators for -// testing the use-auto transform of cpp11-migrate. All std containers and -// iterators are present. Container and iterator implementations vary to cover -// various ways the std container and iterator types are made available: -// -// Variations for how iterator types are presented: -// * Typedef (array, deque, forward_list, list, vector) -// * Nested class (map, multimap, set, multiset) -// * Using declaration {unordered_} X {map, multimap, set, multiset} -// -// Variations for how container types are presented: -// * Defined directly in namespace std -// * Imported into namespace std with using declarations (a la libc++). -// -//===----------------------------------------------------------------------===// - -namespace internal { - -template <typename T, int i> -struct iterator_wrapper { - iterator_wrapper() {} - - // These are required for tests using iteration statements. - bool operator!=(const iterator_wrapper<T, i>&) { return false; } - iterator_wrapper& operator++() { return *this; } - typename T::value_type operator*() { return typename T::value_type(); } -}; - -template <typename T> -class iterator_provider { -public: - class iterator { - public: - iterator() {} - iterator(const iterator&) {} - }; - class const_iterator { - public: - const_iterator(int i=0) {} - const_iterator(const iterator &) {} - const_iterator(const const_iterator &) {} - operator iterator() { return iterator(); } - }; - class reverse_iterator {}; - class const_reverse_iterator {}; -}; - -} // namespace internal - -namespace std {""".lstrip() # Take off leading newline - -for c in typedef_containers: - Definition = """ -template <typename T> -class %(0)s { -public: - typedef T value_type; - typedef typename internal::iterator_wrapper<%(0)s<T>, 0> iterator; - typedef typename internal::iterator_wrapper<%(0)s<T>, 1> const_iterator; - typedef typename internal::iterator_wrapper<%(0)s<T>, 3> reverse_iterator; - typedef typename internal::iterator_wrapper<%(0)s<T>, 2> const_reverse_iterator; - - %(0)s() {} - %(1)s};""" % {'0': c['name'], '1': iterator_generators} - - outputClassDef(Definition, c['name'], c['using']) - -for c in subclass_containers: - Definition = """ -template <typename T> -class %(0)s { -public: - class iterator {}; - class const_iterator {}; - class reverse_iterator {}; - class const_reverse_iterator {}; - - %(0)s() {} - %(1)s};""" % {'0': c['name'], '1': iterator_generators} - - outputClassDef(Definition, c['name'], c['using']) - -for c in using_containers: - Definition = """ -template <typename T> -class %(0)s : internal::iterator_provider<%(0)s<T> > { -public: - using typename internal::iterator_provider<%(0)s<T> >::iterator; - using typename internal::iterator_provider<%(0)s<T> >::const_iterator; - using typename internal::iterator_provider<%(0)s<T> >::reverse_iterator; - using typename internal::iterator_provider<%(0)s<T> >::const_reverse_iterator; - - %(0)s() {} - %(1)s};""" % {'0': c['name'], '1': iterator_generators} - - outputClassDef(Definition, c['name'], c['using']) - -print "} // namespace std" Added: clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/Inputs/test_std_container.h URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/Inputs/test_std_container.h?rev=176627&view=auto ============================================================================== --- clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/Inputs/test_std_container.h (added) +++ clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/Inputs/test_std_c +++ ontainer.h Thu Mar 7 04:09:47 2013 @@ -0,0 +1,117 @@ +//===-----------------------------------------------------------*- C++ +-*--===// // // This file contains a shell implementation of a standard +container with // iterators. This shell is targeted at supporting the +container interfaces // recognized by cpp11-migrate's use-auto +transformation. It requires the // preprocessor to parameterize the +name of the container, and allows the // preprocessor to parameterize +various mechanisms used in the implementation // of the container / +iterator. +// +// Variations for how iterator types are presented: +// * Typedef (array, deque, forward_list, list, vector) // * Nested +class (map, multimap, set, multiset) // * Using declaration +{unordered_} X {map, multimap, set, multiset} // // Variations for how +container types are presented: +// * Defined directly in namespace std +// * Imported into namespace std with using declarations (a la libc++). +// +//===------------------------------------------------------------------ +----===// + +#ifndef CONTAINER +#error You must define CONTAINER to the name of the desired container. +#endif + +// If the test code needs multiple containers, only define our helpers once. +#ifndef TEST_STD_CONTAINER_HELPERS +#define TEST_STD_CONTAINER_HELPERS + +namespace internal { + +template <typename T, int i> +struct iterator_wrapper { + iterator_wrapper() {} + + // These are required for tests using iteration statements. + bool operator!=(const iterator_wrapper<T, i>&) { return false; } + iterator_wrapper& operator++() { return *this; } + typename T::value_type operator*() { return typename T::value_type(); +} }; + +template <typename T> +class iterator_provider { +public: + class iterator { + public: + iterator() {} + iterator(const iterator&) {} + }; + class const_iterator { + public: + const_iterator(int i=0) {} + const_iterator(const iterator &) {} + const_iterator(const const_iterator &) {} + operator iterator() { return iterator(); } + }; + class reverse_iterator {}; + class const_reverse_iterator {}; +}; + +} // namespace internal + +#endif // TEST_STD_CONTAINER_HELPERS + +namespace std { + +#if USE_INLINE_NAMESPACE +namespace _1 { +#endif + +template <typename T> +class CONTAINER +#if USE_BASE_CLASS_ITERATORS + : internal::iterator_provider<CONTAINER<T> > #endif { +public: + +#if USE_BASE_CLASS_ITERATORS + using typename internal::iterator_provider<CONTAINER<T> >::iterator; + using typename internal::iterator_provider<CONTAINER<T> +>::const_iterator; + using typename internal::iterator_provider<CONTAINER<T> +>::reverse_iterator; + using typename internal::iterator_provider<CONTAINER<T> +>::const_reverse_iterator; #elif USE_INNER_CLASS_ITERATORS + class iterator {}; + class const_iterator {}; + class reverse_iterator {}; + class const_reverse_iterator {}; +#else + typedef T value_type; + typedef typename internal::iterator_wrapper<CONTAINER<T>, 0> +iterator; + typedef typename internal::iterator_wrapper<CONTAINER<T>, 1> +const_iterator; + typedef typename internal::iterator_wrapper<CONTAINER<T>, 3> +reverse_iterator; + typedef typename internal::iterator_wrapper<CONTAINER<T>, 2> +const_reverse_iterator; #endif + + // Every class requires these functions. + CONTAINER() {} + + iterator begin() { return iterator(); } iterator end() { return + iterator(); } + + const_iterator begin() const { return const_iterator(); } + const_iterator end() const { return const_iterator(); } + + reverse_iterator rbegin() { return reverse_iterator(); } + reverse_iterator rend() { return reverse_iterator(); } + + const_reverse_iterator rbegin() const { return +const_reverse_iterator(); } + const_reverse_iterator rend() const { return +const_reverse_iterator(); } }; + +#if USE_INLINE_NAMESPACE +} // namespace _1 +using _1::CONTAINER; +#endif + +} // namespace std Added: clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/basic_iterator_tests.cpp URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/basic_iterator_tests.cpp?rev=176627&view=auto ============================================================================== --- clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/basic_iterator_tests.cpp (added) +++ clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/basic_iterator_te +++ sts.cpp Thu Mar 7 04:09:47 2013 @@ -0,0 +1,123 @@ +// This file contains basic positive tests for the use-auto transform's +ability // to replace standard iterators. Variables considered: +// * All std container names +// * All std iterator names +// * Different patterns of defining iterators and containers // // // +The most basic test. +// +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp // RUN: cpp11-migrate +-use-auto %t.cpp -- -DCONTAINER=array -I %S/Inputs // RUN: FileCheck +-input-file=%t.cpp %s // // // Test variations on how the container and +its iterators might be defined. +// +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp // RUN: cpp11-migrate +-use-auto %t.cpp -- -DCONTAINER=array \ +// RUN: -DUSE_INLINE_NAMESPACE -I %S/Inputs +// RUN: FileCheck -input-file=%t.cpp %s // // RUN: grep -Ev "// +*[A-Z-]+:" %s > %t.cpp // RUN: cpp11-migrate -use-auto %t.cpp -- +-DCONTAINER=array \ +// RUN: -DUSE_BASE_CLASS_ITERATORS -I %S/Inputs +// RUN: FileCheck -input-file=%t.cpp %s // // RUN: grep -Ev "// +*[A-Z-]+:" %s > %t.cpp // RUN: cpp11-migrate -use-auto %t.cpp -- +-DCONTAINER=array \ +// RUN: -DUSE_INNER_CLASS_ITERATORS -I %S/Inputs +// RUN: FileCheck -input-file=%t.cpp %s // // // Test all of the other +container names in a basic configuration. +// +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp // RUN: cpp11-migrate +-use-auto %t.cpp -- -DCONTAINER=deque -I %S/Inputs // RUN: FileCheck +-input-file=%t.cpp %s // // RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp // +RUN: cpp11-migrate -use-auto %t.cpp -- -DCONTAINER=forward_list -I +%S/Inputs // RUN: FileCheck -input-file=%t.cpp %s // // RUN: grep -Ev +"// *[A-Z-]+:" %s > %t.cpp // RUN: cpp11-migrate -use-auto %t.cpp -- +-DCONTAINER=list -I %S/Inputs // RUN: FileCheck -input-file=%t.cpp %s +// // RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp // RUN: cpp11-migrate +-use-auto %t.cpp -- -DCONTAINER=vector -I %S/Inputs // RUN: FileCheck +-input-file=%t.cpp %s // // RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp // +RUN: cpp11-migrate -use-auto %t.cpp -- -DCONTAINER=map -I %S/Inputs // +RUN: FileCheck -input-file=%t.cpp %s // // RUN: grep -Ev "// *[A-Z-]+:" +%s > %t.cpp // RUN: cpp11-migrate -use-auto %t.cpp -- +-DCONTAINER=multimap -I %S/Inputs // RUN: FileCheck -input-file=%t.cpp +%s // // RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp // RUN: cpp11-migrate +-use-auto %t.cpp -- -DCONTAINER=set -I %S/Inputs // RUN: FileCheck +-input-file=%t.cpp %s // // RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp // +RUN: cpp11-migrate -use-auto %t.cpp -- -DCONTAINER=multiset -I +%S/Inputs // RUN: FileCheck -input-file=%t.cpp %s // // RUN: grep -Ev +"// *[A-Z-]+:" %s > %t.cpp // RUN: cpp11-migrate -use-auto %t.cpp -- +-DCONTAINER=unordered_map -I %S/Inputs // RUN: FileCheck +-input-file=%t.cpp %s // // RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp // +RUN: cpp11-migrate -use-auto %t.cpp -- -DCONTAINER=unordered_multimap +-I %S/Inputs // RUN: FileCheck -input-file=%t.cpp %s // // RUN: grep +-Ev "// *[A-Z-]+:" %s > %t.cpp // RUN: cpp11-migrate -use-auto %t.cpp +-- -DCONTAINER=unordered_set -I %S/Inputs // RUN: FileCheck +-input-file=%t.cpp %s // // RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp // +RUN: cpp11-migrate -use-auto %t.cpp -- -DCONTAINER=unordered_multiset +-I %S/Inputs // RUN: FileCheck -input-file=%t.cpp %s // // RUN: grep +-Ev "// *[A-Z-]+:" %s > %t.cpp // RUN: cpp11-migrate -use-auto %t.cpp +-- -DCONTAINER=queue -I %S/Inputs // RUN: FileCheck -input-file=%t.cpp +%s // // RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp // RUN: cpp11-migrate +-use-auto %t.cpp -- -DCONTAINER=priority_queue -I %S/Inputs // RUN: +FileCheck -input-file=%t.cpp %s // // RUN: grep -Ev "// *[A-Z-]+:" %s > +%t.cpp // RUN: cpp11-migrate -use-auto %t.cpp -- -DCONTAINER=stack -I +%S/Inputs // RUN: FileCheck -input-file=%t.cpp %s + +#ifndef CONTAINER +#error You must define CONTAINER to the name of the container for testing. +#endif + +#include "test_std_container.h" + +int main(int argc, char **argv) { + { + std::CONTAINER<int> C; + std::CONTAINER<int>::iterator I = C.begin(); + // CHECK: auto I = C.begin(); + } + { + std::CONTAINER<int> C; + std::CONTAINER<int>::reverse_iterator I = C.rbegin(); + // CHECK: auto I = C.rbegin(); + } + { + const std::CONTAINER<int> C; + std::CONTAINER<int>::const_iterator I = C.begin(); + // CHECK: auto I = C.begin(); + } + { + const std::CONTAINER<int> C; + std::CONTAINER<int>::const_reverse_iterator I = C.rbegin(); + // CHECK: auto I = C.rbegin(); + } + + return 0; +} Removed: clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/gen_basic_std_iterator_tests.cpp.py URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/gen_basic_std_iterator_tests.cpp.py?rev=176626&view=auto ============================================================================== --- clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/gen_basic_std_iterator_tests.cpp.py (original) +++ clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/gen_basic_std_ite +++ rator_tests.cpp.py (removed) @@ -1,68 +0,0 @@ -#!/usr/bin/python - -containers = [ - "array", - "deque", - "forward_list", - "list", - "vector", - "map", - "multimap", - "set", - "multiset", - "unordered_map", - "unordered_multimap", - "unordered_set", - "unordered_multiset", - "queue", - "priority_queue", - "stack" -] - -print """ -//===----------------------------------------------------------------------===// -// -// This file was automatically generated from -// gen_basic_std_iterator_tests.cpp.py by the build system as a dependency for -// cpp11-migrate's test suite. -// -// This file contains basic positive tests for the use-auto transform's ability -// to replace standard iterators. Variables considered: -// * All std container names -// * All std iterator names -// -//===----------------------------------------------------------------------===// - -// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp -// RUN: cpp11-migrate -use-auto %t.cpp -- -I %S/Inputs -// RUN: FileCheck -input-file=%t.cpp %s -#include "my_std.h" - -int main(int argc, char **argv) {""".lstrip() # Strip leading newline - -for c in containers: - print """ - { - std::%(0)s<int> C; - std::%(0)s<int>::iterator I = C.begin(); - // CHECK: auto I = C.begin(); - } - { - std::%(0)s<int> C; - std::%(0)s<int>::reverse_iterator I = C.rbegin(); - // CHECK: auto I = C.rbegin(); - } - { - const std::%(0)s<int> C; - std::%(0)s<int>::const_iterator I = C.begin(); - // CHECK: auto I = C.begin(); - } - { - const std::%(0)s<int> C; - std::%(0)s<int>::const_reverse_iterator I = C.rbegin(); - // CHECK: auto I = C.rbegin(); - }""" % {"0": c} - -print """ - return 0; -}""" Modified: clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/iterator.cpp URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/iterator.cpp?rev=176627&r1=176626&r2=176627&view=diff ============================================================================== --- clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/iterator.cpp (original) +++ clang-tools-extra/trunk/test/cpp11-migrate/UseAuto/iterator.cpp Thu +++ Mar 7 04:09:47 2013 @@ -1,7 +1,20 @@ // RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp -// RUN: cpp11-migrate -use-auto %t.cpp -- --std=c++11 -I %gen_root/UseAuto/Inputs +// RUN: cpp11-migrate -use-auto %t.cpp -- --std=c++11 -I %S/Inputs // RUN: FileCheck -input-file=%t.cpp %s -#include "my_std.h" + +#define CONTAINER array +#include "test_std_container.h" +#undef CONTAINER + +#define CONTAINER vector +#include "test_std_container.h" +#undef CONTAINER + +#define CONTAINER unordered_map +#define USE_BASE_CLASS_ITERATORS 1 +#include "test_std_container.h" +#undef USE_BASE_CLASS_ITERATORS +#undef CONTAINER typedef std::vector<int>::iterator int_iterator; Removed: clang-tools-extra/trunk/test/cpp11-migrate/lit.site.cfg.in URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/cpp11-migrate/lit.site.cfg.in?rev=176626&view=auto ============================================================================== --- clang-tools-extra/trunk/test/cpp11-migrate/lit.site.cfg.in (original) +++ clang-tools-extra/trunk/test/cpp11-migrate/lit.site.cfg.in (removed) @@ -1,28 +0,0 @@ -## Autogenerated by LLVM/Clang configuration. -# Do not edit! -config.llvm_src_root = "@LLVM_SOURCE_DIR@" -config.llvm_obj_root = "@LLVM_BINARY_DIR@" -config.llvm_tools_dir = "@LLVM_TOOLS_DIR@" -config.llvm_libs_dir = "@LLVM_LIBS_DIR@" -config.lit_tools_dir = "@LLVM_LIT_TOOLS_DIR@" -config.clang_tools_binary_dir = "@CLANG_TOOLS_BINARY_DIR@" -config.target_triple = "@TARGET_TRIPLE@" - -config.name = "@TESTSUITE_NAME@" -config.test_source_root = "@TEST_SOURCE_ROOT@" -config.test_exec_root = "@TEST_EXEC_ROOT@" - -config.generated_tests_root = "@GENERATED_TESTS_ROOT@" -config.substitutions.append(('%gen_root', config.generated_tests_root)) - -# Support substitution of the tools and libs dirs with user parameters. This is -# used when we can't determine the tool dir at configuration time. -try: - config.llvm_tools_dir = config.llvm_tools_dir % lit.params - config.llvm_libs_dir = config.llvm_libs_dir % lit.params -except KeyError,e: - key, = e.args - lit.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key)) - -# Let the main config do the real work. -lit.load_config(config, "@CLANG_TOOLS_SOURCE_DIR@/test/lit.cfg") Modified: clang-tools-extra/trunk/test/lit.cfg URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/lit.cfg?rev=176627&r1=176626&r2=176627&view=diff ============================================================================== --- clang-tools-extra/trunk/test/lit.cfg (original) +++ clang-tools-extra/trunk/test/lit.cfg Thu Mar 7 04:09:47 2013 @@ -8,9 +8,12 @@ import subprocess # Configuration file for the 'lit' test runner. +# name: The name of this test suite. +config.name = 'Clang Tools' + # Tweak PATH for Win32 if platform.system() == 'Windows': - # Seek sane tools in directories and set to $PATH. + # Seek sane tools in directories and set to $PATH. path = getattr(config, 'lit_tools_dir', None) path = lit.getToolsPath(path, config.environment['PATH'], @@ -31,6 +34,14 @@ config.test_format = lit.formats.ShTest( # suffixes: A list of file extensions to treat as test files. config.suffixes = ['.c', '.cpp', '.m', '.mm', '.cu', '.ll', '.cl', '.s'] +# test_source_root: The root path where tests are located. +config.test_source_root = os.path.dirname(__file__) + +# test_exec_root: The root path where tests should be run. +clang_tools_binary_dir = getattr(config, 'clang_tools_binary_dir', +None) if clang_tools_binary_dir is not None: + config.test_exec_root = os.path.join(clang_tools_binary_dir, +'test') + # Clear some environment variables that might affect Clang. # # This first set of vars are read by Clang, but shouldn't affect tests @@ -61,7 +72,7 @@ for name in possibly_dangerous_env_vars: del config.environment[name] # Tweak the PATH to include the tools dir and the scripts dir. -if hasattr(config, 'clang_tools_binary_dir'): +if clang_tools_binary_dir is not None: llvm_tools_dir = getattr(config, 'llvm_tools_dir', None) if not llvm_tools_dir: lit.fatal('No LLVM tools dir set!') Modified: clang-tools-extra/trunk/test/lit.site.cfg.in URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/lit.site.cfg.in?rev=176627&r1=176626&r2=176627&view=diff ============================================================================== --- clang-tools-extra/trunk/test/lit.site.cfg.in (original) +++ clang-tools-extra/trunk/test/lit.site.cfg.in Thu Mar 7 04:09:47 +++ 2013 @@ -8,10 +8,6 @@ config.lit_tools_dir = "@LLVM_LIT_TOOLS_ config.clang_tools_binary_dir = "@CLANG_TOOLS_BINARY_DIR@" config.target_triple = "@TARGET_TRIPLE@" -config.name = "@TESTSUITE_NAME@" -config.test_source_root = "@TEST_SOURCE_ROOT@" -config.test_exec_root = "@TEST_EXEC_ROOT@" - # Support substitution of the tools and libs dirs with user parameters. This is # used when we can't determine the tool dir at configuration time. try: _______________________________________________ cfe-commits mailing list [email protected] http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits _______________________________________________ cfe-commits mailing list [email protected] http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
