gri3-wm/m4/ax_code_coverage.m4

274 lines
12 KiB
Plaintext
Raw Normal View History

Switch to autotools (GNU build system) This commit probably comes as a surprise to some, given that one of i3’s explicitly stated goals used to be “Do not use programs such as autoconf/automake for configuration and creating unreadable/broken makefiles”. I phrased this goal over 7 years ago, based largely on a grudge that I inherited, which — as I’ve realized in the meantime — was largely held against FOSS in general, and not actually nuanced criticism of autotools. In the meantime, I have come to realize that the knee-jerk reaction of “I could do this better!” (i.e. writing our own build system in this particular case) is usually misguided, and nowadays I strongly suggest trying hard to fix the existing system for the benefit of all existing and future users. Further, I recently got to experience the other side of the coin, as I packaged a new version of FreeRADIUS for Debian, which at the time of writing used autoconf in combination with boilermake, a custom make-based build system that only FreeRADIUS uses. Understanding the build system enough to fix issues and enable parallel compilation took me an entire day. That time is time which potentially every downstream maintainer needs to invest, and the resulting knowledge cannot be applied to any other project. Hence, I believe it’s a good idea switch i3 to autotools. Yes, it might be that particular features were easier to implement/understand in our custom Makefiles, and there might be individuals who have an easier time reading through our custom Makefiles than learning autotools. All of these considerations are outweighed by the benefits we get from using the same build system as literally thousands of other FOSS software packages. Aside from these somewhat philosophical considerations, there’s also practical improvements which this change brings us. See the “changes” section below. ┌──────────────────────────────────────────────────────────────────────────────┐ │ new workflow │ └──────────────────────────────────────────────────────────────────────────────┘ You can now build i3 like you build any other software package which uses autotools. Here’s a memory refresher: autoreconf -fi mkdir -p build && cd build ../configure make -j8 (The autoreconf -fi step is unnecessary if you are building from a release tarball, but shouldn’t hurt either.) ┌──────────────────────────────────────────────────────────────────────────────┐ │ recommended reading │ └──────────────────────────────────────────────────────────────────────────────┘ I very much recommend reading “A Practitioner's Guide to GNU Autoconf, Automake, and Libtool” by John Calcote (https://www.nostarch.com/autotools.htm). That book is from 2010 and, AFAICT, is the most up to date comprehensive description of autotools. Do not read older documentation. In particular, if a document you’re reading mentions configure.in (deprecated filename) or recursive make (now considered harmful), it’s likely outdated. ┌──────────────────────────────────────────────────────────────────────────────┐ │ changes │ └──────────────────────────────────────────────────────────────────────────────┘ This commit implements the following new functionality/changes in behavior: • We use the AX_ENABLE_BUILDDIR macro to enforce builds happening in a separate directory. This is a prerequisite for the AX_EXTEND_SRCDIR macro and building in a separate directory is common practice anyway. In case this causes any trouble when packaging i3 for your distribution, please let me know. • “make check” runs the i3 testsuite. You can still use ./testcases/complete-run.pl to get the interactive progress output. • “make distcheck” (runs testsuite on “make dist” result, tiny bit quicker feedback cycle than waiting for the travis build to catch the issue). • “make uninstall” (occasionally requested by users who compile from source) • “make” will build manpages/docs by default if the tools are installed. Conversely, manpages/docs are not tried to be built for users who don’t want to install all these dependencies to get started hacking on i3. • non-release builds will enable address sanitizer by default. Use the --disable-sanitizers configure option to turn off all sanitizers, and see --help for available sanitizers. • Support for pre-compiled headers (PCH) has been dropped for now in the interest of simplicitly. Maybe we can re-add it later. • coverage reports are now generated using “make check-code-coverage”, which requires specifying --enable-code-coverage when calling configure. ┌──────────────────────────────────────────────────────────────────────────────┐ │ build system feature parity/testing │ └──────────────────────────────────────────────────────────────────────────────┘ In addition to what’s described above, I tested the following features: • “make install” installs the same files (plus documentation and manpages) cd i3-old && make install PREFIX=/tmp/inst/old cd i3-new && ./configure --prefix=/tmp/inst/new cd /tmp/inst (cd old && for f in $(find); do [ -e "../new/$f" ] || echo "$f missing"; done) • make dist generates a tarball which includes the same files cd i3-old && make dist cd i3-new/x86_64-pc-linux-gnu && make dist colordiff -u <(tar tf i3-old/i3-4.12.tar.bz2 | sort) \ <(tar tf i3-new/x86_64-pc-linux-gnu/i3-4.12.tar.gz | sort) There are some expected differences: • Some files have been renamed (e.g. the new etc/ and share/ subdirectories) • Some files will now be generated at build-time, so only their corresponding .in file is shipped (e.g. testcases/complete-run.pl) • The generated parser files are shipped in the dist tarball (they only depend on the parser-specs/* files, not on the target system) • autotools infrastructure is shipped (e.g. “configure”, “missing”, etc.) • DLOG and ELOG statements still produce the same file name in logfiles • Listing source code in gdb still works. • gdb backtraces contain the i3-<version> path component • release.sh still works • version embedding 1. git checkout shows “4.12-136-gf720023 (2016-10-10, branch "autotools")” 2. tarball of a git version shows “4.12-non-git” 3. release tarball shows 4.13 • debug mode is enabled by default for non-release builds • enabling verbose builds via V=1 ┌──────────────────────────────────────────────────────────────────────────────┐ │ speed │ └──────────────────────────────────────────────────────────────────────────────┘ There is no noticeable difference in compilation speed itself (of binaries, documentation and manpages): i3-old $ time make all docs mans -j8 make all docs mans -j8 28.92s user 2.15s system 640% cpu 4.852 total i3-new $ time make -j8 make -j8 27.08s user 1.92s system 620% cpu 4.669 total In terms of one-time costs: configuring the build system (../configure) takes about 2.7s on my machine, generating the build system (autoreconf -fi) takes about 3.1s on my machine. ┌──────────────────────────────────────────────────────────────────────────────┐ │ m4 macros │ └──────────────────────────────────────────────────────────────────────────────┘ All files in m4/ have been copied from the autoconf-archive package in version b6aeb1988f4b6c78bf39d97b6c4f6e1d594d59b9 and should be updated whenever they change. This commit has been tested with autoconf 2.69 and automake 1.15.
2016-10-07 13:48:26 +02:00
# ===========================================================================
# http://www.gnu.org/software/autoconf-archive/ax_code_coverage.html
# ===========================================================================
#
# SYNOPSIS
#
# AX_CODE_COVERAGE()
#
# DESCRIPTION
#
# Defines CODE_COVERAGE_CPPFLAGS, CODE_COVERAGE_CFLAGS,
# CODE_COVERAGE_CXXFLAGS and CODE_COVERAGE_LDFLAGS which should be
# included in the CPPFLAGS, CFLAGS CXXFLAGS and LIBS/LDFLAGS variables of
# every build target (program or library) which should be built with code
# coverage support. Also defines CODE_COVERAGE_RULES which should be
# substituted in your Makefile; and $enable_code_coverage which can be
# used in subsequent configure output. CODE_COVERAGE_ENABLED is defined
# and substituted, and corresponds to the value of the
# --enable-code-coverage option, which defaults to being disabled.
#
# Test also for gcov program and create GCOV variable that could be
# substituted.
#
# Note that all optimisation flags in CFLAGS must be disabled when code
# coverage is enabled.
#
# Usage example:
#
# configure.ac:
#
# AX_CODE_COVERAGE
#
# Makefile.am:
#
# @CODE_COVERAGE_RULES@
# my_program_LIBS = ... $(CODE_COVERAGE_LDFLAGS) ...
# my_program_CPPFLAGS = ... $(CODE_COVERAGE_CPPFLAGS) ...
# my_program_CFLAGS = ... $(CODE_COVERAGE_CFLAGS) ...
# my_program_CXXFLAGS = ... $(CODE_COVERAGE_CXXFLAGS) ...
#
# This results in a "check-code-coverage" rule being added to any
# Makefile.am which includes "@CODE_COVERAGE_RULES@" (assuming the module
# has been configured with --enable-code-coverage). Running `make
# check-code-coverage` in that directory will run the module's test suite
# (`make check`) and build a code coverage report detailing the code which
# was touched, then print the URI for the report.
#
# This code was derived from Makefile.decl in GLib, originally licenced
# under LGPLv2.1+.
#
# LICENSE
#
# Copyright (c) 2012, 2016 Philip Withnall
# Copyright (c) 2012 Xan Lopez
# Copyright (c) 2012 Christian Persch
# Copyright (c) 2012 Paolo Borelli
# Copyright (c) 2012 Dan Winship
# Copyright (c) 2015 Bastien ROUCARIES
#
# This library is free software; you can redistribute it and/or modify it
# under the terms of the GNU Lesser General Public License as published by
# the Free Software Foundation; either version 2.1 of the License, or (at
# your option) any later version.
#
# This library is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser
# General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#serial 15
AC_DEFUN([AX_CODE_COVERAGE],[
dnl Check for --enable-code-coverage
AC_REQUIRE([AC_PROG_SED])
# allow to override gcov location
AC_ARG_WITH([gcov],
[AS_HELP_STRING([--with-gcov[=GCOV]], [use given GCOV for coverage (GCOV=gcov).])],
[_AX_CODE_COVERAGE_GCOV_PROG_WITH=$with_gcov],
[_AX_CODE_COVERAGE_GCOV_PROG_WITH=gcov])
AC_MSG_CHECKING([whether to build with code coverage support])
AC_ARG_ENABLE([code-coverage],
AS_HELP_STRING([--enable-code-coverage],
[Whether to enable code coverage support]),,
enable_code_coverage=no)
AM_CONDITIONAL([CODE_COVERAGE_ENABLED], [test x$enable_code_coverage = xyes])
AC_SUBST([CODE_COVERAGE_ENABLED], [$enable_code_coverage])
AC_MSG_RESULT($enable_code_coverage)
AS_IF([ test "$enable_code_coverage" = "yes" ], [
# check for gcov
AC_CHECK_TOOL([GCOV],
[$_AX_CODE_COVERAGE_GCOV_PROG_WITH],
[:])
AS_IF([test "X$GCOV" = "X:"],
[AC_MSG_ERROR([gcov is needed to do coverage])])
AC_SUBST([GCOV])
dnl Check if gcc is being used
AS_IF([ test "$GCC" = "no" ], [
AC_MSG_ERROR([not compiling with gcc, which is required for gcov code coverage])
])
# List of supported lcov versions.
lcov_version_list="1.6 1.7 1.8 1.9 1.10 1.11 1.12"
AC_CHECK_PROG([LCOV], [lcov], [lcov])
AC_CHECK_PROG([GENHTML], [genhtml], [genhtml])
AS_IF([ test "$LCOV" ], [
AC_CACHE_CHECK([for lcov version], ax_cv_lcov_version, [
ax_cv_lcov_version=invalid
lcov_version=`$LCOV -v 2>/dev/null | $SED -e 's/^.* //'`
for lcov_check_version in $lcov_version_list; do
if test "$lcov_version" = "$lcov_check_version"; then
ax_cv_lcov_version="$lcov_check_version (ok)"
fi
done
])
], [
lcov_msg="To enable code coverage reporting you must have one of the following lcov versions installed: $lcov_version_list"
AC_MSG_ERROR([$lcov_msg])
])
case $ax_cv_lcov_version in
""|invalid[)]
lcov_msg="You must have one of the following versions of lcov: $lcov_version_list (found: $lcov_version)."
AC_MSG_ERROR([$lcov_msg])
LCOV="exit 0;"
;;
esac
AS_IF([ test -z "$GENHTML" ], [
AC_MSG_ERROR([Could not find genhtml from the lcov package])
])
dnl Build the code coverage flags
CODE_COVERAGE_CPPFLAGS="-DNDEBUG"
CODE_COVERAGE_CFLAGS="-O0 -g -fprofile-arcs -ftest-coverage"
CODE_COVERAGE_CXXFLAGS="-O0 -g -fprofile-arcs -ftest-coverage"
CODE_COVERAGE_LDFLAGS="-lgcov"
AC_SUBST([CODE_COVERAGE_CPPFLAGS])
AC_SUBST([CODE_COVERAGE_CFLAGS])
AC_SUBST([CODE_COVERAGE_CXXFLAGS])
AC_SUBST([CODE_COVERAGE_LDFLAGS])
])
[CODE_COVERAGE_RULES='
# Code coverage
#
# Optional:
# - CODE_COVERAGE_DIRECTORY: Top-level directory for code coverage reporting.
# Multiple directories may be specified, separated by whitespace.
# (Default: $(top_builddir))
# - CODE_COVERAGE_OUTPUT_FILE: Filename and path for the .info file generated
# by lcov for code coverage. (Default:
# $(PACKAGE_NAME)-$(PACKAGE_VERSION)-coverage.info)
# - CODE_COVERAGE_OUTPUT_DIRECTORY: Directory for generated code coverage
# reports to be created. (Default:
# $(PACKAGE_NAME)-$(PACKAGE_VERSION)-coverage)
# - CODE_COVERAGE_BRANCH_COVERAGE: Set to 1 to enforce branch coverage,
# set to 0 to disable it and leave empty to stay with the default.
# (Default: empty)
# - CODE_COVERAGE_LCOV_SHOPTS_DEFAULT: Extra options shared between both lcov
# instances. (Default: based on $CODE_COVERAGE_BRANCH_COVERAGE)
# - CODE_COVERAGE_LCOV_SHOPTS: Extra options to shared between both lcov
# instances. (Default: $CODE_COVERAGE_LCOV_SHOPTS_DEFAULT)
# - CODE_COVERAGE_LCOV_OPTIONS_GCOVPATH: --gcov-tool pathtogcov
# - CODE_COVERAGE_LCOV_OPTIONS_DEFAULT: Extra options to pass to the
# collecting lcov instance. (Default: $CODE_COVERAGE_LCOV_OPTIONS_GCOVPATH)
# - CODE_COVERAGE_LCOV_OPTIONS: Extra options to pass to the collecting lcov
# instance. (Default: $CODE_COVERAGE_LCOV_OPTIONS_DEFAULT)
# - CODE_COVERAGE_LCOV_RMOPTS_DEFAULT: Extra options to pass to the filtering
# lcov instance. (Default: empty)
# - CODE_COVERAGE_LCOV_RMOPTS: Extra options to pass to the filtering lcov
# instance. (Default: $CODE_COVERAGE_LCOV_RMOPTS_DEFAULT)
# - CODE_COVERAGE_GENHTML_OPTIONS_DEFAULT: Extra options to pass to the
# genhtml instance. (Default: based on $CODE_COVERAGE_BRANCH_COVERAGE)
# - CODE_COVERAGE_GENHTML_OPTIONS: Extra options to pass to the genhtml
# instance. (Default: $CODE_COVERAGE_GENHTML_OPTIONS_DEFAULT)
# - CODE_COVERAGE_IGNORE_PATTERN: Extra glob pattern of files to ignore
#
# The generated report will be titled using the $(PACKAGE_NAME) and
# $(PACKAGE_VERSION). In order to add the current git hash to the title,
# use the git-version-gen script, available online.
# Optional variables
CODE_COVERAGE_DIRECTORY ?= $(top_builddir)
CODE_COVERAGE_OUTPUT_FILE ?= $(PACKAGE_NAME)-$(PACKAGE_VERSION)-coverage.info
CODE_COVERAGE_OUTPUT_DIRECTORY ?= $(PACKAGE_NAME)-$(PACKAGE_VERSION)-coverage
CODE_COVERAGE_BRANCH_COVERAGE ?=
CODE_COVERAGE_LCOV_SHOPTS_DEFAULT ?= $(if $(CODE_COVERAGE_BRANCH_COVERAGE),\
--rc lcov_branch_coverage=$(CODE_COVERAGE_BRANCH_COVERAGE))
CODE_COVERAGE_LCOV_SHOPTS ?= $(CODE_COVERAGE_LCOV_SHOPTS_DEFAULT)
CODE_COVERAGE_LCOV_OPTIONS_GCOVPATH ?= --gcov-tool "$(GCOV)"
CODE_COVERAGE_LCOV_OPTIONS_DEFAULT ?= $(CODE_COVERAGE_LCOV_OPTIONS_GCOVPATH)
CODE_COVERAGE_LCOV_OPTIONS ?= $(CODE_COVERAGE_LCOV_OPTIONS_DEFAULT)
CODE_COVERAGE_LCOV_RMOPTS_DEFAULT ?=
CODE_COVERAGE_LCOV_RMOPTS ?= $(CODE_COVERAGE_LCOV_RMOPTS_DEFAULT)
CODE_COVERAGE_GENHTML_OPTIONS_DEFAULT ?=\
$(if $(CODE_COVERAGE_BRANCH_COVERAGE),\
--rc genhtml_branch_coverage=$(CODE_COVERAGE_BRANCH_COVERAGE))
CODE_COVERAGE_GENHTML_OPTIONS ?= $(CODE_COVERAGE_GENHTML_OPTIONS_DEFAULTS)
CODE_COVERAGE_IGNORE_PATTERN ?=
code_coverage_v_lcov_cap = $(code_coverage_v_lcov_cap_$(V))
code_coverage_v_lcov_cap_ = $(code_coverage_v_lcov_cap_$(AM_DEFAULT_VERBOSITY))
code_coverage_v_lcov_cap_0 = @echo " LCOV --capture"\
$(CODE_COVERAGE_OUTPUT_FILE);
code_coverage_v_lcov_ign = $(code_coverage_v_lcov_ign_$(V))
code_coverage_v_lcov_ign_ = $(code_coverage_v_lcov_ign_$(AM_DEFAULT_VERBOSITY))
code_coverage_v_lcov_ign_0 = @echo " LCOV --remove /tmp/*"\
$(CODE_COVERAGE_IGNORE_PATTERN);
code_coverage_v_genhtml = $(code_coverage_v_genhtml_$(V))
code_coverage_v_genhtml_ = $(code_coverage_v_genhtml_$(AM_DEFAULT_VERBOSITY))
code_coverage_v_genhtml_0 = @echo " GEN " $(CODE_COVERAGE_OUTPUT_DIRECTORY);
code_coverage_quiet = $(code_coverage_quiet_$(V))
code_coverage_quiet_ = $(code_coverage_quiet_$(AM_DEFAULT_VERBOSITY))
code_coverage_quiet_0 = --quiet
# sanitizes the test-name: replaces with underscores: dashes and dots
code_coverage_sanitize = $(subst -,_,$(subst .,_,$(1)))
# Use recursive makes in order to ignore errors during check
check-code-coverage:
ifeq ($(CODE_COVERAGE_ENABLED),yes)
-$(A''M_V_at)$(MAKE) $(AM_MAKEFLAGS) -k check
$(A''M_V_at)$(MAKE) $(AM_MAKEFLAGS) code-coverage-capture
else
@echo "Need to reconfigure with --enable-code-coverage"
endif
# Capture code coverage data
code-coverage-capture: code-coverage-capture-hook
ifeq ($(CODE_COVERAGE_ENABLED),yes)
$(code_coverage_v_lcov_cap)$(LCOV) $(code_coverage_quiet) $(addprefix --directory ,$(CODE_COVERAGE_DIRECTORY)) --capture --output-file "$(CODE_COVERAGE_OUTPUT_FILE).tmp" --test-name "$(call code_coverage_sanitize,$(PACKAGE_NAME)-$(PACKAGE_VERSION))" --no-checksum --compat-libtool $(CODE_COVERAGE_LCOV_SHOPTS) $(CODE_COVERAGE_LCOV_OPTIONS)
$(code_coverage_v_lcov_ign)$(LCOV) $(code_coverage_quiet) $(addprefix --directory ,$(CODE_COVERAGE_DIRECTORY)) --remove "$(CODE_COVERAGE_OUTPUT_FILE).tmp" "/tmp/*" $(CODE_COVERAGE_IGNORE_PATTERN) --output-file "$(CODE_COVERAGE_OUTPUT_FILE)" $(CODE_COVERAGE_LCOV_SHOPTS) $(CODE_COVERAGE_LCOV_RMOPTS)
-@rm -f $(CODE_COVERAGE_OUTPUT_FILE).tmp
$(code_coverage_v_genhtml)LANG=C $(GENHTML) $(code_coverage_quiet) $(addprefix --prefix ,$(CODE_COVERAGE_DIRECTORY)) --output-directory "$(CODE_COVERAGE_OUTPUT_DIRECTORY)" --title "$(PACKAGE_NAME)-$(PACKAGE_VERSION) Code Coverage" --legend --show-details "$(CODE_COVERAGE_OUTPUT_FILE)" $(CODE_COVERAGE_GENHTML_OPTIONS)
@echo "file://$(abs_builddir)/$(CODE_COVERAGE_OUTPUT_DIRECTORY)/index.html"
else
@echo "Need to reconfigure with --enable-code-coverage"
endif
# Hook rule executed before code-coverage-capture, overridable by the user
code-coverage-capture-hook:
ifeq ($(CODE_COVERAGE_ENABLED),yes)
clean: code-coverage-clean
code-coverage-clean:
-$(LCOV) --directory $(top_builddir) -z
-rm -rf $(CODE_COVERAGE_OUTPUT_FILE) $(CODE_COVERAGE_OUTPUT_FILE).tmp $(CODE_COVERAGE_OUTPUT_DIRECTORY)
-find . -name "*.gcda" -o -name "*.gcov" -delete
endif
GITIGNOREFILES ?=
GITIGNOREFILES += $(CODE_COVERAGE_OUTPUT_FILE) $(CODE_COVERAGE_OUTPUT_DIRECTORY)
A''M_DISTCHECK_CONFIGURE_FLAGS ?=
A''M_DISTCHECK_CONFIGURE_FLAGS += --disable-code-coverage
.PHONY: check-code-coverage code-coverage-capture code-coverage-capture-hook code-coverage-clean
']
AC_SUBST([CODE_COVERAGE_RULES])
m4_ifdef([_AM_SUBST_NOTMAKE], [_AM_SUBST_NOTMAKE([CODE_COVERAGE_RULES])])
])