Experimenting with modules in C++

One of the most awaited features of C++ are modules. One of the problems that modules addresses is that exactly the same headers files are compiled again and again during C++ developments (and some of them like <iostream> , could be extremely big). Caching those compilations will speed up compilation times dramatically.

Modules, in the LLVM implementation, are a generalization of precompiled headers. Precompiled headers are implemented by all the major C++ compiler vendors and they are simply a way of storing the compiler state to disk, so if a .cpp file has an associated precompiled header, the compiler can use this stored state to “fast forward” its state to the one it had after compiling the precompiled header, and skip the compilation of some headers. Because they are just storing the whole compiler state, only one precompiled header can be used per .cpp file.

Although precompiled headers could reduce compilation times dramatically, especially on Windows (where I experienced one order of magnitude improvement on one of my projects when I enabled them), they usually required a lot of manual work and they increase the chances of name collisions as they are usually shared by many files.

The latest versions of clang featured modules. Modules are a much better solution to reduce build times. They store the resulting Abstract Syntax Trees after the parsing of a header, that can be added incrementally to the current compiler state, instead of the whole state. Thusly, multiple modules can be loaded when compiling a translation unit, and a module’s header will be parsed once per language configuration rather than every time a dependant translation unit is recompiled.

Even if we do not use modules in our code, a module-enabled compiler like clang will still be faster than a traditional compiler, because it will still make use of one module, the std, skipping the textual inclusion of standard headers and using their binary representation instead.

You will need the LLVM suite, the clang compiler, the libc++ standard library and the libcxxabi to be able to experiment with modules. Because I wanted to experiment with the latest version, I checked out the trunk of all those projects, but the binary bundles of LLVM 3.7 have modules as well, so you may use them instead. I compiled the whole thing out of the source tree with:

cmake -DLLVM_PATH=../llvm -DLIBCXX_CXX_ABI=libcxxabi -DLIBCXX_CXX_ABI_INCLUDE_PATHS=../llvm/projects/libcxxabi/include -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ ../llvm/projects/libcxx


(note: for some reason a parallel build with make -j of llvm crash gcc on my machine so executed my build serially, but that is probably something wrong my version of gcc, 4.8.4)

Then I tried this simple example, which just makes use of the std module:

#include <iostream>
using namespace std;

int main()
cout &lt;&lt; &quot;Hello, world!!!&quot;;
return 0;
gerardo@GOLIATH:~/llvm_tests$ time clang++ -fmodules -fcxx-modules -fmodules-cache-path=./cache_path -stdlib=libc++ -lc++abi -L/home/gerardo/llvm.libcxxabi.release/lib helloworld.cpp

real    0m2.213s
user    0m2.148s
sys    0m0.064s

This is the first time I created the cache, which entails compiling the whole standard library and thus the long compilation times. If we go to ./cache_path:

total 4
drwxrwx--- 2 gerardo gerardo 4096 Sep 13 16:37 D9DWX79PA5LV
-rw-rw-r-- 1 gerardo gerardo    0 Sep 13 16:37 modules.timestamp&amp;amp;lt;/code&amp;amp;gt;

total 10144
-rw-rw-r-- 1 gerardo gerardo   225476 Sep 13 16:37 modules.idx
-rw-rw-r-- 1 gerardo gerardo 10154840 Sep 13 16:37 std-2WRNL6O46F2FW.pcm

we can see that a module std was compiled in the file std-2WRNL6O46F2FW.pcm.

(Note: on my machine I got this error:

/usr/local/bin/../include/c++/v1/future:515:23: error: a non-type template parameter cannot have type 'std::__1::future_errc'

when clang was generating the std module. I guess this is just a bug in the trunk that will be fixed soon. I disabled the module future at /usr/local/include/c++/v1/module.modulemap and the generation of module std worked)

Subsequent compilations are much faster, thanks to the module cache:

gerardo@GOLIATH:~/llvm_tests$ time clang++ -fmodules -fcxx-modules -fmodules-cache-path=./cache_path -stdlib=libc++ -lc++abi -L/home/gerardo/llvm.libcxxabi.release/lib helloworld.cpp&amp;amp;lt;/code&amp;amp;gt;

real    0m0.093s
user    0m0.080s
sys    0m0.014s

If we want to create our own modules, we have to use the module map language. For example, this module.modulemap file creates a module that exposes the C++ header “myheader.h”

module test {
requires cplusplus
header &quot;myheader.h&quot;;