Posts (summary)
Posts (full)

Typesafe variadic functions in c++11

Variadic functions are cool things, they’re used all over in c (and posix), like execl, printf. However they have problems: you do not know the number of arguments, nor their type. If you try to access an argument that the caller didn’t provide, you invoke undefined behaviour. You can use printf like format arguments, terminating NULL item, etc. But this is not enforced by the compiler, a small mistake in the calling code, and you’re a toast.

C++11 introduces variadic templates. You can use it for thing like perfect argument forwarding in stl container’s emplace methods. But you can even use it to make variadic functions. I didn’t seen articles about this usage of variadic templates, probably because before c++11 you get used to avoid variadic functions at all costs.

Uniform types

Let’s start with a simple example.

void foo() {}

template <typename... Args>
void foo(int x, Args... args)
{
    std::cout << x << std::endl;
    foo(args...);
}

And that’s it. You can call it like foo(1, 2, 3, 4), and it will print numbers from 1 to 4 to the stdout. Let’s see how it works.

Ignore the first line for a moment. You see a template. The typename... tells the compiler it’s a variadic template. When you call foo(1, 2, 3), x will be 1. Args type will be int and int, and args will contain the 2 and 3 numbers. But you can’t access them this way. This is why you need recursion.

foo(1, 2, 3) will call foo(2, 3), which then calls foo(3), which finally calls foo(). Due to how C++ name resolution in templates work, you must declare (or define) foo() before the template.

If you call foo(1, "bar", 3) (or anything that is not implicitly castable to int), you’ll get a compile time fail, instead of random runtime crash. And that’s good.

What if you want to print them in reverse order? Well, unfortunately you can’t write foo(Args... args, int x), but you can do:

template <typename... Args>
void foo(int x, Args... args)
{
    foo(args...);
    std::cout << x << std::endl;
}

If you want to access them in arbitrary order, things are a bit complicated. You can collect args in an array (like int ary[] = { args... };), and operate on ary afterwards. The alternative is to write a function that gets the nth parameter.

int nth(int) { throw std::runtime_error("..."); }

template <typename... Args>
int nth(int n, int x, Args... args)
{
    if (n == 0) return x;
    else nth(n-1, args...);
}

Some functional programming knowledge certainly helps here (for example, this is exactly how you get the nth element of a list in Haskell).

Non uniform types

Of course, you’re not forced to use ints. Just add another template argument. Here is a very simple printf like implementation:

void bar(const char* fmt)
{
    while (*fmt && *fmt != '%')
        std::cout.put(*fmt++);
    if (*fmt == '%')
        throw std::runtime_error("Too many %s");
}

template <typename T, typename... Args>
void bar(const char* fmt, T&& x, Args&&... args)
{
    while (*fmt && *fmt != '%')
        std::cout.put(*fmt++);
    if (*fmt == '%')
    {
        std::cout << std::forward<T>(x);
        bar(fmt+1, std::forward<Args>(args)...);
    }
    else
        throw std::runtime_error("Too many arguments");
}

There are no fancy formatting, %s are replaced with the arguments. Use it like bar("% %!\n", "hello", "world") to print out hello world!. And you can use anything that has operator<<(std::ostream&, thing). Like bar("% % %!\n", 4, 76.88, __FILE__). I also use std::forward to perfect forward arguments (although it’s probably unnecessary here…)

If you pass an invalid type, you’ll get a compile time error. If you pass different number of %s and arguments, you’ll get a run time error. That’s not ideal. But if the format string is not a compile time constant, that’s all we can do. However, can we get a compile time error if the format string is a compile time constant? It’s possible to parse the string compile time using constexpr functions. However there’s a problem: function’s arguments are never constant expressions, so you can’t static_assert them. This means you probably have to use a template argument to somehow tell the function how many arguments shall it have. The best way would be to pass the format string as a template arguments, but that’s not possible.1 The other alternative is to use some macro magic. Not pretty, and has its drawbacks, but works:

constexpr int strcnt(char c, const char* str)
{
    return *str ? (*str == c) + strcnt(c, str+1) : 0;
}

template <int n>
void bar2(const char* fmt)
{
    static_assert(n == 0, "Invalid number of arguments");
    std::cout << fmt;
}

template <int n, typename T, typename... Args>
void bar2(const char* fmt, T&& x, Args&&... args)
{
    static_assert(n == sizeof...(args) + 1, "Invalid number of arguments");

    while (*fmt != '%')
        std::cout.put(*fmt++);

    std::cout << std::forward<T>(x);
    bar2<n-1>(fmt+1, std::forward<Args>(args)...);
}

#define FIRST(x, ...) x
#define bar2_call(...) bar2<strcnt('%', FIRST(__VA_ARGS__))>(__VA_ARGS__)

The strcnt function counts the occurrences of a character in a string. Using the bar2_call macro, I count the number of %s and pass it as template argument n. With every recursion, I decrement n by one. At the end (the bar2(const char*) function) it should be 0, meaning the number of %s and arguments matched. If n is negative, there are extra arguments, if positive, extra %s. I also assert in the template that the numbers match, in gcc this prevents long template traces (as it will fail early in specialization). Clang continues after a failed assert, so it’ll generate a lot of errors. It may be a good idea to only leave the first assert if using clang to avoid flooding the error log

This also uses variadic macros, which is standard in C99 and C++11. I could define it as #define bar2_call(fmt, ...) bar2<strcnt('%', fmt)>(fmt, __VA_ARGS__), but the preprocessor will leave a trailing comma after fmt if you do not supply more arguments (i.e. bar2_call("foo") becomes bar2<strcnt('%', "foo")>("foo", ), which is invalid). That’s why the FIRST hack is there.

Of course this only works with a compile time constant format string. If you ever instantiate bar2 with n, but the format string has different number of %s, it’ll likely crash (if there are less %s), or print out extra %s.


This is, of course, a very simple printf like facility. You do not even have a way to escape %s, or to do any formatting, however it wouldn’t be very hard to add. Reordering is a bit more complicated, as you can’t access arbitrary argument. You can either collect the arguments somehow (if you don’t need fancy formatting you can just do something like std::string strs[] = { boost::lexical_cast<std::string>(args)... };, and you have everything in strs as an std::string). You could also do multiple recursions for each format directive, e.g. you can write something like this

void print_nth(int) { throw std::logic_error("..."); }

template <typename T, typename... Args>
void print_nth(size_t n, const T& x, const Args&... args)
{
    if (n == 0) std::cout << x;
    else print_nth(n-1, args...);
}

and this way you can print any argument in any order. Note that I’ve replaced rvalue references with const lvalue references, since we don’t want to move objects as we need them later. (Actually, it’s probably better to use lvalue references in previous cases aswell since we do no copying/moving…)

Now a more interesting example: let’s wrap boost::format into a variadic function. It turns out to be pretty simple:

template <typename... Args>
std::string boost_format(const std::string& fmt, Args&&... args)
{
    boost::format bf(fmt);
    boost::format x[] = { boost::format(), bf % args... };
    return bf.str();
}

That { bf % args... } trick does bf % arg for each arg (and collect the result in an array, but that’s hopefully optimized away by the compiler). The extra boost::format() is there, because declaring an empty array this way is invalid in C++ (see this stackoverflow question) (gcc and clang accepts empty arrays, but that’s an extension).

Conclusion

Typesafe variadic functions are possible with c++11. Working with parameter packs can be problematic, but if you only need simple iteration, you can use recursion; if you need to access parameters in arbitrary order, you can collect them in an array (if they have the same type) or use a helper function to get the nth argument. Here’s a generic version of the nth function introduced at the beginning:

template <typename T>
T get_nth(int) { throw std::runtime_error("..."); }

template <typename T, typename U, typename... Args>
typename std::enable_if<!std::is_convertible<U, T>::value, T>::type
get_nth(int n, const U&, const Args&... args);


template <typename T, typename U, typename... Args>
typename std::enable_if<std::is_convertible<U, T>::value, T>::type
get_nth(int n, const U& x, const Args&... args)
{
    if (n == 0) return x;
    else return get_nth<T>(n-1, args...);
}

template <typename T, typename U, typename... Args>
typename std::enable_if<!std::is_convertible<U, T>::value, T>::type
get_nth(int n, const U&, const Args&... args)
{
    if (n == 0) throw std::runtime_error("invalid type");
    else return get_nth<T>(n-1, args...);
}

And you can use it like get_nth<type>(n, args...). If the argument has an invalid type, it’ll result in an exception. And you can even take advantage of automatic c++ type conversions, for example get_nth<std::string>(0, "foo") works, even if "foo" passed as a const char array.

The downside? It’s a template


  1. Well, you can give “the address of an object with static storage duration and external or internal linkage” (C++11 14.3.2.1), but then you need const char format[] = "x % y\n"; (outside the function!), then you can do meaningful_function_name<format>(2, 4). But in this case, the format string and function invocation are at a completely different place in the code ruins readability. The other alternative is a vararg char template, like function<'x',' ','%',' ','\n'>(2, 4), probably even worse, especially with longer format strings. And it looks impossible to convert a string literal to template arguments, even with constexpr functions.

Comment!