% We're using KOMA Script to hand-tune footnotes and TOC appearance.
% It should be available in your texlive distribution,
% which is how most distros package LaTeX.
\documentclass[fontsize=11pt, parskip=half]{scrartcl}

% Margins: see http://practicaltypography.com/page-margins.html and
% http://practicaltypography.com/line-length.html
% We're aiming for 80-ish characters per line.
\usepackage[letterpaper, left=1.5in,right=1.5in,top=1in,bottom=1in]{geometry}

% Build with LuaTeX or XeLaTeX, modern Unicode-aware LaTeX engines
% that use system TTF and OTF font files.
\usepackage{fontspec}

% Use Garamond for body text,
% Roboto for sans serif text,
% and mononoki for monospace text.
% Feel free to pick your own fonts,
% or comment these lines out to use TeX's traditional Computer Modern.
\setmainfont[Ligatures=TeX, Numbers=Proportional,
             BoldFont=AGaramond Pro Semibold,
             BoldItalicFont=AGaramond Pro Semibold Italic
            ]{Adobe Garamond Pro}
\setsansfont[Ligatures=TeX, Scale=MatchUppercase,
             BoldFont={* Medium},
             BoldItalicFont={* Medium Italic},
            ]{Roboto}
\setmonofont[Scale=MatchLowercase]{mononoki}

\usepackage{microtype} % Font expansion, protrusion, and other goodness

% Use symbols for footnotes, resetting each page
\usepackage[symbol,perpage]{footmisc}
% Left flush footnotes. See the KOMA Script manual.
\deffootnote[1em]{1em}{1em}{\thefootnotemark}
% Set the width of the rule separating body text and footnotes
\setfootnoterule{0.6\textwidth}

% One of the only annoyances with Adobe Garamond is that the asterisk is already
% set in a "superscripted" form. Superscripting *that* makes it annoyingly small.
% To fix this, we have to redefine footnote marks so that they aren't superscript,
% then raise all the other symbols.
%
% Feel free to remove this if your body type doesn't have this peculiarity,
% but unfortunately many do.
% See http://tex.stackexchange.com/a/16241
%
% We use the Unicode symbols themselves (instead of \dagger, \ddagger, \P, etc.)
% because the latter fall back to Computer Modern/Latin Modern in some cases,
% (e.g., if you're using mathastext instead of unicode-math).
% Alternatively, you could use \textdagger, \textddagger, etc.,
% but this seems more concise.
\DefineFNsymbols*{tweaked}{%
    {*}%
    {\textsuperscript†}%
    {\textsuperscript‡}%
    {\textsuperscript{◊}}%
    {**}%
    {\textsuperscript{†}}%
    {\textsuperscript{‡‡}}%
    {\textsuperscript{◊◊}}%
}
\setfnsymbol{tweaked}
\deffootnotemark{\thefootnotemark}


% Use \punckern to overlap periods, commas, and quotes for a tighter look.
% Care should be taken to not make it too tight - f" and the like can overlap
% if you're not careful.
\newcommand{\punckern}{\kern-0.3ex}

% Use sans serif font for all section headers.
\addtokomafont{disposition}{\sffamily}
% Don't use a sans serif font for description labels.
\setkomafont{descriptionlabel}{\normalfont}

\setcounter{secnumdepth}{0} % Don't number sections

% For ISO-8601 dates (yyyy-mm-dd)
\usepackage{datetime}
\renewcommand{\dateseparator}{-}

\usepackage{shellesc} % See http://tex.stackexchange.com/a/313289
\usepackage{minted} % Syntax highlighting via Pygments

\title{C++ on Embedded Systems}
\author{Matt Kline}
\date{\usdate\today}

% Allows us to reference the last page in the footer
\usepackage{lastpage}

% See http://tex.stackexchange.com/a/68310
\makeatletter
\let\runauthor\@author
\let\runtitle\@title
\makeatother

% Custom footer
\usepackage{scrlayer-scrpage}
% Use the sans serif font for page numbers.
\setkomafont{pagefoot}{\sffamily\upshape\small}
\clearpairofpagestyles
\pagestyle{scrheadings}
% Set the footer to "<page num>/<total>"
\cofoot*{\thepage\,/\,\pageref{LastPage}} % \, is a TeX primitive for a thin space.

% Embed hyperlinks for web pages, the index, footnotes, and intra-document refs.
\usepackage{hyperref}
\usepackage{nameref}
% Hyperlink colors
\usepackage{xcolor}
\definecolor{darkGreen}{HTML}{009000}
\hypersetup{ % Blue web links, green intra-document links
    colorlinks=true,
    linkcolor=darkGreen,
    urlcolor=blue
}

% C++ looks nicer if the ++ is in a monospace font and raised a bit.
\newcommand{\cpp}{C\raisebox{0.2ex}{\texttt{++}}}

% Create an unbreakable string of text in a monospaced font.
% Useful for command --line --args
\newcommand{\monobox}[1]{\mbox{\texttt{#1}}}

% Typeset code sections without inter-line space.
\newcommand{\codesize}{\fontsize{11pt}{11pt}}

% Spend a bit more time to get better word spacing.
% See http://tex.stackexchange.com/a/52855
\emergencystretch=1em

\begin{document}
% Custom title:
\begin{center}
{\LARGE \textbf{\cpp{} on Embedded Systems}} \\

\bigskip
\normalsize
\runauthor{} \\
\href{mailto:matt@bitbashing.io}{matt@bitbashing.io}

February 26, 2017
\end{center}

This October, my team \href{https://www.flukenetworks.com}{at work} switched from C to
\cpp{} for embedded firmware development.
Many of its features, including classes, automatic resource cleanup,
parametric polymorphism, and additional type safety
are just as useful on an RTOS or bare metal as
they are on a desktop running a general-purpose OS.
Using \cpp{} lets us write safer, more expressive firmware.

\cpp{}'s automagic is a double-edged sword, however.
Some language features depend on system facilities that
we don't want to provide in embedded environments.\punckern\footnote{Dynamic memory
allocation is the simplest and most prevalent example.
We usually want to avoid it---at least after startup---on a real time
embedded system,
but exception handling and many other \cpp{} features demand it.}
Wrangling the toolchain can also be difficult.
We don't want to completely discard \monobox{libgcc} and
\monobox{libstdc++} since they provide vital facilities like
\monobox{memcpy}, atomic operations, and hardware-specific floating-point
functions, but we must avoid certain parts of them.

This guide is a short attempt to codify what we've learned while moving
our firmware to \cpp.
Hopefully it provides a solid primer.

\section{Setup}

\subsection{Building a toolchain}

The good news is that GCC works very well as a cross-compiler for all sorts of
targets, including the ARM systems we usually use for embedded
development.\punckern\footnote{Clang also seems to provide solid tools for
cross-compilation, but to date, it hasn't been tried for our firmware.}
While some variant can usually be installed from your Linux distribution's
package manager, it is \emph{highly} recommended that teams build and use
their own cross-compiler.
This has several advantages:
\begin{itemize}
\item By having the entire team use the same version of the same toolchain,
    everyone should get identical builds.
    This is quite helpful for debugging and testing.

\item The pace of compiler development has increased with the recent updates to
    \cpp{},
    and newer versions offer significantly improved code generation
    in some cases.\punckern\footnote{For an example,
    see \url{https://gcc.gnu.org/bugzilla/show_bug.cgi?id=69878}.}
    During the development of a previous project, we also ran into compiler
    bugs on older (4.8.x) versions which caused our system to
    crash.
\end{itemize}

Building an entire cross-compiler toolchain is usually an arduous task,
but we've had good success using
\href{http://crosstool-ng.org/}{\mbox{crosstool-NG}}.
It helps you configure
your toolchain using an interface similar to Linux's \monobox{make nconfig},
manages and downloads dependencies, and performs the builds for you.
Recent versions also allow you to provide arbitrary GCC sources,
such as its latest release.
The resulting binaries can be statically linked,
so deployment becomes as simple as tarballing the toolchain
and placing it somewhere accessible.
Projects using the toolchain can then use a short script to pull it down,
extract it, and run it.

\subsection{Linking C code}

Embedded projects often have plenty of C dependencies, such as
manufacturer-provided drivers and the RTOS.
Build them using \texttt{gcc} and wrap any of their headers you
\mintinline{cpp}{#include} with \mintinline{cpp}{extern "C" { }}.
Similarly, any \cpp{} functions that you want to call from a non-\cpp{}
environment---e.g., RTOS functions or startup assembly---must be tagged
with \mintinline{cpp}{extern "C"}.
This instructs the compiler not to
\href{https://en.wikipedia.org/wiki/Name_mangling}{mangle} symbol names as it
normally would.

\subsection{Compiler flags}

Exception handling and RTTI are difficult to provide without dynamic memory
allocation (much more on that below), so you likely want to disable them with
\monobox{-fno-exceptions}, \monobox{-fno-non-call-exceptions},
and \monobox{-fno-rtti}.
And while it may reboot,
firmware never exits in the same sense as a userspace program does.
Teardown code (including global destructors)
can be omitted with \monobox{-fno-use-cxa-atexit}.
Other useful flags for embedded development include:
\begin{description}
\item[\monobox{-ffreestanding},] which indicates that your program exists in an
    environment where standard library facilities may be absent and where your
    program may not begin at \texttt{main()}.
\item[\monobox{-fstack-protector-strong},] which is discussed later.
\item[\monobox{-fno-common},] which ensures that each global variable is only
    declared once, in a single object.
    This may improve performance on some targets.
\item[\monobox{-ffunction-sections} and \monobox{-fdata-sections},]
    which split functions and data into their own
    \href{https://en.wikipedia.org/wiki/Executable_and_Linkable_Format}{ELF}
    sections.
    This allows the linker to eliminate additional unused code when passed
    \monobox{--gc-sections}.
\end{description}
These aren't specific to \cpp, but are worth mentioning here.

\section{Enabling language features}

As mentioned above, several useful \cpp{} features require underlying system support.
In a bare metal or RTOS environment, we'll need to provide it ourselves.

\textbf{Disclaimer:} Most of this is, of course, implementation-specific.
Everything that follows is based on our experiences using ARM Cortex-M4 boards
with GCC~6.
Hopefully this provides a useful starting point, even if details change.

\subsection{Global object initialization}

Global objects can be quite useful for defining interfaces to hardware resources
in an embedded system, and these objects might have constructors.
The \cpp{} runtime normally guarantees that all global
(or file-local) objects are constructed before entering \monobox{main()},
but in an embedded environment, we must call the constructors ourselves.
GCC groups them into an array of function pointers under the symbol name
\monobox{.init\_array}.
After adding an entry to the linker script resembling:
{\codesize\selectfont
\begin{verbatim}
. = ALIGN(4);
.init_array :
{
    __init_array_start = .;
    KEEP (*(.init_array*))
    __init_array_end = .;
} > FLASH
\end{verbatim}
}% end of \codesize font
\noindent we can call the functions like so:
\begin{minted}[fontsize=\codesize]{cpp}
static void callConstructors()
{
    // Start and end points of the constructor list,
    // defined by the linker script.
    extern void (*__init_array_start)();
    extern void (*__init_array_end)();

    // Call each function in the list.
    // We have to take the address of the symbols, as __init_array_start *is*
    // the first function pointer, not the address of it.
    for (void (**p)() = &__init_array_start; p < &__init_array_end; ++p) {
        (*p)();
    }
}
\end{minted}

When to perform this step is another question.
Should it be after hardware initialization?
After RTOS setup, but before your tasks begin executing?
In the first RTOS task?
Depending on what you decide, it may be prudent to ensure these constructors
don't make OS calls or modify hardware state---besides RAM, of course.

\subsection{Inheritance}

Judicious use of inheritance and run time polymorphism can be quite useful
on embedded systems.
However, \mintinline{cpp}{operator delete} is required whenever we give a base
class a virtual destructor---as is standard practice---even if we never
heap-allocate an object of that
class.\punckern\footnote{\raggedright See \url{http://stackoverflow.com/q/31686508}
and \url{http://eli.thegreenplace.net/2015/c-deleting-destructors-and-virtual-operator-delete/}
for details.
Kind readers have pointed out that we should avoid virtual destructors
altogether if we don't need polymorphic deletion.
Adding one increases the size of the class, since a vtable is needed.}
The \monobox{libgcc} versions assume a Unix-like userspace,
so we should define our own.
If we're trying to avoid dynamic memory allocation,
a call to \mintinline{cpp}{delete} is probably a serious bug,
and we should likely panic.
\begin{minted}[fontsize=\codesize]{cpp}
void operator delete(void* p)
{
    DIE("delete called on pointer %p (was an object heap-allocated?)", p);
}

// Same as above, just a C++14 specialization.
// (See http://en.cppreference.com/w/cpp/memory/new/operator_delete)
void operator delete(void* p, size_t t)
{
    DIE("delete called on pointer %p, size %zu", p, t);
}
\end{minted}
If objects with virtual destructors are stack-allocated,
a version of the destructor without \mintinline{cpp}{operator delete} is used.

\section{Unrecommended language features}

\subsection{Scoped, static objects}

Consider some function with a \mintinline{cpp}{static} variable:
\begin{minted}[fontsize=\codesize]{cpp}
void foo()
{
    static Bar someObject;
    // Do some work with someObject here.
}
\end{minted}

If the object can be trivially initialized via placement in
\monobox{.data} or \monobox{.bss} sections, there is no problem here.
The trouble arises if a constructor must be called at
run time to initialize \texttt{someObject}.
\cpp11 guarantees that the construction of local static objects is race-free.
That is, if multiple threads call \monobox{foo()} at once,
the compiler must provide some locking mechanism\footnote{See
\url{http://preshing.com/20130930/double-checked-locking-is-fixed-in-cpp11/}
for the usual approach.}
to ensure that the object's initial value is decided by a single thread.

Since we might call functions in our system \emph{before} the OS
is running and can provide meaningful locking,
and since we generally want to do all initialization in an embedded system at
startup to make subsequent code as deterministic as possible,
it's instead recommended that any static objects are placed at the file level.
Alternatively, one can compile with \monobox{-fno-threadsafe-statics} if they
are \emph{positive} that functions with static objects will not be called
concurrently.


\subsection{Exceptions}

Modern exception-handling mechanisms are complex,\punckern\footnote{See
 \url{https://mentorembedded.github.io/cxx-abi/abi-eh.html} for the full spec
 used by GCC and Clang.}
and most implementations---including \monobox{glibc}'s---requires dynamic memory
allocation and other facilities we don't have.
Third-party solutions like
\href{http://www.nongnu.org/libunwind/}{\monobox{libunwind}}
also assume they sit atop some Unix-like userspace.
Because of these complexities,
we haven't attempted using exceptions for our embedded projects.

If you are interested in overcoming these hurdles,
work Rian Quinn presented at
\mbox{CppCon 2016}\footnote{The presentation can be found at
\url{https://www.youtube.com/watch?v=uQSQy-7lveQ}.}
seems to be a good starting point.
In order to run \cpp{} code in the Linux kernel,
he built his own stack unwinding library, which can be found at
\url{https://github.com/Bareflank/hypervisor/tree/master/bfunwind}.

\section{Miscellaneous tools and notes}

\subsection{Stack overrun detection}

Consider a function that allocates some storage on the stack:
\begin{minted}[fontsize=\codesize]{cpp}
void bar()
{
    char arrayOnStack[10];
    // ...read and write to the local array...
}
\end{minted}
If we walk off the end of the array,
we can corrupt the memory following the current stack frame,
enter Undefined Behavior land, and can't reason about the state of our program
anymore.
Fun is not had.
Fortunately, the compiler can provide a slight safety net
at the cheap cost of one write and one read per function.
When given one of the flags below,
GCC transforms the function above into something resembling the following:
\begin{minted}[fontsize=\codesize]{cpp}
void bar()
{
    // Assuming the stack grows down, out-of-bounds writes to arrayOnStack
    // may clobber _canary.
    uintptr_t _canary = __stack_chk_guard;

    char arrayOnStack[10];
    // ...read and write to the local array...

    if (_canary != __stack_chk_guard) {
        // We have done terrible things to our stack. Panic.
        __stack_chk_fail();
    }
}
\end{minted}

Different flags offer control over how often the compiler adds these checks.
To save you a trip to the man pages,
\begin{description}
\item[\monobox{-fstack-protector}] emits guards for
    ``functions that call \texttt{alloca},
    and functions with [character] buffers larger than 8 bytes.\punckern''

\item[\monobox{-fstack-protector-strong}] emits guards for
    ``[functions] that have local array definitions,
    or have references to local frame addresses.\punckern''
    This is the generally recommended setting.

\item[\monobox{-fstack-protector-all}] emits guards for every function.
  This is probably overkill and too expensive for an embedded system.

\item[\monobox{-fstack-protector-explicit}] emits guards for functions marked by
    a \verb|stack_protect| attribute.
    This is probably too tedious to use effectively.
\end{description}

To use these guards,
we must provide both of the symbols shown in the example above.
One is the canary, and the other is a function to call when stack corruption
is detected.
Panicking or rebooting is probably the only sane recourse here.
\begin{minted}[fontsize=\codesize]{cpp}
extern "C" {

// The canary value
extern const uintptr_t __stack_chk_guard = 0xdeadbeef;

// Called if the check fails
[[noreturn]]
void __stack_chk_fail()
{
    DIE("Stack overrun!");
}

} // end extern "C"
\end{minted}

The compiler handles the rest.
It's worth noting that if we are particularly unlucky
and manage to read or write past the current stack frame \emph{without}
modifying the canary, this system won't notice the corruption.
Life is hard sometimes.

\subsection{On inlining and optimization}

In embedded systems, we're often under pressure to keep our code as small as
possible.
Inlining may seem antithetical to that goal, but this isn't always the case.
Placing trivial code in headers as \mintinline{cpp}{inline} functions allows
modern \cpp{} compilers to generate incredibly small and efficient output.
See Jason Turner's \emph{Rich Code for Tiny Computers} talk from
\mbox{CppCon 2016}\footnote{See \url{https://www.youtube.com/watch?v=zBkNBP00wJE}}
for an extreme example.

If you're building firmware with \monobox{-Os} (i.e., optimize for size),
consider adding
\begin{description}
\item[\monobox{-finline-small-functions},]
    which inlines functions whenever the compiler thinks the function body is
    smaller than the size of the call sites.

\item[\monobox{-findirect-inlining},] which runs additional inlining passes.
    For example, if \monobox{main()} calls \monobox{a()}
    and \monobox{a()} calls \monobox{b()},
    this compiler could fold the body of \monobox{a()} into \monobox{main()},
    then notice that \monobox{b()} is also a good candidate for inlining
    and fold it in.
    Serious improvements can be gained through such second-order effects.
\end{description}

Current projects are built using \monobox{-O2}, which provides
``nearly all supported optimizations that do not involve a space-speed
tradeoff.\punckern''
As always, test, test, and test some more!
ARM and other RISC architectures produce particularly readable
disassembly---examine it to see what the compiler generates when given
different options.
For quick experiments,
try the \href{https://godbolt.org/}{Godbolt compiler explorer},
which colors the disassembly to show which lines of code generated it.

\vspace{\baselineskip}
Good luck and godspeed!
\end{document}
