Foreign Function Interface Overview

Overview

BridgeStan works by wrapping the Stan Model class generated by the Stan compiler in a C-compatible interface which exposes the desired functionality. BridgeStan does this by using the extern "C" linkage available in C++ to expose functions which are callable from C and C-compatible sources.

BridgeStan clients are then built around their language’s Foreign Function Interface (FFI).

This is a fairly portable solution, since the subset of the C language used in the bindings is small, and most major platforms employ one of the same C calling conventions, namely the “cdecl” convention. This allows the programs, even when they are compiled by different compilers, to talk to each other in an agreeable way, placing arguments and return values in standard locations for communication between languages.

Each of the BridgeStan clients is built around the C-compatible FFI provided by the host language. By sticking to a simple, C-level API, we can avoid writing language specific code required by higher level FFIs such as pybind or Rcpp. These provide additional functionality and somewhat “nicer” bindings, but at the cost of making the source code specific to the language you want to interface with.

Language-specific Notes

Python

The Python FFI documented in the standard library module ctypes.

Our usage is standard with one exception, which is we use the CDLL interface on all platforms, even Windows, due to the fact that BridgeStan models are compiled with MingGW’s gcc.

The NumPy module numpy.ctypeslib is also used for compatibility.

Julia

Julia’s FFI is documented in the Julia manual.

R

R features several built-in forms of foreign function interface. We use the most basic one, called .C, as this is the least dependent on R’s internals.

Documentation on the .C interface can be found by calling help(.C) in an R session. Equivalent documentation is available online, as well as an extended walk-through.

Note: One quirk of the .C interface is the requirement that all inputs and return values are passed by pointers. This is the reason for the bridgestan_R files in the source.

Rust

The Rust interface uses two crates in addition to the built-in FFI types. Some general guidance on Rust FFI can be found in the Rust Book.

These are bindgen, which generates the Rust bindings from the C headers, and libloading, which provides easy dynamic library loading functionality.

Care is taken to provide “safe” Rust wrappers so that users of the crate do not need to use the unsafe keyword.

General Problems

Allocated Memory

Generally speaking, memory allocated on one side of a language barrier must also be freed on that side. This means special consideration is needed to pass strings back and forth between the languages, and inspired some of the design decisions behind ideas like returning the parameter names as a comma separated list, rather than the more “natural” array of strings, and this is why error messages must be passed back to the library in order to be freed.

Output Streams

Printed output from the C++ code cannot always be easily captured in the higher-level language. This is particularly relevant for print statements in Stan, which are printed to the standard output stdout from C++. This does not, for example, correspond to the sys.stdout stream available from Python by default.

We tackle this problem through the use of an interface-provided callback function when necessary. See bs_set_print_callback() for more details.