Tcl CFFI package (v1.2.0)

CookbookTop, Main, Index

While using CFFI to wrap a few ad-hoc functions is fairly straightforward, wrapping a large API into a complete package can be more involved as decisions how C types are translated to script level annotations. This page provides some recipes for mapping C declarations to CFFI declarations depending on the declaration context (parameter, struct field etc.).

Defining a functionTop, Main, Index

A function definition consists of three components.

Figure 1. Function definition syntax

Sections below detail definition of each.

Calling conventionTop, Main, Index

Check the header file or documentation of the function for the expected calling convention. Generally, this is only an issue if the application supports the 32-bit Windows platform. Note the function calling convention may be defined through a macro such as WINAPI etc. Most Windows OS API's are defined using the stdcall convention. The default C language calling convention is either unspecified or _cdecl in C header files. Most third party API's are defined using this convention. CFFI does not support any calling convention other than these two.

Figure 2. Calling convention syntax

Examples

C

WINAPI DWORD GetCurrentProcessId(void);
int getpid(void);

CFFI

KERNEL32 stdcall GetCurrentProcessId ulong {}
LIBC function getpid int {}

Note WINAPI in the Windows SDK headers maps to __stdcall.

Return type declarationTop, Main, Index

Void returnsTop, Main, Index

Figure 3. Syntax for void return type

The void return type does not permit any annotations.

Examples

C

void tzset(void)

CFFI

LIBC function tzset void {}

Integer returnsTop, Main, Index

Figure 4. Syntax for integer return types

Examples

C

int open(const char *pathname, int flags);

CFFI

A basic definition assuming LIBC is the library wrapper object:

LIBC function open int {pathname string flags int}

A more complete definition:

LIBC function open {int nonnegative errno} {pathname string flags int}

Floating point returnsTop, Main, Index

Figure 5. Syntax for floating point return types

Floating point types float and double do not permit any annotations on return type declarations.

Examples

C

double sin(double x);

CFFI

LIBM function sin double {x double}

Pointer returnsTop, Main, Index

Pointer return values may be dealt with at the script level either as raw pointers or as implicitly dereferenced values.

The type declarations for raw and implicitly dereferenced pointers are separately described below.

Raw pointers
Figure 6. Syntax for raw pointer return type

Examples

C

void *malloc(size_t sz);
FILE *fopen(const char *pathname, const char *mode);
git_tree_entry *git_tree_entry_byname(git_tree *, const char *filename);

CFFI

LIBC malloc {pointer errno} {sz size_t}
LIBC fopen {pointer.FILE errno} {pathname string mode string}
LIBGIT git_tree_entry_byname {pointer.git_tree_entry unsafe} {
    pTree {pointer.git_tree counted}
    filename string
    }

The unsafe and counted annotations on git_tree_entry_byname arise from the libgit2 API. The returned pointer is internal to libgit2 and hence marked as unsafe while git_tree structs are reference counted.

Implicitly dereferenced pointers

Raw pointers at script level, while as versatile as their C counterparts, are not as convenient to use as plain old values. Additional steps are needed to access the memory as script level values. Declaring pointer return types as implicitly dereferenced obviates the need for this additional step. As stated earlier, implicit dereferencing should only be done when there is no need for the pointer value itself for freeing later.

For types other than pointers to character (or unicode) strings, the CFFI declaration has the form

Figure 7. Syntax for implicitly dereferenced pointers

where TYPE may be one of the numeric types or a STRUCT. The byref annotation on the return type declaration denotes that the function return value is nominally a pointer that references the true function result.

Examples

C

struct git_oid *git_tree_oid(git_tree *pTree);

CFFI

LIBGIT function git_tree_oid {struct.git_oid byref} {pTree pointer.git_tree}

A call to the git_tree_oid function will then return the git_oid struct (assumed defined previously with ::cffi::Struct) in its script level dictionary form. The actual C level pointer is not accessible but not needed as it is internal to the git_tree and not to be freed or manipulated.

As for raw pointers, a null pointer return will raise an exception by default. The nullok annotation may suppress the exception in the case that TYPE is a struct with defaults defined for all fields (possibly with the -clear option). In that the dictionary corresponding to a struct value with all fields defaulted is returned. For all other types and structs with at least one field without a default, an exception is raised irrespective of nullok.

For C pointers typed as char * which return pointers to character strings, the implicit dereference syntax is

Figure 8. Syntax for character string return types

The byref is not present because string and unistring are already implicit pointers, (string byref would correspond to char **, not char *). An optional nullok annotation will cause a NULL pointer return to be mapped to the empty string instead of generating an exception.

Note that if STRING is string, an encoding identifier may be optionally attached, e.g. string.utf-8.

Examples

C

const char *strerror(int);
char *strstr(const char *haystack, const char *needle);

CFFI

LIBC function strerror string {error_code int}
LIBC function strstr {string nullok} {haystack string needle string}

Note the definition of strstr to return an empty string if the needle is not found instead of raising an exception.

Struct returnsTop, Main, Index

Figure 9. Syntax for struct return type

A struct function return type does not permit any annotations. STRUCT must be a previously defined ::Struct. Returning a struct by value is rarely seen in C API's.

Examples

C

typedef struct {
    int quot;
    int rem;
} div_t;
div_t div (int numer, int denom);

CFFI

Struct create div_t {quot int rem int}
LIBC function struct.div_t {numer int denom int}

Parameter declarationTop, Main, Index

Parameters to a C function may be used to pass values to the function (input parameters), retrieve values from the function (output parameters), or (input-output parameters).

Scalar values and structs that are passed by value are always input parameters. Input parameters may also be passed by reference via a pointer, generally with a const attribute to indicate the function does not modify the referenced value.

Output and input-output parameters are always passed to C function as pointers to a location where the value is stored. At the script level the argument is the name of the variable where the value is stored and not the value itself.

Integer parametersTop, Main, Index

Integer input parameters
Figure 10. Syntax for integer input parameters

Examples

C

int SetHandleInformation(void * hObject, unsigned long dwMask, unsigned long dwFlags);

CFFI

cffi::enum define HANDLE_FLAGS {INHERIT 1 PROTECT_FROM_CLOSE 2}
KERNEL32 stdcall SetHandleInformation {int nonzero lasterror} {
    hObject pointer
    dwMask {ulong {enum HANDLE_FLAGS} bitmask}
    dwFlags {ulong {enum HANDLE_FLAGS} bitmask}
}
Output and input-output integer parameters
Figure 11. Syntax for integer output and input-output parameters

Examples

C

int GetHandleInformation(void*  hObject, unsigned long *lpdwFlags);

CFFI

cffi::enum define HANDLE_FLAGS {INHERIT 1 PROTECT_FROM_CLOSE 2}
KERNEL32 stdcall GetHandleInformation {int nonzero lasterror} {
    hObject pointer
    flags {ulong retval {enum HANDLE_FLAGS} bitmask}
}

Floating point parametersTop, Main, Index

Floating point input parameters
Figure 12. Syntax for floating point input parameters

Examples

C

double sin(double x);

CFFI

LIBM sin double {x double}
Out and in-out floating point parameters
Figure 13. Syntax for floating point output and input-output parameters

Pointer parametersTop, Main, Index

As for return values, pointer parameters to C functions can be dealt with either as raw pointers or as implicitly dereferenced values.

In general, pointers that are only used in the C API to pass arguments by reference, for example to pass large or non-scalar input values (as for structs and arrays) or receive output values, can be declared as implicitly dereferenced. The CFFI declarations for these is described in other sections with use of the byref and out annotations.

This section deals only with raw pointers where the pointer values are directly visible at the script level. This is necessary when the pointers reference resources that need to freed, or in more complex structures in memory with multiple indirections etc.

Raw pointer input parameters
Figure 25. Syntax for input raw pointer parameters

Examples

C

void free(void *ptr);
void *realloc(void *ptr, size_t size);

CFFI

LIBC function free void {ptr {pointer dispose}}
LIBC function realloc {pointer errno} {
    ptr {pointer disposeonsuccess}
    size size_t
}
Raw pointer out and in-out parameters
Figure 15. Syntax for output and input-output raw pointer parameters

Examples

C

int git_repository_open_bare(git_repository **out, const char *bare_path);
int ConvertSidToStringSidA(SID *pSid, char **strSid);

CFFI

LIBGIT2 function git_repository_open_bare {int zero} {
    out {pointer.git_repository counted retval}
    string bare_path
}
ADVAPI32 stdcall ConvertSidToStringSidA {int nonzero lasterror} {
    pSid   pointer.SID
    strSid pointer
}
KERNEL32 stdcall LocalFree {pointer nullok} {p {pointer dispose}}

In the libgit2 example, out is annotated as counted because git_repository is a reference counted opaque structure. The retval annotation is added to have CFFI return the output parameter value as the result of the function.

In the Win32 example, the returned pointer from ConvertSidToStringSidW needs to be freed with LocalFree and hence needs to be declared as a raw pointer. The string can be retrieved from the returned pointer with ::cffi::memory tostring before freeing it.

Struct parametersTop, Main, Index

Struct values are represented as the script level as dictionaries. This requires their definition as described in Structs and Defining structs.

Struct input parameters
Figure 16. Syntax for struct input parameters

Examples

C

int git_checkout_head(git_repository *repo, const git_checkout_options *opts);

CFFI

cffi::Struct create git_checkout_options {...}
LIBGIT function git_checkout_head {int zero} {
    pRepo pointer.git_repository
    opts  {struct.git_checkout_options byref nullifempty}
}
Output and input-output struct parameters
Figure 17. Syntax for struct output and input-output parameters

Examples

C

int GetWindowRect(void *hWnd, RECT *lpRect);

CFFI

cffi::Struct create RECT {left int top int right int bottom int}
LIBUSER32 stdcall GetWindowRect int {hwnd pointer rect {struct.RECT out}}

String parametersTop, Main, Index

Input string parameters
Figure 18. Syntax for input character string parameters

Examples

C

int chdir(const char *path);

CFFI

LIBC function chdir {int zero errno} {path string}
Output and input-output string parameters

Character strings are returned from C functions in one of two ways:

The two are handled differently in CFFI as described below.

Output character arrays

Figure 19. Syntax for character array output parameters

This is the first case described above where the C function expects a pointer to a character (or Tcl_UniChar in the case of unichars) array.

Examples

C

char *getcwd(char *buf, size_t size);

CFFI

LIBC function getcwd {string errno} {buf {chars[size] out} size size_t}

Output pointers to strings

The second case is when the C function parameter is a pointer to pointer to a character string (char **).

How this is best handled depends on whether the returned pointer needs to be available at the script level as a raw pointer, possibly for later freeing. See Pointer returns for a more detailed discussion about raw pointers versus implicit dereferencing. If this is the case, see Raw pointer out and in-out parameters for appropriate declarations.

If retention of the raw pointer is not required (for example, pointer to static storage), then its most convenient to implicitly declare it as an output parameter of

Figure 20. Syntax for output character string parameters

Binary string parametersTop, Main, Index

Binary strings are similar to strings except that instead of treating pointers at the C level as character strings in Tcl, they treat them as binary strings so there is no character encoding/decoding transformation applied and nul characters have no special treatment.

Input binary string parameters
Figure 21. Syntax for input binary string parameters

Examples

C

ssize_t write(int fd, const void *buf, size_t count);

CFFI

LIBC function write {ssize_t nonnegative errno} {
    fd int
    buf binary
    count size_t
}
Output and input-output binary string parameters

Binary strings are returned from C functions in one of two ways:

The two are handled differently in CFFI as described below.

Output byte arrays

Figure 22. Syntax for byte array output parameters

This is the first case described above where the C function expects a pointer to a byte array.

The difference between an array declared as bytes[N] versus schar[N] or uchars[N] is that while both might be arrays of char or unsigned char at the C level, the former declaration will result in a Tcl binary string at the script level while the latter declarations will result in lists of integers.

Examples

C

ssize_t read(int fd, void *buf, size_t count);

CFFI

LIBC function read {ssize_t nonnegative errno} {
    fd int
    buf bytes[count]
    count size_T
}

Output pointers to binary strings

Another way of returning binary data is through a parameter that is a pointer to pointer to the data (char ** etc.). This case has to be handled as raw pointers since, unlike for character strings, there is no way for CFFI to know the size of the data being returned.

Arrays as parametersTop, Main, Index

TYPE[N] type specific annotations ...

Defining structsTop, Main, Index

Native C struct values may be dealt with at the script level either as Tcl dictionary values or kept in their native form and accessed through raw pointers. Both methods require the struct to be defined with ::cffi::Struct. In the former case, the fields are accessed using Tcl's dict command. In the latter case, the ::cffi::Struct accessor methods are used.

Field type declarationsTop, Main, Index

The field type declarations in a struct definition, while similar to function parameter type declarations, differ from them in some respects.

Integer fieldsTop, Main, Index

Figure 23. Syntax for integer fields

Floating point fieldsTop, Main, Index

       +-         -+
float  |{default N}|
double +-         -+
Figure 24. Syntax for floating point fields

Pointer fieldsTop, Main, Index

Figure 25. Syntax for input raw pointer parameters

String pointer fieldsTop, Main, Index

Figure 26. Syntax for string pointer fields

As for function parameters, fields that hold pointers to character strings can be more conveniently declared as one of the CFFI string types if the raw pointer is not required for freeing of resources or other purposes. There are is a restriction imposed when a struct field is declared as a string type. The struct definition may be used to pass and return parameters from functions but cannot be used with the ::cffi::Struct.new or ::cffi::Struct.allocate methods to allocate a native struct in memory.

Character array fieldsTop, Main, Index

Figure 27. Syntax for character array fields

Like string and unistring, the chars and unichars CFFI types are character strings at the script level. However, while the former maps to pointers at the C level, chars and unichars map to char[] and Tcl_UniChar[] arrays.

Byte array fieldsTop, Main, Index

Figure 28. Syntax for byte array fields

The difference between an array declared as bytes[N] versus schar[N] or uchars[N] is that while both might be arrays of char or unsigned char at the C level, the former declaration will result in a Tcl binary string at the script level while the latter declarations will result in lists of integers.

Nested struct fieldsTop, Main, Index

A struct definition may include a field that is declared as another struct.

Figure 29. Syntax for nested struct fields

Array fieldsTop, Main, Index

TYPE[N] type specific annotations ...

Tips and tricksTop, Main, Index

This section to be written