C API GUIDE
===========
Complete guide to the Lingenic-Text C API: 53 functions covering the
full library surface, exported from a standalone static library.
Header: include/lingenic_text.h
Library: lib-c/liblingenic_text_c.a (build: gprbuild -P lingenic_text_c.gpr)
Test: tests/c/smoke_test.c (106 checks, all modules)
The C API is a thin validation shim over the proved SPARK/Ada core.
Every entry point checks its arguments against the precondition of the
proved subprogram it calls; violations return LT_ERR without invoking
the core. The shim is compiled with runtime checks enabled and a
catch-all exception handler, so no Ada exception can propagate into C.
The core itself is compiled with checks suppressed because GNATprove
has proved their absence (9,214 verification conditions, 0 unproved).
================================================================================
CONVENTIONS
================================================================================
Buffers
All text is UTF-8, passed as (const uint8_t *buf, size_t len).
Buffers are not NUL-terminated. The only NUL-terminated strings in
the API are the char* UCD path given to lt_init and the char* name
outputs of lt_script_name and lt_general_category_name.
Offsets
All positions are 0-based byte offsets into the buffer, C-style.
Return codes
LT_ERR (-1) invalid arguments (null pointer, zero/overlong length,
position out of range, bad enum value) or a required
module not initialized. Applies to every function.
Transform functions (normalize, case mapping, IDNA) additionally
distinguish recoverable failures:
0 success
1 output buffer too small -- retry with a larger buffer
2 invalid UTF-8 input
(IDNA uses its own richer LT_IDNA_* result codes; see below.)
Predicate functions return 1 (true), 0 (false), or LT_ERR.
Memory
The caller owns all buffers. The library copies data in and out and
never retains a pointer past the call.
Threading
Initialization (lingenic_text_cinit, lt_init) must complete before
any other call and must not run concurrently. Afterwards all lt_*
functions only read the loaded tables and use local state; they may
be called concurrently from multiple threads.
================================================================================
ELABORATION AND INITIALIZATION
================================================================================
void lingenic_text_cinit (void);
Runs Ada elaboration for the standalone library. Call exactly
once, before any other function. Generated by gprbuild.
void lingenic_text_cfinal (void);
Runs Ada finalization. Optional; call at process shutdown.
int lt_init (const char *ucd_dir);
Initializes all modules from the Unicode Character Database
directory (the directory containing UnicodeData.txt, allkeys.txt,
Scripts.txt, ...). Runs the seven initialization stages in
dependency order.
Returns 0 on success, or the number of the failed stage:
1 = properties 5 = bidi
2 = normalization 6 = case mapping
3 = emoji 7 = collation
4 = idna
Returns LT_ERR if ucd_dir is null or empty.
int lt_initialized (void);
1 if all modules are initialized, 0 otherwise.
================================================================================
UTF-8 (RFC 3629)
================================================================================
int lt_utf8_decode (const uint8_t *buf, size_t len, size_t pos,
uint32_t *cp, size_t *seq_len);
Decodes one UTF-8 sequence starting at pos.
Returns 1 if valid: *cp is the decoded scalar value and *seq_len
the sequence length (1..4). Returns 0 if the sequence at pos is
invalid: *seq_len is set to 1 so callers can resynchronize by
skipping one byte. Returns LT_ERR on bad arguments (null
pointers, len == 0, pos >= len).
Backed by the proved core: validity is exactly RFC 3629
well-formedness (surrogates and overlong encodings rejected), and
the decode/encode round trip is proved.
int lt_utf8_encode (uint32_t cp, uint8_t *buf, size_t cap,
size_t *len);
Encodes cp into buf (capacity cap), writing 1..4 bytes.
Returns 0 on success with *len set. Returns LT_ERR if cp is not
a Unicode scalar value (surrogate or > 0x10FFFF) or cap is too
small for the encoding.
Example: round trip
uint8_t b[4]; size_t n, sl; uint32_t cp;
lt_utf8_encode(0x1F600, b, 4, &n); /* n == 4 */
lt_utf8_decode(b, n, 0, &cp, &sl); /* cp == 0x1F600 */
================================================================================
SEGMENTATION (UAX #29, UAX #14)
================================================================================
int lt_next_grapheme_break (const uint8_t *text, size_t len,
size_t pos, size_t *next_pos);
int lt_next_word_break (const uint8_t *text, size_t len,
size_t pos, size_t *next_pos);
int lt_next_sentence_break (const uint8_t *text, size_t len,
size_t pos, size_t *next_pos);
int lt_next_line_break (const uint8_t *text, size_t len,
size_t pos, size_t *next_pos);
Given a boundary at pos (0 is always a boundary), computes the
next boundary after pos. On success *next_pos satisfies
pos < *next_pos <= len, and returns 0. Returns LT_ERR on bad
arguments (null pointers, len == 0, pos >= len) or when the
Properties module is not initialized.
Progress is proved in the core (the boundary strictly advances),
so the enumeration loop below always terminates:
for (size_t pos = 0, next; pos < len; pos = next) {
if (lt_next_grapheme_break(text, len, pos, &next) != 0)
break; /* cannot happen after init */
/* [pos, next) is one grapheme cluster */
}
Grapheme clusters implement GB1..GB999 including Indic conjunct
breaks (GB9c), extended pictographic runs (GB11) and regional
indicator pairing (GB12/13). Word boundaries implement WB1..WB999
with WB4 transparency and lookahead (WB6/WB7b/WB12). Sentence
boundaries implement SB1..SB11. Line breaking returns break
OPPORTUNITIES per UAX #14 LB1..LB31 (len is always the last
opportunity). Invalid UTF-8 bytes are treated as degenerate
single-byte codepoints rather than rejected.
================================================================================
NORMALIZATION (UAX #15)
================================================================================
Forms: LT_NFD (0), LT_NFC (1), LT_NFKD (2), LT_NFKC (3)
int lt_normalize (int form,
const uint8_t *in_buf, size_t in_len,
uint8_t *out_buf, size_t out_cap, size_t *out_len);
Normalizes in_buf to the given form.
Returns: 0 success, *out_len set
1 output buffer too small (retry with larger out_cap;
decomposed forms can expand the input several-fold)
2 invalid UTF-8 input
LT_ERR bad arguments / not initialized
The core postcondition is proved: on success the output satisfies
the recursive ghost predicate for the requested form (e.g.
Ghost_Is_NFD: fully decomposed, combining classes non-decreasing,
no undecomposed Hangul).
int lt_quick_check (int form, const uint8_t *in_buf, size_t in_len);
UAX #15 Section 9 quick check.
Returns LT_QC_YES (0), LT_QC_NO (1), LT_QC_MAYBE (2), or LT_ERR.
LT_QC_MAYBE means a definitive answer requires normalizing.
int lt_is_normalized (int form, const uint8_t *in_buf,
size_t in_len);
Definitive check (normalizes internally when quick check says
MAYBE). Returns 1 = normalized, 0 = not normalized, LT_ERR on
error or invalid UTF-8.
int lt_ccc (uint32_t cp);
Canonical Combining Class, 0..254. LT_ERR on error.
================================================================================
CASE MAPPING (Unicode Section 3.13)
================================================================================
int lt_uppercase (const uint8_t *in_buf, size_t in_len,
uint8_t *out_buf, size_t out_cap, size_t *out_len);
int lt_lowercase (...same signature...);
int lt_titlecase (...same signature...);
int lt_casefold (...same signature...);
Full case transformations with multi-character mappings
(SpecialCasing.txt, CaseFolding.txt) and Final_Sigma context.
Titlecase uses UAX #29 word boundaries. Output can be up to 3x
the input length; size out_cap accordingly.
Returns: 0 success, 1 output buffer too small, 2 invalid UTF-8,
LT_ERR bad arguments / not initialized.
Example: U+00DF LATIN SMALL LETTER SHARP S uppercases to "SS".
int lt_is_cased (uint32_t cp); /* D135 */
int lt_is_case_ignorable (uint32_t cp); /* D136 */
1/0, LT_ERR on error.
================================================================================
COLLATION (UTS #10, DUCET)
================================================================================
Options: LT_COLLATE_NON_IGNORABLE (0), LT_COLLATE_SHIFTED (1)
int lt_collate (const uint8_t *a, size_t a_len,
const uint8_t *b, size_t b_len,
int option, int *result);
Compares two UTF-8 strings with the Default Unicode Collation
Element Table. On success *result is -1 (a < b), 0 (equal under
the collation), or 1 (a > b), and the function returns 0.
Returns 1 if either input is invalid UTF-8 or exceeds internal
limits; LT_ERR on bad arguments / not initialized.
int lt_sort_key (const uint8_t *in_buf, size_t in_len, int option,
uint8_t *key, size_t key_cap, size_t *key_len);
Builds a binary sort key. Keys compare with memcmp in exactly
the same order lt_collate compares the source strings.
Returns 0 on success with *key_len set, 1 on failure (invalid
input or key buffer too small), LT_ERR on bad arguments.
Sort keys are several times longer than the source string; a
key_cap of 8x the input length is a practical starting point,
retrying on return code 1.
================================================================================
BIDIRECTIONAL ALGORITHM (UAX #9)
================================================================================
Directions: LT_DIR_AUTO (0) -- determine by rules P2/P3
LT_DIR_LTR (1)
LT_DIR_RTL (2)
Limits: paragraphs of at most 4096 codepoints; embedding levels
0..126.
int lt_bidi_resolve (const uint8_t *text, size_t len, int direction,
uint8_t *levels, size_t levels_cap,
size_t *num_cps, uint8_t *para_level);
Resolves embedding levels for one paragraph (rules P2 through I2,
including explicit embeddings, overrides, isolates, and bracket
pair resolution N0/BD16). levels receives one entry per
CODEPOINT, not per byte; levels_cap >= the codepoint count (len
is always sufficient since codepoints >= 1 byte). On success
returns 0 with *num_cps and *para_level set. Returns 1 if the
algorithm rejects the input (invalid UTF-8, more than 4096
codepoints); LT_ERR on bad arguments.
int lt_bidi_reorder (const uint8_t *levels, size_t num_cps,
uint8_t para_level,
uint32_t *order, size_t order_cap);
Computes the visual order (rule L2). order receives num_cps
0-based logical indices: order[i] is the logical index of the
codepoint displayed at visual position i. The core postcondition
proves the result is a permutation -- every logical index appears
exactly once. Returns 0 on success, 1 on failure, LT_ERR on bad
arguments (num_cps must be 1..4096, each levels[i] <= 126).
int lt_bidi_needs_mirror (uint32_t cp, uint8_t level);
Rule L4: 1 if cp should be depicted mirrored at the given
embedding level (Bidi_Mirrored and odd level), 0 if not, LT_ERR
on error.
Example: Hebrew string -- levels come back odd, visual order is
reversed relative to logical order.
================================================================================
EAST ASIAN WIDTH (UAX #11)
================================================================================
int lt_display_width (uint32_t cp);
Display width in fixed-pitch terminal cells: 2 for Wide and
Fullwidth codepoints, 1 otherwise. LT_ERR on error.
int lt_east_asian_width (uint32_t cp);
Raw property value: LT_EAW_NEUTRAL (0), LT_EAW_AMBIGUOUS (1),
LT_EAW_HALFWIDTH (2), LT_EAW_WIDE (3), LT_EAW_FULLWIDTH (4),
LT_EAW_NARROW (5). LT_ERR on error.
================================================================================
EMOJI (UTS #51)
================================================================================
int lt_is_emoji (uint32_t cp);
int lt_is_emoji_presentation (uint32_t cp);
int lt_is_emoji_modifier (uint32_t cp);
int lt_is_emoji_modifier_base (uint32_t cp);
int lt_is_emoji_component (uint32_t cp);
The five binary emoji properties from emoji-data.txt.
1/0, LT_ERR on error.
int lt_emoji_classify (const uint8_t *in_buf, size_t in_len);
Classifies a UTF-8 sequence (at most 256 bytes) as an emoji
sequence. Returns one of:
LT_EMOJI_NOT_EMOJI (0)
LT_EMOJI_CHARACTER (1) single emoji codepoint
LT_EMOJI_PRESENTATION_SEQ (2) emoji + VS16 (U+FE0F)
LT_EMOJI_KEYCAP_SEQ (3) base + VS16 + U+20E3
LT_EMOJI_MODIFIER_SEQ (4) modifier base + skin tone
LT_EMOJI_FLAG_SEQ (5) regional indicator pair
LT_EMOJI_TAG_SEQ (6) tag base + tag spec + cancel
LT_EMOJI_ZWJ_SEQ (7) elements joined by U+200D
or LT_ERR on error. Example: the US flag (U+1F1FA U+1F1F8)
classifies as LT_EMOJI_FLAG_SEQ.
================================================================================
IDENTIFIERS (UAX #31)
================================================================================
int lt_is_identifier_start (uint32_t cp); /* XID_Start */
int lt_is_identifier_part (uint32_t cp); /* XID_Continue */
1/0, LT_ERR on error.
================================================================================
IDNA (UTS #46)
================================================================================
Option bits (OR together; LT_IDNA_DEFAULT = all five):
LT_IDNA_CHECK_HYPHENS 0x01
LT_IDNA_CHECK_BIDI 0x02
LT_IDNA_CHECK_JOINERS 0x04
LT_IDNA_USE_STD3_RULES 0x08
LT_IDNA_VERIFY_DNS_LENGTH 0x10
Result codes:
LT_IDNA_SUCCESS 0
LT_IDNA_INVALID_INPUT 1
LT_IDNA_DISALLOWED_CODEPOINT 2
LT_IDNA_LABEL_TOO_LONG 3
LT_IDNA_DOMAIN_TOO_LONG 4
LT_IDNA_INVALID_HYPHEN 5
LT_IDNA_LEADING_COMBINING_MARK 6
LT_IDNA_NFC_FAILURE 7
LT_IDNA_BIDI_FAILURE 8
LT_IDNA_CONTEXTJ_FAILURE 9
LT_IDNA_PUNYCODE_FAILURE 10
LT_IDNA_BUFFER_OVERFLOW 11
LT_IDNA_STD3_FAILURE 12
LT_IDNA_TOO_MANY_LABELS 13
LT_IDNA_EMPTY_LABEL 14
int lt_idna_to_ascii (const uint8_t *in_buf, size_t in_len,
uint32_t options,
uint8_t *out_buf, size_t out_cap,
size_t *out_len);
Converts a UTF-8 domain name to ASCII-Compatible Encoding
(Punycode, RFC 3492) with nontransitional processing, ContextJ
rules (RFC 5892), and Bidi validation (RFC 5893).
Returns LT_IDNA_SUCCESS with *out_len set, another LT_IDNA_*
code on failure, or LT_ERR on bad arguments / not initialized.
int lt_idna_to_unicode (const uint8_t *in_buf, size_t in_len,
uint32_t options,
uint8_t *out_buf, size_t out_cap,
size_t *out_len);
Converts a domain name (ACE or UTF-8) to Unicode, UTF-8 output.
Same return convention.
Example: "bücher.de" <-> "xn--bcher-kva.de"
================================================================================
CHARACTER PROPERTIES (UAX #44, UAX #24)
================================================================================
All property lookups are O(1) reads of flat tables built at lt_init.
Script and General_Category use RUNTIME NAME TABLES: the index values
are data-derived (discovered while parsing the UCD files), stable
within one lt_init, but not compile-time constants. Compare against
names, not hard-coded indices:
char name[32];
int s = lt_script((uint32_t)'A');
lt_script_name(s, name, sizeof name); /* "Latin" */
int lt_script (uint32_t cp);
int lt_script_count (void);
int lt_script_name (int idx, char *buf, size_t cap);
lt_script_name copies the NUL-terminated name into buf and
returns its length, or LT_ERR if idx is out of range or the
buffer is too small.
int lt_script_extension_count (uint32_t cp);
int lt_script_extension (uint32_t cp, int k);
int lt_in_script_extensions (uint32_t cp, int script_idx);
Script_Extensions set: element count, k-th (0-based) script
index, and membership test (1/0).
int lt_general_category (uint32_t cp);
int lt_general_category_count (void);
int lt_general_category_name (int idx, char *buf, size_t cap);
General_Category with short names ("Lu", "Nd", ...); same
conventions as script.
Enumerated properties return fixed constants (see the header for the
complete lists):
int lt_grapheme_break_property (uint32_t cp); /* LT_GBP_* 0..13 */
int lt_word_break_property (uint32_t cp); /* LT_WBP_* 0..18 */
int lt_sentence_break_property (uint32_t cp); /* LT_SBP_* 0..14 */
int lt_line_break_property (uint32_t cp); /* LT_LBP_* 0..48,
resolved per LB1 */
int lt_east_asian_width (uint32_t cp); /* LT_EAW_* 0..5 */
int lt_bidi_class (uint32_t cp); /* LT_BC_* 0..23 */
int lt_joining_type (uint32_t cp); /* LT_JT_* 0..5 */
int lt_indic_conjunct_break (uint32_t cp); /* LT_INCB_* 0..3 */
Binary properties (1/0, LT_ERR on error):
int lt_extended_pictographic (uint32_t cp);
int lt_bidi_mirrored (uint32_t cp);
All property functions return LT_ERR if cp > 0x10FFFF or the
Properties module is not initialized.
================================================================================
COMPLETE EXAMPLE
================================================================================
#include <lingenic_text.h>
#include <stdio.h>
#include <string.h>
int main(void)
{
lingenic_text_cinit();
int rc = lt_init("ucd");
if (rc != 0) {
fprintf(stderr, "lt_init failed at stage %d\n", rc);
return 1;
}
/* 1. Normalize user input to NFC */
const uint8_t raw[] = { 'c', 'a', 'f', 'e', 0xCC, 0x81 };
uint8_t nfc[32]; size_t nfc_len;
lt_normalize(LT_NFC, raw, sizeof raw, nfc, sizeof nfc,
&nfc_len);
/* 2. Case-fold for caseless comparison */
uint8_t folded[96]; size_t folded_len;
lt_casefold(nfc, nfc_len, folded, sizeof folded, &folded_len);
/* 3. Count grapheme clusters (user-perceived characters) */
size_t clusters = 0;
for (size_t pos = 0, next; pos < nfc_len; pos = next) {
lt_next_grapheme_break(nfc, nfc_len, pos, &next);
clusters++;
}
printf("%zu clusters\n", clusters); /* 4: c a f é */
/* 4. Sort keys for a database index */
uint8_t key[256]; size_t key_len;
if (lt_sort_key(nfc, nfc_len, LT_COLLATE_SHIFTED,
key, sizeof key, &key_len) == 0) {
/* store key; ORDER BY memcmp == UTS #10 order */
}
/* 5. Registered domain name to punycode */
uint8_t ace[256]; size_t ace_len;
rc = lt_idna_to_ascii(nfc, nfc_len, LT_IDNA_DEFAULT,
ace, sizeof ace, &ace_len);
if (rc == LT_IDNA_SUCCESS)
printf("%.*s\n", (int)ace_len, ace); /* xn--caf-dma */
lingenic_text_cfinal();
return 0;
}
Build and run (macOS; from the repository root so "ucd" resolves):
ADALIB="$HOME/.local/share/alire/toolchains/\
gnat_native_15.1.2_60748c54/lib/gcc/aarch64-apple-darwin23.6.0/\
15.0.1/adalib"
cc example.c -Iinclude -Llib-c -llingenic_text_c \
"$ADALIB/libgnat.a" -o example
./example
On Linux, "-lgnat" (with -L pointing at the GNAT adalib directory)
replaces the explicit static archive.
================================================================================
RELATIONSHIP TO THE PROVED CORE
================================================================================
Each C function maps directly onto one proved Ada subprogram:
C function Proved subprogram
---------------------------- ------------------------------------
lt_utf8_decode/encode UTF8.Decode / UTF8.Encode
lt_next_grapheme_break Graphemes.Next_Grapheme_Cluster_Break
lt_next_word_break Words.Next_Word_Break
lt_next_sentence_break Sentences.Next_Sentence_Break
lt_next_line_break Line_Break.Next_Line_Break
lt_normalize Normalization.Normalize
lt_quick_check Normalization.Quick_Check
lt_is_normalized Normalization.Is_Normalized
lt_ccc Normalization.Get_CCC
lt_uppercase .. lt_casefold Case_Mapping.Uppercase / Lowercase /
Titlecase / Casefold
lt_collate, lt_sort_key Collation.Compare / Sort_Key
lt_bidi_* Bidi.Resolve_Levels / Reorder /
Needs_Mirror
lt_display_width EAW.Display_Width
lt_is_emoji*, lt_emoji_* Emoji properties / Classify_Sequence
lt_is_identifier_* Identifiers.Is_Identifier_Start/Part
lt_idna_to_* IDNA.To_ASCII / To_Unicode
lt_script*, lt_*_property... Properties.Get_* lookups
The shim (src-c/lingenic_text_c.adb, SPARK_Mode Off) performs exactly
three jobs: validate C arguments against the Ada preconditions,
convert between 0-based C offsets / C types and 1-based Ada indices /
Ada types, and translate Ada out-parameters and status enums into C
return codes. It adds no logic of its own. The proved postconditions
of the core -- round-trip correctness, boundary correctness against
the ghost Unicode rules, normalization-form predicates, permutation
validity -- therefore apply unchanged to every successful C call.
The full Ada contracts are documented in API_REFERENCE.txt and the
verification approach in VERIFICATION.txt.