TESTING ======= This document describes the conformance test infrastructure, test programs, and test data files used to validate Lingenic-Text against the Unicode standard. PURPOSE ------- Conformance tests form the second chain in the double proof chain. While formal proofs verify that the implementation matches the ghost specification, conformance tests verify that the ghost specification correctly encodes the Unicode standard. Every test program parses an official Unicode Consortium test file and compares the library's output against expected results. TEST PROGRAMS ------------- 12+ test programs, each with a dedicated GPR project file in tests/: Test Program UCD Test File Module Tested ----------------------------------------------------------------------- test_graphemes GraphemeBreakTest.txt Graphemes test_words WordBreakTest.txt Words test_sentences SentenceBreakTest.txt Sentences test_line_break LineBreakTest.txt Line_Break test_normalization NormalizationTest.txt Normalization test_case_mapping (custom test data) Case_Mapping test_collation CollationTest/ Collation test_bidi BidiCharacterTest.txt Bidi test_emoji emoji-test.txt Emoji test_properties (custom property tests) Properties test_idna IdnaTestV2.txt IDNA test_grapheme_props (GBP property tests) Properties (GBP) debug_collation (debugging tool) Collation BUILDING TESTS -------------- Each test has its own GPR file: gprbuild -P tests/test_graphemes.gpr gprbuild -P tests/test_words.gpr gprbuild -P tests/test_sentences.gpr gprbuild -P tests/test_line_break.gpr gprbuild -P tests/test_normalization.gpr gprbuild -P tests/test_case_mapping.gpr gprbuild -P tests/test_collation.gpr gprbuild -P tests/test_bidi.gpr gprbuild -P tests/test_emoji.gpr gprbuild -P tests/test_properties.gpr gprbuild -P tests/test_idna.gpr gprbuild -P tests/test_grapheme_props.gpr Binaries are placed in tests/obj/. RUNNING TESTS ------------- Run from the project root directory (tests expect ucd/ to be relative): ./tests/obj/test_graphemes ./tests/obj/test_words ./tests/obj/test_sentences ./tests/obj/test_line_break ./tests/obj/test_normalization ./tests/obj/test_case_mapping ./tests/obj/test_collation ./tests/obj/test_bidi ./tests/obj/test_emoji ./tests/obj/test_properties ./tests/obj/test_idna ./tests/obj/test_grapheme_props Each test reports pass/fail counts and exits with status 0 on full pass. TEST FILE FORMATS ----------------- Segmentation Tests (GraphemeBreakTest.txt, WordBreakTest.txt, SentenceBreakTest.txt, LineBreakTest.txt): Format: division (/) and multiplication (x) signs between codepoints: / 000D x 000A / 0061 / # comment / = break (boundary exists here) x = no break (no boundary here) Each group of hex codepoints is separated by break/no-break markers. Test approach: 1. Parse the test line to extract codepoints and expected break positions 2. Encode codepoints as UTF-8 3. Call Next_*_Break iteratively 4. Compare returned break positions against expected positions Normalization Test (NormalizationTest.txt): Format: five columns separated by semicolons: source; NFC; NFD; NFKC; NFKD # comment Each column is a space-separated list of hex codepoint values. Test approach: 1. Parse each test line into 5 codepoint sequences 2. Encode as UTF-8 3. For each normalization form, normalize the source 4. Compare the result against the expected column Bidi Test (BidiCharacterTest.txt): Format: input codepoints; paragraph direction; paragraph level; resolved levels; reordering Test approach: 1. Parse codepoints and expected results 2. Call Resolve_Levels 3. Compare resolved levels against expected 4. Call Reorder 5. Compare reordering against expected Collation Test (CollationTest/): Format: sequences of hex codepoints, one per line, in sorted order. Test approach: 1. For each consecutive pair of lines, compare using Collation.Compare 2. Verify the first line sorts Less than or Equal to the second Emoji Test (emoji-test.txt): Format: codepoint sequence; type; description Test approach: 1. Parse codepoint sequences 2. Encode as UTF-8 3. Call Classify_Sequence 4. Compare result against expected type IDNA Test (IdnaTestV2.txt): Format: source; toUnicode; toASCII Test approach: 1. Parse source and expected outputs 2. Call To_ASCII and To_Unicode 3. Compare results against expected outputs TEST DATA FILES --------------- All test data files are stored in ucd/ and downloaded from unicode.org: GraphemeBreakTest.txt 127 KB ~600 test cases WordBreakTest.txt 322 KB ~1,800 test cases SentenceBreakTest.txt 88 KB ~500 test cases LineBreakTest.txt 3.2 MB ~7,600 test cases NormalizationTest.txt 2.8 MB ~18,000 test cases BidiCharacterTest.txt 6.9 MB ~91,000 test cases emoji-test.txt 669 KB ~5,000 test cases IdnaTestV2.txt 776 KB ~6,000 test cases CollationTest/ Multiple files, thousands of test cases These are the official Unicode Consortium conformance test suites. They are the authoritative source for expected behavior. TEST STRUCTURE -------------- Each test program follows the same structure: 1. Initialize required modules (Properties, Normalization, Collation, etc.) 2. Open the test data file 3. For each test line: a. Parse the expected input and output b. Encode input as UTF-8 c. Call the library function d. Compare result against expected e. If mismatch: print diagnostic, increment fail count 4. Report total: N passed, M failed out of T total 5. Exit with 0 if all passed, nonzero if any failed Diagnostic output on failure includes: - Test line number - Input codepoints (hex) - Expected result - Actual result WHAT THE TESTS PROVE -------------------- The conformance tests prove that the ghost specification (the _Spec packages with their expression functions) correctly encodes the Unicode standard's rules. Given that GNATprove has already proved the implementation matches the ghost specification for all inputs, the conformance tests complete the chain: Implementation = Specification (by formal proof, all inputs) Specification = Standard (by conformance test, sampled inputs) --------------------------------------------------------- Implementation = Standard (by transitivity) The test suites are authored by the Unicode Consortium and represent the definitive expected behavior for each algorithm.