Vector Space Semantics

Origin. Harris (1954), Firth, Turney & Pantel, Mikolov. Distributional hypothesis. Word vectors. Compositional distributional. Foundation of computational semantics.

Models. Meaning from distribution. Words as vectors. Similarity as distance. Compositional combination. Learned representations.

Formalism.

Distributional hypothesis: "You shall know a word by the company it keeps" (Firth). Similar contexts → similar meaning. Co-occurrence statistics. Empirical semantics.

Word vector: v_w ∈ ℝⁿ. Vector representing word w. Dimensions: contexts or latent features. Geometric meaning.

Similarity: sim(w₁, w₂) = cos(v_{w₁}, v_{w₂}). Cosine similarity. Angle between vectors. Semantic relatedness.

Word2Vec: Skip-gram: predict context from word. CBOW: predict word from context. Neural training. Dense vectors.

Analogies: v_king - v_man + v_woman ≈ v_queen. Vector arithmetic. Relational similarity. Geometric structure.

Compositional distributional: How to combine word vectors? Matrix-vector multiplication. Tensor composition. Syntax-guided combination.

Type-driven composition: Noun: vector. Adjective: matrix. A N = M_A · v_N. Compositional types.

Symbols.

Symbol	Unicode	Meaning
v_w	—	word vector
cos	—	cosine similarity
⊗	U+2297	tensor product
ℝⁿ	—	vector space

Metatheory. Distribution as meaning. Vector space. Composition. Learned representations.

Applies to. Computational semantics. NLP. Word similarity. Analogy.

Limitations. Compositionality challenges. Discrete meaning. Interpretability. Polysemy.