G17 A Mathematical Model of Language

This lecture rests on two works:

• for the general theory of Merge: M. Marcolli, N. Chomsky, R.C. Berwick, Mathematical Structure of Syntactic Merge. An Algebraic Model of Generative Linguistics, MIT Press, 2025 (cited below as [MCB]);
• for its dynamical-system structure: M. Marcolli, D. Skigin, Merge on workspaces as Hopf algebra Markov chain, arXiv:2512.18861.

Syntax as structure

Syntax is the focus here: the “large-scale structure” of language, where the main structure-building computational process resides. The input from linguistics consists of:

• what the main structure-building and transformation operations of syntax are;
• cross-linguistic variation, and the common universal computational core underlying it;
• the interface between syntax and semantics (structure formation and parsing);
• language acquisition, and the specialization of the core computational mechanism to one particular language.

Strings versus structures

An important development in the modeling of language is the passage from strings to structures. What language appears to look like is a string of symbols,

What language actually looks like is shown on the slide by a photograph of a hanging mobile sculpture: black shapes dangling from wires that branch and branch again from a single suspension point, free to turn in space.

How is language “a structure” beyond the string of words? Look at this sentence:

I shot an elephant in my pajamas – what it was doing in my pajamas, I’ll never know (Groucho Marx)

Why is it funny? Because it conflates two different structures.

A sentence is not just a string of words. We perceive structural relations in a sentence, more than proximity relations in the ordering of its words; the structure reflects the different possible ways in which sentences are generated.

Merge

Chomsky’s Minimalist Program (from the 1990s onward, reformulated after 2013) aims to identify the core computational process of structure formation in syntax. That core is a simple computational structure: Merge.

This process generates structures that are

• called hierarchical structures (or syntactic objects) in linguistics;
• called non-planar (or abstract) binary rooted trees by mathematicians.

The lexical items, at the leaves, are combined into a hierarchical structure. The tree dangles from its root and does not lie in the plane, so the lexical items at the leaves do not form an ordered string of symbols.

Merge is a computational process of structure formation: for instance, merging and and then merging the result with gives the tree at the top of the figure below. It is a generative process of non-planar full binary rooted trees with labeled leaves; below are some of the trees it generates.

What about the linear order of sentences? We can learn any language, and languages vary widely in their word-order structure. So the core computational mechanism, common to all languages, should not depend on one particular word order: word order is learned during language acquisition.

Externalization

• Externalization follows the generative process of structure formation, in which sentences are hierarchical non-planar tree structures.
• Language is externalized through the sensory-motor system (speech, sign) as a temporally ordered sequence of words.
• Equivalently, the trees acquire a planar structure.
• This uses language-dependent forms of word order, plus other rules.
• Example: English is head-initial, Japanese is head-final.

Here VP = verb phrase, TP = tense phrase, DP = determiner phrase. This leads to syntactic variability, syntactic parameters, and a geometry of syntax (which constraints?).

Summary of the picture so far.

• Language starts as hierarchical structures: free-floating (non-planar) binary rooted trees that encode only the structural relations between words.
• The physical constraints of externalization (sound, motion) force a reduction to an ordered sequence.
• During language acquisition our brain quickly learns, from a small collection of such examples, the (non-visible and complex) structure of syntax.

Structure formation: the Merge model

A closer look at structure formation (movement and transformation):

• We do more than form sentences: we modify them, transform them and manipulate them according to precise rules (forming questions, changing to the passive voice, and so on).
• The Merge operation of structure formation has two aspects: Internal Merge and External Merge.
• Internal Merge (responsible for “movement”) needs to reach inside already formed structures (syntactic objects) for constituent parts (accessible terms) that are available for further computation.
• Structure formation is also a “bottom-up” operation that proceeds in steps, using an intermediate scratchpad for its operations (workspaces).

The main aspects of the Merge model of syntax, in its most recent formulation (2013 onward), are:

• syntactic objects;
• workspaces;
• accessible terms;
• the Merge action on workspaces;
• externalization.

All of these notions have a precise mathematical formulation. It shows that many aspects of the linguistic model that have empirical grounds in fact follow from constraints of the algebraic structure.

Syntactic objects

So syntactic objects are abstract binary rooted trees with leaves decorated by lexical items. Abstract means that there is no assigned planar embedding (such trees are also called non-planar), and the leaves do not form an ordered string of elements of .

Workspaces

Merge operations map the set of workspaces to itself: they transform workspaces into new workspaces. This action should account for two types of operation, structure formation (External Merge) and movement or transformation (Internal Merge). The second requires access to substructures.

Accessible terms

In symbols: for a tree and a vertex , let be the subtree rooted at , and let be the set of non-root vertices of . The accessible terms of are For a workspace with ,

What mathematical structure governs workspaces and accessible terms? The answer: workspaces form a Hopf algebra.

Hopf algebras

A Hopf algebra is a mathematical way of describing composition–decomposition:

• a product, an “assemble” operation (two inputs, one output) that says how to put different objects together;
• a coproduct, a “decompose” operation (one input, two outputs) that lists all the possible ways of decomposing an object into two parts;
• compatibility between these two operations (a relation for interchanging the order of product and coproduct).

The slide illustrates this with jigsaw-puzzle pieces. At the top, two pieces are assembled in all possible ways: the product lists the five ways their tabs and slots can be joined. At the bottom, an assembly of three pieces is decomposed in all possible ways into a pair of parts: (the whole, ), (, the whole), and each split into a single piece and the remaining two.

These formal requirements are exactly what makes a good pair of composition and decomposition operations.

Multiplication: associativity and unit.

Comultiplication: coassociativity and counit.

The commutativity of the first of these diagrams is coassociativity:

Compatibility of product and coproduct.

Here is the flip , so permutes the two middle factors of . Writing also for this middle permutation, and for the product (as it will be for workspaces), the diagram says The behavior of the unit and counit with respect to the coproduct and product is given by the two diagrams below, using the identification .

Antipode. A further compatibility is the commutativity of the diagram that is, .

Workspaces form a Hopf algebra

Let be the vector space spanned by the forests.

• Product: workspaces (forests) are assembled by disjoint union of syntactic objects (trees),
• Coproduct: it disassembles workspaces into their constituent parts.

If one uses the coproduct in which the trees are primitive elements, , one gets , that is, This gives only partitions of the workspace, with no access to substructures (accessible terms). With it one would get External Merge, but no movement (Internal Merge).

The coproduct by admissible cuts

In a tree , consider forests of accessible terms,

The Merge action

Because of , only the binary case of applies. From now on the coproduct is a particular version of the admissible-cut coproduct, and is the Hopf algebra of workspaces.

Merge as a Hopf algebra Markov chain

Here the Merge operations are considered as forming a Hopf algebra Markov chain (HAMC), and its dynamical properties are studied. This treats syntax formation and transformation (movement) as a dynamical system.

The Merge graph

• Minimal SM is where both and consist of a single leaf (at least one of them in a tree with more than one leaf).
• There are two possible choices, which affect the properties of : allow cutting both edges below the same vertex, or not. This is only relevant in the SM case. The algebraic properties of are better if one does not; the dynamical-system properties are better if one does (so it is allowed).
• The condition is imposed because every Merge operation creates some edge. So any with is transient: it does not contribute to the long-term dynamical behavior.

Decomposing the Merge graph

Merge operations preserve the number of leaves of forests (using ). So the infinite graph breaks into a disjoint union of finite graphs, The case is trivial (no dynamics), so it is excluded. Merge operations also do not alter the labeling of the leaves, so there is a further decomposition where is the set of -element multisets of lexical items: is the set of labels of the leaves. The graph has the finite set of vertices

Counting vertices: counting forests

Counting the vertices means counting the forests (non-planar, with labeled leaves). A forest has set of leaves so the set of leaf labels is partitioned among the .

Partitions can be represented as Young diagrams: boxes arranged in rows of lengths , in non-increasing order.

There is a projection from workspaces to partitions, , recording the numbers of leaves of the components. For example, both forests below have .

Connectivity of the Merge graphs

For a forest write for its number of connected components, for the set of connected components of with a nontrivial set of edges, and . For let be the numbers of leaves of these components.

This seems complicated, but the key observation is: all these depend only on , not on .

Reducing the graph by projection to partitions

This fact is used to reduce the graph by projecting it to a graph whose vertices are the partitions in .

Sparsity

Unlike what the case suggests, the Merge graphs become rapidly more and more sparse as grows.

Strong connectedness and aperiodicity

Strong connectedness and aperiodicity have important dynamical consequences, discussed next.

From the graph to a dynamical system

There are two different approaches:

1. a random walk on the graph;
2. the maximal entropy random walk (here the same as the Hopf algebra Markov chain).

For the random walk, take the transition probabilities to be a normalization of the adjacency matrix, The drawback: what does the rescaling mean? What the linguistic theory predicts is that the Merge action is Markovian, so it should be itself that is the Markov chain, and not .

The maximal entropy random walk (and HAMCs in general) instead finds the basis in which itself is a stochastic matrix.

Stationary distribution

The first important piece of information about a Markov chain as a dynamical system is the form of its stationary distribution, which rules the long-term behavior of the dynamics.

Ergodicity

Another important property of dynamical systems is ergodicity: the long-term time average along a single trajectory equals the ensemble average across many trajectories.

Entropy

Let be a directed graph with adjacency matrix , and the Markov chain in the Perron–Frobenius rescaled basis. Consider all the Markov chains with the same underlying graph (for example the random walk, or any stochastic matrix with nonzero entries exactly where the adjacency matrix has ).

The maximum over all Markov chains with the same graph is where is the Perron–Frobenius eigenvalue of the adjacency matrix of .

Maximal entropy random walk

A Markov chain on induces a probability distribution on paths in , related to the entropy by

Weighted adjacency matrices: from entropy to free energy

• The different Merge operations (EM, IM, SM) have associated cost functions.
• These cost functions give a weighted adjacency matrix of the Merge graph .
• To incorporate cost functions, the maximal-entropy result has to be replaced by another optimization: Ruelle’s free energy optimization.

This Markov chain represents the core computational process of syntax formation and transformation, prior to its embodiment in the grammar of any one particular language (the core “universal grammar”).

Main question: what are its possible neurocomputational realizations? (For example, realizations of Markov chains in Hopfield networks, or realizations through cross-frequency phase synchronization on wavelets.)

Interfaces

In addition to the core computational mechanism there are the interfaces: externalization (the sensory-motor channel) and semantics (the conceptual channel). The model is motivated by the dual-stream picture of language in the brain (a “what” stream and a “how” stream).

For theoretical linguistics, the externalization interface also incorporates the specialization of the core computational mechanism to one particular language, through positive examples during language acquisition (conditioning of the Markov chain, plus planarization of the trees via word-order rules).

Externalization

The key idea, in a nutshell:

• are the syntactic objects produced by free symmetric Merge.
• are the planar binary rooted trees (with ordered leaves).
• is the canonical projection (a morphism of magmas).
• is a non-canonical (language-dependent) section of (it is not a morphism of magmas).
• The constraints (syntactic parameters, theta theory) give a projection to the language-dependent .

First step, :

• the planarization is a language-dependent, non-unique section of the projection;
• the only requirement on is compatibility with the word-order parameters of the given language ;
• in this way one obtains a planar tree for every syntactic object , with no further restriction.

Second step: other constraints.

• One needs to further eliminate those objects (the planar trees, which form the noncommutative version of ) that violate linguistic constraints of the particular language unrelated to word order (more syntactic parameters).
• There are other language-dependent conditions as well: theta theory, obligatory control, and so on; the trees that fail them are eliminated.

The quotient map (the of the diagram) is the projection that eliminates whatever does not satisfy these further constraints.

Associahedra

The syntax–semantics interface

The overall picture:

• In a similar way, formulating the generative process of syntax in terms of coproduct, grafting and product operations allows syntactic structures and substructures to be mapped consistently to semantic spaces.
• This is called the syntax–semantics interface.
• There are many different models of semantics; the simplest keeps track of proximity relations, agreement, similarity and co-occurrence (for example vector space models).

The slide shows a picture of such a semantic space: points scattered in the plane, grouped into four clusters, each point joined by a line to the center of its cluster.

The conceptual requirements on the syntax–semantics interface are:

1. autonomy of syntax;
2. syntax supports semantic interpretation;
3. semantic interpretation is, to a large extent, independent of externalization;
4. compositionality.

Here:

• autonomy of syntax: the Merge computational generative process of syntax is independent of semantics;
• syntax-first view: the syntax–semantics interface proceeds from syntax to semantics;
• there are two channels from the core Merge mechanism: to the conceptual-intentional system (the syntax–semantics interface) and to the sensory-motor system (externalization);
• compositionality: consistency across syntactic substructures.

Geometry of semantic spaces

• Language is a recent evolutionary step, so it requires a small modification: Merge as a single evolutionary step (Berwick–Chomsky).
• Everything else had already evolved: semantic and conceptual spaces are not specific to language (syntax is specific to language)… for example, conceptual manifolds in vision.
• The basic structures of a semantic space are
  1. proximity measurement (topology, metric);
  2. interpolation (convexity);
  3. agreement and disagreement measurements.

The slide’s picture is a curved surface with the tangent plane at a point , tangent vectors and , the geodesic from in the direction ending at , a nearby point with its own tangent plane and the transported vector , and below, two points and joined by a geodesic and by another curve .

Semantic parsing of syntactic structures

• Lexical items map to the semantic space, (words in context).
• Syntactic objects (trees) also map to , using the proximity and interpolation properties of the semantic space together with additional structure on the syntactic objects (syntactic head, phases, labeling).

Externalization and the syntax–semantics interface together

On the slide, the externalization arrow is decorated with a pentagon (the associahedron ) whose vertices are planar bracketings, one of them circled; the semantic arrow with a pentagon with a five-pointed star inscribed in it; and the combined process with a three-dimensional polyhedron.

The goal is to give a geometric model for the conceptual-intentional and sensory-motor channels and their interaction.

Consistency over substructures

An algebraic approach: a lesson from theoretical physics

The same algebraic structure appears with Feynman diagrams in place of syntactic objects.

• The coproduct decomposition allows the extraction of meaningful physical values of Feynman integrals, consistently over substructures. The slide’s example: for the two-loop photon self-energy graph of quantum electrodynamics (an electron loop crossed by an internal photon line), where is the one-loop vertex subgraph (it occurs twice in ) and , obtained by shrinking it to a point, is the one-loop photon self-energy graph.
• A similar coproduct/grafting/product operation describes the recursive solution of the quantum equations of motion (the Dyson–Schwinger equations). The slide’s example: the grafting operator , decorated by the one-loop vertex graph, applied to the one-loop vertex graph gives times the sum of the three two-loop vertex graphs obtained by inserting it at each of the three vertices.

This is the renormalization problem of quantum field theory.

The key idea: Birkhoff factorization

• Start with an assignment of semantic values to lexical items, .
• We want to extend it to an assignment for syntactic objects that is consistent over substructures.
• Key idea: incorporate consistency checking in an inductively defined function.
• A simple mapping (which will include inconsistent cases) is modified in a recursive way.
• This recursive construction of consistency checking is known in physics as “Bogolyubov preparation.”

The recursive construction needs two main ingredients:

1. a way of extracting the substructures of a given structure , so that and can be checked separately;
2. a way of separating out, in the target space , agreement and disagreement (or of filtering by levels of agreement and disagreement).

The first is on the side of syntax, the second on the side of semantics. The first is provided by the coproduct (the Hopf algebra structure); the second is some form of projection (known as a Rota–Baxter structure).

A basic model of semantic parsing

Its main constructions are:

1. a semantic space (topological or metric, with interpolation or convexity), for instance a geodesically convex Riemannian manifold, with a mapping of syntactic objects to it (using heads and phases);
2. evaluations (of distances, or of agreement and disagreement): an algebra or semiring with some “threshold operation” (a Rota–Baxter structure);
3. semantic probes, which extract (comparative) evaluations from the image of syntactic objects in the semantic space.

Evaluations and thresholds: Rota–Baxter structures

Its effect: and split into subalgebras .

What does this Rota–Baxter identity mean? It is the behavior of a “threshold operator.”

Semantic probes

The semantic space has probes: functions that check the degree of agreement or disagreement with a particular semantic hypothesis. For example, in vector space models one chooses a vector and sets The inner product with a fixed probe vector detects agreement or disagreement; it can be normalized to the cosine similarity.

Characters and Birkhoff factorization

Start with , where is a Hopf algebra and a Rota–Baxter algebra… but is only a morphism of commutative algebras (it is multiplicative: independent structures go to independent values). Such maps are called characters of the Hopf algebra.

The part detects where agreement is right or wrong: the substructures where consistent or inconsistent parsing is located (for a given probe and evaluation).

The Bogolyubov preparation is the expression which inductively incorporates the search for possible inconsistencies in substructures.

In the semiring case, where the runs over all the terms of the coproduct, and . These are multiplicative for the semiring product:

References

• M. Marcolli, N. Chomsky, R.C. Berwick, Mathematical Structure of Syntactic Merge. An Algebraic Model of Generative Linguistics, MIT Press, 2025. [MCB]
• M. Marcolli, D. Skigin, Merge on workspaces as Hopf algebra Markov chain, arXiv:2512.18861.
• R.C. Berwick, N. Chomsky, cited for Merge as a single evolutionary step (Why Only Us: Language and Evolution, MIT Press, 2016).
• Also named on the slides: J. Stasheff (associahedra) and D. Ruelle (free energy optimization).