In mathematics, Vinberg's algorithm is an algorithm, introduced by Ernest Borisovich Vinberg, for finding a fundamental domain of a hyperbolic reflection group.

Conway (1983) used Vinberg's algorithm to describe the automorphism group of the 26-dimensional even unimodular Lorentzian lattice II_25,1 in terms of the Leech lattice.

Let ${\displaystyle \Gamma <\mathrm {Isom} (\mathbb {H} ^{n})}$ be a hyperbolic reflection group. Choose any point ${\displaystyle v_{0}\in \mathbb {H} ^{n}}$; we shall call it the basic (or initial) point. The fundamental domain ${\displaystyle P_{0}}$ of its stabilizer ${\displaystyle \Gamma _{v_{0}}}$ is a polyhedral cone in ${\displaystyle \mathbb {H} ^{n}}$. Let ${\displaystyle H_{1},...,H_{m}}$ be the faces of this cone, and let ${\displaystyle a_{1},...,a_{m}}$ be outer normal vectors to it. Consider the half-spaces ${\displaystyle H_{k}^{-}=\{x\in \mathbb {R} ^{n,1}|(x,a_{k})\leq 0\}.}$

There exists a unique fundamental polyhedron ${\displaystyle P}$ of ${\displaystyle \Gamma }$ contained in ${\displaystyle P_{0}}$ and containing the point ${\displaystyle v_{0}}$. Its faces containing ${\displaystyle v_{0}}$ are formed by faces ${\displaystyle H_{1},...,H_{m}}$ of the cone ${\displaystyle P_{0}}$. The other faces ${\displaystyle H_{m+1},...}$ and the corresponding outward normals ${\displaystyle a_{m+1},...}$ are constructed by induction. Namely, for ${\displaystyle H_{j}}$ we take a mirror such that the root ${\displaystyle a_{j}}$ orthogonal to it satisfies the conditions

(1) ${\displaystyle (v_{0},a_{j})<0}$;

(2) ${\displaystyle (a_{i},a_{j})\leq 0}$ for all ${\displaystyle i<j}$;

(3) the distance ${\displaystyle (v_{0},H_{j})}$ is minimum subject to constraints (1) and (2).

• Conway, John Horton (1983), "The automorphism group of the 26-dimensional even unimodular Lorentzian lattice", Journal of Algebra, 80 (1): 159–163, doi:10.1016/0021-8693(83)90025-X, ISSN 0021-8693, MR 0690711
• Vinberg, È. B. (1975), "Some arithmetical discrete groups in Lobačevskiĭ spaces", in Baily, Walter L. (ed.), Discrete subgroups of Lie groups and applications to moduli (Internat. Colloq., Bombay, 1973), Oxford University Press, pp. 323–348, ISBN 978-0-19-560525-9, MR 0422505