The harmonica first appeared in Vienna, where harmonicas with chambers were sold before 1824 (see also Anton Reinlein and Anton Haeckl). These early instruments were refferred to as mundharmonika (lit. "mouth harmonica") relating to the very popular instrument at the time, the glass harmonika. This harmonica had a series of notes lined up, assigned to holes, which could be trigged by blowing these holes. The first major advancement to the harmonica Richter tuning, invented by Joseph Richter (who also is credited with inventing the blow and draw mechanism), was created in 1826 and was adopted nearly universally in the subsequent years. In Germany, Mr. Meisel of Geschichte des Akkordeonbaus in Klingenthal, Schwarzmeisel and Langhammer, bought a harmonica with chambers (Kanzellen) at the Exhibition in Braunschweig in 1824. He and Langhammer in Graslitz copied the instruments; by 1827 they had produced hundreds of harmonicas. Many others followed in Germany and also nearby in what would later become Czechoslovakia. In 1829, Johann Wilhelm Rudolph Glier also began making harmonicas.

In 1830, Christan Messner, a cloth maker and weaver from Trossingen, copied a harmonica his neighbour had brought from Vienna. He had such success that eventually his brother and some relatives also started to make harmonicas. From 1840 onwards, his nephew Christian Weiss was also involved in the business. By 1855, there were at least three harmonica-making businesses: C. A. Seydel Söhne, Christian Messner & Co., and Württ. Harmonikafabrik Ch. WEISS. Currently, only C.A. Seydel is still in business.

Owing to competition between the harmonica factories in Trossingen and Klingenthal, machines were invented to punch the covers for the reeds. In 1857, Matthias Hohner, a clockmaker from Trossingen, started producing harmonicas, eventually to become the first person to mass-produce them. He used a mass-produced wooden comb that he had made by machine-cutting firms. By 1868, he began supplying the United States. By the 1920s, the diatonic harmonica had largely reached its modern form. Other types followed soon thereafter, including the various tremolo and octave harmonicas.

By the late 19th century, harmonica production was a big business, having evolved into mass-production. New designs were still developed in the 20th century, including the chromatic harmonica, first made by Hohner in 1924, the bass harmonica, and the chord harmonica. In the 21st century, radical new designs are still being introduced into the market, such as the Suzuki Overdrive, Hohner XB-40, and Harrison B-Radical.

Diatonic harmonicas were designed primarily for the playing of German and other European folk music and have succeeded well in those styles. Over time the basic design and tuning proved adaptable to other types of music such as the blues, country, old-time and more. The harmonica was a success almost from the very start of production, and while the centre of the harmonica business has shifted from Germany, the output of the various harmonica manufacturers is still very high. Major companies are now found in Germany (Seydel, Hohner - the dominant manufacturer in the world), Japan (Suzuki, Tombo, Yamaha), China (Huang, Leo Shi, Suzuki, Hohner), Brasil (Hering, Bends) and the United States Harrison Harmonicas. Recently, responding to increasingly demanding performance techniques, the market for high quality instruments has grown.