The invention of the spectroscope by Kirchhoff and Bunsen in 1859 and the development of spectral analysis, along with the development of the periodic table by Mendeleev and Meyer in 1869, provided valuable tools for the study of rare earths. The impact of these advances is clear. In the 60 or so years before their introduction, only six rare earths had been identified, one of them erroneously. In the following 50 years, the number jumped to 15. However, problems remained. The rare earths severely challenged Mendeleev's periodic system, because he did not know where to place them in the table. He made many modifications simply to accommodate them. Meanwhile, spectroscopic analysis was causing difficulties.

Because of the complexity of the spectra of the rare earths and the questionable purity of the samples, many claims were being made for new elements that proved false. Despite these difficulties, by 1901 only two rare earths were still undiscovered. In 1907, almost simultaneously, Georges Urbain in France, Carl Auer von Welsbach in Austria, and Charles James in the United States separated ytterbium into two fractions. Urbain's designation of the new element as lutetium, after the Roman name for his native city, Paris, was accepted by most, although von Welsbach's name cassiopium was used in German-speaking countries for many years. Von Welsbach contested Urbain's priority for the discovery of lutetium. James, although he had accumulated a substantial amount of highly purified lutetium oxide, withdrew a paper he had submitted for publication and made no public claim for his own work.

Lutetium was the 16th of the rare earths to be identified. Using X-ray spectroscopy, Moseley demonstrated that only one rare earth element remained to be discovered: element 61. He determined the atomic numbers of the elements and showed that they were arranged in the periodic table in order of atomic number, not atomic weight, as Mendeleev had thought. In the 1920s, James, B. Smith Hopkins, and Luigi Rolla each thought he had isolated element 61. Hopkins's publication preceded that of James, and so Hopkins's designation of the element as illinium was accepted temporarily. The last of the priority disputes over discovery of a rare earth element ended inconclusively, however, when none of the claims could be substantiated. The authentic discovery of element 61 had to await the development of a new separation technique, ion-exchange chromatography, and work on the atomic bomb during World War II. Finally, in 1947, Marinsky, Glendenin, and Coryell announced the discovery of the unstable element 61, which they named promethium after the Titan who stole fire from the gods.