Neugebauer was born in Göttingen, Germany and is the son of Otto Neugebauer, an Austrian-American mathematician and historian of science, and Grete Bruck. After moving to the United States at age seven, he received his A.B. in physics from Cornell University in 1954 and his Ph.D. in physics from Caltech in 1960, with a thesis on the photoproduction of negative and positive pions from deuterium.
During his military service with the U.S. Army he was stationed at the Jet Propulsion Laboratory and worked for the Ordnance Corps until 1962. He joined the Caltech faculty in 1962 as an assistant professor, becoming a full professor of physics in 1970. He was named Howard Hughes Professor in 1985 and Chairman of the Division of Physics, Math and Astronomy in 1988. At the time of his death, he was the Robert Andrews Millikan Professor of Physics, Emeritus and an adjunct faculty member of the University of Arizona's Steward Observatory.
Neugebauer served as the director of the Palomar Observatory from 1980 to 1994.
Neugebauer was active in infrared astronomy, and played a leading role in infrared studies of the planets. In addition—and largely through his activities with the Infrared Astronomical Satellite (IRAS) and the Infrared Processing and Analysis Center (IPAC)—he led both ground- and space-based infrared studies of the stars, the Milky Way and other galaxies. Observations by him and his colleagues at Mount Wilson and Palomar observatories revealed thousands of infrared sources in the sky, and afforded the first infrared view of the galactic center. Together with Robert B. Leighton, he completed the Two-Micron Sky Survey, the first infrared survey of the sky, which cataloged more than 5,000 infrared sources. Together with Eric Becklin, he discovered the Becklin–Neugebauer Object, an intense source of infrared radiation in the Orion Nebula that is one of the brightest objects in the sky at wavelengths less than 10 micrometres.
Neugebauer played a role in the design and construction of the W. M. Keck Observatory in Hawaii. Among Neugebauer's numerous awards are two NASA Exceptional Scientific Achievement Medals (1972, 1984), the 1985 Space Science Award of the American Institute of Aeronautics and Astronautics, the 1985 Richmyer Lecture Award, the 1986 Rumford Prize, the 1996 Henry Norris Russell Lectureship, the 1998 Herschel Medal, and the 2010 Bruce Medal. He was named California Scientist of the Year for 1986 by the California Museum of Science and Industry, and he was elected to the National Academy of the Sciences, the American Philosophical Society, the American Academy of Arts and Sciences and the Royal Astronomical Society.
Neugebauer was married to the geophysicist Marcia Neugebauer, a pioneer in solar wind research at the Jet Propulsion Laboratory, and the two resided in Tucson, Arizona. He died on 26 September 2014 in Tucson, due to spinocerebellar ataxia.