The H I Lyman alpha (LyA; 1216 Å) and O I (1302, 1305, 1306 Å) transitions have great utility for stellar and exoplanet observations including chromosphere diagnosis and probing exoplanetary mass loss, among many other applications. However, airglow emission from Earth’s exosphere entering through COS’s large aperture typically swamps the stellar LyA and O I signal in COS observations. Airglow subtraction is easily performed for STIS observations of these lines, because STIS employs a narrow slit and has spatial resolution in the cross-dispersion direction that simultaneously captures a spectrum of the airglow. We have applied a proven airglow subtraction technique homogeneously to ~100 archival COS spectra of low-mass stars (F, G, K, and M dwarfs) to extract true stellar LyA and O I profiles, increasing the number of stars with such spectra by a factor of five. A subset of the COS sample has complementary STIS LyA spectra, which we use to gauge the reliability of our subtraction technique. We have measured airglow-free LyA and O I intrinsic line fluxes and statistically compare the fluxes to various stellar properties to uncover variations with mass and age. Finally, we describe the possibility of airglow subtraction with COS for other objects and science goals.