Astronomy is on the cusp of a data revolution, with facilities like the Vera Rubin Observatory (formerly LSST) and the Nancy Grace Roman Space Telescope (formerly WFIRST) posed to provide high-quality, multi band images over thousands of square degrees on the sky. With tens of billions of astronomical objects in these images, astronomers need to devise new methods to identify and classify stars and galaxies at scale. To address these challenges in data analysis, we have developed Morpheus, a new deep learning framework for pixel-level analysis of astronomical images. Morpheus leverages advances in deep learning to perform source detection, source segmentation, and morphological classification pixel-by-pixel via a semantic segmentation algorithm adopted from the field of computer vision. We present the model, demonstrate its performance, and show applications to real astronomical data including the largest Hubble Space Telescope surveys taken to date.