Supernova remnants (SNRs) can provide a unique insight into the formation and evolution of the compact objects within them. The prevailing magnetar formation scenario suggests that magnetars must be born fastly rotating, which in turn implies that they should be accompanied by highly energetic supernova explosions. We construct a comprehensive energy inventory of supernova remnants. To this end, we have fit the spectra of over 20 SNRs in the Milky Way and Large Magellanic Cloud (LMC): 6 with magnetars, 3 with low-magnetic field “central compact objects” (CCOs), 4 with ordinary pulsars, and 9 without any detected compact object. The results of spectral fitting are combined with a Sedov kinematic model in order to derive the energy of the supernova explosion, which are found to range between ESN ≈ 4×1049-4×1051 erg with no obvious correlation with the nature of the central compact source. We use these supernova energies to constrain the initial spin periods of newly born neutron stars and conclude that magnetars as well as CCOs are born in ordinary core collapse explosions. Moreover, our analysis of supernova remnant asymmetries shows that sources hosting magnetars or CCOs are indistinguishable from the core-collapse SNR population in the Milky Way and LMC. All these results cast doubt on the paradigm of magnetar formation via magnetic dynamo amplification.