The connection between binary neutron star mergers and short gamma-ray bursts (GRBs) was solidified by the simultaneous detection of GW170817 and GRB 170817A. These events were followed by bright kilonova emission arising from the radioactive decay of freshly synthesized r-process ejecta, which were expelled during the neutron star merger. Kilonova emission is a fundamental signature of neutron star mergers. The ability to distinguish kilonova emission from the GRB afterglow requires a well characterized multi-wavelength afterglow, and sensitive near-infrared (nIR) observations. The majority of short GRBs lack these features, and, therefore, no meaningful limits on the kilonova ejecta mass can be determined in most cases. As such, evidence for kilonova emission has only been identified in a handful of short GRBs. In this talk I will present a multi-wavelength study of two cosmological short GRBs, GRB 160624A at z = 0.483 and GRB 200522A at z = 0.554, targeted at constraining kilonova emission from these events. Although associated to a similar distance, these events display extremely different emission properties. The optical/nIR limits for GRB 160624A are among the most stringent for short GRBs, and strongly disfavor kilonova ejecta masses larger than 0.1 solar mass. Whereas GRB 200522A displays a bright, nIR emission component that can be explained either by a radioactively powered kilonova with large ejecta mass, ~ 0.1 solar mass, or by intrinsic extinction from its host galaxy. These observations further extend the small sample of short GRBs with nIR observations, and pave the way for future results from the James Webb Space Telescope.