Gas-phase metallicity, or nebular oxygen abundance, is a fundamental property of galaxies crucial to constraining processes responsible for galaxy growth and evolution. At z ~ 2, the peak of cosmic star formation, several statistical surveys have studied gas-phase metallicity and its relation to global galaxy properties for galaxies with log(Mstar / Msolar) >= 9.5. Our work extends these studies at this epoch down into the mass regime of dwarf galaxies, with log(Mstar / Msolar) < 9. We present rest-optical Keck/MOSFIRE spectroscopy of typical, gravitationally-lensed, star-forming dwarf galaxies from 1.7 < z < 2.6. We highlight our study of the z = 2.59 dwarf galaxy A1689-217 (Gburek et al. 2019), for which we detected the faint, electron-temperature-sensitive [OIII]4363 emission line at 4.2-sigma significance, one of only a handful of such detections at z > 1. This detection allowed us to directly calculate a metallicity of 12+log(O/H) = 8.06 ± 0.12 (~ ¼ Zsolar). Using this estimate along with measured strong-line ratios, we used A1689-217 and other galaxies from the literature with [OIII]4363 detections at z ~ 0 – 3.1 to address the applicability of locally-calibrated, strong-line ratio – direct metallicity relations at high-z, providing evidence that such oxygen-based relations are consistent over this redshift interval. We also present a stack of 16 galaxies within our sample (with median values: z ~ 2.40, log(Mstar / Msolar) ~ 8.81, SFR ~ 2.32 M⊙/yr) that have spectroscopic coverage of all strong, rest-optical, nebular emission lines. This stack has a 2.4-sigma detection of [OIII]4363 and yields a direct metallicity of 12+log(O/H) = 7.80 +0.28 -0.19 (~ 0.13 Zsolar). With this stack, we further explore the applicability of locally-calibrated metallicity diagnostics at high-z. We also add a constraint to the low-mass slope of the z~2 mass-metallicity relation and compare these results to predictions by simulations in an effort to provide constraints on galactic outflow and feedback models. Finally, we use the stack to explore the existence of a redshift evolution of the stellar mass – SFR – metallicity relation.