Because of their immense power, Gamma-ray bursts (GRBs) have the potential to be detected, and hence, probe stellar properties and star formation out to high redshifts. To be used as probes, we must understand the evolution of GRBs with metallicity and redshift. We present theoretical results based on the collapsar GRB engine for long-duration GRBs studying a broad range of properties of these GRBs as a function of redshift. Under this engine paradigm and the trends in stellar evolution with metallicity (e.g. metallicity dependence of mass loss from winds), we are able to predict the evolution of GRB properties including distributions of peak luminosity, duration, beaming and the emission from their associated supernovae. Extremely low metallicities, such as those found at high redshift, can also alter the initial mass function of stars formed. We also simulate in our study the effects of an evolving initial mass function on GRB properties, nucleosynthetic yields and the total ionizing radiation. Although the uncertainties in such a study are large, we are able to calibrate our models based on the current set of GRB observations. Finally, we make predictions for the rate of GRBs for high-redshift GRB missions such as the proposed Gamow Explorer.