Studies of the cosmic gamma-ray bursts (GRBs) and their host galaxies are starting to provide interesting or even unique new insights in observational cosmology. GRBs represent a new way of identifying a population of star-forming galaxies at cosmological redshifts. GRB hosts are broadly similar to the normal field galaxy populations at comparable redshifts and magnitudes, and indicate at most a mild luminosity evolution out to z ~ 1.5 - 2. GRB optical afterglows seen in absorption provide a powerful new probe of the ISM in dense, central regions of their host galaxies, complementary to the traditional studies using QSO absorbers. Some GRB hosts are heavily obscured, and provide a new way to select a population of cosmological sub-mm sources, and a novel constraint on the total obscured fraction of star formation over the history of the universe. Finally, detection of GRB afterglows at z > 6 may provide a unique way to probe the primordial star formation, massive IMF, early IGM, and chemical enrichment at the end of the cosmic reionization era.