Hydrogels are commonly used as artificial extracellular matrices for 3D cell culture and for tissue engineering. Viscoelastic hydrogels with tunable stress relaxation have recently been developed, and stress relaxation in the hydrogels has been found to play a key role in regulating cell behaviors such as differentiation, spreading, and proliferation. Here we report a simple but precise materials approach to tuning stress relaxation of alginate hydrogels with polyethylene glycol (PEG) covalently grafted onto the alginate. Hydrogel relaxation was modulated independent of the initial elastic modulus by varying molecular weight and concentration of PEG along with calcium crosslinking of the alginate. Increased concentration and molecular weight of the PEG resulted in faster stress relaxation, a higher loss modulus, and increased creep. Interestingly, we found that stress relaxation of the hydrogels is determined by the total mass amount of PEG in the hydrogel, and not the molecular weight or concentration of PEG chains alone. We then evaluated the utility of these hydrogels for 3D cell culture. Faster relaxation in RGD-coupled alginate-PEG hydrogels led to increased spreading and proliferation of fibroblasts, and enhanced osteogenic differentiation of mesenchymal stem cells (MSCs). Thus, this work establishes a new materials approach to tuning stress relaxation in alginate hydrogels for 3D cell culture.