Two-dimensional (2D) semiconductors are primed to realize a variety of photonic devices that rely on the transient properties of photogenerated charges, yet little is known on the change of the refractive index. The associated optical phase changes can be beneficial or undesired depending on the application, but require proper quantification. Measuring optical phase modulation of dilute 2D materials is, however, not trivial with common methods. Here, we demonstrate that 2D colloidal CdSe quantum wells, a useful model system, can modulate the phase of light across a broad spectrum using a femtosecond interferometry method. Next, we develop a toolbox to calculate the time-dependent refractive index of colloidal 2D materials from widely available transient absorption experiments using a modified effective medium algorithm. Our results show that the excitonic features of 2D materials result in broadband, ultrafast, and sizable phase modulation, even extending to the near infrared because of intraband transitions.