The binding of ligands to nanometer-sized surfaces is a central aspect of colloidal nanocrystal (NC) research, for which CdSe NCs were mostly used as the model system to evaluate different surface chemistries. Here, we take the opposite approach and analyze the binding of a single ligand to two different materials. Using CdSe and CdS NCs of similar size and shape and purified with the same protocol, we show that both NCs are capped with tightly bound cadmium oleate (CdOA2). We systematically find that CdS NCs bind more CdOA2 per surface area and that a larger fraction of these ligands withstand displacement by butylamine (BuNH2) as compared to CdSe NCs. These findings concur with density functional theory simulations, which predict for CdS on average cadmium oleate displacement energy larger by 32 kJ/mol than for CdSe. Even so, the displacement isotherm indicates that for both NCs, the initially displaced ligands have the same displacement equilibrium constant. This result suggests that NC work-up codetermines the actual surface chemistry of NCs by effectively setting a displacement (free) energy threshold for ligands to remain bound throughout the purification process. As such, this work highlights that the actual surface chemistry of NCs post purification is the mixed result of intrinsic ligand-NC binding characteristics and concrete processing and purification methods used.