There is a great fundamental interest in charge dynamics of PbS quantum dots, as they are promising for application in photovoltaics and other optoelectronic devices. The ultrafast charge transport is intriguing, offering insight into the mechanism of electron tunneling processes within the material. In this study, we investigated the charge transfer times of PbS quantum dots of different sizes and non-quantized PbS reference materials by comparing the propensity of localized or delocalized decays of sulfur 1s core hole states excited by X-rays. We show that charge transfer times in PbS quantum dots decrease with excitation energy and are similar at high excitation energy for quantum dots and non-quantized PbS. However, at low excitation energies a distinct difference in charge transfer time is observed with the fastest charge transfer in non-quantized PbS and the slowest in the smallest quantum dots. Our observations can be explained by iodide ligands on the quantum dots creating a barrier for charge transfer, which reduces the probability of interparticle transfer at low excitation energies. The probability of intraparticle charge transfer is limited by the density of available states which we describe according to a wave function in a quantum well model. The stronger quantum confinement effect in smaller PbS quantum dots is manifested as longer charge transfer times relative to the larger quantum dots at low excitation energies.