In order to shed light on the effects of different numbers of fluorine atoms in donor–acceptor (D–A) conjugated polymers on the photophysics and photovoltaic properties, three polymers named PTBFBT-0F, PTBFBT-1F, PTBFBT-2F were synthesized and thoroughly investigated. The nonfluorinated benzothiadiazole (BT) polymer (PTBFBT-0F) has a highest occupied molecular orbital (HOMO) energy level of −4.98 eV and a low bandgap of 1.64 eV. When one of the hydrogen atoms of the BT unit was substituted by a fluorine atom (PTBFBT-1F), a small blue-shift in UV-Vis absorption and a lower HOMO energy level of −5.11 eV were observed, thus a similar V_oc and J_sc were obtained. Nevertheless, the FF of PTBFBT-2F was further improved from 46% to 53% due to the relatively higher and balanced electron/hole charge transport mobility of 1.83 × 10⁻⁵ cm² V⁻¹ s⁻¹ and 1.52 × 10⁻⁵ cm² V⁻¹ s⁻¹. Using a Ca/Al top electrode, devices based on PTBFBT-0F, PTBFBT-1F, PTBFBT-2F as electron donor showed increasing power conversion efficiencies (PCE) of 3.0%, 3.6% and 4.2%, respectively. Furthermore, replacing Ca with a zirconium acetylacetonate film (ZrAcac) as the cathode buffer layer (CBL), a PCE of 5% with PTBFBT-2F as the donor was obtained.