Non-fullerene acceptor-based polymer solar cells (NFA-PSCs) can exhibit high morphological stability under thermal stress, often resulting in the fabrication of thermally stable NFA-PSCs. Here, our stability study with systematic steps shows that high morphological stability of active layers does not necessarily guarantee high thermal stability for NFA-PSCs. Furthermore, a degradation pathway is elucidated by using two NFA-PSCs comprising well-known polymer donors and non-fullerene acceptors such as PTB7-Th, PBDB-T-2Cl, and ITIC-4F. Morphologies of the PTB7-Th:ITIC-4F and PBDB-T-2Cl:ITIC-4F blends are highly stable at a high temperature of 110 °C. However, the corresponding NFA-PSCs degrade severely at the same temperature. For example, all the NFA-PSCs show an efficiency reduction of greater than 30% within an hour, particularly owing to the significant decrease in the short-circuit current density (J_SC) and fill factor (FF). A series of experiments, including time-of-flight secondary ion mass spectrometry measurements, demonstrate that the penetration of metal oxide molecules into the photoactive layer occurs upon heating, resulting in degradation. The diffused molecules can act as charge recombination centers, leading to decreases in J_SC and FF. Finally, we provide insights, from a material design perspective, into the prevention of undesirable penetration of metal oxide molecules.