Stability issues in thermoelectric Na_xCoO₂ thin films have been solved by the addition of an in situ amorphous AlO_x capping layer, which prevents previously reported degradation when exposed to air. These chemically stable thin films enable detailed analysis of the intrinsic thermoelectric properties and form a significant progress towards applications. Single phase Na_xCoO₂ thin films with a low surface roughness are grown by pulsed laser deposition with either an epitaxial or textured crystal structure on Al₂O₃(001) or LaAlO₃(001) single crystal substrates, respectively. For textured thin films a resistivity and thermopower of 0.99 mΩ cm and 69 μV K⁻¹ are observed at room temperature, respectively. Based on an estimated thermal conductivity for thin films, the dimensionless figure of merit is very comparable to Na_xCoO₂ single crystals, demonstrating the effectiveness of the developed capping layer.