Transition metal phosphides exhibit promising catalytic performance for the hydrogen evolution reaction (HER); however their surface structure evolution during electrochemical operation has rarely been studied. In this work, we investigate the surface reconstruction of CoP nanosheets by an in situ electrochemical activation method. After remodeling, CoP nanosheets experience an irreversible and significant evolution of the morphology and composition, and low-valence Co complexes consisting of Co(OH)_x species are formed on the surface of CoP nanosheets, and they largely accelerate the dissociation of water. Benefiting from the synergistic effect of CoP and Co(OH)_x, the working electrode shows a remarkably enhanced HER activity of 100 mV at 10 mA cm⁻² with a Tafel slope of 76 mV dec⁻¹, which is better than that of most transition metal phosphide catalysts. This work would provide a deep understanding of surface reconstruction and a novel perspective for rational design of high performance transition metal phosphide electrocatalysts for water related electrolysis.