To in situ remediate rivers polluted by organic matter and heavy metals, lab-scale sediment microbial fuel cells (SMFCs) were operated under different conditions. Results showed that the operation of SMFCs with anoxic cathodes contributed to the reductive reactions of Hg(II), Cu(II) and Ag(I) in overlying water and obtained the highest removal efficiencies (97.3 ± 2.6% of Hg(II), 87.7% ± 3.2% of Cu(II) and 98.5 ± 1.2% of Ag(I)) after 60 days' operation. Additionally, the SMFCs effectively enhanced the biodegradation of organic matter in the sediment and generated electricity. High-throughput sequencing of anodic microbial communities indicated that electrogenic bacteria and fermentative bacteria coexisted in the chamber and the operation of SMFC increased the content of electrogenic bacteria in the anodic chamber.