In recent years, α-glucosidase inhibitors (AGIs) have played a significant role in the treatment of type II diabetes (T2D), so it is necessary to develop a reliable and sensitive method to find new AGIs. Herein, we establish a novel method based on fluorescent carbon nitride nanoparticles (CNNPs) for the sensitive detection of the activity of α-glucosidase (α-glu) and the screening of its inhibitors. A CNNP-based fluorescent probe is synthesized from green raw materials, urea and lysine, by a one-pot method. In the presence of α-glu, the substrate 4-nitrophenyl-α-D-glucopyranoside (pNPG) is hydrolyzed to generate 4-nitrophenol (pNP), leading to the fluorescence (FL) quenching of CNNPs due to the inner filter effect (IFE). On the other hand, the activity of α-glu is inhibited after the addition of AGIs, which turns on the FL of CNNPs. In this way, the detection of α-glu activity and the screening of AGIs are achieved. The linear range is 1.25–10.00 U L⁻¹ with a limit of detection as low as 0.17 U L⁻¹ and the IC₅₀ values of two typical inhibitors (gallic acid and acarbose) are 813 μM and 465 μM, respectively. The CNNP probe is further applied for the determination of α-glu activity in human serum samples with satisfactory results.