The development of small molecule affinity-based chemical probes as research tools for studying the role of carbonic anhydrases (CAs) in their wider biological context has become an active field of research owing to an increasing awareness of the therapeutic relevance of this enzyme family, particularly in cancer. High CA isozyme selectivity, low nonspecific labeling, and efficient labeling yield are the characteristics of an ideal chemical probe, however achieving an effective balance of all three properties is challenging. The progress of this field, with a focus on the diversity of chemical probe designs and the methodologies employed to accomplish target-selective labeling, is reviewed. The lessons learned from this continuous development have the potential to be more broadly applied to other targets where a delicate balance of the properties impacting the target specificity of a small molecule chemical probe must be managed.