The development of highly active and selective enzyme inhibitors is one of the priorities of medicinal chemistry. Typically, various high-throughput screening methods are used to find lead compounds from a large pool of synthetic compounds, and these are further elaborated and structurally refined to achieve the desired properties. In an effort to streamline this complex and laborious process, new selection strategies based on different principles have recently emerged as an alternative. Herein, we compare three such selection strategies with the aim of identifying potent and selective inhibitors of human carbonic anhydrase II. All three approaches, in situ click chemistry, phage-display libraries and synthetic peptide libraries, led to the identification of more potent inhibitors when compared to the parent compounds. In addition, one of the inhibitor–peptide conjugates identified from the phage libraries showed greater than 100-fold selectivity for the enzyme isoform used for the compound selection. In an effort to rationalize the binding properties of the conjugates, we performed detailed crystallographic and NMR structural analysis, which revealed the structural basis of the compound affinity towards the enzyme and led to the identification of a novel exosite that could be utilized in the development of isoform specific inhibitors.