The development of new antibiotics is of increasing importance due to the growing resistance power of microbes against conventional drugs. To this end, cationic peptides are emerging as clinically potent antimicrobial agents. In the present study, we have synthesized six dipeptide-based cationic amphiphiles with different head group structures by varying combinations of L-amino acid residues. These amphiphiles showed remarkable growth inhibiting activity on several Gram-positive (minimum inhibitory concentration (MIC) = 0.1–10 µg/mL) and Gram-negative (MIC = 5–150 µg/mL) bacteria as well as on fungus (MIC = 1–50 µg/mL). The inherent antimicrobial efficacies of these cationic dipeptides were influenced by the head group architecture of the amphiphiles. Encouragingly, these amphiphiles selectively attacked microbial cells, while showing biocompatibility toward mammalian cells. The results show that the rational designing of short peptide-based cationic amphiphiles might serve as a promising strategy in the development of antimicrobial agents with greater cell specificities. In addition, the amphiphiles showed water gelation ability at room temperature. The formation of non-covalent supramolecular networks in gelation was established by microscopic and spectroscopic studies.