The electric fields and potentials inside and at the interface of matter are relevant to many branches of physics, chemistry, and biology. Accurate quantification of these fields and/or potentials is essential to control and exploit chemical and physical transformations. Before we understand the response of matter to external fields, it is first important to understand the intrinsic interior and interfacial fields and potentials, both classically and quantum mechanically, as well as how they are probed experimentally. Here we compare and contrast, beginning with the hydrogen atom in vacuum and ending with concentrated aqueous NaCl electrolyte, both classical and quantum mechanical electric potentials and fields. We make contact with experimental vibrational Stark, electrochemical, X-ray, and electron spectroscopic probes of these potentials and fields, outline relevant conceptual difficulties, and underscore the advantage of electron holography as a basis to better understand electrostatics in matter.