This tutorial review summarizes the recent advances on the self assembly of magnetic nanoparticles into one-, two-, and three-dimensional architectures using synthetic polymers and biopolymers. The materials have unique stimuli-responsive behavior that emerge in response to internal and external magnetic fields. For example, magnetic fields can be used to elicit a magnetic response from the materials, whether the material rearranges in response to the external field, or provides information regarding the local environment of the nanoparticles. These materials hold great promise for applications ranging from data storage to nanomedicine.