Actuating plasmonic nanoparticles with responsive hydrogels has led to many useful applications but an understanding of how exactly the assembly and disassembly happen remains ambiguous. Conventional views about this system mainly recognize this reversible process as a bi-switch between dispersed and aggregated states. However, in this paper, a hidden vesicle intermediate is revealed by probing changes in plasmon resonances with temperature. The critical factor that influences such a vesicle state originates from the different aggregation modes between polymers and Au NPs. Our theoretical model, along with experimental evidence, further supports this mechanism. This new insight not only provides a kinetic means to modify self-assemblies by engineering the polymer aggregation rate but it also has great implications for the dynamics of colloid/polymer interactions in general.