We report a new design of polymer-patched gold nanoparticles (AuNPs) with controllable interparticle interactions in terms of their direction and strength. Patchy AuNPs (pAuNPs) are prepared through hydrophobicity-driven surface dewetting under deficient ligand exchange conditions. Using the exposed surface on pAuNPs as seeds, a highly controllable growth of AuNPs is carried out via seed-mediated growth while retaining the size of polymer domains. As guided by ligands, these pAuNPs can self-assemble directionally in two ways along the exposed surface (head-to-head) or the polymer-patched surface of pAuNPs (tail-to-tail). Control of the surface asymmetry/coverage on pAuNPs provides an important tool in balancing interparticle interactions (attraction vs. repulsion) that further tunes assembled nanostructures as clusters and nanochains. The self-assembly pathway plays a key role in determining the interparticle distance and therefore plasmon coupling of pAuNPs. Our results demonstrate a new paradigm in the directional self-assembly of anisotropic building blocks for hierarchical nanomaterials with interesting optical properties.