In this paper, a facile route was designed to prepare four new second-order nonlinear optical (NLO) polyaryleneethynylenes containing normal aromatic or perfluoroaromatic rings as isolation groups via a simple Sonogashira coupling reaction. For comparison, their small molecule models were also synthesized to investigate the aromatic/perfluoroaromatic (Ar–Ar^F) self-assembly effect existing between the isolation groups and polymer main chain. Thanks to the presence of the self-assembly effect, the NLO effect and stability of these polymers were enhanced in a large degree. The NLO coefficient of P4 reached up to 166.7 pm V⁻¹, which is, to the best of our knowledge, the new record reported so far for linear polymers using simple azobenzenes as the chromophore moieties.