Novel all-solid-state ion conductive polymer membranes were synthesized for anhydrous high temperature fuel cells. Ion-conductive monomers (ethylene glycol methacrylate phosphates, phosmers) were employed for fixing ion-conductive moieties in the matrix polymer (sulfonated poly(arylene sulfone), sPAS). Inorganic materials, acid-doped titanium oxide and tin indium phosphate were introduced to form anhydrous polymer composite membranes. As a result, poly(phosmer)s significantly improved both the proton conductivity and flexibility of the composite membranes. Sulfonated poly(arylene sulfone)–poly(phosmer)–tin indium phosphate composite membranes showed the highest ion-conductivity value under anhydrous conditions due to the dipolar interaction of poly(phosmer) with the matrix polymer and the inorganic material. The composite membranes exhibited a drastic enhancement in performances of anhydrous high temperature fuel cells compared to those of Nafion-based composite membranes. The fuel cell performances indicate that the poly(arylene sulfone)–poly(phosmer)–inorganic particle composites could be applicable for anhydrous high temperature fuel cells as promising polymer electrolyte membranes.