Microporous molecular sieves (MMS), due to their potential for advanced applications in separation technologies, catalysis, and nanoscience, have attracted much attention in the past decades. In this work, a novel PEO-based all solid-state composite polymer electrolyte by using MMS as filler is developed. Thermal analysis and polarized optical microscopy results show that MMS can increase the amount of PEO spherulites and decrease their size, which is beneficial for decreasing the crystallinity of PEO. There exists a much more continuous amorphous phase of PEO in PEO–LiClO₄/MMS compared with the continuous crystalline phase of PEO in pristine PEO–LiClO₄, which results in enhancement of ionic conductivity. The excellent lithium transporting properties combined with high decomposition voltage ensures the use of PEO–LiClO₄/MMS as a candidate electrolyte material for all solid-state rechargeable lithium polymer batteries.