This paper presents a comprehensive study to evaluate the influence of graphene oxide (GO) concentration on the physiochemical and mechanical properties of cement mortar composites. Scanning electron micrographs (SEM), energy-dispersive X-ray spectroscopy (EDX), X-ray diffraction (XRD), thermogravimetric analysis (TGA), and Fourier transform infrared spectroscopy (FTIR) characterizations were performed to understand the correlation between physicochemical and observed axial tension and compression properties of GO–cement mortar composites. The results show considerable concentration dependence, with the optimum concentration of 0.1% GO that increases the tensile and compressive strength of the composite by 37.5% and 77.7%, respectively. These results are explained by the stronger bonding of calcium silicate hydrate (C–S–H) components in the cement matrix in the presence of GO sheets and the dependence of their dispersions and possible aggregation.