We report a novel water-soluble polysaccharide from Rhizoma panacis japonica, namely, PJ-1 with anti-inflammatory activity. We first obtained a polysaccharide from this herb by eluting with water. After fully characterizing PJ-1 with a series of chromatographic technologies, we evaluated its biological activity by stimulating Raw 264.7 cells to express inflammatory cytokines and found that PJ-1 effectively enhanced the production of anti-inflammatory factors such as IL-10. We further identified its anti-inflammatory effect in a mouse model of sepsis induced by a lipopolysaccharide. The treatment of PJ-1 evidently suppressed the secretion of major inflammatory cytokines and increased the survival rate of mice. Particularly, we verified the STAT3 signal pathway responsible for the anti-inflammatory activity. Thus, PJ-1, which has remarkable anti-inflammatory effect and mechanism of action, may be a promising candidate as an anti-inflammatory agent.