A mini-solid phase extractor (MSPE) was designed and applied in the on-site determination of trace levels of chloramphenicol. The results showed that 1.5 g of the macroporous XAD-16 resin was suitable for capturing chloramphenicol from 500 mL of solution. Methanol was used as the eluent to desorb chloramphenicol from the resin. Chloramphenicol was determined by HPLC at 278 nm with a mobile phase of 0.01 mol L−1 oxalic acid–acetonitrile (40 : 60 v/v), the retention time was around 5.07 min. The detection scope of chloramphenicol was between 0.004 and 0.120 mg L−1. Using the MSPE, chloramphenicol in surface water and hospital sewage was determined with recoveries of 96 to 108% and RSDs of less than 6.8%. The MSPE achieved a rapid on-site preconcentration of chloramphenicol and avoided the inconvenient transportation and storage of large volumes of water sample.