Indium selenide (InSe) single crystals have been considered as promising candidates for future optical, electrical, and optoelectronic device applications. In particular, due to its 2H stacking and non-centrosymmetric feature, ε-InSe is highly desirable for second harmonic generation. However, due to its complex phase diagram, the high chemical activity of the elements, and the large formation energy, single-phase crystal growth of ε-InSe still remains a challenge. In this work, we present our efforts using a horizontal gradient freeze (HGF) method to fabricate InSe single crystals for the first time, whose high-quality crystallinity has been confirmed by both X-ray diffraction (XRD) and transmission electron microscopy (TEM). Moreover, we further combine the photoluminescence, Raman spectroscopy and infrared absorption data to identify this single crystal to be ε-InSe. Finally, thickness-dependent second harmonic generation (SHG) measurements were carried out on our few-layer InSe samples, and the results demonstrate its promising application for non-linear optics.