Size-dependent structural and luminescent properties of the diamond polycrystals produced by shock wave synthesis followed by grinding and separation into fractions of different polycrystal median sizes (25–1000 nm) are studied by comparative Raman and luminescence spectroscopy. The intense 738 nm narrow band luminescence of the SiV-centers are observed for all fractions. The SiV luminescence intensity has a maximum at the median size of about 180 nm that is controlled by competition between deactivation of the SiV-centers by defects in the diamond nanocrystal lattice and that controlled by nonradiative recombination centers in the volume of the intergranular layers.