Mass-selected nanoparticles can be conveniently produced using magnetron sputtering and aggregation techniques. However, numerous pitfalls can compromise the quality of the samples, e.g. double or triple mass production, dendritic structure formation or unpredicted particle composition. We stress the importance of transmission electron microscopy (TEM), X-ray photoelectron spectroscopy (XPS) and ion scattering spectroscopy (ISS) for verifying the morphology, size distribution and chemical composition of the nanoparticles. Furthermore, we correlate the morphology and the composition of the Pt_xY nanoparticles with their catalytic properties for the oxygen reduction reaction. Finally, we propose a completely general diagnostic method, which allows us to minimize the occurrence of undesired masses.