We report on a set of high-precision measurements of nuclear binding and excitation energies, as well as nuclear spins, magnetic dipole and electric quadrupole moments of neutron-rich silver isotopes, 113−123Ag. The measurements were performed using the JYFLTRAP mass spectrometer and the collinear laser spectroscopy beamline at the Ion Guide Isotope Separator On-Line (IGISOL) facility. For the first time, we can firmly establish the ordering of the long-lived 𝐼𝜋 = 1∕2−, 7∕2+ states in these isotopes, and pin down the inversion of these two levels at either 𝐴 = 121 (𝑁 = 74) or 𝐴 = 123 (𝑁 = 76). We compare these findings to calculations performed with density functional theory (DFT), from which we establish the crucial role that the spin-orbit strength and time-odd mean fields play in the simultaneous description of electromagnetic moments and nuclear binding.