We have modeled single-atom radiation damage events in diamond by a molecular-dynamics simulation, using an empirical interatomic potential to describe the interaction between the atoms in diamond. We find that the damage threshold energy needed to displace a single atom is well above simple estimates based on the diamond cohesive energy. The threshold derived from our simulations is approximately 50 eV, and is relatively insensitive to the direction of initial motion of the displaced atom. The high threshold energy is due to a rapid dissipation of kinetic energy from the bombarded atom into incoherent vibrational energy of its neighboring atoms before the displaced atom can overcome the structural energy barrier to defect formation. This rapid dissipation can be understood qualitatively by noting that when the kinetic energy of a carbon atom is comparable to the damage threshold energy, its velocity is comparable to the speed of sound in diamond.