As so often in matters audio, this sounds right, but it isn't. No offense to Handsome.
To illustrate. At the resonance freq of a speaker driver, we can easily have 5 times the impedance compared to the nominal (say 8 Ohm) impedance. So at that freq the amp with a damping factor of 320 has a damping factor of 1600. What does it do to control the movement of the speaker cone? Nokkol niks. Zip. Our only chance of taming the movement of the cone at resonance is correct speaker box volume and port tuning. The amp won't help us at all.
The amp will control cone movement better if the amp is a current source. Like a Howland current pump. But I don't think any standalone commercial amps are current pumps, as they have to be designed with the speaker impedance in mind, to compensate for the above mentioned driver resonance for instance. They go together and you can't mix and match amps and speakers then.
Strangely, an ideal current pump has an infinite output impedance so the most rotten damping factor possible. Could work well in audio as the cone movement (the force applied to the cone) is actually determined by the current in the voice coil, not by the voltage across the coil. The current pump supplies whatever voltage is required to establish the desired current in the voicecoil. My understanding is that this lowers driver distortion appreciably. I've toyed with the idea but never tried it in a power amp. On a much smaller scale, it has made the summing buss in daisy-chain comms systems a total joy though
So my vote goes to "Damping Factor in an audio power amp is complete nonsense." Whether there are formulas and measurements to determine it accurately or not. I love measurement, but more as a sanity check. And sanity is often a slippery idea.
Use the amp that has enough power and that sounds right with your speakers. Understanding why it works is probably impossible. Too many notes. Just fun to try.