I believe a big reason against handloads is that you can't go back and verify what the exact load or bullet balistics, powder residue, etc were. This opens up several problems including the claim that you loaded them extra hot or some other nonsense. Again, it could be a problem if you hit something other than what you were aiming for and the velocity of your bullet was called into question.
The lack of verification is not entirely accurate. This concern has its origins in the New Jersey case where the man worked up a series of reloads with increasing powder charges to test load development. Somehow between the time he reloaded the cartridges and shooting them at the range, his girlfriend loaded up a handgun and committed suicide. There was minimal gun shot residue (GSR). The prosecutor said the lack of GSR was due to the man shooting her at a distance, i.e., a homicide. The man said she shot herself with a load containing a minimal charge. Unfortunately for him, because of the variety of charges he made, there was no way to determine which charge she used. Without the ability to duplicate the load, there was no way to test whether the GSR was consistent with a minimal load or homicide. So, New Jersey man has all sorts of legal problems for years to come. Overall, a fairly unique circumstance.
Compare that to the typical reloader. Most reloaders are going to do batches of at least 100 rounds to use a full tray of primers and/or box of bullets. In this instance, the rest of the rounds in your firearm are the same load as the ones discharged. Now, there may be some problems with testing those rounds because they could be sitting in the evidence locker. However, the rest of the rounds in the box at home are the same load. One would just need to avoid the situation of shooting the bad guy with the last round of the batch.
The bigger problem comes if you have to duplicate the load. The odds of getting primers, powder, bullets, etc. from the same batches is slim. One of the touted benefits of factory ammo is that the manufacturers keep a library of production runs. If one can at least get the lot number from the ammo, then the factory may be able to provide exemplar rounds.