Every time you go down in cartridge/bullet size, the bullet you select becomes more critical. This probably would never be a problem unless you pursued something quite a bit larger than deer, as picking a bad 6.5mm bullet today would be hard.
Having said that, a LOT of Scandinavian moose (about the size of our elk, I've read) have fallen to the 6.5x55, moreso than any other caliber. The game officials there survey and keep "one-shot-stop" data. They find little difference in killing power and the distance the animals run, between the 6.5 and larger calibers.