Sunny skies and warm breezes sound great. But in reality, you might be better off retiring closer to home.