It's cold.
It's freezing cold, cold to the bones, swimming in Arctic waters cold.
Which is exactly what he's doing.
He doesn't know how long he's been doing it. As in, he literally doesn't know how long he's been doing it: he can't remember how he got here, where he was before, where he's from, where here is, exactly. His memories are of cold, his thoughts are of cold, his entire being is focused on trying to make the cold stop. And the only way the cold will stop is if he keeps swimming, or dies.
He's so tired, and he's not sure what "die" means, but his soul is telling him in no uncertain terms that it's bad. Worse than the cold.
So he keeps swimming.