Crossbowmen were professional soldiers. The idea that they either chose to or were unable to perform any other military or fighting role is not even remotely accurate.So, if you're defending a castle or a town, things going south means you've pretty much lost. Time to surrender.
Out on the field of battle, if things go south, you are only a rather small contingent in a larger army. Time to withdraw or just plain run. If you're a crossbowman and the enemy is within arm's reach, you were in trouble a long time ago.
That said, pretty much any man-at-arms would have had a dagger. That's for if you run and they catch you.
For fantasy yeah sure, have at it. I'm just saying how things look from here. I'm pretty sure musketeers and other gunners didn't carry swords either. I'm absolutely open to seeing evidence to the contrary though. I'm a socio-economic historian, not a military one.
Think of all the scenarios that would crop up on a military campaign where a crossbow couldn’t be used:
The sun goes down.
In the woods or in a city where the enemy can get close to they can only get off a single shot before the enemy closes.
After they run out of ammunition.
Their crossbow gets damaged or lost.
The idea that a professional soldier confronted by any of those scenarios would have no choice but to immediately throw their hands in the air and surrender is not realistic.