Sorry Andy, but it's still humour, I could easily have said the same thing quoting exactly the same scenario and I expect many would agree with me. If he'd have said "partner" then I expect no one would have said anything, though, I suppose.
To call it sexist, though, is making a mountain out of a molehill, it was said in jest with no intention to offend.
Trust social media to blow things out of all proportion.