A: If this is your first such invitation, then I assume you haven't been with your company for very long. So do a bit of role-reversal in your head and imagine what your company would think if they knew about your reservations. 'I gather young Gideon's not very happy about having to spend a whole weekend with the people he works with. He obviously doesn't find us of quite the intellectual and social standing he's been accustomed to. Maybe young Gideon needs taking down a peg or two...'
That may be grotesquely unfair, but it's how it seemed to me, and I'm pretty sure it's how it would seem to your employers.
It's not a question of etiquette, it's just plain common sense. You're talking of just one weekend, for heaven's sake. It may be quite appalling or it may be surprisingly instructive. There probably will be appalling bits, and instructive bits too.
But the whole point of such events is that they are concentrated, shared experiences. It's a team-building exercise; and you can't build a team if one member keeps wafting in and out, sampling the bits they fancy and popping back home during the bits they don't.
Of course you must go for it. Put your scepticism on hold and go for it wholeheartedly. And if you end up being the only one who found it an absolute nightmare, you're probably in the wrong company - a view your company would almost certainly share.