As a rider I don't ride unless I've had a chat, and even then I feel very guilty riding without having done some work.
As a builder I don't mind people riding when some of us are about, if people are willing to fix stuff they break, pick their litter up and are decent guys. Hopefully prepared to dig a bit as well. We had some kids come over Easter who asked me and my mate "whether they were ready yet"!!! whilst we were digging. No mate, fuck off- but even then we didn't get aggro, bad attitudes equals stuff being trashed. We're chaining them this year though, mainly because its on private land so we don't want people getting hurt and sueing etc going on. Luckily the village doesn't have a train station so its pretty difficult for people to get out to them if they can't drive (and if you're old enough to drive and still riding you generally have a reasonable attitude, in my experience).
I'm always up for riding with new people though, after a few years where sessions have only consisted of two or three of us its difficult to remain too militant. And for the few people who do make it out to ride its just a one off thing- but if we had people every weekend who hadn't done any digging that would wind me up no end.