Informal human evaluation allows enforcement based roughly in spirit—a “feeling of reasonableness”—but modern institutional bureaucracy demands letter laws for legal accountability, transparency, and consistency of enforcement. What’s more, public guidelines and product advisories must hedge against tail risk, adversarial misinterpretation, and a lack of common sense (for which they might prove legally liable). We shouldn’t underestimate the extent to which this norm changes the incentive and communication landscape.
One class of torque policy I’ve started calling “least common denominator (LCD) messaging.” This is the idea that, for reasons of legal liability and broad public messaging, public guidelines and advisories must cater to uncharitable interpretations and boneheaded users. They must attempt to torque against behavior that most of us would see as lacking in common sense. They must produce populationally robust advisories by paying for a small dip in false negatives with an abundance of false positives.
Warning signs at campsites and National Parks concerning fire hazards and bears are frequently targeted at reducing the sloppiest behavior of society’s worst offenders—put kindly, to prevent idiots and city-slickers from getting mauled. Precautionary signs, trespassing warnings, health advisories, and regulatory hoops may be geared toward making the environment “idiot-proof”: if you’re already a relatively careful, conscientious person in general, you’re probably OK taking these warning more lightly, with grains of salt. Unfortunately, more conscientious individuals are also more likely to update (or “over-update”) on these sorts of warnings, than are less consientious individuals.
Of course, sometimes messaging is not geared to lowest common denominators. Sometimes advisories should be followed to their very letter, uniformly. But the presence of torque policy and LCD messaging leads to a muddied epistemic landscape, where it’s unclear how seriously to take and update on certain messages.