The biggest little guy

by Collin Lysford

Have you heard of Thiomargarita magnifica? At time of writing, it’s the biggest known bacterium in the world. When I say big, I mean it. At around 1cm, it’s visible to the naked eye, over 50 times bigger that anything that had been found before. But the really interesting bit is when it was discovered: 2022.

I think many people have a sense that the natural sciences are now somewhat domesticated. We’ve mapped all sorts of genomes, the Wikipedia page of a living organism has all these nice categories, so we’ve basically figured it all out, right? Well, the first time someone beheld Thiomargarita magnifica, they got the very root level of that classification wrong. It was so big they figured it had to be a fungus. If you’re not looking in the right spot, you’re just not going to find, and that’s true even if you’ve got a principled classififcation for stuff you have found.

I especially love this bit from the Wikipedia page:

Metabolism in bacteria can only occur through the diffusion of molecules of both nutrients and waste through the interior of the bacterial cells, and this places an upper limit on the size of these organisms. The large sulfur bacterium T. namibiensis, discovered in 1999, overcomes this limit by including a large sac filled with water and nitrates. This sac pushes the cell contents to the cell wall, so that the diffusion can work; life processes occur only “along the edge” of the cell. T. magnifica ’s cell includes a similar vacuole[3] that occupies most of the cell (65–80% by volume) and pushes the cytoplasm to the periphery of the cell (the thickness of cytoplasm varies from 1.8 to 4.8 microns.)

You can so easily imagine a biologist in 1998 saying something like “While we can never get a definitive answer, we have a strong suspicion that we have seen the largest bacterium, due to fundamental constraints on the geometry of the cell membrame. This isn’t any sort of scientific egotism, but a direct consequence of the square-cube law, burned into the mathematical framework of the universe.” And then T. namibiensis rolls up like “I got big water in my tummy :)” and now T. magnifica’s like “wow look I made my tum tum extra big :) :).” Once again, a natural law ended up being more of a natural wet floor sign. Evolution walked a little differently than normal to get where it was going, but the pressure of millions of years has a creativity all it’s own.

The moral of the story: you can still find an order-of-magnitude extreme value across an entire domain of life by just picking up the wet thing and checking whether it’s fungus or not. The age of anecdote is not remotely at an end. It’s not just learning about the natural laws, but also sniffing around for every single way to cheat ’em.

ecology stamp collecting

Samuel Paredes, director of DIARIES (notes and sketches)

by Cristóbal

The artifacts that remain of this artist’s work are compact—a few terabytes of documents, storable in a single hard drive. The artists’ friends relate his obsession with recreating Jonas Mekas’ DIARIES (notes and sketches). His viewing of the Mekas retrospective at Lincoln Center in early 2022 coincided with the introduction of synthetic representation techniques. Emerging from Walter Reade, the artist had felt compelled to describe what he had seen. Such an earnest rendition of domestic life was foreign to him. He began to write, hoping that the black-box would render the imprint left on his mind’s eye. He pirated the film in a measly 480p to step through, frame by frame, and describe the movements of the people, the flickering of the lights, the hazy gouache of film. Friends relate that he would spend whole evenings with a single frame projected on his apartment’s wall, the wall itself plastered in graph paper, to describe with affine accuracy the composition of the frames. The scene descriptions, often hundreds of pages long, were fed into the black-box, which produced a film which was itself projected back onto the wall for inspection. Frustration arose from the imprecision of the prompts. One of the last social events the artist is known to have attended is a reading group of early 60s editions of L’Express. It was then that he began to strip down the language, incorporating the gaze of the camera into an intricate system of object notation. It wasn’t sufficient. If only he could know what Mekas was thinking the instant before capturing those handful of frames on his Bolex, then perhaps he could conjure the words that would induce the film. This is when the artist’s late phase begins. He hires set designers and actors to re-enact scenes from the film, himself occupying the role of Mekas. Long sessions were performed, followed by maniacal bouts of writing. The description was fed back into the black-box, and viewed again. This process continues until the author’s death. Careful to keep track of all of his progress, the artist stored all of his prompts via an open-source versioning system. The git repository is what was found in the hard drive. It is available online on the archive’s website. Our archival team avoids bit rot through periodic replacement and has replicated his work across five different disks, distributed between five cloud service providers.

I scream, you scream, we still underrate indexicality

by Suspended Reason

Here’s a tautology we can all sign-off on: “Whether a meal is healthy or unhealthy depends on the metabolism of the organism consuming it.” Might as well say “The solution to a problem is indexical to the problem it solves.”

This is in theory. In theory we say, “The word healthy tracks how a food item contributes to a given organism’s health!” In practice, we go around thinking about food as being generalizably healthy or unhealthy, as in a vacuum, as if it were an inherent or eternal property of the food item itself. I agree—the generalization is a shortcut, heuristic, approximation—it saves you time, it’s often necessary or optimal or “rational” or whatever for us to employ these mental shortcuts. But maybe every once in a while there’s alpha in re-grounding ourselves with a more indexical relation to the world.

(We also tend to think of it as a purely biological designation, but it’s not; see below. Is it also true that there’s serious correlation in how a given food item is metabolized across organisms and species, for reasons that are ~obvious given what we know about evolution? Yes—and there’s no contradiction here.)

So is ice cream healthy?

Well, it’s associated with desserts. It has a decent amount of sugar. (So does fruit.) It has a non-trivial amount of fat, but not a massive amount—significantly less than peanut butter. It has a moderate amount of calories but again, not super excessive—a cup of ice cream will run you at about two slices of toast and OJ.

On the other hand, ice cream’s very high in protein, it’s maximally hydrating, and a lot of us could use more protein and fluids. If you’d been fasting in the desert a week, lost some weight, and were severely dehydrated, ice cream would be one of the best things you could eat on that swollen stomach of yours.

Do we eat it as part of our nutritious breakfast? Maybe we should. It’s a nearly identical nutritional profile to yogurt. Some people who feel they get too much sugar, or are hoping to lose weight, might want to moderate their intake. But when we say “ice cream is unhealthy” we are really saying something like, ice cream as it is conventionally, socially consumed—that is, as a regular and additional (supplementary) dessert on top of an average diet and lifestyle—is usually detrimental to a person’s health.

I think it’s fair to say this is an obvious point. That at some level no one really believes the non-indexical, non-constructed account of “healthy.” And yet, note how weak a statement it is—the number of conditionals tacked. The narrowness of circumstance in which it applies. For instance, if you ate ice cream for breakfast. (That violates the additional/supplementary dessert clause.) But how many of us do eat ice cream for breakfast? And we can all imagine the reactions we’d get if friends or family saw us eating ice cream for breakfast.

This is a playful example, but many of the words we use and depend on have similar levels of indexicality, social construction, and porting problems. We reify their simplifications and end up mistaking recommended rituals for first principles.

A post-script:

Chris Beiser links to an article in The Atlantic that talks about how nutrition science ends up handling a pro-ice cream result. It’s symptomatic to say the least.

As Ardisson Korat spelled out on the day of his defense, his debunking efforts had been largely futile. The ice-cream signal was robust.

This was obviously not what a budding nutrition expert or his super-credentialed committee members were hoping to discover. “He and his committee had done, like, every type of analysis—they had thrown every possible test at this finding to try to make it go away. And there was nothing they could do to make it go away.”

And:

The Harvard researchers didn’t like the ice-cream finding: It seemed wrong. But the same paper had given them another result that they liked much better. The team was going all in on yogurt. With a growing reputation as a boon for microbiomes, yogurt was the anti-ice-cream—the healthy person’s dairy treat.

indexicality nutrition torque policy

‘Do You See What I See?’: the conjunction fallacy & latent incentive structures in experimental settings

by RIPDCB

The conjunction fallacy — ever heard of it? One of the more controversial bias experiments of the past 40 years (RIP DK), the conjunction fallacy elicits the strongest responses from believers and non-believers alike. The initial findings were so offensive that Kahneman & Tversky were tasked with replicating their results again, and then again, and then again…concocting variation upon variation of the original experiment until they had managed to isolate every possible interpretation of ‘probable’ while also testing different social contexts (casuals & industry-specific folks alike). This is all to say: most of, if not all, of the pushbacks against Kahneman & Tversky’s findings have been responded to in some experimental form or another.

Here’s a helpful explanation of the conjunction fallacy, by way of Yudowsky:

In the conventional interpretation of the Linda experiment, subjects substitute judgment of representativeness for judgment of probability: Their feelings of similarity between each of the propositions and Linda’s description, determines how plausible it feels that each of the propositions is true of Linda.

Representativeness and probability get mistaken for each other by subjects who select the description that they think is most representative of the situation over the description that is, in fact, most probable. That this would happen is believable, especially for those who do not carry with them a mathematically rigorous conception of probability (experiments on those with backgrounds in statistics have also been run and the results are similar, for what it’s worth).

This representational fallacy is probably real. But why do we mistake representativeness for probability? For Yudowsky, Kahneman, Tversky, and Scott Alexander, the answer is cognitive bias. And while that very well may be the case — that in certain situations we instinctively go against our most rational selves — there might be an alternate explanation, or at least a complementary one, that deals with the structures of experimental settings.

When Pierre Bourdieu was first getting started, he went to study the Kabyle people in present-day Algeria. He wasn’t the first, though. Structuralist anthropologists had been working in the area for decades, speaking with informants in order to build genealogy charts. These charts would become quite influential in understanding the marriage rituals and structures of the Kabyle people, namely the relative prestiges of different arrangements (paternal cross cousin down to maternal same-side cousin). They deduced that your social rank would define which partnerships were available to you, and those partnerships would then reinforce your social rank. For the structuralist anthropologists, they believed that this hierarchy was culturally inscribed—i.e. that it was adhered to predominantly because of a shared belief system and community-wide buy-in.

Bourdieu found something else entirely, however: most marriage arrangements were not a function of ritual & social rank, but instead born out of very practical concerns. How people were paired often came down to capital (who had it and who needed it), protection (again, who had it and who needed it), and environmental factors (e.g. a hard-hitting drought might push someone to accept a marriage arrangement they otherwise might not). The disjunction between his findings and those of previous anthropologists came down to a matter of formalization. Bourdieu believed that the informants––in sitting down with social scientists who were trying to map a culture through a series of rigid, objective relationships––tended to reify the practical circumstances of life in order to fit the formalizing desires of their interlocutors. Sensitive to the power structures underlying the relationship between anthropologist and tribal informant, the latter wanted to meet the former at his level, ultimately providing a misrepresentation of the logic driving the informant’s culture. This misrepresentation was probably in good faith (they wanted to be helpful), but a misrepresentation nonetheless, one that the informants themselves may have even believed despite their experience of the material realities that influenced their lives.

Around the time Bourdieu was studying the Kabyle people, psychologist Martin T. Orne was growing frustrated with his scientific practice. Social experiments he would run on UPenn students kept coming up all-too in line with his hypotheses. What he kept butting up against was quite similar to what Bourdieu found in Kabyle: his subjects were providing him with the answers that they thought he wanted. Subjects were picking up on the expectations of the experiment and Orne’s expectations more generally, and were trying to anticipate the ‘right’ response, rather than provide an intuitive or rational response. Orne came up with the term demand characteristics to describe the structural features of experimental settings that might push subjects to give certain responses over others. Examples range from subject-experimenter relationships to foreknowledge of the experiment to the physical features of the lab itself. Incentives are latent yet legible in the structures of experiments, despite a scientist’s best intentions, and accidentally incentivized responses may very well muddy the validity of a given study’s findings.

This is all to suggest that the conjunction fallacy may be a bias circumscribed by the payouts subjects project onto experiments. In the Linda experiment, subjects may have been flexing their interpretive skills when they selected “bank teller + feminist” for the most probable proposition, emphasizing for Kahneman & Tversky that they were capable of making connections between the experiment’s frame and the possible answers. In the version of the experiment run on medical internists–where the internists were asked to assess the probability of different symptoms after a patient’s pulmonary embolism—the subjects overwhelmingly picked “dyspnea and hemiparesis” (the former is quite common to pulmonary embolisms, the latter not at all), and perhaps they did so because they wanted to prove that they had, in fact, studied their books. Parallel incentive structures can also be unearthed in the ‘Russia invades Poland’ example that Yudowsky talks about extensively.

Ultimately, I think it’s unclear the extent to which subjects are confusing representation for probability instead of selecting for representation over probability. I want to emphasize that this demand-characteristic reading of the conjunction fallacy does not preclude such a bias existing in an experimental setting. However, it puts the experiments in a different light, one that might help us better contextualize social scientific findings. If in fact the conjunction fallacy is in any part the product of subjects responding rationally to incentive structures unintentionally baked into an experiment, then the relevancy of cognitive biases in non-experimental settings might look quite different.

Daniel Kahneman conjunction fallacy behavioral economics bias probability Pierre Bourdieu

Adversarial asymmetry

by Collin Lysford

A comment from RIP DCB on the original draft of The Mongolian meta:

i don’t know much about large language modeling, but i would guess part of your point is that there’s a strategic advantage in being the adversary who can extrapolate your enemies structural flaws when those flaws are so site-dependent?

This is certainly true, as far as it goes. The Street View employee acting as a predator model doesn’t need to come up with the whole meta to start inverting it. But there’s a deeper and more profound lesson here, so I want to take the time to clearly parcel it out.

One way to tell where you are in Mongolia are the mountains. These are giant and striking and effectively impossible to move. Your relative position to the mountains are exactly what it means to be at a certain location. Breaking this correlation takes a massive amount of energy — you’d need to level the mountains themselves.

Imagine this correlation on a scale, and on the other end of that scale is the position of the spare tire of the car doing the mapping. This has absolutely nothing to do with where you are in Mongolia. You just have a spare tire in some spot or another that just happens to correlate because you took many pictures in the same location with the tire wherever you threw it that trip. If you’re going to take a new picture of that spot, you’re going to just throw the spare tire wherever. It would take more energy to keep that correlation intact, because you’d have to remember how you did it before.

When you’re trying to find the geographical location of a photo using elements in that photo, some of the elements are tightly coupled with that element in the world (it would take years of industry or many high-yield bombs to change a mountain’s relationship to a location in Mongolia). Other elements aren’t coupled at all in the world, only the dataset (you can put your spare tire wherever you want as you drive through Mongolia). As a predator model, your goal is to take a picture in Mongolia that gets incorrectly classified by the meta. If everything in your picture was as invariant as the mountains, you’d be stuck. If the meta had an account of variation that properly privileged the mountains and shut it’s eyes to the other stuff, you’d be stuck. But the meta is trying to get all of the juice it can out of the data there is right now, and that means it’s using the information with respect to its predictive power, not its invariance. So you can take every single measure that’s totally uncorrelated with location in the world and change it to the exact opposite the meta expects, for free.

Note that if the initial dataset was bigger, the meta would be forced to adapt and become less strictly predictive but more robust to variation. Because the spare tire really is arbitrary with respect to location, there will be pictures of the spare tire in different places in different locations. You’ll get a clue that it doesn’t bind to location as strictly as “spare tire in a net means Western Mongolia.” But unless your pictures have the infinite permutations all mapped, with every single possible spare tire orientation present at every single location, there will still be some correlation, some combos that happen to exist and some that don’t. And the predator model will still be free to take those spurious correlations and reverse them.

This is why even ultra-powerful AIs can be easily snowed by dedicated adversaries. Physically existing and trying to move stuff gives us strong indications of what correlations are extremely difficult to change, which are dead-simple to change, which are linked by social convention but could be reversed by a sufficiently eccentric person, which are linked now but could change dramatically later. I am not saying these distinctions are impossible for AIs to ever learn; I am saying that doing so necessarily diminishes their predictive power over the data that exists today, because it entails throwing away the parts of the meta that won’t survive change. So an AI that is trying to optimize score does so exactly by considering every single bit of the meta, thus becoming more susceptible to predator models. They don’t need to survive change and so they’re not trying to. The strategy to build a living thing that will endure is altogether different.

predator models games anti-inductivity Geoguessr ecology unit of survival mesa-optimization

Intergenerational knowledge transfer in hunting

by Feast of Assumption

One of the primary themes I write about is the intergenerational transfer of knowledge. Hunting is an interesting case: in the modern era, it’s practiced only 9 days per year. In this short season, enough information must be transmitted to enough people that the tradition will be preserved even if the patriarch doesn’t survive the next year. And nearly every hunting camp that you pass in the orange-dotted woods has executed this transfer successfully—I see many more successfully propagated hunting traditions than I see family businesses.

Knowledge Transfer Mechanisms:

Trad,

Plenty of information (affectionately termed ‘knowledge’) is easy to pass via book-learning. (What caliber of ammunition will be able to deliver enough force to take a particular quarry at a particular distance, and thus what rifles are suitable for hunting which quarry.)

Modern,

What Samo Burja describes as tacit knowledge¹ used to be hard to pass down, but has become easy with the Youtube Revolution.

And video transmission of some forms of tacit knowledge is splendidly effective. Letting novices watch a few videos of the field-dressing process before offering them a knife for the first time greatly reduces the instances of unpleasant smells produced by an unsure hand in an unfamiliar gut.

and Trad Again

But what makes a hunting party successful isn’t field dressing technique—it’s hunting technique²You can have the right rifle all day long, and you can be the slickest dresser in the county—but these do you no good if no deer passes in front of you and presents a clear shot.

And while a non-hunter might think deer appearance is stochastic, it isn’t. Deer behave in ways that are predictable when fully zoomed out. “Deer are active at dawn and dusk, still during day and night”; this is simple enough to write down, convey across time and space. But where they’re traveling from and to, their preferred routes to take, are pattern-ful—but not in a way that can be shared outside of a specific ~200 acre ecosystem. When you know the land and the patterns, though, you can place yourself and your party members in the right spots. To a well-placed hunter, shots will present themselves.

The knowledge of how local deer move and live can only be transmitted by your hunting party interacting with those specific local deer, on that specific land, over decades.

A Description of the Knowledge

It may be ‘intellectual dark matter,’ but I’m at least going to gesture at the shape of it.

In my state, there are three main forms of hunting discussed. The first is no longer practiced, but it still comes up regularly as a contrast object. The three happen to chunk neatly into our “knowledge, tacit knowledge, hypercontextual knowledge’ categories.

• still hunting: a nearly extinct technique because it requires a critical mass of unrelated people, usually on public land, walking and pausing, walking and pausing, waiting for deer. Occasionally you’ll flush a deer for another unknown hunter, occasionally another unknown hunter will flush a deer to you. This was a popular technique in my grandfather’s day, but since it’s not practiced anymore, if you were to try it, nobody would flush a deer to you, and any deer you flushed would go unnoticed. (Or, be noticed by someone in a tree stand who will never return the favor.) If still hunting were still practiced today, you wouldn’t need anything except a rifle, marksmanship, and patience to succeed.
• stand hunting: popular because it can be practiced alone. Deer tend to travel at dawn and dusk, and remain bedded down or grazing during the day and night. Stand hunters will install stands in trees before hunting season, along known deer paths. Climbing the stand an hour before dawn to patiently wait for a deer to pass along the path, coming home for lunch, and returning to the stand an hour before dusk—this comprises hunting for the solo hunter. It can be learned from books and youtube, especially if you integrate local information from trailcams.
• drive hunting: “Posters” wait, alert, at geographic funnel points. “Drivers” enter a stretch of woods and walk in formation, noisily, to startle bedded deer and send them into the waiting sights of the posters. ³

The benefits of drive hunting are plenty. You can hunt all day instead of being confined only to the natural movement hours of dawn and dusk. You enjoy more scenery, and while driving you keep warm from exertion. You harvest many more deer per hunter per year than even a well placed stand. Effectively, improving the design and execution of your drives allows the ‘technique’ part of the equation to overwhelm the ‘luck’ part. Since the goal of hunting is to put meat in the freezer, drive hunting is the clear winner for anyone who can execute it.

But you can’t execute a drive well unless you know your land’s geographic funnel points. Where can deer be funneled? Which hillsides do they like, which hillsides do they avoid? Where are their natural paths? These answers can be earned by walks in the woods in the springtime before leaves have crowded the view, if they weren’t handed down from your hunting party who were on the land before you. But you can’t learn everything in one year. A mast year⁴ for oaks drives different deer behavior than a no-acorns year will. Corn vs oats vs alfalfa on nearby farm fields will change your local herd’s browsing patterns. Even funnel points can change year to year—we have a drive where a poster stands on a ridge over a hill if ice hasn’t formed yet. But if ice has formed, the deer will charge across the lake. So in an early-ice November, our poster will stand at the edge of the bay.⁵

Coordinating A Hunting Party

This is a lot of knowledge. The mechanism for conveying it (the hunting party) has to be robust. The successful way I’ve seen this achieved is with a lot of built-in redundancy.

Cousins are invited. A mix of age ranges is a necessity. To succeed in the long term in the modern era, a hunting party has to be robust to members leaving and coming back. “Oh no he started dating a vegetarian” is a real hurdle, as is “they moved across country for a job.”

A successful hunting party will have invited people for an occasional day of hunting, to assess if they’d be a good long term fit. Success breeds more success, as everyone wants to be a member of a cheerful party with meat in its freezers. But hunting parties cannot turn into ‘big tents.’ There is a correct size for a hunting party that varies with the land, but rarely exceeds 8. A hunting party wants its members to be safe, patient, cheerful, willing to do their share of gruntwork, and good shots.

A successful hunting party must also invite youth. A good patriarch will go out of his way to give the newest party members the most advantageous positions, so the kid can taste success early, whetting his or her appetite for future years. I’m not a patriarch so I’m only speculating, but I’d ascribe the definition of success shifting from “how big is this trophy” to “did my newest hunter get a shot at a deer?” being based largely in Honor and Pride at passing on enthusiasm for the traditions.

While bending over backward to show 12-year-olds the joys of the hunt, a hunting party (and family) must also be robust to slow starts. My dad is an avid hunter—but if he had forced me to hunt before I was interested, I would have hated it and never returned. Happily, I was allowed to loaf around and bake pies while the rest of the family hunted. When I eventually did develop an interest, I was able to start with a blank slate. I was able to celebrate the milestones of “my first deer” and “my first buck” when I earned them—I got no fewer congratulations at 31 than I would have gotten if I’d started at 12.

A Testament to Humanity

Considering all the folk arts which didn’t make it from 1400 to 2000, I’m in awe that hunting has survived and flourished. Against the odds—practiced 9 days a year instead of 365 (or even 52)—the fact that hunting parties assemble and keep the forest population balanced by harvesting wild meat fills me with pride in humanity. I feel honored to be receiving and carrying forth knowledge as a member of a successful hunting tradition.

folk knowledge folklore hunting knowledge logistics

The ecological perspective

by Suspended Reason

The ecological thought does, indeed, consist in the ramifications of the “truly wonderful fact” of the mesh. All life forms are the mesh, and so are all the dead ones, as are their habitats, which are also made up of living and nonliving beings. (Timothy Morton)

Riffing off “Erving Goffman, Ecologist”, I thought I’d write up the core principles I find most useful for thinking ecologically. I’m particularly curious which principles readers think I’m missing—I’d love to hear from you etc etc; my email’s suspendedreason@gmail.

1. The unit of survival is the organism-and-environment. Fitness is always contextual to a given environment. Because a given organism is only ever suited to a given distribution of environments, its survival depends also on that environment surviving (staying in-distribution). See also Bateson’s “Form, Substance, & Difference.”

2. Drift & disruption drive an adaptation ripple. When an environment is disrupted, the average organism loses fitness. Previous strategies and adaptations no longer work. Because the environment is always changing, it is useful to describe organisms as mesa-optimizers: optimizers who have themselves been optimized by a “base” process (e.g. natural selection). Drift is inevitable because ecology is too complex a system to ever fully stabilize. Due to drift and disruption, an organism’s optimization function differs (often quite dramatically) from what is optimal from the perspective of the base optimizer (e.g. natural selection).

3. Selection is frequency-dependent. Because the unit of survival is the organism-and-environment, the composition of that environment is critical. The population of a given species will have a strong effect on the survival chances of a single member. The population of a given prey species will have strong effect on the chances of a predator species. And so on.

4. What is true for an organism is true for a strategy. Organisms are merely bundles of strategies. A strategy, being a solution fitted to a problem context, only works if that context is maintained; if the environment is disrupted, the strategy no longer works. The effect of a given strategy depends on the history of strategies deployed. Is this a new, unseen strategy to which other organisms are naive, or is this a tired strategy to which they are well-accustomed and habituated?

5. What is true for a strategy is true for a resource. The poison lies in the dose (the frequency), e.g. too much water or too much salt are both fatal for mammals. For nearly any chemical, there is an (at least theoretical, but usually actual) organism for whom it is a sought-after nutrient, and also for whom it is a filtered-out toxin. (The few exceptions on earth might be carbon & water, since they’re so fundamental to the planet’s evolutionary history?) Oxygenation of the atmosphere led to one of the greatest die-offs in planetary history.

6. Every available resource ends up a seized opportunity. All waste products are nutrient opportunities. “Grifts” tile ecosystems—food stuck in an alligator’s teeth becomes birdfood; if plastic sits around long enough in landfills, bacteria will evolve the capacity to break down its hydrocarbons.

ecology frequency dependency Gregory Bateson fitness unit of survival mesa-optimization

The conjunction fallacy & Grice

by Frances Kafka

In “Here’s Why Automaticity Is Real Actually”, Scott Alexander describes the conjunction fallacy as “not only well-replicated, but easy to viscerally notice in your own reasoning.” Can we better understand what’s going on here?

It’s important to distinguish the various meanings that we attribute to the word “probable.” In fact, “probability” is a shaggy, irregular word which requires a lot of conceptual engineering for various specific domains, and it’s no coincidence that Carnap, who gave much stimulus to conceptual engineering, wrote a book that attempted to carve out probability (he distinguished probability1 and probability2, which I’m not going to touch on).

A cognitive scientist like Kahneman thinks of the word “probable” in a mathematical way, and identifies this statistical interpretation with the common notion of the word. Suppose we asked him the Linda question. His thought process would develop in this way:

“The experimenter is asking me which has a higher value, P(Linda is a bank teller) or P(Linda is a bank teller AND Linda is active in the feminist movement).” To him, the question is in the genre of a mathematics word problem, which is why he throws away all the extraneous, humanizing detail about Linda. You don’t have to be a Bayesian to think in this way; you can either ask in the frequentist way, “In a population of people with Linda’s broad characteristics, and I take samples, would I get more bank tellers or bank tellers AND active feminists?” A Bayesian would wonder, “What is the subjective likelihood (consistent with the laws of probability theory) would I assign to both P(A) and P(A and B)?”

Suppose, however, I drag Average Alice and Basic Bob into my laboratory and I present them with this exact question. Mind you, this does not preclude the possibility that Alice and Bob implement some sort of Bayesian-ish plausible reasoning in their brains (on that, see E. T. Jaynes’ paper here). For Alice and Bob, the word “probable” is essentially asking a social question. Of course a woman who is “deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations” is likely to be an active feminist! I’ll also point out that the first option, “Linda is a bank teller”, when contrasted with “Linda is a bank teller and is active in the feminist movement”, actually implicitly sounds like, “Linda is a bank teller and is not a feminist.” The average person, the average intelligent person who doesn’t stake their life on the genre of mathematical word problems is very reasonable in interpreting the options in this way for Gricean reasons. In fact, from a mathematical point of view their answers are quite reasonable, because the implied mathematical question is not “Which is larger, P(A and B) or P(A)?” but rather “Which is larger, P(A and B) or P(A and not B)?” You can use this to point out that it’s not the subjects who are wrong, but Kahneman.

In fact, I suspect that people would give a better answer if you made it into a matter of numbers. “Suppose I have a room of 100 women. How many of them are likely to be bank tellers, and how many of them are likely to be bank tellers and active feminists?”

The fact of Gricean norms comes up elsewhere, in Yishan Wong’s attempt to show that people don’t know how to understand counterfactuals. If you go up to your coworker and ask them, “If I were to go to France, where would I get a baguette?”, it’s a reasonable assumption that I am planning on going to France, because if not I would not be asking this question. (Unless, of course, you know that I’m asking this question to be insufferable, for which the appropriate answer is “Sod off.”)

A bigger divergence comes from Kahneman’s forecasting questions. Kahneman’s experiment involved asking two different groups of forecasters to rate the probability of two different statements, each group only being shown one statement.

1. “A complete suspension of diplomatic relations between the USA and the Soviet Union, sometime in 1983.”

2. “A Russian invasion of Poland, and a complete suspension of diplomatic relations between the USA and the Soviet Union, sometime in 1983.”

For Kahneman, a good cognitive scientist, of course all these disparate activities, the “probable” involved in the bank teller problem is the same as the probability involved here. Perhaps Kahneman (or perhaps a LessWronger, because they seem to be more keen in discussing counterfactual worlds) would reason this way: Imagine there are a hundred possible worlds in the future, and I tag off each world where the USA and the USSR suspend relations with a A, and I tag off each world where Russia invades Poland with a B, which is larger, P(A) or P(A and B)?”

Yudkowsky helpfully tells us this;

The scenario is not “The US and Soviet Union suddenly suspend diplomatic relations for no reason,” but “The US and Soviet Union suspend diplomatic relations for any reason.”

But this is not what the words imply; Yudkowsky and Kahneman might mean the second statement, but that is not how the first would be interpreted. In fact, I suspect that the forecasters involved would do better if 1) was simply “The US and Soviet Union suspend diplomatic relations for any reason.” without them knowing anything about Bayes’ theorem or the conjunction fallacy. A sentence is not a mathematical proposition but a narrative in nuce. The story presented by 1) is that there’s a sudden suspension of diplomatic relations. The story presented in 2) is more plausible, because again, these are narratives we are talking about, so of course you’d be more willing to press this. Moreover, it’s not clear that our forecasters are thinking in terms of “checking off” possible futures, but are thinking causally. It’s also possible that for sentence 2, they’re not thinking of P(A and B) but rather P(A|B): given that the Russians invade Poland, it’s quite likely that diplomatic relations will be suspended. These forecasters are probably intelligent people, people who are rather good at solving mathematics word problems, but even they have to be pushed to consider this like a word problem.

Paul Grice probability Rudolf Carnap conceptual engineering Daniel Kahneman behavioral economics language conjunction fallacy bias

KataGo and The Hook

by Collin Lysford

When I was a kid, I played this game called Backyard Football. You draft teams of children, come up with your own plays, and play football against a CPU or a human. I only ever played the CPU, and pretty quickly I only ever played one way. One of my random plays I invented was “The Hook”, where one wide receiver runs off at a 45 degree angle while everyone else gets gummed up in the center. Trying it out, I quickly noticed that something about The Hook short-circuited the poor CPU and it would never, ever see the receiver on the hook. So I’d play The Hook and whip the ball to Pablo Sanchez, who would have no defenders near him and inevitably run it and get a touchdown. I would end every single game over a hundred points up and was the undefeated master of Backyard Sports, at least on my own machine.¹

Turns out this kind of thing still works! Here’s a paper about adversarial policies designed to beat KataGo, a best-in-class Go AI:

We attack the state-of-the-art Go-playing AI system KataGo by training adversarial policies against it, achieving a >97% win rate against KataGo running at superhuman settings. Our adversaries do not win by playing Go well. Instead, they trick KataGo into making serious blunders. Our attack transfers zero-shot to other superhuman Go-playing AIs, and is comprehensible to the extent that human experts can implement it without algorithmic assistance to consistently beat superhuman AIs. The core vulnerability uncovered by our attack persists even in KataGo agents adversarially trained to defend against our attack.

I use the term “predator models” for these adversarial policies. They don’t beat their prey models by playing the game better than they do, in the same way that cheetahs doesn’t hunt zebras by eating the grass super quickly so the zebras starve to death. They attack the weak points, the arbitrary leverage the model used to get so good in the first place. One predator model just tricks KataGo into passing so it loses. Another one “wins by first coaxing KataGo into creating a large group of stones in a circular pattern, and then exploiting a weakness in KataGo’s network which allows the adversary to capture the group.” KataGo learns by self-playing, but it never coaxed itself into creating large groups of stones in a circular pattern, so it never learned how to play against it. By modifying KataGo’s training run to include some games in the process of exploiting this weakness, it gained a brief immunity to the predator model. But a predator model trained on this “immunized” KataGo was able to rebound from a 0% win rate back to 47%.

The most important thing to understand about predator models is this diagram from the paper:

KataGo didn’t learn how to defend itself from predators, and is helpless against them; the predator model didn’t learn how to play Go, and is helpless against Go players. If I had dipped my toes into Backyard Football multiplayer running my little predator model, The Hook would only work once before they start leaving some defenders on Pablo, and I would lose badly — even against players not capable of beating the CPU themselves. Finding a single ungoverned state in a hyper-optimized book of strategies is a lot faster than writing that book of strategies, which makes predator models energy efficient killing machines. But then you don’t have the book of strategies! So the cheetah ain’t eating grass and Pablo’s got no clue how to dodge a block.

Each year, the energy we put into AI grows dramatically, both in literal kilowatt-hours and in our conceptions, affordances, and imagination. Understanding this adversarial dynamic is going to become more and more important to get the leverage to move anything sufficiently big and computational. Take it from me, the world’s greatest football coach.

predator models games anti-inductivity football strategic interaction

Strategic advantages in slanted information states

by RIPDCB

In games of strategy, we tend to assume anti-inductivity: As new edges are discovered, they end up quickly neutralized by either widespread adoption, or the development of counter-moves. But sometimes, that process is much slower than we’d naively expect.

One of the oldest truisms in football is that you need to establish the run in order to pass. One of analytics’ first major wins in football was showing that an efficient passing game actually did not require an effective run game in order succeed. But then data pointed to an even more radical conclusion: that there was no meaningful relationship between an effective play-action pass game and an effective run game, which seems absurd on the surface. A play-action pass is when a QB fakes handing the ball off to a runner to get defenders to mistakenly step up so there’s more open space to pass. It’s a play type that, on the surface, relies on deception and the subversion of expectations in a contextually-rich environment. But what the data over the past 10 years has born out is that you can have a very efficient play-action passing game without an effective run game.

The dynamic at play here is that NFL defenders will instinctively defend the run before the pass if they sense any potential for a run. Given that the run game was the most important part of NFL offenses up until about ~15-20 years ago, that dynamic makes sense, but there’s so much data available to coaches and front offices that it’s weird there would still be no meaningful relationship between play-action passes and the run game. You’d think you could just coach defenders to not bite on the fake. Defend the pass first, the run second. And while in isolated contexts this approach might work, there have been no major shifts in play-action efficiency over the past 5+ years.

From the lowest levels of youth football up to the second-highest tier of college football, game planning starts “in the trenches”–i.e. it’s about the big guys up front being able to block the defenders in front of them, and the big defenders being able to beat those blocks. The game is largely a war of physical strength, and usually, at the lower levels of competition, quite mismatched given the talent disparities between good players and average players. In other words, for 99.9% of all people who play football, the game is one of brute strength based around running the football, because that’s all you need to win. You can apply high-level analysis to the game at this level but it wouldn’t make much of a difference, because analytic edges are not useful at a level where the dynamics at play are relatively simple.

So why are NFL defenders, the top .01% of the entire field, so thoroughly and consistently deceived by the mere threat of the run? Maybe it’s because they’ve been threatened by the run for the majority of their lives. They’ve been enculturated for years into a version of the game that changes radically when you get the highest level, and they just can’t adjust quickly enough. This disparity between past realities and current reality becomes a major edge for coaches who know they can manipulate defenders with just a little bit of ambiguity.

A couple takeaways: Suspended has spoken before about how lying is often prohibitively expensive enough to prevent people from consistently lying, given that they would rather be believed in the long term than not believed at all. In the case of play-action passing, it seems that the threshold for deception is so low that you can, within reason, continually deceive a defense given how embodied their knowledge is of the game. Their enculturation is so deeply embedded in their perception of reality that it can be manipulated without much of a long-term penalty incurred. Further, the majority of football coaches have had parallel experiences to the majority of football players, and as such may also privilege the run over the pass given their past experiences. Feedback loop exists that maintains at the professional level despite the edges that are being exploited–players believe the play-action pass, and coaches do, too, so the perceived reality maintains.

Second, I think the point about analytics applied to lower level of game states is interesting, sort of a reverse scenario of Collin’s WIFOM piece. You can gain an edge in WIFOM games by applying a higher level ontology than others. In the case of high school football, there’s no advantage in finding unexplored edges because the game state is too simple. Could you gain an edge by spamming play-action passes in high school? Sure. But the effective difference wouldn’t be great enough to warrant the work that goes into implementing a radically different game plan, and the discursive effort needed to convince an entire team of unbelievers. In other words, contextual fit will always supersede ontological complexity (if you can win with less, you should), even if the latter is better suited for comprehending the entire game state.

strategic interaction adversarial epistemology anti-inductivity sports football frequency dependency WIFOM games

Still against automaticity

by Suspended Reason

The psychology studies that got debunked in the replication crisis weren’t random; they were part of a theory or worldview that’s still pervasive, and we need to consider what it means for that worldview to be false. (Sarah Constantin, Twitter)

In rationalist and evpsych discourse, “signaling” is often motte-and-bailey’d. If pressed, the idea is treated as a neutral biological phenomenon, but in casual usage, it’s typically used accusatorily, as a pejorative, a synonym of “wasteful boasting.” Activities and individuals are summarily dismissed for spending significant energy on signaling. But the ethologist knows: “Signaling” is just another word for “communication”! It would be very strange to dismiss an activity for being communication-intensive, given that the modern educated class (i.e. everyone involved in signaling discourse) spends the vast majority of their time communicating. Communication games may be more or less necessary, more or less wasteful. Sometimes they incline toward zero-sum, but usually they’re meaningfully positive sum. But describing communication we dislike, or don’t understand the point of, as “signaling” just muddies the waters. We should strive to more clearly articulate the kinds of games, and the kinds of gameplay, that we find unethical. My hope is that we can make progress on this during S2.

A similar thing happens with behavioral economic (BE) discourse around the concept of a nudge. NB: I cannot speak for BE’s steelmen; instead, I’m responding for the way nudge and similar BE’s concepts are treated in casual, para-academic discourse, including governance and public policy. This is the “automaticity” worldview critiqued by Literal Banana and defended by Scott Alexander. In this view, the nudged individual is an irrational actor controlled by unconscious cues—a sheep in need of herding. This is why nudge theory, and behavioral economics more broadly, is so appealing to the Professional Managerial Class.

But note that metaphor of the herded sheep. The sheep “nudged” by the dog is neither mindless nor irrational; rather, it is trying to avoid being nipped or bitten by the heeler. The “manipulation” of the sheep happens by objectively changing the sheep’s “game state”—by moving a predatory animal (the canine) closer to one part of the herd or another, changing the objective incentive structure of the herded animal.

Similarly, I want to argue that successful nudges are only rarely ‘magical’ manipulations of irrational individuals. Rather, they are typically interventions in the objective structure of an individual’s incentives, honestly communicated to that individual. Indeed, they are not even properly classed as psychological effects, and there may be nothing in need of study. No one needs to study why, when the NBA changes traveling rules and communicates these new rules to players, said players change their dribbling and driving patterns.

I’ll give an example to illustrate what I mean. Let’s talk about the new Square-based/digital & tablet-based tipping system, because it’s one we’re all very familiar with. Moreover, because it’s a relatively new innovation (read: environmental disruption), we’re still adjusting to it, and IMO are in a pretty frustrating equilibrium. The combination of novelty + bad equilibrium leads to a lot of frustration and consciousness, which I think all of us are familiar with. (Well-established, working equilibria are often naturalized—they are unconscious and taken for granted.)

Scott Alexander, in his defense of the behavioral economics worldview (i.e. “automaticity”), writes about the strongest nudge finding yet:

The default option significantly changed the average tip customers gave. This isn’t a p = 0.04 effect in a lab, this is a real example with real money with 13 million data points. The authors then go on to replicate this in a different dataset of millions of taxi rides. Also, I met a psychologist who worked at Uber or Lyft (I can’t remember which), who confirmed that their company had replicated this research, and put lots of effort into deciding which default tip options to give customers because obviously it affected customer behavior.

If you set default tips to 18, 20, and 22% you get higher tips on average. Surprise! But I think some really basic phenomenological introspection (which Banana advocates for, and Scott dismisses) tells us why (1) this isn’t at all surprising (2) this isn’t a psychological effect, and isn’t evidence of the automaticity worldview. So why do people tip more?

1. It’s objectively a bit of a hassle to choose non-default tipping options. (It takes more thought and more time.)
2. There’s implicit social pressure not to—it feels rude to give less than the default social expectation. 15% is a polite tip in Mexico (where the standard is 10%) but rude in the United States if you have a large dinner party (where the expectation is 20%).

In other words, the default tip communicates a baseline of social expectation. How someone behaves relative to baseline social expectation determines whether they’re being “nice” or “mean,” “polite” or “rude.” The effect is particularly acute if you’re out buying a coffee or taking a taxi ride with a friend who might look over your shoulder and see whether you’re a generous tipper or a stingy little shit. In other words, the customer rationally behaves in a way as to communicate politeness—politeness being tethered to social expectation.

This is all pretty obvious stuff, but it actively undermines the automaticity worldview. Insofar as this is a “psychological effect” or manipulation, it is downstream of objective changes to the game, that is, to the space of payouts. What the phenomenological introspection conveys is that in fact, default-tippers are being intensely rational—they just understand the social game better than social scientists.

Strictly speaking, perhaps it’s fair to say that you can “manipulate” people into making certain choices if you change the game’s structure of payouts and then communicate those changed payouts to them. But this is less a psychological effect deserving extensive study, and more just a tautological extension of what incentives are.

And this is the point of Simler’s “Ads Don’t Work That Way” as well (cited in Banana’s OP). Changing the social meaning of options is equivalent to changing their payouts:

Whether you drink Corona or Heineken or Budweiser “says” something about you. But you aren’t in control of that message; it just sits there, out in the world, having been imprinted on the broader culture by an ad campaign. It’s then up to you to decide whether you want to align yourself with it. Do you want to be seen as a “chill” person? Then bring Corona to a party.

Anyway, I think this whole disagreement is ripe for a torque-style deconstruction, because it’s arguing over connotations and spirits: Are we irrational and biased (as Kahneman might have it), or are we boundedly rational and using heuristics that sometimes fail (as Gigerenzer might have it)? There’s a synthesis here that’s useful, and it’s probably Bourdieusean habitus: “Production [i.e. gameplay] is neither freely agentic nor structurally determined, rather, the field constitutes a space of possibles—the potential moves which might be understood by others as moves—from which the [player], according to his disposition and his assessment of the field, selects.” But I wanted to first lay out an antithesis, and argue it forcefully. Insofar as “nudge” effects belong to a field of study, they belong to game design and communication.

Collin offers a useful path forward:

The question is whether [a nudge] is something that survives deliberate anti-inductivity—that is, whether someone who has already decided to tip 10% for a conscious reason will [see their decision] overridden.

automaticity communication signaling ethology tipping ACiM anti-inductive Literal Banana Scott Alexander Sarah Constantin nudge behavioral economics disruption bias games

Monty Hall as investigative journalism

by Collin Lysford

In honor of one of the best jokes I’ve seen in a while, I’d like to offer a reframe of the Monty Hall problem that makes it less paradoxical. Especially because so many explainers out there just make it more confusing.

Monty Hall feels paradoxical because the host has information; we can show this by seeing what happens when we take that information away. Imagine you and your buddy stumbling blindly onto a naturally-occurring Monty Hall problem in the jungle. There are three doors, one with a car and two with a goat. You pick your favorite door. Then your buddy picks their favorite door, but they’re forced to pick a different one, because you’ve got your hand on yours already. Then they open their door. One third of the time, your buddy gets a car and you’re left with a weird choice: do you want to switch which door you’ve chosen, even though you’re definitely not getting a car, and you’re just picking between two goats you can’t see? ¹ But the other 2/3rds of the time, you either picked the right door or you didn’t, and switching makes no difference. You’re perfectly balanced between “Car if you stay, goat if you switch” and “Goat if you stay, car if you switch”. This is what most people’s intuition wants the normal Monty Hall problem to be like.

But the Monty Hall problem is different than my version because it’s not your buddy up there; it’s a game show host taking orders from on high. The boss doesn’t want the anti-climax of the host revealing a car and turning the game to Guess the Goat, so in cases where the host’s favorite door has a car behind it, the boss makes them change. This can never happen when your favorite door has the car (because you picked first), so it has the effect of taking Guess the Goat and turning it into “Goat if you stay, car if you switch”. And since “Car if you stay, goat if you switch” and “Goat if you stay, car if you switch” were perfectly balanced before, adding some more “Goat if you stay, car if you switch” into the mix makes switching a better deal.

The real lesson: if the boss is allowed to put their thumb on the scales, assuming the prize is randomly distributed makes you a chump who deserves every goat you get. Figure out where they aren’t letting you look, and assume that the treasure is a lot more likely to be hiding there than your first blind guess. Paradox, my ass! That’s just the first rule of investigative journalism.

Monty Hall games epistemology adversarial epistemology paradox