The Long Game 92: Managing People, Surrogate Endpoints, Complaining, Heuristics

📦 Drone Delivery, Good Climate News, Writing, North Pole, TikTok & Teens, AskAnything, and Much More!

Feb 14, 2022

Hi there, it’s Mehdi Yacoubi, co-founder at Vital, and this is The Long Game Newsletter. To receive it in your inbox each week, subscribe here:

In this episode, we explore:

The problem with surrogate endpoints
Complaining makes things worse
Managing people
Heuristics that almost always work
Drone delivery
TikTok

Let’s dive in!

🥑 Health

🎯 The Problem of Surrogate Endpoints

I came across this excellent thread, and it encapsulates something that happens even outside of machine learning research. This is the problem of optimizing for a model rather than for reality.

Christoph Molnar @ChristophMolnar

A lot of machine learning research has detached itself from solving real problems, and created their own "benchmark-islands". How does this happen? And why are researchers not escaping this pattern? A thread 🧵

Image with the following text:

Establish a research topic. For example: Detect COVID-19 from x-ray images with machine learning.

First papers on this topic appear, and provide motivation for this research topic.

A precedent has been established: The initial papers pave the way for further research papers on the same topic.

The original motivation and assumptions for developing these predictive models are no longer questioned.

More researchers publish data and predictive models.

A community forms that cites and reviews each other.

Certain datasets become benchmarks and certain predictive models become state-of-the-art.

Predictive performance becomes the sole measure of progress, although improvements are becoming smaller.

Actual progress in solving the initial research questions has become irrelevant, even discouraged.

Decoupling from reality is complete.

In health research, it’s widespread. We learn how a mechanism works, and we get a good idea of which levers can be pulled to improve an endpoint, and we consider that the improvement of the endpoint naturally makes us “healthier.”

This phenomenon was described at length in the context of cancer research and therapies by Vinayak K. Prasad in Malignant: How Bad Policy and Bad Evidence Harm People with Cancer. He shows how big pharma manages to get drugs approved that improve surrogate endpoints (an indicator or sign used in place of another to tell if a treatment works) but, for the most part, aren’t really increasing the quality-adjusted life-year of the patients.

“In an analysis of 71 drugs consecutively approved between 2002 and 2014 for solid cancers, the median improvement in survival was just 2.1 months.”

The solutions he provides are numerous, but the general idea is that the hypotheses should be tested in randomized controlled trials that target the actual outcome we’re looking for (more years in good health) rather than the improvement of a surrogate endpoint.

This is a fundamental challenge because it determines how scientists and companies work. If the efforts are engaged in a process that’s not reliable, we’re wasting time and money while not improving health outcomes.

Pair with: Vinay Prasad, M.D., M.P.H: Hallmarks of successful cancer policy

Vinay Prasad, MD MPH 🎙️📷 @VPrasadMDMPH

I am reading these Booster docs in disbelief Are we really going to deploy a national booster campaign on the basis of an uncontrolled study showing that Ab titers (surrogate) are Non-inferior to what they were after dose 2 (historical control)? Insane fda.gov/media/152161/d…

Other noteworthy news 💊

Resveratrol — The Unfortunate & Scandalous Story

🌱 Wellness

🚫 Complaining Really Does Make Things Worse

I must admit that I was delighted to find this research paper explaining that complaining does make things worse. I try to stay away from complaining as much as possible, and now we have new research demonstrating why it’s important:

Affective events theory (AET) argues that everyday negative events are likely to lower both daily work engagement and momentary positive affect. These problems can then persist on subsequent days.
However, AET also argues that individual strategies can diminish the ill effects of negative events.
We explicitly focused on good sportsmanship or abstaining from unnecessary complaints and criticism as a possible moderator of the effects of daily negative work events on daily work engagement and positive affect.
We tested this possibility with a 3-day diary study among 112 employees. As expected, we found that daily negative events lowered daily engagement and momentary positive affect for two consecutive days.
However, this effect only held on days that people exhibited low sportsmanship. For days that people exhibited high sportsmanship, there were no significant effects.
Creating a resource rich work environment that enhances individuals’ sportsmanship behaviour can help to minimize the unfavourable impact of daily negative events.

Bottom line: stop complaining. It will make your life better!

Andrew D. Huberman, Ph.D. @hubermanlab

My deepest wish is that academics & scientists & clinicians that don’t like what they see in the world related to science & health communication, will stop complaining. If people do not understand something, that’s generally not their fault, it’s ours. Do better & make it free.

🏴 Heuristics That Are Almost Always Right

There’s no one better than Slate Star Codex to explain why relying on experts who follow heuristics can be so dangerous.

The Futurist
He comments on the latest breathless press releases from tech companies. This will change everything! say the press releases. “No it won’t”, he comments. This is the greatest invention ever to exist! say the press releases. “It’s a scam,” he says.
Whatever upheaval is predicted, he denies it. Soon we’ll all have flying cars! “Our cars will remain earthbound as always”. Soon we’ll all use cryptocurrency! “We’ll continue using dollars and Visa cards, just like before.” We’re collapsing into dictatorship! “No, we’ll be the same boring oligarchic pseudo-democracy we are now” A new utopian age of citizen governance will flourish. “You’re drunk, go back to bed.”
When all the Brier scores are calculated and all the Bayes points added up, he is the best futurist of all. Everyone else occasionally gets bamboozled by some scam or hype train, but he never does. His heuristic is truly superb.
But - say it with me - he could be profitably replaced with a rock. “NOTHING EVER CHANGES OR IS INTERESTING”, says the rock, in letters chiseled into its surface. Why hire a squishy drooling human being, when this beautiful glittering rock is right there?

Heuristics that almost always work are dangerous because people stop being sharp and stop paying attention. Most viruses don’t cause global pandemics, most new technologies don’t change the world, most physical pain isn’t serious, most UFO sightings are not legit…

These things are true, but the rare times they’re false, they cause a tremendous impact on the world, the person, etc.

Sometimes there’s a Heuristic That Almost Always Works, like “this technology won’t change everything” or “there won’t be a hurricane tomorrow”.
And sometimes the rare exceptions are so important to spot that we charge experts with the task. But the heuristics are so hard to beat that the experts themselves might be tempted to secretly rely on them, while publicly pretending to use more subtle forms of expertise. “My statistical model, accounting for chaos theory, barometric pressure, and the price of tea in China, says there won’t be a hurricane tomorrow. Rejoice!”
Maybe this is because the experts are stupid and lazy. Or maybe it’s social pressure: failure because you didn’t follow a well-known heuristic that even a rock can get right is more humiliating than failure because you didn’t predict a subtle phenomenon that nobody else predicted either. Or maybe it’s because false positives are more common (albeit less important) than false negatives, and so over any “reasonable” timescale the people who never give false positives look more accurate and get selected for.
This is bad for several reasons.
First, because it means everyone is wasting their time and money having experts at all.
But second, because it builds false confidence.

⚡️ Startup Stuff

👥 Managing People

This piece on managing people by Andreas Klinger is excellent and a must-read for all managers out there.

Here’s how it starts: Everything is your fault.

As a manager, everything is your fault
I know… very positive start 👀
There is no point being angry at your team – ever
You are in charge of processes and people
And you got more information than they do, always
You either created the processes where this outcome happened
or you hired (or did not fire) the wrong people
Ultimately everything is your fault

I also liked this part on trust, which echoed Tobi Lutke’s trust battery concept:

Trust is not binary
We tend to think of trust as binary
I either trust someone, or i don't
But this isn't true
You trust different people with different things differently over time.
Think of trust as something that you systemize
Eg what kind of trust do you give a new team member?
What are they expected to do in the first weeks? first month?

Here’s how Tobi puts it:

We talk about the trust battery as a metaphor quite casually. I know from people who are coming into the company that it may seem really strange. But it's something you just observe over time. Personally, I have found it really, really useful to be able to reason about a relationship without getting egos involved too much. I can have a conversation with someone saying, “Hey, you made a commitment to ship this thing, and you did. That's awesome. That's a super big charge on the trust battery, but you’re actually late for every meeting. Even though that's relatively minor—like it decreases 0.1% on your battery—you should fix that.”

Unrelated, but I found this tweet excellent and underlining a vital reality:

Fahd Ananta @fahdananta

Good early stage founders look unsuccessful for a long time Bad early stage founders look successful before doing the work

📚 What I Read

✍️ Putting Ideas Into Words

Your ideas are not clear until you can write about them.

Writing about something, even something you know well, usually shows you that you didn't know it as well as you thought. Putting ideas into words is a severe test. The first words you choose are usually wrong; you have to rewrite sentences over and over to get them exactly right. And your ideas won't just be imprecise, but incomplete too. Half the ideas that end up in an essay will be ones you thought of while you were writing it. Indeed, that's why I write them.

🌍 People Need to Hear the Good News About Climate Change

Less guilt and tribalism, more efforts where it really matters:

Calculating individual carbon footprints is overrated, and trying to guilt people about personal lifestyle choices is pretty pointless. If you’re worried about other people’s impact on the climate, you need to push for policy changes that make it easier for them to do what they want to do in an ecologically sustainable way or else that outright ban them from doing something destructive.
But as a matter of personal well-being, it’s critical to recognize that if you are, in fact, gripped with worry and guilt about climate change, you can do something about it.
If you’re young, you can pursue a career path where you’re working directly on zero-carbon electricity generation, on batteries, on green hydrogen, on new ways to manufacture ammonia, on carbon-neutral steel, or on direct air capture projects. There are probably a dozen other areas with an urgent need for technical and engineering progress and for the work of more good people as scientists and engineers, but also in management, marketing, sales, and everything else. There are tremendous opportunities to make a difference.

🏥 Teen Girls’ Sexy TikTok Videos Take a Mental-Health Toll

TikTok has very detrimental effects on kids. It’s no wonder that China bans it at specific hours of the day and tweaks their algorithm to promote learning and scientific content while doing the exact opposite for the version they export to the West.

When Jula Anderson joined TikTok at age 16, her first video featured her family’s home renovations. It got five likes. After seeing others post risqué videos and get more likes, she tried it, too.
“I wanted to get famous on TikTok, and I learned that if you post stuff showing your body, people will start liking it,” Jula, now an 18-year-old high-school senior near Sacramento, Calif., said.

@jason (too old, too tired, too talented) @Jason

TikTok should be banned in the USA because it’s Chinese psyops, Chinese spyware, & because our apps aren’t allowed in China. Also, it’s horrible for kids and at a minimum should be age gated at 18 — this @WSJ story explains how bad it is.

wsj.comTeen Girls’ Sexy TikTok Videos Take a Mental-Health TollGirls are often anxious and overwhelmed by the attention they get after posting suggestive videos; therapists say more are suffering emotionally.

🎙 Podcast Episodes of the Week

This week in podcasts:

An Evolutionary Psychologist's Dating Advice - Geoffrey Miller
- An excellent episode about evolutionary psychology and dating practices.
Garry Nolan: UFOs and Aliens 👽🛸
- Your monthly dose of UFOs with Garry Nolan. I particularly liked Garry’s focus on the data and his open mind. Unfortunately, far too many scientists ridicule this field of research.
- Pair with: Conquering the Barrier Belief

Mike Solana @micsolana

shout out to my armchair twitter intellectuals tweeting things along the lines of "pentagon just casually admitted UFOs are real and no one cares???"... welcome to the party. that actually happened a year ago, and serious reporting has been breaking on the topic since 2017.

🍭 Brain Food

📉 When There Were More Deaths Than Births in the U.S.

For the first time (ever?) in the US, deaths surpassed births from 12/2020 - 03/2021.

Births have declined over the years, whereas number of deaths have gone up with the population. So the two counts over time have been getting closer to each other. The past couple of years of covid accelerated the process.

Pair with: What the decline in population means for South Korea

The number of young men being conscripted into military service is expected to decline by almost 100,000 by 2025 as Korea's population declines, and potential university students will dwindle by 570,000.
The Ministry of Economy and Finance, which made the projection, on Thursday said it will come up with a response later this month.
"The military needs to overhaul its organizational structure, because the declining number of conscripts and shorter service period make it difficult to maintain troop levels."

🎥 What I’m Watching

📦 Drone Delivery Was Supposed to be the Future. What Went Wrong?

Sometimes, the “future” never materializes. Drone delivery is a perfect example.

Pair with: The counterpoint from Bryne Hobart, Drone Delivery: Accelerating to Inevitable

🥶 58 Days Alone in the North Pole

In 1978, Naomi Uemura attempted to reach the North Pole with a sled, 17 dogs, and determination.

🔧 The Tool of the Week

❓ Ask Anything — VC

Building a company is about learning different things as you go. It can quickly get tricky when it comes to legal/VC terms, and the small mistakes you make in the early days can have massive impacts later on.

AskAnything.VC is a great place to find resources to answer your questions.

I also highly recommend Secrets of Sand Hill Road: Venture Capital and How to Get It by Scott Kupor.

🪐 Quote I’m Pondering

The only way to discover the limits of the possible is to go beyond them into the impossible.

— Arthur C. Clarke

If you enjoyed this newsletter, make sure to subscribe if you haven’t 👇

👋 EndNote

Thanks for reading!

If you like The Long Game, please share it on social media or forward this email to someone who might enjoy it. Podcast reviews are also gratefully received. You can also “like” this newsletter by clicking the heart just below this, which helps me get visibility on Substack.

Feel free to email me or find me on Twitter if you have any feedback or questions.

Until next week,

Mehdi Yacoubi

PS: Lots of newsletters get stuck in Gmail’s Promotions tab. If you find it in there, please help train the algorithm by dragging it to Primary. It makes a big difference.

The Long Game by Mehdi Yacoubi