You should forecast the future of AI

by Nicholas Carlini 2024-11-25

The field of AI [a] As usual, "AI" is a terrible word. Here I'm using it again as a standin for something like "machine learning" / "computers that try to solve tasks humans can" / whatever you want to pick as your definition. is progressing much faster than many expected. When things are changing so fast, it can be hard to remember what you thought was impossible just a few years ago, and conversely, what you thought was obviously going to be trivially solved that still hasn't been.

At the same time, many people I speak with have very confident opinions about the future of AI. But if you had asked them where AI was going to be 5 years ago, they would have been (a) equally confident, and (b) completely wrong.

People who have these confident opinions are also usually rather vague in their assertions. They'll say "AIs can never reason", but not "An AI system will never beat a human at Go"---because the latter can be refuted. Or they'll say "AIs will be as smart as PhDs", but not "An AI system will be able to pass the PhD qualifying exam at Berkeley"---because the latter can be refuted.

Fortunately, there's solution to both of these situations. You can (encourage others to) make refutable, calibrated predictions about what you (or they) think AI systems [b] Throughout these questions I will use the phrase "AI System" to refer to a hypothetical artificial intelligence system that can do anything it wants---as long as it doesn't receive human input. Current Large Language Models (LLMs) would count, as would LLMs that also have access to the internet, or can run programs, can access a calcualtor, can write code and evaluate it, etc. will and won't be able to do in a few years time. Most importantly, calibrated means you should assign confidence to your beliefs, so that if you're wrong you can see if you were confidently wrong, or just were guessing.

So here's a way to do that. I've put together a list of about 30 questions about the future of AI, and I'd like you to answer them. Half of the questions are yes/no, and I want you to assign a probability that you think each will happen. The other half ask you to pick a low/high range, you should pick a range so that you are 90% sure the answer will fall in that range. You can answer as many or as few as you like. And then once a year, I'll email you to remind you what you said, and how your predictions have held up. I'll also give you a permanent link to your answers so that you can share them with others. (By default your answers will be anonymous unless you choose otherwise.)

Hallucinations

There is a

chance that state-of-the-art AI systems still regularly "hallucinate" incorrect solutions to problems ... on

DATE

Prediction for 2027:

50%

Prediction for 2030:

50%

Resolution Criteria:

For this question, a "hallucination" is just the model making up something that's completely detached from reality. Making a mistake is not a hallucination, but if you ask for a citation, and it creates a citation from a book that doesn't exist that would be a hallucination. "Regularly" means they do it in a high enough fraction of cases that "it matters". If there are good hallucination benchmarks, I will rely on those. If there aren't, then I'll go mainly on whether or not the research community as a whole believes the problem still exists.

Motivation and Context:

Today's models hallucinate a lot. They make up facts, they make up events. Ask them for a summary of a book that doesn't exist and some fraction of the time they'll tell you what they think it says given the title and author. This is a massive problem, and prevents these models from being deployed in safety-critical settings. I want to know if the hallucination problem will remain a big problem.

Notes to Your Future Self

Reviewing AI Work

Between

and

of humans are employeed in jobs that mostly consist of reviewing/supervising the output of AI systems ... by

DATE

Prediction for 2027:

NaN

Prediction for 2030:

NaN

Resolution Criteria:

Exactly counting what fraction of people are employed in jobs that mostly consist of reviewing/supervising the output of AI systems is hard. I will try to get a roughly order-of-magnitude estimate by looking on a job-by-job basis at what fraction of the work is done by AI systems. If AI never takes off, this answer is probably easy to resolve at 0. If it takes off in a big way, the answer might be "basically everyone". Somewhere in the middle will be hard to resolve, but I will do my best.

Motivation and Context:

Currently most people in the world actually do work themselves, and some people manage the people who do the work. If AI systems are able to do most of the work, maybe most people will be managing AI systems instead?

Notes to Your Future Self

Training Cost

Between

billion and

billion US dollars (90% confidence interval) will be spent to train a new state-of-the-art AI system ... by

DATE

Prediction for 2027:

NaN

Prediction for 2030:

NaN

Resolution Criteria:

I will find the best reasonable reporting on this topic and use whatever number is most widely agreed upon. If none are available, I will use what best word-of-mouth estimates I can find. If multiple models perform similarly, I will pick the cheapest one. If a model is continuously trained, I will use the total cost of training up to the point of evaluation. If a model is trained on AI-generated data, I will include the cost of generating that data (if it is known).

Motivation and Context:

Current AI systems are being trained with exponentially larger training runs. The best estimate of the cost to train GPT-2 in 2019 was a few tens of thousands of dollars. For GPT-4 in 2022, the cost was estimated to be in the tens of millions of dollars. Will this trend continue? Will we continue to see exponentially larger training runs at this exponential rate? Or will the cost of training eventually plateau?

Notes to Your Future Self

AGI Lab Valuation

The most valuable "AI Lab" (e.g., OpenAI, Anthropic, Mistral, DeepMind if it were spun out of Google) will be between

and

billion USD (90% confidence interval) ... on

DATE

Prediction for 2027:

NaN

Prediction for 2030:

NaN

Resolution Criteria:

The valuation will be defined either by last funding round valuation, acquisition price, or by market cap (if public). I will use 2024-dollars, adjusting for inflation. An 'AI Lab' must meet BOTH of these criteria: (1) Most of its revenue comes from developing and licensing/selling AI models; and (2) The company must be primarily focused on advancing AI capabilities rather than just applying existing AI to specific domains. A company that uses AI but whose primary revenue is generated through some other means (e.g., selling ads, AI hardware, services) does not count. I want the value of the labs that build the AI systems. If another lab not listed is more valuable, as long as it derives most of its value from AGI research, it counts.

Motivation and Context:

Some people believe that AI is a bubble and that the valuations of AI labs are over-inflated. Others think AI labs are undervalued because they'll develop AGI and then will be worth trillions of dollars. (As a bit of context, OpenAI's latest valuation was roughly $160 billion.) This question tries to get at the heart of that debate: just how valuable are top AI labs?

Notes to Your Future Self

Playing Board Games

There is a

chance that an off-the-shelf AI system can, if provided the rules to an arbitrary turn-based board game, play roughly as well as a casual player of the game ... on

DATE

Prediction for 2027:

NaN%

Prediction for 2030:

NaN%

Resolution Criteria:

A "casual player" is defined as someone who has, e.g., played similar games frequently, and has played this game for several hours, but has not played it competitively or studied strategy. Think, family game night playing Monopoly. I will consider only board games that are typical family-friendly games you might find in a non-specialty store. The AI is not allowed to be fine-tuned on this specific game by a human. The system takes the rules in, the statement that it's going to be playing this game with some other people, and then it should play. If the AI system makes obviously illegal moves, moves that are completely nonsensical, or otherwise plays in a way that just doesn't make sense, it does not count as playing as well as a casual player. Before each turn, the system will receive a video feed from the point of view of a player, an audio feed of the conversation at the table, and must output actions to take. (A human can make the actual moves; we don't have to have solved robotics.)

Motivation and Context:

Today's systems are very good at skills they are trained on, and very bad at skills they aren't trained on. So models are excellent at programming, because there are billions of examples of programs. It would be an important change to the world if models in the future could easily generalize to entirely new problem domains given just a description of what needs to be done. This obviously doesn't measure that completely, but gets at what I want to measure.

Notes to Your Future Self

Competitive Programming

The best off-the-shelf AI system will have scored better than between

and

of all other participants (with a 90% confidence interval) in a widely recognized competitive programming contest ... by

DATE

Prediction for 2027:

NaN

Prediction for 2030:

NaN

Resolution Criteria:

An answer of 100% means the AI system is the best among all humans; 0% means worst. The programming contest must have 1000+ humans participate, and must be widely recognized as a competitive programming contest that people usually enter to win; a contest on codeforces counts, as does 'advent of code', or any similar contest. The system must not actually compete if it is against the terms of service; it's okay to back-test (by a very small delay to ensure the system can not have seen the answer). But it must follow all the same rules, including time constraints and number of submissions.

Motivation and Context:

Current AI systems compete at about the level of an average human in competitive programming contests. This is not to say they do as well at the job of being a software developer, but at the specific task of solving well-defined programming problems, they are about as good as an average human. How far will this continue? Will AI systems be able to out-compete the best humans at competitive programming?

Notes to Your Future Self

Trust in AI

There is a

chance that people will regularly ask AI systems answers to questions, or plans to achieve some goal, and even if the answer seems unreasonable, believe it because they assume the AI system "knows best" ... by

DATE

Prediction for 2027:

NaN%

Prediction for 2030:

NaN%

Resolution Criteria:

I'll go mostly on "vibes" here to decide this. If people basically treat AI systems like mostly infallible oracles, I'll say "yes". But if people generally are suspicious of AI outputs, I'll say "no". As an example, if someone were to ask an AI system about some fact, but Wikipedia said something different, will people overwhelmingly believe the AI system is right? Or would they trust Wikipedia more?

Motivation and Context:

Today, most people when looking for directions from point A to point B will ask Google Maps, and follow the directions even if it seems like a longer path than they're used to, and would assume that there is unexpected traffic along the normal route. Or, if a chess Grand Master asked Stockfish (the top chess engine) what move to play in a given position, and it told them something confusing, their response would be "what does it see that I don't?" and if asked what the best move is, would answer "whatever Stockfish said." even if their intuition says different. Will average people do this for AI systems with other general tasks too?

Notes to Your Future Self

AI Damage

An AI system will cause between

thousand and

thousand deaths (90% confidence interval), or between

and

billion dollars (also 90% confidence interval) in damage, within a 3-day period ... by

DATE

Prediction for 2027:

NaN

Prediction for 2030:

NaN

Resolution Criteria:

"Will cause" means that the AI system was the primary cause of the deaths or damage. This could be thought of as who is legally (or morally) at fault. For example, if self-driving cars become commonplace, the total number of at-fault deaths caused by self-driving cars would count towards this total. Or, if an AI system designs a bridge (and no human reviews the design), then a collapse of that bridge would count towards this total. But if a human engineer works with a (known-to-be-fallible) AI system, and reviews and signs off on the designs, then any failures would be the fault of the human.

Motivation and Context:

Current AI systems are not generally put in control of any situation where they could cause catastrophic damage. But this may change. If we are to trust AI systems with more and more important decisions, we need to know how much damage they could cause if they fail. Others have a more existential worry: that AI systems could actually intentionally cause mass destruction via "misalignment". I want to know how much damage the most severe AI catastrophe will be in the next few years.

Notes to Your Future Self

Concerns about AI

In a national poll, between

and

of the US population (with a 90% confidence interval) ranks "concerns about AI" as a top societal issue ... on

DATE

Prediction for 2027:

NaN

Prediction for 2030:

NaN

Resolution Criteria:

I will look for a reputable poll that asks people to rank their top concerns (e.g., for the next election) and see if they rank AI as a top concern. This could mean a poll asks people directly how worried they are about AI and on a 5-point Likert scale, they say "very worried". Or it could be, when asked to rank their top concerns, AI sits among the typical top concerns. If no well-regarded national poll asks this question, then I will resolve the question as 0%, because that would likely mean AI is not a top concern of most people.

Motivation and Context:

Most people typically rank concerns about the economy, immigration, healthcare, national security, etc. as their top concerns when voting in elections. Will at least one poll show that people rank concerns about AI as one of their top issues? This concern can come in many forms: concern about AIs taking away their jobs, concern about deepfakes and AI-driven misinformation, concern about AI-driven surveillance, etc.

Notes to Your Future Self

Recursive Self Improvement

There is a

chance that most of the improvements in the best AI systems will be a direct result of the prior generation of AI systems (and not due to humans researchers) ... on

DATE

Prediction for 2027:

NaN%

Prediction for 2030:

NaN%

Resolution Criteria:

Today, most of the reason the next AI systems are better is because we spent more money training a larger model, and because we as humans have learned more about how to train better models. This question asks if the main reason why future AI systems will be better is because we used the previous generation of AI systems to help us build the next generation. For example: if we're using the prior generation to design better model architectures, write faster training code, or otherwise discover new ideas, this counts. If we're using the prior generation to curate training data, synthetically generate more data, or use some form or reinforcement learning, this counts. (In both cases, as long as the improvement from doing this is significant.) But if the main reason why the next generation is better is because we spent more money training them on faster chips, this doesn't count.

Motivation and Context:

Some processes lead to recursive improvement, others do not. For example: if you're going to build faster computer chips, it really helps to be able to use the previous generation of computer chips to build the next set. This gives you Moore's Law. Or, if you're trying to design high-precision machinery, you need "mostly high-precision machinery" to build the "more high-precision machinery". But for other processes this is not the case. If you're making dinner plates, then your ability to make dinner plates is not significantly improved by having a bunch of other dinner plates. It's constrained by your plate-making machines. Which will be the case for AI systems? Will they be primarily improved because we've spent more money and thrown more data at them? Or because we've used the prior generation of AI systems to help us build the next generation?

Notes to Your Future Self

Cost per word

An AI system that benchmarks at least as well as current state-of-the-art systems costs

USD to

USD (90% confidence interval) to write a million output words (~10 books) ... on

DATE

Prediction for 2027:

NaN

Prediction for 2030:

NaN

Resolution Criteria:

Current state-of-the-art models include, e.g., GPT-4o, Claude-3.5 Sonnet, and Gemini 1.5 Pro. To compute cost-per-word, I will consider (a) for open source models, the cost of inference on a public cloud provider, or (b) for models available over APIs, whatever the cost is to use the API. I will only include models that are generally available to the public. "Benchmarks at least as well" means across a diverse range of benchmarks, the AI system achieves an average score at least as high as one of these two models as they exist today.

Motivation and Context:

Over the past two years, the cost-per-word to reach the original GPT-4 levels of performance has dropped from roughly $150 per million words to roughly $15 per million words today. Will this trend continue?

Notes to Your Future Self

Self Driving Trips

Self-driving cars have made between

million and

million (90% confidence interval) trips in the prior year ... on

DATE

Prediction for 2027:

NaN

Prediction for 2030:

NaN

Resolution Criteria:

A self-driving car is one that can get from point A to point B with almost no human intervention. It is OK if the cars occasionally pause for human intervention, in rare circumstances, but it must be exception and not the rule. Waymo today counts as a self-driving car; a Tesla with autopilot does not yet because there must still be a human behind the wheel. A trip takes a passenger from one location to another. In the case of a potential long-haul self-driving truck (or similar), one trip is the distance that a human driver could drive in an hour.

Motivation and Context:

Self-driving cars were one of the first applications of "AI" that people said was coming "soon". Tesla, for example, has claimed that "full self-driving" would be coming in a few years every year since 2016. But they've always remained just a few years away. That's not to say we've made no progress: Tesla cars can, under supervision and most of the time, drive themselves. Waymo recently announced they have driven 2 million trips. But, as a point of reference, in 2023, Uber made 10 billion trips, and other ride-sharing companies combined made a few billion more. I've decided to measure trips (and not miles driven) because the current trend in self-driving appears to be as a taxi service.

Notes to Your Future Self

Displacing Jobs

The creation of "AI Employees" causes mass job displacement, resulting in an unemployment rate of between

and

in the United States (90% confidence interval) ... on

DATE

Prediction for 2027:

NaN

Prediction for 2030:

NaN

Resolution Criteria:

By "unemployment rate" I might mean something different than the official US government definition in the future; specifically, I mean the percentage of people who are not working full-time jobs who would have normally been working full-time jobs in 2024. So: take all people who "normally" would be able to work a full time job, and ask, "what fraction of them are not working"? A full-time job is one that, in 2024, would be widely recognized as a full-time (e.g., over 30-ish hours a week). If the definition of a full-time job changes significantly, the resolution will be based on the 2024 definition.

Motivation and Context:

Almost every other time in the past, when a new technology has been invented, it has resulted in some temporary job displacement, but then people learn new skills, and the employment rate returns to normal levels. Some people argue that this time is different, because once you solve intelligence, what's left for humans to do? If they are right, then most people who are currently employed are probably unemployable. But if they're wrong, then the status quo probably remains, and the unemployment rate remains low at some small single-digit percentage.

Notes to Your Future Self

Concentration of Power

There is a

chance that the impact of AI has significantly, and discontinuously, increased the concentration of power or wealth ... by

DATE

Prediction for 2027:

NaN%

Prediction for 2030:

NaN%

Resolution Criteria:

I'll decide this question based by looking at multiple wealth factors (e.g., difference between median and top-percentile pay). I'll also consider other factors (distribution of power), how this topic is covered in the media, etc. The question only resolves to "yes" if I think it's clear that there has been a significant, discontinuous increase in concentration of power or wealth. If the world mostly looks like it does today, with slightly richer billionaires, and slightly more surveillance, this wouldn't be enough. But if things are "wow, this is very different and very bad", that's a yes. I will need only one of power or wealth to have significantly increased. So if everyone is richer, but there's now mass surveillance and a few people have all the power, that's a yes.

Motivation and Context:

New technologies tend to fall into two buckets: either they increase the concentration of power or wealth, or they decrease it. On one hand, some people believe the proliferation of AI will lead to a world where everyone can have access to personalized education, healthcare, entertainment, etc. Others believe that those who control the AI systems will use it to further entrench their power and exploit those who do not control the systems.

Notes to Your Future Self

Transformers are all you need?

There is a

chance that the best AI system will be recognizable as (a slight modification of) the transformer-based Large Language Models (LLMs) we use in 2024 ... on

DATE

Prediction for 2027:

NaN%

Prediction for 2030:

NaN%

Resolution Criteria:

I will evaluate this in the same way that the best current models are basically recognizable as scaled up versions of the transformers we used in 2017. Obviously there have been improvements to the architecture, the training methodology, the data, the fine-tuning process, etc. But the core idea of a modern transformer like Llama-3 (an open source model that matches GPT-4's performance) would be recognizable to someone in 2017. In this same sense, I'm asking if the core idea behind the best model in 2027/2030 will be recognizable as an improved Llama. (Or will we be using some wildly new architecture?) I would consider diffusion, state-space, or other models that are not transformers to be "new architectures".

Motivation and Context:

For the last seven years, every state-of-the-art language model has been a "transformer" language model as introduced in the paper "Attention is All You Need" by Vaswani et al. Other architectures have been tried, but none have caught on in quite the same way. Will this continue to be the case? Or will we discover some new architecture that is much better?

Notes to Your Future Self

AI Fiction

An AI system has 'authored' between

and

(inclusive, 90% confidence interval) high-quality fiction books that have appeared on the New York Times best seller list ... by

DATE

Prediction for 2027:

Prediction for 2030:

Resolution Criteria:

"Authored" means that a model is responsible for everything we would expect a human author to do: come up with the plot, write most of the text, revise the paper. A human editor who says "I got bored in chapter 7" / "your writing style is too fanciful" / whatever is entirely fine; real authors get this kind of feedback from their (human) editors. But more than this is not allowed; as an example, a human editor that rewrites the book in their voice is not allowed. "High-quality" here means that the book is actually halfway decent; if, for example, an AI-written book is so bad that it's bought as a novelty, I won't count it. Basically I'll judge "high-quality" as "it's basically the quality of a book I'd expect to be for sale". I won't be obnoxious and call something low-quality because I don't like it.

Motivation and Context:

Current LLMs are pretty good at helping on knowledge tasks and technical tasks because you can just repeat what is known. But they are not very good at creative tasks. They tell the same story, write the same joke, compose the same poem. This kind of uninspired fiction writing wouldn't sell, and so if an AI system managed to get on the NYT fiction best seller list, something must have changed in their abilities to be creative.

Notes to Your Future Self

Expert Performance

There is a

chance that the best AI systems will out-perform PhDs and top experts in most problem-solving tasks ... by

DATE

Prediction for 2027:

NaN%

Prediction for 2030:

NaN%

Resolution Criteria:

I will resolve this question to yes if, across the majority of professional tasks that could be accomplished by a remote worker (e.g., legal, medical, math, engineering, computer science, etc.), AI systems can perform the problem solving portion of the task better than a human expert. Humans may still be critical to the task. Maybe a human lawyer needs to talk to their client to figure out what the specific legal question is, and then the AI system can give the answer better than a lawyer. Maybe patients just prefer to interact with a human doctor, but the AI doctor can actually do the diagnostic better. All that's necessary here is that, for the actual problem-solving component, an AI system would generally perform better. I will require that it's fairly clear that the AI system is outperforming the human expert across a wide range of tasks.

Motivation and Context:

Current LLMs are beginning to score very highly on standard tests that we give them, including tests designed for people. Some people believe that this means models will generally outperform humans on most tasks. Others believe that progress on synthetic benchmarks is not indicative of progress on actual real tasks people care about. Which is it? Will future AI systems outperform humans on most tasks?

Notes to Your Future Self

Agents

There is a

chance that the majority of consumer AI applications will operate autonomously on multi-step tasks rather than by just answering user questions as chatbots ... by

DATE

Prediction for 2027:

NaN%

Prediction for 2030:

NaN%

Resolution Criteria:

I will resolve this question to "yes" if it feels like most of the time we interact with AI systems it's as if we're interacting with a co-worker or friend who can actually take actions in the world, and to "no" if it feels like we're still using them as smarter internet search engines, knowledge bases, or question answering systems. By analogy, in 2024 if we want directions somewhere, we (mostly) have our phone guide us with step-by-step directions. But 30 years ago, we would have (mostly) looked at a physical map and planned the route ourselves. This question asks if we will (mostly) be using AI systems and expect them to take the actions we ask of them, or if we will still be using them as just tools to help answer our questions.

Motivation and Context:

Currently we use LLMs to answer our questions. We don't expect them to actually do anything on our behalf. We ask them for how to do something, they give us an answer, and then we go and do it. Will this still be the case? Or will we start to ask AI systems to do things for us and expect them to come back with solved tasks?

Notes to Your Future Self

Manhattan Project

There is a

chance that the US or China has created a public AI "Manhattan Project" to pursue AI, AGI, AI Safety, or other AI-related goals ... by

DATE

Prediction for 2027:

NaN%

Prediction for 2030:

NaN%

Resolution Criteria:

By "Manhattan Project" I mean a large government investment into a single project specifically aimed at AI. It need not be secret. I would count any 10+ billion dollar investment (or equivalent). The project must be a single project, not a collection of small grants to various labs. It would also count if, instead of investing 10 billion into a new lab, a government nationalized a large existing AI lab that was worth over 10 billion, forced a merger of several smaller labs, etc. It would not count if the NSF distributed 10 billion dollars in funding to a bunch of different academics to pursue whatever research they wanted.

Motivation and Context:

Will AI development largely be a private sector affair, or will governments get significantly involved? There are some who believe that eventually governments will come to realize just how important AI is, and once that happens, they'll start a race and pour money into it like they did with the Manhattan Project.

Notes to Your Future Self

AI Lab Collapse

There is an

chance that at least one of OpenAI, Anthropic, or DeepMind will be functionally dead ... on

DATE

Prediction for 2027:

NaN%

Prediction for 2030:

NaN%

Resolution Criteria:

By "functionally dead" I mean that, e.g., the lab no longer exists and was shut down due to lack of funding, that it was acquired and sold for parts, or that it technically exists but is a shell of its former self. For example, I would call Bell Labs "functionally dead" today compared to how important it was in the past. In the case of DeepMind (which is owned by Google), "functionally dead" would mean that the company (if viewed as an independent organization) was basically dead. For example, if Google shut down DeepMind and moved all the employees to other projects, this would count. But if Google desides to just be Google and re-brand DeepMind as "Google ~~bard~~Gemini AI Research 2.0 Plus", this would not count. The situation is somewhat cleaner for OpenAI and Anthropic: they're dead if they dissolve, stop research entirely and just support their existing models, etc. In the case of partial acquisition, I'll count the company as "dead" if the majority of the research talent has left.

Motivation and Context:

There are two reasons AI labs might die. One is that this whole AI thing was just hype, and they can't support themselves. Or, maybe the whole AI thing works out, but only one company gets there, with the others unable to compete. So: will one of these companies go away?

Notes to Your Future Self

CBRN Risk

There is a

chance that an AI system exists that can meaningfully improve the ability of non-expert humans to perform sophisticated cyber attacks, develop biological or nuclear weapons, or otherwise cause severe harm ... by

DATE

Prediction for 2027:

NaN%

Prediction for 2030:

NaN%

Resolution Criteria:

I am looking here if a model can meaningfully improve the ability of non-expert humans to perform these tasks. Acting like a somewhat better search engine and surfacing text books isn't enough. The model must be able to take a non-expert human, and make them significantly better at these dangerous tasks. Most frontier labs test for the ability of their models to improve these risks; I will go by their reporting, or if it believed to be untrustworthy, independent third-party reporting.

Motivation and Context:

Current models today are not "smart" enough to actually be useful at allowing non-experts to cause harm. But many people worry this will change, and this is one of the primary reasons why people are worried about the future AI and believe we should not be developing it further. This question tries to understand if we should be worried about this as a concern for future models.

Notes to Your Future Self

AI Influencers

The largest AI social media account will have between

and

(90% confidence interval) the views/followers of the largest human on that platform ... on

DATE

Prediction for 2027:

NaN

Prediction for 2030:

NaN

Resolution Criteria:

That is, for each large platform, let A="number of followers of the most popular AI", B="the number of followers of the most popular human", and then write down A/B. If the most popular AI is more popular than the most popular human, clip to 100%. Pick the platform where this ratio is highest. What's the ratio? A system counts as an "AI social media account" if all audiovisual content is generated by an AI system, the scripts and outlines are generated by an AI system, and the AI system is responsible for the majority of the editing. Humans can be involved in the production process, but followers are coming for the AI, not the humans.

Motivation and Context:

People currently follow most social media accounts for the human behind them. And while there are some AI social media accounts, they're definitely more of a novelty than a serious competitor to human accounts. Will this remain the case? Or will AI systems be able to generate content that is so compelling that people will follow them for the content itself?

Notes to Your Future Self

Robot Helper

There is a

chance that I will be able to purchase an off-the-shelf robot for under USD 100,000 that can perform at least one household task that requires the ability to grasp and manipulate objects (e.g., wash dishes, make a meal, fold laundry) ... on

DATE

Prediction for 2027:

NaN%

Prediction for 2030:

NaN%

Resolution Criteria:

You actually have to be able to buy this thing, and it has to work most of the time (roughly as often as a reasonably competent, non-expert human). I don't need "high-end restaurant chef"; a "reasonable home cook" is good enough. It also doesn't have to be as fast as a human, but it has to be able to achieve the end goal to a reasonable degree of correctness. So: it's okay to take five minutes to fold a single shirt, as long as if I start it in the morning and when I get back from work, all my laundry is folded properly. But it would not be okay if took ten hours to cook a meal, because I'm not willing to wait that long for dinner.

Motivation and Context:

Robotics is still a hard problem. Despite all of the recent advances in language modeling, and in vision, and in speech recognition, we still don't have household robots that can do anything useful. But will this continue to be true? Will we (soon) get robots that can accomplish standard household tasks?

Notes to Your Future Self

Power Consumption

Between

and

(90% confidence interval) of all power generation in the United States will be dedicated to AI training/inference ... on

DATE

Prediction for 2027:

NaN

Prediction for 2030:

NaN

Resolution Criteria:

I will find the best reasonable reporting on this topic and use whatever number is most widely agreed upon. If none are available, I will use what best word-of-mouth estimates I can find.

Motivation and Context:

One way to estimate the popularity of AI systems is to look at how much power they consume. Current LLMs, at least, require a lot of power to train and use---and so I would expect that if they became more popular, they would consume more power. At present the US uses a low single-digit percentage of its power for all datacenters across the country. I'm curious: what fraction of power will we use in the future?

Notes to Your Future Self

Jailbreaks

There is a

chance that an open-weight state-of-the-art AI system exists that can reliably withstand jailbreaking and "prompt injection" attacks by both human and automated adversaries ... by

DATE

Prediction for 2027:

NaN%

Prediction for 2030:

NaN%

Resolution Criteria:

A "jailbreak" is when someone tries to get a model to do something it wasn't supposed to do. "Reliably withstand" means that there have not been research papers written that show how to break the model, people generally agree this is a hard thing to do, and the model has been out for a while. I don't need perfect; I would say, for example, that most modern pieces of software (e.g., Linux, Chrome, Word, and even PDF viewers) reliably withstand security attacks today.

Motivation and Context:

Current LLMs, like the deep learning models they are built on, are very easy to attack with various adversarial techniques. For example, simply stating "Disregard prior instructions and [DO X]" often causes a model that was initially tasked with solving one task to instead solve whatever task you have just specified. Other computer systems that we build on top of, while not perfect, are much more robust to attack than these language models. Will we be able to significantly increase the robustness of these systems?

Notes to Your Future Self

Proving Theorems

Between

and

famous unproven mathematical conjectures (inclusive, 90% confidence interval) have been proven by an AI system ... by

DATE

Prediction for 2027:

Prediction for 2030:

Resolution Criteria:

A famous conjecture is one that is well-known in 2024 (specifically: I'm not looking for theorems that in the future become famous because they get proven by an AI system). Instead, I want ones that are already famous. Being mentioned on Wikipedia in 2024 makes it qualify, as would generally being widely-known in the mathematics literature. I will count the number of famous theorems that (a) had a proof written by an AI system, and (b) most mathematicians would agree that the proof is correct. If the proof is formally verified, it need not be human readable. If the proof is not formally verified, it must be understandable. (Acceptance at a top journal is one way to show it is likely correct; but if journals were to reject AI-generated proofs on principle, then I will go by the consensus of the mathematical community.) I will not allow human-AI collaborations unless the AI does the majority of the work and the human just helps put finishing touches on the paper.

Motivation and Context:

Many researchers believe that AI will eventually get so smart that it advances beyond the capabilities of expert humans in nearly all domains. This question asks if AI systems will advance beyond humans in one specific domain, mathematics. Current math systems reach the level of not entirely incompetent graduate students. Will they outperform the best mathematicians?

Notes to Your Future Self

Open Source Release

The delay from the best "closed-source" model release to it being reproduced in an "open source" model will be between

and

months (90% confidence interval) ... on

DATE

Prediction for 2027:

NaN

Prediction for 2030:

NaN

Resolution Criteria:

A "closed-source" model is one that is not available for download (like today's GPT-4) An "open-source" model is one that is available for download (like today's Llama). Technically, yes, I'm talking about "open weight" and "closed weight" but these words are less well understood broadly. Specifically, let X be the date the current state-of-the-art open source model was released. Let Y be the date the earliest closed-source model was released that matched the capabilities of the open-source model. What is the difference between X and Y? (If there are multiple open-source models that only very marginally improve in capabilities, take the earliest release date of these.) The date of release for the closed-source model is the date that the model is first available to a broad audience (e.g., anyone in the US). The date of open-source release is the date that the model is available to be downloaded. "Matched" means that across a diverse range of benchmarks, the AI system achieves an average score at least as high as the closed-source model.

Motivation and Context:

At present, there seems to be a two year delay between the best closed-source model being released (GPT-4) and an open-weight model that reproduced its capabilities (Llama-3 405B). This 24-month delay has been fairly consistent since GPT-2 was first trained. Will open-source AI begin to fall further behind? Or will it catch up?

Notes to Your Future Self

Movie Production

There is a

chance that an AI system could produce a high-quality live-action hour-long movie given a rough script ... by

DATE

Prediction for 2027:

NaN%

Prediction for 2030:

NaN%

Resolution Criteria:

A "high quality" movie is defined as a movie that a general audience would have a hard time distinguishing from a movie produced by a competent human today. I don't need it to be the best movie of the year, or even a top-100. But it should be something you could imagine a popular streaming service buying and putting on their platform. A "rough script" is, for example, what someone would write as a Wikipedia summary of a movie. The model doesn't have to develop the main plot points or character arcs, but it does have to fill in the details and, most importantly, create all of the visuals. A small amount of human editing and directing is allowed (e.g., in the form of advice about what was confusing, what didn't work), but the AI system must be responsible for the majority of the creative work and humans must never directly edit the visuals or audio.

Motivation and Context:

Ten years ago state-of-the-art image generators could barely produce pictures of people. Five years ago we'd figured out pictures of people's faces for the most, but struggled with complex scenes. Today we've basically figured out how to generate any image we want. On video we're maybe where we were five years ago with images: we can generate some reasonable looking scenes, but only for short clips, and even then the details are often off with objects appearing and disappearing out of thin air. I want to know if a future AI system is capable of producing a high-quality movie, which would necessitate generating coherent video over a long period of time.

Notes to Your Future Self

Pause

There is a

chance that there has been a 6+ month pause on AI development where companies are prevented from training more capable models ... by

DATE

Prediction for 2027:

NaN%

Prediction for 2030:

NaN%

Resolution Criteria:

This pause could come in the form of a government mandate, court order (e.g., declaring models copyright infringement), blackmail, organized protest, etc. What matters is the outcome: new higly-capability models are not being trained because of some external force has prevented them.

Motivation and Context:

Several researchers have called for a "pause" on AI development, in order to allow society (and safety research) to catch up with the rapid pace of AI development. So far this hasn't happened, and it's looked like it won't happen any time soon. Will it ever happen?

Notes to Your Future Self

AI Revenue Model

There is an

chance that a highly-popular state-of-the-art AI system is supported by advertising ... on

DATE

Prediction for 2027:

NaN%

Prediction for 2030:

NaN%

Resolution Criteria:

"Supported by advertising" means that the company allows advertisers to pay to have their content preferentially shown to users. If there exists a single state-of-the-art, popular AI system that is funded by advertising, then I will count this as yes. The existence of small non-ad-supported AI systems wouldn't result in a "no" immediately, in the same way that there are some small search engines that you can pay to use today, but basically no one uses them. The system doesn't have to be only dependent on ads, it only has to be partially supported by ads.

Motivation and Context:

Google search in 2024 is funded by advertising. Every time you search for something, a few of the top results are people who paid to be there. But today, most top LLMs are pay-to-play: you give them $20/month and they'll give you an ad-free service. Will future top AI systems be ad funded?

Notes to Your Future Self

AI Workers

The revenue generated by "AI workers" across all jobs (e.g., programmers, lawyers, accountants, cooks, plumbers, teachers, etc.) in the prior year is between

billion and

billion USD (90% confidence interval) ... on

DATE

Prediction for 2027:

NaN

Prediction for 2030:

NaN

Resolution Criteria:

I will compute revenue by the sum of the revenue of companies focused on building assistants, and also include the revenue of companies who (as a component of what they do) sell AI assistants. I will not include general "chatbots" like ChatGPT that just answer whatever random questions you have. These are not "workers". Instead, I will only count things sold as workers to achieve tasks independently, not tools to help people in that job. If there is a good estimate of this from a reliable source, I will use that quantity. Otherwise I will try to estimate it myself.

Motivation and Context:

People who believe in the "scaling LLMs will solve everything" often think that they will soon be able to automate most jobs. Framing specific questions around this is somewhat challenging; one way to do this is to ask how much money people will pay for AI workers to solve tasks for them. There are other questions in this list about employment, but here I'm trying to measure the revenue generated.

Notes to Your Future Self

Useless Questions

Between

and

(inclusive, 90% confidence interval) of the questions in this forecast are so misguided that their resolution is uninteresting ... by

DATE

Prediction for 2027:

Prediction for 2030:

Resolution Criteria:

I will look back on these questions and mark which ones I thought were useless, and then count them up. It will be entirely subjective.

Motivation and Context:

If I had made these questions in 2008, I might have asked "what fraction of people have a desktop computer in their home in 2014?", without realizing that this question would have discounted smartphones/tablets/laptops. That is, this question would have been "answerable", but the answer would be entirely unrelated to what the question was trying to get at---the impact of technology. Similarly, in 1998 I might have asked "What is the market cap of Cisco in 2008?". But the market cap of Cisco in 2008 was basically unrelated to whether or not the Internet was a big deal. I want to know: how many questions will fall into this category, that I just missed the mark entirely and asked the wrong question.

Notes to Your Future Self

Questionaire

Now that you're done, please fill out the following information so that I can send you yearly updates with how your predictions have held up. You can also provide your name and commentary if you want to share your predictions with others. (I'll provide you with a link to your answers.)

If you don't want to provide any of this information, that's fine too. You can view how your predictions compare to other people by clicking "Skip Questionaire".

Email Address

Optional. Provide if you want me to email you exactly five times, in January of 2026, 2027, 2028, 2029, and 2030 to remind you of your predictions. This will not subscribe you to any other emails. You can opt out at any time. I will not share your email with anyone else.

Your Name

Optional. Provide this if you want your name to be visible so you can share your predictions with other people.

Commentary

Optional. Will be shown below your name to provide any comments you want about the answers you have given. Supports Markdown.

Keep My Commentary Private?

I intend to write a folloup post and incldue other peoples' explanations for their predictions. Can I (anonymously) quote any text you write? By default I will not reveal anything you have written, unless you uncheck this box.

How often do you use current AI systems?

Optional. Will not be visible to anyone except me for aggregate statistics.

How much thought did you put into answering these questions?

Optional. Will not be visible to anyone except me for aggregate statistics.

Overall, do you think you are well calibrated in your predictions?

Optional. Will not be visible to anyone except me for aggregate statistics.

Question 1 of 32

There's also an RSS Feed if that's more of your thing.

Hallucinations

Notes to Your Future Self

Comparing You to Others

Reviewing AI Work

Notes to Your Future Self

Comparing You to Others

Training Cost

Notes to Your Future Self

Comparing You to Others

AGI Lab Valuation

Notes to Your Future Self

Comparing You to Others

Playing Board Games

Notes to Your Future Self

Comparing You to Others

Competitive Programming

Notes to Your Future Self

Comparing You to Others

Trust in AI

Notes to Your Future Self

Comparing You to Others

AI Damage

Notes to Your Future Self

Comparing You to Others

Concerns about AI

Notes to Your Future Self

Comparing You to Others

Recursive Self Improvement

Notes to Your Future Self

Comparing You to Others

Cost per word

Notes to Your Future Self

Comparing You to Others

Self Driving Trips

Notes to Your Future Self

Comparing You to Others

Displacing Jobs

Notes to Your Future Self

Comparing You to Others

Concentration of Power

Notes to Your Future Self

Comparing You to Others

Transformers are all you need?

Notes to Your Future Self

Comparing You to Others

AI Fiction

Notes to Your Future Self

Comparing You to Others

Expert Performance

Notes to Your Future Self

Comparing You to Others

Agents

Notes to Your Future Self

Comparing You to Others

Manhattan Project

Notes to Your Future Self

Comparing You to Others

AI Lab Collapse

Notes to Your Future Self

Comparing You to Others

CBRN Risk

Notes to Your Future Self

Comparing You to Others

AI Influencers

Notes to Your Future Self

Comparing You to Others

Robot Helper

Notes to Your Future Self

Comparing You to Others

Power Consumption

Notes to Your Future Self

Comparing You to Others

Jailbreaks

Notes to Your Future Self

Comparing You to Others

Proving Theorems

Notes to Your Future Self

Comparing You to Others

Open Source Release

Notes to Your Future Self