Examples of me doing forecasting
Here are some examples of questions I forecasted on last year, my rationale for the forecasts I made, and how I feel like the forecast turned out. All forecasts were made on 2021 Aug 5.
I think doing this kind of practice (writing your rationale out and then examining how it went) is good for learning forecasting skills.
-
Question: What is your likelihood that GIobal Priorities Institute will publish a new paper on https://globalprioritiesinstitute.org/papers/ between 2021 Aug 6 and 2021 Sep 6?
Rationale: Seems like GPI does publish 1-2 papers a month, suggesting an initial base rate of ~70% or so. I think there are more papers for September after the summer is over (for visiting fellows), which makes me raise this to ~80%. I do worry we may narrowly miss by being too early for visiting fellow papers but too late for the papers they just published.
Aftermath: This resolved “yes”. Things basically played out as I expected and I’m happy with my reasoning here.
-
Question: What is your likelihood that the government of Israel calls an election for Prime Minister, with the announcement occurring between 2021 Aug 6 and 2021 Sep 6? Measured per credible media reporting.
Rationale: The Israeli government has seemed pretty stable so far, seems to have a ~10% chance of falling apart this year and a ~2% chance of falling apart during this particular period. I imagine the government is particularly likely to be stable while still focused on defeating COVID.
Aftermath: My reasoning looks exactly right.
-
Question: What is your likelihood that LeVar Burton is announced as the next permanent host of Jeopardy, with the announcement occurring between 2021 Aug 6 and 2021 Sep 6?
Rationale: Recent reporting looks really bad for LeVar Burton as it looks like another person is in negotiations to become the permanent host. I think this drops to 5% that it is Burton and 2% it is announced during the forecasting period.
Aftermath: Indeed it was not Burton. I’m glad I read the news on this one.
-
What is your likelihood that Open Philanthropy publishes a new blog post on https://www.openphilanthropy.org/blog between 2021 Aug 6 and 2021 Sep 6? --- A repost of an old post will not count, nor will a link post to material already published by OP prior to 2021 Aug 6.
Rationale: So far this year there have been five posts in June, one in April, one in March, and one in January. The large amount of posts in June suggest that they’d be unlikely to publish something again, though I know OP is looking to publish their model on giving now vs. later soon, though unsure if this will happen within the forecasting period. 30% feels about right for this.
Aftermath: My reasoning looks exactly right. Maybe I should have thought even more about OP apparently having a preference for publishing some things on the EA Forum now?
-
Question: What is your likelihood that a new Variant of Interest appears and is officially assigned a new Greek letter between Aug 6 and Sep 6?
Rationale: Current variants of interest and variants of concern:
- Alpha and Beta - December 2020
- Gamma - January 2021
- Eta and Iota - March 2021
- Delta and Kappa - April 2021
- Lambda - June 2021
So far in 2021, variants have emerged 4 out of the 7 past months, suggesting a naive base rate of 63%. I think it could be higher as there is more pressure for variants to produce immune escape, but also lower as infections are decreasing, Delta is already so dominant, and we are no longer in an accelerated season for infection (winter in northern hemisphere). I think decreasing to 50% is more reasonable.
Aftermath: My reasoning feels right here. After I made the prediction I felt like my odds were too high, but then it happened anyway, so I guess things were about right.
-
Question: What is your likelihood that the United States has more total Olympic gold medals than any other country by the end of 2021 Sep 6?
I messed up this question too by making the forecast date way too close to the end of the Olympics. We already know that the US needs to gain a net of 6 gold medals over China in just one more day, which seems quite unlikely. I give them a 5% chance of doing that.
Aftermath: This prediction of mine was bad. This resolved positively. Apparently there were a lot more medal opportunities for the US than for China on the last day, making the last day not representative of the other days. I was way too certain that the last day would be like the previous three and there was no way my 5% odds was justified.
-
Question: What is your likelihood that the UK has more total Olympic gold medals than China by the end of 2021 Sep 6?
Rationale: The UK would have to net 18 more gold medals over China in a day, which is more than they currently have. I give them a <1% chance of doing that.
Aftermath: This impossibility did prove to be an impossibility, as I expected.
-
Question: What is your likelihood that, on 2021 Sep 6, Michael Aird still has more EA Forum karma than Peter Wildeford, according to Issa Rice’s leaderboard on https://eaforum.issarice.com/userlist?sort=karma ?
Rationale: Right now I am 416 karma short. I may make 1-2 posts in the forecasting period, but it is exceedingly unlikely they would combine to net me 416 karma, not to mention any additional karma Michael will generate. Since I have pretty much total control over this question and can steer things to not be competitive, I can comfortably give this a <1% chance of happening. It is possible that there could be a mass downvoting attempt to manipulate this question, but I’ll only give that a 3% chance of happening and succeeding, so Michael Aird should have a ~97% chance of keeping his spot.
Aftermath: My reasoning and tactics were right here.
-
Question: What is your likelihood that Mekelle, Ethiopia will remain under the control of the Tigray Defense Forces on 2021 Sep 6? Measured per credible media reporting.
Rationale: Tigray Defense Forces seem to have the upper hand on the war and Ethiopia does not seem to be contesting control of Mekelle. I think it is safe to assume this has an 80% chance of continuing.
Aftermath: My reasoning feels right here. I guess I could’ve been even more confident the war was essentially over [at least for this month].
-
Question: What is your likelihood that North Korea launches an intercontinental ballistic missile between 2021 Aug 6 and 2021 Sep 6? Measured per credible media reporting.
Rationale: North Korea hasn’t launched an ICBM since November 28, 2017. NK launched three total ICBMs in two different months in 2017 but have not launched any in any other year. Laplace’s rule suggests that since it has been 44 months since the last ICBM launch, a launch in the next month has a 1/46 chance of happening (~2%). A naive base rate that over the 49 months since the first launch in July 2017, there have been two months with launches, suggesting a 2/49 chance of happening (~4%). It was thought they might launch one early in the Biden adminstration to test their resolve, but this hasn’t happened yet - this could be one reason to make the risk higher than the base rate. However, NK also seems a bit too busy handling COVID and famine right now, which might pull away their resources from launching an ICBM, which makes the risk lower than the base rate. Overall, odds of 2% seem right.
Aftermath: My reasoning feels right here.
-
Question: What is your likelihood that, according to https://www.ochaopt.org/data/casualties, the sum of Palestinian and Israeli fatalities exceed 8000 by the end of 6 Sep?
Rationale: Crossing 8K fatalities would require 1786 more fatalities, a ~29% increase in one month. There have only been ~14 fatalities since the conflict came to an end at the end of May, suggesting that conflict would have to rise again to previous levels to create the necessary amount of fatalities. The last time an event caused the necessary amount of fatalities in the necessary amount of time was the 2014 Gaza War. Previous to that was the 2008 Gaza War. This suggests a base rate since 2000 of 3 events per 21 years, or 1/84, or ~1%. It’s possible right now that the tensions are a bit higher, but ~1% still sounds right.
Aftermath: My reasoning feels right here.
-
Question: What is your likelihood that Ethereum is over $2400 on 2021 6 Sep, per coinmarketcap at 11:59pm UTC?
Rationale: ETH is currently at $2800, having been over $2400 since 30 July. A naive random walk model suggests it has a ~70% chance of being over $2400 on 6 Sep. It is likely possible to improve upon this model using option prices, but I don’t feel like bothering to do that.
Aftermath: ETH went way up, but I’m not sure how I could’ve better predicted that, so my reasoning feels right here.
-
Question: What is your likelihood that, by the end of 2021 6 Sep, at least one country with population >=200,000 has covered more than 85% of its population with at least one vaccine dose, per https://ig.ft.com/coronavirus-vaccine-tracker/?
Rationale: So far, UAE, the country that has vaccinated the most percentage of population with at least one dose, has vaccinated 80.6% of their population. They got here from 75% in the past month, which makes us think that maybe they could add another 5 percentage points in the next month? But unfortunately they are slowing down, having only increased 2.2 percentage points over the past 20 days. If they keep that rate up over the 32 day period, we’d expect them to vaccinate an additional 3.52 percentage points, or 84.12% of the population, coming short by just a little bit. To get to 85%, they’d need to increase their current vaccination rate by 42%, which seems unlikely. We’d also expect that they are in a declining trend rather than a linear trend to get the last bit of their population vaccinated, so our actual expectation should be a bit lower. Overall, I give this a ~10% chance of happening.
Aftermath: This did happen and this forecast didn’t do too well. It looks like the UAE vaccination curve was more linear than I thought, contrary to my overall prior that it would be logistic where reaching the last 15% of the population is very difficult. I guess UAE is a more unusual country where this prior wouldn’t apply, which makes sense. I also overfit to the recent data and there was another inflection point back to more linear growth.
-
Question: What is your likelihood that, by the end of 6 Sep, Brazil has more cumulative reported COVID cases than the US, per https://ig.ft.com/coronavirus-chart/? --- A tie results in this question answering “no”.
Rationale: Brazil overtaking the US within a month would require something far more insane than Delta - Brazil would need a 50% increase in cases above any corresponding increase in US cases. I give this <1%.
Aftermath: My reasoning feels right here.
-
Question: What is your likelihood that, by the end of 2021 6 Sep, India has more cumulative reported COVID cases than the US, per https://ig.ft.com/coronavirus-chart/? --- A tie results in this question answering “no”.
Rationale: This would require a 10% increase in India cases above any corresponding increase in US cases, which seems unlikely given that India has already had their Delta wave and the US is still in their Delta wave. I give this ~5%.
Aftermath: My reasoning feels right here… maybe I was underconfident the US would have a bad delta wave.