Predictoor Dynamics Have Shifted Towards Accuracy
The “maximize accuracy” game is now outcompeting the “50/50 maximize stake” game
Introduction
In Ocean Predictoor, people run AI-powered prediction bots or trading bots on crypto price feeds to earn $. In Predictoor’s rewards formula, the more accurately you predict and the more you stake, the more you earn; you’re always in competition with others and their predictions.
From this rewards formula, we recognized that accuracy should be the key value proposition. It was a natural fit: staking caused accountability, and the game around it could nudge up accuracy over time.
Alas, “maximize accuracy” wasn’t happening. Rather, another game emerged that didn’t help Predictoor’s value proposition.
Observing the “50/50 High Stake” Game
People started running bots that used a “50/50” strategy, where at “prediction” the bot would stake exactly half its OCEAN up, and half down. This was popular, because it was nicely profitable and relatively safe.
To maximize $ earned, the 50/50 bot-runners simply maximized stake. Competition among 50/50 bot runners led staking volume to grow exponentially, to nearly $1B per month from $0 at launch six months earlier 😲.
These people had found that optimizing on stake (with no value-add to accuracy) was more profitable than optimizing on accuracy.
While the high volumes were exciting to see, they weren’t helping accuracy. And ultimately it was accuracy that mattered for Predictoor’s long-term success. What held back the model-based bots from competing? What could we do about it? Did it work?
Identifying the Problem
Was anyone running model-based prediction bots? We knew this answer to be “yes” because we were among them! Nothing like eating your own dog food:)
And, we found that it was hard to be profitable with model-based bots. Here’s why: it’s easier to be profitable predicting on both up & down, rather than just one side.
Let’s elaborate.
- The 50/50 bots staked on both sides, for each prediction. They were always there to catch the good and the bad. Having both sides is predictable, and helps profitability.
- In contrast, the model-based bots staked on just one side: up or down, but not both at once. Each time a prediction was wrong (almost but not quite 50% of the time), it was slashed its full stake amount. This meant high variance in winnings from epoch to epoch: big win or big loss. High variance hurts profitability.
We also recognized that higher prediction accuracy could help compete against the 50/50 bots. So we continued our diligent line of research to increase model accuracy.
If you can’t be profitable, then you can’t stake meaningful amounts to optimize profitability.
Addressing the Problem
We did two things to address the problem: two-sided prediction bots, and higher accuracy models.
Two-sided prediction bots. We introduced these a couple months ago. They made it easy to run bots that submit both up *and* down predictions based on model confidence.
Here’s an example. If the bot calculates up=30% chance, and therefore down=70%, and has 1000 OCEAN to stake, then the bot stakes 0.30*1000 = 300 OCEAN to up, and 700 to down. Critically, the bot is always on both the winning side and the losing side. This reduces its variance and increases its profitability. The image below illustrates.
Model accuracy research. We continued to improve our own internal models’ accuracy. We also learned that for models with 52% accuracy, two-sided bots could compete against 50/50 bots, but one-sided could not. For models of 56% accuracy, even one-sided bots could compete, and two-sided did not help.
What Happened Next? Accuracy Went Up
We rolled out two-sided model-based bots, and provided affordances for more accurate models.
What happened then?
New predictoors armed with two-sided bots and more accurate prediction models joined the game. These new model-based bots drove aggregate prediction accuracy up, nicely above 50%.
The image below shows accuracy for each of the ten 5-minute feeds from April 8, 2024 until May 1, 2024. Most notable is in the right 1/3 of the image where all accuracies trend upwards nicely.
The following image shows accuracy for each of the ten 1 hour feeds from April 8, 2024 until May 1, 2024. As with 5min, accuracies trend up nicely.
Then, What Happened? “50/50 Maximize-Stake” Game Faded
As accuracy increased, the 50/50 strategy became unprofitable, because the accurate model-based bots ate the stake of the 50/50 bots. Let’s elaborate:
- In an environment where every predictoor has a 50% accuracy, using a 50/50 strategy means a predictor will be right as often as wrong. Each correct prediction offsets the loss from a wrong one, while also earning a share of the rewards based on the stake amount.
- However, in an environment, where some predictors have accuracy sufficiently higher than 50%, they will win part of the losses incurred by the 50/50 strategy. Consequently, the gains from correct predictions in the 50/50 strategy do not fully cover the losses from incorrect ones, rendering it unprofitable.
With “50/50 maximize-stake” strategy, becoming unprofitable, the 50/50 bot runners left or sharply reduced their stake.
The image below illustrates. Below:top shows stake for each of the ten 5-minute feeds from April 8, 2024 until May 1, 2024. It dropped from 4000 OCEAN / feed / epoch to <1000 OCEAN. As of May 6 there’s about 700 OCEAN / feed.
Below:bottom shows a similar trend for 1 hour feeds. It also dropped similarly.
Discussion / Learnings
Accuracy is rewarded. At its heart, Predictoor is built on the principle of “reward accuracy”. The system rewards predictoors who make accurate predictions and penalizes those who don’t. We just had to reduce friction for making this happen.
Predictoors must adapt. From the above graphs, it can be seen that after 50/50 predictoors quit the game, the rest of the predictoors have lowered their stake amounts to maximize their revenue. Adapting to these changes and employing smarter strategies is a key part of the game. Understanding these dynamics is essential to maximizing your potential returns in the game. So, dive into it, tweak your strategies, and watch your profits improve!
Conclusion
In this article, we described how Predictoor rewards for accuracy and stake, and with a main value proposition of accuracy. We then described how the “50/50 maximize-stake” strategy was nicely profitable, alas, optimizing for stake not accuracy. We described steps we took to help make model-based predictions more profitable: two-sided predictions and more accurate models. Finally, we showed positive results of how accuracy has increased and the “50/50 maximize-stake” bots have been leaving.
The game is now firmly “maximize accuracy” and that’s a great thing.
About Ocean Protocol
Ocean was founded to level the playing field for AI and data. Ocean tools enable businesses and individuals to trade tokenized data assets seamlessly to manage data all along the AI model life-cycle. Ocean-powered apps include enterprise-grade data exchanges, data science competitions, and data DAOs. Follow Ocean on Twitter or TG, and chat in Discord.
In Ocean Predictoor, people run AI-powered prediction bots or trading bots on crypto price feeds to earn $. Predictoor has over $400+ million in monthly volume, just six months after launch with a roadmap to scale foundation models globally. Follow Predictoor on Twitter.
Data Farming is Ocean’s incentives program.
