I noticed that the formulas for various leaderboard elements were available. I have two suggestions that I think will really improve their robustness.
=== 1: Low ratings should suppress leaderboard position, not raise it. ===
In the case of model generators, you have the term
(averageRating × ratingCount × 10)Say that someone has produced an awful model. If this model got 100 1-star ratings, then this term's influence on the leaderboards would be equivalent to a fairly good model: say, one that got 20 5-star ratings.
Since the medium score is 3, I think this term should be changed to:
((averageRating - 3) × ratingCount × 10)This way, 100 1-star and 20 5-star votes will now separate model rankings rather then put them together.
Alternately, to address the point Maxfield made about ratings, you could instead replace "3" with the median value of all voted on models in the category. This of course requires an additional calculation, aggregating all individual user ratings on all models in the category of interest (e.g. NSFW).
=== 2: Counts should be square-rooted. ===
This is inspired by signal-to-noise ratio calculations. In astrophysics, for example, counts of photons entering detector pixels N are uncertain by Poisson error: √N due to various influences. This means that the signal-to-noise ratio (SNR) of a measurement is N/√N = √N.
If my above formula is changed to:
((averageRating - 3) × √(ratingCount) × 10)then what this does is still award the highest boost influence to models with more ratings, but it also suppresses the runaway leader/Pareto effect. It'll be harder for the highest-ranked models/etc to hopelessly covet their positions from contenders due to, for example, lucky head starts.
Ideally, this square-root would be applied to all counting elements (e.g. download number, reaction count) done on all leaderboard values.
Please authenticate to join the conversation.
Awaiting Dev Review
💡 Feature Request
Over 2 years ago
AstroTibs
Get notified by email when there are changes.
Awaiting Dev Review
💡 Feature Request
Over 2 years ago
AstroTibs
Get notified by email when there are changes.