Monthly Archives: December 2010

2009 BPR Correlation Analysis

For more on the BPR correlation analysis, see the 2010 BPR Correlation Analysis post.

BPR POWER RATING / RACE FINISH POSITION CORRELATION TABLE

(click to enlarge)

BPR RANKING / RACE FINISH POSITION CORRELATION TABLE

(click to enlarge)

COMBINED CORRELATION LINE PLOT

(click to enlarge)

BPR RATING / BPR RATING CORRELATION TABLE

(click to enlarge)

posted by Trey Blincoe

Advertisements

Leave a comment

Filed under 2009 Season, BPR

Comment – Codemaster’s F1 2010

For the Formula 1 geek, the off-season can be a truly depressing time; it’s cold outside, and there’s not an F1 car on-track for months.  One way to pass the time is to jump into an F1 car yourself, but for those without a grand prix car in the garage, or access to one of the team’s impressive simulators, the opportunity to do so has been fairly limited without a true F1 video game on retailers’ shelves in recent years.  While the private ‘modding’ community has done a very respectable job of producing bits and pieces of the F1 world for games like rFactor, a full-on, turn-key F1 experience has been lacking for all video game platforms for some time.

That all changed when Codemasters took the F1 license over from Sony, and after almost 2 years of development, “F1 2010” hit the shelves early this fall for PC, Xbox 360, and Playstation 3.  F1 2010 was released to wide acclaim, with the game receiving an aggregate rating of 84 (out of 100) on metacritic; and more importantly for Codemasters, F1 2010 reached sales figures previously thought unreachable for an F1-based game.

Unfortunately, video game reviews are generally conducted by dedicated game-reviewers who play a game for a relatively short period of time before writing up their corresponding review.  Furthermore, in the case of F1 2010, the vast majority of reviewers are not well-versed on the world of F1 and unlikely to care about the things that a true follower of the sport would.  In an attempt to reverse that trend, I decided to write a review from the standpoint of a true F1 geek, and wait until I’d thoroughly played F1 2010 before writing my review.

THE GOOD…

Make no mistake based on what’s to come later in this review review: F1 2010 is impressive considering this is Codemaster’s first attempt at simulating the world of grand prix racing.  The game’s environment provides a complete impression of the F1 world that I haven’t felt since playing MicroProse’s “Grand Prix 2” some 15 years ago.  The graphics are absolutely superb, with levels of detail in track and car modeling that are typically only seen in big-league titles like the Call of Duty franchise.  While the tracks aren’t modeled with the assistance of laser-scanning and are therefore less than accurate at times, for the most part, the entirety of each circuit is reproduced in excruciatingly-brilliant detail.  F1 2010’s visual effects are just as impressive, with animated crowds and objects, heat waves and exhaust overrun, and a beautiful, although ultimately flawed dynamic track and weather system (more on that later).  More important than just the visual representation of the F1 world is the fact that more so than any other racing game I’ve played, the scaling of F1 2010’s environment just feels realistic.  In summary, the level of visual immersion in F1 2010 is truly unparalleled in the world of racing games.

F1 2010 also gets my general praise for the game’s physics.  That statement comes with a pragmatic caveat that in order to move product, Codemasters couldn’t create an all-out simulation in light of the general public’s taste for arcade racing.  From the moment I started up F1 2010 for the first time, I’ve used a Logitech Formula Force GP wheel and pedal set  with the game set to the ‘hardest’ settings without the help of any of the available driving aids.  With that in mind, a direct comparison of driving an F1 car in the ultra-realistic world of “iRacing” isn’t all that different than the experience in F1 2010.  In fact, the only appreciable difference between the two platforms is iRacing’s level of physics modeling allowing it to more completely replicate true on-the-limit driving to an extent that F1 2010 can’t.  Up until the limits of tire grip and corresponding wheel slippage, both games feel remarkably the same.  While iRacing’s physics are more complete and will reward the expert driver more than F1 2010, the process of finding the limit in an F1 car is substantially the same process in both games, which is a truly impressive feat for F1 2010.

In addition to the visuals and feeling of driving an F1 car, F1 2010 also gets high marks in the sound department.  It wasn’t until Simbin’s revolutionary sportscar racing game “GTR” that a developer truly attempted to replicate the audio effects of motor racing.  The systematic lack of attention to sound in racing games has always baffled me; when you ask a racing fan what he or she enjoys about going to the track, near the top of everyone’s list is the sound of motor racing.  F1 2010 does an excellent job of representing the sounds of F1, both inside and out of the car.  The player in F1 2010 is treated to very representative engine notes, curb vibration noises, downshifting effects, and much more.  While upcoming editions of Codemaster’s F1 series can stand to include more effects, the game sounds right at this early stage, and that gets a big nod of appreciation from me.

Finally, F1 2010 gets high marks for its “Live the Life” feature of making the game about more than just going from menu screen to track.  F1 2010 attempts to model what its like to be an F1 driver, from career planning and contract negotiation, to dealing with the team and press, to the development of rivalries with other drivers.  Sure, the implementation of these features isn’t always perfect and becomes repetitious at times, I certainly appreciate any attempt to broaden the fairly static approach developers have taken to racing games.

THE BAD…

While F1 2010 does a lot of stuff right, it also gets a lot of stuff wrong.  The flaws in the F1 2010 game labeled as being part of “the bad” aren’t deal-breakers, and are merely areas where Codemasters can improve on for F1 2011 and beyond.

As even the most casual fan of F1 knows, the world of Formula 1 racing is extremely complicated.  A large part of Formula 1’s sophistication is owed to the rules of the sport, which are constantly changing and are nothing if not complex.  Codemasters gets a lot of credit for attempting to implement many of those rules into F1 2010, which features engine limits, tire allocations, flag and penalty rules, etc.  However, it is in the implementation of those rules that there is a lot of room for improvement.  For instance, F1 2010’s tire allocation system governing the usage of tires over a race weekend is error-prone, with tires randomly becoming available or unavailable to the player without regard for the actual rules.  This makes developing a tire management strategy quite difficult to say the least.

On the subject of strategy, F1 2010 also gets a certain amount of kudos for attempting to replicate the team experience of having a race engineer guide the player in areas such as race management, strategy, and even car setup.  However, F1 2010 doesn’t go far enough and often fails at implementation of the limited goals it set out for this feature.  As an example, the race engineer is completely oblivious to differences in fuel loads or track conditions when comparing your times to your teammate’s, and will often suggest copying your teammate’s setup to chase the difference.  Another problem with that suggestion?  Copying your teammate’s setup isn’t an option in 2010.

F1 2010 also keeps the player from fully controlling the car at all times, including most-importantly, driving the car in the pit-lane.  It’s been a long time since I’ve played a game that doesn’t allow for player control of the car in the pits, and while lack of full control doesn’t completely detract from the experience, a game of F1 2010’s caliber should obviously include that functionality.

There are a number of other fixes, additions, etc. that Codemasters could include in upcoming F1 titles.  Just a few things I’d like to see addressed: pre-season testing is completely left out of F1 2010, and could provide for a number of interesting game-play additions; not to mention invaluable acclimatization time for a player getting set to tackle a full World Championship season.  The lack of pit-lane action in light of the high level of detail in the game is also noticeable; if the crowd is animated, why isn’t there a single team member on the pit-wall?  And where is the safety car, reconnaissance laps, and other fundamental aspects of F1 racing that make the sport what it is?

THE UGLY…

Head on over to Codemaster’s F1 2010 forum and you’ll be privy to an angry mob of protesting players, the likes of which are usually seen in old black and white horror films. While many of the calls for the heads of those in charge at Codemasters aren’t deserved, there are more than a few issues in F1 2010 which warrant the ire the developers have received in the past several months.

To say F1 2010 was bug-ridden at the time of release is an understatement.  Yes, all games have minor glitches that slip through the beta-testing process; but F1 2010 was released with major, game-breaking bugs that should never be an issue for a game of F1 2010’s caliber.  Saved games would randomly become corrupted under certain circumstances, erasing a player’s considerable time and effort without any hope of recovery.  As just another example of the kinds of bugs F1 2010 was released with: after completing several 100% distance grand prix, I experienced no less than 6 punctures without going off-track even once.  Tire allocations were completely incorrect, driver aids would randomly be turned on by the game despite player selections, and the list goes on…

Codemasters did release a single patch aimed at addressing some of the aforementioned problems, and true to the initial release of F1 2010, the patch created just as many problems as it fixed.  Inexcusably, the patch completely broke the PC version’s dynamic track and weather system.  Goodbye racing lines, wet tracks, marbles off-line, etc.  Codemasters eventually identified a “fix” for the problem, which cut frame rates by more than half; rendering the most capable PC impotent on even modest settings.  It’s hard to imagine how a publisher like Codemasters can let such massive and obvious oversights to continually crop up.  It’s embarrassing.

More important than the aforementioned fiasco is the fact that F1 2010’s artificial intelligence system is an unmitigated disaster.  The AI drivers in F1 2010 perform well enough on-track, although driver behavior is nothing spectacular in terms of what we’ve seen before in racing games.  Computer drivers in F1 2010 do a better job than most of reacting to the player’s movements; from defending the racing line, to moving over when being lapped.  However, the absolute deal-breaker with F1 2010’s AI is such a major oversight on the part of the developer, that its palpably frustrating to even write about it several months after the game’s initial release.  The distinguishing characteristic of the 2010 Formula 1 season is that in-race refueling has been banned, and that cars start the race with several hundred pounds of fuel on board which eventually burns off lap-by-lap over a race distance.  Fuel weight changes in conjunction with tire wear have substantial effects on car behavior and lap times throughout a race, which the player can certainly feel to good effect in F1 2010.  What’s the problem then?  The AI drivers are seemingly unaffected by the fuel or tire simulation experienced by the human player.  Whereas the player will run laps some 4 to 5 seconds slower at the beginning of the race as compared to the end, the AI will set lap times within 1-2 seconds of their qualifying pace throughout the entirety of the race.

In addition to that monstrous oversight, the rest of the AI system is faked by artificially-generated lap times in qualifying, track-position ghosting, and a rubber-band system to control relative performance during a race.  In qualifying, an AI car’s lap-times aren’t represented by the car crossing the timing stripe; no matter how long it actually takes the car to get around the track, the computer will fake the lap time to correspond to pre-determined probable grid positions.  Furthermore, AI track positions aren’t controlled by in-game physics, but rather by computer-generated guidance which can lead to cars ghosting from one point to another when certain circumstances don’t fit the computer’s predictions.  In a race, the rubber-band system will also determine driver performance based on position and not the driver/entry’s true performance.  To illustrate, if Mark Webber gets into a first-lap squabble and drops to 20th place, he will consistently run laps on the level of the surrounding HRT, Lotus, or Virgin drivers.  Never before have I seen such fundamentally flawed design in an AI system.  Not even close.

…THE SUMMARY

In the end, F1 2010 is a potentially amazing game that is held back from greatness by a number of substantial design flaws.  The game looks, sounds, and feels like a modern Formula 1 game should.  Codemasters also gets a lot of credit for trying to replicate the ultra-complicated world of F1 racing while adding gameplay-enhancing features previously unseen in reality-based racing games.  That said, F1 2010 is Codemasters’ first offering in an ongoing F1 franchise and their inexperience shows in numerous aspects of the game.  One has to remember though, that while this is Codemasters’ first F1 game, the company has been around since 1986 and has produced a number of racing titles covering a wide variety of motorsports.  The persistent level of design, implementation, and testing flaws contained in F1 2010 is absolutely inexcusable for a title as big as F1 2010 is; especially for a publisher of Codemaster’s ilk and experience.

In many ways we can be both disappointed and hopeful that there is so much room for improvement for next-year’s F1 2011.  If Codemasters can right the state of affairs established by F1 2010, then this F1 fan won’t be dreading the next off-season quite as much as usual…

posted by Trey Blincoe

Leave a comment

Filed under Comment

2009 BPR Charts – Round 17: Abu Dhabi Grand Prix

In addition to a number of planned non-BPR items, over the upcoming off-season BPR F1 will be offering a ‘way-back-machine’ of the BPR by archiving past Formula 1 seasons.

As a supplement to the regularly posted BPR tables, bprf1.com will chart the progress of the BPR POWER rating throughout the season. For further explanation of the tables, see this post.

DRIVERS – FULL RANGE (50-100)

(click to enlarge)

DRIVERS – POSSIBLE POINT SCORERS (75-100)

(click to enlarge)

DRIVERS – POSSIBLE PODIUM FINISHERS (85-100)

(click to enlarge)

DRIVERS – PROBABLE RACE WINNERS (90-100)

(click to enlarge)

DRIVERS – RANKINGS

(click to enlarge)

TEAMS – FULL RANGE (0-100)

(click to enlarge)

TEAMS – RANKINGS

(click to enlarge)

posted by Trey Blincoe

Leave a comment

Filed under 2009 Season, BPR

2009 BPR Season Summary

For more on the season summary table, see the 2010 summary table post.

(click to enlarge)

posted by Trey Blincoe

Leave a comment

Filed under 2009 Season, BPR

2009 BPR Update – Round 17: Abu Dhabi Grand Prix

In addition to a number of planned non-BPR items, over the upcoming off-season BPR F1 will be offering a ‘way-back-machine’ of the BPR by archiving past Formula 1 seasons.

2009 FIA Formula 1 World Championship – BPR Through Results of Rd. 17: Abu Dhabi Grand Prix

(click to enlarge)

Don’t understand what’s going on here? The Blincoe Performance Rating (“BPR”) is a statistical motorsport performance rating system that monitors driver/entry performance during each Formula 1 season. For more information: see this explanation.

posted by Trey Blincoe

Leave a comment

Filed under 2009 Season, BPR

2010 BPR Trend Plots

As a final wrap-up of the 2010 Formula 1 season, I compiled several line plots with polynomial trend lines of the BPR POWER rating to help show the general progression of the past season’s results.  I chose a 4-order polynomial trend line for the plots to essentially break down the season trends into quarters.  The line plots focus first on the 5-way title fight between Lewis Hamilton, Jenson Button, Sebastian Vettel, Mark Webber, and Fernando Alonso, with an additional plot displaying the championship points progression for the same group of drivers.  The focus then shifts to team trends, with the first plot displaying the season’s 3 elite teams, the mid-field, and the 3 new teams.

DRIVERS CHAMPIONSHIP CONTENDERS TREND LINES

(click to enlarge)

DRIVERS CHAMPIONSHIP CONTENDERS POINTS

(click to enlarge)

ELITE TEAMS

(click to enlarge)

MID-FIELD TEAMS

(click to enlarge)

THE NEW TEAMS

(click to enlarge)

by Trey Blincoe

Leave a comment

Filed under 2010 Season, BPR

2010 BPR Correlation Analysis

As part of BPR F1’s ongoing season wrap-up, I thought that reviewing the correlation between the BPR POWER rating and the actual on-track results during the 2010 Formula 1 season would be a worthwhile exercise.

For the less mathematically-inclined, correlation is a study of the relationship between two sets of variables (a good, brief explanation of correlation can be found here).  In evaluating the POWER rating’s relationship to on-track results, I used one of the most common methods of displaying correlation: a correlation coefficient.  A correlation coefficient ranges from -1.0 to +1.0. The closer the coefficient is to +1 or -1, the more closely the two sets of variables are related.  Once again, the foregoing link provides a good explanation of what a correlation coefficient represents.

My main purpose in running a correlation analysis was to evaluate how well the BPR POWER rating predicted the next round’s results throughout the 2010 Formula 1 season.  It wouldn’t make sense to run a correlation between the individual-round BPR ratings and the on-track results from that same event, as the BPR formula is based solely on on-track performance and inherently correlated.  Therefore, I ran the “CORREL” function in Microsoft Excel for the POWER rating preceding each round against the next round’s results.  To illustrate: the post-Bahrain POWER rating was correlated to the results of Australia, the post-Australia POWER rating was correlated to the results of Malaysia, and so on.  For those that are interested, Excel’s CORREL function uses the covariance of the variable sets and the standard deviations of each set to arrive at its correlation coefficient output.

I evaluated the POWER rating’s correlation to the finishing position of all 24 driver/entries at subsequent rounds in two ways: by utilizing the actual POWER rating as well as the rankings based on the POWER rating which are included in each BPR table posted on BPR F1.  The results of each correlation analysis are provided in the tables below, with the corresponding results being utilized to generate a line plot also displayed below.  The POWER rating’s correlations were inversed from negative to positive to allow for continuity in graphing.

BPR POWER RATING / RACE FINISH POSITION CORRELATION TABLE

(click to enlarge)

BPR RANKING / RACE FINISH POSITION CORRELATION TABLE

(click to enlarge)

COMBINED CORRELATION LINE PLOT

(click to enlarge)

(Note that because there was no pre-round POWER rating for the season-opening Bahrain Grand Prix, that round does not have a corresponding correlation coefficient.  Asterisks next to round abbreviations indicates a year-to-year change in track configuration, and the letters in parentheses next to some round abbreviations indicates the track used for events which have alternated tracks in recent history.  Coefficient boxes shaded in light-blue indicate a race that was run in wet conditions.)

With nearly identical average coefficients of 0.779 and 0.778, the POWER rating and its associated rankings are certainly correlated to finishing positions at consecutive races.  Take the correlations one step further and the POWER rating is indeed a “predictor” of future results.  However, the POWER rating was never designed to be a prediction model and its predicting abilities are solely related to its reflection of current performance trends prior to each consecutive race.

With the foregoing in mind, I intend to adapt the existing BPR formula to create a predictor model for the 2011 Formula 1 season.  At this stage, the model will incorporate the POWER rating as well as a separate formula based on track-type performances which positively correlate to other rounds.  Including the second calculation is intended to address the simple fact that there are completely different tracks on the Formula 1 calendar which tend to produce different results based on individual car and driver characteristics.  Although it’s relatively obvious that the results of the Monaco Grand Prix are unlikely to reflect the results of say, the Italian Grand Prix at Monza, result relationships amongst other tracks aren’t as obvious.  Therefore, I decided to run a correlation of all the BPR scores for each round on the 2010 Formula 1 calendar to find which events’ results correlate.  Remember that the BPR incorporates a lot of data for all 24 participating drivers at each round and therefore the correlation results should be fairly dependable.

BPR RATING / BPR RATING CORRELATION TABLE

(click to enlarge)

(Note the light-blue shading used once again to show races which ran in wet conditions, with darker blue shading indicating a strong correlation between two races which were run in the wet.)

I’ll let the reader digest some of the more interesting correlations in the above table and what they entail, but I highlighted all correlation coefficients of 0.900+ as being especially relevant.

Look for the upcoming BPR predictor at BPR F1 in addition to the regularly-posted BPR materials throughout the 2011 Formula 1 season.

posted by Trey Blincoe

6 Comments

Filed under 2010 Season, BPR