Fallout 4 PC Performance Analysis

[section label=1. Introduction]

Fallout 4; Bethesda’s highly anticipated latest installment of its  post-apocalyptic, open-world franchise, has finally been released. The Fallout series has long been known to be one of the most welcoming to the PC mod development scene, and unfortunately that is because these games are often launched with bugs, performance issues, and other annoyances that are sometimes only fixed by the community. While Bethesda’s games have always been hugely popular on PC, it is no secret that they can have issues on some configurations at the time of their launch. Anyone who’s played Skyrim with an AMD processor probably knows exactly what I’m talking about.

So, today we’ll be conducting the first of a series of performance analysis, taking a look at the game’s graphics, settings, and performance. This will not be a review of the game’s overall quality, gameplay or story. We’ll be judging it solely on its technical merits. It is worth noting that while we we’ll be following this report up with a number of articles that will be testing various hardware configurations, today’s tests will be focusing on a fixed set of hardware. These tests are simply to get a baseline of the game’s performance on modern hardware, and to figure out just how taxing it is on the CPU. So, if today’s report doesn’t answer all of your questions, be sure to check back soon. 
[section label=2. Graphics Options]

Fallout 4 PC Graphics Options

Any PC game worth its salt will have a number of customizable graphical features, and settings which allow the user to fine-tune the performance and visual fidelity of the game. Fallout 4 does have a number of these, although it does not allow the user to change them while in-game. Instead, you’ll be forced to make any graphical changes you want in the launcher, prior to starting the game. This is unfortunate as it means you’ll have to restart the game in order to make simple changes such as resolution, or  anti-aliasing. It is admittedly a small issue, but still one that could have been easily avoided.

Another thing worth nothing is that this game features a hard V-Sync cap, which means the maximum frame-rate will be limited to your monitors refresh rate. Now, that doesn’t sound like a huge deal as any frame-rate above your monitors refresh rate will not be visible. However, this creates a lot of other issues as it means that dropping too far below your refresh rate will result in huge spikes. For example, if your monitor is 60Hz, and you’re getting an average of 50 FPS, you’ll actually be dropped to as low as 30, as it will need to buffer the same frame twice, in order to display one frame per refresh cycle.

So you might ask, why would they do this? Well, in our experience with the game so far, Fallout 4 seems to have the player movement speed, along with a number of other animations, tied to the game’s frame rate. This is a rather archaic way of doing things, as it does not consider users with higher refresh rate monitors such as 120, and 144Hz, to say the least of variable refresh rate solutions such as NVIDIA’s G-Sync, and AMD’s FreeSync technologies. You can of course disable V-Sync with some tweaking, but it is not natively supported by the game, and users who are curious will either have to disable it globally in the NVIDIA control panel, or be forced to rummage through the games .ini files.  Either way, the experience will vary, and movement speed will increase drastically at higher frame-rates.

The game offers four graphical presets, which can be automatically selected based on your video card. The presets include Low,  Medium, High and Ultra and they will each automatically change the quality and severity of a number of the available graphical settings.

Detail

Antialiasing:  Allows for selection between Off (Best performance), FXAA (low), or NVIDIA’s TXAA (Best quality).

Anisotropic Filtering: A method of enhancing the image quality of textures on surfaces at certain angles; can be set to Off, 2, 4, 8, 12 or 16 samples.

Texture Quality:  Adjusts  the resolution of textures applied to objects in-game; can be set to either Medium, High or Ultra.

Shadow Quality:  Adjusts the resolution and detail shadows cast by in-game objects; can of be set to either Low, Medium, High or Ultra.

Shadow Distance: Adjusts the distance in which shadows are rendered; can be set to either Medium, High or Ultra.

Decal Quality: Decals are essentially objects which are not always present in-game such as bullets, blood and anything else that is created as result of an action. This setting dictates how long a decal will stay visible, how many will accumulate at once, as well as general quality; can be set to None, Medium, High or Ultra.

Lighting Quality: Adjusts the quality of cascade effects and light presence; can be set to Medium, High or Ultra.

Godrays Quality: Adjust the quality, and number of light shafts which are casts down from above, through obstacles such as windows, trees, and even thick fog; can be set to Off, Low, Medium, High or Ultra.

Depth of Field: Applies a film like filter which blurs objects that are at a distance in order to simulate depth; can be set to Standard (Low), or Bokeh (Best quality).

Ambient Occlusion: Adjusts the light and shading application to adjacent surfaces, such as the point where two walls meet; can be set to Off (Best performance), or SSAO (Best quality).

Screen Space Reflections: Toggles screen space aware reflections, which appear when light interacts with reflective surfaces.

Wetness: Toggles the appearance of wetness on objects which encounter water.

Rain Occlusion: Toggles whether or not rain can be subject to occlusion effects.

Motion Blur: Applies a filter which blurs screen space and objects during motion.

Lens Flare: Applies a film-like lens flare when the camera is at certain angles to light sources.

View Distance

Object Fade: Adjusts the distance at which objects such as signs, trash cans, and other non-NPCs pop-in or disappear from view.

Actor Fade: Adjusts the distance at which NPCs pop-in or disappear from camera view.

Grass Fade: Adjusts the distance at which grass pops in or disappears from view.

Item Fade: Adjusts the distance at which dropped or placed items pop-in or fade from view.

Distant Object Detail: Adjusts the level of detail of far away objects.

Object Detail Fade: Adjusts how quickly grittier details of objects fade away  from the player’s view.

[section label=3. Graphics Comparison]

Fallout 4 PC Graphics Comparison

Taking a look at our view of Diamond City on both Low, and Ultra, you can see that there’s not really a huge visible difference. I noticed that objects such as shadows seem to render at greater distances, and in much greater detail. Lighting is also better, but overall quality, and detail doesn’t seem to change too much.

Textures

Comparing texture quality between the lowest setting (Medium) and the highest (Ultra) gives us much of the same result. In fact, there is absolutely no discernible difference between the various texture quality settings on just about any of the objects, buildings or models that we witnessed. It’s a wonder Bethesda even decided to include Texture Quality settings at all, as they seem to have no actual effect on graphical fidelity.

Godrays

Of all of the graphics technologies used in Fallout 4, it seems the  NVIDIA GameWorks powered Godrays are the most intensive. Taking a look at the various settings (there’s also an Off option) we can see that at Ultra there is more detail in the light beams, with less pixelation at certain angles than at lower settings.

[section label=4. Testing Setup and Methodology]

Fallout 4 PC Performance

Testing Setup

Haswell-E X99 Test Bench

CPU	Intel Core i7 5960X @ 4.2GHz
Motherboard	Asrock X99 OC Formula
Memory	Crucial Ballistix Elite 16GB DDR4-2666
Graphics	2x ASUS STRIX GTX 970 4GB
Boot Drive	Samsung 850 EVO 500GB M.2 SSD
Storage Drive	ADATA Premier SP610 1TB SSD
Power Supply	DEEPCOOL DQ1250
CPU Cooler	DEEPCOOL GamerStorm Captain 360
Case	Phanteks Enthoo Pro
Operating System	Windows 10 Pro

Special thanks to Phanteks, DEEPCOOL, Asrock, and Crucial for supplying vital components for this test bench.

Testing Methodology

Game performance is measured in both average FPS, as well as Frame Time, both of which are logged using the benchmark and recording tool FRAPS, and then analyzed using FRAFS. Each game is tested a minimum of 3 times with the average of those results being displayed in our scores. We’ll test the game at both 1080p, and 1440p using our GTX 970 in both single, and SLI configurations at all of the available graphical presets, which include: Low, Medium, High, and Ultra. We’ll then do further testing at 1080p with various CPU configurations in order to test how many cores the game can scale to. In order to determine the actual performance of the game, we’ll be disabling the hard V-Sync cap via the NVIDIA control panel.

All testing was done using the recently released NVIDIA GeForce Game Ready 358.91 for Fallout 4.

Frame Time Analysis

Frame Time Analysis is a recent addition to our graphics testing suite, it allows us to measure the overall gaming experience, and fill in the gaps that traditional FPS performance measuring leaves. Frame Time itself, is simply the amount of time it takes (in milliseconds) for the GPU to render a given frame. Higher or inconsistent Frame Time can result in a poor experience, which will be perceived as micro-stutter. In order to measure micro-stutter, we’ll be measuring both the 99th, and 99.9th percentile. These results are essentially the worst Frame Time a user can expect both 99%, and 99.9% of the time. While every user is different, it is unlikely that most would notice anything below 30 ms, so that is what we look to achieve. In addition, we’ll want to look out for large gaps between the 99th, and 99.9th percentile as that can also result in noticeable stutter, or spikes in performance.

[section label=5. Performance]

Fallout 4 PC Performance

For testing, we do a  lap around the area “Diamond City” which is the first area in the game that displays a good mix of population, long draw distances, lighting effects, and other variables that are necessary for properly gauging the game’s performance.

Average FPS

 

SLI Scaling

CPU Scaling

For CPU scaling tests, we test the game on low at 1080p. We then configure our Core i7 5960X in a way that simulates other popular CPU configurations by disabling a number of cores or hyper-threading. Our goal here is to find the difference in performance as it pertains to CPU Cores, be it physical or logical (hyper-threading). We are not trying to simulate the performance of any specific SKU, therefore clock speed will be kept at our standard 4.2GHz. In other words, if you’re using a Core i3, performance displayed here may not be indicative of what your experience will be, as our CPU will be clocked higher than those locked multiplier parts can achieve.

Frame Time Variance

[section label=6. Conclusion]

 

Alright, so there you have it. After our initial run of testing, we found that Fallout for has a few surprises, and a few not so surprising issues. While, we have admittedly not gone over a large vareity of hardware here, or testing environments — those tests are on the way! — we did get some very useful insights. First, we were able to confirm that for the most part, the game does run well without too many issues. While we only tested with an NVIDIA graphics card, it is still good to see that unlike some other major releases this year (looking at you Arkham Knight) the game can be played with relative ease on for at least the majority of users.

While we look forward to testing further with AMD hardware in the mix, it is good to see that our GTX 970 was able to easily maintain high frame-rates at even the highest settings at both 1080p, and 1440p. This means it is safe to assume that if you’re targeting 1080p, and 60 FPS, something like a GTX 960 should be able to meet your needs at relatively high settings. That being said, we’ll have to do some further testing to confirm these findings.

Furthermore, we were able to confirm what we knew all along, Fallout 4 does not support SLI at all, at least for now. While this isn’t the worst issue ever, it is still rather unfortunate, especially considering this game was developed along side NVIDIA for the PC. You’d think that they’d make the extra effort in ensuring it supported all of NVIDIA’s hardware features, or at the very least, one as popular as SLI. Although, while I can forgive the lack of initial SLI support, one thing I cannot forgive is the texture quality, not only are the textures in the game pretty awful, despite various settings, the textures look exactly the same no matter which you select. This is very disappointing. It begs the question, that if they could overlook something so simple as this, what else did they overlook?

Finally, we have our CPU scaling tests, which were definitely a surprise. We found that, not only does the game seem to perform better with more cores, it actually had significant performance increases all the way up to six. In fact, there was noticeable gains with a hyper-threaded quad-core over a standard quad-core, which is not usually the case. This means that users with a Core i7 can expect greater performance than those with a Core i5, at least up until they’re GPU limited, which if you’re playing at higher settings, will be sooner than you’d think. What is definitely worth mentioning is the relatively horrible performance of the dual-core configuration, compared to everything else. Sure, if you’re on a lower-end GPU such as a GTX 750, then users with a Pentium dual-core might not have any issues, but in our experience we saw a lot of stutter, and hit 100% CPU usage at nearly all times. At this point, it is hard to recommend anyone buy something like a Pentium G3258, if playing AAA titles is your goal.

As previously stated, we’ll be doing a few more tests of Fallout 4 with a wider range of hardware, and in various testing environments. So, if we didn’t answer all of your questions in this report, be sure to check back soon for more.