How does energy modeling work?

After modeling the potential impacts of the federal permitting bill that’s scheduled for a Senate vote on Monday, I’ve been thinking a lot about how influential energy modeling can be—and how contentious. In recent conversations, I’ve found that building a shared understanding of how energy modeling works can alleviate some points of contention, particularly around the role of assumptions in modeling.

Here, I explain energy modeling, note the susceptibility of model conclusions to modelers’ assumptions, and discuss modeling practices that I believe make conclusions more trustworthy.

The energy modeling process

The sketch above illustrates the basic process of modeling. The modeler makes assumptions and feeds them to a model, computer code that runs logic and returns conclusions to the modeler. The modeler looks at those conclusions, stares into space for a while, talks with friends, adjusts assumptions, and repeats. After many iterations, the modeler decides which conclusions to communicate to the audience and how to communicate them.

For energy modeling in support of policymaking, the modeler might work at a think tank, university, or national laboratory. The audience might include policymakers, reporters, or social media influencers. The modeler’s communication with the audience might be as formal as a peer-reviewed scientific paper or as informal as a conversation.

The sketch below illustrates the internal structure of a model. It consists of equations and data sources, both of which the modeler chooses. I use the blanket term assumptions to denote modeling choices about either equations or data sources. Data are numbers that enter equations. Equations map data into conclusions.

For energy modeling, the equations may be as simple as “output equals efficiency times input” or as complex as “the solution equals a minimizer of the objective function over the feasible region.” Example data include energy demands or prices, energy conversion efficiencies, pollutant intensities of various fuels, or capacity constraints on power plants, power lines, production wells, or pipelines. Some modeling assumptions directly specify data, such as “this power plant’s efficiency is 37%.” Other assumptions specify data sources, such as online weather databases or spreadsheets from the Energy Information Administration, from which code feeds data into equations.

Good practices for energy modeling

All energy modeling has the form “If this, then that.” All modeling conclusions are contingent on the modeler’s assumptions. This fundamental structure makes the modeling process highly susceptible to influence by the modeler. Bad-faith modelers can make assumptions that steer model conclusions into alignment with their ideological, political, or financial goals.

Does this mean that all energy modeling is useless, untrustworthy? I don’t think so, but I do think audiences for energy modeling should be on the lookout for a few red flags, including deviations from the following four practices that I believe make energy modeling more trustworthy.

1. Transparency. Transparency lets people audit assumptions and reproduce or falsify conclusions. Put another way, opacity makes it difficult or impossible to catch modelers’ thumbs on scales. In my opinion, reporting the conclusions of an opaque energy model amounts to little more than holding up a “Trust me” sign adorned with graphs, statistics, and the other trappings of science. I write here about what I consider reasonable standards for transparency in energy modeling. They include clearly explaining all equations and data, making all code open source, structuring and commenting code well, and disclosing funding sources and relevant financial entanglements.
2. Uncertainty quantification. Energy modeling often involves highly uncertain data, such as future energy demands or prices. Modelers sometimes pick one numerical value for each uncertain variable and report a single set of conclusions, projecting an illusion of certainty. I prefer modeling that explores a range of values for uncertain parameters, either by specifying distributions in a probabilistic framework, or by specifying scenarios (such as “optimistic,” “middle of the road,” and “pessimistic”) in a deterministic framework. At the very least, I believe modelers should point out and discuss sources of uncertainty and how they may influence model conclusions.
3. Sensitivity analysis. All too often in energy modeling, contentious assumptions are unavoidable. In such cases, I believe it is good practice to analyze how sensitive model conclusions are to contentious assumptions. A straightforward approach is to sweep a range of plausible values for a contentious number, then show how model conclusions change in response.
4. Parsimony. More complex models—ones with more equations for the modeler to structure and more input data for the modeler to specify—present modelers with more opportunities to make assumptions that may bias conclusions. Unnecessary complexity also complicates auditing, reproducibility, uncertainty quantification, and sensitivity analysis.