New Realities Impact Our Ability to Forecast Natural Gas Markets.

Part III: Methodologies Get Challenged by Industry Critical Events. 

Complexities of forecasting natural gas markets.

Keeping track of ever-changing composition of supply and demand drivers.

Analysis in Part II showed that the natural gas supply-demand dynamics during critical events (cold snaps) do not always produce prices that are otherwise expected in accordance with the basic laws of economics. Does it mean that these dependencies are broken? if so, how and when did this happen? In this article, we explore developments of the natural gas sector that have led to a complicated combination of price drivers and uncertain outcomes from their impact.

Uncertainty starts at the highest level: the global supply data is not clear

There is probably nothing less gratifying than building predictions for energy resources, natural gas being one of them. Difficulties start right at the basic fundamental parameter – supply. No doubt, it is rather challenging to come up with the exact assessment of deposits concealed deep underground, hence we have to accept that the best numbers we can get are estimates. Any estimate is burdened with the truckload of uncertainties and probabilities (P95 or P90 vs. P50). The results get even dimmer when the industry cannot agree on a valuation methodology. Various combinations of "proven", "probable", "possible", "developed producing", "undeveloped", and "discovered" resources bring in a multitude of classifications offered by different assessors.

 It is not even so important to learn on whether it is the inability or absence of any desire among estimators to agree on the standard calculation methodology and how much geopolitical flavour is added to it. What matter is that we end up with a wide spread of reserves estimates. As a result, CIA, EIA, IEA, OPEC, Russia, BP and multiple third-party consultants cannot even agree on the country, Russia or Iran, leading in these reserves. According to some, it is Russia (60 tcm vs. 42 tcm), according to others it is Iran (34 tcm vs. 32 tcm). What do you think about such spread?

Commercialization of shale gas exploration has added more supply of natural gas to the markets. Meanwhile, environmental regulations and standards, varying in their strictness by country, have a divergent effect on the natural gas recovery volume thus adding to the complexity.

Demand drivers have been going through transformation 

Another side of the fundamentals’ balance – demand – has been going through rather confusing alterations. On the one hand, traditional drivers -economic activities and demographic factors – do not cease to contribute to its constant growth. New businesses, automation of manual labor, population growth and longer life expectancy increase the natural gas usage rate. Renaissance of the domestic industry with production operations run on natural gas or its derivatives has also been a contributing factor. In some parts of the world, including USA, shale gas exploration boom has created an excess of supply allowing for its shipment in the form of LNG to other parts of the globe; it contributes to the demand growth in the regions abundant with this resource.

On the other hand, the global and local drives for the improved environmental stance result in direct and indirect impact on demand; they also have been pushing demand in opposite directions. Replacement of the petroleum- with natural gas-driven transportation fleet inevitably drives it up. Some local governments and municipalities' standards and codes require replacement of the natural gas -fed heating and cooking infrastructure for residential and commercial customers with that run on electricity. Distributed power generation has started to replace the natural gas – fueled heating and cooking infrastructure. Because this replacement is made not on a mandatory, but rather voluntary basis, it is challenging to forecast its expanse. 

Environmental laws and regulations have an uplifting impact on natural gas demand from its prominent customer – power industry. Replacement of the coal-fired generation with renewables requires a parallel increase in natural gas-fired power generators to remediate their intermittency. As shown in graphs below, coal-fueled generation has been in decline mirrored by the growth of its two opponents, natural gas and renewables, on a net basis.  

Figure 1: US Coal-Fired Power Generation, Natural-Gas-Fired Power Generation and Power Generation by Renewable Sources (Source: EIA)

Figure 2: US Power Generation by Energy Source (Source: EIA)

Natural gas pricing mechanism in a constant state of update 

With all these fast-pacing changes and adjustments, it is not surprising that the natural gas pricing mechanism has been through a series of alterations and is yet to be settled down. 

First of all, there is an inherited logical loop in the price setting mechanism in relation to supply. "Proven" reserves, which are most frequently used for defining the basic supply, according to some definitions have to be not only technically recoverable, but also economically profitable. Economical profitability has to do with the regulatory and contractual conditions as well as market prices. From this perspective, natural gas price, being impacted by supply-demand curves, at the same time becomes a parameter affecting the supply estimates. 

Further to that, it was not so long ago when oil was acting as the generally accepted underlying commodity for the natural gas pricing.

Probably the easiest explanation of this relation is that oil and natural gas both originate from the same source, hydrocarbon reservoirs. In majority of cases, natural gas is found compressed on the top of crude oil horizon. As the economics of both these natural resources are bound together, it was quite logical to link their prices. Oil, easily transported across the globe and remaining the primary energy source in a multitude of industrial applications, has developed a universal marketability or liquidity. Unlike oil, transportation of natural gas has been, at least until recently, dependent on a network of pipelines; this made it a local commodity. With such limitations, it seemed reasonable to link the natural gas contracts' pricing algorithms to prevailing oil prices.  

At times, a weighted average of crude oil or its products (gasoil, gasoline, naphtha, fuel oil, etc.) was used for natural gas pricing.  Natural gas analysis and price projections were linked to the macroeconomic models of the oil markets. Those models usually incorporated features of oil supply, such as exploration, extraction and depletion, and oil demand. The general expectations for a consistent gas price increase were aligned with the projected decline in oil supply based on the theory of “peak oil”. For a long time, those models worked without fail; risk management departments used futures contracts to hedge their positions and effectively offset risks. Then something happened. Previously made long-term projections turned out much higher than the actual natural gas prices. It meant that the forecasting models started failing in their reflection of the new reality.

The new reality included growth in shale production that brought in an influx of this resource. Expansion of LNG infrastructure worldwide offerred completely new opportunity to these additional volumes. It has effectively turned natural gas into a global commodity. With open markets gaining more momentum, more pricing hubs have been set up around the world with some of them gaining recognition of global benchmarks. As a result, more contracts get priced to the natural gas markets while oil-linked trades are slowing becoming a fact from the past. The new natural gas pricing models start incorporating supply and demand dynamics of the natural gas itself, with more attention given to its global nature. 

Long-term vs. short-term natural gas projections

Like any other commodity, which is used to support the immediate human needs, there are few time horizons for settling natural gas prices. Longer terms (mid-term and long-term) are somewhat similar in their approach. Those trades are executed predominately to serve the expected demand levels and risk management purposes. Short-term trades mainly balance the current fluctuations in demand. Accordingly, price projection mechanisms differ for short-term versus long-term time horizons. Short-term insight has a deeper look into behaviour of all market trading parties reflected in technical parameters, or technicals. Balance between fundamental factors, while present in the both approaches, has a much heavier impact in long-term forecasts. Fundamentals usually carry a much more profound bearing towards expectations for changes in the fossil fuel recovery and adjustments to its utilization rate by different classes of customers. These changes are driven by different factors. Some of these factors can be referred to as "classical drivers" while others have been added to the analytical plate recently, playing a role of "disruptors". Currently, the clean and environmental agenda is the most dominating factor influencing the natural gas movements; it has its impact on the both timelines.

Long-term expectations

There are several drivers currently on the demand and supply sides that are likely to keep counter pressure on prices over longer terms. Figure 3: Drivers of the Long-Term Natural Gas Price Projections summarizes these factors.

Figure 3: Drivers of the Long-Term Natural Gas Price Projections

Classical drivers that have been and are unlikely to cease influencing the general demand in the future are economic activities and demographics. Population continues growing with the life expectancy getting longer. Population effectively constitutes residential customers; their increase is accompanied by the industrial and commercial sectors growth required to sustain the increase in the humankind thus also partially stimulating economic growth. Economic growth is greatly influenced by political and geopolitical events and developments, even though at times it can be curbed by natural disasters. 

Population growth rate will probably slow down as a result of various measures being put in place to curb its unsustainability. Regardless of the rate, population increase will continue putting an upward pressure on the natural gas demand in all consumer classes. At the same time, the overall economy shifting towards less energy-intensive technologies creates a counter force for the demand level. Another factor, which has mixed outcomes is the cost of regulatory compliance with tighter environmental requirements. These outcomes differ depending on the specifics of the industry and applications. In some cases, there is a tendency of replacing oil-fed equipment emitting some of the highest levels of pollutants with natural gas-based technologies characterized by lower than oil-driven damage to the environment. In cases of much stricter requirements, natural gas can be viewed as a culprit; its replacement with cleaner technologies creates potential for suppressing the natural gas usages. As the baseline conditions are not universal across the industries and within each industry, the combined effects will likely continue counterbalancing each other.   

Don’t forget the globalization trend of the natural gas production and consumption: LNG tankers will continue carrying liquefied gas to the locations with the shortage of supply boosting the local production demand.

Classical factors have a direct effect on gas consumption; however, they also indirectly influence industries, which rely on natural gas as a fuel. For example, in circumstances of growing residential and industrial demand, electric power industry will have to jack up its production to serve the growing customer demand. Any changes within this industry, such as a shift of generation stack composition towards natural gas production, will have a compounding effect on the total natural gas consumption. Gas-fired power generation has been growing and is expected to remain on this path, at least for some time. It is mainly because of the low fuel prices and environmental regulations requiring increase in renewable units, which, in their turn, have to be supplemented by auxiliary power production; natural gas being the best candidate for this job. Natural gas-fired power production is also boosted by the preference granted to it when the decisions on replacing the retiring nuclear and coal-fed power units are made. Even though there are chances that more aggressive environmental standards are introduced globally, natural gas is likely to remain dominant source in the next one or two decades. Some opposing pressure on the demand increase will continue coming from the energy efficiency efforts.

Supply drivers are established and have been traditionally skimmed down to the resource availability, recovery and transportation availability. A newly emerged global nature of natural gas offers more opportunities thus opens new supply options. Constant improvements to the resource recovery technology are likely here to stay. Natural gas extraction technology upgrades, declining cost of shale gas development will be somewhat restrained by environmental concerns; however, these restrains differ across the globe and get altered with changes in political agendas. Natural gas transportation is involved and expensive. Infrastructural changes also can act as either deterrents or boosters. For example, new pipelines are built to connect new gas fields in Alaska and the Canadian Northwest Territories. At the same time, some LNG conversion facilities face pushback from the approving authorities.

Global nature of the natural gas brings more geopolitical events to the table, which start affecting natural market balance; their influence will get stronger with the continued growth of LNG sector. Critical political developments in the fuel-bearing locations can have drastic impact on supply. Also, gas cartels might gain more power to control gas production.

Short-term expectations

Short-term traders rely not only on information about the immediate supply and demand dynamics; they also watch trades executed by their counterparties and other trading entities. Technical parameters alongside current and nearing developments in fundamentals are less predictable than long-term factors. It makes short-term forecasts very involved and instant in their execution.

Figure 4: Drivers of the Short-Term Natural Gas Price Projections

Some of the load (demand) patterns are rather predictable as they have been developed over the long time to establish rather stable consumption profiles for different consumer classes. There are daily, weekly and seasonal profiles. Residential customers exhibit consumption peaks in mornings and evenings during weekdays, on weekends and holidays. Profiles of industrial and commercial classes are somewhat reversed to those of residents. It does make sense as we have the same group of people moving from one location to another (home – work). Then there are winter months requiring more gas to be sent to the burners to keep homes and workplaces warm. Heating and cooling demand can be augmented when weather exhibits an outlier behaviour, which would require quick adjustments to supply either through withdrawing more gas out of the storage or buying it in the market. In such cases, natural gas consumption can be affected not only by the direct utilization, but also via increases in gas-fired power generation demand.

Weather affects not only demand, but in some cases supply and its delivery to the consumers. Extremely cold temperatures and storms can lead to infrastructure failure causing delays in production, transportation and operation of storage facilities.

Like the long-term forecasts, short-term projections can be affected by the geopolitical events bringing chaos to natural gas market. Thus, domestic disputes on the side of the shipping party, can prevent an LNG tanker from being shipped to its destination thus cutting off the expected supply on the receiving side. Cases of that nature have not been reported yet, but it is only a matter of time when they become the reality.

Natural gas short-term expectations are heavily driven by market volatilities. Like equity market players, natural gas traders study the price, volume, as well as the speed or the momentum of price changes. There is no specific set of rules or markers to follow; each can choose whatever works for the region or the specific office. Some follow the movements of volatility measurements, such as Exponential Moving Averages (EMA), Relative Strength Index (RSI), Moving Average Convergence Divergence (MACD), Stochastic Oscillator to name the key ones.


With all information gathered and analysis performed, how can we explain that the current forecasting models turn out to be not so useful during critical events (cold snaps) failing to predict price spikes at specific time points? They carry a high price tag as risk management cannot react adequately in ensuring the uninterrupted supply at reasonable cost. Usually, these impacts are location-specific as utilities have to deal with the local natural gas-substituting power generation technologies and other fuels (oil-based) for replacement of heating fuel; the type and cost of these substitute fuels vary and should be tracked on a more granular level. 

The industry critical events leading to multiple outages and significant price spikes, such as those occurring during cold snaps, effectively represent the bifurcation points. Analysis of critical events / bifurcation points is an important step in constructing proper projections. Meanwhile, bifurcation points are treated in the short-term models where the technicals do take over the fundamentals allowing for market behaviour imperfections prevail leading to distorted market reaction. When these bifurcation points are included in longer projections as they are, it creates a distorted long-term picture of the market by shifting and diluting the averages.

The natural gas industry is expanding inward and outward at the same time. It becomes global and at the same time more localized with the power industry relying more on it for the real-time balancing and fuel substitution. The trick is to bear in mind both trajectories when building models for the long-term investments or preparing for and handling critical events. The growing complexities of gas markets and intertwined operations with the power sector require new approaches to forecast methodologies, as well as coordinated operations with the power producers. For better results, each critical event should be analyzed from the perspective of acting natural gas operators and electric generators to see dependencies in operations between these two industries. Utilities with their risk management departments should be alerted in advance of what to expect for the region, how the existing and interrelated systems will be affected and the likelihood of the response time.

​​​​​​​ Industry market forecasts have to go under a serious overhaul as the currently prevailing methodologies for short-term and longer term projects use different sets of drivers, inputs and algorithms.  Short-term weather changes and historical system responses should be adjusted by parameters used for mid and long- term modeling. Ultimately, long-term, mid-term and short-term models should fluently integrate with each other providing better insights into long-term projections and current developments.

Contact us if you want to share your point of view or need more insight and assistance for developing better projections.