Gas is the relatively clean fossil fuel many hoped would smooth the transition to low-carbon energy. Instead it is turning out to have its own transition problems.

Natural-gas prices around the world have jumped in recent months, lifting energy bills, stoking worries about higher inflation and even prompting factory shutdowns. The European benchmark rose more than 30% this week to a fresh record. The tight market can be attributed mainly to unexpectedly extreme weather, maintenance catch-up following the pandemic, supply bottlenecks...

Gas is the relatively clean fossil fuel many hoped would smooth the transition to low-carbon energy. Instead it is turning out to have its own transition problems.

Natural-gas prices around the world have jumped in recent months, lifting energy bills, stoking worries about higher inflation and even prompting factory shutdowns. The European benchmark rose more than 30% this week to a fresh record. The tight market can be attributed mainly to unexpectedly extreme weather, maintenance catch-up following the pandemic, supply bottlenecks and a dash of geopolitics.

The good news is that conditions could look different next year: Spring will arrive and a wave of new supply will come onstream. The bad news is that gas prices are likely to get only more volatile as the world bumbles its way toward a low-carbon energy system by 2050 or thereabouts.

The gas market has a low tolerance for imbalances of supply and demand, as its history of price volatility illustrates. But mismatches are becoming harder to avoid. Increasingly uncertain demand raises the risk profile of future investments, likely making supply adjustments even slower and more erratic than they are today.

Gas is already widely used in heating, power generation and industries including glass, fertilizers and plastics. Yet it is also at the heart of the energy transition, as a cleaner-burning substitute for coal and oil, a fuel for backup electricity plants to bridge the gaps from intermittent renewables and possibly even a low-carbon fuel if its combustion is paired with carbon capture and storage.

Predicting demand for gas is currently challenging and becoming more so. Heat waves and frigid weather often cause demand spikes, and weather is getting more extreme and unpredictable. Demand from poorer countries will likely grow with their economies. However, many have joined developed nations in promising to reduce their carbon emissions by using renewables, increasing energy efficiency and other decarbonization measures. The pace of those shifts is anyone’s guess, as is the price of a ton of carbon and what regulations or incentives might be added to the mix.

Likewise, it can be tricky to forecast the supply of gas, which is typically slow to react. No global cartel coordinates gas supply as it does oil production. While it may take only a few months to get gas from a nearby shale well, it takes many years to build the pipelines or liquefied natural gas facilities that transport the fuel to importers around the globe.

More uncertain demand increases the risk of gas investing. It already goes in waves: Low prices in 2020, as in late 2015, quelled new investment commitments, while high prices have encouraged them—witness a recent spate of LNG investment announcements by U.S. producers. But most of those investments don’t come online until years later. More expensive gas also improves the economics for new wind and solar farms, displacing some future gas demand and possibly hastening its decline. A further risk is that gas-dependent producer nations, such as Qatar, might flood the market to fill their coffers before demand tapers.

All these moving parts rarely seem likely to result in a balanced market. Producers are in for a roller-coaster ride, though those with trading divisions, like Royal Dutch Shell and BP, should be able to profit from some of those swings. The rest of us should buckle up.

Related Video

A Swiss startup has created a giant vacuum cleaner to capture carbon dioxide from the air, helping companies offset their emissions. WSJ visits the facility to see how it traps the gas for sale to clients like Coca-Cola, which uses it in fizzy drinks. Composite: Clément Bürge The Wall Street Journal Interactive Edition

Write to Rochelle Toplensky at rochelle.toplensky@wsj.com