All postsAI explained

Physics-informed neural networks: when PINNs help or fail

APJ-RAISE team · · 7 min read

A physics-informed neural network (PINN) is a neural network trained to fit data and obey a known physics equation at the same time, by adding the equation's error to the loss function.

White smoke flowing in swirls over a black surface
Photo: cottonbro studio, Pexels

That one extra loss term is the whole trick. It lets a network learn a flow field from almost no data. It can also make training fail on problems that a textbook solver handles in seconds. This post is about that tension: when does adding physics to the loss help, and when does it break?

Maziar Raissi, Paris Perdikaris and George Em Karniadakis introduced PINNs in two arXiv preprints in November 2017, and the full paper appeared in the Journal of Computational Physics in 2019. Interest is rising again: in October 2026 Gartner grouped three of its 2027 trends under what Network World reports as the nontextual AI frontier, meaning AI for numbers and the physical world.

How do physics-informed neural networks work?

A PINN takes coordinates, such as time t and position x, and outputs the quantity you care about, such as u(t, x). The physics comes from a partial differential equation (PDE), which says how u changes in space and time.

Training adds up three errors. The data term compares the output with any measured values. The boundary term checks known values at the edges and at the start time. The residual term is the new part: put the network's output into the PDE and see what is left over. A perfect solution leaves zero.

The residual sits at many sample points inside the domain, called collocation points. To compute it you need derivatives such as ∂u/∂t and ∂²u/∂x². Automatic differentiation, the same tool that computes training gradients, gives these exactly at any point, with no grid.

In the original paper the only data are initial and boundary values, so the loss is MSE = MSE_u + MSE_f, the error on known points plus the mean squared residual.

Worked example: Burgers' equation, the first PINN

Burgers' equation is a simple fluid model that can form a sharp front, like a shock. The paper notes it comes from Navier-Stokes when the pressure term is dropped. The problem: u_t + u * u_x - (0.01/π) * u_xx = 0, for x from -1 to 1 and t from 0 to 1, with u(0, x) = -sin(πx) and u = 0 at both edges.

  1. 1Network: inputs (t, x), output u. The paper used 9 layers, 20 tanh neurons per hidden layer, 3,021 trainable parameters.
  2. 2Labelled data: 100 random points on the start line and the two edges, where u is known.
  3. 3Collocation: 10,000 points inside the domain, spread by Latin hypercube sampling.
  4. 4Residual: f = u_t + u * u_x - (0.01/π) * u_xx at each collocation point, via automatic differentiation.
  5. 5Loss: mean squared error on the 100 known points plus mean squared f on the 10,000 points.
  6. 6Optimiser: L-BFGS, which uses the full dataset each step.
  7. 7Result: relative L2 error of 6.7 × 10⁻⁴, after about 60 seconds on one NVIDIA Titan X GPU.

Look at the ratios, which are our arithmetic from the paper's numbers. There are 100 collocation points for every labelled point. And 3,021 parameters learned from 100 labels is about 30 parameters per label. A plain network with that ratio would overfit badly. The physics term supplies the missing information. The network never sees a single value of u inside the domain.

The paper also notes that with zero collocation points, the model is just an ordinary network that ignores the equation. Running both versions side by side is the cleanest way to see what the physics term buys you. Raissi's TensorFlow code is on GitHub at maziarraissi/PINNs.

When does adding physics to the loss help?

The evidence points to three situations.

Sparse data is the first. If you have a handful of noisy sensors and a trusted equation, the residual fills the gaps, as Burgers' shows with 100 labels.

Inverse problems are the second, and probably the strongest case. Given measurements, a PINN can recover an unknown constant inside the equation, such as viscosity. Part II of Raissi's preprints does this for Burgers'. Classical solvers need that constant up front, so they must wrap the solve in a search.

Smooth solutions are the third. Neural networks learn smooth patterns easily, and the physics term keeps them from wandering where data is missing.

PINN failure modes: when the physics term breaks training

On harder problems, PINNs often fail even when the network is large enough. Three causes are well documented.

  • Sharp features: neural networks learn slow, smooth patterns first and struggle with fast ones. Nasim Rahaman and colleagues called this spectral bias (ICML 2019). Our reading: this is one reason shocks and thin layers are hard for PINNs.
  • Tug of war between terms: Sifan Wang, Yujun Teng and Paris Perdikaris (2020) traced failures to stiff gradients that let one loss term swamp the others. Reweighting the terms during training gave them 50 to 100 times better accuracy on several problems.
  • The physics term itself: Aditi Krishnapriyan and colleagues (NeurIPS 2021) found PINNs can miss the relevant physics even on slightly harder convection, reaction and diffusion problems. Adding the residual made the loss surface harder to optimise.

Krishnapriyan's team offered two fixes. Curriculum regularisation starts with an easy version of the problem and makes it harder step by step. Sequence-to-sequence training solves short time windows one after another. They report errors one to two orders of magnitude lower.

So the same term that helps on Burgers' with low viscosity can hurt when convection gets strong. It depends on how sharp the solution is and how the loss terms are balanced.

What most PINN tutorials leave out

Many tutorials present PINNs as a faster replacement for solvers. For a plain forward problem, where you know the full equation and every boundary value, a classical finite-difference or finite-element solver is usually the safer and faster first choice. A fair paper compares against one.

Tutorials also tend to show the 6.7 × 10⁻⁴ result without the setup that made it work: L-BFGS, a full batch and a smooth start condition. Swap in a different PDE with the same code and the error can jump by orders of magnitude.

A quick guide from the sources above:

  • Likely to help: inverse problems, sparse noisy data, smooth solutions, a trusted equation.
  • Likely to struggle: shocks or thin layers, strong convection, very different scales between loss terms, long time spans in one go.
  • Use something else: many similar problems to solve fast, where a neural operator fits better, or a full forward problem, where a classical solver wins.

PINN vs neural operator

A PINN learns one solution to one problem and needs retraining for each new setup. A neural operator learns from many solved examples to map an input function, such as a start state, to a solution function. It then answers new cases without retraining. DeepONet (Lu Lu and colleagues, Nature Machine Intelligence, 2021) and the Fourier Neural Operator (Li and colleagues, 2020) are the best known; the FNO authors report up to three orders of magnitude faster than traditional solvers on their tests.

Data-only models are a third path. Google's WeatherNext 3, announced on 3 September 2026, gives hourly forecasts at up to 5 km resolution. Google does not describe it as physics-informed, so it is not a PINN. A January 2026 arXiv paper by Dai and colleagues treats PINNs and neural operators as one family, which is a good read after the basics.

PINN code on GitHub and a first project

Three routes, as of November 2026. DeepXDE (lululxvi/deepxde), by Lu Lu, Xuhui Meng, Zhiping Mao and George Karniadakis (SIAM Review, 2021), supports TensorFlow, PyTorch, JAX and PaddlePaddle. NVIDIA PhysicsNeMo (NVIDIA/physicsnemo), formerly Modulus, is Apache 2.0, with releases v2.2.0 (27 August 2026) and v2.2.2 (11 September 2026) and a lid-driven cavity PINN example. Plain PyTorch or JAX is best for learning.

A first project that tests this post's question directly:

  1. 1Rewrite the Burgers' example in PyTorch. Set requires_grad=True on t and x, and use torch.autograd.grad with create_graph=True for u_t, u_x and u_xx.
  2. 2Train with Adam, then finish with L-BFGS. Aim for an error near 6.7 × 10⁻⁴.
  3. 3Lower the viscosity step by step and record where the error jumps. That is your failure point.
  4. 4Add adaptive loss weights or a curriculum and see how far the failure point moves.
  5. 5Solve the same cases with a finite-difference solver and compare error and time honestly.

Our designing experiments course covers baselines and ablations. For AI work on the full fluid equations, see AI and the Navier-Stokes proof; for another 2027 trend about the physical world, read our world models explainer. To build a PINN project with others, join APJ-RAISE.

Questions people ask

What are PINN failure modes?
The main ones are spectral bias against sharp features, unbalanced loss terms, and harder optimisation when the physics term is strong, such as high convection. Curriculum training and adaptive weights help.
What is the difference between a PINN and a neural operator?
A PINN solves one problem and is retrained for each new setup. A neural operator, such as DeepONet or the Fourier Neural Operator, learns from many examples and answers new cases without retraining.
Which is the main physics informed neural networks research paper?
Raissi, Perdikaris and Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, Journal of Computational Physics 378 (2019) 686-707.

Sources

  1. 1.Raissi, M., Perdikaris, P. and Karniadakis, G. E. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics 378, 686-707, 2019.
  2. 2.Raissi, M., Perdikaris, P. and Karniadakis, G. E. Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations. arXiv:1711.10561, 2017.
  3. 3.Raissi, M., Perdikaris, P. and Karniadakis, G. E. Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations. arXiv:1711.10566, 2017.
  4. 4.Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S. and Yang, L. Physics-informed machine learning. Nature Reviews Physics 3, 422-440, 2021.
  5. 5.Krishnapriyan, A. S., Gholami, A., Zhe, S., Kirby, R. M. and Mahoney, M. W. Characterizing possible failure modes in physics-informed neural networks. NeurIPS 2021. arXiv:2109.01050.
  6. 6.Wang, S., Teng, Y. and Perdikaris, P. Understanding and mitigating gradient pathologies in physics-informed neural networks. arXiv:2001.04536, 2020.
  7. 7.Rahaman, N., Baratin, A., Arpit, D. and others. On the Spectral Bias of Neural Networks. ICML 2019. arXiv:1806.08734.
  8. 8.Li, Z., Kovachki, N., Azizzadenesheli, K. and others. Fourier Neural Operator for Parametric Partial Differential Equations. arXiv:2010.08895, 2020.
  9. 9.Lu, L., Jin, P., Pang, G., Zhang, Z. and Karniadakis, G. E. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence 3, 218-229, 2021.
  10. 10.Lu, L., Meng, X., Mao, Z. and Karniadakis, G. E. DeepXDE: A deep learning library for solving differential equations. SIAM Review 63(1), 208-228, 2021.
  11. 11.Dai, Y., Chen, S., Wang, Z. and others. Learning PDE Solvers with Physics and Data: A Unifying View of Physics-Informed Neural Networks and Neural Operators. arXiv:2601.14517, 2026.
  12. 12.NVIDIA PhysicsNeMo releases on GitHub.
  13. 13.Google. Introducing WeatherNext 3, our most advanced and accurate global weather AI model. 3 September 2026.
  14. 14.Bednarz, A. Gartner's top 10 strategic technology trends for 2027. Network World, 20 October 2026.
All posts