How do we know the causal graph is right?
We don't. But we know when we're wrong. How disproof, not proof, drives causal discovery on real data, and why that is the honest answer to the question every buyer asks.
We don't.
Nobody does, at least from observational data. Under the standard assumptions (more on those below), a causal graph of a real system is almost never identifiable from data collected by watching it.
So while you can’t know if a causal graph is right, you can determine other, more useful things: you can rule out which graphs are definitely wrong, and whether the graphs you’re left with disagree about the things you care about.
What is a causal claim?#
A “causal claim” is a claim about what happens when you change something - and you can't verify that kind of statement just by reading off historical data (i.e. what happened in the past). Two things moving together historically doesn't necessarily imply one caused the other.
Say the manager of an ore-grinding plant tells you that feed rate drives energy use. Feed rate is how fast ore goes into the mill. They're claiming that if someone throws more ore into the grinder, and changed nothing else, you need more energy to grind up more ore.
But that’s not something the data is actually going to show you. Think about it: the plant’s data will show that energy use was high on days that the feed rate happened to be high. It doesn’t technically say that more ore being ground up caused more energy consumption.
But what if the feed rate was lower on cool days, and temperature affects how much energy the mill was drawing?
Now suddenly you have days with higher feed rates and lower energy consumption. Grinding more ore is using up more energy, but the rate of power consumed is being influenced by something else.
That mix-up is called confounding, and it's why watching and doing give different answers.1 (Figure 1).
A causal graph is a set of claims like this one, drawn as a diagram. Every arrow in it is a claim about doing, which is why historical data can't confirm it on its own.
What is a causal graph?#
A causal graph is a diagram of causal claims.
Each box is a variable. An arrow from one box to another means changing the first changes the second directly. Each variable is set by whatever has arrows pointing into it.
If feed rate and ambient temperature are the only arrows into energy, the graph is saying:
That means energy is determined by feed rate and ambient temperature, through some function f. f is whatever the physics actually is, a straight line or something far messier. The graph doesn't need to know what f looks like, only which inputs go into it.
An arrow says an effect might exist, but not how big it is. A missing arrow is a claim too: no arrow from ore grade to energy means ore grade makes no direct difference to energy once you've accounted for feed rate and temperature.
Each missing arrow predicts that certain variables won't move together once you account for others. You can test that without changing anything.
What can the data tell you?#
Historical data can tell you two things:
- Which variables move together.
- Whether they still move together once you only look at records where some third variable was the same.
It can tell you a graph is wrong.
Take the graph from the last section. It has no arrow from ore grade to energy. That's a claim: ore grade does not affect energy, except through feed rate and temperature.
You can check that in old data. Take the days where feed rate and temperature were about the same. If the graph is right, ore grade and energy should have nothing to do with each other on those days. If high-grade days still used more energy, then the graph is wrong.
Every missing arrow gives you a test like this. Fail one, and the graph is out and no experiments are needed.
But it usually can't tell you which graph is right.
Feed rate and energy move together in the data. Two different wirings produce exactly that:
- Feed rate drives energy.
- Temperature drives both, and feed rate does nothing.
These are opposite answers for the plant manager. The data looks identical under both. Both pass every missing-arrow test. More data doesn't help.
Here's the general version. Three variables in a line can be wired three ways.
| Shape | Example | Do the outer two move together? | After accounting for the middle one? |
|---|---|---|---|
| A. Chain | feed rate → mill load → energy | Yes | No |
| B. Fork | feed rate ← ambient temperature → energy | Yes | No |
| C. Collider | feed rate → energy ← ambient temperature | No | Yes |
A and B give the same answers. If the data shows "yes, then no," you can't tell which one you're looking at.
C is the one shape that gives different answers. Feed rate and temperature don't move together, because nothing links them. But if you look only at days with the same energy use, then suddenly they do: on a hot day, feed rate must have been lower to land on that number. That only happens when two arrows point into the same variable. It's the only pattern in historical data that tells you which way an arrow goes.
Discovery software runs these tests (Figure 2) on every pair of variables, then keeps every graph consistent with the results2. What comes back is a set of multiple graphs, not just one.
Each graph will agree on which variables are connected and on every collider. Everything else is left undirected.
And all of that rests on three assumptions2:
- Every relationship in the data comes from the graph. Nothing moves together by accident.
- Every absence of a relationship is real, rather than two effects that happen to cancel out.
- Anything that drives two measured variables is itself measured.
In real plant data, the third assumption is the one that fails. There is almost always something driving your measured variables that nobody measured.
What's missing from the data#
So that plant manager from before hands their data to a model and gets an answer: feed rate costs this much energy per tonne. The model also reports how much of the energy it couldn't explain. It calls that part noise, and it treats it as random.
Written out, the energy equation now looks like this (assuming the noise is additive for simplicity):
The first part is what feed rate and temperature explain. The noise is the gap between that prediction and what the energy meter recorded. Treating it as random is fine as long as it has nothing to do with feed rate.
But think about what else changes how hard the mill has to work. Harder ore takes more energy to grind. And when the ore is hard, operators slow the feed to stop the mill choking. So ore hardness pushes energy up and feed rate down at the same time.
Nobody logs ore hardness. So where did it go? Into the noise. Write it in and the equation becomes:
Now look at what the model saw. On hard-ore days, energy was high and feed rate was low. The model doesn't know about the ore, so it reads that as feed rate mattering less than it does. The manager gets a confident number, but the number is wrong, and nothing in the output says so.
An unmeasured variable that drives two measured ones like this is a hidden confounder. The stronger its pull, the further the estimate drifts, and measuring it is what brings the estimate back (Figure 3).
Now there's a second problem, and it's in the feed rate column itself. What if the feed rate in the data isn't the feed rate? It's a flow meter reading, and the meter is off by a little every hour. Some of the movement in that column is the meter wobbling, not the feed changing. Energy doesn't respond to the wobble, so energy looks less responsive to the reading than it really is to feed rate, and the estimate comes out too small.
Both problems end the same way: the thing the model called noise turns out to be tangled up with the lever the manager is about to pull.
Side note: the smallest useful set of inputs. If energy depended only on feed rate and temperature, nothing else in the plant would improve a forecast once you knew those two. That idea generalizes. For any variable there's a smallest set of others that tells you everything the rest of the system can. It's the variable's direct causes plus a few close neighbors, and it's called the Markov blanket3. It's the set of inputs a good forecast should use.
If the notation looks cryptic, it's shorthand, and each piece maps onto the graph. MB stands for Markov blanket. PA(Y) is Y's parents, the variables with an arrow pointing into Y. CH(Y) is Y's children, the variables Y points into. PA(CH(Y)) is the other parents of those children. The ∪ sign means "together with" and the backslash means "leave out", so the first line reads: the blanket is the parents, the children and the children's other parents, minus Y itself.
In the plant, take energy as Y. Its parents are feed rate and ambient temperature. Say energy has one child, the electricity bill, which also depends on the tariff. The tariff is then a child's other parent, so energy's blanket is feed rate, ambient temperature, the bill and the tariff. On its own the tariff says nothing about energy, but once you know the bill it does: a high bill on a cheap tariff means a lot of energy. That's the collider pattern from Figure 2.
The second line borrows the double-bar symbol from the chain, fork and collider equations, where it means "unrelated to". The vertical bar means "given", and V is every variable in the system. Read together: given Y's blanket, nothing else in V tells you anything more about Y.
The flow meter is one case of a wider problem. Almost every variable a plant logs is a stand-in for the thing you actually care about.
When the patterns mislead#
Discovery finds patterns among the variables you give it, and industrial data reaches you through sensors, controllers and logging systems that bend those patterns. Stand-ins, control loops and logging intervals cause most of the trouble.
A temperature sensor stands in for temperature, a logged setpoint for valve position, a promotion flag for the promotion. Stand-ins distort discovery in two opposite ways, depending on how good they are.
A noisy stand-in only partly accounts for the thing it measures4. Say the lab does test ore hardness, once a shift, and the result is rough. Adjusting for the lab result removes some of the ore's influence but not all of it. What's left still ties feed rate to energy through the ore, so discovery can draw arrows that shouldn't exist, and the feed rate estimate stays confounded.
A precise stand-in has the opposite problem. A good temperature sensor carries almost everything about the real temperature, so discovery attaches the downstream arrows to the sensor. The graph says the sensor reading drives the outcome. As a summary of the data, that's fair. As advice it's wrong: pushing the reading up by recalibrating the sensor changes nothing in the plant.
Control loops hide the thing they're correcting for. In a building with a thermostat, a cold day pulls room temperature down and the heating cancels it out. In the data, room temperature barely moves with outside temperature, even though outside temperature is acting on the room the whole time (the gas bill shows it). This is the second assumption failing: two effects canceling out and leaving no trace (Figure 4).
How often the plant logs its data can hide which way an arrow points. A feedback loop that plays out over minutes, recorded as hourly averages, looks like two variables driving each other with no clear direction. Part of the graph you recover reflects how the data was logged, not how the plant works.
None of these problems show up in the graph itself. Each one does leave a trace somewhere else in the data, and that's what makes a graph testable.
Disproving a graph#
You can't prove a graph right, but you can catch it being wrong. Every arrow and every missing arrow is a claim you can test, and each problem above leaves a trace that one of these checks picks up.
- Check every pair against the data. For any two variables, the graph's paths say whether they should move together, and whether that should change once you account for others. This is called d-separation, and each of those predictions can be tested directly.
- Test the missing arrows on fresh data. Each missing arrow predicts that certain variables won't move together. Check that on data the search never saw.
- Rule out the impossible before searching. Effects don't come before causes, and some arrows break physics. Building these into the search removes whole sets of candidate graphs before any test runs.
- Score the graph against natural experiments. Setpoint changes, shutdowns, maintenance and policy changes are moments when someone actually changed something. The graph predicts what should have happened next, and you can check it. This is the most direct test of stand-ins and control loops.
- Check the graph holds across sites and seasons. A real cause-and-effect relationship should look the same across sites, seasons and shifts5. If the link between feed rate and energy shifts with context, something hidden in the noise is shifting too. This is where hidden confounders show up.
The checks catch different things. D-separation checks, held-out tests and physics rules work on the graph's structure. Natural experiments and stability checks go after what's hidden in the data: stand-ins, control loops and confounders the search couldn't see. A graph that passes all five hasn't been proved right, but every part of it has had a fair chance to fail.
Passing every check still leaves arrows the data can't settle. Whether those matter depends on the decision the graph is for.
What matters in practice#
You don't need the entire graph to be right. You need only the part that connects your decision to the outcome you care about.
Returning to the plant manager, the question is whether increasing feed rate will use more energy, and by how much. For this question, most of the graph is irrelevant. They need:
- The levers they control: feed rate, setpoints, schedules.
- The conditions they can't control but can measure: ambient temperature, ore grade, demand.
- The paths from both to energy, and what has to be accounted for to read them correctly.
That question is small enough to check whether the uncertainty in the graph matters. Take every graph the data allows, work out the effect of feed rate under each one, and look at the spread. If they all give the same answer, the arrows you couldn't settle don't affect this decision. If they disagree, you know which uncertainty is costing you and which experiment would settle it. This is the idea behind a method called IDA6, and Figure 5 runs it on a small plant model.
So how do we know the graph is right? We don't. We know which parts the data supports, which parts it rules out, and whether the parts we're unsure about change the answer to the question being asked.
One thing this post left alone is what's still in the noise once hidden causes and measurement error are taken out. That remainder is supposed to be random. In plant data it usually isn't. That's the next post.
References#
- 1Pearl, J. (2009). Causality: Models, Reasoning, and Inference, 2nd ed. Cambridge University Press. doi.org/10.1017/CBO9780511803161↩︎ Back
- 2Spirtes, P., Glymour, C. and Scheines, R. (2000). Causation, Prediction, and Search, 2nd ed. MIT Press.↩︎ Back↩︎ 2
- 3Peters, J., Janzing, D. and Schölkopf, B. (2017). Elements of Causal Inference: Foundations and Learning Algorithms. MIT Press.↩︎ Back
- 4Kuroki, M. and Pearl, J. (2014). Measurement bias and effect restoration in causal inference. Biometrika, 101(2), 423–437. doi.org/10.1093/biomet/ast066↩︎ Back
- 5Peters, J., Bühlmann, P. and Meinshausen, N. (2016). Causal inference by using invariant prediction: identification and confidence intervals. Journal of the Royal Statistical Society: Series B, 78(5), 947–1012. doi.org/10.1111/rssb.12167↩︎ Back
- 6Maathuis, M. H., Kalisch, M. and Bühlmann, P. (2009). Estimating high-dimensional intervention effects from observational data. Annals of Statistics, 37(6A), 3133–3164. doi.org/10.1214/09-AOS685↩︎ Back