serial-bench / CAN / msg #8
The segment had run for three years. A cabinet was re-wired over a weekend, and on Monday the two nodes at the far end started logging error frames - but only when the payload was long. Short status frames went through all day.
That asymmetry is what made it take me an afternoon. A bus that is simply broken is easy. A bus that works for six-bit answers and fails for eight-byte ones feels like a software problem, and I spent the first hour reading somebody else's driver instead of putting a probe on the pair.
Ninety-five metres of 120 Ω twisted pair, daisy-chained through seventeen nodes, 500 kbit/s, so a bit cell is 2 000 ns. Stubs into each node are short - I measured the longest at 0.6 m. Nominally there is one 120 Ω resistor at each physical end of the trunk and nothing anywhere else. After the re-wire there was one at the near end, and at the far end there was a neat screw terminal with no resistor in it. The second terminator had been moved to a node roughly in the middle of the chain, where it does almost nothing useful.
Cable delay on this type of pair is close to 5 ns/m, which I take as given rather than measured. Ninety-five metres is therefore about 475 ns in one direction and 950 ns there and back - just under half a bit cell. That number is the whole story.
I put the differential probe across CANH and CANL at the last node and triggered on a recessive-to-dominant edge, 500 ns/div. The top trace below is the segment with both terminators in place. The bottom trace is the same node, same frame, with the far-end resistor removed. An open end reflects with a coefficient of +1, so the incident edge comes back at full amplitude and adds to whatever the driver is doing.
The overshoot by itself is ugly but harmless: the receiver reads dominant, and dominant is what the driver meant. The dangerous part is the second crossing. At 950 ns the reflected edge drags the differential voltage below 0.9 V for about 120 ns, and that looks to the receiver like an edge that nobody transmitted.
Every extra edge is an opportunity for the receiver to resynchronise on the wrong thing, and an eight-byte data frame simply contains far more edges than a one-byte status frame - more payload transitions, and more stuff bits inserted around them. So the probability of at least one bad sample per frame scales with frame length, which is exactly the pattern the operators described without knowing why.
I want to be careful here. The ghost crossing is in the capture and I can reproduce it by pulling and replacing the resistor. What each controller does internally with that crossing I cannot observe from outside, so I will not claim a precise chain of events inside the synchroniser. The useful statement is weaker and still enough: the physical layer presents an edge that was never transmitted, and frames with more edges lose more often.
I logged receive error counters at the far node over ten minutes in both states, driving the bus with a fixed mix of frames.
| Frame payload | Both terminators | Far end open |
|---|---|---|
| 1 byte | 0 | 0 |
| 4 bytes | 0 | 3 |
| 8 bytes | 0 | 14 |
| 8 bytes, 0x55 pattern | 0 | 31 |
The last row is the one I would show a sceptic. A payload of alternating bits has an edge in almost every bit cell and no stuff bits at all, and it is more than twice as bad as the mixed eight-byte case. Nothing about software changed between any two rows.
Fix took four minutes: a 120 Ω, 1 % resistor back into the far-end terminal, the stray one removed from the middle node. The ghost crossing disappeared, the counters went to zero and stayed there through the following week.
Two things. First, round-trip delay is a number worth writing on the inside of the cabinet door: at 500 kbit/s a 95 m segment reflects back in half a bit, and anything that reflects is visible to every receiver. Second, when a fault correlates with payload length, suspect the copper before the code. Length correlates with edge count, and edge count is a physical-layer quantity.