hello@dothex.net

+44 (0) 20 3290 3080

London, UK

FREE CONSULTATION

Embedded Systems & Product Development Consultancy · London, UK

Embedded debugging and troubleshooting

Faults that won’t show up on the bench

The hardest faults are the ones nobody can trigger on demand. They appear at a customer site, in the climate chamber or after days of uptime, and the logs say nothing useful.

  • Devices that reset, freeze or drop off the network a few times a week.
  • Behaviour that changes at low temperature, on a low battery or with a noisy supply.
  • A radio link that works in the office and fails at the customer’s site.
  • A failure at the test house, with the launch date already fixed.
  • Units returned as “no fault found” that fail again once they’re back in service.

Each of these has a cause that can be measured. Finding it takes method, the right instruments and an engineer who reads the code and the schematic together.

See how we work

What we fix

Firmware, hardware or the space between them. We go where the evidence points.

Resets, lock-ups and crashes.

Hard faults, watchdog resets, stack overflows and memory corruption traced to the line of code or the event behind them.

Environmental failures

Faults that appear at low or high temperature, in humidity or under vibration during environmental testing.

EMC and certification failures

Emissions and immunity problems found at the test house, diagnosed and fixed in hardware, firmware or both.

How we find the root cause

Guessing is expensive. We follow the evidence and change one thing at a time.

  1. Reproduce it. Recreate the failure under the same temperature, supply and load conditions, or from captured field logs, so every fix can be proven.
  2. Instrument it. Add tracing, logging and test points, and capture what the firmware and the hardware are actually doing at the moment it fails.
  3. Isolate the cause. Narrow it down one variable at a time until a single cause explains every symptom.
  4. Fix it properly. Correct the cause in code, hardware or both, rather than adding a delay, a retry or a reset that hides it.
  5. Prove it and write it up. Show the fault is gone under the original conditions, add a test so it stays gone, and record what happened and why.

Instruments and debug tools

Instruments: oscilloscopes, logic analysers, battery analyser, current-limited bench supplies, BLE and Wi-Fi diagnostic tools

Debug: J-Link, ST-Link, SWD and JTAG, IAR, Keil, MCUXpresso, MPLAB X, Eclipse with GCC

Analysis: RTOS-aware debugging, trace logging, LDRA static analysis and unit testing, Python scripts for log and protocol analysis

Platforms: STM32, NXP i.MX and i.MX RT, Nordic nRF52, ESP32, Silicon Labs EFR32, Microchip PIC and SAM, embedded Linux

Fixes in shipped products

Client names stay private. The engineering doesn’t.

Root-causing a Wi-Fi connection issue in a marine transponder already on the market

An ESP32 Wi-Fi fault traced and fixed in half the time allowed, with throughput improved for multiple connected devices.

Read the case study

An ATM security device, from schematic to 40,000 units in 24/7 service

Noise from older ATM power supplies diagnosed and designed out, with hardware and firmware issues solved at the CE test house and across 40,000 devices in service.

Read the case study

More debugging work

View case studies

Frequently asked debugging questions

Can you help with a product that’s already in the field?

Yes, and much of our debugging work is on products that have already shipped. We work from field logs, returned units and your bug reports, and plan the fix around your update process.

How quickly can you start?

We reply within one business day and can usually begin an urgent investigation within a few days. A short call and a description of the symptoms are enough to get going.

The original developer has left. Can you still help?

es. We regularly work on code and boards we didn’t write or design. We build it, measure it and document what we learn as we go.

What do you need from us?

A few units, the firmware source and build setup, schematics and anything you know about when the fault happens. We’re happy to sign an NDA first.

Do you fix the problem or just report it?

Whichever suits you. We can implement and test the fix ourselves, or hand over a root-cause report with a recommended fix for your team.

Can you help us pass EMC or environmental testing?

Yes. We’ve solved faults at the CE test house and during environmental testing down to −25 °C. We work out whether hardware, firmware or both need to change and support the retest.

Have a new board on the bench, or a system that won’t behave?

Talk to an engineer

Latest from the blog