Case study: In search of a lost bug – RevDeBug & Redgate Oracle Team

With over eight hundred thousand users Redgate Software is a market leader in database develops solutions, helping organizations satisfy their Compliant Database DevOps needs.

This case study describes how the Oracle tools development team at Redgate has successfully integrated RevDeBug into its development practices.

Redgate’s Source Control for Oracle tools is the cornerstone of Redgate’s Database DevOps offering, allowing customers to seamlessly manage their development databases in version control. It works by linking a development database with a version control repository and automatically translates detected changes into their constituent object creation scripts and migration scripts.

When errors occur, either as part of development or reported by a customer, they can be extremely tricky to reproduce, requiring access to a specific Oracle server instance and database.

One recent error was discovered by a client for whom the tool crashed each time they tried to view changes in one of his tables. In some specific rare conditions, the highlighting of the changes in SQL had its UI area calculated incorrectly and ran outside the bounds of the displayed SQL text, causing the UI element to crash. To make matters worse this error caused an immediate crash outside the managed code as it happened during the refresh of the whole UI component tree. It crashed not only the application but also .NET’s virtual machine without leaving a trace to aid the debugging process.

 

The crash RevDeBug has helped with

 

The root of this problem from a developers perspective

After building a new version of the application with an implementation of RevDeBug, the developer obtained a recording from the clients’ machine, revealing, just one unhandled exception:

 

RevDeBug recording from the client's machine

The RevDeBug recording contains all values, method calls and exceptions, which can be filtered.

 

It is possible to navigate to the code by drilling down into the exception.

 

RevDeBug value prompts with historical values

Values from the production recordings are annotated on the applicable code in Visual Studio

 

Armed with this data, developers quickly came to the conclusion that the problem lays in difference with the name of the database schemas.

The first port of call was to attempt to reproduce the bug in development, which can sometimes be the simplest way of debugging and diagnosing a problem, but in this case efforts were fruitless. Historically, the team would have spent a lot more time experimenting in the dark, sometimes days, in a desperate attempt to identify the root cause. Today a much more effective solution has been adopted, which leverages RevDeBug’s reverse debugging technology.

This works by using pre-built binaries of the tools in question, instrumented with RevDeBug. If a bug is found by a customer, they can themselves supply a complete recording of the sequence of events that led to the elusive crash. This provides crucial information allowing the developers to trace back the executed code from the moment the application crashed through every line of code and every variable value. With RevDeBug developers are able to identify and fix such issues far quicker than using traditional means, which benefits development efficiency and hence reducing the customer feedback loop.

“My team uses RevDeBug for all those elusive and hard to reproduce bugs – it speeds up the time to diagnose issues, allowing us to be much more responsive to our customers.”

David Atkinson, Head of Product Strategy, Redgate

Alice Gajos

By training production engineer, at heart .net programmer. Currently technology advocate at RevDeBug.