Through the Interface

Advanced Visual Studio debugging: how to stop stepping into certain functions

While debugging it's sometimes very frustrating to find yourself repeatedly stepping into an irrelevant function. For instance, complex statements pretty commonly include object constructors etc. that you know function perfectly well, but the debugger routinely takes you into them.

The Visual Studio IDE has an undocumented (and unsupported) mechanism to help with this. During the VC 6.0 timeframe it was implemented via our old friend the autoexp.dat file (see my previous post on this), in a special section called [ExecutionControl]. Since VC 7.0 this has been moved to the Registry.

For VC 7.0 and 7.1, it was in the HKCU (HKEY_CURRENT_USER) section:

HKCU\Software\Microsoft\VisualStudio\7.0\NativeDE\StepOver
HKCU\Software\Microsoft\VisualStudio\7.1\NativeDE\StepOver

For VC 8.0 (Visual Studio 2005), it shifted across to HKLM (HKEY_LOCAL_MACHINE):

HKLM\SOFTWARE\Microsoft\VisualStudio\8.0\NativeDE\StepOver

Inside this key you will need to create string values that specify the functions and class methods to exclude (or include) while stepping through code in the debugger. For VC7.0/7.1 you needed to use a numeric identifier for the name, but with VC8.0 you can thankfully use something more meaningful:

Name                Value
AcDbOP cons    AcDbObjectPointer\<.*\>\:\:AcDbObjectPointer.*=NoStepInto
App init              InitApplication=NoStepInto

[ Note: I would have used something even more meaningful than AcDbOP above, but I wanted the text to all fit on one line for clarity... :-) ]

The first part of the string value needs to be defined as a regular expression: the text I'm looking for in the first instance is really "AcDbObjectPointer<*>::AcDbObjectPointer*", but I had to use an escape character to prefix some of the symbols. Check MSDN for more information on regular expressions.

This is also the string as you would enter it via the Registry Editor - if you're using a .reg file then you'll need to double-up the slashes.

The second part of the string value is likely to be "=NoStepInto" or "=StepInto", depending on whether you are telling the debugger to exclude or include the locations defined by the expression. If you want to get really clever, you can use wildcards to exclude all the functions of a particular class, and then specifically re-include the ones that interest you. This CodeGuru forum post shows some good examples of that. I'd also recommend reading this entry from Andy Pennell's blog.

Incidentally, you don't have to restart Visual Studio for the changes to be picked up - both autoexp.dat and this section of the Registry are read in as a debug session is started (although the autoexp.dat timestamp needs to be more recent than the end of the last debugging session, interestingly enough... so if you make changes and save while a debug session is running, they won't be picked up in the next session unless a save is done between the sessions <phew!>).

When you come to execute your code, you should find that it no longer steps into the constructor of any class defined using the AcDbObjectPointer template (such as AcDbObjectPointer<AcDbViewportTableRecord>).

[ Side note: This template is one of the ObjectARX "smart pointers", and are a great way to handle the opening and closing of persistent ObjectARX objects. I won't go into detail here, but they are well worth checking out in the ObjectARX SDK documentation. ]

When I say "step into", I literally mean you would need to be stepping through the code using the debug toolbar or F11 - execution still stops in that particular function if you set a break-point there.

Advanced Visual Studio debugging: automatic expansion of watched variables

I've been using Visual C++ (and afterwards Visual Studio) since it was 16-bit, back in version 1.52. OK, maybe that's not so long ago, relatively (11 short years), but the point is that in spite of having followed the Visual Studio technology over this period, I've so far been completely unaware of the autoexp.dat file.

This feature of the Visual Studio was brought to my attention by Ahsan Ali, a programmer in the Inventor Engineering team who was based over in Bangalore at the same time I was (we had both previously worked in the US - he had come across from Tualatin while I had moved there from San Rafael). During a recent technology discussion, Ahsan shared some information with our Bangalore-based team on some advanced debugging techniques, and I thought it would be a great topic for this blog.

The autoexp.dat file's default location (for Visual Studio 2005) is:

C:\Program Files\Microsoft Visual Studio 8\Common7\Packages\Debugger\autoexp.dat

It's basically a text file that informs the Visual Studio IDE which information is especially relevant in a particular class, so that this information can be displayed automatically (and in the appropriate format) in the watch window, without you having to expand the various datatypes. The clicks you use to get down to data all add up, especially when dealing with a particularly repetitive debugging task, so this feature can really be helpful. MSDN refers to it here.

The technique seems to be specific for native (presumably C++?) code, but the above link refers to techniques to implement this type of functionality for VB.NET and C#.

So let's look at an example of the problem - as it relates to ObjectARX - and how this technique helps. Here's the default view of two very common ObjectARX datatypes, AcGePoint3d and AcGeMatrix3d:

The default view of the pickpnt variable (an AcGePoint3d) is OK, but all those zeroes do make your eyes hurt. The data displayed for xform (an AcGeMatrix3D) tells you nothing whatsoever of any use - it's the address of the 2D array holding the matrix contents. Here's what you get when you expand these two variables:

You need 5 clicks just to determine that the xform variable is actually the unit matrix, which is way too much.

So what can we do? The autoexp.dat rules format is very simple and is documented in the header of the file itself. I won't reproduce the whole description of the format here, aside from this brief statement:

An AutoExpand rule is a line with the name of a type, an equals sign, and text with replaceable parts in angle brackets. The part in angle brackets names a member of the type and an  optional Watch format specifier.

Thereafter follow some specifics regarding the syntax - for an alternative source of information regarding this, you might also try this CodeGuru article. I'd also recommend looking at the various types listed in the file itself - as usual real-life examples often paint better pictures than abstract descriptions.

So, let's look at what we can do for AcGePoint3d and AcGeMatrix3d. Here are the two entries I added to the autoexp.dat file:

AcGePoint3d =x=<x,g> y=<y,g> z=<z,g>

AcGeMatrix3d =<entry[0][0],g> <entry[0][1],g> <entry[0][2],g> <entry[0][3],g>,<entry[1][0],g> <entry[1][1],g> <entry[1][2],g> <entry[1][3],g>,<entry[2][0],g> <entry[2][1],g> <entry[2][2],g> <entry[2][3],g>,<entry[3][0],g> <entry[3][1],g> <entry[3][2],g> <entry[3][3],g>

The first is fairly simple: it simply tells Visual Studio to display each of the X, Y and Z values of the point, but using the "g" type specifier. Without going into details, this means that they are floating-point values that should be abbreviated to significant digits only. I chose to leave in the labels, x, y & z, to improve readability.

The second is clearly longer - once again it uses the "g" type specifier for each of the 16 entries in the 4x4 matrix. Given the volume of information in the matrix class, I decided to leave out the labels, simply listing the contents with each row separated each by a comma.

Here are the results:

When we have something more meaningful contained in the variables, the descriptions will get longer, of course:

A few tips about the entries in autoexp.dat:

• Don't just add them to the end of the file: they should be part of the [AutoExpand] section, not the [Visualizer] section. I placed mine at line 147.
• It seems that the string itself is limited to 256 characters to the right of the first equals sign (in this day and age - can you imagine?). I was lucky - the AcGeMatrix3d rule eventually came to exactly 256 characters (which is how I found out about the limitation - I had to squeeze out a few redundant spaces for it to work).
• I haven't been able to get multiple lines to display - which would be especially useful for the matrix class, of course. If someone can work it out or find the information on the internet, please post a comment!
• It would also be great to do conditional display of data, especially for union types such as the good old resbuf. While this is not supported directly, you can develop an AddIn to make it work, apparently. See this article for more details.

Debugging using Express Editions

Most of our desktop products support a "plug-in" model of development: you create a DLL (which may or may not be renamed with a number of extensions, such as DBX or ARX) which gets loaded into the calling executable's memory space. This allows the process to share memory with the loaded modules, improving performance over the more archaic IPC (inter-process communication)-based architectures.

[Here begins retracted information...]

While Visual C++ Express Edition supports debugging DLL projects using an external executable, Visual C# Express and Visual VB.NET Express Editions do not. When developing with our products it is extremely common to launch an external executable (for instance AutoCAD's executable, acad.exe), which in turn loads your class module/DLL into its memory space, allowing you to step through the code, watching the contents of variables and all the good stuff that comes with a professional debugging tool.

This essentially means these two tools (Visual C#/VB.NET Express Editions) are crippled when it comes to doing serious development work with a plug-in architecture such as AutoCAD's. Visual C++ Express Edition, on the other hand, apparently allows you to debug DLL projects fully.

[End of retracted information.]

OK - I'm adding some information to this post... the paragraphs above belonged to my initial post, and aren't strictly true. Thanks to Ray Mendoza for raising this issue.

While Visual Basic 2005 (and presumably Visual C#) Express Edition does not make it easy for you to set up and use an external application to debug a class library, it is possible to do. Here's the initial message you get, when you simply try to debug a class library with no addition project set-up performed:

While at first you think "OK, great - let me just add in acad.exe and have it as the startup object", when you come to modify the project settings, there's no easy way to do it. You may be able to add in a separate EXE project to debug, but we don't have or need that for a pre-built EXE such as AutoCAD.

So what can we do? Well, it turns out that all you need to enable debugging in Visual Basic Express Edition is a separate file in the project folder, called "MyProjectName.vbproj.user". This contains the user-specific project settings for the MyProjectName project. In my test I used the default project name for a Class Library, and so the file was called "ClassLibrary1.vbproj.user" and in the same folder as "ClassLibrary1.vbproj". Here are the contents of that file:

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
    <StartAction>Program</StartAction>
    <StartProgram>C:\Program Files\AutoCAD 2007\acad.exe</StartProgram>
  </PropertyGroup>
</Project>

After creating this file (please make sure the executable path is correct for your system, of course), you will need to reopen the project for it to be detected and read. But once it's found, you can debug your class library project with an external executable, just by hitting F5. Once AutoCAD is loaded, NETLOAD your DLL and away you go.

This file does get created automatically using the Autodesk Managed AppWizard (part of the ObjectARX Wizard - see the previous post for more information).

Now - if someone out there knows an easier way to set this up directly in VB Express, please do let me know, by email or by adding a comment to this post. This is the first time I've used VB Express myself, so there may well be something obvious that I've missed.