August 2014

Sun Mon Tue Wed Thu Fri Sat
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31            








« Autodesk Building Systems .NET API Webcast | Main | Breathing fresh life into LISP applications with a modern GUI »

July 28, 2006

Advanced Visual Studio debugging: how to stop stepping into certain functions

While debugging it's sometimes very frustrating to find yourself repeatedly stepping into an irrelevant function. For instance, complex statements pretty commonly include object constructors etc. that you know function perfectly well, but the debugger routinely takes you into them.

The Visual Studio IDE has an undocumented (and unsupported) mechanism to help with this. During the VC 6.0 timeframe it was implemented via our old friend the autoexp.dat file (see my previous post on this), in a special section called [ExecutionControl]. Since VC 7.0 this has been moved to the Registry.

For VC 7.0 and 7.1, it was in the HKCU (HKEY_CURRENT_USER) section:

HKCU\Software\Microsoft\VisualStudio\7.0\NativeDE\StepOver
HKCU\Software\Microsoft\VisualStudio\7.1\NativeDE\StepOver

For VC 8.0 (Visual Studio 2005), it shifted across to HKLM (HKEY_LOCAL_MACHINE):

HKLM\SOFTWARE\Microsoft\VisualStudio\8.0\NativeDE\StepOver

Inside this key you will need to create string values that specify the functions and class methods to exclude (or include) while stepping through code in the debugger. For VC7.0/7.1 you needed to use a numeric identifier for the name, but with VC8.0 you can thankfully use something more meaningful:

Name                Value
AcDbOP cons    AcDbObjectPointer\<.*\>\:\:AcDbObjectPointer.*=NoStepInto
App init              InitApplication=NoStepInto

[ Note: I would have used something even more meaningful than AcDbOP above, but I wanted the text to all fit on one line for clarity... :-) ]

The first part of the string value needs to be defined as a regular expression: the text I'm looking for in the first instance is really "AcDbObjectPointer<*>::AcDbObjectPointer*", but I had to use an escape character to prefix some of the symbols. Check MSDN for more information on regular expressions.

This is also the string as you would enter it via the Registry Editor - if you're using a .reg file then you'll need to double-up the slashes.

The second part of the string value is likely to be "=NoStepInto" or "=StepInto", depending on whether you are telling the debugger to exclude or include the locations defined by the expression. If you want to get really clever, you can use wildcards to exclude all the functions of a particular class, and then specifically re-include the ones that interest you. This CodeGuru forum post shows some good examples of that. I'd also recommend reading this entry from Andy Pennell's blog.

Incidentally, you don't have to restart Visual Studio for the changes to be picked up - both autoexp.dat and this section of the Registry are read in as a debug session is started (although the autoexp.dat timestamp needs to be more recent than the end of the last debugging session, interestingly enough... so if you make changes and save while a debug session is running, they won't be picked up in the next session unless a save is done between the sessions <phew!>).

When you come to execute your code, you should find that it no longer steps into the constructor of any class defined using the AcDbObjectPointer template (such as AcDbObjectPointer<AcDbViewportTableRecord>).

[ Side note: This template is one of the ObjectARX "smart pointers", and are a great way to handle the opening and closing of persistent ObjectARX objects. I won't go into detail here, but they are well worth checking out in the ObjectARX SDK documentation. ]

When I say "step into", I literally mean you would need to be stepping through the code using the debug toolbar or F11 - execution still stops in that particular function if you set a break-point there.

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d83452464869e200d83530ead153ef

Listed below are links to weblogs that reference Advanced Visual Studio debugging: how to stop stepping into certain functions:

blog comments powered by Disqus

Feed/Share

10 Random Posts