This solution is specific to the Microsoft Visual Studio 6.0 debugger on Windows. For other configurations, see the Debugging section of Tech Note 1605, MEX-files Guide:
<http://www.mathworks.com/support/tech-notes/1600/1605.html>
The following steps detail how to debug a MEX-file in the Microsoft Visual Studio 6.0 environment:
1. Build the library using "mex -g filename.c" from the MATLAB command prompt.
2. Open Microsoft Visual Studio 6.0.
3. Create a new "empty" project of type "Win32 Console Application".
4. Select Project -> Settings, or right-click on the project to go to the Settings window.
5. Select the "Debug" tab.
6. Under "Executable for Debug Session", enter the full path to matlab.exe (in the bin\win32 directory).
7. Select "Additional DLLs" from the "Category" drop-down.
8. Add the name of the DLL/MEX-file you want to debug.
9. Open the source file.
10. Select "Run", or "Go". Click NO if prompted to build or rebuild the project.
This should bring up the MATLAB environment.
11. In Visual Studio, set breakpoints in the source file.
12. In the MATLAB environment, change to the directory location of the MEX-file using the CD command.
13. Execute the call to the MEX-file you want to debug. Execution will stop at the break point in Microsoft Visual Studio 6.0.
NOTE: These steps are not directly supported by MathWorks. Refer to the Microsoft Visual Studio 6.0 manual for more information on debugging.
