Earlier this year I suffered from a case of a Visual Studio properties window that appeared to get “stuck” in the middle of developing an application. It would show up blank and the only way to get it to work was to fix the project and restart Visual Studio (not just close and reopen the solution). For more details, please read the first article.

Although I haven’t written about it, after applying the solution from the first article, my problems weren’t entirely over. Every once in a while, the properties window would get stuck again, and after tweaking the .xsd file a little or reverting it to a previous revision, I could get it to work again. But I never got the feeling that I really understood what I was doing. It would just work sometimes, or not. I even got hesitant to make any changes to the dataset, unless absolutely necessary. But it only happened on that single solution.

Until last week. In an entirely different solution, all of a sudden the properties window was blank again. It’s subtle, you don’t always notice immediately and because of that I could not simply undo the last the thing I did. This solution contained a dataset file too, so after cursing a little (I admit) the first thing I did was revert that and reapply the changes one by one. But this time, that didn’t work. I reverted the dataset to points way before the property editor got broken but still nothing. It was starting to look like the dataset file being the solution was not a generic one, but just the solution for that first project. This time, it was something else.

Finding out where the trouble starts

Instead of just mucking around with the dataset file, I reverted the whole project to the previous revision and started reapplying the changes from the latest revision on a file-by-file basis where possible. I was fortunate this was possible because the changes were fairly segmented. After every change, I built the project and tested if the property editor would still work as expected. A couple of hours later, I managed to break it again, but this time because I was paying attention I knew exactly when. I had a UserControl that grouped a couple of editors, and a form that used that UserControl. The moment I switched from code view to design view of the Form, the property editor was dead. I now learned something new: exact steps to reproduce the error’s first occurance. I could close and reopen Visual Studio and do it all over again every time.

Okay, so obviously, usercontrols are compiled and semi-running in Visual Studio the moment you add them to a form, so I must have a bug in my usercontrol that caused a crash in Visual Studio. But what? Remember, the program would build and run absolutely fine, without even a compiler warning. What could be wrong with it?

Debugging Visual Studio

The following morning (really, a good night’s sleep helps!) I got an idea. What would I have done if a seamingly invisible error occurred in my own project…? Why not try the same here? So I fired up two instances of Visual Studio. The first loaded the project, built it, but I would stop right before switching from code to design view on the form that contained the user control. I switched to the second VS (no solution open) and used Debug / Attach to process. Behind Attach to, clicked Select, set it to Automatically determine the type of code to debug. I then chose the first VS instance from the list of Available Processes (devenv.exe) and clicked Attach.

What happened was exactly what I hoped for. Switched back to VS instance number one, to the form file, switch from code to design view, and bam! The second VS window started flashing: it had intercepted the exception in the first. Whoohoo. It got even better: because the “bug” was in user code (my user control), it had already opened the matching source file and showed me the exact point where things went wrong:

Because I know knew what was causing the property editor to crash, I could fix it, and get it to work again (after a VS restart). Before, I was in the dark — hey, you try finding a bug in your solution without knowing where it is or what error it causes…! For those of you that want to know what was wrong specifically with my project, read on below (it turned out a small known issue in Visual Studio that hit me), but the important thing is that I now have learned a method I can use if this ever happens again.

Signs of trouble

You know when you’re in trouble when…

1. Property editor stays blank when you select an object in form design view. Only after focusing the property editor and pressing Esc, the properties are shown. You need to do this every time you select another object, or it will keep the previous object selected.
2. The dropdownbox above the property editor does not contain items.
3. The Document Outline does not update if you add, move or remove a control on a form.
4. The Dataset Designer shows “Please wait while loading” forever. (This is what made me think the dataset file was causing all this.)
5. The above conditions, once they have first occurred, will remain until you restart Visual Studio. Just closing the offending solution and opening another is not enough.

To try to fix this

1. Close all files, especially designer files
2. Close the solution (if you skip step 1, you might break it as soon as you perform step 4, because it will reopen the files you had open last time).
3. Close Visual Studio and restart it
4. Reopen the offending the solution but do not open any files.
5. Fire up a second Visual Studio
6. Menu Debug, Attach to Process
7. Attach to: Automatically select the type of code to debug
8. Process: the devenv.exe instance of the first Visual Studio
9. Click Attach
10. Switch back to VS instance 1 and start opening files and switching back and forth to designers until VS instance 2 intercepts the exception.
11. See in VS instance 2 which error occurs and where.
12. Stop the VS 2 debugger, so VS 1 can continue running.
13. Go to VS instance 1 and fix it.
14. Rebuild the project.
15. Close and reopen VS instance 1 and test if this solves it. If not, repeat steps.

Now back to my specific error. As it turns out:

Visual Studio issue: DesignMode property is sometimes wrong

The bug I bumped into was the fact that the Control.DesignMode property is sometimes wrong. There’s even a Microsoft Connect page on it.

I was using the great Action Library by Cradle77 (Marco De Sanctis). I was used to the concept of Actions coming from the Delphi world and frankly, I have a hard time understanding how a VS developer doing a WinForms project can do without it 😉 The idea is that you use a central point (the action list’s Update event handler) where you can set which actions are checked, visible and/or enabled or not and one (Execute) event handler per action to determine what it does. After that, you only need to link your menu items, tool strip buttons, context menu items, buttons etc. etc. to an action and your action state and execution is managed centrally without extra coding.

In this Update event handler I was hiding certain actions from a user that had no authorization (pseudo):

private void aclItemList_Update(object sender, EventArgs e) {
actAddItem.Visible = Program.Context.CurrentUser.CanEditItems;
actEditItem.Visible = Program.Context.CurrentUser.CanEditItems;

It error-ed out on the 2nd line giving a null reference exception. Well duh, of course Program.Context is null at designtime. Program.Context is a static property that I set before I call Application.Run() in Program.cs, so a null check is redundant.  But why is this code run in the designer anyway!? The action list update event handlers are executed when the application is running idle, never at design time and it uses the DesignMode property to make sure. But, as I found out here, this property does not always work correctly when controls are nested. And this specifc action list was in a user control, that was in a form.

So I could have replaced the design time check with the workaround that is suggested in the stackoverflow thread, but I chose to simply test for Program.Context being null as the first line in the event handler and returning without doing anything if that’s the case.

private void aclItemList_Update(object sender, EventArgs e) {
if (Program.Context == null) return; // can be null at design-time...
actAddItem.Visible = Program.Context.CurrentUser.CanEditItems;

This also perfectly explained why the project compiled and ran absolutely fine. This could only go wrong at design time. I recompiled, restarted Visual Studio and my troubles were over…