Explain how the 'Secure Desktop' mode in windows works?

As a quick primer to how Windows GUIs are built, basically everything runs through a function called CreateWindow() (I mean everything, every button, every menu, everything) and is given a hWnd or Window Handle. Modifying these Windows is done via another function, SendMessage().

Here's the catch. As a user mode application, making the right API calls I can fairly easily send messages to other Windows. It's fairly trivial to make buttons disappear from other people's forms. It is a little harder to perform DLL injection and hook the message loop that receives messages (the OS sends Windows messages when things happen to them) but not that much harder. If I can hook those events, I could automatically submit your "yes/no" form. Or, I could change the label from ReallyDodgyVirus.exe to explorer.exe and you'd be none the wiser.

Insert: A really good article on the various techniques of getting your code into the address space of a running process.

Now, what are KeePass doing?

A very brief perusal of the source shows they are using CreateDesktop(), SwitchDesktop() and CloseDesktop() to create a second desktop connected to the physical viewing device you're on. In English, they're asking the kernel to create for them an isolated desktop whose hWnd objects are outside of the findable range of any other application's SendMessage().

I should point out that SwitchDesktop suspends the updating of the UI of the default desktop. I'm not sure if the message loops are also frozen - I suspect not since the desktop is created as a new thread.

In this instance, KeePass is drawing the UI, so the execution is not, as I understand it, as NT AUTHORITY/SYSTEM. Instead, the new desktop is created in isolation from basically the rest of the current desktop, which protects it. I'll be happy to be corrected on that. However, see the MSDN for SwitchDesktop:

The SwitchDesktop function fails if the desktop belongs to an invisible window station. SwitchDesktop also fails when called from a process that is associated with a secured desktop such as the WinLogon and ScreenSaver desktops. Processes that are associated with a secured desktop include custom UserInit processes. Such calls typically fail with an "access denied" error.

I believe this means that these dialogs (screensavers, Windows Logon) are built more deeply into Windows such that they always execute as NT AUTHORITYSYSTEM and the UserInit process creates the sub processes on valid authentication at the required privilege level.

The reason I bring this up is because I believe there are two issues: different desktops and privilege separation. From Mark Russinovich's discussion of the topic of Secure Desktop:

The Windows Integrity Mechanism and UIPI were designed to create a protective barrier around elevated applications. One of its original goals was to prevent software developers from taking shortcuts and leveraging already-elevated applications to accomplish administrative tasks. An application running with standard user rights cannot send synthetic mouse or keyboard inputs into an elevated application to make it do its bidding or inject code into an elevated application to perform administrative operations.

As SteveS says, UAC runs a separate desktop process as NT AUTHORITY/SYSTEM.

Escalating privileges as a process I don't have the specifics of, but here is what I think I understand: I believe the process of privilege escalation in the Windows API causes a process running as NT AUTHORITY/SYSTEM (therefore able to execute the new process under whatever privileges it wants to, in this case an Administrator). When an application asks for higher privileges, that question is asked to you on a new desktop locally, to which none of your applications can get either the Desktop Handle or any of the GUI element handles. When you consent, consent.exe creates the process as the privileged user. Thus, the process running as NT AUTHORITYSYSTEM is a consequence of the need to create a new privileged process, not as a method of creating a secure desktop. The fact the desktop is different to the default is what adds security in both cases.

I believe what Mark means above is that, in addition to these secure desktops, two things are happening:

Your default administrator desktop is in fact running unprivileged, contrary to Windows XP and earlier and

Unprivileged and privileged applications now exist on separate desktops (disclaimer: could just be ACLs on the objects in memory, I'm not sure), ensuring that unprivileged code can't access privileged objects.

The Windows Logon UI is different again in Vista/7.

Clearly, none of these methods will defend you against kernel mode rootkits, but they do prevent privilege escalation and UI integrity compromise by isolating privileged applications, or in the case of KeePass, the sensitive dialog.

Edit

Having looked harder at the KeePass code, I saw this handy piece of C#:
Bitmap bmpBack = UIUtil.CreateScreenshot();
if(bmpBack != null) UIUtil.DimImage(bmpBack);
/* ... */
SecureThreadParams stp = new SecureThreadParams();
stp.BackgroundBitmap = bmpBack;
stp.ThreadDesktop = pNewDesktop;
From this you can see that in fact in order to mimic consent.exe, KeePass takes a screenshot of the background, dims it and creates its new desktop with the background of the old desktop. I therefore suspect the old desktop continues running even while it isn't being rendered. This I think confirms that no magic NT AUTHORITYSYSTEM action is happening both with KeePass and consent.exe (I suspect consent.exe is doing the same thing UI-wise, it just happens to be launched in the context of NT AUTHORITYSYSTEM).

Edit 2

When I say DLL Injection, I'm specifically thinking of DLL injection to corrupt the UI. DLL Injection remains possible on KeePass as a process, I'm just not sure whether it could be used to influence that secure UI. It could, however, be used to access the memory of the process and its threads, thereby grabbing the entered password pre-encryption. Hard, but I think it's possible. I'd appreciate someone advising on this if they know.