I won't go through the week of poking and testing to identify this issue, I'll just cut straight to what I've found;

1) when I launch a game, Nvidia takes over the display (I assume). The screen goes black for a second, the game loads... when I close the game the screen goes black for a second, I get the HDMI tag at top left, my window comes back. Since about the 12th of April (Windows updates, new Nvidia display drivers) when I open a game it runs fine, then when I close the game my entire system gets very stuttery, browser pages are extremely slow to open (Firefox, Chrome is effected too, just not quite as bad), opening any other process is slow. Checking my system processes shows hardly any usage of RAM or CPU, just everything is slow and jittery.

2) I tested lots of things but the only thing that has worked (so far) is to remove DisplayFusion. Not uninstall, just go into msconfig and remove it from the startup list. Did that, no game issues. Put it back on and rebooted, problem was back.

I don't know what information you might need about this, but for now DisplayFusion is off the startup list.