Sounds like you are overloading the power supply.
Fans usually use the +12V connection, but a few use a lower voltage. First, check it is the correct spec.
Next, check whether you are running the graphics card and fan from the same PSU output. Sometimes a PSU will have more than one output circuit and changing to a different output connector can ease things.
Otherwise, for now just disconnect the case fan and controller and run without it.
Whether you really need the case fan or not depends on how hot things like the CPU and graphics card get without it. Even if it kinda runs OK, these things will last longer if you can cool them that bit more. Your system may well have some temperature sensors and software to monitor them and set upper limits. The old rule of thumb used to be 100 deg C max but it's worth googling around to find out more. If necessary, the short-term remedy is to leave the case open (I have one old PC permanently in this sorry state).
For a proper fix I would consider upgrading to a more powerful PSU. Even if you get things kinda running OK it is probably too close to its limits for comfort. It's worth digging through the small print on both old and new, to make sure that the overloaded output gets a larger Amp (A) rating.
"Klinger, do you know how many zoots were killed to make that one suit?" — BJ Hunnicutt, 4077 M*A*S*H