Fix a strange check in the auto DH codepath

Fix a strange check in the auto DH codepath

The code assumes that the server certificate has an RSA key and bases
the calculation of the size of the ephemeral DH key on this assumption.
So instead of checking whether we have any key by inspecting the dh
part of the union, let's check that we actually have an RSA key.
While here, make sure that its length is non-negative.

ok jsing