X509_check_ca() has 5 return values but still can't fail

The values 0, 1, 3, 4, 5 all have some meaning, none of which is failure.
If caching of X509v3 extensions fails, returning X509_V_ERR_UNSPECIFIED,
i.e., 1 is a bad idea since that means the cert is a CA with appropriate
basic constraints. Revert to OpenSSL behavior which is to ignore failure
to cache extensions at the risk of reporting lies.

Since no return value can indicate failure, we can't fix this in
X509_check_ca() itself. Application code will have to call (and check)
the magic X509_check_purpose(x, -1, -1) to ensure extensions are cached,
then X509_check_ca() can't lie.

ok jsing

diff --git a/lib/libcrypto/x509/x509_purp.c b/lib/libcrypto/x509/x509_purp.c
index ffd986b..ab5e7cb 100644 (file)
--- a/lib/libcrypto/x509/x509_purp.c
+++ b/lib/libcrypto/x509/x509_purp.c
@@ -1,4 +1,4 @@
-/* $OpenBSD: x509_purp.c,v 1.15 2022/04/21 04:48:12 tb Exp $ */
+/* $OpenBSD: x509_purp.c,v 1.16 2022/05/10 19:42:52 tb Exp $ */
 /* Written by Dr Stephen N Henson (steve@openssl.org) for the OpenSSL
  * project 2001.
  */
@@ -671,8 +671,6 @@ X509_check_ca(X509 *x)
                CRYPTO_w_lock(CRYPTO_LOCK_X509);
                x509v3_cache_extensions(x);
                CRYPTO_w_unlock(CRYPTO_LOCK_X509);
-               if (x->ex_flags & EXFLAG_INVALID)
-                       return X509_V_ERR_UNSPECIFIED;
        }
 
        return check_ca(x);