Remove a bogus memcmp in range_should_be_prefix()

range_should_be_prefix() currently always fails. The reason for this
is that OpenSSL commit 42d7d7dd incorrectly moved a memcmp() out of
an assertion.  As a consequence, the library emits and accepts
incorrectly encoded ipAddrBlock extensions since it will never detect
ranges that MUST be encoded as a prefix according to RFC 3779, 2.2.3.7.

The return -1 from this memcmp() indicates to the callers that the
range should be expressed as a range, so callers must check beforehand
that min <= max to be able to fail. Thus, remove this memcmp() and
add a check to make_addressRange(), the only caller that didn't already
ensure that min <= max.

This fixes the noisy output in regress/lib/libcrypto/x509/rfc3779.

ok inoguchi jsing

diff --git a/lib/libcrypto/x509/x509_addr.c b/lib/libcrypto/x509/x509_addr.c
index 705fc7d..c6eac91 100644 (file)
--- a/lib/libcrypto/x509/x509_addr.c
+++ b/lib/libcrypto/x509/x509_addr.c
@@ -1,4 +1,4 @@
-/*     $OpenBSD: x509_addr.c,v 1.61 2022/01/05 07:37:01 tb Exp $ */
+/*     $OpenBSD: x509_addr.c,v 1.62 2022/01/05 07:47:15 tb Exp $ */
 /*
  * Contributed to the OpenSSL Project by the American Registry for
  * Internet Numbers ("ARIN").
@@ -755,6 +755,8 @@ v6IPAddressOrRange_cmp(const IPAddressOrRange *const *a,
 /*
  * Calculate whether a range collapses to a prefix.
  * See last paragraph of RFC 3779 2.2.3.7.
+ *
+ * It's the caller's responsibility to ensure that min <= max.
  */
 static int
 range_should_be_prefix(const unsigned char *min, const unsigned char *max,
@@ -763,8 +765,6 @@ range_should_be_prefix(const unsigned char *min, const unsigned char *max,
        unsigned char mask;
        int i, j;
 
-       if (memcmp(min, max, length) <= 0)
-               return -1;
        for (i = 0; i < length && min[i] == max[i]; i++)
                continue;
        for (j = length - 1; j >= 0 && min[j] == 0x00 && max[j] == 0xff; j--)
@@ -863,6 +863,9 @@ make_addressRange(IPAddressOrRange **result, unsigned char *min,
        IPAddressOrRange *aor;
        int i, prefix_len;
 
+       if (memcmp(min, max, length) > 0)
+               return 0;
+
        if ((prefix_len = range_should_be_prefix(min, max, length)) >= 0)
                return make_addressPrefix(result, min, afi, prefix_len);