Correctly detect b < a in BN_usub().

BN_usub() requires that a >= b and should return an error in the case that
b < a. This is currently only detected by checking the number of words in
a versus b - if they have the same number of words, the top word is not
checked and b < a, which then succeeds and produces an incorrect result.

Fix this by checking for the case where a and b have an equal number of
words, yet there is a borrow returned from bn_sub_words().

ok miod@ tb@

diff --git a/lib/libcrypto/bn/bn_add.c b/lib/libcrypto/bn/bn_add.c
index c5bc024..cfc04fd 100644 (file)
--- a/lib/libcrypto/bn/bn_add.c
+++ b/lib/libcrypto/bn/bn_add.c
@@ -1,4 +1,4 @@
-/* $OpenBSD: bn_add.c,v 1.19 2023/01/23 10:34:21 jsing Exp $ */
+/* $OpenBSD: bn_add.c,v 1.20 2023/01/31 05:16:52 jsing Exp $ */
 /* Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com)
  * All rights reserved.
  *
@@ -291,6 +291,10 @@ BN_usub(BIGNUM *r, const BIGNUM *a, const BIGNUM *b)
        rp = r->d;
 
        borrow = bn_sub_words(rp, ap, bp, min);
+       if (dif == 0 && borrow > 0) {
+               BNerror(BN_R_ARG2_LT_ARG3);
+               return 0;
+       }
        ap += min;
        rp += min;
 

diff --git a/lib/libcrypto/man/BN_add.3 b/lib/libcrypto/man/BN_add.3
index c875147..a06b8af 100644 (file)
--- a/lib/libcrypto/man/BN_add.3
+++ b/lib/libcrypto/man/BN_add.3
@@ -1,4 +1,4 @@
-.\" $OpenBSD: BN_add.3,v 1.17 2022/11/16 14:19:22 schwarze Exp $
+.\" $OpenBSD: BN_add.3,v 1.18 2023/01/31 05:16:52 jsing Exp $
 .\" full merge up to: OpenSSL e9b77246 Jan 20 19:58:49 2017 +0100
 .\"
 .\" This file is a derived work.
@@ -66,7 +66,7 @@
 .\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
 .\" OF THE POSSIBILITY OF SUCH DAMAGE.
 .\"
-.Dd $Mdocdate: November 16 2022 $
+.Dd $Mdocdate: January 31 2023 $
 .Dt BN_ADD 3
 .Os
 .Sh NAME
@@ -318,8 +318,7 @@ It requires the absolute value of
 .Fa a
 to be greater than the absolute value of
 .Fa b ;
-otherwise, it will sometimes fail
-and sometimes silently produce wrong results.
+otherwise it will fail.
 .Fa r
 may be the same
 .Vt BIGNUM