From: jsing <jsing@openbsd.org>
Date: Wed, 30 Nov 2022 02:52:25 +0000 (+0000)
Subject: Fix return values bug in BN_ucmp().
X-Git-Url: http://artulab.com/gitweb/?a=commitdiff_plain;h=f38a8e3eb6df96ad80d3272d958185b711965eba;p=openbsd

Fix return values bug in BN_ucmp().

BN_ucmp() is supposed to return -1/0/1 on a < b, a == b and a > b, however
it currently returns other negative and positive values when the top of
a and b differ. Correct this.

ok tb@
---

diff --git a/lib/libcrypto/bn/bn_lib.c b/lib/libcrypto/bn/bn_lib.c
index 0ac3977c195..df43da5db66 100644
--- a/lib/libcrypto/bn/bn_lib.c
+++ b/lib/libcrypto/bn/bn_lib.c
@@ -1,4 +1,4 @@
-/* $OpenBSD: bn_lib.c,v 1.64 2022/11/30 01:47:19 jsing Exp $ */
+/* $OpenBSD: bn_lib.c,v 1.65 2022/11/30 02:52:25 jsing Exp $ */
 /* Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com)
  * All rights reserved.
  *
@@ -691,9 +691,11 @@ BN_ucmp(const BIGNUM *a, const BIGNUM *b)
 	BN_ULONG t1, t2, *ap, *bp;
 
 
-	i = a->top - b->top;
-	if (i != 0)
-		return (i);
+	if (a->top < b->top)
+		return -1;
+	if (a->top > b->top)
+		return 1;
+
 	ap = a->d;
 	bp = b->d;
 	for (i = a->top - 1; i >= 0; i--) {