From 091f637a5964825bc94bebd06624b0aea21eb127 Mon Sep 17 00:00:00 2001 From: tb Date: Sat, 16 Mar 2024 20:42:33 +0000 Subject: [PATCH] Fix signed integer overflow in bnrand() If more bits than INT_MAX - 7 are requested, the calculation of number of bytes required to store the bignum triggers undefined behavior due to signed integer overflow. This will typically result in bytes becoming negative which will then make malloc() fail. If the ulimit should be high enough to make malloc() succeed, there is a bad out of bounds write in case bottom is set (an odd number was requested). On jsing's request this does not deal with another bug which we could catch with a similar check due to BN_bn2bin() failing later on as the number of words in a BIGNUM is some fraction of INT_MAX. ok jsing --- lib/libcrypto/bn/bn_rand.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/lib/libcrypto/bn/bn_rand.c b/lib/libcrypto/bn/bn_rand.c index a5b163c8202..9cfcd8e2c00 100644 --- a/lib/libcrypto/bn/bn_rand.c +++ b/lib/libcrypto/bn/bn_rand.c @@ -1,4 +1,4 @@ -/* $OpenBSD: bn_rand.c,v 1.29 2023/08/03 18:53:55 tb Exp $ */ +/* $OpenBSD: bn_rand.c,v 1.30 2024/03/16 20:42:33 tb Exp $ */ /* Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com) * All rights reserved. * @@ -109,6 +109,7 @@ * */ +#include #include #include #include @@ -133,6 +134,10 @@ bnrand(int pseudorand, BIGNUM *rnd, int bits, int top, int bottom) BNerror(BN_R_BITS_TOO_SMALL); return (0); } + if (bits > INT_MAX - 7) { + BNerror(BN_R_BIGNUM_TOO_LONG); + return (0); + } if (bits == 0) { BN_zero(rnd); -- 2.20.1