From 0334e08b415550dd4e3ca9f961ff47ffdb00dc26 Mon Sep 17 00:00:00 2001 From: schwarze Date: Sun, 20 Nov 2022 22:23:43 +0000 Subject: [PATCH] Fix an off-by-one bug in BN_GF2m_poly2arr(3). If the last argument, the size of the output array, is too small to contain all degrees present in the input polynomial plus one for the terminating -1, the function is documented to return the size of the output array that would be needed (in comments in the source code, in the new manual page, and by the way how the function is used by other functions in the same file). However, in case of overflow, the existing code failed to include the element needed for the terminating -1 in the return value, wrongly indicating success if everything but the -1 did fit and reporting failure with a size that was still too small otherwise. According to tb@, this is very unlikely to cause vulnerabilities in practical applications because there is no real reason to pick a reducing polynomial longer than a pentanomial, because all known callers use either fixed size arrays of size 6 or dynamic allocation, because use of GF(2^m) is rare in practice, and GF(2^m) with custom reducing polynomials even more so. OK tb@ --- lib/libcrypto/bn/bn_gf2m.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/lib/libcrypto/bn/bn_gf2m.c b/lib/libcrypto/bn/bn_gf2m.c index 8562b3f87e2..1fd7105a313 100644 --- a/lib/libcrypto/bn/bn_gf2m.c +++ b/lib/libcrypto/bn/bn_gf2m.c @@ -1,4 +1,4 @@ -/* $OpenBSD: bn_gf2m.c,v 1.23 2017/01/29 17:49:22 beck Exp $ */ +/* $OpenBSD: bn_gf2m.c,v 1.24 2022/11/20 22:23:43 schwarze Exp $ */ /* ==================================================================== * Copyright 2002 Sun Microsystems, Inc. ALL RIGHTS RESERVED. * @@ -1291,10 +1291,9 @@ BN_GF2m_poly2arr(const BIGNUM *a, int p[], int max) } } - if (k < max) { + if (k < max) p[k] = -1; - k++; - } + k++; return k; } -- 2.20.1