From: tb <tb@openbsd.org>
Date: Thu, 20 Jul 2023 06:26:27 +0000 (+0000)
Subject: Cap the size of numbers we check for primality
X-Git-Url: http://artulab.com/gitweb/?a=commitdiff_plain;h=050d837a6cf064b603a4b238874fb149634381d3;p=openbsd

Cap the size of numbers we check for primality

We refuse to generate RSA keys larger than 16k and DH keys larger than 10k.
Primality checking with adversarial input is a DoS vector, so simply don't
do this. Introduce a cap of 32k for numbers we try to test for primality,
which should be more than large enough for use withing a non-toolkit crypto
library. This is one way of mitigating the DH_check()/EVP_PKEY_param_check()
issue.

ok jsing miod
---

diff --git a/lib/libcrypto/bn/bn_prime.c b/lib/libcrypto/bn/bn_prime.c
index a09bac4ae9e..5a4aa50bf10 100644
--- a/lib/libcrypto/bn/bn_prime.c
+++ b/lib/libcrypto/bn/bn_prime.c
@@ -1,4 +1,4 @@
-/* $OpenBSD: bn_prime.c,v 1.33 2023/07/08 12:21:58 beck Exp $ */
+/* $OpenBSD: bn_prime.c,v 1.34 2023/07/20 06:26:27 tb Exp $ */
 /* Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com)
  * All rights reserved.
  *
@@ -240,6 +240,8 @@ BN_is_prime_ex(const BIGNUM *a, int checks, BN_CTX *ctx_passed, BN_GENCB *cb)
 }
 LCRYPTO_ALIAS(BN_is_prime_ex);
 
+#define BN_PRIME_MAXIMUM_BITS (32 * 1024)
+
 int
 BN_is_prime_fasttest_ex(const BIGNUM *a, int checks, BN_CTX *ctx_passed,
     int do_trial_division, BN_GENCB *cb)
@@ -249,6 +251,15 @@ BN_is_prime_fasttest_ex(const BIGNUM *a, int checks, BN_CTX *ctx_passed,
 	if (checks < 0)
 		return -1;
 
+	/*
+	 * Prime numbers this large do not appear in everyday cryptography
+	 * and checking such numbers for primality is very expensive.
+	 */
+	if (BN_num_bits(a) > BN_PRIME_MAXIMUM_BITS) {
+		BNerror(BN_R_BIGNUM_TOO_LONG);
+		return -1;
+	}
+
 	if (checks == BN_prime_checks)
 		checks = BN_prime_checks_for_size(BN_num_bits(a));