Make i2d_ASN1_OBJECT() return -1 on error

This is what the (not quite appropriately) referenced ASN1_item_i2d()
page documents for errors, matches what the RETURN VALUE section has
been documenting for ages, matches BoringSSL, it's the usal behavior
for i2d_*. It's also what OpenSSL (of course incorrectly) documents.

discussed with jsing

diff --git a/lib/libcrypto/asn1/a_object.c b/lib/libcrypto/asn1/a_object.c
index 00b5e7a..aae1b8b 100644 (file)
--- a/lib/libcrypto/asn1/a_object.c
+++ b/lib/libcrypto/asn1/a_object.c
@@ -1,4 +1,4 @@
-/* $OpenBSD: a_object.c,v 1.52 2024/05/29 16:04:50 tb Exp $ */
+/* $OpenBSD: a_object.c,v 1.53 2024/05/29 16:10:41 tb Exp $ */
 /* Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com)
  * All rights reserved.
  *
@@ -619,7 +619,7 @@ i2d_ASN1_OBJECT(const ASN1_OBJECT *a, unsigned char **pp)
        int objsize;
 
        if (a == NULL || a->data == NULL)
-               return 0;
+               return -1;
 
        objsize = ASN1_object_size(0, a->length, V_ASN1_OBJECT);