Avoid an overflow in the ELF SYSV ABI hash function.

The hash function is supposed to return a value less than or equal
to 0x0fffffff.  Due to a bug in the sample code supplied with the
ELF SYSV ABI documentation, the hash function can overflow on 64-bit
systems.  Apply the same fix used by GNU libc, MUSL libc and FreeBSD.
Prompted by https://maskray.me/blog/2023-04-12-elf-hash-function
OK tb@ miod@

diff --git a/libexec/ld.so/resolve.c b/libexec/ld.so/resolve.c
index 709402d..70e8dd0 100644 (file)
--- a/libexec/ld.so/resolve.c
+++ b/libexec/ld.so/resolve.c
@@ -1,4 +1,4 @@
-/*     $OpenBSD: resolve.c,v 1.98 2023/04/09 23:41:47 gnezdo Exp $ */
+/*     $OpenBSD: resolve.c,v 1.99 2023/04/13 19:57:30 millert Exp $ */
 
 /*
  * Copyright (c) 1998 Per Fogelstrom, Opsycon AB
@@ -650,13 +650,11 @@ _dl_find_symbol(const char *name, int flags, const Elf_Sym *ref_sym,
 
        /* Calculate both hashes in one pass */
        for (p = (const unsigned char *)name; (c = *p) != '\0'; p++) {
-               unsigned long g;
                sl.sl_elf_hash = (sl.sl_elf_hash << 4) + c;
-               if ((g = sl.sl_elf_hash & 0xf0000000))
-                       sl.sl_elf_hash ^= g >> 24;
-               sl.sl_elf_hash &= ~g;
+               sl.sl_elf_hash ^= (sl.sl_elf_hash >> 24) & 0xf0;
                sl.sl_gnu_hash = sl.sl_gnu_hash * 33 + c;
        }
+       sl.sl_elf_hash &= 0x0fffffff;
 
        if (req_obj->dyn.symbolic)
                if (_dl_find_symbol_obj(req_obj, &sl))