From: bluhm <bluhm@openbsd.org>
Date: Tue, 26 Mar 2024 18:18:30 +0000 (+0000)
Subject: Improve spinning in mtx_enter().
X-Git-Url: http://artulab.com/gitweb/?a=commitdiff_plain;h=55c63681fc0c76278cd2b515f4696a742954c980;p=openbsd

Improve spinning in mtx_enter().

Instead of calling mtx_enter_try() in each spinning loop, do it
only if the result of a lockless read indicates that the mutex has
been released.  This avoids some expensive atomic compare-and-swap
operations.  Up to 5% reduction of spinning time during kernel build
can been seen on a 8 core amd64 machine.  On other machines there
was no visible effect.

Test on powerpc64 has revealed a bug in mtx_owner declaration.  Not
the variable was volatile, but the object it points to.  Move the
volatile declaration in struct mutex to avoid a hang when going to
multiuser.

from Mateusz Guzik; input kettenis@ jca@; OK mpi@
---

diff --git a/sys/kern/kern_lock.c b/sys/kern/kern_lock.c
index b21e1aa5542..19f441bd5ce 100644
--- a/sys/kern/kern_lock.c
+++ b/sys/kern/kern_lock.c
@@ -1,4 +1,4 @@
-/*	$OpenBSD: kern_lock.c,v 1.72 2022/04/26 15:31:14 dv Exp $	*/
+/*	$OpenBSD: kern_lock.c,v 1.73 2024/03/26 18:18:30 bluhm Exp $	*/
 
 /*
  * Copyright (c) 2017 Visa Hankala
@@ -264,15 +264,17 @@ mtx_enter(struct mutex *mtx)
 
 	spc->spc_spinning++;
 	while (mtx_enter_try(mtx) == 0) {
-		CPU_BUSY_CYCLE();
-
+		do {
+			CPU_BUSY_CYCLE();
 #ifdef MP_LOCKDEBUG
-		if (--nticks == 0) {
-			db_printf("%s: %p lock spun out\n", __func__, mtx);
-			db_enter();
-			nticks = __mp_lock_spinout;
-		}
+			if (--nticks == 0) {
+				db_printf("%s: %p lock spun out\n",
+				    __func__, mtx);
+				db_enter();
+				nticks = __mp_lock_spinout;
+			}
 #endif
+		} while (mtx->mtx_owner != NULL);
 	}
 	spc->spc_spinning--;
 }
diff --git a/sys/sys/mutex.h b/sys/sys/mutex.h
index a90714c82b1..cff33404c3c 100644
--- a/sys/sys/mutex.h
+++ b/sys/sys/mutex.h
@@ -1,4 +1,4 @@
-/*	$OpenBSD: mutex.h,v 1.20 2024/02/03 22:50:09 mvs Exp $	*/
+/*	$OpenBSD: mutex.h,v 1.21 2024/03/26 18:18:30 bluhm Exp $	*/
 
 /*
  * Copyright (c) 2004 Artur Grabowski <art@openbsd.org>
@@ -40,7 +40,7 @@
 #include <sys/_lock.h>
 
 struct mutex {
-	volatile void *mtx_owner;
+	void *volatile mtx_owner;
 	int mtx_wantipl;
 	int mtx_oldipl;
 #ifdef WITNESS