From: deraadt <deraadt@openbsd.org>
Date: Mon, 23 Sep 2024 21:18:33 +0000 (+0000)
Subject: If during parsing lines in the script, ksh finds a NUL byte on the
X-Git-Url: http://artulab.com/gitweb/?a=commitdiff_plain;h=e6679286379553a84039637d6b8e4533a1847d0b;p=openbsd

If during parsing lines in the script, ksh finds a NUL byte on the
line, it should abort ("syntax error: NUL byte unexpected").  There
appears to be one piece of software which is misinterpreting guidance
of this, and trying to depend upon embedded NUL.  During research,
every shell we tested has one or more cases where a NUL byte in the
input or inside variable contents will create divergent behaviour from
other shells.  (ie. gets converted to a space, is silently skipped, or
aborts script parsing or later execution).  All the shells are written
in C, and majority of them use C strings for everything, which means
they cannot embed a NUL, so this is not surprising.  It is quite
unbelievable there are people trying to rewrite history on a lark, and
expecting the world to follow alone.

If there is ONE THING the Unix world needs, it is for bash/ksh/sh to
stop diverging further by permitting STUPID INPUT that cannot
plausibly work in all other shells.  We are in a post-Postel world.

It remains possible to put arbitrary bytes *AFTER* the parts of the
shell script that get parsed & executed (like some Solaris patch files
do).  But you can't put arbirary bytes in the middle, ahead of shell
script parsed lines, because shells can't jump to arbitrary offsets
inside the input file, they go THROUGH all the 'valid shell script
text lines' to get there.

This was in snapshots for more than 2 months, and only spotted one
other program depending on the behaviour (and that test program did
not observe that it was therefore depending in incorrect behaviour!!)

ok ingo.  Softer ok's from various others.
---

diff --git a/bin/ksh/shf.c b/bin/ksh/shf.c
index b2670b93fd0..a9347444cdb 100644
--- a/bin/ksh/shf.c
+++ b/bin/ksh/shf.c
@@ -1,4 +1,4 @@
-/*	$OpenBSD: shf.c,v 1.34 2019/06/28 13:34:59 deraadt Exp $	*/
+/*	$OpenBSD: shf.c,v 1.35 2024/09/23 21:18:33 deraadt Exp $	*/
 
 /*
  *  Shell file I/O routines
@@ -450,6 +450,10 @@ shf_read(char *buf, int bsize, struct shf *shf)
 		ncopy = shf->rnleft;
 		if (ncopy > bsize)
 			ncopy = bsize;
+		if (memchr((char *)shf->rp, '\0', ncopy) != NULL) {
+			errorf("syntax error: NUL byte unexpected");
+			return EOF;
+		}
 		memcpy(buf, shf->rp, ncopy);
 		buf += ncopy;
 		bsize -= ncopy;
@@ -493,6 +497,10 @@ shf_getse(char *buf, int bsize, struct shf *shf)
 		ncopy = end ? end - shf->rp + 1 : shf->rnleft;
 		if (ncopy > bsize)
 			ncopy = bsize;
+		if (memchr((char *)shf->rp, '\0', ncopy) != NULL) {
+			errorf("syntax error: NUL byte unexpected");
+			return NULL;
+		}
 		memcpy(buf, (char *) shf->rp, ncopy);
 		shf->rp += ncopy;
 		shf->rnleft -= ncopy;