Improve consistency of the substitution command further.

Improve consistency of the substitution command further.

When the opening square bracket ('[') is abused as the delimiter, the regular
expression contains a bracket expression, and the bracket expression contains
another opening square bracket (sick! - i mean, sic!), then do not require
escaping that innermost bracket and treat a preceding backslash as a literal
backslash character, in accordance with POSIX:

   $ printf 'x[x\\x\n' | sed 's[\[[][R[g'
  xRx\x
   $ printf 'x[x\\x\n' | sed 's[\[\[][R[g'
  xRxRx

While here, also make the implementation more readable and insert
some comments.

Joint work with martijn@ (started during g2k18) and OK martijn@.