Do not held the vm_map lock while flushing pages in msync(2) and madvise(2).
Mark the VM map as busy instead to prevent any sibling thread to request an
exclusive version of the vm_map. This is necessary to let any PG_BUSY page,
found in the UVM vnode object, to be released by a sibling in the middle of
a page-fault.
Note: the page-fault handler releases & re-grab a shared version of the
vm_map lock and expect it to be available to make progress.
Prevent a 3-Threads deadlock between msync(2), page-fault and mmap(2). The
deadlock reported on bugs@ by many occured as follow:
..ThreadA faults & grabs the shared `vmmaplk' then release it before calling
..uvn_get() which might sleep to allocate pages and mark them as PG_BUSY.
..Once the lock is released, threadB calls uvn_flush(). It sees at least a
..PG_BUSY page and sleeps on the `vmmaplk' waiting for threadA to un-busy
..the page.
..At the same time threadC asked for an exclusive version of the lock and
..sleeps until all reader are done with it. This prevents threadA to
..acquire a shared-version of the lock and finish the page fault.
This issue is similar to NetBSD's PR #56952 and the fix is from Chuck Silvers.
Tested by many on bugs@, thanks!
ok kettenis@