Zoom Virtual Backgrounds Causing BSOD on Windows

[williamson1]

Our organization has several new laptops (Lenovo P1 Gen 5) with the AMD Radeon™ 780M graphics chip that are running into a BSOD when using virtual backgrounds in Zoom Workplace. This is across multiple users and computers, and all available drivers have been updated to their latest versions on all devices with no change in behavior. Our current version of Windows is 23H2, but we are upgrading soon to 24H2.

Steps to reproduce (so far):

• Connect device to a docking station of some kind. If not docked, issue does not occur.
• Connect external camera to device.
  • Does not matter if through docking station or a USB port on the laptop.
  • Occurs across multiple models of external cameras.
  • Issue does not occur with integrated camera.
• Open Zoom Workplace version 6.4.3 (latest version)
• Open settings or a new meeting with the external camera enabled
  • External camera must be selected as active camera
  • Issue still occurs with hardware acceleration disabled/difference video rendering methods.
• Attempt to change/use virtual background.
  • Issue still occurs if a virtual background was already selected for use before zoom is opened.
• Zoom will freeze completely, then eventually the computer will BSOD with the error code VIDEO_SCHEDULER_INTERNAL_ERROR
  • This is the error every time, it does not change
  • Even if Zoom is force quite immediately after the freeze through task manager, the BSOD will still happen.

After hours of troubleshooting we have not found a fix. This is largely disruptive to us, as some of our users train others on how to use Zoom and currently cannot show them how to use virtual backgrounds because the software is causing their computer to crash.

Microsoft Teams does not have this issue.

[mitjmr]

My organization is experiencing the same problem with the same generation AMD laptop (T14 Gen 5 rather than P1 though).  Just about all of the symptoms you describe are also happening with us, except:

- We can get it happening without the laptop being docked, an external camera merely needs to be plugged in to a USB port on the laptop.

- We are also primarily 23H2, but the issue also happens with 24H2.

- This has only been happening since the March Windows Cumulative Updates.  We have machines set up for deployment before that update, and the issue does not happen with those machines until after the update is applied.

- In addition to not seeing this with Microsoft Teams, we are also not seeing it with Slack Huddles with virtual backgrounds.

I have tickets in with both Zoom and Lenovo.  Zoom has not been very responsive at all. Lenovo is having me do testing with the factory image, which is not practical, but I've been trying anyway.  It seems to be better with the factory image even after the Windows Update, as long as you prevent video drivers from being updated from any method (either Windows Update, Lenovo System Update, or directly from AMD).  If you do that, then the freezes eventually come back, though they seem to not result in a blue screen, just a long delay and an error about it taking a while to load.  This isn't really practical though, since we need to deploy these with Windows 11 Enterprise, not the Windows 11 Pro image that these ship with.

I've also spent many hours over the past couple weeks working on this, and can relate to the frustration.

[williamson1]

Hello mitjmr,

Thank you for adding some more information, I think we are running into the same issue due to how similar your description is. I wish Zoom was more responsive about this issue. It seems like the Zoom app is not playing nice with the AMD graphics drivers or chipset. My next plan is to see if this issue occurs with earlier versions of Zoom if I can find them. I will let you know if we find any other information.

[jflippen]

Our organization has also been experiencing this since last week.  P16s Gen 2 with the 780m graphics.  Windows 11 23H2.  It seems inconsistent on whether internal camera has the issue vs external camera (for my laptop, external always crashes and internal only sometimes).  I have tried other computers with a different chipset and have no issues. We have examine BIOS versions, but that doesn't seem to matter as it happens across multiple versions.  Drivers have been updated for several months, which leaves possibly a windows update or the latest version of Zoom Workspace.

I have also noticed that the issue seems to go away if you don't use ANY effects. just video and nothing else.  I also noticed that the web version of zoom works just fine with any effects (app.zoom.us).  Teams, Webex, Google Chat... all have zero issues. Just Zoom Workspace.  I am looking at windows updates from end of last month but not seeing anything glaring.  Going to also try a windows 10 image to see if it's unique to Windows 11. 

[jflippen]

Didn't mean to post twice... I just didn't see my reply the first time I tried and so I thought it didn't go through.  12 hours later and they both show up... oh well.

[jflippen]

Our organization is experiencing this issue as well on our Lenovo P16s Gen 2, also 780m graphics.  I have noticed that it works just fine with other video conference applications and even works with the web version of zoom no problem.  For us it doesn't seem to always matter if it's USB or internal camera, but removing any kind of effects including background blur seems to be a temporary band aid.  I have had to go into several local app data folders to clear out zoom Workspace so that we could remove the effects from cameras.
Intel chipsets seem fine.
 All effected users are on Windows 11 23H2.  I have yet to try on Windows 10 on the same chipset or deploying a Lenovo image to the laptop.  We don't think it's the drivers or BIOS as our BIOS is several different versions and the drivers have been the same for months.  We are considering last month's cumulative windows update as well.

[mitjmr]

We tried Windows 10, and there were no problems with that OS.  Not that it really helps as Windows 10 is EOL in October, and I'm not going to downgrade a bunch of users.

[williamson1]

Updates on my end separated into sections to make it easier to read.

Docking station doesn't matter:

I was mistaken. A docking station or no docking station doesn't seem to affect the outcome of this BSOD. I also accidentally put the wrong model of laptop: a P14s Gen 5 is what we have.

External Camera connected before Zoom is opened:

The only requirement for a BSOD is for an external camera to be connected to the laptop before starting Zoom, then opening up the camera with a virtual background enabled to cause a BSOD. However, if you start a virtual background with the external webcam disconnected, then plug in an external camera, the BSOD does not happen and it works like normal.

AMD Driver seems to be part of the problem:

If you switch the Display Adapter driver to the Generic Microsoft Display adapter that comes with Windows, the issue does not happen. Lenovo only provides 1 AMD driver to use, but I was able to install a newer version from AMD's adrenaline software. The issue still occurs on the newer driver.

No real solution:

At this point I don't really care if it is AMD's fault or Zoom's fault: I just want the problem to be fixed. I tried several more things to try and fix the issue, including disabling Windows Hardware Accelerated Video Scheduling. Nothing seems to fix it (ripping the AMD driver from the system is not a real solution), besides telling my customers "sorry you can't use Zoom backgrounds".

It is the same error every time as well: VIDEO_SCHEDULING_INTERNAL_ERROR. 

I have read the dump log in Windbg and it is cryptic and unhelpful.

[mitjmr]

I tried some of the things you tried to see what happens on my end.

External Camera connected before Zoom is opened:
Tested plugging in the external camera after launching a meeting (or just the video settings page) with a virtual background on.  This does work!  Except just for that one meeting/instance.  Once the meeting is over and another is launched, the freeze comes back.

AMD Driver seems to be part of the problem:
I've tried reverting back to the Microsoft Basic Display Adapter, but still have the freezing.  I've even gone as far as using Display Driver Uninstaller to completely nuke all video drivers, and still have it happen.  I do feel like it is related to the video driver, but no manipulating of it seems to help me.

I've installed Lenovo's Recovery Image, and the crash does not happen, until I apply the March Windows Updates, then it happens.  I've had  users using these machines without these issues just fine until the March updates hit, that's when we started seeing this.  Unfortunately the April updates are out now, and do not solve the problem.

FYI, I've used a utility called WhoCrashed to analyze the memory dumps, and the problems are being trigged by two drivers/modules, watchdog.sys, and dxgmms2.sys.

My Zoom support contact has continued to not provide any helpful suggestions as of yet.

[jflippen]

I have gone through my minidump files as well and do often see dxgmms2.sys in the list of things... however the arguments listed in the debug are less than helpful.  So far the only things that is working for me is to roll back to the driver 32.0.11022.4 or to tell people not to use any effects as a band-aid until a fix is rolled out.

[jflippen]

Docking station doesn't matter:

Can confirm docking station does not seem to matter (or the amount of screens, or if the lid is open or closed).

External Camera connected before Zoom is opened:

I have noticed this as well, though it doesn't always seem consistent.  Additionally, sometimes I have had people BSOD with just the internal camera and effects, though not as common.

AMD Driver seems to be part of the problem:

I grabbed one of our laptops that hadn't been imaged yet and set it up without domain joining or anything.  I noticed it worked fine out of the box until the AMD display driver updated to version 32.0.11030.7006.  I rolled back the driver to 32.0.11022.4 from June of 24 and that one seems to work without issues.  I used DISM to export the driver and then added it to my laptop and manually switched the driver to the good one.  At first my laptop froze for a second, but I told it to keep the video and then it unfroze and did not BSOD.

The question now is do I try to roll back 250 devices to an older display driver?  The latest driver straight from AMD does not fix the issue, and it could be months before Lenovo rolls out a fixed driver.

[jflippen]

scratch that... after a few more tests my laptop BSOD on the old driver again.

[mitjmr]

Yeah, that might be one of the most annoying things about this issue, you think you might have figured out a combination of versions of software/drivers that works, but after a couple restarts of the Zoom client or the computer or removing/re-adding the external webcam, it finally BSOD's.

[williamson1]

Update: found a fix for my situation

I was able to fix the issue for two of my users by installing an earlier version of Zoom.

Finding earlier versions is difficult, because Zoom doesn't post them anywhere. However, the hint is in the installer link:

https://zoom.us/client/latest/ZoomInstaller.exe ---- This is the default link for installing Zoom. However, you can download earlier versions by editing the URL. For example, I can look at the changelog for version 6.2.11 and find the last section of digits there under the collum "Windows". (50939)

That means the url changes to https://zoom.us/client/6.2.11.50939/ZoomInstaller.exe. This version fixed the issue for me, but I arbitrarily chose it. Newer versions might work, but I don't yet have another computer with me to test with without stopping someone else's work. Could I have some of the other users of this forum see what the latest working version that works for them is?

Here is the link to the changelog to search through versions:

https://support.zoom.com/hc/en/article?id=zm_kb&sysparm_article=KB0061222

For example, 6.3.0 appears to have several important bug fixes, but I don't know if it works or not. I also want to make sure that any earlier versions don't have serious security vulnerabilities, so the later the version that works, the better.

Also, the old version of Zoom must be uninstalled before you can install the older version. Check under Settings > Help > About Zoom to ensure you are on an older version, as it looks identical to newer versions.

[mitjmr]

I wanted to try older versions, but didn't know of a way to grab them from the official channel, thanks for finding this!

I have yet to receive any crashes with 6.3.1 *so far*.  (I reserve the right to update this if it decides to give up on me after a while like the other times I thought things seemed stable).

I decided to jump up to 6.4.0, and that did trigger a crash, so that potentially limits when this starts to between 6.3.1 to 6.3.11.  I can do more testing tomorrow.

[williamson1]

I will also do some testing with 6.3 versions of Zoom to see if the issue is limited to 6.4 and beyond. The more information I get on this issue the more inclined I am to believe that this might be a combination problem between AMD drivers and Zoom, not necessarily one or the other. If others could let us know if older versions of Zoom play nice, that would be great.

[jflippen]

Removed 6.4.63669 and installed 6.2.11.  Older zoom seems to be stable so far.

[mitjmr]

Installed 6.3.11 this morning, have not gotten it to crash yet on a couple of machines with a lot of exiting/relaunching of the application and detaching/reattaching of the external camera.  Usually those actions will trigger a freeze at some point, but not yet so far.  Looks like 6.3.11 might be the last stable version on this platform?

[MGSR]

Hello Community Members! 

Thank you for bringing this to our attention, and we sincerely apologize for the inconvenience.

We've escalated the issue internally and will keep you informed as soon as we have any updates.

In the meantime, we appreciate your patience and understanding.

[williamson1]

Thank you for the assistance, let us know if there is any data that we can provide on our end to help troubleshoot.