Print Server CPU spikes on reboot -- Spoolsv.exe

Solution

Problem in my case turned out to be a bad driver. Luckily it was a driver from an old printer that was no longer on the network.

Cleaning out print drivers is pretty easy: Open up two instances of print manager, go to the Drivers menu on one, and the printers on the other. Arrange the columns on the printers page so that you can see the printer name and the driver name. Sort by driver name, then compare the list to the ones in the Drivers menu. This makes it easy to see which drivers are no longer in use.

If you've cleaned out old drivers, and the problem is still occurring, you may need to start removing printers until the problem stops. Remove a printer, reboot the server, and see if the problem still exists. Of course, make sure you copy the printer information so you can re-add it later. Once you've determined which printer/driver combo is causing the issue, see if you can find an alternative driver for that printer. Most printer manufacturers recommend using their "Universal Print Driver" in a print server environment, rather than the 'named' or 'model specific' driver.

 Problem

In brief, our print server would hang with 100% CPU usage whenever it was rebooted. This necessitated spoolsv.exe (the 100% process) to be manually terminated, and the spooler and all dependant services to be restarted. After manual termination/restart, the CPU levels would remain normal. Printer shares would be unavailable during the CPU spike.

Machine Specs:

• Server 2008 R2 -- fully updated
• VM on top of XenServer 6.2 - Intel based system
• 2 core
• 3 GB of RAM
• Additional Printer Related Software
• Print Manager (print tracking software)
• Citrix Universal Print Server

Things I tried that didn't help

• Windows updates
• Updating paravirtualization driver.
• SFC / Checkdisk/ other system file scans
• Re-installing drivers/print queues  

Problem ended up being an issue with some unused print driver. Apparently the spooler service loads up drivers even if they are not being used by any print queue. So the only way to fix the problem was to remove all drivers from printers that had been retired -- see details in Solution above.

Printing issues on Win 7 Virtual machines with XenDesktop -- Citrix Universal Print Client

Solution

The  fix for this ends up being to remove the "Citrix Universal Print Client" from the XenDesktop clients. According to certain sources, this happens when the UPC is attempting to contact the Universal Print Server, even when a Universal Print Driver isn't being used. The server obviously doesn't respond, and so there's a considerable timeout before it falls back to windows printing.

I'm skeptical that this is actually what's happening, at least in my environment, for a couple of reasons.

1. The timeout occurs even when using the UPS/UPD/UPC.
2. The timeout occurs for some printers/drivers and not for others

The only way I've found to remove the UPC is to reinstall the XenDesktop Client without the UPC. From command line it looks something like:

#note: reboot before running the first command, and between each command. If you don't specify the "/noreboot" flag, it will reboot automatically after each command.
#<XenDesktopDir> = unzipped iso location
#<XDInstaller> = <XenDesktopDir>\x86\XenDesktop Setup\XenDesktopVdaSetup.exe
#   or for 64-bit = <XenDesktopDir>\x64\XenDesktop Setup\XenDesktopVdaSetup.exe

#Remove Current install
<XDInstaller> /quiet /removeall

#Reinstall Without UPC
#You can check here for the flags you need for any other customizations you make
<XDInstaller> /quiet /Components vda /EXCLUDE "Citrix Universal Print Client" /logpath "c:\temp\xdinstalllogs\"

#Configure Controller hostname/port
#If you set this through group policy, you shouldn't need this step
<XDInstaller> /quiet /reconfigure /controllers "mycontroller.mydomain.com" /portnumber 9999

 

Introduction

Been having some issues with printing from our thin clients. The most common symptom is that whatever program is trying to print (word, notepad, browser, appears to effect all about the same) locks up for 30-60 seconds. During this time the a window saying "connecting to printer" may be present (though not always), and the main program window appears unresponsive (not responding in task manger).

 I'm writing this as I troubleshoot so, apologies if it's a bit schizophrenic.

Setup

Client: Windows 7 x86 - fully updated -- mostly, some are not 100% updated, but all have at least SP1, and it doesn't appear to make much of a difference. Clients have XenDesktop Client installed (7.0/7.1 -- I end up trying 7.5 as well)

HyperVisor: XenServer 6.2.0 (fully updated, SP1 + a couple more updates). Clients all have XenTools installed.

Print Server: Server 2008R2 (VM) x64. Has "Print Manager Plus" software, also has Citrix Universal Print Server.

New Printer Server: Server 2012R2 (VM) x64. Does not have Print Manager Plus.

Drivers: To be discussed

A More Detailed Description of the Problem

My general view of the problem is that the printer is taking a long time to respond to the program trying to print to it. The main way this appears, is when you click "print" in a program, it tries to contact the printer to get availability, status, capabilities, etc. This takes a long time to finish when the problem occurs, which means when you click "print" the program stops working for 30-60 seconds. The window will show "not responding". Sometimes a box saying "connecting to printer" will appear, but not always.

This problem will happen several times when trying to print, because the computer appears to talk out to the printer several times. First when you go to the print menu to select a printer, then again when you click on the printer itself (to select it) then again when you click print to actually send the job. This means there can be a delay of several minutes for a user trying to print a basic document. This is understandably very annoying.

What I've tried so far

My first thought was that it had something to do with the Citrix Universal Print Server. That's why I built up the 2012R2 server. However the problem is present when mapping printers from that server. This only happens with some printers, other printers work just fine.

Mapping the printers directly (via IP on the client) appears to work fine as well.

Most of our printers are HP, and thus most use the HP Universal Print Driver. This is currently my main suspect, but it's unclear as to why that would cause such a problem with only VMs. Physical Clients (your standard desktop machines) do not have this problem.

I also thought it might be the universal print manager (Client side of the Citrix Universal Print Server), but I have disabled that service and deleted the Citrix Universal Print Driver from a machine and the problem persists. It's possible that the Citrix software still causes some problems, currently installing a vanilla windows 7 machine to test this theory.

Problem occurs whether connected over XenDesktop, Remote Desktop, or Directly through XenCenter Console.

Being an Administrator or  not does not appear to have an effect.

Using the FQDN or IP of the server rather than the hostname doesn't appear to have an effect.

I've tried Type 3 and Type 4 drivers (What does that mean?). Both word fine from the print server (that is the print server never has problems printing test pages). Type-4 drivers are not technically supported on Windows 7, so when a windows 7 machine trys to connect to a type-4 printer, they are given a "enhanced Point and Print Compatibility driver". These work fine, however this is not an apples-to-apples comparison, because there is no type-4 HP universal print driver. So the Type-3 not working where the Type-4 does is as much comparing device-specific to Universal as it is type-3 to type-4 (trying to find type-3 device specific drivers). But for what it's worth the HP UPD (type-3) has the problem, the device specific (type-4) do not.

Older version of the HP UPD appear to have the same problem. Nor does PCL 5 vs PCL 6 vs PS.

For some reason HP doesn't always have device-specific drivers on their website -- they'll just like the UPD. So it's really hard to find a printer that has the UPD, Type-3 Device Specific, and Type-4 Device Specific to do some real testing on. Testing the difference between type-3 DS and type-3 UPD at least..... Type-3 DS driver has the problem as well, at least on my "HP LaserJet 400 M401".

Problem also occurs on one of my terminal servers (2008R2),  also running XenDesktop client, but is a physical server.

Problem does not appear to occur on another terminal server (2008R2) which is a virtual machine, but is not running the XenDesktop Client. Doing further testing to verify. For all intents, problem does not exist on this machine. You can see the "connecting to printer..." dialog come up for a split second, but it's almost as fast as a physical machine, not enough to make a difference user experience wise. So it looks like we may be looking at the XenDesktop software as the culprit. It's strange that it only has problems with certain (mostly HP) drivers.

As I mentioned, I've also done tried using the Citrix UPD, but it shows the same problem. I'd have to do more testing to verify whether the CUPD locks up only when using it to print to printers that otherwise exhibit those symptoms, or whether the CUPD is just broken in general.

For now, I'm getting a clean windows 7 machine built up and will install software one-by-one to determine when the problem starts happening.

....

OK, fully updated windows 7 machine with nothing else on it is ready. My method here is

1. Map the printer
2. Wait a minute
3. Open notepad
4. Try to print
5. Wait another minute
6. Open Printer properties
7. Unmap Printer
8. Reboot after each full test (between changing variables, not between each printer)

If either 4 or 6 take more than about 10 seconds, I'll consider the problem to exist. I try this with three different printers, all of which have exhibited the problem in the past. Here are the variables and results of the tests.

1. Clean machine
1. Problem does not exist
2. Install XenTools
1. Problem does not exist
3. Domain Join
1. Problem does not exist
4. Move to Correct OU
1. Problem does not exist
5. Install XenDesktop Client (VDA)
1. Problem defiantly exists.

So I guess that settles it. It's something in the XenDesktop VDA that breaks printing. Keep in mind I did not run any of the XenDesktop Optimizations, so it's not one of those; the problem is within the client itself, or some change it makes without the option not to. So let's see if I can narrow down specifically what's causing it here.

• This Forum suggests stopping "Net Driver HPZ12" service (some sort of HP monitor thing)
• Did one better and disabled "Pml Driver HPZ12" as well (another HP monitoring thing)
• This did not help
• Lets try disabling the service and rebooting.
• No Dice
• This Post on experts exchange says to turn off bidirectional support in the printer properties. Lets try that.
• It's under the "ports" tab in printer properties. I changed it on the server, then deleted/re-added it to the client.
• This doesn't appear to have any effect
• Downloading XenDesktop 7.5 -- just the VDA I'm not upgrading my whole installation yet. Since This "clean" machine isn't even joined to the XenDesktop Controller, I don't see how that would have any effect anyway.
• Installed 7.5 VDA, doesn't appear to have had any effect.
• Just to be sure I removed the device and uninstalled all drivers and tried again
• Still locks up
• Disabled Citrix Print Manager Service
• I've tried this before, but thought I'd try again under 7.5
• Still locks up
• Disabled "Citrix Personal vDisk" Service
• At this point I'm just disabling Citrix Services one-by-one to see if there's any change
• Still locks up
• Disabled "Citrix Profile Management" Service
• Still Locks up
• To reduce redundancy, disabled each "Citrix Service" one-by-one
• Still Locks up
• Tried disabling "allow direct connection to printers" in Citrix Policy.
• This made it slightly less terrible. It still hangs for a bit while "connecting to printer" but the program doesn't stop responding (or at least, windows doesn't think it has). Not a perfect solution but it's progress at least.
• Found this forum post, trying the solution listed at the bottom - installing the VDA without the universal printing component
• Completely uninstalled current VDA first
• Verified problem had gone away
• Reinstalled using "XenDesktopVdaSetup.exe /components vda /EXCLUDE "Citrix Universal Print Client" /logpath "c:\ctxinstall.log" /quiet /noreboot"
•  Sweet baby Jesus I think that actually worked
• Yep, that appears to solve the problem

I've applied this fix to my main two XenDesktop Terminal Servers (aka ServerOS Hosted Desktops) and it appears to have solved the problem. You no longer get the "connecting to printer..." dialog, the program doesn't go to a not responding state, users will hopefully got breaking things by clicking a bunch of buttons while it appears to be frozen.

If the forum post I linked above is to be believed, the issue is that the XenDesktop client attempts to talk to the universal print server, even when the UPD is not being used. I'm a little skeptical that this is the entire problem, because this error would happen even when using the UPD/UPS. But at any rate it's fixed. Obviously this precludes using the Citrix Universal Print Server in the future, but it's honestly been such a pain to manage/get working that I have to call that a 100% positive effect.

Microsoft Access 2013 on Server 2008 R2 SP 1 remote access through XenDesktop Screen Freeze

Solution

New Solution:

"Legacy Graphics Mode" is an option is Citrix Policy. Enabling this for the desktop group will stop Access from freezing, and doesn't break 'browse' 'save-as' etc. Unsure exactly what legacy graphics mode does, still trying to get someone to answer that for me, or why it doesn't work in the first place, but this is a less bad work around.

Appears to be some visual theme here that messes up the ICA connection. The "fix" - probably more of a workaround - is to run the program in compatibility mode (don't forget to set for all users). You'll want to select compatibility mode for windows 7 + disable visual themes.

Edit: People on Citrix forums claim that this effects office 2010 as well; I can't confirm that it does, or that this fix works for 2010 if the problem does exist, but there you go.

Warning: This workaround has been found to cause some other undesirable behavior. Notably: "browse" buttons will no longer launch the file browser to allow you to select files. This effects a number of things such as the "save as..." feature and "import from (excel/text/etc)" wizards. This is arguably less detrimental than the screen freeze but please be aware.

The Problem / Full story

As you may guess from the title, this is an insanely specific bug that we ran into in one of our labs. Students were trying to use MS Access (because we teach that for some reason) and their sessions kept on locking up. Here's the sequence of events:

Users log into Wyse Xenith 2 thin clients connecting to a 2008 R2 SP 1 terminal server type environment through XenDesktop (Citrix seems to refer to it as ServerOS connection, or Shared Published Desktops). This works great.

Users then open Access and everything seems to be working fine, but when they try to open a table in design view the screen freezes after a few seconds. At this point the machine is completely unresponsive, the mouse still moves, but no mouse clicks or keystrokes appear to have any effect.

I say "appear to have any effect" because they actually do the user just can't see them. If I use lanschool to remotely watch/control the user's session I can see them clicking around and typing things, and when I control I can interact with the session just fine; The user cannot see anything changing on the screen. It's totally bizarre. 

The only way I've found to restore functionality is to remotely log the user off so they could log in again. 

The really weird thing is that none of the factors individually cause the problem. Using Access on the same server but over remote desktop (RDP) works just fine. Using Access on a windows 7 machine over xendesktop works just fine. No other program I've used on the server over XenDesktop has exhibited this behavior.

I figured out the workaround above pretty much by trial and error. Then figured out the correct solution from citrix forums; Yay citrix forums!