SharePoint 2010 SP1 troubleshooting

So our upgrade from 2007 to 2010 a couple of years back wasn't without errors in the log, but most of these referenced outdated, unused features or web templates that had been decommisioned from any production site. They didn't prevent the upgrade from succeeding.

But when I applied SP1 recently, I realized some of these lingering references were again surfacing to give me trouble.  The service pack installed successfully, but the PSConfig wizard failed. Log files contained 3 errors:

• A missing site feature
• A missing web template
• A missing web.config file

The result of this service pack install and failed PSConfig was a sharepoint farm that was completely down. A request to any page generated an error and referenced a correlation ID. Tracking this error down in the log was a looong dead end because it was just referencing services that were down as a result of the failed upgrade. I cursed under my breathe about testing this install on a development system that wasn't identical to production, after all, the development system upgraded to SP1 without a hitch.  Prior to implementing the rollback strategy (yay VM snapshots!) I decided to dive in and see if the upgrade was salvageable.

Our farm consists of an App server and a WFE. Although PSConfig failed on the WFE, the upgrade status in Central Administration said it was fully up to date, so I focused on the App server.

The web template was installed on the WFE, but not the App server. So I copied over the folder from the C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\SiteTemplates folder.

Since the WFE was responsible for Central Admin and all sites, I was surprised to find a site reference in IIS on the App server. Checking the site and folder revealed it was empty, and the site status was stopped.  After ensuring this site was indeed running on the WFE, I deleted the site from IIS on the App server and removed the folder from the wss\Virtual Directories in inetpub.

I performed a powershell search for the missing site feature:
Get-SPFeature | Sort -Property Id

The feature was a FBA Management package developed by Visigo to manage FBA user accounts from the Site Settings page in SharePoint.  This had been deployed to the WFE but not the App server.

It was then time to run PSConfig again. Success. All sites functional again.

Next steps:

1. June 2011 Cumulative Update
2. Office Web Apps with SP1