Microknowledge

Using FSUtil to fix a failed SharePoint 2010 upgrade – Windows Process Activation Service Error – and the Blue Comet

We recently had to rescue a stalled SharePoint upgrade that went awry after a partial disk rebuild.  During the upgrade, Central Administration refused to start, reporting the error:

Windows could not start the Windows Process Activation Service - Error 6801: Transaction support within the specified resource manager is not started or was shut down due to an error.

We traced this to the local file system journals being out of sync due to the partial restore.  As result, IIS itself refused to start after reboot because the WAS wouldn’t start – and the WAS acts as the ignition for other processes, like IIS.  In part, this is done to protect the data in case the file system has become corrupted.  However, there’s also a simple fix.  At a command line, run:

fsutil resource setautoreset true c:\

assuming the system drive is C.  After a reboot, you will able to start IIS and continue with the upgrade.

So, why the Blue Comet?  As you might recall, the penultimate episode of the Sopranos was The Blue Comet.  And this will be my next to last post for Microknowledge under the KMA banner. 

 

The Blue Comet, from http://forum.atlasrr.com/forum/topic.asp?ARCHIVE=true&TOPIC_ID=53193

There will be more to come later today – but I promise neither a bloody confrontation nor a sudden cut to black.  Stay tuned – there’s still one more.

Don’t Forget About Me, If, By 'Me', You Mean the Page File on a SharePoint Server

Recently I was asked to help troubleshoot performance issues on an older, seasoned SharePoint 2007 installation.  (No Developer Dashboards here!).  We ran through a lot of performance counters, checked the SQL performance, and made sure that indexing and backups weren’t coinciding with business hours.  All good.  So why were the pages so sluggish?

Looking at counters on the web front end server showed the server was swapping memory.  A lot.  The server originally began life with only 2-4GB of RAM and a single 60GB hard disk.  Over time, it had grown to 16GB of RAM with an extra 500GB disk for index files (so, a lot of free space there!)  However, we found that the page file (previously known as the swap file!) was still locked at 2GB.  Expanding this to about 24GB, spread across multiple local disks, really improved not only system performance but also perceptions of system responsiveness.

To sum up -- do NOT assume you don’t need page files, even on virtualized servers.  The general rule of thumb – Microsoft recommends the total page file size should be 1-3X the size of total physical RAM.  (If you allow Windows to automatically regulate this, the size is usually OK.  Further details can be found at http://support.microsoft.com/kb/2021748 (see below for summary tables).

Default Page File Sizes for Windows XP and Windows Server 2003.

System Memory Size (RAM)	Minimum Page File Size	Maximum Page File Size
Windows XP and Windows Server 2003 <1GB	1.5 x RAM	3 x RAM
Windows XP and Windows Server 2003 >= 1GB	1 x RAM	3 x RAM

Default Page File Sizes for Windows Vista, Windows Server 2008, Windows 7 and Windows Server 2008 R2.

System Memory Size (RAM)	Minimum Page File Size	Maximum Page File Size
<1GB	Large enough to hold a kernel-memory crash dump and is RAM plus 300MB or 1GB, whichever is larger	3 x RAM or 4GB, whichever is larger
>= 1GB	Large enough to hold a kernel-memory crash dump and is RAM plus 300MB or 1GB, whichever is larger	3 x RAM or 4GB, whichever is larger

I’m at SPTechCon on Boston all week – will share more shortly!

Adding Microsoft Pivot Business Intelligence to SharePoint

Pivot is a unique graphical query interface released by Microsoft Live Labs in 2010.  Microsoft certainly seems to love the word – this follows in the wake of PowerPivot and PivotTables, two very different business intelligence subsystems. 

Pivot is entirely different.  At its initial introduction at WPC2010, Microsoft described it as “powerful, informative and fun”.  Fun?  Yes, I think it is!

Pivot lets you build an interface to items that can be represent with a graphical tag.  As you drill into the items, the pictures dynamically fly around the screen to rearrange themselves according to your query.  Pivot provides a local client, an Excel design interface, and Silverlight controls to allow you to integrate Pivot into your SharePoint site.  I’ve pasted a screenshot below – you can also “play along” with the interactive demos online at http://www.microsoft.com/silverlight/pivotviewer/

 

Let’s walk through building your first Pivot collection and viewing it through a Silverlight control from your SharePoint site.  Since one of my hobbies is digital photography (see www.flickr.com/photos/cmcnulty) I used Pivot to build a simple viewer for some of my favorite pictures. 

Step 1: Install the Excel Add-In

The simplest way to start building a Pivot Collection is to install the Excel Add-In from http://www.silverlight.net/learn/pivotviewer/download-excel-tool/ The Excel Add-In add the Ribbon control to allow you to create a New Collection.  What’s a collection?  A collection is maintained in an Excel spreadsheet, but is exported to Pivot as a .cxml file and related directory that contains all the related support fields and graphic element required by the Pivot interface.   You use Excel to maintain the data, and then users can query the data using the Pivot Collection.

Once you’ve installed the add-in, select New Collection from the Pivot Collections tab.  The Pivot template gives you two spreadsheets in your workbook:

• Collection Items.  This is the main screen where you will add image files and define the principal columns for data exploration.
• Collection Properties  The tab allows you to define collection-wide information, such as Collection Title, Icon,  Brand Image, Additional Search Text, and Copyright.

 
 

Use Import Images to browse for pictures you want to add to the Collection.  (You may note that some columns are hidden by default.)  Once the photos are imported, you need to start adding data for the existing columns:

• Name – A title for the image
• HRef – A link to follow if the image is clicked
• Description

There is pop-up help to remind you about each of those columns on the header line.  Most important, you can add additional columns to the right of the Description Column.  Here’s what our sample collection looks like after I imported photos and added several additional columns for State, Time (of day) and Year:

 

Once the collection is defined, you can play with it by using the Quick Preview option (which actually doesn’t show the images, just the query.

Sept 2: Publish Collection

First, save your Excel file – and it doesn’t need to be in the location where the finished collection will live.  Then, choose the Publish Collection item.  Pick a new local folder, and give the Collection .CXML file a name.  This process can take a few minutes while the engine parses and resizes all the graphical elements.  Once it finishes, your collection should open up in a new Pivot Viewer local window.  You can explore your data and test to make sure your collection is ready to share.

One of the things you may note is that you’re not limited to “check box” queries – if you define number values the Pivot query interface lets you use a slider control to filter a range of values.

Step 3: Publish to Web

The first thing to do is to copy the entire Collection directory to your SharePoint server – somewhere on the file system, outside of SharePoint.  There are a LOT of files, so this process can take a while.

In our example, I created a new C:\Pivot directory.  The entire collection goes into C:\Pivot\PivotPix. 

You’re also going to need to download the sample Silverlight Pivot viewer and viewer page from Microsoft (see http://www.silverlight.net/learn/pivotviewer/release-notes/) In our example, we put the XAP file (a compiled Silverlight app) and HTML page in C:\Pivot\Viewers.  We’re really taking a shortcut here – however, recompiling a hew viewer is pretty simple in the Silverlight 4 Toolkit.

Step 4:  IIS Configuration

Now the web server needs to know about the files we just added.  In IIS7 manager, I add two virtual directories:

• Pivot – mapped to the location of the viewer files
• PivotPix – mapped to the location of the CXML file and the rest of the collection

 

To do this:

• Right click on the web site where you want to add the alias
• Select Add Virtual Directory
• Enter the Alias and Directory as needed

 

Step 5: Edit the HTML File

There’s a lot in the sample HTML file, but the key line is:

<param name="initparams" value="initialCollectionUri=http://msshome2010/PivotPix/FavoritePhotos.cxml" />

Change the value of the initiaCollectionUri to match the “http://website/alias/nameofcollection.cxml” path for your local server.  (Substitute your own local server name and alias.)

Step 6:  Add links on SharePoint

Test the path to your edited HTML file – it should be http://yoursite/your-alias/PivotViewerSample.HTML [match to your own actual server names etc.] If you’ve set up the aliasing correctly, add a link in SharePoint and you should get your browser to render the Silverlight view to your collection:

 
 

What next?  There’s a lot more you can do.  You may start by adding custom columns to your collection.  Also, you’ll want to redesign and recompile the Silverlight viewer to patch your interface requirements  but we’ll leave that for another day.

Alternative Approach

Use the Silverlight Pivot Viewer for SharePoint on Codeplex http://pivotviewersp.codeplex.com/  This control is desgined to let you skip the Excel collection generation and IIS process entirely.  Instead, this solution consumes its images and data directly from:

• A picture library
• The User Profile List
• Site navigation

The web part is not hard to figure out once you’ve downloaded and deployed the WSP.  (Hint – although the documentation tells you how to install it using Stsadm, why not follow through on that New Years’ Resolution and use PowerShell:  Add-SPSolution and Install-SPSolution).

Anyway, once you’ve installed the part and added to a page, you can connect to a library, and get almost the same experience as you had using the HTML-hosted XAP file:

 
 

 

Have powerful, informative fun!

SharePoint 2010 Troubleshooting with ULS, Event, Usage and Health Logs

Despite your best efforts, users may eventually get the 2010 error screen:

 

SharePoint errors always include a unique “correlation ID” to help identify related logged events in multiple areas.  However, in a medium or large farm, the number of local log files to cross reference can become cumbersome.

However, in SharePoint 2010, the unified logging and monitoring service integrates information from a broad range of sources:

• Windows Event Logs
• Performance Monitors & Counters
• SharePoint Features and Web Parts
• Unified Logging Service (Diagnostics)

This process relies on the Usage and Health Shared Service Application, and by default, information is centralized in the WSS_Logging database.  SharePoint uses timer jobs to aggregate information from all these systems across all the servers in your farm.  The table structure of the database is complex; however, the database has prebuilt views for ULS, Analysis Service, Site Structure, etc. However, if you look at an out-of-the-box logging database, you will NOT see the NTEventLog and ULSTraceLog views shown here:

 

How do you get them?

By design, the Timer jobs for aggregating event logs and ULS logs are disabled.  In part, this is because the potential for runaway database growth is significant if verbose logging is used across a number of servers.

However, if you enabled the following two Timer jobs, the jobs will create the “missing” SQL views:

• Diagnostic Data Provider: Trace Log
• Diagnostic Data Provider: Event Log

Once you have the correct data and views in the database, you can run a SQL query like the one below to find all the logged events that correspond to a given correlation ID:

SELECT * FROM [WSS_Logging].[dbo].[ULSTraceLog] WHERE CorrelationID = '04377DAE-C2FD-4DBE-A57E-101B3005059E’

Watch the database sizes, but this can be a very effective technique when you need more information about a logged error.  Good luck!

Super Geeky LDAP-ADAM Configuration with SharePoint 2007

We were recently working on a client SharePoint farm that required an extranet authentication provider to link to an LDAP-oriented Active Directory Application Mode (ADAM) server.  For those of you keeping score, ADAM is a lightweight version of Active Directory dedicated to user authentication.

We began by extending the primary intranet application to a new top level URL, also on port 80, as part of the Internet zone.  .  This automatically adds the Alternate Access Mapping and creates the appropriate IIS Web Site.

 

Next we added the right configuration settings inside web.config for the primary site, the extended site, and Central Admin.  (All of these can be found by default at C:\inetpub\wwwroot\wss\virtualdirectorories).  The following setting go at very beginning of the systems.web node (e.g. immediately after the <system.web> tag.)   [Your server and userContainer tags will vary from this sample, fyi, but trust me, this is correct.]

 

<membership defaultProvider="ADAMMembership">

    <providers>

    <add name="ADAMMembership"

     type="Microsoft.Office.Server.Security.LDAPMembershipProvider, Microsoft.Office.Server, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71E9BCE111E9429C"

     server="SERVERNAME"

         port="389"

         useSSL="false"

         userDNAttribute="distinguishedName"

         userNameAttribute="cn"

         userContainer="OU=users,DC=[SAMPLE],DC=[SAMPLE],DC=com"

         userObjectClass="user"

         userFilter="(ObjectClass=user)"

         scope="Subtree"

         otherRequiredUserAttributes="sn,givenname,cn" />

        </providers>

   </membership>

 

 

 

Then we configured the Internet zone to use Forms authentication against the ADAMMembership provider.  And users still couldn’t login.

 So we checked again, and wanted to verify that the central admin and web pools had the right permissions to read from the ADAM server.    In ADSI Edit, we connected to the LDAP server  and switched from “Default  Naming Context” to “Configuration.  Next, ,we selected the "Roles" container and right-clicked the “Readers”.  Edit the Members  property and click Add Windows Account to verify which accounts can access the ADAM server.

And, like that, ADAM users can login to SharePoint. 

Installing SharePoint 2007 on Windows 2008 Server R2 - Slipstream Patches

If you're reading this, it may be because you had issues trying to install SharePoint 2007 on Windows 2008 R2 Server. 

You're not crazy - the original RTM version isn't supported.  What you need to do is slipstream patch the installation files to at least SP2. 

In the sequence below, let's presume that you've already extracted the installation files to a directory names c:\software\x64.  Assuming you kept the default file names, the following commands will patch the installation to the post SP2 February 2010 Cumulative Update:

· wssv3sp2-kb953338-x64-fullfile-en-us.exe /extract:c:\software\x64\updates

· officeserver2007sp2-kb953334-x64-fullfile-en-us.exe /extract: c:\software\x64\updates

· wss-kb978396-fullfile-x64-glb.exe /extract:c:\software\x64\updates

· office-kb978395-fullfile-x64-glb.exe /extract:c:\software\x64\updates

The extract command puts a lot of update packages in the updates folder.  When you run the install, you'll probably note the progress bar gets to about 95% and stays there for a long time.  That's because the patches take a while.  However at the end, you are ready to launch the SharePoint Configuration Wizard and start using a fully updated version of SharePoint on Windows 2008 R2 without further patching.  Good luck!

Server Virtualization Tips and Tricks

I promised a followup to my previous post on virtualization for SharePoint developers.  Here are a few additional tips and tricks to keep in mind if you’re starting a development server virtualization project.

· Check for antivirus updates.  Older version of some server antivirus software had issues with the first releases of Hyper-V.

· Turn on hardware-assisted virtualization in the host system BIOS

· Turn on PING for ICMP (provide batch file)

· For Hyper-V or Microsoft Virtual Server, install Integration Services on each VPC.  This gives each guest system access to shared folder, cut and paste, drag and drop, and mouse support.

· Remove older virtual server extensions from any Virtual Server 2007 (or older) hosted guests before migrating to Hyper-V

· When virtualizing Exchange 2007 or SQL Server 2008, use actual host physical disks or LUNs, instead of VHD virtual disks, for best performance results.

Server Virtualization for the SharePoint/.NET Developer

Virtualization is often thought to be an infrastructure topic,  Recently, my colleague Liz Hatch and I presented a virtualization walkthrough to the Boston .NET User Group.  At KMA, virtualization provides us with the essential scale to support relatively dense server volumes with a comparatively small physical footprint.  We work principally on the Microsoft Virtual Server/Hyper-V platform, although most of the basic concepts are valid in VMware,  Xen, or other virtualization hosts.

At KMA, virtualization allows us to support multiple operational needs:

· Multiple Software development systems

· Testing environments that replicate an entire ecosystem, including login servers, database, web, mail, and clients

· Demonstrations and proof-of-concept environment

In a virtualized environment, a single server can simultaneously support one or multiple guest systems.

In Hyper-V, each guest OS hard disk is maintained on the host as a .VHD file.  As a result, its easy to snapshot, copy and rollback a server before or after deplying test software or running an new build.

IN addition to running Windows, there’s another possible approach.  Hypervisors provide the essential support for server hardware to support multiple guest operating systems, but without a host operating system.  Hypervisors make the most efficient possible use of the server hardware, since they don’t need to support a multiuser GUI, desktop applications, etc.  However, we’ve found the comparative ease of use of Windows 2008 R2 makes allowing our developers to self-support their own development environments. 

In large enterprise environments, production systems operate under stringent control to support high availability.  However, this can mean that lower availability is thought to be OK for test or development regions.   Hyper-V makes it easy for a developer to maintain, apply, and rollback their own virtualized images without relying on an overextended (or nonexistent) infrastructure support team.  For developers, the “dev region” IS production.

In our virtualization design, we emphasize centrally hosted virtual servers as opposed to Windows 7-hosted Virtual PCs.  The ubiquity of high speed Net connections, along with multiple VPN connections make accessing our central server core from the office, home, client site, or mobile location seamless.  (For those few locations without remote connectivity, we make light use of local Virtual PC hosts for self contained demonstrations.)  Central servers provide better security, backup, power, memory usage, and performance.

In addition, multi-instance applications like Microsoft Office SharePoint Server 2007 (MOSS) or SQL Server 2005/2008 allow us to achieve even greater server density on each single server.

In a departure from past practice, we no longer name guest servers to designate their host OS.  (I had once used designators like VDEVSQL08-H03 to show that the server was hosted on “H03”.)  But guest OS are no longer tied to a single host.  Modern virtual machine management, like VMware vMotion System Center Virtual Machine Manager, make it easier than ever to switch a running guest OS to a new host on the fly, with no down time.  As a result, hard coding the host name into the guest system name is no longer relevant.

Even with greater server density, multiple guests systems can rapidly consume more RAM than the host system supplies.  Developers need to designate always-on vs. on demand systems, and work cooperatively to temporarily shut down or suspend unused systems.

I’ll have more on virtualization tips and tricks in a future post.  Until then, thanks for reading.