Microknowledge

SharePoint Conference 2011 – Day Two Wrapup

Welcome to Day Two of the Microsoft SharePoint Conference 2011.   Tons of content yesterday, and there are two emergent themes here – the pervasive importance of Project Server 2010, and all the amazing core and business intelligence functions in SQL Server 2001 “Denali”. 

Session Highlights

We’re off to Disneyland now, but here’s today’s quick recap:

FAST Search Sizing

Barry Waldbaum and Steve Fowle led us through an overview of Microsoft; own experience engineering a huge Shared Services farm for their in house FAST implementation, published to many consuming farms internally at Microsoft.  Here’s a picture:

Some quick scaling rules

• Run SQLIO to test hardware, NAS is usually too slow for optimal performance
• For database sizing:

o   Web needs 4.5GB per million docs

o   Crawl needs 3GB doc per million

o   Indexers need 120GB per million docs

o   Rule of thumb 10 million Office docs equal about 1.5TB of search database usage in FAST

Deploy multiple server rows to ensure high availability.  The Admin component is not fault tolerant, automatically, and when it goes down new content isn’t added to the index, but the role can easily be manually failed over with PowerShell.  Most roles are stateless and failover automatically – except indexing, which requires Manual failover.

Document processing, Indexing and Link Analysis are hardware intensive and good candidates for dedicated HW in big enviroemnets.

There are backup tools “in the box” – they use  PowerShell remoting and SQL 2008 Management Object package.  But practically, SAN tools are more manageable, since you need to suspend search during a backup!!!!!

SQL Denali 2011 Reporting Services (SSRS)

Carolyn Chau, Microsoft SQL Prgraom Manager, led us through SQL Server Reporting Services (SSRS) enhancements in Denali.

Microsoft understands that self service functions were dropped from SQL Server 2008 R2 to ensure an on time release.  However, user self-service reporting is back and completely revamped as the “Crescent” project to be release as part of Denali.  Denali enhances Excel BI and delivers:

• Better SharePoint integration by rearchitecting integrated mode as a SharePoint 2010 Shared Services Application (SSA).  Reporting Services admin happens in SharePoint Central Administration.
• Making use of Denali’s new “Always On” Failover mode
• Integrating report-based data aware alerts via the SharePoint integration
•  Business Inteligence Developer Studio “BIDS” is brought up to date with the Visual Studio 2010/2011 Dev10/11 updates (no more reliance on VS2008.  This got an ovation.

Performance – SSRS SharePoint integrated mode is now as fast as native mode, and30-60% faster than R2. 

User alerts – but I’ll have more to say on this tomorrow!

Crescent, the new self-service BI interface, is a jaw dropper – live timeline slider animated bubble charts, etc. 

 

Some key points:

 

o   It requires Silverlight 5

o   Tightly integrated with PowerPivot data sources

o   Presentation ready at all times

o   No out of box PivotViewer integration in Crescent

o   Web part integration to SharePoint isn’t ready yet in CTP3 but will ship later

o   No direct access to SharePoint lists but those can be integrated into a PowerPivot model and then accessed via PowerPivot

 

JQuery to Enhance SharePoint without Code on the Server

Eric Harlan & Mark Rackley  were a knowledgeable and entertaining duo.  Quick summary about the reasons for and against jQuery as a tool on SharePoint:

Pros

o   Quick solutions

o   Powerful satisfied users

o   Rapid prototyping

Cons

o   A lot of easily found untested code in the wild

o   Debugging hard at best

o   Performance degradation

o   Inconsistent compatibility

SharePoint Conference 2011 – Day One Wrapup

Welcome to Day One of the Microsoft SharePoint Conference 2011. 

It’s already been a fantastic conference. I arrived here yesterday morning.  I got to spend a little time with family here in LA during the day yesterday – I was lucky enough to get a ride on my brother’s neighbor’s personally owned fire truck on my way down here.  (A long story.)  Hey, I got to SPC11 (in part) on a firetruck!

 

This is an interesting event – there’s no new release of SharePoint, so we expect a deep review of how SharePoint usage and enhancements.  Also, a lot of tablets here – iPads, and a few lucky Slate Windows 8 tablets. 

I already posted a summary of this morning’s keynote address, led by Jared Spataro with Jeff Teper and Kurt Delbene.  Here are some highlights:

• Expect a public preview of the next version of SharePoint at the SharePoint Conference 2012, November 12-15, at Mandalay Bay in Las Vegas, Nevada.
• 125 million SharePoint Users worldwide currently.
• The next version of SharePoint has the largest development history in SharePoint history to support v15 (the next release) and a new prelease of Office 365. Expect a lot in user interface enhancements and branding.
• Microsoft is launching the Microsoft Certified Architect (MCA) program for SharePoint
• Office 265 will support full two way Business Connectivity Services (BCS) connections to web-service based data by year end
• New Microsoft Visio server architectures visualization tools to be available this week for download from Microsoft.com
• New scaling and architecture whitepapers were used to support 14TB content databases with full failover of a live 15 server farm under load running on SQL Denali CTP

Session Highlights

Here are highlights from a few sessions I visited:

SharePoint for Knowledge Communities

The best take away from this is to organize your intranet around functional “communities” rather than traditional department sub sites.  Jim Kane from Paragon Consulting highlighted the following:

• Knowledge communities have to breakdown challenges of wikis and folders – they are repositories, not communities
• Knowledge hoarding – people pull in far more information than they push back
• Tacit information = information that is needed but not actively sought
• Tacit information is the core problem – how do you get information into the right hands without someone having to search for it.
• Organize your taxonomy around content repositories by project, practice, or topic centric
• Don't create department sites and throw a template at it - create a community
• Don't make it an -IT project
• Use a community to introduce mysites

Always On in SQL 2011 “Denali”

Bill Baer from Microsoft did a fantastic job showing how simply live failover is configured in SQL 2011 "Denali”  The best part isn’t just “disaster” failover, but graceful failover to smoothly transition between servers from planned maintenance, etc. “Always On” mode is setup via easy wizard.

But wait, there-s more – on October 6, Microsoft will release a new PowerShell command line builder – the PowerShell Design Surface.  It looks like Windows 8/Metro, and will be available from the Windows PowerShell Resource Center soon.

 

Developing and Extending Enterprise Content Management Features

Paul Swider led this session – and it was great to see such a big crowd for a great friend to the SharePoint community.

 Paul led through a series of practical tips and demos:

• You can easily customize a site and save the template to the SharePoint solution Gallery – then the template can be exported as a WSP and imported into Visual Studio 2010 for further enhancement
• The DocumentID function provides a static URL that can always reference a document by ID regardless of its physical position.  It is coded by referencing the Office.DocumentManagement library in the ISAPI folder of the SharePoint root/14 Hive.  It uses the server object model.  Paul showed a great implementation trick – implement GetDocumentURLs by ID to return the documents URL reference. 
• In addition, the end user web part that allows user to type in a document ID directly for retrieval can itself be overridden.

Other Stuff

A final note – I saw in the tweet stream that the Productivity Hub – a series of free end user training tools for SharePoint that can be added directly to your farm – is available for download now, and will be available for SharePoint 2010 SP1 in November from  http://www.microsoft.com/download/en/details.aspx?id=7122

Thanks! 

Welcome to Day One of the SharePoint Conference 2011 – Fantastic Keynote Summary

Executive Highlights

• Expect a public preview of the next version of SharePoint at the SharePoint Conference 2012, November 12-15, at Mandalay Bay in Las Vegas, Nevada.
• 125 million SharePoint Users worldwide currently.
• The next version of SharePoint has the largest development history in SharePoint history to support v15 (the next release) and a new prelease of Office 365. Expect a lot in user interface enhancements and branding.
• Microsoft is launching the Microsoft Certified Architect (MCA) program for SharePoint
• Office3265 will support full two way Business Connectivity Services (BCS) connections to web-service based data by year end
• New Microsoft Visio server architecture visualization tools to be available this week for download from Microsoft.com
• New scaling and architecture whitepapers were used to support 14TB content databases with full failover of a live 15 server farm under load running on SQL Denali CTP

(Word meme of the day – “fantastic”. Expect that word throughout the week.)

It’s already been quite a conference – maybe even fantastic! I arrived here yesterday morning.  I got to spend a little time with family here in LA during the day yesterday – I was lucky enough to get a ride on my brother’s neighbor’s personally owned fire truck on my way down here.  (A long story.) 

 

This is an interesting event – there’s no new release of SharePoint, so we expect a deep review of how Microsoft and the community have SharePoint 2010’s capabilities into practice.   And in addition to fantastic, the other new thing here is tablets – a decent number of iPads, and a lucky few attendees carrying Slate Windows 8 tablets. 

OK, the stage is set, the backdrop looks like a Mango-colored Windows 8 screen, and here we go!

 

Jared Spataro, Director SharePoint Product Management

Jared just took the stage dressed casually to host the keynote.  The theme of this year’s conference is Productivity Delivered.    He reflected on the history of SharePoint 2010 and revisited his three hour demo of SharePoint 2010 at the 2009 SharePoint Conference in Las Vegas.  (Remember Huey Lewis and the SharePoint wedding?)

 

Here are some highlights of the last few years:

• SharePoint is the fastest product to reach the billion dollar level at Microsoft, and grows continues at double digit rates
• 125MM licenses to 65,000 customers
• As a standalone business, SharePoint would be one of the top 50 software firms in the world
• Enterprise deployment has reached 67%, and is #1 in satisfaction awareness, recommendations and market share.
• Office 365 was launched

Wow, he just mentioned SharePoint Saturday and SharePint – is that the first ever official acknowledgement of those community events?

Anyway, he’s showing a short film about finding a celebrity and auditioning celebrities

• Luke Perry struggles with the name
• Carmen Electra tried to invent a tag line, and Jared just rejected her.
• Alan Thicke doesn’t know what an API is, or the Interweb!
• Arpan Shah (product manager for MS Project)
• Florence Henderson just was escorted from the audition while singing the Brady Bunch theme

Jeff Teper, Corporate BP Office Business Systems

And our celebrity keynote is Jeff Peper!  He’s retelling the history of a SharePoint presentation in Redmond to a huge enterprise client, and the rise of the phrase “I didn’t know SharePoint could do that!”

 

Redefine collaboration

Self service is key – IT deploys SharePoint as a utility so users can create their own sites.  Need to balance user empowerment and technical governance. 

Self-service BI is a key differentiator for SharePoint 2010 – especially with PowerPivot.  He also talked about a large bank that makes tremendous use of Access Services, which, to be fair, is somewhat neglected in the SharePoint functional space lately. 

Social fabric began with MySite, as an alternative to personalizing the home page.  He’s really excited about the Outlook Social Connector, which brings collaboration that may start in Outlook into the SharePoint sphere.

Also, part of the vision should extend collaboration outside the firewall to clients and partners.  In this realm, SharePoint for Internet Sites (FIS) has been a great takeoff success.

Now we’re reviewing SpaceX – the private rocket company – and how they use SharePoint for marketing and collaboration.  Interesting screen shots – SpaceX uses Document Sets, manual KPIs for tracing project milestones, and ratings on all project documents.  They also use partner sites to collaborate outside the firewall.

Here comes the big bet for the future!  Microsoft currently has the largest development history in SharePoint history to support v15 (the next release) and a new prelease of Office 365. Expect a lot in user interface enhancements and branding.

Unleash the ecosystem

There are 750,000 SharePoint developers active in the ecosystem.  He’s announcing BCS connectivity from the cloud to Web-based services for data access as part of the Office 365 release later this year.

Integrators and clients want to know who the deepest black belts are to design SharePoint so9lutions.  To support this, Microsoft is launching the Microsoft Certified Architect (MCA) program.

The next version of SharePoint will run all code developed for SharePoint 2010 seamlessly.

Build a world class platform

Currently, there are 1000 people in SharePoint R&D while the engineering has grown from single server to huge farms.  Spotlights have included the acquisition of Groove Networks and FAST to enhance synchronization and search.  But the support for tuning and administering the system has also grown.  Jeff introduced Richard Riley, Director of SharePoint to demo scaling, performance and high availability.

Richard’s demo environment:

• Content for the demos was taken from Wikipedia --  14.5TB in 108MM items retrieved from Wikipedia in two document centers
• Two SQL clusters running Denali CTP3 in Always On mode, VM Host with 80 cores and 1TB RAM
• 5 web front ends
• 5 FAST Search Servers
• 2 Admin/FAST Servers
• 2 Application servers
• All running on a big EMC SAN and NEC servers

 

 

Richard showed this using a new Visio SharePoint visualization template (not yet available from Microsoft.com at press time!)

 

On his test bed, he has 7500 live simulated users, to represent a user installed base of about 150,000 given average concurrency.

(Quick side note ‘#” in FAST search returns all results in the index!)  A ’#’ search in FAST retrieved all 108 million items in about .02 seconds.  Wow.  Or should we say, fantastic!

And it’s time for a high availability demo of fully replicated failover  of all databases and the server farms…under load, 14TB, full farm replicas.  And the volunteer has pulled the primary network cable… disconnected and fully reloaded inn less than 60 seconds.  Again, fantastic!

Kurt Delbene, President, MS Office Division

Social collaboration and remote work have become mission critical business requirements.  80% of enterprises users are engaging or planning for the cloud.

His major themes for future products will address these concepts:

• Best productivity across PC phone and browser
• Cloud services should be always available
• Deliver a unified client and server platform

Kurt closed by introducing Bill Bridley, the CEO of NetHope, a global umbrella social services provider.  NetHope uses Office 365 to unify response teams for disasters like the Haitian earthquake.  And Microsoft is distributing $50,000 to distribute via NetHope to user-sourced charitable organizations.  There’s a live HTML WebPart Office265 applkicvation to supply the user voting interface on the MySPC conference user portal, and we got a peek under the hood of the SharePoint app integrated from Azure with JavaScript and HTML5.

Finales

Jared’s back.  One last thing - the next SharePoint Conference will be next year, 2012, back in Lax Vegas at Mandalay Bay November 12-15, 2012.  If the past is any guide, expect the SharePoint v5/2012/Wave 15 public preview to debut there.

Using SSRS Reports as a PowerPivot Data Source – Kerberos Free Edition

When developing SharePoint business intelligence solutions, most of the time the data connections are fairly routine.  However, I was recently building a PowerPivot solution where some of the data modelers, working in PowerPivot, live in a different domain from the source database and SharePoint farm. 

Also, this environment doesn’t use Kerberos.  Kerberos, as you may recall, is a security protocol that allows users to request and receive authorization “tickets” across multiple server hops – from machine to machine, and across domains.  Because of its occasional complexity, the protocol was named after Cerberus, the three-headed beast of ancient Greek mythology. 

  
Hercules captures Cerberus. (Hans) Sebald Beham, 1545. Public Domain

OK, let’s not slay any monsters today.  Let’s suppose we want to enable Excel users to develop PowerPivot models against a remote, cross-domain SQL data source for publication to a remote domain’s SharePoint farm.   How best to proceed?

Well, PowerPivot does have the ability to connect directly to a SQL Server Reporting Services report as its data source, in addition to remotely addressing the SQL Server.  So, here’s our successful workaround.

Add the data to a SQL Server Reporting Services report.

This is pretty simple.  In this case, we’re already running SSRS in SharePoint integrated mode.   And let’s assume we already have a library that supports Report Server Reports and Report Data Source files, and that our data source was built and published to SharePoint as an .rsds file.  When we set up the report, we’re going to grab all the data we think we want in the report.  We can filter later in Excel.

1.       Create a new report in SharePoint library by selecting New Documents | Report Builder Reports from the library Ribbon

2.       Select the Table or Matrix Wizard.

3.       On the next screen, choose ‘Create a dataset’.

4.       Select the data connection, and, if re-prompted, login.

5.       On the Design a query screen, select the source table or view and choose Next.

6.       On the Arrange fields screen, select all the fields and drag them into the Values section at lower right.  Choose next.

7.       Choose Next on the Layout screen (if you’re getting all the data and adding it to the Values section, there should be no options for grouping or subtotaling.)

8.       Pick any color scheme – it doesn’t matter to PowerPivot.

Your report is built!  Now save it back to the SharePoint report-document library, and remember what you named it and where it was saved for the next steps.  Again, we don’t really intended to run or print this report, so the formatting etc., doesn’t matter.

Create a PowerPivot model in Excel 2010

Click the PowerPivot menu item, and select the PowerPivot button in the Ribbon.  From there, select the From Report option in the Get External Data section of the PowerPivot window.

In the Table Import Wizard, click the browse button and find the SSRS report we saved in the first half of this exercise.  The report will render in preview mode – click Next.  The next step will show the table – the “Tablix” – we built in the report.  You may choose to eliminate columns from the Preview and Filter button, or click Next to open up the source data in PowerPivot.

And that’s it!  You can slice and dice your PowerPivot model as you like now, and publisn the finished models back to a PowerPivot library on SharePoint if needed.  Just don’t wake up the sleeping three headed dog on your way back to the SharePoint document library…

SharePoint 2010 – Show me your (document) ID!

One of the welcomed new features in SharePoint 2010 is the Document ID.  This feature assigns a unique serial number to each document upon being created or first uploaded. 

SharePoint has lots of Enterprise Content Management that can automatically route a document along multiple folders, sites, and libraries throughout its lifetime.   Throughout it all, the Document ID stays constant – providing a guaranteed way to get back to a document without remembering where it lives.  But how do you find out what the number is in the first place?  That’s simple, but let’s first walk through how to create them.

The feature is easily enabled in Site Collection Administration:

 

Once activated, you can also specify the format for the serial number prefix – and, as suggested, you should use different prefixes for different site collections to avoid duplicates.

Document ID’s can be viewed using the View Properties in SharePoint:

 
If you know the document ID, but not the location you can always retrieve the document (with the right permissionns) using a custom URL:

http://SITEURL/_layouts/DocIdRedir.aspx?ID=[DocumentID]

Or you can enter the URL using the search functions on the home page of a Document Center:

 
Again, all of this presumes you know what the document ID is in the first place!  The best way to do this is to insert the ID directly into the document:

The ID can then be inserted into any document:  In Word 2010:

• Insert
• Quick Parts
• Document ID Value.  For example, in the example below, you can see a document ID inserted into document footer.

 

Thanks again – and don’t forget to show your document ID!

It's the One Thing - PowerPivot for SharePoint 2010

(with apologies to INXS fans!)

Last weekend, I was honored to be asked to contribute to Christian Buckley's ongoinng SharePoint video series.  The One Thing collects a series of interviews with SharePoint community members about one thing they think is essential for people to know in SharePoint, and is sponsored by SharePoint ISV Axceler.

The videos are all nicely edited and scored - Christian does a great job of editing and posting them in less than an hour!  You can find the videos on YouTube or Axceler's website. 

My topic is PowerPivot.  PowerPivot makes it easy for users to query any standard database (SQL, Oracle, XML etc.), build an interface in Excel, and publish the dashboard for users in any supported browser - no local Excel required for web users!

 

Shabooh shoobah!  (Another INXS reference...)

Adding Microsoft Pivot Business Intelligence to SharePoint

Pivot is a unique graphical query interface released by Microsoft Live Labs in 2010.  Microsoft certainly seems to love the word – this follows in the wake of PowerPivot and PivotTables, two very different business intelligence subsystems. 

Pivot is entirely different.  At its initial introduction at WPC2010, Microsoft described it as “powerful, informative and fun”.  Fun?  Yes, I think it is!

Pivot lets you build an interface to items that can be represent with a graphical tag.  As you drill into the items, the pictures dynamically fly around the screen to rearrange themselves according to your query.  Pivot provides a local client, an Excel design interface, and Silverlight controls to allow you to integrate Pivot into your SharePoint site.  I’ve pasted a screenshot below – you can also “play along” with the interactive demos online at http://www.microsoft.com/silverlight/pivotviewer/

 

Let’s walk through building your first Pivot collection and viewing it through a Silverlight control from your SharePoint site.  Since one of my hobbies is digital photography (see www.flickr.com/photos/cmcnulty) I used Pivot to build a simple viewer for some of my favorite pictures. 

Step 1: Install the Excel Add-In

The simplest way to start building a Pivot Collection is to install the Excel Add-In from http://www.silverlight.net/learn/pivotviewer/download-excel-tool/ The Excel Add-In add the Ribbon control to allow you to create a New Collection.  What’s a collection?  A collection is maintained in an Excel spreadsheet, but is exported to Pivot as a .cxml file and related directory that contains all the related support fields and graphic element required by the Pivot interface.   You use Excel to maintain the data, and then users can query the data using the Pivot Collection.

Once you’ve installed the add-in, select New Collection from the Pivot Collections tab.  The Pivot template gives you two spreadsheets in your workbook:

• Collection Items.  This is the main screen where you will add image files and define the principal columns for data exploration.
• Collection Properties  The tab allows you to define collection-wide information, such as Collection Title, Icon,  Brand Image, Additional Search Text, and Copyright.

 
 

Use Import Images to browse for pictures you want to add to the Collection.  (You may note that some columns are hidden by default.)  Once the photos are imported, you need to start adding data for the existing columns:

• Name – A title for the image
• HRef – A link to follow if the image is clicked
• Description

There is pop-up help to remind you about each of those columns on the header line.  Most important, you can add additional columns to the right of the Description Column.  Here’s what our sample collection looks like after I imported photos and added several additional columns for State, Time (of day) and Year:

 

Once the collection is defined, you can play with it by using the Quick Preview option (which actually doesn’t show the images, just the query.

Sept 2: Publish Collection

First, save your Excel file – and it doesn’t need to be in the location where the finished collection will live.  Then, choose the Publish Collection item.  Pick a new local folder, and give the Collection .CXML file a name.  This process can take a few minutes while the engine parses and resizes all the graphical elements.  Once it finishes, your collection should open up in a new Pivot Viewer local window.  You can explore your data and test to make sure your collection is ready to share.

One of the things you may note is that you’re not limited to “check box” queries – if you define number values the Pivot query interface lets you use a slider control to filter a range of values.

Step 3: Publish to Web

The first thing to do is to copy the entire Collection directory to your SharePoint server – somewhere on the file system, outside of SharePoint.  There are a LOT of files, so this process can take a while.

In our example, I created a new C:\Pivot directory.  The entire collection goes into C:\Pivot\PivotPix. 

You’re also going to need to download the sample Silverlight Pivot viewer and viewer page from Microsoft (see http://www.silverlight.net/learn/pivotviewer/release-notes/) In our example, we put the XAP file (a compiled Silverlight app) and HTML page in C:\Pivot\Viewers.  We’re really taking a shortcut here – however, recompiling a hew viewer is pretty simple in the Silverlight 4 Toolkit.

Step 4:  IIS Configuration

Now the web server needs to know about the files we just added.  In IIS7 manager, I add two virtual directories:

• Pivot – mapped to the location of the viewer files
• PivotPix – mapped to the location of the CXML file and the rest of the collection

 

To do this:

• Right click on the web site where you want to add the alias
• Select Add Virtual Directory
• Enter the Alias and Directory as needed

 

Step 5: Edit the HTML File

There’s a lot in the sample HTML file, but the key line is:

<param name="initparams" value="initialCollectionUri=http://msshome2010/PivotPix/FavoritePhotos.cxml" />

Change the value of the initiaCollectionUri to match the “http://website/alias/nameofcollection.cxml” path for your local server.  (Substitute your own local server name and alias.)

Step 6:  Add links on SharePoint

Test the path to your edited HTML file – it should be http://yoursite/your-alias/PivotViewerSample.HTML [match to your own actual server names etc.] If you’ve set up the aliasing correctly, add a link in SharePoint and you should get your browser to render the Silverlight view to your collection:

 
 

What next?  There’s a lot more you can do.  You may start by adding custom columns to your collection.  Also, you’ll want to redesign and recompile the Silverlight viewer to patch your interface requirements  but we’ll leave that for another day.

Alternative Approach

Use the Silverlight Pivot Viewer for SharePoint on Codeplex http://pivotviewersp.codeplex.com/  This control is desgined to let you skip the Excel collection generation and IIS process entirely.  Instead, this solution consumes its images and data directly from:

• A picture library
• The User Profile List
• Site navigation

The web part is not hard to figure out once you’ve downloaded and deployed the WSP.  (Hint – although the documentation tells you how to install it using Stsadm, why not follow through on that New Years’ Resolution and use PowerShell:  Add-SPSolution and Install-SPSolution).

Anyway, once you’ve installed the part and added to a page, you can connect to a library, and get almost the same experience as you had using the HTML-hosted XAP file:

 
 

 

Have powerful, informative fun!

SharePoint Saturday Webinar Series - Business Intelligence in SharePoint 2010 - 8 Nov 2010

As a sponsor of SharePoint Saturday Boston, KMA is proud to help announce and kickoff the SPSBOS Webinar Series this Monday, November 8, 2010 at 11:45am EST.

 

http://www.sharepointsaturday.org/pages/spsbos3webinarseries.aspx

Sadie Van Buren (my colleague at Burntsand) and I will be reprising our presentation of "See Beyond The Numbers: Data Visualization in SharePoint 2010".  Its a guide to Business Intelligence tools and tips, from the Chart Web Part all the way through custom solutions leveraging PerformancePoint and mapping controls.  Register at https://www1.gotomeeting.com/register/970462616!

Most of this year's presentations are scheduled for online presentations - the schedule is on the SharePoint Saturday site.  Sadie and I are excited to bring this to you, and hope you can join us!

The Ultimate SharePoint 2010 Enterprise Content Management ECM Cheat Sheet

Hi again to everyone from SharePoint Saturday NY!  It was a great chance to get out with the SharePoint user community.  Great content – and flawless weather!  Midsummer in New York and I didn’t miss the beach once.  Ok, once.

Managed Metadata and Taxonomies in SharePoint 2010 remain a very popular topic, and I had great participation during my session.  (Presentation on SlideShare.)

Due to another speaker’s family emergency (unfortunately) I was asked to take over a previously announced presentation on Enterprise Content Management (ECM), Web Content Management (WCM) and Migration Strategies.  So I didn’t’ have as much time as I’d like to prepare -- but I did put together the ultimate SharePoint 2010 ECM “cheat sheet”.  Here goes. (Presentation on SlideShare.)

First, there’s a rich range of ECM features in SharePoint 2010.  Here’s a quick road map in order of increasing complexity and organizational maturity.  (Also, although some folks consider InfoPath and forms management to be an ECM feature, I’ve left it out for brevity.)

 

One general rule before we move through all of these – SharePoint 2010 now supports auto-generated document IDs.  When a document is first uploaded to SharePoint, the server assigns a unique identifier that follows the document throughout its lifecycle – even as it moves from site to site as part of workflows, organizer roles, or lifecycle management policies.

ECM Configuration Roadmap

Function	Feature Scope	Where to configure it
Content Organizer	Site	Site Settings | Content Organizer Rules
Hold and eDiscovery	Site	Site Settings | Hold and eDiscovery Section
In place Records Management	Site Collection	Context Menu | Compliance Details
Send to Records Center	Define in Central Admin	Central Admin: General Settings | Send To Connections
ILM	Variable	Content Type definition; or override at library/folder level in library settings – Information Management Policy Settings

Versioning and Metadata

Document versioning remains one of the oldest and strongest functions in SharePoint.   As users collaborate on a document, you can:

· Require the document be checked out to a user before edits

· Save a running copy of all draft and final published versions

· Require administrative approval before other users can see approved, published content

· Retain or delete prior drafts or published versions

In addition, you can – and should - use “Content Types” to establish sets of properties about a document – e.g.  Draft/final, presentation date, subject keywords, etc.  These types and properties will greatly help in using all the other features that let you manage content after it’s created and/or published.

Drop off library and Content organizer rules

In large, complex sites, users who occasionally write new content are faced with a dilemma – where should I post this?  They often pick one of two options:

· Post it nowhere for fear of “breaking” something

· Post it to multiple places – one of them is bound to be right.

The good news in SharePoint 2010 – users don’t need to remember all the “right” places to post a document.  Instead, by activating the Content Organizer, you can direct users to a single “drop-off library”, and then use rules to route the document to the correct location.  Rules are flexible, and can be driven by content types, document properties, titles, etc.  I’ve listed an example below for a rule to route audio files to a media library if they are copyrighted in 1997:

 
I can’t stress enough how important it is to be able to write rules based on properties, and not just Content Type.  In 2007, enterprises often had a proliferation of content types, or avoided them all together since that was the only way to route documents via Record Center, document retention policies.  The result was a separate content type for each department, subcategory etc. that might have a different policy.  The new rules sets for Content Organizer and related ECM functions are much more flexible.  Before, you would have needed to establish a content type just for audio files from 1997.

Finally, Content Organizer functions have to be activated as site features.

eDiscovery and Hold

Hold and eDiscovery allow an administrator or compliance staff to “freeze” a document it place, or route it to a holding area.  Holds can be implemented directly on a document.  You can also preview a set of search rules, and let the system add documents to a “hold” automatically (it saves an Excel report when done!)

This function can help outside of litigation support – we have clients who use this when they need to find and purge documents from cancelled deals – the documents are moved to a pending area and then deleted.

Records Management

Records management was included in SharePoint 2007, but the functions were so limited that they seldom saw enterprise use.  Essentially, you could copy content types to a new site collection and manage them as records there.  However, the original documents were left intact, and could be edited further.  SharePoint 2010 presumes you will do one of two things (most likely):

· Ship the document to a Record Center site collection and leave behind a link. 

· Declare the documents to be a record “in place” and manage it there.

Here’s another question – why?  Simple.  Once you’ve declared a document to be a record, you can more easily prevent the document form being deleted interactively, or exempt the record from other document retention policies.

Send to Records Center

Record Center is a custom, in the box site template that you can add to an existing SharePoint 2010 farm.  Once you’ve established it, you can define a custom “Send To” action for the document context menu (the hove).  These settings can be found in Central Administration | General Settings.  You can pick a target location, and decide whether documents can be copied, moved, or moved with a link left behind.

 

Records In Place:

You can also let users declare records interactively through the context menu for Compliance Details.  (This can also be a great use of a Ribbon-based Custom Action if you want to customize the UI).

 
  
Information Lifecycle Management (ILM) Policy

As documents age, you can define a lot of things to happen automatically via Information Lifecycle Policies.  Historically you could only define these on a per content type basis – but now you can also define this at a library or folder level.   These settings are reached from the Ribbon for Library Settings.

 

Polices can be applied uniformly to all content, or can be excluded for files already flagged as a “Record”.  You can define stages and test for properties – usually based on document creation, modification or approval dates.

Most importantly, what can you do to these documents?  A lot:

ILM Actions

· Move to Recycle Bin

· Permanently Delete

· Transfer to another site or Records Center

· Start Workflow

· Skip to Next Stage

· Declare Record

· Delete prior drafts

· Delete Prior versions

Finally – don’t forget to generate a File Plan Report – it rolls up a summary of all the retention policies that govern a given library:

Good luck – there’s a lot to learn for ECM in SharePoint 2010!

Microsoft TechEd 2010 - Day 3 Recap

Hello from summer-like New Orleans!  Last night was filled with fantastic food, explorations of the French Quarter, and introductions to many of the leading figures of the SharePoint community.   We got to watch the founding meeting of the New Orleans SharePoint User Group, and were generously hosted at the Quest Software party on Bourbon Street.  Well-earned relaxation, I think, since we’re all so tired walking around the Ernest Morial Convention Center.  Here’s a picture – note – the building runs well beyond the bridge in the distance:

OK, on to yesterday’s highlights.  It was another heavy day of business intelligence sessions at TechEd.

PowerPivot

PowerPivot is Microsoft’s new tool, released as part of SQL Server 2008 R2, which allows fast, in-memory access to multimillion row data sets in Excel 2010 and SharePoint 2010.  The Excel client add in can be downloaded at http://go.microsoft.com/fwlink/?LinkId=155905 [requires Excel 2010].  Once installed, you can work with SharePoint hosted PowerPivot just by uploading the spreadsheet to a PowerPivot Gallery on SharePoint.

There’s a lot of subtlety to troubleshooting the Excel client – make sure the add-in is enabled; and disable spurious error messages by adding the following environment variable:

VSTO_suppressdisplayalerts = 0

There are other techniques, like opening up the local in memory Analysis Services cube backup file and attaching in to the AS PowerPivot instance on SharePoint 2010 to open up additional diagnostics.  A great resource for more tips is http://powerpivotgeek.com/

SharePoint Adoption

My own session on SharePoint adoption strategies.  [I had a lot of fun!]    

SharePoint 2010 Document Sets

Andrew Connell had a great presentation on document sets in SP2010.  If you don’t know, document sets allow you to define and work with a collection of documents with common properties and workflow.  For example, you might define a product portfolio to always include at least a product overview, sales presentation, and pricing spreadsheet.   Here’s one I built – notice that KMAProductAudience is a property of the set, and each element has individual properties.

Equally valuable was the overview of using Visual Studio 2010 templates, which are much more SharePoint friendly than ever before.  There’s a compelling case for using declarative features to define custom columns, pages, etc. because the difficulty is greatly reduced – if it takes only 15 minutes to define a site column definition as a deployable feature across the farm, it makes maintenance late much easier.  Great session all around.

Project Dallas

Forget TechEd 2011 in Atlanta or SPC2011 in California – the future is in Dallas.  “Dallas” is the codename for Microsoft’s ambitious project to host public reference datasets on Azure – on a public domain and or subscription basis.  In their architecture, all the datasets are directly referenceable for download, browser view, access from PowerPivot in Excel and/or SharePoint.  Also, you can easily code .NET solutions to them using OData since they’re also URI addressable as WCF endpoints.

Enough geek speak – just visit at http://www.sqlazureservices.com.  For example, here’s recent information on global ice cream production from the UN on “Dallas”.

PerformancePoint Dashboards

Performance Point Dashboards are really simple, once you get started.  I sat in on a great walkthrough to create a live dashboard of twitter sentiment.  We used the following steps:

· Build a PowerPivot Excel chart to pull in data from AS cube – see a similar screen shot – and upload to SharePoint.

· Build a SQL Reporting Services (SSRS) report – use URL argument “rs:ClearSession=true” to avoid caching real-time data.

· Use Dashboard Designer to integrate the elements into a single web part page

· Add auto refresh filters to keep data real-time.

That’s all for now – onward through Day 4 and tonight’s closing party.   See you all when I’m back in Boston!