FAST Search for SharePoint promises custom search without custom code

Microsoft is trying to marry SharePoint admin with FAST’s search expertise for manageable search.

At the SharePoint conference last month we got more details about the new Fast Search for SharePoint offerings – the first integration of the industry-leading search technology it gained with last year’s acquisition of FAST.

Due in the first half of next year along with the other SharePoint 2010 versions, Fast Search Server adds richer search to SharePoint Server and FAST Search Server for Internet Business does the same for external Web sites built on SharePoint Server For Internet Sites. “SharePoint search is good,” claimed senior product manager Jeff Fried, “but FAST is better. Some things are in both SharePoint search and FAST, but they’re better in FAST and some things are only in FAST”.

Results from a FAST-enhanced SharePoint site will include thumbnail images of documents, PowerPoint previews you can scroll through on the results page to see if the presentation has the information you need without opening it, related documents, the option to sort by any document property specified by the site administrator and a visual ‘best bet’ highlighted at the top of the page, plus more detailed options for refining the search by category (with the number of results for each refined search – all customised for the role and user profile of the searcher. Sort by relevance and the ranking differs based on the role and user profile of who’s searching; an engineer will get different documents in the first results and different people suggested as relevant experts.

FAST offers sophisticated search algorithms like lemmatisation (searches for ‘better’ will include results for ‘good’), expansion of nouns but not verbs (searches for ‘book’ will match ‘books’ but not ‘booked’) and takes into account ‘social’ factors like how other users have tagged documents and which search results they’ve clicked on.

Fried said these tools are more sophisticated than SharePoint’s built-in search and FAST does more advanced content processing to identify dates, stock tickers and company names and generate other metadata automatically as it indexes documents. (All of that gets exposed in the new Backstage view in Office 2010, by the way). It also takes into account ‘social’ factors like how other users have tagged documents and which search results they’ve clicked on.

Many of these features were in the original FAST ESP product, which powers search on many commercial Web sites, like Deutsche Telekom and Dell, but they had to be coded by hand. Fried compares it to a race car. “FAST ESP is a high-performance machine that could do almost anything; it’s highly flexible – but it’s also highly engineered. High end search was a specialised machine that used specialised tools… it was not something you could manage without special training or special services. This stuff used to be really hard and we've made it so that if you can administer a SharePoint farm, you can do this.”

The search refinements use metadata (like author) from SharePoint, but they also use ‘enriched’ metadata generated automatically by FAST as it indexes documents, identifying entities like dates, stock tickers and company names. You can specify other concepts and products relevant to your company to use for refining searches and map them to properties that FAST discovers in your documents through the graphical interface or using PowerShell commands rather than having to write custom code.

The high-level Fast Query Language allows you to create complex search tools like location aware search in three dimensions that can take account of other information (like which properties your business owns) to sort the results. Or you could re-arrange items on a Web page as users select topics and people, creating what the director of SharePoint Server, Tom Rizzo calls “query-less search, where you're getting rich navigation, but not ever writing a query in the search box”.

FAST also uses the same APIs, development tools and management options as SharePoint, including System Center Operations Manager, so it looks like the rest of your SharePoint farm to manage.

Search in previous version of SharePoint has been far less powerful and the FAST acquisition was seen as a significant boost to Microsoft’s plans for search across its product range. FAST Search is one of the reasons Gartner currently gives SharePoint a leadership position in its magic quadrant for information access technology as “a formidable competitor” with “a particularly broad capability to index from a variety of content sources; these include records management applications that relatively few, if any, other vendors connect with”. The Google Search Appliance integrates with SharePoint and some other document libraries like ECM and it offers similar search algorithms, including social search, but it doesn’t offer the integrated administration, the breadth of data sources or the same customisation as FQL (though it’s not quite the black box it used to be).

Gartner analyst Whit Andrews told me he thinks the integration in SharePoint is promising: “Users appear to have easy access to Fast's capabilities through the SharePoint chrome. Using it in a sophisticated fashion will still require significant investment in effort and time, but more users will have access to it as a result of the SharePoint integration, and more users will be familiar with the controls and development levers and knobs.” But he struck a note of caution. “Complex search applications require committed information architects and developers to make their visions complete.”

Will you be looking at FAST to build powerful business search tools or looking to do ‘search without the search box’ or do you just want a system that builds the index and does the basics?
-Mary