Ten years ago, Microsoft revolutionized business intelligence (BI) by making OLAP accessible to the IT professional right in the SQL Server relational database product. Now the company wants to make BI available throughout an organization. What’s next for BI? Tom Casey, the Microsoft general manager for SQL Server BI, met with SQL Server Magazine’s Technical Director Michael Otey and Executive Editor Sheila Molnar to discuss Microsoft’s BI strategy—the history of BI, its vital role in the SQL Server 2008 R2 release, and its future. For an in-depth look at the SQL Server 2008 R2 release, watch for our SQL Server 2008 R2 new features article in the June issue of SQL Server Magazine.

SQL Server Magazine: When we spoke with Donald Farmer in 2009, we asked him about his background, and we’d like to start there with you as well. Did you move from being a relational database guy to a BI guy? How did that happen?

Tom Casey: I was a file server database guy for a long time. I worked on embedded systems, and then I transitioned to relational databases prior to coming to Microsoft to work on SQL Server 7.0. I focused on our relational technologies, on replication, and on distributed computing. Working on file systems and embedded systems early on, I got the notion of connection to the end user that’s critical for BI. So when the opportunity came to work on making accumulated relational data \\[available\\] to more users, which is a big part of BI, I jumped at it. It’s an exciting convergence.

SQL Server Magazine: Can you tell us more about your role as the leader of the cross-group BI effort at Microsoft? Can you describe how this virtual team works and how that effort is progressing?

Casey: We focus on making BI pervasive and available to everyone. We want users to get value out of the data that they already have, but we also want to turn that data into broadly available information that informs daily business productivity. We don’t want to deliver a suite that requires specialized training and tools for BI; we want to deliver something that’s ingrained in the tools that people use every day. Not just the tools that the information worker uses in Office, but also SharePoint as the infrastructure that the IT professional relies upon, and SQL Server as a mission-critical platform for the whole thing. We don’t want specialization in the stack because it gets in the way of users consuming and getting value out of what they need.

At Microsoft, we’ve organized ourselves so that we don’t deliver BI in just one team but we do it in a virtual team comprised of leaders and developers in Office, SharePoint, and SQL Server. We drive a holistic virtual engineering team. First, we identified key leaders—general managers in the Access, Excel, and SharePoint teams; me in SQL Server; a marketing counterpart and a distinguished engineer, Amir Netz. We focus on BI as a whole for Microsoft, giving teams guiding principles on the necessary priorities. We work closely to make sure that what we’re delivering is consistent and compelling. There are end-to-end scenarios or experiences that guide what we’re doing in the SQL Server 2008 R2 release. We communicate those things broadly and make sure that we’re adhering to those principles. We’re shipping Office 2010 at the same time that we’re shipping SQL Server 2008 R2. It’s the first time in 10 years that we’ve shipped those two products together.

SQL Server Magazine: What’s Microsoft’s strategic thinking behind BI in SQL Server 2008 R2?

Casey: The strategic initiative here is to make sure we serve the needs of all the roles in the organization. The consumers of BI, the IT professionals that put the systems and infrastructure together, and the developers and analysts who build the solutions that people consume. We deliver it in the right place in the stack so that you have familiar tools and the right tools. We’re aligning release cycles and development efforts to make sure that BI comes through as a set of holistic experiences from Microsoft. It is a key thing that will differentiate us from what other organizations do.

SQL Server Magazine: How does an organization get started with BI? Do they have to become OLAP experts?

Casey: There’s a much faster path available for them. Requiring the whole organization—the IT professional, the developer, and the end user—to become experts in a tool that’s specialized for an OLAP engine is an inhibitor. The SQL Server 2008 R2 PowerPivot add-in for Excel lets users simply start working in Excel, an environment that they’re familiar with. We’ve masked the fact that there’s a very powerful OLAP engine underneath. People will start with experiences that are ingrained in their daily work, and they won’t have that abrupt sense of making a transition to becoming a BI developer or BI user.

Making this seamless is key to our information platform vision. The second thing that provides a faster path is reporting. In the past, reporting has been separate but related: You build your application, and then you build your reports. You never got any leverage between the two. By driving to a common model, user-accessible tools, and a shared platform infrastructure for reporting and OLAP, we made it possible to build a BI solution and generate a report off of it with the new Report Builder. And vice versa—you can take a report that’s off of relational data and turn it into a data feed that lets you do rich analysis.

SQL Server Magazine: We’ve seen a huge uptick with our readers using SQL Server Reporting Services. The same thing could work with PowerPivot for Excel.

Casey: They can’t be separate and isolated experiences. You have the appropriate tools—Report Builder to design and create reports and present information broadly to users, and Excel to design solutions building and analysis. They need to be complementary and interoperate well.

SQL Server Magazine: How is Microsoft’s vision for BI different from that of its major competitors?

Casey: Our vision is about breadth of adoption and supporting higher-end utilization. Where we differ from some competitors is that we’ll rely on our rich partner ecosystem to build and deliver the specialized solutions that people want in vertical markets. That’s very different from other vendors who focus on specialized tools and applications. We focus on breadth: We want to make it possible to drive BI from 10 to 20 percent of users in an organization gaining value from their BI tools toward something that’s in the range of 40, 60, or 80 percent of people.

SQL Server Magazine: That opens up opportunities for smaller companies that want to get in this game.

Casey: SQL Mag readers, IT pros, as well as developers benefit from organizations that standardize on the BI platform as a part of their infrastructure. That’s part of what we’re driving at with SQL Server and SharePoint. We want to make BI part of the infrastructure and a set of services that are available for any app you build and consume in an organization. By optimizing for that, we’ll allow that ecosystem for our partners and custom application builders.

SQL Server Magazine: Is this approach to BI something that can promote career growth throughout an organization?

Casey: I absolutely think it’s a career-growth opportunity for IT professionals. For more than 10 years, we’ve been asking the SQL Server community to develop expertise beyond the relational engine. Now we deliver on the vision that BI is really part of the infrastructure that enables new capacity and new classes of apps, and the expertise that they’ve gained by cross-training themselves suddenly allows them to enable new classes of agility and decision-making.

SQL Server Magazine: What opportunities are available for BI developers?

Casey: There are two classes of developers in the BI realm. We’ve got to serve both and merge them. One class is the people who build the specialized BI app—the traditional budgeting, planning, forecasting, and scorecarding application. And then there’s the other class, where there’s opportunity around data visualization and reporting. We’ve taken steps to enrich the data platform at the same time as we’re building the BI tooling. Look at data visualization, charting controls, and gauge controls. They’re available in the Report Designer for the developer who wants to write a report that generates RDL that uses those components, but they’re also available in .NET, and they follow the same common .NET programming model. You can access them in your project system normally just by referencing those assemblies. Developers can take an incremental step, one that’s small compared to the value they get from doing reporting that’s well-integrated with their .NET applications. That provides a foundation where the semantic models they’re building and the skills they’ve developed are transferable to custom or traditional BI applications. We’re going to keep investing in that.

SQL Server Magazine: So would those things involve Windows Presentation Foundation (WPF) and Silverlight

Casey: If you want to hint at futures, I think your readers would like that. I’m a big believer in Silverlight and WPF and carrying those things forward. I think you’ll see some investments in that direction.

SQL Server Magazine: How about BI for SharePoint?

Casey: You and your readership know that SharePoint has become a very critical piece of data access, access to BI information, and delivery of that information. More and more organizations are asking how to deliver BI through SharePoint. Last November was the first time we had a BI presence at the SharePoint Conference. We were mobbed on the show floor. That audience has already figured out that reporting is integrated and is taking advantage of it. Just as in the old days when we used to expect the systems administrator and the DBA to work together, the SharePoint administrator and the DBA are becoming two critical pieces that fit together and overlap very well.

SQL Server Magazine: You’re saying that many of your inquiries are related to SharePoint?

Casey: Some people see SharePoint as a way to get to data. The platform underneath that stores data, allows you to access it, and provides enrichment services, like analytics, is SQL Server. What we’ve done in SQL Server 2008 R2, and what we’ll continue to do, is drive an integrated set of experiences. I want the IT pro to know that SQL Server is key, and that SharePoint is key to the BI story.

SQL Server Magazine: A lot of our readers are interested in surfacing management dashboards through SharePoint for SQL Server BI information and reports.

Casey: That’s where it starts. With PowerPivot integration, not just with Excel but with PowerPivot for SharePoint, we give the IT administrator the ability to gain visibility into and exert some control over those reporting applications. We make those manageable through the SharePoint environment with the PowerPivot add-in for SharePoint. We make it possible for the IT admin to understand how data is being reused in the organization and create a cycle for turning data into a new information asset that is published back to SharePoint, which is implicitly backed by SQL Server Analysis Services.

SQL Server Magazine: Can you explain what Master Data Services is all about?

Casey: The addition of Master Data Services to the integration portfolio that underlies BI is brand-new with SQL Server 2008 R2, and it’s getting a lot of good initial buzz and traction from our user community. One of the most common things organizations run into when they try to get consistent reporting done over a period of time is the fact that hierarchies, dimensions, and perspectives change. For example, say you’ve acquired another company that looked at the sales regions slightly differently than you do. That creates a mismatch and a boundary that you can’t report across. Master Data Services lets you map and create a master view that defines the hierarchies you want to project and work with going forward.

You have the model that you want to present to the organization, and you map that to one or many different models underneath. That lets you look at more dimensions and more attributes in a very consistent way. You can create an infrastructure using Master Data Services that gives you a consistent view of not just the schema that you want to present (the hierarchies you care about) but also the data that’s relevant for those things. We’re providing the infrastructure that’s necessary for people to build master data management solutions, which are almost always custom to each organization. And hence we talk about Master Data Services just like we talk about providing SQL Server Reporting Services, which lets people build reports.

SQL Server Magazine: Would you call this a data integration product?

Casey: It’s part of SQL Server. It’s another service in the box that’s very closely related to SQL Server Integration Services \\[SSIS\\]. In fact, it uses SSIS for data movement. It’s closely related but it’s a discrete and separate service. We’ll continue to add more services as it’s relevant to enrich the data platform.

SQL Server Magazine: Let’s discuss the Parallel Data Warehouse. Do you see any challenges with getting people to buy an appliance along with the software?

Casey: I see it as an opportunity. We entered the reference architectures business in the SQL Server 2005 release where we announced partnerships with HP, Dell, and others. We refreshed those partnerships with the SQL Server 2008 release, and we’ve announced even more people working with SQL Server for the reference architectures or appliance-like offerings on our Fast Track Data Warehouse solutions. Those solutions are very popular with customers—the complexity of getting high-scale systems matched between processing hardware, memory, and disk layouts, along with properly configuring the software in it to run on those things, takes time for IT professionals. It’s a big up-front cost that carries most of its cost one time.

The partners are optimizing themselves and working with us to optimize SQL Server running in those configurations. Those vendors provide dedicated solutions that just work out of the box, so you’re up and running quickly. Where parallel data warehousing comes in is the ability to scale out. To take an incremental step to expand, you add another appliance-like node into the mesh. It extends and adds your ability to scale out as well as scale up, which SQL Server already had. Over the last couple of years, we’ve worked closely with our partners around the total package solution and how that lands in the data center. This lets you get better TCO in that total package.

SQL Server Magazine: Is the Parallel Data Warehouse targeted at a certain size of organization? And when a customer has support needs, to whom do they look?

Casey: Both great questions. In terms of the size of the organization: It’s less about size in terms of people or even revenues, and it’s much more about data. You can have a fairly small data clearinghouse that has huge volumes of data. Look at web analytics companies. SQL Server scales up very well today, as evidenced by what folks like PREMIER Bankcard are doing with teens of terabytes of data deployed in their environment currently via scale-up with SQL Server 2008. Into the tens of terabytes, we do alright. But if you’re in the many tens or into the hundreds of terabytes or a petabyte of information, those cases are where parallel data warehousing is the preferred solution. You can use a Fast Track solution as a spoke or a node in your Parallel Data Warehouse topology as well in order to provide some domain-specific focusing of data that you want to deliver for a particular department’s usage. We’ve been very deliberate about taking that acquisition and normalizing it on Windows and SQL Server so that it follows the same semantic and has many familiar manageability characteristics for our users. The roadmap for moving forward on both scale up and scale out is very clear, and we continue to articulate it to customers and partners.

In terms of whom they call: We ship the software and we stand behind that software. And we’ve set up the programs very carefully so that when they call us they’re getting the right class of escalation that will support their solution and any hand-off that needs to be done in either direction, regardless of where the call originated. Those hand-offs with partners are as seamless as we can make them. It’s about enterprise commitment.

SQL Server Magazine: You mentioned the Fast Track Data Warehouse; would you describe it for us?

Casey: Fast Track Data Warehouse is a moniker around a set of reference architectures. They range from the single terabyte offering all the way into the many tens of terabyte offerings; 48 terabytes being the largest. They’re available from multiple different vendors: HP, Dell, and IBM.

SQL Server Magazine: Do the vendors have different packages? How do the customers buy them?

Casey: With the Parallel Data Warehouse, as it has been with Fast Track, we’re offering choice because we know some of our customers have preferred vendors. They’ve standardized on them, and we want them to be able to continue to do that. If you’re an HP shop, continue to be an HP shop. If you’re a Dell shop, continue to be a Dell shop. If you’re an IBM shop, continue to be an IBM shop. We don’t want to offer a whole separate solution. That’s why we don’t roll out a Microsoft-branded and stamped piece of hardware. We know people have good relationships; we want them to use those relationships and offer them the right capability that’s optimized.

SQL Server Magazine: Shifting gears a bit: Can you give us your perspective on using BI with the SQL Azure platform?

Casey: We recently made some announcements about Windows Azure and SQL Azure: Starting in January, SQL Azure is broadly available for public deployments. Our roadmap says that we’ll continue to deliver additional platform services in that data platform tier under SQL Azure. The first offering is focused on the relational capabilities; subsequently we’ll add reporting and analytics. We’ll add BI in a way that it surfaces in the right places. So you may see end-user experiences surfacing through Office online, and the back end of that being served out of analytics running in SQL Azure.

SQL Server Magazine: So will these announcements be forthcoming this year?

Casey: This year, we’re focused around what’s coming with the compelling set of releases in the first half of the year: Office 2010 and SQL Server 2008 R2. We’ll begin talking about the roadmap for the next wave of things that we do in the cloud. In BI, most organizations are still struggling with getting their data together and making it usable inside the firewall.

We’re just seeing SQL Azure take off—the ability for someone to provision a database in the cloud and be up and running. Suddenly it’s there, and you don’t have to publish it to anybody else and move it around. It’s available from anywhere from any device. And it’s hugely powerful. Now bring that to your reports. Now bring that to your analytics. It’s just a natural evolution.

SQL Server Magazine: It’s a new year and a new decade for BI. Based on the past 10 years, can you venture a few predictions about where BI will be at the end of the next decade?

Casey: We were the first vendor to integrate OLAP Services right next to the data store. We did that with SQL Server 7.0, and we still have the leading OLAP server on the market. Good things have happened: There’s been consolidation of tools and consolidation in the market. But BI is still only reaching 10 to 20 percent of users in an organization. I’m pleased, but I’m not at all satisfied. The future’s going to be about delivering not just capabilities and applications to an organization, but rich, compelling experiences. I’ll be greatly disappointed if we end this new decade without BI at least having doubled in terms of its value to an organization and the number of users it reaches to take advantage of it. I won’t be surprised if people are accessing BI through any application that’s delivered anywhere, whether it’s on the cloud, on premises, or on a device. And they won’t even know that it’s BI. I think 10 years from now we’ll be talking about BI as a natural part of business productivity and application usage. Today, we’re still talking about a separate thing called BI. You need it to become part of the infrastructure of IT and part of the implicit IP \\[intellectual property\\] of information workers. It needs to become how businesses are agile and productive.

I think we’re on the cusp of a major change right now. It’s a pivotal point. It harkens back to your question about SharePoint earlier, where people have accepted applications and information in their organization as coming through a portal and not littered across a bunch of file shares or discrete applications. It means that there’s a place where users connect to users and users connect to data. Now we can drive a different kind of transformation—from where BI was something separate toward where it is truly integrated in applications. And that’s exciting to me. That’s what our mission really is.

SQL Server Magazine: It looks like we have another exciting 10 years to look forward to!