A look at the Jim Gray Systems Lab, Denali’s columnar index feature, and more
Have you considered how the features found in the CTP of the latest version of SQL Server, such as SQL Server “Denali,” came to be? Someone has to come up with the ideas, design the features, and test them until they reach a high level of quality and can be considered for inclusion in the next version of SQL Server. Some of this research is done at Microsoft’s Jim Gray Systems Lab in Madison, Wisconsin. SQL Server Magazine’s Senior Technical Director Michael Otey and SQL Server Content Manager Megan Keller spoke with David DeWitt, a Technical Fellow in Microsoft’s Data and Storage Platform Division, who leads the lab. We discussed his work at the lab, query optimization, and how he sees SQL Server progressing in upcoming years.
SQL Server Magazine: David, can you tell our readers a bit about your technical background?
David DeWitt: I received my PhD from Michigan in 1976, joined the faculty at the University of Wisconsin-Madison that fall, and was a faculty member for almost 32 years before I retired and joined Microsoft in the spring of 2008. Along the way I built three different parallel data warehouse systems, including the first one that actually ran queries in the early 1980s, a second one called Gamma in the mid-to-late 1980s, and a third one called Paradise in the 1990s that we sold to NCR. I’ve been on the board of a number of startups, most recently Vertica. I joined Microsoft in March 2008 and I am part of the SQL Server organization, reporting into Quentin Clark, who reports into Ted \\[Kummert\\] who reports into Bob \\[Muglia\\]. So, that’s sort of the high-level background.
When I joined Microsoft, I was not given a specific charter. I was just told to go set up a lab and do interesting work. So far I have hired seven full-time employees, plus an admin. I have a small lab in Madison, Wisconsin, named after Jim Gray, a former Microsoft Technical Fellow who was lost sailing off the coast of San Francisco in January 2007, that’s affiliated with the university. There are also eight graduate students and three faculty members from the university’s database group involved in the activities of the lab. The lab is a collaboration between the university and Microsoft. So far it has proven really interesting.
SQL Server Magazine: What kind of things do you and your team do at the lab?
DeWitt: We’re involved in a number of different projects. We worked on some features for version 1 of the parallel data warehousing product, SQL Server 2008 R2 PDW \\[Parallel Data Warehouse\\], including analytics and semi-joins, and we are currently doing technology explorations for future versions of PDW. Even though I’m part of the development organization, my mission in the lab is to really work on features that may or may not be part of future releases of SQL Server. We’ve done some work on buffer pool extensions to SQL Server using solid state devices and are exploring how novel memory technologies, such as phase-change memory, might affect the design of SQL Server down the road. We are also exploring a variety of parallel database technologies.
Our mission is similar to a typical research lab, but with a slightly shorter-term focus. The graduate students involved pretty much do whatever they want to do under the supervision of their advisors. They are not Microsoft employees. They’re exploring a variety of ideas including doing query optimization with power constraints, text retrieval algorithms for parallel database systems, and advanced designs for highly scalable transaction processing systems. So the graduate students are being typical graduate students. The staff is focused on issues more directly aligned with the mission of SQL Server and technologies that can be used to improve SQL Server.
SQL Server Magazine: It’s pretty interesting that you use graduate students in the lab.
DeWitt: It’s an unusual situation; totally unique for Microsoft, and probably anywhere, in that the students aren’t Microsoft employees as I said previously. So Microsoft gives a grant to the university just as the National Science Foundation would give a grant. The graduate students are free to work on whatever interests them with their advisors. There is an IP \\[intellectual property\\] agreement in place between Microsoft and the university such that both sides benefit. The students benefit because they have funding that is not constrained by what their advisor proposed in a grant application to the NSF or DARPA. The students also benefit because, if they want to (and not all do, but about half do), they can actually take advantage of their ability to access Microsoft source code and actually do their research projects inside SQL Server. So that gives them access to a real commercial database solution. Microsoft benefits because we do get tiered rights to the patents, we get to look at the graduate students early on, and get early exposure to the creative new ideas that the graduate students come up with. So we’re looking over their shoulders, seeing what they’re working on, and giving them guidance as they go.
Pretty much everybody in my lab has a PhD, and some have experience beyond the PhD, so we get the privilege to act as collaborators and advisors, and we get the opportunity of seeing where their minds are taking them. All the best ideas come from graduate students—well, I shouldn’t say that, but more so than their faculty advisors. It is an unusual situation—it’s been an operation going on three years now, and both sides (we had a review recently) are very happy with the arrangement. We think it’ll give us a little commercial advantage, obviously. It pays back because a lot of people from Wisconsin have gone on to work at Microsoft, but we see a business opportunity, and the graduate students really benefit by having this sort of unlimited source of funding without any constraints.
