Paging XML Data to a Web Client

Consider the following scenario. Your customer has a Web-based reporting system that uses client-side Internet Explorer (IE) data islands. The current version pulls all the records to the client at once and uses Extensible Style Language Transformations (XSLT) on the client side to transform the data to a columnar format. This version also has client-side data sorting (by clicking on the field column). Your customer wants to change this system to one that pages the data to the client, 25 records at a time, to speed up the report's initial presentation—downloading and performing transformations on a few thousand records is too slow. But the customer, of course, still wants to sort the data by the field columns and to be able to view all the records.

When the report returns the records from the query, you could simply display the first 25 records. Then if a user pages ahead, you could display the next 25 records. But running the query every time seems like a waste of resources. You could use ADO to page the records down, but you need a stateless design (this Web site is, after all, WELL designed). You could use some fancy, elaborate, "smart" stored procedure that creates a temporary table for every user who runs the report and then perform selects out of the temporary table for the paged records. But that's a lot of hits to the database and a lot of maintenance to get rid of the huge number of temporary tables. I'm going to share with you an XML solution that I used to tackle this very scenario—using Windows 2000, SQL Server 2000, IE 5.0+, and Microsoft XML Parser (MSXML) 3.0.

The Solution


My solution uses a data island—not a client-side, IE data island, but a server-side file data island. Below is the pseudocode for the solution:

If this is the first run of the query then
     Run the query
         Save the resulting XML to a file on the server
         Set the start record number to 0 and the last record number to 25
Else if paging forward
     Add 25 to the start record number and the last record number
Else if paging back
     Subtract 25 from the start record number and the last record number
End if

If resorting then
     Transform the XML data island file with an XSLT file that resorts the records
         Overwrite the XML file on the server with the resulting XML
End if

If displaying all records then
     Use the entire XML data island file
Else
     Use an XSLT file to select the records between start record number and last record
         number from the XML data island file
End if
Send the XML to the client

Voila! Trust me, this approach is much simpler than it sounds. You simply need five hidden input tags on the report page that you post over:

  • a hidden input that holds the current first-record number on the page
  • a hidden input that holds the desired action (e.g., show first page, page up, page down)
  • a hidden input that holds the field to sort by
  • a hidden input that holds the sort direction
  • a hidden input that holds the sort field's date type

You need to either pass parameters to your Extensible Style Language (XSL) files or use the Document Object Model (DOM) to change the paging range of the pagination XSL file and the sorting properties of the re-sorting XSL file.

The pagination and the re-sorting code appear below.

PAGINATION.XSL

<?xml version="1.0"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:output method="html"/>
<xsl:param name="thestart"/>

<xsl:template match="*">
     <!-- only get some of the records from the master XML file -->
         <!--  the 0 and 2 below can be parameters passed to this XSL file -->
           <xsl:for-each select="row\[position() &gt; 0 and position() &lt; 2\]" >
           
           <!-- copy the records to a new xml output -->
              <xsl:copy>
                       <xsl:copy-of select="@*"/>
                           <xsl:apply-templates/>
                           </xsl:copy>
                </xsl:for-each>
                <!-- calculate the grandtotal for all records-->
        <grandtotal><xsl:value-of select="sum(//@Amount_Paid)"/></grandtotal>
</xsl:template>

</xsl:stylesheet>

RE-SORT.XSL

<?xml version="1.0"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:output method="html"/>
<xsl:template match="/">
     <xsl:for-each select="ROOT/row">
         <!--  the 3 attributes below can be parameters passed to this file -->
         <xsl:sort select="@ListPrice" order="ascending" data-type="number"/>
         <xsl:copy>
              <xsl:copy-of select="@*"/>
                  <xsl:apply-templates/>
         </xsl:copy>
     </xsl:for-each>
</xsl:template>
</xsl:stylesheet>

Discuss this Article 6

James Handley (not verified)
on Apr 5, 2001
I think that this is a good example of an alternative method of munipulating reporting data. I have myself created a similar application that does all of this except sort in a data island. I chose to return my records to the DOM based on the absolutepage and pagesize properties of the recordset object. The navigation bar reloads the page based on the page requested and transform the records in the DOM. I guess in my situation I felt that there would be a performance increase by not including a physical XML file. However, I will try this example out. I am happy to see SQL Mag share real world examples opposed to ones that work great in theory. good job JH
Anonymous User (not verified)
on Nov 8, 2004
To make the solution work on a web farm, couldnt you store the XML file on a file server, or as a field in a db table.. where the application clears the table, same as you would have to get rid of all the old XMl files anwyay.???
Anonymous
on Jun 1, 2004
Will this solution hold on a weblogic cluster if the cluster is homogenous and if store the server-side xml data island obj in a servlet instance ??
Jenna Jameson (not verified)
on Aug 30, 2002
This XML method is perhaps the sexiest solution to the problem in terms of girth and speed.
Anonymous User (not verified)
on Jan 17, 2005
In respect to the last question, storing the XML file on a file server or in a database table - this defeats the purpose of using XML for paging in the first place. Accessing XML from a file server (especially a very large XML file) is possibly going to cause more delays than accessing the data directly from the database. Alternatively, accessing the XML from a database table, you may as well simply populate the table with the original report data and retrieve specific 'rows' of data during each page event, rather than retrieveing and parsing the entire XML document each time. This approach is also mentioned at the top of the article as an approach that the author is attempting to avoid. The homogenous weblogic cluster with xml data island servlets sounds sexy, dunno if it'll work though.
Benny Sasson (not verified)
on Mar 25, 2001
Hi, This solution isn't scalable in a Web farm because you hold on the IIS the user state using the XML file. When the user page he actually can have other server that deal with the request.

Please or Register to post comments.

IT/Dev Connections

Las Vegas
September 30th - October 4th

Paul ThurottOur Experts will show you:
• Common SQL Server
Problems
• Best Practices for T-SQL
• SQL Server Integration
Services
• Database Development

Come See Michael Otey & Tim Ford in Person!

Early Registration Now Open

From the Blogs
May 21, 2013
blog

A Common Misconception about MAXDOP

Out of the box, SQL Server is (and has been) able to take advantage of multiple processors/cores without any effort on behalf of administrators....More
May 9, 2013
blog

My ISO 8601-Compliant Signature 2

My family recently just "officially" announced that we're in the process of adopting a child from South Africa. We're quite excited, of course, but there's a ton of paperwork to do—along with the need for gobs of signatures....More
May 8, 2013
blog

Use SSIS for ETL from Hadoop

In this blog post, Mark Kromer walks you through using SSIS as a way to use ETL techniques using Microsoft's Hadoop on Windows (HDInsight) as a source using Hive connectors...More
SQL Server Pro Forums

Get answers to questions, share tips, and engage with the SQL Server community in our Forums.