|
Introduction
After a few years of building Web applications using the Windows DNA
approach, we've realized several inefficiencies about how our teams
work with the technologies. They usually manifest as late nights
trying to integrate the code that our back-end and front-end Web
developers put together. After all, missing nights of Unreal
Tournament and Icewind Dale because the project is behind is very bad
for morale.
Finally, it was time for action. We decided to use some newer
technologies: XML, XSLT, and XML Schemas. We've come up with a
development paradigm that makes it easier for team development and
improves the standard Windows DNA approach.
Click here for all
code, images, schema, and documentation associated with this
article
Windows DNA
We assume you are familiar with Windows DNA and direct you to the
many white papers and articles available in MSDN
(http://msdn.microsoft.com/dna).
Briefly, Windows DNA is broken up into three development tiers: a
presentation tier, a business tier and a data tier. The presentation
tier is the technology behind a client-facing application or browser.
The business tier is the logic and manipulation of the data. The
data tier persists the data to a database or some type of storage
mechanism.
When building a Windows DNA solution, you typically end up with a set
of Active Server Pages (ASP) that access a COM object model to
exchange information and to build dynamic HTML. The COM objects, in
turn, interact with the data layer of the application, usually a
relational database.
For example, your ASP page could call a method on an object to get an
ADO recordset full of information about a set of albums for a certain
band. Using VBScript, loop through the ADO recordset and build an
HTML table. This generated HTML will make up one small part of the
entire HTML page that ASP is generating. To make the Windows DNA
approach work, we need to discuss how to put teams together.
Teams
For large-size projects it makes sense to split up the development
team into distinct groups with distinct responsibilities. It allows
you to channel each person's talents to build a Web site.
Our typical development team is made up of a group of designers, Web
programmers and back-end programmers. In Windows DNA terms, the Web
programmers are focusing on the Presentation Tier and the back-end
programmers are focusing on the Business Logic and Data Tiers.
Depending on the project size, we may have more or fewer roles
involved, but for the purposes of this article these three groups
will suffice. Here are some descriptions of the roles and the
expectations we have of each:
Designers
Designers are responsible for the user's experience on the site.
This involves everything from color schemes, graphics, and typography
to Information Architecture. Information Architecture involves the
creation of the site's layout and navigation. The Design staff is
generally not concerned about the technologies to be used to drive
the site.
Web Programmers
The Web programmer is the one who makes the designer's vision a
reality. These programmers are experts in various markup languages
and client side scripting languages, such as HTML, HDML, WML,
JavaScript, VBScript, and more. The Web programmer's main concern is
the presentation of data, not necessarily how the data is gathered.
Because of this, Web programmers are generally not experts in server
side scripting.
Back-end Programmers
The back-end programmers are not only the best looking, they are the
geeks who make the underlying technologies work. They create the
architecture for the site, the COM object model, implementany nasty
algorithms or code, and build any extra "tools" the Web programmers
may need. These people are into heavy COM development using
languages like Visual C++, Visual Basic, and the recently announced
C# (pronounced "see sharp"). While designers and Web programmers are
concerned with presentation, the back-end programmer is concerned
with gathering data and implementing business logic. No pretty
colors, no HTML; just code.
Team Development Issues
So far, this is sounding pretty ideal, and it can be. In reality,
there are many problems this team will face. Each member of the team
makes an equal contribution to the success of a project. With such a
diverse team there are bound to be issues. Here are some of the most
common issues with team development
Blurry team roles
Blurry team roles often occur between the Web and back-end
disciplines. Both understand the scripting languages; but who
actually writes the code? Does the Web programmer write "all" of the
VBScript code for the ASP or does the back-end programmer need to be
involved? Exactly where does the Web programmer stop and let the
back-end programmer take over?
Learning Curve
Often the blurriness of roles leads to larger learning curve issues.
While Web programmers are very adept at scripting languages, they may
often not know much about databases, working with different object
models or, even worse, 3rd party technologies. The Web programmer
may have to deal with RDO, ADO, DAO, ODBC, HTTP requests, screen
scraping, and various other APIs just to get data. Even if the
back-end programmer has spent time setting up the data access to
remove the complexity; the Web programmer still has to work with the
technology to render it to the user. The Web programmer now has a
whole new set of things to learn.
Likewise, if you have the back-end programmers do the work, they are
not as adept at markup languages like HTML, DHTML and may have
similar learning curves dealing with making the Web pages work. This
is especially true when supporting multiple browsers. The back-end
programmer now must deal with all the complexities of how to create
HTML/DHTML that is viewable to the same degree on all browsers.
Miscommunication and Change Requests
It is common for different people from different teams to contribute
work for a Web page or functions within it. When requirements are
not communicated properly or changes are necessary, who should update
the work? You've now got multiple people from multiple disciplines
responsible for some aspect of the work.
Round Tripping
Now that we've described the team's makeup and potential issues, it's
time to build a scenario to frame our discussion.
Our team wants to build a search page to find rock albums. The
search page will find and return information using the typical
Windows DNA approach. We'll have some ASP pages, ADO recordsets,
HTML and some VB COM objects.
The search page will return the band, an album name, a song, whether
or not it is a "live" song, and the date the song was released.
It is Monday morning when we start out. Our designer hands us an
image that represents what the search results page should look like
(figure 1). Both the Web programmer and the back-end programmer take
a gander at it and say "Wow, that looks really cool."
The Web programmer takes the image back to his seat and starts
building a big HTML table containing rows and columns of made up
information. When finished, the Web programmer saves this as an ASP
page that represents the image.
However, the ASP page doesn't really have any programming logic for
the search results yet, because the back-end programmer has not
completed the search object. The ASP page contains a set of static
HTML tables, cells, anchor tags, etc., to represent what the final
programming inside of the ASP page should produce.
The back-end programmer is writing the search object in Visual Basic,
and it will probably take more time to complete the search object
than it will take for the Web programmer to finish the static
representation of the page. A couple of weeks later, the search
object is finished and the back-end programmer begins to change the
ASP page so that it makes use of the search object.
The new search object has a method called "search". Given certain
search criteria, it will return a disconnected ADO recordset filled
with album data. A server-side VBScript function called
"BuildSearchResults()" is created, which iterates through the
disconnected ADO recordset and generates a string containing the HTML
table. After a few hours, there is a triumphant exclamation from the
back-end developer, "I'm done!"
The Web programmer notices that there is a strange problem rendering
the search results on Netscape, which he tested thoroughly. He
checks out the HTML and finds out it is a problem with the code
generated by the function. So he goes into the function to find out
what's wrong. Although the Web programmer is not a server-side
scripting expert, he does the best that he can to fix the problem.
The resulting table is fixed, but there are some errors introduced
into the function.
The back-end programmer gets the function and corrects the code
mistakes and cleans it up a little bit, screwing up the HTML table
once again. This causes the Web programmer to again correct the HTML.
This passing of the code back and forth between these team members is
what we refer to as "Round Tripping". Sometimes there are very few
iterations, and other times there are many. Of course, the more
iterations, the more time is needed and the greater the
inefficiencies in the process.
Analysis
The team development issues degenerate into "round tripping," which,
as you can see, is not a very productive process. Much time is spent
relearning code and revisiting the designs each team has produced.
After a two-week delay, do you think that the Web programmer
remembers exactly how the code for the HTML table for the search
results should be? Probably not.
The work between the Web programmer and the back-end programmer is
essentially data binding, and this causes a majority of the problems.
The big question is who does the data binding? Does the Web
programmer do it? Does the back-end programmer do it? Or, do both
groups do this?
If you look at the different roles for the Web programmer and the
back-end programmer, ideally these should be two separate tasks. The
Web programmer should focus on the markup languages while the
back-end programmer should focus on producing the data and the object
model. This is clearly not the case in the above scenario and it is
difficult to achieve that type of separation using VBScript, ADO, and
HTML.
When building a large Web site and integrating many different
technologies, the issue is exacerbated further. A staff may have to
invest time and effort to learn these external technologies, which
may also be changing frequently. Third-party technologies are rarely
static. Over time, most vary their APIs, object models, command
lines, and more.
To avoid a mess, we need to have a better split between the
presentation layer and the data. There should be something between
the two programmers that performs the data binding. The Web
programmer could describe how the binding should take place (XSLT)
and the back-end programmer could provide the data to be bound (XML).
By using a combination of the newer Web technologies -- XML, XML
Schemas and XSLT -- we can accomplish a much better division of labor
for the teams.
First let's define these technologies and how they're used:
XML
The eXtensible Markup Language (XML) is a way to express data in an
easy-to-read, self-describing text that is made up of elements,
attributes and text data. An XML document with this self-describing
text can be either a file or a text string. All elements, attributes,
and data combine to form a large tree structure of data. Developers
can use the Microsoft XML Document Object Model (DOM) to load and
parse an XML document; it even allows developers to easily access the
data inside an XML document.
XML Schemas
Schemas are XML documents that define a structure to another XML document. The
schema dictates which elements and attributes are required, what
values an attribute can contain, how many instances of a certain
element can be in the document, etc. XML Schemas are so exact that a
powerful database like Microsoft SQL Server 2000 can read and write
an XML document back and forth to the database based on schema data.
XSLT
The eXtensible Style Sheet Template (XSLT) language is a very
exciting display technology. It is a declarative language that allows
you to transform an XML document into a precise visual representation
and is especially helpful with hierarchical data. With XSLT, a
developer can transform an XML document into many things: an HTML
page, an HTML table, WML page, a series of SQL insert statements,
another XML document, etc.
" . . . when I first saw the XSL Transformation language, XSLT, I
realized that this was going to be the SQL of the web, the high-level
data manipulation language that would turn XML from being merely a
storage and transmission format for data into an active information
source that could be queried and manipulated in a flexible,
declarative way." (XSLT Programmer's Reference, Introduction; Wrox
Press; Michael Kay)
XPath
XPath is an interesting technology that is heavily used by XSLT and
the XML DOM to query an XML document to find a specific node or a set
of nodes. XPath is to the XML documents as Structured Query Language
(SQL) is to relational databases. We will use XPath throughout the
album finder demo.
Windows XDNA
We've been using and building an XML version of Windows DNA that we
call "XDNA", and it works out really well. Instead of exchanging
data to and from our object model using ADO recordsets we're using
XML documents. The ASP pages will be written as usual except in the
places where data binding is done. We replace these parts with an
XML/XSLT transformation (figure 2). Each ASP page may contain
several transformations to make up an entire HTML page.
The XDNA Process
In the XDNA process, a Web site is backed by an object model that is
used to exchange data between the back-end system and the browser.
As a part of the design process, the developer needs to define the
interfaces of the objects, and the datasets that represent the
information coming in and out. These datasets are defined using XML
schemas and implemented using XML documents. These two form the
contract between the Web and back-end programmers.
The back-end programmers building the object model can generate the
XML documents any way they wish. One quick and easy way will be SQL
Server 2000's new stored procedures, which can generate rich XML
documents (
http://msdn.microsoft.com/library/default.asp?URL=/library/techart/d51ctlgbrowse.htm).
The back-end programmers just need to make sure the XML documents
that are created are well formed and validate against the schemas.
The Web programmers then build XSLT templates to render the XML
documents into HTML (or some other markup language). Then they will
place several XSLT transformations within an ASP page to produce the
final HTML page to the browser.
The MSXML parser can then take the XSLT and the XML from both
programmers and perform the transformation. In our demonstration we
wrote an xlate() function that takes in two parameters, the XSLT and
the XML, and then performs the transformation.
<HTML>
<HEAD>
<LINK REL="stylesheet" TYPE="text/css" HREF="/rare.css">
<!-- #include virtual="\include\tform.asp"-->
<TITLE>Album Search</TITLE>
</HEAD>
<BODY>
<table width="800" cellspacing="2" cellpadding="2" border="1">
<tr>
<td width="100"><%=
tform("myLogo.xml", "logo.xsl") %></td>
<td width="700"><%=
tform("myBanner.xml", "content.xsl") %></td>
</tr>
<tr>
<td width="100"><%=
tform("myMenu.xml", "menu.xsl") %></td>
<td width="700"><%=
tform("myMainContent.xml", "content.xsl") %></td>
</tr>
</table>
</BODY>
</HTML>
Figure 2: Sample ASP page
The Result
Instead of using shared VBScript to do the data binding, there is a
transformation function sitting between the two programmers. The
function takes the XML document and the XSL template as input and
produces a chunk of HTML as output. This is possible because XML
documents - instead of ADO recordsets - are sent in and out of the
object model. The Web programmer can create sample XML documents
based on how the object model will return the data and be confident
that it won't change too much. The back-end programmer can work on
the object model in any form or fashion that is best suited for this
problem.
Both parties are now free to work on what they do best. Web
programmers can handle the markup languages XML, XSLT, HTML, WML,
etc. Back-end programmers can build the object model to provide the
data. As long as the schemas don't change too much, the two groups
should integrate perfectly!
The traditional DNA development methods are varied slightly by this
method, but it has large implications, such as reducing "round
tripping." In the tables below, we've listed some comparisons
between using XML documents vs. ADO recordsets.
Figure 3: XML Document Pluses
|
XML Documents |
ADO Recordsets |
|
XML document can be easily defined with a text editor
and can be used by Web programmers right away. |
Need to have a prototype object or database backing
the recordset. |
|
Easier to keep object models stateless. This is
especially helpful with nested data. |
Not easy to achieve when working with nested data.
See the Album Example. |
|
Fewer function calls to get data and fewer trips to
the database. |
Multiple function calls that use an object model that
returns flat ADO recordsets will be necessary to get the required
data. |
|
XML is in a self-described, text format. |
ADO recordsets are a binary format that requires
debuggers, etc., to determine the format and structure. |
|
XML was built to represent data that is tree-like in
nature. |
ADO struggles to do so. |
|
You can build an XML document on any client or
platform. |
ADO requires an install. |
|
Up-level browsers have support for XML data
islands. |
ADO requires an install. |
|
A declarative language like XSLT can be used to
transform the contents ofan XML document. This works well on many
platforms. |
ADO must have procedural code written in a
COM-compliant language on the Windows Platform. |
|
XSLT helps enforce better programming practices. The
language is naturally geared towards this. |
Using a procedural language like VBScript in
combination with an ADO recordset can lead to many programming
errors. The binding code is much more fragile. |
Figure 3: XML Document Minuses
|
XML Documents |
ADO Recordsets |
|
The XML DOM object model is more difficult to use
than the ADO API. |
ADO has a simple and easy to use object model. |
|
XML, XSLT and the DOM are relatively new on the
technology scene. |
ADO is a more mature technology. |
|
XML and XSLT have less mature developer tools. |
ADO works great in Visual Studio. |
Album Finder
We've put together a set of pages to scroll through some favorite
rock albums and do a search. The pages are mostly simple XSLT
transformations. To demonstrate the power of XSLT, we threw in a
cool album template. In this demo, we're using the MSXML
3.0 to render the XML as HTML using XSL templates. You'll need
to compile the AlbumFinder.vbp.
First we defined a rough draft of what the pages would look like. .
We were concerned with how the data was represented on the pages, not
about the look and feel of the site We've included the original XSLT
templates, just for a laugh. (figure 5). It's a good thing we're
not the people picking the color scheme on your Web site.
Figure 5: Sample Search Results page
The next thing we did was to define the interface to the object model
and the data sets. We define what the data looks like exactly, using
an XML schema and an object model on how to get the data. Here are
the schemas we came up with:
|
XML Schema For Data
Set |
Purpose |
|
SearchResults_Schema.xml |
Defines how the object model will return search
results. |
|
SearchPost_Schema.xml |
Defines the type of query that the search object will
accept. |
|
Group_Schema.xml |
Defines how the object model will return a list of
rock groups. |
|
Album_Schema.xml |
Defines how the object model will return an album. |
|
AlbumList_Schema.xml |
Defines how the object model will return a list of
albums for a specific rock group. |
Figure 6: XML Schema Listings
The last thing we did was use XSLT to transform the data sets from
the object model into HTML. We also defined the schemas and the
interfaces on the object model (figure 6).
Highlights
The album finder demo was built using the XDNA architecture and
serves as a good example. To get a list of songs, we call a search
method that takes in an XML document that represents the user's
query. The returned list of results is also an XML document. We
then use XSLT to render an HTML table out to the browser.
The query could be much more complex than what we have here. It
could also be built on a client browser and posted to the ASP page.
Another example would be getting album information through the object
model, where we would call a single method GetAlbum() and pass in the
album name. This builds an XML document that represents the album
information (figures 7 & 9). For some of our songs, we even included
the lyrics. These were optional in the album schema and our XSL
template accounted for this very easily. After the document is
returned, write an XSL template (figure 8) that displays this
information. To accomplish the same tasks in ADO would be both
complicated and error prone, would include several method calls,
incur multiple database hits and use a number of recordsets together
in order to get the HTML output.
In the AlbumFinder object, we've got a monster path query (see the
Search() in the VB project) where we do a case-insensitive search on
the songs and descriptions in the XML Catalog. When you write your
own code, make sure you call SetProperty("SelectionLanguage",
"XPath") if you are using the XML DOM. This is necessary because the
default selection language is an XSL Pattern not an XPath expression,
which means that you won't have access to several XSLT functions like
contains(), string() and others.
<album group="Rush" title="Moving Pictures"
year="1981" month="1" day="1">
<image>.\images\MovingPictures.gif</image>
<description>
This is one of my favorites! Well, they all are.
</description>
<song_list>
<song title="Tom Sawyer" track="1"
length="277" author="Woods/Peart"/>
<song title="Red Barchetta" track="2"
length="371" author="Neil Peart"/>
<song title="YYZ" track="3" length="265"
author="Geddy Lee/Neil Peart"/>
<song title="Limelight" track="4"
length="263" author="Neil Peart"/>
<song title="The Camera Eye" track="5"
length="659" author="Neil Peart"/>
<song title="Witch Hunt" track="6"
length="286" author="Neil Peart"/>
<song title="Vital Signs" track="7"
length="287" author="Neil Peart"/>
</song_list>
</album>
Figure 7: Sample Album XML "MovingPictures.xml"
<xsl:transform version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:msxsl="urn:schemas-microsoft-com:xslt">
<xsl:template match="line">
<xsl:value-of select="."/><br/>
</xsl:template>
<xsl:template match="chorus">
<div style="padding-left:15">
<font color="red">
<xsl:apply-templates select="line"/>
</font>
</div>
</xsl:template>
<xsl:template match="song">
<xsl:if test="lyrics">
<hr/><br/>
<font size="4px"><xsl:value-of select="@title"/><br/></font>
<div style="padding-left:20">
<xsl:apply-templates select="*"/>
</div>
</xsl:if>
</xsl:template>
<xsl:template match="/">
<table valign="top" border="0" cellspacing="5" cellpadding="5" width="100%">
<tr valign="top">
<td>
<font color="black" size="4pt">
<xsl:value-of select="album/@title"/>
</font>
</td>
</tr>
<tr valign="top">
<td>
<img><xsl:attribute name="src">
<xsl:value-of select="album/image"/>
</xsl:attribute></img>
</td>
<td align="left" width="100%">
<table width="100%" >
<tr valign="center" bgcolor="#bfbfbf" height="20" width="100%">
<td>
<p><img src="/article/images/albumnotes.gif"/></p>
</td>
</tr>
<tr>
<td>
<xsl:value-of select="album/description"/>
</td>
</tr>
<tr valign="center" bgcolor="#bfbfbf" height="20">
<td>
<p><img src="/article/images/tracklist.gif"/></p>
</td>
</tr>
<tr>
<td>
<table border="0" cellspacing="2" cellpadding="2">
<xsl:for-each select="album/song_list/song">
<tr>
<td><xsl:value-of select="@track" />. </td>
<td><xsl:value-of select="@title" /></td>
<td><xsl:value-of select=
"format-number(floor(@length div 60),'###0')"/>
:<xsl:value-of select=
"format-number(floor(@length mod 60),'00')"/> min</td>
</tr>
</xsl:for-each>
</table>
</td>
</tr>
</table>
</td>
</tr>
<tr> <!-- do the lyrics -->
<td>
<xsl:apply-templates select="/album/song_list/song"/>
</td>
</tr>
</table>
</xsl:template>
</xsl:transform>
Figure 8: Simple Album XSLT "Album.xsl"
<%
<!-- #include file="\include\tform.asp" -->
dim oAlbum
dim sXML
set oAlbum = Server.CreateObject( "AlbumFinder.IAlbumFinder" )
sXML = oAlbum.GetAlbum("Moving Pictures")
set oAlbum = nothing
response.write tform( sXML, "\article\xsl\album.xsl" )
%>
Figure 9: Clean & Simple - Transformations in ASP
Optimizations
We've found that it is better to keep the XML data to be transformed
small. By keeping it small, the data can be re-used, the parsing is
faster, and the XSLT is much more manageable to write and maintain.
Trying to render whole HTML pages using a single XSL template is a
bad idea in quite a few ways - performance topping the list.
Mike Moore, director of development for Microsoft.com, used XML and
XSLT for portions of the site. He relates some good experiences in
the MSDN article "XML and
www.Microsoft.com.". The typical page had between 4 and 6 small
transformations and they tried to keep their XML documents under 50k
One area of concern is the repeated loading, parsing, and
transforming of the XML data each time the page is rendered. These
concerns are addressed with the upcoming releases of MSXML 3.0 and
ASP+.
MSXML 3.0 provides support for caching XSLT templates and ASP+ has a
very good built-in caching mechanism. ASP+ allows for both output
caching as well as caching XML documents. Both of these technologies
will help put a dent in the speed issue.
For those who are not sold on XSLT as a display technology, there is
another option, ASP+ Server Controls will be a part of Microsoft's
upcoming .net platform. The XDNA approach won't change. The Web
programmers will instead use Server Controls on an ASP+ page to
render the XML documents. Attendees of the Microsoft Professional
Developer's Conference in Orlando saw a preview of this technology,
and it can be downloaded from http://msdn.Microsoft.com.
Conclusion
The XDNA approach puts a more definitive division of labor between
the teams, helps to eliminate "round tripping," offers some
architectural benefits, and allows programmers to focus on their
individual talents. Overall, it is a much cleaner programming model
and allows you to produce more maintainable code, faster!
Acknowledgments
Special thanks to Chris Henry for editing expertise, Aaron Boodman (a.k.a.
"Brockavich") for helping with the "Path Query from hell." Grant Hill for
the demo page's facelift and the rest of the Rare Medium staff that helped
us make this article happen.
About The Authors
Chris Faranetta is a Lead Software Engineer for Rare Medium, Inc. and enjoys
creating video games, XML/XSLT, C++, C#, and syrup. Rare Medium (NASDAQ:
RRRR) is an award-winning Internet services firm that combines the best of
online management consulting, user-focused design, online brand management
and leading-edge technology. Chris can be reached at
chris.faranetta@raremedium.com.
Mathew Upchurch is a Lead Software Engineer for Rare Medium, Inc. and when
he's not knee deep in ATL and XML, he enjoys spending time with his wife and
two daughters. Rare Medium (NASDAQ: RRRR) is an award-winning Internet
services firm that combines the best of online management consulting,
user-focused design, online brand management and leading-edge technology.
Mat can be reached at mathew.upchurch@raremedium.com.
|
