"Science and technology multiply around us. To an increasing extent they dictate the languages in which we speak and think. Either we use those languages or we remain mute." -J.G. Ballard
Besides SoBig and W32.Blaster.Worm, the latest hot news remains "WS-I Publishes Basic Profile 1.0" (http://boston.internet.com/news/article.php/2247551). Both news items pertain to the distributed-computing arena. The prior piece is distributed denial of service (DDoS) and later one is distributed EDI. As the name suggests, this article is about the later one. It digs into Web services for EDI and provides some examples and code.
Through the basic profile, the Web Services Interoperability (WS-I) organisation provided a framework guideline to Web services vendors and developers for Messaging, Description, Discovery and Security. The W3C defines a Web service as "a software application identified by a URI, whose interfaces and binding are capable of being defined, described and discovered by XML artifacts and supports direct interactions with other software applications using XML based messages via Internet-based protocols." This idea of URL-addressable software applications is thriving by leaps and bounds. In my previous article, Evolution of Web Services, I explained the EDI progression from socket based Internet transfer controls to .NET Web client class usage. This article takes it further and shows some financial-world scenarios then concludes with Web services security.
This article will explain important Web services terminology inline. XML Web services are, no doubt, one of the most thriving and developing technologies in distributed computing. Due to their simplicity and their capability of bridging the gap between different platforms and languages, they provide strong hope for an ideal interoperable world. They are ideal for workflow solutions, enterprise application integration, and messaging between Web and application. Even a brief coverage of each of their dimensions is not possible in this article; therefore, I'll concentrate on major aspects regarding implementation. For theory, the further readings section will help.
With Microsoft's .NET Framework, writing a Web service has become quite an easy task. Instead of being concerned with client and server stubs and end-point naming conventions, ease of implementation yields focused development on business processes. Reasons for DCOM's and CORBA's limited success were proprietary standards and interoperability hurdles. Web services rely heavily on open Internet standards, which provide an improved messaging and RPC solution. XML Web services are open-text based, firewall friendly, XML encoded, and URI-addressable resources.
In a typical .NET implementation, a Web service is a .asmx file that starts with an "@ WebService" directive with a language parameter. A .asmx file is the entry point of a Web service, the addressable resource. Keeping in mind that we are dealing with distributed content, the importance of namespaces can't be ignored. These are used for unique identification of a Web service, and although not necessary for test purposes, developers are strongly recommended to set namespace definitions on the production box.
The procedure for setting up and executing an XML Web service is simple and similar to that of deploying an ASP.NET application. Creating a virtual directory using IIS management console and setting it as a Web application are two straightforward steps involved in the process. The virtual directory acts as the root directory for an XML Web service, and relevant files are to be placed in this directory.
Figure: Screenshot showing IIS Application configuration, application name and execute permissions.
Thanks again to the .NET Framework, invoking Web services is just a matter of addressing the resource (the .asmx file) in the following format:
The developer hasn't written any code for invoking the Web service. The .NET Framework took care of this. It shows a service's public methods, arguments, help, and SOAP headers. To elaborate, let's create a sample Web service called TemperatureService, which will return a random number when invoked. The first step in creating a Web service is writing the .asmx file. This .asmx file holds the definition of the Web service, its public methods, and other details.
Here is the listing of TemperatureService.asmx file. It's a simple C# file with one public Web method, i.e. a method accessible and usable by clients querying this Web service. This method takes no argument but returns a randomly generated number as a string. This is accomplished by instantiating an object of class Random and by calling its method Next. This method takes the required range's upper and lower bounds as arguments. This code is very similar to that of an ASP.NET page, but instead of having a page_load event, it has a public method to be used by clients calling this Web service.
The framework provides built-in support for viewing and accessing methods exposed by a Web service. By accessing the Web service .asmx file, the framework automatically displays the public methods, as shown in figure below.
Figure: The Web Service Help Page
Besides the function name, it provides further exposure through the "Service Method Help Page", which is a deeper look into the Web service. The service method help page shows the XML representation of the request and response for the Web service, its arguments, and return types.
Figure: Service Method Help Page
Listed below is the POST request for the TemperatureService Web service. Its headers contain character set, length, and required method. Since this method doesn't take any parameters, they are not defined in the request; otherwise, the method body also contains child nodes for all the arguments. This type of argument passing will be discussed later in this article.
POST
/sortcodewebservice/TemperatureService.asmx HTTP/1.1
Automatically Generated Code - Post Request for TemperatureService
The response XML file is also crucial for building an application which reads the response from a Web service. This response definition explains what is being returned by the Web service and which data type is to be handled.
These pages demonstrate the usage of messaging protocols i.e. HTTP-GET, HTTP-POST and SOAP to access Web services. By setting the element inside the Element in web.config, messaging protocols and the service help page for XML Web services can be configured.
Role of WSDL (Web Services Description Language)
According to the W3C, WSDL is "an XML format for describing network services as a set of endpoints operating on messages containing either document-oriented or procedure-oriented information". This refers to Document or RPC style Web service definitions, which will be discussed later in this article. WSDL, in principle, is the platform independent definition of Data and Methods. This declaration is vital for web service description and it's the foundation of UDDI, universal description, discovery and integration. The .NET Framework provides the capability to generate a WSDL definition, i.e. "Service Description", for the corresponding .asmx Web service source file. While viewing a Web service .asmx file, the .NET Framework creates a corresponding WSDL, which can easily be accessed by sending WSDL parameters in an .asmx URI. This is demonstrated below.
While .asmx is specific to .NET, WSDL is a generic Web service entry point that holds key information about the Web service. The difference is that .asmx files contain implementation, while WSDLs have an abstraction of members and methods. Dissecting a WSDL file reveals the anatomy of Web service methods, details of parameters, and return data types. Once the .asmx handler is called by the .NET HTTP pipeline, the .NET Framework handles message dispatching, mapping XML to objects, and automatic WSDL and documentation.
Passing WSDL as a parameter to a .asmx file returns a service's description, which is a detailed XML representation of data types used, method names, their respective parameters, and return types.
Now we've reached the final step of invoking the method, i.e. either calling it using HTTP-GET or pressing "invoke" on the service method help page. The result of this method call is the XML file shown below.
Figure: Invoking a Web Service Method from service method help page
The XML file returned by the method call contains the namespace of the Web service and the random number which is generated as the temperature. On multiple calls, the result of this Web service will vary, providing a uniform statistical distribution for random numbers.
Important XML Web Services Terminologies
XML
The text format used when communicating with XML Web
services using SOAP. When communicating with XML Web services using the
HTTP-GET and HTTP-POST protocols, XML is used to encode the response.
SOAP
An XML-based message exchange protocol used for
communication between XML Web services and their clients.
Web Services Description Language (WSDL)
Describes the contract of messages that an XML Web
service can interpret when communicating with an XML Web service client.
XML Schema Definition language (XSD)
Provides a universal type system, allowing data types
to be defined and passed across platforms. For an XML Web service, XSD
defines the structure and data types for the XML encapsulated within a SOAP
message sent to and from an XML Web service.
application/x-www-form-urlencoded
A MIME type used for encoding parameters on a URL.
This encoding is used for encoding request parameters to XML Web services
using the HTTP-GET and HTTP-POST protocols.
Courtesy:
MSDN
Invoking the Web Service from an ASP.NET Web Page
As described above, the .NET Framework implicitly provides support for XML Web service invocation through the message help pages. In order to use the Web service more efficiently within code, tempService.aspx shows the implementation in ASP.NET. Bear in mind that an XML Web service is accessible using HTTP-GET and HTTP-POST; therefore, it can also be invoked from static HTML pages.
Invoking a Web service from an application, either Web or desktop, is a simple matter of object instantiation and invocation. But which Object? This is where the proxy class comes into play. Since XML Web services are part of a distributed application framework, the service is usually residing on machine A while we are trying to invoke it from machine B, physically located somewhere else in the planet, connected by the Internet. In order to invoke it, we have to have a stub/skeleton in CORBA lingo, or proxy classes in Web services dialect. The proxy class can be created using the WSDL tool, which comes with the .NET Framework. Wsdl.exe is a utility that generates code for XML Web service clients and XML Web services using ASP.NET from WSDL "contract" (Microsoft usually calls WSDL files as contracts for sharing the data) files and XSD schemas.
Generic syntax for WSDL is
wsdl.exe <options> <url or path> <url or path> ...
There are several options for WSDL. I'll just define some for our particular scenario. For more options, type wsdl /? in the command prompt.
<url or path> - A url or path to a WSDL document.
/language:<language> - The language to use for the generated proxy class. Choose from 'CS', 'VB', 'JS' or provide a fully-qualified name for a class implementing System.CodeDom.Compiler.CodeDomProvider. The default is 'CS' (CSharp). Short form is '/l:'.
/namespace:<namespace> - The namespace for the generated proxy or template. The default namespace is the global namespace. Short form is '/n:'.
/out:<fileName> - The filename for the generated proxy code. The default name is derived from the service name. Short form is '/o:'
WSDL.exe will return code for the proxy class, i.e. the cs file. Once we have the source code we need to compile this code into a DLL. Csc.exe is short for Visual C# .NET Compiler for Microsoft .NET Framework . The compiler options used are:
/out:<file> - Output file name (default: base name of file with main class or first file)
/target:exe - Build a console executable (default) (Short form: /t:exe)
/reference: <file list> - Reference metadata from the specified assembly files (Short form: /r)
To create the corresponding DLL, csc is invoked with the following parameters.
Once we have the proxy class compiled into a DLL,we must move it to the \bin directory. This is the location where DLLs need to reside in order to be picked up automatically by ASP.NET. Bin directory is automatically checked for the dynamic link libraries and for definitions of methods that will be used. This reminds me of the bin\etc folder in JDK where it picks up the extended jar files. The figure below shows the creation of proxy and corresponding DLLs. Although you can see Visual Studio .NET's IDE in background, its only used here as a text editor to keep things simple and more understandable.
Figure: Screen shot for creating proxy and DLLs with VS.NET
Now that the Web service's method has a proxy class and corresponding DLLs, it can be accessed as a local class method. The statement for creating TemperatureService Object, written below, is the same as creating an object in any .NET Framework library class.
TemperatureService tempObj = new TemperatureService ();
Below is the code for TempClient.aspx, the example for invoking a temperature Web service from an ASP.NET Web page.
<%@
Page Language="C#" Debug="True"%>
<%@
import Namespace="TemperatureService" %>
<html>
<head>
<title>Invoking Web Service from
ASP Page - An Example</title>
Response.Write
("Today, the temperature is: " + tempObj.sayTemperature());
%>
</body>
</html>
Listing: TempClient.aspx
This code is simple and self explanatory. It just creates a temperature object, sets the credentials of the proxy class object to access the Web service, and then calls the sayTemperature() method of the Web service. This method will return a string which will be concatenated with a string in the Response.Write method of the system library. Executing the above example will result in the following page.
This simple example demonstrates how easily Web services can be accessed from ASP.NET pages and can be embedded into Web applications for enhanced usability. In the next section, we'll discuss how to communicate with a Web service from desktop applications.
Using Web Services from a Desktop Application
It can't be emphasized enough that Web services are a great method of communicating between various applications, for instance peer-to-peer, client-server, Web based, etc. In this particular example, we'll see how to run an intelligent bot from a C# console application using Web services. Below is a demonstration using Eliza, a Web service chat client found listed on xmethods. It is implemented using Microsoft Soap Toolkit in Visual Basic .NET.
Microsoft SOAP Central http://msdn.microsoft.com/soap is a good place to check for toolkit specifications. SOAP toolkit makes it possible to build SOAP-client applications to access Web services. Microsoft defines the functionality of MS Soap Toolkit 3.0 as, "The SOAP Toolkit can be used to add XML Web Service functionality to COM components and applications. The major features in the Microsoft SOAP Toolkit 3.0 release are: send and receive attachments, WSDL and XML Schema 'import' and 'include' elements support, 'One way' message support, and more."
"Eliza - Chat with a webservice!" is an RPC (remote procedure call) style Web service. It's defined as "a mock Rogerian psychotherapist. It prompts for user input, and uses a special transformation algorithm to change user input into a follow-up question." Like before, I tried to keep the code simple and easy to understand. This console application has a do-loop iterating over the console input for chatting. It will keep reading questions, invoking the Web service method for replies and printing them. The mssoapinit method initializes the SOAPClient object using the Web Services Description Language (WSDL) file as input. All of the operations in the identified service are bound to the SOAPClient object during initialization. Thus, you can call the operations defined in the service using the SOAPClient object.
Console.WriteLine("Lets Have a
Chat with Eliza - Web Service!")
Do
Console.Write("Q. ")
question = Console.ReadLine()
response =
SOAPClient.Eliza(question)
Console.Write("Answer.
" & response)
Console.WriteLine()
LoopUntil Convert.ToString(question).Length = 0
Console.WriteLine("Press Enter
to terminate the programme")
Console.ReadLine()
EndSub
EndModule
Listing - Module1.VB
Figure: Visual Studio.NET IDE executing console VB.NET application for Eliza
Using Web services in desktop applications has if not opened, then at least extended new horizons in development of distributed software through a standard. A desktop application can communicate with URI addressable resources, consume their services, and send and receive data without knowing the implementation details on the other end. It's a messaging mechanism to inter-communicate and request and respond in uniform fashion.
Developers will find it interesting that the SOAP toolkit provides a Trace utility to view port data and SOAP transfers. Setting it up on 8080 and accessing Web services with this port in the URL will give you access to raw headers and data, both in hex and ascii.
Figure: Soap Toolkit, Trace utility.
With the strong coupling and communication strengths of Web services, many service providers have started using them as the source of their service "fronts", such as inventory sharing, pricing bots, etc. For further information on writing soap clients, "Overview of SOAP Client in Windows XP" by Roger Wolter in MSDN is a good read.
In this article, we will see how Web services can be used professionally in applications and how service providers are offering them for use in applications. We will also take a brief look into revenue generating Web services, monthly trial based services, and security.
Using Search Engine Services
Spelling Suggestions Using Google APIs
Google, a Stanford project, is famous for adapting, evolving, and using the latest technology. From statistical algorithms for clustering to intelligent context based searches, Google is a proven technology prodigy. Its development team has made Google APIs public for developers to invoke methods from their respective applications.
Google APIs are defined as:
"Google Web APIs are implemented as a web service. The service supports several SOAP methods; these are described in an accompanying WSDL file that can be imported into your favorite web services programming environment. "
Execution of a Google Web service will be similar to those discussed above; the only difference will be that we are not aware of any implementation methodology of Google's Web service. It's nothing to be worried about because Web services rely on implementation-neutral communication. Our entry point will be the GoogleSearch.wsdl file, which exposes operations, names and data types. Google keeps track of hit limits, so you need to have a license key for its usage, which can be obtained by signing up on Google APIs home page.
First we have to create a proxy class from google's WSDL
Now it's time to code the Google API into ASP.NET. In the Google API examples, there are samples for Java, Visual Basic .NET, etc., but none for ASP, even though there is not much difference in implementation.
Googleclient.aspx contains a Web form; this constitutes a text box control named txtPhrase, which reads the user input, and a label control titled labelSpelling, which renders results into text.
Later, the doSpellingSuggestion method is instantiated. This method takes two arguments; the first one is the google API license key, and the second one is the text pharase to be checked for correct spelling.
Some of the test results for the google spelling client are as follows:
Figure: Function result for a spelling mistake
Figure: Function suggestion returns null if there is no spelling suggestion.
Above we've seen steps for simple Web service construction and invocation. Examples were for general purpose usage of Web services. Later, this article explains Web services applications in the financial and business sectors.
XML Web Services - EAI & EDI Translation in Enterprise Applications
Being a software engineer with an avid interest in real sciences, I see Web services as the missing link between interoperability and scientific-inclusion. Web services are the niche where an application can utilise various algorithms and approaches simultaneously in a distributed, asynchronous fashion.
Examples of various different kind of projects that need intercommunication with client applications or with servers to receive data are Berkley's SETI resultants, email corpus comparison of Naïve Bayes and Chi-squared [Cormac O' Brien & Carl Vogel] /Rule-learning for spam filtering, travelling salesman mathematical problem or grid computing messaging protocol. Instead of proprietary data exchange mechanisms, Web services could be a effective tool for these to inter-communicate.
XML Web Services are to be treated as software components developed by their respective vendors to solve business problems. Financial sector and businesses most importantly need such interoperability to share their data to maximize exposure, sales, and profits. These services can exist on different systems and can be implemented by different technologies or toolkits, but since they are deployed using standard and widely used Web messaging protocols like XML and HTTP, they can easily be implemented by a third party vendor in its software or Web application.
Another major use of XML Web services in enterprise development could be in legacy systems, for instance, communicating to and from an archaic UNIX box to a hi-tech e-commerce Web site. The old system is still required because the enterprise is used to it and staff training for a new system is not cost effective; therefore, sharing inventory seamlessly might be a developer's nightmare. It could involve hundreds of man-hours building an application or using expensive third-party tools not customized for your specific needs. Using Web services can get rid of all these qualms. Data sharing through Web services is platform independent, location and vendor neutral, and the cost is much lower than that of a typical Enterprise Application Integration system. Therefore, it's a new niche for EAI. Intercommunication between businesses, or B2B Integration, is also an arena which is being affected by the development of Web services. It helps lowers the cost of data sharing on mutual agreements, eases the complexity, and cuts costs.
Within large organisations, different departments or systems can use Web services as a standard messaging protocol over HTTP. How interesting it would be when ERP and CRM systems can talk to each other while Enterprise Commerce Management can communicate with digital asset management to read certain fields regarding a company's digital assets costing analysis. With Web services, sharing data collaboration with users, associate vendors, or clients is easy and straightforward.
XML Web services allow software embedding in both Web and desktop applications. Delivery Service providers, such as UPS.COM, also provide tracking services via XML request response. This can be integrated more easily in a Web site using WSDL methodology. Some of the Online Trade Listings, for instance, Amazon's external dealers, update their inventory with XML files, which can easily be incorporated into very effective Web services for various critical daily tasks, such as checking stock and delivery status.
Our next example is based on this application integration scenario. It's about developing and integrating a sort code Web service in a Web site to setup direct debit forms.
The sort code is a number which indicates a bank's address during payment transactions. The idea is similar to that of an ATM locator for a particular bank; such applications are available for PDA and also in the form of WAP and desktop clients. Most banks provide such services on their Web sites. For instance, two of UK's major high street banks, National West Minister and Lloyds TSB, show their versions of branch locators.
It's not difficult to spot the difference among their input parameters and search domain. Similarly, we still don't have a central repository for sort codes. Like ATM locators, branch locating using sort codes is done by the bank's proprietary services and are difficult to integrate with software applications. Ideally, our model proposes a distributed framework that will provide such information to software applications, in this case, a direct-debit-setting charity Web site.
According to UK government data standards catalogue, a sort code schema can be defined as
Translating XML Schema Definition into simple English, this defines a sort code has having six numeric characters.
Bank Details
Information needed to identify one bank account. It mostly consists of the account number and the bank code, sometimes even a branch code. In some countries the bank code information is included within the account number. The bank code is named differently in each country: >"Routing Number" (US), >"Sort Code" (UK), "Bankleitzahl" (Germany).
The sort code is a number which indicates a bank's address during payment transactions. Through the use of electronic data-processing (e.g. opto-automatic document sorting, magnetic tape exchange and remote data transmission) it should allow the faultless, rapid and fully automatic transfer of payments. ~Courtesy: HSBC
I found a similar US-based counterpart for routing-number tracing. On XMethods.net, an informative directory of Web services, a routing-number tracing Web service (ABA lookup) is available. This Web service is designed to allow the user to validate United States bank routing numbers (ABA numbers) . This Web service allows the user to validate a routing number or search for a participant bank by phone number or bankname/city/state.
It supports the following three functions:
GetABADetailsByPhoneNumber
Validate United States bank routing numbers(ABA numbers) by Phone Number
GetABADetailsByRoutingNumber
Validate United States bank routing numbers(ABA numbers) by Routing Number
GetABADetailsByBankDetails
Validate United States bank routing numbers(ABA numbers) by Bank Name and Address
The following is a sample HTTP POST request and response. The placeholders shown need to be replaced with actual values. Placeholders are strings in blue.
As background, I'd like to explain that the application's emphasis is on a sort code number to retrieve a donor's address to setup a direct debit on his account for regular donations. In contrast with regular online donations, setting up direct debit is somewhat different. First, the beneficiary organisation has to have consent of donation from the donor, i.e. the direct debit mandate. Then they have to contact the donor's bank along with benefactor's bank details and organisation OIN to setup this particular standing order for his account. This branch has to be the one in which the customer holds the account; therefore, we trace the branch using the sort code and post to them the standing-order mandate to process. Mandate is the signed authority given by a customer to the bank instructing the bank to make regular payments on behalf of the customer.
Direct debit is an easy and reliable way of paying because of the following features.
Automated regular payments
Reduces time and cost of administering bulk payments
Helps manage cash flow and improve financial control
Reduces risk of loss, late payment, and theft for customers
Because the BACS (Banks Automated Clearing System) process is electronic, it removes the manual processing issues. Similarly, through online banking, direct debits can very easily and effectively be managed.
Figure: Direct Debits controls through Online Banking
Figure: Showing control over setting up standing order activities by Online Banking
Creating a Sort Code Lookup XML Web Service
To proceed further with this XML Web service there needs to be a database which holds the information regarding sort code and address details. This Web service will extract information from there. In an enterprise application that may be used from a PDA to secure banking transactions, the database design would be done in a very sophisticated and precise manner. Keeping virtually all existing scenarios in mind, developers will proceed towards identifying Entities, Relationships and Attributes. Put simply, we analyse that a bank is organized into branches. Each branch is located in a city and is identified by a unique name and sort code. Sort code cannot be the primary-key of the branch entity set because there can be more than one sort codes associated with a branch. For instance there are two sort codes (30-84-12, 30-92-21) for Lloyds TSB Commercial Road branch. For the sake of simplicity, we try to keep our database schema as simple as possible, which can always be elaborated with branch contact details, (phone, fax), facilities (disabled access), and mailing address if different from the branch.
Figure below shows a normal banking process flow diagram.
This process flow is courtesy of Lecture 2, The Entity Relationship Model, and its translation to Relational Database Uppsala University Sweden
CREATE
TABLE Branch (
sortcodevarchar(10) NOT NULL,
titlevarchar(256) NOT NULL,
addressvarchar(256) NULL,
postcodevarchar(256)
NULL
)
ALTER TABLE Branch
ADD PRIMARY KEY NONCLUSTERED (title)
Listing: Schema.sql
Now that we have a database to query, the Web service will do the rest. Like any ASP.NET page, we can query a Web service from an .asmx file. Therefore, we have defined a public method called public String getAddress(String sortcode), which needs SqlClient namespace to access the database. The getAddress() function takes one argument i.e. sortcode, and returns the address as a string.
To accomplish this reading operation, an SqlDataReader object helps us read a recordset from the database. Therefore, if there is any record available for the corresponding sortcode, i.e. if (myReader.Read()), it will add up the three fields using myReader.GetString()and return the address for the corresponding bank.
