twitter
facebook 
| New Notification Services to Enhance SQL Server 2000 |
| Jun. 3, 2002 |
A new add-on for SQL Server 2000 enables developers to create notification services, which extend applications to notify users of events that interest them. For example, an airline could extend its flight scheduling system to alert passengers of delays or schedule changes. Notification services can help organizations win and keep customer loyalty or help employees or partners react more quickly to changes and thus boost productivity. However, the new SQL Server Notification Services (SSNS) requires a significant amount of programming and integration, and it lacks a few features, such as reliable delivery. The product is currently in beta and due to ship in summer 2002. The Need for Notification Services A notification service sends brief text messages (notifications) to users when important events occur, based on the users’ criteria of what is considered important. A notification service is most viable when users want to be personally alerted to time-sensitive events, such as the following examples:
To be useful, notifications must be short (especially if a user is receiving them on a portable device). When users need more detail or to act on the information, the notification can contain a URL path that allows the user to browse to a Web application for further action or information. More important, notification services must not deluge users with unwanted or useless information. Users must be able to set rules such as "notify me only when the stock price of company XYZ reaches US$50." Furthermore, sometimes notifications need to be based on more complex criteria. For example, customers might want to take historical information into account, setting rules such as "notify me when the stock of company XYZ moves more than 5% from the previous day’s closing price." A notification service is a special form of "push" technology. Although widely hyped late-1990s push technologies, such as PointCast, were a flop, notification services are more likely to become popular because they focus on pushing only critical, time-sensitive information and exploit the growing availability of "always on" wireless services to portable devices, such as personal digital assistants, cell phones, and pagers. As a result, they have much higher utility to users. Why Microsoft Created SSNS Organizations attempting to build a notification service from scratch will find it a difficult task, especially when the service must handle high volumes of users and events. SSNS is designed to make building high-volume services easier. SSNS was the platform on which Microsoft built notification services for the MSN Mobile and CarPoint Web sites, where it proved well suited for large-scale operations. The SQL Server product team has now turned it into a product, which it hopes will attract more developers of large-scale Web sites and applications to SQL Server and thus boost SQL Server sales. Furthermore, SSNS could also gain customers for Microsoft's fee-based message delivery service, .NET Alerts. .NET Alerts is an online service that delivers short text messages to users based on their "presence" information—that is, it delivers messages to the network address of the specific device the user is working with at any given time. SSNS provides all the components needed to deliver notifications over the .NET Alerts service without any extra coding or integration. (See the sidebar ".NET Alerts Service Complements SSNS".) Although the technology for large-scale notification services is still emerging, Microsoft is not alone in this field. Oracle has Oracle Alerts, which provides some of the functions of SSNS, and other smaller companies, such as Alerts, Inc., Apama, LetMeKnow, and Tibco, also market notification engines. IBM has also done research and pilot projects in this area. How SSNS Works SSNS uses a paradigm called "publish-subscribe," in which certain data, in this case the events, are "published," i.e., made available in such a way that individuals can "subscribe" to receive events matching their subscription criteria. In this context, events are usually published by server applications and consist of time-stamped, structured or semi-structured data records that someone may be interested in. Unlike notification systems designed for alerting employees when there is a problem with a process (such as when a server or a piece of equipment is down), the publish-subscribe model gives users direct control over their notification criteria. Microsoft also designed the SSNS architecture around its relational database, SQL Server, which does most of the heavy lifting. This design enables the service to quickly process high volumes of events and subscribers. It achieves this efficiency by running a limited number of "parameterized" queries against batches of events. For example, a delayed flight notification application might use a single generalized query form, such as "Notify subscriber X of flight delay events when the flight number equals Y and the flight delay time exceeds Z." This model depends on the premise that large numbers of subscribers can use the same basic query template, and that only the query parameters will vary among the individuals subscribing with that template; users simply submit their desired parameters using a simple Web application. Using this approach, SSNS can handle millions of subscribers, evaluate thousands of events per minute, and generate the appropriate notifications with a single multiprocessor server. Although batching of incoming events introduces some initial latency, SSNS makes up for it by more efficient processing. This database-centric architecture provides two other important benefits: Event and notification history. Notifications can be based on historical information in addition to recent events. SSNS can be programmed to maintain "chronicle" tables of state information generated from past events, such as the previous period’s low- or high-water marks (allowing notifications to be based on a measured change from the previous period) or records of the previous notifications (so that subscribers don’t keep getting repeat notifications of identical events). For example, the stock alerting application mentioned earlier must have access to the stock’s previous closing price to determine when to send a notification. Scheduled queries. Programmers can also create SSNS subscription templates that are evaluated against accumulated events at a scheduled time. This is useful when a subscriber only wants a daily summary of information, rather than a notification each time something of interest is generated. This capability is needed to provide the single daily summary of stock prices in the example mentioned earlier. SSNS Architecture SSNS is a server application that runs on the .NET Framework. It requires SQL Server, but the two do not necessarily have to run on the same computer. The service consists of three components: the Event Provider, the Generator, and the Distributor. In addition, any notification system requires a subscription application, which is not part of SSNS. (For a graphical overview of the SSNS architecture, see "SQL Server Notification Services Architecture".) Event Provider. The Event Provider is an SSNS module that polls for new events from a variety of sources, translates them into the correct format, accumulates them into batches, and inserts the batches into SQL Server. The Event Provider comes standard with two modules for gathering events: an XML file watcher (to gather events from applications that generate XML event files) and a SQL Server Event Provider (that periodically runs SQL queries against other SQL Server databases and loads the results as input events.) The Event Provider also supports APIs that developers can use to build additional modules that gather events from .NET or COM applications. Generator. The Generator matches the event, chronicle, and subscription data by running Transact-SQL queries, known as "match rules." The output of the match rule is a set of notifications that are written to a notifications table in SQL Server. Distributor. The Distributor reads the raw notifications from the notifications table, transforms them into XML, applies appropriate style sheets to format notifications for the subscribers’ target devices and languages, and then hands off the notifications to the subscribers’ delivery channels. The Distributor includes connectors to support Simple Mail Transport Protocol (SMTP) for e-mail, and Hypertext Transfer Protocol (HTTP), which provides a means for linking to Microsoft’s .NET Alerts service or other paging or instant messaging services. The Distributor can also write the notifications to the file system (as XML) for debugging or pickup by another application. The Distributor has logic that can make it more efficient or robust, with features such as the following:
Subscription application. Not included with SSNS, this is the application that users interact with to create or modify their subscription preferences. Subscribers select which queries they want to use, the query parameters, and the devices on which they want to receive notifications. All of this information gets loaded into SQL Server tables. The subscriber application can be any type of application that can access SQL Server, but it would typically be a Web application, possibly a part of the company’s e-commerce site. The subscription application dictates the method of user authentication and controls access to the subscriber data. End users have no direct access to SSNS. Further Scalability and Availability Although the basic architecture of SSNS allows organizations to sustain high notification volumes with a single server, SSNS can scale to support even higher volumes if needed. By analyzing SSNS use at MSN, Microsoft has found that SQL Server processing constitutes approximately half the workload, with the other half performed by the three SSNS components (Event Provider, Generator, and Distributor). This suggests that significantly greater throughput can be achieved by locating the SQL Server database on its own server, separate from the other components. The SSNS components can also be spread over multiple servers and databases for further scalability. In the future, Microsoft plans to support partitioning of the database over multiple SQL Server machines, increasing scalability even further. Since both SQL Server and the SSNS are "cluster enabled," organizations needing especially high availability of their notification system can also run it on multiple redundant servers using the Microsoft Cluster Service. Programming Required SSNS is not a finished application but rather an "engine" that can form the core of a notification system. The amount of custom development needed will depend on the organization’s environment, event sources, and needs, and will fall into three basic categories: the subscription application, the core SSNS logic, and the event and notification interfaces. Subscription application. Microsoft includes sample code for an ASP.NET-based subscription application, but SSNS customers still must build and maintain a custom application to allow users to set their notification preferences. This application could be merely an extension of an existing e-commerce Web site, offering notification configuration options on the same pages that customers use to set their other preferences. Core SSNS logic. Programmers building the core notification application perform all coding "declaratively" by building two types of XML files that define the data schemas, query match rules, content formatting instructions, database paths, transport channels, version information, and more. Once these files are created, SSNS takes care of creating the SQL Server databases and tables and creates the links to the event sources and distribution channels. It also creates the SSNS Windows Service in which the Event Provider, Generator, and Distributor run. Event Provider and transport channel interfaces. If the customer’s event sources and transport channels are compliant with the standard SSNS interfaces, then little or no additional programming of these modules is needed. However, if the customer needs to link to other types of event sources (such as Oracle databases) or transport channels (such as via Microsoft Message Queue [MSMQ] or Simple Object Access Protocol [SOAP]) then the customer must build custom interfaces for them. Limitations Remain Although this will be SSNS’s first public release, Microsoft considers it a second version because of its experience running the first version at MSN, a very taxing, high-volume production environment. However, potential customers should understand some limitations of the product. No notification acknowledgement. In this version of SSNS, notification messages are passed on to a message transport, such as SMTP-based e-mail, and then forgotten. SSNS has no way of knowing if the user actually received or read the notification. Ideally, the system should support "closed loop" notifications, in which the user manually or automatically acknowledges a notification. The acknowledgement, once sent back to the notification system, would allow the construction of more intelligent notification applications. For example, notifications could be rerouted down alternate channels if the first notification goes unacknowledged within some time limit. Any notification application that conveys critical information should have this feature. .NET Alerts needed for presence information. In this version of SSNS, the Distributor simply sends notifications to the channel or channels specified by the subscriber. It has no capacity to use logic to select a distribution channel and device address based on the subscriber’s location and active device. .NET Alerts can provide this capability, but this incurs recurring service fees, and more could be done if the presence information were part of the core logic of the SSNS application. For instance, SSNS could send a critical notification to an alternate transport that sends the message to a user’s phone in text-to-speech format if it determined that none of the user’s data devices were active. Availability, Licensing, and Resources SSNS is currently in beta and will be available as a download later in summer 2002. It will also come standard with future versions of SQL Server. SSNS will be free to licensed SQL Server 2000 customers when run on the same machine as SQL Server. However, if the customer installs SSNS on a separate server, it will need to purchase additional SQL Server licenses for the SSNS server. For more information on .NET Alerts, see www.microsoft.com/myservices/alerts/default.asp. |
