.NET is complex. Not so much in the same way that COM is complex. Not in the way that makes you want to cry as you realize you’re going to have to scrub the registry for references toan old version of COM server again. Not in the way that gives you nightmares about ghoulish GUIDs taunting you from a misplaced type library. No, .NET’s complexity is based more onits sheer size and scale. There are more than 3,000 types in the Framework class library, and these types are designed to do just about anything. The major learning curve to becoming productive in the .NET Framework is not the language, regardless of your language of choice (although moving from VBScript to VB .NET Web Forms has been challenging for more than a few); it’s the Framework class library. It calls to question, What’s out there? When do I use it? How does it work?

Distributed applications are also complex. A layered architecture results in an application with a lot of moving parts. Simply displaying a data point within a web browser can involve using an object graph with dozens of instances, a call stack that’s easily five layers deep, code involving markup, one or more managed languages, Structured Query Language (SQL), and maybe even a proprietary database language such as Transact SQL (TSQL). The path of this stack may span processes, machines within a LAN, platforms and operating systems, and maybe even several LANs. As the architect, you have the task of designing the path of these requests and the rules of interaction for each step of the way. You’ll need to consider more than the business-based, functional requirements. When you’re designing the architecture, functional requirements may be relevant, but they’re usually secondary to other application requirements, which aren’t captured in the “Use Cases.” You’ll also need to address a multitude of questions:

• What are the scalability requirements for the application?
• How available must the application be, and how will someone know when it’s “down”?
• Is it customizable, and if so, when and by whom?
• Is it easy to deploy, upgrade, and maintain over time?
• Are there development cost constraints?
• What connectivity mechanisms will it employ (i.e., will users be fully connected, be
partially connected/mobile, use PDAs, etc.)?

These are the questions that keep you awake at night, at your desk, going over your solution one more time.

After you’ve established the technical requirements of the application, you must map these requirements to the technologies at your disposal, have them drive the adoption of logical tiers in the design, andthen decide how to physically deploy these logical tiers. You’ll need to address many questions in this process:

• What servers and services will be used at what tiers of the application?
• How will the data be accessed?
• How will requests be marshaled between the logical layers of the application?
• What about when those layers span processes—or span machines?

In this book, we provide a complete traversal of this call stack in the context of a .NET application. Each step of the way, we examine the options and services available to meet differentnonfunctional requirements at each tier and between adjoining tiers. This is not to say it’s a blueprint for a specific application architecture. You have many choices, and many more possibilities when you consider the number of ways you can combine those choices. The architecture you design depends on the requirements of your application. And believe it or not, all applications are unique.

The answer to any question when it comes to the architecture is, “It depends.” This is why you make the big bucks. This is why your job seems so hard. And it is. In this book, we give you the background to make it easier when the .NET Framework is your primary toolset. In the chapters that follow, we offer you a road map that enables you to navigate the application layers, choose what to use with each layer, and move information between layers.

Part 1 starts with this chapter, which provides you with an overview of .NET architectural problems and some examples of solutions. This gives you some concrete endpoints that show what your journey may look like. Then, we’ll move the focus to the point where a request comes into a web server. This may be a request for a Web Form (an ASPX page), but it may also be a request for a Web Service, a Remoted component, or some custom processor specific to your application. We take broad view of ASP.NET in this book, looking at its full features rather than focusing just on Web Forms.

Regardless of what’s being requested, web server requests travel a common pathway. How do these requests travel from Internet Information Services (IIS) to ASP.NET, and what are thepoints of extensibility along the way? We’ll examine this process in Chapter 2. In Chapter 3 and Chapter 4, we’ll focus on ASP.NET as a presentation tier. We’ll look at how you can maximize code reuse at the presentation tier, as well as discuss the internals for how an instance of a Page object becomes a stream of HTML. We’ll show you how the full features of the browser can be leveraged for the presentation tier, and will cover some of the improvements and new features of version 2.0 of the Framework.

In Part 2, we begin by discussing security in the .NET Framework, as security is the “vertical slice” that passes across all the layers in the stack. We’ll then look at Microsoft’s options and solutions for the middle tier of a distributed app, examining both those in the present and in the not-so-distant future. In this part of the book, we’ll also examine your options for marshaling calls for services and data between layers. Here, we’ll consider the options and the technologies in their different permutations. This is the most precarious balancing you’ll undertake during application design. Even after you’ve decided on the technologies to use for the individual tiers and the individual layers, the wrong choices for marshaling can impair your scalability on the one hand, and/or impair your dependability and availability on the other. The perils and importance of these decisions provide a fine segue into Microsoft’s next generation messaging stack: Windows Communication Foundation (formerly known as Indigo). Windows Communication Foundation’s design goal is to unify the call stack that’s used whenever out-of-process communication occurs, be it across application domains, inter-process, across machine boundaries, or across the world.

In Part 3, we move to the final critical tier of our application: the data access layer. This is where all the real action resides, where all the bits people actually want to see are housed, where even the nontechnical business users in your enterprise know the heart of the business lies. This is your critical resource in a distributed application, as it is the transactional nerve center of the enterprise. This is also the layer of a distributed application that tends to get the most reuse. Once you write something down, people tend to want to get to it. The decisions you make in this part of your architecture will make or break your suite of applications. ADO.NET exposes the data access object model in .NET. We’ll examine the ins and outs of using these managed providers of data access, how to pick the right tool for the job, and how to employ best practices when using them. We’ll also survey other data access services in .NET, and get a sneak peak at the next generation database server: SQL Server 2005.

ADO.NET exposes the data access object model in .NET. We’ll examine the ins and outs of using these managed providers of data access, how to pick the right tool for the job, and how to employ best practices when using them. We’ll also survey other data access services in .NET, and get a sneak peak at the next generation database server: SQL Server 2005.

Nonfunctional Requirements

Architectural requirements are defined by nonfunctional requirements, or quality attributes. These are the requirements of the application the business or functional requirements do not describe. It is your job to capture these, and to define a technical infrastructure that meets the captured requirements. A key deliverable you need to provide is a definition of the different atomic pieces of the application that will be used, and justification for using them by explaining how they meet different nonfunctional requirements. You also need to define how these elements will interact. You need to address how type information will be discovered by a calling processes; how the information will be marshaled to and from the service; the constraints of this interaction; the platforms that must be supported in this communication; as well as which pieces are public knowledge (part of the interface) and which are internal (hidden details of the implementation). You’ll need to answer all of these questions in order to design the technical infrastructure. Many things can fall into the realm of nonfunctional requirements. While these requirements can be considered separately, it’s their interactions that become the critical influence on the design: Many of them work against one another, creating a tension between them, and a balance must be struck. Let’s look at a few.

Availability

Availability concerns system failure. This brings to question: What are the points of failure in the system? How often do they become unavailable? And how does someone know when a system is unavailable? Further, when there’s a failure, how much time passes before the system becomes available again? Also, what percentage of the time does the application need to be available? In an environment where availability is a high priority, this is usually expressed in the “ninety nine and n nines” form. In these environments, it’s usually a given that a system has to be available more than 99 percent of the time. A system that’s down for three minutes every five hours will likely be problematic. That’s almost fifteen minutes per day. This measure doesn’t usually include planned downtime, for backups and routine maintenance, for example. On the other hand, some environments don’t need to be highly available. In many environments, as long as an application is available during business hours, folks are happy. However, as applications become more connected, more global, and more automated, requirements for availability will increase. Failure is inevitable. Hardware fails. Disk drives crash. Networks go down. Accepting this is a given; you provide for availability by adding redundancy to the system. You must add redundancy at each point of failure. Having redundancy at every point of failure is frequently called n+1 reliability. N is a measure of the amount of resources needed to do the job. The plus one provides the availability when failure occurs. Given n+1 reliability isn’t cheap; it’s only put in place for mission-critical applications. If a company’s entire revenue stream is web based, an unavailable website means zero dollars in the revenue stream. Suddenly the expense of n+1 reliability doesn’t seem like so much money, after all. ISPs that host the big websites typically have four T4 lines running to the building, one from each direction on the compass. They may have several diesel generators in case the power fails, and then in case a generator (or two) fails. They may also be fortified, to guard against sabotage, like a bank. The company adds capacity for the servers automatically when a spike in traffic occurs. All of this redundancy allows them to guarantee the sites they host will be available. Availability of static pieces of the system can be attained by scaling out, which is a fancy term for throwing more servers at the problem. Since one web server can go down, another is added to the mix, and a load balancer is put in front of them. A single IP now maps to more than one machine. Failover is provided by the load balancer, which will send all traffic to the live machine when one dies. If these machines are not physically colocated, some type of persistent connection needs to be maintained between them. This can be a factor when a failover strategy also needs to account for a disaster recovery scenario.

These requirements and the decisions made to meet them can affect the design of the software systems that will be hosted on this physical infrastructure. The introduction of a load balancer means anything stored in the memory of the web server can be in the memory of more than one physical machine. This may be fine for read-only information, where more than one copy is acceptable. But for mutable, or user-specific, information, this situation introduces a definite problem. The web server’s memory becomes an unsuitable location to store this information. It must be marshaled out-of-process, and stored in a central location. Failure to account for this means failure of the application when a web server blows a gasket. This problem is accounted for with out-of-process session state available in ASP.NET. State information can be stored either in an out-of-process state server (no redundancy), or SQL Server, which can be made redundant with clustering. This introduces a definite performance hit, but architecture is frequently about trade-offs. You must find a balance. Maybe this hit is not acceptable, and the application will be designed not to use session information at all. It depends.

Clustering will actually need to be present in a highly available system, regardless of how session state is being dealt with. Scaling out will not work on the database tier of the application for the same reason it doesn’t work with user-specific session information. It changes. There can’t be n copies of it, because these copies would start to deviate from one another. Clustering maintains a hot backup of a single server and a mirrored copy of the information written to disk that the system is dependent upon. Drive crashes? Switch over to the mirror. Server crashes? Switch to the backup. This is also called scaling up. “We can scale out at the application tier and scale up at the data tier,” says the architect who has done his homework (and has a big budget).

Clustering works great when you’re working with a single physical location. In a disaster recovery scenario with multiple geographic locations, it gets a lot harder and may not even be possible depending on the situation and budget. In such instances, you may still be able to consider clustering, but you’ll need to explicitly define the following:

• A geographically shared RAID or SAN, using a hardware- or software-centric approach
to data synchronization
• The interaction between Windows clustering and SQL Server clustering
• The size of the “pipe” between two or more locations
• Failover vs. failback

 There are other exceptions to these points as well. For example, you may be able to have more than one copy of a database if it’s a reporting server whose data is updated nightly with data from the previous day’s transactions and it doesn’t change during the course of the day. Scaling out is also possible in the data tier by horizontally partitioning your data. Application architecture is all about your enterprise’s exceptions to the rules. And exceptions to those exceptions. And repeat.

Scaling out and scaling up are used for more than availability. They’ll make another appearance when we discuss (oddly enough) scalability. false false false MicrosoftInternetExplorer4

Performance

Performance is frequently the most difficult metric to ensure, as it is often ill-defined, and many development shops don’t have the skill set, experience, time, and/or motivation to design and run performance tests. Consequently, performance problems often first rear their ugly heads after an application is deployed. Further, these problems tend not to show up right away—only after sufficient data storage thresholds have been met. “Of course the system should respond in less than seven seconds. That’s a given,” says the consternated manager whose direct reports have been complaining about system performance.

Performance strategies can often work directly against other system design decisions implemented to solve other requirements. You can use the layered software architecture to increase maintainability and reuse, though introducing new layers of abstraction does not necessarily increase performance. An overly deep call stack can actually impede it. However, other driving factors may actually increase performance. It depends.

Performance is closely related to other aspects of the system, like availability, scalability, and robustness. Often a problem in one of these other areas first shows up as a performance problem. A bad memory chip is an availability issue, but it may first be reported as a performance problem. Similarly, for availability or scalability reasons, you may choose to persist data via asynchronous mechanisms. This decision may actually be perceived by the user as a “performance” issue. For example: “I made an update to this widget record, but my changes weren’t immediate. I had to wait 4.3422 seconds for my changes to take effect.” While maybe it’s not ideal for the user, an asynchronous mechanism like Message Queuing (MSMQ) allows for peak load balancing, guaranteed message delivery, and a looser coupling between the presentation tier and service availability. Weighing these against users’ experiences and their tendency to crave immediate validation of their work is another architectural trade-off you must sometimes make.

Measurable requirements are necessary in order to test the performance of the system before putting it into production. These measures have to be more concrete than “It has to be as fast as possible,” or “There must be no noticeable delay.” There are many measures you can use, and you must design a load test that accurately reflects the expected use of the application in the real world. If you’re expecting peaks in traffic, you must test a load representing these peaks. You can measure the number of expected concurrent users, the transactions that are processed per second, or the raw throughput of the web server’s response stream. Ideally, you should test the system in its less-than-optimum states as well. If you have the website balanced between two web servers, what happens when one of them goes down? Can a single box deal with the entire load? If not, you may need to add a third server to the mix, two to deal with the expected load, and a third for availability (n+1).

There are many tools available from Microsoft that can help you with this process (see Table 1-1).

These are but a few of the tools available from Microsoft, and there is, of course, a rich third-party market for more advanced testing in this area.

Remember, if you don’t calibrate the performance of your application before putting it into production, you can make no claims about how it will perform once it’s deployed. A simple failure to adhere to a best practice, hidden deep in the call stack, could impede the performance of the entire system. This won’t show up as developers sit at their workstations and press F5 within IE as fast as they can. It must be put under load. This load should approximate the traffic you expect when the application goes into production.

While performance measures how fast the current system is, scalability is a measure of how much performance improves as resources are added to the system.

Scalability

Scalability describes the system’s capability to deal with additional load. You may have only a dozen hits a day when your application goes into production, but as it generates a buzz, you may shortly find yourself with hundreds or thousands of concurrent users. Will your application be able to handle this additional load? Have you designed it to scale up and/or scale out, and given it the capability to add capacity?

Scalability is closely related to performance, but they aren’t the same thing. Performance is a measure of a static deployment of the system. Scalability is the measure of how well adding resources to the infrastructure of the system improves its capability to service additional requests. As you add resources, do you get a corresponding bump in the throughput of the application? Or are you losing bang for the bucks that go into these resources?

There are two types of scaling: scaling up, which is also called vertical scaling, and scaling out, also known as horizontal scaling.

Vertical Scaling

This is used for the mutable portions of a system, or those that change over time. The database is a tier of the application that must usually be scaled up to deal with additional load, as scaling out at the database tier is difficult and complex. When the database reaches its capacity, you can add more memory, pop another CPU into the box, or purchase a more powerful box. A clustering solution can be introduced for failover, but this won’t help with the current load on the system. Scaling up is also used for the middle layer of an application when it hasn’t been designed to scale out. This is not a happy situation to find your system in.

Horizontal Scaling

This involves adding servers to the system, and balancing the load of traffic on the system between them. It’s sometimes called a Web Farm when it’s used for web servers, but scaling out can also be used for an application server. When more than one machine is performing work identical to other machines, the IP representing the farm is “virtualized.” That is, a load balancer is the first to receive all requests into the system, and that load balancer doles out the request to the servers configured in the Farm. Load balancers can be hardware- or softwarebased. How the load is distributed depends on the algorithm used by load balancer in use. Some take a “round robin” approach, and just rotate between the servers. Some will “poll” all of the servers in the Farm, and those responding quickly get hit with more traffic (the idea is to approach equilibrium between the servers over time).

Any state information maintained on these servers must be centralized when you’re using a Web Farm. Since requests can be sent to any machine in the mix, a change in the memory of one machine won’t show up in the memory of another. Session state, a feature of both classic ASP and ASP.NET, is an example of this type of user-specific, volatile information that’s stored per browser instance. The canonical example of what goes in the Session is the user’s shopping cart. This information must be maintained across different requests that the user makes. It must survive for the entire “session” that the user maintains with the application.

While no solution was built into ASP for out-of-process sessions, it’s accounted for in ASP.NET. Session information can be stored on a centralized state server, or it can be stored in SQL Server. When deciding to use out-of-process session state, keep in mind that only SQL Server provides a solution for n+1 reliability. If you’re using the NT State Server and that box goes down, your session information is unavailable, which will, in all likelihood, severely impair (or take down) an application that’s dependent on this information. Also be aware that session state in ASP.NET is application specific. ASP.NET provides no solution “out of the box” for sharing session information across IIS applications, which is unfortunate, because this is a common need. If you find yourself needing this type of information sharing, you’ll have to code your own solution.

If it’s entirely untenable to move session information out of process, there is one other option. Load balancers support an option called client affinity, or sticky sessions. This means that once a load balancer sends a given user to a specific server, it continues to send that user to that server for the duration of the user’s session (it sticks the user to that server). While this allows you to use in-process session information and still have an application that can scale out, it’s not the most efficient load balancing algorithm you can employ. It’s possible that some servers in the Farm will be under significantly more load than others. It depends on how long users “stuck” to a particular server use the application. If more users stuck to server A stick around, and those stuck to server B leave the application, server A could be under much more load than server B.

This solution also doesn’t provide for redundant availability of the application. If a server that a user is “stuck” to goes down, the user’s session information goes down with it. While a good load balancer sends those users to a server that’s still available, their session information will be lost, and depending on what’s being stored there, their experience with the application will likely be somewhat less than ideal. Once again, storing session state in SQL Server is the only way to provide for redundancy when using this feature.

Security

Security attempts to prevent nefarious persons from performing nefarious acts, and simpleminded fools from the tools they shouldn’t use. This runs a wide range of activities, from detecting and preventing a denial of service attack on a public web server, to keeping a known user from accessing a function he’s not meant to. You’ll also need to establish what happens once a security breach does occur. Do you have enough information to detect what happens? Do you have enough to recover? Can you restore your data to a previously known, good state?

you have enough to recover? Can you restore your data to a previously known, good state? There are three main steps to security: authentication, authorization, and auditing. Authentication involves identifying the users of your system, and denying access to functionality to those users who cannot identify themselves. Authorization concerns making sure authenticated users have permissions to run the function or see the data they’re attempting to access. Auditing ensures your ability to investigate and recover if something goes wrong with the first two. Can you tell there was unauthorized access or use of the system? Can you undo what was done?

Data must also be secured. You can secure data by encrypting it, or by keeping it in an otherwise secure data store. An opportune time for encryption is when you’re moving data around on the network, or sharing data with partners. Typically, when you’re done moving it around, you write it down in a source of persistence that keeps it safe for you, like within a relational database that requires credentials for access.

Maintainability

Maintainability is concerned with the evolution of your system over time. It’s highly unusual to ship an application and have all business stakeholders and users simultaneously proclaim “Yes! Yes! This is exactly what we needed! It does the job perfectly!” It’s more likely that they’ll start requesting changes right away. Sometimes they’ll wait a day while your team recovers from pulling an all-nighter to get the thing working in production in the first place, but when they do make requests, what type of changes to the system can you expect?

Your initial requirements may be quite a bit more ambitious than what you’ve committed to ship on the application’s first iteration. Office was not built in a day. However, knowing the requirements that will be present in future iterations can be of great benefit during the architectural design phase, as you can take some of these features into account in the solution.

The application may also have a subset of features that are so volatile that it may be worth the investment to create some user interfaces that are entirely polymorphic in their behavior, and create a tool for end users (or power users) to control how this portion of the interface gets rendered. There may even be a vendor-supplied tool that meets these requirements for you. Content management systems and Web portal packages are just a couple of examples of generalized solutions that let qualified users affect the application at a higher level of abstraction than cranking out and compiling code.

Your application may have requirements that can be met by network or application administrators via configuration files or Microsoft Management Console (MMC) snap-ins. These are tasks technical people need to perform, but they don’t necessarily require a developer to change code that then needs to be compiled and shipped.

Checking a code file out of source control and having a developer make changes to it is the most expensive kind of change that can be made to a system. It requires a developer (not cheap). It requires the recompilation of binaries. It requires regression testing of all areas of the application that are affected by a change to that binary (testers, automation, and time: aka more money). And then it takes a deployment and all of the requisite worry, heartache, and long hours that can accompany that.

“Customizability” frequently comes up when discussing these types of features. A fully customizable solution is a totally nontrivial undertaking that can doom a project to never even ship a decent V1 (think of how long it took Microsoft to get Access right … oh wait … that hasn’t happened yet …). But there may be key features of your application that you can move into configuration, or you can create an administrative applet to tweak, or for which a vendorsupplied solution nicely fits the bill.

The other type of change anticipation involves minimizing how many components of a system will be affected when the inevitable change is requested. Even if the anticipated change does require developer involvement, isolating that change from other functional areas of the system minimizes the number of binaries affected and, therefore, the complexity of the regression testing that must be done. This may be a choice as simple as making some set of functionality interface based, so that a developer can create new implementations of the interface, and the production system can use late-binding and Reflection to pick up and execute an assembly with the new implementation. Intertech Software has an ingenious shell application that can have entirely new applications added to it just by dropping an assembly that implements a predefined set of interfaces into a directory that the shell is “watching.” XCopy and you’re done; everything else flows from there.

This leads to an important tenet of service design, that of not creating tight couplings between service components in the system. You don’t want to be in a situation where a change to a single class causes a cascade effect of changes to other components. This atomic design is the thrust behind “loosely coupled” services. Not only does this increase maintainability, but it also increases the reuse of your services, as the more atomic and independent they are in the work that they do, the more ways they’ll be able to be combined with other services.

Other Nonfunctional Requirements

There are many other requirements that may be relevant to your system. We summarize some of the others that may come up in Table 1-3.

Nonfunctional requirements will definitely influence the design of any architecture. Some of these requirements don’t affect the software architecture; others do so indirectly; while some have a direct influence. Over the course of the book, as we examine different servers, services, and packages available within and provided for the .NET Framework, we’ll revisit these requirements in-as-much as how the features a solution provides are relevant.

Some are far more important than others. Performance is always a critical consideration when you design a system. If it does not perform, it will not be adopted. We’ll address this throughout the text in discussions of extensibility, how to know what services to use when, and applied best practices in different areas.

Security is another critically important feature of any system. Not only because it spans all of the tiers of a distributed application, but also because it’s the mechanism for protecting critical enterprise data. The reputation or even the continued existence of an enterprise can depend on it. We dedicate Chapter 5 to a discussion of security and how it works in the .NET Framework, and subsequent topics address it specifically as well. You have many different considerations for keeping an application secure as you move across the tiers.