The world wide web has created a standardized communications infrastructure that has enabled a wide range of applications, especially for business to business eCommerce, customer support, and entertainment. The rapid design and deployment of web applications has been done largely in the absence of performance considerations. In addition, there have been great difficulties with forecasting site access patterns and dealing with the scalability issues of an almost unlimited audience. Thus, it is not surprising that web-based applications frequently experience problems with poor availability and long response times. Below we describe the performance metrics for detecting and resolving such problems.
The intent of this article is to provide a systematic and somewhat deeper look at the performance of web-based applications. This is made possible because of a newly developed technology described in HMMT99 that has been employed for client side instrumentation of browser interactions. In essence, probes are placed in the sockets layer of the IP stack. This provides a way to track key events (e.g., name resolution) as well as recognize other protocols and their key events (e.g., HTTP and SOCKS). These events are then used to define the duration of activities of interest (e.g., time for name resolution). In addition, the technology in HMMT99 also provides a way to report message sizes. Doing so allows us to compute efficiency metrics.
Performance Metrics: Efficiency Metrics
This section describes performance metrics that quantify the overheads of a page retrieval. These overheads are expressed in terms of a ratio between what’s required for the payload (e.g., application data) and the entire message sent. We use the data in Table 1 to illustrate the computation of many of the metrics described.
Application Data Encapsulation Efficiency (ADEE)
A web transaction comprises a request and related response that communicate application specific data by using web standard protocols (e.g., HTTP, SSL, Socks, DNS). Data Encapsulation Efficiency is the ratio of application data bytes to the sum of one or more overhead bytes (from protocol or otherwise) and application. Overhead data is broadly defined as anything that isn’t application data and is typically used to get the “payload” data back and forth to the requesting application (e.g., the browser).
Take the following example using the information from Table 1 above, the retrieval of a web page component (e.g., the HTML Document) may involve sending an HTTP Get Request (G), followed by retrieving the HTTP Response comprising an HTTP Reply header (R) and the HTML document’s data (i.e., the HTML document’s data) (H). In this example, the Application Data Encapsulation Efficiency would be computed as (H)/((H)+(G+R)) and given the example numbers would be .817 or around 82%. This metric will by definition be in the range [0,1].
One variation on Application Data Encapsulation Efficiency might include allowing the cookie (C), which is sent in the HTTP Get Request, to be considered part of the application data, thereby making the computation: (C+H)/((H)+(G+R)) and given the example numbers would be .857 or around 86%. Yet another variation might include the overhead associated with navigating a firewall using the socks protocol (S), which would yield the computation (assuming we also wish to consider the cookie as application data): (C+H)/((H)+(G+R+S)) and given the example numbers would be .839 or around 84%.
Transport Data Encapsulation Efficiency (TDEE)
This performance metric is similar conceptually to Application Data Encapsulation Efficiency except the divisor is increased to include more overhead data, namely, the sum of bytes used to transport the data (e.g., TCP and IP header byte counts). The difference between Application and Transport Encapsulation Efficiencies is that application developers may have control over the ADEE, whereas the TDEE is also controlled by the design and implementation of the communications environment used to carry the transaction.
When the ADEE or TDEE are small, they imply a high contribution of overhead in requesting and retrieving web pages (and their component pieces). This might lead designers to investigate combining component pieces into larger payloads, thereby requiring fewer request and responses. An example might be to use a menu bar GIF rather than individual menu item GIF’s arranged on the page to form a menu bar. By retrieving the entire menu bar in one operation, only one pair of HTTP headers would be used, rather than a pair for each menu item.
Low TDEE values or percentages might imply the communications layer needs to be reconfigured to package larger payloads within the TCP/IP protocol packets. There is a concept of Maximum Transmission Unit (MTU) that is often set differently depending upon the connectivity medium (e.g., dialup vs LAN). It is possible that the setting is too small (e.g., for the dialup environment) when the communications could support a larger value (e.g., for the LAN environment). Sending more payload per protocol packet reduces the number of protocol headers, but increases the likelihood that an error could be introduced requiring retransmission. This is why the less dependable communications channels tend to reduce the MTU. At any rate, the low TDEE value would indicate that greater efficiency could be had by increasing the MTU if the medium is capable.
Application Image Weight (AIW)
Web transactions may communicate information using images as well as text. This performance metric calculates the ratio of image-based (e.g., GIFs, JPEGs) application data bytes to the sum of image-based and non-image (e.g., HTML text) application data bytes. Given the information from Table 1 above, this would be given by I/(I+H) or .684 or about 68% image weighting. This ratio may be useful in measuring policies for content generation. For example, web sites may elect to have pages heavily geared toward graphical navigation at the “top” of their web site page hierarchy, with pages containing more textual details lower in the hierarchy. Another use of this metric might be to gauge whether or not non-image data is being transmitted containing information such as formatting whitespace, or comments on pages expected to be predominantly image based.
Encapsulated Application Image Weight (EAIW)
This performance metric adds the additional byte counts for protocol overhead involved in requesting and retrieving the image-based application data to the image-based application data bytes and uses the result as the numerator to calculate a ratio by dividing it by the total bytes (protocol overhead and application data) for the transaction. Again, the some or all of the different overhead bytes (protocol and otherwise) might be chosen.
Referring now to Table 2 below, displays measurements of certain data quantities in a web transaction to retrieve a web page. These activities include measuring the bytes used to go negotiate security protocols, the bytes sent when the application data is fully encrypted (i.e., includes encrypted bytes from the Get Request and Reply headers), the unencrypted bytes in the HTTP Get Request, the unencrypted bytes in the HTTP Reply, the unencrypted bytes in the web page, and the total unencrypted bytes corresponding in the fully encrypted application data (i.e., includes bytes in the Get Requests and Reply headers).
Security Negotiation Efficiency (SNE):
This performance metric is the ratio of the encrypted application data bytes (E) alone, to the sum of bytes used to negotiate security (N) and encrypted application data bytes (E). For example, using the Secure Socket Layer protocol, security is negotiated between the client and server, including initial hello’s, key exchanges, and cypher selections. This will vary depending upon the reuse of sockets for subsequent requests. This ratio expresses the cost to “set up” a secure connection and is given by the equation E/(N+E). Using the example numbers from Table 2 above, the SNE would be .763 or around 76%.
A lower SNE value would indicate security negotiations are taking place too often while a higher SNE value would indicate efficient reuse of the security negotiation on keep-alive sockets.. One could use the SNE ratio to see how well their secured web page retrievals were managing the security negotiations. There are some cases where improperly configured security options in a browser or session time out settings at the web server will cause security to be renegotiated more often than necessary.
Performance Metrics: Concurrency metrics
As observed in Section 2, web browsers attempt concurrent retrieval of objects in order to reduce the total time to render a page. This section describes Component Activity Concurrency Efficiency (CACE), a metric the quantifies the efficiency with which concurrency is exploited.
The Component Activity Concurrency Efficiency (CACE) performance metric is a measure of how efficiently the transaction has scheduled its component activities so that serial processing is kept to a minimum. An example is a browser that uses multiple threads or sockets to retrieve component URL’s required to fully render a requested page from a web site. Essentially, it is desirable to maximize concurrency in order to increase performance. An example would be sending out one or more requests to the server just before beginning rendering of data just received so the browser can work while the requests are transmitted to the web server, the web server generates its responses, and the responses are returned to the browser.
The CACE performance metric is a relationship between the total serial execution time (SET) for a set of activities, the total concurrent execution time (CET) for the same activities, and the non-concurrent execution time (NCET) for the same activities. The total serial execution time is the summation of the individual activity times without regard for any overlap in time (i.e., as if the had been executed serially one right after the other). In Figure 4, the total serial execution time for the activities B-E is simply the addition of the respective duration in time units and the SET value is therefore 34 time units (34=5+14+8+7).
Activity Duration Aggregation (ADA) reflects the amount of linear time when at least one activity is active and involves summing the durations of non-overlapped portions of these activities added to the sum of the duration of the intersections of these activities (e.g., the duration in linear time when at least two activities overlapped). In other words, it is the sum of the total concurrent execution time (i.e., those times when more than one activity is executing) with the total non-concurrent execution time (i.e., those times when only one of the set of activities is executing) for the set of activities, or ADA = CET + NCET.
The distributed nature of the web and the structure of HTML documents create challenges for analyzing the performance of web-based applications. Herein, we describe a number of performance metrics that can aid in this analysis. Briefly, these are:
- Response time components: DNS resolution time, connection time to SOCKS server, connection time to web server, time to receive the first byte of data, and data delivery time).
- Efficiency metrics that gauge various overheads: application data encapsulation efficiency, transport data encapsulation efficiency, application image weight, encapsulated application image weight, and security negotiation efficiency.
- Concurrency metrics: component activity concurrency efficiency, activity duration aggregation, and serial execution time.
As indicated in the text, our experience has been that these performance metrics provide key insights into the cause of performance problems and the actions to take for their resolution.
Information from www.research.ibm.com