\documentclass[11pt]{article} \usepackage{times} \input amssym.def \input amssym.tex \usepackage{epsfig,psfig} \setlength{\textwidth}{6.2in} \setlength{\textheight}{9in} \setlength{\oddsidemargin}{.2in} \setlength{\topmargin}{-0.25in} \setlength{\headheight}{0in} \makeatletter \def\section{\@startsection {section}{1}{\z@}{-3.5ex plus -1ex minus -.2ex}{2.3ex plus .2ex}{\normalsize\bf}} \makeatother \makeatletter \def\subsection{\@startsection {subsection}{1}{\z@}{-3.5ex plus -1ex minus -.2ex}{2.3ex plus .2ex}{\normalsize\bf}} % put a period after section or subsection number in header \def\@sect#1#2#3#4#5#6[#7]#8{\ifnum #2>\c@secnumdepth \def\@svsec{}\else \refstepcounter{#1}\edef\@svsec{\csname the#1\endcsname.\hskip .75em }\fi \@tempskipa #5\relax \ifdim \@tempskipa>\z@ \begingroup #6\relax \@hangfrom{\hskip #3\relax\@svsec}{\interlinepenalty \@M #8\par}% \endgroup \csname #1mark\endcsname{#7}\addcontentsline {toc}{#1}{\ifnum #2>\c@secnumdepth \else \protect\numberline{\csname the#1\endcsname}\fi #7}\else \def\@svsechd{#6\hskip #3\@svsec #8\csname #1mark\endcsname {#7}\addcontentsline {toc}{#1}{\ifnum #2>\c@secnumdepth \else \protect\numberline{\csname the#1\endcsname}\fi #7}}\fi \@xsect{#5}} % put a period after theorem and theorem-like numbers \def\@begintheorem#1#2{\it \trivlist \item[\hskip \labelsep{\bf #1\ #2.}]} \makeatother \thispagestyle{empty} \begin{document} \begin{center} {\Large {\bf The Internet and other networks: Utilization rates and their implications}} \\ \vspace{\baselineskip} Andrew Odlyzko \\ \vspace{0.5\baselineskip} AT\&T Labs - Research \\ amo@research.att.com \\ \vspace{\baselineskip} Revised version, February 26, 2000. \vspace{.5\baselineskip} \end{center} \setlength{\baselineskip}{1.5\baselineskip} \paragraph{Abstract.} Costs of communications networks are determined by the maximal capacities of those networks. On the other hand, the traffic those networks carry depends on how heavily those networks are used. Hence utilization rates and utilization patterns determine the costs of providing services, and therefore are crucial in understanding the economics of communications networks. A comparison of utilization rates and costs of various networks helps disprove many popular myths about the Internet. Although packet networks are often extolled for the efficiency of their transport, it often costs more to send data over internal corporate networks than using modems on the switched voice network. Packet networks are growing explosively not because they utilize underlying transport capacity more efficiently, but because they provide much greater flexibility in offering new services. Study of utilization patterns shows there are large opportunities for increasing the efficiency of data transport and making the Internet less expensive and more useful. On the other hand, many popular techniques, such as some Quality of Service measures and ATM, are likely to be of limited usefulness. \section{Introduction} \hspace*{\parindent} An extensive study of data networks is documented in \cite{CoffmanO,FishburnO,Odlyzko2,Odlyzko3,Odlyzko4}. This paper presents only a brief summary of the results of that study, and concentrates on their implications for present and future data networks. %For sources of the data %and details of the arguments, see those papers. Utilization rates of networks have been strangely absent from most papers on the economics of the Internet, such as those in \cite{MacKieM,McKnightB,Varian}. However, these rates determine costs of services, since transmission links are priced by their maximal capacity. Furthermore, utilization rates are the primary means by which network managers determine quality of transmission. Therefore it seemed important to consider current utilization rates on the Internet and the resulting costs. Such a study was carried out in \cite{Odlyzko2,Odlyzko4}. It uncovered a number of surprising results. For example, corporations in the U.S. spend more to transmit large files over their packet networks than they would if they used modems over the switched voice network. The primary culprit behind this phenomenon is the low utilization of most data networks. A key point of the investigation of \cite{CoffmanO,FishburnO,Odlyzko2,Odlyzko3,Odlyzko4} was the need to consider not just the public Internet, but the full universe of data networks and their role in the economy. For simplicity, only networks inside the U.S. were considered. Although they do attract intense attention, they are a small part of the entire IT (information technologies) industry. While the IT sector of the U.S. economy accounts for about \$600 billion per year \cite{DOC}, data communications costs about \$80 billion, and of that, transmission comes to about \$16 billion \cite{Odlyzko4}. (By comparison, the telephone system revenues are close to \$250 billion per year. Even technically sophisticated corporations are still spending more on voice communications than on data.) Thus any modification to data networks has to be considered in light of the total costs it imposes on the economy, not just in terms of what it does to networks. Data networks are not only still a small part of the economy, they do not operate in isolation. For voice calls, the basic service is easy to describe (Fig. 1), and there is general agreement on desirable quality. On the other hand, data networks increasingly are becoming just enablers (although crucial ones) of other services, and few users care about the network by itself. What is important is how the entire application is perceived by the user, and network transmission is only a part of the system that makes that application possible. %It is thus impossible to provide any simple quantitative criteria for %what is required of a data network. Even in the restricted realm of data networks, the public Internet (those parts of the Internet accessible to general users) is a small fraction of the total system. Measuring networks by their maximal transmission capacity, it was estimated in \cite{CoffmanO} that at the end of 1997 in the U.S., the switched voice network was probably still the largest, but the private line networks were about as large, and the public Internet was considerably smaller. Updating the estimates of \cite{CoffmanO}, using the same methodology, yields the following figures for the effective bandwidths (see \cite{CoffmanO} for definition), measured in Gbps (gigabits per second) at the end of 1998: \begin{center} \begin{tabular}{lr} network & bandwidth (Gbps) \\ \hline US voice & 375 \\ public Internet & 150 \\ other public data networks & 80 \\ private line & 400 \\ \end{tabular} \end{center} Thus looking just at the public Internet does not give a proper perspective on data networks, especially since utilization patterns of private networks are considerably different. Although data networks are about as large as the voice network in bandwidth, the voice network still dominates in carried load, and is likely to do so for a few more years. The traffic, measured in TB/month (terabytes per month), through various networks at the end of 1998 was (again updating the estimates of \cite{CoffmanO}): \begin{center} \begin{tabular}{ll} network & traffic (TB/month) \\ \hline US voice & 43,000 \\ public Internet & 5,000 - 8,000 \\ other public data networks & 1000 \\ private line & 4,000 - 7,000 \\ \end{tabular} \end{center} A comparison of the two tables above shows that there are substantial differences in utilization rates between the voice network and data networks. The question is what this means. It is my contention is that by considering these statistics as well as more detailed ones, we can deduce much about user preferences in data services, and about their willingness to pay for various quality levels. George Gilder's thesis is that bandwidth supply will soon be increasing so rapidly that we will not have to worry about network congestion. However, that agrument is not entirely convincing. While technology will indeed increase supply, costs and prices are not the same in industries with high fixed costs, regulatory concerns, and substantial barriers to entry. More important, while supply will be increasing, so will demand. Hence it is the balance between the two that will help determine the future of networks. Section 2 is devoted to disproving a variety of common myths about the Internet. It is based on evidence that is quantitative, although not as precise as one might hope for. Section 3 presents an evaluation of the reasons for the different utilization levels of voice and data networks. Later sections are increasingly speculative, dealing with the likely evolution of the Internet. Section 4 uses the observations about the utilization patterns of current data networks to explain the failure of ATM and the poor prospects for many Quality of Service (QoS) schemes. Section 5 discusses the opportunities for increasing the quality and lowering the costs of the Internet by learning from the experience of the switched voice network. Section 6 deals with the role of differential service levels and usage sensitive pricing. Section 7 speculates on the most important factors that are likely to influence the evolution of the Internet. Finally, Section 8 presents some final conclusions and predictions. \section{Common wisdom or common misconceptions?} \hspace*{\parindent} Much of the ``folk knowledge'' about the Internet is simply false. This section discusses the most important examples. {\em Packet networks are not necessarily more efficient than the switched voice network.} A key point, to be addressed in Section 3, is what is meant by efficiency. In general publications, though, it is often asserted without qualification that packet networks are less expensive then the switched voice network. Some of the new packet-only carriers have been showing comparisons in which IP transport saves more than 90\% over the cost of traditional switched networks. In particular, savings on transport costs are widely perceived as the main advantages of carrying voice over packet networks. On the other hand, when one considers existing networks, it turns out that most corporations spend more on transferring large files over their internal IP networks than they would if they used modems over the public switched voice network. This is an astounding result, since modems use only a small fraction of the bandwidth of the digital channel that is provided for voice calls. Further, network costs of voice calls are small compared to the prices charged. The estimates for the cost of transmitting a megabyte of data over various networks are estimated in \cite{Odlyzko4} as follows: \begin{center} \begin{tabular}{ll} network & dollars/MB \\ \hline modem & 0.25 - 0.50 \\ private line & 0.50 - 1.00 \\ Frame Relay & 0.30 \\ Internet & 0.04 - 0.15 \\ \end{tabular} \end{center} This table suggests an obvious question: Why don't corporations junk their private networks and send data via modems over the public switched voice network? The answer is that the cost estimates of the table apply only to large file transfers, and do not take into account other factors, such as latency. As an example, a credit card authorization involves transfer of only a few hundred bytes, and so would cost far more over a modem than the table might suggest. It would also take far longer, tens of seconds instead of seconds, and thus lead to lower productivity of the sales force and customer dissatisfaction. There are thus unbeatable advantages to packet networks, but they are not in network costs, but in flexibility. %That is where common wisdom %goes seriously astray. {\em The public Internet is small relative to other data networks.} Although it is the public Internet that has caught all the attention, it is still dwarfed in transmission capacity and costs by the private line networks, as was shown in the tables in the Introduction. It is also far smaller than the switched voice network. However, it is growing much faster, about 100\% a year, than either the voice network, which is growing at around 10\% per year, or the private line networks, which are growing at around 20-30\% per year. (See \cite{CoffmanO} for details.) Therefore in a few years the public Internet will be the dominant communication network, but it is not that yet. {\em Few data networks are congested.} A surprising fact is that even though it provides high quality service, the switched voice network has considerably higher average utilization than any large collection of data networks. There is a general perception that the public Internet is hopelessly crowded, and even most network experts believe that private line networks are congested as well. Reality is different, as is shown in \cite{Odlyzko2} and summarized in the table below. (The utilization rates in the table above, and elsewhere in this paper, refer to averages over a full week.) \begin{center} \begin{tabular}{lc} network & utilization \\ \hline local phone line & 4\% \\ U.S. long distance switched voice & 33\% \\ Internet backbones & 10-15\% \\ private line networks & 3-5\% \\ LANs & 1\% \end{tabular} \end{center} Some parts of the Internet are highly congested, especially the public peering points, the NAPs and MAEs. Many university links to the public Internet are also heavily loaded, which may have persuaded generations of students that all networks are heavily utilized. However, the backbones of the Internet are relatively lightly loaded. The estimates of their utilization rates in \cite{Odlyzko2} (based partially on estimates of sizes of various networks in \cite{CoffmanO}) are consistent with recent measurements which show that as long as transmission stays on a single backbone, latency and jitter are not a problem. What is congested are many of the feeder links to the backbones from smaller ISPs, especially those that aggregate modem traffic. Fig. 2 shows the traffic pattern on a T1 line (1.5 Mbps) belonging to an ISP. It runs at a high fraction of its capacity for large parts of the day, but still manages to provide relatively high quality service, with minor delays and packet losses. (Average utilization is in the 40-45\% range.) Other ISP links show even higher utilization, frequent saturation, and high packet loss rates. (Other examples of traffic patterns of ISPs, as well as those of other users, are in \cite{Odlyzko2,Odlyzko4}.) As the tables in the Introduction show, most of the data transmission capacity is in private corporate networks. Their traffic patterns tend to be far different from those of the ISP line profiled in Fig. 2. Fig. 3 shows utilization of a corporate T1 line. The average utilization of this line is slightly under 1\%. Comparing the graphs of figures 2 and 3, it is easy to grasp that the performance of those two lines will be different, and that traffic control algorithms suitable for one might not fit the other one. Utilization of the T1 line in Fig. 3 is lower than the average 3-5\% cited for corporate networks in the table. Thus a more typical link would show somewhat higher traffic than Fig. 3, but it would still look very low compared to the traffic in Fig. 1. It is worth noting that the line in Fig. 3 does have occasional spikes of higher utilization, but they tend to be shorter than the 5-minutes averaging interval. On some days it also has 5-minute spikes much higher than those of Fig. 3, which represents a pretty typical business day for that link. While most corporate networks are run at low average utilizations, there are many exceptions. The most prominent are international lines, such as the one profiled in Fig. 4. The average traffic shown there is 59 Kbps, or 46\% of capacity during the day that is profiled. On this particular link there is little traffic in the reverse direction, so average utilization of the entire line (which, as is always the case in current data network as a legacy of the switched voice network, consists of two one-directional links), during a business day is around 25\%. Over a full week, average utilization might therefore be expected to be around 20\%. Some international links have higher utilizations, over 30\%. I do not have enough data to be certain, but it appears that the average utilization of trans-oceanic corporate private lines might be in the 10-20\% range. {\em Congestion is not necessarily the biggest problem on the Internet.} The ``World Wide Wait'' is often caused by problems other than lack of bandwidth. A study carried out in 1997 by Christian Huitema [Huitema] about accessing some popular servers showed that 20\% were not reachable. Among the 80\% that could be reached, 42\% of the delays were caused by network transmission, with DNS accounting for 13\% and servers for the remaining 45\%. Further, there are some indications that in the last couple of years, the performance of the backbones has improved, while servers are falling behind. If bandwidth were the critical resource in corporate networks, then surely it would be monitored closely. Yet the study of utilization levels in \cite{Odlyzko2} was hampered by lack of data. Existing data was hard to obtain, since it is usually regarded as sensitive. In most cases, though, there was no data to release, since utilizations are not even measured consistently. Hence bandwidth is presumably just one of many problems that network managers have to deal with, and not necessarily the most important. In general, although the Internet is acclaimed as the simple network, it is extraordinarily complicated and hard to keep operational. A simple example is the description in \cite{WiltziusBD} of the difficulties in getting a network operational, but any network operator can provide a multitude of stories of problems that crop up. One of the major problems with the Internet is that most of the work of keeping it running has to be done at the edges, and so is wastefully duplicated, with network administrators at thousands of institutions doing essentially the same planning, implementation, and operational tasks \cite{Odlyzko3}. Often (and perhaps almost always) networks are not optimized because there are not enough skilled experts to do it. {\em ``The tragedy of the commons'' may not be an insurmountable threat for the Internet.} It is widely believed that queueing by congestion is the only way to run the Internet, since demand, driven by flat rate pricing, is insatiable. There are many situations where adding capacity does not help. For example, road congestion in metropolitan areas can be relieved only temporarily by building more highways. The problem is that when travel speeds increase, people move further away, to take advantage of less expensive housing, opportunities to be closer to family, and other reasons \cite{Gibbs, SchaferV}. However, in a dynamic environment with growing bandwidth, this argument is questionable. The ``tragedy of the commons'' argument is even suspect in slower growing industries. Consider, for example, the following statistics on local calling in the United States, based on data in \cite{FCC}. Such calling is overwhelmingly paid for by a fixed monthly fee, and is thus insensitive to the number and length of calls. \begin{center} \begin{tabular}{cccc} ~ & ~ & local calls & local calls \\ ~ & lines & (minutes per & (minutes per \\ year & (millions) & day per line) & day per person) \\ [+.15in] 1980 & 102.2 & 39 & 17.5 \\ 1988 & 127.1 & 39 & 20.2 \\ 1996 & 166.3 & 40 & 25.1 \\ 1997 & 173.9 & 42 & 27.3 \end{tabular} \end{center} The amount of calling per line has not changed appreciably in 17 years, although the ``tragedy of the commons'' analogy suggests it should have grown. The standard reply to this is that people have limited time, and so their demand for phone service has already reached saturation. Yet that is not the case, since the amount of calling per person has grown vigorously, over 55\% between 1980 and 1997. (This includes fax and modem calls). Individuals and institutions have voted with their pocketbooks for low utilization, presumably because it was perceived to provide higher quality of life and work. In data traffic, growth has been much faster than in telephone lines, and links have tended to get saturated. This is often cited as a prototypical ``tragedy of the commons'' problem. Yet the situation is less clear than it might seem. Growth has been orderly. Fig. 6 shows the average traffic from the public Internet to the University of Waterloo. (See \cite{CoffmanO,Odlyzko4} for more details.) Although the capacity of the link has had several sudden jumps, usage has grown at a pretty steady 100\% a year. Similar steady growth rates have been seen in other networks, see \cite{CoffmanO}. Thus these networks have not in general had to cope with sudden surges in demand that saturated new capacity as soon as it became available. Even when such surges materialized (as they did at the University of Waterloo when student dorms were hooked up to the campus Ethernet), they were contained by simple local measures. The orderly, although very rapid, growth in data traffic is probably caused by a combination of several factors. Available content, user skills and awareness, and the network infrastructure all have to develop in parallel. The last factor is especially important. Users don't just go and exploit the Internet. Except for cases of malice, they usually need to have a substantial local infrastructure in order to make use of the wider networks, and this limits the growth in demand for connectivity. Not only is growth of data traffic steady, actual traffic is generally predictable once it is sufficiently aggregated. Several of the graphs in this paper, as well as many of those in \cite{Odlyzko2,Odlyzko4} combine displays of traffic for several days. It is noteworthy that the traffic patterns are generally consistent from week to week, with Monday through Thursday usually behaving the same, and Friday, Saturday, and Sunday each having its own particular load graph. This is the same behavior that has been observed on the switched voice network, and goes counter to the claim that data traffic is chaotic and is only constrained by congestion. {\em The ``bursty nature of data traffic'' is not the culprit behind low utilization rates of data networks.} Data traffic does not smooth out as well as switched voice traffic, and it shows long range dependence \cite{FeldmannGWK,LelandTWW}. However, that does not mean that high utilization cannot be achieved. Figures 4 and 5 show that it can. In Fig. 5, we see essentially full utilization over 9 hours during the business day, and in Fig. 4, for a much smaller link, more than 80\% utilization over a comparable period. Most of data transport uses TCP, which fills available bandwidth and can produce high load factors. Thus low utilization has to come from a different source, and the next section will be devoted to this topic. As a preliminary step, let us note that high utilization carries a penalty. For example, during the hours of peak usage in Fig. 4, the average packet drop rate was around 5\%, so service quality was substandard (and the throughput figure was deceptively high, since it included substantial retransmissions). High utilization thus had to come from explicit or implicit choices about the quality of data transport to be provided. \section{Economic efficiency versus engineering efficiency} \hspace*{\parindent} Economic efficiency means satisfying customers' demands. Engineering efficiency means providing a service with the minimal amount of resources. The two are often in conflict. The switched voice system provides a compromise that is more efficient than the Internet in both economic and engineering aspects. It gives a high quality service to its customers any time they want to use it, and yet manages to have a higher utilization rate of the transmission facilities. It does so at a rather high monetary cost, though, and offers little flexibility. There are basic reasons for low utilization rates of data networks that are caused by the rapid growth rate of data traffic and the economies of scale in purchases of transmission links. These reasons are discussed in \cite{Odlyzko2}. However, the main cause of this low utilization rate of private line networks (and thus of most of the Internet) is the attempt to satisfy user needs. Business customers with traffic patterns such as the one in Fig. 3 could carry all their traffic on 56 Kbps lines instead of T1s. Their decision not to do so shows what they find desirable and affordable. The manager of a branch lab of a major software producer described the private line from that lab to company headquarters as follows: \begin{quote} I see peak bandwidth as the basic commodity I buy. ... When we had a 256Kb data line it was too slow (it interfered with productivity). With a T1 line, no one has complained. I guess our T1 line is less than 1\% utilized. ... I would not go for a T3 line (it would not improve our productivity) but I would not cut back on the T1 line. \end{quote} Note that this manager does not care about average utilization of the line, in common with most people in similar positions. (The previous section noted already how seldom utilization data is collected.) What he cares about is that he and his coworkers get what they need quickly. Thus latency is the key issue. However, it is not packet latency, the concern of most of networking literature, that matters. It is transaction latency, the time it takes to complete whatever one cares about (transmitting an email message or downloading a Web page), that is important. Transaction latency can be lowered to a large extent by going for higher bandwidth. Since most of the traffic on the Internet is HTTP, it is often dismissed as just Web-surfing. However, Web-surfing includes customers downloading product information or placing orders, as well as employees accessing corporate databases. Therefore it often deserves high priority. (In general, trying to assign priorities to packets based just on the application is unlikely to be productive.) However, Web-surfing does produce traffic patterns such as that of Fig. 3, with low average utilizations, when it meets user demands. An important feature of the quote above is that it did not refer to any quantitative measures of network performance. There are certainly studies of what performance is required for tasks such as transaction processing, and how different network technologies compare in satisfying those requirements \cite{Cavanagh}. It appears, though, that increasingly interactions with networks are becoming more complex, and can only be judged by subjective criteria of user satisfaction. The importance of the subjective factor in judging network performance is also apparent in the choices of circuit speeds. They are usually chosen in simple multiples of some basic rate (such as 56, 128, 256, 512 Kbps) even when intermediate speeds are available. The human perceptual system operates on a logarithmic scale, and therefore it requires large steps to achieve a noticeable improvement in performance. Low utilizations of data networks should not be surprising. Note that the family car and the phone set are used on average around 4\% of the time. The Pentium III PC on an office desktop is idle most of the time, and does little that a 486 machine could not do. However, when it is called upon to typeset a document or recalculate a spreadsheet, it can do so much faster than the 486 machine, and that justifies its purchase. Comparisons of PCs never discuss how much they will be used, and instead concentrate on benchmarks showing how much time those PCs take to perform various tasks. What the resulting low utilizations mean is that the lightly utilized resource is inexpensive enough compared to the value people place on its availability and their time. Note that two or three decades ago, computers were largely mainframes, were kept in computer centers, and were run at high utilization rates. By moving to PCs we have lowered the utilization of the equipment, but have provided much more flexibility. Low utilizations of corporate lines, such as that of Fig. 3, show the value of high quality transmission. On the other hand, the high utilizations of international links, such as that of Fig. 4, show the limits of what even corporations are willing to pay. When links are as expensive as they are now across the Pacific or the Atlantic, it appears that IT managers either implicitly or explicitly decide to make their users put up with congested networks. The high utilizations of ISPs links most likely reflect a combination of extreme price sensitivity of the residential modem customers and of the difficulty those customers would have in deriving any benefit from uncongested links. The high latencies and low transmission speeds of modems would guarantee low quality experience in any event. The comparison of utilization rates of trans-oceanic and domestic corporate links suggests that if prices come down, users will opt for higher quality transmission and thereby lower utilizations. This is what appears to have happened in the LAN environment. I have much less data here than for long haul networks, especially historical data. However, many people say that average utilizations of LANs a decade ago were in the 5\% to 10\% range. (It is not clear, though, whether this refers to business hours alone or a full day). That is also consistent with a few scraps of hard data, such as the statistics for the Bellcore LANs in the early 1990s in [LelandTWW]. On the other hand, today LAN utilizations at the institutions that I was able to obtain the data for (which is not many, and consist primarily of those places that use the MRTG tool \cite{MRTG}), tend to be around 1\%. (This is not to say that this is universal, as there are insitutions with higher rates.) Further, in those institutions that have both 10 Mbps and 100 Mbps Ethernets, the average utilizations of the 100 Mbps links tend to be around half or a third of the rate of 10 Mbps links. LAN equipment has decreased drastically in price, and so it has become easier to satisfy people's desire for bursty transmission. It has also become less expensive to solve problems by tossing bandwidth at them instead of using scarce and expensive network manager time. There is an attempt to economize, and connections get upgraded to 100 Mbps Ethernet only when there is a need for such speeds. In general, though, other pressing network problems are clearly more important than maximizing utilization of network bandwidth. The general conclusion about low utilization rates of corporate networks is that they are not a sign of waste, but of the value of high quality data communications and of the complexity of running networks. However, these low rates do provide a substantial business opportunity, as will be discussed in Section 5. \section{ATM and QoS in current data networks} \hspace*{\parindent} Utilization rates and utilization patterns may explain why some technologies have flourished and others have fallen by the wayside. During the 1980s, there was a serious competition between Ethernet and Token Ring technologies for the LAN market. One of the claimed advantages of Token Ring was supposed to be its ability to carry a higher fraction of its peak capacity in routine operations. Another advantage was that Token Ring had QoS support built in. Still, Token Ring lost out. The technical and economic issues were complex, but the primary reason it lost out appears to be that it was more complicated. Its greater engineering efficiency did not save it, and it is easy to see why. In an environment like that of Fig. 3 (note that today LANs operate at about the utilization rate of that figure, although 10 years ago they probably operated at higher rates) average throughput is basically irrelevant. It is only the peak rate that matters. Similarly, it appears that ATM has failed to take off largely because it is inappropriate for most of today's networks. ATM was conceived with the idea, inspired by voice and multimedia, that traffic would consist of long-lived flows with reasonably well defined bandwidth, latency, and jitter requirements. However, that is not what we have on our networks today. Most of the traffic consists of Web page downloads that are small, and what matters is how quickly the entire page is delivered. Therefore ATM is irrelevant from users' perspective. It finds its greatest applications in core networks, where aggregate traffic flows do resemble the traffic conditions for which ATM was designed. Most QoS measures (see \cite{FergusonH} for a survey) are also of doubtful utility in the current environment. In an environment such as that of figures 2, 4, and 5, it is intuitively appealing to create a special lane for high priority traffic. In the environment of Fig. 3, though, which is much more representative of the universe of data networks today than figures 2, 4, and 5, that is much more questionable. High priority and low priority traffic do go through just about whenever they need to. Even when two demands coincide, it is just as likely that they will both be of high priority, so no prioritization scheme would help. Applications would still have to cope with occasional congestion. Bandwidth reservations are especially questionable in the environment of Fig. 3. In a recent work on the guaranteed service features being developed for the vBNS high speed network, the authors say that \cite{SongCW} \begin{quote} ... we give a relatively firm commitment of bandwidth. The word ``relatively'' suggests that we have not excluded the use of bandwidth overbooking for the benefit of statistical sharing. However, we must make sure that an equivalent throughput is not compromised. \end{quote} For traffic like that in Fig. 3, it is impractical to avoid bandwidth overbooking, as firm guarantees would involve tiny utilization rates and astronomical costs. If we allow overbooking, though, then we are basically dealing with a best-effort network for the high priority traffic, with low utilization providing an expected high quality of transmission. That, however, can be accomplished by much simpler methods, such as the Paris Metro Pricing scheme of \cite{Odlyzko1}, to be discussed later. So far I have been arguing that ATM and QoS are inappropriate for today's Internet. However, there are limits to these arguments. Both ATM and QoS might become much more relevant if the Internet changes (as will be discussed in Section 7). ATM already plays a big role in the core of the network, as the basic transport mechanism used by the backbones to carry IP traffic. The arguments in this paper do not say anything about the advantages of ATM in that context as compared to packet-over-SONET or other technologies. The focus here is on how the Internet appears to the users and system administrators at the edges of the network, and from their point of view ATM does not offer serious advantages, and does have serious shortcomings. I am also not suggesting that QoS is useless. It will be vital in the many situations where there are stringent bandwidth constraints, such as at the edges of the network, especially in the wireless arena. Further, techniques such as Weighted Fair Queueing (WFQ) and Random Early Detection (RED) can be invaluable in controlling congestion, and can be implemented inside the network without destroying the exceedingly valuable stateless nature of the Internet, and without complicating the lives of users or even the lives of users' system administrators. Such simple techniques might suffice to provide congestion controls for networks that offer uniformly high quality of service to all traffic. Some congestion responsive techniques are essential. Even the highest priority transmissions will occasionally have to compete for limited bandwidth with other transmission of similar priority. Therefore essentially all applications will have to possess some mechanism for limiting their bandwidth demands in the presence of congestion. (The few exception that do require absolute bandwidth guarantees are likely to stay with private lines or some special channels on the public network, as has historically been the case.) Since the human perceptual system is insensitive to small changes, very simple, although suboptimal, algorithms like the ones in TCP, WFQ, and RED should suffice, especially if the network is not heavily loaded. Why do I keep insisting on keeping the Internet simple? It is not just that the Internet is too complicated; it is that all the other things that rely on the Internet are too complicated! To make the applications that people care about interact effectively with most of the QoS schemes that are being proposed would be an undesirable additional burden. \section{Lessons to be learned from switched voice networks} \hspace*{\parindent} The Internet community appears to be learning precisely the wrong lessons from the switched voice networks. The ideas that data moves in flows and that precise guarantees of service quality are required appear to be inappropriate for the Internet, and are leading to development efforts that are likely to be of little use. On the other hand, there are valuable lessons that can be learned from the traditional phone networks. One of them is simplicity. The Internet is just too complicated. Not only is it expensive (as the figures in the first table in Section 2 show), but it requires many experts at the edges to keep it running. A telling sign is that only about two thirds of U.S. households that have PCs also have Internet accounts. In contrast, the phone system has developed simple user interfaces and standards that allow billions of people to use it easily. There is an unavoidable conflict between simplicity and flexibility, and the phone system is not flexible enough to survive in its present form. On the other hand, the Internet will surely have to become simpler to attract more users. Furthermore, there should be ways to do that without sacrificing much flexibility, by moving towards standards such as IPv6, providing more security inside the network, and so on. Another important lesson that the Internet can learn from the switched voice network is about economics. It is worth remembering that initially the phone was an extremely expensive technology. A hundred years ago, monthly charges for phone service in New York City amounted to half of the average monthly wage. Yet with diligent effort, phone service has become affordable for the masses. As is discussed in \cite{Odlyzko2}, some factors that went into the high utilization of the switched voice network (which was a substantial, although not the dominant contributor to the lowering of costs) are the slow and predictable rate of growth of voice traffic, factors that do not apply to the Internet. However, there are other factors that can also be taken advantage of in data networks, such as complementary traffic patterns and statistical aggregation. Fig. 7 shows the traffic patterns on the switched voice network separated into the residential and business customers. By carrying both types of calls on the same network, over 40\% of capacity is saved. (See \cite{Odlyzko4} for a more detailed discussion.) Yet in data networks, we have corporate private line networks that are used mostly just during the business day. We also have ISP networks used primarily by residential customers, who use them largely in the evenings and on weekends. (See \cite{Odlyzko2, Odlyzko4} for graphs of usage patterns.) Those networks are disjoint. If they were combined, they could provide a uniformly high quality of service to all current traffic at all times and do so with lower total capacity than the separated networks have now. Aggregation of traffic is another great opportunity for the Internet. If data networks were as congested as is widely believed (supposedly with 70\% peak hour utilizations on most circuits, cf. \cite{Odlyzko2}) there would be little that could be done. However, corporate users purchase lines for their burst capacity, so utilizations are low, and the situation is vastly different. A dozen lines like the one in Fig. 3 can be aggregated onto a single T1 with all dozen users obtaining essentially the same service as with their own lines, since their traffic peaks are uncorrelated. The line in Fig. 3 has lower utilization than the average for private line networks, but experiments with combining traffic traces for different lines show that statistical aggregation has great promise for producing higher utilization. It appears that by aggregating traffic from different private lines onto larger common links, one should be able to produce average utilizations at least two or three times as high as the 3-5\% range that is typical of corporate networks. (Note that this is just about the operating range of current Internet backbones, which do appear to provide high quality services.) Furthermore, this would still be corporate traffic only, which is significant only during the regular business day. By mixing in the complementary household traffic patterns, it might be possible to raise utilization levels by factors of four compared to the current levels, and provide uniformly high quality service for all traffic. A large common network would also serve to reduce the enormous administrative costs of running separate corporate networks with point-to-point connections. It might also make it easier to provide some form of the dynamic routing that has been so important in reducing costs and increasing utilization rates in switched networks \cite{Ash}. In general, the cost advantages of a single common network have been demonstrated overwhelmingly with electric and other utilities, but are still waiting to be realized in data networks. The costs advantages of a large common data network are already being partially realized by the Internet backbones. A large part of the reason for the huge differential in transport costs shown in the first table in Section 2 is precisely because the public Internet aggregates many sources of traffic, both business and residential, does operate large links, and centralizes many network management functions. However, to be a convincing substitute for private line networks, the public Internet will have to provide higher quality and security. Quality on many backbones already appears to be sufficient, and VPN (virtual private network) technology exists to assure necessary security. \section{Differential service, usage sensitive charging, and Paris Metro Pricing} \hspace*{\parindent} The main objection to QoS is that it would complicate the Internet. As a simple example, if traffic priority were to be set according to application, then either encryption technologies such as IPSec, which conceal packet payloads, would have to be banned, or else an elaborate additional signaling scheme would be required. The ideal solution is to keep the Internet as close to a dumb network as possible, one that just accepts packets and delivers then to the destination. As in the switched voice network, this might require building in more intelligence inside the network (to deal with security issues, for example), but it should be intelligence that is invisible to the end users. Although the ideal of a simple network is very attractive, there are advantages to more complicated systems. In particular, several levels of service quality would lead to more efficient use of network capacity. Today, all corporate traffic, including delay-insensitive file transfers, receive the same high quality service, and in addition, networks are idle most of the time. On the other hand, on the public Internet, there are choke points that lead to frustration with the ``World Wide Wait,'' yet there is no way for users to obtain better service. A network offering different levels of service to different types of traffic therefore has attractions. All the standard economic arguments argue in favor of such a solution \cite{MacKieM,McKnightB,Varian}. Charges do not have to be high to have a noticeable effect on human behavior. Still, the question is whether the gains are worth the cost. Any universal differential service scheme will almost inevitably involve a usage sensitive pricing system. So far such pricing has been applied only in limited cases, most notably in countries such as Chile, New Zealand, and Australia, where communication costs to the U.S. (which is where most of their international and often of their total Internet traffic comes from) are very high. However, with traffic increasing and transmission costs still growing, charging per byte is spreading. Even corporate IT managers are implementing it, in order to allocate costs to divisions. So far, though, all such charges are simply for each byte sent or received (although sometimes it is just for bytes received from international links). This has obvious attractions all by itself, as it would promote fairness. As it is, flat rate pricing means that corporations that use their Internet links lightly pay the same as ISPs and universities that transmit at high fractions of the capacity of their connections. This is already leading to modifications of the standard flat rate approach, with ISPs often facing higher charges than other users. Even with simple per-byte charging, there are problems caused by the lack of a reliable measurement infrastructure. (As a simple example, in the two years' of data I have received for the link of Fig. 3, about 15\% of the values are missing, and there are some obviously erroneous entries, such as some indicating data transfer rates of over 100 Mbps.) There are also questions of fairness, since packets that are dropped further on in their journey get counted and charged for, and this overcharge gets worse precisely when the network is congested and provides the worst service. With differential services, the accounting difficulties increase. Traffic counts would have to become more robust, and there would surely have to be a substantial infrastructure to allow either the sender or the receiver to pay. If it turns out that it is worth modifying the Internet to that extent, then, in a compromise with the overwhelming need for simplicity, I propose using the Paris Metro Pricing (PMP) scheme of \cite{Odlyzko1}. In PMP, the backbones would be divided into several logically separate channels, each with a different price per byte. Users would be free to select for each packet which channel to send it on. The expectation is that the more expensive channels would attract less traffic, and therefore would be much less congested. The details of PMP and in particular further justifications for it are contained in \cite{Odlyzko1}. The basic intuition of PMP is tohave a scheme that is as simple as possible. If there are going to be different service levels on the Internet, there will have to be differential pricing. In that case, though, why not take advantage of that pricing to deal with congestion control, and preserve the stateless nature of the Internet? PMP keeps the pricing part, which seems unavoidable, and dispenses with everything else. My expectation is that if a differentiated service system is introduced on the Internet, it will eventually evolve towards PMP (or degenerate towards it, depending on one's view). \section{The rapidly evolving Internet} \hspace*{\parindent} The preceding sections dealt with the current Internet, and explained the utilization rates and patterns that dominate on it. They suggested why ATM is not a solution to the bulk of the problems on the current Internet, %which, %statistically speaking, consists of lightly utilized corporate %links, and that most of the QoS measures are also of questionable applicability. Will this also be true on the future Internet? That will depend on prices of transmission and the nature of data traffic. {\em Prices:} In Section 2, we saw the crucial role that pricing plays in utilization patterns of data links. If prices of transmission continue to go up, as they have been doing recently, then most data networks might be used like the corporate trans-Pacific link of Fig. 4, and then differentiated services might become necessary. Predicting data transport prices is hazardous. As an example, in 1993 Irvin \cite{Irvin} published a study of private line prices in the U.S.. He constructed two plausible models that fit the historical record well up to that point, and used them to predict a continuation of the declining trend in prices. Unfortunately, that happened to be just the time when prices hit their absolute minimum. Since 1992, prices have increased over 50\%, and in 1999 were three times as high as Irvin's models predicted. (See \cite{CoffmanO} for a graph of historical private line prices.) However, as is discussed in \cite{FishburnO,Odlyzko4}, we are entering a new era, with new technologies and new competitors. Until recently the Internet was so small, that even its 100\% per year growth rate did not affect the much smaller growth rate of the underlying telecommunications network. Very soon, though, that growth rate for the Internet will mean a similar growth rate for the entire network, which is likely to lead to rapid introduction of new equipment and a rapid decline in prices. (It is also likely to lead to an increase in total revenues from data transport, in analogy to what has been happening in microprocessors, hard disks, and other high tech areas.) %If prices decreaes rapidly enough, then lightly %utilized networks that provide uniformly high quality service %for all services might become might not just feasible, but %economically optimal. The paper \cite{FishburnO} presents some simple economic models which demonstrate that when prices fall rapidly, a lightly loaded network with uniformly high quality of service can often be economically optimal. Unfortunately, it is impossible to predict how soon transmission prices will start declining. Even when they do, it is not clear how fast they will do so. {\em Nature of Internet traffic:} ATM and many QoS schemes were dismissed in an earlier section as irrelevant for today's Internet on the grounds that current traffic does not consist of extended flows with well-defined rate requirements. However, the Internet can change extremely rapidly. After all, hardly anyone had heard of the Web half a dozen years ago, and yet it is now the dominant application on the Internet. What if multimedia traffic begins to dominate? Under those conditions ATM might be the right answer, but this appears an unlikely scenario. St Arnaud \cite{StArnaud,StArnaudCFM} has already argued convincingly that multimedia is not the future of the Internet, and instead computer-to-computer communication will dominate. To his arguments I would add another one, namely that there are limits on how much multimedia material people will want to be consume. On the other hand, as long as computers keep growing in numbers and power, their potential communication demands will grow, and are likely to fill available bandwidth. However, my views differ from those of St Arnaud in an important respect. He expects that the dominant computer-to-computer traffic will be insensitive to delay and jitter. My prediction is that while that is true in principle, it will not be so in practice. I expect that future computer-to-computer traffic will in some important respects be similar to today's Web surfing. It is likely to be overwhelmingly generated in response to human demands. An example might be a surgeon sending data from the operating room to a specialist for a consultation, who in turn sends out software agents to scour databases for similar data. Another example might be a salesman trying to generate a quote for a prospective customer, and kicking off a flurry of communications between the ERP (enterprise resource planning) systems of his company and its suppliers. All such communications will have the same feature that we see on the Web today, features that corporations spend much for, namely delivering results as quickly as possible. Networks engineered to provide that level of service should be able to provide low latency and jitter as automatic byproducts with only a few simple mechanisms such as Fair Queueing that are completely invisible to the users. \section{Conclusions} \hspace*{\parindent} The current state of the entire Internet, and the utilization patterns on it, show that ATM and QoS are of limited utility. On the other hand, there are huge inefficiencies in the Internet that can be alleviated without intrusive measures that affect users. There is enough data transport capacity to provide high quality transmission for all current traffic as well as allow for substantial growth, if only all data networks were combined into a common network. There is also overwhelming evidence of users' desire and willingness to pay for high quality services. How the Internet will evolve is likely to be determined by the trends in transmission prices and in the nature of traffic. Given the huge potential costs to the entire IT system of any modifications to the Internet, though, simplicity will surely be at a premium. If prices do decrease sufficiently rapidly compared to traffic growth, then it might be economically optimal to continue with the present system of flat rate pricing, and to provide high quality service to all packets. If prices do not decline sufficiently, then something like the ``expected usage profile'' proposal of \cite{Odlyzko4} might be appropriate. In this scheme, all traffic would still get the same high quality transmission. However, users would pay ISPs according to their past usage (based on sampling, say), with lowered rates for sending their traffic at night, say, or for making sure most of the traffic is congestion-sensitive (such as TCP). Finally, if traffic growth outpaces price declines, then some version of Paris Metro Pricing might be called for as a last resort. \paragraph{Acknowledgements:} I thank my coauthors, Kerry Coffman and Peter Fishburn, the many people who have already been acknowledged in previous papers \cite{CoffmanO,FishburnO,Odlyzko2,Odlyzko3,Odlyzko4} on which this one is based, as well as David Charlton, Jon Crowcroft, Tim Dorcey, Vince Fedele, Bob Frankston, Jim Gray, Eric Grosse, Fotios Harmantzis, Alan Kotok, Jacek Kowalski, Michael Lesk, Chris Ramming, Frank Schmidt, Bill St Arnaud, Ed Vielmetti, Ivan Vukovic, Damon Wischik, and Ed Zajac for their comments. \clearpage \begin{thebibliography}{MM} \bibitem[Ash]{Ash} G. R. Ash, {\em Dynamic Routing in Telecommunications Networks,} McGraw Hill, 1998. \bibitem[Cavanagh]{Cavanagh} J. P. Cavanagh, {\em Frame Relay Applications: Business and Technical Case Studies,} Morgan Kaufman, 1998. \bibitem[CoffmanO]{CoffmanO} K. G. Coffman and A. M. Odlyzko, The size and growth rate of the Internet. {\em First Monday,} Oct. 1998, $\langle$http://firstmonday.org/$\rangle$. Also available at $\langle$http://www.research.att.com/$\sim$amo$\rangle$. \bibitem[DOC]{DOC} U.S. Department of Commerce, {\em The Emerging Digital Economy,} April 1998. Available from $\langle$http://www.ecommerce.gov/emerging.htm\#$\rangle$. \bibitem[FCC]{FCC} U.S. Federal Communications Commission, {\em Trends in Telephone Service,} Sept. 1999. Available from $\langle$http://www.fcc.gov/ccb/stats$\rangle$. \bibitem[FeldmannGWK]{FeldmannGWK} A. Feldmann, A. C. Gilbert, W. Willinger, and T. G. Kurtz, The changing nature of network traffic: Scaling phenomena, {\em Computer Communication Review,} 28, no. 2 (April 1998), pp. 5--29. Available at $\langle$http://www.research.att.com/$\sim$agilbert$\rangle$. \bibitem[FergusonH]{FergusonH} P. Ferguson and G. Huston, {\em Quality of Service: Delivering QoS on the Internet and in Corporate Networks,} Wiley, 1998. \bibitem[FishburnO]{FishburnO} P. C. Fishburn and A. M. Odlyzko, Dynamic behavior of differential pricing and Quality of Service options for the Internet, in {\em Proc. First Intern. Conf. on Information and Computation Economies (ICE-98)}, ACM Press, 1998. To appear. Available at $\langle$http://www.research.att.com/$\sim$amo$\rangle$. \bibitem[Gibbs]{Gibbs} W. W. Gibbs, Transportation's perennial problems, {\em Scientific American,} 277, no. 4 (Oct. 1997), pp. 54-57. Available at $\langle$http://www.sciam.com/1097issue/1097gibbs.html$\rangle$. \bibitem[Huitema]{Huitema} C. Huitema, The required steps towards high quality Internet services, unpublished Bellcore report, 1997. \bibitem[Irvin]{Irvin} D. R. Irvin, Modeling the cost of data communication for multi-node computer networks operating in the United States, {\em IBM J. Res. Develop.} 37 (1993), pp. 537-546. \bibitem[Leida]{Leida} B. Leida, A cost model of Internet service providers: Implications for Internet telephony and yield management, M.S. thesis, department of Electr. Eng. and Comp. Sci. and Technology and Policy Program, MIT, 1998. Available at $\langle$http://www.nmis.org/AboutNMIS/Team/BrettL/contents.html$\rangle$. \bibitem[LelandTWW]{LelandTWW} W. E. Leland, M. S. Taqqu, W. Willinger, and D. V. Wilson, On the self-similar nature of Ethernet traffic (extended version), {\em IEEE/ACM Trans. Networking} 2 (1994), 1-15. \bibitem[MacKieM]{MacKieM} J. MacKie-Mason, Telecom Information Resources on the Internet, Web site with links to online sources, $\langle$http://china.si.umich.edu/telecom/telecom-info.html$\rangle$. \bibitem[McKnightB]{McKnightB} L. W. McKnight and J. P. Bailey, eds., {\em Internet Economics,} MIT Press, 1997. Preliminary version of many papers available in {\em J. Electronic Publishing,} special issue on Internet economics, $\langle$http://www.press.umich.edu/jep/$\rangle$. \bibitem[MRTG]{MRTG} The Multi Router Traffic Grapher of Tobias Oetiker and Dave Rand, information and links to sites using it at $\langle$http://ee-staff.ethz.ch/$\sim$oetiker/webtools/mrtg/mrtg.html$\rangle$. \bibitem[Odlyzko1]{Odlyzko1} A. M. Odlyzko, Paris Metro Pricing for the Internet, in {\em Proc. ACM Conference on Electronic Commerce (EC-99),} ACM, 1999, pp. 140-147. Based on a 1997 unpublished manuscript, A modest proposal for preventing Internet congestion. Both available at $\langle$http://www.research.att.com/$\sim$amo$\rangle$. \bibitem[Odlyzko2]{Odlyzko2} A. M. Odlyzko, Data networks are lightly utilized, and will stay that way. Available at $\langle$http://www.research.att.com/$\sim$amo$\rangle$. \bibitem[Odlyzko3]{Odlyzko3} A. M. Odlyzko, Smart and stupid networks: Why the Internet is like Microsoft. {\em ACM netWorke} 2(5) (Dec. 1998), 38-46. Available at $\langle$http://www.research.att.com/$\sim$amo$\rangle$. \bibitem[Odlyzko4]{Odlyzko4} A. M. Odlyzko, The economics of the Internet: Utility, utilization, pricing, and Quality of Service. Available at $\langle$http://www.research.att.com/$\sim$amo$\rangle$. \bibitem[Paxson]{Paxson} V. Paxson, Measurements and Dynamics of End-to-End Internet Dynamics, Ph.D. thesis, Computer Science Division, Univ. Calif. Berkeley, April 1997. Available at $\langle$ftp://ftp.ee.lbl.gov/papers/vp-thesis/$\rangle$. \bibitem[SchaferV]{SchaferV} A. Schafer and D. Victor, The past and future of global moobility, {\em Scientific American}, vol. 277, no. 4 (Oct. 1997), pp. 58-61. Available at $\langle$http://www.sciam.com/1097issue/1097schafer.html$\rangle$. \bibitem[SongCW]{SongCW} C. Song, L. Cunningham, and R. Wilder, Quality of Service development in the vBNS, {\em IEEE Communications Magazine} 36 (May 1998). Available at $\langle$http://www.vbns.net/presentations/papers/$\rangle$. \bibitem[StArnaud]{StArnaud} B. St Arnaud, The future of the Internet is NOT multimedia, {\em Network World}, Nov. 1997. Available at $\langle$http://tweetie.canarie.ca/$\sim$bstarn/publications.html$\rangle$. \bibitem[StArnaudCFM]{StArnaudCFM} B. St Arnaud, J. Coulter, J. Fitchett, and S. Mokbel, Architectural and engineering issues for building an optical Internet. Short version in {\em Proc. Soc. Optical Engineering,} (1998). Full version available at $\langle$http://wwww.canet2.net$\rangle$. \bibitem[Steinberg]{Steinberg} S. G. Steinberg, Netheads vs. Bellheads, {\em Wired,} 4, no. 10 (Oct. 1996), pp. 144-147, 206-213. Available at $\langle$http://wwww.wired.com/wired/4.10/features/atm.html$\rangle$. \bibitem[Varian]{Varian} H. R. Varian, The economics of the Internet, information goods, intellectual property and related issues, reference Web pages with links, $\langle$http://www.sims.berkeley.edu/resources/infoecon/$\rangle$. \bibitem[WiltziusBD]{WiltziusBD} D. Wiltzius, L. Berc, and S. Devadhar, BAGNet: Experiences with an ATM metropolitan-area network, {\em ConneXions,} vol. 10, no. 3, (March 1996). Available at $\langle$http://www-itg.lbl.gov/BAGNet.html$\rangle$. \end{thebibliography} \clearpage \begin{figure} \centerline{\psfig{file=phone,width=5in,height=3.5in}} \caption{The analogy between the Internet and the switched voice network is weak and often misleading.} \end{figure} \begin{figure} \centerline{\psfig{file=Pzoc5,width=5in,height=3.5in}} \caption{Traffic on an ISP's T1 line on Tuesdays of April 14 and 21, 1998. 5-minute averages.} \end{figure} \clearpage \begin{figure} \centerline{\psfig{file=Pzoc4,width=5in,height=3.5in}} \caption{Traffic on a corporate T1 line in the continental U.S. during Thursday, May 28, 1998. 5-minute averages.} \end{figure} \begin{figure} \centerline{\psfig{file=Pnortel1,width=5in,height=3.5in}} \caption{Traffic from the U.S. to the Far East on a corporate 128 Kbps line during a weekday. The peak traffic hours between 1800 and 2400 coincide with the busy hours in Far East location. Hourly averages.} \end{figure} \clearpage \begin{figure} \centerline{\psfig{file=Pswitch2,width=5in,height=3.5in}} \caption{Traffic on the 8 Mbps link from the U.S. to SWITCH, the Swiss academic and research network, during Tuesdays of February 3, 10, and 17, 1998. Hourly averages, Swiss time. By permission of SWITCH.} \end{figure} \begin{figure} \centerline{\psfig{file=Pwat3,width=5in,height=3.5in}} \caption{Traffic on the link from the public Internet to the University of Waterloo. The line with circles shows average traffic during the month of heaviest traffic in each school term. The step function is the full capacity of the link. By permission of University of Waterloo.} \end{figure} \clearpage \begin{figure} \centerline{\psfig{file=Patt5,width=5in,height=3.5in}} \caption{Residential (thin line) and business (line with circles) voice traffic on U.S. long distance switched voice networks, as percentage of peak traffic on those networks.} \end{figure} \end{document}