Archives

By John J. Downey, Cisco Systems

In part two of our series, Cisco's John Downey looks at issues and processes that influence both upstream and downstream path performance.

In part one of this series, we examined the basics of symbol rate and bit rate (see November CT). We began explaining how downstream (DS) performance can be influenced by the generation of map messages and how upstream (US) performance is affected by the request/grant cycle that must be performed in a Data Over Cable Service Interface Specification (DOCSIS) environment.

This month, we examine other factors related to DS and US paths.

TCP or UDP?

A point that is often overlooked when testing for throughput performance is the protocol being used. Is it a connection-oriented protocol such as transmission control protocol (TCP), or connectionless such as user datagram protocol (UDP)?

UDP sends information with no regard to received quality. This process is often referred to as "best effort" delivery. If some bits were received in error, you make do and move on, which is typical for, say, real-time audio or streaming video.

TCP, on the other hand, requires an acknowledgment (ack) to prove that the sent packet was received correctly. If the network is maintained well, the protocol may be dynamic enough to send more packets consecutively before an ack is requested. This is referred to as "increasing the window size," which is a standard part of the transmission control protocol.

The point is that DS traffic will generate US traffic in the form of more acks. Also, if a brief interruption of the upstream results in a TCP ack being dropped, the TCP flow will slow; this would not happen with UDP.

If your reverse path is disrupted, the cable modem will eventually fail the keep-alive polling after about 45 seconds and start scanning the DS again. Both TCP and UDP will survive brief interruptions, because TCP packets will be queued or lost and DS UDP traffic will chug along.

The US throughput could limit the DS throughput as well. For instance, if the downstream traffic travels via coax or satellite and the upstream traffic travels via telephone line, the 28.8 kilobits per second (kbps) US throughput could limit the DS throughput to less than 1.5 megabits per second (Mbps) even though it may have been advertised as 10 Mbps max. This happens because the low-speed link adds latency to the ack US flow, which then causes TCP to slow the DS flow.

Interleaving effect

Electrical burst or impulse noise in the downstream path may cause errors in blocks, causing worse problems with throughput quality than errors that are spread out from thermal noise.

A technique known as interleaving, which spreads data over time, is used to minimize the effect of burst errors. By intermixing the symbols on the transmit end then reassembling them on the receive end, the errors will appear spread apart.

Forward error correction (FEC) is very effective on errors that are spread apart. A relatively long burst of interference can cause errors that may be corrected by FEC when using interleaving. Because most errors occur in bursts, interleaving is an efficient way to improve the error rate.

DOCSIS specifies five different levels of interleaving. The highest amount of interleaving is 128:1 and 8:16 is the lowest. This indicates that 128 codewords, made up of 128 symbols each, will be intermixed on a one-for-one basis, whereas the 8:16 level of interleaving indicates that 16 symbols will be kept in a row per codeword and intermixed with 16 symbols from eight other codewords.

Interleaving doesn't add overhead bits like FEC, but it does add latency, which could affect voice and real-time video. It also increases the request/grant round-trip time (RTT).

Increasing the RTT may cause you to go from every other map opportunity to every third or fourth map. That is a secondary effect, and it is that effect that can cause a decrease in peak US data throughput. Therefore, you can slightly increase the US throughput (in a packets-per-second-per-modem way) when the value is set to a number lower then the typical default of 32.

To get around the impulse noise issue, you may increase the interleaving value to 64 or 128. However, increasing this value may degrade performance (throughput) but increase noise stability in the DS. Put another way, maintain the plant properly, or else the customer will see more uncorrectable errors (lost packets) in the DS, to the point where modems start losing connectivity and/or you end up with more retransmissions.

By increasing the interleave depth to compensate for a noisy DS path, a decrease in peak cable modem US throughput must be factored in. In most residential cases, that is not an issue, but it's good to understand the trade-off. Because the plant RTT is on the order of 0.8 ms, going to a lower interleaver value of less than the typical interleave depth of 1 ms may not provide much throughput increase. Going to the maximum interleaver depth of 16 ms will have a significant, negative impact on US throughput.

For upstream robustness to noise, DOCSIS modems allow variable or no FEC. Turning off US FEC will get rid of some overhead and allow more packets to be passed, but at the expense of robustness to noise. It's also advantageous to have different amounts of FEC associated with the type of burst. Is the burst for actual data or for station maintenance? Is the data packet made up of 64 bytes or 1518 bytes? You may want more protection for larger packets.

No interleaving appears in the upstream because the transmission occurs in bursts. As well, there isn't enough latency within a burst to support interleaving. Some chip manufacturers are attempting to add this feature in the near future, which could have a huge impact considering all the impulse noise from home appliances and other sources.

Concatenation effect

DOCSIS 1.1 introduces concatenation, which allows several smaller DOCSIS frames to be combined into one larger DOCSIS frame, and be sent together with one request.

Because the number of bytes being requested has a maximum of 240 mini-slots, and there are typically 8 or 16 bytes per minislot, the maximum number of bytes that may be transferred in one upstream transmission interval is 1920 or 3840. That means although concatenation is not too useful for large packets, it is an excellent tool for short TCP "acks."

By allowing multiple packets per transmission opportunity, concatenation increases the basic packets-per-second (PPS) value by that multiple. For example, if the upstream is limited to 250 PPS because of the RTT, and concatenation allows 10 32-byte TCP acks to be grouped together, then the effective PPS just became 250 PPS multiplied by 10 equals 2500 PPS. Not bad!

Ultimately, variables that may be manipulated exist, but there is always a tradeoff. If we could change the maps/sec and interleaving values, we may get better US rates, but at the expense of DS rate or robustness.

Decreasing the map interval won't make much difference in a real network and it will increase CPU and bandwidth overhead on both the cable modem termination system (CMTS) and cable modem. We also could incorporate US FEC at the expense of more US overhead.

Throughput calculations

Let's assume we are using a CMTS card that has one downstream and six upstream ports. The one downstream port is split to feed about 12 nodes. Half of this network is shown in Figure 1. The 500 homes/node multiplied by an 80 percent cable take-rate and multiplied by a 20 percent modem take-rate equals 80 modems per node. And the 12 nodes multiplied by the 80 modems per node equals 960 modems per DS port.

Note: Many MSOs are now quantifying their systems by homes passed per node. This is the only constant in today's architectures where you may have direct broadcast satellite (DBS) subscribers buying data service or only telephony without video service.

The reverse signal from each one of those nodes will probably be combined on a two-to-one ratio so that two nodes feed 1 upstream port. Six upstream ports multiplied by two nodes per upstream equals 12 nodes. And 80 modems per node multiplied by two nodes per upstream equals 160 modems per US port.

Downstream

The DS symbol rate equals 5.057 Msymbols/sec or Mbaud. A filter roll-off (alpha) of about 18 percent gives 5.057 multiplied by (1+0.18) equals about a 6 MHz-wide "haystack" as would be seen on a spectrum analyzer.

Six bits per symbol for 64-QAM would give 5.057 multiplied by 6 equals 30.3 Mbps. After the entire FEC and Moving Picture Experts Group (MPEG) overhead is calculated, about 28 Mbps is left for payload. This payload is further reduced because it's also shared with DOCSIS signaling.

Note: ITU-J.83 Annex B indicates Reed-Solomon FEC with a 128/122 code, which means 6 symbols of overhead for every 128 symbols, hence 6/128 equals 4.7 percent. MPEG-2 is made up of 188 byte packets with 4 bytes of overhead, sometimes 5, giving 4.5/188 equals 2.4 percent.

Add another possible 4 percent of overhead for DOCSIS map traffic, which is why you'll see the speed listed for 64-QAM as 27 Mbps. Remember, Ethernet packets also have 18 bytes of overhead whether it's for a 1500-byte packet or a 48-byte packet. There are also 6 bytes of DOCSIS overhead. That could be about 1.1 percent to 2.8 percent extra overhead.

In the unlikely event that all 960 modems were downloading data at precisely the same time, they would each get only about 28 kbps! Consider a more realistic scenario that assumes a 10 percent peak usage, and we get a theoretical throughput of 280 kbps as a worst-case scenario during busy times.

If only one customer were on, the customer theoretically would get 27 Mbps, but the reverse acks that must be transmitted when doing TCP limits the forward throughput. In reality, the cable company will rate-limit this to 1 Mbps or 2 Mbps so as not to create a perception that will never be achievable when more subscribers sign up.

Upstream

The DOCSIS upstream modulation of quadrature phase shift keying (QPSK) at 2 bits/symbol would give about 2.56 Mbps. This is calculated from the symbol rate of 1.28 Msymbols/sec multiplied by 2 bits/symbol. The filter alpha is about 25 percent, giving a bandwidth of 1.28 multiplied by (1+0.25), which equals 1.6 MHz bandwidth.

Subtract about 8 percent for the FEC and another approximately 10 percent overhead for maintenance, reserved time slots for contention, and acks and we're now down to about 2.2 Mbps, which is shared amongst 160 potential customers per upstream port.

Assuming 10 percent peak usage, we have 2.2 Mbps divided by (160 * 0.1), which equals 137.5 kbps worst-case payload per subscriber. For typical residential, we probably don't need as much US throughput as DS. This speed may be sufficient for residential usage, but not for commercial deployments or when a host behind a cable modem becomes a server because they're logged into Napster or WinMX!

Keep in mind that when we refer to file size, we are usually referring to bytes made up of 8 bits, so 128 kbps would equal 16 kBps (kilobytes per second). If I try to download a 5 Mb file, it's actually 5 multiplied by 8 equals 40 Mbits and could take longer to download than anticipated. Also, 1 kB actually is equal to 1,024 bytes (not 1,000), because binary numbers are a power of two.

Conclusion

Knowing what to expect of throughput is the first step in determining why a customer's speed is what it is. You must know what theoretically is possible before you can design or manage a plant that will be changing on a daily basis. The next step is to monitor the actual traffic loading to determine what's being transported and when more components are necessary to alleviate bottlenecks.

DOCSIS now has mechanisms that assure levels of quality for certain services (that is, your VoIP call isn't going to collapse during the busy hour, it's always going to get through).

Having a technical grasp of all the issues will help keep our heads above water even though we're all still a little wet behind the ears!

John J. Downey is a broadband network engineer with Cisco Systems. He may be reached at m.

Data Throughput, Part 2

More factors that influence your DOCSIS data throughput to add to last month's list:

1. How downstream throughout may be affected by upstream throughput.

2. What effect the protocol has on it. Is it TCP or UDP?

3. How interleaving may affect data throughput.

4. How concatenation may affect data throughput.

Know what is possible before you design or manage a plant that will be dynamically changing with different requirements on a daily basis.

 Back to December 2001 Issue

Access Intelligence's CABLE GROUP
Communications Technology | CableFAX Daily | CableFAX's CableWORLD | CT's Pipeline
CableFAX Magazine | CableFAX databriefs | Broadband Leaders Retreat | CableFAX Leaders Retreat