Security

(LinuxWorld) — About a month ago, the SANS Institute, in cooperation with the U.S. Federal Bureau of Investigation, released its list of "The Twenty Most Critical Internet Security Vulnerabilities (Updated) - The Experts' Consensus" for 2002.

The information provided was picked up and relayed to the public by many news sites and major newspapers across the United States and Canada. Although the SANS Institute notes, further down in the top-20 page, that this is actually two top-ten lists, even sophisticated publications such as Computerworld, which referred to the list as the "top 20" throughout its front page treatment of the story, didn't make that distinction clear to readers.

In addition, you have to dig fairly deeply into the announcement to see the top 10 Windows list is limited to a few current variants of major Windows-brand server operating systems, while the Unix list includes applications, desktops and bugs going back at least as far as 1990. More subtly, the title coupled with the silent omission of all information about the relative costs and risks represented by the listed vulnerabilities invites readers to impute a rational basis, such as cost or risk, for the rankings shown.

In my opinion, the result was misleading in that many readers and editors would have seen this as an FBI certification of the relative equality of security problems between systems running Microsoft Windows and those running Unix.

For example, here's how Business Week headlined its report:

FBI names most wanted security flaws
The bureau and a prestigious computer-security research group announce the 20 most serious security flaws affecting both Windows and Unix systems.

This, of course, is obvious nonsense, but a majority of the popular press picking up the story reported it without warning readers or viewers that there were two distinct top-ten lists or that the two lists do not cover comparable product sets.

Because I couldn't tell from the top-20 page whether the list-compilers set out to create a false impression of parity or merely stumbled into doing so, I decided to ask the people at the institute for help:

After reviewing your listings carefully I'm unable to determine:

1. the basis on which vulnerabilities were included in the list; or,
2. the basis for the rankings.

Could you please help me understand the answers to these two questions?

I have, however, concluded that your basis for classifying a specific vulnerability as pertaining to either Windows or Unix consisted of the rule: if it involves at least one product not sold by Microsoft, classify it as a Unix issue. Please help me understand what the rule was, if this is incorrect.

In addition I would have thought that a list of "internet vulnerabilities" might have included:

1. Microsoft desktop vulnerabilities; and,
2. Cisco and other router vulnerabilities.

Could you please help me understand why these were omitted?

And got a prompt reply:

From: [suppressed]
Date: Mon, 14 Oct 2002 09:53:55 EDT
Subject: Re: Top 20 lists
To: murph@winface.com

Thanks for the note, Paul.

The system we use is to empower the people who have to clean up after the attacks and the people who staff red teams, to rate the ones that they feel are most critical — the ones that lead to significant numbers of successful compromises.

The Cisco ones are planned for group two (early 2003). And the desktop ones will probably be there, too.

What is your role?

Alan

This reply seemed to indicate that the choices reflect the distribution of successful attacks, so I asked for the numbers:

Thanks for the quick response.

However, I wonder if you could be a little more precise in your answers since your comment about empowering people seems more of a restatement of the ascription to expertise in the your top-20 page than an answer to my questions. Of course, I may just be missing the obvious because I have no information on the number of successful system compromises related to each vulnerability on your list. Since you must have it, would it be possible for you to share it with me?

... and that question also got a quick response:

Paul,

Tell me more about your role in the community?

Alan

I've heard the macroLenat "Just who do you think you are ... ?" speech since high school (you're surprised, right?), so I fired back a request that he explain how my role would affect the answer and decided to explore the issue by looking at the clues in their top-20 page to see if I could determine:

1. the criteria used to classify a vulnerability as pertaining to Unix- or Windows-based systems;
2. the criteria used to select vulnerabilities for inclusion; and/or,
3. the criteria used to rank vulnerabilities.

How the lists were created

Two things that may have a bearing on these decision processes stand out from the page:

1. the two sublists are not headed "Windows Vulnerabilities" and "Unix Vulnerabilities" respectively. Instead, both refer to systems, something which makes no practical difference on the Windows side but allows weaknesses in third-party applications running on Unix to be presented as Unix System vulnerabilities.
2. only technical vulnerabilities are considered, thus exempting structural issues, such as those that drive people to re-install various Windows operating environments from an original source rather than from a fully patched back-up.

   This type of system-recovery work, rare in Unix but common in the Microsoft Windows environment, reloads previously resolved vulnerabilities and is a key reason Unix web servers around the world continue to be bombarded by new instances of both Code Red and Nimda.

The SANS document combines the two top-ten lists with recognition and remediation information about each vulnerability cited. That information uses the CVE — Common Vulnerability and Exposures — nomenclature developed by Mitre and refers to the detail maintained by the ICAT team at the National Institute of Standards and Technology.

As a result, a naive reader could reasonably assume that the CVE information forms the basis for both the rankings and the Windows/Unix distinctions.

In total, the SANS Institute's top-20 page cites 89 different ICAT listings in support of its Microsoft top-ten list and 147 unique listings with respect to the Unix top ten. Among these, 16 (17 percent) of the Microsoft citations and 75 (51 percent) of the Unix citations have 1999 prefixes, while two Microsoft citations and eight Unix citations are used to document more than one vulnerability.

To see how the detailed CVE (or CAN, for "Candidate CVE(?)") listings illuminate the top-ten lists, consider just the two highest ranked vulnerabilities for each environment:

From the Windows list:

1. W1 Internet Information Services (IIS)
2. W2 Microsoft Data Access Components (MDAC) — Remote Data Services

... and from the Unix list:

1. U1 Remote Procedure Calls (RPC)
2. U2 Apache Web Server

Notice that these seem quite parallel; to a non-technical reader it would appear that the order is reversed for the two environments, but both have roughly comparable Web-server and remote-access vulnerabilities. Unfortunately, this implicit conclusion is completely wrong.

The SANS top-20 page cites only 7 of the 17 current (year 2002) CVEs listed for IIS in the ICAT database in support of its decision to award this product set top billing on that list.

On the Unix side, however, SANS cites nine-year 2002 ICAT vulnerabilities against Apache. There are currently 21 listed, of which eight were added on or after Sept. 5, 2002. Of the four missing ones, two of them (2002-661 and 654) have the words "Apache 2.0 through 2.0.39 on Windows, OS2 and Netware" in the first sentence, but I could see nothing to suggest why the other two were omitted.

The nine are:

CAN/CVE	Nature of issue?	Applies to?
CAN-2002-0061	Apache for Win32 before 1.3.24, and 2.0.x before 2.0.34-beta	Windows Only
CVE-2002-0082	mod_ssl pre 2.8.7-1.3.23, and Apache-SSL before 1.3.22+1.46	Cross Platform
CAN-2002-0257	Cross-site scripting vulnerability in auction.pl	Third party Application
CAN-2002-0392	Apache 1.3 through 1.3.24, and Apache 2.0 through 2.0.36	Windows Only
CAN-2002-0513	PHP administration script in popper_mod 1.2.1	Third party Application
CAN-2002-0655	OpenSSL pre 9.7beta2	Cross Platform
CAN-2002-0656	OpenSSL pre 9.7beta2	Cross Platform
CAN-2002-0657	OpenSSL pre 9.7beta2	Cross Platform
CAN-2002-0682	Cross-site scripting vulnerability in Apache Tomcat 4.0.3	Cross Platform

Only two of these unambiguously apply to Apache running on Unix. Two apply uniquely to Apache on Windows. The three rather-dubiously applicable OpenSSL CVEs are repeated in support of the SANS decision to list OpenSSL on Unix as the No. 3 Unix vulnerability, and two represent problems with third-party (and cross platform) applications that have little to do with either Unix or Apache.

'Unix' includes all Unix apps

I don't see anything here that remotely justifies considering Apache the No. 2 source of Unix vulnerability, but once I looked at a few other rankings the general pattern seemed clear. The vulnerabilities cited against Microsoft products generally underrepresent the ITAC database — sometimes by considerable margins — while those cited in support of Unix vulnerabilities often seem to have little or nothing to do with Unix. Instead, problems affecting applications, problems that have long since been remediated, problems affecting network gear and problems affecting cross-platform libraries, are all cited in support of the Unix system-vulnerability list.

The No. 4 Unix vulnerability, for example, is given as SNMP — something about which the SANS Institute's top-20 page says:

SNMP is not unique to Unix; it is extensively used on Windows, in networking equipment, printers and embedded devices. But the majority of SNMP-related attacks seen thus far have occurred on Unix systems with poor SNMP configurations.

The first statement above is certainly true and supported by the ICAT data, but the other is neither supported by the evidence provided nor pertinent to the question. At first glance it looks as if the writer is trying to explain why SNMP problems are classified as Unix-system problems, but a second look will show that it fails to give the kind of factual information needed. Instead, a matter of opinion is supported by a statement of opinion and the real question — "how important are SNMP attacks relative to other attacks on Unix?" — is silently elided.

I cannot tell from the document whether this kind of wording reflects sharp practice or just fuzzy thinking, but the same kind of thing occurs in other places. For example, the word "system" is used in the list titles; something which I'm guessing was done as cover in case anyone questioned counting Unix application vulnerabilities as Unix vulnerabilities.

Remote thought

Consider the word "remote" in both the second-ranked Microsoft systems vulnerability ("Microsoft Data Access Components (MDAC) — Remote Data Services") and in the label used for the No. 1 Unix vulnerability ("Remote Procedure Calls (RPC).") I think that this repeated use of the key word "remote" makes the two sound fairly parallel, but they're not.

In the Unix world, "RPC" has a reasonably precise and widely understood meaning that has been consistent over time. Thus a 1994 RPC vulnerability in Ultrix 4.3A (CVE-1999-0170) can be cited to support the idea that RPCs represent the No. 1 source of Unix vulnerabilities.

In contrast, Microsoft's tendency to rename things between product announcements means that "Microsoft Data Access Components (MDAC) — Remote Data Services" has no real meaning beyond its reference to a specific and short-lived product release. Here's the SANS description for what they see as the second-highest ranked Microsoft Internet vulnerability:

The Remote Data Services (RDS) component in older versions of Microsoft Data Access Components (MDAC) has a program flaw ...affecting... Most Microsoft Windows NT 4.0 systems running IIS 3.0 or 4.0, Remote Data Services 1.5, or Visual Studio 6.0.

Request for comment

I asked Mr. Paller to review the draft for this article but received no reply. However, when the editor for LinuxWorld asked for comment, he got this response. We love criticism; it forms the critical building blocks for excellence. I'll let the folks who built the Top 20 read the material and where they see things in it that would make the list better, they will undoubtedly use the ideas — with full credit. The part that is most useful is the detailed assessment of the CVEs. We allowed the contributors to add CVEs they found that were relevant and now we are (while vetting the scanning tools that claim to test for the top 20) systematically generating an improved list. So his work is particularly helpful right now Thanks for sharing it with me. By the way, the question about role in the community was to make a choice how to connect him into the teams of people working on the project. Alan I've never seen it done better: not only is this graceful and even eloquent, but it still manages to put me firmly in my place. Now if he would only answer the question...

SANS cites one CVE, from 1999, in support of this artifact's claim to second place. The ITAC database, in contrast, has a lot of current CVEs relating to Microsoft's own RPC implementations for NT, Windows 2000 and XP, in addition to RPC-like products and APIs (including 45 just for ActiveX).

In sharp contrast to the lack of text supporting the status accorded MDAC, the top-20 page lists 26 CVEs (65 percent of them from 1999) in support of the idea that RPCs represent the No. 1 Unix vulnerability.

It's not obvious that RPC represents the No. 1 Unix vulnerability. If you check the targets list at incidents.org, RPC attacks are No. 6 on the list for the year (dropping to No. 7 on the front page current stats), well behind attacks on other common functions such as FTP and SMTP.

The more of these RPC related CVEs I read, the odder this choice for first place seemed. Then I noticed that almost all of the cited material referred explicitly to Sun — not least by naming the protocols "SunRPC" — while the SANS Institute top-20 page uses "RPC". The only mention of Sun outside a URL context contains a logical error suggesting a bit of last-minute dictation slipping past an editor:

In addition to this inherent transmission insecurity, critical flaws have been found in many versions of FTP server software, both those provided by operating system vendors (Sun, HP-UX, etc) and those developed by the open source community (WU-FTPD, ProFTPD, etc)

As a result, I started to think that having a finger pointing at Sun but not Microsoft might have been a primary selection criterion that someone later attempted to disguise by sanitizing the top-20 page to remove direct references to Sun that occur in the source documents. If so, you have to ask why anyone would bother or, more cynically, what they might have wanted to hide.

Consider CVE CAN-2002-0391. It is cited by SANS as showing the Unix vulnerability to RPC exploits and refers directly to Sun as the source of the offending SunRPC code, but it doesn't say that the vulnerability is peculiar to Unix:

Integer overflow in xdr_array function in RPC servers for operating systems that use libc, glibc, or other code based on SunRPC, allows remote attackers to execute arbitrary code by passing a large number of arguments to xdr_array through RPC services such as rpc.cmsd and dmispd.

There is more detail in the matching CERT advisory. This makes it clear that SunRPC is found in a lot of operating systems:

Because SunRPC-derived XDR libraries are used by a variety of vendors in a variety of applications, this defect may lead to a number of differing security problems. Exploiting this vulnerability will lead to denial of service, execution of arbitrary code, or the disclosure of sensitive information.

In fact, Netmanage wrote an RPC/XDR dynamic link library for Windows 3.0, and RPC capabilities have been common in Microsoft operating environments released since. But this CVE (CAN-2002-1141), illustrating CERT's "variety of vendors," isn't mentioned in the SANS document. It says:

An input validation error in the Sun Microsystems RPC library Services for Unix 3.0 Interix SD, as implemented on Microsoft Windows NT4, 2000, and XP, allows remote attackers to cause a denial of service via malformed fragmented RPC client packets, aka "Denial of service by sending an invalid RPC request."

There may or may not be a pattern here, in which CVEs that point at Sun are selected while those pointing at Microsoft are skipped. But even if this pattern exists, it would not explain why these particular vulnerabilities were selected or how the ranking was done.

In an effort to understand the latter, I considered — and rejected — seven alternative hypotheses:

1. Could the criteria have been technical interest?

The ICAT group at NIST maintains a top-ten list ranked according to "the number of requests made for a particular vulnerability in ICAT." Six of the ICAT top ten for 2002 relate to issues affecting Microsoft products. Two refer to OpenSSL, one refers to Apache and one refers to CDE.

Clearly, one can conclude that technical interest had little or nothing to do with the SANS ranking.

2. Did someone construct a Windows list and then decide to create a comparable Unix one?

The Windows list seems limited to current, major-market server-OS variants, while CDE desktops are featured front-and-center on the Unix side.

3. How about currency? Maybe these are just the more recently discovered vulnerabilities?

The Windows list is fairly current with the exception of item two, which is technically limited to an NT variant marketed in 1999. On the other hand, the Unix list ranges all the way back to 1991 and seems to consider anything with a defect that could be attached to or run on a Unix machine as core Unix.

For example, the first CVE listed on the top-20 page in support of the RPC ranking is CVE-1999-0166. This pertains to a CERT advisory from 1991 reporting an already-patched problem with SunOS 4.1 and 4.0.3. The next two refer to the same issue on SunOS, and No. 4 reflects a closely related 1994 issue with Ultrix.

In fact, most of the issues listed for Unix systems were resolved years ago. No. 10 has one CVE, a candidate from 1999, that isn't found by searching the online database. Meanwhile, all five of the CVEs supporting number 6 (trust services) are from 1999.

Therefore, it's clear that neither actual currency nor concurrency with the Microsoft list governed the Unix selections or rankings.

4. How about demonstrated risk?

Of five 2002 CANs listed for SunRPC, four have the words "Currently we are not aware of any exploits for this issue" attached to them on the security focus exploits database. One of these four has a rumored but unknown exploit, and the fifth has a sketch for a possible exploit. I could find no record of a single successful attack based on any of these.

Clearly, demonstrated risk had little or nothing to do with the rankings.

5. Damage or remediation cost?

Here are the two lists in full with the original labels:

Top Vulnerabilities to Windows Systems

1. W1 Internet Information Services (IIS)
2. W2 Microsoft Data Access Components (MDAC) — Remote Data Services
3. W3 Microsoft SQL Server
4. W4 NETBIOS — Unprotected Windows Networking Shares
5. W5 Anonymous Logon — Null Sessions
6. W6 LAN Manager Authentication — Weak LM Hashing
7. W7 General Windows Authentication — Accounts with No Passwords or Weak Passwords
8. W8 Internet Explorer
9. W9 Remote Registry Access
10. W10 Windows Scripting Host

Top Vulnerabilities to Unix Systems

1. U1 Remote Procedure Calls (RPC)
2. U2 Apache Web Server
3. U3 Secure Shell (SSH)
4. U4 Simple Network Management Protocol (SNMP)
5. U5 File Transfer Protocol (FTP)
6. U6 R-Services — Trust Relationships
7. U7 Line Printer Daemon (LPD)
8. U8 Sendmail
9. U9 BIND/DNS
10. U10 General Unix Authentication — Accounts with No Passwords or Weak Passwords

A cost or risk basis is what most people would expect: a 20 worst list should be ordered according to damage or cost. Unfortunately, even casual inspection suggests that this doesn't apply here.

We don't have actual numbers for any of these things, but we know that problems with sendmail, especially open relays, have cost far more to recognize and remediate than problems with secure shell, Apache or R-services.

6. Maybe it's something simple, like the number of CVEs?

If you search ICAT according to the top three Unix vulnerabilities, you start out with a nice declining series: 68, 67 and 49 hits respectively. But just as you get that "a-ha" feeling, No. 4 gets 58 hits and blows that explanation away. Furthermore, neither list matches the incident reports at incident.org.

7. Perhaps something sophisticated like damage done to third parties

Again, this sounds like a logical explanation; third parties, the people who have their bandwidth reduced by things like Code Red but have no Microsoft products on site, are inarguably the "innocent victims" here. However, this list appears to equate weaknesses in IIS that affect millions of third parties to Unix RPC buffer vulnerabilities that have yet to victimize their first third party.

If it wasn't one of the above, what basis did they use first for the selection and then for the ranking? I still don't know, but I have two hypotheses:

1. It is possible that the Unix list started out as nothing more than an exercise in political correctness. After all, not all of the Internet's vulnerabilities are due to Microsoft products and processes. Under this guess, I imagine a top-ten list that gets out of control and becomes a top-40 list (most of them Windows-related) until someone decides that fairness requires a comparable Unix list and then edits both lists to produce the false impression of parity presented in the resulting "top-20" announcement.

2. The more cynical alternative is less-credible because it requires the assumption of malice or commercial interest rather than just sloppy thinking. In this interpretation, the apparent pattern is taken as real, and the whole announcement is seen as an attempt to balance the bad press Microsoft gets on its security vulnerabilities. A prestigious FBI announcement seems to be showing that Unix isn't any better.

Personally I'm more inclined to believe in stupidity than malice but, either way, this "top 20" announcement seems to me to have been an extremely misleading and quite possibly disingenuous piece of work that the Institute and the FBI should correct promptly.