About the Ranking Methods
How Your Rankings are Computed
Unlike other ranking systems, we do not use a fixed set of criteria and weights for computing rankings. Instead, you choose a set of criteria of interest and indicate how important each criterion is to you. We then use your weights to rank a set of programs according to your priorities.
We compute a custom score for each institution based on your weights. Your customized program score is a weighted sum of points for each item. For an item with importance weight W, a program receives W × S points for being S standard deviations above average.
Missing Values
Data on the site come from a number of outside sources. Values in these data sets are sometimes missing. There are two main reasons for missing values:
- Survey nonresponse
- Many of the data sets we are using are generated from surveys of institutions or doctoral students. When questions in the surveys are unanswered, we end up with missing data.
- Confidentiality measures
- Some data sets (e.g., the Survey of Earned Doctorates) have values suppressed to protect the confidentiality of survey responders. No values from the SED are reported unless they come from at least 5 survey responses. When reporting sets of percentages that add up to 100, additional suppression is done to prevent one from deducing a suppressed value by subtracting the sum of the remaining values from 100.
Replacements for Missing Values
In some cases we display a replacement value when values are missing. The goal is to give you an idea of the typical magnitude of the missing value. These replacement values are marked with an asterisk (*) and are shown in gray text to distinguish them from known values. Here is how the replacement values are computed:
We compute the average of all the known values for other programs in the same field. If the value in question is not part of a group of percentages that must sum to 100, we display this average in place of the missing value.
For example, suppose the total number of graduate students is not known for a chemistry program. We compute the average number of graduate students at chemistry programs with known numbers of students and display this average in place of the missing value.
- When there are one or more missing values from a group of percentages that sum to 100, we proceed as follows:
- We first compute the average value for each missing variable using known values.
- We then compute the sum of the group, including both the known values and the replacement values. We know the sum must be 100, so we rescale the replacement values to satisfy this constraint.
For example, suppose we know the following about a biology program:
Employment status at graduation Job / contract / definite commitment 40% Negotiating with specific organizations 40% Still seeking Missing value Other Missing value We first compute the average percentage of people in the "Still seeking" category over all biology programs with known values. We repeat the process for the "Other" category. Suppose that these averages are 30% and 10%, respectively.
We know that the sum of the percentages for "Job," "Negotiating," "Seeking," and "Other" must be 100. The sum of the known values is 80%, so the sum of the missing values must be 20%. Because the replacement values sum to 40%, not 20%, they must be too big. To correct this problem, we divide each replacement value by 2, which gives us replacement values of 15% and 5%, respectively. These replacement values (a) sum to the appropriate value and (b) are proportional to the averages for the field.
Missing Values in Rankings
Missing values are handled in three different ways in the rankings. You decide which method will be used when you set your priorities.
- Use averages
- When a value is missing, we use in its place a replacement value generated via the process described above.
- Skip and reweight
When a value is missing, we skip it and increase the weights you assign to other items.
Suppose, for example, that you give a weight of 5 to "placement rate at graduation," a weight of 2 to "time to degree", and a weight of 3 to "perceived faculty quality." Suppose further that the placement rate for one program is not available. We compute the score for the program with the missing value by skipping over the missing value and increasing the weights for the other items. In this case, the sum of the magnitudes of all your weights is 5 + 2 + 3 = 10, and the sum of the magnitudes of the weights for the known values is 2 + 3 = 5. We skip the item with the missing value and increase the weights for the remaining items by a factor of 10 / 5 = 2. Thus, for the program with the missing item, we use weights of 4 for "time to degree" and 6 for "perceived faculty quality."
- Do not rank
When a value is missing, we do not rank the program and instead display it in a separate, alphabetically sorted list.
About the Graduate School Guide
