Appendix E of the 20162017
USGA Handicap System presents estimates of the probability of a player of achieving
various negative Net Differentials or zero or better.[1] These estimates differ substantially from the
estimates the USGA has presented in the past.
Specifically, players no longer have an equal chance of beating their
Index, and the chances for having a large negative Net Differential are
substantially increased. This post
examines possible causes for the discrepancies between the two sets of estimates.
The Probability of Beating Your Index The 20122015 Edition of the USGA Handicap System estimated the probability of beating one’s Index to be .20 for all Index ranges as shown in Table 1. In the 20162017 Edition, that probability varies with a player’s Index (see Table 1). The lowIndex player now has a 64 percent better chance of beating his Index than his highIndex competitor. What causes the highIndex player to perform so poorly? Is he confused by too many tips from Golf Digest? Can he not hit the pressure shots essential for low scores due to the low selfesteem of habitually being placed in the last flight?
Table 1
Probability of Beating Your Index[2]
Method

Index
(Average Differential)


+2.0
(0)

3.0
(5)

7.0
(10)

17.0
(20)

27.0
(30)


20122015 Ed.

.20

.20

.20

.20

.20

20162018 Ed.

.21

.23

.23

.19

.14

Normal Dist.

.22

.20

.19

.17

.15

No, psychological or physical traits do not explain the
phenomenon. The poor performance of the
highIndex players is most likely the direct result of USGA handicap policy,
specifically the Bonus for Excellence (BFE).
To see how the
probability of beating one’s Index is affected
by the BFE, assume a player’s differential follows a normal distribution with a
mean of X and a standard deviation of σ.[3] It can be shown (see Appendix A) that the
average of a player’s better half of differentials is X  .8·σ. A player’s Index is then:
1) Index = BFE·(X.8·σ)
Where,
BFE =
Bonus for Excellence
A player’s scoring differential is
2) Player’s
Scoring Differential = X  S·σ
Where,
S = Number of standard deviation
from the mean.
For a player to beat or equal his Index:
3) XS·σ ≤ BFE·(X
 .8·σ)
Solving for S:
4) S ≥ (1 BFE)
·X/σ + BFE .8
If the BFE was 1.0, all players would have the same chance
of beating their Index. They would have
to have a scoring differential equal or better than .8·σ below their mean scores. From the Normal Distribution Table, the
probability of this occurrence is .21.
Substantially the same probability the USGA has maintained in previous
years.
The BFE is not 1.0, however, but .96. For the purposes of illustration, it is assumed
that a player’s σ is a linear function of his average differential, X:[4]
5) σ = 3 + 0.05 ·X
A player with an average differential of 0.0 would have σ of
3.0. A player with an average
differential of 30.0 would have σ = 4.5.
Substituting equation 5) into equation 4), the new equation
for S becomes:
6) S ≥ (1 
.96) ·X/(3+(0.05·X)) +.96·.8
For a player with an average differential of 30, S equals
1.04. From the Standard Normal Table,
the probability of such a differential or better is .15. The probabilities for a range of average
differentials are presented in Table 1. The
estimates from the Normal Distribution analysis mirror the empirical findings
in the 20162017 Appendix E. Both estimates reveal the highIndex player
has less of a chance of beating his Index than the lowIndex player. The
Normal Distribution analysis identifies the BFE as the likely culprit. There is no theoretical or empirical evidence
suggesting the BFE promotes equity.[5]
It was born from political compromise
and not statistical analysis.
Why Do the Estimates Differ by So Much? There is no apparent correlation between the two sets of probabilities. There are major differences, however, between the estimates. For example, in the 20122015 Edition the frequency of a player with a 20.0 Index having a Net Differential of 10 or better was 1 in 37,000. In the 20162017 Edition, the frequency is now estimated at 1 in 1,950. If one of the estimates is correct, the other is off by a factor of 19.
The best explanation is the old estimates were not accurate.[6] They appear to be based on a study by
Bogevold in 1974 (unpublished). Most of the research back in the ‘70s and ‘80s
depended upon collecting scorecards from various sources. The cost of collecting and processing
scorecards necessarily limited the sample size.
It is doubtful Bogevold had a sufficient sample to estimate the
probability of large negative Net Differentials with any accuracy. Moreover, these scores did not reflect the
Slope System which was not in existence at the time and were reported using a
different stroke adjustment procedure than used today. All of these factors make the probability
estimates suspect. There is no evidence the USGA ever tried to
replicate the results with another study so the probability estimates were
always of questionable validity.
While the old estimates may be in error, the 20162017 Edition estimates also have
problems:
Measurement
Errors The USGA Handicap System
only estimates a player’s true Index. If
the USGA used an unbiased estimate of a player’s Index there would not be a
significant problem. If the USGA used a
player’s Current Index, however, the probability estimates for negative Net
Differentials would be biased downward.
Model
Error  Appendix E assumes the probabilities are a function of a player’s
Index. For the most part, a player’s
probability of achieving negative Net Differentials is determined by his
standard deviation and not his Index. It
is quite possible a wild 15.0 Index has a better chance of having a 12 Net
Differential than a steady 30.0 Index player.
The probabilities in Appendix E are for an average player within each
Index range. There can be a large error if
they are applied to an individual’s performance.
No
Estimate of the Error  Appendix E does not give the reader any estimate of
the accuracy of its results. Most
likely, the USGA just took 7.3 million differentials, placed them into cells,
and did the necessary long division to produce a frequency. Appendix E implies 100 percent confidence in
the estimates, which is clearly not true. The fluctuating nature of the probabilities as
a player’s Index increases cannot be explained by any theory, and should be
attributed to random error.
Sample
Size  The handicap Index Classifications do not include the same number of
players. The number of players with a plus Index is approximately 1 percent of
the population. Appendix E shows such
players have a 1 in 9,216 chance of having a Net Differential of 7 or
better. The frequency of a Net
Differential of 8 or better is “off the charts.” The frequency should not be “off the charts”
since the chart goes up to 1 in 46,328 for the 10.0 to 14.9 Index player. It is likely the sample of players with plus
Indexes was too small to capture any Net Differential of 8 or better.
Women
 It is unlikely women were included in the sample, but if they were it would
corrupt the analysis. If women were not
included, the USGA should make clear Appendix E only applies to men.
Conclusion  There are two conclusions. First, the BFE should go the way of the stymie since it does not serve its intended purpose of promoting excellence[7] and is a source of inequity within the USGA Handicap System.
Second, trying to estimate the probability of a Net
Differential by a player’s Index is a fool’s errand. There is too much variance within Index ranges
to produce meaningful results. Appendix
E should be eliminated since it serves no real purpose in the Handicap System.[8]
A better approach would be to choose a
player with an average standard deviation and estimate the probabilities for
various Net Differentials using the Standard Normal Table. This procedure produces estimates very similar
to those found in the 20162017 Appendix
E. These results should be presented
as a ruleofthumb to give players a rough idea of the likelihood of various
Net Differentials. To strive for more
precision is both misleading and a waste of USGA resources.
Appendix A
The Probability of Beating Your Index
Assuming a Normal Distribution
Assume the distribution of a player’s differentials has a
standard deviation of σ. The average of
a player’s better half of differentials is found by multiplying possible
differentials measured in standard deviations from the mean by the probability
of making that score. For example, the
probability a player has a net differential between 0 and .1 standard
deviations below his mean differential is 0.0396. The probability that one of the player’s best
scores is between 0 and .1 standard deviations below the average differential
is 0.0796 (i.e., the probability is multiplied by two since only scores below
the average are included). The player’s
average differential is found by summing all expected value for all
intervals. If the value within an
interval is approximated by the mean value within the interval, a player’s
average of his best differentials can be
estimated as shown in Table A1. The estimate is approximately .8·σ.
Table A1
Calculation of Expected Differential
(Abbreviated)
Interval

Mean Value

Probability

Expected Value

0 to .1

0.05·σ

.07960

.003980·σ

.1 to .2

0.15·σ

.07900

.011850·σ

.2 to .3

0.25·σ

.07722

.019305·σ

…

…

…

…

2.9 to 3.0

2.95·σ

.00104

.003068·σ

Total

.789681·σ

[1]
Net Differential = (Adjusted Score – Course Rating)·113/Slope Rating – Index
[2]
The groupings of players are different from that shown in Appendix E. The analysis presented here is concerned with
average differentials. The average
differential consistent with the Index is shown in the heading of Table 1. For example, an Index of 27.0 corresponds to
an average differential of approximately 30.
Therefore USGA results for a player with a 27.0 Index are compared to a
player with a 30.0 average differential.
[3]
The USGA has maintained golf scores follow the normal distribution. See Scheid, F.J., “On the normality and
independence of golf scores with various applications,” Science and Golf, E &F Spon, London, 1990, pp. 147152.
[4]
From the probabilities presented in Appendix E, it is possible to estimate the
standard deviation by Index range.
Equation 5) is a reasonable representation of the relationship between a
player’s average differential and his standard deviation.
[5] In
a rudimentary study, Dr. Francis Scheid estimated a BFE of 1.07 was the most
equitable (“You’re not getting enough strokes!, “Golf Digest, June 1971). Given
the small sample size, a BFE of 1.0 probably could not be eliminated as the
most equitable.
[6] A
possible error was the old estimates were really the odds of negative Handicap
Differentials (i.e., (Adj. Score – Course Rating) – Handicap = Handicap
Differential) and not negative Net Differentials. On a course with a Slope Rating greater than
113, however, a player has a better chance of scoring a Handicap Differential
of –N than a Net Differential of –N (N>0).
If the old estimates were for Handicap Differentials, the old
probabilities should be systematically higher than the new probabilities. They are not, so the explanation for the
differences must lie elsewhere.
[7]
See Dougharty, Laurence, “The Bonus for Excellence Ruse,” www.ongolfhandicaps.com, January 15,
2013.
[8]
Appendix E has had many problems over the years as documented in “The Unmaking
of the USGA’s Appendix E,” www.ongolfhandicaps.com,
March 2, 2016.
[9]
Newport, John Paul, “Fighting Back Against Sandbaggers,” Wall Street Journal, July 2, 2011.
The equation and/or calculations in steps 4 and 5 appear to be in error
ReplyDeleteCorrection: steps 5 and 6
ReplyDeleteYou may be right. It would be helpful if you identified the error so it could be verified.
ReplyDeleteOkay, you are right. The problem appears to be a typo. The coefficient of X in equation 5 should be .05 and not 0.15. The denominator in the first variable of equation 6 should be (3 +.05 X). Though the equations are in error, the result is not. For a player with and average differential of 30, S equals1.04. Thanks for the catch.
ReplyDeleteThanks for the reply. Strictly speaking a players index is not the average of the better half of their differentials. The highest and lowest are thrown out. My assumption however is that this has a negligible impact on the conclusion. I appreciate your analysis.
ReplyDelete