(Note: This post revises a previous post based on comments by the USGA and a paper written by the USGA's Handicap Research Team.)

The problems in USGA handicap research are illustrated by two tables prepared by the USGA on the probability of a player making an exceptional score. The table presented in Appendix E of the

*USGA Handicap System*is entitled “Odds of shooting an Exceptional Tournament Score.” Part of that table is shown as Table 1 below.**Table 1**

**Appendix E - Exceptional Tournament Score Probability Table**

**[4]**Net Differential | 0-5 | 6-12 | 13-21 | 22-30 | >30 |

0 | 5 | 5 | 6 | 5 | 5 |

-1 | 10 | 10 | 10 | 8 | 7 |

-2 | 23 | 22 | 21 | 13 | 10 |

-3 | 57 | 51 | 43 | 23 | 15 |

The values in the table are the odds of shooting a net differential EQUAL OR BETTER than the number in the left column.

The definition of net differential appears to have changed over time leading to significant errors. The original research defines net differential as the difference between a player’s net score and the course rating.[5] The current definition of net differential in Appendix E is the handicap differential for a score minus the player’s Handicap Index. The two definitions are different as can be seen by an example. Assume a 17.0 index player scores a net 64 on a course with a slope rating of 150 and a course rating of 72.0. The player is a 23 handicap and records a gross score of 87. His net differential under the old system is -8.0. His net differential under the current definition is -5.7 -- (87-72) 113/150 – 17.0. Using the original definition of net differential, a player would have a 1 in 1138 chance of such a score or better. Under the current definition of net differential, a player would have about a 1 in 300 chance of such a score or better. I suspect the old definition is the correct one, and the current Appendix E underestimates the rarity of low scores for courses with a Slope Rating greater than 113.

The USGA shows some confusion about the distinction between odds and probabilities.[6] The title of appendix references “probabilities,” but the body of the appendix is supposed to represent “odds.” The USGA apparently does not know there is a difference between these two concepts. “Odds against” is the ratio of the number of times an event does not occur to the number of times it does occur. Probability is the ratio of the number of times an event occurs to the total number of trials.[7] If the odds are 6:1, a golfer will have a net differential of zero or better 1 in 7 (not 6 as shown in Appendix E) rounds. I assume, however, that the USGA meant probability and will accept the numbers in the table as that.

Looking at Table 1, an inconsistency should be obvious. If a player has an index between 13 and 21, he is 25 percent less likely to score a net differential of zero or better than a player with either a lower or higher index. This is the only case where the probability of any level of performance (e.g., -2,-3…-10 net differential) does not increase or stay the same as a player’s index is increased. Why is this index range cursed with such poor performance? I suspect any difference in probability among groups for scoring a net differential of zero or better may not be statistically significant.[8]

Exactly how these probabilities were derived is not known. The USGA does not have access to raw scores, only adjusted scores through its GHIN system. Since adjusted scores are lower or equal to actual score, their use would overestimate the likelihood of an exceptional score. If the USGA used actual scores, where did it get them, and how large was the sample? Without a description of the research methodology, little credence should be given to the table.

The table “Probability of Two Best Scores Beating Handicap” which appears on the USGA’s website presents more problems. The table is shown as Table 2 below:

**Is the Table Accurate?**– Table 2 represents the probability of scoring exactly 2 net differentials of various values. This methodology underestimates the probability of exceptional scores. The errors, however, are small in the range of interest of the USGA. It also appears that Table 2 is based on Table 1. Errors in Table 1 translate into errors in Table 2. While Table 2 may be accurate, it is misleading. For example, the table shows the probability of two net differentials of -6 is 1 in 3385. The probability of two net differentials of -6 or better is 1 in 573—i.e., Table 2 decreases the probability by restricting the scores to a narrow range. A more complete discussion of this issue is contained in the appendix.

**Probability of Equivalent Pairs Are Not Equal**– The probability of two best scores being (a,b) should be the same as the probability of (b,a). This is the case for most of the pairs in Table 2. There is an exception, however. The probability of (6,9) is 17109. The probability of (9,6) is 17189. This is clearly just a typographical error made by the USGA, but such errors seem to plague the USGA handicap publications.[9]

**Table 2**

**Probability of Two Best Scores Beating Handicap**

0 | -1 | -2 | -3 | -4 | -5 | -6 | -7 | -8 | -9 | -10 | |

0 | 27 | 46 | 92 | 199 | 408 | 869 | 1808 | 2480 | 3871 | 9180 | 85779 |

-1 | 46 | 13 | 26 | 58 | 118 | 253 | 526 | 722 | 1126 | 2672 | 24987 |

-2 | 92 | 26 | 20 | 43 | 89 | 191 | 398 | 546 | 853 | 2023 | 18907 |

-3 | 199 | 58 | 43 | 59 | 121 | 258 | 537 | 737 | 1150 | 2728 | 25492 |

-4 | 408 | 118 | 89 | 121 | 200 | 427 | 888 | 1219 | 1903 | 4512 | 42163 |

-5 | 869 | 253 | 191 | 258 | 427 | 821 | 1708 | 2343 | 3667 | 8872 | 81030 |

-6 | 1808 | 526 | 398 | 537 | 888 | 1708 | 3385 | 4644 | 7249 | 17189 | **** |

-7 | 2480 | 722 | 546 | 737 | 1219 | 2343 | 4644 | 6225 | 9716 | 23041 | **** |

-8 | 3871 | 1126 | 853 | 1150 | 1903 | 3667 | 7249 | 9716 | 1492 | 35361 | **** |

-9 | 9180 | 2672 | 2023 | 2728 | 4512 | 8672 | 17109 | 23041 | 35361 | 82951 | **** |

-10 | 85779 | 24967 | 18907 | 25492 | 42163 | 81030 | **** | **** | **** | **** | **** |

**Appendix: Probability of Having Two Best Net Differentials**

The probability of a player having his two best two differentials equal –n in 20 rounds is found by the following equation:

P(-n) = (sum from k=2 to 20)(20!/((20-k)!k!)) x

Where,^{k}a

^{20-k}

P(-n) = The probability the best two differentials out of 20 equal -n

x = Probability of a net differential of -n or higher (i.e., a worse score)

a = Probability to a net differential of –n

k = Number of –n differentials

x = Probability of a net differential of -n or higher (i.e., a worse score)

a = Probability to a net differential of –n

k = Number of –n differentials

The probability P(-n) covers the occurrences of 2,3,….20 differentials of –n in 20 rounds –i.e., even if a player has four rounds of with a net differential of –n, his two best differentials are still –n.

To find the probability of any particular net differential, we use the probabilities of scoring a net differential of various values or better. These are taken from Appendix E of the USGA Handicap System and are shown in Table 1A below.[1]

**Table 1A**

**Probability of Net Differentials or Better (13- 21.9 Index Player)**

Net Differential | 0 | -1 | -2 | -3 | -4 | -5 | -6 | -7 | -8 | -9 | -10 |

Probability | 1/5 | 1/10 | 1/21 | 1/43 | 1/87 | 1/174 | 1/323 | 1/552 | 1/1138 | 1/3577 | 1/37000 |

**Table 2A**

**Probability of Net Differentials**

Net Differential | 0 | -1 | -2 | -3 | -4 | -5 | -6 | -7 | -8 | -9 | -10 |

Probability | .1000 | .05238 | .02436 | .01176 | .00575 | .00265 | .00128 | .00093 | .00060 | .00025 | na |

**Table 3A**

**Probability of k Scores with a Differential of 0**

k | Combinations | P(0) ^{n·}P(>0)^{20-n} | Probability |

2 | 190 | .00018 | .03427 |

3 | 1140 | 2.2518E-5 | .02567 |

4 | 4845 | 2.81475E-6 | .01363 |

5 | 15504 | 3.51844E-7 | .00545 |

6 | 38760 | 1.15292E-8 | .00044 |

7 | 77520 | 5.49756E-9 | .00042 |

Total | .07997 |

The probability of having two best net differentials of 0 is 1 in 12. This differs from the USGA‘s estimate (1 in 27) since that estimate is based on only the occurrence of exactly 2 net differentials of 0. [2]

The size of any error in the USGA calculations can be seen in Table 4A which presents the Equation estimate, the Exactly 2 estimate, and the USGA estimates.

**Table 4A**

**Probability of Two Best Scores**

Two Best Scores | Equation | Exactly 2 | USGA |

0,0 | 12 | 29 | 27 |

-1,-1 | 9 | 13 | 13 |

-2,-2 | 18 | 21 | 20 |

-3,-3 | 54 | 58 | 59 |

-4,-4 | 189 | 186 | 200 |

-5,-5 | 818 | 831 | 821 |

-6.-6 | 3349 | 3375 | 3385 |

-7,-7 | 6210 | 6245 | 6225 |

-8,-8 | 14855 | 14908 | 14912 |

-9,-9 | 82671 | 82545 | 82951 |

The USGA presents the probability of the two best scores out of 20 being equal to -6 is 1 in 3385. But, using the same methodology, the probability of a player having two best scores of -6 or better is 1 in 572 or about six times more likely than having scores exactly equal to -6.[3]

The rarity the USGA finds is due in part to the interval it has chosen. Assume for example, the USGA had decimal intervals. The probability of a player having a net differential of -3 is the probability of a net differential between -3 and -3.1. The probability of two best net differentials of -3 (making simplifying assumptions) with this shortened interval is about 1 in 6800. Basing any modification of a player’s handicap on this probability would clearly not be fair.

The better method is to find the probability of two scores of –n or better. If that probability is found unreasonable, a player should have his index reduced. In examining the

*Handicap Reduction Table*,[4] it appears the probability of –n__or better__has been used. For example, for a player with 20 tournaments and an average net differential of -6, there is no handicap reduction. That is, the probability of scoring two net differentials of -6 or better is 1 in 570 and within the range of reasonability (where 1 in 3385 would not be). [5][1]

**The USGA Handicap System, 2012-2015**, USGA, Far Hills, NJ, 2012, p.117.[2] Knuth,

*loc. cit.*[3] The confusion about what these numbers mean is shown in a USGA Publication (

*How Well Should You Play?*):*“For example, the odds of our example player with a course handicap of 14 beating it by eight strokes (-8 net) once is 1,138 to one. Put another way, the average player posts 21 scores a year. That means to score this well, assuming the Handicap Index is correct, would take 54 years of golf to do once.”*

This is incorrect. The player is expected to accomplish the feat once in 54 years, but that is different than saying the feat will be accomplished at the end of 54 years. The player may well do it tomorrow. There is a 17 percent chance he will accomplish this feat in the first ten years. There is a 37 percent chance he will not accomplish the feat even in 54 years.

[4]

**USGA Handicap System**, p. 81.[5]

*Appendix E*rates performance by negative net differentials (score differential - handicap index). The*Handicap Reduction Table*uses positive net differentials (handicap – average of two differentials) in measuring performance. For consistency the USGA should use one type of differential. It is also not clear why the*Handicap Reduction Table*goes up to an average net differential of -14 when such scores are highly unlikely according to Appendix E.[1] It is USGA policy not to release its research to any outside agency.

[2] The USGA recommends a 90 percent allowance for four-ball stroke play. This allowance appears to come from a small study conducted by Francis Scheid, a one time member of the USGA Handicap Research Team, in 1971 (See F. Scheid, “You’re Not Getting Enough Strokes,”

*Golf Digest*, Trumbull, Connecticut, June 1971, p. 52). In this study, Scheid took 50 scorecards from members of his home club and simulated matches among them. Using such a small sample drawn from non-tournament conditions makes any result suspect.[3] In four-ball stroke play, for example, men play to 90 percent of their handicap, and women play to 95 percent. The USGA has never presented any theory on why the absence of a Y-Chromosome should alter a player’s handicap allowance.

[4] Note: Appendix E has been abbreviated here. It actually shows probabilities up to a -10 net differential

**.**[5] Knuth, D. L.,

*et. al., Outlier Identification Procedure for Reduction of Handicaps*,**Science and Golf II**, E & F Spon, Chapman and Hall, London 1994, p.230. This definition is affirmed in an article written by John Paul Newport in the*Wall Street Journal*(July 2, 2011). Dean Knuth, former Senior Director of Handicapping at the USGA, is quoted saying the odds of a midrange player shooting eight strokes better than his course handicap are 1 in 1,138.[6] Since the first draft of this paper, the USGA has dropped “odds” and now uses “probabilities” in Appendix E.

[7] Goldberg, Samuel,

*Probability: An Introduction*, Prentice Hall, Englewood Cliffs, N.J, 1960, p. 70. Let E be any event and P(E) be the probability of that event. We say that the odds for E are*a*to*b*if and only if: P(E)=a/(a+b). If odds for E are a to b, then the odds against E are b to a.

[8] Since the first draft of this paper, the probability for this 13 to 21 index range has been dropped from 1 in 6 to 1 in 5. No explanation was given for the change.

[9] This mistake is not an isolated example. For at least 5 years, Appendix E has shown the following incorrect equation:

74 -71.2 = 2.8 x 113/126 = 2.5 Handicap Differential

The equation should be:

(74-71.2) X 113/Slope Rating = 2.8 x 113/126 = 2.5 Handicap Differential

(Note: This error was corrected for the 2012-2015 edition of the

**Handicap System**. Unfortunately, current edition contains new errors).
## No comments:

## Post a Comment