The only thing the USGA hates more than making a mistake is
admitting to one. Examples of this
behavior are numerous. The most famous (or
infamous) was the USGA’s defense of the Masters for not disqualifying Tiger
Woods in 2013. The USGA argued this was an exceptional case and would determine
whether any “adjustment to the Rules
and/or Decisions is appropriate.”[1] The USGA’s curious logic, however, was never
memorialized in the Decisions. Its omission was due to the only exceptional
circumstance being the Masters Tournament Committee was headed by a former
president of the USGA.[2] Since this is a circumstance unlikely to be
repeated, the USGA apparently thought it best to paper over the error and move
on.
The USGA’s proclivity for dissembling even reaches down to
those responsible for the handicap system.
This post examines the truthfulness of the USGA in defending Appendix E,
Exceptional Tournament Score Probability
Table, of the USGA Handicap System. This example is not chosen because of its
importance to the handicap system. It
really has none. Its selection is based
on two other reasons. First, if the USGA
is less than straightforward on minor issues like Appendix E, it does not
portend well for its actions on major issues.
Second, Appendix E should be a place for the USGA to demonstrate its
competence in probability theory which is the backbone of the handicap
system. If the USGA errs here, similar
missteps can be inferred in the more important elements of the handicap system
(e.g., validity of the Slope System, allocation of handicap strokes, handicap
allowances).
For many years the probability table in the Appendix E looked
like that shown in Table 1 below. The
text explained “The values in the table are the odds (sic) of shooting a net
differential equal to or better than the number in the left column.”[3] The USGA wrote “odds” when it meant
“probability.” When this error was
pointed out, the USGA made the correction without admitting its previous error.
Table 1
Exceptional Tournament Score Probability
Table 20062007 (Abbreviated)
Net
Differential

Handicap Index Ranges


05

612

1321

2230

30 or Greater


0

5

5

5

5

5

1

10

10

10

8

7

2

23

22

21

13

10

While the probabilities (i.e., the inverse of the table
values) presented in the Table 1 could be questioned, at least they made sense. This changed with the 20122015 edition of
the USGA Handicap System where the probability
table took on a new form as shown in Table 2.
Table 2
Exceptional Tournament Score Probability
Table 20122015 (Abbreviated)
Net
Differential

Handicap Index Ranges


5.9 or Less

6.012.9

13.021.9

22.030.9

31.0 or Greater


0 to .9

5

5

5

5

5

1.0 to 1.9

10

10

10

8

7

2.0 to 2.9

23

22

21

13

10

The text in Appendix E again explained “The values in the
table represent the probability of shooting a Net Differential equal to or
better than the number in the left column.”[4]
The problem is there is no longer a
“number” in the left column but a “range” of numbers. A simple examination of the normal curve
would show the probability of shooting a net differential of 0 or better differs
greatly from the probability of shooting a net differential of .9 or better. This discrepancy in the Appendix was pointed
out to the USGA along with many typographical errors in the table.[5] Scott
Hovde, Assistant Director of Handicap and Course Rating, did not accept the
criticism and defended the change as an improvement:
When
reviewing the entire Handicap manual for the 2012 update, we looked at the
table in Appendix E to determine if it needed some tweaking or updating.
The initial thought was to remove the table from the book altogether and only
supply an online version once the table was fully reviewed. In consulting
with a few of our HRT members (who were involved with the original table), it
was decided to keep Appendix E in the book, but expand the rows to ranges, as
the formula (and example) listed underneath the table to determine the net
differential is taken to the tenth place. The values in each cell
represented an average of the probability of net differentials within that
range, and not just a single net differential. In reality each 0.1
difference in net differential will have a separate probability, but that table
would be far too large to be of use.
In
the interest of ease of use by our target audience, the ranges were there to
keep them from having to interpolate and just plug in the net differential, and
probability was listed in each cell as the divisor, instead of probability
itself (so 1200 instead of .0008333…)…The typos
in the manual were unfortunately made at the printer when they formatted the
table for final printing, the table we submitted and that was reviewed by all
involved had the correct values. [6]
Hovde does admit to the typos, but argues the
mistake was not the USGA’s, but the printer’s.
For this to be true, it must be believed the USGA did not request,
receive, and copy edit a proof copy before publication. In other words, Hovde’s defense implies the
USGA was guilty of gross negligence.
The most charitable interpretation of Hovde’s
meandering syntax is that cell values represent the probability of a Net
Differential equal to or better than some average value within the range. In the case of a range from 0 to .9, for
example, the cell value represents the probability of a Net Differential equal
to or better than .45 (i.e., an average value). The user will still have to interpolate if he
wants a more accurate estimate of the probability. So listing ranges is not a benefit to the
user as Hovde claims.
Hovde argues members of the Handicap Research
Team (HRT) agreed with this new formulation.
The probabilities, however, did not change with the new edition. If the cell values are really “average”
probabilities, all previous editions have been incorrect. For example, the USGA has previously maintained
a player should play to his Index 20 percent of the time. The 20122015
Edition now implies a player has a much better chance to play to his Index
than .20. It is difficult to believe the
HRT would be party to such a ruse.
Hovde also maintains two or more of the
originators of the table agreed with using ranges. The only surviving members of the HRT who did
research on outlier identification are Dean Knuth and Frank Engel. Knuth has implied he had nothing to do with
the change.[7] Perhaps Engel supported the change, but that
would still be far from the unanimous approval Hovde alleges. It is more likely Hovde tried to use the
imprimatur of the HRT to add credence to his argument.
The final piece of evidence showing Hovde’s
defense was disingenuous came with the publication of the USGA Handicap System 20162017.
The table in Appendix E now looks like that shown in Table 3 below.[8] Table 3 shows ranges have disappeared and the
left column has reverted to integers. If
there were major benefits to be gained by using ranges, why have they
disappeared?
Table 3
Score Frequency and Probability Table 20162017
(Abbreviated)
Net Diff.

Handicap
Index Ranges


<
0.0

0.0 4.9

5.09.9

10.014.9

1519.9

20.024.9

25.029.9

>29.9


0

4.8

4.3

4.3

4.7

5.2

6.2

7.1

6.8

1

8.8

7.6

7.2

7.6

8.3

9.7

11

9.8

2

19

15

13

13

14

16

17

15

In early 2012, the USGA must have realized
Appendix E was in error. To make a correction,
however, would mean running a revised Appendix E by the Handicap Procedure
Committee and admitting to a mistake.
The better option, and the one chosen, would be to bury the change in
the 20162017 Edition without
explanation and hope no one would notice.
And so it goes…
[1]
“USGA, The R&A Issue Statement Addressing Tiger Woods Ruling at the 2013
Masters Tournament,” USGA, Far Hills, N.J., May 1, 2013.
[2]
For a full discussion of this ruling see “Bureaucracy, Augusta National, and
Tiger Woods,” www.ongolfhandicps.com,
April 8, 2013.
[3] The USGA Handicap System, 20062007,
USGA, Par Hills, NJ, p. 126.
[4] The USGA Handicap System, 20122015, USGA,
Far Hills, NJ, p. 117. There is some
evidence the column headings were once Handicap Ranges and were switched to
Index Ranges to be consistent with Slope System nomenclature. See “The
Reliability and Accuracy of USGA Handicap Research,” www.ongolfhandicaps.com, March 6,
2012. The USGA has not made any study
supporting Appendix E available to the public, so its validity remains in
question.
[5]
Dougharty, Laurence, email to Scott Hovde of the SSGA, February 1, 2012.
[6]
Hovde, Scott, email to author, February 9, 2012.
[7]
Knuth, Dean, email to author, February 13, 2012.
[8] The
USGA will not release the study behind the revised Appendix. Even without seeing the study, however, some
problems with its results are evident see “The USGA’s Appendix E: Problematic
at Best,” www.ongolfhandicaps.com,
forthcoming.