Friday, May 27, 2016

The R&A Battles for Gender Equality: Or Does It?


The Royal and Ancient Golf Club of St. Andrews(R&A) recently announced Muirfield Golf Club would no longer be considered as a site for The Open Championship.  The announcement was pure political theater since Muirfield was not up in the rotation for a considerable number of years.  The R&A’s intent was to show its enlightenment on issues of gender equality to placate sponsors of its golden goose—The Open.  The R&A had a lot of work to do to clean-up its bona fides as a civil rights champion.  After all, for 260 years the R&A had the same policy of barring women from membership that Muirfield was now being pilloried for.  Here are some alternatives the R&A might have considered for image control and why it chose to do what it did (i.e., alternative 4):

1. Cancel this year’s Open Championship at Royal Troon Golf Club which also does not allow women members - This step would dramatically demonstrate how seriously the R&A takes the issue.  While it might punish Royal Troon, it would wreak more economic harm on the R&A.  Better to argue Troon was awarded the Open by a previous administration of the R&A, have the television announcers “tsk, tsk” the men only policy, and reap the millions The Open will bring. 

2. Atone for years of discrimination by putting itself on probation and granting sole control over the Rules of Golf for 5 years to the United States Golf Association (USGA) - The problem with this alternative is some of the nabobs of the USGA were members of the R&A during the heyday of sexual discrimination.  The USGA should not benefit from a crime in which it abetted.  David Fay, former Executive Director of the USGA comes to mind.  While he famously resigned form the all male Pine Valley Club, he kept his membership in the R&A. It is more likely the notoriously parsimonious Fay left Pine Valley because he didn’t want to pay the dues rather than from any moral convictions.

Mike Davis, the current Executive Director of the USGA does not appear to have cleaner hands.  He too is probably a member of Pine Valley since he posts scores from there.  He was a member at Trump National, New Jersey until Donald Trump found disfavor with the liberal elites.  When The Donald was a fringe candidate and given little hope of winning, Davis was quick to announce the USGA was reconsidering holding the US Women’s Open at Trump National.[1]  Since Trump won the Republican nomination, Davis has been mute on relocating the tournament. 
 
If the R&A were to cede control, it must find an organization with a much better record on gender equality and that also has the courage of its convictions.  That would not be the USGA.

3. Assume control the Women’s British Open and give equal prize money to men and women - The R&A is in the process of merging with the Ladies Golf Union which owns the rights to the Women’s British Open.  Currently, the difference in total purses between the two Opens is about $7 million.  To get rid of the title sponsor of the Women’s Open (Ricoh) would probably cost an additional $1 million or more.  What price is the R&A willing to play to demonstrate its commitment to full gender equality?  It is probably somewhere south of $8 million a year.

4. Issue a press release chastising Muirfield and announce its removal from the rota until Club members vote for a more inclusive membership policy – This alternative would give the R&A moral high ground and be cost free.   In the meantime, Muirfield would be encouraged to follow the R&A’s lead by inviting a few women to join. The three main arguments would go like this:
  • Women want the right to be a member more than they want to be a member.   If the membership becomes open to women, the Club may have to recruit women to prove it has an inclusive membership policy. It is likely women memberships will top off at 1 percent of the total membership as they do at the R&A and Augusta.  Does Muirfield really want to take to the barricades over this minor incursion?
  • It’s not like Muirfield is all male club where women are not permitted on the grounds.  Female tourists and member’s wives are mulling around all the time.  A few more women will not be noticed and they have the upside of paying dues.  True, Sir Edgar will have to more careful about where he takes a leak.  Lose the Open or fit Sir Edgar with a catheter?  Take your pick.
  • If Muirfield is not in the rota, tourists will not pay as much to play it.  This will be an income hit for the club.  The only way to make up for the loss is for the members to pay more for dues and/or liquor.  Will single malt scotch taste as good at twice the price?  

There is little doubt the Muirfield membership will eventually find these arguments compelling.  With that settled, Muirfield will resume its place in the rota.  The R&A will be praised for its tough stand that brought about change, and Muirfield will be extolled for its enlightened, if belated, policy.   In essence, everything appears to change, but nothing really changes--the perfect bureaucratic solution.
                    




[1] Mell, Randell, “Davis: USGA ‘evaluating things’ regarding Trump, Golf Central Blog, July 9, 2015.

Tuesday, May 17, 2016

The Probability of Beating Your USGA Index


Appendix E of the 2016-2017 USGA Handicap System presents estimates of the probability of a player of achieving various negative Net Differentials or zero or better.[1]  These estimates differ substantially from the estimates the USGA has presented in the past.  Specifically, players no longer have an equal chance of beating their Index, and the chances for having a large negative Net Differential are substantially increased.  This post examines possible causes for the discrepancies between the two sets of estimates.          

The Probability of Beating Your Index -The 2012-2015 Edition of the USGA Handicap System estimated the probability of beating one’s Index to be .20 for all Index ranges as shown in Table 1.  In the 2016-2017 Edition, that probability varies with a player’s Index (see Table 1).  The low-Index player now has a 64 percent better chance of beating his Index than his high-Index competitor.  What causes the high-Index player to perform so poorly?   Is he confused by too many tips from Golf Digest?  Can he not hit the pressure shots essential for low scores due to the low self-esteem of habitually being placed in the last flight? 
Table 1
Probability of Beating Your Index[2]


Method
Index (Average Differential)
+2.0 (0)
3.0 (5)
7.0 (10)
17.0 (20)
27.0 (30)
2012-2015 Ed.
.20
.20
.20
.20
.20
2016-2018 Ed.
.21
.23
.23
.19
.14
Normal Dist.
.22
.20
.19
.17
.15

No, psychological or physical traits do not explain the phenomenon.  The poor performance of the high-Index players is most likely the direct result of USGA handicap policy, specifically the Bonus for Excellence (BFE). 
 To see how the probability of beating  one’s Index is affected by the BFE, assume a player’s differential follows a normal distribution with a mean of X and a standard deviation of σ.[3]  It can be shown (see Appendix A) that the average of a player’s better half of differentials is X - .8·σ.  A player’s Index is then:
1)           Index = BFE·(X-.8·σ)
Where,
              BFE = Bonus for Excellence
A player’s scoring differential is
2)           Player’s Scoring Differential = X - S·σ 
Where,
S = Number of standard deviation from the mean.
For a player to beat or equal his Index: 
3)           X-S·σ ≤ BFE·(X - .8·σ)
Solving for S:
4)           S ≥ (1- BFE) ·X/σ + BFE .8
If the BFE was 1.0, all players would have the same chance of beating their Index.  They would have to have a scoring differential equal or better than .8·σ below their mean scores.  From the Normal Distribution Table, the probability of this occurrence is .21.  Substantially the same probability the USGA has maintained in previous years.
The BFE is not 1.0, however, but .96.  For the purposes of illustration, it is assumed that a player’s σ is a linear function of his average differential, X:[4]
5)           σ = 3 + 0.05 ·X 
A player with an average differential of 0.0 would have σ of 3.0.  A player with an average differential of 30.0 would have σ = 4.5.
Substituting equation 5) into equation 4), the new equation for S becomes:
6)           S ≥ (1 - .96) ·X/(3+(0.05·X)) +.96·.8 
For a player with an average differential of 30, S equals 1.04.  From the Standard Normal Table, the probability of such a differential or better is .15.  The probabilities for a range of average differentials are presented in Table 1.  The estimates from the Normal Distribution analysis mirror the empirical findings in the 2016-2017 Appendix E.  Both estimates reveal the high-Index player has less of a chance of beating his Index than the low-Index player.   The Normal Distribution analysis identifies the BFE as the likely culprit.  There is no theoretical or empirical evidence suggesting the BFE promotes equity.[5]  It was born from political compromise and not statistical analysis.   

Why Do the Estimates Differ by So Much?  There is no apparent correlation between the two sets of probabilities.  There are major differences, however, between the estimates.   For example, in the 2012-2015 Edition the frequency of a player with a 20.0 Index having a Net Differential of -10 or better was 1 in 37,000.  In the 2016-2017 Edition, the frequency is now estimated at 1 in 1,950.   If one of the estimates is correct, the other is off by a factor of 19.
The best explanation is the old estimates were not accurate.[6]   They appear to be based on a study by Bogevold in 1974 (unpublished).   Most of the research back in the ‘70s and ‘80s depended upon collecting scorecards from various sources.   The cost of collecting and processing scorecards necessarily limited the sample size.  It is doubtful Bogevold had a sufficient sample to estimate the probability of large negative Net Differentials with any accuracy.  Moreover, these scores did not reflect the Slope System which was not in existence at the time and were reported using a different stroke adjustment procedure than used today.  All of these factors make the probability estimates suspect.   There is no evidence the USGA ever tried to replicate the results with another study so the probability estimates were always of questionable validity.
While the old estimates may be in error, the 2016-2017 Edition estimates also have problems:
Measurement Errors -The USGA Handicap System only estimates a player’s true Index.  If the USGA used an unbiased estimate of a player’s Index there would not be a significant problem.  If the USGA used a player’s Current Index, however, the probability estimates for negative Net Differentials would be biased downward.
Model Error - Appendix E assumes the probabilities are a function of a player’s Index.  For the most part, a player’s probability of achieving negative Net Differentials is determined by his standard deviation and not his Index.  It is quite possible a wild 15.0 Index has a better chance of having a -12 Net Differential than a steady 30.0 Index player.  The probabilities in Appendix E are for an average player within each Index range.  There can be a large error if they are applied to an individual’s performance.  
No Estimate of the Error - Appendix E does not give the reader any estimate of the accuracy of its results.  Most likely, the USGA just took 7.3 million differentials, placed them into cells, and did the necessary long division to produce a frequency.   Appendix E implies 100 percent confidence in the estimates, which is clearly not true.  The fluctuating nature of the probabilities as a player’s Index increases cannot be explained by any theory, and should be attributed to random error. 
Sample Size - The handicap Index Classifications do not include the same number of players. The number of players with a plus Index is approximately 1 percent of the population.  Appendix E shows such players have a 1 in 9,216 chance of having a Net Differential of -7 or better.  The frequency of a Net Differential of -8 or better is “off the charts.”  The frequency should not be “off the charts” since the chart goes up to 1 in 46,328 for the 10.0 to 14.9 Index player.  It is likely the sample of players with plus Indexes was too small to capture any Net Differential of -8 or better.
Women - It is unlikely women were included in the sample, but if they were it would corrupt the analysis.  If women were not included, the USGA should make clear Appendix E only applies to men.

Conclusion - There are two conclusions.  First, the BFE should go the way of the stymie since it does not serve its intended purpose of promoting excellence[7] and is a source of inequity within the USGA Handicap System.
Second, trying to estimate the probability of a Net Differential by a player’s Index is a fool’s errand.  There is too much variance within Index ranges to produce meaningful results.  Appendix E should be eliminated since it serves no real purpose in the Handicap System.[8]   A better approach would be to choose a player with an average standard deviation and estimate the probabilities for various Net Differentials using the Standard Normal Table.  This procedure produces estimates very similar to those found in the 2016-2017 Appendix E.  These results should be presented as a rule-of-thumb to give players a rough idea of the likelihood of various Net Differentials.   To strive for more precision is both misleading and a waste of USGA resources.   

Appendix A
The Probability of Beating Your Index Assuming a Normal Distribution

Assume the distribution of a player’s differentials has a standard deviation of σ.  The average of a player’s better half of differentials is found by multiplying possible differentials measured in standard deviations from the mean by the probability of making that score.   For example, the probability a player has a net differential between 0 and .1 standard deviations below his mean differential is 0.0396.  The probability that one of the player’s best scores is between 0 and .1 standard deviations below the average differential is 0.0796 (i.e., the probability is multiplied by two since only scores below the average are included).  The player’s average differential is found by summing all expected value for all intervals.  If the value within an interval is approximated by the mean value within the interval, a player’s average  of his best differentials can be estimated as shown in Table A-1. The estimate is approximately -.8·σ.
Table A-1
Calculation of Expected Differential (Abbreviated)

Interval
Mean Value
Probability
Expected Value
0 to -.1
-0.05·σ
.07960
-.003980·σ
-.1 to -.2
-0.15·σ
.07900
-.011850·σ
-.2 to -.3
-0.25·σ
.07722
-.019305·σ
…
…
…
…
-2.9 to- 3.0
-2.95·σ
.00104
-.003068·σ


Total
-.789681·σ






[1] Net Differential = (Adjusted Score – Course Rating)·113/Slope Rating – Index
[2] The groupings of players are different from that shown in Appendix E.  The analysis presented here is concerned with average differentials.  The average differential consistent with the Index is shown in the heading of Table 1.  For example, an Index of 27.0 corresponds to an average differential of approximately 30.   Therefore USGA results for a player with a 27.0 Index are compared to a player with a 30.0 average differential.
[3] The USGA has maintained golf scores follow the normal distribution.  See Scheid, F.J., “On the normality and independence of golf scores with various applications,” Science and Golf, E &F Spon, London, 1990, pp. 147-152.
[4] From the probabilities presented in Appendix E, it is possible to estimate the standard deviation by Index range.  Equation 5) is a reasonable representation of the relationship between a player’s average differential and his standard deviation.
[5] In a rudimentary study, Dr. Francis Scheid estimated a BFE of 1.07 was the most equitable (“You’re not getting enough strokes!, “Golf Digest, June 1971).  Given the small sample size, a BFE of 1.0 probably could not be eliminated as the most equitable.
[6] A possible error was the old estimates were really the odds of negative Handicap Differentials (i.e., (Adj. Score – Course Rating) – Handicap = Handicap Differential) and not negative Net Differentials.  On a course with a Slope Rating greater than 113, however, a player has a better chance of scoring a Handicap Differential of –N than a Net Differential of –N (N>0).  If the old estimates were for Handicap Differentials, the old probabilities should be systematically higher than the new probabilities.  They are not, so the explanation for the differences must lie elsewhere.
[7] See Dougharty, Laurence, “The Bonus for Excellence Ruse,” www.ongolfhandicaps.com, January 15, 2013.
[8] Appendix E has had many problems over the years as documented in “The Unmaking of the USGA’s Appendix E,” www.ongolfhandicaps.com, March 2, 2016. 
[9] Newport, John Paul, “Fighting Back Against Sandbaggers,” Wall Street Journal, July 2, 2011.

Wednesday, March 2, 2016

The Unmaking of the USGA's Appendix E

The only thing the USGA hates more than making a mistake is admitting to one.  Examples of this behavior are numerous.  The most famous (or infamous) was the USGA’s defense of the Masters for not disqualifying Tiger Woods in 2013. The USGA argued this was an exceptional case and would determine whether any “adjustment to the Rules and/or Decisions is appropriate.”[1]  The USGA’s curious logic, however, was never memorialized in the Decisions.  Its omission was due to the only exceptional circumstance being the Masters Tournament Committee was headed by a former president of the USGA.[2]  Since this is a circumstance unlikely to be repeated, the USGA apparently thought it best to paper over the error and move on.

The USGA’s proclivity for dissembling even reaches down to those responsible for the handicap system.  This post examines the truthfulness of the USGA in defending Appendix E, Exceptional Tournament Score Probability Table, of the USGA Handicap System.  This example is not chosen because of its importance to the handicap system.  It really has none.  Its selection is based on two other reasons.  First, if the USGA is less than straightforward on minor issues like Appendix E, it does not portend well for its actions on major issues.  Second, Appendix E should be a place for the USGA to demonstrate its competence in probability theory which is the backbone of the handicap system.  If the USGA errs here, similar missteps can be inferred in the more important elements of the handicap system (e.g., validity of the Slope System, allocation of handicap strokes, handicap allowances).

For many years the probability table in the Appendix E looked like that shown in Table 1 below.  The text explained “The values in the table are the odds (sic) of shooting a net differential equal to or better than the number in the left column.”[3]  The USGA wrote “odds” when it meant “probability.”  When this error was pointed out, the USGA made the correction without admitting its previous error. 

Table 1
Exceptional Tournament Score Probability Table 2006-2007 (Abbreviated)

Net
Differential
Handicap Index Ranges
0-5
6-12
13-21
22-30
30 or Greater
0
5
5
5
5
5
-1
10
10
10
8
7
-2
23
22
21
13
10

While the probabilities (i.e., the inverse of the table values) presented in the Table 1 could be questioned, at least they made sense.  This changed with the 2012-2015 edition of the USGA Handicap System where the probability table took on a new form as shown in Table 2.

Table 2
Exceptional Tournament Score Probability Table 2012-2015 (Abbreviated)

Net
Differential
Handicap Index Ranges
5.9 or Less
6.0-12.9
13.0-21.9
22.0-30.9
31.0 or Greater
0 to -.9
5
5
5
5
5
-1.0 to -1.9
10
10
10
8
7
-2.0 to -2.9
23
22
21
13
10

The text in Appendix E again explained “The values in the table represent the probability of shooting a Net Differential equal to or better than the number in the left column.”[4]  The problem is there is no longer a “number” in the left column but a “range” of numbers.  A simple examination of the normal curve would show the probability of shooting a net differential of 0 or better differs greatly from the probability of shooting a net differential of -.9 or better.   This discrepancy in the Appendix was pointed out to the USGA along with many typographical errors in the table.[5]    Scott Hovde, Assistant Director of Handicap and Course Rating, did not accept the criticism and defended the change as an improvement:

When reviewing the entire Handicap manual for the 2012 update, we looked at the table in Appendix E to determine if it needed some tweaking or updating.  The initial thought was to remove the table from the book altogether and only supply an online version once the table was fully reviewed.  In consulting with a few of our HRT members (who were involved with the original table), it was decided to keep Appendix E in the book, but expand the rows to ranges, as the formula (and example) listed underneath the table to determine the net differential is taken to the tenth place.  The values in each cell represented an average of the probability of net differentials within that range, and not just a single net differential.  In reality each 0.1 difference in net differential will have a separate probability, but that table would be far too large to be of use.  
In the interest of ease of use by our target audience, the ranges were there to keep them from having to interpolate and just plug in the net differential, and probability was listed in each cell as the divisor, instead of probability itself (so 1200 instead of .0008333…)…The typos in the manual were unfortunately made at the printer when they formatted the table for final printing, the table we submitted and that was reviewed by all involved had the correct values. [6] 

Hovde does admit to the typos, but argues the mistake was not the USGA’s, but the printer’s.  For this to be true, it must be believed the USGA did not request, receive, and copy edit a proof copy before publication.  In other words, Hovde’s defense implies the USGA was guilty of gross negligence.    

The most charitable interpretation of Hovde’s meandering syntax is that cell values represent the probability of a Net Differential equal to or better than some average value within the range.  In the case of a range from 0 to -.9, for example, the cell value represents the probability of a Net Differential equal to or better than -.45 (i.e., an average value).   The user will still have to interpolate if he wants a more accurate estimate of the probability.  So listing ranges is not a benefit to the user as Hovde claims.

Hovde argues members of the Handicap Research Team (HRT) agreed with this new formulation.  The probabilities, however, did not change with the new edition.  If the cell values are really “average” probabilities, all previous editions have been incorrect.  For example, the USGA has previously maintained a player should play to his Index 20 percent of the time.  The 2012-2015 Edition now implies a player has a much better chance to play to his Index than .20.  It is difficult to believe the HRT would be party to such a ruse. 

Hovde also maintains two or more of the originators of the table agreed with using ranges.  The only surviving members of the HRT who did research on outlier identification are Dean Knuth and Frank Engel.  Knuth has implied he had nothing to do with the change.[7]  Perhaps Engel supported the change, but that would still be far from the unanimous approval Hovde alleges.  It is more likely Hovde tried to use the imprimatur of the HRT to add credence to his argument.

The final piece of evidence showing Hovde’s defense was disingenuous came with the publication of the USGA Handicap System 2016-2017.  The table in Appendix E now looks like that shown in Table 3 below.[8]  Table 3 shows ranges have disappeared and the left column has reverted to integers.  If there were major benefits to be gained by using ranges, why have they disappeared? 

Table 3
Score Frequency and Probability Table 2016-2017 (Abbreviated)


Net Diff.
Handicap Index Ranges
<  0.0
0.0- 4.9
5.0-9.9
10.0-14.9
15-19.9
20.0-24.9
25.0-29.9
>29.9
0
4.8
4.3
4.3
4.7
5.2
6.2
7.1
6.8
-1
8.8
7.6
7.2
7.6
8.3
9.7
11
9.8
-2
19
15
13
13
14
16
17
15

In early 2012, the USGA must have realized Appendix E was in error.  To make a correction, however, would mean running a revised Appendix E by the Handicap Procedure Committee and admitting to a mistake.  The better option, and the one chosen, would be to bury the change in the 2016-2017 Edition without explanation and hope no one would notice.  And so it goes…





[1] “USGA, The R&A Issue Statement Addressing Tiger Woods Ruling at the 2013 Masters Tournament,” USGA, Far Hills, N.J., May 1, 2013. 
[2] For a full discussion of this ruling see “Bureaucracy, Augusta National, and Tiger Woods,” www.ongolfhandicps.com, April 8, 2013.
[3] The USGA Handicap System, 2006-2007, USGA, Par Hills, NJ, p. 126.
[4] The USGA Handicap System, 2012-2015, USGA, Far Hills, NJ, p. 117.  There is some evidence the column headings were once Handicap Ranges and were switched to Index Ranges to be consistent with Slope System nomenclature. See “The Reliability and Accuracy of USGA Handicap Research,” www.ongolfhandicaps.com, March 6, 2012.  The USGA has not made any study supporting Appendix E available to the public, so its validity remains in question.
[5] Dougharty, Laurence, e-mail to Scott Hovde of the SSGA, February 1, 2012.
[6] Hovde, Scott, e-mail to author, February 9, 2012.
[7] Knuth, Dean, e-mail to author, February 13, 2012.
[8] The USGA will not release the study behind the revised Appendix.  Even without seeing the study, however, some problems with its results are evident-- see “The USGA’s Appendix E: Problematic at Best,” www.ongolfhandicaps.com, forthcoming.