Saturday, January 17, 2015

Problems with the USGA’s Stroke Allocation Procedure

Note: The World Handicap System (WHS) listened to complaints and did away with the USGA recommendation for stroke allocations and adopted those recommended below. This blog likes to think it played a part in the WHS revision, but probably not.

Introduction and Summary – For 30 years the United States Golf Association (USGA) has recommended the following handicap stroke allocation procedure for match play:[1]

A handicap stroke should be an equalizer rather than a winning stroke and should be available on a hole where it most likely will be need (sic) by the higher-handicapped player to obtain a half (sic) in singles… [2]

The USGA has never demonstrated its recommended procedure does in fact assign strokes where they are most needed, nor has it demonstrated the procedure is superior to other procedures in ensuring the equity of competition.  This post attempts to evaluate the efficacy of the USGA procedure by asking and answering a series of questions about handicap stroke allocation.

1.     Does the USGA procedure actually identify where a high-handicap player will most likely need a stroke to obtain a halve?  It does not.  As shown in the USGA’s example in Section 17, the procedure assigns the first handicap stroke to a hole where it will be of the least use to the higher-handicap player. 

2.     Is there a better procedure for allocating handicap strokes for match play?  Probably not.  If the USGA wanted handicap strokes to serve as equalizers, allocations should be made on the basis of how close the average difference in hole scores between higher and lower handicap players is to 1.0.  This average difference, however, will change with each handicap pairing.  Even if this equalizer objective is adopted—and there is no evidence it should be—there would not be one best stroke allocation suited for all matches.  Golf Australia recognized this when it recommended the same match play stroke allocation for all courses.  

3.     Is the USGA procedure defined with sufficient specificity to be implemented?  No.  The USGA advocates that the first handicap stroke should be allocated so that this stroke is most useful in a match between players of almost equal ability.   The USGA, however, does not identify any process for determining where a stroke would be most useful in such a match.

4.     Can a case be made for eliminating the stroke allocation procedure for match play?  Yes.  The USGA’s recommended method yields some strange results that are not easily understood by the average player—e.g., why don’t I get a stroke on a difficult hole?  Moreover, given the random nature of scoring it does not appear to make a significant difference to the equity of competition where strokes are given.  Given the many failings of its recommendation, the USGA should eliminate its match play stroke allocation procedure and replace it with general guidelines for ranking holes. 

Each question is discussed in turn:

1.  Does the USGA procedure actually identify where a high-handicap player will most likely need a stroke to obtain a halve?

The USGA’s recommended procedure for allocating handicap strokes does not ensure a player will get a stroke where it is most needed to ensure a halve.  This can be seen in the example presented in Sec. 17 of the Handicap System which is reproduced in part in the Table 1 below.[3]

Table 1
Stroke Allocation Based on Average Score Difference

(1)

Hole
(2)
Average  Score Difference
(3)

Rank
(4)

Strokes
(5)
Net Average Score Difference
(6)
Match
Standing
1
1.13
8
1
0.13
-1
2
1.41
4
1
0.41
-2
3
0.75
14
1
-0.25
-1
4
2.10
2
1
1.10
-2
5
0.74
16
1
-0.26
-1
6
0.73
18
0
0.73
-2
7
0.93
10
1
-0.07
-1
8
1.22
6
1
0.22
-2
9
0.88
12
1
-0.12
-1
10
1.05
13
1
0.05
-2
11
1.45
7
1
0.45
-3
12
2.04
3
1
1.04
-4
13
1.39
9
1
0.39
-5
14
0.8
15
1
-0.20
-4
15
1.22
11
1
0.22
-5
16
0.75
17
0
0.75
-6
17
1.84
5
1
0.84
-7
18
2.38
1
1
1.38
-8

The USGA recommends the stroke allocation for match play be determined by the average difference in score between low- and high-handicap players.  The average difference is used as a proxy for where the high handicap player needs a stroke.   To demonstrate the computation procedure, the USGA used two groups of players.  The first group had an average handicap of 6.  The second group had an average handicap of 22.  The difference in average score between the two groups for each hole is shown in Column 2 of the Table.

Within each nine, the hole with the largest average difference gets the lowest stroke allocation.  For example, hole 4 has the largest average difference among the holes on the first nine.  Therefore, it is assigned a stroke allocation of 2.  (Note: The second nine is considered more difficult so it is assigned the odd-number strokes.)  The next largest difference (i.e., 1.41) is assigned a stroke allocation of 4 and so on as shown in Column 3.

Does this stroke allocation promote equity?  Let’s assume a match between a 6- and a 22-handicap with the same average score differences shown in Column 2 of the Table.  Column 5 presents the net average difference in score after the 16 handicap strokes are applied according to the recommended stroke allocation.  The last column shows the standing in an average match.  The 22-handicap loses 5 and 3 and would lose 8 down if the match continued.[4] 

The problem with the USGA’s method is that it does not automatically assign strokes to where they would be of most use to the high-handicap player.  In the USGA’s example, the stroke allocation method ensures that a high-handicap player will get a stroke where it probably will be of little use.  The high-handicap player gets a stroke on hole 18, but the average difference in scores will still be 1.38 strokes even after the handicap stroke is applied.  With such a large difference in scores, the high-handicap player can be expected to have a higher average score than his competitor 92 percent of the time.   In summary, the USGA procedure assigned the first handicap stroke to where it will likely be of least use to the higher-handicap player.        

2. Is there a better procedure for allocating handicap strokes for match play?

The USGA procedure allegedly assigns handicap strokes where they most likely will be needed to obtain a halve.  The USGA measures “need” by the difference in scores between low- and high- handicap players.  As discussed above, the average difference in score is not a good proxy for “need.” 

If the USGA wants a stroke to be an equalizer, it should be applied to a hole where it evens the probability of winning between competitors.  Using the average score differences  in Sec. 17, Table 2 shows the probability of winning (i.e., having an equal or lower average score than the lower handicap player) for the higher-handicap player if a stroke was given on a hole.  Holes are then ranked by the closeness of the probability of winning to .50.  These changes would make the match more competitive.   The higher-handicap player no longer wastes a handicap stroke on holes 4 and 18.  The 22-handicap would lose 3 and 1 under the allocations shown in Table 2 rather than 5 and 3 under the USGA’s recommended procedure.
Table 2
Using the Probability of Winning a Hole as a Stroke Allocation Rule

Hole
Probability
Allocation
Hole
Probability
Allocation
1
.45
6
10
.48
1
2
.34
16
11
.33
11
3
.60
10
12
.15
15
4
.14
18
13
.35
9
5
.60
12
14
.58
3
6
.61
14
15
.41
5
7
.53
2
16
.60
7
8
.41
8
17
.20
13
9
.55
4
18
.08
17

To determine the allocations shown in Table 2 it is not necessary to calculate the probabilities of winning for each hole.  Instead, holes can be ranked by how close its average score difference is to 1.0.  For example, if two holes had average score differences of .85 and 1.1, the hole with the 1.1 score difference would receive the lower allocation.  This “proximity to 1.0” rule will yield the same allocations as the probability rule shown in Table 2.

Unfortunately, the “proximity to 1.0” rule gives different results depending upon the difference in handicaps.  If the difference in handicap between competitors is halved, for example, the average score differences are also halved under the USGA assumption.  Different holes would now have an average score difference closer to 1.0.  So even if the “proximity to 1.0” test was the most equitable, it is administratively infeasible and could not be implemented. 

Golf Australia (GA)—Australia’s counterpart to the USGA— recognized that there is no perfect allocation.   It recommended the same stroke allocation for all courses.  GA argued that it disregarded hole difficulties (acknowledging that a 30-marker receives 5 strokes from a 25-marker, which is what a 5- marker receives from a scratch-marker, but that there can be clear differences in the holes a 5-marker and a scratch-marker will find most challenging).[5]  GA’s recommended stroke allocation avoids allocating low-numbered strokes to the last two holes and allocating low-numbered strokes to the first three holes in case a match goes to extra holes.  In essence, GA is arguing that any match play allocation, including a “difficulty” allocation, would be acceptable as long as its recommendations on the opening and closing holes were followed. 
     
The USGA’s recommendation is also alleged to promote equity in four-ball matches.  The USGA, however, has never presented any theoretical or empirical evidence to prove that assertion.  The USGA argues that the optimal allocation in a four-ball match can be determined by the average difference in scores between low- and high-handicap players.  As noted above, however, the best allocation may vary by the difference in the handicaps of the competitors.  When you introduce two more players to the competition, the analytical problem becomes intractable.   Fortunately, there is so much randomness in scoring the actual allocation is probably not determinant of the final outcome of the four-ball match.    

 3. Is the USGA procedure defined with sufficient specificity to be implemented?

 The questionable logic behind the USGA’s recommendation becomes apparent in the allocation criterion presented in Sec. 17-1bii:[6]

The first handicap stroke should be allocated so that this stroke is most useful in a match between players of almost equal ability (e.g., a match involving players with a Course Handicap of 0 and 1, 10 and 11, and 29 and 30).  The second handicap stroke should be allocated so that this stroke is most useful in a match between players having a slightly greater difference in Course Handicap (e.g., a match involving players with a Course Handicap of 0 and 2, 10 and 12, or 29 and 31).  This process should be continued until the first six strokes have been assigned. (Editorial note: The USGA does not delineate how stroke allocations for the remaining 12 holes are to be determined.)

The USGA does not present any procedure to determine where a stroke will be most useful in a match between players of almost the same ability.  The USGA also assumes, without evidence, that a handicap stroke would be most useful over a broad range of ability—i.e., would a 1 handicap and a 30 handicap find a handicap stroke most useful on the same hole? 

When 17-1bii is applied to the rankings shown in the USGA example, the allocations remain the same on the front nine, and only modified on the back nine to eliminate Hole 18 as the number 1 stroke hole.  Stroke allocations were essentially based on the average difference between players of dissimilar ability and not of equal ability.  There is no mention in the example of assigning the first stroke hole to where it would be useful in a match between players of comparable ability.  This criterion is never used, never explained, and should be deleted from the Handicap System.

4.  Can a case be made for eliminating the stroke allocation procedure for match play?

In matches between players of similar ability (i.e., a 1 to 2 stroke difference in handicap), the difference in average scores is minimal—maybe around .1 strokes at the most.   Therefore, wherever the handicap stroke is assigned, it is expected to be a winning stroke.  If most matches are between players of comparable ability, the USGA’s allocation procedure will not increase the number of halved holes.  Strokes could just as well be allocated randomly and the results of the matches would not be changed significantly.[7]  

The USGA’s match play recommendation creates other problems.   Often the number one stroke hole is a moderately easy hole.  The handicap chairmen must then explain—often and repeatedly—the arcane procedure recommended by the USGA.  In many cases a tough hole will get a high stroke allocation.  This leads to the player with the higher handicap (i.e., he gets a stroke) carrying the load on the difficult holes in a four-ball stroke-play competition.  The USGA states the Committee may develop a separate allocation based on difficulty relative to par to handle stroke-play competitions.   This would be adding more complexity to the scorecard without any benefit. 

The match play allocation procedure recommended by the USGA should be eliminated.[8]  In its place, the USGA should suggest a ranking of holes by difficulty subject to certain guidelines such as spreading low stroke holes evenly over the 18 holes.[9]  This can be done by consensus of the Handicap Committee.  The USGA procedure of evaluating numerous scorecards implies a level of scientific certainty in the outcome that is not justified.  As long as the recommendations on the placement of low strokes are followed (Sec. 17-1bii), one set of allocations should serve both for match and stroke play.[10]




[1] An early description of the procedure was presented in “New Handicap Surveys May Change the Stroke Allocations at your Course,” Golf Digest, Trumbull, Connecticut, July 1985, pp. 32-33.  The allocation procedure was described in the 1994 USGA Handicap System.  The same example used in 1994 appears in the USGA Handicap System of 2012.
[2] The USGA Handicap System, 2012-2015, The United States Golf Association, Far Hills, NJ, 2012, p. 103.
[3] Ibid, p. 107.
[4] The main reason the 22-handicap loses is that the average scores presented in Table 1 are not consistent with the theory behind the Slope System.  The average score of the 6-handicap is 74.56.  The average score of the 22-handicap is 98.37.  If the average score of the 6-handicap is correct, the average score of the 22-handicap should be 93.63 (75.56 + 1.13·16).  As it is, the average score difference is approximately 23 strokes, yet the 22-handicap only receives 16 strokes.  It is also hard to believe a bogey player (a 22 handicap by the USGA’s definition) averages about a triple bogey on hole 18. If this section is kept, the USGA should employ a more realistic example.  
[5] Course Management-Golf Australia Recommendations, Golf Australia, Melbourne, Australia, June 1, 2014.
[6] Op. cit., p. 103.  This recommendation is in conflict with the USGA’s previous assertion that the first handicap stroke should be assigned to a hole where a higher-handicap player most needs a stroke as an equalizer.
[7] Dougharty, Laurence, “One Set of Stroke Allocations is Enough,” www.ongolfhandicaps.com, July 24, 2013.  Dougharty reported that changing the stroke allocations had little effect on the outcome of the match.
[8] If the USGA decides to keep both allocation methods, it should illustrate each method with the same data.  This would give the reader some idea of the differences between methods.  Sec. 17-5 was added to the Handicap System, but used different data than that used to illustrate the allocation method set forth in Sec. 17-2.
[9] A sensible set of guidelines is presented in Appendix G of the Council of National Golf Union’s Unified Handicap System, 2012.
[10] To judge the efficacy of its stroke allocation procedure, the USGA should undertake simulation studies similar to those conducted by Dr. Francis Scheid, a past member of Handicap Research Team.  Scheid took scores from a golf club and used a computer to simulate matches between players.  This technique could be employed to determine if using the “average difference in scores” or the “difficulty to par” for stroke allocation has a significant impact on the outcome of a match.

Friday, December 5, 2014

The Pacific Amateur Golf Tournament: If you go…

The Pacific Amateur (Pac-Am) golf tournament is held in Bend, Oregon every year.  It is run by the Central Oregon Visitors Association (COVA) to promote tourism in the shoulder season between golf and skiing.  If you go, you can expect good golf courses, good food, and a great selection of ales from Bend’s many breweries.  What you cannot expect is an equitable competition.

The Pac-Am attracts approximately 400 golfers each year and they compete in flights based on age, sex, and handicap.   The typical flight has 24 players.  After three rounds, the top four players in each flight compete in the finals at the Crosswater Course at the Sunriver Resort.

With so many golfers, it is likely that a few will not be in strict adherence to the USGA Handicap System. The Pac-Am assures its participants that they will have to pass a handicap background check before they are allowed to enter.  Here are a few warnings from the Pac-Am website: 

Any unusual posting procedures, such as ceasing to post scores for a period leading up to the event, may be grounds for disqualification.  Any participant found using a fraudulent Handicap will be expelled from the tournament and all fees will be forfeited.

Participants in the competitive net divisions will compete using an assigned Tournament Handicap.  The participant will need to provide a complete scoring history to the Handicap Committee.  This handicap is calculated using one to two methods.  The committee may assign a handicap from a previous month if it deems the participant’s historical handicap is a better representation of scoring potential.  (Alternatively), the Tournament Handicap can be calculated by using past Pac-Am tournament scores and scores from other tournaments.  Tournament scores (especially those from the Pac-Am) are weighted more heavily in the calculation to help prevent golfers from having “repeat career round” during tournaments.

The vetting procedure sounds so strict that no sandbagger could make it through.  In truth, the Pac-Am’s policy is essentially a paper tiger.  With a limited staff, the Tournament Committee cannot check the bona fides of all entrants.  Basically, if your entry check clears, you are in.
    
The 2014 Pac-Am demonstrated the Tournament Committee’s inability to conduct an equitable tournament.  In this tournament, the same player was a repeat winner of the over-all net competition. The Bend Bulletin (September 26, 2014) in an act of journalistic naiveté extolled the accomplishment.  Asked to explain his unlikely performance, the player said  “I really can’t explain it.  I love the course and I think playing four, five or six days in a row just helps me swing better.”  The Tournament Committee was equally impressed and featured the winner’s picture on its website along with an article on the remarkable accomplishment of being the first repeat winner in the Tournament’s history.

No one seemed to ask the obvious question “Is the player’s handicap legitimate?” [1] Though the player was from Washington State, he did not have a handicap with any golf club belonging to the Washington State Golf Association.  When this was brought to the attention of COVA, it responded:

We are happy to provide you with (the player’s) USGA handicap card, with his index dated 9/15/2014, issued by his home course.

COVA was incorrect when it wrote the player’s index was issued by his home course (emphasis added).  His index was issued by an affiliate club (i.e., no course) through a company named MyScorecard.  MyScorecard sells a handicapping service for $14.95 a year.   Supposedly clubs formed under the aegis of MyScorecard have peer review.  But this is in theory only.  In practice, I bought a MyScorecard index that had no relationship to my potential ability.  I never received a query from my alleged handicap chairman who could possibly be a figment of someone’s imagination.
  
Even though the lack of peer review makes it is easy to cheat with MyScorecard, it does not necessarily follow that this player cheated.  To determine the authenticity of the index, it is necessary to examine his posted scores.  Here is what that examination revealed:

1. The player never posted a score from Crosswater even though he shot two net 66s in winning the Pac-Am finals in 2013 and 2014.
2. He never posted any round as a T-Score even though the Pac-Am Tournament Committee was supposedly to weight these scores heavily in arriving at the player’s tournament handicap.
3. The player played at a higher index in 2014 than in 2013 even though he won the tournament in 2013.  Apparently the Pac-Am Tournament Committee is not as vigilant in adjusting handicaps based on past performance as its website claims.

The missing 2013 tournament score (i.e., had it been posted it would have been in the last twenty scores and used in calculating the player’s index used for the 2014 tournament) means the player’s handicap was fraudulent and he should have been stripped of his title as the Pac-Am rules stipulate.  This did not happen.  COVA continued to defend the player’s performance as not extraordinary.  By any measure, however, the performance was extraordinary.  First, the player had to finish in the top four of his flight.  The probability of doing this is 1/6.  Then he had to win the finals against approximately 40 competitors.  The probability of doing this is 1/40.  The probability of doing this twice in a row is 1/57,600.  Even the probability of a defending champion repeating is 1/240—i.e., highly unlikely.

COVA did not hand down a harsh verdict on this player’s performance, and the reason is fairly clear.   To do so would reveal handicaps are not getting the close scrutiny the Pac-Am claims.  Prospective players could be turned off and participation could be reduced if they believed the tournament was not an equitable competition.   Since COVA’s real purpose is to fill motel rooms (from which it gets a percentage of the Transient Occupancy Tax), advertising the misconduct of the player and the ineffectiveness of its staff would not be in its interest.  Best just to forget about it and hope the player does not come back and “threepeat.”














[1] Dean Knuth, former Senior Director of Handicapping of the USGA, did pioneering work on the probability of exceptional scores.—see www.popeoftheslope.com.  Knuth recognized the weakness of the USGA’s handicap system in catching flagrant sandbaggers.  He invented the Knuth Point System where a player’s handicap is reduced based on what he wins and not on what he scores. 

Sunday, November 16, 2014

The Randomness of Course Ratings

Course and Slope Ratings are determined through a process that measures the yardage and obstacle values of a course.   The yardage of courses can be measured with some precision.  There are some judgments (e.g., roll, elevation) in measuring effective yardage, but any errors will be relatively small.  It is in the measurement of obstacle values where the greatest chance of random errors occurs. 

Errors in measuring obstacle values arise from the lack of precision in defining obstacles, confusing standards for rating each obstacle, model errors, and differences in subjective judgment among Rating Committees:

Lack of Precision in Defining Obstacles - The size, firmness and shape of a green in relation to the length of the approach shot is one obstacle.  The three characteristics -- size, firmness, and shape—of the obstacle are not defined with any specificity.[1]  If it is not clear what is to be measured, a lack of precision in the estimate is ensured.
Confusing Standards The Rating Committee must assign a value between 0 and 10 to each obstacle. The ratings criteria are both confusing and lack specificity.   Obstacle values are increased, for example, if a green is in poor condition or a player’s stance is moderately awkward.  Much of the confusion arises because the obstacles are not independent.  “Trees,” for example, present their own obstacle but also have an impact on the “fairway” and “green target” obstacle ratings.  It is not clear in the Course Rating Model how the independent effect of “trees” should be evaluated. [2]
Model Errors - The Psychological Obstacle Value is determined by the value of the other nine obstacle values.   This covariance among variables (Obstacle Values) leads to errors in the estimate the Scratch and Bogey Obstacle values. 
Differences Among Rating Committees - It is likely some Rating Committee members will weigh obstacle differently.  Given the subjective nature of the ratings process, this is a foregone conclusion.

To examine the “randomness” hypothesis the Course and Bogey Ratings of a course have been studied over the past rating cycle.  The course has had no change in its yardage, and there has been no significant change in the rated obstacles between ratings.[3] The changes in the men’s ratings are presented in Table 1.

Table 1
Change in Men’s Ratings

Tees
CR Old
CR New
Difference
BR Old
BR New
Difference
Gold
65.0
65.1
+0.1
86.4
85.7
-.6
Silver
68.4
68.2
-.0.2
90.9
91.1
.2
Green
70.7
71.0
+0.3
95.4
95.9
.5
Black
73.8
73.7
-0.1
99.6
100.1
.5

The differences in Course Ratings can be described as random.  That is, the course did not get uniformly tougher or easier for the scratch player or the bogey player.  Instead, the course was judged to be more difficult from two sets of tees and easier from two sets of tees for the scratch player.  For the bogey player, the first set of tees (gold) is rated easier while the remaining tees are rated more difficult.
A similar random pattern is shown in the change of ratings for women as shown in Table 2.  The new Course Ratings are higher from the gold and silver tees, but lower from the green tees.  For the bogey player, the course is now rated more difficult from the gold and green tees, but easier from the silver tees.     

Table 2
Change in Women’s Ratings

Tees
CR Old
CR New
Difference
BR Old
BR New
Difference
Gold
70.1
70.4
+0.3
100.6
101.1
+0.5
Silver
73.3
73.4
+0.1
107.3
107.1
-0.1
Green
77.6
77.1
-0.5
112.5
113.2
+0.7

The random variation probably stems from the subjective nature of the ratings procedure.  For example, assume the new obstacle ratings for topography are higher than the old obstacle ratings by one point on each hole.  Further assume the old and new ratings are identical for the nine other obstacles.  The increase in the obstacle value for the scratch player and in the course rating would be 0.2 strokes (1 x 0.1 x 18 x 0.11 = .198 rounded to .2).[4]  A similar error would lead to an increase of the bogey rating of .6 strokes (1 x .12 x 18 x .26 = .56 rounded to .6).  The Slope Rating would increase by 2 points ((.6 - .2) x 5.381 = 2.15 rounded to 2.0).  In essence, it only takes a small difference in the subjective ratings, rather than real changes in the obstacles, to lead to the small Course Rating changes shown in Tables 1 and 2.

The USGA could argue the systematic error in the measurement of topography described above is unlikely.  The error is more likely to be random with one hole being rated too high and another being too low.  The net result would be a much smaller change in the obstacle value.  There are two problems with this defense.  First, reliance on random errors discredits the measurement process—i.e.,”errors will cancel out” is not a rigorous defense for the Course Rating System.  Second, random errors do not always cancel out.  The 18-hole total of the weighted obstacle values only has to differ by 2.0 to get a 0.2 stroke change in the Course Rating.  A difference of 2.0 is not unlikely given the variance in the rating of individual obstacles.  For example, if the rating was 3 for each obstacle, the weighted obstacle value of the course would be 54.  A difference of 2.0 would only be an error of approximately 4 percent.   Such a small difference should be within the 95 percent confidence interval of the estimate of the total weighted obstacle value.  Therefore, small changes in the ratings are more likely due to the “randomness” of the rating process than to any physical changes in the course.      

The importance of random errors in the measurement process raises two questions for the USGA and local golf associations to consider.  First, if the re-rating results in small and apparently random differences from the old ratings, should the ratings be changed?   Unless the Rating Committee can point to physical changes that caused the differences, the prudent course would be leave the ratings unchanged. [5] After all, there are some costs (new scorecards, player confusion) in making changes to the ratings.   Second, are the required periodic re-ratings the best use of a Rating Committee’s time?   It would be more efficient and effective to re-rate a course if 1) its ratings seem out of line (e.g., visitors score higher or lower than expected or team performance is exceptionally good or bad) or 2) the club professional believes there have been significant alterations in the course since the last rating.  To rate for ratings sake better serves the bureaucratic interest of golf associations, but is not the most effective method for ensuring the equity of the handicap system. 



[1] USGA Course Rating System: 2012-2015, United States Golf Association, Far Hills, NJ, 2012.
[2] Op. cit., p. 27.
[3] A tree was removed from one fairway.  The tree was not an obstacle for the scratch player, and only affected the bogey player when he played from the green or black tees.  
[4] USGA Course Rating System: 2012-2015, p. 72.  The weight for the scratch topography obstacle is 0.1.  The sum of the weighted obstacle values is multiplied by .11 in the formula for the course obstacle value.
[5] Golf associations rarely explain why ratings have changed.  This is due in part to a lack of understanding of the USGA’s Course Rating Model.  Numbers from the field are entered into the model, and then the model produces Course and Bogey Ratings. This makes it difficult for the association to make a defense of the ratings based on physical changes in the course.  Instead, associations will defend the “process,” but not identify physical changes that led to the new ratings.  

Thursday, October 23, 2014

Can the USGA Slope Rating Decrease as Yardage Increases?

There are cases where the USGA Slope Rating decreases as the yardage increases.   It is never clear, however, whether this anomaly is due to an error by the Rating Committee or an oddity in the course design.  This post examines one such situation to determine the most likely explanation.
Our example is drawn from the files of the Oregon Golf Association (OGA) in 2006.  The yardages and Slope Ratings of the course in question are shown in Table 1 below.

Table 1
Yardages and Slope Ratings for Women

Tee
Yardage
Slope Rating
Green/Silver
6249
150
Green
6559
148

As Table 1 shows, the Slope Rating decreases as yardage increases.  A player with a 14.0 Index would receive 19 strokes if she played the Green/Silver tees, but only 18 Strokes if she moved back to the Green tees.   An examination of the Slope Rating formula reveals why the decrease in Slope Rating is unlikely.

                Slope Rating = 4.24·((Y/120 + BOV +51.3) – ( Y/180 + SOV +40.1))
                                    = .0118·Y + 4.24·(BOV –SOV) +47.5
Where,
                             Y = Effective Course Yardage
                       BOV = Bogey Obstacle Value
                       SOV = Scratch Obstacle Value

The Slope Rating is an increasing function of yardage.  For each 100 yard increase in yardage, the Slope Rating—all things being equal—will increase by approximately 1.2 rating points.  For the 310 yard increase in length in the example, the Slope Rating would increase by 3.7 rating points.  The only way for the Slope Rating to increase with distance is for the SOV to increase more than the BOV.  It is difficult to conceive of a course design where this could happen.

Where an error may have occurred can be found by examining the SOVs and BOVs implicit in the new ratings.  Table 2 shows the BOV for the Green tees is actually lower than from the Green/Silver tees. This is highly unlikely.  The bogey player should have shorter approach shots from the Green/Silver tees which should reduce the BOV.  If the shorter tees bring more hazards in to play, the bogey player can just use less club so that her landing area is the same as from the longer tees (i.e., there is no change in obstacle values). 

Table 2
Obstacle Values

Tees
SOV
BOV
Green/Silver
0.5
7.2
Green
1.1
6.5

The case for an error in ratings is strong.  The OGA, however, did not see it that way. 
Jim Gibbons, Executive Director, of the OGA wrote the following:

We have received your review of the course ratings. Nancy Holmes has
started the process to double check our ratings, but initial review shows we
are correct.  The bogey rating for women from the Green/Silver tees of 110.6
slopes to a 150 based upon the course rating of 75.3 at 6249 yards.  This
relates to a 19 handicap to shoot(sic) a net 75.3 for women.
The Green tees at 6559 have a course rating of 77.6 with a bogey rating of
112.5 (because some obstacles are located with less impact).  This provides a
handicap of 18 to shoot (sic) the net score of 77.6.
The rating process has changed since the previous ratings were done. Green
speeds and rough heights may be different. A check of the BOV between the
Green/Silver and the Green (tees) will be made and if there is a change that will
be posted.  Realize that the BOV sometimes is lower from the longer tees, but
many organizations will adjust the numbers to avoid having to explain the
reason.  We choose to do the rating correctly. 

Mr. Gibbons does not really give a defense of the Slope Ratings. He is correct that the various course and bogey ratings yield the peculiar Slope Ratings. This math was never in question. The issue was whether the Obstacle Values were estimated correctly. He states this will be addressed in the future, but never did. Gibbons ends with the audacious claim that the BOV is frequently lower from the longer tees than reported because other associations fudge the numbers. I suspect he never filed a complaint to the USGA to that effect.

The course was re-rated in 2014. The new BOVs and Slope Ratings are shown in Table 3. Both the BOV and the Slope Rating increase with distance as expected. The difference in the Slope Ratings between the two sets of tees is 7 rating points (+2 to -5). Since the course was essentially unchanged between ratings, a difference of 7 rating points is too large to be attributed to random error. Given the more reasonable 2014 ratings, it is likely that the 2006 ratings were due to Committee error rather than course design.

Table 3
2014 BOVs and Slope Ratings

Tees
BOV
Slope Rating
Green/Silver
6.7
147
Green
7.2
153
 
While the 2014 ratings are more sensible, it is difficult to prove they are more accurate than the ratings of 2006. Rating Committees take measurements, assign number to subjective judgments, and plug those results into a model that has never been empirically verified. Ratings come out of the computer and are sent to the golf course. You can’t argue with a rating, but only the logic behind the rating. Since the OGA never presents that logic, there can be no debate.

Thursday, October 2, 2014

Why Does the USGA Treat Women Differently?


The USGA does not have an enviable record when it comes to its view of women:
  1. The USGA has a “separate but equal” handicap system for women that codifies them as the weaker sex. 
  2. While the United States Tennis Association provides equal prize money for men and women at its Opens, the USGA awards more than twice as much prize money to men than women at its Opens.
  3. No woman has ever served on the USGA’s Handicap Research Team.
  4. In recommended handicap allocations, the USGA seems to presume women and men are different psychologically (i.e., men are bigger risk takers and women are more conservative on the golf course). 
The first two actions of the USGA can at least be defended on physiological and economic grounds.  The third result could stem from no women wanting to work in area where the possibility of publishable research is nil.  It is the fourth action—the USGA’s perceived difference is the psychological make-up of men and women –that is the subject of this post.


In many competitions, the USGA recommends different handicap allowances for men and women.  For example, in four-ball stroke play men are allowed 90 percent of their handicap while women are allowed 95 percent of their handicap.  Why are women treated differently?  Much of the USGA’s research on multi-team events was done over 35 years ago and there appears to be no mention of any differences due to the gender of the player. [1]   It is likely the USGA had no empirical evidence for the women’s allocation, and the percentage was just a consensus guess by members of the Handicap Procedure Committee.   If women were studied, it is probable any difference in the estimated optimal allowance for men and women was not statistically significant.  Remember, all of the studies used to justify four-ball allowances were completed long before the introduction of the Slope System.  With this error and others, it is likely any difference as small as five percent was not significant.  Since the USGA does not release its research for peer review, the accuracy and validity of the USGA’s allowance may never be known.  

The typical reason given for reducing handicaps in multi-ball events is that the higher handicap player has a larger standard deviation in his/her scores and hence an advantage.  Given that women get a smaller reduction in handicap, the USGA must believe women have as smaller standard deviation in their scoring.  Women must be steadier and/or less prone to risk taking as noted above.  In the appendix below, it is shown that the difference in standard deviations between teams is the same regardless of gender.  Therefore, it is difficult to defend the different allocations based on differences in standard deviations of scoring.[2]

While the allocations should be reviewed and revised, it is doubtful the USGA will take such action.  The allocations were never based on sound science, but rather on the internal politics at the USGA.  The allowances are considered “settled law” by the myriad of attorneys that guide the USGA.  To make a small step toward the equal treatment of women, however, the USGA could keep the hallowed men’s allowances and simply eliminate any allowance specific to women. 



[1] Ewen, Gordan, What the Multi-ball Allowances Mean to You, www.usga.org, Far Hills NJ, 1978.  The USGA has not released the original research for peer review. 
[2] The USGA has the data to examine if there are differences in scoring patterns between men and women.  It has chosen not to do so.
    


Appendix

The Slope Handicap System assumes that the standard deviation of a player’s scores increases linearly with handicap.  The standard deviation for each gender would be:
1)            σ (m,h) = σ(m,0)·(1 + a·h)
                                σ(m,h) = standard deviation of a male player with handicap h
                                σ(m,0) = standard deviation of a scratch male player
                                          h = handicap of the player
2)            σ (f,h) = σ(m,0)·(1 + b·h)
                                σ(f,h) = standard deviation of a female player with handicap h
                                σ(f,0) = standard deviation of a scratch female player
                                        h = handicap of the player
The USGA assumes that the line plotting average scores versus handicap would have a slope of 1.13.  The equation for males reflecting this assumption is:
3)            1.13 = (Average Score(h) –Average Score(0))/h
If a normal distribution of scores is assumed, then:
4)            Average Score(h) = ATBD(h) + .8 ·σ(m,h)
Where,
                                ATBD(h) = Average of Ten Best Differentials of a player with a h-handicap
Substituting eq. 4 into eq. 3:
5)            1.13 =(ATBD(h) + .8·σ(m,0)·(1 +a·h)  - (ATBD(0) + .8·σ(m,0))/h
Since,
6)            h = ATBD(h)·.96
Eq.  5 can be rewritten as:
7)            1.13 = (h/.96 +.8·σ(m,0)·(1 + a·h) – .8·σ(m,0))/h
                1.13 = 1.04 + .8·σ(m,0)·a                               
Since the same equality must hold for women, it follows that:
8)            σ(m,0)·a = σ(f,0)·b
Using eq. 8, the equations for the standard deviations can be rewritten as:
9)            σ(m,h) = σ(m,0) ·(1 +a·h)
10           σ(f,h) = σ(m,0)·(a/b) (1 +b·h) = σ(m,0)·(a/b +a·h)
For simplicity, assume we have a team where both players have a handicap of h1, and another team where both players have a handicap of h2.  The difference in the average standard deviation of the two teams is:
11)          Average Male Difference = σ(m,0)·(h1 - h2)
12)          Average Female Difference = σ(m,0)·(h1 - h2)
Therefore, the difference in average standard deviation between teams is the same regardless of gender (i.e., any advantage a team has is the same for both genders).  This finding makes it difficult to justify different handicap allocations for men and women in four-ball stroke play.







 






[1] Ewen, Gordan, What the Multi-ball Allowances Mean to You, www.usga.org, Far Hills NJ, 1978.  The USGA has not released the original research for peer review. 
[2] The USGA has the data to examine if there are differences in scoring patterns between men and women.  It has chosen not to do so.