Friday, February 26, 2021

World Handicap System's Stroke Allocation Method: Part II

The World Handicap System (WHS) suggests the Stroke Index Allocation be determined by hole difficulty (Rules of Handicapping, Appendix E).  The WHS defines hole difficulty by the following equation.

              Hole Difficulty = Scratch Vale + Bogey Value – 2 x Par

As discussed in a previous post (USGA Covers-Up for the World Handicap System, February 12, 2021), the WHS has never defined how Scratch and Bogey Values are determined.  Nor has the USGA produced research indicating the accuracy of its methods in predicting difficulty as indicated by actual scores.  It has also been demonstrated that regional golf associations have made errors in implementing the WHS method (see World Handicap System’s Stroke Allocation Method: Part 1, January 25, 2021).  All of this indicates the WHS method is suspect and its validity questionable.

This post presents a preliminary analysis of the WHS method.  Because of the small sample size any conclusion cannot be definitive.  There is evidence the WHS procedure produces inaccurate and inconsistent results.  At a minimum, the WHS should review its recommendation with an eye to eliminating Appendix E in its entirety.

A sample of 41 rounds from two courses was used to assess the accuracy of hole difficulty estimates.  The players had course handicaps from 3 to 24.  The average handicap was approximately 14.  This is also approximately the average handicap of a scratch player and a bogey player.  Therefore, course difficulty determined from the sample was the average score on a hole minus par.  The rankings of the holes by using the sample were compared with that provided by the Southern California Golf Association (SCGA) using the WHS method.  The rankings are compared in the table below.

Table

Hole Difficulty

Course 1

Course 2

Sample

SCGA

Sample

SCGA

Front

9

9

8

8

5

5

9

9

6

7

4

1

1

1

6

7

2

3

2

6

3

6

3

4

7

8

7

3

8

4

1

2

4

2

5

5

Back

 

 

 

18

11

15

12

11

16

18

11

10

15

12

15

15

18

17

18

17

10

14

16

16

17

16

17

14

13

13

14

12

14

11

13

13

12

10

10

 

Most of the rankings on Course 1 are within two placements of one another.  The exceptions are holes 2 and 16.  At Course 2, holes 2, 4, and 11 differ by three or more placements.  The larger differences could be explained by either sample error or problems with the WHS method. 

While the exact WHS methodology is unknown, it appears to be largely a function of the yardage of the approach shot.  For example, the USGA assumes a scratch player drives the ball 250 yards at sea level and can hit a fairway shot 220 yards. On a 400-yard hole, the scratch player would have a 150-yard approach shot.  On a 500-yard hole, he would have a 30-yard approach shot.  The bogey player is assumed to drive the ball 200 yards and can hit a fairway shot 170 yards.  At the 400-yard hole he would have a 30-yard approach but would lie two.  On the 500-yard hole he would have a 130-yard approach. Let’s see if this explains any of the discrepancies in hole rankings.

Course 1, Hole 2:  The Sample rated this hole as the 5th hardest on the front nine.  The SCGA rated it as the 9th hardest.  As shown in the Appendix, the ratings by the SCGA are highly correlated with yardage—as it should be.  Hole 2 is only 125 yards.  Under the USGA’s assumptions, this is shorter than the approach shot the bogey player will have on any par 4 or par 5 on the front nine.  This is probably why the SGA rated it as the easiest.  The Sample ranking suggests the SCGA underestimated the obstacle value.  The tee shot on this hole is all carry over water.  The bailout area on the left of the green leaves a treacherous pitch to a hard green sloped toward the water.  The Sample ranking seems to be the better placement. 

Course 1, Hole 16:  The Sample rates this hole as the 6th most difficult on the back nine.  The SCGA rated it as the 2nd most difficult.  If the bogey player his a 200-yard drive, he would have a 152-yard approach shot (the effective distance is probably longer since the hole plays uphill.)  This is the longest approach shot (or par 3 tee shot) he would have on the back nine except for holes 11 and 15.  This would explain the SCGA’s high difficulty ranking.  The Sample ranks holes 18 and 17 above hole 16.  Both 18 and 17 both present obstacles (e.g., carry over water) that bring high scores into play.  Hole 16 does not have similar obstacles.  This indicates the SCGA d rating does not properly weight the obstacle value of a hole.  The Sample gives a more realistic estimate of hole difficulty.

Course 2, Hole 2:  This is a par 5 hole the SCGA ranks as the 8th most difficult on the front nine.  The hole is only 442 yards.  The WHS assume the scratch player can reach the hole in two shots.  The bogey player should have a 72-yard approach shot.  It is no wonder the SCGA rated it so easy.  The Sample ranks this hole as 5th hardest on the front nine.  The green is sloped toward the water on the right so holding the green is difficult for anything except a short iron.   A deep bunker on the left precludes playing it safe. Again, it looks like the SCGA has underestimated the obstacle value for this hole.

Course 2, Hole 4:  The Sample ranking is 3rd most difficult.  The SCGA outs it at 6th.   Since Sample ranking between the 3rd and 6th place is only 0.12 strokes, the difference in rankings could be due to sample error.

Course 2, Hole 11:  The SCGA ranks this hole as the 2nd most difficult on the back nine, while the Sample ranking is 8th.  The SCGA also ranks this hole as more difficult than the longer hole 15. SCGA must see obstacle values that are not experienced by the player.

The WHS rankings are also inconsistent.  The most difficult hole on Course 2 is Hole 12 with a difficulty rating of 1.29.  This is a par 5 with a yardage of 518 yards.  The other par 5 (Hole 17) on the back nine is ranked the easiest (difficulty rating of 0.67) and given the stroke allocation of 18. The only obvious difference between the two holes is Hole 12 is 25-yards longer.  Apparently, the SCGA believes this is the toughest 25 yards in golf.

In conclusion the WHS hole ranking procedure yields inaccurate, inconsistent, and curious stroke allocations.  There is no reason for the WHS to get into the stroke allocation business.  The mechanical tools given to the Ratings Committee are not adequate—whatever they may be.  Rating Committees do not have knowledge of the nuances of a hole that can make it more difficult.  Stroke allocations should be data driven and made by the Handicap Committee.  Given the electronic scoring being used at many courses, rating holes by difficulty would not be the laborious process it once was.  It is highly recommended that Appendix E of the Rules of Handicapping eliminate it current formula for stroke allocations.  

  

Appendix A 

Table A-1

Par 3 Difficulty Rating

Course 1

Course 2

Hole

Diff.

Yardage

Hole

Diff

Yardage

15

1.08

169

16

0.94

146

17

0.74

137

6

0.89

156

4

0.61

145

14

0.74

155

2

0.54

125

3

0.48

134

12

0.30

120

 

 

 

 

Table A-2

Par 4 Difficulty Rating 

Course 1

Course 2

Hole

Diff.

Yardage

Hole

Diff

Yardage

11

1.59

394

8

1.33

361

9

1.52

355

9

1.30

358

5

1.48

395

1

1.23

350

7

1.28

363

7

1.18

349

16

1.22

352

11

1.18

349

1

1.07

352

15

1.10

364

6

0.88

312

18

1.03

342

13

0.60

295

13

0.67

324

 

 

 

10

0.38

280

 

 

 

5

0.20

276

 

Table A-3

Par 5 Difficulty Rating

Course 1

Course 2

Hole

Diff.

Yardage

Hole

Diff

Yardage

3

0.90

498

12

1.29

518

8

0.87

507

17

0.67

493

18

0.80

486

4

0.50

473

10

0.77

493

2

0.27

442

14

0.59

488

 

 

 

  

Friday, February 12, 2021

USGA Cover-Up for the World Handicap System

The World Handicap System defines the difficulty of as:[1]

              Hole Difficulty – Scratch Value + Bogey Value – 2 x Par

This equation implies holes are rated for the scratch player and the bogey player.  For example, a hole could be rated 4.2 strokes for the scratch player and 5.3 strokes for the bogey player. However, an examination of previous USGA Course Rating Manuals[2] and a description of the USGA Course Rating Model by its creator[3], Dean Knuth, does not reveal how holes are rated for difficulty.  The USGA Course Rating Model does indicate how “courses” are rated, but not “holes.”  The USGA’s Course Rating equation is:

              Course Rating – Effective Distance/220 + 40.9 + Scratch Obstacle Value (SOV)

Course Rating Committees does not assign a stroke value to the obstacles on each hole.  Instead, the process goes as follows:

  1. The Rating Committee assigns a value (usually between 0 and 10) to each of the ten obstacles for every hole.
  2. These values are weighted in accordance with importance (e.g., for the scratch rating use topography .10, trees .09, etc.)
  3. The sum of the weighted values and multiplied by .11.
  4. The SOV is the product found in step 3 minus 4.9.

If this sounds more like a cookbook recipe than an empirically based econometric model, that is because it is. But put that aside.  The USGA model will at least give consistent, if not accurate, estimates of the SOV.  That is, courses with similar obstacles will get closely the same SOVs.  The USGA model as described above will not, however, yield a stroke obstacle value to each hole.

So how are ratings for each hole determined?  The question was put to Scott Hovde, Director of Course Rating and Handicap Education, at the USGA. Below are excerpts from the email exchange:

OnGolfHandicaps: Appendix E of the Rules of Handicapping gives an example of how playing difficulty is determined.  In the example, the scratch value of a par four hole is 4.2 as "provided by the Course Rating procedure."...Can you direct me to a source that details how such scratch values are determined?  

 

Hovde: A scratch value for a hole would be based on the combination of a (sic) the length rating and the obstacle stroke value.  The length rating has not changed in a while (other than name, which used to be yardage rating)...It is probably listed as an 18-hole formula, so would have to be scaled down to a single hole, but a hole over 420 yards for men (effective playing length) would have a length rating of 4.2 or greater.  The obstacle stroke value can add or subtract from that, but on average it adds to it.

 

OnGolfHandicaps: Your example implies there is a formula for converting hole length into the scratch and bogey values mentioned in Appendix E.  Can you cite a reference that explains how the course distance rating is scaled down to a single hole?

 

Hovde has not replied to this request for details on how hole estimates are actually made.  Equally disappointing were responses from the Southern California Golf Association (SCGA) and the Oregon Golf Association (OGA).  Doug Sullivan, Director of Course Rating at the SCGA, has published hole ratings but has tacitly refused to explain his method for doing so.


Gretchen Yoder, Manager of Handicapping and Course Rating at the OGA, wrote the following:


I guess the best way to think of it is separately for both Scratch and Bogey, the number of the full shots, plus the distance of the shots to the green and then +/- any difficulty in addition to yardage.

 

Yoder’s description is not definitive of the method.  How does a 140-yard distance to the green translate into an incremental hole rating?  And how is the +/- difficulty determined?

 

Why are the USGA and regional golf associations so secretive about the hole-rating process?  One explanation for the cover-up is the USGA does not want to expose the lack of scientific rigor in the Course Rating process. The USGA, for example, has never published any research indicating its hole ratings  predict scores with any accuracy.  The USGA thrives on a on a reputation of omnipotence (Chambers Bay aside).    To be forthcoming about much of its work on the handicap system is not in its own best interest.

 

 

 



[1] Rules of Handicapping, Appendix E, United State Golf Association, p. 98.

[2] USGA Course Rating Manuals, 21012-2015, 2016-2017.

[3] Knuth, Dean, “A two parameter golf course rating system,” Science and Golf, E & F Spon, London, 1990, pp. 141-146. 

Sunday, February 7, 2021

World Handicap System is Confused About Course Ratings

 

The World Handicap System (WHS) seems confused about what the Course Rating should measure.  In the WHS’s Rules of Handicapping (p.103) the Course Rating is defined as the expected score of a scratch player.  By “expected score” it is assumed the WHS means “average score.”  The WHS also defines a scratch player as one with a 0.0 Handicap Index (p.15).   To have a 0.0 Handicap Index a player’s low 8 differentials out of 20 must equal 0.0.  In equation form:

Avg. of 8 low differentials = 0.0 = ((Score1 – CR) + (Score2 – CR)…+ (Score8 – CR))/8 x 113/SR

                            CR = Course Rating

                            SR = Slope Rating

For this equation to hold. the average of the player’s 8 low scores must equal the Course Rating.  His expected score, therefore, would be higher than the Course Rating.  So, what is the Course Rating?  Is it a scratch player’s expected score of the average of his 8 best scores?

Complicating matters is the USGA Course Rating Model does not use either of the WHS’s definitions of the Course Rating.  That Model estimates the Course Rating as the average of the better half of scores of a scratch player’s latest 20 scores.  Has the Course Rating Model been updated to reflect the new definition (or definitions) of the Course Rating?  There is no evidence of any change in the Model.

Does any of this affect the accuracy of the Course Rating?  Probably not. A Course Rating is an imaginary number that cannot be measured like height, weight, and temperature.  The Model provides estimates of the Course Rating that should be consistent, but not necessarily accurate.  By that it is meant that if two courses are similar in distance and obstruction, the Course Ratings should be roughly the same.  The confusion about Course Ratings in the WHS will not affect handicaps, but does suggest the WHS needs to hire a better copy editor for its next edition.