Friday, February 12, 2021

USGA Cover-Up for the World Handicap System

The World Handicap System defines the difficulty of as:[1]

              Hole Difficulty – Scratch Value + Bogey Value – 2 x Par

This equation implies holes are rated for the scratch player and the bogey player.  For example, a hole could be rated 4.2 strokes for the scratch player and 5.3 strokes for the bogey player. However, an examination of previous USGA Course Rating Manuals[2] and a description of the USGA Course Rating Model by its creator[3], Dean Knuth, does not reveal how holes are rated for difficulty.  The USGA Course Rating Model does indicate how “courses” are rated, but not “holes.”  The USGA’s Course Rating equation is:

              Course Rating – Effective Distance/220 + 40.9 + Scratch Obstacle Value (SOV)

Course Rating Committees does not assign a stroke value to the obstacles on each hole.  Instead, the process goes as follows:

  1. The Rating Committee assigns a value (usually between 0 and 10) to each of the ten obstacles for every hole.
  2. These values are weighted in accordance with importance (e.g., for the scratch rating use topography .10, trees .09, etc.)
  3. The sum of the weighted values and multiplied by .11.
  4. The SOV is the product found in step 3 minus 4.9.

If this sounds more like a cookbook recipe than an empirically based econometric model, that is because it is. But put that aside.  The USGA model will at least give consistent, if not accurate, estimates of the SOV.  That is, courses with similar obstacles will get closely the same SOVs.  The USGA model as described above will not, however, yield a stroke obstacle value to each hole.

So how are ratings for each hole determined?  The question was put to Scott Hovde, Director of Course Rating and Handicap Education, at the USGA. Below are excerpts from the email exchange:

OnGolfHandicaps: Appendix E of the Rules of Handicapping gives an example of how playing difficulty is determined.  In the example, the scratch value of a par four hole is 4.2 as "provided by the Course Rating procedure."...Can you direct me to a source that details how such scratch values are determined?  

 

Hovde: A scratch value for a hole would be based on the combination of a (sic) the length rating and the obstacle stroke value.  The length rating has not changed in a while (other than name, which used to be yardage rating)...It is probably listed as an 18-hole formula, so would have to be scaled down to a single hole, but a hole over 420 yards for men (effective playing length) would have a length rating of 4.2 or greater.  The obstacle stroke value can add or subtract from that, but on average it adds to it.

 

OnGolfHandicaps: Your example implies there is a formula for converting hole length into the scratch and bogey values mentioned in Appendix E.  Can you cite a reference that explains how the course distance rating is scaled down to a single hole?

 

Hovde has not replied to this request for details on how hole estimates are actually made.  Equally disappointing were responses from the Southern California Golf Association (SCGA) and the Oregon Golf Association (OGA).  Doug Sullivan, Director of Course Rating at the SCGA, has published hole ratings but has tacitly refused to explain his method for doing so.


Gretchen Yoder, Manager of Handicapping and Course Rating at the OGA, wrote the following:


I guess the best way to think of it is separately for both Scratch and Bogey, the number of the full shots, plus the distance of the shots to the green and then +/- any difficulty in addition to yardage.

 

Yoder’s description is not definitive of the method.  How does a 140-yard distance to the green translate into an incremental hole rating?  And how is the +/- difficulty determined?

 

Why are the USGA and regional golf associations so secretive about the hole-rating process?  One explanation for the cover-up is the USGA does not want to expose the lack of scientific rigor in the Course Rating process. The USGA, for example, has never published any research indicating its hole ratings  predict scores with any accuracy.  The USGA thrives on a on a reputation of omnipotence (Chambers Bay aside).    To be forthcoming about much of its work on the handicap system is not in its own best interest.

 

 

 



[1] Rules of Handicapping, Appendix E, United State Golf Association, p. 98.

[2] USGA Course Rating Manuals, 21012-2015, 2016-2017.

[3] Knuth, Dean, “A two parameter golf course rating system,” Science and Golf, E & F Spon, London, 1990, pp. 141-146. 

Sunday, February 7, 2021

World Handicap System is Confused About Course Ratings

 

The World Handicap System (WHS) seems confused about what the Course Rating should measure.  In the WHS’s Rules of Handicapping (p.103) the Course Rating is defined as the expected score of a scratch player.  By “expected score” it is assumed the WHS means “average score.”  The WHS also defines a scratch player as one with a 0.0 Handicap Index (p.15).   To have a 0.0 Handicap Index a player’s low 8 differentials out of 20 must equal 0.0.  In equation form:

Avg. of 8 low differentials = 0.0 = ((Score1 – CR) + (Score2 – CR)…+ (Score8 – CR))/8 x 113/SR

                            CR = Course Rating

                            SR = Slope Rating

For this equation to hold. the average of the player’s 8 low scores must equal the Course Rating.  His expected score, therefore, would be higher than the Course Rating.  So, what is the Course Rating?  Is it a scratch player’s expected score of the average of his 8 best scores?

Complicating matters is the USGA Course Rating Model does not use either of the WHS’s definitions of the Course Rating.  That Model estimates the Course Rating as the average of the better half of scores of a scratch player’s latest 20 scores.  Has the Course Rating Model been updated to reflect the new definition (or definitions) of the Course Rating?  There is no evidence of any change in the Model.

Does any of this affect the accuracy of the Course Rating?  Probably not. A Course Rating is an imaginary number that cannot be measured like height, weight, and temperature.  The Model provides estimates of the Course Rating that should be consistent, but not necessarily accurate.  By that it is meant that if two courses are similar in distance and obstruction, the Course Ratings should be roughly the same.  The confusion about Course Ratings in the WHS will not affect handicaps, but does suggest the WHS needs to hire a better copy editor for its next edition.

Monday, January 25, 2021

World Handicap System's Stroke Allocation Method: Part I

 

The World Handicap System (WHS) proscribes a new method for determining how handicap strokes are allocated among holes.  Appendix E of the Rules of Golf states that each hole should be ranked by playing difficulty.  Hole difficulty is defined by:

              Hole Difficulty = Stroke Value + Bogey Value – 2 x Par

The Rules of Golf, however, does not define Stroke or Bogey Values or give any indication on how they are to be estimated.  It does say these values “can be determined objectively using hole-by-hole data from the Course Rating Procedure”.  Part II will examine if this assertion has any validity. 

The WHS removes the stroke allocation analysis from the club’s handicap committee and places it with the regional golf association (i.e., from those to regularly play the course to those who walk the course once every five years.  Part I examines the work of the Southern California Golf Association (SCGA) as a test of whether the switch to a regional association made for any improvement in the allocation process.  The SCGA prepared a Stroke Index Progression Report (5/18/2020) for two courses.[i]  The errors and inconsistencies in the report indicate removing the analysis from the handicap committee was problematical.  The troubling difficulties with the report are detailed below:

1. Mistake in Assigning Hole Difficulty - As shown in Table 1, the SCGA assigned different difficulties to the exact same tees.  For example, on hole 1 of Course 1 the tournament tees are the black tees, yet the Report shows different difficulties.  These errors do not affect the results but do indicate a lack of care in preparing the report.  A handicap committee would be familiar with the course and not make such a basic error.

Table 1

Hole Difficulty Errors 

Course

Hole

Tee

Difficulty

Tee

Difficulty

Course 1

1

Black

1.28

Tournament

1.24

Course 1

18

Tournament

0.77

White

0.80

Course 2

1

Black

1.53

Tournament

1.50

Course 2

2

Black

0.51

Tournament

0.41*

Course 2

18

Black

1.68

Tournament

1.64

 

* The difficulty was .51 on the Tournament Tee rating but was entered as .41 on the summary sheet.

 

2. The Weighted Hole Difficulties Are in Error - The report only shows the hole difficulty for four sets of tees.  When the hole difficulties are weighted by the number of rounds played, the results for the Course 2 differ from those shown in the report.  Table 2 below gives an example for hole 3 on the Course 2.

Table 2

Weighted Hole Difficulty 

Tee

Report Difficulty(D)

Number of Rounds(R)

Weighted Avg. D*R/8934

Black

0.48

1146

.06

Tournament

0.48

2838

.15

White

0.48

3549

.19

Green

0.27

1401

.04

 

 

Weighted Difficulty

.44

 

The weighted average is .44 and not .32 shown in the report.  Since the weighted hole difficulties for the Course 1 are correct, there must have been computational errors in the Course 2 calculations.  Why did not the SCGA double check its calculations?  Probably because it has to turn out hundreds of such reports and knew the recipient clubs would not the report’s accuracy.   It is doubtful that handicap committees concerned with only one course would be so cavalier, but that remains to be tested.

3. Inconsistencies in Course Rating Estimates - The sum of the Scratch Values should equal the Course Rating.  The sum of the Bogey Values should equal the Bogey Rating.  Then an equation for the Course Rating would be:

 

Eq. 1   Course Rating = ((2 x Par) + HD - Slope Rating/5.381)/2

Where,

                             HD = Hole Difficulty summed over all holes

The report provided hole difficulty for two courses with four sets of tees.  Table 3 shows the Course Rating as calculated by Eq. 1 and the SCGA Course Rating.

Table 3

Course Ratings

 

Course

Rating

Course 1

Black

Course 1

Tourn.

Course 1

White

Course 1

 Green

Course 2

Black

Course 2

Tourn.

Course 2

White

Course 2

 Green

Eq. 1

71.6

70.3

69.3

67.5

71.5

70.4

68.5

65.8

SCGA

71.0

69.6

68.7

66.9

71.2

70.0

68.4

65.7

Difference

0.6

.07

0.6

0.6

0.3

.04

0.1

0.1

 

Eq. 1 always overestimates the SCGA Course Rating.  The difference in Course Ratings at Course 2 could be due to rounding errors.  The differences at Course are too large to be explained by rounding.  The hole difficulties would have to be reduced by 1.2 strokes to eliminate the difference.  The inconsistencies in Course Ratings between the two methods may indicate Course Ratings are not the sum of hole ratings. If that is the case, the accuracy of the Scratch and Bogey Values are brought into question.

4. Incorrect Distance Measurements – The SCGA has rated the courses based on the yardages shown on the card.  Many of those yardages are in error due to the way the courses are setup.  The most egregious example is hole 5 of the Course 2.  The green tees are set alongside the red tees and not the white tees as specified on the card.   The misplacement of tees leads to errors in both the Course Ratings and anomalies in the stroke allocation (e.g., the relatively benign Hole 5 on the Course 2 is rated more difficult than Hole 2 where a hard green is surrounded by sand and water).   A handicap committee would be aware of the course set-up and would not make the same mistake.   

 

Conclusions on the efficacy of the WHS’s method cannot be drawn from a single example.  The example does support the contention Handicap Committees are better suited to doing the stroke allocation analysis because of their familiarity with the course and a greater sense of responsibility to doing it right.  It is likely, however, the analysis will remain in the hands of golf associations.  The WHS recommended method relieves the handicap committee from the arduous task of collecting scorecards and analyzing hole-by-hole scores.  Many committees will adopt this easier road believing the regional golf association cannot be wrong and if there is the blame it will be placed on the association and not us.   Part II will examine the accuracy of the WHS method and determine if there is another reason not to cede control of the stroke allocation process to the regional association.     



[i] Stroke Index is a new term introduced in the WHS.  Formerly, the USGA simply referenced the “The Allocation of Handicap Strokes.”  Other golf associations (Golf Australia and the Council of National Golf Unions) did use the term “Stroke Index.”  However, the Stroke Index is not an index.   Its use is both wrong and confusing with the more prominent Handicap Index. 

Tuesday, November 10, 2020

The Southern California Golf Association and Kevin O'Connor Can Do Better

 


A previous post (The World Handicap System Penalizes the Honest Player, July 21, 2020) discussed how the exceptional score reduction in Rule 5.9 of the World Handicap System (WHS) was unfair to the honest player.  The intentions behind the adoption of Rule 5.9 were good but misguided.  The WHS eliminated Section 10-3 of the USGA Handicap System that reduced a player’s Handicap Index for exceptional tournament scores.    While the USGA’s Section 10-3 was not effective, it was at least an attempt to ensnare the unethical player. 

The WHS needed something to replace the USGA’s efforts and thus Rule 5-9 was born.  Rule 5.9 differed from the old Section 10-3 in three important respects.  First, exceptional performance was no longer limited to tournament rounds. All rounds would be examined.  Second, only one exceptional round was required for an Index reduction.  Section 10-3 required two exceptional rounds.  Third the definition of what constituted an exceptional round was drastically changed.  Rule 5-9 requires a differential at least 7.0. below a player’s Index to be counted as “exceptional.”  Section 10-3 counted differentials of at least 3.0 below a player’s Index as “exceptional.” Neither attempt at controlling the unethical player was likely to be effective.  But that was not the point.  Rule 5-9 would give the illusion of cracking down on sandbaggers even though it was practically toothless. This is just another example of bureaucracies confusing motion with progress. 

When the WHS was announced, there was no defense of Rule 5.9.  That was wise since no reasonable defense exists.  That did not dissuade Kevin O’Connor, Director of Member Services at the Southern California Golf Association (SCGA), from trying to construct such a defense.  O’Connor was once Director of Handicapping at the USGA.  He has administered the handicap services of the SCGA since 2012.   You would think O’Connor would write with intelligence and insight on handicap issues given his experience.  Unfortunately, that is not the case.

O’Connor wrote an article on Rule 5.9 for the fall edition of Fore, the SCGA’s magazine.  His article exhibits three basic errors shown below in italics:

1.                   Part of the rationale for a reduction is research showing that someone doing this well (having a score differential 7.0 better than his Handicap Index) is likely to do so again in the future and this indicates the Handicap Index isn’t reflective of the player’s demonstrated ability.

Debaters without a good argument for their case often will resort to “experts say,” or as in this case “research proves” my position.   Somehow the experts are never identified, or the research cited.  This is because there is no research backing O’Connor’s claim.  The research that is available shows Rule 5.9 is both discriminatory and of little value.  It is discriminatory  according to the USGA research since high handicap players are 21 times more likely to have an exceptional score reduction than a low-handicap player.[1]  It is of little value because that same research indicates the rarity of such exceptional scores.   A midrange Index player has .003 chance of scoring a differential 7.0 or better than his Index.[2]   

2.                   The reduction makes the Handicap Index more statistically accurate.

The use of the term “statistically” is O’Connor’s attempt to indicate a highly sophisticated analysis behind Rule 5.9 where there is none.   Accuracy of a player’s Handicap Index is measured by the error in predicting a player’s next differential.  Arbitrary cut-offs for reductions in Rule 5.9 lead to decreasing rather than increasing accuracy.  For example, assume two players have the same Handicap Index.  In the next round, one player has a differential 7.0 below his Handicap Index while the other player has a differential 6.9 below his Handicap Index.   The penalized player gets a reduction of 1-2 strokes more than the unpenalized player.  The WHS generates Handicap Indexes that cannot be accurate (or equitable) for both players.

3.                   Scores posted after an Exceptional Score Reduction has been applied are not impacted.

While technically correct, O’Connor fails to mention a player’s Handicap Index can be affected long after an additional 20 scores are added to his scoring record.  For example, assume a player has a 16.0 Handicap Index and an Exceptional Score Reduction reduces his Index to 13.0 which is now his Low Index of the year.  If the exceptional score is an anomaly, the player’s Index will revert to 16.0.   Under Rule 5.8, any increase of 3.0 or more in his Index will be reduced by 50 percent and his Index will be capped at 18.0.  Such restraints on the player’s Index will last until the anniversary of his exceptional round.  

 

O’Connor ends by arguing Rule 5-9 “protects all players in relation to one another and results in a better Handicap Index more indicative of a player’s demonstrated ability.”  The examples above prove this statement is not true.[3]  O’Connor and the SCGA should educate and not indoctrinate golfers on the workings of the WHS.   Misleading articles such as O’Connor's weaken the credibility of the SCGA.  Both the SCGA and O’Connor can and should do better.

 

 



[1]  The USGA Handicap System 2016-2017, Appendix E.  A player if an index between 0-4.9 has a 1 in 2837 chance of having a net differential of 7.0 or better. A player with a 30.0 Handicap Index has a 1 in 133 chance.

[2] Ibid.

[3] The SCGA’s record in explaining the handicap system is not a good one. See “Southern California Golf Association Errs Badly,” ongolfhandicaps.com, May 10, 2017.

Saturday, October 31, 2020

World Handicap System - Clean-up on Rule 5.8

The World Handicap System (WHS) has been in effect for about a year.  If the WHS was a product, the manufacturer would be examining if it made handicaps more equitable, more accurate, and easier to use.  If history is any guide, golf's governing bodies are not making such an effort. Without the data within GHIN, independent analysis of the WHS cannot be undertaken.  It is possible, however, to theorize about the efficacy of any Rule.  This post examines Rule 5-8 Limit on the Upward Movement of a Handicap Index.  It recommends eliminating the Soft Cap  and adopting a simple  Hard Cap.  If the Soft Cap is to be kept, editorial changes are suggested to make the it more understandable and transparent.    

Eliminating the Soft Cap - Rule 5-8 limits the upward movement of a Handicap Index by employing Soft and Hard Caps.  The Soft Cap is triggered when a player’s calculated Handicap Index is 3.0 or more above his Low Index of the year.  The value above 3.0 is restricted to 50% of the increase.  There does not appear to be empirical evidence justifying the Soft Cap.  The adoption of this Soft Cap was probably based on two reasons.  First, it made the WHS complicated and to many players it would imply validity.  Second, since the WHS eliminated any handicap controls based on tournament performance,  the Soft could be defended as the WHS effort in controlling sandbagging. Neither argument is convincing. 

The Soft Cap would not be an effective instrument in controlling sandbagging..  The astute sandbagger  knows how to maintain his Low Index and will not let it drop due to a series of good scores. The Soft Cap, however, can ensnare the honest player.   Relatively modest increases (a region between +3.0 and +4.0) in a player's Index can be caused by random variations in scoring, slumps, changes in playing conditions (e.g. summer versus winter), and by a player manipulating his scores.  Anecdotal evidence suggests, a large percentage of Soft Cap reductions in this region are due to causes that should not be penalized.  Therefore, the Soft Cap will ensnare the honest player  and not advance the equity of competition.

The Table below shows a player's reduced Index under the WHS compared to only having a hard cap of 5.0.  There are two regions of interest.  For increases in Index between 3.0 and 4.0 there is only a minimal effect on a players handicap-- zero to one-stroke.  This is the rregion where the honest play will most likely to be penalized.  Adopting the 5.0 Hard Cap would eliminate this inequity.   

For increases in Index from 5.0 and on up, a Hard Cap of 5.0 leads to a slightly smaller reduction than the application of the Soft Cap.  For example if a player had a calculated Index increase of 6.0 the difference between the Soft Cap and Hard Cap reduction would only be 0.5. That is the Hard Cap is almost a severe a penalty on those players most likely to be manipulating their Index.  In essence, the Soft Cap under Rule 5-8 can be replaced by a simple Hard Cap of 5.0 with no significant effect on equity.

Under the Soft Cap a player can have an increased Index of 7. before reaching the Hard Cap comes into play.  Statistically such an  increase is so rare it should come to the attention of the Handicap Committee.  This was done under the old Golf Australia Handicap System.  That system  had a 5.0 Hard Cap, but it was but the authority to impose the Hard Cap was given to the Handicap Committee and not a WHS computer.  Data from  Golf Australia should be used to see if the simpler 5.0 Hard Cap worked sufficiently well that the Soft Cap can be eliminated from Rule 5.8. 

 

                                                         

Suggested Editorial Changes-

Notification of Reduction - Under the previous USGA Handicap System, a reduction in Index was signified by placing an "R" next to the player's Index.  Under the WHS, a player is not notified of a cap reduction with similar clarity.  The WHS places a small icon "!" next to a player's Index.  If the player notices the icon and clicks on it, a message will appear stating a cap reduction has been applied.  The player is not informed of the size of the reduction, however.  This lack of transparency could prevent a player from appealing the reduction to the Handicap Committee if he believed it unjust.  The WHS should follow the basic tenet of a good handicap system by fully informing a player about his reduction in Index. This will allow players to provide feedback on the perceived value of the soft cap in ensuring equitable competition. 

Terminology - Rule 5-8 states “When a calculated Handicap Index increase is greater than 3.0 strokes, the value above 3.0 strokes is restricted to 50% of the increase.” The use of the term “strokes” is a mistake in nomenclature and should be corrected.  Indexes and differentials are not defined by “strokes.”  A player has a 10.0 Handicap Index and not a 10.0 Stroke Handicap Index.[1]  To be correct, any association of “strokes” with a player’s Handicap Index or differential should be eliminated.

Rounding - Rule 5-8 does not detail how the 50% reduction is calculated.  For example, if the increase is 1.3, Rule 5.8 prescribes a 0.65 reduction to a player’s Handicap Index.  The Rule is silent on whether this reduction should be rounded up (0.7) or rounded down (0.6).  From a sample of scoring records, it appears the WHS rounds down.  This should be made explicit when Rule 5-8 is revised.

All of this assumes there will be a revised WHS.  The USGA and R&A have not indicated any timetable for revising the WHS.



[1] The USGA Handicap System did not use the term "strokes."  Golf Australia’s previous Handicap System did.  This is another example of Golf Australia’s heavy influence (e.g., counting 8 of 20 scores, daily course ratings, hard caps) in the construction of the WHS.