Thursday, March 7, 2013

What a Difference an ESC Makes


Introduction - A previous post detailed the various equitable stroke control procedures (ESC) implemented by the United States Golf Association (USGA) over the past twenty years--See What Does the Equitable Stroke Control Procedure Mean for Golf? February 13, 2013.  This post presents an analysis of the impact of different ESCs on handicaps and the overall equity of competition.  A simulation model was used to estimate the effects of different ESCs on adjusted scores and handicaps.  It was found that the latest ESC (ESC97) reduced the advantage of the low-handicap player over the high handicap player.  This conclusion was affirmed in an empirical test using a small sample of players. 

Simulation Model - The first step in creating the simulation model was to estimate the probabilities of hole scores relative to par for golfers of varying ability.  These probabilities were estimated first by drawing from a small sample of scorecards, and then reviewing and adjusting these estimates after discussions with numerous players.  The results are dependent on these estimates.  The overall conclusions of this analysis, however, were found to be fairly insensitive to moderate changes in the estimated probabilities.  The probabilities for a low-handicap (5-handicap), a medium-handicap (15-handicap) and a high-handicap (25-handicap) are shown in Table 1.
Table 1
Probability of Hole Score Relative to Par 

Score Relative to Par
Low
Medium
High
-2
.005
.000
.000
-1
.140
.060
.010
0
.450
.250
.150
+1
.310
.430
.380
+2
.070
.200
.300
+3
.020
.040
.100
+4
.005
.020
.050
+5
.000
.000
.010


Each player had 1000 simulated rounds. His score was adjusted after each round under the three ESC formulas: 1) OLDESC, 2) ESC93, and 3) ESC97.  The players’ average score and adjusted scores are presented in Table 2.
Table 2
Average Scores 

Player Handicap
Average Score
OLDESC
ESC93
ESC97
Low
78.77
78.26
77.98
78.26
Medium
89.36
87.91
88.86
88.86
High
99.43
98.09
99.02
99.02
 

As Table 2 shows, the adoption ESC93 lowered the adjusted score of the low-handicap player (.28 strokes), but raised the adjusted scores for the medium-handicap player (.95 strokes) and the high- handicap player (.93 strokes).  After complaints by low-handicap players, ESC93 was dropped and ESC97 was adopted.  (ESC93 limited the low-handicap player to a maximum of 6-strokes, while ESC97 allowed him to take a double bogey on any hole).  This brought the low-handicap player back to where he was under the OLDESC.  The low-handicap player, however, still lost approximately one-stroke vis-à-vis the medium and low handicap player. 
Handicaps were computed after each 20 rounds.  The average handicap for each player under the three ESCs is shown in Table 3.
Table 3
Average Handicap (Course Rating = 70.0) 


Player Handicap

OLDESC

ESC93

ESC97
Low
5.34
5.14
5.34
Medium
14.60
15.19
15.19
High
23.92
24.52
24.52
 

Comparing handicaps under the OLDESC and ESC97, the low-handicap player lost approximately .6 stroke advantage in handicap to both the medium- and high-handicap player.

One measure of over-all equity is the difference between a player’s average differential (average score - course rating of 70,0) and his handicap. These differences are shown in Table 4.

Table 4
Difference in Average Differential and Handicap

Player Handicap
OLDESC
ESC93
ESC97
Low
3.43
3.63
3.43
Medium
4.76
4.17
4.17
High
5.51
4.91
4.91

As Table 4 indicates, the low-handicap player has a handicap closer to his average differential, and therefore has an advantage over the medium- and high-handicap player.  ESC97, however, has reduced the size of the advantage.  The relationship between average differential and handicap can be represented in equation form for each ESC procedure:[1]

OLDESC                Average Differential = 2.90 + 1.11 x Handicap

ESC93                    Average Differential = 3.25 + 1.07 x Handicap   

 ESC97                   Average Differential = 3.01 + 1.08 x Handicap

The equations show the adoption of ESC97 decreased the penalty against the higher handicap player by 3 percentage points.

Empirical Test - To test if the results of the simulation model  are confirmed by real world results, a sample of 21 golfers was used. This sample was not random.  It consisted of players known to the author and celebrity players such as John Paul Newport, golf columnist of the Wall Street Journal, George Peper, former editor of Golf Magazine, and Dean Knuth, former USGA Senior Director of Handicapping.  The average differential ((score -course rating) x 113/Slope Rating) of each player was regressed against their index (2/15/2013) as shown in the figure below. (Note: If the ESC equations above are multplied by 113/Slope Rating, they become identical in form to the equation shown in  the figure below).


This empirical test shows the low-handicap player has a 9.5 percent edge as opposed to the 8 percent edge predicted by the theoretical model.  Both the theoretical and empirical models, however, predict a smaller edge for the low handicap player than the 13 percent edge assumed in the development of the Slope Handicap System when OLDESC was in force.  Clearly, the small sample size limits the confidence in the estimate of the coefficient (i.e., .095).  Since ESC97 increased the cap on scoring for medium- and high-handicapper players,  a reduction in the edge given to the low-handicap player was inevitable.  Therefore, it is reasonable to conclude ESC97, as opposed to OLDESC, made competition more equitable, but not equitable. How golfers have reacted to this inequity will be the subject of a forthcoming post--In Defense of Sandbagging.

 

 



[1] These equations were determined by finding the trendline among three data points. 

Friday, February 22, 2013

Equitable Stroke Control - A Postmortem


          The previous post, What Does the New Equitable Stroke Control Mean for Golf?,  struck a responsive chord at Golf Digest.  Jerry Tarde, editor of Golf Digest, lifted many parts of the paper for his column entitled “A formula for slow play and higher scores.”[1]  Tarde used the material without either attribution or compensation (if you don’t count an autographed copy of a book he had ghost written for Sam Snead).
I did not take offense, however, for here was a major golf publication at least questioning the USGA.  I was further encouraged when Golf Digest implied it would be conducting an independent review of ESC93.  In a letter to Mr. Tarde, I urged:

"Golf Digest to demand to see the three studies the USGA claimed supported  ESC93. I believe you will find that while the USGA will trumpet the findings of studies supporting their position, they will not make these studies available for peer review.  I hope you will be made privy to these studies so Golf Digest could come to an independent and informed evaluation of the new ESC procedure. [2]

            I was to be greatly disappointed by Golf Digest’s effort.  Golf Digest assigned the story to Guy Yocom.  Yocom did not review any USGA study, and his article was simply a listing of unchallenged replies by Dean Knuth, Director of Handicapping for the USGA.[3]  Yocom’s article lacked insight and demonstrated an interview style better suited to People Magazine.  An example will demonstrate Yocom’s deficiency:

                        Question by Yocom: Since even low handicappers now can score a 6 on a par 3, won’t this be a boon for sandbaggers?

                        Knuth: No…Par 3s are where golfers receive the fewest number of handicap   strokes, where 99 percent of players are likely to try their hardest to score well.

            Now there are two problems with Knuth’s reply.  First, there is no reason to believe a golfer’s effort on a hole is related to whether he gets a handicap stroke.  Second, and more importantly, Knuth is charging that 99 percent of all golfers are unethical since they violate the basic precept of the handicap system that “every player will endeavor to make the best score he can at each hole.”[4]  Knuth’s defense rests on the lack of integrity of the American golfer.  Poor Yocom did not identify either problem.[5]
            Yocom did a follow-up article that was equally unsatisfactory.  He even made a small technical error in trying to explain why the ESC was changed.  When I pointed out the error, it was apparently too much for Yocom.  He wrote:

                       "Just a crazy, giddy guess on my part, but is there a Dealey Plaza (where you live)…Has your company conducted any research on the subject (ESC).  I would be glad to look it over, as you must have some basis for your persistent opinion opposing the new ESC."[6]

I had to respond in a letter to Jerry Tarde:[7]

                       "Though he does not write with great clarity, Yocom seems to liken me to Lee  Harvey Oswald.  I find this personally offensive and beyond the pale of  responsible debate.  It is unfortunate when a representative of your fine magazine does not bring intelligence or insight to an issue, but only personal invective.

                        Mr. Yocom claims that I oppose ESC.  He apparently had not read my paper.  What I oppose is change that does not represent progress.  I do not believe the USGA proved any substantive benefit from the new ESC…The new ESC may be a success as (the USGA) claims.  That judgment, however, should have rested upon an independent evaluation of the data Yocom had promised in March.  Yocom’s January article failed to deliver on that promise."

            The net result was Yocom still gets a press pass to the U.S. Open, and I canceled my subscription to Golf digest.
Despite the USGA claims of great support, ESC93 did not make many people happy.  The USGA was to try a different ESC in 1997.  Currently, ESC consists of a hybrid system.  Low handicappers are now allowed to take a double bogey on any hole.  This was in answer to the many complaints that taking a 6 on a par five was not equitable.  All handicaps of ten and above have the same limits as ESC93.  No research was ever published on why low handicap players should be treated differently than high handicapped players.  It appears the USGA Handicap Procedure Committee was only responding to a political problem with low handicapped players rather than seeking a just solution to the ESC problem.  Given the track history of this most political institution, such a resolution should come as no surprise.




[1] Jerry Tarde, “A formula for slow play and higher scores,” Golf Digest, Trumbull, CT, June 1993.
[2] Letter from the author to Jerry Tarde, January 15, 1993.
[3] Guy Yocom, “Why your handicap will change this year,” Golf Digest, Trumbull, CT, March 1993
[4] USGA Handicap Manual, p. 1
[5] Yocom has no technical background and could not review any studies if the USGA gave them to him.  His principle background is that of ghost writer.  He wrote a book with Corey Pavin on shotmaking shortly before Mr. Pavin’s career started to ebb.  Therefore, the lack of analysis and evaluation is the fault of those at Golf Digest who assigned Yocom to this inquiry.
[6] Letter to the author from Guy Yocom, December 15, 1993
[7] Letter from the author to Jerry Tarde, December 27, 1993.

Wednesday, February 13, 2013

What Does the New Equitable Stroke Control Procedure Mean for Golf?


(Note: This post is a reprint of an article first published in 1993.  It is posted here because 1) It will be referenced in an upcoming post,  and 2) It iilustrates the political nature of decisions on handicapping at the USGA--a recurring theme of this blog.)
Introduction - The United States Golf Association (USGA) adopted changes in the Equitable Stroke Control (ESC) procedure for adjusting scores in 1993.  The USGA maintained that the new procedure would increase the equity of competition, and should have no effect on playing time or the prevalence of sandbagging.[1]

The USGA's conclusions, however, appear to be based on questionable analysis and an unrealistic view of the golfing world.  A more probable outcome is that the new ESC (termed ESC93 in this paper) will not lead to any significant improvement in the equity of competition and might actually lessen it.  ESC93 procedures will undoubtedly increase the problem of slow play, and make life easier for the sandbagger.  To see why all of this will happen, this paper examines the old ESC and ESC93 in light of four criteria mentioned by the USGA in defending the change: equity of competition, simplicity, speed of play, and the effect on the practice of sandbagging.

Equity of Competition - The purpose of ESC is to prevent unduly high scores from having an abnormal effect on a player's Handicap Index.  The USGA presented three arguments for why the ESC93 procedure will increase the equity of competition.  Each is examined in turn:

1. Under the Old ESC (see Table 1 below) high-handicap players frequently were not able to post their average hole-score on difficult holes. 

An example of the inequity ESC93 was designed to correct would be a player with a thirty-four handicap who averages 8 strokes on a par four, but can only post a maximum of 7 strokes (i.e., a triple bogey) under the old ESC.  Under ESC93 he would be allowed to post a maximum hole-score of 9 (See Table 2).

           

Table 1

OLD ESC PROCEDURE


Course Handicap
Limitation on Hole Score
Plus or Scratch
Limit of one over par on any hole.
1 through 18
Limit of two over par on number of holes equal to Course Handicap.  Limit of one over par on balance of holes.
19 through 36
Limit of three over par on as many holes as course handicap exceeds 18 strokes.  Limit of two over par on balance of holes.
37 through 54
Limit of four over par on as many holes as the Course Handicap exceeds 36 strokes.  Limit of three over par on balance of holes.


Table 2

ESC93 PROCEDURE


Course
Handicap
Maximum Score Posted on Any Hole
9 or less
6
10-19
7
20-29
8
30-39
9
40-49
10
50+
11


            The size of any correction to equity will depend in part on the frequency with which a player's average score exceeds the maximum score permitted by ESC.  In our example, a thirty-four handicap player averages approximately two over par on the holes he plays.  A hole that would require 4 or more strokes over par on average should be a statistical rarity.  Therefore ESC93 would not appear to have a large impact on equity because it corrects for a problem that seldom occurs.[2]

            Even the small contribution to equity of ESC93 in stroke play may be outweighed by the inequity it could cause in match play.  The player in our example is now allowed to post a quintuple bogey on a par four hole.  To demonstrate a possible inequity created by ESC93, assume the player in the example has two holes where he always posts the ESC limit of 9-strokes, and that he is still a thirty-four handicap.  He will then average approximately 1.6 strokes over par on the remaining 16 holes (i.e., he plays like a 28 handicap).  Our player should have a big advantage over the player who earns a thirty-four handicap with a more even scoring pattern.

            In match play a score of 5 strokes over par should be considered "abnormal."  Any strokes over a triple bogey are irrelevant since the hole most probably would have been lost by then.  Allowing a player to post a nine would seem to be just the thing that ESC was designed to eliminate.  In effect, our player is rewarded because of his abysmal play on two holes.  This is not only inequitable, but it sets up an incentive system that is detrimental to promoting good golf. 

2. The old ESC depends upon the par of each hole, and par ratings are often set higher than they should be. 

To make the argument, the USGA gives the example of a hole under 401 yards for women that is set as a par five at one course but as a par four at another course.  This would result in different equitable stroke control limits because of the difference in par ratings. If this is a problem, it does not appear to be a compelling reason to change the old ESC system.  A simpler solution would be to require governing golf associations to approve the par rating of each hole in accordance with USGA guidelines.[3]

3. The old ESC causes unfairly low course handicaps just below the break-points.

            In fact, the ESCC93 procedure has the same problem, if not worse, as the old ESC procedure at the "break-points."  Let's examine the breakpoint problem under the old ESC procedure first.  Assume we have an eighteen and a nineteen-handicap -- one player above and below the breakpoint under the old ESC system.  The nineteen-handicap can post one triple bogey while the eighteen-handicap cannot post any triple-bogeys.  Now assume the two players have identical eighteen-hole scores and each player has a triple bogey.  The handicap difference of one stroke would remain even though the players had identical rounds.  While this is inequitable, it is also fairly insignificant.  Even in the unlikely scenario of our example, the handicap inequity is at most one-stroke.  In trying to correct this inequity, however, the USGA has only made matters worse.

            Assume a nine and a ten-handicap as an example of players at the breakpoint under ESC93.  If each player has the same score which includes a double-bogey on a par five, the nine-handicap would post a score one-stroke lower than the ten-handicap.  If they both scored the same but each had a triple bogey on a par four, the same one-stroke disparity would appear in their adjusted scores.  Just as with the old ESC, the handicap of the player below the breakpoint is underestimated.

            While the old ESC limited the adjusted score difference of identical scores to one-stroke, under ESC93 the difference in adjusted scores could be much larger.  If both players double-bogeyed all of the par fives, for example, then the difference in adjusted scores would be four-strokes.  In the words of the USGA, the nine-handicap would receive an "unfairly low handicap."  Therefore, ESC93 does not eliminate the breakpoint inequities as the USGA maintains, but actually increases the possibility and the size of such breakpoint inequities.

            Additionally, the ESC93 procedure exacerbates the problem by adding more breakpoints.  Under the old ESC procedure, only the breakpoints between 18-19 and 36-37 affected a significant number of golfers.  The ESC93 procedure includes four breakpoints (9-10, 19-20, 29-30, and 39-40) for this same group.  In summary, the ESC93 procedure makes the problem at each breakpoint more severe and adds more breakpoints.

ESC93 also has the problem of not treating all players evenhandedly.  While the ESC93 tends to raise the index of most players, it will reduce the indexes of high single-digit handicap players.  Based on a simple probabilistic model, a high single-digit player will be giving his low double-digit handicap competitor one more stroke when the ESC93 procedure is fully implemented. [4] 

Simplicity - The USGA argued that "clubs have difficulty teaching golfers to use the old ESC procedure correctly."  The argument that a growing innumeracy among the golfing public requires a switch to a mathematically less demanding ESC formula seems strange coming from an organization that just foisted the Slope System on that same public.  Certainly more golfers understand the old ESC system than can correctly explain the Slope System.  Does this mean the Slope System should be abolished?  While simplicity is a virtue, it should not be the primary criterion for selecting appropriate handicapping procedures.

Pace of Play - The USGA states that the pace of play has nothing to do with ESC.  The USGA's position simply is not logical. ESC93 will require players to play more strokes.  That more strokes require more time seems irrefutable.  Under the old ESC, a thirty-handicap could pick up after five strokes on a par three (scoring a 6-x).  Under ESC93, the player picks up after eight strokes (scoring a 9-x).  Watching a player play out for his nine will be as painful as it is time consuming. 

            The USGA has defended its position on the specious argument that high handicappers can play as fast as low handicappers.  While that assertion may be true under certain assumptions, it avoids the appropriate research question.  The USGA should have asked, "Does it take the same player longer to play 8 strokes than 5?"  Framed in this way, the direction, if not the size, of the effect of the ESC93 on the pace of play is obvious.

            The USGA has also argued "ESC does not tell a player when to quit hitting a ball."  Again, the USGA misses the point.  While ESC does not prohibit a player from exceeding the limit, it does act as an informal barrier to such action.  If a player is not involved in stroke play competition, he will often quit hitting the ball when he reaches the ESC limit.  To quit sooner than the ESC limit would cause an unjustifiable low handicap.  To quit after the limit is reached would serve only to irritate his playing partners.  The USGA has chosen not to recognize the role of ESC in shortening playing time because such recognition would not support their change in policy on ESC. 

Effect on Sandbagging - Under the old ESC, the unethical player (assuming a ten-handicap) was limited to two-strokes over par on any one hole.  ESC93 allows him to take four-strokes over par on a par three and three-strokes over par on a par four.  Under the ESC93 procedure, he can have fewer bad holes and still maintain his handicap.  This increases the strategic advantage the sandbagger already holds over the ethical player.  The USGA recognizes ESC93 makes it easier for the sandbagger to ply his trade, but makes the argument "that it would be a step backward to write rules or to develop a handicap system solely to trap sandbaggers."

            Of course no one has argued that trapping sandbaggers should be the sole purpose of the handicap system. The system should have the dual objectives of estimating a player's ability and making it difficult for the unethical player to manipulate the system to his advantage. 

            Having two objectives can often force tradeoffs.  For example, in calculating a player's handicap only the ten best scores out of the last twenty are used.  This is inequitable to the player who has large variation in his scoring.  His handicap will overestimate his actual ability, and he will be at a disadvantage to the steadier player.  By selecting only ten scores, however, the USGA was making it difficult on the sandbagger to artificially inflate his handicap.

            While no system can be devised that is "sandbagger-proof," any new system should make it tougher for the sandbagger to ply his trade.  ESC93 does not meet this simple test.

Conclusions - ESC93 has the advantage of being simpler, but will not make the contribution to equity the USGA claims.  Simplicity may lead to a more uniform and consistent application of ESC.  This benefit may be outweighed, however, by the detrimental effects on the pace of play and the prevalence of sandbagging.

            ESC93 is also an example of the type of tinkering with the Handicap System that should be avoided.  The conservative administration of the game of golf should be the guiding philosophy of the USGA.  Changes should only be undertaken if there is documented evidence that new policies will make significant contribution to the objectives of the USGA.  The USGA has a good record in that regard with their equipment testing program, rules and competitions, and turf grass research.[5]  It is only in the area of handicap administration where too often the USGA has confused motion with progress. 

           




[1]  See Position Paper on the New Equitable Stroke Control Procedure, United States Golf Association, Far Hills, NJ, 1992. 
[2] And if it does occur, it reduces the probability that the round would be in his lowest ten of twenty differentials.
[3]  The USGA went back to par-based ESC in 1997 for players with handicaps under ten. Consistency in reasoning has never been a USGA strongpoint.
[4] This is a rare time the Handicap System was changed to benefit the high-handicap player.  A more complete history of the discrimination against the high-handicap player will be documented in a future post, In Defense of Sandbagging, forthcoming.
[5] The USGA’s record in equipment testing is open to debate.  Geoff Shackelford makes a strong argument that the USGA was too late in stopping the increased distance due to changes in clubs and the ball.  See Shackelford, Geoff, The Future of Golf, Sasquatch Books, Seattle, WA, 2005.