Tuesday, March 24, 2015

The Selling of the USGA’s Slope Handicap System: The Big Con?

Introduction and Summary - The USGA has sold the Slope Handicap System as the solution to the portability problem.  By measuring the “slope effect” a player’s handicap will be adjusted to reflect the difficulty of any course and equity will be achieved.  The USGA, however, has never proven the existence of the slope effect nor demonstrated it could measure it with any accuracy.   If the Slope Handicap System was a consumer product, the Federal Trade Commission would step in and demand proof of the USGA’s claim before the Slope System could be marketed to consumers.  Sadly, the USGA is an unregulated monopoly and can do pretty much what it wants—no questions asked.  

This post asks those questions and presents a methodology for examining the existence, size, and efficacy of the Slope system.  A small and unrepresentative sample of scoring records is used to address three questions:  1) Is there a measurable slope effect, 2) Does the slope effect increase with handicap?  3) Does the slope effect have a significant impact on a player’s handicap?  

This limited study found it is likely there is a slope effect, but it only explains a small percentage of the variance in a player’s scoring differentials.  The slope effect does increase with a player’s Index as Slope Theory would predict.   A player’s index, however, does not explain much of the variance in the marginal effect of the Slope Rating on a player’s scoring differentials (i.e., the Slope Rating is not a perfect indicator of the difficulty of a course for player’s of the same Index).  Attributes of an individual’s game (e.g., tendency to slice, distance of the tee) probably explain more of this variance than the player’s Index.
 
There are two conditions that can negate the effect of the Slope System.  First, if a player plays courses with a narrow range Slope Ratings, the Slope System will not have a significant effect on the player’s handicap.  Second, if a player competes against players with about the same Index, the Slope Rating will not have much effect on the equity of competition.  The efficacy of the Slope system depends upon those conditions not being met.  How often that occurs is an empirical question beyond the scope of this study. 

Why does the Slope System which has only marginal benefits sell so well?  There are probably three reasons.  First, it has intuitive appeal to golfers who believe there are courses where they deserve more strokes.  Second, it sounds like it was developed with scientific rigor and is therefore credible.  And third, since a player’s index is typically lower than his pre-Slope Handicap, the Slope System caters to his vanity.   The estimate of a player’s handicap is so riddled with errors that tacking on the Slope System will not do any harm or make for a significant increase in equity.  An analogy for the introduction of the Slope System is “putting lipstick on a pig.”  It looks good, but the results are pretty much the same.

Is There a Slope Effect? – Before the introduction of the Slope System, a player’s handicap was just a function of his scoring differentials (Adjusted Score – Course Rating).  The USGA argued scoring differentials should be adjusted for the difficulty a bogey player encounters relative to the scratch player.  Without the adjustment for the slope effect the scoring differentials would not reflect a player’s true ability.  If there is a slope effect then, the scoring differentials of typical player (i.e., seldom if ever has a gross score below the course rating) should be positively correlated with the Slope Rating.  For example, if you are a 10.0 index player, your scoring differential should be higher on a course with a Slope Rating of 142 than on a course with a Slope Rating of 113.[1] In equation form this would be:

1)  Scoring Differential = a + b·(Slope Rating – 113) + є

Where є is the random error.

If there is a strong slope effect, the estimate of the coefficient “b” will be positive and statistically significant.  To estimate the coefficient with any validity would take a large random sample of players.  Since such a study is not practical for researchers outside of the USGA, a much smaller study was undertaken to 1) demonstrate the methodology, and 2) get some hint about the existence of the slope effect.  The small study consisted of the scoring records of handicap administrators and golf journalists, .  Each player in the sample was required to have played courses with a range of Slope Ratings. The sample is detailed in Table 1.

Table 1
The Slope Effect As Measured by Scoring Records (1/15/2015)

Position
Estimate of “b”
Significant?
R2
Director of Handicapping, Oregon GA
-.162
No
.06
Director of Handicapping, Washington State GA
-.238
No
.04
Director of Handicapping, NCGA
.054
No
.02
Director of Handicapping, SCGA
.160
No
.04
Exec. Director, SCGA
.040
No
.00
Executive Director, SCGA Ret.
.180
No
.05
Executive Director, USGA
.150
Yes
.17
Executive Director, USGA Ret.
.020
No
.00
Member, USGA Handicap Research Team
.120
No
.05
Editor, Golf Digest
.050
No
.03
Columnist, Wall street Journal
.310
No
.11
Member, Golf Digest Course Rating Comm.
-.046
No
.01
Director of Handicap, USGA Ret.
.140
No
.02
Asst. Director of Handicapping, USGA
.410
Yes
.33

Equation 1 was estimated using the scoring records of each player.  The estimates of “b” for each player are shown in Table 1.  For 11 of the 14 players sampled, the estimate of “b” was positive and therefore consistent with Slope Theory.  The estimate of “b”, however, is only statistically significant at the 5 percent level of confidence for two players. 

Another test of the power of the slope effect is the coefficient of determination (R2). This statistic measures the percentage of the variation in scoring differentials that is explained by the  Slope Rating.  The average value of R2 for the sample is .07.  That is, only 7 percent of the variation in scoring differentials is explained by the Slope Rating.  In summary, there likely is a slope effect, but it is small in comparison to the random error in determining a player’s scoring differential.

There are two possible explanations for the low values of R2.  First, there is not much variation in the Slope Ratings of the courses played by each player.  Players do not play a course with a Slope Rating of 70 one day, and 150 the next day.  More likely, a player will choose a set of tees within a fairly narrow range of Slope Ratings. The slope effect will be small in such cases.  Second, errors in the Slope Ratings can lead to small R2.   Regression analysis assumes the independent variable (i.e., the Slope Rating) is measured without error.   The Slope Ratings, however, are prone to error.[2]  If the Slope Ratings are listed as 125 and 130, but the true Slope Ratings are 127 and 127, regression analysis would assume there was no slope effect if the scoring differentials at the two courses were the same. 


Does the slope effect increase with handicap? - If you accept the accuracy of Slope Ratings, then the slope effect should increase with a player’s Index.  For example, the USGA assumes a large increase in the Slope Rating will have no effect on the scoring differentials of a scratch player, but should increase the scoring differentials of a high-handicap player.  Keeping with the USGA’s assumption that a player’s average score is 113 percent of the average of ten best out of twenty scores, a player’s scoring differential is expected to be:[3]

2)            Expected Scoring Differential = Index· (Slope Rating/113) ·1.13 = Index·Slope Rating·.01

The marginal increase in the scoring differential for a one point increase in the Slope Rating is then:

3)            Marginal Increase = Index·.01

Now the estimates of “b” shown in Table 1 are estimates of the marginal increase in a player’s scoring differential for a one point increase in the Slope Rating.  In equation form:

4)            “b” = C + D·Index

To estimate the coefficients C and D, the values of “b” shown in Table 1 were regressed against the player’s Index. The estimated regression equation was:

5)            “b” = -.01 + .008·Index

Equation 5) is an empirical test of the validity of the assumption behind the Slope System as represented in equation 3).  In Slope Theory, the constant C should be zero.  The estimate of C is .01, but is not statistically different from zero.  The theoretical value of D should be .01 as shown in eq. 3.  The estimated value of D is .008.  The estimated value of D is close to the theoretical value, and is statistically significant (i.e., the hypothesis that the true value is .01 cannot be rejected with a 5 percent level of confidence.).

The Figure below shows the plot of the regression line and the values of “b”.  As the figure indicates, a player’s Index does not explain much in the variance of “b”—i.e., the values of “b” are not clustered close to the regression line.  The value of R2 for this equation is only .04.  This indicates there may be a large random error associated with each player’s “b.”   The Slope Rating is for some average player, but the true Slope Rating for an individual player may vary by his particular playing characteristics.

With such a small and unscientific sample as used here, no final conclusions can be drawn.  It appears the slope effect is an increasing function of a player’s handicap, but does not explain a major part of the variance in the marginal effect of the Slope Rating.

Does the Slope Effect have a significant impact on a player’s handicap?   Table 2 (column 3) below shows the player’s handicaps computed without the Slope Rating of a course (i.e., his or her pre-Slope handicap).  Column 4 shows the range of Slope Ratings where a player’s Slope adjusted handicap would be the same as his pre-Slope handicap.  Column 5 presents the percentage of the 20 courses in a player’s scoring file where his Slope Adjusted handicap and his Old Handicap were the same.   The percentage will vary depending upon a player’s index and the variance in the Slope Ratings of the courses played.  The higher a player’s index, the smaller the range of Slope Ratings that will make the two handicaps equal and therefore the greater the chance that a Slope Rating will fall outside of this range.  If a player has a small range in the Slope Ratings he encounters, the two handicaps are more likely to be equal.  In no case, however, did a player’s pre-Slope Handicap differ from his Slope Adjusted Handicap by more than one stroke.

Therefore, the efficacy of the Slope System depends upon the Slope Rating being accurate and a player having a large range in the Slope Ratings of the courses he plays.   It is an empirical question as to whether a player’s scoring record has a wide range in Slope Ratings.   If a large percentage of players have a wide range, the Slope System may be correcting an inequity.  If the percentage is small, a lot of effort t is being made for little or no increase in equity.
   
Table 2
Handicaps and the Slope System


(1)
Position

(2)
Index
(3)
Pre-Slope Handicap

(4)
Slope Range

(5)
Percent Same
Director of Handicap, Oregon GA
6.1
7
121-138
75
Director of Handicap, Washington GA
7.3
8
117-131
100
Director of Handicap, NCGA
4.5
5
113-138
80
Director of Handicap, SCGA
9.6
11
124-135
85
Exec. Director, SCGA
14.9
17
126-132
45
Executive Director, SCGA Ret.
18.2
20
122-127
55
Executive Director, USGA
3.0
4
132-155
65
Executive Director, USGA Ret.
11.6
13
122-131
55
Member, USGA Hand.  Research Team
11.3
12
116-124
75
Editor, Golf Digest
7.6
9
127-140
45
Columnist, Wall street Journal
4.9
6
127-149
90
Member, Golf Digest Course Rating
4.8
6
130-153
45
Director of Handicap, USGA Ret.
9.5
11
125-136
95
Asst. Director of Handicapping, USGA
11.0
12
119-128
55

Why does the Slope System Sell so Well? -  The Slope System was adopted in the United States with no serious opposition.  It has now spread to many more countries.  There are a few notable holdouts, but they may go the way of the Betamax as the drive for a universal handicap system gains momentum.  There are probably three reasons for its acceptance.  First, it has intuitive appeal to golfers who believe there are courses where they deserve more strokes.  Whether the Slope Rating is an accurate measure of how many strokes they should receive is not questioned.  If they get more stokes, this group is satisfied.

Second, it sounds like the Slope System was developed with scientific rigor and is therefore credible.   The ubiquitous “113,” gives the illusion of precision, though in reality it was just a number picked out of the air.    And third, since a player’s index is typically lower than his Old Handicap, the Slope System caters to his vanity.  For example, a player is no longer an 11 handicap, but a 9 index.  The 9 index sounds like a better player than an 11 handicap to the uninitiated though they are in fact equivalent.

The estimate of a player’s handicap is riddled with errors.  The USGA Handicap System contains many self-admitted biases.  There is sampling error in using only twenty scores.  The handicap estimate is also a lagging indicator of a player’s current ability.  There are numerous rounding errors made in the calculation of a player’s index and his subsequent course handicap.[4]  There are errors in the estimates of the Course and Slope Ratings that in turn lead to errors in the estimate of a player’s handicap.  Then there the errors caused by the dubious character of some players (sandbaggers).  So tacking on the Slope System will not do much harm or bring a significant increase in equity.  Incorporating the Slope System into the existing Handicap system is like “putting lipstick on a pig.”   It looks good, but the results are pretty much the same.





[1] If you are a player with a plus index, your differential should be negatively related to the Slope Rating.  This is an anomaly of the Slope System explained in Dougharty, Laurence, “Why the Slope Handicap System Doesn't Work for Plus Indices, www.ongolfhandicaps.com, August 26, 2012.
[2] The accuracy of Slope Ratings is examined in Dougharty, Laurence, “How Accurate is the Slope System,” www.ongolfhandicaps.com, October 8, 2012.
[3] Stroud, R.C., and L.J. Riccio, “Mathematical underpinnings of the slope handicap,” Science and Golf, The Proceedings of the World Scientific Congress of Golf, E.FN. Spon, London, 1990, pp. 135-140.
[4] If Player A (Index=8.9) played Player B (Index=5.1) at a benign course with a Slope Rating of 121, Player A would get 5 strokes. If the course had a Slope Rating of 146, Player A would only get 4 strokes.  Rounding produces this result which is contrary to the philosophy of the Slope System that supposedly aids the higher-handicap player on courses with a higher Slope Rating.

Monday, February 16, 2015

John Paul Newport and "Going for 100": Is this Column Really Necessary?

John Paul Newport’s recent column “63 Women—And Me—Go On a Buddy Trip” (Wall Street Journal, February 7-8, 2015) raises questions of journalistic integrity and editorial judgment.  Let’s look at a few.

Why was this trip the subject of a column?  Women and men go on golf trips all of the time.  There is no apparent reason to document this one in a major newspaper.  Sure, 63 women is a fairly large number, but it not a Guinness Book record.   “Going for 100” was allegedly started 5 years ago, has 170 members, but the Bandon Dunes trip was the group’s first outing.   What has the group been doing for the past 5 years?  Were the participants truly buddies or just members of a loosely affiliated group?—e.g., Costco members.
The founder of the organization has playing privileges at Cypress Point, San Francisco Golf Club, and Burlingame Country Club.  Another participant is a member of the Wente (as in Wente Vineyards wine) family with privileges at Cypress Point, Monterey Peninsula CC, and Ruby Hill Golf Club.  It doesn't sound like any of the participants play in a 9-hole league after work.  Had I not known better, I would have thought the column was the product of Robin Leach rather than a writer who shares a byline with the likes of Kimberley Strassel, Bret Stephens, and Mary O’Grady. 

It is true this column fits the genre of many of Newport’s columns (i.e., basically a diary of his golfing adventures with Trump, Trevino, et al.).  But there must be a back story here that Newport fails to explain.  Did he fail to disclose any previous connection with the group  or was he offered any inducement to promote the trip?  The trip cost $1000, but that probably was for double occupancy.  Who did Newport bunk with?

Why does Newport think he is Margaret Mead?  Much of Newport’s column is devoted to his observation on the differences between men and women.  He makes no claim “of broader significance or surprising insights,” but goes ahead anyway.  He has to get to 800 words somehow.  Women prefer chardonnay while men prefer beer and women tend to be more verbally supportive of each other than men are, or so writes Newport.  His sample consists of one group of women of high socioeconomic status—a statistically insignificant sample in both size and composition.  Newport also demonstrates he has never heard of the Hawthorne Effect in which individuals improve their performance when they know they are being observed.  If these women knew they were going to be a subject of a WSJ column—why else would Newport be invited?—they probably checked any bad behavior at the airport departure counter. 
   
Why does Newport, the golf writer for the WSJ, not understand the USGA Handicap System? Newport wrote “Handicaps ranged from 4.7 to a few higher than 30.”  A handicap is always an integer so no player has a “4.7” handicap.  Newport confused a player’s index with her course handicap.   Newport also failed to post any score from Bandon Dunes.  This is a serious breach, but maybe Newport has an explanation for his handicap chairman--“I was just playing for fun with the girls and was going to post before the start of the active season.  Truly, I was!”

To end on a positive note, while Newport details the problems with the commuter flight to Bandon Dunes, he does not say the plane came under RPG fire.

Friday, January 30, 2015

What is the Value of a Handicap Stroke?

Attempts to put a value on a handicap stroke fall within the set of problems that are not worth solving.  There is no general solution since the value will depend upon the playing characteristics of both yourself and your opponent.  In this post, only the specific case presented in Section 17 of the USGA Handicap System is examined.   The value of a handicap stroke is evaluated using 1) a continuous probability function associated with average scores and 2) a discrete probability function describing hole scores.

Continuous Probability Function - The value of a handicap stroke depends on the average difference in score between players and their respective standard deviations of scoring.  In the general case, assume there are two competitors, Player A and Player B.  Player A has the lower hole scores.  They have an average score of ASa and ASb, respectively.  Their standard deviations of scoring are σa and σb.  The distribution of the difference in scores is normally distributed with the following mean and standard deviation:

                D = ASb  - ASa = Mean of the difference in average hole scores
And,
                σd = (σa2  + σ b2)1/2  = standard deviation of the distribution of the difference in average scores

As an example, assume the mean difference in average holes scores 2.38 strokes (i.e., hole 18 in the USGA example).  Further assume each player has a standard deviation of 0.7 strokes.  The standard deviation of the difference in hole scores, σd, is then approximately 1.0.  For Player B to win a hole, the difference in average hole scores must be zero or less than zero.  The probability of that happening is the probability of drawing an average difference more than 2.38 (2.38/1.0) standard deviations from the mean.  That probability is .01.  If Player B is given a stroke on the hole, he must now draw an average difference more than 1.38 standard deviations from the mean.  The probability of that occurrence is .08.  Table 1 below shows how getting a handicap stroke affects a player B’s probability of having a lower average score on each hole.

If a player had the opportunity to buy a handicap stroke, he would look for the hole where the stroke increases his winning by the largest margin.   If the prize for winning a hole was $1, then the stroke on hole 6 would be valued at $0.38.  A handicap stroke on hole 18, on the other hand, would only be worth $0.07. In summary, the USGA’s recommended procedure assigns the higher-handicap player strokes where they are of minimal value (i.e., holes 4 and 18).

Table 1
Probability of Winning a Hole


Hole

Avg. Difference
Probability of Winning the Hole

Gain
Without Stroke
With Stroke
1
1.13
.13
.45
.32
2
1.41
.08
.34
.26
3
0.75
.23
.60
.37
4
2.10
.02
.14
.12
5
0.74
.23
.60
.37
6
0.73
.23
.61
.38
7
0.93
.18
.53
.35
8
1.22
.11
.41
.30
9
0.88
.19
.58
.39
10
1.05
.15
.48
.33
11
1.45
.07
.33
.26
12
2.04
.02
.15
.13
13
1.39
.08
.35
.27
14
0.80
.21
.58
.37
15
1.22
.11
.41
.30
16
0.75
.23
.60
.37
17
1.84
.03
.20
.17
18
2.38
.01
.08
.07

Discrete Probability Function - The first case examined is when there is a large difference in average scores between a high handicapper and a low handicapper on a hole.  One set of probabilities that make for a 2.0 stroke difference in average scores is shown in Table 2 below.
Table 2
Discrete Probabilities When Average Difference in Scoring is Equal to 2.0

Low Handicapper Probabilities and Avg. Hole Score
High Handicapper Probabilities and Avg. Hole Score
4
5
6
7
Score
4
5
6
7
Score
0.55
0.45
0.00
0.00
4.45
0.05
0.10
0.20
.65
6.45

Now let’s calculate the probabilities of a win, halve, and loss for the high handicapper with and without a stroke. 
Without a stroke the probabilities are:
                P(win) = .45 ·.05 = .0225
                P(halve) = .55 · .05  + .45·.10 = .0725
                P(lose) = .55· 95 + .45·.85 = .905

With a stroke the probabilities for the three outcomes are:
                P(win) = .05 + .10·.45 = .095
                P(halve) = .10·.55 + .20·.45 = .145
                P(loss) =  .20·.55 + .65 = .76

Now let’s change the probabilities on the hole for each player so the average difference in scores is only 1.0 stroke.  One set of probabilities that meet this criterion is shown in Table 3.
Table 3
Discrete Probabilities When Average Difference in Scoring is Equal to 1.0

Low Handicapper Probabilities and Avg. Hole Score
High Handicapper Probabilities and Avg. Hole Score
4
5
6
7
Score
4
5
6
7
Score
.1
.7
.1
.1
5.2
.01
.20
.36
.43
6.2

Without a stroke the probabilities for the three outcomes are: 
                P(win) = .01·.9 + .2·.2 +.36·.1 = .085
                P(halve) = .1·.01 + .7·.2 +.1·.36 + .1·.43 = .220
                P(lose) = .1 ·.99 +.7·.79 +.1·.43 = .695

With a stroke the probabilities of three outcomes are:
                P(win) = .01 + .2·.9 + .36·.2 + .43·.1 = .305
                P(halve) = .2·.1 +.36·.7 + .43·.1 = .315
                P(lose) = .36·.1 +.43·.8 = .380

Table 4 shows the change in probabilities when a handicap stroke is applied.  As expected, the stroke applied when there is a 2.0 stroke difference in average score is less helpful to the high handicapper than where there is a 1.0 stroke difference.  For example, a stroke increases his winning percentage by .15 when applied to the 1.0 hole rather than the 2.0 hole.  Interestingly, the smallest impact is on the number of halved holes though this may be an artifact of the selected probabilities.

As in the continuous probability analysis, assume that a player receives $1 for winning a hole.  If the average difference in holes scores is 2.0, a handicap stroke would only increase the player’s expected winning by $0.07.  On a hole with a 1.0 difference in average score, a handicap stroke would increase the player’s expected winning by $0.22.   Given a choice, the high handicapper would take his stroke on the 1.0 hole.  Given a choice, the USGA procedure would assign the player a stroke on the 2.0 hole.

Table 4
Change in Probabilities

Average Score Difference
Change in P(win)
Change in P(halve)
Change  in P(lose)
2.0
.0725
.0725
-.145
1.0
.2200
.0950
-.315