Monday, April 28, 2014

Why Section 10-3 is Ineffective


A major problem with Section 10-3 is a player has to be the Jack the Ripper of sandbaggers to be affected.  More moderate sandbagging, while still a winning strategy, escapes the clutches of the Section.  A review of the methodology of Section 10-3 will expose its weakness.
The reduction in index for exceptional tournament scores is a function of the difference between a player’s index and the average differential of his two best tournament scores (T-scores).  For a reduction to occur, this difference must be greater than the “Standard Difference” set by the USGA.  The Standard Difference varies with the number of T-Scores a player has in his file as shown in the Table below.  If a player has three T-Scores, for example, the difference must be equal to or greater than 4.5.  If a player competes on the same course, the following inequality must be met for an index reduction to occur:

Eq. 1)     Standard Difference ≤ Handicap Index – (Avg. 2 Gross T-Scores – Course Rating) · (113/Slope                                      Rating)
For simplicity, assume the two gross scores are the same.  Since a player’s handicap is his index multiplied by the Slope Rating divided by 113, Equation 1 can be reduced to:

Eq. 2)    Standard Difference · (Slope Rating/113) ≤ Handicap – (Gross T-Score – Course Rating)
Since a player’s Net Score is his Gross T-Score minus his handicap, Equation 2 becomes):

Eq. 3)    Net Score ≤ Course Rating – Standard Difference · (Slope Rating/113)

Table
Avg. of 2 Net Scores for Reduction in Handicap Index
(Course Rating = 68.7   Slope Rating = 119)

Number of T-Scores
2
3
4
5-9
10-19
Standard Difference
4.0
4.5
5.0
5.5
6.0
Avg. of 2 Best Net Scores[1]
64.5
64.0
63.4
62.9
62.4

Now let’s see what Net Scores are necessary to trigger the reduction in index.   For illustration, the Course Rating is 68.7 and the Slope Rating is 119.  Assume a player has a 17.5 index and 14 T-scores.  The Table indicates that if he has two Net Scores of 63, he will not see a reduction in his index (i.e., his two net scores average above 62.4).  A 17.5 index leads to a handicap of 18.  To have a Net Score of 63, he would have a Gross Score of 81.  Two Gross Scores of 81 would lead to an average tournament differential of 11.7.  To determine if there would be a reduction, the average tournament differential is subtracted from his index.  In this case, the difference is 5.8 (17.5 – 11.7).  To receive a reduction this difference must be 6.0 or greater.  Therefore, the player’s index is not reduced under Section 10-3 as the Table predicts.

The Table indicates great scores do not necessarily lead to a reduction in index.  The player can further reduce his chances of getting a reduction by 1) playing in only one tournament a year , 2) managing his score with discretion in four-ball competitions (e.g., finding a water hazard when your partner is safely on the green), and 3) posting regular rounds as tournament rounds when the handicap committee is not paying attention.

While Section 10-3 does not always punish the guilty, it can snare the innocent.  For example, assume a player is a 10.4 index and an 11 handicap on the course described above.  He has two net scores of 68 (gross scores of 79) in a tournament, and did not win anything.  His two tournament differentials average 9.8.  Later in the season his Handicap Index increases to 13.9.  At this point, Section 10-3 kicks in and the player’s index is reduced by 1.0. He is assigned a 12.9R. If the player continues to slump and his Handicap Index increases to 14.9, he is whacked even harder and given a 12.3R.

When Section 10-3 was introduced, its aim was to identify the flagrant sandbagger and hang a scarlet “R” around his neck. This goal was never met.  Instead, getting an “R” implies either 1) you are currently in a slump, or 2) you were not clever enough to avoid being caught.
 






[1] Both differentials must be 3.0 below a player’s Handicap Index.   

Tuesday, April 15, 2014

The Truth about Section 10-3

An article by Kevin O’Connor appearing in Fore Magazine entitled “The Truth Behind those Tournament Scores” contains a statement that ironically appears to be untrue:[1]

Sometimes golfers impacted express the concern that at the time the score was posted it did not seem exceptional, but as a player’s normal calculation increased a score/differential may have become exceptional.  This was purposely implemented as part of the formula, which is all about potential ability.

In other words, players have objected to receiving a reduced index under Sec. 10-3 even though they did not do well in a tournament.[2]  O’Connor argues this reduction was a purpose of Sec. 10-3.  Evidence indicates  such reductions were not the result of well crafted regulation, but rather an unintended consequence.  Let’s examine three pieces of evidence:

1. The USGA entitled Sec. 10-3 “Reduction of USGA Handicap Index Based on Exceptional Tournament Scores.”  It was not entitled “Using Tournament Scores as a Measure of a Player’s Potential. “  If the USGA wanted to measure a player’s potential, a sample of two scores is too small to be reliable.  A more reasonable approach would have been to adopt a “cap” where a player’s index is limited to a certain number of strokes over his low-index of the year. [3]

2. The example in the Handicap System implies the tournament scores were “exceptional.”  After all, the purpose of Sec. 10-3 was to impose a penalty on a player who won with scores that were much lower than expected.  Nowhere in the Handicap System does it explicitly discuss how tournament scores can become exceptional as a player’s index increases sometime later.  If the USGA really thought of Sec. 10-3 a measure of a player’s potential ability, it seems likely it would have mentioned this added benefit somewhere.  More to the point, the USGA indentified the problem early on.  Dean Knuth wrote in 1994 “The Handicap Research Team….(is) seeking ways to keep from reducing declining players whose early  year tournaments are much better than their current performance.” [4] 

3. Sec. 10-3 creates inequities contrary to the purpose the Handicap System.  Let’s take an example of a player with two T-score differentials of 10.0 made while he was an 11.0 index.  It is likely our player did not win anything in the two tournaments.  When this player plays a competitor with the exact same scoring record (though no T-scores), however, he must give strokes.  If both players had a 16.0 index, for example, the non-tournament player would receive four strokes (assuming the Slope Rating is 113) from the tournament player as shown in the Table below.  This is contrary to the purpose the Handicap System which is to “enable golfers of all levels to compete on a comparatively equal basis.” [5]  It seems unlikely the USGA would “purposefully” create such inequity.
Table
Reduction in Index


Sec. 10-2 Index

 Sec. 10-3 Reduction
Strokes Given to Non Tournament Player
13
13.0
0
14
13.0
1
15
12.4
2
16
11.9
4
17
11.5
5

The Table also reveals another paradox of Sec. 10-3.  If Sec. 10-3 is to measure a player’s potential, why is that potential dependent upon his Sec. 10-2 index?  As shown in the Table, a player with a 17 index is assigned more potential (i.e., a lower index) than a player with a 14.0 index.  If two players had the same tournament scores, shouldn't their potential be the same?  The  initial Sec. 10-3 is at least consistent by keeping a player’s potential (i.e., reduced index) independent of his Sec. 10-2 index once the net differential (handicap index – average of 2 best T-score differentials) threshold of 3.0 is reached. [6]

Based on this evidence, it is likely the USGA didn't consider the collateral damage to the innocent bystanders described above.  The good news is the incidence of such damage is very small.  The bad news is the lack of casualties stems from Sec. 10-3 being ineffectual in punishing both the guilty and the innocent.  The low apprehension rate of Sec. 10-3 can be traced to at least three causes:

·              Some tournaments use a format where scores cannot be posted (e.g., Chapman, scrambles).
·              Four-ball tournaments allow the unethical player opportunities to manipulate his score.
·             Sec. 10-3 is not applied uniformly.   Sec. 10-3 is often ignored at some clubs, and used too often at                   others.  This makes the Section both ineffective and inequitable.

The USGA has never published any research on the effectiveness of Sec. 10-3.  When asked recently how many players receive a reduced index, the USGA replied “We do not keep such statistics.”[7]  Apparently the USGA does not want to know the effectiveness of this section.  Sec. 10-3 lives on since it: 1) gives the illusion of curing the sandbagging problem, 2) does not generate negative feedback since so few are affected, and 3) relieves the indolent handicap committee of the responsibility for rooting out the unethical player.  In essence, it is the perfect bureaucratic solution.     




[1] O’Connor, Kevin The Truth Behind “Those Tournament Scores,” Fore Magazine, Southern California Golf Association, Winter 2014, p. 66.  O’Connor is Director of Handicap and Membership for the Southern California golf Association.  He previously serve as Senior Director of Handicapping for the United states Golf Association.
[2] Sec. 10-3, “Reduction of Handicap Index Based on Exceptional Tournament Scores,” USGA Handicap System -2012-2015, pp. 78-83.
[3] Such a cap is part of the handicap system of Australia—see Golf Australia Handicap System, 2014.
[4] Knuth, Dean, et. al., “Outlier identification procedure for reduction of handicaps,” Science and Golf II, E and F Spon, London, 1994, p. 233.
[5] Lahman, Eric, “Purpose of the USGA Handicap System,” www.USGA.org, 2009.
[6] Sec. 10-3, “Reduction of USGA Handicap Index Based on Exceptional Tournament Scores,” USGA Handicap System, 1994, pp. 41-43.  The player in the example would play to the lower of his current index or 13.0 (i.e., average of two T-scores + 3.0).
[7] E-mail to author from Annie Pollock, USGA, November 20, 2013.
.

Tuesday, March 4, 2014

The Problem with Section 3-5


A recent article in FORE Magazine (“A Perfect Match,” November/December 2013) argues the Handicap System makes for “perfect “competition even when players compete from different tees. You can trace the push for Sec. 3-5 (Sec. 8-4f back in 1990) to Alice Dye and her article “Tees for Two or More” in the July 1990 PGA Magazine.  Ms. Dye, like the article, used an example of individual stroke play to demonstrate the equity of Sec. 3-5.  If a competition is social (e.g., husband v. wife, buddy v. buddy) stroke play, Sec. 3-5, while not perfect, is good enough.  The problem comes when the competition is more serious and when other forms of competition are introduced.  Articles like this can unintentionally mislead the reader into believing Sec. 3-5 cures all of the equity problems of such competition.   I believe the use of Sec. 3-5 should be discouraged and only used as a last resort.  Here are some reasons why:

1. Biased Handicaps - Rounding errors and the Bonus for Excellence can give one player an advantage when competition is from different tees.  The Appendix presents an example of two players with 13.5 indexes.  When they play from different tees, one player has an expected net score 1.7 strokes below that of his competitor.  Based on certain assumptions, that player should win 65 percent of the time.  Had they played from the same tees, neither player would have an advantage.

2. Equity of Competition – Sec. 3-5 depends upon the accuracy of the Course and Slope Ratings. The standard error of the estimate of Course and Slope Ratings is much larger than the USGA wants to admit.  Given these error, the efficacy of Sec. 3-5 becomes in doubt.  Significant errors can occur in assigning the proper handicap when a player establishes his index playing one set of tees, but plays another set of tees in a competition.

 3. Misuse of Sec. 3-5 – I have seen Sec. 3-5 applied to various forms of competition –scrambles, Chapman, foursome, and four-ball.  I have not seen any evidence indicating Sec. 3-5 makes for equitable competition in these formats.  I have conducted small sample studies that show Sec. 3-5 did not always equalize competition in scramble and four-ball events.  To be fair, the USGA does not recommend using Sec. 3-5 for scramble competitions, but it also does not explicitly prohibit its use.  When Sec. 3-5 is used in a four-man scramble, the handicap allowances for the C and D players are often the same from both the front and back tees. The team that recognizes this anomaly, and places it C and D players on the front tees has a significant advantage.

4. Handicap Stroke Allocation Issue – All the theory behind stroke allocation goes out the window when players compete from different tees.  This is not a great concern, because I do not believe stroke allocations are a major determinant in who wins a match.  Advocating competition from different tees, however, sends the message that Section 17 (Allocation of Handicap Strokes) of the USGA Handicap System isn’t all that important.

5.  Pace of Play – One of the arguments for “Play It Forward” is that rounds would be played faster.  When players compete from different tees in the same foursome, it takes longer to play a round and any time benefit from Play it Forward could be lost.

6. Smaller Scoring Variance from the Front Tees - In some four-ball competitions, the second ball is used as a tie-breaker.  It is probable the scoring variance is smaller from the front tees. This should give the team playing the front tees a small edge when the second ball is used as a tie-breaker. 

Appendix
 Section 3-5 (Players competing from different tees) argues competition would be equitable if the difference in Course Ratings is added to the handicap of the player competing from the tees with the higher course rating.  Applying Section 3-5, however, introduces three possible errors
1. Rounding the Difference in Course Rating Error – To demonstrate the equity of Sec. 3-5, examples are manufactured to give both players the same net score. The example shown in Table 1 is taken from the USGA explanation of Sec. 3-5.  Both players have an expected net score of 71, and equity is allegedly assured.
Table 1
USGA’s 3-5 Procedure

Gary (Gold Tees)
Vs.
Bob (Blue Tees)
10.4
Index
10.4
130/113
X Slope Rating/113
140/113
12
Course Handicap
13
71.1
Course Rating
73.2
83
Target Score
86
12
Adjusted Handicap
13 + 2 =15
71
Net Score
71

The findings, however, depend upon the choice of the Course Ratings.  Table 2 demonstrates that changing the Course Ratings can lead to different results.
Table 2
3-5 with Different Course Ratings

Gary (Gold Tees)
Vs.
Bob (Blue Tees)
10.4
Index
10.4
130/113
X Slope Rating/113
140/113
12
Course Handicap
13
71.4
Course Rating
73.8
83
Target Score
87
12
Adjusted Handicap
13 + 2 =15
71
Net Score
72


In Table 2, the Net Scores of the two players are not equal after making the Sec. 3-5 adjustment.  The difference is due to rounding the difference in Course Ratings.  Therefore, competition is not as perfect as the article implies.

2. Bonus for Excellence Error - For equity, however, it is not the target score that is important, but a player’s expected net score.  Ideally, the expected net score of both competitors should be the same.  For simplicity, we assume a player’s expected gross score is the average of his ten best out of twenty scores.[1]  A player’s expected gross score is given by the handicap formula:
                Expected Gross Score = (Index · (Slope Rating/113))/.96 + Course Rating
The .96 in the equation is the Bonus for Excellence.
A player’s handicap is:
                Handicap = Index · (Slope Rating)/113
A player’s Expected Net Score is:
                Expected Net Score = Expected Gross Score – Handicap
                                 = (Index· (Slope Rating/113)/.96 + Course Rating - Index · (Slope Rating/113)
                                 = 0.04· (Index · Slope Rating)/113 + Course Rating
When the Slope Ratings of the two tees are fairly close, the Bonus for Excellence introduces only a minor error.  This is not always the case, however.  As an example, assume two payers have an index of 13.5.  One player plays the gold tees with a Slope Rating of 88.  The other player plays from the blue tees with a Slope Rating of 146. The player from the gold tees would have an expected net score .4 strokes over the Course Rating.  The player competing from the blue tees would have an expected net score .7 strokes over the Course Rating.   The player competing from the gold tees will have a slight edge (.3 strokes) that will not be corrected by Section 3.5. (Note: This bias is due to the USGA’s Bonus for Excellence which is yet another argument for its elimination.[2]  The bias favors the player with the lower handicap, and not necessarily the player with the lower index.  This is contrary to the intent of the Bonus for Excellence that supposedly rewards the better player.)

3.  Handicap Rounding Problem – When players with the same index play from the same tees any rounding error is the same for both players.  When they play from different tees, rounding can give one player up to a 1-stroke advantage.  Let’s use the example above  The unrounded handicap of the player playing the gold tees would be 10.51.  The unrounded handicap for the player competing from the white tees would be 17.44.  The difference in unrounded handicaps is 6.93 strokes or approximately 7 strokes.  The difference in their rounded handicap, however, is 6 strokes.  In this case the player competing from the gold tees gains a 1-stroke advantage because of rounding. 


4. Total Error - Table 3 shows how all of these errors can add-up to give the player from the gold tees an advantage  He gains approximately 1-stroke when his handicap is rounded up while his competitor’s handicap is rounded down.  If we assume the difference in Course Ratings is 2.4 strokes, he gains .4 strokes when the difference in Course Ratings is rounded down.  He gains another .3 strokes because of the Bonus for Excellence.  His total advantage is 1.7 strokes.  This is about as large as an advantage that can occur.  All of the rounding errors were in favor of the player competing from the gold tees.  In most cases the bias will be smaller.  The point is, however, is that these biases disappear when these competitors play the same tees.[3]

Table 4 also shows how the errors can affect the probability that a player wins a match. It is assumed that each player’s scoring record has standard deviation of 3.0. In the extreme case, the probability of winning can reach .65.   In this instance, playing from different tees has made the competition less equitable. 
 
Table 4
Probability of Gold Tee Player Winning[4]

Bias
Net Score Advantage
Probability of Winning
Bonus for Excellence (BE)
.3
.53
BE + Course Rating  Rounding(CRR)
.3 + .4 = .7
.57
BE + CRR + Handicap Rounding
.3 +  .4 + 1.0 = 1.7
.65









[1] A player’s actual expected score, under certain assumptions, is the average of his ten best scores plus .8 times a player’s scoring variance.  If both player’s have the same variance, the average of a player’s ten best scores is an adequate proxy for his actual expected score for the purposes here.
[2] Dougharty Laurence, “The USGA’s Bonus for Excellence Ruse,” www.ongolfhandicaps.com, January 15, 2013
[3] There will still be rounding problems when players play from the same tees.  The example was a special case where the indexes of both players were the same. 
[4] The methodology behind the estimates can be found in “A Sandbagger’s Guide to Winning,” www,ongolfhandicaps.com, December 19, 2013

Wednesday, February 19, 2014

John Paul Newport: Not the Sharpest Wedge in the Bag II

John Paul Newport is the golf columnist for the Wall Street Journal.  His lack of analytics can often lead him astray as it did in his column of July 2-3, 2011 (“Fighting Back Against Sandbaggers”).  First, he took some tables off the Internet, and assumed they were valid—this is never a good idea or good journalism.  Second, some knave misled him on the interpretation of the tables and the vagaries of the Slope System.  This resulted in multiple errors. Third, Newport pronounced the Tournament Point System (TPS) the Salk vaccine for sandbagging on the basis of two phone calls.  I doubt the efficacy of the TPS, and strongly believe sandbaggers will be eating the few remaining cockroaches when our planet comes to its end.  More detailed comments are made below.   Excerpts from Newport’s column are in italics.   

Among the most fascinating pages of Knuth’s website…is one titled “Odds of Shooting an Exceptional Tournament Score.”    A player in the 13-21 handicap range, for example, will shoot better than his course handicap only one round in every six.

The table Newport cites on Knuth’s website is also Appendix E of the USGA Handicap System.  Part of that table is shown as Table 1 below.  When the probabilities shown in the Table were first developed there was no Slope System and the probabilities were related to handicaps.  When the Slope System was introduced handicap ranges were switched to index ranges.  Such a conversion is not theoretically correct, but intellectual consistency was never the USGA’s strong suit.   Newport should have used an index range of 13 to 21.9 to be in accordance with the USGA

Table 1
Appendix E - Exceptional Tournament Score Probability Table[1]

Net Differential
Handicap Index Ranges
0-5
6-12
13-21
22-30
>30
0
5
5
6
5
5
-1
10
10
10
8
7
-2
23
22
21
13
10
-3
57
51
43
23
15
The values in the table are the odds of shooting a net differential EQUAL OR BETTER than the number in the left column.

Looking at Table 1, an inconsistency should be obvious.   If a player has an index between 13 and 21, he is 20 percent less likely to score a net differential of zero or better than a player with either a lower or higher index.  This is the only case where the probability of any level of performance (e.g., -2,-3…-10 net differential) does not increase or stay the same as index is increased.  Why is this index range cursed with such poor performance?  I suspect it has to do with poor research technique.  Any difference in probability among groups for scoring a net differential of 0 or better may not be statistically significant.  This hypothesis cannot be confirmed, however, since the USGA does not release its studies for peer review.    When I mentioned this anomaly to the USGA, the probability for the 13.-21.9 index player was changed from 6:1 to 5:1 in the 2012-2015 edition of The USGA Handicap System.  The USGA never admitted a mistake nor cited new research for the change.

The odds of a midrange player shooting eight strokes better than his course handicap are 1 in 1,138.  The odds of him doing that twice in 20 rounds (much less than in a two- or four-round tournament are 1 in 14,912).

There are several errors in this quote.  Table 1 show the probability of shooting a net differential of  -8 or better, not just -8.  Newport also errs in using handicap instead of index.  An example will make this clear.  Assume a player is a 13.0 index.  He plays a course with a course rating of 72.0 and a Slope Rating of 150.  His course handicap is 17.  Assume he shoots eight strokes better than his handicap (i.e., his net score is 64 and his gross score is 81).  The player’s handicap and net differentials are:
Handicap differential = (81-72) x 113/150 = 6.8
Net differential = 6.8 -13.0 = - 6.2 rounded to -6.0
So in this case,  the probability of a player shooting 8 strokes or better than his handicap is 323:1 and not 1,381:1.  The probability of the player doing that twice in 20 rounds is 1 in 3,385, and not 1 in 14,912. (Note: These latter probabilities are taken from a Table shown in Knuth’s website that also has technical problems that are not discussed here.)   In his defense, Newport was just using “handicaps” as instructed by Knuth.  The fact that Newport did not notice the inconsistency between “handicap” and “index” demonstrates Newport’s shortcomings as journalist.

But clubs or associations that run numerous handicapped or net tournaments are using another more effective Knuth invention called the Tournament Point System.

Newport judged the Tournament Point System (TPS) effective based on little evidence.  It appears the TPS has been around for at least 7 years and possibly much longer.  If it was that effective, it should have swept the nation.  It appears Newport only found two clubs using the Point System, one of which may have been Knuth’s home club in California.  The other club that extolled the TPS was Lost Creek Country Club in Austin Texas.  A call to the Lost Creek pro shop in 2014, however, revealed the TPS it is no longer in use.   
A club in Oregon implemented a version of the TPS nine years ago.  It was not a fair test, but it did reveal critical flaws in the system:
Not Effective at Many Clubs – Many clubs do not have a large number of major tournaments necessary for effective implementation of the TPS.  Often, the Member-Guest is the only tournament with significant prize money.  Under the TPS, a player could win the Member-Guest every year and not be penalized.  If minor events are put under the TPS, the strategic player could chose to sit out these tournaments rather than risk a penalty for an up-coming major tournament.  The artful sandbagger could also go unpunished under the TPS.  If the TPS cuts two strokes, he merely submits scores that raise his handicap by two strokes.
Double Whammy – Assume a player with a 15 index had two net tournament differentials of -8.  Under section 10-3 of the USGA handicap system, a player’s index could be reduced to 8.2.  On a course with a 133 slope rating his handicap would come down from 18 to 10.  Under the TPS his handicap would be further reduced by another three strokes making him a 7 handicap.  This is draconian.  Now the tournament committee may think this unfair and not apply the TPS reduction.  But if the objective of the TPS was to remove discretion, it would have failed.
Too Complicated and Time Consuming – The TPS is not as simple as suggested in the Newport’s column.  When the Oregon club used the TPS for weekly and flighted events, just about everyone was getting a penalty.  The latest version of the TPS has now been changed to alter the point distribution and penalties as a function of flight size and number of tournaments played.  And who is going to keep track of all of this?  Pros certainly don’t want to do it.  Volunteers were relied upon at the Oregon club.  This led to haphazard record keeping.  Some tournaments were recorded, others were not.  Members showed their dissatisfaction by declining to participate in tournaments.  The TPS was quietly killed at the end of the season.

In summary, Newport’s main fault appears to be he believes anything that contains a modicum of mathematics or statistics.  This should be a disqualifying fault for being a Wall Street Journal columnist.  Unfortunately, it is not.





[1] Note: Appendix E has been abbreviated here.  It actually shows odds up to a -10 net differential.