Sunday, January 10, 2016

Tournament Point System: Mistaking Motion for Progress?


The Nobel Prize for Peace awaits the inventor of a handicap system that ensures equity and identifies and punishes scoundrels.  The search for such a handicap system has been as productive as that for the Loch Ness Monster.  The futility does not stem from a lack of trying.   The USGA introduced the Reduction in Player Index for Exceptional Tournament Performance (Sec. 10-3) to penalize players who have handicaps higher than their potential would suggest.   Penalties under this Section are easily avoided by the skilled sandbagger, however, so it has little or no effect on tournament equity.1] 

The most common complaint faced by Tournament Committees is about the “same old winners.”   Certain players always seem to finish high in the tournament standings which draws the wrath of other players who feel the playing field is not even.  The USGA Handicap System provides one solution for the besieged Committee.  The Committee can reduce a player’s handicap if it believes the player is deliberately manipulating his scores (Sec. 8-4c).  Such reductions are rarely imposed because the Committee typically does not have the ability to collect and analyze large data sets for evidence of manipulation.  It may also want to avoid the personal confrontation of labeling a member as a “cheater.”  

To avoid the appearance of being feckless, some Committees have adopted a tournament point system (TPS) patterned after the point system developed by Dean Knuth, former Director of Handicapping at the USGA.  Under such a system, a player accumulates points based on where he finishes in the tournament.  After certain levels of points are accumulated, his handicap is reduced.   Initially Knuth’s system was applied to large stroke play events—i.e., more than 50 players.  If a player consistently finishes high in such events, the statistical evidence strongly suggests the player’s handicap is out of line and a reduction is warranted.
 
Clubs, however, do not usually hold large stroke play events.  The theory behind the TPS breaks down when a club only has a few tournaments a year, the flight size within the tournament is small, and the tournaments are sporadically entered by the membership.  In such cases, many problems can occur: 1) the TPS may not punish the guilty, 2) the TPS may punish the innocent, 3) the TPS can drive down tournament participation, and 4) the TPS can adversely affect the culture of the club. 

To examine the likelihood and severity of each of these problems, a case study of a club that recently adopted a TPS has been undertaken. The parameters of the adopted TPS fit those where the TPS is not likely to be effective.  Only four tournaments a year are part of the TPS.  The size of each flight ranges from 5 to 8 competitors.  And on average, a member only enters two tournaments a year.  The points awarded vary by flight size as shown in Table 1. The reduction in handicap for points accumulated over two years is shown in Table 2.   Now let’s examine each potential problem:


Table 1
Points Awarded by Finish

Finish
Flight Size up to 8
Flight Size Greater than 8
1st
2
3
2nd
1
2
3rd
0
1

Table 2
Reduction in Handicap for Accumulated Point Totals

Accumulated Points
Handicap Reduction
7
2
9
3
11
4
13
5

Not Punishing the Guilty – In theory, the more tournaments a player enters, the more points he should be allowed before a reduction in handicap is imposed.  When the number of tournaments considered is only eight, however, both the Knuth Point System and the TPS studied here do not consider how many tournaments are entered.  This anomaly in the TPS allows the unethical player to manage his tournament schedule to greatly decrease his chances of a handicap reduction.  If he only enters two tournaments a year, his chance of receiving a handicap reduction is quite small.  And since most tournament players do not enter all of the tournaments, the TPS effectiveness will be greatly limited (i.e., even if a player has a suspect handicap, he will not play in a sufficient number of tournaments to be caught by the TPS).

Moreover, the TPS does not stipulate the handicap from which any strokes must be taken.[2]  If a 15-handicap receives a two-stroke reduction, he only has to increase his handicap to 17 to avoid any punishment.  To counteract such manipulation, the club has to keep a record of the player’s handicap at each tournament, make a decision on whether the player’s current handicap has been manipulated, and if so, confront the player.   The TPS was supposed to relieve the Committee of such duties.  It does not.

Punishing the Innocent– Punishing the innocent has been a problem for modified TPSs.  One club in Oregon, for example, implemented a TPS where most members had a handicap reduction by the end of the season.  Knuth was so irate he wrote the Oregon Golf Association:

“I am quite happy when clubs use my Knuth Tournament Point System.  However, I am not happy when the system gets bastardized to the point that it doesn't work.  Would you please contact this club and tell them to use the system properly.”[3]

The club under study has followed the Knuth Point System, but it still has a problem.  When the flights are small (e.g., 6 or less), the TPS has a good chance of returning a false positive (i.e., identifying a player as a sandbagger when his good fortune is due to chance).   A typical player if he enters all 8 events has about a 13 percent chance of receiving a handicap reduction if the flight size is 6 (see Appendix).  Perhaps that is a reasonable price to pay if the TPS also captured the flagrant sandbagger.  As noted above, however, that may not be the case.  The TPS may impose costs without any accompanying benefit.

When the flight size is even smaller (e.g., 4), approximately 40 percent of the competitors would receive a reduction in handicap.  In this case, the TPS goes from a method of detecting suspect handicaps to one of ensuring “everyone wins a trophy.”   

 Tournament Participation – The TPS could suppress tournament participation.  The unethical player is unlikely to play in all eight events.  His absence should work to the benefit of ethical players.  But this assumes all events are of approximately equal value (e.g., a weekly sweeps).  That is not the case here.  While three of the tournaments require significant entry fees and award substantial prizes,[4] the fourth (Net Club Championship) does neither.  In the Net Club Championship the entry fee is $40 and the payouts are $100 for first and $75 for second.  .  Yet the winners of the Net Club Championship flights are assigned the same TPS points as the flight winners in the Member/Guest with a $1095 entry fee per team.  The prudent player who wants to minimize his chances for a reduction while maximizing his economic well-being will skip the Net Club Championship.  Players on the cusp may also skip tournaments to avoid a reduction in handicap. 

The TPS could increase participation if ethical players enter more tournaments believing they now have a fair chance.  While a stronger case can be made for declining participation, the actual participation is an empirical question.  This question should be answered at the end of year two of the TPS.

Club Culture –Adopting a TPS is an admission that “cockroaches live among us.”  This is never a good thing for a club.  Knuth developed his system for large statewide events (e.g., Pacific Northwest Golf Championship).  In such events, the players typically do not know one another or whether the Committee has reduced someone’s handicap.  This is not the case when the TPS is employed at the club level.  Point totals can be readily viewed by the membership.   Will club members hang a scarlet “S” for sandbagger on a player whose only sins were being lucky and playing in all of the tournaments?  Will the penalized player give up tournaments to avoid the grief?   Will a penalized player be forsaken for partner events?  Will the TPS end the carping over suspect handicaps that pervade every tournament?  (Answer: No.)  The effect of the TPS on the culture of the club is unknown, but it is something the Committee should examine closely in the early years of implementation.

Measuring the effectiveness of the TPS is difficult.  If no player gets a reduction, can it be said the TPS has eliminated sandbagging?  Alternatively, can it be argued the TPS was a waste of time?  Let’s look at the club under study to see how many players could get a reduction in handicap.

In implementing the TPS, the Club used tournament results from 2015.[5]  If the point totals are duplicated in 2016, there would be 6 players with 8 points, and 1 player with 10 points.  Four of the players are unlikely to repeat their 2015 performance due to regression toward the mean—i.e., a good year is more likely to be followed by a bad year than another good year.  The others can avoid the TPS reduction in handicap penalty by skipping some tournaments when they are on the cusp of a penalty.  If some players do receive a reduction, the Committee will have to determine if these players have been on the Committee’s radar or are innocents snared by their heavy tournament participation.  The tournament season will end in April 2016.  Then it can be determined if the adoption of the TPS was just another case of mistaking motion for progress. 

Update: June 3, 2016
1. One player received 8 points and 4 players received 7 points. At least 3 of the players participated in all of the tournaments.  The TPS penalty may be for participation as much as for performance.
2. Only one penalty was applied during the season.  That player was assigned a higher Index after the reduction that his Index in the previous tournament (i.e., his current Index had increased so the reduction was nullified).
3. Players were not informed of their point status unless they went to the pro shop and asked.  Most players were not aware to the TPS.  Participation was down in the tournaments, but it is not clear the TPS was a cause.  The Tournament Committee never explained why the TPS was kept such a secret.
4. The pro shop was charged with keeping track of the points.  The pro shop, however, made several mistakes.  It would be best to assign this task to a member.  Assistant pros are not all that interested in performing this task.  If the point totals were posted in the locker room, peer review would have caught the mistakes.
5. The Tournament Committee never evaluated the efficacy of the TPS at the end of the season.  Since the Committee never established any criteria for success, an analytical evaluation was not in the cards.  The Committee just assumed the TPS was working and was looking forward to next year.  And so it goes.

Update: March 6, 2017
1. A penalized player won his flight by 11 strokes even with a two stroke reduction in handicap.
2. Some players played have played to a  higher reduced handicap than the handicap with which they won previous events.
3. The tournament results would have been the same in the absence of the TPS.  I believe every player with a reduced handicap except one (i.e., see #1 above) skipped the handicap flights.

Is the TPS worth the effort?  That is for higher powers to decide.


Appendix
The Probability of Tournament Outcomes

In this analysis it is assumed a flight contains 6 competitors.  The player competes in 8 events.  He can finish first and gain 2 points with a probability of 1/6.  He can finish second and gain 1 point also with a probability of 1/6.  He can finish third or worse and gain no points with a probability of 2/3.  The various outcomes are shown in Table A-1.  In the third row, for example, the player earns 14 points by finishing first 6 times and second 2 times.  There are 28 ways this performance pattern can be achieved.  The probability of achieving 14 points in this exact manner would 28*(1/6)5*(1/6)3.  The player can also achieve 14 points with seven first place finishes and one third place or worse finish.  The probability of gaining 14 points is the sum the probabilities of the two ways to score 14 points.  The probabilities of gaining various point levels are shown in Table A-2.  The probability a player gains 7 or more points and receives a reduction is approximately 13 percent.  The probabilities of gaining 7 or more points for various flight sizes are presented in Table A-3.

Table A-1
Tournament Outcomes


Pts.

Ways
Tournament Number

1
2
3
4
5
6
7
8

16
1
2
2
2
2
2
2
2
2
15
8
2
2
2
2
2
2
2
1
14
28
2
2
2
2
2
2
1
1
13
56
2
2
2
2
2
1
1
1
12
70
2
2
2
2
1
1
1
1
11
56
2
2
2
1
1
1
1
1
10
28
2
2
1
1
1
1
1
1
9
8
2
1
1
1
1
1
1
1
8
1
1
1
1
1
1
1
1
1
12
28
2
2
2
2
2
2
0
0
10
56
2
2
2
2
2
0
0
0
8
70
2
2
2
2
0
0
0
0
6
56
2
2
2
0
0
0
0
0
4
28
2
2
0
0
0
0
0
0
2
8
2
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
13
56
2
2
2
2
2
2
1
0
12
168
2
2
2
2
2
1
1
0
11
280
2
2
2
2
1
1
1
0
10
280
2
2
2
1
1
1
1
0
9
168
2
2
1
1
1
1
1
0
8
56
2
1
1
1
1
1
1
0
7
8
1
1
1
1
1
1
1
0
11
168
2
2
2
2
2
1
0
0
10
420
2
2
2
2
1
1
0
0
9
560
2
2
2
1
1
1
0
0
8
420
2
2
1
1
1
1
0
0
7
168
2
1
1
1
1
1
0
0
6
28
1
1
1
1
1
1
0
0
9
280
2
2
2
2
1
0
0
0
8
560
2
2
2
1
1
0
0
0
7
560
2
2
1
1
1
0
0
0
6
280
2
1
1
1
1
0
0
0
5
56
1
1
1
1
1
0
0
0
7
280
2
2
2
1
0
0
0
0
6
420
2
2
1
1
0
0
0
0
5
280
2
1
1
1
0
0
0
0
4
70
1
1
1
1
0
0
0
0
5
168
2
2
1
0
0
0
0
0
4
168
2
1
1
0
0
0
0
0
3
56
1
1
1
0
0
0
0
0
3
56
2
1
0
0
0
0
0
0
2
28
1
1
0
0
0
0
0
0
1
8
1
0
0
0
0
0
0
0



Table A-2
Probability of Each Point Total

Points
Probability
Points
Probability
16
.00000
7
.06563
15
.00000
6
.10909
14
.00004
5
.14723
13
.00017
4
.18137
12
.00071
3
.17070
11
.00230
2
.14632
10
.00682
1
.07804
9
.01641
0
.03901
8
.03614




Table A-3
Probability of Gaining 7 or More Points by Flight Size

Flight Size
Probability of 7 or More Points
5
22.2
6
12.8
7
9.4
8
5.0





[1] See “Why Sec. 10-3 is Ineffective,”  www.ongolfhandicaps, April 28, 2014.
[2] The USGA’s Reduction of Handicap Index for Exceptional Tournament Scores penalizes a player if his handicap goes up after making two qualifying T-scores. The TPS does not. 
[3] E-mail from Dean Knuth to Kelly Neely, Director of Handicapping, Oregon Golf Association, June 27, 2005.
[4] Significant is defined here as the size of an average Time Warner cable bill. 
[5] Using tournament results from a period when the TPS was not operational biases the results.  Players are likely to behave differently when they know their performance is being studied, a phenomenon termed the Hawthorne Effect.  It does provide a baseline for tournament participation, however, from which to make judgments about the TPS’s effect.

Saturday, December 26, 2015

What's "Age" Got to Do With It?

A Member/Guest tournament has a condition that if a player’s combined age and handicap equals 90 or higher, he can play the forward tees.   The Tournament Committee explains the inclusion of age as a determinant of the tees played is intended to increase participation from an increasingly older membership.  (This begs the question of why the tee prize for participants is a quasi-walking golf bag.)  

This attempt at affirmative action for the elderly is misguided for two reasons.  First, it is “tacky.”  In a Member/Guest, do you really want to inquire about your partner’s age?  And how would you vet the age of a player?   Require a photo-id like he is checking into a Motel 6? 

Second, to ensure equitable competition, golfers should be measured by their ability (i.e., their Index) and not some extraneous attribute.  An example should demonstrate why age is an inappropriate criterion:

Assume Lance and his twin Larry are 16-handicaps from both the back and forward tees—there is only a small difference between the two Slope Ratings.  Since they are both 75, they qualify to play the forward tees. There is a one stroke difference in the Course Ratings so each would be assigned a 15 handicap for the tournament.  Since the event consists of a series of nine hole matches, Lance and Larry would each be assigned an 8 handicap for each match (15/2 = 7.5 which is rounded to 8).

Twins Justin and Jim are a couple of years younger than Lance and Larry, but are also 16 handicaps.  They too would play the back tees as 8 handicaps for their match.   Both teams are of equal ability and handicap, yet Lance and Larry would have the advantage of playing from the forward tees.   This would not be fair on its face.  (Justin and Jim could be assigned ½-stroke, but that presents other problems in equity, ease of administration, and user friendliness not discussed here.)

Competitions between players from different sets of tees are inherently inequitable.  The handicap adjustment under Sec. 3-5 , Players Competing from Different Tees, may not be equitable even for singles stroke play, and there is no evidence  its makes for equitable competition in four-ball.[1]  To obtain the best chance for a fair competition, all matches within a flight should be played from the same set of tees.  This would eliminate the need to invoke Sec. 3-5, increase equity, and ease the burden on tournament administrators. 

If there is a playoff among flight winners for an overall championship, it too should be conducted from one set of tees.  If three sets of tees are used, for example, the middle set could be selected for the playoff.  This would minimize grumbling from those who might otherwise have to give a stroke and distance to their less skilled opponents.




[1]  The theoretical problems with Sec. 3-5 are discussed in “The Problems with Section 3-5,” www.ongolfhandicaps.com, March 4, 2014.  A small empirical study reports Sec. 3-5 did not correct for the difference in tees.  In this case, the players competing from the back tees were at a disadvantage--see “Is Your Tournament Equitable,” www.ongolfhandicap.com, October 22, 2012.

Wednesday, December 2, 2015

Eliminating the Blind Draw-Continued

In commenting on a previous post (“Eliminating the Blind Draw' October 31, 2012 now published with the date October 22, 2019), an astute reader suggested an alternative method:

My club had been messing around with options to handle this problem until I suggested that we should try using the second best score as the score of the third player. For example, in Stableford Scoring if Player A gets 2 points, Player B gets 2 points, and Player C gets 1 point, then the team gets 6 points.
This has now been accepted by all the informal groups in the club as results seem to suggest that teams of 3 win a prize in proportion to the number of teams of 3 or 4 entered. The number of prizes is in proportion to the size of the field.  I would be interested to know if you thought there was any mathematical support for this.

Let’s look at a simplified example to get a preliminary judgment of equity.  Assume only four scores are possible (3 = net birdie, 2 = net par, 1 = net bogey, and 0 = net double bogey or worse).  Further assume the probability of each score is the same (i.e., 1/4).  The question is whether the expected second best score of the threesome is greater, equal, or less than the expected score of a fourth player.  Column 4 of Table 1 shows the probability of each outcome for the threesome.[1]

Table 1
Probability of Outcomes

(1)
 Highest
(2)
 Second Best
(3)
 Third Best
(4)
Probability 
3
3
3
1/64
3
3
2
3/64
3
3
1
3/64
3
3
0
3/64
3
2
2
3/64
3
2
1
6/64
3
2
0
6/64
3
1
1
3/64
3
1
0
6/64
3
0
0
3/64
2
2
2
1/64
2
2
1
3/64
2
2
0
3/64
2
1
1
3/64
2
1
0
6/64
2
0
0
3/64
1
1
1
1/64
1
1
0
3/64
1
0
0
3/64
0
0
0
1/64

The expected second best score is the sum of the value of each score multiplied by the probability of each score occurring:

                Expected Second Best = 3·(10/64) +2·(22/64) + 1·(22/64) + 0·(10/64) =  96/64 = 1.5

The expected score of the fourth player is:

                Expected Score of the Fourth Player = 3·(1/4) + 2·(1/4) + 1·(1/4) + 0·(1/4) = 1.5

Under all of the assumptions made, using the second best score would be equitable.  This result is not surprising.  The expected second best score is also the expected average score of any player.   (It would not be equitable under a modified Stableford System where the difference between hole scores is not constant.  For example, assume a net birdie is now worth 5 points while all the points for the other scores remain the same.   The expected second best score would be 1.81.  The expected score of the fourth player would be 2.00.) 

To get a more realistic estimate of any advantage, the number of possible hole scores is expanded as shown in Table 2.  Since not all hole scores are equally likely, however, Table 2 presents reasonable probability estimates for an easy course (Course Rating less than par) and a more difficult course.

Table 2
Probability of Hole Scores (Stableford Points) for Easy and Difficult Courses



Course


Net Eagle(4)


Net Birdie(3)


Net Par(2)


Net Bogey(1)
Net Double Bogey or Worse(0)
Expected Score
Par = 72
Easy
.10
.30
.40
.15
.05
67.5
Difficult
.05
.20
.45
.20
.10
73.8

Using the same methodology presented in the simplified example, the expected score of the second best hole score of the threesome and the expected score of the fourth player can be estimated.  Those holes scores are shown in Table 3.

Table 3
Expected Holes Scores


Course

2nd  Best

Fourth Player
Threesome Advantage per Hole
Easy
2.27
2.25
0.02
Difficult
1.91
1.90
0.01

Table 3 shows a small edge for the threesome on both courses.   Given all of the uncertainties embedded in the model (e.g., probability assumptions, assumption that all players have identical scoring patterns and variances), such a small edge is not significant.  And given the random nature of team scoring, such a small edge would be hard to notice without a large number of trials.  If threesomes are winning in proportion to their numbers as the reader suggests, there is no reason to believe his method is not equitable. [2]

The one remaining problem is how prize money is distributed.  If first place wins the same amount regardless of team size, the threesomes have a big edge.  A threesome is just as likely to win, but only has to split the prize money three ways instead of four.  Since the reader was clever enough to devise an equitable competition, I am sure he has solved this problem too.




[1] The probability of an outcome is the probability of any one outcome multiplied by the number of ways that outcome can be achieved.  For example, the outcome 3,3,2 can be achieved in three ways—Player A gets a 2 and the other players get  3s, Player B gets a 2 and the other players get 3s, Player C gets a 2 and the other players get 3s).   The probability of any one outcome is 1/4·1/4·1/4 = 1/64.  Therefore, the probability a team has two 3s and a 2 is 3x1/64 =3/64. 

[2]  Probabilities for courses can be constructed where there is a significant difference in hole scores between the second best score and the score of the fourth player.  A course where net bogey and net double bogey are the likeliest scores will favor the foursome.  A course where net eagle and net birdie are the likeliest scores would favor the threesome.  Neither course is likely to exist in the real world.