Monday, September 24, 2012

The USGA - The Dishonarable Company of Golfers


Introduction - The history of golf is replete with stories of golfers calling rulings on themselves.  Bobby Jones is noted for calling penalty strokes on himself for violations that no one else saw.  One such call may even have cost him the 1925 U.S. Open.  Indeed it is the honor in the game that most distinguishes golf from its competitors on Sunday television.  In football and baseball, it is all right to claim you caught the ball even if you didn’t.  Lying is an accepted if not an integral part of those games.
            Given the history and tenets of golf, one would expect the United States Golf Association (USGA) to exemplify all of the virtuous attributes ascribed to the game.  That certainly was my expectation, but it was dashed on the rocks of reality by the USGA’s reaction to my paper questioning the value of the Slope System (see Is the Slope System Worth the Effort - posted 9/20/2012).  The USGA engaged in lying, fraud, and character assassination in an attempt to bury my work.  In the USGA’s case, the bureaucratic instincts of survival clearly overwhelmed any inclination to do the honorable thing.

 In the Beginning - My research on the Slope System was born out of curiosity.  The Slope System was hard to use and difficult to understand.  To be implemented it had to have significant benefits.  I wrote to the USGA asking for studies of the Slope System that demonstrated its efficacy.  In response, the USGA only sent a description of the Slope System. 

Having observed bureaucratic behavior for over ten years as a policy analyst at the Rand Corporation, I found the USGA’s failure to send any studies probably meant there were not any.  After all, if there were strong scientific support for the Slope System there should have been no hesitancy to release it.

            Given the USGA’s reluctance, I went ahead with two tests of the Slope System reported in Is the Slope System Worth the Effort- posted 9/20/2012.  The two tests were not meant to be definitive, but to 1) demonstrate a methodology for testing the Slope System and 2) determine if more research needed to be done. The research concluded, qualified by the limited sample size, that the Slope System was a second order improvement in the handicap system at best. 

            The report “The Slope System: A Review and Evaluation” was submitted to the USGA for its review.  The USGA’s response at this was encouraging.  Francis Scheid of the USGA’s Handicap Research Team wrote:[1]

It is always a pleasure to find that someone else is involved in handicap research and your work shows the competence that the subject deserves…In summary, you have obviously made an important contribution.  I hope you can avoid the temptation to go for the front page of the Washington Post and suggest that instead we continue to refine what is basically a sound idea.

            This was in essence a pat on the head.  I might have accepted it if Scheid had not gone on with some rather silly reasoning in defense of the Slope System:

The reality of the slope concept should not be doubted.  Common sense points directly to it… (If) A outbowls B by 20 pins a string on average, and B bests C by 30, then Las Vegas will expect this field of three to show gaps of 60 and 90 strings over a three string event.  (This example)…illustrates a general principle, that differences in ability are magnified by increasing the level of difficulty of the assignment…”
 
            In response to Mr. Scheid, I had to point out his example did not support the concept of a Slope System.[2]   To be kind, I said nothing about his choice of a bowling analogy nor did I take it as an insult.  I did say:

Las Vegas is simply assuming an independence among trials (i.e., strings), and computing the expected value of three strings to be three times the expected valued of one string.  Your implied logic that if this bowling example is true, then the Slope System is valid is simply false. The Slope System tried to deal with how to standardize a trial (e.g., what should your bowling handicap be if the lane was lengthened by 20 feet?), rather than cumulative differences over a number of trials
 
            Apparently any criticism was too much for Mr. Scheid.  He characterized my effort as hate mail[3] even though my letter concluded with:

I have the greatest respect for the USGA and for your work for the past twenty years in studies of handicapping.  For the love of the game you dedicated your talent to making improvements to ensure more equitable competition.  I hope you can believe my relatively modest effort was undertaken with the same goal in mind.

Not exactly my definition of hate mail type material, but perhaps I have not led the cloistered existence of Mr. Scheid.

            But that was not enough for the USGA.  Dean Knuth, the Director of Handicapping for the USGA, had to pile on: [4]

“I share Dr. Scheid’s repulsion of your style of correspondence and also question who would even be interested enough in your handicapping work to give you the ‘economic advantage’ that you admit to seek.”

Of course I had not sought and economic advantage – though that term really has no meaning – since there is no money to be made in research on golf handicapping.  Knuth tried to discredit my research by impugning my integrity. It was something he would repeat on several occasions in the future. 

While ideas certainly should have been debated on a level playing field, the USGA chose instead to shovel dirt on a minor opponent.  And remember these are the men who so often remind us that they are guardians of the integrity of the game.

The World Scientific Congress of Golf - I had also sent my research paper to Dr.
Alastair Cochran, author of The Search for the Perfect Swing.[5]  He wrote back:

If repeated with a similar outcome elsewhere, the results of your study of play from two different sets of tees certainly would cast doubt on the need for the Slope System….You may wish to consider submitting one or more of your papers to the Second World Scientific Congress of Golf, to be held in St. Andrews in July 1994.  I have taken the liberty of putting your name on the list to receive advance notice of this.

Since Cochran was one of the leaders of applying science to the study of golf, I was encouraged by his generally positive response.  I was particularly excited by the opportunity to share my research at the Congress.

            The paper was submitted to the Congress for review in November of 1993.  The Conference Director, Martin Farrally, informed me that all decisions would be made by February 15, 1994.  I also learned at this time that the USGA was a co-sponsor of the Congress.  While I found this troubling, I did not believe the USGA would be so blatant as to use their power have my paper excluded from the Congress.  I was wrong.

            The paper was rejected by the Congress, but the reviewer who rejected the paper demonstrated both his bias and ignorance of the subject area.  The reviewer carped about the small sample size.  This argument was not compelling for the following reasons:

1.   The Congress Has Published Papers With Far Less Data and Smaller Sample Sizes - At the First Congress, Stroud and Riccio, members of the USGA Handicap Procedures Committee, published a paper entitled "Mathematical underpinnings of the slope handicap system."  While they described the system and extolled its benefits, they did not present one shred of empirical evidence that the system would increase the equity of competition.  Moreover they were allowed to cite articles that are not available to the general public, but only to select members of the USGA community.  This is hardly consistent with the principles of scientific inquiry that the Congress should represent.

            Dean Knuth, Director of Handicapping of the USGA, had a paper entitled "A two-parameter golf course rating system" presented at the First Congress.  Again there is no research on the benefits of the Slope system, but merely a description of how ratings are done.  For the course rating system, Knuth only used data from seven U.S. Amateur Championship courses in his sample.  He then threw out 41 percent of the hole scores sampled without providing a sound reason.  The estimates of the coefficient of the bogey rating model were based on the scores of only 60 players!  If such a skimpy sample is of sufficient size on which to base the entire USGA Handicap System, then my sample size should be adequate to question the efficacy of that system.

                  To criticize my paper because of its limited sample size is to hold it to a higher standard than research submitted by the USGA.

 

2.   The Sample Size is Not that Small -  The inter-club test of the Slope System involved most courses in Southern California.  This is probably the largest such team competition held anywhere.  And any statistical problems certainly do not stem from sample size.

            My study concluded that tough courses (i.e., high slope ratings) did not have the big advantage that the Riccio and Stroud paper postulated.  Riccio and Stroud only gave theoretical arguments, of course, and had no empirical evidence on which to base their claim.  My results were based on over 100 team matches in each year of the study.  Therefore, a reasonable person would have to conclude that it is not sample size that is the deciding factor in the selection of papers.

 

3.    The Value of the Paper is in Presenting a Methodology that Can be Replicated by Others to Test the Merits of the Slope System -  My paper presents a methodology for testing the slope and course ratings among various sets of tees at a course.  I believe this is important for golf associations in validating the accuracy of their ratings.  I have demonstrated this methodology for only one course.  While my result raises questions about the Slope System, it cannot serve as a general indictment as I readily admit.  With my methodology, however, golf associations could test the slope ratings at thousands of courses.  By rejecting my paper, however, the associations are deprived of the methodology to perform such tests.

             The reviewer also claimed that I had erred since the Slope System was introduced in Southern California in 1990 and not 1991 as I reported in my paper.  I gave Farrally the phone number of the Southern California Golf Association (SCGA) so he could check who was right.  Farrally never made the call.  Since the SCGA had both supplied the data for the study and reviewed the results no such error could have occurred.

            If Farrally was embarrassed by his reviewer’s mistake, he did not show it.  Instead he went on to claim independence from the USGA:[6]

“May I assure you there is no politics involved in the rejection of your works as a paper.  We make no secret that the R & A and the USGA are major sponsors but the congress is run by a university with full academic freedom.” 
 
            Farrally’s claim of independence rings hollow.  Without the USGA and R & A sponsorship, the Congress and Farrally’s job as director would not exist.  Moreover , the USGA is not without an institutional interest.  The USGA also has an economic interest in maintaining its omnipotence on all handicapping matters.  The USGA sells handicap services much like Ping or Cobra sells golf clubs.  Therefore, the Congress conference has the appearance, if not the reality, of a conflict of interest in the area of handicapping. But let’s examine the evidence.

            Every paper submitted by the USGA was accepted.  These papers were hardly earthshaking.  L.J. Riccio of the USGA Handicap Procedures Committee did a study of Tom Watson’s play at the US. Open – The Ageing of a Great Player: Tom Watson’s Play in the U.S. Open from 1980 to 1993.  A longitudinal study of one player cannot say anything about the effects of ageing for the population as a whole.  Since the venue and weather change each year, the study is of even limited value in assessing Tom Watson’s performance over time.  Therefore, the criticism of my paper because of its limited sample size, was simply a ruse to the USGA bidding.

Dean Knuth submitted a paper that was just a description of the Adjustment for Exceptional Tournament Performance.  Knuth presented no evidence that the new procedure had any effect on the equity of competition.  Though I was criticized for not having sufficient empirical findings, Knuth paper was embraced even though it had no findings.

Finally, Francis Sheid of the USGA submitted a paper, The Search for the Perfect Handicap, that by its title was not the definitive research that was demanded of my paper.  It was to turn out that Scheid’s paper was just a rehash of a paper he had written 16 years earlier.  A good reviewer would have refused Scheid’s paper because it was not original.  Since USGA paper’s probably got minimal scrutiny at best, it was accepted.

In taking the USGA’s money, the Congress received a batch of mediocre papers of questionable scientific merit and little policy value.  The Congress made a bad bargain that exposed that it is a creature of politics and not science.[7]

            Farrally tried to defend the review process:

“Your paper was reviewed by two separate academics, both of them well respected statisticians working in Universities, on British and one American.  Because of the sensitivity of this issue I am prepared to inform you that the American reviewer was not Frank Scheid.  I can also confirm that having spoken to both reviewers, I have been assured that the content of neither review was disclosed to anybody else but myself.”

 
            Farrally’s defense strongly implies the reviewers were independent of the USGA.  I learned, however, that my paper was reviewed by Burt Lieberman who is not a statistician but an associate professor of mathematics at Polytechnic University – a commuter school in Brooklyn, New York.  Mr. Lieberman has no publications in either econometrics, quasi-experimental design, or the measurement of human performance in golf.  He has, however, been a consultant to the USGA since 1979!  I guess this little fact slipped Farrally’s mind.

            I further learned that the review of my paper had been discussed at length with members of the USGA Handicap section before being sent to Farrally.  This is easily proved by a letter Dean Knuth sent to the head of the Connecticut State Golf Association:[8]

“An independent Congress of Golf review committee composed of professors from Great Britain and the United States ‘peer reviewed” his (Dougharty’s) report and found it unfit for their proceedings.”

 
            Now how would Knuth know of the reviews unless he had contact with the reviewers?  Farrally continued to maintain that “neither of the review were seen by anyone else except myself and Alastair Cochran.[9]”  Did Farrally take the most direct step and ask Knuth how he knew of the results?  Of course not.  This would only embarrass the USGA and was something Farrally could not afford to do.  Farrally continued to maintain the USGA had nothing to do with the review even against overwhelming evidence to the contrary.  Farrally was certainly a good soldier in the cause of the USGA.  It is not surprising he continues as Congress Director and the USGA continued its financial support.

            The response of David Fay, Executive Director of the USGA, was equally disappointing.  After Fay was sent all of the supporting evidence he wrote:[10]

”In our judgement (sic), we concluded that yet another reply to you would accomplish nothing.  We know your feelings; we happen to disagree.”

 
Though Fay must have been aware of the USGA’s involvement, he took no corrective steps nor did he offer an apology for the actions taken by his organization.  If the rejection of my paper was not embarrassing enough for the USGA, the acceptance of Francis Sheid’s was even more so.  That sordid affair is detailed in the next post, The Francis Scheid Affair.



[1] Letter to the author from Francis Sheid, February 20, 1991.
[2] Letter from the author to Francis Scheid, February 27, 1991.
[3] Letter to the author From Francis Scheid, March 18, 1991.
[4] Letter to the author from Dean Knuth, April 18, 1991.
[5] Cochran, Alastair and Stobbs, John, The Search for the Perfect Swing, The Bootlegger (reprinted), Grass Valley, CA 1989.
[6] Letter to the author from Martin Farrally, February 4, 1994.
[7]  The Congress showed a similar bias in a paper submitted by a writer from one of the sponsoring organizations, Golf Digest (Australia).  The paper contained no substantive research, and was generally a lament about the poor television coverage of Australian tournaments.  The writer concluded his paper with, “As journalists continue to be so badly paid, golf writers will continue to look for other public relations work to supplement their paltry incomes.”  I assume this writer’s paltry income was supplemented by a free trip to the St. Andrews, the Congress site.  Others used the Congress as a tax write-off.  U.S. taxpayers were picking up part of the tab for many Congress participants to play the Old Course.
[8] Letter from Dean Knuth, USGA Director of Handicapping to Russell Palmer, Executive Director of the Connecticut State Golf Association, August 25, 1994.
[9] Letter to the author from Martin Farrally, November 22, 1994.
[10] Letter to the author from David Fay, December 7, 1998.

Thursday, September 20, 2012

Is the Slope System Worth the Effort?


 
(Note: This is the first in a series of four posts examining possible deficiencies in the Slope Handicap System and detailing efforts by the USGA to suppress criticism of the System. This post presents an empirical study on the efficacy of the System.  The next post, The USGA - The Dishonorable Company of Golfers, documents how the USGA went to great lengths to bury the study.  The third post, The Francis Sheid Affair, presents an example of the poor quality of handicap research performed by the USGA.  Lastly, How Accurate is the Slope System, uses USGA data to show the Slope System may be doing more harm than good to the equity of competition.)  
 
Introduction - The Slope System for golf handicapping has been introduced by the United States Golf Association (USGA) as a more equitable method of assessing the relative abilities of competing golfers.  This introduction, however, has not been without cost.  There has been increased data handling costs for associations maintaining handicaps, to say nothing of the time and effort spent by individual golfers and tournament committees converting indexes to handicaps.

            Are these costs commensurate with the benefits from the Slope System?  Unfortunately,. The USGA has not released any research validating the Slope System.  Instead they have produced normative articles explaining the reasoning behind the Slope System.  In essence, the golfers of the United States are asked to accept the Slope System as an of religious faith.  It is not without irony that the leading proponent of the Slope System within the USGA, and its Director of Handicapping, is known as the “Pope of the Slope.”[1]

            It is difficult to imagine why the USGA has stifled scientific inquiry on the Slope System.  Would the USGA, for example, recommend a type of grass and not make the field test public?  Of course not.  Yet the USGA has forced the Slope System on the country without providing any compelling evidence of its efficacy.

            The purpose here is to start the debate on the merits of the Slope System.  This research empirically tests the size and direction of the impact of the Slope System on the equity of competition.

            To set the state for later analysis, a brief description of the assumptions underlying the Slope System is presented.  Then two tenets of the Slope System are tested.  First, the Slope System assumes the handicap of a given player would be an increasing function of the slope rating.  Data from a club in Southern California are used to see if this “slope effect” (i.e., higher handicaps increasing proportionately more that lower handicaps on difficult courses) can be measured.

            Second, the Slope System presumes that players from “high slope” courses should have an advantage over players from “low slope” courses before the advent of the.  Inter-club matches from Southern California are examined to see if indeed players from the more difficult courses had the advantage slope theory would predict.

            Lastly, conclusions are drawn from the research to address the title of this article – Is the Slope System Worth the Effort?

The Slope System - Before the Slope System was introduced, handicaps were based on the difference between the player’s score and the course rating.[2]  The course rating was determined by the difficulty of the course for a scratch golfer.  Difficulty was estimated by a distance formula and then modified by obstacle factors such as topography, width of fairways, out-of-bounds, water hazards, trees, bunkers, and size of greens.

            This handicapping system assumed a player’s “expected” score would exceed the course rating by the same number of strokes regardless of the difficulty of the course for the non-scratch player.  If a player scored 10 strokes over the course rating at the local municipal golf course, he would be expected to exceed the course rating by 10 strokes from the back tees at Pine Valley.

            Proponents of the Slope System argued, however, the course rating may not be a good measure of difficulty of a course for a bogey golfer (i.e., a player with a 18 handicap).  Some hazards can be disastrous for the bogey golfer, but be of little consequence to the scratch player.  A water hazard in front of the tee, for example requiring an 180 yard carry may not be noticed by the scratch player, but may cause the body golfer to use up his supply of water balls.  Therefore, it was argued, another parameter should be used in handicapping to indicate the difficulty for the bogey golfer.  That parameter is the “slope” rating of the course.

            The slope system is designed to give the bogey golfer more strokes the higher the slope rating.[3]  The Slope System assumes players coming from easier courses need more strokes than their handicaps would provide when they travel to a more difficult course (difficulty will be assumed to be synonymous with high slope ratings for this discussion).  Proponents of the Slope System give the following example of two courses: Perfect Valley – a medium difficulty course with a moderate rating of 70.0, and 2)Panther Mountain – a difficult test of golf with a high course rating of 72.0.[4]

            Figure 2-1 is a hypothetical plot of scores against handicap for players from Perfect Valley and Panther Mountain playing at Panther Mountain.  The graph shows players from Perfect Valley are at a disadvantage to players with the same handicap from Panther Mountain.  This disparity increases as the handicaps of players increase.
 
Fig. 2-1 - Average Score Versus Handicap for Play at Panther Mountain

 

            Under the Slope System, every course receives a slope rating indicating the difficulty of the course.  For example, Panther Mountain might be rated at a 139 slope while Perfect Valley is rated at 113.  The old handicap differential  is converted to an index differential by multiplying the ratio of the standard course slope rating of 113 and the slope rating of the course having been played.  In equation form:

            Index Differential = (Adjusted Score - Course Rating) x (113/Slope Rating)

            A player’s index is computed by taking the average of the l0 lowest differentials out of the last 20 rounds, multiplying by .96, and then truncating the result after the first decimal.  A player’s handicap is found by multiplying the player’s index by the ratio of the slope rating of the course to be played and the standard slope rating of 113 – the resulting product is rounded to the nearest integer to find the handicap.

            Under the Slope System, players from Panther Mountain would tend to get fewer strokes when playing Perfect Valley that they do when they play their home course.  While the Slope System solves the problem depicted in Figure 2-1, three nagging questions remain:

1.    Is there a slope effect?  Figure 2-1 showed that Perfect Valley members could not play to their handicap at the tougher Panther Mountain.  Is that indeed the case?  No empirical evidence validating the slope effect has ever been published by the USGA.  If the slope effect does not exist, then the Slope System will lead to less equitable competition.

2.    Is the assumption of linearity in the Slope System reasonable?  The slope rating is based on the difficulty of a course for the bogey golfer.  The Slope System assumes the ratio of difficulty between course is linear over all levels of ability.  For examples, assume that a player from Perfect Valley had an index of 3.0 for a handicap of 3 when playing Perfect Valley.  At Panther Mountain under the Slope System the player’s handicap would be 4.  Realistically, the playing hazards for a 3 handicap should by the same as for a scratch player.  To give this player an extra stroke at Panther Mountain because of difficulties encountered by the bogey golfer is an assumption embedded in the Slope system.

If the slope rating adjustment is justified for high handicappers but not for low handicappers, then the Slope System would lead to less equitable competition for the latter group of players.

3.    Can the Slope Rating be estimated accurately?  Even if the slope phenomenon exists, if it cannot be estimated with sufficient accuracy, the Slope System could lead to less equitable competition.  For example, assume two courses actually had the same slope rating.  If one course was rated higher than the other, then that club would be at a disadvantage when playing members for the course with the lower rating. 

            These questions are difficult to answer, but the empirical studies can shed some light on these issues.  

 

A One-Course Test of the Slope System - To test the slope hypothesis represented in Figure 2-1, it would be necessary for members of one course (e.g. perfect Valley) play a large number of rounds at another course (e.g. Panther Mountain).  Then statistical methods could be used to test whether scores of the visiting players followed the slope line shown in the figure.  Such a test has two problems.,  First, few clubs would want to give-up so many starting times for visitors from another course.  Second, the scores by the visiting players are likely to be higher because they lack local knowledge.  If there is an upward bias in scores due to a local knowledge factor, it would be difficult to isolate the “slope effect.”

            One way overcome these two problems is to have players from the same club play two different sets of tees at their home course.  In essence, one set of tees becomes Panther Mountain and the other set Perfect Valley.  By using the same course for the test, moreover, the bias of “local knowledge” is eliminated. 

            To test the slope hypothesis data from a private club in Southern California was collected.  Of approximately 500 members, 59 members had a sufficient number of rounds (20) within one calendar year from both sets of tees to compute a handicap.  Handicap were computed for each player from the scores from the blue tees (slope 119, 6520 yards) and from the white tees (slope=111, 6059 yards).  Handicaps were computed to the nearest tenth rather than the nearest integer in order to preserve any minor variation that may be due to the slope phenomenon.

            A player’s handicap from the white tees was plotted against the difference between the player’s blue tee and white tee handicaps as shown in Figure 2-2.  Under slope theory, these points should follow a line with a zero intercept (i.e. the scratch golfer is not affected by the slope rating) and with a slope equal to the ratio of the two slopes minus one (i.e. ,119/111 -1) or .072.  That is the handicap from the blue tees should exceed that from the white tees by approximately 7 percent.


Figure 2-2 - Handicap Difference Versus White Tee Handicap

 

In equation form:

 

            DIFF = a + b WTH                                                                                                     

 

Where,

                                 DIFF= Difference between a player's blue and white tee                     handicap

                                 WTH= Player's Handicap from the White Tees

 

The coefficient "b" represents the slope effect.  .

            The coefficients of this model were estimated using standard linear regression techniques.  The estimated equation was:

 

            DIFF= .89 - .024 WTH                                                                                               

 

            The estimated model shows a slope of -2.4 percent rather than the 7 percent slope theory would predict.  The hypothesis that the true slope is 7 percent can be rejected at the 1 percent level of significance.  An alternative hypothesis is there is no slope phenomenon at work so that the coefficient "b" should be zero.  This hypothesis cannot be rejected at the 20 percent level of significance.  In other words, it is likely that there is no slope effect present.

            The introduction of the Slope System has not led to more equitable competition at this club.  White tee players are given additional strokes when playing the blue tees.  Similarly, strokes are taken from blue tee players when they play the white tees.  Neither adjustment can be justified by the data.  The Slope System, in this one case, has made competition less equitable.

            Clearly the results form one club out of thousands in not definitive.  It is hoped, however that the results will encourage the USGA and other golf associations to conduct similar studies.  The methodology used above would be helpful in determined if the slope effect existed and the accuracy of the relative slope ratings among different sets of tees at a course.

 

An Inter-Club Test of the Slope System - A purported major benefit of the Slope System is its elimination of the handicap advantage that players from difficult courses enjoyed over players from easy courses.  To test for the existence of this advantage, data from inter-club team matches sponsored by the Southern California Golf Association (SCGA) were used.  If the slope theory is valid, the courses with higher slope ratings should have had an advantage before the introduction of the Slope System.  The introduction of the Slope System should have erased any advantage belonging to the higher slope courses and led to more equitable competition.

            Data from three years of team matches were analyzed to test the influence of the Slope System on team match results.  In the years 1988 and 1989 the SCGA was not on the Slope System.  Therefore, an advantage for teams from higher slope courses would be expected in those years.  With the introduction of the Slope System in 1990, the advantage would be expected to disappear.

            Before examining data from the team matches, an explanation of how they are scored is in order.  A total of 96 points is available in two matches played on a home-and-home basis (48 points at each club).  The impact of the Slope System was estimated using a model where the number of points won over 48 (i.e., the number of points won over those expected by chance) is considered to be a linear function of the difference in course Slope Ratings.

            Slope theory would predict that courses with higher slopes would do better in the years before the Slope System was introduced.  The size of the impact would be measured by the coefficient of the variable measuring the difference in slope.  In equation form the model was:

 

            (Score1 - 48) = a + b(Slope1 -Slope2)                                                                      

 

Where,

                        Score1 = Points won by Course 1 Against Course 2

                        Slope1 = Slope of Course 1

                        Slope2 = Slope of Course 2

 

            The SCGA conducts competition for both Thursday and Saturday teams.  The data from three years for both teams were fitted to the model using standard linear regression techniques.

            In 1988 (a pre-Slope System year) the coefficient measuring the impact of the slope rating (i.e., "b") was not statistically significant.  That is, teams from courses with higher slopes did not outperform teams from courses with lower slopes.

            In 1989, the slope coefficient was significant for both Thursday and Saturday Team results.  The impact was small, however.  The model predicts a ten-point differential in slope (e.g., 128 versus 118) would lead to only a total differential of three points over the 96 point match.

            In 1990, the first year of the Slope System, the slope effect would be expected to disappear.  That is, the Slope System should have equalized competition.  The slope coefficient was not significant (10 percent level of significance) indicating that the slope was not a factor in explaining match results.

            Any conclusion concerning the efficacy of the Slope System depends in part on which two years are being compared.  Between 1988 and 1990, the introduction of the Slope System had no measurable impact on the results of team play.  Between 1989 and 1990, however, the Slope System appears to have made a minor contribution to more equitable competition.  The inconsistency and the small size of the impact, however, raises doubts about the net benefits of the Slope System.

            Another test of the impact of the Slope System is to examine whether “higher slope” teams were more successful in team play.  Table 2-1 presents the winning percentage (i.e., where the team score 49 or more points) of higher slope competitors.

Table 2-1

Winning Percentage of Higher Slope Teams

 
 
Winning Percentage
 
Year
 
Team
Slope Difference
>0
Slope Difference
>9
1988
Saturday
.469
.607
1988
Thursday
.500
.516
1989
Saturday
.533
.632
1989
Thursday
.504
.628
1990
Saturday
.541
.520
1990
Thursday
.508
.607

 

            The winning percentage is computed for two cases: 1) the slope difference is greater than zero, and 2) the slope difference is greater than nine  In the case where the slope difference is greater than zero, teams with a higher slope do no not have a significant edge in any of the years studied.

            When the set of matches is reduced to those where the difference in slope is greater than nine, the teams from the higher slope courses do better than chance would predict.[5]  But, most important, it appears they do better over all years.  That is, any slope effect was not corrected by the Slope System.

            In summary, the Slope System has not led to any large or consistent improvement in the equity of team play competition.

 

Conclusions - No definite conclusion on the Slope System can be drawn from the two tests reported here.  The research did show, however, that little benefit has been brought to the equity of competition in the two instances.

            In the “one-course test” of the Slope System problems in measuring the slope were revealed.  Since the Slope System relies in large part on human judgment, the possibility of error seems quite large.  Can the formulae used to calculate the slope really distinguish between courses?  Can ratings committees consistently and reliably distinguish between a 130 and 125 slope rating? 

The results of this research suggest that such distinctions cannot be made with precision.  In the one course test no actual difference in slope was found, the slope ratings differed by eight points.  In the test using inter-club matches, any effect of the Slope System was marginal at best.

There may be something else at work other than the slope effect.  In the inter-club matches, teams with high slopes (i.e., greater nine point differential over their competitor) continued to do well even after the introduction of the Slope System.  One possible explanation is that higher slope course may also have a “local knowledge effect” (LKE).  LKE means that players score gradually does down with the number of times a course is played until a steady state is reached.  High LKE are associated with courses where there is a large penalty for choosing the wrong club, where there is strategic value in being on certain parts of the fairways and greens, and where breaks in the green are not easily read. Figure 2-3 show the pattern of scores for courses with high and low LKE.  If the slope rating is based on the steady state score then the LKE may overwhelm any contribution made by the Slope System.

Figure 2-3 - Score Versus the Number of Rounds Played
 

            The failure of the Slope System may also be due to the wide variety of “bogey golfers.”  Some bogey golfers are slicer while others are hookers.  Some hit the ball short and straight while others hit long and wild.  A course should have a slope rating for each category of bogey golfer, but instead a slope rating for a mythical average bogey golfer.  This implies the slope rating will be too high for some and too low for others.

             For those for who the slope rating is too low, the Slope System will underestimate their ability.  An example will make this clear.  Assume a golfer who slices the ball on a course where all the trouble is on the right.  The slope rating for the average golfer at this course would be 120, but for the slicer the slope rating should be 130.  His index differential is found by adjusting is handicap differential by the ratio of 113/120.  His index will then be higher than if the “true” slope ratio of 130 were used.  His ability on away courses is underestimated (i.e., his handicap is overestimated)f by the Slope System.  The effectiveness of the Slope System depends in part upon a uniformity among bogey golfers.  The USGA has never presented any evidence that this assumption is met.

            The Slope System brought more complexity to the handicapping system, but not necessarily more accuracy in evaluation the ability of competing golfers.  The evidence presented here cannot say conclusively that the Slope system was not worth the effort, but it certainly raises doubt.  It also raises the question  as to the size of the contribution further refinements in the handicapping system can make.  The inherent measurement errors in course ratings and the wide distribution in golfing characteristics among players of the same scoring ability may make refinements such as the Slope System only second order corrections at best.



[1] Golf Digest, June 1991, p. 146.  This referred to Dean Knuth who has since been replaced as the Director of Handicapping).
[2] Handicaps were then determined by first taking the lowest 10 differentials (adjusted score minus the course rating) out of the last 20 scores.  The total of the 10 differentials was divided by 10 to find the average differential.  The handicap was computed by multiplying the average differential by .96 and rounding to the nearest integer.
[3] This is true for stoke play, but not necessarily true for match play.  Under the Slope System a low handicapper can give a higher handicapped fewer strokes on a more difficult course.  For example a 10 index would give a 15 index 6 strokes on a course rating of 117.  On a course with a slope rating of 131 (i.e., a more difficult course he would only have to give 5 strokes.
[4] Stroud, R.C., and L.J. Riccio, “Mathematical Underpinnings of the Slope Handicap System,” in Science and Golf: The Proceeding so the first World Scientific Congress, Rutledge, Chapman, and Hall, London, 1990.
[5] Pooling data over all years and teams, the teams with a greater that nine point slope differential won 58.7 of the time (128 out of 218 matches).  The null hypothesis that the probability of winning is less than or equal to .5 can be rejected at the 2.5 percent level of confidence.