Monday, September 24, 2012

The USGA - The Dishonarable Company of Golfers

Introduction - The history of golf is replete with stories of golfers calling rulings on themselves.  Bobby Jones is noted for calling penalty strokes on himself for violations that no one else saw.  One such call may even have cost him the 1925 U.S. Open.  Indeed it is the honor in the game that most distinguishes golf from its competitors on Sunday television.  In football and baseball, it is all right to claim you caught the ball even if you didn’t.  Lying is an accepted if not an integral part of those games.
            Given the history and tenets of golf, one would expect the United States Golf Association (USGA) to exemplify all of the virtuous attributes ascribed to the game.  That certainly was my expectation, but it was dashed on the rocks of reality by the USGA’s reaction to my paper questioning the value of the Slope System (see Is the Slope System Worth the Effort - posted 9/20/2012).  The USGA engaged in lying, fraud, and character assassination in an attempt to bury my work.  In the USGA’s case, the bureaucratic instincts of survival clearly overwhelmed any inclination to do the honorable thing.

 In the Beginning - My research on the Slope System was born out of curiosity.  The Slope System was hard to use and difficult to understand.  To be implemented it had to have significant benefits.  I wrote to the USGA asking for studies of the Slope System that demonstrated its efficacy.  In response, the USGA only sent a description of the Slope System. 

Having observed bureaucratic behavior for over ten years as a policy analyst at the Rand Corporation, I found the USGA’s failure to send any studies probably meant there were not any.  After all, if there were strong scientific support for the Slope System there should have been no hesitancy to release it.

            Given the USGA’s reluctance, I went ahead with two tests of the Slope System reported in Is the Slope System Worth the Effort- posted 9/20/2012.  The two tests were not meant to be definitive, but to 1) demonstrate a methodology for testing the Slope System and 2) determine if more research needed to be done. The research concluded, qualified by the limited sample size, that the Slope System was a second order improvement in the handicap system at best. 

            The report “The Slope System: A Review and Evaluation” was submitted to the USGA for its review.  The USGA’s response at this was encouraging.  Francis Scheid of the USGA’s Handicap Research Team wrote:[1]

It is always a pleasure to find that someone else is involved in handicap research and your work shows the competence that the subject deserves…In summary, you have obviously made an important contribution.  I hope you can avoid the temptation to go for the front page of the Washington Post and suggest that instead we continue to refine what is basically a sound idea.

            This was in essence a pat on the head.  I might have accepted it if Scheid had not gone on with some rather silly reasoning in defense of the Slope System:

The reality of the slope concept should not be doubted.  Common sense points directly to it… (If) A outbowls B by 20 pins a string on average, and B bests C by 30, then Las Vegas will expect this field of three to show gaps of 60 and 90 strings over a three string event.  (This example)…illustrates a general principle, that differences in ability are magnified by increasing the level of difficulty of the assignment…”
            In response to Mr. Scheid, I had to point out his example did not support the concept of a Slope System.[2]   To be kind, I said nothing about his choice of a bowling analogy nor did I take it as an insult.  I did say:

Las Vegas is simply assuming an independence among trials (i.e., strings), and computing the expected value of three strings to be three times the expected valued of one string.  Your implied logic that if this bowling example is true, then the Slope System is valid is simply false. The Slope System tried to deal with how to standardize a trial (e.g., what should your bowling handicap be if the lane was lengthened by 20 feet?), rather than cumulative differences over a number of trials
            Apparently any criticism was too much for Mr. Scheid.  He characterized my effort as hate mail[3] even though my letter concluded with:

I have the greatest respect for the USGA and for your work for the past twenty years in studies of handicapping.  For the love of the game you dedicated your talent to making improvements to ensure more equitable competition.  I hope you can believe my relatively modest effort was undertaken with the same goal in mind.

Not exactly my definition of hate mail type material, but perhaps I have not led the cloistered existence of Mr. Scheid.

            But that was not enough for the USGA.  Dean Knuth, the Director of Handicapping for the USGA, had to pile on: [4]

“I share Dr. Scheid’s repulsion of your style of correspondence and also question who would even be interested enough in your handicapping work to give you the ‘economic advantage’ that you admit to seek.”

Of course I had not sought and economic advantage – though that term really has no meaning – since there is no money to be made in research on golf handicapping.  Knuth tried to discredit my research by impugning my integrity. It was something he would repeat on several occasions in the future. 

While ideas certainly should have been debated on a level playing field, the USGA chose instead to shovel dirt on a minor opponent.  And remember these are the men who so often remind us that they are guardians of the integrity of the game.

The World Scientific Congress of Golf - I had also sent my research paper to Dr.
Alastair Cochran, author of The Search for the Perfect Swing.[5]  He wrote back:

If repeated with a similar outcome elsewhere, the results of your study of play from two different sets of tees certainly would cast doubt on the need for the Slope System….You may wish to consider submitting one or more of your papers to the Second World Scientific Congress of Golf, to be held in St. Andrews in July 1994.  I have taken the liberty of putting your name on the list to receive advance notice of this.

Since Cochran was one of the leaders of applying science to the study of golf, I was encouraged by his generally positive response.  I was particularly excited by the opportunity to share my research at the Congress.

            The paper was submitted to the Congress for review in November of 1993.  The Conference Director, Martin Farrally, informed me that all decisions would be made by February 15, 1994.  I also learned at this time that the USGA was a co-sponsor of the Congress.  While I found this troubling, I did not believe the USGA would be so blatant as to use their power have my paper excluded from the Congress.  I was wrong.

            The paper was rejected by the Congress, but the reviewer who rejected the paper demonstrated both his bias and ignorance of the subject area.  The reviewer carped about the small sample size.  This argument was not compelling for the following reasons:

1.   The Congress Has Published Papers With Far Less Data and Smaller Sample Sizes - At the First Congress, Stroud and Riccio, members of the USGA Handicap Procedures Committee, published a paper entitled "Mathematical underpinnings of the slope handicap system."  While they described the system and extolled its benefits, they did not present one shred of empirical evidence that the system would increase the equity of competition.  Moreover they were allowed to cite articles that are not available to the general public, but only to select members of the USGA community.  This is hardly consistent with the principles of scientific inquiry that the Congress should represent.

            Dean Knuth, Director of Handicapping of the USGA, had a paper entitled "A two-parameter golf course rating system" presented at the First Congress.  Again there is no research on the benefits of the Slope system, but merely a description of how ratings are done.  For the course rating system, Knuth only used data from seven U.S. Amateur Championship courses in his sample.  He then threw out 41 percent of the hole scores sampled without providing a sound reason.  The estimates of the coefficient of the bogey rating model were based on the scores of only 60 players!  If such a skimpy sample is of sufficient size on which to base the entire USGA Handicap System, then my sample size should be adequate to question the efficacy of that system.

                  To criticize my paper because of its limited sample size is to hold it to a higher standard than research submitted by the USGA.


2.   The Sample Size is Not that Small -  The inter-club test of the Slope System involved most courses in Southern California.  This is probably the largest such team competition held anywhere.  And any statistical problems certainly do not stem from sample size.

            My study concluded that tough courses (i.e., high slope ratings) did not have the big advantage that the Riccio and Stroud paper postulated.  Riccio and Stroud only gave theoretical arguments, of course, and had no empirical evidence on which to base their claim.  My results were based on over 100 team matches in each year of the study.  Therefore, a reasonable person would have to conclude that it is not sample size that is the deciding factor in the selection of papers.


3.    The Value of the Paper is in Presenting a Methodology that Can be Replicated by Others to Test the Merits of the Slope System -  My paper presents a methodology for testing the slope and course ratings among various sets of tees at a course.  I believe this is important for golf associations in validating the accuracy of their ratings.  I have demonstrated this methodology for only one course.  While my result raises questions about the Slope System, it cannot serve as a general indictment as I readily admit.  With my methodology, however, golf associations could test the slope ratings at thousands of courses.  By rejecting my paper, however, the associations are deprived of the methodology to perform such tests.

             The reviewer also claimed that I had erred since the Slope System was introduced in Southern California in 1990 and not 1991 as I reported in my paper.  I gave Farrally the phone number of the Southern California Golf Association (SCGA) so he could check who was right.  Farrally never made the call.  Since the SCGA had both supplied the data for the study and reviewed the results no such error could have occurred.

            If Farrally was embarrassed by his reviewer’s mistake, he did not show it.  Instead he went on to claim independence from the USGA:[6]

“May I assure you there is no politics involved in the rejection of your works as a paper.  We make no secret that the R & A and the USGA are major sponsors but the congress is run by a university with full academic freedom.” 
            Farrally’s claim of independence rings hollow.  Without the USGA and R & A sponsorship, the Congress and Farrally’s job as director would not exist.  Moreover , the USGA is not without an institutional interest.  The USGA also has an economic interest in maintaining its omnipotence on all handicapping matters.  The USGA sells handicap services much like Ping or Cobra sells golf clubs.  Therefore, the Congress conference has the appearance, if not the reality, of a conflict of interest in the area of handicapping. But let’s examine the evidence.

            Every paper submitted by the USGA was accepted.  These papers were hardly earthshaking.  L.J. Riccio of the USGA Handicap Procedures Committee did a study of Tom Watson’s play at the US. Open – The Ageing of a Great Player: Tom Watson’s Play in the U.S. Open from 1980 to 1993.  A longitudinal study of one player cannot say anything about the effects of ageing for the population as a whole.  Since the venue and weather change each year, the study is of even limited value in assessing Tom Watson’s performance over time.  Therefore, the criticism of my paper because of its limited sample size, was simply a ruse to the USGA bidding.

Dean Knuth submitted a paper that was just a description of the Adjustment for Exceptional Tournament Performance.  Knuth presented no evidence that the new procedure had any effect on the equity of competition.  Though I was criticized for not having sufficient empirical findings, Knuth paper was embraced even though it had no findings.

Finally, Francis Sheid of the USGA submitted a paper, The Search for the Perfect Handicap, that by its title was not the definitive research that was demanded of my paper.  It was to turn out that Scheid’s paper was just a rehash of a paper he had written 16 years earlier.  A good reviewer would have refused Scheid’s paper because it was not original.  Since USGA paper’s probably got minimal scrutiny at best, it was accepted.

In taking the USGA’s money, the Congress received a batch of mediocre papers of questionable scientific merit and little policy value.  The Congress made a bad bargain that exposed that it is a creature of politics and not science.[7]

            Farrally tried to defend the review process:

“Your paper was reviewed by two separate academics, both of them well respected statisticians working in Universities, on British and one American.  Because of the sensitivity of this issue I am prepared to inform you that the American reviewer was not Frank Scheid.  I can also confirm that having spoken to both reviewers, I have been assured that the content of neither review was disclosed to anybody else but myself.”

            Farrally’s defense strongly implies the reviewers were independent of the USGA.  I learned, however, that my paper was reviewed by Burt Lieberman who is not a statistician but an associate professor of mathematics at Polytechnic University – a commuter school in Brooklyn, New York.  Mr. Lieberman has no publications in either econometrics, quasi-experimental design, or the measurement of human performance in golf.  He has, however, been a consultant to the USGA since 1979!  I guess this little fact slipped Farrally’s mind.

            I further learned that the review of my paper had been discussed at length with members of the USGA Handicap section before being sent to Farrally.  This is easily proved by a letter Dean Knuth sent to the head of the Connecticut State Golf Association:[8]

“An independent Congress of Golf review committee composed of professors from Great Britain and the United States ‘peer reviewed” his (Dougharty’s) report and found it unfit for their proceedings.”

            Now how would Knuth know of the reviews unless he had contact with the reviewers?  Farrally continued to maintain that “neither of the review were seen by anyone else except myself and Alastair Cochran.[9]  Did Farrally take the most direct step and ask Knuth how he knew of the results?  Of course not.  This would only embarrass the USGA and was something Farrally could not afford to do.  Farrally continued to maintain the USGA had nothing to do with the review even against overwhelming evidence to the contrary.  Farrally was certainly a good soldier in the cause of the USGA.  It is not surprising he continues as Congress Director and the USGA continued its financial support.

            The response of David Fay, Executive Director of the USGA, was equally disappointing.  After Fay was sent all of the supporting evidence he wrote:[10]

”In our judgement (sic), we concluded that yet another reply to you would accomplish nothing.  We know your feelings; we happen to disagree.”

Though Fay must have been aware of the USGA’s involvement, he took no corrective steps nor did he offer an apology for the actions taken by his organization.  If the rejection of my paper was not embarrassing enough for the USGA, the acceptance of Francis Sheid’s was even more so.  That sordid affair is detailed in the next post, The Francis Scheid Affair.

[1] Letter to the author from Francis Sheid, February 20, 1991.
[2] Letter from the author to Francis Scheid, February 27, 1991.
[3] Letter to the author From Francis Scheid, March 18, 1991.
[4] Letter to the author from Dean Knuth, April 18, 1991.
[5] Cochran, Alastair and Stobbs, John, The Search for the Perfect Swing, The Bootlegger (reprinted), Grass Valley, CA 1989.
[6] Letter to the author from Martin Farrally, February 4, 1994.
[7]  The Congress showed a similar bias in a paper submitted by a writer from one of the sponsoring organizations, Golf Digest (Australia).  The paper contained no substantive research, and was generally a lament about the poor television coverage of Australian tournaments.  The writer concluded his paper with, “As journalists continue to be so badly paid, golf writers will continue to look for other public relations work to supplement their paltry incomes.”  I assume this writer’s paltry income was supplemented by a free trip to the St. Andrews, the Congress site.  Others used the Congress as a tax write-off.  U.S. taxpayers were picking up part of the tab for many Congress participants to play the Old Course.
[8] Letter from Dean Knuth, USGA Director of Handicapping to Russell Palmer, Executive Director of the Connecticut State Golf Association, August 25, 1994.
[9] Letter to the author from Martin Farrally, November 22, 1994.
[10] Letter to the author from David Fay, December 7, 1998.

No comments:

Post a Comment