Introduction - The history of golf is
replete with stories of golfers calling rulings on themselves. Bobby Jones is noted for calling penalty
strokes on himself for violations that no one else saw. One such call may even have cost him the 1925
U.S. Open. Indeed it is the honor in the
game that most distinguishes golf from its competitors on Sunday television. In football and baseball, it is all right to
claim you caught the ball even if you didn’t.
Lying is an accepted if not an integral part of those games.
Given the history and tenets of
golf, one would expect the United States Golf Association (USGA) to exemplify
all of the virtuous attributes ascribed to the game. That certainly was my expectation, but it was
dashed on the rocks of reality by the USGA’s reaction to my paper questioning
the value of the Slope System (see Is the Slope System Worth the Effort - posted 9/20/2012).
The USGA engaged in lying, fraud, and character assassination in an attempt
to bury my work. In the USGA’s case, the
bureaucratic instincts of survival clearly overwhelmed any inclination to do
the honorable thing.
Having
observed bureaucratic behavior for over ten years as a policy analyst at the
Rand Corporation, I found the USGA’s failure to send any studies probably meant
there were not any. After all, if there
were strong scientific support for the Slope System there should have been no
hesitancy to release it.
Given the USGA’s reluctance, I went
ahead with two tests of the Slope System reported in Is the Slope System Worth the Effort- posted 9/20/2012. The two tests were not meant to be
definitive, but to 1) demonstrate a methodology for testing the Slope System
and 2) determine if more research needed to be done. The research concluded,
qualified by the limited sample size, that the Slope System was a second order
improvement in the handicap system at best.
The report “The Slope System: A
Review and Evaluation” was submitted to the USGA for its review. The USGA’s response at this was
encouraging. Francis Scheid of the
USGA’s Handicap Research Team wrote:[1]
It is always a
pleasure to find that someone else is involved in handicap research and your
work shows the competence that the subject deserves…In summary, you have
obviously made an important contribution.
I hope you can avoid the temptation to go for the front page of the
Washington Post and suggest that instead we continue to refine what is
basically a sound idea.
This was in essence a pat on the
head. I might have accepted it if Scheid
had not gone on with some rather silly reasoning in defense of the Slope
System:
The reality of
the slope concept should not be doubted.
Common sense points directly to it… (If) A outbowls B by 20 pins a
string on average, and B bests C by 30, then Las Vegas will expect this field
of three to show gaps of 60 and 90 strings over a three string event. (This example)…illustrates a general
principle, that differences in ability are magnified by increasing the level of
difficulty of the assignment…”
In response to Mr. Scheid, I had to
point out his example did not support the concept of a Slope System.[2] To be kind, I said nothing about his choice
of a bowling analogy nor did I take it as an insult. I did say:
Las Vegas is
simply assuming an independence among trials (i.e., strings), and computing the
expected value of three strings to be three times the expected valued of one
string. Your implied logic that if this
bowling example is true, then the Slope System is valid is simply false. The
Slope System tried to deal with how to standardize a trial (e.g., what should
your bowling handicap be if the lane was lengthened by 20 feet?), rather than
cumulative differences over a number of trials
Apparently any criticism was too
much for Mr. Scheid. He characterized my
effort as hate mail[3]
even though my letter concluded with:
I have the
greatest respect for the USGA and for your work for the past twenty years in
studies of handicapping. For the love of
the game you dedicated your talent to making improvements to ensure more
equitable competition. I hope you can
believe my relatively modest effort was undertaken with the same goal in mind.
Not exactly my
definition of hate mail type material, but perhaps I have not led the cloistered
existence of Mr. Scheid.
But that was not enough for the
USGA. Dean Knuth, the Director of
Handicapping for the USGA, had to pile on: [4]
“I share Dr.
Scheid’s repulsion of your style of correspondence and also question who would
even be interested enough in your handicapping work to give you the ‘economic
advantage’ that you admit to seek.”
Of
course I had not sought and economic advantage – though that term really has no
meaning – since there is no money to be made in research on golf
handicapping. Knuth tried to discredit
my research by impugning my integrity. It was something he would repeat on
several occasions in the future.
While
ideas certainly should have been debated on a level playing field, the USGA
chose instead to shovel dirt on a minor opponent. And remember these are the men who so often
remind us that they are guardians of the integrity of the game.
The World Scientific Congress of Golf -
I had also sent my research paper to Dr.
If repeated
with a similar outcome elsewhere, the results of your study of play from two
different sets of tees certainly would cast doubt on the need for the Slope
System….You may wish to consider submitting one or more of your papers to the
Second World Scientific Congress of Golf, to be held in St. Andrews in July
1994. I have taken the liberty of
putting your name on the list to receive advance notice of this.
Since Cochran
was one of the leaders of applying science to the study of golf, I was
encouraged by his generally positive response.
I was particularly excited by the opportunity to share my research at
the Congress.
The paper was submitted to the
Congress for review in November of 1993.
The Conference Director, Martin Farrally, informed me that all decisions
would be made by February 15, 1994. I
also learned at this time that the USGA was a co-sponsor of the Congress. While I found this troubling, I did not
believe the USGA would be so blatant as to use their power have my paper
excluded from the Congress. I was wrong.
The paper was rejected by the
Congress, but the reviewer who rejected the paper demonstrated both his bias
and ignorance of the subject area. The
reviewer carped about the small sample size.
This argument was not compelling for the following reasons:
1. The
Congress Has Published Papers With Far Less Data and Smaller Sample Sizes - At
the First Congress, Stroud and Riccio, members of the USGA Handicap Procedures
Committee, published a paper entitled "Mathematical underpinnings of the
slope handicap system." While they
described the system and extolled its benefits, they did not present one shred
of empirical evidence that the system would increase the equity of
competition. Moreover they were allowed
to cite articles that are not available to the general public, but only to
select members of the USGA community.
This is hardly consistent with the principles of scientific inquiry that
the Congress should represent.
Dean Knuth, Director of Handicapping
of the USGA, had a paper entitled "A two-parameter golf course rating
system" presented at the First Congress.
Again there is no research on the benefits of the Slope system, but
merely a description of how ratings are done.
For the course rating system, Knuth only used data from seven U.S.
Amateur Championship courses in his sample.
He then threw out 41 percent of the hole scores sampled without
providing a sound reason. The estimates
of the coefficient of the bogey rating model were based on the scores of only
60 players! If such a skimpy sample is
of sufficient size on which to base the entire USGA Handicap System, then my
sample size should be adequate to question the efficacy of that system.
To criticize my paper because
of its limited sample size is to hold it to a higher standard than research
submitted by the USGA.
2. The
Sample Size is Not that Small - The
inter-club test of the Slope System involved most courses in Southern
California. This is probably the largest
such team competition held anywhere. And
any statistical problems certainly do not stem from sample size.
My study concluded that tough
courses (i.e., high slope ratings) did not have the big advantage that the
Riccio and Stroud paper postulated.
Riccio and Stroud only gave theoretical arguments, of course, and had no
empirical evidence on which to base their claim. My results were based on over 100 team
matches in each year of the study.
Therefore, a reasonable person would have to conclude that it is not
sample size that is the deciding factor in the selection of papers.
3.
The
Value of the Paper is in Presenting a Methodology that Can be Replicated by
Others to Test the Merits of the Slope System - My paper presents a methodology for testing
the slope and course ratings among various sets of tees at a course. I believe this is important for golf
associations in validating the accuracy of their ratings. I have demonstrated this methodology for only
one course. While my result raises
questions about the Slope System, it cannot serve as a general indictment as I
readily admit. With my methodology, however,
golf associations could test the slope ratings at thousands of courses. By rejecting my paper, however, the
associations are deprived of the methodology to perform such tests.
If Farrally was embarrassed by his
reviewer’s mistake, he did not show it.
Instead he went on to claim independence from the USGA:[6]
“May I assure
you there is no politics involved in the rejection of your works as a
paper. We make no secret that the R
& A and the USGA are major sponsors but the congress is run by a university
with full academic freedom.”
Farrally’s claim of independence
rings hollow. Without the USGA and R
& A sponsorship, the Congress and Farrally’s job as director would not
exist. Moreover , the USGA is not
without an institutional interest. The
USGA also has an economic interest in maintaining its omnipotence on all
handicapping matters. The USGA sells
handicap services much like Ping or Cobra sells golf clubs. Therefore, the Congress conference has the
appearance, if not the reality, of a conflict of interest in the area of
handicapping. But let’s examine the evidence.
Every paper submitted by the USGA
was accepted. These papers were hardly
earthshaking. L.J. Riccio of the USGA
Handicap Procedures Committee did a study of Tom Watson’s play at the US. Open
– The Ageing of a Great Player: Tom
Watson’s Play in the U.S. Open from 1980 to 1993. A longitudinal study of one player cannot say
anything about the effects of ageing for the population as a whole. Since the venue and weather change each year,
the study is of even limited value in assessing Tom Watson’s performance over
time. Therefore, the criticism of my
paper because of its limited sample size, was simply a ruse to the USGA
bidding.
Dean
Knuth submitted a paper that was just a description of the Adjustment for
Exceptional Tournament Performance.
Knuth presented no evidence that the new procedure had any effect on the
equity of competition. Though I was
criticized for not having sufficient empirical findings, Knuth paper was
embraced even though it had no findings.
Finally,
Francis Sheid of the USGA submitted a paper, The Search for the Perfect Handicap, that by its title was not the
definitive research that was demanded of my paper. It was to turn out that Scheid’s paper was
just a rehash of a paper he had written 16 years earlier. A
good reviewer would have refused Scheid’s paper because it was not original. Since USGA paper’s probably got minimal
scrutiny at best, it was accepted.
In
taking the USGA’s money, the Congress received a batch of mediocre papers of
questionable scientific merit and little policy value. The Congress made a bad bargain that exposed
that it is a creature of politics and not science.[7]
Farrally tried to defend the review
process:
“Your paper was
reviewed by two separate academics, both of them well respected statisticians
working in Universities, on British and one American. Because of the sensitivity of this issue I am
prepared to inform you that the American reviewer was not Frank Scheid. I can also confirm that having spoken to both
reviewers, I have been assured that the content of neither review was disclosed
to anybody else but myself.”
Farrally’s defense strongly implies
the reviewers were independent of the USGA.
I learned, however, that my paper was reviewed by Burt Lieberman who is
not a statistician but an associate professor of mathematics at Polytechnic
University – a commuter school in Brooklyn, New York. Mr. Lieberman has no publications in either
econometrics, quasi-experimental design, or the measurement of human
performance in golf. He has, however,
been a consultant to the USGA since 1979!
I guess this little fact slipped Farrally’s mind.
I further learned that the review of
my paper had been discussed at length with members of the USGA Handicap section
before being sent to Farrally. This is
easily proved by a letter Dean Knuth sent to the head of the Connecticut State
Golf Association:[8]
“An independent
Congress of Golf review committee composed of professors from Great Britain and
the United States ‘peer reviewed” his (Dougharty’s) report and found it unfit
for their proceedings.”
Now how would Knuth know of the
reviews unless he had contact with the reviewers? Farrally continued to maintain that “neither
of the review were seen by anyone else except myself and Alastair Cochran.[9]” Did Farrally take the most direct step and
ask Knuth how he knew of the results? Of
course not. This would only embarrass
the USGA and was something Farrally could not afford to do. Farrally continued to maintain the USGA had
nothing to do with the review even against overwhelming evidence to the
contrary. Farrally was certainly a good
soldier in the cause of the USGA. It is
not surprising he continues as Congress Director and the USGA continued its
financial support.
The response of David Fay, Executive
Director of the USGA, was equally disappointing. After Fay was sent all of the supporting
evidence he wrote:[10]
”In our
judgement (sic), we concluded that yet another reply to you would accomplish
nothing. We know your feelings; we
happen to disagree.”
Though
Fay must have been aware of the USGA’s involvement, he took no corrective steps
nor did he offer an apology for the actions taken by his organization. If the rejection of my paper was not
embarrassing enough for the USGA, the acceptance of Francis Sheid’s was even
more so. That sordid affair is detailed
in the next post, The Francis Scheid Affair.
[1]
Letter to the author from Francis Sheid, February 20, 1991.
[2]
Letter from the author to Francis Scheid, February 27, 1991.
[3]
Letter to the author From Francis Scheid, March 18, 1991.
[4]
Letter to the author from Dean Knuth, April 18, 1991.
[5]
Cochran, Alastair and Stobbs, John, The
Search for the Perfect Swing, The Bootlegger (reprinted), Grass Valley, CA
1989.
[6]
Letter to the author from Martin Farrally, February 4, 1994.
[7] The Congress showed a similar bias in a paper
submitted by a writer from one of the sponsoring organizations, Golf Digest
(Australia). The paper contained no
substantive research, and was generally a lament about the poor television
coverage of Australian tournaments. The
writer concluded his paper with, “As journalists continue to be so badly paid,
golf writers will continue to look for other public relations work to
supplement their paltry incomes.” I
assume this writer’s paltry income was supplemented by a free trip to the St.
Andrews, the Congress site. Others used
the Congress as a tax write-off. U.S.
taxpayers were picking up part of the tab for many Congress participants to
play the Old Course.
[8]
Letter from Dean Knuth, USGA Director of Handicapping to Russell Palmer,
Executive Director of the Connecticut State Golf Association, August 25, 1994.
[9]
Letter to the author from Martin Farrally, November 22, 1994.
[10]
Letter to the author from David Fay, December 7, 1998.
No comments:
Post a Comment