|
Ballester-Guasp Evaluation of Completed Reports Javascript Report Evaluator by Terry Groff |
"It has been a very nice surprise to see the Ballester-Guasp quantification method on your site. I applaud very much your excellent work being done on it, and I will certainly report it to others, in order to facilitate scientific-oriented investigators to use this tool. Thanks and keep up the good work! Very best wishes."
- Vicente-Juan Ballester Olmos
- Welcome to the Javascript Report Evaluator. (at bottom of page)
- This calculator is based on the MUFON FIELD INVESTIGATORS MANUAL: SECTION XVIII ADDITIONAL GUIDELINES: PART 2: BALLESTER-GUASP EVALUATION OF COMPLETED REPORTS
- Field Investigators should find this tool useful in evaluating reports prior to submission to ensure that all essential questions have been asked and answered.
- Use the button after all selections have been made. Also note that the Certainty Index will remain at 0% until at least one selection has been made in each of the three sections.
- Click to start over.
- Click button for more information about a particular selection.
- This script has been tested in most browsers and while it's appearance is slightly different in some, the essential functions still execute. If you have any problems running this program email me or use my feedback form.
- Just make all the necessary selections in the following tables and read the result at the bottom
| Evaluation Guidelines |
 FIELD INVESTIGATORS MANUAL XVIII ADDITIONAL GUIDELINES PART 2: BALLESTER-GUASP EVALUATION OF COMPLETED REPORTS By Jerold R. Johnson |
| EVALUATION PROCEDURES FOR REPORTS |
This section details the evaluation procedure applied to reports at headquarters level prior to their being entered into the computer file and physically filed in the MUFON headquarters office. This is not a task required of the Field Investigator, or any of the volunteers at State or Province level, but is included so that all users of this manual will know how the evaluation is done and why. Familiarity with the data required in each report to perform this evaluation is encouraged of the Field Investigator, as is a "practice run-through" on reports prepared for submission to make sure that the data are present.
| NUMERIC EVALUATION OF REPORTS |
Since 1992, reports processed into the MUFON files have been given a numeric evaluation
1. The volume and quality of the data recorded, based on the methods employed and the time spent investigating the case.
2. The inherent abnormality or "strangeness" of the event, making it unlikely to have a natural or conventional explanation.
3. The credibility of the report, based on the reliability, maturity, and circumstances of the witnesses interviewed.
4. An overall "score" for the report derived by multiplying the three previous values together, representing the degree of certainty that the report indeed represents an anomalous event that happened as recorded.
Each of these quantities is expressed as a decimal fraction between zero and one, thus when multiplied together, any parameter near the zero end of its range will bring the overall "certainty" score down, and if a parameter is zero, as when there are no elements of strangeness" to differentiate the reported sighting from some mundane event, then the final certainty index becomes zero, marking the report as an "IFO" or otherwise spurious.
This system of numeric evaluation and the standards and definitions it is based on was developed and codified by Vincente-Juan Ballester Olmos (MUFON's Representative for Spain) and Miguel Guasp. Its detailed derivation may be found in two publications: Hynek, Mimi (ed.) The Spectrum of UFO Research, J. Allen Hynek Center for UFO Studies, Chicago, 1988, p. 175182, and Ballester, V. J. and Guasp, M. Los OVNIs y la Ciencia (UFOs and Science), Plaza & Janes, S.A., Barcelona, 1981, 1989, p. 117-135.
They describe the system as a formula for self-assessment based on the contents of the report itself, which can be done by any person regardless of their level of "expertise" in the flying saucer subject and lore. The system gives reproducible numbers when evaluated by different individuals, at different times, as long as they are following the standards as published. The method is relatively "quick and easy" given a calculator and a few tables and definitions extracted from the publications and kept handy as notes. The ultimate value comes from using the numbers derived and attached to the reports recorded in a catalog to compare the value of each report against the others within that catalog. It may also be compared with other such catalogs maintained by other organizations using the same internationally recognized standards for evaluation, in order to select the highest value reports for detailed studies, statistical processing, and analysis of trends.
By presenting the three components of information quality, event strangeness, and witness reliability as well as the product: certainty index, one can tell what it is about the report that gives it a high, or a low score, and reports can be regrouped numerically by each of these factors for additional analysis. If nothing else, an organization of volunteer investigators can keep track, statistically, of the one factor they have some control over: the quality of the investigation and report preparation, and takes steps to maximize those numbers when the significance of the case demands a "quality" effort.
Note the particular details of information which must be recorded by the investigator, and preserved even on the computer file, so that this evaluation may be completed, both upon initial entry and later by any future researcher that may want to double-check the numbers or determine their source from the internal details of the report. This requires reporting details not only of the anomaly observed, but the status and circumstance of the witness(es) involved, and the situation and extent of the field investigation as well. Avoid the urge to argue with the selection, ordering, or number value assigned to the various factors. The standards must be stable and rigorously adhered to for there to be any usefulness in comparisons of reports using the numbers at different times, by different people, ,possibly in different countries, but all using the same standardized evaluation procedure. Statistical manipulation of the numbers also requires that the "rules" not change partway through the "game." Considerable knowledge, experience, and testing has gone into the selection and organization of these factors and their limitation to what can be reasonably expected of both the investigator to record and the evaluator to deal with in terms of complexity.
| INFORMATION QUALITY INDEX |
The first parameter is the Information Quality Index, symbolized as W. It takes a value between zero and one, and indicates the "strength" that a report has for analysis based on how it was acquired.
| Source | Direct Investigation | At the site | >= 2 hours | 1.0 |
| < 2hours | 0.9 | |||
| Interview Person to person | >= 1 hour | 0.9 | ||
| < hour | 0.8 | |||
| By Telephone | >= 1/2 hour | 0.7 | ||
| < 1/2 hour | 0.6 | |||
| Indirect Investigation | Questionnaire w/ follow-up | Extensive | 0.7 | |
| Brief | 0.6 | |||
| Letter w/ follow-up | Extensive | 0.6 | ||
| Brief | 0.5 | |||
| Other Investigation | Questionnaire no follow-up | 0.6 | ||
| Letter/Narrative no follow-up | >= 1 page | 0.4 | ||
| < 1 page | 0.3 | |||
| Newspaper | >= 500 words | 0.2 | ||
| < 500 words | 0.1 | |||
| Radio/TV | 0.1 | |||
| Witness Relative | 0.1 | |||
| Verbal/Rumor/Unknown | 0.0 | |||
The catalog of reports being collected by MUFON contains only investigated reports, from investigators known to MUFON, so the value of W should always be 0.5 or greater.
| STRANGENESS INDEX |
The second parameter is the Strangeness Index, symbolized as S (sigma). It is a value between zero and one that indicates the "abnormality" level of a report compared to normal processes, familiar phenomena and known manufactured objects. Seven factors commonly found in sighting reports have been carefully defined, and one simply counts up the number of these factors recognized in reading the report and divides by seven to record S. The seven factors (quoting verbatim from Ballester and Guasp) are:
The anomalous appearance will be the first clue for the researcher to doubt the normality of what has been observed. An apparently physical phenomenon will have such an anomalous aspect when its shape or dimensions do not correlate with any identifiable flying craft. Its shape may be that of disk, cigar, etc., or it can include in its description, for instance, lights for which intensity or situation immediately exclude other conventional lights to be sighted in the skies.
Anomalous movements are those dynamic characteristics of the observed phenomenon which make it impossible to receive a logical explanation, based on actual knowledge. Definition must include enormous horizontal or ascending speeds (higher than those of the fastest aircraft); enormous accelerations (idem), strange trajectories (mixed, broken, helicoidal, etc.); ascent or descent in the "dead leaf" fashion (as if no gravity existed); and in general, any contradiction with the usual movements of aeronautical devices, astronomical bodies, birds, etc.
By physical-spatial incongruities we mean those elements which are in flagrant contradiction with the intuitive sense of the dimensions and the volumes (such as evident, well observed apparitions and disappearances sur place, the merging of two objects into one, etc.), and/or that which seems to violate the known behavior of the physical entities (such as deformations of apparently material objects, "solid light" cases, etc.).
The technological detection refers to the observing and/or recording of the passage of the UFO through calibrated precision instruments (technical or scientific): radar or laser tracking, observation through telescope or theodolite, record in photograph, film or videotape, light spectrum analysis, audio recording, etc.
A close encounter occurs when the witness has enjoyed a privileged position, thanks to the proximity of the event, to observe in detail without the distortion of distance. Quantitatively, this factor should follow Hynek's range (i.e., within 500 feet or 150 meters).
By the presence of beings we mean the association of presumed occupants with objects that conform to at least some of the requirements of the items anomalous appearance, anomalous movements and/or physical-spatial incongruities.
By traces and effects we mean those lasting physical or chemical characteristics or residues left by a UFO after its disappearance, provided that there exists some testimony that the traces or effects were produced by the presence of the UFO. These should include effects in people, effects of a mechanical nature, physical changes in inanimate bodies, and electromagnetic effects on motors, vehicles, or electric circuits.
The factors to be counted are those which remain unexplained after analysis by experienced investigators and directors (with expert consultation when required) within the MUFON Field Investigator's Network, not just initial witness claims. Any valid "unknown" must have at least one factor of "strangeness" (usually anomalous appearance), If the analyst cannot, in all honesty, find any factors in the report that conform to these somewhat conservative definitions then a
= 0.0 is recorded and the report is filed as a "probable IFO." Otherwise, the number of factors recognized and accepted are divided by seven and the resulting decimal number recorded for
| RELIABILITY INDEX |
The third parameter is the Reliability Index, symbolized as p (pi). It will also be a decimal value between zero, and one, indicating the witness "credibility." There are six categories within this parameter and each is assigned a "weight factor." From the information recorded, in the report, one selects the appropriate number from each category, multiplies it by its "weight factor," and ultimately adds the six results together to produce p. Obviously, if little or nothing is recorded about the witnesses and their activity at the time of the sighting, they will receive a very low p, so it is incumbent on the Field Investigator to take as much care in reporting this information as that which just describes the anomaly. Again quoting extensively from Ballester and Guasp:
Number of witnesses (a sighting is more believable if it has more witnesses)
0.0 - none or unknown
0.3 - one
0.5 - two
0.7 - three to five; "several"
0.9 - six to ten
1.0 - more than ten(multiply by weight factor 0.25)
(Note that MUFON requires this to be the number of witnesses interviewed or having signed report forms. A witness that is interviewed and says that others were present remains a single witness unless those others are somehow contacted and confirmed.
Profession or occupation of the witnesses (indicates their level of job responsibility, from which can be inferred a measure of their dependability or social status).
0.0 - not specified
0.3 - students (pre-college)
0.5 - laborers, farmers and housewives
0.6 - university students
0.7 - traders, businessmen, employees and artists
0.9 - technicians, police and pilots
1.0 - university graduates and military personnel(multiply by weight factor 0.2)
(Note that "retired" is not a profession. Investigators should ask and record what the witness' career was before they retired, as well as military service, and anything else that would help to assess that person's reliability as an observer and reporter of phenomenal events. MUFON report forms provide extensive space for "additional comments;" in too many cases unused).
Relationship between witnesses (provides indication of the theoretical tendency to generate a hoax together, based on the different types of ties between them).
0.0 - unknown
0.3 - friends
0.6 - family relationship; also applies to cases with a single witness
0.8 - professional relationship
1.0 - no relationship(multiply by weight factor 0.15)
Geographic relation between witnesses (when there are multiple observers, their relative location affects the certainty of the event).
0.0 - unknown
0.5 - together; also applies when there is a single witness
1.0 - independent (separate)(multiply by weight factor 0.15)
Activity at the time of the sighting (measures the opportunity for a hoax)
0.0 - not specified
0.3 - recreational activity (walk, rest, outing, hunting, sport, at home, on vacation, etc.)
0.6 - traveling (moving, by any means)
0.8 - cultural or intellectual activity
1.0 - working (at work or on the way to or from)(Multiply by weight factor 0.15)
Age of the witness (indicates their degree of maturity and the validity of their testimony, based on their capability).
0.0 - unknown
0.2 - under 10 years or over 75 years
0.4 - between 10 and 17 years
0.6 - between 18 and 34 years
0.8 between 65 and 74 years
1.0 between 35 and 64 years(multiply by weight factor 0.1)
(Note that this must be the age when making the report, not the age at the time of the sighting, if the two are different. Present age should be recorded in witness information blocks on MUFON forms, age at time of sighting written into "Narrative" or "Comment" areas).
In the case of multiple witnesses, for Profession and Age, the values are taken from the witnesses who provide the higher absolute value.
| CERTAINTY INDEX |
The three parameters W, Sand p are recorded for the report under consideration, and then a fourth, the Certainty Index, symbolized as D (delta) is derived by multiplying the three others together W * S * p = D, to provide a measure of the overall degree of "certainty" of an anomalous event behind the report. D is often expressed as a percentage of unity by moving the decimal two places to the right and appending "%," example D = .125 indicates a "certainty" of 12.5%, a respectable percentage as these things go. The Certainty Index might be used as a quick way to order the reports in a catalog from "least promising" to "most promising," while the other three parameters will indicate, why each report received the D value that it has.
| HOW A REPORT EVALUATES ITSELF |
To demonstrate how a report "evaluates itself" within this system, a brief hypothetical example is in order.
Field Investigator comments, under the "General" heading on a Form 2, indicate that a personal interview was conducted with the witnesses requiring more than an hour of the investigator's time, but due to the sighting occurring on a road trip, a site visit was considered impractical. From the first table, Information quality W = 0.9.
A detailed description of a daytime encounter with a metallic, "spool of thread" shaped object, larger than a baseball diamond, which approached within 100 feet of a married couple's car at an "incredible rate of speed," then slowed to pace their car at that distance for "several minutes" before "vanishing to nothingness" as another car approached. Summarized in "Event" and "Anomalous lights or objects" fields on the Form 2, allows us to fit the definitions of "anomalous appearance," "anomalous movements" and "close encounter"; hence three "sevenths" for a Strangeness S = 0,4286.
The report includes two signed Form l's so there are two witnesses recorded (0.5 X 0.25 - 0.125). "Homemaker" is listed for the woman's occupation, "College professor" for the man's, but they are both university graduates, so (1.0 X 0.2 = 0.2). Husband and wife is a family relationship (0.6 X 0.15 = 0.09). Both were together in a vehicle (0.5 X 0.15 ~: 0.075), traveling but not to or from work (0.6 X 0.15 = 0.09), and the woman's age of 43 allows the highest category (1. X 0.1 = 0.1). Adding up 0.125 + 0.2 + 0.09 + 0.075 + 0.09 + 0.1 provides a Reliability p = 0.68.
Finally, 0.9 X 0.4286 X 0.68 multiplies out to a Certainty D = 0.2623, or 26.23%.
| ANALYSIS AND COMPUTATION |
Though analysis and computation of this evaluation formula is not required of them, the Field Investigators might wish to run through it on their own reports before submission, to check if all the questions required for the evaluation have been answered somewhere in the report. The MUFON report forms were designed before this formula was recognized as worthwhile and there are not specific blanks on the forms for all the required data, especially in the "how investigated, for how long" category, so some Field Investigator "write-ins" are necessary.
BALLESTER-GUASP EVALUATION OF COMPLETED REPORTS
Copyright February 1995 by The Mutual UFO Network Inc.
Groff Javascript Report Evaluator
Copyright 2003 by Terry Groff