METHODOLOGY 73 measures has been divided into two groups, signs and diagnosis. The·_ signs were expected to be relatively reliable measures for which good agreement could be expected. The diagnosis measures, primarily based on the anatomical cause, etiology, and visual acuity assessments, were expected to be somewhat less reliable. However, a comparison of kappa values between cataract signs (Table 2-16, items 19 through 24) and diagnosis (items 25 through 34) demonstrate the opposite effect. The diagnosis measures show excellent levels of agreement. For example, the presence or absence of any cataract (including aphakia and after-cataract within the category of cataract) has a kappa value of 0.91 and 99.2 percent agreement. The determination of cataract backlog status A, B, and C (items 32 through 34) have kappa values of 0.85, a reassuring finding in view of their importance to program planning.11 The cataract-stage measures Table 2-16, items 26 through 31) have generally excellent levels of agreement, although somewhat lower than those observed for the other measures in the group. It is interesting to observe that when the cataract-stage variable with five categories "incipient, immature, mature, aphakic, and after- cataract" (items 26 and 27) is collapsed to the four categories "early, mature, aphakic and after-cataract" (items 28 and 29) the kappa values increase from the good range to the excellent or almost perfect range. Most of the statistical disagreement for cataract stage is in distinguishing incipient and immature cataracts. Finally, when a subject is assigned a cataract bilateral progression score with the five categories "none/none, none/early, early/early, early/mature, and mature/mature," the percentage agreement is 99 percent and the kappa value is 0.86. The agreement for cataract signs, although in the good range, is not as strong as for cataract diagnosis. Generally the percentage agreement is better than 95 percent and kappa is in the 0.60 to 0. 70 range. This is again the opposite of what was expected; NBS ophthalmologists tended to disagree slightly more on what level of red light reflex is "markedly" or "slightly" reduced, and they tended to agree almost perfectly about which cataract is mature, which is early, and which is not present at all. Trachoma Measures: As in the case of cataract measures, ob- server agreement on trachoma measures has been divided into two groups of signs and diagnoses. Neither the trachoma signs nor the trachoma diagnosis measures demonstrate the excellent levels of agreement found in cataract, and some trachoma measures provide poor levels of agreement. Trachoma sign measures are presented in sets of four measures
