CHAPTER TWO 80 showed substantial agreement (kappa from 0.61 to 0.80); 29 showed moderate agreement (kappa from 0.41 to 0.60); 7 (9%) showed fair agreement (kappa from 0.21 to 0.40); 8 (11%) showed slight agreement (kappa from 0.00 to 0.20); and none showed poor agreement (kappa less than zero). A summary of these findings is presented in Table 2-20. For the most part, excellent levels of agreement occur for the visual acuity findings, cataract diagnoses, and diagnoses of other diseases. The good measures are those related to major cause, etiology, COD-S, cataract signs, certain trachoma signs and diagnoses, and intervention- level trachoma intensities. The measures with poor or no interobserver agreement include adult xerophthalmia, anterior synechia, and the four-category intensity of trachoma measure. As might have been anticipated, there was more variability be- tween ophthalmologists for the measures that had many categories (e.g., a scale of four to seven classes) than for those with only two categories (absent or present). In many cases, a measure that was "poor" when many classes were possible became "good" or excellent" when categories were collapsed. Notes 3One ophthalmologist who suffered a broken foot en route to the training site spent two months reviewing completed proformae in the Kathmandu office. 4Royal Australian College of Ophthalmologists (1980), The National Trachoma and Eye Health Program of the Royal Australian College of Ophthalmologists, Sydney: Royal Australian College of Ophthalmologists. 5Kupka, K., Nizetic, B., and Reinhards, J. (1968), Sampling Studies on Epidemiology and Control of Trachoma in Southern Morocco, World Health Or- ganization Bulletin, 39, 547-566. 6Fleiss, J. L. Statistical Methods for Rates and Proportions, New York: Columbia University. 7Landis, J. R., and Koch, G. G. (1977), The Measurement of Observer Agree- ment for Categorical Data, Biometrics, SS, 159-174. 8Fleiss, J. L., and Cohen, J. (1973), The Equivalence of Weighted Kappa and the Intraclass Correlation Coefficient as Measures of Reliability, Educational and Psychological Measurement, 613-619. 9World Health Organization (1980), Methods of Assessment of Avoidable Blind- ness (WHO Offset Publication No. 54), Geneva: World Health Organization. 10The development of the COD and COD-S is described in Chapter 5. 11The definition of "backlog" includes both presence of cataract as the COD and visual impairment levels corresponding to the various categories of backlog A, B, or C.
