Reliability of using Standardized Mortality Ratio in Estimation of Relative Risk of Under-Five Mortality in Uganda; an Empirical Analysis

In Uganda, using survey data, estimates of under-five mortality have only been available at national and regional levels. However, estimation of under-five mortality can be made for districts using small area estimation techniques. The simpler way is to use the Standardized Mortality Ratio (SMR). Literature has shown that use of SMR is subject to unreliable results but no empirical study has verified that this is so. The author used Uganda Demographic and Health Survey data of 1995, 2001 and 2006 in the investigations to explore empirically how reliable is the use of SMR in estimation of relative risk of under-5 mortality. The author applied the coefficient of variation to measure reliability of the SMR estimates. Utilization of the traditional SMR could potentially be associated with very high coefficient of variations. The author recommends that before utilization of SMR, there is need to explore reliability of the results using coefficient of variation.


Introduction
Standardized Mortality Ratio is one of the basic methodologies used in small area estimation technique and is simply a ratio of observed to the expected deaths and could be used to provide estimates of local area such as district or sub-county whose estimates at that level may not be derived by direct estimation from the available survey data. Small area estimation is statistical techniques involving the estimation for small sub-populations, generally used when the sub-population of interest is included in a larger survey. For example a national survey may derive estimates for regional cluster statistics but not for district level but using small area estimation indicators for the latter could be estimated. Small area estimates in general may be useful for government agencies to allocate resources or identify hazardous areas related to high under-five mortality so that appropriate action may be taken [1][2][3]. Mapping mortality and disease rates to display geographic variability is an increasingly common epidemiological tool and falls under a broad subject of small area estimation.
Understanding spatial clustering of childhood or/and under-five mortality can provide a guide in targeting interventions in a more strategic approach to the population where mortality is highest and the interventions are most likely to make an impact [4].
National surveys are widely used to provide estimates for the entire population parameters of interest but also for subpopulations (domains) such as regions, rural or urban, sex and age groups. However, such subpopulations are generally too large to provide a sense of particular lower level localities (small areas) like district or counties or sub-counties where the actual problem can easily be located. Intervention can easily be accomplished when a small locality has been identified with a particular problem. Small area estimation provides a solution to using survey data to furnish estimates at such lower localities. The idea is that small area estimation techniques in particular "borrow strength" by using values of the variable of interest, y, from related areas to increase "effective" sample size. The value of, y, is by itself "small" to provide a reliable direct estimate for a particular locality. For example, the number of under-five deaths, y, derived from a national survey data may be too small to provide estimate of under-five mortality for a particular district. However, using small area estimation, the value of, y, can help derive a reliable estimate for relative risk of under-five mortality for the district. Relative Risk is the ratio of the incidence of disease in the exposed population to the incidence in the non-exposed population [5]. It is a ratio of two probabilities. In utilizing small area estimation techniques, the values, y, are brought into the estimation process through a model that provides a link to the related areas. A study by Asiimwe et al., [6] show that use of Poisson-gamma and log-normal models offer reliable with less 'noise' in the estimates for small area under-5 mortality data though these methodologies appear to be more complex. The simplest form is to use SMR which is a ratio of the observed value, y, to the expected deaths to provide an estimate of the relative risk of under-five mortality for a given locality. However, literature has shown that use of SMR is subject to unreliable results but no empirical study has verified that this is so.

Materials and Method
The study utilized data obtained from Uganda Demographic and Health Surveys of 1995, 2001 and 2006. This section provides discussion on the three data sets that were used in the study; their sources and weakness. A summary of the key characteristics of the three UDHS surveys of 1995, 2001 and 2006 are given in Table 1.
The UDHS survey of 1995 covered a total of 37 districts and due to armed conflict; the district of Kitgum located in the northern part of the country was not covered. By the time of the survey, Uganda had a total of 38 districts. A sample of 303 Primary Sampling Units (PSUs) consisting of Enumeration Areas (EAs) were selected from a sampling frame of the 1991 Population and Housing Census and covered a total of 7,070 women in the reproductive age group of 15-49 years. An EA in most cases is equivalent to a village but where the size of the village is big two or more EAs are created. The survey also obtained data from a total of 7,550 households and 1,996 men in a reproductive age group of 15-54 years. The country was clustered into four regions consisting of Central, Eastern, Northern and Western. To permit calculation of contraceptive prevalence rates under a USAID-funded project called DISH (Delivery of Improved Services for Health) a sample design allowed for over sampling of households in the nine districts. These districts were Kasese, Mbarara, Masaka, Rakai, Luwero, Masindi, Jinja, Kamuli and Kampala. Over sampling allowed for a reliable sample size that would purposively allow derive estimates for such areas or districts.
where e i refers to the expected number of deaths in a given district.

Coefficient of variation is defined as;
. *100

Results
Using the 1995 UDHS data, SMR had lower CVs (<100%). This may largely be attributed to the fact that the number of districts was still few by 1995 and the observed counts were fairly substantial. By 1995, there were a total of 38 districts although the demographic and health survey covered 37 due to armed conflict in one of the district. Compared to 2006 where we had a total of 56 districts and even if the sample size had slightly increased, SMR showed a more reliable and stable estimates with fewer districts for either 1995 or 2001. The highest CV was 59.8% obtained for Kabale district.
SMR results using the 2001 UDHS data showed very high variability (>100%) in three districts of Kapchorwa, Kotido and Hoima as shown in Table 2. Overall, other district's coefficient of variation was relatively low indicating low level of 'noise' in the SMR computations. Despite high CVs in the three districts, overall the other districts showed lower CVs and again this might be attributed to the fact that the numbers of districts were still few (34) and subsequently a large number of observations per district to reduce the noise.
Results obtained from the standard deviation of SMR show high values for the districts of Adjumani, Kaberamaido, Kisoro, Mayuge, Moyo and Yumbe. The coefficient of variation (CV) for these districts were relatively very high (>100%) as shown in Table  2 depicting unreliability in utilization of SMR to estimate relative risk of under-five mortality. The 'noise' from SMR results can be attributed to the fact that more districts (56) were introduced that reduced the sample size per district.

Discussion
Coefficient of variation (CV) using SMR for the 1995 UDHS data showed little variability or simply the standard deviations were small. In all the cases for the 37 districts none of them exceeded 60% when using CV. These results further show that when few districts are involved in estimation of SMR and when substantial data points are provided, SMR estimates appear to be a good estimate of relative risk of under-five mortality.
SMR results using the 2001 UDHS data showed very high variability (>100%) in three districts of Kapchorwa, Kotido and Hoima as shown in Table 2. Overall, other district's coefficient of variation was relatively low indicating low level of 'noise' in the SMR computations. Again this may largely be attributed to the fact that the number of districts was still few by 2001 and the observed count (y i ) was fairly substantial.
SMR results using the 2006 UDHS data showed very high CV for the districts of Adjumani, Kaberamaido, Kisoro, Mayuge, Moyo and Yumbe. The coefficient of variation (CV) for these districts were relatively very high (>100%) depicting unreliability in utilization of SMR to estimate relative risk of under-five mortality.

Conclusion
There were 37, 34 and then 56 districts in the UDHS data of 1995, 2001 and 2006 respectively. Since there were fewer districts in the UDHS of 1995 and 2001, there were more observations per districts for these periods. More observations allowed for less volatility in SMR measure compared to the UDHS data of 2006. With UDHS data of 2006 less observation per districts were giving raise to increase 'noise' in the SMR results.

Recommendation
The author therefore recommend that before utilization of SMR, there is need to explore empirically the reliability of the results using simple techniques such as coefficient of variation. We also recommend use of alternative Bayesian approaches like Besang, York, Mollie, Poisson-gamma or the Log-normal models to smoothen the estimates.    The symbol "-" indicates that the district was not yet created