Additional Information on Classification and Regression Tree (CART) Analysis

Contribute to mljs/decision-tree-cart development by creating an account on GitHub. Clone or download. Ml-cart (Classification and regression trees) Decision trees using CART implementation. Npm i ml-cart. API documentation Usage As a classifier. To predict the classification or regression based on the tree (Mdl) and the new data, enter. Ynew = predict(Mdl,Xnew) For each row of data in Xnew, predict runs through the decisions in Mdl and gives the resulting prediction in the corresponding element of Ynew. For more information on classification tree prediction, see the predict.

Helpful Links

Back to: CART Analysis

Classification and regression tree (CART) analysis recursively partitions observations in a matched data set, consisting of a categorical (for classification trees) or continuous (for regression trees) dependent (response) variable and one or more independent (explanatory) variables, into progressively smaller groups (De’ath and Fabricius 2000, Prasad et al. 2006). Each partition is a binary split. During each recursion, splits for each explanatory variable are examined and the split that maximizes the homogeneity of the two resulting groups with respect to the dependent variable is chosen. A typical output from these analyses is shown below (in Figure 1).

Figure 1. A tree diagram for relative abundance of lithophilous fish (i.e., fish that broadcast spawn on gravel beds) with respect to % sand and fines (% S&F, a measure of fine bedded sediment) and watershed area (WA). Branches are annotated showing the decision rules (e.g. % sand and fines < 22.3). Nodes are annotated showing the mean of the dependent variable (n = number of observations, x = mean value, MSE = mean squared error). Data set provided by the Minnesota Pollution Control Agency.

Using CART Analysis in Causal Analysis

CART analysis is used in data exploration to classify systems that differ due to natural causes. CART analaysis may be used to determine the relative importance of different variables for identifying homogeneous groups within the data set.

In Figure 1, the CART analysis results suggest that the relationship between the relative abundance of lithophilous fish and % sand and fines depends on watershed area. Based on this finding, one might consider classifying these sites based on drainage area into sites greater than or less than about 40 km².

CART analyses also might be used to help identify variables that may confound estimates of stressor-response relationships.

Figure 2. Scatterplot of % sand and fines and % of lithophilous fish. Based on the CART analysis depicted in Figure M.8-1, observations with % sand and fines < 22.3% are plotted as closed circles, and observations with % sand and fines > 22.3 % are plotted as open circles. Linear regression (solid lines) and 90th-percentile quantile regression (dashed lines) reveal different slopes and intercepts for the two categories.CART analysis also can help describe stressor-response relationships by identifying the levels of the stressor at which its functional relationship with the biological response might change. This application may be used to help identify inflection points or nonlinearities in a stressor-response relationship (Brenden et al. 2008). Apparent change points then can be investigated using other techniques (e.g., regression analysis) to determine whether they represent thresholds or other change points in the stressor-response relationship. For example, the previous CART analysis (in Figure 1) identified a split in the data set at % sand and fines = 22.3%. Regression analyses demonstrate that two groups are best described by different models: the y intercepts of the mean regression line and both the intercept and slope of the 90th-percentile line decreased for sites where the percentage of sand and fines exceeded 22.3% (in Figure 2). After the model is derived, it would be interpreted in the same way as the results from regression or quantile regression analyses.

Assumptions in Classification and Regression Tree Analysis (CART)

Unlike linear regression techniques, CART analysis does not assume a particular form of relationship between the independent and dependent variables. Therefore, CART can often be used even in cases where data are not suitable for analysis by linear regression. The objective of CART analysis is to create a decision tree that predicts the characteristics of the population of sites being studied. Therefore, the more sites (i.e., examples or observations) presented to the algorithm, the more accurately it will predict the characteristics of the population.

Simplification or “Pruning” of Classification and Regression Trees

Theoretically, CART algorithms could continue to split a data set until there are groups or nodes containing every observation in the data set. In causal analysis, one is generally most interested in the first few splits of a data set. Moreover, to avoid such overfitting of the data, algorithms used in CART generally simplify or “prune” the tree that contains all possible splits of the data to an optimal tree that contains a sufficient number of splits to describe the data.

Many algorithms use stopping criteria that may set the minimum number of observations needed in a group or node in order to split that group, the minimum number of observations needed in a terminal group or node in order to retain that group, or the minimum decrease in the overall lack of fit (i.e., usually the mean squared error) needed for a split of a group or node in order to retain that split. These stopping criteria usually have default values, but they can be set by the user. As some of these criteria are based on the number of observations in a group or node, the number of splits will be dependent on the total number of observations in a data set.

Other options for pruning trees include the use of cross-validation, where the CART analysis is conducted iteratively on random subsets of the data set, or validation of the resulting tree against a second, independent data set. As a result, finding a final tree is a balance between model fit and the size of the data set available for analysis.

Citation: Hu W, O'Leary RA, Mengersen K, Low Choy S (2011) Bayesian Classification and Regression Trees for Predicting Incidence of Cryptosporidiosis. PLoS ONE 6(8):e23903.Zheng Su, Genentech Inc., United States of AmericaReceived: January 24, 2011; Accepted: July 28, 2011; Published: August 31, 2011Copyright: © 2011 Hu et al.

This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.Funding: These authors have no support or funding to report.Competing interests: The authors have declared that no competing interests exist. IntroductionCryptosporidium causes gastrointestinal infection in humans and animals and is now the most common protozoan parasite associated with gastroenteritis. Cryptosporidiosis diseases are sensitive to weather variability as temperature and/or rainfall can influence the development and transmissibility of cryptosporidium and may also affect people's health-related behaviour.

However, there are complex spatio-temporal interactions between the potential explanatory variables of these diseases that motivate further investigation.Spatial dependence and heterogeneity are well known as major features of in spatial analysis of disease risk,. Spatial dependence can arise from the delineation of spatial units of observation (such as suburbs, statistical local areas and counties), spatial aggregation, and the presence of spatial exploratory factors. Spatial heterogeneity is related to the lack of stability over space of the spatial relationships between the observations,.Bayesian methods have been shown to account more sensibly and comprehensively for uncertainty in inference than frequentist methods, particularly with regard to the handling of parameter and model uncertainty,. Bayesian algorithms such as Markov Chain Monte Carlo (MCMC) have allowed for more widespread application of Bayesian methods to many fields of scientific investigation, including public health.Bayesian spatial conditional autoregressive (CAR) models are increasingly being used to estimate spatial variation in disease risk between spatially aggregated units,.

These models are typically represented as a linear regression between the response and explanatory variables with additional terms to explain spatial correlation. These models thus incorporate and estimate spatial correlation while simultaneously estimating covariate effects. Recently, Bayesian spatial and spatiotemporal models have been used to study the geographical distribution of tropical diseases including Ross River virus, malaria and schistosomiasis,.Classification and regression tree (CART) models provide an alternative representation of the relationship between a response variable and potential explanatory variables. These models have been shown to be very useful in identifying and estimating complex hierarchical (high order nonlinear interaction effect) relationships in ecological and medical contexts,. CART models are accepted in many fields of research because they are easy to interpret, more flexible than conventional parametric regression models and have a good predictive power. Bayesian CART models have also been developed, but have yet to be widely applied,.In a previous study we used a frequentist CART model to assess the relationship between social-ecological factors and cryptosporidiosis. In this study we apply the Bayesian CART algorithm developed by O'Leary to predict the spatial distribution of the cryptosporidiosis infection using selected social-ecological factors and climate variables.

We also compare the outcomes of the spatial CART model with those of the Bayesian spatial CAR model. Data collectionThe dataset considered here has been described elsewhere.

Briefly, we obtained the computerised dataset on notified cryptosporidiosis cases by local government areas (LGAs) in Queensland for the period of 1 st January–31 st December 2001 from the Queensland Department of Health. The dataset includes the onset date and place of onset of the notified cases of cryptosporidiosis infection, age and sex of the patients and laboratory test date. Weather (daily temperature and daily rainfall) and socio-economic index for areas (SEIFA) data were obtained for the same period from the Australian Bureau of Meteorology and the Australian Bureau of Statistics, respectively. Confusion or loss matrix – classification of observed versus predicted presence (‘Yes’) and absences (‘No’) from Bayesian CART model.For each tree in the set of good classification trees SC G the following summary statistics can be examined: tree structure (variables, splitting rules and number of terminal nodes), sensitivity, specificity, deviance (−2×log likelihood p( y K, θ k)), log likelihood and log posterior probability.

Bayesian CAR modelAn initial descriptive analysis of cryptosporidiosis was performed. Crude standardised morbidity ratios (SMRs) for each LGA for the whole study period were calculated using standard methods, where SMR = (the observed number of cryptosporidiosis cases)/(the expected number of cryptosporidiosis cases). This model assumed that the observed counts of cases ( O kt) for the kth LGA ( k = 1125) in the tth month in 2001 follow a Poisson distribution with mean ( μ kt), that is, and where α is the intercept, β 1 is the coefficient for temperature, β 2 is the coefficient for rainfall, β 3 is the coefficient for SEIFA, β 4 is the interaction coefficient of temperature and SEFIA, γ is a LGA-level temporal trend coefficients, u is LGA-level variation that is spatially structured (ie. Spatially-structured factors not explained by the model covariates), v is spatially unstructured LGA-level variation, and δ is the amplitude of seasonal oscillation in the month-specific random effects, which was modelled by a sinusoidal term cosine(2π×t/12). Spatial correlation between LGAs was modelled using a CAR prior for u, using a simple adjacency weights matrix.Parameter estimation was obtained via MCMC simulation using an initial burn-in of 5000 iterations and subsequent set 100,000 interactions for estimation.

Convergence was assessed by examining posterior density plots, history plots and autocorrelation of selected parameters. Model selection was performed using the deviance information criterion (DIC), where a lower DIC suggests a better trade-off between model fit and parsimony.

Poisson regression models were developed in a Bayesian framework, using the WinBUGS software version 1.4. Bayesian classification treeA set of five good Bayesian classification trees, with the highest sensitivity, specificity and lowest deviance, are displayed in. The first tree has the highest sensitivity and specificity, and lowest deviance. Since the focus of this case study was on correct prediction of presence (highest sensitivity) the first tree was selected as the best. This tree, depicted in, indicates that presence of cryptosporidiosis was predominantly explained by a high-order nonlinear interaction between temperature, SEIFA and rainfall.

The probability of cryptosporidiosis was largest when temperature was high and rainfall was low, temperature was low and SEIFA was very low, and temperature was low and SEIFA was mid-range but rainfall was low. Quantiles of sensitivity, specificity and log posterior for training and validation datasets over all accepted trees, for Bayesian classification trees.Overfitting of Bayesian classification trees was explored by investigating the quantiles of sensitivity and specificity for training and validation dataset, over all accepted trees. Reveals similar 95% CIs for sensitivity and specificity between the training and validation datasets, indicating no over-fitting.

However, for the validation dataset, the fourth and fifth trees have slightly higher sensitivity than the first tree. Bayesian regression treeThe Bayesian CART algorithm was applied to positive incidence rates of cryptosporidium. The set of five best regression trees (with lowest RSS and deviance) have the same log RSS (−58.96 and −58.47), log posterior (−16.18 and −13.56) and deviance (22.58 and 17.35) for both training and validation dataset respectively. The only difference between these trees is the splitting rules, which have all resulted in the same y observations being classified into the same terminal nodes.

Over the 300,000 iterations, the iteration number for each of these five trees are very different, indicating that the Bayesian CART did not get trapped in local maxima. The first and second trees were designated as the ‘best trees’ since they were most consistently accepted in the set of good trees.The best regression tree modeling positive incidence rates of cryptosporidium is displayed in. There are three groups of positive incidence rates of cryptosporidium, ranging from low to high incidence. A monthly mean incidence rate of cryptosporidium of 78.22/100,000 (n = 105; far left terminal node) occurs in areas with temperatures less than or equal to 28.5° and SEIFA less than or equal to 1033.8.

The monthly mean incidence rate is reduced to 4.73/100,000 when temperatures are the same but SEIFA is greater than 1033.8. The highest monthly mean incidence rate (134.76/100,000) occurs when the temperature is greater than 28.5°.The quantiles of log RSS, deviance and log posterior (distribution of data given the tree structure) over all accepted regression tees are displayed in.

The Bayesian regression tree algorithm search space includes trees with low to high RSS, deviance and log posterior. There was no evidence over-fitting with Bayesian regression trees since there was little difference in log RSS and deviance between training and validation datasets. Comparison with frequentist CART modelsWe also compared the outcomes of the Bayesian CART model with those of the traditional CART model. Both the Bayesian CART and traditional CART models show that SEIFA and temperature were associated with the cryptosporidiosis disease.

However, the analyses indicate that Bayesian CART gave slightly better prediction accuracy (ie. High sensitivity) (sensitivity Bayeisan:79%; specificity Bayesian: 50%) than the CART accuracy (sensitivity frequentist: 10%; specificity frequentist: 99%) established using the more traditional frequentist approach. An important difference between the two models was that the frequentist tree gave equal weighting to correct classification of all observations, whereas the Bayesian tree differentially weighted the groups of presences and absences based on the respective sample size. DiscussionBoth the Bayesian CART and Bayesian CAR models show that temperature was significantly associated with the cryptosporidiosis disease. The analyses indicate that the nature and magnitude of the effect estimates were similar for the two methods used in this study. However, the Bayesian CART allowed more flexible identification and description of nonlinear interactions between explanatory or predictor variables, while still allowing for local smoothing.The Bayesian CART model revealed a strong nonlinear interaction between SEIFA and temperature, and a weaker interaction with rainfall, in predicting incidence rate of cryptosporidiosis. In contrast, because only main effect term and one interaction term (ie.

Temperature and SEIFA) were included in the spatial CAR model, other interactions were not identified. Although other interactions (ie. Temperature, rainfall and SEIFA) could of course be included in the CAR model, it is difficult to identify a priori which interactions to include and evaluation of all possible interactions would require a much larger dataset than was available here.We also considered including these interactions in a spatial CAR hurdle model, which allows for zero-inflation by having a probability mass at zero, but found this to be difficult to fit in terms of stability and interpretability of the estimates and corresponding predictions. This is possibly not surprising given that the discretisation of the data into two components (zero and non-zero) may impact on the representation of the spatial component in the model, especially when taking into interactions into account. This requires further future investigation. In the meantime, a posteriori inclusion of interactions, based on the CART, into the CAR model analyses is a potentially useful alternative.A strong advantage of a Bayesian framework for the CAR and CART models is that all the parameters of the model are treated as variables, so that probabilistic inferences are made on the basis of the corresponding posterior distributions. Moreover, by virtue of the MCMC computation, the distributions used to describe these variables are no longer constrained to analytically tractable (e.g., normal) formulations.

Furthermore, under a Bayesian CART framework, a diverse range of tree structures can be readily explored. The typical frequentist approach of fitting the CART model uses single recursive partitioning algorithms, in which the choices of the splitting rules at nodes further down the tree are constrained by the choices made at nodes above it, and only get one optimal tree. In contrast, the Bayesian CART approach investigates a wide variety of tree structures with different variables, splitting rules and number of terminal nodes. At any splitting node, the variable and splitting rules are randomly selected from the prior and trees that perform well in terms of high likelihood (low deviance) and posterior probabilities are chosen. Accounting for model uncertainty in this manner can improve predictive performance.A Bayesian CART model for identification and estimation of the spatial distribution of disease risk can be useful in monitoring and assessment of infectious diseases and in decision-making about prevention and control. The methodology developed through this study may be directly applicable to research on other infectious diseases, with further potential for application to a wider range of public health problems. References.

Telestream Wirecast Pro 13.1.3 Crack competent video broadcasting tool from Telestream, designed for all needs and everybody power ranges. Additionally, the Program features a superior broadcast atmosphere but is prominent and very easy to use. Telestream wirecast pro serial number key for mac pro.

1.Meinhardt P, Casemore D, Miller K (1996) Epidemiologic aspects of human cryptosporidiosis and the role of waterborne transmission. Epidemiol Rev 18: 118–136. 2.Mabaso M, Vounatsou P, Midzi S, Silva J, Smith T (2006) Spatio-temporal analysis of the role of climate in inter-annual variation of malaria incidence in Zimbabwe. Int J Health Geog 5: 20. 3.Moore D, Carpenter T (1999) Spatial analytical methods and geographic information systems: use in health research and epidemiology. Epidemiol Rev 21: 143–161. 4.Anselin L (2002) Under the hood - Issues in the specification and interpretation of spatial regression models.

Agric Econo 27: 247–267. 5.Anselin L (2005) Exploring spatial data with GeoDa: a workbook. Urbana, USA. 6.Duc H, Jalaludin B, Morgan G (2009) Associations between Air Pollution and Hospital Visits for Cardiovascular Diseases in the Elderly in Sydney Using Bayesian Statistical Methods.

Aust N Z J Stat 51: 289–303. 7.Hoeting J, Raftery AE, Madigan D (1996) A method for simultaneous variable selection and outlier identification in linear regression. Comput Stat Data An 22: 251–270. 8.Lamon EC 3rd, Stow CA (2004) Bayesian methods for regional-scale eutrophication models. Water Res 38: 2764–2774. 9.Lawson A, Browne W, Vidal Rodeiro C (2003) Disease mapping with WinBUGS and MLwiN.

England: John Wiley & Sons Ltd. 10.Escaramis G, Carrasco J, Ascaso C (2007) Detection of significant disease risks using a spatial conditional autoregressive model. Biometrics 64: 1043–1053. 11.Beale CM, Lennon JJ, Yearsley JM, Brewer MJ, Elston DA (2010) Regression analysis of spatial data. Ecol Lett 13: 246–264. 12.Yang G, Vounatsou P, Zhou X, Tanner M, Utzinger J (2005) A Bayesian-based approach for spatio-temporal modeling of county level prevalence of Schistosoma japonicum infection in Jiangsu province, China. Int J Parasitol 35: 155–162.

13.Clements A, Lwambo N, Blair L, Nyandindi U, Kaatano G, et al. (2006) Bayesian spatial analysis and disease mapping: tools to enhance planning and implementation of a schistosomiasis control programme in Tanzania. Trop Med Int Health 11: 490–503. 14.Hu W, Clements A, Williams G, Tong S, Mengersen K (2010) Bayesian spatiotemporal analysis of socio-ecologic drivers of Ross River virus transmission in Queensland, Australia. Am J Trop Med Hyg 83: 722–728. 15.Breiman L, Fredman J, Olshen R, Stone C (1984) Classification and regression trees. New York: Chapman & Hall (Wardworth, Inc).

16.De'ath G, Fabricius K (2000) Classification and regression trees: a powerful yet simple technique for ecological data analysis. Ecology 81: 3178–3192.

17.Hu W, Mengersen K, Dale P, Tong S (2010) Difference in mosquito species (Diptera: Culicidae) and the transmission of Ross River virus between coastline and inland areas in Brisbane, Australia. Environ Entomol 39: 88–97. 18.Hu W, Tong S, Mengersen K, Oldenburg B, Dale P (2006) Mosquito species (Diptera: Culicidae) and the transmission of Ross River virus in Brisbane, Australia. J Med Entomol 43: 375–381. 19.Chipman HA, George EI, McCulloch RE (1998) Bayesian CART model search. J Am Stat Assoc 93: 935–948. 20.Denison DGT, Mallick BK, Smith AFM (1998) A Bayesian CART algorithm.

Biometrika 85: 363–377. 21.O'Leary R, Francis R, K C, Firth M, Kees U, et al.

(2009) A comparison of Bayesian classification trees and random forest to identify classifiers for childhood leukaemia. 18th World IMACS/MODSIM Congress. Cairns, Australia. 22.O'Leary R (2008) Informed statistical modelling of habitat suitability for rare and threatened species PhD Thesis.

Brisbane: Queensland University of Technology. 23.O'Leary R, Murray J, Low Choy S, Mengersen K (2008) Expert elicitation for Bayesian classification trees. J Appl Probab Stat 3: 95–106. 24.Hu W, Mengersen K, Tong S (2010) Risk factor analysis and spatiotemporal CART model of cryptosporidiosis in Queensland, Australia.

BMC Infect Dis 10: 311. 25.Cordell H (2009) Detecting gene-gene interactions that underlie human diseases. Nat Rev Genet 10: 392–404. 26.Green P (1995) Reversible jump Markov chain Monte Carlo computation and Bayesian model determination.

Biometrika 82: 711–732. 27.Chipman HA, George EI, McCulloch RE (2010) Bart: Bayesian Additive Regression Trees. Annals of Applied Statistics 4: 266–298. 28.Gelman A, Carlin J, Stern H, Rubin D (2004) Bayesian data analysis (2nd ed). Florida: Chapman & Hall/CRC.

29.Cameron A, Trivedi P (1998) Regression Analysis of Count Data. Cambridge: Cambridge University Press. 30.WinBUGs (2008) MRC Biostatistics Unit.

Imperial College London, Cambridge, UK. 31.Therneau T, Atkinson E (1997) An Introduction to Recursive Partitioning Using the rpart Routine. Rochester. 32.Therneau T, Atkinson E (2003) The rpart package. Software manual.