Comparing K Nearest Neighbours Methods and Linear Regression – Is There Reason To Select One Over the Other?
Parole chiave:
Modelling, Regression, Imputation, Balanced Data, K Nearest NeighbourAbstract
Non-parametric k nearest neighbours (k-nn) techniques are increasingly used in forestry problems, especially in remote sensing. Parametric regression analysis has the advantage of well-known statistical theory behind it, whereas the statistical properties of k-nn are less studied. In this study, we compared the relative performance of k-nn and linear regression in an experiment. We examined the effect of three different properties of the data and problem: 1) the effect of increasing non-linearity of the modelling task, 2) the effect of the assumptions concerning the population and 3) the effect of balance of the sample data. In order to be able to determine the effect of these three aspects, we used simulated data and simple modelling problems. K-nn and linear regression gave fairly similar results with respect to the average RMSEs. In both cases, balanced modelling dataset gave better results than unbalanced dataset. When the results were examined within diameter classes, the k-nn results were less biased than regression model results, especially with extreme values of diameter. The differences increased with increasing non-linearity of the model and increasing unbalance of the data. The difference between the methods was more obvious when the assumed model form was not exactly correct.
Riferimenti bibliografici
Altman, N.S. 1992. An introduction to kernel and nearest-neighbor nonparametric regression. Journal of the American Statistical Association 46: 175-185.
Dobbertin, M. and G.S. Biging. 1997. Using the non-parametric classifier CART to model forest tree mortality. For. Sci. 44(4): 507-516.
Eskelson, B.N.I., H. Temesgen, V. LeMay, T.M. Barrett, N.L. Crookston, and A.T. Hudak. 2009. The roles of nearest neighbor methods in imputing missing data in forest inventory and monitoring databases. Scand. J. For. Res. 24(3): 235-246.
Fan, J. 2000. Prospects of nonparametric modelling. Journal of American Statistical Association 95: 1296-1300.
Fehrmann, L., A. Lehtonen, C. Kleinn, and E. Tomppo. 2008. Comparison of linear and mixed-effect regression models and a k-nearest neighbour approach for estimation of single-tree biomass. Can. J. For. Res. 38: 1-9.
Gibbons, J.D. and S. Chakraborti. 1992. Nonparametric statistical inference. Third Edition. Marcel Dekker, Inc. New York, 544 p.
Haara, A. 2002. Kasvuennusteiden luotettavuuden selvittäminen k-nn-menetelmällä ja monitavoiteoptimoinnilla. Metsätieteen aikakauskirja 3/2002: 391-406. (In Finnish)
Haara, A., M. Maltamo, and T. Tokola. 1997. The k-nearest-neighbour method for estimating basal-area diameter distribution. Scand. J. For. Res. 12: 200-208.
Hamilton, D.A. 1990. Extending the range of applicability of an individual tree mortality model. Can. J. For. Res. 20: 1212-1218.
Härdle, W. 1989. Applied nonparametric regression. Cambridge University Press. Cambridge, 323 p.
Hooke, R. and T.A. Jeeves. 1961. Direct search solution of numerical and statistical problems. Journal of the ACM 8: 212-229.
Magnussen, S., E. Tomppo, and R.E. McRoberts. 2010. A model-assisted k-nearest neighbour approach to remove extrapolation bias. Scand. J. For. Res. 25: 174-184.
Malinen, J. 2003. Locally Adaptable Non-parametric Methods for Estimating Stand Characteristics for Wood Procurement Planning. Silva Fennica 37: 109-120.
Maltamo, M., E. Næsset, O.M. Bollandsås, T. Gobakken, and P. Packalén. 2009. Non-parametric prediction of diameter distributions using airborne laser scanner data. Scand. J. For. Res. 24(6): 541-553.
McRoberts, R.E., M.D. Nelson, and D.G. Wendt. 2002. Stratified estimation of forest area using satellite imagery, inventory data, and the k-Nearest Neighbors technique. Remote Sens. Environ. 82: 457-468.
McRoberts, R.E. 2009. Diagnostic tools for nearest neighbors techniques when used with satellite imagery. Remote Sens. Environ. 113: 489-499.
Metcalf, C.J.E., S.M. McMahon, and J.S. Clark. 2009. Overcoming data sparseness and parametric constraints in modeling of tree mortality: a new nonparametric Bayesian model. Can. J. For. Res. 39: 1677-1687.
Moeur, M. and A.R. Stage. 1995. Most Similar Neighbor. An improved sampling inference procedure for natural resource planning. For. Sci. 41: 337-359.
Osyczka, A. 1984. Multicriterion optimization in engineering with Fortran programs. Ellis Horwood, Chichester, 178 p.
Sironen, S., A. Kangas, M. Maltamo, and J. Kalliovirta. 2008. Localizing of growth estimates using non-parametric imputation methods. For. Ecol. Manage. 256: 674-684.
Sironen, S., A. Kangas, and M. Maltamo. 2010. Comparison of different non-parametric growth imputation methods in the presence of dependent observations. Forestry 83: 39-51.
Temesgen, H. 2003. Estimating stand tables from aerial attributes: a comparison of a parametric prediction and most similar neighbour methods. Scand. J. For. Res. 18: 279-288.
Tomppo, E. and M. Halme. 2004. Using coarse scale forest variables as ancillary information and weighting of variables in k-nn estimation: a genetic algorithm approach. Remote Sens. Environ. 92: 1-20.
Tomppo, E. 2006. Finnish NFI, p. 295-308 in: Kangas, A. and M. Maltamo. (eds.), Forest Inventory. Methodology and Applications. Managing Forest Ecosystems, vol. 10, Springer (2006).
Valtakunnan metsien 8. inventointi. Pysyvien koealojen kenttätyön ohjeet 1985-86. 2. painos. Finnish Forest Research Institute. (in Finnish)
Vieilledent, G., B. Courbaud, G. Kunstler, J-F. Dhôte, and J.S. Clark. 2009. Biases in the estimation of size-dependent mortality models: advantages of a semi-parametric approach. Can. J. For. Res. 39: 1430-1443.
Dowloads
Pubblicato
Fascicolo
Sezione
Licenza
Authors who publish with this journal agree to the following terms:- Authors grant the journal right of first publication, with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) following the publication by the journal, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
