A penalized estimation of the variogram and effective sample size
No Thumbnail Available
Date
2025
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
The variogram function plays a key role in modeling intrinsically stationary random fields, especially in spatial prediction using kriging equations. However, determining whether a computed variogram accurately fits the underlying dependence structure can be challenging. Current nonparametric estimators often fail to guarantee a conditionally negative definite function. In this paper, we propose a new valid variogram estimator, constructed as a linear combination of functions from a predefined class, ensuring it meets essential mathematical properties. A penalty coefficient is introduced to prevent overfitting, reducing spurious fluctuations in the estimated variogram. We also extend the concept of effective sample size (ESS), an important metric in spatial regression, to a nonparametric framework. Our ESS estimator is based on the reciprocal of the average correlation and is calculated using a plug-in approach, with the consistency of the estimator being demonstrated. The performance of these estimates is investigated through Monte Carlo simulations across various scenarios. Finally, we apply the methodology to rasterized forest images, illustrating both the strengths and limitations of the proposed approach.
Description
Keywords
Nonparametric estimates, Covariance functions, Information, Bessel functions, Penalty coefficient