Implications of sampling design and sample size for national carbon accounting systems
 Michael Köhl^{1, 2}Email author,
 Andrew Lister^{3},
 Charles T Scott^{3},
 Thomas Baldauf^{2} and
 Daniel Plugge^{1}
https://doi.org/10.1186/17500680610
© Köhl et al; licensee BioMed Central Ltd. 2011
Received: 25 August 2011
Accepted: 8 November 2011
Published: 8 November 2011
Abstract
Background
Countries willing to adopt a REDD regime need to establish a national Measurement, Reporting and Verification (MRV) system that provides information on forest carbon stocks and carbon stock changes. Due to the extensive areas covered by forests the information is generally obtained by sample based surveys. Most operational sampling approaches utilize a combination of earthobservation data and insitu field assessments as data sources.
Results
We compared the costefficiency of four different sampling design alternatives (simple random sampling, regression estimators, stratified sampling, 2phase sampling with regression estimators) that have been proposed in the scope of REDD. Three of the design alternatives provide for a combination of insitu and earthobservation data. Under different settings of remote sensing coverage, cost per field plot, cost of remote sensing imagery, correlation between attributes quantified in remote sensing and field data, as well as population variability and the percent standard error over total survey cost was calculated. The costefficiency of forest carbon stock assessments is driven by the sampling design chosen. Our results indicate that the cost of remote sensing imagery is decisive for the costefficiency of a sampling design. The variability of the sample population impairs costefficiency, but does not reverse the pattern of costefficiency of the individual design alternatives.
Conclusions, brief summary and potential implications
Our results clearly indicate that it is important to consider costefficiency in the development of forest carbon stock assessments and the selection of remote sensing techniques. The development of MRVsystems for REDD need to be based on a sound optimization process that compares different data sources and sampling designs with respect to their costefficiency. This helps to reduce the uncertainties related with the quantification of carbon stocks and to increase the financial benefits from adopting a REDD regime.
Keywords
Background
In the 1990's tropical deforestation was estimated to cause approximately 20 percent of the global anthropogenic carbon emissions [1]. Between 1997 and 2006, deforestation, forest degradation and peatland fires contributed between 8 and 20 percent to the global anthropogenic carbon emissions [2]. FAO [3] estimated an annual loss of carbon stocks in forest biomass of 0.5 Gt between 1990 and 2010, which is considered to be mainly a result of tropical deforestation. At their 16th meeting in Cancun in 2010, the Parties of the United Nations Framework Convention on Climate change (UNFCCC) approved the inclusion of a reduction of emissions from deforestation and forest degradation (REDD) mechanism as an eligible action to prevent climate changes and global warming in post2012 commitment periods of the Kyoto Protocol (KP).
So far no financial value has been assigned to the carbon stored in forests. Decisions about future land use are driven by the potential income from alternative forms of land management rather than maintaining forests as nondisposable intangible assets. REDD introduces a new land use paradigm in which developed countries provide financial resources for incentives to developing countries to reduce carbon emissions from deforestation and forest degradation. Financial benefits are based on quantified carbon emission reductions relative to a preestablished reference level [4]. Due to the financial arrangements between developed and developing countries participating in a future REDD mechanism, there is a requirement for reliable and verifiable data on carbon emission reduction efforts [5]. Countries willing to adopt a REDD regime need to establish a national system for Measurement, Reporting and Verification (MRV) that provides information on forest carbon stock changes. While some authors see MRV systems as easytoapply tools [6], others describe the difficulties of implementation and operational applications [7–9].
The objective of this paper is to demonstrate the implications of sampling designs and sample sizes on the costefficiency of the measurement component of MRV systems. We chose four sampling approaches and anticipated different cost schemes for field surveys and remote sensing imagery to show the effect of both the inventory designs and the associated costs on the costefficiency and reliability of carbon inventory and monitoring systems. Assumptions and methods used in our study are compatible with those laid down in the IPCC GPG (Intergovernmental Panel on Climate Change Good Practice Guidance) [10].
Estimating forest carbon stock changes includes assessments of deforestation rates and associated carbon stock loss, afforestation and reforestation rates and associated carbon stock gains, and changes of carbon stocks in forests that remain forests. The approach presented in the IPCC GPG quantifies emissions or removals from carbon stocks within a given period as the product of the extent of human activity (activity data, AD) and the emissionsremovals ratio per unit of activity (emission factor, EF). Information on AD and EF can be obtained in different ways, the most complex and reliable being from detailed, spatially dense forest monitoring and modeling data. The GPG classify the approaches in three categories (so called "Tiers") [11] with respect to requirements for data, analysis procedures, and reliability. Since continuous forest inventory data collected with a valid statistical sampling design allows for complex assessment and analysis procedures and results in reliable estimates with known (sampling) errors, they are assigned the highest Tier, i.e. Tier 3.
The IPCC Guidelines use six broad landuse categories to report emissions and removals from land use and land use conversions: forest land, cropland, grassland, wetlands, settlements, and other land. The six categories can be further subdivided on the national level to capture differences between climate, soil, ecological zones, and management practices [11]. In addition, IPCC defines five carbon pools which are to be considered for reporting carbon stock changes on forest land: aboveground and belowground biomass, dead wood, litter, and soil organic matter.
In forest inventories changes are generally assessed as the difference of an attribute (e.g. forest area, timber or biomass volume, stand age, timber value, carbon stock) between successive occasions [12–15]. This approach conforms to the socalled "stock difference method", which is along with the "the gainloss method" presented by IPCC [11] to assess carbon stock changes.
Inferences from monitoring data and associates errors
Inference  

No change  Change  
Actual state  No change  correct  False positive (Type 1 error, α) 
Change  False negative (Type 2 error, β)  correct 
The reliability of results can be quantified by giving their precision, accuracy, mean square error or bias. These words are often used synonymously in colloquial speech, but they are deliberately contrasted in the context of sampling statistics. In the following, we show the definitions of precision, bias, mean square error, and accuracy.
Precision
Precision refers to the size of deviations from the estimated mean, $\hat{\mu}$, obtained by repeated application of a sampling procedure. It is quantified by the standard error or confidence intervals. The precision of a statistical estimate can be increased by increasing the number of observations.
Bias
Bias, B, is the difference between the estimated mean and the true mean, thus is directly related to the accuracy of an estimate, as $\mathsf{\text{B}}=\hat{\mu}\mu $. A problem in surveys is that the presence of bias, i.e. the lack of accuracy, is often not known.
Mean square error
Accuracy
Accuracy refers to the size of deviations from the true mean, μ. It relates directly to the MSE. When comparing two estimators, the one with the smaller MSE is said to be more accurate [17].
For unbiased estimates the MSE and the precision are asymptotically identical. As the concept of MSE and the underlying figures are often not intuitively understood by many stakeholders, the use of confidence intervals is suggested [16, 18]. Confidence intervals give an estimated range of values, which is calculated from the sample data and which are likely to include the unknown population (true) value. Albeit the GPG and other publications suggest differently [10, 11], confidence intervals account for precision only and do not address bias or other nonsampling errors. The selection of a confidence level (e.g., 95%) specifies the probability that the confidence interval calculated will include the true parameter value. In forestry applications, especially in research, a common choice is a 95%confidence level, which says that in 95% of the time, if repeated samples are taken with the same methods, the confidence interval that is generated will contain the true parameter value [15–17].
Dawkins [19] introduced the lower bound of a confidence interval as a surrogate for the minimum quantity to be expected with a given probability. The lower bound of confidence intervals can serve as a proxy for the Reliable Minimum Estimate (RME) which the IPCCGood Practice Guidance suggests for addressing uncertainties in the assessment of changes in soil carbon. In the context of afforestation and reforestation activities under the Clean Development Mechanism (CDM) [20, 21] the RME as a conservative measure has already been reflected in several UNFCCC documents. Grassi et al. [22] propose using the principle of conservativeness in order to "address the potential incompleteness and high uncertainties of REDD estimates".
Confidence intervals and standard errors are strongly influenced by the variability of the target population. IPCC [10] presents the variability of above ground biomass stock for different forest formations. Inventory concepts need to take into account both the required precision and budget constraints, in order to come up with an optimal inventory design. Countries in a REDDreadiness or demonstration phase [23] need to pay special attention to the costefficiency of proposed REDD monitoring concepts. It is good practice to evaluate alternative sampling concepts under the criterion of cost efficiency [24]. However, in the vast number of publications on REDD monitoring schemes the aspect of inventory cost seems to have been neglected. An exception is Hardcastle and Baird [25], who present a cost assessment for measuring and monitoring forest carbon for 25 countries. The cost figures they present are indicative of the levels of funding that would be required to achieve reporting at different Tierlevels ignoring and including degradation.
Sampling design alternatives
Data sources and sampling designs for REDD monitoring
Data sources  Sampling design  Estimation procedures for means and variances of the mean (from [15]) 

Field plots  Simple random sampling  $\overline{\text{y}}\text{=}\frac{{\displaystyle \sum _{\text{i}=1}^{\text{n}}{\text{y}}_{\text{i}}}}{\text{n}};\text{v(}\overline{\text{y}}\text{)=}\frac{{\displaystyle \sum _{\text{i}=1}^{\text{n}}{({\text{y}}_{\text{i}}\overline{\text{y}})}^{2}}}{\text{n}(\text{n}1)}$ 
Systematic sampling  
Field plots & full coverage remote sensing imagery  Stratified sampling Example: field plots and Spot satellite imagery  ${\overline{\text{y}}}_{\text{st}}={\displaystyle \sum _{\text{h}=\text{1}}^{\text{L}}\frac{{\text{A}}_{\text{h}}}{\text{A}}\frac{{\displaystyle \sum _{\text{i}=\text{1}}^{{\text{n}}_{\text{h}}}{\text{y}}_{\text{hi}}}}{{\text{n}}_{\text{h}}}}$ $\text{v}\left({\overline{\text{y}}}_{\text{st}}\right)={\displaystyle \sum _{\text{h}=\text{1}}^{\text{L}}\frac{{{\left(\frac{{\text{A}}_{\text{h}}}{\text{A}}\right)}^{\text{2}}}^{\phantom{\rule{0.1em}{0ex}}}{\text{s}}_{\text{h}}^{\text{2}}}{{\text{n}}_{\text{h}}}}$; 
Regression estimators Example: field plots and TerraSarX (radar)  ${\overline{\text{y}}}_{\text{lr}}=\overline{\text{y}}+\text{b(}\overline{\text{X}}\overline{\text{x}}\text{)}$ $\text{v}\left({\overline{\text{y}}}_{\text{lr}}\right)\approx \frac{{\displaystyle \sum _{\text{i}=\text{1}}^{\text{n}}{\left[\left({\text{y}}_{\text{i}}\overline{\text{y}}\right)\text{b}\left({\text{x}}_{\text{i}}\overline{\text{x}}\right)\right]}^{\text{2}}}}{\text{n(n}\text{2)}}$  
Field plots & partial coverage remote sensing imagery  2phase sampling for stratification (double sampling for stratification) Example: field plots & aerial photography  ${\stackrel{\u0304}{\mathsf{\text{y}}}}_{\mathsf{\text{ds}}}=\sum _{\mathsf{\text{h}}=1}^{\mathsf{\text{L}}}\frac{{\mathsf{\text{n}}}_{\mathsf{\text{h}}}^{\prime}}{\mathsf{\text{n\u2019}}}{\stackrel{\u0304}{\mathsf{\text{y}}}}_{\mathsf{\text{h}}}=\sum _{\mathsf{\text{h}}=1}^{\mathsf{\text{L}}}{\mathsf{\text{w}}}_{\mathsf{\text{h}}}{\stackrel{\u0304}{\mathsf{\text{y}}}}_{\mathsf{\text{h}}}$ $v\left({\overline{\mathsf{\text{y}}}}_{\mathsf{\text{ds}}}\right)=\frac{N1}{N}\sum _{\mathsf{\text{h=1}}}^{\mathsf{\text{L}}}\left(\frac{{\mathsf{\text{n}}}_{\mathsf{\text{h}}}1}{\mathsf{\text{n}}\prime \mathsf{\text{1}}}\frac{{\mathsf{\text{n}}}_{\mathsf{\text{L}}}^{\prime}1}{\mathsf{\text{N}}1}\right)\frac{{\mathsf{\text{W}}}_{\mathsf{\text{h}}}^{\prime}{\mathsf{\text{s}}}_{\mathsf{\text{h}}}^{2}}{\mathsf{\text{n}}\prime}+\frac{\mathsf{\text{Nn}}\prime}{\mathsf{\text{N(n}}\prime 1\mathsf{\text{)}}}\sum _{\mathsf{\text{h=1}}}^{\mathsf{\text{L}}}{\mathsf{\text{W}}}_{\mathsf{\text{h}}\left({\overline{\mathsf{\text{y}}}}_{\mathsf{\text{h}}}{\overline{\mathsf{\text{y}}}}_{\mathsf{\text{ds}}}\right)}$ where ${s}_{h}^{2}=\frac{1}{{n}_{h}^{\prime}}\sum _{h=1}^{L}{\left({y}_{hi}{\overline{y}}_{\mathsf{\text{h}}}\right)}^{2}$ 
2phase sampling with regression estimators Example: field plots & LiDAR  ${\overline{\text{y}}}_{\text{lr}}=\overline{\text{y}}+\text{b(}\overline{\text{x}}\text{'}\overline{\text{x}}\text{)}$ $v\left({\overline{y}}_{lr}\right)={s}_{y.x}^{2}\left[\frac{1}{n}+\frac{{\left(\overline{x}\text{'}\overline{x}\right)}^{2}}{{\displaystyle \sum {\left(\overline{x}\text{'}\overline{x}\right)}^{2}}}\right]+\frac{{s}_{y}^{2}{s}_{y.x}^{2}}{n\text{'}}\frac{{s}_{y}^{2}}{N}$ with ${s}_{y.x}^{2}=\frac{1}{n2}\left[{\displaystyle \sum _{i=1}^{n}{\left({y}_{i}\overline{y}\right)}^{2}{b}^{2}}{\displaystyle \sum _{i=1}^{n}{\left({x}_{i}\overline{x}\right)}^{2}}\right]$ 
In forest surveys, simple random sampling (SRS) and, more commonly, systematic sampling, are typically used. In SRS, sampling units are chosen randomly. In systematic sampling, they are arranged in a systematic pattern, usually on a square grid or other regular geometric network. The starting point of the geometric network of sampling units is generally the only element of randomization in systematic sampling. However, some, such as the US Forest Inventory and Analysis (FIA) program, randomize within each cell of a hexagonal tessellation of the study area [26]. Ranneby [27] and Matérn and Ranneby [28] studied exact approaches to calculate variances from systematic sampling in a forest inventory context, and determined that using SRS variance equations results in overestimates of sampling error. In forest surveys it is good practice to approximate the standard errors of systematic sampling designs by the SRS equations and accept the overestimation of the sampling errors.
Combined insitu/earth observation sample designs use auxiliary information obtained by remote sensing and field sampling systems simultaneously. The earth observation data can consist of derived data, such as a classification of remote sensing data into landuse strata, or unprocessed reflectance data from optical, radar or LiDAR sensors. Variables of interest such as biomass or carbon stock are assessed on a small sample of field plots, and these data are combined with the more denselysampled earth observation data using statistical estimation procedures in order to generate estimates.
The use of spatially continuous earth observation datasets generally leads to stratified sampling or regression sampling designs. Regression estimators relate an auxiliary variable, which is measured or known for all population elements, N, to a variable of interest, which is assessed on a subsample of size n. Regression estimators are applicable whenever the constraints for the application of linear regression are satisfied. In practical applications, the assumptions and constraints of linear regression such as sufficient data across the entire value range, or homoscedasticity, can easily be violated for small units.
In stratified sampling thematic classes are obtained by classifying the remote sensing imagery and assigning the individual pixels (or polygons) into a fixed number of groups (strata). Thus, the idea of stratified sampling is to divide the population of N units into nonoverlapping subpopulations of N1, N2, ..., NL units. The subpopulations are called strata. The strata are constructed to minimize the variance within strata, thus maximizing the differences between strata means. The characteristics of classes suitable for stratification do not necessarily reflect thematic information that is suitable for map production. In many cases combinations of thematic maps (i.e., different classification schemes) or totally artificial (i.e., thematically "meaningless") classes prove best for stratification purposes [15]. The n samples can be assigned to the strata equally, proportional (i.e., in proportion to strata sizes), optimal (i.e., by strata sizes and strata variances), or by Neyman allocation that in addition to strata size and variance includes the assessment costs per stratum [16]. For monitoring purposes proportional allocation proves most feasible, because changes in stratum assignments over time do not affect the probabilities of selection, thus complicating estimation [29, 30].
In extensive surveys of large areas it is sometimes not possible to acquire fullcoverage remote sensing imagery. That holds especially true when airborne instead of spaceborne remote sensing data are to be used. Here, twophase sampling designs offer an alternative by sampling both the variable of interest as well as the auxiliary variable. Stratified sampling and regression estimators can be applied as twophase sampling for stratification and twophase sampling with regression estimators.
In twophase sampling with regression estimators the auxiliary variable x_{i} is measured on a subsample of N. In this first phase a large sample of size n' is selected. In the second phase a random subsample of n' is selected where both x_{i} and y_{i} are measured and related via regression models. Twophase sampling with regression estimation results in specific problems when used in practical applications. Among those problems are the need for calculating regression estimates for any variable considered, the assumptions for regression may be violated, no additive tables are obtained (table cells and margins are modeled separately), or not being able to analyze data on nominal and ordinal scales [31].
Twophase sampling for stratification is similar to stratified sampling  the difference being that the strata sizes are not measured but estimated by the large first phase sample. The variance $\text{v(}{\overline{\text{y}}}_{\text{ds}}\text{)}$ combines the within and between strata variation. For large N, $\text{v(}{\overline{\text{y}}}_{\text{st}}\text{)}$ can be used as an approximation for $\text{v(}{\overline{\text{y}}}_{\text{ds}}\text{)}$.
For sample sizes that are sufficiently large (n > 60), the Student's tvalue corresponds to the value of the normal deviate with the desired probability, e.g., t = 1.96 for 95% confidence levels with large sample sizes.
Selecting the optimal design
Many inventory concepts have been presented for monitoring carbon stocks and carbon stock changes in the scope of REDD. Irrespective of the objective of a survey alternative, inventory concepts exist to choose from, including the utilized data sources (field assessments, remote sensing, maps etc.), the design of the sampling units (plot configuration), sampling rules and sample sizes. The potential design alternatives are influenced by a variety of factors such as the variability of the target population, budget allowance, or availability of auxiliary data sources and information (e.g. maps, remote sensing imagery, biomass models). A rational decision about the optimal design can be made only by comparing the set of alternatives under objective selection criteria that combine information on survey cost and the achievable reliability of the results. This allows for selecting the most costefficient design that either provides the best reliability under a given budget or provides the desired reliability by least cost. Discussions on survey design alternatives that lack the inclusion of cost are not very helpful for developing operational MRVsystems under a national REDD regime.
Survey Costs
Survey costs are made up of fixed and variable cost components. Fixed costs are those that do not vary with sample sizes and design alternatives, but are common to all alternatives, for example cost for administration or research. As fixed costs are design independent they are not to be considered in the optimization process [24, 32]. Design dependent costs include additional fixed costs for specific design alternatives and variable costs. Costs for visiting and measuring field samples are a typical example of variable costs, which are proportional to the number of field samples assessed. For stratified sampling, additional costs include acquisition, enhancement, and classification of remote sensing data as well as validation of the classification results.
 1.
Tier 2, Approach A: an accurate landcover map is available, 300 sample plots are assessed insitu, all carbon measurements are performed once at the beginning of the programme, future monitoring is focused on the assessment of human activities (activity data, AD) such as area changes by remote sensing data and requires only minimal field work.
 2.
Tier 2, Approach B: no accurate landcover map is available, insitu assessments are performed when activity monitoring by remote sensing identifies locations under change, the insitu sampling intensity is considerably lower than under Tier 2, Approach A.
 3.
Tier 3, ignoring degradation: AD and emissions per unit of the activity (emission factors, EF) are assessed as under alternative 1 (Tier 2 Approach A), but remeasurements are made in permanent insitu sample plots (about 1/3 of the original sample locations)
 4.
Tier 3, including degradation: alternative 3 is enhanced by further stratification of forests into the two classes "intact forests" and "nonintact forests", the number of field plots is moderately increased
The inventory concepts applied by Hardcastle and Baird [25] are generic rather than casespecific, as they do not result from an optimization process on the individual national levels. However, they are used for an approximate comparison of cost required to implement an operational REDD MRV scheme on the national level. Hardcastle and Baird [25] present respective costs for four alternatives over forest area. The cost per unit area decrease with increasing forest area, as the share of fixed costs in total costs decreases.
Variability of the target population
Sample sizes and thus survey costs are directly linked to the variability of the sample population. Variability data for a population can be obtained by prior knowledge or by a pilot survey. For each variance component that is included in the estimation procedures, variability figures have to be specified. For stratified sampling this means specifying the variance by stratum for each key attribute of interest.
Optimization
 1.
minimizing cost for a specified level of precision, or
 2.
minimizing variance for a specified cost.
In either case, the optimization requires that the cost and precision be expressed in terms of the sampling design and sample sizes.
Results and Discussion
From Figure 1 it can be seen that r^{2}values have a pronounced effect on standard errors. An increase of r^{2} from 0.3 to 0.9 reduces the percent standard error by approximately 50 percent. The functional pattern of sample size and percent standard error is similar for all design alternatives except stratified sampling; under stratified sampling the gain in precision with increasing sample size is more pronounced. Under any sampling design the relative gain in efficiency decreases with increasing sample sizes. For our example there exists a dropoff point at a sample size of around n = 200, after which the percent standard error drop would not account for the increased cost to collect additional samples.
The design alternatives show similar behavior  rising cost reduces via the increased number of field plots assessed the percent standard error. The impact of r^{2}values as seen in Figure 1 can be translated into cost: for the same cost a r^{2}value of 0.9 reduces the percent standard error by half compared to an r^{2}value of 0.3. The gain in standard error per cost unit decreases with increasing cost until it reaches a more or less steady state.
For low costs of remote sensing as opposed to per field plot cost (Figure 2A, Figure 3C, Figure 5A,Figure 6C) the design alternatives utilizing remote sensing perform better than SRS, with the exception of stratified sampling for costs below 0.5 US$/ha and regression estimators with r^{2} = 0.3 for costs below 0.4 US$/ha (Figure 4E, Figure 7E). The pattern of gain in percent standard error over cost is similar for all design alternatives except stratified sampling. Here the rate of reduction in sampling error is greater than the other alternatives, although there are higher initial costs (Figure 4EF, Figure 7EF). This makes stratified sampling the least costefficient alternative for low costs (fewer field plots) and the most costefficient for high costs (more field plots).
When the cost of remote sensing imagery is assumed to be 1 US$ per hectare, the design alternatives requiring full coverage of the auxiliary variable (regression estimation and stratification) (Figure 4F, Figure 7F) differ considerably in costefficiency from the 2phase designs (Figure 2B, Figure 3D, Figure 5B, Figure 6D), which require only partial coverage. Stratification and regression estimators with r^{2}values of 0.6 reach the percent standard errors of the other designs at a cost level of 1.6 US$/ha, while regression estimation with an r^{2}value of 0.3 fail to achieve parity with the other designs until they reach a much higher costlevel. Where remote sensing information is costly, design alternatives utilizing only field plots or partial coverage with remote sensing imagery can thus be the more costefficient alternatives.
Figures 5, 6 and 7 show results of the sample design alternative comparison when field plots costs are assumed to be 500 US$/plot. In this case, cost efficiency is necessarily consistently better than for more expensive per plot costs (Figures 2, 3 and 4). The differences between design alternatives are less pronounced with lowcost remote sensing data; here differences in costefficiency between regression estimators and 2phase sampling with regression estimators become negligible when cost are 0.3 US$/ha or higher (Figure 5A, Figure 6C, Figure 7E). Where the cost of remote sensing are higher (Figure 5B, Figure 6D, Figure 7F) fullcoverage design alternatives are competitive only for higher total per hectare cost. Stratification and regression estimates with low r^{2}values are less efficient than SRS for moderate costs.
When remote sensing costs are assumed to be 1 US$/ha, stratified sampling and regression estimators can no longer compete with the other design alternatives (Figure 4F, Figure 7F). For a remote sensing coverage of 1 percent of the study area, 2phase sampling with regression estimators is consistently more costefficient than SRS (Figure 2B, Figure 5B), while for a 10percent remote sensing coverage this holds true only for r^{2}values of 0.9 (Figure 3D, Figure 6D).
From the equations given in Table 2 it is intuitively clear that changes in population variances affect standard errors but do not change the pattern of costefficiency. To illustrate this obvious matter of fact we simulated cost efficiency under different variance assumptions. The variances presented in Tab. 4 were by 50% inflated and decreased. The effects on costefficiency can be seen in Figures 8, 9 and 10. The absolute standard errors change, but the general pattern of the costefficiency curves is maintained. Similarly the relative order of the designing alternatives with respect to costefficiency does not change.
Conclusions
In our simulation study we compared different sampling design alternatives in the scope of REDD and linked information on sampling variance with information on cost. This allowed us to characterize the effect of sampling design alternatives and sample sizes on the costefficiency of a REDD MRV system. This approach facilitates the selection of the optimal design alternative for specific populations and monitoring objectives.
The optimization process offers a set of potential starting points for improvement. Sampling intensity, field plot design and sample design (including the potential for use of a remote sensing product as auxiliary data) are the most important control variables for developing a costefficient inventory and monitoring methodology. Given the assumptions we chose to adopt, our cost analysis study revealed that incorporating expensive (i.e. airborne) remote sensing data into the sample design for a forest carbon measurement survey can unnecessarily inflate the costs compared to other alternatives.
The results indicate that it is important to include costefficiency aspects in the selection of the remote sensing alternative to be used. It needs special justification if expensive remote sensing alternatives are suggested. Either they improve costefficiency or there are additional benefits beyond the mere estimation of carbon stock changes.
The development of MRVsystems for REDD needs to be based on a sound optimization function, where either costs are minimized for a desired level of precision, or variances are minimized for a specified budget. Design optimization has to consider the marginal benefit for improving the costefficiency. Increasing the budget for an assessment results in substantial improvements of standard errors in the beginning, but the marginal benefits become negligible for high costs. The definition of the ideal turning point is such essential for the design optimization. The turning point could be selected by applying the principles of capital budgeting or by expert opinion.
Monitoring cost are especially important in the context of REDD, as an MRVsystem can be seen as an investment that aims to generate financial benefits. The amount of investment and the resulting reliability of the estimated carbon stock drives the financial gains, and thus rules the success of a REDD regime. This holds especially true in situations where deforestation is driven by the expectation of financial profits due to landuse change.
Uncertainty is a major issue in MRVsystems. Given the decreasing marginal benefit with increasing budgets indicates that increasing the sampling intensity is not the ultimate solution to improve the reliability of estimates. The application of models and functions renders necessary to transfer data assessments into estimates of carbon stock changes. The uncertainty underlying those models and functions has widely been discussed and was recognized by IPCC [11]. In relation to design optimization it could be a better choice to accept lower sampling intensities and resulting higher standard errors and invest into the improvement of models and functions. Extending the cost considerations from the costefficiency of sampling to the overall cost of a MRVsystem turns design optimization into a process that is part of the entire desire to reduce uncertainties and make estimates of carbon stock changes more reliable.
Materials and methods
Designing a monitoring system renders decisions on data sources, sample sizes, and sampling designs necessary, which in turn control inventory cost and costefficiency. To represent the interrelations between these inventory design components in a general and transferable way, we chose a simulation study approach. The simulation study was designed to repeatedly generate estimates on sampling errors with different combinations of design alternatives, samples sizes and costs. By analyzing the results of the simulation runs we hope to indentify principles that can help to guide design choices for REDD monitoring.
True population data on variance structures were taken from the Third Forest Inventory of Puerto Rico [33, 34], which covers a total land area of 886, 996 ha. The forest life zones found on the mainland of Puerto Rico are "subtropical dry forest, subtropical moist forest, subtropical wet forest, subtropical rain forest, subtropical lower montane wet forest, and subtropical lower montane rain forest" [34], while on the islands of Vieques and Culebra subtropical dry forest conditions prevail. Field data were collected by FIA. Each FIA plots consists of four circular 14.6 m diameter subplots, with one subplot located in the center and three equidistant subplots distributed symmetrically around and located 31.6 m from the center subplot. The subplots occupy 0.07 ha, and the subplot array can be subtended by a circle of 0.4 ha in area [35, 36].
Per plot aboveground biomass (AGB) figures were taken from the FIA data set. FIA estimates AGB by regression models that are either developed by the FIA program or compiled from the literature. The models predict aboveground biomass from individual tree dbh and total height measurements and provide the total ovendry biomass in kilograms of all live aboveground tree parts, including stem, stump, branches, bark, seeds, and foliage. Carbon is calculated by multiplying estimated total biomass of all trees with dbh ≥ 2.5 cm by a factor of 0.5 [34]. Per plot values were expanded to unit area (hectare).
Summary statistics for above ground biomass*
Forest Type  N  Mean  Variance  Coeff. of Variation  Std Error 

Mangrove  3  45.99  3, 949.77  136.64  36.28 
Dry forest  54  54.17  2, 451.60  91.40  6.74 
Moist forest  141  136.33  19, 248.57  101.77  11.68 
Wet and rain forest  81  200.58  27, 611.02  82.84  18.46 
Lower montane wet and rain forest  4  220.36  7, 675.41  39.76  43.80 
Mixed  5  68.55  2, 021.61  65.59  20.11 
All (288 forest plots; 678 nonforest plots)  956  41.59  10, 137.60  242.11  3.26 

Simple random sampling (SRS); this alternative would represent a solely fieldbased assessment

Regression estimators; under this alternative auxiliary data (e.g. LiDAR or RADAR backscatter) are assessed on a walltowall coverage of remote sensing imagery and linked via regression estimates to the variable of interest (e.g., AGB) that is assessed on a (small) subsample of field plots.

Stratified sampling; a walltowall coverage of remote sensing imagery is utilized to separate the entire population into homogeneous strata. In each stratum field plots are assessed. The classification of multispectral, optical remote sensing data would be a common procedure to obtain the stratification of the inventory area.

2phase sampling with regression estimators; the alternative is similar to regression estimators, but requires only a partial coverage of the inventory area by remote sensing imagery. Where airborne remote sensing systems such as LiDAR render data assessment on flight lines rather than full coverage necessary, this sampling alternative is the preferred method. For the simulations study we used a phase1 coverage of 1 percent and 10 percent of the entire inventory area.
Simple random sample will serve as the baseline for comparing alternative sampling designs. The performance of both, regression estimators and 2phase sampling with regression estimators depends on the correlation between the auxiliary variable and the variable of interest. Drake et al. [37] used metrics from largefootprint LiDAR and modeled plotlevel biomass with r^{2} = 0.93 for a 1, 536 ha area in Costa Rica stocked by primary and secondary wet tropical forest, abandoned pasture and plantations, and agroforestry. Even higher r^{2}values could be found in boreal and temperate monospecies forests. For example, Means et al. [38] found on 26 plots (approx. 6.5 ha) primarily of Douglasfir and western hemlock r^{2} values of 0.96 for the estimation of AGB. Considerably lower r^{2}values were found for volume (0.66) and biomass (0.59) by van Aardt et al. [39], who used smallfootprint LiDAR to study a LiDARbased, objectoriented approach to forest volume and aboveground biomass modeling in temperate forests. We included r^{2}values of 0.9, 0.6, and 0.3 in our simulation study to extent the informative value to operational applications and to show the effect of the underlying correlation between auxiliary and field data on the costefficiency of the design.
Stratification by Jenk's Natural Breaks Classification Method
Strata by Jenk's  N  Mean  Variance  Coeff of Variation  Std Error 

0  676  0.00  0.00  .  0.00 
1  80  25.36  225.26  59.18  1.68 
2  81  85.74  390.56  23.05  2.20 
3  51  153.43  539.90  15.14  3.25 
4  40  248.44  923.50  12.23  4.80 
5  28  465.04  31, 521.72  38.18  33.55 
For each design alternative the initial number of field plots was set to n = 20, except for stratified sampling, where a minimum of 40 field plots was predefined to obtain a sufficient withinstrata sample size. A maximum sample of n = 6, 000 was sufficient to show the effect of increasing sample size on the percent standard error. We sampled n = 20 to 6, 000 in increments of 5.
Cost figures used in the simulation study
Cost component  Cost  

Min.  Max.  
Field assessment  500 US$/plot  5, 000 US$/plot 
Remote sensing imagery, Alternative 1  0.1 US$/ha  1 US$/ha 
Remote sensing imagery, Alternative 2  0.01 US$/ha   
The following settings of the simulation study were realized:
Field sample size: n = {20, ... 6, 000}
Remote sensing coverage: 1%, 10%, full
Cost per field plot: 500 US$, 5, 000 US$
Cost remote sensing imagery: 0.1 US$/ha, 1 US$/ha
r^{2}value, AGB = f(remote sensing signal): 0.3, 0.6, 0.9
Sampling designs: simple random sampling, stratified sampling, regression sampling, 2phase sampling with regression estimators
Based on the coefficients of variation for the sampling population (Table 3) and the stratification rules presented in Table 4 we calculated the costefficiency in terms of total cost and percent standard error or each combination of settings. The simulation was run under SAS^{©}[42].
Declarations
Acknowledgements
We thank John Stanovik, USDA Forest Service, Newtown Square, USA, for carefully reviewing a first draft of the manuscript and for helpful comments. We are grateful to three anonymous reviewers, which helped to improve the quality of the text.
Authors’ Affiliations
References
 IPCC: Climate Change 2007: Impacts, Adaptation and Vulnerability: Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change: IPCC National Greenhouse Gas Inventories Programme. Cambridge, UK: Cambridge University Press; 2007.Google Scholar
 van der Werf GR, Morton DC, DeFries RS, Olivier JGJ, Kasibhatla PS, Jackson RB, Collatz GJ, Randerson JT: CO2 emissions from forest loss. Nature Geoscience 2009, 2: 737–738. 10.1038/ngeo671View ArticleGoogle Scholar
 FAO: Global forest resources assessment 2010: Main Report. Rome: Food and Agriculture Organization of the United Nations; 2010.Google Scholar
 Griscom B, Shoch D, Stanley B, Cortez R, Virgilio N: Sensitivity of amounts and distribution of tropical forest carbon credits depending on baseline rules. Environmental Science & Policy 2009, 12: 897–911. 10.1016/j.envsci.2009.07.008View ArticleGoogle Scholar
 Baker DJ, Richards G, Grainger A, Gonzalez P, Brown S, DeFries R, Held A, Kellndorfer J, Ndunda P, Ojima D, et al.: Achieving forest carbon information with higher certainty: A fivepart plan. Environmental Science & Policy 2010, 13: 249–260. 10.1016/j.envsci.2010.03.004View ArticleGoogle Scholar
 Böttcher H, Eisbrenner K, Fritz S, Kindermann G, Kraxner F, McCallum I, Obersteiner M: An assessment of monitoring requirements and costs of 'Reduced Emissions from Deforestation and Degradation'. Carbon Balance and Management 2009, 4: 7. 10.1186/1750068047View ArticleGoogle Scholar
 Miles L, Kapos V: Reducing Greenhouse Gas Emissions from Deforestation and Forest Degradation: Global LandUse Implications. Science 2008, 320: 1454–1455. 10.1126/science.1155358View ArticleGoogle Scholar
 Maniatis D, Mollicone D: Options for sampling and stratification for national forest inventories to implement REDD+ under the UNFCCC. Carbon Balance and Management 2010, 5: 9. 10.1186/1750068059View ArticleGoogle Scholar
 Köhl M, Baldauf T, Plugge D, Krug J: Reduced emissions from deforestation and forest degradation (REDD): a climate change mitigation strategy on a critical track. Carbon Balance and Management 2009, 4: 10. 10.1186/17500680410View ArticleGoogle Scholar
 IPCC: Good practice guidance for land use, landuse change and forestry: IPCC National Greenhouse Gas Inventories Programme. Hayama, Japan: Institute for Global Environmental Strategies (IGES); 2003.Google Scholar
 IPCC: Guidelines for National Greenhouse Gas Inventories: IPCC National Greenhouse Gas Inventories Programme. Japan: Institute for Global Environmental Strategies (IGES); 2006.Google Scholar
 Stott CB, Ryan EJ: A Permanent Sample Technique Adapted to Commercial Timber Stands. J For 1939, 347–349.Google Scholar
 Ware KD, Cunia T: Continuous Forest Inventory, Partial Replacement of Samples and Multiple Regression. Forest Science 1965, 11: 480–502.Google Scholar
 Scott CT: A New Look at Sampling with Partial Replacement. Forest Science 1984, 30: 157–166.Google Scholar
 Köhl M, Magnussen S, Marchetti M: Sampling Methods, Remote Sensing and GIS Multiresource Forest Inventory. Berlin, Heidelberg: Springer; 2006.View ArticleGoogle Scholar
 Cochran WG: Sampling techniques. 3rd edition. New York: Wiley; 1977.Google Scholar
 Gregoire TG, Valentine HT: Sampling strategies for natural resources and the environment. Boca Raton, Fla: Chapman & Hall CRC; 2008.Google Scholar
 Walshe T, Wintle B, Fidler F, Burgman M: Use of confidence intervals to demonstrate performance against forest management standards. Forest Ecology and Management 2007, 247: 237–245. 10.1016/j.foreco.2007.04.048View ArticleGoogle Scholar
 Dawkins HC: Some results of stratified random sampling of tropical highforest. Seventh British Commonwealth Forestry Conference Item 7 (iii), Oxford; 1957.Google Scholar
 UNFCCC: Good practice guidance and adjustments under Article 5, paragraph 2, of the Kyoto Protocol: FCCC/KP/CMP/2005/8/Add.3 Decision 20/CMP.1. 2006.Google Scholar
 UNFCCC: Modalities and procedures for afforestation and reforestation project activities under the clean development mechanism in the first commitment period of the Kyoto Protocol: Decision 5/CMP.1. 2006.Google Scholar
 Grassi G, Monni S, Federici S, Achard F, Mollicone D: Applying the conservativeness principle to REDD to deal with the uncertainties of the estimates. Environmental Research Letters 2008, 3: 035005. 10.1088/17489326/3/3/035005View ArticleGoogle Scholar
 Cerbu GA, Swallow BM, Thompson DY: Locating REDD: A global survey and analysis of REDD readiness and demonstration activities: Governing and Implementing REDD+. Environmental Science & Policy 2011, 14: 168–180. 10.1016/j.envsci.2010.09.007View ArticleGoogle Scholar
 Scott CT, Köhl M: A method for comparing sampling design alternatives for extensive inventories. Birmensdorf: Eidg. Forschungsanstalt für Wald, Schnee und Landschaft; 1993.Google Scholar
 Hardcastle PD, Baird D: Capability and cost assessment of the major forest nations to measure and monitor their forest carbon: Report prepared for the Office of Climate Change. Penicuick, UK;Google Scholar
 Reams GA, Smith WD, Hansen MH, Bechtold WA, Roesch FA, Moisen GG: The Forest Inventory and Analyis Sampling Frame. In The enhanced forest inventory and analysis programnational sampling design and estimation procedures. Volume 80. Edited by: Bechtold WA, Patterson PL. USDA Forest Service, Southern Research Station; 2005:11–26.Google Scholar
 Ranneby B: Den topografiska variationen inom olika områden: En redovisning av skattade korrelationsfunktioner. Sveriges Lantbruksuniversitet, Institutionen för skogstaxering 1981, 54.Google Scholar
 Matern B, Ranneby B: Variational structure in forests: Implications for sampling. In Forest inventory for improved management. IUFRO; 1983.Google Scholar
 Schreuder HT, Li J, Scott CT: Estimation with Different Stratification at Two Occasions. Forest Science 1993, 39: 368–382.Google Scholar
 Scott CT, Köhl M: Sampling with Partial Replacement and Stratification. Forest Science 1994, 40: 30–46.Google Scholar
 Green E, Köhl M, Strawderman WE: Improved Estimates for Cell Values in a TwoWayTable. Biometrie und Informatik in Medizin und Biologie 1991, 23: 24–30.Google Scholar
 Groves RM: Survey errors and survey costs. Edited by: Hoboken NJ. WileyInterscience; 2004.Google Scholar
 USDA Forest Service: Forest inventory and analysis national core field guide, volume 1: field data collection procedures for phase 2 plots: version 5.0. U.S. Department of Agriculture FSWO ed 2010.Google Scholar
 Brandeis TJ, Helmer EH, Oswalt SN: The status of Puerto Rico's forests, 2003. US Department of Agriculture Forest Service, Southern Research Station, Resource Bulletin SRS119 2007.Google Scholar
 Gregoire TG, Scott CT: Sampling at the Stand Boundary: A Comparison of the Statistical Performance of Eight Methods. In Proceedings of the XIX IUFRO World Forestry Congress, Publication FWS3–90. IUFRO. Blacksburg, VA: Virginia Polytechnic Institute and University; 1990:78–85.Google Scholar
 McRoberts RE, Bechtold WA, Patterson PL, Scott CT, Reams GA: The Enhanced Forest Inventory and Analysis Program of the USDA Forest Service: Historical Perspective and Announcement of Statistical Documentation. Journal of Forestry 2005, 103: 304–308.Google Scholar
 Drake JB, Dubayah RO, Clark DB, Knox RG, Blair JB, Hofton MA, Chazdon RL, Weishampel JF, Prince S: Estimation of tropical forest structural characteristics using largefootprint lidar. Remote Sensing of Environment 2002, 79: 305–319. 10.1016/S00344257(01)002814View ArticleGoogle Scholar
 Means JE, Acker SA, Harding DJ, Blair JB, Lefsky MA, Cohen WB, Harmon ME, Mckee WA: Use of LargeFootprint Scanning Airborne Lidar To Estimate Forest Stand Characteristics in the Western Cascades of Oregon  biomass distribution and production budgets. Remote Sensing of Environment 1999, 67: 298–308. 10.1016/S00344257(98)000911View ArticleGoogle Scholar
 van Aardt JAN, Wynne RH, Oderwald RG: Forest Volume and Biomass Estimation Using SmallFootprint LidarDistributional Parameters on a PerSegment Basis. Forest Science 2006, 52: 636–649.Google Scholar
 Jenks GF: The data model concept in statistical mapping. In International Yearbook of Cartography. Volume 7. Edited by: Frenzel K. Rand McNally & Co; 1967.Google Scholar
 Asner GP, Hughes RF, Mascaro J, Uowolo AL, Knapp DE, Jacobson J, KennedyBowdoin T, Clark JK: Highresolution carbon mapping on the millionhectare Island of Hawaii: Frontiers in Ecology and the Environment. Frontiers in Ecology and the Environment 2011.Google Scholar
 SAS Institute Inc: SAS/STAT® 9.2 User's Guide. Cary, NC; 2008.Google Scholar
Copyright
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.