Avoiding treatment bias of REDD+ monitoring by sampling with partial replacement
 Michael Köhl^{1}Email author,
 Charles T Scott^{2},
 Andrew J Lister^{2},
 Inez Demon^{3} and
 Daniel Plugge^{1}
https://doi.org/10.1186/s130210150020y
© Köhl et al.; licensee Springer. 2015
Received: 19 December 2014
Accepted: 8 April 2015
Published: 8 May 2015
Abstract
Background
Implementing REDD+ renders the development of a measurement, reporting and verification (MRV) system necessary to monitor carbon stock changes. MRV systems generally apply a combination of remote sensing techniques and insitu field assessments. Insitu assessments can be based on 1) permanent plots, which are assessed on all successive occasions, 2) temporary plots, which are assessed only once, and 3) a combination of both. The current study focuses on insitu assessments and addresses the effect of treatment bias, which is introduced by managing permanent sampling plots differently than the surrounding forests. Temporary plots are not subject to treatment bias, but are associated with large sampling errors and low costefficiency. Sampling with partial replacement (SPR) utilizes both permanent and temporary plots.
Results
We apply a scenario analysis with different intensities of deforestation and forest degradation to show that SPR combines costefficiency with the handling of treatment bias. Without treatment bias permanent plots generally provide lower sampling errors for change estimates than SPR and temporary plots, but do not provide reliable estimates, if treatment bias occurs, SPR allows for change estimates that are comparable to those provided by permanent plots, offers the flexibility to adjust sample sizes in the course of time, and allows to compare data on permanent versus temporary plots for detecting treatment bias. Equivalence of biomass or carbon stock estimates between permanent and temporary plots serves as an indication for the absence of treatment bias while differences suggest that there is evidence for treatment bias.
Conclusions
SPR is a flexible tool for estimating emission factors from successive measurements. It does not entirely depend on sample plots that are installed at the first occasion but allows for the adjustment of sample sizes and placement of new plots at any occasion. This ensures that insitu samples provide representative estimates over time. SPR offers the possibility to increase sampling intensity in areas with high degradation intensities or to establish new plots in areas where permanent plots are lost due to deforestation. SPR is also an ideal approach to mitigate concerns about treatment bias.
Keywords
Measurement Reporting and verification (MRV) Forest carbon stock and carbon stock change estimation Representativeness over timeBackground
In November 2013, the nineteenth session of the Conference of the Parties (COP 19) agreed on the “Warsaw Framework for REDD Plus”, which consists of seven decisions relating to the implementation of REDD+ [1]. Together with the UNFCCC Cancun Agreements [2] and Durban outcomes [3], these decisions are a major step forward for the implementation of REDD+ at the national level. According to decision 11/CP.19 [1], the modalities for national forest monitoring systems should in the full implementation phase provide data and information that are transparent, consistent over time, suitable for measurement, reporting and verification (MRV) and build upon existing systems while being flexible and allowing for improvement [1]. In accordance with national circumstances and respective capabilities, robust and transparent national forest monitoring systems are to be developed (Decision 4/CP.15) [4].
MRV systems as an integral part of REDD+ implementation mainly focus on the assessment of carbon stock change. Decisions to adopt specific operational systems at national and local levels are subject to the country’s unique circumstances, such as differences in forest types, drivers of deforestation and forest degradation, or livelihood impacts. Therefore, finding an operational approach that adheres to the international (IPCC) requirements is a matter of much debate.
MRV systems generally apply a combination of remote sensing techniques and insitu field assessments to provide information on activity data and emission factors. The current study focuses on insitu assessments and addresses the effect of treatment bias, which is introduced by managing forests on permanent sampling plots differently than the surrounding forests.
A major obstacle to the use of permanent plots is the likelihood that they will become nonrepresentative due to treatment bias [5]. Land management on permanent plots with known locations might differ from that of the surrounding forests. When payments are linked to the results of field assessments, the potential for treatment bias can be high. For example, activities like tree cutting that would normally be classified as forest degradation could be deliberately excluded from areas on or around permanent plots in order to maintain biomass and secure payments. For an operational and sound MRV system, it is critical that the shortcomings of inventory designs are not exploited for the generation of (unjustified) financial benefits. Therefore, MRV systems have to be immune to treatment bias and thus produce objective stock change estimates.
It has been shown that permanent sample plots guarantee the highest precision for change estimates [68]. Continuous Forest Inventory (CFI), or the use of permanent plots in forestry, was introduced in the middle of the last century [911]. In this design, the number of sampling units and their allocation is determined at inventory establishment and retained over time. Therefore, CFI designs show a limited ability to adapt to changing population conditions. This holds especially true in the scope of REDD+. Due to deforestation, forested permanent sample plots might become deforested over time and thus sampling intensity might become too low to provide forest change estimates with a desired reliability. Although an increase of sampling intensity is called for in locations with high degradation activities, the use of permanent plots selected with fixed selection probabilities precludes this type of adaptation. This inflexibility, together with the potential for treatment bias, is a considerable disadvantage compared to alternatives.
Temporary plots are an alternative to permanent plots. At each occasion, plots are allocated independently from assessments at previous occasions, allowing for a flexible adjustment of sampling intensities over time. As an individual plot is assessed only once, treatment bias is not an issue. However, one major disadvantage of temporary plots is the high sampling error associated with the estimation of change. Avoidance of observer bias by utilizing temporary plots only is therefore achieved at the expense of a loss in precision and reliability, making designs based on temporary plots less costefficient than those with an equal number of permanent plots [12].
The desire to exploit the advantages and avoid the disadvantages of temporary and permanent plots motivated the development of a sampling approach that combines both approaches. In this design, a subset of the plots allocated at inventory establishment are remeasured (permanent plots), and the remaining subset is replaced by new, temporary plots. This proceeds in repeated inventories in an alternating fashion over successive inventory cycles. The procedure is known as Sampling with Partial Replacement (SPR) and was introduced into forestry by Ware and Cunia [13]. Scott [14] presented a samplebased estimator that combines the variance from the permanent (matched) and temporary (unmatched) plots for change estimation. SPR has been recommended as a flexible tool to meet precision requirements of current forest status and trend estimates in a costefficient way [1315].
This paper will present the statistical background of sampling on successive occasions in the scope of MRV. A descriptive example is used to illustrate the procedure and identify the pros and cons of the 3 alternative sampling approaches. It is demonstrated how SPR can be used in MRV systems in order to mitigate problems such as treatment bias and loss of optimality of sample intensity over time with designs using permanent plots, and provide a costeffective improvement over designs using only temporary plots.
Results and discussion

Continuous forest inventory (CFI) design, utilizing permanent (remeasured) plots only

Temporary plot design, where an independent sample is drawn at each occasion

Sampling with Partial Replacement (SPR) design, utilizing a mixture of permanent and temporary plots
is illustrated by a set of permanent plots located in the Suriname’s forest belt for change estimation under different deforestation and degradation scenarios. The permanent plots were established in the late 1970’s, when several silvicultural treatments with different logging intensities were applied. Since then, no forest management has taken place. For our study we utilized the observations from the most recent measurements, which were made in 2000 and in 2013. On 750 plots with a size of 400 m^{2} different degradation and deforestation intensities were simulated by selectively removing certain trees based on diameter thresholds.
To assess the effects of the alternative sampling designs, 6 scenarios that reflect realistic deforestation and degradation activities (including no intervention) were created. For a subset of the plots (5% to 20%), different levels of treatment bias were simulated on the plot level. Treatment bias was simulated by deliberately excluding the permanent plots from being affected by forest loss or degradation that occurred elsewhere in the study area, leading to a nonrepresentative sample. For scenarios with no treatment bias applied, simulated deforestation and degradation occurred on permanent plots with the same frequency at which it occurred elsewhere, leading to a representative sample.
The absolute value of biomass loss differs by scenario (Figure 2). Degradation and deforestation lead to a decreasing total biomass, but only under heavy degradation activities where on 20% of the forest area all trees with dbh >45 cm were removed the growth of the remaining forest could not compensate for the biomass loss by degradation activities. Treatment bias consistently leads to an overestimation of biomass changes and does not capture the true development. Under a REDD+ regime this would result in unjustified benefits. A rogue stakeholder could use this effect to manipulate carbon budgets.
The absolute (Figure 3) and percent (Figure 4) difference between estimated biomass changes and true population values illustrate the effects of treatment bias. Treatment bias results in substantial overestimation of biomass gains; the overestimation is higher for CFI than for SPR, as SPR utilizes some new (temporary) plots at the second occasion, which replace some of those plots from treatment bias. Where only temporary plots are utilized assessments are not subject to treatment bias as the location of the location of the set of plots installed at the second occasion is not known in advance.
Figure 5 presents the percent standard errors for the estimation of biomass change under the different scenarios and sampling design alternatives. Beside population variability the standard errors in change estimates are generally affected by two variance components: (1) the variance introduced by biomass growth, which is influenced by stand structure and site quality, and (2) the variance due to disturbances, i.e., degradation and deforestation. The second variance component is not present when plots are subject to treatment bias. Thus the percent standard errors of CFI and SPR under treatment bias show comparable magnitudes, as they refer to similar, nondisturbed biomass development.
Where no treatment bias is present, the variability introduced by disturbances inflates the resulting standard errors. CFI results in the smallest standard errors, as the estimator utilizes the covariance between observations at successive occasions (Eq. 3). Temporary plots show the largest sampling errors; the standard errors obtained by SPR, which apply an update of unremeasured plots based on the regression relationship with remeasured plots (Eq. 4), are at an intermediate level.
Treatment bias considerably affects change estimates and their standard errors. Even substantial biomass losses outside the remeasured plots remain undetected. This is reinforced by the fact that standard errors of treatmentbiased change estimates ignore the presence of degradation and deforestation activities and thus remain small due to the pronounced correlation between plot values on successive occasions.
Conclusions
MRV systems for REDD+ aim at the provision of consistent and reliable estimates of biomass and carbon stock changes at successive occasions. As the nature of these estimates affects a country’s financial benefits associated with participation in REDD, it is important that inventory methods are subject to careful validation.
It is a widespread practice in forestry to utilize permanent plots for change estimates. However, permanent plots are subject to treatment bias as they may be excluded from degradation or deforestation activities once their location is known. This opens the potential for nonrepresentative samples and associated estimates due to either honest mistakes or fraudulent activities. Biased estimates linked with small sampling errors have an unknown level of risk where only permanent plots are used. Since one of the main goals of an MRV is to accurately characterize the carbon dynamics of an area of interest, using solely permanent plots can thus call into question the scientific validity of an MRV system if treatment bias is not controlled.
Given the absence of a selection bias the problem of nonrepresentativeness is not a concern when temporary plots are used. However, an assessment built on only temporary plots will result in substantial sampling errors and low costefficiency. SPR, on the other hand, offers a solution to both low costefficiency and potential treatment bias, as it combines temporary and permanent plots. SPR can be used to guard against treatment bias on permanent plots and improves the reliability of change estimates.
Another concern about MRV systems based solely on permanent plots is the determination of sample size and plot location at inventory establishment. This design lacks flexibility because the set of plots installed at the first occasion has to be remeasured at successive occasions, regardless of changes that occur on the landscape. This holds especially true in situations where forest plots are lost due to deforestation activities or where degradation activities are shifting.
Under REDD+ monitoring the application of SPR offers compelling advantages over designs based solely on permanent plots. SPR designs are flexible as new plots can be established at any occasion and old plots can be replaced by new plots whenever necessary. The SPR estimation procedures help detect and guard against treatment bias and can generate costeffective estimates of forest carbon dynamics as well as help provide verification for the scientific validity of change estimates.
Where degradation is concentrated in specific regions, SPR can be combined with stratification for further reductions in sampling error [15]. Stratification rules can be designed that incorporate the magnitude of degradation intensities and utilize auxiliary information e.g. from remote sensing [16,17]. In each stratum an independent SPR design can be applied and the number of remeasured and temporary plots can be optimized [18,19].
Methods
State of the art
\( \widehat{X} \) = first occasion (time 1) estimate of the mean, \( \widehat{X}=\frac{{\displaystyle {\sum}_{i=1}^{n_1}}{X}_i}{n_1} \)
Ŷ = second occasion (time 2) estimate of the mean, \( \widehat{Y}=\frac{{\displaystyle {\sum}_{i=1}^{n_2}}{Y}_i}{n_2} \)
X _{ i } = measurement on plot i at time 1, i = 1,… n _{ 1 }
Y _{ i } = measurement on plot i at time 2, i = 1,… n _{ 2 }
n _{ 1 } = number of plots at time 1
n _{2} = number of plots at time 2
The CFI method, despite its obvious advantage, encounters practical and inferential problems. Over time the locations of sample plots may become known beyond the surveyors and, as a result, they may be deliberately treated differently than the surrounding forest. This nontrivial risk is especially acute for visibly marked sample plots. The latent potential of an inferential problem therefore exists because, as paraphrased by [20], “there is no guarantee that sample plots, visible or not, will remain representative of the target population”. Inventories that potentially don’t represent reality will lose credibility. This holds especially true for inventories in the scope of REDD+; nonrepresentative treatment applied on permanent plots could corrupt the reliability of emission estimates.
Those problems can be controlled and mitigated by Sampling with Partial Replacement (SPR), which utilizes a mixture of both permanent and temporary plots. New, temporary plots established at the second and every following occasion can be utilized to assess the potential for treatment bias on the remeasured subset. In addition, plots that are lost due to landuse change can be “replaced” by new plots by increasing the sampling intensity on the new set of permanent plots so that the number of forested plots does not diminish over time.
SPR was introduced into forest inventory around 1960 [13,21]. Scott [14] presented a consistent set of estimators for SPR, which will be presented in the following.

Sample plots that are measured on the first occasion as well as on the second occasion (permanent, matched sample plots referred to as the n _{ 12 } sample).

Sample plots that are only measured on the first occasion (unmatched plots referred to as the n _{ 1− } sample).

Sample plots that are only measured on the second occasion (new, unmatched plots referred to as the n _{ −2 } sample).
 (1)
The current state is obtained by 2 means. One mean is based on the measurements of permanent (remeasured) plots and the updated values of the temporary (not remeasured) plots (Eq.4). The first mean, \( {\widehat{\overline{Y}}}_{12} \), is formed by updating the time 1 mean using the simple linear regression between time 1 and 2 on the remeasured plots. This regression, in effect, updates the values of the sample plots that are not remeasured (Y _{ 1−}). A second mean is derived from the new (temporary) sample plots (Eq.5).
X _{ 12j } = measurement of permanent plot j at time 1, j = 1,…, n _{ 12 }
 (2)
For both means, the variance is calculated.
 (3)
Through weighting both means with their inverse variance, a combined estimator is derived. If the regression estimator has a larger variance, it therefore receives a lower weight and vice versa. These weights minimize the variance of the combined estimator.
 (4)
As the last step the variance of the combined estimator is calculated.
While SPR is straightforward for 2 applications, estimation procedures become cumbersome for 3 or more occasions. For example, where SPR is applied for 3 occasions, 7 different plot types (n _{ 123 } , n _{ 12− } , n _{ 1–3 } , n _{ −23 } , n _{ 3 } , n _{ 1 } , n _{ 2}) need to be considered [15].
Methods and data
The comparison of the different sampling approaches renders the availability of information on population variances necessary. A longterm forest growth and yield experiment from Suriname was selected to provide the necessary information. The experiment, for which trees were initially measured in 1978 and remeasured in 2000 and 2013, was on a former concession site on which no forest management practices were applied since the establishment of the experiment. The basic treatments applied at that time were silvicultural treatments implemented at different intensity levels, and included release cuts – harvests similar to thinnings in temperate and boreal forests – to stimulate growth of the remaining stand. The treatments were allocated in 3 blocks, each containing 9 experimental plots. Each of the 9 experimental plots within a block was 1 hectare in size and surrounded by a buffer strip. In addition to the blocks, 3 plots were established in undisturbed natural forests, resulting in a total of thirty 1ha plots.
Tree level statsitics
Time  Number of trees  Aboveground biomass ( AGB )  

Min  Max  Mean  Median  Standard deviation  
[kg]  [kg]  [kg]  [kg]  
2000  8650  70.1  9777.8  417.4  185.7  664.1 
2013  8191  70.1  9581.5  485.3  223.9  717.1 
DBH  
Min  Max  Mean  Median  Standard Deviation  
[cm]  [cm]  [cm]  [cm]  
2000  8650  15.0  149,1  29.3  23.6  16.4 
2013  8191  15.0  147.7  31.6  25.7  17.4 
Plot statistics
Time  Number of plots  Aboveground biomass ( AGB )  

Min  Max  Mean  Median  Standard Deviation  Correlation  
[t/ha]  [t/ha]  [t/ha]  [t/ha]  
2000  750  10.8  401.6  120.3  108.5  60.9  0.804 
2013  750  17.5  542.7  132.5  118.6  65.5 
Sample sizes
Plot type  Sampling design alternative  

SPR  CFI  Temporary plots  
Temporary, time 1 (n _{ 1− })  125  375  
Permanent (n _{ 12 })  250  375  
Temporary, time 2 (n _{ −2 })  125  375 
Deforestation and degradation scenarios
Scenario  Description  Anticipated degradation/deforestation pattern 

No intervention  Original plot data from both treated and untreated stands are used without modification  No degradation and deforestation activities 
10% degradation, dbh < 35 cm  On 10 percent of the plots (n = 75) the biomass of trees with dbh < 35 cm was set to zero at occasion 2  Degradation by harvesting trees with small dbh for fuelwood 
10% degradation, dbh > 45 cm  On 10 percent of the plots (n = 75) the biomass of trees with dbh > 45 cm was set to zero at occasion 2  Degradation by selectively harvesting trees with large dbh for timber procurement 
20% degradation, dbh < 35 cm  On 20 percent of the plots (n = 150) the biomass of trees with dbh < 35 cm was set to zero at occasion 2  Degradation by harvesting trees with small dbh for fuelwood 
20% degradation, dbh > 45 cm  On 20 percent of the plots (n = 150) the biomass of trees with dbh > 45 cm was set to zero at occasion 2  Degradation by harvesting trees with large dbh for timber procurement 
5% deforestation  On 5 percent of the plots (n = 37) the biomass of all trees is set to zero at the second occasion  Deforestation and landuse change 
Numerical results of the combinations of scenarios and sample design alternatives were obtained by a Monte Carlo experiment. The experiment was realized with 1000 iterations for each combination. The 750 sample plots served as input for the simulations. In each iteration plots were randomly selected (simple random sampling without replacement) for applying the treatments of the 6 scenarios. In addition the original measurement values were maintained in order to provide input for the realizations with treatment bias. From the modified set of plots (n = 750) samples were selected by simple random sampling without replacement according to the sample sizes and plot types (permanent, temporary) given in 0. For each iteration population (true) values as well as sample estimates (current values and change between time 1 and time 2, and corresponding variances) were calculated.
The realizations of percentages of disturbance (deforestation, degradation) given in column 1 of Table 4 refer to the entire population of 750 plots, not to the selected samples. The original, undisturbed measurements at time 2 were utilized to simulate treatment bias. Hence, no degradation and deforestation activities take place on any of the plots assigned to the alternatives with treatment bias. Thereby, the endpoints of the conceivable range of treatment bias effects on permanent plots are depicted. Under realistic conditions, treatment bias will occur between these endpoints.
The Monte Carlo experiment was conducted in SAS™.
Declarations
Acknowledgements
We are grateful to Kirstin Höwler, Christian Hack, Jan Wirjosentono, Merdy Sewotaroeno, Hubert Jubithana, Paul Prika, and Aniel Soekhlal for the careful assessment of the field data under often not easy conditions. Kirstin and Christian entered the data into a database. John Alimoenardjo provided both the memorable ambience and the outstanding food in the camp. We thank Dr. Volker Mues and Dr. Phillip Mundhenk, University of Hamburg, and two anonymous reviewers for carefully reviewing a first draft of the manuscript and for helpful comments.
Authors’ Affiliations
References
 UNFCCC. Report of the Conference of the Parties on its nineteenth session, held in Warsaw from 11 to 23 November 2013. 2014, FCCC/CP/2013/10/Add.1:43. http://unfccc.int/resource/docs/2013/cop19/eng/10a01.pdf.
 UNFCCC. Fact Sheet. Reducing Emissions from deforestation in developing countries: approaches to stimulate action 2011, https://unfccc.int/files/press/backgrounders/application/pdf/fact_sheet_reducing_emissions_from_deforestation.pdf.
 UNFCCC. Report of the Conference of the Parties on its seventeenth session, held in Durban from 28 November to 11 December 2011. 2012, FCCC/CP/2011/9/Add.1. http://unfccc.int/resource/docs/2011/cop17/eng/09a01.pdf.
 UNFCCC: Report of the Conference of the Parties on its fifteenth session, held in Copenhagen from 7 to 19 December 2009. 2010, FCCC/CP/2009/11/Add.1. City; 2010, http://unfccc.int/resource/docs/2009/cop15/eng/11a01.pdf.
 Schmid P. Die Weiterentwicklung der Leistungskontrolle in der Schweiz. Wiss Zeitschrift d techn Univ Dresden. 1969;16:545–9.Google Scholar
 Cochran WG. Sampling techniques. 3rd ed. New York: Wiley; 1977.Google Scholar
 Schreuder H, Gregoire TG, Wood GB. Sampling methods for multiresource forest inventory. New York: Wiley; 1992.Google Scholar
 Köhl M, Magnussen S, Marchetti M. Sampling methods, remote sensing and GIS multiresource forest inventory. Berlin, Heidelberg: Springer; 2006.View ArticleGoogle Scholar
 Stott CB. Permanent growth and monitoring plots in half the time. J For. 1947;37:669–73.Google Scholar
 Stott CB, Ryan EJ. A permanent sample technique adapted to commercial timber stands. J For 1939;37(4)347–349.Google Scholar
 Stott CB, Semmes G: Our Changing Inventory Methods and the CFI System in North America. In Proceedings 5th World Forest Congress. Seatle: University of Washington; 1962.Google Scholar
 Scott CT, Köhl M, Schnellbächer HJ. A comparison of permanent versus periodic surveys. For Sci. 1999;45:433–51.Google Scholar
 Ware KD, Cunia T. Continuous forest inventory, partial replacement of samples and multiple regression. For Sci. 1965;11:480–502.Google Scholar
 Scott CT. A new look at sampling with partial replacement. For Sci. 1984;30:157–66.Google Scholar
 Scott CT, Köhl M. Sampling with partial replacement and stratification. For Sci. 1994;40:30–46.Google Scholar
 Hewson J, Steininger M, Pesmajoglou S. REDD+ measurement, reporting and verification (MRV) manual. In: Book REDD+ measurement, reporting and verification (MRV) manual (Editor ed.^eds.). City: USAID; 2013. p. 160.Google Scholar
 GOFCGOLD. A sourcebook of methods and procedures for monitoring and reporting anthropogenic greenhouse gas emissions and removals associated with deforestation, gains and losses of carbon stocks in forests remaining forests, and forestation. In: Book A sourcebook of methods and procedures for monitoring and reporting anthropogenic greenhouse gas emissions and removals associated with deforestation, gains and losses of carbon stocks in forests remaining forests, and forestation (Editor ed.^eds.). City: GOFCGOLD Land Cover Project Office; 2014.Google Scholar
 Scott CT, Köhl M. A method for comparing sampling design alternatives for extensive inventories. Birmensdorf: Eidg. Forschungsanstalt für Wald, Schnee und Landschaft; 1993.Google Scholar
 Köhl M, Lister A, Charles TS, Baldauf T, Plugge D. Implications of sampling design and sample size for national carbon accounting systems. Carbon Balance and Management 2011;6:10. doi:10.1186/17500680610.Google Scholar
 SchmidHaas P. Swiss continuous forest inventory twenty years experience. In: Atterury T, Bell JF (Eds.). Renewable Resources Monitoring Changes and Trends. College of Forestry, Corvallis, OR: University of Corvallis; 1983.Google Scholar
 Bickford CA, Mayer CF, Ware KD. An efficient sampling design for forest inventory: the northeastern forest survey. J For. 1963;61:826–33.Google Scholar
 Chave J, Andalo C, Brown S, Cairns MA, Chambers JQ, Eamus D, et al. Tree allometry and improved estimation of carbon stocks and balance in tropical forests. Oecologia. 2005;145:87–99.View ArticleGoogle Scholar
Copyright
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.