Abstract
Download PDF
Full Article
Environmental LCA and Financial Analysis to Evaluate the Feasibility of Bio-based Sugar Feedstock Biomass Supply Globally: Part 2. Application of Multi-Criteria Decision-Making Analysis as a Method for Biomass Feedstock Comparisons
Carter W. Reeb, Richard Venditti,* Ronalds Gonzalez, and Stephen Kelley
Early-stage evaluation of biofuel and bioproduct technologies is extremely complicated and involves many disparate feasibility criteria, including technical, financial, environmental, logistic, legal, social, and other aspects. Problems can arise for decision-makers when evaluating renewable technologies at this early stage due to bias, shifting preferences or priorities, occurrence of trade-offs, and decision-making complexity. Thus, a method is needed for evaluating disparate, typically non-comparable criteria concurrently. In Part 1 of this research, cradle-to-grave environmental LCA was conducted for biomass delivery to a biosugar refinery using Ecoinvent v2.2 data and the TRACI 2 impact assessment method for midpoint impacts. Biomass availability, delivered cost, sugar yield, transportation distance, harvestable months per year, and other aspects of supply chain feasibility were measured for eighteen feedstock biomass types. In Part 2, stochastic multi-attribute analysis (SMAA) was used alongside LCA to develop an environmental preference single-score probability distribution function for feedstock alternatives. Weighted single-scoring and ranking, using multi-criteria decision-making analysis (MCDA), was conducted considering five criteria of biomass supply feasibility: biomass delivered cost, biosugar yield, harvestable months, transport distance, and environmental preference single-score. Corn was shown to cost the most, followed by switchgrass and U.S. primary forest products. Transport distance was found to be highest for residues due to low yield per acre and low covered area. Results of MCDA show that Brazilian eucalyptus and Malaysian empty fruit bunch biomass types were consistently preferred relative to other biomass types. In the U.S., Genera biomass sorghum is most holistically preferred. It is shown that SMAA is helpful for translating LCA data for decision science. It was shown that MCDA can be useful for early-stage biorefinery technology commercialization decision-making, using the novel decision science tool described herein.
Keywords: Biomass supply feasibility; Multi-criteria decision analysis; Life cycle assessment; Delivered cost; Biorefining
Contact information: Department of Forest Biomaterials, North Carolina State University, Campus Box 8001, NCSU Campus, Raleigh, NC 27695 USA; *Corresponding author: Richard_Venditti@ncsu.edu
INTRODUCTION
It is critical in the biofuels or biomaterials arena to develop methods to fairly and consistently compare different biofuel and bio-based chemical feedstocks and conversion pathways. In the first part of this study (Reeb et al. 2015), eighteen biomass feedstocks from three continents were compared based upon supply logistic data, delivered cost, environmental impacts, estimated monomeric sugar yields from a biochemical conversion process, transportation distance, and estimated regional biomass availability. In Part 2, data generated for each feedstock alternative are used to develop a robust and statistically-relevant weighted single-score ranking of alternatives, accounting for five selected feasibility criteria concurrently.
Life cycle assessment (LCA) is a methodology for which practitioners must typically take multiple conflicting or non-comparable criteria into account when interpreting the results and basing decisions on study results. The accounting of human health and environmental impacts provides stakeholders with a broad perspective on the trade-offs associated with each biomass feedstock scenario (alternative) modeled. An unavoidable conflict that arises when multiple non-comparable metrics are provided simultaneously is that of interpretation fidelity. This problem that users of an LCA are faced with during the interpretation of LCIA results has been primarily ignored or external normalization and subjective weight sets are used. The same is true when considering feasibility analysis more broadly, the analysis of multiple criteria of project feasibility including financial performance, logistics, economics, social impacts, environmental impacts, and technological aspects (You et al. 2012).
In operations research, artificial intelligence research and other fields where conflicting metrics must be evaluated concurrently to make a sound decision, often rapidly, various forms of multi-criteria decision-making analysis (MCDA) are often employed (Linkov et al. 2006). Bernoulli’s research (Bernoulli 1738) into the mathematical problem known as the St. Petersburg game led to the first MCDA techniques, which have today been expanded to dozens of techniques and hundreds of tools (Tzeng and Huang 2011). These tools can be categorized generally as either multiple attribute utility theory techniques or outranking techniques. While no existing tool is without problems, the concerns faced by each may possibly be avoided by concurrently employing multiple sorting/ranking techniques and utilizing the same raw value matrix and weight set to reflect the stakeholders’ values.
Previous research by Seager and co-workers (Rogers and Seager 2009; Prado-Lopez et al. 2014) and others have introduced stochastic modeling and outranking as part of a technique called internal normalization for LCA. A tool was developed called stochastic multi-attribute analysis for life cycle impact assessment (SMAA-LCIA) that uses Monte Carlo analysis, mean values for each impact category, and standard deviations of these mean values to create a cumulative environmental preference distribution. While this technique has helped improve the interpretation stage of LCIA, we purport that stakeholders are still left with a decision based upon disparate criteria because these techniques do not incorporate financial analysis, logistical feasibility, technical feasibility, etc. The SMAA technique is herein explored as part of a larger MCDA methodology.
MCDA enables a decision-maker to standardize their set of decision values (value set), create a weight set based upon these values, and consistently apply the weight set against a matrix of multiple criteria for multiple alternatives to facilitate more objective decision-making (Kaplan 2006; Giarola et al. 2011; Čuček et al. 2012; Prado-Lopez et al. 2012). There are numerous MCDA tools available that can be used to categorize and define criteria, generate an appropriate weight set, and conduct statistically meaningful, single-scoring, and ranking activities for multiple criteria concurrently (Ishizaka and Nemery 2013).
Even if multiple disparate criteria are provided to a stakeholder from feasibility analysis or from LCIA results, that stakeholder must either eliminate criteria deemed less important under that stakeholder’s value system to make a decision or at best apply a weighting method and employ single-scoring. Because the raw value results and even the normalized, weighted results of LCIA and feasibility analysis in general are often overly complex, a subsequent interpretation method must be employed to facilitate better interpretation.
It is hypothesized that the use of multiple MCDA methods to develop multiple rank orders of studied alternatives will provide evidence, through a lack of re-ranking between alternatives, of the robustness of the rank order and usability of this rank order for decision-making purposes. Herein, this is tested with a case study of biomass supply, comparing eighteen specific alternatives, taking into account biomass delivered cost, sugar yield for each feedstock via a biochemical conversion process, harvestable months, transportation distance, and environmental preference.
METHODS
The first steps toward multi-criteria decision making that encompasses environmental impacts, financial analysis, and other criteria of biorefinery feasibility is to define the problem being addressed, identify the criteria for analysis, and evaluate these criteria for potential overlap to avoid double-counting of attribute values during weighted scoring and rank ordering (Dubois and Prade 1980).
Detailed supply chain models were developed for various feedstocks (Daystar 2014; Daystar et al. 2014; Reeb et al. 2014, 2015), and life cycle assessment was conducted in SimaPro v7.2 (PRé Consultants 2014) using Ecoinvent 2.2 data (Frischknecht et al. 2005) and the TRACI 2.0 impact assessment method (Bare et al. 2002; Ryberg et al. 2014). Environmental impact categories included are shown in Table 1. SMAA-LCIA (Rogers and Seager 2009; Prado-Lopez et al. 2014) was conducted using midpoint impact values for each impact category and the Gloria et al. (2007) weighting set for producers. Environmental preference probability distribution functions were developed for each alternative and a single score was calculated from this for ranking purposes.
Table 1. TRACI Impact Categories, Acronyms and Units (Bare et al. 2002).
A representation of the combined supply chain, financial analysis, environmental LCA, and MCDA methodology used to rank biomass alternatives is shown in Fig. 1.
Fig. 1. Graphic representation of the combined feasibility assessment method for n biomass alternatives and the application of weighting values for m criteria during the multi-criteria decision-making ranking process
Delivered cost of the feedstocks and feedstock availability were calculated and sugar yield estimated as described by Reeb et al. (2014, 2015). The environmental preference single score, delivered cost (financial indicator), transportation distance (technical supply chain indicator), months of harvest (technical supply chain indicator), and sugar yield (technical conversion indicator) raw values were together considered the raw values matrix for the criteria set. Three primary MCDA methodologies were employed to explore the effect of various methods on rank order, to apply various weighting sets, and to employ different scoring distribution methods, Table 2.
Table 2. Scenario Parameters and Assumptions Used for Each MCDA Method
Stochastic Multi-Attribute Analysis for Environmental LCIA Results
Stochastic multi-attribute analysis of life cycle impact assessment results (SMAA-LCIA) was used to generate a multi-attribute single score of “environmental preference” through stochastic modeling of weighted environmental impact values and Monte Carlo simulation of the effect of data uncertainty (Prado-Lopez et al. 2014).
Uncertainty values used in the stochastic modeling of environmental preference probability distribution functions according to the Prado-Lopez et al. (2014) method are provided in Table 2. Coefficient of variation (CV) values, defined as the standard deviation as a percentage of the mean, are provided in Table 3 and can be used to understand the dispersion of the probability distributions generated for SMAA-LCIA.
Table 3. Aggregate Uncertainty Values (Frischknecht et al. 2005) as a Single Standard Deviation from the Mean (from Ecoinvent V.2.2 Data for Each Feedstock Scenario Model). Used in SMAA-LCIA for Stochastic Modeling of Cumulative, Weighted Environmental Preference
Units for each impact category left to right, respectively, are: kg CO2 eq., moles of H+ eq., kg N eq., kg 2,4-D eq., kg CFC-11 eq., kg NOx eq., kg benzene eq., kg toluene eq., and kg PM2.5 eq.
Weighting values used for the SMAA-LCIA analysis to obtain a single environmental preference values are shown in Fig. 2. These weighting values reflect a biorefiner’s values with respect to the importance of various environmental impact categories. The LCIA impact category weight values used herein during SMAA were derived from weighting values developed by Gloria et al. (2007) for producers (herein “biorefiners”).
Fig. 2. Probability distribution function of each TRACI impact factor used to determine
environmental preference single-score. Frequency is the probability (y-axis) that a TRACI impact category will have a particular weight contribution in % to the net environmental preference score (x-axis) is shown. The Prado-Lopez et al. (2014) method was followed.
The SMAA-LCIA tool does not allow for direct input of numerical weighting values such as the biorefinery weighting set. Instead, a “- -”, “-”, “=”, “+”, “+ +” weighting scheme (lower to higher) is used. Thus, relative weighting of the TRACI impact categories for a biorefinery that was applied towards the environmental preference single score were converted to this +/- scale, as outlined in Table 4. No weights of “- -” or “+ +” were assigned because weighting for each impact category for the biorefiner was relative to greater and lesser category weight values for LCA users and LCA experts (Gloria et al. 2007). The coefficient of variation values for LCIA mean values for each impact category for each feedstock alternative are provided in Table 5. Monte Carlo simulation was then conducted to develop weighted environmental preference probability distribution functions.
Table 4. The Biorefinery Weight Set from Gloria et al. (2007) and the Converted Form Used for SMAA-LCIA
Table 5. Coefficient of Variation (CV) of Impact for Each Feedstock by Impact Category, as Determined by Standard Deviation as a Percentage of the Mean
Note: Gray highlighting indicates CV values greater than 100%, meaning the standard deviation is greater than the mean impact value.
It is clear from this analysis of CV that some impact assessment methods are inherently more uncertain than others and that perhaps LCI data for specific feedstocks is less certain than others. Higher variation is observed for ecotoxicity, carcinogenics, and non-carcinogenics and for Thai bagasse and Brazilian sugarcane most specifically. The implications of this for interpreting the LCIA results are that any feedstock option which has a large uncertainty must be cautiously analyzed. For example, if the environmental preference score for product system A is benefited largely by a low carcinogenics value as compared to product system B, the uncertainty of the carcinogenics impact category discounts this environmental preference for A.
Through SMAA, LCIA results are aggregated to a weighted single-score that reflects the value set of a biorefinery, termed the environmental preference score, Table 6.
Table 6. Cumulative Environmental Preference Score and Probability Based Upon the Results of Stochastic Multi-Attribute Analysis Coupled with Life Cycle Impact Assessment (SMAA-LCIA). Rank Order of One Indicates the Most Environmentally Preferred Feedstock, Relative to Alternatives Considered
Mean environmental preference scores for each alternative, based upon stochastic modeling under uncertainty and utilizing impact category-specific weighting factors, were then used as single environmental preference values for MCDA. Because of the stochastic nature of the single-score calculation, uncertainty is taken into account but probability distribution function overlap between alternatives requires further analysis to determine how certain the rank order is with relation to adjacently ranked alternatives. The first order rank probability is a simple measure of the area under the probability distribution function curve, which overlaps with the alternatives ranked one rank position above or below by mean single-score value. The second order rank probability takes into account the two rank positions on either side of the alternative. High probability indicates high rank position certainty. Using these rank order probability values, it can be determined whether there is likely global or local rank uncertainty as measured for only these environmental single-score values produced using SMAA-LCIA. Consideration for uncertainty with respect to environmental single-scores was given during subsequent MCDA modeling and analysis of rank order results.
Data Generation
Criteria were selected for MCDA to reflect either technical feasibility, financial feasibility, or environmental feasibility of biomass supply. These criteria are:
- biomass delivered cost – financial indicator
- estimated sugar yield – technical indicator
- transportation distance – technical indicator
- harvestable months per year –technical indicator
- environmental preference single score – environmental indicator
Raw values for MCDA for these five criteria (Table 7) were generated through the creation of detailed supply chain models and subsequent financial and environmental life cycle analysis (Daystar et al. 2014, 2015; Reeb et al. 2014, 2015).
Table 7. Raw Values for MCDA Criteria for Each Feedstock Alternative Analyzed (Reeb et al.2014, 2015)
Units: Cost is US$ per bone-dry metric tonne biomass delivered, Sugar Yield is kg sugar per bone-dry metric tonne of biomass, Transport Distance is kilometers (km) per bone-dry metric tonne of biomass delivered, Harvestable Months is calendar months per year, and Environmental Preference is a unit-less value between 0 and 1, relative to other studied scenarios, with higher values being preferred.
Multi-Criteria Decision-Making Analysis
This study explores the usability of various MCDA methods and tools in the interpretation of a broad feasibility study that incorporates financial analysis, environmental LCA, and technical feasibility analysis. To accomplish this, a customization of the Pugh decision matrix method (Pugh 1991) was developed that calculates a combined preference score for each alternative through a sum of the product of raw values and the criteria weight set. This criteria weight set was developed using a decision-maker value set matrix and constrained randomization of weight values.
Multi-criteria decision-making analysis (MCDA) is a method of comparing alternatives based on multiple incomparable metrics to take all into account concurrently during decision making (Čuček et al. 2012; Benítez et al. 2014; Cinelli et al. 2014;). Herein, MCDA is used to interpret the results of feasibility analysis for the various metrics of analysis concurrently and interpret feasibility as a single value or rank position relative to other alternatives. Three methods (Table 2) and four experimental weighting schemes for each weighted method were employed and are described herein.
Method 1: Unweighted scoring method
An unweighted rank ordering of alternatives of the five criteria was conducted (bio-based sugar yield values, environmental midpoint impacts, biomass availability values, and delivered cost values) followed by a ranking of the sum of these rank order values for all alternatives. This ranking hinges on the de facto assumption that all five of these feasibility criteria are weighted the same in the ranking process, which may not necessarily be true for all stakeholders. For instance, a biorefinery operator might prioritize the delivered cost value and estimated sugar yield greater than the environmental impacts, in which case the equal weighted single-score ranking is not reflective of their interests. The single rank score calculation for alternative j and N criteria is provided in Eq. 1,
(1)
where Sj is the rank score for alternative j and Xi,j is the rank applied for criterion i for alternative j.
Method 2: Weighted, rank-order distributed scoring method
A weighted MCDA method was derived from the Pugh decision matrix method (Yang and Singh 1994; Pohekar and Ramachandran 2004; Doumpos and Grigoroudis 2013; Yao and Huang 2014). This method, using previous research based on economic considerations (Ishizaka and Nemery 2013), ranks the various alternatives by applying a weighting scheme against the multiple criteria being quantitatively summed to provide a single rank score. MCDA was conducted for the various feedstock options using weighted decision grid ranking that reflected a biorefinery operator viewpoint, whereby biomass cost was weighted 30%, sugar yield 25%, harvestable months 20%, transportation distance 15%, and environmental impact 10%. This arbitrary set of weighting factors was chosen such that the economics (embodied in the biomass cost and separately in the sugar yield) of the process is at the forefront of the decision. It is acknowledged that more research might be done to refine this weighting set. In this study, we also investigated the use of 55/20/10/10/5%, 45/30/10/10/5%, and 35/25/20/10/10% weight sets for the five criteria, respectively, to evaluate the sensitivity of ranking for different weight set values.
Feedstock alternatives for each criterion were rank ordered 1 to 5, where 5 is the most appropriate and 1 is the least appropriate. With this ranking, 20% of the alternatives were equally distributed by raw value rank order among each of the five possible ranking bins for each criterion. Weights were applied for each criterion such that the sum of the fractional weights equaled 1. The single rank score calculation for alternative j with N criteria is provided in Eq. 2,
(2)
where Sj is the weighted rank score for alternative j, N is the number of criteria evaluated, Xi,j is the score applied for a criterion i for alternative j, and Yi is the weighting value for criterion i.
Method 3: Weighted, raw value distributed scoring method
Method 2 improves decision-making over Method 1 by incorporating weighting and a quintile scoring step to normalize disparate and non-comparable raw values. However, Method 2 does not account for the relative magnitude of differences between raw values by distributing alternatives to the quintiles by raw value rank order. Thus, Method 3 was developed that incorporates weighting and a quintile scoring method but which distributes alternatives for each criterion across the quintile scoring space by criteria raw values instead of rank order. In this case, the alternative with the worst desirability will have a 1 and the best will have a 5, thus defining the range for those criteria. All other alternatives will have values of 1 through 5, depending on their magnitude in those criteria. Note that in this criterial scoring, there is not an equally distributed amount of alternatives in each quintile. As discussed above, sensitivity to weighting scheme was also investigated using alternative experimental weight sets. Further analysis of sensitivity was conducted using constrained weight set randomization in a stochastic method (see below).
In this analysis, a rational decision-maker for whom the weight sets were experimentally developed is considered to be the owner and operator of a bio-sugar refinery at a 500,000 BDMT yr-1 scale. Alternative criteria weights were generated using iterative constrained randomization to understand the impact of weight set uncertainty on rank order and rank order stability (Appendix Table A1). Constrained randomization entails the development of ranges of possible weighting values for each criterion in the set and sampling randomly within each range for a criteria weighting values, the sum of which converges on one. Randomization was conducted around the baseline weight set for biomass cost of 30%, sugar yield of 25%, harvestable months of 20%, transportation distance of 15%, and environmental impact of 10%. A weighted sum product calculation was then conducted (see above, Eq. 1 and 2) to calculate the weighted preference score and a ranking of alternatives by this multi-criteria preference score, obtaining a final rank order.
Sensitivity Analysis and Rank Stability
Sensitivity analysis for the constrained, randomized weight set used for Methods 2 and 3 was conducted through the use of three additional randomization matrices to develop constrained median weight values for the five criteria. Standard deviation of rank order between the four ranking scenarios was used to determine the sensitivity of weighted multi-criteria rank order to weighting factors.
A pair-wise Spearman’s rank correlation coefficient (ρ) of the resulting rank orders from the various MCDA techniques examined herein and weight sets used was then conducted to determine the stability of rank order between MCDA techniques (Sielska 2010), known as the re-rank rate,
(3)
where n is the number of alternatives and is the difference between the ranks of alternative ai in the pair of rankings compared. This way, pairwise rank stability calculations were conducted and a rank stability matrix was created to identify rank comparison, which yielded significant re-ranking (herein determined as ρ values less than 0.700). Significant re-ranking is indicative of high uncertainty in the rank order and should be considered in investment and operational decision-making for biomass conversion technology commercialization.
RESULTS AND DISCUSSION
The primary goal of this study was to provide some objective analysis methods for traditionally non-comparable data in the bioproducts arena for decision-makers in industry, government, and academia to help direct research and development efforts. Multi-criteria decision-making analysis (MCDA) methods were used to develop overall unweighted and weighted multi-criteria rankings of these feedstock options for the biochemical conversion pathway to monomeric sugars from cellulose and hemicelluloses.
Method 1: Unweighted ranking method
In the first unweighted ranking method, criteria contributed equally to aggregate rank scores and to the final rank order. Criteria raw values were ranked and ranks summed to an overall rank score assuming equal weighting, Eq. 1. The criterion-wise rank order and overall rank order assuming equal criterion weighting show the highest ranked biomass types (Table 8).
Table 8. Criterion-specific Ranking and Overall Unweighted Single-score Rank Results
Note: A rank of 1 is best.
From this ranking, Brazilian eucalyptus, forest residues, and Malaysian empty fruit bunch are the highest ranking feedstocks for South America, North America, and Southeast Asia, respectively. As described in the Methods section, this simplified method has limitations; therefore, weighted ranking methods are evaluated below.
Method 2: Weighted, rank-order distributed scoring method
Weighted rank analysis provides a more easily understood and accurate interpretation of the relative feasibility of alternatives studied based on a stakeholder’s priorities. MCDA weighting values used for this first weighted MCDA activity were 30% for cost, 25% sugar yield, 20% transportation distance, 15% harvestable months, and 10% environmental preference. The weighted summation of criterion-specific rank scores for each alternative were then used to rank alternatives in descending order by single rank score, Table 9 and Fig. 3.
Table 9. Scoring of Alternatives for Each Criterion to Generate a Total Weighted Single Score and Overall Ranking, Using MCDA Method 2 (M2) and Weight Set 1 (W1). A 5 is best for the Individual Criteria and the Single Score. Overall Rank of 1 is Best.
Single-scores for corn and other primary crops in North America are clearly disadvantaged by high delivered cost, whereas residues are advantaged by a lower delivered cost. Forest species and sugarcane single-scores are contributed to more substantially for sugar yield than residues. Rank position was not greatly affected by environmental preference, partially due to a low criterion weight and partially due to low variability in criterion score. For the weight set used in Table 9 (W1), Brazilian eucalyptus, oil palm empty fruit bunch, and Genera biomass sorghum are most preferred, respectively, in South America, Southeast Asia, and North America. In Fig. 3, a larger multi-criteria cumulative preference score indicates a more preferred feedstock, based upon the assumptions used herein.
Fig. 3. Multi-criteria weighted decision making cumulative preference scores by feedstock option. Weighting values were 30% delivered cost, 25% sugar yield, 20% transportation distance, 15% harvestable months, and 10% environmental preference. Higher preference score is better.
Three additional weighting schemes were developed to explore the sensitivity of rank order to weighting. Experimental weight sets (Appendix Table A1) may reflect the viewpoint of decision makers with different values. The results of this weighted uniform ranking activity are provided in Appendix Table A2. It is evident that those feedstocks that are financially advantaged, such as residue biomass types and those from Southeast Asia and South America, increased their rank order, as the second value set increased the prioritization of low biomass cost. Some re-ranking of scenarios was seen between weight set 1 (W1), 2 (W2), 3 (W3) and 4 (W4), in particular for rice hulls and Brazilian sugarcane, for which rank position ranged from 7-13 and 5-10, respectively (Appendix Table A2). Switchgrass was consistently ranked lowest (18th rank position) for all weight sets for Method 2.
Method 3: Weighted, raw value distributed scoring method
The merit of adding weighting values to a cumulative rank scoring method is that it more closely approximates the priorities (values) of the decision maker, as in Method 2. However, the rank order distributed scoring method (Method 2) does not account for the magnitude difference of criterion a for alternative 1 as compared to the same criterion for alternative 2. For equally-weighted rank scoring systems this is not a detriment to the accuracy of the final rank order. However for weighted scoring the shifting of benefit from criterion a to criterion b for the purposes of an aggregate weighted score means that the magnitude difference between criterion afor alternative 1 and for alternative 2 could be misleading if simply sorted by rank order for each criterion. Therefore, Method 3 was developed and takes into account the magnitude difference between alternatives for each criterion, the results of which are provided herein for weight set 1 (W1) (Table 10). This methodological change caused re-ranking to occur relative to Method 2, even when holding weighting set values constant. In Table 10, the scores for any of the criteria are not evenly distributed through the scoring space (1-5), but are reflective of the magnitudes of criteria values relative to alternatives. For instance, harvestable months scores are mainly scores of 1 since many of the biomass types can only be harvested for 2-3 months per year. Some biomass types can be harvested 12 months per year, resulting in values of 5.
Table 10. Scoring of Alternatives for Each Criterion to Generate a Total Weighted Single Score and Overall Ranking for MCDA Method 3 (M3), Using Weight Set 1 (W1). A 5 is best for the Individual Criteria and the Single Score. Overall Rank of 1 is Best.
As noted above, harvestable month criteria scores contribute substantially less to overall single-score using Method 3 than for Method 2, due to the clustering of alternatives within scoring quintile 1. More polarization is also seen for environmental preference score due to lack of raw value differentiation across raw value range. For Method 3 and weight set 1, Brazilian eucalyptus, Genera biomass sorghum, and oil palm empty fruit bunch biomass types were found to be most holistically preferred, respectively, for South America, North America and Southeast Asia. Whereas for Method 2 switchgrass was found to be consistently ranked lowest, for Method 3 the lowest ranked was consistently corn. As for Method 2, three additional weight sets (W2, W3, and W4) were used for single-scoring and ranking using Method 3. The use of different weight sets, for some biomass types, specifically rice hulls and Genera switchgrass, resulted in rank position variability with ranges of 6-12 and 8-14, respectively.
In Fig. 4, single-scores for each alternative and contribution of each criterion to single-score are shown, using Method 3 and weight set 1. Herein, a single-score of 5 is most preferred.
Fig. 4. Cumulative preference scores, distributed for each criterion across a criterion-specific quintile, and final alternative rank order. The weighting set was delivered cost 30%, sugar yield 25%, transportation distance 20%, harvestable months 15%, and environmental impact 10%.
As can be seen in Fig. 4, other than the Genera biomass sorghum biomass scenario, residues and non-domestic biomass types dominate the upper rank positions. Additionally, despite the fact that biomass cost is weighted higher than sugar yield, sugar yield often contributes more substantially to cumulative preference score. For some biomass types, transportation distance, which is weighted only 20%, and delivered cost, weighted 30%, contribute to single-score equally. This is likely due to more polarization among alternatives for Method 3 as compared to Method 2. This equality of contribution from delivered cost and transportation distance to single-score practically disappears for weight sets 2 and 3, for which delivered cost is weighted even more heavily than transport distance as compared to weight set 1.
Follow-up research will investigate the sensitivity of the results to other alternative parameters (biomass availability, various cost drivers, and chemicals and energy necessary to produce the biomass, among others) and the impact of MCDA methodology used.
Ranking stability analysis
Comparing the rank order results from the unweighted method (Method 1), weighted, rank-order distributed method (Method 2), and weighted, criterion value magnitude distributed method (Method 3) shows some re-ranking. This rank order instability is an indication of rank sensitivity to weighting and MCDA methodology (Appendix Table A2). Consistent among the results of these analyses is that Brazilian sugarcane bagasse, Brazilian eucalyptus, and Malaysian empty fruit bunch are ranked within the top five biomass types, though localized re-ranking occurs between methods and weight sets.
Spearman’s rank order correlation coefficient (ρ, see Eq. 3) was calculated for pairwise comparison of rank order change between the various cumulative unweighted and weighted rank orders developed using either differing methodologies or differing weight set assumptions. Then average ρ values (Table 11) were calculated for changing study parameters to identify the parameter that most affects rank order stability.
Table 11. Spearman’s Rank Order Correlation Coefficient (Sielska 2010) Calculated for Pairwise Comparisons of the Rank Order
The value of Spearman’s correlation coefficient (ρ) is that the rank order stability can be determined for differences in MCDA method and results as well as for differences in results due to different weight sets, holding MCDA method constant.
- The average ρ value for parameter number 1 was calculated by comparing the pair-wise rank order stability between Methods 1, 2, and 3 for each weight set, holding weight set constant. This value was 0.10 and indicates significant rank instability between the three methods.
- For parameter 2, four different weight sets (W1, W2, W3, and W4) were used, holding MCDA method (2 and 3) constant. The average ρ value was 0.84, indicating only moderate rank instability due to weight set choice and that the use of distinctively different weight sets might not affect the outcome in this study to the same extent as method choice. This is also an indicator that weight set uncertainty may not be causative of rank order uncertainty.
- For parameters 3 and 4, the four different weight sets are again used, this time for only Method 2 and Method 3, respectively, including the ρ value for rank order stability between Method 1 and Method 2 and Method 1 and Method 3, respectively. These ρ values were determined to be 0.91 and 0.78, respectively.
- For parameter 5 and 6, the same analysis was conducted as for parameters 3 and 4, for Method 2 and Method 3, respectively; however Method 1 was not included in the calculation of average ρ values. These ρ values were 0.95 and 0.91, signifying a high level of rank order stability, indicating minimal risk of re-ranking when weight set is changed.
- For parameters 7 and 8, constrained randomization (CR) was conducted around weight set 1 (W1), for only Method 2 or Method 3, respectively. The ρ values were 0.81 for Method 2 and 0.88 for Method 3, representing a low risk of re-ranking as found for parameters 5 and 6. It is important to note that randomization minimally affected the average ρ value for Method 2 and Method 3. This is likely due to a decrease in weight set variability between CR1, CR2 and CR3 as compared to W1, W2, W3, and W4.
- For parameter 9, the average ρ value is calculated for all pair-wise comparisons of rank order as constrained randomization is applied (CR1, CR2, and CR3) for Method 2 and Method 3. The ρ value was 0.10, indicating very low rank order stability. Since rank order stability was not found to change drastically due to weight change, and even less due to constrained randomization, this low rank order stability is most likely due to the differences between Method 1 and Methods 2 and 3 with respect to criterion values and single-scoring.
Only Brazilian eucalyptus and Malaysian empty fruit bunch were ranked in the top five feedstocks for every method and weight set combinations (see Appendix Table A2). Alternatively, only corn and corn stover ranked in the bottom five for all method and weight set combinations. With the amount of re-ranking caused by method and weight set changes, as shown above, the fact that these feedstocks did not change rank outside of a narrow rank order range indicates a high level of confidence in their preference or rejection as viable biomass feedstock for biorefining.
Weight set change minimally impacts rank order stability (and therefore the confidence that a decision maker can have in the original rank order) and method change more drastically impacts rank order stability. Therefore, a decision maker should use a consistent MCDA method and apply a best approximated weight set for a rank order that is most indicative of the decision-makers’ values.
It can be reasoned that weight set 2 (55/20/10/10/5% for biomass cost, sugar yield, transport distance, harvestable months, and environmental single-score, respectively) is most representative of the priorities of a biorefinery owner/operator, and that MCDA Method 3 handles magnitude difference between alternative values most appropriately. Using these decision model parameters, Genera biomass sorghum, Malaysian empty fruit bunch, forest residues, Brazilian eucalyptus, or Brazilian sugarcane bagasse are most preferred biomass feedstock for a sugar biorefinery. It should be noted that the methods and results presented herein are for screening purposes. Entities intending to invest in a commercial-scale biorefinery or working in the stage-gate process for technology scale-up would conduct more stringent as well as more site- and feedstock-specific analyses to determine feasibility.
What is evident from these unweighted and weighted multi-criteria ranking activities is that additional and clearer interpretative tools are necessary both for multi-criteria decision making and for the interpretation stage of LCA. Future work will explore the use of SMAA alongside other internal and external normalization and weighting schemes. By doing so, more insightful and useful interpretation of feasibility and LCA study results for decision makers and other stakeholders will be established. Further retrospective analysis of the value of these MCDA methods for biorefining could also confirm the efficacy of these methods.
CONCLUSIONS
- A weighted single-scoring method (Method 3: weighted, raw criterion value distributed scoring method) was developed that integrates typically non-comparable feasibility metrics and facilitates more objective scenario comparison and ranking.
- Only Brazilian eucalyptus and Malaysian empty fruit bunch were ranked in the top five biomass types for all weight sets and multi-criteria decision-making analysis ranking methodologies, confirming their overall preference through a lack of non-localized re-ranking.
- The weighted multi-criteria single-score ranking system is flexible in that the weighting parameters can be modified to better reflect the values of the stakeholder.
- Using Spearman’s correlation coefficient, it was possible to identify the study parameters that most affected rank order stability and therefore uncertainty in final rank order.
- Rank order for biomass selection is more sensitive to multi-criteria decision-making analysis method used than to the weight sets explored, meaning that use of a reasonable experimental weight set and the weighted, raw value distributed method (method 3) will approximate a rank order that reflects the decision-makers values.
ACKNOWLEDGMENTS
The authors are grateful for the support of the Eastman Chemical Company and editing help from Barclay Satterfield, Lauren Johnson, Rebecca Glaspie, and Randy Waymire. The authors are also grateful to Kelly Tiller and Sam Jackson of Genera Energy for primary biomass supply system data.
REFERENCES CITED
Bare, J. C., Norris, G. A., Pennington, D. W., and McKone, T. (2002). “TRACI: The tool for the reduction and assessment of chemical and other environmental impacts,” Journal of Industrial Ecology 6(3-4), 49-78. DOI: 10.1162/108819802766269539
Benitez, J., Carrión, L., Izquierdo, J., and Pérez-Garcia, R. (2014). “Characterization of consistent completion of reciprocal comparison matrices,” Abstract and Applied Analysis 2014, 349729. DOI: 10.1155/2014/349729
Bernoulli, D. (1738). “Specimen theoriae novae de mensura sortis (Exposition of a new theory on the measurement of risk),” Commentarri Academiae Scientiarum Imperialis Petropolitanae 5, 175-192 (translated for Econometrica 22(1), 23-36, 1954). DOI: 10.2307/1909829
Cinelli, M., Coles, S. R., and Kirwan, K. (2014). “Analysis of the potentials of multi criteria decision analysis methods to conduct sustainability assessment,” Ecological Indicators 46, 138-148. DOI: 10.1016/j.ecolind.2014.06.011
Čuček, L., Varbanov, P. S., Klemeš, J. J., and Kravanja, Z. (2012). “Total footprints-based multi-criteria optimization of regional biomass energy supply chains,” Energy 44(1), 135-145. DOI: 10.1016/j.energy.2012.01.040
Daystar, J. S. (2014). Environmental Impacts of Cellulosic Biofuels Made in the South East: Implications of Impact Assessment Methods and Study Assumptions, Ph.D. dissertation, North Carolina State University, Raleigh, NC. http://repository.lib.ncsu.edu/ir/bitstream/1840.16/9698/1/etd.pdf
Daystar, J. S., Gonzalez, R., Reeb, C. W., Venditti, R. A., Treasure, T., Abt, R., and Kelley, S. (2014). “Economics, environmental impacts, and supply chain analysis of cellulosic biomass for biofuels in the southern US: Pine, eucalyptus, unmanaged hardwoods, forest residues, switchgrass, and sweet sorghum,” BioResources 9(1), 393-444. DOI: 10.15376/biores.9.1.393-444
Daystar, J. S., Reeb, C. W., Gonzalez, R., and Venditti, R. (2015). “Environmental life cycle impacts of cellulosic ethanol in the Southern US produced from loblolly pine, eucalyptus, unmanaged hardwoods, forest residues, and switchgrass using a thermochemical conversion pathway,” Fuel Processing Technology 138, 164-174. DOI: 10.1016/j.fuproc.2015.04.019
Dubois, D., and Prade, H. (1980). Fuzzy Sets and Systems: Theory and Applications, Academic Press, New York, NY.
Frischknecht, R., Jungbluth, N., Althaus, H.-J., Doka, G., Dones, R., Heck, T., Hellweg, S., Hischier, R., Nemecek, T., Rebitzer, G., and Spielmann, M. (2005). “The ecoinvent database: Overview and methodological framework,” International Journal of Life Cycle Assessment 10(1), 3-9. DOI: 10.1065/lca2004.10.181.1
Giarola, S., Zamboni, A., and Bezzo, F. (2011). “Spatially explicit multi-objective optimization for design and planning of hybrid first and second generation biorefineries,” Computers and Chemical Engineering 35(9), 1782-1797. DOI: 10.1016/j.compchemeng.2011.01.020
Gloria, T., Lippiatt, B., and Cooper, J. (2007). “Life cycle impact assessment weights to support environmentally preferable purchasing in the United States,” Environmental Science and Technology 41(21), 7551-7557. DOI: 10.1021/es070750+
Ishizaka, A., and Nemery, P. (2013). Multi-Criteria Decision Analysis: Methods and Software, John Wiley & Sons, Ltd., West Sussex, UK.
Kaplan, P. O. (2006). A New Multiple Criteria Decision Making Methodology for Environmental Decision Support, Ph.D. dissertation, Department of Civil Engineering, North Carolina State University, Raleigh, NC. http://repository.lib.ncsu.edu/ir/bitstream/1840.16/3123/1/etd.pdf
Linkov, I., Satterstrom, F. K., Kiker, G., Batchelor, C., Bridges, T., and Ferguson, E. (2006). “From comparative risk assessment to multi-criteria decision analysis and adaptive management: Recent developments and applications,” Environment International 32(8), 1072-1093. DOI: 10.1016/j.envint.2006.06.013
Pohekar, S. D., and Ramachandran, M. (2004). “Application of multi-criteria decision making to sustainable energy planning – A review,” Renewable and Sustainable Energy Reviews 8(4), 365-381. DOI: 10.1016/j.rser.2003.12.007
Prado-Lopez, V., Rogers, K., and Seager, T.P. (2012). “Integration of MCDA tools in valuation of comparative life cycle assessment,” Chapter 19 in: Life Cycle Assessment Handbook: A Guide for Environmentally Sustainable Products, M. A. Curran (ed.), John Wiley & Sons, Inc., Hoboken, NJ, pp. 413-431.
Prado-Lopez, V., Seager, T., Chester, M., Laurin, L., Bernardo, M., and Tylock, S. (2014). “Stochastic multi-attribute analysis (SMAA) as an interpretation method for comparative life-cycle assessment (LCA),” International Journal of Life Cycle Assessment 19(2), 405-416. DOI: 10.1007/s11367-013-0641-x
PRé Consultants. (2014). “SimaPro v. 7.2 Life Cycle Assessment Software,” Amersfoort, The Netherlands.
Pugh, S. (1991). Total Design – Integrated Methods for Successful Product Engineering, Addison-Wesley Publishing Company, Harlow, UK.
Reeb, C. W., Hays, T., Venditti, R., Gonzalez, R., and Kelley, S. (2014). “Supply chain analysis, delivered cost and life cycle assessment of oil palm empty fruit bunch biomass for green chemical production in Malaysia,” BioResources 9(3), 5385-5416. DOI: 10.15376/biores.9.3.5385-5416
Reeb, C. W., Venditti, R., Hays, T., Daystar, J., Gonzalez, R., and Kelley, S. (2015). “Environmental LCA and financial analysis to evaluate the feasibility of bio-based sugar feedstock biomass supply globally: Part 1. Supply chain analysis,” BioResources 10(4), 8098-8134. DOI: 10.15376/biores.10.4.8098-8134
Rogers, K., and Seager, T. P. (2009). “Environmental decision-making using life cycle impact assessment and stochastic multiattribute decision analysis: A case study on alternative transportation fuels,” Environmental Science and Technology 43(6), 1718-1723. DOI: 10.1021/es801123h
Ryberg, M., Vieira, M. D. M., Zgola, M., Bare, J., and Rosenbaum, R. K. (2014). “Updated US and Canadian normalization factors for TRACI 2.1,” Clean Technology and Environmental Policy16(2), 329-339. DOI: 10.1007/s10098-013-0629-z
Sielska, A. (2010). “Multicriteria rankings of open-end investment funds and their stability,” Operations Research and Decisions 1, 111-129. ISSN: 1230-1868.
Tableau Software (2014). “Tableau Desktop v. 8.3 Data Analysis Software,” Seattle, WA (www.tableausoftware.com).
Tzeng, G.-H., and Huang, J.-J. (2011). Multiple Attribute Decision Making: Methods and Applications, Taylor and Francis, New York, NY.
Yang, J.-B., and Singh, M. G. (1994). “An evidential reasoning approach for multi-attribute decision making with uncertainty,” IEEE Transactions of Systems, Man, and Cybernetics 34(1), 1-18. DOI: 10.1109/21.259681
Yao, S., and Huang, W.-Q. (2014). “Induced ordered weighted evidential reasoning approach for multiple attribute decision analysis with uncertainty,” International Journal of Intelligent Systems29(10), 906-925. DOI: 10.1002/int.21669
You, F., Tao, L., Graziano, D. J., and Snyder, S. W. (2012). “Optimal design of sustainable cellulosic biofuel supply chains: Multiobjective optimization coupled with life cycle assessment and input-output analysis,” AIChE Journal 58(4), 1157-1180. DOI: 10.1002/aic.12637
Article submitted: June 19, 2015; Peer review completed: August 30, 2015; Revisions accepted: May 19, 2016; Published: May 24, 2016.
DOI: 10.15376/biores.11.3.6062-6084
APPENDIX
Table A1. Experimental Weight Set Values (W1, W2, W3, and W4) for the Selected Criteria and Weight Set Values Generated using Iterative, Constrained Randomization as Described in the Methods Section (CR1, CR2, and CR3)
Note: All weight sets sum to 1.00.
Table A2. Rank Order for All Method and Weighting Combinations and for Three Constrained Randomization Scenarios, Based on Delivered Cost, Sugar Yield, Transport Distance, Harvestable Months, and Environmental Preference
Note: A rank of 1 is best. M1 = unweighted ranking method, M2 = weighted scoring by criterion-specific rank order, M3 = weighted scoring by raw criterion value across fixed scoring space; W1, W2, W3 and W4 are described in Table A1; CR1, CR2 and CR3 represent three weight sets, described in Table A1, developed using constrained randomization.