Energy Technology

Energy Technology Demonstrations: The Business Case

 

Contents


Resources for project managers:

Reports:

 

Resources for project engineers:

 

Introduction to business case analysis

The cost-benefit analysis of energy demonstrations (CBA-ED) project is a part of the Office of Naval Research’s Energy Systems Technology and Evaluation Program (ESTEP). The purpose of the CBA-ED is to develop methods and tools to estimate the value of ESTEP demonstration projects to Department of Navy installations, capturing non-monetary impacts and learning value. Currently, we are interviewing ESTEP project teams to:

  • validate the installation energy objectives and associated costs and benefits; and
  • identify and validate learning-value mechanisms

For more information, contact principal investigators Eva Regnier, Dan Nussbaum, Geraldo Ferrer, and Brandon Naylor of the Naval Postgraduate School's Energy Academic Group.


What is Cost-Benefit Analysis?

A cost-benefit analysis (CBA), sometimes called a Benefit-Cost Analysis or a business case analysis (BCA) is a comparison of alternatives in terms of all their benefits and costs. When only one alternative (e.g. a technology) is considered, the comparison is between the proposed alternative and the status quo—which serves as a reference case.

Office of Management and Budget (OMB) Circular A-94, Guidelines and Discount Rates for Benefit-Cost Analysis of Federal Programs, is the official guidance on how U.S. federal agencies (including the Department of Defense) should evaluate and select projects and programs.

 

Official A-94 definition of BCA

“The standard criterion for deciding whether a government program can be justified on economic principles is net present value—the discounted monetized value of expected net benefits (i.e., benefits minus costs). Net present value is computed by assigning monetary values to benefits and costs, discounting future benefits and costs using an appropriate discount rate, and subtracting the sum total of discounted costs from the sum total of discounted benefits.” (bold added)

“Although net present value is not always computable…efforts to measure it can produce useful insights even when the monetary values of some benefits or costs cannot be determined. In these cases:

A comprehensive enumeration of the different types of benefits and costs, monetized or not, can be helpful in identifying the full range of program effects.

Quantifying benefits and costs is worthwhile, even when it is not feasible to assign monetary values; physical measurements may be possible and useful.”

Consistent with this guidance, the project's goal is to estimate the value of a technology demonstration project in monetary units, specifically net present value, adjusting for the time-profile of costs and benefits, with costs generally occurring early and benefits accruing over time. All costs and benefits will be measured relative to existing solution for providing relevant services as the baseline or reference technology.

 


On-site energy storage 

---

 


Adoption readiness

 


Benefits of installation energy technologies


Strategic Objectives

Energy is a critical input for almost every mission and support capability of at United States Department of Navy (Navy) installations. We have enjoyed nearly uninterrupted access to electrical power in recent years, especially in the US but installations' dependence on the “fragile and vulnerable” commercial power grid (DSB 2008, p. 3) is also a vulnerability.

As part of the process of addressing the key vulnerability of operational and installation energy requirements, Department of Defense (DoD) and Navy leaders have issued guidance documents outlining goals and objectives relating to energy. For example, in 2009, the Secretary of the Navy’s set forth five energy targets for the DoN, briefly

  • Contracts: include energy evaluation factors in contracts.
  • Green Strike Group: in 2012, sail a strike group on nuclear and biofuel power only, and in 2016, deploy a fleet including aircraft flying on only biofuels.
  • Consumption: reduce petroleum use by 50%.
  • Alternative sources: half of shore-based energy produced on-installation and from non-fossil-fuel sources by 2020.
  • Alternative sources: half of all DON energy derived from non-fossil-fuel sources by 2020.


The Secretary’s goals and other strategic documents specify a wide range of energy-related objectives that overlap only partially, while identical terms appear in many documents, but with inconsistent definitions. As part of the CBA-ED project, we reviewed strategic guidance documents and defined a comprehensive set of common objectives for energy decisions in the DoD. The objectives and associated definitions are intended to facilitate horizontal and vertical communication within the DoD. For more documentation on the process, see Whitney et al., 2013.

The resulting objectives hierarchy and means-ends network and definitions can help provide structure to decisions at all levels of the organization by identifying objectives, illustrating connections between fundamental and means objectives, suggesting possible ways of quantifying objectives, and providing a basis for evaluating specific tradeoffs among competing objectives.


Installation Energy Objectives

The strategic guidance communication has been very successful in bringing attention to energy and activating decisions that improve the DoD’s energy profile throughout the organization. challenges remain. It is not always clear what objectives and metrics a decision maker within the DoD should use to guide and justify decisions in which energy is a factor, and it is difficult to ensure that decision makers across different parts of the DoD are consistent in their implementation of the strategies provided by DoD leadership.

Based on the strategic objectives and interviews with the ESTEP project teams, we developed a means-ends network for energy technology adoption decisions at Navy installations, and definitions of the objectives that suggest cost and benefit metrics.

Each ESTEP project provides benefits in one or more of these areas, and a checklist of the key ESTEP-relevant benefits can help identify the key impacts (costs and benefits) of new ESTEP projects.

Theses:

  • Campbell, J.O. & Hutcheon, A.R. (2018, September) MS in Information Technology Thesis. Energy Storage Systems and Utility Cost Savings for DoD Installations. Outstanding Thesis Award.
  • Gordon, K.L. (2018, June) MBA Thesis. Case Studies on Technology Adoption in Navy Energy and Environmental Technology Projects.
  • Adams, B.J. & Hartner, C.C. (2017, December) MBA Thesis. Cyber-Defense Return-on-Investment for NAVFAC Energy Technologies.

 
Other:

  • Hudgens, B.J., Hartner, C., Adams, B., Regnier, E.D. (2019, January) Investing in Cyber Defense: A Value Focused Analysis of Investment Decisions for Microgrids in Proceedings, 2019 52nd Hawaii International Conference on System Science (HICSS), pp. 1222–1230. IEEE.
  • Gannon, A., Regnier, E., Taber, S. (2019) Electricity Spikes and the Power of Collaboration. Defense Acquisition (formerly Defense AT&L) 48(2): 20–23.

 

 

Drivers of value in S&T projects

Many Non-monetary Costs and Benefits

The costs and benefits of installation energy project should be summarized with a small set of relevant metrics that are tightly linked to fundamental objectives.
 

Why?

Using many metrics dilutes the impact of each metric on the final valuation of a project. Results are usually better with a small number of important metrics.

In addition, estimating each metric requires more time and data and introduces measurement/estimation error that further reduces the validity of the final result.

How?

One of the most important parts of an analysis is deciding what is and is not important enough to be included in the analysis. Spending time identifying fundamental values — and distinguishing between supporting (means) objectives and fundamental (ends) objectives — can uncover duplication among objectives (e.g. meeting a mandate to generate renewable power and maximizing renewable power generation capacity) and reveal the relative importance of objectives, suggesting some that may be excluded from analysis.

Reference: Chapter 3 in Keeney, R. L. (2009). Value-focused thinking: A path to creative decisionmaking. Harvard University Press.


Non-monetary impacts of the technology should be measured in real-world units as much as possible. For example, some installation energy technologies improve reliability of mission-critical power. The frequency of outages is a natural-unit metric of reliability.
 

Why?

Translating an impact into a non-natural (constructed) measurement scale, such as a score of 1–10 adds a layer that can obscure the meaning of the underlying impact, making it harder for consumers of the analysis to interpret the results. Perhaps more important, if decision-makers who are not very familiar with the real-world meaning associated with the constructed scale need to provide feedback such as preferences about trade-offs, their feedback will be less reliable. It is harder to give good input about the relative importance of improving reliability from 6 to 7 than about reducing the frequency of outages from twice per year to once per year.

How?

Whenever possible, identify a real-world measure that is tightly linked to the objective in question. Sometimes it is tempting to use a constructed scale in place of an objective that is difficult to predict or measure. Using a constructed scale disguises and compounds the measurement or estimation error and makes the measurement less precise and accurate, not more.

References: Hubbard, D. W. (2010). How to Measure Anything: Finding the Value of Intangibles. Business, 2nd Edition. Hoboken, New Jersey: John Wiley & Sons, Inc.
http://www.howtomeasureanything.com


Each technology project's direct net benefits in operation should be summarized using a quantitative preference model.
 

Why?

In many public-sector decisions, it is difficult to estimate the equivalent monetary value of some important impacts. However, there is an established method for modeling decision makers' preferences, and use this model to compare alternatives' value based on their achievement of multiple incommensurable objectives.   

How?

The process of modeling decision-maker preferences can be quite involved. An example of the model structure is offered. A number of introductory references are listed below. In addition, the Naval Postgraduate School's Defense Resources Management Institute offers a two-week tuition-free course on this topic.

  • Keeney, R. L. (2009). Value-focused thinking: A path to creative decisionmaking. Harvard University Press.
  • Kirkwood, C. W. (1997). Strategic decision making. Belmont, CA: Duxbury Press.
  • Parnell, G. S. 2007. Value-focused thinking using multiple objective decision analysis. A. Loerch, L. Rainey eds. Methods for conducting military operational analysis: Best practices in use throughout the Department of Defense. Military Operations Research Society, Alexandria, VA, 619-656.
 
The next step is to validate our installation energy objectives with project teams and work with teams to identify ways to estimate or measure the impact the technology could have on these objectives. Based on project team interviews, we have a preliminary report for each ESTEP FY13 start project team to review on the objectives relevant to each project.
 
Navy installation energy objectives identification describes our process for identifying a small set of objectives with associated real-world units. These objectives are reported for most of the FY13 ESTEP project teams.
 

Value Through Learning

Estimate impact of learning on future installation energy investments.
Much of the value of a demonstration projects comes from learning and therefore will be realized in future adoption of the technology at a larger scale over more installations.
 

Why?

A key distinction between demonstration projects and other projects is that much of the value of a demonstration projects comes from learning and therefore will be realized in future adoption of the technology at a larger scale over more installations. In general, the net direct impacts of a demonstration project will be negative for a number of reasons. They usually require expensive, skilled labor to implement, and incur initial costs to develop new specifications. They are generally small in scale, and their technical performance is more uncertain than for non-demonstration investments. Failing to capture the impact of the learning gained on future energy investments would seriously understate the value of a demonstration project.

How?

In order to estimate this value, we need a model of future installation energy investment decisions and the impacts of the technology once adopted. The model is described below. To use this model, we need to understand the mechanisms by which learning will contribute to value for the Navy for each demonstration technology.


Measure value of demonstration relative to a reference technology.
 

Why?

Per the guidance and economic best practices, the impacts of any course of action (e.g. an energy technology investment) should be evaluated relative to an alternative. For example, we should not assume that, in the absence of a photovoltaic array, the installation will have no electrical power. Costs and benefits are net and should be measured as increments relative to some reference case (also called a base case)—usually the status quo—for how the function that a given technology performs would be performed if the course of action is not adopted.

How?

For each energy technology, we need to understand the reference technology, or base case, relative to which costs and benefits should be estimated.

The three first-order learning mechanisms identified are:

  • Navy-specific adaptation: Most installation energy technologies have civilian technologies and are being developed commercially. However in many cases, the technology requires military-specific adaptations or experience using the technology makes the Navy more efficient and effective in procuring, installing, and using it. For example, learning how to use energy management software consistent for Navy installation applications can be shared among installations, making them all more effective.
  • Competing technologies: If a demonstration project compares multiple instances of a technology (e.g. equipment from multiple manufacturers, or comparing a new technology with the existing technology it would replace), then it provides information that helps the Navy determine which is the better choice for future adoption and increases the likelihood that the Navy will adopt the most effective and efficient technology in future investments. For example, comparing multiple photovoltaic roof-mounting technologies helps the Navy acquire the one that is best.
  • Justification for wider investment: Technologies that have been demonstrated in a Navy installation may fare better in competition for limited funds in the Navy project selection processes. In addition, they may pass more quickly through the specification and contracting processes. For example, an information assurance accreditation for a demonstration project could speed the process of getting accreditation for wider adoption. In this way, learning can accelerate the adoption of more efficient and effective installation energy technology. The next step is to interview each project team to identify the relevant learning mechanisms for their technology.

Two further important learning mechanisms for ESTEP projects are:

  • Enabling technologies: For some energy technologies, such as energy management systems, cost savings are indirect (this may be called second-order). In other words, they enable either behavioral changes (e.g. conservation) or changes in the installation energy infrastructure (procurement of new equipment, or operational/configuration modifications) by providing information or control capabilities. The impact of these technologies is difficult to predict. One possible approach is to study empirically the impact of similar technologies that have been adopted in the DoN or elsewhere. The value of enabling technologies is not captured in the current model.


Taxonomy of enabling technologies

   

Locus of changes
    Behavorial (end user) Installation energy systems (configuration / equipment)

Enabling Mechanism

 Information / Feedback Feedback to enduser, e.g. local power consumption readings or power unit price System operational data, e.g. smart metering, or other power system performance data that can be used to make design choices and/or justify procurement requests
  Control User control, e.g. individual room or cubicle thermostats System management capability, e.g. capability for faster, or remote modification of operational systems for conservation or greater reliability

 

  • Workforce development: One of the ways the ESTEP program is distinct from other energy technology demonstration programs is its emphasis on Navy workforce development. In ESTEP there are at least three types of workforce development being supported: internships, master's theses, and experience by Navy civilian personnel using the technologies under study. While these opportunities have value to the individuals who take advantage of them, in the context of this analysis we are interested in estimating the value of workforce development to the DoN. Separating the impact of workforce development from other contributors to organizational success is a challenge, and there is little quantitative analysis in this area. The value of enabling technologies is not captured in most ROI, CBA and BCA approaches.


Slide
 

Slide

 

 

 


Reports:

 

Business case analysis

 

 

 


ESTEP retrospective

---

 


Other reports

  • Hudgens, B.J., Hartner, C., Adams, B., Regnier, E.D. (2019, January) Investing in Cyber Defense: A Value Focused Analysis of Investment Decisions for Microgrids in Proceedings, 2019 52nd Hawaii International Conference on System Science (HICSS), pp. 1222–1230. IEEE.
  • Gannon, A., Regnier, E., Taber, S. (2019) Electricity Spikes and the Power of Collaboration. Defense Acquisition (formerly Defense AT&L) 48(2): 20–23.
Contact Us - Sidebar

Questions

How can we help with your energy-related education, research, and outreach?
Talk with us