|
|
 |
Services:
|
|
Risk Analysis Services
Major capital projects are reviewed at two key stages of project development: 1) Before Planning Authorization, for which a Value Planning Workshop is conducted at the conceptual phase of a project to focus on higher order functions and 2) Before Project Authorization, for which a Value Engineering Workshop is conducted at the preliminary design phase prior to the Contract Document preparation phase of a project to focus on more technical issues and a narrower scope of functions. In either case, the products of the value management process
include presentations to the ultimate policy making body of the Authority, its Board of Commissioners. For any capital project, the Line Department Director presents to the Board a Project Proposal that discusses the goals and objectives of the project all alternatives, its financial implications, project alternatives, the outcome of the VA Workshop and the risks related to the project. It is these latter considerations that have been enhanced by the merger of the Value Management and Risk Analysis process.
Risk Analysis involves not only an evaluation of risk issues and the appropriate mitigations, but also includes a quantification of risk as reflected in the project contingency. Risk is evaluated two ways: 1) in a Project Risk Assessment that discusses qualitatively such issues as political considerations, environmental impacts, economic trade-offs, market demands, revenue forecasts, construction phasing, etc. and 2) in a Construction Risk Analysis that is a more formal risk calculation process using range estimating and the Monte Carlo simulation process to determine the Project Contingency. Although many PA Project Proposals in previous years have included project risk assessments, the VA Workshops have proven to be the ideal settings for determining or validating these project risks, "brainstorming" risk mitigations, and developing the input to risk analysis computations using the collective judgment of the experts that make up the VA Team. Therefore, for most Owners, the Project Risk Assessment and Construction Risk Analysis have become integral parts of the Value Analysis process, and the multi-disciplined interactive workshop setting has become almost indispensable to an effective risk study.
The risk analysis process provides a mechanism for calculating potential cost saving or cost avoidance related to risk issues. In the management of capital programs, funding for risk is reflected in the Project Contingency. Most Owners include a Project Contingency, an amount added to a project's cost to compensate for unknown factors (e.g. delays, changes in scope, etc.) associated with the project, is usually calculated as a percentage of the combined total of construction cost, planning/engineering costs and general and administrative expenses. Until the introduction of risk analysis into the capital program Value Management Process, contingency had been traditionally determined by a less rigorous process. It would be either a mandated or "rule of thumb" method (e.g. 30% in conceptual design stage, 15% in preliminary design stage, etc.) or a "gut" call by a program executive based on his best assessment of potential overruns.
Since different kinds of projects have inherently different risks (e.g. aircraft runway construction vs. the renovation of an active airline terminal), a project contingency determined by risk analysis increases the accuracy of the Total Project Cost. Simply put, the contingency should help answer the question "What will the Total Project Cost and duration be at an 80th percentile of probability of attainments" (The 80th percentile criterion is a corporate policy determination based on a general pattern of costs over the years in which approximately 80% of all projects do not use all of their allocated contingency and, not coincidentally, also based on Pareto's Law). This is achieved by performing a Monte Carlo simulation of project line item costs (both "above the line hard and below the line soft costs") and schedule elements based on a range estimate of all key line items. The range estimating process is performed by the Value Analysis Team members in a facilitated interactive workshop environment. The result is a Project Contingency that is now calculated in a rational, analytical way by VA Team members who have been fully "sensitized" to the risk issues during the VA Job Plan steps.
Risk Analysis Terminology
To clarify the terminology and methodologies referred to in this paper the following definitions are provided as they relate to the integrated process:
Deterministic Estimating - The traditional method of estimating involved in the computation of one finite "bottom line" number made up of the arithmetic spreadsheet calculation of many "absolute" line item quantities, unit prices and other factors to determine the "right" number. Each element is based on the measurement or best judgment of the estimator whose estimate represents his/her best thinking at the time of the calculation. A deterministic estimate will invariably be different from the actual cost of the project.
Probabilistic Estimating - A method of estimating and a technique of analysis that accounts for the uncertainty or the likelihood of occurrence of the costs of a project. It is a "real world" computation of cost based on probabilities and ranges of possibilities of the outcome of a project. Probabilistic estimating provides a means for measuring uncertainty by using range estimating and Monte Carlo simulation techniques.
Range Estimating - A decision making technique in which the uncertainty of each line item of an estimate is determined by specifying the lowest and highest values that each element could assume based on an assessment of the related risks. The low end of the range is an estimate of the most optimistic outcome or a "best case" scenario. The high end of the range is an estimate of the most pessimistic outcome or "worst case" scenario. The boundaries of the range are often measured in relation to a deterministic estimate of each line item. The ranges become the input data to a Monte Carlo program which then simulates various combinations of scenarios. It should be noted that the deterministic sum of all "best case" or "worst case" scenarios is statistically impossible to the point that it is not even presented as a possible outcome.
Monte Carlo Simulation - The risk analysis methodology for providing a means to quantify project risks, managing multiple numerical uncertainties, and determining project contingency using cost and schedule range estimating data inputs. A proprietary "Monte Carlo" program (marketed by the software company Primavera Systems, Inc.), "@Risk for Microsoft Project" (sold by Palisade Corp.), the program "REFI/PC" (developed by Decision Sciences Corporation) and other similar risk analysis software packages combine generic Monte Carlo sampling and simulation techniques for project specific scheduling and construction considerations. These programs have the capability of sampling and simulating as many as thousands of likely permutations and combinations of ranged cost elements in such a way that they can plot a probabilistic distribution groups of simulations that fall within incremental cost ranges. The usual distribution of the simulation groups of samples of cost combinations can be programmed to fall into "modified" triangular or normal "bell" curve distribution. As an example of $21 Million project, approximately 2.5% of the samples were $9.6M and $21.8M, 10.5% were $20.4M and $20.9M, and 12.8% (the most) were $20.6M. The commutative probability shows a 50% (or "most likely") probability of $20.6M and an 80% probability of $21.I M. In other words, a project contingency of $500,000 would have to be added to the $20.6M "most likely" cost to obtain an 80% level of confidence so that the project cost will not exceed $20.6M. This example is an attempt to show how to interpret the output of the range estimates of the individual line item inputs to develop a contingency in a rational way, depending
on any desired level of certainty.
AN INTEGRATED VALUE/RISK MANAGEMENT PROCEDURE
The effectiveness of the integration of risk analysis (RA) with a value management program can best be explained by describing its "real-world" application to the VA Job Plan. For many Owners, the Project Risk Assessment and Construction Risk Analysis have become a regular part of the value management process and more refinements and improvements have been achieved with each successive workshop. Any agency or owner-would have to "customize" the process to fit with its unique organizational requirements. The following is a generic
step-by-step risk management procedure to identify, analyze and respond to risk factors throughout the life of a project as it has been integrated into the VA Job Plan:
STEP 1: During the Information Phase, identify risk issues and determine risk impacts on the project.
Before the VA team can understand the risk issues, it must be prepared to ask the right questions and to extract the relevant information concerning risk issues (e.g., project management, design, environmental, political, property acquisition, etc.) from the presenting Resource Team. In addition to identifying what the risk issues are, it can further evaluate the general degree of risk (low, medium, or high). A generic check list of risk issues is used by the workshop team to obtain an awareness of project risk factors and develop a list of these issues that are unique to the project being analyzed. The team is now in a position to effectively discuss and analyze risks and to develop and recommend mitigations.
STEP 2. During the Information Phase, perform a risk analysis of the designer project cost estimate.
Armed with the knowledge of risk issues and impacts, the team can then subject the original deterministic design estimate to a line-by-line risk analysis using range estimating of both the project cost and the project schedule. In preparation for this effort the estimate is formatted such that construction line items, if possible, are often broken down into labor and material costs along with an identification of construction duration. The estimates are best analyzed at a building system, "UNIFORMAT" or WBS level that are identifiable as relatively large enough unit items that are capable of being placed into a CPM precedence network.
It should be noted that this step also provides the estimator an opportunity to adjust the original design estimate (with the concurrence of the designer) for any major errors or omissions in the line item deterministic calculation. Minor errors or omissions will be accounted for in the range estimating process and a protracted "redo" of the designer's estimate is inappropriate and not necessary.
For each line item (labor, material, and duration, if possible) prepare range estimate of the "best case" and "worse case" scenarios. Then conduct a Monte Carlo simulation risk analysis of the total project cost (without the contingency) and schedule. Note that the process includes an analysis of all project line item costs, both "hard" and "soft", exclusive of the project contingency.
An output of the risk analysis is a probability curve for the original project design of both the project cost (without contingency) and the project schedule. (The risk analysis of construction, or "hard" costs, is an optional report). From this curve one can determine a recommended project contingency based on the level of certainty desired. As mentioned earlier many Owners have established a policy that the confidence should be that which provides the 80th percentile level of certainty that a project will be within the approved project cost and schedule. The difference between the original project's deterministic cost (without contingency) and the 80th percent cost probability is what the original design project contingency should have been, based on the VA Team's assessment of risks. For the purpose of comparison to the VA proposed design cost, yet to be developed, a "risk adjustment" (up or down) is made to the original deterministic project cost. This "risk adjusted" version of the original estimate will be compared later with the VA Team's estimate of its proposed alternative design, subject to the same range estimating and risk analysis procedures.
STEP 3: During the Creativity Phase "brainstorm" risk mitigations along with other idea generation efforts.
Now that the VA Team has awareness of the risk issues, as well as the impacts of risk design elements on the original proposal, it is in an excellent position to interactively generate creative ways to mitigate risks, in addition to the usually creative "brainstorming" of ideas to achieve functions for improved value. These creative risk mitigations are treated just like any other ideas that evolve during the Creativity Phase of a workshop.
STEP 4: During the Evaluation Phase consider risk as a weighted criterion.
Whether the facilitator uses a fully developed weighted matrix or a more informal system for weighing alternatives, risk should be a major consideration. Ideas which mitigate risk can often achieve cost savings. Furthermore, the risk analysis process provides the tools to quantify those savings. The advantages and disadvantages of workshop generated ideas as they relate to risk of risk mitigation can clearly be discussed in a rational way.
STEP 5: During the Development Phase conduct a risk analysis of the VA Alternative Design.
Assuming that all the ideas which the VA Team has proposed can be synthesized into a complete alternative VA Team proposal, a risk analysis can be performed on the new proposal similarly to that which was performed on the original design alternative in Step 2. The VA Team estimator need not worry if his/her deterministic quantities and unit prices are the same as those of the original designer, since all line items will be subjected to the same "best case/worst case" scenario analysis as the original design. Of course, if these workshop ideas mitigate risks, the ranges of the VA proposals would be much narrower for the same line item in the original estimate. Again, a similar probability curve is produced based on the project cost, without contingency.
Since there is no deterministic contingency produced for the VA design, the probabilistic 80th percentile project cost can immediately determine the recommended project contingency for the VA proposal (i.e. there is no need to make a "@Risk adjustment"). Thus, the total project cost of the two designs (the "risk adjusted" original and the "risk analyzed" new design) can be compared on an "apples-to-apples" basis. It is possible to develop an equitable amount of potential savings or cost avoidance by comparing the two risk analyzed proposals.
STEP 6: During the Presentation Phase, present the risk assessment with suggested mitigations, the risk analysis with recommended project costs and contingency, and the total overall potential project cost savings along with the VA Team's developed proposals.
After presenting the qualitative discussion of risk issues, a quantitative cost comparison of the original design can be presented in a very simple tabular form to quantify the Total Project Cost potential savings resulting from both the standard value analysis functional improvements and the mitigation of risks.
In the example above, the potential cost savings (or "cost avoidance") is $10.4 million for the Total Project Cost ("soft" and "hard") when comparing the original design to the VA Team's alternative design at the 80th percentile probability of attainment. A similar comparison can made of the project schedule. The identification, discussion, and mitigation of risk issues are valuable additions to the presentation to senior management in making policy decisions on critical issues, which are also discussed in a clear rational way in the risk assessment. The risk analysis clearly and objectively quantifies the recommended total project cost along with an appropriate Project Contingency.
STEP 7: During the Implementation Phase perform a second risk analysis of the final reconciled proposal.
In most Owner Value Management Programs, value analysis does not end with the delivery of the workshop report. Rather, there is Reconciliation Meeting in which the disposition of all recommendations is brought to closure. Resource Team representatives, including financial, technical, and operations staff are assembled together with the VA Team of experts to "accept", "accept with modifications", or "reject" every value analysis recommendation, risk mitigation proposal and design suggestion. The ideas and concepts not rejected become part of the final resolved project proposal for Value Board review and for Board of Commissioner's authorization. If the final, "resolved" design, consisting of only accepted and modified proposals, differs substantially form the VA Workshop alternate design proposal, there may be a need to perform another Construction Risk Analysis to determine the Project Contingency for the 80th percentile level of confidence in the accepted "reconciled" design.
|
|
|
|
