Uncertainty in Fire Standards

Transcription

1 Hall Journal of Testing and Evaluation Selected Technical Papers STP 1541 JTE Uncertainty in Fire Standards and What to do About It ISBN: Stock #: STP1541 STP Uncertainty in Fire Standards and What to do About It John R. Hall, Jr. JTE Guest Editor

2 Journal of Testing and Evaluation Selected Technical Papers STP1541 Uncertainty in Fire Standards and What to Do About It JTE Guest Editor: John R. Hall, Jr. ASTM International 100 Barr Harbor Drive PO Box C700 West Conshohocken, PA Printed in the U.S.A. ASTM Stock #: STP1541

3 Library of Congress Cataloging-in-Publication Data ISBN: Copyright 2012 ASTM INTERNATIONAL, West Conshohocken, PA. All rights reserved. This material may not be reproduced or copied, in whole or in part, in any printed, mechanical, electronic, film, or other distribution and storage media, without the written consent of the publisher. Journal of Testing and Evaluation (JTE) Scope This flagship ASTM journal is a multi-disciplinary forum for the applied sciences and engineering. Published bimonthly, JOTE presents new technical information, derived from field and laboratory testing, on the performance, quantitative characterization, and evaluation of materials. Papers present new methods and data along with critical evaluations; report users experience with test methods and results of interlaboratory testing and analysis; and stimulate new ideas in the fields of testing and evaluation. Major topic areas are fatigue and fracture, mechanical testing, and fire testing. Also publishes review articles, technical notes, research briefs and commentary. All papers are peer-reviewed. Photocopy Rights Authorization to photocopy items for internal, personal, or educational classroom use, or the internal, personal, or educational classroom use of specific clients, is granted by ASTM International provided that the appropriate fee is paid to ASTM International, 100 Barr Harbor Drive, P.O. Box C700, West Conshohocken, PA , Tel: ; online: The Society is not responsible, as a body, for the statements and opinions expressed in this publication. ASTM International does not endorse any products represented in this publication. Peer Review Policy Each paper published in this volume was evaluated by two peer reviewers and at least one editor. The authors addressed all of the reviewers comments to the satisfaction of both the technical editor(s) and the ASTM International Committee on Publications. The quality of the papers in this publication reflects not only the obvious efforts of the authors and the technical editor(s), but also the work of the peer reviewers. In keeping with long-standing publication practices, ASTM International maintains the anonymity of the peer reviewers. The ASTM International Committee on Publications acknowledges with appreciation their dedication and contribution of time and effort on behalf of ASTM International. Citation of Papers When citing papers from this publication, the appropriate citation includes the paper authors, paper title, J. ASTM Intl., volume and number, Paper doi, ASTM International, West Conshohocken, PA, Paper, year listed in the footnote of the paper. A citation is provided as a footnote on page one of each paper. Printed in Bay Shore, NY February, 2012

4 Foreword THIS COMPILATION OF THE Journal of Testing and Evaluation, (JTE), STP 1541, Uncertainty in Fire Standards and What to Do About It, contains only the papers published in JTE that were presented at a symposium in Anaheim, CA on June 16, 2011 and sponsored by ASTM Committee E05 on Fire Standards. The Symposium Chairman and Guest Editor is John R. Hall, Jr., Division Director of Fire Analysis and Research, National Fire Protection Association, Quincy, MA, USA.

5

6 Contents Overview... vii Who Gets the Benefit of the Doubt from Uncertainty? J. R. Hall, Jr Measurement Uncertainty in Fire Tests A Fire Laboratory Point of View J. O. Trevino and R. Curkeet Bench Tests for Characterizing the Thermophysical Properties of Type X Special Fire Resistant Gypsum Board Exposed to Fire P. H. Shipp and Q. Yu Measurement Uncertainty and Statistical Process Control for the Steiner Tunnel J. V. Resing, P. D. Gandhi, D. E. Sloan, and R. K. Laymon Precision of the Cone Calorimeter and ICAL Test Methods J. Urbas Uncertainty in Fire Protection Engineering Design M. J. Hurley Fire Pattern Repeatability: A Study in Uncertainty D. Madrzykowski and C. Fleischmann In Search of Standard Reference Materials for ASTM E05 Fire Standards N. J. Alvares and H. K. Hasegawa What Have We Learned About Uncertainty? Are We Still Playing with Fire? N. Keltner Heat Flux Measurements and Their Uncertainty in a Large-Scale Fire Test C. S. Lam and E. J. Weckman Development of a Proposed ASTM Guide to Continued Applicability of Reports on Fire Test Standards T. T. Earl and M. M. Hirschler

7

8 Overview When it comes to measuring product fire performance in standard tests or standard calculations, uncertainty is the elephant in the room. We honor the need for uncertainty measurement and interpretation in the abstract and set uncertainty-related requirements for every standard (such as the precision and bias requirements). When it comes to actually calculating uncertainty and incorporating uncertainty into our use of test and calculation results, however, we often fall short. We act as if the elephant really isn t there, because we don t know what to do about it if it is there. On July 16, 2011, ASTM Committee E05 on Fire Standards conducted an all-day symposium with 15 papers on the subject of uncertainty in fire standards and what to do about it. The objective of the symposium was to discuss different issues related to uncertainty in fire standards and to cover how different parties testing laboratories, enforcement authorities, manufacturers, practicing engineers incorporate uncertainty into their use of results from fire safety tests and calculations. The symposium was also designed to look at larger implications of different approaches and provide overviews of some of the newest methods and approaches for handling uncertainty. An effort has been made to post all 15 presentations at the E05 website for a limited time at The first four presentations provided a basic familiarity with existing methods and procedures and with relevant ASTM and other standards. Because these presentations were not designed to provide new information only to lay a solid foundation for the later presentations they were not converted into published papers. This STP contains papers based on the other eleven presentations. Because you, the reader, may not have access to those first four presentations, this Overview will provide a brief description of the contents of those presentations as well as places to go for more information. William Guthrie of NIST led off with his presentation on Assessing Uncertainty in Measurement Results: The Big Picture. He began by linking the need for uncertainty to situations where the threshold for acceptable product fire performance lies within the uncertainty range around the point estimate or single-value measurement of that performance. He identified the major factors that contribute to test uncertainty, including variations in the sample, the test method, the test environment, and the calibration of the instruments. vii

9 The analyst needs to select a statistical approach that will accurately describe how uncertainty in each of these factors combines (or propagates) to produce combined uncertainty in the final measurement. He discussed both frequentist and Bayesian approaches and provided example calculations for both. He introduced the audience to ISO s GUM standard, which is the shorthand name for ISO JCGM (Joint Committee for Guides in Metrology) 100, Evaluation of Measurement Data Guide to the Expression of Uncertainty in Measurement, an essential document for frequentist calculations of uncertainty, which is the approach used with nearly all uncertainty calculations. The GUM is accessible at org/utils/common/documents/jcgm/jcgm_100_2008_e.pdf. For Bayesian analysis, he referred the audience to D.J. Lunn, A. Thomas, N. Best, and D. Spiegelhalter, WinBUGS A Bayesian Modelling Framework: Concepts, Structure and Extensibility, Statistics and Computing, volume 10 (2000), pp Marc Janssens of Southwest Research Institute followed with two presentations Relevance of ASTM 2536 in Fire Safety Engineering Analysis and Precision and Uncertainty of Fire Tests What s the Difference? He described an example application using the cone calorimeter to develop input data for use in a fire dynamics model, either CFD or zone. ASTM E2536, Standard Guide for Assessment of Measurement Uncertainty in Fire Tests, is the principal ASTM reference for such an exercise. ASTM E2536 fully addresses measurement uncertainty but only partially (if at all) addresses uncertainty associated with the test specimen or the test procedure. Picking up on Guthrie s key step of selecting an appropriate statistical approach, Janssens illustrated the complex calculations required to estimate uncertainty more comprehensively in this example case. He noted that seemingly small changes in the test conditions can have dramatic effects on the test results. In his second presentation, he explained the difference between uncertainty, which measures the magnitude of errors associated with a value, and precision, which focuses on variations between or within laboratories in repeated applications of a specified test method to a specified material. He then illustrated the calculation of both measures for a very simple example, which was the application of ASTM E691 to the total burning viii

10 time for 50 ml of 91% IPA alcohol in an empty tuna can taken from a specific brand and type of tuna. Hershal Brewer of the International Accreditation Service provided the last of the presentations on basics in Measurement Uncertainty for Fire Test Laboratories in the Accredited Environment Under ISO/IEC 17025:2005. ISO/IEC 17025, General Requirements for the Competence of Testing and Calibration Laboratories, is the standard that governs laboratory accreditation. Clause of ISO/IEC requires laboratories to have and apply procedures to estimate uncertainty of measurement. ISO/IEC refers users to the GUM for methods to discharge its requirements. Brewer also cited ANSI/NCSL Z , which is the U.S. edition of the GUM, and NIST Technical Note He then walked through the application of these references to ASTM E84 tests. The first paper in the STP is based on the fifth and final presentation in the introductory section of the symposium. John Hall s paper Who Gets the Benefit of the Doubt from Uncertainty? focuses less on the calculation and more on the framing and interpretation of uncertainty information, including the points in the decision-making process where imbalances in knowledge or in access can introduce biases in the decisions. The next four papers in the STP were presented in the Applications to Specific ASTM Fire Tests section of the symposium. Measurement Uncertainty in Fire Tests A Fire Laboratory Point of View, by Javier Trevino and Rick Curkeet, provides a perspective on the way that measurement uncertainty rules are applied, simplified and sometimes declared non-applicable in practice. Bench Tests for Characterizing the Thermophysical Properties of Type X Special Fire Resistant Gypsum Board Exposed to Fire, by Paul Shipp and Qiang Yu, is a detailed description of research conducted to address difficulties in estimating precision for ASTM E119, ASTM s most used standard fire test, relative to a specific product. Measurement Uncertainty and Statistical Process Control for the Steiner Tunnel (UL 723, ASTM E84), by John Resing and colleagues at Underwriters Laboratories, examines uncertainty measurement issues for ASTM E84, ASTM s second most used standard fire test. Precision of the Cone Calorimeter and ICAL Test Methods, by Joe Urbas, examines uncertainty measurement issues for two of the relatively newer ASTM fire test methods, including the cone calorimeter, ix

11 which is probably the most used source for input data for fire modeling and calculation. The next two papers in the STP were presented in the Uncertainty in Engineering Calculations section of the symposium. Uncertainty in Fire Protection Engineering Design, by Morgan Hurley, walks through the uses of uncertainty in fire modeling and other fire protection engineering calculations. He references Uncertainty, Chapter 5-4 in the 4 th edition of the SFPE Handbook of Fire Protection Engineering. He also references the SFPE Engineering Guide G , Guidelines for Substantiating a Fire Model for a Given Application, and ASTM E1355, Predictive Capability of Fire Models. Both guides are about verification, validation, and assessment of firerelated models. Fire Pattern Repeatability: A Study in Uncertainty, by Daniel Madrzykowski and Charles Fleischman, begins with a review of uncertainties in calorimeter tests of various materials, then switches to a discussion of the change in uncertainties when test results are used instead as input data for computer model analysis and estimation. The final four papers in the STP were presented in the New Methods and Other Issues section of the symposium. In Search of Standard Reference Materials for ASTM E05 Fire Standards, by Norman Alvares and Harry Hasegawa, reviews the history of work in identifying, accessing and using standard reference materials to calibrate test instruments and test operators, providing better measurement and control of measurement uncertainty. What Have We Learned About Uncertainty? Are We Still Playing with Fire?, by Ned Keltner, reviews a wide range of issues in thermal measurement, along with some candidate solutions and strategies to improve performance. Heat Flux Measurements and Their Uncertainty in a Large-Scale Fire Test, by Cecilia Lam and Elizabeth Weckman, focuses on the contribution of heat flux gages to uncertainty in heat flux measurements. Development of a Proposed ASTM Guide to Continued Applicability of Reports on Fire Test Standards, by Marcelo Hirschler and Timothy Earl, describes ongoing work on a proposed new ASTM guide that would translate uncertainty and variation considerations into guidance on the continued use of, or the need for new, fire test reports. Also worthy of note is the one poster at the symposium, which was Parametric Uncertainty of Moisture Content Ignition Time Relationship for Cellulosic Materials, by Joseph Kahan and M. Ziefman of FM Global. x

12 The intent of the symposium was to provide something useful for every member of the audience, regardless of their level of experience or their work responsibilities. For newer professionals, the symposium s front half was designed to give them a quick exposure to the basics and leads to major guidance documents and references that would allow them to be fully capable practitioners. For experienced professionals, the symposium s second half was designed to provide an overview of current activities designed to improve practice and identify the leaders of those efforts, so that interested audience members could become part of the change. The intent of this STP is the same helping the newer practitioners to build their skills quickly and helping the leading-edge researchers to attract additional minds and energy to their important work. If we are successful, we may soon be able to acknowledge the elephant in the room without flinching and set about bringing it under our control. Many thanks are herein offered to all the speakers who made up this outstanding program; to the attendees, whose good questions and comments added value to the program; and to the several members of ASTM staff who made the symposium and this STP possible. It was a pleasure and a privilege to be part of this team. John R. Hall, Jr. Division Director Fire Analysis and Research National Fire Protection Association Quincy, MA, USA xi

13 Hall Journal of Testing and Evaluation Selected Technical Papers STP 1541 JTE Uncertainty in Fire Standards and What to do About It ISBN: Stock #: STP1541 STP Uncertainty in Fire Standards and What to do About It John R. Hall, Jr. JTE Guest Editor