Operating Committee Meeting Presentations

Transcription

1 Operating Committee Meeting Presentations June 5-6, 2018 New Orleans, LA *All presentations are posted with the written consent of the presenters.

2 EAS Status Update Rich Hydzik, EAS Chair NERC OC Meeting June 5, 2018

3 Lessons Learned Hurricane Irma presentation from Glenn Dooley - Duke Three Lessons Learned published since last OC meeting LL Back Office EMS Support Tools Impact Real-Time Situational Awareness o Reliability Coordinator (RC) contacted entity about a particular contingency o Entity identified that half of the contingency lists in the EMS list were disabled o Back-office study user had opened and modified the contingency list control display o Inadvertently impacted the contingencies being displayed in the real-time environment LL Loss of Communication to Multiple SCADA RTU s at a Switching Center o Grid Operations lost communication with 87 RTUs for 78 minutes o Primary station service power source was taken out of service for maintenance o UPS general alarm that was acknowledged by operators but not investigated o 5.5 hours later UPS batteries failed o UPS auto/manual restart switch had been left in the factory default setting: manual tripping AC breaker 2 RELIABILITY ACCOUNTABILITY

4 Lessons Learned LL External Model Data causing State Estimator to Not Converge o Several entities have experienced state estimator (SE) outages involving Inter-Control Center Communications Protocol (ICCP) data received from neighboring entities o Topology of the neighboring system had changed but the receiving entities had not updated their representation o ICCP data no longer matched with the older model, and the SE was unable to converge Details on the Lessons Learned have been included in OC materials EAS has contacted Mark Child (CIPC Chair) to establish a CIPC resource Lessons Learned development as needed TOP R20 group o Network SME o CIP SME 3 RELIABILITY ACCOUNTABILITY

5 Ongoing Activities OC Assignment to review TOP R20 and R21 as requested to clarify redundant and diversely routed language as well as testing requirements Group has formed membership, developed scope, and is investigating issue Indications that Regional Entities are not entirely consistent in guidance Ryan Mauldin (NERC Compliance) is working with group Group is evaluating whether they will pursue a guideline or guidance Review of Reliability Guideline: Generating Unit Operations During Complete Loss of Communications EAS is planning to approve changes and forward this to OC by June 30 EMSWG Met last week in Atlanta to develop October Conference Materials Extended to one and one-half days based on attendee feedback Focus is Evolution of EMS Systems o New standards like TOP-001/TOP-010 o Highlight new tools/innovations regarding voltage stability analysis, alarm tools, etc. o Include Telecomm vendors 4 RELIABILITY ACCOUNTABILITY

6 Activities From Yesterday s Meeting Event Analysis Process Review Categories for Inverter Disturbances and DC Asynchronous Tie Events EAP Changes to Category 1.h? o Change 30 minutes to 45 minutes? o What does the data say? o Will we miss something? Near Miss Category? o Question from PAS (Paul Kure) EAS Scope Document Revisions SPP RE changes Review scope overall 5 RELIABILITY ACCOUNTABILITY

7 Rich Hydzik 6 RELIABILITY ACCOUNTABILITY

8 Lessons Learned Summary Update to EAS Richard Hackman Event Analysis Subcommittee Meeting June 4, 2018

9 NERC Lessons Learned Three new NERC Lessons Learned (LL) published in June 2018: LL Back Office EMS Support Tools Impact Real-Time Situational Awareness LL Loss of Communication to Multiple SCADA RTU s at a Switching Center LL External Model Data causing State Estimator to not converge 2 RELIABILITY ACCOUNTABILITY

10 LL Back Office EMS Support Tools Impact Real-Time Situational Awareness Description Upon receiving a call from its Reliability Coordinator (RC) about a particular contingency, a registered entity identified that half of the contingency lists in the energy management system (EMS) list were disabled. This was due to a back-office study where the contingency list control display was opened and modified, inadvertently impacting the contingencies being displayed in the real-time environment. 3 RELIABILITY ACCOUNTABILITY

11 LL Back Office EMS Support Tools Impact Real-Time Situational Awareness Corrective actions Because the lack of application separation for study case vs realtime made the need to maintain the real-time case locked absolute, and this condition was not well understood, the registered entity trained its operations support staff regarding this issue. The EMS support staff have been instructed not to release control of the real-time case when they complete their work as changes can inadvertently be made to real-time case when no one has control of real-time case. 4 RELIABILITY ACCOUNTABILITY

12 LL Back Office EMS Support Tools Impact Real-Time Situational Awareness Lessons Learned Several entities have had similar events where a study tool or simulation tool has impacted the real-time EMS environment. The tools often share functionality and data with the real-time EMS tool to closely mimic real-time conditions, and are used to test out new features and technology; apply patches and fixes; run near-term studies looking for thermal, voltage, or stability concerns; and train the operators on interesting scenarios, including some stress-cases. Because these tools look and feel the same and often share some common data links, there is a risk that the real-time EMS environment can be compromised by changes to the test, study, or simulation systems, or the persons making the changes may think they are in a non-real-time environment when really he or she is making changes to the production environment. 5 RELIABILITY ACCOUNTABILITY

13 LL Back Office EMS Support Tools Impact Real-Time Situational Awareness The following controls may help prevent, detect, and correct this risk: Build a separate QA or testing environment for back-office study. Discuss any vulnerabilities with the EMS vendor. What changes made to the test, study, or simulation tools could have consequences in the realtime environment? When possible, use different visual effects, such as different colored borders or backgrounds, to help differentiate between real-time EMS environment and test, study, or simulation environments. Train operations support personnel and transmission System Operators on the risks of any setting changes or contingency list updates made to any of these environments. Ensure opening & closing breakers in the simulation tool is not capable of opening & closing them in the production server. 6 RELIABILITY ACCOUNTABILITY

14 LL Back Office EMS Support Tools Impact Real-Time Situational Awareness Provide guidelines on who is authorized to work in which environment & which study clones are assigned to which operators or support personnel. Ensure there is no overlap where two or more persons could be working on the same study in real-time apps. Work with internal EMS IT expertise or the vendor to design/assign application/environment permissions (read, write, modify) to support personnel and transmission System Operators. Continue to work with RCs and TOPs who have overlapping coverage of the entity s system. If observations of the RC or the TOP differ from the registered entity s, investigate whether the differences are the result of inadvertent disabling of tools or features. When performing start-up testing of a new EMS app, monitor other EMS tools & apps to ensure the new app is not impacting other systems. 7 RELIABILITY ACCOUNTABILITY

15 LL Loss of Communication to Multiple SCADA RTU s at a Switching Center Description Grid Operations lost communication with 87 remote terminal units (RTUs) when conditions allowed a pre-existing configuration error to express itself. The various substations affected had operating voltages ranging from 4 kv to 500 kv. This resulted in loss SCADA functionality. The total event duration was 78 minutes. A regional switching center that contained the EMS platform for the area s RTU communication was relocated to a newly constructed control room 2 months prior to the event. A transformer that supplied the primary station service power source was taken out of service for maintenance. This created a UPS general alarm that was acknowledged by operators. 8 RELIABILITY ACCOUNTABILITY

16 LL Loss of Communication to Multiple SCADA RTU s at a Switching Center Due to distractions with other scheduled switching that was also taking place, the cause of the UPS general alarm was not investigated. 5.5 hours later communication with 87 RTUs was lost when the UPS batteries were unable to maintain proper voltage. A technician was sent to determine the cause of the power failure and found the main circuit breaker to the UPS tripped. When installed, the UPS auto/manual restart switch had been left in the factory default setting: manual. In the manual configuration, the UPS system is designed to trip the main circuit breaker for a momentary loss of ac power, and go on battery power as experienced during the event. 9 RELIABILITY ACCOUNTABILITY

17 LL Loss of Communication to Multiple SCADA RTU s at a Switching Center Corrective Actions The UPS system auto/manual restart switch at this facility was placed in the auto position. Future installations of similar UPS systems have been flagged to ensure correct auto/manual restart switch position selection prior to releasing equipment for regular service. Mandatory operator response to alarms of this nature have been reinforced, including communication requirements. Training was conducted with operations personnel about the importance of the on-site UPS facilities and their impact on area RTUs. Contacts for when UPS facility issues are encountered have been specifically identified and posted. 10 RELIABILITY ACCOUNTABILITY

18 LL Loss of Communication to Multiple SCADA RTU s at a Switching Center Lessons Learned Alarm False Sense of Security Assure that each alarm adequately describes the equipment or system operating status. Some alarm conditions have no immediate undesired consequence. This may lead to a false sense of security if the alarm does not adequately describe the equipment or system operating status. The RTU equipment continued to operate normally until the UPS battery ran out of power. It is likely that if the alarm actually stated that the UPS was operating on battery power that different actions would have been taken. Properly categorize all alarms so that prioritized and proper action can be taken based on the significance of their cause. 11 RELIABILITY ACCOUNTABILITY

19 LL Loss of Communication to Multiple SCADA RTU s at a Switching Center Commissioning When new equipment is commissioned & testing is performed by a contractor, the entity should ensure that the contractor s startup checklist includes all necessary settings, including switch positions. Otherwise, a latent error may set up conditions for a later failure. When an Operating Event Is Assessed and Corrected Examine the extent of the condition look beyond the particular adverse event & inspect other areas of your system for similar issues. Periodically review processes for integrating new equipment into the system. Work to remove ambiguity in alarm conditions for new equipment. 12 RELIABILITY ACCOUNTABILITY

20 LL Loss of Communication to Multiple SCADA RTU s at a Switching Center When Planning Work that Removes an Important Power Supply The work planner should know what loads are on that supply (what could be lost?), which ones are critical loads (what absolutely needs to be kept powered?), & how those critical loads will be supplied during the configuration change. If there is a critical load or UPS that should automatically switchover to another source, include steps in the work instructions to verify that before turning off the normal source, the alternate source is available. Then when the normal power is removed, check to ensure the critical load or UPS does transfer over. If it does not do so automatically, then do it manually and then check that the alternate source is indeed powering it. If there is a maintain positive control policy to not trust auto transfers, then transfer to the other source manually before removing normal power. 13 RELIABILITY ACCOUNTABILITY

21 LL External Model Data causing State Estimator to Not Converge Description Several entities have experienced state estimator (SE) outages involving Inter-Control Center Communications Protocol (ICCP) data received from neighboring entities. Upon investigation, the topology of the neighboring system had changed but the receiving entities had not updated their representation so the ICCP data no longer matched with the older model, and the SE was unable to converge (or solve). 14 RELIABILITY ACCOUNTABILITY

22 LL External Model Data causing State Estimator to Not Converge Corrective Actions Upon determination that data points or external model were incorrect, there were several actions that allowed the SE to converge (not all methods were used by every entity nor in all cases): Removing the data points from the model to allow the SE to converge around the area, hopefully producing a quality solution. Phone calls were made to the RC or neighboring entities to assess the situation. It is possible that RTU or other communications-type maintenance may be the reason for the external data point to be invalid (or bad ) even though its quality shows as good; however, often it is due to some type of external topology change outside of the entity s footprint. Discuss topology with neighbors so the model can be adjusted as necessary. 15 RELIABILITY ACCOUNTABILITY

23 LL External Model Data causing State Estimator to Not Converge Some entities have the ability to freeze external data (either globally or for a specific area) and revert to the last known good solution. This allows the SE to converge while operations support personnel troubleshoot the external model and incoming data. Data bounds and limits were placed where possible on external data so as to throw away any unreasonable data points (e.g., hundreds of MVARs on a subtransmission transformer). Applying lower weighting to external measurements was used in some cases (not all EMS allows that method). Increase the number of measurements in unsolved areas of the model. 16 RELIABILITY ACCOUNTABILITY

24 LL External Model Data causing State Estimator to Not Converge Lessons Learned RCs & TOPs have expanded their EMS models to monitor impact of events & outages outside their footprints. This has increased potential exposure to bad data points, inaccurate topology modeling, & communication issues which may cause SE nonconvergence. Changes to the BES (incl. new substations, other new or removed facilities) need to be communicated to neighboring entities well in advance so they can update their external EMS models & ensure data from ICCP is accurately matched to appropriate data points in the model. Models can be shared between the RC and its TOPs using standardized formats such as the Common Information Model. This information can be communicated to all entities that include that area in their external model. 17 RELIABILITY ACCOUNTABILITY

25 LL External Model Data causing State Estimator to Not Converge Consider standardizing on model names for facilities. Due to different EMS vendors and platforms, different data points may be called different cryptic names (often truncated due to space limitations). Standardization of these model names could help reduce confusion and lead to fewer SE outages. External models should have sufficient detail (topology and SCADA) or should have very little SCADA. Explicitly modeling without sufficient SCADA can lead to solution errors. Automatically importing external (topology and SCADA) data while convenient can lead to problems, especially when troubleshooting unsolved areas. Ensure that imports contain all the types of data and default assumptions expected by your system (normally open switches, market model deficiencies). 18 RELIABILITY ACCOUNTABILITY

26 Lessons Learned Metrics (incl. early June 2018) Total FRCC MRO NERC NPCC RF SERC SPP TRE WECC Total RELIABILITY ACCOUNTABILITY

27 On the NERC website, Go to > Click on the Program Areas & Departments tab and click Reliability Risk Management Then on the left side menu under Event Analysis click Lessons Learned Ways to Access Lessons Learned 20 Lower left corner of NERC website homepage RELIABILITY ACCOUNTABILITY

28 21 RELIABILITY ACCOUNTABILITY

29 Reliability Assessment Subcommittee Status Report Tim Fryfogle, RAS Chair Planning Committee Meeting June 5-6, 2018

30 Outline Summary 2018 Summer Reliability Assessment 2018 Long Term Reliability Assessment Probabilistic Assessment Working Group Schedule Essential Reliability Services Measure 6 Parameter Update 2 RELIABILITY ACCOUNTABILITY

31 2018 Summer Reliability Assessment 2018 Summer Reliability Assessment Most assessment areas demonstrate resource adequacy by maintaining sufficient Anticipated Resources to meet and exceed their planning Reference Margin Levels for this summer. The Anticipated Reserve Margin for ERCOT falls to 10.9 percent, below their Reference Margin Level of percent for this summer. During extreme weather, there is an increasing risk of operational issues when reserve margins are tight. If forecasted summer conditions materialize, ERCOT may need to rely on emergency operating procedures (EOPs) and import capabilities from neighboring Areas. 3 RELIABILITY ACCOUNTABILITY

32 2018 Long-Term Reliability Assessment 4 RELIABILITY ACCOUNTABILITY

33 PAWG Work Plan Technical Reference Report on Probabilistic Adequacy and Measures PC vote June 2018 Core ProbA High priority in 2018 RAS Approved Scenario Base Case Integration into LTRA RAS full report review by November 2018 PC Approval 4Q, 2018 Data Collection Approaches and Recommendations Report Moderate priority in RELIABILITY ACCOUNTABILITY

34 2018 ProbA Schedule 6 RELIABILITY ACCOUNTABILITY

35 Schedule July Toronto, Canada LTRA Peer Review August Valley Forge, PJM Review ProbA dashboards Review LTRA WRA kickoff November San Antonio, Texas ProbA comprehensive report 7 RELIABILITY ACCOUNTABILITY

36 ERS Measure 6 Essential Reliability Services Measure 6 Sub team finalized o Data Forms o Narrative Questions Kick off call for volunteers of the pilot study o May 30 o ERCOT, PJM, MISO, CAISO, Ontario, and ISO-New England 8 RELIABILITY ACCOUNTABILITY

37 9 RELIABILITY ACCOUNTABILITY

38

39

40

41 Hurricane Irma

42 Topics Overview of Irma Impact Preparation Operations and Restoration Lessons Learned FRCC NERC Irma Analysis Report coming soon 2

43 Irma Facts Florida Landfalls on September 10, 2017 Florida Keys Marco Island 1 of 5 Hurricanes with Maximum Sustained Winds of 185mph Maintained >185mph for 37 Hours

44

45 Preparation 5

46 FRCC Hurricane Plan Daily Calls BA/TOP Reporting to RC TM Status Gen Status Planned Shutdowns Coordination with NG Pipelines Post Storm Calls Damage Tie Lines Customers Out Facilities Out (Lines, Subs, Generation) Restoration This information relayed to NERC and other agencies through FRCC RE representative 6

47

48

49 9

50

51

52

53 Magnitude

54

55 Loss of Load FRCC 64% decline in energy usage NG facilities performed well no hindrance to electric operations 15

56

57 Transmission Line Outages 9/10 9/11 9/12 9/13 9/14 9/15 500kV kV kV kV Total

58 Additional Preparations Communications with Neighbors Additional Staffing Generation Planned Shutdowns and Prep Mutual Aid Assistance Efforts Largest MAA effort ever in Florida >6M Customers without Power Over 6.5M Mandatory Evacuations 18

59 Operations and Restoration 19

60 Operations & Restoration 500-kV lines (1) 230-kV lines (48) 138-kV lines (48) 115-kV lines (21) BES substations (12*) *full or partial outages 20

61 Number of BES Transmission Lines Transmission Restoration 120 Transmission Line Outages /10/2017 9/11/2017 9/12/2017 9/13/2017 9/14/2017 9/15/ kV 230kV 138kV 115kV Total 21

62 Number of Customers (Millions) Customers Out Customers Out

63 IRMA Power Outages

64 Water Levels Area Maximum Water Levels in Feet Caribbean 11 FL Keys 8 SW FL - Unpopulated 10 SW FL - Populated 5 FL East Coast 6 FL Central West Coast 2 Georgia and South Carolina 5 [1] Some areas of the Florida Central West Coast had a low water level of 7 feet below ground level. 24

65 25

66 26

67 27

68 28

69

70

71 Lessons Learned Pre-staging of equipment streamlined restoration Preemptively removing generation protected equipment from damage which shortened restoration Continuous communications between the RC, TOPs and BAs in the FRCC Region Advanced meters and intelligent grid devices were effective to pinpoint outages and operate equipment remotely 31

72 Lessons Learned Flood monitors in substations located within the 100 year flood plain allowed utilities to de-energize substations at notification of rising water Leveraging social media provided customers with the most current outage and restoration information 32

73 Lessons Learned Aerial drones helped assess damage, evaluate work conditions, and enable real-time situational awareness Infrared capabilities helped identify equipment that needed further inspection Hardening and resiliency programs implemented prior to the hurricane significantly reduced the storm damage due to high winds and storm surge 33

74 Glenn Dooley Hassan Hamdar

75 ALARM MANAGEMENT DISTILLED How a simple tool can change everything.

76 Insulator String

77 Insulator String

78 Serene picture

79 Dramatic picture of the damage!

80

81 BRANDON HARRINGTON Associated Electric Cooperative Springfield, MO

82 Operator Out Then a picture of an Operator?

83 Picture of Operator at console with arms raised in surprise Then an arrow points to the alarms, with a *ping* Or, zoom into the screen IN THE CONTROL ROOM

84 Alarms Scrolling

85 System Operators Plant Operators Network Operators Smooth Operators

86 Flash forward Video of same event with current alarm configuration

87 A Tool Tips

88 First, educate yourself

89 The seven steps 1. Define Philosophy 2. Benchmark System 3. Resolve Bad Actors 4. Document and Rationalize 5. Audit and Enforce 6. Real-Time Management 7. Control and Maintain

90

91 It is a spreadsheet!

92 Excel is a Swiss Army Knife

93 4 sections Gray Green Blue Orange Collect Catorganize Rationalize Configure

94 4 sections Gray Green Blue Orange Collect Catorganize Rationalize Configure

95 Collect COLLECT

96 Collect First Step: Export Points!!! Dump SCADA points database into your spreadsheet SCADA key Point Description Alarm Text Station Configuration Data Trigger, etc

97 Collect Second Step: Create a Benchmarking Workbench Create a second sheet and dump your alarm history. We ll need this next.

98 Collect Look for Patterns Pick a time frame to benchmark such as three months

99 Collect Pattern: Frequent alarms

100 Collect Pattern: Frequent alarms The Vital Few: Obviously Broken

101 Collect 1/2/2016 1/2/2016 0:20 1/2/2016 0:40 1/2/2016 1:00 1/2/2016 1:20 1/2/2016 1:40 1/2/2016 2:00 1/2/2016 2:20 1/2/2016 2:40 1/2/2016 3:00 1/2/2016 3:20 1/2/2016 3:40 1/2/2016 4:00 1/2/2016 4:20 1/2/2016 4:40 1/2/2016 5:00 1/2/2016 5:20 1/2/2016 5:40 1/2/2016 6:00 1/2/2016 6:20 1/2/2016 6:40 1/2/2016 7:00 1/2/2016 7:20 1/2/2016 7:40 1/2/2016 8:00 1/2/2016 8:20 1/2/2016 8:40 1/2/2016 9:00 1/2/2016 9:20 1/2/2016 9:40 1/2/ :00 1/2/ :20 1/2/ :40 1/2/ :00 1/2/ :20 1/2/ :40 1/2/ :00 1/2/ :20 1/2/ :40 1/2/ :00 1/2/ :20 1/2/ :40 1/2/ :00 1/2/ :20 1/2/ :40 1/2/ :00 1/2/ :20 1/2/ :40 1/2/ :00 1/2/ :20 1/2/ :40 1/2/ :00 1/2/ :20 1/2/ :40 1/2/ :00 1/2/ :20 1/2/ :40 1/2/ :00 1/2/ :20 1/2/ :40 1/2/ :00 1/2/ :20 1/2/ :40 1/2/ :00 1/2/ :20 1/2/ :40 1/2/ :00 1/2/ :20 1/2/ :40 1/2/ :00 1/2/ :20 1/2/ :40 Pattern: Alarm floods This ain t good

102 Collect Operator alarm response 1. Read it 2. Acknowledge it 3. Navigate to the right screen 4. Decide if its even real 5. Determine what to do 6. Make it happen 7. Watch and wait At least a few minutes

103 Collect How many is too many? 1 alarm every 10 minutes is ideal 1 alarm every 5 minutes is manageable No more than 300 alarms per day

104 Collect Recapitulation Logged in spreadsheet: Created master alarm database with Data points that are generating alarms Any useful metadata History and benchmark Created benchmark workbench with Used Excel to find patterns Found the Vital Few Learned how deep the rabbit hole goes

105 Catorganize CATORGANIZE

106 Catorganize catorganize -verb cat or ga nize \ kat ˈȯr-gə-ˌnīz \ : to place into categories and organize

107 Catorganize CATORGANIZE

108 Catorganize Categorize and Organize Categorize all points into buckets of similar points The Vital Few: 80% of your points fall to 20% of your buckets

109 Catorganize Category Examples MVA line loading Voltage limits Tie line MW shift Circuit Breakers and Switches Load tap changers EMS system Comm. and health XFMR and CB Alarms Relaying and protection Plant cycle Duplicates and points to disable

110 Collect Catorganize No time to go solo Find your experts Seek out other stakeholders

111 Collect Catorganize Recapitulation Logged to spreadsheet: Contact Person or team Categories All of em. Notes

112 Collect Rationalize RATIONALIZE

113 Collect Rationalize Determine Two Things: Should an alarm exist? How important is an alarm?

114 Collect Rationalize Alarm Textbook Definition An audible and/or visible means of indicating to the operator an abnormal condition requiring a response. (ISA 18.2)

115 Collect Rationalize Should an alarm exist? An audible and/or visible means of indicating to the operator an abnormal condition requiring a response. (ISA 18.2)

116 Collect Rationalize To the spreadsheet! Cause of the abnormal condition Action the Operator should take

117 Collect Rationalize How important is it? Priority Why Focus on what needs to be done next How The operator s ability to quickly tell things apart

118 Collect Rationalize We re bad at this.

119 Collect Rationalize 3 things is simple.

120 Collect Rationalize Color! Woah!

121 Collect Rationalize For some reason we do this.

122 Collect Rationalize Priority 3 levels

123 Collect Rationalize Priority 3 levels 80% High Medium Low 15% 5% Low Medium High

124 Collect Rationalize

125 Collect Rationalize

126 Collect Rationalize Priority Make Color Count High Medium 80% Low 15% 5% Low Medium High

127 Collect Rationalize Severity How Severe? Priority Consequence High Medium Low Time to Respond How long before it happens?

128 Collect Rationalize Severity Minor Major Severe

129 Collect Rationalize Severity Area of Impact Minor Major Severe Compliance Operation Generation Loss Gen Impact to Grid Cost

130 Collect Rationalize Severity Area of Impact Minor Major Severe Compliance File report Report and Fines Operation Warning Over/under voltage Continued Violation Line overloading Generation Loss <2% Capacity In between Loss of Unit Gen Impact to Grid Fixed by Auto Gen Control Re-dispatching or purchase Potential of brown/blackout Cost <$10k <=$100k >$100k

131 Collect Rationalize Severity Minor Major Severe

132 Collect Rationalize Severity + Time Minor Major Severe 30 minutes minutes minutes 3 minutes < 3 minutes Low Low Medium Low Medium High Medium Medium High

133 Collect Rationalize Severity + Time = Priority Minor Major Severe 30 minutes Low Low 10 minutes Low < 3 minutes

134 Collect Rationalize Recapitulation Logged to spreadsheet: Cause Action Consequence Severity (Minor, Major, Severe) Time (3, 10, 30) Priority (L, M, H)

135 Collect Configurize CONFIGURIZE

136 Collect Configure CONFIGURE

137 Collect Configure Translate Convert rationalize data into your specific system settings: Alarm Group and Classes Priority and Color Sound Operating Guide or Procedure Deadbanding

138 Collect FEP OpenVIEW OpenNERC HIGH MTR N2_3 A4_TNAN A4_N1AA T2_NCAA R3_1 ICCP S2_10 V2_10 N1_3 AL_NCAA Gen_Con T1_TNAN T2_1 T1_NCAA A4_TNB AL_10 N3_3 T1_N_A AL A4_N0AA UNIT T1_TNB T1_1 ALERT2 AL_TNAN AL_1 T2_TNAN 1/2/2016 1/2/2016 0:40 1/2/2016 1:20 1/2/2016 2:00 1/2/2016 2:40 1/2/2016 3:20 1/2/2016 4:00 1/2/2016 4:40 1/2/2016 5:20 1/2/2016 6:00 1/2/2016 6:40 1/2/2016 7:20 1/2/2016 8:00 1/2/2016 8:40 1/2/2016 9:20 1/2/ :00 1/2/ :40 1/2/ :20 1/2/ :00 1/2/ :40 1/2/ :20 1/2/ :00 1/2/ :40 1/2/ :20 1/2/ :00 1/2/ :40 1/2/ :20 1/2/ :00 1/2/ :40 1/2/ :20 1/2/ :00 1/2/ :40 1/2/ :20 1/2/ :00 1/2/ :40 1/2/ :20 Configure Deadbanding Slow these down!

139 Collect Configure Deadbanding A little goes a long way. 1-2% on the value 20 second delay or less Wait to call it Normal Alarms immediately Prevents chatter

140 Collect Configure Recapitulation Logged to spreadsheet: Configuration Alarm Groups and Classes Color, Priority, Sound, Deadbands

141 All done right?! Gray Green Blue Orange Collect Catorganize Rationalize Configure

142 One Last Thing

143

144

145 Goodbye 1. It s not so hard. 2. You can have my stuff.

146 Brandon Harrington QUESTIONS?

147 Community Confidentiality Candor Commitment NATF Update NERC OC Meeting June 5-6, 2018 Ken Keels NATF Director, Practices and Initiatives Open Distribution Copyright 2018 North American Transmission Forum. Not for sale or commercial use. All rights reserved.

148 Topics Development and Publication of Practices and Reference Documents Update on Supplemental Operating Strategies (Spare Tire) Changing Resources Mix Activities Open Distribution 2

149 Development and Publication of Practices and Reference Documents Open Distribution 3

150 Recently published practices and reference documents Risk and Controls Matrix Examples for Transmission Operations Governance: Impacting Effective Oversight within Your Organization How to Design and Develop a Risk Assessment, Internal Controls, and Compliance Metrics Program Distributed Energy Resource (DER) Modeling and Study Practice Reference Document (Public version coming soon!) Vegetation Management Clearances Vegetation Management Managing Off-Right of Way Hazard Trees Open Distribution 4

151 Practices and reference documents nearing completion Control Center Design, Construction and Operations Environment Practices Contingency Analysis Performance Monitoring Practices Risk Assessment, Internal Controls, and Compliance Peer Team Effectiveness Training SF6 Breaker Insulation Testing Lines Inspection Cyber Supply Chain Risk Management Guidance White Paper CIP Compliance Implementation Guidance Open Distribution 5

152 Practices and reference documents under development (related to system operations) Human Performance Observation and Coaching Practices Conservative Operations and Emergency Tools Practices Situational Awareness Key Elements White Paper Situational Awareness Tools Practices Systematic Approach to Training Open Distribution 6

153 Supplemental Operating Strategies (Spare Tire) Open Distribution 7

154 Spare Tire Report Recommendations 1. Develop alternative voice and data communications 2. Develop additional reliability tools/data availability for situational awareness 3. Establish formal strategies and plans for Spare Tire operations scenarios 4. Formalize data sharing on Spare Tire operations strategies 5. Harden EMS hardware components and develop streamlined EMS recovery processes Open Distribution 8

155 Project Team: Formalize Knowledge Sharing Compile individual NATF member "Spare Tire" and Conservative System Operations plans Compile various industry "Spare Tire" type activities (i.e., EPRI, EIDSN, NERC PRASE, DOE, etc.) Schedule targeted assistance visits to several companies to review Conservative Operations plans and "Spare Tire" plans GridEx IV Share learnings from related member exercises and drills Identify, document and share best practices and best ideas with all NATF members. Open Distribution

156 Project Team: Establish Formal Strategies and Plans Develop Conservative Operations Reference Document Conservative Operations and Emergency Tools Practices Develop a "Spare Tire" plan Reference Document Hold webinar/workshop to share these best practices Collaborate with EPRI on Spare Tire Project: Assessment of Extreme Contingencies Affecting EMS- Control Center Functionalities Open Distribution

157 Conservative Operations and Emergency Tools Practices Identify Conditions to Enable and Disable Roles and Responsibilities Internal Communications Common Conservative Operations Strategies Emergency Communications Load Management Open Distribution

158 NATF Changing Resource Mix Activities Key topic area covering several practice groups System Operations Modeling and Planning System Protection Resiliency Presentations at workshops and web meetings Example: Joint NATF-EPRI-NERC-UVIG webinar series Not yet a formal practice area or initiative

159 Questions? Open Distribution 13