Rapid Prototyping of NASA Next Generation Sensors for the SERVIR System of Fire Detection in Mesoamerica

Transcription

1 Rapid Prototyping of NASA Next Generation Sensors for the SERVIR System of Fire Detection in Mesoamerica Joel S. Kuszmaul, Henrique Momm, Greg Easson University of Mississippi, University, MS Timothy Gubbels Science Systems and Applications, Inc., Lanham, MD Executive Summary The SERVIR Fire Rapid Response System web site displays and distributes daily images of near-real-time MODIS data processed by the MODIS Rapid Response System. The MODIS Rapid Response System was developed by Science Systems and Applications Inc. scientists at the Goddard Space Flight Center and implanted as the SERVIR extension for the use of personnel involved in environmental monitoring and disaster management in Mesoamerica. This Rapid Prototyping Capability Experiment is designed to examine the continued performance of SERVIR s fire detection system associated with the anticipated transition from MODIS to VIIRS sensor data. There is an inherent limitation in attempting this analysis prior to the commissioning of the actual VIIRS-based sensors. Our efforts in simulating the VIIRS data require us to make substitutions of sensor type and sensor resolution that cause the comparisons to be subject to this limitation. Our comparisons can, however, identify issues to be addressed in the preparation for the new sensors. We compared MODIS- to VIIRS-based fire products directly for four study dates during the 2003 Guatemala fire season. We find that there is reasonable agreement between the MODISand VIIRS-based products on the basis of direct comparison. While agreement rates are extremely high, largely due to the fact that fires are spatially uncommon, the kappa statistic was used to provide a measure of agreement that takes account of chance agreement. The calculated values of revealed good to excellent agreement beyond chance agreement. The highest values were obtained when the MODIS- and VIIRS-based assessments of high confidence fires were compared. The VIIRS-based fire products result in relatively few nominal-confidence fires and almost no low-confidence fires. The excellent agreement on high-confidence fires provides encouragement regarding the continued value of SERVIR s fire-detection program beyond the life of the MODIS sensors. The difference between the two fire product results was found to be most significant for low and nominal confidence fires, where the overall results revealed that the classifications failed goodness-of-fit tests applied to the MODIS- and VIIRS-based classification distributions. We also compared the accuracy of both the MODIS- and VIIRS-based products against fires observed in both ASTER and LANDSAT-7 imagery. The MODIS-based products compared reasonably well with the ASTER imagery, with results roughly comparable to those reported by other investigators. When the VIIRS-based products were compared to the same fires detected using ASTER imagery, distinctly poorer results were observed with lower values of the kappa statistic and higher omission error values. This indicates that the challenge of detecting small 1

2 fires, which presumably may translate to lower confidence fires, is problematic with the VIIRS data and the current VIIRS-based algorithm. Finally, the two sets of fire products were compared to fires identified by two experts working independently with LANDSAT-7 imagery from the four study days. A total of 60 fires were identified by both experts (each fire independently identified). When these fires were compared to fires detected with the MODIS- and VIIRS-based decision support tool, the results demonstrated moderate, but more importantly nearly equal, success rates for the Terra-based products. The validation fire set was compared to the results obtained using the Aqua-based products yielding distinctly poorer success rates, but those were not considered as significant given the time lag between the collection of the LANDSAT-7 imagery and the Aqua data. The overall outcome of this RPC is the demonstration that SERVIR s fire-detection system is expected to continue to perform well detecting fires that are currently being detected at a highconfidence level. Small fires or fires of low intensity do not appear to be as readily detected using the combination of the planned VIIRS sensor and the fire-detection algorithm designed for that sensor. The limitations of this experiment must also be considered. First, we used simulated VIIRS imagery where the current MODIS sensor was used to generate the pseudo-viirs dataset. Second, we did not have a confirmed set of fires detected at the surface at the time of imagery collection. Finally, we did not examine the consequences of the improved resolution associated with the expected VIIRS sensor. 1.0 Introduction This Rapid Prototyping Capability (RPC) experiment explores the feasibility of using Visible/Infrared Imager Radiometer Suite (VIIRS) data from the future National Polar-orbiting Operational Environmental Satellite System (NPOESS) to support the Rapid Response Fire Products within the SERVIR architecture. The VIIRS data is intended to replace current Moderate Resolution Imaging Spectroradiometer (MODIS) products. SERVIR is a NASA managed program integrating satellite and geospatial data with the aim to develop scientific and decision making knowledge for issues affecting Mesoamerica. The issues addressed by this program include disasters, ecosystems, biodiversity, weather, water, climate, oceans, health, agriculture and energy of the region. The Rapid Response Fire Products include Mesoamerican Web Fire Mapper and MODIS Rapid Response Fire Mapper. Both have been effectively implemented in recent years to monitor hot spots and active fires that threaten the natural resources of Mesoamerica. The products were developed as part of a collaborative effort by researchers from the University of Maryland, NASA and Center for the Humid Tropics of Latin America and the Caribbean (CATHALAC). The MODIS-based Rapid Response Fire Products of the SERVIR program were selected for the application of a NASA-funded Rapid Prototype Capability experiment. Our goal in this experiment is to compare the value of the MODIS-based tool to the planned VIIRS-based algorithm that may one day be used as a substitute fire detection system for the SERVIR program. There is an inherent limitation in attempting this analysis prior to the commissioning of the actual VIIRS-based sensors. Our efforts in simulating the VIIRS data require us to make substitutions of sensor type and sensor resolution that cause the comparisons to be subject to this 2

3 limitation. These substitutions create a limitation to our analysis that is an inherent consequence of the design of this Rapid Prototyping Experiment. In particular, the resolution differences between the proposed VIIRS-based sensor and the MODIS-based sensors used to create the simulated VIIRS data contributes an approximation to our comparison that cannot be quantified. Our comparisons can, however, identify issues to be addressed in the preparation for the new sensors. Both the similarity and differences in the MODIS- and VIIRS-based DSTs are significant. A record of good agreement between the two DSTs will provide assurance that the comparison has significance, the disagreement found can be used to identify areas where a difference between the success of the two DST may be expected. 1.1 Relevance to NASA This RPC experiment is directed at the sustained performance of a decision support tool (DST) with a well established track record of credibility and utility. This RPC experiment directly contributes to the NASA Applied Sciences Programs mission to extend the results of NASA Earth Science Division s (ESD) contribution to national priority applications by supporting the ongoing mission of SERVIR s Rapid Response Fire Products relate to National Application Areas of air quality, disaster management, ecological forecasting, and public health. Within the six National Focus Areas established by NASA ESD, this RPC contributes to an improved understanding within the categories Atmospheric Composition, Earth Surface and Interior and Weather. 1.2 Introduction to the SERVIR Fire Rapid Response System The SERVIR Fire Rapid Response System web site displays and distributes daily images of near-real-time MODIS data processed by the MODIS Rapid Response System. The MODIS Rapid Response System was developed by Science Systems and Applications Inc. (SSAI) scientists at the Goddard Space Flight Center (GSFC) based on experience gained during the wildfires in Montana in The SERVIR extension of the MODIS Rapid Response System has been developed for the use of personnel involved in environmental monitoring and disaster management in Mesoamerica. The MODIS SERVIR Fire Extent Product was developed to meet the requirements of the Guatemalan Park Service and was later extended throughout Mesoamerica. During 2003, the fire season in Guatemala was particularly severe (Figure 1), necessitating the need for greater reporting and evaluation of fire locations and extent in the sparsely populated northern part of the country. In that year, the Guatemalan Park Service did not possess the capability to effectively evaluate fire potential and location because they relied on visual and oral reports. 3

4 Figure 1. Fires in Guatemala, Honduras, and Yucatan, Mexico on March 19, 2003 as detected by MODIS visible and thermal bands. The Guatemalan Park Service requested the SERVIR project to provide MODIS fires data sets to significantly enhance their current system in providing a twice daily mapping capability that had higher temporal and spatial characteristics. The MODIS data sets were evaluated to see if they met the technical requirements of the Guatemalan Park Service. It was determined that the MODIS thermal anomalies data burned into a land cover map twice daily would enhance the country s ability to identify and evaluate fire events in a timely and effective manner. This data was also merged into the Guatemalan Park Service s existing GIS to allow them to respond and allocate their resources more efficiently to fire events. The SERVIR Fire Rapid Response System uses daily MODIS data from both the Terra and Aqua spacecrafts. The data are processed in the MODIS Rapid Response system into fire products within 3-4 hours after acquisition. Image subsets with active fire overlay are usually generated in approximately two hours. SERVIR s Mesoamerican Web Fire Mapper, which was originally developed by the University of Maryland, NASA, and CATHALAC, now serves Guatemala and the rest of Mesoamerica with on-line, interactive maps of active fires in the region. 1.3 Partners in the Interdisciplinary RPC Experiment Our partners in this RPC Experiment bring a diverse set of skills. We have partnered with experts in the science and application of the SERVIR fire detection tools. These include Dan Irwin of NASA s Marshall Space Flight Center (NASA-MSFC) and project manager for SERVIR, to provide an understanding of how SERVIR is applied. NASA-MSFC is partnered with CATHALAC to manage the SERVIR project and serve information products to users in Central America. For understanding of the current MODIS-based fire detection system and associated SERVIR fire products we have relied on Tim Gubbels, of SSAI. Tim Gubbels supplied the MODIS-based fire products used for the comparison discussed in Section 4.0. We also incorporated experts in the NASA next-generation sensors. Robert Ryan, SSAI, provided 4

5 the simulated VIIRS data that was used to create the results for the VIIRS-based fire detection system. 2.0 SERVIR and Fire Detection: the MODIS Active Fire Products for Mesoamerica The MODIS Active Fire Products for Mesoamerica are derived from reflected and emitted radiation received by the MODIS instrument from the Earth s land surface. The reflected radiation provides a representation of surface features and is conveyed by the Rapid Response Corrected Reflectance Product. The emitted radiation provides the fire detections and is conveyed by the Rapid Response Fire Detection Product. The two products are described below. Both products are produced in near-real-time within the Rapid Response system from the MODIS L0 data stream. 2.1 The Corrected Reflectance Product The imagery product for the fire detection system is based on a modified version of the MODIS Corrected Reflectance product (MOD09). The MOD09 product includes the visible and nearinfrared bands (1 to 7). The transformation from radiance to reflectance uses the ratio of the measured at-sensor radiance and the known solar irradiance at the time of the observation. The MODIS Rapid Response processing algorithm (Giglio et al., 2003) corrects for molecular (Rayleigh) scattering and gaseous absorption by water vapor and ozone and does not require real-time input or ancillary data. The algorithm uses climatological values for gas parameters. The near-real time MODIS surface reflectance product is similar to MOD09 surface reflectance in clear atmospheric conditions (based on 5S/6S radiative transfer model), but departs from MOD09 in the presence of aerosols. In order to preserve the signature of smoke plumes caused by fires, no aerosol correction is made. For ease of interpretation, the Rapid Response reflectance product is depicted as true-color imagery. This means that the colors red, green, and blue as observed by the sensor, in MODIS bands 1 (670 nm), 4 (565 nm), and 3 (479 nm), are depicted in the same colors in the resulting digital color-composite image. The advantage of this method is that it produces intuitive, natural-looking images of land surface, oceanic and atmospheric features. This imagery serves as the backdrop, or reference image, upon which the fire outlines are burned-in as red polygons. 2.2 The Fire Detection Product The fire detection strategy is based on a combination of absolute detection of fire size and intensity and on detection relative to the background, to account for variability of the ambient surface temperature and reflection by sunlight. The principal inputs to the Enhanced Contextual Detection Algorithm are the 4 and 11-micrometer brightness temperatures. Additional MODIS channels are further used to condition the data and to provide numerous tests to identify and reject typical false alarms. Cloud masking is provided by a combination of bands 31 and 32, while reflected bands 1, 2, and 7, assist in cloud masking, sun glint, bright surface, and coastal false alarm detection (Giglio et al., 2003). 5

6 Because the Terra MODIS instrument acquires data twice daily (10:30 am and 10:30 pm), as does the Aqua MODIS (2:30 pm and 2:30 am), four daily MODIS observations are available to contribute to global fire monitoring. The MODIS/Terra Thermal Anomalies/Fire 5-Min L2 Swath product, MOD14, is the most basic data in which active fires and other thermal anomalies, such as volcanoes can be identified. It is used to generate all of the higher-level fire products. The MODIS Rapid Response image products are daytime only. The sensitivity threshold is quantified as the probability of detection, which is strongly dependent upon the temperature and area of the fire being observed. The detection of any given fire is dependent on its magnitude, which is the product of its temperature and size. Hot fires as small as ~10 m in diameter can be readily detected, while cooler fires of larger sizes, such as ~30 m in diameter, are also readily detected. In any given area, the detection of a fire is related to biome type and characteristics. As a consequence, it must be noted that this RPC experiment addresses fire detection in Guatemala, and the outcomes might be different than what could be expected in another region. The fire detection algorithm used in the MOD14 algorithm in both MODAPS and Rapid Response is an improved algorithm that offers increased sensitivity to smaller, cooler fires, as well as a significantly lower false alarm rate. Performance of both the original and replacement algorithm has been evaluated using theoretical simulations and ASTER scenes (Giglio et al., 2003). In general, the new algorithm can detect fires roughly half the minimum size that could be detected with the original algorithm while having an overall false alarm rate times smaller. The algorithm performance has been measured by numerous field validation experiments (eg., Morisette et al., 2005a) 2.3 SERVIR Rapid Response Tailored Products The Corrected Reflectance true color image and the Fire Detection product are used as the primary inputs to the Fire Extent product as produced in MODIS Rapid Response. The daytime Terra and Aqua MODIS observations are used to make the morning and afternoon products, respectively. The outlines of the fires are depicted in red, superimposed on the true color image. This has the effect of showing the fire extent, clouds, and smoke. The L0 data stream upon which the products are based is intercepted at the NOAA Data server at the Goddard DAAC. The data are then ingested in the Rapid Response system and processed in the Rapid Response system into fire products within 3-4 hours after acquisition. Image subsets with active fire overlay are then generated after a short time delay (about two hours). To assist in the interpretation, the coastlines and political boundaries are fused into each map in blue and the resultant JPEG images posted to the web on the SERVIR site at Georeferencing information is stored in JPEG World Files that may be downloaded to ease the import of the data into a Geographic Information Systems (GIS). It was determined by the SERVIR project team that Mesoamerica was simply too large to convey the fire locations in adequate detail in a single frame. Six subframes were designed to 6

7 capture the region in greater detail. They are named Mexico, GuatBelSal, Honduras, Nicaragua, Costa Rica, and Panama. When a region is selected, the archive of subsets is presented chronologically as a list of image dates, with the most recent image at the top of the list. By clicking on a date, the corresponding Terra and Aqua images are displayed with annotation of calendar date as well. Project information is available by clicking the More Info button. These products have been produced since March 1, 2004 for each Mesoamerican fire season. 2.4 Related Decision Support Systems The VIIRS data products may also serve as an input to other operational decision support systems that currently use MODIS products for fire detecting. Some of such systems are Web Fire Mapper (University of Maryland based), Mexico National Commission for the Knowledge and Use of Biodiversity (CONABIO) that monitors fires in Mexico, Belize and Guatemala, National Meteorological Service of Mexico, Wild-land Fire Information System in Mexico, and Wildfire Alternatives, which is a fire-monitoring research initiative taken up at the University of Arizona. 3.0 The Ongoing NASA Contribution to SERVIR s Fire Detection Tools The ongoing contribution of NASA to SERVIR s mission in providing fire detection tools depends upon the sustained performance of MODIS sensor on both the Terra and Aqua satellites. VIIRS represents the next generation of satellites designed to combine and to replace the current MODIS Terra and Aqua missions (Yu et al., 2005) VIIRS will have significant improvements over its predecessors that will lead to new and improved products. These improvements include reduced data delivery time, improved scan geometry, and the 22 channels designed based on experience acquired from past missions (Lee et al., 2006). 3.1 Application of Current SERVIR Fire Detection Tools to the 2003 Guatemala Fire Season One of the characteristics of the design of this RPC experiment is the incorporation of data from the 2003 Guatemala fire season. We used field information provided by the SERVIR project regarding the number and location of fires that were encountered and surveyed. Scene selection constituted a vital part for the success of the experiment. Four dates were selected for investigation during the 2003 fire season: March 20, April 21, April 28, and April 30. These dates were selected according to the following criteria: Guatemala had to be entirely contained by a single MODIS scene; The cloud coverage of the selected dates (specially over Guatemala) should be as limited as possible; Guarantee of fire existence in the selected dates based on information obtained from different sources; and Availability of higher spatial resolution imagery such as LANDSAT and ASTER data to be used as validation data. Table 1 lists the dates selected and the higher resolution imagery used for our validation analyses. Our goal is to compare the results of the MODIS-based fire detection system with the simulated-viirs-based fire detection system. We achieve the goal by comparing the 7

8 consistency of classification of the two systems (Section 4.1), then by comparing the success that each of the models has compared to fires detected in the ASTER imagery listed in Table 1 (Section 4.2), and finally by examining the frequency with which fires found using the LANDSAT imagery listed in Table 1 are detected by each of the systems (Section 4.3). This last comparison provides a check on the rate of missed fires, or a check on the error of omission. We also attempted to compare the rate of falsely detected fires or a check on the error commission, but that effort was not successful for the reasons discussed below (Section 3.2). The LANDSAT coverage within Guatemala is shown in Figure 2 for each of the four study dates. These images show that inconsistent coverage of Guatemala is available and the number of confirmed fires will not be consistent from one day to another as a result. Landfires in Guatemala can change significantly within a few hours. Some span a broad area for more than a week, while others flare and are extinguished quickly. This presents a challenge when seeking to use two sets of satellite-based sensors to compare fire detection. If the time of acquisition is too great, the possibility exists that the discrepancy may be attributable in part due to changes in the fire itself. Further, the intensity of fire burns is affected by weather (including rainfall, humidity and air temperature). As a result, the time of day at which fires are detected can play a role in the position of the fires as well as the potential for success of a fire-detection system, the time of acquisition of the LANDSAT and ASTER images are included in Table 2. This information will be considered in the evaluation of the fire-detection DSTs discussed in Section 4. 8

9 Table 1 List of available LANDSAT and ASTER scenes for the selected study date Study Date March 20, 2003 April 21,2003 April 28,2003 April 30, 2003 Landsat or ASTER Scene ID Acquisition Time (Metadata) Acquisition Time (Local Time) L _ or LE ASN00 16:11:47 16:12:14 10:11:47 10:12:14 L _ or LE ASN00 16:12:11 16:12:38 10:12:11 10:12:38 L _ or LE ASN00 16:12:37 16:13:04 10:12:37 10:13:04 L _ or LE ASN00 16:12:13 16:12:40 10:12:13 10:12:40 L _ or LE ASN00 16:11:49 16:12:16 10:11:49 10:12:16 AST_L1B_ _ _4525 AST_L1B_ _ _5672 AST_L1B_ _ _4850 AST_L1B_ _ _5586 AST_L1B_ _ _5582 AST_L1B_ _ _5575 AST_L1B_ _ _5958 AST_L1B_ _ _ :41: :41: :41: :41: :41: :41: :41: :42: :41: :41: :41: :41: :41: :41: :41: :42: L _ or LE ASN00 16:18:26 16:18:53 10:18:26 10:18:53 L _ or LE ASN00 16:18:02 16:18:29 10:18:02 10:18:29 L _ or LE EDC00 16:06:04 16:06:32 10:06:04 10:06:32 L _ or LE EDC00 16:05:41 16:06:08 10:05:41 10:06:08 AST_L1B_ _ _8916 AST_L1B_ _ _8130 AST_L1B_ _ _7631 AST_L1B_ _ _7629 AST_L1B_ _ _8914 AST_L1B_ _ _8908 AST_L1B_ _ _8128 AST_L1B_ _ _8903 AST_L1B_ _ _8899 AST_L1B_ _ _ :34: :34: :34: :35: :35: :35: :35: :35: :35: :35: :34: :34: :34: :35: :35: :35: :35: :35: :35: :35:

10 Figure 2. The LANDSAT and ASTTER imagery coverage for the four dates in The LANDSAT imagery is shown for each of the four dates. The ASTER imagery coverage area is shown as a blue polygon on the two dates where imagery is available. The LANDSAT imagery, with the more extensive coverage, is to be used to form a validation data set for comparing the error of omission for both DSTs. The ASTER imagery, is used in section 4.0 for more complete validation of both DSTs. 10

11 Section 3.2 Calculating the error of commission The commission error represents the situation in which fires are detected from remotely sensed imagery when actually there is no fire in the field. The commission error is also referred to as a false positive error. The approach taken to evaluate the commission error was to identify and select areas where there were no fires during the entire 2003 fire season in northern Guatemala. Thus, assuming normal conditions, no change in biomass would occur in areas not burned before and after the fire season. The indicator of the presence and the condition of the green vegetation used was the Normalized Difference Vegetation Index (NDVI). NDVI uses a mathematical combination of red and infra-red channels which yields high values for vegetated areas, negative values for features such as clouds, water and near zero values for features such as rock and bare soil. (Lillesand and Kiefer, 2000). MODIS NDVI 16 day composites at 250 meters spatial resolution covering Guatemala prior and after the 2003 fire season were selected. The composites were chosen to reduce the cloud coverage and to use 16-day averaged NDVI values instead of individual day values. These two scenes were preprocessed to remove the negative values representing non-vegetated areas. The green vegetation situation was evaluated by computing the NDVI difference by subtracting the pre-fire NDVI from the post-fire NDVI image. Values above zero in the NDVI difference image relates to vegetation changes or has increased in the timeframe considered. However, to capture even a small decrease in the vegetation (which could be related to season changes or other climate reasons) a threshold of was used. Visual investigation of the final healthy vegetation image revealed that small scars (smaller than MODIS pixel resolution) were missed when comparing to ASTER and LANDSAT imagery. Thus, known fires in a location did not yield a significant NDVI difference. In addition, the small commission errors reported by published literature (Morisette et al., 2005a) led us to discontinue the commission error evaluation through this approach. 3.3 Plans to Utilize NASA Next-Generation Sensors The Visible Infrared Imager Radiometer Suite (VIIRS) is designed to collect visible/infrared and radiometric data to obtain measurements of Earth s oceans, land surface and atmosphere with global coverage at least once per day, and with better than 1-km spatial resolution (Welsch et al., 2001). The type of environmental data measurements to be made by VIIRS include atmospheric, clouds, earth radiation budget, clear-air land and water surfaces, sea surface temperatures, ocean color, and low light visible imagery (Lee et al., 2006). VIIRS will fly on the National Polar- Orbiting Operational Environmental Satellite System (NPOESS) in three polar sun-syncronous orbit planes (at an altitude of 833 km), with equator crossing times of 1730, 0930, and It will also fly on the NPOEESS Preparatory Project with a 1030 equator crossing time. VIIRS will have multi-channel imaging capabilities to support the acquisition of high resolution atmospheric imagery and generation of a variety of applied products including visible and infrared imaging of hurricanes and detection of fires, smoke, and atmospheric aerosols. It will acquire data in 22 spectral channels between 0.3 and 14 microns with a spatial resolution of 400 m to 800 m and 3040-km scan swath. VIIRS draws heavily on the experience gained in building and operating MODIS that also combined diverse functions into a single sensor. It also draws on the experience with various sensors with specific tasks such as the Sea-viewing Wide Field-ofview Sensor for measuring ocean color, the Along Track Scanning Radiometer for measuring sea 11

12 surface temperature, and the Operational Linescan System for terminator imaging. The difference in the spectral content in the expected VIIRS sensor and the current MODIS sensors is significant. MODIS is a 36 band spectrometer with a spatial resolution (pixel size at nadir) of 250m for channels 1 and 2 (0.6µm - 0.9µm), 500m for channels 3 through 7 (0.4µm - 2.1µm) and 1,000m for channels 8 to 36 (0.4µm µm). The MODIS instrument consists of a cross-track scan mirror that generates a simple rectangular array of pixels following the time sequence of the scan mirror rotation and the spacecraft alongtrack movement. At nadir position, a nominal 1-km pixel has dimensions of 1 x 1 km but as the scan angle increases from nadir the pixel dimensions grows to approximately 4.8-km by 2-km along track (Masuoka et al., 1998). Due to this feature, a single swath covers 10-km along track at nadir and it expands to 20-km along track creating a bowtie-shaped footprint which overlaps the swath below and above it. In simple terms a single feature on Earth may appear in several scan lines if it is located near the edge of the scene. VIIRS is designed to optimize both spatial resolution and signal to noise ratio across the scan. In this unique scanning approach, at nadir three detector footprints are aggregated to form a single VIIRS pixel. As the scan angle increases to approximately 32 degrees, the aggregation scheme is changed from 3x1 to 2x1 and at a scan angle near 48 degrees the aggregation scheme changes again from 2x1 to 1x1 (Miller and Liu, 2002). As consequence VIIRS exhibits a pixel growth actor of two along and across track. Miller and Liu (2002) also explain that the increase in spatial resolution is beneficial to the identification of active fires however the same improved spatial resolution at the edge of the images increase the changes of band saturations. The algorithm described by Giglio et al. (2003) uses brightness temperatures derived from MODIS channels at 4-µm and 11- µm to detect active fires. MODIS has two 4-µm channels: bands 21 and 22. The difference between these two bands is the temperature saturation values: band 21 saturates at 500K while band 22 saturates at 331K. Because band 22 is less noisy and has a smaller quantization error (Giglio et al., 2003) the algorithm uses this band unless it saturates then band 22 is replaced by band 21. The channel 11-µm in MODIS is represented by band 31 and it saturates at 400K for Terra MODIS and 340K for Aqua MODIS. Table 2 shows a comparison between VIIRS and MODIS spectral bands. Table 2 VIIRS spectral bands comparison to MODIS. Spectral bands used in active fire detection are highlighted in gray. Source: modified from Byerly et al. (2002) 12

13 VIIRS MODIS Spectral Bands Spectral Range (um) Nadir HDR (m) Spectral Bands Spectral Range (um) Nadir HDR (m) M M M M I M M I M M M I M M I M M M M I M The SERVIR Fire Detection System Using NASA Next-Generation Sensors The simulated VIIRS data was produced by our SSAI-Stennis partners led by Robert Ryan. They used the MODIS datasets from both Terra and Aqua sensors for the four study dates to generate a simulated VIIRS or proxy VIIRS dataset. Data was simulated for five of the moderate-resolution VIIRS bands needed for the fire detection algorithm; the red and NIR bands, SWIR (2.25 μm) band, MWIR (4.05 μm) band, and LWIR ( μm) bands. As these bands are very similar spectrally to the MODIS bands used in the algorithm, a spectral band matching technique was used for the simulation. In this technique, the most similar MODIS band is used create each band of the VIIRS proxy data. The change in ground sample distance (GSD) from 1- km MODIS data to km VIIRS data was not considered, with the exception of the noise model described below. Corrections to account for the increased noise predicted for the VIIRS sensor were applied to the MWIR and LWIR bands. The noise correction is based on the NEDTs (noise equivalent delta temperatures) of MODIS and VIIRS for both extended and point source targets. Because the GSD of VIIRS is smaller than that of MODIS, one noise correction model could not account for all different types of targets. Thus, best case and worst case noise models were developed to bound the simulations. The extended targets, which are larger than a pixel and for which the improvement in GSD has no effect, are represented with the worst case noise model. Point source targets, which have a higher signal to noise ratio due to the improved GSD, are represented with the best case noise model. The relation between the NEDT of the two systems for an extended target is given in Equation (1) and for a point source in Equation (2). 13

14 (1) where, = VIIRS NEDT = MODIS NEDT = Difference between MODIS and VIIRS NEDTs (2) where, = VIIRS ground sample distance = MODIS ground sample distance From each of these equations, the value was found and used to scale a matrix of normally distributed random numbers the size of the original image. This scaled matrix was added to MODIS data which was converted to brightness temperature to produce the simulated VIIRS data set with either point source or extended target noise. Noise corrections were not applied to the red, NIR, and SWIR bands, which were simulated solely via spectral band matching. For the fire detection algorithm, these bands are used to detect water, clouds, and sun glint, and differences in sensor noise was determined to have little or no effect on algorithm output. The SSAI recommended (Kara Holekamp and Robert Ryan, personal communication, May, 2007) error model was the point source model. Two independent VIIRS simulations were performed by SSAI-Stennis. The first one produced simulated VIIIRS data in grid format. This data was then used as input to a MODIS fire algorithm modified for VIIRS thresholds and to accept gridded data instead of swath format (MODIS original format). The second simulation was performed after considerable modifications to the simulation algorithm to accept as input MODIS in swath format and produce as output simulated VIIRS also in swath format. This allowed pixel-by-pixel comparisons since there were no resampling operations involved. The simulated VIIRS in swath format was a requirement of the VIIRS fire detection algorithm. The goal of this RPC Experiment was to compare the effect of the change in sensor on the SERVIR fire detection DST. Our goal was not to make any modifications to the DST itself, but simply to assess how the DST would perform with simulated VIIRS data instead of MODIS data. It is important to note, that the scientists working in cooperation with SERVIR have already modified the DST in anticipation of the expected VIIRS sensor characteristics. The VIIRS fire detection algorithm (NASA Goddard Space Flight Center, electronic communication, May, 2007) used in this project is very similar in design to the MODIS fire detection algorithm. However, there are some changes in the saturation thresholds to meet VIIRS requirements. VIIRS is currently designed to have only one channel at 4-µm: band M13. Band M13 saturates at 634K while M15 (VIIRS equivalent to MODIS band 31) saturates at 343K (Miller and Liu, 2002). This saturation difference and other algorithm differences make the application of the new DST and the current DST genuinely different, and the comparison of the results is considered in the next section. 14

15 4.1 Comparing MODIS- and VIIRS-based Detection Tools To compare the results obtained from the MODIS- and VIIRS-based fire detection tools, we applied both algorithms to the data from each of the four days for both sensors. Thus, a total of 8 MODIS fire products (one for Terra and Aqua on each of the four study days) were compared with the 16 simulated VIIRS fire products (with two simulated VIIRS products for every one MODIS product due to the two different errors models). As an example of how the MODIS product can be compared to the simulated VIIRS product, consider the results for March 20, 2003 using the Terra satellite data. The results can be compared using an error matrix (also known as the confusion matrix) as shown in Table 3. This matrix reveals the number of pixels in agreement or disagreement for each of four pixel classifications. For each of the pixels where both algorithms make a classification, the results are compared class by class. In these models, fires are classed as high-confidence fires (Class 9), nominal-confidence fires (Class 8) and low-confidence fires (Class 7). Pixels classed as no fire are assigned to Class 5. In the error matrix of Table 3, agreement between the algorithms results in pixels being recorded along the diagonal of the matrix. Pixels in disagreement are shown off the diagonal. One measure of agreement that can be applied in this type of case is the overall frequency of pixels in agreement, which is a measure of the fraction of total pixels in which both algorithms made the same classification. In the case shown in Table 3, the agreement rate is or %. While these results appear to indicate extremely close agreement, high agreement rates for the prediction of rare events such as fires is to be expected. Much of the agreement stems from the fact that most of both fire products show agreement in that most of Guatemala is not on fire. Instead of comparing the MODIS- and VIIRS-based products on the basis of agreement rate, we seek a more complete measure of the extent to which the two DSTs are or are not achieving the similar fire detection results. A more complete comparison of the data sets is possible using the kappa statistic. The kappa statistic, first proposed by Cohen (1960), is a variable designed to account for the rate of chance agreement. The kappa statistic varies between -1 and +1. For two classifications that agree merely by chance, a kappa value of 0 would result. Classifications with agreement poorer than chance would result in a negative kappa value, while agreement better than chance would result in a positive kappa value. Landis and Koch (1977) proposed the use of kappa values greater than 0.75 to represent excellent agreement beyond chance, kappa values between 0.4 and 0.75 to be fair to good agreement beyond chance, and kappa values between 0 and 0.4 to be poor agreement beyond chance. To calculate the kappa value in this and all other comparisons of MODIS and simulated VIIRS fire products, we used the ArcView extension supplied by the USGS (Jenness and Wynne, 2005). This module calculates the sample s kappa statistic,, with other supporting statistics. For the March 20, 2005 Terra case shown in Table 3, the calculated kappa statistic is , which reveals good agreement beyond chance between the models. Table 3. The error matrix result comparing the MODIS and simulated VIIRS fire products for March 20, 2003, using the Terra server data and the extended error model for the simulated VIIRS data. VIIRS MODIS

16 We applied the Jenness and Wynne analysis tool (2005) to all comparisons to yield the results shown in Figure 3, where values (along with the confidence interval about each value) are shown for all cases. These cases include the four study dates, both Terra and Aqua sensors, and both the extended and point errors models applied to the simulated VIIRS data. One observation that can readily be made is that the type of error model applied to the simulated VIIRS data does not significantly affect the results. The SSAI recommended (Kara Holekamp, communication, May 11, 2007) error model was the point source model as the most representative of the expected level of error in the VIIRS sensor s Figure 3. The overall kappa calculations (with point estimate of and the associated confidence interval) from the Jenness and Wynne tool (2005).Demonstrates that the choice of the point or extended error models applied to the VIIRS data is not significant. Using only the point model of error, the associated estimates and confidence intervals are shown in Figure 4, where the results from the two sensors on each of the four dates are illustrated. One observation that can be made of the results shown in Figure 4 is that there seems to be a fairly consistent size of the confidence interval of in each case. In addition, the values are reasonably consistent from day to day and sensor to sensor, with the exception of April 28 Terra and April 30 Terra values. The April 30 Terra value may differ from other results reported in part due to the fact that this analysis does not fully cover all of Guatemala. For the April 28 Terra products, there are distinctly lower estimates of. No obvious explanation for this poorer performance has been identified, but one possible explanation is that more fires were detected on April 28, 2003 than on any other day. Whether the poor agreement results from the presence of many fires on that date, or whether the large number of detected fires is a symptom 16

17 of the challenges to accuracy of the algorithm cannot be known. However, the association between many detected fires and lower associated estimates seems more than a coincidence Figure 4. Results from the overall kappa calculations (with point estimate of and the associated confidence interval) from the Jenness and Wynne tool (2005) for the case of the point source error model. Only comparisons involving the point source error model are considered. Another way to compare the results of the fire detection algorithms is by the efficacy of their fire detection. The current MODIS detection algorithm produces three classes of fires: highconfidence fires (class 9), nominal-confidence fires (class 8), and low-confidence fires (class 7). In the current MODIS fire products, all three fire classes are grouped together and assigned the class of fire. We could also compare the efficacy in detecting fires as defined by current practice, by grouping pixels classed as 7, 8 and 9 into a new combined fire class, to compare fire versus no fire. It is important to note that many pixels are not assigned a classification for a variety of reasons such as excessive clouds, the presence of a water body, and other reasons that emerge in the application of the algorithm (Giglio et al., 2003). The previous kappa values discussed were based upon similarity in class of fire. For example, a pixel declared to be a class 7 fire (low-confidence fire) and a class 8 fire (nominal-confidence fire) would have been considered an error. If we group classes 7, 8 and 9 into a single fire class, the new kappa value should show improved success (as it would not distinguish between classes of fires). While we made this comparison, we also chose to make two additional comparisons: comparing fires detected at the nominal confidence level and above and comparing fires detected 17

18 at the high-confidence level only. In these last two comparisons, we simply disregarded the locations found to have possible fires. The result of the comparison is shown in Table 4. Table 4. The overall results comparison for the ability to detect fires using the four different definitions of fires. The upper left case considers the differences between classifications on a class-by-class basis. The other cases restrict the definition of a fire as discussed in the text, with the bottom right comparing the ability to detect high-confidence fires. Overall Low-confidence and higher DAY TERRA AQUA DAY TERRA AQUA March 20, March 20, April 21, April 21, April 28, April 28, April 30, April 30, Nominal-confidence and higher High-confidence DAY TERRA AQUA DAY TERRA AQUA March 20, March 20, April 21, April 21, April 28, April 28, April 30, April 30, The upper left result, labeled Overall, considers each class distinctly so that a MODIS ranking of nominal-confidence fire and a VIIRS ranking of a high-confidence fire would be recorded as an error. If however, the distinction between different confidence levels in the fire classes (classes 7, 8 and 9 in the algorithm result) are ignored, a higher success rate results. This is the assumption in the upper right result shown in Table 4. If low-confidence fire classes are ignored and nominal- and high-confidence fire classes are grouped together, the result is shown in the bottom left portion of Table 4. While this comparison shows slightly improved results, there is little impact in ignoring low-confidence fire results because so few pixels are assigned this classification. If only high-confidence fires are considered (with low- and nominal-confidence fires being ignored), the result is shown in the bottom right portion of Table 4. The very high values of for the case of high-confidence fire detection suggests that the ability to assess highconfidence fire should remain unchanged with the introduction of the VIIRS sensor. Because the simulated VIIRS data was generated using MODIS data as a significant input, the difference in the 4-µm channels saturation temperatures (discussed in Section 3.3) is not fully examined in this analysis. Nonetheless, this aspect of the two algorithms appears to be extremely consistent across the sensors. We sought to examine this band-saturation issue more closely. We examined the extent to which difference levels of fire confidence were occurring in the MODIS- and VIIRS-based products, presented in Table 5. We discovered a significant difference with respect to low and nominal confidence fires. The VIIRS-based product yields nominal confidence fires at a rate of 20% less than the MODIS-based products. Further, the VIIRS-based products did not yield any low confidence fires for the data considered in this RPC. To further examine this difference the two TERRA-based results were compared using a Chi-Squared goodness-of-fit test and found to be significantly different (P < ). The two AQUA-based results were compared to find similar results. In each case, the largest contribution to the chi-squared statistic came from the 18

19 disagreement attributable to the nominal-confidence fires (class 8). This reveals a distinct difference between the MODIS and VIIRS-based fire products. Table 5. A comparison of the overall frequency of classes 5, 7, 8, and 9 across the four study dates for each of the data types. Comparison of Overall Classification Frequency Classification MODIS:Terra VIIRS:Terra MODIS:Aqua VIIRS:Aqua No Fire (5) Low Conf Fire (7) Nominal Conf Fire (8) High Conf Fire (9) Comparing Detection Tools Using Validation Data Sets: Results from the ASTER Imagery The MODIS fire product and the simulated VIIRS fire product were evaluated by comparison to ASTER (Advanced Spaceborne Thermal Emission and Reflection Radiometer) derived fire maps following the procedure described by Morisette et al. (2005a). In this procedure ASTER bands 3N and 8 were converted from calibrated radiance to top-of-atmosphere reflectance and band 3N was aggregated to 30 meters nominal spatial resolution to match the spatial resolution of band 8. Eleven scenes were used: four scenes of April 30, 2003 covering the western part of Guatemala and seven scenes of April 21, 2003 covering a north-south strip over Guatemala. The algorithm described by Morisette et al. (2005a) and others starts by creating two intermediate images: ratio and difference of band 8 and band 3N. The algorithm performs two passes over each pixel. In the first pass a simple threshold operation is performed over the ration and difference image. Pixels that exceed these thresholds are flagged as obvious fires and therefore excluded from the second pass. In the second pass the neighboring pixels are considered (61x61 pixel square window centered in each pixel) and another sequence of thresholds is applied. The final map is a binary image containing fire pixels either flagged as active fires or non-fires. The final active map was then visually inspected to make sure that the no false fire pixels were present. The VIIRS and MODIS fire products were then evaluated by using the ASTER fire map as reference. The VIIRS and MODIS fire products pixels had to meet two criteria: (1) the pixel must lie within the spatial areas represented by the ASTER footprint; and (2) the pixel value must be 5, 7, 8, or 9 to be considered in the evaluation process. For each pixel meeting these criteria, the number of ASTER fire pixels was then counted and values calculated for the associated statistic, the omission error rate, and the commission error rate for different ASTER threshold values, where the threshold value is the minimum number of fire pixels to consider fire. The results in Figure 5 indicate that an optimal threshold could be obtained with a number of ASTER fire pixels between 10 and 20. This method of assessing detection success is consistent with results described by Morisette et al. (2005a), except that we have added the statistics and the concept of an optimum threshold value. 19

20 MODIS - TERRA April, April, VIIRS - TERRA April, April, Kappa Statistic pp ASTER Threshold ASTER Threshold Probability (Omission Error) Probability (Commission Error) ASTER Threshold ASTER Threshold y( ) ASTER Threshold Figure 5 Summary of the Kappa statistic, Omission Error Probability, and Commission Error Probability values for different ASTER thresholds 4.2 Comparing Detection Tools Using Validation Data Sets: Results from the LANDSAT Imagery To provide a further basis of comparison for of the success rate of the fire detection systems, we used the available LANDSAT imagery introduced in Section 3.1. Two different experts familiar with the task of classifying imagery were asked to identify wildfires in the LANDSAT imagery. Efforts to collect reliable groundtruth information (i.e., date, location, and area of active fire sites) from Guatemala did not yield promising results. Hence it was decided to create ASTER Threshold 20

21 groundtruth data by visually interpreting LANDSAT ETM and ASTER data that were collected for the same dates previously considered: 20 March, 21 April, 28 April, and 30 April. The LANDSAT data were imported and geometrically corrected using ERDAS Imagine. The false color composite (FCC) (B1-Blue, B2-Green, and B7-Red) for each date was loaded onto ArcMap and clipped along the boundary line of Guatemala. Each FCC was examined closely from left to right and top to bottom to look for active fires. An active fire was identified based on its association with smoke that was visible in the FCC. Figure 6 presents examples of active fires. Figure 6. Examples of LANDSAT false color composite images showing active fires in Guatemala. Yellow color in the left image is a polygon representing an active fire. Polygon data of fire polygons were generated for all of the four dates using ArcMap. Finally, the area and the central coordinates of the polygons were determined using Arc tools. When active wildfires that both experts identified were compared, confirmed fires were defined as those cases where both fire fighters identified fires at a location. These confirmed fires were then compared to the MODIS- and simulated-viirs-based fire products. A total of 60 fires were identified by both of independent expert classifiers. We then compared the validation data set of 60 fires to the classified pixels from the SERVIR DST results. For each fire location listed in the field data it should be assessed as to whether a fire was detected in the imagery or not and also the size of the detected fire. Because the real conditions of fire (or no fire) can clearly influence sensors readings in adjacent pixels (Townshend et al., 2000) we proposed that pixels be considered using a 5-by-5 pixel window centered at the location of interest, as shown in Figure 7. 21

22 Figure 7 5-by-5 MODIS pixels window centered at each field fire location If a recorded location of a confirmed fire (marked with * in Figure 7) falls within the 5-by-5 grid it would be considered a true positive. In other words for each confirmed fire identified, the pixel information of the 25 nearest pixels are recorded. Where any of the 25 pixels have values of either 7, 8, or 9 a finding or agreement is indicated. Figure 8 shows an example of the proposed approach. The success rate of both of the DSTs were applied to the Aqua and Terra cumulative results for the four dates to produce the success rates shown in Table 6. Figure 8 - Fire location and the 25 nearest MODIS pixels collected to investigate agreement with field data Table 6. Comparison of fires identified in LANDSAT imagery and fires detected by the MODIS- and VIIRS-based DST. All Fires March 20, 2003 April 21, 2003 April 28, 2003 April 30, 2003 Overall Accuracy Sensor No. Found Fires No. Found Fires No. Found Fires No. Found Fires No. Found Fires (%) MODIS:Aqua % VIIRS:Aqua % MODIS:Terra % VIIRS:Terra % The results demonstrate inconsistent success across the four dates. Further, the success rate for data from the Aqua sensor appears to be significantly poorer than the Terra server. It must be noted, however, that the LANDSAT imagery was collected closer in time to the Terra-based sensor than the Aqua-based sensor. Both the MODIS-based data and the LANDSAT data are collected during morning hours (NASA, 2007), but the AQUA-based data is collected hours later 22