A Building Fire Risk Validation Project was carried out by Fire Underwriters Survey/Opta Information Intelligence in partnership with Vancouver Fire Rescue Service and New Westminster Fire Rescue Service in British Columbia.
Photo Credit: Fire Underwriters SurveyIn Canada, similar to the United States, fire codes are enforced through fire prevention inspections conducted by fire department personnel. In certain provinces, changes are happening with fire-related legislation where a municipality will be tasked with completing a risk-based approach to providing fire-protection services.
What is the best approach to completing a risk assessment for prioritizing fire prevention inspections?
Consider an approach to a fire-risk assessment. The buildings to identify are those at most risk of fire.
Generally, an experienced firefighter can tell which buildings pose a higher risk of fire. Factors such as building age, building occupancy, type of business, previous violations and incidents, location, demographics and socio-economics, would all be expected to contribute.
Most often, these features can be found in various municipal datasets. The problem is that the municipal datasets are not readily accessible, and the data is rarely in a useable format. Other datasets related to demographics and socio-economics may be in datasets outside of the municipality such as census data. The issue becomes one of data aggregation.
Once all the data is aggregated the second issue relates to how each feature should be weighted to arrive at a final risk score.
The models we built with VFRS and NWFRS were informed by recent work in predictive modeling of building fire risk completed in New York City, the City of Atlanta and the City of Pittsburgh. In all three U.S. cities, a data-driven, risk-based approach proved to be better at targeting risks than previous practice.
We wanted to validate the results to building fire risk assessment by completing similar risk-assessment-based prediction work for two municipalities in Canada. We invited two cities to participate based on population size. Could a data-driven, building-based risk assessment better prioritize inspections in both these municipalities?
Both the VFRS and NWFRS projects took the following approach: data extraction, data cleaning and merging, feature engineering and modeling, and results.
- Data extraction: The team met with both VFRS and NWFRS staff in May 2018 to discuss the datasets and other available building level information. The datasets included fire incident data, fire inspection data, property assessment data, 2016 census data (demographic and socio-economic), parcel data, 311 data (including sanitation data), crime data, as well as other available property data.
- Data cleaning and joining: As can be expected, address data from different municipal departments can be kept in various formats. Also, some datasets have block-level aggregation (or a higher level in the case of census data). We built a geo-coding service and used GIS to join the data.
- Feature engineering and modeling: Both city models used data for the period 2013-2017. The data was initially split into a training dataset (2014-2016) and a test dataset (2017). The target was to predict fire incidents (IncidentType = FIRE – Fire Department Responded Reportable – 1000) for 2017. Model parameters were tuned, and features selected by splitting the training set and using the 2016 fire incidents for validations. The final model was generated using the best parameters and the full training set. Finally, the model was tested against the 2017 fire incident data.
- Results: Seventy per cent of fires were accurately predicted for 2017 for the City of Vancouver (with a false positive rate of 25.19 per cent). Seventy per cent of fires were accurately predicted for 2017 for the City of New Westminster (with a false positive rate of 24.45 per cent). The high prediction rates offset the low false prediction rates. Even though approximately 25 per cent of high-risk classified buildings for 2017 did not have a fire, the model still classifies these as risky buildings and should therefore be prioritized for an inspection; these false positives may still have a fire.
The influential features and weights differ for both cities.
The model provided a map of property fire predictions in inspectable properties for 2017 and where the property had a fire in 2017. The City of Vancouver has a grouping of fires and predictions in certain areas. In the City of New Westminster, the actual fires are more dispersed. There are good predictions in both cases.
Using the model results, we assign a risk score to each building in both cities and then re-order the inspections for 2017, targeting the highest risk first.
Under the current VFRS inspection program, the first 25 per cent of inspections had visited 20 per cent of buildings that had a fire in 2017. By using a risk-based approach and re-prioritizing inspections, the first 25 per cent of inspections would theoretically visit 65 per cent of inspectable buildings that had a fire incident in 2017.
As the model was trained on fire events, we expect an increase in this value. There was an increase in discovering severe violations from 15 to 47 per cent in the first 25 per cent of inspections. This result also shows a good correlation between fire events and severe violations.
A similar impact is seen for the City of New Westminster. A subset list of severe violations was not available for the New Westminster data. However, looking at all violations the increase is from 18 to 34 per cent.
Overall, implementing a fire-risk prediction/assessment approach shows that, for the year 2017, more violations (and severe violations) would have been identified earlier in the inspection year. This is similar to the impact seen by NYFD using its risk-based inspection system.
This report further validates building fire prediction methods in a comparison study of two municipalities/fire departments in Canada. We would recommend this approach to any sizable municipality looking to create a risk-based approach and drive insights from municipal data in order to better target inspections.
We demonstrated in the report how machine learning can contribute to improving the provision of public fire services using existing municipal data and data exchange processes.
The FUS team is now looking forward to the next stage and implementing live models.
Robert McGuinness, P.Eng., PMSFPE, is technical director at the Fire Underwriters Survey in Burnaby, B.C., a national organization that provides data on public fire protection for fire insurance statistical work and underwriting processes. A summary of the recent survey can be found at www.fireunderwriters.ca. Contact Robert at robert.mcguinness@scm.ca.
