The City of Edmonton developed advanced AI software to predict whether home builders will pass low-risk inspections.
The City of Edmonton developed advanced AI software to predict whether home builders will pass low-risk inspections. This software enables the city to automatically pass inspections for builders with good track records. We'll catch up with Ben Gready and Juan Monterrosa to discuss how this software frees up staff resources to focus on more complicated inspections that pose a higher risk to public safety and shortens timelines for builders.
Read more about Edmonton's AI framework and data-driven solutions.
More info:
Featured government: Edmonton, AB
Episode guests: Ben Gready, Data Scientist at the City of Edmonton, and Juan Monterrosa, Director of Codes and Inspections at the City of Edmonton
Visit govlaunch.com for more stories and examples of local government innovation.
Lindsay (00:05):
Welcome to the Govlaunch podcast. Govlaunch is the wiki for local government innovation and on this podcast, we're sharing the stories of local government innovators and their efforts to build smarter governments. I'm Lindsay Pica-Alfano, co-founder of Govlaunch and your host.
In this episode, we're excited to be talking about artificial intelligence. More and more local governments are turning to this technology as a way to boost efficiency of services rendered to the public.
Today, Olivia from our team is talking to Ben Gready and Juan Monterrosa from Edmonton in the Alberta province of Canada. They were part of a team that created advanced AI software to predict whether home builders will pass low risk inspections. Their data driven work has allowed much of the inspection process to be streamlined, saving valuable time and money. Now let's dive into the concept and find out how Edmonton started integrating AI into their frontline services.
Olivia (01:04):
Hi, I'm Olivia from Govlaunch and I'm here with Ben Gready and Juan Monterrosa. So how are things going in Edmonton's analytics center of excellence today and just more broadly in this sort of data science piece at the city of Edmonton?
Ben (01:19):
Sure. Thanks for the question. I think I should preface this with just about everything that's happening right now for every organization, which is just the impact that COVID-19 has had. It's been an unprecedented challenge to the city of Edmonton and our kind of area, data and analytics and AI have been affected. We've had staff laid off just like almost every area of the city. So that's the current background that a lot of people are also experiencing. But I would say we are actually busier than we've ever been. So our area is working to use data and analytics to help the city in the response to COVID-19. So things like using data and analytics to help redeploy staff to different business areas, staff that have been temporarily laid off, looking at neighborhoods around the city and seeing which ones are more at risk for COVID-19 based upon kind of demographic information. And we suspect that going forward, data and analytics and AI are going to be more important than they were before when we think about re-imagining the city of Edmonton. And I think it's going to be a common theme for many municipalities.
Olivia (02:48):
So what's interesting is that your team and actually Edmonton overall, although the context right now, isn't great. It is worth mentioning that you are gaining recognition for the work in the realm of AI specifically. So how did Edmonton actually begin integrating AI to improve its services at the city?
Ben (03:06):
Yeah, thanks. So there's obviously a huge potential for applying AI and data science in municipal government. These kind of AI models are, you know, in the category of advanced analytics. And if you kind of had a spectrum from basic data reporting all the way through to putting out some artificial intelligence algorithm that's going to help with some form of decision making, It’s way on the one end of the spectrum, but it requires a foundation to be able to sort of get to there. So some of the foundational elements are things like quality of data and good data management practices. So at the city of Edmonton, we have an internal data portal that we can use to organize and give access to analysts and data scientists at the city. We've also worked a lot on general data literacy around the city, too, so that people are aware of what's available and what's possible.
We also need to have invested business partners. So through that kind of data literacy and education process, we've tried to get different business areas interested and able to see the potential. So kind of alongside all of that work that as a business area we've done, we've also been in the lucky position of having leadership from the very top, including the mayor and the city manager who have seen the potential for this sort of technology to help to transform the city.
So using this direction from the top of the organization and the hard work that we've done around putting analytics to action. This kind of framework allows us to work with areas as diverse as fire rescue, pest management, police and security, all sorts of areas. And of course, the safety codes area that we're talking about today. So it's been great. We've been able to develop this framework that allows us to go into really diverse areas of the city and kind of leverage this technology.
Olivia (05:19):
Excellent. So that's a great segue to our next question. Edmonton has gotten a lot of attention for the AI project that relates to the inspection codes. Could you tell us a little bit more about that specific project?
Juan (05:32):
Yeah, so the project itself the short term for it is SCIE project so S-C-I-E - safety codes, inspections, efficiency project, and it is really going through and helping our team out to look at how we can look at our different levels of risk when it comes to inspections and using a database approach for it. So what we did originally was just look at a lot of data that we had and Ben took that on and looked at things was 10 years worth of data and seeing what it is that we can do with that data. So really taking that approach of what is the inherent risk of us going out and doing an inspection or not doing it. You know, it's kinda like what's the worst that can happen if we don't go out and do it and then balancing that risk. So it was a pretty long process. It took a big team. A lot of very smart, dedicated people to make it happen, meeting weekly on it at one point.
But the first hurdle really is the change management piece and having the willingness to lead it. So, you know, when you think about something like building code, it's pretty black and white. So going through and looking at different levels of risks with each item in the code in each item that we inspect takes a lot of searching and knowing that some things are going to be worse than others. So we did a lot of change management training as part of that. And I think that the piece of it was going through and providing auditing with it. So like on the back end of it, although we're using AI to help us out with this, there was an auditing piece that helps along with some of the monthly and quarterly touch points that we do with Ben's team. To date, there's been about 40% of all the applicable inspections have passed through the AI model. So yeah, it helps out greatly in how we manage our risk right now.
Olivia (07:40):
That's interesting, especially around you mentioning that there is a very sort of robust due diligence. So sometimes people assume that with AI or technology and removing people, we might not have that control in place, but it seems like everything was taken into account and that there was a robust auditing process in place with this AI software that complimented the technology and some of the predictive processes. So in terms of, you mentioned change management as being a big piece of this project, was there any resistance or misconception about AI within your local government when you were approaching this to begin with?
Ben (08:18):
I think I can highlight three kind of main misconceptions that come to mind. So one of them is probably the most obvious one, is this model going to come and take all of our jobs? And so we worked really hard to kind of work with the frontline staff and educate them about what this was and the fact that it's not designed to take everybody's jobs it's designed to help them deal with the just incredible volume of work that they're trying to get through. And if you think about, you know, all the inspections that come in for these inspectors, some of them are super high risk and some of them are just really routine, low risk inspections. So the idea here is to leverage the AI, to help them move their focus to the higher risk inspections and take that focus away from the lower risk inspections that they're just turning up, having a look around and then they're passing.
The second one is the misconception that AI is a golden bullet. The truth of the matter is that AI is very helpful, but it's only as good as the data that it is fed, and it's only as good as kind of the design of the project and how it's rolled out. So, um, so in this case, we did a lot of work to make sure that the data we were feeding it in was relevant and kind of was teaching them all the right thing. So basically it's the garbage in garbage out rule. If you feed them all the wrong thing,it's not gonna be helpful for you. Um, and then finally, there's kind of two misconceptions that can go hand in hand. Basically it's around AI being fair by design.Some people say, okay, it's a computer making the decision and so therefore it doesn't have any human bias in there. And on the opposite end of the spectrum you have the idea that, why are we letting a robot make these decisions that need some human judgment? If it's designed correctly, it will make the decisions that you want it to make. However, it's really easy to actually make an algorithm that has bias built in because the data you fed it has biases kind of in the historical data. So for example, we were worried in this project that if we fed it all of the data, all of the attributes that we had about all of the inspections that we may disadvantage a certain area of society. So for example, if we told the algorithm exactly which neighborhood every past inspection had been in, and that neighborhood happened to have a couple of bad contractors that worked there or something like that, that neighborhood could be disadvantaged by the model going forward, because the model would see, Hey, in this particular neighborhood, all of the inspections failed in the past so we're going to make sure we never drop that inspection going forward. So we were very conscious about actually taking out some of the attributes because we didn't want to accidentally bias the model. Basically it can be fair as long as it's designed with a lot of thought.
Olivia (11:28):
Excellent. I think that that's something that people are becoming more comfortable with understanding AI, and also the fact that there can be human biases built into the algorithm. So how are creative ways that your team is actually debunking these AI myths? How did you tackle that? I know that change management was at the center of this project, but how did you actually go and sort of change that narrative within your local government?
Ben (11:42):
I'll start with this particular project, but just then expand a little bit to our kind of broader approach. So in this project, we worked really hard with Juan's team, in order to basically go to all of the staff meetings and talk and present the model to all of the frontline staff, and basically give them an understanding of what, what the project is about the basics of how the model works and just be there to answer questions. And it's a simple thing to do, but actually having that human contact and taking questions and trying to answer people's concerns, etc. I think it's really important and I think that was one of the main reasons that this project was successful.
Olivia (19:07):
I think that's great. So taking a step back, can you walk me through the process that led to the AI software being developed in the context of the home inspection? So I know the center of analytics, you mentioned works with a lot of different groups across the city, but how did that process start?
Juan (12:33):
I think the one piece that we were talking about was looking at data. So we started off with just having that huge data set and trying to get a feel for what type of conclusions we can make based off of it. And so we looked at all the different types of inspections that we did and we looked at what the first time pass rate was on them and we found that there was one that passed over 90% of the time. And that was the footing foundation inspection. So that at the end of day, like that is what your house sits on, right? And most builders were passing it most of the time. And we started to feel comfortable along those lines of like, how can we provide a program where these builders don't get inspected every time, but we provide again an audit function of this and making sure that we go out and we check to see like, you know, every 20th one, are you doing it right? And if you aren't doing it right, then there are consequences and you come off the program and you have to almost gain her trust back. So that was like the initial step of it.
And I think we looked At, okay, well, what, how can we use this data to actually start to predict when something's going to pass or fail? And when we started looking at the data, we had to start looking again, the risk piece of it. And we had kind of started talking about it when we looked at the footing and foundation piece. Because it is like at the end day, it's low risk in terms of how many times it was passing, but it was high impact cause the whole house sits on your foundation. It was a huge change management piece. Because when we first talked about it, we said, okay, well what happens if we don't do this inspection?
Juan (14:37):
And we did a risk based on zero to five, five being the worst thing, you know, like, somebody could die from this zero being, there's not much of an issue with it. And the first thing we got was everything was five. We don't do inspections, all these houses are gonna fall apart. So we said, well, let's look at the data again in more detail and see for all these ones that did fail, why did they fail? And based on that fail, what is the impact of them failing and what we started getting numbers in a bit more detail, we realized that some of those fails weren't high risk. So again, just going through the model over and over again, it was important to keep showing that and as Ben, was saying, having the team come out and talk to my team and talking through what the data was saying. And, you know, and I guess sometimes like it got really detailed into, um, how we were calculating our comfort level. And as long as people started understanding what that comfort level was and started seeing it being used, I think that's one big piece that really helped us out. So at the end of the day, it was really just looking at that huge data set that we have and you know, there's many municipalities have all this data. It's just a matter of how do you actually use it to make risk based decisions?
Olivia (15:57):
Great. So the software is in place. Can you share what some of the initial benefits are from approaching the safety code inspection service from an AI lens?
Juan (16:06):
Yeah, so I think one of the first ones that just is the breaking some misconceptions of the team. When you think about building permits and inspectors is like I said, it could be very black and white and very rigid. And it really was being able to, to create a team that looks at this, with a change management lens, but also at the risk management piece. So as we move forward with any other projects now, we have a base to work off of, right? So I think that has been a huge piece in terms of the team and being able to speak to it, and be able to use their experience and their expertise are more complex applications.
Another piece of it that came in handy during, during the pandemic was, now we have AI to use, like we don't have to put our inspectors in harm's way when we can look at a risk based lens with AI to help us make those decisions. Something that we didn't even think about in our risk matrix originally, you know, it was like not being able to physically go out and do an inspection was never part of the conversation. So, it was that other piece that really helped us out the past few months.
Olivia (17:25):
The project and implementation that keeps on giving. So do you have any advice or resources for local governments that are wishing to learn more about AI? Just generally. I know that this is quite a new concept for a lot of governments and a lot of local governments are actually just starting out this process. They actually, in some cases might not even have started it. So do you have any resources or words of wisdom that you'd like to share with our audience today?
Ben (17:52):
Sure. So I would say a lot of organizations already have analysts and other data professionals that can begin the process of putting in those foundational kind of elements to lead up to these more advanced kind of AI and just advanced analytics in general. So, um, there's that kind of spectrum and, you know, the foundational pieces are to have your data in a good, in a good place, so organized, good data quality, and to begin just educating the organization about, you know, basic data literacy, how to interpret data, how to report on data. In order to apply AI, you would often hire or work with a data scientist and the data scientist should be able to work with your data and with the business area in tandem to basically see what the possibilities are.
Ben (18:58):
So, you know, as an example, using this project, we didn't know what the exact model was going to look like and what the exact business case was going to be at the very beginning. But we knew there was a good data set and we knew there was a business problem to be solved. And along those lines, I think the other thing to highlight is, don't expect to kind of hit a home run right off the bat and I don't even play baseball as you can probably tell from my accent, but I know what a home run is, I guess. So there's most applications when you, when you sort of first look into a project where the data sets and a business problem, often it won't work out.
So there's a lot of exploration and I think a key is to move quickly at the start and try to find the potential kind of match between the data science and the business problem. And if it's not there, then move on. Municipalities are a ripe with opportunities to apply AI. I kind of think of it like a giant fruit tree and like there's lots of low hanging fruit you can pick. So don't, don't aim for the one right at the top of the tree that might not be good in the first place, find, find the really great chunky ripe fruit at the bottom of the tree that there's going to give you a lot of business value in a reasonably short amount of time.
Olivia (20:30):
Thank you so much for that. I'm sure our audience appreciates your words of wisdom because I know as you mentioned with the current situation that many municipalities find themselves with COVID-19, really leaning on the data scientists, the analysts, and you know, the technology teams to try to figure out some creative solutions to some ongoing problems. So thanks for that. So beyond developing award-winning software, Edmonton is busy innovating and some other areas as well. Can you share with us something that you've tried that didn't work?
Ben (21:04):
I can answer this during the pandemic, cutting my hair during a pandemic. Seriously, though, as I mentioned earlier, many projects will fail when you're rolling out AI algorithms. And I think the key is to identify early that it's going to fail and then move on and not kind of go down a road that’s not going to work out. So we've had lots of projects that maybe we tried for a couple of weeks. We sort of looked at the data, worked with the business area and eventually we just came to the conclusion that there's not an opportunity here in this case.
Olivia (21:49):
Good and really important. And this ties nicely with the piece of advice that you shared earlier with our audience around focusing on the low hanging fruit and the pieces that are going to work quickly and effectively. What's something that excites you both about the future of civic innovation in Edmonton?
Ben (22:09):
So let's say the, you know, the impact of COVID-19 on local government is huge and that being said, civic innovation and so I'm thinking, you know, AI robotic process automation has a huge potential and generally moving to online services is going to be a central part of how municipalities reimagine the way that they deliver services to their citizens.
So what excites me, it's more of a silver lining, you know, I think everybody's feeling the pain of the current situation that we're in, but I do think that a silver lining of this is that t's going to kind of accelerate our move towards moving services online and applying the sorts of models that we've worked with Juan’s area in this case to roll out. I think the appetite is going to be there. We need to kind of compact and really improve the way we deliver the efficiency with which we deliver services to citizens. My hope is that this whole situation is just gonna move us a little further down the road and, kind of help us go in that direction.
Olivia (23:23):
Thank you for sharing your insights and for taking the time to talk to Govlaunch today, it's been really interesting to hear how Edmonton is improving its service through the use of AI and predictive analytics and all the different pieces that stem from that from changing culture to data literacy and just even now mentioning how it's had an impact throughout the pandemic as well. We look forward to hearing about more projects like this. So thank you both for joining us.
Ben (23:48):
Thank you!
Lindsay (23:56):
There are several ways local governments can begin integrating AI and predictive analytics into their work. Step one is to compile and understand the data. This is no small feat. So if you need a little help, you can search Govlaunch for ways other local governments like yours have moved toward more transparent and strategic use of data.
Once you've tackled this important step, you can begin looking to technologies such as AI to promote data driven decisions, increase efficiency, and reduce costs for your local government. Thank you to the team in Edmonton for sharing their great work.
I'm Lindsay Pica-Alfano and this podcast was produced by Govlaunch, the wiki for local government innovation. You can subscribe to your more stories like this, wherever you get your podcasts. If you're a local government innovator, we hope you'll help us on our mission to build the largest free resource for local governments globally. You can join to search and contribute to the wiki at govlaunch.com. Thanks for tuning in. We hope to see you next time on the Govlaunch podcast.